2024-04-25T09:54:59.703 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T09:54:59.703 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048 --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} --name neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --verbose --owner scheduled_neesingh@teuthology -- /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048/orig.config.yaml 2024-04-25T09:54:59.747 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:54:59.828 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7673048' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec kclient: syntax: v1 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1124 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIVju1AAd0gE/+C4CJtuXZwjKdnXue5BP/hJ477CXirK3URoeIOvWXXyYI9gK3V9TJoD16idtOV6jlg7lEEpetQ= smithi132.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAVgsCkyRVusmNSE6EqqVSz6QgjmdNFwA6Zk7noeSi57ejVm+rctGYzRk7JRJ2swgrM67UiylRqY/ZBm1K6VRtM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_09:36:50 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T09:54:59.828 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa; will attempt to use it 2024-04-25T09:54:59.829 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa/tasks 2024-04-25T09:54:59.829 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T09:54:59.830 INFO:teuthology.task.internal:Checking packages... 2024-04-25T09:54:59.855 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T09:54:59.855 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T09:54:59.855 INFO:teuthology.packaging:ref: None 2024-04-25T09:54:59.855 INFO:teuthology.packaging:tag: None 2024-04-25T09:54:59.855 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T09:54:59.856 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:54:59.856 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T09:54:59.998 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628.g97ae6cc9 2024-04-25T09:55:00.000 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T09:55:00.006 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T09:55:00.006 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T09:55:00.025 INFO:teuthology.task.internal:Saving configuration 2024-04-25T09:55:00.044 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T09:55:00.050 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:55:00.073 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:50:05.855804', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIVju1AAd0gE/+C4CJtuXZwjKdnXue5BP/hJ477CXirK3URoeIOvWXXyYI9gK3V9TJoD16idtOV6jlg7lEEpetQ='} 2024-04-25T09:55:00.096 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:50:05.857393', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAVgsCkyRVusmNSE6EqqVSz6QgjmdNFwA6Zk7noeSi57ejVm+rctGYzRk7JRJ2swgrM67UiylRqY/ZBm1K6VRtM='} 2024-04-25T09:55:00.096 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T09:55:00.102 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T09:55:00.102 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T09:55:00.103 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T09:55:00.163 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f31ceb67160>, signals=[15]) 2024-04-25T09:55:00.163 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T09:55:00.169 INFO:teuthology.task.internal:Opening connections... 2024-04-25T09:55:00.170 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T09:55:00.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:55:00.248 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi132.front.sepia.ceph.com 2024-04-25T09:55:00.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:55:00.322 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T09:55:00.329 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-04-25T09:55:00.347 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-04-25T09:55:00.347 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-04-25T09:55:00.404 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2024-04-25T09:55:00.404 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2024-04-25T09:55:00.404 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2024-04-25T09:55:00.404 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2024-04-25T09:55:00.404 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2024-04-25T09:55:00.404 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2024-04-25T09:55:00.404 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T09:55:00.405 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2024-04-25T09:55:00.405 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2024-04-25T09:55:00.405 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T09:55:00.405 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2024-04-25T09:55:00.405 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T09:55:00.405 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T09:55:00.405 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T09:55:00.406 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-04-25T09:55:00.434 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2024-04-25T09:55:00.452 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2024-04-25T09:55:00.453 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2024-04-25T09:55:00.509 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream" 2024-04-25T09:55:00.510 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="9" 2024-04-25T09:55:00.510 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2024-04-25T09:55:00.510 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2024-04-25T09:55:00.510 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="9" 2024-04-25T09:55:00.510 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el9" 2024-04-25T09:55:00.510 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T09:55:00.510 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2024-04-25T09:55:00.510 INFO:teuthology.orchestra.run.smithi132.stdout:LOGO="fedora-logo-icon" 2024-04-25T09:55:00.510 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T09:55:00.510 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/" 2024-04-25T09:55:00.510 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T09:55:00.510 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T09:55:00.511 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T09:55:00.511 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2024-04-25T09:55:00.533 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T09:55:00.542 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T09:55:00.547 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T09:55:00.548 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:55:00.550 DEBUG:teuthology.orchestra.run.smithi132:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:55:00.566 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T09:55:00.582 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T09:55:00.583 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:55:00.608 DEBUG:teuthology.orchestra.run.smithi132:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:55:00.638 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T09:55:00.687 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T09:55:00.701 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T09:55:00.702 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-25T09:55:00.702 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-25T09:55:00.703 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:55:00.703 DEBUG:teuthology.orchestra.run.smithi132:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:55:00.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:55:00.719 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T09:55:00.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:55:00.723 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T09:55:00.899 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:55:00.900 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T09:55:00.900 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-04-25T09:55:00.908 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:55:00.909 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T09:55:00.909 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-04-25T09:55:00.916 INFO:teuthology.orchestra.run.smithi132.stdout:5.14.0-437.el9.x86_64 2024-04-25T09:55:00.917 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:55:00.917 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T09:55:00.917 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum upgrade -y linux-firmware 2024-04-25T09:55:00.926 INFO:teuthology.orchestra.run.smithi119.stdout:5.14.0-437.el9.x86_64 2024-04-25T09:55:00.926 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:55:00.926 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T09:55:00.927 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum upgrade -y linux-firmware 2024-04-25T09:55:03.508 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.0 MB 00:00 2024-04-25T09:55:03.549 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 8.1 MB/s | 8.0 MB 00:00 2024-04-25T09:55:05.467 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T09:55:05.791 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-25T09:55:09.676 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T09:55:10.016 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-04-25T09:55:11.368 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-25T09:55:11.692 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-25T09:55:15.729 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 5.3 MB/s | 21 MB 00:03 2024-04-25T09:55:17.869 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 3.3 MB/s | 21 MB 00:06 2024-04-25T09:55:20.760 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T09:55:22.730 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-25T09:55:22.731 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-04-25T09:55:22.731 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-25T09:55:22.828 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T09:55:22.828 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:55:22.935 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-25T09:55:22.975 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T09:55:22.975 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T09:55:22.977 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi119.front.sepia.ceph.com to complete... 2024-04-25T09:55:24.917 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-04-25T09:55:24.918 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-04-25T09:55:24.918 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-04-25T09:55:25.027 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T09:55:25.027 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:55:25.185 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T09:55:25.186 DEBUG:teuthology.orchestra.run.smithi132:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T09:55:25.188 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi132.front.sepia.ceph.com to complete... 2024-04-25T09:55:37.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25 09:55:37 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T09:55:37.703 INFO:teuthology.orchestra.run.smithi119.stderr:FINISHED --2024-04-25 09:55:37-- 2024-04-25T09:55:37.703 INFO:teuthology.orchestra.run.smithi119.stderr:Total wall clock time: 15s 2024-04-25T09:55:37.704 INFO:teuthology.orchestra.run.smithi119.stderr:Downloaded: 1 files, 341M in 14s (23.6 MB/s) 2024-04-25T09:55:37.707 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T09:55:37.707 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T09:55:39.791 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying... ######################################## 2024-04-25T09:55:39.924 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing... ######################################## 2024-04-25T09:55:40.039 INFO:teuthology.orchestra.run.smithi119.stdout:Updating / installing... 2024-04-25T09:55:46.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25 09:55:46 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T09:55:46.964 INFO:teuthology.orchestra.run.smithi132.stderr:FINISHED --2024-04-25 09:55:46-- 2024-04-25T09:55:46.964 INFO:teuthology.orchestra.run.smithi132.stderr:Total wall clock time: 22s 2024-04-25T09:55:46.964 INFO:teuthology.orchestra.run.smithi132.stderr:Downloaded: 1 files, 341M in 21s (16.0 MB/s) 2024-04-25T09:55:46.967 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T09:55:46.968 DEBUG:teuthology.orchestra.run.smithi132:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T09:55:48.002 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T09:55:49.088 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying... ######################################## 2024-04-25T09:55:49.230 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing... ######################################## 2024-04-25T09:55:49.337 INFO:teuthology.orchestra.run.smithi132.stdout:Updating / installing... 2024-04-25T09:55:57.280 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T09:56:27.319 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T09:56:27.320 DEBUG:teuthology.orchestra.run.smithi119:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T09:56:27.404 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:27.404 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:27.404 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:27.404 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T09:56:27.404 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T09:56:27.404 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T09:56:27.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T09:56:27.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T09:56:27.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T09:56:27.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T09:56:27.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T09:56:27.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T09:56:27.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T09:56:27.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T09:56:27.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T09:56:27.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T09:56:27.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T09:56:27.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T09:56:27.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T09:56:27.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T09:56:27.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T09:56:27.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T09:56:27.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T09:56:27.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T09:56:27.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T09:56:27.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T09:56:27.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T09:56:27.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T09:56:27.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T09:56:27.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T09:56:27.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T09:56:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T09:56:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T09:56:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T09:56:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T09:56:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T09:56:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T09:56:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T09:56:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T09:56:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T09:56:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T09:56:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T09:56:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T09:56:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T09:56:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T09:56:27.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T09:56:27.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T09:56:27.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T09:56:27.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T09:56:27.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T09:56:27.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T09:56:27.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T09:56:27.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T09:56:27.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T09:56:27.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T09:56:27.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T09:56:27.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T09:56:27.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T09:56:27.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T09:56:27.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T09:56:27.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T09:56:27.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T09:56:27.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T09:56:27.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T09:56:27.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T09:56:27.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T09:56:27.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T09:56:27.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T09:56:27.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T09:56:27.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T09:56:27.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T09:56:27.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T09:56:27.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T09:56:27.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T09:56:27.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T09:56:27.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T09:56:27.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T09:56:27.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T09:56:27.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T09:56:27.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T09:56:27.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T09:56:27.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T09:56:27.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T09:56:27.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T09:56:27.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T09:56:27.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T09:56:27.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T09:56:27.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T09:56:27.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T09:56:27.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T09:56:27.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T09:56:27.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T09:56:27.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T09:56:27.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T09:56:27.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T09:56:27.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T09:56:27.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T09:56:27.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T09:56:27.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T09:56:27.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T09:56:27.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T09:56:27.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T09:56:27.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T09:56:27.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T09:56:27.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T09:56:27.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T09:56:27.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T09:56:27.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T09:56:27.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T09:56:27.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T09:56:27.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T09:56:27.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T09:56:27.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T09:56:27.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T09:56:27.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T09:56:27.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T09:56:27.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T09:56:27.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T09:56:27.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T09:56:27.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T09:56:27.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T09:56:27.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T09:56:27.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T09:56:27.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T09:56:27.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T09:56:27.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T09:56:27.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T09:56:27.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T09:56:27.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T09:56:27.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T09:56:27.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T09:56:27.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T09:56:27.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T09:56:27.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T09:56:27.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T09:56:27.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T09:56:27.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T09:56:27.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T09:56:27.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T09:56:27.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T09:56:27.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T09:56:27.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T09:56:27.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T09:56:27.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T09:56:27.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T09:56:27.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T09:56:27.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T09:56:27.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T09:56:27.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T09:56:27.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T09:56:27.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T09:56:27.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T09:56:27.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T09:56:27.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T09:56:27.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T09:56:27.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T09:56:27.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T09:56:27.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T09:56:27.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T09:56:27.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T09:56:27.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T09:56:27.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T09:56:27.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T09:56:27.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T09:56:27.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T09:56:27.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T09:56:27.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T09:56:27.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T09:56:27.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T09:56:27.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T09:56:27.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T09:56:27.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T09:56:27.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T09:56:27.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T09:56:27.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T09:56:27.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T09:56:27.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T09:56:27.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T09:56:27.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T09:56:27.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T09:56:27.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T09:56:27.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T09:56:27.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T09:56:27.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T09:56:27.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T09:56:27.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T09:56:27.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T09:56:27.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T09:56:27.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T09:56:27.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T09:56:27.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T09:56:27.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T09:56:27.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T09:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T09:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T09:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T09:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T09:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T09:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T09:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T09:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T09:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T09:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T09:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T09:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T09:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T09:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T09:56:27.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T09:56:27.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T09:56:27.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T09:56:27.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T09:56:27.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T09:56:27.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T09:56:27.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T09:56:27.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T09:56:27.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T09:56:27.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T09:56:27.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T09:56:27.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T09:56:27.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T09:56:27.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T09:56:27.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T09:56:27.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T09:56:27.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T09:56:27.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T09:56:27.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T09:56:27.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T09:56:27.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T09:56:27.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T09:56:27.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T09:56:27.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T09:56:27.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T09:56:27.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T09:56:27.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T09:56:27.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T09:56:27.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T09:56:27.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T09:56:27.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T09:56:27.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T09:56:27.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T09:56:27.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T09:56:27.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T09:56:27.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T09:56:27.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T09:56:27.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T09:56:27.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T09:56:27.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T09:56:27.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T09:56:27.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T09:56:27.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T09:56:27.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T09:56:27.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T09:56:27.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T09:56:27.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T09:56:27.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T09:56:27.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T09:56:27.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T09:56:27.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T09:56:27.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T09:56:27.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T09:56:27.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T09:56:27.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T09:56:27.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T09:56:27.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T09:56:27.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T09:56:27.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T09:56:27.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T09:56:27.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T09:56:27.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T09:56:27.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T09:56:27.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T09:56:27.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T09:56:27.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T09:56:27.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T09:56:27.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T09:56:27.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T09:56:27.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T09:56:27.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T09:56:27.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T09:56:27.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T09:56:27.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T09:56:27.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T09:56:27.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T09:56:27.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T09:56:27.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T09:56:27.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T09:56:27.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T09:56:27.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T09:56:27.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T09:56:27.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T09:56:27.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T09:56:27.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T09:56:27.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T09:56:27.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T09:56:27.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T09:56:27.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T09:56:27.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T09:56:27.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T09:56:27.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T09:56:27.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T09:56:27.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T09:56:27.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T09:56:27.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T09:56:27.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T09:56:27.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T09:56:27.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T09:56:27.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T09:56:27.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T09:56:27.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T09:56:27.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T09:56:27.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T09:56:27.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T09:56:27.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T09:56:27.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T09:56:27.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T09:56:27.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T09:56:27.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T09:56:27.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T09:56:27.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T09:56:27.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T09:56:27.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T09:56:27.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T09:56:27.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T09:56:27.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T09:56:27.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T09:56:27.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T09:56:27.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T09:56:27.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T09:56:27.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T09:56:27.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T09:56:27.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T09:56:27.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T09:56:27.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T09:56:27.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T09:56:27.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T09:56:27.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T09:56:27.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T09:56:27.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T09:56:27.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T09:56:27.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T09:56:27.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T09:56:27.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T09:56:27.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T09:56:27.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T09:56:27.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T09:56:27.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T09:56:27.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T09:56:27.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T09:56:27.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T09:56:27.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T09:56:27.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T09:56:27.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T09:56:27.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T09:56:27.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T09:56:27.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T09:56:27.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T09:56:27.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T09:56:27.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T09:56:27.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T09:56:27.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T09:56:27.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T09:56:27.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T09:56:27.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T09:56:27.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T09:56:27.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T09:56:27.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T09:56:27.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T09:56:27.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T09:56:27.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T09:56:27.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T09:56:27.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T09:56:27.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T09:56:27.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T09:56:27.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T09:56:27.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T09:56:27.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T09:56:27.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T09:56:27.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T09:56:27.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T09:56:27.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T09:56:27.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T09:56:27.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T09:56:27.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T09:56:27.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T09:56:27.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T09:56:27.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T09:56:27.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T09:56:27.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T09:56:27.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T09:56:27.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T09:56:27.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T09:56:27.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T09:56:27.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T09:56:27.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T09:56:27.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T09:56:27.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T09:56:27.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T09:56:27.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T09:56:27.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T09:56:27.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T09:56:27.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T09:56:27.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T09:56:27.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T09:56:27.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T09:56:27.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T09:56:27.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T09:56:27.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T09:56:27.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T09:56:27.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T09:56:27.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T09:56:27.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T09:56:27.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T09:56:27.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T09:56:27.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T09:56:27.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T09:56:27.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T09:56:27.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T09:56:27.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T09:56:27.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T09:56:27.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T09:56:27.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T09:56:27.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T09:56:27.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T09:56:27.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T09:56:27.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T09:56:27.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T09:56:27.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T09:56:27.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T09:56:27.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T09:56:27.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T09:56:27.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T09:56:27.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T09:56:27.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T09:56:27.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T09:56:27.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T09:56:27.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T09:56:27.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T09:56:27.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T09:56:27.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T09:56:27.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T09:56:27.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T09:56:27.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T09:56:27.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T09:56:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T09:56:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T09:56:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T09:56:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T09:56:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T09:56:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T09:56:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T09:56:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T09:56:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T09:56:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T09:56:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T09:56:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T09:56:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T09:56:27.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T09:56:27.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T09:56:27.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T09:56:27.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T09:56:27.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T09:56:27.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T09:56:27.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T09:56:27.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T09:56:27.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T09:56:27.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T09:56:27.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T09:56:27.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T09:56:27.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T09:56:27.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T09:56:27.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T09:56:27.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T09:56:27.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T09:56:27.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T09:56:27.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T09:56:27.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T09:56:27.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T09:56:27.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T09:56:27.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T09:56:27.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T09:56:27.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T09:56:27.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T09:56:27.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T09:56:27.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T09:56:27.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T09:56:27.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T09:56:27.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T09:56:27.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T09:56:27.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T09:56:27.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T09:56:27.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T09:56:27.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T09:56:27.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T09:56:27.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T09:56:27.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T09:56:27.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T09:56:27.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T09:56:27.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T09:56:27.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T09:56:27.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T09:56:27.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T09:56:27.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T09:56:27.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T09:56:27.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T09:56:27.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T09:56:27.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T09:56:27.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T09:56:27.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T09:56:27.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T09:56:27.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T09:56:27.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T09:56:27.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T09:56:27.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T09:56:27.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T09:56:27.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T09:56:27.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T09:56:27.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T09:56:27.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T09:56:27.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T09:56:27.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T09:56:27.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T09:56:27.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T09:56:27.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T09:56:27.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T09:56:27.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T09:56:27.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T09:56:27.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T09:56:27.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T09:56:27.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T09:56:27.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T09:56:27.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T09:56:27.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T09:56:27.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T09:56:27.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T09:56:27.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T09:56:27.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T09:56:27.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T09:56:27.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T09:56:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T09:56:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T09:56:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T09:56:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T09:56:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T09:56:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T09:56:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T09:56:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T09:56:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T09:56:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T09:56:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T09:56:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T09:56:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T09:56:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T09:56:27.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T09:56:27.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T09:56:27.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T09:56:27.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T09:56:27.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T09:56:27.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T09:56:27.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T09:56:27.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T09:56:27.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T09:56:27.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T09:56:27.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T09:56:27.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T09:56:27.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T09:56:27.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T09:56:27.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T09:56:27.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T09:56:27.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T09:56:27.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T09:56:27.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T09:56:27.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T09:56:27.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T09:56:27.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T09:56:27.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T09:56:27.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T09:56:27.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T09:56:27.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T09:56:27.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T09:56:27.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T09:56:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T09:56:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T09:56:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T09:56:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T09:56:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T09:56:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T09:56:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T09:56:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T09:56:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T09:56:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T09:56:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T09:56:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T09:56:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T09:56:27.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T09:56:27.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T09:56:27.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T09:56:27.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T09:56:27.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T09:56:27.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T09:56:27.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T09:56:27.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T09:56:27.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T09:56:27.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T09:56:27.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T09:56:27.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T09:56:27.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T09:56:27.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T09:56:27.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T09:56:27.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T09:56:27.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T09:56:27.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T09:56:27.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T09:56:27.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T09:56:27.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T09:56:27.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T09:56:27.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T09:56:27.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T09:56:27.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T09:56:27.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T09:56:27.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T09:56:27.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T09:56:27.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T09:56:27.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T09:56:27.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T09:56:27.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T09:56:27.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T09:56:27.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T09:56:27.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T09:56:27.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T09:56:27.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T09:56:27.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T09:56:27.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T09:56:27.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T09:56:27.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T09:56:27.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T09:56:27.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T09:56:27.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T09:56:27.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T09:56:27.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T09:56:27.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T09:56:27.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T09:56:27.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T09:56:27.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T09:56:27.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T09:56:27.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T09:56:27.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T09:56:27.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T09:56:27.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T09:56:27.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T09:56:27.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T09:56:27.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T09:56:27.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T09:56:27.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T09:56:27.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T09:56:27.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T09:56:27.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T09:56:27.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T09:56:27.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T09:56:27.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T09:56:27.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T09:56:27.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T09:56:27.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T09:56:27.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T09:56:27.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T09:56:27.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T09:56:27.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T09:56:27.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T09:56:27.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T09:56:27.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T09:56:27.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T09:56:27.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T09:56:27.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T09:56:27.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T09:56:27.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T09:56:27.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T09:56:27.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T09:56:27.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T09:56:27.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T09:56:27.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T09:56:27.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T09:56:27.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T09:56:27.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T09:56:27.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T09:56:27.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T09:56:27.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T09:56:27.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T09:56:27.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T09:56:27.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T09:56:27.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T09:56:27.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T09:56:27.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T09:56:27.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T09:56:27.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T09:56:27.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T09:56:27.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T09:56:27.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T09:56:27.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T09:56:27.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T09:56:27.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T09:56:27.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T09:56:27.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T09:56:27.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T09:56:27.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T09:56:27.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T09:56:27.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T09:56:27.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T09:56:27.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T09:56:27.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T09:56:27.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T09:56:27.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T09:56:27.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T09:56:27.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T09:56:27.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T09:56:27.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T09:56:27.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T09:56:27.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T09:56:27.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T09:56:27.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T09:56:27.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T09:56:27.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T09:56:27.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T09:56:27.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T09:56:27.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T09:56:27.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T09:56:27.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T09:56:27.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T09:56:27.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T09:56:27.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T09:56:27.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T09:56:27.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T09:56:27.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T09:56:27.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T09:56:27.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T09:56:27.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T09:56:27.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T09:56:27.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T09:56:27.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T09:56:27.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T09:56:27.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T09:56:27.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T09:56:27.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T09:56:27.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T09:56:27.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T09:56:27.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T09:56:27.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T09:56:27.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T09:56:27.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T09:56:27.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T09:56:27.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T09:56:27.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T09:56:27.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T09:56:27.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T09:56:27.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T09:56:27.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T09:56:27.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T09:56:27.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T09:56:27.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T09:56:27.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T09:56:27.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T09:56:27.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T09:56:27.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T09:56:27.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T09:56:27.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T09:56:27.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T09:56:27.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T09:56:27.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T09:56:27.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T09:56:27.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T09:56:27.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T09:56:27.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T09:56:27.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T09:56:27.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T09:56:27.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T09:56:27.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T09:56:27.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T09:56:27.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T09:56:27.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T09:56:27.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T09:56:27.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T09:56:27.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T09:56:27.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T09:56:27.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T09:56:27.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T09:56:27.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T09:56:27.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T09:56:27.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T09:56:27.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T09:56:27.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T09:56:27.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T09:56:27.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T09:56:27.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T09:56:27.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T09:56:27.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T09:56:27.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T09:56:27.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T09:56:27.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T09:56:27.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T09:56:27.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T09:56:27.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T09:56:27.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T09:56:27.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T09:56:27.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T09:56:27.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T09:56:27.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T09:56:27.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T09:56:27.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T09:56:27.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T09:56:27.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T09:56:27.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T09:56:27.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T09:56:27.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T09:56:27.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T09:56:27.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T09:56:27.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T09:56:27.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T09:56:27.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T09:56:27.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T09:56:27.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T09:56:27.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T09:56:27.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T09:56:27.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T09:56:27.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T09:56:27.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T09:56:27.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T09:56:27.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T09:56:27.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T09:56:27.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T09:56:27.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T09:56:27.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T09:56:27.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T09:56:27.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T09:56:27.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T09:56:27.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T09:56:27.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T09:56:27.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T09:56:27.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T09:56:27.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T09:56:27.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T09:56:27.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T09:56:27.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T09:56:27.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T09:56:27.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T09:56:27.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T09:56:27.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T09:56:27.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T09:56:27.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T09:56:27.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T09:56:27.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T09:56:27.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T09:56:27.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T09:56:27.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T09:56:27.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T09:56:27.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T09:56:27.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T09:56:27.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T09:56:27.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T09:56:27.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T09:56:27.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T09:56:27.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T09:56:27.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T09:56:27.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T09:56:27.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T09:56:27.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T09:56:27.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T09:56:27.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T09:56:27.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T09:56:27.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T09:56:27.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T09:56:27.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T09:56:27.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T09:56:27.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T09:56:27.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T09:56:27.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T09:56:27.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T09:56:27.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T09:56:27.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T09:56:27.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T09:56:27.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T09:56:27.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T09:56:27.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T09:56:27.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T09:56:27.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T09:56:27.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T09:56:27.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T09:56:27.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T09:56:27.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T09:56:27.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T09:56:27.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T09:56:27.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T09:56:27.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T09:56:27.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T09:56:27.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T09:56:27.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T09:56:27.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T09:56:27.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T09:56:27.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T09:56:27.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T09:56:27.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T09:56:27.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T09:56:27.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T09:56:27.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T09:56:27.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T09:56:27.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T09:56:27.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T09:56:27.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T09:56:27.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T09:56:27.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T09:56:27.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T09:56:27.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T09:56:27.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T09:56:27.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T09:56:27.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T09:56:27.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T09:56:27.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T09:56:27.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T09:56:27.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T09:56:27.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T09:56:27.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T09:56:27.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T09:56:27.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T09:56:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T09:56:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T09:56:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T09:56:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T09:56:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T09:56:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T09:56:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T09:56:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T09:56:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T09:56:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T09:56:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T09:56:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T09:56:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T09:56:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T09:56:27.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T09:56:27.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T09:56:27.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T09:56:27.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T09:56:27.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T09:56:27.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T09:56:27.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T09:56:27.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T09:56:27.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T09:56:27.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T09:56:27.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T09:56:27.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T09:56:27.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T09:56:27.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T09:56:27.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T09:56:27.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T09:56:27.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T09:56:27.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T09:56:27.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T09:56:27.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T09:56:27.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T09:56:27.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T09:56:27.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T09:56:27.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T09:56:27.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T09:56:27.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T09:56:27.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T09:56:27.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T09:56:27.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T09:56:27.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T09:56:27.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T09:56:27.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T09:56:27.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T09:56:27.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T09:56:27.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T09:56:27.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T09:56:27.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T09:56:27.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T09:56:27.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T09:56:27.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T09:56:27.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T09:56:27.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T09:56:27.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T09:56:27.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T09:56:27.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T09:56:27.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T09:56:27.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T09:56:27.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T09:56:27.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T09:56:27.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T09:56:27.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T09:56:27.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T09:56:27.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T09:56:27.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T09:56:27.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T09:56:27.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T09:56:27.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T09:56:27.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T09:56:27.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T09:56:27.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T09:56:27.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T09:56:27.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T09:56:27.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T09:56:27.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T09:56:27.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T09:56:27.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T09:56:27.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T09:56:27.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T09:56:27.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T09:56:27.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T09:56:27.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T09:56:27.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T09:56:27.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T09:56:27.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T09:56:27.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T09:56:27.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T09:56:27.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T09:56:27.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T09:56:27.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T09:56:27.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T09:56:27.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T09:56:27.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T09:56:27.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T09:56:27.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T09:56:27.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T09:56:27.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T09:56:27.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T09:56:27.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T09:56:27.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T09:56:27.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T09:56:27.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T09:56:27.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T09:56:27.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T09:56:27.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T09:56:27.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T09:56:27.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T09:56:27.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T09:56:27.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T09:56:27.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T09:56:27.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T09:56:27.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T09:56:27.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T09:56:27.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T09:56:27.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T09:56:27.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T09:56:27.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T09:56:27.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T09:56:27.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T09:56:27.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T09:56:27.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T09:56:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T09:56:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T09:56:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T09:56:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T09:56:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T09:56:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T09:56:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T09:56:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T09:56:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T09:56:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T09:56:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T09:56:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T09:56:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T09:56:27.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T09:56:27.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T09:56:27.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T09:56:27.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T09:56:27.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T09:56:27.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T09:56:27.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T09:56:27.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T09:56:27.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T09:56:27.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T09:56:27.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T09:56:27.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T09:56:27.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T09:56:27.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T09:56:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T09:56:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T09:56:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T09:56:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T09:56:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T09:56:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T09:56:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T09:56:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T09:56:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T09:56:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T09:56:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T09:56:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T09:56:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T09:56:27.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T09:56:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T09:56:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T09:56:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T09:56:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T09:56:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T09:56:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T09:56:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T09:56:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T09:56:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T09:56:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T09:56:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T09:56:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T09:56:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T09:56:27.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T09:56:27.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T09:56:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T09:56:27.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T09:56:27.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T09:56:27.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T09:56:27.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T09:56:27.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T09:56:27.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T09:56:27.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T09:56:27.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T09:56:27.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T09:56:27.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T09:56:27.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T09:56:27.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T09:56:27.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T09:56:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T09:56:27.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T09:56:27.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T09:56:27.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T09:56:27.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T09:56:27.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T09:56:27.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T09:56:27.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T09:56:27.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T09:56:27.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T09:56:27.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T09:56:27.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T09:56:27.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T09:56:27.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T09:56:27.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T09:56:27.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T09:56:27.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T09:56:27.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T09:56:27.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T09:56:27.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T09:56:27.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T09:56:27.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T09:56:27.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T09:56:27.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T09:56:27.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T09:56:27.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T09:56:27.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T09:56:27.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T09:56:27.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T09:56:27.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T09:56:27.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T09:56:27.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T09:56:27.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T09:56:27.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T09:56:27.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T09:56:27.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T09:56:27.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T09:56:27.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T09:56:27.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T09:56:27.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T09:56:27.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T09:56:27.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T09:56:27.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T09:56:27.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T09:56:27.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T09:56:27.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T09:56:27.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T09:56:27.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T09:56:27.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T09:56:27.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T09:56:27.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T09:56:27.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T09:56:27.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T09:56:27.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T09:56:27.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T09:56:27.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T09:56:27.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T09:56:27.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T09:56:27.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T09:56:27.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T09:56:27.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T09:56:27.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T09:56:27.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T09:56:27.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T09:56:27.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T09:56:27.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T09:56:27.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T09:56:27.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T09:56:27.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T09:56:27.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T09:56:27.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T09:56:27.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T09:56:27.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T09:56:27.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T09:56:27.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T09:56:27.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T09:56:27.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T09:56:27.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T09:56:27.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T09:56:27.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T09:56:27.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T09:56:27.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T09:56:27.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T09:56:27.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T09:56:27.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T09:56:27.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T09:56:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T09:56:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T09:56:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T09:56:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T09:56:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T09:56:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T09:56:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T09:56:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T09:56:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T09:56:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T09:56:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T09:56:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T09:56:27.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T09:56:27.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T09:56:27.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T09:56:27.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T09:56:27.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T09:56:27.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T09:56:27.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T09:56:27.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T09:56:27.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T09:56:27.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T09:56:27.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T09:56:27.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T09:56:27.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T09:56:27.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T09:56:27.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T09:56:27.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T09:56:27.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T09:56:27.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T09:56:27.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T09:56:27.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T09:56:27.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T09:56:27.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T09:56:27.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T09:56:27.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T09:56:27.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T09:56:27.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T09:56:27.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T09:56:27.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T09:56:27.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T09:56:27.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T09:56:27.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T09:56:27.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T09:56:27.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T09:56:27.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T09:56:27.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T09:56:27.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T09:56:27.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T09:56:27.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T09:56:27.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T09:56:27.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T09:56:27.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T09:56:27.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T09:56:27.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T09:56:27.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T09:56:27.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T09:56:27.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T09:56:27.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T09:56:27.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T09:56:27.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T09:56:27.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T09:56:27.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T09:56:27.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T09:56:27.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T09:56:27.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T09:56:27.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T09:56:27.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T09:56:27.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T09:56:27.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T09:56:27.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T09:56:27.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T09:56:27.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T09:56:27.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T09:56:27.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T09:56:27.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T09:56:27.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T09:56:27.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T09:56:27.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T09:56:27.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T09:56:27.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T09:56:27.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T09:56:27.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T09:56:27.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T09:56:27.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T09:56:27.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T09:56:27.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T09:56:27.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T09:56:27.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T09:56:27.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T09:56:27.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T09:56:27.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T09:56:27.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T09:56:27.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T09:56:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T09:56:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T09:56:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T09:56:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T09:56:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T09:56:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T09:56:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T09:56:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T09:56:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T09:56:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T09:56:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T09:56:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T09:56:27.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T09:56:27.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T09:56:27.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T09:56:27.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T09:56:27.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T09:56:27.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T09:56:27.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T09:56:27.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T09:56:27.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T09:56:27.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T09:56:27.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T09:56:27.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T09:56:27.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T09:56:27.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T09:56:27.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T09:56:27.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T09:56:27.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T09:56:27.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T09:56:27.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T09:56:27.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T09:56:27.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T09:56:27.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T09:56:27.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T09:56:27.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T09:56:27.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T09:56:27.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T09:56:27.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T09:56:27.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T09:56:27.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T09:56:27.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T09:56:27.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T09:56:27.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T09:56:27.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T09:56:27.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T09:56:27.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T09:56:27.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T09:56:27.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T09:56:27.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T09:56:27.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T09:56:27.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T09:56:27.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T09:56:27.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T09:56:27.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T09:56:27.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T09:56:27.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T09:56:27.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T09:56:27.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T09:56:27.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T09:56:27.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T09:56:27.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T09:56:27.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T09:56:27.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T09:56:27.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T09:56:27.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T09:56:27.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T09:56:27.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T09:56:27.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T09:56:27.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T09:56:27.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T09:56:27.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T09:56:27.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T09:56:27.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T09:56:27.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T09:56:27.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T09:56:27.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T09:56:27.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T09:56:27.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T09:56:27.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T09:56:27.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T09:56:27.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T09:56:27.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T09:56:27.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T09:56:27.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T09:56:27.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T09:56:27.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T09:56:27.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T09:56:27.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T09:56:27.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T09:56:27.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T09:56:27.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T09:56:27.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T09:56:27.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T09:56:27.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T09:56:27.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T09:56:27.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T09:56:27.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T09:56:27.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T09:56:27.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T09:56:27.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T09:56:27.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T09:56:27.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T09:56:27.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T09:56:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T09:56:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T09:56:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T09:56:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T09:56:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T09:56:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T09:56:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T09:56:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T09:56:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T09:56:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T09:56:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T09:56:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T09:56:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T09:56:27.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T09:56:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T09:56:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T09:56:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T09:56:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T09:56:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T09:56:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T09:56:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T09:56:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T09:56:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T09:56:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T09:56:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T09:56:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T09:56:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T09:56:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T09:56:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T09:56:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T09:56:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T09:56:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T09:56:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T09:56:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T09:56:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T09:56:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T09:56:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T09:56:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T09:56:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T09:56:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T09:56:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T09:56:27.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T09:56:27.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T09:56:27.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T09:56:27.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T09:56:27.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T09:56:27.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T09:56:27.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T09:56:27.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T09:56:27.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T09:56:27.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T09:56:27.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T09:56:27.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T09:56:27.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T09:56:27.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T09:56:27.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T09:56:27.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T09:56:27.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T09:56:27.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T09:56:27.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T09:56:27.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T09:56:27.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T09:56:27.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T09:56:27.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T09:56:27.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T09:56:27.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T09:56:27.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T09:56:27.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T09:56:27.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T09:56:27.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T09:56:27.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T09:56:27.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T09:56:27.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T09:56:27.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T09:56:27.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T09:56:27.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T09:56:27.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T09:56:27.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T09:56:27.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T09:56:27.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T09:56:27.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T09:56:27.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T09:56:27.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T09:56:27.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T09:56:27.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T09:56:27.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T09:56:27.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T09:56:27.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T09:56:27.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T09:56:27.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T09:56:27.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T09:56:27.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T09:56:27.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T09:56:27.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T09:56:27.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T09:56:27.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T09:56:27.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T09:56:27.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T09:56:27.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T09:56:27.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T09:56:27.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T09:56:27.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T09:56:27.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T09:56:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T09:56:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T09:56:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T09:56:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T09:56:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T09:56:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T09:56:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T09:56:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T09:56:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T09:56:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T09:56:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T09:56:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T09:56:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T09:56:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T09:56:27.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T09:56:27.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T09:56:27.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T09:56:27.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T09:56:27.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T09:56:27.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T09:56:27.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T09:56:27.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T09:56:27.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T09:56:27.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T09:56:27.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T09:56:27.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T09:56:27.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T09:56:27.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T09:56:27.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T09:56:27.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T09:56:27.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T09:56:27.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T09:56:27.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T09:56:27.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T09:56:27.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T09:56:27.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T09:56:27.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T09:56:27.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T09:56:27.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T09:56:27.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T09:56:27.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T09:56:27.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T09:56:27.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T09:56:27.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T09:56:27.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T09:56:27.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T09:56:27.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T09:56:27.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T09:56:27.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T09:56:27.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T09:56:27.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T09:56:27.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T09:56:27.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T09:56:27.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T09:56:27.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T09:56:27.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T09:56:27.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T09:56:27.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T09:56:27.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T09:56:27.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T09:56:27.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T09:56:27.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T09:56:27.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T09:56:27.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T09:56:27.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T09:56:27.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T09:56:27.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T09:56:27.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T09:56:27.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T09:56:27.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T09:56:27.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T09:56:27.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T09:56:27.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T09:56:27.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T09:56:27.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T09:56:27.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T09:56:27.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T09:56:27.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T09:56:27.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T09:56:27.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T09:56:27.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T09:56:27.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T09:56:27.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T09:56:27.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T09:56:27.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T09:56:27.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T09:56:27.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T09:56:27.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T09:56:27.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T09:56:27.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T09:56:27.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T09:56:27.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T09:56:27.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T09:56:27.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T09:56:27.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T09:56:27.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T09:56:27.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T09:56:27.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T09:56:27.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T09:56:27.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T09:56:27.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T09:56:27.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T09:56:27.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T09:56:27.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T09:56:27.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T09:56:27.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T09:56:27.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T09:56:27.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T09:56:27.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T09:56:27.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T09:56:27.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T09:56:27.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T09:56:27.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T09:56:27.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T09:56:27.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T09:56:27.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T09:56:27.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T09:56:27.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T09:56:27.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T09:56:27.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T09:56:27.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T09:56:27.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T09:56:27.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T09:56:27.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T09:56:27.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T09:56:27.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T09:56:27.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T09:56:27.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T09:56:27.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T09:56:27.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T09:56:27.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T09:56:27.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T09:56:27.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T09:56:27.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T09:56:27.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T09:56:27.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T09:56:27.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T09:56:27.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T09:56:27.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T09:56:27.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T09:56:27.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T09:56:27.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T09:56:27.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T09:56:27.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T09:56:27.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T09:56:27.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T09:56:27.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T09:56:27.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T09:56:27.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T09:56:27.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T09:56:27.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T09:56:27.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T09:56:27.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T09:56:27.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T09:56:27.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T09:56:27.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T09:56:27.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T09:56:27.556 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:27.556 DEBUG:teuthology.orchestra.run.smithi119:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T09:56:27.579 INFO:teuthology.orchestra.run.smithi119.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T09:56:27.579 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T09:56:27.579 INFO:teuthology.orchestra.run.smithi119.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T09:56:27.580 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-04-25T09:56:27.580 INFO:teuthology.orchestra.run.smithi119.stdout:for file in vmlinuz System.map config; do 2024-04-25T09:56:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T09:56:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:56:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: fi 2024-04-25T09:56:27.580 INFO:teuthology.orchestra.run.smithi119.stdout:done 2024-04-25T09:56:27.580 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T09:56:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T09:56:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:56:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:56:27.580 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-04-25T09:56:27.580 INFO:teuthology.orchestra.run.smithi119.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T09:56:27.580 INFO:teuthology.orchestra.run.smithi119.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:27.581 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-04-25T09:56:27.581 INFO:teuthology.orchestra.run.smithi119.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T09:56:27.581 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T09:56:27.581 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:27.581 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-04-25T09:56:27.582 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -qi grub2-tools 2024-04-25T09:56:27.654 INFO:teuthology.orchestra.run.smithi119.stdout:Name : grub2-tools 2024-04-25T09:56:27.654 INFO:teuthology.orchestra.run.smithi119.stdout:Epoch : 1 2024-04-25T09:56:27.654 INFO:teuthology.orchestra.run.smithi119.stdout:Version : 2.06 2024-04-25T09:56:27.654 INFO:teuthology.orchestra.run.smithi119.stdout:Release : 68.el9 2024-04-25T09:56:27.654 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: x86_64 2024-04-25T09:56:27.654 INFO:teuthology.orchestra.run.smithi119.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T09:56:27.654 INFO:teuthology.orchestra.run.smithi119.stdout:Group : Unspecified 2024-04-25T09:56:27.654 INFO:teuthology.orchestra.run.smithi119.stdout:Size : 8264254 2024-04-25T09:56:27.654 INFO:teuthology.orchestra.run.smithi119.stdout:License : GPLv3+ 2024-04-25T09:56:27.655 INFO:teuthology.orchestra.run.smithi119.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T09:56:27.655 INFO:teuthology.orchestra.run.smithi119.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T09:56:27.655 INFO:teuthology.orchestra.run.smithi119.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T09:56:27.655 INFO:teuthology.orchestra.run.smithi119.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T09:56:27.655 INFO:teuthology.orchestra.run.smithi119.stdout:Packager : builder@centos.org 2024-04-25T09:56:27.655 INFO:teuthology.orchestra.run.smithi119.stdout:Vendor : CentOS 2024-04-25T09:56:27.655 INFO:teuthology.orchestra.run.smithi119.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T09:56:27.655 INFO:teuthology.orchestra.run.smithi119.stdout:Summary : Support tools for GRUB. 2024-04-25T09:56:27.655 INFO:teuthology.orchestra.run.smithi119.stdout:Description : 2024-04-25T09:56:27.655 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T09:56:27.655 INFO:teuthology.orchestra.run.smithi119.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T09:56:27.655 INFO:teuthology.orchestra.run.smithi119.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T09:56:27.655 INFO:teuthology.orchestra.run.smithi119.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T09:56:27.655 INFO:teuthology.orchestra.run.smithi119.stdout:hardware devices. 2024-04-25T09:56:27.656 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T09:56:27.656 INFO:teuthology.orchestra.run.smithi119.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T09:56:27.656 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T09:56:27.657 INFO:teuthology.task.kernel:Updating grub on smithi119 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:27.657 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T09:56:28.262 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2024-04-25T09:56:29.668 INFO:teuthology.orchestra.run.smithi119.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T09:56:29.709 INFO:teuthology.orchestra.run.smithi119.stderr:done 2024-04-25T09:56:29.712 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-04-25T09:56:29.727 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.RFX7oNl370 2024-04-25T09:56:29.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RFX7oNl370 2024-04-25T09:56:29.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.RFX7oNl370 2024-04-25T09:56:29.932 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.RFX7oNl370 is 6KB 2024-04-25T09:56:29.981 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.RFX7oNl370 2024-04-25T09:56:29.994 DEBUG:teuthology.orchestra.run.smithi119:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T09:56:30.055 INFO:teuthology.orchestra.run.smithi119.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T09:56:30.055 INFO:teuthology.orchestra.run.smithi119.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T09:56:30.055 INFO:teuthology.orchestra.run.smithi119.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T09:56:30.055 INFO:teuthology.orchestra.run.smithi119.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T09:56:30.056 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:30.219 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2024-04-25T09:56:37.087 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi132.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T09:56:37.087 DEBUG:teuthology.orchestra.run.smithi132:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T09:56:37.162 INFO:teuthology.orchestra.run.smithi132.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:37.163 INFO:teuthology.orchestra.run.smithi132.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:37.163 INFO:teuthology.orchestra.run.smithi132.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:37.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T09:56:37.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T09:56:37.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T09:56:37.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T09:56:37.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T09:56:37.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T09:56:37.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T09:56:37.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T09:56:37.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T09:56:37.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T09:56:37.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T09:56:37.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T09:56:37.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T09:56:37.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T09:56:37.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T09:56:37.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T09:56:37.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T09:56:37.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T09:56:37.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T09:56:37.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T09:56:37.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T09:56:37.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T09:56:37.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T09:56:37.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T09:56:37.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T09:56:37.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T09:56:37.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T09:56:37.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T09:56:37.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T09:56:37.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T09:56:37.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T09:56:37.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T09:56:37.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T09:56:37.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T09:56:37.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T09:56:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T09:56:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T09:56:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T09:56:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T09:56:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T09:56:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T09:56:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T09:56:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T09:56:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T09:56:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T09:56:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T09:56:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T09:56:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T09:56:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T09:56:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T09:56:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T09:56:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T09:56:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T09:56:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T09:56:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T09:56:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T09:56:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T09:56:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T09:56:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T09:56:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T09:56:37.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T09:56:37.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T09:56:37.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T09:56:37.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T09:56:37.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T09:56:37.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T09:56:37.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T09:56:37.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T09:56:37.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T09:56:37.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T09:56:37.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T09:56:37.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T09:56:37.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T09:56:37.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T09:56:37.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T09:56:37.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T09:56:37.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T09:56:37.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T09:56:37.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T09:56:37.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T09:56:37.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T09:56:37.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T09:56:37.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T09:56:37.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T09:56:37.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T09:56:37.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T09:56:37.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T09:56:37.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T09:56:37.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T09:56:37.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T09:56:37.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T09:56:37.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T09:56:37.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T09:56:37.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T09:56:37.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T09:56:37.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T09:56:37.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T09:56:37.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T09:56:37.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T09:56:37.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T09:56:37.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T09:56:37.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T09:56:37.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T09:56:37.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T09:56:37.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T09:56:37.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T09:56:37.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T09:56:37.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T09:56:37.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T09:56:37.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T09:56:37.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T09:56:37.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T09:56:37.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T09:56:37.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T09:56:37.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T09:56:37.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T09:56:37.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T09:56:37.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T09:56:37.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T09:56:37.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T09:56:37.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T09:56:37.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T09:56:37.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T09:56:37.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T09:56:37.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T09:56:37.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T09:56:37.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T09:56:37.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T09:56:37.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T09:56:37.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T09:56:37.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T09:56:37.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T09:56:37.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T09:56:37.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T09:56:37.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T09:56:37.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T09:56:37.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T09:56:37.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T09:56:37.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T09:56:37.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T09:56:37.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T09:56:37.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T09:56:37.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T09:56:37.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T09:56:37.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T09:56:37.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T09:56:37.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T09:56:37.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T09:56:37.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T09:56:37.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T09:56:37.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T09:56:37.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T09:56:37.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T09:56:37.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T09:56:37.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T09:56:37.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T09:56:37.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T09:56:37.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T09:56:37.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T09:56:37.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T09:56:37.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T09:56:37.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T09:56:37.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T09:56:37.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T09:56:37.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T09:56:37.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T09:56:37.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T09:56:37.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T09:56:37.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T09:56:37.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T09:56:37.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T09:56:37.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T09:56:37.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T09:56:37.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T09:56:37.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T09:56:37.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T09:56:37.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T09:56:37.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T09:56:37.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T09:56:37.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T09:56:37.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T09:56:37.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T09:56:37.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T09:56:37.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T09:56:37.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T09:56:37.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T09:56:37.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T09:56:37.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T09:56:37.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T09:56:37.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T09:56:37.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T09:56:37.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T09:56:37.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T09:56:37.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T09:56:37.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T09:56:37.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T09:56:37.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T09:56:37.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T09:56:37.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T09:56:37.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T09:56:37.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T09:56:37.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T09:56:37.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T09:56:37.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T09:56:37.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T09:56:37.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T09:56:37.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T09:56:37.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T09:56:37.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T09:56:37.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T09:56:37.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T09:56:37.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T09:56:37.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T09:56:37.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T09:56:37.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T09:56:37.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T09:56:37.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T09:56:37.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T09:56:37.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T09:56:37.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T09:56:37.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T09:56:37.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T09:56:37.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T09:56:37.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T09:56:37.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T09:56:37.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T09:56:37.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T09:56:37.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T09:56:37.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T09:56:37.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T09:56:37.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T09:56:37.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T09:56:37.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T09:56:37.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T09:56:37.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T09:56:37.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T09:56:37.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T09:56:37.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T09:56:37.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T09:56:37.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T09:56:37.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T09:56:37.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T09:56:37.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T09:56:37.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T09:56:37.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T09:56:37.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T09:56:37.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T09:56:37.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T09:56:37.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T09:56:37.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T09:56:37.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T09:56:37.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T09:56:37.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T09:56:37.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T09:56:37.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T09:56:37.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T09:56:37.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T09:56:37.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T09:56:37.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T09:56:37.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T09:56:37.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T09:56:37.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T09:56:37.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T09:56:37.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T09:56:37.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T09:56:37.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T09:56:37.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T09:56:37.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T09:56:37.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T09:56:37.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T09:56:37.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T09:56:37.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T09:56:37.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T09:56:37.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T09:56:37.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T09:56:37.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T09:56:37.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T09:56:37.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T09:56:37.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T09:56:37.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T09:56:37.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T09:56:37.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T09:56:37.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T09:56:37.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T09:56:37.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T09:56:37.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T09:56:37.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T09:56:37.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T09:56:37.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T09:56:37.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T09:56:37.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T09:56:37.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T09:56:37.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T09:56:37.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T09:56:37.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T09:56:37.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T09:56:37.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T09:56:37.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T09:56:37.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T09:56:37.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T09:56:37.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T09:56:37.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T09:56:37.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T09:56:37.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T09:56:37.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T09:56:37.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T09:56:37.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T09:56:37.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T09:56:37.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T09:56:37.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T09:56:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T09:56:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T09:56:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T09:56:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T09:56:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T09:56:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T09:56:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T09:56:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T09:56:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T09:56:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T09:56:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T09:56:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T09:56:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T09:56:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T09:56:37.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T09:56:37.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T09:56:37.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T09:56:37.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T09:56:37.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T09:56:37.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T09:56:37.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T09:56:37.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T09:56:37.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T09:56:37.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T09:56:37.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T09:56:37.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T09:56:37.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T09:56:37.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T09:56:37.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T09:56:37.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T09:56:37.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T09:56:37.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T09:56:37.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T09:56:37.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T09:56:37.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T09:56:37.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T09:56:37.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T09:56:37.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T09:56:37.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T09:56:37.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T09:56:37.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T09:56:37.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T09:56:37.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T09:56:37.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T09:56:37.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T09:56:37.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T09:56:37.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T09:56:37.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T09:56:37.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T09:56:37.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T09:56:37.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T09:56:37.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T09:56:37.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T09:56:37.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T09:56:37.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T09:56:37.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T09:56:37.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T09:56:37.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T09:56:37.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T09:56:37.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T09:56:37.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T09:56:37.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T09:56:37.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T09:56:37.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T09:56:37.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T09:56:37.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T09:56:37.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T09:56:37.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T09:56:37.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T09:56:37.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T09:56:37.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T09:56:37.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T09:56:37.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T09:56:37.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T09:56:37.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T09:56:37.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T09:56:37.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T09:56:37.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T09:56:37.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T09:56:37.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T09:56:37.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T09:56:37.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T09:56:37.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T09:56:37.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T09:56:37.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T09:56:37.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T09:56:37.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T09:56:37.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T09:56:37.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T09:56:37.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T09:56:37.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T09:56:37.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T09:56:37.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T09:56:37.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T09:56:37.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T09:56:37.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T09:56:37.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T09:56:37.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T09:56:37.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T09:56:37.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T09:56:37.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T09:56:37.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T09:56:37.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T09:56:37.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T09:56:37.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T09:56:37.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T09:56:37.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T09:56:37.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T09:56:37.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T09:56:37.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T09:56:37.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T09:56:37.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T09:56:37.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T09:56:37.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T09:56:37.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T09:56:37.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T09:56:37.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T09:56:37.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T09:56:37.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T09:56:37.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T09:56:37.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T09:56:37.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T09:56:37.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T09:56:37.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T09:56:37.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T09:56:37.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T09:56:37.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T09:56:37.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T09:56:37.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T09:56:37.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T09:56:37.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T09:56:37.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T09:56:37.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T09:56:37.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T09:56:37.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T09:56:37.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T09:56:37.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T09:56:37.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T09:56:37.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T09:56:37.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T09:56:37.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T09:56:37.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T09:56:37.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T09:56:37.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T09:56:37.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T09:56:37.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T09:56:37.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T09:56:37.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T09:56:37.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T09:56:37.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T09:56:37.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T09:56:37.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T09:56:37.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T09:56:37.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T09:56:37.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T09:56:37.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T09:56:37.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T09:56:37.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T09:56:37.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T09:56:37.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T09:56:37.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T09:56:37.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T09:56:37.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T09:56:37.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T09:56:37.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T09:56:37.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T09:56:37.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T09:56:37.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T09:56:37.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T09:56:37.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T09:56:37.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T09:56:37.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T09:56:37.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T09:56:37.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T09:56:37.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T09:56:37.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T09:56:37.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T09:56:37.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T09:56:37.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T09:56:37.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T09:56:37.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T09:56:37.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T09:56:37.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T09:56:37.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T09:56:37.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T09:56:37.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T09:56:37.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T09:56:37.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T09:56:37.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T09:56:37.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T09:56:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T09:56:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T09:56:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T09:56:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T09:56:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T09:56:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T09:56:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T09:56:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T09:56:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T09:56:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T09:56:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T09:56:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T09:56:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T09:56:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T09:56:37.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T09:56:37.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T09:56:37.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T09:56:37.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T09:56:37.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T09:56:37.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T09:56:37.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T09:56:37.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T09:56:37.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T09:56:37.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T09:56:37.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T09:56:37.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T09:56:37.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T09:56:37.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T09:56:37.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T09:56:37.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T09:56:37.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T09:56:37.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T09:56:37.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T09:56:37.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T09:56:37.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T09:56:37.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T09:56:37.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T09:56:37.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T09:56:37.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T09:56:37.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T09:56:37.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T09:56:37.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T09:56:37.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T09:56:37.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T09:56:37.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T09:56:37.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T09:56:37.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T09:56:37.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T09:56:37.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T09:56:37.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T09:56:37.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T09:56:37.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T09:56:37.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T09:56:37.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T09:56:37.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T09:56:37.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T09:56:37.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T09:56:37.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T09:56:37.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T09:56:37.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T09:56:37.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T09:56:37.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T09:56:37.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T09:56:37.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T09:56:37.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T09:56:37.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T09:56:37.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T09:56:37.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T09:56:37.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T09:56:37.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T09:56:37.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T09:56:37.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T09:56:37.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T09:56:37.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T09:56:37.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T09:56:37.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T09:56:37.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T09:56:37.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T09:56:37.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T09:56:37.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T09:56:37.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T09:56:37.215 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T09:56:37.216 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T09:56:37.216 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T09:56:37.216 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T09:56:37.216 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T09:56:37.216 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T09:56:37.216 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T09:56:37.216 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T09:56:37.216 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T09:56:37.216 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T09:56:37.216 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T09:56:37.216 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T09:56:37.216 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T09:56:37.217 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T09:56:37.218 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T09:56:37.218 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T09:56:37.218 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T09:56:37.218 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T09:56:37.218 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T09:56:37.218 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T09:56:37.218 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T09:56:37.218 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T09:56:37.218 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T09:56:37.218 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T09:56:37.218 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T09:56:37.219 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T09:56:37.219 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T09:56:37.219 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T09:56:37.219 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T09:56:37.219 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T09:56:37.219 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T09:56:37.219 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T09:56:37.219 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T09:56:37.219 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T09:56:37.219 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T09:56:37.219 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T09:56:37.220 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T09:56:37.220 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T09:56:37.220 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T09:56:37.220 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T09:56:37.220 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T09:56:37.220 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T09:56:37.220 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T09:56:37.220 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T09:56:37.220 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T09:56:37.220 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T09:56:37.220 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T09:56:37.220 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T09:56:37.220 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T09:56:37.221 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T09:56:37.221 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T09:56:37.221 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T09:56:37.221 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T09:56:37.221 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T09:56:37.221 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T09:56:37.221 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T09:56:37.221 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T09:56:37.221 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T09:56:37.221 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T09:56:37.221 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T09:56:37.222 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T09:56:37.222 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T09:56:37.222 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T09:56:37.222 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T09:56:37.222 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T09:56:37.222 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T09:56:37.222 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T09:56:37.222 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T09:56:37.222 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T09:56:37.222 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T09:56:37.222 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T09:56:37.222 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T09:56:37.222 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T09:56:37.223 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T09:56:37.223 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T09:56:37.223 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T09:56:37.223 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T09:56:37.223 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T09:56:37.223 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T09:56:37.223 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T09:56:37.223 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T09:56:37.223 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T09:56:37.223 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T09:56:37.223 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T09:56:37.224 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T09:56:37.224 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T09:56:37.224 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T09:56:37.224 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T09:56:37.224 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T09:56:37.224 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T09:56:37.224 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T09:56:37.224 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T09:56:37.224 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T09:56:37.224 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T09:56:37.224 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T09:56:37.224 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T09:56:37.224 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T09:56:37.224 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T09:56:37.225 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T09:56:37.225 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T09:56:37.225 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T09:56:37.225 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T09:56:37.225 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T09:56:37.225 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T09:56:37.225 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T09:56:37.225 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T09:56:37.225 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T09:56:37.226 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T09:56:37.226 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T09:56:37.226 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T09:56:37.226 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T09:56:37.226 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T09:56:37.226 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T09:56:37.226 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T09:56:37.226 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T09:56:37.226 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T09:56:37.226 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T09:56:37.226 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T09:56:37.227 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T09:56:37.227 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T09:56:37.227 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T09:56:37.227 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T09:56:37.227 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T09:56:37.227 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T09:56:37.227 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T09:56:37.227 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T09:56:37.227 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T09:56:37.227 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T09:56:37.227 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T09:56:37.228 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T09:56:37.228 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T09:56:37.228 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T09:56:37.228 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T09:56:37.228 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T09:56:37.228 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T09:56:37.228 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T09:56:37.228 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T09:56:37.228 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T09:56:37.228 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T09:56:37.229 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T09:56:37.229 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T09:56:37.229 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T09:56:37.229 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T09:56:37.229 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T09:56:37.229 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T09:56:37.229 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T09:56:37.229 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T09:56:37.229 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T09:56:37.229 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T09:56:37.230 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T09:56:37.230 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T09:56:37.230 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T09:56:37.230 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T09:56:37.230 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T09:56:37.230 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T09:56:37.230 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T09:56:37.230 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T09:56:37.230 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T09:56:37.230 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T09:56:37.230 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T09:56:37.230 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T09:56:37.231 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T09:56:37.231 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T09:56:37.231 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T09:56:37.231 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T09:56:37.231 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T09:56:37.231 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T09:56:37.231 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T09:56:37.231 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T09:56:37.231 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T09:56:37.231 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T09:56:37.231 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T09:56:37.231 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T09:56:37.231 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T09:56:37.231 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T09:56:37.232 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T09:56:37.232 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T09:56:37.232 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T09:56:37.232 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T09:56:37.232 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T09:56:37.232 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T09:56:37.232 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T09:56:37.232 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T09:56:37.232 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T09:56:37.232 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T09:56:37.232 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T09:56:37.232 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T09:56:37.232 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T09:56:37.232 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T09:56:37.233 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T09:56:37.233 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T09:56:37.233 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T09:56:37.233 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T09:56:37.233 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T09:56:37.233 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T09:56:37.233 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T09:56:37.233 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T09:56:37.233 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T09:56:37.233 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T09:56:37.233 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T09:56:37.233 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T09:56:37.233 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T09:56:37.234 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T09:56:37.234 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T09:56:37.234 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T09:56:37.234 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T09:56:37.234 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T09:56:37.234 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T09:56:37.234 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T09:56:37.234 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T09:56:37.234 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T09:56:37.234 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T09:56:37.234 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T09:56:37.234 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T09:56:37.234 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T09:56:37.235 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T09:56:37.235 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T09:56:37.235 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T09:56:37.235 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T09:56:37.235 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T09:56:37.235 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T09:56:37.235 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T09:56:37.235 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T09:56:37.235 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T09:56:37.235 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T09:56:37.235 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T09:56:37.235 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T09:56:37.235 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T09:56:37.235 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T09:56:37.236 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T09:56:37.237 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T09:56:37.238 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T09:56:37.238 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T09:56:37.238 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T09:56:37.238 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T09:56:37.238 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T09:56:37.238 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T09:56:37.238 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T09:56:37.238 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T09:56:37.238 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T09:56:37.238 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T09:56:37.238 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T09:56:37.238 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T09:56:37.238 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T09:56:37.239 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T09:56:37.239 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T09:56:37.239 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T09:56:37.239 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T09:56:37.239 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T09:56:37.239 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T09:56:37.239 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T09:56:37.239 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T09:56:37.239 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T09:56:37.239 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T09:56:37.239 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T09:56:37.239 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T09:56:37.239 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T09:56:37.239 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T09:56:37.240 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T09:56:37.240 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T09:56:37.240 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T09:56:37.240 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T09:56:37.240 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T09:56:37.240 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T09:56:37.240 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T09:56:37.240 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T09:56:37.240 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T09:56:37.240 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T09:56:37.240 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T09:56:37.240 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T09:56:37.240 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T09:56:37.240 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T09:56:37.241 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T09:56:37.242 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T09:56:37.242 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T09:56:37.242 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T09:56:37.242 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T09:56:37.242 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T09:56:37.242 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T09:56:37.242 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T09:56:37.242 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T09:56:37.242 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T09:56:37.242 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T09:56:37.242 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T09:56:37.242 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T09:56:37.242 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T09:56:37.243 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T09:56:37.243 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T09:56:37.243 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T09:56:37.243 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T09:56:37.243 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T09:56:37.243 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T09:56:37.243 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T09:56:37.243 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T09:56:37.243 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T09:56:37.243 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T09:56:37.243 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T09:56:37.243 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T09:56:37.243 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T09:56:37.243 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T09:56:37.244 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T09:56:37.244 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T09:56:37.244 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T09:56:37.244 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T09:56:37.244 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T09:56:37.244 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T09:56:37.244 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T09:56:37.244 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T09:56:37.244 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T09:56:37.244 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T09:56:37.244 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T09:56:37.244 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T09:56:37.244 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T09:56:37.245 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T09:56:37.245 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T09:56:37.245 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T09:56:37.245 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T09:56:37.245 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T09:56:37.245 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T09:56:37.245 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T09:56:37.245 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T09:56:37.245 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T09:56:37.245 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T09:56:37.245 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T09:56:37.245 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T09:56:37.245 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T09:56:37.246 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T09:56:37.246 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T09:56:37.246 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T09:56:37.246 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T09:56:37.246 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T09:56:37.246 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T09:56:37.246 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T09:56:37.246 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T09:56:37.246 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T09:56:37.246 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T09:56:37.246 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T09:56:37.246 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T09:56:37.247 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T09:56:37.247 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T09:56:37.247 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T09:56:37.247 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T09:56:37.247 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T09:56:37.247 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T09:56:37.247 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T09:56:37.247 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T09:56:37.247 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T09:56:37.247 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T09:56:37.247 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T09:56:37.247 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T09:56:37.247 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T09:56:37.247 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T09:56:37.248 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T09:56:37.248 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T09:56:37.248 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T09:56:37.248 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T09:56:37.248 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T09:56:37.248 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T09:56:37.248 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T09:56:37.248 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T09:56:37.248 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T09:56:37.248 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T09:56:37.248 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T09:56:37.248 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T09:56:37.248 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T09:56:37.248 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T09:56:37.249 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T09:56:37.249 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T09:56:37.249 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T09:56:37.249 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T09:56:37.249 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T09:56:37.249 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T09:56:37.249 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T09:56:37.249 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T09:56:37.249 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T09:56:37.249 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T09:56:37.249 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T09:56:37.249 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T09:56:37.249 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T09:56:37.249 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T09:56:37.250 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T09:56:37.250 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T09:56:37.250 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T09:56:37.250 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T09:56:37.250 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T09:56:37.250 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T09:56:37.250 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T09:56:37.250 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T09:56:37.250 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T09:56:37.250 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T09:56:37.250 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T09:56:37.250 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T09:56:37.250 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T09:56:37.251 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T09:56:37.251 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T09:56:37.251 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T09:56:37.251 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T09:56:37.251 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T09:56:37.251 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T09:56:37.251 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T09:56:37.251 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T09:56:37.251 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T09:56:37.251 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T09:56:37.251 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T09:56:37.251 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T09:56:37.251 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T09:56:37.252 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T09:56:37.253 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T09:56:37.253 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T09:56:37.253 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T09:56:37.253 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T09:56:37.253 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T09:56:37.253 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T09:56:37.253 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T09:56:37.253 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T09:56:37.253 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T09:56:37.253 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T09:56:37.253 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T09:56:37.253 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T09:56:37.254 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T09:56:37.254 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T09:56:37.254 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T09:56:37.254 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T09:56:37.254 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T09:56:37.254 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T09:56:37.254 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T09:56:37.254 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T09:56:37.254 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T09:56:37.254 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T09:56:37.255 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T09:56:37.255 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T09:56:37.255 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T09:56:37.255 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T09:56:37.255 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T09:56:37.255 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T09:56:37.255 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T09:56:37.255 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T09:56:37.255 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T09:56:37.255 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T09:56:37.255 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T09:56:37.255 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T09:56:37.255 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T09:56:37.255 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T09:56:37.256 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T09:56:37.256 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T09:56:37.256 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T09:56:37.256 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T09:56:37.256 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T09:56:37.256 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T09:56:37.256 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T09:56:37.256 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T09:56:37.256 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T09:56:37.256 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T09:56:37.257 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T09:56:37.257 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T09:56:37.257 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T09:56:37.257 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T09:56:37.257 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T09:56:37.257 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T09:56:37.257 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T09:56:37.257 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T09:56:37.257 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T09:56:37.257 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T09:56:37.257 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T09:56:37.257 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T09:56:37.257 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T09:56:37.258 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T09:56:37.258 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T09:56:37.258 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T09:56:37.258 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T09:56:37.258 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T09:56:37.258 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T09:56:37.258 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T09:56:37.258 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T09:56:37.258 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T09:56:37.258 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T09:56:37.258 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T09:56:37.259 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T09:56:37.259 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T09:56:37.259 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T09:56:37.259 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T09:56:37.259 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T09:56:37.259 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T09:56:37.259 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T09:56:37.259 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T09:56:37.259 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T09:56:37.259 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T09:56:37.259 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T09:56:37.260 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T09:56:37.260 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T09:56:37.260 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T09:56:37.260 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T09:56:37.260 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T09:56:37.260 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T09:56:37.260 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T09:56:37.260 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T09:56:37.260 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T09:56:37.260 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T09:56:37.260 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T09:56:37.260 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T09:56:37.261 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T09:56:37.261 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T09:56:37.261 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T09:56:37.261 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T09:56:37.261 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T09:56:37.261 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T09:56:37.261 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T09:56:37.261 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T09:56:37.261 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T09:56:37.261 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T09:56:37.261 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T09:56:37.262 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T09:56:37.262 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T09:56:37.262 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T09:56:37.262 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T09:56:37.262 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T09:56:37.262 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T09:56:37.262 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T09:56:37.262 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T09:56:37.262 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T09:56:37.262 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T09:56:37.263 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T09:56:37.263 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T09:56:37.263 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T09:56:37.263 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T09:56:37.263 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T09:56:37.263 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T09:56:37.263 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T09:56:37.263 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T09:56:37.263 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T09:56:37.263 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T09:56:37.263 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T09:56:37.264 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T09:56:37.264 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T09:56:37.264 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T09:56:37.264 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T09:56:37.264 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T09:56:37.264 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T09:56:37.264 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T09:56:37.264 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T09:56:37.264 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T09:56:37.264 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T09:56:37.264 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T09:56:37.264 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T09:56:37.265 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T09:56:37.266 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T09:56:37.266 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T09:56:37.266 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T09:56:37.266 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T09:56:37.266 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T09:56:37.266 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T09:56:37.266 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T09:56:37.266 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T09:56:37.266 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T09:56:37.266 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T09:56:37.266 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T09:56:37.266 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T09:56:37.266 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T09:56:37.267 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T09:56:37.268 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T09:56:37.268 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T09:56:37.268 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T09:56:37.268 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T09:56:37.268 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T09:56:37.268 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T09:56:37.268 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T09:56:37.268 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T09:56:37.268 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T09:56:37.268 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T09:56:37.268 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T09:56:37.268 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T09:56:37.268 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T09:56:37.269 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T09:56:37.269 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T09:56:37.269 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T09:56:37.269 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T09:56:37.269 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T09:56:37.269 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T09:56:37.269 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T09:56:37.269 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T09:56:37.269 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T09:56:37.270 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T09:56:37.270 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T09:56:37.270 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T09:56:37.270 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T09:56:37.270 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T09:56:37.270 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T09:56:37.270 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T09:56:37.270 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T09:56:37.270 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T09:56:37.270 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T09:56:37.270 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T09:56:37.270 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T09:56:37.270 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T09:56:37.271 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T09:56:37.272 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T09:56:37.272 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T09:56:37.272 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T09:56:37.272 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T09:56:37.272 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T09:56:37.272 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T09:56:37.272 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T09:56:37.272 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T09:56:37.272 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T09:56:37.272 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T09:56:37.272 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T09:56:37.272 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T09:56:37.272 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T09:56:37.272 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T09:56:37.273 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T09:56:37.273 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T09:56:37.273 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T09:56:37.273 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T09:56:37.273 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T09:56:37.273 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T09:56:37.273 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T09:56:37.273 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T09:56:37.273 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T09:56:37.273 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T09:56:37.273 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T09:56:37.273 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T09:56:37.273 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T09:56:37.273 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T09:56:37.274 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T09:56:37.274 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T09:56:37.274 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T09:56:37.274 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T09:56:37.274 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T09:56:37.274 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T09:56:37.274 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T09:56:37.274 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T09:56:37.274 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T09:56:37.274 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T09:56:37.274 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T09:56:37.274 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T09:56:37.274 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T09:56:37.275 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T09:56:37.276 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T09:56:37.277 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T09:56:37.278 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T09:56:37.278 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T09:56:37.278 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T09:56:37.278 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T09:56:37.278 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T09:56:37.278 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T09:56:37.278 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T09:56:37.278 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T09:56:37.278 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T09:56:37.278 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T09:56:37.278 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T09:56:37.278 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T09:56:37.278 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T09:56:37.278 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T09:56:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T09:56:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T09:56:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T09:56:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T09:56:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T09:56:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T09:56:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T09:56:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T09:56:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T09:56:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T09:56:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T09:56:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T09:56:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T09:56:37.279 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T09:56:37.280 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T09:56:37.280 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T09:56:37.280 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T09:56:37.280 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T09:56:37.280 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T09:56:37.280 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T09:56:37.280 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T09:56:37.280 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T09:56:37.280 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T09:56:37.280 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T09:56:37.280 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T09:56:37.280 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T09:56:37.280 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T09:56:37.280 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T09:56:37.281 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T09:56:37.281 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T09:56:37.281 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T09:56:37.281 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T09:56:37.281 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T09:56:37.281 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T09:56:37.281 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T09:56:37.281 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T09:56:37.281 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T09:56:37.281 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T09:56:37.281 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T09:56:37.281 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T09:56:37.281 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T09:56:37.281 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T09:56:37.282 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T09:56:37.282 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T09:56:37.282 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T09:56:37.282 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T09:56:37.282 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T09:56:37.282 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T09:56:37.282 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T09:56:37.282 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T09:56:37.282 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T09:56:37.282 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T09:56:37.282 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T09:56:37.282 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T09:56:37.282 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T09:56:37.282 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T09:56:37.283 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T09:56:37.283 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T09:56:37.283 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T09:56:37.283 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T09:56:37.283 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T09:56:37.283 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T09:56:37.283 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T09:56:37.283 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T09:56:37.283 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T09:56:37.283 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T09:56:37.283 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T09:56:37.283 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T09:56:37.283 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T09:56:37.283 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T09:56:37.284 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T09:56:37.284 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T09:56:37.284 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T09:56:37.284 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T09:56:37.284 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T09:56:37.284 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T09:56:37.284 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T09:56:37.284 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T09:56:37.284 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T09:56:37.284 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T09:56:37.284 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T09:56:37.284 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T09:56:37.284 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T09:56:37.284 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T09:56:37.285 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T09:56:37.286 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T09:56:37.286 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T09:56:37.286 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T09:56:37.286 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T09:56:37.286 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T09:56:37.286 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T09:56:37.286 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T09:56:37.286 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T09:56:37.286 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T09:56:37.286 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T09:56:37.286 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T09:56:37.286 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T09:56:37.286 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T09:56:37.286 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T09:56:37.287 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T09:56:37.287 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T09:56:37.287 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T09:56:37.287 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T09:56:37.287 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T09:56:37.287 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T09:56:37.287 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T09:56:37.287 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T09:56:37.287 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T09:56:37.287 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T09:56:37.287 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T09:56:37.287 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T09:56:37.287 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T09:56:37.287 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T09:56:37.288 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T09:56:37.289 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T09:56:37.289 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T09:56:37.289 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T09:56:37.289 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T09:56:37.289 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T09:56:37.289 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T09:56:37.289 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T09:56:37.289 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T09:56:37.289 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T09:56:37.289 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T09:56:37.289 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T09:56:37.289 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T09:56:37.289 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T09:56:37.289 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T09:56:37.290 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T09:56:37.290 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T09:56:37.290 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T09:56:37.290 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T09:56:37.290 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T09:56:37.290 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T09:56:37.290 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T09:56:37.290 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T09:56:37.290 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T09:56:37.290 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T09:56:37.290 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T09:56:37.291 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T09:56:37.291 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T09:56:37.291 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T09:56:37.291 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T09:56:37.291 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T09:56:37.291 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T09:56:37.291 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T09:56:37.291 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T09:56:37.291 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T09:56:37.291 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T09:56:37.291 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T09:56:37.291 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T09:56:37.291 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T09:56:37.292 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T09:56:37.292 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T09:56:37.292 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T09:56:37.292 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T09:56:37.292 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T09:56:37.292 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T09:56:37.292 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T09:56:37.292 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T09:56:37.292 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T09:56:37.292 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T09:56:37.292 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T09:56:37.292 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T09:56:37.293 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T09:56:37.293 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T09:56:37.293 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T09:56:37.293 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T09:56:37.293 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T09:56:37.293 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T09:56:37.293 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T09:56:37.293 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T09:56:37.293 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T09:56:37.293 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T09:56:37.293 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T09:56:37.293 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T09:56:37.293 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T09:56:37.294 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T09:56:37.294 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T09:56:37.294 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T09:56:37.294 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T09:56:37.294 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T09:56:37.294 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T09:56:37.294 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T09:56:37.294 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T09:56:37.294 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T09:56:37.294 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T09:56:37.294 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T09:56:37.294 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T09:56:37.295 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T09:56:37.295 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T09:56:37.295 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T09:56:37.295 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T09:56:37.295 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T09:56:37.295 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T09:56:37.295 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T09:56:37.295 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T09:56:37.295 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T09:56:37.295 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T09:56:37.295 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T09:56:37.295 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T09:56:37.296 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T09:56:37.296 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T09:56:37.296 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T09:56:37.296 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T09:56:37.296 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T09:56:37.296 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T09:56:37.296 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T09:56:37.296 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T09:56:37.296 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T09:56:37.296 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T09:56:37.296 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T09:56:37.297 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T09:56:37.297 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T09:56:37.297 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T09:56:37.297 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T09:56:37.297 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T09:56:37.297 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T09:56:37.297 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T09:56:37.297 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T09:56:37.297 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T09:56:37.297 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T09:56:37.297 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T09:56:37.297 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T09:56:37.297 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T09:56:37.298 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T09:56:37.298 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T09:56:37.298 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T09:56:37.298 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T09:56:37.298 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T09:56:37.298 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T09:56:37.298 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T09:56:37.298 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T09:56:37.298 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T09:56:37.298 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T09:56:37.299 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T09:56:37.299 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T09:56:37.299 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T09:56:37.299 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T09:56:37.299 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T09:56:37.299 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T09:56:37.299 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T09:56:37.299 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T09:56:37.299 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T09:56:37.299 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T09:56:37.299 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T09:56:37.299 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T09:56:37.300 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T09:56:37.300 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T09:56:37.300 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T09:56:37.300 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T09:56:37.300 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T09:56:37.300 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T09:56:37.300 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T09:56:37.300 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T09:56:37.300 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T09:56:37.300 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T09:56:37.301 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T09:56:37.301 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T09:56:37.301 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T09:56:37.301 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T09:56:37.301 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T09:56:37.301 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T09:56:37.301 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T09:56:37.301 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T09:56:37.301 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T09:56:37.301 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T09:56:37.301 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T09:56:37.302 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T09:56:37.302 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T09:56:37.302 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T09:56:37.302 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T09:56:37.302 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T09:56:37.302 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T09:56:37.302 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T09:56:37.302 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T09:56:37.302 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T09:56:37.303 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T09:56:37.303 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T09:56:37.303 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T09:56:37.303 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T09:56:37.303 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T09:56:37.303 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T09:56:37.303 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T09:56:37.303 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T09:56:37.303 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T09:56:37.303 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T09:56:37.303 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T09:56:37.303 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T09:56:37.303 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T09:56:37.303 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T09:56:37.304 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T09:56:37.304 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T09:56:37.304 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T09:56:37.304 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T09:56:37.304 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T09:56:37.304 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T09:56:37.304 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T09:56:37.304 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T09:56:37.304 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T09:56:37.304 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T09:56:37.304 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T09:56:37.304 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T09:56:37.304 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T09:56:37.304 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T09:56:37.305 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T09:56:37.305 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T09:56:37.305 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T09:56:37.305 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T09:56:37.305 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T09:56:37.305 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T09:56:37.305 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T09:56:37.305 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T09:56:37.305 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T09:56:37.305 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T09:56:37.305 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T09:56:37.305 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T09:56:37.305 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T09:56:37.305 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T09:56:37.306 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T09:56:37.306 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T09:56:37.306 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T09:56:37.306 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T09:56:37.306 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T09:56:37.306 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T09:56:37.306 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T09:56:37.306 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T09:56:37.306 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T09:56:37.306 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T09:56:37.306 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T09:56:37.306 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T09:56:37.306 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T09:56:37.307 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T09:56:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T09:56:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T09:56:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T09:56:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T09:56:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T09:56:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T09:56:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T09:56:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T09:56:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T09:56:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T09:56:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T09:56:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T09:56:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T09:56:37.308 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T09:56:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T09:56:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T09:56:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T09:56:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T09:56:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T09:56:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T09:56:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T09:56:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T09:56:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T09:56:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T09:56:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T09:56:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T09:56:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T09:56:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T09:56:37.310 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T09:56:37.310 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T09:56:37.310 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T09:56:37.310 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T09:56:37.310 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T09:56:37.310 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T09:56:37.310 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T09:56:37.310 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T09:56:37.310 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T09:56:37.310 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T09:56:37.310 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T09:56:37.310 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T09:56:37.310 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T09:56:37.310 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T09:56:37.311 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T09:56:37.312 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T09:56:37.313 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T09:56:37.313 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T09:56:37.313 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T09:56:37.313 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T09:56:37.313 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T09:56:37.313 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T09:56:37.313 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T09:56:37.313 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T09:56:37.313 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T09:56:37.313 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T09:56:37.313 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T09:56:37.317 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:37.317 DEBUG:teuthology.orchestra.run.smithi132:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T09:56:37.339 INFO:teuthology.orchestra.run.smithi132.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T09:56:37.339 INFO:teuthology.orchestra.run.smithi132.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T09:56:37.339 INFO:teuthology.orchestra.run.smithi132.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T09:56:37.339 INFO:teuthology.orchestra.run.smithi132.stdout:fi 2024-04-25T09:56:37.339 INFO:teuthology.orchestra.run.smithi132.stdout:for file in vmlinuz System.map config; do 2024-04-25T09:56:37.339 INFO:teuthology.orchestra.run.smithi132.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T09:56:37.339 INFO:teuthology.orchestra.run.smithi132.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:56:37.339 INFO:teuthology.orchestra.run.smithi132.stdout: fi 2024-04-25T09:56:37.339 INFO:teuthology.orchestra.run.smithi132.stdout:done 2024-04-25T09:56:37.339 INFO:teuthology.orchestra.run.smithi132.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T09:56:37.340 INFO:teuthology.orchestra.run.smithi132.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T09:56:37.340 INFO:teuthology.orchestra.run.smithi132.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:56:37.340 INFO:teuthology.orchestra.run.smithi132.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:56:37.340 INFO:teuthology.orchestra.run.smithi132.stdout:fi 2024-04-25T09:56:37.340 INFO:teuthology.orchestra.run.smithi132.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T09:56:37.340 INFO:teuthology.orchestra.run.smithi132.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:37.340 INFO:teuthology.orchestra.run.smithi132.stdout:fi 2024-04-25T09:56:37.340 INFO:teuthology.orchestra.run.smithi132.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T09:56:37.340 INFO:teuthology.orchestra.run.smithi132.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T09:56:37.340 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:37.340 INFO:teuthology.orchestra.run.smithi132.stdout:fi 2024-04-25T09:56:37.341 DEBUG:teuthology.orchestra.run.smithi132:> sudo rpm -qi grub2-tools 2024-04-25T09:56:37.412 INFO:teuthology.orchestra.run.smithi132.stdout:Name : grub2-tools 2024-04-25T09:56:37.412 INFO:teuthology.orchestra.run.smithi132.stdout:Epoch : 1 2024-04-25T09:56:37.412 INFO:teuthology.orchestra.run.smithi132.stdout:Version : 2.06 2024-04-25T09:56:37.412 INFO:teuthology.orchestra.run.smithi132.stdout:Release : 68.el9 2024-04-25T09:56:37.412 INFO:teuthology.orchestra.run.smithi132.stdout:Architecture: x86_64 2024-04-25T09:56:37.412 INFO:teuthology.orchestra.run.smithi132.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T09:56:37.412 INFO:teuthology.orchestra.run.smithi132.stdout:Group : Unspecified 2024-04-25T09:56:37.412 INFO:teuthology.orchestra.run.smithi132.stdout:Size : 8264254 2024-04-25T09:56:37.412 INFO:teuthology.orchestra.run.smithi132.stdout:License : GPLv3+ 2024-04-25T09:56:37.412 INFO:teuthology.orchestra.run.smithi132.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T09:56:37.412 INFO:teuthology.orchestra.run.smithi132.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T09:56:37.413 INFO:teuthology.orchestra.run.smithi132.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T09:56:37.413 INFO:teuthology.orchestra.run.smithi132.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T09:56:37.413 INFO:teuthology.orchestra.run.smithi132.stdout:Packager : builder@centos.org 2024-04-25T09:56:37.413 INFO:teuthology.orchestra.run.smithi132.stdout:Vendor : CentOS 2024-04-25T09:56:37.413 INFO:teuthology.orchestra.run.smithi132.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T09:56:37.413 INFO:teuthology.orchestra.run.smithi132.stdout:Summary : Support tools for GRUB. 2024-04-25T09:56:37.413 INFO:teuthology.orchestra.run.smithi132.stdout:Description : 2024-04-25T09:56:37.413 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T09:56:37.413 INFO:teuthology.orchestra.run.smithi132.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T09:56:37.413 INFO:teuthology.orchestra.run.smithi132.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T09:56:37.413 INFO:teuthology.orchestra.run.smithi132.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T09:56:37.413 INFO:teuthology.orchestra.run.smithi132.stdout:hardware devices. 2024-04-25T09:56:37.413 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T09:56:37.413 INFO:teuthology.orchestra.run.smithi132.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T09:56:37.414 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T09:56:37.414 INFO:teuthology.task.kernel:Updating grub on smithi132 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:37.414 DEBUG:teuthology.orchestra.run.smithi132:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T09:56:37.939 INFO:teuthology.orchestra.run.smithi132.stderr:Generating grub configuration file ... 2024-04-25T09:56:39.318 INFO:teuthology.orchestra.run.smithi132.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T09:56:39.344 INFO:teuthology.orchestra.run.smithi132.stderr:done 2024-04-25T09:56:39.346 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-04-25T09:56:39.360 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.kLTJzaRp0J 2024-04-25T09:56:39.360 DEBUG:teuthology.orchestra.run.smithi132:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kLTJzaRp0J 2024-04-25T09:56:39.426 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.kLTJzaRp0J 2024-04-25T09:56:39.569 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.kLTJzaRp0J is 6KB 2024-04-25T09:56:39.618 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.kLTJzaRp0J 2024-04-25T09:56:39.632 DEBUG:teuthology.orchestra.run.smithi132:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T09:56:39.701 INFO:teuthology.orchestra.run.smithi132.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T09:56:39.702 INFO:teuthology.orchestra.run.smithi132.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T09:56:39.702 INFO:teuthology.orchestra.run.smithi132.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T09:56:39.702 INFO:teuthology.orchestra.run.smithi132.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T09:56:39.703 DEBUG:teuthology.orchestra.run.smithi132:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T09:56:39.854 DEBUG:teuthology.orchestra.run.smithi132:> sudo shutdown -r now 2024-04-25T09:57:00.242 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T09:57:00.244 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-04-25T09:57:00.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:57:09.866 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-25T09:57:09.867 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-04-25T09:57:09.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:57:18.750 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-04-25T09:57:27.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-04-25T09:57:27.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:57:28.222 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.132 2024-04-25T09:57:37.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-04-25T09:57:37.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:58:27.811 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T09:58:37.245 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T09:58:39.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-04-25T09:58:39.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:58:40.237 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-04-25T09:58:40.897 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-04-25T09:58:40.897 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T09:58:40.897 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-04-25T09:58:40.947 INFO:teuthology.orchestra.run.smithi119.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:40.947 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:40.948 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T09:58:40.948 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T09:58:41.949 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:49.246 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-04-25T09:58:49.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:58:49.641 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-04-25T09:58:50.304 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-04-25T09:58:50.304 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T09:58:50.305 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-04-25T09:58:50.357 INFO:teuthology.orchestra.run.smithi132.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:50.357 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T09:58:50.357 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T09:58:50.357 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T09:58:51.358 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:51.359 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T09:58:51.366 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T09:58:51.367 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:58:51.374 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:58:51.430 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T09:58:51.442 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T09:58:51.448 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T09:58:51.448 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:58:51.453 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:58:51.524 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T09:58:51.532 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T09:58:51.532 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:58:51.540 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:58:51.656 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:58:51.666 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:58:51.708 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:58:51.717 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:58:51.722 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T09:58:51.730 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T09:58:51.730 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:58:51.737 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:58:51.828 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T09:58:51.838 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T09:58:51.838 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:58:51.844 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:58:51.896 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:58:51.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:58:52.072 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:58:52.153 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:58:52.237 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T09:58:52.237 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:58:52.321 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:58:52.400 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:58:52.503 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:58:52.587 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:58:52.671 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T09:58:52.671 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:58:52.755 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-04-25T09:58:52.762 DEBUG:teuthology.orchestra.run.smithi132:> sudo service rsyslog restart 2024-04-25T09:58:52.874 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:58:52.901 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:58:53.323 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T09:58:53.332 INFO:teuthology.task.internal:Starting timer... 2024-04-25T09:58:53.332 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T09:58:53.358 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T09:58:53.367 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2024-04-25T09:58:53.473 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2024-04-25T09:58:53.477 DEBUG:teuthology.orchestra.run.smithi132:> sudo service auditd rotate 2024-04-25T09:58:53.579 INFO:teuthology.orchestra.run.smithi132.stdout:Rotating logs: 2024-04-25T09:58:53.583 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T09:58:53.583 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2024-04-25T09:58:53.640 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2024-04-25T09:58:53.640 DEBUG:teuthology.orchestra.run.smithi132:> /usr/sbin/getenforce 2024-04-25T09:58:53.694 INFO:teuthology.orchestra.run.smithi132.stdout:Permissive 2024-04-25T09:58:53.694 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi119.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi132.front.sepia.ceph.com': 'permissive'} 2024-04-25T09:58:53.695 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:58:53.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:58:53.773 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:58:53.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:58:53.860 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T09:58:53.860 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2024-04-25T09:58:53.930 DEBUG:teuthology.orchestra.run.smithi132:> sudo /usr/sbin/setenforce permissive 2024-04-25T09:58:54.017 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T09:58:54.031 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T09:58:54.051 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T09:58:54.052 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi119.front.sepia.ceph.com,smithi132.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T10:04:01.222 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi132.front.sepia.ceph.com')] 2024-04-25T10:04:01.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-04-25T10:04:01.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:04:01.341 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-04-25T10:04:01.484 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-04-25T10:04:01.485 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-04-25T10:04:01.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:04:01.603 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-04-25T10:04:01.760 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-04-25T10:04:01.760 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T10:04:01.772 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T10:04:01.773 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:04:01.773 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:04:01.781 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:04:01.781 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:04:01.871 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:04:01.871 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:04:01.912 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:04:01.912 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:04:01.981 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2024-04-25T10:04:01.983 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: ntpd: command not found 2024-04-25T10:04:02.011 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2024-04-25T10:04:02.012 INFO:teuthology.orchestra.run.smithi132.stdout:506 Cannot talk to daemon 2024-04-25T10:04:02.051 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:04:02.053 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:04:02.089 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:04:02.094 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:04:02.221 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:04:02.231 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:04:02.231 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-04-25T10:04:02.231 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:04:02.231 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:04:02.231 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:04:02.232 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:04:02.235 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:04:02.244 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:04:02.244 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-04-25T10:04:02.244 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:04:02.244 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:04:02.244 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:04:02.244 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:04:02.246 INFO:teuthology.run_tasks:Running task install... 2024-04-25T10:04:02.255 DEBUG:teuthology.task.install:project ceph 2024-04-25T10:04:02.256 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T10:04:02.256 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T10:04:02.256 INFO:teuthology.task.install:Using flavor: default 2024-04-25T10:04:02.262 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T10:04:02.262 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T10:04:02.263 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T10:04:02.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:04:02.266 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T10:04:02.266 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:04:02.413 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T10:04:02.413 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T10:04:02.435 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T10:04:02.435 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T10:04:02.704 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:04:02.704 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:04:02.704 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:04:02.726 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:04:02.727 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:04:02.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:04:02.794 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:04:02.794 DEBUG:teuthology.orchestra.run.smithi132:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:04:02.817 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:04:02.817 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:04:02.923 DEBUG:teuthology.orchestra.run.smithi132:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:04:02.946 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:04:03.034 DEBUG:teuthology.orchestra.run.smithi132:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:04:03.055 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:04:03.114 INFO:teuthology.orchestra.run.smithi132.stdout:check_obsoletes = 1 2024-04-25T10:04:03.118 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum clean all 2024-04-25T10:04:03.135 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2024-04-25T10:04:03.140 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2024-04-25T10:04:03.434 INFO:teuthology.orchestra.run.smithi132.stdout:48 files removed 2024-04-25T10:04:03.457 INFO:teuthology.orchestra.run.smithi119.stdout:48 files removed 2024-04-25T10:04:03.472 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:04:03.495 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:04:04.367 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 189 kB/s | 82 kB 00:00 2024-04-25T10:04:04.373 INFO:teuthology.orchestra.run.smithi132.stdout:ceph packages for x86_64 175 kB/s | 82 kB 00:00 2024-04-25T10:04:04.809 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 29 kB/s | 10 kB 00:00 2024-04-25T10:04:04.850 INFO:teuthology.orchestra.run.smithi132.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-04-25T10:04:05.360 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-04-25T10:04:05.383 INFO:teuthology.orchestra.run.smithi132.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-04-25T10:04:05.998 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-25T10:04:09.274 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 9.3 MB/s | 19 MB 00:02 2024-04-25T10:04:09.932 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.0 MB 00:04 2024-04-25T10:04:11.992 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-25T10:04:14.366 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 5.2 MB/s | 6.2 MB 00:01 2024-04-25T10:04:16.084 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 16 kB 00:00 2024-04-25T10:04:16.443 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T10:04:18.368 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-04-25T10:04:18.605 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 21 MB 00:02 2024-04-25T10:04:19.893 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T10:04:24.619 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-25T10:04:25.836 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-25T10:04:27.010 INFO:teuthology.orchestra.run.smithi132.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:04:27.011 INFO:teuthology.orchestra.run.smithi132.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:04:27.030 INFO:teuthology.orchestra.run.smithi132.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:04:27.031 INFO:teuthology.orchestra.run.smithi132.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:04:27.032 INFO:teuthology.orchestra.run.smithi132.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:04:27.033 INFO:teuthology.orchestra.run.smithi132.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:04:27.035 INFO:teuthology.orchestra.run.smithi132.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:04:27.039 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:04:27.040 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:04:27.111 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-04-25T10:04:27.132 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================= 2024-04-25T10:04:27.132 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-04-25T10:04:27.132 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================= 2024-04-25T10:04:27.132 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-04-25T10:04:27.132 INFO:teuthology.orchestra.run.smithi132.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:04:27.132 INFO:teuthology.orchestra.run.smithi132.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T10:04:27.132 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T10:04:27.132 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T10:04:27.132 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T10:04:27.132 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T10:04:27.132 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T10:04:27.133 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T10:04:27.133 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T10:04:27.133 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T10:04:27.133 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T10:04:27.133 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T10:04:27.133 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T10:04:27.133 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T10:04:27.133 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T10:04:27.133 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T10:04:27.133 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T10:04:27.133 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:04:27.133 INFO:teuthology.orchestra.run.smithi132.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:04:27.134 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:04:27.134 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T10:04:27.134 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T10:04:27.134 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T10:04:27.134 INFO:teuthology.orchestra.run.smithi132.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:04:27.134 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:04:27.134 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:04:27.134 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:04:27.134 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:04:27.134 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:04:27.134 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T10:04:27.135 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T10:04:27.135 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T10:04:27.135 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T10:04:27.135 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T10:04:27.135 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T10:04:27.135 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T10:04:27.135 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:04:27.135 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrading: 2024-04-25T10:04:27.135 INFO:teuthology.orchestra.run.smithi132.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T10:04:27.135 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T10:04:27.135 INFO:teuthology.orchestra.run.smithi132.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:04:27.135 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:04:27.135 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-04-25T10:04:27.136 INFO:teuthology.orchestra.run.smithi132.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:04:27.136 INFO:teuthology.orchestra.run.smithi132.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:04:27.136 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:04:27.136 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T10:04:27.136 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T10:04:27.136 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T10:04:27.136 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T10:04:27.136 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T10:04:27.136 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T10:04:27.136 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T10:04:27.136 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T10:04:27.137 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:04:27.137 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:04:27.137 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:04:27.137 INFO:teuthology.orchestra.run.smithi132.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:04:27.137 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:04:27.137 INFO:teuthology.orchestra.run.smithi132.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:04:27.137 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:04:27.137 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:04:27.137 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T10:04:27.137 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:04:27.137 INFO:teuthology.orchestra.run.smithi132.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:04:27.137 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:04:27.137 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:04:27.138 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:04:27.138 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T10:04:27.138 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:04:27.138 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T10:04:27.138 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:04:27.138 INFO:teuthology.orchestra.run.smithi132.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:04:27.138 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:04:27.138 INFO:teuthology.orchestra.run.smithi132.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:04:27.138 INFO:teuthology.orchestra.run.smithi132.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:04:27.138 INFO:teuthology.orchestra.run.smithi132.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:04:27.138 INFO:teuthology.orchestra.run.smithi132.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:04:27.138 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:04:27.139 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:04:27.139 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:04:27.139 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:04:27.139 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:04:27.139 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:04:27.139 INFO:teuthology.orchestra.run.smithi132.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:04:27.139 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:04:27.139 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:04:27.139 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:04:27.139 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T10:04:27.139 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T10:04:27.139 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:04:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:04:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:04:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:04:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:04:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:04:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:04:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:04:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:04:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:04:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:04:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:04:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:04:27.140 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:04:27.141 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:04:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:04:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:04:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:04:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:04:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:04:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:04:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:04:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:04:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:04:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:04:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:04:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:04:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:04:27.142 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:04:27.143 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:04:27.143 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:04:27.143 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:04:27.143 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:04:27.143 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:04:27.143 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:04:27.143 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:04:27.143 INFO:teuthology.orchestra.run.smithi132.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:04:27.143 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:04:27.143 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:04:27.143 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:04:27.143 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:04:27.143 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:04:27.143 INFO:teuthology.orchestra.run.smithi132.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:04:27.144 INFO:teuthology.orchestra.run.smithi132.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:04:27.144 INFO:teuthology.orchestra.run.smithi132.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:04:27.144 INFO:teuthology.orchestra.run.smithi132.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:04:27.144 INFO:teuthology.orchestra.run.smithi132.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:04:27.144 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:04:27.144 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:04:27.144 INFO:teuthology.orchestra.run.smithi132.stdout:Installing weak dependencies: 2024-04-25T10:04:27.144 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:04:27.144 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:04:27.144 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-04-25T10:04:27.144 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================= 2024-04-25T10:04:27.144 INFO:teuthology.orchestra.run.smithi132.stdout:Install 148 Packages 2024-04-25T10:04:27.144 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrade 4 Packages 2024-04-25T10:04:27.144 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:04:27.145 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 215 M 2024-04-25T10:04:27.145 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-04-25T10:04:27.954 INFO:teuthology.orchestra.run.smithi132.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-25T10:04:28.171 INFO:teuthology.orchestra.run.smithi132.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 4.0 MB/s | 887 kB 00:00 2024-04-25T10:04:28.203 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:04:28.205 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:04:28.224 INFO:teuthology.orchestra.run.smithi119.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:04:28.226 INFO:teuthology.orchestra.run.smithi119.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:04:28.227 INFO:teuthology.orchestra.run.smithi119.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:04:28.228 INFO:teuthology.orchestra.run.smithi119.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:04:28.230 INFO:teuthology.orchestra.run.smithi132.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 2.5 MB/s | 145 kB 00:00 2024-04-25T10:04:28.231 INFO:teuthology.orchestra.run.smithi119.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:04:28.234 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:04:28.235 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:04:28.305 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-25T10:04:28.320 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-04-25T10:04:28.320 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-04-25T10:04:28.320 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-04-25T10:04:28.320 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-25T10:04:28.320 INFO:teuthology.orchestra.run.smithi119.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:04:28.320 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T10:04:28.320 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T10:04:28.320 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T10:04:28.320 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T10:04:28.320 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T10:04:28.320 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T10:04:28.320 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T10:04:28.321 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T10:04:28.321 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T10:04:28.321 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T10:04:28.321 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T10:04:28.321 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T10:04:28.321 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T10:04:28.321 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T10:04:28.321 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T10:04:28.321 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T10:04:28.321 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:04:28.321 INFO:teuthology.orchestra.run.smithi119.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:04:28.321 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:04:28.321 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T10:04:28.322 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T10:04:28.322 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T10:04:28.322 INFO:teuthology.orchestra.run.smithi119.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:04:28.322 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:04:28.322 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:04:28.322 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:04:28.322 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:04:28.322 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:04:28.322 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T10:04:28.322 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T10:04:28.322 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T10:04:28.322 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T10:04:28.322 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T10:04:28.323 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T10:04:28.323 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T10:04:28.323 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:04:28.323 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2024-04-25T10:04:28.323 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T10:04:28.323 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T10:04:28.323 INFO:teuthology.orchestra.run.smithi119.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:04:28.323 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:04:28.323 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-04-25T10:04:28.323 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:04:28.323 INFO:teuthology.orchestra.run.smithi119.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:04:28.323 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:04:28.323 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:04:28.324 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:04:28.325 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T10:04:28.325 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:04:28.325 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:04:28.325 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:04:28.325 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:04:28.325 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:04:28.325 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T10:04:28.325 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:04:28.325 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T10:04:28.325 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:04:28.325 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:04:28.325 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:04:28.325 INFO:teuthology.orchestra.run.smithi119.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:04:28.325 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:04:28.326 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:04:28.326 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:04:28.326 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:04:28.326 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:04:28.326 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:04:28.326 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:04:28.326 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:04:28.326 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:04:28.326 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:04:28.326 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:04:28.326 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:04:28.326 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:04:28.327 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T10:04:28.327 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T10:04:28.327 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:04:28.327 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:04:28.327 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:04:28.327 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:04:28.327 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:04:28.327 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:04:28.327 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:04:28.327 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:04:28.327 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:04:28.327 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:04:28.328 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:04:28.328 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:04:28.328 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:04:28.328 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:04:28.328 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:04:28.328 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:04:28.328 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:04:28.328 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:04:28.328 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:04:28.328 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:04:28.328 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:04:28.328 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:04:28.328 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:04:28.328 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:04:28.329 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:04:28.329 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:04:28.329 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:04:28.329 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:04:28.329 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:04:28.329 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:04:28.329 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:04:28.329 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:04:28.329 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:04:28.329 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:04:28.329 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:04:28.330 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:04:28.330 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:04:28.330 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:04:28.330 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:04:28.330 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:04:28.330 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:04:28.330 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:04:28.330 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:04:28.330 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:04:28.330 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:04:28.330 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:04:28.330 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:04:28.330 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:04:28.330 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:04:28.331 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:04:28.331 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:04:28.331 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:04:28.331 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:04:28.331 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:04:28.331 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:04:28.331 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:04:28.331 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:04:28.331 INFO:teuthology.orchestra.run.smithi119.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:04:28.331 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:04:28.331 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:04:28.331 INFO:teuthology.orchestra.run.smithi119.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:04:28.331 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:04:28.331 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:04:28.332 INFO:teuthology.orchestra.run.smithi119.stdout:Installing weak dependencies: 2024-04-25T10:04:28.332 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:04:28.332 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:04:28.332 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-25T10:04:28.332 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-04-25T10:04:28.332 INFO:teuthology.orchestra.run.smithi119.stdout:Install 148 Packages 2024-04-25T10:04:28.332 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 4 Packages 2024-04-25T10:04:28.332 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:04:28.333 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 215 M 2024-04-25T10:04:28.333 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-25T10:04:28.538 INFO:teuthology.orchestra.run.smithi132.stdout:(4/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 6.9 MB/s | 2.1 MB 00:00 2024-04-25T10:04:28.688 INFO:teuthology.orchestra.run.smithi132.stdout:(5/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 5.4 MB/s | 5.3 MB 00:00 2024-04-25T10:04:28.772 INFO:teuthology.orchestra.run.smithi132.stdout:(6/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 6.4 MB/s | 1.5 MB 00:00 2024-04-25T10:04:29.634 INFO:teuthology.orchestra.run.smithi132.stdout:(7/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 5.0 MB/s | 4.7 MB 00:00 2024-04-25T10:04:29.855 INFO:teuthology.orchestra.run.smithi119.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-25T10:04:29.881 INFO:teuthology.orchestra.run.smithi132.stdout:(8/152): ceph-common-19.0.0-1628.g97ae6cc9.el9. 9.1 MB/s | 20 MB 00:02 2024-04-25T10:04:29.940 INFO:teuthology.orchestra.run.smithi132.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 441 kB/s | 25 kB 00:00 2024-04-25T10:04:30.563 INFO:teuthology.orchestra.run.smithi119.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.2 MB/s | 887 kB 00:00 2024-04-25T10:04:30.747 INFO:teuthology.orchestra.run.smithi119.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 798 kB/s | 145 kB 00:00 2024-04-25T10:04:31.022 INFO:teuthology.orchestra.run.smithi119.stdout:(4/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 3.8 MB/s | 5.3 MB 00:01 2024-04-25T10:04:31.073 INFO:teuthology.orchestra.run.smithi132.stdout:(10/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 7.3 MB/s | 17 MB 00:02 2024-04-25T10:04:31.265 INFO:teuthology.orchestra.run.smithi132.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 1.1 MB/s | 220 kB 00:00 2024-04-25T10:04:31.281 INFO:teuthology.orchestra.run.smithi119.stdout:(5/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 5.8 MB/s | 1.5 MB 00:00 2024-04-25T10:04:31.332 INFO:teuthology.orchestra.run.smithi132.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 414 kB/s | 27 kB 00:00 2024-04-25T10:04:31.491 INFO:teuthology.orchestra.run.smithi132.stdout:(13/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 5.3 MB/s | 9.8 MB 00:01 2024-04-25T10:04:31.516 INFO:teuthology.orchestra.run.smithi132.stdout:(14/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 171 kB/s | 31 kB 00:00 2024-04-25T10:04:31.700 INFO:teuthology.orchestra.run.smithi132.stdout:(15/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 3.5 MB/s | 744 kB 00:00 2024-04-25T10:04:31.800 INFO:teuthology.orchestra.run.smithi132.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.2 MB/s | 127 kB 00:00 2024-04-25T10:04:31.906 INFO:teuthology.orchestra.run.smithi119.stdout:(6/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 7.5 MB/s | 4.7 MB 00:00 2024-04-25T10:04:31.934 INFO:teuthology.orchestra.run.smithi132.stdout:(17/152): libradosstriper1-19.0.0-1628.g97ae6cc 3.7 MB/s | 501 kB 00:00 2024-04-25T10:04:32.092 INFO:teuthology.orchestra.run.smithi132.stdout:(18/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 284 kB/s | 163 kB 00:00 2024-04-25T10:04:32.184 INFO:teuthology.orchestra.run.smithi132.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 498 kB/s | 45 kB 00:00 2024-04-25T10:04:32.284 INFO:teuthology.orchestra.run.smithi132.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 1.3 MB/s | 131 kB 00:00 2024-04-25T10:04:32.289 INFO:teuthology.orchestra.run.smithi119.stdout:(7/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 1.4 MB/s | 2.1 MB 00:01 2024-04-25T10:04:32.376 INFO:teuthology.orchestra.run.smithi132.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 1.7 MB/s | 162 kB 00:00 2024-04-25T10:04:32.551 INFO:teuthology.orchestra.run.smithi132.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 1.8 MB/s | 321 kB 00:00 2024-04-25T10:04:32.727 INFO:teuthology.orchestra.run.smithi132.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 1.7 MB/s | 298 kB 00:00 2024-04-25T10:04:32.828 INFO:teuthology.orchestra.run.smithi132.stdout:(24/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.0 MB/s | 99 kB 00:00 2024-04-25T10:04:32.928 INFO:teuthology.orchestra.run.smithi132.stdout:(25/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 863 kB/s | 85 kB 00:00 2024-04-25T10:04:33.078 INFO:teuthology.orchestra.run.smithi132.stdout:(26/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 4.5 MB/s | 5.1 MB 00:01 2024-04-25T10:04:33.178 INFO:teuthology.orchestra.run.smithi132.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 1.7 MB/s | 170 kB 00:00 2024-04-25T10:04:33.279 INFO:teuthology.orchestra.run.smithi132.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 316 kB/s | 31 kB 00:00 2024-04-25T10:04:33.379 INFO:teuthology.orchestra.run.smithi132.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.4 MB/s | 138 kB 00:00 2024-04-25T10:04:33.640 INFO:teuthology.orchestra.run.smithi119.stdout:(8/152): ceph-common-19.0.0-1628.g97ae6cc9.el9. 4.9 MB/s | 20 MB 00:04 2024-04-25T10:04:33.723 INFO:teuthology.orchestra.run.smithi119.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 306 kB/s | 25 kB 00:00 2024-04-25T10:04:33.746 INFO:teuthology.orchestra.run.smithi132.stdout:(30/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 3.8 MB/s | 3.1 MB 00:00 2024-04-25T10:04:33.805 INFO:teuthology.orchestra.run.smithi132.stdout:(31/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 4.2 MB/s | 1.8 MB 00:00 2024-04-25T10:04:33.921 INFO:teuthology.orchestra.run.smithi132.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 2.1 MB/s | 248 kB 00:00 2024-04-25T10:04:34.022 INFO:teuthology.orchestra.run.smithi132.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 495 kB/s | 49 kB 00:00 2024-04-25T10:04:34.089 INFO:teuthology.orchestra.run.smithi132.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 233 kB/s | 15 kB 00:00 2024-04-25T10:04:34.180 INFO:teuthology.orchestra.run.smithi132.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 2.8 MB/s | 264 kB 00:00 2024-04-25T10:04:34.190 INFO:teuthology.orchestra.run.smithi119.stdout:(10/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 7.4 MB/s | 17 MB 00:02 2024-04-25T10:04:34.299 INFO:teuthology.orchestra.run.smithi119.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 2.0 MB/s | 220 kB 00:00 2024-04-25T10:04:34.339 INFO:teuthology.orchestra.run.smithi132.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 4.6 MB/s | 748 kB 00:00 2024-04-25T10:04:34.382 INFO:teuthology.orchestra.run.smithi119.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 327 kB/s | 27 kB 00:00 2024-04-25T10:04:34.423 INFO:teuthology.orchestra.run.smithi132.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 220 kB/s | 18 kB 00:00 2024-04-25T10:04:34.466 INFO:teuthology.orchestra.run.smithi119.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 378 kB/s | 31 kB 00:00 2024-04-25T10:04:34.582 INFO:teuthology.orchestra.run.smithi132.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 510 kB/s | 80 kB 00:00 2024-04-25T10:04:34.658 INFO:teuthology.orchestra.run.smithi119.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 3.8 MB/s | 744 kB 00:00 2024-04-25T10:04:34.758 INFO:teuthology.orchestra.run.smithi119.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 1.6 MB/s | 163 kB 00:00 2024-04-25T10:04:34.823 INFO:teuthology.orchestra.run.smithi132.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 803 kB 00:00 2024-04-25T10:04:34.841 INFO:teuthology.orchestra.run.smithi119.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.5 MB/s | 127 kB 00:00 2024-04-25T10:04:34.899 INFO:teuthology.orchestra.run.smithi132.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-04-25T10:04:34.950 INFO:teuthology.orchestra.run.smithi119.stdout:(17/152): libradosstriper1-19.0.0-1628.g97ae6cc 4.6 MB/s | 501 kB 00:00 2024-04-25T10:04:34.974 INFO:teuthology.orchestra.run.smithi132.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-04-25T10:04:35.241 INFO:teuthology.orchestra.run.smithi132.stdout:(42/152): ceph-mgr-diskprediction-local-19.0.0- 4.9 MB/s | 7.4 MB 00:01 2024-04-25T10:04:35.291 INFO:teuthology.orchestra.run.smithi132.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 3.9 MB/s | 1.2 MB 00:00 2024-04-25T10:04:35.509 INFO:teuthology.orchestra.run.smithi119.stdout:(18/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 9.2 MB/s | 5.1 MB 00:00 2024-04-25T10:04:35.584 INFO:teuthology.orchestra.run.smithi119.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 611 kB/s | 45 kB 00:00 2024-04-25T10:04:35.659 INFO:teuthology.orchestra.run.smithi119.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 1.7 MB/s | 131 kB 00:00 2024-04-25T10:04:35.743 INFO:teuthology.orchestra.run.smithi119.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 1.9 MB/s | 162 kB 00:00 2024-04-25T10:04:35.767 INFO:teuthology.orchestra.run.smithi132.stdout:(44/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 8.1 MB/s | 47 MB 00:05 2024-04-25T10:04:35.800 INFO:teuthology.orchestra.run.smithi132.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 266 kB/s | 135 kB 00:00 2024-04-25T10:04:35.835 INFO:teuthology.orchestra.run.smithi119.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 3.4 MB/s | 321 kB 00:00 2024-04-25T10:04:35.851 INFO:teuthology.orchestra.run.smithi132.stdout:(46/152): python3-ply-3.11-14.el9.noarch.rpm 175 kB/s | 106 kB 00:00 2024-04-25T10:04:35.901 INFO:teuthology.orchestra.run.smithi132.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-25T10:04:35.935 INFO:teuthology.orchestra.run.smithi132.stdout:(48/152): python3-requests-2.25.1-8.el9.noarch. 751 kB/s | 125 kB 00:00 2024-04-25T10:04:35.935 INFO:teuthology.orchestra.run.smithi119.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 3.0 MB/s | 298 kB 00:00 2024-04-25T10:04:36.018 INFO:teuthology.orchestra.run.smithi132.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-04-25T10:04:36.027 INFO:teuthology.orchestra.run.smithi119.stdout:(24/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.1 MB/s | 99 kB 00:00 2024-04-25T10:04:36.044 INFO:teuthology.orchestra.run.smithi132.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 989 kB/s | 107 kB 00:00 2024-04-25T10:04:36.086 INFO:teuthology.orchestra.run.smithi132.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 325 kB/s | 21 kB 00:00 2024-04-25T10:04:36.119 INFO:teuthology.orchestra.run.smithi119.stdout:(25/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 937 kB/s | 85 kB 00:00 2024-04-25T10:04:36.127 INFO:teuthology.orchestra.run.smithi132.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-04-25T10:04:36.153 INFO:teuthology.orchestra.run.smithi132.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 494 kB/s | 33 kB 00:00 2024-04-25T10:04:36.186 INFO:teuthology.orchestra.run.smithi132.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-04-25T10:04:36.212 INFO:teuthology.orchestra.run.smithi132.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 315 kB/s | 18 kB 00:00 2024-04-25T10:04:36.254 INFO:teuthology.orchestra.run.smithi132.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 168 kB/s | 11 kB 00:00 2024-04-25T10:04:36.287 INFO:teuthology.orchestra.run.smithi132.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-25T10:04:36.313 INFO:teuthology.orchestra.run.smithi132.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 785 kB/s | 45 kB 00:00 2024-04-25T10:04:36.388 INFO:teuthology.orchestra.run.smithi132.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-04-25T10:04:36.447 INFO:teuthology.orchestra.run.smithi132.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 443 kB/s | 25 kB 00:00 2024-04-25T10:04:36.547 INFO:teuthology.orchestra.run.smithi132.stdout:(61/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.1 MB/s | 3.0 MB 00:00 2024-04-25T10:04:36.562 INFO:teuthology.orchestra.run.smithi119.stdout:(26/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 7.0 MB/s | 3.1 MB 00:00 2024-04-25T10:04:36.572 INFO:teuthology.orchestra.run.smithi132.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 381 kB/s | 47 kB 00:00 2024-04-25T10:04:36.605 INFO:teuthology.orchestra.run.smithi132.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 1.9 MB/s | 582 kB 00:00 2024-04-25T10:04:36.637 INFO:teuthology.orchestra.run.smithi119.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 2.3 MB/s | 170 kB 00:00 2024-04-25T10:04:36.639 INFO:teuthology.orchestra.run.smithi132.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-25T10:04:36.665 INFO:teuthology.orchestra.run.smithi132.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 611 kB/s | 35 kB 00:00 2024-04-25T10:04:36.703 INFO:teuthology.orchestra.run.smithi119.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 478 kB/s | 31 kB 00:00 2024-04-25T10:04:36.706 INFO:teuthology.orchestra.run.smithi132.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2024-04-25T10:04:36.788 INFO:teuthology.orchestra.run.smithi119.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.6 MB/s | 138 kB 00:00 2024-04-25T10:04:36.932 INFO:teuthology.orchestra.run.smithi132.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 7.1 MB/s | 1.6 MB 00:00 2024-04-25T10:04:37.013 INFO:teuthology.orchestra.run.smithi119.stdout:(30/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 7.9 MB/s | 1.8 MB 00:00 2024-04-25T10:04:37.032 INFO:teuthology.orchestra.run.smithi132.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 9.0 MB/s | 911 kB 00:00 2024-04-25T10:04:37.157 INFO:teuthology.orchestra.run.smithi132.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 9.2 MB/s | 4.7 MB 00:00 2024-04-25T10:04:37.191 INFO:teuthology.orchestra.run.smithi132.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 845 kB/s | 133 kB 00:00 2024-04-25T10:04:37.288 INFO:teuthology.orchestra.run.smithi119.stdout:(31/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 2.0 MB/s | 9.8 MB 00:04 2024-04-25T10:04:37.341 INFO:teuthology.orchestra.run.smithi132.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-04-25T10:04:37.397 INFO:teuthology.orchestra.run.smithi119.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 2.3 MB/s | 248 kB 00:00 2024-04-25T10:04:37.400 INFO:teuthology.orchestra.run.smithi132.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T10:04:37.455 INFO:teuthology.orchestra.run.smithi119.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 861 kB/s | 49 kB 00:00 2024-04-25T10:04:37.522 INFO:teuthology.orchestra.run.smithi119.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 233 kB/s | 15 kB 00:00 2024-04-25T10:04:37.525 INFO:teuthology.orchestra.run.smithi132.stdout:(73/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-04-25T10:04:37.631 INFO:teuthology.orchestra.run.smithi119.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 2.4 MB/s | 264 kB 00:00 2024-04-25T10:04:37.667 INFO:teuthology.orchestra.run.smithi132.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-04-25T10:04:37.692 INFO:teuthology.orchestra.run.smithi132.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-04-25T10:04:37.726 INFO:teuthology.orchestra.run.smithi132.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-25T10:04:37.760 INFO:teuthology.orchestra.run.smithi132.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 386 kB/s | 26 kB 00:00 2024-04-25T10:04:37.779 INFO:teuthology.orchestra.run.smithi132.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 371 kB/s | 19 kB 00:00 2024-04-25T10:04:37.781 INFO:teuthology.orchestra.run.smithi119.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 9.6 MB/s | 7.4 MB 00:00 2024-04-25T10:04:37.821 INFO:teuthology.orchestra.run.smithi132.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 172 kB 00:00 2024-04-25T10:04:37.840 INFO:teuthology.orchestra.run.smithi119.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 318 kB/s | 18 kB 00:00 2024-04-25T10:04:37.846 INFO:teuthology.orchestra.run.smithi132.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 528 kB/s | 35 kB 00:00 2024-04-25T10:04:37.905 INFO:teuthology.orchestra.run.smithi132.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-25T10:04:37.915 INFO:teuthology.orchestra.run.smithi119.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 80 kB 00:00 2024-04-25T10:04:37.949 INFO:teuthology.orchestra.run.smithi119.stdout:(39/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 2.3 MB/s | 748 kB 00:00 2024-04-25T10:04:38.024 INFO:teuthology.orchestra.run.smithi119.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-04-25T10:04:38.042 INFO:teuthology.orchestra.run.smithi132.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 9.3 MB/s | 6.0 MB 00:00 2024-04-25T10:04:38.066 INFO:teuthology.orchestra.run.smithi119.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.2 MB/s | 803 kB 00:00 2024-04-25T10:04:38.080 INFO:teuthology.orchestra.run.smithi132.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-04-25T10:04:38.105 INFO:teuthology.orchestra.run.smithi132.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 77 kB 00:00 2024-04-25T10:04:38.139 INFO:teuthology.orchestra.run.smithi132.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 732 kB/s | 42 kB 00:00 2024-04-25T10:04:38.173 INFO:teuthology.orchestra.run.smithi132.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-04-25T10:04:38.206 INFO:teuthology.orchestra.run.smithi132.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 159 kB 00:00 2024-04-25T10:04:38.240 INFO:teuthology.orchestra.run.smithi132.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-25T10:04:38.266 INFO:teuthology.orchestra.run.smithi132.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 869 kB/s | 51 kB 00:00 2024-04-25T10:04:38.291 INFO:teuthology.orchestra.run.smithi132.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T10:04:38.341 INFO:teuthology.orchestra.run.smithi132.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 854 kB/s | 42 kB 00:00 2024-04-25T10:04:38.392 INFO:teuthology.orchestra.run.smithi132.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 898 kB/s | 44 kB 00:00 2024-04-25T10:04:38.501 INFO:teuthology.orchestra.run.smithi132.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.5 MB/s | 5.1 MB 00:00 2024-04-25T10:04:38.534 INFO:teuthology.orchestra.run.smithi132.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-04-25T10:04:38.550 INFO:teuthology.orchestra.run.smithi119.stdout:(42/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 9.8 MB/s | 47 MB 00:04 2024-04-25T10:04:38.559 INFO:teuthology.orchestra.run.smithi132.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 622 kB/s | 36 kB 00:00 2024-04-25T10:04:38.583 INFO:teuthology.orchestra.run.smithi119.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 454 kB/s | 253 kB 00:00 2024-04-25T10:04:38.584 INFO:teuthology.orchestra.run.smithi132.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 955 kB/s | 47 kB 00:00 2024-04-25T10:04:38.618 INFO:teuthology.orchestra.run.smithi132.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-25T10:04:38.642 INFO:teuthology.orchestra.run.smithi119.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-25T10:04:38.647 INFO:teuthology.orchestra.run.smithi132.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-25T10:04:38.667 INFO:teuthology.orchestra.run.smithi119.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-25T10:04:38.677 INFO:teuthology.orchestra.run.smithi132.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T10:04:38.702 INFO:teuthology.orchestra.run.smithi132.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T10:04:38.726 INFO:teuthology.orchestra.run.smithi119.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 1.9 MB/s | 1.2 MB 00:00 2024-04-25T10:04:38.727 INFO:teuthology.orchestra.run.smithi132.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-04-25T10:04:38.753 INFO:teuthology.orchestra.run.smithi132.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 450 kB/s | 22 kB 00:00 2024-04-25T10:04:38.776 INFO:teuthology.orchestra.run.smithi119.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 938 kB/s | 125 kB 00:00 2024-04-25T10:04:38.810 INFO:teuthology.orchestra.run.smithi119.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-04-25T10:04:38.820 INFO:teuthology.orchestra.run.smithi132.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 9.2 MB/s | 863 kB 00:00 2024-04-25T10:04:38.985 INFO:teuthology.orchestra.run.smithi119.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 613 kB/s | 107 kB 00:00 2024-04-25T10:04:39.035 INFO:teuthology.orchestra.run.smithi119.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 438 kB/s | 21 kB 00:00 2024-04-25T10:04:39.094 INFO:teuthology.orchestra.run.smithi119.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-04-25T10:04:39.161 INFO:teuthology.orchestra.run.smithi119.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 499 kB/s | 33 kB 00:00 2024-04-25T10:04:39.194 INFO:teuthology.orchestra.run.smithi119.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 313 kB 00:00 2024-04-25T10:04:39.254 INFO:teuthology.orchestra.run.smithi132.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-04-25T10:04:39.335 INFO:teuthology.orchestra.run.smithi119.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 131 kB/s | 18 kB 00:00 2024-04-25T10:04:39.338 INFO:teuthology.orchestra.run.smithi132.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 598 kB/s | 309 kB 00:00 2024-04-25T10:04:39.387 INFO:teuthology.orchestra.run.smithi119.stdout:(55/152): bison-3.7.4-5.el9.x86_64.rpm 1.5 MB/s | 943 kB 00:00 2024-04-25T10:04:39.414 INFO:teuthology.orchestra.run.smithi119.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 143 kB/s | 11 kB 00:00 2024-04-25T10:04:39.473 INFO:teuthology.orchestra.run.smithi119.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 788 kB/s | 45 kB 00:00 2024-04-25T10:04:39.498 INFO:teuthology.orchestra.run.smithi119.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-25T10:04:39.557 INFO:teuthology.orchestra.run.smithi119.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-25T10:04:39.607 INFO:teuthology.orchestra.run.smithi119.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 518 kB/s | 25 kB 00:00 2024-04-25T10:04:39.638 INFO:teuthology.orchestra.run.smithi132.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 621 kB/s | 548 kB 00:00 2024-04-25T10:04:39.657 INFO:teuthology.orchestra.run.smithi119.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 965 kB/s | 47 kB 00:00 2024-04-25T10:04:39.716 INFO:teuthology.orchestra.run.smithi119.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.0 MB/s | 292 kB 00:00 2024-04-25T10:04:39.774 INFO:teuthology.orchestra.run.smithi119.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-25T10:04:39.816 INFO:teuthology.orchestra.run.smithi119.stdout:(64/152): libtool-2.4.6-45.el9.x86_64.rpm 1.8 MB/s | 582 kB 00:00 2024-04-25T10:04:39.833 INFO:teuthology.orchestra.run.smithi119.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 35 kB 00:00 2024-04-25T10:04:39.958 INFO:teuthology.orchestra.run.smithi119.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-04-25T10:04:40.038 INFO:teuthology.orchestra.run.smithi132.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 65 kB/s | 26 kB 00:00 2024-04-25T10:04:40.072 INFO:teuthology.orchestra.run.smithi132.stdout:(108/152): grpc-data-1.46.7-10.el9.noarch.rpm 24 kB/s | 19 kB 00:00 2024-04-25T10:04:40.164 INFO:teuthology.orchestra.run.smithi132.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 390 kB/s | 49 kB 00:00 2024-04-25T10:04:40.231 INFO:teuthology.orchestra.run.smithi132.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 426 kB/s | 67 kB 00:00 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi119.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T10:04:40.297 INFO:teuthology.orchestra.run.smithi132.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-04-25T10:04:40.300 INFO:teuthology.orchestra.run.smithi119.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 4.7 MB/s | 1.6 MB 00:00 2024-04-25T10:04:40.368 INFO:teuthology.orchestra.run.smithi119.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 7.2 MB/s | 911 kB 00:00 2024-04-25T10:04:40.393 INFO:teuthology.orchestra.run.smithi119.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-25T10:04:40.510 INFO:teuthology.orchestra.run.smithi119.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-04-25T10:04:40.568 INFO:teuthology.orchestra.run.smithi119.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T10:04:40.660 INFO:teuthology.orchestra.run.smithi119.stdout:(73/152): openblas-openmp-0.3.21-2.el9.x86_64.r 5.6 MB/s | 4.7 MB 00:00 2024-04-25T10:04:40.777 INFO:teuthology.orchestra.run.smithi119.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-04-25T10:04:40.811 INFO:teuthology.orchestra.run.smithi119.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.6 MB/s | 244 kB 00:00 2024-04-25T10:04:40.844 INFO:teuthology.orchestra.run.smithi119.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-04-25T10:04:40.870 INFO:teuthology.orchestra.run.smithi119.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 444 kB/s | 26 kB 00:00 2024-04-25T10:04:40.903 INFO:teuthology.orchestra.run.smithi119.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 331 kB/s | 19 kB 00:00 2024-04-25T10:04:40.937 INFO:teuthology.orchestra.run.smithi119.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-25T10:04:40.962 INFO:teuthology.orchestra.run.smithi119.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 599 kB/s | 35 kB 00:00 2024-04-25T10:04:41.021 INFO:teuthology.orchestra.run.smithi119.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-04-25T10:04:41.079 INFO:teuthology.orchestra.run.smithi119.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.8 MB/s | 222 kB 00:00 2024-04-25T10:04:41.196 INFO:teuthology.orchestra.run.smithi119.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 9.5 MB/s | 6.0 MB 00:00 2024-04-25T10:04:41.222 INFO:teuthology.orchestra.run.smithi119.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 547 kB/s | 77 kB 00:00 2024-04-25T10:04:41.222 INFO:teuthology.orchestra.run.smithi132.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 846 kB/s | 838 kB 00:00 2024-04-25T10:04:41.247 INFO:teuthology.orchestra.run.smithi119.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-04-25T10:04:41.256 INFO:teuthology.orchestra.run.smithi132.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 572 kB/s | 548 kB 00:00 2024-04-25T10:04:41.281 INFO:teuthology.orchestra.run.smithi119.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T10:04:41.323 INFO:teuthology.orchestra.run.smithi119.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T10:04:41.323 INFO:teuthology.orchestra.run.smithi132.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 487 kB/s | 32 kB 00:00 2024-04-25T10:04:41.356 INFO:teuthology.orchestra.run.smithi119.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-04-25T10:04:41.382 INFO:teuthology.orchestra.run.smithi119.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 884 kB/s | 51 kB 00:00 2024-04-25T10:04:41.407 INFO:teuthology.orchestra.run.smithi119.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-25T10:04:41.440 INFO:teuthology.orchestra.run.smithi132.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2024-04-25T10:04:41.465 INFO:teuthology.orchestra.run.smithi132.stdout:(116/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 180 kB/s | 43 kB 00:00 2024-04-25T10:04:41.466 INFO:teuthology.orchestra.run.smithi119.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 732 kB/s | 42 kB 00:00 2024-04-25T10:04:41.516 INFO:teuthology.orchestra.run.smithi119.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 897 kB/s | 44 kB 00:00 2024-04-25T10:04:41.566 INFO:teuthology.orchestra.run.smithi132.stdout:(117/152): libarrow-9.0.0-10.el9.x86_64.rpm 2.0 MB/s | 4.4 MB 00:02 2024-04-25T10:04:41.599 INFO:teuthology.orchestra.run.smithi132.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-04-25T10:04:41.624 INFO:teuthology.orchestra.run.smithi119.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.4 MB/s | 5.1 MB 00:00 2024-04-25T10:04:41.633 INFO:teuthology.orchestra.run.smithi132.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-04-25T10:04:41.658 INFO:teuthology.orchestra.run.smithi119.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-04-25T10:04:41.683 INFO:teuthology.orchestra.run.smithi119.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 622 kB/s | 36 kB 00:00 2024-04-25T10:04:41.709 INFO:teuthology.orchestra.run.smithi119.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 956 kB/s | 47 kB 00:00 2024-04-25T10:04:41.758 INFO:teuthology.orchestra.run.smithi132.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 223 kB/s | 35 kB 00:00 2024-04-25T10:04:41.759 INFO:teuthology.orchestra.run.smithi119.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T10:04:41.792 INFO:teuthology.orchestra.run.smithi119.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.8 MB/s | 305 kB 00:00 2024-04-25T10:04:41.818 INFO:teuthology.orchestra.run.smithi119.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T10:04:41.843 INFO:teuthology.orchestra.run.smithi119.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-25T10:04:41.868 INFO:teuthology.orchestra.run.smithi119.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-25T10:04:41.894 INFO:teuthology.orchestra.run.smithi119.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 451 kB/s | 22 kB 00:00 2024-04-25T10:04:42.036 INFO:teuthology.orchestra.run.smithi119.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-04-25T10:04:42.361 INFO:teuthology.orchestra.run.smithi119.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-25T10:04:42.394 INFO:teuthology.orchestra.run.smithi119.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-25T10:04:42.420 INFO:teuthology.orchestra.run.smithi119.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 810 kB/s | 309 kB 00:00 2024-04-25T10:04:42.437 INFO:teuthology.orchestra.run.smithi119.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 279 kB/s | 19 kB 00:00 2024-04-25T10:04:42.637 INFO:teuthology.orchestra.run.smithi119.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 120 kB/s | 26 kB 00:00 2024-04-25T10:04:42.654 INFO:teuthology.orchestra.run.smithi119.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 224 kB/s | 49 kB 00:00 2024-04-25T10:04:42.795 INFO:teuthology.orchestra.run.smithi119.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 517 kB/s | 67 kB 00:00 2024-04-25T10:04:42.888 INFO:teuthology.orchestra.run.smithi119.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 9.5 MB/s | 4.4 MB 00:00 2024-04-25T10:04:42.913 INFO:teuthology.orchestra.run.smithi119.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 610 kB/s | 151 kB 00:00 2024-04-25T10:04:42.980 INFO:teuthology.orchestra.run.smithi119.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 838 kB 00:00 2024-04-25T10:04:43.064 INFO:teuthology.orchestra.run.smithi119.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-04-25T10:04:43.081 INFO:teuthology.orchestra.run.smithi119.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 274 kB/s | 43 kB 00:00 2024-04-25T10:04:43.098 INFO:teuthology.orchestra.run.smithi119.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2024-04-25T10:04:43.115 INFO:teuthology.orchestra.run.smithi119.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 303 kB/s | 15 kB 00:00 2024-04-25T10:04:43.182 INFO:teuthology.orchestra.run.smithi119.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T10:04:43.215 INFO:teuthology.orchestra.run.smithi119.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-25T10:04:43.249 INFO:teuthology.orchestra.run.smithi119.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-25T10:04:43.274 INFO:teuthology.orchestra.run.smithi119.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 400 kB/s | 35 kB 00:00 2024-04-25T10:04:43.308 INFO:teuthology.orchestra.run.smithi119.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.3 MB/s | 206 kB 00:00 2024-04-25T10:04:43.450 INFO:teuthology.orchestra.run.smithi119.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-04-25T10:04:43.475 INFO:teuthology.orchestra.run.smithi119.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 730 kB/s | 144 kB 00:00 2024-04-25T10:04:43.500 INFO:teuthology.orchestra.run.smithi119.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-04-25T10:04:43.563 INFO:teuthology.orchestra.run.smithi119.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 165 kB/s | 18 kB 00:00 2024-04-25T10:04:43.592 INFO:teuthology.orchestra.run.smithi119.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 199 kB/s | 23 kB 00:00 2024-04-25T10:04:43.618 INFO:teuthology.orchestra.run.smithi119.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2024-04-25T10:04:43.651 INFO:teuthology.orchestra.run.smithi119.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-25T10:04:44.010 INFO:teuthology.orchestra.run.smithi119.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 2.5 MB/s | 1.0 MB 00:00 2024-04-25T10:04:44.035 INFO:teuthology.orchestra.run.smithi119.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 111 kB/s | 46 kB 00:00 2024-04-25T10:04:44.060 INFO:teuthology.orchestra.run.smithi119.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 193 kB/s | 79 kB 00:00 2024-04-25T10:04:44.427 INFO:teuthology.orchestra.run.smithi119.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 139 kB/s | 58 kB 00:00 2024-04-25T10:04:44.478 INFO:teuthology.orchestra.run.smithi119.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 654 kB/s | 272 kB 00:00 2024-04-25T10:04:44.503 INFO:teuthology.orchestra.run.smithi119.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 37 kB/s | 16 kB 00:00 2024-04-25T10:04:44.608 INFO:teuthology.orchestra.run.smithi132.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 69 kB/s | 206 kB 00:02 2024-04-25T10:04:44.642 INFO:teuthology.orchestra.run.smithi132.stdout:(122/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 99 kB/s | 303 kB 00:03 2024-04-25T10:04:44.653 INFO:teuthology.orchestra.run.smithi119.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 400 kB/s | 90 kB 00:00 2024-04-25T10:04:44.678 INFO:teuthology.orchestra.run.smithi119.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 141 kB/s | 28 kB 00:00 2024-04-25T10:04:44.704 INFO:teuthology.orchestra.run.smithi119.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 154 kB/s | 31 kB 00:00 2024-04-25T10:04:44.754 INFO:teuthology.orchestra.run.smithi119.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.9 MB/s | 188 kB 00:00 2024-04-25T10:04:44.767 INFO:teuthology.orchestra.run.smithi132.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 694 kB/s | 2.0 MB 00:03 2024-04-25T10:04:44.779 INFO:teuthology.orchestra.run.smithi119.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 591 kB/s | 59 kB 00:00 2024-04-25T10:04:44.804 INFO:teuthology.orchestra.run.smithi119.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 358 kB/s | 36 kB 00:00 2024-04-25T10:04:44.830 INFO:teuthology.orchestra.run.smithi119.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 690 kB/s | 51 kB 00:00 2024-04-25T10:04:44.834 INFO:teuthology.orchestra.run.smithi132.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 643 kB/s | 144 kB 00:00 2024-04-25T10:04:44.859 INFO:teuthology.orchestra.run.smithi132.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 49 kB/s | 11 kB 00:00 2024-04-25T10:04:44.880 INFO:teuthology.orchestra.run.smithi119.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T10:04:44.914 INFO:teuthology.orchestra.run.smithi119.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 835 kB/s | 90 kB 00:00 2024-04-25T10:04:44.934 INFO:teuthology.orchestra.run.smithi132.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-04-25T10:04:44.960 INFO:teuthology.orchestra.run.smithi132.stdout:(127/152): python3-jaraco-classes-3.2.1-5.el9.n 92 kB/s | 18 kB 00:00 2024-04-25T10:04:44.968 INFO:teuthology.orchestra.run.smithi119.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.0 MB/s | 427 kB 00:00 2024-04-25T10:04:44.985 INFO:teuthology.orchestra.run.smithi132.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 155 kB/s | 19 kB 00:00 2024-04-25T10:04:44.990 INFO:teuthology.orchestra.run.smithi119.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 193 kB/s | 20 kB 00:00 2024-04-25T10:04:45.010 INFO:teuthology.orchestra.run.smithi132.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 263 kB/s | 20 kB 00:00 2024-04-25T10:04:45.035 INFO:teuthology.orchestra.run.smithi132.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 931 kB/s | 46 kB 00:00 2024-04-25T10:04:45.064 INFO:teuthology.orchestra.run.smithi119.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-25T10:04:45.311 INFO:teuthology.orchestra.run.smithi132.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 263 kB/s | 79 kB 00:00 2024-04-25T10:04:45.336 INFO:teuthology.orchestra.run.smithi132.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 192 kB/s | 58 kB 00:00 2024-04-25T10:04:45.406 INFO:teuthology.orchestra.run.smithi119.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-04-25T10:04:45.503 INFO:teuthology.orchestra.run.smithi132.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-04-25T10:04:45.528 INFO:teuthology.orchestra.run.smithi132.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 86 kB/s | 16 kB 00:00 2024-04-25T10:04:45.540 INFO:teuthology.orchestra.run.smithi119.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 10 MB/s | 1.3 MB 00:00 2024-04-25T10:04:45.554 INFO:teuthology.orchestra.run.smithi132.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-25T10:04:45.596 INFO:teuthology.orchestra.run.smithi132.stdout:(136/152): python3-kubernetes-26.1.0-2.el9.noar 1.6 MB/s | 1.0 MB 00:00 2024-04-25T10:04:45.699 INFO:teuthology.orchestra.run.smithi119.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-04-25T10:04:45.763 INFO:teuthology.orchestra.run.smithi132.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 121 kB/s | 28 kB 00:00 2024-04-25T10:04:45.788 INFO:teuthology.orchestra.run.smithi132.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 132 kB/s | 31 kB 00:00 2024-04-25T10:04:45.822 INFO:teuthology.orchestra.run.smithi132.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 836 kB/s | 188 kB 00:00 2024-04-25T10:04:45.840 INFO:teuthology.orchestra.run.smithi119.stdout:(151/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 4.0 MB/s | 3.1 MB 00:00 2024-04-25T10:04:46.447 INFO:teuthology.orchestra.run.smithi132.stdout:(140/152): python3-typing-extensions-4.4.0-2.el 82 kB/s | 51 kB 00:00 2024-04-25T10:04:46.474 INFO:teuthology.orchestra.run.smithi119.stdout:(152/152): librados2-19.0.0-1628.g97ae6cc9.el9. 2.3 MB/s | 3.4 MB 00:01 2024-04-25T10:04:46.482 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:04:46.482 INFO:teuthology.orchestra.run.smithi119.stdout:Total 12 MB/s | 215 MB 00:18 2024-04-25T10:04:46.489 INFO:teuthology.orchestra.run.smithi132.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 81 kB/s | 59 kB 00:00 2024-04-25T10:04:46.506 INFO:teuthology.orchestra.run.smithi132.stdout:(142/152): python3-tempora-5.0.0-2.el9.noarch.r 50 kB/s | 36 kB 00:00 2024-04-25T10:04:46.540 INFO:teuthology.orchestra.run.smithi132.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-25T10:04:46.632 INFO:teuthology.orchestra.run.smithi132.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 219 kB/s | 20 kB 00:00 2024-04-25T10:04:46.657 INFO:teuthology.orchestra.run.smithi132.stdout:(145/152): python3-websocket-client-1.2.3-2.el9 537 kB/s | 90 kB 00:00 2024-04-25T10:04:46.765 INFO:teuthology.orchestra.run.smithi132.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-25T10:04:47.157 INFO:teuthology.orchestra.run.smithi132.stdout:(147/152): librados2-19.0.0-1628.g97ae6cc9.el9. 8.6 MB/s | 3.4 MB 00:00 2024-04-25T10:04:47.291 INFO:teuthology.orchestra.run.smithi132.stdout:(148/152): python3-werkzeug-2.0.3-3.el9.1.noarc 545 kB/s | 427 kB 00:00 2024-04-25T10:04:47.475 INFO:teuthology.orchestra.run.smithi132.stdout:(149/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 9.9 MB/s | 3.1 MB 00:00 2024-04-25T10:04:47.608 INFO:teuthology.orchestra.run.smithi132.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 4.3 MB/s | 1.3 MB 00:00 2024-04-25T10:04:47.675 INFO:teuthology.orchestra.run.smithi132.stdout:(151/152): thrift-0.15.0-2.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:01 2024-04-25T10:04:47.825 INFO:teuthology.orchestra.run.smithi132.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.8 MB/s | 2.4 MB 00:00 2024-04-25T10:04:47.832 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:04:47.832 INFO:teuthology.orchestra.run.smithi132.stdout:Total 10 MB/s | 215 MB 00:20 2024-04-25T10:04:49.356 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-25T10:04:49.593 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-25T10:04:49.593 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-25T10:04:50.650 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-04-25T10:04:50.891 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-04-25T10:04:50.891 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-04-25T10:04:52.259 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-25T10:04:52.260 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-25T10:04:53.477 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-04-25T10:04:53.478 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-04-25T10:04:58.733 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-25T10:04:58.841 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:04:58.934 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:04:59.058 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:04:59.119 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:04:59.157 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:04:59.280 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:04:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:04:59.963 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-04-25T10:05:00.105 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:05:00.116 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:05:00.225 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:05:00.291 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:05:00.321 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:05:00.339 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:05:00.374 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:05:00.408 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:05:00.470 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:05:00.502 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:05:00.521 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T10:05:00.878 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:05:00.943 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:05:00.946 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:05:00.978 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:05:01.073 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:05:01.115 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:05:01.144 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T10:05:01.228 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T10:05:01.253 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:05:01.254 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:05:01.299 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:05:01.325 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:05:01.413 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:05:01.457 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T10:05:01.634 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:05:01.696 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:05:01.792 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:05:01.862 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:05:01.895 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:05:01.915 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:05:02.019 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:05:02.071 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:05:02.075 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:05:02.104 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T10:05:02.122 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:05:02.192 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T10:05:02.222 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:05:02.227 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:05:02.283 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:05:02.301 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:05:02.348 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:05:02.399 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:05:02.437 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:05:02.554 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:05:02.600 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:05:02.632 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:05:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:05:02.671 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:05:02.729 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:05:02.835 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:05:02.854 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:05:02.874 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T10:05:02.929 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T10:05:02.964 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T10:05:02.976 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:05:03.023 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:05:03.071 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:05:03.083 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:05:03.144 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:05:03.174 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:05:03.217 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:05:03.278 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:05:03.279 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:05:03.323 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:05:03.354 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:05:03.466 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:05:03.467 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:05:03.532 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:05:03.538 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:05:03.564 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:05:03.616 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:05:03.753 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:05:03.793 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T10:05:03.846 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T10:05:03.883 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T10:05:04.000 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:05:04.063 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:05:04.170 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:05:04.250 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:05:04.353 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:05:04.360 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:05:04.414 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:05:04.462 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:05:04.511 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:05:04.560 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T10:05:04.634 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:05:04.916 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:05:04.982 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:05:05.060 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:05:05.122 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:05:05.153 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:05:05.198 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:05:05.220 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:05:05.254 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:05:05.264 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:05:05.311 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:05:05.323 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:05:05.371 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:05:05.382 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T10:05:05.431 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:05:05.445 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:05:05.490 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:05:05.684 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:05:05.702 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:05:05.743 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:05:05.775 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:05:05.780 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:05:05.837 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:05:05.843 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:05:05.927 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:05:05.965 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:05:05.966 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:05:06.005 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:05:06.006 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:05:06.070 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:05:06.080 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:05:06.115 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:05:06.139 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:05:06.172 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:05:06.175 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:05:06.227 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:05:06.381 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:05:06.446 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:05:06.496 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:05:06.533 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:05:06.558 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:05:06.589 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:05:06.619 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:05:06.667 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:05:06.715 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:05:06.718 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:05:06.767 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:05:06.782 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:05:06.813 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:05:06.838 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:05:06.857 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:05:06.881 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:05:06.890 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:05:06.927 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:05:06.975 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:05:07.050 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:05:07.080 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:05:07.099 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:05:07.225 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:05:07.266 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:05:07.295 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:05:07.313 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:05:07.339 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:05:07.352 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:05:07.398 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:05:07.415 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:05:07.457 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:05:07.509 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:05:07.566 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:05:07.568 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:05:07.612 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:05:07.637 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:05:07.662 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:05:07.738 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:05:07.786 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:05:07.925 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:05:07.990 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:05:08.034 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:05:08.105 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:05:08.250 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:05:08.314 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:05:08.613 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:05:08.669 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:05:08.702 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:05:08.826 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:05:08.889 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:05:08.985 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:05:09.037 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:05:09.254 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:05:09.269 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:05:09.306 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:05:09.309 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:05:09.339 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:05:09.418 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:05:09.462 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:05:09.533 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:05:09.629 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:05:09.674 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:05:09.903 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:05:09.937 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:05:10.042 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:05:10.250 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:05:10.399 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:05:10.847 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:05:10.994 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:05:12.256 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:05:12.366 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:05:12.406 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:05:12.451 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:05:12.550 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:05:12.687 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:05:12.731 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:05:12.785 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:05:12.844 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:05:12.878 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T10:05:13.000 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T10:05:13.035 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:05:13.169 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:05:13.207 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T10:05:13.329 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T10:05:13.365 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T10:05:13.425 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:05:13.607 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:05:13.639 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T10:05:13.730 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T10:05:13.775 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T10:05:13.920 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:05:13.952 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T10:05:14.052 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T10:05:14.079 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T10:05:16.311 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:05:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:05:16.664 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:05:16.701 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:05:17.017 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:05:17.067 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T10:05:17.220 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T10:05:17.220 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:05:17.221 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:05:17.221 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:05:17.276 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T10:05:17.356 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:05:17.405 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T10:05:17.644 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T10:05:17.644 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:05:17.644 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:05:17.644 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:05:17.689 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T10:05:30.524 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T10:05:30.525 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2024-04-25T10:05:30.525 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2024-04-25T10:05:30.525 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2024-04-25T10:05:30.525 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2024-04-25T10:05:30.525 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2024-04-25T10:05:30.525 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /root 2024-04-25T10:05:30.525 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2024-04-25T10:05:30.525 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:05:30.747 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:05:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:05:30.796 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:30.796 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:05:30.796 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:05:30.831 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T10:05:30.831 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /sys 2024-04-25T10:05:30.832 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /proc 2024-04-25T10:05:30.832 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /mnt 2024-04-25T10:05:30.832 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /var/tmp 2024-04-25T10:05:30.832 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /home 2024-04-25T10:05:30.832 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /root 2024-04-25T10:05:30.832 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /tmp 2024-04-25T10:05:30.832 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:05:31.056 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:05:31.105 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:05:31.105 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:31.105 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:05:31.105 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:05:32.405 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:05:32.481 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:05:32.481 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:32.482 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:05:32.482 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:05:32.482 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:05:32.482 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:05:32.698 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:05:32.714 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:05:32.774 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:05:32.775 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:32.775 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:05:32.775 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:05:32.775 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:05:32.775 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:05:32.780 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:05:32.780 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:32.781 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:05:32.781 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:05:32.781 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:05:32.781 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:05:33.024 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:05:33.100 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:05:33.100 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:33.100 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:05:33.100 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:05:33.100 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:05:33.100 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:05:33.281 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:05:33.349 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:05:33.350 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:33.350 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:05:33.350 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:05:33.350 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:05:33.350 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:05:33.491 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:05:33.553 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:05:33.608 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:05:33.661 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:05:33.689 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:05:33.689 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:33.689 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:05:33.689 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:05:33.689 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:05:33.689 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:05:33.834 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:05:33.898 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:05:33.910 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:05:33.970 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:05:33.972 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:05:34.055 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:05:34.156 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:05:34.201 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:05:34.254 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:05:34.332 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:05:34.364 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:05:34.386 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:05:34.425 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:05:34.451 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:05:34.465 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:05:34.628 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:05:34.683 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:05:34.719 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:05:34.761 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:05:35.072 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:05:35.104 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:05:35.141 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T10:05:35.245 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T10:05:35.281 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:05:35.384 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:05:35.414 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:05:35.456 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T10:05:35.548 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T10:05:35.583 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:05:35.606 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:05:35.640 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:05:35.905 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:05:35.943 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:05:36.778 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:05:36.815 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:05:36.961 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:05:37.053 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:05:37.085 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:05:37.151 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T10:05:37.185 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:05:37.228 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:05:37.278 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:05:37.279 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:37.279 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:05:37.279 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:05:37.279 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:05:37.279 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:05:37.335 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:05:37.376 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:05:37.426 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T10:05:37.454 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:05:37.509 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T10:05:37.529 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:05:37.529 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:37.529 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:05:37.529 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:05:37.530 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:05:37.530 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:05:37.576 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T10:05:37.588 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:05:37.627 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:05:37.777 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T10:05:37.857 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T10:05:38.545 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:05:38.588 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:05:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:05:38.656 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:38.656 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:05:38.656 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:05:38.656 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:05:38.656 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:05:38.706 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:05:38.773 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:05:38.773 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:38.773 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:05:38.773 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:05:38.838 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:05:38.839 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:05:38.874 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:05:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:05:38.911 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:38.911 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:05:38.911 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:05:38.911 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:05:38.911 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:05:38.965 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:05:38.965 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:38.965 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:05:38.965 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:05:38.965 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:05:38.965 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:05:39.016 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:05:39.110 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:05:39.110 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:39.110 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:05:39.110 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:05:39.183 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:05:39.287 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:05:39.287 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:39.287 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:05:39.288 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:05:39.288 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:05:39.288 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:05:39.315 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:05:39.384 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:05:39.385 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:05:39.676 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:05:39.761 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:05:39.761 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:05:39.761 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:05:39.761 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:05:39.761 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:05:39.761 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:05:44.212 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T10:05:44.264 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T10:05:44.339 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T10:05:44.381 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:05:44.429 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:05:44.581 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T10:05:44.625 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T10:05:44.698 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T10:05:44.741 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:05:44.757 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T10:05:44.789 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:05:44.808 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:05:44.837 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T10:05:45.103 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T10:05:45.182 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:05:45.214 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T10:05:45.721 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:05:45.770 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:05:45.821 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:05:46.098 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:05:46.146 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:05:46.205 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:05:47.165 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T10:05:47.280 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:05:47.360 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:05:47.525 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:05:47.531 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T10:05:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:05:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:05:47.645 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:05:47.703 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:05:47.724 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:05:47.742 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:05:47.743 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:05:47.879 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:05:47.912 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:05:47.960 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:05:47.960 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:05:48.007 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:05:48.095 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:05:48.142 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:05:48.142 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:05:48.282 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:05:48.298 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:05:48.299 INFO:teuthology.orchestra.run.smithi119.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T10:05:48.299 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:05:48.413 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:05:48.676 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:05:48.676 INFO:teuthology.orchestra.run.smithi132.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T10:05:48.676 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:05:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T10:05:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T10:05:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T10:05:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T10:05:53.578 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T10:05:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T10:05:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T10:05:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:05:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:05:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T10:05:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T10:05:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T10:05:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:05:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T10:05:53.579 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T10:05:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T10:05:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T10:05:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T10:05:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T10:05:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T10:05:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T10:05:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T10:05:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T10:05:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T10:05:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T10:05:53.580 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T10:05:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T10:05:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T10:05:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T10:05:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T10:05:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T10:05:53.581 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T10:05:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T10:05:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T10:05:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T10:05:53.595 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T10:05:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T10:05:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T10:05:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T10:05:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:05:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:05:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:05:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:05:53.596 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:05:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:05:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:05:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:05:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:05:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:05:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:05:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:05:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:05:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:05:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:05:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:05:53.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:05:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:05:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:05:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:05:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:05:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:05:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:05:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:05:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:05:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:05:53.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:05:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:05:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:05:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:05:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:05:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:05:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:05:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:05:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:05:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:05:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:05:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:05:53.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:05:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:05:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:05:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:05:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:05:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:05:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:05:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:05:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:05:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:05:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:05:53.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:05:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:05:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:05:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:05:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:05:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:05:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:05:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:05:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:05:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:05:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:05:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:05:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:05:53.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:05:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:05:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:05:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:05:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:05:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:05:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:05:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:05:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:05:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:05:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:05:53.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:05:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:05:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:05:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:05:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:05:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:05:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:05:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:05:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:05:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:05:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:05:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:05:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:05:53.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:05:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:05:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:05:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:05:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:05:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:05:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:05:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:05:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:05:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:05:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:05:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:05:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:05:53.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:05:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:05:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:05:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:05:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:05:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:05:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:05:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:05:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:05:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:05:53.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T10:05:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:05:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T10:05:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:05:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:05:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:05:53.606 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:05:53.887 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T10:05:53.887 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T10:05:53.887 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T10:05:53.887 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T10:05:53.887 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T10:05:53.887 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T10:05:53.887 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T10:05:53.887 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:05:53.887 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:05:53.887 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T10:05:53.887 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T10:05:53.888 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T10:05:53.888 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:05:53.888 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T10:05:53.888 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T10:05:53.888 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T10:05:53.888 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T10:05:53.888 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T10:05:53.888 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T10:05:53.888 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T10:05:53.888 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T10:05:53.888 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T10:05:53.888 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T10:05:53.889 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T10:05:53.889 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T10:05:53.889 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T10:05:53.889 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T10:05:53.901 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T10:05:53.901 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T10:05:53.902 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T10:05:53.902 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T10:05:53.902 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T10:05:53.902 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T10:05:53.902 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T10:05:53.902 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T10:05:53.902 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T10:05:53.902 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T10:05:53.902 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T10:05:53.902 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T10:05:53.902 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:05:53.902 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:05:53.902 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:05:53.903 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:05:53.903 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:05:53.903 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:05:53.903 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:05:53.903 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:05:53.903 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:05:53.903 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:05:53.903 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:05:53.903 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:05:53.903 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:05:53.903 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:05:53.903 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:05:53.903 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:05:53.904 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:05:53.904 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:05:53.904 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:05:53.904 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:05:53.904 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:05:53.904 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:05:53.904 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:05:53.904 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:05:53.904 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:05:53.904 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:05:53.904 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:05:53.904 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:05:53.904 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:05:53.905 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:05:53.905 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:05:53.905 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:05:53.905 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:05:53.905 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:05:53.905 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:05:53.905 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:05:53.905 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:05:53.905 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:05:53.905 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:05:53.905 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:05:53.905 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:05:53.905 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:05:53.905 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:05:53.906 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:05:53.906 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:05:53.906 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:05:53.906 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:05:53.906 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:05:53.906 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:05:53.906 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:05:53.906 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:05:53.906 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:05:53.906 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:05:53.906 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:05:53.906 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:05:53.906 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:05:53.907 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:05:53.907 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:05:53.907 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:05:53.907 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:05:53.907 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:05:53.907 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:05:53.907 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:05:53.907 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:05:53.907 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:05:53.907 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:05:53.907 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:05:53.907 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:05:53.908 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:05:53.908 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:05:53.908 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:05:53.908 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:05:53.908 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:05:53.908 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:05:53.908 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:05:53.908 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:05:53.908 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:05:53.908 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:05:53.908 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:05:53.908 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:05:53.908 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:05:53.909 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:05:53.909 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:05:53.909 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:05:53.909 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:05:53.909 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:05:53.909 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:05:53.909 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:05:53.909 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:05:53.909 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:05:53.909 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:05:53.909 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:05:53.909 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:05:53.910 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:05:53.910 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:05:53.910 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:05:53.910 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:05:53.910 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:05:53.910 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:05:53.910 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:05:53.910 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:05:53.910 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:05:53.910 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:05:53.910 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:05:53.910 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:05:53.911 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:05:53.911 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:05:53.911 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:05:53.911 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:05:53.911 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T10:05:53.911 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:05:53.911 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T10:05:53.911 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:05:53.911 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:05:53.911 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:05:53.911 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:05:55.982 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:05:55.982 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:05:55.982 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2024-04-25T10:05:55.982 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.982 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.982 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:05:55.982 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:05:55.982 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-25T10:05:55.983 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:05:55.983 INFO:teuthology.orchestra.run.smithi119.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:05:55.983 INFO:teuthology.orchestra.run.smithi119.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:05:55.983 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:05:55.983 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.983 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.983 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.983 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.983 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:55.983 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.983 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.983 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.983 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:55.983 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:55.984 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:55.984 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:55.984 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:55.984 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.984 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.984 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:55.984 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.984 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.984 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.984 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:55.984 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:55.984 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.985 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.985 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:55.985 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:05:55.985 INFO:teuthology.orchestra.run.smithi119.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:05:55.985 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:05:55.985 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:05:55.985 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:05:55.985 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:05:55.985 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:05:55.985 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:05:55.985 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:05:55.985 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:05:55.985 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:05:55.986 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.986 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.986 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.986 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:05:55.986 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:05:55.986 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:05:55.986 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:05:55.986 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:05:55.986 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.986 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.986 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:05:55.986 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.986 INFO:teuthology.orchestra.run.smithi119.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:05:55.987 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:05:55.987 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:05:55.987 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:05:55.987 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:05:55.987 INFO:teuthology.orchestra.run.smithi119.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:05:55.987 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:05:55.987 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:05:55.987 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:05:55.988 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:05:55.988 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:05:55.988 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:05:55.988 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:05:55.988 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:05:55.988 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:05:55.988 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:05:55.988 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:05:55.988 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:05:55.988 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:05:55.989 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:05:55.989 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:05:55.989 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:05:55.989 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:05:55.989 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.989 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.989 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.989 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:05:55.989 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:05:55.989 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:05:55.989 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:05:55.989 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:05:55.989 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:05:55.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:05:55.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:05:55.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:05:55.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:05:55.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:05:55.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:05:55.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:05:55.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:05:55.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:05:55.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:05:55.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:05:55.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:05:55.990 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:05:55.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:05:55.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:05:55.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:05:55.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:05:55.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:05:55.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:05:55.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:05:55.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:05:55.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:05:55.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:05:55.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:05:55.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:05:55.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:05:55.991 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:05:55.992 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:05:55.992 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:05:55.992 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:05:55.992 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:05:55.992 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:05:55.992 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:05:55.992 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:05:55.992 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.992 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.992 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:05:55.992 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:05:55.992 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:05:55.992 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.993 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:05:55.993 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:05:55.993 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:05:55.993 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:05:55.993 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:05:55.993 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:05:55.993 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:05:55.993 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:05:55.993 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:05:55.993 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:05:55.994 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:05:55.994 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:05:55.994 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:05:55.994 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:05:55.994 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:05:55.994 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.994 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.994 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:55.994 INFO:teuthology.orchestra.run.smithi119.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:05:55.994 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:05:55.994 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:05:55.994 INFO:teuthology.orchestra.run.smithi119.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:05:55.994 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:05:55.995 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:05:55.995 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:05:55.995 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:05:55.995 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-25T10:05:56.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:05:56.419 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:05:56.419 INFO:teuthology.orchestra.run.smithi132.stdout:Upgraded: 2024-04-25T10:05:56.419 INFO:teuthology.orchestra.run.smithi132.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.419 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.419 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:05:56.419 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:05:56.419 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-04-25T10:05:56.419 INFO:teuthology.orchestra.run.smithi132.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:05:56.419 INFO:teuthology.orchestra.run.smithi132.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:05:56.419 INFO:teuthology.orchestra.run.smithi132.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:05:56.419 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:05:56.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:56.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:56.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:56.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:56.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:56.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:56.421 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.421 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.421 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:56.421 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.421 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.421 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.421 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:56.421 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:56.421 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.421 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.421 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:05:56.421 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:05:56.421 INFO:teuthology.orchestra.run.smithi132.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:05:56.422 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:05:56.422 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:05:56.422 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:05:56.422 INFO:teuthology.orchestra.run.smithi132.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:05:56.422 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:05:56.422 INFO:teuthology.orchestra.run.smithi132.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:05:56.422 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:05:56.423 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:05:56.423 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:05:56.423 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.423 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.423 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.423 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:05:56.423 INFO:teuthology.orchestra.run.smithi132.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:05:56.423 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:05:56.423 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:05:56.423 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:05:56.423 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.425 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.425 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:05:56.425 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.425 INFO:teuthology.orchestra.run.smithi132.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:05:56.425 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:05:56.425 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:05:56.425 INFO:teuthology.orchestra.run.smithi132.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:05:56.425 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:05:56.426 INFO:teuthology.orchestra.run.smithi132.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:05:56.426 INFO:teuthology.orchestra.run.smithi132.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:05:56.426 INFO:teuthology.orchestra.run.smithi132.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:05:56.426 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:05:56.426 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:05:56.426 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:05:56.426 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:05:56.426 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:05:56.426 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:05:56.426 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:05:56.426 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:05:56.426 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:05:56.426 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:05:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:05:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:05:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:05:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:05:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:05:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:05:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:05:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:05:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:05:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:05:56.427 INFO:teuthology.orchestra.run.smithi132.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:05:56.428 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:05:56.428 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:05:56.428 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:05:56.428 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:05:56.428 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:05:56.428 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:05:56.428 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:05:56.428 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:05:56.428 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:05:56.428 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:05:56.428 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:05:56.428 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:05:56.428 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:05:56.429 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:05:56.429 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:05:56.429 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:05:56.429 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:05:56.429 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:05:56.429 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:05:56.429 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:05:56.429 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:05:56.429 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:05:56.429 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:05:56.430 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:05:56.430 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:05:56.430 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:05:56.430 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:05:56.430 INFO:teuthology.orchestra.run.smithi132.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:05:56.430 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:05:56.430 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:05:56.430 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:05:56.430 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:05:56.430 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:05:56.430 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:05:56.430 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.430 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.431 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:05:56.431 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:05:56.431 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:05:56.431 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.431 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:05:56.431 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:05:56.431 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:05:56.431 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:05:56.431 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:05:56.431 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:05:56.431 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:05:56.431 INFO:teuthology.orchestra.run.smithi132.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:05:56.431 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:05:56.431 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:05:56.432 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:05:56.432 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:05:56.432 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:05:56.432 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:05:56.432 INFO:teuthology.orchestra.run.smithi132.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:05:56.432 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.432 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.432 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:05:56.432 INFO:teuthology.orchestra.run.smithi132.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:05:56.432 INFO:teuthology.orchestra.run.smithi132.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:05:56.432 INFO:teuthology.orchestra.run.smithi132.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:05:56.432 INFO:teuthology.orchestra.run.smithi132.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:05:56.432 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:05:56.433 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:05:56.433 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:05:56.433 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:05:56.434 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-04-25T10:05:56.604 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:57.000 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:57.001 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:05:57.172 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:05:57.227 INFO:teuthology.orchestra.run.smithi119.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:05:57.228 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:05:57.228 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T10:05:57.230 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:05:57.382 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:05:57.438 INFO:teuthology.orchestra.run.smithi132.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:05:57.439 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:05:57.439 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T10:05:57.441 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T10:05:57.441 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:05:57.441 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:05:57.513 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:05:57.513 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:05:57.582 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T10:05:57.582 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:05:57.582 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:05:57.650 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:05:57.755 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:05:57.755 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:05:57.838 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:05:57.948 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T10:05:57.948 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:05:57.948 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:05:58.019 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:05:58.130 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:05:58.130 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:05:58.202 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:05:58.308 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T10:05:58.308 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:05:58.308 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:05:58.386 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:05:58.492 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:05:58.492 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:05:58.567 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:05:58.685 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T10:05:58.793 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T10:05:58.794 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /var/log/ceph 2024-04-25T10:05:58.799 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 777 /var/log/ceph 2024-04-25T10:05:58.870 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T10:05:58.871 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:05:58.875 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:05:58.937 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T10:05:58.937 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:05:58.941 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:05:59.010 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T10:05:59.011 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T10:05:59.011 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T10:05:59.011 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:05:59.011 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:05:59.050 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:59.236 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:59.296 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:05:59.296 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:05:59.342 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:59.525 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:59.595 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T10:05:59.595 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T10:05:59.595 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7673048', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1124, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi119.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIVju1AAd0gE/+C4CJtuXZwjKdnXue5BP/hJ477CXirK3URoeIOvWXXyYI9gK3V9TJoD16idtOV6jlg7lEEpetQ=', 'smithi132.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAVgsCkyRVusmNSE6EqqVSz6QgjmdNFwA6Zk7noeSi57ejVm+rctGYzRk7JRJ2swgrM67UiylRqY/ZBm1K6VRtM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_09:36:50', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T10:05:59.596 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:05:59.601 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:05:59.643 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:05:59.684 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:05:59.754 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:05:59.755 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:05:59.792 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:05:59.792 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-04-25T10:05:59.869 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:05:59.869 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:05:59.869 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-25T10:05:59.869 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:05:59.870 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:05:59.870 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-04-25 10:05:48.800516416 +0000 2024-04-25T10:05:59.870 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-04-25 10:03:32.965175070 +0000 2024-04-25T10:05:59.870 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-04-25 10:03:32.965175070 +0000 2024-04-25T10:05:59.870 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-04-25 10:03:32.965175070 +0000 2024-04-25T10:05:59.870 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:05:59.976 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-04-25T10:05:59.976 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-04-25T10:05:59.977 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000377387 s, 1.4 MB/s 2024-04-25T10:05:59.979 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:06:00.019 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-04-25T10:06:00.099 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:06:00.099 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:06:00.099 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-25T10:06:00.101 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:06:00.101 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:06:00.101 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-04-25 10:05:48.801516392 +0000 2024-04-25T10:06:00.101 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-04-25 10:03:33.586159608 +0000 2024-04-25T10:06:00.101 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-04-25 10:03:33.586159608 +0000 2024-04-25T10:06:00.101 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-04-25 10:03:33.586159608 +0000 2024-04-25T10:06:00.101 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:06:00.200 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-04-25T10:06:00.201 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-04-25T10:06:00.201 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000315901 s, 1.6 MB/s 2024-04-25T10:06:00.203 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:06:00.241 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-04-25T10:06:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:06:00.323 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:06:00.323 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-25T10:06:00.324 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:06:00.324 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:06:00.324 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-04-25 10:05:48.801516392 +0000 2024-04-25T10:06:00.324 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-04-25 10:03:34.181144793 +0000 2024-04-25T10:06:00.324 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-04-25 10:03:34.181144793 +0000 2024-04-25T10:06:00.324 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-04-25 10:03:34.181144793 +0000 2024-04-25T10:06:00.324 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:06:00.432 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-04-25T10:06:00.432 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-04-25T10:06:00.432 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000338798 s, 1.5 MB/s 2024-04-25T10:06:00.436 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:06:00.483 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-04-25T10:06:00.525 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:06:00.525 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:06:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-25T10:06:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:06:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:06:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-04-25 10:05:48.802516368 +0000 2024-04-25T10:06:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-04-25 10:03:34.793129555 +0000 2024-04-25T10:06:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-04-25 10:03:34.793129555 +0000 2024-04-25T10:06:00.525 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-04-25 10:03:34.793129555 +0000 2024-04-25T10:06:00.526 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:06:00.629 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-04-25T10:06:00.629 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-04-25T10:06:00.629 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000521165 s, 982 kB/s 2024-04-25T10:06:00.632 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:06:00.671 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:06:00.671 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:00.671 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:06:00.718 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:06:00.718 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_1 2024-04-25T10:06:00.799 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:06:00.799 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:06:00.799 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-25T10:06:00.799 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:06:00.799 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:06:00.800 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-04-25 10:05:49.127247196 +0000 2024-04-25T10:06:00.800 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-04-25 10:03:32.971887021 +0000 2024-04-25T10:06:00.800 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-04-25 10:03:32.971887021 +0000 2024-04-25T10:06:00.800 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-04-25 10:03:32.971887021 +0000 2024-04-25T10:06:00.800 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:06:00.901 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-04-25T10:06:00.901 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-04-25T10:06:00.902 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000438591 s, 1.2 MB/s 2024-04-25T10:06:00.904 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:06:00.946 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_2 2024-04-25T10:06:01.030 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:06:01.031 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:06:01.031 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-25T10:06:01.031 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:06:01.031 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:06:01.031 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-04-25 10:05:49.128247170 +0000 2024-04-25T10:06:01.031 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-04-25 10:03:33.561871825 +0000 2024-04-25T10:06:01.031 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-04-25 10:03:33.561871825 +0000 2024-04-25T10:06:01.031 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-04-25 10:03:33.561871825 +0000 2024-04-25T10:06:01.031 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:06:01.136 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-04-25T10:06:01.137 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-04-25T10:06:01.137 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000347761 s, 1.5 MB/s 2024-04-25T10:06:01.139 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:06:01.182 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_3 2024-04-25T10:06:01.262 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:06:01.262 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:06:01.262 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-25T10:06:01.262 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:06:01.263 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:06:01.263 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-04-25 10:05:49.128247170 +0000 2024-04-25T10:06:01.263 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-04-25 10:03:34.180855881 +0000 2024-04-25T10:06:01.263 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-04-25 10:03:34.180855881 +0000 2024-04-25T10:06:01.263 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-04-25 10:03:34.180855881 +0000 2024-04-25T10:06:01.263 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:06:01.362 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-04-25T10:06:01.362 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-04-25T10:06:01.362 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000291959 s, 1.8 MB/s 2024-04-25T10:06:01.365 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:06:01.409 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_4 2024-04-25T10:06:01.489 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:06:01.489 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:06:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-25T10:06:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:06:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:06:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-04-25 10:05:49.129247145 +0000 2024-04-25T10:06:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-04-25 10:03:34.762840891 +0000 2024-04-25T10:06:01.490 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-04-25 10:03:34.762840891 +0000 2024-04-25T10:06:01.490 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-04-25 10:03:34.762840891 +0000 2024-04-25T10:06:01.490 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:06:01.590 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-04-25T10:06:01.590 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-04-25T10:06:01.590 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.00026714 s, 1.9 MB/s 2024-04-25T10:06:01.594 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:06:01.637 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:06:01.637 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi132.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:06:01.637 INFO:tasks.ceph:Generating config... 2024-04-25T10:06:01.638 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T10:06:01.639 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T10:06:01.639 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T10:06:01.639 INFO:tasks.ceph:[client] pid file = 2024-04-25T10:06:01.639 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T10:06:01.639 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T10:06:01.639 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T10:06:01.639 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T10:06:01.639 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T10:06:01.639 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T10:06:01.639 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T10:06:01.639 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T10:06:01.639 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T10:06:01.639 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T10:06:01.640 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T10:06:01.640 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T10:06:01.640 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T10:06:01.640 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T10:06:01.640 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T10:06:01.640 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T10:06:01.640 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T10:06:01.640 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T10:06:01.640 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T10:06:01.640 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T10:06:01.640 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T10:06:01.640 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T10:06:01.640 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T10:06:01.641 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T10:06:01.641 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T10:06:01.641 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T10:06:01.641 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T10:06:01.641 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T10:06:01.641 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T10:06:01.641 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T10:06:01.641 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T10:06:01.641 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T10:06:01.641 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T10:06:01.641 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T10:06:01.641 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T10:06:01.642 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T10:06:01.642 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T10:06:01.758 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T10:06:01.765 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T10:06:01.854 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:06:01.912 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.119'), ('mon.b', '172.21.15.132'), ('mon.c', '[v2:172.21.15.132:3301,v1:172.21.15.132:6790]')] 2024-04-25T10:06:01.913 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.119,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T10:06:01.914 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:01.914 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:06:01.951 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.119 --add b 172.21.15.132 --addv c '[v2:172.21.15.132:3301,v1:172.21.15.132:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:06:02.094 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:06:02.095 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: generated fsid e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:06:02.095 INFO:teuthology.orchestra.run.smithi119.stdout:setting min_mon_release = quincy 2024-04-25T10:06:02.095 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 0 2024-04-25T10:06:02.095 INFO:teuthology.orchestra.run.smithi119.stdout:fsid e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:06:02.095 INFO:teuthology.orchestra.run.smithi119.stdout:last_changed 2024-04-25T10:06:02.090427+0000 2024-04-25T10:06:02.095 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-04-25T10:06:02.090427+0000 2024-04-25T10:06:02.095 INFO:teuthology.orchestra.run.smithi119.stdout:min_mon_release 17 (quincy) 2024-04-25T10:06:02.095 INFO:teuthology.orchestra.run.smithi119.stdout:election_strategy: 1 2024-04-25T10:06:02.095 INFO:teuthology.orchestra.run.smithi119.stdout:0: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.a 2024-04-25T10:06:02.095 INFO:teuthology.orchestra.run.smithi119.stdout:1: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.b 2024-04-25T10:06:02.095 INFO:teuthology.orchestra.run.smithi119.stdout:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-04-25T10:06:02.095 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T10:06:02.100 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:06:02.140 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e0b0f428-b441-4237-a4d1-ed53321f87a5... 2024-04-25T10:06:02.141 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:06:02.185 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:06:02.287 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-04-25T10:06:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2024-04-25T10:06:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:06:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2024-04-25T10:06:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2024-04-25T10:06:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2024-04-25T10:06:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.288 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2024-04-25T10:06:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2024-04-25T10:06:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2024-04-25T10:06:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2024-04-25T10:06:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:06:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2024-04-25T10:06:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:06:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:06:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2024-04-25T10:06:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2024-04-25T10:06:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2024-04-25T10:06:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:06:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:06:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2024-04-25T10:06:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:06:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2024-04-25T10:06:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:06:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:06:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2024-04-25T10:06:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:06:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.290 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.119,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on skipped message = False 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2024-04-25T10:06:02.291 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2024-04-25T10:06:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:06:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2024-04-25T10:06:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2024-04-25T10:06:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2024-04-25T10:06:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2024-04-25T10:06:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2024-04-25T10:06:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2024-04-25T10:06:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2024-04-25T10:06:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2024-04-25T10:06:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.292 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:06:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2024-04-25T10:06:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2024-04-25T10:06:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:06:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2024-04-25T10:06:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2024-04-25T10:06:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2024-04-25T10:06:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: bdev async discard = True 2024-04-25T10:06:02.293 INFO:teuthology.orchestra.run.smithi119.stdout: bdev enable discard = True 2024-04-25T10:06:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore allocator = bitmap 2024-04-25T10:06:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2024-04-25T10:06:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2024-04-25T10:06:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 1/20 2024-04-25T10:06:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 1/20 2024-04-25T10:06:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:06:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2024-04-25T10:06:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 4/10 2024-04-25T10:06:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:06:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2024-04-25T10:06:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:06:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:06:02.294 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2024-04-25T10:06:02.295 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:06:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-04-25T10:06:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2024-04-25T10:06:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-04-25T10:06:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2024-04-25T10:06:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2024-04-25T10:06:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:06:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2024-04-25T10:06:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2024-04-25T10:06:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:06:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:06:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:06:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:06:02.296 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: mon op complaint time = 120 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2024-04-25T10:06:02.297 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: client mount timeout = 600 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: debug client = 20 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = "" 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2024-04-25T10:06:02.298 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi119.stdout:[mds] 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds = 20 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds balancer = 20 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug frag = True 2024-04-25T10:06:02.299 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug scatterstat = True 2024-04-25T10:06:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: mds op complaint time = 180 2024-04-25T10:06:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: mds verify scatter = True 2024-04-25T10:06:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2024-04-25T10:06:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2024-04-25T10:06:02.300 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2024-04-25T10:06:02.301 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:06:02.302 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.303 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.119,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2024-04-25T10:06:02.304 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2024-04-25T10:06:02.305 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.305 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2024-04-25T10:06:02.305 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2024-04-25T10:06:02.305 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:06:02.305 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.305 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2024-04-25T10:06:02.305 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2024-04-25T10:06:02.305 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.305 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2024-04-25T10:06:02.305 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2024-04-25T10:06:02.305 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2024-04-25T10:06:02.305 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug trim objects = true 2024-04-25T10:06:02.305 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.305 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2024-04-25T10:06:02.306 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2024-04-25T10:06:02.306 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.306 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:06:02.306 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.306 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2024-04-25T10:06:02.306 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.306 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2024-04-25T10:06:02.306 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.306 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:06:02.306 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2024-04-25T10:06:02.306 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.306 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2024-04-25T10:06:02.306 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi132.stdout: bdev async discard = True 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi132.stdout: bdev enable discard = True 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore allocator = bitmap 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 1/20 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 1/20 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 4/10 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:06:02.307 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:06:02.308 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2024-04-25T10:06:02.308 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2024-04-25T10:06:02.308 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.308 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2024-04-25T10:06:02.308 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:06:02.308 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2024-04-25T10:06:02.308 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-04-25T10:06:02.308 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-04-25T10:06:02.308 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:06:02.308 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:06:02.308 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2024-04-25T10:06:02.308 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.308 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2024-04-25T10:06:02.308 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.309 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:06:02.310 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:06:02.310 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:06:02.310 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.310 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2024-04-25T10:06:02.310 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:06:02.310 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.310 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:06:02.310 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2024-04-25T10:06:02.310 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:06:02.310 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2024-04-25T10:06:02.311 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2024-04-25T10:06:02.311 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2024-04-25T10:06:02.311 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2024-04-25T10:06:02.311 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:06:02.311 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:06:02.311 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2024-04-25T10:06:02.311 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2024-04-25T10:06:02.311 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:06:02.311 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2024-04-25T10:06:02.311 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2024-04-25T10:06:02.311 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2024-04-25T10:06:02.312 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2024-04-25T10:06:02.312 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2024-04-25T10:06:02.312 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2024-04-25T10:06:02.312 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2024-04-25T10:06:02.312 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2024-04-25T10:06:02.312 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2024-04-25T10:06:02.312 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2024-04-25T10:06:02.312 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2024-04-25T10:06:02.312 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2024-04-25T10:06:02.312 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2024-04-25T10:06:02.312 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:06:02.312 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2024-04-25T10:06:02.312 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = True 2024-04-25T10:06:02.312 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2024-04-25T10:06:02.313 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = True 2024-04-25T10:06:02.313 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2024-04-25T10:06:02.313 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2024-04-25T10:06:02.313 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2024-04-25T10:06:02.333 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T10:06:02.333 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T10:06:02.439 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T10:06:02.439 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:02.439 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T10:06:02.478 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:02.478 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T10:06:02.558 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:06:02.558 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:02.558 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:06:02.559 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:06:02.697 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:02.697 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:06:02.782 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi132.front.sepia.ceph.com 2024-04-25T10:06:02.782 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:02.782 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:06:02.782 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:06:02.874 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:02.874 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:06:02.951 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T10:06:02.951 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T10:06:02.951 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:06:03.082 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:06:03.090 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:06:03.214 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:06:03.221 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T10:06:03.222 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:06:03.356 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:06:03.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T10:06:03.436 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:06:03.577 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:06:03.585 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T10:06:03.657 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:06:03.791 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:06:03.797 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T10:06:03.860 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:06:03.968 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:06:03.974 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T10:06:04.034 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T10:06:04.034 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T10:06:04.151 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T10:06:04.190 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T10:06:04.293 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T10:06:04.346 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T10:06:04.453 INFO:teuthology.orchestra.run.smithi132.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T10:06:04.485 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T10:06:04.591 INFO:teuthology.orchestra.run.smithi132.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T10:06:04.631 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T10:06:04.631 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi132.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:06:04.632 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T10:06:04.720 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:06:04.720 INFO:tasks.ceph:role: osd.0 2024-04-25T10:06:04.720 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:06:04.721 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:06:04.803 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:06:04.803 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:06:04.803 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:06:04.803 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:06:04.803 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:06:04.803 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:06:04.803 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:06:04.803 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:06:04.804 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:06:04.804 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:06:05.217 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-04-25T10:06:05.222 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-04-25T10:06:05.222 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T10:06:05.363 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T10:06:05.441 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T10:06:05.561 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:06:05.561 INFO:tasks.ceph:role: osd.1 2024-04-25T10:06:05.561 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:06:05.561 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:06:05.635 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:06:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:06:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:06:05.635 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:06:05.635 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:06:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:06:05.636 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:06:05.636 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:06:05.636 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:06:05.636 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:06:05.992 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-04-25T10:06:05.998 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-04-25T10:06:05.998 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T10:06:06.106 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T10:06:06.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T10:06:06.255 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:06:06.255 INFO:tasks.ceph:role: osd.2 2024-04-25T10:06:06.255 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:06:06.255 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:06:06.334 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:06:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:06:06.334 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:06:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:06:06.335 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:06:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:06:06.335 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:06:06.335 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:06:06.335 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:06:06.335 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:06:06.721 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-04-25T10:06:06.727 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-04-25T10:06:06.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T10:06:06.845 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T10:06:06.919 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T10:06:06.977 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:06:06.978 INFO:tasks.ceph:role: osd.3 2024-04-25T10:06:06.978 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:06:06.978 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:06:07.045 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:06:07.045 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:06:07.045 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:06:07.045 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:06:07.045 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:06:07.045 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:06:07.045 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:06:07.046 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:06:07.046 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:06:07.046 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:06:07.456 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-04-25T10:06:07.463 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-04-25T10:06:07.465 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T10:06:07.611 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T10:06:07.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:06:07.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:07.827+0000 7f10c332c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T10:06:07.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:07.828+0000 7f10c332c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T10:06:07.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:07.828+0000 7f10c332c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:06:07.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:07.828+0000 7f10c332c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:06:07.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:07.828+0000 7f10c332c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:06:07.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:07.831+0000 7f10c332c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T10:06:11.045 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T10:06:11.131 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:06:11.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:11.255+0000 7f296b2b5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T10:06:11.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:11.255+0000 7f296b2b5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T10:06:11.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:11.255+0000 7f296b2b5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:06:11.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:11.255+0000 7f296b2b5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:06:11.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:11.255+0000 7f296b2b5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:06:11.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:11.258+0000 7f296b2b5640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T10:06:14.233 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T10:06:14.322 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:06:14.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:14.444+0000 7f62ecc10640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T10:06:14.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:14.445+0000 7f62ecc10640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T10:06:14.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:14.445+0000 7f62ecc10640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:06:14.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:14.445+0000 7f62ecc10640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:06:14.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:14.445+0000 7f62ecc10640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:06:14.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:14.447+0000 7f62ecc10640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T10:06:17.676 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T10:06:17.745 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:06:17.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:17.847+0000 7fb521884640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T10:06:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:17.847+0000 7fb521884640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T10:06:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:17.847+0000 7fb521884640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:06:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:17.848+0000 7fb521884640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:06:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:17.848+0000 7fb521884640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:06:17.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:17.850+0000 7fb521884640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T10:06:20.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T10:06:20.896 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T10:06:20.986 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:06:20.986 INFO:tasks.ceph:role: osd.4 2024-04-25T10:06:20.986 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-25T10:06:20.986 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:06:21.075 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:06:21.076 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:06:21.076 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:06:21.076 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:06:21.076 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:06:21.076 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:06:21.076 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:06:21.076 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:06:21.076 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:06:21.076 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:06:21.490 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-04-25T10:06:21.498 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-04-25T10:06:21.498 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T10:06:21.640 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T10:06:21.729 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T10:06:21.804 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:06:21.804 INFO:tasks.ceph:role: osd.5 2024-04-25T10:06:21.804 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-25T10:06:21.804 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:06:21.886 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:06:21.886 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:06:21.886 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:06:21.886 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:06:21.886 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:06:21.886 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:06:21.886 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:06:21.887 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:06:21.887 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:06:21.887 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:06:22.240 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-04-25T10:06:22.247 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-04-25T10:06:22.247 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T10:06:22.346 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T10:06:22.416 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T10:06:22.480 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:06:22.480 INFO:tasks.ceph:role: osd.6 2024-04-25T10:06:22.480 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-25T10:06:22.480 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:06:22.552 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:06:22.552 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:06:22.552 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:06:22.552 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:06:22.552 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:06:22.553 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:06:22.553 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:06:22.553 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:06:22.553 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:06:22.553 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:06:22.934 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-04-25T10:06:22.941 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-04-25T10:06:22.942 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T10:06:23.049 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T10:06:23.120 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T10:06:23.188 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:06:23.188 INFO:tasks.ceph:role: osd.7 2024-04-25T10:06:23.188 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-25T10:06:23.188 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:06:23.269 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:06:23.269 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:06:23.269 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:06:23.269 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:06:23.269 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:06:23.269 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:06:23.269 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:06:23.269 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:06:23.269 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:06:23.269 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:06:23.680 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-04-25T10:06:23.685 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-04-25T10:06:23.685 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T10:06:23.844 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T10:06:23.935 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:06:24.089 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:24.084+0000 7ff07b3fc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T10:06:24.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:24.084+0000 7ff07b3fc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T10:06:24.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:24.085+0000 7ff07b3fc640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:06:24.090 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:24.085+0000 7ff07b3fc640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:06:24.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:24.085+0000 7ff07b3fc640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:06:24.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:24.088+0000 7ff07b3fc640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T10:06:27.059 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T10:06:27.145 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:06:27.274 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:27.269+0000 7f7095a18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T10:06:27.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:27.269+0000 7f7095a18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T10:06:27.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:27.269+0000 7f7095a18640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:06:27.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:27.269+0000 7f7095a18640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:06:27.275 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:27.269+0000 7f7095a18640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:06:27.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:27.272+0000 7f7095a18640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T10:06:29.597 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:29.605 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:30.496 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T10:06:30.580 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:06:30.707 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:30.701+0000 7fd75000c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T10:06:30.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:30.702+0000 7fd75000c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T10:06:30.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:30.702+0000 7fd75000c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:06:30.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:30.702+0000 7fd75000c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:06:30.708 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:30.702+0000 7fd75000c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:06:30.710 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:30.705+0000 7fd75000c640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T10:06:33.929 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T10:06:34.013 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:06:34.140 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:34.135+0000 7f08ec65f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T10:06:34.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:34.135+0000 7f08ec65f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T10:06:34.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:34.135+0000 7f08ec65f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:06:34.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:34.135+0000 7f08ec65f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:06:34.141 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:34.135+0000 7f08ec65f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:06:34.143 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:34.138+0000 7f08ec65f640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T10:06:37.371 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T10:06:37.457 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T10:06:37.457 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:37.457 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T10:06:37.545 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:37.546 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T10:06:37.626 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:37.626 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T10:06:37.702 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:37.702 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T10:06:37.783 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:37.783 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T10:06:37.866 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:37.866 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T10:06:37.949 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:37.949 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T10:06:38.029 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:38.030 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T10:06:38.117 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:38.118 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T10:06:38.201 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:38.201 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T10:06:38.284 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:38.284 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T10:06:38.367 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:38.367 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T10:06:38.455 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:38.455 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T10:06:38.540 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:38.540 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T10:06:38.615 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:38.616 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T10:06:38.675 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:38.675 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T10:06:38.729 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:38.729 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T10:06:38.787 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:38.787 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T10:06:38.839 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T10:06:38.839 DEBUG:teuthology.orchestra.run.smithi119:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:06:38.847 DEBUG:teuthology.orchestra.run.smithi132:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:06:38.922 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.y] 2024-04-25T10:06:38.923 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQALKypmr8SqBBAA8l4CjUEidXz1obWJFM4DHA== 2024-04-25T10:06:38.923 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2024-04-25T10:06:38.923 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQALKypmFDz9FBAAdC/N+8UnkYkK5wrDlgpkgA== 2024-04-25T10:06:38.923 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2024-04-25T10:06:38.923 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQALKypmojotIhAAdSIhFOXoYZN+01nbCs3e+Q== 2024-04-25T10:06:38.923 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.0] 2024-04-25T10:06:38.923 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAPKypmOI9YMRAAmMrg0xRUG+GG+waJ12llJQ== 2024-04-25T10:06:38.923 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.1] 2024-04-25T10:06:38.923 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQATKypmHV5GDxAAW0XzpAxF7oxgVBKy5yzavw== 2024-04-25T10:06:38.923 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.2] 2024-04-25T10:06:38.924 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAWKypmXTiRGhAAZxct/Ck3A3DMBjqNW3HXrg== 2024-04-25T10:06:38.924 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.3] 2024-04-25T10:06:38.924 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAZKypmEeGSMhAAwVNTny5OmP/KONYhIXzrrQ== 2024-04-25T10:06:38.924 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.x] 2024-04-25T10:06:38.924 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQALKypm/KyKDBAA646fW6qrcXB31usxeIApaA== 2024-04-25T10:06:38.924 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2024-04-25T10:06:38.924 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQALKypmmh7lLhAAkzsFzTL6Ko/J/0ZJOpvR4A== 2024-04-25T10:06:38.924 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2024-04-25T10:06:38.924 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQALKypm1gp7ORAAvtG28IuhGtR1/Vj6JCPq5A== 2024-04-25T10:06:38.924 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.4] 2024-04-25T10:06:38.924 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAgKypmh7gPBRAApx1VM3WpYW7hIQzu7+6J7w== 2024-04-25T10:06:38.924 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.5] 2024-04-25T10:06:38.924 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAjKypmQ2EaEBAAoFYMlnh5/ar0pAlynwjYOA== 2024-04-25T10:06:38.925 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.6] 2024-04-25T10:06:38.925 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAmKypmDATkKRAAsIDViOBeDrd5N91wf8zGLA== 2024-04-25T10:06:38.925 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.7] 2024-04-25T10:06:38.925 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAqKypmNskbCBAAQIYhJ9lfHHr9EP/n5QJoRw== 2024-04-25T10:06:38.925 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-04-25T10:06:38.925 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA== 2024-04-25T10:06:38.925 INFO:teuthology.orchestra.run.smithi119.stdout:[client.2] 2024-04-25T10:06:38.925 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAMKypmMZ89ERAAICfWm/ZgOLQMVSv5xG0Crg== 2024-04-25T10:06:38.925 INFO:teuthology.orchestra.run.smithi119.stdout:[client.1] 2024-04-25T10:06:38.925 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAMKypmm0DLGhAApkxEcc1jUU5T/9ErkTkT4A== 2024-04-25T10:06:38.925 INFO:teuthology.orchestra.run.smithi119.stdout:[client.3] 2024-04-25T10:06:38.926 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAMKypmrokCIxAAwi8FtvYSVhMguvZ0OyYt+g== 2024-04-25T10:06:38.927 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr.y] 2024-04-25T10:06:38.927 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQALKypmr8SqBBAA8l4CjUEidXz1obWJFM4DHA== 2024-04-25T10:06:38.927 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2024-04-25T10:06:38.928 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQALKypmFDz9FBAAdC/N+8UnkYkK5wrDlgpkgA== 2024-04-25T10:06:38.928 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2024-04-25T10:06:38.928 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQALKypmojotIhAAdSIhFOXoYZN+01nbCs3e+Q== 2024-04-25T10:06:38.928 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.0] 2024-04-25T10:06:38.928 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQAPKypmOI9YMRAAmMrg0xRUG+GG+waJ12llJQ== 2024-04-25T10:06:38.928 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.1] 2024-04-25T10:06:38.928 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQATKypmHV5GDxAAW0XzpAxF7oxgVBKy5yzavw== 2024-04-25T10:06:38.928 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.2] 2024-04-25T10:06:38.928 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQAWKypmXTiRGhAAZxct/Ck3A3DMBjqNW3HXrg== 2024-04-25T10:06:38.928 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.3] 2024-04-25T10:06:38.928 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQAZKypmEeGSMhAAwVNTny5OmP/KONYhIXzrrQ== 2024-04-25T10:06:38.928 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr.x] 2024-04-25T10:06:38.928 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQALKypm/KyKDBAA646fW6qrcXB31usxeIApaA== 2024-04-25T10:06:38.929 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2024-04-25T10:06:38.929 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQALKypmmh7lLhAAkzsFzTL6Ko/J/0ZJOpvR4A== 2024-04-25T10:06:38.929 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2024-04-25T10:06:38.929 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQALKypm1gp7ORAAvtG28IuhGtR1/Vj6JCPq5A== 2024-04-25T10:06:38.929 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.4] 2024-04-25T10:06:38.929 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQAgKypmh7gPBRAApx1VM3WpYW7hIQzu7+6J7w== 2024-04-25T10:06:38.929 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.5] 2024-04-25T10:06:38.929 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQAjKypmQ2EaEBAAoFYMlnh5/ar0pAlynwjYOA== 2024-04-25T10:06:38.929 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.6] 2024-04-25T10:06:38.929 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQAmKypmDATkKRAAsIDViOBeDrd5N91wf8zGLA== 2024-04-25T10:06:38.929 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.7] 2024-04-25T10:06:38.929 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQAqKypmNskbCBAAQIYhJ9lfHHr9EP/n5QJoRw== 2024-04-25T10:06:38.929 INFO:teuthology.orchestra.run.smithi132.stdout:[client.0] 2024-04-25T10:06:38.930 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA== 2024-04-25T10:06:38.930 INFO:teuthology.orchestra.run.smithi132.stdout:[client.2] 2024-04-25T10:06:38.930 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQAMKypmMZ89ERAAICfWm/ZgOLQMVSv5xG0Crg== 2024-04-25T10:06:38.930 INFO:teuthology.orchestra.run.smithi132.stdout:[client.1] 2024-04-25T10:06:38.930 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQAMKypmm0DLGhAApkxEcc1jUU5T/9ErkTkT4A== 2024-04-25T10:06:38.930 INFO:teuthology.orchestra.run.smithi132.stdout:[client.3] 2024-04-25T10:06:38.930 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQAMKypmrokCIxAAwi8FtvYSVhMguvZ0OyYt+g== 2024-04-25T10:06:38.930 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:06:38.978 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:06:39.116 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:06:39.122 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:06:39.258 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:06:39.263 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:06:39.393 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:39.400 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:39.531 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:39.536 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:39.665 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:39.669 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:39.801 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:39.806 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:39.939 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:06:39.946 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:06:40.077 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:06:40.083 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:06:40.217 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:06:40.222 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:06:40.359 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:40.364 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:40.495 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:40.500 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:40.630 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:40.637 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:40.767 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:40.776 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:06:40.908 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:06:40.913 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:06:41.046 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:06:41.052 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:06:41.185 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:06:41.193 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:06:41.330 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:06:41.335 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:06:41.470 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T10:06:41.470 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T10:06:41.554 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:06:41.698 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T10:06:41.767 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T10:06:41.856 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:06:42.004 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T10:06:42.083 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T10:06:42.166 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:06:42.312 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T10:06:42.391 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:06:42.399 DEBUG:teuthology.orchestra.run.smithi132:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:06:42.450 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T10:06:42.450 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T10:06:42.450 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T10:06:42.459 INFO:tasks.ceph.mon.a:Started 2024-04-25T10:06:42.459 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T10:06:42.459 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T10:06:42.464 INFO:tasks.ceph.mon.b:Started 2024-04-25T10:06:42.464 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T10:06:42.464 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T10:06:42.468 INFO:tasks.ceph.mon.c:Started 2024-04-25T10:06:42.468 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T10:06:42.469 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T10:06:42.469 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T10:06:42.476 INFO:tasks.ceph.mgr.y:Started 2024-04-25T10:06:42.476 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T10:06:42.476 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T10:06:42.480 INFO:tasks.ceph.mgr.x:Started 2024-04-25T10:06:42.480 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T10:06:42.481 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T10:06:42.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.644+0000 7f23a1e00640 1 Processor -- start 2024-04-25T10:06:42.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.644+0000 7f23a1e00640 1 -- start start 2024-04-25T10:06:42.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.645+0000 7f23a1e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f239c161120 0x7f239c1617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.645+0000 7f23a1e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f239c161ce0 0x7f239c0b5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.645+0000 7f23a1e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239c161500 con 0x7f239c0b5a90 2024-04-25T10:06:42.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.645+0000 7f23a1e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f239c0b5f00 con 0x7f239c161ce0 2024-04-25T10:06:42.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.645+0000 7f23a1e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f239c162740 con 0x7f239c161120 2024-04-25T10:06:42.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.645+0000 7f2393e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f239c161ce0 0x7f239c0b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.645+0000 7f2393e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f239c161ce0 0x7f239c0b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36228/0 (socket says 172.21.15.119:36228) 2024-04-25T10:06:42.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.645+0000 7f2393e00640 1 -- 172.21.15.119:0/2223022619 learned_addr learned my addr 172.21.15.119:0/2223022619 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:06:42.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.646+0000 7f2393e00640 1 -- 172.21.15.119:0/2223022619 >> v2:172.21.15.119:3300/0 conn(0x7f239c161ce0 msgr2=0x7f239c0b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:06:42.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.646+0000 7f2393e00640 1 -- 172.21.15.119:0/2223022619 >> v2:172.21.15.119:3300/0 conn(0x7f239c161ce0 msgr2=0x7f239c0b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:06:42.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.646+0000 7f2393e00640 1 --2- 172.21.15.119:0/2223022619 >> v2:172.21.15.119:3300/0 conn(0x7f239c161ce0 0x7f239c0b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:06:42.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.646+0000 7f2393e00640 1 --2- 172.21.15.119:0/2223022619 >> v2:172.21.15.119:3300/0 conn(0x7f239c161ce0 0x7f239c0b5550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:06:42.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.646+0000 7f239aa00640 1 --2- 172.21.15.119:0/2223022619 >> v2:172.21.15.132:3300/0 conn(0x7f239c161120 0x7f239c1617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.646+0000 7f239aa00640 1 -- 172.21.15.119:0/2223022619 >> v2:172.21.15.132:3300/0 conn(0x7f239c161120 msgr2=0x7f239c1617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:06:42.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.646+0000 7f239aa00640 1 -- 172.21.15.119:0/2223022619 >> v2:172.21.15.132:3300/0 conn(0x7f239c161120 msgr2=0x7f239c1617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:06:42.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.646+0000 7f239aa00640 1 --2- 172.21.15.119:0/2223022619 >> v2:172.21.15.132:3300/0 conn(0x7f239c161120 0x7f239c1617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:06:42.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.646+0000 7f239aa00640 1 --2- 172.21.15.119:0/2223022619 >> v2:172.21.15.132:3300/0 conn(0x7f239c161120 0x7f239c1617a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:06:42.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.847+0000 7f239aa00640 1 --2- 172.21.15.119:0/2223022619 >> v2:172.21.15.132:3300/0 conn(0x7f239c161120 0x7f239c1617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.847+0000 7f2393e00640 1 --2- 172.21.15.119:0/2223022619 >> v2:172.21.15.119:3300/0 conn(0x7f239c161ce0 0x7f239c0b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.848+0000 7f239aa00640 1 -- 172.21.15.119:0/2223022619 >> v2:172.21.15.132:3300/0 conn(0x7f239c161120 msgr2=0x7f239c1617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:06:42.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.848+0000 7f239aa00640 1 -- 172.21.15.119:0/2223022619 >> v2:172.21.15.132:3300/0 conn(0x7f239c161120 msgr2=0x7f239c1617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:06:42.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.848+0000 7f239aa00640 1 --2- 172.21.15.119:0/2223022619 >> v2:172.21.15.132:3300/0 conn(0x7f239c161120 0x7f239c1617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:06:42.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.848+0000 7f239aa00640 1 --2- 172.21.15.119:0/2223022619 >> v2:172.21.15.132:3300/0 conn(0x7f239c161120 0x7f239c1617a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:06:42.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.848+0000 7f2393e00640 1 -- 172.21.15.119:0/2223022619 >> v2:172.21.15.119:3300/0 conn(0x7f239c161ce0 msgr2=0x7f239c0b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:06:42.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.848+0000 7f2393e00640 1 -- 172.21.15.119:0/2223022619 >> v2:172.21.15.119:3300/0 conn(0x7f239c161ce0 msgr2=0x7f239c0b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:06:42.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.848+0000 7f2393e00640 1 --2- 172.21.15.119:0/2223022619 >> v2:172.21.15.119:3300/0 conn(0x7f239c161ce0 0x7f239c0b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:06:42.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.848+0000 7f2393e00640 1 --2- 172.21.15.119:0/2223022619 >> v2:172.21.15.119:3300/0 conn(0x7f239c161ce0 0x7f239c0b5550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:06:42.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.904+0000 7f239a000640 1 -- 172.21.15.119:0/2223022619 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352001486 0 0) 0x7f239c161500 con 0x7f239c0b5a90 2024-04-25T10:06:42.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.905+0000 7f239a000640 1 -- 172.21.15.119:0/2223022619 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2380003520 con 0x7f239c0b5a90 2024-04-25T10:06:42.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.913+0000 7f239a000640 1 -- 172.21.15.119:0/2223022619 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3194360582 0 0) 0x7f2380003520 con 0x7f239c0b5a90 2024-04-25T10:06:42.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.913+0000 7f239a000640 1 -- 172.21.15.119:0/2223022619 >> v2:172.21.15.132:3300/0 conn(0x7f239c161120 msgr2=0x7f239c1617a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:06:42.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.913+0000 7f239a000640 1 --2- 172.21.15.119:0/2223022619 >> v2:172.21.15.132:3300/0 conn(0x7f239c161120 0x7f239c1617a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.913+0000 7f239a000640 1 -- 172.21.15.119:0/2223022619 >> v2:172.21.15.119:3300/0 conn(0x7f239c161ce0 msgr2=0x7f239c0b5550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:06:42.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.913+0000 7f239a000640 1 --2- 172.21.15.119:0/2223022619 >> v2:172.21.15.119:3300/0 conn(0x7f239c161ce0 0x7f239c0b5550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.913+0000 7f239a000640 1 -- 172.21.15.119:0/2223022619 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239c162180 con 0x7f239c0b5a90 2024-04-25T10:06:42.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.913+0000 7f239a000640 1 -- 172.21.15.119:0/2223022619 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f23940019f0 con 0x7f239c0b5a90 2024-04-25T10:06:42.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.914+0000 7f239a000640 1 -- 172.21.15.119:0/2223022619 >> v1:172.21.15.132:6789/0 conn(0x7f239c0b5a90 legacy=0x7f239c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.914+0000 7f239a000640 1 --2- 172.21.15.119:0/2223022619 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2380003a00 0x7f2380003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.914+0000 7f239a000640 1 --2- 172.21.15.119:0/2223022619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2380004800 0x7f2380004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.914+0000 7f239a000640 1 --2- 172.21.15.119:0/2223022619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2380005210 0x7f238000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.914+0000 7f239a000640 1 -- 172.21.15.119:0/2223022619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c0b5f00 con 0x7f2380004800 2024-04-25T10:06:42.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.914+0000 7f239a000640 1 -- 172.21.15.119:0/2223022619 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c162740 con 0x7f2380005210 2024-04-25T10:06:42.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.914+0000 7f239a000640 1 -- 172.21.15.119:0/2223022619 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238000c1c0 con 0x7f2380003a00 2024-04-25T10:06:42.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.915+0000 7f239aa00640 1 --2- 172.21.15.119:0/2223022619 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2380003a00 0x7f2380003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.915+0000 7f239b400640 1 --2- 172.21.15.119:0/2223022619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2380005210 0x7f238000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.915+0000 7f2393e00640 1 --2- 172.21.15.119:0/2223022619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2380004800 0x7f2380004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.915+0000 7f239aa00640 1 -- 172.21.15.119:0/2223022619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2380005210 msgr2=0x7f238000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.916+0000 7f239aa00640 1 --2- 172.21.15.119:0/2223022619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2380005210 0x7f238000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.916+0000 7f239aa00640 1 -- 172.21.15.119:0/2223022619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2380004800 msgr2=0x7f2380004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.916+0000 7f239aa00640 1 --2- 172.21.15.119:0/2223022619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2380004800 0x7f2380004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.916+0000 7f239aa00640 1 -- 172.21.15.119:0/2223022619 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2380003520 con 0x7f2380003a00 2024-04-25T10:06:42.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.916+0000 7f239b400640 1 --2- 172.21.15.119:0/2223022619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2380005210 0x7f238000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:42.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.916+0000 7f239b400640 1 -- 172.21.15.119:0/2223022619 reap_dead start 2024-04-25T10:06:42.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.919+0000 7f239aa00640 1 --2- 172.21.15.119:0/2223022619 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2380003a00 0x7f2380003e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f238c00da90 tx=0x7f2380005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dccc4e4b5f22b74c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.921+0000 7f239a000640 1 -- 172.21.15.119:0/2223022619 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c038070 con 0x7f2380003a00 2024-04-25T10:06:42.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.921+0000 7f239a000640 1 -- 172.21.15.119:0/2223022619 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c033410 con 0x7f2380003a00 2024-04-25T10:06:42.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.921+0000 7f239a000640 1 -- 172.21.15.119:0/2223022619 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c00d690 con 0x7f2380003a00 2024-04-25T10:06:42.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.922+0000 7f23a1e00640 1 -- 172.21.15.119:0/2223022619 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2380003a00 msgr2=0x7f2380003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.922+0000 7f23a1e00640 1 --2- 172.21.15.119:0/2223022619 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2380003a00 0x7f2380003e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f238c00da90 tx=0x7f2380005830 comp rx=0 tx=0).stop 2024-04-25T10:06:42.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.923+0000 7f23a1e00640 1 -- 172.21.15.119:0/2223022619 shutdown_connections 2024-04-25T10:06:42.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.923+0000 7f23a1e00640 1 --2- 172.21.15.119:0/2223022619 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2380003a00 0x7f2380003e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.923+0000 7f23a1e00640 1 -- 172.21.15.119:0/2223022619 >> 172.21.15.119:0/2223022619 conn(0x7f239c0d0550 msgr2=0x7f239c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.923+0000 7f23a1e00640 1 -- 172.21.15.119:0/2223022619 shutdown_connections 2024-04-25T10:06:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.923+0000 7f23a1e00640 1 -- 172.21.15.119:0/2223022619 wait complete. 2024-04-25T10:06:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.926+0000 7f23a1e00640 1 Processor -- start 2024-04-25T10:06:42.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.926+0000 7f23a1e00640 1 -- start start 2024-04-25T10:06:42.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.927+0000 7f23a1e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2380003a00 0x7f239c1548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.927+0000 7f23a1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f239c0cb5b0 0x7f239c1f7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.927+0000 7f23a1e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f239c1f7940 0x7f239c0c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.927+0000 7f23a1e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c0d2040 con 0x7f239c0cb5b0 2024-04-25T10:06:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.927+0000 7f23a1e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c0d1d60 con 0x7f239c1f7940 2024-04-25T10:06:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.927+0000 7f23a1e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239c0b95f0 con 0x7f2380003a00 2024-04-25T10:06:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.927+0000 7f239aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2380003a00 0x7f239c1548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.928+0000 7f2393e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f239c0cb5b0 0x7f239c1f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.928+0000 7f239aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2380003a00 0x7f239c1548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:33134/0 (socket says 172.21.15.119:33134) 2024-04-25T10:06:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.928+0000 7f239aa00640 1 -- 172.21.15.119:0/2647400199 learned_addr learned my addr 172.21.15.119:0/2647400199 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:06:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.928+0000 7f239b400640 1 --2- 172.21.15.119:0/2647400199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f239c1f7940 0x7f239c0c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.928+0000 7f239b400640 1 -- 172.21.15.119:0/2647400199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2380003a00 msgr2=0x7f239c1548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.928+0000 7f239b400640 1 --2- 172.21.15.119:0/2647400199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2380003a00 0x7f239c1548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.928+0000 7f239b400640 1 -- 172.21.15.119:0/2647400199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f239c0cb5b0 msgr2=0x7f239c1f7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.928+0000 7f239b400640 1 --2- 172.21.15.119:0/2647400199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f239c0cb5b0 0x7f239c1f7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.928+0000 7f239b400640 1 -- 172.21.15.119:0/2647400199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239c0cbab0 con 0x7f239c1f7940 2024-04-25T10:06:42.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.928+0000 7f2393e00640 1 --2- 172.21.15.119:0/2647400199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f239c0cb5b0 0x7f239c1f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:42.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.929+0000 7f239b400640 1 --2- 172.21.15.119:0/2647400199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f239c1f7940 0x7f239c0c7d60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f239c06bb50 tx=0x7f239400ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.930+0000 7f2393400640 1 -- 172.21.15.119:0/2647400199 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2394003b80 con 0x7f239c1f7940 2024-04-25T10:06:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.930+0000 7f2393400640 1 -- 172.21.15.119:0/2647400199 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2394018070 con 0x7f239c1f7940 2024-04-25T10:06:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.930+0000 7f2393400640 1 -- 172.21.15.119:0/2647400199 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23940050d0 con 0x7f239c1f7940 2024-04-25T10:06:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.930+0000 7f23a1e00640 1 -- 172.21.15.119:0/2647400199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239c0cbd90 con 0x7f239c1f7940 2024-04-25T10:06:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.931+0000 7f23a1e00640 1 -- 172.21.15.119:0/2647400199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239c0cc350 con 0x7f239c1f7940 2024-04-25T10:06:42.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.931+0000 7f2393400640 1 -- 172.21.15.119:0/2647400199 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2394005270 con 0x7f239c1f7940 2024-04-25T10:06:42.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.931+0000 7f2393400640 1 -- 172.21.15.119:0/2647400199 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f239401d460 con 0x7f239c1f7940 2024-04-25T10:06:42.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.932+0000 7f23a1e00640 1 -- 172.21.15.119:0/2647400199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239c067490 con 0x7f239c1f7940 2024-04-25T10:06:42.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:42.938+0000 7f2393400640 1 -- 172.21.15.119:0/2647400199 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f239c067490 con 0x7f239c1f7940 2024-04-25T10:06:43.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:43.012+0000 7f23a1e00640 1 -- 172.21.15.119:0/2647400199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f239c0750b0 con 0x7f239c1f7940 2024-04-25T10:06:43.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:43.025+0000 7f2393400640 1 -- 172.21.15.119:0/2647400199 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f239400cde0 con 0x7f239c1f7940 2024-04-25T10:06:43.030 INFO:teuthology.orchestra.run.smithi119.stderr:adjusted tunables profile to default 2024-04-25T10:06:43.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:43.026+0000 7f23a1e00640 1 -- 172.21.15.119:0/2647400199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f239c1f7940 msgr2=0x7f239c0c7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:43.026+0000 7f23a1e00640 1 --2- 172.21.15.119:0/2647400199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f239c1f7940 0x7f239c0c7d60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f239c06bb50 tx=0x7f239400ef00 comp rx=0 tx=0).stop 2024-04-25T10:06:43.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:43.026+0000 7f23a1e00640 1 -- 172.21.15.119:0/2647400199 shutdown_connections 2024-04-25T10:06:43.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:43.026+0000 7f23a1e00640 1 --2- 172.21.15.119:0/2647400199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f239c1f7940 0x7f239c0c7d60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:43.026+0000 7f23a1e00640 1 --2- 172.21.15.119:0/2647400199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f239c0cb5b0 0x7f239c1f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:43.026+0000 7f23a1e00640 1 --2- 172.21.15.119:0/2647400199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2380003a00 0x7f239c1548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:43.026+0000 7f23a1e00640 1 -- 172.21.15.119:0/2647400199 >> 172.21.15.119:0/2647400199 conn(0x7f239c0d0550 msgr2=0x7f239c0ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:43.027+0000 7f23a1e00640 1 -- 172.21.15.119:0/2647400199 shutdown_connections 2024-04-25T10:06:43.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:43.027+0000 7f23a1e00640 1 -- 172.21.15.119:0/2647400199 wait complete. 2024-04-25T10:06:43.050 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T10:06:43.050 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T10:06:43.051 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:43.051 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T10:06:43.077 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:43.071+0000 7f84f3e96040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:06:43.111 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:43.104+0000 7f5c573e3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:06:43.120 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:43.120 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T10:06:43.236 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:43.236 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T10:06:43.280 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:43.274+0000 7f84f3e96040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:06:43.307 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:06:43.307 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T10:06:43.337 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:43.331+0000 7f5c573e3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:06:43.347 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:43.342+0000 7f84f3e96040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:06:43.404 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:43.397+0000 7f5c573e3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:06:43.419 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:43.419 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T10:06:43.492 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:43.492 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T10:06:43.610 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:43.610 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T10:06:43.641 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:43.635+0000 7f84f3e96040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:06:43.693 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:43.687+0000 7f84f3e96040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:06:43.724 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:06:43.724 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T10:06:43.730 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:43.724+0000 7f5c573e3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:06:43.787 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:43.780+0000 7f5c573e3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:06:43.808 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 6587d006-498c-4076-8aa1-6f385b1a5002 0 2024-04-25T10:06:44.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.022+0000 7f7cd2200640 1 Processor -- start 2024-04-25T10:06:44.029 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.022+0000 7f7cd2200640 1 -- start start 2024-04-25T10:06:44.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.023+0000 7f7cd2200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ccc15dc50 0x7f7ccc099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.030 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.023+0000 7f7cd2200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7ccc099f80 0x7f7ccc06d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.023+0000 7f7cd2200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ccc0cbcd0 con 0x7f7ccc06d580 2024-04-25T10:06:44.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.023+0000 7f7cd2200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc0b0bc0 con 0x7f7ccc15dc50 2024-04-25T10:06:44.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.023+0000 7f7cd2200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc09a3e0 con 0x7f7ccc099f80 2024-04-25T10:06:44.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.024+0000 7f7ccbe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7ccc099f80 0x7f7ccc06d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.024+0000 7f7cd1800640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f7ccc06d580 0x7f7ccc0cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:47358/0 (socket says 172.21.15.132:47358) 2024-04-25T10:06:44.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.024+0000 7f7ccbe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7ccc099f80 0x7f7ccc06d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:58994/0 (socket says 172.21.15.132:58994) 2024-04-25T10:06:44.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.024+0000 7f7cd1800640 1 -- 172.21.15.132:0/1665284723 learned_addr learned my addr 172.21.15.132:0/1665284723 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-04-25T10:06:44.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.024+0000 7f7cd0e00640 1 --2- 172.21.15.132:0/1665284723 >> v2:172.21.15.119:3300/0 conn(0x7f7ccc15dc50 0x7f7ccc099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.024+0000 7f7ccbe00640 1 -- 172.21.15.132:0/1665284723 >> v2:172.21.15.119:3300/0 conn(0x7f7ccc15dc50 msgr2=0x7f7ccc099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.024+0000 7f7ccbe00640 1 --2- 172.21.15.132:0/1665284723 >> v2:172.21.15.119:3300/0 conn(0x7f7ccc15dc50 0x7f7ccc099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.031 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.024+0000 7f7ccbe00640 1 -- 172.21.15.132:0/1665284723 >> v1:172.21.15.132:6789/0 conn(0x7f7ccc06d580 legacy=0x7f7ccc0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.024+0000 7f7ccbe00640 1 -- 172.21.15.132:0/1665284723 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc0cbfb0 con 0x7f7ccc099f80 2024-04-25T10:06:44.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.024+0000 7f7cd0e00640 1 --2- 172.21.15.132:0/1665284723 >> v2:172.21.15.119:3300/0 conn(0x7f7ccc15dc50 0x7f7ccc099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:44.032 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.024+0000 7f7ccb400640 1 -- 172.21.15.132:0/1665284723 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152706164 0 0) 0x7f7ccc0cbcd0 con 0x7f7ccc06d580 2024-04-25T10:06:44.033 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.025+0000 7f7ccbe00640 1 --2- 172.21.15.132:0/1665284723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ccc099f80 0x7f7ccc06d040 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0724f0 tx=0x7f7cb802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=973a3b6c033fd7e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.025+0000 7f7ccb400640 1 -- 172.21.15.132:0/1665284723 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8037070 con 0x7f7ccc099f80 2024-04-25T10:06:44.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.025+0000 7f7ccb400640 1 -- 172.21.15.132:0/1665284723 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb8032440 con 0x7f7ccc099f80 2024-04-25T10:06:44.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.025+0000 7f7ccb400640 1 -- 172.21.15.132:0/1665284723 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8032810 con 0x7f7ccc099f80 2024-04-25T10:06:44.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.026+0000 7f7cd2200640 1 -- 172.21.15.132:0/1665284723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ccc099f80 msgr2=0x7f7ccc06d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.026+0000 7f7cd2200640 1 --2- 172.21.15.132:0/1665284723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ccc099f80 0x7f7ccc06d040 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0724f0 tx=0x7f7cb802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.026+0000 7f7cd2200640 1 -- 172.21.15.132:0/1665284723 shutdown_connections 2024-04-25T10:06:44.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.026+0000 7f7cd2200640 1 --2- 172.21.15.132:0/1665284723 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ccc099f80 0x7f7ccc06d040 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.026+0000 7f7cd2200640 1 --2- 172.21.15.132:0/1665284723 >> v2:172.21.15.119:3300/0 conn(0x7f7ccc15dc50 0x7f7ccc099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.026+0000 7f7cd2200640 1 -- 172.21.15.132:0/1665284723 >> 172.21.15.132:0/1665284723 conn(0x7f7ccc0b3170 msgr2=0x7f7ccc0b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.026+0000 7f7cd2200640 1 -- 172.21.15.132:0/1665284723 shutdown_connections 2024-04-25T10:06:44.034 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.026+0000 7f7cd2200640 1 -- 172.21.15.132:0/1665284723 wait complete. 2024-04-25T10:06:44.037 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.028+0000 7f7cd2200640 1 Processor -- start 2024-04-25T10:06:44.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.028+0000 7f7cd2200640 1 -- start start 2024-04-25T10:06:44.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.029+0000 7f7cd2200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc06d580 0x7f7ccc157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.029+0000 7f7cd2200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ccc099f80 0x7f7ccc0b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.029+0000 7f7cd2200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ccc15dc50 0x7f7ccc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.029+0000 7f7cd2200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0cc570 con 0x7f7ccc06d580 2024-04-25T10:06:44.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.029+0000 7f7cd2200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc15e030 con 0x7f7ccc15dc50 2024-04-25T10:06:44.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.029+0000 7f7cd2200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0cc850 con 0x7f7ccc099f80 2024-04-25T10:06:44.038 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.029+0000 7f7ccbe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ccc099f80 0x7f7ccc0b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.029+0000 7f7ccbe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ccc099f80 0x7f7ccc0b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:33578/0 (socket says 172.21.15.132:33578) 2024-04-25T10:06:44.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.029+0000 7f7cd1800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ccc15dc50 0x7f7ccc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.029+0000 7f7ccbe00640 1 -- 172.21.15.132:0/338595940 learned_addr learned my addr 172.21.15.132:0/338595940 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-25T10:06:44.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.029+0000 7f7cd0e00640 1 --2- 172.21.15.132:0/338595940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc06d580 0x7f7ccc157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.030+0000 7f7ccbe00640 1 -- 172.21.15.132:0/338595940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ccc15dc50 msgr2=0x7f7ccc0b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.030+0000 7f7ccbe00640 1 --2- 172.21.15.132:0/338595940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ccc15dc50 0x7f7ccc0b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.030+0000 7f7ccbe00640 1 -- 172.21.15.132:0/338595940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc06d580 msgr2=0x7f7ccc157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.030+0000 7f7ccbe00640 1 --2- 172.21.15.132:0/338595940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc06d580 0x7f7ccc157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.030+0000 7f7ccbe00640 1 -- 172.21.15.132:0/338595940 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc0b7400 con 0x7f7ccc099f80 2024-04-25T10:06:44.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.030+0000 7f7ccbe00640 1 --2- 172.21.15.132:0/338595940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ccc099f80 0x7f7ccc0b4970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8000c00 tx=0x7f7cb80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.030+0000 7f7cc8c00640 1 -- 172.21.15.132:0/338595940 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8037040 con 0x7f7ccc099f80 2024-04-25T10:06:44.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.030+0000 7f7cc8c00640 1 -- 172.21.15.132:0/338595940 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb8042070 con 0x7f7ccc099f80 2024-04-25T10:06:44.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.030+0000 7f7cd2200640 1 -- 172.21.15.132:0/338595940 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc15ffe0 con 0x7f7ccc099f80 2024-04-25T10:06:44.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.030+0000 7f7cc8c00640 1 -- 172.21.15.132:0/338595940 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8030bd0 con 0x7f7ccc099f80 2024-04-25T10:06:44.040 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.030+0000 7f7cd2200640 1 -- 172.21.15.132:0/338595940 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc1605a0 con 0x7f7ccc099f80 2024-04-25T10:06:44.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.031+0000 7f7cc8c00640 1 -- 172.21.15.132:0/338595940 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7cb8047460 con 0x7f7ccc099f80 2024-04-25T10:06:44.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.031+0000 7f7cc8c00640 1 -- 172.21.15.132:0/338595940 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7cb803d440 con 0x7f7ccc099f80 2024-04-25T10:06:44.041 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.032+0000 7f7cd2200640 1 -- 172.21.15.132:0/338595940 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc0749b0 con 0x7f7ccc099f80 2024-04-25T10:06:44.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.034+0000 7f7cc8c00640 1 -- 172.21.15.132:0/338595940 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7cb8032e20 con 0x7f7ccc099f80 2024-04-25T10:06:44.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.112+0000 7f7cd2200640 1 -- 172.21.15.132:0/338595940 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6587d006-498c-4076-8aa1-6f385b1a5002", "id": 0} v 0) v1 -- 0x7f7ccc070480 con 0x7f7ccc099f80 2024-04-25T10:06:44.125 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.119+0000 7f7cc8c00640 1 -- 172.21.15.132:0/338595940 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6587d006-498c-4076-8aa1-6f385b1a5002", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7cb8030dc0 con 0x7f7ccc099f80 2024-04-25T10:06:44.126 INFO:teuthology.orchestra.run.smithi132.stdout:0 2024-04-25T10:06:44.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.120+0000 7f7cd2200640 1 -- 172.21.15.132:0/338595940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ccc099f80 msgr2=0x7f7ccc0b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.120+0000 7f7cd2200640 1 --2- 172.21.15.132:0/338595940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ccc099f80 0x7f7ccc0b4970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8000c00 tx=0x7f7cb80046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.120+0000 7f7cd2200640 1 -- 172.21.15.132:0/338595940 shutdown_connections 2024-04-25T10:06:44.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.120+0000 7f7cd2200640 1 --2- 172.21.15.132:0/338595940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ccc15dc50 0x7f7ccc0b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.120+0000 7f7cd2200640 1 --2- 172.21.15.132:0/338595940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ccc099f80 0x7f7ccc0b4970 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.120+0000 7f7cd2200640 1 --2- 172.21.15.132:0/338595940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc06d580 0x7f7ccc157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.120+0000 7f7cd2200640 1 -- 172.21.15.132:0/338595940 >> 172.21.15.132:0/338595940 conn(0x7f7ccc0b3170 msgr2=0x7f7ccc0b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.120+0000 7f7cd2200640 1 -- 172.21.15.132:0/338595940 shutdown_connections 2024-04-25T10:06:44.127 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.121+0000 7f7cd2200640 1 -- 172.21.15.132:0/338595940 wait complete. 2024-04-25T10:06:44.143 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 7744b5a3-61e9-4e73-9764-ef4e11325b30 1 2024-04-25T10:06:44.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.314+0000 7f0735c00640 1 Processor -- start 2024-04-25T10:06:44.321 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.314+0000 7f0735c00640 1 -- start start 2024-04-25T10:06:44.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.316+0000 7f0735c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0730161120 0x7f07301617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.316+0000 7f0735c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f07300b5c60 0x7f07300d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.316+0000 7f0735c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0730161500 con 0x7f0730161ce0 2024-04-25T10:06:44.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.316+0000 7f0735c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0730162740 con 0x7f07300b5c60 2024-04-25T10:06:44.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.316+0000 7f0735c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0730162180 con 0x7f0730161120 2024-04-25T10:06:44.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.316+0000 7f072ea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0730161120 0x7f07301617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.316+0000 7f072ea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0730161120 0x7f07301617a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59004/0 (socket says 172.21.15.132:59004) 2024-04-25T10:06:44.323 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.316+0000 7f072ea00640 1 -- 172.21.15.132:0/3121788043 learned_addr learned my addr 172.21.15.132:0/3121788043 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-25T10:06:44.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.317+0000 7f072ea00640 1 -- 172.21.15.132:0/3121788043 >> v2:172.21.15.119:3300/0 conn(0x7f07300b5c60 msgr2=0x7f07300d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.317+0000 7f072f400640 1 --2- 172.21.15.132:0/3121788043 >> v2:172.21.15.119:3300/0 conn(0x7f07300b5c60 0x7f07300d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.317+0000 7f072ea00640 1 --2- 172.21.15.132:0/3121788043 >> v2:172.21.15.119:3300/0 conn(0x7f07300b5c60 0x7f07300d2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.317+0000 7f072ea00640 1 -- 172.21.15.132:0/3121788043 >> v1:172.21.15.119:6789/0 conn(0x7f0730161ce0 legacy=0x7f07300b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.324 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.317+0000 7f072ea00640 1 -- 172.21.15.132:0/3121788043 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0730162460 con 0x7f0730161120 2024-04-25T10:06:44.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.319+0000 7f072ea00640 1 --2- 172.21.15.132:0/3121788043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0730161120 0x7f07301617a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f073006dac0 tx=0x7f072002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=509120d38dc110e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.319+0000 7f072e000640 1 -- 172.21.15.132:0/3121788043 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720037070 con 0x7f0730161120 2024-04-25T10:06:44.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.319+0000 7f072e000640 1 -- 172.21.15.132:0/3121788043 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0720032440 con 0x7f0730161120 2024-04-25T10:06:44.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.319+0000 7f072e000640 1 -- 172.21.15.132:0/3121788043 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720032810 con 0x7f0730161120 2024-04-25T10:06:44.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.319+0000 7f0735c00640 1 -- 172.21.15.132:0/3121788043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0730161120 msgr2=0x7f07301617a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.319+0000 7f0735c00640 1 --2- 172.21.15.132:0/3121788043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0730161120 0x7f07301617a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f073006dac0 tx=0x7f072002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.319+0000 7f0735c00640 1 -- 172.21.15.132:0/3121788043 shutdown_connections 2024-04-25T10:06:44.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.319+0000 7f0735c00640 1 --2- 172.21.15.132:0/3121788043 >> v2:172.21.15.119:3300/0 conn(0x7f07300b5c60 0x7f07300d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.319+0000 7f0735c00640 1 --2- 172.21.15.132:0/3121788043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0730161120 0x7f07301617a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.319+0000 7f0735c00640 1 -- 172.21.15.132:0/3121788043 >> 172.21.15.132:0/3121788043 conn(0x7f07300d0550 msgr2=0x7f07300d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.319+0000 7f0735c00640 1 -- 172.21.15.132:0/3121788043 shutdown_connections 2024-04-25T10:06:44.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.319+0000 7f0735c00640 1 -- 172.21.15.132:0/3121788043 wait complete. 2024-04-25T10:06:44.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.320+0000 7f0735c00640 1 Processor -- start 2024-04-25T10:06:44.326 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.320+0000 7f0735c00640 1 -- start start 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.321+0000 7f0735c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07300b5c60 0x7f07300d2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.321+0000 7f0735c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0730161120 0x7f07300d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.321+0000 7f0735c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0730161ce0 0x7f07301f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.321+0000 7f0735c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07300b95f0 con 0x7f07300b5c60 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.321+0000 7f0735c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07300d2040 con 0x7f0730161120 2024-04-25T10:06:44.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.321+0000 7f0735c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07300b9770 con 0x7f0730161ce0 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.322+0000 7f072f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0730161ce0 0x7f07301f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.322+0000 7f0727e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0730161120 0x7f07300d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.322+0000 7f072f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0730161ce0 0x7f07301f40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:33594/0 (socket says 172.21.15.132:33594) 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.322+0000 7f072f400640 1 -- 172.21.15.132:0/3696973450 learned_addr learned my addr 172.21.15.132:0/3696973450 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.322+0000 7f072ea00640 1 --2- 172.21.15.132:0/3696973450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07300b5c60 0x7f07300d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.322+0000 7f0727e00640 1 -- 172.21.15.132:0/3696973450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0730161ce0 msgr2=0x7f07301f40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.322+0000 7f0727e00640 1 --2- 172.21.15.132:0/3696973450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0730161ce0 0x7f07301f40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.322+0000 7f0727e00640 1 -- 172.21.15.132:0/3696973450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07300b5c60 msgr2=0x7f07300d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.322+0000 7f0727e00640 1 --2- 172.21.15.132:0/3696973450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07300b5c60 0x7f07300d2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.322+0000 7f0727e00640 1 -- 172.21.15.132:0/3696973450 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07301f4de0 con 0x7f0730161120 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.322+0000 7f072f400640 1 --2- 172.21.15.132:0/3696973450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0730161ce0 0x7f07301f40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.322+0000 7f0727e00640 1 --2- 172.21.15.132:0/3696973450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0730161120 0x7f07300d2ca0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f071800b3e0 tx=0x7f071800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.323+0000 7f0726a00640 1 -- 172.21.15.132:0/3696973450 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718015020 con 0x7f0730161120 2024-04-25T10:06:44.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.323+0000 7f0726a00640 1 -- 172.21.15.132:0/3696973450 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0718011830 con 0x7f0730161120 2024-04-25T10:06:44.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.323+0000 7f0726a00640 1 -- 172.21.15.132:0/3696973450 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718011c20 con 0x7f0730161120 2024-04-25T10:06:44.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.323+0000 7f0735c00640 1 -- 172.21.15.132:0/3696973450 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0730148dc0 con 0x7f0730161120 2024-04-25T10:06:44.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.323+0000 7f0735c00640 1 -- 172.21.15.132:0/3696973450 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0730149380 con 0x7f0730161120 2024-04-25T10:06:44.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.323+0000 7f0726a00640 1 -- 172.21.15.132:0/3696973450 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0718018020 con 0x7f0730161120 2024-04-25T10:06:44.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.323+0000 7f0726a00640 1 -- 172.21.15.132:0/3696973450 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f0718019990 con 0x7f0730161120 2024-04-25T10:06:44.331 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.324+0000 7f0735c00640 1 -- 172.21.15.132:0/3696973450 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07300b6630 con 0x7f0730161120 2024-04-25T10:06:44.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.326+0000 7f0726a00640 1 -- 172.21.15.132:0/3696973450 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f07300b6630 con 0x7f0730161120 2024-04-25T10:06:44.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.406+0000 7f0735c00640 1 -- 172.21.15.132:0/3696973450 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7744b5a3-61e9-4e73-9764-ef4e11325b30", "id": 1} v 0) v1 -- 0x7f0730149580 con 0x7f0730161120 2024-04-25T10:06:44.418 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.411+0000 7f0726a00640 1 -- 172.21.15.132:0/3696973450 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7744b5a3-61e9-4e73-9764-ef4e11325b30", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0730149580 con 0x7f0730161120 2024-04-25T10:06:44.418 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-04-25T10:06:44.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.413+0000 7f0735c00640 1 -- 172.21.15.132:0/3696973450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0730161120 msgr2=0x7f07300d2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.413+0000 7f0735c00640 1 --2- 172.21.15.132:0/3696973450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0730161120 0x7f07300d2ca0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f071800b3e0 tx=0x7f071800b8a0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.413+0000 7f0735c00640 1 -- 172.21.15.132:0/3696973450 shutdown_connections 2024-04-25T10:06:44.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.413+0000 7f0735c00640 1 --2- 172.21.15.132:0/3696973450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0730161ce0 0x7f07301f40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.413+0000 7f0735c00640 1 --2- 172.21.15.132:0/3696973450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0730161120 0x7f07300d2ca0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.413+0000 7f0735c00640 1 --2- 172.21.15.132:0/3696973450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07300b5c60 0x7f07300d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.413+0000 7f0735c00640 1 -- 172.21.15.132:0/3696973450 >> 172.21.15.132:0/3696973450 conn(0x7f07300d0550 msgr2=0x7f07300cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.420 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.414+0000 7f0735c00640 1 -- 172.21.15.132:0/3696973450 shutdown_connections 2024-04-25T10:06:44.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.414+0000 7f0735c00640 1 -- 172.21.15.132:0/3696973450 wait complete. 2024-04-25T10:06:44.436 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new edb818b9-6266-4e52-88bf-d3659485dc44 2 2024-04-25T10:06:44.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.594+0000 7f9d70200640 1 Processor -- start 2024-04-25T10:06:44.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.595+0000 7f9d70200640 1 -- start start 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.596+0000 7f9d70200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d680b5550 0x7f9d680b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.596+0000 7f9d70200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d68161500 con 0x7f9d68161120 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.596+0000 7f9d70200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d68161eb0 con 0x7f9d680d2560 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.596+0000 7f9d70200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d680b5ef0 con 0x7f9d680b5550 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.596+0000 7f9d6ce00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d680b5550 0x7f9d680b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.597+0000 7f9d6e200640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f9d680d2560 0x7f9d680d29f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:47362/0 (socket says 172.21.15.132:47362) 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.597+0000 7f9d6e200640 1 -- 172.21.15.132:0/1645317871 learned_addr learned my addr 172.21.15.132:0/1645317871 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-04-25T10:06:44.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.597+0000 7f9d6ce00640 1 -- 172.21.15.132:0/1645317871 >> v1:172.21.15.132:6789/0 conn(0x7f9d680d2560 legacy=0x7f9d680d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.597+0000 7f9d6ce00640 1 -- 172.21.15.132:0/1645317871 >> v1:172.21.15.119:6789/0 conn(0x7f9d68161120 legacy=0x7f9d681617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.597+0000 7f9d6ce00640 1 -- 172.21.15.132:0/1645317871 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d68162180 con 0x7f9d680b5550 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.600+0000 7f9d6ce00640 1 --2- 172.21.15.132:0/1645317871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d680b5550 0x7f9d680b59b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9d540097d0 tx=0x7f9d5402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ab24a61539a7641 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.600+0000 7f9d67e00640 1 -- 172.21.15.132:0/1645317871 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54037070 con 0x7f9d680b5550 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.600+0000 7f9d67e00640 1 -- 172.21.15.132:0/1645317871 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d54032440 con 0x7f9d680b5550 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.600+0000 7f9d67e00640 1 -- 172.21.15.132:0/1645317871 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54032810 con 0x7f9d680b5550 2024-04-25T10:06:44.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.600+0000 7f9d70200640 1 -- 172.21.15.132:0/1645317871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d680b5550 msgr2=0x7f9d680b59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.600+0000 7f9d70200640 1 --2- 172.21.15.132:0/1645317871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d680b5550 0x7f9d680b59b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9d540097d0 tx=0x7f9d5402a180 comp rx=0 tx=0).stop 2024-04-25T10:06:44.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.601+0000 7f9d70200640 1 -- 172.21.15.132:0/1645317871 shutdown_connections 2024-04-25T10:06:44.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.601+0000 7f9d70200640 1 --2- 172.21.15.132:0/1645317871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d680b5550 0x7f9d680b59b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.601+0000 7f9d70200640 1 -- 172.21.15.132:0/1645317871 >> 172.21.15.132:0/1645317871 conn(0x7f9d680d0550 msgr2=0x7f9d680d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.601+0000 7f9d70200640 1 -- 172.21.15.132:0/1645317871 shutdown_connections 2024-04-25T10:06:44.606 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.601+0000 7f9d70200640 1 -- 172.21.15.132:0/1645317871 wait complete. 2024-04-25T10:06:44.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.602+0000 7f9d70200640 1 Processor -- start 2024-04-25T10:06:44.607 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.602+0000 7f9d70200640 1 -- start start 2024-04-25T10:06:44.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.603+0000 7f9d70200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d680b5550 0x7f9d680c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.603+0000 7f9d70200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d680d2560 0x7f9d681f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.603+0000 7f9d70200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d68161120 0x7f9d681f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.603+0000 7f9d70200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d680d1d60 con 0x7f9d680d2560 2024-04-25T10:06:44.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.603+0000 7f9d70200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d68162740 con 0x7f9d68161120 2024-04-25T10:06:44.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.603+0000 7f9d70200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d680d2040 con 0x7f9d680b5550 2024-04-25T10:06:44.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.603+0000 7f9d6d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d680b5550 0x7f9d680c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.604+0000 7f9d6d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d680b5550 0x7f9d680c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:33616/0 (socket says 172.21.15.132:33616) 2024-04-25T10:06:44.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.604+0000 7f9d6d800640 1 -- 172.21.15.132:0/3642392735 learned_addr learned my addr 172.21.15.132:0/3642392735 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-25T10:06:44.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.604+0000 7f9d6e200640 1 --2- 172.21.15.132:0/3642392735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d68161120 0x7f9d681f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.604+0000 7f9d6ce00640 1 --2- 172.21.15.132:0/3642392735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d680d2560 0x7f9d681f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.604+0000 7f9d6d800640 1 -- 172.21.15.132:0/3642392735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d68161120 msgr2=0x7f9d681f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.604+0000 7f9d6d800640 1 --2- 172.21.15.132:0/3642392735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d68161120 0x7f9d681f3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.604+0000 7f9d6d800640 1 -- 172.21.15.132:0/3642392735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d680d2560 msgr2=0x7f9d681f83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.604+0000 7f9d6d800640 1 --2- 172.21.15.132:0/3642392735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d680d2560 0x7f9d681f83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.604+0000 7f9d6d800640 1 -- 172.21.15.132:0/3642392735 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d681f42d0 con 0x7f9d680b5550 2024-04-25T10:06:44.611 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.604+0000 7f9d6d800640 1 --2- 172.21.15.132:0/3642392735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d680b5550 0x7f9d680c7600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00c400 tx=0x7f9d5c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.605+0000 7f9d65600640 1 -- 172.21.15.132:0/3642392735 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c017070 con 0x7f9d680b5550 2024-04-25T10:06:44.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.605+0000 7f9d65600640 1 -- 172.21.15.132:0/3642392735 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5c0052d0 con 0x7f9d680b5550 2024-04-25T10:06:44.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.605+0000 7f9d70200640 1 -- 172.21.15.132:0/3642392735 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d680c9fa0 con 0x7f9d680b5550 2024-04-25T10:06:44.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.605+0000 7f9d65600640 1 -- 172.21.15.132:0/3642392735 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c003830 con 0x7f9d680b5550 2024-04-25T10:06:44.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.605+0000 7f9d70200640 1 -- 172.21.15.132:0/3642392735 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d68159660 con 0x7f9d680b5550 2024-04-25T10:06:44.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.605+0000 7f9d65600640 1 -- 172.21.15.132:0/3642392735 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9d5c01a070 con 0x7f9d680b5550 2024-04-25T10:06:44.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.605+0000 7f9d65600640 1 -- 172.21.15.132:0/3642392735 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f9d5c011590 con 0x7f9d680b5550 2024-04-25T10:06:44.612 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.606+0000 7f9d70200640 1 -- 172.21.15.132:0/3642392735 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d680750b0 con 0x7f9d680b5550 2024-04-25T10:06:44.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.608+0000 7f9d65600640 1 -- 172.21.15.132:0/3642392735 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9d5c021070 con 0x7f9d680b5550 2024-04-25T10:06:44.692 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.687+0000 7f9d70200640 1 -- 172.21.15.132:0/3642392735 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "edb818b9-6266-4e52-88bf-d3659485dc44", "id": 2} v 0) v1 -- 0x7f9d680d17e0 con 0x7f9d680b5550 2024-04-25T10:06:44.693 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:44.688+0000 7f84f3e96040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:06:44.699 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.694+0000 7f9d65600640 1 -- 172.21.15.132:0/3642392735 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "edb818b9-6266-4e52-88bf-d3659485dc44", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9d5c011b80 con 0x7f9d680b5550 2024-04-25T10:06:44.699 INFO:teuthology.orchestra.run.smithi132.stdout:2 2024-04-25T10:06:44.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.695+0000 7f9d70200640 1 -- 172.21.15.132:0/3642392735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d680b5550 msgr2=0x7f9d680c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.695+0000 7f9d70200640 1 --2- 172.21.15.132:0/3642392735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d680b5550 0x7f9d680c7600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00c400 tx=0x7f9d5c00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.695+0000 7f9d70200640 1 -- 172.21.15.132:0/3642392735 shutdown_connections 2024-04-25T10:06:44.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.695+0000 7f9d70200640 1 --2- 172.21.15.132:0/3642392735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d68161120 0x7f9d681f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.695+0000 7f9d70200640 1 --2- 172.21.15.132:0/3642392735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d680d2560 0x7f9d681f83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.695+0000 7f9d70200640 1 --2- 172.21.15.132:0/3642392735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d680b5550 0x7f9d680c7600 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.695+0000 7f9d70200640 1 -- 172.21.15.132:0/3642392735 >> 172.21.15.132:0/3642392735 conn(0x7f9d680d0550 msgr2=0x7f9d680b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.695+0000 7f9d70200640 1 -- 172.21.15.132:0/3642392735 shutdown_connections 2024-04-25T10:06:44.701 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.695+0000 7f9d70200640 1 -- 172.21.15.132:0/3642392735 wait complete. 2024-04-25T10:06:44.718 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 2febff65-e13e-435b-8556-8b0b3399b875 3 2024-04-25T10:06:44.784 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:44.778+0000 7f5c573e3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:06:44.796 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:44.790+0000 7f84f3e96040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:06:44.845 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:44.839+0000 7f84f3e96040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:06:44.900 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:44.894+0000 7f5c573e3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:06:44.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.894+0000 7f703d800640 1 Processor -- start 2024-04-25T10:06:44.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.895+0000 7f703d800640 1 -- start start 2024-04-25T10:06:44.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.896+0000 7f703d800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70380b5550 0x7f70380b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.896+0000 7f703d800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7038161500 con 0x7f7038161120 2024-04-25T10:06:44.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.896+0000 7f703d800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7038161eb0 con 0x7f70380d2560 2024-04-25T10:06:44.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.896+0000 7f703d800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70380b5ef0 con 0x7f70380b5550 2024-04-25T10:06:44.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.896+0000 7f7037400640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f70380d2560 0x7f70380d2980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:47370/0 (socket says 172.21.15.132:47370) 2024-04-25T10:06:44.902 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.896+0000 7f7037400640 1 -- 172.21.15.132:0/3762205378 learned_addr learned my addr 172.21.15.132:0/3762205378 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-04-25T10:06:44.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.896+0000 7f702fe00640 1 --2- 172.21.15.132:0/3762205378 >> v2:172.21.15.119:3300/0 conn(0x7f70380b5550 0x7f70380b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 -- 172.21.15.132:0/3762205378 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008015582 0 0) 0x7f7038161eb0 con 0x7f70380d2560 2024-04-25T10:06:44.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 -- 172.21.15.132:0/3762205378 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f701c003520 con 0x7f70380d2560 2024-04-25T10:06:44.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 -- 172.21.15.132:0/3762205378 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1959654673 0 0) 0x7f701c003520 con 0x7f70380d2560 2024-04-25T10:06:44.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 -- 172.21.15.132:0/3762205378 >> v2:172.21.15.119:3300/0 conn(0x7f70380b5550 msgr2=0x7f70380b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 --2- 172.21.15.132:0/3762205378 >> v2:172.21.15.119:3300/0 conn(0x7f70380b5550 0x7f70380b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 -- 172.21.15.132:0/3762205378 >> v1:172.21.15.119:6789/0 conn(0x7f7038161120 legacy=0x7f70381617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 -- 172.21.15.132:0/3762205378 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7038162180 con 0x7f70380d2560 2024-04-25T10:06:44.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 -- 172.21.15.132:0/3762205378 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f70300019f0 con 0x7f70380d2560 2024-04-25T10:06:44.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 -- 172.21.15.132:0/3762205378 >> v1:172.21.15.132:6789/0 conn(0x7f70380d2560 legacy=0x7f70380d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 --2- 172.21.15.132:0/3762205378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f701c003a00 0x7f701c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.904 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 --2- 172.21.15.132:0/3762205378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f701c004800 0x7f701c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 --2- 172.21.15.132:0/3762205378 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f701c005210 0x7f701c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 -- 172.21.15.132:0/3762205378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c00c130 con 0x7f701c004800 2024-04-25T10:06:44.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 -- 172.21.15.132:0/3762205378 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c00c2e0 con 0x7f701c003a00 2024-04-25T10:06:44.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.897+0000 7f7036000640 1 -- 172.21.15.132:0/3762205378 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701c00c4e0 con 0x7f701c005210 2024-04-25T10:06:44.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.898+0000 7f7036a00640 1 --2- 172.21.15.132:0/3762205378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f701c003a00 0x7f701c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.898+0000 7f7037400640 1 --2- 172.21.15.132:0/3762205378 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f701c005210 0x7f701c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.898+0000 7f702fe00640 1 --2- 172.21.15.132:0/3762205378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f701c004800 0x7f701c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.898+0000 7f7036a00640 1 -- 172.21.15.132:0/3762205378 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f701c005210 msgr2=0x7f701c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.898+0000 7f7036a00640 1 --2- 172.21.15.132:0/3762205378 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f701c005210 0x7f701c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.898+0000 7f7036a00640 1 -- 172.21.15.132:0/3762205378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f701c004800 msgr2=0x7f701c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.898+0000 7f7036a00640 1 --2- 172.21.15.132:0/3762205378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f701c004800 0x7f701c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.898+0000 7f7036a00640 1 -- 172.21.15.132:0/3762205378 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c003520 con 0x7f701c003a00 2024-04-25T10:06:44.905 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.898+0000 7f7037400640 1 -- 172.21.15.132:0/3762205378 reap_dead start 2024-04-25T10:06:44.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.900+0000 7f7036a00640 1 --2- 172.21.15.132:0/3762205378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f701c003a00 0x7f701c003e20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7028001720 tx=0x7f701c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5097363d2c783de5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.900+0000 7f7036000640 1 -- 172.21.15.132:0/3762205378 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7028001370 con 0x7f701c003a00 2024-04-25T10:06:44.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.900+0000 7f7036000640 1 -- 172.21.15.132:0/3762205378 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70280052d0 con 0x7f701c003a00 2024-04-25T10:06:44.908 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.900+0000 7f7036000640 1 -- 172.21.15.132:0/3762205378 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702802abc0 con 0x7f701c003a00 2024-04-25T10:06:44.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.901+0000 7f703d800640 1 -- 172.21.15.132:0/3762205378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f701c003a00 msgr2=0x7f701c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.901+0000 7f703d800640 1 --2- 172.21.15.132:0/3762205378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f701c003a00 0x7f701c003e20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7028001720 tx=0x7f701c005830 comp rx=0 tx=0).stop 2024-04-25T10:06:44.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.901+0000 7f703d800640 1 -- 172.21.15.132:0/3762205378 shutdown_connections 2024-04-25T10:06:44.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.901+0000 7f703d800640 1 --2- 172.21.15.132:0/3762205378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f701c003a00 0x7f701c003e20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.901+0000 7f703d800640 1 -- 172.21.15.132:0/3762205378 >> 172.21.15.132:0/3762205378 conn(0x7f70380d0550 msgr2=0x7f70380d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.901+0000 7f703d800640 1 -- 172.21.15.132:0/3762205378 shutdown_connections 2024-04-25T10:06:44.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.902+0000 7f703d800640 1 -- 172.21.15.132:0/3762205378 wait complete. 2024-04-25T10:06:44.909 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.904+0000 7f703d800640 1 Processor -- start 2024-04-25T10:06:44.910 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.904+0000 7f703d800640 1 -- start start 2024-04-25T10:06:44.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.905+0000 7f703d800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f701c003a00 0x7f70380c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.905+0000 7f703d800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70380c9c60 0x7f70380ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.905+0000 7f703d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f703815a580 0x7f703815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.905+0000 7f703d800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70380d1d60 con 0x7f70380c9c60 2024-04-25T10:06:44.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.905+0000 7f703d800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7038162740 con 0x7f701c003a00 2024-04-25T10:06:44.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.905+0000 7f703d800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70380d2040 con 0x7f703815a580 2024-04-25T10:06:44.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.905+0000 7f7037400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f703815a580 0x7f703815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.905+0000 7f7036a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f701c003a00 0x7f70380c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.905+0000 7f7037400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f703815a580 0x7f703815aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:33636/0 (socket says 172.21.15.132:33636) 2024-04-25T10:06:44.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.905+0000 7f7037400640 1 -- 172.21.15.132:0/3833592224 learned_addr learned my addr 172.21.15.132:0/3833592224 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-25T10:06:44.917 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.905+0000 7f702fe00640 1 --2- 172.21.15.132:0/3833592224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70380c9c60 0x7f70380ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.906+0000 7f7037400640 1 -- 172.21.15.132:0/3833592224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f701c003a00 msgr2=0x7f70380c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.906+0000 7f7037400640 1 --2- 172.21.15.132:0/3833592224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f701c003a00 0x7f70380c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.906+0000 7f7037400640 1 -- 172.21.15.132:0/3833592224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70380c9c60 msgr2=0x7f70380ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.906+0000 7f7037400640 1 --2- 172.21.15.132:0/3833592224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70380c9c60 0x7f70380ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.906+0000 7f7037400640 1 -- 172.21.15.132:0/3833592224 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70380c7640 con 0x7f703815a580 2024-04-25T10:06:44.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.906+0000 7f7037400640 1 --2- 172.21.15.132:0/3833592224 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f703815a580 0x7f703815aa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f70380cab30 tx=0x7f7030007bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.906+0000 7f702ea00640 1 -- 172.21.15.132:0/3833592224 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703000aad0 con 0x7f703815a580 2024-04-25T10:06:44.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.906+0000 7f702ea00640 1 -- 172.21.15.132:0/3833592224 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7030018070 con 0x7f703815a580 2024-04-25T10:06:44.918 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.906+0000 7f702ea00640 1 -- 172.21.15.132:0/3833592224 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70300133f0 con 0x7f703815a580 2024-04-25T10:06:44.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.906+0000 7f703d800640 1 -- 172.21.15.132:0/3833592224 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70380c7920 con 0x7f703815a580 2024-04-25T10:06:44.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.906+0000 7f703d800640 1 -- 172.21.15.132:0/3833592224 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70380c7ee0 con 0x7f703815a580 2024-04-25T10:06:44.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.907+0000 7f702ea00640 1 -- 172.21.15.132:0/3833592224 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7030013590 con 0x7f703815a580 2024-04-25T10:06:44.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.907+0000 7f702ea00640 1 -- 172.21.15.132:0/3833592224 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f703001d460 con 0x7f703815a580 2024-04-25T10:06:44.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.908+0000 7f703d800640 1 -- 172.21.15.132:0/3833592224 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70380750b0 con 0x7f703815a580 2024-04-25T10:06:44.919 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.910+0000 7f702ea00640 1 -- 172.21.15.132:0/3833592224 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7030013880 con 0x7f703815a580 2024-04-25T10:06:44.959 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:44.954+0000 7f5c573e3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:06:44.994 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.989+0000 7f703d800640 1 -- 172.21.15.132:0/3833592224 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2febff65-e13e-435b-8556-8b0b3399b875", "id": 3} v 0) v1 -- 0x7f70380b6090 con 0x7f703815a580 2024-04-25T10:06:45.001 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.996+0000 7f702ea00640 1 -- 172.21.15.132:0/3833592224 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2febff65-e13e-435b-8556-8b0b3399b875", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7030013bb0 con 0x7f703815a580 2024-04-25T10:06:45.001 INFO:teuthology.orchestra.run.smithi132.stdout:3 2024-04-25T10:06:45.003 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.997+0000 7f703d800640 1 -- 172.21.15.132:0/3833592224 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f703815a580 msgr2=0x7f703815aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.003 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.997+0000 7f703d800640 1 --2- 172.21.15.132:0/3833592224 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f703815a580 0x7f703815aa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f70380cab30 tx=0x7f7030007bd0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.003 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.997+0000 7f703d800640 1 -- 172.21.15.132:0/3833592224 shutdown_connections 2024-04-25T10:06:45.003 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.997+0000 7f703d800640 1 --2- 172.21.15.132:0/3833592224 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f703815a580 0x7f703815aa30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.003 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.998+0000 7f703d800640 1 --2- 172.21.15.132:0/3833592224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70380c9c60 0x7f70380ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.003 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.998+0000 7f703d800640 1 --2- 172.21.15.132:0/3833592224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f701c003a00 0x7f70380c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.004 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.998+0000 7f703d800640 1 -- 172.21.15.132:0/3833592224 >> 172.21.15.132:0/3833592224 conn(0x7f70380d0550 msgr2=0x7f70380b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.004 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.998+0000 7f703d800640 1 -- 172.21.15.132:0/3833592224 shutdown_connections 2024-04-25T10:06:45.004 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:44.998+0000 7f703d800640 1 -- 172.21.15.132:0/3833592224 wait complete. 2024-04-25T10:06:45.020 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 36485de8-b602-4be0-80ae-e309a8f26352 4 2024-04-25T10:06:45.036 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:45.030+0000 7f84f3e96040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:06:45.101 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:45.095+0000 7f84f3e96040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:06:45.161 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:45.155+0000 7f84f3e96040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:06:45.170 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:45.165+0000 7f5c573e3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:06:45.194 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.189+0000 7f32d8c00640 1 Processor -- start 2024-04-25T10:06:45.195 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.189+0000 7f32d8c00640 1 -- start start 2024-04-25T10:06:45.196 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.190+0000 7f32d8c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f32d40b10b0 0x7f32d40b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.196 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.190+0000 7f32d8c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d40cc740 con 0x7f32d406b5b0 2024-04-25T10:06:45.196 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.191+0000 7f32d8c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d40cca20 con 0x7f32d415bb20 2024-04-25T10:06:45.196 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.191+0000 7f32d8c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d40cc510 con 0x7f32d40b10b0 2024-04-25T10:06:45.196 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.191+0000 7f32d1e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f32d415bb20 0x7f32d40ad570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:47376/0 (socket says 172.21.15.132:47376) 2024-04-25T10:06:45.196 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.191+0000 7f32d1e00640 1 -- 172.21.15.132:0/242871008 learned_addr learned my addr 172.21.15.132:0/242871008 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-04-25T10:06:45.196 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.191+0000 7f32d2800640 1 --2- 172.21.15.132:0/242871008 >> v2:172.21.15.132:3300/0 conn(0x7f32d40b10b0 0x7f32d40b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.191+0000 7f32d0a00640 1 -- 172.21.15.132:0/242871008 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2063116322 0 0) 0x7f32d40cca20 con 0x7f32d415bb20 2024-04-25T10:06:45.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.191+0000 7f32d0a00640 1 -- 172.21.15.132:0/242871008 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32b8003520 con 0x7f32d415bb20 2024-04-25T10:06:45.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.192+0000 7f32d2800640 1 -- 172.21.15.132:0/242871008 >> v1:172.21.15.132:6789/0 conn(0x7f32d415bb20 legacy=0x7f32d40ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.192+0000 7f32d2800640 1 -- 172.21.15.132:0/242871008 >> v1:172.21.15.119:6789/0 conn(0x7f32d406b5b0 legacy=0x7f32d40cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.192+0000 7f32d2800640 1 -- 172.21.15.132:0/242871008 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d40ccd00 con 0x7f32d40b10b0 2024-04-25T10:06:45.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.193+0000 7f32d2800640 1 --2- 172.21.15.132:0/242871008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32d40b10b0 0x7f32d40b1770 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009d90 tx=0x7f32cc02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d69ee8d95b1acd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.193+0000 7f32d0a00640 1 -- 172.21.15.132:0/242871008 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc007740 con 0x7f32d40b10b0 2024-04-25T10:06:45.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.193+0000 7f32d0a00640 1 -- 172.21.15.132:0/242871008 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc0078e0 con 0x7f32d40b10b0 2024-04-25T10:06:45.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.193+0000 7f32d0a00640 1 -- 172.21.15.132:0/242871008 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc007c90 con 0x7f32d40b10b0 2024-04-25T10:06:45.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.194+0000 7f32d8c00640 1 -- 172.21.15.132:0/242871008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32d40b10b0 msgr2=0x7f32d40b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.194+0000 7f32d8c00640 1 --2- 172.21.15.132:0/242871008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32d40b10b0 0x7f32d40b1770 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009d90 tx=0x7f32cc02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.194+0000 7f32d8c00640 1 -- 172.21.15.132:0/242871008 shutdown_connections 2024-04-25T10:06:45.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.194+0000 7f32d8c00640 1 --2- 172.21.15.132:0/242871008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32d40b10b0 0x7f32d40b1770 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.194+0000 7f32d8c00640 1 -- 172.21.15.132:0/242871008 >> 172.21.15.132:0/242871008 conn(0x7f32d4099a40 msgr2=0x7f32d40d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.194+0000 7f32d8c00640 1 -- 172.21.15.132:0/242871008 shutdown_connections 2024-04-25T10:06:45.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.194+0000 7f32d8c00640 1 -- 172.21.15.132:0/242871008 wait complete. 2024-04-25T10:06:45.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.195+0000 7f32d8c00640 1 Processor -- start 2024-04-25T10:06:45.201 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.195+0000 7f32d8c00640 1 -- start start 2024-04-25T10:06:45.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.196+0000 7f32d8c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d406b5b0 0x7f32d4154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.196+0000 7f32d8c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32d40b10b0 0x7f32d40ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.196+0000 7f32d8c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32d415bb20 0x7f32d40b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.196+0000 7f32d8c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d415bf00 con 0x7f32d406b5b0 2024-04-25T10:06:45.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.196+0000 7f32d8c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d40cd350 con 0x7f32d40b10b0 2024-04-25T10:06:45.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.196+0000 7f32d8c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d409a010 con 0x7f32d415bb20 2024-04-25T10:06:45.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.197+0000 7f32d1400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32d40b10b0 0x7f32d40ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.197+0000 7f32d1400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32d40b10b0 0x7f32d40ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59046/0 (socket says 172.21.15.132:59046) 2024-04-25T10:06:45.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.197+0000 7f32d1400640 1 -- 172.21.15.132:0/1065296138 learned_addr learned my addr 172.21.15.132:0/1065296138 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-25T10:06:45.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.197+0000 7f32d2800640 1 --2- 172.21.15.132:0/1065296138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32d415bb20 0x7f32d40b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.197+0000 7f32d1400640 1 -- 172.21.15.132:0/1065296138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32d415bb20 msgr2=0x7f32d40b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.197+0000 7f32d1400640 1 --2- 172.21.15.132:0/1065296138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32d415bb20 0x7f32d40b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.197+0000 7f32d1400640 1 -- 172.21.15.132:0/1065296138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d406b5b0 msgr2=0x7f32d4154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.197+0000 7f32d1400640 1 --2- 172.21.15.132:0/1065296138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d406b5b0 0x7f32d4154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.197+0000 7f32d1400640 1 -- 172.21.15.132:0/1065296138 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d40c88a0 con 0x7f32d40b10b0 2024-04-25T10:06:45.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.197+0000 7f32d1400640 1 --2- 172.21.15.132:0/1065296138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32d40b10b0 0x7f32d40ce3d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c440 tx=0x7f32c0001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.198+0000 7f32ca000640 1 -- 172.21.15.132:0/1065296138 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00052d0 con 0x7f32d40b10b0 2024-04-25T10:06:45.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.198+0000 7f32ca000640 1 -- 172.21.15.132:0/1065296138 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0005470 con 0x7f32d40b10b0 2024-04-25T10:06:45.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.198+0000 7f32ca000640 1 -- 172.21.15.132:0/1065296138 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0007260 con 0x7f32d40b10b0 2024-04-25T10:06:45.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.198+0000 7f32d8c00640 1 -- 172.21.15.132:0/1065296138 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d40c8b80 con 0x7f32d40b10b0 2024-04-25T10:06:45.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.198+0000 7f32d8c00640 1 -- 172.21.15.132:0/1065296138 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d40b45e0 con 0x7f32d40b10b0 2024-04-25T10:06:45.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.198+0000 7f32ca000640 1 -- 172.21.15.132:0/1065296138 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f32c0007400 con 0x7f32d40b10b0 2024-04-25T10:06:45.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.198+0000 7f32ca000640 1 -- 172.21.15.132:0/1065296138 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f32c00035f0 con 0x7f32d40b10b0 2024-04-25T10:06:45.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.199+0000 7f32d8c00640 1 -- 172.21.15.132:0/1065296138 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d40750b0 con 0x7f32d40b10b0 2024-04-25T10:06:45.207 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.201+0000 7f32ca000640 1 -- 172.21.15.132:0/1065296138 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f32c0003a50 con 0x7f32d40b10b0 2024-04-25T10:06:45.250 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:45.243+0000 7f5c573e3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:06:45.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.277+0000 7f32d8c00640 1 -- 172.21.15.132:0/1065296138 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "36485de8-b602-4be0-80ae-e309a8f26352", "id": 4} v 0) v1 -- 0x7f32d40d0d20 con 0x7f32d40b10b0 2024-04-25T10:06:45.288 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:45.282+0000 7f84f3e96040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:06:45.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.284+0000 7f32ca000640 1 -- 172.21.15.132:0/1065296138 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "36485de8-b602-4be0-80ae-e309a8f26352", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f32c0003c30 con 0x7f32d40b10b0 2024-04-25T10:06:45.289 INFO:teuthology.orchestra.run.smithi132.stdout:4 2024-04-25T10:06:45.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.285+0000 7f32d8c00640 1 -- 172.21.15.132:0/1065296138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32d40b10b0 msgr2=0x7f32d40ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.285+0000 7f32d8c00640 1 --2- 172.21.15.132:0/1065296138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32d40b10b0 0x7f32d40ce3d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c440 tx=0x7f32c0001e70 comp rx=0 tx=0).stop 2024-04-25T10:06:45.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.285+0000 7f32d8c00640 1 -- 172.21.15.132:0/1065296138 shutdown_connections 2024-04-25T10:06:45.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.285+0000 7f32d8c00640 1 --2- 172.21.15.132:0/1065296138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32d415bb20 0x7f32d40b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.285+0000 7f32d8c00640 1 --2- 172.21.15.132:0/1065296138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32d40b10b0 0x7f32d40ce3d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.291 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.285+0000 7f32d8c00640 1 --2- 172.21.15.132:0/1065296138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d406b5b0 0x7f32d4154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.285+0000 7f32d8c00640 1 -- 172.21.15.132:0/1065296138 >> 172.21.15.132:0/1065296138 conn(0x7f32d4099a40 msgr2=0x7f32d40b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.286+0000 7f32d8c00640 1 -- 172.21.15.132:0/1065296138 shutdown_connections 2024-04-25T10:06:45.292 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.286+0000 7f32d8c00640 1 -- 172.21.15.132:0/1065296138 wait complete. 2024-04-25T10:06:45.314 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6 5 2024-04-25T10:06:45.322 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:45.317+0000 7f5c573e3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:06:45.456 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:45.451+0000 7f5c573e3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.493+0000 7fd368e00640 1 Processor -- start 2024-04-25T10:06:45.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.494+0000 7fd368e00640 1 -- start start 2024-04-25T10:06:45.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.495+0000 7fd368e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd3640d2e20 0x7fd3640b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.495+0000 7fd368e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3640b8330 con 0x7fd36415da10 2024-04-25T10:06:45.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.495+0000 7fd368e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3640b8610 con 0x7fd3640b9530 2024-04-25T10:06:45.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.495+0000 7fd368e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36415ddf0 con 0x7fd3640d2e20 2024-04-25T10:06:45.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.495+0000 7fd35aa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd3640d2e20 0x7fd3640b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.496+0000 7fd363e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd3640b9530 0x7fd3640b6e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:47386/0 (socket says 172.21.15.132:47386) 2024-04-25T10:06:45.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.496+0000 7fd363e00640 1 -- 172.21.15.132:0/800736555 learned_addr learned my addr 172.21.15.132:0/800736555 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-04-25T10:06:45.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.496+0000 7fd35aa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd3640d2e20 0x7fd3640b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59052/0 (socket says 172.21.15.132:59052) 2024-04-25T10:06:45.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.496+0000 7fd35aa00640 1 -- 172.21.15.132:0/800736555 >> v1:172.21.15.132:6789/0 conn(0x7fd3640b9530 legacy=0x7fd3640b6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.496+0000 7fd35aa00640 1 -- 172.21.15.132:0/800736555 >> v1:172.21.15.119:6789/0 conn(0x7fd36415da10 legacy=0x7fd3640ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.496+0000 7fd35aa00640 1 -- 172.21.15.132:0/800736555 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3640b88f0 con 0x7fd3640d2e20 2024-04-25T10:06:45.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.497+0000 7fd35aa00640 1 --2- 172.21.15.132:0/800736555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3640d2e20 0x7fd3640b6920 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0097a0 tx=0x7fd34c02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f08e53485ddc867 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.497+0000 7fd362a00640 1 -- 172.21.15.132:0/800736555 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c037070 con 0x7fd3640d2e20 2024-04-25T10:06:45.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.497+0000 7fd362a00640 1 -- 172.21.15.132:0/800736555 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c032640 con 0x7fd3640d2e20 2024-04-25T10:06:45.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.497+0000 7fd362a00640 1 -- 172.21.15.132:0/800736555 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c032a10 con 0x7fd3640d2e20 2024-04-25T10:06:45.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.498+0000 7fd368e00640 1 -- 172.21.15.132:0/800736555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3640d2e20 msgr2=0x7fd3640b6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.498+0000 7fd368e00640 1 --2- 172.21.15.132:0/800736555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3640d2e20 0x7fd3640b6920 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0097a0 tx=0x7fd34c02a2c0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.498+0000 7fd368e00640 1 -- 172.21.15.132:0/800736555 shutdown_connections 2024-04-25T10:06:45.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.498+0000 7fd368e00640 1 --2- 172.21.15.132:0/800736555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3640d2e20 0x7fd3640b6920 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.498+0000 7fd368e00640 1 -- 172.21.15.132:0/800736555 >> 172.21.15.132:0/800736555 conn(0x7fd3640726c0 msgr2=0x7fd364072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.498+0000 7fd368e00640 1 -- 172.21.15.132:0/800736555 shutdown_connections 2024-04-25T10:06:45.504 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.498+0000 7fd368e00640 1 -- 172.21.15.132:0/800736555 wait complete. 2024-04-25T10:06:45.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.500+0000 7fd368e00640 1 Processor -- start 2024-04-25T10:06:45.505 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.500+0000 7fd368e00640 1 -- start start 2024-04-25T10:06:45.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd368e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3640b9530 0x7fd3640c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd368e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3640d2e20 0x7fd36414a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd368e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd36415da10 0x7fd36414a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd368e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640cec90 con 0x7fd3640d2e20 2024-04-25T10:06:45.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd368e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640d1930 con 0x7fd3640b9530 2024-04-25T10:06:45.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd368e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640cee10 con 0x7fd36415da10 2024-04-25T10:06:45.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd363e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd36415da10 0x7fd36414a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.508 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd363400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3640b9530 0x7fd3640c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd363e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd36415da10 0x7fd36414a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:33652/0 (socket says 172.21.15.132:33652) 2024-04-25T10:06:45.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd363e00640 1 -- 172.21.15.132:0/938469037 learned_addr learned my addr 172.21.15.132:0/938469037 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-25T10:06:45.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd35aa00640 1 --2- 172.21.15.132:0/938469037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3640d2e20 0x7fd36414a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd363e00640 1 -- 172.21.15.132:0/938469037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3640b9530 msgr2=0x7fd3640c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd363e00640 1 --2- 172.21.15.132:0/938469037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3640b9530 0x7fd3640c9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd363e00640 1 -- 172.21.15.132:0/938469037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3640d2e20 msgr2=0x7fd36414a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.501+0000 7fd363e00640 1 --2- 172.21.15.132:0/938469037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3640d2e20 0x7fd36414a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.502+0000 7fd363e00640 1 -- 172.21.15.132:0/938469037 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364158340 con 0x7fd36415da10 2024-04-25T10:06:45.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.502+0000 7fd363400640 1 --2- 172.21.15.132:0/938469037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3640b9530 0x7fd3640c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:45.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.502+0000 7fd363e00640 1 --2- 172.21.15.132:0/938469037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd36415da10 0x7fd36414a910 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00c420 tx=0x7fd35c00c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.502+0000 7fd35be00640 1 -- 172.21.15.132:0/938469037 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c017070 con 0x7fd36415da10 2024-04-25T10:06:45.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.502+0000 7fd35be00640 1 -- 172.21.15.132:0/938469037 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c005ce0 con 0x7fd36415da10 2024-04-25T10:06:45.509 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.502+0000 7fd35be00640 1 -- 172.21.15.132:0/938469037 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0072d0 con 0x7fd36415da10 2024-04-25T10:06:45.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.502+0000 7fd368e00640 1 -- 172.21.15.132:0/938469037 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd364160490 con 0x7fd36415da10 2024-04-25T10:06:45.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.502+0000 7fd368e00640 1 -- 172.21.15.132:0/938469037 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3640c92c0 con 0x7fd36415da10 2024-04-25T10:06:45.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.502+0000 7fd35be00640 1 -- 172.21.15.132:0/938469037 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd35c01a070 con 0x7fd36415da10 2024-04-25T10:06:45.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.503+0000 7fd35be00640 1 -- 172.21.15.132:0/938469037 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fd35c01cc10 con 0x7fd36415da10 2024-04-25T10:06:45.510 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.504+0000 7fd368e00640 1 -- 172.21.15.132:0/938469037 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3640749b0 con 0x7fd36415da10 2024-04-25T10:06:45.511 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.506+0000 7fd35be00640 1 -- 172.21.15.132:0/938469037 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd35c007470 con 0x7fd36415da10 2024-04-25T10:06:45.590 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.584+0000 7fd368e00640 1 -- 172.21.15.132:0/938469037 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6", "id": 5} v 0) v1 -- 0x7fd3640ce6d0 con 0x7fd36415da10 2024-04-25T10:06:45.597 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.591+0000 7fd35be00640 1 -- 172.21.15.132:0/938469037 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd35c007650 con 0x7fd36415da10 2024-04-25T10:06:45.597 INFO:teuthology.orchestra.run.smithi132.stdout:5 2024-04-25T10:06:45.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.592+0000 7fd368e00640 1 -- 172.21.15.132:0/938469037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd36415da10 msgr2=0x7fd36414a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.592+0000 7fd368e00640 1 --2- 172.21.15.132:0/938469037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd36415da10 0x7fd36414a910 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00c420 tx=0x7fd35c00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.592+0000 7fd368e00640 1 -- 172.21.15.132:0/938469037 shutdown_connections 2024-04-25T10:06:45.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.592+0000 7fd368e00640 1 --2- 172.21.15.132:0/938469037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd36415da10 0x7fd36414a910 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.592+0000 7fd368e00640 1 --2- 172.21.15.132:0/938469037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3640d2e20 0x7fd36414a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.599 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.592+0000 7fd368e00640 1 --2- 172.21.15.132:0/938469037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3640b9530 0x7fd3640c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.592+0000 7fd368e00640 1 -- 172.21.15.132:0/938469037 >> 172.21.15.132:0/938469037 conn(0x7fd3640726c0 msgr2=0x7fd36415c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.592+0000 7fd368e00640 1 -- 172.21.15.132:0/938469037 shutdown_connections 2024-04-25T10:06:45.600 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.593+0000 7fd368e00640 1 -- 172.21.15.132:0/938469037 wait complete. 2024-04-25T10:06:45.615 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new c275df55-3aca-4426-8c41-79da30ff0909 6 2024-04-25T10:06:45.739 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:45.731+0000 7f84f3e96040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:06:45.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.835+0000 7f7abac00640 1 Processor -- start 2024-04-25T10:06:45.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.835+0000 7f7abac00640 1 -- start start 2024-04-25T10:06:45.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.837+0000 7f7abac00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ab415da10 0x7f7ab40ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.837+0000 7f7abac00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7ab40b9530 0x7f7ab40ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.837+0000 7f7abac00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab40b8330 con 0x7f7ab40d1e10 2024-04-25T10:06:45.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.837+0000 7f7abac00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab415ddf0 con 0x7f7ab415da10 2024-04-25T10:06:45.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.837+0000 7f7abac00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab40b8610 con 0x7f7ab40b9530 2024-04-25T10:06:45.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.837+0000 7f7ab8e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7ab40b9530 0x7f7ab40ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.837+0000 7f7ab8e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7ab40b9530 0x7f7ab40ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59066/0 (socket says 172.21.15.132:59066) 2024-04-25T10:06:45.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.837+0000 7f7ab8e00640 1 -- 172.21.15.132:0/3412191644 learned_addr learned my addr 172.21.15.132:0/3412191644 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-25T10:06:45.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.837+0000 7f7ab9800640 1 --2- 172.21.15.132:0/3412191644 >> v2:172.21.15.119:3300/0 conn(0x7f7ab415da10 0x7f7ab40ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.848 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.837+0000 7f7ab8e00640 1 -- 172.21.15.132:0/3412191644 >> v2:172.21.15.119:3300/0 conn(0x7f7ab415da10 msgr2=0x7f7ab40ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.848 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.837+0000 7f7ab8e00640 1 --2- 172.21.15.132:0/3412191644 >> v2:172.21.15.119:3300/0 conn(0x7f7ab415da10 0x7f7ab40ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.848 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.837+0000 7f7ab8e00640 1 -- 172.21.15.132:0/3412191644 >> v1:172.21.15.119:6789/0 conn(0x7f7ab40d1e10 legacy=0x7f7ab40d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.848 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.837+0000 7f7ab8e00640 1 -- 172.21.15.132:0/3412191644 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab40b88f0 con 0x7f7ab40b9530 2024-04-25T10:06:45.848 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.838+0000 7f7ab8e00640 1 --2- 172.21.15.132:0/3412191644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab40b9530 0x7f7ab40ceac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40d49b0 tx=0x7f7aa002a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e9d3cf7cfa8f5ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.848 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.839+0000 7f7ab3e00640 1 -- 172.21.15.132:0/3412191644 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0037070 con 0x7f7ab40b9530 2024-04-25T10:06:45.849 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.839+0000 7f7ab3e00640 1 -- 172.21.15.132:0/3412191644 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa0032440 con 0x7f7ab40b9530 2024-04-25T10:06:45.849 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.839+0000 7f7ab3e00640 1 -- 172.21.15.132:0/3412191644 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0032810 con 0x7f7ab40b9530 2024-04-25T10:06:45.849 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.839+0000 7f7abac00640 1 -- 172.21.15.132:0/3412191644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab40b9530 msgr2=0x7f7ab40ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.839+0000 7f7abac00640 1 --2- 172.21.15.132:0/3412191644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab40b9530 0x7f7ab40ceac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40d49b0 tx=0x7f7aa002a2c0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.840+0000 7f7abac00640 1 -- 172.21.15.132:0/3412191644 shutdown_connections 2024-04-25T10:06:45.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.840+0000 7f7abac00640 1 --2- 172.21.15.132:0/3412191644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab40b9530 0x7f7ab40ceac0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.840+0000 7f7abac00640 1 --2- 172.21.15.132:0/3412191644 >> v2:172.21.15.119:3300/0 conn(0x7f7ab415da10 0x7f7ab40ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.840+0000 7f7abac00640 1 -- 172.21.15.132:0/3412191644 >> 172.21.15.132:0/3412191644 conn(0x7f7ab40726c0 msgr2=0x7f7ab4072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.840+0000 7f7abac00640 1 -- 172.21.15.132:0/3412191644 shutdown_connections 2024-04-25T10:06:45.850 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.840+0000 7f7abac00640 1 -- 172.21.15.132:0/3412191644 wait complete. 2024-04-25T10:06:45.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.841+0000 7f7abac00640 1 Processor -- start 2024-04-25T10:06:45.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.841+0000 7f7abac00640 1 -- start start 2024-04-25T10:06:45.855 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.842+0000 7f7abac00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab40b9530 0x7f7ab414c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.842+0000 7f7abac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab40d1e10 0x7f7ab40c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.842+0000 7f7abac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ab415da10 0x7f7ab40d3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.842+0000 7f7abac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab40b7930 con 0x7f7ab40d1e10 2024-04-25T10:06:45.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.842+0000 7f7abac00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab40d18f0 con 0x7f7ab40b9530 2024-04-25T10:06:45.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.842+0000 7f7abac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab40b7ab0 con 0x7f7ab415da10 2024-04-25T10:06:45.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.843+0000 7f7ab9800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab40b9530 0x7f7ab414c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.843+0000 7f7ab9800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab40b9530 0x7f7ab414c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59072/0 (socket says 172.21.15.132:59072) 2024-04-25T10:06:45.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.843+0000 7f7aba200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ab415da10 0x7f7ab40d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.856 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.843+0000 7f7ab9800640 1 -- 172.21.15.132:0/3244799566 learned_addr learned my addr 172.21.15.132:0/3244799566 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-25T10:06:45.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.843+0000 7f7ab8e00640 1 --2- 172.21.15.132:0/3244799566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab40d1e10 0x7f7ab40c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.843+0000 7f7ab9800640 1 -- 172.21.15.132:0/3244799566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ab415da10 msgr2=0x7f7ab40d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.843+0000 7f7ab9800640 1 --2- 172.21.15.132:0/3244799566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ab415da10 0x7f7ab40d3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.843+0000 7f7ab9800640 1 -- 172.21.15.132:0/3244799566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab40d1e10 msgr2=0x7f7ab40c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.843+0000 7f7ab9800640 1 --2- 172.21.15.132:0/3244799566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab40d1e10 0x7f7ab40c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.857 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.843+0000 7f7ab9800640 1 -- 172.21.15.132:0/3244799566 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab40c7920 con 0x7f7ab40b9530 2024-04-25T10:06:45.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.843+0000 7f7ab8e00640 1 --2- 172.21.15.132:0/3244799566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab40d1e10 0x7f7ab40c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:45.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.844+0000 7f7ab9800640 1 --2- 172.21.15.132:0/3244799566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab40b9530 0x7f7ab414c890 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8007e40 tx=0x7f7aa800e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.858 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.844+0000 7f7ab1600640 1 -- 172.21.15.132:0/3244799566 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8014620 con 0x7f7ab40b9530 2024-04-25T10:06:45.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.844+0000 7f7ab1600640 1 -- 172.21.15.132:0/3244799566 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa8019070 con 0x7f7ab40b9530 2024-04-25T10:06:45.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.844+0000 7f7abac00640 1 -- 172.21.15.132:0/3244799566 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab40cbe40 con 0x7f7ab40b9530 2024-04-25T10:06:45.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.844+0000 7f7ab1600640 1 -- 172.21.15.132:0/3244799566 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8014a00 con 0x7f7ab40b9530 2024-04-25T10:06:45.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.844+0000 7f7abac00640 1 -- 172.21.15.132:0/3244799566 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab40cc0e0 con 0x7f7ab40b9530 2024-04-25T10:06:45.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.844+0000 7f7ab1600640 1 -- 172.21.15.132:0/3244799566 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7aa8014ba0 con 0x7f7ab40b9530 2024-04-25T10:06:45.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.844+0000 7f7ab1600640 1 -- 172.21.15.132:0/3244799566 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f7aa801d3c0 con 0x7f7ab40b9530 2024-04-25T10:06:45.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.845+0000 7f7abac00640 1 -- 172.21.15.132:0/3244799566 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab40749b0 con 0x7f7ab40b9530 2024-04-25T10:06:45.860 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.847+0000 7f7ab1600640 1 -- 172.21.15.132:0/3244799566 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7aa801e460 con 0x7f7ab40b9530 2024-04-25T10:06:45.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.927+0000 7f7abac00640 1 -- 172.21.15.132:0/3244799566 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c275df55-3aca-4426-8c41-79da30ff0909", "id": 6} v 0) v1 -- 0x7f7ab414d370 con 0x7f7ab40b9530 2024-04-25T10:06:45.938 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:45.932+0000 7f5c573e3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:06:45.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.935+0000 7f7ab1600640 1 -- 172.21.15.132:0/3244799566 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c275df55-3aca-4426-8c41-79da30ff0909", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7aa801e640 con 0x7f7ab40b9530 2024-04-25T10:06:45.941 INFO:teuthology.orchestra.run.smithi132.stdout:6 2024-04-25T10:06:45.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.936+0000 7f7abac00640 1 -- 172.21.15.132:0/3244799566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab40b9530 msgr2=0x7f7ab414c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.936+0000 7f7abac00640 1 --2- 172.21.15.132:0/3244799566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab40b9530 0x7f7ab414c890 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8007e40 tx=0x7f7aa800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.937+0000 7f7abac00640 1 -- 172.21.15.132:0/3244799566 shutdown_connections 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.937+0000 7f7abac00640 1 --2- 172.21.15.132:0/3244799566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ab415da10 0x7f7ab40d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.937+0000 7f7abac00640 1 --2- 172.21.15.132:0/3244799566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab40d1e10 0x7f7ab40c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.937+0000 7f7abac00640 1 --2- 172.21.15.132:0/3244799566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab40b9530 0x7f7ab414c890 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.937+0000 7f7abac00640 1 -- 172.21.15.132:0/3244799566 >> 172.21.15.132:0/3244799566 conn(0x7f7ab40726c0 msgr2=0x7f7ab40b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.937+0000 7f7abac00640 1 -- 172.21.15.132:0/3244799566 shutdown_connections 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:45.937+0000 7f7abac00640 1 -- 172.21.15.132:0/3244799566 wait complete. 2024-04-25T10:06:45.959 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 37d1ddce-fbb1-498b-aece-09e4831bd340 7 2024-04-25T10:06:46.027 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:46.021+0000 7f84f3e96040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:06:46.086 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:46.081+0000 7f84f3e96040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:06:46.143 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.138+0000 7faa6dc00640 1 Processor -- start 2024-04-25T10:06:46.145 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.138+0000 7faa6dc00640 1 -- start start 2024-04-25T10:06:46.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.139+0000 7faa6dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa6815da10 0x7faa680ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.139+0000 7faa6dc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faa680b9530 0x7faa680ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.140+0000 7faa6dc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa680b8330 con 0x7faa680d1e10 2024-04-25T10:06:46.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.140+0000 7faa6dc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6815ddf0 con 0x7faa680b9530 2024-04-25T10:06:46.146 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.140+0000 7faa6dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa680b8610 con 0x7faa6815da10 2024-04-25T10:06:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.140+0000 7faa67e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa6815da10 0x7faa680ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.140+0000 7faa67e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa6815da10 0x7faa680ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:33670/0 (socket says 172.21.15.132:33670) 2024-04-25T10:06:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.140+0000 7faa67e00640 1 -- 172.21.15.132:0/2104501071 learned_addr learned my addr 172.21.15.132:0/2104501071 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-25T10:06:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.140+0000 7faa67400640 1 --2- 172.21.15.132:0/2104501071 >> v2:172.21.15.132:3300/0 conn(0x7faa680b9530 0x7faa680ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.140+0000 7faa6d200640 1 --1- 172.21.15.132:0/2104501071 >> v1:172.21.15.132:6789/0 conn(0x7faa680d1e10 0x7faa680d22a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:47396/0 (socket says 172.21.15.132:47396) 2024-04-25T10:06:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.140+0000 7faa67e00640 1 -- 172.21.15.132:0/2104501071 >> v2:172.21.15.132:3300/0 conn(0x7faa680b9530 msgr2=0x7faa680ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.140+0000 7faa67e00640 1 --2- 172.21.15.132:0/2104501071 >> v2:172.21.15.132:3300/0 conn(0x7faa680b9530 0x7faa680ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.140+0000 7faa67e00640 1 -- 172.21.15.132:0/2104501071 >> v1:172.21.15.132:6789/0 conn(0x7faa680d1e10 legacy=0x7faa680d22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.140+0000 7faa67e00640 1 -- 172.21.15.132:0/2104501071 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa680b88f0 con 0x7faa6815da10 2024-04-25T10:06:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.141+0000 7faa67e00640 1 --2- 172.21.15.132:0/2104501071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa6815da10 0x7faa680ce580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faa58009870 tx=0x7faa5802a2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e81ca8f642c27955 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.142+0000 7faa66a00640 1 -- 172.21.15.132:0/2104501071 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58037070 con 0x7faa6815da10 2024-04-25T10:06:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.142+0000 7faa66a00640 1 -- 172.21.15.132:0/2104501071 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa58032640 con 0x7faa6815da10 2024-04-25T10:06:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.142+0000 7faa66a00640 1 -- 172.21.15.132:0/2104501071 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58032a10 con 0x7faa6815da10 2024-04-25T10:06:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.142+0000 7faa6dc00640 1 -- 172.21.15.132:0/2104501071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa6815da10 msgr2=0x7faa680ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.142+0000 7faa6dc00640 1 --2- 172.21.15.132:0/2104501071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa6815da10 0x7faa680ce580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faa58009870 tx=0x7faa5802a2f0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.142+0000 7faa6dc00640 1 -- 172.21.15.132:0/2104501071 shutdown_connections 2024-04-25T10:06:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.142+0000 7faa6dc00640 1 --2- 172.21.15.132:0/2104501071 >> v2:172.21.15.132:3300/0 conn(0x7faa680b9530 0x7faa680ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.142+0000 7faa6dc00640 1 --2- 172.21.15.132:0/2104501071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa6815da10 0x7faa680ce580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.142+0000 7faa6dc00640 1 -- 172.21.15.132:0/2104501071 >> 172.21.15.132:0/2104501071 conn(0x7faa680726c0 msgr2=0x7faa68072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.143+0000 7faa6dc00640 1 -- 172.21.15.132:0/2104501071 shutdown_connections 2024-04-25T10:06:46.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.143+0000 7faa6dc00640 1 -- 172.21.15.132:0/2104501071 wait complete. 2024-04-25T10:06:46.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.144+0000 7faa6dc00640 1 Processor -- start 2024-04-25T10:06:46.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.144+0000 7faa6dc00640 1 -- start start 2024-04-25T10:06:46.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.145+0000 7faa6dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa680b9530 0x7faa680cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.145+0000 7faa6dc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa680d1e10 0x7faa681f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.145+0000 7faa6dc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa6815da10 0x7faa681f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.145+0000 7faa6dc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa680d29b0 con 0x7faa6815da10 2024-04-25T10:06:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.145+0000 7faa6dc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa680d18f0 con 0x7faa680d1e10 2024-04-25T10:06:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.145+0000 7faa6dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa680d2b30 con 0x7faa680b9530 2024-04-25T10:06:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.145+0000 7faa67e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa680b9530 0x7faa680cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.146+0000 7faa67400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa680d1e10 0x7faa681f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.146+0000 7faa67e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa680b9530 0x7faa680cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:33682/0 (socket says 172.21.15.132:33682) 2024-04-25T10:06:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.146+0000 7faa67400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa680d1e10 0x7faa681f3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:59094/0 (socket says 172.21.15.132:59094) 2024-04-25T10:06:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.146+0000 7faa67e00640 1 -- 172.21.15.132:0/2650581062 learned_addr learned my addr 172.21.15.132:0/2650581062 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-25T10:06:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.146+0000 7faa67e00640 1 -- 172.21.15.132:0/2650581062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa680d1e10 msgr2=0x7faa681f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.146+0000 7faa67e00640 1 --2- 172.21.15.132:0/2650581062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa680d1e10 0x7faa681f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.146+0000 7faa67e00640 1 -- 172.21.15.132:0/2650581062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa6815da10 msgr2=0x7faa681f3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.146+0000 7faa67e00640 1 --2- 172.21.15.132:0/2650581062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa6815da10 0x7faa681f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.146+0000 7faa67e00640 1 -- 172.21.15.132:0/2650581062 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa680cd7b0 con 0x7faa680b9530 2024-04-25T10:06:46.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.146+0000 7faa67e00640 1 --2- 172.21.15.132:0/2650581062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa680b9530 0x7faa680cce00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faa5802a330 tx=0x7faa580046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.147+0000 7faa5fe00640 1 -- 172.21.15.132:0/2650581062 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58037040 con 0x7faa680b9530 2024-04-25T10:06:46.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.147+0000 7faa5fe00640 1 -- 172.21.15.132:0/2650581062 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa58042070 con 0x7faa680b9530 2024-04-25T10:06:46.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.147+0000 7faa5fe00640 1 -- 172.21.15.132:0/2650581062 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa580309b0 con 0x7faa680b9530 2024-04-25T10:06:46.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.147+0000 7faa6dc00640 1 -- 172.21.15.132:0/2650581062 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa680cda90 con 0x7faa680b9530 2024-04-25T10:06:46.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.147+0000 7faa6dc00640 1 -- 172.21.15.132:0/2650581062 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa680c87b0 con 0x7faa680b9530 2024-04-25T10:06:46.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.147+0000 7faa5fe00640 1 -- 172.21.15.132:0/2650581062 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7faa58030b50 con 0x7faa680b9530 2024-04-25T10:06:46.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.147+0000 7faa5fe00640 1 -- 172.21.15.132:0/2650581062 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7faa58047be0 con 0x7faa680b9530 2024-04-25T10:06:46.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.148+0000 7faa6dc00640 1 -- 172.21.15.132:0/2650581062 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa680749b0 con 0x7faa680b9530 2024-04-25T10:06:46.155 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.150+0000 7faa5fe00640 1 -- 172.21.15.132:0/2650581062 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7faa58047460 con 0x7faa680b9530 2024-04-25T10:06:46.215 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:46.209+0000 7f84f3e96040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:06:46.234 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.228+0000 7faa6dc00640 1 -- 172.21.15.132:0/2650581062 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "37d1ddce-fbb1-498b-aece-09e4831bd340", "id": 7} v 0) v1 -- 0x7faa680d2490 con 0x7faa680b9530 2024-04-25T10:06:46.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.240+0000 7faa5fe00640 1 -- 172.21.15.132:0/2650581062 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "37d1ddce-fbb1-498b-aece-09e4831bd340", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7faa58047640 con 0x7faa680b9530 2024-04-25T10:06:46.246 INFO:teuthology.orchestra.run.smithi132.stdout:7 2024-04-25T10:06:46.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.241+0000 7faa6dc00640 1 -- 172.21.15.132:0/2650581062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa680b9530 msgr2=0x7faa680cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.241+0000 7faa6dc00640 1 --2- 172.21.15.132:0/2650581062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa680b9530 0x7faa680cce00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faa5802a330 tx=0x7faa580046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.241+0000 7faa6dc00640 1 -- 172.21.15.132:0/2650581062 shutdown_connections 2024-04-25T10:06:46.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.241+0000 7faa6dc00640 1 --2- 172.21.15.132:0/2650581062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa6815da10 0x7faa681f3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.241+0000 7faa6dc00640 1 --2- 172.21.15.132:0/2650581062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa680d1e10 0x7faa681f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.241+0000 7faa6dc00640 1 --2- 172.21.15.132:0/2650581062 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa680b9530 0x7faa680cce00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.247 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.241+0000 7faa6dc00640 1 -- 172.21.15.132:0/2650581062 >> 172.21.15.132:0/2650581062 conn(0x7faa680726c0 msgr2=0x7faa680d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.242+0000 7faa6dc00640 1 -- 172.21.15.132:0/2650581062 shutdown_connections 2024-04-25T10:06:46.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-25T10:06:46.242+0000 7faa6dc00640 1 -- 172.21.15.132:0/2650581062 wait complete. 2024-04-25T10:06:46.248 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:46.243+0000 7f5c573e3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:06:46.264 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T10:06:46.264 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T10:06:46.272 INFO:tasks.ceph.osd.0:Started 2024-04-25T10:06:46.272 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T10:06:46.272 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T10:06:46.279 INFO:tasks.ceph.osd.1:Started 2024-04-25T10:06:46.279 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T10:06:46.279 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T10:06:46.286 INFO:tasks.ceph.osd.2:Started 2024-04-25T10:06:46.286 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T10:06:46.286 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T10:06:46.291 INFO:tasks.ceph.osd.3:Started 2024-04-25T10:06:46.291 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T10:06:46.291 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T10:06:46.298 INFO:tasks.ceph.osd.4:Started 2024-04-25T10:06:46.298 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T10:06:46.298 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T10:06:46.305 INFO:tasks.ceph.osd.5:Started 2024-04-25T10:06:46.305 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T10:06:46.305 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T10:06:46.310 INFO:tasks.ceph.osd.6:Started 2024-04-25T10:06:46.310 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T10:06:46.310 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T10:06:46.314 INFO:tasks.ceph.osd.7:Started 2024-04-25T10:06:46.315 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:46.315 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:46.309+0000 7f5c573e3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:06:46.474 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:46.468+0000 7f5c573e3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:06:46.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.508+0000 7fa9a2000640 1 Processor -- start 2024-04-25T10:06:46.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.508+0000 7fa9a2000640 1 -- start start 2024-04-25T10:06:46.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.509+0000 7fa9a2000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa99c0d35b0 0x7fa99c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.509+0000 7fa9a2000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99c15e0c0 con 0x7fa99c15dce0 2024-04-25T10:06:46.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.509+0000 7fa9a2000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99c0d2510 con 0x7fa99c0b5990 2024-04-25T10:06:46.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.509+0000 7fa9a2000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa99c0d27f0 con 0x7fa99c0d35b0 2024-04-25T10:06:46.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.510+0000 7fa99b400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa99c15dce0 0x7fa99c0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33314/0 (socket says 172.21.15.119:33314) 2024-04-25T10:06:46.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.510+0000 7fa99b400640 1 -- 172.21.15.119:0/1995398472 learned_addr learned my addr 172.21.15.119:0/1995398472 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:06:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.510+0000 7fa99aa00640 1 --2- 172.21.15.119:0/1995398472 >> v2:172.21.15.132:3300/0 conn(0x7fa99c0d35b0 0x7fa99c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.510+0000 7fa99aa00640 1 -- 172.21.15.119:0/1995398472 >> v1:172.21.15.132:6789/0 conn(0x7fa99c0b5990 legacy=0x7fa99c0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.510+0000 7fa99aa00640 1 -- 172.21.15.119:0/1995398472 >> v1:172.21.15.119:6789/0 conn(0x7fa99c15dce0 legacy=0x7fa99c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.510+0000 7fa99aa00640 1 -- 172.21.15.119:0/1995398472 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c0d2ad0 con 0x7fa99c0d35b0 2024-04-25T10:06:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.510+0000 7fa99a000640 1 -- 172.21.15.119:0/1995398472 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348609770 0 0) 0x7fa99c15e0c0 con 0x7fa99c15dce0 2024-04-25T10:06:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.511+0000 7fa99aa00640 1 --2- 172.21.15.119:0/1995398472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa99c0d35b0 0x7fa99c0b5450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa9880097a0 tx=0x7fa98802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0d143b2b224aba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.511+0000 7fa99a000640 1 -- 172.21.15.119:0/1995398472 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988037070 con 0x7fa99c0d35b0 2024-04-25T10:06:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.511+0000 7fa99a000640 1 -- 172.21.15.119:0/1995398472 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa988032440 con 0x7fa99c0d35b0 2024-04-25T10:06:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.511+0000 7fa99a000640 1 -- 172.21.15.119:0/1995398472 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988032810 con 0x7fa99c0d35b0 2024-04-25T10:06:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.512+0000 7fa9a2000640 1 -- 172.21.15.119:0/1995398472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa99c0d35b0 msgr2=0x7fa99c0b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.512+0000 7fa9a2000640 1 --2- 172.21.15.119:0/1995398472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa99c0d35b0 0x7fa99c0b5450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa9880097a0 tx=0x7fa98802a230 comp rx=0 tx=0).stop 2024-04-25T10:06:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.512+0000 7fa9a2000640 1 -- 172.21.15.119:0/1995398472 shutdown_connections 2024-04-25T10:06:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.512+0000 7fa9a2000640 1 --2- 172.21.15.119:0/1995398472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa99c0d35b0 0x7fa99c0b5450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.512+0000 7fa9a2000640 1 -- 172.21.15.119:0/1995398472 >> 172.21.15.119:0/1995398472 conn(0x7fa99c072900 msgr2=0x7fa99c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.512+0000 7fa9a2000640 1 -- 172.21.15.119:0/1995398472 shutdown_connections 2024-04-25T10:06:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.512+0000 7fa9a2000640 1 -- 172.21.15.119:0/1995398472 wait complete. 2024-04-25T10:06:46.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.514+0000 7fa9a2000640 1 Processor -- start 2024-04-25T10:06:46.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.514+0000 7fa9a2000640 1 -- start start 2024-04-25T10:06:46.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.515+0000 7fa9a2000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa99c0b5990 0x7fa99c159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.515+0000 7fa9a2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa99c0d35b0 0x7fa99c157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.515+0000 7fa9a2000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa99c15dce0 0x7fa99c0b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.515+0000 7fa9a2000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c0d2160 con 0x7fa99c0d35b0 2024-04-25T10:06:46.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.515+0000 7fa9a2000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c0d3090 con 0x7fa99c0b5990 2024-04-25T10:06:46.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.515+0000 7fa9a2000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c157ec0 con 0x7fa99c15dce0 2024-04-25T10:06:46.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.515+0000 7fa99b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa99c0b5990 0x7fa99c159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.515+0000 7fa99aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa99c0d35b0 0x7fa99c157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.515+0000 7fa99b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa99c0b5990 0x7fa99c159190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:58274/0 (socket says 172.21.15.119:58274) 2024-04-25T10:06:46.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.515+0000 7fa99b400640 1 -- 172.21.15.119:0/1165201207 learned_addr learned my addr 172.21.15.119:0/1165201207 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:06:46.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.515+0000 7fa99be00640 1 --2- 172.21.15.119:0/1165201207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa99c15dce0 0x7fa99c0b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.516+0000 7fa99b400640 1 -- 172.21.15.119:0/1165201207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa99c15dce0 msgr2=0x7fa99c0b6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.516+0000 7fa99b400640 1 --2- 172.21.15.119:0/1165201207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa99c15dce0 0x7fa99c0b6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.516+0000 7fa99b400640 1 -- 172.21.15.119:0/1165201207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa99c0d35b0 msgr2=0x7fa99c157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.516+0000 7fa99b400640 1 --2- 172.21.15.119:0/1165201207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa99c0d35b0 0x7fa99c157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.516+0000 7fa99b400640 1 -- 172.21.15.119:0/1165201207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c1580f0 con 0x7fa99c0b5990 2024-04-25T10:06:46.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.516+0000 7fa99b400640 1 --2- 172.21.15.119:0/1165201207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa99c0b5990 0x7fa99c159190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00c3f0 tx=0x7fa98c00c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.516+0000 7fa993400640 1 -- 172.21.15.119:0/1165201207 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0074e0 con 0x7fa99c0b5990 2024-04-25T10:06:46.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.516+0000 7fa993400640 1 -- 172.21.15.119:0/1165201207 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c0047e0 con 0x7fa99c0b5990 2024-04-25T10:06:46.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.516+0000 7fa9a2000640 1 -- 172.21.15.119:0/1165201207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99c1583d0 con 0x7fa99c0b5990 2024-04-25T10:06:46.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.516+0000 7fa993400640 1 -- 172.21.15.119:0/1165201207 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c004c10 con 0x7fa99c0b5990 2024-04-25T10:06:46.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.516+0000 7fa9a2000640 1 -- 172.21.15.119:0/1165201207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99c158990 con 0x7fa99c0b5990 2024-04-25T10:06:46.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.516+0000 7fa993400640 1 -- 172.21.15.119:0/1165201207 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa98c011070 con 0x7fa99c0b5990 2024-04-25T10:06:46.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.517+0000 7fa993400640 1 -- 172.21.15.119:0/1165201207 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fa98c013a90 con 0x7fa99c0b5990 2024-04-25T10:06:46.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.517+0000 7fa9a2000640 1 -- 172.21.15.119:0/1165201207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99c0749b0 con 0x7fa99c0b5990 2024-04-25T10:06:46.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.520+0000 7fa993400640 1 -- 172.21.15.119:0/1165201207 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa98c004db0 con 0x7fa99c0b5990 2024-04-25T10:06:46.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.584+0000 7fa9a2000640 1 -- 172.21.15.119:0/1165201207 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa99c0b4e90 con 0x7fa99c0b5990 2024-04-25T10:06:46.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.585+0000 7fa993400640 1 -- 172.21.15.119:0/1165201207 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fa98c01ee10 con 0x7fa99c0b5990 2024-04-25T10:06:46.591 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:06:46.591 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":10,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:06:46.230936+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:46.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.587+0000 7fa9a2000640 1 -- 172.21.15.119:0/1165201207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa99c0b5990 msgr2=0x7fa99c159190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.587+0000 7fa9a2000640 1 --2- 172.21.15.119:0/1165201207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa99c0b5990 0x7fa99c159190 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00c3f0 tx=0x7fa98c00c8b0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.587+0000 7fa9a2000640 1 -- 172.21.15.119:0/1165201207 shutdown_connections 2024-04-25T10:06:46.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.587+0000 7fa9a2000640 1 --2- 172.21.15.119:0/1165201207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa99c15dce0 0x7fa99c0b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.587+0000 7fa9a2000640 1 --2- 172.21.15.119:0/1165201207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa99c0d35b0 0x7fa99c157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.587+0000 7fa9a2000640 1 --2- 172.21.15.119:0/1165201207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa99c0b5990 0x7fa99c159190 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.587+0000 7fa9a2000640 1 -- 172.21.15.119:0/1165201207 >> 172.21.15.119:0/1165201207 conn(0x7fa99c072900 msgr2=0x7fa99c0b8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.587+0000 7fa9a2000640 1 -- 172.21.15.119:0/1165201207 shutdown_connections 2024-04-25T10:06:46.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:06:46.587+0000 7fa9a2000640 1 -- 172.21.15.119:0/1165201207 wait complete. 2024-04-25T10:06:46.610 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T10:06:46.610 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T10:06:46.612 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:46.607+0000 7f84f3e96040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:06:46.669 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:46.663+0000 7f84f3e96040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:06:46.801 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:46.796+0000 7f5c573e3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:06:46.852 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:46.846+0000 7f84f3e96040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:06:46.858 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:46.853+0000 7f5c573e3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:06:46.971 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-04-25T10:06:46.965+0000 7f2aa0b33640 -1 Falling back to public interface 2024-04-25T10:06:46.982 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-04-25T10:06:46.976+0000 7ff69ba86640 -1 Falling back to public interface 2024-04-25T10:06:46.994 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-04-25T10:06:46.989+0000 7fa92521c640 -1 Falling back to public interface 2024-04-25T10:06:47.005 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-04-25T10:06:47.000+0000 7fa138086640 -1 Falling back to public interface 2024-04-25T10:06:47.006 INFO:tasks.ceph.osd.6.smithi132.stderr:2024-04-25T10:06:47.001+0000 7f9737faf640 -1 Falling back to public interface 2024-04-25T10:06:47.012 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-04-25T10:06:47.006+0000 7f1e39e0d640 -1 Falling back to public interface 2024-04-25T10:06:47.016 INFO:tasks.ceph.osd.7.smithi132.stderr:2024-04-25T10:06:47.011+0000 7fa1bc615640 -1 Falling back to public interface 2024-04-25T10:06:47.019 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-04-25T10:06:47.014+0000 7f7714e10640 -1 Falling back to public interface 2024-04-25T10:06:47.026 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:47.021+0000 7f84f3e96040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:06:47.051 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:47.046+0000 7f5c573e3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:06:47.086 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:47.080+0000 7f84f3e96040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:06:47.166 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:47.159+0000 7f84f3e96040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:06:47.217 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:47.211+0000 7f84f3e96040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:06:47.217 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:47.211+0000 7f5c573e3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:06:47.275 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:47.270+0000 7f5c573e3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:06:47.358 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:47.353+0000 7f5c573e3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:06:47.411 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:47.406+0000 7f5c573e3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:06:47.415 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:47.568 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.562+0000 7fb291200640 1 Processor -- start 2024-04-25T10:06:47.568 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.562+0000 7fb291200640 1 -- start start 2024-04-25T10:06:47.570 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.563+0000 7fb291200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb28c15de50 0x7fb28c0d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.570 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.564+0000 7fb291200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb28c0d0970 0x7fb28c0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.570 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.564+0000 7fb291200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb28c0d2bf0 con 0x7fb28c0d23c0 2024-04-25T10:06:47.570 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.564+0000 7fb291200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb28c0d0dd0 con 0x7fb28c0d0970 2024-04-25T10:06:47.570 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.564+0000 7fb291200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c0d3800 con 0x7fb28c15de50 2024-04-25T10:06:47.570 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.564+0000 7fb28a800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb28c0d23c0 0x7fb28c0ce450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33320/0 (socket says 172.21.15.119:33320) 2024-04-25T10:06:47.570 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.564+0000 7fb28a800640 1 -- 172.21.15.119:0/86008453 learned_addr learned my addr 172.21.15.119:0/86008453 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:06:47.571 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.564+0000 7fb289e00640 1 --2- 172.21.15.119:0/86008453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb28c15de50 0x7fb28c0d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.571 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.564+0000 7fb289400640 1 --2- 172.21.15.119:0/86008453 >> v2:172.21.15.132:3300/0 conn(0x7fb28c0d0970 0x7fb28c0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.571 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.564+0000 7fb288a00640 1 -- 172.21.15.119:0/86008453 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107201611 0 0) 0x7fb28c0d2bf0 con 0x7fb28c0d23c0 2024-04-25T10:06:47.571 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.564+0000 7fb288a00640 1 -- 172.21.15.119:0/86008453 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb270003520 con 0x7fb28c0d23c0 2024-04-25T10:06:47.571 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.564+0000 7fb289e00640 1 -- 172.21.15.119:0/86008453 >> v2:172.21.15.132:3300/0 conn(0x7fb28c0d0970 msgr2=0x7fb28c0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.571 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.564+0000 7fb289e00640 1 --2- 172.21.15.119:0/86008453 >> v2:172.21.15.132:3300/0 conn(0x7fb28c0d0970 0x7fb28c0d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.571 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.565+0000 7fb289e00640 1 -- 172.21.15.119:0/86008453 >> v1:172.21.15.119:6789/0 conn(0x7fb28c0d23c0 legacy=0x7fb28c0ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.571 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.565+0000 7fb289e00640 1 -- 172.21.15.119:0/86008453 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28c0d2ed0 con 0x7fb28c15de50 2024-04-25T10:06:47.572 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.566+0000 7fb289e00640 1 --2- 172.21.15.119:0/86008453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb28c15de50 0x7fb28c0d0430 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0097a0 tx=0x7fb27c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfd572e82d8302ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.572 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.566+0000 7fb288a00640 1 -- 172.21.15.119:0/86008453 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c037070 con 0x7fb28c15de50 2024-04-25T10:06:47.573 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.566+0000 7fb288a00640 1 -- 172.21.15.119:0/86008453 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27c032440 con 0x7fb28c15de50 2024-04-25T10:06:47.573 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.566+0000 7fb288a00640 1 -- 172.21.15.119:0/86008453 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c032810 con 0x7fb28c15de50 2024-04-25T10:06:47.573 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.566+0000 7fb291200640 1 -- 172.21.15.119:0/86008453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb28c15de50 msgr2=0x7fb28c0d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.573 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.566+0000 7fb291200640 1 --2- 172.21.15.119:0/86008453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb28c15de50 0x7fb28c0d0430 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0097a0 tx=0x7fb27c02a180 comp rx=0 tx=0).stop 2024-04-25T10:06:47.573 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.567+0000 7fb291200640 1 -- 172.21.15.119:0/86008453 shutdown_connections 2024-04-25T10:06:47.573 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.567+0000 7fb291200640 1 --2- 172.21.15.119:0/86008453 >> v2:172.21.15.132:3300/0 conn(0x7fb28c0d0970 0x7fb28c0d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.573 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.567+0000 7fb291200640 1 --2- 172.21.15.119:0/86008453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb28c15de50 0x7fb28c0d0430 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.573 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.567+0000 7fb291200640 1 -- 172.21.15.119:0/86008453 >> 172.21.15.119:0/86008453 conn(0x7fb28c06bfd0 msgr2=0x7fb28c06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.573 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.567+0000 7fb291200640 1 -- 172.21.15.119:0/86008453 shutdown_connections 2024-04-25T10:06:47.573 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.567+0000 7fb291200640 1 -- 172.21.15.119:0/86008453 wait complete. 2024-04-25T10:06:47.574 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.568+0000 7fb291200640 1 Processor -- start 2024-04-25T10:06:47.574 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.569+0000 7fb291200640 1 -- start start 2024-04-25T10:06:47.576 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.569+0000 7fb291200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28c0d0970 0x7fb28c14ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.576 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.569+0000 7fb291200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb28c0d23c0 0x7fb28c0c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.576 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.569+0000 7fb291200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb28c15de50 0x7fb28c0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.576 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.570+0000 7fb291200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c0d2850 con 0x7fb28c0d23c0 2024-04-25T10:06:47.576 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.570+0000 7fb291200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c0d3490 con 0x7fb28c0d0970 2024-04-25T10:06:47.576 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.570+0000 7fb291200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c0d29d0 con 0x7fb28c15de50 2024-04-25T10:06:47.576 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.570+0000 7fb289400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb28c0d23c0 0x7fb28c0c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.576 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.570+0000 7fb289400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb28c0d23c0 0x7fb28c0c9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36316/0 (socket says 172.21.15.119:36316) 2024-04-25T10:06:47.576 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.570+0000 7fb289400640 1 -- 172.21.15.119:0/2854758964 learned_addr learned my addr 172.21.15.119:0/2854758964 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:06:47.577 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.570+0000 7fb28a800640 1 --2- 172.21.15.119:0/2854758964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb28c15de50 0x7fb28c0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.577 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.570+0000 7fb289e00640 1 --2- 172.21.15.119:0/2854758964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28c0d0970 0x7fb28c14ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.577 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.570+0000 7fb289400640 1 -- 172.21.15.119:0/2854758964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb28c15de50 msgr2=0x7fb28c0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.577 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.570+0000 7fb289400640 1 --2- 172.21.15.119:0/2854758964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb28c15de50 0x7fb28c0ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.577 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.570+0000 7fb289400640 1 -- 172.21.15.119:0/2854758964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28c0d0970 msgr2=0x7fb28c14ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.577 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.570+0000 7fb289400640 1 --2- 172.21.15.119:0/2854758964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28c0d0970 0x7fb28c14ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.577 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.570+0000 7fb289400640 1 -- 172.21.15.119:0/2854758964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28c0c76f0 con 0x7fb28c0d23c0 2024-04-25T10:06:47.577 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.570+0000 7fb289e00640 1 --2- 172.21.15.119:0/2854758964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28c0d0970 0x7fb28c14ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:47.577 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.571+0000 7fb289400640 1 --2- 172.21.15.119:0/2854758964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb28c0d23c0 0x7fb28c0c9ad0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb27800e9b0 tx=0x7fb27800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.578 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.571+0000 7fb282000640 1 -- 172.21.15.119:0/2854758964 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278017070 con 0x7fb28c0d23c0 2024-04-25T10:06:47.578 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.571+0000 7fb282000640 1 -- 172.21.15.119:0/2854758964 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27800c9a0 con 0x7fb28c0d23c0 2024-04-25T10:06:47.579 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.571+0000 7fb291200640 1 -- 172.21.15.119:0/2854758964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28c155600 con 0x7fb28c0d23c0 2024-04-25T10:06:47.579 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.571+0000 7fb282000640 1 -- 172.21.15.119:0/2854758964 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27800cd90 con 0x7fb28c0d23c0 2024-04-25T10:06:47.579 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.571+0000 7fb291200640 1 -- 172.21.15.119:0/2854758964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb28c0c79b0 con 0x7fb28c0d23c0 2024-04-25T10:06:47.579 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.572+0000 7fb282000640 1 -- 172.21.15.119:0/2854758964 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb27801a070 con 0x7fb28c0d23c0 2024-04-25T10:06:47.579 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.572+0000 7fb282000640 1 -- 172.21.15.119:0/2854758964 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb278006660 con 0x7fb28c0d23c0 2024-04-25T10:06:47.579 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.572+0000 7fb291200640 1 -- 172.21.15.119:0/2854758964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28c068530 con 0x7fb28c0d23c0 2024-04-25T10:06:47.582 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.577+0000 7fb282000640 1 -- 172.21.15.119:0/2854758964 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb278005990 con 0x7fb28c0d23c0 2024-04-25T10:06:47.651 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.646+0000 7fb291200640 1 -- 172.21.15.119:0/2854758964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb28c0c7b80 con 0x7fb28c0d23c0 2024-04-25T10:06:47.653 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.647+0000 7fb282000640 1 -- 172.21.15.119:0/2854758964 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fb278005b70 con 0x7fb28c0d23c0 2024-04-25T10:06:47.653 INFO:teuthology.misc.health.smithi119.stdout: 2024-04-25T10:06:47.653 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":10,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:06:46.230936+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:47.654 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.648+0000 7fb291200640 1 -- 172.21.15.119:0/2854758964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb28c0d23c0 msgr2=0x7fb28c0c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.654 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.648+0000 7fb291200640 1 --2- 172.21.15.119:0/2854758964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb28c0d23c0 0x7fb28c0c9ad0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb27800e9b0 tx=0x7fb27800ee70 comp rx=0 tx=0).stop 2024-04-25T10:06:47.654 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.648+0000 7fb291200640 1 -- 172.21.15.119:0/2854758964 shutdown_connections 2024-04-25T10:06:47.654 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.648+0000 7fb291200640 1 --2- 172.21.15.119:0/2854758964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb28c15de50 0x7fb28c0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.654 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.648+0000 7fb291200640 1 --2- 172.21.15.119:0/2854758964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb28c0d23c0 0x7fb28c0c9ad0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.655 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.648+0000 7fb291200640 1 --2- 172.21.15.119:0/2854758964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb28c0d0970 0x7fb28c14ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.655 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.648+0000 7fb291200640 1 -- 172.21.15.119:0/2854758964 >> 172.21.15.119:0/2854758964 conn(0x7fb28c06bfd0 msgr2=0x7fb28c0b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.655 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.648+0000 7fb291200640 1 -- 172.21.15.119:0/2854758964 shutdown_connections 2024-04-25T10:06:47.655 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:47.648+0000 7fb291200640 1 -- 172.21.15.119:0/2854758964 wait complete. 2024-04-25T10:06:47.666 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:06:47.769 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:47.763+0000 7f84f3e96040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:06:47.902 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:47.896+0000 7f84f3e96040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:06:47.958 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:47.952+0000 7f5c573e3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:06:48.088 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-25T10:06:48.083+0000 7f5c573e3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:06:48.954 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:48.948+0000 7f8495000640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T10:06:48.954 INFO:tasks.ceph.mgr.y.smithi119.stderr:2024-04-25T10:06:48.948+0000 7f8495000640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T10:06:50.602 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-04-25T10:06:50.597+0000 7ff69ba86640 -1 osd.0 0 log_to_monitors true 2024-04-25T10:06:50.613 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-04-25T10:06:50.608+0000 7f2aa0b33640 -1 osd.1 0 log_to_monitors true 2024-04-25T10:06:50.613 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-04-25T10:06:50.609+0000 7fa92521c640 -1 osd.3 0 log_to_monitors true 2024-04-25T10:06:50.614 INFO:tasks.ceph.osd.6.smithi132.stderr:2024-04-25T10:06:50.609+0000 7f9737faf640 -1 osd.6 0 log_to_monitors true 2024-04-25T10:06:50.638 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-04-25T10:06:50.632+0000 7f1e39e0d640 -1 osd.5 0 log_to_monitors true 2024-04-25T10:06:50.642 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-04-25T10:06:50.638+0000 7f7714e10640 -1 osd.2 0 log_to_monitors true 2024-04-25T10:06:50.824 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-04-25T10:06:50.818+0000 7fa138086640 -1 osd.4 0 log_to_monitors true 2024-04-25T10:06:50.879 INFO:tasks.ceph.osd.7.smithi132.stderr:2024-04-25T10:06:50.873+0000 7fa1bc615640 -1 osd.7 0 log_to_monitors true 2024-04-25T10:06:54.263 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-04-25T10:06:54.257+0000 7f2a9ce00640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T10:06:54.270 INFO:tasks.ceph.osd.1.smithi119.stderr:2024-04-25T10:06:54.265+0000 7f2a91a00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:06:54.279 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-04-25T10:06:54.274+0000 7fa91f000640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T10:06:54.291 INFO:tasks.ceph.osd.3.smithi119.stderr:2024-04-25T10:06:54.285+0000 7fa915a00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:06:54.322 INFO:tasks.ceph.osd.7.smithi132.stderr:2024-04-25T10:06:54.317+0000 7fa1b6400640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T10:06:54.330 INFO:tasks.ceph.osd.7.smithi132.stderr:2024-04-25T10:06:54.325+0000 7fa1ace00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:06:54.334 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-04-25T10:06:54.329+0000 7fa132600640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T10:06:54.334 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-04-25T10:06:54.329+0000 7f1e33e00640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T10:06:54.338 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-04-25T10:06:54.332+0000 7ff697e00640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T10:06:54.339 INFO:tasks.ceph.osd.6.smithi132.stderr:2024-04-25T10:06:54.334+0000 7f9732400640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T10:06:54.342 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-04-25T10:06:54.337+0000 7f7710a00640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T10:06:54.344 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-04-25T10:06:54.338+0000 7fa129000640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:06:54.344 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-04-25T10:06:54.338+0000 7f1e2a800640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:06:54.349 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-04-25T10:06:54.344+0000 7ff68ca00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:06:54.356 INFO:tasks.ceph.osd.2.smithi119.stderr:2024-04-25T10:06:54.350+0000 7f7705600640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:06:54.357 INFO:tasks.ceph.osd.6.smithi132.stderr:2024-04-25T10:06:54.352+0000 7f9728e00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:06:54.474 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:54.642 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.636+0000 7f58dfe00640 1 Processor -- start 2024-04-25T10:06:54.642 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.636+0000 7f58dfe00640 1 -- start start 2024-04-25T10:06:54.644 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.637+0000 7f58dfe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f58e0143fc0 0x7f58e00ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.644 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58dfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58e00aed50 0x7f58e0071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.644 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58dfe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f58e0071660 0x7f58e006d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.644 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58dfe00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e0071a90 con 0x7f58e0143fc0 2024-04-25T10:06:54.644 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58dfe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e00b2680 con 0x7f58e0071660 2024-04-25T10:06:54.644 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58dfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e00b2940 con 0x7f58e00aed50 2024-04-25T10:06:54.644 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58dea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f58e0143fc0 0x7f58e00ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.644 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58dea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f58e0143fc0 0x7f58e00ae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60340/0 (socket says 172.21.15.119:60340) 2024-04-25T10:06:54.645 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58dea00640 1 -- 172.21.15.119:0/906561613 learned_addr learned my addr 172.21.15.119:0/906561613 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:06:54.645 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58de000640 1 --2- 172.21.15.119:0/906561613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58e00aed50 0x7f58e0071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.645 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58df400640 1 --2- 172.21.15.119:0/906561613 >> v2:172.21.15.132:3300/0 conn(0x7f58e0071660 0x7f58e006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.645 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58dea00640 1 -- 172.21.15.119:0/906561613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58e00aed50 msgr2=0x7f58e0071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.645 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58dea00640 1 --2- 172.21.15.119:0/906561613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58e00aed50 0x7f58e0071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.645 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58dea00640 1 -- 172.21.15.119:0/906561613 >> v2:172.21.15.132:3300/0 conn(0x7f58e0071660 msgr2=0x7f58e006d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.645 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58dea00640 1 --2- 172.21.15.119:0/906561613 >> v2:172.21.15.132:3300/0 conn(0x7f58e0071660 0x7f58e006d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.645 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.638+0000 7f58dea00640 1 -- 172.21.15.119:0/906561613 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e00b2c20 con 0x7f58e0143fc0 2024-04-25T10:06:54.646 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.640+0000 7f58dea00640 1 --2- 172.21.15.119:0/906561613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58e0143fc0 0x7f58e00ae810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f58e00adad0 tx=0x7f58d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9350705ac0368d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.647 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.640+0000 7f58dd600640 1 -- 172.21.15.119:0/906561613 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0037070 con 0x7f58e0143fc0 2024-04-25T10:06:54.647 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.640+0000 7f58dd600640 1 -- 172.21.15.119:0/906561613 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d0032440 con 0x7f58e0143fc0 2024-04-25T10:06:54.647 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.640+0000 7f58dd600640 1 -- 172.21.15.119:0/906561613 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0032810 con 0x7f58e0143fc0 2024-04-25T10:06:54.647 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.641+0000 7f58dfe00640 1 -- 172.21.15.119:0/906561613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58e0143fc0 msgr2=0x7f58e00ae810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.647 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.641+0000 7f58dfe00640 1 --2- 172.21.15.119:0/906561613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58e0143fc0 0x7f58e00ae810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f58e00adad0 tx=0x7f58d002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.647 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.641+0000 7f58dfe00640 1 -- 172.21.15.119:0/906561613 shutdown_connections 2024-04-25T10:06:54.647 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.641+0000 7f58dfe00640 1 --2- 172.21.15.119:0/906561613 >> v2:172.21.15.132:3300/0 conn(0x7f58e0071660 0x7f58e006d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.647 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.641+0000 7f58dfe00640 1 --2- 172.21.15.119:0/906561613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58e00aed50 0x7f58e0071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.647 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.641+0000 7f58dfe00640 1 --2- 172.21.15.119:0/906561613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58e0143fc0 0x7f58e00ae810 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.647 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.641+0000 7f58dfe00640 1 -- 172.21.15.119:0/906561613 >> 172.21.15.119:0/906561613 conn(0x7f58e006aa70 msgr2=0x7f58e006ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.647 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.641+0000 7f58dfe00640 1 -- 172.21.15.119:0/906561613 shutdown_connections 2024-04-25T10:06:54.647 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.641+0000 7f58dfe00640 1 -- 172.21.15.119:0/906561613 wait complete. 2024-04-25T10:06:54.648 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.642+0000 7f58dfe00640 1 Processor -- start 2024-04-25T10:06:54.648 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.643+0000 7f58dfe00640 1 -- start start 2024-04-25T10:06:54.651 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.643+0000 7f58dfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58e0071660 0x7f58e0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.651 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.643+0000 7f58dfe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58e00aed50 0x7f58e0158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.651 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.643+0000 7f58dfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58e0143fc0 0x7f58e00ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.651 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.643+0000 7f58dfe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e006e9a0 con 0x7f58e0071660 2024-04-25T10:06:54.651 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.643+0000 7f58dfe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e00b31e0 con 0x7f58e00aed50 2024-04-25T10:06:54.651 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.643+0000 7f58dfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e01443a0 con 0x7f58e0143fc0 2024-04-25T10:06:54.651 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.644+0000 7f58dea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58e0071660 0x7f58e0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.651 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.644+0000 7f58dea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58e0071660 0x7f58e0156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60350/0 (socket says 172.21.15.119:60350) 2024-04-25T10:06:54.651 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.644+0000 7f58dea00640 1 -- 172.21.15.119:0/2225642617 learned_addr learned my addr 172.21.15.119:0/2225642617 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:06:54.651 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.644+0000 7f58de000640 1 --2- 172.21.15.119:0/2225642617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58e00aed50 0x7f58e0158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.651 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.644+0000 7f58df400640 1 --2- 172.21.15.119:0/2225642617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58e0143fc0 0x7f58e00ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.651 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.644+0000 7f58dea00640 1 -- 172.21.15.119:0/2225642617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58e0143fc0 msgr2=0x7f58e00ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.651 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.644+0000 7f58dea00640 1 --2- 172.21.15.119:0/2225642617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58e0143fc0 0x7f58e00ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.652 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.644+0000 7f58dea00640 1 -- 172.21.15.119:0/2225642617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58e00aed50 msgr2=0x7f58e0158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.652 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.644+0000 7f58dea00640 1 --2- 172.21.15.119:0/2225642617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58e00aed50 0x7f58e0158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.652 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.644+0000 7f58dea00640 1 -- 172.21.15.119:0/2225642617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e0158f70 con 0x7f58e0071660 2024-04-25T10:06:54.652 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.644+0000 7f58dea00640 1 --2- 172.21.15.119:0/2225642617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58e0071660 0x7f58e0156bc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f58d002a220 tx=0x7f58d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.652 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.645+0000 7f58d7400640 1 -- 172.21.15.119:0/2225642617 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0037040 con 0x7f58e0071660 2024-04-25T10:06:54.652 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.645+0000 7f58d7400640 1 -- 172.21.15.119:0/2225642617 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d0042070 con 0x7f58e0071660 2024-04-25T10:06:54.652 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.645+0000 7f58d7400640 1 -- 172.21.15.119:0/2225642617 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0030bb0 con 0x7f58e0071660 2024-04-25T10:06:54.652 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.645+0000 7f58dfe00640 1 -- 172.21.15.119:0/2225642617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e015f080 con 0x7f58e0071660 2024-04-25T10:06:54.652 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.645+0000 7f58dfe00640 1 -- 172.21.15.119:0/2225642617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e00cf0a0 con 0x7f58e0071660 2024-04-25T10:06:54.653 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.646+0000 7f58dfe00640 1 -- 172.21.15.119:0/2225642617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e0074f10 con 0x7f58e0071660 2024-04-25T10:06:54.653 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.647+0000 7f58d7400640 1 -- 172.21.15.119:0/2225642617 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f58d0047460 con 0x7f58e0071660 2024-04-25T10:06:54.653 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.648+0000 7f58d7400640 1 --2- 172.21.15.119:0/2225642617 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f58c006b470 0x7f58c006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.653 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.648+0000 7f58d7400640 1 -- 172.21.15.119:0/2225642617 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f58d00a9ff0 con 0x7f58e0071660 2024-04-25T10:06:54.653 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.648+0000 7f58de000640 1 --2- 172.21.15.119:0/2225642617 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f58c006b470 0x7f58c006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.654 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.648+0000 7f58de000640 1 --2- 172.21.15.119:0/2225642617 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f58c006b470 0x7f58c006d930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58e01589e0 tx=0x7f58cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.658 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.653+0000 7f58d7400640 1 -- 172.21.15.119:0/2225642617 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58d0045360 con 0x7f58e0071660 2024-04-25T10:06:54.841 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.835+0000 7f58dfe00640 1 -- 172.21.15.119:0/2225642617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58e00b4960 con 0x7f58e0071660 2024-04-25T10:06:54.842 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.837+0000 7f58d7400640 1 -- 172.21.15.119:0/2225642617 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f58d0079c40 con 0x7f58e0071660 2024-04-25T10:06:54.843 INFO:teuthology.misc.health.smithi119.stdout: 2024-04-25T10:06:54.843 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":12,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:06:51.963825+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:54.845 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.839+0000 7f58dfe00640 1 -- 172.21.15.119:0/2225642617 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f58c006b470 msgr2=0x7f58c006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.845 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.839+0000 7f58dfe00640 1 --2- 172.21.15.119:0/2225642617 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f58c006b470 0x7f58c006d930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58e01589e0 tx=0x7f58cc008040 comp rx=0 tx=0).stop 2024-04-25T10:06:54.845 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.839+0000 7f58dfe00640 1 -- 172.21.15.119:0/2225642617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58e0071660 msgr2=0x7f58e0156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.845 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.839+0000 7f58dfe00640 1 --2- 172.21.15.119:0/2225642617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58e0071660 0x7f58e0156bc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f58d002a220 tx=0x7f58d00046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.845 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.839+0000 7f58dfe00640 1 -- 172.21.15.119:0/2225642617 shutdown_connections 2024-04-25T10:06:54.846 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.839+0000 7f58dfe00640 1 --2- 172.21.15.119:0/2225642617 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f58c006b470 0x7f58c006d930 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.846 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.839+0000 7f58dfe00640 1 --2- 172.21.15.119:0/2225642617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58e0143fc0 0x7f58e00ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.846 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.839+0000 7f58dfe00640 1 --2- 172.21.15.119:0/2225642617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58e00aed50 0x7f58e0158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.846 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.839+0000 7f58dfe00640 1 --2- 172.21.15.119:0/2225642617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58e0071660 0x7f58e0156bc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.846 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.839+0000 7f58dfe00640 1 -- 172.21.15.119:0/2225642617 >> 172.21.15.119:0/2225642617 conn(0x7f58e006aa70 msgr2=0x7f58e00b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.846 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.839+0000 7f58dfe00640 1 -- 172.21.15.119:0/2225642617 shutdown_connections 2024-04-25T10:06:54.846 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:06:54.840+0000 7f58dfe00640 1 -- 172.21.15.119:0/2225642617 wait complete. 2024-04-25T10:06:54.859 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:06:59.730 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:59.737 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:01.667 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:01.851 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.846+0000 7f0231000640 1 Processor -- start 2024-04-25T10:07:01.852 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.846+0000 7f0231000640 1 -- start start 2024-04-25T10:07:01.853 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.847+0000 7f0231000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c071260 0x7f022c15ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.853 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.847+0000 7f0231000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f022c15d4d0 0x7f022c15d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.853 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.847+0000 7f0231000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022c06ed30 con 0x7f022c06d2c0 2024-04-25T10:07:01.853 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.847+0000 7f0231000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022c15cf60 con 0x7f022c15d4d0 2024-04-25T10:07:01.853 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.847+0000 7f0231000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c06e300 con 0x7f022c071260 2024-04-25T10:07:01.853 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.848+0000 7f022a800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f022c15d4d0 0x7f022c15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.854 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.848+0000 7f022a800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f022c15d4d0 0x7f022c15d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49108/0 (socket says 172.21.15.119:49108) 2024-04-25T10:07:01.854 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.848+0000 7f022a800640 1 -- 172.21.15.119:0/594131511 learned_addr learned my addr 172.21.15.119:0/594131511 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:01.854 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.848+0000 7f0229e00640 1 --2- 172.21.15.119:0/594131511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c071260 0x7f022c15ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.855 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.848+0000 7f022a800640 1 -- 172.21.15.119:0/594131511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c071260 msgr2=0x7f022c15ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.855 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.848+0000 7f022a800640 1 --2- 172.21.15.119:0/594131511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c071260 0x7f022c15ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.855 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.848+0000 7f022a800640 1 -- 172.21.15.119:0/594131511 >> v1:172.21.15.132:6789/0 conn(0x7f022c06d2c0 legacy=0x7f022c071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.855 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.848+0000 7f022a800640 1 -- 172.21.15.119:0/594131511 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c06d780 con 0x7f022c15d4d0 2024-04-25T10:07:01.856 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.849+0000 7f022a800640 1 --2- 172.21.15.119:0/594131511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f022c15d4d0 0x7f022c15d960 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0224009e50 tx=0x7f022402a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f229b870d974ee33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.856 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.850+0000 7f0228a00640 1 -- 172.21.15.119:0/594131511 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224036070 con 0x7f022c15d4d0 2024-04-25T10:07:01.856 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.850+0000 7f0228a00640 1 -- 172.21.15.119:0/594131511 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0224031600 con 0x7f022c15d4d0 2024-04-25T10:07:01.856 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.850+0000 7f0228a00640 1 -- 172.21.15.119:0/594131511 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02240319d0 con 0x7f022c15d4d0 2024-04-25T10:07:01.856 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.850+0000 7f0231000640 1 -- 172.21.15.119:0/594131511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f022c15d4d0 msgr2=0x7f022c15d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.856 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.850+0000 7f0231000640 1 --2- 172.21.15.119:0/594131511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f022c15d4d0 0x7f022c15d960 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0224009e50 tx=0x7f022402a640 comp rx=0 tx=0).stop 2024-04-25T10:07:01.857 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.850+0000 7f0231000640 1 -- 172.21.15.119:0/594131511 shutdown_connections 2024-04-25T10:07:01.857 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.850+0000 7f0231000640 1 --2- 172.21.15.119:0/594131511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f022c15d4d0 0x7f022c15d960 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.857 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.850+0000 7f0231000640 1 --2- 172.21.15.119:0/594131511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c071260 0x7f022c15ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.857 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.850+0000 7f0231000640 1 -- 172.21.15.119:0/594131511 >> 172.21.15.119:0/594131511 conn(0x7f022c0b1490 msgr2=0x7f022c0b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.857 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.851+0000 7f0231000640 1 -- 172.21.15.119:0/594131511 shutdown_connections 2024-04-25T10:07:01.857 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.851+0000 7f0231000640 1 -- 172.21.15.119:0/594131511 wait complete. 2024-04-25T10:07:01.858 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.852+0000 7f0231000640 1 Processor -- start 2024-04-25T10:07:01.858 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.852+0000 7f0231000640 1 -- start start 2024-04-25T10:07:01.860 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.853+0000 7f0231000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f022c06d2c0 0x7f022c1e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.860 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.853+0000 7f0231000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c071260 0x7f022c1516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.860 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.853+0000 7f0231000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c15d4d0 0x7f022c1e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.860 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.853+0000 7f0231000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c06e020 con 0x7f022c06d2c0 2024-04-25T10:07:01.860 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.853+0000 7f0231000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c06dd40 con 0x7f022c071260 2024-04-25T10:07:01.861 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.853+0000 7f0231000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c15f190 con 0x7f022c15d4d0 2024-04-25T10:07:01.861 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.854+0000 7f0229e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f022c06d2c0 0x7f022c1e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.861 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.854+0000 7f0229e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f022c06d2c0 0x7f022c1e7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49124/0 (socket says 172.21.15.119:49124) 2024-04-25T10:07:01.861 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.854+0000 7f0229e00640 1 -- 172.21.15.119:0/2700037882 learned_addr learned my addr 172.21.15.119:0/2700037882 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:01.861 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.854+0000 7f0229e00640 1 -- 172.21.15.119:0/2700037882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c15d4d0 msgr2=0x7f022c1e53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.861 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.854+0000 7f022a800640 1 --2- 172.21.15.119:0/2700037882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c15d4d0 0x7f022c1e53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.861 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.854+0000 7f0229400640 1 --2- 172.21.15.119:0/2700037882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c071260 0x7f022c1516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.861 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.854+0000 7f0229e00640 1 --2- 172.21.15.119:0/2700037882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c15d4d0 0x7f022c1e53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.861 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.854+0000 7f0229e00640 1 -- 172.21.15.119:0/2700037882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c071260 msgr2=0x7f022c1516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.861 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.854+0000 7f0229e00640 1 --2- 172.21.15.119:0/2700037882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c071260 0x7f022c1516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.861 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.854+0000 7f0229e00640 1 -- 172.21.15.119:0/2700037882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c1e5c20 con 0x7f022c06d2c0 2024-04-25T10:07:01.861 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.854+0000 7f0229e00640 1 --2- 172.21.15.119:0/2700037882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f022c06d2c0 0x7f022c1e7b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f021c00c7f0 tx=0x7f021c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.861 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.855+0000 7f0222000640 1 -- 172.21.15.119:0/2700037882 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c015070 con 0x7f022c06d2c0 2024-04-25T10:07:01.861 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.855+0000 7f0222000640 1 -- 172.21.15.119:0/2700037882 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021c004220 con 0x7f022c06d2c0 2024-04-25T10:07:01.862 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.855+0000 7f0222000640 1 -- 172.21.15.119:0/2700037882 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c007620 con 0x7f022c06d2c0 2024-04-25T10:07:01.862 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.855+0000 7f0231000640 1 -- 172.21.15.119:0/2700037882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022c1e5f00 con 0x7f022c06d2c0 2024-04-25T10:07:01.862 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.855+0000 7f0231000640 1 -- 172.21.15.119:0/2700037882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022c0a89e0 con 0x7f022c06d2c0 2024-04-25T10:07:01.862 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.856+0000 7f0231000640 1 -- 172.21.15.119:0/2700037882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022c0680d0 con 0x7f022c06d2c0 2024-04-25T10:07:01.863 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.857+0000 7f0222000640 1 -- 172.21.15.119:0/2700037882 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f021c0079d0 con 0x7f022c06d2c0 2024-04-25T10:07:01.864 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.858+0000 7f0222000640 1 --2- 172.21.15.119:0/2700037882 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f020c06b4d0 0x7f020c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.864 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.858+0000 7f0222000640 1 -- 172.21.15.119:0/2700037882 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f021c08a250 con 0x7f022c06d2c0 2024-04-25T10:07:01.864 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.858+0000 7f0229400640 1 --2- 172.21.15.119:0/2700037882 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f020c06b4d0 0x7f020c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.864 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.859+0000 7f0229400640 1 --2- 172.21.15.119:0/2700037882 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f020c06b4d0 0x7f020c06d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f022c0aea70 tx=0x7f0218008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.869 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:01.863+0000 7f0222000640 1 -- 172.21.15.119:0/2700037882 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f021c059410 con 0x7f022c06d2c0 2024-04-25T10:07:02.071 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:02.065+0000 7f0231000640 1 -- 172.21.15.119:0/2700037882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f022c068530 con 0x7f022c06d2c0 2024-04-25T10:07:02.073 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:02.067+0000 7f0222000640 1 -- 172.21.15.119:0/2700037882 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12920 (secure 0 0 0) 0x7f021c058d10 con 0x7f022c06d2c0 2024-04-25T10:07:02.073 INFO:teuthology.misc.health.smithi119.stdout: 2024-04-25T10:07:02.074 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":16,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:06:58.306787+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:02.075 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:02.070+0000 7f0231000640 1 -- 172.21.15.119:0/2700037882 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f020c06b4d0 msgr2=0x7f020c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.075 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:02.070+0000 7f0231000640 1 --2- 172.21.15.119:0/2700037882 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f020c06b4d0 0x7f020c06d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f022c0aea70 tx=0x7f0218008040 comp rx=0 tx=0).stop 2024-04-25T10:07:02.075 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:02.070+0000 7f0231000640 1 -- 172.21.15.119:0/2700037882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f022c06d2c0 msgr2=0x7f022c1e7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.075 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:02.070+0000 7f0231000640 1 --2- 172.21.15.119:0/2700037882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f022c06d2c0 0x7f022c1e7b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f021c00c7f0 tx=0x7f021c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.076 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:02.070+0000 7f0231000640 1 -- 172.21.15.119:0/2700037882 shutdown_connections 2024-04-25T10:07:02.076 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:02.070+0000 7f0231000640 1 --2- 172.21.15.119:0/2700037882 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f020c06b4d0 0x7f020c06d990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.076 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:02.070+0000 7f0231000640 1 --2- 172.21.15.119:0/2700037882 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f022c15d4d0 0x7f022c1e53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.076 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:02.070+0000 7f0231000640 1 --2- 172.21.15.119:0/2700037882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f022c071260 0x7f022c1516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.076 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:02.070+0000 7f0231000640 1 --2- 172.21.15.119:0/2700037882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f022c06d2c0 0x7f022c1e7b10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.076 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:02.070+0000 7f0231000640 1 -- 172.21.15.119:0/2700037882 >> 172.21.15.119:0/2700037882 conn(0x7f022c0b1490 msgr2=0x7f022c0ce180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.077 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:02.070+0000 7f0231000640 1 -- 172.21.15.119:0/2700037882 shutdown_connections 2024-04-25T10:07:02.077 INFO:teuthology.misc.health.smithi119.stderr:2024-04-25T10:07:02.070+0000 7f0231000640 1 -- 172.21.15.119:0/2700037882 wait complete. 2024-04-25T10:07:02.090 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T10:07:02.090 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T10:07:02.091 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:07:02.091 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:07:02.098 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:07:02.098 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:07:02.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:07:02.105 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:07:02.106 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:07:02.106 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:07:02.113 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:07:02.113 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:07:02.114 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:07:02.121 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:07:02.121 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T10:07:02.124 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:02.241 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:07:02.242 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2024-04-25T10:07:02.244 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:07:02.239+0000 7fe2b8400640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6835/0: (98) Address already in use 2024-04-25T10:07:02.272 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-04-25T10:07:02.273 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-04-25T10:07:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.286+0000 7fbef1800640 1 Processor -- start 2024-04-25T10:07:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.286+0000 7fbef1800640 1 -- start start 2024-04-25T10:07:02.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.287+0000 7fbef1800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeec0ce560 0x7fbeec0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.287+0000 7fbef1800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbeec0d3340 0x7fbeec0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.287+0000 7fbef1800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeec0cee80 con 0x7fbeec15dc80 2024-04-25T10:07:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.287+0000 7fbef1800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeec0d0e30 con 0x7fbeec0d3340 2024-04-25T10:07:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.287+0000 7fbef1800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeec0d2b50 con 0x7fbeec0ce560 2024-04-25T10:07:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.287+0000 7fbeeb400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbeec15dc80 0x7fbeec0b53a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56314/0 (socket says 172.21.15.119:56314) 2024-04-25T10:07:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.287+0000 7fbeeb400640 1 -- 172.21.15.119:0/1939323861 learned_addr learned my addr 172.21.15.119:0/1939323861 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.288+0000 7fbeebe00640 1 --2- 172.21.15.119:0/1939323861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeec0ce560 0x7fbeec0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.288+0000 7fbef0e00640 1 --2- 172.21.15.119:0/1939323861 >> v2:172.21.15.132:3300/0 conn(0x7fbeec0d3340 0x7fbeec0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.288+0000 7fbeeaa00640 1 -- 172.21.15.119:0/1939323861 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976365007 0 0) 0x7fbeec0cee80 con 0x7fbeec15dc80 2024-04-25T10:07:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.288+0000 7fbeeaa00640 1 -- 172.21.15.119:0/1939323861 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbed0003520 con 0x7fbeec15dc80 2024-04-25T10:07:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.288+0000 7fbeebe00640 1 -- 172.21.15.119:0/1939323861 >> v2:172.21.15.132:3300/0 conn(0x7fbeec0d3340 msgr2=0x7fbeec0b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.288+0000 7fbeebe00640 1 --2- 172.21.15.119:0/1939323861 >> v2:172.21.15.132:3300/0 conn(0x7fbeec0d3340 0x7fbeec0b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.288+0000 7fbeebe00640 1 -- 172.21.15.119:0/1939323861 >> v1:172.21.15.119:6789/0 conn(0x7fbeec15dc80 legacy=0x7fbeec0b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.288+0000 7fbeebe00640 1 -- 172.21.15.119:0/1939323861 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeec0d1c80 con 0x7fbeec0ce560 2024-04-25T10:07:02.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.289+0000 7fbeebe00640 1 --2- 172.21.15.119:0/1939323861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeec0ce560 0x7fbeec0ce940 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0097a0 tx=0x7fbedc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78c1b4e80792c72c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.289+0000 7fbeeaa00640 1 -- 172.21.15.119:0/1939323861 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc037070 con 0x7fbeec0ce560 2024-04-25T10:07:02.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.290+0000 7fbeeaa00640 1 -- 172.21.15.119:0/1939323861 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc032440 con 0x7fbeec0ce560 2024-04-25T10:07:02.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.290+0000 7fbeeaa00640 1 -- 172.21.15.119:0/1939323861 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc032810 con 0x7fbeec0ce560 2024-04-25T10:07:02.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.290+0000 7fbef1800640 1 -- 172.21.15.119:0/1939323861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeec0ce560 msgr2=0x7fbeec0ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.290+0000 7fbef1800640 1 --2- 172.21.15.119:0/1939323861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeec0ce560 0x7fbeec0ce940 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0097a0 tx=0x7fbedc02a180 comp rx=0 tx=0).stop 2024-04-25T10:07:02.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.290+0000 7fbef1800640 1 -- 172.21.15.119:0/1939323861 shutdown_connections 2024-04-25T10:07:02.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.290+0000 7fbef1800640 1 --2- 172.21.15.119:0/1939323861 >> v2:172.21.15.132:3300/0 conn(0x7fbeec0d3340 0x7fbeec0b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.290+0000 7fbef1800640 1 --2- 172.21.15.119:0/1939323861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeec0ce560 0x7fbeec0ce940 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.290+0000 7fbef1800640 1 -- 172.21.15.119:0/1939323861 >> 172.21.15.119:0/1939323861 conn(0x7fbeec06bde0 msgr2=0x7fbeec06c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.290+0000 7fbef1800640 1 -- 172.21.15.119:0/1939323861 shutdown_connections 2024-04-25T10:07:02.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.290+0000 7fbef1800640 1 -- 172.21.15.119:0/1939323861 wait complete. 2024-04-25T10:07:02.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.292+0000 7fbef1800640 1 Processor -- start 2024-04-25T10:07:02.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.292+0000 7fbef1800640 1 -- start start 2024-04-25T10:07:02.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbef1800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbeec0ce560 0x7fbeec150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbef1800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeec0d3340 0x7fbeec1f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbef1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbeec15dc80 0x7fbeec0cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbef1800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeec0d2500 con 0x7fbeec15dc80 2024-04-25T10:07:02.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbef1800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeec0d2220 con 0x7fbeec0ce560 2024-04-25T10:07:02.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbef1800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeec0d27e0 con 0x7fbeec0d3340 2024-04-25T10:07:02.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbef0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbeec15dc80 0x7fbeec0cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbef0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbeec15dc80 0x7fbeec0cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49150/0 (socket says 172.21.15.119:49150) 2024-04-25T10:07:02.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbef0e00640 1 -- 172.21.15.119:0/543198100 learned_addr learned my addr 172.21.15.119:0/543198100 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:02.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbeeb400640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeec0d3340 0x7fbeec1f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbeebe00640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbeec0ce560 0x7fbeec150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbef0e00640 1 -- 172.21.15.119:0/543198100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeec0d3340 msgr2=0x7fbeec1f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbef0e00640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeec0d3340 0x7fbeec1f4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbef0e00640 1 -- 172.21.15.119:0/543198100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbeec0ce560 msgr2=0x7fbeec150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbef0e00640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbeec0ce560 0x7fbeec150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbef0e00640 1 -- 172.21.15.119:0/543198100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeec0cab90 con 0x7fbeec15dc80 2024-04-25T10:07:02.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.293+0000 7fbeebe00640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbeec0ce560 0x7fbeec150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:02.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.294+0000 7fbef0e00640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbeec15dc80 0x7fbeec0cc840 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009dd0 tx=0x7fbee400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.294+0000 7fbee3e00640 1 -- 172.21.15.119:0/543198100 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee4018020 con 0x7fbeec15dc80 2024-04-25T10:07:02.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.294+0000 7fbee3e00640 1 -- 172.21.15.119:0/543198100 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee4014480 con 0x7fbeec15dc80 2024-04-25T10:07:02.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.294+0000 7fbef1800640 1 -- 172.21.15.119:0/543198100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeec0cae70 con 0x7fbeec15dc80 2024-04-25T10:07:02.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.294+0000 7fbee3e00640 1 -- 172.21.15.119:0/543198100 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee4014870 con 0x7fbeec15dc80 2024-04-25T10:07:02.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.294+0000 7fbef1800640 1 -- 172.21.15.119:0/543198100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeec1f4e90 con 0x7fbeec15dc80 2024-04-25T10:07:02.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.295+0000 7fbef1800640 1 -- 172.21.15.119:0/543198100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeec0749b0 con 0x7fbeec15dc80 2024-04-25T10:07:02.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.296+0000 7fbee3e00640 1 -- 172.21.15.119:0/543198100 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbee4005d10 con 0x7fbeec15dc80 2024-04-25T10:07:02.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.297+0000 7fbee3e00640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbecc06b4d0 0x7fbecc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.297+0000 7fbee3e00640 1 -- 172.21.15.119:0/543198100 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbee408ae10 con 0x7fbeec15dc80 2024-04-25T10:07:02.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.297+0000 7fbeebe00640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbecc06b4d0 0x7fbecc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.298+0000 7fbeebe00640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbecc06b4d0 0x7fbecc06d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbedc034000 tx=0x7fbedc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.302+0000 7fbee3e00640 1 -- 172.21.15.119:0/543198100 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbee408e030 con 0x7fbeec15dc80 2024-04-25T10:07:02.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.481+0000 7fbef1800640 1 -- 172.21.15.119:0/543198100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbeec06ae10 con 0x7fbeec15dc80 2024-04-25T10:07:02.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.483+0000 7fbee3e00640 1 -- 172.21.15.119:0/543198100 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12920 (secure 0 0 0) 0x7fbee4022030 con 0x7fbeec15dc80 2024-04-25T10:07:02.489 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:02.489 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:06:58.306787+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:02.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.485+0000 7fbef1800640 1 -- 172.21.15.119:0/543198100 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbecc06b4d0 msgr2=0x7fbecc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.485+0000 7fbef1800640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbecc06b4d0 0x7fbecc06d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbedc034000 tx=0x7fbedc02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:02.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.485+0000 7fbef1800640 1 -- 172.21.15.119:0/543198100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbeec15dc80 msgr2=0x7fbeec0cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.485+0000 7fbef1800640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbeec15dc80 0x7fbeec0cc840 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009dd0 tx=0x7fbee400c490 comp rx=0 tx=0).stop 2024-04-25T10:07:02.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.485+0000 7fbef1800640 1 -- 172.21.15.119:0/543198100 shutdown_connections 2024-04-25T10:07:02.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.485+0000 7fbef1800640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbecc06b4d0 0x7fbecc06d990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.485+0000 7fbef1800640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbeec15dc80 0x7fbeec0cc840 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.485+0000 7fbef1800640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbeec0d3340 0x7fbeec1f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.485+0000 7fbef1800640 1 --2- 172.21.15.119:0/543198100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbeec0ce560 0x7fbeec150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.485+0000 7fbef1800640 1 -- 172.21.15.119:0/543198100 >> 172.21.15.119:0/543198100 conn(0x7fbeec06bde0 msgr2=0x7fbeec0d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.486+0000 7fbef1800640 1 -- 172.21.15.119:0/543198100 shutdown_connections 2024-04-25T10:07:02.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.486+0000 7fbef1800640 1 -- 172.21.15.119:0/543198100 wait complete. 2024-04-25T10:07:02.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:02.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:02.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.695+0000 7f811ee00640 1 Processor -- start 2024-04-25T10:07:02.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.695+0000 7f811ee00640 1 -- start start 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.696+0000 7f811ee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8118144e70 0x7f811806d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.696+0000 7f811ee00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f81180cf0e0 0x7f81180cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.696+0000 7f811ee00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811806dcb0 con 0x7f811806d7a0 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.696+0000 7f811ee00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811815bfe0 con 0x7f81180cf0e0 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.696+0000 7f811ee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811806df90 con 0x7f8118144e70 2024-04-25T10:07:02.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.697+0000 7f811ce00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f81180cf0e0 0x7f81180cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.697+0000 7f8117400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f811806d7a0 0x7f81180ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56318/0 (socket says 172.21.15.119:56318) 2024-04-25T10:07:02.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.697+0000 7f8117400640 1 -- 172.21.15.119:0/898651394 learned_addr learned my addr 172.21.15.119:0/898651394 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:02.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.697+0000 7f8117e00640 1 --2- 172.21.15.119:0/898651394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8118144e70 0x7f811806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.697+0000 7f8116a00640 1 -- 172.21.15.119:0/898651394 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159767030 0 0) 0x7f811806dcb0 con 0x7f811806d7a0 2024-04-25T10:07:02.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.697+0000 7f8116a00640 1 -- 172.21.15.119:0/898651394 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80fc003520 con 0x7f811806d7a0 2024-04-25T10:07:02.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.697+0000 7f811ce00640 1 -- 172.21.15.119:0/898651394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8118144e70 msgr2=0x7f811806d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.697+0000 7f811ce00640 1 --2- 172.21.15.119:0/898651394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8118144e70 0x7f811806d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.697+0000 7f811ce00640 1 -- 172.21.15.119:0/898651394 >> v1:172.21.15.119:6789/0 conn(0x7f811806d7a0 legacy=0x7f81180ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.697+0000 7f811ce00640 1 -- 172.21.15.119:0/898651394 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811806e270 con 0x7f81180cf0e0 2024-04-25T10:07:02.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.698+0000 7f811ce00640 1 --2- 172.21.15.119:0/898651394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81180cf0e0 0x7f81180cc300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8110009d90 tx=0x7f811002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc33dc62f0c83273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.699+0000 7f8116a00640 1 -- 172.21.15.119:0/898651394 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8110031600 con 0x7f81180cf0e0 2024-04-25T10:07:02.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.699+0000 7f8116a00640 1 -- 172.21.15.119:0/898651394 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81100317a0 con 0x7f81180cf0e0 2024-04-25T10:07:02.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.699+0000 7f8116a00640 1 -- 172.21.15.119:0/898651394 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8110031b70 con 0x7f81180cf0e0 2024-04-25T10:07:02.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.699+0000 7f811ee00640 1 -- 172.21.15.119:0/898651394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81180cf0e0 msgr2=0x7f81180cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.699+0000 7f811ee00640 1 --2- 172.21.15.119:0/898651394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81180cf0e0 0x7f81180cc300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8110009d90 tx=0x7f811002a5d0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.699+0000 7f811ee00640 1 -- 172.21.15.119:0/898651394 shutdown_connections 2024-04-25T10:07:02.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.699+0000 7f811ee00640 1 --2- 172.21.15.119:0/898651394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81180cf0e0 0x7f81180cc300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.699+0000 7f811ee00640 1 --2- 172.21.15.119:0/898651394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8118144e70 0x7f811806d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.699+0000 7f811ee00640 1 -- 172.21.15.119:0/898651394 >> 172.21.15.119:0/898651394 conn(0x7f811806aba0 msgr2=0x7f811806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.699+0000 7f811ee00640 1 -- 172.21.15.119:0/898651394 shutdown_connections 2024-04-25T10:07:02.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.699+0000 7f811ee00640 1 -- 172.21.15.119:0/898651394 wait complete. 2024-04-25T10:07:02.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.701+0000 7f811ee00640 1 Processor -- start 2024-04-25T10:07:02.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.701+0000 7f811ee00640 1 -- start start 2024-04-25T10:07:02.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f811ee00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f811806d7a0 0x7f81181f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f811ee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81180cf0e0 0x7f81180b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f811ee00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8118144e70 0x7f811813e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f811ee00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8118145630 con 0x7f811806d7a0 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f811ee00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81181454b0 con 0x7f8118144e70 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f811ee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81180cc840 con 0x7f81180cf0e0 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f8117e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f811806d7a0 0x7f81181f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f8117e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f811806d7a0 0x7f81181f4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49172/0 (socket says 172.21.15.119:49172) 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f8117e00640 1 -- 172.21.15.119:0/3421317496 learned_addr learned my addr 172.21.15.119:0/3421317496 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f8117400640 1 --2- 172.21.15.119:0/3421317496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81180cf0e0 0x7f81180b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f8117e00640 1 -- 172.21.15.119:0/3421317496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81180cf0e0 msgr2=0x7f81180b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f8117e00640 1 --2- 172.21.15.119:0/3421317496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81180cf0e0 0x7f81180b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f8117e00640 1 -- 172.21.15.119:0/3421317496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8118144e70 msgr2=0x7f811813e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f8117e00640 1 --2- 172.21.15.119:0/3421317496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8118144e70 0x7f811813e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f8117e00640 1 -- 172.21.15.119:0/3421317496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81181f4a40 con 0x7f811806d7a0 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.702+0000 7f8117400640 1 --2- 172.21.15.119:0/3421317496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81180cf0e0 0x7f81180b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:02.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.703+0000 7f8117e00640 1 --2- 172.21.15.119:0/3421317496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f811806d7a0 0x7f81181f4500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f810800e870 tx=0x7f810800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.703+0000 7f810fe00640 1 -- 172.21.15.119:0/3421317496 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108017070 con 0x7f811806d7a0 2024-04-25T10:07:02.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.703+0000 7f810fe00640 1 -- 172.21.15.119:0/3421317496 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810800c8e0 con 0x7f811806d7a0 2024-04-25T10:07:02.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.703+0000 7f811ee00640 1 -- 172.21.15.119:0/3421317496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811813da40 con 0x7f811806d7a0 2024-04-25T10:07:02.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.703+0000 7f810fe00640 1 -- 172.21.15.119:0/3421317496 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810800ccd0 con 0x7f811806d7a0 2024-04-25T10:07:02.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.703+0000 7f811ee00640 1 -- 172.21.15.119:0/3421317496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811813dce0 con 0x7f811806d7a0 2024-04-25T10:07:02.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.704+0000 7f811ee00640 1 -- 172.21.15.119:0/3421317496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81180750b0 con 0x7f811806d7a0 2024-04-25T10:07:02.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.705+0000 7f810fe00640 1 -- 172.21.15.119:0/3421317496 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f810801a070 con 0x7f811806d7a0 2024-04-25T10:07:02.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.706+0000 7f810fe00640 1 --2- 172.21.15.119:0/3421317496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f80f806b520 0x7f80f806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.706+0000 7f810fe00640 1 -- 172.21.15.119:0/3421317496 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8108089ca0 con 0x7f811806d7a0 2024-04-25T10:07:02.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.706+0000 7f8117400640 1 --2- 172.21.15.119:0/3421317496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f80f806b520 0x7f80f806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.707+0000 7f8117400640 1 --2- 172.21.15.119:0/3421317496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f80f806b520 0x7f80f806d9e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f811806f2e0 tx=0x7f8104009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.710+0000 7f810fe00640 1 -- 172.21.15.119:0/3421317496 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81181f4a40 con 0x7f811806d7a0 2024-04-25T10:07:02.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.906+0000 7f811ee00640 1 -- 172.21.15.119:0/3421317496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f81180aebe0 con 0x7f811806d7a0 2024-04-25T10:07:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.907+0000 7f810fe00640 1 -- 172.21.15.119:0/3421317496 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8108058ee0 con 0x7f811806d7a0 2024-04-25T10:07:02.913 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:07:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.910+0000 7f811ee00640 1 -- 172.21.15.119:0/3421317496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f80f806b520 msgr2=0x7f80f806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.910+0000 7f811ee00640 1 --2- 172.21.15.119:0/3421317496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f80f806b520 0x7f80f806d9e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f811806f2e0 tx=0x7f8104009040 comp rx=0 tx=0).stop 2024-04-25T10:07:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.910+0000 7f811ee00640 1 -- 172.21.15.119:0/3421317496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f811806d7a0 msgr2=0x7f81181f4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.910+0000 7f811ee00640 1 --2- 172.21.15.119:0/3421317496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f811806d7a0 0x7f81181f4500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f810800e870 tx=0x7f810800ed30 comp rx=0 tx=0).stop 2024-04-25T10:07:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.910+0000 7f811ee00640 1 -- 172.21.15.119:0/3421317496 shutdown_connections 2024-04-25T10:07:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.910+0000 7f811ee00640 1 --2- 172.21.15.119:0/3421317496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f80f806b520 0x7f80f806d9e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.910+0000 7f811ee00640 1 --2- 172.21.15.119:0/3421317496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8118144e70 0x7f811813e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.910+0000 7f811ee00640 1 --2- 172.21.15.119:0/3421317496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f81180cf0e0 0x7f81180b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.910+0000 7f811ee00640 1 --2- 172.21.15.119:0/3421317496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f811806d7a0 0x7f81181f4500 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.910+0000 7f811ee00640 1 -- 172.21.15.119:0/3421317496 >> 172.21.15.119:0/3421317496 conn(0x7f811806aba0 msgr2=0x7f811813fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.910+0000 7f811ee00640 1 -- 172.21.15.119:0/3421317496 shutdown_connections 2024-04-25T10:07:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:02.910+0000 7f811ee00640 1 -- 172.21.15.119:0/3421317496 wait complete. 2024-04-25T10:07:02.933 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:03.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.124+0000 7fbd5f000640 1 Processor -- start 2024-04-25T10:07:03.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.124+0000 7fbd5f000640 1 -- start start 2024-04-25T10:07:03.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.126+0000 7fbd5f000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd58161db0 0x7fbd5806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.126+0000 7fbd5f000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbd580cb820 0x7fbd580cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.126+0000 7fbd5f000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd58166770 con 0x7fbd580b4b60 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.126+0000 7fbd5f000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5806e030 con 0x7fbd580cb820 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.126+0000 7fbd5f000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd581631a0 con 0x7fbd58161db0 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.127+0000 7fbd5dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd58161db0 0x7fbd5806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.127+0000 7fbd5dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd58161db0 0x7fbd5806e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:44414/0 (socket says 172.21.15.119:44414) 2024-04-25T10:07:03.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.127+0000 7fbd5dc00640 1 -- 172.21.15.119:0/4156735791 learned_addr learned my addr 172.21.15.119:0/4156735791 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:03.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.127+0000 7fbd5d200640 1 --2- 172.21.15.119:0/4156735791 >> v2:172.21.15.132:3300/0 conn(0x7fbd580cb820 0x7fbd580cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.128+0000 7fbd5dc00640 1 -- 172.21.15.119:0/4156735791 >> v2:172.21.15.132:3300/0 conn(0x7fbd580cb820 msgr2=0x7fbd580cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.128+0000 7fbd5dc00640 1 --2- 172.21.15.119:0/4156735791 >> v2:172.21.15.132:3300/0 conn(0x7fbd580cb820 0x7fbd580cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.128+0000 7fbd5dc00640 1 -- 172.21.15.119:0/4156735791 >> v1:172.21.15.132:6789/0 conn(0x7fbd580b4b60 legacy=0x7fbd580cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.128+0000 7fbd5dc00640 1 -- 172.21.15.119:0/4156735791 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5806d420 con 0x7fbd58161db0 2024-04-25T10:07:03.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.129+0000 7fbd5dc00640 1 --2- 172.21.15.119:0/4156735791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd58161db0 0x7fbd5806e4c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009870 tx=0x7fbd4c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=469fe06cba0fb3e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.129+0000 7fbd57e00640 1 -- 172.21.15.119:0/4156735791 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c037070 con 0x7fbd58161db0 2024-04-25T10:07:03.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.129+0000 7fbd57e00640 1 -- 172.21.15.119:0/4156735791 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c032440 con 0x7fbd58161db0 2024-04-25T10:07:03.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.129+0000 7fbd57e00640 1 -- 172.21.15.119:0/4156735791 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c032810 con 0x7fbd58161db0 2024-04-25T10:07:03.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.130+0000 7fbd5f000640 1 -- 172.21.15.119:0/4156735791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd58161db0 msgr2=0x7fbd5806e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.130+0000 7fbd5f000640 1 --2- 172.21.15.119:0/4156735791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd58161db0 0x7fbd5806e4c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009870 tx=0x7fbd4c02a270 comp rx=0 tx=0).stop 2024-04-25T10:07:03.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.130+0000 7fbd5f000640 1 -- 172.21.15.119:0/4156735791 shutdown_connections 2024-04-25T10:07:03.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.130+0000 7fbd5f000640 1 --2- 172.21.15.119:0/4156735791 >> v2:172.21.15.132:3300/0 conn(0x7fbd580cb820 0x7fbd580cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.130+0000 7fbd5f000640 1 --2- 172.21.15.119:0/4156735791 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd58161db0 0x7fbd5806e4c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.130+0000 7fbd5f000640 1 -- 172.21.15.119:0/4156735791 >> 172.21.15.119:0/4156735791 conn(0x7fbd5806b1d0 msgr2=0x7fbd580b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.130+0000 7fbd5f000640 1 -- 172.21.15.119:0/4156735791 shutdown_connections 2024-04-25T10:07:03.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.130+0000 7fbd5f000640 1 -- 172.21.15.119:0/4156735791 wait complete. 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.132+0000 7fbd5f000640 1 Processor -- start 2024-04-25T10:07:03.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.132+0000 7fbd5f000640 1 -- start start 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.133+0000 7fbd5f000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd580b4b60 0x7fbd581f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.133+0000 7fbd5f000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd580cb820 0x7fbd58155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.133+0000 7fbd5f000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd58161db0 0x7fbd581fae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.133+0000 7fbd5f000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5806dcc0 con 0x7fbd580cb820 2024-04-25T10:07:03.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.133+0000 7fbd5f000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5806d9e0 con 0x7fbd58161db0 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.133+0000 7fbd5f000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5806ea00 con 0x7fbd580b4b60 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.133+0000 7fbd5d200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd580cb820 0x7fbd58155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.133+0000 7fbd5d200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd580cb820 0x7fbd58155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49174/0 (socket says 172.21.15.119:49174) 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.133+0000 7fbd5d200640 1 -- 172.21.15.119:0/2848685936 learned_addr learned my addr 172.21.15.119:0/2848685936 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.134+0000 7fbd5e600640 1 --2- 172.21.15.119:0/2848685936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd58161db0 0x7fbd581fae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.134+0000 7fbd5d200640 1 -- 172.21.15.119:0/2848685936 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd580b4b60 msgr2=0x7fbd581f4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.134+0000 7fbd5d200640 1 --2- 172.21.15.119:0/2848685936 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd580b4b60 0x7fbd581f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.134+0000 7fbd5d200640 1 -- 172.21.15.119:0/2848685936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd58161db0 msgr2=0x7fbd581fae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.134+0000 7fbd5d200640 1 --2- 172.21.15.119:0/2848685936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd58161db0 0x7fbd581fae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.134+0000 7fbd5d200640 1 -- 172.21.15.119:0/2848685936 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5815fee0 con 0x7fbd580cb820 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.134+0000 7fbd5e600640 1 --2- 172.21.15.119:0/2848685936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd58161db0 0x7fbd581fae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:03.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.134+0000 7fbd5d200640 1 --2- 172.21.15.119:0/2848685936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd580cb820 0x7fbd58155e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400e9b0 tx=0x7fbd4400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.134+0000 7fbd55600640 1 -- 172.21.15.119:0/2848685936 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44017070 con 0x7fbd580cb820 2024-04-25T10:07:03.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.134+0000 7fbd55600640 1 -- 172.21.15.119:0/2848685936 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4400c9a0 con 0x7fbd580cb820 2024-04-25T10:07:03.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.134+0000 7fbd5f000640 1 -- 172.21.15.119:0/2848685936 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd58160180 con 0x7fbd580cb820 2024-04-25T10:07:03.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.134+0000 7fbd55600640 1 -- 172.21.15.119:0/2848685936 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400cd90 con 0x7fbd580cb820 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.134+0000 7fbd5f000640 1 -- 172.21.15.119:0/2848685936 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd58160740 con 0x7fbd580cb820 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.135+0000 7fbd5f000640 1 -- 172.21.15.119:0/2848685936 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5806bb30 con 0x7fbd580cb820 2024-04-25T10:07:03.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.136+0000 7fbd55600640 1 -- 172.21.15.119:0/2848685936 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd4401a070 con 0x7fbd580cb820 2024-04-25T10:07:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.137+0000 7fbd55600640 1 --2- 172.21.15.119:0/2848685936 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd3806b4d0 0x7fbd3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.137+0000 7fbd55600640 1 -- 172.21.15.119:0/2848685936 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbd44089f40 con 0x7fbd580cb820 2024-04-25T10:07:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.138+0000 7fbd5dc00640 1 --2- 172.21.15.119:0/2848685936 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd3806b4d0 0x7fbd3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.138+0000 7fbd5dc00640 1 --2- 172.21.15.119:0/2848685936 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd3806b4d0 0x7fbd3806d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c034000 tx=0x7fbd4c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.141+0000 7fbd55600640 1 -- 172.21.15.119:0/2848685936 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd44059070 con 0x7fbd580cb820 2024-04-25T10:07:03.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.379+0000 7fbd5f000640 1 -- 172.21.15.119:0/2848685936 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd580749b0 con 0x7fbd580cb820 2024-04-25T10:07:03.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.381+0000 7fbd55600640 1 -- 172.21.15.119:0/2848685936 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3703 (secure 0 0 0) 0x7fbd44058970 con 0x7fbd580cb820 2024-04-25T10:07:03.387 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:03.387 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1385207078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1385207078},{"type":"v1","addr":"172.21.15.132:6834","nonce":1385207078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1580923823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1580923823},{"type":"v1","addr":"172.21.15.119:6836","nonce":1580923823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4260,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/954053568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":954053568},{"type":"v1","addr":"172.21.15.132:6835","nonce":954053568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1632024048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1632024048},{"type":"v1","addr":"172.21.15.119:6837","nonce":1632024048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T10:07:03.387 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 3 2024-04-25T10:07:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.383+0000 7fbd5f000640 1 -- 172.21.15.119:0/2848685936 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd3806b4d0 msgr2=0x7fbd3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.383+0000 7fbd5f000640 1 --2- 172.21.15.119:0/2848685936 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd3806b4d0 0x7fbd3806d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c034000 tx=0x7fbd4c02c000 comp rx=0 tx=0).stop 2024-04-25T10:07:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.383+0000 7fbd5f000640 1 -- 172.21.15.119:0/2848685936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd580cb820 msgr2=0x7fbd58155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.383+0000 7fbd5f000640 1 --2- 172.21.15.119:0/2848685936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd580cb820 0x7fbd58155e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400e9b0 tx=0x7fbd4400ee70 comp rx=0 tx=0).stop 2024-04-25T10:07:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.384+0000 7fbd5f000640 1 -- 172.21.15.119:0/2848685936 shutdown_connections 2024-04-25T10:07:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.384+0000 7fbd5f000640 1 --2- 172.21.15.119:0/2848685936 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd3806b4d0 0x7fbd3806d990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.384+0000 7fbd5f000640 1 --2- 172.21.15.119:0/2848685936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd58161db0 0x7fbd581fae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.384+0000 7fbd5f000640 1 --2- 172.21.15.119:0/2848685936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd580cb820 0x7fbd58155e00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.384+0000 7fbd5f000640 1 --2- 172.21.15.119:0/2848685936 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd580b4b60 0x7fbd581f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.384+0000 7fbd5f000640 1 -- 172.21.15.119:0/2848685936 >> 172.21.15.119:0/2848685936 conn(0x7fbd5806b1d0 msgr2=0x7fbd580cd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.384+0000 7fbd5f000640 1 -- 172.21.15.119:0/2848685936 shutdown_connections 2024-04-25T10:07:03.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.384+0000 7fbd5f000640 1 -- 172.21.15.119:0/2848685936 wait complete. 2024-04-25T10:07:03.407 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:03.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.599+0000 7f94a7e00640 1 Processor -- start 2024-04-25T10:07:03.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.599+0000 7f94a7e00640 1 -- start start 2024-04-25T10:07:03.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.600+0000 7f94a7e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f94a806ea30 0x7f94a81653e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.601+0000 7f94a7e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a8144420 con 0x7f94a815acd0 2024-04-25T10:07:03.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.601+0000 7f94a7e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a80b3d80 con 0x7f94a8144040 2024-04-25T10:07:03.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.601+0000 7f94a7e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a80b36d0 con 0x7f94a806ea30 2024-04-25T10:07:03.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.601+0000 7f949e000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f94a815acd0 0x7f94a806e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56324/0 (socket says 172.21.15.119:56324) 2024-04-25T10:07:03.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.601+0000 7f949e000640 1 -- 172.21.15.119:0/3673244208 learned_addr learned my addr 172.21.15.119:0/3673244208 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:03.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.601+0000 7f94a6000640 1 -- 172.21.15.119:0/3673244208 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924268781 0 0) 0x7f94a8144420 con 0x7f94a815acd0 2024-04-25T10:07:03.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.601+0000 7f94a6000640 1 -- 172.21.15.119:0/3673244208 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9488003520 con 0x7f94a815acd0 2024-04-25T10:07:03.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.601+0000 7f94a7400640 1 --2- 172.21.15.119:0/3673244208 >> v2:172.21.15.132:3300/0 conn(0x7f94a806ea30 0x7f94a81653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f94a6000640 1 -- 172.21.15.119:0/3673244208 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3914219444 0 0) 0x7f9488003520 con 0x7f94a815acd0 2024-04-25T10:07:03.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f94a6000640 1 -- 172.21.15.119:0/3673244208 >> v2:172.21.15.132:3300/0 conn(0x7f94a806ea30 msgr2=0x7f94a81653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f94a6000640 1 --2- 172.21.15.119:0/3673244208 >> v2:172.21.15.132:3300/0 conn(0x7f94a806ea30 0x7f94a81653e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f94a6000640 1 -- 172.21.15.119:0/3673244208 >> v1:172.21.15.132:6789/0 conn(0x7f94a8144040 legacy=0x7f94a80cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f94a6000640 1 -- 172.21.15.119:0/3673244208 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a80b4060 con 0x7f94a815acd0 2024-04-25T10:07:03.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f94a6000640 1 -- 172.21.15.119:0/3673244208 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f9490001670 con 0x7f94a815acd0 2024-04-25T10:07:03.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f94a6000640 1 -- 172.21.15.119:0/3673244208 >> v1:172.21.15.119:6789/0 conn(0x7f94a815acd0 legacy=0x7f94a806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f94a6000640 1 --2- 172.21.15.119:0/3673244208 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9488003a70 0x7f9488003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f94a6000640 1 --2- 172.21.15.119:0/3673244208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9488004800 0x7f9488004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f94a6000640 1 --2- 172.21.15.119:0/3673244208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94880052e0 0x7f948800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f94a6000640 1 -- 172.21.15.119:0/3673244208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80b36d0 con 0x7f9488004800 2024-04-25T10:07:03.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f94a6000640 1 -- 172.21.15.119:0/3673244208 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948800c1b0 con 0x7f94880052e0 2024-04-25T10:07:03.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f94a6000640 1 -- 172.21.15.119:0/3673244208 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948800c3b0 con 0x7f9488003a70 2024-04-25T10:07:03.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f949e000640 1 --2- 172.21.15.119:0/3673244208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9488004800 0x7f9488004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.602+0000 7f94a7400640 1 --2- 172.21.15.119:0/3673244208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94880052e0 0x7f948800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.603+0000 7f949e000640 1 -- 172.21.15.119:0/3673244208 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9488003a70 msgr2=0x7f9488003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.603+0000 7f949e000640 1 --2- 172.21.15.119:0/3673244208 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9488003a70 0x7f9488003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.603+0000 7f949e000640 1 -- 172.21.15.119:0/3673244208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94880052e0 msgr2=0x7f948800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.603+0000 7f949e000640 1 --2- 172.21.15.119:0/3673244208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94880052e0 0x7f948800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.603+0000 7f949e000640 1 -- 172.21.15.119:0/3673244208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9488003520 con 0x7f9488004800 2024-04-25T10:07:03.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.603+0000 7f94a7400640 1 --2- 172.21.15.119:0/3673244208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94880052e0 0x7f948800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:03.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.603+0000 7f94a7400640 1 -- 172.21.15.119:0/3673244208 reap_dead start 2024-04-25T10:07:03.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.604+0000 7f949e000640 1 --2- 172.21.15.119:0/3673244208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9488004800 0x7f9488004c80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f949000c110 tx=0x7f9488005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf030dd3071f67d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.604+0000 7f94a6000640 1 -- 172.21.15.119:0/3673244208 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490007c40 con 0x7f9488004800 2024-04-25T10:07:03.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.604+0000 7f94a6000640 1 -- 172.21.15.119:0/3673244208 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9490005af0 con 0x7f9488004800 2024-04-25T10:07:03.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.604+0000 7f94a6000640 1 -- 172.21.15.119:0/3673244208 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490007070 con 0x7f9488004800 2024-04-25T10:07:03.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.604+0000 7f94a7e00640 1 -- 172.21.15.119:0/3673244208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9488004800 msgr2=0x7f9488004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.604+0000 7f94a7e00640 1 --2- 172.21.15.119:0/3673244208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9488004800 0x7f9488004c80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f949000c110 tx=0x7f9488005830 comp rx=0 tx=0).stop 2024-04-25T10:07:03.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.604+0000 7f94a7e00640 1 -- 172.21.15.119:0/3673244208 shutdown_connections 2024-04-25T10:07:03.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.604+0000 7f94a7e00640 1 --2- 172.21.15.119:0/3673244208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9488004800 0x7f9488004c80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.604+0000 7f94a7e00640 1 -- 172.21.15.119:0/3673244208 >> 172.21.15.119:0/3673244208 conn(0x7f94a806b1d0 msgr2=0x7f94a80b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.605+0000 7f94a7e00640 1 -- 172.21.15.119:0/3673244208 shutdown_connections 2024-04-25T10:07:03.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.605+0000 7f94a7e00640 1 -- 172.21.15.119:0/3673244208 wait complete. 2024-04-25T10:07:03.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.606+0000 7f94a7e00640 1 Processor -- start 2024-04-25T10:07:03.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.606+0000 7f94a7e00640 1 -- start start 2024-04-25T10:07:03.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.607+0000 7f94a7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9488004800 0x7f94a813ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.607+0000 7f94a7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a8141750 0x7f94a8141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.607+0000 7f94a7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94a8159d60 0x7f94a815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.607+0000 7f94a7e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80b4900 con 0x7f9488004800 2024-04-25T10:07:03.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.607+0000 7f94a7e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80b4620 con 0x7f94a8141750 2024-04-25T10:07:03.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.607+0000 7f94a7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80b5180 con 0x7f94a8159d60 2024-04-25T10:07:03.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.607+0000 7f94a6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9488004800 0x7f94a813ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.608+0000 7f94a6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9488004800 0x7f94a813ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49186/0 (socket says 172.21.15.119:49186) 2024-04-25T10:07:03.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.608+0000 7f94a6a00640 1 -- 172.21.15.119:0/1499226496 learned_addr learned my addr 172.21.15.119:0/1499226496 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:03.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.608+0000 7f94a6a00640 1 -- 172.21.15.119:0/1499226496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94a8159d60 msgr2=0x7f94a815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.608+0000 7f94a6a00640 1 --2- 172.21.15.119:0/1499226496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94a8159d60 0x7f94a815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.608+0000 7f94a6a00640 1 -- 172.21.15.119:0/1499226496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a8141750 msgr2=0x7f94a8141bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.608+0000 7f94a6a00640 1 --2- 172.21.15.119:0/1499226496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a8141750 0x7f94a8141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.608+0000 7f94a6a00640 1 -- 172.21.15.119:0/1499226496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a815a720 con 0x7f9488004800 2024-04-25T10:07:03.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.608+0000 7f94a6a00640 1 --2- 172.21.15.119:0/1499226496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9488004800 0x7f94a813ca90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f949800c3f0 tx=0x7f949800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.608+0000 7f949f400640 1 -- 172.21.15.119:0/1499226496 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94980052d0 con 0x7f9488004800 2024-04-25T10:07:03.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.608+0000 7f949f400640 1 -- 172.21.15.119:0/1499226496 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9498005490 con 0x7f9488004800 2024-04-25T10:07:03.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.608+0000 7f949f400640 1 -- 172.21.15.119:0/1499226496 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498007260 con 0x7f9488004800 2024-04-25T10:07:03.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.608+0000 7f94a7e00640 1 -- 172.21.15.119:0/1499226496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a8142210 con 0x7f9488004800 2024-04-25T10:07:03.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.609+0000 7f94a7e00640 1 -- 172.21.15.119:0/1499226496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a81427d0 con 0x7f9488004800 2024-04-25T10:07:03.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.610+0000 7f94a7e00640 1 -- 172.21.15.119:0/1499226496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a806bb30 con 0x7f9488004800 2024-04-25T10:07:03.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.611+0000 7f949f400640 1 -- 172.21.15.119:0/1499226496 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9498007400 con 0x7f9488004800 2024-04-25T10:07:03.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.612+0000 7f949f400640 1 --2- 172.21.15.119:0/1499226496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f948406b500 0x7f948406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.612+0000 7f949f400640 1 -- 172.21.15.119:0/1499226496 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9498089330 con 0x7f9488004800 2024-04-25T10:07:03.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.612+0000 7f949e000640 1 --2- 172.21.15.119:0/1499226496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f948406b500 0x7f948406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.612+0000 7f949e000640 1 --2- 172.21.15.119:0/1499226496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f948406b500 0x7f948406d9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9490032000 tx=0x7f9490007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.616+0000 7f949f400640 1 -- 172.21.15.119:0/1499226496 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9498058460 con 0x7f9488004800 2024-04-25T10:07:03.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.796+0000 7f94a7e00640 1 -- 172.21.15.119:0/1499226496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94a80cd0a0 con 0x7f9488004800 2024-04-25T10:07:03.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.799+0000 7f949f400640 1 -- 172.21.15.119:0/1499226496 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12920 (secure 0 0 0) 0x7f9498057d60 con 0x7f9488004800 2024-04-25T10:07:03.804 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:03.805 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:06:58.306787+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.801+0000 7f94a7e00640 1 -- 172.21.15.119:0/1499226496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f948406b500 msgr2=0x7f948406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.801+0000 7f94a7e00640 1 --2- 172.21.15.119:0/1499226496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f948406b500 0x7f948406d9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9490032000 tx=0x7f9490007000 comp rx=0 tx=0).stop 2024-04-25T10:07:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.801+0000 7f94a7e00640 1 -- 172.21.15.119:0/1499226496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9488004800 msgr2=0x7f94a813ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.801+0000 7f94a7e00640 1 --2- 172.21.15.119:0/1499226496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9488004800 0x7f94a813ca90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f949800c3f0 tx=0x7f949800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.801+0000 7f94a7e00640 1 -- 172.21.15.119:0/1499226496 shutdown_connections 2024-04-25T10:07:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.801+0000 7f94a7e00640 1 --2- 172.21.15.119:0/1499226496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f948406b500 0x7f948406d9c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.801+0000 7f94a7e00640 1 --2- 172.21.15.119:0/1499226496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94a8159d60 0x7f94a815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.801+0000 7f94a7e00640 1 --2- 172.21.15.119:0/1499226496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a8141750 0x7f94a8141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.801+0000 7f94a7e00640 1 --2- 172.21.15.119:0/1499226496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9488004800 0x7f94a813ca90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.801+0000 7f94a7e00640 1 -- 172.21.15.119:0/1499226496 >> 172.21.15.119:0/1499226496 conn(0x7f94a806b1d0 msgr2=0x7f94a80adb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.801+0000 7f94a7e00640 1 -- 172.21.15.119:0/1499226496 shutdown_connections 2024-04-25T10:07:03.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:03.801+0000 7f94a7e00640 1 -- 172.21.15.119:0/1499226496 wait complete. 2024-04-25T10:07:03.825 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:03.825 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:04.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.020+0000 7f8259c00640 1 Processor -- start 2024-04-25T10:07:04.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.020+0000 7f8259c00640 1 -- start start 2024-04-25T10:07:04.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.022+0000 7f8259c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f82541450e0 0x7f825406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.022+0000 7f8259c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825406b0e0 0x7f82540cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.022+0000 7f8259c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825415f400 con 0x7f82540cc810 2024-04-25T10:07:04.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.022+0000 7f8259c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82541454c0 con 0x7f82541450e0 2024-04-25T10:07:04.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.022+0000 7f8259c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825406b540 con 0x7f825406b0e0 2024-04-25T10:07:04.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.023+0000 7f8253e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f82541450e0 0x7f825406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.023+0000 7f8259200640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f82540cc810 0x7f8254071050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:46908/0 (socket says 172.21.15.119:46908) 2024-04-25T10:07:04.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.023+0000 7f8259200640 1 -- 172.21.15.119:0/1026349144 learned_addr learned my addr 172.21.15.119:0/1026349144 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:04.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.023+0000 7f8253400640 1 --2- 172.21.15.119:0/1026349144 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825406b0e0 0x7f82540cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.023+0000 7f8253e00640 1 -- 172.21.15.119:0/1026349144 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825406b0e0 msgr2=0x7f82540cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.023+0000 7f8253e00640 1 --2- 172.21.15.119:0/1026349144 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825406b0e0 0x7f82540cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.023+0000 7f8253e00640 1 -- 172.21.15.119:0/1026349144 >> v1:172.21.15.132:6789/0 conn(0x7f82540cc810 legacy=0x7f8254071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.023+0000 7f8253e00640 1 -- 172.21.15.119:0/1026349144 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825415f6e0 con 0x7f82541450e0 2024-04-25T10:07:04.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.025+0000 7f8253e00640 1 --2- 172.21.15.119:0/1026349144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82541450e0 0x7f825406aba0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f82440097a0 tx=0x7f824402a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed62af861e7f364 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.025+0000 7f8252a00640 1 -- 172.21.15.119:0/1026349144 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244037070 con 0x7f82541450e0 2024-04-25T10:07:04.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.025+0000 7f8252a00640 1 -- 172.21.15.119:0/1026349144 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824402adb0 con 0x7f82541450e0 2024-04-25T10:07:04.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.025+0000 7f8252a00640 1 -- 172.21.15.119:0/1026349144 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244032660 con 0x7f82541450e0 2024-04-25T10:07:04.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.026+0000 7f8259c00640 1 -- 172.21.15.119:0/1026349144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82541450e0 msgr2=0x7f825406aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.026+0000 7f8259c00640 1 --2- 172.21.15.119:0/1026349144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82541450e0 0x7f825406aba0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f82440097a0 tx=0x7f824402a060 comp rx=0 tx=0).stop 2024-04-25T10:07:04.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.026+0000 7f8259c00640 1 -- 172.21.15.119:0/1026349144 shutdown_connections 2024-04-25T10:07:04.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.026+0000 7f8259c00640 1 --2- 172.21.15.119:0/1026349144 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825406b0e0 0x7f82540cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.026+0000 7f8259c00640 1 --2- 172.21.15.119:0/1026349144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82541450e0 0x7f825406aba0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.026+0000 7f8259c00640 1 -- 172.21.15.119:0/1026349144 >> 172.21.15.119:0/1026349144 conn(0x7f8254066ff0 msgr2=0x7f82540b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.026+0000 7f8259c00640 1 -- 172.21.15.119:0/1026349144 shutdown_connections 2024-04-25T10:07:04.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.026+0000 7f8259c00640 1 -- 172.21.15.119:0/1026349144 wait complete. 2024-04-25T10:07:04.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.028+0000 7f8259c00640 1 Processor -- start 2024-04-25T10:07:04.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.028+0000 7f8259c00640 1 -- start start 2024-04-25T10:07:04.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.029+0000 7f8259c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825406b0e0 0x7f82541e5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.029+0000 7f8259c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82540cc810 0x7f8254142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.029+0000 7f8259c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82541450e0 0x7f82541f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.029+0000 7f8259c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82540ccca0 con 0x7f82541450e0 2024-04-25T10:07:04.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.029+0000 7f8259c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825415fca0 con 0x7f82540cc810 2024-04-25T10:07:04.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.029+0000 7f8259c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8254143bd0 con 0x7f825406b0e0 2024-04-25T10:07:04.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.029+0000 7f8259200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82541450e0 0x7f82541f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.029+0000 7f8259200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82541450e0 0x7f82541f3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49192/0 (socket says 172.21.15.119:49192) 2024-04-25T10:07:04.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.029+0000 7f8259200640 1 -- 172.21.15.119:0/2368856187 learned_addr learned my addr 172.21.15.119:0/2368856187 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:04.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.030+0000 7f8253e00640 1 --2- 172.21.15.119:0/2368856187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825406b0e0 0x7f82541e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.030+0000 7f8253400640 1 --2- 172.21.15.119:0/2368856187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82540cc810 0x7f8254142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.030+0000 7f8259200640 1 -- 172.21.15.119:0/2368856187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825406b0e0 msgr2=0x7f82541e5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.030+0000 7f8259200640 1 --2- 172.21.15.119:0/2368856187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825406b0e0 0x7f82541e5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.030+0000 7f8259200640 1 -- 172.21.15.119:0/2368856187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82540cc810 msgr2=0x7f8254142260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.030+0000 7f8259200640 1 --2- 172.21.15.119:0/2368856187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82540cc810 0x7f8254142260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.030+0000 7f8259200640 1 -- 172.21.15.119:0/2368856187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82541438f0 con 0x7f82541450e0 2024-04-25T10:07:04.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.030+0000 7f8259200640 1 --2- 172.21.15.119:0/2368856187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82541450e0 0x7f82541f3270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f824c00cbc0 tx=0x7f824c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.030+0000 7f824be00640 1 -- 172.21.15.119:0/2368856187 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c017070 con 0x7f82541450e0 2024-04-25T10:07:04.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.030+0000 7f824be00640 1 -- 172.21.15.119:0/2368856187 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c00ad40 con 0x7f82541450e0 2024-04-25T10:07:04.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.030+0000 7f8259c00640 1 -- 172.21.15.119:0/2368856187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8254141780 con 0x7f82541450e0 2024-04-25T10:07:04.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.030+0000 7f824be00640 1 -- 172.21.15.119:0/2368856187 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c012600 con 0x7f82541450e0 2024-04-25T10:07:04.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.030+0000 7f8259c00640 1 -- 172.21.15.119:0/2368856187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82541427a0 con 0x7f82541450e0 2024-04-25T10:07:04.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.032+0000 7f8259c00640 1 -- 172.21.15.119:0/2368856187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82540749b0 con 0x7f82541450e0 2024-04-25T10:07:04.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.033+0000 7f824be00640 1 -- 172.21.15.119:0/2368856187 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f824c01a070 con 0x7f82541450e0 2024-04-25T10:07:04.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.033+0000 7f824be00640 1 --2- 172.21.15.119:0/2368856187 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f823406b4d0 0x7f823406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.034+0000 7f824be00640 1 -- 172.21.15.119:0/2368856187 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f824c089b60 con 0x7f82541450e0 2024-04-25T10:07:04.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.034+0000 7f8253e00640 1 --2- 172.21.15.119:0/2368856187 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f823406b4d0 0x7f823406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.034+0000 7f8253e00640 1 --2- 172.21.15.119:0/2368856187 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f823406b4d0 0x7f823406d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8244034000 tx=0x7f824402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.038+0000 7f824be00640 1 -- 172.21.15.119:0/2368856187 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f824c0586e0 con 0x7f82541450e0 2024-04-25T10:07:04.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.229+0000 7f8259c00640 1 -- 172.21.15.119:0/2368856187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f825406c050 con 0x7f82541450e0 2024-04-25T10:07:04.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.230+0000 7f824be00640 1 -- 172.21.15.119:0/2368856187 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f824c05c780 con 0x7f82541450e0 2024-04-25T10:07:04.235 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.232+0000 7f8259c00640 1 -- 172.21.15.119:0/2368856187 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f823406b4d0 msgr2=0x7f823406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.232+0000 7f8259c00640 1 --2- 172.21.15.119:0/2368856187 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f823406b4d0 0x7f823406d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8244034000 tx=0x7f824402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.232+0000 7f8259c00640 1 -- 172.21.15.119:0/2368856187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82541450e0 msgr2=0x7f82541f3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.232+0000 7f8259c00640 1 --2- 172.21.15.119:0/2368856187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82541450e0 0x7f82541f3270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f824c00cbc0 tx=0x7f824c00a430 comp rx=0 tx=0).stop 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.232+0000 7f8259c00640 1 -- 172.21.15.119:0/2368856187 shutdown_connections 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.232+0000 7f8259c00640 1 --2- 172.21.15.119:0/2368856187 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f823406b4d0 0x7f823406d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.232+0000 7f8259c00640 1 --2- 172.21.15.119:0/2368856187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82541450e0 0x7f82541f3270 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.232+0000 7f8259c00640 1 --2- 172.21.15.119:0/2368856187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82540cc810 0x7f8254142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.232+0000 7f8259c00640 1 --2- 172.21.15.119:0/2368856187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f825406b0e0 0x7f82541e5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.232+0000 7f8259c00640 1 -- 172.21.15.119:0/2368856187 >> 172.21.15.119:0/2368856187 conn(0x7f8254066ff0 msgr2=0x7f82540b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.232+0000 7f8259c00640 1 -- 172.21.15.119:0/2368856187 shutdown_connections 2024-04-25T10:07:04.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.232+0000 7f8259c00640 1 -- 172.21.15.119:0/2368856187 wait complete. 2024-04-25T10:07:04.254 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:07:04.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.443+0000 7f4a28000640 1 Processor -- start 2024-04-25T10:07:04.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.443+0000 7f4a28000640 1 -- start start 2024-04-25T10:07:04.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.444+0000 7f4a28000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a200b15c0 0x7f4a2006d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.444+0000 7f4a28000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a2006d9d0 0x7f4a200cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.444+0000 7f4a28000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a200ca4a0 con 0x7f4a200b11e0 2024-04-25T10:07:04.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.444+0000 7f4a28000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a200ca250 con 0x7f4a200b15c0 2024-04-25T10:07:04.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.444+0000 7f4a28000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a200ca780 con 0x7f4a2006d9d0 2024-04-25T10:07:04.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.445+0000 7f4a24c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a200b15c0 0x7f4a2006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.445+0000 7f4a24c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a200b15c0 0x7f4a2006d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49204/0 (socket says 172.21.15.119:49204) 2024-04-25T10:07:04.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.445+0000 7f4a24c00640 1 -- 172.21.15.119:0/1089101692 learned_addr learned my addr 172.21.15.119:0/1089101692 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:04.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.445+0000 7f4a26000640 1 --2- 172.21.15.119:0/1089101692 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a2006d9d0 0x7f4a200cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.445+0000 7f4a24c00640 1 -- 172.21.15.119:0/1089101692 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a2006d9d0 msgr2=0x7f4a200cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.445+0000 7f4a24c00640 1 --2- 172.21.15.119:0/1089101692 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a2006d9d0 0x7f4a200cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.445+0000 7f4a24c00640 1 -- 172.21.15.119:0/1089101692 >> v1:172.21.15.132:6789/0 conn(0x7f4a200b11e0 legacy=0x7f4a2015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.445+0000 7f4a24c00640 1 -- 172.21.15.119:0/1089101692 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a200caa60 con 0x7f4a200b15c0 2024-04-25T10:07:04.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.446+0000 7f4a24c00640 1 --2- 172.21.15.119:0/1089101692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a200b15c0 0x7f4a2006d490 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009870 tx=0x7f4a0c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce655b358715856 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.446+0000 7f4a1fe00640 1 -- 172.21.15.119:0/1089101692 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c037070 con 0x7f4a200b15c0 2024-04-25T10:07:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.447+0000 7f4a1fe00640 1 -- 172.21.15.119:0/1089101692 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c032440 con 0x7f4a200b15c0 2024-04-25T10:07:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.447+0000 7f4a1fe00640 1 -- 172.21.15.119:0/1089101692 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c032810 con 0x7f4a200b15c0 2024-04-25T10:07:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.447+0000 7f4a28000640 1 -- 172.21.15.119:0/1089101692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a200b15c0 msgr2=0x7f4a2006d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.447+0000 7f4a28000640 1 --2- 172.21.15.119:0/1089101692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a200b15c0 0x7f4a2006d490 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009870 tx=0x7f4a0c02a250 comp rx=0 tx=0).stop 2024-04-25T10:07:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.447+0000 7f4a28000640 1 -- 172.21.15.119:0/1089101692 shutdown_connections 2024-04-25T10:07:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.447+0000 7f4a28000640 1 --2- 172.21.15.119:0/1089101692 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a2006d9d0 0x7f4a200cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.447+0000 7f4a28000640 1 --2- 172.21.15.119:0/1089101692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a200b15c0 0x7f4a2006d490 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.447+0000 7f4a28000640 1 -- 172.21.15.119:0/1089101692 >> 172.21.15.119:0/1089101692 conn(0x7f4a200cf5a0 msgr2=0x7f4a200cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.447+0000 7f4a28000640 1 -- 172.21.15.119:0/1089101692 shutdown_connections 2024-04-25T10:07:04.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.447+0000 7f4a28000640 1 -- 172.21.15.119:0/1089101692 wait complete. 2024-04-25T10:07:04.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.449+0000 7f4a28000640 1 Processor -- start 2024-04-25T10:07:04.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.449+0000 7f4a28000640 1 -- start start 2024-04-25T10:07:04.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.450+0000 7f4a28000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a2006d9d0 0x7f4a20149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.450+0000 7f4a28000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a200b11e0 0x7f4a200af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.450+0000 7f4a28000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a200b15c0 0x7f4a200afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.450+0000 7f4a28000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a201597b0 con 0x7f4a2006d9d0 2024-04-25T10:07:04.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.450+0000 7f4a28000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a200b1a30 con 0x7f4a200b15c0 2024-04-25T10:07:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.450+0000 7f4a28000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a20159930 con 0x7f4a200b11e0 2024-04-25T10:07:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.450+0000 7f4a25600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a2006d9d0 0x7f4a20149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.450+0000 7f4a25600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a2006d9d0 0x7f4a20149f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49216/0 (socket says 172.21.15.119:49216) 2024-04-25T10:07:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.450+0000 7f4a25600640 1 -- 172.21.15.119:0/3334475191 learned_addr learned my addr 172.21.15.119:0/3334475191 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.451+0000 7f4a24c00640 1 --2- 172.21.15.119:0/3334475191 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a200b11e0 0x7f4a200af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.451+0000 7f4a26000640 1 --2- 172.21.15.119:0/3334475191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a200b15c0 0x7f4a200afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.451+0000 7f4a25600640 1 -- 172.21.15.119:0/3334475191 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a200b11e0 msgr2=0x7f4a200af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.451+0000 7f4a25600640 1 --2- 172.21.15.119:0/3334475191 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a200b11e0 0x7f4a200af6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.451+0000 7f4a25600640 1 -- 172.21.15.119:0/3334475191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a200b15c0 msgr2=0x7f4a200afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.451+0000 7f4a25600640 1 --2- 172.21.15.119:0/3334475191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a200b15c0 0x7f4a200afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.451+0000 7f4a25600640 1 -- 172.21.15.119:0/3334475191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a201568d0 con 0x7f4a2006d9d0 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.451+0000 7f4a25600640 1 --2- 172.21.15.119:0/3334475191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a2006d9d0 0x7f4a20149f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400c450 tx=0x7f4a1400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.451+0000 7f4a1d600640 1 -- 172.21.15.119:0/3334475191 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140052d0 con 0x7f4a2006d9d0 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.451+0000 7f4a1d600640 1 -- 172.21.15.119:0/3334475191 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a14005470 con 0x7f4a2006d9d0 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.451+0000 7f4a28000640 1 -- 172.21.15.119:0/3334475191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a20156bd0 con 0x7f4a2006d9d0 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.452+0000 7f4a1d600640 1 -- 172.21.15.119:0/3334475191 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14012870 con 0x7f4a2006d9d0 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.452+0000 7f4a28000640 1 -- 172.21.15.119:0/3334475191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a20157410 con 0x7f4a2006d9d0 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.453+0000 7f4a28000640 1 -- 172.21.15.119:0/3334475191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2006dde0 con 0x7f4a2006d9d0 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.454+0000 7f4a1d600640 1 -- 172.21.15.119:0/3334475191 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a14012a10 con 0x7f4a2006d9d0 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.455+0000 7f4a1d600640 1 --2- 172.21.15.119:0/3334475191 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4a0006b500 0x7f4a0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.455+0000 7f4a1d600640 1 -- 172.21.15.119:0/3334475191 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4a14089be0 con 0x7f4a2006d9d0 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.455+0000 7f4a24c00640 1 --2- 172.21.15.119:0/3334475191 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4a0006b500 0x7f4a0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.456+0000 7f4a24c00640 1 --2- 172.21.15.119:0/3334475191 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4a0006b500 0x7f4a0006d9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c034000 tx=0x7f4a0c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.459+0000 7f4a1d600640 1 -- 172.21.15.119:0/3334475191 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a2006dde0 con 0x7f4a2006d9d0 2024-04-25T10:07:04.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.656+0000 7f4a28000640 1 -- 172.21.15.119:0/3334475191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4a200749b0 con 0x7f4a2006d9d0 2024-04-25T10:07:04.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.657+0000 7f4a1d600640 1 -- 172.21.15.119:0/3334475191 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4a20156bd0 con 0x7f4a2006d9d0 2024-04-25T10:07:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:04.663 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2024-04-25T10:07:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:07:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2024-04-25T10:07:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2024-04-25T10:07:04.664 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:07:04.664 INFO:teuthology.orchestra.run.smithi119.stdout:] 2024-04-25T10:07:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.660+0000 7f4a28000640 1 -- 172.21.15.119:0/3334475191 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4a0006b500 msgr2=0x7f4a0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.660+0000 7f4a28000640 1 --2- 172.21.15.119:0/3334475191 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4a0006b500 0x7f4a0006d9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c034000 tx=0x7f4a0c02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.660+0000 7f4a28000640 1 -- 172.21.15.119:0/3334475191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a2006d9d0 msgr2=0x7f4a20149f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.660+0000 7f4a28000640 1 --2- 172.21.15.119:0/3334475191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a2006d9d0 0x7f4a20149f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400c450 tx=0x7f4a1400c910 comp rx=0 tx=0).stop 2024-04-25T10:07:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.660+0000 7f4a28000640 1 -- 172.21.15.119:0/3334475191 shutdown_connections 2024-04-25T10:07:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.660+0000 7f4a28000640 1 --2- 172.21.15.119:0/3334475191 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4a0006b500 0x7f4a0006d9c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.660+0000 7f4a28000640 1 --2- 172.21.15.119:0/3334475191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a200b15c0 0x7f4a200afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.660+0000 7f4a28000640 1 --2- 172.21.15.119:0/3334475191 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a200b11e0 0x7f4a200af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.660+0000 7f4a28000640 1 --2- 172.21.15.119:0/3334475191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a2006d9d0 0x7f4a20149f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.660+0000 7f4a28000640 1 -- 172.21.15.119:0/3334475191 >> 172.21.15.119:0/3334475191 conn(0x7f4a200cf5a0 msgr2=0x7f4a200b5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.660+0000 7f4a28000640 1 -- 172.21.15.119:0/3334475191 shutdown_connections 2024-04-25T10:07:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.660+0000 7f4a28000640 1 -- 172.21.15.119:0/3334475191 wait complete. 2024-04-25T10:07:04.683 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:07:04.683 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:07:04.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.871+0000 7f6c2b600640 1 Processor -- start 2024-04-25T10:07:04.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.871+0000 7f6c2b600640 1 -- start start 2024-04-25T10:07:04.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.873+0000 7f6c2b600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c2415dec0 0x7f6c240d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.873+0000 7f6c2b600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6c240d0b60 0x7f6c240b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.873+0000 7f6c2b600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c240d2760 con 0x7f6c240d22e0 2024-04-25T10:07:04.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.873+0000 7f6c2b600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c240d2a20 con 0x7f6c240d0b60 2024-04-25T10:07:04.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.873+0000 7f6c2b600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240d2d00 con 0x7f6c2415dec0 2024-04-25T10:07:04.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.873+0000 7f6c23e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6c240d22e0 0x7f6c240d0450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56332/0 (socket says 172.21.15.119:56332) 2024-04-25T10:07:04.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.873+0000 7f6c23e00640 1 -- 172.21.15.119:0/3601617115 learned_addr learned my addr 172.21.15.119:0/3601617115 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:04.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.873+0000 7f6c23400640 1 -- 172.21.15.119:0/3601617115 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129062713 0 0) 0x7f6c240d2760 con 0x7f6c240d22e0 2024-04-25T10:07:04.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.873+0000 7f6c23400640 1 -- 172.21.15.119:0/3601617115 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c04003520 con 0x7f6c240d22e0 2024-04-25T10:07:04.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.873+0000 7f6c28c00640 1 --2- 172.21.15.119:0/3601617115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c2415dec0 0x7f6c240d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.873+0000 7f6c29600640 1 --2- 172.21.15.119:0/3601617115 >> v2:172.21.15.132:3300/0 conn(0x7f6c240d0b60 0x7f6c240b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c23400640 1 -- 172.21.15.119:0/3601617115 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 701602423 0 0) 0x7f6c04003520 con 0x7f6c240d22e0 2024-04-25T10:07:04.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c23400640 1 -- 172.21.15.119:0/3601617115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c2415dec0 msgr2=0x7f6c240d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c23400640 1 --2- 172.21.15.119:0/3601617115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c2415dec0 0x7f6c240d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c23400640 1 -- 172.21.15.119:0/3601617115 >> v2:172.21.15.132:3300/0 conn(0x7f6c240d0b60 msgr2=0x7f6c240b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c23400640 1 --2- 172.21.15.119:0/3601617115 >> v2:172.21.15.132:3300/0 conn(0x7f6c240d0b60 0x7f6c240b6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c23400640 1 -- 172.21.15.119:0/3601617115 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c240d2fe0 con 0x7f6c240d22e0 2024-04-25T10:07:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c23400640 1 -- 172.21.15.119:0/3601617115 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f6c0c0019f0 con 0x7f6c240d22e0 2024-04-25T10:07:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c23400640 1 -- 172.21.15.119:0/3601617115 >> v1:172.21.15.119:6789/0 conn(0x7f6c240d22e0 legacy=0x7f6c240d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c23400640 1 --2- 172.21.15.119:0/3601617115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c040039b0 0x7f6c04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c23400640 1 --2- 172.21.15.119:0/3601617115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c04004800 0x7f6c04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c23400640 1 --2- 172.21.15.119:0/3601617115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c2415dec0 0x7f6c040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c23400640 1 -- 172.21.15.119:0/3601617115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240d2a20 con 0x7f6c040039b0 2024-04-25T10:07:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c23400640 1 -- 172.21.15.119:0/3601617115 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240d2d00 con 0x7f6c04004800 2024-04-25T10:07:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c23400640 1 -- 172.21.15.119:0/3601617115 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0400ba70 con 0x7f6c2415dec0 2024-04-25T10:07:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.874+0000 7f6c28c00640 1 --2- 172.21.15.119:0/3601617115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c040039b0 0x7f6c04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.875+0000 7f6c23e00640 1 --2- 172.21.15.119:0/3601617115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c04004800 0x7f6c04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.875+0000 7f6c28c00640 1 -- 172.21.15.119:0/3601617115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c2415dec0 msgr2=0x7f6c040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.875+0000 7f6c28c00640 1 --2- 172.21.15.119:0/3601617115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c2415dec0 0x7f6c040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.875+0000 7f6c28c00640 1 -- 172.21.15.119:0/3601617115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c04004800 msgr2=0x7f6c04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.875+0000 7f6c28c00640 1 --2- 172.21.15.119:0/3601617115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c04004800 0x7f6c04004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.875+0000 7f6c28c00640 1 -- 172.21.15.119:0/3601617115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c04003520 con 0x7f6c040039b0 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.875+0000 7f6c23e00640 1 --2- 172.21.15.119:0/3601617115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c04004800 0x7f6c04004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.876+0000 7f6c28c00640 1 --2- 172.21.15.119:0/3601617115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c040039b0 0x7f6c04003dd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400ba00 tx=0x7f6c04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47a37cb75e4adc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.876+0000 7f6c23400640 1 -- 172.21.15.119:0/3601617115 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14030070 con 0x7f6c040039b0 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.876+0000 7f6c23400640 1 -- 172.21.15.119:0/3601617115 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1402adb0 con 0x7f6c040039b0 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.876+0000 7f6c23400640 1 -- 172.21.15.119:0/3601617115 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400b600 con 0x7f6c040039b0 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.876+0000 7f6c2b600640 1 -- 172.21.15.119:0/3601617115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c040039b0 msgr2=0x7f6c04003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.876+0000 7f6c2b600640 1 --2- 172.21.15.119:0/3601617115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c040039b0 0x7f6c04003dd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400ba00 tx=0x7f6c04005830 comp rx=0 tx=0).stop 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.876+0000 7f6c29600640 1 -- 172.21.15.119:0/3601617115 reap_dead start 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.876+0000 7f6c2b600640 1 -- 172.21.15.119:0/3601617115 shutdown_connections 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.876+0000 7f6c2b600640 1 -- 172.21.15.119:0/3601617115 >> 172.21.15.119:0/3601617115 conn(0x7f6c24072900 msgr2=0x7f6c240b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.876+0000 7f6c2b600640 1 -- 172.21.15.119:0/3601617115 shutdown_connections 2024-04-25T10:07:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.877+0000 7f6c2b600640 1 -- 172.21.15.119:0/3601617115 wait complete. 2024-04-25T10:07:04.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.878+0000 7f6c2b600640 1 Processor -- start 2024-04-25T10:07:04.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.878+0000 7f6c2b600640 1 -- start start 2024-04-25T10:07:04.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.879+0000 7f6c2b600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c040039b0 0x7f6c240cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.879+0000 7f6c2b600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c241462a0 0x7f6c24160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.879+0000 7f6c2b600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c24145d80 0x7f6c24160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.879+0000 7f6c2b600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240d1da0 con 0x7f6c24145d80 2024-04-25T10:07:04.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.879+0000 7f6c2b600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240d35a0 con 0x7f6c040039b0 2024-04-25T10:07:04.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.879+0000 7f6c2b600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240d1f20 con 0x7f6c241462a0 2024-04-25T10:07:04.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.879+0000 7f6c29600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c24145d80 0x7f6c24160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.879+0000 7f6c29600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c24145d80 0x7f6c24160840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49238/0 (socket says 172.21.15.119:49238) 2024-04-25T10:07:04.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.879+0000 7f6c29600640 1 -- 172.21.15.119:0/721350655 learned_addr learned my addr 172.21.15.119:0/721350655 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:04.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.880+0000 7f6c29600640 1 -- 172.21.15.119:0/721350655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c241462a0 msgr2=0x7f6c24160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.880+0000 7f6c29600640 1 --2- 172.21.15.119:0/721350655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c241462a0 0x7f6c24160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.880+0000 7f6c29600640 1 -- 172.21.15.119:0/721350655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c040039b0 msgr2=0x7f6c240cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.880+0000 7f6c29600640 1 --2- 172.21.15.119:0/721350655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c040039b0 0x7f6c240cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.880+0000 7f6c29600640 1 -- 172.21.15.119:0/721350655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c240cc5d0 con 0x7f6c24145d80 2024-04-25T10:07:04.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.880+0000 7f6c29600640 1 --2- 172.21.15.119:0/721350655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c24145d80 0x7f6c24160840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c240ce140 tx=0x7f6c1c00d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.880+0000 7f6c20c00640 1 -- 172.21.15.119:0/721350655 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c004020 con 0x7f6c24145d80 2024-04-25T10:07:04.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.880+0000 7f6c20c00640 1 -- 172.21.15.119:0/721350655 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c0041c0 con 0x7f6c24145d80 2024-04-25T10:07:04.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.880+0000 7f6c20c00640 1 -- 172.21.15.119:0/721350655 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0043b0 con 0x7f6c24145d80 2024-04-25T10:07:04.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.880+0000 7f6c2b600640 1 -- 172.21.15.119:0/721350655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c240cd900 con 0x7f6c24145d80 2024-04-25T10:07:04.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.881+0000 7f6c2b600640 1 -- 172.21.15.119:0/721350655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c240cb890 con 0x7f6c24145d80 2024-04-25T10:07:04.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.881+0000 7f6c2b600640 1 -- 172.21.15.119:0/721350655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c240750b0 con 0x7f6c24145d80 2024-04-25T10:07:04.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.882+0000 7f6c20c00640 1 -- 172.21.15.119:0/721350655 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c1c01a020 con 0x7f6c24145d80 2024-04-25T10:07:04.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.883+0000 7f6c20c00640 1 --2- 172.21.15.119:0/721350655 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6c0006b500 0x7f6c0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.883+0000 7f6c20c00640 1 -- 172.21.15.119:0/721350655 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6c1c089470 con 0x7f6c24145d80 2024-04-25T10:07:04.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.884+0000 7f6c28c00640 1 --2- 172.21.15.119:0/721350655 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6c0006b500 0x7f6c0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.884+0000 7f6c28c00640 1 --2- 172.21.15.119:0/721350655 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6c0006b500 0x7f6c0006d9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c040057e0 tx=0x7f6c1402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:04.888+0000 7f6c20c00640 1 -- 172.21.15.119:0/721350655 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c1c0586d0 con 0x7f6c24145d80 2024-04-25T10:07:05.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.083+0000 7f6c2b600640 1 -- 172.21.15.119:0/721350655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6c2415a070 con 0x7f6c24145d80 2024-04-25T10:07:05.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.273+0000 7f6c20c00640 1 -- 172.21.15.119:0/721350655 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6c1c057fd0 con 0x7f6c24145d80 2024-04-25T10:07:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2024-04-25T10:07:05.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.278+0000 7f6c2b600640 1 -- 172.21.15.119:0/721350655 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6c0006b500 msgr2=0x7f6c0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.278+0000 7f6c2b600640 1 --2- 172.21.15.119:0/721350655 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6c0006b500 0x7f6c0006d9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c040057e0 tx=0x7f6c1402c000 comp rx=0 tx=0).stop 2024-04-25T10:07:05.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.278+0000 7f6c2b600640 1 -- 172.21.15.119:0/721350655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c24145d80 msgr2=0x7f6c24160840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.278+0000 7f6c2b600640 1 --2- 172.21.15.119:0/721350655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c24145d80 0x7f6c24160840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c240ce140 tx=0x7f6c1c00d820 comp rx=0 tx=0).stop 2024-04-25T10:07:05.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.278+0000 7f6c2b600640 1 -- 172.21.15.119:0/721350655 shutdown_connections 2024-04-25T10:07:05.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.278+0000 7f6c2b600640 1 --2- 172.21.15.119:0/721350655 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6c0006b500 0x7f6c0006d9c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.278+0000 7f6c2b600640 1 --2- 172.21.15.119:0/721350655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c24145d80 0x7f6c24160840 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.278+0000 7f6c2b600640 1 --2- 172.21.15.119:0/721350655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c241462a0 0x7f6c24160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.278+0000 7f6c2b600640 1 --2- 172.21.15.119:0/721350655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c040039b0 0x7f6c240cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.278+0000 7f6c2b600640 1 -- 172.21.15.119:0/721350655 >> 172.21.15.119:0/721350655 conn(0x7f6c24072900 msgr2=0x7f6c240ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.279+0000 7f6c2b600640 1 -- 172.21.15.119:0/721350655 shutdown_connections 2024-04-25T10:07:05.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.279+0000 7f6c2b600640 1 -- 172.21.15.119:0/721350655 wait complete. 2024-04-25T10:07:05.310 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:07:05.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.500+0000 7f9626c00640 1 Processor -- start 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.501+0000 7f9626c00640 1 -- start start 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.502+0000 7f9626c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962015a850 0x7f962015e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.502+0000 7f9626c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f96200cd790 0x7f96200cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.502+0000 7f9626c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96200b26f0 con 0x7f962006e400 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.502+0000 7f9626c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96200b29b0 con 0x7f96200cd790 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.502+0000 7f9626c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96200b2c90 con 0x7f962015a850 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.503+0000 7f961fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962015a850 0x7f962015e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.503+0000 7f961fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962015a850 0x7f962015e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:44492/0 (socket says 172.21.15.119:44492) 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.503+0000 7f9624c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f96200cd790 0x7f96200cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.503+0000 7f961fe00640 1 -- 172.21.15.119:0/2508090389 learned_addr learned my addr 172.21.15.119:0/2508090389 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.504+0000 7f961fe00640 1 -- 172.21.15.119:0/2508090389 >> v2:172.21.15.132:3300/0 conn(0x7f96200cd790 msgr2=0x7f96200cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.504+0000 7f961fe00640 1 --2- 172.21.15.119:0/2508090389 >> v2:172.21.15.132:3300/0 conn(0x7f96200cd790 0x7f96200cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.504+0000 7f961fe00640 1 -- 172.21.15.119:0/2508090389 >> v1:172.21.15.132:6789/0 conn(0x7f962006e400 legacy=0x7f962015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.504+0000 7f961fe00640 1 -- 172.21.15.119:0/2508090389 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96200b2f70 con 0x7f962015a850 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.504+0000 7f9624c00640 1 --2- 172.21.15.119:0/2508090389 >> v2:172.21.15.132:3300/0 conn(0x7f96200cd790 0x7f96200cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:05.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.505+0000 7f961fe00640 1 --2- 172.21.15.119:0/2508090389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962015a850 0x7f962015e490 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9610009e50 tx=0x7f961002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f067be0e944a593 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.506+0000 7f961f400640 1 -- 172.21.15.119:0/2508090389 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610036070 con 0x7f962015a850 2024-04-25T10:07:05.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.506+0000 7f961f400640 1 -- 172.21.15.119:0/2508090389 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9610031600 con 0x7f962015a850 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.506+0000 7f961f400640 1 -- 172.21.15.119:0/2508090389 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96100319d0 con 0x7f962015a850 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.506+0000 7f9626c00640 1 -- 172.21.15.119:0/2508090389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962015a850 msgr2=0x7f962015e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.506+0000 7f9626c00640 1 --2- 172.21.15.119:0/2508090389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962015a850 0x7f962015e490 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9610009e50 tx=0x7f961002a680 comp rx=0 tx=0).stop 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.507+0000 7f9626c00640 1 -- 172.21.15.119:0/2508090389 shutdown_connections 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.507+0000 7f9626c00640 1 --2- 172.21.15.119:0/2508090389 >> v2:172.21.15.132:3300/0 conn(0x7f96200cd790 0x7f96200cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.507+0000 7f9626c00640 1 --2- 172.21.15.119:0/2508090389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962015a850 0x7f962015e490 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.507+0000 7f9626c00640 1 -- 172.21.15.119:0/2508090389 >> 172.21.15.119:0/2508090389 conn(0x7f962006aba0 msgr2=0x7f962006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.507+0000 7f9626c00640 1 -- 172.21.15.119:0/2508090389 shutdown_connections 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.507+0000 7f9626c00640 1 -- 172.21.15.119:0/2508090389 wait complete. 2024-04-25T10:07:05.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.508+0000 7f9626c00640 1 Processor -- start 2024-04-25T10:07:05.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.509+0000 7f9626c00640 1 -- start start 2024-04-25T10:07:05.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.509+0000 7f9626c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962006e400 0x7f962015cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f9626c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96200cd790 0x7f96200ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f9626c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f962015a850 0x7f96200acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f9626c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962015ac30 con 0x7f96200cd790 2024-04-25T10:07:05.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f9626c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96200af050 con 0x7f962015a850 2024-04-25T10:07:05.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f9626c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962015f0e0 con 0x7f962006e400 2024-04-25T10:07:05.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f9617400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96200cd790 0x7f96200ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f9617400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96200cd790 0x7f96200ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49242/0 (socket says 172.21.15.119:49242) 2024-04-25T10:07:05.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f9617400640 1 -- 172.21.15.119:0/3728105257 learned_addr learned my addr 172.21.15.119:0/3728105257 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:05.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f9624c00640 1 --2- 172.21.15.119:0/3728105257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f962015a850 0x7f96200acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f961fe00640 1 --2- 172.21.15.119:0/3728105257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962006e400 0x7f962015cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f9617400640 1 -- 172.21.15.119:0/3728105257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962006e400 msgr2=0x7f962015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f9617400640 1 --2- 172.21.15.119:0/3728105257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962006e400 0x7f962015cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f9617400640 1 -- 172.21.15.119:0/3728105257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f962015a850 msgr2=0x7f96200acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f9617400640 1 --2- 172.21.15.119:0/3728105257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f962015a850 0x7f96200acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.510+0000 7f9617400640 1 -- 172.21.15.119:0/3728105257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96201f37c0 con 0x7f96200cd790 2024-04-25T10:07:05.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.511+0000 7f9617400640 1 --2- 172.21.15.119:0/3728105257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96200cd790 0x7f96200ac8b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f960800c400 tx=0x7f960800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.511+0000 7f961cc00640 1 -- 172.21.15.119:0/3728105257 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96080052d0 con 0x7f96200cd790 2024-04-25T10:07:05.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.511+0000 7f961cc00640 1 -- 172.21.15.119:0/3728105257 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9608005490 con 0x7f96200cd790 2024-04-25T10:07:05.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.511+0000 7f9626c00640 1 -- 172.21.15.119:0/3728105257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96201f3aa0 con 0x7f96200cd790 2024-04-25T10:07:05.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.511+0000 7f961cc00640 1 -- 172.21.15.119:0/3728105257 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608007260 con 0x7f96200cd790 2024-04-25T10:07:05.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.511+0000 7f9626c00640 1 -- 172.21.15.119:0/3728105257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962015d3e0 con 0x7f96200cd790 2024-04-25T10:07:05.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.512+0000 7f9626c00640 1 -- 172.21.15.119:0/3728105257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96200749b0 con 0x7f96200cd790 2024-04-25T10:07:05.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.514+0000 7f961cc00640 1 -- 172.21.15.119:0/3728105257 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9608007400 con 0x7f96200cd790 2024-04-25T10:07:05.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.514+0000 7f961cc00640 1 --2- 172.21.15.119:0/3728105257 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f95fc06b4d0 0x7f95fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.515+0000 7f961cc00640 1 -- 172.21.15.119:0/3728105257 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9608089d90 con 0x7f96200cd790 2024-04-25T10:07:05.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.515+0000 7f961fe00640 1 --2- 172.21.15.119:0/3728105257 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f95fc06b4d0 0x7f95fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.515+0000 7f961fe00640 1 --2- 172.21.15.119:0/3728105257 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f95fc06b4d0 0x7f95fc06d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9610033000 tx=0x7f9610009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.518+0000 7f961cc00640 1 -- 172.21.15.119:0/3728105257 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9608058e60 con 0x7f96200cd790 2024-04-25T10:07:05.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:05.717+0000 7f9626c00640 1 -- 172.21.15.119:0/3728105257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f96200cc580 con 0x7f96200cd790 2024-04-25T10:07:06.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.281+0000 7f961cc00640 1 -- 172.21.15.119:0/3728105257 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9608058760 con 0x7f96200cd790 2024-04-25T10:07:06.287 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2024-04-25T10:07:06.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.285+0000 7f9626c00640 1 -- 172.21.15.119:0/3728105257 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f95fc06b4d0 msgr2=0x7f95fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.285+0000 7f9626c00640 1 --2- 172.21.15.119:0/3728105257 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f95fc06b4d0 0x7f95fc06d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9610033000 tx=0x7f9610009000 comp rx=0 tx=0).stop 2024-04-25T10:07:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.285+0000 7f9626c00640 1 -- 172.21.15.119:0/3728105257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96200cd790 msgr2=0x7f96200ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.285+0000 7f9626c00640 1 --2- 172.21.15.119:0/3728105257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96200cd790 0x7f96200ac8b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f960800c400 tx=0x7f960800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.285+0000 7f9626c00640 1 -- 172.21.15.119:0/3728105257 shutdown_connections 2024-04-25T10:07:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.285+0000 7f9626c00640 1 --2- 172.21.15.119:0/3728105257 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f95fc06b4d0 0x7f95fc06d990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.285+0000 7f9626c00640 1 --2- 172.21.15.119:0/3728105257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f962015a850 0x7f96200acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.286+0000 7f9626c00640 1 --2- 172.21.15.119:0/3728105257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96200cd790 0x7f96200ac8b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.286+0000 7f9626c00640 1 --2- 172.21.15.119:0/3728105257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962006e400 0x7f962015cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.286+0000 7f9626c00640 1 -- 172.21.15.119:0/3728105257 >> 172.21.15.119:0/3728105257 conn(0x7f962006aba0 msgr2=0x7f96200ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:06.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.286+0000 7f9626c00640 1 -- 172.21.15.119:0/3728105257 shutdown_connections 2024-04-25T10:07:06.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.286+0000 7f9626c00640 1 -- 172.21.15.119:0/3728105257 wait complete. 2024-04-25T10:07:06.324 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:07:06.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.495+0000 7f2dc1000640 1 Processor -- start 2024-04-25T10:07:06.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.495+0000 7f2dc1000640 1 -- start start 2024-04-25T10:07:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.496+0000 7f2dc1000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2dbc0ce9d0 0x7f2dbc0cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.496+0000 7f2dc1000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dbc06dcb0 con 0x7f2dbc144e70 2024-04-25T10:07:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.496+0000 7f2dc1000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dbc06df90 con 0x7f2dbc0cc300 2024-04-25T10:07:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.496+0000 7f2dc1000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dbc15bfe0 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.497+0000 7f2db9e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2dbc144e70 0x7f2dbc06d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56348/0 (socket says 172.21.15.119:56348) 2024-04-25T10:07:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.497+0000 7f2db9e00640 1 -- 172.21.15.119:0/450358016 learned_addr learned my addr 172.21.15.119:0/450358016 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.497+0000 7f2db3e00640 1 --2- 172.21.15.119:0/450358016 >> v2:172.21.15.119:3300/0 conn(0x7f2dbc0ce9d0 0x7f2dbc0cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.497+0000 7f2db9400640 1 -- 172.21.15.119:0/450358016 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284027995 0 0) 0x7f2dbc06dcb0 con 0x7f2dbc144e70 2024-04-25T10:07:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.497+0000 7f2db9400640 1 -- 172.21.15.119:0/450358016 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2da0003520 con 0x7f2dbc144e70 2024-04-25T10:07:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.497+0000 7f2db3e00640 1 -- 172.21.15.119:0/450358016 >> v1:172.21.15.132:6789/0 conn(0x7f2dbc0cc300 legacy=0x7f2dbc0cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.497+0000 7f2db3e00640 1 -- 172.21.15.119:0/450358016 >> v1:172.21.15.119:6789/0 conn(0x7f2dbc144e70 legacy=0x7f2dbc06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.497+0000 7f2db3e00640 1 -- 172.21.15.119:0/450358016 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc06e270 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.498+0000 7f2db3e00640 1 --2- 172.21.15.119:0/450358016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc0ce9d0 0x7f2dbc0cee60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc15c160 tx=0x7f2da402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6671de40e1a6f178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.499+0000 7f2db9400640 1 -- 172.21.15.119:0/450358016 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4037070 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.499+0000 7f2db9400640 1 -- 172.21.15.119:0/450358016 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da402ae40 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.499+0000 7f2db9400640 1 -- 172.21.15.119:0/450358016 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4032660 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.499+0000 7f2dc1000640 1 -- 172.21.15.119:0/450358016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc0ce9d0 msgr2=0x7f2dbc0cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.499+0000 7f2dc1000640 1 --2- 172.21.15.119:0/450358016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc0ce9d0 0x7f2dbc0cee60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc15c160 tx=0x7f2da402a130 comp rx=0 tx=0).stop 2024-04-25T10:07:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.499+0000 7f2dc1000640 1 -- 172.21.15.119:0/450358016 shutdown_connections 2024-04-25T10:07:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.499+0000 7f2dc1000640 1 --2- 172.21.15.119:0/450358016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc0ce9d0 0x7f2dbc0cee60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.499+0000 7f2dc1000640 1 -- 172.21.15.119:0/450358016 >> 172.21.15.119:0/450358016 conn(0x7f2dbc06aba0 msgr2=0x7f2dbc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.499+0000 7f2dc1000640 1 -- 172.21.15.119:0/450358016 shutdown_connections 2024-04-25T10:07:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.499+0000 7f2dc1000640 1 -- 172.21.15.119:0/450358016 wait complete. 2024-04-25T10:07:06.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.501+0000 7f2dc1000640 1 Processor -- start 2024-04-25T10:07:06.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.502+0000 7f2dc1000640 1 -- start start 2024-04-25T10:07:06.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2dc1000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dbc0cc300 0x7f2dbc134fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2dc1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc0ce9d0 0x7f2dbc13d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2dc1000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2dbc144e70 0x7f2dbc157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2dc1000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc06d970 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2dc1000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0cf3a0 con 0x7f2dbc0cc300 2024-04-25T10:07:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2dc1000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc1454b0 con 0x7f2dbc144e70 2024-04-25T10:07:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2db3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc0ce9d0 0x7f2dbc13d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2db3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc0ce9d0 0x7f2dbc13d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49268/0 (socket says 172.21.15.119:49268) 2024-04-25T10:07:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2db3e00640 1 -- 172.21.15.119:0/2072774913 learned_addr learned my addr 172.21.15.119:0/2072774913 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2dba800640 1 --2- 172.21.15.119:0/2072774913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2dbc144e70 0x7f2dbc157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2db3e00640 1 -- 172.21.15.119:0/2072774913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2dbc144e70 msgr2=0x7f2dbc157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2db9e00640 1 --2- 172.21.15.119:0/2072774913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dbc0cc300 0x7f2dbc134fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2db3e00640 1 --2- 172.21.15.119:0/2072774913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2dbc144e70 0x7f2dbc157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2db3e00640 1 -- 172.21.15.119:0/2072774913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dbc0cc300 msgr2=0x7f2dbc134fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2db3e00640 1 --2- 172.21.15.119:0/2072774913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dbc0cc300 0x7f2dbc134fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.503+0000 7f2db3e00640 1 -- 172.21.15.119:0/2072774913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc158390 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.504+0000 7f2db3e00640 1 --2- 172.21.15.119:0/2072774913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc0ce9d0 0x7f2dbc13d310 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2da4000c00 tx=0x7f2da40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.504+0000 7f2db2000640 1 -- 172.21.15.119:0/2072774913 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4041020 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.504+0000 7f2db2000640 1 -- 172.21.15.119:0/2072774913 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da403c670 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.504+0000 7f2db2000640 1 -- 172.21.15.119:0/2072774913 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4037040 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.504+0000 7f2dc1000640 1 -- 172.21.15.119:0/2072774913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dbc158670 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.504+0000 7f2dc1000640 1 -- 172.21.15.119:0/2072774913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc13d850 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.505+0000 7f2dc1000640 1 -- 172.21.15.119:0/2072774913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dbc13ce40 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.506+0000 7f2db2000640 1 -- 172.21.15.119:0/2072774913 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2da4044020 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.508+0000 7f2db2000640 1 --2- 172.21.15.119:0/2072774913 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d9806b500 0x7f2d9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.508+0000 7f2db2000640 1 -- 172.21.15.119:0/2072774913 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2da40aaa60 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.508+0000 7f2db9e00640 1 --2- 172.21.15.119:0/2072774913 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d9806b500 0x7f2d9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.509+0000 7f2db9e00640 1 --2- 172.21.15.119:0/2072774913 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d9806b500 0x7f2d9806d9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2dac000f60 tx=0x7f2dac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.513+0000 7f2db2000640 1 -- 172.21.15.119:0/2072774913 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2da404b030 con 0x7f2dbc0ce9d0 2024-04-25T10:07:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:06.740+0000 7f2dc1000640 1 -- 172.21.15.119:0/2072774913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2dbc0750b0 con 0x7f2dbc0ce9d0 2024-04-25T10:07:07.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.301+0000 7f2db2000640 1 -- 172.21.15.119:0/2072774913 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2da4079810 con 0x7f2dbc0ce9d0 2024-04-25T10:07:07.307 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T10:07:07.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.304+0000 7f2dc1000640 1 -- 172.21.15.119:0/2072774913 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d9806b500 msgr2=0x7f2d9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.304+0000 7f2dc1000640 1 --2- 172.21.15.119:0/2072774913 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d9806b500 0x7f2d9806d9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2dac000f60 tx=0x7f2dac009040 comp rx=0 tx=0).stop 2024-04-25T10:07:07.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.304+0000 7f2dc1000640 1 -- 172.21.15.119:0/2072774913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc0ce9d0 msgr2=0x7f2dbc13d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.304+0000 7f2dc1000640 1 --2- 172.21.15.119:0/2072774913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc0ce9d0 0x7f2dbc13d310 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2da4000c00 tx=0x7f2da40046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.304+0000 7f2dc1000640 1 -- 172.21.15.119:0/2072774913 shutdown_connections 2024-04-25T10:07:07.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.304+0000 7f2dc1000640 1 --2- 172.21.15.119:0/2072774913 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d9806b500 0x7f2d9806d9c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.304+0000 7f2dc1000640 1 --2- 172.21.15.119:0/2072774913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2dbc144e70 0x7f2dbc157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.304+0000 7f2dc1000640 1 --2- 172.21.15.119:0/2072774913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc0ce9d0 0x7f2dbc13d310 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.304+0000 7f2dc1000640 1 --2- 172.21.15.119:0/2072774913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dbc0cc300 0x7f2dbc134fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.304+0000 7f2dc1000640 1 -- 172.21.15.119:0/2072774913 >> 172.21.15.119:0/2072774913 conn(0x7f2dbc06aba0 msgr2=0x7f2dbc0ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.304+0000 7f2dc1000640 1 -- 172.21.15.119:0/2072774913 shutdown_connections 2024-04-25T10:07:07.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.304+0000 7f2dc1000640 1 -- 172.21.15.119:0/2072774913 wait complete. 2024-04-25T10:07:07.351 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:07.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.516+0000 7f422b200640 1 Processor -- start 2024-04-25T10:07:07.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.516+0000 7f422b200640 1 -- start start 2024-04-25T10:07:07.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.517+0000 7f422b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f422c144470 0x7f422c15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.517+0000 7f422b200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c0cd440 con 0x7f422c0b5050 2024-04-25T10:07:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.517+0000 7f422b200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c0cd720 con 0x7f422c06e430 2024-04-25T10:07:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.517+0000 7f422b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c145340 con 0x7f422c144470 2024-04-25T10:07:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.518+0000 7f422a800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f422c0b5050 0x7f422c15bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56354/0 (socket says 172.21.15.119:56354) 2024-04-25T10:07:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.518+0000 7f422a800640 1 -- 172.21.15.119:0/1748897405 learned_addr learned my addr 172.21.15.119:0/1748897405 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.518+0000 7f4229e00640 1 --2- 172.21.15.119:0/1748897405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f422c144470 0x7f422c15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.518+0000 7f4228a00640 1 -- 172.21.15.119:0/1748897405 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3100832246 0 0) 0x7f422c0cd440 con 0x7f422c0b5050 2024-04-25T10:07:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.518+0000 7f4228a00640 1 -- 172.21.15.119:0/1748897405 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4210003520 con 0x7f422c0b5050 2024-04-25T10:07:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.518+0000 7f4228a00640 1 -- 172.21.15.119:0/1748897405 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2507373832 0 0) 0x7f4210003520 con 0x7f422c0b5050 2024-04-25T10:07:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.518+0000 7f4228a00640 1 -- 172.21.15.119:0/1748897405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f422c144470 msgr2=0x7f422c15fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.518+0000 7f4228a00640 1 --2- 172.21.15.119:0/1748897405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f422c144470 0x7f422c15fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.518+0000 7f4228a00640 1 -- 172.21.15.119:0/1748897405 >> v1:172.21.15.132:6789/0 conn(0x7f422c06e430 legacy=0x7f422c0b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.518+0000 7f4228a00640 1 -- 172.21.15.119:0/1748897405 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c0cda00 con 0x7f422c0b5050 2024-04-25T10:07:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.518+0000 7f4228a00640 1 -- 172.21.15.119:0/1748897405 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f42240019f0 con 0x7f422c0b5050 2024-04-25T10:07:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f4228a00640 1 -- 172.21.15.119:0/1748897405 >> v1:172.21.15.119:6789/0 conn(0x7f422c0b5050 legacy=0x7f422c15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f4228a00640 1 --2- 172.21.15.119:0/1748897405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4210003a50 0x7f4210003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f4228a00640 1 --2- 172.21.15.119:0/1748897405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4210004800 0x7f4210004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f4228a00640 1 --2- 172.21.15.119:0/1748897405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f422c144470 0x7f4210005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f4228a00640 1 -- 172.21.15.119:0/1748897405 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421000bb50 con 0x7f4210004800 2024-04-25T10:07:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f4228a00640 1 -- 172.21.15.119:0/1748897405 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421000bd00 con 0x7f4210003a50 2024-04-25T10:07:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f4228a00640 1 -- 172.21.15.119:0/1748897405 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421000bf00 con 0x7f422c144470 2024-04-25T10:07:07.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f4229400640 1 --2- 172.21.15.119:0/1748897405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4210004800 0x7f4210004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f422a800640 1 --2- 172.21.15.119:0/1748897405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f422c144470 0x7f4210005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f4229400640 1 -- 172.21.15.119:0/1748897405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f422c144470 msgr2=0x7f4210005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f4229e00640 1 --2- 172.21.15.119:0/1748897405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4210003a50 0x7f4210003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f4229400640 1 --2- 172.21.15.119:0/1748897405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f422c144470 0x7f4210005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f4229400640 1 -- 172.21.15.119:0/1748897405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4210003a50 msgr2=0x7f4210003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f4229400640 1 --2- 172.21.15.119:0/1748897405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4210003a50 0x7f4210003e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f4229400640 1 -- 172.21.15.119:0/1748897405 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4210003520 con 0x7f4210004800 2024-04-25T10:07:07.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.519+0000 7f422a800640 1 --2- 172.21.15.119:0/1748897405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f422c144470 0x7f4210005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:07.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.520+0000 7f4229400640 1 --2- 172.21.15.119:0/1748897405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4210004800 0x7f4210004c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4218001750 tx=0x7f4210005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adfcd6e4ad25141 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.521+0000 7f4228a00640 1 -- 172.21.15.119:0/1748897405 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4218001370 con 0x7f4210004800 2024-04-25T10:07:07.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.521+0000 7f4228a00640 1 -- 172.21.15.119:0/1748897405 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42180052d0 con 0x7f4210004800 2024-04-25T10:07:07.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.521+0000 7f4228a00640 1 -- 172.21.15.119:0/1748897405 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421802abe0 con 0x7f4210004800 2024-04-25T10:07:07.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.521+0000 7f422b200640 1 -- 172.21.15.119:0/1748897405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4210004800 msgr2=0x7f4210004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.521+0000 7f422b200640 1 --2- 172.21.15.119:0/1748897405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4210004800 0x7f4210004c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4218001750 tx=0x7f4210005830 comp rx=0 tx=0).stop 2024-04-25T10:07:07.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.521+0000 7f422a800640 1 -- 172.21.15.119:0/1748897405 reap_dead start 2024-04-25T10:07:07.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.521+0000 7f422b200640 1 -- 172.21.15.119:0/1748897405 shutdown_connections 2024-04-25T10:07:07.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.521+0000 7f422b200640 1 -- 172.21.15.119:0/1748897405 >> 172.21.15.119:0/1748897405 conn(0x7f422c06b1d0 msgr2=0x7f422c0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.521+0000 7f422b200640 1 -- 172.21.15.119:0/1748897405 shutdown_connections 2024-04-25T10:07:07.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.521+0000 7f422b200640 1 -- 172.21.15.119:0/1748897405 wait complete. 2024-04-25T10:07:07.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.523+0000 7f422b200640 1 Processor -- start 2024-04-25T10:07:07.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.523+0000 7f422b200640 1 -- start start 2024-04-25T10:07:07.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.524+0000 7f422b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4210004800 0x7f422c15b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.524+0000 7f422b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f422c139370 0x7f422c1397d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.524+0000 7f422b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c139d10 0x7f422c13b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.524+0000 7f422b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0ce200 con 0x7f422c139d10 2024-04-25T10:07:07.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.524+0000 7f422b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c144850 con 0x7f4210004800 2024-04-25T10:07:07.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.524+0000 7f422b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c15fc90 con 0x7f422c139370 2024-04-25T10:07:07.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.524+0000 7f422a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c139d10 0x7f422c13b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.524+0000 7f422a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c139d10 0x7f422c13b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49280/0 (socket says 172.21.15.119:49280) 2024-04-25T10:07:07.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.524+0000 7f422a800640 1 -- 172.21.15.119:0/3546546256 learned_addr learned my addr 172.21.15.119:0/3546546256 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:07.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.525+0000 7f4229400640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f422c139370 0x7f422c1397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.525+0000 7f4229e00640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4210004800 0x7f422c15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.525+0000 7f422a800640 1 -- 172.21.15.119:0/3546546256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f422c139370 msgr2=0x7f422c1397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.525+0000 7f422a800640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f422c139370 0x7f422c1397d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.525+0000 7f422a800640 1 -- 172.21.15.119:0/3546546256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4210004800 msgr2=0x7f422c15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.525+0000 7f422a800640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4210004800 0x7f422c15b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.525+0000 7f422a800640 1 -- 172.21.15.119:0/3546546256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c137eb0 con 0x7f422c139d10 2024-04-25T10:07:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.525+0000 7f4229e00640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4210004800 0x7f422c15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.525+0000 7f422a800640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c139d10 0x7f422c13b080 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f422c06f500 tx=0x7f422400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.525+0000 7f4222000640 1 -- 172.21.15.119:0/3546546256 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422400de50 con 0x7f422c139d10 2024-04-25T10:07:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.525+0000 7f4222000640 1 -- 172.21.15.119:0/3546546256 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4224004b60 con 0x7f422c139d10 2024-04-25T10:07:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.525+0000 7f4222000640 1 -- 172.21.15.119:0/3546546256 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224007070 con 0x7f422c139d10 2024-04-25T10:07:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.525+0000 7f422b200640 1 -- 172.21.15.119:0/3546546256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c138190 con 0x7f422c139d10 2024-04-25T10:07:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.526+0000 7f422b200640 1 -- 172.21.15.119:0/3546546256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c138750 con 0x7f422c139d10 2024-04-25T10:07:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.527+0000 7f422b200640 1 -- 172.21.15.119:0/3546546256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422c0749b0 con 0x7f422c139d10 2024-04-25T10:07:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.528+0000 7f4222000640 1 -- 172.21.15.119:0/3546546256 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f422401b020 con 0x7f422c139d10 2024-04-25T10:07:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.528+0000 7f4222000640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f420c06b4d0 0x7f420c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.529+0000 7f4222000640 1 -- 172.21.15.119:0/3546546256 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f422408a480 con 0x7f422c139d10 2024-04-25T10:07:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.529+0000 7f4229e00640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f420c06b4d0 0x7f420c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.529+0000 7f4229e00640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f420c06b4d0 0x7f420c06d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f421c006fd0 tx=0x7f421c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.533+0000 7f4222000640 1 -- 172.21.15.119:0/3546546256 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4224059260 con 0x7f422c139d10 2024-04-25T10:07:07.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.721+0000 7f422b200640 1 -- 172.21.15.119:0/3546546256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f422c143c90 con 0x7f422c139d10 2024-04-25T10:07:07.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.723+0000 7f4222000640 1 -- 172.21.15.119:0/3546546256 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16654 (secure 0 0 0) 0x7f4224058b60 con 0x7f422c139d10 2024-04-25T10:07:07.729 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:07.729 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":19,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:06.742751+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:07.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.726+0000 7f422b200640 1 -- 172.21.15.119:0/3546546256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f420c06b4d0 msgr2=0x7f420c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.726+0000 7f422b200640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f420c06b4d0 0x7f420c06d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f421c006fd0 tx=0x7f421c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:07.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.726+0000 7f422b200640 1 -- 172.21.15.119:0/3546546256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c139d10 msgr2=0x7f422c13b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.726+0000 7f422b200640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c139d10 0x7f422c13b080 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f422c06f500 tx=0x7f422400b4a0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.726+0000 7f422b200640 1 -- 172.21.15.119:0/3546546256 shutdown_connections 2024-04-25T10:07:07.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.726+0000 7f422b200640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f420c06b4d0 0x7f420c06d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.726+0000 7f422b200640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f422c139d10 0x7f422c13b080 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.726+0000 7f422b200640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f422c139370 0x7f422c1397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.726+0000 7f422b200640 1 --2- 172.21.15.119:0/3546546256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4210004800 0x7f422c15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.726+0000 7f422b200640 1 -- 172.21.15.119:0/3546546256 >> 172.21.15.119:0/3546546256 conn(0x7f422c06b1d0 msgr2=0x7f422c0b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.726+0000 7f422b200640 1 -- 172.21.15.119:0/3546546256 shutdown_connections 2024-04-25T10:07:07.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.726+0000 7f422b200640 1 -- 172.21.15.119:0/3546546256 wait complete. 2024-04-25T10:07:07.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:07.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.938+0000 7f25a3400640 1 Processor -- start 2024-04-25T10:07:07.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.938+0000 7f25a3400640 1 -- start start 2024-04-25T10:07:07.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.940+0000 7f25a3400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c15d980 0x7f259c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.940+0000 7f25a3400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f259c15ed70 0x7f259c0acf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.940+0000 7f25a3400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c0712c0 con 0x7f259c0aeb20 2024-04-25T10:07:07.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.940+0000 7f25a3400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259c071ed0 con 0x7f259c15ed70 2024-04-25T10:07:07.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.940+0000 7f25a3400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0715a0 con 0x7f259c15d980 2024-04-25T10:07:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.940+0000 7f25a0a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c15d980 0x7f259c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.940+0000 7f259be00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f259c0aeb20 0x7f259c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:46952/0 (socket says 172.21.15.119:46952) 2024-04-25T10:07:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.940+0000 7f259be00640 1 -- 172.21.15.119:0/1484931920 learned_addr learned my addr 172.21.15.119:0/1484931920 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.941+0000 7f25a1400640 1 --2- 172.21.15.119:0/1484931920 >> v2:172.21.15.132:3300/0 conn(0x7f259c15ed70 0x7f259c0acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.941+0000 7f25a0a00640 1 -- 172.21.15.119:0/1484931920 >> v2:172.21.15.132:3300/0 conn(0x7f259c15ed70 msgr2=0x7f259c0acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.941+0000 7f25a0a00640 1 --2- 172.21.15.119:0/1484931920 >> v2:172.21.15.132:3300/0 conn(0x7f259c15ed70 0x7f259c0acf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.941+0000 7f25a0a00640 1 -- 172.21.15.119:0/1484931920 >> v1:172.21.15.132:6789/0 conn(0x7f259c0aeb20 legacy=0x7f259c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.941+0000 7f25a0a00640 1 -- 172.21.15.119:0/1484931920 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c071880 con 0x7f259c15d980 2024-04-25T10:07:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.942+0000 7f25a0a00640 1 --2- 172.21.15.119:0/1484931920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c15d980 0x7f259c0ae5e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f25900097a0 tx=0x7f259002a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0409ca062d6de69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.943+0000 7f259b400640 1 -- 172.21.15.119:0/1484931920 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590037070 con 0x7f259c15d980 2024-04-25T10:07:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.943+0000 7f259b400640 1 -- 172.21.15.119:0/1484931920 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259002ae40 con 0x7f259c15d980 2024-04-25T10:07:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.943+0000 7f259b400640 1 -- 172.21.15.119:0/1484931920 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590032660 con 0x7f259c15d980 2024-04-25T10:07:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.944+0000 7f25a3400640 1 -- 172.21.15.119:0/1484931920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c15d980 msgr2=0x7f259c0ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.944+0000 7f25a3400640 1 --2- 172.21.15.119:0/1484931920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c15d980 0x7f259c0ae5e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f25900097a0 tx=0x7f259002a0f0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.944+0000 7f25a3400640 1 -- 172.21.15.119:0/1484931920 shutdown_connections 2024-04-25T10:07:07.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.944+0000 7f25a3400640 1 --2- 172.21.15.119:0/1484931920 >> v2:172.21.15.132:3300/0 conn(0x7f259c15ed70 0x7f259c0acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.944+0000 7f25a3400640 1 --2- 172.21.15.119:0/1484931920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c15d980 0x7f259c0ae5e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.944+0000 7f25a3400640 1 -- 172.21.15.119:0/1484931920 >> 172.21.15.119:0/1484931920 conn(0x7f259c06aba0 msgr2=0x7f259c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.944+0000 7f25a3400640 1 -- 172.21.15.119:0/1484931920 shutdown_connections 2024-04-25T10:07:07.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.944+0000 7f25a3400640 1 -- 172.21.15.119:0/1484931920 wait complete. 2024-04-25T10:07:07.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.946+0000 7f25a3400640 1 Processor -- start 2024-04-25T10:07:07.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.946+0000 7f25a3400640 1 -- start start 2024-04-25T10:07:07.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a3400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c0aeb20 0x7f259c141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a3400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c15d980 0x7f259c0b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f259c15c1b0 0x7f259c141010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a3400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0cfb10 con 0x7f259c15c1b0 2024-04-25T10:07:07.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a3400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0cf990 con 0x7f259c15d980 2024-04-25T10:07:07.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a3400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0ad4b0 con 0x7f259c0aeb20 2024-04-25T10:07:07.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f259c15c1b0 0x7f259c141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f259c15c1b0 0x7f259c141010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49284/0 (socket says 172.21.15.119:49284) 2024-04-25T10:07:07.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a1400640 1 -- 172.21.15.119:0/629719706 learned_addr learned my addr 172.21.15.119:0/629719706 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:07.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a0a00640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c0aeb20 0x7f259c141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f259be00640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c15d980 0x7f259c0b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a1400640 1 -- 172.21.15.119:0/629719706 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c0aeb20 msgr2=0x7f259c141c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a1400640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c0aeb20 0x7f259c141c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a1400640 1 -- 172.21.15.119:0/629719706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c15d980 msgr2=0x7f259c0b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a1400640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c15d980 0x7f259c0b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f25a1400640 1 -- 172.21.15.119:0/629719706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c0b2a90 con 0x7f259c15c1b0 2024-04-25T10:07:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.947+0000 7f259be00640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c15d980 0x7f259c0b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.948+0000 7f25a1400640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f259c15c1b0 0x7f259c141010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f259400efc0 tx=0x7f259400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.948+0000 7f2598c00640 1 -- 172.21.15.119:0/629719706 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594018020 con 0x7f259c15c1b0 2024-04-25T10:07:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.948+0000 7f2598c00640 1 -- 172.21.15.119:0/629719706 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2594014480 con 0x7f259c15c1b0 2024-04-25T10:07:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.948+0000 7f25a3400640 1 -- 172.21.15.119:0/629719706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c0b2d70 con 0x7f259c15c1b0 2024-04-25T10:07:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.948+0000 7f2598c00640 1 -- 172.21.15.119:0/629719706 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594014870 con 0x7f259c15c1b0 2024-04-25T10:07:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.948+0000 7f25a3400640 1 -- 172.21.15.119:0/629719706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c0b3330 con 0x7f259c15c1b0 2024-04-25T10:07:07.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.949+0000 7f25a3400640 1 -- 172.21.15.119:0/629719706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259c0749b0 con 0x7f259c15c1b0 2024-04-25T10:07:07.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.950+0000 7f2598c00640 1 -- 172.21.15.119:0/629719706 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2594005d10 con 0x7f259c15c1b0 2024-04-25T10:07:07.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.951+0000 7f2598c00640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f257c06b4d0 0x7f257c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.951+0000 7f2598c00640 1 -- 172.21.15.119:0/629719706 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f259408b0c0 con 0x7f259c15c1b0 2024-04-25T10:07:07.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.951+0000 7f25a0a00640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f257c06b4d0 0x7f257c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.952+0000 7f25a0a00640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f257c06b4d0 0x7f257c06d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2590034000 tx=0x7f259002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:07.956+0000 7f2598c00640 1 -- 172.21.15.119:0/629719706 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2594022030 con 0x7f259c15c1b0 2024-04-25T10:07:08.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.152+0000 7f25a3400640 1 -- 172.21.15.119:0/629719706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f259c06d4c0 con 0x7f259c15c1b0 2024-04-25T10:07:08.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.153+0000 7f2598c00640 1 -- 172.21.15.119:0/629719706 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16654 (secure 0 0 0) 0x7f2594010040 con 0x7f259c15c1b0 2024-04-25T10:07:08.160 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:08.160 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":19,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:06.742751+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:08.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.156+0000 7f25a3400640 1 -- 172.21.15.119:0/629719706 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f257c06b4d0 msgr2=0x7f257c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.156+0000 7f25a3400640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f257c06b4d0 0x7f257c06d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2590034000 tx=0x7f259002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:08.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.156+0000 7f25a3400640 1 -- 172.21.15.119:0/629719706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f259c15c1b0 msgr2=0x7f259c141010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.156+0000 7f25a3400640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f259c15c1b0 0x7f259c141010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f259400efc0 tx=0x7f259400c490 comp rx=0 tx=0).stop 2024-04-25T10:07:08.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.156+0000 7f25a3400640 1 -- 172.21.15.119:0/629719706 shutdown_connections 2024-04-25T10:07:08.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.156+0000 7f25a3400640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f257c06b4d0 0x7f257c06d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.156+0000 7f25a3400640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f259c15c1b0 0x7f259c141010 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.156+0000 7f25a3400640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c15d980 0x7f259c0b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.156+0000 7f25a3400640 1 --2- 172.21.15.119:0/629719706 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f259c0aeb20 0x7f259c141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.156+0000 7f25a3400640 1 -- 172.21.15.119:0/629719706 >> 172.21.15.119:0/629719706 conn(0x7f259c06aba0 msgr2=0x7f259c0af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.156+0000 7f25a3400640 1 -- 172.21.15.119:0/629719706 shutdown_connections 2024-04-25T10:07:08.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.156+0000 7f25a3400640 1 -- 172.21.15.119:0/629719706 wait complete. 2024-04-25T10:07:08.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:07:08.266 INFO:tasks.ceph.mds.c.smithi119.stderr:2024-04-25T10:07:08.260+0000 7f09d7800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:08.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.374+0000 7ff9b3e00640 1 Processor -- start 2024-04-25T10:07:08.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.374+0000 7ff9b3e00640 1 -- start start 2024-04-25T10:07:08.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.376+0000 7ff9b3e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff9ac0b5c00 0x7ff9ac168c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.376+0000 7ff9b3e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac1627a0 con 0x7ff9ac0b97c0 2024-04-25T10:07:08.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.376+0000 7ff9b3e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac1621e0 con 0x7ff9ac15dc80 2024-04-25T10:07:08.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.376+0000 7ff9b3e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac161f80 con 0x7ff9ac0b5c00 2024-04-25T10:07:08.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.376+0000 7ff9b1e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff9ac0b5c00 0x7ff9ac168c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.376+0000 7ff9b1e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff9ac0b5c00 0x7ff9ac168c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49294/0 (socket says 172.21.15.119:49294) 2024-04-25T10:07:08.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.376+0000 7ff9b0a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff9ac0b97c0 0x7ff9ac0b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56362/0 (socket says 172.21.15.119:56362) 2024-04-25T10:07:08.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.376+0000 7ff9b1e00640 1 -- 172.21.15.119:0/3193810496 learned_addr learned my addr 172.21.15.119:0/3193810496 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:08.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.376+0000 7ff9abe00640 1 -- 172.21.15.119:0/3193810496 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455575404 0 0) 0x7ff9ac1627a0 con 0x7ff9ac0b97c0 2024-04-25T10:07:08.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.376+0000 7ff9abe00640 1 -- 172.21.15.119:0/3193810496 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff990003520 con 0x7ff9ac0b97c0 2024-04-25T10:07:08.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.377+0000 7ff9b1e00640 1 -- 172.21.15.119:0/3193810496 >> v1:172.21.15.132:6789/0 conn(0x7ff9ac15dc80 legacy=0x7ff9ac161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.377+0000 7ff9b1e00640 1 -- 172.21.15.119:0/3193810496 >> v1:172.21.15.119:6789/0 conn(0x7ff9ac0b97c0 legacy=0x7ff9ac0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.377+0000 7ff9b1e00640 1 -- 172.21.15.119:0/3193810496 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac1624c0 con 0x7ff9ac0b5c00 2024-04-25T10:07:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.378+0000 7ff9b1e00640 1 --2- 172.21.15.119:0/3193810496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0b5c00 0x7ff9ac168c00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009870 tx=0x7ff9a402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d221cd4a09f1d2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.378+0000 7ff9abe00640 1 -- 172.21.15.119:0/3193810496 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4037070 con 0x7ff9ac0b5c00 2024-04-25T10:07:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.378+0000 7ff9abe00640 1 -- 172.21.15.119:0/3193810496 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a402ae50 con 0x7ff9ac0b5c00 2024-04-25T10:07:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.378+0000 7ff9abe00640 1 -- 172.21.15.119:0/3193810496 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4032860 con 0x7ff9ac0b5c00 2024-04-25T10:07:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.378+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3193810496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0b5c00 msgr2=0x7ff9ac168c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.378+0000 7ff9b3e00640 1 --2- 172.21.15.119:0/3193810496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0b5c00 0x7ff9ac168c00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009870 tx=0x7ff9a402a300 comp rx=0 tx=0).stop 2024-04-25T10:07:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.378+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3193810496 shutdown_connections 2024-04-25T10:07:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.378+0000 7ff9b3e00640 1 --2- 172.21.15.119:0/3193810496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0b5c00 0x7ff9ac168c00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.378+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3193810496 >> 172.21.15.119:0/3193810496 conn(0x7ff9ac0d04f0 msgr2=0x7ff9ac0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.378+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3193810496 shutdown_connections 2024-04-25T10:07:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.379+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3193810496 wait complete. 2024-04-25T10:07:08.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.380+0000 7ff9b3e00640 1 Processor -- start 2024-04-25T10:07:08.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.380+0000 7ff9b3e00640 1 -- start start 2024-04-25T10:07:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.381+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac0b5c00 0x7ff9ac0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.381+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0b97c0 0x7ff9ac1497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.381+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac15dc80 0x7ff9ac160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.381+0000 7ff9b3e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0d3480 con 0x7ff9ac0b97c0 2024-04-25T10:07:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.381+0000 7ff9b3e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac15e060 con 0x7ff9ac15dc80 2024-04-25T10:07:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.381+0000 7ff9b3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0d3600 con 0x7ff9ac0b5c00 2024-04-25T10:07:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.381+0000 7ff9b0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0b97c0 0x7ff9ac1497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.381+0000 7ff9b0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0b97c0 0x7ff9ac1497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49304/0 (socket says 172.21.15.119:49304) 2024-04-25T10:07:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.381+0000 7ff9b0a00640 1 -- 172.21.15.119:0/3421248361 learned_addr learned my addr 172.21.15.119:0/3421248361 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.382+0000 7ff9b0a00640 1 -- 172.21.15.119:0/3421248361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac0b5c00 msgr2=0x7ff9ac0c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.382+0000 7ff9b0a00640 1 --2- 172.21.15.119:0/3421248361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac0b5c00 0x7ff9ac0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.382+0000 7ff9b0a00640 1 -- 172.21.15.119:0/3421248361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac15dc80 msgr2=0x7ff9ac160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.382+0000 7ff9b0a00640 1 --2- 172.21.15.119:0/3421248361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac15dc80 0x7ff9ac160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.382+0000 7ff9b0a00640 1 -- 172.21.15.119:0/3421248361 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac0c9f40 con 0x7ff9ac0b97c0 2024-04-25T10:07:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.382+0000 7ff9b0a00640 1 --2- 172.21.15.119:0/3421248361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0b97c0 0x7ff9ac1497f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff99800c870 tx=0x7ff99800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.382+0000 7ff9a9600640 1 -- 172.21.15.119:0/3421248361 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998017070 con 0x7ff9ac0b97c0 2024-04-25T10:07:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.382+0000 7ff9a9600640 1 -- 172.21.15.119:0/3421248361 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99800a640 con 0x7ff9ac0b97c0 2024-04-25T10:07:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.382+0000 7ff9a9600640 1 -- 172.21.15.119:0/3421248361 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99800aa10 con 0x7ff9ac0b97c0 2024-04-25T10:07:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.382+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3421248361 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac149d30 con 0x7ff9ac0b97c0 2024-04-25T10:07:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.382+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3421248361 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac0c8960 con 0x7ff9ac0b97c0 2024-04-25T10:07:08.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.384+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3421248361 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ac0750b0 con 0x7ff9ac0b97c0 2024-04-25T10:07:08.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.385+0000 7ff9a9600640 1 -- 172.21.15.119:0/3421248361 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff99801a070 con 0x7ff9ac0b97c0 2024-04-25T10:07:08.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.385+0000 7ff9a9600640 1 --2- 172.21.15.119:0/3421248361 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff98c06b4d0 0x7ff98c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.386+0000 7ff9a9600640 1 -- 172.21.15.119:0/3421248361 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff998089a50 con 0x7ff9ac0b97c0 2024-04-25T10:07:08.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.386+0000 7ff9b1400640 1 --2- 172.21.15.119:0/3421248361 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff98c06b4d0 0x7ff98c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.386+0000 7ff9b1400640 1 --2- 172.21.15.119:0/3421248361 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff98c06b4d0 0x7ff98c06d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff99c000f60 tx=0x7ff99c0079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.390+0000 7ff9a9600640 1 -- 172.21.15.119:0/3421248361 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff998058830 con 0x7ff9ac0b97c0 2024-04-25T10:07:08.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:08.626+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3421248361 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff9ac0d1780 con 0x7ff9ac0b97c0 2024-04-25T10:07:09.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.629+0000 7ff9a9600640 1 -- 172.21.15.119:0/3421248361 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff998058130 con 0x7ff9ac0b97c0 2024-04-25T10:07:09.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.632+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3421248361 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff98c06b4d0 msgr2=0x7ff98c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.632+0000 7ff9b3e00640 1 --2- 172.21.15.119:0/3421248361 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff98c06b4d0 0x7ff98c06d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff99c000f60 tx=0x7ff99c0079b0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.632+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3421248361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0b97c0 msgr2=0x7ff9ac1497f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.632+0000 7ff9b3e00640 1 --2- 172.21.15.119:0/3421248361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0b97c0 0x7ff9ac1497f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff99800c870 tx=0x7ff99800cd30 comp rx=0 tx=0).stop 2024-04-25T10:07:09.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.633+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3421248361 shutdown_connections 2024-04-25T10:07:09.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.633+0000 7ff9b3e00640 1 --2- 172.21.15.119:0/3421248361 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff98c06b4d0 0x7ff98c06d990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.633+0000 7ff9b3e00640 1 --2- 172.21.15.119:0/3421248361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9ac15dc80 0x7ff9ac160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.633+0000 7ff9b3e00640 1 --2- 172.21.15.119:0/3421248361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0b97c0 0x7ff9ac1497f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.633+0000 7ff9b3e00640 1 --2- 172.21.15.119:0/3421248361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff9ac0b5c00 0x7ff9ac0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.633+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3421248361 >> 172.21.15.119:0/3421248361 conn(0x7ff9ac0d04f0 msgr2=0x7ff9ac0d3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.633+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3421248361 shutdown_connections 2024-04-25T10:07:09.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.633+0000 7ff9b3e00640 1 -- 172.21.15.119:0/3421248361 wait complete. 2024-04-25T10:07:09.660 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:07:09.661 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:07:09.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.862+0000 7f3215e00640 1 Processor -- start 2024-04-25T10:07:09.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.862+0000 7f3215e00640 1 -- start start 2024-04-25T10:07:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.864+0000 7f3215e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3210144e70 0x7f321006d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.864+0000 7f3215e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321006d7a0 0x7f32100ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.864+0000 7f3215e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321006dcb0 con 0x7f32100cef10 2024-04-25T10:07:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.864+0000 7f3215e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321015bfe0 con 0x7f3210144e70 2024-04-25T10:07:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.864+0000 7f3215e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321006df90 con 0x7f321006d7a0 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.864+0000 7f320ea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3210144e70 0x7f321006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.864+0000 7f320ea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3210144e70 0x7f321006d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49320/0 (socket says 172.21.15.119:49320) 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.864+0000 7f320ea00640 1 -- 172.21.15.119:0/1787677444 learned_addr learned my addr 172.21.15.119:0/1787677444 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.864+0000 7f320e000640 1 --2- 172.21.15.119:0/1787677444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321006d7a0 0x7f32100ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.865+0000 7f320ea00640 1 -- 172.21.15.119:0/1787677444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321006d7a0 msgr2=0x7f32100ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.865+0000 7f320ea00640 1 --2- 172.21.15.119:0/1787677444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321006d7a0 0x7f32100ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.865+0000 7f320ea00640 1 -- 172.21.15.119:0/1787677444 >> v1:172.21.15.119:6789/0 conn(0x7f32100cef10 legacy=0x7f3210071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.865+0000 7f320ea00640 1 -- 172.21.15.119:0/1787677444 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321006e270 con 0x7f3210144e70 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.865+0000 7f320d600640 1 -- 172.21.15.119:0/1787677444 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344908086 0 0) 0x7f321006dcb0 con 0x7f32100cef10 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.866+0000 7f320ea00640 1 --2- 172.21.15.119:0/1787677444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3210144e70 0x7f321006d260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009870 tx=0x7f31fc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55b6abd340fa117 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.866+0000 7f320d600640 1 -- 172.21.15.119:0/1787677444 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc037070 con 0x7f3210144e70 2024-04-25T10:07:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.866+0000 7f320d600640 1 -- 172.21.15.119:0/1787677444 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc032440 con 0x7f3210144e70 2024-04-25T10:07:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.866+0000 7f320d600640 1 -- 172.21.15.119:0/1787677444 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc032810 con 0x7f3210144e70 2024-04-25T10:07:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.867+0000 7f3215e00640 1 -- 172.21.15.119:0/1787677444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3210144e70 msgr2=0x7f321006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.867+0000 7f3215e00640 1 --2- 172.21.15.119:0/1787677444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3210144e70 0x7f321006d260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009870 tx=0x7f31fc02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.867+0000 7f3215e00640 1 -- 172.21.15.119:0/1787677444 shutdown_connections 2024-04-25T10:07:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.867+0000 7f3215e00640 1 --2- 172.21.15.119:0/1787677444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f321006d7a0 0x7f32100ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.867+0000 7f3215e00640 1 --2- 172.21.15.119:0/1787677444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3210144e70 0x7f321006d260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.867+0000 7f3215e00640 1 -- 172.21.15.119:0/1787677444 >> 172.21.15.119:0/1787677444 conn(0x7f321006aba0 msgr2=0x7f321006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.867+0000 7f3215e00640 1 -- 172.21.15.119:0/1787677444 shutdown_connections 2024-04-25T10:07:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.867+0000 7f3215e00640 1 -- 172.21.15.119:0/1787677444 wait complete. 2024-04-25T10:07:09.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.868+0000 7f3215e00640 1 Processor -- start 2024-04-25T10:07:09.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.869+0000 7f3215e00640 1 -- start start 2024-04-25T10:07:09.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.869+0000 7f3215e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f321006d7a0 0x7f32100c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.869+0000 7f3215e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32100cef10 0x7f32100c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.869+0000 7f3215e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3210144e70 0x7f32100aa040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.870+0000 7f3215e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32101454b0 con 0x7f32100cef10 2024-04-25T10:07:09.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.870+0000 7f3215e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100cf380 con 0x7f321006d7a0 2024-04-25T10:07:09.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.870+0000 7f3215e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3210145630 con 0x7f3210144e70 2024-04-25T10:07:09.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.870+0000 7f320e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32100cef10 0x7f32100c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.870+0000 7f320e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32100cef10 0x7f32100c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49336/0 (socket says 172.21.15.119:49336) 2024-04-25T10:07:09.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.870+0000 7f320e000640 1 -- 172.21.15.119:0/3790687060 learned_addr learned my addr 172.21.15.119:0/3790687060 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:09.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.870+0000 7f320ea00640 1 --2- 172.21.15.119:0/3790687060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f321006d7a0 0x7f32100c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.870+0000 7f320f400640 1 --2- 172.21.15.119:0/3790687060 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3210144e70 0x7f32100aa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.870+0000 7f320e000640 1 -- 172.21.15.119:0/3790687060 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3210144e70 msgr2=0x7f32100aa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.870+0000 7f320e000640 1 --2- 172.21.15.119:0/3790687060 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3210144e70 0x7f32100aa040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.870+0000 7f320e000640 1 -- 172.21.15.119:0/3790687060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f321006d7a0 msgr2=0x7f32100c4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.870+0000 7f320e000640 1 --2- 172.21.15.119:0/3790687060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f321006d7a0 0x7f32100c4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.870+0000 7f320e000640 1 -- 172.21.15.119:0/3790687060 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32100acab0 con 0x7f32100cef10 2024-04-25T10:07:09.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.870+0000 7f320e000640 1 --2- 172.21.15.119:0/3790687060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32100cef10 0x7f32100c52f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f320000d7d0 tx=0x7f320000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.871+0000 7f3207400640 1 -- 172.21.15.119:0/3790687060 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320000b6c0 con 0x7f32100cef10 2024-04-25T10:07:09.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.871+0000 7f3207400640 1 -- 172.21.15.119:0/3790687060 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3200019070 con 0x7f32100cef10 2024-04-25T10:07:09.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.871+0000 7f3215e00640 1 -- 172.21.15.119:0/3790687060 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32100acd90 con 0x7f32100cef10 2024-04-25T10:07:09.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.871+0000 7f3207400640 1 -- 172.21.15.119:0/3790687060 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320000bac0 con 0x7f32100cef10 2024-04-25T10:07:09.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.871+0000 7f3215e00640 1 -- 172.21.15.119:0/3790687060 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32100ad350 con 0x7f32100cef10 2024-04-25T10:07:09.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.872+0000 7f3215e00640 1 -- 172.21.15.119:0/3790687060 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32100750b0 con 0x7f32100cef10 2024-04-25T10:07:09.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.873+0000 7f3207400640 1 -- 172.21.15.119:0/3790687060 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f320000bc60 con 0x7f32100cef10 2024-04-25T10:07:09.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.874+0000 7f3207400640 1 --2- 172.21.15.119:0/3790687060 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f31f006b4d0 0x7f31f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.874+0000 7f3207400640 1 -- 172.21.15.119:0/3790687060 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f320008a780 con 0x7f32100cef10 2024-04-25T10:07:09.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.874+0000 7f320ea00640 1 --2- 172.21.15.119:0/3790687060 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f31f006b4d0 0x7f31f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.875+0000 7f320ea00640 1 --2- 172.21.15.119:0/3790687060 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f31f006b4d0 0x7f31f006d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f31fc034000 tx=0x7f31fc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:09.878+0000 7f3207400640 1 -- 172.21.15.119:0/3790687060 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3200059560 con 0x7f32100cef10 2024-04-25T10:07:10.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.117+0000 7f3215e00640 1 -- 172.21.15.119:0/3790687060 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f321006f130 con 0x7f32100cef10 2024-04-25T10:07:10.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.633+0000 7f3207400640 1 -- 172.21.15.119:0/3790687060 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3200058e60 con 0x7f32100cef10 2024-04-25T10:07:10.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.636+0000 7f3215e00640 1 -- 172.21.15.119:0/3790687060 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f31f006b4d0 msgr2=0x7f31f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.636+0000 7f3215e00640 1 --2- 172.21.15.119:0/3790687060 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f31f006b4d0 0x7f31f006d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f31fc034000 tx=0x7f31fc02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:10.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.636+0000 7f3215e00640 1 -- 172.21.15.119:0/3790687060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32100cef10 msgr2=0x7f32100c52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.636+0000 7f3215e00640 1 --2- 172.21.15.119:0/3790687060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32100cef10 0x7f32100c52f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f320000d7d0 tx=0x7f320000dc90 comp rx=0 tx=0).stop 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.637+0000 7f3215e00640 1 -- 172.21.15.119:0/3790687060 shutdown_connections 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.637+0000 7f3215e00640 1 --2- 172.21.15.119:0/3790687060 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f31f006b4d0 0x7f31f006d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.637+0000 7f3215e00640 1 --2- 172.21.15.119:0/3790687060 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3210144e70 0x7f32100aa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.637+0000 7f3215e00640 1 --2- 172.21.15.119:0/3790687060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32100cef10 0x7f32100c52f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.637+0000 7f3215e00640 1 --2- 172.21.15.119:0/3790687060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f321006d7a0 0x7f32100c4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.637+0000 7f3215e00640 1 -- 172.21.15.119:0/3790687060 >> 172.21.15.119:0/3790687060 conn(0x7f321006aba0 msgr2=0x7f321015cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.637+0000 7f3215e00640 1 -- 172.21.15.119:0/3790687060 shutdown_connections 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.637+0000 7f3215e00640 1 -- 172.21.15.119:0/3790687060 wait complete. 2024-04-25T10:07:10.664 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:07:10.664 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:10.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.853+0000 7f0b4fe00640 1 Processor -- start 2024-04-25T10:07:10.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.853+0000 7f0b4fe00640 1 -- start start 2024-04-25T10:07:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.855+0000 7f0b4fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b48161ee0 0x7f0b480b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.855+0000 7f0b4fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0b480b2570 0x7f0b481632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.855+0000 7f0b4fe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b48071330 con 0x7f0b481664d0 2024-04-25T10:07:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.855+0000 7f0b4fe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b480720f0 con 0x7f0b480b2570 2024-04-25T10:07:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.855+0000 7f0b4fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b48071610 con 0x7f0b48161ee0 2024-04-25T10:07:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.856+0000 7f0b4d400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b48161ee0 0x7f0b480b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.856+0000 7f0b4d400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b48161ee0 0x7f0b480b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:48884/0 (socket says 172.21.15.119:48884) 2024-04-25T10:07:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.856+0000 7f0b4d400640 1 -- 172.21.15.119:0/1034060032 learned_addr learned my addr 172.21.15.119:0/1034060032 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.856+0000 7f0b4de00640 1 --2- 172.21.15.119:0/1034060032 >> v2:172.21.15.132:3300/0 conn(0x7f0b480b2570 0x7f0b481632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.857+0000 7f0b4d400640 1 -- 172.21.15.119:0/1034060032 >> v2:172.21.15.132:3300/0 conn(0x7f0b480b2570 msgr2=0x7f0b481632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.857+0000 7f0b4d400640 1 --2- 172.21.15.119:0/1034060032 >> v2:172.21.15.132:3300/0 conn(0x7f0b480b2570 0x7f0b481632d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.857+0000 7f0b4d400640 1 -- 172.21.15.119:0/1034060032 >> v1:172.21.15.132:6789/0 conn(0x7f0b481664d0 legacy=0x7f0b4806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.857+0000 7f0b4d400640 1 -- 172.21.15.119:0/1034060032 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b480718f0 con 0x7f0b48161ee0 2024-04-25T10:07:10.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.858+0000 7f0b4d400640 1 --2- 172.21.15.119:0/1034060032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b48161ee0 0x7f0b480b2f60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009870 tx=0x7f0b3c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=556a713ca8b80f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.858+0000 7f0b47e00640 1 -- 172.21.15.119:0/1034060032 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c037070 con 0x7f0b48161ee0 2024-04-25T10:07:10.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.858+0000 7f0b47e00640 1 -- 172.21.15.119:0/1034060032 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c032440 con 0x7f0b48161ee0 2024-04-25T10:07:10.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.858+0000 7f0b47e00640 1 -- 172.21.15.119:0/1034060032 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c032810 con 0x7f0b48161ee0 2024-04-25T10:07:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.859+0000 7f0b4fe00640 1 -- 172.21.15.119:0/1034060032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b48161ee0 msgr2=0x7f0b480b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.859+0000 7f0b4fe00640 1 --2- 172.21.15.119:0/1034060032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b48161ee0 0x7f0b480b2f60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009870 tx=0x7f0b3c02a250 comp rx=0 tx=0).stop 2024-04-25T10:07:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.859+0000 7f0b4fe00640 1 -- 172.21.15.119:0/1034060032 shutdown_connections 2024-04-25T10:07:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.859+0000 7f0b4fe00640 1 --2- 172.21.15.119:0/1034060032 >> v2:172.21.15.132:3300/0 conn(0x7f0b480b2570 0x7f0b481632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.859+0000 7f0b4fe00640 1 --2- 172.21.15.119:0/1034060032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b48161ee0 0x7f0b480b2f60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.859+0000 7f0b4fe00640 1 -- 172.21.15.119:0/1034060032 >> 172.21.15.119:0/1034060032 conn(0x7f0b4806aba0 msgr2=0x7f0b4806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.859+0000 7f0b4fe00640 1 -- 172.21.15.119:0/1034060032 shutdown_connections 2024-04-25T10:07:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.859+0000 7f0b4fe00640 1 -- 172.21.15.119:0/1034060032 wait complete. 2024-04-25T10:07:10.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.861+0000 7f0b4fe00640 1 Processor -- start 2024-04-25T10:07:10.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.861+0000 7f0b4fe00640 1 -- start start 2024-04-25T10:07:10.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.862+0000 7f0b4fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b480b2570 0x7f0b4814bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.862+0000 7f0b4fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b48161ee0 0x7f0b4814a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.862+0000 7f0b4fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b481664d0 0x7f0b4814c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.862+0000 7f0b4fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480b3d30 con 0x7f0b481664d0 2024-04-25T10:07:10.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.862+0000 7f0b4fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480b3bb0 con 0x7f0b48161ee0 2024-04-25T10:07:10.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.862+0000 7f0b4fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480b34a0 con 0x7f0b480b2570 2024-04-25T10:07:10.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.862+0000 7f0b4de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b481664d0 0x7f0b4814c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.862+0000 7f0b4de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b481664d0 0x7f0b4814c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34386/0 (socket says 172.21.15.119:34386) 2024-04-25T10:07:10.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.862+0000 7f0b4de00640 1 -- 172.21.15.119:0/150091636 learned_addr learned my addr 172.21.15.119:0/150091636 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:10.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.863+0000 7f0b4d400640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b480b2570 0x7f0b4814bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.863+0000 7f0b4ca00640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b48161ee0 0x7f0b4814a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.863+0000 7f0b4de00640 1 -- 172.21.15.119:0/150091636 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b480b2570 msgr2=0x7f0b4814bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.863+0000 7f0b4de00640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b480b2570 0x7f0b4814bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.863+0000 7f0b4de00640 1 -- 172.21.15.119:0/150091636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b48161ee0 msgr2=0x7f0b4814a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.863+0000 7f0b4de00640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b48161ee0 0x7f0b4814a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.863+0000 7f0b4de00640 1 -- 172.21.15.119:0/150091636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4814c120 con 0x7f0b481664d0 2024-04-25T10:07:10.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.863+0000 7f0b4ca00640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b48161ee0 0x7f0b4814a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:10.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.863+0000 7f0b4de00640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b481664d0 0x7f0b4814c5f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000efc0 tx=0x7f0b4000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.863+0000 7f0b45600640 1 -- 172.21.15.119:0/150091636 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40018020 con 0x7f0b481664d0 2024-04-25T10:07:10.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.863+0000 7f0b45600640 1 -- 172.21.15.119:0/150091636 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b40014480 con 0x7f0b481664d0 2024-04-25T10:07:10.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.863+0000 7f0b4fe00640 1 -- 172.21.15.119:0/150091636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4814ab40 con 0x7f0b481664d0 2024-04-25T10:07:10.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.863+0000 7f0b45600640 1 -- 172.21.15.119:0/150091636 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40014870 con 0x7f0b481664d0 2024-04-25T10:07:10.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.864+0000 7f0b4fe00640 1 -- 172.21.15.119:0/150091636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4814b3e0 con 0x7f0b481664d0 2024-04-25T10:07:10.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.865+0000 7f0b4fe00640 1 -- 172.21.15.119:0/150091636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b481657f0 con 0x7f0b481664d0 2024-04-25T10:07:10.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.866+0000 7f0b45600640 1 -- 172.21.15.119:0/150091636 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b40005d10 con 0x7f0b481664d0 2024-04-25T10:07:10.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.866+0000 7f0b45600640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b2806b4d0 0x7f0b2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.867+0000 7f0b45600640 1 -- 172.21.15.119:0/150091636 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b4008b0b0 con 0x7f0b481664d0 2024-04-25T10:07:10.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.867+0000 7f0b4d400640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b2806b4d0 0x7f0b2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.867+0000 7f0b4d400640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b2806b4d0 0x7f0b2806d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c034000 tx=0x7f0b3c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:10.871+0000 7f0b45600640 1 -- 172.21.15.119:0/150091636 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b40022030 con 0x7f0b481664d0 2024-04-25T10:07:11.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.110+0000 7f0b4fe00640 1 -- 172.21.15.119:0/150091636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b4806d4c0 con 0x7f0b481664d0 2024-04-25T10:07:11.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.111+0000 7f0b45600640 1 -- 172.21.15.119:0/150091636 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4941 (secure 0 0 0) 0x7f0b40010040 con 0x7f0b481664d0 2024-04-25T10:07:11.117 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:11.117 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1385207078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1385207078},{"type":"v1","addr":"172.21.15.132:6834","nonce":1385207078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1580923823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1580923823},{"type":"v1","addr":"172.21.15.119:6836","nonce":1580923823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4260,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/954053568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":954053568},{"type":"v1","addr":"172.21.15.132:6835","nonce":954053568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:06.743184+0000","modified":"2024-04-25T10:07:10.118630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1632024048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1632024048},{"type":"v1","addr":"172.21.15.119:6837","nonce":1632024048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:07:11.117 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2024-04-25T10:07:11.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.114+0000 7f0b4fe00640 1 -- 172.21.15.119:0/150091636 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b2806b4d0 msgr2=0x7f0b2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.114+0000 7f0b4fe00640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b2806b4d0 0x7f0b2806d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c034000 tx=0x7f0b3c02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:11.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.114+0000 7f0b4fe00640 1 -- 172.21.15.119:0/150091636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b481664d0 msgr2=0x7f0b4814c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.114+0000 7f0b4fe00640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b481664d0 0x7f0b4814c5f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000efc0 tx=0x7f0b4000c490 comp rx=0 tx=0).stop 2024-04-25T10:07:11.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.114+0000 7f0b4fe00640 1 -- 172.21.15.119:0/150091636 shutdown_connections 2024-04-25T10:07:11.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.114+0000 7f0b4fe00640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b2806b4d0 0x7f0b2806d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.114+0000 7f0b4fe00640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b481664d0 0x7f0b4814c5f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.114+0000 7f0b4fe00640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b48161ee0 0x7f0b4814a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.114+0000 7f0b4fe00640 1 --2- 172.21.15.119:0/150091636 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b480b2570 0x7f0b4814bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.114+0000 7f0b4fe00640 1 -- 172.21.15.119:0/150091636 >> 172.21.15.119:0/150091636 conn(0x7f0b4806aba0 msgr2=0x7f0b480b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.114+0000 7f0b4fe00640 1 -- 172.21.15.119:0/150091636 shutdown_connections 2024-04-25T10:07:11.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.114+0000 7f0b4fe00640 1 -- 172.21.15.119:0/150091636 wait complete. 2024-04-25T10:07:11.138 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:11.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.331+0000 7f2e9d000640 1 Processor -- start 2024-04-25T10:07:11.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.331+0000 7f2e9d000640 1 -- start start 2024-04-25T10:07:11.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.332+0000 7f2e9d000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e98144040 0x7f2e980cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.332+0000 7f2e9d000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2e9806ea30 0x7f2e980b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.332+0000 7f2e9d000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e98144420 con 0x7f2e9815acd0 2024-04-25T10:07:11.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.332+0000 7f2e9d000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e980b3d80 con 0x7f2e9806ea30 2024-04-25T10:07:11.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.332+0000 7f2e9d000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980b4060 con 0x7f2e98144040 2024-04-25T10:07:11.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.333+0000 7f2e8fe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2e9815acd0 0x7f2e9806e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42062/0 (socket says 172.21.15.119:42062) 2024-04-25T10:07:11.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.333+0000 7f2e8fe00640 1 -- 172.21.15.119:0/1910826292 learned_addr learned my addr 172.21.15.119:0/1910826292 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:11.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.333+0000 7f2e97400640 1 --2- 172.21.15.119:0/1910826292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e98144040 0x7f2e980cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.333+0000 7f2e97e00640 1 --2- 172.21.15.119:0/1910826292 >> v2:172.21.15.132:3300/0 conn(0x7f2e9806ea30 0x7f2e980b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.333+0000 7f2e96a00640 1 -- 172.21.15.119:0/1910826292 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280529034 0 0) 0x7f2e98144420 con 0x7f2e9815acd0 2024-04-25T10:07:11.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.333+0000 7f2e96a00640 1 -- 172.21.15.119:0/1910826292 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e7c003520 con 0x7f2e9815acd0 2024-04-25T10:07:11.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.333+0000 7f2e96a00640 1 -- 172.21.15.119:0/1910826292 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 134064617 0 0) 0x7f2e7c003520 con 0x7f2e9815acd0 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.333+0000 7f2e96a00640 1 -- 172.21.15.119:0/1910826292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e98144040 msgr2=0x7f2e980cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.333+0000 7f2e96a00640 1 --2- 172.21.15.119:0/1910826292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e98144040 0x7f2e980cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.333+0000 7f2e96a00640 1 -- 172.21.15.119:0/1910826292 >> v2:172.21.15.132:3300/0 conn(0x7f2e9806ea30 msgr2=0x7f2e980b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.333+0000 7f2e96a00640 1 --2- 172.21.15.119:0/1910826292 >> v2:172.21.15.132:3300/0 conn(0x7f2e9806ea30 0x7f2e980b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.333+0000 7f2e96a00640 1 -- 172.21.15.119:0/1910826292 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e980b4340 con 0x7f2e9815acd0 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.333+0000 7f2e96a00640 1 -- 172.21.15.119:0/1910826292 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f2e80001670 con 0x7f2e9815acd0 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.333+0000 7f2e96a00640 1 -- 172.21.15.119:0/1910826292 >> v1:172.21.15.119:6789/0 conn(0x7f2e9815acd0 legacy=0x7f2e9806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.334+0000 7f2e96a00640 1 --2- 172.21.15.119:0/1910826292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e7c003a20 0x7f2e7c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.334+0000 7f2e96a00640 1 --2- 172.21.15.119:0/1910826292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e98144040 0x7f2e7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.334+0000 7f2e96a00640 1 --2- 172.21.15.119:0/1910826292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e7c004d40 0x7f2e7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.334+0000 7f2e96a00640 1 -- 172.21.15.119:0/1910826292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980b3d80 con 0x7f2e7c003a20 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.334+0000 7f2e96a00640 1 -- 172.21.15.119:0/1910826292 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980b4060 con 0x7f2e7c004d40 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.334+0000 7f2e96a00640 1 -- 172.21.15.119:0/1910826292 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c00dc20 con 0x7f2e98144040 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.334+0000 7f2e97400640 1 --2- 172.21.15.119:0/1910826292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e7c003a20 0x7f2e7c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.334+0000 7f2e8fe00640 1 --2- 172.21.15.119:0/1910826292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e98144040 0x7f2e7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.334+0000 7f2e97400640 1 -- 172.21.15.119:0/1910826292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e98144040 msgr2=0x7f2e7c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.334+0000 7f2e97400640 1 --2- 172.21.15.119:0/1910826292 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e98144040 0x7f2e7c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.334+0000 7f2e97400640 1 -- 172.21.15.119:0/1910826292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e7c004d40 msgr2=0x7f2e7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.334+0000 7f2e97400640 1 --2- 172.21.15.119:0/1910826292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e7c004d40 0x7f2e7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.334+0000 7f2e97400640 1 -- 172.21.15.119:0/1910826292 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c003520 con 0x7f2e7c003a20 2024-04-25T10:07:11.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.335+0000 7f2e97400640 1 --2- 172.21.15.119:0/1910826292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e7c003a20 0x7f2e7c003e40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e88009ac0 tx=0x7f2e7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81da041e195e9b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.336+0000 7f2e96a00640 1 -- 172.21.15.119:0/1910826292 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88036070 con 0x7f2e7c003a20 2024-04-25T10:07:11.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.336+0000 7f2e96a00640 1 -- 172.21.15.119:0/1910826292 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e88031400 con 0x7f2e7c003a20 2024-04-25T10:07:11.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.336+0000 7f2e96a00640 1 -- 172.21.15.119:0/1910826292 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88009690 con 0x7f2e7c003a20 2024-04-25T10:07:11.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.336+0000 7f2e9d000640 1 -- 172.21.15.119:0/1910826292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e7c003a20 msgr2=0x7f2e7c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.336+0000 7f2e9d000640 1 --2- 172.21.15.119:0/1910826292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e7c003a20 0x7f2e7c003e40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e88009ac0 tx=0x7f2e7c005830 comp rx=0 tx=0).stop 2024-04-25T10:07:11.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.336+0000 7f2e97e00640 1 -- 172.21.15.119:0/1910826292 reap_dead start 2024-04-25T10:07:11.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.336+0000 7f2e9d000640 1 -- 172.21.15.119:0/1910826292 shutdown_connections 2024-04-25T10:07:11.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.336+0000 7f2e9d000640 1 -- 172.21.15.119:0/1910826292 >> 172.21.15.119:0/1910826292 conn(0x7f2e9806b1d0 msgr2=0x7f2e980b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.337+0000 7f2e9d000640 1 -- 172.21.15.119:0/1910826292 shutdown_connections 2024-04-25T10:07:11.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.337+0000 7f2e9d000640 1 -- 172.21.15.119:0/1910826292 wait complete. 2024-04-25T10:07:11.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.338+0000 7f2e9d000640 1 Processor -- start 2024-04-25T10:07:11.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.339+0000 7f2e9d000640 1 -- start start 2024-04-25T10:07:11.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.339+0000 7f2e9d000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e7c003a20 0x7f2e981397a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e9d000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e98159c00 0x7f2e98159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e9d000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e98138410 0x7f2e980b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e9d000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980b5180 con 0x7f2e98138410 2024-04-25T10:07:11.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e9d000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980b4900 con 0x7f2e7c003a20 2024-04-25T10:07:11.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e9d000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980b5300 con 0x7f2e98159c00 2024-04-25T10:07:11.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e97e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e98138410 0x7f2e980b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e97e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e98138410 0x7f2e980b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34400/0 (socket says 172.21.15.119:34400) 2024-04-25T10:07:11.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e97e00640 1 -- 172.21.15.119:0/3155399564 learned_addr learned my addr 172.21.15.119:0/3155399564 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:11.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e8fe00640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e98159c00 0x7f2e98159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e97e00640 1 -- 172.21.15.119:0/3155399564 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e98159c00 msgr2=0x7f2e98159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e97e00640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e98159c00 0x7f2e98159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e97400640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e7c003a20 0x7f2e981397a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e97e00640 1 -- 172.21.15.119:0/3155399564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e7c003a20 msgr2=0x7f2e981397a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e97e00640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e7c003a20 0x7f2e981397a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e97e00640 1 -- 172.21.15.119:0/3155399564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e98139e10 con 0x7f2e98138410 2024-04-25T10:07:11.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.340+0000 7f2e8fe00640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e98159c00 0x7f2e98159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:11.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.341+0000 7f2e97e00640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e98138410 0x7f2e980b2420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2e9815f360 tx=0x7f2e9000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.341+0000 7f2e8f400640 1 -- 172.21.15.119:0/3155399564 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90018020 con 0x7f2e98138410 2024-04-25T10:07:11.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.341+0000 7f2e8f400640 1 -- 172.21.15.119:0/3155399564 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9000bd10 con 0x7f2e98138410 2024-04-25T10:07:11.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.341+0000 7f2e9d000640 1 -- 172.21.15.119:0/3155399564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9813a0f0 con 0x7f2e98138410 2024-04-25T10:07:11.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.341+0000 7f2e8f400640 1 -- 172.21.15.119:0/3155399564 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90014640 con 0x7f2e98138410 2024-04-25T10:07:11.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.341+0000 7f2e9d000640 1 -- 172.21.15.119:0/3155399564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e98137ef0 con 0x7f2e98138410 2024-04-25T10:07:11.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.342+0000 7f2e9d000640 1 -- 172.21.15.119:0/3155399564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e981380f0 con 0x7f2e98138410 2024-04-25T10:07:11.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.343+0000 7f2e8f400640 1 -- 172.21.15.119:0/3155399564 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e9001b020 con 0x7f2e98138410 2024-04-25T10:07:11.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.344+0000 7f2e8f400640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2e7406b500 0x7f2e7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.344+0000 7f2e8f400640 1 -- 172.21.15.119:0/3155399564 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e9008a960 con 0x7f2e98138410 2024-04-25T10:07:11.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.344+0000 7f2e97400640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2e7406b500 0x7f2e7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.345+0000 7f2e97400640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2e7406b500 0x7f2e7406d9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e88033000 tx=0x7f2e88008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.348+0000 7f2e8f400640 1 -- 172.21.15.119:0/3155399564 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e981380f0 con 0x7f2e98138410 2024-04-25T10:07:11.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.548+0000 7f2e9d000640 1 -- 172.21.15.119:0/3155399564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e98142090 con 0x7f2e98138410 2024-04-25T10:07:11.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.551+0000 7f2e8f400640 1 -- 172.21.15.119:0/3155399564 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f2e90022020 con 0x7f2e98138410 2024-04-25T10:07:11.557 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:11.557 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:08.319104+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.554+0000 7f2e9d000640 1 -- 172.21.15.119:0/3155399564 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2e7406b500 msgr2=0x7f2e7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.554+0000 7f2e9d000640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2e7406b500 0x7f2e7406d9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e88033000 tx=0x7f2e88008030 comp rx=0 tx=0).stop 2024-04-25T10:07:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.554+0000 7f2e9d000640 1 -- 172.21.15.119:0/3155399564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e98138410 msgr2=0x7f2e980b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.554+0000 7f2e9d000640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e98138410 0x7f2e980b2420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2e9815f360 tx=0x7f2e9000b480 comp rx=0 tx=0).stop 2024-04-25T10:07:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.554+0000 7f2e9d000640 1 -- 172.21.15.119:0/3155399564 shutdown_connections 2024-04-25T10:07:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.554+0000 7f2e9d000640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2e7406b500 0x7f2e7406d9c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.554+0000 7f2e9d000640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e98138410 0x7f2e980b2420 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.554+0000 7f2e9d000640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e98159c00 0x7f2e98159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.554+0000 7f2e9d000640 1 --2- 172.21.15.119:0/3155399564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e7c003a20 0x7f2e981397a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.554+0000 7f2e9d000640 1 -- 172.21.15.119:0/3155399564 >> 172.21.15.119:0/3155399564 conn(0x7f2e9806b1d0 msgr2=0x7f2e980af360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.554+0000 7f2e9d000640 1 -- 172.21.15.119:0/3155399564 shutdown_connections 2024-04-25T10:07:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.554+0000 7f2e9d000640 1 -- 172.21.15.119:0/3155399564 wait complete. 2024-04-25T10:07:11.579 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:07:11.579 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:11.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.780+0000 7f803ac00640 1 Processor -- start 2024-04-25T10:07:11.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.780+0000 7f803ac00640 1 -- start start 2024-04-25T10:07:11.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.781+0000 7f803ac00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f803415a870 0x7f80340cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.781+0000 7f803ac00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80340cdb10 con 0x7f803406e2e0 2024-04-25T10:07:11.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.781+0000 7f803ac00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80340cddf0 con 0x7f80340cfc40 2024-04-25T10:07:11.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.781+0000 7f803ac00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80340ce720 con 0x7f803415a870 2024-04-25T10:07:11.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.782+0000 7f802b400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f803406e2e0 0x7f80340cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42072/0 (socket says 172.21.15.119:42072) 2024-04-25T10:07:11.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.782+0000 7f802b400640 1 -- 172.21.15.119:0/2765900041 learned_addr learned my addr 172.21.15.119:0/2765900041 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:11.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.782+0000 7f8033e00640 1 --2- 172.21.15.119:0/2765900041 >> v2:172.21.15.119:3300/0 conn(0x7f803415a870 0x7f80340cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.783+0000 7f8033e00640 1 -- 172.21.15.119:0/2765900041 >> v1:172.21.15.132:6789/0 conn(0x7f80340cfc40 legacy=0x7f803415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.783+0000 7f8033400640 1 -- 172.21.15.119:0/2765900041 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 259439175 0 0) 0x7f80340cdb10 con 0x7f803406e2e0 2024-04-25T10:07:11.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.783+0000 7f8033e00640 1 -- 172.21.15.119:0/2765900041 >> v1:172.21.15.119:6789/0 conn(0x7f803406e2e0 legacy=0x7f80340cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.783+0000 7f8033e00640 1 -- 172.21.15.119:0/2765900041 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80340ce0d0 con 0x7f803415a870 2024-04-25T10:07:11.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.784+0000 7f8033e00640 1 --2- 172.21.15.119:0/2765900041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803415a870 0x7f80340cebb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f80240097d0 tx=0x7f802402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911fa498a461fdfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.785+0000 7f8033400640 1 -- 172.21.15.119:0/2765900041 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024037070 con 0x7f803415a870 2024-04-25T10:07:11.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.785+0000 7f8033400640 1 -- 172.21.15.119:0/2765900041 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8024032440 con 0x7f803415a870 2024-04-25T10:07:11.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.785+0000 7f8033400640 1 -- 172.21.15.119:0/2765900041 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024032810 con 0x7f803415a870 2024-04-25T10:07:11.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.785+0000 7f803ac00640 1 -- 172.21.15.119:0/2765900041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803415a870 msgr2=0x7f80340cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.785+0000 7f803ac00640 1 --2- 172.21.15.119:0/2765900041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803415a870 0x7f80340cebb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f80240097d0 tx=0x7f802402a210 comp rx=0 tx=0).stop 2024-04-25T10:07:11.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.785+0000 7f803ac00640 1 -- 172.21.15.119:0/2765900041 shutdown_connections 2024-04-25T10:07:11.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.785+0000 7f803ac00640 1 --2- 172.21.15.119:0/2765900041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803415a870 0x7f80340cebb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.785+0000 7f803ac00640 1 -- 172.21.15.119:0/2765900041 >> 172.21.15.119:0/2765900041 conn(0x7f803406aba0 msgr2=0x7f803406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.786+0000 7f803ac00640 1 -- 172.21.15.119:0/2765900041 shutdown_connections 2024-04-25T10:07:11.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.786+0000 7f803ac00640 1 -- 172.21.15.119:0/2765900041 wait complete. 2024-04-25T10:07:11.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.787+0000 7f803ac00640 1 Processor -- start 2024-04-25T10:07:11.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.787+0000 7f803ac00640 1 -- start start 2024-04-25T10:07:11.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.788+0000 7f803ac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803406e2e0 0x7f803414b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.788+0000 7f803ac00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80340cfc40 0x7f803415c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.788+0000 7f803ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f803415a870 0x7f8034168ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.788+0000 7f803ac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80340cd700 con 0x7f803406e2e0 2024-04-25T10:07:11.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.788+0000 7f803ac00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80340cd580 con 0x7f80340cfc40 2024-04-25T10:07:11.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.788+0000 7f803ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80340cd880 con 0x7f803415a870 2024-04-25T10:07:11.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.788+0000 7f8033e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803406e2e0 0x7f803414b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.788+0000 7f8033e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803406e2e0 0x7f803414b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34416/0 (socket says 172.21.15.119:34416) 2024-04-25T10:07:11.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.788+0000 7f8033e00640 1 -- 172.21.15.119:0/751239942 learned_addr learned my addr 172.21.15.119:0/751239942 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:11.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.789+0000 7f802b400640 1 --2- 172.21.15.119:0/751239942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80340cfc40 0x7f803415c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.789+0000 7f8033e00640 1 -- 172.21.15.119:0/751239942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f803415a870 msgr2=0x7f8034168ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.789+0000 7f8033e00640 1 --2- 172.21.15.119:0/751239942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f803415a870 0x7f8034168ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.789+0000 7f8033e00640 1 -- 172.21.15.119:0/751239942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80340cfc40 msgr2=0x7f803415c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.789+0000 7f8033e00640 1 --2- 172.21.15.119:0/751239942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80340cfc40 0x7f803415c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.789+0000 7f8033e00640 1 -- 172.21.15.119:0/751239942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803415d030 con 0x7f803406e2e0 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.789+0000 7f8033e00640 1 --2- 172.21.15.119:0/751239942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803406e2e0 0x7f803414b630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f80240098f0 tx=0x7f80240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.789+0000 7f8030c00640 1 -- 172.21.15.119:0/751239942 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024041070 con 0x7f803406e2e0 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.789+0000 7f8030c00640 1 -- 172.21.15.119:0/751239942 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802403c930 con 0x7f803406e2e0 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.789+0000 7f8030c00640 1 -- 172.21.15.119:0/751239942 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024037040 con 0x7f803406e2e0 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.789+0000 7f803ac00640 1 -- 172.21.15.119:0/751239942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803415cd50 con 0x7f803406e2e0 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.790+0000 7f803ac00640 1 -- 172.21.15.119:0/751239942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80340accc0 con 0x7f803406e2e0 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.791+0000 7f803ac00640 1 -- 172.21.15.119:0/751239942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80340749b0 con 0x7f803406e2e0 2024-04-25T10:07:11.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.791+0000 7f8030c00640 1 -- 172.21.15.119:0/751239942 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8024044070 con 0x7f803406e2e0 2024-04-25T10:07:11.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.792+0000 7f8030c00640 1 --2- 172.21.15.119:0/751239942 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f801006b4d0 0x7f801006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.792+0000 7f8030c00640 1 -- 172.21.15.119:0/751239942 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80240aaf70 con 0x7f803406e2e0 2024-04-25T10:07:11.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.793+0000 7f802b400640 1 --2- 172.21.15.119:0/751239942 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f801006b4d0 0x7f801006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.793+0000 7f802b400640 1 --2- 172.21.15.119:0/751239942 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f801006b4d0 0x7f801006d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f801c007fd0 tx=0x7f801c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.797+0000 7f8030c00640 1 -- 172.21.15.119:0/751239942 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8024079f00 con 0x7f803406e2e0 2024-04-25T10:07:11.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:11.990+0000 7f803ac00640 1 -- 172.21.15.119:0/751239942 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8034071e40 con 0x7f801006b4d0 2024-04-25T10:07:12.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.006+0000 7f8030c00640 1 -- 172.21.15.119:0/751239942 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251947 (secure 0 0 0) 0x7f8034071e40 con 0x7f801006b4d0 2024-04-25T10:07:12.015 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:12.016 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:07:12.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.011+0000 7f803ac00640 1 -- 172.21.15.119:0/751239942 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f801006b4d0 msgr2=0x7f801006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.011+0000 7f803ac00640 1 --2- 172.21.15.119:0/751239942 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f801006b4d0 0x7f801006d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f801c007fd0 tx=0x7f801c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:12.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.012+0000 7f803ac00640 1 -- 172.21.15.119:0/751239942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803406e2e0 msgr2=0x7f803414b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.012+0000 7f803ac00640 1 --2- 172.21.15.119:0/751239942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803406e2e0 0x7f803414b630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f80240098f0 tx=0x7f80240046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.012+0000 7f803ac00640 1 -- 172.21.15.119:0/751239942 shutdown_connections 2024-04-25T10:07:12.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.012+0000 7f803ac00640 1 --2- 172.21.15.119:0/751239942 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f801006b4d0 0x7f801006d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.012+0000 7f803ac00640 1 --2- 172.21.15.119:0/751239942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f803415a870 0x7f8034168ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.012+0000 7f803ac00640 1 --2- 172.21.15.119:0/751239942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80340cfc40 0x7f803415c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.012+0000 7f803ac00640 1 --2- 172.21.15.119:0/751239942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803406e2e0 0x7f803414b630 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.012+0000 7f803ac00640 1 -- 172.21.15.119:0/751239942 >> 172.21.15.119:0/751239942 conn(0x7f803406aba0 msgr2=0x7f80340cb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.012+0000 7f803ac00640 1 -- 172.21.15.119:0/751239942 shutdown_connections 2024-04-25T10:07:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.012+0000 7f803ac00640 1 -- 172.21.15.119:0/751239942 wait complete. 2024-04-25T10:07:12.036 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T10:07:11.924137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2812,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2879488,"data_stored":1746728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001866"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117497+0000","last_change":"2024-04-25T10:07:07.357761+0000","last_active":"2024-04-25T10:07:10.117497+0000","last_peered":"2024-04-25T10:07:10.117497+0000","last_clean":"2024-04-25T10:07:10.117497+0000","last_became_active":"2024-04-25T10:07:07.357419+0000","last_became_peered":"2024-04-25T10:07:07.357419+0000","last_unstale":"2024-04-25T10:07:10.117497+0000","last_undegraded":"2024-04-25T10:07:10.117497+0000","last_fullsized":"2024-04-25T10:07:10.117497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:31.725733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340438+0000","last_change":"2024-04-25T10:07:07.324906+0000","last_active":"2024-04-25T10:07:08.340438+0000","last_peered":"2024-04-25T10:07:08.340438+0000","last_clean":"2024-04-25T10:07:08.340438+0000","last_became_active":"2024-04-25T10:07:07.324635+0000","last_became_peered":"2024-04-25T10:07:07.324635+0000","last_unstale":"2024-04-25T10:07:08.340438+0000","last_undegraded":"2024-04-25T10:07:08.340438+0000","last_fullsized":"2024-04-25T10:07:08.340438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013703+0000","last_change":"2024-04-25T10:07:07.324537+0000","last_active":"2024-04-25T10:07:09.013703+0000","last_peered":"2024-04-25T10:07:09.013703+0000","last_clean":"2024-04-25T10:07:09.013703+0000","last_became_active":"2024-04-25T10:07:07.324290+0000","last_became_peered":"2024-04-25T10:07:07.324290+0000","last_unstale":"2024-04-25T10:07:09.013703+0000","last_undegraded":"2024-04-25T10:07:09.013703+0000","last_fullsized":"2024-04-25T10:07:09.013703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340924+0000","last_change":"2024-04-25T10:07:07.336252+0000","last_active":"2024-04-25T10:07:08.340924+0000","last_peered":"2024-04-25T10:07:08.340924+0000","last_clean":"2024-04-25T10:07:08.340924+0000","last_became_active":"2024-04-25T10:07:07.335889+0000","last_became_peered":"2024-04-25T10:07:07.335889+0000","last_unstale":"2024-04-25T10:07:08.340924+0000","last_undegraded":"2024-04-25T10:07:08.340924+0000","last_fullsized":"2024-04-25T10:07:08.340924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116666+0000","last_change":"2024-04-25T10:07:07.354080+0000","last_active":"2024-04-25T10:07:10.116666+0000","last_peered":"2024-04-25T10:07:10.116666+0000","last_clean":"2024-04-25T10:07:10.116666+0000","last_became_active":"2024-04-25T10:07:07.351587+0000","last_became_peered":"2024-04-25T10:07:07.351587+0000","last_unstale":"2024-04-25T10:07:10.116666+0000","last_undegraded":"2024-04-25T10:07:10.116666+0000","last_fullsized":"2024-04-25T10:07:10.116666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340152+0000","last_change":"2024-04-25T10:07:07.329505+0000","last_active":"2024-04-25T10:07:08.340152+0000","last_peered":"2024-04-25T10:07:08.340152+0000","last_clean":"2024-04-25T10:07:08.340152+0000","last_became_active":"2024-04-25T10:07:07.327756+0000","last_became_peered":"2024-04-25T10:07:07.327756+0000","last_unstale":"2024-04-25T10:07:08.340152+0000","last_undegraded":"2024-04-25T10:07:08.340152+0000","last_fullsized":"2024-04-25T10:07:08.340152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116453+0000","last_change":"2024-04-25T10:07:07.354368+0000","last_active":"2024-04-25T10:07:10.116453+0000","last_peered":"2024-04-25T10:07:10.116453+0000","last_clean":"2024-04-25T10:07:10.116453+0000","last_became_active":"2024-04-25T10:07:07.354048+0000","last_became_peered":"2024-04-25T10:07:07.354048+0000","last_unstale":"2024-04-25T10:07:10.116453+0000","last_undegraded":"2024-04-25T10:07:10.116453+0000","last_fullsized":"2024-04-25T10:07:10.116453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430715+0000","last_change":"2024-04-25T10:07:07.337913+0000","last_active":"2024-04-25T10:07:08.430715+0000","last_peered":"2024-04-25T10:07:08.430715+0000","last_clean":"2024-04-25T10:07:08.430715+0000","last_became_active":"2024-04-25T10:07:07.337268+0000","last_became_peered":"2024-04-25T10:07:07.337268+0000","last_unstale":"2024-04-25T10:07:08.430715+0000","last_undegraded":"2024-04-25T10:07:08.430715+0000","last_fullsized":"2024-04-25T10:07:08.430715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:01.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117721+0000","last_change":"2024-04-25T10:07:07.352856+0000","last_active":"2024-04-25T10:07:10.117721+0000","last_peered":"2024-04-25T10:07:10.117721+0000","last_clean":"2024-04-25T10:07:10.117721+0000","last_became_active":"2024-04-25T10:07:07.352555+0000","last_became_peered":"2024-04-25T10:07:07.352555+0000","last_unstale":"2024-04-25T10:07:10.117721+0000","last_undegraded":"2024-04-25T10:07:10.117721+0000","last_fullsized":"2024-04-25T10:07:10.117721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.763141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627954+0000","last_change":"2024-04-25T10:07:07.335997+0000","last_active":"2024-04-25T10:07:09.627954+0000","last_peered":"2024-04-25T10:07:09.627954+0000","last_clean":"2024-04-25T10:07:09.627954+0000","last_became_active":"2024-04-25T10:07:07.334634+0000","last_became_peered":"2024-04-25T10:07:07.334634+0000","last_unstale":"2024-04-25T10:07:09.627954+0000","last_undegraded":"2024-04-25T10:07:09.627954+0000","last_fullsized":"2024-04-25T10:07:09.627954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627400+0000","last_change":"2024-04-25T10:07:07.336470+0000","last_active":"2024-04-25T10:07:09.627400+0000","last_peered":"2024-04-25T10:07:09.627400+0000","last_clean":"2024-04-25T10:07:09.627400+0000","last_became_active":"2024-04-25T10:07:07.334875+0000","last_became_peered":"2024-04-25T10:07:07.334875+0000","last_unstale":"2024-04-25T10:07:09.627400+0000","last_undegraded":"2024-04-25T10:07:09.627400+0000","last_fullsized":"2024-04-25T10:07:09.627400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115690+0000","last_change":"2024-04-25T10:07:07.342067+0000","last_active":"2024-04-25T10:07:10.115690+0000","last_peered":"2024-04-25T10:07:10.115690+0000","last_clean":"2024-04-25T10:07:10.115690+0000","last_became_active":"2024-04-25T10:07:07.341655+0000","last_became_peered":"2024-04-25T10:07:07.341655+0000","last_unstale":"2024-04-25T10:07:10.115690+0000","last_undegraded":"2024-04-25T10:07:10.115690+0000","last_fullsized":"2024-04-25T10:07:10.115690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:37.664841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431262+0000","last_change":"2024-04-25T10:07:07.337652+0000","last_active":"2024-04-25T10:07:08.431262+0000","last_peered":"2024-04-25T10:07:08.431262+0000","last_clean":"2024-04-25T10:07:08.431262+0000","last_became_active":"2024-04-25T10:07:07.337057+0000","last_became_peered":"2024-04-25T10:07:07.337057+0000","last_unstale":"2024-04-25T10:07:08.431262+0000","last_undegraded":"2024-04-25T10:07:08.431262+0000","last_fullsized":"2024-04-25T10:07:08.431262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.012844+0000","last_change":"2024-04-25T10:07:07.341549+0000","last_active":"2024-04-25T10:07:09.012844+0000","last_peered":"2024-04-25T10:07:09.012844+0000","last_clean":"2024-04-25T10:07:09.012844+0000","last_became_active":"2024-04-25T10:07:07.341228+0000","last_became_peered":"2024-04-25T10:07:07.341228+0000","last_unstale":"2024-04-25T10:07:09.012844+0000","last_undegraded":"2024-04-25T10:07:09.012844+0000","last_fullsized":"2024-04-25T10:07:09.012844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:50.697002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339480+0000","last_change":"2024-04-25T10:07:07.348054+0000","last_active":"2024-04-25T10:07:08.339480+0000","last_peered":"2024-04-25T10:07:08.339480+0000","last_clean":"2024-04-25T10:07:08.339480+0000","last_became_active":"2024-04-25T10:07:07.347739+0000","last_became_peered":"2024-04-25T10:07:07.347739+0000","last_unstale":"2024-04-25T10:07:08.339480+0000","last_undegraded":"2024-04-25T10:07:08.339480+0000","last_fullsized":"2024-04-25T10:07:08.339480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339967+0000","last_change":"2024-04-25T10:07:07.326858+0000","last_active":"2024-04-25T10:07:08.339967+0000","last_peered":"2024-04-25T10:07:08.339967+0000","last_clean":"2024-04-25T10:07:08.339967+0000","last_became_active":"2024-04-25T10:07:07.326107+0000","last_became_peered":"2024-04-25T10:07:07.326107+0000","last_unstale":"2024-04-25T10:07:08.339967+0000","last_undegraded":"2024-04-25T10:07:08.339967+0000","last_fullsized":"2024-04-25T10:07:08.339967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.342255+0000","last_change":"2024-04-25T10:07:07.348628+0000","last_active":"2024-04-25T10:07:08.342255+0000","last_peered":"2024-04-25T10:07:08.342255+0000","last_clean":"2024-04-25T10:07:08.342255+0000","last_became_active":"2024-04-25T10:07:07.348356+0000","last_became_peered":"2024-04-25T10:07:07.348356+0000","last_unstale":"2024-04-25T10:07:08.342255+0000","last_undegraded":"2024-04-25T10:07:08.342255+0000","last_fullsized":"2024-04-25T10:07:08.342255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:16.375835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.338486+0000","last_change":"2024-04-25T10:07:07.341185+0000","last_active":"2024-04-25T10:07:08.338486+0000","last_peered":"2024-04-25T10:07:08.338486+0000","last_clean":"2024-04-25T10:07:08.338486+0000","last_became_active":"2024-04-25T10:07:07.340797+0000","last_became_peered":"2024-04-25T10:07:07.340797+0000","last_unstale":"2024-04-25T10:07:08.338486+0000","last_undegraded":"2024-04-25T10:07:08.338486+0000","last_fullsized":"2024-04-25T10:07:08.338486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:49.701271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.118018+0000","last_change":"2024-04-25T10:07:07.356663+0000","last_active":"2024-04-25T10:07:10.118018+0000","last_peered":"2024-04-25T10:07:10.118018+0000","last_clean":"2024-04-25T10:07:10.118018+0000","last_became_active":"2024-04-25T10:07:07.356381+0000","last_became_peered":"2024-04-25T10:07:07.356381+0000","last_unstale":"2024-04-25T10:07:10.118018+0000","last_undegraded":"2024-04-25T10:07:10.118018+0000","last_fullsized":"2024-04-25T10:07:10.118018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:51.771853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627639+0000","last_change":"2024-04-25T10:07:07.348329+0000","last_active":"2024-04-25T10:07:09.627639+0000","last_peered":"2024-04-25T10:07:09.627639+0000","last_clean":"2024-04-25T10:07:09.627639+0000","last_became_active":"2024-04-25T10:07:07.347033+0000","last_became_peered":"2024-04-25T10:07:07.347033+0000","last_unstale":"2024-04-25T10:07:09.627639+0000","last_undegraded":"2024-04-25T10:07:09.627639+0000","last_fullsized":"2024-04-25T10:07:09.627639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013394+0000","last_change":"2024-04-25T10:07:07.350923+0000","last_active":"2024-04-25T10:07:09.013394+0000","last_peered":"2024-04-25T10:07:09.013394+0000","last_clean":"2024-04-25T10:07:09.013394+0000","last_became_active":"2024-04-25T10:07:07.350102+0000","last_became_peered":"2024-04-25T10:07:07.350102+0000","last_unstale":"2024-04-25T10:07:09.013394+0000","last_undegraded":"2024-04-25T10:07:09.013394+0000","last_fullsized":"2024-04-25T10:07:09.013394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340239+0000","last_change":"2024-04-25T10:07:07.351751+0000","last_active":"2024-04-25T10:07:08.340239+0000","last_peered":"2024-04-25T10:07:08.340239+0000","last_clean":"2024-04-25T10:07:08.340239+0000","last_became_active":"2024-04-25T10:07:07.349688+0000","last_became_peered":"2024-04-25T10:07:07.349688+0000","last_unstale":"2024-04-25T10:07:08.340239+0000","last_undegraded":"2024-04-25T10:07:08.340239+0000","last_fullsized":"2024-04-25T10:07:08.340239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430799+0000","last_change":"2024-04-25T10:07:07.346636+0000","last_active":"2024-04-25T10:07:08.430799+0000","last_peered":"2024-04-25T10:07:08.430799+0000","last_clean":"2024-04-25T10:07:08.430799+0000","last_became_active":"2024-04-25T10:07:07.346448+0000","last_became_peered":"2024-04-25T10:07:07.346448+0000","last_unstale":"2024-04-25T10:07:08.430799+0000","last_undegraded":"2024-04-25T10:07:08.430799+0000","last_fullsized":"2024-04-25T10:07:08.430799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.370945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339964+0000","last_change":"2024-04-25T10:07:07.327120+0000","last_active":"2024-04-25T10:07:08.339964+0000","last_peered":"2024-04-25T10:07:08.339964+0000","last_clean":"2024-04-25T10:07:08.339964+0000","last_became_active":"2024-04-25T10:07:07.326471+0000","last_became_peered":"2024-04-25T10:07:07.326471+0000","last_unstale":"2024-04-25T10:07:08.339964+0000","last_undegraded":"2024-04-25T10:07:08.339964+0000","last_fullsized":"2024-04-25T10:07:08.339964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431521+0000","last_change":"2024-04-25T10:07:07.346515+0000","last_active":"2024-04-25T10:07:08.431521+0000","last_peered":"2024-04-25T10:07:08.431521+0000","last_clean":"2024-04-25T10:07:08.431521+0000","last_became_active":"2024-04-25T10:07:07.346218+0000","last_became_peered":"2024-04-25T10:07:07.346218+0000","last_unstale":"2024-04-25T10:07:08.431521+0000","last_undegraded":"2024-04-25T10:07:08.431521+0000","last_fullsized":"2024-04-25T10:07:08.431521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:49.296728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627914+0000","last_change":"2024-04-25T10:07:07.352410+0000","last_active":"2024-04-25T10:07:09.627914+0000","last_peered":"2024-04-25T10:07:09.627914+0000","last_clean":"2024-04-25T10:07:09.627914+0000","last_became_active":"2024-04-25T10:07:07.352106+0000","last_became_peered":"2024-04-25T10:07:07.352106+0000","last_unstale":"2024-04-25T10:07:09.627914+0000","last_undegraded":"2024-04-25T10:07:09.627914+0000","last_fullsized":"2024-04-25T10:07:09.627914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339814+0000","last_change":"2024-04-25T10:07:07.339497+0000","last_active":"2024-04-25T10:07:08.339814+0000","last_peered":"2024-04-25T10:07:08.339814+0000","last_clean":"2024-04-25T10:07:08.339814+0000","last_became_active":"2024-04-25T10:07:07.339168+0000","last_became_peered":"2024-04-25T10:07:07.339168+0000","last_unstale":"2024-04-25T10:07:08.339814+0000","last_undegraded":"2024-04-25T10:07:08.339814+0000","last_fullsized":"2024-04-25T10:07:08.339814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431449+0000","last_change":"2024-04-25T10:07:07.349878+0000","last_active":"2024-04-25T10:07:08.431449+0000","last_peered":"2024-04-25T10:07:08.431449+0000","last_clean":"2024-04-25T10:07:08.431449+0000","last_became_active":"2024-04-25T10:07:07.349602+0000","last_became_peered":"2024-04-25T10:07:07.349602+0000","last_unstale":"2024-04-25T10:07:08.431449+0000","last_undegraded":"2024-04-25T10:07:08.431449+0000","last_fullsized":"2024-04-25T10:07:08.431449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013693+0000","last_change":"2024-04-25T10:07:07.331427+0000","last_active":"2024-04-25T10:07:09.013693+0000","last_peered":"2024-04-25T10:07:09.013693+0000","last_clean":"2024-04-25T10:07:09.013693+0000","last_became_active":"2024-04-25T10:07:07.329911+0000","last_became_peered":"2024-04-25T10:07:07.329911+0000","last_unstale":"2024-04-25T10:07:09.013693+0000","last_undegraded":"2024-04-25T10:07:09.013693+0000","last_fullsized":"2024-04-25T10:07:09.013693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340563+0000","last_change":"2024-04-25T10:07:07.329226+0000","last_active":"2024-04-25T10:07:08.340563+0000","last_peered":"2024-04-25T10:07:08.340563+0000","last_clean":"2024-04-25T10:07:08.340563+0000","last_became_active":"2024-04-25T10:07:07.327329+0000","last_became_peered":"2024-04-25T10:07:07.327329+0000","last_unstale":"2024-04-25T10:07:08.340563+0000","last_undegraded":"2024-04-25T10:07:08.340563+0000","last_fullsized":"2024-04-25T10:07:08.340563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:49.296728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337552+0000","last_change":"2024-04-25T10:07:07.327804+0000","last_active":"2024-04-25T10:07:08.337552+0000","last_peered":"2024-04-25T10:07:08.337552+0000","last_clean":"2024-04-25T10:07:08.337552+0000","last_became_active":"2024-04-25T10:07:07.327516+0000","last_became_peered":"2024-04-25T10:07:07.327516+0000","last_unstale":"2024-04-25T10:07:08.337552+0000","last_undegraded":"2024-04-25T10:07:08.337552+0000","last_fullsized":"2024-04-25T10:07:08.337552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:46.704814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627063+0000","last_change":"2024-04-25T10:07:07.333594+0000","last_active":"2024-04-25T10:07:09.627063+0000","last_peered":"2024-04-25T10:07:09.627063+0000","last_clean":"2024-04-25T10:07:09.627063+0000","last_became_active":"2024-04-25T10:07:07.333302+0000","last_became_peered":"2024-04-25T10:07:07.333302+0000","last_unstale":"2024-04-25T10:07:09.627063+0000","last_undegraded":"2024-04-25T10:07:09.627063+0000","last_fullsized":"2024-04-25T10:07:09.627063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:01.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.114966+0000","last_change":"2024-04-25T10:07:07.333101+0000","last_active":"2024-04-25T10:07:10.114966+0000","last_peered":"2024-04-25T10:07:10.114966+0000","last_clean":"2024-04-25T10:07:10.114966+0000","last_became_active":"2024-04-25T10:07:07.332714+0000","last_became_peered":"2024-04-25T10:07:07.332714+0000","last_unstale":"2024-04-25T10:07:10.114966+0000","last_undegraded":"2024-04-25T10:07:10.114966+0000","last_fullsized":"2024-04-25T10:07:10.114966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:53.471918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431440+0000","last_change":"2024-04-25T10:07:06.322470+0000","last_active":"2024-04-25T10:07:08.431440+0000","last_peered":"2024-04-25T10:07:08.431440+0000","last_clean":"2024-04-25T10:07:08.431440+0000","last_became_active":"2024-04-25T10:07:06.321092+0000","last_became_peered":"2024-04-25T10:07:06.321092+0000","last_unstale":"2024-04-25T10:07:08.431440+0000","last_undegraded":"2024-04-25T10:07:08.431440+0000","last_fullsized":"2024-04-25T10:07:08.431440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:28.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013819+0000","last_change":"2024-04-25T10:07:07.340326+0000","last_active":"2024-04-25T10:07:09.013819+0000","last_peered":"2024-04-25T10:07:09.013819+0000","last_clean":"2024-04-25T10:07:09.013819+0000","last_became_active":"2024-04-25T10:07:07.339889+0000","last_became_peered":"2024-04-25T10:07:07.339889+0000","last_unstale":"2024-04-25T10:07:09.013819+0000","last_undegraded":"2024-04-25T10:07:09.013819+0000","last_fullsized":"2024-04-25T10:07:09.013819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.626769+0000","last_change":"2024-04-25T10:07:06.333386+0000","last_active":"2024-04-25T10:07:09.626769+0000","last_peered":"2024-04-25T10:07:09.626769+0000","last_clean":"2024-04-25T10:07:09.626769+0000","last_became_active":"2024-04-25T10:07:06.333014+0000","last_became_peered":"2024-04-25T10:07:06.333014+0000","last_unstale":"2024-04-25T10:07:09.626769+0000","last_undegraded":"2024-04-25T10:07:09.626769+0000","last_fullsized":"2024-04-25T10:07:09.626769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:53.885696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430668+0000","last_change":"2024-04-25T10:07:06.314017+0000","last_active":"2024-04-25T10:07:08.430668+0000","last_peered":"2024-04-25T10:07:08.430668+0000","last_clean":"2024-04-25T10:07:08.430668+0000","last_became_active":"2024-04-25T10:07:06.313712+0000","last_became_peered":"2024-04-25T10:07:06.313712+0000","last_unstale":"2024-04-25T10:07:08.430668+0000","last_undegraded":"2024-04-25T10:07:08.430668+0000","last_fullsized":"2024-04-25T10:07:08.430668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:05.470696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430845+0000","last_change":"2024-04-25T10:07:07.323982+0000","last_active":"2024-04-25T10:07:08.430845+0000","last_peered":"2024-04-25T10:07:08.430845+0000","last_clean":"2024-04-25T10:07:08.430845+0000","last_became_active":"2024-04-25T10:07:07.323664+0000","last_became_peered":"2024-04-25T10:07:07.323664+0000","last_unstale":"2024-04-25T10:07:08.430845+0000","last_undegraded":"2024-04-25T10:07:08.430845+0000","last_fullsized":"2024-04-25T10:07:08.430845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340135+0000","last_change":"2024-04-25T10:07:06.312709+0000","last_active":"2024-04-25T10:07:08.340135+0000","last_peered":"2024-04-25T10:07:08.340135+0000","last_clean":"2024-04-25T10:07:08.340135+0000","last_became_active":"2024-04-25T10:07:06.311785+0000","last_became_peered":"2024-04-25T10:07:06.311785+0000","last_unstale":"2024-04-25T10:07:08.340135+0000","last_undegraded":"2024-04-25T10:07:08.340135+0000","last_fullsized":"2024-04-25T10:07:08.340135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:52.458435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339243+0000","last_change":"2024-04-25T10:07:07.332051+0000","last_active":"2024-04-25T10:07:08.339243+0000","last_peered":"2024-04-25T10:07:08.339243+0000","last_clean":"2024-04-25T10:07:08.339243+0000","last_became_active":"2024-04-25T10:07:07.331656+0000","last_became_peered":"2024-04-25T10:07:07.331656+0000","last_unstale":"2024-04-25T10:07:08.339243+0000","last_undegraded":"2024-04-25T10:07:08.339243+0000","last_fullsized":"2024-04-25T10:07:08.339243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627326+0000","last_change":"2024-04-25T10:07:06.316859+0000","last_active":"2024-04-25T10:07:09.627326+0000","last_peered":"2024-04-25T10:07:09.627326+0000","last_clean":"2024-04-25T10:07:09.627326+0000","last_became_active":"2024-04-25T10:07:06.314454+0000","last_became_peered":"2024-04-25T10:07:06.314454+0000","last_unstale":"2024-04-25T10:07:09.627326+0000","last_undegraded":"2024-04-25T10:07:09.627326+0000","last_fullsized":"2024-04-25T10:07:09.627326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.867432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341342+0000","last_change":"2024-04-25T10:07:07.352036+0000","last_active":"2024-04-25T10:07:08.341342+0000","last_peered":"2024-04-25T10:07:08.341342+0000","last_clean":"2024-04-25T10:07:08.341342+0000","last_became_active":"2024-04-25T10:07:07.351244+0000","last_became_peered":"2024-04-25T10:07:07.351244+0000","last_unstale":"2024-04-25T10:07:08.341342+0000","last_undegraded":"2024-04-25T10:07:08.341342+0000","last_fullsized":"2024-04-25T10:07:08.341342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:31.725733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341722+0000","last_change":"2024-04-25T10:07:06.313325+0000","last_active":"2024-04-25T10:07:08.341722+0000","last_peered":"2024-04-25T10:07:08.341722+0000","last_clean":"2024-04-25T10:07:08.341722+0000","last_became_active":"2024-04-25T10:07:06.312922+0000","last_became_peered":"2024-04-25T10:07:06.312922+0000","last_unstale":"2024-04-25T10:07:08.341722+0000","last_undegraded":"2024-04-25T10:07:08.341722+0000","last_fullsized":"2024-04-25T10:07:08.341722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013388+0000","last_change":"2024-04-25T10:07:07.334790+0000","last_active":"2024-04-25T10:07:09.013388+0000","last_peered":"2024-04-25T10:07:09.013388+0000","last_clean":"2024-04-25T10:07:09.013388+0000","last_became_active":"2024-04-25T10:07:07.334587+0000","last_became_peered":"2024-04-25T10:07:07.334587+0000","last_unstale":"2024-04-25T10:07:09.013388+0000","last_undegraded":"2024-04-25T10:07:09.013388+0000","last_fullsized":"2024-04-25T10:07:09.013388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340029+0000","last_change":"2024-04-25T10:07:06.317802+0000","last_active":"2024-04-25T10:07:08.340029+0000","last_peered":"2024-04-25T10:07:08.340029+0000","last_clean":"2024-04-25T10:07:08.340029+0000","last_became_active":"2024-04-25T10:07:06.317291+0000","last_became_peered":"2024-04-25T10:07:06.317291+0000","last_unstale":"2024-04-25T10:07:08.340029+0000","last_undegraded":"2024-04-25T10:07:08.340029+0000","last_fullsized":"2024-04-25T10:07:08.340029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.867432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115871+0000","last_change":"2024-04-25T10:07:07.351461+0000","last_active":"2024-04-25T10:07:10.115871+0000","last_peered":"2024-04-25T10:07:10.115871+0000","last_clean":"2024-04-25T10:07:10.115871+0000","last_became_active":"2024-04-25T10:07:07.351141+0000","last_became_peered":"2024-04-25T10:07:07.351141+0000","last_unstale":"2024-04-25T10:07:10.115871+0000","last_undegraded":"2024-04-25T10:07:10.115871+0000","last_fullsized":"2024-04-25T10:07:10.115871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:46.704814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627332+0000","last_change":"2024-04-25T10:07:06.319663+0000","last_active":"2024-04-25T10:07:09.627332+0000","last_peered":"2024-04-25T10:07:09.627332+0000","last_clean":"2024-04-25T10:07:09.627332+0000","last_became_active":"2024-04-25T10:07:06.317321+0000","last_became_peered":"2024-04-25T10:07:06.317321+0000","last_unstale":"2024-04-25T10:07:09.627332+0000","last_undegraded":"2024-04-25T10:07:09.627332+0000","last_fullsized":"2024-04-25T10:07:09.627332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117185+0000","last_change":"2024-04-25T10:07:07.350498+0000","last_active":"2024-04-25T10:07:10.117185+0000","last_peered":"2024-04-25T10:07:10.117185+0000","last_clean":"2024-04-25T10:07:10.117185+0000","last_became_active":"2024-04-25T10:07:07.349929+0000","last_became_peered":"2024-04-25T10:07:07.349929+0000","last_unstale":"2024-04-25T10:07:10.117185+0000","last_undegraded":"2024-04-25T10:07:10.117185+0000","last_fullsized":"2024-04-25T10:07:10.117185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:23.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013849+0000","last_change":"2024-04-25T10:07:06.312357+0000","last_active":"2024-04-25T10:07:09.013849+0000","last_peered":"2024-04-25T10:07:09.013849+0000","last_clean":"2024-04-25T10:07:09.013849+0000","last_became_active":"2024-04-25T10:07:06.312088+0000","last_became_peered":"2024-04-25T10:07:06.312088+0000","last_unstale":"2024-04-25T10:07:09.013849+0000","last_undegraded":"2024-04-25T10:07:09.013849+0000","last_fullsized":"2024-04-25T10:07:09.013849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:05.470696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013146+0000","last_change":"2024-04-25T10:07:07.345439+0000","last_active":"2024-04-25T10:07:09.013146+0000","last_peered":"2024-04-25T10:07:09.013146+0000","last_clean":"2024-04-25T10:07:09.013146+0000","last_became_active":"2024-04-25T10:07:07.345205+0000","last_became_peered":"2024-04-25T10:07:07.345205+0000","last_unstale":"2024-04-25T10:07:09.013146+0000","last_undegraded":"2024-04-25T10:07:09.013146+0000","last_fullsized":"2024-04-25T10:07:09.013146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:56.522956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340351+0000","last_change":"2024-04-25T10:07:06.313807+0000","last_active":"2024-04-25T10:07:08.340351+0000","last_peered":"2024-04-25T10:07:08.340351+0000","last_clean":"2024-04-25T10:07:08.340351+0000","last_became_active":"2024-04-25T10:07:06.313376+0000","last_became_peered":"2024-04-25T10:07:06.313376+0000","last_unstale":"2024-04-25T10:07:08.340351+0000","last_undegraded":"2024-04-25T10:07:08.340351+0000","last_fullsized":"2024-04-25T10:07:08.340351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:44.436918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116968+0000","last_change":"2024-04-25T10:07:07.351251+0000","last_active":"2024-04-25T10:07:10.116968+0000","last_peered":"2024-04-25T10:07:10.116968+0000","last_clean":"2024-04-25T10:07:10.116968+0000","last_became_active":"2024-04-25T10:07:07.350404+0000","last_became_peered":"2024-04-25T10:07:07.350404+0000","last_unstale":"2024-04-25T10:07:10.116968+0000","last_undegraded":"2024-04-25T10:07:10.116968+0000","last_fullsized":"2024-04-25T10:07:10.116968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:49.296728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.626713+0000","last_change":"2024-04-25T10:07:06.322457+0000","last_active":"2024-04-25T10:07:09.626713+0000","last_peered":"2024-04-25T10:07:09.626713+0000","last_clean":"2024-04-25T10:07:09.626713+0000","last_became_active":"2024-04-25T10:07:06.318116+0000","last_became_peered":"2024-04-25T10:07:06.318116+0000","last_unstale":"2024-04-25T10:07:09.626713+0000","last_undegraded":"2024-04-25T10:07:09.626713+0000","last_fullsized":"2024-04-25T10:07:09.626713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:44.436918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627151+0000","last_change":"2024-04-25T10:07:07.328355+0000","last_active":"2024-04-25T10:07:09.627151+0000","last_peered":"2024-04-25T10:07:09.627151+0000","last_clean":"2024-04-25T10:07:09.627151+0000","last_became_active":"2024-04-25T10:07:07.328088+0000","last_became_peered":"2024-04-25T10:07:07.328088+0000","last_unstale":"2024-04-25T10:07:09.627151+0000","last_undegraded":"2024-04-25T10:07:09.627151+0000","last_fullsized":"2024-04-25T10:07:09.627151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337372+0000","last_change":"2024-04-25T10:07:06.306731+0000","last_active":"2024-04-25T10:07:08.337372+0000","last_peered":"2024-04-25T10:07:08.337372+0000","last_clean":"2024-04-25T10:07:08.337372+0000","last_became_active":"2024-04-25T10:07:06.306314+0000","last_became_peered":"2024-04-25T10:07:06.306314+0000","last_unstale":"2024-04-25T10:07:08.337372+0000","last_undegraded":"2024-04-25T10:07:08.337372+0000","last_fullsized":"2024-04-25T10:07:08.337372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:28.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117145+0000","last_change":"2024-04-25T10:07:07.356371+0000","last_active":"2024-04-25T10:07:10.117145+0000","last_peered":"2024-04-25T10:07:10.117145+0000","last_clean":"2024-04-25T10:07:10.117145+0000","last_became_active":"2024-04-25T10:07:07.352731+0000","last_became_peered":"2024-04-25T10:07:07.352731+0000","last_unstale":"2024-04-25T10:07:10.117145+0000","last_undegraded":"2024-04-25T10:07:10.117145+0000","last_fullsized":"2024-04-25T10:07:10.117145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115672+0000","last_change":"2024-04-25T10:07:06.320436+0000","last_active":"2024-04-25T10:07:10.115672+0000","last_peered":"2024-04-25T10:07:10.115672+0000","last_clean":"2024-04-25T10:07:10.115672+0000","last_became_active":"2024-04-25T10:07:06.319958+0000","last_became_peered":"2024-04-25T10:07:06.319958+0000","last_unstale":"2024-04-25T10:07:10.115672+0000","last_undegraded":"2024-04-25T10:07:10.115672+0000","last_fullsized":"2024-04-25T10:07:10.115672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:02.968109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117278+0000","last_change":"2024-04-25T10:07:07.352988+0000","last_active":"2024-04-25T10:07:10.117278+0000","last_peered":"2024-04-25T10:07:10.117278+0000","last_clean":"2024-04-25T10:07:10.117278+0000","last_became_active":"2024-04-25T10:07:07.352585+0000","last_became_peered":"2024-04-25T10:07:07.352585+0000","last_unstale":"2024-04-25T10:07:10.117278+0000","last_undegraded":"2024-04-25T10:07:10.117278+0000","last_fullsized":"2024-04-25T10:07:10.117278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:35.056084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.338565+0000","last_change":"2024-04-25T10:07:06.308389+0000","last_active":"2024-04-25T10:07:08.338565+0000","last_peered":"2024-04-25T10:07:08.338565+0000","last_clean":"2024-04-25T10:07:08.338565+0000","last_became_active":"2024-04-25T10:07:06.308046+0000","last_became_peered":"2024-04-25T10:07:06.308046+0000","last_unstale":"2024-04-25T10:07:08.338565+0000","last_undegraded":"2024-04-25T10:07:08.338565+0000","last_fullsized":"2024-04-25T10:07:08.338565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:53.885696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.118014+0000","last_change":"2024-04-25T10:07:07.357349+0000","last_active":"2024-04-25T10:07:10.118014+0000","last_peered":"2024-04-25T10:07:10.118014+0000","last_clean":"2024-04-25T10:07:10.118014+0000","last_became_active":"2024-04-25T10:07:07.356970+0000","last_became_peered":"2024-04-25T10:07:07.356970+0000","last_unstale":"2024-04-25T10:07:10.118014+0000","last_undegraded":"2024-04-25T10:07:10.118014+0000","last_fullsized":"2024-04-25T10:07:10.118014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:16.375835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340105+0000","last_change":"2024-04-25T10:07:06.310585+0000","last_active":"2024-04-25T10:07:08.340105+0000","last_peered":"2024-04-25T10:07:08.340105+0000","last_clean":"2024-04-25T10:07:08.340105+0000","last_became_active":"2024-04-25T10:07:06.310265+0000","last_became_peered":"2024-04-25T10:07:06.310265+0000","last_unstale":"2024-04-25T10:07:08.340105+0000","last_undegraded":"2024-04-25T10:07:08.340105+0000","last_fullsized":"2024-04-25T10:07:08.340105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:48.687788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341312+0000","last_change":"2024-04-25T10:07:07.337180+0000","last_active":"2024-04-25T10:07:08.341312+0000","last_peered":"2024-04-25T10:07:08.341312+0000","last_clean":"2024-04-25T10:07:08.341312+0000","last_became_active":"2024-04-25T10:07:07.335230+0000","last_became_peered":"2024-04-25T10:07:07.335230+0000","last_unstale":"2024-04-25T10:07:08.341312+0000","last_undegraded":"2024-04-25T10:07:08.341312+0000","last_fullsized":"2024-04-25T10:07:08.341312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:16.166171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337856+0000","last_change":"2024-04-25T10:07:06.311469+0000","last_active":"2024-04-25T10:07:08.337856+0000","last_peered":"2024-04-25T10:07:08.337856+0000","last_clean":"2024-04-25T10:07:08.337856+0000","last_became_active":"2024-04-25T10:07:06.311182+0000","last_became_peered":"2024-04-25T10:07:06.311182+0000","last_unstale":"2024-04-25T10:07:08.337856+0000","last_undegraded":"2024-04-25T10:07:08.337856+0000","last_fullsized":"2024-04-25T10:07:08.337856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:44.436918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117507+0000","last_change":"2024-04-25T10:07:07.350766+0000","last_active":"2024-04-25T10:07:10.117507+0000","last_peered":"2024-04-25T10:07:10.117507+0000","last_clean":"2024-04-25T10:07:10.117507+0000","last_became_active":"2024-04-25T10:07:07.350181+0000","last_became_peered":"2024-04-25T10:07:07.350181+0000","last_unstale":"2024-04-25T10:07:10.117507+0000","last_undegraded":"2024-04-25T10:07:10.117507+0000","last_fullsized":"2024-04-25T10:07:10.117507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:16.166171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116058+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:07:10.116058+0000","last_peered":"2024-04-25T10:07:10.116058+0000","last_clean":"2024-04-25T10:07:10.116058+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:07:10.116058+0000","last_undegraded":"2024-04-25T10:07:10.116058+0000","last_fullsized":"2024-04-25T10:07:10.116058+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340032+0000","last_change":"2024-04-25T10:07:06.306253+0000","last_active":"2024-04-25T10:07:08.340032+0000","last_peered":"2024-04-25T10:07:08.340032+0000","last_clean":"2024-04-25T10:07:08.340032+0000","last_became_active":"2024-04-25T10:07:06.305849+0000","last_became_peered":"2024-04-25T10:07:06.305849+0000","last_unstale":"2024-04-25T10:07:08.340032+0000","last_undegraded":"2024-04-25T10:07:08.340032+0000","last_fullsized":"2024-04-25T10:07:08.340032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:28.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430815+0000","last_change":"2024-04-25T10:07:07.337895+0000","last_active":"2024-04-25T10:07:08.430815+0000","last_peered":"2024-04-25T10:07:08.430815+0000","last_clean":"2024-04-25T10:07:08.430815+0000","last_became_active":"2024-04-25T10:07:07.337609+0000","last_became_peered":"2024-04-25T10:07:07.337609+0000","last_unstale":"2024-04-25T10:07:08.430815+0000","last_undegraded":"2024-04-25T10:07:08.430815+0000","last_fullsized":"2024-04-25T10:07:08.430815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115960+0000","last_change":"2024-04-25T10:07:06.325347+0000","last_active":"2024-04-25T10:07:10.115960+0000","last_peered":"2024-04-25T10:07:10.115960+0000","last_clean":"2024-04-25T10:07:10.115960+0000","last_became_active":"2024-04-25T10:07:06.324740+0000","last_became_peered":"2024-04-25T10:07:06.324740+0000","last_unstale":"2024-04-25T10:07:10.115960+0000","last_undegraded":"2024-04-25T10:07:10.115960+0000","last_fullsized":"2024-04-25T10:07:10.115960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.691331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.012839+0000","last_change":"2024-04-25T10:07:07.337867+0000","last_active":"2024-04-25T10:07:09.012839+0000","last_peered":"2024-04-25T10:07:09.012839+0000","last_clean":"2024-04-25T10:07:09.012839+0000","last_became_active":"2024-04-25T10:07:07.337521+0000","last_became_peered":"2024-04-25T10:07:07.337521+0000","last_unstale":"2024-04-25T10:07:09.012839+0000","last_undegraded":"2024-04-25T10:07:09.012839+0000","last_fullsized":"2024-04-25T10:07:09.012839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:05.611848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337610+0000","last_change":"2024-04-25T10:07:06.308483+0000","last_active":"2024-04-25T10:07:08.337610+0000","last_peered":"2024-04-25T10:07:08.337610+0000","last_clean":"2024-04-25T10:07:08.337610+0000","last_became_active":"2024-04-25T10:07:06.308156+0000","last_became_peered":"2024-04-25T10:07:06.308156+0000","last_unstale":"2024-04-25T10:07:08.337610+0000","last_undegraded":"2024-04-25T10:07:08.337610+0000","last_fullsized":"2024-04-25T10:07:08.337610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:25.307131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339704+0000","last_change":"2024-04-25T10:07:07.351559+0000","last_active":"2024-04-25T10:07:08.339704+0000","last_peered":"2024-04-25T10:07:08.339704+0000","last_clean":"2024-04-25T10:07:08.339704+0000","last_became_active":"2024-04-25T10:07:07.351271+0000","last_became_peered":"2024-04-25T10:07:07.351271+0000","last_unstale":"2024-04-25T10:07:08.339704+0000","last_undegraded":"2024-04-25T10:07:08.339704+0000","last_fullsized":"2024-04-25T10:07:08.339704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340604+0000","last_change":"2024-04-25T10:07:06.314916+0000","last_active":"2024-04-25T10:07:08.340604+0000","last_peered":"2024-04-25T10:07:08.340604+0000","last_clean":"2024-04-25T10:07:08.340604+0000","last_became_active":"2024-04-25T10:07:06.314572+0000","last_became_peered":"2024-04-25T10:07:06.314572+0000","last_unstale":"2024-04-25T10:07:08.340604+0000","last_undegraded":"2024-04-25T10:07:08.340604+0000","last_fullsized":"2024-04-25T10:07:08.340604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:53.885696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115644+0000","last_change":"2024-04-25T10:07:07.342482+0000","last_active":"2024-04-25T10:07:10.115644+0000","last_peered":"2024-04-25T10:07:10.115644+0000","last_clean":"2024-04-25T10:07:10.115644+0000","last_became_active":"2024-04-25T10:07:07.341566+0000","last_became_peered":"2024-04-25T10:07:07.341566+0000","last_unstale":"2024-04-25T10:07:10.115644+0000","last_undegraded":"2024-04-25T10:07:10.115644+0000","last_fullsized":"2024-04-25T10:07:10.115644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:14.880915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339445+0000","last_change":"2024-04-25T10:07:06.317437+0000","last_active":"2024-04-25T10:07:08.339445+0000","last_peered":"2024-04-25T10:07:08.339445+0000","last_clean":"2024-04-25T10:07:08.339445+0000","last_became_active":"2024-04-25T10:07:06.316494+0000","last_became_peered":"2024-04-25T10:07:06.316494+0000","last_unstale":"2024-04-25T10:07:08.339445+0000","last_undegraded":"2024-04-25T10:07:08.339445+0000","last_fullsized":"2024-04-25T10:07:08.339445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341452+0000","last_change":"2024-04-25T10:07:07.337051+0000","last_active":"2024-04-25T10:07:08.341452+0000","last_peered":"2024-04-25T10:07:08.341452+0000","last_clean":"2024-04-25T10:07:08.341452+0000","last_became_active":"2024-04-25T10:07:07.334884+0000","last_became_peered":"2024-04-25T10:07:07.334884+0000","last_unstale":"2024-04-25T10:07:08.341452+0000","last_undegraded":"2024-04-25T10:07:08.341452+0000","last_fullsized":"2024-04-25T10:07:08.341452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.763141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117681+0000","last_change":"2024-04-25T10:07:06.331016+0000","last_active":"2024-04-25T10:07:10.117681+0000","last_peered":"2024-04-25T10:07:10.117681+0000","last_clean":"2024-04-25T10:07:10.117681+0000","last_became_active":"2024-04-25T10:07:06.330712+0000","last_became_peered":"2024-04-25T10:07:06.330712+0000","last_unstale":"2024-04-25T10:07:10.117681+0000","last_undegraded":"2024-04-25T10:07:10.117681+0000","last_fullsized":"2024-04-25T10:07:10.117681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.867432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339661+0000","last_change":"2024-04-25T10:07:07.351856+0000","last_active":"2024-04-25T10:07:08.339661+0000","last_peered":"2024-04-25T10:07:08.339661+0000","last_clean":"2024-04-25T10:07:08.339661+0000","last_became_active":"2024-04-25T10:07:07.351585+0000","last_became_peered":"2024-04-25T10:07:07.351585+0000","last_unstale":"2024-04-25T10:07:08.339661+0000","last_undegraded":"2024-04-25T10:07:08.339661+0000","last_fullsized":"2024-04-25T10:07:08.339661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340221+0000","last_change":"2024-04-25T10:07:06.317585+0000","last_active":"2024-04-25T10:07:08.340221+0000","last_peered":"2024-04-25T10:07:08.340221+0000","last_clean":"2024-04-25T10:07:08.340221+0000","last_became_active":"2024-04-25T10:07:06.316964+0000","last_became_peered":"2024-04-25T10:07:06.316964+0000","last_unstale":"2024-04-25T10:07:08.340221+0000","last_undegraded":"2024-04-25T10:07:08.340221+0000","last_fullsized":"2024-04-25T10:07:08.340221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:58.603977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431612+0000","last_change":"2024-04-25T10:07:07.352730+0000","last_active":"2024-04-25T10:07:08.431612+0000","last_peered":"2024-04-25T10:07:08.431612+0000","last_clean":"2024-04-25T10:07:08.431612+0000","last_became_active":"2024-04-25T10:07:07.352486+0000","last_became_peered":"2024-04-25T10:07:07.352486+0000","last_unstale":"2024-04-25T10:07:08.431612+0000","last_undegraded":"2024-04-25T10:07:08.431612+0000","last_fullsized":"2024-04-25T10:07:08.431612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:23.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627361+0000","last_change":"2024-04-25T10:07:06.327061+0000","last_active":"2024-04-25T10:07:09.627361+0000","last_peered":"2024-04-25T10:07:09.627361+0000","last_clean":"2024-04-25T10:07:09.627361+0000","last_became_active":"2024-04-25T10:07:06.326227+0000","last_became_peered":"2024-04-25T10:07:06.326227+0000","last_unstale":"2024-04-25T10:07:09.627361+0000","last_undegraded":"2024-04-25T10:07:09.627361+0000","last_fullsized":"2024-04-25T10:07:09.627361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013232+0000","last_change":"2024-04-25T10:07:07.326642+0000","last_active":"2024-04-25T10:07:09.013232+0000","last_peered":"2024-04-25T10:07:09.013232+0000","last_clean":"2024-04-25T10:07:09.013232+0000","last_became_active":"2024-04-25T10:07:07.326261+0000","last_became_peered":"2024-04-25T10:07:07.326261+0000","last_unstale":"2024-04-25T10:07:09.013232+0000","last_undegraded":"2024-04-25T10:07:09.013232+0000","last_fullsized":"2024-04-25T10:07:09.013232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:01.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115657+0000","last_change":"2024-04-25T10:07:06.325264+0000","last_active":"2024-04-25T10:07:10.115657+0000","last_peered":"2024-04-25T10:07:10.115657+0000","last_clean":"2024-04-25T10:07:10.115657+0000","last_became_active":"2024-04-25T10:07:06.324704+0000","last_became_peered":"2024-04-25T10:07:06.324704+0000","last_unstale":"2024-04-25T10:07:10.115657+0000","last_undegraded":"2024-04-25T10:07:10.115657+0000","last_fullsized":"2024-04-25T10:07:10.115657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:52.458435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115184+0000","last_change":"2024-04-25T10:07:07.340986+0000","last_active":"2024-04-25T10:07:10.115184+0000","last_peered":"2024-04-25T10:07:10.115184+0000","last_clean":"2024-04-25T10:07:10.115184+0000","last_became_active":"2024-04-25T10:07:07.332637+0000","last_became_peered":"2024-04-25T10:07:07.332637+0000","last_unstale":"2024-04-25T10:07:10.115184+0000","last_undegraded":"2024-04-25T10:07:10.115184+0000","last_fullsized":"2024-04-25T10:07:10.115184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:17.677957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339302+0000","last_change":"2024-04-25T10:07:06.324149+0000","last_active":"2024-04-25T10:07:08.339302+0000","last_peered":"2024-04-25T10:07:08.339302+0000","last_clean":"2024-04-25T10:07:08.339302+0000","last_became_active":"2024-04-25T10:07:06.323900+0000","last_became_peered":"2024-04-25T10:07:06.323900+0000","last_unstale":"2024-04-25T10:07:08.339302+0000","last_undegraded":"2024-04-25T10:07:08.339302+0000","last_fullsized":"2024-04-25T10:07:08.339302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431004+0000","last_change":"2024-04-25T10:07:07.337614+0000","last_active":"2024-04-25T10:07:08.431004+0000","last_peered":"2024-04-25T10:07:08.431004+0000","last_clean":"2024-04-25T10:07:08.431004+0000","last_became_active":"2024-04-25T10:07:07.333283+0000","last_became_peered":"2024-04-25T10:07:07.333283+0000","last_unstale":"2024-04-25T10:07:08.431004+0000","last_undegraded":"2024-04-25T10:07:08.431004+0000","last_fullsized":"2024-04-25T10:07:08.431004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339925+0000","last_change":"2024-04-25T10:07:06.317292+0000","last_active":"2024-04-25T10:07:08.339925+0000","last_peered":"2024-04-25T10:07:08.339925+0000","last_clean":"2024-04-25T10:07:08.339925+0000","last_became_active":"2024-04-25T10:07:06.316862+0000","last_became_peered":"2024-04-25T10:07:06.316862+0000","last_unstale":"2024-04-25T10:07:08.339925+0000","last_undegraded":"2024-04-25T10:07:08.339925+0000","last_fullsized":"2024-04-25T10:07:08.339925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.691331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431273+0000","last_change":"2024-04-25T10:07:07.329541+0000","last_active":"2024-04-25T10:07:08.431273+0000","last_peered":"2024-04-25T10:07:08.431273+0000","last_clean":"2024-04-25T10:07:08.431273+0000","last_became_active":"2024-04-25T10:07:07.329296+0000","last_became_peered":"2024-04-25T10:07:07.329296+0000","last_unstale":"2024-04-25T10:07:08.431273+0000","last_undegraded":"2024-04-25T10:07:08.431273+0000","last_fullsized":"2024-04-25T10:07:08.431273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117039+0000","last_change":"2024-04-25T10:07:06.331241+0000","last_active":"2024-04-25T10:07:10.117039+0000","last_peered":"2024-04-25T10:07:10.117039+0000","last_clean":"2024-04-25T10:07:10.117039+0000","last_became_active":"2024-04-25T10:07:06.330637+0000","last_became_peered":"2024-04-25T10:07:06.330637+0000","last_unstale":"2024-04-25T10:07:10.117039+0000","last_undegraded":"2024-04-25T10:07:10.117039+0000","last_fullsized":"2024-04-25T10:07:10.117039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627101+0000","last_change":"2024-04-25T10:07:07.334097+0000","last_active":"2024-04-25T10:07:09.627101+0000","last_peered":"2024-04-25T10:07:09.627101+0000","last_clean":"2024-04-25T10:07:09.627101+0000","last_became_active":"2024-04-25T10:07:07.332086+0000","last_became_peered":"2024-04-25T10:07:07.332086+0000","last_unstale":"2024-04-25T10:07:09.627101+0000","last_undegraded":"2024-04-25T10:07:09.627101+0000","last_fullsized":"2024-04-25T10:07:09.627101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.370945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341238+0000","last_change":"2024-04-25T10:07:07.349145+0000","last_active":"2024-04-25T10:07:08.341238+0000","last_peered":"2024-04-25T10:07:08.341238+0000","last_clean":"2024-04-25T10:07:08.341238+0000","last_became_active":"2024-04-25T10:07:07.348916+0000","last_became_peered":"2024-04-25T10:07:07.348916+0000","last_unstale":"2024-04-25T10:07:08.341238+0000","last_undegraded":"2024-04-25T10:07:08.341238+0000","last_fullsized":"2024-04-25T10:07:08.341238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:35.056084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116709+0000","last_change":"2024-04-25T10:07:06.330571+0000","last_active":"2024-04-25T10:07:10.116709+0000","last_peered":"2024-04-25T10:07:10.116709+0000","last_clean":"2024-04-25T10:07:10.116709+0000","last_became_active":"2024-04-25T10:07:06.329886+0000","last_became_peered":"2024-04-25T10:07:06.329886+0000","last_unstale":"2024-04-25T10:07:10.116709+0000","last_undegraded":"2024-04-25T10:07:10.116709+0000","last_fullsized":"2024-04-25T10:07:10.116709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340935+0000","last_change":"2024-04-25T10:07:07.347234+0000","last_active":"2024-04-25T10:07:08.340935+0000","last_peered":"2024-04-25T10:07:08.340935+0000","last_clean":"2024-04-25T10:07:08.340935+0000","last_became_active":"2024-04-25T10:07:07.346961+0000","last_became_peered":"2024-04-25T10:07:07.346961+0000","last_unstale":"2024-04-25T10:07:08.340935+0000","last_undegraded":"2024-04-25T10:07:08.340935+0000","last_fullsized":"2024-04-25T10:07:08.340935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:23.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117348+0000","last_change":"2024-04-25T10:07:06.330806+0000","last_active":"2024-04-25T10:07:10.117348+0000","last_peered":"2024-04-25T10:07:10.117348+0000","last_clean":"2024-04-25T10:07:10.117348+0000","last_became_active":"2024-04-25T10:07:06.330360+0000","last_became_peered":"2024-04-25T10:07:06.330360+0000","last_unstale":"2024-04-25T10:07:10.117348+0000","last_undegraded":"2024-04-25T10:07:10.117348+0000","last_fullsized":"2024-04-25T10:07:10.117348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:48.687788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013341+0000","last_change":"2024-04-25T10:07:07.337072+0000","last_active":"2024-04-25T10:07:09.013341+0000","last_peered":"2024-04-25T10:07:09.013341+0000","last_clean":"2024-04-25T10:07:09.013341+0000","last_became_active":"2024-04-25T10:07:07.336767+0000","last_became_peered":"2024-04-25T10:07:07.336767+0000","last_unstale":"2024-04-25T10:07:09.013341+0000","last_undegraded":"2024-04-25T10:07:09.013341+0000","last_fullsized":"2024-04-25T10:07:09.013341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.370945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431572+0000","last_change":"2024-04-25T10:07:06.322321+0000","last_active":"2024-04-25T10:07:08.431572+0000","last_peered":"2024-04-25T10:07:08.431572+0000","last_clean":"2024-04-25T10:07:08.431572+0000","last_became_active":"2024-04-25T10:07:06.321990+0000","last_became_peered":"2024-04-25T10:07:06.321990+0000","last_unstale":"2024-04-25T10:07:08.431572+0000","last_undegraded":"2024-04-25T10:07:08.431572+0000","last_fullsized":"2024-04-25T10:07:08.431572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:25.307131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431571+0000","last_change":"2024-04-25T10:07:07.337246+0000","last_active":"2024-04-25T10:07:08.431571+0000","last_peered":"2024-04-25T10:07:08.431571+0000","last_clean":"2024-04-25T10:07:08.431571+0000","last_became_active":"2024-04-25T10:07:07.336894+0000","last_became_peered":"2024-04-25T10:07:07.336894+0000","last_unstale":"2024-04-25T10:07:08.431571+0000","last_undegraded":"2024-04-25T10:07:08.431571+0000","last_fullsized":"2024-04-25T10:07:08.431571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013544+0000","last_change":"2024-04-25T10:07:06.324947+0000","last_active":"2024-04-25T10:07:09.013544+0000","last_peered":"2024-04-25T10:07:09.013544+0000","last_clean":"2024-04-25T10:07:09.013544+0000","last_became_active":"2024-04-25T10:07:06.324464+0000","last_became_peered":"2024-04-25T10:07:06.324464+0000","last_unstale":"2024-04-25T10:07:09.013544+0000","last_undegraded":"2024-04-25T10:07:09.013544+0000","last_fullsized":"2024-04-25T10:07:09.013544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:25.307131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":64760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:12.041 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:12.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.276+0000 7f6508a00640 1 Processor -- start 2024-04-25T10:07:12.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.276+0000 7f6508a00640 1 -- start start 2024-04-25T10:07:12.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.278+0000 7f6508a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f650415dce0 0x7f65040d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.278+0000 7f6508a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f65040b5b60 0x7f65040b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.278+0000 7f6508a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650415e0c0 con 0x7f65040d35b0 2024-04-25T10:07:12.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.278+0000 7f6508a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65040d2510 con 0x7f65040b5b60 2024-04-25T10:07:12.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.278+0000 7f6508a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65040d27f0 con 0x7f650415dce0 2024-04-25T10:07:12.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.278+0000 7f6502800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f65040b5b60 0x7f65040b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.278+0000 7f6502800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f65040b5b60 0x7f65040b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34432/0 (socket says 172.21.15.119:34432) 2024-04-25T10:07:12.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.278+0000 7f6502800640 1 -- 172.21.15.119:0/2949157744 learned_addr learned my addr 172.21.15.119:0/2949157744 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:12.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.279+0000 7f6501e00640 1 --2- 172.21.15.119:0/2949157744 >> v2:172.21.15.132:3300/0 conn(0x7f650415dce0 0x7f65040d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.279+0000 7f6502800640 1 -- 172.21.15.119:0/2949157744 >> v2:172.21.15.132:3300/0 conn(0x7f650415dce0 msgr2=0x7f65040d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.279+0000 7f6502800640 1 --2- 172.21.15.119:0/2949157744 >> v2:172.21.15.132:3300/0 conn(0x7f650415dce0 0x7f65040d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.279+0000 7f6502800640 1 -- 172.21.15.119:0/2949157744 >> v1:172.21.15.132:6789/0 conn(0x7f65040d35b0 legacy=0x7f65040b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.279+0000 7f6502800640 1 -- 172.21.15.119:0/2949157744 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65040d2ad0 con 0x7f65040b5b60 2024-04-25T10:07:12.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.280+0000 7f6502800640 1 --2- 172.21.15.119:0/2949157744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65040b5b60 0x7f65040b6b00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f65040ce220 tx=0x7f64fc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185edfd897f8d8bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.280+0000 7f6501400640 1 -- 172.21.15.119:0/2949157744 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc036070 con 0x7f65040b5b60 2024-04-25T10:07:12.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.281+0000 7f6501400640 1 -- 172.21.15.119:0/2949157744 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc031600 con 0x7f65040b5b60 2024-04-25T10:07:12.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.281+0000 7f6501400640 1 -- 172.21.15.119:0/2949157744 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc0319d0 con 0x7f65040b5b60 2024-04-25T10:07:12.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.281+0000 7f6508a00640 1 -- 172.21.15.119:0/2949157744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65040b5b60 msgr2=0x7f65040b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.281+0000 7f6508a00640 1 --2- 172.21.15.119:0/2949157744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65040b5b60 0x7f65040b6b00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f65040ce220 tx=0x7f64fc02a730 comp rx=0 tx=0).stop 2024-04-25T10:07:12.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.281+0000 7f6508a00640 1 -- 172.21.15.119:0/2949157744 shutdown_connections 2024-04-25T10:07:12.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.281+0000 7f6508a00640 1 --2- 172.21.15.119:0/2949157744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65040b5b60 0x7f65040b6b00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.281+0000 7f6508a00640 1 --2- 172.21.15.119:0/2949157744 >> v2:172.21.15.132:3300/0 conn(0x7f650415dce0 0x7f65040d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.281+0000 7f6508a00640 1 -- 172.21.15.119:0/2949157744 >> 172.21.15.119:0/2949157744 conn(0x7f6504072900 msgr2=0x7f65040d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.281+0000 7f6508a00640 1 -- 172.21.15.119:0/2949157744 shutdown_connections 2024-04-25T10:07:12.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.281+0000 7f6508a00640 1 -- 172.21.15.119:0/2949157744 wait complete. 2024-04-25T10:07:12.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.283+0000 7f6508a00640 1 Processor -- start 2024-04-25T10:07:12.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.283+0000 7f6508a00640 1 -- start start 2024-04-25T10:07:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.284+0000 7f6508a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65040b5b60 0x7f65040c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.284+0000 7f6508a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65040d35b0 0x7f650415bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.284+0000 7f6508a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f650415dce0 0x7f65041ea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.284+0000 7f6508a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040b9820 con 0x7f650415dce0 2024-04-25T10:07:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.284+0000 7f6508a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040d3090 con 0x7f65040b5b60 2024-04-25T10:07:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.284+0000 7f6508a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65040b99a0 con 0x7f65040d35b0 2024-04-25T10:07:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.285+0000 7f6502800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f650415dce0 0x7f65041ea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.285+0000 7f6502800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f650415dce0 0x7f65041ea7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34446/0 (socket says 172.21.15.119:34446) 2024-04-25T10:07:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.285+0000 7f6502800640 1 -- 172.21.15.119:0/2931924350 learned_addr learned my addr 172.21.15.119:0/2931924350 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.285+0000 7f6501e00640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65040b5b60 0x7f65040c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.285+0000 7f64f9400640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65040d35b0 0x7f650415bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.285+0000 7f6502800640 1 -- 172.21.15.119:0/2931924350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65040d35b0 msgr2=0x7f650415bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.285+0000 7f6502800640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65040d35b0 0x7f650415bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.285+0000 7f6502800640 1 -- 172.21.15.119:0/2931924350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65040b5b60 msgr2=0x7f65040c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.285+0000 7f6502800640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65040b5b60 0x7f65040c8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.285+0000 7f6502800640 1 -- 172.21.15.119:0/2931924350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650415f130 con 0x7f650415dce0 2024-04-25T10:07:12.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.285+0000 7f64f9400640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65040d35b0 0x7f650415bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:12.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.285+0000 7f6502800640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f650415dce0 0x7f65041ea7d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009f40 tx=0x7f64fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.286+0000 7f64faa00640 1 -- 172.21.15.119:0/2931924350 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc036050 con 0x7f650415dce0 2024-04-25T10:07:12.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.286+0000 7f64faa00640 1 -- 172.21.15.119:0/2931924350 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc042070 con 0x7f650415dce0 2024-04-25T10:07:12.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.286+0000 7f64faa00640 1 -- 172.21.15.119:0/2931924350 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc009510 con 0x7f650415dce0 2024-04-25T10:07:12.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.286+0000 7f6508a00640 1 -- 172.21.15.119:0/2931924350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650415f410 con 0x7f650415dce0 2024-04-25T10:07:12.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.286+0000 7f6508a00640 1 -- 172.21.15.119:0/2931924350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650415f9d0 con 0x7f650415dce0 2024-04-25T10:07:12.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.287+0000 7f6508a00640 1 -- 172.21.15.119:0/2931924350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65040749b0 con 0x7f650415dce0 2024-04-25T10:07:12.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.288+0000 7f64faa00640 1 -- 172.21.15.119:0/2931924350 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f64fc007bf0 con 0x7f650415dce0 2024-04-25T10:07:12.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.289+0000 7f64faa00640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f64e006b4d0 0x7f64e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.289+0000 7f64faa00640 1 -- 172.21.15.119:0/2931924350 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64fc0ab130 con 0x7f650415dce0 2024-04-25T10:07:12.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.289+0000 7f6501e00640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f64e006b4d0 0x7f64e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.290+0000 7f6501e00640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f64e006b4d0 0x7f64e006d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f64f4006fd0 tx=0x7f64f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.293+0000 7f64faa00640 1 -- 172.21.15.119:0/2931924350 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f64fc079e80 con 0x7f650415dce0 2024-04-25T10:07:12.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.483+0000 7f6508a00640 1 -- 172.21.15.119:0/2931924350 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6504067f30 con 0x7f64e006b4d0 2024-04-25T10:07:12.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.491+0000 7f64faa00640 1 -- 172.21.15.119:0/2931924350 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251947 (secure 0 0 0) 0x7f6504067f30 con 0x7f64e006b4d0 2024-04-25T10:07:12.501 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:12.502 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:07:12.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.497+0000 7f6508a00640 1 -- 172.21.15.119:0/2931924350 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f64e006b4d0 msgr2=0x7f64e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.497+0000 7f6508a00640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f64e006b4d0 0x7f64e006d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f64f4006fd0 tx=0x7f64f4008040 comp rx=0 tx=0).stop 2024-04-25T10:07:12.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.497+0000 7f6508a00640 1 -- 172.21.15.119:0/2931924350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f650415dce0 msgr2=0x7f65041ea7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.497+0000 7f6508a00640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f650415dce0 0x7f65041ea7d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009f40 tx=0x7f64fc004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:12.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.498+0000 7f6508a00640 1 -- 172.21.15.119:0/2931924350 shutdown_connections 2024-04-25T10:07:12.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.498+0000 7f6508a00640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f64e006b4d0 0x7f64e006d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.498+0000 7f6508a00640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f650415dce0 0x7f65041ea7d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.498+0000 7f6508a00640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65040d35b0 0x7f650415bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.498+0000 7f6508a00640 1 --2- 172.21.15.119:0/2931924350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65040b5b60 0x7f65040c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.498+0000 7f6508a00640 1 -- 172.21.15.119:0/2931924350 >> 172.21.15.119:0/2931924350 conn(0x7f6504072900 msgr2=0x7f65040ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.498+0000 7f6508a00640 1 -- 172.21.15.119:0/2931924350 shutdown_connections 2024-04-25T10:07:12.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.498+0000 7f6508a00640 1 -- 172.21.15.119:0/2931924350 wait complete. 2024-04-25T10:07:12.522 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T10:07:11.924137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2812,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2879488,"data_stored":1746728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001866"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117497+0000","last_change":"2024-04-25T10:07:07.357761+0000","last_active":"2024-04-25T10:07:10.117497+0000","last_peered":"2024-04-25T10:07:10.117497+0000","last_clean":"2024-04-25T10:07:10.117497+0000","last_became_active":"2024-04-25T10:07:07.357419+0000","last_became_peered":"2024-04-25T10:07:07.357419+0000","last_unstale":"2024-04-25T10:07:10.117497+0000","last_undegraded":"2024-04-25T10:07:10.117497+0000","last_fullsized":"2024-04-25T10:07:10.117497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:31.725733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340438+0000","last_change":"2024-04-25T10:07:07.324906+0000","last_active":"2024-04-25T10:07:08.340438+0000","last_peered":"2024-04-25T10:07:08.340438+0000","last_clean":"2024-04-25T10:07:08.340438+0000","last_became_active":"2024-04-25T10:07:07.324635+0000","last_became_peered":"2024-04-25T10:07:07.324635+0000","last_unstale":"2024-04-25T10:07:08.340438+0000","last_undegraded":"2024-04-25T10:07:08.340438+0000","last_fullsized":"2024-04-25T10:07:08.340438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013703+0000","last_change":"2024-04-25T10:07:07.324537+0000","last_active":"2024-04-25T10:07:09.013703+0000","last_peered":"2024-04-25T10:07:09.013703+0000","last_clean":"2024-04-25T10:07:09.013703+0000","last_became_active":"2024-04-25T10:07:07.324290+0000","last_became_peered":"2024-04-25T10:07:07.324290+0000","last_unstale":"2024-04-25T10:07:09.013703+0000","last_undegraded":"2024-04-25T10:07:09.013703+0000","last_fullsized":"2024-04-25T10:07:09.013703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340924+0000","last_change":"2024-04-25T10:07:07.336252+0000","last_active":"2024-04-25T10:07:08.340924+0000","last_peered":"2024-04-25T10:07:08.340924+0000","last_clean":"2024-04-25T10:07:08.340924+0000","last_became_active":"2024-04-25T10:07:07.335889+0000","last_became_peered":"2024-04-25T10:07:07.335889+0000","last_unstale":"2024-04-25T10:07:08.340924+0000","last_undegraded":"2024-04-25T10:07:08.340924+0000","last_fullsized":"2024-04-25T10:07:08.340924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116666+0000","last_change":"2024-04-25T10:07:07.354080+0000","last_active":"2024-04-25T10:07:10.116666+0000","last_peered":"2024-04-25T10:07:10.116666+0000","last_clean":"2024-04-25T10:07:10.116666+0000","last_became_active":"2024-04-25T10:07:07.351587+0000","last_became_peered":"2024-04-25T10:07:07.351587+0000","last_unstale":"2024-04-25T10:07:10.116666+0000","last_undegraded":"2024-04-25T10:07:10.116666+0000","last_fullsized":"2024-04-25T10:07:10.116666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340152+0000","last_change":"2024-04-25T10:07:07.329505+0000","last_active":"2024-04-25T10:07:08.340152+0000","last_peered":"2024-04-25T10:07:08.340152+0000","last_clean":"2024-04-25T10:07:08.340152+0000","last_became_active":"2024-04-25T10:07:07.327756+0000","last_became_peered":"2024-04-25T10:07:07.327756+0000","last_unstale":"2024-04-25T10:07:08.340152+0000","last_undegraded":"2024-04-25T10:07:08.340152+0000","last_fullsized":"2024-04-25T10:07:08.340152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116453+0000","last_change":"2024-04-25T10:07:07.354368+0000","last_active":"2024-04-25T10:07:10.116453+0000","last_peered":"2024-04-25T10:07:10.116453+0000","last_clean":"2024-04-25T10:07:10.116453+0000","last_became_active":"2024-04-25T10:07:07.354048+0000","last_became_peered":"2024-04-25T10:07:07.354048+0000","last_unstale":"2024-04-25T10:07:10.116453+0000","last_undegraded":"2024-04-25T10:07:10.116453+0000","last_fullsized":"2024-04-25T10:07:10.116453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430715+0000","last_change":"2024-04-25T10:07:07.337913+0000","last_active":"2024-04-25T10:07:08.430715+0000","last_peered":"2024-04-25T10:07:08.430715+0000","last_clean":"2024-04-25T10:07:08.430715+0000","last_became_active":"2024-04-25T10:07:07.337268+0000","last_became_peered":"2024-04-25T10:07:07.337268+0000","last_unstale":"2024-04-25T10:07:08.430715+0000","last_undegraded":"2024-04-25T10:07:08.430715+0000","last_fullsized":"2024-04-25T10:07:08.430715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:01.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117721+0000","last_change":"2024-04-25T10:07:07.352856+0000","last_active":"2024-04-25T10:07:10.117721+0000","last_peered":"2024-04-25T10:07:10.117721+0000","last_clean":"2024-04-25T10:07:10.117721+0000","last_became_active":"2024-04-25T10:07:07.352555+0000","last_became_peered":"2024-04-25T10:07:07.352555+0000","last_unstale":"2024-04-25T10:07:10.117721+0000","last_undegraded":"2024-04-25T10:07:10.117721+0000","last_fullsized":"2024-04-25T10:07:10.117721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.763141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627954+0000","last_change":"2024-04-25T10:07:07.335997+0000","last_active":"2024-04-25T10:07:09.627954+0000","last_peered":"2024-04-25T10:07:09.627954+0000","last_clean":"2024-04-25T10:07:09.627954+0000","last_became_active":"2024-04-25T10:07:07.334634+0000","last_became_peered":"2024-04-25T10:07:07.334634+0000","last_unstale":"2024-04-25T10:07:09.627954+0000","last_undegraded":"2024-04-25T10:07:09.627954+0000","last_fullsized":"2024-04-25T10:07:09.627954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627400+0000","last_change":"2024-04-25T10:07:07.336470+0000","last_active":"2024-04-25T10:07:09.627400+0000","last_peered":"2024-04-25T10:07:09.627400+0000","last_clean":"2024-04-25T10:07:09.627400+0000","last_became_active":"2024-04-25T10:07:07.334875+0000","last_became_peered":"2024-04-25T10:07:07.334875+0000","last_unstale":"2024-04-25T10:07:09.627400+0000","last_undegraded":"2024-04-25T10:07:09.627400+0000","last_fullsized":"2024-04-25T10:07:09.627400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115690+0000","last_change":"2024-04-25T10:07:07.342067+0000","last_active":"2024-04-25T10:07:10.115690+0000","last_peered":"2024-04-25T10:07:10.115690+0000","last_clean":"2024-04-25T10:07:10.115690+0000","last_became_active":"2024-04-25T10:07:07.341655+0000","last_became_peered":"2024-04-25T10:07:07.341655+0000","last_unstale":"2024-04-25T10:07:10.115690+0000","last_undegraded":"2024-04-25T10:07:10.115690+0000","last_fullsized":"2024-04-25T10:07:10.115690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:37.664841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431262+0000","last_change":"2024-04-25T10:07:07.337652+0000","last_active":"2024-04-25T10:07:08.431262+0000","last_peered":"2024-04-25T10:07:08.431262+0000","last_clean":"2024-04-25T10:07:08.431262+0000","last_became_active":"2024-04-25T10:07:07.337057+0000","last_became_peered":"2024-04-25T10:07:07.337057+0000","last_unstale":"2024-04-25T10:07:08.431262+0000","last_undegraded":"2024-04-25T10:07:08.431262+0000","last_fullsized":"2024-04-25T10:07:08.431262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.012844+0000","last_change":"2024-04-25T10:07:07.341549+0000","last_active":"2024-04-25T10:07:09.012844+0000","last_peered":"2024-04-25T10:07:09.012844+0000","last_clean":"2024-04-25T10:07:09.012844+0000","last_became_active":"2024-04-25T10:07:07.341228+0000","last_became_peered":"2024-04-25T10:07:07.341228+0000","last_unstale":"2024-04-25T10:07:09.012844+0000","last_undegraded":"2024-04-25T10:07:09.012844+0000","last_fullsized":"2024-04-25T10:07:09.012844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:50.697002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339480+0000","last_change":"2024-04-25T10:07:07.348054+0000","last_active":"2024-04-25T10:07:08.339480+0000","last_peered":"2024-04-25T10:07:08.339480+0000","last_clean":"2024-04-25T10:07:08.339480+0000","last_became_active":"2024-04-25T10:07:07.347739+0000","last_became_peered":"2024-04-25T10:07:07.347739+0000","last_unstale":"2024-04-25T10:07:08.339480+0000","last_undegraded":"2024-04-25T10:07:08.339480+0000","last_fullsized":"2024-04-25T10:07:08.339480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339967+0000","last_change":"2024-04-25T10:07:07.326858+0000","last_active":"2024-04-25T10:07:08.339967+0000","last_peered":"2024-04-25T10:07:08.339967+0000","last_clean":"2024-04-25T10:07:08.339967+0000","last_became_active":"2024-04-25T10:07:07.326107+0000","last_became_peered":"2024-04-25T10:07:07.326107+0000","last_unstale":"2024-04-25T10:07:08.339967+0000","last_undegraded":"2024-04-25T10:07:08.339967+0000","last_fullsized":"2024-04-25T10:07:08.339967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.342255+0000","last_change":"2024-04-25T10:07:07.348628+0000","last_active":"2024-04-25T10:07:08.342255+0000","last_peered":"2024-04-25T10:07:08.342255+0000","last_clean":"2024-04-25T10:07:08.342255+0000","last_became_active":"2024-04-25T10:07:07.348356+0000","last_became_peered":"2024-04-25T10:07:07.348356+0000","last_unstale":"2024-04-25T10:07:08.342255+0000","last_undegraded":"2024-04-25T10:07:08.342255+0000","last_fullsized":"2024-04-25T10:07:08.342255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:16.375835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.338486+0000","last_change":"2024-04-25T10:07:07.341185+0000","last_active":"2024-04-25T10:07:08.338486+0000","last_peered":"2024-04-25T10:07:08.338486+0000","last_clean":"2024-04-25T10:07:08.338486+0000","last_became_active":"2024-04-25T10:07:07.340797+0000","last_became_peered":"2024-04-25T10:07:07.340797+0000","last_unstale":"2024-04-25T10:07:08.338486+0000","last_undegraded":"2024-04-25T10:07:08.338486+0000","last_fullsized":"2024-04-25T10:07:08.338486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:49.701271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.118018+0000","last_change":"2024-04-25T10:07:07.356663+0000","last_active":"2024-04-25T10:07:10.118018+0000","last_peered":"2024-04-25T10:07:10.118018+0000","last_clean":"2024-04-25T10:07:10.118018+0000","last_became_active":"2024-04-25T10:07:07.356381+0000","last_became_peered":"2024-04-25T10:07:07.356381+0000","last_unstale":"2024-04-25T10:07:10.118018+0000","last_undegraded":"2024-04-25T10:07:10.118018+0000","last_fullsized":"2024-04-25T10:07:10.118018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:51.771853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627639+0000","last_change":"2024-04-25T10:07:07.348329+0000","last_active":"2024-04-25T10:07:09.627639+0000","last_peered":"2024-04-25T10:07:09.627639+0000","last_clean":"2024-04-25T10:07:09.627639+0000","last_became_active":"2024-04-25T10:07:07.347033+0000","last_became_peered":"2024-04-25T10:07:07.347033+0000","last_unstale":"2024-04-25T10:07:09.627639+0000","last_undegraded":"2024-04-25T10:07:09.627639+0000","last_fullsized":"2024-04-25T10:07:09.627639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013394+0000","last_change":"2024-04-25T10:07:07.350923+0000","last_active":"2024-04-25T10:07:09.013394+0000","last_peered":"2024-04-25T10:07:09.013394+0000","last_clean":"2024-04-25T10:07:09.013394+0000","last_became_active":"2024-04-25T10:07:07.350102+0000","last_became_peered":"2024-04-25T10:07:07.350102+0000","last_unstale":"2024-04-25T10:07:09.013394+0000","last_undegraded":"2024-04-25T10:07:09.013394+0000","last_fullsized":"2024-04-25T10:07:09.013394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340239+0000","last_change":"2024-04-25T10:07:07.351751+0000","last_active":"2024-04-25T10:07:08.340239+0000","last_peered":"2024-04-25T10:07:08.340239+0000","last_clean":"2024-04-25T10:07:08.340239+0000","last_became_active":"2024-04-25T10:07:07.349688+0000","last_became_peered":"2024-04-25T10:07:07.349688+0000","last_unstale":"2024-04-25T10:07:08.340239+0000","last_undegraded":"2024-04-25T10:07:08.340239+0000","last_fullsized":"2024-04-25T10:07:08.340239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430799+0000","last_change":"2024-04-25T10:07:07.346636+0000","last_active":"2024-04-25T10:07:08.430799+0000","last_peered":"2024-04-25T10:07:08.430799+0000","last_clean":"2024-04-25T10:07:08.430799+0000","last_became_active":"2024-04-25T10:07:07.346448+0000","last_became_peered":"2024-04-25T10:07:07.346448+0000","last_unstale":"2024-04-25T10:07:08.430799+0000","last_undegraded":"2024-04-25T10:07:08.430799+0000","last_fullsized":"2024-04-25T10:07:08.430799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.370945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339964+0000","last_change":"2024-04-25T10:07:07.327120+0000","last_active":"2024-04-25T10:07:08.339964+0000","last_peered":"2024-04-25T10:07:08.339964+0000","last_clean":"2024-04-25T10:07:08.339964+0000","last_became_active":"2024-04-25T10:07:07.326471+0000","last_became_peered":"2024-04-25T10:07:07.326471+0000","last_unstale":"2024-04-25T10:07:08.339964+0000","last_undegraded":"2024-04-25T10:07:08.339964+0000","last_fullsized":"2024-04-25T10:07:08.339964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431521+0000","last_change":"2024-04-25T10:07:07.346515+0000","last_active":"2024-04-25T10:07:08.431521+0000","last_peered":"2024-04-25T10:07:08.431521+0000","last_clean":"2024-04-25T10:07:08.431521+0000","last_became_active":"2024-04-25T10:07:07.346218+0000","last_became_peered":"2024-04-25T10:07:07.346218+0000","last_unstale":"2024-04-25T10:07:08.431521+0000","last_undegraded":"2024-04-25T10:07:08.431521+0000","last_fullsized":"2024-04-25T10:07:08.431521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:49.296728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627914+0000","last_change":"2024-04-25T10:07:07.352410+0000","last_active":"2024-04-25T10:07:09.627914+0000","last_peered":"2024-04-25T10:07:09.627914+0000","last_clean":"2024-04-25T10:07:09.627914+0000","last_became_active":"2024-04-25T10:07:07.352106+0000","last_became_peered":"2024-04-25T10:07:07.352106+0000","last_unstale":"2024-04-25T10:07:09.627914+0000","last_undegraded":"2024-04-25T10:07:09.627914+0000","last_fullsized":"2024-04-25T10:07:09.627914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339814+0000","last_change":"2024-04-25T10:07:07.339497+0000","last_active":"2024-04-25T10:07:08.339814+0000","last_peered":"2024-04-25T10:07:08.339814+0000","last_clean":"2024-04-25T10:07:08.339814+0000","last_became_active":"2024-04-25T10:07:07.339168+0000","last_became_peered":"2024-04-25T10:07:07.339168+0000","last_unstale":"2024-04-25T10:07:08.339814+0000","last_undegraded":"2024-04-25T10:07:08.339814+0000","last_fullsized":"2024-04-25T10:07:08.339814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431449+0000","last_change":"2024-04-25T10:07:07.349878+0000","last_active":"2024-04-25T10:07:08.431449+0000","last_peered":"2024-04-25T10:07:08.431449+0000","last_clean":"2024-04-25T10:07:08.431449+0000","last_became_active":"2024-04-25T10:07:07.349602+0000","last_became_peered":"2024-04-25T10:07:07.349602+0000","last_unstale":"2024-04-25T10:07:08.431449+0000","last_undegraded":"2024-04-25T10:07:08.431449+0000","last_fullsized":"2024-04-25T10:07:08.431449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013693+0000","last_change":"2024-04-25T10:07:07.331427+0000","last_active":"2024-04-25T10:07:09.013693+0000","last_peered":"2024-04-25T10:07:09.013693+0000","last_clean":"2024-04-25T10:07:09.013693+0000","last_became_active":"2024-04-25T10:07:07.329911+0000","last_became_peered":"2024-04-25T10:07:07.329911+0000","last_unstale":"2024-04-25T10:07:09.013693+0000","last_undegraded":"2024-04-25T10:07:09.013693+0000","last_fullsized":"2024-04-25T10:07:09.013693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340563+0000","last_change":"2024-04-25T10:07:07.329226+0000","last_active":"2024-04-25T10:07:08.340563+0000","last_peered":"2024-04-25T10:07:08.340563+0000","last_clean":"2024-04-25T10:07:08.340563+0000","last_became_active":"2024-04-25T10:07:07.327329+0000","last_became_peered":"2024-04-25T10:07:07.327329+0000","last_unstale":"2024-04-25T10:07:08.340563+0000","last_undegraded":"2024-04-25T10:07:08.340563+0000","last_fullsized":"2024-04-25T10:07:08.340563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:49.296728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337552+0000","last_change":"2024-04-25T10:07:07.327804+0000","last_active":"2024-04-25T10:07:08.337552+0000","last_peered":"2024-04-25T10:07:08.337552+0000","last_clean":"2024-04-25T10:07:08.337552+0000","last_became_active":"2024-04-25T10:07:07.327516+0000","last_became_peered":"2024-04-25T10:07:07.327516+0000","last_unstale":"2024-04-25T10:07:08.337552+0000","last_undegraded":"2024-04-25T10:07:08.337552+0000","last_fullsized":"2024-04-25T10:07:08.337552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:46.704814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627063+0000","last_change":"2024-04-25T10:07:07.333594+0000","last_active":"2024-04-25T10:07:09.627063+0000","last_peered":"2024-04-25T10:07:09.627063+0000","last_clean":"2024-04-25T10:07:09.627063+0000","last_became_active":"2024-04-25T10:07:07.333302+0000","last_became_peered":"2024-04-25T10:07:07.333302+0000","last_unstale":"2024-04-25T10:07:09.627063+0000","last_undegraded":"2024-04-25T10:07:09.627063+0000","last_fullsized":"2024-04-25T10:07:09.627063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:01.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.114966+0000","last_change":"2024-04-25T10:07:07.333101+0000","last_active":"2024-04-25T10:07:10.114966+0000","last_peered":"2024-04-25T10:07:10.114966+0000","last_clean":"2024-04-25T10:07:10.114966+0000","last_became_active":"2024-04-25T10:07:07.332714+0000","last_became_peered":"2024-04-25T10:07:07.332714+0000","last_unstale":"2024-04-25T10:07:10.114966+0000","last_undegraded":"2024-04-25T10:07:10.114966+0000","last_fullsized":"2024-04-25T10:07:10.114966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:53.471918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431440+0000","last_change":"2024-04-25T10:07:06.322470+0000","last_active":"2024-04-25T10:07:08.431440+0000","last_peered":"2024-04-25T10:07:08.431440+0000","last_clean":"2024-04-25T10:07:08.431440+0000","last_became_active":"2024-04-25T10:07:06.321092+0000","last_became_peered":"2024-04-25T10:07:06.321092+0000","last_unstale":"2024-04-25T10:07:08.431440+0000","last_undegraded":"2024-04-25T10:07:08.431440+0000","last_fullsized":"2024-04-25T10:07:08.431440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:28.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013819+0000","last_change":"2024-04-25T10:07:07.340326+0000","last_active":"2024-04-25T10:07:09.013819+0000","last_peered":"2024-04-25T10:07:09.013819+0000","last_clean":"2024-04-25T10:07:09.013819+0000","last_became_active":"2024-04-25T10:07:07.339889+0000","last_became_peered":"2024-04-25T10:07:07.339889+0000","last_unstale":"2024-04-25T10:07:09.013819+0000","last_undegraded":"2024-04-25T10:07:09.013819+0000","last_fullsized":"2024-04-25T10:07:09.013819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.626769+0000","last_change":"2024-04-25T10:07:06.333386+0000","last_active":"2024-04-25T10:07:09.626769+0000","last_peered":"2024-04-25T10:07:09.626769+0000","last_clean":"2024-04-25T10:07:09.626769+0000","last_became_active":"2024-04-25T10:07:06.333014+0000","last_became_peered":"2024-04-25T10:07:06.333014+0000","last_unstale":"2024-04-25T10:07:09.626769+0000","last_undegraded":"2024-04-25T10:07:09.626769+0000","last_fullsized":"2024-04-25T10:07:09.626769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:53.885696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430668+0000","last_change":"2024-04-25T10:07:06.314017+0000","last_active":"2024-04-25T10:07:08.430668+0000","last_peered":"2024-04-25T10:07:08.430668+0000","last_clean":"2024-04-25T10:07:08.430668+0000","last_became_active":"2024-04-25T10:07:06.313712+0000","last_became_peered":"2024-04-25T10:07:06.313712+0000","last_unstale":"2024-04-25T10:07:08.430668+0000","last_undegraded":"2024-04-25T10:07:08.430668+0000","last_fullsized":"2024-04-25T10:07:08.430668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:05.470696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430845+0000","last_change":"2024-04-25T10:07:07.323982+0000","last_active":"2024-04-25T10:07:08.430845+0000","last_peered":"2024-04-25T10:07:08.430845+0000","last_clean":"2024-04-25T10:07:08.430845+0000","last_became_active":"2024-04-25T10:07:07.323664+0000","last_became_peered":"2024-04-25T10:07:07.323664+0000","last_unstale":"2024-04-25T10:07:08.430845+0000","last_undegraded":"2024-04-25T10:07:08.430845+0000","last_fullsized":"2024-04-25T10:07:08.430845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340135+0000","last_change":"2024-04-25T10:07:06.312709+0000","last_active":"2024-04-25T10:07:08.340135+0000","last_peered":"2024-04-25T10:07:08.340135+0000","last_clean":"2024-04-25T10:07:08.340135+0000","last_became_active":"2024-04-25T10:07:06.311785+0000","last_became_peered":"2024-04-25T10:07:06.311785+0000","last_unstale":"2024-04-25T10:07:08.340135+0000","last_undegraded":"2024-04-25T10:07:08.340135+0000","last_fullsized":"2024-04-25T10:07:08.340135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:52.458435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339243+0000","last_change":"2024-04-25T10:07:07.332051+0000","last_active":"2024-04-25T10:07:08.339243+0000","last_peered":"2024-04-25T10:07:08.339243+0000","last_clean":"2024-04-25T10:07:08.339243+0000","last_became_active":"2024-04-25T10:07:07.331656+0000","last_became_peered":"2024-04-25T10:07:07.331656+0000","last_unstale":"2024-04-25T10:07:08.339243+0000","last_undegraded":"2024-04-25T10:07:08.339243+0000","last_fullsized":"2024-04-25T10:07:08.339243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627326+0000","last_change":"2024-04-25T10:07:06.316859+0000","last_active":"2024-04-25T10:07:09.627326+0000","last_peered":"2024-04-25T10:07:09.627326+0000","last_clean":"2024-04-25T10:07:09.627326+0000","last_became_active":"2024-04-25T10:07:06.314454+0000","last_became_peered":"2024-04-25T10:07:06.314454+0000","last_unstale":"2024-04-25T10:07:09.627326+0000","last_undegraded":"2024-04-25T10:07:09.627326+0000","last_fullsized":"2024-04-25T10:07:09.627326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.867432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341342+0000","last_change":"2024-04-25T10:07:07.352036+0000","last_active":"2024-04-25T10:07:08.341342+0000","last_peered":"2024-04-25T10:07:08.341342+0000","last_clean":"2024-04-25T10:07:08.341342+0000","last_became_active":"2024-04-25T10:07:07.351244+0000","last_became_peered":"2024-04-25T10:07:07.351244+0000","last_unstale":"2024-04-25T10:07:08.341342+0000","last_undegraded":"2024-04-25T10:07:08.341342+0000","last_fullsized":"2024-04-25T10:07:08.341342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:31.725733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341722+0000","last_change":"2024-04-25T10:07:06.313325+0000","last_active":"2024-04-25T10:07:08.341722+0000","last_peered":"2024-04-25T10:07:08.341722+0000","last_clean":"2024-04-25T10:07:08.341722+0000","last_became_active":"2024-04-25T10:07:06.312922+0000","last_became_peered":"2024-04-25T10:07:06.312922+0000","last_unstale":"2024-04-25T10:07:08.341722+0000","last_undegraded":"2024-04-25T10:07:08.341722+0000","last_fullsized":"2024-04-25T10:07:08.341722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013388+0000","last_change":"2024-04-25T10:07:07.334790+0000","last_active":"2024-04-25T10:07:09.013388+0000","last_peered":"2024-04-25T10:07:09.013388+0000","last_clean":"2024-04-25T10:07:09.013388+0000","last_became_active":"2024-04-25T10:07:07.334587+0000","last_became_peered":"2024-04-25T10:07:07.334587+0000","last_unstale":"2024-04-25T10:07:09.013388+0000","last_undegraded":"2024-04-25T10:07:09.013388+0000","last_fullsized":"2024-04-25T10:07:09.013388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340029+0000","last_change":"2024-04-25T10:07:06.317802+0000","last_active":"2024-04-25T10:07:08.340029+0000","last_peered":"2024-04-25T10:07:08.340029+0000","last_clean":"2024-04-25T10:07:08.340029+0000","last_became_active":"2024-04-25T10:07:06.317291+0000","last_became_peered":"2024-04-25T10:07:06.317291+0000","last_unstale":"2024-04-25T10:07:08.340029+0000","last_undegraded":"2024-04-25T10:07:08.340029+0000","last_fullsized":"2024-04-25T10:07:08.340029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.867432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115871+0000","last_change":"2024-04-25T10:07:07.351461+0000","last_active":"2024-04-25T10:07:10.115871+0000","last_peered":"2024-04-25T10:07:10.115871+0000","last_clean":"2024-04-25T10:07:10.115871+0000","last_became_active":"2024-04-25T10:07:07.351141+0000","last_became_peered":"2024-04-25T10:07:07.351141+0000","last_unstale":"2024-04-25T10:07:10.115871+0000","last_undegraded":"2024-04-25T10:07:10.115871+0000","last_fullsized":"2024-04-25T10:07:10.115871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:46.704814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627332+0000","last_change":"2024-04-25T10:07:06.319663+0000","last_active":"2024-04-25T10:07:09.627332+0000","last_peered":"2024-04-25T10:07:09.627332+0000","last_clean":"2024-04-25T10:07:09.627332+0000","last_became_active":"2024-04-25T10:07:06.317321+0000","last_became_peered":"2024-04-25T10:07:06.317321+0000","last_unstale":"2024-04-25T10:07:09.627332+0000","last_undegraded":"2024-04-25T10:07:09.627332+0000","last_fullsized":"2024-04-25T10:07:09.627332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117185+0000","last_change":"2024-04-25T10:07:07.350498+0000","last_active":"2024-04-25T10:07:10.117185+0000","last_peered":"2024-04-25T10:07:10.117185+0000","last_clean":"2024-04-25T10:07:10.117185+0000","last_became_active":"2024-04-25T10:07:07.349929+0000","last_became_peered":"2024-04-25T10:07:07.349929+0000","last_unstale":"2024-04-25T10:07:10.117185+0000","last_undegraded":"2024-04-25T10:07:10.117185+0000","last_fullsized":"2024-04-25T10:07:10.117185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:23.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013849+0000","last_change":"2024-04-25T10:07:06.312357+0000","last_active":"2024-04-25T10:07:09.013849+0000","last_peered":"2024-04-25T10:07:09.013849+0000","last_clean":"2024-04-25T10:07:09.013849+0000","last_became_active":"2024-04-25T10:07:06.312088+0000","last_became_peered":"2024-04-25T10:07:06.312088+0000","last_unstale":"2024-04-25T10:07:09.013849+0000","last_undegraded":"2024-04-25T10:07:09.013849+0000","last_fullsized":"2024-04-25T10:07:09.013849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:05.470696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013146+0000","last_change":"2024-04-25T10:07:07.345439+0000","last_active":"2024-04-25T10:07:09.013146+0000","last_peered":"2024-04-25T10:07:09.013146+0000","last_clean":"2024-04-25T10:07:09.013146+0000","last_became_active":"2024-04-25T10:07:07.345205+0000","last_became_peered":"2024-04-25T10:07:07.345205+0000","last_unstale":"2024-04-25T10:07:09.013146+0000","last_undegraded":"2024-04-25T10:07:09.013146+0000","last_fullsized":"2024-04-25T10:07:09.013146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:56.522956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340351+0000","last_change":"2024-04-25T10:07:06.313807+0000","last_active":"2024-04-25T10:07:08.340351+0000","last_peered":"2024-04-25T10:07:08.340351+0000","last_clean":"2024-04-25T10:07:08.340351+0000","last_became_active":"2024-04-25T10:07:06.313376+0000","last_became_peered":"2024-04-25T10:07:06.313376+0000","last_unstale":"2024-04-25T10:07:08.340351+0000","last_undegraded":"2024-04-25T10:07:08.340351+0000","last_fullsized":"2024-04-25T10:07:08.340351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:44.436918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116968+0000","last_change":"2024-04-25T10:07:07.351251+0000","last_active":"2024-04-25T10:07:10.116968+0000","last_peered":"2024-04-25T10:07:10.116968+0000","last_clean":"2024-04-25T10:07:10.116968+0000","last_became_active":"2024-04-25T10:07:07.350404+0000","last_became_peered":"2024-04-25T10:07:07.350404+0000","last_unstale":"2024-04-25T10:07:10.116968+0000","last_undegraded":"2024-04-25T10:07:10.116968+0000","last_fullsized":"2024-04-25T10:07:10.116968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:49.296728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.626713+0000","last_change":"2024-04-25T10:07:06.322457+0000","last_active":"2024-04-25T10:07:09.626713+0000","last_peered":"2024-04-25T10:07:09.626713+0000","last_clean":"2024-04-25T10:07:09.626713+0000","last_became_active":"2024-04-25T10:07:06.318116+0000","last_became_peered":"2024-04-25T10:07:06.318116+0000","last_unstale":"2024-04-25T10:07:09.626713+0000","last_undegraded":"2024-04-25T10:07:09.626713+0000","last_fullsized":"2024-04-25T10:07:09.626713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:44.436918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627151+0000","last_change":"2024-04-25T10:07:07.328355+0000","last_active":"2024-04-25T10:07:09.627151+0000","last_peered":"2024-04-25T10:07:09.627151+0000","last_clean":"2024-04-25T10:07:09.627151+0000","last_became_active":"2024-04-25T10:07:07.328088+0000","last_became_peered":"2024-04-25T10:07:07.328088+0000","last_unstale":"2024-04-25T10:07:09.627151+0000","last_undegraded":"2024-04-25T10:07:09.627151+0000","last_fullsized":"2024-04-25T10:07:09.627151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337372+0000","last_change":"2024-04-25T10:07:06.306731+0000","last_active":"2024-04-25T10:07:08.337372+0000","last_peered":"2024-04-25T10:07:08.337372+0000","last_clean":"2024-04-25T10:07:08.337372+0000","last_became_active":"2024-04-25T10:07:06.306314+0000","last_became_peered":"2024-04-25T10:07:06.306314+0000","last_unstale":"2024-04-25T10:07:08.337372+0000","last_undegraded":"2024-04-25T10:07:08.337372+0000","last_fullsized":"2024-04-25T10:07:08.337372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:28.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117145+0000","last_change":"2024-04-25T10:07:07.356371+0000","last_active":"2024-04-25T10:07:10.117145+0000","last_peered":"2024-04-25T10:07:10.117145+0000","last_clean":"2024-04-25T10:07:10.117145+0000","last_became_active":"2024-04-25T10:07:07.352731+0000","last_became_peered":"2024-04-25T10:07:07.352731+0000","last_unstale":"2024-04-25T10:07:10.117145+0000","last_undegraded":"2024-04-25T10:07:10.117145+0000","last_fullsized":"2024-04-25T10:07:10.117145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115672+0000","last_change":"2024-04-25T10:07:06.320436+0000","last_active":"2024-04-25T10:07:10.115672+0000","last_peered":"2024-04-25T10:07:10.115672+0000","last_clean":"2024-04-25T10:07:10.115672+0000","last_became_active":"2024-04-25T10:07:06.319958+0000","last_became_peered":"2024-04-25T10:07:06.319958+0000","last_unstale":"2024-04-25T10:07:10.115672+0000","last_undegraded":"2024-04-25T10:07:10.115672+0000","last_fullsized":"2024-04-25T10:07:10.115672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:02.968109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117278+0000","last_change":"2024-04-25T10:07:07.352988+0000","last_active":"2024-04-25T10:07:10.117278+0000","last_peered":"2024-04-25T10:07:10.117278+0000","last_clean":"2024-04-25T10:07:10.117278+0000","last_became_active":"2024-04-25T10:07:07.352585+0000","last_became_peered":"2024-04-25T10:07:07.352585+0000","last_unstale":"2024-04-25T10:07:10.117278+0000","last_undegraded":"2024-04-25T10:07:10.117278+0000","last_fullsized":"2024-04-25T10:07:10.117278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:35.056084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.338565+0000","last_change":"2024-04-25T10:07:06.308389+0000","last_active":"2024-04-25T10:07:08.338565+0000","last_peered":"2024-04-25T10:07:08.338565+0000","last_clean":"2024-04-25T10:07:08.338565+0000","last_became_active":"2024-04-25T10:07:06.308046+0000","last_became_peered":"2024-04-25T10:07:06.308046+0000","last_unstale":"2024-04-25T10:07:08.338565+0000","last_undegraded":"2024-04-25T10:07:08.338565+0000","last_fullsized":"2024-04-25T10:07:08.338565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:53.885696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.118014+0000","last_change":"2024-04-25T10:07:07.357349+0000","last_active":"2024-04-25T10:07:10.118014+0000","last_peered":"2024-04-25T10:07:10.118014+0000","last_clean":"2024-04-25T10:07:10.118014+0000","last_became_active":"2024-04-25T10:07:07.356970+0000","last_became_peered":"2024-04-25T10:07:07.356970+0000","last_unstale":"2024-04-25T10:07:10.118014+0000","last_undegraded":"2024-04-25T10:07:10.118014+0000","last_fullsized":"2024-04-25T10:07:10.118014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:16.375835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340105+0000","last_change":"2024-04-25T10:07:06.310585+0000","last_active":"2024-04-25T10:07:08.340105+0000","last_peered":"2024-04-25T10:07:08.340105+0000","last_clean":"2024-04-25T10:07:08.340105+0000","last_became_active":"2024-04-25T10:07:06.310265+0000","last_became_peered":"2024-04-25T10:07:06.310265+0000","last_unstale":"2024-04-25T10:07:08.340105+0000","last_undegraded":"2024-04-25T10:07:08.340105+0000","last_fullsized":"2024-04-25T10:07:08.340105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:48.687788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341312+0000","last_change":"2024-04-25T10:07:07.337180+0000","last_active":"2024-04-25T10:07:08.341312+0000","last_peered":"2024-04-25T10:07:08.341312+0000","last_clean":"2024-04-25T10:07:08.341312+0000","last_became_active":"2024-04-25T10:07:07.335230+0000","last_became_peered":"2024-04-25T10:07:07.335230+0000","last_unstale":"2024-04-25T10:07:08.341312+0000","last_undegraded":"2024-04-25T10:07:08.341312+0000","last_fullsized":"2024-04-25T10:07:08.341312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:16.166171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337856+0000","last_change":"2024-04-25T10:07:06.311469+0000","last_active":"2024-04-25T10:07:08.337856+0000","last_peered":"2024-04-25T10:07:08.337856+0000","last_clean":"2024-04-25T10:07:08.337856+0000","last_became_active":"2024-04-25T10:07:06.311182+0000","last_became_peered":"2024-04-25T10:07:06.311182+0000","last_unstale":"2024-04-25T10:07:08.337856+0000","last_undegraded":"2024-04-25T10:07:08.337856+0000","last_fullsized":"2024-04-25T10:07:08.337856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:44.436918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117507+0000","last_change":"2024-04-25T10:07:07.350766+0000","last_active":"2024-04-25T10:07:10.117507+0000","last_peered":"2024-04-25T10:07:10.117507+0000","last_clean":"2024-04-25T10:07:10.117507+0000","last_became_active":"2024-04-25T10:07:07.350181+0000","last_became_peered":"2024-04-25T10:07:07.350181+0000","last_unstale":"2024-04-25T10:07:10.117507+0000","last_undegraded":"2024-04-25T10:07:10.117507+0000","last_fullsized":"2024-04-25T10:07:10.117507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:16.166171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116058+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:07:10.116058+0000","last_peered":"2024-04-25T10:07:10.116058+0000","last_clean":"2024-04-25T10:07:10.116058+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:07:10.116058+0000","last_undegraded":"2024-04-25T10:07:10.116058+0000","last_fullsized":"2024-04-25T10:07:10.116058+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340032+0000","last_change":"2024-04-25T10:07:06.306253+0000","last_active":"2024-04-25T10:07:08.340032+0000","last_peered":"2024-04-25T10:07:08.340032+0000","last_clean":"2024-04-25T10:07:08.340032+0000","last_became_active":"2024-04-25T10:07:06.305849+0000","last_became_peered":"2024-04-25T10:07:06.305849+0000","last_unstale":"2024-04-25T10:07:08.340032+0000","last_undegraded":"2024-04-25T10:07:08.340032+0000","last_fullsized":"2024-04-25T10:07:08.340032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:28.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430815+0000","last_change":"2024-04-25T10:07:07.337895+0000","last_active":"2024-04-25T10:07:08.430815+0000","last_peered":"2024-04-25T10:07:08.430815+0000","last_clean":"2024-04-25T10:07:08.430815+0000","last_became_active":"2024-04-25T10:07:07.337609+0000","last_became_peered":"2024-04-25T10:07:07.337609+0000","last_unstale":"2024-04-25T10:07:08.430815+0000","last_undegraded":"2024-04-25T10:07:08.430815+0000","last_fullsized":"2024-04-25T10:07:08.430815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115960+0000","last_change":"2024-04-25T10:07:06.325347+0000","last_active":"2024-04-25T10:07:10.115960+0000","last_peered":"2024-04-25T10:07:10.115960+0000","last_clean":"2024-04-25T10:07:10.115960+0000","last_became_active":"2024-04-25T10:07:06.324740+0000","last_became_peered":"2024-04-25T10:07:06.324740+0000","last_unstale":"2024-04-25T10:07:10.115960+0000","last_undegraded":"2024-04-25T10:07:10.115960+0000","last_fullsized":"2024-04-25T10:07:10.115960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.691331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.012839+0000","last_change":"2024-04-25T10:07:07.337867+0000","last_active":"2024-04-25T10:07:09.012839+0000","last_peered":"2024-04-25T10:07:09.012839+0000","last_clean":"2024-04-25T10:07:09.012839+0000","last_became_active":"2024-04-25T10:07:07.337521+0000","last_became_peered":"2024-04-25T10:07:07.337521+0000","last_unstale":"2024-04-25T10:07:09.012839+0000","last_undegraded":"2024-04-25T10:07:09.012839+0000","last_fullsized":"2024-04-25T10:07:09.012839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:05.611848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337610+0000","last_change":"2024-04-25T10:07:06.308483+0000","last_active":"2024-04-25T10:07:08.337610+0000","last_peered":"2024-04-25T10:07:08.337610+0000","last_clean":"2024-04-25T10:07:08.337610+0000","last_became_active":"2024-04-25T10:07:06.308156+0000","last_became_peered":"2024-04-25T10:07:06.308156+0000","last_unstale":"2024-04-25T10:07:08.337610+0000","last_undegraded":"2024-04-25T10:07:08.337610+0000","last_fullsized":"2024-04-25T10:07:08.337610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:25.307131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339704+0000","last_change":"2024-04-25T10:07:07.351559+0000","last_active":"2024-04-25T10:07:08.339704+0000","last_peered":"2024-04-25T10:07:08.339704+0000","last_clean":"2024-04-25T10:07:08.339704+0000","last_became_active":"2024-04-25T10:07:07.351271+0000","last_became_peered":"2024-04-25T10:07:07.351271+0000","last_unstale":"2024-04-25T10:07:08.339704+0000","last_undegraded":"2024-04-25T10:07:08.339704+0000","last_fullsized":"2024-04-25T10:07:08.339704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340604+0000","last_change":"2024-04-25T10:07:06.314916+0000","last_active":"2024-04-25T10:07:08.340604+0000","last_peered":"2024-04-25T10:07:08.340604+0000","last_clean":"2024-04-25T10:07:08.340604+0000","last_became_active":"2024-04-25T10:07:06.314572+0000","last_became_peered":"2024-04-25T10:07:06.314572+0000","last_unstale":"2024-04-25T10:07:08.340604+0000","last_undegraded":"2024-04-25T10:07:08.340604+0000","last_fullsized":"2024-04-25T10:07:08.340604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:53.885696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115644+0000","last_change":"2024-04-25T10:07:07.342482+0000","last_active":"2024-04-25T10:07:10.115644+0000","last_peered":"2024-04-25T10:07:10.115644+0000","last_clean":"2024-04-25T10:07:10.115644+0000","last_became_active":"2024-04-25T10:07:07.341566+0000","last_became_peered":"2024-04-25T10:07:07.341566+0000","last_unstale":"2024-04-25T10:07:10.115644+0000","last_undegraded":"2024-04-25T10:07:10.115644+0000","last_fullsized":"2024-04-25T10:07:10.115644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:14.880915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339445+0000","last_change":"2024-04-25T10:07:06.317437+0000","last_active":"2024-04-25T10:07:08.339445+0000","last_peered":"2024-04-25T10:07:08.339445+0000","last_clean":"2024-04-25T10:07:08.339445+0000","last_became_active":"2024-04-25T10:07:06.316494+0000","last_became_peered":"2024-04-25T10:07:06.316494+0000","last_unstale":"2024-04-25T10:07:08.339445+0000","last_undegraded":"2024-04-25T10:07:08.339445+0000","last_fullsized":"2024-04-25T10:07:08.339445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341452+0000","last_change":"2024-04-25T10:07:07.337051+0000","last_active":"2024-04-25T10:07:08.341452+0000","last_peered":"2024-04-25T10:07:08.341452+0000","last_clean":"2024-04-25T10:07:08.341452+0000","last_became_active":"2024-04-25T10:07:07.334884+0000","last_became_peered":"2024-04-25T10:07:07.334884+0000","last_unstale":"2024-04-25T10:07:08.341452+0000","last_undegraded":"2024-04-25T10:07:08.341452+0000","last_fullsized":"2024-04-25T10:07:08.341452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.763141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117681+0000","last_change":"2024-04-25T10:07:06.331016+0000","last_active":"2024-04-25T10:07:10.117681+0000","last_peered":"2024-04-25T10:07:10.117681+0000","last_clean":"2024-04-25T10:07:10.117681+0000","last_became_active":"2024-04-25T10:07:06.330712+0000","last_became_peered":"2024-04-25T10:07:06.330712+0000","last_unstale":"2024-04-25T10:07:10.117681+0000","last_undegraded":"2024-04-25T10:07:10.117681+0000","last_fullsized":"2024-04-25T10:07:10.117681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.867432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339661+0000","last_change":"2024-04-25T10:07:07.351856+0000","last_active":"2024-04-25T10:07:08.339661+0000","last_peered":"2024-04-25T10:07:08.339661+0000","last_clean":"2024-04-25T10:07:08.339661+0000","last_became_active":"2024-04-25T10:07:07.351585+0000","last_became_peered":"2024-04-25T10:07:07.351585+0000","last_unstale":"2024-04-25T10:07:08.339661+0000","last_undegraded":"2024-04-25T10:07:08.339661+0000","last_fullsized":"2024-04-25T10:07:08.339661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340221+0000","last_change":"2024-04-25T10:07:06.317585+0000","last_active":"2024-04-25T10:07:08.340221+0000","last_peered":"2024-04-25T10:07:08.340221+0000","last_clean":"2024-04-25T10:07:08.340221+0000","last_became_active":"2024-04-25T10:07:06.316964+0000","last_became_peered":"2024-04-25T10:07:06.316964+0000","last_unstale":"2024-04-25T10:07:08.340221+0000","last_undegraded":"2024-04-25T10:07:08.340221+0000","last_fullsized":"2024-04-25T10:07:08.340221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:58.603977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431612+0000","last_change":"2024-04-25T10:07:07.352730+0000","last_active":"2024-04-25T10:07:08.431612+0000","last_peered":"2024-04-25T10:07:08.431612+0000","last_clean":"2024-04-25T10:07:08.431612+0000","last_became_active":"2024-04-25T10:07:07.352486+0000","last_became_peered":"2024-04-25T10:07:07.352486+0000","last_unstale":"2024-04-25T10:07:08.431612+0000","last_undegraded":"2024-04-25T10:07:08.431612+0000","last_fullsized":"2024-04-25T10:07:08.431612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:23.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627361+0000","last_change":"2024-04-25T10:07:06.327061+0000","last_active":"2024-04-25T10:07:09.627361+0000","last_peered":"2024-04-25T10:07:09.627361+0000","last_clean":"2024-04-25T10:07:09.627361+0000","last_became_active":"2024-04-25T10:07:06.326227+0000","last_became_peered":"2024-04-25T10:07:06.326227+0000","last_unstale":"2024-04-25T10:07:09.627361+0000","last_undegraded":"2024-04-25T10:07:09.627361+0000","last_fullsized":"2024-04-25T10:07:09.627361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013232+0000","last_change":"2024-04-25T10:07:07.326642+0000","last_active":"2024-04-25T10:07:09.013232+0000","last_peered":"2024-04-25T10:07:09.013232+0000","last_clean":"2024-04-25T10:07:09.013232+0000","last_became_active":"2024-04-25T10:07:07.326261+0000","last_became_peered":"2024-04-25T10:07:07.326261+0000","last_unstale":"2024-04-25T10:07:09.013232+0000","last_undegraded":"2024-04-25T10:07:09.013232+0000","last_fullsized":"2024-04-25T10:07:09.013232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:01.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115657+0000","last_change":"2024-04-25T10:07:06.325264+0000","last_active":"2024-04-25T10:07:10.115657+0000","last_peered":"2024-04-25T10:07:10.115657+0000","last_clean":"2024-04-25T10:07:10.115657+0000","last_became_active":"2024-04-25T10:07:06.324704+0000","last_became_peered":"2024-04-25T10:07:06.324704+0000","last_unstale":"2024-04-25T10:07:10.115657+0000","last_undegraded":"2024-04-25T10:07:10.115657+0000","last_fullsized":"2024-04-25T10:07:10.115657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:52.458435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115184+0000","last_change":"2024-04-25T10:07:07.340986+0000","last_active":"2024-04-25T10:07:10.115184+0000","last_peered":"2024-04-25T10:07:10.115184+0000","last_clean":"2024-04-25T10:07:10.115184+0000","last_became_active":"2024-04-25T10:07:07.332637+0000","last_became_peered":"2024-04-25T10:07:07.332637+0000","last_unstale":"2024-04-25T10:07:10.115184+0000","last_undegraded":"2024-04-25T10:07:10.115184+0000","last_fullsized":"2024-04-25T10:07:10.115184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:17.677957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339302+0000","last_change":"2024-04-25T10:07:06.324149+0000","last_active":"2024-04-25T10:07:08.339302+0000","last_peered":"2024-04-25T10:07:08.339302+0000","last_clean":"2024-04-25T10:07:08.339302+0000","last_became_active":"2024-04-25T10:07:06.323900+0000","last_became_peered":"2024-04-25T10:07:06.323900+0000","last_unstale":"2024-04-25T10:07:08.339302+0000","last_undegraded":"2024-04-25T10:07:08.339302+0000","last_fullsized":"2024-04-25T10:07:08.339302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431004+0000","last_change":"2024-04-25T10:07:07.337614+0000","last_active":"2024-04-25T10:07:08.431004+0000","last_peered":"2024-04-25T10:07:08.431004+0000","last_clean":"2024-04-25T10:07:08.431004+0000","last_became_active":"2024-04-25T10:07:07.333283+0000","last_became_peered":"2024-04-25T10:07:07.333283+0000","last_unstale":"2024-04-25T10:07:08.431004+0000","last_undegraded":"2024-04-25T10:07:08.431004+0000","last_fullsized":"2024-04-25T10:07:08.431004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339925+0000","last_change":"2024-04-25T10:07:06.317292+0000","last_active":"2024-04-25T10:07:08.339925+0000","last_peered":"2024-04-25T10:07:08.339925+0000","last_clean":"2024-04-25T10:07:08.339925+0000","last_became_active":"2024-04-25T10:07:06.316862+0000","last_became_peered":"2024-04-25T10:07:06.316862+0000","last_unstale":"2024-04-25T10:07:08.339925+0000","last_undegraded":"2024-04-25T10:07:08.339925+0000","last_fullsized":"2024-04-25T10:07:08.339925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.691331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431273+0000","last_change":"2024-04-25T10:07:07.329541+0000","last_active":"2024-04-25T10:07:08.431273+0000","last_peered":"2024-04-25T10:07:08.431273+0000","last_clean":"2024-04-25T10:07:08.431273+0000","last_became_active":"2024-04-25T10:07:07.329296+0000","last_became_peered":"2024-04-25T10:07:07.329296+0000","last_unstale":"2024-04-25T10:07:08.431273+0000","last_undegraded":"2024-04-25T10:07:08.431273+0000","last_fullsized":"2024-04-25T10:07:08.431273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117039+0000","last_change":"2024-04-25T10:07:06.331241+0000","last_active":"2024-04-25T10:07:10.117039+0000","last_peered":"2024-04-25T10:07:10.117039+0000","last_clean":"2024-04-25T10:07:10.117039+0000","last_became_active":"2024-04-25T10:07:06.330637+0000","last_became_peered":"2024-04-25T10:07:06.330637+0000","last_unstale":"2024-04-25T10:07:10.117039+0000","last_undegraded":"2024-04-25T10:07:10.117039+0000","last_fullsized":"2024-04-25T10:07:10.117039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627101+0000","last_change":"2024-04-25T10:07:07.334097+0000","last_active":"2024-04-25T10:07:09.627101+0000","last_peered":"2024-04-25T10:07:09.627101+0000","last_clean":"2024-04-25T10:07:09.627101+0000","last_became_active":"2024-04-25T10:07:07.332086+0000","last_became_peered":"2024-04-25T10:07:07.332086+0000","last_unstale":"2024-04-25T10:07:09.627101+0000","last_undegraded":"2024-04-25T10:07:09.627101+0000","last_fullsized":"2024-04-25T10:07:09.627101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.370945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341238+0000","last_change":"2024-04-25T10:07:07.349145+0000","last_active":"2024-04-25T10:07:08.341238+0000","last_peered":"2024-04-25T10:07:08.341238+0000","last_clean":"2024-04-25T10:07:08.341238+0000","last_became_active":"2024-04-25T10:07:07.348916+0000","last_became_peered":"2024-04-25T10:07:07.348916+0000","last_unstale":"2024-04-25T10:07:08.341238+0000","last_undegraded":"2024-04-25T10:07:08.341238+0000","last_fullsized":"2024-04-25T10:07:08.341238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:35.056084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116709+0000","last_change":"2024-04-25T10:07:06.330571+0000","last_active":"2024-04-25T10:07:10.116709+0000","last_peered":"2024-04-25T10:07:10.116709+0000","last_clean":"2024-04-25T10:07:10.116709+0000","last_became_active":"2024-04-25T10:07:06.329886+0000","last_became_peered":"2024-04-25T10:07:06.329886+0000","last_unstale":"2024-04-25T10:07:10.116709+0000","last_undegraded":"2024-04-25T10:07:10.116709+0000","last_fullsized":"2024-04-25T10:07:10.116709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340935+0000","last_change":"2024-04-25T10:07:07.347234+0000","last_active":"2024-04-25T10:07:08.340935+0000","last_peered":"2024-04-25T10:07:08.340935+0000","last_clean":"2024-04-25T10:07:08.340935+0000","last_became_active":"2024-04-25T10:07:07.346961+0000","last_became_peered":"2024-04-25T10:07:07.346961+0000","last_unstale":"2024-04-25T10:07:08.340935+0000","last_undegraded":"2024-04-25T10:07:08.340935+0000","last_fullsized":"2024-04-25T10:07:08.340935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:23.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117348+0000","last_change":"2024-04-25T10:07:06.330806+0000","last_active":"2024-04-25T10:07:10.117348+0000","last_peered":"2024-04-25T10:07:10.117348+0000","last_clean":"2024-04-25T10:07:10.117348+0000","last_became_active":"2024-04-25T10:07:06.330360+0000","last_became_peered":"2024-04-25T10:07:06.330360+0000","last_unstale":"2024-04-25T10:07:10.117348+0000","last_undegraded":"2024-04-25T10:07:10.117348+0000","last_fullsized":"2024-04-25T10:07:10.117348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:48.687788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013341+0000","last_change":"2024-04-25T10:07:07.337072+0000","last_active":"2024-04-25T10:07:09.013341+0000","last_peered":"2024-04-25T10:07:09.013341+0000","last_clean":"2024-04-25T10:07:09.013341+0000","last_became_active":"2024-04-25T10:07:07.336767+0000","last_became_peered":"2024-04-25T10:07:07.336767+0000","last_unstale":"2024-04-25T10:07:09.013341+0000","last_undegraded":"2024-04-25T10:07:09.013341+0000","last_fullsized":"2024-04-25T10:07:09.013341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.370945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431572+0000","last_change":"2024-04-25T10:07:06.322321+0000","last_active":"2024-04-25T10:07:08.431572+0000","last_peered":"2024-04-25T10:07:08.431572+0000","last_clean":"2024-04-25T10:07:08.431572+0000","last_became_active":"2024-04-25T10:07:06.321990+0000","last_became_peered":"2024-04-25T10:07:06.321990+0000","last_unstale":"2024-04-25T10:07:08.431572+0000","last_undegraded":"2024-04-25T10:07:08.431572+0000","last_fullsized":"2024-04-25T10:07:08.431572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:25.307131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431571+0000","last_change":"2024-04-25T10:07:07.337246+0000","last_active":"2024-04-25T10:07:08.431571+0000","last_peered":"2024-04-25T10:07:08.431571+0000","last_clean":"2024-04-25T10:07:08.431571+0000","last_became_active":"2024-04-25T10:07:07.336894+0000","last_became_peered":"2024-04-25T10:07:07.336894+0000","last_unstale":"2024-04-25T10:07:08.431571+0000","last_undegraded":"2024-04-25T10:07:08.431571+0000","last_fullsized":"2024-04-25T10:07:08.431571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013544+0000","last_change":"2024-04-25T10:07:06.324947+0000","last_active":"2024-04-25T10:07:09.013544+0000","last_peered":"2024-04-25T10:07:09.013544+0000","last_clean":"2024-04-25T10:07:09.013544+0000","last_became_active":"2024-04-25T10:07:06.324464+0000","last_became_peered":"2024-04-25T10:07:06.324464+0000","last_unstale":"2024-04-25T10:07:09.013544+0000","last_undegraded":"2024-04-25T10:07:09.013544+0000","last_fullsized":"2024-04-25T10:07:09.013544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:25.307131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":64760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:12.526 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:07:12.526 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T10:07:12.526 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T10:07:12.527 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T10:07:12.527 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T10:07:12.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.763+0000 7f0bea800640 1 Processor -- start 2024-04-25T10:07:12.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.763+0000 7f0bea800640 1 -- start start 2024-04-25T10:07:12.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.765+0000 7f0bea800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0be4161ee0 0x7f0be40b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.765+0000 7f0bea800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0be40b2570 0x7f0be41632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.765+0000 7f0bea800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be4071330 con 0x7f0be41664d0 2024-04-25T10:07:12.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.765+0000 7f0bea800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be40720f0 con 0x7f0be4161ee0 2024-04-25T10:07:12.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.765+0000 7f0bea800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be4071610 con 0x7f0be40b2570 2024-04-25T10:07:12.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.765+0000 7f0be3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0be4161ee0 0x7f0be40b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.765+0000 7f0bdaa00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0be41664d0 0x7f0be406dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42080/0 (socket says 172.21.15.119:42080) 2024-04-25T10:07:12.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.765+0000 7f0bdaa00640 1 -- 172.21.15.119:0/1401142629 learned_addr learned my addr 172.21.15.119:0/1401142629 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.766+0000 7f0be3e00640 1 --2- 172.21.15.119:0/1401142629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0be40b2570 0x7f0be41632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.766+0000 7f0be3400640 1 -- 172.21.15.119:0/1401142629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0be40b2570 msgr2=0x7f0be41632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.766+0000 7f0be3400640 1 --2- 172.21.15.119:0/1401142629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0be40b2570 0x7f0be41632d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.766+0000 7f0be3400640 1 -- 172.21.15.119:0/1401142629 >> v1:172.21.15.119:6789/0 conn(0x7f0be41664d0 legacy=0x7f0be406dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.766+0000 7f0be2a00640 1 -- 172.21.15.119:0/1401142629 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621756914 0 0) 0x7f0be4071330 con 0x7f0be41664d0 2024-04-25T10:07:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.766+0000 7f0be3400640 1 -- 172.21.15.119:0/1401142629 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be40718f0 con 0x7f0be4161ee0 2024-04-25T10:07:12.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.767+0000 7f0be3400640 1 --2- 172.21.15.119:0/1401142629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be4161ee0 0x7f0be40b2f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40097a0 tx=0x7f0bd402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4faf5e33c9304f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.767+0000 7f0be2a00640 1 -- 172.21.15.119:0/1401142629 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4037070 con 0x7f0be4161ee0 2024-04-25T10:07:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.767+0000 7f0be2a00640 1 -- 172.21.15.119:0/1401142629 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd4032440 con 0x7f0be4161ee0 2024-04-25T10:07:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.767+0000 7f0be2a00640 1 -- 172.21.15.119:0/1401142629 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4032810 con 0x7f0be4161ee0 2024-04-25T10:07:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.768+0000 7f0bea800640 1 -- 172.21.15.119:0/1401142629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be4161ee0 msgr2=0x7f0be40b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.768+0000 7f0bea800640 1 --2- 172.21.15.119:0/1401142629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be4161ee0 0x7f0be40b2f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40097a0 tx=0x7f0bd402a180 comp rx=0 tx=0).stop 2024-04-25T10:07:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.768+0000 7f0bea800640 1 -- 172.21.15.119:0/1401142629 shutdown_connections 2024-04-25T10:07:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.768+0000 7f0bea800640 1 --2- 172.21.15.119:0/1401142629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0be40b2570 0x7f0be41632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.768+0000 7f0bea800640 1 --2- 172.21.15.119:0/1401142629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be4161ee0 0x7f0be40b2f60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.768+0000 7f0bea800640 1 -- 172.21.15.119:0/1401142629 >> 172.21.15.119:0/1401142629 conn(0x7f0be406aba0 msgr2=0x7f0be406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.768+0000 7f0bea800640 1 -- 172.21.15.119:0/1401142629 shutdown_connections 2024-04-25T10:07:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.768+0000 7f0bea800640 1 -- 172.21.15.119:0/1401142629 wait complete. 2024-04-25T10:07:12.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.769+0000 7f0bea800640 1 Processor -- start 2024-04-25T10:07:12.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.770+0000 7f0bea800640 1 -- start start 2024-04-25T10:07:12.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.770+0000 7f0bea800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0be40b2570 0x7f0be4156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.770+0000 7f0bea800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be4161ee0 0x7f0be41f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.771+0000 7f0bea800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0be41664d0 0x7f0be41f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.771+0000 7f0bea800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be40b3d30 con 0x7f0be4161ee0 2024-04-25T10:07:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.771+0000 7f0bea800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be40b3bb0 con 0x7f0be41664d0 2024-04-25T10:07:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.771+0000 7f0bea800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be40b34a0 con 0x7f0be40b2570 2024-04-25T10:07:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.771+0000 7f0bdaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be4161ee0 0x7f0be41f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.771+0000 7f0bdaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be4161ee0 0x7f0be41f9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34464/0 (socket says 172.21.15.119:34464) 2024-04-25T10:07:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.771+0000 7f0bdaa00640 1 -- 172.21.15.119:0/3806661643 learned_addr learned my addr 172.21.15.119:0/3806661643 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.771+0000 7f0be3400640 1 --2- 172.21.15.119:0/3806661643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0be40b2570 0x7f0be4156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.771+0000 7f0bdaa00640 1 -- 172.21.15.119:0/3806661643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0be40b2570 msgr2=0x7f0be4156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.771+0000 7f0bdaa00640 1 --2- 172.21.15.119:0/3806661643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0be40b2570 0x7f0be4156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.771+0000 7f0bdaa00640 1 -- 172.21.15.119:0/3806661643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0be41664d0 msgr2=0x7f0be41f8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.771+0000 7f0bdaa00640 1 --2- 172.21.15.119:0/3806661643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0be41664d0 0x7f0be41f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.771+0000 7f0bdaa00640 1 -- 172.21.15.119:0/3806661643 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be41f97d0 con 0x7f0be4161ee0 2024-04-25T10:07:12.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.771+0000 7f0bdaa00640 1 --2- 172.21.15.119:0/3806661643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be4161ee0 0x7f0be41f9290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00c430 tx=0x7f0bcc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.772+0000 7f0bdbe00640 1 -- 172.21.15.119:0/3806661643 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0074e0 con 0x7f0be4161ee0 2024-04-25T10:07:12.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.772+0000 7f0bdbe00640 1 -- 172.21.15.119:0/3806661643 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bcc0047e0 con 0x7f0be4161ee0 2024-04-25T10:07:12.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.772+0000 7f0bdbe00640 1 -- 172.21.15.119:0/3806661643 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc004bd0 con 0x7f0be4161ee0 2024-04-25T10:07:12.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.772+0000 7f0bea800640 1 -- 172.21.15.119:0/3806661643 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be41f9060 con 0x7f0be4161ee0 2024-04-25T10:07:12.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.772+0000 7f0bea800640 1 -- 172.21.15.119:0/3806661643 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be40abba0 con 0x7f0be4161ee0 2024-04-25T10:07:12.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.773+0000 7f0bea800640 1 -- 172.21.15.119:0/3806661643 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be40749b0 con 0x7f0be4161ee0 2024-04-25T10:07:12.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.774+0000 7f0bdbe00640 1 -- 172.21.15.119:0/3806661643 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bcc011070 con 0x7f0be4161ee0 2024-04-25T10:07:12.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.775+0000 7f0bdbe00640 1 --2- 172.21.15.119:0/3806661643 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0bc006b4d0 0x7f0bc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.775+0000 7f0bdbe00640 1 -- 172.21.15.119:0/3806661643 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bcc089e40 con 0x7f0be4161ee0 2024-04-25T10:07:12.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.775+0000 7f0be3400640 1 --2- 172.21.15.119:0/3806661643 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0bc006b4d0 0x7f0bc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.776+0000 7f0be3400640 1 --2- 172.21.15.119:0/3806661643 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0bc006b4d0 0x7f0bc006d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4034000 tx=0x7f0bd402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:12.779+0000 7f0bdbe00640 1 -- 172.21.15.119:0/3806661643 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0bcc058cb0 con 0x7f0be4161ee0 2024-04-25T10:07:13.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.026+0000 7f0bea800640 1 -- 172.21.15.119:0/3806661643 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0be406d4c0 con 0x7f0be4161ee0 2024-04-25T10:07:13.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.031+0000 7f0bdbe00640 1 -- 172.21.15.119:0/3806661643 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171791 (secure 0 0 0) 0x7f0bcc0585b0 con 0x7f0be4161ee0 2024-04-25T10:07:13.039 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:13.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.035+0000 7f0bea800640 1 -- 172.21.15.119:0/3806661643 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0bc006b4d0 msgr2=0x7f0bc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.035+0000 7f0bea800640 1 --2- 172.21.15.119:0/3806661643 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0bc006b4d0 0x7f0bc006d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4034000 tx=0x7f0bd402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:13.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.035+0000 7f0bea800640 1 -- 172.21.15.119:0/3806661643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be4161ee0 msgr2=0x7f0be41f9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.035+0000 7f0bea800640 1 --2- 172.21.15.119:0/3806661643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be4161ee0 0x7f0be41f9290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00c430 tx=0x7f0bcc00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.035+0000 7f0bea800640 1 -- 172.21.15.119:0/3806661643 shutdown_connections 2024-04-25T10:07:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.035+0000 7f0bea800640 1 --2- 172.21.15.119:0/3806661643 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0bc006b4d0 0x7f0bc006d990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.035+0000 7f0bea800640 1 --2- 172.21.15.119:0/3806661643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0be41664d0 0x7f0be41f8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.035+0000 7f0bea800640 1 --2- 172.21.15.119:0/3806661643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0be4161ee0 0x7f0be41f9290 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.035+0000 7f0bea800640 1 --2- 172.21.15.119:0/3806661643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0be40b2570 0x7f0be4156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.035+0000 7f0bea800640 1 -- 172.21.15.119:0/3806661643 >> 172.21.15.119:0/3806661643 conn(0x7f0be406aba0 msgr2=0x7f0be40ce560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.036+0000 7f0bea800640 1 -- 172.21.15.119:0/3806661643 shutdown_connections 2024-04-25T10:07:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.036+0000 7f0bea800640 1 -- 172.21.15.119:0/3806661643 wait complete. 2024-04-25T10:07:13.061 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2600847164},{"type":"v1","addr":"172.21.15.119:6833","nonce":2600847164}]},"active_addr":"172.21.15.119:6833/2600847164","active_change":"2024-04-25T10:06:47.900634+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":2230283100}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":292667231}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":3585995603}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1615148544}]}]} 2024-04-25T10:07:13.065 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T10:07:13.065 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T10:07:13.065 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:13.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.251+0000 7fc608000640 1 Processor -- start 2024-04-25T10:07:13.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.251+0000 7fc608000640 1 -- start start 2024-04-25T10:07:13.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.252+0000 7fc608000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc60006e8f0 0x7fc60015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.253+0000 7fc608000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6000ce6c0 con 0x7fc60015bf00 2024-04-25T10:07:13.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.253+0000 7fc608000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6000cc4e0 con 0x7fc60015c820 2024-04-25T10:07:13.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.253+0000 7fc608000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6001455e0 con 0x7fc60006e8f0 2024-04-25T10:07:13.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.253+0000 7fc605600640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc60015bf00 0x7fc6000cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42094/0 (socket says 172.21.15.119:42094) 2024-04-25T10:07:13.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.253+0000 7fc605600640 1 -- 172.21.15.119:0/1295729097 learned_addr learned my addr 172.21.15.119:0/1295729097 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:13.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.253+0000 7fc604c00640 1 --2- 172.21.15.119:0/1295729097 >> v2:172.21.15.132:3300/0 conn(0x7fc60006e8f0 0x7fc60015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.253+0000 7fc5ffe00640 1 -- 172.21.15.119:0/1295729097 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185142425 0 0) 0x7fc5f4001110 con 0x7fc60015bf00 2024-04-25T10:07:13.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.253+0000 7fc5ffe00640 1 -- 172.21.15.119:0/1295729097 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5e8003520 con 0x7fc60015bf00 2024-04-25T10:07:13.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.254+0000 7fc5ffe00640 1 -- 172.21.15.119:0/1295729097 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 881473063 0 0) 0x7fc5e8003520 con 0x7fc60015bf00 2024-04-25T10:07:13.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.254+0000 7fc5ffe00640 1 -- 172.21.15.119:0/1295729097 >> v2:172.21.15.132:3300/0 conn(0x7fc60006e8f0 msgr2=0x7fc60015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.254+0000 7fc5ffe00640 1 --2- 172.21.15.119:0/1295729097 >> v2:172.21.15.132:3300/0 conn(0x7fc60006e8f0 0x7fc60015c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.254+0000 7fc5ffe00640 1 -- 172.21.15.119:0/1295729097 >> v1:172.21.15.132:6789/0 conn(0x7fc60015c820 legacy=0x7fc6001654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.254+0000 7fc5ffe00640 1 -- 172.21.15.119:0/1295729097 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6000cc7b0 con 0x7fc60015bf00 2024-04-25T10:07:13.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.254+0000 7fc5ffe00640 1 -- 172.21.15.119:0/1295729097 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fc5f4001840 con 0x7fc60015bf00 2024-04-25T10:07:13.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.254+0000 7fc5ffe00640 1 -- 172.21.15.119:0/1295729097 >> v1:172.21.15.119:6789/0 conn(0x7fc60015bf00 legacy=0x7fc6000cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.254+0000 7fc5ffe00640 1 --2- 172.21.15.119:0/1295729097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5e8003a00 0x7fc5e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.254+0000 7fc5ffe00640 1 --2- 172.21.15.119:0/1295729097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5e8004800 0x7fc5e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.254+0000 7fc5ffe00640 1 --2- 172.21.15.119:0/1295729097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5e80051c0 0x7fc5e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.254+0000 7fc5ffe00640 1 -- 172.21.15.119:0/1295729097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6001455e0 con 0x7fc5e80051c0 2024-04-25T10:07:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.254+0000 7fc5ffe00640 1 -- 172.21.15.119:0/1295729097 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e800c0c0 con 0x7fc5e8004800 2024-04-25T10:07:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.254+0000 7fc5ffe00640 1 -- 172.21.15.119:0/1295729097 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e800c2c0 con 0x7fc5e8003a00 2024-04-25T10:07:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.254+0000 7fc606000640 1 --2- 172.21.15.119:0/1295729097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5e80051c0 0x7fc5e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.255+0000 7fc605600640 1 --2- 172.21.15.119:0/1295729097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5e8003a00 0x7fc5e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.255+0000 7fc606000640 1 -- 172.21.15.119:0/1295729097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5e8003a00 msgr2=0x7fc5e8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.255+0000 7fc606000640 1 --2- 172.21.15.119:0/1295729097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5e8003a00 0x7fc5e8003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.255+0000 7fc606000640 1 -- 172.21.15.119:0/1295729097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5e8004800 msgr2=0x7fc5e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.255+0000 7fc606000640 1 --2- 172.21.15.119:0/1295729097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5e8004800 0x7fc5e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.255+0000 7fc606000640 1 -- 172.21.15.119:0/1295729097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e8003520 con 0x7fc5e80051c0 2024-04-25T10:07:13.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.256+0000 7fc606000640 1 -- 172.21.15.119:0/1295729097 reap_dead start 2024-04-25T10:07:13.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.256+0000 7fc606000640 1 --2- 172.21.15.119:0/1295729097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5e80051c0 0x7fc5e800b9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8001e50 tx=0x7fc5e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ad04fecb9360ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.256+0000 7fc5ffe00640 1 -- 172.21.15.119:0/1295729097 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8001a70 con 0x7fc5e80051c0 2024-04-25T10:07:13.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.256+0000 7fc5ffe00640 1 -- 172.21.15.119:0/1295729097 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f8003970 con 0x7fc5e80051c0 2024-04-25T10:07:13.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.256+0000 7fc5ffe00640 1 -- 172.21.15.119:0/1295729097 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8003d60 con 0x7fc5e80051c0 2024-04-25T10:07:13.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.257+0000 7fc608000640 1 -- 172.21.15.119:0/1295729097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5e80051c0 msgr2=0x7fc5e800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.257+0000 7fc608000640 1 --2- 172.21.15.119:0/1295729097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5e80051c0 0x7fc5e800b9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8001e50 tx=0x7fc5e8005830 comp rx=0 tx=0).stop 2024-04-25T10:07:13.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.257+0000 7fc608000640 1 -- 172.21.15.119:0/1295729097 shutdown_connections 2024-04-25T10:07:13.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.257+0000 7fc608000640 1 --2- 172.21.15.119:0/1295729097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5e80051c0 0x7fc5e800b9c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.257+0000 7fc608000640 1 -- 172.21.15.119:0/1295729097 >> 172.21.15.119:0/1295729097 conn(0x7fc60006aba0 msgr2=0x7fc60006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.257+0000 7fc608000640 1 -- 172.21.15.119:0/1295729097 shutdown_connections 2024-04-25T10:07:13.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.257+0000 7fc608000640 1 -- 172.21.15.119:0/1295729097 wait complete. 2024-04-25T10:07:13.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.258+0000 7fc608000640 1 Processor -- start 2024-04-25T10:07:13.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.259+0000 7fc608000640 1 -- start start 2024-04-25T10:07:13.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.259+0000 7fc608000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5e80051c0 0x7fc60013f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.259+0000 7fc608000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc600141090 0x7fc600138d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.259+0000 7fc608000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc600139290 0x7fc60013b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.259+0000 7fc608000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6000ccd70 con 0x7fc5e80051c0 2024-04-25T10:07:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.259+0000 7fc608000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60006d450 con 0x7fc600141090 2024-04-25T10:07:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.259+0000 7fc608000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6000cd050 con 0x7fc600139290 2024-04-25T10:07:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.260+0000 7fc605600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5e80051c0 0x7fc60013f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.260+0000 7fc605600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5e80051c0 0x7fc60013f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34488/0 (socket says 172.21.15.119:34488) 2024-04-25T10:07:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.260+0000 7fc605600640 1 -- 172.21.15.119:0/1533054173 learned_addr learned my addr 172.21.15.119:0/1533054173 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.260+0000 7fc605600640 1 -- 172.21.15.119:0/1533054173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc600139290 msgr2=0x7fc60013b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.260+0000 7fc605600640 1 --2- 172.21.15.119:0/1533054173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc600139290 0x7fc60013b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.260+0000 7fc605600640 1 -- 172.21.15.119:0/1533054173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc600141090 msgr2=0x7fc600138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.260+0000 7fc605600640 1 --2- 172.21.15.119:0/1533054173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc600141090 0x7fc600138d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.260+0000 7fc605600640 1 -- 172.21.15.119:0/1533054173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60013fff0 con 0x7fc5e80051c0 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.260+0000 7fc605600640 1 --2- 172.21.15.119:0/1533054173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5e80051c0 0x7fc60013f8f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4000ff0 tx=0x7fc5f400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.261+0000 7fc5fd600640 1 -- 172.21.15.119:0/1533054173 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4019070 con 0x7fc5e80051c0 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.261+0000 7fc5fd600640 1 -- 172.21.15.119:0/1533054173 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f40072d0 con 0x7fc5e80051c0 2024-04-25T10:07:13.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.261+0000 7fc5fd600640 1 -- 172.21.15.119:0/1533054173 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40047e0 con 0x7fc5e80051c0 2024-04-25T10:07:13.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.261+0000 7fc608000640 1 -- 172.21.15.119:0/1533054173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6001402d0 con 0x7fc5e80051c0 2024-04-25T10:07:13.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.261+0000 7fc608000640 1 -- 172.21.15.119:0/1533054173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc600140890 con 0x7fc5e80051c0 2024-04-25T10:07:13.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.262+0000 7fc608000640 1 -- 172.21.15.119:0/1533054173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60006bb30 con 0x7fc5e80051c0 2024-04-25T10:07:13.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.263+0000 7fc5fd600640 1 -- 172.21.15.119:0/1533054173 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5f401c070 con 0x7fc5e80051c0 2024-04-25T10:07:13.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.264+0000 7fc5fd600640 1 --2- 172.21.15.119:0/1533054173 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc5e006b500 0x7fc5e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.264+0000 7fc5fd600640 1 -- 172.21.15.119:0/1533054173 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5f4089a20 con 0x7fc5e80051c0 2024-04-25T10:07:13.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.264+0000 7fc604c00640 1 --2- 172.21.15.119:0/1533054173 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc5e006b500 0x7fc5e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.265+0000 7fc604c00640 1 --2- 172.21.15.119:0/1533054173 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc5e006b500 0x7fc5e006d9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec006a90 tx=0x7fc5ec006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.269+0000 7fc5fd600640 1 -- 172.21.15.119:0/1533054173 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6000ce6c0 con 0x7fc5e80051c0 2024-04-25T10:07:13.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.462+0000 7fc608000640 1 -- 172.21.15.119:0/1533054173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6000cdd30 con 0x7fc5e80051c0 2024-04-25T10:07:13.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.464+0000 7fc5fd600640 1 -- 172.21.15.119:0/1533054173 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fc6000cdd30 con 0x7fc5e80051c0 2024-04-25T10:07:13.470 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:13.470 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:08.319104+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:13.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.467+0000 7fc608000640 1 -- 172.21.15.119:0/1533054173 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc5e006b500 msgr2=0x7fc5e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.467+0000 7fc608000640 1 --2- 172.21.15.119:0/1533054173 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc5e006b500 0x7fc5e006d9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec006a90 tx=0x7fc5ec006a20 comp rx=0 tx=0).stop 2024-04-25T10:07:13.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.467+0000 7fc608000640 1 -- 172.21.15.119:0/1533054173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5e80051c0 msgr2=0x7fc60013f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.467+0000 7fc608000640 1 --2- 172.21.15.119:0/1533054173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5e80051c0 0x7fc60013f8f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4000ff0 tx=0x7fc5f400dc00 comp rx=0 tx=0).stop 2024-04-25T10:07:13.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.467+0000 7fc608000640 1 -- 172.21.15.119:0/1533054173 shutdown_connections 2024-04-25T10:07:13.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.467+0000 7fc608000640 1 --2- 172.21.15.119:0/1533054173 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc5e006b500 0x7fc5e006d9c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.467+0000 7fc608000640 1 --2- 172.21.15.119:0/1533054173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc600139290 0x7fc60013b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.467+0000 7fc608000640 1 --2- 172.21.15.119:0/1533054173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc600141090 0x7fc600138d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.467+0000 7fc608000640 1 --2- 172.21.15.119:0/1533054173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5e80051c0 0x7fc60013f8f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.467+0000 7fc608000640 1 -- 172.21.15.119:0/1533054173 >> 172.21.15.119:0/1533054173 conn(0x7fc60006aba0 msgr2=0x7fc600071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.467+0000 7fc608000640 1 -- 172.21.15.119:0/1533054173 shutdown_connections 2024-04-25T10:07:13.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.467+0000 7fc608000640 1 -- 172.21.15.119:0/1533054173 wait complete. 2024-04-25T10:07:13.491 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T10:07:13.491 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:13.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.676+0000 7fd66d800640 1 Processor -- start 2024-04-25T10:07:13.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.676+0000 7fd66d800640 1 -- start start 2024-04-25T10:07:13.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.677+0000 7fd66d800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd66806e8f0 0x7fd66815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.677+0000 7fd66d800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd66815c820 0x7fd6681654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.677+0000 7fd66d800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6680ce6c0 con 0x7fd66815bf00 2024-04-25T10:07:13.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.678+0000 7fd66d800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6681455e0 con 0x7fd66806e8f0 2024-04-25T10:07:13.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.678+0000 7fd66d800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd66806d450 con 0x7fd66815c820 2024-04-25T10:07:13.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.678+0000 7fd666000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd66806e8f0 0x7fd66815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.678+0000 7fd666000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd66806e8f0 0x7fd66815c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34494/0 (socket says 172.21.15.119:34494) 2024-04-25T10:07:13.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.678+0000 7fd666000640 1 -- 172.21.15.119:0/3924377298 learned_addr learned my addr 172.21.15.119:0/3924377298 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:13.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.678+0000 7fd666000640 1 -- 172.21.15.119:0/3924377298 >> v2:172.21.15.132:3300/0 conn(0x7fd66815c820 msgr2=0x7fd6681654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.678+0000 7fd666000640 1 --2- 172.21.15.119:0/3924377298 >> v2:172.21.15.132:3300/0 conn(0x7fd66815c820 0x7fd6681654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.678+0000 7fd666000640 1 -- 172.21.15.119:0/3924377298 >> v1:172.21.15.132:6789/0 conn(0x7fd66815bf00 legacy=0x7fd6680cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.678+0000 7fd666000640 1 -- 172.21.15.119:0/3924377298 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6680cc4e0 con 0x7fd66806e8f0 2024-04-25T10:07:13.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.679+0000 7fd666000640 1 --2- 172.21.15.119:0/3924377298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd66806e8f0 0x7fd66815c2e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd6680cdf50 tx=0x7fd65402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c523c00a47bf51f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.680+0000 7fd665600640 1 -- 172.21.15.119:0/3924377298 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654037070 con 0x7fd66806e8f0 2024-04-25T10:07:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.680+0000 7fd665600640 1 -- 172.21.15.119:0/3924377298 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd654032440 con 0x7fd66806e8f0 2024-04-25T10:07:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.680+0000 7fd665600640 1 -- 172.21.15.119:0/3924377298 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654032810 con 0x7fd66806e8f0 2024-04-25T10:07:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.680+0000 7fd66d800640 1 -- 172.21.15.119:0/3924377298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd66806e8f0 msgr2=0x7fd66815c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.680+0000 7fd66d800640 1 --2- 172.21.15.119:0/3924377298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd66806e8f0 0x7fd66815c2e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd6680cdf50 tx=0x7fd65402a230 comp rx=0 tx=0).stop 2024-04-25T10:07:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.680+0000 7fd66d800640 1 -- 172.21.15.119:0/3924377298 shutdown_connections 2024-04-25T10:07:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.680+0000 7fd66d800640 1 --2- 172.21.15.119:0/3924377298 >> v2:172.21.15.132:3300/0 conn(0x7fd66815c820 0x7fd6681654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.680+0000 7fd66d800640 1 --2- 172.21.15.119:0/3924377298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd66806e8f0 0x7fd66815c2e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.680+0000 7fd66d800640 1 -- 172.21.15.119:0/3924377298 >> 172.21.15.119:0/3924377298 conn(0x7fd66806aba0 msgr2=0x7fd66806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.681+0000 7fd66d800640 1 -- 172.21.15.119:0/3924377298 shutdown_connections 2024-04-25T10:07:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.681+0000 7fd66d800640 1 -- 172.21.15.119:0/3924377298 wait complete. 2024-04-25T10:07:13.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.682+0000 7fd66d800640 1 Processor -- start 2024-04-25T10:07:13.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.683+0000 7fd66d800640 1 -- start start 2024-04-25T10:07:13.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.684+0000 7fd66d800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd66806e8f0 0x7fd6681f0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.684+0000 7fd66d800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd66815bf00 0x7fd668141450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.684+0000 7fd66d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd66815c820 0x7fd6681f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.684+0000 7fd66d800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680ccd70 con 0x7fd66806e8f0 2024-04-25T10:07:13.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.684+0000 7fd66d800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680cca90 con 0x7fd66815bf00 2024-04-25T10:07:13.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.684+0000 7fd66d800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680cd050 con 0x7fd66815c820 2024-04-25T10:07:13.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.684+0000 7fd666a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd66806e8f0 0x7fd6681f0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.684+0000 7fd666a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd66806e8f0 0x7fd6681f0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34502/0 (socket says 172.21.15.119:34502) 2024-04-25T10:07:13.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.684+0000 7fd666a00640 1 -- 172.21.15.119:0/3394052316 learned_addr learned my addr 172.21.15.119:0/3394052316 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:13.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.684+0000 7fd666a00640 1 -- 172.21.15.119:0/3394052316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd66815c820 msgr2=0x7fd6681f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.684+0000 7fd666000640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd66815bf00 0x7fd668141450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.684+0000 7fd667400640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd66815c820 0x7fd6681f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.685+0000 7fd666a00640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd66815c820 0x7fd6681f6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.685+0000 7fd666a00640 1 -- 172.21.15.119:0/3394052316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd66815bf00 msgr2=0x7fd668141450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.685+0000 7fd666a00640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd66815bf00 0x7fd668141450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.685+0000 7fd666a00640 1 -- 172.21.15.119:0/3394052316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd668141e80 con 0x7fd66806e8f0 2024-04-25T10:07:13.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.685+0000 7fd666000640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd66815bf00 0x7fd668141450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:13.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.685+0000 7fd666a00640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd66806e8f0 0x7fd6681f0d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd65800c410 tx=0x7fd65800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.685+0000 7fd65ea00640 1 -- 172.21.15.119:0/3394052316 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658017070 con 0x7fd66806e8f0 2024-04-25T10:07:13.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.685+0000 7fd65ea00640 1 -- 172.21.15.119:0/3394052316 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6580035f0 con 0x7fd66806e8f0 2024-04-25T10:07:13.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.685+0000 7fd65ea00640 1 -- 172.21.15.119:0/3394052316 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658003a20 con 0x7fd66806e8f0 2024-04-25T10:07:13.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.685+0000 7fd66d800640 1 -- 172.21.15.119:0/3394052316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd668142160 con 0x7fd66806e8f0 2024-04-25T10:07:13.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.685+0000 7fd66d800640 1 -- 172.21.15.119:0/3394052316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66813d120 con 0x7fd66806e8f0 2024-04-25T10:07:13.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.686+0000 7fd66d800640 1 -- 172.21.15.119:0/3394052316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66806bb30 con 0x7fd66806e8f0 2024-04-25T10:07:13.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.687+0000 7fd65ea00640 1 -- 172.21.15.119:0/3394052316 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd658003bc0 con 0x7fd66806e8f0 2024-04-25T10:07:13.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.688+0000 7fd65ea00640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd64806b4d0 0x7fd64806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.688+0000 7fd65ea00640 1 -- 172.21.15.119:0/3394052316 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd658089510 con 0x7fd66806e8f0 2024-04-25T10:07:13.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.689+0000 7fd666000640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd64806b4d0 0x7fd64806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.689+0000 7fd666000640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd64806b4d0 0x7fd64806d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd654034000 tx=0x7fd65402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.693+0000 7fd65ea00640 1 -- 172.21.15.119:0/3394052316 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd658058390 con 0x7fd66806e8f0 2024-04-25T10:07:13.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.874+0000 7fd66d800640 1 -- 172.21.15.119:0/3394052316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd668071440 con 0x7fd66806e8f0 2024-04-25T10:07:13.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.876+0000 7fd65ea00640 1 -- 172.21.15.119:0/3394052316 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fd658057c90 con 0x7fd66806e8f0 2024-04-25T10:07:13.883 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:13.883 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:08.319104+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.879+0000 7fd66d800640 1 -- 172.21.15.119:0/3394052316 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd64806b4d0 msgr2=0x7fd64806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.879+0000 7fd66d800640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd64806b4d0 0x7fd64806d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd654034000 tx=0x7fd65402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.879+0000 7fd66d800640 1 -- 172.21.15.119:0/3394052316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd66806e8f0 msgr2=0x7fd6681f0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.879+0000 7fd66d800640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd66806e8f0 0x7fd6681f0d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd65800c410 tx=0x7fd65800c8d0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.879+0000 7fd66d800640 1 -- 172.21.15.119:0/3394052316 shutdown_connections 2024-04-25T10:07:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.879+0000 7fd66d800640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd64806b4d0 0x7fd64806d990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.879+0000 7fd66d800640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd66815c820 0x7fd6681f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.879+0000 7fd66d800640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd66815bf00 0x7fd668141450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.879+0000 7fd66d800640 1 --2- 172.21.15.119:0/3394052316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd66806e8f0 0x7fd6681f0d90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.879+0000 7fd66d800640 1 -- 172.21.15.119:0/3394052316 >> 172.21.15.119:0/3394052316 conn(0x7fd66806aba0 msgr2=0x7fd6680cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.879+0000 7fd66d800640 1 -- 172.21.15.119:0/3394052316 shutdown_connections 2024-04-25T10:07:13.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:13.880+0000 7fd66d800640 1 -- 172.21.15.119:0/3394052316 wait complete. 2024-04-25T10:07:13.904 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T10:07:13.905 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T10:07:13.905 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T10:07:13.905 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T10:07:13.906 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T10:07:13.907 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T10:07:13.907 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T10:07:13.907 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T10:07:14.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.180+0000 7f2517e00640 1 Processor -- start 2024-04-25T10:07:14.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.180+0000 7f2517e00640 1 -- start start 2024-04-25T10:07:14.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.181+0000 7f5adbe00640 1 Processor -- start 2024-04-25T10:07:14.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.182+0000 7f5adbe00640 1 -- start start 2024-04-25T10:07:14.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.182+0000 7f2517e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f251815b7e0 0x7f251815e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.182+0000 7f2517e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25180ce860 0x7f25180cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.182+0000 7f2517e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f25180d00a0 0x7f25180d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.183+0000 7f2516a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f251815b7e0 0x7f251815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.183+0000 7f2516000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25180ce860 0x7f25180cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.183+0000 7f2517e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25180d0a70 con 0x7f25180d00a0 2024-04-25T10:07:14.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.183+0000 7f2517e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25180b5710 con 0x7f251815b7e0 2024-04-25T10:07:14.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.183+0000 7f2517e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25180b59f0 con 0x7f25180ce860 2024-04-25T10:07:14.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.183+0000 7f2516000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25180ce860 0x7f25180cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49016/0 (socket says 172.21.15.119:49016) 2024-04-25T10:07:14.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.183+0000 7f2516a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f251815b7e0 0x7f251815e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:50634/0 (socket says 172.21.15.119:50634) 2024-04-25T10:07:14.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.183+0000 7f2516000640 1 -- 172.21.15.119:0/2975179243 learned_addr learned my addr 172.21.15.119:0/2975179243 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.183+0000 7f2517400640 1 --2- 172.21.15.119:0/2975179243 >> v2:172.21.15.119:3300/0 conn(0x7f25180d00a0 0x7f25180d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.184+0000 7f5adbe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5ad4005e90 0x7f5ad40062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.184+0000 7f5adbe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ad4006810 0x7f5ad4006c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.184+0000 7f5adbe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad4053720 con 0x7f5ad40ead00 2024-04-25T10:07:14.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.184+0000 7f5adbe00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ad4054330 con 0x7f5ad4006810 2024-04-25T10:07:14.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.184+0000 7f5adbe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ad4053a00 con 0x7f5ad4005e90 2024-04-25T10:07:14.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.184+0000 7f2517400640 1 -- 172.21.15.119:0/2975179243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25180ce860 msgr2=0x7f25180cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.184+0000 7f2517400640 1 --2- 172.21.15.119:0/2975179243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25180ce860 0x7f25180cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.184+0000 7f2517400640 1 -- 172.21.15.119:0/2975179243 >> v2:172.21.15.132:3300/0 conn(0x7f251815b7e0 msgr2=0x7f251815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.184+0000 7f2517400640 1 --2- 172.21.15.119:0/2975179243 >> v2:172.21.15.132:3300/0 conn(0x7f251815b7e0 0x7f251815e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.184+0000 7f2517400640 1 -- 172.21.15.119:0/2975179243 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25180b5cd0 con 0x7f25180d00a0 2024-04-25T10:07:14.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.185+0000 7f2cdbe00640 1 Processor -- start 2024-04-25T10:07:14.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.185+0000 7f5adb400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ad4006810 0x7f5ad4006c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.185+0000 7f5adb400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ad4006810 0x7f5ad4006c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34532/0 (socket says 172.21.15.119:34532) 2024-04-25T10:07:14.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.185+0000 7f5adb400640 1 -- 172.21.15.119:0/1824501411 learned_addr learned my addr 172.21.15.119:0/1824501411 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.186+0000 7f5adb400640 1 -- 172.21.15.119:0/1824501411 >> v2:172.21.15.132:3300/0 conn(0x7f5ad4005e90 msgr2=0x7f5ad40062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.186+0000 7f5adb400640 1 --2- 172.21.15.119:0/1824501411 >> v2:172.21.15.132:3300/0 conn(0x7f5ad4005e90 0x7f5ad40062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.186+0000 7f5adb400640 1 -- 172.21.15.119:0/1824501411 >> v1:172.21.15.132:6789/0 conn(0x7f5ad40ead00 legacy=0x7f5ad404fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.186+0000 7f5adb400640 1 -- 172.21.15.119:0/1824501411 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad4053ce0 con 0x7f5ad4006810 2024-04-25T10:07:14.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.187+0000 7f5adb400640 1 --2- 172.21.15.119:0/1824501411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad4006810 0x7f5ad4006c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5adc000b60 tx=0x7f5adc0a5430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee74312462d42cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.186+0000 7f2517400640 1 --2- 172.21.15.119:0/2975179243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25180d00a0 0x7f25180d0530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f251808bac0 tx=0x7f250802a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16055b621b4fe891 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.187+0000 7f2515600640 1 -- 172.21.15.119:0/2975179243 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508037070 con 0x7f25180d00a0 2024-04-25T10:07:14.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.187+0000 7f2515600640 1 -- 172.21.15.119:0/2975179243 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250802ad50 con 0x7f25180d00a0 2024-04-25T10:07:14.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.187+0000 7f2515600640 1 -- 172.21.15.119:0/2975179243 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508032660 con 0x7f25180d00a0 2024-04-25T10:07:14.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.188+0000 7f2517e00640 1 -- 172.21.15.119:0/2975179243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25180d00a0 msgr2=0x7f25180d0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.188+0000 7f2517e00640 1 --2- 172.21.15.119:0/2975179243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25180d00a0 0x7f25180d0530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f251808bac0 tx=0x7f250802a000 comp rx=0 tx=0).stop 2024-04-25T10:07:14.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.188+0000 7f5ad9600640 1 -- 172.21.15.119:0/1824501411 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0b1070 con 0x7f5ad4006810 2024-04-25T10:07:14.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.188+0000 7f5ad9600640 1 -- 172.21.15.119:0/1824501411 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc0ac600 con 0x7f5ad4006810 2024-04-25T10:07:14.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.188+0000 7f5ad9600640 1 -- 172.21.15.119:0/1824501411 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0b1070 con 0x7f5ad4006810 2024-04-25T10:07:14.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.185+0000 7f2cdbe00640 1 -- start start 2024-04-25T10:07:14.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.187+0000 7f2cdbe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2cd40df420 0x7f2cd4053260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.188+0000 7f2517e00640 1 -- 172.21.15.119:0/2975179243 shutdown_connections 2024-04-25T10:07:14.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.188+0000 7f2517e00640 1 --2- 172.21.15.119:0/2975179243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25180d00a0 0x7f25180d0530 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.188+0000 7f2517e00640 1 --2- 172.21.15.119:0/2975179243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25180ce860 0x7f25180cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.188+0000 7f2517e00640 1 --2- 172.21.15.119:0/2975179243 >> v2:172.21.15.132:3300/0 conn(0x7f251815b7e0 0x7f251815e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.188+0000 7f2517e00640 1 -- 172.21.15.119:0/2975179243 >> 172.21.15.119:0/2975179243 conn(0x7f25180722d0 msgr2=0x7f25180726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.187+0000 7f2cdbe00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd4006af0 con 0x7f2cd404fa90 2024-04-25T10:07:14.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.187+0000 7f2cdbe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd4006dd0 con 0x7f2cd40392d0 2024-04-25T10:07:14.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.187+0000 7f2cdbe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd40068a0 con 0x7f2cd40df420 2024-04-25T10:07:14.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.189+0000 7f2517e00640 1 -- 172.21.15.119:0/2975179243 shutdown_connections 2024-04-25T10:07:14.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.189+0000 7f2cda000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2cd40df420 0x7f2cd4053260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.189+0000 7f5adbe00640 1 -- 172.21.15.119:0/1824501411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad4006810 msgr2=0x7f5ad4006c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.189+0000 7f5adbe00640 1 --2- 172.21.15.119:0/1824501411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad4006810 0x7f5ad4006c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5adc000b60 tx=0x7f5adc0a5430 comp rx=0 tx=0).stop 2024-04-25T10:07:14.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.189+0000 7f2cda000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2cd40df420 0x7f2cd4053260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:50658/0 (socket says 172.21.15.119:50658) 2024-04-25T10:07:14.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.189+0000 7f2cda000640 1 -- 172.21.15.119:0/1354168399 learned_addr learned my addr 172.21.15.119:0/1354168399 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.189+0000 7f5adbe00640 1 -- 172.21.15.119:0/1824501411 shutdown_connections 2024-04-25T10:07:14.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.189+0000 7f5adbe00640 1 --2- 172.21.15.119:0/1824501411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad4006810 0x7f5ad4006c80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.189+0000 7f5adbe00640 1 --2- 172.21.15.119:0/1824501411 >> v2:172.21.15.132:3300/0 conn(0x7f5ad4005e90 0x7f5ad40062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.189+0000 7f5adbe00640 1 -- 172.21.15.119:0/1824501411 >> 172.21.15.119:0/1824501411 conn(0x7f5ad4009fb0 msgr2=0x7f5ad400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.190+0000 7f5adbe00640 1 -- 172.21.15.119:0/1824501411 shutdown_connections 2024-04-25T10:07:14.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.190+0000 7f5adbe00640 1 -- 172.21.15.119:0/1824501411 wait complete. 2024-04-25T10:07:14.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.189+0000 7f2517e00640 1 -- 172.21.15.119:0/2975179243 wait complete. 2024-04-25T10:07:14.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.191+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709388111 0 0) 0x7f2cd4006af0 con 0x7f2cd404fa90 2024-04-25T10:07:14.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.191+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cc0003520 con 0x7f2cd404fa90 2024-04-25T10:07:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452603754 0 0) 0x7f2cd4006dd0 con 0x7f2cd40392d0 2024-04-25T10:07:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd4006af0 con 0x7f2cd40392d0 2024-04-25T10:07:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 156314287 0 0) 0x7f2cc0003520 con 0x7f2cd404fa90 2024-04-25T10:07:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 >> v2:172.21.15.132:3300/0 conn(0x7f2cd40df420 msgr2=0x7f2cd4053260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 --2- 172.21.15.119:0/1354168399 >> v2:172.21.15.132:3300/0 conn(0x7f2cd40df420 0x7f2cd4053260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 >> v1:172.21.15.132:6789/0 conn(0x7f2cd40392d0 legacy=0x7f2cd40537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd40070b0 con 0x7f2cd404fa90 2024-04-25T10:07:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f2ccc001670 con 0x7f2cd404fa90 2024-04-25T10:07:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 >> v1:172.21.15.119:6789/0 conn(0x7f2cd404fa90 legacy=0x7f2cd404fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 --2- 172.21.15.119:0/1354168399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc0004d20 0x7f2cc00037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 --2- 172.21.15.119:0/1354168399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc0003d20 0x7f2cc00041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 --2- 172.21.15.119:0/1354168399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cc0005220 0x7f2cc00056d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd40068a0 con 0x7f2cc0004d20 2024-04-25T10:07:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc0005fb0 con 0x7f2cc0003d20 2024-04-25T10:07:14.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc00061b0 con 0x7f2cc0005220 2024-04-25T10:07:14.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.191+0000 7f2517e00640 1 Processor -- start 2024-04-25T10:07:14.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.193+0000 7f5adbe00640 1 Processor -- start 2024-04-25T10:07:14.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.192+0000 7f2517e00640 1 -- start start 2024-04-25T10:07:14.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.193+0000 7f5adbe00640 1 -- start start 2024-04-25T10:07:14.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.193+0000 7f2cda000640 1 --2- 172.21.15.119:0/1354168399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc0003d20 0x7f2cc00041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.194+0000 7f2cdaa00640 1 --2- 172.21.15.119:0/1354168399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc0004d20 0x7f2cc00037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.194+0000 7f2cdb400640 1 --2- 172.21.15.119:0/1354168399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cc0005220 0x7f2cc00056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.194+0000 7f5adbe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad4005e90 0x7f5ad40e0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.194+0000 7f5adbe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad4006810 0x7f5ad40dc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f5adbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad40ead00 0x7f5ad40dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f5adbe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad40547c0 con 0x7f5ad40ead00 2024-04-25T10:07:14.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f5adbe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad40501a0 con 0x7f5ad4005e90 2024-04-25T10:07:14.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f5adbe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad4054940 con 0x7f5ad4006810 2024-04-25T10:07:14.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f5adaa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad4005e90 0x7f5ad40e0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f5adb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad40ead00 0x7f5ad40dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f5adb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad40ead00 0x7f5ad40dcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34556/0 (socket says 172.21.15.119:34556) 2024-04-25T10:07:14.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f5adb400640 1 -- 172.21.15.119:0/3725187851 learned_addr learned my addr 172.21.15.119:0/3725187851 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f5ada000640 1 --2- 172.21.15.119:0/3725187851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad4006810 0x7f5ad40dc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f2cdaa00640 1 -- 172.21.15.119:0/1354168399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cc0005220 msgr2=0x7f2cc00056d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f2cdaa00640 1 --2- 172.21.15.119:0/1354168399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cc0005220 0x7f2cc00056d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f2cdaa00640 1 -- 172.21.15.119:0/1354168399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc0003d20 msgr2=0x7f2cc00041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f2cdaa00640 1 --2- 172.21.15.119:0/1354168399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc0003d20 0x7f2cc00041a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f2cdaa00640 1 -- 172.21.15.119:0/1354168399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd40070b0 con 0x7f2cc0004d20 2024-04-25T10:07:14.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.195+0000 7f2cdb400640 1 -- 172.21.15.119:0/1354168399 reap_dead start 2024-04-25T10:07:14.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.194+0000 7f2a13000640 1 Processor -- start 2024-04-25T10:07:14.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.194+0000 7f2a13000640 1 -- start start 2024-04-25T10:07:14.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.196+0000 7f3279a00640 1 Processor -- start 2024-04-25T10:07:14.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.196+0000 7f5adb400640 1 -- 172.21.15.119:0/3725187851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad4006810 msgr2=0x7f5ad40dc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.196+0000 7f5adb400640 1 --2- 172.21.15.119:0/3725187851 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ad4006810 0x7f5ad40dc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.196+0000 7f5adb400640 1 -- 172.21.15.119:0/3725187851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad4005e90 msgr2=0x7f5ad40e0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.196+0000 7f5adb400640 1 --2- 172.21.15.119:0/3725187851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad4005e90 0x7f5ad40e0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.196+0000 7f5adb400640 1 -- 172.21.15.119:0/3725187851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad4036ea0 con 0x7f5ad40ead00 2024-04-25T10:07:14.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.196+0000 7f3279a00640 1 -- start start 2024-04-25T10:07:14.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.196+0000 7f2a13000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2a0c161090 0x7f2a0c161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.196+0000 7f2a13000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2a0c0b9720 0x7f2a0c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.196+0000 7f2a13000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0c0cf0f0 con 0x7f2a0c0cece0 2024-04-25T10:07:14.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.196+0000 7f2a13000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0c0d21a0 con 0x7f2a0c161090 2024-04-25T10:07:14.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.196+0000 7f2a13000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0c161f80 con 0x7f2a0c0b9720 2024-04-25T10:07:14.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.196+0000 7f5adb400640 1 --2- 172.21.15.119:0/3725187851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad40ead00 0x7f5ad40dcda0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5adc069780 tx=0x7f5adc068980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.197+0000 7f2cdaa00640 1 --2- 172.21.15.119:0/1354168399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc0004d20 0x7f2cc00037a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00c5a0 tx=0x7f2cc0004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3acbcd734acb37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.197+0000 7f2517e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25180ce860 0x7f25181f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.197+0000 7f5ad2a00640 1 -- 172.21.15.119:0/3725187851 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0ba070 con 0x7f5ad40ead00 2024-04-25T10:07:14.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.197+0000 7f5ad2a00640 1 -- 172.21.15.119:0/3725187851 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc0acbc0 con 0x7f5ad40ead00 2024-04-25T10:07:14.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.197+0000 7f5ad2a00640 1 -- 172.21.15.119:0/3725187851 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0b1040 con 0x7f5ad40ead00 2024-04-25T10:07:14.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.197+0000 7f2a11000640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f2a0c0cece0 0x7f2a0c06c0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:41342/0 (socket says 172.21.15.119:41342) 2024-04-25T10:07:14.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.197+0000 7f2a11000640 1 -- 172.21.15.119:0/1260610456 learned_addr learned my addr 172.21.15.119:0/1260610456 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:14.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.197+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc033020 con 0x7f2cc0004d20 2024-04-25T10:07:14.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.197+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc007270 con 0x7f2cc0004d20 2024-04-25T10:07:14.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.197+0000 7f2cd9600640 1 -- 172.21.15.119:0/1354168399 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc007ca0 con 0x7f2cc0004d20 2024-04-25T10:07:14.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.197+0000 7f2a0b400640 1 --2- 172.21.15.119:0/1260610456 >> v2:172.21.15.132:3300/0 conn(0x7f2a0c0b9720 0x7f2a0c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.197+0000 7f2517e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25180d00a0 0x7f25180b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.197+0000 7f2517e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f251815b7e0 0x7f25180b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.197+0000 7f2a0be00640 1 --2- 172.21.15.119:0/1260610456 >> v2:172.21.15.119:3300/0 conn(0x7f2a0c161090 0x7f2a0c161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.198+0000 7f2a0aa00640 1 -- 172.21.15.119:0/1260610456 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000667804 0 0) 0x7f2a04001490 con 0x7f2a0c0cece0 2024-04-25T10:07:14.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.198+0000 7f2a0aa00640 1 -- 172.21.15.119:0/1260610456 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29f0003520 con 0x7f2a0c0cece0 2024-04-25T10:07:14.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.198+0000 7f2a0b400640 1 -- 172.21.15.119:0/1260610456 >> v2:172.21.15.119:3300/0 conn(0x7f2a0c161090 msgr2=0x7f2a0c161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.198+0000 7f2a0b400640 1 --2- 172.21.15.119:0/1260610456 >> v2:172.21.15.119:3300/0 conn(0x7f2a0c161090 0x7f2a0c161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.198+0000 7f2a0b400640 1 -- 172.21.15.119:0/1260610456 >> v1:172.21.15.132:6789/0 conn(0x7f2a0c0cece0 legacy=0x7f2a0c06c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.198+0000 7f5adbe00640 1 -- 172.21.15.119:0/3725187851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad40da870 con 0x7f5ad40ead00 2024-04-25T10:07:14.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.198+0000 7f5adbe00640 1 -- 172.21.15.119:0/3725187851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad40dae30 con 0x7f5ad40ead00 2024-04-25T10:07:14.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.198+0000 7f2517e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25180b53c0 con 0x7f25180d00a0 2024-04-25T10:07:14.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.199+0000 7f2516000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25180d00a0 0x7f25180b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.198+0000 7f2cdbe00640 1 -- 172.21.15.119:0/1354168399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc0004d20 msgr2=0x7f2cc00037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.198+0000 7f2cdbe00640 1 --2- 172.21.15.119:0/1354168399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc0004d20 0x7f2cc00037a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00c5a0 tx=0x7f2cc0004820 comp rx=0 tx=0).stop 2024-04-25T10:07:14.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.199+0000 7f2517e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251815e540 con 0x7f251815b7e0 2024-04-25T10:07:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.199+0000 7f2517e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2518161960 con 0x7f25180ce860 2024-04-25T10:07:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.199+0000 7f2517400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f251815b7e0 0x7f25180b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.199+0000 7f2516000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25180d00a0 0x7f25180b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34580/0 (socket says 172.21.15.119:34580) 2024-04-25T10:07:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.199+0000 7f2516000640 1 -- 172.21.15.119:0/441850779 learned_addr learned my addr 172.21.15.119:0/441850779 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.198+0000 7f2a0b400640 1 -- 172.21.15.119:0/1260610456 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c162260 con 0x7f2a0c0b9720 2024-04-25T10:07:14.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.199+0000 7f2cdbe00640 1 -- 172.21.15.119:0/1354168399 shutdown_connections 2024-04-25T10:07:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.199+0000 7f2cdbe00640 1 --2- 172.21.15.119:0/1354168399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc0004d20 0x7f2cc00037a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.199+0000 7f2cdbe00640 1 -- 172.21.15.119:0/1354168399 >> 172.21.15.119:0/1354168399 conn(0x7f2cd4009fb0 msgr2=0x7f2cd400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.199+0000 7f2cdbe00640 1 -- 172.21.15.119:0/1354168399 shutdown_connections 2024-04-25T10:07:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.199+0000 7f2516a00640 1 --2- 172.21.15.119:0/441850779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25180ce860 0x7f25181f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.200+0000 7f2517400640 1 -- 172.21.15.119:0/441850779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25180ce860 msgr2=0x7f25181f2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.200+0000 7f2517400640 1 --2- 172.21.15.119:0/441850779 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f25180ce860 0x7f25181f2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.200+0000 7f2517400640 1 -- 172.21.15.119:0/441850779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25180d00a0 msgr2=0x7f25180b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.200+0000 7f2517400640 1 --2- 172.21.15.119:0/441850779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25180d00a0 0x7f25180b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.200+0000 7f2517400640 1 -- 172.21.15.119:0/441850779 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25181f3b60 con 0x7f251815b7e0 2024-04-25T10:07:14.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.200+0000 7f5adbe00640 1 -- 172.21.15.119:0/3725187851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5ad4005390 con 0x7f5ad40ead00 2024-04-25T10:07:14.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.200+0000 7f3279a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f327406aba0 0x7f3274159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.200+0000 7f3279a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f327415a730 con 0x7f327406d920 2024-04-25T10:07:14.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.200+0000 7f3279a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f327415aa10 con 0x7f32740b3f70 2024-04-25T10:07:14.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.200+0000 7f3279a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f327406d120 con 0x7f327406aba0 2024-04-25T10:07:14.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.200+0000 7f2a0b400640 1 --2- 172.21.15.119:0/1260610456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a0c0b9720 0x7f2a0c0ce7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c08cf10 tx=0x7f29f802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2728a2db1d3fde2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.200+0000 7f2cdbe00640 1 -- 172.21.15.119:0/1354168399 wait complete. 2024-04-25T10:07:14.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.201+0000 7f3273e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f327406aba0 0x7f3274159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.201+0000 7f3273400640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f32740b3f70 0x7f3274153c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:41356/0 (socket says 172.21.15.119:41356) 2024-04-25T10:07:14.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.201+0000 7f3273400640 1 -- 172.21.15.119:0/3870476108 learned_addr learned my addr 172.21.15.119:0/3870476108 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:14.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.201+0000 7f2a0aa00640 1 -- 172.21.15.119:0/1260610456 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8037070 con 0x7f2a0c0b9720 2024-04-25T10:07:14.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.201+0000 7f2a0aa00640 1 -- 172.21.15.119:0/1260610456 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f8032440 con 0x7f2a0c0b9720 2024-04-25T10:07:14.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.202+0000 7f2a0aa00640 1 -- 172.21.15.119:0/1260610456 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8037070 con 0x7f2a0c0b9720 2024-04-25T10:07:14.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.201+0000 7f2517400640 1 --2- 172.21.15.119:0/441850779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f251815b7e0 0x7f25180b4490 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f251808c1e0 tx=0x7f25080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.202+0000 7f3273e00640 1 -- 172.21.15.119:0/3870476108 >> v1:172.21.15.132:6789/0 conn(0x7f32740b3f70 legacy=0x7f3274153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.202+0000 7f3273e00640 1 -- 172.21.15.119:0/3870476108 >> v1:172.21.15.119:6789/0 conn(0x7f327406d920 legacy=0x7f3274071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.202+0000 7f3273e00640 1 -- 172.21.15.119:0/3870476108 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327406d400 con 0x7f327406aba0 2024-04-25T10:07:14.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.202+0000 7f2cdbe00640 1 Processor -- start 2024-04-25T10:07:14.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.202+0000 7f250f400640 1 -- 172.21.15.119:0/441850779 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508040020 con 0x7f251815b7e0 2024-04-25T10:07:14.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.203+0000 7f250f400640 1 -- 172.21.15.119:0/441850779 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2508038040 con 0x7f251815b7e0 2024-04-25T10:07:14.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.203+0000 7f250f400640 1 -- 172.21.15.119:0/441850779 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508037070 con 0x7f251815b7e0 2024-04-25T10:07:14.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.202+0000 7f2cdbe00640 1 -- start start 2024-04-25T10:07:14.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.203+0000 7f2a13000640 1 -- 172.21.15.119:0/1260610456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a0c0b9720 msgr2=0x7f2a0c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.203+0000 7f2a13000640 1 --2- 172.21.15.119:0/1260610456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a0c0b9720 0x7f2a0c0ce7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c08cf10 tx=0x7f29f802a110 comp rx=0 tx=0).stop 2024-04-25T10:07:14.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.203+0000 7f2a13000640 1 -- 172.21.15.119:0/1260610456 shutdown_connections 2024-04-25T10:07:14.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.203+0000 7f2a13000640 1 --2- 172.21.15.119:0/1260610456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a0c0b9720 0x7f2a0c0ce7a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.203+0000 7f2a13000640 1 --2- 172.21.15.119:0/1260610456 >> v2:172.21.15.119:3300/0 conn(0x7f2a0c161090 0x7f2a0c161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.203+0000 7f2a13000640 1 -- 172.21.15.119:0/1260610456 >> 172.21.15.119:0/1260610456 conn(0x7f2a0c0b55e0 msgr2=0x7f2a0c0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.203+0000 7f2cdbe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc0004d20 0x7f2cd40d8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.204+0000 7f2517e00640 1 -- 172.21.15.119:0/441850779 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25181f35a0 con 0x7f251815b7e0 2024-04-25T10:07:14.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.204+0000 7f2cdbe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd404e6a0 0x7f2cd40e0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.204+0000 7f2cdbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd40e1e90 0x7f2cd4174a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.204+0000 7f2cdbe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4052cc0 con 0x7f2cd40e1e90 2024-04-25T10:07:14.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.204+0000 7f2cdbe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4052b40 con 0x7f2cc0004d20 2024-04-25T10:07:14.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.204+0000 7f2cdbe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd400aaa0 con 0x7f2cd404e6a0 2024-04-25T10:07:14.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.204+0000 7f2cdaa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc0004d20 0x7f2cd40d8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.204+0000 7f2517e00640 1 -- 172.21.15.119:0/441850779 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2518153be0 con 0x7f251815b7e0 2024-04-25T10:07:14.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.204+0000 7f2cdaa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc0004d20 0x7f2cd40d8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:50692/0 (socket says 172.21.15.119:50692) 2024-04-25T10:07:14.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.204+0000 7f2cdaa00640 1 -- 172.21.15.119:0/281717558 learned_addr learned my addr 172.21.15.119:0/281717558 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.204+0000 7f2cdb400640 1 --2- 172.21.15.119:0/281717558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd40e1e90 0x7f2cd4174a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.204+0000 7f2cda000640 1 --2- 172.21.15.119:0/281717558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd404e6a0 0x7f2cd40e0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.204+0000 7f5ad2a00640 1 -- 172.21.15.119:0/3725187851 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5adc0b4070 con 0x7f5ad40ead00 2024-04-25T10:07:14.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.205+0000 7f3273e00640 1 --2- 172.21.15.119:0/3870476108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327406aba0 0x7f3274159eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f326c009d90 tx=0x7f326c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527c3865e8c5dab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.205+0000 7f2cdaa00640 1 -- 172.21.15.119:0/281717558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd404e6a0 msgr2=0x7f2cd40e0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.205+0000 7f2517e00640 1 -- 172.21.15.119:0/441850779 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25180d0530 con 0x7f251815b7e0 2024-04-25T10:07:14.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.205+0000 7f2cdaa00640 1 --2- 172.21.15.119:0/281717558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cd404e6a0 0x7f2cd40e0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.205+0000 7f2cdaa00640 1 -- 172.21.15.119:0/281717558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd40e1e90 msgr2=0x7f2cd4174a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.205+0000 7f2cdaa00640 1 --2- 172.21.15.119:0/281717558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd40e1e90 0x7f2cd4174a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.205+0000 7f2a13000640 1 -- 172.21.15.119:0/1260610456 shutdown_connections 2024-04-25T10:07:14.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.205+0000 7f2cdaa00640 1 -- 172.21.15.119:0/281717558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd40d5810 con 0x7f2cc0004d20 2024-04-25T10:07:14.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.206+0000 7f3272a00640 1 -- 172.21.15.119:0/3870476108 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c007740 con 0x7f327406aba0 2024-04-25T10:07:14.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.206+0000 7f3272a00640 1 -- 172.21.15.119:0/3870476108 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c0078e0 con 0x7f327406aba0 2024-04-25T10:07:14.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.206+0000 7f3272a00640 1 -- 172.21.15.119:0/3870476108 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c007cb0 con 0x7f327406aba0 2024-04-25T10:07:14.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.206+0000 7f2a13000640 1 -- 172.21.15.119:0/1260610456 wait complete. 2024-04-25T10:07:14.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.206+0000 7f5ad2a00640 1 --2- 172.21.15.119:0/3725187851 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5abc06b520 0x7f5abc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.206+0000 7f2cdaa00640 1 --2- 172.21.15.119:0/281717558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc0004d20 0x7f2cd40d8530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc032040 tx=0x7f2ccc001970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.206+0000 7f5ad2a00640 1 -- 172.21.15.119:0/3725187851 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5adc125580 con 0x7f5ad40ead00 2024-04-25T10:07:14.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.207+0000 7f3279a00640 1 -- 172.21.15.119:0/3870476108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327406aba0 msgr2=0x7f3274159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.207+0000 7f3279a00640 1 --2- 172.21.15.119:0/3870476108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327406aba0 0x7f3274159eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f326c009d90 tx=0x7f326c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.207+0000 7f3279a00640 1 -- 172.21.15.119:0/3870476108 shutdown_connections 2024-04-25T10:07:14.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.207+0000 7f3279a00640 1 --2- 172.21.15.119:0/3870476108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327406aba0 0x7f3274159eb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.207+0000 7f3279a00640 1 -- 172.21.15.119:0/3870476108 >> 172.21.15.119:0/3870476108 conn(0x7f32740b1350 msgr2=0x7f32740b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.207+0000 7f5ad2a00640 1 --2- 172.21.15.119:0/3725187851 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x7f5abc073740 0x7f5abc075b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.207+0000 7f5ad2a00640 1 -- 172.21.15.119:0/3725187851 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5abc076210 con 0x7f5abc073740 2024-04-25T10:07:14.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.206+0000 7f2cd2a00640 1 -- 172.21.15.119:0/281717558 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc033040 con 0x7f2cc0004d20 2024-04-25T10:07:14.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.208+0000 7f3279a00640 1 -- 172.21.15.119:0/3870476108 shutdown_connections 2024-04-25T10:07:14.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.208+0000 7f3279a00640 1 -- 172.21.15.119:0/3870476108 wait complete. 2024-04-25T10:07:14.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.207+0000 7f2cd2a00640 1 -- 172.21.15.119:0/281717558 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc005270 con 0x7f2cc0004d20 2024-04-25T10:07:14.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.207+0000 7f2cd2a00640 1 -- 172.21.15.119:0/281717558 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0069d0 con 0x7f2cc0004d20 2024-04-25T10:07:14.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.209+0000 7f2cdbe00640 1 -- 172.21.15.119:0/281717558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd40e10d0 con 0x7f2cc0004d20 2024-04-25T10:07:14.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.209+0000 7f2cdbe00640 1 -- 172.21.15.119:0/281717558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd40e1690 con 0x7f2cc0004d20 2024-04-25T10:07:14.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.207+0000 7f5ad2a00640 1 -- 172.21.15.119:0/3725187851 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5adc125930 con 0x7f5ad40ead00 2024-04-25T10:07:14.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.208+0000 7f5adaa00640 1 --2- 172.21.15.119:0/3725187851 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5abc06b520 0x7f5abc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.209+0000 7f2a13000640 1 Processor -- start 2024-04-25T10:07:14.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.210+0000 7f250f400640 1 -- 172.21.15.119:0/441850779 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2508043020 con 0x7f251815b7e0 2024-04-25T10:07:14.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.209+0000 7f5ada000640 1 --2- 172.21.15.119:0/3725187851 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x7f5abc073740 0x7f5abc075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.210+0000 7f5adaa00640 1 --2- 172.21.15.119:0/3725187851 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5abc06b520 0x7f5abc06d9e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8006fd0 tx=0x7f5ac8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.210+0000 7f2a13000640 1 -- start start 2024-04-25T10:07:14.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.211+0000 7f2cdbe00640 1 -- 172.21.15.119:0/281717558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2cd4005390 con 0x7f2cc0004d20 2024-04-25T10:07:14.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.211+0000 7f3279a00640 1 Processor -- start 2024-04-25T10:07:14.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.212+0000 7f3279a00640 1 -- start start 2024-04-25T10:07:14.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.211+0000 7f2a13000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a0c0b9720 0x7f2a0c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.211+0000 7f2a13000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a0c0cece0 0x7f2a0c071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.211+0000 7f2a13000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a0c161090 0x7f2a0c06e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.210+0000 7f8e3fe00640 1 Processor -- start 2024-04-25T10:07:14.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.211+0000 7f8e3fe00640 1 -- start start 2024-04-25T10:07:14.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.212+0000 7f2a13000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c161470 con 0x7f2a0c0cece0 2024-04-25T10:07:14.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.212+0000 7f2a13000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0d1e30 con 0x7f2a0c0b9720 2024-04-25T10:07:14.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.212+0000 7f2a13000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c161c70 con 0x7f2a0c161090 2024-04-25T10:07:14.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.212+0000 7f2a0b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a0c0cece0 0x7f2a0c071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.211+0000 7f5ada000640 1 --2- 172.21.15.119:0/3725187851 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x7f5abc073740 0x7f5abc075b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f5ad2a00640 1 -- 172.21.15.119:0/3725187851 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5abc076210 con 0x7f5abc073740 2024-04-25T10:07:14.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.212+0000 7f2a0b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a0c0cece0 0x7f2a0c071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34614/0 (socket says 172.21.15.119:34614) 2024-04-25T10:07:14.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.212+0000 7f2a0b400640 1 -- 172.21.15.119:0/224425657 learned_addr learned my addr 172.21.15.119:0/224425657 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.212+0000 7f2a11000640 1 --2- 172.21.15.119:0/224425657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a0c161090 0x7f2a0c06e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.212+0000 7f2a0be00640 1 --2- 172.21.15.119:0/224425657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a0c0b9720 0x7f2a0c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f587f200640 1 Processor -- start 2024-04-25T10:07:14.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f2a0b400640 1 -- 172.21.15.119:0/224425657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a0c161090 msgr2=0x7f2a0c06e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f2a0b400640 1 --2- 172.21.15.119:0/224425657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a0c161090 0x7f2a0c06e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f2a0b400640 1 -- 172.21.15.119:0/224425657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a0c0b9720 msgr2=0x7f2a0c0cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f2a0b400640 1 --2- 172.21.15.119:0/224425657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a0c0b9720 0x7f2a0c0cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f2a0b400640 1 -- 172.21.15.119:0/224425657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c0708c0 con 0x7f2a0c0cece0 2024-04-25T10:07:14.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f587f200640 1 -- start start 2024-04-25T10:07:14.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.214+0000 7f2a0b400640 1 --2- 172.21.15.119:0/224425657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a0c0cece0 0x7f2a0c071660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f29f8035040 tx=0x7f29f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f8e3fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8e4015c330 0x7f8e4006b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f8e3fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e40162a70 0x7f8e400b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f8e3fe00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e401630b0 con 0x7f8e4006b710 2024-04-25T10:07:14.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f8e3fe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4006bb80 con 0x7f8e4015c330 2024-04-25T10:07:14.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f8e3fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e40162e80 con 0x7f8e40162a70 2024-04-25T10:07:14.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f8e3e000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8e4006b710 0x7f8e40162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42118/0 (socket says 172.21.15.119:42118) 2024-04-25T10:07:14.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f8e3e000640 1 -- 172.21.15.119:0/213289178 learned_addr learned my addr 172.21.15.119:0/213289178 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:14.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.213+0000 7f8e3f400640 1 --2- 172.21.15.119:0/213289178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e40162a70 0x7f8e400b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.214+0000 7f2cd2a00640 1 -- 172.21.15.119:0/281717558 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ccc036020 con 0x7f2cc0004d20 2024-04-25T10:07:14.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.214+0000 7f3279a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f327406aba0 0x7f327414d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.214+0000 7f3279a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f327406d920 0x7f3274158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.214+0000 7f3279a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32740b3f70 0x7f32741560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.214+0000 7f3279a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3274071760 con 0x7f32740b3f70 2024-04-25T10:07:14.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.214+0000 7f8e3f400640 1 -- 172.21.15.119:0/213289178 >> v2:172.21.15.132:3300/0 conn(0x7f8e4015c330 msgr2=0x7f8e4006b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.214+0000 7f8e3f400640 1 --2- 172.21.15.119:0/213289178 >> v2:172.21.15.132:3300/0 conn(0x7f8e4015c330 0x7f8e4006b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.214+0000 7f8e3f400640 1 -- 172.21.15.119:0/213289178 >> v1:172.21.15.119:6789/0 conn(0x7f8e4006b710 legacy=0x7f8e40162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.214+0000 7f8e3f400640 1 -- 172.21.15.119:0/213289178 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4006d950 con 0x7f8e40162a70 2024-04-25T10:07:14.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.214+0000 7f3279a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327415acf0 con 0x7f327406aba0 2024-04-25T10:07:14.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.214+0000 7f3279a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32740718e0 con 0x7f327406d920 2024-04-25T10:07:14.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.214+0000 7f250f400640 1 --2- 172.21.15.119:0/441850779 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f24f806b520 0x7f24f806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.215+0000 7f3273400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f327406d920 0x7f3274158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.215+0000 7f3273400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f327406d920 0x7f3274158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49094/0 (socket says 172.21.15.119:49094) 2024-04-25T10:07:14.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.215+0000 7f3273400640 1 -- 172.21.15.119:0/2934970748 learned_addr learned my addr 172.21.15.119:0/2934970748 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.215+0000 7f250f400640 1 -- 172.21.15.119:0/441850779 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25080aa410 con 0x7f251815b7e0 2024-04-25T10:07:14.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.215+0000 7f587f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f587404f860 0x7f587404fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.215+0000 7f587f200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5874037ba0 0x7f5874038030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.215+0000 7f587f200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5874053690 con 0x7f58740e32c0 2024-04-25T10:07:14.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.215+0000 7f587f200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58740e36a0 con 0x7f5874037ba0 2024-04-25T10:07:14.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.215+0000 7f587f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5874053970 con 0x7f587404f860 2024-04-25T10:07:14.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.215+0000 7f3273e00640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f327406aba0 0x7f327414d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.215+0000 7f2cd2a00640 1 --2- 172.21.15.119:0/281717558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2cbc06b4d0 0x7f2cbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f2cd2a00640 1 -- 172.21.15.119:0/281717558 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ccc0aadb0 con 0x7f2cc0004d20 2024-04-25T10:07:14.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f2a03e00640 1 -- 172.21.15.119:0/224425657 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8041070 con 0x7f2a0c0cece0 2024-04-25T10:07:14.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f2a03e00640 1 -- 172.21.15.119:0/224425657 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f803d630 con 0x7f2a0c0cece0 2024-04-25T10:07:14.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f2a13000640 1 -- 172.21.15.119:0/224425657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0c070b80 con 0x7f2a0c0cece0 2024-04-25T10:07:14.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.215+0000 7f250f400640 1 --2- 172.21.15.119:0/441850779 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x7f24f8073740 0x7f24f8075b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.215+0000 7f250f400640 1 -- 172.21.15.119:0/441850779 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25080067f0 con 0x7f24f8073740 2024-04-25T10:07:14.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.215+0000 7f250f400640 1 -- 172.21.15.119:0/441850779 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25080aa8f0 con 0x7f251815b7e0 2024-04-25T10:07:14.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f2516a00640 1 --2- 172.21.15.119:0/441850779 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f24f806b520 0x7f24f806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f2cd2a00640 1 --2- 172.21.15.119:0/281717558 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x7f2cbc073750 0x7f2cbc075b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f2cd2a00640 1 -- 172.21.15.119:0/281717558 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ccc009eb0 con 0x7f2cbc073750 2024-04-25T10:07:14.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f2cd2a00640 1 -- 172.21.15.119:0/281717558 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ccc0ab1a0 con 0x7f2cc0004d20 2024-04-25T10:07:14.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f587de00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f58740e32c0 0x7f5874052c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:41364/0 (socket says 172.21.15.119:41364) 2024-04-25T10:07:14.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f587de00640 1 -- 172.21.15.119:0/3250286028 learned_addr learned my addr 172.21.15.119:0/3250286028 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:14.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f587e800640 1 --2- 172.21.15.119:0/3250286028 >> v2:172.21.15.132:3300/0 conn(0x7f5874037ba0 0x7f5874038030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f2a03e00640 1 -- 172.21.15.119:0/224425657 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f802ae60 con 0x7f2a0c0cece0 2024-04-25T10:07:14.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f2a13000640 1 -- 172.21.15.119:0/224425657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0c071140 con 0x7f2a0c0cece0 2024-04-25T10:07:14.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f3279000640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32740b3f70 0x7f32741560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f3273400640 1 -- 172.21.15.119:0/2934970748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f327406aba0 msgr2=0x7f327414d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f3273400640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f327406aba0 0x7f327414d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f3273400640 1 -- 172.21.15.119:0/2934970748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32740b3f70 msgr2=0x7f32741560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f3273400640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32740b3f70 0x7f32741560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f3273400640 1 -- 172.21.15.119:0/2934970748 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3274154e30 con 0x7f327406d920 2024-04-25T10:07:14.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f587d400640 1 --2- 172.21.15.119:0/3250286028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f587404f860 0x7f587404fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f2cda000640 1 --2- 172.21.15.119:0/281717558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2cbc06b4d0 0x7f2cbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f2cdb400640 1 --2- 172.21.15.119:0/281717558 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x7f2cbc073750 0x7f2cbc075b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.216+0000 7f8e3f400640 1 --2- 172.21.15.119:0/213289178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e40162a70 0x7f8e400b3560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8e280097a0 tx=0x7f8e2802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef07615fdefa0d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.217+0000 7f8e3d600640 1 -- 172.21.15.119:0/213289178 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28037070 con 0x7f8e40162a70 2024-04-25T10:07:14.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.217+0000 7f2cdb400640 1 --2- 172.21.15.119:0/281717558 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x7f2cbc073750 0x7f2cbc075b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.217+0000 7f587ca00640 1 -- 172.21.15.119:0/3250286028 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179527867 0 0) 0x7f5874053690 con 0x7f58740e32c0 2024-04-25T10:07:14.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.217+0000 7f587ca00640 1 -- 172.21.15.119:0/3250286028 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5860003520 con 0x7f58740e32c0 2024-04-25T10:07:14.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.217+0000 7f8e3d600640 1 -- 172.21.15.119:0/213289178 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e28032440 con 0x7f8e40162a70 2024-04-25T10:07:14.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.217+0000 7f8e3d600640 1 -- 172.21.15.119:0/213289178 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28032810 con 0x7f8e40162a70 2024-04-25T10:07:14.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.218+0000 7f587e800640 1 -- 172.21.15.119:0/3250286028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f587404f860 msgr2=0x7f587404fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.218+0000 7f587e800640 1 --2- 172.21.15.119:0/3250286028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f587404f860 0x7f587404fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.218+0000 7f587e800640 1 -- 172.21.15.119:0/3250286028 >> v1:172.21.15.132:6789/0 conn(0x7f58740e32c0 legacy=0x7f5874052c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.217+0000 7f3273400640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f327406d920 0x7f3274158d80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f326400c4b0 tx=0x7f326400c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.218+0000 7f2516000640 1 --2- 172.21.15.119:0/441850779 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x7f24f8073740 0x7f24f8075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.218+0000 7f587e800640 1 -- 172.21.15.119:0/3250286028 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5874053c50 con 0x7f5874037ba0 2024-04-25T10:07:14.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.218+0000 7f587d400640 1 --2- 172.21.15.119:0/3250286028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f587404f860 0x7f587404fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:14.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.218+0000 7f3279000640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32740b3f70 0x7f32741560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:14.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.219+0000 7f2cda000640 1 --2- 172.21.15.119:0/281717558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2cbc06b4d0 0x7f2cbc06d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4033a40 tx=0x7f2cc8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.218+0000 7f8e3fe00640 1 -- 172.21.15.119:0/213289178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e40162a70 msgr2=0x7f8e400b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.218+0000 7f8e3fe00640 1 --2- 172.21.15.119:0/213289178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e40162a70 0x7f8e400b3560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8e280097a0 tx=0x7f8e2802a230 comp rx=0 tx=0).stop 2024-04-25T10:07:14.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.219+0000 7f326be00640 1 -- 172.21.15.119:0/2934970748 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264017070 con 0x7f327406d920 2024-04-25T10:07:14.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.219+0000 7f326be00640 1 -- 172.21.15.119:0/2934970748 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3264005ce0 con 0x7f327406d920 2024-04-25T10:07:14.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.219+0000 7f326be00640 1 -- 172.21.15.119:0/2934970748 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264007270 con 0x7f327406d920 2024-04-25T10:07:14.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.219+0000 7f2516000640 1 --2- 172.21.15.119:0/441850779 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x7f24f8073740 0x7f24f8075b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.219+0000 7f8e3fe00640 1 -- 172.21.15.119:0/213289178 shutdown_connections 2024-04-25T10:07:14.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.219+0000 7f8e3fe00640 1 --2- 172.21.15.119:0/213289178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e40162a70 0x7f8e400b3560 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.219+0000 7f8e3fe00640 1 --2- 172.21.15.119:0/213289178 >> v2:172.21.15.132:3300/0 conn(0x7f8e4015c330 0x7f8e4006b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.219+0000 7f8e3fe00640 1 -- 172.21.15.119:0/213289178 >> 172.21.15.119:0/213289178 conn(0x7f8e400b1350 msgr2=0x7f8e400b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.220+0000 7f2cd2a00640 1 -- 172.21.15.119:0/281717558 <== osd.2 v2:172.21.15.119:6824/1001762963 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2ccc009eb0 con 0x7f2cbc073750 2024-04-25T10:07:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.219+0000 7f3279a00640 1 -- 172.21.15.119:0/2934970748 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3274155110 con 0x7f327406d920 2024-04-25T10:07:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.219+0000 7f3279a00640 1 -- 172.21.15.119:0/2934970748 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3274156600 con 0x7f327406d920 2024-04-25T10:07:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.219+0000 7f587e800640 1 --2- 172.21.15.119:0/3250286028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5874037ba0 0x7f5874038030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f58680097a0 tx=0x7f586802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daa0337a2caa01a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.220+0000 7f587ca00640 1 -- 172.21.15.119:0/3250286028 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868037070 con 0x7f5874037ba0 2024-04-25T10:07:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.220+0000 7f587ca00640 1 -- 172.21.15.119:0/3250286028 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5868032440 con 0x7f5874037ba0 2024-04-25T10:07:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.220+0000 7f587ca00640 1 -- 172.21.15.119:0/3250286028 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58680327f0 con 0x7f5874037ba0 2024-04-25T10:07:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.221+0000 7f3279a00640 1 -- 172.21.15.119:0/2934970748 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f324c001030 con 0x7f327406d920 2024-04-25T10:07:14.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.219+0000 7f2a13000640 1 -- 172.21.15.119:0/224425657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f29e4001030 con 0x7f2a0c0cece0 2024-04-25T10:07:14.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.221+0000 7f2516a00640 1 --2- 172.21.15.119:0/441850779 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f24f806b520 0x7f24f806d9e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2504009aa0 tx=0x7f2504009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.221+0000 7f587f200640 1 -- 172.21.15.119:0/3250286028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5874037ba0 msgr2=0x7f5874038030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.221+0000 7f587f200640 1 --2- 172.21.15.119:0/3250286028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5874037ba0 0x7f5874038030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f58680097a0 tx=0x7f586802a210 comp rx=0 tx=0).stop 2024-04-25T10:07:14.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.221+0000 7f250f400640 1 -- 172.21.15.119:0/441850779 <== osd.0 v2:172.21.15.119:6808/2890625333 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f25080067f0 con 0x7f24f8073740 2024-04-25T10:07:14.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.221+0000 7f587f200640 1 -- 172.21.15.119:0/3250286028 shutdown_connections 2024-04-25T10:07:14.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.221+0000 7f587f200640 1 --2- 172.21.15.119:0/3250286028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5874037ba0 0x7f5874038030 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.221+0000 7f587f200640 1 --2- 172.21.15.119:0/3250286028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f587404f860 0x7f587404fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.221+0000 7f587f200640 1 -- 172.21.15.119:0/3250286028 >> 172.21.15.119:0/3250286028 conn(0x7f5874009fa0 msgr2=0x7f587400a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.221+0000 7f587f200640 1 -- 172.21.15.119:0/3250286028 shutdown_connections 2024-04-25T10:07:14.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.222+0000 7f8e3fe00640 1 -- 172.21.15.119:0/213289178 shutdown_connections 2024-04-25T10:07:14.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.223+0000 7f2a03e00640 1 -- 172.21.15.119:0/224425657 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29f803da00 con 0x7f2a0c0cece0 2024-04-25T10:07:14.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.224+0000 7f5d36200640 1 Processor -- start 2024-04-25T10:07:14.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.224+0000 7f5d36200640 1 -- start start 2024-04-25T10:07:14.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.222+0000 7f587f200640 1 -- 172.21.15.119:0/3250286028 wait complete. 2024-04-25T10:07:14.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.224+0000 7f587f200640 1 Processor -- start 2024-04-25T10:07:14.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.224+0000 7f587f200640 1 -- start start 2024-04-25T10:07:14.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.226+0000 7f8e3fe00640 1 -- 172.21.15.119:0/213289178 wait complete. 2024-04-25T10:07:14.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.226+0000 7f5d36200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5d300ceeb0 0x7f5d30168a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.226+0000 7f587f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5874037ba0 0x7f5874185a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.226+0000 7f587f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f587404f860 0x7f58741887a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.226+0000 7f5d36200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d30161f80 con 0x7f5d30161090 2024-04-25T10:07:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.223+0000 7f326be00640 1 -- 172.21.15.119:0/2934970748 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f326401a070 con 0x7f327406d920 2024-04-25T10:07:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.224+0000 7f326be00640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f325406b520 0x7f325406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.224+0000 7f326be00640 1 -- 172.21.15.119:0/2934970748 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3264089a80 con 0x7f327406d920 2024-04-25T10:07:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.225+0000 7f326be00640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x7f3254073740 0x7f3254075b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.225+0000 7f326be00640 1 -- 172.21.15.119:0/2934970748 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3264015e50 con 0x7f3254073740 2024-04-25T10:07:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.225+0000 7f326be00640 1 -- 172.21.15.119:0/2934970748 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3264089f60 con 0x7f327406d920 2024-04-25T10:07:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.225+0000 7f3279000640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x7f3254073740 0x7f3254075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.227+0000 7f5d36200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d30162260 con 0x7f5d300b9720 2024-04-25T10:07:14.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.227+0000 7f5d36200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d300d21a0 con 0x7f5d300ceeb0 2024-04-25T10:07:14.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.227+0000 7f587f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58740e32c0 0x7f587404dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.227+0000 7f587f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5874053350 con 0x7f587404f860 2024-04-25T10:07:14.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.226+0000 7f3279000640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x7f3254073740 0x7f3254075b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.227+0000 7f587f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5874054210 con 0x7f58740e32c0 2024-04-25T10:07:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.227+0000 7f587f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5874054730 con 0x7f5874037ba0 2024-04-25T10:07:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.227+0000 7f587e800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58740e32c0 0x7f587404dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.228+0000 7f587de00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5874037ba0 0x7f5874185a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.228+0000 7f587e800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58740e32c0 0x7f587404dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:50768/0 (socket says 172.21.15.119:50768) 2024-04-25T10:07:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.228+0000 7f587e800640 1 -- 172.21.15.119:0/4035815116 learned_addr learned my addr 172.21.15.119:0/4035815116 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.227+0000 7f326be00640 1 -- 172.21.15.119:0/2934970748 <== osd.3 v2:172.21.15.119:6816/1142971786 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3264015e50 con 0x7f3254073740 2024-04-25T10:07:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.227+0000 7f2a03e00640 1 --2- 172.21.15.119:0/224425657 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f29ec06b5a0 0x7f29ec06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.228+0000 7f2a03e00640 1 -- 172.21.15.119:0/224425657 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29f80aa630 con 0x7f2a0c0cece0 2024-04-25T10:07:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.228+0000 7f587d400640 1 --2- 172.21.15.119:0/4035815116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f587404f860 0x7f58741887a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.228+0000 7f587e800640 1 -- 172.21.15.119:0/4035815116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5874037ba0 msgr2=0x7f5874185a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.228+0000 7f587e800640 1 --2- 172.21.15.119:0/4035815116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5874037ba0 0x7f5874185a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.228+0000 7f587e800640 1 -- 172.21.15.119:0/4035815116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f587404f860 msgr2=0x7f58741887a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.229+0000 7f587e800640 1 --2- 172.21.15.119:0/4035815116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f587404f860 0x7f58741887a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.229+0000 7f587e800640 1 -- 172.21.15.119:0/4035815116 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58740df680 con 0x7f58740e32c0 2024-04-25T10:07:14.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.229+0000 7f3273e00640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f325406b520 0x7f325406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.229+0000 7f587d400640 1 --2- 172.21.15.119:0/4035815116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f587404f860 0x7f58741887a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.229+0000 7f2a03e00640 1 --2- 172.21.15.119:0/224425657 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x7f29ec0737c0 0x7f29ec075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.229+0000 7f2a03e00640 1 -- 172.21.15.119:0/224425657 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29f80067f0 con 0x7f29ec0737c0 2024-04-25T10:07:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.229+0000 7f2a03e00640 1 -- 172.21.15.119:0/224425657 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29f80b1050 con 0x7f2a0c0cece0 2024-04-25T10:07:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.229+0000 7f2a0be00640 1 --2- 172.21.15.119:0/224425657 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f29ec06b5a0 0x7f29ec06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.229+0000 7f587e800640 1 --2- 172.21.15.119:0/4035815116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58740e32c0 0x7f587404dcc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5868035040 tx=0x7f58680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.227+0000 7f5d2fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5d300ceeb0 0x7f5d30168a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.227+0000 7f5d2fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5d300ceeb0 0x7f5d30168a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:50756/0 (socket says 172.21.15.119:50756) 2024-04-25T10:07:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.227+0000 7f5d2fe00640 1 -- 172.21.15.119:0/3518770797 learned_addr learned my addr 172.21.15.119:0/3518770797 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.228+0000 7f8e3fe00640 1 Processor -- start 2024-04-25T10:07:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.228+0000 7f5d2fe00640 1 -- 172.21.15.119:0/3518770797 >> v1:172.21.15.132:6789/0 conn(0x7f5d300b9720 legacy=0x7f5d300ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.228+0000 7f5d2e000640 1 -- 172.21.15.119:0/3518770797 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972547237 0 0) 0x7f5d30161f80 con 0x7f5d30161090 2024-04-25T10:07:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.228+0000 7f5d2fe00640 1 -- 172.21.15.119:0/3518770797 >> v1:172.21.15.119:6789/0 conn(0x7f5d30161090 legacy=0x7f5d30161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.228+0000 7f5d2fe00640 1 -- 172.21.15.119:0/3518770797 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d30162540 con 0x7f5d300ceeb0 2024-04-25T10:07:14.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.230+0000 7f8e3fe00640 1 -- start start 2024-04-25T10:07:14.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.230+0000 7f5d2fe00640 1 --2- 172.21.15.119:0/3518770797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d300ceeb0 0x7f5d30168a60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d300d2320 tx=0x7f5d2802a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc96064b6899d6ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.230+0000 7f5872000640 1 -- 172.21.15.119:0/4035815116 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868037040 con 0x7f58740e32c0 2024-04-25T10:07:14.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.230+0000 7f5872000640 1 -- 172.21.15.119:0/4035815116 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5868042070 con 0x7f58740e32c0 2024-04-25T10:07:14.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.230+0000 7f5872000640 1 -- 172.21.15.119:0/4035815116 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58680044e0 con 0x7f58740e32c0 2024-04-25T10:07:14.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.230+0000 7f3273e00640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f325406b520 0x7f325406d9e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f326c034000 tx=0x7f326c02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.230+0000 7f587f200640 1 -- 172.21.15.119:0/4035815116 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58740e4050 con 0x7f58740e32c0 2024-04-25T10:07:14.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.230+0000 7f5d2e000640 1 -- 172.21.15.119:0/3518770797 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28036070 con 0x7f5d300ceeb0 2024-04-25T10:07:14.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.230+0000 7f587f200640 1 -- 172.21.15.119:0/4035815116 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58740e4610 con 0x7f58740e32c0 2024-04-25T10:07:14.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.230+0000 7f5d2e000640 1 -- 172.21.15.119:0/3518770797 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d28031600 con 0x7f5d300ceeb0 2024-04-25T10:07:14.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.230+0000 7f5d2e000640 1 -- 172.21.15.119:0/3518770797 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28036070 con 0x7f5d300ceeb0 2024-04-25T10:07:14.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.231+0000 7f2a11000640 1 --2- 172.21.15.119:0/224425657 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x7f29ec0737c0 0x7f29ec075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.231+0000 7f8e3fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e4006b710 0x7f8e401533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.231+0000 7f8e3fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e4015c330 0x7f8e401538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.231+0000 7f8e3fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e40162a70 0x7f8e401eb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.231+0000 7f8e3fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4006df10 con 0x7f8e4015c330 2024-04-25T10:07:14.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.231+0000 7f8e3fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4015c710 con 0x7f8e40162a70 2024-04-25T10:07:14.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.231+0000 7f8e3fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4006e1f0 con 0x7f8e4006b710 2024-04-25T10:07:14.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.232+0000 7f8e3e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e4015c330 0x7f8e401538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.232+0000 7f2a11000640 1 --2- 172.21.15.119:0/224425657 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x7f29ec0737c0 0x7f29ec075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.232+0000 7f587f200640 1 -- 172.21.15.119:0/4035815116 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5854001030 con 0x7f58740e32c0 2024-04-25T10:07:14.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.232+0000 7f8e3ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e4006b710 0x7f8e401533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.232+0000 7f8e3ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e4006b710 0x7f8e401533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49122/0 (socket says 172.21.15.119:49122) 2024-04-25T10:07:14.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.232+0000 7f8e3ea00640 1 -- 172.21.15.119:0/1729368690 learned_addr learned my addr 172.21.15.119:0/1729368690 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.232+0000 7f8e3f400640 1 --2- 172.21.15.119:0/1729368690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e40162a70 0x7f8e401eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.233+0000 7f2a0be00640 1 --2- 172.21.15.119:0/224425657 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f29ec06b5a0 0x7f29ec06da60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009a30 tx=0x7f29fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.233+0000 7f5d36200640 1 -- 172.21.15.119:0/3518770797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d300ceeb0 msgr2=0x7f5d30168a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.233+0000 7f5d36200640 1 --2- 172.21.15.119:0/3518770797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d300ceeb0 0x7f5d30168a60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d300d2320 tx=0x7f5d2802a730 comp rx=0 tx=0).stop 2024-04-25T10:07:14.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.233+0000 7f8e3e000640 1 -- 172.21.15.119:0/1729368690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e4006b710 msgr2=0x7f8e401533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.233+0000 7f8e3e000640 1 --2- 172.21.15.119:0/1729368690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e4006b710 0x7f8e401533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.233+0000 7f8e3e000640 1 -- 172.21.15.119:0/1729368690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e40162a70 msgr2=0x7f8e401eb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.233+0000 7f8e3e000640 1 --2- 172.21.15.119:0/1729368690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e40162a70 0x7f8e401eb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.233+0000 7f8e3e000640 1 -- 172.21.15.119:0/1729368690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e40165360 con 0x7f8e4015c330 2024-04-25T10:07:14.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.233+0000 7f5d36200640 1 -- 172.21.15.119:0/3518770797 shutdown_connections 2024-04-25T10:07:14.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.233+0000 7f5d36200640 1 --2- 172.21.15.119:0/3518770797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d300ceeb0 0x7f5d30168a60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.233+0000 7f5d36200640 1 -- 172.21.15.119:0/3518770797 >> 172.21.15.119:0/3518770797 conn(0x7f5d300b55e0 msgr2=0x7f5d300b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.233+0000 7f5d36200640 1 -- 172.21.15.119:0/3518770797 shutdown_connections 2024-04-25T10:07:14.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.233+0000 7f5d36200640 1 -- 172.21.15.119:0/3518770797 wait complete. 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.234+0000 7f8e3e000640 1 --2- 172.21.15.119:0/1729368690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e4015c330 0x7f8e401538f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000cb10 tx=0x7f8e3000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.234+0000 7f8e37400640 1 -- 172.21.15.119:0/1729368690 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30017070 con 0x7f8e4015c330 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.234+0000 7f8e37400640 1 -- 172.21.15.119:0/1729368690 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3000ac70 con 0x7f8e4015c330 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.234+0000 7f8e37400640 1 -- 172.21.15.119:0/1729368690 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30017070 con 0x7f8e4015c330 2024-04-25T10:07:14.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.234+0000 7f8e3fe00640 1 -- 172.21.15.119:0/1729368690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e40163590 con 0x7f8e4015c330 2024-04-25T10:07:14.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.235+0000 7f8e3fe00640 1 -- 172.21.15.119:0/1729368690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e40163b50 con 0x7f8e4015c330 2024-04-25T10:07:14.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.235+0000 7f2a03e00640 1 -- 172.21.15.119:0/224425657 <== osd.4 v2:172.21.15.132:6800/1740646323 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f29f80067f0 con 0x7f29ec0737c0 2024-04-25T10:07:14.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.237+0000 7f5d36200640 1 Processor -- start 2024-04-25T10:07:14.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.237+0000 7f5d36200640 1 -- start start 2024-04-25T10:07:14.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.235+0000 7f5872000640 1 -- 172.21.15.119:0/4035815116 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58680327f0 con 0x7f58740e32c0 2024-04-25T10:07:14.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.238+0000 7f8e3fe00640 1 -- 172.21.15.119:0/1729368690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e400749b0 con 0x7f8e4015c330 2024-04-25T10:07:14.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.238+0000 7f5872000640 1 --2- 172.21.15.119:0/4035815116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f585c06b5a0 0x7f585c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.238+0000 7f5872000640 1 -- 172.21.15.119:0/4035815116 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58680ab2a0 con 0x7f58740e32c0 2024-04-25T10:07:14.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.239+0000 7f8e37400640 1 -- 172.21.15.119:0/1729368690 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e3001a070 con 0x7f8e4015c330 2024-04-25T10:07:14.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.239+0000 7f5d36200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d300b9720 0x7f5d301f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.239+0000 7f5d36200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d300ceeb0 0x7f5d3015fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.239+0000 7f5d36200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d30161090 0x7f5d30155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.239+0000 7f5872000640 1 --2- 172.21.15.119:0/4035815116 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x7f585c073820 0x7f585c075c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.239+0000 7f5872000640 1 -- 172.21.15.119:0/4035815116 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5868032440 con 0x7f585c073820 2024-04-25T10:07:14.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.239+0000 7f5872000640 1 -- 172.21.15.119:0/4035815116 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58680ab690 con 0x7f58740e32c0 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.239+0000 7f5d36200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d300d3640 con 0x7f5d300ceeb0 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.239+0000 7f5d36200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d30161470 con 0x7f5d30161090 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.239+0000 7f5d36200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d300d37c0 con 0x7f5d300b9720 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.240+0000 7f5d2f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d300b9720 0x7f5d301f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.240+0000 7f5d2fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d30161090 0x7f5d30155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.240+0000 7f5d2fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d30161090 0x7f5d30155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:50776/0 (socket says 172.21.15.119:50776) 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.240+0000 7f5d2fe00640 1 -- 172.21.15.119:0/2762626517 learned_addr learned my addr 172.21.15.119:0/2762626517 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.240+0000 7f5d2ea00640 1 --2- 172.21.15.119:0/2762626517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d300ceeb0 0x7f5d3015fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.240+0000 7f587d400640 1 --2- 172.21.15.119:0/4035815116 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x7f585c073820 0x7f585c075c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.240+0000 7f8e37400640 1 --2- 172.21.15.119:0/1729368690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8e1c06b700 0x7f8e1c06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.240+0000 7f5d2fe00640 1 -- 172.21.15.119:0/2762626517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d300b9720 msgr2=0x7f5d301f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.240+0000 7f5d2fe00640 1 --2- 172.21.15.119:0/2762626517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d300b9720 0x7f5d301f4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.241+0000 7f5d2fe00640 1 -- 172.21.15.119:0/2762626517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d300ceeb0 msgr2=0x7f5d3015fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.241+0000 7f5d2fe00640 1 --2- 172.21.15.119:0/2762626517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d300ceeb0 0x7f5d3015fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.241+0000 7f5d2fe00640 1 -- 172.21.15.119:0/2762626517 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d30156be0 con 0x7f5d30161090 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.241+0000 7f8e37400640 1 -- 172.21.15.119:0/1729368690 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e30056360 con 0x7f8e4015c330 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.241+0000 7f8e37400640 1 --2- 172.21.15.119:0/1729368690 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x7f8e1c074570 0x7f8e1c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.241+0000 7f8e37400640 1 -- 172.21.15.119:0/1729368690 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e1c0770e0 con 0x7f8e1c074570 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.241+0000 7f8e37400640 1 -- 172.21.15.119:0/1729368690 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e30054cf0 con 0x7f8e4015c330 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.241+0000 7f5d2ea00640 1 --2- 172.21.15.119:0/2762626517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d300ceeb0 0x7f5d3015fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:14.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.241+0000 7f587de00640 1 --2- 172.21.15.119:0/4035815116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f585c06b5a0 0x7f585c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.241+0000 7f5d2fe00640 1 --2- 172.21.15.119:0/2762626517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d30161090 0x7f5d30155e70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009f40 tx=0x7f5d28004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.242+0000 7f8e3ea00640 1 --2- 172.21.15.119:0/1729368690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8e1c06b700 0x7f8e1c06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.242+0000 7f5d27400640 1 -- 172.21.15.119:0/2762626517 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28036050 con 0x7f5d30161090 2024-04-25T10:07:14.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.242+0000 7f5d27400640 1 -- 172.21.15.119:0/2762626517 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d28037040 con 0x7f5d30161090 2024-04-25T10:07:14.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.242+0000 7f5d36200640 1 -- 172.21.15.119:0/2762626517 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3015cf60 con 0x7f5d30161090 2024-04-25T10:07:14.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.242+0000 7f5d36200640 1 -- 172.21.15.119:0/2762626517 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3015d7a0 con 0x7f5d30161090 2024-04-25T10:07:14.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.242+0000 7f8e3f400640 1 --2- 172.21.15.119:0/1729368690 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x7f8e1c074570 0x7f8e1c076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.242+0000 7f5d27400640 1 -- 172.21.15.119:0/2762626517 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28009510 con 0x7f5d30161090 2024-04-25T10:07:14.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.242+0000 7f587d400640 1 --2- 172.21.15.119:0/4035815116 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x7f585c073820 0x7f585c075c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.243+0000 7f587de00640 1 --2- 172.21.15.119:0/4035815116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f585c06b5a0 0x7f585c06da60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5878050190 tx=0x7f5878073fc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.243+0000 7f8e3f400640 1 --2- 172.21.15.119:0/1729368690 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x7f8e1c074570 0x7f8e1c076a10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.244+0000 7f5d36200640 1 -- 172.21.15.119:0/2762626517 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d301f4fd0 con 0x7f5d30161090 2024-04-25T10:07:14.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.245+0000 7f5872000640 1 -- 172.21.15.119:0/4035815116 <== osd.5 v2:172.21.15.132:6815/2978152964 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5868032440 con 0x7f585c073820 2024-04-25T10:07:14.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.245+0000 7f5adbe00640 1 -- 172.21.15.119:0/3725187851 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5ad40e0b60 con 0x7f5abc073740 2024-04-25T10:07:14.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.246+0000 7f8e3ea00640 1 --2- 172.21.15.119:0/1729368690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8e1c06b700 0x7f8e1c06dbc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009a40 tx=0x7f8e380091c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.246+0000 7f8e37400640 1 -- 172.21.15.119:0/1729368690 <== osd.6 v2:172.21.15.132:6806/353653608 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8e1c0770e0 con 0x7f8e1c074570 2024-04-25T10:07:14.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.247+0000 7f5ad2a00640 1 -- 172.21.15.119:0/3725187851 <== osd.1 v2:172.21.15.119:6800/298545098 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5ad40e0b60 con 0x7f5abc073740 2024-04-25T10:07:14.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.247+0000 7f5d27400640 1 -- 172.21.15.119:0/2762626517 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d28044030 con 0x7f5d30161090 2024-04-25T10:07:14.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.249+0000 7f5d27400640 1 --2- 172.21.15.119:0/2762626517 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5d0c06b5a0 0x7f5d0c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.249+0000 7f2517e00640 1 -- 172.21.15.119:0/441850779 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f25180749b0 con 0x7f24f8073740 2024-04-25T10:07:14.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.248+0000 7f5adbe00640 1 -- 172.21.15.119:0/3725187851 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x7f5abc073740 msgr2=0x7f5abc075b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.249+0000 7f5d27400640 1 -- 172.21.15.119:0/2762626517 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d280aaac0 con 0x7f5d30161090 2024-04-25T10:07:14.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.248+0000 7f5adbe00640 1 --2- 172.21.15.119:0/3725187851 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x7f5abc073740 0x7f5abc075b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.248+0000 7f5adbe00640 1 -- 172.21.15.119:0/3725187851 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5abc06b520 msgr2=0x7f5abc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.248+0000 7f5adbe00640 1 --2- 172.21.15.119:0/3725187851 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5abc06b520 0x7f5abc06d9e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8006fd0 tx=0x7f5ac8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:14.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.249+0000 7f5adbe00640 1 -- 172.21.15.119:0/3725187851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad40ead00 msgr2=0x7f5ad40dcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.249+0000 7f5adbe00640 1 --2- 172.21.15.119:0/3725187851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad40ead00 0x7f5ad40dcda0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5adc069780 tx=0x7f5adc068980 comp rx=0 tx=0).stop 2024-04-25T10:07:14.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.249+0000 7f5adb400640 1 -- 172.21.15.119:0/3725187851 reap_dead start 2024-04-25T10:07:14.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.249+0000 7f5adbe00640 1 -- 172.21.15.119:0/3725187851 shutdown_connections 2024-04-25T10:07:14.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.249+0000 7f5adbe00640 1 -- 172.21.15.119:0/3725187851 >> 172.21.15.119:0/3725187851 conn(0x7f5ad4009fb0 msgr2=0x7f5ad4049b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.249+0000 7f5d27400640 1 --2- 172.21.15.119:0/2762626517 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x7f5d0c073820 0x7f5d0c075c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.249+0000 7f5d27400640 1 -- 172.21.15.119:0/2762626517 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d0c0762f0 con 0x7f5d0c073820 2024-04-25T10:07:14.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.250+0000 7f5d2f400640 1 --2- 172.21.15.119:0/2762626517 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5d0c06b5a0 0x7f5d0c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.250+0000 7f5d27400640 1 -- 172.21.15.119:0/2762626517 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d280aaeb0 con 0x7f5d30161090 2024-04-25T10:07:14.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.249+0000 7f5adbe00640 1 -- 172.21.15.119:0/3725187851 shutdown_connections 2024-04-25T10:07:14.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.250+0000 7f5adbe00640 1 -- 172.21.15.119:0/3725187851 wait complete. 2024-04-25T10:07:14.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.250+0000 7f5d2ea00640 1 --2- 172.21.15.119:0/2762626517 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x7f5d0c073820 0x7f5d0c075c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.251+0000 7f250f400640 1 -- 172.21.15.119:0/441850779 <== osd.0 v2:172.21.15.119:6808/2890625333 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f25180749b0 con 0x7f24f8073740 2024-04-25T10:07:14.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.251+0000 7f5d2f400640 1 --2- 172.21.15.119:0/2762626517 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5d0c06b5a0 0x7f5d0c06da60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d20000f60 tx=0x7f5d20009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.251+0000 7f2517e00640 1 -- 172.21.15.119:0/441850779 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x7f24f8073740 msgr2=0x7f24f8075b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.251+0000 7f2517e00640 1 --2- 172.21.15.119:0/441850779 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x7f24f8073740 0x7f24f8075b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.252+0000 7f5d2ea00640 1 --2- 172.21.15.119:0/2762626517 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x7f5d0c073820 0x7f5d0c075c40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.255+0000 7f2517e00640 1 -- 172.21.15.119:0/441850779 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f24f806b520 msgr2=0x7f24f806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.255+0000 7f2517e00640 1 --2- 172.21.15.119:0/441850779 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f24f806b520 0x7f24f806d9e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2504009aa0 tx=0x7f2504009190 comp rx=0 tx=0).stop 2024-04-25T10:07:14.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.255+0000 7f2517e00640 1 -- 172.21.15.119:0/441850779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f251815b7e0 msgr2=0x7f25180b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.255+0000 7f2517e00640 1 --2- 172.21.15.119:0/441850779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f251815b7e0 0x7f25180b4490 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f251808c1e0 tx=0x7f25080046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.255+0000 7f2517400640 1 -- 172.21.15.119:0/441850779 reap_dead start 2024-04-25T10:07:14.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.255+0000 7f2517e00640 1 -- 172.21.15.119:0/441850779 shutdown_connections 2024-04-25T10:07:14.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.255+0000 7f2517e00640 1 -- 172.21.15.119:0/441850779 >> 172.21.15.119:0/441850779 conn(0x7f25180722d0 msgr2=0x7f2518155f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.256+0000 7f2517e00640 1 -- 172.21.15.119:0/441850779 shutdown_connections 2024-04-25T10:07:14.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.256+0000 7f2517e00640 1 -- 172.21.15.119:0/441850779 wait complete. 2024-04-25T10:07:14.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.257+0000 7f2cdbe00640 1 -- 172.21.15.119:0/281717558 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2cd400a6c0 con 0x7f2cbc073750 2024-04-25T10:07:14.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.257+0000 7f2cd2a00640 1 -- 172.21.15.119:0/281717558 <== osd.2 v2:172.21.15.119:6824/1001762963 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2cd400a6c0 con 0x7f2cbc073750 2024-04-25T10:07:14.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.257+0000 7f5d27400640 1 -- 172.21.15.119:0/2762626517 <== osd.7 v2:172.21.15.132:6821/2727385616 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5d0c0762f0 con 0x7f5d0c073820 2024-04-25T10:07:14.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.258+0000 7f2cdbe00640 1 -- 172.21.15.119:0/281717558 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x7f2cbc073750 msgr2=0x7f2cbc075b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.258+0000 7f2cdbe00640 1 --2- 172.21.15.119:0/281717558 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x7f2cbc073750 0x7f2cbc075b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.259+0000 7f2cdbe00640 1 -- 172.21.15.119:0/281717558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2cbc06b4d0 msgr2=0x7f2cbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.259+0000 7f2cdbe00640 1 --2- 172.21.15.119:0/281717558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2cbc06b4d0 0x7f2cbc06d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4033a40 tx=0x7f2cc8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:14.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.259+0000 7f2cdbe00640 1 -- 172.21.15.119:0/281717558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc0004d20 msgr2=0x7f2cd40d8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.259+0000 7f2cdbe00640 1 --2- 172.21.15.119:0/281717558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc0004d20 0x7f2cd40d8530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc032040 tx=0x7f2ccc001970 comp rx=0 tx=0).stop 2024-04-25T10:07:14.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.259+0000 7f2cdb400640 1 -- 172.21.15.119:0/281717558 reap_dead start 2024-04-25T10:07:14.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.259+0000 7f2cdbe00640 1 -- 172.21.15.119:0/281717558 shutdown_connections 2024-04-25T10:07:14.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.259+0000 7f2cdbe00640 1 -- 172.21.15.119:0/281717558 >> 172.21.15.119:0/281717558 conn(0x7f2cd4009fb0 msgr2=0x7f2cd404f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.261+0000 7f2cdbe00640 1 -- 172.21.15.119:0/281717558 shutdown_connections 2024-04-25T10:07:14.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.262+0000 7f2cdbe00640 1 -- 172.21.15.119:0/281717558 wait complete. 2024-04-25T10:07:14.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.268+0000 7f3279a00640 1 -- 172.21.15.119:0/2934970748 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f324c002dd0 con 0x7f3254073740 2024-04-25T10:07:14.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.269+0000 7f326be00640 1 -- 172.21.15.119:0/2934970748 <== osd.3 v2:172.21.15.119:6816/1142971786 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f324c002dd0 con 0x7f3254073740 2024-04-25T10:07:14.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.270+0000 7f3279a00640 1 -- 172.21.15.119:0/2934970748 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x7f3254073740 msgr2=0x7f3254075b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.270+0000 7f3279a00640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x7f3254073740 0x7f3254075b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.270+0000 7f2a13000640 1 -- 172.21.15.119:0/224425657 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f29e4002dd0 con 0x7f29ec0737c0 2024-04-25T10:07:14.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.271+0000 7f2a03e00640 1 -- 172.21.15.119:0/224425657 <== osd.4 v2:172.21.15.132:6800/1740646323 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f29e4002dd0 con 0x7f29ec0737c0 2024-04-25T10:07:14.295 INFO:teuthology.orchestra.run.smithi119.stdout:55834574855 2024-04-25T10:07:14.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f3279a00640 1 -- 172.21.15.119:0/2934970748 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f325406b520 msgr2=0x7f325406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f3279a00640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f325406b520 0x7f325406d9e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f326c034000 tx=0x7f326c02f000 comp rx=0 tx=0).stop 2024-04-25T10:07:14.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f3279a00640 1 -- 172.21.15.119:0/2934970748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f327406d920 msgr2=0x7f3274158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f3279a00640 1 --2- 172.21.15.119:0/2934970748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f327406d920 0x7f3274158d80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f326400c4b0 tx=0x7f326400c970 comp rx=0 tx=0).stop 2024-04-25T10:07:14.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f3279000640 1 -- 172.21.15.119:0/2934970748 reap_dead start 2024-04-25T10:07:14.296 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:07:14.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f3279a00640 1 -- 172.21.15.119:0/2934970748 shutdown_connections 2024-04-25T10:07:14.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f3279a00640 1 -- 172.21.15.119:0/2934970748 >> 172.21.15.119:0/2934970748 conn(0x7f32740b1350 msgr2=0x7f32740b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f2a13000640 1 -- 172.21.15.119:0/224425657 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x7f29ec0737c0 msgr2=0x7f29ec075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f2a13000640 1 --2- 172.21.15.119:0/224425657 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x7f29ec0737c0 0x7f29ec075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f3279a00640 1 -- 172.21.15.119:0/2934970748 shutdown_connections 2024-04-25T10:07:14.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f3279a00640 1 -- 172.21.15.119:0/2934970748 wait complete. 2024-04-25T10:07:14.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f2a13000640 1 -- 172.21.15.119:0/224425657 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f29ec06b5a0 msgr2=0x7f29ec06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f2a13000640 1 --2- 172.21.15.119:0/224425657 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f29ec06b5a0 0x7f29ec06da60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009a30 tx=0x7f29fc008040 comp rx=0 tx=0).stop 2024-04-25T10:07:14.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f2a13000640 1 -- 172.21.15.119:0/224425657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a0c0cece0 msgr2=0x7f2a0c071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.272+0000 7f2a13000640 1 --2- 172.21.15.119:0/224425657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a0c0cece0 0x7f2a0c071660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f29f8035040 tx=0x7f29f80046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.273+0000 7f2a11000640 1 -- 172.21.15.119:0/224425657 reap_dead start 2024-04-25T10:07:14.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.273+0000 7f2a13000640 1 -- 172.21.15.119:0/224425657 shutdown_connections 2024-04-25T10:07:14.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.273+0000 7f2a13000640 1 -- 172.21.15.119:0/224425657 >> 172.21.15.119:0/224425657 conn(0x7f2a0c0b55e0 msgr2=0x7f2a0c15b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.273+0000 7f2a13000640 1 -- 172.21.15.119:0/224425657 shutdown_connections 2024-04-25T10:07:14.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.273+0000 7f2a13000640 1 -- 172.21.15.119:0/224425657 wait complete. 2024-04-25T10:07:14.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.276+0000 7f587f200640 1 -- 172.21.15.119:0/4035815116 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5854002dd0 con 0x7f585c073820 2024-04-25T10:07:14.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.277+0000 7f5872000640 1 -- 172.21.15.119:0/4035815116 <== osd.5 v2:172.21.15.132:6815/2978152964 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5854002dd0 con 0x7f585c073820 2024-04-25T10:07:14.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.279+0000 7f587f200640 1 -- 172.21.15.119:0/4035815116 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x7f585c073820 msgr2=0x7f585c075c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.279+0000 7f587f200640 1 --2- 172.21.15.119:0/4035815116 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x7f585c073820 0x7f585c075c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.279+0000 7f587f200640 1 -- 172.21.15.119:0/4035815116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f585c06b5a0 msgr2=0x7f585c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.279+0000 7f587f200640 1 --2- 172.21.15.119:0/4035815116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f585c06b5a0 0x7f585c06da60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5878050190 tx=0x7f5878073fc0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.279+0000 7f587f200640 1 -- 172.21.15.119:0/4035815116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58740e32c0 msgr2=0x7f587404dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.279+0000 7f587f200640 1 --2- 172.21.15.119:0/4035815116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58740e32c0 0x7f587404dcc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5868035040 tx=0x7f58680046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.279+0000 7f587e800640 1 -- 172.21.15.119:0/4035815116 reap_dead start 2024-04-25T10:07:14.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.280+0000 7f587f200640 1 -- 172.21.15.119:0/4035815116 shutdown_connections 2024-04-25T10:07:14.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.280+0000 7f587f200640 1 -- 172.21.15.119:0/4035815116 >> 172.21.15.119:0/4035815116 conn(0x7f5874009fa0 msgr2=0x7f5874052c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.280+0000 7f587f200640 1 -- 172.21.15.119:0/4035815116 shutdown_connections 2024-04-25T10:07:14.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.280+0000 7f587f200640 1 -- 172.21.15.119:0/4035815116 wait complete. 2024-04-25T10:07:14.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.281+0000 7f8e3fe00640 1 -- 172.21.15.119:0/1729368690 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8e401ebe90 con 0x7f8e1c074570 2024-04-25T10:07:14.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.282+0000 7f8e37400640 1 -- 172.21.15.119:0/1729368690 <== osd.6 v2:172.21.15.132:6806/353653608 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8e401ebe90 con 0x7f8e1c074570 2024-04-25T10:07:14.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.283+0000 7f8e3fe00640 1 -- 172.21.15.119:0/1729368690 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x7f8e1c074570 msgr2=0x7f8e1c076a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.283+0000 7f8e3fe00640 1 --2- 172.21.15.119:0/1729368690 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x7f8e1c074570 0x7f8e1c076a10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.303 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2024-04-25T10:07:14.303 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:07:14.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.283+0000 7f8e3fe00640 1 -- 172.21.15.119:0/1729368690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8e1c06b700 msgr2=0x7f8e1c06dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.283+0000 7f8e3fe00640 1 --2- 172.21.15.119:0/1729368690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8e1c06b700 0x7f8e1c06dbc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009a40 tx=0x7f8e380091c0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.283+0000 7f8e3fe00640 1 -- 172.21.15.119:0/1729368690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e4015c330 msgr2=0x7f8e401538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.283+0000 7f8e3fe00640 1 --2- 172.21.15.119:0/1729368690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e4015c330 0x7f8e401538f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000cb10 tx=0x7f8e3000a060 comp rx=0 tx=0).stop 2024-04-25T10:07:14.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.283+0000 7f8e3f400640 1 -- 172.21.15.119:0/1729368690 reap_dead start 2024-04-25T10:07:14.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.283+0000 7f8e3fe00640 1 -- 172.21.15.119:0/1729368690 shutdown_connections 2024-04-25T10:07:14.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.283+0000 7f8e3fe00640 1 -- 172.21.15.119:0/1729368690 >> 172.21.15.119:0/1729368690 conn(0x7f8e400b1350 msgr2=0x7f8e40160ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.284+0000 7f8e3fe00640 1 -- 172.21.15.119:0/1729368690 shutdown_connections 2024-04-25T10:07:14.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.284+0000 7f8e3fe00640 1 -- 172.21.15.119:0/1729368690 wait complete. 2024-04-25T10:07:14.305 INFO:teuthology.orchestra.run.smithi119.stdout:55834574855 2024-04-25T10:07:14.306 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:07:14.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.290+0000 7f5d36200640 1 -- 172.21.15.119:0/2762626517 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5d300750b0 con 0x7f5d0c073820 2024-04-25T10:07:14.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.291+0000 7f5d27400640 1 -- 172.21.15.119:0/2762626517 <== osd.7 v2:172.21.15.132:6821/2727385616 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5d300750b0 con 0x7f5d0c073820 2024-04-25T10:07:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.292+0000 7f5d36200640 1 -- 172.21.15.119:0/2762626517 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x7f5d0c073820 msgr2=0x7f5d0c075c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.292+0000 7f5d36200640 1 --2- 172.21.15.119:0/2762626517 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x7f5d0c073820 0x7f5d0c075c40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.292+0000 7f5d36200640 1 -- 172.21.15.119:0/2762626517 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5d0c06b5a0 msgr2=0x7f5d0c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.292+0000 7f5d36200640 1 --2- 172.21.15.119:0/2762626517 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5d0c06b5a0 0x7f5d0c06da60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d20000f60 tx=0x7f5d20009040 comp rx=0 tx=0).stop 2024-04-25T10:07:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.292+0000 7f5d36200640 1 -- 172.21.15.119:0/2762626517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d30161090 msgr2=0x7f5d30155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.292+0000 7f5d36200640 1 --2- 172.21.15.119:0/2762626517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d30161090 0x7f5d30155e70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009f40 tx=0x7f5d28004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.292+0000 7f5d2fe00640 1 -- 172.21.15.119:0/2762626517 reap_dead start 2024-04-25T10:07:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.293+0000 7f5d36200640 1 -- 172.21.15.119:0/2762626517 shutdown_connections 2024-04-25T10:07:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.293+0000 7f5d36200640 1 -- 172.21.15.119:0/2762626517 >> 172.21.15.119:0/2762626517 conn(0x7f5d300b55e0 msgr2=0x7f5d300b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.293+0000 7f5d36200640 1 -- 172.21.15.119:0/2762626517 shutdown_connections 2024-04-25T10:07:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.293+0000 7f5d36200640 1 -- 172.21.15.119:0/2762626517 wait complete. 2024-04-25T10:07:14.309 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2024-04-25T10:07:14.309 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:07:14.310 INFO:teuthology.orchestra.run.smithi119.stdout:55834574855 2024-04-25T10:07:14.310 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:07:14.311 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2024-04-25T10:07:14.312 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:07:14.313 INFO:teuthology.orchestra.run.smithi119.stdout:55834574855 2024-04-25T10:07:14.313 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:07:14.320 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2024-04-25T10:07:14.320 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:07:14.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.567+0000 7f11e7400640 1 Processor -- start 2024-04-25T10:07:14.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.568+0000 7f11e7400640 1 -- start start 2024-04-25T10:07:14.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.570+0000 7f11e7400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e0159910 0x7f11e015e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.570+0000 7f11e7400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e0071bd0 con 0x7f11e00cef80 2024-04-25T10:07:14.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.570+0000 7f11e7400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e00718f0 con 0x7f11e00b1740 2024-04-25T10:07:14.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.570+0000 7f11e7400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e015d5d0 con 0x7f11e0159910 2024-04-25T10:07:14.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.571+0000 7f11e4a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e0159910 0x7f11e015e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.571+0000 7f11e4a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e0159910 0x7f11e015e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49128/0 (socket says 172.21.15.119:49128) 2024-04-25T10:07:14.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.571+0000 7f11dfe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f11e00cef80 0x7f11e0071050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42126/0 (socket says 172.21.15.119:42126) 2024-04-25T10:07:14.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.571+0000 7f11dfe00640 1 -- 172.21.15.119:0/1672622193 learned_addr learned my addr 172.21.15.119:0/1672622193 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:14.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.572+0000 7f84ba200640 1 Processor -- start 2024-04-25T10:07:14.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.573+0000 7f84ba200640 1 -- start start 2024-04-25T10:07:14.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.573+0000 7f11df400640 1 -- 172.21.15.119:0/1672622193 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760323831 0 0) 0x7f11e0071bd0 con 0x7f11e00cef80 2024-04-25T10:07:14.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.573+0000 7f11df400640 1 -- 172.21.15.119:0/1672622193 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11c4003520 con 0x7f11e00cef80 2024-04-25T10:07:14.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.573+0000 7f11e4a00640 1 -- 172.21.15.119:0/1672622193 >> v1:172.21.15.132:6789/0 conn(0x7f11e00b1740 legacy=0x7f11e0159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.573+0000 7f11e4a00640 1 -- 172.21.15.119:0/1672622193 >> v1:172.21.15.119:6789/0 conn(0x7f11e00cef80 legacy=0x7f11e0071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.573+0000 7f11e4a00640 1 -- 172.21.15.119:0/1672622193 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e015d8b0 con 0x7f11e0159910 2024-04-25T10:07:14.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.574+0000 7f84ba200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84ac0e1090 0x7f84ac038630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.575+0000 7f84ba200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f84ac03a5d0 0x7f84ac005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.575+0000 7f84ba200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ac04f540 con 0x7f84ac050dd0 2024-04-25T10:07:14.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.575+0000 7f84ba200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84ac04f820 con 0x7f84ac03a5d0 2024-04-25T10:07:14.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.575+0000 7f84ba200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac04fb00 con 0x7f84ac0e1090 2024-04-25T10:07:14.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.575+0000 7f84b8e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84ac0e1090 0x7f84ac038630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.575+0000 7f84b8e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84ac0e1090 0x7f84ac038630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49132/0 (socket says 172.21.15.119:49132) 2024-04-25T10:07:14.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.575+0000 7f84b8e00640 1 -- 172.21.15.119:0/1999933372 learned_addr learned my addr 172.21.15.119:0/1999933372 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.575+0000 7f11e4a00640 1 --2- 172.21.15.119:0/1672622193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e0159910 0x7f11e015e0b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00ba70 tx=0x7f11cc02e840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d247b7717bed45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.576+0000 7f84b9800640 1 --2- 172.21.15.119:0/1999933372 >> v2:172.21.15.132:3300/0 conn(0x7f84ac03a5d0 0x7f84ac005f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.576+0000 7f11df400640 1 -- 172.21.15.119:0/1672622193 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc03d070 con 0x7f11e0159910 2024-04-25T10:07:14.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.576+0000 7f84b8e00640 1 -- 172.21.15.119:0/1999933372 >> v2:172.21.15.132:3300/0 conn(0x7f84ac03a5d0 msgr2=0x7f84ac005f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.576+0000 7f84b8e00640 1 --2- 172.21.15.119:0/1999933372 >> v2:172.21.15.132:3300/0 conn(0x7f84ac03a5d0 0x7f84ac005f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.576+0000 7f84b8e00640 1 -- 172.21.15.119:0/1999933372 >> v1:172.21.15.132:6789/0 conn(0x7f84ac050dd0 legacy=0x7f84ac039ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.576+0000 7f84b8e00640 1 -- 172.21.15.119:0/1999933372 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac04fde0 con 0x7f84ac0e1090 2024-04-25T10:07:14.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.577+0000 7f11df400640 1 -- 172.21.15.119:0/1672622193 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc035670 con 0x7f11e0159910 2024-04-25T10:07:14.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.577+0000 7f11df400640 1 -- 172.21.15.119:0/1672622193 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc035a40 con 0x7f11e0159910 2024-04-25T10:07:14.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.578+0000 7f84b8e00640 1 --2- 172.21.15.119:0/1999933372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84ac0e1090 0x7f84ac038630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009870 tx=0x7f84a802a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7402fae00f3ee3fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.578+0000 7f11e7400640 1 -- 172.21.15.119:0/1672622193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e0159910 msgr2=0x7f11e015e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.578+0000 7f84b3400640 1 -- 172.21.15.119:0/1999933372 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8037070 con 0x7f84ac0e1090 2024-04-25T10:07:14.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.578+0000 7f84b3400640 1 -- 172.21.15.119:0/1999933372 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a8032440 con 0x7f84ac0e1090 2024-04-25T10:07:14.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.579+0000 7f84b3400640 1 -- 172.21.15.119:0/1999933372 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8032850 con 0x7f84ac0e1090 2024-04-25T10:07:14.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.578+0000 7f11e7400640 1 --2- 172.21.15.119:0/1672622193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e0159910 0x7f11e015e0b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00ba70 tx=0x7f11cc02e840 comp rx=0 tx=0).stop 2024-04-25T10:07:14.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.579+0000 7f11e7400640 1 -- 172.21.15.119:0/1672622193 shutdown_connections 2024-04-25T10:07:14.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.579+0000 7f11e7400640 1 --2- 172.21.15.119:0/1672622193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e0159910 0x7f11e015e0b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.579+0000 7f11e7400640 1 -- 172.21.15.119:0/1672622193 >> 172.21.15.119:0/1672622193 conn(0x7f11e006aba0 msgr2=0x7f11e006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.579+0000 7f8b60e00640 1 Processor -- start 2024-04-25T10:07:14.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.580+0000 7f11e7400640 1 -- 172.21.15.119:0/1672622193 shutdown_connections 2024-04-25T10:07:14.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.580+0000 7f8b60e00640 1 -- start start 2024-04-25T10:07:14.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.580+0000 7f11e7400640 1 -- 172.21.15.119:0/1672622193 wait complete. 2024-04-25T10:07:14.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.581+0000 7f84ba200640 1 -- 172.21.15.119:0/1999933372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84ac0e1090 msgr2=0x7f84ac038630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.581+0000 7f84ba200640 1 --2- 172.21.15.119:0/1999933372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84ac0e1090 0x7f84ac038630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009870 tx=0x7f84a802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.582+0000 7f8b60e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8b5c0b54a0 0x7f8b5c0b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.582+0000 7f87f5000640 1 Processor -- start 2024-04-25T10:07:14.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.582+0000 7f11e7400640 1 Processor -- start 2024-04-25T10:07:14.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.582+0000 7f84ba200640 1 -- 172.21.15.119:0/1999933372 shutdown_connections 2024-04-25T10:07:14.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.582+0000 7f84ba200640 1 --2- 172.21.15.119:0/1999933372 >> v2:172.21.15.132:3300/0 conn(0x7f84ac03a5d0 0x7f84ac005f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.582+0000 7f84ba200640 1 --2- 172.21.15.119:0/1999933372 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84ac0e1090 0x7f84ac038630 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.582+0000 7f84ba200640 1 -- 172.21.15.119:0/1999933372 >> 172.21.15.119:0/1999933372 conn(0x7f84ac009fb0 msgr2=0x7f84ac00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.582+0000 7f8b60e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b5c0b7e00 0x7f8b5c06c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.582+0000 7f8b60e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5c0b5dc0 con 0x7f8b5c0d2fd0 2024-04-25T10:07:14.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.582+0000 7f8b60e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5c0d1e90 con 0x7f8b5c0b54a0 2024-04-25T10:07:14.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.582+0000 7f8b60e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c0d2170 con 0x7f8b5c0b7e00 2024-04-25T10:07:14.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.582+0000 7f11e7400640 1 -- start start 2024-04-25T10:07:14.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.582+0000 7f87f5000640 1 -- start start 2024-04-25T10:07:14.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.583+0000 7fbe9f000640 1 Processor -- start 2024-04-25T10:07:14.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.583+0000 7f84ba200640 1 -- 172.21.15.119:0/1999933372 shutdown_connections 2024-04-25T10:07:14.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.583+0000 7f84ba200640 1 -- 172.21.15.119:0/1999933372 wait complete. 2024-04-25T10:07:14.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.583+0000 7f8b59400640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f8b5c0d2fd0 0x7f8b5c06c020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:41392/0 (socket says 172.21.15.119:41392) 2024-04-25T10:07:14.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.583+0000 7fbe9f000640 1 -- start start 2024-04-25T10:07:14.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.583+0000 7f8b59400640 1 -- 172.21.15.119:0/4130201440 learned_addr learned my addr 172.21.15.119:0/4130201440 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:14.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.583+0000 7f8b5b200640 1 --2- 172.21.15.119:0/4130201440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b5c0b7e00 0x7f8b5c06c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f8b5b200640 1 -- 172.21.15.119:0/4130201440 >> v2:172.21.15.119:3300/0 conn(0x7f8b5c0b54a0 msgr2=0x7f8b5c0b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f8b5b200640 1 --2- 172.21.15.119:0/4130201440 >> v2:172.21.15.119:3300/0 conn(0x7f8b5c0b54a0 0x7f8b5c0b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f8b5b200640 1 -- 172.21.15.119:0/4130201440 >> v1:172.21.15.132:6789/0 conn(0x7f8b5c0d2fd0 legacy=0x7f8b5c06c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f8b5b200640 1 -- 172.21.15.119:0/4130201440 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c0d2450 con 0x7f8b5c0b7e00 2024-04-25T10:07:14.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f11e7400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e00b1740 0x7f11e0157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f11e7400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e00cef80 0x7f11e00b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f11e7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e0159910 0x7f11e0154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f11e7400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e006b840 con 0x7f11e0159910 2024-04-25T10:07:14.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f11e7400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e006b6c0 con 0x7f11e00cef80 2024-04-25T10:07:14.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f11e7400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00b1ba0 con 0x7f11e00b1740 2024-04-25T10:07:14.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f87f5000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f87f00d01d0 0x7f87f00cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f87f5000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f00d05b0 0x7f87f015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f87f5000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87f00b5440 0x7f87f00b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f87f5000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f0162160 con 0x7f87f00b5440 2024-04-25T10:07:14.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f87f5000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f01625f0 con 0x7f87f00d01d0 2024-04-25T10:07:14.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.584+0000 7f87f5000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f00cef60 con 0x7f87f00d05b0 2024-04-25T10:07:14.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7f87eea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f00d05b0 0x7f87f015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7f87eea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f00d05b0 0x7f87f015ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49158/0 (socket says 172.21.15.119:49158) 2024-04-25T10:07:14.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7f87eea00640 1 -- 172.21.15.119:0/1859547108 learned_addr learned my addr 172.21.15.119:0/1859547108 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7f87efe00640 1 --2- 172.21.15.119:0/1859547108 >> v2:172.21.15.119:3300/0 conn(0x7f87f00b5440 0x7f87f00b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7f84ba200640 1 Processor -- start 2024-04-25T10:07:14.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7f11e5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e0159910 0x7f11e0154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7f11dfe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e00cef80 0x7f11e00b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7f11dfe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e00cef80 0x7f11e00b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:50786/0 (socket says 172.21.15.119:50786) 2024-04-25T10:07:14.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7f11dfe00640 1 -- 172.21.15.119:0/297328809 learned_addr learned my addr 172.21.15.119:0/297328809 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7f11e4a00640 1 --2- 172.21.15.119:0/297328809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e00b1740 0x7f11e0157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7fbe9f000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbe98161090 0x7fbe98161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7fbe9f000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe98161f80 con 0x7fbe980ceeb0 2024-04-25T10:07:14.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7fbe9f000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe98162260 con 0x7fbe980b9720 2024-04-25T10:07:14.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7fbe9f000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe980d21a0 con 0x7fbe98161090 2024-04-25T10:07:14.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.585+0000 7f87ef400640 1 --2- 172.21.15.119:0/1859547108 >> v2:172.21.15.132:3300/0 conn(0x7f87f00d01d0 0x7f87f00cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.586+0000 7f8b5b200640 1 --2- 172.21.15.119:0/4130201440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b5c0b7e00 0x7f8b5c06c730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009d90 tx=0x7f8b5402a550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed7fb50e80948771 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.586+0000 7fbe9d000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbe980ceeb0 0x7fbe980d2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42142/0 (socket says 172.21.15.119:42142) 2024-04-25T10:07:14.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.586+0000 7fbe9d000640 1 -- 172.21.15.119:0/4286291468 learned_addr learned my addr 172.21.15.119:0/4286291468 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:14.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.586+0000 7fbe97e00640 1 --2- 172.21.15.119:0/4286291468 >> v2:172.21.15.119:3300/0 conn(0x7fbe98161090 0x7fbe98161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.586+0000 7f87eea00640 1 -- 172.21.15.119:0/1859547108 >> v2:172.21.15.132:3300/0 conn(0x7f87f00d01d0 msgr2=0x7f87f00cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.586+0000 7f87eea00640 1 --2- 172.21.15.119:0/1859547108 >> v2:172.21.15.132:3300/0 conn(0x7f87f00d01d0 0x7f87f00cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.586+0000 7f87eea00640 1 -- 172.21.15.119:0/1859547108 >> v2:172.21.15.119:3300/0 conn(0x7f87f00b5440 msgr2=0x7f87f00b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.586+0000 7f87eea00640 1 --2- 172.21.15.119:0/1859547108 >> v2:172.21.15.119:3300/0 conn(0x7f87f00b5440 0x7f87f00b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.586+0000 7f87eea00640 1 -- 172.21.15.119:0/1859547108 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f00cf200 con 0x7f87f00d05b0 2024-04-25T10:07:14.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.587+0000 7f8b58a00640 1 -- 172.21.15.119:0/4130201440 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b54007740 con 0x7f8b5c0b7e00 2024-04-25T10:07:14.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.587+0000 7f8b58a00640 1 -- 172.21.15.119:0/4130201440 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b540078e0 con 0x7f8b5c0b7e00 2024-04-25T10:07:14.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.587+0000 7f8b58a00640 1 -- 172.21.15.119:0/4130201440 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b54007cb0 con 0x7f8b5c0b7e00 2024-04-25T10:07:14.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.587+0000 7f11e4a00640 1 -- 172.21.15.119:0/297328809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e00cef80 msgr2=0x7f11e00b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.587+0000 7f11e4a00640 1 --2- 172.21.15.119:0/297328809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e00cef80 0x7f11e00b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.587+0000 7f11e4a00640 1 -- 172.21.15.119:0/297328809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e0159910 msgr2=0x7f11e0154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.587+0000 7f11e4a00640 1 --2- 172.21.15.119:0/297328809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e0159910 0x7f11e0154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.587+0000 7f11e4a00640 1 -- 172.21.15.119:0/297328809 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e0157c20 con 0x7f11e00b1740 2024-04-25T10:07:14.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.587+0000 7fbe96a00640 1 -- 172.21.15.119:0/4286291468 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873202441 0 0) 0x7fbe98161f80 con 0x7fbe980ceeb0 2024-04-25T10:07:14.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.587+0000 7fbe96a00640 1 -- 172.21.15.119:0/4286291468 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe7c003520 con 0x7fbe980ceeb0 2024-04-25T10:07:14.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.587+0000 7fbe96a00640 1 -- 172.21.15.119:0/4286291468 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4293800371 0 0) 0x7fbe7c003520 con 0x7fbe980ceeb0 2024-04-25T10:07:14.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.587+0000 7fbe96a00640 1 -- 172.21.15.119:0/4286291468 >> v2:172.21.15.119:3300/0 conn(0x7fbe98161090 msgr2=0x7fbe98161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7fbe96a00640 1 --2- 172.21.15.119:0/4286291468 >> v2:172.21.15.119:3300/0 conn(0x7fbe98161090 0x7fbe98161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7fbe96a00640 1 -- 172.21.15.119:0/4286291468 >> v1:172.21.15.132:6789/0 conn(0x7fbe980b9720 legacy=0x7fbe980ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7fbe96a00640 1 -- 172.21.15.119:0/4286291468 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe98162540 con 0x7fbe980ceeb0 2024-04-25T10:07:14.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7fbe96a00640 1 -- 172.21.15.119:0/4286291468 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fbe84001670 con 0x7fbe980ceeb0 2024-04-25T10:07:14.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7fbe96a00640 1 -- 172.21.15.119:0/4286291468 >> v1:172.21.15.119:6789/0 conn(0x7fbe980ceeb0 legacy=0x7fbe980d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7f8b60e00640 1 -- 172.21.15.119:0/4130201440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b5c0b7e00 msgr2=0x7f8b5c06c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7f8b60e00640 1 --2- 172.21.15.119:0/4130201440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b5c0b7e00 0x7f8b5c06c730 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009d90 tx=0x7f8b5402a550 comp rx=0 tx=0).stop 2024-04-25T10:07:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7f8b60e00640 1 -- 172.21.15.119:0/4130201440 shutdown_connections 2024-04-25T10:07:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7f8b60e00640 1 --2- 172.21.15.119:0/4130201440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b5c0b7e00 0x7f8b5c06c730 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7f8b60e00640 1 --2- 172.21.15.119:0/4130201440 >> v2:172.21.15.119:3300/0 conn(0x7f8b5c0b54a0 0x7f8b5c0b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7f8b60e00640 1 -- 172.21.15.119:0/4130201440 >> 172.21.15.119:0/4130201440 conn(0x7f8b5c0d04a0 msgr2=0x7f8b5c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7fbe96a00640 1 --2- 172.21.15.119:0/4286291468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe7c0039b0 0x7fbe7c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7fbe96a00640 1 --2- 172.21.15.119:0/4286291468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe7c004800 0x7fbe7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7fbe96a00640 1 --2- 172.21.15.119:0/4286291468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe7c0053c0 0x7fbe7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7fbe96a00640 1 -- 172.21.15.119:0/4286291468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe980d21a0 con 0x7fbe7c0039b0 2024-04-25T10:07:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7fbe96a00640 1 -- 172.21.15.119:0/4286291468 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c00c260 con 0x7fbe7c004800 2024-04-25T10:07:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7f8b60e00640 1 -- 172.21.15.119:0/4130201440 shutdown_connections 2024-04-25T10:07:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7f8b60e00640 1 -- 172.21.15.119:0/4130201440 wait complete. 2024-04-25T10:07:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7fbe96a00640 1 -- 172.21.15.119:0/4286291468 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c00c460 con 0x7fbe7c0053c0 2024-04-25T10:07:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.588+0000 7fbe97e00640 1 --2- 172.21.15.119:0/4286291468 >> v2:172.21.15.119:3300/0 conn(0x7fbe98161090 0x7fbe98161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.589+0000 7fbe97400640 1 --2- 172.21.15.119:0/4286291468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe7c004800 0x7fbe7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.589+0000 7fbe9d000640 1 --2- 172.21.15.119:0/4286291468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe7c0053c0 0x7fbe7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.589+0000 7f11e4a00640 1 --2- 172.21.15.119:0/297328809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e00b1740 0x7f11e0157f00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f11cc037000 tx=0x7f11cc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.589+0000 7fbe97e00640 1 --2- 172.21.15.119:0/4286291468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe7c0039b0 0x7fbe7c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.589+0000 7f84ba200640 1 -- start start 2024-04-25T10:07:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.589+0000 7f87eea00640 1 --2- 172.21.15.119:0/1859547108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f00d05b0 0x7f87f015ea00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f87f006ce70 tx=0x7f87e002a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc95d2e92e45abe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.589+0000 7f11dcc00640 1 -- 172.21.15.119:0/297328809 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc03d030 con 0x7f11e00b1740 2024-04-25T10:07:14.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.589+0000 7f11dcc00640 1 -- 172.21.15.119:0/297328809 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc03f070 con 0x7f11e00b1740 2024-04-25T10:07:14.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.589+0000 7f11dcc00640 1 -- 172.21.15.119:0/297328809 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc004030 con 0x7f11e00b1740 2024-04-25T10:07:14.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.590+0000 7f87ee000640 1 -- 172.21.15.119:0/1859547108 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0037070 con 0x7f87f00d05b0 2024-04-25T10:07:14.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.590+0000 7fbe97e00640 1 -- 172.21.15.119:0/4286291468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe7c0053c0 msgr2=0x7fbe7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.590+0000 7fbe97e00640 1 --2- 172.21.15.119:0/4286291468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe7c0053c0 0x7fbe7c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.590+0000 7f87ee000640 1 -- 172.21.15.119:0/1859547108 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e0032440 con 0x7f87f00d05b0 2024-04-25T10:07:14.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.590+0000 7f87ee000640 1 -- 172.21.15.119:0/1859547108 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0032810 con 0x7f87f00d05b0 2024-04-25T10:07:14.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.591+0000 7f84ba200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84ac03a5d0 0x7f84ac0e3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.590+0000 7fbe97e00640 1 -- 172.21.15.119:0/4286291468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe7c004800 msgr2=0x7fbe7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.590+0000 7fbe97e00640 1 --2- 172.21.15.119:0/4286291468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe7c004800 0x7fbe7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.590+0000 7fbe97e00640 1 -- 172.21.15.119:0/4286291468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7c003520 con 0x7fbe7c0039b0 2024-04-25T10:07:14.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.590+0000 7fbe97400640 1 --2- 172.21.15.119:0/4286291468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe7c004800 0x7fbe7c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:14.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.591+0000 7fbe9d000640 1 -- 172.21.15.119:0/4286291468 reap_dead start 2024-04-25T10:07:14.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.591+0000 7f738a000640 1 Processor -- start 2024-04-25T10:07:14.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.591+0000 7f11e7400640 1 -- 172.21.15.119:0/297328809 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e00c8700 con 0x7f11e00b1740 2024-04-25T10:07:14.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.591+0000 7f11e7400640 1 -- 172.21.15.119:0/297328809 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e0158850 con 0x7f11e00b1740 2024-04-25T10:07:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.591+0000 7f8b60e00640 1 Processor -- start 2024-04-25T10:07:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.591+0000 7f87f5000640 1 -- 172.21.15.119:0/1859547108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f00d05b0 msgr2=0x7f87f015ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.591+0000 7f87f5000640 1 --2- 172.21.15.119:0/1859547108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f00d05b0 0x7f87f015ea00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f87f006ce70 tx=0x7f87e002a140 comp rx=0 tx=0).stop 2024-04-25T10:07:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.591+0000 7f738a000640 1 -- start start 2024-04-25T10:07:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.591+0000 7f87f5000640 1 -- 172.21.15.119:0/1859547108 shutdown_connections 2024-04-25T10:07:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.591+0000 7f87f5000640 1 --2- 172.21.15.119:0/1859547108 >> v2:172.21.15.119:3300/0 conn(0x7f87f00b5440 0x7f87f00b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.591+0000 7f87f5000640 1 --2- 172.21.15.119:0/1859547108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f00d05b0 0x7f87f015ea00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.591+0000 7f87f5000640 1 --2- 172.21.15.119:0/1859547108 >> v2:172.21.15.132:3300/0 conn(0x7f87f00d01d0 0x7f87f00cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.591+0000 7f87f5000640 1 -- 172.21.15.119:0/1859547108 >> 172.21.15.119:0/1859547108 conn(0x7f87f006bed0 msgr2=0x7f87f006c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.592+0000 7f84ba200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84ac050dd0 0x7f84ac04e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.592+0000 7f84ba200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84ac0e1090 0x7f84ac04bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.592+0000 7f84ba200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac0064d0 con 0x7f84ac03a5d0 2024-04-25T10:07:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.592+0000 7f84ba200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac0503a0 con 0x7f84ac050dd0 2024-04-25T10:07:14.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.592+0000 7f84ba200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac006650 con 0x7f84ac0e1090 2024-04-25T10:07:14.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.592+0000 7fbe97e00640 1 --2- 172.21.15.119:0/4286291468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe7c0039b0 0x7fbe7c003db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000c410 tx=0x7fbe7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5038a4263010f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.592+0000 7f84b8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84ac03a5d0 0x7f84ac0e3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.592+0000 7f84b8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84ac03a5d0 0x7f84ac0e3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34702/0 (socket says 172.21.15.119:34702) 2024-04-25T10:07:14.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.592+0000 7f84b8e00640 1 -- 172.21.15.119:0/3499157082 learned_addr learned my addr 172.21.15.119:0/3499157082 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.592+0000 7f11e7400640 1 -- 172.21.15.119:0/297328809 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e006b1d0 con 0x7f11e00b1740 2024-04-25T10:07:14.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.592+0000 7f8b60e00640 1 -- start start 2024-04-25T10:07:14.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.593+0000 7f84b9800640 1 --2- 172.21.15.119:0/3499157082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84ac0e1090 0x7f84ac04bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.593+0000 7fbe96a00640 1 -- 172.21.15.119:0/4286291468 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90007c00 con 0x7fbe7c0039b0 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.593+0000 7fbe96a00640 1 -- 172.21.15.119:0/4286291468 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe90003b90 con 0x7fbe7c0039b0 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.593+0000 7fbe96a00640 1 -- 172.21.15.119:0/4286291468 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9002b910 con 0x7fbe7c0039b0 2024-04-25T10:07:14.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.593+0000 7f738a000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f738415dbe0 0x7f738415ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.593+0000 7f738a000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73840d3580 0x7f73840b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.593+0000 7f738a000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73840b9980 con 0x7f73840b9410 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.593+0000 7f738a000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738415f210 con 0x7f738415dbe0 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.593+0000 7f738a000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73840ce140 con 0x7f73840d3580 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.593+0000 7f84b8e00640 1 -- 172.21.15.119:0/3499157082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84ac0e1090 msgr2=0x7f84ac04bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.593+0000 7f84b8e00640 1 --2- 172.21.15.119:0/3499157082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84ac0e1090 0x7f84ac04bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.593+0000 7f84b8e00640 1 -- 172.21.15.119:0/3499157082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84ac050dd0 msgr2=0x7f84ac04e440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.593+0000 7f84b8e00640 1 --2- 172.21.15.119:0/3499157082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84ac050dd0 0x7f84ac04e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.593+0000 7f84b8e00640 1 -- 172.21.15.119:0/3499157082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac0496e0 con 0x7f84ac03a5d0 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.592+0000 7f87f5000640 1 -- 172.21.15.119:0/1859547108 shutdown_connections 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.592+0000 7f87f5000640 1 -- 172.21.15.119:0/1859547108 wait complete. 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.594+0000 7f7382a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73840d3580 0x7f73840b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.594+0000 7fbe9f000640 1 -- 172.21.15.119:0/4286291468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe7c0039b0 msgr2=0x7fbe7c003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.594+0000 7fbe9f000640 1 --2- 172.21.15.119:0/4286291468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe7c0039b0 0x7fbe7c003db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000c410 tx=0x7fbe7c005830 comp rx=0 tx=0).stop 2024-04-25T10:07:14.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.594+0000 7f7382a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73840d3580 0x7f73840b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49192/0 (socket says 172.21.15.119:49192) 2024-04-25T10:07:14.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.594+0000 7f7382a00640 1 -- 172.21.15.119:0/408596406 learned_addr learned my addr 172.21.15.119:0/408596406 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.594+0000 7fbe9f000640 1 -- 172.21.15.119:0/4286291468 shutdown_connections 2024-04-25T10:07:14.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.594+0000 7fbe9f000640 1 --2- 172.21.15.119:0/4286291468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe7c0039b0 0x7fbe7c003db0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000c410 tx=0x7fbe7c005830 comp rx=0 tx=0).stop 2024-04-25T10:07:14.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.594+0000 7fbe9f000640 1 -- 172.21.15.119:0/4286291468 >> 172.21.15.119:0/4286291468 conn(0x7fbe980b55e0 msgr2=0x7fbe980b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.594+0000 7f7383400640 1 --2- 172.21.15.119:0/408596406 >> v2:172.21.15.132:3300/0 conn(0x7f738415dbe0 0x7f738415ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.594+0000 7fbe9f000640 1 -- 172.21.15.119:0/4286291468 shutdown_connections 2024-04-25T10:07:14.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.595+0000 7f7383400640 1 -- 172.21.15.119:0/408596406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73840d3580 msgr2=0x7f73840b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.595+0000 7f7383400640 1 --2- 172.21.15.119:0/408596406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73840d3580 0x7f73840b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.595+0000 7f7383400640 1 -- 172.21.15.119:0/408596406 >> v1:172.21.15.132:6789/0 conn(0x7f73840b9410 legacy=0x7f738406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.595+0000 7f7383400640 1 -- 172.21.15.119:0/408596406 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73840ce420 con 0x7f738415dbe0 2024-04-25T10:07:14.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.594+0000 7f84b8e00640 1 --2- 172.21.15.119:0/3499157082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84ac03a5d0 0x7f84ac0e3040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f84a8000c00 tx=0x7f84a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.595+0000 7f8b60e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b5c0b54a0 0x7f8b5c0cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.595+0000 7f8b60e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b5c0b7e00 0x7f8b5c155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.595+0000 7f87f5000640 1 Processor -- start 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.595+0000 7f84b0c00640 1 -- 172.21.15.119:0/3499157082 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8041020 con 0x7f84ac03a5d0 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.596+0000 7f84ba200640 1 -- 172.21.15.119:0/3499157082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ac0499c0 con 0x7f84ac03a5d0 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.596+0000 7f87f5000640 1 -- start start 2024-04-25T10:07:14.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.596+0000 7f11dcc00640 1 -- 172.21.15.119:0/297328809 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11cc041070 con 0x7f11e00b1740 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.596+0000 7f84ba200640 1 -- 172.21.15.119:0/3499157082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ac049f80 con 0x7f84ac03a5d0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.596+0000 7f84b0c00640 1 -- 172.21.15.119:0/3499157082 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a8032e10 con 0x7f84ac03a5d0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.596+0000 7f84b0c00640 1 -- 172.21.15.119:0/3499157082 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8041020 con 0x7f84ac03a5d0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.596+0000 7f8b60e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b5c0d2fd0 0x7f8b5c152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.596+0000 7f8b59e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b5c0b54a0 0x7f8b5c0cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.596+0000 7f8b59e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b5c0b54a0 0x7f8b5c0cb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49196/0 (socket says 172.21.15.119:49196) 2024-04-25T10:07:14.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.597+0000 7f7383400640 1 --2- 172.21.15.119:0/408596406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f738415dbe0 0x7f738415ecd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f737c009d90 tx=0x7f737c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e06f6159df072c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.597+0000 7f8b59e00640 1 -- 172.21.15.119:0/1576953786 learned_addr learned my addr 172.21.15.119:0/1576953786 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.597+0000 7f8b60e00640 1 -- 172.21.15.119:0/1576953786 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c0d1c30 con 0x7f8b5c0b7e00 2024-04-25T10:07:14.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.595+0000 7fbe9f000640 1 -- 172.21.15.119:0/4286291468 wait complete. 2024-04-25T10:07:14.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.597+0000 7f11dcc00640 1 --2- 172.21.15.119:0/297328809 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f11c006b5a0 0x7f11c006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.597+0000 7fbe9f000640 1 Processor -- start 2024-04-25T10:07:14.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.597+0000 7f8b60e00640 1 -- 172.21.15.119:0/1576953786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c0d1ab0 con 0x7f8b5c0d2fd0 2024-04-25T10:07:14.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.597+0000 7f8b5b200640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b5c0d2fd0 0x7f8b5c152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.598+0000 7f8b60e00640 1 -- 172.21.15.119:0/1576953786 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c15dd30 con 0x7f8b5c0b54a0 2024-04-25T10:07:14.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.598+0000 7f37f5600640 1 Processor -- start 2024-04-25T10:07:14.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.598+0000 7f8b59400640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b5c0b7e00 0x7f8b5c155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.598+0000 7f11dcc00640 1 -- 172.21.15.119:0/297328809 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11cc0aa230 con 0x7f11e00b1740 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.598+0000 7f11dfe00640 1 --2- 172.21.15.119:0/297328809 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f11c006b5a0 0x7f11c006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.598+0000 7f87f5000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f00b5440 0x7f87f0157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.597+0000 7f7382000640 1 -- 172.21.15.119:0/408596406 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c007740 con 0x7f738415dbe0 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.597+0000 7f7382000640 1 -- 172.21.15.119:0/408596406 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c0078e0 con 0x7f738415dbe0 2024-04-25T10:07:14.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.597+0000 7f7382000640 1 -- 172.21.15.119:0/408596406 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c007cb0 con 0x7f738415dbe0 2024-04-25T10:07:14.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.599+0000 7fbe9f000640 1 -- start start 2024-04-25T10:07:14.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.599+0000 7f87f5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87f00d01d0 0x7f87f015a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.599+0000 7f87f5000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87f00d05b0 0x7f87f00c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.598+0000 7f37f5600640 1 -- start start 2024-04-25T10:07:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.599+0000 7f87f5000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0161df0 con 0x7f87f00d01d0 2024-04-25T10:07:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.599+0000 7f87f5000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0161b10 con 0x7f87f00d05b0 2024-04-25T10:07:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.599+0000 7f87f5000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f015ef40 con 0x7f87f00b5440 2024-04-25T10:07:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.599+0000 7f87ef400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f00b5440 0x7f87f0157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.599+0000 7f87eea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87f00d01d0 0x7f87f015a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.598+0000 7f84ba200640 1 -- 172.21.15.119:0/3499157082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8490005290 con 0x7f84ac03a5d0 2024-04-25T10:07:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f87eea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87f00d01d0 0x7f87f015a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34716/0 (socket says 172.21.15.119:34716) 2024-04-25T10:07:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f87eea00640 1 -- 172.21.15.119:0/881254888 learned_addr learned my addr 172.21.15.119:0/881254888 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f87efe00640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87f00d05b0 0x7f87f00c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f8b5b200640 1 -- 172.21.15.119:0/1576953786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b5c0b54a0 msgr2=0x7f8b5c0cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f8b5b200640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b5c0b54a0 0x7f8b5c0cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f8b5b200640 1 -- 172.21.15.119:0/1576953786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b5c0b7e00 msgr2=0x7f8b5c155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f8b5b200640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b5c0b7e00 0x7f8b5c155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f8b5b200640 1 -- 172.21.15.119:0/1576953786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c1f7640 con 0x7f8b5c0d2fd0 2024-04-25T10:07:14.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.598+0000 7f738a000640 1 -- 172.21.15.119:0/408596406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f738415dbe0 msgr2=0x7f738415ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.598+0000 7f738a000640 1 --2- 172.21.15.119:0/408596406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f738415dbe0 0x7f738415ecd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f737c009d90 tx=0x7f737c02a660 comp rx=0 tx=0).stop 2024-04-25T10:07:14.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f738a000640 1 -- 172.21.15.119:0/408596406 shutdown_connections 2024-04-25T10:07:14.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f738a000640 1 --2- 172.21.15.119:0/408596406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73840d3580 0x7f73840b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f738a000640 1 --2- 172.21.15.119:0/408596406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f738415dbe0 0x7f738415ecd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f738a000640 1 -- 172.21.15.119:0/408596406 >> 172.21.15.119:0/408596406 conn(0x7f7384072900 msgr2=0x7f73840d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f37f5600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37e80e0e20 0x7f37e803c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f37f5600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37e8054d90 0x7f37e8056580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f37f5600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e80554e0 con 0x7f37e80519b0 2024-04-25T10:07:14.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f37f5600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e8056ac0 con 0x7f37e8054d90 2024-04-25T10:07:14.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f37f5600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80557c0 con 0x7f37e80e0e20 2024-04-25T10:07:14.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f87efe00640 1 -- 172.21.15.119:0/881254888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f00b5440 msgr2=0x7f87f0157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f8b59400640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b5c0b7e00 0x7f8b5c155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:14.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f37ef400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37e8054d90 0x7f37e8056580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.600+0000 7f11dfe00640 1 --2- 172.21.15.119:0/297328809 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f11c006b5a0 0x7f11c006da60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f11e00b4070 tx=0x7f11d00095e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f87efe00640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f00b5440 0x7f87f0157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f87efe00640 1 -- 172.21.15.119:0/881254888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87f00d01d0 msgr2=0x7f87f015a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f37ef400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37e8054d90 0x7f37e8056580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34726/0 (socket says 172.21.15.119:34726) 2024-04-25T10:07:14.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f37ef400640 1 -- 172.21.15.119:0/133316603 learned_addr learned my addr 172.21.15.119:0/133316603 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f738a000640 1 -- 172.21.15.119:0/408596406 shutdown_connections 2024-04-25T10:07:14.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f37efe00640 1 --2- 172.21.15.119:0/133316603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37e80e0e20 0x7f37e803c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7fbe9f000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe981ffe70 0x7fbe980cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f37ef400640 1 -- 172.21.15.119:0/133316603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37e80e0e20 msgr2=0x7f37e803c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f37ef400640 1 --2- 172.21.15.119:0/133316603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37e80e0e20 0x7f37e803c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f37ef400640 1 -- 172.21.15.119:0/133316603 >> v1:172.21.15.119:6789/0 conn(0x7f37e80519b0 legacy=0x7f37e8039e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f37ef400640 1 -- 172.21.15.119:0/133316603 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e8055aa0 con 0x7f37e8054d90 2024-04-25T10:07:14.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7fbe9f000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe980c9e50 0x7fbe980cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f87efe00640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87f00d01d0 0x7f87f015a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f87efe00640 1 -- 172.21.15.119:0/881254888 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f01592a0 con 0x7f87f00d05b0 2024-04-25T10:07:14.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7fbe9f000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe980b3af0 0x7fbe980b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f8b5b200640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b5c0d2fd0 0x7f8b5c152a80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b54031000 tx=0x7f8b54004a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f87ef400640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f00b5440 0x7f87f0157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:14.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.602+0000 7f87eea00640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87f00d01d0 0x7f87f015a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:14.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.601+0000 7f738a000640 1 -- 172.21.15.119:0/408596406 wait complete. 2024-04-25T10:07:14.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.602+0000 7f84b0c00640 1 -- 172.21.15.119:0/3499157082 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84a8044020 con 0x7f84ac03a5d0 2024-04-25T10:07:14.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.602+0000 7fbe9f000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe980ce7a0 con 0x7fbe981ffe70 2024-04-25T10:07:14.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.602+0000 7fbe9f000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe98161470 con 0x7fbe980c9e50 2024-04-25T10:07:14.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.602+0000 7fbe9f000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe980ce920 con 0x7fbe980b3af0 2024-04-25T10:07:14.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.602+0000 7fbe97e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe981ffe70 0x7fbe980cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.602+0000 7fbe97400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe980c9e50 0x7fbe980cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.602+0000 7fbe97e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe981ffe70 0x7fbe980cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34736/0 (socket says 172.21.15.119:34736) 2024-04-25T10:07:14.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.602+0000 7fbe97e00640 1 -- 172.21.15.119:0/3458043096 learned_addr learned my addr 172.21.15.119:0/3458043096 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.602+0000 7fbe9d000640 1 --2- 172.21.15.119:0/3458043096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe980b3af0 0x7fbe980b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.603+0000 7f87efe00640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87f00d05b0 0x7f87f00c91e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f87d800d550 tx=0x7f87d800da10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.603+0000 7fbe97e00640 1 -- 172.21.15.119:0/3458043096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe980b3af0 msgr2=0x7fbe980b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.603+0000 7fbe97e00640 1 --2- 172.21.15.119:0/3458043096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe980b3af0 0x7fbe980b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.603+0000 7fbe97e00640 1 -- 172.21.15.119:0/3458043096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe980c9e50 msgr2=0x7fbe980cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.603+0000 7fbe97e00640 1 --2- 172.21.15.119:0/3458043096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe980c9e50 0x7fbe980cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.603+0000 7fbe97e00640 1 -- 172.21.15.119:0/3458043096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe980ccc30 con 0x7fbe981ffe70 2024-04-25T10:07:14.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.603+0000 7f738a000640 1 Processor -- start 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.603+0000 7f37ef400640 1 --2- 172.21.15.119:0/133316603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e8054d90 0x7f37e8056580 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009870 tx=0x7f37e402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca07320a14df937 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.604+0000 7fbe9d000640 1 --2- 172.21.15.119:0/3458043096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe980b3af0 0x7fbe980b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.602+0000 7f8b52000640 1 -- 172.21.15.119:0/1576953786 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5403f070 con 0x7f8b5c0d2fd0 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.602+0000 7f8b52000640 1 -- 172.21.15.119:0/1576953786 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b54031c80 con 0x7f8b5c0d2fd0 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.602+0000 7f8b52000640 1 -- 172.21.15.119:0/1576953786 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b540446a0 con 0x7f8b5c0d2fd0 2024-04-25T10:07:14.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.604+0000 7f87e7400640 1 -- 172.21.15.119:0/881254888 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8018020 con 0x7f87f00d05b0 2024-04-25T10:07:14.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.604+0000 7f738a000640 1 -- start start 2024-04-25T10:07:14.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.604+0000 7f87e7400640 1 -- 172.21.15.119:0/881254888 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d8014620 con 0x7f87f00d05b0 2024-04-25T10:07:14.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.604+0000 7f87e7400640 1 -- 172.21.15.119:0/881254888 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8014a10 con 0x7f87f00d05b0 2024-04-25T10:07:14.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.604+0000 7fbe97e00640 1 --2- 172.21.15.119:0/3458043096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe981ffe70 0x7fbe980cc600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbe9002b580 tx=0x7fbe90004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.604+0000 7f8b60e00640 1 -- 172.21.15.119:0/1576953786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5c0cad00 con 0x7f8b5c0d2fd0 2024-04-25T10:07:14.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.604+0000 7f8b60e00640 1 -- 172.21.15.119:0/1576953786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c0cb2c0 con 0x7f8b5c0d2fd0 2024-04-25T10:07:14.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.603+0000 7f84b0c00640 1 --2- 172.21.15.119:0/3499157082 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f849406b700 0x7f849406dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.603+0000 7f84b0c00640 1 -- 172.21.15.119:0/3499157082 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84a80aa640 con 0x7f84ac03a5d0 2024-04-25T10:07:14.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.605+0000 7f87f5000640 1 -- 172.21.15.119:0/881254888 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f0159540 con 0x7f87f00d05b0 2024-04-25T10:07:14.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.605+0000 7f87f5000640 1 -- 172.21.15.119:0/881254888 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f0159b00 con 0x7f87f00d05b0 2024-04-25T10:07:14.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.606+0000 7f738a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73840b9410 0x7f7384158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.606+0000 7f84b3e00640 1 --2- 172.21.15.119:0/3499157082 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f849406b700 0x7f849406dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.606+0000 7f8b47400640 1 -- 172.21.15.119:0/1576953786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b30005290 con 0x7f8b5c0d2fd0 2024-04-25T10:07:14.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.606+0000 7f87f5000640 1 -- 172.21.15.119:0/881254888 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f00749b0 con 0x7f87f00d05b0 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.606+0000 7f7383400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73840b9410 0x7f7384158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.606+0000 7f738a000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73840d3580 0x7f738415b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.606+0000 7f738a000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f738415dbe0 0x7f73840b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.606+0000 7f7383400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73840b9410 0x7f7384158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34748/0 (socket says 172.21.15.119:34748) 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.607+0000 7f738a000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738415dfc0 con 0x7f73840b9410 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.607+0000 7f738a000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73840d3060 con 0x7f738415dbe0 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.607+0000 7f738a000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738415e140 con 0x7f73840d3580 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.607+0000 7f7383400640 1 -- 172.21.15.119:0/2631389021 learned_addr learned my addr 172.21.15.119:0/2631389021 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.607+0000 7f7382a00640 1 --2- 172.21.15.119:0/2631389021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73840d3580 0x7f738415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.608+0000 7f84b3e00640 1 --2- 172.21.15.119:0/3499157082 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f849406b700 0x7f849406dbc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f84a0000f60 tx=0x7f84a00079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.607+0000 7f7383400640 1 -- 172.21.15.119:0/2631389021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73840d3580 msgr2=0x7f738415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.607+0000 7f7383400640 1 --2- 172.21.15.119:0/2631389021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73840d3580 0x7f738415b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.607+0000 7f7383400640 1 -- 172.21.15.119:0/2631389021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f738415dbe0 msgr2=0x7f73840b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.607+0000 7f7383400640 1 --2- 172.21.15.119:0/2631389021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f738415dbe0 0x7f73840b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.607+0000 7f7383400640 1 -- 172.21.15.119:0/2631389021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7384158e80 con 0x7f73840b9410 2024-04-25T10:07:14.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.608+0000 7f7383400640 1 --2- 172.21.15.119:0/2631389021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73840b9410 0x7f7384158940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f737c009d90 tx=0x7f737c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.608+0000 7f8b52000640 1 -- 172.21.15.119:0/1576953786 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b54044840 con 0x7f8b5c0d2fd0 2024-04-25T10:07:14.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.609+0000 7f87e7400640 1 -- 172.21.15.119:0/881254888 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87d801b020 con 0x7f87f00d05b0 2024-04-25T10:07:14.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.609+0000 7f8b52000640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b3c06b4d0 0x7f8b3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.609+0000 7f8b52000640 1 -- 172.21.15.119:0/1576953786 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b540abb30 con 0x7f8b5c0d2fd0 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.610+0000 7f8b59e00640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b3c06b4d0 0x7f8b3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.610+0000 7f37eea00640 1 -- 172.21.15.119:0/133316603 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e4037070 con 0x7f37e8054d90 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.610+0000 7f37eea00640 1 -- 172.21.15.119:0/133316603 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e4032440 con 0x7f37e8054d90 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.610+0000 7f37eea00640 1 -- 172.21.15.119:0/133316603 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e4032810 con 0x7f37e8054d90 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.610+0000 7fbe8fe00640 1 -- 172.21.15.119:0/3458043096 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90042070 con 0x7fbe981ffe70 2024-04-25T10:07:14.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.610+0000 7f737b400640 1 -- 172.21.15.119:0/2631389021 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c040070 con 0x7f73840b9410 2024-04-25T10:07:14.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.610+0000 7f84b0c00640 1 -- 172.21.15.119:0/3499157082 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84a804b030 con 0x7f84ac03a5d0 2024-04-25T10:07:14.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.610+0000 7f737b400640 1 -- 172.21.15.119:0/2631389021 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c031c40 con 0x7f73840b9410 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.610+0000 7f737b400640 1 -- 172.21.15.119:0/2631389021 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c009450 con 0x7f73840b9410 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.610+0000 7f738a000640 1 -- 172.21.15.119:0/2631389021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7384159370 con 0x7f73840b9410 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.610+0000 7f87e7400640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87cc06b520 0x7f87cc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.610+0000 7f87e7400640 1 -- 172.21.15.119:0/881254888 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87d808a560 con 0x7f87f00d05b0 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.610+0000 7fbe9f000640 1 -- 172.21.15.119:0/3458043096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe980c9090 con 0x7fbe981ffe70 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.611+0000 7fbe9f000640 1 -- 172.21.15.119:0/3458043096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe980c9650 con 0x7fbe981ffe70 2024-04-25T10:07:14.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.611+0000 7f37f5600640 1 -- 172.21.15.119:0/133316603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e8054d90 msgr2=0x7f37e8056580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.611+0000 7f37f5600640 1 --2- 172.21.15.119:0/133316603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e8054d90 0x7f37e8056580 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009870 tx=0x7f37e402a180 comp rx=0 tx=0).stop 2024-04-25T10:07:14.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.612+0000 7f37f5600640 1 -- 172.21.15.119:0/133316603 shutdown_connections 2024-04-25T10:07:14.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.612+0000 7f37f5600640 1 --2- 172.21.15.119:0/133316603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e8054d90 0x7f37e8056580 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.612+0000 7f37f5600640 1 --2- 172.21.15.119:0/133316603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37e80e0e20 0x7f37e803c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.612+0000 7f37f5600640 1 -- 172.21.15.119:0/133316603 >> 172.21.15.119:0/133316603 conn(0x7f37e8053760 msgr2=0x7f37e8053b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.611+0000 7f738a000640 1 -- 172.21.15.119:0/2631389021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73840cb0f0 con 0x7f73840b9410 2024-04-25T10:07:14.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.612+0000 7fbe8fe00640 1 -- 172.21.15.119:0/3458043096 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe90003990 con 0x7fbe981ffe70 2024-04-25T10:07:14.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.612+0000 7fbe8fe00640 1 -- 172.21.15.119:0/3458043096 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90007c00 con 0x7fbe981ffe70 2024-04-25T10:07:14.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.612+0000 7f738a000640 1 -- 172.21.15.119:0/2631389021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73840750b0 con 0x7f73840b9410 2024-04-25T10:07:14.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.612+0000 7f87ef400640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87cc06b520 0x7f87cc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.613+0000 7f8b59e00640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b3c06b4d0 0x7f8b3c06d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0099a0 tx=0x7f8b4c006f90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.613+0000 7f37f5600640 1 -- 172.21.15.119:0/133316603 shutdown_connections 2024-04-25T10:07:14.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.613+0000 7f87ef400640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87cc06b520 0x7f87cc06d9e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f87f01fb5c0 tx=0x7f87e80039f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.613+0000 7f37f5600640 1 -- 172.21.15.119:0/133316603 wait complete. 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.614+0000 7fbe9f000640 1 -- 172.21.15.119:0/3458043096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe70005290 con 0x7fbe981ffe70 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.614+0000 7fbe8fe00640 1 -- 172.21.15.119:0/3458043096 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe90045070 con 0x7fbe981ffe70 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.615+0000 7f37f5600640 1 Processor -- start 2024-04-25T10:07:14.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.615+0000 7f37f5600640 1 -- start start 2024-04-25T10:07:14.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.616+0000 7fbe8fe00640 1 --2- 172.21.15.119:0/3458043096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe7806b550 0x7fbe7806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.616+0000 7f737b400640 1 -- 172.21.15.119:0/2631389021 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f737c039040 con 0x7f73840b9410 2024-04-25T10:07:14.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.616+0000 7fbe8fe00640 1 -- 172.21.15.119:0/3458043096 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe900ab830 con 0x7fbe981ffe70 2024-04-25T10:07:14.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.617+0000 7fbe97400640 1 --2- 172.21.15.119:0/3458043096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe7806b550 0x7fbe7806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.617+0000 7f37f5600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37e80519b0 0x7f37e804ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.617+0000 7f37f5600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37e8054d90 0x7f37e8046ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.617+0000 7f37f5600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e80e0e20 0x7f37e80dbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.617+0000 7f37f5600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80551f0 con 0x7f37e80e0e20 2024-04-25T10:07:14.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.617+0000 7f37f5600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e8056060 con 0x7f37e80519b0 2024-04-25T10:07:14.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.617+0000 7f737b400640 1 --2- 172.21.15.119:0/2631389021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f736406b4d0 0x7f736406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.618+0000 7f37f5600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80e1200 con 0x7f37e8054d90 2024-04-25T10:07:14.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.618+0000 7f37efe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37e80519b0 0x7f37e804ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.618+0000 7f737b400640 1 -- 172.21.15.119:0/2631389021 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f737c0abb90 con 0x7f73840b9410 2024-04-25T10:07:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.618+0000 7f37efe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37e80519b0 0x7f37e804ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:50856/0 (socket says 172.21.15.119:50856) 2024-04-25T10:07:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.618+0000 7f37efe00640 1 -- 172.21.15.119:0/2545386218 learned_addr learned my addr 172.21.15.119:0/2545386218 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.618+0000 7f37ef400640 1 --2- 172.21.15.119:0/2545386218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37e8054d90 0x7f37e8046ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.618+0000 7f37f4c00640 1 --2- 172.21.15.119:0/2545386218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e80e0e20 0x7f37e80dbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.618+0000 7f7382a00640 1 --2- 172.21.15.119:0/2631389021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f736406b4d0 0x7f736406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.618+0000 7fbe97400640 1 --2- 172.21.15.119:0/3458043096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe7806b550 0x7fbe7806da10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe980ce2c0 tx=0x7fbe88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.619+0000 7f37efe00640 1 -- 172.21.15.119:0/2545386218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37e8054d90 msgr2=0x7f37e8046ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.619+0000 7f37efe00640 1 --2- 172.21.15.119:0/2545386218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37e8054d90 0x7f37e8046ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.619+0000 7f37efe00640 1 -- 172.21.15.119:0/2545386218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e80e0e20 msgr2=0x7f37e80dbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.619+0000 7f37efe00640 1 --2- 172.21.15.119:0/2545386218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e80e0e20 0x7f37e80dbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.619+0000 7f37efe00640 1 -- 172.21.15.119:0/2545386218 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e8045f50 con 0x7f37e80519b0 2024-04-25T10:07:14.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.620+0000 7f7382a00640 1 --2- 172.21.15.119:0/2631389021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f736406b4d0 0x7f736406d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f73840cdbc0 tx=0x7f7374008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.620+0000 7f37efe00640 1 --2- 172.21.15.119:0/2545386218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37e80519b0 0x7f37e804ce60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f37e000d800 tx=0x7f37e000dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.620+0000 7f11dcc00640 1 -- 172.21.15.119:0/297328809 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11cc078bf0 con 0x7f11e00b1740 2024-04-25T10:07:14.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.622+0000 7f737b400640 1 -- 172.21.15.119:0/2631389021 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f737c07a9f0 con 0x7f73840b9410 2024-04-25T10:07:14.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.626+0000 7fbe8fe00640 1 -- 172.21.15.119:0/3458043096 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe9007a260 con 0x7fbe981ffe70 2024-04-25T10:07:14.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.630+0000 7f87e7400640 1 -- 172.21.15.119:0/881254888 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87d8022030 con 0x7f87f00d05b0 2024-04-25T10:07:14.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.636+0000 7f754fc00640 1 Processor -- start 2024-04-25T10:07:14.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.637+0000 7f754fc00640 1 -- start start 2024-04-25T10:07:14.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.638+0000 7f754fc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7548161090 0x7f7548161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.639+0000 7f754fc00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7548161f80 con 0x7f75480ceeb0 2024-04-25T10:07:14.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.639+0000 7f754fc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7548162260 con 0x7f75480b9720 2024-04-25T10:07:14.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.639+0000 7f754fc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75480d21a0 con 0x7f7548161090 2024-04-25T10:07:14.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.639+0000 7f754dc00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f75480ceeb0 0x7f75480d2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42162/0 (socket says 172.21.15.119:42162) 2024-04-25T10:07:14.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.639+0000 7f754dc00640 1 -- 172.21.15.119:0/3772397926 learned_addr learned my addr 172.21.15.119:0/3772397926 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:14.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f754d200640 1 --2- 172.21.15.119:0/3772397926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7548161090 0x7f7548161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 -- 172.21.15.119:0/3772397926 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081553634 0 0) 0x7f7548161f80 con 0x7f75480ceeb0 2024-04-25T10:07:14.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 -- 172.21.15.119:0/3772397926 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7530003520 con 0x7f75480ceeb0 2024-04-25T10:07:14.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 -- 172.21.15.119:0/3772397926 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 814380216 0 0) 0x7f7530003520 con 0x7f75480ceeb0 2024-04-25T10:07:14.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 -- 172.21.15.119:0/3772397926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7548161090 msgr2=0x7f7548161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 --2- 172.21.15.119:0/3772397926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7548161090 0x7f7548161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 -- 172.21.15.119:0/3772397926 >> v1:172.21.15.132:6789/0 conn(0x7f75480b9720 legacy=0x7f75480ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 -- 172.21.15.119:0/3772397926 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7548162540 con 0x7f75480ceeb0 2024-04-25T10:07:14.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 -- 172.21.15.119:0/3772397926 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f75400019f0 con 0x7f75480ceeb0 2024-04-25T10:07:14.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 -- 172.21.15.119:0/3772397926 >> v1:172.21.15.119:6789/0 conn(0x7f75480ceeb0 legacy=0x7f75480d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 --2- 172.21.15.119:0/3772397926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75300039e0 0x7f7530003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 --2- 172.21.15.119:0/3772397926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7530004800 0x7f7530004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 --2- 172.21.15.119:0/3772397926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75300051c0 0x7f753000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 -- 172.21.15.119:0/3772397926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75480d21a0 con 0x7f75300039e0 2024-04-25T10:07:14.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 -- 172.21.15.119:0/3772397926 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753000c030 con 0x7f75300051c0 2024-04-25T10:07:14.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.640+0000 7f7547400640 1 -- 172.21.15.119:0/3772397926 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753000c230 con 0x7f7530004800 2024-04-25T10:07:14.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.641+0000 7f754d200640 1 --2- 172.21.15.119:0/3772397926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75300039e0 0x7f7530003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.641+0000 7f754dc00640 1 --2- 172.21.15.119:0/3772397926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75300051c0 0x7f753000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.641+0000 7f754d200640 1 -- 172.21.15.119:0/3772397926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7530004800 msgr2=0x7f7530004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.641+0000 7f754d200640 1 --2- 172.21.15.119:0/3772397926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7530004800 0x7f7530004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.641+0000 7f754d200640 1 -- 172.21.15.119:0/3772397926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75300051c0 msgr2=0x7f753000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.641+0000 7f754d200640 1 --2- 172.21.15.119:0/3772397926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75300051c0 0x7f753000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.641+0000 7f754d200640 1 -- 172.21.15.119:0/3772397926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7530003520 con 0x7f75300039e0 2024-04-25T10:07:14.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.643+0000 7f754d200640 1 --2- 172.21.15.119:0/3772397926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75300039e0 0x7f7530003e20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f75480d4b40 tx=0x7f7530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5aca2dbdb406f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.644+0000 7f7547400640 1 -- 172.21.15.119:0/3772397926 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c036070 con 0x7f75300039e0 2024-04-25T10:07:14.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.644+0000 7f8b52000640 1 -- 172.21.15.119:0/1576953786 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b5407a350 con 0x7f8b5c0d2fd0 2024-04-25T10:07:14.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.645+0000 7f37dfe00640 1 -- 172.21.15.119:0/2545386218 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e000b6c0 con 0x7f37e80519b0 2024-04-25T10:07:14.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.645+0000 7f37dfe00640 1 -- 172.21.15.119:0/2545386218 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e0019070 con 0x7f37e80519b0 2024-04-25T10:07:14.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.645+0000 7f37dfe00640 1 -- 172.21.15.119:0/2545386218 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e000ba80 con 0x7f37e80519b0 2024-04-25T10:07:14.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.644+0000 7f7547400640 1 -- 172.21.15.119:0/3772397926 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c031400 con 0x7f75300039e0 2024-04-25T10:07:14.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.644+0000 7f7547400640 1 -- 172.21.15.119:0/3772397926 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c009630 con 0x7f75300039e0 2024-04-25T10:07:14.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.645+0000 7f754fc00640 1 -- 172.21.15.119:0/3772397926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75300039e0 msgr2=0x7f7530003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.645+0000 7f754fc00640 1 --2- 172.21.15.119:0/3772397926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75300039e0 0x7f7530003e20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f75480d4b40 tx=0x7f7530005830 comp rx=0 tx=0).stop 2024-04-25T10:07:14.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.645+0000 7f754dc00640 1 -- 172.21.15.119:0/3772397926 reap_dead start 2024-04-25T10:07:14.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.646+0000 7f754fc00640 1 -- 172.21.15.119:0/3772397926 shutdown_connections 2024-04-25T10:07:14.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.646+0000 7f754fc00640 1 -- 172.21.15.119:0/3772397926 >> 172.21.15.119:0/3772397926 conn(0x7f75480b55e0 msgr2=0x7f75480b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.646+0000 7f754fc00640 1 -- 172.21.15.119:0/3772397926 shutdown_connections 2024-04-25T10:07:14.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.646+0000 7f754fc00640 1 -- 172.21.15.119:0/3772397926 wait complete. 2024-04-25T10:07:14.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.645+0000 7f37f5600640 1 -- 172.21.15.119:0/2545386218 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e8046230 con 0x7f37e80519b0 2024-04-25T10:07:14.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.645+0000 7f37f5600640 1 -- 172.21.15.119:0/2545386218 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e80467f0 con 0x7f37e80519b0 2024-04-25T10:07:14.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.648+0000 7f37f5600640 1 -- 172.21.15.119:0/2545386218 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e80064d0 con 0x7f37e80519b0 2024-04-25T10:07:14.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.649+0000 7f754fc00640 1 Processor -- start 2024-04-25T10:07:14.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.649+0000 7f754fc00640 1 -- start start 2024-04-25T10:07:14.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.651+0000 7f754fc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75300039e0 0x7f75480c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.651+0000 7f754fc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75480c7990 0x7f75481f7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.652+0000 7f754fc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75481f7d50 0x7f75481f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.650+0000 7f37dfe00640 1 -- 172.21.15.119:0/2545386218 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37e000bc20 con 0x7f37e80519b0 2024-04-25T10:07:14.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.652+0000 7f37dfe00640 1 --2- 172.21.15.119:0/2545386218 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f37d406b5d0 0x7f37d406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.652+0000 7f37dfe00640 1 -- 172.21.15.119:0/2545386218 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37e008b020 con 0x7f37e80519b0 2024-04-25T10:07:14.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.652+0000 7f754fc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7548161730 con 0x7f75480c7990 2024-04-25T10:07:14.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.652+0000 7f754fc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7548161470 con 0x7f75300039e0 2024-04-25T10:07:14.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.652+0000 7f754fc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75481618b0 con 0x7f75481f7d50 2024-04-25T10:07:14.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.653+0000 7f754dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75481f7d50 0x7f75481f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.653+0000 7f37ef400640 1 --2- 172.21.15.119:0/2545386218 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f37d406b5d0 0x7f37d406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.653+0000 7f754dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75481f7d50 0x7f75481f81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49262/0 (socket says 172.21.15.119:49262) 2024-04-25T10:07:14.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.653+0000 7f754dc00640 1 -- 172.21.15.119:0/2123915632 learned_addr learned my addr 172.21.15.119:0/2123915632 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:14.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.653+0000 7f754dc00640 1 -- 172.21.15.119:0/2123915632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75300039e0 msgr2=0x7f75480c7450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:07:14.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.653+0000 7f754dc00640 1 --2- 172.21.15.119:0/2123915632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75300039e0 0x7f75480c7450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.653+0000 7f754dc00640 1 -- 172.21.15.119:0/2123915632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75480c7990 msgr2=0x7f75481f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.654+0000 7f7547e00640 1 --2- 172.21.15.119:0/2123915632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75480c7990 0x7f75481f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.654+0000 7f37ef400640 1 --2- 172.21.15.119:0/2545386218 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f37d406b5d0 0x7f37d406da90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f37e4034000 tx=0x7f37e402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.654+0000 7f754dc00640 1 --2- 172.21.15.119:0/2123915632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75480c7990 0x7f75481f7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.654+0000 7f754dc00640 1 -- 172.21.15.119:0/2123915632 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75481e7d60 con 0x7f75481f7d50 2024-04-25T10:07:14.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.654+0000 7f7547e00640 1 --2- 172.21.15.119:0/2123915632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75480c7990 0x7f75481f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:14.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.655+0000 7f754dc00640 1 --2- 172.21.15.119:0/2123915632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75481f7d50 0x7f75481f81d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f75480c7420 tx=0x7f7540007b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.655+0000 7f7544c00640 1 -- 172.21.15.119:0/2123915632 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754000eaa0 con 0x7f75481f7d50 2024-04-25T10:07:14.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.655+0000 7f754fc00640 1 -- 172.21.15.119:0/2123915632 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7548142510 con 0x7f75481f7d50 2024-04-25T10:07:14.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.655+0000 7f7544c00640 1 -- 172.21.15.119:0/2123915632 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7540018070 con 0x7f75481f7d50 2024-04-25T10:07:14.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.655+0000 7f7544c00640 1 -- 172.21.15.119:0/2123915632 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754000ee60 con 0x7f75481f7d50 2024-04-25T10:07:14.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.655+0000 7f754fc00640 1 -- 172.21.15.119:0/2123915632 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75481427f0 con 0x7f75481f7d50 2024-04-25T10:07:14.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.658+0000 7f754fc00640 1 -- 172.21.15.119:0/2123915632 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7524005290 con 0x7f75481f7d50 2024-04-25T10:07:14.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.660+0000 7f7544c00640 1 -- 172.21.15.119:0/2123915632 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f754001d460 con 0x7f75481f7d50 2024-04-25T10:07:14.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.661+0000 7f7544c00640 1 --2- 172.21.15.119:0/2123915632 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f752806b4d0 0x7f752806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.661+0000 7f7544c00640 1 -- 172.21.15.119:0/2123915632 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f754008a4a0 con 0x7f75481f7d50 2024-04-25T10:07:14.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.663+0000 7f754d200640 1 --2- 172.21.15.119:0/2123915632 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f752806b4d0 0x7f752806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.664+0000 7f754d200640 1 --2- 172.21.15.119:0/2123915632 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f752806b4d0 0x7f752806d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f753c033000 tx=0x7f753c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.668+0000 7f37dfe00640 1 -- 172.21.15.119:0/2545386218 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37e003c030 con 0x7f37e80519b0 2024-04-25T10:07:14.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:14.683+0000 7f7544c00640 1 -- 172.21.15.119:0/2123915632 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7540059310 con 0x7f75481f7d50 2024-04-25T10:07:15.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.013+0000 7f84ba200640 1 -- 172.21.15.119:0/3499157082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8490005850 con 0x7f84ac03a5d0 2024-04-25T10:07:15.021 INFO:teuthology.orchestra.run.smithi119.stdout:55834574853 2024-04-25T10:07:15.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.014+0000 7f84b0c00640 1 -- 172.21.15.119:0/3499157082 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f84a80794c0 con 0x7f84ac03a5d0 2024-04-25T10:07:15.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.018+0000 7f84ba200640 1 -- 172.21.15.119:0/3499157082 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f849406b700 msgr2=0x7f849406dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.018+0000 7f84ba200640 1 --2- 172.21.15.119:0/3499157082 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f849406b700 0x7f849406dbc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f84a0000f60 tx=0x7f84a00079b0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.018+0000 7f84ba200640 1 -- 172.21.15.119:0/3499157082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84ac03a5d0 msgr2=0x7f84ac0e3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.018+0000 7f84ba200640 1 --2- 172.21.15.119:0/3499157082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84ac03a5d0 0x7f84ac0e3040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f84a8000c00 tx=0x7f84a80046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.019+0000 7f84ba200640 1 -- 172.21.15.119:0/3499157082 shutdown_connections 2024-04-25T10:07:15.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.019+0000 7f84ba200640 1 --2- 172.21.15.119:0/3499157082 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f849406b700 0x7f849406dbc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.019+0000 7f84ba200640 1 --2- 172.21.15.119:0/3499157082 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84ac0e1090 0x7f84ac04bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.019+0000 7f84ba200640 1 --2- 172.21.15.119:0/3499157082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84ac050dd0 0x7f84ac04e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.019+0000 7f84ba200640 1 --2- 172.21.15.119:0/3499157082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84ac03a5d0 0x7f84ac0e3040 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.019+0000 7f84ba200640 1 -- 172.21.15.119:0/3499157082 >> 172.21.15.119:0/3499157082 conn(0x7f84ac009fb0 msgr2=0x7f84ac035ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.020+0000 7f84ba200640 1 -- 172.21.15.119:0/3499157082 shutdown_connections 2024-04-25T10:07:15.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.020+0000 7f84ba200640 1 -- 172.21.15.119:0/3499157082 wait complete. 2024-04-25T10:07:15.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-04-25T10:07:15.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.048+0000 7f11e7400640 1 -- 172.21.15.119:0/297328809 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f11e0074930 con 0x7f11e00b1740 2024-04-25T10:07:15.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.049+0000 7f738a000640 1 -- 172.21.15.119:0/2631389021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f73840d1790 con 0x7f73840b9410 2024-04-25T10:07:15.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.049+0000 7f11dcc00640 1 -- 172.21.15.119:0/297328809 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f11cc07cc90 con 0x7f11e00b1740 2024-04-25T10:07:15.055 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2024-04-25T10:07:15.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.049+0000 7f737b400640 1 -- 172.21.15.119:0/2631389021 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f737c0ab070 con 0x7f73840b9410 2024-04-25T10:07:15.058 INFO:teuthology.orchestra.run.smithi119.stdout:55834574853 2024-04-25T10:07:15.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f11e7400640 1 -- 172.21.15.119:0/297328809 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f11c006b5a0 msgr2=0x7f11c006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f11e7400640 1 --2- 172.21.15.119:0/297328809 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f11c006b5a0 0x7f11c006da60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f11e00b4070 tx=0x7f11d00095e0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f738a000640 1 -- 172.21.15.119:0/2631389021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f736406b4d0 msgr2=0x7f736406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f738a000640 1 --2- 172.21.15.119:0/2631389021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f736406b4d0 0x7f736406d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f73840cdbc0 tx=0x7f7374008040 comp rx=0 tx=0).stop 2024-04-25T10:07:15.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f738a000640 1 -- 172.21.15.119:0/2631389021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73840b9410 msgr2=0x7f7384158940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f738a000640 1 --2- 172.21.15.119:0/2631389021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73840b9410 0x7f7384158940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f737c009d90 tx=0x7f737c004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:15.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7fbe9f000640 1 -- 172.21.15.119:0/3458043096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbe70005850 con 0x7fbe981ffe70 2024-04-25T10:07:15.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f738a000640 1 -- 172.21.15.119:0/2631389021 shutdown_connections 2024-04-25T10:07:15.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f738a000640 1 --2- 172.21.15.119:0/2631389021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f736406b4d0 0x7f736406d990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f738a000640 1 --2- 172.21.15.119:0/2631389021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f738415dbe0 0x7f73840b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f738a000640 1 --2- 172.21.15.119:0/2631389021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73840d3580 0x7f738415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f738a000640 1 --2- 172.21.15.119:0/2631389021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73840b9410 0x7f7384158940 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f737c009d90 tx=0x7f737c004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:15.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f738a000640 1 -- 172.21.15.119:0/2631389021 >> 172.21.15.119:0/2631389021 conn(0x7f7384072900 msgr2=0x7f7384160910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f11e7400640 1 -- 172.21.15.119:0/297328809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e00b1740 msgr2=0x7f11e0157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f11e7400640 1 --2- 172.21.15.119:0/297328809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e00b1740 0x7f11e0157f00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f11cc037000 tx=0x7f11cc004730 comp rx=0 tx=0).stop 2024-04-25T10:07:15.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.053+0000 7f11e7400640 1 -- 172.21.15.119:0/297328809 shutdown_connections 2024-04-25T10:07:15.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.054+0000 7f738a000640 1 -- 172.21.15.119:0/2631389021 shutdown_connections 2024-04-25T10:07:15.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.054+0000 7f11e7400640 1 --2- 172.21.15.119:0/297328809 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f11c006b5a0 0x7f11c006da60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.054+0000 7f11e7400640 1 --2- 172.21.15.119:0/297328809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e0159910 0x7f11e0154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.054+0000 7f11e7400640 1 --2- 172.21.15.119:0/297328809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11e00cef80 0x7f11e00b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.054+0000 7f738a000640 1 -- 172.21.15.119:0/2631389021 wait complete. 2024-04-25T10:07:15.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.054+0000 7f11e7400640 1 --2- 172.21.15.119:0/297328809 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11e00b1740 0x7f11e0157f00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.054+0000 7fbe8fe00640 1 -- 172.21.15.119:0/3458043096 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbe9007e300 con 0x7fbe981ffe70 2024-04-25T10:07:15.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.054+0000 7f11e7400640 1 -- 172.21.15.119:0/297328809 >> 172.21.15.119:0/297328809 conn(0x7f11e006aba0 msgr2=0x7f11e00ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.068 INFO:teuthology.orchestra.run.smithi119.stdout:55834574853 2024-04-25T10:07:15.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.054+0000 7f11e7400640 1 -- 172.21.15.119:0/297328809 shutdown_connections 2024-04-25T10:07:15.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.054+0000 7f11e7400640 1 -- 172.21.15.119:0/297328809 wait complete. 2024-04-25T10:07:15.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.055+0000 7f87f5000640 1 -- 172.21.15.119:0/881254888 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f87f00ce800 con 0x7f87f00d05b0 2024-04-25T10:07:15.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.056+0000 7f87e7400640 1 -- 172.21.15.119:0/881254888 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f87d8059550 con 0x7f87f00d05b0 2024-04-25T10:07:15.071 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2024-04-25T10:07:15.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.057+0000 7fbe9f000640 1 -- 172.21.15.119:0/3458043096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe7806b550 msgr2=0x7fbe7806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.058+0000 7fbe9f000640 1 --2- 172.21.15.119:0/3458043096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe7806b550 0x7fbe7806da10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe980ce2c0 tx=0x7fbe88009040 comp rx=0 tx=0).stop 2024-04-25T10:07:15.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.058+0000 7fbe9f000640 1 -- 172.21.15.119:0/3458043096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe981ffe70 msgr2=0x7fbe980cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.058+0000 7fbe9f000640 1 --2- 172.21.15.119:0/3458043096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe981ffe70 0x7fbe980cc600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbe9002b580 tx=0x7fbe90004ab0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.058+0000 7fbe9f000640 1 -- 172.21.15.119:0/3458043096 shutdown_connections 2024-04-25T10:07:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.058+0000 7fbe9f000640 1 --2- 172.21.15.119:0/3458043096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe7806b550 0x7fbe7806da10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.058+0000 7fbe9f000640 1 --2- 172.21.15.119:0/3458043096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe980b3af0 0x7fbe980b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.058+0000 7fbe9f000640 1 --2- 172.21.15.119:0/3458043096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe980c9e50 0x7fbe980cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.058+0000 7fbe9f000640 1 --2- 172.21.15.119:0/3458043096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe981ffe70 0x7fbe980cc600 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.058+0000 7fbe9f000640 1 -- 172.21.15.119:0/3458043096 >> 172.21.15.119:0/3458043096 conn(0x7fbe980b55e0 msgr2=0x7fbe9815b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.059+0000 7fbe9f000640 1 -- 172.21.15.119:0/3458043096 shutdown_connections 2024-04-25T10:07:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.059+0000 7fbe9f000640 1 -- 172.21.15.119:0/3458043096 wait complete. 2024-04-25T10:07:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.059+0000 7f8b47400640 1 -- 172.21.15.119:0/1576953786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8b30005580 con 0x7f8b5c0d2fd0 2024-04-25T10:07:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.060+0000 7f8b52000640 1 -- 172.21.15.119:0/1576953786 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8b5407e3f0 con 0x7f8b5c0d2fd0 2024-04-25T10:07:15.076 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2024-04-25T10:07:15.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.062+0000 7f87f5000640 1 -- 172.21.15.119:0/881254888 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87cc06b520 msgr2=0x7f87cc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.062+0000 7f87f5000640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87cc06b520 0x7f87cc06d9e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f87f01fb5c0 tx=0x7f87e80039f0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.062+0000 7f87f5000640 1 -- 172.21.15.119:0/881254888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87f00d05b0 msgr2=0x7f87f00c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.062+0000 7f87f5000640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87f00d05b0 0x7f87f00c91e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f87d800d550 tx=0x7f87d800da10 comp rx=0 tx=0).stop 2024-04-25T10:07:15.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.062+0000 7f87f5000640 1 -- 172.21.15.119:0/881254888 shutdown_connections 2024-04-25T10:07:15.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.062+0000 7f87f5000640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87cc06b520 0x7f87cc06d9e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.062+0000 7f87f5000640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87f00d05b0 0x7f87f00c91e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.062+0000 7f87f5000640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87f00d01d0 0x7f87f015a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.062+0000 7f87f5000640 1 --2- 172.21.15.119:0/881254888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f00b5440 0x7f87f0157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.062+0000 7f87f5000640 1 -- 172.21.15.119:0/881254888 >> 172.21.15.119:0/881254888 conn(0x7f87f006bed0 msgr2=0x7f87f015d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.063+0000 7f87f5000640 1 -- 172.21.15.119:0/881254888 shutdown_connections 2024-04-25T10:07:15.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.063+0000 7f87f5000640 1 -- 172.21.15.119:0/881254888 wait complete. 2024-04-25T10:07:15.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.064+0000 7f8b47400640 1 -- 172.21.15.119:0/1576953786 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b3c06b4d0 msgr2=0x7f8b3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.064+0000 7f8b47400640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b3c06b4d0 0x7f8b3c06d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0099a0 tx=0x7f8b4c006f90 comp rx=0 tx=0).stop 2024-04-25T10:07:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.064+0000 7f8b47400640 1 -- 172.21.15.119:0/1576953786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b5c0d2fd0 msgr2=0x7f8b5c152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.064+0000 7f8b47400640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b5c0d2fd0 0x7f8b5c152a80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b54031000 tx=0x7f8b54004a70 comp rx=0 tx=0).stop 2024-04-25T10:07:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.065+0000 7f8b47400640 1 -- 172.21.15.119:0/1576953786 shutdown_connections 2024-04-25T10:07:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.065+0000 7f8b47400640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b3c06b4d0 0x7f8b3c06d990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.065+0000 7f8b47400640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b5c0d2fd0 0x7f8b5c152a80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.065+0000 7f8b47400640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b5c0b7e00 0x7f8b5c155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.065+0000 7f8b47400640 1 --2- 172.21.15.119:0/1576953786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b5c0b54a0 0x7f8b5c0cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.065+0000 7f8b47400640 1 -- 172.21.15.119:0/1576953786 >> 172.21.15.119:0/1576953786 conn(0x7f8b5c0d04a0 msgr2=0x7f8b5c0ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.066+0000 7f8b47400640 1 -- 172.21.15.119:0/1576953786 shutdown_connections 2024-04-25T10:07:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.067+0000 7f8b47400640 1 -- 172.21.15.119:0/1576953786 wait complete. 2024-04-25T10:07:15.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.072+0000 7f37f5600640 1 -- 172.21.15.119:0/2545386218 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f37e80decb0 con 0x7f37e80519b0 2024-04-25T10:07:15.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.073+0000 7f37dfe00640 1 -- 172.21.15.119:0/2545386218 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f37e0059f10 con 0x7f37e80519b0 2024-04-25T10:07:15.080 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2024-04-25T10:07:15.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.076+0000 7f37f5600640 1 -- 172.21.15.119:0/2545386218 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f37d406b5d0 msgr2=0x7f37d406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.076+0000 7f37f5600640 1 --2- 172.21.15.119:0/2545386218 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f37d406b5d0 0x7f37d406da90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f37e4034000 tx=0x7f37e402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:15.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.076+0000 7f37f5600640 1 -- 172.21.15.119:0/2545386218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37e80519b0 msgr2=0x7f37e804ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.076+0000 7f37f5600640 1 --2- 172.21.15.119:0/2545386218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37e80519b0 0x7f37e804ce60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f37e000d800 tx=0x7f37e000dc00 comp rx=0 tx=0).stop 2024-04-25T10:07:15.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.076+0000 7f37f5600640 1 -- 172.21.15.119:0/2545386218 shutdown_connections 2024-04-25T10:07:15.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.076+0000 7f37f5600640 1 --2- 172.21.15.119:0/2545386218 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f37d406b5d0 0x7f37d406da90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.076+0000 7f37f5600640 1 --2- 172.21.15.119:0/2545386218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e80e0e20 0x7f37e80dbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.076+0000 7f37f5600640 1 --2- 172.21.15.119:0/2545386218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37e8054d90 0x7f37e8046ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.076+0000 7f37f5600640 1 --2- 172.21.15.119:0/2545386218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37e80519b0 0x7f37e804ce60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.076+0000 7f37f5600640 1 -- 172.21.15.119:0/2545386218 >> 172.21.15.119:0/2545386218 conn(0x7f37e8053760 msgr2=0x7f37e80dd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.078+0000 7f37f5600640 1 -- 172.21.15.119:0/2545386218 shutdown_connections 2024-04-25T10:07:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.079+0000 7f37f5600640 1 -- 172.21.15.119:0/2545386218 wait complete. 2024-04-25T10:07:15.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-04-25T10:07:15.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-04-25T10:07:15.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-04-25T10:07:15.096 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-04-25T10:07:15.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-04-25T10:07:15.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.095+0000 7f754fc00640 1 -- 172.21.15.119:0/2123915632 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7524005580 con 0x7f75481f7d50 2024-04-25T10:07:15.102 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-04-25T10:07:15.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.096+0000 7f7544c00640 1 -- 172.21.15.119:0/2123915632 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7540058c10 con 0x7f75481f7d50 2024-04-25T10:07:15.103 INFO:teuthology.orchestra.run.smithi119.stdout:55834574853 2024-04-25T10:07:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.098+0000 7f754fc00640 1 -- 172.21.15.119:0/2123915632 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f752806b4d0 msgr2=0x7f752806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.098+0000 7f754fc00640 1 --2- 172.21.15.119:0/2123915632 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f752806b4d0 0x7f752806d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f753c033000 tx=0x7f753c008030 comp rx=0 tx=0).stop 2024-04-25T10:07:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.098+0000 7f754fc00640 1 -- 172.21.15.119:0/2123915632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75481f7d50 msgr2=0x7f75481f81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.098+0000 7f754fc00640 1 --2- 172.21.15.119:0/2123915632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75481f7d50 0x7f75481f81d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f75480c7420 tx=0x7f7540007b70 comp rx=0 tx=0).stop 2024-04-25T10:07:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.098+0000 7f754fc00640 1 -- 172.21.15.119:0/2123915632 shutdown_connections 2024-04-25T10:07:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.098+0000 7f754fc00640 1 --2- 172.21.15.119:0/2123915632 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f752806b4d0 0x7f752806d990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.098+0000 7f754fc00640 1 --2- 172.21.15.119:0/2123915632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75481f7d50 0x7f75481f81d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.098+0000 7f754fc00640 1 --2- 172.21.15.119:0/2123915632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75480c7990 0x7f75481f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.098+0000 7f754fc00640 1 --2- 172.21.15.119:0/2123915632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75300039e0 0x7f75480c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.098+0000 7f754fc00640 1 -- 172.21.15.119:0/2123915632 >> 172.21.15.119:0/2123915632 conn(0x7f75480b55e0 msgr2=0x7f754815b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.098+0000 7f754fc00640 1 -- 172.21.15.119:0/2123915632 shutdown_connections 2024-04-25T10:07:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:15.098+0000 7f754fc00640 1 -- 172.21.15.119:0/2123915632 wait complete. 2024-04-25T10:07:15.120 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-04-25T10:07:16.052 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:07:16.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:07:16.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:07:16.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:07:16.096 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:07:16.101 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:07:16.103 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:07:16.121 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:07:16.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.328+0000 7f8246600640 1 Processor -- start 2024-04-25T10:07:16.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.328+0000 7f8246600640 1 -- start start 2024-04-25T10:07:16.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.331+0000 7f8246600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8240159910 0x7f824015e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.331+0000 7f8246600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f82400cef80 0x7f8240071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.331+0000 7f8246600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f82400b1740 0x7f82400b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.332+0000 7f823f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8240159910 0x7f824015e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.332+0000 7f823f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8240159910 0x7f824015e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49272/0 (socket says 172.21.15.119:49272) 2024-04-25T10:07:16.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.332+0000 7f823f400640 1 -- 172.21.15.119:0/8766332 learned_addr learned my addr 172.21.15.119:0/8766332 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:16.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.332+0000 7f8246600640 1 -- 172.21.15.119:0/8766332 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82400b20e0 con 0x7f82400b1740 2024-04-25T10:07:16.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.333+0000 7f8246600640 1 -- 172.21.15.119:0/8766332 --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8240071bd0 con 0x7f82400cef80 2024-04-25T10:07:16.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.333+0000 7f8246600640 1 -- 172.21.15.119:0/8766332 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82400718f0 con 0x7f8240159910 2024-04-25T10:07:16.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.333+0000 7f823ea00640 1 --2- 172.21.15.119:0/8766332 >> v2:172.21.15.132:3300/0 conn(0x7f82400cef80 0x7f8240071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.333+0000 7f823fe00640 1 --2- 172.21.15.119:0/8766332 >> v2:172.21.15.119:3300/0 conn(0x7f82400b1740 0x7f82400b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.334+0000 7f823fe00640 1 -- 172.21.15.119:0/8766332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8240159910 msgr2=0x7f824015e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.334+0000 7f823fe00640 1 --2- 172.21.15.119:0/8766332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8240159910 0x7f824015e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.334+0000 7f823fe00640 1 -- 172.21.15.119:0/8766332 >> v2:172.21.15.132:3300/0 conn(0x7f82400cef80 msgr2=0x7f8240071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.335+0000 7f823fe00640 1 --2- 172.21.15.119:0/8766332 >> v2:172.21.15.132:3300/0 conn(0x7f82400cef80 0x7f8240071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.335+0000 7f823fe00640 1 -- 172.21.15.119:0/8766332 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824015d5d0 con 0x7f82400b1740 2024-04-25T10:07:16.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.336+0000 7f823ea00640 1 --2- 172.21.15.119:0/8766332 >> v2:172.21.15.132:3300/0 conn(0x7f82400cef80 0x7f8240071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:16.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.336+0000 7f823f400640 1 --2- 172.21.15.119:0/8766332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8240159910 0x7f824015e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:16.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.337+0000 7f823fe00640 1 --2- 172.21.15.119:0/8766332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82400b1740 0x7f82400b1ba0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f82400724f0 tx=0x7f823802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb70764a1b0dd307 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.337+0000 7f823e000640 1 -- 172.21.15.119:0/8766332 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238007740 con 0x7f82400b1740 2024-04-25T10:07:16.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.338+0000 7f823e000640 1 -- 172.21.15.119:0/8766332 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82380078e0 con 0x7f82400b1740 2024-04-25T10:07:16.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.339+0000 7f8246600640 1 -- 172.21.15.119:0/8766332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82400b1740 msgr2=0x7f82400b1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.339+0000 7f8246600640 1 --2- 172.21.15.119:0/8766332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82400b1740 0x7f82400b1ba0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f82400724f0 tx=0x7f823802a610 comp rx=0 tx=0).stop 2024-04-25T10:07:16.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.339+0000 7f8246600640 1 -- 172.21.15.119:0/8766332 shutdown_connections 2024-04-25T10:07:16.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.339+0000 7f8246600640 1 --2- 172.21.15.119:0/8766332 >> v2:172.21.15.132:3300/0 conn(0x7f82400cef80 0x7f8240071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.339+0000 7f8246600640 1 --2- 172.21.15.119:0/8766332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82400b1740 0x7f82400b1ba0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.339+0000 7f8246600640 1 --2- 172.21.15.119:0/8766332 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8240159910 0x7f824015e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.339+0000 7f8246600640 1 -- 172.21.15.119:0/8766332 >> 172.21.15.119:0/8766332 conn(0x7f824006aba0 msgr2=0x7f824006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.340+0000 7f8246600640 1 -- 172.21.15.119:0/8766332 shutdown_connections 2024-04-25T10:07:16.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.340+0000 7f8246600640 1 -- 172.21.15.119:0/8766332 wait complete. 2024-04-25T10:07:16.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.342+0000 7f8246600640 1 Processor -- start 2024-04-25T10:07:16.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.343+0000 7f8246600640 1 -- start start 2024-04-25T10:07:16.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.345+0000 7f8246600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82400b1740 0x7f82401f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.346+0000 7f8246600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82400cef80 0x7f8240152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.346+0000 7f823f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82400b1740 0x7f82401f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.346+0000 7f823f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82400b1740 0x7f82401f55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49282/0 (socket says 172.21.15.119:49282) 2024-04-25T10:07:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.346+0000 7f823ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82400cef80 0x7f8240152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.347+0000 7f823ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82400cef80 0x7f8240152be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34786/0 (socket says 172.21.15.119:34786) 2024-04-25T10:07:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.347+0000 7f823f400640 1 -- 172.21.15.119:0/628051869 learned_addr learned my addr 172.21.15.119:0/628051869 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.347+0000 7f8246600640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8240159910 0x7f82400b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.348+0000 7f8246600640 1 -- 172.21.15.119:0/628051869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8240071590 con 0x7f82400cef80 2024-04-25T10:07:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.348+0000 7f8246600640 1 -- 172.21.15.119:0/628051869 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824015db90 con 0x7f8240159910 2024-04-25T10:07:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.348+0000 7f8246600640 1 -- 172.21.15.119:0/628051869 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824006b6c0 con 0x7f82400b1740 2024-04-25T10:07:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.348+0000 7f823fe00640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8240159910 0x7f82400b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.349+0000 7f823ea00640 1 -- 172.21.15.119:0/628051869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82400b1740 msgr2=0x7f82401f55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.349+0000 7f823ea00640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82400b1740 0x7f82401f55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.350+0000 7f823ea00640 1 -- 172.21.15.119:0/628051869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8240159910 msgr2=0x7f82400b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.350+0000 7f823ea00640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8240159910 0x7f82400b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.350+0000 7f823ea00640 1 -- 172.21.15.119:0/628051869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8240153120 con 0x7f82400cef80 2024-04-25T10:07:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.350+0000 7f823fe00640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8240159910 0x7f82400b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.351+0000 7f823ea00640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82400cef80 0x7f8240152be0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f822c006fd0 tx=0x7f822c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.351+0000 7f8237400640 1 -- 172.21.15.119:0/628051869 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00f040 con 0x7f82400cef80 2024-04-25T10:07:16.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.352+0000 7f8237400640 1 -- 172.21.15.119:0/628051869 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c004030 con 0x7f82400cef80 2024-04-25T10:07:16.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.352+0000 7f8237400640 1 -- 172.21.15.119:0/628051869 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c0043e0 con 0x7f82400cef80 2024-04-25T10:07:16.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.352+0000 7f8246600640 1 -- 172.21.15.119:0/628051869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82401f4550 con 0x7f82400cef80 2024-04-25T10:07:16.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.352+0000 7f8246600640 1 -- 172.21.15.119:0/628051869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82401f4b10 con 0x7f82400cef80 2024-04-25T10:07:16.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.356+0000 7f8234c00640 1 -- 172.21.15.119:0/628051869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8214005290 con 0x7f82400cef80 2024-04-25T10:07:16.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.359+0000 7f8237400640 1 -- 172.21.15.119:0/628051869 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f822c00c6d0 con 0x7f82400cef80 2024-04-25T10:07:16.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.360+0000 7f8237400640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f822006b4d0 0x7f822006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.360+0000 7f8237400640 1 -- 172.21.15.119:0/628051869 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f822c089ed0 con 0x7f82400cef80 2024-04-25T10:07:16.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.363+0000 7f823f400640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f822006b4d0 0x7f822006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.368+0000 7f823f400640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f822006b4d0 0x7f822006d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82401ed060 tx=0x7f8230008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.369+0000 7f922f200640 1 Processor -- start 2024-04-25T10:07:16.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.369+0000 7f3acf200640 1 Processor -- start 2024-04-25T10:07:16.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.369+0000 7f3acf200640 1 -- start start 2024-04-25T10:07:16.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.369+0000 7f922f200640 1 -- start start 2024-04-25T10:07:16.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.370+0000 7f8237400640 1 -- 172.21.15.119:0/628051869 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f822c017020 con 0x7f82400cef80 2024-04-25T10:07:16.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.371+0000 7f922f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240e31c0 0x7f9224006c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.371+0000 7f922f200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f92240e7530 0x7f9224057b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.371+0000 7f922f200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92240076f0 con 0x7f9224007190 2024-04-25T10:07:16.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.371+0000 7f922f200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9224057670 con 0x7f92240e7530 2024-04-25T10:07:16.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.371+0000 7f922f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92240079d0 con 0x7f92240e31c0 2024-04-25T10:07:16.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.371+0000 7f3acf200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3ad015d920 0x7f3ad0071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.372+0000 7f3acf200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad006d780 0x7f3ad0071590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.372+0000 7f3acf200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad00b3960 con 0x7f3ad00adeb0 2024-04-25T10:07:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.372+0000 7f3acf200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad00b4850 con 0x7f3ad015d920 2024-04-25T10:07:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.372+0000 7f3acf200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad00b3c40 con 0x7f3ad006d780 2024-04-25T10:07:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.372+0000 7f3acde00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3ad015d920 0x7f3ad0071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.372+0000 7f3acde00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3ad015d920 0x7f3ad0071050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34802/0 (socket says 172.21.15.119:34802) 2024-04-25T10:07:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.372+0000 7f3acde00640 1 -- 172.21.15.119:0/2517499983 learned_addr learned my addr 172.21.15.119:0/2517499983 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.372+0000 7f922e800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f92240e7530 0x7f9224057b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.372+0000 7f922de00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240e31c0 0x7f9224006c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.372+0000 7f6ad3e00640 1 Processor -- start 2024-04-25T10:07:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.372+0000 7f3ace800640 1 --2- 172.21.15.119:0/2517499983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad006d780 0x7f3ad0071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.373+0000 7f3acde00640 1 -- 172.21.15.119:0/2517499983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad006d780 msgr2=0x7f3ad0071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.372+0000 7f922de00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240e31c0 0x7f9224006c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49284/0 (socket says 172.21.15.119:49284) 2024-04-25T10:07:16.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.373+0000 7f922de00640 1 -- 172.21.15.119:0/2709137355 learned_addr learned my addr 172.21.15.119:0/2709137355 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:16.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.373+0000 7f3acde00640 1 --2- 172.21.15.119:0/2517499983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad006d780 0x7f3ad0071590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.373+0000 7f3acde00640 1 -- 172.21.15.119:0/2517499983 >> v1:172.21.15.119:6789/0 conn(0x7f3ad00adeb0 legacy=0x7f3ad01615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.373+0000 7f3acde00640 1 -- 172.21.15.119:0/2517499983 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad00b3f20 con 0x7f3ad015d920 2024-04-25T10:07:16.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.373+0000 7f6ad3e00640 1 -- start start 2024-04-25T10:07:16.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.374+0000 7f922de00640 1 -- 172.21.15.119:0/2709137355 >> v2:172.21.15.119:3300/0 conn(0x7f92240e7530 msgr2=0x7f9224057b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.375+0000 7f3acde00640 1 --2- 172.21.15.119:0/2517499983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad015d920 0x7f3ad0071050 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009870 tx=0x7f3ab802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3e0d7d48d7807a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.374+0000 7f922de00640 1 --2- 172.21.15.119:0/2709137355 >> v2:172.21.15.119:3300/0 conn(0x7f92240e7530 0x7f9224057b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.374+0000 7f922de00640 1 -- 172.21.15.119:0/2709137355 >> v1:172.21.15.132:6789/0 conn(0x7f9224007190 legacy=0x7f92240e6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.374+0000 7f922de00640 1 -- 172.21.15.119:0/2709137355 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9224057300 con 0x7f92240e31c0 2024-04-25T10:07:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.375+0000 7f922e800640 1 --2- 172.21.15.119:0/2709137355 >> v2:172.21.15.119:3300/0 conn(0x7f92240e7530 0x7f9224057b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.375+0000 7f3acca00640 1 -- 172.21.15.119:0/2517499983 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8037070 con 0x7f3ad015d920 2024-04-25T10:07:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.376+0000 7f3acca00640 1 -- 172.21.15.119:0/2517499983 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab8032440 con 0x7f3ad015d920 2024-04-25T10:07:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.376+0000 7f3acca00640 1 -- 172.21.15.119:0/2517499983 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8032810 con 0x7f3ad015d920 2024-04-25T10:07:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.374+0000 7f6ad3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad40d01d0 0x7f6ad40cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.374+0000 7f6ad3e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6ad40d05b0 0x7f6ad415ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.374+0000 7f6ad3e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6ad40b5440 0x7f6ad40b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.374+0000 7f6ad3e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad4162160 con 0x7f6ad40d05b0 2024-04-25T10:07:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.374+0000 7f6ad3e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad41625f0 con 0x7f6ad40b5440 2024-04-25T10:07:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.374+0000 7f6ad3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad40cef60 con 0x7f6ad40d01d0 2024-04-25T10:07:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.376+0000 7f6ad2000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6ad40d05b0 0x7f6ad415ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.376+0000 7f6ad2000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6ad40d05b0 0x7f6ad415ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34812/0 (socket says 172.21.15.119:34812) 2024-04-25T10:07:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.376+0000 7f6ad2000640 1 -- 172.21.15.119:0/3698195431 learned_addr learned my addr 172.21.15.119:0/3698195431 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.376+0000 7f6ad2a00640 1 --2- 172.21.15.119:0/3698195431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad40d01d0 0x7f6ad40cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.376+0000 7f6ad3400640 1 --2- 172.21.15.119:0/3698195431 >> v2:172.21.15.132:3300/0 conn(0x7f6ad40b5440 0x7f6ad40b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f3acf200640 1 -- 172.21.15.119:0/2517499983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad015d920 msgr2=0x7f3ad0071050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f3acf200640 1 --2- 172.21.15.119:0/2517499983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad015d920 0x7f3ad0071050 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009870 tx=0x7f3ab802a210 comp rx=0 tx=0).stop 2024-04-25T10:07:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f6ad3400640 1 -- 172.21.15.119:0/3698195431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad40d01d0 msgr2=0x7f6ad40cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f3acf200640 1 -- 172.21.15.119:0/2517499983 shutdown_connections 2024-04-25T10:07:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f3acf200640 1 --2- 172.21.15.119:0/2517499983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad006d780 0x7f3ad0071590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f3acf200640 1 --2- 172.21.15.119:0/2517499983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad015d920 0x7f3ad0071050 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f3acf200640 1 -- 172.21.15.119:0/2517499983 >> 172.21.15.119:0/2517499983 conn(0x7f3ad00b1350 msgr2=0x7f3ad00b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f922de00640 1 --2- 172.21.15.119:0/2709137355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240e31c0 0x7f9224006c50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f921c009870 tx=0x7f921c02a310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50ed616e5d7b0f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f3acf200640 1 -- 172.21.15.119:0/2517499983 shutdown_connections 2024-04-25T10:07:16.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f3acf200640 1 -- 172.21.15.119:0/2517499983 wait complete. 2024-04-25T10:07:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f6ad3400640 1 --2- 172.21.15.119:0/3698195431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad40d01d0 0x7f6ad40cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f6ad3400640 1 -- 172.21.15.119:0/3698195431 >> v2:172.21.15.119:3300/0 conn(0x7f6ad40d05b0 msgr2=0x7f6ad415ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f6ad3400640 1 --2- 172.21.15.119:0/3698195431 >> v2:172.21.15.119:3300/0 conn(0x7f6ad40d05b0 0x7f6ad415ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f6ad3400640 1 -- 172.21.15.119:0/3698195431 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad40cf200 con 0x7f6ad40b5440 2024-04-25T10:07:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.378+0000 7f6ad2000640 1 --2- 172.21.15.119:0/3698195431 >> v2:172.21.15.119:3300/0 conn(0x7f6ad40d05b0 0x7f6ad415ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.377+0000 7f922ca00640 1 -- 172.21.15.119:0/2709137355 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c037070 con 0x7f92240e31c0 2024-04-25T10:07:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.379+0000 7f922ca00640 1 -- 172.21.15.119:0/2709137355 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c032640 con 0x7f92240e31c0 2024-04-25T10:07:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.379+0000 7f922ca00640 1 -- 172.21.15.119:0/2709137355 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c02ae60 con 0x7f92240e31c0 2024-04-25T10:07:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.379+0000 7f6ad2a00640 1 --2- 172.21.15.119:0/3698195431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad40d01d0 0x7f6ad40cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.379+0000 7f922f200640 1 -- 172.21.15.119:0/2709137355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240e31c0 msgr2=0x7f9224006c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.379+0000 7f922f200640 1 --2- 172.21.15.119:0/2709137355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240e31c0 0x7f9224006c50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f921c009870 tx=0x7f921c02a310 comp rx=0 tx=0).stop 2024-04-25T10:07:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.380+0000 7f922f200640 1 -- 172.21.15.119:0/2709137355 shutdown_connections 2024-04-25T10:07:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.380+0000 7f3acf200640 1 Processor -- start 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.380+0000 7f922f200640 1 --2- 172.21.15.119:0/2709137355 >> v2:172.21.15.119:3300/0 conn(0x7f92240e7530 0x7f9224057b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.380+0000 7f922f200640 1 --2- 172.21.15.119:0/2709137355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240e31c0 0x7f9224006c50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.380+0000 7f922f200640 1 -- 172.21.15.119:0/2709137355 >> 172.21.15.119:0/2709137355 conn(0x7f922403aa30 msgr2=0x7f922403ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.380+0000 7f6ad3400640 1 --2- 172.21.15.119:0/3698195431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad40b5440 0x7f6ad40b58d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40d1520 tx=0x7f6acc02a580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a84532ea0d2de81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.380+0000 7f922f200640 1 -- 172.21.15.119:0/2709137355 shutdown_connections 2024-04-25T10:07:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.380+0000 7f3acf200640 1 -- start start 2024-04-25T10:07:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.380+0000 7f922f200640 1 -- 172.21.15.119:0/2709137355 wait complete. 2024-04-25T10:07:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.381+0000 7f6ad1600640 1 -- 172.21.15.119:0/3698195431 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc007740 con 0x7f6ad40b5440 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.381+0000 7f6ad1600640 1 -- 172.21.15.119:0/3698195431 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc0078e0 con 0x7f6ad40b5440 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.381+0000 7f6ad1600640 1 -- 172.21.15.119:0/3698195431 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc007cf0 con 0x7f6ad40b5440 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.382+0000 7f6ad3e00640 1 -- 172.21.15.119:0/3698195431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad40b5440 msgr2=0x7f6ad40b58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.382+0000 7f6ad3e00640 1 --2- 172.21.15.119:0/3698195431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad40b5440 0x7f6ad40b58d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40d1520 tx=0x7f6acc02a580 comp rx=0 tx=0).stop 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.382+0000 7f922f200640 1 Processor -- start 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.382+0000 7f645a600640 1 Processor -- start 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.382+0000 7f645a600640 1 -- start start 2024-04-25T10:07:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.383+0000 7f922f200640 1 -- start start 2024-04-25T10:07:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.383+0000 7f3acf200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ad006d780 0x7f3ad014a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.383+0000 7f3acf200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad00adeb0 0x7f3ad014ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.383+0000 7f3acf200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad015d920 0x7f3ad01f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.383+0000 7f3acf200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad015dd00 con 0x7f3ad015d920 2024-04-25T10:07:16.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.383+0000 7f3acf200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad00b44e0 con 0x7f3ad006d780 2024-04-25T10:07:16.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.383+0000 7f3acf200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad015de80 con 0x7f3ad00adeb0 2024-04-25T10:07:16.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.384+0000 7f3acde00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ad006d780 0x7f3ad014a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.384+0000 7f922f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9224007190 0x7f92240e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.384+0000 7f922f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92240e31c0 0x7f922404f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.384+0000 7f922f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240e7530 0x7f92240e5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.384+0000 7f922f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922403c240 con 0x7f92240e31c0 2024-04-25T10:07:16.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.384+0000 7f922f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922403c0c0 con 0x7f9224007190 2024-04-25T10:07:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.384+0000 7f922f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92240e35a0 con 0x7f92240e7530 2024-04-25T10:07:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.384+0000 7f3ace800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad015d920 0x7f3ad01f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.384+0000 7f3acde00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ad006d780 0x7f3ad014a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:50916/0 (socket says 172.21.15.119:50916) 2024-04-25T10:07:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.384+0000 7f922e800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240e7530 0x7f92240e5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.384+0000 7f3acde00640 1 -- 172.21.15.119:0/1580253965 learned_addr learned my addr 172.21.15.119:0/1580253965 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f645a600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645406b690 0x7f645415e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f645a600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f645406dfa0 0x7f6454071200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f922e800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240e7530 0x7f92240e5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49300/0 (socket says 172.21.15.119:49300) 2024-04-25T10:07:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f922e800640 1 -- 172.21.15.119:0/3615753806 learned_addr learned my addr 172.21.15.119:0/3615753806 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.384+0000 7f3acd400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad00adeb0 0x7f3ad014ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f645a600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6454071740 0x7f6454071ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f645a600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64540b3fd0 con 0x7f645406dfa0 2024-04-25T10:07:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f645a600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64540b3170 con 0x7f6454071740 2024-04-25T10:07:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f645a600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64540b33c0 con 0x7f645406b690 2024-04-25T10:07:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f6ad3e00640 1 -- 172.21.15.119:0/3698195431 shutdown_connections 2024-04-25T10:07:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f6ad3e00640 1 --2- 172.21.15.119:0/3698195431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad40b5440 0x7f6ad40b58d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f6ad3e00640 1 --2- 172.21.15.119:0/3698195431 >> v2:172.21.15.119:3300/0 conn(0x7f6ad40d05b0 0x7f6ad415ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f6ad3e00640 1 --2- 172.21.15.119:0/3698195431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad40d01d0 0x7f6ad40cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f6ad3e00640 1 -- 172.21.15.119:0/3698195431 >> 172.21.15.119:0/3698195431 conn(0x7f6ad406bed0 msgr2=0x7f6ad406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f3ace800640 1 -- 172.21.15.119:0/1580253965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad00adeb0 msgr2=0x7f3ad014ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.385+0000 7f922de00640 1 --2- 172.21.15.119:0/3615753806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9224007190 0x7f92240e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.386+0000 7f922d400640 1 --2- 172.21.15.119:0/3615753806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92240e31c0 0x7f922404f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.386+0000 7f3ace800640 1 --2- 172.21.15.119:0/1580253965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad00adeb0 0x7f3ad014ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.386+0000 7f3ace800640 1 -- 172.21.15.119:0/1580253965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ad006d780 msgr2=0x7f3ad014a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.386+0000 7f3ace800640 1 --2- 172.21.15.119:0/1580253965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ad006d780 0x7f3ad014a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.386+0000 7f3ace800640 1 -- 172.21.15.119:0/1580253965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad0156560 con 0x7f3ad015d920 2024-04-25T10:07:16.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.386+0000 7f3acd400640 1 --2- 172.21.15.119:0/1580253965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad00adeb0 0x7f3ad014ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:16.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.386+0000 7f3acde00640 1 --2- 172.21.15.119:0/1580253965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ad006d780 0x7f3ad014a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:16.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.387+0000 7f6ad3e00640 1 -- 172.21.15.119:0/3698195431 shutdown_connections 2024-04-25T10:07:16.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.386+0000 7f6453e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6454071740 0x7f6454071ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.387+0000 7f6452a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f645406dfa0 0x7f6454071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.387+0000 7f6453400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645406b690 0x7f645415e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.387+0000 7f6453e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6454071740 0x7f6454071ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:50944/0 (socket says 172.21.15.119:50944) 2024-04-25T10:07:16.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.387+0000 7f6453e00640 1 -- 172.21.15.119:0/2665894587 learned_addr learned my addr 172.21.15.119:0/2665894587 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:16.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.386+0000 7f922de00640 1 -- 172.21.15.119:0/3615753806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240e7530 msgr2=0x7f92240e5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.386+0000 7f922de00640 1 --2- 172.21.15.119:0/3615753806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240e7530 0x7f92240e5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.386+0000 7f922de00640 1 -- 172.21.15.119:0/3615753806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92240e31c0 msgr2=0x7f922404f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.386+0000 7f922de00640 1 --2- 172.21.15.119:0/3615753806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92240e31c0 0x7f922404f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.387+0000 7f922de00640 1 -- 172.21.15.119:0/3615753806 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922404f7d0 con 0x7f9224007190 2024-04-25T10:07:16.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.387+0000 7f3ace800640 1 --2- 172.21.15.119:0/1580253965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad015d920 0x7f3ad01f71d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800c590 tx=0x7f3ac800ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.387+0000 7f35a2000640 1 Processor -- start 2024-04-25T10:07:16.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.387+0000 7f3ac6000640 1 -- 172.21.15.119:0/1580253965 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8015070 con 0x7f3ad015d920 2024-04-25T10:07:16.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.387+0000 7f922de00640 1 --2- 172.21.15.119:0/3615753806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9224007190 0x7f92240e39c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f921c031000 tx=0x7f921c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.388+0000 7f35a2000640 1 -- start start 2024-04-25T10:07:16.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.388+0000 7f3acf200640 1 -- 172.21.15.119:0/1580253965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad0156840 con 0x7f3ad015d920 2024-04-25T10:07:16.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.388+0000 7f6453400640 1 -- 172.21.15.119:0/2665894587 >> v2:172.21.15.132:3300/0 conn(0x7f6454071740 msgr2=0x7f6454071ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.388+0000 7f3ac6000640 1 -- 172.21.15.119:0/1580253965 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac8004220 con 0x7f3ad015d920 2024-04-25T10:07:16.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.388+0000 7f3ac6000640 1 -- 172.21.15.119:0/1580253965 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8015070 con 0x7f3ad015d920 2024-04-25T10:07:16.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.388+0000 7f6453400640 1 --2- 172.21.15.119:0/2665894587 >> v2:172.21.15.132:3300/0 conn(0x7f6454071740 0x7f6454071ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.388+0000 7f6ad3e00640 1 -- 172.21.15.119:0/3698195431 wait complete. 2024-04-25T10:07:16.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.388+0000 7f9222000640 1 -- 172.21.15.119:0/3615753806 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c041020 con 0x7f9224007190 2024-04-25T10:07:16.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.388+0000 7f6453400640 1 -- 172.21.15.119:0/2665894587 >> v2:172.21.15.119:3300/0 conn(0x7f645406dfa0 msgr2=0x7f6454071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.389+0000 7f9222000640 1 -- 172.21.15.119:0/3615753806 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c0309b0 con 0x7f9224007190 2024-04-25T10:07:16.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.389+0000 7f9222000640 1 -- 172.21.15.119:0/3615753806 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c037040 con 0x7f9224007190 2024-04-25T10:07:16.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.389+0000 7f3acf200640 1 -- 172.21.15.119:0/1580253965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad0156e00 con 0x7f3ad015d920 2024-04-25T10:07:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.389+0000 7f35a2000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359c06b2b0 0x7f359c0cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.389+0000 7f35a2000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f359c0cc810 0x7f359c0b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.390+0000 7f35a2000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c15f400 con 0x7f359c1450e0 2024-04-25T10:07:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.390+0000 7f35a2000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359c1454c0 con 0x7f359c0cc810 2024-04-25T10:07:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.390+0000 7f35a2000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c15f6e0 con 0x7f359c06b2b0 2024-04-25T10:07:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.390+0000 7f6453400640 1 --2- 172.21.15.119:0/2665894587 >> v2:172.21.15.119:3300/0 conn(0x7f645406dfa0 0x7f6454071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.390+0000 7f6453400640 1 -- 172.21.15.119:0/2665894587 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64540b36a0 con 0x7f645406b690 2024-04-25T10:07:16.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.390+0000 7f35a0c00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f359c1450e0 0x7f359c06aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:41434/0 (socket says 172.21.15.119:41434) 2024-04-25T10:07:16.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.390+0000 7f35a0c00640 1 -- 172.21.15.119:0/2154627496 learned_addr learned my addr 172.21.15.119:0/2154627496 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:16.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.390+0000 7f359be00640 1 --2- 172.21.15.119:0/2154627496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359c06b2b0 0x7f359c0cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.390+0000 7f6453e00640 1 --2- 172.21.15.119:0/2665894587 >> v2:172.21.15.132:3300/0 conn(0x7f6454071740 0x7f6454071ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:16.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.390+0000 7f6452a00640 1 --2- 172.21.15.119:0/2665894587 >> v2:172.21.15.119:3300/0 conn(0x7f645406dfa0 0x7f6454071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:16.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.391+0000 7f359b400640 1 -- 172.21.15.119:0/2154627496 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972698365 0 0) 0x7f359c15f400 con 0x7f359c1450e0 2024-04-25T10:07:16.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.391+0000 7f359b400640 1 -- 172.21.15.119:0/2154627496 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3584003520 con 0x7f359c1450e0 2024-04-25T10:07:16.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.391+0000 7f3acf200640 1 -- 172.21.15.119:0/1580253965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad00749b0 con 0x7f3ad015d920 2024-04-25T10:07:16.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.391+0000 7f922f200640 1 -- 172.21.15.119:0/3615753806 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922416e630 con 0x7f9224007190 2024-04-25T10:07:16.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.391+0000 7f6ad3e00640 1 Processor -- start 2024-04-25T10:07:16.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.392+0000 7f922f200640 1 -- 172.21.15.119:0/3615753806 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922404ce40 con 0x7f9224007190 2024-04-25T10:07:16.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.391+0000 7f359be00640 1 -- 172.21.15.119:0/2154627496 >> v2:172.21.15.119:3300/0 conn(0x7f359c0cc810 msgr2=0x7f359c0b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.391+0000 7f359be00640 1 --2- 172.21.15.119:0/2154627496 >> v2:172.21.15.119:3300/0 conn(0x7f359c0cc810 0x7f359c0b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.391+0000 7f359be00640 1 -- 172.21.15.119:0/2154627496 >> v1:172.21.15.132:6789/0 conn(0x7f359c1450e0 legacy=0x7f359c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.391+0000 7f359be00640 1 -- 172.21.15.119:0/2154627496 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c15f9c0 con 0x7f359c06b2b0 2024-04-25T10:07:16.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.392+0000 7f6453400640 1 --2- 172.21.15.119:0/2665894587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645406b690 0x7f645415e060 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f645406dce0 tx=0x7f644c02e380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e124590aae2c66b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.393+0000 7f6452000640 1 -- 172.21.15.119:0/2665894587 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c03b070 con 0x7f645406b690 2024-04-25T10:07:16.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.393+0000 7f6452000640 1 -- 172.21.15.119:0/2665894587 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c036640 con 0x7f645406b690 2024-04-25T10:07:16.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.393+0000 7f6452000640 1 -- 172.21.15.119:0/2665894587 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c036a10 con 0x7f645406b690 2024-04-25T10:07:16.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.393+0000 7f359be00640 1 --2- 172.21.15.119:0/2154627496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359c06b2b0 0x7f359c0cc2d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f35880097a0 tx=0x7f358802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c6eb36bfdd1d8b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.393+0000 7f6ad3e00640 1 -- start start 2024-04-25T10:07:16.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.394+0000 7f922f200640 1 -- 172.21.15.119:0/3615753806 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92240057e0 con 0x7f9224007190 2024-04-25T10:07:16.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.394+0000 7f645a600640 1 -- 172.21.15.119:0/2665894587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645406b690 msgr2=0x7f645415e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.394+0000 7f645a600640 1 --2- 172.21.15.119:0/2665894587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645406b690 0x7f645415e060 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f645406dce0 tx=0x7f644c02e380 comp rx=0 tx=0).stop 2024-04-25T10:07:16.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.395+0000 7f3ac6000640 1 -- 172.21.15.119:0/1580253965 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ac8011600 con 0x7f3ad015d920 2024-04-25T10:07:16.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.395+0000 7f359b400640 1 -- 172.21.15.119:0/2154627496 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3588037070 con 0x7f359c06b2b0 2024-04-25T10:07:16.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.395+0000 7f359b400640 1 -- 172.21.15.119:0/2154627496 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358802add0 con 0x7f359c06b2b0 2024-04-25T10:07:16.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.395+0000 7f359b400640 1 -- 172.21.15.119:0/2154627496 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3588032660 con 0x7f359c06b2b0 2024-04-25T10:07:16.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.396+0000 7f9222000640 1 -- 172.21.15.119:0/3615753806 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f921c044020 con 0x7f9224007190 2024-04-25T10:07:16.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.396+0000 7f3ac6000640 1 --2- 172.21.15.119:0/1580253965 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3aac06b6d0 0x7f3aac06db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.396+0000 7f35a2000640 1 -- 172.21.15.119:0/2154627496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359c06b2b0 msgr2=0x7f359c0cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.395+0000 7f6ad3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad40b5440 0x7f6ad40c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.395+0000 7f6ad3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ad40d01d0 0x7f6ad40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.395+0000 7f6ad3e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad40d05b0 0x7f6ad40cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f35a2000640 1 --2- 172.21.15.119:0/2154627496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359c06b2b0 0x7f359c0cc2d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f35880097a0 tx=0x7f358802a080 comp rx=0 tx=0).stop 2024-04-25T10:07:16.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.395+0000 7f6ad3e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad4161df0 con 0x7f6ad40d01d0 2024-04-25T10:07:16.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.395+0000 7f6ad3e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad4161b10 con 0x7f6ad40d05b0 2024-04-25T10:07:16.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.395+0000 7f6ad3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad415ef40 con 0x7f6ad40b5440 2024-04-25T10:07:16.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.396+0000 7f6ad2a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad40b5440 0x7f6ad40c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.396+0000 7f6ad2a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad40b5440 0x7f6ad40c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49328/0 (socket says 172.21.15.119:49328) 2024-04-25T10:07:16.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.396+0000 7f6ad2a00640 1 -- 172.21.15.119:0/4014283096 learned_addr learned my addr 172.21.15.119:0/4014283096 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:16.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f3ac6000640 1 -- 172.21.15.119:0/1580253965 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ac808a040 con 0x7f3ad015d920 2024-04-25T10:07:16.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f645a600640 1 -- 172.21.15.119:0/2665894587 shutdown_connections 2024-04-25T10:07:16.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f645a600640 1 --2- 172.21.15.119:0/2665894587 >> v2:172.21.15.132:3300/0 conn(0x7f6454071740 0x7f6454071ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f645a600640 1 --2- 172.21.15.119:0/2665894587 >> v2:172.21.15.119:3300/0 conn(0x7f645406dfa0 0x7f6454071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f645a600640 1 --2- 172.21.15.119:0/2665894587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645406b690 0x7f645415e060 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f645a600640 1 -- 172.21.15.119:0/2665894587 >> 172.21.15.119:0/2665894587 conn(0x7f645406aba0 msgr2=0x7f645406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f6ad2a00640 1 -- 172.21.15.119:0/4014283096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad40d05b0 msgr2=0x7f6ad40cafd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:07:16.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f6ad2a00640 1 --2- 172.21.15.119:0/4014283096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad40d05b0 0x7f6ad40cafd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f6ad2a00640 1 -- 172.21.15.119:0/4014283096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ad40d01d0 msgr2=0x7f6ad40caa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:07:16.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f6ad2a00640 1 --2- 172.21.15.119:0/4014283096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ad40d01d0 0x7f6ad40caa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f6ad2a00640 1 -- 172.21.15.119:0/4014283096 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad40c8430 con 0x7f6ad40b5440 2024-04-25T10:07:16.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f3acde00640 1 --2- 172.21.15.119:0/1580253965 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3aac06b6d0 0x7f3aac06db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f645a600640 1 -- 172.21.15.119:0/2665894587 shutdown_connections 2024-04-25T10:07:16.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.397+0000 7f645a600640 1 -- 172.21.15.119:0/2665894587 wait complete. 2024-04-25T10:07:16.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.398+0000 7f6ad2a00640 1 --2- 172.21.15.119:0/4014283096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad40b5440 0x7f6ad40c6940 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400e840 tx=0x7f6ac400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.398+0000 7f6acaa00640 1 -- 172.21.15.119:0/4014283096 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac400f040 con 0x7f6ad40b5440 2024-04-25T10:07:16.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.398+0000 7f6ad3e00640 1 -- 172.21.15.119:0/4014283096 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad40bf670 con 0x7f6ad40b5440 2024-04-25T10:07:16.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.398+0000 7f6ad3e00640 1 -- 172.21.15.119:0/4014283096 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad40cda20 con 0x7f6ad40b5440 2024-04-25T10:07:16.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.399+0000 7f35a2000640 1 -- 172.21.15.119:0/2154627496 shutdown_connections 2024-04-25T10:07:16.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.399+0000 7f35a2000640 1 --2- 172.21.15.119:0/2154627496 >> v2:172.21.15.119:3300/0 conn(0x7f359c0cc810 0x7f359c0b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.399+0000 7f35a2000640 1 --2- 172.21.15.119:0/2154627496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359c06b2b0 0x7f359c0cc2d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.399+0000 7f35a2000640 1 -- 172.21.15.119:0/2154627496 >> 172.21.15.119:0/2154627496 conn(0x7f359c066ff0 msgr2=0x7f359c0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.399+0000 7f6acaa00640 1 -- 172.21.15.119:0/4014283096 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac4004030 con 0x7f6ad40b5440 2024-04-25T10:07:16.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.399+0000 7f6acaa00640 1 -- 172.21.15.119:0/4014283096 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac400f040 con 0x7f6ad40b5440 2024-04-25T10:07:16.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.399+0000 7f3acde00640 1 --2- 172.21.15.119:0/1580253965 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3aac06b6d0 0x7f3aac06db90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80090b0 tx=0x7f3ab802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.399+0000 7f3ac6000640 1 -- 172.21.15.119:0/1580253965 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ac8058eb0 con 0x7f3ad015d920 2024-04-25T10:07:16.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.399+0000 7f9222000640 1 --2- 172.21.15.119:0/3615753806 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f920c06b500 0x7f920c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.400+0000 7f9222000640 1 -- 172.21.15.119:0/3615753806 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f921c0aa790 con 0x7f9224007190 2024-04-25T10:07:16.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.400+0000 7f645a600640 1 Processor -- start 2024-04-25T10:07:16.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.400+0000 7f6ad3e00640 1 -- 172.21.15.119:0/4014283096 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aac005290 con 0x7f6ad40b5440 2024-04-25T10:07:16.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.400+0000 7f645a600640 1 -- start start 2024-04-25T10:07:16.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.400+0000 7f922d400640 1 --2- 172.21.15.119:0/3615753806 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f920c06b500 0x7f920c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.402+0000 7f922d400640 1 --2- 172.21.15.119:0/3615753806 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f920c06b500 0x7f920c06d9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f92240e4fd0 tx=0x7f9218009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.400+0000 7f35a2000640 1 -- 172.21.15.119:0/2154627496 shutdown_connections 2024-04-25T10:07:16.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.400+0000 7f35a2000640 1 -- 172.21.15.119:0/2154627496 wait complete. 2024-04-25T10:07:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.402+0000 7f645a600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f645406b690 0x7f64540cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.402+0000 7f645a600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645406dfa0 0x7f64541f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.402+0000 7f645a600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6454071740 0x7f64541f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.402+0000 7f645a600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645415e5a0 con 0x7f645406b690 2024-04-25T10:07:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.402+0000 7f645a600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64540b3c60 con 0x7f6454071740 2024-04-25T10:07:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.402+0000 7f645a600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645415d7d0 con 0x7f645406dfa0 2024-04-25T10:07:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.403+0000 7f6453400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f645406b690 0x7f64540cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.403+0000 7f6453400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f645406b690 0x7f64540cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34854/0 (socket says 172.21.15.119:34854) 2024-04-25T10:07:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.403+0000 7f6453400640 1 -- 172.21.15.119:0/2702489819 learned_addr learned my addr 172.21.15.119:0/2702489819 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.403+0000 7f6453e00640 1 --2- 172.21.15.119:0/2702489819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6454071740 0x7f64541f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.403+0000 7f6452a00640 1 --2- 172.21.15.119:0/2702489819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645406dfa0 0x7f64541f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.403+0000 7f6acaa00640 1 -- 172.21.15.119:0/4014283096 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ac400c6d0 con 0x7f6ad40b5440 2024-04-25T10:07:16.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.403+0000 7f6453400640 1 -- 172.21.15.119:0/2702489819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645406dfa0 msgr2=0x7f64541f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.403+0000 7f6453400640 1 --2- 172.21.15.119:0/2702489819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645406dfa0 0x7f64541f6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.403+0000 7f6453400640 1 -- 172.21.15.119:0/2702489819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6454071740 msgr2=0x7f64541f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.403+0000 7f6453400640 1 --2- 172.21.15.119:0/2702489819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6454071740 0x7f64541f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.403+0000 7f6453400640 1 -- 172.21.15.119:0/2702489819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64540cda90 con 0x7f645406b690 2024-04-25T10:07:16.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.404+0000 7f6453400640 1 --2- 172.21.15.119:0/2702489819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f645406b690 0x7f64540cf560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f644c00fa80 tx=0x7f644c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.404+0000 7f35a2000640 1 Processor -- start 2024-04-25T10:07:16.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.404+0000 7f644b400640 1 -- 172.21.15.119:0/2702489819 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c03b070 con 0x7f645406b690 2024-04-25T10:07:16.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.404+0000 7f644b400640 1 -- 172.21.15.119:0/2702489819 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c006040 con 0x7f645406b690 2024-04-25T10:07:16.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.404+0000 7f644b400640 1 -- 172.21.15.119:0/2702489819 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c006410 con 0x7f645406b690 2024-04-25T10:07:16.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.404+0000 7f35a2000640 1 -- start start 2024-04-25T10:07:16.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.404+0000 7f645a600640 1 -- 172.21.15.119:0/2702489819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64540cdd70 con 0x7f645406b690 2024-04-25T10:07:16.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.404+0000 7f645a600640 1 -- 172.21.15.119:0/2702489819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64540ce330 con 0x7f645406b690 2024-04-25T10:07:16.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.405+0000 7f6acaa00640 1 --2- 172.21.15.119:0/4014283096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6ab406b6d0 0x7f6ab406db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.405+0000 7f6acaa00640 1 -- 172.21.15.119:0/4014283096 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ac408a050 con 0x7f6ad40b5440 2024-04-25T10:07:16.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.406+0000 7f645a600640 1 -- 172.21.15.119:0/2702489819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64540750b0 con 0x7f645406b690 2024-04-25T10:07:16.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.406+0000 7f6ad2000640 1 --2- 172.21.15.119:0/4014283096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6ab406b6d0 0x7f6ab406db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.406+0000 7f35a2000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359c06b2b0 0x7f359c1f8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.407+0000 7f6ad2000640 1 --2- 172.21.15.119:0/4014283096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6ab406b6d0 0x7f6ab406db90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ad41689f0 tx=0x7f6ac0009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.408+0000 7f35a2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f359c0cc810 0x7f359c0aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.408+0000 7f35a2000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f359c1450e0 0x7f359c0ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.408+0000 7f35a2000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0cce00 con 0x7f359c0cc810 2024-04-25T10:07:16.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.408+0000 7f35a2000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0ccc80 con 0x7f359c1450e0 2024-04-25T10:07:16.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.408+0000 7f35a2000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0b33c0 con 0x7f359c06b2b0 2024-04-25T10:07:16.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.408+0000 7f644b400640 1 -- 172.21.15.119:0/2702489819 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f644c0065b0 con 0x7f645406b690 2024-04-25T10:07:16.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.408+0000 7f35a0c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359c06b2b0 0x7f359c1f8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.409+0000 7f359be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f359c0cc810 0x7f359c0aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.409+0000 7f359be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f359c0cc810 0x7f359c0aaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34860/0 (socket says 172.21.15.119:34860) 2024-04-25T10:07:16.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.409+0000 7f359be00640 1 -- 172.21.15.119:0/4243158432 learned_addr learned my addr 172.21.15.119:0/4243158432 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:16.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.409+0000 7f644b400640 1 --2- 172.21.15.119:0/2702489819 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f643006b5a0 0x7f643006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.409+0000 7f644b400640 1 -- 172.21.15.119:0/2702489819 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f644c0aa6c0 con 0x7f645406b690 2024-04-25T10:07:16.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.409+0000 7f6452a00640 1 --2- 172.21.15.119:0/2702489819 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f643006b5a0 0x7f643006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.410+0000 7f6452a00640 1 --2- 172.21.15.119:0/2702489819 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f643006b5a0 0x7f643006da60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6444009430 tx=0x7f644400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.411+0000 7f35a1600640 1 --2- 172.21.15.119:0/4243158432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f359c1450e0 0x7f359c0ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.412+0000 7f359be00640 1 -- 172.21.15.119:0/4243158432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359c06b2b0 msgr2=0x7f359c1f8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.412+0000 7f359be00640 1 --2- 172.21.15.119:0/4243158432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359c06b2b0 0x7f359c1f8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.412+0000 7f359be00640 1 -- 172.21.15.119:0/4243158432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f359c1450e0 msgr2=0x7f359c0ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.412+0000 7f359be00640 1 --2- 172.21.15.119:0/4243158432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f359c1450e0 0x7f359c0ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.412+0000 7f359be00640 1 -- 172.21.15.119:0/4243158432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c0a8d30 con 0x7f359c0cc810 2024-04-25T10:07:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.413+0000 7f359be00640 1 --2- 172.21.15.119:0/4243158432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f359c0cc810 0x7f359c0aaf60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f358802a670 tx=0x7f35880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.414+0000 7f9222000640 1 -- 172.21.15.119:0/3615753806 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f921c04b030 con 0x7f9224007190 2024-04-25T10:07:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.414+0000 7f3598c00640 1 -- 172.21.15.119:0/4243158432 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3588040020 con 0x7f359c0cc810 2024-04-25T10:07:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.414+0000 7f3598c00640 1 -- 172.21.15.119:0/4243158432 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3588038040 con 0x7f359c0cc810 2024-04-25T10:07:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.414+0000 7f35a2000640 1 -- 172.21.15.119:0/4243158432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359c0ac090 con 0x7f359c0cc810 2024-04-25T10:07:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.414+0000 7f35a2000640 1 -- 172.21.15.119:0/4243158432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359c0abad0 con 0x7f359c0cc810 2024-04-25T10:07:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.415+0000 7f3598c00640 1 -- 172.21.15.119:0/4243158432 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3588037070 con 0x7f359c0cc810 2024-04-25T10:07:16.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.415+0000 7f644b400640 1 -- 172.21.15.119:0/2702489819 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f644c079520 con 0x7f645406b690 2024-04-25T10:07:16.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.416+0000 7f6acaa00640 1 -- 172.21.15.119:0/4014283096 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ac4017020 con 0x7f6ad40b5440 2024-04-25T10:07:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.416+0000 7f35a2000640 1 -- 172.21.15.119:0/4243158432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359c0749b0 con 0x7f359c0cc810 2024-04-25T10:07:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.420+0000 7f3598c00640 1 -- 172.21.15.119:0/4243158432 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3588043020 con 0x7f359c0cc810 2024-04-25T10:07:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.421+0000 7f3598c00640 1 --2- 172.21.15.119:0/4243158432 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f357c06b720 0x7f357c06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.421+0000 7f3598c00640 1 -- 172.21.15.119:0/4243158432 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3588009360 con 0x7f359c0cc810 2024-04-25T10:07:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.422+0000 7f35a0c00640 1 --2- 172.21.15.119:0/4243158432 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f357c06b720 0x7f357c06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.425+0000 7f35a0c00640 1 --2- 172.21.15.119:0/4243158432 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f357c06b720 0x7f357c06dbe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3590005880 tx=0x7f3590009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.429+0000 7f3598c00640 1 -- 172.21.15.119:0/4243158432 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f358804a030 con 0x7f359c0cc810 2024-04-25T10:07:16.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.438+0000 7f05d9200640 1 Processor -- start 2024-04-25T10:07:16.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.438+0000 7f05d9200640 1 -- start start 2024-04-25T10:07:16.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.440+0000 7f05d9200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f05d4154060 0x7f05d406d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.440+0000 7f05d9200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05d40b3f70 0x7f05d4071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.441+0000 7f05d9200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05d415a730 con 0x7f05d4153c80 2024-04-25T10:07:16.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.441+0000 7f05d9200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d415aa10 con 0x7f05d4154060 2024-04-25T10:07:16.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.441+0000 7f05d9200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d406d120 con 0x7f05d40b3f70 2024-04-25T10:07:16.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.441+0000 7f05d3400640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f05d4153c80 0x7f05d4159eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:41446/0 (socket says 172.21.15.119:41446) 2024-04-25T10:07:16.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.441+0000 7f05d3400640 1 -- 172.21.15.119:0/2330024316 learned_addr learned my addr 172.21.15.119:0/2330024316 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:16.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.441+0000 7f05d2a00640 1 --2- 172.21.15.119:0/2330024316 >> v2:172.21.15.132:3300/0 conn(0x7f05d4154060 0x7f05d406d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.442+0000 7f05d3e00640 1 --2- 172.21.15.119:0/2330024316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05d40b3f70 0x7f05d4071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.442+0000 7f05d2000640 1 -- 172.21.15.119:0/2330024316 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1382372202 0 0) 0x7f05d415a730 con 0x7f05d4153c80 2024-04-25T10:07:16.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.442+0000 7f05d2000640 1 -- 172.21.15.119:0/2330024316 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05b8003440 con 0x7f05d4153c80 2024-04-25T10:07:16.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.442+0000 7f05d2a00640 1 -- 172.21.15.119:0/2330024316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05d40b3f70 msgr2=0x7f05d4071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.442+0000 7f05d2a00640 1 --2- 172.21.15.119:0/2330024316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05d40b3f70 0x7f05d4071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.443+0000 7f05d2a00640 1 -- 172.21.15.119:0/2330024316 >> v1:172.21.15.132:6789/0 conn(0x7f05d4153c80 legacy=0x7f05d4159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.443+0000 7f05d2a00640 1 -- 172.21.15.119:0/2330024316 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d406d400 con 0x7f05d4154060 2024-04-25T10:07:16.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.444+0000 7f05d2a00640 1 --2- 172.21.15.119:0/2330024316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05d4154060 0x7f05d406d920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f05c00090e0 tx=0x7f05c002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbb08eee5a30ce47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.445+0000 7f05d2000640 1 -- 172.21.15.119:0/2330024316 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c0037070 con 0x7f05d4154060 2024-04-25T10:07:16.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.445+0000 7f05d2000640 1 -- 172.21.15.119:0/2330024316 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c0032440 con 0x7f05d4154060 2024-04-25T10:07:16.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.445+0000 7f05d2000640 1 -- 172.21.15.119:0/2330024316 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c0032810 con 0x7f05d4154060 2024-04-25T10:07:16.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.446+0000 7f05d9200640 1 -- 172.21.15.119:0/2330024316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05d4154060 msgr2=0x7f05d406d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.446+0000 7f05d9200640 1 --2- 172.21.15.119:0/2330024316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05d4154060 0x7f05d406d920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f05c00090e0 tx=0x7f05c002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.446+0000 7f05d9200640 1 -- 172.21.15.119:0/2330024316 shutdown_connections 2024-04-25T10:07:16.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.446+0000 7f05d9200640 1 --2- 172.21.15.119:0/2330024316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05d40b3f70 0x7f05d4071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.446+0000 7f05d9200640 1 --2- 172.21.15.119:0/2330024316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05d4154060 0x7f05d406d920 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.446+0000 7f05d9200640 1 -- 172.21.15.119:0/2330024316 >> 172.21.15.119:0/2330024316 conn(0x7f05d40b1350 msgr2=0x7f05d40b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.446+0000 7f05d9200640 1 -- 172.21.15.119:0/2330024316 shutdown_connections 2024-04-25T10:07:16.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.446+0000 7f05d9200640 1 -- 172.21.15.119:0/2330024316 wait complete. 2024-04-25T10:07:16.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.449+0000 7f05d9200640 1 Processor -- start 2024-04-25T10:07:16.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.450+0000 7f05d9200640 1 -- start start 2024-04-25T10:07:16.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.452+0000 7f8b42c00640 1 Processor -- start 2024-04-25T10:07:16.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.452+0000 7f8b42c00640 1 -- start start 2024-04-25T10:07:16.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.453+0000 7f05d9200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05d40b3f70 0x7f05d41ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.453+0000 7f05d9200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05d4153c80 0x7f05d4156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.453+0000 7f05d9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05d4154060 0x7f05d40b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.454+0000 7f05d2a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05d4153c80 0x7f05d4156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.454+0000 7f8b42c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b3c0d05b0 0x7f8b3c15ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.454+0000 7f05d2a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05d4153c80 0x7f05d4156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:50998/0 (socket says 172.21.15.119:50998) 2024-04-25T10:07:16.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.454+0000 7f05d2a00640 1 -- 172.21.15.119:0/932808588 learned_addr learned my addr 172.21.15.119:0/932808588 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:16.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.454+0000 7f05d9200640 1 -- 172.21.15.119:0/932808588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d406dfe0 con 0x7f05d4154060 2024-04-25T10:07:16.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.454+0000 7f05d3400640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05d40b3f70 0x7f05d41ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.454+0000 7f8b42c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3c1625f0 con 0x7f8b3c0d1b90 2024-04-25T10:07:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.454+0000 7f8b42c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3c0cf200 con 0x7f8b3c0d01d0 2024-04-25T10:07:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.454+0000 7f8b42c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c162160 con 0x7f8b3c0d05b0 2024-04-25T10:07:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.454+0000 7f8b41800640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f8b3c0d01d0 0x7f8b3c0cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:41456/0 (socket says 172.21.15.119:41456) 2024-04-25T10:07:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.454+0000 7f8b41800640 1 -- 172.21.15.119:0/268990517 learned_addr learned my addr 172.21.15.119:0/268990517 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.454+0000 7f05d9200640 1 -- 172.21.15.119:0/932808588 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d406de60 con 0x7f05d4153c80 2024-04-25T10:07:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.454+0000 7f05d9200640 1 -- 172.21.15.119:0/932808588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d406aba0 con 0x7f05d40b3f70 2024-04-25T10:07:16.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.455+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903806098 0 0) 0x7f8b3c0cf200 con 0x7f8b3c0d01d0 2024-04-25T10:07:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.455+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b24003520 con 0x7f8b3c0d01d0 2024-04-25T10:07:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.455+0000 7f05d3e00640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05d4154060 0x7f05d40b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.455+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368768425 0 0) 0x7f8b3c1625f0 con 0x7f8b3c0d1b90 2024-04-25T10:07:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.455+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b3c0cf200 con 0x7f8b3c0d1b90 2024-04-25T10:07:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2127346983 0 0) 0x7f8b24003520 con 0x7f8b3c0d01d0 2024-04-25T10:07:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b3c0d05b0 msgr2=0x7f8b3c15ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f8b40e00640 1 --2- 172.21.15.119:0/268990517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b3c0d05b0 0x7f8b3c15ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f8b3be00640 1 --2- 172.21.15.119:0/268990517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b3c0d05b0 0x7f8b3c15ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 >> v1:172.21.15.119:6789/0 conn(0x7f8b3c0d1b90 legacy=0x7f8b3c0d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c161830 con 0x7f8b3c0d01d0 2024-04-25T10:07:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f8b30001670 con 0x7f8b3c0d01d0 2024-04-25T10:07:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 >> v1:172.21.15.132:6789/0 conn(0x7f8b3c0d01d0 legacy=0x7f8b3c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f8b3be00640 1 --2- 172.21.15.119:0/268990517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b24003b70 0x7f8b24003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f8b3be00640 1 --2- 172.21.15.119:0/268990517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b24004800 0x7f8b24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f8b3be00640 1 --2- 172.21.15.119:0/268990517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b3c0d05b0 0x7f8b2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c162160 con 0x7f8b24004800 2024-04-25T10:07:16.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b24005600 con 0x7f8b24003b70 2024-04-25T10:07:16.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2400bf00 con 0x7f8b3c0d05b0 2024-04-25T10:07:16.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f05d3400640 1 -- 172.21.15.119:0/932808588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05d4153c80 msgr2=0x7f05d4156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f05d3400640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05d4153c80 0x7f05d4156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f05d3400640 1 -- 172.21.15.119:0/932808588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05d4154060 msgr2=0x7f05d40b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f05d3400640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05d4154060 0x7f05d40b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f05d3400640 1 -- 172.21.15.119:0/932808588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d415b240 con 0x7f05d40b3f70 2024-04-25T10:07:16.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f05d3e00640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05d4154060 0x7f05d40b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:16.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.456+0000 7f05d3400640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05d40b3f70 0x7f05d41ffff0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f05c400c3e0 tx=0x7f05c400c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.457+0000 7f8b42200640 1 --2- 172.21.15.119:0/268990517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b3c0d05b0 0x7f8b2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.457+0000 7f8b41800640 1 --2- 172.21.15.119:0/268990517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b24003b70 0x7f8b24003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.457+0000 7f05cb400640 1 -- 172.21.15.119:0/932808588 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c40074e0 con 0x7f05d40b3f70 2024-04-25T10:07:16.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.457+0000 7f05cb400640 1 -- 172.21.15.119:0/932808588 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c40047e0 con 0x7f05d40b3f70 2024-04-25T10:07:16.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.457+0000 7f05cb400640 1 -- 172.21.15.119:0/932808588 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c4004c50 con 0x7f05d40b3f70 2024-04-25T10:07:16.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.458+0000 7f8b42200640 1 -- 172.21.15.119:0/268990517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b24003b70 msgr2=0x7f8b24003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.458+0000 7f8b42200640 1 --2- 172.21.15.119:0/268990517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b24003b70 0x7f8b24003f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.458+0000 7f8b42200640 1 -- 172.21.15.119:0/268990517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b24004800 msgr2=0x7f8b24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.458+0000 7f8b42200640 1 --2- 172.21.15.119:0/268990517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b24004800 0x7f8b24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.458+0000 7f8b42200640 1 -- 172.21.15.119:0/268990517 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c161830 con 0x7f8b3c0d05b0 2024-04-25T10:07:16.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.460+0000 7f05d9200640 1 -- 172.21.15.119:0/932808588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d41e4950 con 0x7f05d40b3f70 2024-04-25T10:07:16.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.460+0000 7f8b42200640 1 --2- 172.21.15.119:0/268990517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b3c0d05b0 0x7f8b2400b9c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8b34002000 tx=0x7f8b24005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0c6f1cb47407e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.460+0000 7f05d9200640 1 -- 172.21.15.119:0/932808588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d41e4f10 con 0x7f05d40b3f70 2024-04-25T10:07:16.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.461+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34001bf0 con 0x7f8b3c0d05b0 2024-04-25T10:07:16.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.461+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b34003b70 con 0x7f8b3c0d05b0 2024-04-25T10:07:16.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.461+0000 7f8b3be00640 1 -- 172.21.15.119:0/268990517 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b340050d0 con 0x7f8b3c0d05b0 2024-04-25T10:07:16.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.461+0000 7f05d9200640 1 -- 172.21.15.119:0/932808588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d415b410 con 0x7f05d40b3f70 2024-04-25T10:07:16.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.462+0000 7f8b42c00640 1 -- 172.21.15.119:0/268990517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b3c0d05b0 msgr2=0x7f8b2400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.462+0000 7f8b42c00640 1 --2- 172.21.15.119:0/268990517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b3c0d05b0 0x7f8b2400b9c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8b34002000 tx=0x7f8b24005830 comp rx=0 tx=0).stop 2024-04-25T10:07:16.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.462+0000 7f8b42200640 1 -- 172.21.15.119:0/268990517 reap_dead start 2024-04-25T10:07:16.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.462+0000 7f8b42c00640 1 -- 172.21.15.119:0/268990517 shutdown_connections 2024-04-25T10:07:16.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.462+0000 7f8b42c00640 1 -- 172.21.15.119:0/268990517 >> 172.21.15.119:0/268990517 conn(0x7f8b3c06bed0 msgr2=0x7f8b3c06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.463+0000 7f8b42c00640 1 -- 172.21.15.119:0/268990517 shutdown_connections 2024-04-25T10:07:16.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.463+0000 7f8b42c00640 1 -- 172.21.15.119:0/268990517 wait complete. 2024-04-25T10:07:16.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.465+0000 7f8b42c00640 1 Processor -- start 2024-04-25T10:07:16.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.466+0000 7f8b42c00640 1 -- start start 2024-04-25T10:07:16.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.466+0000 7f05cb400640 1 -- 172.21.15.119:0/932808588 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05c4011070 con 0x7f05d40b3f70 2024-04-25T10:07:16.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.467+0000 7f05cb400640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f05b406b4d0 0x7f05b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.467+0000 7f8b42c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b3c0d05b0 0x7f8b3c158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.467+0000 7f8b42c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b3c153270 0x7f8b3c0c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.467+0000 7f8b42c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b3c0c86e0 0x7f8b3c15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.467+0000 7f8b42c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0d0a10 con 0x7f8b3c0c86e0 2024-04-25T10:07:16.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.467+0000 7f8b42c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c161df0 con 0x7f8b3c153270 2024-04-25T10:07:16.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.467+0000 7f8b42c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0d0b90 con 0x7f8b3c0d05b0 2024-04-25T10:07:16.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.467+0000 7f05cb400640 1 -- 172.21.15.119:0/932808588 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05c408a380 con 0x7f05d40b3f70 2024-04-25T10:07:16.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.468+0000 7f05d2a00640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f05b406b4d0 0x7f05b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.468+0000 7f8b42200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b3c0c86e0 0x7f8b3c15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.468+0000 7f8b41800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b3c0d05b0 0x7f8b3c158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.468+0000 7f8b42200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b3c0c86e0 0x7f8b3c15cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34888/0 (socket says 172.21.15.119:34888) 2024-04-25T10:07:16.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.468+0000 7f8b42200640 1 -- 172.21.15.119:0/1571016642 learned_addr learned my addr 172.21.15.119:0/1571016642 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:16.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.468+0000 7f8b40e00640 1 --2- 172.21.15.119:0/1571016642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b3c153270 0x7f8b3c0c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.469+0000 7f05d2a00640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f05b406b4d0 0x7f05b406d990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f05d415cda0 tx=0x7f05c002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.469+0000 7f8b42200640 1 -- 172.21.15.119:0/1571016642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b3c0d05b0 msgr2=0x7f8b3c158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.469+0000 7f8b42200640 1 --2- 172.21.15.119:0/1571016642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b3c0d05b0 0x7f8b3c158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.469+0000 7f8b42200640 1 -- 172.21.15.119:0/1571016642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b3c153270 msgr2=0x7f8b3c0c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.469+0000 7f8b42200640 1 --2- 172.21.15.119:0/1571016642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b3c153270 0x7f8b3c0c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.469+0000 7f8b42200640 1 -- 172.21.15.119:0/1571016642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c0c8b60 con 0x7f8b3c0c86e0 2024-04-25T10:07:16.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.470+0000 7f8b42200640 1 --2- 172.21.15.119:0/1571016642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b3c0c86e0 0x7f8b3c15cd10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8b34033040 tx=0x7f8b34001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.470+0000 7f8b39600640 1 -- 172.21.15.119:0/1571016642 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34001bf0 con 0x7f8b3c0c86e0 2024-04-25T10:07:16.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.470+0000 7f8b39600640 1 -- 172.21.15.119:0/1571016642 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b34043070 con 0x7f8b3c0c86e0 2024-04-25T10:07:16.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.470+0000 7f8b42c00640 1 -- 172.21.15.119:0/1571016642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3c0cc010 con 0x7f8b3c0c86e0 2024-04-25T10:07:16.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.471+0000 7f8b42c00640 1 -- 172.21.15.119:0/1571016642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c0cc5d0 con 0x7f8b3c0c86e0 2024-04-25T10:07:16.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.471+0000 7f8b39600640 1 -- 172.21.15.119:0/1571016642 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34005520 con 0x7f8b3c0c86e0 2024-04-25T10:07:16.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.474+0000 7f8b39600640 1 -- 172.21.15.119:0/1571016642 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b340056c0 con 0x7f8b3c0c86e0 2024-04-25T10:07:16.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.475+0000 7f8b42c00640 1 -- 172.21.15.119:0/1571016642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b18005290 con 0x7f8b3c0c86e0 2024-04-25T10:07:16.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.476+0000 7f05cb400640 1 -- 172.21.15.119:0/932808588 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05c4059270 con 0x7f05d40b3f70 2024-04-25T10:07:16.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.475+0000 7f8b39600640 1 --2- 172.21.15.119:0/1571016642 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b1c06b520 0x7f8b1c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.476+0000 7f8b39600640 1 -- 172.21.15.119:0/1571016642 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b340aaae0 con 0x7f8b3c0c86e0 2024-04-25T10:07:16.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.477+0000 7f8b41800640 1 --2- 172.21.15.119:0/1571016642 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b1c06b520 0x7f8b1c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.478+0000 7f8b41800640 1 --2- 172.21.15.119:0/1571016642 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b1c06b520 0x7f8b1c06d9e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8b30001eb0 tx=0x7f8b30009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.487+0000 7f8b39600640 1 -- 172.21.15.119:0/1571016642 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b340799b0 con 0x7f8b3c0c86e0 2024-04-25T10:07:16.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.809+0000 7f8234c00640 1 -- 172.21.15.119:0/628051869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8214005580 con 0x7f82400cef80 2024-04-25T10:07:16.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.810+0000 7f8237400640 1 -- 172.21.15.119:0/628051869 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f822c017020 con 0x7f82400cef80 2024-04-25T10:07:16.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.810+0000 7f6ad3e00640 1 -- 172.21.15.119:0/4014283096 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6aac005850 con 0x7f6ad40b5440 2024-04-25T10:07:16.816 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2024-04-25T10:07:16.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.813+0000 7f8234c00640 1 -- 172.21.15.119:0/628051869 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f822006b4d0 msgr2=0x7f822006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.814+0000 7f8234c00640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f822006b4d0 0x7f822006d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82401ed060 tx=0x7f8230008040 comp rx=0 tx=0).stop 2024-04-25T10:07:16.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.814+0000 7f8234c00640 1 -- 172.21.15.119:0/628051869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82400cef80 msgr2=0x7f8240152be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.814+0000 7f8234c00640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82400cef80 0x7f8240152be0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f822c006fd0 tx=0x7f822c00ec70 comp rx=0 tx=0).stop 2024-04-25T10:07:16.820 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2024-04-25T10:07:16.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.811+0000 7f6acaa00640 1 -- 172.21.15.119:0/4014283096 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6ac4058f40 con 0x7f6ad40b5440 2024-04-25T10:07:16.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.814+0000 7f8234c00640 1 -- 172.21.15.119:0/628051869 shutdown_connections 2024-04-25T10:07:16.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.815+0000 7f8234c00640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f822006b4d0 0x7f822006d990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.815+0000 7f8234c00640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8240159910 0x7f82400b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.815+0000 7f8234c00640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82400cef80 0x7f8240152be0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.815+0000 7f8234c00640 1 --2- 172.21.15.119:0/628051869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82400b1740 0x7f82401f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.815+0000 7f8234c00640 1 -- 172.21.15.119:0/628051869 >> 172.21.15.119:0/628051869 conn(0x7f824006aba0 msgr2=0x7f82400ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.815+0000 7f8234c00640 1 -- 172.21.15.119:0/628051869 shutdown_connections 2024-04-25T10:07:16.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.815+0000 7f8234c00640 1 -- 172.21.15.119:0/628051869 wait complete. 2024-04-25T10:07:16.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.815+0000 7f6ad3e00640 1 -- 172.21.15.119:0/4014283096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6ab406b6d0 msgr2=0x7f6ab406db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.815+0000 7f6ad3e00640 1 --2- 172.21.15.119:0/4014283096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6ab406b6d0 0x7f6ab406db90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ad41689f0 tx=0x7f6ac0009240 comp rx=0 tx=0).stop 2024-04-25T10:07:16.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.815+0000 7f6ad3e00640 1 -- 172.21.15.119:0/4014283096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad40b5440 msgr2=0x7f6ad40c6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.816+0000 7f6ad3e00640 1 --2- 172.21.15.119:0/4014283096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad40b5440 0x7f6ad40c6940 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400e840 tx=0x7f6ac400ed00 comp rx=0 tx=0).stop 2024-04-25T10:07:16.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.816+0000 7f6ad3e00640 1 -- 172.21.15.119:0/4014283096 shutdown_connections 2024-04-25T10:07:16.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.816+0000 7f6ad3e00640 1 --2- 172.21.15.119:0/4014283096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6ab406b6d0 0x7f6ab406db90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.816+0000 7f6ad3e00640 1 --2- 172.21.15.119:0/4014283096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ad40d05b0 0x7f6ad40cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.816+0000 7f6ad3e00640 1 --2- 172.21.15.119:0/4014283096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ad40d01d0 0x7f6ad40caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.817+0000 7f6ad3e00640 1 --2- 172.21.15.119:0/4014283096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6ad40b5440 0x7f6ad40c6940 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.817+0000 7f6ad3e00640 1 -- 172.21.15.119:0/4014283096 >> 172.21.15.119:0/4014283096 conn(0x7f6ad406bed0 msgr2=0x7f6ad415f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.817+0000 7f6ad3e00640 1 -- 172.21.15.119:0/4014283096 shutdown_connections 2024-04-25T10:07:16.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.817+0000 7f6ad3e00640 1 -- 172.21.15.119:0/4014283096 wait complete. 2024-04-25T10:07:16.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.830+0000 7f3acf200640 1 -- 172.21.15.119:0/1580253965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3ad006ecc0 con 0x7f3ad015d920 2024-04-25T10:07:16.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.831+0000 7f3ac6000640 1 -- 172.21.15.119:0/1580253965 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3ac80587b0 con 0x7f3ad015d920 2024-04-25T10:07:16.836 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2024-04-25T10:07:16.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.834+0000 7f3acf200640 1 -- 172.21.15.119:0/1580253965 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3aac06b6d0 msgr2=0x7f3aac06db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.834+0000 7f3acf200640 1 --2- 172.21.15.119:0/1580253965 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3aac06b6d0 0x7f3aac06db90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80090b0 tx=0x7f3ab802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:16.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.834+0000 7f3acf200640 1 -- 172.21.15.119:0/1580253965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad015d920 msgr2=0x7f3ad01f71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.834+0000 7f3acf200640 1 --2- 172.21.15.119:0/1580253965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad015d920 0x7f3ad01f71d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800c590 tx=0x7f3ac800ca50 comp rx=0 tx=0).stop 2024-04-25T10:07:16.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.834+0000 7f3acf200640 1 -- 172.21.15.119:0/1580253965 shutdown_connections 2024-04-25T10:07:16.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.834+0000 7f3acf200640 1 --2- 172.21.15.119:0/1580253965 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3aac06b6d0 0x7f3aac06db90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.834+0000 7f3acf200640 1 --2- 172.21.15.119:0/1580253965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ad015d920 0x7f3ad01f71d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.834+0000 7f3acf200640 1 --2- 172.21.15.119:0/1580253965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad00adeb0 0x7f3ad014ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.834+0000 7f3acf200640 1 --2- 172.21.15.119:0/1580253965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ad006d780 0x7f3ad014a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.834+0000 7f3acf200640 1 -- 172.21.15.119:0/1580253965 >> 172.21.15.119:0/1580253965 conn(0x7f3ad00b1350 msgr2=0x7f3ad00aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.835+0000 7f3acf200640 1 -- 172.21.15.119:0/1580253965 shutdown_connections 2024-04-25T10:07:16.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.835+0000 7f3acf200640 1 -- 172.21.15.119:0/1580253965 wait complete. 2024-04-25T10:07:16.847 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-04-25T10:07:16.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.842+0000 7f922f200640 1 -- 172.21.15.119:0/3615753806 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9224056ca0 con 0x7f9224007190 2024-04-25T10:07:16.848 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:16.848 INFO:teuthology.orchestra.run.smithi119.stdout:55834574855 2024-04-25T10:07:16.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.842+0000 7f9222000640 1 -- 172.21.15.119:0/3615753806 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f921c079610 con 0x7f9224007190 2024-04-25T10:07:16.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.843+0000 7f645a600640 1 -- 172.21.15.119:0/2702489819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f64540b0600 con 0x7f645406b690 2024-04-25T10:07:16.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.844+0000 7f644b400640 1 -- 172.21.15.119:0/2702489819 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f644c078e20 con 0x7f645406b690 2024-04-25T10:07:16.849 INFO:teuthology.orchestra.run.smithi119.stdout:55834574855 2024-04-25T10:07:16.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-04-25T10:07:16.851 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:16.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.846+0000 7f922f200640 1 -- 172.21.15.119:0/3615753806 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f920c06b500 msgr2=0x7f920c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.846+0000 7f922f200640 1 --2- 172.21.15.119:0/3615753806 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f920c06b500 0x7f920c06d9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f92240e4fd0 tx=0x7f9218009040 comp rx=0 tx=0).stop 2024-04-25T10:07:16.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.846+0000 7f922f200640 1 -- 172.21.15.119:0/3615753806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9224007190 msgr2=0x7f92240e39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.846+0000 7f922f200640 1 --2- 172.21.15.119:0/3615753806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9224007190 0x7f92240e39c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f921c031000 tx=0x7f921c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.847+0000 7f922f200640 1 -- 172.21.15.119:0/3615753806 shutdown_connections 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.847+0000 7f922f200640 1 --2- 172.21.15.119:0/3615753806 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f920c06b500 0x7f920c06d9c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.847+0000 7f922f200640 1 --2- 172.21.15.119:0/3615753806 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240e7530 0x7f92240e5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.847+0000 7f922f200640 1 --2- 172.21.15.119:0/3615753806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92240e31c0 0x7f922404f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.847+0000 7f922f200640 1 --2- 172.21.15.119:0/3615753806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9224007190 0x7f92240e39c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.847+0000 7f922f200640 1 -- 172.21.15.119:0/3615753806 >> 172.21.15.119:0/3615753806 conn(0x7f922403aa30 msgr2=0x7f922403d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.847+0000 7f922f200640 1 -- 172.21.15.119:0/3615753806 shutdown_connections 2024-04-25T10:07:16.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.847+0000 7f922f200640 1 -- 172.21.15.119:0/3615753806 wait complete. 2024-04-25T10:07:16.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.847+0000 7f645a600640 1 -- 172.21.15.119:0/2702489819 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f643006b5a0 msgr2=0x7f643006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.847+0000 7f645a600640 1 --2- 172.21.15.119:0/2702489819 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f643006b5a0 0x7f643006da60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6444009430 tx=0x7f644400b040 comp rx=0 tx=0).stop 2024-04-25T10:07:16.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.847+0000 7f645a600640 1 -- 172.21.15.119:0/2702489819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f645406b690 msgr2=0x7f64540cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.847+0000 7f645a600640 1 --2- 172.21.15.119:0/2702489819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f645406b690 0x7f64540cf560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f644c00fa80 tx=0x7f644c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.849+0000 7f645a600640 1 -- 172.21.15.119:0/2702489819 shutdown_connections 2024-04-25T10:07:16.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.849+0000 7f645a600640 1 --2- 172.21.15.119:0/2702489819 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f643006b5a0 0x7f643006da60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.849+0000 7f645a600640 1 --2- 172.21.15.119:0/2702489819 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6454071740 0x7f64541f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.849+0000 7f645a600640 1 --2- 172.21.15.119:0/2702489819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f645406dfa0 0x7f64541f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.849+0000 7f645a600640 1 --2- 172.21.15.119:0/2702489819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f645406b690 0x7f64540cf560 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.849+0000 7f645a600640 1 -- 172.21.15.119:0/2702489819 >> 172.21.15.119:0/2702489819 conn(0x7f645406aba0 msgr2=0x7f645415ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.849+0000 7f645a600640 1 -- 172.21.15.119:0/2702489819 shutdown_connections 2024-04-25T10:07:16.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.849+0000 7f645a600640 1 -- 172.21.15.119:0/2702489819 wait complete. 2024-04-25T10:07:16.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-04-25T10:07:16.865 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:16.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.866+0000 7f35a2000640 1 -- 172.21.15.119:0/4243158432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f359c0aece0 con 0x7f359c0cc810 2024-04-25T10:07:16.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.867+0000 7f3598c00640 1 -- 172.21.15.119:0/4243158432 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3588078740 con 0x7f359c0cc810 2024-04-25T10:07:16.872 INFO:teuthology.orchestra.run.smithi119.stdout:55834574855 2024-04-25T10:07:16.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.870+0000 7f35a2000640 1 -- 172.21.15.119:0/4243158432 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f357c06b720 msgr2=0x7f357c06dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.870+0000 7f35a2000640 1 --2- 172.21.15.119:0/4243158432 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f357c06b720 0x7f357c06dbe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3590005880 tx=0x7f3590009040 comp rx=0 tx=0).stop 2024-04-25T10:07:16.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.870+0000 7f35a2000640 1 -- 172.21.15.119:0/4243158432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f359c0cc810 msgr2=0x7f359c0aaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.870+0000 7f35a2000640 1 --2- 172.21.15.119:0/4243158432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f359c0cc810 0x7f359c0aaf60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f358802a670 tx=0x7f35880046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.870+0000 7f35a2000640 1 -- 172.21.15.119:0/4243158432 shutdown_connections 2024-04-25T10:07:16.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.870+0000 7f35a2000640 1 --2- 172.21.15.119:0/4243158432 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f357c06b720 0x7f357c06dbe0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.870+0000 7f35a2000640 1 --2- 172.21.15.119:0/4243158432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f359c1450e0 0x7f359c0ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.870+0000 7f35a2000640 1 --2- 172.21.15.119:0/4243158432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f359c0cc810 0x7f359c0aaf60 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f358802a670 tx=0x7f35880046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.870+0000 7f35a2000640 1 --2- 172.21.15.119:0/4243158432 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f359c06b2b0 0x7f359c1f8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.870+0000 7f35a2000640 1 -- 172.21.15.119:0/4243158432 >> 172.21.15.119:0/4243158432 conn(0x7f359c066ff0 msgr2=0x7f359c0b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.870+0000 7f35a2000640 1 -- 172.21.15.119:0/4243158432 shutdown_connections 2024-04-25T10:07:16.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.870+0000 7f35a2000640 1 -- 172.21.15.119:0/4243158432 wait complete. 2024-04-25T10:07:16.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-25T10:07:16.877 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:16.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-04-25T10:07:16.881 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:16.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.877+0000 7f8b42c00640 1 -- 172.21.15.119:0/1571016642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f8b18005850 con 0x7f8b3c0c86e0 2024-04-25T10:07:16.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.877+0000 7f05d9200640 1 -- 172.21.15.119:0/932808588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f05d40749b0 con 0x7f05d40b3f70 2024-04-25T10:07:16.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.877+0000 7f8b39600640 1 -- 172.21.15.119:0/1571016642 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8b340792b0 con 0x7f8b3c0c86e0 2024-04-25T10:07:16.883 INFO:teuthology.orchestra.run.smithi119.stdout:55834574855 2024-04-25T10:07:16.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.878+0000 7f05cb400640 1 -- 172.21.15.119:0/932808588 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f05c4058b70 con 0x7f05d40b3f70 2024-04-25T10:07:16.884 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2024-04-25T10:07:16.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.880+0000 7f8b42c00640 1 -- 172.21.15.119:0/1571016642 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b1c06b520 msgr2=0x7f8b1c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.880+0000 7f8b42c00640 1 --2- 172.21.15.119:0/1571016642 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b1c06b520 0x7f8b1c06d9e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8b30001eb0 tx=0x7f8b30009040 comp rx=0 tx=0).stop 2024-04-25T10:07:16.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.880+0000 7f8b42c00640 1 -- 172.21.15.119:0/1571016642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b3c0c86e0 msgr2=0x7f8b3c15cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.880+0000 7f8b42c00640 1 --2- 172.21.15.119:0/1571016642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b3c0c86e0 0x7f8b3c15cd10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8b34033040 tx=0x7f8b34001650 comp rx=0 tx=0).stop 2024-04-25T10:07:16.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.880+0000 7f8b42c00640 1 -- 172.21.15.119:0/1571016642 shutdown_connections 2024-04-25T10:07:16.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.880+0000 7f8b42c00640 1 --2- 172.21.15.119:0/1571016642 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b1c06b520 0x7f8b1c06d9e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.880+0000 7f8b42c00640 1 --2- 172.21.15.119:0/1571016642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b3c0c86e0 0x7f8b3c15cd10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.880+0000 7f8b42c00640 1 --2- 172.21.15.119:0/1571016642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b3c153270 0x7f8b3c0c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.880+0000 7f8b42c00640 1 --2- 172.21.15.119:0/1571016642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b3c0d05b0 0x7f8b3c158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.880+0000 7f8b42c00640 1 -- 172.21.15.119:0/1571016642 >> 172.21.15.119:0/1571016642 conn(0x7f8b3c06bed0 msgr2=0x7f8b3c06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.881+0000 7f8b42c00640 1 -- 172.21.15.119:0/1571016642 shutdown_connections 2024-04-25T10:07:16.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.881+0000 7f8b42c00640 1 -- 172.21.15.119:0/1571016642 wait complete. 2024-04-25T10:07:16.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.881+0000 7f05d9200640 1 -- 172.21.15.119:0/932808588 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f05b406b4d0 msgr2=0x7f05b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.881+0000 7f05d9200640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f05b406b4d0 0x7f05b406d990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f05d415cda0 tx=0x7f05c002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:16.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.881+0000 7f05d9200640 1 -- 172.21.15.119:0/932808588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05d40b3f70 msgr2=0x7f05d41ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.881+0000 7f05d9200640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05d40b3f70 0x7f05d41ffff0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f05c400c3e0 tx=0x7f05c400c8a0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.881+0000 7f05d9200640 1 -- 172.21.15.119:0/932808588 shutdown_connections 2024-04-25T10:07:16.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.881+0000 7f05d9200640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f05b406b4d0 0x7f05b406d990 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.881+0000 7f05d9200640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05d4154060 0x7f05d40b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.881+0000 7f05d9200640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05d4153c80 0x7f05d4156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.881+0000 7f05d9200640 1 --2- 172.21.15.119:0/932808588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05d40b3f70 0x7f05d41ffff0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.881+0000 7f05d9200640 1 -- 172.21.15.119:0/932808588 >> 172.21.15.119:0/932808588 conn(0x7f05d40b1350 msgr2=0x7f05d40b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.881+0000 7f05d9200640 1 -- 172.21.15.119:0/932808588 shutdown_connections 2024-04-25T10:07:16.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:16.881+0000 7f05d9200640 1 -- 172.21.15.119:0/932808588 wait complete. 2024-04-25T10:07:16.894 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-25T10:07:16.894 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:16.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-04-25T10:07:16.904 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:16.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-04-25T10:07:16.905 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:16.905 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:07:16.905 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:17.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.128+0000 7f8bf1600640 1 Processor -- start 2024-04-25T10:07:17.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.128+0000 7f8bf1600640 1 -- start start 2024-04-25T10:07:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.130+0000 7f8bf1600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bec15c8a0 0x7f8bec0acd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.130+0000 7f8bf1600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8bec0ad2c0 0x7f8bec0b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.130+0000 7f8bf1600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bec06af10 con 0x7f8bec141b80 2024-04-25T10:07:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.130+0000 7f8bf1600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bec06bb20 con 0x7f8bec0ad2c0 2024-04-25T10:07:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.130+0000 7f8bf1600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec06b1f0 con 0x7f8bec15c8a0 2024-04-25T10:07:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.130+0000 7f8bf0c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8bec0ad2c0 0x7f8bec0b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.130+0000 7f8bf0c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8bec0ad2c0 0x7f8bec0b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34894/0 (socket says 172.21.15.119:34894) 2024-04-25T10:07:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.131+0000 7f8bf0c00640 1 -- 172.21.15.119:0/3857071107 learned_addr learned my addr 172.21.15.119:0/3857071107 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.130+0000 7f8bebe00640 1 --1- 172.21.15.119:0/3857071107 >> v1:172.21.15.119:6789/0 conn(0x7f8bec141b80 0x7f8bec06bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42190/0 (socket says 172.21.15.119:42190) 2024-04-25T10:07:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.131+0000 7f8beb400640 1 --2- 172.21.15.119:0/3857071107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bec15c8a0 0x7f8bec0acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.131+0000 7f8bf0c00640 1 -- 172.21.15.119:0/3857071107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bec15c8a0 msgr2=0x7f8bec0acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.131+0000 7f8bf0c00640 1 --2- 172.21.15.119:0/3857071107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bec15c8a0 0x7f8bec0acd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.131+0000 7f8bf0c00640 1 -- 172.21.15.119:0/3857071107 >> v1:172.21.15.119:6789/0 conn(0x7f8bec141b80 legacy=0x7f8bec06bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.131+0000 7f8bf0c00640 1 -- 172.21.15.119:0/3857071107 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec06b4d0 con 0x7f8bec0ad2c0 2024-04-25T10:07:17.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.131+0000 7f8beaa00640 1 -- 172.21.15.119:0/3857071107 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563848711 0 0) 0x7f8bec06af10 con 0x7f8bec141b80 2024-04-25T10:07:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.132+0000 7f8bf0c00640 1 --2- 172.21.15.119:0/3857071107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bec0ad2c0 0x7f8bec0b2d30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009870 tx=0x7f8be402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ad201787790c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.132+0000 7f8beaa00640 1 -- 172.21.15.119:0/3857071107 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4037070 con 0x7f8bec0ad2c0 2024-04-25T10:07:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.132+0000 7f8beaa00640 1 -- 172.21.15.119:0/3857071107 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be4032440 con 0x7f8bec0ad2c0 2024-04-25T10:07:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.132+0000 7f8beaa00640 1 -- 172.21.15.119:0/3857071107 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4032810 con 0x7f8bec0ad2c0 2024-04-25T10:07:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.133+0000 7f8bf1600640 1 -- 172.21.15.119:0/3857071107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bec0ad2c0 msgr2=0x7f8bec0b2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.133+0000 7f8bf1600640 1 --2- 172.21.15.119:0/3857071107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bec0ad2c0 0x7f8bec0b2d30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009870 tx=0x7f8be402a250 comp rx=0 tx=0).stop 2024-04-25T10:07:17.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.133+0000 7f8bf1600640 1 -- 172.21.15.119:0/3857071107 shutdown_connections 2024-04-25T10:07:17.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.133+0000 7f8bf1600640 1 --2- 172.21.15.119:0/3857071107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bec0ad2c0 0x7f8bec0b2d30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.133+0000 7f8bf1600640 1 --2- 172.21.15.119:0/3857071107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bec15c8a0 0x7f8bec0acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.133+0000 7f8bf1600640 1 -- 172.21.15.119:0/3857071107 >> 172.21.15.119:0/3857071107 conn(0x7f8bec0cfdc0 msgr2=0x7f8bec0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.133+0000 7f8bf1600640 1 -- 172.21.15.119:0/3857071107 shutdown_connections 2024-04-25T10:07:17.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.133+0000 7f8bf1600640 1 -- 172.21.15.119:0/3857071107 wait complete. 2024-04-25T10:07:17.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.135+0000 7f8bf1600640 1 Processor -- start 2024-04-25T10:07:17.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.135+0000 7f8bf1600640 1 -- start start 2024-04-25T10:07:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8bf1600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bec0ad2c0 0x7f8bec15d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8bf1600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bec141b80 0x7f8bec15fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8bf1600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bec15c8a0 0x7f8bec1405b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8bf1600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec06aba0 con 0x7f8bec141b80 2024-04-25T10:07:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8bf1600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec141f60 con 0x7f8bec0ad2c0 2024-04-25T10:07:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8bf1600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec0af3d0 con 0x7f8bec15c8a0 2024-04-25T10:07:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8beb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bec141b80 0x7f8bec15fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8beb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bec141b80 0x7f8bec15fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34904/0 (socket says 172.21.15.119:34904) 2024-04-25T10:07:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8beb400640 1 -- 172.21.15.119:0/953919376 learned_addr learned my addr 172.21.15.119:0/953919376 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8bebe00640 1 --2- 172.21.15.119:0/953919376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bec0ad2c0 0x7f8bec15d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8beb400640 1 -- 172.21.15.119:0/953919376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bec15c8a0 msgr2=0x7f8bec1405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8beb400640 1 --2- 172.21.15.119:0/953919376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bec15c8a0 0x7f8bec1405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8beb400640 1 -- 172.21.15.119:0/953919376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bec0ad2c0 msgr2=0x7f8bec15d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8beb400640 1 --2- 172.21.15.119:0/953919376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bec0ad2c0 0x7f8bec15d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.136+0000 7f8beb400640 1 -- 172.21.15.119:0/953919376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec15cf10 con 0x7f8bec141b80 2024-04-25T10:07:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.137+0000 7f8beb400640 1 --2- 172.21.15.119:0/953919376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bec141b80 0x7f8bec15fcb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8007e40 tx=0x7f8bd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.137+0000 7f8be3e00640 1 -- 172.21.15.119:0/953919376 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8014620 con 0x7f8bec141b80 2024-04-25T10:07:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.137+0000 7f8be3e00640 1 -- 172.21.15.119:0/953919376 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd8019070 con 0x7f8bec141b80 2024-04-25T10:07:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.137+0000 7f8be3e00640 1 -- 172.21.15.119:0/953919376 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8014a00 con 0x7f8bec141b80 2024-04-25T10:07:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.137+0000 7f8bf1600640 1 -- 172.21.15.119:0/953919376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bec15d1f0 con 0x7f8bec141b80 2024-04-25T10:07:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.137+0000 7f8bf1600640 1 -- 172.21.15.119:0/953919376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bec140af0 con 0x7f8bec141b80 2024-04-25T10:07:17.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.138+0000 7f8bf1600640 1 -- 172.21.15.119:0/953919376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bec0749b0 con 0x7f8bec141b80 2024-04-25T10:07:17.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.139+0000 7f8be3e00640 1 -- 172.21.15.119:0/953919376 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8bd8014ba0 con 0x7f8bec141b80 2024-04-25T10:07:17.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.140+0000 7f8be3e00640 1 --2- 172.21.15.119:0/953919376 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8bcc06b4d0 0x7f8bcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.140+0000 7f8be3e00640 1 -- 172.21.15.119:0/953919376 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bd808a9d0 con 0x7f8bec141b80 2024-04-25T10:07:17.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.140+0000 7f8bebe00640 1 --2- 172.21.15.119:0/953919376 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8bcc06b4d0 0x7f8bcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.141+0000 7f8bebe00640 1 --2- 172.21.15.119:0/953919376 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8bcc06b4d0 0x7f8bcc06d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc007fd0 tx=0x7f8bdc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.144+0000 7f8be3e00640 1 -- 172.21.15.119:0/953919376 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8bd8059840 con 0x7f8bec141b80 2024-04-25T10:07:17.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.322+0000 7f8bf1600640 1 -- 172.21.15.119:0/953919376 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8bec15ba80 con 0x7f8bcc06b4d0 2024-04-25T10:07:17.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.328+0000 7f8be3e00640 1 -- 172.21.15.119:0/953919376 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f8bec15ba80 con 0x7f8bcc06b4d0 2024-04-25T10:07:17.338 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:17.340 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:07:17.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.334+0000 7f8bf1600640 1 -- 172.21.15.119:0/953919376 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8bcc06b4d0 msgr2=0x7f8bcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.334+0000 7f8bf1600640 1 --2- 172.21.15.119:0/953919376 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8bcc06b4d0 0x7f8bcc06d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc007fd0 tx=0x7f8bdc009040 comp rx=0 tx=0).stop 2024-04-25T10:07:17.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.334+0000 7f8bf1600640 1 -- 172.21.15.119:0/953919376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bec141b80 msgr2=0x7f8bec15fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.334+0000 7f8bf1600640 1 --2- 172.21.15.119:0/953919376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bec141b80 0x7f8bec15fcb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8007e40 tx=0x7f8bd800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.334+0000 7f8bf1600640 1 -- 172.21.15.119:0/953919376 shutdown_connections 2024-04-25T10:07:17.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.334+0000 7f8bf1600640 1 --2- 172.21.15.119:0/953919376 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8bcc06b4d0 0x7f8bcc06d990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.334+0000 7f8bf1600640 1 --2- 172.21.15.119:0/953919376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bec15c8a0 0x7f8bec1405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.334+0000 7f8bf1600640 1 --2- 172.21.15.119:0/953919376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bec141b80 0x7f8bec15fcb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.334+0000 7f8bf1600640 1 --2- 172.21.15.119:0/953919376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bec0ad2c0 0x7f8bec15d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.334+0000 7f8bf1600640 1 -- 172.21.15.119:0/953919376 >> 172.21.15.119:0/953919376 conn(0x7f8bec0cfdc0 msgr2=0x7f8bec0b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.334+0000 7f8bf1600640 1 -- 172.21.15.119:0/953919376 shutdown_connections 2024-04-25T10:07:17.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.335+0000 7f8bf1600640 1 -- 172.21.15.119:0/953919376 wait complete. 2024-04-25T10:07:17.358 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-25T10:07:15.925479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218988,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755732,"statfs":{"total":773094113280,"available":772869869568,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.621466"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117497+0000","last_change":"2024-04-25T10:07:07.357761+0000","last_active":"2024-04-25T10:07:10.117497+0000","last_peered":"2024-04-25T10:07:10.117497+0000","last_clean":"2024-04-25T10:07:10.117497+0000","last_became_active":"2024-04-25T10:07:07.357419+0000","last_became_peered":"2024-04-25T10:07:07.357419+0000","last_unstale":"2024-04-25T10:07:10.117497+0000","last_undegraded":"2024-04-25T10:07:10.117497+0000","last_fullsized":"2024-04-25T10:07:10.117497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:31.725733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340438+0000","last_change":"2024-04-25T10:07:07.324906+0000","last_active":"2024-04-25T10:07:08.340438+0000","last_peered":"2024-04-25T10:07:08.340438+0000","last_clean":"2024-04-25T10:07:08.340438+0000","last_became_active":"2024-04-25T10:07:07.324635+0000","last_became_peered":"2024-04-25T10:07:07.324635+0000","last_unstale":"2024-04-25T10:07:08.340438+0000","last_undegraded":"2024-04-25T10:07:08.340438+0000","last_fullsized":"2024-04-25T10:07:08.340438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013703+0000","last_change":"2024-04-25T10:07:07.324537+0000","last_active":"2024-04-25T10:07:09.013703+0000","last_peered":"2024-04-25T10:07:09.013703+0000","last_clean":"2024-04-25T10:07:09.013703+0000","last_became_active":"2024-04-25T10:07:07.324290+0000","last_became_peered":"2024-04-25T10:07:07.324290+0000","last_unstale":"2024-04-25T10:07:09.013703+0000","last_undegraded":"2024-04-25T10:07:09.013703+0000","last_fullsized":"2024-04-25T10:07:09.013703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340924+0000","last_change":"2024-04-25T10:07:07.336252+0000","last_active":"2024-04-25T10:07:08.340924+0000","last_peered":"2024-04-25T10:07:08.340924+0000","last_clean":"2024-04-25T10:07:08.340924+0000","last_became_active":"2024-04-25T10:07:07.335889+0000","last_became_peered":"2024-04-25T10:07:07.335889+0000","last_unstale":"2024-04-25T10:07:08.340924+0000","last_undegraded":"2024-04-25T10:07:08.340924+0000","last_fullsized":"2024-04-25T10:07:08.340924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116666+0000","last_change":"2024-04-25T10:07:07.354080+0000","last_active":"2024-04-25T10:07:10.116666+0000","last_peered":"2024-04-25T10:07:10.116666+0000","last_clean":"2024-04-25T10:07:10.116666+0000","last_became_active":"2024-04-25T10:07:07.351587+0000","last_became_peered":"2024-04-25T10:07:07.351587+0000","last_unstale":"2024-04-25T10:07:10.116666+0000","last_undegraded":"2024-04-25T10:07:10.116666+0000","last_fullsized":"2024-04-25T10:07:10.116666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340152+0000","last_change":"2024-04-25T10:07:07.329505+0000","last_active":"2024-04-25T10:07:08.340152+0000","last_peered":"2024-04-25T10:07:08.340152+0000","last_clean":"2024-04-25T10:07:08.340152+0000","last_became_active":"2024-04-25T10:07:07.327756+0000","last_became_peered":"2024-04-25T10:07:07.327756+0000","last_unstale":"2024-04-25T10:07:08.340152+0000","last_undegraded":"2024-04-25T10:07:08.340152+0000","last_fullsized":"2024-04-25T10:07:08.340152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116453+0000","last_change":"2024-04-25T10:07:07.354368+0000","last_active":"2024-04-25T10:07:10.116453+0000","last_peered":"2024-04-25T10:07:10.116453+0000","last_clean":"2024-04-25T10:07:10.116453+0000","last_became_active":"2024-04-25T10:07:07.354048+0000","last_became_peered":"2024-04-25T10:07:07.354048+0000","last_unstale":"2024-04-25T10:07:10.116453+0000","last_undegraded":"2024-04-25T10:07:10.116453+0000","last_fullsized":"2024-04-25T10:07:10.116453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430715+0000","last_change":"2024-04-25T10:07:07.337913+0000","last_active":"2024-04-25T10:07:08.430715+0000","last_peered":"2024-04-25T10:07:08.430715+0000","last_clean":"2024-04-25T10:07:08.430715+0000","last_became_active":"2024-04-25T10:07:07.337268+0000","last_became_peered":"2024-04-25T10:07:07.337268+0000","last_unstale":"2024-04-25T10:07:08.430715+0000","last_undegraded":"2024-04-25T10:07:08.430715+0000","last_fullsized":"2024-04-25T10:07:08.430715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:01.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117721+0000","last_change":"2024-04-25T10:07:07.352856+0000","last_active":"2024-04-25T10:07:10.117721+0000","last_peered":"2024-04-25T10:07:10.117721+0000","last_clean":"2024-04-25T10:07:10.117721+0000","last_became_active":"2024-04-25T10:07:07.352555+0000","last_became_peered":"2024-04-25T10:07:07.352555+0000","last_unstale":"2024-04-25T10:07:10.117721+0000","last_undegraded":"2024-04-25T10:07:10.117721+0000","last_fullsized":"2024-04-25T10:07:10.117721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.763141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627954+0000","last_change":"2024-04-25T10:07:07.335997+0000","last_active":"2024-04-25T10:07:09.627954+0000","last_peered":"2024-04-25T10:07:09.627954+0000","last_clean":"2024-04-25T10:07:09.627954+0000","last_became_active":"2024-04-25T10:07:07.334634+0000","last_became_peered":"2024-04-25T10:07:07.334634+0000","last_unstale":"2024-04-25T10:07:09.627954+0000","last_undegraded":"2024-04-25T10:07:09.627954+0000","last_fullsized":"2024-04-25T10:07:09.627954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627400+0000","last_change":"2024-04-25T10:07:07.336470+0000","last_active":"2024-04-25T10:07:09.627400+0000","last_peered":"2024-04-25T10:07:09.627400+0000","last_clean":"2024-04-25T10:07:09.627400+0000","last_became_active":"2024-04-25T10:07:07.334875+0000","last_became_peered":"2024-04-25T10:07:07.334875+0000","last_unstale":"2024-04-25T10:07:09.627400+0000","last_undegraded":"2024-04-25T10:07:09.627400+0000","last_fullsized":"2024-04-25T10:07:09.627400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115690+0000","last_change":"2024-04-25T10:07:07.342067+0000","last_active":"2024-04-25T10:07:10.115690+0000","last_peered":"2024-04-25T10:07:10.115690+0000","last_clean":"2024-04-25T10:07:10.115690+0000","last_became_active":"2024-04-25T10:07:07.341655+0000","last_became_peered":"2024-04-25T10:07:07.341655+0000","last_unstale":"2024-04-25T10:07:10.115690+0000","last_undegraded":"2024-04-25T10:07:10.115690+0000","last_fullsized":"2024-04-25T10:07:10.115690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:37.664841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431262+0000","last_change":"2024-04-25T10:07:07.337652+0000","last_active":"2024-04-25T10:07:08.431262+0000","last_peered":"2024-04-25T10:07:08.431262+0000","last_clean":"2024-04-25T10:07:08.431262+0000","last_became_active":"2024-04-25T10:07:07.337057+0000","last_became_peered":"2024-04-25T10:07:07.337057+0000","last_unstale":"2024-04-25T10:07:08.431262+0000","last_undegraded":"2024-04-25T10:07:08.431262+0000","last_fullsized":"2024-04-25T10:07:08.431262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.012844+0000","last_change":"2024-04-25T10:07:07.341549+0000","last_active":"2024-04-25T10:07:09.012844+0000","last_peered":"2024-04-25T10:07:09.012844+0000","last_clean":"2024-04-25T10:07:09.012844+0000","last_became_active":"2024-04-25T10:07:07.341228+0000","last_became_peered":"2024-04-25T10:07:07.341228+0000","last_unstale":"2024-04-25T10:07:09.012844+0000","last_undegraded":"2024-04-25T10:07:09.012844+0000","last_fullsized":"2024-04-25T10:07:09.012844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:50.697002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339480+0000","last_change":"2024-04-25T10:07:07.348054+0000","last_active":"2024-04-25T10:07:08.339480+0000","last_peered":"2024-04-25T10:07:08.339480+0000","last_clean":"2024-04-25T10:07:08.339480+0000","last_became_active":"2024-04-25T10:07:07.347739+0000","last_became_peered":"2024-04-25T10:07:07.347739+0000","last_unstale":"2024-04-25T10:07:08.339480+0000","last_undegraded":"2024-04-25T10:07:08.339480+0000","last_fullsized":"2024-04-25T10:07:08.339480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339967+0000","last_change":"2024-04-25T10:07:07.326858+0000","last_active":"2024-04-25T10:07:08.339967+0000","last_peered":"2024-04-25T10:07:08.339967+0000","last_clean":"2024-04-25T10:07:08.339967+0000","last_became_active":"2024-04-25T10:07:07.326107+0000","last_became_peered":"2024-04-25T10:07:07.326107+0000","last_unstale":"2024-04-25T10:07:08.339967+0000","last_undegraded":"2024-04-25T10:07:08.339967+0000","last_fullsized":"2024-04-25T10:07:08.339967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.342255+0000","last_change":"2024-04-25T10:07:07.348628+0000","last_active":"2024-04-25T10:07:08.342255+0000","last_peered":"2024-04-25T10:07:08.342255+0000","last_clean":"2024-04-25T10:07:08.342255+0000","last_became_active":"2024-04-25T10:07:07.348356+0000","last_became_peered":"2024-04-25T10:07:07.348356+0000","last_unstale":"2024-04-25T10:07:08.342255+0000","last_undegraded":"2024-04-25T10:07:08.342255+0000","last_fullsized":"2024-04-25T10:07:08.342255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:16.375835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.338486+0000","last_change":"2024-04-25T10:07:07.341185+0000","last_active":"2024-04-25T10:07:08.338486+0000","last_peered":"2024-04-25T10:07:08.338486+0000","last_clean":"2024-04-25T10:07:08.338486+0000","last_became_active":"2024-04-25T10:07:07.340797+0000","last_became_peered":"2024-04-25T10:07:07.340797+0000","last_unstale":"2024-04-25T10:07:08.338486+0000","last_undegraded":"2024-04-25T10:07:08.338486+0000","last_fullsized":"2024-04-25T10:07:08.338486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:49.701271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.118018+0000","last_change":"2024-04-25T10:07:07.356663+0000","last_active":"2024-04-25T10:07:10.118018+0000","last_peered":"2024-04-25T10:07:10.118018+0000","last_clean":"2024-04-25T10:07:10.118018+0000","last_became_active":"2024-04-25T10:07:07.356381+0000","last_became_peered":"2024-04-25T10:07:07.356381+0000","last_unstale":"2024-04-25T10:07:10.118018+0000","last_undegraded":"2024-04-25T10:07:10.118018+0000","last_fullsized":"2024-04-25T10:07:10.118018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:51.771853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627639+0000","last_change":"2024-04-25T10:07:07.348329+0000","last_active":"2024-04-25T10:07:09.627639+0000","last_peered":"2024-04-25T10:07:09.627639+0000","last_clean":"2024-04-25T10:07:09.627639+0000","last_became_active":"2024-04-25T10:07:07.347033+0000","last_became_peered":"2024-04-25T10:07:07.347033+0000","last_unstale":"2024-04-25T10:07:09.627639+0000","last_undegraded":"2024-04-25T10:07:09.627639+0000","last_fullsized":"2024-04-25T10:07:09.627639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013394+0000","last_change":"2024-04-25T10:07:07.350923+0000","last_active":"2024-04-25T10:07:09.013394+0000","last_peered":"2024-04-25T10:07:09.013394+0000","last_clean":"2024-04-25T10:07:09.013394+0000","last_became_active":"2024-04-25T10:07:07.350102+0000","last_became_peered":"2024-04-25T10:07:07.350102+0000","last_unstale":"2024-04-25T10:07:09.013394+0000","last_undegraded":"2024-04-25T10:07:09.013394+0000","last_fullsized":"2024-04-25T10:07:09.013394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340239+0000","last_change":"2024-04-25T10:07:07.351751+0000","last_active":"2024-04-25T10:07:08.340239+0000","last_peered":"2024-04-25T10:07:08.340239+0000","last_clean":"2024-04-25T10:07:08.340239+0000","last_became_active":"2024-04-25T10:07:07.349688+0000","last_became_peered":"2024-04-25T10:07:07.349688+0000","last_unstale":"2024-04-25T10:07:08.340239+0000","last_undegraded":"2024-04-25T10:07:08.340239+0000","last_fullsized":"2024-04-25T10:07:08.340239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430799+0000","last_change":"2024-04-25T10:07:07.346636+0000","last_active":"2024-04-25T10:07:08.430799+0000","last_peered":"2024-04-25T10:07:08.430799+0000","last_clean":"2024-04-25T10:07:08.430799+0000","last_became_active":"2024-04-25T10:07:07.346448+0000","last_became_peered":"2024-04-25T10:07:07.346448+0000","last_unstale":"2024-04-25T10:07:08.430799+0000","last_undegraded":"2024-04-25T10:07:08.430799+0000","last_fullsized":"2024-04-25T10:07:08.430799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.370945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339964+0000","last_change":"2024-04-25T10:07:07.327120+0000","last_active":"2024-04-25T10:07:08.339964+0000","last_peered":"2024-04-25T10:07:08.339964+0000","last_clean":"2024-04-25T10:07:08.339964+0000","last_became_active":"2024-04-25T10:07:07.326471+0000","last_became_peered":"2024-04-25T10:07:07.326471+0000","last_unstale":"2024-04-25T10:07:08.339964+0000","last_undegraded":"2024-04-25T10:07:08.339964+0000","last_fullsized":"2024-04-25T10:07:08.339964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431521+0000","last_change":"2024-04-25T10:07:07.346515+0000","last_active":"2024-04-25T10:07:08.431521+0000","last_peered":"2024-04-25T10:07:08.431521+0000","last_clean":"2024-04-25T10:07:08.431521+0000","last_became_active":"2024-04-25T10:07:07.346218+0000","last_became_peered":"2024-04-25T10:07:07.346218+0000","last_unstale":"2024-04-25T10:07:08.431521+0000","last_undegraded":"2024-04-25T10:07:08.431521+0000","last_fullsized":"2024-04-25T10:07:08.431521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:49.296728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627914+0000","last_change":"2024-04-25T10:07:07.352410+0000","last_active":"2024-04-25T10:07:09.627914+0000","last_peered":"2024-04-25T10:07:09.627914+0000","last_clean":"2024-04-25T10:07:09.627914+0000","last_became_active":"2024-04-25T10:07:07.352106+0000","last_became_peered":"2024-04-25T10:07:07.352106+0000","last_unstale":"2024-04-25T10:07:09.627914+0000","last_undegraded":"2024-04-25T10:07:09.627914+0000","last_fullsized":"2024-04-25T10:07:09.627914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339814+0000","last_change":"2024-04-25T10:07:07.339497+0000","last_active":"2024-04-25T10:07:08.339814+0000","last_peered":"2024-04-25T10:07:08.339814+0000","last_clean":"2024-04-25T10:07:08.339814+0000","last_became_active":"2024-04-25T10:07:07.339168+0000","last_became_peered":"2024-04-25T10:07:07.339168+0000","last_unstale":"2024-04-25T10:07:08.339814+0000","last_undegraded":"2024-04-25T10:07:08.339814+0000","last_fullsized":"2024-04-25T10:07:08.339814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431449+0000","last_change":"2024-04-25T10:07:07.349878+0000","last_active":"2024-04-25T10:07:08.431449+0000","last_peered":"2024-04-25T10:07:08.431449+0000","last_clean":"2024-04-25T10:07:08.431449+0000","last_became_active":"2024-04-25T10:07:07.349602+0000","last_became_peered":"2024-04-25T10:07:07.349602+0000","last_unstale":"2024-04-25T10:07:08.431449+0000","last_undegraded":"2024-04-25T10:07:08.431449+0000","last_fullsized":"2024-04-25T10:07:08.431449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013693+0000","last_change":"2024-04-25T10:07:07.331427+0000","last_active":"2024-04-25T10:07:09.013693+0000","last_peered":"2024-04-25T10:07:09.013693+0000","last_clean":"2024-04-25T10:07:09.013693+0000","last_became_active":"2024-04-25T10:07:07.329911+0000","last_became_peered":"2024-04-25T10:07:07.329911+0000","last_unstale":"2024-04-25T10:07:09.013693+0000","last_undegraded":"2024-04-25T10:07:09.013693+0000","last_fullsized":"2024-04-25T10:07:09.013693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340563+0000","last_change":"2024-04-25T10:07:07.329226+0000","last_active":"2024-04-25T10:07:08.340563+0000","last_peered":"2024-04-25T10:07:08.340563+0000","last_clean":"2024-04-25T10:07:08.340563+0000","last_became_active":"2024-04-25T10:07:07.327329+0000","last_became_peered":"2024-04-25T10:07:07.327329+0000","last_unstale":"2024-04-25T10:07:08.340563+0000","last_undegraded":"2024-04-25T10:07:08.340563+0000","last_fullsized":"2024-04-25T10:07:08.340563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:49.296728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337552+0000","last_change":"2024-04-25T10:07:07.327804+0000","last_active":"2024-04-25T10:07:08.337552+0000","last_peered":"2024-04-25T10:07:08.337552+0000","last_clean":"2024-04-25T10:07:08.337552+0000","last_became_active":"2024-04-25T10:07:07.327516+0000","last_became_peered":"2024-04-25T10:07:07.327516+0000","last_unstale":"2024-04-25T10:07:08.337552+0000","last_undegraded":"2024-04-25T10:07:08.337552+0000","last_fullsized":"2024-04-25T10:07:08.337552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:46.704814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627063+0000","last_change":"2024-04-25T10:07:07.333594+0000","last_active":"2024-04-25T10:07:09.627063+0000","last_peered":"2024-04-25T10:07:09.627063+0000","last_clean":"2024-04-25T10:07:09.627063+0000","last_became_active":"2024-04-25T10:07:07.333302+0000","last_became_peered":"2024-04-25T10:07:07.333302+0000","last_unstale":"2024-04-25T10:07:09.627063+0000","last_undegraded":"2024-04-25T10:07:09.627063+0000","last_fullsized":"2024-04-25T10:07:09.627063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:01.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.114966+0000","last_change":"2024-04-25T10:07:07.333101+0000","last_active":"2024-04-25T10:07:10.114966+0000","last_peered":"2024-04-25T10:07:10.114966+0000","last_clean":"2024-04-25T10:07:10.114966+0000","last_became_active":"2024-04-25T10:07:07.332714+0000","last_became_peered":"2024-04-25T10:07:07.332714+0000","last_unstale":"2024-04-25T10:07:10.114966+0000","last_undegraded":"2024-04-25T10:07:10.114966+0000","last_fullsized":"2024-04-25T10:07:10.114966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:53.471918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431440+0000","last_change":"2024-04-25T10:07:06.322470+0000","last_active":"2024-04-25T10:07:08.431440+0000","last_peered":"2024-04-25T10:07:08.431440+0000","last_clean":"2024-04-25T10:07:08.431440+0000","last_became_active":"2024-04-25T10:07:06.321092+0000","last_became_peered":"2024-04-25T10:07:06.321092+0000","last_unstale":"2024-04-25T10:07:08.431440+0000","last_undegraded":"2024-04-25T10:07:08.431440+0000","last_fullsized":"2024-04-25T10:07:08.431440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:28.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013819+0000","last_change":"2024-04-25T10:07:07.340326+0000","last_active":"2024-04-25T10:07:09.013819+0000","last_peered":"2024-04-25T10:07:09.013819+0000","last_clean":"2024-04-25T10:07:09.013819+0000","last_became_active":"2024-04-25T10:07:07.339889+0000","last_became_peered":"2024-04-25T10:07:07.339889+0000","last_unstale":"2024-04-25T10:07:09.013819+0000","last_undegraded":"2024-04-25T10:07:09.013819+0000","last_fullsized":"2024-04-25T10:07:09.013819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.626769+0000","last_change":"2024-04-25T10:07:06.333386+0000","last_active":"2024-04-25T10:07:09.626769+0000","last_peered":"2024-04-25T10:07:09.626769+0000","last_clean":"2024-04-25T10:07:09.626769+0000","last_became_active":"2024-04-25T10:07:06.333014+0000","last_became_peered":"2024-04-25T10:07:06.333014+0000","last_unstale":"2024-04-25T10:07:09.626769+0000","last_undegraded":"2024-04-25T10:07:09.626769+0000","last_fullsized":"2024-04-25T10:07:09.626769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:53.885696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430668+0000","last_change":"2024-04-25T10:07:06.314017+0000","last_active":"2024-04-25T10:07:08.430668+0000","last_peered":"2024-04-25T10:07:08.430668+0000","last_clean":"2024-04-25T10:07:08.430668+0000","last_became_active":"2024-04-25T10:07:06.313712+0000","last_became_peered":"2024-04-25T10:07:06.313712+0000","last_unstale":"2024-04-25T10:07:08.430668+0000","last_undegraded":"2024-04-25T10:07:08.430668+0000","last_fullsized":"2024-04-25T10:07:08.430668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:05.470696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430845+0000","last_change":"2024-04-25T10:07:07.323982+0000","last_active":"2024-04-25T10:07:08.430845+0000","last_peered":"2024-04-25T10:07:08.430845+0000","last_clean":"2024-04-25T10:07:08.430845+0000","last_became_active":"2024-04-25T10:07:07.323664+0000","last_became_peered":"2024-04-25T10:07:07.323664+0000","last_unstale":"2024-04-25T10:07:08.430845+0000","last_undegraded":"2024-04-25T10:07:08.430845+0000","last_fullsized":"2024-04-25T10:07:08.430845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340135+0000","last_change":"2024-04-25T10:07:06.312709+0000","last_active":"2024-04-25T10:07:08.340135+0000","last_peered":"2024-04-25T10:07:08.340135+0000","last_clean":"2024-04-25T10:07:08.340135+0000","last_became_active":"2024-04-25T10:07:06.311785+0000","last_became_peered":"2024-04-25T10:07:06.311785+0000","last_unstale":"2024-04-25T10:07:08.340135+0000","last_undegraded":"2024-04-25T10:07:08.340135+0000","last_fullsized":"2024-04-25T10:07:08.340135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:52.458435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339243+0000","last_change":"2024-04-25T10:07:07.332051+0000","last_active":"2024-04-25T10:07:08.339243+0000","last_peered":"2024-04-25T10:07:08.339243+0000","last_clean":"2024-04-25T10:07:08.339243+0000","last_became_active":"2024-04-25T10:07:07.331656+0000","last_became_peered":"2024-04-25T10:07:07.331656+0000","last_unstale":"2024-04-25T10:07:08.339243+0000","last_undegraded":"2024-04-25T10:07:08.339243+0000","last_fullsized":"2024-04-25T10:07:08.339243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627326+0000","last_change":"2024-04-25T10:07:06.316859+0000","last_active":"2024-04-25T10:07:09.627326+0000","last_peered":"2024-04-25T10:07:09.627326+0000","last_clean":"2024-04-25T10:07:09.627326+0000","last_became_active":"2024-04-25T10:07:06.314454+0000","last_became_peered":"2024-04-25T10:07:06.314454+0000","last_unstale":"2024-04-25T10:07:09.627326+0000","last_undegraded":"2024-04-25T10:07:09.627326+0000","last_fullsized":"2024-04-25T10:07:09.627326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.867432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341342+0000","last_change":"2024-04-25T10:07:07.352036+0000","last_active":"2024-04-25T10:07:08.341342+0000","last_peered":"2024-04-25T10:07:08.341342+0000","last_clean":"2024-04-25T10:07:08.341342+0000","last_became_active":"2024-04-25T10:07:07.351244+0000","last_became_peered":"2024-04-25T10:07:07.351244+0000","last_unstale":"2024-04-25T10:07:08.341342+0000","last_undegraded":"2024-04-25T10:07:08.341342+0000","last_fullsized":"2024-04-25T10:07:08.341342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:31.725733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341722+0000","last_change":"2024-04-25T10:07:06.313325+0000","last_active":"2024-04-25T10:07:08.341722+0000","last_peered":"2024-04-25T10:07:08.341722+0000","last_clean":"2024-04-25T10:07:08.341722+0000","last_became_active":"2024-04-25T10:07:06.312922+0000","last_became_peered":"2024-04-25T10:07:06.312922+0000","last_unstale":"2024-04-25T10:07:08.341722+0000","last_undegraded":"2024-04-25T10:07:08.341722+0000","last_fullsized":"2024-04-25T10:07:08.341722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013388+0000","last_change":"2024-04-25T10:07:07.334790+0000","last_active":"2024-04-25T10:07:09.013388+0000","last_peered":"2024-04-25T10:07:09.013388+0000","last_clean":"2024-04-25T10:07:09.013388+0000","last_became_active":"2024-04-25T10:07:07.334587+0000","last_became_peered":"2024-04-25T10:07:07.334587+0000","last_unstale":"2024-04-25T10:07:09.013388+0000","last_undegraded":"2024-04-25T10:07:09.013388+0000","last_fullsized":"2024-04-25T10:07:09.013388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340029+0000","last_change":"2024-04-25T10:07:06.317802+0000","last_active":"2024-04-25T10:07:08.340029+0000","last_peered":"2024-04-25T10:07:08.340029+0000","last_clean":"2024-04-25T10:07:08.340029+0000","last_became_active":"2024-04-25T10:07:06.317291+0000","last_became_peered":"2024-04-25T10:07:06.317291+0000","last_unstale":"2024-04-25T10:07:08.340029+0000","last_undegraded":"2024-04-25T10:07:08.340029+0000","last_fullsized":"2024-04-25T10:07:08.340029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.867432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115871+0000","last_change":"2024-04-25T10:07:07.351461+0000","last_active":"2024-04-25T10:07:10.115871+0000","last_peered":"2024-04-25T10:07:10.115871+0000","last_clean":"2024-04-25T10:07:10.115871+0000","last_became_active":"2024-04-25T10:07:07.351141+0000","last_became_peered":"2024-04-25T10:07:07.351141+0000","last_unstale":"2024-04-25T10:07:10.115871+0000","last_undegraded":"2024-04-25T10:07:10.115871+0000","last_fullsized":"2024-04-25T10:07:10.115871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:46.704814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627332+0000","last_change":"2024-04-25T10:07:06.319663+0000","last_active":"2024-04-25T10:07:09.627332+0000","last_peered":"2024-04-25T10:07:09.627332+0000","last_clean":"2024-04-25T10:07:09.627332+0000","last_became_active":"2024-04-25T10:07:06.317321+0000","last_became_peered":"2024-04-25T10:07:06.317321+0000","last_unstale":"2024-04-25T10:07:09.627332+0000","last_undegraded":"2024-04-25T10:07:09.627332+0000","last_fullsized":"2024-04-25T10:07:09.627332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117185+0000","last_change":"2024-04-25T10:07:07.350498+0000","last_active":"2024-04-25T10:07:10.117185+0000","last_peered":"2024-04-25T10:07:10.117185+0000","last_clean":"2024-04-25T10:07:10.117185+0000","last_became_active":"2024-04-25T10:07:07.349929+0000","last_became_peered":"2024-04-25T10:07:07.349929+0000","last_unstale":"2024-04-25T10:07:10.117185+0000","last_undegraded":"2024-04-25T10:07:10.117185+0000","last_fullsized":"2024-04-25T10:07:10.117185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:23.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013849+0000","last_change":"2024-04-25T10:07:06.312357+0000","last_active":"2024-04-25T10:07:09.013849+0000","last_peered":"2024-04-25T10:07:09.013849+0000","last_clean":"2024-04-25T10:07:09.013849+0000","last_became_active":"2024-04-25T10:07:06.312088+0000","last_became_peered":"2024-04-25T10:07:06.312088+0000","last_unstale":"2024-04-25T10:07:09.013849+0000","last_undegraded":"2024-04-25T10:07:09.013849+0000","last_fullsized":"2024-04-25T10:07:09.013849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:05.470696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013146+0000","last_change":"2024-04-25T10:07:07.345439+0000","last_active":"2024-04-25T10:07:09.013146+0000","last_peered":"2024-04-25T10:07:09.013146+0000","last_clean":"2024-04-25T10:07:09.013146+0000","last_became_active":"2024-04-25T10:07:07.345205+0000","last_became_peered":"2024-04-25T10:07:07.345205+0000","last_unstale":"2024-04-25T10:07:09.013146+0000","last_undegraded":"2024-04-25T10:07:09.013146+0000","last_fullsized":"2024-04-25T10:07:09.013146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:56.522956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340351+0000","last_change":"2024-04-25T10:07:06.313807+0000","last_active":"2024-04-25T10:07:08.340351+0000","last_peered":"2024-04-25T10:07:08.340351+0000","last_clean":"2024-04-25T10:07:08.340351+0000","last_became_active":"2024-04-25T10:07:06.313376+0000","last_became_peered":"2024-04-25T10:07:06.313376+0000","last_unstale":"2024-04-25T10:07:08.340351+0000","last_undegraded":"2024-04-25T10:07:08.340351+0000","last_fullsized":"2024-04-25T10:07:08.340351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:44.436918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116968+0000","last_change":"2024-04-25T10:07:07.351251+0000","last_active":"2024-04-25T10:07:10.116968+0000","last_peered":"2024-04-25T10:07:10.116968+0000","last_clean":"2024-04-25T10:07:10.116968+0000","last_became_active":"2024-04-25T10:07:07.350404+0000","last_became_peered":"2024-04-25T10:07:07.350404+0000","last_unstale":"2024-04-25T10:07:10.116968+0000","last_undegraded":"2024-04-25T10:07:10.116968+0000","last_fullsized":"2024-04-25T10:07:10.116968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:49.296728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.626713+0000","last_change":"2024-04-25T10:07:06.322457+0000","last_active":"2024-04-25T10:07:09.626713+0000","last_peered":"2024-04-25T10:07:09.626713+0000","last_clean":"2024-04-25T10:07:09.626713+0000","last_became_active":"2024-04-25T10:07:06.318116+0000","last_became_peered":"2024-04-25T10:07:06.318116+0000","last_unstale":"2024-04-25T10:07:09.626713+0000","last_undegraded":"2024-04-25T10:07:09.626713+0000","last_fullsized":"2024-04-25T10:07:09.626713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:44.436918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627151+0000","last_change":"2024-04-25T10:07:07.328355+0000","last_active":"2024-04-25T10:07:09.627151+0000","last_peered":"2024-04-25T10:07:09.627151+0000","last_clean":"2024-04-25T10:07:09.627151+0000","last_became_active":"2024-04-25T10:07:07.328088+0000","last_became_peered":"2024-04-25T10:07:07.328088+0000","last_unstale":"2024-04-25T10:07:09.627151+0000","last_undegraded":"2024-04-25T10:07:09.627151+0000","last_fullsized":"2024-04-25T10:07:09.627151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337372+0000","last_change":"2024-04-25T10:07:06.306731+0000","last_active":"2024-04-25T10:07:08.337372+0000","last_peered":"2024-04-25T10:07:08.337372+0000","last_clean":"2024-04-25T10:07:08.337372+0000","last_became_active":"2024-04-25T10:07:06.306314+0000","last_became_peered":"2024-04-25T10:07:06.306314+0000","last_unstale":"2024-04-25T10:07:08.337372+0000","last_undegraded":"2024-04-25T10:07:08.337372+0000","last_fullsized":"2024-04-25T10:07:08.337372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:28.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117145+0000","last_change":"2024-04-25T10:07:07.356371+0000","last_active":"2024-04-25T10:07:10.117145+0000","last_peered":"2024-04-25T10:07:10.117145+0000","last_clean":"2024-04-25T10:07:10.117145+0000","last_became_active":"2024-04-25T10:07:07.352731+0000","last_became_peered":"2024-04-25T10:07:07.352731+0000","last_unstale":"2024-04-25T10:07:10.117145+0000","last_undegraded":"2024-04-25T10:07:10.117145+0000","last_fullsized":"2024-04-25T10:07:10.117145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115672+0000","last_change":"2024-04-25T10:07:06.320436+0000","last_active":"2024-04-25T10:07:10.115672+0000","last_peered":"2024-04-25T10:07:10.115672+0000","last_clean":"2024-04-25T10:07:10.115672+0000","last_became_active":"2024-04-25T10:07:06.319958+0000","last_became_peered":"2024-04-25T10:07:06.319958+0000","last_unstale":"2024-04-25T10:07:10.115672+0000","last_undegraded":"2024-04-25T10:07:10.115672+0000","last_fullsized":"2024-04-25T10:07:10.115672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:02.968109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117278+0000","last_change":"2024-04-25T10:07:07.352988+0000","last_active":"2024-04-25T10:07:10.117278+0000","last_peered":"2024-04-25T10:07:10.117278+0000","last_clean":"2024-04-25T10:07:10.117278+0000","last_became_active":"2024-04-25T10:07:07.352585+0000","last_became_peered":"2024-04-25T10:07:07.352585+0000","last_unstale":"2024-04-25T10:07:10.117278+0000","last_undegraded":"2024-04-25T10:07:10.117278+0000","last_fullsized":"2024-04-25T10:07:10.117278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:35.056084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.338565+0000","last_change":"2024-04-25T10:07:06.308389+0000","last_active":"2024-04-25T10:07:08.338565+0000","last_peered":"2024-04-25T10:07:08.338565+0000","last_clean":"2024-04-25T10:07:08.338565+0000","last_became_active":"2024-04-25T10:07:06.308046+0000","last_became_peered":"2024-04-25T10:07:06.308046+0000","last_unstale":"2024-04-25T10:07:08.338565+0000","last_undegraded":"2024-04-25T10:07:08.338565+0000","last_fullsized":"2024-04-25T10:07:08.338565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:53.885696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.118014+0000","last_change":"2024-04-25T10:07:07.357349+0000","last_active":"2024-04-25T10:07:10.118014+0000","last_peered":"2024-04-25T10:07:10.118014+0000","last_clean":"2024-04-25T10:07:10.118014+0000","last_became_active":"2024-04-25T10:07:07.356970+0000","last_became_peered":"2024-04-25T10:07:07.356970+0000","last_unstale":"2024-04-25T10:07:10.118014+0000","last_undegraded":"2024-04-25T10:07:10.118014+0000","last_fullsized":"2024-04-25T10:07:10.118014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:16.375835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340105+0000","last_change":"2024-04-25T10:07:06.310585+0000","last_active":"2024-04-25T10:07:08.340105+0000","last_peered":"2024-04-25T10:07:08.340105+0000","last_clean":"2024-04-25T10:07:08.340105+0000","last_became_active":"2024-04-25T10:07:06.310265+0000","last_became_peered":"2024-04-25T10:07:06.310265+0000","last_unstale":"2024-04-25T10:07:08.340105+0000","last_undegraded":"2024-04-25T10:07:08.340105+0000","last_fullsized":"2024-04-25T10:07:08.340105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:48.687788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341312+0000","last_change":"2024-04-25T10:07:07.337180+0000","last_active":"2024-04-25T10:07:08.341312+0000","last_peered":"2024-04-25T10:07:08.341312+0000","last_clean":"2024-04-25T10:07:08.341312+0000","last_became_active":"2024-04-25T10:07:07.335230+0000","last_became_peered":"2024-04-25T10:07:07.335230+0000","last_unstale":"2024-04-25T10:07:08.341312+0000","last_undegraded":"2024-04-25T10:07:08.341312+0000","last_fullsized":"2024-04-25T10:07:08.341312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:16.166171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337856+0000","last_change":"2024-04-25T10:07:06.311469+0000","last_active":"2024-04-25T10:07:08.337856+0000","last_peered":"2024-04-25T10:07:08.337856+0000","last_clean":"2024-04-25T10:07:08.337856+0000","last_became_active":"2024-04-25T10:07:06.311182+0000","last_became_peered":"2024-04-25T10:07:06.311182+0000","last_unstale":"2024-04-25T10:07:08.337856+0000","last_undegraded":"2024-04-25T10:07:08.337856+0000","last_fullsized":"2024-04-25T10:07:08.337856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:44.436918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117507+0000","last_change":"2024-04-25T10:07:07.350766+0000","last_active":"2024-04-25T10:07:10.117507+0000","last_peered":"2024-04-25T10:07:10.117507+0000","last_clean":"2024-04-25T10:07:10.117507+0000","last_became_active":"2024-04-25T10:07:07.350181+0000","last_became_peered":"2024-04-25T10:07:07.350181+0000","last_unstale":"2024-04-25T10:07:10.117507+0000","last_undegraded":"2024-04-25T10:07:10.117507+0000","last_fullsized":"2024-04-25T10:07:10.117507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:16.166171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116058+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:07:10.116058+0000","last_peered":"2024-04-25T10:07:10.116058+0000","last_clean":"2024-04-25T10:07:10.116058+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:07:10.116058+0000","last_undegraded":"2024-04-25T10:07:10.116058+0000","last_fullsized":"2024-04-25T10:07:10.116058+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340032+0000","last_change":"2024-04-25T10:07:06.306253+0000","last_active":"2024-04-25T10:07:08.340032+0000","last_peered":"2024-04-25T10:07:08.340032+0000","last_clean":"2024-04-25T10:07:08.340032+0000","last_became_active":"2024-04-25T10:07:06.305849+0000","last_became_peered":"2024-04-25T10:07:06.305849+0000","last_unstale":"2024-04-25T10:07:08.340032+0000","last_undegraded":"2024-04-25T10:07:08.340032+0000","last_fullsized":"2024-04-25T10:07:08.340032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:28.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430815+0000","last_change":"2024-04-25T10:07:07.337895+0000","last_active":"2024-04-25T10:07:08.430815+0000","last_peered":"2024-04-25T10:07:08.430815+0000","last_clean":"2024-04-25T10:07:08.430815+0000","last_became_active":"2024-04-25T10:07:07.337609+0000","last_became_peered":"2024-04-25T10:07:07.337609+0000","last_unstale":"2024-04-25T10:07:08.430815+0000","last_undegraded":"2024-04-25T10:07:08.430815+0000","last_fullsized":"2024-04-25T10:07:08.430815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115960+0000","last_change":"2024-04-25T10:07:06.325347+0000","last_active":"2024-04-25T10:07:10.115960+0000","last_peered":"2024-04-25T10:07:10.115960+0000","last_clean":"2024-04-25T10:07:10.115960+0000","last_became_active":"2024-04-25T10:07:06.324740+0000","last_became_peered":"2024-04-25T10:07:06.324740+0000","last_unstale":"2024-04-25T10:07:10.115960+0000","last_undegraded":"2024-04-25T10:07:10.115960+0000","last_fullsized":"2024-04-25T10:07:10.115960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.691331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.012839+0000","last_change":"2024-04-25T10:07:07.337867+0000","last_active":"2024-04-25T10:07:09.012839+0000","last_peered":"2024-04-25T10:07:09.012839+0000","last_clean":"2024-04-25T10:07:09.012839+0000","last_became_active":"2024-04-25T10:07:07.337521+0000","last_became_peered":"2024-04-25T10:07:07.337521+0000","last_unstale":"2024-04-25T10:07:09.012839+0000","last_undegraded":"2024-04-25T10:07:09.012839+0000","last_fullsized":"2024-04-25T10:07:09.012839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:05.611848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337610+0000","last_change":"2024-04-25T10:07:06.308483+0000","last_active":"2024-04-25T10:07:08.337610+0000","last_peered":"2024-04-25T10:07:08.337610+0000","last_clean":"2024-04-25T10:07:08.337610+0000","last_became_active":"2024-04-25T10:07:06.308156+0000","last_became_peered":"2024-04-25T10:07:06.308156+0000","last_unstale":"2024-04-25T10:07:08.337610+0000","last_undegraded":"2024-04-25T10:07:08.337610+0000","last_fullsized":"2024-04-25T10:07:08.337610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:25.307131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339704+0000","last_change":"2024-04-25T10:07:07.351559+0000","last_active":"2024-04-25T10:07:08.339704+0000","last_peered":"2024-04-25T10:07:08.339704+0000","last_clean":"2024-04-25T10:07:08.339704+0000","last_became_active":"2024-04-25T10:07:07.351271+0000","last_became_peered":"2024-04-25T10:07:07.351271+0000","last_unstale":"2024-04-25T10:07:08.339704+0000","last_undegraded":"2024-04-25T10:07:08.339704+0000","last_fullsized":"2024-04-25T10:07:08.339704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340604+0000","last_change":"2024-04-25T10:07:06.314916+0000","last_active":"2024-04-25T10:07:08.340604+0000","last_peered":"2024-04-25T10:07:08.340604+0000","last_clean":"2024-04-25T10:07:08.340604+0000","last_became_active":"2024-04-25T10:07:06.314572+0000","last_became_peered":"2024-04-25T10:07:06.314572+0000","last_unstale":"2024-04-25T10:07:08.340604+0000","last_undegraded":"2024-04-25T10:07:08.340604+0000","last_fullsized":"2024-04-25T10:07:08.340604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:53.885696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115644+0000","last_change":"2024-04-25T10:07:07.342482+0000","last_active":"2024-04-25T10:07:10.115644+0000","last_peered":"2024-04-25T10:07:10.115644+0000","last_clean":"2024-04-25T10:07:10.115644+0000","last_became_active":"2024-04-25T10:07:07.341566+0000","last_became_peered":"2024-04-25T10:07:07.341566+0000","last_unstale":"2024-04-25T10:07:10.115644+0000","last_undegraded":"2024-04-25T10:07:10.115644+0000","last_fullsized":"2024-04-25T10:07:10.115644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:14.880915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339445+0000","last_change":"2024-04-25T10:07:06.317437+0000","last_active":"2024-04-25T10:07:08.339445+0000","last_peered":"2024-04-25T10:07:08.339445+0000","last_clean":"2024-04-25T10:07:08.339445+0000","last_became_active":"2024-04-25T10:07:06.316494+0000","last_became_peered":"2024-04-25T10:07:06.316494+0000","last_unstale":"2024-04-25T10:07:08.339445+0000","last_undegraded":"2024-04-25T10:07:08.339445+0000","last_fullsized":"2024-04-25T10:07:08.339445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341452+0000","last_change":"2024-04-25T10:07:07.337051+0000","last_active":"2024-04-25T10:07:08.341452+0000","last_peered":"2024-04-25T10:07:08.341452+0000","last_clean":"2024-04-25T10:07:08.341452+0000","last_became_active":"2024-04-25T10:07:07.334884+0000","last_became_peered":"2024-04-25T10:07:07.334884+0000","last_unstale":"2024-04-25T10:07:08.341452+0000","last_undegraded":"2024-04-25T10:07:08.341452+0000","last_fullsized":"2024-04-25T10:07:08.341452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.763141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117681+0000","last_change":"2024-04-25T10:07:06.331016+0000","last_active":"2024-04-25T10:07:10.117681+0000","last_peered":"2024-04-25T10:07:10.117681+0000","last_clean":"2024-04-25T10:07:10.117681+0000","last_became_active":"2024-04-25T10:07:06.330712+0000","last_became_peered":"2024-04-25T10:07:06.330712+0000","last_unstale":"2024-04-25T10:07:10.117681+0000","last_undegraded":"2024-04-25T10:07:10.117681+0000","last_fullsized":"2024-04-25T10:07:10.117681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.867432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339661+0000","last_change":"2024-04-25T10:07:07.351856+0000","last_active":"2024-04-25T10:07:08.339661+0000","last_peered":"2024-04-25T10:07:08.339661+0000","last_clean":"2024-04-25T10:07:08.339661+0000","last_became_active":"2024-04-25T10:07:07.351585+0000","last_became_peered":"2024-04-25T10:07:07.351585+0000","last_unstale":"2024-04-25T10:07:08.339661+0000","last_undegraded":"2024-04-25T10:07:08.339661+0000","last_fullsized":"2024-04-25T10:07:08.339661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340221+0000","last_change":"2024-04-25T10:07:06.317585+0000","last_active":"2024-04-25T10:07:08.340221+0000","last_peered":"2024-04-25T10:07:08.340221+0000","last_clean":"2024-04-25T10:07:08.340221+0000","last_became_active":"2024-04-25T10:07:06.316964+0000","last_became_peered":"2024-04-25T10:07:06.316964+0000","last_unstale":"2024-04-25T10:07:08.340221+0000","last_undegraded":"2024-04-25T10:07:08.340221+0000","last_fullsized":"2024-04-25T10:07:08.340221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:58.603977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431612+0000","last_change":"2024-04-25T10:07:07.352730+0000","last_active":"2024-04-25T10:07:08.431612+0000","last_peered":"2024-04-25T10:07:08.431612+0000","last_clean":"2024-04-25T10:07:08.431612+0000","last_became_active":"2024-04-25T10:07:07.352486+0000","last_became_peered":"2024-04-25T10:07:07.352486+0000","last_unstale":"2024-04-25T10:07:08.431612+0000","last_undegraded":"2024-04-25T10:07:08.431612+0000","last_fullsized":"2024-04-25T10:07:08.431612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:23.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627361+0000","last_change":"2024-04-25T10:07:06.327061+0000","last_active":"2024-04-25T10:07:09.627361+0000","last_peered":"2024-04-25T10:07:09.627361+0000","last_clean":"2024-04-25T10:07:09.627361+0000","last_became_active":"2024-04-25T10:07:06.326227+0000","last_became_peered":"2024-04-25T10:07:06.326227+0000","last_unstale":"2024-04-25T10:07:09.627361+0000","last_undegraded":"2024-04-25T10:07:09.627361+0000","last_fullsized":"2024-04-25T10:07:09.627361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013232+0000","last_change":"2024-04-25T10:07:07.326642+0000","last_active":"2024-04-25T10:07:09.013232+0000","last_peered":"2024-04-25T10:07:09.013232+0000","last_clean":"2024-04-25T10:07:09.013232+0000","last_became_active":"2024-04-25T10:07:07.326261+0000","last_became_peered":"2024-04-25T10:07:07.326261+0000","last_unstale":"2024-04-25T10:07:09.013232+0000","last_undegraded":"2024-04-25T10:07:09.013232+0000","last_fullsized":"2024-04-25T10:07:09.013232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:01.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115657+0000","last_change":"2024-04-25T10:07:06.325264+0000","last_active":"2024-04-25T10:07:10.115657+0000","last_peered":"2024-04-25T10:07:10.115657+0000","last_clean":"2024-04-25T10:07:10.115657+0000","last_became_active":"2024-04-25T10:07:06.324704+0000","last_became_peered":"2024-04-25T10:07:06.324704+0000","last_unstale":"2024-04-25T10:07:10.115657+0000","last_undegraded":"2024-04-25T10:07:10.115657+0000","last_fullsized":"2024-04-25T10:07:10.115657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:52.458435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115184+0000","last_change":"2024-04-25T10:07:07.340986+0000","last_active":"2024-04-25T10:07:10.115184+0000","last_peered":"2024-04-25T10:07:10.115184+0000","last_clean":"2024-04-25T10:07:10.115184+0000","last_became_active":"2024-04-25T10:07:07.332637+0000","last_became_peered":"2024-04-25T10:07:07.332637+0000","last_unstale":"2024-04-25T10:07:10.115184+0000","last_undegraded":"2024-04-25T10:07:10.115184+0000","last_fullsized":"2024-04-25T10:07:10.115184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:17.677957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339302+0000","last_change":"2024-04-25T10:07:06.324149+0000","last_active":"2024-04-25T10:07:08.339302+0000","last_peered":"2024-04-25T10:07:08.339302+0000","last_clean":"2024-04-25T10:07:08.339302+0000","last_became_active":"2024-04-25T10:07:06.323900+0000","last_became_peered":"2024-04-25T10:07:06.323900+0000","last_unstale":"2024-04-25T10:07:08.339302+0000","last_undegraded":"2024-04-25T10:07:08.339302+0000","last_fullsized":"2024-04-25T10:07:08.339302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431004+0000","last_change":"2024-04-25T10:07:07.337614+0000","last_active":"2024-04-25T10:07:08.431004+0000","last_peered":"2024-04-25T10:07:08.431004+0000","last_clean":"2024-04-25T10:07:08.431004+0000","last_became_active":"2024-04-25T10:07:07.333283+0000","last_became_peered":"2024-04-25T10:07:07.333283+0000","last_unstale":"2024-04-25T10:07:08.431004+0000","last_undegraded":"2024-04-25T10:07:08.431004+0000","last_fullsized":"2024-04-25T10:07:08.431004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339925+0000","last_change":"2024-04-25T10:07:06.317292+0000","last_active":"2024-04-25T10:07:08.339925+0000","last_peered":"2024-04-25T10:07:08.339925+0000","last_clean":"2024-04-25T10:07:08.339925+0000","last_became_active":"2024-04-25T10:07:06.316862+0000","last_became_peered":"2024-04-25T10:07:06.316862+0000","last_unstale":"2024-04-25T10:07:08.339925+0000","last_undegraded":"2024-04-25T10:07:08.339925+0000","last_fullsized":"2024-04-25T10:07:08.339925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.691331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431273+0000","last_change":"2024-04-25T10:07:07.329541+0000","last_active":"2024-04-25T10:07:08.431273+0000","last_peered":"2024-04-25T10:07:08.431273+0000","last_clean":"2024-04-25T10:07:08.431273+0000","last_became_active":"2024-04-25T10:07:07.329296+0000","last_became_peered":"2024-04-25T10:07:07.329296+0000","last_unstale":"2024-04-25T10:07:08.431273+0000","last_undegraded":"2024-04-25T10:07:08.431273+0000","last_fullsized":"2024-04-25T10:07:08.431273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117039+0000","last_change":"2024-04-25T10:07:06.331241+0000","last_active":"2024-04-25T10:07:10.117039+0000","last_peered":"2024-04-25T10:07:10.117039+0000","last_clean":"2024-04-25T10:07:10.117039+0000","last_became_active":"2024-04-25T10:07:06.330637+0000","last_became_peered":"2024-04-25T10:07:06.330637+0000","last_unstale":"2024-04-25T10:07:10.117039+0000","last_undegraded":"2024-04-25T10:07:10.117039+0000","last_fullsized":"2024-04-25T10:07:10.117039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627101+0000","last_change":"2024-04-25T10:07:07.334097+0000","last_active":"2024-04-25T10:07:09.627101+0000","last_peered":"2024-04-25T10:07:09.627101+0000","last_clean":"2024-04-25T10:07:09.627101+0000","last_became_active":"2024-04-25T10:07:07.332086+0000","last_became_peered":"2024-04-25T10:07:07.332086+0000","last_unstale":"2024-04-25T10:07:09.627101+0000","last_undegraded":"2024-04-25T10:07:09.627101+0000","last_fullsized":"2024-04-25T10:07:09.627101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.370945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341238+0000","last_change":"2024-04-25T10:07:07.349145+0000","last_active":"2024-04-25T10:07:08.341238+0000","last_peered":"2024-04-25T10:07:08.341238+0000","last_clean":"2024-04-25T10:07:08.341238+0000","last_became_active":"2024-04-25T10:07:07.348916+0000","last_became_peered":"2024-04-25T10:07:07.348916+0000","last_unstale":"2024-04-25T10:07:08.341238+0000","last_undegraded":"2024-04-25T10:07:08.341238+0000","last_fullsized":"2024-04-25T10:07:08.341238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:35.056084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116709+0000","last_change":"2024-04-25T10:07:06.330571+0000","last_active":"2024-04-25T10:07:10.116709+0000","last_peered":"2024-04-25T10:07:10.116709+0000","last_clean":"2024-04-25T10:07:10.116709+0000","last_became_active":"2024-04-25T10:07:06.329886+0000","last_became_peered":"2024-04-25T10:07:06.329886+0000","last_unstale":"2024-04-25T10:07:10.116709+0000","last_undegraded":"2024-04-25T10:07:10.116709+0000","last_fullsized":"2024-04-25T10:07:10.116709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340935+0000","last_change":"2024-04-25T10:07:07.347234+0000","last_active":"2024-04-25T10:07:08.340935+0000","last_peered":"2024-04-25T10:07:08.340935+0000","last_clean":"2024-04-25T10:07:08.340935+0000","last_became_active":"2024-04-25T10:07:07.346961+0000","last_became_peered":"2024-04-25T10:07:07.346961+0000","last_unstale":"2024-04-25T10:07:08.340935+0000","last_undegraded":"2024-04-25T10:07:08.340935+0000","last_fullsized":"2024-04-25T10:07:08.340935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:23.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117348+0000","last_change":"2024-04-25T10:07:06.330806+0000","last_active":"2024-04-25T10:07:10.117348+0000","last_peered":"2024-04-25T10:07:10.117348+0000","last_clean":"2024-04-25T10:07:10.117348+0000","last_became_active":"2024-04-25T10:07:06.330360+0000","last_became_peered":"2024-04-25T10:07:06.330360+0000","last_unstale":"2024-04-25T10:07:10.117348+0000","last_undegraded":"2024-04-25T10:07:10.117348+0000","last_fullsized":"2024-04-25T10:07:10.117348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:48.687788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013341+0000","last_change":"2024-04-25T10:07:07.337072+0000","last_active":"2024-04-25T10:07:09.013341+0000","last_peered":"2024-04-25T10:07:09.013341+0000","last_clean":"2024-04-25T10:07:09.013341+0000","last_became_active":"2024-04-25T10:07:07.336767+0000","last_became_peered":"2024-04-25T10:07:07.336767+0000","last_unstale":"2024-04-25T10:07:09.013341+0000","last_undegraded":"2024-04-25T10:07:09.013341+0000","last_fullsized":"2024-04-25T10:07:09.013341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.370945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431572+0000","last_change":"2024-04-25T10:07:06.322321+0000","last_active":"2024-04-25T10:07:08.431572+0000","last_peered":"2024-04-25T10:07:08.431572+0000","last_clean":"2024-04-25T10:07:08.431572+0000","last_became_active":"2024-04-25T10:07:06.321990+0000","last_became_peered":"2024-04-25T10:07:06.321990+0000","last_unstale":"2024-04-25T10:07:08.431572+0000","last_undegraded":"2024-04-25T10:07:08.431572+0000","last_fullsized":"2024-04-25T10:07:08.431572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:25.307131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431571+0000","last_change":"2024-04-25T10:07:07.337246+0000","last_active":"2024-04-25T10:07:08.431571+0000","last_peered":"2024-04-25T10:07:08.431571+0000","last_clean":"2024-04-25T10:07:08.431571+0000","last_became_active":"2024-04-25T10:07:07.336894+0000","last_became_peered":"2024-04-25T10:07:07.336894+0000","last_unstale":"2024-04-25T10:07:08.431571+0000","last_undegraded":"2024-04-25T10:07:08.431571+0000","last_fullsized":"2024-04-25T10:07:08.431571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013544+0000","last_change":"2024-04-25T10:07:06.324947+0000","last_active":"2024-04-25T10:07:09.013544+0000","last_peered":"2024-04-25T10:07:09.013544+0000","last_clean":"2024-04-25T10:07:09.013544+0000","last_became_active":"2024-04-25T10:07:06.324464+0000","last_became_peered":"2024-04-25T10:07:06.324464+0000","last_unstale":"2024-04-25T10:07:09.013544+0000","last_undegraded":"2024-04-25T10:07:09.013544+0000","last_fullsized":"2024-04-25T10:07:09.013544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:25.307131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:17.363 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:17.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.584+0000 7f35be000640 1 Processor -- start 2024-04-25T10:07:17.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.584+0000 7f35be000640 1 -- start start 2024-04-25T10:07:17.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.586+0000 7f35be000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815a870 0x7f35b80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.586+0000 7f35be000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f35b806e2e0 0x7f35b815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.586+0000 7f35be000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b80cdb10 con 0x7f35b80cd580 2024-04-25T10:07:17.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.586+0000 7f35be000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b80ce720 con 0x7f35b806e2e0 2024-04-25T10:07:17.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.586+0000 7f35be000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b80cddf0 con 0x7f35b815a870 2024-04-25T10:07:17.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.586+0000 7f35afe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f35b806e2e0 0x7f35b815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.586+0000 7f35afe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f35b806e2e0 0x7f35b815ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34918/0 (socket says 172.21.15.119:34918) 2024-04-25T10:07:17.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.586+0000 7f35afe00640 1 -- 172.21.15.119:0/1937523798 learned_addr learned my addr 172.21.15.119:0/1937523798 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:17.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.586+0000 7f35afe00640 1 -- 172.21.15.119:0/1937523798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815a870 msgr2=0x7f35b80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.586+0000 7f35b7400640 1 --2- 172.21.15.119:0/1937523798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815a870 0x7f35b80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.586+0000 7f35afe00640 1 --2- 172.21.15.119:0/1937523798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815a870 0x7f35b80cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.586+0000 7f35afe00640 1 -- 172.21.15.119:0/1937523798 >> v1:172.21.15.132:6789/0 conn(0x7f35b80cd580 legacy=0x7f35b80cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.586+0000 7f35afe00640 1 -- 172.21.15.119:0/1937523798 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b80ce0d0 con 0x7f35b806e2e0 2024-04-25T10:07:17.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.587+0000 7f35afe00640 1 --2- 172.21.15.119:0/1937523798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b806e2e0 0x7f35b815ac50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009870 tx=0x7f35a002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d1a20ae88741ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.587+0000 7f35b6a00640 1 -- 172.21.15.119:0/1937523798 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0037070 con 0x7f35b806e2e0 2024-04-25T10:07:17.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.588+0000 7f35b6a00640 1 -- 172.21.15.119:0/1937523798 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a0032440 con 0x7f35b806e2e0 2024-04-25T10:07:17.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.588+0000 7f35b6a00640 1 -- 172.21.15.119:0/1937523798 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0032810 con 0x7f35b806e2e0 2024-04-25T10:07:17.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.588+0000 7f35be000640 1 -- 172.21.15.119:0/1937523798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b806e2e0 msgr2=0x7f35b815ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.588+0000 7f35be000640 1 --2- 172.21.15.119:0/1937523798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b806e2e0 0x7f35b815ac50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009870 tx=0x7f35a002a210 comp rx=0 tx=0).stop 2024-04-25T10:07:17.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.588+0000 7f35be000640 1 -- 172.21.15.119:0/1937523798 shutdown_connections 2024-04-25T10:07:17.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.588+0000 7f35be000640 1 --2- 172.21.15.119:0/1937523798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b806e2e0 0x7f35b815ac50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.588+0000 7f35be000640 1 --2- 172.21.15.119:0/1937523798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815a870 0x7f35b80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.588+0000 7f35be000640 1 -- 172.21.15.119:0/1937523798 >> 172.21.15.119:0/1937523798 conn(0x7f35b806aba0 msgr2=0x7f35b806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.589+0000 7f35be000640 1 -- 172.21.15.119:0/1937523798 shutdown_connections 2024-04-25T10:07:17.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.589+0000 7f35be000640 1 -- 172.21.15.119:0/1937523798 wait complete. 2024-04-25T10:07:17.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.591+0000 7f35be000640 1 Processor -- start 2024-04-25T10:07:17.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.591+0000 7f35be000640 1 -- start start 2024-04-25T10:07:17.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.592+0000 7f35be000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b806e2e0 0x7f35b814b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.592+0000 7f35be000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35b80cd580 0x7f35b80ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.592+0000 7f35be000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815a870 0x7f35b80c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.592+0000 7f35be000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b80cfdc0 con 0x7f35b806e2e0 2024-04-25T10:07:17.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.592+0000 7f35be000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b80cfc40 con 0x7f35b80cd580 2024-04-25T10:07:17.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.592+0000 7f35be000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b815bf10 con 0x7f35b815a870 2024-04-25T10:07:17.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.592+0000 7f35b7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b806e2e0 0x7f35b814b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.592+0000 7f35b7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b806e2e0 0x7f35b814b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34932/0 (socket says 172.21.15.119:34932) 2024-04-25T10:07:17.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.592+0000 7f35b7400640 1 -- 172.21.15.119:0/2393805482 learned_addr learned my addr 172.21.15.119:0/2393805482 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:17.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.593+0000 7f35b7e00640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815a870 0x7f35b80c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.593+0000 7f35afe00640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35b80cd580 0x7f35b80ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.593+0000 7f35b7400640 1 -- 172.21.15.119:0/2393805482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815a870 msgr2=0x7f35b80c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.593+0000 7f35b7400640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815a870 0x7f35b80c4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.593+0000 7f35b7400640 1 -- 172.21.15.119:0/2393805482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35b80cd580 msgr2=0x7f35b80ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.593+0000 7f35b7400640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35b80cd580 0x7f35b80ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.593+0000 7f35b7400640 1 -- 172.21.15.119:0/2393805482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b80ad020 con 0x7f35b806e2e0 2024-04-25T10:07:17.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.593+0000 7f35b7e00640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815a870 0x7f35b80c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:17.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.593+0000 7f35b7400640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b806e2e0 0x7f35b814b9c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f35a800c7f0 tx=0x7f35a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.593+0000 7f35af400640 1 -- 172.21.15.119:0/2393805482 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8015070 con 0x7f35b806e2e0 2024-04-25T10:07:17.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.593+0000 7f35af400640 1 -- 172.21.15.119:0/2393805482 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a8004220 con 0x7f35b806e2e0 2024-04-25T10:07:17.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.593+0000 7f35af400640 1 -- 172.21.15.119:0/2393805482 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8007620 con 0x7f35b806e2e0 2024-04-25T10:07:17.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.593+0000 7f35be000640 1 -- 172.21.15.119:0/2393805482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b80ad300 con 0x7f35b806e2e0 2024-04-25T10:07:17.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.593+0000 7f35be000640 1 -- 172.21.15.119:0/2393805482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b80ad8c0 con 0x7f35b806e2e0 2024-04-25T10:07:17.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.594+0000 7f35be000640 1 -- 172.21.15.119:0/2393805482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b80749b0 con 0x7f35b806e2e0 2024-04-25T10:07:17.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.595+0000 7f35af400640 1 -- 172.21.15.119:0/2393805482 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35a80079d0 con 0x7f35b806e2e0 2024-04-25T10:07:17.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.596+0000 7f35af400640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f359406b500 0x7f359406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.596+0000 7f35af400640 1 -- 172.21.15.119:0/2393805482 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35a808a200 con 0x7f35b806e2e0 2024-04-25T10:07:17.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.596+0000 7f35afe00640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f359406b500 0x7f359406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.597+0000 7f35afe00640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f359406b500 0x7f359406d9c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f35a0034000 tx=0x7f35a002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.600+0000 7f35af400640 1 -- 172.21.15.119:0/2393805482 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f35a8059070 con 0x7f35b806e2e0 2024-04-25T10:07:17.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.779+0000 7f35be000640 1 -- 172.21.15.119:0/2393805482 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f35b8148b50 con 0x7f359406b500 2024-04-25T10:07:17.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.796+0000 7f35af400640 1 -- 172.21.15.119:0/2393805482 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f35b8148b50 con 0x7f359406b500 2024-04-25T10:07:17.806 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:17.808 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:07:17.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.801+0000 7f35be000640 1 -- 172.21.15.119:0/2393805482 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f359406b500 msgr2=0x7f359406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.801+0000 7f35be000640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f359406b500 0x7f359406d9c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f35a0034000 tx=0x7f35a002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:17.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.801+0000 7f35be000640 1 -- 172.21.15.119:0/2393805482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b806e2e0 msgr2=0x7f35b814b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.801+0000 7f35be000640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b806e2e0 0x7f35b814b9c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f35a800c7f0 tx=0x7f35a800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.801+0000 7f35be000640 1 -- 172.21.15.119:0/2393805482 shutdown_connections 2024-04-25T10:07:17.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.801+0000 7f35be000640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f359406b500 0x7f359406d9c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.801+0000 7f35be000640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f35b815a870 0x7f35b80c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.801+0000 7f35be000640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35b80cd580 0x7f35b80ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.801+0000 7f35be000640 1 --2- 172.21.15.119:0/2393805482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b806e2e0 0x7f35b814b9c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.801+0000 7f35be000640 1 -- 172.21.15.119:0/2393805482 >> 172.21.15.119:0/2393805482 conn(0x7f35b806aba0 msgr2=0x7f35b80cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.801+0000 7f35be000640 1 -- 172.21.15.119:0/2393805482 shutdown_connections 2024-04-25T10:07:17.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:17.801+0000 7f35be000640 1 -- 172.21.15.119:0/2393805482 wait complete. 2024-04-25T10:07:17.825 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-25T10:07:15.925479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218988,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755732,"statfs":{"total":773094113280,"available":772869869568,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.621466"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117497+0000","last_change":"2024-04-25T10:07:07.357761+0000","last_active":"2024-04-25T10:07:10.117497+0000","last_peered":"2024-04-25T10:07:10.117497+0000","last_clean":"2024-04-25T10:07:10.117497+0000","last_became_active":"2024-04-25T10:07:07.357419+0000","last_became_peered":"2024-04-25T10:07:07.357419+0000","last_unstale":"2024-04-25T10:07:10.117497+0000","last_undegraded":"2024-04-25T10:07:10.117497+0000","last_fullsized":"2024-04-25T10:07:10.117497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:31.725733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340438+0000","last_change":"2024-04-25T10:07:07.324906+0000","last_active":"2024-04-25T10:07:08.340438+0000","last_peered":"2024-04-25T10:07:08.340438+0000","last_clean":"2024-04-25T10:07:08.340438+0000","last_became_active":"2024-04-25T10:07:07.324635+0000","last_became_peered":"2024-04-25T10:07:07.324635+0000","last_unstale":"2024-04-25T10:07:08.340438+0000","last_undegraded":"2024-04-25T10:07:08.340438+0000","last_fullsized":"2024-04-25T10:07:08.340438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013703+0000","last_change":"2024-04-25T10:07:07.324537+0000","last_active":"2024-04-25T10:07:09.013703+0000","last_peered":"2024-04-25T10:07:09.013703+0000","last_clean":"2024-04-25T10:07:09.013703+0000","last_became_active":"2024-04-25T10:07:07.324290+0000","last_became_peered":"2024-04-25T10:07:07.324290+0000","last_unstale":"2024-04-25T10:07:09.013703+0000","last_undegraded":"2024-04-25T10:07:09.013703+0000","last_fullsized":"2024-04-25T10:07:09.013703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340924+0000","last_change":"2024-04-25T10:07:07.336252+0000","last_active":"2024-04-25T10:07:08.340924+0000","last_peered":"2024-04-25T10:07:08.340924+0000","last_clean":"2024-04-25T10:07:08.340924+0000","last_became_active":"2024-04-25T10:07:07.335889+0000","last_became_peered":"2024-04-25T10:07:07.335889+0000","last_unstale":"2024-04-25T10:07:08.340924+0000","last_undegraded":"2024-04-25T10:07:08.340924+0000","last_fullsized":"2024-04-25T10:07:08.340924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116666+0000","last_change":"2024-04-25T10:07:07.354080+0000","last_active":"2024-04-25T10:07:10.116666+0000","last_peered":"2024-04-25T10:07:10.116666+0000","last_clean":"2024-04-25T10:07:10.116666+0000","last_became_active":"2024-04-25T10:07:07.351587+0000","last_became_peered":"2024-04-25T10:07:07.351587+0000","last_unstale":"2024-04-25T10:07:10.116666+0000","last_undegraded":"2024-04-25T10:07:10.116666+0000","last_fullsized":"2024-04-25T10:07:10.116666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340152+0000","last_change":"2024-04-25T10:07:07.329505+0000","last_active":"2024-04-25T10:07:08.340152+0000","last_peered":"2024-04-25T10:07:08.340152+0000","last_clean":"2024-04-25T10:07:08.340152+0000","last_became_active":"2024-04-25T10:07:07.327756+0000","last_became_peered":"2024-04-25T10:07:07.327756+0000","last_unstale":"2024-04-25T10:07:08.340152+0000","last_undegraded":"2024-04-25T10:07:08.340152+0000","last_fullsized":"2024-04-25T10:07:08.340152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116453+0000","last_change":"2024-04-25T10:07:07.354368+0000","last_active":"2024-04-25T10:07:10.116453+0000","last_peered":"2024-04-25T10:07:10.116453+0000","last_clean":"2024-04-25T10:07:10.116453+0000","last_became_active":"2024-04-25T10:07:07.354048+0000","last_became_peered":"2024-04-25T10:07:07.354048+0000","last_unstale":"2024-04-25T10:07:10.116453+0000","last_undegraded":"2024-04-25T10:07:10.116453+0000","last_fullsized":"2024-04-25T10:07:10.116453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430715+0000","last_change":"2024-04-25T10:07:07.337913+0000","last_active":"2024-04-25T10:07:08.430715+0000","last_peered":"2024-04-25T10:07:08.430715+0000","last_clean":"2024-04-25T10:07:08.430715+0000","last_became_active":"2024-04-25T10:07:07.337268+0000","last_became_peered":"2024-04-25T10:07:07.337268+0000","last_unstale":"2024-04-25T10:07:08.430715+0000","last_undegraded":"2024-04-25T10:07:08.430715+0000","last_fullsized":"2024-04-25T10:07:08.430715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:01.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117721+0000","last_change":"2024-04-25T10:07:07.352856+0000","last_active":"2024-04-25T10:07:10.117721+0000","last_peered":"2024-04-25T10:07:10.117721+0000","last_clean":"2024-04-25T10:07:10.117721+0000","last_became_active":"2024-04-25T10:07:07.352555+0000","last_became_peered":"2024-04-25T10:07:07.352555+0000","last_unstale":"2024-04-25T10:07:10.117721+0000","last_undegraded":"2024-04-25T10:07:10.117721+0000","last_fullsized":"2024-04-25T10:07:10.117721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.763141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627954+0000","last_change":"2024-04-25T10:07:07.335997+0000","last_active":"2024-04-25T10:07:09.627954+0000","last_peered":"2024-04-25T10:07:09.627954+0000","last_clean":"2024-04-25T10:07:09.627954+0000","last_became_active":"2024-04-25T10:07:07.334634+0000","last_became_peered":"2024-04-25T10:07:07.334634+0000","last_unstale":"2024-04-25T10:07:09.627954+0000","last_undegraded":"2024-04-25T10:07:09.627954+0000","last_fullsized":"2024-04-25T10:07:09.627954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627400+0000","last_change":"2024-04-25T10:07:07.336470+0000","last_active":"2024-04-25T10:07:09.627400+0000","last_peered":"2024-04-25T10:07:09.627400+0000","last_clean":"2024-04-25T10:07:09.627400+0000","last_became_active":"2024-04-25T10:07:07.334875+0000","last_became_peered":"2024-04-25T10:07:07.334875+0000","last_unstale":"2024-04-25T10:07:09.627400+0000","last_undegraded":"2024-04-25T10:07:09.627400+0000","last_fullsized":"2024-04-25T10:07:09.627400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115690+0000","last_change":"2024-04-25T10:07:07.342067+0000","last_active":"2024-04-25T10:07:10.115690+0000","last_peered":"2024-04-25T10:07:10.115690+0000","last_clean":"2024-04-25T10:07:10.115690+0000","last_became_active":"2024-04-25T10:07:07.341655+0000","last_became_peered":"2024-04-25T10:07:07.341655+0000","last_unstale":"2024-04-25T10:07:10.115690+0000","last_undegraded":"2024-04-25T10:07:10.115690+0000","last_fullsized":"2024-04-25T10:07:10.115690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:37.664841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431262+0000","last_change":"2024-04-25T10:07:07.337652+0000","last_active":"2024-04-25T10:07:08.431262+0000","last_peered":"2024-04-25T10:07:08.431262+0000","last_clean":"2024-04-25T10:07:08.431262+0000","last_became_active":"2024-04-25T10:07:07.337057+0000","last_became_peered":"2024-04-25T10:07:07.337057+0000","last_unstale":"2024-04-25T10:07:08.431262+0000","last_undegraded":"2024-04-25T10:07:08.431262+0000","last_fullsized":"2024-04-25T10:07:08.431262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.012844+0000","last_change":"2024-04-25T10:07:07.341549+0000","last_active":"2024-04-25T10:07:09.012844+0000","last_peered":"2024-04-25T10:07:09.012844+0000","last_clean":"2024-04-25T10:07:09.012844+0000","last_became_active":"2024-04-25T10:07:07.341228+0000","last_became_peered":"2024-04-25T10:07:07.341228+0000","last_unstale":"2024-04-25T10:07:09.012844+0000","last_undegraded":"2024-04-25T10:07:09.012844+0000","last_fullsized":"2024-04-25T10:07:09.012844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:50.697002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339480+0000","last_change":"2024-04-25T10:07:07.348054+0000","last_active":"2024-04-25T10:07:08.339480+0000","last_peered":"2024-04-25T10:07:08.339480+0000","last_clean":"2024-04-25T10:07:08.339480+0000","last_became_active":"2024-04-25T10:07:07.347739+0000","last_became_peered":"2024-04-25T10:07:07.347739+0000","last_unstale":"2024-04-25T10:07:08.339480+0000","last_undegraded":"2024-04-25T10:07:08.339480+0000","last_fullsized":"2024-04-25T10:07:08.339480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339967+0000","last_change":"2024-04-25T10:07:07.326858+0000","last_active":"2024-04-25T10:07:08.339967+0000","last_peered":"2024-04-25T10:07:08.339967+0000","last_clean":"2024-04-25T10:07:08.339967+0000","last_became_active":"2024-04-25T10:07:07.326107+0000","last_became_peered":"2024-04-25T10:07:07.326107+0000","last_unstale":"2024-04-25T10:07:08.339967+0000","last_undegraded":"2024-04-25T10:07:08.339967+0000","last_fullsized":"2024-04-25T10:07:08.339967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.342255+0000","last_change":"2024-04-25T10:07:07.348628+0000","last_active":"2024-04-25T10:07:08.342255+0000","last_peered":"2024-04-25T10:07:08.342255+0000","last_clean":"2024-04-25T10:07:08.342255+0000","last_became_active":"2024-04-25T10:07:07.348356+0000","last_became_peered":"2024-04-25T10:07:07.348356+0000","last_unstale":"2024-04-25T10:07:08.342255+0000","last_undegraded":"2024-04-25T10:07:08.342255+0000","last_fullsized":"2024-04-25T10:07:08.342255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:16.375835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.338486+0000","last_change":"2024-04-25T10:07:07.341185+0000","last_active":"2024-04-25T10:07:08.338486+0000","last_peered":"2024-04-25T10:07:08.338486+0000","last_clean":"2024-04-25T10:07:08.338486+0000","last_became_active":"2024-04-25T10:07:07.340797+0000","last_became_peered":"2024-04-25T10:07:07.340797+0000","last_unstale":"2024-04-25T10:07:08.338486+0000","last_undegraded":"2024-04-25T10:07:08.338486+0000","last_fullsized":"2024-04-25T10:07:08.338486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:49.701271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.118018+0000","last_change":"2024-04-25T10:07:07.356663+0000","last_active":"2024-04-25T10:07:10.118018+0000","last_peered":"2024-04-25T10:07:10.118018+0000","last_clean":"2024-04-25T10:07:10.118018+0000","last_became_active":"2024-04-25T10:07:07.356381+0000","last_became_peered":"2024-04-25T10:07:07.356381+0000","last_unstale":"2024-04-25T10:07:10.118018+0000","last_undegraded":"2024-04-25T10:07:10.118018+0000","last_fullsized":"2024-04-25T10:07:10.118018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:51.771853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627639+0000","last_change":"2024-04-25T10:07:07.348329+0000","last_active":"2024-04-25T10:07:09.627639+0000","last_peered":"2024-04-25T10:07:09.627639+0000","last_clean":"2024-04-25T10:07:09.627639+0000","last_became_active":"2024-04-25T10:07:07.347033+0000","last_became_peered":"2024-04-25T10:07:07.347033+0000","last_unstale":"2024-04-25T10:07:09.627639+0000","last_undegraded":"2024-04-25T10:07:09.627639+0000","last_fullsized":"2024-04-25T10:07:09.627639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013394+0000","last_change":"2024-04-25T10:07:07.350923+0000","last_active":"2024-04-25T10:07:09.013394+0000","last_peered":"2024-04-25T10:07:09.013394+0000","last_clean":"2024-04-25T10:07:09.013394+0000","last_became_active":"2024-04-25T10:07:07.350102+0000","last_became_peered":"2024-04-25T10:07:07.350102+0000","last_unstale":"2024-04-25T10:07:09.013394+0000","last_undegraded":"2024-04-25T10:07:09.013394+0000","last_fullsized":"2024-04-25T10:07:09.013394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340239+0000","last_change":"2024-04-25T10:07:07.351751+0000","last_active":"2024-04-25T10:07:08.340239+0000","last_peered":"2024-04-25T10:07:08.340239+0000","last_clean":"2024-04-25T10:07:08.340239+0000","last_became_active":"2024-04-25T10:07:07.349688+0000","last_became_peered":"2024-04-25T10:07:07.349688+0000","last_unstale":"2024-04-25T10:07:08.340239+0000","last_undegraded":"2024-04-25T10:07:08.340239+0000","last_fullsized":"2024-04-25T10:07:08.340239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430799+0000","last_change":"2024-04-25T10:07:07.346636+0000","last_active":"2024-04-25T10:07:08.430799+0000","last_peered":"2024-04-25T10:07:08.430799+0000","last_clean":"2024-04-25T10:07:08.430799+0000","last_became_active":"2024-04-25T10:07:07.346448+0000","last_became_peered":"2024-04-25T10:07:07.346448+0000","last_unstale":"2024-04-25T10:07:08.430799+0000","last_undegraded":"2024-04-25T10:07:08.430799+0000","last_fullsized":"2024-04-25T10:07:08.430799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.370945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339964+0000","last_change":"2024-04-25T10:07:07.327120+0000","last_active":"2024-04-25T10:07:08.339964+0000","last_peered":"2024-04-25T10:07:08.339964+0000","last_clean":"2024-04-25T10:07:08.339964+0000","last_became_active":"2024-04-25T10:07:07.326471+0000","last_became_peered":"2024-04-25T10:07:07.326471+0000","last_unstale":"2024-04-25T10:07:08.339964+0000","last_undegraded":"2024-04-25T10:07:08.339964+0000","last_fullsized":"2024-04-25T10:07:08.339964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431521+0000","last_change":"2024-04-25T10:07:07.346515+0000","last_active":"2024-04-25T10:07:08.431521+0000","last_peered":"2024-04-25T10:07:08.431521+0000","last_clean":"2024-04-25T10:07:08.431521+0000","last_became_active":"2024-04-25T10:07:07.346218+0000","last_became_peered":"2024-04-25T10:07:07.346218+0000","last_unstale":"2024-04-25T10:07:08.431521+0000","last_undegraded":"2024-04-25T10:07:08.431521+0000","last_fullsized":"2024-04-25T10:07:08.431521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:49.296728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627914+0000","last_change":"2024-04-25T10:07:07.352410+0000","last_active":"2024-04-25T10:07:09.627914+0000","last_peered":"2024-04-25T10:07:09.627914+0000","last_clean":"2024-04-25T10:07:09.627914+0000","last_became_active":"2024-04-25T10:07:07.352106+0000","last_became_peered":"2024-04-25T10:07:07.352106+0000","last_unstale":"2024-04-25T10:07:09.627914+0000","last_undegraded":"2024-04-25T10:07:09.627914+0000","last_fullsized":"2024-04-25T10:07:09.627914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339814+0000","last_change":"2024-04-25T10:07:07.339497+0000","last_active":"2024-04-25T10:07:08.339814+0000","last_peered":"2024-04-25T10:07:08.339814+0000","last_clean":"2024-04-25T10:07:08.339814+0000","last_became_active":"2024-04-25T10:07:07.339168+0000","last_became_peered":"2024-04-25T10:07:07.339168+0000","last_unstale":"2024-04-25T10:07:08.339814+0000","last_undegraded":"2024-04-25T10:07:08.339814+0000","last_fullsized":"2024-04-25T10:07:08.339814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431449+0000","last_change":"2024-04-25T10:07:07.349878+0000","last_active":"2024-04-25T10:07:08.431449+0000","last_peered":"2024-04-25T10:07:08.431449+0000","last_clean":"2024-04-25T10:07:08.431449+0000","last_became_active":"2024-04-25T10:07:07.349602+0000","last_became_peered":"2024-04-25T10:07:07.349602+0000","last_unstale":"2024-04-25T10:07:08.431449+0000","last_undegraded":"2024-04-25T10:07:08.431449+0000","last_fullsized":"2024-04-25T10:07:08.431449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:41.064369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013693+0000","last_change":"2024-04-25T10:07:07.331427+0000","last_active":"2024-04-25T10:07:09.013693+0000","last_peered":"2024-04-25T10:07:09.013693+0000","last_clean":"2024-04-25T10:07:09.013693+0000","last_became_active":"2024-04-25T10:07:07.329911+0000","last_became_peered":"2024-04-25T10:07:07.329911+0000","last_unstale":"2024-04-25T10:07:09.013693+0000","last_undegraded":"2024-04-25T10:07:09.013693+0000","last_fullsized":"2024-04-25T10:07:09.013693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340563+0000","last_change":"2024-04-25T10:07:07.329226+0000","last_active":"2024-04-25T10:07:08.340563+0000","last_peered":"2024-04-25T10:07:08.340563+0000","last_clean":"2024-04-25T10:07:08.340563+0000","last_became_active":"2024-04-25T10:07:07.327329+0000","last_became_peered":"2024-04-25T10:07:07.327329+0000","last_unstale":"2024-04-25T10:07:08.340563+0000","last_undegraded":"2024-04-25T10:07:08.340563+0000","last_fullsized":"2024-04-25T10:07:08.340563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:49.296728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337552+0000","last_change":"2024-04-25T10:07:07.327804+0000","last_active":"2024-04-25T10:07:08.337552+0000","last_peered":"2024-04-25T10:07:08.337552+0000","last_clean":"2024-04-25T10:07:08.337552+0000","last_became_active":"2024-04-25T10:07:07.327516+0000","last_became_peered":"2024-04-25T10:07:07.327516+0000","last_unstale":"2024-04-25T10:07:08.337552+0000","last_undegraded":"2024-04-25T10:07:08.337552+0000","last_fullsized":"2024-04-25T10:07:08.337552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:46.704814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627063+0000","last_change":"2024-04-25T10:07:07.333594+0000","last_active":"2024-04-25T10:07:09.627063+0000","last_peered":"2024-04-25T10:07:09.627063+0000","last_clean":"2024-04-25T10:07:09.627063+0000","last_became_active":"2024-04-25T10:07:07.333302+0000","last_became_peered":"2024-04-25T10:07:07.333302+0000","last_unstale":"2024-04-25T10:07:09.627063+0000","last_undegraded":"2024-04-25T10:07:09.627063+0000","last_fullsized":"2024-04-25T10:07:09.627063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:01.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.114966+0000","last_change":"2024-04-25T10:07:07.333101+0000","last_active":"2024-04-25T10:07:10.114966+0000","last_peered":"2024-04-25T10:07:10.114966+0000","last_clean":"2024-04-25T10:07:10.114966+0000","last_became_active":"2024-04-25T10:07:07.332714+0000","last_became_peered":"2024-04-25T10:07:07.332714+0000","last_unstale":"2024-04-25T10:07:10.114966+0000","last_undegraded":"2024-04-25T10:07:10.114966+0000","last_fullsized":"2024-04-25T10:07:10.114966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:53.471918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431440+0000","last_change":"2024-04-25T10:07:06.322470+0000","last_active":"2024-04-25T10:07:08.431440+0000","last_peered":"2024-04-25T10:07:08.431440+0000","last_clean":"2024-04-25T10:07:08.431440+0000","last_became_active":"2024-04-25T10:07:06.321092+0000","last_became_peered":"2024-04-25T10:07:06.321092+0000","last_unstale":"2024-04-25T10:07:08.431440+0000","last_undegraded":"2024-04-25T10:07:08.431440+0000","last_fullsized":"2024-04-25T10:07:08.431440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:28.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013819+0000","last_change":"2024-04-25T10:07:07.340326+0000","last_active":"2024-04-25T10:07:09.013819+0000","last_peered":"2024-04-25T10:07:09.013819+0000","last_clean":"2024-04-25T10:07:09.013819+0000","last_became_active":"2024-04-25T10:07:07.339889+0000","last_became_peered":"2024-04-25T10:07:07.339889+0000","last_unstale":"2024-04-25T10:07:09.013819+0000","last_undegraded":"2024-04-25T10:07:09.013819+0000","last_fullsized":"2024-04-25T10:07:09.013819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.626769+0000","last_change":"2024-04-25T10:07:06.333386+0000","last_active":"2024-04-25T10:07:09.626769+0000","last_peered":"2024-04-25T10:07:09.626769+0000","last_clean":"2024-04-25T10:07:09.626769+0000","last_became_active":"2024-04-25T10:07:06.333014+0000","last_became_peered":"2024-04-25T10:07:06.333014+0000","last_unstale":"2024-04-25T10:07:09.626769+0000","last_undegraded":"2024-04-25T10:07:09.626769+0000","last_fullsized":"2024-04-25T10:07:09.626769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:53.885696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430668+0000","last_change":"2024-04-25T10:07:06.314017+0000","last_active":"2024-04-25T10:07:08.430668+0000","last_peered":"2024-04-25T10:07:08.430668+0000","last_clean":"2024-04-25T10:07:08.430668+0000","last_became_active":"2024-04-25T10:07:06.313712+0000","last_became_peered":"2024-04-25T10:07:06.313712+0000","last_unstale":"2024-04-25T10:07:08.430668+0000","last_undegraded":"2024-04-25T10:07:08.430668+0000","last_fullsized":"2024-04-25T10:07:08.430668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:05.470696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430845+0000","last_change":"2024-04-25T10:07:07.323982+0000","last_active":"2024-04-25T10:07:08.430845+0000","last_peered":"2024-04-25T10:07:08.430845+0000","last_clean":"2024-04-25T10:07:08.430845+0000","last_became_active":"2024-04-25T10:07:07.323664+0000","last_became_peered":"2024-04-25T10:07:07.323664+0000","last_unstale":"2024-04-25T10:07:08.430845+0000","last_undegraded":"2024-04-25T10:07:08.430845+0000","last_fullsized":"2024-04-25T10:07:08.430845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:29.610283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340135+0000","last_change":"2024-04-25T10:07:06.312709+0000","last_active":"2024-04-25T10:07:08.340135+0000","last_peered":"2024-04-25T10:07:08.340135+0000","last_clean":"2024-04-25T10:07:08.340135+0000","last_became_active":"2024-04-25T10:07:06.311785+0000","last_became_peered":"2024-04-25T10:07:06.311785+0000","last_unstale":"2024-04-25T10:07:08.340135+0000","last_undegraded":"2024-04-25T10:07:08.340135+0000","last_fullsized":"2024-04-25T10:07:08.340135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:52.458435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339243+0000","last_change":"2024-04-25T10:07:07.332051+0000","last_active":"2024-04-25T10:07:08.339243+0000","last_peered":"2024-04-25T10:07:08.339243+0000","last_clean":"2024-04-25T10:07:08.339243+0000","last_became_active":"2024-04-25T10:07:07.331656+0000","last_became_peered":"2024-04-25T10:07:07.331656+0000","last_unstale":"2024-04-25T10:07:08.339243+0000","last_undegraded":"2024-04-25T10:07:08.339243+0000","last_fullsized":"2024-04-25T10:07:08.339243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627326+0000","last_change":"2024-04-25T10:07:06.316859+0000","last_active":"2024-04-25T10:07:09.627326+0000","last_peered":"2024-04-25T10:07:09.627326+0000","last_clean":"2024-04-25T10:07:09.627326+0000","last_became_active":"2024-04-25T10:07:06.314454+0000","last_became_peered":"2024-04-25T10:07:06.314454+0000","last_unstale":"2024-04-25T10:07:09.627326+0000","last_undegraded":"2024-04-25T10:07:09.627326+0000","last_fullsized":"2024-04-25T10:07:09.627326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.867432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341342+0000","last_change":"2024-04-25T10:07:07.352036+0000","last_active":"2024-04-25T10:07:08.341342+0000","last_peered":"2024-04-25T10:07:08.341342+0000","last_clean":"2024-04-25T10:07:08.341342+0000","last_became_active":"2024-04-25T10:07:07.351244+0000","last_became_peered":"2024-04-25T10:07:07.351244+0000","last_unstale":"2024-04-25T10:07:08.341342+0000","last_undegraded":"2024-04-25T10:07:08.341342+0000","last_fullsized":"2024-04-25T10:07:08.341342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:31.725733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341722+0000","last_change":"2024-04-25T10:07:06.313325+0000","last_active":"2024-04-25T10:07:08.341722+0000","last_peered":"2024-04-25T10:07:08.341722+0000","last_clean":"2024-04-25T10:07:08.341722+0000","last_became_active":"2024-04-25T10:07:06.312922+0000","last_became_peered":"2024-04-25T10:07:06.312922+0000","last_unstale":"2024-04-25T10:07:08.341722+0000","last_undegraded":"2024-04-25T10:07:08.341722+0000","last_fullsized":"2024-04-25T10:07:08.341722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013388+0000","last_change":"2024-04-25T10:07:07.334790+0000","last_active":"2024-04-25T10:07:09.013388+0000","last_peered":"2024-04-25T10:07:09.013388+0000","last_clean":"2024-04-25T10:07:09.013388+0000","last_became_active":"2024-04-25T10:07:07.334587+0000","last_became_peered":"2024-04-25T10:07:07.334587+0000","last_unstale":"2024-04-25T10:07:09.013388+0000","last_undegraded":"2024-04-25T10:07:09.013388+0000","last_fullsized":"2024-04-25T10:07:09.013388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340029+0000","last_change":"2024-04-25T10:07:06.317802+0000","last_active":"2024-04-25T10:07:08.340029+0000","last_peered":"2024-04-25T10:07:08.340029+0000","last_clean":"2024-04-25T10:07:08.340029+0000","last_became_active":"2024-04-25T10:07:06.317291+0000","last_became_peered":"2024-04-25T10:07:06.317291+0000","last_unstale":"2024-04-25T10:07:08.340029+0000","last_undegraded":"2024-04-25T10:07:08.340029+0000","last_fullsized":"2024-04-25T10:07:08.340029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.867432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115871+0000","last_change":"2024-04-25T10:07:07.351461+0000","last_active":"2024-04-25T10:07:10.115871+0000","last_peered":"2024-04-25T10:07:10.115871+0000","last_clean":"2024-04-25T10:07:10.115871+0000","last_became_active":"2024-04-25T10:07:07.351141+0000","last_became_peered":"2024-04-25T10:07:07.351141+0000","last_unstale":"2024-04-25T10:07:10.115871+0000","last_undegraded":"2024-04-25T10:07:10.115871+0000","last_fullsized":"2024-04-25T10:07:10.115871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:46.704814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627332+0000","last_change":"2024-04-25T10:07:06.319663+0000","last_active":"2024-04-25T10:07:09.627332+0000","last_peered":"2024-04-25T10:07:09.627332+0000","last_clean":"2024-04-25T10:07:09.627332+0000","last_became_active":"2024-04-25T10:07:06.317321+0000","last_became_peered":"2024-04-25T10:07:06.317321+0000","last_unstale":"2024-04-25T10:07:09.627332+0000","last_undegraded":"2024-04-25T10:07:09.627332+0000","last_fullsized":"2024-04-25T10:07:09.627332+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117185+0000","last_change":"2024-04-25T10:07:07.350498+0000","last_active":"2024-04-25T10:07:10.117185+0000","last_peered":"2024-04-25T10:07:10.117185+0000","last_clean":"2024-04-25T10:07:10.117185+0000","last_became_active":"2024-04-25T10:07:07.349929+0000","last_became_peered":"2024-04-25T10:07:07.349929+0000","last_unstale":"2024-04-25T10:07:10.117185+0000","last_undegraded":"2024-04-25T10:07:10.117185+0000","last_fullsized":"2024-04-25T10:07:10.117185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:23.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013849+0000","last_change":"2024-04-25T10:07:06.312357+0000","last_active":"2024-04-25T10:07:09.013849+0000","last_peered":"2024-04-25T10:07:09.013849+0000","last_clean":"2024-04-25T10:07:09.013849+0000","last_became_active":"2024-04-25T10:07:06.312088+0000","last_became_peered":"2024-04-25T10:07:06.312088+0000","last_unstale":"2024-04-25T10:07:09.013849+0000","last_undegraded":"2024-04-25T10:07:09.013849+0000","last_fullsized":"2024-04-25T10:07:09.013849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:05.470696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013146+0000","last_change":"2024-04-25T10:07:07.345439+0000","last_active":"2024-04-25T10:07:09.013146+0000","last_peered":"2024-04-25T10:07:09.013146+0000","last_clean":"2024-04-25T10:07:09.013146+0000","last_became_active":"2024-04-25T10:07:07.345205+0000","last_became_peered":"2024-04-25T10:07:07.345205+0000","last_unstale":"2024-04-25T10:07:09.013146+0000","last_undegraded":"2024-04-25T10:07:09.013146+0000","last_fullsized":"2024-04-25T10:07:09.013146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:56.522956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340351+0000","last_change":"2024-04-25T10:07:06.313807+0000","last_active":"2024-04-25T10:07:08.340351+0000","last_peered":"2024-04-25T10:07:08.340351+0000","last_clean":"2024-04-25T10:07:08.340351+0000","last_became_active":"2024-04-25T10:07:06.313376+0000","last_became_peered":"2024-04-25T10:07:06.313376+0000","last_unstale":"2024-04-25T10:07:08.340351+0000","last_undegraded":"2024-04-25T10:07:08.340351+0000","last_fullsized":"2024-04-25T10:07:08.340351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:44.436918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116968+0000","last_change":"2024-04-25T10:07:07.351251+0000","last_active":"2024-04-25T10:07:10.116968+0000","last_peered":"2024-04-25T10:07:10.116968+0000","last_clean":"2024-04-25T10:07:10.116968+0000","last_became_active":"2024-04-25T10:07:07.350404+0000","last_became_peered":"2024-04-25T10:07:07.350404+0000","last_unstale":"2024-04-25T10:07:10.116968+0000","last_undegraded":"2024-04-25T10:07:10.116968+0000","last_fullsized":"2024-04-25T10:07:10.116968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:49.296728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.626713+0000","last_change":"2024-04-25T10:07:06.322457+0000","last_active":"2024-04-25T10:07:09.626713+0000","last_peered":"2024-04-25T10:07:09.626713+0000","last_clean":"2024-04-25T10:07:09.626713+0000","last_became_active":"2024-04-25T10:07:06.318116+0000","last_became_peered":"2024-04-25T10:07:06.318116+0000","last_unstale":"2024-04-25T10:07:09.626713+0000","last_undegraded":"2024-04-25T10:07:09.626713+0000","last_fullsized":"2024-04-25T10:07:09.626713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:44.436918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627151+0000","last_change":"2024-04-25T10:07:07.328355+0000","last_active":"2024-04-25T10:07:09.627151+0000","last_peered":"2024-04-25T10:07:09.627151+0000","last_clean":"2024-04-25T10:07:09.627151+0000","last_became_active":"2024-04-25T10:07:07.328088+0000","last_became_peered":"2024-04-25T10:07:07.328088+0000","last_unstale":"2024-04-25T10:07:09.627151+0000","last_undegraded":"2024-04-25T10:07:09.627151+0000","last_fullsized":"2024-04-25T10:07:09.627151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337372+0000","last_change":"2024-04-25T10:07:06.306731+0000","last_active":"2024-04-25T10:07:08.337372+0000","last_peered":"2024-04-25T10:07:08.337372+0000","last_clean":"2024-04-25T10:07:08.337372+0000","last_became_active":"2024-04-25T10:07:06.306314+0000","last_became_peered":"2024-04-25T10:07:06.306314+0000","last_unstale":"2024-04-25T10:07:08.337372+0000","last_undegraded":"2024-04-25T10:07:08.337372+0000","last_fullsized":"2024-04-25T10:07:08.337372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:28.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117145+0000","last_change":"2024-04-25T10:07:07.356371+0000","last_active":"2024-04-25T10:07:10.117145+0000","last_peered":"2024-04-25T10:07:10.117145+0000","last_clean":"2024-04-25T10:07:10.117145+0000","last_became_active":"2024-04-25T10:07:07.352731+0000","last_became_peered":"2024-04-25T10:07:07.352731+0000","last_unstale":"2024-04-25T10:07:10.117145+0000","last_undegraded":"2024-04-25T10:07:10.117145+0000","last_fullsized":"2024-04-25T10:07:10.117145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115672+0000","last_change":"2024-04-25T10:07:06.320436+0000","last_active":"2024-04-25T10:07:10.115672+0000","last_peered":"2024-04-25T10:07:10.115672+0000","last_clean":"2024-04-25T10:07:10.115672+0000","last_became_active":"2024-04-25T10:07:06.319958+0000","last_became_peered":"2024-04-25T10:07:06.319958+0000","last_unstale":"2024-04-25T10:07:10.115672+0000","last_undegraded":"2024-04-25T10:07:10.115672+0000","last_fullsized":"2024-04-25T10:07:10.115672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:02.968109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117278+0000","last_change":"2024-04-25T10:07:07.352988+0000","last_active":"2024-04-25T10:07:10.117278+0000","last_peered":"2024-04-25T10:07:10.117278+0000","last_clean":"2024-04-25T10:07:10.117278+0000","last_became_active":"2024-04-25T10:07:07.352585+0000","last_became_peered":"2024-04-25T10:07:07.352585+0000","last_unstale":"2024-04-25T10:07:10.117278+0000","last_undegraded":"2024-04-25T10:07:10.117278+0000","last_fullsized":"2024-04-25T10:07:10.117278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:35.056084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.338565+0000","last_change":"2024-04-25T10:07:06.308389+0000","last_active":"2024-04-25T10:07:08.338565+0000","last_peered":"2024-04-25T10:07:08.338565+0000","last_clean":"2024-04-25T10:07:08.338565+0000","last_became_active":"2024-04-25T10:07:06.308046+0000","last_became_peered":"2024-04-25T10:07:06.308046+0000","last_unstale":"2024-04-25T10:07:08.338565+0000","last_undegraded":"2024-04-25T10:07:08.338565+0000","last_fullsized":"2024-04-25T10:07:08.338565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:53.885696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.118014+0000","last_change":"2024-04-25T10:07:07.357349+0000","last_active":"2024-04-25T10:07:10.118014+0000","last_peered":"2024-04-25T10:07:10.118014+0000","last_clean":"2024-04-25T10:07:10.118014+0000","last_became_active":"2024-04-25T10:07:07.356970+0000","last_became_peered":"2024-04-25T10:07:07.356970+0000","last_unstale":"2024-04-25T10:07:10.118014+0000","last_undegraded":"2024-04-25T10:07:10.118014+0000","last_fullsized":"2024-04-25T10:07:10.118014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:16.375835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340105+0000","last_change":"2024-04-25T10:07:06.310585+0000","last_active":"2024-04-25T10:07:08.340105+0000","last_peered":"2024-04-25T10:07:08.340105+0000","last_clean":"2024-04-25T10:07:08.340105+0000","last_became_active":"2024-04-25T10:07:06.310265+0000","last_became_peered":"2024-04-25T10:07:06.310265+0000","last_unstale":"2024-04-25T10:07:08.340105+0000","last_undegraded":"2024-04-25T10:07:08.340105+0000","last_fullsized":"2024-04-25T10:07:08.340105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:48.687788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341312+0000","last_change":"2024-04-25T10:07:07.337180+0000","last_active":"2024-04-25T10:07:08.341312+0000","last_peered":"2024-04-25T10:07:08.341312+0000","last_clean":"2024-04-25T10:07:08.341312+0000","last_became_active":"2024-04-25T10:07:07.335230+0000","last_became_peered":"2024-04-25T10:07:07.335230+0000","last_unstale":"2024-04-25T10:07:08.341312+0000","last_undegraded":"2024-04-25T10:07:08.341312+0000","last_fullsized":"2024-04-25T10:07:08.341312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:16.166171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337856+0000","last_change":"2024-04-25T10:07:06.311469+0000","last_active":"2024-04-25T10:07:08.337856+0000","last_peered":"2024-04-25T10:07:08.337856+0000","last_clean":"2024-04-25T10:07:08.337856+0000","last_became_active":"2024-04-25T10:07:06.311182+0000","last_became_peered":"2024-04-25T10:07:06.311182+0000","last_unstale":"2024-04-25T10:07:08.337856+0000","last_undegraded":"2024-04-25T10:07:08.337856+0000","last_fullsized":"2024-04-25T10:07:08.337856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:44.436918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117507+0000","last_change":"2024-04-25T10:07:07.350766+0000","last_active":"2024-04-25T10:07:10.117507+0000","last_peered":"2024-04-25T10:07:10.117507+0000","last_clean":"2024-04-25T10:07:10.117507+0000","last_became_active":"2024-04-25T10:07:07.350181+0000","last_became_peered":"2024-04-25T10:07:07.350181+0000","last_unstale":"2024-04-25T10:07:10.117507+0000","last_undegraded":"2024-04-25T10:07:10.117507+0000","last_fullsized":"2024-04-25T10:07:10.117507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:16.166171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116058+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:07:10.116058+0000","last_peered":"2024-04-25T10:07:10.116058+0000","last_clean":"2024-04-25T10:07:10.116058+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:07:10.116058+0000","last_undegraded":"2024-04-25T10:07:10.116058+0000","last_fullsized":"2024-04-25T10:07:10.116058+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340032+0000","last_change":"2024-04-25T10:07:06.306253+0000","last_active":"2024-04-25T10:07:08.340032+0000","last_peered":"2024-04-25T10:07:08.340032+0000","last_clean":"2024-04-25T10:07:08.340032+0000","last_became_active":"2024-04-25T10:07:06.305849+0000","last_became_peered":"2024-04-25T10:07:06.305849+0000","last_unstale":"2024-04-25T10:07:08.340032+0000","last_undegraded":"2024-04-25T10:07:08.340032+0000","last_fullsized":"2024-04-25T10:07:08.340032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:28.708810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.430815+0000","last_change":"2024-04-25T10:07:07.337895+0000","last_active":"2024-04-25T10:07:08.430815+0000","last_peered":"2024-04-25T10:07:08.430815+0000","last_clean":"2024-04-25T10:07:08.430815+0000","last_became_active":"2024-04-25T10:07:07.337609+0000","last_became_peered":"2024-04-25T10:07:07.337609+0000","last_unstale":"2024-04-25T10:07:08.430815+0000","last_undegraded":"2024-04-25T10:07:08.430815+0000","last_fullsized":"2024-04-25T10:07:08.430815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:49.865533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115960+0000","last_change":"2024-04-25T10:07:06.325347+0000","last_active":"2024-04-25T10:07:10.115960+0000","last_peered":"2024-04-25T10:07:10.115960+0000","last_clean":"2024-04-25T10:07:10.115960+0000","last_became_active":"2024-04-25T10:07:06.324740+0000","last_became_peered":"2024-04-25T10:07:06.324740+0000","last_unstale":"2024-04-25T10:07:10.115960+0000","last_undegraded":"2024-04-25T10:07:10.115960+0000","last_fullsized":"2024-04-25T10:07:10.115960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.691331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.012839+0000","last_change":"2024-04-25T10:07:07.337867+0000","last_active":"2024-04-25T10:07:09.012839+0000","last_peered":"2024-04-25T10:07:09.012839+0000","last_clean":"2024-04-25T10:07:09.012839+0000","last_became_active":"2024-04-25T10:07:07.337521+0000","last_became_peered":"2024-04-25T10:07:07.337521+0000","last_unstale":"2024-04-25T10:07:09.012839+0000","last_undegraded":"2024-04-25T10:07:09.012839+0000","last_fullsized":"2024-04-25T10:07:09.012839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:05.611848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.337610+0000","last_change":"2024-04-25T10:07:06.308483+0000","last_active":"2024-04-25T10:07:08.337610+0000","last_peered":"2024-04-25T10:07:08.337610+0000","last_clean":"2024-04-25T10:07:08.337610+0000","last_became_active":"2024-04-25T10:07:06.308156+0000","last_became_peered":"2024-04-25T10:07:06.308156+0000","last_unstale":"2024-04-25T10:07:08.337610+0000","last_undegraded":"2024-04-25T10:07:08.337610+0000","last_fullsized":"2024-04-25T10:07:08.337610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:25.307131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339704+0000","last_change":"2024-04-25T10:07:07.351559+0000","last_active":"2024-04-25T10:07:08.339704+0000","last_peered":"2024-04-25T10:07:08.339704+0000","last_clean":"2024-04-25T10:07:08.339704+0000","last_became_active":"2024-04-25T10:07:07.351271+0000","last_became_peered":"2024-04-25T10:07:07.351271+0000","last_unstale":"2024-04-25T10:07:08.339704+0000","last_undegraded":"2024-04-25T10:07:08.339704+0000","last_fullsized":"2024-04-25T10:07:08.339704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.250284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340604+0000","last_change":"2024-04-25T10:07:06.314916+0000","last_active":"2024-04-25T10:07:08.340604+0000","last_peered":"2024-04-25T10:07:08.340604+0000","last_clean":"2024-04-25T10:07:08.340604+0000","last_became_active":"2024-04-25T10:07:06.314572+0000","last_became_peered":"2024-04-25T10:07:06.314572+0000","last_unstale":"2024-04-25T10:07:08.340604+0000","last_undegraded":"2024-04-25T10:07:08.340604+0000","last_fullsized":"2024-04-25T10:07:08.340604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:53.885696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115644+0000","last_change":"2024-04-25T10:07:07.342482+0000","last_active":"2024-04-25T10:07:10.115644+0000","last_peered":"2024-04-25T10:07:10.115644+0000","last_clean":"2024-04-25T10:07:10.115644+0000","last_became_active":"2024-04-25T10:07:07.341566+0000","last_became_peered":"2024-04-25T10:07:07.341566+0000","last_unstale":"2024-04-25T10:07:10.115644+0000","last_undegraded":"2024-04-25T10:07:10.115644+0000","last_fullsized":"2024-04-25T10:07:10.115644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:14.880915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339445+0000","last_change":"2024-04-25T10:07:06.317437+0000","last_active":"2024-04-25T10:07:08.339445+0000","last_peered":"2024-04-25T10:07:08.339445+0000","last_clean":"2024-04-25T10:07:08.339445+0000","last_became_active":"2024-04-25T10:07:06.316494+0000","last_became_peered":"2024-04-25T10:07:06.316494+0000","last_unstale":"2024-04-25T10:07:08.339445+0000","last_undegraded":"2024-04-25T10:07:08.339445+0000","last_fullsized":"2024-04-25T10:07:08.339445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341452+0000","last_change":"2024-04-25T10:07:07.337051+0000","last_active":"2024-04-25T10:07:08.341452+0000","last_peered":"2024-04-25T10:07:08.341452+0000","last_clean":"2024-04-25T10:07:08.341452+0000","last_became_active":"2024-04-25T10:07:07.334884+0000","last_became_peered":"2024-04-25T10:07:07.334884+0000","last_unstale":"2024-04-25T10:07:08.341452+0000","last_undegraded":"2024-04-25T10:07:08.341452+0000","last_fullsized":"2024-04-25T10:07:08.341452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:32.763141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117681+0000","last_change":"2024-04-25T10:07:06.331016+0000","last_active":"2024-04-25T10:07:10.117681+0000","last_peered":"2024-04-25T10:07:10.117681+0000","last_clean":"2024-04-25T10:07:10.117681+0000","last_became_active":"2024-04-25T10:07:06.330712+0000","last_became_peered":"2024-04-25T10:07:06.330712+0000","last_unstale":"2024-04-25T10:07:10.117681+0000","last_undegraded":"2024-04-25T10:07:10.117681+0000","last_fullsized":"2024-04-25T10:07:10.117681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.867432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339661+0000","last_change":"2024-04-25T10:07:07.351856+0000","last_active":"2024-04-25T10:07:08.339661+0000","last_peered":"2024-04-25T10:07:08.339661+0000","last_clean":"2024-04-25T10:07:08.339661+0000","last_became_active":"2024-04-25T10:07:07.351585+0000","last_became_peered":"2024-04-25T10:07:07.351585+0000","last_unstale":"2024-04-25T10:07:08.339661+0000","last_undegraded":"2024-04-25T10:07:08.339661+0000","last_fullsized":"2024-04-25T10:07:08.339661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340221+0000","last_change":"2024-04-25T10:07:06.317585+0000","last_active":"2024-04-25T10:07:08.340221+0000","last_peered":"2024-04-25T10:07:08.340221+0000","last_clean":"2024-04-25T10:07:08.340221+0000","last_became_active":"2024-04-25T10:07:06.316964+0000","last_became_peered":"2024-04-25T10:07:06.316964+0000","last_unstale":"2024-04-25T10:07:08.340221+0000","last_undegraded":"2024-04-25T10:07:08.340221+0000","last_fullsized":"2024-04-25T10:07:08.340221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:58.603977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431612+0000","last_change":"2024-04-25T10:07:07.352730+0000","last_active":"2024-04-25T10:07:08.431612+0000","last_peered":"2024-04-25T10:07:08.431612+0000","last_clean":"2024-04-25T10:07:08.431612+0000","last_became_active":"2024-04-25T10:07:07.352486+0000","last_became_peered":"2024-04-25T10:07:07.352486+0000","last_unstale":"2024-04-25T10:07:08.431612+0000","last_undegraded":"2024-04-25T10:07:08.431612+0000","last_fullsized":"2024-04-25T10:07:08.431612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:23.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627361+0000","last_change":"2024-04-25T10:07:06.327061+0000","last_active":"2024-04-25T10:07:09.627361+0000","last_peered":"2024-04-25T10:07:09.627361+0000","last_clean":"2024-04-25T10:07:09.627361+0000","last_became_active":"2024-04-25T10:07:06.326227+0000","last_became_peered":"2024-04-25T10:07:06.326227+0000","last_unstale":"2024-04-25T10:07:09.627361+0000","last_undegraded":"2024-04-25T10:07:09.627361+0000","last_fullsized":"2024-04-25T10:07:09.627361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013232+0000","last_change":"2024-04-25T10:07:07.326642+0000","last_active":"2024-04-25T10:07:09.013232+0000","last_peered":"2024-04-25T10:07:09.013232+0000","last_clean":"2024-04-25T10:07:09.013232+0000","last_became_active":"2024-04-25T10:07:07.326261+0000","last_became_peered":"2024-04-25T10:07:07.326261+0000","last_unstale":"2024-04-25T10:07:09.013232+0000","last_undegraded":"2024-04-25T10:07:09.013232+0000","last_fullsized":"2024-04-25T10:07:09.013232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:01.102598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115657+0000","last_change":"2024-04-25T10:07:06.325264+0000","last_active":"2024-04-25T10:07:10.115657+0000","last_peered":"2024-04-25T10:07:10.115657+0000","last_clean":"2024-04-25T10:07:10.115657+0000","last_became_active":"2024-04-25T10:07:06.324704+0000","last_became_peered":"2024-04-25T10:07:06.324704+0000","last_unstale":"2024-04-25T10:07:10.115657+0000","last_undegraded":"2024-04-25T10:07:10.115657+0000","last_fullsized":"2024-04-25T10:07:10.115657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:52.458435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.115184+0000","last_change":"2024-04-25T10:07:07.340986+0000","last_active":"2024-04-25T10:07:10.115184+0000","last_peered":"2024-04-25T10:07:10.115184+0000","last_clean":"2024-04-25T10:07:10.115184+0000","last_became_active":"2024-04-25T10:07:07.332637+0000","last_became_peered":"2024-04-25T10:07:07.332637+0000","last_unstale":"2024-04-25T10:07:10.115184+0000","last_undegraded":"2024-04-25T10:07:10.115184+0000","last_fullsized":"2024-04-25T10:07:10.115184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:17.677957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339302+0000","last_change":"2024-04-25T10:07:06.324149+0000","last_active":"2024-04-25T10:07:08.339302+0000","last_peered":"2024-04-25T10:07:08.339302+0000","last_clean":"2024-04-25T10:07:08.339302+0000","last_became_active":"2024-04-25T10:07:06.323900+0000","last_became_peered":"2024-04-25T10:07:06.323900+0000","last_unstale":"2024-04-25T10:07:08.339302+0000","last_undegraded":"2024-04-25T10:07:08.339302+0000","last_fullsized":"2024-04-25T10:07:08.339302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431004+0000","last_change":"2024-04-25T10:07:07.337614+0000","last_active":"2024-04-25T10:07:08.431004+0000","last_peered":"2024-04-25T10:07:08.431004+0000","last_clean":"2024-04-25T10:07:08.431004+0000","last_became_active":"2024-04-25T10:07:07.333283+0000","last_became_peered":"2024-04-25T10:07:07.333283+0000","last_unstale":"2024-04-25T10:07:08.431004+0000","last_undegraded":"2024-04-25T10:07:08.431004+0000","last_fullsized":"2024-04-25T10:07:08.431004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:21.718875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.339925+0000","last_change":"2024-04-25T10:07:06.317292+0000","last_active":"2024-04-25T10:07:08.339925+0000","last_peered":"2024-04-25T10:07:08.339925+0000","last_clean":"2024-04-25T10:07:08.339925+0000","last_became_active":"2024-04-25T10:07:06.316862+0000","last_became_peered":"2024-04-25T10:07:06.316862+0000","last_unstale":"2024-04-25T10:07:08.339925+0000","last_undegraded":"2024-04-25T10:07:08.339925+0000","last_fullsized":"2024-04-25T10:07:08.339925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.691331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431273+0000","last_change":"2024-04-25T10:07:07.329541+0000","last_active":"2024-04-25T10:07:08.431273+0000","last_peered":"2024-04-25T10:07:08.431273+0000","last_clean":"2024-04-25T10:07:08.431273+0000","last_became_active":"2024-04-25T10:07:07.329296+0000","last_became_peered":"2024-04-25T10:07:07.329296+0000","last_unstale":"2024-04-25T10:07:08.431273+0000","last_undegraded":"2024-04-25T10:07:08.431273+0000","last_fullsized":"2024-04-25T10:07:08.431273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:39.263734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117039+0000","last_change":"2024-04-25T10:07:06.331241+0000","last_active":"2024-04-25T10:07:10.117039+0000","last_peered":"2024-04-25T10:07:10.117039+0000","last_clean":"2024-04-25T10:07:10.117039+0000","last_became_active":"2024-04-25T10:07:06.330637+0000","last_became_peered":"2024-04-25T10:07:06.330637+0000","last_unstale":"2024-04-25T10:07:10.117039+0000","last_undegraded":"2024-04-25T10:07:10.117039+0000","last_fullsized":"2024-04-25T10:07:10.117039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:16.664475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.627101+0000","last_change":"2024-04-25T10:07:07.334097+0000","last_active":"2024-04-25T10:07:09.627101+0000","last_peered":"2024-04-25T10:07:09.627101+0000","last_clean":"2024-04-25T10:07:09.627101+0000","last_became_active":"2024-04-25T10:07:07.332086+0000","last_became_peered":"2024-04-25T10:07:07.332086+0000","last_unstale":"2024-04-25T10:07:09.627101+0000","last_undegraded":"2024-04-25T10:07:09.627101+0000","last_fullsized":"2024-04-25T10:07:09.627101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.370945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.341238+0000","last_change":"2024-04-25T10:07:07.349145+0000","last_active":"2024-04-25T10:07:08.341238+0000","last_peered":"2024-04-25T10:07:08.341238+0000","last_clean":"2024-04-25T10:07:08.341238+0000","last_became_active":"2024-04-25T10:07:07.348916+0000","last_became_peered":"2024-04-25T10:07:07.348916+0000","last_unstale":"2024-04-25T10:07:08.341238+0000","last_undegraded":"2024-04-25T10:07:08.341238+0000","last_fullsized":"2024-04-25T10:07:08.341238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:35.056084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.116709+0000","last_change":"2024-04-25T10:07:06.330571+0000","last_active":"2024-04-25T10:07:10.116709+0000","last_peered":"2024-04-25T10:07:10.116709+0000","last_clean":"2024-04-25T10:07:10.116709+0000","last_became_active":"2024-04-25T10:07:06.329886+0000","last_became_peered":"2024-04-25T10:07:06.329886+0000","last_unstale":"2024-04-25T10:07:10.116709+0000","last_undegraded":"2024-04-25T10:07:10.116709+0000","last_fullsized":"2024-04-25T10:07:10.116709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:36.651359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.340935+0000","last_change":"2024-04-25T10:07:07.347234+0000","last_active":"2024-04-25T10:07:08.340935+0000","last_peered":"2024-04-25T10:07:08.340935+0000","last_clean":"2024-04-25T10:07:08.340935+0000","last_became_active":"2024-04-25T10:07:07.346961+0000","last_became_peered":"2024-04-25T10:07:07.346961+0000","last_unstale":"2024-04-25T10:07:08.340935+0000","last_undegraded":"2024-04-25T10:07:08.340935+0000","last_fullsized":"2024-04-25T10:07:08.340935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:23.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.117348+0000","last_change":"2024-04-25T10:07:06.330806+0000","last_active":"2024-04-25T10:07:10.117348+0000","last_peered":"2024-04-25T10:07:10.117348+0000","last_clean":"2024-04-25T10:07:10.117348+0000","last_became_active":"2024-04-25T10:07:06.330360+0000","last_became_peered":"2024-04-25T10:07:06.330360+0000","last_unstale":"2024-04-25T10:07:10.117348+0000","last_undegraded":"2024-04-25T10:07:10.117348+0000","last_fullsized":"2024-04-25T10:07:10.117348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:48.687788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013341+0000","last_change":"2024-04-25T10:07:07.337072+0000","last_active":"2024-04-25T10:07:09.013341+0000","last_peered":"2024-04-25T10:07:09.013341+0000","last_clean":"2024-04-25T10:07:09.013341+0000","last_became_active":"2024-04-25T10:07:07.336767+0000","last_became_peered":"2024-04-25T10:07:07.336767+0000","last_unstale":"2024-04-25T10:07:09.013341+0000","last_undegraded":"2024-04-25T10:07:09.013341+0000","last_fullsized":"2024-04-25T10:07:09.013341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.370945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431572+0000","last_change":"2024-04-25T10:07:06.322321+0000","last_active":"2024-04-25T10:07:08.431572+0000","last_peered":"2024-04-25T10:07:08.431572+0000","last_clean":"2024-04-25T10:07:08.431572+0000","last_became_active":"2024-04-25T10:07:06.321990+0000","last_became_peered":"2024-04-25T10:07:06.321990+0000","last_unstale":"2024-04-25T10:07:08.431572+0000","last_undegraded":"2024-04-25T10:07:08.431572+0000","last_fullsized":"2024-04-25T10:07:08.431572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:25.307131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:08.431571+0000","last_change":"2024-04-25T10:07:07.337246+0000","last_active":"2024-04-25T10:07:08.431571+0000","last_peered":"2024-04-25T10:07:08.431571+0000","last_clean":"2024-04-25T10:07:08.431571+0000","last_became_active":"2024-04-25T10:07:07.336894+0000","last_became_peered":"2024-04-25T10:07:07.336894+0000","last_unstale":"2024-04-25T10:07:08.431571+0000","last_undegraded":"2024-04-25T10:07:08.431571+0000","last_fullsized":"2024-04-25T10:07:08.431571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:06.274514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:06.274514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:26.671430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.013544+0000","last_change":"2024-04-25T10:07:06.324947+0000","last_active":"2024-04-25T10:07:09.013544+0000","last_peered":"2024-04-25T10:07:09.013544+0000","last_clean":"2024-04-25T10:07:09.013544+0000","last_became_active":"2024-04-25T10:07:06.324464+0000","last_became_peered":"2024-04-25T10:07:06.324464+0000","last_unstale":"2024-04-25T10:07:09.013544+0000","last_undegraded":"2024-04-25T10:07:09.013544+0000","last_fullsized":"2024-04-25T10:07:09.013544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:05.261032+0000","last_clean_scrub_stamp":"2024-04-25T10:07:05.261032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:25.307131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:17.830 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:07:17.830 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T10:07:17.830 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T10:07:17.830 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:07:18.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.055+0000 7f6306600640 1 Processor -- start 2024-04-25T10:07:18.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.056+0000 7f6306600640 1 -- start start 2024-04-25T10:07:18.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.057+0000 7f6306600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6300147270 0x7f63000b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.057+0000 7f6306600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f630015dee0 0x7f630015e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.057+0000 7f6306600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630015e890 0x7f63001614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.057+0000 7f6306600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630015ed20 con 0x7f6300147270 2024-04-25T10:07:18.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.057+0000 7f6306600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63000af000 con 0x7f630015dee0 2024-04-25T10:07:18.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.057+0000 7f6306600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63000b4a80 con 0x7f630015e890 2024-04-25T10:07:18.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.057+0000 7f6305200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6300147270 0x7f63000b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.058+0000 7f6305200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6300147270 0x7f63000b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34948/0 (socket says 172.21.15.119:34948) 2024-04-25T10:07:18.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.058+0000 7f6305200640 1 -- 172.21.15.119:0/849737576 learned_addr learned my addr 172.21.15.119:0/849737576 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:18.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.058+0000 7f6305c00640 1 --2- 172.21.15.119:0/849737576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630015e890 0x7f63001614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.058+0000 7f62f7e00640 1 --2- 172.21.15.119:0/849737576 >> v2:172.21.15.132:3300/0 conn(0x7f630015dee0 0x7f630015e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.058+0000 7f6305200640 1 -- 172.21.15.119:0/849737576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630015e890 msgr2=0x7f63001614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.058+0000 7f6305200640 1 --2- 172.21.15.119:0/849737576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630015e890 0x7f63001614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.058+0000 7f6305200640 1 -- 172.21.15.119:0/849737576 >> v2:172.21.15.132:3300/0 conn(0x7f630015dee0 msgr2=0x7f630015e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.058+0000 7f6305200640 1 --2- 172.21.15.119:0/849737576 >> v2:172.21.15.132:3300/0 conn(0x7f630015dee0 0x7f630015e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.058+0000 7f6305200640 1 -- 172.21.15.119:0/849737576 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63000b4d60 con 0x7f6300147270 2024-04-25T10:07:18.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.058+0000 7f62f7e00640 1 --2- 172.21.15.119:0/849737576 >> v2:172.21.15.132:3300/0 conn(0x7f630015dee0 0x7f630015e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:18.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.060+0000 7f6305200640 1 --2- 172.21.15.119:0/849737576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6300147270 0x7f63000b4410 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f630006f500 tx=0x7f62f002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2503b9fd9b41250b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.060+0000 7f62ffe00640 1 -- 172.21.15.119:0/849737576 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0037070 con 0x7f6300147270 2024-04-25T10:07:18.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.060+0000 7f62ffe00640 1 -- 172.21.15.119:0/849737576 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f0032440 con 0x7f6300147270 2024-04-25T10:07:18.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.060+0000 7f62ffe00640 1 -- 172.21.15.119:0/849737576 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0032810 con 0x7f6300147270 2024-04-25T10:07:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.061+0000 7f6306600640 1 -- 172.21.15.119:0/849737576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6300147270 msgr2=0x7f63000b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.061+0000 7f6306600640 1 --2- 172.21.15.119:0/849737576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6300147270 0x7f63000b4410 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f630006f500 tx=0x7f62f002a140 comp rx=0 tx=0).stop 2024-04-25T10:07:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.061+0000 7f6306600640 1 -- 172.21.15.119:0/849737576 shutdown_connections 2024-04-25T10:07:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.061+0000 7f6306600640 1 --2- 172.21.15.119:0/849737576 >> v2:172.21.15.132:3300/0 conn(0x7f630015dee0 0x7f630015e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.061+0000 7f6306600640 1 --2- 172.21.15.119:0/849737576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630015e890 0x7f63001614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.061+0000 7f6306600640 1 --2- 172.21.15.119:0/849737576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6300147270 0x7f63000b4410 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.061+0000 7f6306600640 1 -- 172.21.15.119:0/849737576 >> 172.21.15.119:0/849737576 conn(0x7f630006b1d0 msgr2=0x7f63000b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.061+0000 7f6306600640 1 -- 172.21.15.119:0/849737576 shutdown_connections 2024-04-25T10:07:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.061+0000 7f6306600640 1 -- 172.21.15.119:0/849737576 wait complete. 2024-04-25T10:07:18.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.063+0000 7f6306600640 1 Processor -- start 2024-04-25T10:07:18.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.063+0000 7f6306600640 1 -- start start 2024-04-25T10:07:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6306600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6300147270 0x7f6300134970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6306600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f630015dee0 0x7f63000c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6306600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630015e890 0x7f6300135270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6306600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63000b5600 con 0x7f6300147270 2024-04-25T10:07:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6306600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63000b5320 con 0x7f630015dee0 2024-04-25T10:07:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6306600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63001619e0 con 0x7f630015e890 2024-04-25T10:07:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6305200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6300147270 0x7f6300134970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6305200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6300147270 0x7f6300134970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34964/0 (socket says 172.21.15.119:34964) 2024-04-25T10:07:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6305200640 1 -- 172.21.15.119:0/154269529 learned_addr learned my addr 172.21.15.119:0/154269529 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6305c00640 1 --2- 172.21.15.119:0/154269529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630015e890 0x7f6300135270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6305200640 1 -- 172.21.15.119:0/154269529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630015e890 msgr2=0x7f6300135270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6305200640 1 --2- 172.21.15.119:0/154269529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630015e890 0x7f6300135270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6305200640 1 -- 172.21.15.119:0/154269529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f630015dee0 msgr2=0x7f63000c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6305200640 1 --2- 172.21.15.119:0/154269529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f630015dee0 0x7f63000c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.064+0000 7f6305200640 1 -- 172.21.15.119:0/154269529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63000c59f0 con 0x7f6300147270 2024-04-25T10:07:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.065+0000 7f6305200640 1 --2- 172.21.15.119:0/154269529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6300147270 0x7f6300134970 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f62f0000c00 tx=0x7f62f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.065+0000 7f62fd600640 1 -- 172.21.15.119:0/154269529 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0041020 con 0x7f6300147270 2024-04-25T10:07:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.065+0000 7f62fd600640 1 -- 172.21.15.119:0/154269529 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f00306c0 con 0x7f6300147270 2024-04-25T10:07:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.065+0000 7f62fd600640 1 -- 172.21.15.119:0/154269529 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0037040 con 0x7f6300147270 2024-04-25T10:07:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.065+0000 7f6306600640 1 -- 172.21.15.119:0/154269529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63000c5c90 con 0x7f6300147270 2024-04-25T10:07:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.065+0000 7f6306600640 1 -- 172.21.15.119:0/154269529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63000c6250 con 0x7f6300147270 2024-04-25T10:07:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.066+0000 7f6306600640 1 -- 172.21.15.119:0/154269529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63000749b0 con 0x7f6300147270 2024-04-25T10:07:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.067+0000 7f62fd600640 1 -- 172.21.15.119:0/154269529 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62f0044020 con 0x7f6300147270 2024-04-25T10:07:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.068+0000 7f62fd600640 1 --2- 172.21.15.119:0/154269529 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62dc06b4d0 0x7f62dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.068+0000 7f62fd600640 1 -- 172.21.15.119:0/154269529 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62f00aaa10 con 0x7f6300147270 2024-04-25T10:07:18.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.068+0000 7f62f7e00640 1 --2- 172.21.15.119:0/154269529 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62dc06b4d0 0x7f62dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.069+0000 7f62f7e00640 1 --2- 172.21.15.119:0/154269529 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62dc06b4d0 0x7f62dc06d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f63001442e0 tx=0x7f62e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.072+0000 7f62fd600640 1 -- 172.21.15.119:0/154269529 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62f004b030 con 0x7f6300147270 2024-04-25T10:07:18.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.290+0000 7f6306600640 1 -- 172.21.15.119:0/154269529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f63000b2180 con 0x7f6300147270 2024-04-25T10:07:18.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.291+0000 7f62fd600640 1 -- 172.21.15.119:0/154269529 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f62f0079880 con 0x7f6300147270 2024-04-25T10:07:18.296 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:18.296 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:07:18.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.293+0000 7f6306600640 1 -- 172.21.15.119:0/154269529 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62dc06b4d0 msgr2=0x7f62dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.293+0000 7f6306600640 1 --2- 172.21.15.119:0/154269529 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62dc06b4d0 0x7f62dc06d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f63001442e0 tx=0x7f62e8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:18.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.293+0000 7f6306600640 1 -- 172.21.15.119:0/154269529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6300147270 msgr2=0x7f6300134970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.293+0000 7f6306600640 1 --2- 172.21.15.119:0/154269529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6300147270 0x7f6300134970 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f62f0000c00 tx=0x7f62f00046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.293+0000 7f6306600640 1 -- 172.21.15.119:0/154269529 shutdown_connections 2024-04-25T10:07:18.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.293+0000 7f6306600640 1 --2- 172.21.15.119:0/154269529 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62dc06b4d0 0x7f62dc06d990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.293+0000 7f6306600640 1 --2- 172.21.15.119:0/154269529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630015e890 0x7f6300135270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.293+0000 7f6306600640 1 --2- 172.21.15.119:0/154269529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f630015dee0 0x7f63000c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.293+0000 7f6306600640 1 --2- 172.21.15.119:0/154269529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6300147270 0x7f6300134970 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.293+0000 7f6306600640 1 -- 172.21.15.119:0/154269529 >> 172.21.15.119:0/154269529 conn(0x7f630006b1d0 msgr2=0x7f6300160aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.294+0000 7f6306600640 1 -- 172.21.15.119:0/154269529 shutdown_connections 2024-04-25T10:07:18.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.294+0000 7f6306600640 1 -- 172.21.15.119:0/154269529 wait complete. 2024-04-25T10:07:18.317 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T10:07:18.317 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:18.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.512+0000 7fec93e00640 1 Processor -- start 2024-04-25T10:07:18.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.512+0000 7fec93e00640 1 -- start start 2024-04-25T10:07:18.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.514+0000 7fec93e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fec940b5050 0x7fec9415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.514+0000 7fec93e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fec9406e430 0x7fec9415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.514+0000 7fec93e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec940cd440 con 0x7fec94144470 2024-04-25T10:07:18.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.514+0000 7fec93e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec94145340 con 0x7fec9406e430 2024-04-25T10:07:18.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.514+0000 7fec93e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec940cd720 con 0x7fec940b5050 2024-04-25T10:07:18.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.514+0000 7fec93400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fec9406e430 0x7fec9415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.514+0000 7fec93400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fec9406e430 0x7fec9415bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34966/0 (socket says 172.21.15.119:34966) 2024-04-25T10:07:18.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.514+0000 7fec93400640 1 -- 172.21.15.119:0/2650922731 learned_addr learned my addr 172.21.15.119:0/2650922731 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:18.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.514+0000 7fec8a000640 1 --2- 172.21.15.119:0/2650922731 >> v2:172.21.15.132:3300/0 conn(0x7fec940b5050 0x7fec9415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.514+0000 7fec93400640 1 -- 172.21.15.119:0/2650922731 >> v2:172.21.15.132:3300/0 conn(0x7fec940b5050 msgr2=0x7fec9415fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.515+0000 7fec93400640 1 --2- 172.21.15.119:0/2650922731 >> v2:172.21.15.132:3300/0 conn(0x7fec940b5050 0x7fec9415fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.515+0000 7fec93400640 1 -- 172.21.15.119:0/2650922731 >> v1:172.21.15.132:6789/0 conn(0x7fec94144470 legacy=0x7fec940b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.515+0000 7fec93400640 1 -- 172.21.15.119:0/2650922731 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec940cda00 con 0x7fec9406e430 2024-04-25T10:07:18.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.515+0000 7fec8a000640 1 --2- 172.21.15.119:0/2650922731 >> v2:172.21.15.132:3300/0 conn(0x7fec940b5050 0x7fec9415fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:18.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.516+0000 7fec93400640 1 --2- 172.21.15.119:0/2650922731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec9406e430 0x7fec9415bcf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fec9406f500 tx=0x7fec8c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add474e9f7404923 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.516+0000 7fec92000640 1 -- 172.21.15.119:0/2650922731 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c037070 con 0x7fec9406e430 2024-04-25T10:07:18.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.516+0000 7fec92000640 1 -- 172.21.15.119:0/2650922731 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c032440 con 0x7fec9406e430 2024-04-25T10:07:18.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.516+0000 7fec92000640 1 -- 172.21.15.119:0/2650922731 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c032810 con 0x7fec9406e430 2024-04-25T10:07:18.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.516+0000 7fec93e00640 1 -- 172.21.15.119:0/2650922731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec9406e430 msgr2=0x7fec9415bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.516+0000 7fec93e00640 1 --2- 172.21.15.119:0/2650922731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec9406e430 0x7fec9415bcf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fec9406f500 tx=0x7fec8c02a230 comp rx=0 tx=0).stop 2024-04-25T10:07:18.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.517+0000 7fec93e00640 1 -- 172.21.15.119:0/2650922731 shutdown_connections 2024-04-25T10:07:18.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.517+0000 7fec93e00640 1 --2- 172.21.15.119:0/2650922731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec9406e430 0x7fec9415bcf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.517+0000 7fec93e00640 1 --2- 172.21.15.119:0/2650922731 >> v2:172.21.15.132:3300/0 conn(0x7fec940b5050 0x7fec9415fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.517+0000 7fec93e00640 1 -- 172.21.15.119:0/2650922731 >> 172.21.15.119:0/2650922731 conn(0x7fec9406b1d0 msgr2=0x7fec940b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.517+0000 7fec93e00640 1 -- 172.21.15.119:0/2650922731 shutdown_connections 2024-04-25T10:07:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.517+0000 7fec93e00640 1 -- 172.21.15.119:0/2650922731 wait complete. 2024-04-25T10:07:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.518+0000 7fec93e00640 1 Processor -- start 2024-04-25T10:07:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.518+0000 7fec93e00640 1 -- start start 2024-04-25T10:07:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.519+0000 7fec93e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fec9406e430 0x7fec9415d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.519+0000 7fec93e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec940b5050 0x7fec9415b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.519+0000 7fec93e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec94144470 0x7fec941e59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.519+0000 7fec93e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec940ce200 con 0x7fec94144470 2024-04-25T10:07:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.519+0000 7fec93e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec94144850 con 0x7fec940b5050 2024-04-25T10:07:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.519+0000 7fec93e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9415c230 con 0x7fec9406e430 2024-04-25T10:07:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.520+0000 7fec93400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec94144470 0x7fec941e59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.520+0000 7fec93400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec94144470 0x7fec941e59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34972/0 (socket says 172.21.15.119:34972) 2024-04-25T10:07:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.520+0000 7fec93400640 1 -- 172.21.15.119:0/438362691 learned_addr learned my addr 172.21.15.119:0/438362691 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.520+0000 7fec92a00640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fec9406e430 0x7fec9415d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.520+0000 7fec8a000640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec940b5050 0x7fec9415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.520+0000 7fec93400640 1 -- 172.21.15.119:0/438362691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fec9406e430 msgr2=0x7fec9415d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.520+0000 7fec93400640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fec9406e430 0x7fec9415d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.520+0000 7fec93400640 1 -- 172.21.15.119:0/438362691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec940b5050 msgr2=0x7fec9415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.520+0000 7fec93400640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec940b5050 0x7fec9415b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.520+0000 7fec93400640 1 -- 172.21.15.119:0/438362691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9415b5d0 con 0x7fec94144470 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.520+0000 7fec8a000640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec940b5050 0x7fec9415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.520+0000 7fec93400640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec94144470 0x7fec941e59c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0090b0 tx=0x7fec8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.521+0000 7fec8b400640 1 -- 172.21.15.119:0/438362691 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c041020 con 0x7fec94144470 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.521+0000 7fec8b400640 1 -- 172.21.15.119:0/438362691 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c030990 con 0x7fec94144470 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.521+0000 7fec93e00640 1 -- 172.21.15.119:0/438362691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec941e5f00 con 0x7fec94144470 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.521+0000 7fec8b400640 1 -- 172.21.15.119:0/438362691 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c037040 con 0x7fec94144470 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.521+0000 7fec93e00640 1 -- 172.21.15.119:0/438362691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9415da80 con 0x7fec94144470 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.522+0000 7fec93e00640 1 -- 172.21.15.119:0/438362691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec940ccb50 con 0x7fec94144470 2024-04-25T10:07:18.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.523+0000 7fec8b400640 1 -- 172.21.15.119:0/438362691 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec8c044020 con 0x7fec94144470 2024-04-25T10:07:18.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.524+0000 7fec8b400640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fec7006b4d0 0x7fec7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.524+0000 7fec8b400640 1 -- 172.21.15.119:0/438362691 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec8c0aab80 con 0x7fec94144470 2024-04-25T10:07:18.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.524+0000 7fec92a00640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fec7006b4d0 0x7fec7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.525+0000 7fec92a00640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fec7006b4d0 0x7fec7006d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec84009f20 tx=0x7fec840094d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.528+0000 7fec8b400640 1 -- 172.21.15.119:0/438362691 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec940ccb50 con 0x7fec94144470 2024-04-25T10:07:18.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.725+0000 7fec93e00640 1 -- 172.21.15.119:0/438362691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec940b2090 con 0x7fec94144470 2024-04-25T10:07:18.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.727+0000 7fec8b400640 1 -- 172.21.15.119:0/438362691 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fec8c04b030 con 0x7fec94144470 2024-04-25T10:07:18.733 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:18.733 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:08.319104+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:18.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.730+0000 7fec93e00640 1 -- 172.21.15.119:0/438362691 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fec7006b4d0 msgr2=0x7fec7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.730+0000 7fec93e00640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fec7006b4d0 0x7fec7006d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec84009f20 tx=0x7fec840094d0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.730+0000 7fec93e00640 1 -- 172.21.15.119:0/438362691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec94144470 msgr2=0x7fec941e59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.730+0000 7fec93e00640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec94144470 0x7fec941e59c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0090b0 tx=0x7fec8c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.731+0000 7fec93e00640 1 -- 172.21.15.119:0/438362691 shutdown_connections 2024-04-25T10:07:18.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.731+0000 7fec93e00640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fec7006b4d0 0x7fec7006d990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.731+0000 7fec93e00640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec94144470 0x7fec941e59c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.731+0000 7fec93e00640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec940b5050 0x7fec9415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.731+0000 7fec93e00640 1 --2- 172.21.15.119:0/438362691 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fec9406e430 0x7fec9415d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.731+0000 7fec93e00640 1 -- 172.21.15.119:0/438362691 >> 172.21.15.119:0/438362691 conn(0x7fec9406b1d0 msgr2=0x7fec941411c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.731+0000 7fec93e00640 1 -- 172.21.15.119:0/438362691 shutdown_connections 2024-04-25T10:07:18.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.731+0000 7fec93e00640 1 -- 172.21.15.119:0/438362691 wait complete. 2024-04-25T10:07:18.757 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:05.135832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:05.768083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:18.758 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:18.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.950+0000 7f7105600640 1 Processor -- start 2024-04-25T10:07:18.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.950+0000 7f7105600640 1 -- start start 2024-04-25T10:07:18.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.952+0000 7f7105600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71000b42c0 0x7f710015ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.952+0000 7f7105600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f710015aef0 0x7f71000cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.952+0000 7f7105600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710015b370 con 0x7f71000b46a0 2024-04-25T10:07:18.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.952+0000 7f7105600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71000d05d0 con 0x7f71000b42c0 2024-04-25T10:07:18.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.952+0000 7f7105600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710015b650 con 0x7f710015aef0 2024-04-25T10:07:18.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.952+0000 7f70fea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71000b42c0 0x7f710015ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.952+0000 7f70fea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71000b42c0 0x7f710015ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34978/0 (socket says 172.21.15.119:34978) 2024-04-25T10:07:18.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.952+0000 7f70fea00640 1 -- 172.21.15.119:0/1057181194 learned_addr learned my addr 172.21.15.119:0/1057181194 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:18.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.952+0000 7f70fe000640 1 --2- 172.21.15.119:0/1057181194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f710015aef0 0x7f71000cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.953+0000 7f70fea00640 1 -- 172.21.15.119:0/1057181194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f710015aef0 msgr2=0x7f71000cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.953+0000 7f70fea00640 1 --2- 172.21.15.119:0/1057181194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f710015aef0 0x7f71000cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.953+0000 7f70fea00640 1 -- 172.21.15.119:0/1057181194 >> v1:172.21.15.132:6789/0 conn(0x7f71000b46a0 legacy=0x7f71001549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.953+0000 7f70fea00640 1 -- 172.21.15.119:0/1057181194 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710015b930 con 0x7f71000b42c0 2024-04-25T10:07:18.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.953+0000 7f70fe000640 1 --2- 172.21.15.119:0/1057181194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f710015aef0 0x7f71000cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:18.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.954+0000 7f70fea00640 1 --2- 172.21.15.119:0/1057181194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71000b42c0 0x7f710015ef40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f70f00098a0 tx=0x7f70f002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e90814167ed66b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.954+0000 7f70fd600640 1 -- 172.21.15.119:0/1057181194 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0037070 con 0x7f71000b42c0 2024-04-25T10:07:18.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.954+0000 7f70fd600640 1 -- 172.21.15.119:0/1057181194 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f0032440 con 0x7f71000b42c0 2024-04-25T10:07:18.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.954+0000 7f70fd600640 1 -- 172.21.15.119:0/1057181194 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0032810 con 0x7f71000b42c0 2024-04-25T10:07:18.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.955+0000 7f7105600640 1 -- 172.21.15.119:0/1057181194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71000b42c0 msgr2=0x7f710015ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.955+0000 7f7105600640 1 --2- 172.21.15.119:0/1057181194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71000b42c0 0x7f710015ef40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f70f00098a0 tx=0x7f70f002a160 comp rx=0 tx=0).stop 2024-04-25T10:07:18.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.955+0000 7f7105600640 1 -- 172.21.15.119:0/1057181194 shutdown_connections 2024-04-25T10:07:18.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.955+0000 7f7105600640 1 --2- 172.21.15.119:0/1057181194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f710015aef0 0x7f71000cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.955+0000 7f7105600640 1 --2- 172.21.15.119:0/1057181194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71000b42c0 0x7f710015ef40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.955+0000 7f7105600640 1 -- 172.21.15.119:0/1057181194 >> 172.21.15.119:0/1057181194 conn(0x7f710006aba0 msgr2=0x7f710006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.955+0000 7f7105600640 1 -- 172.21.15.119:0/1057181194 shutdown_connections 2024-04-25T10:07:18.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.955+0000 7f7105600640 1 -- 172.21.15.119:0/1057181194 wait complete. 2024-04-25T10:07:18.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.957+0000 7f7105600640 1 Processor -- start 2024-04-25T10:07:18.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.957+0000 7f7105600640 1 -- start start 2024-04-25T10:07:18.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.958+0000 7f7105600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71000b42c0 0x7f710014e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.958+0000 7f7105600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71000b46a0 0x7f71000cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.958+0000 7f7105600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f710015aef0 0x7f710015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.958+0000 7f7105600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710006eae0 con 0x7f71000b46a0 2024-04-25T10:07:18.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.958+0000 7f7105600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71000cf980 con 0x7f710015aef0 2024-04-25T10:07:18.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.958+0000 7f7105600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710006ec60 con 0x7f71000b42c0 2024-04-25T10:07:18.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.958+0000 7f70fe000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71000b46a0 0x7f71000cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.958+0000 7f70fe000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71000b46a0 0x7f71000cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34992/0 (socket says 172.21.15.119:34992) 2024-04-25T10:07:18.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.958+0000 7f70fe000640 1 -- 172.21.15.119:0/2480014431 learned_addr learned my addr 172.21.15.119:0/2480014431 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:18.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.959+0000 7f70ff400640 1 --2- 172.21.15.119:0/2480014431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f710015aef0 0x7f710015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.959+0000 7f70fe000640 1 -- 172.21.15.119:0/2480014431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71000b42c0 msgr2=0x7f710014e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.959+0000 7f70fe000640 1 --2- 172.21.15.119:0/2480014431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71000b42c0 0x7f710014e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.959+0000 7f70fe000640 1 -- 172.21.15.119:0/2480014431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f710015aef0 msgr2=0x7f710015cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.959+0000 7f70fe000640 1 --2- 172.21.15.119:0/2480014431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f710015aef0 0x7f710015cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.959+0000 7f70fe000640 1 -- 172.21.15.119:0/2480014431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71000a9790 con 0x7f71000b46a0 2024-04-25T10:07:18.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.959+0000 7f70ff400640 1 --2- 172.21.15.119:0/2480014431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f710015aef0 0x7f710015cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:18.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.959+0000 7f70fe000640 1 --2- 172.21.15.119:0/2480014431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71000b46a0 0x7f71000cb320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00c820 tx=0x7f70ec00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.959+0000 7f70f6a00640 1 -- 172.21.15.119:0/2480014431 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec015070 con 0x7f71000b46a0 2024-04-25T10:07:18.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.959+0000 7f70f6a00640 1 -- 172.21.15.119:0/2480014431 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec007a30 con 0x7f71000b46a0 2024-04-25T10:07:18.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.959+0000 7f7105600640 1 -- 172.21.15.119:0/2480014431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71000a86f0 con 0x7f71000b46a0 2024-04-25T10:07:18.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.959+0000 7f70f6a00640 1 -- 172.21.15.119:0/2480014431 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec007de0 con 0x7f71000b46a0 2024-04-25T10:07:18.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.960+0000 7f7105600640 1 -- 172.21.15.119:0/2480014431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71000a8cb0 con 0x7f71000b46a0 2024-04-25T10:07:18.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.960+0000 7f7105600640 1 -- 172.21.15.119:0/2480014431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71000750b0 con 0x7f71000b46a0 2024-04-25T10:07:18.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.961+0000 7f70f6a00640 1 -- 172.21.15.119:0/2480014431 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70ec018070 con 0x7f71000b46a0 2024-04-25T10:07:18.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.962+0000 7f70f6a00640 1 --2- 172.21.15.119:0/2480014431 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70e006b4d0 0x7f70e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.962+0000 7f70f6a00640 1 -- 172.21.15.119:0/2480014431 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70ec08a120 con 0x7f71000b46a0 2024-04-25T10:07:18.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.963+0000 7f70fea00640 1 --2- 172.21.15.119:0/2480014431 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70e006b4d0 0x7f70e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.963+0000 7f70fea00640 1 --2- 172.21.15.119:0/2480014431 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70e006b4d0 0x7f70e006d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f70f0034000 tx=0x7f70f002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:18.966+0000 7f70f6a00640 1 -- 172.21.15.119:0/2480014431 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70ec058fa0 con 0x7f71000b46a0 2024-04-25T10:07:19.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.164+0000 7f7105600640 1 -- 172.21.15.119:0/2480014431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7100159d30 con 0x7f71000b46a0 2024-04-25T10:07:19.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.165+0000 7f70f6a00640 1 -- 172.21.15.119:0/2480014431 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f70ec0588a0 con 0x7f71000b46a0 2024-04-25T10:07:19.170 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:07:19.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.167+0000 7f7105600640 1 -- 172.21.15.119:0/2480014431 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70e006b4d0 msgr2=0x7f70e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.167+0000 7f7105600640 1 --2- 172.21.15.119:0/2480014431 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70e006b4d0 0x7f70e006d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f70f0034000 tx=0x7f70f002c000 comp rx=0 tx=0).stop 2024-04-25T10:07:19.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.167+0000 7f7105600640 1 -- 172.21.15.119:0/2480014431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71000b46a0 msgr2=0x7f71000cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.167+0000 7f7105600640 1 --2- 172.21.15.119:0/2480014431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71000b46a0 0x7f71000cb320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00c820 tx=0x7f70ec00cce0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.167+0000 7f7105600640 1 -- 172.21.15.119:0/2480014431 shutdown_connections 2024-04-25T10:07:19.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.167+0000 7f7105600640 1 --2- 172.21.15.119:0/2480014431 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70e006b4d0 0x7f70e006d990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.167+0000 7f7105600640 1 --2- 172.21.15.119:0/2480014431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f710015aef0 0x7f710015cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.167+0000 7f7105600640 1 --2- 172.21.15.119:0/2480014431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71000b46a0 0x7f71000cb320 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.167+0000 7f7105600640 1 --2- 172.21.15.119:0/2480014431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71000b42c0 0x7f710014e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.168+0000 7f7105600640 1 -- 172.21.15.119:0/2480014431 >> 172.21.15.119:0/2480014431 conn(0x7f710006aba0 msgr2=0x7f710006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.168+0000 7f7105600640 1 -- 172.21.15.119:0/2480014431 shutdown_connections 2024-04-25T10:07:19.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.168+0000 7f7105600640 1 -- 172.21.15.119:0/2480014431 wait complete. 2024-04-25T10:07:19.191 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:19.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.376+0000 7f8108e00640 1 Processor -- start 2024-04-25T10:07:19.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.376+0000 7f8108e00640 1 -- start start 2024-04-25T10:07:19.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.377+0000 7f8108e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f810415dbd0 0x7f810406c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.377+0000 7f8108e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8104066e10 0x7f81040cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.377+0000 7f8108e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f810415efc0 con 0x7f810406aba0 2024-04-25T10:07:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.378+0000 7f8108e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810406b0a0 con 0x7f810415dbd0 2024-04-25T10:07:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.378+0000 7f8108e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810406b380 con 0x7f8104066e10 2024-04-25T10:07:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.378+0000 7f8101e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f810415dbd0 0x7f810406c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.378+0000 7f8101e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f810415dbd0 0x7f810406c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35000/0 (socket says 172.21.15.119:35000) 2024-04-25T10:07:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.378+0000 7f8101e00640 1 -- 172.21.15.119:0/1831379970 learned_addr learned my addr 172.21.15.119:0/1831379970 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.378+0000 7f8101e00640 1 -- 172.21.15.119:0/1831379970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8104066e10 msgr2=0x7f81040cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.378+0000 7f8101e00640 1 --2- 172.21.15.119:0/1831379970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8104066e10 0x7f81040cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.378+0000 7f8101e00640 1 -- 172.21.15.119:0/1831379970 >> v1:172.21.15.132:6789/0 conn(0x7f810406aba0 legacy=0x7f810406dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.378+0000 7f8101e00640 1 -- 172.21.15.119:0/1831379970 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810406b660 con 0x7f810415dbd0 2024-04-25T10:07:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.379+0000 7f8101e00640 1 --2- 172.21.15.119:0/1831379970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f810415dbd0 0x7f810406c140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009870 tx=0x7f80f402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869da3de146e4dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.379+0000 7f8100a00640 1 -- 172.21.15.119:0/1831379970 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f4037070 con 0x7f810415dbd0 2024-04-25T10:07:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.379+0000 7f8100a00640 1 -- 172.21.15.119:0/1831379970 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f4032440 con 0x7f810415dbd0 2024-04-25T10:07:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.379+0000 7f8100a00640 1 -- 172.21.15.119:0/1831379970 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f4032810 con 0x7f810415dbd0 2024-04-25T10:07:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.380+0000 7f8108e00640 1 -- 172.21.15.119:0/1831379970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f810415dbd0 msgr2=0x7f810406c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.380+0000 7f8108e00640 1 --2- 172.21.15.119:0/1831379970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f810415dbd0 0x7f810406c140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009870 tx=0x7f80f402a200 comp rx=0 tx=0).stop 2024-04-25T10:07:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.380+0000 7f8108e00640 1 -- 172.21.15.119:0/1831379970 shutdown_connections 2024-04-25T10:07:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.380+0000 7f8108e00640 1 --2- 172.21.15.119:0/1831379970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8104066e10 0x7f81040cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.380+0000 7f8108e00640 1 --2- 172.21.15.119:0/1831379970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f810415dbd0 0x7f810406c140 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.380+0000 7f8108e00640 1 -- 172.21.15.119:0/1831379970 >> 172.21.15.119:0/1831379970 conn(0x7f81040cca00 msgr2=0x7f81040cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.380+0000 7f8108e00640 1 -- 172.21.15.119:0/1831379970 shutdown_connections 2024-04-25T10:07:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.380+0000 7f8108e00640 1 -- 172.21.15.119:0/1831379970 wait complete. 2024-04-25T10:07:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.382+0000 7f8108e00640 1 Processor -- start 2024-04-25T10:07:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.382+0000 7f8108e00640 1 -- start start 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.383+0000 7f8108e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8104066e10 0x7f81041f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.383+0000 7f8108e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f810406aba0 0x7f8104159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.383+0000 7f8108e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f810415dbd0 0x7f8104160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.383+0000 7f8108e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8104161720 con 0x7f810415dbd0 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.383+0000 7f8108e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810406bc20 con 0x7f810406aba0 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.383+0000 7f8108e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81041618a0 con 0x7f8104066e10 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.383+0000 7f8102800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f810415dbd0 0x7f8104160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.383+0000 7f8102800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f810415dbd0 0x7f8104160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35012/0 (socket says 172.21.15.119:35012) 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.383+0000 7f8102800640 1 -- 172.21.15.119:0/2204530095 learned_addr learned my addr 172.21.15.119:0/2204530095 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.383+0000 7f8101e00640 1 --2- 172.21.15.119:0/2204530095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8104066e10 0x7f81041f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.383+0000 7f8101400640 1 --2- 172.21.15.119:0/2204530095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f810406aba0 0x7f8104159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.383+0000 7f8102800640 1 -- 172.21.15.119:0/2204530095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8104066e10 msgr2=0x7f81041f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.383+0000 7f8102800640 1 --2- 172.21.15.119:0/2204530095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8104066e10 0x7f81041f7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.384+0000 7f8102800640 1 -- 172.21.15.119:0/2204530095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f810406aba0 msgr2=0x7f8104159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.384+0000 7f8102800640 1 --2- 172.21.15.119:0/2204530095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f810406aba0 0x7f8104159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.384+0000 7f8102800640 1 -- 172.21.15.119:0/2204530095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81041f81a0 con 0x7f810415dbd0 2024-04-25T10:07:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.384+0000 7f8102800640 1 --2- 172.21.15.119:0/2204530095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f810415dbd0 0x7f8104160590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00b990 tx=0x7f80fc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.384+0000 7f80fa000640 1 -- 172.21.15.119:0/2204530095 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc019070 con 0x7f810415dbd0 2024-04-25T10:07:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.384+0000 7f80fa000640 1 -- 172.21.15.119:0/2204530095 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80fc00cba0 con 0x7f810415dbd0 2024-04-25T10:07:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.384+0000 7f80fa000640 1 -- 172.21.15.119:0/2204530095 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc014440 con 0x7f810415dbd0 2024-04-25T10:07:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.384+0000 7f8108e00640 1 -- 172.21.15.119:0/2204530095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81040c7050 con 0x7f810415dbd0 2024-04-25T10:07:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.384+0000 7f8108e00640 1 -- 172.21.15.119:0/2204530095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81040c75d0 con 0x7f810415dbd0 2024-04-25T10:07:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.385+0000 7f8108e00640 1 -- 172.21.15.119:0/2204530095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81040750b0 con 0x7f810415dbd0 2024-04-25T10:07:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.386+0000 7f80fa000640 1 -- 172.21.15.119:0/2204530095 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80fc01c070 con 0x7f810415dbd0 2024-04-25T10:07:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.387+0000 7f80fa000640 1 --2- 172.21.15.119:0/2204530095 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f80e406b4d0 0x7f80e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.387+0000 7f80fa000640 1 -- 172.21.15.119:0/2204530095 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80fc089f90 con 0x7f810415dbd0 2024-04-25T10:07:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.387+0000 7f8101e00640 1 --2- 172.21.15.119:0/2204530095 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f80e406b4d0 0x7f80e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.388+0000 7f8101e00640 1 --2- 172.21.15.119:0/2204530095 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f80e406b4d0 0x7f80e406d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f80f4034000 tx=0x7f80f402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.391+0000 7f80fa000640 1 -- 172.21.15.119:0/2204530095 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81041f81a0 con 0x7f810415dbd0 2024-04-25T10:07:19.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.574+0000 7f8108e00640 1 -- 172.21.15.119:0/2204530095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f81040b00e0 con 0x7f810415dbd0 2024-04-25T10:07:19.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.575+0000 7f80fa000640 1 -- 172.21.15.119:0/2204530095 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80fc058e00 con 0x7f810415dbd0 2024-04-25T10:07:19.580 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:07:19.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.577+0000 7f8108e00640 1 -- 172.21.15.119:0/2204530095 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f80e406b4d0 msgr2=0x7f80e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.577+0000 7f8108e00640 1 --2- 172.21.15.119:0/2204530095 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f80e406b4d0 0x7f80e406d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f80f4034000 tx=0x7f80f402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:19.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.577+0000 7f8108e00640 1 -- 172.21.15.119:0/2204530095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f810415dbd0 msgr2=0x7f8104160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.577+0000 7f8108e00640 1 --2- 172.21.15.119:0/2204530095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f810415dbd0 0x7f8104160590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00b990 tx=0x7f80fc00be50 comp rx=0 tx=0).stop 2024-04-25T10:07:19.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.577+0000 7f8108e00640 1 -- 172.21.15.119:0/2204530095 shutdown_connections 2024-04-25T10:07:19.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.577+0000 7f8108e00640 1 --2- 172.21.15.119:0/2204530095 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f80e406b4d0 0x7f80e406d990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.577+0000 7f8108e00640 1 --2- 172.21.15.119:0/2204530095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f810415dbd0 0x7f8104160590 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.577+0000 7f8108e00640 1 --2- 172.21.15.119:0/2204530095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f810406aba0 0x7f8104159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.577+0000 7f8108e00640 1 --2- 172.21.15.119:0/2204530095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8104066e10 0x7f81041f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.577+0000 7f8108e00640 1 -- 172.21.15.119:0/2204530095 >> 172.21.15.119:0/2204530095 conn(0x7f81040cca00 msgr2=0x7f81040cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.577+0000 7f8108e00640 1 -- 172.21.15.119:0/2204530095 shutdown_connections 2024-04-25T10:07:19.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.577+0000 7f8108e00640 1 -- 172.21.15.119:0/2204530095 wait complete. 2024-04-25T10:07:19.600 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:19.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.781+0000 7f8fcee00640 1 Processor -- start 2024-04-25T10:07:19.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.781+0000 7f8fcee00640 1 -- start start 2024-04-25T10:07:19.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fcee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc815dc80 0x7f8fc8161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fcee00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8fc80b97c0 0x7f8fc80b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fcee00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8fc80b5a30 0x7f8fc806bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fcee00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc80b5ea0 con 0x7f8fc80b5a30 2024-04-25T10:07:19.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fcee00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc81627a0 con 0x7f8fc80b97c0 2024-04-25T10:07:19.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fcee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc81621e0 con 0x7f8fc815dc80 2024-04-25T10:07:19.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fcce00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8fc80b5a30 0x7f8fc806bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fcce00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8fc80b5a30 0x7f8fc806bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35018/0 (socket says 172.21.15.119:35018) 2024-04-25T10:07:19.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fcce00640 1 -- 172.21.15.119:0/3709758879 learned_addr learned my addr 172.21.15.119:0/3709758879 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:19.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fc7e00640 1 --2- 172.21.15.119:0/3709758879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc815dc80 0x7f8fc8161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fc7400640 1 --2- 172.21.15.119:0/3709758879 >> v2:172.21.15.132:3300/0 conn(0x7f8fc80b97c0 0x7f8fc80b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fcce00640 1 -- 172.21.15.119:0/3709758879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc815dc80 msgr2=0x7f8fc8161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fcce00640 1 --2- 172.21.15.119:0/3709758879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc815dc80 0x7f8fc8161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fcce00640 1 -- 172.21.15.119:0/3709758879 >> v2:172.21.15.132:3300/0 conn(0x7f8fc80b97c0 msgr2=0x7f8fc80b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fcce00640 1 --2- 172.21.15.119:0/3709758879 >> v2:172.21.15.132:3300/0 conn(0x7f8fc80b97c0 0x7f8fc80b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.783+0000 7f8fcce00640 1 -- 172.21.15.119:0/3709758879 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc81624c0 con 0x7f8fc80b5a30 2024-04-25T10:07:19.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.784+0000 7f8fcce00640 1 --2- 172.21.15.119:0/3709758879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc80b5a30 0x7f8fc806bff0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8fc806d9c0 tx=0x7f8fc002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f6d1ad30a72d85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.785+0000 7f8fc6a00640 1 -- 172.21.15.119:0/3709758879 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0037070 con 0x7f8fc80b5a30 2024-04-25T10:07:19.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.785+0000 7f8fc6a00640 1 -- 172.21.15.119:0/3709758879 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc0032440 con 0x7f8fc80b5a30 2024-04-25T10:07:19.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.785+0000 7f8fc6a00640 1 -- 172.21.15.119:0/3709758879 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0032810 con 0x7f8fc80b5a30 2024-04-25T10:07:19.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.785+0000 7f8fcee00640 1 -- 172.21.15.119:0/3709758879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc80b5a30 msgr2=0x7f8fc806bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.785+0000 7f8fcee00640 1 --2- 172.21.15.119:0/3709758879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc80b5a30 0x7f8fc806bff0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8fc806d9c0 tx=0x7f8fc002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.785+0000 7f8fcee00640 1 -- 172.21.15.119:0/3709758879 shutdown_connections 2024-04-25T10:07:19.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.785+0000 7f8fcee00640 1 --2- 172.21.15.119:0/3709758879 >> v2:172.21.15.132:3300/0 conn(0x7f8fc80b97c0 0x7f8fc80b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.785+0000 7f8fcee00640 1 --2- 172.21.15.119:0/3709758879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc80b5a30 0x7f8fc806bff0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.785+0000 7f8fcee00640 1 --2- 172.21.15.119:0/3709758879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc815dc80 0x7f8fc8161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.785+0000 7f8fcee00640 1 -- 172.21.15.119:0/3709758879 >> 172.21.15.119:0/3709758879 conn(0x7f8fc80d04f0 msgr2=0x7f8fc80d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.785+0000 7f8fcee00640 1 -- 172.21.15.119:0/3709758879 shutdown_connections 2024-04-25T10:07:19.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.785+0000 7f8fcee00640 1 -- 172.21.15.119:0/3709758879 wait complete. 2024-04-25T10:07:19.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.787+0000 7f8fcee00640 1 Processor -- start 2024-04-25T10:07:19.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.787+0000 7f8fcee00640 1 -- start start 2024-04-25T10:07:19.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.788+0000 7f8fcee00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc80b5a30 0x7f8fc80b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.788+0000 7f8fcee00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc80b97c0 0x7f8fc80b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.788+0000 7f8fcee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc815dc80 0x7f8fc814c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.788+0000 7f8fcee00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc8161db0 con 0x7f8fc80b97c0 2024-04-25T10:07:19.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.788+0000 7f8fcee00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc815e060 con 0x7f8fc80b5a30 2024-04-25T10:07:19.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.788+0000 7f8fcee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc8161f30 con 0x7f8fc815dc80 2024-04-25T10:07:19.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.788+0000 7f8fc7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc80b97c0 0x7f8fc80b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.788+0000 7f8fc7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc80b97c0 0x7f8fc80b7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35030/0 (socket says 172.21.15.119:35030) 2024-04-25T10:07:19.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.788+0000 7f8fc7400640 1 -- 172.21.15.119:0/1596531752 learned_addr learned my addr 172.21.15.119:0/1596531752 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:19.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.789+0000 7f8fc7e00640 1 --2- 172.21.15.119:0/1596531752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc80b5a30 0x7f8fc80b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.789+0000 7f8fcce00640 1 --2- 172.21.15.119:0/1596531752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc815dc80 0x7f8fc814c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.789+0000 7f8fc7400640 1 -- 172.21.15.119:0/1596531752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc815dc80 msgr2=0x7f8fc814c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.789+0000 7f8fc7400640 1 --2- 172.21.15.119:0/1596531752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc815dc80 0x7f8fc814c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.789+0000 7f8fc7400640 1 -- 172.21.15.119:0/1596531752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc80b5a30 msgr2=0x7f8fc80b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.789+0000 7f8fc7400640 1 --2- 172.21.15.119:0/1596531752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc80b5a30 0x7f8fc80b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.789+0000 7f8fc7400640 1 -- 172.21.15.119:0/1596531752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc80cd780 con 0x7f8fc80b97c0 2024-04-25T10:07:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.789+0000 7f8fc7400640 1 --2- 172.21.15.119:0/1596531752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc80b97c0 0x7f8fc80b7da0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000c820 tx=0x7f8fb000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.789+0000 7f8fbfe00640 1 -- 172.21.15.119:0/1596531752 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0015070 con 0x7f8fc80b97c0 2024-04-25T10:07:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.789+0000 7f8fbfe00640 1 -- 172.21.15.119:0/1596531752 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0004020 con 0x7f8fc80b97c0 2024-04-25T10:07:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.789+0000 7f8fbfe00640 1 -- 172.21.15.119:0/1596531752 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0004410 con 0x7f8fc80b97c0 2024-04-25T10:07:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.789+0000 7f8fcee00640 1 -- 172.21.15.119:0/1596531752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc80ca1b0 con 0x7f8fc80b97c0 2024-04-25T10:07:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.790+0000 7f8fcee00640 1 -- 172.21.15.119:0/1596531752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc814a730 con 0x7f8fc80b97c0 2024-04-25T10:07:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.791+0000 7f8fcee00640 1 -- 172.21.15.119:0/1596531752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc80750b0 con 0x7f8fc80b97c0 2024-04-25T10:07:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.792+0000 7f8fbfe00640 1 -- 172.21.15.119:0/1596531752 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fb0007830 con 0x7f8fc80b97c0 2024-04-25T10:07:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.793+0000 7f8fbfe00640 1 --2- 172.21.15.119:0/1596531752 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fa406b4d0 0x7f8fa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.793+0000 7f8fbfe00640 1 -- 172.21.15.119:0/1596531752 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fb008a0a0 con 0x7f8fc80b97c0 2024-04-25T10:07:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.793+0000 7f8fc7e00640 1 --2- 172.21.15.119:0/1596531752 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fa406b4d0 0x7f8fa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.794+0000 7f8fc7e00640 1 --2- 172.21.15.119:0/1596531752 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fa406b4d0 0x7f8fa406d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80ccfa0 tx=0x7f8fb8007900 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.797+0000 7f8fbfe00640 1 -- 172.21.15.119:0/1596531752 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8fc80cd780 con 0x7f8fc80b97c0 2024-04-25T10:07:19.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.992+0000 7f8fcee00640 1 -- 172.21.15.119:0/1596531752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8fc8067c70 con 0x7f8fc80b97c0 2024-04-25T10:07:19.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.993+0000 7f8fbfe00640 1 -- 172.21.15.119:0/1596531752 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8fb0058e80 con 0x7f8fc80b97c0 2024-04-25T10:07:19.998 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:07:20.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.995+0000 7f8fcee00640 1 -- 172.21.15.119:0/1596531752 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fa406b4d0 msgr2=0x7f8fa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.995+0000 7f8fcee00640 1 --2- 172.21.15.119:0/1596531752 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fa406b4d0 0x7f8fa406d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80ccfa0 tx=0x7f8fb8007900 comp rx=0 tx=0).stop 2024-04-25T10:07:20.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.995+0000 7f8fcee00640 1 -- 172.21.15.119:0/1596531752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc80b97c0 msgr2=0x7f8fc80b7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.995+0000 7f8fcee00640 1 --2- 172.21.15.119:0/1596531752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc80b97c0 0x7f8fc80b7da0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000c820 tx=0x7f8fb000cce0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.995+0000 7f8fcee00640 1 -- 172.21.15.119:0/1596531752 shutdown_connections 2024-04-25T10:07:20.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.995+0000 7f8fcee00640 1 --2- 172.21.15.119:0/1596531752 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fa406b4d0 0x7f8fa406d990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.995+0000 7f8fcee00640 1 --2- 172.21.15.119:0/1596531752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc815dc80 0x7f8fc814c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.995+0000 7f8fcee00640 1 --2- 172.21.15.119:0/1596531752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc80b97c0 0x7f8fc80b7da0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.995+0000 7f8fcee00640 1 --2- 172.21.15.119:0/1596531752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc80b5a30 0x7f8fc80b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.995+0000 7f8fcee00640 1 -- 172.21.15.119:0/1596531752 >> 172.21.15.119:0/1596531752 conn(0x7f8fc80d04f0 msgr2=0x7f8fc80d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.995+0000 7f8fcee00640 1 -- 172.21.15.119:0/1596531752 shutdown_connections 2024-04-25T10:07:20.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:19.995+0000 7f8fcee00640 1 -- 172.21.15.119:0/1596531752 wait complete. 2024-04-25T10:07:20.021 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:20.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.210+0000 7fe525400640 1 Processor -- start 2024-04-25T10:07:20.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.210+0000 7fe525400640 1 -- start start 2024-04-25T10:07:20.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.211+0000 7fe525400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe52015db60 0x7fe5200d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.212+0000 7fe525400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe5200d2fa0 0x7fe5200d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.212+0000 7fe525400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe52015ef10 con 0x7fe5200b9270 2024-04-25T10:07:20.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.212+0000 7fe525400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe52015e390 con 0x7fe5200d2fa0 2024-04-25T10:07:20.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.212+0000 7fe525400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52015e670 con 0x7fe52015db60 2024-04-25T10:07:20.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.212+0000 7fe524a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe5200d2fa0 0x7fe5200d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.212+0000 7fe524a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe5200d2fa0 0x7fe5200d3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35038/0 (socket says 172.21.15.119:35038) 2024-04-25T10:07:20.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.212+0000 7fe524a00640 1 -- 172.21.15.119:0/2534373793 learned_addr learned my addr 172.21.15.119:0/2534373793 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:20.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.212+0000 7fe51fe00640 1 --2- 172.21.15.119:0/2534373793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe52015db60 0x7fe5200d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.212+0000 7fe524a00640 1 -- 172.21.15.119:0/2534373793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe52015db60 msgr2=0x7fe5200d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.212+0000 7fe524a00640 1 --2- 172.21.15.119:0/2534373793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe52015db60 0x7fe5200d3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.212+0000 7fe524a00640 1 -- 172.21.15.119:0/2534373793 >> v1:172.21.15.132:6789/0 conn(0x7fe5200b9270 legacy=0x7fe5200b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.212+0000 7fe524a00640 1 -- 172.21.15.119:0/2534373793 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52015e950 con 0x7fe5200d2fa0 2024-04-25T10:07:20.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.213+0000 7fe524a00640 1 --2- 172.21.15.119:0/2534373793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5200d2fa0 0x7fe5200d3db0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe518009e50 tx=0x7fe51802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87aae0ae28605300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.214+0000 7fe51f400640 1 -- 172.21.15.119:0/2534373793 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518036070 con 0x7fe5200d2fa0 2024-04-25T10:07:20.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.214+0000 7fe51f400640 1 -- 172.21.15.119:0/2534373793 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe518031600 con 0x7fe5200d2fa0 2024-04-25T10:07:20.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.214+0000 7fe51f400640 1 -- 172.21.15.119:0/2534373793 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5180319d0 con 0x7fe5200d2fa0 2024-04-25T10:07:20.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.214+0000 7fe525400640 1 -- 172.21.15.119:0/2534373793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5200d2fa0 msgr2=0x7fe5200d3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.214+0000 7fe525400640 1 --2- 172.21.15.119:0/2534373793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5200d2fa0 0x7fe5200d3db0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe518009e50 tx=0x7fe51802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.214+0000 7fe525400640 1 -- 172.21.15.119:0/2534373793 shutdown_connections 2024-04-25T10:07:20.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.214+0000 7fe525400640 1 --2- 172.21.15.119:0/2534373793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5200d2fa0 0x7fe5200d3db0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.214+0000 7fe525400640 1 --2- 172.21.15.119:0/2534373793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe52015db60 0x7fe5200d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.214+0000 7fe525400640 1 -- 172.21.15.119:0/2534373793 >> 172.21.15.119:0/2534373793 conn(0x7fe5200726c0 msgr2=0x7fe520072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.214+0000 7fe525400640 1 -- 172.21.15.119:0/2534373793 shutdown_connections 2024-04-25T10:07:20.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.214+0000 7fe525400640 1 -- 172.21.15.119:0/2534373793 wait complete. 2024-04-25T10:07:20.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.216+0000 7fe525400640 1 Processor -- start 2024-04-25T10:07:20.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.216+0000 7fe525400640 1 -- start start 2024-04-25T10:07:20.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.217+0000 7fe525400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5200b9270 0x7fe5201493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.217+0000 7fe525400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5200d2fa0 0x7fe5201e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.217+0000 7fe525400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe52015db60 0x7fe5201e7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.217+0000 7fe525400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200b3120 con 0x7fe5200b9270 2024-04-25T10:07:20.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.217+0000 7fe525400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5200b2fa0 con 0x7fe5200d2fa0 2024-04-25T10:07:20.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.217+0000 7fe525400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52015df40 con 0x7fe52015db60 2024-04-25T10:07:20.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.217+0000 7fe51fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5200b9270 0x7fe5201493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.217+0000 7fe51fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5200b9270 0x7fe5201493f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35050/0 (socket says 172.21.15.119:35050) 2024-04-25T10:07:20.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.217+0000 7fe51fe00640 1 -- 172.21.15.119:0/4104276774 learned_addr learned my addr 172.21.15.119:0/4104276774 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:20.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.218+0000 7fe51fe00640 1 -- 172.21.15.119:0/4104276774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe52015db60 msgr2=0x7fe5201e7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.218+0000 7fe51fe00640 1 --2- 172.21.15.119:0/4104276774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe52015db60 0x7fe5201e7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.218+0000 7fe51fe00640 1 -- 172.21.15.119:0/4104276774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5200d2fa0 msgr2=0x7fe5201e6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.218+0000 7fe51fe00640 1 --2- 172.21.15.119:0/4104276774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5200d2fa0 0x7fe5201e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.218+0000 7fe51fe00640 1 -- 172.21.15.119:0/4104276774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52014a050 con 0x7fe5200b9270 2024-04-25T10:07:20.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.218+0000 7fe51fe00640 1 --2- 172.21.15.119:0/4104276774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5200b9270 0x7fe5201493f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe51000c7f0 tx=0x7fe51000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.218+0000 7fe51cc00640 1 -- 172.21.15.119:0/4104276774 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510015070 con 0x7fe5200b9270 2024-04-25T10:07:20.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.218+0000 7fe51cc00640 1 -- 172.21.15.119:0/4104276774 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe510004220 con 0x7fe5200b9270 2024-04-25T10:07:20.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.218+0000 7fe51cc00640 1 -- 172.21.15.119:0/4104276774 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510007620 con 0x7fe5200b9270 2024-04-25T10:07:20.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.218+0000 7fe525400640 1 -- 172.21.15.119:0/4104276774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe52014a330 con 0x7fe5200b9270 2024-04-25T10:07:20.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.218+0000 7fe525400640 1 -- 172.21.15.119:0/4104276774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5201e4150 con 0x7fe5200b9270 2024-04-25T10:07:20.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.220+0000 7fe525400640 1 -- 172.21.15.119:0/4104276774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5200749b0 con 0x7fe5200b9270 2024-04-25T10:07:20.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.220+0000 7fe51cc00640 1 -- 172.21.15.119:0/4104276774 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5100079d0 con 0x7fe5200b9270 2024-04-25T10:07:20.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.221+0000 7fe51cc00640 1 --2- 172.21.15.119:0/4104276774 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe4fc06b500 0x7fe4fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.221+0000 7fe51cc00640 1 -- 172.21.15.119:0/4104276774 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe51008a290 con 0x7fe5200b9270 2024-04-25T10:07:20.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.221+0000 7fe517e00640 1 --2- 172.21.15.119:0/4104276774 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe4fc06b500 0x7fe4fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.222+0000 7fe517e00640 1 --2- 172.21.15.119:0/4104276774 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe4fc06b500 0x7fe4fc06d9c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe5200b2680 tx=0x7fe508007a00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.225+0000 7fe51cc00640 1 -- 172.21.15.119:0/4104276774 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe510059070 con 0x7fe5200b9270 2024-04-25T10:07:20.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.438+0000 7fe525400640 1 -- 172.21.15.119:0/4104276774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5200d1a60 con 0x7fe5200b9270 2024-04-25T10:07:20.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.439+0000 7fe51cc00640 1 -- 172.21.15.119:0/4104276774 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4941 (secure 0 0 0) 0x7fe510058970 con 0x7fe5200b9270 2024-04-25T10:07:20.728 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:20.728 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1385207078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1385207078},{"type":"v1","addr":"172.21.15.132:6834","nonce":1385207078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1580923823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1580923823},{"type":"v1","addr":"172.21.15.119:6836","nonce":1580923823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4260,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/954053568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":954053568},{"type":"v1","addr":"172.21.15.132:6835","nonce":954053568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:06.743184+0000","modified":"2024-04-25T10:07:10.118630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1632024048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1632024048},{"type":"v1","addr":"172.21.15.119:6837","nonce":1632024048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:07:20.731 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2024-04-25T10:07:20.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.441+0000 7fe525400640 1 -- 172.21.15.119:0/4104276774 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe4fc06b500 msgr2=0x7fe4fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.441+0000 7fe525400640 1 --2- 172.21.15.119:0/4104276774 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe4fc06b500 0x7fe4fc06d9c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe5200b2680 tx=0x7fe508007a00 comp rx=0 tx=0).stop 2024-04-25T10:07:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.441+0000 7fe525400640 1 -- 172.21.15.119:0/4104276774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5200b9270 msgr2=0x7fe5201493f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.441+0000 7fe525400640 1 --2- 172.21.15.119:0/4104276774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5200b9270 0x7fe5201493f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe51000c7f0 tx=0x7fe51000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.441+0000 7fe525400640 1 -- 172.21.15.119:0/4104276774 shutdown_connections 2024-04-25T10:07:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.441+0000 7fe525400640 1 --2- 172.21.15.119:0/4104276774 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe4fc06b500 0x7fe4fc06d9c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.441+0000 7fe525400640 1 --2- 172.21.15.119:0/4104276774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe52015db60 0x7fe5201e7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.442+0000 7fe525400640 1 --2- 172.21.15.119:0/4104276774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5200d2fa0 0x7fe5201e6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.442+0000 7fe525400640 1 --2- 172.21.15.119:0/4104276774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5200b9270 0x7fe5201493f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.442+0000 7fe525400640 1 -- 172.21.15.119:0/4104276774 >> 172.21.15.119:0/4104276774 conn(0x7fe5200726c0 msgr2=0x7fe5200cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.442+0000 7fe525400640 1 -- 172.21.15.119:0/4104276774 shutdown_connections 2024-04-25T10:07:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.442+0000 7fe525400640 1 -- 172.21.15.119:0/4104276774 wait complete. 2024-04-25T10:07:20.734 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:20.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.934+0000 7f3802a00640 1 Processor -- start 2024-04-25T10:07:20.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.934+0000 7f3802a00640 1 -- start start 2024-04-25T10:07:20.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.935+0000 7f3802a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37fc144470 0x7f37fc15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3802a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f37fc06e430 0x7f37fc0b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3802a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37fc0b4e80 0x7f37fc06d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3802a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37fc145340 con 0x7f37fc144470 2024-04-25T10:07:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3802a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37fc0cd440 con 0x7f37fc06e430 2024-04-25T10:07:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3802a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc0cd720 con 0x7f37fc0b4e80 2024-04-25T10:07:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3801600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37fc144470 0x7f37fc15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3801600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37fc144470 0x7f37fc15fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53280/0 (socket says 172.21.15.119:53280) 2024-04-25T10:07:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3801600640 1 -- 172.21.15.119:0/3862218698 learned_addr learned my addr 172.21.15.119:0/3862218698 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3800c00640 1 --2- 172.21.15.119:0/3862218698 >> v2:172.21.15.132:3300/0 conn(0x7f37fc06e430 0x7f37fc0b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3802000640 1 --2- 172.21.15.119:0/3862218698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37fc0b4e80 0x7f37fc06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3801600640 1 -- 172.21.15.119:0/3862218698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37fc0b4e80 msgr2=0x7f37fc06d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3801600640 1 --2- 172.21.15.119:0/3862218698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37fc0b4e80 0x7f37fc06d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3801600640 1 -- 172.21.15.119:0/3862218698 >> v2:172.21.15.132:3300/0 conn(0x7f37fc06e430 msgr2=0x7f37fc0b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3801600640 1 --2- 172.21.15.119:0/3862218698 >> v2:172.21.15.132:3300/0 conn(0x7f37fc06e430 0x7f37fc0b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3801600640 1 -- 172.21.15.119:0/3862218698 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc0cda00 con 0x7f37fc144470 2024-04-25T10:07:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.936+0000 7f3802000640 1 --2- 172.21.15.119:0/3862218698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37fc0b4e80 0x7f37fc06d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.937+0000 7f3801600640 1 --2- 172.21.15.119:0/3862218698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc144470 0x7f37fc15fc90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f37fc06f500 tx=0x7f37f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0fc4baee3d5d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.938+0000 7f37fbe00640 1 -- 172.21.15.119:0/3862218698 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0037070 con 0x7f37fc144470 2024-04-25T10:07:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.938+0000 7f37fbe00640 1 -- 172.21.15.119:0/3862218698 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f0032440 con 0x7f37fc144470 2024-04-25T10:07:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.938+0000 7f37fbe00640 1 -- 172.21.15.119:0/3862218698 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0032810 con 0x7f37fc144470 2024-04-25T10:07:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.938+0000 7f3802a00640 1 -- 172.21.15.119:0/3862218698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc144470 msgr2=0x7f37fc15fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.938+0000 7f3802a00640 1 --2- 172.21.15.119:0/3862218698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc144470 0x7f37fc15fc90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f37fc06f500 tx=0x7f37f002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.938+0000 7f3802a00640 1 -- 172.21.15.119:0/3862218698 shutdown_connections 2024-04-25T10:07:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.938+0000 7f3802a00640 1 --2- 172.21.15.119:0/3862218698 >> v2:172.21.15.132:3300/0 conn(0x7f37fc06e430 0x7f37fc0b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.938+0000 7f3802a00640 1 --2- 172.21.15.119:0/3862218698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37fc0b4e80 0x7f37fc06d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.938+0000 7f3802a00640 1 --2- 172.21.15.119:0/3862218698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc144470 0x7f37fc15fc90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.938+0000 7f3802a00640 1 -- 172.21.15.119:0/3862218698 >> 172.21.15.119:0/3862218698 conn(0x7f37fc06b1d0 msgr2=0x7f37fc0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.938+0000 7f3802a00640 1 -- 172.21.15.119:0/3862218698 shutdown_connections 2024-04-25T10:07:20.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.938+0000 7f3802a00640 1 -- 172.21.15.119:0/3862218698 wait complete. 2024-04-25T10:07:20.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.940+0000 7f3802a00640 1 Processor -- start 2024-04-25T10:07:20.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.940+0000 7f3802a00640 1 -- start start 2024-04-25T10:07:20.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.941+0000 7f3802a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37fc06e430 0x7f37fc13c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.941+0000 7f3802a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc0b4e80 0x7f37fc1e8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.941+0000 7f3802a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37fc144470 0x7f37fc1e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.941+0000 7f3802a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc0ce200 con 0x7f37fc0b4e80 2024-04-25T10:07:20.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.941+0000 7f3802a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc144850 con 0x7f37fc06e430 2024-04-25T10:07:20.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.941+0000 7f3802a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc06d750 con 0x7f37fc144470 2024-04-25T10:07:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.941+0000 7f3800c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc0b4e80 0x7f37fc1e8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.941+0000 7f3800c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc0b4e80 0x7f37fc1e8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53284/0 (socket says 172.21.15.119:53284) 2024-04-25T10:07:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.941+0000 7f3800c00640 1 -- 172.21.15.119:0/908720098 learned_addr learned my addr 172.21.15.119:0/908720098 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.942+0000 7f3801600640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37fc06e430 0x7f37fc13c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.942+0000 7f3802000640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37fc144470 0x7f37fc1e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.942+0000 7f3800c00640 1 -- 172.21.15.119:0/908720098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37fc144470 msgr2=0x7f37fc1e6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.942+0000 7f3800c00640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37fc144470 0x7f37fc1e6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.942+0000 7f3800c00640 1 -- 172.21.15.119:0/908720098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37fc06e430 msgr2=0x7f37fc13c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.942+0000 7f3800c00640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37fc06e430 0x7f37fc13c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.942+0000 7f3800c00640 1 -- 172.21.15.119:0/908720098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc13cf20 con 0x7f37fc0b4e80 2024-04-25T10:07:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.942+0000 7f3801600640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37fc06e430 0x7f37fc13c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.942+0000 7f3800c00640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc0b4e80 0x7f37fc1e8330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37e800d7d0 tx=0x7f37e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.942+0000 7f37f9600640 1 -- 172.21.15.119:0/908720098 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8004020 con 0x7f37fc0b4e80 2024-04-25T10:07:20.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.942+0000 7f37f9600640 1 -- 172.21.15.119:0/908720098 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e80041e0 con 0x7f37fc0b4e80 2024-04-25T10:07:20.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.942+0000 7f37f9600640 1 -- 172.21.15.119:0/908720098 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e80043d0 con 0x7f37fc0b4e80 2024-04-25T10:07:20.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.942+0000 7f3802a00640 1 -- 172.21.15.119:0/908720098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37fc13d200 con 0x7f37fc0b4e80 2024-04-25T10:07:20.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.943+0000 7f3802a00640 1 -- 172.21.15.119:0/908720098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37fc13d7c0 con 0x7f37fc0b4e80 2024-04-25T10:07:20.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.943+0000 7f3802a00640 1 -- 172.21.15.119:0/908720098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37fc13dfc0 con 0x7f37fc0b4e80 2024-04-25T10:07:20.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.944+0000 7f37f9600640 1 -- 172.21.15.119:0/908720098 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37e801b020 con 0x7f37fc0b4e80 2024-04-25T10:07:20.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.945+0000 7f37f9600640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f37dc06b4d0 0x7f37dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.945+0000 7f37f9600640 1 -- 172.21.15.119:0/908720098 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37e8089fe0 con 0x7f37fc0b4e80 2024-04-25T10:07:20.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.945+0000 7f3801600640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f37dc06b4d0 0x7f37dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.946+0000 7f3801600640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f37dc06b4d0 0x7f37dc06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f37f0034000 tx=0x7f37f002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:20.950+0000 7f37f9600640 1 -- 172.21.15.119:0/908720098 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37e8058e90 con 0x7f37fc0b4e80 2024-04-25T10:07:21.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.134+0000 7f3802a00640 1 -- 172.21.15.119:0/908720098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37fc15b7a0 con 0x7f37fc0b4e80 2024-04-25T10:07:21.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.136+0000 7f37f9600640 1 -- 172.21.15.119:0/908720098 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f37e8058790 con 0x7f37fc0b4e80 2024-04-25T10:07:21.143 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:21.143 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:08.319104+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:21.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.139+0000 7f3802a00640 1 -- 172.21.15.119:0/908720098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f37dc06b4d0 msgr2=0x7f37dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.139+0000 7f3802a00640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f37dc06b4d0 0x7f37dc06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f37f0034000 tx=0x7f37f002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:21.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.139+0000 7f3802a00640 1 -- 172.21.15.119:0/908720098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc0b4e80 msgr2=0x7f37fc1e8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.139+0000 7f3802a00640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc0b4e80 0x7f37fc1e8330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37e800d7d0 tx=0x7f37e800dc90 comp rx=0 tx=0).stop 2024-04-25T10:07:21.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.139+0000 7f3802a00640 1 -- 172.21.15.119:0/908720098 shutdown_connections 2024-04-25T10:07:21.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.139+0000 7f3802a00640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f37dc06b4d0 0x7f37dc06d990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.139+0000 7f3802a00640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37fc144470 0x7f37fc1e6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.139+0000 7f3802a00640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc0b4e80 0x7f37fc1e8330 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.139+0000 7f3802a00640 1 --2- 172.21.15.119:0/908720098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37fc06e430 0x7f37fc13c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.139+0000 7f3802a00640 1 -- 172.21.15.119:0/908720098 >> 172.21.15.119:0/908720098 conn(0x7f37fc06b1d0 msgr2=0x7f37fc13f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.139+0000 7f3802a00640 1 -- 172.21.15.119:0/908720098 shutdown_connections 2024-04-25T10:07:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.139+0000 7f3802a00640 1 -- 172.21.15.119:0/908720098 wait complete. 2024-04-25T10:07:21.165 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:05.135832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:05.768083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:21.166 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:21.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.356+0000 7f718c200640 1 Processor -- start 2024-04-25T10:07:21.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.356+0000 7f718c200640 1 -- start start 2024-04-25T10:07:21.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.357+0000 7f718c200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7184144e70 0x7f718406d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.357+0000 7f718c200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71840cf0e0 0x7f71840cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.358+0000 7f718c200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718406dcb0 con 0x7f718406d7a0 2024-04-25T10:07:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.358+0000 7f718c200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718415bfe0 con 0x7f71840cf0e0 2024-04-25T10:07:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.358+0000 7f718c200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718406df90 con 0x7f7184144e70 2024-04-25T10:07:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.358+0000 7f718a200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71840cf0e0 0x7f71840cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.358+0000 7f718a200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71840cf0e0 0x7f71840cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53288/0 (socket says 172.21.15.119:53288) 2024-04-25T10:07:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.358+0000 7f718a200640 1 -- 172.21.15.119:0/3112768476 learned_addr learned my addr 172.21.15.119:0/3112768476 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:21.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.358+0000 7f7189800640 1 --2- 172.21.15.119:0/3112768476 >> v2:172.21.15.132:3300/0 conn(0x7f7184144e70 0x7f718406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.358+0000 7f718a200640 1 -- 172.21.15.119:0/3112768476 >> v2:172.21.15.132:3300/0 conn(0x7f7184144e70 msgr2=0x7f718406d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:21.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.358+0000 7f718a200640 1 --2- 172.21.15.119:0/3112768476 >> v2:172.21.15.132:3300/0 conn(0x7f7184144e70 0x7f718406d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.358+0000 7f718a200640 1 -- 172.21.15.119:0/3112768476 >> v1:172.21.15.119:6789/0 conn(0x7f718406d7a0 legacy=0x7f71840ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.358+0000 7f718a200640 1 -- 172.21.15.119:0/3112768476 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718406e270 con 0x7f71840cf0e0 2024-04-25T10:07:21.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.358+0000 7f7189800640 1 --2- 172.21.15.119:0/3112768476 >> v2:172.21.15.132:3300/0 conn(0x7f7184144e70 0x7f718406d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:21.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.359+0000 7f718a200640 1 --2- 172.21.15.119:0/3112768476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71840cf0e0 0x7f71840cc300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f71840b4a20 tx=0x7f717c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb883acf6bafc91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.360+0000 7f7183e00640 1 -- 172.21.15.119:0/3112768476 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c036070 con 0x7f71840cf0e0 2024-04-25T10:07:21.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.360+0000 7f7183e00640 1 -- 172.21.15.119:0/3112768476 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c031600 con 0x7f71840cf0e0 2024-04-25T10:07:21.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.360+0000 7f7183e00640 1 -- 172.21.15.119:0/3112768476 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c0319d0 con 0x7f71840cf0e0 2024-04-25T10:07:21.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.360+0000 7f718c200640 1 -- 172.21.15.119:0/3112768476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71840cf0e0 msgr2=0x7f71840cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.360+0000 7f718c200640 1 --2- 172.21.15.119:0/3112768476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71840cf0e0 0x7f71840cc300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f71840b4a20 tx=0x7f717c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.360+0000 7f718c200640 1 -- 172.21.15.119:0/3112768476 shutdown_connections 2024-04-25T10:07:21.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.360+0000 7f718c200640 1 --2- 172.21.15.119:0/3112768476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71840cf0e0 0x7f71840cc300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.360+0000 7f718c200640 1 --2- 172.21.15.119:0/3112768476 >> v2:172.21.15.132:3300/0 conn(0x7f7184144e70 0x7f718406d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.360+0000 7f718c200640 1 -- 172.21.15.119:0/3112768476 >> 172.21.15.119:0/3112768476 conn(0x7f718406aba0 msgr2=0x7f718406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.360+0000 7f718c200640 1 -- 172.21.15.119:0/3112768476 shutdown_connections 2024-04-25T10:07:21.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.361+0000 7f718c200640 1 -- 172.21.15.119:0/3112768476 wait complete. 2024-04-25T10:07:21.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.362+0000 7f718c200640 1 Processor -- start 2024-04-25T10:07:21.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.362+0000 7f718c200640 1 -- start start 2024-04-25T10:07:21.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.363+0000 7f718c200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f718406d7a0 0x7f718413f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.363+0000 7f718c200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71840cf0e0 0x7f71841565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.363+0000 7f718c200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184144e70 0x7f7184156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.363+0000 7f718c200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7184145630 con 0x7f718406d7a0 2024-04-25T10:07:21.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.363+0000 7f718c200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71841454b0 con 0x7f7184144e70 2024-04-25T10:07:21.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.363+0000 7f718c200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71840cc840 con 0x7f71840cf0e0 2024-04-25T10:07:21.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.364+0000 7f7189800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f718406d7a0 0x7f718413f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.364+0000 7f7189800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f718406d7a0 0x7f718413f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53294/0 (socket says 172.21.15.119:53294) 2024-04-25T10:07:21.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.364+0000 7f7189800640 1 -- 172.21.15.119:0/4160288508 learned_addr learned my addr 172.21.15.119:0/4160288508 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:21.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.364+0000 7f7188e00640 1 --2- 172.21.15.119:0/4160288508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71840cf0e0 0x7f71841565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.364+0000 7f7189800640 1 -- 172.21.15.119:0/4160288508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71840cf0e0 msgr2=0x7f71841565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.364+0000 7f7189800640 1 --2- 172.21.15.119:0/4160288508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71840cf0e0 0x7f71841565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.364+0000 7f7189800640 1 -- 172.21.15.119:0/4160288508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184144e70 msgr2=0x7f7184156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.364+0000 7f718a200640 1 --2- 172.21.15.119:0/4160288508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184144e70 0x7f7184156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.364+0000 7f7189800640 1 --2- 172.21.15.119:0/4160288508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184144e70 0x7f7184156c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.364+0000 7f7189800640 1 -- 172.21.15.119:0/4160288508 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71841571c0 con 0x7f718406d7a0 2024-04-25T10:07:21.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.364+0000 7f7189800640 1 --2- 172.21.15.119:0/4160288508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f718406d7a0 0x7f718413f980 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f717800c820 tx=0x7f717800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.364+0000 7f7181600640 1 -- 172.21.15.119:0/4160288508 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178015070 con 0x7f718406d7a0 2024-04-25T10:07:21.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.365+0000 7f7181600640 1 -- 172.21.15.119:0/4160288508 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7178007a30 con 0x7f718406d7a0 2024-04-25T10:07:21.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.365+0000 7f7181600640 1 -- 172.21.15.119:0/4160288508 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178007de0 con 0x7f718406d7a0 2024-04-25T10:07:21.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.365+0000 7f718c200640 1 -- 172.21.15.119:0/4160288508 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7184157460 con 0x7f718406d7a0 2024-04-25T10:07:21.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.365+0000 7f718c200640 1 -- 172.21.15.119:0/4160288508 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7184157a20 con 0x7f718406d7a0 2024-04-25T10:07:21.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.366+0000 7f718c200640 1 -- 172.21.15.119:0/4160288508 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71840750b0 con 0x7f718406d7a0 2024-04-25T10:07:21.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.367+0000 7f7181600640 1 -- 172.21.15.119:0/4160288508 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7178018070 con 0x7f718406d7a0 2024-04-25T10:07:21.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.368+0000 7f7181600640 1 --2- 172.21.15.119:0/4160288508 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f716406b4d0 0x7f716406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.368+0000 7f7181600640 1 -- 172.21.15.119:0/4160288508 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f717808a150 con 0x7f718406d7a0 2024-04-25T10:07:21.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.368+0000 7f7188e00640 1 --2- 172.21.15.119:0/4160288508 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f716406b4d0 0x7f716406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.368+0000 7f7188e00640 1 --2- 172.21.15.119:0/4160288508 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f716406b4d0 0x7f716406d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f71841440c0 tx=0x7f7170009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.372+0000 7f7181600640 1 -- 172.21.15.119:0/4160288508 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7178058f30 con 0x7f718406d7a0 2024-04-25T10:07:21.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.566+0000 7f718c200640 1 -- 172.21.15.119:0/4160288508 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f718415baa0 con 0x7f718406d7a0 2024-04-25T10:07:21.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.567+0000 7f7181600640 1 -- 172.21.15.119:0/4160288508 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7178058830 con 0x7f718406d7a0 2024-04-25T10:07:21.573 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:07:21.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.570+0000 7f718c200640 1 -- 172.21.15.119:0/4160288508 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f716406b4d0 msgr2=0x7f716406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.570+0000 7f718c200640 1 --2- 172.21.15.119:0/4160288508 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f716406b4d0 0x7f716406d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f71841440c0 tx=0x7f7170009040 comp rx=0 tx=0).stop 2024-04-25T10:07:21.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.570+0000 7f718c200640 1 -- 172.21.15.119:0/4160288508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f718406d7a0 msgr2=0x7f718413f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.570+0000 7f718c200640 1 --2- 172.21.15.119:0/4160288508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f718406d7a0 0x7f718413f980 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f717800c820 tx=0x7f717800cce0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.570+0000 7f718c200640 1 -- 172.21.15.119:0/4160288508 shutdown_connections 2024-04-25T10:07:21.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.570+0000 7f718c200640 1 --2- 172.21.15.119:0/4160288508 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f716406b4d0 0x7f716406d990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.570+0000 7f718c200640 1 --2- 172.21.15.119:0/4160288508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7184144e70 0x7f7184156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.570+0000 7f718c200640 1 --2- 172.21.15.119:0/4160288508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71840cf0e0 0x7f71841565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.570+0000 7f718c200640 1 --2- 172.21.15.119:0/4160288508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f718406d7a0 0x7f718413f980 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.570+0000 7f718c200640 1 -- 172.21.15.119:0/4160288508 >> 172.21.15.119:0/4160288508 conn(0x7f718406aba0 msgr2=0x7f71840cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.570+0000 7f718c200640 1 -- 172.21.15.119:0/4160288508 shutdown_connections 2024-04-25T10:07:21.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.570+0000 7f718c200640 1 -- 172.21.15.119:0/4160288508 wait complete. 2024-04-25T10:07:21.594 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:21.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.790+0000 7f087c800640 1 Processor -- start 2024-04-25T10:07:21.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.790+0000 7f087c800640 1 -- start start 2024-04-25T10:07:21.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.792+0000 7f087c800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0874144e70 0x7f087406d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.792+0000 7f087c800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087406dcb0 con 0x7f08740cf0e0 2024-04-25T10:07:21.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.792+0000 7f087c800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087406df90 con 0x7f087406d7a0 2024-04-25T10:07:21.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.792+0000 7f087c800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087415bfe0 con 0x7f0874144e70 2024-04-25T10:07:21.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.792+0000 7f087a800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f08740cf0e0 0x7f08740cc300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48690/0 (socket says 172.21.15.119:48690) 2024-04-25T10:07:21.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.792+0000 7f087a800640 1 -- 172.21.15.119:0/2530539555 learned_addr learned my addr 172.21.15.119:0/2530539555 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:21.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.792+0000 7f0879e00640 1 --2- 172.21.15.119:0/2530539555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0874144e70 0x7f087406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.793+0000 7f0878a00640 1 -- 172.21.15.119:0/2530539555 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769799317 0 0) 0x7f087406dcb0 con 0x7f08740cf0e0 2024-04-25T10:07:21.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.793+0000 7f0878a00640 1 -- 172.21.15.119:0/2530539555 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f085c003520 con 0x7f08740cf0e0 2024-04-25T10:07:21.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.793+0000 7f0878a00640 1 -- 172.21.15.119:0/2530539555 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657953212 0 0) 0x7f087406df90 con 0x7f087406d7a0 2024-04-25T10:07:21.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.793+0000 7f0878a00640 1 -- 172.21.15.119:0/2530539555 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f087406dcb0 con 0x7f087406d7a0 2024-04-25T10:07:21.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.793+0000 7f0879e00640 1 -- 172.21.15.119:0/2530539555 >> v1:172.21.15.132:6789/0 conn(0x7f087406d7a0 legacy=0x7f08740ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.793+0000 7f0879e00640 1 -- 172.21.15.119:0/2530539555 >> v1:172.21.15.119:6789/0 conn(0x7f08740cf0e0 legacy=0x7f08740cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.793+0000 7f0878a00640 1 -- 172.21.15.119:0/2530539555 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 752747282 0 0) 0x7f085c003520 con 0x7f08740cf0e0 2024-04-25T10:07:21.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.793+0000 7f0879e00640 1 -- 172.21.15.119:0/2530539555 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087406e270 con 0x7f0874144e70 2024-04-25T10:07:21.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.795+0000 7f0879e00640 1 --2- 172.21.15.119:0/2530539555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0874144e70 0x7f087406d260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f08680097d0 tx=0x7f086802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=423a81309506740d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.795+0000 7f0878a00640 1 -- 172.21.15.119:0/2530539555 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0868037070 con 0x7f0874144e70 2024-04-25T10:07:21.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.795+0000 7f0878a00640 1 -- 172.21.15.119:0/2530539555 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086802ab90 con 0x7f0874144e70 2024-04-25T10:07:21.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.795+0000 7f0878a00640 1 -- 172.21.15.119:0/2530539555 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870001a10 con 0x7f0874144e70 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.796+0000 7f087c800640 1 -- 172.21.15.119:0/2530539555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0874144e70 msgr2=0x7f087406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.796+0000 7f087c800640 1 --2- 172.21.15.119:0/2530539555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0874144e70 0x7f087406d260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f08680097d0 tx=0x7f086802a100 comp rx=0 tx=0).stop 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.796+0000 7f087c800640 1 -- 172.21.15.119:0/2530539555 shutdown_connections 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.796+0000 7f087c800640 1 --2- 172.21.15.119:0/2530539555 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0874144e70 0x7f087406d260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.796+0000 7f087c800640 1 -- 172.21.15.119:0/2530539555 >> 172.21.15.119:0/2530539555 conn(0x7f087406aba0 msgr2=0x7f087406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.796+0000 7f087c800640 1 -- 172.21.15.119:0/2530539555 shutdown_connections 2024-04-25T10:07:21.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.796+0000 7f087c800640 1 -- 172.21.15.119:0/2530539555 wait complete. 2024-04-25T10:07:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.798+0000 7f087c800640 1 Processor -- start 2024-04-25T10:07:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.798+0000 7f087c800640 1 -- start start 2024-04-25T10:07:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.799+0000 7f087c800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f087406d7a0 0x7f08740c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.799+0000 7f087c800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08740cf0e0 0x7f08740c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.799+0000 7f087c800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0874144e70 0x7f08740b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.799+0000 7f087c800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0874145630 con 0x7f08740cf0e0 2024-04-25T10:07:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.799+0000 7f087c800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08741454b0 con 0x7f0874144e70 2024-04-25T10:07:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.799+0000 7f087c800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08740cca10 con 0x7f087406d7a0 2024-04-25T10:07:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.799+0000 7f0879400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08740cf0e0 0x7f08740c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.800+0000 7f0879400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08740cf0e0 0x7f08740c45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53308/0 (socket says 172.21.15.119:53308) 2024-04-25T10:07:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.800+0000 7f0879400640 1 -- 172.21.15.119:0/487474039 learned_addr learned my addr 172.21.15.119:0/487474039 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.800+0000 7f0879e00640 1 --2- 172.21.15.119:0/487474039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f087406d7a0 0x7f08740c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.800+0000 7f087a800640 1 --2- 172.21.15.119:0/487474039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0874144e70 0x7f08740b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.800+0000 7f0879400640 1 -- 172.21.15.119:0/487474039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f087406d7a0 msgr2=0x7f08740c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.800+0000 7f0879400640 1 --2- 172.21.15.119:0/487474039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f087406d7a0 0x7f08740c5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.800+0000 7f0879400640 1 -- 172.21.15.119:0/487474039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0874144e70 msgr2=0x7f08740b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.800+0000 7f0879400640 1 --2- 172.21.15.119:0/487474039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0874144e70 0x7f08740b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.800+0000 7f0879400640 1 -- 172.21.15.119:0/487474039 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08740c4ec0 con 0x7f08740cf0e0 2024-04-25T10:07:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.800+0000 7f0879400640 1 --2- 172.21.15.119:0/487474039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08740cf0e0 0x7f08740c45c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f086400c400 tx=0x7f086400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.801+0000 7f086e000640 1 -- 172.21.15.119:0/487474039 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864017070 con 0x7f08740cf0e0 2024-04-25T10:07:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.801+0000 7f086e000640 1 -- 172.21.15.119:0/487474039 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08640052d0 con 0x7f08740cf0e0 2024-04-25T10:07:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.801+0000 7f086e000640 1 -- 172.21.15.119:0/487474039 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864003810 con 0x7f08740cf0e0 2024-04-25T10:07:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.801+0000 7f087c800640 1 -- 172.21.15.119:0/487474039 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08740c51a0 con 0x7f08740cf0e0 2024-04-25T10:07:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.801+0000 7f087c800640 1 -- 172.21.15.119:0/487474039 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08740c5760 con 0x7f08740cf0e0 2024-04-25T10:07:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.802+0000 7f087c800640 1 -- 172.21.15.119:0/487474039 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08740750b0 con 0x7f08740cf0e0 2024-04-25T10:07:21.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.803+0000 7f086e000640 1 -- 172.21.15.119:0/487474039 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f086401a070 con 0x7f08740cf0e0 2024-04-25T10:07:21.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.804+0000 7f086e000640 1 --2- 172.21.15.119:0/487474039 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f085806b4d0 0x7f085806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.804+0000 7f086e000640 1 -- 172.21.15.119:0/487474039 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0864089610 con 0x7f08740cf0e0 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.804+0000 7f0879e00640 1 --2- 172.21.15.119:0/487474039 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f085806b4d0 0x7f085806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.805+0000 7f0879e00640 1 --2- 172.21.15.119:0/487474039 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f085806b4d0 0x7f085806d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0868034000 tx=0x7f086802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:21.808+0000 7f086e000640 1 -- 172.21.15.119:0/487474039 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0864057da0 con 0x7f08740cf0e0 2024-04-25T10:07:22.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.006+0000 7f087c800640 1 -- 172.21.15.119:0/487474039 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08740cc5b0 con 0x7f08740cf0e0 2024-04-25T10:07:22.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.007+0000 7f086e000640 1 -- 172.21.15.119:0/487474039 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f086405be40 con 0x7f08740cf0e0 2024-04-25T10:07:22.013 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.010+0000 7f087c800640 1 -- 172.21.15.119:0/487474039 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f085806b4d0 msgr2=0x7f085806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.010+0000 7f087c800640 1 --2- 172.21.15.119:0/487474039 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f085806b4d0 0x7f085806d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0868034000 tx=0x7f086802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.010+0000 7f087c800640 1 -- 172.21.15.119:0/487474039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08740cf0e0 msgr2=0x7f08740c45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.010+0000 7f087c800640 1 --2- 172.21.15.119:0/487474039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08740cf0e0 0x7f08740c45c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f086400c400 tx=0x7f086400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.010+0000 7f087c800640 1 -- 172.21.15.119:0/487474039 shutdown_connections 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.010+0000 7f087c800640 1 --2- 172.21.15.119:0/487474039 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f085806b4d0 0x7f085806d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.010+0000 7f087c800640 1 --2- 172.21.15.119:0/487474039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0874144e70 0x7f08740b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.010+0000 7f087c800640 1 --2- 172.21.15.119:0/487474039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08740cf0e0 0x7f08740c45c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.010+0000 7f087c800640 1 --2- 172.21.15.119:0/487474039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f087406d7a0 0x7f08740c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.010+0000 7f087c800640 1 -- 172.21.15.119:0/487474039 >> 172.21.15.119:0/487474039 conn(0x7f087406aba0 msgr2=0x7f08740ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.010+0000 7f087c800640 1 -- 172.21.15.119:0/487474039 shutdown_connections 2024-04-25T10:07:22.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.010+0000 7f087c800640 1 -- 172.21.15.119:0/487474039 wait complete. 2024-04-25T10:07:22.035 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:22.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.210+0000 7fcd26000640 1 Processor -- start 2024-04-25T10:07:22.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.210+0000 7fcd26000640 1 -- start start 2024-04-25T10:07:22.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.211+0000 7fcd26000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcd20144e70 0x7fcd2006d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.211+0000 7fcd26000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd2006d7a0 0x7fcd200ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.211+0000 7fcd26000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd2006dcb0 con 0x7fcd200cef10 2024-04-25T10:07:22.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.211+0000 7fcd26000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2015bfe0 con 0x7fcd2006d7a0 2024-04-25T10:07:22.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.211+0000 7fcd26000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2006df90 con 0x7fcd20144e70 2024-04-25T10:07:22.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.212+0000 7fcd17e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd2006d7a0 0x7fcd200ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.212+0000 7fcd17e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd2006d7a0 0x7fcd200ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53320/0 (socket says 172.21.15.119:53320) 2024-04-25T10:07:22.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.212+0000 7fcd17e00640 1 -- 172.21.15.119:0/3674279953 learned_addr learned my addr 172.21.15.119:0/3674279953 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:22.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.212+0000 7fcd1f400640 1 --2- 172.21.15.119:0/3674279953 >> v2:172.21.15.132:3300/0 conn(0x7fcd20144e70 0x7fcd2006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.212+0000 7fcd17e00640 1 -- 172.21.15.119:0/3674279953 >> v2:172.21.15.132:3300/0 conn(0x7fcd20144e70 msgr2=0x7fcd2006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.212+0000 7fcd17e00640 1 --2- 172.21.15.119:0/3674279953 >> v2:172.21.15.132:3300/0 conn(0x7fcd20144e70 0x7fcd2006d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.212+0000 7fcd17e00640 1 -- 172.21.15.119:0/3674279953 >> v1:172.21.15.132:6789/0 conn(0x7fcd200cef10 legacy=0x7fcd20071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.212+0000 7fcd17e00640 1 -- 172.21.15.119:0/3674279953 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2006e270 con 0x7fcd2006d7a0 2024-04-25T10:07:22.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.213+0000 7fcd17e00640 1 --2- 172.21.15.119:0/3674279953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd2006d7a0 0x7fcd200ce9d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcd200b4a20 tx=0x7fcd0802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e42a9fe1c4363f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.214+0000 7fcd1ea00640 1 -- 172.21.15.119:0/3674279953 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08037070 con 0x7fcd2006d7a0 2024-04-25T10:07:22.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.214+0000 7fcd1ea00640 1 -- 172.21.15.119:0/3674279953 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd08032440 con 0x7fcd2006d7a0 2024-04-25T10:07:22.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.214+0000 7fcd1ea00640 1 -- 172.21.15.119:0/3674279953 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08032810 con 0x7fcd2006d7a0 2024-04-25T10:07:22.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.214+0000 7fcd26000640 1 -- 172.21.15.119:0/3674279953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd2006d7a0 msgr2=0x7fcd200ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.214+0000 7fcd26000640 1 --2- 172.21.15.119:0/3674279953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd2006d7a0 0x7fcd200ce9d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcd200b4a20 tx=0x7fcd0802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.215+0000 7fcd26000640 1 -- 172.21.15.119:0/3674279953 shutdown_connections 2024-04-25T10:07:22.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.215+0000 7fcd26000640 1 --2- 172.21.15.119:0/3674279953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd2006d7a0 0x7fcd200ce9d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.215+0000 7fcd26000640 1 --2- 172.21.15.119:0/3674279953 >> v2:172.21.15.132:3300/0 conn(0x7fcd20144e70 0x7fcd2006d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.215+0000 7fcd26000640 1 -- 172.21.15.119:0/3674279953 >> 172.21.15.119:0/3674279953 conn(0x7fcd2006aba0 msgr2=0x7fcd2006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.215+0000 7fcd26000640 1 -- 172.21.15.119:0/3674279953 shutdown_connections 2024-04-25T10:07:22.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.215+0000 7fcd26000640 1 -- 172.21.15.119:0/3674279953 wait complete. 2024-04-25T10:07:22.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.216+0000 7fcd26000640 1 Processor -- start 2024-04-25T10:07:22.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.217+0000 7fcd26000640 1 -- start start 2024-04-25T10:07:22.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.217+0000 7fcd26000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd2006d7a0 0x7fcd201f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.217+0000 7fcd26000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd200cef10 0x7fcd200b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.217+0000 7fcd26000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd20144e70 0x7fcd200af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.218+0000 7fcd26000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd201454b0 con 0x7fcd20144e70 2024-04-25T10:07:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.218+0000 7fcd26000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd200cf370 con 0x7fcd200cef10 2024-04-25T10:07:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.218+0000 7fcd26000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd20145630 con 0x7fcd2006d7a0 2024-04-25T10:07:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.218+0000 7fcd1fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd20144e70 0x7fcd200af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.218+0000 7fcd1fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd20144e70 0x7fcd200af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53334/0 (socket says 172.21.15.119:53334) 2024-04-25T10:07:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.218+0000 7fcd1fe00640 1 -- 172.21.15.119:0/976032568 learned_addr learned my addr 172.21.15.119:0/976032568 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.218+0000 7fcd17e00640 1 --2- 172.21.15.119:0/976032568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd200cef10 0x7fcd200b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.218+0000 7fcd1f400640 1 --2- 172.21.15.119:0/976032568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd2006d7a0 0x7fcd201f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.218+0000 7fcd1fe00640 1 -- 172.21.15.119:0/976032568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd2006d7a0 msgr2=0x7fcd201f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.218+0000 7fcd1fe00640 1 --2- 172.21.15.119:0/976032568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd2006d7a0 0x7fcd201f7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.218+0000 7fcd1fe00640 1 -- 172.21.15.119:0/976032568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd200cef10 msgr2=0x7fcd200b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.218+0000 7fcd1fe00640 1 --2- 172.21.15.119:0/976032568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd200cef10 0x7fcd200b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.218+0000 7fcd1fe00640 1 -- 172.21.15.119:0/976032568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd20154a50 con 0x7fcd20144e70 2024-04-25T10:07:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.219+0000 7fcd1fe00640 1 --2- 172.21.15.119:0/976032568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd20144e70 0x7fcd200af360 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800c780 tx=0x7fcd1800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.219+0000 7fcd17400640 1 -- 172.21.15.119:0/976032568 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18004b60 con 0x7fcd20144e70 2024-04-25T10:07:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.219+0000 7fcd17400640 1 -- 172.21.15.119:0/976032568 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd18004d00 con 0x7fcd20144e70 2024-04-25T10:07:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.219+0000 7fcd17400640 1 -- 172.21.15.119:0/976032568 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1800aa90 con 0x7fcd20144e70 2024-04-25T10:07:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.219+0000 7fcd26000640 1 -- 172.21.15.119:0/976032568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2013e570 con 0x7fcd20144e70 2024-04-25T10:07:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.219+0000 7fcd26000640 1 -- 172.21.15.119:0/976032568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2013ce30 con 0x7fcd20144e70 2024-04-25T10:07:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.220+0000 7fcd26000640 1 -- 172.21.15.119:0/976032568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2013d110 con 0x7fcd20144e70 2024-04-25T10:07:22.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.221+0000 7fcd17400640 1 -- 172.21.15.119:0/976032568 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd1800ac30 con 0x7fcd20144e70 2024-04-25T10:07:22.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.222+0000 7fcd17400640 1 --2- 172.21.15.119:0/976032568 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fccfc06b4d0 0x7fccfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.222+0000 7fcd17400640 1 -- 172.21.15.119:0/976032568 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd1808a4e0 con 0x7fcd20144e70 2024-04-25T10:07:22.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.222+0000 7fcd1f400640 1 --2- 172.21.15.119:0/976032568 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fccfc06b4d0 0x7fccfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.223+0000 7fcd1f400640 1 --2- 172.21.15.119:0/976032568 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fccfc06b4d0 0x7fccfc06d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcd10006fd0 tx=0x7fcd10008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.226+0000 7fcd17400640 1 -- 172.21.15.119:0/976032568 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd2013d110 con 0x7fcd20144e70 2024-04-25T10:07:22.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.416+0000 7fcd26000640 1 -- 172.21.15.119:0/976032568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd200750b0 con 0x7fcd20144e70 2024-04-25T10:07:22.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.417+0000 7fcd17400640 1 -- 172.21.15.119:0/976032568 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd18020070 con 0x7fcd20144e70 2024-04-25T10:07:22.423 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:07:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.420+0000 7fcd26000640 1 -- 172.21.15.119:0/976032568 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fccfc06b4d0 msgr2=0x7fccfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.420+0000 7fcd26000640 1 --2- 172.21.15.119:0/976032568 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fccfc06b4d0 0x7fccfc06d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcd10006fd0 tx=0x7fcd10008040 comp rx=0 tx=0).stop 2024-04-25T10:07:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.420+0000 7fcd26000640 1 -- 172.21.15.119:0/976032568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd20144e70 msgr2=0x7fcd200af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.420+0000 7fcd26000640 1 --2- 172.21.15.119:0/976032568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd20144e70 0x7fcd200af360 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800c780 tx=0x7fcd1800cc40 comp rx=0 tx=0).stop 2024-04-25T10:07:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.420+0000 7fcd26000640 1 -- 172.21.15.119:0/976032568 shutdown_connections 2024-04-25T10:07:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.420+0000 7fcd26000640 1 --2- 172.21.15.119:0/976032568 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fccfc06b4d0 0x7fccfc06d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.420+0000 7fcd26000640 1 --2- 172.21.15.119:0/976032568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd20144e70 0x7fcd200af360 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.420+0000 7fcd26000640 1 --2- 172.21.15.119:0/976032568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd200cef10 0x7fcd200b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.420+0000 7fcd26000640 1 --2- 172.21.15.119:0/976032568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd2006d7a0 0x7fcd201f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.420+0000 7fcd26000640 1 -- 172.21.15.119:0/976032568 >> 172.21.15.119:0/976032568 conn(0x7fcd2006aba0 msgr2=0x7fcd2013fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.420+0000 7fcd26000640 1 -- 172.21.15.119:0/976032568 shutdown_connections 2024-04-25T10:07:22.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.420+0000 7fcd26000640 1 -- 172.21.15.119:0/976032568 wait complete. 2024-04-25T10:07:22.443 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:22.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.628+0000 7efd7f200640 1 Processor -- start 2024-04-25T10:07:22.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.628+0000 7efd7f200640 1 -- start start 2024-04-25T10:07:22.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.630+0000 7efd7f200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efd8015b940 0x7efd800b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.630+0000 7efd7f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd800cf300 0x7efd800ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.630+0000 7efd7f200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efd800ce660 0x7efd80099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.630+0000 7efd7f200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd800d0800 con 0x7efd800ce660 2024-04-25T10:07:22.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.630+0000 7efd7f200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd800cf7a0 con 0x7efd8015b940 2024-04-25T10:07:22.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.630+0000 7efd7f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd800cfa40 con 0x7efd800cf300 2024-04-25T10:07:22.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.630+0000 7efd7e800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efd800ce660 0x7efd80099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.630+0000 7efd7e800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efd800ce660 0x7efd80099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53342/0 (socket says 172.21.15.119:53342) 2024-04-25T10:07:22.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.630+0000 7efd7e800640 1 -- 172.21.15.119:0/3287448873 learned_addr learned my addr 172.21.15.119:0/3287448873 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:22.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.631+0000 7efd7de00640 1 --2- 172.21.15.119:0/3287448873 >> v2:172.21.15.132:3300/0 conn(0x7efd8015b940 0x7efd800b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.631+0000 7efd7e800640 1 -- 172.21.15.119:0/3287448873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd800cf300 msgr2=0x7efd800ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.631+0000 7efd7d400640 1 --2- 172.21.15.119:0/3287448873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd800cf300 0x7efd800ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.631+0000 7efd7e800640 1 --2- 172.21.15.119:0/3287448873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd800cf300 0x7efd800ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.631+0000 7efd7e800640 1 -- 172.21.15.119:0/3287448873 >> v2:172.21.15.132:3300/0 conn(0x7efd8015b940 msgr2=0x7efd800b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.631+0000 7efd7e800640 1 --2- 172.21.15.119:0/3287448873 >> v2:172.21.15.132:3300/0 conn(0x7efd8015b940 0x7efd800b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.631+0000 7efd7e800640 1 -- 172.21.15.119:0/3287448873 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd800cfd20 con 0x7efd800ce660 2024-04-25T10:07:22.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.631+0000 7efd7de00640 1 --2- 172.21.15.119:0/3287448873 >> v2:172.21.15.132:3300/0 conn(0x7efd8015b940 0x7efd800b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:22.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.631+0000 7efd7d400640 1 --2- 172.21.15.119:0/3287448873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd800cf300 0x7efd800ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:22.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.632+0000 7efd7e800640 1 --2- 172.21.15.119:0/3287448873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd800ce660 0x7efd80099a40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efd800d2700 tx=0x7efd7802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79f95f79de2bfc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.632+0000 7efd7ca00640 1 -- 172.21.15.119:0/3287448873 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78037070 con 0x7efd800ce660 2024-04-25T10:07:22.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.632+0000 7efd7ca00640 1 -- 172.21.15.119:0/3287448873 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd78032440 con 0x7efd800ce660 2024-04-25T10:07:22.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.632+0000 7efd7ca00640 1 -- 172.21.15.119:0/3287448873 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78032810 con 0x7efd800ce660 2024-04-25T10:07:22.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.633+0000 7efd7f200640 1 -- 172.21.15.119:0/3287448873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd800ce660 msgr2=0x7efd80099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.633+0000 7efd7f200640 1 --2- 172.21.15.119:0/3287448873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd800ce660 0x7efd80099a40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efd800d2700 tx=0x7efd7802a140 comp rx=0 tx=0).stop 2024-04-25T10:07:22.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.633+0000 7efd7f200640 1 -- 172.21.15.119:0/3287448873 shutdown_connections 2024-04-25T10:07:22.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.633+0000 7efd7f200640 1 --2- 172.21.15.119:0/3287448873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd800ce660 0x7efd80099a40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.633+0000 7efd7f200640 1 --2- 172.21.15.119:0/3287448873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd800cf300 0x7efd800ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.633+0000 7efd7f200640 1 --2- 172.21.15.119:0/3287448873 >> v2:172.21.15.132:3300/0 conn(0x7efd8015b940 0x7efd800b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.633+0000 7efd7f200640 1 -- 172.21.15.119:0/3287448873 >> 172.21.15.119:0/3287448873 conn(0x7efd800d1730 msgr2=0x7efd800d1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.633+0000 7efd7f200640 1 -- 172.21.15.119:0/3287448873 shutdown_connections 2024-04-25T10:07:22.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.633+0000 7efd7f200640 1 -- 172.21.15.119:0/3287448873 wait complete. 2024-04-25T10:07:22.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.635+0000 7efd7f200640 1 Processor -- start 2024-04-25T10:07:22.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.635+0000 7efd7f200640 1 -- start start 2024-04-25T10:07:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd800ce660 0x7efd80152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd800cf300 0x7efd801f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd8015b940 0x7efd801f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd800b2530 con 0x7efd800ce660 2024-04-25T10:07:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8015bd20 con 0x7efd8015b940 2024-04-25T10:07:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd800b26b0 con 0x7efd800cf300 2024-04-25T10:07:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd800ce660 0x7efd80152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd800ce660 0x7efd80152c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53358/0 (socket says 172.21.15.119:53358) 2024-04-25T10:07:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7de00640 1 -- 172.21.15.119:0/2880891253 learned_addr learned my addr 172.21.15.119:0/2880891253 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7d400640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd800cf300 0x7efd801f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7de00640 1 -- 172.21.15.119:0/2880891253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd800cf300 msgr2=0x7efd801f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7e800640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd8015b940 0x7efd801f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7de00640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd800cf300 0x7efd801f6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7de00640 1 -- 172.21.15.119:0/2880891253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd8015b940 msgr2=0x7efd801f70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7de00640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd8015b940 0x7efd801f70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7de00640 1 -- 172.21.15.119:0/2880891253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8015dee0 con 0x7efd800ce660 2024-04-25T10:07:22.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.636+0000 7efd7d400640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd800cf300 0x7efd801f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:22.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.637+0000 7efd7de00640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd800ce660 0x7efd80152c70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd7000c820 tx=0x7efd7000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.637+0000 7efd76000640 1 -- 172.21.15.119:0/2880891253 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70015070 con 0x7efd800ce660 2024-04-25T10:07:22.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.637+0000 7efd76000640 1 -- 172.21.15.119:0/2880891253 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd70007a30 con 0x7efd800ce660 2024-04-25T10:07:22.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.637+0000 7efd76000640 1 -- 172.21.15.119:0/2880891253 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70007e00 con 0x7efd800ce660 2024-04-25T10:07:22.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.637+0000 7efd7f200640 1 -- 172.21.15.119:0/2880891253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8015e1c0 con 0x7efd800ce660 2024-04-25T10:07:22.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.637+0000 7efd7f200640 1 -- 172.21.15.119:0/2880891253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8015e780 con 0x7efd800ce660 2024-04-25T10:07:22.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.638+0000 7efd7f200640 1 -- 172.21.15.119:0/2880891253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd800749b0 con 0x7efd800ce660 2024-04-25T10:07:22.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.639+0000 7efd76000640 1 -- 172.21.15.119:0/2880891253 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd70018070 con 0x7efd800ce660 2024-04-25T10:07:22.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.640+0000 7efd76000640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd5c06b4d0 0x7efd5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.640+0000 7efd76000640 1 -- 172.21.15.119:0/2880891253 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd7008a180 con 0x7efd800ce660 2024-04-25T10:07:22.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.640+0000 7efd7d400640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd5c06b4d0 0x7efd5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.641+0000 7efd7d400640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd5c06b4d0 0x7efd5c06d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd8015bf50 tx=0x7efd68008450 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.644+0000 7efd76000640 1 -- 172.21.15.119:0/2880891253 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd70059000 con 0x7efd800ce660 2024-04-25T10:07:22.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.878+0000 7efd7f200640 1 -- 172.21.15.119:0/2880891253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd800cdac0 con 0x7efd800ce660 2024-04-25T10:07:22.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.879+0000 7efd76000640 1 -- 172.21.15.119:0/2880891253 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4941 (secure 0 0 0) 0x7efd70058900 con 0x7efd800ce660 2024-04-25T10:07:22.884 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:22.885 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1385207078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1385207078},{"type":"v1","addr":"172.21.15.132:6834","nonce":1385207078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1580923823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1580923823},{"type":"v1","addr":"172.21.15.119:6836","nonce":1580923823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4260,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/954053568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":954053568},{"type":"v1","addr":"172.21.15.132:6835","nonce":954053568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:06.743184+0000","modified":"2024-04-25T10:07:10.118630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1632024048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1632024048},{"type":"v1","addr":"172.21.15.119:6837","nonce":1632024048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:07:22.885 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2024-04-25T10:07:22.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.882+0000 7efd7f200640 1 -- 172.21.15.119:0/2880891253 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd5c06b4d0 msgr2=0x7efd5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.882+0000 7efd7f200640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd5c06b4d0 0x7efd5c06d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd8015bf50 tx=0x7efd68008450 comp rx=0 tx=0).stop 2024-04-25T10:07:22.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.882+0000 7efd7f200640 1 -- 172.21.15.119:0/2880891253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd800ce660 msgr2=0x7efd80152c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.882+0000 7efd7f200640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd800ce660 0x7efd80152c70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd7000c820 tx=0x7efd7000cc20 comp rx=0 tx=0).stop 2024-04-25T10:07:22.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.882+0000 7efd7f200640 1 -- 172.21.15.119:0/2880891253 shutdown_connections 2024-04-25T10:07:22.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.882+0000 7efd7f200640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd5c06b4d0 0x7efd5c06d990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.882+0000 7efd7f200640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd8015b940 0x7efd801f70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.882+0000 7efd7f200640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd800cf300 0x7efd801f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.882+0000 7efd7f200640 1 --2- 172.21.15.119:0/2880891253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd800ce660 0x7efd80152c70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.882+0000 7efd7f200640 1 -- 172.21.15.119:0/2880891253 >> 172.21.15.119:0/2880891253 conn(0x7efd800d1730 msgr2=0x7efd800b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.882+0000 7efd7f200640 1 -- 172.21.15.119:0/2880891253 shutdown_connections 2024-04-25T10:07:22.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:22.882+0000 7efd7f200640 1 -- 172.21.15.119:0/2880891253 wait complete. 2024-04-25T10:07:22.906 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.094+0000 7f7ebfe00640 1 Processor -- start 2024-04-25T10:07:23.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.094+0000 7f7ebfe00640 1 -- start start 2024-04-25T10:07:23.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.095+0000 7f7ebfe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ec0144470 0x7f7ec015fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.095+0000 7f7ebfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec006e430 0x7f7ec00b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.095+0000 7f7ebfe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7ec00b4e80 0x7f7ec006d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.095+0000 7f7ebfe00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec0145340 con 0x7f7ec0144470 2024-04-25T10:07:23.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.095+0000 7f7ebfe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec00cd440 con 0x7f7ec00b4e80 2024-04-25T10:07:23.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.095+0000 7f7ebfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00cd720 con 0x7f7ec006e430 2024-04-25T10:07:23.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.096+0000 7f7ebea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ec0144470 0x7f7ec015fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.096+0000 7f7ebea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ec0144470 0x7f7ec015fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53368/0 (socket says 172.21.15.119:53368) 2024-04-25T10:07:23.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.096+0000 7f7ebea00640 1 -- 172.21.15.119:0/3916171740 learned_addr learned my addr 172.21.15.119:0/3916171740 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:23.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.096+0000 7f7ebe000640 1 --2- 172.21.15.119:0/3916171740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec006e430 0x7f7ec00b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.096+0000 7f7ebea00640 1 -- 172.21.15.119:0/3916171740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec006e430 msgr2=0x7f7ec00b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.096+0000 7f7ebea00640 1 --2- 172.21.15.119:0/3916171740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec006e430 0x7f7ec00b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.096+0000 7f7ebf400640 1 --2- 172.21.15.119:0/3916171740 >> v2:172.21.15.132:3300/0 conn(0x7f7ec00b4e80 0x7f7ec006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.096+0000 7f7ebea00640 1 -- 172.21.15.119:0/3916171740 >> v2:172.21.15.132:3300/0 conn(0x7f7ec00b4e80 msgr2=0x7f7ec006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.096+0000 7f7ebea00640 1 --2- 172.21.15.119:0/3916171740 >> v2:172.21.15.132:3300/0 conn(0x7f7ec00b4e80 0x7f7ec006d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.096+0000 7f7ebea00640 1 -- 172.21.15.119:0/3916171740 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec00cda00 con 0x7f7ec0144470 2024-04-25T10:07:23.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.096+0000 7f7ebe000640 1 --2- 172.21.15.119:0/3916171740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec006e430 0x7f7ec00b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.097+0000 7f7ebea00640 1 --2- 172.21.15.119:0/3916171740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec0144470 0x7f7ec015fc90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00b26e0 tx=0x7f7eb002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6747877175323dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.097+0000 7f7ebd600640 1 -- 172.21.15.119:0/3916171740 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0036070 con 0x7f7ec0144470 2024-04-25T10:07:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.097+0000 7f7ebd600640 1 -- 172.21.15.119:0/3916171740 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb0031600 con 0x7f7ec0144470 2024-04-25T10:07:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.097+0000 7f7ebd600640 1 -- 172.21.15.119:0/3916171740 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb00319d0 con 0x7f7ec0144470 2024-04-25T10:07:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.098+0000 7f7ebfe00640 1 -- 172.21.15.119:0/3916171740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec0144470 msgr2=0x7f7ec015fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.098+0000 7f7ebfe00640 1 --2- 172.21.15.119:0/3916171740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec0144470 0x7f7ec015fc90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00b26e0 tx=0x7f7eb002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.098+0000 7f7ebfe00640 1 -- 172.21.15.119:0/3916171740 shutdown_connections 2024-04-25T10:07:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.098+0000 7f7ebfe00640 1 --2- 172.21.15.119:0/3916171740 >> v2:172.21.15.132:3300/0 conn(0x7f7ec00b4e80 0x7f7ec006d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.098+0000 7f7ebfe00640 1 --2- 172.21.15.119:0/3916171740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec006e430 0x7f7ec00b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.098+0000 7f7ebfe00640 1 --2- 172.21.15.119:0/3916171740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec0144470 0x7f7ec015fc90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.098+0000 7f7ebfe00640 1 -- 172.21.15.119:0/3916171740 >> 172.21.15.119:0/3916171740 conn(0x7f7ec006b1d0 msgr2=0x7f7ec00b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.098+0000 7f7ebfe00640 1 -- 172.21.15.119:0/3916171740 shutdown_connections 2024-04-25T10:07:23.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.098+0000 7f7ebfe00640 1 -- 172.21.15.119:0/3916171740 wait complete. 2024-04-25T10:07:23.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.100+0000 7f7ebfe00640 1 Processor -- start 2024-04-25T10:07:23.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.100+0000 7f7ebfe00640 1 -- start start 2024-04-25T10:07:23.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.101+0000 7f7ebfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec006e430 0x7f7ec0139d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.101+0000 7f7ebfe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec00b4e80 0x7f7ec015ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.101+0000 7f7ebfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec0144470 0x7f7ec0136e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.101+0000 7f7ebfe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00ce200 con 0x7f7ec006e430 2024-04-25T10:07:23.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.101+0000 7f7ebfe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec0144850 con 0x7f7ec00b4e80 2024-04-25T10:07:23.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.101+0000 7f7ebfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec006d750 con 0x7f7ec0144470 2024-04-25T10:07:23.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.101+0000 7f7ebea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec006e430 0x7f7ec0139d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.101+0000 7f7ebea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec006e430 0x7f7ec0139d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53382/0 (socket says 172.21.15.119:53382) 2024-04-25T10:07:23.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.101+0000 7f7ebea00640 1 -- 172.21.15.119:0/133413179 learned_addr learned my addr 172.21.15.119:0/133413179 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:23.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.101+0000 7f7ebea00640 1 -- 172.21.15.119:0/133413179 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec0144470 msgr2=0x7f7ec0136e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.101+0000 7f7ebea00640 1 --2- 172.21.15.119:0/133413179 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec0144470 0x7f7ec0136e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.101+0000 7f7ebea00640 1 -- 172.21.15.119:0/133413179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec00b4e80 msgr2=0x7f7ec015ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.101+0000 7f7ebea00640 1 --2- 172.21.15.119:0/133413179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec00b4e80 0x7f7ec015ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.101+0000 7f7ebea00640 1 -- 172.21.15.119:0/133413179 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec0137380 con 0x7f7ec006e430 2024-04-25T10:07:23.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.102+0000 7f7ebea00640 1 --2- 172.21.15.119:0/133413179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec006e430 0x7f7ec0139d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00093e0 tx=0x7f7eb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.102+0000 7f7eb6a00640 1 -- 172.21.15.119:0/133413179 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0041020 con 0x7f7ec006e430 2024-04-25T10:07:23.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.102+0000 7f7eb6a00640 1 -- 172.21.15.119:0/133413179 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb0007800 con 0x7f7ec006e430 2024-04-25T10:07:23.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.102+0000 7f7ebfe00640 1 -- 172.21.15.119:0/133413179 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec0137620 con 0x7f7ec006e430 2024-04-25T10:07:23.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.102+0000 7f7eb6a00640 1 -- 172.21.15.119:0/133413179 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0036040 con 0x7f7ec006e430 2024-04-25T10:07:23.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.102+0000 7f7ebfe00640 1 -- 172.21.15.119:0/133413179 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec0137be0 con 0x7f7ec006e430 2024-04-25T10:07:23.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.103+0000 7f7ebfe00640 1 -- 172.21.15.119:0/133413179 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec0137e20 con 0x7f7ec006e430 2024-04-25T10:07:23.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.104+0000 7f7eb6a00640 1 -- 172.21.15.119:0/133413179 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7eb0044020 con 0x7f7ec006e430 2024-04-25T10:07:23.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.105+0000 7f7eb6a00640 1 --2- 172.21.15.119:0/133413179 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7ea006b500 0x7f7ea006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.105+0000 7f7eb6a00640 1 -- 172.21.15.119:0/133413179 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7eb00aae60 con 0x7f7ec006e430 2024-04-25T10:07:23.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.105+0000 7f7ebe000640 1 --2- 172.21.15.119:0/133413179 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7ea006b500 0x7f7ea006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.106+0000 7f7ebe000640 1 --2- 172.21.15.119:0/133413179 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7ea006b500 0x7f7ea006d9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7eac006fd0 tx=0x7f7eac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.109+0000 7f7eb6a00640 1 -- 172.21.15.119:0/133413179 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7eb004b070 con 0x7f7ec006e430 2024-04-25T10:07:23.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.304+0000 7f7ebfe00640 1 -- 172.21.15.119:0/133413179 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ec00749b0 con 0x7f7ec006e430 2024-04-25T10:07:23.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.305+0000 7f7eb6a00640 1 -- 172.21.15.119:0/133413179 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f7eb0079d90 con 0x7f7ec006e430 2024-04-25T10:07:23.312 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:23.312 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:08.319104+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:23.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.308+0000 7f7ebfe00640 1 -- 172.21.15.119:0/133413179 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7ea006b500 msgr2=0x7f7ea006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.308+0000 7f7ebfe00640 1 --2- 172.21.15.119:0/133413179 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7ea006b500 0x7f7ea006d9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7eac006fd0 tx=0x7f7eac008040 comp rx=0 tx=0).stop 2024-04-25T10:07:23.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.308+0000 7f7ebfe00640 1 -- 172.21.15.119:0/133413179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec006e430 msgr2=0x7f7ec0139d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.308+0000 7f7ebfe00640 1 --2- 172.21.15.119:0/133413179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec006e430 0x7f7ec0139d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00093e0 tx=0x7f7eb0004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:23.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.308+0000 7f7ebfe00640 1 -- 172.21.15.119:0/133413179 shutdown_connections 2024-04-25T10:07:23.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.308+0000 7f7ebfe00640 1 --2- 172.21.15.119:0/133413179 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7ea006b500 0x7f7ea006d9c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.308+0000 7f7ebfe00640 1 --2- 172.21.15.119:0/133413179 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ec0144470 0x7f7ec0136e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.308+0000 7f7ebfe00640 1 --2- 172.21.15.119:0/133413179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ec00b4e80 0x7f7ec015ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.308+0000 7f7ebfe00640 1 --2- 172.21.15.119:0/133413179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec006e430 0x7f7ec0139d80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.308+0000 7f7ebfe00640 1 -- 172.21.15.119:0/133413179 >> 172.21.15.119:0/133413179 conn(0x7f7ec006b1d0 msgr2=0x7f7ec00b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:23.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.308+0000 7f7ebfe00640 1 -- 172.21.15.119:0/133413179 shutdown_connections 2024-04-25T10:07:23.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.308+0000 7f7ebfe00640 1 -- 172.21.15.119:0/133413179 wait complete. 2024-04-25T10:07:23.334 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:23.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.524+0000 7f7555a00640 1 Processor -- start 2024-04-25T10:07:23.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.524+0000 7f7555a00640 1 -- start start 2024-04-25T10:07:23.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.526+0000 7f7555a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f755015db60 0x7f75500d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.526+0000 7f7555a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f75500b9270 0x7f75500d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.526+0000 7f7555a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f755015ef10 con 0x7f75500b5360 2024-04-25T10:07:23.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.526+0000 7f7555a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755015e390 con 0x7f75500b9270 2024-04-25T10:07:23.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.526+0000 7f7555a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755015e670 con 0x7f755015db60 2024-04-25T10:07:23.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.526+0000 7f7555000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f75500b5360 0x7f75500b57d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48698/0 (socket says 172.21.15.119:48698) 2024-04-25T10:07:23.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.526+0000 7f7555000640 1 -- 172.21.15.119:0/3690070921 learned_addr learned my addr 172.21.15.119:0/3690070921 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:23.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.526+0000 7f754fe00640 1 --2- 172.21.15.119:0/3690070921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f755015db60 0x7f75500d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.526+0000 7f754ea00640 1 -- 172.21.15.119:0/3690070921 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4169785073 0 0) 0x7f755015ef10 con 0x7f75500b5360 2024-04-25T10:07:23.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 -- 172.21.15.119:0/3690070921 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7534003520 con 0x7f75500b5360 2024-04-25T10:07:23.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754f400640 1 --2- 172.21.15.119:0/3690070921 >> v2:172.21.15.132:3300/0 conn(0x7f75500b9270 0x7f75500d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 -- 172.21.15.119:0/3690070921 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2455458279 0 0) 0x7f7534003520 con 0x7f75500b5360 2024-04-25T10:07:23.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 -- 172.21.15.119:0/3690070921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f755015db60 msgr2=0x7f75500d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 --2- 172.21.15.119:0/3690070921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f755015db60 0x7f75500d3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 -- 172.21.15.119:0/3690070921 >> v2:172.21.15.132:3300/0 conn(0x7f75500b9270 msgr2=0x7f75500d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 --2- 172.21.15.119:0/3690070921 >> v2:172.21.15.132:3300/0 conn(0x7f75500b9270 0x7f75500d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 -- 172.21.15.119:0/3690070921 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755015e950 con 0x7f75500b5360 2024-04-25T10:07:23.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 -- 172.21.15.119:0/3690070921 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f7548001a10 con 0x7f75500b5360 2024-04-25T10:07:23.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 -- 172.21.15.119:0/3690070921 >> v1:172.21.15.119:6789/0 conn(0x7f75500b5360 legacy=0x7f75500b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 --2- 172.21.15.119:0/3690070921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f755015db60 0x7f7534003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 --2- 172.21.15.119:0/3690070921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7534003fa0 0x7f7534004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 --2- 172.21.15.119:0/3690070921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7534004d40 0x7f75340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 -- 172.21.15.119:0/3690070921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755015e390 con 0x7f7534004d40 2024-04-25T10:07:23.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 -- 172.21.15.119:0/3690070921 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755015e670 con 0x7f7534003fa0 2024-04-25T10:07:23.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f754ea00640 1 -- 172.21.15.119:0/3690070921 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753400baa0 con 0x7f755015db60 2024-04-25T10:07:23.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.527+0000 7f7555000640 1 --2- 172.21.15.119:0/3690070921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7534004d40 0x7f75340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.528+0000 7f754fe00640 1 --2- 172.21.15.119:0/3690070921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f755015db60 0x7f7534003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.528+0000 7f754f400640 1 --2- 172.21.15.119:0/3690070921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7534003fa0 0x7f7534004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.528+0000 7f7555000640 1 -- 172.21.15.119:0/3690070921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f755015db60 msgr2=0x7f7534003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.528+0000 7f7555000640 1 --2- 172.21.15.119:0/3690070921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f755015db60 0x7f7534003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.528+0000 7f7555000640 1 -- 172.21.15.119:0/3690070921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7534003fa0 msgr2=0x7f7534004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.528+0000 7f7555000640 1 --2- 172.21.15.119:0/3690070921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7534003fa0 0x7f7534004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.528+0000 7f7555000640 1 -- 172.21.15.119:0/3690070921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7534003520 con 0x7f7534004d40 2024-04-25T10:07:23.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.528+0000 7f754f400640 1 --2- 172.21.15.119:0/3690070921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7534003fa0 0x7f7534004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:23.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.529+0000 7f7555000640 1 --2- 172.21.15.119:0/3690070921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7534004d40 0x7f75340051c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f754800a890 tx=0x7f7534005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9030bb369fceaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.529+0000 7f754ea00640 1 -- 172.21.15.119:0/3690070921 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754800a4e0 con 0x7f7534004d40 2024-04-25T10:07:23.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.529+0000 7f754ea00640 1 -- 172.21.15.119:0/3690070921 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754802f070 con 0x7f7534004d40 2024-04-25T10:07:23.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.529+0000 7f754ea00640 1 -- 172.21.15.119:0/3690070921 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754802bcc0 con 0x7f7534004d40 2024-04-25T10:07:23.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.529+0000 7f7555a00640 1 -- 172.21.15.119:0/3690070921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7534004d40 msgr2=0x7f75340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.529+0000 7f7555a00640 1 --2- 172.21.15.119:0/3690070921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7534004d40 0x7f75340051c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f754800a890 tx=0x7f7534005830 comp rx=0 tx=0).stop 2024-04-25T10:07:23.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.529+0000 7f7555000640 1 -- 172.21.15.119:0/3690070921 reap_dead start 2024-04-25T10:07:23.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.530+0000 7f7555a00640 1 -- 172.21.15.119:0/3690070921 shutdown_connections 2024-04-25T10:07:23.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.530+0000 7f7555a00640 1 -- 172.21.15.119:0/3690070921 >> 172.21.15.119:0/3690070921 conn(0x7f75500726c0 msgr2=0x7f7550072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:23.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.530+0000 7f7555a00640 1 -- 172.21.15.119:0/3690070921 shutdown_connections 2024-04-25T10:07:23.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.530+0000 7f7555a00640 1 -- 172.21.15.119:0/3690070921 wait complete. 2024-04-25T10:07:23.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.531+0000 7f7555a00640 1 Processor -- start 2024-04-25T10:07:23.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.531+0000 7f7555a00640 1 -- start start 2024-04-25T10:07:23.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.532+0000 7f7555a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7534004d40 0x7f75500ce420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.532+0000 7f7555a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75500ca9b0 0x7f755014e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.532+0000 7f7555a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7550160970 0x7f755014fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.532+0000 7f7555a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75500b3120 con 0x7f7550160970 2024-04-25T10:07:23.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.532+0000 7f7555a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75500b2fa0 con 0x7f7534004d40 2024-04-25T10:07:23.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.532+0000 7f7555a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755015df40 con 0x7f75500ca9b0 2024-04-25T10:07:23.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.533+0000 7f7555000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7550160970 0x7f755014fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.533+0000 7f7555000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7550160970 0x7f755014fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53392/0 (socket says 172.21.15.119:53392) 2024-04-25T10:07:23.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.533+0000 7f7555000640 1 -- 172.21.15.119:0/232951641 learned_addr learned my addr 172.21.15.119:0/232951641 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:23.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.533+0000 7f7555000640 1 -- 172.21.15.119:0/232951641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75500ca9b0 msgr2=0x7f755014e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.533+0000 7f754fe00640 1 --2- 172.21.15.119:0/232951641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7534004d40 0x7f75500ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.533+0000 7f7555000640 1 --2- 172.21.15.119:0/232951641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75500ca9b0 0x7f755014e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.533+0000 7f7555000640 1 -- 172.21.15.119:0/232951641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7534004d40 msgr2=0x7f75500ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.533+0000 7f7555000640 1 --2- 172.21.15.119:0/232951641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7534004d40 0x7f75500ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.533+0000 7f7555000640 1 -- 172.21.15.119:0/232951641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755014ea70 con 0x7f7550160970 2024-04-25T10:07:23.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.533+0000 7f7555000640 1 --2- 172.21.15.119:0/232951641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7550160970 0x7f755014fd90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f754802b530 tx=0x7f7548001de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.534+0000 7f7547e00640 1 -- 172.21.15.119:0/232951641 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754800a4e0 con 0x7f7550160970 2024-04-25T10:07:23.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.534+0000 7f7547e00640 1 -- 172.21.15.119:0/232951641 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7548001a40 con 0x7f7550160970 2024-04-25T10:07:23.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.534+0000 7f7547e00640 1 -- 172.21.15.119:0/232951641 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548002030 con 0x7f7550160970 2024-04-25T10:07:23.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.534+0000 7f7555a00640 1 -- 172.21.15.119:0/232951641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75500cae10 con 0x7f7550160970 2024-04-25T10:07:23.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.534+0000 7f7555a00640 1 -- 172.21.15.119:0/232951641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755014efd0 con 0x7f7550160970 2024-04-25T10:07:23.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.535+0000 7f7555a00640 1 -- 172.21.15.119:0/232951641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755015f100 con 0x7f7550160970 2024-04-25T10:07:23.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.536+0000 7f7547e00640 1 -- 172.21.15.119:0/232951641 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f754803e030 con 0x7f7550160970 2024-04-25T10:07:23.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.537+0000 7f7547e00640 1 --2- 172.21.15.119:0/232951641 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f753006b500 0x7f753006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.537+0000 7f7547e00640 1 -- 172.21.15.119:0/232951641 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75480a99b0 con 0x7f7550160970 2024-04-25T10:07:23.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.537+0000 7f754fe00640 1 --2- 172.21.15.119:0/232951641 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f753006b500 0x7f753006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.538+0000 7f754fe00640 1 --2- 172.21.15.119:0/232951641 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f753006b500 0x7f753006d9c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7540009950 tx=0x7f7540008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.541+0000 7f7547e00640 1 -- 172.21.15.119:0/232951641 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f755015f100 con 0x7f7550160970 2024-04-25T10:07:23.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.772+0000 7f7555a00640 1 -- 172.21.15.119:0/232951641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75500c6ac0 con 0x7f7550160970 2024-04-25T10:07:23.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.772+0000 7f7547e00640 1 -- 172.21.15.119:0/232951641 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4941 (secure 0 0 0) 0x7f755014ea70 con 0x7f7550160970 2024-04-25T10:07:23.778 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:23.779 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1385207078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1385207078},{"type":"v1","addr":"172.21.15.132:6834","nonce":1385207078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1580923823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1580923823},{"type":"v1","addr":"172.21.15.119:6836","nonce":1580923823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4260,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/954053568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":954053568},{"type":"v1","addr":"172.21.15.132:6835","nonce":954053568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:06.743184+0000","modified":"2024-04-25T10:07:10.118630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1632024048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1632024048},{"type":"v1","addr":"172.21.15.119:6837","nonce":1632024048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:07:23.779 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2024-04-25T10:07:23.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.775+0000 7f7555a00640 1 -- 172.21.15.119:0/232951641 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f753006b500 msgr2=0x7f753006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.775+0000 7f7555a00640 1 --2- 172.21.15.119:0/232951641 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f753006b500 0x7f753006d9c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7540009950 tx=0x7f7540008040 comp rx=0 tx=0).stop 2024-04-25T10:07:23.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.775+0000 7f7555a00640 1 -- 172.21.15.119:0/232951641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7550160970 msgr2=0x7f755014fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.775+0000 7f7555a00640 1 --2- 172.21.15.119:0/232951641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7550160970 0x7f755014fd90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f754802b530 tx=0x7f7548001de0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.775+0000 7f7555a00640 1 -- 172.21.15.119:0/232951641 shutdown_connections 2024-04-25T10:07:23.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.775+0000 7f7555a00640 1 --2- 172.21.15.119:0/232951641 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f753006b500 0x7f753006d9c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.775+0000 7f7555a00640 1 --2- 172.21.15.119:0/232951641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7550160970 0x7f755014fd90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.775+0000 7f7555a00640 1 --2- 172.21.15.119:0/232951641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75500ca9b0 0x7f755014e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.775+0000 7f7555a00640 1 --2- 172.21.15.119:0/232951641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7534004d40 0x7f75500ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.775+0000 7f7555a00640 1 -- 172.21.15.119:0/232951641 >> 172.21.15.119:0/232951641 conn(0x7f75500726c0 msgr2=0x7f75500b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:23.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.776+0000 7f7555a00640 1 -- 172.21.15.119:0/232951641 shutdown_connections 2024-04-25T10:07:23.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:23.776+0000 7f7555a00640 1 -- 172.21.15.119:0/232951641 wait complete. 2024-04-25T10:07:24.201 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:07:06.743184+0000', 'modified': '2024-04-25T10:07:10.118630+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4261}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4261': {'gid': 4261, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6837/1632024048', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 1632024048}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 1632024048}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:24.202 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:07:24.202 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:07:24.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.405+0000 7f111fe00640 1 Processor -- start 2024-04-25T10:07:24.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.406+0000 7f111fe00640 1 -- start start 2024-04-25T10:07:24.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.407+0000 7f111fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112006b2b0 0x7f11200acb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.407+0000 7f111fe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f11200b2f90 0x7f11200b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.407+0000 7f111fe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11200b3940 con 0x7f11200ada20 2024-04-25T10:07:24.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.407+0000 7f111fe00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112015dec0 con 0x7f11200b2f90 2024-04-25T10:07:24.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.407+0000 7f111fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112015e120 con 0x7f112006b2b0 2024-04-25T10:07:24.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.408+0000 7f111f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f11200b2f90 0x7f11200b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.408+0000 7f111f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f11200b2f90 0x7f11200b3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53396/0 (socket says 172.21.15.119:53396) 2024-04-25T10:07:24.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.408+0000 7f111f400640 1 -- 172.21.15.119:0/4188297953 learned_addr learned my addr 172.21.15.119:0/4188297953 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:24.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.408+0000 7f111e000640 1 --2- 172.21.15.119:0/4188297953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112006b2b0 0x7f11200acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.408+0000 7f111f400640 1 -- 172.21.15.119:0/4188297953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112006b2b0 msgr2=0x7f11200acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.408+0000 7f111f400640 1 --2- 172.21.15.119:0/4188297953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112006b2b0 0x7f11200acb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.408+0000 7f111f400640 1 -- 172.21.15.119:0/4188297953 >> v1:172.21.15.132:6789/0 conn(0x7f11200ada20 legacy=0x7f112006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.408+0000 7f111f400640 1 -- 172.21.15.119:0/4188297953 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112015e400 con 0x7f11200b2f90 2024-04-25T10:07:24.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.408+0000 7f111e000640 1 --2- 172.21.15.119:0/4188297953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112006b2b0 0x7f11200acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:24.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.409+0000 7f111f400640 1 --2- 172.21.15.119:0/4188297953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11200b2f90 0x7f11200b3400 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1118009e50 tx=0x7f111802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6a28ec29a24fab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.409+0000 7f111d600640 1 -- 172.21.15.119:0/4188297953 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118036070 con 0x7f11200b2f90 2024-04-25T10:07:24.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.409+0000 7f111d600640 1 -- 172.21.15.119:0/4188297953 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1118031600 con 0x7f11200b2f90 2024-04-25T10:07:24.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.409+0000 7f111d600640 1 -- 172.21.15.119:0/4188297953 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11180319d0 con 0x7f11200b2f90 2024-04-25T10:07:24.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.410+0000 7f111fe00640 1 -- 172.21.15.119:0/4188297953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11200b2f90 msgr2=0x7f11200b3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.410+0000 7f111fe00640 1 --2- 172.21.15.119:0/4188297953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11200b2f90 0x7f11200b3400 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1118009e50 tx=0x7f111802a6d0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.410+0000 7f111fe00640 1 -- 172.21.15.119:0/4188297953 shutdown_connections 2024-04-25T10:07:24.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.410+0000 7f111fe00640 1 --2- 172.21.15.119:0/4188297953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11200b2f90 0x7f11200b3400 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.410+0000 7f111fe00640 1 --2- 172.21.15.119:0/4188297953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112006b2b0 0x7f11200acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.410+0000 7f111fe00640 1 -- 172.21.15.119:0/4188297953 >> 172.21.15.119:0/4188297953 conn(0x7f1120066ff0 msgr2=0x7f11200b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.410+0000 7f111fe00640 1 -- 172.21.15.119:0/4188297953 shutdown_connections 2024-04-25T10:07:24.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.410+0000 7f111fe00640 1 -- 172.21.15.119:0/4188297953 wait complete. 2024-04-25T10:07:24.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.412+0000 7f111fe00640 1 Processor -- start 2024-04-25T10:07:24.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.413+0000 7f111fe00640 1 -- start start 2024-04-25T10:07:24.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.414+0000 7f111fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f112006b2b0 0x7f11201466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.414+0000 7f111fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11200ada20 0x7f1120143ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.414+0000 7f111fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11200b2f90 0x7f11201452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.414+0000 7f111fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112015eca0 con 0x7f112006b2b0 2024-04-25T10:07:24.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.414+0000 7f111fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112015e9c0 con 0x7f11200b2f90 2024-04-25T10:07:24.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.414+0000 7f111fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11200ade00 con 0x7f11200ada20 2024-04-25T10:07:24.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.414+0000 7f111ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f112006b2b0 0x7f11201466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.414+0000 7f111ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f112006b2b0 0x7f11201466b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53404/0 (socket says 172.21.15.119:53404) 2024-04-25T10:07:24.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.414+0000 7f111ea00640 1 -- 172.21.15.119:0/1914016311 learned_addr learned my addr 172.21.15.119:0/1914016311 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:24.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.414+0000 7f111e000640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11200ada20 0x7f1120143ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.414+0000 7f111f400640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11200b2f90 0x7f11201452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.415+0000 7f111ea00640 1 -- 172.21.15.119:0/1914016311 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11200ada20 msgr2=0x7f1120143ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.415+0000 7f111ea00640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11200ada20 0x7f1120143ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.415+0000 7f111ea00640 1 -- 172.21.15.119:0/1914016311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11200b2f90 msgr2=0x7f11201452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.415+0000 7f111ea00640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11200b2f90 0x7f11201452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.415+0000 7f111ea00640 1 -- 172.21.15.119:0/1914016311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1120144230 con 0x7f112006b2b0 2024-04-25T10:07:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.415+0000 7f111e000640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11200ada20 0x7f1120143ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.415+0000 7f111ea00640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f112006b2b0 0x7f11201466b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f111000c430 tx=0x7f111000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.415+0000 7f1116a00640 1 -- 172.21.15.119:0/1914016311 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110011790 con 0x7f112006b2b0 2024-04-25T10:07:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.415+0000 7f1116a00640 1 -- 172.21.15.119:0/1914016311 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1110017070 con 0x7f112006b2b0 2024-04-25T10:07:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.415+0000 7f1116a00640 1 -- 172.21.15.119:0/1914016311 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110011bb0 con 0x7f112006b2b0 2024-04-25T10:07:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.415+0000 7f111fe00640 1 -- 172.21.15.119:0/1914016311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1120144510 con 0x7f112006b2b0 2024-04-25T10:07:24.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.415+0000 7f111fe00640 1 -- 172.21.15.119:0/1914016311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1120144ad0 con 0x7f112006b2b0 2024-04-25T10:07:24.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.417+0000 7f1116a00640 1 -- 172.21.15.119:0/1914016311 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f111001c460 con 0x7f112006b2b0 2024-04-25T10:07:24.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.419+0000 7f1116a00640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f110006b4d0 0x7f110006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.419+0000 7f1116a00640 1 -- 172.21.15.119:0/1914016311 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f111008a100 con 0x7f112006b2b0 2024-04-25T10:07:24.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.419+0000 7f111e000640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f110006b4d0 0x7f110006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.420+0000 7f1128a33740 1 Processor -- start 2024-04-25T10:07:24.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.420+0000 7f1128a33740 1 -- start start 2024-04-25T10:07:24.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.420+0000 7f111e000640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f110006b4d0 0x7f110006d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f110c006fd0 tx=0x7f110c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f1128a33740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc1523b30 0x561cc154b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f1128a33740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc1523f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f1128a33740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156c7a0 0x561cc15222f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f1128a33740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc1487620 con 0x561cc154beb0 2024-04-25T10:07:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f1128a33740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc1430ca0 con 0x561cc156c7a0 2024-04-25T10:07:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f1128a33740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cc14836c0 con 0x561cc1523b30 2024-04-25T10:07:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f111e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc1523f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f111e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc1523f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53414/0 (socket says 172.21.15.119:53414) 2024-04-25T10:07:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f111e000640 1 -- 172.21.15.119:0/2606362356 learned_addr learned my addr 172.21.15.119:0/2606362356 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f111ea00640 1 --2- 172.21.15.119:0/2606362356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc1523b30 0x561cc154b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f111e000640 1 -- 172.21.15.119:0/2606362356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc1523b30 msgr2=0x561cc154b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f111e000640 1 --2- 172.21.15.119:0/2606362356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc1523b30 0x561cc154b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f111e000640 1 -- 172.21.15.119:0/2606362356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156c7a0 msgr2=0x561cc15222f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:07:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f111e000640 1 --2- 172.21.15.119:0/2606362356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156c7a0 0x561cc15222f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.421+0000 7f111e000640 1 -- 172.21.15.119:0/2606362356 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cc1437f40 con 0x561cc154beb0 2024-04-25T10:07:24.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.422+0000 7f111e000640 1 --2- 172.21.15.119:0/2606362356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc1523f10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f110c00db30 tx=0x7f110c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24501097f7a33c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.422+0000 7f110b400640 1 -- 172.21.15.119:0/2606362356 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c019050 con 0x561cc154beb0 2024-04-25T10:07:24.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.422+0000 7f110b400640 1 -- 172.21.15.119:0/2606362356 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110c023070 con 0x561cc154beb0 2024-04-25T10:07:24.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.422+0000 7f110b400640 1 -- 172.21.15.119:0/2606362356 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c012c90 con 0x561cc154beb0 2024-04-25T10:07:24.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.423+0000 7f1128a33740 1 -- 172.21.15.119:0/2606362356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 msgr2=0x561cc1523f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.423+0000 7f1128a33740 1 --2- 172.21.15.119:0/2606362356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc1523f10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f110c00db30 tx=0x7f110c010000 comp rx=0 tx=0).stop 2024-04-25T10:07:24.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.423+0000 7f1128a33740 1 -- 172.21.15.119:0/2606362356 shutdown_connections 2024-04-25T10:07:24.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.423+0000 7f1128a33740 1 --2- 172.21.15.119:0/2606362356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156c7a0 0x561cc15222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.423+0000 7f1128a33740 1 --2- 172.21.15.119:0/2606362356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc1523f10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.423+0000 7f1128a33740 1 --2- 172.21.15.119:0/2606362356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc1523b30 0x561cc154b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.423+0000 7f1128a33740 1 -- 172.21.15.119:0/2606362356 >> 172.21.15.119:0/2606362356 conn(0x561cc1520d50 msgr2=0x561cc1569730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.423+0000 7f1128a33740 1 -- 172.21.15.119:0/2606362356 shutdown_connections 2024-04-25T10:07:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.423+0000 7f1128a33740 1 -- 172.21.15.119:0/2606362356 wait complete. 2024-04-25T10:07:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.423+0000 7f1128a33740 1 -- start start 2024-04-25T10:07:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.424+0000 7f1128a33740 1 Processor -- start 2024-04-25T10:07:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f1128a33740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc157b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f1128a33740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc156c7a0 0x561cc1542930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f1128a33740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156b830 0x561cc156bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f1128a33740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc154b4e0 con 0x561cc154beb0 2024-04-25T10:07:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f1128a33740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc14261f0 con 0x561cc156b830 2024-04-25T10:07:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f1128a33740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cc154aed0 con 0x561cc156c7a0 2024-04-25T10:07:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f1128a33740 10 client.0 _ll_register_callbacks cb 0x561cc1443ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f111ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc157b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f111ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc157b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53424/0 (socket says 172.21.15.119:53424) 2024-04-25T10:07:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f111ea00640 1 -- 172.21.15.119:0/1064228320 learned_addr learned my addr 172.21.15.119:0/1064228320 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f111ea00640 1 -- 172.21.15.119:0/1064228320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc156c7a0 msgr2=0x561cc1542930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f111ea00640 1 --2- 172.21.15.119:0/1064228320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc156c7a0 0x561cc1542930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f111ea00640 1 -- 172.21.15.119:0/1064228320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156b830 msgr2=0x561cc156bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f111ea00640 1 --2- 172.21.15.119:0/1064228320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156b830 0x561cc156bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f111ea00640 1 -- 172.21.15.119:0/1064228320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cc1445c50 con 0x561cc154beb0 2024-04-25T10:07:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f111ea00640 1 --2- 172.21.15.119:0/1064228320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc157b050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f111000a040 tx=0x7f1110058930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6d71c4c4dca91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.425+0000 7f10ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:07:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.426+0000 7f10ffe00640 1 -- 172.21.15.119:0/1064228320 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110070070 con 0x561cc154beb0 2024-04-25T10:07:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.426+0000 7f10ffe00640 1 -- 172.21.15.119:0/1064228320 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1110054ee0 con 0x561cc154beb0 2024-04-25T10:07:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.426+0000 7f1128a33740 1 -- 172.21.15.119:0/1064228320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f110c023070 con 0x561cc154beb0 2024-04-25T10:07:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.426+0000 7f10ffe00640 1 -- 172.21.15.119:0/1064228320 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1110088170 con 0x561cc154beb0 2024-04-25T10:07:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.426+0000 7f1128a33740 10 client.4602 fetch_fsmap learned FSMap version 7 2024-04-25T10:07:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.426+0000 7f1128a33740 1 -- 172.21.15.119:0/1064228320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x561cc119c2d0 con 0x561cc154beb0 2024-04-25T10:07:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.426+0000 7f10ffe00640 1 -- 172.21.15.119:0/1064228320 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110c023070 con 0x561cc154beb0 2024-04-25T10:07:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.426+0000 7f10ffe00640 1 -- 172.21.15.119:0/1064228320 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1110070210 con 0x561cc154beb0 2024-04-25T10:07:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.426+0000 7f1128a33740 10 client.4602 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T10:07:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.426+0000 7f1128a33740 10 client.4602 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:07:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.426+0000 7f1128a33740 20 client.4602 populate_metadata read hostname 'smithi119' 2024-04-25T10:07:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.426+0000 7f1128a33740 1 --2- 172.21.15.119:0/1064228320 >> [v2:172.21.15.119:6835/1632024048,v1:172.21.15.119:6837/1632024048] conn(0x561cc1540ca0 0x561cc1541080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.426+0000 7f1128a33740 4 client.4602 mds_command: new command op to 4261 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.426+0000 7f1128a33740 1 -- 172.21.15.119:0/1064228320 --> [v2:172.21.15.119:6835/1632024048,v1:172.21.15.119:6837/1632024048] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cc1494b60 con 0x561cc1540ca0 2024-04-25T10:07:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.427+0000 7f111e000640 1 --2- 172.21.15.119:0/1064228320 >> [v2:172.21.15.119:6835/1632024048,v1:172.21.15.119:6837/1632024048] conn(0x561cc1540ca0 0x561cc1541080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.427+0000 7f111e000640 1 --2- 172.21.15.119:0/1064228320 >> [v2:172.21.15.119:6835/1632024048,v1:172.21.15.119:6837/1632024048] conn(0x561cc1540ca0 0x561cc1541080 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bd405a3aa2cfc04 server_cookie=d70f6cbb8f1ccdc9 in_seq=0 out_seq=0 2024-04-25T10:07:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.427+0000 7f10ffe00640 10 client.4602 ms_handle_connect on v2:172.21.15.119:6835/1632024048 2024-04-25T10:07:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.428+0000 7f10ffe00640 1 -- 172.21.15.119:0/1064228320 <== mds.0 v2:172.21.15.119:6835/1632024048 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561cc1494b60 con 0x561cc1540ca0 2024-04-25T10:07:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.428+0000 7f10ffe00640 10 client.4602 handle_command_reply: tid=0 2024-04-25T10:07:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.428+0000 7f1128a33740 1 client.4602 shutdown 2024-04-25T10:07:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.428+0000 7f1128a33740 1 -- 172.21.15.119:0/1064228320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 msgr2=0x561cc157b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.428+0000 7f1128a33740 1 --2- 172.21.15.119:0/1064228320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc157b050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f111000a040 tx=0x7f1110058930 comp rx=0 tx=0).stop 2024-04-25T10:07:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.428+0000 7f1128a33740 1 -- 172.21.15.119:0/1064228320 shutdown_connections 2024-04-25T10:07:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.428+0000 7f1128a33740 1 --2- 172.21.15.119:0/1064228320 >> [v2:172.21.15.119:6835/1632024048,v1:172.21.15.119:6837/1632024048] conn(0x561cc1540ca0 0x561cc1541080 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.428+0000 7f1128a33740 1 --2- 172.21.15.119:0/1064228320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156b830 0x561cc156bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.428+0000 7f1128a33740 1 --2- 172.21.15.119:0/1064228320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc156c7a0 0x561cc1542930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.428+0000 7f1128a33740 1 --2- 172.21.15.119:0/1064228320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc157b050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.428+0000 7f10ffe00640 0 client.4602 ms_handle_reset on v2:172.21.15.119:6835/1632024048 2024-04-25T10:07:24.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.428+0000 7f1128a33740 1 -- 172.21.15.119:0/1064228320 >> 172.21.15.119:0/1064228320 conn(0x561cc1523b30 msgr2=0x561cc151d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.429+0000 7f1128a33740 1 -- 172.21.15.119:0/1064228320 shutdown_connections 2024-04-25T10:07:24.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.429+0000 7f1128a33740 1 -- 172.21.15.119:0/1064228320 wait complete. 2024-04-25T10:07:24.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.429+0000 7f1128a33740 20 client.4602 trim_cache size 0 max 16384 2024-04-25T10:07:24.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.445+0000 7f1128a33740 1 Processor -- start 2024-04-25T10:07:24.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.445+0000 7f1128a33740 1 -- start start 2024-04-25T10:07:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.446+0000 7f1128a33740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc1583db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.446+0000 7f1128a33740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156b830 0x561cc151d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.446+0000 7f1128a33740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc156c7a0 0x561cc151ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.446+0000 7f1128a33740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11201f7970 con 0x561cc154beb0 2024-04-25T10:07:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.446+0000 7f1128a33740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1120159590 con 0x561cc156b830 2024-04-25T10:07:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.446+0000 7f1128a33740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11201f7370 con 0x561cc156c7a0 2024-04-25T10:07:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.446+0000 7f111ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc1583db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.446+0000 7f111ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc1583db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53428/0 (socket says 172.21.15.119:53428) 2024-04-25T10:07:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.446+0000 7f111ea00640 1 -- 172.21.15.119:0/2338710618 learned_addr learned my addr 172.21.15.119:0/2338710618 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.446+0000 7f111ea00640 1 -- 172.21.15.119:0/2338710618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc156c7a0 msgr2=0x561cc151ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.446+0000 7f111ea00640 1 --2- 172.21.15.119:0/2338710618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc156c7a0 0x561cc151ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.446+0000 7f111ea00640 1 -- 172.21.15.119:0/2338710618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156b830 msgr2=0x561cc151d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.446+0000 7f111ea00640 1 --2- 172.21.15.119:0/2338710618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156b830 0x561cc151d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.446+0000 7f111ea00640 1 -- 172.21.15.119:0/2338710618 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cc12666b0 con 0x561cc154beb0 2024-04-25T10:07:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.447+0000 7f111ea00640 1 --2- 172.21.15.119:0/2338710618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc1583db0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1110058570 tx=0x7f1110013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa62d7b4f532028 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.447+0000 7f110aa00640 1 -- 172.21.15.119:0/2338710618 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110077030 con 0x561cc154beb0 2024-04-25T10:07:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.447+0000 7f110aa00640 1 -- 172.21.15.119:0/2338710618 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1110088310 con 0x561cc154beb0 2024-04-25T10:07:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.447+0000 7f110aa00640 1 -- 172.21.15.119:0/2338710618 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110070040 con 0x561cc154beb0 2024-04-25T10:07:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.447+0000 7f1128a33740 1 -- 172.21.15.119:0/2338710618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 msgr2=0x561cc1583db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.447+0000 7f1128a33740 1 --2- 172.21.15.119:0/2338710618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc1583db0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1110058570 tx=0x7f1110013000 comp rx=0 tx=0).stop 2024-04-25T10:07:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.447+0000 7f1128a33740 1 -- 172.21.15.119:0/2338710618 shutdown_connections 2024-04-25T10:07:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.447+0000 7f1128a33740 1 --2- 172.21.15.119:0/2338710618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc156c7a0 0x561cc151ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.447+0000 7f1128a33740 1 --2- 172.21.15.119:0/2338710618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156b830 0x561cc151d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.447+0000 7f1128a33740 1 --2- 172.21.15.119:0/2338710618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc154beb0 0x561cc1583db0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.447+0000 7f1128a33740 1 -- 172.21.15.119:0/2338710618 >> 172.21.15.119:0/2338710618 conn(0x561cc1523b30 msgr2=0x561cc1569530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.447+0000 7f1128a33740 1 -- 172.21.15.119:0/2338710618 shutdown_connections 2024-04-25T10:07:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.448+0000 7f1128a33740 1 -- 172.21.15.119:0/2338710618 wait complete. 2024-04-25T10:07:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.448+0000 7f1128a33740 1 -- start start 2024-04-25T10:07:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.448+0000 7f1128a33740 1 Processor -- start 2024-04-25T10:07:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.449+0000 7f1128a33740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc154beb0 0x561cc15bd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.449+0000 7f1128a33740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156b830 0x561cc151d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.449+0000 7f1128a33740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc156c7a0 0x561cc151ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.449+0000 7f1128a33740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc1542e70 con 0x561cc156c7a0 2024-04-25T10:07:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.449+0000 7f1128a33740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc15425e0 con 0x561cc156b830 2024-04-25T10:07:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.449+0000 7f1128a33740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cc157a820 con 0x561cc154beb0 2024-04-25T10:07:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.449+0000 7f1128a33740 10 client.0 _ll_register_callbacks cb 0x7f1110054b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.450+0000 7f111f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc156c7a0 0x561cc151ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.450+0000 7f111f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc156c7a0 0x561cc151ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53444/0 (socket says 172.21.15.119:53444) 2024-04-25T10:07:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.450+0000 7f111f400640 1 -- 172.21.15.119:0/1504860864 learned_addr learned my addr 172.21.15.119:0/1504860864 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.450+0000 7f111e000640 1 --2- 172.21.15.119:0/1504860864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156b830 0x561cc151d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.450+0000 7f111f400640 1 -- 172.21.15.119:0/1504860864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc154beb0 msgr2=0x561cc15bd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.450+0000 7f111f400640 1 --2- 172.21.15.119:0/1504860864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc154beb0 0x561cc15bd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.450+0000 7f111f400640 1 -- 172.21.15.119:0/1504860864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156b830 msgr2=0x561cc151d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.450+0000 7f111f400640 1 --2- 172.21.15.119:0/1504860864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156b830 0x561cc151d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.450+0000 7f111f400640 1 -- 172.21.15.119:0/1504860864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cc153fa40 con 0x561cc156c7a0 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.450+0000 7f111f400640 1 --2- 172.21.15.119:0/1504860864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc156c7a0 0x561cc151ddf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1118009e50 tx=0x7f1118031ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1f1a8a95959c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.450+0000 7f10ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.450+0000 7f10ffe00640 1 -- 172.21.15.119:0/1504860864 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118041020 con 0x561cc156c7a0 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.450+0000 7f10ffe00640 1 -- 172.21.15.119:0/1504860864 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f111803c470 con 0x561cc156c7a0 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.450+0000 7f1128a33740 1 -- 172.21.15.119:0/1504860864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1110088310 con 0x561cc156c7a0 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.451+0000 7f10ffe00640 1 -- 172.21.15.119:0/1504860864 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f111804b050 con 0x561cc156c7a0 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.451+0000 7f1128a33740 10 client.4608 fetch_fsmap learned FSMap version 7 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.451+0000 7f1128a33740 1 -- 172.21.15.119:0/1504860864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x561cc1541ce0 con 0x561cc156c7a0 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.451+0000 7f10ffe00640 1 -- 172.21.15.119:0/1504860864 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1110088310 con 0x561cc156c7a0 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.451+0000 7f10ffe00640 1 -- 172.21.15.119:0/1504860864 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f111804e020 con 0x561cc156c7a0 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.451+0000 7f1128a33740 10 client.4608 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.451+0000 7f1128a33740 10 client.4608 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.451+0000 7f1128a33740 20 client.4608 populate_metadata read hostname 'smithi119' 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.451+0000 7f1128a33740 1 --2- 172.21.15.119:0/1504860864 >> [v2:172.21.15.119:6835/1632024048,v1:172.21.15.119:6837/1632024048] conn(0x561cc1541680 0x561cc1578be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.451+0000 7f1128a33740 4 client.4608 mds_command: new command op to 4261 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.451+0000 7f1128a33740 1 -- 172.21.15.119:0/1504860864 --> [v2:172.21.15.119:6835/1632024048,v1:172.21.15.119:6837/1632024048] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561cc148eeb0 con 0x561cc1541680 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.451+0000 7f111ea00640 1 --2- 172.21.15.119:0/1504860864 >> [v2:172.21.15.119:6835/1632024048,v1:172.21.15.119:6837/1632024048] conn(0x561cc1541680 0x561cc1578be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f111ea00640 1 --2- 172.21.15.119:0/1504860864 >> [v2:172.21.15.119:6835/1632024048,v1:172.21.15.119:6837/1632024048] conn(0x561cc1541680 0x561cc1578be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f648afd863026cd8 server_cookie=3940313365bc5169 in_seq=0 out_seq=0 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f10ffe00640 10 client.4608 ms_handle_connect on v2:172.21.15.119:6835/1632024048 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f10ffe00640 1 -- 172.21.15.119:0/1504860864 <== mds.0 v2:172.21.15.119:6835/1632024048 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x561cc148eeb0 con 0x561cc1541680 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f10ffe00640 10 client.4608 handle_command_reply: tid=0 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f1128a33740 1 client.4608 shutdown 2024-04-25T10:07:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f1128a33740 1 -- 172.21.15.119:0/1504860864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc156c7a0 msgr2=0x561cc151ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f1128a33740 1 --2- 172.21.15.119:0/1504860864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc156c7a0 0x561cc151ddf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1118009e50 tx=0x7f1118031ae0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f1128a33740 1 -- 172.21.15.119:0/1504860864 shutdown_connections 2024-04-25T10:07:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f1128a33740 1 --2- 172.21.15.119:0/1504860864 >> [v2:172.21.15.119:6835/1632024048,v1:172.21.15.119:6837/1632024048] conn(0x561cc1541680 0x561cc1578be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f1128a33740 1 --2- 172.21.15.119:0/1504860864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cc156c7a0 0x561cc151ddf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f1128a33740 1 --2- 172.21.15.119:0/1504860864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561cc156b830 0x561cc151d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f1128a33740 1 --2- 172.21.15.119:0/1504860864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561cc154beb0 0x561cc15bd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f10ffe00640 0 client.4608 ms_handle_reset on v2:172.21.15.119:6835/1632024048 2024-04-25T10:07:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f1128a33740 1 -- 172.21.15.119:0/1504860864 >> 172.21.15.119:0/1504860864 conn(0x561cc1523b30 msgr2=0x561cc15481a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f1128a33740 1 -- 172.21.15.119:0/1504860864 shutdown_connections 2024-04-25T10:07:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f1128a33740 1 -- 172.21.15.119:0/1504860864 wait complete. 2024-04-25T10:07:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.452+0000 7f1128a33740 20 client.4608 trim_cache size 0 max 16384 2024-04-25T10:07:24.466 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:07:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "e0b0f428-b441-4237-a4d1-ed53321f87a5", 2024-04-25T10:07:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-04-25T10:07:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 4261, 2024-04-25T10:07:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-04-25T10:07:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-04-25T10:07:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-04-25T10:07:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 17.148782126, 2024-04-25T10:07:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 7, 2024-04-25T10:07:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 20, 2024-04-25T10:07:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T10:07:24.466 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 22.210358708000001 2024-04-25T10:07:24.467 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:07:24.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.453+0000 7f111fe00640 1 -- 172.21.15.119:0/1914016311 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f110006b4d0 msgr2=0x7f110006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.453+0000 7f111fe00640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f110006b4d0 0x7f110006d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f110c006fd0 tx=0x7f110c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.453+0000 7f111fe00640 1 -- 172.21.15.119:0/1914016311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f112006b2b0 msgr2=0x7f11201466b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.453+0000 7f111fe00640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f112006b2b0 0x7f11201466b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f111000c430 tx=0x7f111000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.453+0000 7f111fe00640 1 -- 172.21.15.119:0/1914016311 shutdown_connections 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.453+0000 7f111fe00640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f110006b4d0 0x7f110006d990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.453+0000 7f111fe00640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11200b2f90 0x7f11201452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.453+0000 7f111fe00640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f11200ada20 0x7f1120143ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.453+0000 7f111fe00640 1 --2- 172.21.15.119:0/1914016311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f112006b2b0 0x7f11201466b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.453+0000 7f111fe00640 1 -- 172.21.15.119:0/1914016311 >> 172.21.15.119:0/1914016311 conn(0x7f1120066ff0 msgr2=0x7f112006aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.453+0000 7f111fe00640 1 -- 172.21.15.119:0/1914016311 shutdown_connections 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:24.453+0000 7f111fe00640 1 -- 172.21.15.119:0/1914016311 wait complete. 2024-04-25T10:07:24.475 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T10:07:24.502 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T10:07:24.502 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T10:07:24.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:24.503 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:07:24.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:24.557 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2024-04-25T10:07:24.636 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2024-04-25T10:07:24.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:07:24.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:24.640 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:07:24.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:24.688 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2024-04-25T10:07:24.776 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2024-04-25T10:07:24.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:07:24.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:24.781 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-25T10:07:24.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:24.842 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-04-25T10:07:24.929 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2024-04-25T10:07:24.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:07:24.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:24.933 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-25T10:07:24.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:24.986 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-04-25T10:07:25.068 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2024-04-25T10:07:25.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:07:25.074 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-25T10:07:25.075 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:07:25.075 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:25.075 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:07:25.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.277+0000 7f5e47200640 1 Processor -- start 2024-04-25T10:07:25.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.277+0000 7f5e47200640 1 -- start start 2024-04-25T10:07:25.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e47200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5e480b5050 0x7f5e4815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e47200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e4806e430 0x7f5e4815bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e47200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e480cd440 con 0x7f5e48144470 2024-04-25T10:07:25.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e47200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e48145340 con 0x7f5e4806e430 2024-04-25T10:07:25.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e47200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e480cd720 con 0x7f5e480b5050 2024-04-25T10:07:25.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e45e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5e48144470 0x7f5e480b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48702/0 (socket says 172.21.15.119:48702) 2024-04-25T10:07:25.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e45e00640 1 -- 172.21.15.119:0/3522978882 learned_addr learned my addr 172.21.15.119:0/3522978882 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:25.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e46800640 1 --2- 172.21.15.119:0/3522978882 >> v2:172.21.15.119:3300/0 conn(0x7f5e4806e430 0x7f5e4815bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e3d400640 1 --2- 172.21.15.119:0/3522978882 >> v2:172.21.15.132:3300/0 conn(0x7f5e480b5050 0x7f5e4815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e45400640 1 -- 172.21.15.119:0/3522978882 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464478296 0 0) 0x7f5e480cd440 con 0x7f5e48144470 2024-04-25T10:07:25.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e45400640 1 -- 172.21.15.119:0/3522978882 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e2c003520 con 0x7f5e48144470 2024-04-25T10:07:25.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e46800640 1 -- 172.21.15.119:0/3522978882 >> v2:172.21.15.132:3300/0 conn(0x7f5e480b5050 msgr2=0x7f5e4815fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:25.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e46800640 1 --2- 172.21.15.119:0/3522978882 >> v2:172.21.15.132:3300/0 conn(0x7f5e480b5050 0x7f5e4815fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e46800640 1 -- 172.21.15.119:0/3522978882 >> v1:172.21.15.119:6789/0 conn(0x7f5e48144470 legacy=0x7f5e480b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.279+0000 7f5e46800640 1 -- 172.21.15.119:0/3522978882 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e480cda00 con 0x7f5e4806e430 2024-04-25T10:07:25.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.280+0000 7f5e46800640 1 --2- 172.21.15.119:0/3522978882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4806e430 0x7f5e4815bcf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5e4806f500 tx=0x7f5e4002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cfea4fd50413db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.281+0000 7f5e45400640 1 -- 172.21.15.119:0/3522978882 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40037070 con 0x7f5e4806e430 2024-04-25T10:07:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.281+0000 7f5e45400640 1 -- 172.21.15.119:0/3522978882 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e40032440 con 0x7f5e4806e430 2024-04-25T10:07:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.281+0000 7f5e45400640 1 -- 172.21.15.119:0/3522978882 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40032810 con 0x7f5e4806e430 2024-04-25T10:07:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.281+0000 7f5e47200640 1 -- 172.21.15.119:0/3522978882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4806e430 msgr2=0x7f5e4815bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.281+0000 7f5e47200640 1 --2- 172.21.15.119:0/3522978882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4806e430 0x7f5e4815bcf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5e4806f500 tx=0x7f5e4002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.281+0000 7f5e47200640 1 -- 172.21.15.119:0/3522978882 shutdown_connections 2024-04-25T10:07:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.281+0000 7f5e47200640 1 --2- 172.21.15.119:0/3522978882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4806e430 0x7f5e4815bcf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.281+0000 7f5e47200640 1 --2- 172.21.15.119:0/3522978882 >> v2:172.21.15.132:3300/0 conn(0x7f5e480b5050 0x7f5e4815fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.281+0000 7f5e47200640 1 -- 172.21.15.119:0/3522978882 >> 172.21.15.119:0/3522978882 conn(0x7f5e4806b1d0 msgr2=0x7f5e480b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.282+0000 7f5e47200640 1 -- 172.21.15.119:0/3522978882 shutdown_connections 2024-04-25T10:07:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.282+0000 7f5e47200640 1 -- 172.21.15.119:0/3522978882 wait complete. 2024-04-25T10:07:25.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.283+0000 7f5e47200640 1 Processor -- start 2024-04-25T10:07:25.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.283+0000 7f5e47200640 1 -- start start 2024-04-25T10:07:25.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.284+0000 7f5e47200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e4806e430 0x7f5e481f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.284+0000 7f5e47200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e480b5050 0x7f5e480b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.284+0000 7f5e47200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e48144470 0x7f5e4815e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.284+0000 7f5e47200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e480ce200 con 0x7f5e480b5050 2024-04-25T10:07:25.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.284+0000 7f5e47200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e48144850 con 0x7f5e48144470 2024-04-25T10:07:25.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.284+0000 7f5e47200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4815c230 con 0x7f5e4806e430 2024-04-25T10:07:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.285+0000 7f5e3d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e480b5050 0x7f5e480b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.285+0000 7f5e3d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e480b5050 0x7f5e480b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53460/0 (socket says 172.21.15.119:53460) 2024-04-25T10:07:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.285+0000 7f5e3d400640 1 -- 172.21.15.119:0/2496266343 learned_addr learned my addr 172.21.15.119:0/2496266343 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.285+0000 7f5e3d400640 1 -- 172.21.15.119:0/2496266343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e4806e430 msgr2=0x7f5e481f2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.285+0000 7f5e3d400640 1 --2- 172.21.15.119:0/2496266343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e4806e430 0x7f5e481f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.285+0000 7f5e3d400640 1 -- 172.21.15.119:0/2496266343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e48144470 msgr2=0x7f5e4815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.285+0000 7f5e3d400640 1 --2- 172.21.15.119:0/2496266343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e48144470 0x7f5e4815e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.285+0000 7f5e3d400640 1 -- 172.21.15.119:0/2496266343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e480b42e0 con 0x7f5e480b5050 2024-04-25T10:07:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.285+0000 7f5e3d400640 1 --2- 172.21.15.119:0/2496266343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e480b5050 0x7f5e480b3140 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000d800 tx=0x7f5e3000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.286+0000 7f5e3ea00640 1 -- 172.21.15.119:0/2496266343 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3000b6c0 con 0x7f5e480b5050 2024-04-25T10:07:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.286+0000 7f5e3ea00640 1 -- 172.21.15.119:0/2496266343 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e30019070 con 0x7f5e480b5050 2024-04-25T10:07:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.286+0000 7f5e47200640 1 -- 172.21.15.119:0/2496266343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e480b3a40 con 0x7f5e480b5050 2024-04-25T10:07:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.286+0000 7f5e3ea00640 1 -- 172.21.15.119:0/2496266343 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3000ba80 con 0x7f5e480b5050 2024-04-25T10:07:25.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.286+0000 7f5e47200640 1 -- 172.21.15.119:0/2496266343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e480b4000 con 0x7f5e480b5050 2024-04-25T10:07:25.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.287+0000 7f5e47200640 1 -- 172.21.15.119:0/2496266343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e480749b0 con 0x7f5e480b5050 2024-04-25T10:07:25.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.288+0000 7f5e3ea00640 1 -- 172.21.15.119:0/2496266343 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e3000bc20 con 0x7f5e480b5050 2024-04-25T10:07:25.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.289+0000 7f5e3ea00640 1 --2- 172.21.15.119:0/2496266343 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e2406b4d0 0x7f5e2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.289+0000 7f5e3ea00640 1 -- 172.21.15.119:0/2496266343 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e3008a780 con 0x7f5e480b5050 2024-04-25T10:07:25.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.289+0000 7f5e45e00640 1 --2- 172.21.15.119:0/2496266343 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e2406b4d0 0x7f5e2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.290+0000 7f5e45e00640 1 --2- 172.21.15.119:0/2496266343 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e2406b4d0 0x7f5e2406d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5e380012e0 tx=0x7f5e380015b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:25.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.293+0000 7f5e3ea00640 1 -- 172.21.15.119:0/2496266343 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e3001c360 con 0x7f5e480b5050 2024-04-25T10:07:25.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.527+0000 7f5e47200640 1 -- 172.21.15.119:0/2496266343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5e480b19c0 con 0x7f5e480b5050 2024-04-25T10:07:25.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.527+0000 7f5e3ea00640 1 -- 172.21.15.119:0/2496266343 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5e30059560 con 0x7f5e480b5050 2024-04-25T10:07:25.533 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:07:25.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.530+0000 7f5e47200640 1 -- 172.21.15.119:0/2496266343 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e2406b4d0 msgr2=0x7f5e2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.530+0000 7f5e47200640 1 --2- 172.21.15.119:0/2496266343 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e2406b4d0 0x7f5e2406d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5e380012e0 tx=0x7f5e380015b0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.530+0000 7f5e47200640 1 -- 172.21.15.119:0/2496266343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e480b5050 msgr2=0x7f5e480b3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.530+0000 7f5e47200640 1 --2- 172.21.15.119:0/2496266343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e480b5050 0x7f5e480b3140 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000d800 tx=0x7f5e3000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.530+0000 7f5e47200640 1 -- 172.21.15.119:0/2496266343 shutdown_connections 2024-04-25T10:07:25.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.530+0000 7f5e47200640 1 --2- 172.21.15.119:0/2496266343 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e2406b4d0 0x7f5e2406d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.530+0000 7f5e47200640 1 --2- 172.21.15.119:0/2496266343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e48144470 0x7f5e4815e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.530+0000 7f5e47200640 1 --2- 172.21.15.119:0/2496266343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e480b5050 0x7f5e480b3140 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.530+0000 7f5e47200640 1 --2- 172.21.15.119:0/2496266343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e4806e430 0x7f5e481f2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.530+0000 7f5e47200640 1 -- 172.21.15.119:0/2496266343 >> 172.21.15.119:0/2496266343 conn(0x7f5e4806b1d0 msgr2=0x7f5e48143080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:25.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.530+0000 7f5e47200640 1 -- 172.21.15.119:0/2496266343 shutdown_connections 2024-04-25T10:07:25.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:25.531+0000 7f5e47200640 1 -- 172.21.15.119:0/2496266343 wait complete. 2024-04-25T10:07:25.555 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:07:25.555 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:07:25.555 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:07:25.555 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:07:25.555 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:25.555 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:07:25.555 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:07:25.555 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:07:25.556 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:25.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:25.556 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:07:25.603 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:25.604 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:25.604 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:25.604 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:25.604 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:25.604 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:25.604 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:25.604 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:25.604 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:07:25.605 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:25.605 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:25.605 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:07:25.605 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:25.605 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:25.605 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:07:25.605 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:25.605 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:07:25.606 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42644sec preferred_lft 42644sec 2024-04-25T10:07:25.606 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:07:25.606 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:25.606 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:07:25.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:25.607 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:07:25.607 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:07:25.607 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:07:25.607 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr flush dev ceph-brx 2024-04-25T10:07:25.607 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set ceph-brx up 2024-04-25T10:07:25.608 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:07:25.608 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:07:26.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:26.033 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:07:26.100 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:07:26.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:26.104 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:07:26.155 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:07:26.155 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:07:26.155 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:07:26.155 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:07:26.155 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:07:26.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:26.156 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:07:26.156 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:07:26.156 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:07:26.156 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:07:26.156 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:07:26.156 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:07:26.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:26.841 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:07:26.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:26.894 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2024-04-25T10:07:26.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:26.940 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:07:26.941 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:07:26.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:26.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T10:07:26.941 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:07:27.109 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T10:07:27.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:27.109 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:07:27.109 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:07:27.109 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T10:07:27.109 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:07:27.109 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T10:07:27.109 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T10:07:27.109 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T10:07:27.110 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:07:27.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:27.344 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:07:27.344 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:07:27.344 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:07:27.344 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:07:27.344 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:07:27.475 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:27.475 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:27.522 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:27.522 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:27.523 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:27.614 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:07:27.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:27.614 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:27.744 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:27.745 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:27.745 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:07:28.445 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:28.445 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:28.445 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:07:28.445 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:07:28.445 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:28.487 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:28.487 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:28.487 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:07:28.487 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:07:28.487 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:28.487 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:07:28.487 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:07:28.487 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:07:28.496 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:07:28.496 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:07:28.496 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:07:28.496 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:07:28.496 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:07:28.501 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:07:28.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:28.501 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:28.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:28.575 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:28.635 INFO:teuthology.orchestra.run.smithi119.stdout:client4427 2024-04-25T10:07:28.635 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:07:28.635 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:28.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:28.687 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:28.795 INFO:teuthology.orchestra.run.smithi119.stdout:client4427 2024-04-25T10:07:28.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:28.796 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:28.861 INFO:teuthology.orchestra.run.smithi119.stdout:client4427 2024-04-25T10:07:28.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:28.861 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e0b0f428-b441-4237-a4d1-ed53321f87a5.client4427/status') 2024-04-25T10:07:28.939 INFO:teuthology.orchestra.run.smithi119.stdout:instance: client.4427 (3)172.21.15.119:0/1308172419 2024-04-25T10:07:28.939 INFO:teuthology.orchestra.run.smithi119.stdout:blocklisted: false 2024-04-25T10:07:28.939 INFO:teuthology.orchestra.run.smithi119.stderr:0+1 records in 2024-04-25T10:07:28.939 INFO:teuthology.orchestra.run.smithi119.stderr:0+1 records out 2024-04-25T10:07:28.940 INFO:teuthology.orchestra.run.smithi119.stderr:71 bytes copied, 6.4416e-05 s, 1.1 MB/s 2024-04-25T10:07:28.943 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-25T10:07:28.943 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:07:28.943 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:07:28.943 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:07:29.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.138+0000 7f03f1800640 1 Processor -- start 2024-04-25T10:07:29.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.138+0000 7f03f1800640 1 -- start start 2024-04-25T10:07:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.139+0000 7f03f1800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03ec144470 0x7f03ec15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.139+0000 7f03f1800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f03ec0b5050 0x7f03ec15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.139+0000 7f03f1800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ec0cd440 con 0x7f03ec06e430 2024-04-25T10:07:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.139+0000 7f03f1800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ec145340 con 0x7f03ec0b5050 2024-04-25T10:07:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.139+0000 7f03f1800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec0cd720 con 0x7f03ec144470 2024-04-25T10:07:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.139+0000 7f03f0e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f03ec0b5050 0x7f03ec15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.140+0000 7f03f0e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f03ec0b5050 0x7f03ec15bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53474/0 (socket says 172.21.15.119:53474) 2024-04-25T10:07:29.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.140+0000 7f03f0e00640 1 -- 172.21.15.119:0/3288314358 learned_addr learned my addr 172.21.15.119:0/3288314358 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:29.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.140+0000 7f03ebe00640 1 --2- 172.21.15.119:0/3288314358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03ec144470 0x7f03ec15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.140+0000 7f03f0e00640 1 -- 172.21.15.119:0/3288314358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03ec144470 msgr2=0x7f03ec15fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:29.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.140+0000 7f03f0e00640 1 --2- 172.21.15.119:0/3288314358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03ec144470 0x7f03ec15fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.140+0000 7f03f0e00640 1 -- 172.21.15.119:0/3288314358 >> v1:172.21.15.132:6789/0 conn(0x7f03ec06e430 legacy=0x7f03ec0b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.140+0000 7f03f0e00640 1 -- 172.21.15.119:0/3288314358 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec0cda00 con 0x7f03ec0b5050 2024-04-25T10:07:29.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.141+0000 7f03f0e00640 1 --2- 172.21.15.119:0/3288314358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ec0b5050 0x7f03ec15bcf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009e50 tx=0x7f03e402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb3c7cc4b38d97d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.141+0000 7f03eb400640 1 -- 172.21.15.119:0/3288314358 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e4036070 con 0x7f03ec0b5050 2024-04-25T10:07:29.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.142+0000 7f03eb400640 1 -- 172.21.15.119:0/3288314358 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e4031600 con 0x7f03ec0b5050 2024-04-25T10:07:29.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.142+0000 7f03eb400640 1 -- 172.21.15.119:0/3288314358 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e40319d0 con 0x7f03ec0b5050 2024-04-25T10:07:29.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.142+0000 7f03f1800640 1 -- 172.21.15.119:0/3288314358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ec0b5050 msgr2=0x7f03ec15bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.142+0000 7f03f1800640 1 --2- 172.21.15.119:0/3288314358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ec0b5050 0x7f03ec15bcf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009e50 tx=0x7f03e402a710 comp rx=0 tx=0).stop 2024-04-25T10:07:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.142+0000 7f03f1800640 1 -- 172.21.15.119:0/3288314358 shutdown_connections 2024-04-25T10:07:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.142+0000 7f03f1800640 1 --2- 172.21.15.119:0/3288314358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ec0b5050 0x7f03ec15bcf0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.142+0000 7f03f1800640 1 --2- 172.21.15.119:0/3288314358 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03ec144470 0x7f03ec15fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.142+0000 7f03f1800640 1 -- 172.21.15.119:0/3288314358 >> 172.21.15.119:0/3288314358 conn(0x7f03ec06b1d0 msgr2=0x7f03ec0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.142+0000 7f03f1800640 1 -- 172.21.15.119:0/3288314358 shutdown_connections 2024-04-25T10:07:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.142+0000 7f03f1800640 1 -- 172.21.15.119:0/3288314358 wait complete. 2024-04-25T10:07:29.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.144+0000 7f03f1800640 1 Processor -- start 2024-04-25T10:07:29.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.144+0000 7f03f1800640 1 -- start start 2024-04-25T10:07:29.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.145+0000 7f03f1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ec06e430 0x7f03ec1f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.145+0000 7f03f1800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03ec0b5050 0x7f03ec0b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.145+0000 7f03f1800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03ec144470 0x7f03ec142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.145+0000 7f03f1800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec0ce200 con 0x7f03ec06e430 2024-04-25T10:07:29.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.145+0000 7f03f1800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec144850 con 0x7f03ec0b5050 2024-04-25T10:07:29.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.145+0000 7f03f1800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec15c230 con 0x7f03ec144470 2024-04-25T10:07:29.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.145+0000 7f03ebe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ec06e430 0x7f03ec1f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.145+0000 7f03ebe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ec06e430 0x7f03ec1f6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53476/0 (socket says 172.21.15.119:53476) 2024-04-25T10:07:29.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.145+0000 7f03ebe00640 1 -- 172.21.15.119:0/3577507148 learned_addr learned my addr 172.21.15.119:0/3577507148 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:29.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.146+0000 7f03e3400640 1 --2- 172.21.15.119:0/3577507148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03ec0b5050 0x7f03ec0b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.146+0000 7f03ebe00640 1 -- 172.21.15.119:0/3577507148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03ec144470 msgr2=0x7f03ec142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.146+0000 7f03ebe00640 1 --2- 172.21.15.119:0/3577507148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03ec144470 0x7f03ec142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.146+0000 7f03ebe00640 1 -- 172.21.15.119:0/3577507148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03ec0b5050 msgr2=0x7f03ec0b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.146+0000 7f03ebe00640 1 --2- 172.21.15.119:0/3577507148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03ec0b5050 0x7f03ec0b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.146+0000 7f03ebe00640 1 -- 172.21.15.119:0/3577507148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec15e870 con 0x7f03ec06e430 2024-04-25T10:07:29.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.146+0000 7f03ebe00640 1 --2- 172.21.15.119:0/3577507148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ec06e430 0x7f03ec1f6590 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00d5d0 tx=0x7f03dc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.146+0000 7f03e8c00640 1 -- 172.21.15.119:0/3577507148 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0125f0 con 0x7f03ec06e430 2024-04-25T10:07:29.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.146+0000 7f03e8c00640 1 -- 172.21.15.119:0/3577507148 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc019070 con 0x7f03ec06e430 2024-04-25T10:07:29.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.147+0000 7f03e8c00640 1 -- 172.21.15.119:0/3577507148 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0129f0 con 0x7f03ec06e430 2024-04-25T10:07:29.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.147+0000 7f03f1800640 1 -- 172.21.15.119:0/3577507148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ec15eb50 con 0x7f03ec06e430 2024-04-25T10:07:29.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.147+0000 7f03f1800640 1 -- 172.21.15.119:0/3577507148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ec15f110 con 0x7f03ec06e430 2024-04-25T10:07:29.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.148+0000 7f03f1800640 1 -- 172.21.15.119:0/3577507148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ec0749b0 con 0x7f03ec06e430 2024-04-25T10:07:29.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.149+0000 7f03e8c00640 1 -- 172.21.15.119:0/3577507148 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03dc012b90 con 0x7f03ec06e430 2024-04-25T10:07:29.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.150+0000 7f03e8c00640 1 --2- 172.21.15.119:0/3577507148 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f03c806b4d0 0x7f03c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.150+0000 7f03e8c00640 1 -- 172.21.15.119:0/3577507148 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03dc08a6b0 con 0x7f03ec06e430 2024-04-25T10:07:29.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.150+0000 7f03e3400640 1 --2- 172.21.15.119:0/3577507148 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f03c806b4d0 0x7f03c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.150+0000 7f03e3400640 1 --2- 172.21.15.119:0/3577507148 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f03c806b4d0 0x7f03c806d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03ec143990 tx=0x7f03d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.154+0000 7f03e8c00640 1 -- 172.21.15.119:0/3577507148 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03dc059490 con 0x7f03ec06e430 2024-04-25T10:07:29.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.389+0000 7f03f1800640 1 -- 172.21.15.119:0/3577507148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f03ec0b2090 con 0x7f03ec06e430 2024-04-25T10:07:29.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.390+0000 7f03e8c00640 1 -- 172.21.15.119:0/3577507148 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f03dc058d90 con 0x7f03ec06e430 2024-04-25T10:07:29.396 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:07:29.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.393+0000 7f03f1800640 1 -- 172.21.15.119:0/3577507148 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f03c806b4d0 msgr2=0x7f03c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.393+0000 7f03f1800640 1 --2- 172.21.15.119:0/3577507148 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f03c806b4d0 0x7f03c806d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03ec143990 tx=0x7f03d4009040 comp rx=0 tx=0).stop 2024-04-25T10:07:29.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.393+0000 7f03f1800640 1 -- 172.21.15.119:0/3577507148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ec06e430 msgr2=0x7f03ec1f6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.393+0000 7f03f1800640 1 --2- 172.21.15.119:0/3577507148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ec06e430 0x7f03ec1f6590 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00d5d0 tx=0x7f03dc00da90 comp rx=0 tx=0).stop 2024-04-25T10:07:29.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.393+0000 7f03f1800640 1 -- 172.21.15.119:0/3577507148 shutdown_connections 2024-04-25T10:07:29.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.393+0000 7f03f1800640 1 --2- 172.21.15.119:0/3577507148 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f03c806b4d0 0x7f03c806d990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.393+0000 7f03f1800640 1 --2- 172.21.15.119:0/3577507148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03ec144470 0x7f03ec142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.393+0000 7f03f1800640 1 --2- 172.21.15.119:0/3577507148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03ec0b5050 0x7f03ec0b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.393+0000 7f03f1800640 1 --2- 172.21.15.119:0/3577507148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03ec06e430 0x7f03ec1f6590 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.393+0000 7f03f1800640 1 -- 172.21.15.119:0/3577507148 >> 172.21.15.119:0/3577507148 conn(0x7f03ec06b1d0 msgr2=0x7f03ec0b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:29.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.393+0000 7f03f1800640 1 -- 172.21.15.119:0/3577507148 shutdown_connections 2024-04-25T10:07:29.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:29.393+0000 7f03f1800640 1 -- 172.21.15.119:0/3577507148 wait complete. 2024-04-25T10:07:29.418 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:07:29.418 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:07:29.418 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:07:29.418 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T10:07:29.418 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:07:29.418 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:07:29.418 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:07:29.418 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:07:29.418 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:07:29.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.419 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:07:29.467 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:29.467 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:29.468 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:07:29.468 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:07:29.468 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:29.468 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:07:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:07:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42640sec preferred_lft 42640sec 2024-04-25T10:07:29.468 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:07:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:29.469 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:07:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:07:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:29.469 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:07:29.469 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:29.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.470 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:07:29.545 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:07:29.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.549 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:07:29.602 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:07:29.602 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:07:29.602 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:07:29.602 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:07:29.602 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:07:29.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.603 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:07:29.603 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:07:29.603 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:07:29.603 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:07:29.603 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:07:29.603 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:07:29.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.751 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:07:29.797 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:07:29.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.798 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2024-04-25T10:07:29.894 INFO:teuthology.orchestra.run.smithi119.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T10:07:29.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.896 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:07:29.896 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:07:29.896 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:29.896 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T10:07:29.896 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:07:30.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:30.019 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:07:30.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:30.071 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:30.093 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:30.093 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:30.093 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:30.093 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:30.093 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:30.093 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:30.093 INFO:teuthology.orchestra.run.smithi119.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:30.093 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 7a:f9:03:9d:f0:2e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:07:30.094 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:07:30.094 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:30.094 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::78f9:3ff:fe9d:f02e/64 scope link 2024-04-25T10:07:30.094 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:30.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:30.096 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:07:30.209 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:30.209 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:30.209 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:30.209 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:30.209 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:30.209 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:30.209 INFO:teuthology.orchestra.run.smithi119.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:30.209 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 7a:f9:03:9d:f0:2e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:07:30.209 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:07:30.209 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:30.209 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::78f9:3ff:fe9d:f02e/64 scope link 2024-04-25T10:07:30.210 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:30.212 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T10:07:30.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:30.212 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:07:30.212 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:07:30.213 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T10:07:30.213 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:07:30.213 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T10:07:30.213 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T10:07:30.213 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T10:07:30.213 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:07:30.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:30.471 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:07:30.471 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:07:30.472 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.1 up 2024-04-25T10:07:30.472 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:07:30.472 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:07:30.616 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:30.616 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T10:07:30.665 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:07:30.666 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:30.666 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:07:30.713 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:07:30.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:30.713 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:07:30.838 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:07:30.838 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:07:30.838 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:07:30.859 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:07:30.859 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:07:30.859 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:07:30.860 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:07:30.860 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:30.886 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:07:30.886 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:07:30.886 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:07:30.886 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:07:30.886 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:30.886 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:07:30.886 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:07:30.886 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:07:30.887 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-25T10:07:30.888 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:07:30.888 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:07:30.888 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:07:30.888 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:07:30.891 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:07:30.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:30.891 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:07:30.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:30.966 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:31.024 INFO:teuthology.orchestra.run.smithi119.stdout:client4626 2024-04-25T10:07:31.024 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:07:31.024 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:31.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:31.076 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:31.127 INFO:teuthology.orchestra.run.smithi119.stdout:client4626 2024-04-25T10:07:31.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:31.127 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:31.180 INFO:teuthology.orchestra.run.smithi119.stdout:client4626 2024-04-25T10:07:31.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:31.180 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e0b0f428-b441-4237-a4d1-ed53321f87a5.client4626/status') 2024-04-25T10:07:31.257 INFO:teuthology.orchestra.run.smithi119.stdout:instance: client.4626 (3)192.168.0.2:0/4240800352 2024-04-25T10:07:31.257 INFO:teuthology.orchestra.run.smithi119.stdout:blocklisted: false 2024-04-25T10:07:31.257 INFO:teuthology.orchestra.run.smithi119.stderr:0+1 records in 2024-04-25T10:07:31.257 INFO:teuthology.orchestra.run.smithi119.stderr:0+1 records out 2024-04-25T10:07:31.257 INFO:teuthology.orchestra.run.smithi119.stderr:69 bytes copied, 7.9904e-05 s, 864 kB/s 2024-04-25T10:07:31.261 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-25T10:07:31.261 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:07:31.261 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:07:31.261 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:07:31.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.459+0000 7f70e0c00640 1 Processor -- start 2024-04-25T10:07:31.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.459+0000 7f70e0c00640 1 -- start start 2024-04-25T10:07:31.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.461+0000 7f70e0c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f70dc15a870 0x7f70dc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.461+0000 7f70e0c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0cfc40 0x7f70dc15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.461+0000 7f70e0c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70dc0cdb10 con 0x7f70dc06e2e0 2024-04-25T10:07:31.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.461+0000 7f70e0c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70dc0ce720 con 0x7f70dc15a870 2024-04-25T10:07:31.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.461+0000 7f70e0c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0cddf0 con 0x7f70dc0cfc40 2024-04-25T10:07:31.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.461+0000 7f70da800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0cfc40 0x7f70dc15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.461+0000 7f70d9400640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f70dc06e2e0 0x7f70dc0cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:53690/0 (socket says 172.21.15.119:53690) 2024-04-25T10:07:31.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.462+0000 7f70d9400640 1 -- 172.21.15.119:0/563427897 learned_addr learned my addr 172.21.15.119:0/563427897 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:31.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.462+0000 7f70d9e00640 1 --2- 172.21.15.119:0/563427897 >> v2:172.21.15.132:3300/0 conn(0x7f70dc15a870 0x7f70dc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.462+0000 7f70da800640 1 -- 172.21.15.119:0/563427897 >> v2:172.21.15.132:3300/0 conn(0x7f70dc15a870 msgr2=0x7f70dc0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:31.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.462+0000 7f70da800640 1 --2- 172.21.15.119:0/563427897 >> v2:172.21.15.132:3300/0 conn(0x7f70dc15a870 0x7f70dc0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.462+0000 7f70da800640 1 -- 172.21.15.119:0/563427897 >> v1:172.21.15.132:6789/0 conn(0x7f70dc06e2e0 legacy=0x7f70dc0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.462+0000 7f70da800640 1 -- 172.21.15.119:0/563427897 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0ce0d0 con 0x7f70dc0cfc40 2024-04-25T10:07:31.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.463+0000 7f70da800640 1 --2- 172.21.15.119:0/563427897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0cfc40 0x7f70dc15ac50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f70d400bfa0 tx=0x7f70d402c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=347a100ea180c8d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.464+0000 7f70d8a00640 1 -- 172.21.15.119:0/563427897 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4039070 con 0x7f70dc0cfc40 2024-04-25T10:07:31.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.464+0000 7f70d8a00640 1 -- 172.21.15.119:0/563427897 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d4009540 con 0x7f70dc0cfc40 2024-04-25T10:07:31.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.464+0000 7f70d8a00640 1 -- 172.21.15.119:0/563427897 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4009910 con 0x7f70dc0cfc40 2024-04-25T10:07:31.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.465+0000 7f70e0c00640 1 -- 172.21.15.119:0/563427897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0cfc40 msgr2=0x7f70dc15ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.465+0000 7f70e0c00640 1 --2- 172.21.15.119:0/563427897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0cfc40 0x7f70dc15ac50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f70d400bfa0 tx=0x7f70d402c870 comp rx=0 tx=0).stop 2024-04-25T10:07:31.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.465+0000 7f70e0c00640 1 -- 172.21.15.119:0/563427897 shutdown_connections 2024-04-25T10:07:31.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.465+0000 7f70e0c00640 1 --2- 172.21.15.119:0/563427897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0cfc40 0x7f70dc15ac50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.465+0000 7f70e0c00640 1 --2- 172.21.15.119:0/563427897 >> v2:172.21.15.132:3300/0 conn(0x7f70dc15a870 0x7f70dc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.465+0000 7f70e0c00640 1 -- 172.21.15.119:0/563427897 >> 172.21.15.119:0/563427897 conn(0x7f70dc06aba0 msgr2=0x7f70dc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:31.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.465+0000 7f70e0c00640 1 -- 172.21.15.119:0/563427897 shutdown_connections 2024-04-25T10:07:31.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.465+0000 7f70e0c00640 1 -- 172.21.15.119:0/563427897 wait complete. 2024-04-25T10:07:31.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.467+0000 7f70e0c00640 1 Processor -- start 2024-04-25T10:07:31.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.467+0000 7f70e0c00640 1 -- start start 2024-04-25T10:07:31.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.468+0000 7f70e0c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70dc06e2e0 0x7f70dc147140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.468+0000 7f70e0c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0cfc40 0x7f70dc147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.468+0000 7f70e0c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15a870 0x7f70dc06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.468+0000 7f70e0c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0cd700 con 0x7f70dc15a870 2024-04-25T10:07:31.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.468+0000 7f70e0c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0cd580 con 0x7f70dc06e2e0 2024-04-25T10:07:31.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.468+0000 7f70e0c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0cd880 con 0x7f70dc0cfc40 2024-04-25T10:07:31.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.469+0000 7f70da800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15a870 0x7f70dc06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.469+0000 7f70da800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15a870 0x7f70dc06d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55190/0 (socket says 172.21.15.119:55190) 2024-04-25T10:07:31.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.469+0000 7f70da800640 1 -- 172.21.15.119:0/2341405235 learned_addr learned my addr 172.21.15.119:0/2341405235 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:31.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.469+0000 7f70d9e00640 1 --2- 172.21.15.119:0/2341405235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70dc06e2e0 0x7f70dc147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.469+0000 7f70d9400640 1 --2- 172.21.15.119:0/2341405235 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0cfc40 0x7f70dc147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.469+0000 7f70da800640 1 -- 172.21.15.119:0/2341405235 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0cfc40 msgr2=0x7f70dc147ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.469+0000 7f70da800640 1 --2- 172.21.15.119:0/2341405235 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0cfc40 0x7f70dc147ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.469+0000 7f70da800640 1 -- 172.21.15.119:0/2341405235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70dc06e2e0 msgr2=0x7f70dc147140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.469+0000 7f70da800640 1 --2- 172.21.15.119:0/2341405235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70dc06e2e0 0x7f70dc147140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.469+0000 7f70da800640 1 -- 172.21.15.119:0/2341405235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0ac9b0 con 0x7f70dc15a870 2024-04-25T10:07:31.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.470+0000 7f70da800640 1 --2- 172.21.15.119:0/2341405235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15a870 0x7f70dc06d1b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f70d4037040 tx=0x7f70d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.470+0000 7f70d2000640 1 -- 172.21.15.119:0/2341405235 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4039040 con 0x7f70dc15a870 2024-04-25T10:07:31.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.470+0000 7f70d2000640 1 -- 172.21.15.119:0/2341405235 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d40092b0 con 0x7f70dc15a870 2024-04-25T10:07:31.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.470+0000 7f70d2000640 1 -- 172.21.15.119:0/2341405235 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d40348a0 con 0x7f70dc15a870 2024-04-25T10:07:31.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.470+0000 7f70e0c00640 1 -- 172.21.15.119:0/2341405235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70dc0acc90 con 0x7f70dc15a870 2024-04-25T10:07:31.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.470+0000 7f70e0c00640 1 -- 172.21.15.119:0/2341405235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70dc0ad250 con 0x7f70dc15a870 2024-04-25T10:07:31.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.472+0000 7f70e0c00640 1 -- 172.21.15.119:0/2341405235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70dc0749b0 con 0x7f70dc15a870 2024-04-25T10:07:31.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.472+0000 7f70d2000640 1 -- 172.21.15.119:0/2341405235 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70d4032070 con 0x7f70dc15a870 2024-04-25T10:07:31.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.474+0000 7f70d2000640 1 --2- 172.21.15.119:0/2341405235 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70bc06b500 0x7f70bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.474+0000 7f70d2000640 1 -- 172.21.15.119:0/2341405235 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70d40aac60 con 0x7f70dc15a870 2024-04-25T10:07:31.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.474+0000 7f70d9e00640 1 --2- 172.21.15.119:0/2341405235 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70bc06b500 0x7f70bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.475+0000 7f70d9e00640 1 --2- 172.21.15.119:0/2341405235 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70bc06b500 0x7f70bc06d9c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009b40 tx=0x7f70cc009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.478+0000 7f70d2000640 1 -- 172.21.15.119:0/2341405235 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70d40793f0 con 0x7f70dc15a870 2024-04-25T10:07:31.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.712+0000 7f70e0c00640 1 -- 172.21.15.119:0/2341405235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f70dc0a9760 con 0x7f70dc15a870 2024-04-25T10:07:31.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.713+0000 7f70d2000640 1 -- 172.21.15.119:0/2341405235 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f70d407d490 con 0x7f70dc15a870 2024-04-25T10:07:31.718 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:07:31.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.716+0000 7f70e0c00640 1 -- 172.21.15.119:0/2341405235 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70bc06b500 msgr2=0x7f70bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.716+0000 7f70e0c00640 1 --2- 172.21.15.119:0/2341405235 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70bc06b500 0x7f70bc06d9c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009b40 tx=0x7f70cc009190 comp rx=0 tx=0).stop 2024-04-25T10:07:31.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.716+0000 7f70e0c00640 1 -- 172.21.15.119:0/2341405235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15a870 msgr2=0x7f70dc06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.716+0000 7f70e0c00640 1 --2- 172.21.15.119:0/2341405235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15a870 0x7f70dc06d1b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f70d4037040 tx=0x7f70d4004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:31.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.716+0000 7f70e0c00640 1 -- 172.21.15.119:0/2341405235 shutdown_connections 2024-04-25T10:07:31.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.716+0000 7f70e0c00640 1 --2- 172.21.15.119:0/2341405235 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70bc06b500 0x7f70bc06d9c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.716+0000 7f70e0c00640 1 --2- 172.21.15.119:0/2341405235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15a870 0x7f70dc06d1b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.716+0000 7f70e0c00640 1 --2- 172.21.15.119:0/2341405235 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0cfc40 0x7f70dc147ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.716+0000 7f70e0c00640 1 --2- 172.21.15.119:0/2341405235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70dc06e2e0 0x7f70dc147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.716+0000 7f70e0c00640 1 -- 172.21.15.119:0/2341405235 >> 172.21.15.119:0/2341405235 conn(0x7f70dc06aba0 msgr2=0x7f70dc159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:31.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.716+0000 7f70e0c00640 1 -- 172.21.15.119:0/2341405235 shutdown_connections 2024-04-25T10:07:31.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:31.716+0000 7f70e0c00640 1 -- 172.21.15.119:0/2341405235 wait complete. 2024-04-25T10:07:31.740 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:07:31.740 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:07:31.740 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2024-04-25T10:07:31.740 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T10:07:31.740 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:07:31.740 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:07:31.740 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:07:31.740 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:07:31.741 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:07:31.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:31.741 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2024-04-25T10:07:31.797 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:31.797 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:31.797 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:31.797 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:31.797 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:31.797 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:31.797 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:31.797 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:31.797 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f0 2024-04-25T10:07:31.797 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:31.797 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:31.798 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f1 2024-04-25T10:07:31.798 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:31.798 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:31.798 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:07:31.798 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:31.798 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:07:31.798 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42646sec preferred_lft 42646sec 2024-04-25T10:07:31.798 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ec4:7aff:fe88:773f/64 scope link 2024-04-25T10:07:31.798 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:31.799 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:07:31.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:31.799 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-04-25T10:07:31.799 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-25T10:07:31.799 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:07:31.799 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr flush dev ceph-brx 2024-04-25T10:07:31.800 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set ceph-brx up 2024-04-25T10:07:31.800 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:07:31.800 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-25T10:07:32.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:32.210 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:07:32.274 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-04-25T10:07:32.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:32.278 DEBUG:teuthology.orchestra.run.smithi132:> route 2024-04-25T10:07:32.340 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2024-04-25T10:07:32.341 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:07:32.341 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:07:32.341 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:07:32.341 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:07:32.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:32.341 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-04-25T10:07:32.341 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-25T10:07:32.341 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:07:32.341 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:07:32.341 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:07:32.342 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-25T10:07:32.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:32.957 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-25T10:07:33.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:33.012 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list-id 2024-04-25T10:07:33.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:33.064 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-04-25T10:07:33.064 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-25T10:07:33.064 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:07:33.064 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T10:07:33.064 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-25T10:07:33.185 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T10:07:33.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:33.185 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-04-25T10:07:33.185 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-25T10:07:33.186 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T10:07:33.186 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:07:33.186 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T10:07:33.186 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T10:07:33.186 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T10:07:33.186 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-25T10:07:33.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:33.416 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-04-25T10:07:33.416 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-25T10:07:33.416 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.0 up 2024-04-25T10:07:33.416 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:07:33.416 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-25T10:07:33.558 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:33.559 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T10:07:33.603 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:07:33.603 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:33.604 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:07:33.696 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:07:33.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:33.697 DEBUG:teuthology.orchestra.run.smithi132:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:07:33.836 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:07:33.836 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:07:33.836 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2024-04-25T10:07:34.576 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:07:34.576 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:07:34.576 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2024-04-25T10:07:34.576 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:07:34.576 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:34.615 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:07:34.615 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:07:34.615 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2024-04-25T10:07:34.615 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:07:34.615 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:34.615 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:07:34.615 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:07:34.615 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:07:34.620 INFO:teuthology.orchestra.run.smithi132.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T10:07:34.620 INFO:teuthology.orchestra.run.smithi132.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:07:34.620 INFO:teuthology.orchestra.run.smithi132.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:07:34.620 INFO:teuthology.orchestra.run.smithi132.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:07:34.620 INFO:teuthology.orchestra.run.smithi132.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:07:34.625 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:07:34.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:34.626 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:07:34.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:34.704 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:34.761 INFO:teuthology.orchestra.run.smithi132.stdout:client4439 2024-04-25T10:07:34.762 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:07:34.762 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:34.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:34.814 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:34.869 INFO:teuthology.orchestra.run.smithi132.stdout:client4439 2024-04-25T10:07:34.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:34.870 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:34.925 INFO:teuthology.orchestra.run.smithi132.stdout:client4439 2024-04-25T10:07:34.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:34.926 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e0b0f428-b441-4237-a4d1-ed53321f87a5.client4439/status') 2024-04-25T10:07:35.006 INFO:teuthology.orchestra.run.smithi132.stdout:instance: client.4439 (3)192.168.0.1:0/747579787 2024-04-25T10:07:35.006 INFO:teuthology.orchestra.run.smithi132.stdout:blocklisted: false 2024-04-25T10:07:35.006 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records in 2024-04-25T10:07:35.006 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records out 2024-04-25T10:07:35.007 INFO:teuthology.orchestra.run.smithi132.stderr:68 bytes copied, 8.1267e-05 s, 837 kB/s 2024-04-25T10:07:35.009 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-25T10:07:35.010 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:07:35.010 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:07:35.010 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:07:35.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.215+0000 7faa9f200640 1 Processor -- start 2024-04-25T10:07:35.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.215+0000 7faa9f200640 1 -- start start 2024-04-25T10:07:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.217+0000 7faa9f200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faaa00b9270 0x7faaa00d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.217+0000 7faa9f200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa015ef10 con 0x7faaa00d3f80 2024-04-25T10:07:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.217+0000 7faa9f200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa015e390 con 0x7faaa015db60 2024-04-25T10:07:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.217+0000 7faa9f200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa015e670 con 0x7faaa00b9270 2024-04-25T10:07:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.217+0000 7faa95400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7faaa00d3f80 0x7faaa00d1f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51542/0 (socket says 172.21.15.119:51542) 2024-04-25T10:07:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.217+0000 7faa95400640 1 -- 172.21.15.119:0/622319 learned_addr learned my addr 172.21.15.119:0/622319 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.217+0000 7faa9e800640 1 --2- 172.21.15.119:0/622319 >> v2:172.21.15.119:3300/0 conn(0x7faaa00b9270 0x7faaa00d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.218+0000 7faa9e800640 1 -- 172.21.15.119:0/622319 >> v1:172.21.15.132:6789/0 conn(0x7faaa015db60 legacy=0x7faaa00d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.218+0000 7faa9d400640 1 -- 172.21.15.119:0/622319 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984722250 0 0) 0x7faaa015ef10 con 0x7faaa00d3f80 2024-04-25T10:07:35.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.218+0000 7faa9e800640 1 -- 172.21.15.119:0/622319 >> v1:172.21.15.119:6789/0 conn(0x7faaa00d3f80 legacy=0x7faaa00d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.218+0000 7faa9e800640 1 -- 172.21.15.119:0/622319 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa015e950 con 0x7faaa00b9270 2024-04-25T10:07:35.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.219+0000 7faa9e800640 1 --2- 172.21.15.119:0/622319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaa00b9270 0x7faaa00d26a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa98009d90 tx=0x7faa9802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738bc4236bc69b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.219+0000 7faa9d400640 1 -- 172.21.15.119:0/622319 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98007740 con 0x7faaa00b9270 2024-04-25T10:07:35.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.219+0000 7faa9d400640 1 -- 172.21.15.119:0/622319 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa980078e0 con 0x7faaa00b9270 2024-04-25T10:07:35.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.219+0000 7faa9d400640 1 -- 172.21.15.119:0/622319 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98007cb0 con 0x7faaa00b9270 2024-04-25T10:07:35.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.220+0000 7faa9f200640 1 -- 172.21.15.119:0/622319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaa00b9270 msgr2=0x7faaa00d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.220+0000 7faa9f200640 1 --2- 172.21.15.119:0/622319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaa00b9270 0x7faaa00d26a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa98009d90 tx=0x7faa9802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.220+0000 7faa9f200640 1 -- 172.21.15.119:0/622319 shutdown_connections 2024-04-25T10:07:35.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.220+0000 7faa9f200640 1 --2- 172.21.15.119:0/622319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaa00b9270 0x7faaa00d26a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.220+0000 7faa9f200640 1 -- 172.21.15.119:0/622319 >> 172.21.15.119:0/622319 conn(0x7faaa00726c0 msgr2=0x7faaa0072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.220+0000 7faa9f200640 1 -- 172.21.15.119:0/622319 shutdown_connections 2024-04-25T10:07:35.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.220+0000 7faa9f200640 1 -- 172.21.15.119:0/622319 wait complete. 2024-04-25T10:07:35.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.222+0000 7faa9f200640 1 Processor -- start 2024-04-25T10:07:35.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.222+0000 7faa9f200640 1 -- start start 2024-04-25T10:07:35.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.223+0000 7faa9f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaa00b9270 0x7faaa0158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.223+0000 7faa9f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaa00d3f80 0x7faaa00b8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.223+0000 7faa9f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faaa015db60 0x7faaa00b88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.223+0000 7faa9f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa00b3120 con 0x7faaa00d3f80 2024-04-25T10:07:35.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.223+0000 7faa9f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa00b2fa0 con 0x7faaa015db60 2024-04-25T10:07:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.223+0000 7faa9f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa015df40 con 0x7faaa00b9270 2024-04-25T10:07:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.223+0000 7faa95400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaa00d3f80 0x7faaa00b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.223+0000 7faa95400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaa00d3f80 0x7faaa00b8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55208/0 (socket says 172.21.15.119:55208) 2024-04-25T10:07:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.223+0000 7faa95400640 1 -- 172.21.15.119:0/4230604176 learned_addr learned my addr 172.21.15.119:0/4230604176 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.223+0000 7faa95400640 1 -- 172.21.15.119:0/4230604176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaa00b9270 msgr2=0x7faaa0158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.223+0000 7faa95400640 1 --2- 172.21.15.119:0/4230604176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaa00b9270 0x7faaa0158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.223+0000 7faa95400640 1 -- 172.21.15.119:0/4230604176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faaa015db60 msgr2=0x7faaa00b88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.223+0000 7faa95400640 1 --2- 172.21.15.119:0/4230604176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faaa015db60 0x7faaa00b88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.223+0000 7faa95400640 1 -- 172.21.15.119:0/4230604176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa00cb6e0 con 0x7faaa00d3f80 2024-04-25T10:07:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.224+0000 7faa95400640 1 --2- 172.21.15.119:0/4230604176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaa00d3f80 0x7faaa00b8380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faa8800c400 tx=0x7faa8800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.224+0000 7faa96a00640 1 -- 172.21.15.119:0/4230604176 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88017070 con 0x7faaa00d3f80 2024-04-25T10:07:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.224+0000 7faa96a00640 1 -- 172.21.15.119:0/4230604176 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa880074e0 con 0x7faaa00d3f80 2024-04-25T10:07:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.224+0000 7faa9f200640 1 -- 172.21.15.119:0/4230604176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa00cb9c0 con 0x7faaa00d3f80 2024-04-25T10:07:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.224+0000 7faa96a00640 1 -- 172.21.15.119:0/4230604176 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa880059e0 con 0x7faaa00d3f80 2024-04-25T10:07:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.224+0000 7faa9f200640 1 -- 172.21.15.119:0/4230604176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa00cbf80 con 0x7faaa00d3f80 2024-04-25T10:07:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.225+0000 7faa9f200640 1 -- 172.21.15.119:0/4230604176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa00749b0 con 0x7faaa00d3f80 2024-04-25T10:07:35.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.226+0000 7faa96a00640 1 -- 172.21.15.119:0/4230604176 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa8801a070 con 0x7faaa00d3f80 2024-04-25T10:07:35.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.227+0000 7faa96a00640 1 --2- 172.21.15.119:0/4230604176 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa7c06b4d0 0x7faa7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.227+0000 7faa96a00640 1 -- 172.21.15.119:0/4230604176 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa88089610 con 0x7faaa00d3f80 2024-04-25T10:07:35.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.227+0000 7faa9de00640 1 --2- 172.21.15.119:0/4230604176 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa7c06b4d0 0x7faa7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.228+0000 7faa9de00640 1 --2- 172.21.15.119:0/4230604176 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa7c06b4d0 0x7faa7c06d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa90003cc0 tx=0x7faa900079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.232+0000 7faa96a00640 1 -- 172.21.15.119:0/4230604176 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa88058540 con 0x7faaa00d3f80 2024-04-25T10:07:35.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.465+0000 7faa9f200640 1 -- 172.21.15.119:0/4230604176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faaa00d1a60 con 0x7faaa00d3f80 2024-04-25T10:07:35.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.466+0000 7faa96a00640 1 -- 172.21.15.119:0/4230604176 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7faa88057e40 con 0x7faaa00d3f80 2024-04-25T10:07:35.472 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:07:35.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.469+0000 7faa9f200640 1 -- 172.21.15.119:0/4230604176 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa7c06b4d0 msgr2=0x7faa7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.469+0000 7faa9f200640 1 --2- 172.21.15.119:0/4230604176 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa7c06b4d0 0x7faa7c06d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa90003cc0 tx=0x7faa900079b0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.469+0000 7faa9f200640 1 -- 172.21.15.119:0/4230604176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaa00d3f80 msgr2=0x7faaa00b8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.469+0000 7faa9f200640 1 --2- 172.21.15.119:0/4230604176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaa00d3f80 0x7faaa00b8380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faa8800c400 tx=0x7faa8800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.469+0000 7faa9f200640 1 -- 172.21.15.119:0/4230604176 shutdown_connections 2024-04-25T10:07:35.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.469+0000 7faa9f200640 1 --2- 172.21.15.119:0/4230604176 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa7c06b4d0 0x7faa7c06d990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.469+0000 7faa9f200640 1 --2- 172.21.15.119:0/4230604176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faaa015db60 0x7faaa00b88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.469+0000 7faa9f200640 1 --2- 172.21.15.119:0/4230604176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaa00d3f80 0x7faaa00b8380 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.469+0000 7faa9f200640 1 --2- 172.21.15.119:0/4230604176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faaa00b9270 0x7faaa0158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.469+0000 7faa9f200640 1 -- 172.21.15.119:0/4230604176 >> 172.21.15.119:0/4230604176 conn(0x7faaa00726c0 msgr2=0x7faaa00b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.470+0000 7faa9f200640 1 -- 172.21.15.119:0/4230604176 shutdown_connections 2024-04-25T10:07:35.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:35.470+0000 7faa9f200640 1 -- 172.21.15.119:0/4230604176 wait complete. 2024-04-25T10:07:35.494 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:07:35.494 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:07:35.494 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2024-04-25T10:07:35.495 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T10:07:35.495 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:07:35.495 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:07:35.495 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:07:35.495 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:07:35.495 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:07:35.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:35.495 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2024-04-25T10:07:35.553 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:35.553 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:35.554 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:35.554 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:35.554 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:35.554 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:35.554 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:35.554 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:35.554 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f0 2024-04-25T10:07:35.554 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:35.555 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:35.555 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f1 2024-04-25T10:07:35.555 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:35.555 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:35.555 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:07:35.555 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:35.555 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:07:35.555 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42642sec preferred_lft 42642sec 2024-04-25T10:07:35.555 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ec4:7aff:fe88:773f/64 scope link 2024-04-25T10:07:35.556 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:35.556 INFO:teuthology.orchestra.run.smithi132.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:35.556 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 56:ac:03:23:4e:f2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:35.556 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:07:35.556 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:35.556 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::643d:d4ff:fe56:4d63/64 scope link 2024-04-25T10:07:35.556 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:35.556 INFO:teuthology.orchestra.run.smithi132.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:35.556 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 56:ac:03:23:4e:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:07:35.556 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::54ac:3ff:fe23:4ef2/64 scope link 2024-04-25T10:07:35.556 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:35.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:35.557 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:07:35.631 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-04-25T10:07:35.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:35.635 DEBUG:teuthology.orchestra.run.smithi132:> route 2024-04-25T10:07:35.684 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2024-04-25T10:07:35.684 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:07:35.684 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:07:35.685 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:07:35.685 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:07:35.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:35.685 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-04-25T10:07:35.685 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-25T10:07:35.685 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:07:35.685 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:07:35.685 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:07:35.685 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-25T10:07:35.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:35.831 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-25T10:07:35.881 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:07:35.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:35.882 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list-id 2024-04-25T10:07:35.934 INFO:teuthology.orchestra.run.smithi132.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T10:07:35.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:35.935 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-04-25T10:07:35.935 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-25T10:07:35.935 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:07:35.935 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T10:07:35.935 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-25T10:07:36.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.056 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:07:36.133 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:36.133 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:36.133 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:36.133 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:36.133 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:36.133 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:36.133 INFO:teuthology.orchestra.run.smithi132.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:36.133 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether ce:a6:a1:de:0a:2c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:07:36.133 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:07:36.133 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:36.133 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::cca6:a1ff:fede:a2c/64 scope link 2024-04-25T10:07:36.134 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:36.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.137 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:07:36.218 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:36.218 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:36.218 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:36.218 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:36.218 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:36.218 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:36.218 INFO:teuthology.orchestra.run.smithi132.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:36.218 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether ce:a6:a1:de:0a:2c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:07:36.218 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:07:36.218 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:36.219 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::cca6:a1ff:fede:a2c/64 scope link 2024-04-25T10:07:36.219 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:36.222 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T10:07:36.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.222 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-04-25T10:07:36.222 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-25T10:07:36.222 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T10:07:36.222 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:07:36.222 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T10:07:36.222 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T10:07:36.222 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T10:07:36.223 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-25T10:07:36.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.434 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-04-25T10:07:36.435 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-25T10:07:36.435 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.1 up 2024-04-25T10:07:36.435 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:07:36.435 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-25T10:07:36.579 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:36.579 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T10:07:36.622 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:07:36.623 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:36.623 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:07:36.714 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:07:36.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.714 DEBUG:teuthology.orchestra.run.smithi132:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:07:36.838 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:07:36.839 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:07:36.839 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2024-04-25T10:07:36.859 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:07:36.859 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:07:36.859 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2024-04-25T10:07:36.859 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:07:36.859 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:36.889 INFO:teuthology.orchestra.run.smithi132.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:07:36.889 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:07:36.889 INFO:teuthology.orchestra.run.smithi132.stdout:invalid new device string format 2024-04-25T10:07:36.889 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:07:36.889 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:36.889 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:07:36.889 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:07:36.889 INFO:teuthology.orchestra.run.smithi132.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:07:36.892 INFO:teuthology.orchestra.run.smithi132.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-25T10:07:36.892 INFO:teuthology.orchestra.run.smithi132.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:07:36.892 INFO:teuthology.orchestra.run.smithi132.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:07:36.892 INFO:teuthology.orchestra.run.smithi132.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:07:36.892 INFO:teuthology.orchestra.run.smithi132.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:07:36.896 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:07:36.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.896 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:07:36.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.966 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:37.015 INFO:teuthology.orchestra.run.smithi132.stdout:client4641 2024-04-25T10:07:37.016 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:07:37.016 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:37.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:37.065 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:37.166 INFO:teuthology.orchestra.run.smithi132.stdout:client4641 2024-04-25T10:07:37.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:37.166 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:37.226 INFO:teuthology.orchestra.run.smithi132.stdout:client4641 2024-04-25T10:07:37.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:37.227 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/e0b0f428-b441-4237-a4d1-ed53321f87a5.client4641/status') 2024-04-25T10:07:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:instance: client.4641 (3)172.21.15.132:0/3183697209 2024-04-25T10:07:37.309 INFO:teuthology.orchestra.run.smithi132.stdout:blocklisted: false 2024-04-25T10:07:37.309 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records in 2024-04-25T10:07:37.310 INFO:teuthology.orchestra.run.smithi132.stderr:0+1 records out 2024-04-25T10:07:37.310 INFO:teuthology.orchestra.run.smithi132.stderr:71 bytes copied, 0.000111666 s, 636 kB/s 2024-04-25T10:07:37.313 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T10:07:37.331 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:37.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.537+0000 7f9130a00640 1 Processor -- start 2024-04-25T10:07:37.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.537+0000 7f9130a00640 1 -- start start 2024-04-25T10:07:37.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.539+0000 7f9130a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f912c06e8f0 0x7f912c15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.539+0000 7f9130a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912c0ce6c0 con 0x7f912c15c820 2024-04-25T10:07:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.539+0000 7f9130a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912c0cc4e0 con 0x7f912c15bf00 2024-04-25T10:07:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.539+0000 7f9130a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c1455e0 con 0x7f912c06e8f0 2024-04-25T10:07:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.539+0000 7f912a800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f912c15c820 0x7f912c1654b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51554/0 (socket says 172.21.15.119:51554) 2024-04-25T10:07:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.539+0000 7f912a800640 1 -- 172.21.15.119:0/3365891122 learned_addr learned my addr 172.21.15.119:0/3365891122 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.539+0000 7f9129400640 1 --2- 172.21.15.119:0/3365891122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f912c06e8f0 0x7f912c15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.539+0000 7f9128a00640 1 -- 172.21.15.119:0/3365891122 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202934045 0 0) 0x7f9124001490 con 0x7f912c15c820 2024-04-25T10:07:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.539+0000 7f9128a00640 1 -- 172.21.15.119:0/3365891122 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9110003520 con 0x7f912c15c820 2024-04-25T10:07:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.540+0000 7f9128a00640 1 -- 172.21.15.119:0/3365891122 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1792838077 0 0) 0x7f9110003520 con 0x7f912c15c820 2024-04-25T10:07:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.540+0000 7f9128a00640 1 -- 172.21.15.119:0/3365891122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f912c06e8f0 msgr2=0x7f912c15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.540+0000 7f9128a00640 1 --2- 172.21.15.119:0/3365891122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f912c06e8f0 0x7f912c15c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.540+0000 7f9128a00640 1 -- 172.21.15.119:0/3365891122 >> v1:172.21.15.132:6789/0 conn(0x7f912c15bf00 legacy=0x7f912c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.540+0000 7f9128a00640 1 -- 172.21.15.119:0/3365891122 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c0cc7b0 con 0x7f912c15c820 2024-04-25T10:07:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.540+0000 7f9128a00640 1 -- 172.21.15.119:0/3365891122 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f9124001bc0 con 0x7f912c15c820 2024-04-25T10:07:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.540+0000 7f9128a00640 1 -- 172.21.15.119:0/3365891122 >> v1:172.21.15.119:6789/0 conn(0x7f912c15c820 legacy=0x7f912c1654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.540+0000 7f9128a00640 1 --2- 172.21.15.119:0/3365891122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91100039e0 0x7f9110003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.540+0000 7f9128a00640 1 --2- 172.21.15.119:0/3365891122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f912c06e8f0 0x7f9110004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.540+0000 7f9128a00640 1 --2- 172.21.15.119:0/3365891122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9110004d40 0x7f91100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.540+0000 7f9128a00640 1 -- 172.21.15.119:0/3365891122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c1455e0 con 0x7f91100039e0 2024-04-25T10:07:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.540+0000 7f9128a00640 1 -- 172.21.15.119:0/3365891122 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911000ba70 con 0x7f9110004d40 2024-04-25T10:07:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.540+0000 7f9128a00640 1 -- 172.21.15.119:0/3365891122 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911000bc70 con 0x7f912c06e8f0 2024-04-25T10:07:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.540+0000 7f9129e00640 1 --2- 172.21.15.119:0/3365891122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91100039e0 0x7f9110003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.541+0000 7f9129400640 1 --2- 172.21.15.119:0/3365891122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f912c06e8f0 0x7f9110004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.541+0000 7f9129e00640 1 -- 172.21.15.119:0/3365891122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f912c06e8f0 msgr2=0x7f9110004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.541+0000 7f9129e00640 1 --2- 172.21.15.119:0/3365891122 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f912c06e8f0 0x7f9110004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.541+0000 7f9129e00640 1 -- 172.21.15.119:0/3365891122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9110004d40 msgr2=0x7f91100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.541+0000 7f9129e00640 1 --2- 172.21.15.119:0/3365891122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9110004d40 0x7f91100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.541+0000 7f9129e00640 1 -- 172.21.15.119:0/3365891122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9110003520 con 0x7f91100039e0 2024-04-25T10:07:37.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.543+0000 7f9129e00640 1 --2- 172.21.15.119:0/3365891122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91100039e0 0x7f9110003e00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f911c001720 tx=0x7f9110005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2564c385df2e7ebe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.543+0000 7f9128a00640 1 -- 172.21.15.119:0/3365891122 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c001370 con 0x7f91100039e0 2024-04-25T10:07:37.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.543+0000 7f9128a00640 1 -- 172.21.15.119:0/3365891122 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c0052d0 con 0x7f91100039e0 2024-04-25T10:07:37.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.543+0000 7f9128a00640 1 -- 172.21.15.119:0/3365891122 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c02abc0 con 0x7f91100039e0 2024-04-25T10:07:37.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.544+0000 7f9130a00640 1 -- 172.21.15.119:0/3365891122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91100039e0 msgr2=0x7f9110003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.544+0000 7f9130a00640 1 --2- 172.21.15.119:0/3365891122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91100039e0 0x7f9110003e00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f911c001720 tx=0x7f9110005830 comp rx=0 tx=0).stop 2024-04-25T10:07:37.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.544+0000 7f912a800640 1 -- 172.21.15.119:0/3365891122 reap_dead start 2024-04-25T10:07:37.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.544+0000 7f9130a00640 1 -- 172.21.15.119:0/3365891122 shutdown_connections 2024-04-25T10:07:37.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.544+0000 7f9130a00640 1 -- 172.21.15.119:0/3365891122 >> 172.21.15.119:0/3365891122 conn(0x7f912c06aba0 msgr2=0x7f912c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.544+0000 7f9130a00640 1 -- 172.21.15.119:0/3365891122 shutdown_connections 2024-04-25T10:07:37.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.544+0000 7f9130a00640 1 -- 172.21.15.119:0/3365891122 wait complete. 2024-04-25T10:07:37.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.546+0000 7f9130a00640 1 Processor -- start 2024-04-25T10:07:37.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.546+0000 7f9130a00640 1 -- start start 2024-04-25T10:07:37.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.547+0000 7f9130a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f912c1f70f0 0x7f912c1e4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.547+0000 7f9130a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f912c0b3080 0x7f912c137bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.547+0000 7f9130a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f912c1f74d0 0x7f912c138110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.547+0000 7f9130a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c0ccd70 con 0x7f912c0b3080 2024-04-25T10:07:37.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.547+0000 7f9130a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c06d450 con 0x7f912c1f70f0 2024-04-25T10:07:37.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.547+0000 7f9130a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c0cd050 con 0x7f912c1f74d0 2024-04-25T10:07:37.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.547+0000 7f9129400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f912c0b3080 0x7f912c137bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.547+0000 7f9129400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f912c0b3080 0x7f912c137bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55224/0 (socket says 172.21.15.119:55224) 2024-04-25T10:07:37.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.547+0000 7f9129400640 1 -- 172.21.15.119:0/908331125 learned_addr learned my addr 172.21.15.119:0/908331125 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:37.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.547+0000 7f9129400640 1 -- 172.21.15.119:0/908331125 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f912c1f74d0 msgr2=0x7f912c138110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.547+0000 7f9129400640 1 --2- 172.21.15.119:0/908331125 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f912c1f74d0 0x7f912c138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.547+0000 7f9129400640 1 -- 172.21.15.119:0/908331125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f912c1f70f0 msgr2=0x7f912c1e4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.547+0000 7f9129400640 1 --2- 172.21.15.119:0/908331125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f912c1f70f0 0x7f912c1e4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.547+0000 7f9129400640 1 -- 172.21.15.119:0/908331125 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c1f4a50 con 0x7f912c0b3080 2024-04-25T10:07:37.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.548+0000 7f9129400640 1 --2- 172.21.15.119:0/908331125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f912c0b3080 0x7f912c137bd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f91180115d0 tx=0x7f9118011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.548+0000 7f9122000640 1 -- 172.21.15.119:0/908331125 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118015870 con 0x7f912c0b3080 2024-04-25T10:07:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.548+0000 7f9122000640 1 -- 172.21.15.119:0/908331125 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9118015a10 con 0x7f912c0b3080 2024-04-25T10:07:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.548+0000 7f9122000640 1 -- 172.21.15.119:0/908331125 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118015e00 con 0x7f912c0b3080 2024-04-25T10:07:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.548+0000 7f9130a00640 1 -- 172.21.15.119:0/908331125 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912c0b35f0 con 0x7f912c0b3080 2024-04-25T10:07:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.548+0000 7f9130a00640 1 -- 172.21.15.119:0/908331125 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c0b3bb0 con 0x7f912c0b3080 2024-04-25T10:07:37.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.549+0000 7f9130a00640 1 -- 172.21.15.119:0/908331125 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912c0749b0 con 0x7f912c0b3080 2024-04-25T10:07:37.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.550+0000 7f9122000640 1 -- 172.21.15.119:0/908331125 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f911800f480 con 0x7f912c0b3080 2024-04-25T10:07:37.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.551+0000 7f9122000640 1 --2- 172.21.15.119:0/908331125 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f910c06b500 0x7f910c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.551+0000 7f9122000640 1 -- 172.21.15.119:0/908331125 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f911808aaa0 con 0x7f912c0b3080 2024-04-25T10:07:37.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.551+0000 7f9129e00640 1 --2- 172.21.15.119:0/908331125 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f910c06b500 0x7f910c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.552+0000 7f9129e00640 1 --2- 172.21.15.119:0/908331125 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f910c06b500 0x7f910c06d9c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f911c032000 tx=0x7f911c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.555+0000 7f9122000640 1 -- 172.21.15.119:0/908331125 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9118019030 con 0x7f912c0b3080 2024-04-25T10:07:37.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.754+0000 7f9130a00640 1 -- 172.21.15.119:0/908331125 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f912c1e46d0 con 0x7f912c0b3080 2024-04-25T10:07:37.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.757+0000 7f9122000640 1 -- 172.21.15.119:0/908331125 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f9118059880 con 0x7f912c0b3080 2024-04-25T10:07:37.764 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:37.764 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:08.319104+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:37.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.760+0000 7f9130a00640 1 -- 172.21.15.119:0/908331125 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f910c06b500 msgr2=0x7f910c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.760+0000 7f9130a00640 1 --2- 172.21.15.119:0/908331125 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f910c06b500 0x7f910c06d9c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f911c032000 tx=0x7f911c02c000 comp rx=0 tx=0).stop 2024-04-25T10:07:37.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.760+0000 7f9130a00640 1 -- 172.21.15.119:0/908331125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f912c0b3080 msgr2=0x7f912c137bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.760+0000 7f9130a00640 1 --2- 172.21.15.119:0/908331125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f912c0b3080 0x7f912c137bd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f91180115d0 tx=0x7f9118011a90 comp rx=0 tx=0).stop 2024-04-25T10:07:37.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.760+0000 7f9130a00640 1 -- 172.21.15.119:0/908331125 shutdown_connections 2024-04-25T10:07:37.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.760+0000 7f9130a00640 1 --2- 172.21.15.119:0/908331125 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f910c06b500 0x7f910c06d9c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.760+0000 7f9130a00640 1 --2- 172.21.15.119:0/908331125 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f912c1f74d0 0x7f912c138110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.760+0000 7f9130a00640 1 --2- 172.21.15.119:0/908331125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f912c0b3080 0x7f912c137bd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.760+0000 7f9130a00640 1 --2- 172.21.15.119:0/908331125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f912c1f70f0 0x7f912c1e4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.760+0000 7f9130a00640 1 -- 172.21.15.119:0/908331125 >> 172.21.15.119:0/908331125 conn(0x7f912c06aba0 msgr2=0x7f912c0ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.761+0000 7f9130a00640 1 -- 172.21.15.119:0/908331125 shutdown_connections 2024-04-25T10:07:37.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.761+0000 7f9130a00640 1 -- 172.21.15.119:0/908331125 wait complete. 2024-04-25T10:07:37.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:05.135832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:05.768083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:37.787 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:37.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.987+0000 7f440d800640 1 Processor -- start 2024-04-25T10:07:37.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.987+0000 7f440d800640 1 -- start start 2024-04-25T10:07:37.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.989+0000 7f440d800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44080ce9d0 0x7f44080cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.989+0000 7f440d800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440806dcb0 con 0x7f44080cc300 2024-04-25T10:07:37.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.989+0000 7f440d800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440806df90 con 0x7f4408144e70 2024-04-25T10:07:37.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.989+0000 7f440d800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440815bfe0 con 0x7f44080ce9d0 2024-04-25T10:07:37.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.989+0000 7f43ffe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44080ce9d0 0x7f44080cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.989+0000 7f43ffe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44080ce9d0 0x7f44080cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55230/0 (socket says 172.21.15.119:55230) 2024-04-25T10:07:37.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.989+0000 7f43ffe00640 1 -- 172.21.15.119:0/871616722 learned_addr learned my addr 172.21.15.119:0/871616722 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:37.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.989+0000 7f4407400640 1 --1- 172.21.15.119:0/871616722 >> v1:172.21.15.119:6789/0 conn(0x7f44080cc300 0x7f44080cc740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51560/0 (socket says 172.21.15.119:51560) 2024-04-25T10:07:37.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.990+0000 7f43ffe00640 1 -- 172.21.15.119:0/871616722 >> v1:172.21.15.132:6789/0 conn(0x7f4408144e70 legacy=0x7f440806d260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:07:37.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.990+0000 7f43ffe00640 1 -- 172.21.15.119:0/871616722 >> v1:172.21.15.119:6789/0 conn(0x7f44080cc300 legacy=0x7f44080cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.990+0000 7f43ffe00640 1 -- 172.21.15.119:0/871616722 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440806e270 con 0x7f44080ce9d0 2024-04-25T10:07:37.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.991+0000 7f43ffe00640 1 --2- 172.21.15.119:0/871616722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44080ce9d0 0x7f44080cee60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f440815c160 tx=0x7f43f002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94b411a92153302 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.991+0000 7f4406000640 1 -- 172.21.15.119:0/871616722 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0037070 con 0x7f44080ce9d0 2024-04-25T10:07:37.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.991+0000 7f4406000640 1 -- 172.21.15.119:0/871616722 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f0032440 con 0x7f44080ce9d0 2024-04-25T10:07:37.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.991+0000 7f4406000640 1 -- 172.21.15.119:0/871616722 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0032810 con 0x7f44080ce9d0 2024-04-25T10:07:37.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.992+0000 7f440d800640 1 -- 172.21.15.119:0/871616722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44080ce9d0 msgr2=0x7f44080cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.992+0000 7f440d800640 1 --2- 172.21.15.119:0/871616722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44080ce9d0 0x7f44080cee60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f440815c160 tx=0x7f43f002a210 comp rx=0 tx=0).stop 2024-04-25T10:07:37.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.992+0000 7f440d800640 1 -- 172.21.15.119:0/871616722 shutdown_connections 2024-04-25T10:07:37.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.992+0000 7f440d800640 1 --2- 172.21.15.119:0/871616722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44080ce9d0 0x7f44080cee60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.992+0000 7f440d800640 1 -- 172.21.15.119:0/871616722 >> 172.21.15.119:0/871616722 conn(0x7f440806aba0 msgr2=0x7f440806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.992+0000 7f440d800640 1 -- 172.21.15.119:0/871616722 shutdown_connections 2024-04-25T10:07:37.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.992+0000 7f440d800640 1 -- 172.21.15.119:0/871616722 wait complete. 2024-04-25T10:07:37.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.994+0000 7f440d800640 1 Processor -- start 2024-04-25T10:07:37.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.994+0000 7f440d800640 1 -- start start 2024-04-25T10:07:38.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f440d800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44080cc300 0x7f440815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f440d800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44080ce9d0 0x7f44080b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f440d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4408144e70 0x7f44081f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f440d800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440806d970 con 0x7f44080ce9d0 2024-04-25T10:07:38.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f440d800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44080cf3a0 con 0x7f44080cc300 2024-04-25T10:07:38.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f440d800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44081454b0 con 0x7f4408144e70 2024-04-25T10:07:38.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f43ffe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44080ce9d0 0x7f44080b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f43ffe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44080ce9d0 0x7f44080b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55242/0 (socket says 172.21.15.119:55242) 2024-04-25T10:07:38.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f43ffe00640 1 -- 172.21.15.119:0/3272242755 learned_addr learned my addr 172.21.15.119:0/3272242755 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:38.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f43ffe00640 1 -- 172.21.15.119:0/3272242755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4408144e70 msgr2=0x7f44081f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f4406a00640 1 --2- 172.21.15.119:0/3272242755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44080cc300 0x7f440815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f43ffe00640 1 --2- 172.21.15.119:0/3272242755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4408144e70 0x7f44081f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f43ffe00640 1 -- 172.21.15.119:0/3272242755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44080cc300 msgr2=0x7f440815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f43ffe00640 1 --2- 172.21.15.119:0/3272242755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44080cc300 0x7f440815e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.995+0000 7f43ffe00640 1 -- 172.21.15.119:0/3272242755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440815dcf0 con 0x7f44080ce9d0 2024-04-25T10:07:38.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.996+0000 7f43ffe00640 1 --2- 172.21.15.119:0/3272242755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44080ce9d0 0x7f44080b1b50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f43f0000c00 tx=0x7f43f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.996+0000 7f43fea00640 1 -- 172.21.15.119:0/3272242755 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0041070 con 0x7f44080ce9d0 2024-04-25T10:07:38.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.996+0000 7f43fea00640 1 -- 172.21.15.119:0/3272242755 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f003c870 con 0x7f44080ce9d0 2024-04-25T10:07:38.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.996+0000 7f43fea00640 1 -- 172.21.15.119:0/3272242755 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0037040 con 0x7f44080ce9d0 2024-04-25T10:07:38.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.996+0000 7f440d800640 1 -- 172.21.15.119:0/3272242755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440813cb10 con 0x7f44080ce9d0 2024-04-25T10:07:38.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.996+0000 7f440d800640 1 -- 172.21.15.119:0/3272242755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440815df90 con 0x7f44080ce9d0 2024-04-25T10:07:38.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.997+0000 7f440d800640 1 -- 172.21.15.119:0/3272242755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44080750b0 con 0x7f44080ce9d0 2024-04-25T10:07:38.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.998+0000 7f43fea00640 1 -- 172.21.15.119:0/3272242755 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43f0044070 con 0x7f44080ce9d0 2024-04-25T10:07:38.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.999+0000 7f43fea00640 1 --2- 172.21.15.119:0/3272242755 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43e406b4d0 0x7f43e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.999+0000 7f43fea00640 1 -- 172.21.15.119:0/3272242755 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43f00aae70 con 0x7f44080ce9d0 2024-04-25T10:07:38.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:37.999+0000 7f4406a00640 1 --2- 172.21.15.119:0/3272242755 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43e406b4d0 0x7f43e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.000+0000 7f4406a00640 1 --2- 172.21.15.119:0/3272242755 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43e406b4d0 0x7f43e406d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f43f800a730 tx=0x7f43f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.004+0000 7f43fea00640 1 -- 172.21.15.119:0/3272242755 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43f0079c50 con 0x7f44080ce9d0 2024-04-25T10:07:38.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.196+0000 7f440d800640 1 -- 172.21.15.119:0/3272242755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f440806f130 con 0x7f44080ce9d0 2024-04-25T10:07:38.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.197+0000 7f43fea00640 1 -- 172.21.15.119:0/3272242755 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43f0079550 con 0x7f44080ce9d0 2024-04-25T10:07:38.203 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:07:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.199+0000 7f440d800640 1 -- 172.21.15.119:0/3272242755 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43e406b4d0 msgr2=0x7f43e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.199+0000 7f440d800640 1 --2- 172.21.15.119:0/3272242755 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43e406b4d0 0x7f43e406d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f43f800a730 tx=0x7f43f8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.199+0000 7f440d800640 1 -- 172.21.15.119:0/3272242755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44080ce9d0 msgr2=0x7f44080b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.199+0000 7f440d800640 1 --2- 172.21.15.119:0/3272242755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44080ce9d0 0x7f44080b1b50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f43f0000c00 tx=0x7f43f00046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.199+0000 7f440d800640 1 -- 172.21.15.119:0/3272242755 shutdown_connections 2024-04-25T10:07:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.199+0000 7f440d800640 1 --2- 172.21.15.119:0/3272242755 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43e406b4d0 0x7f43e406d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.199+0000 7f440d800640 1 --2- 172.21.15.119:0/3272242755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4408144e70 0x7f44081f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.199+0000 7f440d800640 1 --2- 172.21.15.119:0/3272242755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44080ce9d0 0x7f44080b1b50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.199+0000 7f440d800640 1 --2- 172.21.15.119:0/3272242755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44080cc300 0x7f440815e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.199+0000 7f440d800640 1 -- 172.21.15.119:0/3272242755 >> 172.21.15.119:0/3272242755 conn(0x7f440806aba0 msgr2=0x7f44080ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.199+0000 7f440d800640 1 -- 172.21.15.119:0/3272242755 shutdown_connections 2024-04-25T10:07:38.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.199+0000 7f440d800640 1 -- 172.21.15.119:0/3272242755 wait complete. 2024-04-25T10:07:38.225 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:38.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.425+0000 7f1556c00640 1 Processor -- start 2024-04-25T10:07:38.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.425+0000 7f1556c00640 1 -- start start 2024-04-25T10:07:38.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.426+0000 7f1556c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f15500ce3a0 0x7f15500b1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.426+0000 7f1556c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155015d450 0x7f155015d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.426+0000 7f1556c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15500b1f60 con 0x7f155006aba0 2024-04-25T10:07:38.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.426+0000 7f1556c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15500b0130 con 0x7f15500ce3a0 2024-04-25T10:07:38.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.426+0000 7f1556c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15500b09d0 con 0x7f155015d450 2024-04-25T10:07:38.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.427+0000 7f154f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f15500ce3a0 0x7f15500b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.427+0000 7f154f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f15500ce3a0 0x7f15500b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55256/0 (socket says 172.21.15.119:55256) 2024-04-25T10:07:38.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.427+0000 7f154f400640 1 -- 172.21.15.119:0/3695331379 learned_addr learned my addr 172.21.15.119:0/3695331379 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:38.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.427+0000 7f154f400640 1 -- 172.21.15.119:0/3695331379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155015d450 msgr2=0x7f155015d8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:07:38.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.427+0000 7f154f400640 1 --2- 172.21.15.119:0/3695331379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155015d450 0x7f155015d8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.427+0000 7f154f400640 1 -- 172.21.15.119:0/3695331379 >> v1:172.21.15.119:6789/0 conn(0x7f155006aba0 legacy=0x7f155006af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.427+0000 7f154f400640 1 -- 172.21.15.119:0/3695331379 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15500b0380 con 0x7f15500ce3a0 2024-04-25T10:07:38.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.428+0000 7f154f400640 1 --2- 172.21.15.119:0/3695331379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15500ce3a0 0x7f15500b1980 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1538009870 tx=0x7f153802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652df064414c58ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.428+0000 7f154ea00640 1 -- 172.21.15.119:0/3695331379 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538037070 con 0x7f15500ce3a0 2024-04-25T10:07:38.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.428+0000 7f154ea00640 1 -- 172.21.15.119:0/3695331379 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1538032440 con 0x7f15500ce3a0 2024-04-25T10:07:38.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.428+0000 7f154ea00640 1 -- 172.21.15.119:0/3695331379 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538032810 con 0x7f15500ce3a0 2024-04-25T10:07:38.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.429+0000 7f1556c00640 1 -- 172.21.15.119:0/3695331379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15500ce3a0 msgr2=0x7f15500b1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.429+0000 7f1556c00640 1 --2- 172.21.15.119:0/3695331379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15500ce3a0 0x7f15500b1980 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1538009870 tx=0x7f153802a270 comp rx=0 tx=0).stop 2024-04-25T10:07:38.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.429+0000 7f1556c00640 1 -- 172.21.15.119:0/3695331379 shutdown_connections 2024-04-25T10:07:38.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.429+0000 7f1556c00640 1 --2- 172.21.15.119:0/3695331379 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155015d450 0x7f155015d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.429+0000 7f1556c00640 1 --2- 172.21.15.119:0/3695331379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15500ce3a0 0x7f15500b1980 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.429+0000 7f1556c00640 1 -- 172.21.15.119:0/3695331379 >> 172.21.15.119:0/3695331379 conn(0x7f15500d07d0 msgr2=0x7f15500d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.429+0000 7f1556c00640 1 -- 172.21.15.119:0/3695331379 shutdown_connections 2024-04-25T10:07:38.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.429+0000 7f1556c00640 1 -- 172.21.15.119:0/3695331379 wait complete. 2024-04-25T10:07:38.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.431+0000 7f1556c00640 1 Processor -- start 2024-04-25T10:07:38.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.431+0000 7f1556c00640 1 -- start start 2024-04-25T10:07:38.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.432+0000 7f1556c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155006aba0 0x7f15500b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.432+0000 7f1556c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15500ce3a0 0x7f1550154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.432+0000 7f1556c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155015d450 0x7f1550159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.432+0000 7f1556c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15500b0e60 con 0x7f155015d450 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.432+0000 7f1556c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155006b880 con 0x7f15500ce3a0 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.432+0000 7f1556c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15500b0fe0 con 0x7f155006aba0 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.432+0000 7f1554c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155015d450 0x7f1550159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.433+0000 7f1554c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155015d450 0x7f1550159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55260/0 (socket says 172.21.15.119:55260) 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.433+0000 7f1554c00640 1 -- 172.21.15.119:0/433510099 learned_addr learned my addr 172.21.15.119:0/433510099 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.433+0000 7f1554c00640 1 -- 172.21.15.119:0/433510099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155006aba0 msgr2=0x7f15500b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.433+0000 7f1554c00640 1 --2- 172.21.15.119:0/433510099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155006aba0 0x7f15500b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.433+0000 7f1554c00640 1 -- 172.21.15.119:0/433510099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15500ce3a0 msgr2=0x7f1550154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.433+0000 7f1554c00640 1 --2- 172.21.15.119:0/433510099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15500ce3a0 0x7f1550154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.433+0000 7f1554c00640 1 -- 172.21.15.119:0/433510099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1550155390 con 0x7f155015d450 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.433+0000 7f1554c00640 1 --2- 172.21.15.119:0/433510099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155015d450 0x7f1550159530 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f154800b9c0 tx=0x7f154800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.433+0000 7f1547e00640 1 -- 172.21.15.119:0/433510099 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548019070 con 0x7f155015d450 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.434+0000 7f1547e00640 1 -- 172.21.15.119:0/433510099 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154800cba0 con 0x7f155015d450 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.434+0000 7f1547e00640 1 -- 172.21.15.119:0/433510099 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548014440 con 0x7f155015d450 2024-04-25T10:07:38.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.434+0000 7f1556c00640 1 -- 172.21.15.119:0/433510099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15500c6560 con 0x7f155015d450 2024-04-25T10:07:38.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.434+0000 7f1556c00640 1 -- 172.21.15.119:0/433510099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15500c6b20 con 0x7f155015d450 2024-04-25T10:07:38.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.435+0000 7f1556c00640 1 -- 172.21.15.119:0/433510099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15500750b0 con 0x7f155015d450 2024-04-25T10:07:38.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.436+0000 7f1547e00640 1 -- 172.21.15.119:0/433510099 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f154801c070 con 0x7f155015d450 2024-04-25T10:07:38.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.437+0000 7f1547e00640 1 --2- 172.21.15.119:0/433510099 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f152c06b4d0 0x7f152c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.437+0000 7f1547e00640 1 -- 172.21.15.119:0/433510099 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1548089fe0 con 0x7f155015d450 2024-04-25T10:07:38.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.437+0000 7f154fe00640 1 --2- 172.21.15.119:0/433510099 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f152c06b4d0 0x7f152c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.438+0000 7f154fe00640 1 --2- 172.21.15.119:0/433510099 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f152c06b4d0 0x7f152c06d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1540000f60 tx=0x7f15400079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.442+0000 7f1547e00640 1 -- 172.21.15.119:0/433510099 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1548058e20 con 0x7f155015d450 2024-04-25T10:07:38.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.638+0000 7f1556c00640 1 -- 172.21.15.119:0/433510099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1550072f40 con 0x7f155015d450 2024-04-25T10:07:38.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.639+0000 7f1547e00640 1 -- 172.21.15.119:0/433510099 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1548058720 con 0x7f155015d450 2024-04-25T10:07:38.645 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:07:38.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.641+0000 7f1556c00640 1 -- 172.21.15.119:0/433510099 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f152c06b4d0 msgr2=0x7f152c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.641+0000 7f1556c00640 1 --2- 172.21.15.119:0/433510099 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f152c06b4d0 0x7f152c06d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1540000f60 tx=0x7f15400079d0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.641+0000 7f1556c00640 1 -- 172.21.15.119:0/433510099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155015d450 msgr2=0x7f1550159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.641+0000 7f1556c00640 1 --2- 172.21.15.119:0/433510099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155015d450 0x7f1550159530 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f154800b9c0 tx=0x7f154800be80 comp rx=0 tx=0).stop 2024-04-25T10:07:38.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.642+0000 7f1556c00640 1 -- 172.21.15.119:0/433510099 shutdown_connections 2024-04-25T10:07:38.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.642+0000 7f1556c00640 1 --2- 172.21.15.119:0/433510099 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f152c06b4d0 0x7f152c06d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.642+0000 7f1556c00640 1 --2- 172.21.15.119:0/433510099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155015d450 0x7f1550159530 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.642+0000 7f1556c00640 1 --2- 172.21.15.119:0/433510099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f15500ce3a0 0x7f1550154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.642+0000 7f1556c00640 1 --2- 172.21.15.119:0/433510099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f155006aba0 0x7f15500b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.642+0000 7f1556c00640 1 -- 172.21.15.119:0/433510099 >> 172.21.15.119:0/433510099 conn(0x7f15500d07d0 msgr2=0x7f15500cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.642+0000 7f1556c00640 1 -- 172.21.15.119:0/433510099 shutdown_connections 2024-04-25T10:07:38.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.642+0000 7f1556c00640 1 -- 172.21.15.119:0/433510099 wait complete. 2024-04-25T10:07:38.665 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:38.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.866+0000 7fccc2200640 1 Processor -- start 2024-04-25T10:07:38.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.866+0000 7fccc2200640 1 -- start start 2024-04-25T10:07:38.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.867+0000 7fccc2200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fccbc1450e0 0x7fccbc06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.867+0000 7fccc2200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccbc06b0e0 0x7fccbc0cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.867+0000 7fccc2200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccbc15f400 con 0x7fccbc0cc810 2024-04-25T10:07:38.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.867+0000 7fccc2200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccbc1454c0 con 0x7fccbc1450e0 2024-04-25T10:07:38.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.868+0000 7fccc2200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc06b540 con 0x7fccbc06b0e0 2024-04-25T10:07:38.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.868+0000 7fccc0e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fccbc1450e0 0x7fccbc06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.868+0000 7fccc0e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fccbc1450e0 0x7fccbc06aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55268/0 (socket says 172.21.15.119:55268) 2024-04-25T10:07:38.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.868+0000 7fccc0e00640 1 -- 172.21.15.119:0/1927019032 learned_addr learned my addr 172.21.15.119:0/1927019032 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:38.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.868+0000 7fccbbe00640 1 --2- 172.21.15.119:0/1927019032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccbc06b0e0 0x7fccbc0cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.868+0000 7fccc0e00640 1 -- 172.21.15.119:0/1927019032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccbc06b0e0 msgr2=0x7fccbc0cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.868+0000 7fccc0e00640 1 --2- 172.21.15.119:0/1927019032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccbc06b0e0 0x7fccbc0cc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.868+0000 7fccc0e00640 1 -- 172.21.15.119:0/1927019032 >> v1:172.21.15.119:6789/0 conn(0x7fccbc0cc810 legacy=0x7fccbc071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.868+0000 7fccc0e00640 1 -- 172.21.15.119:0/1927019032 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc15f6e0 con 0x7fccbc1450e0 2024-04-25T10:07:38.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.868+0000 7fccbb400640 1 -- 172.21.15.119:0/1927019032 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673920405 0 0) 0x7fccbc15f400 con 0x7fccbc0cc810 2024-04-25T10:07:38.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.869+0000 7fccc0e00640 1 --2- 172.21.15.119:0/1927019032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccbc1450e0 0x7fccbc06aba0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009870 tx=0x7fccb002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2476b1df477c2ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.870+0000 7fccbb400640 1 -- 172.21.15.119:0/1927019032 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0037070 con 0x7fccbc1450e0 2024-04-25T10:07:38.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.870+0000 7fccbb400640 1 -- 172.21.15.119:0/1927019032 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb0032440 con 0x7fccbc1450e0 2024-04-25T10:07:38.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.870+0000 7fccbb400640 1 -- 172.21.15.119:0/1927019032 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0032810 con 0x7fccbc1450e0 2024-04-25T10:07:38.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.870+0000 7fccc2200640 1 -- 172.21.15.119:0/1927019032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccbc1450e0 msgr2=0x7fccbc06aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.870+0000 7fccc2200640 1 --2- 172.21.15.119:0/1927019032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccbc1450e0 0x7fccbc06aba0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009870 tx=0x7fccb002a130 comp rx=0 tx=0).stop 2024-04-25T10:07:38.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.870+0000 7fccc2200640 1 -- 172.21.15.119:0/1927019032 shutdown_connections 2024-04-25T10:07:38.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.870+0000 7fccc2200640 1 --2- 172.21.15.119:0/1927019032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccbc06b0e0 0x7fccbc0cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.870+0000 7fccc2200640 1 --2- 172.21.15.119:0/1927019032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccbc1450e0 0x7fccbc06aba0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.870+0000 7fccc2200640 1 -- 172.21.15.119:0/1927019032 >> 172.21.15.119:0/1927019032 conn(0x7fccbc066ff0 msgr2=0x7fccbc0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.870+0000 7fccc2200640 1 -- 172.21.15.119:0/1927019032 shutdown_connections 2024-04-25T10:07:38.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.870+0000 7fccc2200640 1 -- 172.21.15.119:0/1927019032 wait complete. 2024-04-25T10:07:38.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.872+0000 7fccc2200640 1 Processor -- start 2024-04-25T10:07:38.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.872+0000 7fccc2200640 1 -- start start 2024-04-25T10:07:38.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.873+0000 7fccc2200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccbc06b0e0 0x7fccbc141200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.873+0000 7fccc2200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccbc0cc810 0x7fccbc1f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.873+0000 7fccc2200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccbc1450e0 0x7fccbc13dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.873+0000 7fccc2200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc0ccca0 con 0x7fccbc06b0e0 2024-04-25T10:07:38.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.873+0000 7fccc2200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc15fca0 con 0x7fccbc1450e0 2024-04-25T10:07:38.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.873+0000 7fccc2200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc06dc50 con 0x7fccbc0cc810 2024-04-25T10:07:38.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.873+0000 7fccc0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccbc06b0e0 0x7fccbc141200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.874+0000 7fccc0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccbc06b0e0 0x7fccbc141200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55282/0 (socket says 172.21.15.119:55282) 2024-04-25T10:07:38.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.874+0000 7fccc0e00640 1 -- 172.21.15.119:0/1338705852 learned_addr learned my addr 172.21.15.119:0/1338705852 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:38.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.874+0000 7fccc0e00640 1 -- 172.21.15.119:0/1338705852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccbc0cc810 msgr2=0x7fccbc1f4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.874+0000 7fccc0e00640 1 --2- 172.21.15.119:0/1338705852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccbc0cc810 0x7fccbc1f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.874+0000 7fccc0e00640 1 -- 172.21.15.119:0/1338705852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccbc1450e0 msgr2=0x7fccbc13dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.874+0000 7fccc0e00640 1 --2- 172.21.15.119:0/1338705852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccbc1450e0 0x7fccbc13dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.874+0000 7fccc0e00640 1 -- 172.21.15.119:0/1338705852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc13e520 con 0x7fccbc06b0e0 2024-04-25T10:07:38.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.874+0000 7fccc0e00640 1 --2- 172.21.15.119:0/1338705852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccbc06b0e0 0x7fccbc141200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fccb002a170 tx=0x7fccb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.874+0000 7fccb8c00640 1 -- 172.21.15.119:0/1338705852 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0037040 con 0x7fccbc06b0e0 2024-04-25T10:07:38.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.874+0000 7fccb8c00640 1 -- 172.21.15.119:0/1338705852 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb0042070 con 0x7fccbc06b0e0 2024-04-25T10:07:38.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.874+0000 7fccc2200640 1 -- 172.21.15.119:0/1338705852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccbc06d080 con 0x7fccbc06b0e0 2024-04-25T10:07:38.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.874+0000 7fccb8c00640 1 -- 172.21.15.119:0/1338705852 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0030bd0 con 0x7fccbc06b0e0 2024-04-25T10:07:38.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.875+0000 7fccc2200640 1 -- 172.21.15.119:0/1338705852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccbc06d600 con 0x7fccbc06b0e0 2024-04-25T10:07:38.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.876+0000 7fccc2200640 1 -- 172.21.15.119:0/1338705852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccbc0749b0 con 0x7fccbc06b0e0 2024-04-25T10:07:38.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.877+0000 7fccb8c00640 1 -- 172.21.15.119:0/1338705852 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccb0047460 con 0x7fccbc06b0e0 2024-04-25T10:07:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.878+0000 7fccb8c00640 1 --2- 172.21.15.119:0/1338705852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcc9c06b4d0 0x7fcc9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.878+0000 7fccb8c00640 1 -- 172.21.15.119:0/1338705852 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccb00aaf00 con 0x7fccbc06b0e0 2024-04-25T10:07:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.878+0000 7fccbbe00640 1 --2- 172.21.15.119:0/1338705852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcc9c06b4d0 0x7fcc9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.879+0000 7fccbbe00640 1 --2- 172.21.15.119:0/1338705852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcc9c06b4d0 0x7fcc9c06d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcca8006fd0 tx=0x7fcca8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:38.882+0000 7fccb8c00640 1 -- 172.21.15.119:0/1338705852 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fccbc13e520 con 0x7fccbc06b0e0 2024-04-25T10:07:39.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.071+0000 7fccc2200640 1 -- 172.21.15.119:0/1338705852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fccbc0cbec0 con 0x7fccbc06b0e0 2024-04-25T10:07:39.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.072+0000 7fccb8c00640 1 -- 172.21.15.119:0/1338705852 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fccb0045360 con 0x7fccbc06b0e0 2024-04-25T10:07:39.078 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:07:39.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.075+0000 7fccc2200640 1 -- 172.21.15.119:0/1338705852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcc9c06b4d0 msgr2=0x7fcc9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.075+0000 7fccc2200640 1 --2- 172.21.15.119:0/1338705852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcc9c06b4d0 0x7fcc9c06d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcca8006fd0 tx=0x7fcca8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:39.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.075+0000 7fccc2200640 1 -- 172.21.15.119:0/1338705852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccbc06b0e0 msgr2=0x7fccbc141200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.075+0000 7fccc2200640 1 --2- 172.21.15.119:0/1338705852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccbc06b0e0 0x7fccbc141200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fccb002a170 tx=0x7fccb00046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.075+0000 7fccc2200640 1 -- 172.21.15.119:0/1338705852 shutdown_connections 2024-04-25T10:07:39.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.075+0000 7fccc2200640 1 --2- 172.21.15.119:0/1338705852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcc9c06b4d0 0x7fcc9c06d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.075+0000 7fccc2200640 1 --2- 172.21.15.119:0/1338705852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccbc1450e0 0x7fccbc13dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.075+0000 7fccc2200640 1 --2- 172.21.15.119:0/1338705852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccbc0cc810 0x7fccbc1f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.075+0000 7fccc2200640 1 --2- 172.21.15.119:0/1338705852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccbc06b0e0 0x7fccbc141200 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.075+0000 7fccc2200640 1 -- 172.21.15.119:0/1338705852 >> 172.21.15.119:0/1338705852 conn(0x7fccbc066ff0 msgr2=0x7fccbc0af090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:39.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.075+0000 7fccc2200640 1 -- 172.21.15.119:0/1338705852 shutdown_connections 2024-04-25T10:07:39.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.075+0000 7fccc2200640 1 -- 172.21.15.119:0/1338705852 wait complete. 2024-04-25T10:07:39.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:39.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.291+0000 7fea25e00640 1 Processor -- start 2024-04-25T10:07:39.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.291+0000 7fea25e00640 1 -- start start 2024-04-25T10:07:39.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.292+0000 7fea25e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fea2015c9f0 0x7fea201664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.292+0000 7fea25e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea200ce6c0 con 0x7fea2015bf00 2024-04-25T10:07:39.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.292+0000 7fea25e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea200cc4e0 con 0x7fea2006e8f0 2024-04-25T10:07:39.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.292+0000 7fea25e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea201455e0 con 0x7fea2015c9f0 2024-04-25T10:07:39.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.293+0000 7fea1f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fea2015c9f0 0x7fea201664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.293+0000 7fea1f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fea2015c9f0 0x7fea201664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55298/0 (socket says 172.21.15.119:55298) 2024-04-25T10:07:39.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.293+0000 7fea1f400640 1 -- 172.21.15.119:0/2071890536 learned_addr learned my addr 172.21.15.119:0/2071890536 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:39.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.293+0000 7fea1f400640 1 -- 172.21.15.119:0/2071890536 >> v1:172.21.15.132:6789/0 conn(0x7fea2006e8f0 legacy=0x7fea2015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:39.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.293+0000 7fea1f400640 1 -- 172.21.15.119:0/2071890536 >> v1:172.21.15.119:6789/0 conn(0x7fea2015bf00 legacy=0x7fea200cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.293+0000 7fea1f400640 1 -- 172.21.15.119:0/2071890536 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea200cc7b0 con 0x7fea2015c9f0 2024-04-25T10:07:39.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.294+0000 7fea1f400640 1 --2- 172.21.15.119:0/2071890536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2015c9f0 0x7fea201664c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fea18009e50 tx=0x7fea1802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8698280ac4b3b2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.294+0000 7fea1e000640 1 -- 172.21.15.119:0/2071890536 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18036070 con 0x7fea2015c9f0 2024-04-25T10:07:39.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.295+0000 7fea1e000640 1 -- 172.21.15.119:0/2071890536 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea18031600 con 0x7fea2015c9f0 2024-04-25T10:07:39.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.295+0000 7fea1e000640 1 -- 172.21.15.119:0/2071890536 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea180319d0 con 0x7fea2015c9f0 2024-04-25T10:07:39.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.295+0000 7fea25e00640 1 -- 172.21.15.119:0/2071890536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2015c9f0 msgr2=0x7fea201664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.295+0000 7fea25e00640 1 --2- 172.21.15.119:0/2071890536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2015c9f0 0x7fea201664c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fea18009e50 tx=0x7fea1802a760 comp rx=0 tx=0).stop 2024-04-25T10:07:39.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.295+0000 7fea25e00640 1 -- 172.21.15.119:0/2071890536 shutdown_connections 2024-04-25T10:07:39.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.295+0000 7fea25e00640 1 --2- 172.21.15.119:0/2071890536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2015c9f0 0x7fea201664c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.295+0000 7fea25e00640 1 -- 172.21.15.119:0/2071890536 >> 172.21.15.119:0/2071890536 conn(0x7fea2006aba0 msgr2=0x7fea2006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:39.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.295+0000 7fea25e00640 1 -- 172.21.15.119:0/2071890536 shutdown_connections 2024-04-25T10:07:39.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.295+0000 7fea25e00640 1 -- 172.21.15.119:0/2071890536 wait complete. 2024-04-25T10:07:39.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.297+0000 7fea25e00640 1 Processor -- start 2024-04-25T10:07:39.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.297+0000 7fea25e00640 1 -- start start 2024-04-25T10:07:39.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.298+0000 7fea25e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea2006e8f0 0x7fea20137fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.298+0000 7fea25e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2015bf00 0x7fea20143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.298+0000 7fea25e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2015c9f0 0x7fea20071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.298+0000 7fea25e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea200ccd70 con 0x7fea2015c9f0 2024-04-25T10:07:39.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.298+0000 7fea25e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2006d450 con 0x7fea2015bf00 2024-04-25T10:07:39.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.298+0000 7fea25e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea200cd050 con 0x7fea2006e8f0 2024-04-25T10:07:39.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.299+0000 7fea1f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2015c9f0 0x7fea20071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.299+0000 7fea1f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2015c9f0 0x7fea20071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55304/0 (socket says 172.21.15.119:55304) 2024-04-25T10:07:39.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.299+0000 7fea1f400640 1 -- 172.21.15.119:0/3592826598 learned_addr learned my addr 172.21.15.119:0/3592826598 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:39.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.299+0000 7fea1ea00640 1 --2- 172.21.15.119:0/3592826598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea2006e8f0 0x7fea20137fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.299+0000 7fea17e00640 1 --2- 172.21.15.119:0/3592826598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2015bf00 0x7fea20143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.299+0000 7fea1f400640 1 -- 172.21.15.119:0/3592826598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea2006e8f0 msgr2=0x7fea20137fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.299+0000 7fea1f400640 1 --2- 172.21.15.119:0/3592826598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea2006e8f0 0x7fea20137fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.299+0000 7fea1f400640 1 -- 172.21.15.119:0/3592826598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2015bf00 msgr2=0x7fea20143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.299+0000 7fea1f400640 1 --2- 172.21.15.119:0/3592826598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2015bf00 0x7fea20143cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.299+0000 7fea1f400640 1 -- 172.21.15.119:0/3592826598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea20144760 con 0x7fea2015c9f0 2024-04-25T10:07:39.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.299+0000 7fea1f400640 1 --2- 172.21.15.119:0/3592826598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2015c9f0 0x7fea20071190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fea18009e50 tx=0x7fea18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.300+0000 7fea17400640 1 -- 172.21.15.119:0/3592826598 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18041020 con 0x7fea2015c9f0 2024-04-25T10:07:39.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.300+0000 7fea17400640 1 -- 172.21.15.119:0/3592826598 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea18007780 con 0x7fea2015c9f0 2024-04-25T10:07:39.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.300+0000 7fea17400640 1 -- 172.21.15.119:0/3592826598 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18036040 con 0x7fea2015c9f0 2024-04-25T10:07:39.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.300+0000 7fea25e00640 1 -- 172.21.15.119:0/3592826598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea20144a40 con 0x7fea2015c9f0 2024-04-25T10:07:39.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.300+0000 7fea25e00640 1 -- 172.21.15.119:0/3592826598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2014d710 con 0x7fea2015c9f0 2024-04-25T10:07:39.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.301+0000 7fea25e00640 1 -- 172.21.15.119:0/3592826598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2006bb30 con 0x7fea2015c9f0 2024-04-25T10:07:39.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.302+0000 7fea17400640 1 -- 172.21.15.119:0/3592826598 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea18044020 con 0x7fea2015c9f0 2024-04-25T10:07:39.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.303+0000 7fea17400640 1 --2- 172.21.15.119:0/3592826598 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe9fc06b4d0 0x7fe9fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.304+0000 7fea17400640 1 -- 172.21.15.119:0/3592826598 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea180aaeb0 con 0x7fea2015c9f0 2024-04-25T10:07:39.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.304+0000 7fea1ea00640 1 --2- 172.21.15.119:0/3592826598 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe9fc06b4d0 0x7fe9fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.304+0000 7fea1ea00640 1 --2- 172.21.15.119:0/3592826598 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe9fc06b4d0 0x7fe9fc06d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fea10001640 tx=0x7fea10009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.308+0000 7fea17400640 1 -- 172.21.15.119:0/3592826598 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea1804b030 con 0x7fea2015c9f0 2024-04-25T10:07:39.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.508+0000 7fea25e00640 1 -- 172.21.15.119:0/3592826598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea2014d910 con 0x7fea2015c9f0 2024-04-25T10:07:39.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.511+0000 7fea17400640 1 -- 172.21.15.119:0/3592826598 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fea2014d910 con 0x7fea2015c9f0 2024-04-25T10:07:39.517 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:39.517 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:08.319104+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.513+0000 7fea25e00640 1 -- 172.21.15.119:0/3592826598 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe9fc06b4d0 msgr2=0x7fe9fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.513+0000 7fea25e00640 1 --2- 172.21.15.119:0/3592826598 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe9fc06b4d0 0x7fe9fc06d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fea10001640 tx=0x7fea10009040 comp rx=0 tx=0).stop 2024-04-25T10:07:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.513+0000 7fea25e00640 1 -- 172.21.15.119:0/3592826598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2015c9f0 msgr2=0x7fea20071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.513+0000 7fea25e00640 1 --2- 172.21.15.119:0/3592826598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2015c9f0 0x7fea20071190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fea18009e50 tx=0x7fea18004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.514+0000 7fea25e00640 1 -- 172.21.15.119:0/3592826598 shutdown_connections 2024-04-25T10:07:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.514+0000 7fea25e00640 1 --2- 172.21.15.119:0/3592826598 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe9fc06b4d0 0x7fe9fc06d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.514+0000 7fea25e00640 1 --2- 172.21.15.119:0/3592826598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2015c9f0 0x7fea20071190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.514+0000 7fea25e00640 1 --2- 172.21.15.119:0/3592826598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea2015bf00 0x7fea20143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.514+0000 7fea25e00640 1 --2- 172.21.15.119:0/3592826598 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea2006e8f0 0x7fea20137fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.514+0000 7fea25e00640 1 -- 172.21.15.119:0/3592826598 >> 172.21.15.119:0/3592826598 conn(0x7fea2006aba0 msgr2=0x7fea2015cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.514+0000 7fea25e00640 1 -- 172.21.15.119:0/3592826598 shutdown_connections 2024-04-25T10:07:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.514+0000 7fea25e00640 1 -- 172.21.15.119:0/3592826598 wait complete. 2024-04-25T10:07:39.538 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:05.135832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:05.768083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:39.539 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:39.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.739+0000 7f6a57200640 1 Processor -- start 2024-04-25T10:07:39.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.740+0000 7f6a57200640 1 -- start start 2024-04-25T10:07:39.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.741+0000 7f6a57200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5806b2b0 0x7f6a580cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.741+0000 7f6a57200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6a580cc810 0x7f6a580b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.741+0000 7f6a57200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5815f400 con 0x7f6a581450e0 2024-04-25T10:07:39.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.741+0000 7f6a57200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a581454c0 con 0x7f6a580cc810 2024-04-25T10:07:39.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.741+0000 7f6a57200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5815f6e0 con 0x7f6a5806b2b0 2024-04-25T10:07:39.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.742+0000 7f6a55e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6a581450e0 0x7f6a5806aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51602/0 (socket says 172.21.15.119:51602) 2024-04-25T10:07:39.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.742+0000 7f6a55e00640 1 -- 172.21.15.119:0/707327930 learned_addr learned my addr 172.21.15.119:0/707327930 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:39.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.742+0000 7f6a56800640 1 --2- 172.21.15.119:0/707327930 >> v2:172.21.15.119:3300/0 conn(0x7f6a580cc810 0x7f6a580b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.742+0000 7f6a54a00640 1 -- 172.21.15.119:0/707327930 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813316826 0 0) 0x7f6a5815f400 con 0x7f6a581450e0 2024-04-25T10:07:39.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.742+0000 7f6a54a00640 1 -- 172.21.15.119:0/707327930 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a38003520 con 0x7f6a581450e0 2024-04-25T10:07:39.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.742+0000 7f6a55400640 1 --2- 172.21.15.119:0/707327930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5806b2b0 0x7f6a580cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.742+0000 7f6a56800640 1 -- 172.21.15.119:0/707327930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5806b2b0 msgr2=0x7f6a580cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:39.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.742+0000 7f6a56800640 1 --2- 172.21.15.119:0/707327930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5806b2b0 0x7f6a580cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.742+0000 7f6a56800640 1 -- 172.21.15.119:0/707327930 >> v1:172.21.15.119:6789/0 conn(0x7f6a581450e0 legacy=0x7f6a5806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.742+0000 7f6a56800640 1 -- 172.21.15.119:0/707327930 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5815f9c0 con 0x7f6a580cc810 2024-04-25T10:07:39.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.742+0000 7f6a55400640 1 --2- 172.21.15.119:0/707327930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5806b2b0 0x7f6a580cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:39.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.743+0000 7f6a56800640 1 --2- 172.21.15.119:0/707327930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a580cc810 0x7f6a580b2e80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6a500097a0 tx=0x7f6a5002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c4f58b3c98ccb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.744+0000 7f6a54a00640 1 -- 172.21.15.119:0/707327930 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50037070 con 0x7f6a580cc810 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.744+0000 7f6a54a00640 1 -- 172.21.15.119:0/707327930 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5002adb0 con 0x7f6a580cc810 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.744+0000 7f6a54a00640 1 -- 172.21.15.119:0/707327930 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50032660 con 0x7f6a580cc810 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.744+0000 7f6a57200640 1 -- 172.21.15.119:0/707327930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a580cc810 msgr2=0x7f6a580b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.744+0000 7f6a57200640 1 --2- 172.21.15.119:0/707327930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a580cc810 0x7f6a580b2e80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6a500097a0 tx=0x7f6a5002a060 comp rx=0 tx=0).stop 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.744+0000 7f6a57200640 1 -- 172.21.15.119:0/707327930 shutdown_connections 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.744+0000 7f6a57200640 1 --2- 172.21.15.119:0/707327930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a580cc810 0x7f6a580b2e80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.744+0000 7f6a57200640 1 --2- 172.21.15.119:0/707327930 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a5806b2b0 0x7f6a580cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.744+0000 7f6a57200640 1 -- 172.21.15.119:0/707327930 >> 172.21.15.119:0/707327930 conn(0x7f6a58066ff0 msgr2=0x7f6a580b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:39.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.744+0000 7f6a57200640 1 -- 172.21.15.119:0/707327930 shutdown_connections 2024-04-25T10:07:39.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.744+0000 7f6a57200640 1 -- 172.21.15.119:0/707327930 wait complete. 2024-04-25T10:07:39.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.746+0000 7f6a57200640 1 Processor -- start 2024-04-25T10:07:39.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.746+0000 7f6a57200640 1 -- start start 2024-04-25T10:07:39.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.747+0000 7f6a57200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a5806b2b0 0x7f6a58140c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.747+0000 7f6a57200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a580cc810 0x7f6a581f1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.747+0000 7f6a57200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a581450e0 0x7f6a580a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.747+0000 7f6a57200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580cce00 con 0x7f6a580cc810 2024-04-25T10:07:39.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.747+0000 7f6a57200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580ccc80 con 0x7f6a5806b2b0 2024-04-25T10:07:39.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.747+0000 7f6a57200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580b33c0 con 0x7f6a581450e0 2024-04-25T10:07:39.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.747+0000 7f6a55400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a580cc810 0x7f6a581f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.747+0000 7f6a55400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a580cc810 0x7f6a581f1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55316/0 (socket says 172.21.15.119:55316) 2024-04-25T10:07:39.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.747+0000 7f6a55400640 1 -- 172.21.15.119:0/1384585820 learned_addr learned my addr 172.21.15.119:0/1384585820 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:39.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.748+0000 7f6a55400640 1 -- 172.21.15.119:0/1384585820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a581450e0 msgr2=0x7f6a580a93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.748+0000 7f6a55400640 1 --2- 172.21.15.119:0/1384585820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a581450e0 0x7f6a580a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.748+0000 7f6a55400640 1 -- 172.21.15.119:0/1384585820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a5806b2b0 msgr2=0x7f6a58140c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.748+0000 7f6a55400640 1 --2- 172.21.15.119:0/1384585820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a5806b2b0 0x7f6a58140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.748+0000 7f6a55400640 1 -- 172.21.15.119:0/1384585820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a580aa3b0 con 0x7f6a580cc810 2024-04-25T10:07:39.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.748+0000 7f6a55400640 1 --2- 172.21.15.119:0/1384585820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a580cc810 0x7f6a581f1010 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000cf60 tx=0x7f6a4000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.748+0000 7f6a4e000640 1 -- 172.21.15.119:0/1384585820 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4001a070 con 0x7f6a580cc810 2024-04-25T10:07:39.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.748+0000 7f6a4e000640 1 -- 172.21.15.119:0/1384585820 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4000d840 con 0x7f6a580cc810 2024-04-25T10:07:39.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.748+0000 7f6a57200640 1 -- 172.21.15.119:0/1384585820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a580aa690 con 0x7f6a580cc810 2024-04-25T10:07:39.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.748+0000 7f6a4e000640 1 -- 172.21.15.119:0/1384585820 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000dc10 con 0x7f6a580cc810 2024-04-25T10:07:39.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.748+0000 7f6a57200640 1 -- 172.21.15.119:0/1384585820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a580a9e40 con 0x7f6a580cc810 2024-04-25T10:07:39.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.750+0000 7f6a57200640 1 -- 172.21.15.119:0/1384585820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a580749b0 con 0x7f6a580cc810 2024-04-25T10:07:39.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.751+0000 7f6a4e000640 1 -- 172.21.15.119:0/1384585820 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a4001d070 con 0x7f6a580cc810 2024-04-25T10:07:39.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.751+0000 7f6a4e000640 1 --2- 172.21.15.119:0/1384585820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6a3406b4d0 0x7f6a3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.752+0000 7f6a4e000640 1 -- 172.21.15.119:0/1384585820 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a4008aa30 con 0x7f6a580cc810 2024-04-25T10:07:39.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.752+0000 7f6a55e00640 1 --2- 172.21.15.119:0/1384585820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6a3406b4d0 0x7f6a3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.752+0000 7f6a55e00640 1 --2- 172.21.15.119:0/1384585820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6a3406b4d0 0x7f6a3406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6a48000f60 tx=0x7f6a480078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.756+0000 7f6a4e000640 1 -- 172.21.15.119:0/1384585820 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a40059250 con 0x7f6a580cc810 2024-04-25T10:07:39.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.949+0000 7f6a57200640 1 -- 172.21.15.119:0/1384585820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a580b2010 con 0x7f6a580cc810 2024-04-25T10:07:39.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.950+0000 7f6a4e000640 1 -- 172.21.15.119:0/1384585820 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a4005d2f0 con 0x7f6a580cc810 2024-04-25T10:07:39.956 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:07:39.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.952+0000 7f6a57200640 1 -- 172.21.15.119:0/1384585820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6a3406b4d0 msgr2=0x7f6a3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.952+0000 7f6a57200640 1 --2- 172.21.15.119:0/1384585820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6a3406b4d0 0x7f6a3406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6a48000f60 tx=0x7f6a480078f0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.952+0000 7f6a57200640 1 -- 172.21.15.119:0/1384585820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a580cc810 msgr2=0x7f6a581f1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.952+0000 7f6a57200640 1 --2- 172.21.15.119:0/1384585820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a580cc810 0x7f6a581f1010 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000cf60 tx=0x7f6a4000cf90 comp rx=0 tx=0).stop 2024-04-25T10:07:39.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.953+0000 7f6a57200640 1 -- 172.21.15.119:0/1384585820 shutdown_connections 2024-04-25T10:07:39.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.953+0000 7f6a57200640 1 --2- 172.21.15.119:0/1384585820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6a3406b4d0 0x7f6a3406d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.953+0000 7f6a57200640 1 --2- 172.21.15.119:0/1384585820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a581450e0 0x7f6a580a93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.953+0000 7f6a57200640 1 --2- 172.21.15.119:0/1384585820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a580cc810 0x7f6a581f1010 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.953+0000 7f6a57200640 1 --2- 172.21.15.119:0/1384585820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a5806b2b0 0x7f6a58140c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.953+0000 7f6a57200640 1 -- 172.21.15.119:0/1384585820 >> 172.21.15.119:0/1384585820 conn(0x7f6a58066ff0 msgr2=0x7f6a580714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:39.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.953+0000 7f6a57200640 1 -- 172.21.15.119:0/1384585820 shutdown_connections 2024-04-25T10:07:39.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:39.953+0000 7f6a57200640 1 -- 172.21.15.119:0/1384585820 wait complete. 2024-04-25T10:07:39.977 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:40.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.165+0000 7f6408e00640 1 Processor -- start 2024-04-25T10:07:40.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.165+0000 7f6408e00640 1 -- start start 2024-04-25T10:07:40.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.166+0000 7f6408e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f640406aba0 0x7f640406af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.166+0000 7f6408e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f640406f040 0x7f640415e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.166+0000 7f6408e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f640406b4c0 con 0x7f64040cec60 2024-04-25T10:07:40.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.166+0000 7f6408e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640406d080 con 0x7f640406aba0 2024-04-25T10:07:40.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.166+0000 7f6408e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640406d360 con 0x7f640406f040 2024-04-25T10:07:40.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.167+0000 7f6403400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f640406aba0 0x7f640406af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.167+0000 7f6403400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f640406aba0 0x7f640406af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55318/0 (socket says 172.21.15.119:55318) 2024-04-25T10:07:40.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.167+0000 7f6403400640 1 -- 172.21.15.119:0/846625282 learned_addr learned my addr 172.21.15.119:0/846625282 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:40.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.167+0000 7f6403400640 1 -- 172.21.15.119:0/846625282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f640406f040 msgr2=0x7f640415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:40.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.167+0000 7f6403400640 1 --2- 172.21.15.119:0/846625282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f640406f040 0x7f640415e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.167+0000 7f6403400640 1 -- 172.21.15.119:0/846625282 >> v1:172.21.15.132:6789/0 conn(0x7f64040cec60 legacy=0x7f640415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:40.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.167+0000 7f6403400640 1 -- 172.21.15.119:0/846625282 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640406d640 con 0x7f640406aba0 2024-04-25T10:07:40.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.168+0000 7f6403400640 1 --2- 172.21.15.119:0/846625282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f640406aba0 0x7f640406af80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009870 tx=0x7f63f402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfb9ee12d0d2858 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.168+0000 7f6402000640 1 -- 172.21.15.119:0/846625282 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4037070 con 0x7f640406aba0 2024-04-25T10:07:40.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.168+0000 7f6402000640 1 -- 172.21.15.119:0/846625282 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f4032440 con 0x7f640406aba0 2024-04-25T10:07:40.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.168+0000 7f6402000640 1 -- 172.21.15.119:0/846625282 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4032810 con 0x7f640406aba0 2024-04-25T10:07:40.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.169+0000 7f6408e00640 1 -- 172.21.15.119:0/846625282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f640406aba0 msgr2=0x7f640406af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.169+0000 7f6408e00640 1 --2- 172.21.15.119:0/846625282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f640406aba0 0x7f640406af80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009870 tx=0x7f63f402a200 comp rx=0 tx=0).stop 2024-04-25T10:07:40.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.169+0000 7f6408e00640 1 -- 172.21.15.119:0/846625282 shutdown_connections 2024-04-25T10:07:40.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.169+0000 7f6408e00640 1 --2- 172.21.15.119:0/846625282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f640406f040 0x7f640415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.169+0000 7f6408e00640 1 --2- 172.21.15.119:0/846625282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f640406aba0 0x7f640406af80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.169+0000 7f6408e00640 1 -- 172.21.15.119:0/846625282 >> 172.21.15.119:0/846625282 conn(0x7f6404066ff0 msgr2=0x7f64040b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:40.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.169+0000 7f6408e00640 1 -- 172.21.15.119:0/846625282 shutdown_connections 2024-04-25T10:07:40.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.169+0000 7f6408e00640 1 -- 172.21.15.119:0/846625282 wait complete. 2024-04-25T10:07:40.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.171+0000 7f6408e00640 1 Processor -- start 2024-04-25T10:07:40.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.171+0000 7f6408e00640 1 -- start start 2024-04-25T10:07:40.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.172+0000 7f6408e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f640406aba0 0x7f64040a9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.172+0000 7f6408e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f640406f040 0x7f6404071050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.172+0000 7f6408e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64040cec60 0x7f6404071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.172+0000 7f6408e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640415a980 con 0x7f640406aba0 2024-04-25T10:07:40.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.172+0000 7f6408e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640406dc00 con 0x7f640406f040 2024-04-25T10:07:40.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.172+0000 7f6408e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64040aed40 con 0x7f64040cec60 2024-04-25T10:07:40.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.172+0000 7f6403400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f640406aba0 0x7f64040a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.172+0000 7f6403400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f640406aba0 0x7f64040a9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55334/0 (socket says 172.21.15.119:55334) 2024-04-25T10:07:40.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.172+0000 7f6403400640 1 -- 172.21.15.119:0/967310242 learned_addr learned my addr 172.21.15.119:0/967310242 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:40.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.172+0000 7f6403400640 1 -- 172.21.15.119:0/967310242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64040cec60 msgr2=0x7f6404071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.172+0000 7f6403400640 1 --2- 172.21.15.119:0/967310242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64040cec60 0x7f6404071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.172+0000 7f6403400640 1 -- 172.21.15.119:0/967310242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f640406f040 msgr2=0x7f6404071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.172+0000 7f6403400640 1 --2- 172.21.15.119:0/967310242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f640406f040 0x7f6404071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.172+0000 7f6403400640 1 -- 172.21.15.119:0/967310242 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64040a9ca0 con 0x7f640406aba0 2024-04-25T10:07:40.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.173+0000 7f6403400640 1 --2- 172.21.15.119:0/967310242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f640406aba0 0x7f64040a9760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f63f4000c00 tx=0x7f63f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.173+0000 7f63fb400640 1 -- 172.21.15.119:0/967310242 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4041070 con 0x7f640406aba0 2024-04-25T10:07:40.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.173+0000 7f63fb400640 1 -- 172.21.15.119:0/967310242 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f403c870 con 0x7f640406aba0 2024-04-25T10:07:40.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.173+0000 7f6408e00640 1 -- 172.21.15.119:0/967310242 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64041f25f0 con 0x7f640406aba0 2024-04-25T10:07:40.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.173+0000 7f63fb400640 1 -- 172.21.15.119:0/967310242 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4037040 con 0x7f640406aba0 2024-04-25T10:07:40.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.173+0000 7f6408e00640 1 -- 172.21.15.119:0/967310242 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64041f2eb0 con 0x7f640406aba0 2024-04-25T10:07:40.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.174+0000 7f6408e00640 1 -- 172.21.15.119:0/967310242 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64041f29e0 con 0x7f640406aba0 2024-04-25T10:07:40.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.175+0000 7f63fb400640 1 -- 172.21.15.119:0/967310242 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63f4044070 con 0x7f640406aba0 2024-04-25T10:07:40.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.176+0000 7f63fb400640 1 --2- 172.21.15.119:0/967310242 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f63e406b4d0 0x7f63e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.176+0000 7f63fb400640 1 -- 172.21.15.119:0/967310242 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63f40aacc0 con 0x7f640406aba0 2024-04-25T10:07:40.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.176+0000 7f6402a00640 1 --2- 172.21.15.119:0/967310242 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f63e406b4d0 0x7f63e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.177+0000 7f6402a00640 1 --2- 172.21.15.119:0/967310242 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f63e406b4d0 0x7f63e406d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f64040b10a0 tx=0x7f63f0006c90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.180+0000 7f63fb400640 1 -- 172.21.15.119:0/967310242 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63f40b1080 con 0x7f640406aba0 2024-04-25T10:07:40.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.373+0000 7f6408e00640 1 -- 172.21.15.119:0/967310242 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64040cf1b0 con 0x7f640406aba0 2024-04-25T10:07:40.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.373+0000 7f63fb400640 1 -- 172.21.15.119:0/967310242 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f63f4079b30 con 0x7f640406aba0 2024-04-25T10:07:40.379 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:07:40.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.376+0000 7f6408e00640 1 -- 172.21.15.119:0/967310242 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f63e406b4d0 msgr2=0x7f63e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.376+0000 7f6408e00640 1 --2- 172.21.15.119:0/967310242 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f63e406b4d0 0x7f63e406d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f64040b10a0 tx=0x7f63f0006c90 comp rx=0 tx=0).stop 2024-04-25T10:07:40.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.376+0000 7f6408e00640 1 -- 172.21.15.119:0/967310242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f640406aba0 msgr2=0x7f64040a9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.376+0000 7f6408e00640 1 --2- 172.21.15.119:0/967310242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f640406aba0 0x7f64040a9760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f63f4000c00 tx=0x7f63f40046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.376+0000 7f6408e00640 1 -- 172.21.15.119:0/967310242 shutdown_connections 2024-04-25T10:07:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.376+0000 7f6408e00640 1 --2- 172.21.15.119:0/967310242 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f63e406b4d0 0x7f63e406d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.376+0000 7f6408e00640 1 --2- 172.21.15.119:0/967310242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64040cec60 0x7f6404071590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.376+0000 7f6408e00640 1 --2- 172.21.15.119:0/967310242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f640406f040 0x7f6404071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.376+0000 7f6408e00640 1 --2- 172.21.15.119:0/967310242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f640406aba0 0x7f64040a9760 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.376+0000 7f6408e00640 1 -- 172.21.15.119:0/967310242 >> 172.21.15.119:0/967310242 conn(0x7f6404066ff0 msgr2=0x7f64040ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.377+0000 7f6408e00640 1 -- 172.21.15.119:0/967310242 shutdown_connections 2024-04-25T10:07:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.377+0000 7f6408e00640 1 -- 172.21.15.119:0/967310242 wait complete. 2024-04-25T10:07:40.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:40.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.598+0000 7f522a800640 1 Processor -- start 2024-04-25T10:07:40.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.598+0000 7f522a800640 1 -- start start 2024-04-25T10:07:40.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.600+0000 7f522a800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522415b7b0 0x7f52240ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.600+0000 7f522a800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f522415e820 0x7f52240cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.600+0000 7f522a800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522415f000 con 0x7f52240cf760 2024-04-25T10:07:40.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.600+0000 7f522a800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52240d0100 con 0x7f522415e820 2024-04-25T10:07:40.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.600+0000 7f522a800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52240ced00 con 0x7f522415b7b0 2024-04-25T10:07:40.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.601+0000 7f5223400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522415b7b0 0x7f52240ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.601+0000 7f5223400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522415b7b0 0x7f52240ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:42434/0 (socket says 172.21.15.119:42434) 2024-04-25T10:07:40.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.601+0000 7f5223400640 1 -- 172.21.15.119:0/3670662880 learned_addr learned my addr 172.21.15.119:0/3670662880 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:40.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.601+0000 7f5223e00640 1 --2- 172.21.15.119:0/3670662880 >> v2:172.21.15.132:3300/0 conn(0x7f522415e820 0x7f52240cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.603+0000 7f5223e00640 1 -- 172.21.15.119:0/3670662880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522415b7b0 msgr2=0x7f52240ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:40.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.603+0000 7f5223e00640 1 --2- 172.21.15.119:0/3670662880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522415b7b0 0x7f52240ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.603+0000 7f5223e00640 1 -- 172.21.15.119:0/3670662880 >> v1:172.21.15.132:6789/0 conn(0x7f52240cf760 legacy=0x7f522415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.603+0000 7f5223e00640 1 -- 172.21.15.119:0/3670662880 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522415ed20 con 0x7f522415e820 2024-04-25T10:07:40.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.603+0000 7f5223400640 1 --2- 172.21.15.119:0/3670662880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522415b7b0 0x7f52240ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:40.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.606+0000 7f5223e00640 1 --2- 172.21.15.119:0/3670662880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522415e820 0x7f52240cfbc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f521c009d90 tx=0x7f521c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0bafd6d45c2a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.608+0000 7f5222a00640 1 -- 172.21.15.119:0/3670662880 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c007740 con 0x7f522415e820 2024-04-25T10:07:40.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.608+0000 7f5222a00640 1 -- 172.21.15.119:0/3670662880 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c0078e0 con 0x7f522415e820 2024-04-25T10:07:40.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.608+0000 7f5222a00640 1 -- 172.21.15.119:0/3670662880 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c007cb0 con 0x7f522415e820 2024-04-25T10:07:40.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.608+0000 7f522a800640 1 -- 172.21.15.119:0/3670662880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522415e820 msgr2=0x7f52240cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.608+0000 7f522a800640 1 --2- 172.21.15.119:0/3670662880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522415e820 0x7f52240cfbc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f521c009d90 tx=0x7f521c02a660 comp rx=0 tx=0).stop 2024-04-25T10:07:40.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.609+0000 7f522a800640 1 -- 172.21.15.119:0/3670662880 shutdown_connections 2024-04-25T10:07:40.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.609+0000 7f522a800640 1 --2- 172.21.15.119:0/3670662880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522415e820 0x7f52240cfbc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.609+0000 7f522a800640 1 --2- 172.21.15.119:0/3670662880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522415b7b0 0x7f52240ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.609+0000 7f522a800640 1 -- 172.21.15.119:0/3670662880 >> 172.21.15.119:0/3670662880 conn(0x7f522406f4a0 msgr2=0x7f522406f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:40.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.609+0000 7f522a800640 1 -- 172.21.15.119:0/3670662880 shutdown_connections 2024-04-25T10:07:40.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.609+0000 7f522a800640 1 -- 172.21.15.119:0/3670662880 wait complete. 2024-04-25T10:07:40.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.611+0000 7f522a800640 1 Processor -- start 2024-04-25T10:07:40.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.611+0000 7f522a800640 1 -- start start 2024-04-25T10:07:40.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.612+0000 7f522a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52240cf760 0x7f52241f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.612+0000 7f522a800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522415b7b0 0x7f5224158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.612+0000 7f522a800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522415e820 0x7f52240b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.612+0000 7f522a800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52240cdfc0 con 0x7f52240cf760 2024-04-25T10:07:40.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.612+0000 7f522a800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52240ce330 con 0x7f522415e820 2024-04-25T10:07:40.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.612+0000 7f522a800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52241619f0 con 0x7f522415b7b0 2024-04-25T10:07:40.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.612+0000 7f5223400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52240cf760 0x7f52241f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.612+0000 7f5223400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52240cf760 0x7f52241f3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55340/0 (socket says 172.21.15.119:55340) 2024-04-25T10:07:40.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.612+0000 7f5223400640 1 -- 172.21.15.119:0/3913813293 learned_addr learned my addr 172.21.15.119:0/3913813293 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:40.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.613+0000 7f5223400640 1 -- 172.21.15.119:0/3913813293 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522415b7b0 msgr2=0x7f5224158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.613+0000 7f5223400640 1 --2- 172.21.15.119:0/3913813293 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522415b7b0 0x7f5224158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.613+0000 7f5223e00640 1 --2- 172.21.15.119:0/3913813293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522415e820 0x7f52240b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.613+0000 7f5223400640 1 -- 172.21.15.119:0/3913813293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522415e820 msgr2=0x7f52240b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.613+0000 7f5223400640 1 --2- 172.21.15.119:0/3913813293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522415e820 0x7f52240b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.613+0000 7f5223400640 1 -- 172.21.15.119:0/3913813293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240b3b30 con 0x7f52240cf760 2024-04-25T10:07:40.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.613+0000 7f5223400640 1 --2- 172.21.15.119:0/3913813293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52240cf760 0x7f52241f3a80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f521400e9d0 tx=0x7f521400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.613+0000 7f521b400640 1 -- 172.21.15.119:0/3913813293 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214017070 con 0x7f52240cf760 2024-04-25T10:07:40.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.614+0000 7f521b400640 1 -- 172.21.15.119:0/3913813293 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521400ca60 con 0x7f52240cf760 2024-04-25T10:07:40.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.614+0000 7f521b400640 1 -- 172.21.15.119:0/3913813293 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521400ce10 con 0x7f52240cf760 2024-04-25T10:07:40.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.614+0000 7f522a800640 1 -- 172.21.15.119:0/3913813293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52240b3e10 con 0x7f52240cf760 2024-04-25T10:07:40.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.614+0000 7f522a800640 1 -- 172.21.15.119:0/3913813293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52240b44f0 con 0x7f52240cf760 2024-04-25T10:07:40.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.615+0000 7f522a800640 1 -- 172.21.15.119:0/3913813293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52240750b0 con 0x7f52240cf760 2024-04-25T10:07:40.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.616+0000 7f521b400640 1 -- 172.21.15.119:0/3913813293 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f521401a070 con 0x7f52240cf760 2024-04-25T10:07:40.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.617+0000 7f521b400640 1 --2- 172.21.15.119:0/3913813293 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f520006b4d0 0x7f520006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.617+0000 7f521b400640 1 -- 172.21.15.119:0/3913813293 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f521408a4c0 con 0x7f52240cf760 2024-04-25T10:07:40.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.617+0000 7f521be00640 1 --2- 172.21.15.119:0/3913813293 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f520006b4d0 0x7f520006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.618+0000 7f521be00640 1 --2- 172.21.15.119:0/3913813293 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f520006b4d0 0x7f520006d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f520c000f60 tx=0x7f520c0079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.621+0000 7f521b400640 1 -- 172.21.15.119:0/3913813293 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5214059340 con 0x7f52240cf760 2024-04-25T10:07:40.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.799+0000 7f522a800640 1 -- 172.21.15.119:0/3913813293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52240b7870 con 0x7f52240cf760 2024-04-25T10:07:40.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.800+0000 7f521b400640 1 -- 172.21.15.119:0/3913813293 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5214058c40 con 0x7f52240cf760 2024-04-25T10:07:40.806 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:07:40.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.802+0000 7f522a800640 1 -- 172.21.15.119:0/3913813293 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f520006b4d0 msgr2=0x7f520006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.802+0000 7f522a800640 1 --2- 172.21.15.119:0/3913813293 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f520006b4d0 0x7f520006d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f520c000f60 tx=0x7f520c0079d0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.802+0000 7f522a800640 1 -- 172.21.15.119:0/3913813293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52240cf760 msgr2=0x7f52241f3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.802+0000 7f522a800640 1 --2- 172.21.15.119:0/3913813293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52240cf760 0x7f52241f3a80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f521400e9d0 tx=0x7f521400ee90 comp rx=0 tx=0).stop 2024-04-25T10:07:40.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.803+0000 7f522a800640 1 -- 172.21.15.119:0/3913813293 shutdown_connections 2024-04-25T10:07:40.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.803+0000 7f522a800640 1 --2- 172.21.15.119:0/3913813293 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f520006b4d0 0x7f520006d990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.803+0000 7f522a800640 1 --2- 172.21.15.119:0/3913813293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522415e820 0x7f52240b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.803+0000 7f522a800640 1 --2- 172.21.15.119:0/3913813293 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522415b7b0 0x7f5224158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.803+0000 7f522a800640 1 --2- 172.21.15.119:0/3913813293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52240cf760 0x7f52241f3a80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.803+0000 7f522a800640 1 -- 172.21.15.119:0/3913813293 >> 172.21.15.119:0/3913813293 conn(0x7f522406f4a0 msgr2=0x7f522406c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:40.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.803+0000 7f522a800640 1 -- 172.21.15.119:0/3913813293 shutdown_connections 2024-04-25T10:07:40.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.803+0000 7f522a800640 1 -- 172.21.15.119:0/3913813293 wait complete. 2024-04-25T10:07:40.827 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:41.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.999+0000 7f8efce00640 1 Processor -- start 2024-04-25T10:07:41.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:40.999+0000 7f8efce00640 1 -- start start 2024-04-25T10:07:41.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.001+0000 7f8efce00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8ef815dc80 0x7f8ef80b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.001+0000 7f8efce00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ef80d3340 0x7f8ef80b7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.001+0000 7f8efce00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef80d1c80 con 0x7f8ef80ce560 2024-04-25T10:07:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.001+0000 7f8efce00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef80d0e30 con 0x7f8ef815dc80 2024-04-25T10:07:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.001+0000 7f8efce00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80d2b50 con 0x7f8ef80d3340 2024-04-25T10:07:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.001+0000 7f8ef7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ef80d3340 0x7f8ef80b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.001+0000 7f8ef7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ef80d3340 0x7f8ef80b7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:36662/0 (socket says 172.21.15.119:36662) 2024-04-25T10:07:41.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.001+0000 7f8ef7e00640 1 -- 172.21.15.119:0/2887913625 learned_addr learned my addr 172.21.15.119:0/2887913625 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:41.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.002+0000 7f8eeea00640 1 --2- 172.21.15.119:0/2887913625 >> v2:172.21.15.132:3300/0 conn(0x7f8ef815dc80 0x7f8ef80b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:41.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.002+0000 7f8ef7e00640 1 -- 172.21.15.119:0/2887913625 >> v2:172.21.15.132:3300/0 conn(0x7f8ef815dc80 msgr2=0x7f8ef80b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:41.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.002+0000 7f8ef7e00640 1 --2- 172.21.15.119:0/2887913625 >> v2:172.21.15.132:3300/0 conn(0x7f8ef815dc80 0x7f8ef80b6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.002+0000 7f8ef7e00640 1 -- 172.21.15.119:0/2887913625 >> v1:172.21.15.132:6789/0 conn(0x7f8ef80ce560 legacy=0x7f8ef80ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.002+0000 7f8ef7e00640 1 -- 172.21.15.119:0/2887913625 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef80d1f40 con 0x7f8ef80d3340 2024-04-25T10:07:41.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.004+0000 7f8ef7e00640 1 --2- 172.21.15.119:0/2887913625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ef80d3340 0x7f8ef80b7000 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009870 tx=0x7f8ef002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ab28d1135cd2902 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:41.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.004+0000 7f8ef6a00640 1 -- 172.21.15.119:0/2887913625 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0037070 con 0x7f8ef80d3340 2024-04-25T10:07:41.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.004+0000 7f8ef6a00640 1 -- 172.21.15.119:0/2887913625 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef0032440 con 0x7f8ef80d3340 2024-04-25T10:07:41.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.004+0000 7f8ef6a00640 1 -- 172.21.15.119:0/2887913625 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0032810 con 0x7f8ef80d3340 2024-04-25T10:07:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.005+0000 7f8efce00640 1 -- 172.21.15.119:0/2887913625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ef80d3340 msgr2=0x7f8ef80b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.005+0000 7f8efce00640 1 --2- 172.21.15.119:0/2887913625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ef80d3340 0x7f8ef80b7000 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009870 tx=0x7f8ef002a270 comp rx=0 tx=0).stop 2024-04-25T10:07:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.005+0000 7f8efce00640 1 -- 172.21.15.119:0/2887913625 shutdown_connections 2024-04-25T10:07:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.005+0000 7f8efce00640 1 --2- 172.21.15.119:0/2887913625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ef80d3340 0x7f8ef80b7000 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.005+0000 7f8efce00640 1 --2- 172.21.15.119:0/2887913625 >> v2:172.21.15.132:3300/0 conn(0x7f8ef815dc80 0x7f8ef80b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.005+0000 7f8efce00640 1 -- 172.21.15.119:0/2887913625 >> 172.21.15.119:0/2887913625 conn(0x7f8ef806bde0 msgr2=0x7f8ef806c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.005+0000 7f8efce00640 1 -- 172.21.15.119:0/2887913625 shutdown_connections 2024-04-25T10:07:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.005+0000 7f8efce00640 1 -- 172.21.15.119:0/2887913625 wait complete. 2024-04-25T10:07:41.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.007+0000 7f8efce00640 1 Processor -- start 2024-04-25T10:07:41.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.007+0000 7f8efce00640 1 -- start start 2024-04-25T10:07:41.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.008+0000 7f8efce00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ef80ce560 0x7f8ef8159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.008+0000 7f8efce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef80d3340 0x7f8ef80b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.008+0000 7f8efce00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ef815dc80 0x7f8ef80cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.008+0000 7f8efce00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80d27e0 con 0x7f8ef80d3340 2024-04-25T10:07:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.008+0000 7f8efce00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80d2500 con 0x7f8ef815dc80 2024-04-25T10:07:41.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.008+0000 7f8efce00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80d2fe0 con 0x7f8ef80ce560 2024-04-25T10:07:41.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.008+0000 7f8eeea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef80d3340 0x7f8ef80b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:41.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.008+0000 7f8eeea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef80d3340 0x7f8ef80b3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58848/0 (socket says 172.21.15.119:58848) 2024-04-25T10:07:41.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.008+0000 7f8eeea00640 1 -- 172.21.15.119:0/1241977245 learned_addr learned my addr 172.21.15.119:0/1241977245 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:41.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.009+0000 7f8ef7e00640 1 --2- 172.21.15.119:0/1241977245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ef815dc80 0x7f8ef80cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:41.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.009+0000 7f8ef7400640 1 --2- 172.21.15.119:0/1241977245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ef80ce560 0x7f8ef8159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:41.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.009+0000 7f8eeea00640 1 -- 172.21.15.119:0/1241977245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ef80ce560 msgr2=0x7f8ef8159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.009+0000 7f8eeea00640 1 --2- 172.21.15.119:0/1241977245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ef80ce560 0x7f8ef8159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.009+0000 7f8eeea00640 1 -- 172.21.15.119:0/1241977245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ef815dc80 msgr2=0x7f8ef80cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.009+0000 7f8eeea00640 1 --2- 172.21.15.119:0/1241977245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ef815dc80 0x7f8ef80cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.009+0000 7f8eeea00640 1 -- 172.21.15.119:0/1241977245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef80b4700 con 0x7f8ef80d3340 2024-04-25T10:07:41.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.009+0000 7f8eeea00640 1 --2- 172.21.15.119:0/1241977245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef80d3340 0x7f8ef80b3d60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000e870 tx=0x7f8ee000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:41.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.009+0000 7f8eefe00640 1 -- 172.21.15.119:0/1241977245 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0017070 con 0x7f8ef80d3340 2024-04-25T10:07:41.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.009+0000 7f8eefe00640 1 -- 172.21.15.119:0/1241977245 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee000c940 con 0x7f8ef80d3340 2024-04-25T10:07:41.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.009+0000 7f8eefe00640 1 -- 172.21.15.119:0/1241977245 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee000cd30 con 0x7f8ef80d3340 2024-04-25T10:07:41.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.009+0000 7f8efce00640 1 -- 172.21.15.119:0/1241977245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef80b49e0 con 0x7f8ef80d3340 2024-04-25T10:07:41.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.010+0000 7f8efce00640 1 -- 172.21.15.119:0/1241977245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef80b4fa0 con 0x7f8ef80d3340 2024-04-25T10:07:41.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.011+0000 7f8efce00640 1 -- 172.21.15.119:0/1241977245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef80b52e0 con 0x7f8ef80d3340 2024-04-25T10:07:41.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.012+0000 7f8eefe00640 1 -- 172.21.15.119:0/1241977245 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ee001a070 con 0x7f8ef80d3340 2024-04-25T10:07:41.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.013+0000 7f8eefe00640 1 --2- 172.21.15.119:0/1241977245 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8ed406b4d0 0x7f8ed406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.013+0000 7f8eefe00640 1 -- 172.21.15.119:0/1241977245 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ee008a5b0 con 0x7f8ef80d3340 2024-04-25T10:07:41.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.013+0000 7f8ef7400640 1 --2- 172.21.15.119:0/1241977245 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8ed406b4d0 0x7f8ed406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:41.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.014+0000 7f8ef7400640 1 --2- 172.21.15.119:0/1241977245 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8ed406b4d0 0x7f8ed406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8001d40 tx=0x7f8ee8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:41.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.018+0000 7f8eefe00640 1 -- 172.21.15.119:0/1241977245 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ee0059300 con 0x7f8ef80d3340 2024-04-25T10:07:41.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.215+0000 7f8efce00640 1 -- 172.21.15.119:0/1241977245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ef80749b0 con 0x7f8ef80d3340 2024-04-25T10:07:41.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.217+0000 7f8eefe00640 1 -- 172.21.15.119:0/1241977245 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f8ee0058c00 con 0x7f8ef80d3340 2024-04-25T10:07:41.223 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:41.223 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:08.319104+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:41.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.219+0000 7f8efce00640 1 -- 172.21.15.119:0/1241977245 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8ed406b4d0 msgr2=0x7f8ed406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.220+0000 7f8efce00640 1 --2- 172.21.15.119:0/1241977245 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8ed406b4d0 0x7f8ed406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8001d40 tx=0x7f8ee8009040 comp rx=0 tx=0).stop 2024-04-25T10:07:41.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.220+0000 7f8efce00640 1 -- 172.21.15.119:0/1241977245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef80d3340 msgr2=0x7f8ef80b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.220+0000 7f8efce00640 1 --2- 172.21.15.119:0/1241977245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef80d3340 0x7f8ef80b3d60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000e870 tx=0x7f8ee000ed30 comp rx=0 tx=0).stop 2024-04-25T10:07:41.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.220+0000 7f8efce00640 1 -- 172.21.15.119:0/1241977245 shutdown_connections 2024-04-25T10:07:41.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.220+0000 7f8efce00640 1 --2- 172.21.15.119:0/1241977245 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8ed406b4d0 0x7f8ed406d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.220+0000 7f8efce00640 1 --2- 172.21.15.119:0/1241977245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ef815dc80 0x7f8ef80cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.220+0000 7f8efce00640 1 --2- 172.21.15.119:0/1241977245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef80d3340 0x7f8ef80b3d60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.220+0000 7f8efce00640 1 --2- 172.21.15.119:0/1241977245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ef80ce560 0x7f8ef8159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.220+0000 7f8efce00640 1 -- 172.21.15.119:0/1241977245 >> 172.21.15.119:0/1241977245 conn(0x7f8ef806bde0 msgr2=0x7f8ef8160290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:41.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.220+0000 7f8efce00640 1 -- 172.21.15.119:0/1241977245 shutdown_connections 2024-04-25T10:07:41.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.220+0000 7f8efce00640 1 -- 172.21.15.119:0/1241977245 wait complete. 2024-04-25T10:07:41.244 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:05.135832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:05.768083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:41.244 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:41.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.434+0000 7faf95a00640 1 Processor -- start 2024-04-25T10:07:41.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.434+0000 7faf95a00640 1 -- start start 2024-04-25T10:07:41.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.436+0000 7faf95a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faf9006b2b0 0x7faf900cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.436+0000 7faf95a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faf900cc810 0x7faf900b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.436+0000 7faf95a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9015f400 con 0x7faf901450e0 2024-04-25T10:07:41.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.436+0000 7faf95a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf901454c0 con 0x7faf9006b2b0 2024-04-25T10:07:41.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.436+0000 7faf95a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9015f6e0 con 0x7faf900cc810 2024-04-25T10:07:41.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.436+0000 7faf8fe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7faf901450e0 0x7faf9006aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36638/0 (socket says 172.21.15.119:36638) 2024-04-25T10:07:41.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.436+0000 7faf8fe00640 1 -- 172.21.15.119:0/1384946670 learned_addr learned my addr 172.21.15.119:0/1384946670 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:41.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.436+0000 7faf8f400640 1 --2- 172.21.15.119:0/1384946670 >> v2:172.21.15.119:3300/0 conn(0x7faf9006b2b0 0x7faf900cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:41.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.436+0000 7faf8ea00640 1 -- 172.21.15.119:0/1384946670 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830822401 0 0) 0x7faf9015f400 con 0x7faf901450e0 2024-04-25T10:07:41.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.436+0000 7faf95000640 1 --2- 172.21.15.119:0/1384946670 >> v2:172.21.15.132:3300/0 conn(0x7faf900cc810 0x7faf900b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:41.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.436+0000 7faf8ea00640 1 -- 172.21.15.119:0/1384946670 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf74003520 con 0x7faf901450e0 2024-04-25T10:07:41.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.436+0000 7faf8f400640 1 -- 172.21.15.119:0/1384946670 >> v2:172.21.15.132:3300/0 conn(0x7faf900cc810 msgr2=0x7faf900b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:41.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.437+0000 7faf8f400640 1 --2- 172.21.15.119:0/1384946670 >> v2:172.21.15.132:3300/0 conn(0x7faf900cc810 0x7faf900b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.437+0000 7faf8f400640 1 -- 172.21.15.119:0/1384946670 >> v1:172.21.15.119:6789/0 conn(0x7faf901450e0 legacy=0x7faf9006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.437+0000 7faf8f400640 1 -- 172.21.15.119:0/1384946670 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9015f9c0 con 0x7faf9006b2b0 2024-04-25T10:07:41.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.437+0000 7faf95000640 1 --2- 172.21.15.119:0/1384946670 >> v2:172.21.15.132:3300/0 conn(0x7faf900cc810 0x7faf900b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:41.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.438+0000 7faf8f400640 1 --2- 172.21.15.119:0/1384946670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf9006b2b0 0x7faf900cc2d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faf900cbe90 tx=0x7faf7c02a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f366e54c03279c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:41.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.438+0000 7faf8ea00640 1 -- 172.21.15.119:0/1384946670 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c007740 con 0x7faf9006b2b0 2024-04-25T10:07:41.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.438+0000 7faf8ea00640 1 -- 172.21.15.119:0/1384946670 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7c0078e0 con 0x7faf9006b2b0 2024-04-25T10:07:41.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.438+0000 7faf8ea00640 1 -- 172.21.15.119:0/1384946670 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c007cb0 con 0x7faf9006b2b0 2024-04-25T10:07:41.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.438+0000 7faf95a00640 1 -- 172.21.15.119:0/1384946670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf9006b2b0 msgr2=0x7faf900cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.438+0000 7faf95a00640 1 --2- 172.21.15.119:0/1384946670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf9006b2b0 0x7faf900cc2d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faf900cbe90 tx=0x7faf7c02a550 comp rx=0 tx=0).stop 2024-04-25T10:07:41.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.439+0000 7faf95a00640 1 -- 172.21.15.119:0/1384946670 shutdown_connections 2024-04-25T10:07:41.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.439+0000 7faf95a00640 1 --2- 172.21.15.119:0/1384946670 >> v2:172.21.15.132:3300/0 conn(0x7faf900cc810 0x7faf900b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.439+0000 7faf95a00640 1 --2- 172.21.15.119:0/1384946670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf9006b2b0 0x7faf900cc2d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.439+0000 7faf95a00640 1 -- 172.21.15.119:0/1384946670 >> 172.21.15.119:0/1384946670 conn(0x7faf90066ff0 msgr2=0x7faf900b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:41.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.439+0000 7faf95a00640 1 -- 172.21.15.119:0/1384946670 shutdown_connections 2024-04-25T10:07:41.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.439+0000 7faf95a00640 1 -- 172.21.15.119:0/1384946670 wait complete. 2024-04-25T10:07:41.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.440+0000 7faf95a00640 1 Processor -- start 2024-04-25T10:07:41.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.441+0000 7faf95a00640 1 -- start start 2024-04-25T10:07:41.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.441+0000 7faf95a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf9006b2b0 0x7faf901345e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.442+0000 7faf95a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900cc810 0x7faf900c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.442+0000 7faf95a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf901450e0 0x7faf900c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.442+0000 7faf95a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf900ccde0 con 0x7faf900cc810 2024-04-25T10:07:41.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.442+0000 7faf95a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf900ccc60 con 0x7faf9006b2b0 2024-04-25T10:07:41.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.442+0000 7faf95a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf900b33c0 con 0x7faf901450e0 2024-04-25T10:07:41.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.442+0000 7faf8f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900cc810 0x7faf900c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:41.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.442+0000 7faf8f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900cc810 0x7faf900c43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58866/0 (socket says 172.21.15.119:58866) 2024-04-25T10:07:41.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.442+0000 7faf8f400640 1 -- 172.21.15.119:0/16837270 learned_addr learned my addr 172.21.15.119:0/16837270 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:41.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.442+0000 7faf8f400640 1 -- 172.21.15.119:0/16837270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf901450e0 msgr2=0x7faf900c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.442+0000 7faf8f400640 1 --2- 172.21.15.119:0/16837270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf901450e0 0x7faf900c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.442+0000 7faf8f400640 1 -- 172.21.15.119:0/16837270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf9006b2b0 msgr2=0x7faf901345e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.442+0000 7faf8f400640 1 --2- 172.21.15.119:0/16837270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf9006b2b0 0x7faf901345e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.442+0000 7faf8f400640 1 -- 172.21.15.119:0/16837270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf900aaad0 con 0x7faf900cc810 2024-04-25T10:07:41.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.443+0000 7faf8f400640 1 --2- 172.21.15.119:0/16837270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900cc810 0x7faf900c43e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009e80 tx=0x7faf7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:41.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.443+0000 7faf87e00640 1 -- 172.21.15.119:0/16837270 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c040070 con 0x7faf900cc810 2024-04-25T10:07:41.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.443+0000 7faf87e00640 1 -- 172.21.15.119:0/16837270 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7c031c40 con 0x7faf900cc810 2024-04-25T10:07:41.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.443+0000 7faf87e00640 1 -- 172.21.15.119:0/16837270 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c009450 con 0x7faf900cc810 2024-04-25T10:07:41.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.443+0000 7faf95a00640 1 -- 172.21.15.119:0/16837270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf900aadb0 con 0x7faf900cc810 2024-04-25T10:07:41.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.443+0000 7faf95a00640 1 -- 172.21.15.119:0/16837270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf900ab370 con 0x7faf900cc810 2024-04-25T10:07:41.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.444+0000 7faf95a00640 1 -- 172.21.15.119:0/16837270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf900749b0 con 0x7faf900cc810 2024-04-25T10:07:41.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.445+0000 7faf87e00640 1 -- 172.21.15.119:0/16837270 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf7c039040 con 0x7faf900cc810 2024-04-25T10:07:41.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.446+0000 7faf87e00640 1 --2- 172.21.15.119:0/16837270 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf7006b4d0 0x7faf7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.446+0000 7faf87e00640 1 -- 172.21.15.119:0/16837270 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf7c0ab740 con 0x7faf900cc810 2024-04-25T10:07:41.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.446+0000 7faf8fe00640 1 --2- 172.21.15.119:0/16837270 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf7006b4d0 0x7faf7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:41.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.447+0000 7faf8fe00640 1 --2- 172.21.15.119:0/16837270 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf7006b4d0 0x7faf7006d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faf80000f60 tx=0x7faf800078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:41.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.451+0000 7faf87e00640 1 -- 172.21.15.119:0/16837270 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faf7c07a5b0 con 0x7faf900cc810 2024-04-25T10:07:41.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.644+0000 7faf95a00640 1 -- 172.21.15.119:0/16837270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf900cda90 con 0x7faf900cc810 2024-04-25T10:07:41.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.645+0000 7faf87e00640 1 -- 172.21.15.119:0/16837270 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf7c079eb0 con 0x7faf900cc810 2024-04-25T10:07:41.651 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:07:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.647+0000 7faf95a00640 1 -- 172.21.15.119:0/16837270 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf7006b4d0 msgr2=0x7faf7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.647+0000 7faf95a00640 1 --2- 172.21.15.119:0/16837270 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf7006b4d0 0x7faf7006d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faf80000f60 tx=0x7faf800078f0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.647+0000 7faf95a00640 1 -- 172.21.15.119:0/16837270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900cc810 msgr2=0x7faf900c43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.647+0000 7faf95a00640 1 --2- 172.21.15.119:0/16837270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900cc810 0x7faf900c43e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009e80 tx=0x7faf7c004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.648+0000 7faf95a00640 1 -- 172.21.15.119:0/16837270 shutdown_connections 2024-04-25T10:07:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.648+0000 7faf95a00640 1 --2- 172.21.15.119:0/16837270 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf7006b4d0 0x7faf7006d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.648+0000 7faf95a00640 1 --2- 172.21.15.119:0/16837270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf901450e0 0x7faf900c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.648+0000 7faf95a00640 1 --2- 172.21.15.119:0/16837270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf900cc810 0x7faf900c43e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.648+0000 7faf95a00640 1 --2- 172.21.15.119:0/16837270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf9006b2b0 0x7faf901345e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.648+0000 7faf95a00640 1 -- 172.21.15.119:0/16837270 >> 172.21.15.119:0/16837270 conn(0x7faf90066ff0 msgr2=0x7faf9015e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.648+0000 7faf95a00640 1 -- 172.21.15.119:0/16837270 shutdown_connections 2024-04-25T10:07:41.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.648+0000 7faf95a00640 1 -- 172.21.15.119:0/16837270 wait complete. 2024-04-25T10:07:41.672 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:41.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.864+0000 7fc380c00640 1 Processor -- start 2024-04-25T10:07:41.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.864+0000 7fc380c00640 1 -- start start 2024-04-25T10:07:41.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.866+0000 7fc380c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc37c15b7e0 0x7fc37c15e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.866+0000 7fc380c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc37c0d07b0 0x7fc37c0cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.866+0000 7fc380c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37c0b5710 con 0x7fc37c0ce860 2024-04-25T10:07:41.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.866+0000 7fc380c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37c0b59f0 con 0x7fc37c15b7e0 2024-04-25T10:07:41.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.866+0000 7fc380c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c0b5cd0 con 0x7fc37c0d07b0 2024-04-25T10:07:41.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.866+0000 7fc37b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc37c15b7e0 0x7fc37c15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:41.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.866+0000 7fc37b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc37c15b7e0 0x7fc37c15e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58874/0 (socket says 172.21.15.119:58874) 2024-04-25T10:07:41.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.866+0000 7fc37b400640 1 -- 172.21.15.119:0/149755301 learned_addr learned my addr 172.21.15.119:0/149755301 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:41.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.866+0000 7fc37aa00640 1 --1- 172.21.15.119:0/149755301 >> v1:172.21.15.119:6789/0 conn(0x7fc37c0ce860 0x7fc37c0d00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36646/0 (socket says 172.21.15.119:36646) 2024-04-25T10:07:41.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.866+0000 7fc37b400640 1 -- 172.21.15.119:0/149755301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc37c0d07b0 msgr2=0x7fc37c0cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:41.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.866+0000 7fc37b400640 1 --2- 172.21.15.119:0/149755301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc37c0d07b0 0x7fc37c0cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.866+0000 7fc37b400640 1 -- 172.21.15.119:0/149755301 >> v1:172.21.15.119:6789/0 conn(0x7fc37c0ce860 legacy=0x7fc37c0d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.866+0000 7fc37b400640 1 -- 172.21.15.119:0/149755301 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c0b5fb0 con 0x7fc37c15b7e0 2024-04-25T10:07:41.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.867+0000 7fc37b400640 1 --2- 172.21.15.119:0/149755301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc37c15b7e0 0x7fc37c15e000 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009870 tx=0x7fc36c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4046ec155f504599 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:41.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.868+0000 7fc37a000640 1 -- 172.21.15.119:0/149755301 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c037070 con 0x7fc37c15b7e0 2024-04-25T10:07:41.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.868+0000 7fc37a000640 1 -- 172.21.15.119:0/149755301 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c032440 con 0x7fc37c15b7e0 2024-04-25T10:07:41.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.868+0000 7fc37a000640 1 -- 172.21.15.119:0/149755301 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c032810 con 0x7fc37c15b7e0 2024-04-25T10:07:41.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.868+0000 7fc380c00640 1 -- 172.21.15.119:0/149755301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc37c15b7e0 msgr2=0x7fc37c15e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.868+0000 7fc380c00640 1 --2- 172.21.15.119:0/149755301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc37c15b7e0 0x7fc37c15e000 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009870 tx=0x7fc36c02a130 comp rx=0 tx=0).stop 2024-04-25T10:07:41.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.868+0000 7fc380c00640 1 -- 172.21.15.119:0/149755301 shutdown_connections 2024-04-25T10:07:41.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.868+0000 7fc380c00640 1 --2- 172.21.15.119:0/149755301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc37c0d07b0 0x7fc37c0cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.868+0000 7fc380c00640 1 --2- 172.21.15.119:0/149755301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc37c15b7e0 0x7fc37c15e000 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.868+0000 7fc380c00640 1 -- 172.21.15.119:0/149755301 >> 172.21.15.119:0/149755301 conn(0x7fc37c0722d0 msgr2=0x7fc37c0726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:41.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.868+0000 7fc380c00640 1 -- 172.21.15.119:0/149755301 shutdown_connections 2024-04-25T10:07:41.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.868+0000 7fc380c00640 1 -- 172.21.15.119:0/149755301 wait complete. 2024-04-25T10:07:41.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.870+0000 7fc380c00640 1 Processor -- start 2024-04-25T10:07:41.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.870+0000 7fc380c00640 1 -- start start 2024-04-25T10:07:41.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.871+0000 7fc380c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc37c0ce860 0x7fc37c0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.871+0000 7fc380c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc37c0d07b0 0x7fc37c1f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.871+0000 7fc380c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc37c15b7e0 0x7fc37c0cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.871+0000 7fc380c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c161960 con 0x7fc37c0ce860 2024-04-25T10:07:41.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.871+0000 7fc380c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c0b53c0 con 0x7fc37c0d07b0 2024-04-25T10:07:41.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.871+0000 7fc380c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c161ae0 con 0x7fc37c15b7e0 2024-04-25T10:07:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.871+0000 7fc37b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc37c0ce860 0x7fc37c0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.871+0000 7fc37b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc37c0ce860 0x7fc37c0cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58888/0 (socket says 172.21.15.119:58888) 2024-04-25T10:07:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.871+0000 7fc37b400640 1 -- 172.21.15.119:0/162400828 learned_addr learned my addr 172.21.15.119:0/162400828 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.872+0000 7fc37b400640 1 -- 172.21.15.119:0/162400828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc37c15b7e0 msgr2=0x7fc37c0cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.872+0000 7fc37b400640 1 --2- 172.21.15.119:0/162400828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc37c15b7e0 0x7fc37c0cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.872+0000 7fc37b400640 1 -- 172.21.15.119:0/162400828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc37c0d07b0 msgr2=0x7fc37c1f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.872+0000 7fc37b400640 1 --2- 172.21.15.119:0/162400828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc37c0d07b0 0x7fc37c1f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.872+0000 7fc37b400640 1 -- 172.21.15.119:0/162400828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c1f46d0 con 0x7fc37c0ce860 2024-04-25T10:07:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.872+0000 7fc37b400640 1 --2- 172.21.15.119:0/162400828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc37c0ce860 0x7fc37c0cc6c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009870 tx=0x7fc36c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.872+0000 7fc373400640 1 -- 172.21.15.119:0/162400828 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c041070 con 0x7fc37c0ce860 2024-04-25T10:07:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.872+0000 7fc373400640 1 -- 172.21.15.119:0/162400828 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c03c870 con 0x7fc37c0ce860 2024-04-25T10:07:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.872+0000 7fc373400640 1 -- 172.21.15.119:0/162400828 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c037040 con 0x7fc37c0ce860 2024-04-25T10:07:41.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.872+0000 7fc380c00640 1 -- 172.21.15.119:0/162400828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37c1f49b0 con 0x7fc37c0ce860 2024-04-25T10:07:41.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.872+0000 7fc380c00640 1 -- 172.21.15.119:0/162400828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37c0cae20 con 0x7fc37c0ce860 2024-04-25T10:07:41.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.873+0000 7fc380c00640 1 -- 172.21.15.119:0/162400828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37c1f4b80 con 0x7fc37c0ce860 2024-04-25T10:07:41.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.874+0000 7fc373400640 1 -- 172.21.15.119:0/162400828 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc36c044070 con 0x7fc37c0ce860 2024-04-25T10:07:41.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.875+0000 7fc373400640 1 --2- 172.21.15.119:0/162400828 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc35806b4d0 0x7fc35806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:41.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.875+0000 7fc373400640 1 -- 172.21.15.119:0/162400828 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc36c0aafe0 con 0x7fc37c0ce860 2024-04-25T10:07:41.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.875+0000 7fc37aa00640 1 --2- 172.21.15.119:0/162400828 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc35806b4d0 0x7fc35806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:41.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.876+0000 7fc37aa00640 1 --2- 172.21.15.119:0/162400828 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc35806b4d0 0x7fc35806d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc364000f60 tx=0x7fc364007a00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:41.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:41.880+0000 7fc373400640 1 -- 172.21.15.119:0/162400828 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc36c079fb0 con 0x7fc37c0ce860 2024-04-25T10:07:42.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.059+0000 7fc380c00640 1 -- 172.21.15.119:0/162400828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc37c0749b0 con 0x7fc37c0ce860 2024-04-25T10:07:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.060+0000 7fc373400640 1 -- 172.21.15.119:0/162400828 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc36c0798b0 con 0x7fc37c0ce860 2024-04-25T10:07:42.066 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:07:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.062+0000 7fc380c00640 1 -- 172.21.15.119:0/162400828 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc35806b4d0 msgr2=0x7fc35806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.062+0000 7fc380c00640 1 --2- 172.21.15.119:0/162400828 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc35806b4d0 0x7fc35806d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc364000f60 tx=0x7fc364007a00 comp rx=0 tx=0).stop 2024-04-25T10:07:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.062+0000 7fc380c00640 1 -- 172.21.15.119:0/162400828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc37c0ce860 msgr2=0x7fc37c0cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.062+0000 7fc380c00640 1 --2- 172.21.15.119:0/162400828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc37c0ce860 0x7fc37c0cc6c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009870 tx=0x7fc36c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.063+0000 7fc380c00640 1 -- 172.21.15.119:0/162400828 shutdown_connections 2024-04-25T10:07:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.063+0000 7fc380c00640 1 --2- 172.21.15.119:0/162400828 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc35806b4d0 0x7fc35806d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.063+0000 7fc380c00640 1 --2- 172.21.15.119:0/162400828 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc37c15b7e0 0x7fc37c0cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.063+0000 7fc380c00640 1 --2- 172.21.15.119:0/162400828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc37c0d07b0 0x7fc37c1f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.063+0000 7fc380c00640 1 --2- 172.21.15.119:0/162400828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc37c0ce860 0x7fc37c0cc6c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.063+0000 7fc380c00640 1 -- 172.21.15.119:0/162400828 >> 172.21.15.119:0/162400828 conn(0x7fc37c0722d0 msgr2=0x7fc37c1507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.063+0000 7fc380c00640 1 -- 172.21.15.119:0/162400828 shutdown_connections 2024-04-25T10:07:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.063+0000 7fc380c00640 1 -- 172.21.15.119:0/162400828 wait complete. 2024-04-25T10:07:42.087 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:42.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.271+0000 7f021ac00640 1 Processor -- start 2024-04-25T10:07:42.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.271+0000 7f021ac00640 1 -- start start 2024-04-25T10:07:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.273+0000 7f021ac00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f021406b2b0 0x7f02140cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.273+0000 7f021ac00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021415f400 con 0x7f02141450e0 2024-04-25T10:07:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.273+0000 7f021ac00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021415f6e0 con 0x7f02140cc810 2024-04-25T10:07:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.273+0000 7f021ac00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02141454c0 con 0x7f021406b2b0 2024-04-25T10:07:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.273+0000 7f0219800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f02141450e0 0x7f021406aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36654/0 (socket says 172.21.15.119:36654) 2024-04-25T10:07:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.273+0000 7f0219800640 1 -- 172.21.15.119:0/2152243513 learned_addr learned my addr 172.21.15.119:0/2152243513 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.273+0000 7f0218e00640 1 --2- 172.21.15.119:0/2152243513 >> v2:172.21.15.132:3300/0 conn(0x7f021406b2b0 0x7f02140cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.273+0000 7f0213e00640 1 -- 172.21.15.119:0/2152243513 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252109561 0 0) 0x7f021415f400 con 0x7f02141450e0 2024-04-25T10:07:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.273+0000 7f0213e00640 1 -- 172.21.15.119:0/2152243513 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01fc003520 con 0x7f02141450e0 2024-04-25T10:07:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.274+0000 7f0213e00640 1 -- 172.21.15.119:0/2152243513 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3328073177 0 0) 0x7f01fc003520 con 0x7f02141450e0 2024-04-25T10:07:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.274+0000 7f0213e00640 1 -- 172.21.15.119:0/2152243513 >> v2:172.21.15.132:3300/0 conn(0x7f021406b2b0 msgr2=0x7f02140cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.274+0000 7f0213e00640 1 --2- 172.21.15.119:0/2152243513 >> v2:172.21.15.132:3300/0 conn(0x7f021406b2b0 0x7f02140cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.274+0000 7f0213e00640 1 -- 172.21.15.119:0/2152243513 >> v1:172.21.15.132:6789/0 conn(0x7f02140cc810 legacy=0x7f02140b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.274+0000 7f0213e00640 1 -- 172.21.15.119:0/2152243513 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021415f9c0 con 0x7f02141450e0 2024-04-25T10:07:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.274+0000 7f0213e00640 1 -- 172.21.15.119:0/2152243513 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f0208001670 con 0x7f02141450e0 2024-04-25T10:07:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.274+0000 7f0213e00640 1 -- 172.21.15.119:0/2152243513 >> v1:172.21.15.119:6789/0 conn(0x7f02141450e0 legacy=0x7f021406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.274+0000 7f0213e00640 1 --2- 172.21.15.119:0/2152243513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01fc003a00 0x7f01fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.274+0000 7f0213e00640 1 --2- 172.21.15.119:0/2152243513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01fc004800 0x7f01fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.274+0000 7f0213e00640 1 --2- 172.21.15.119:0/2152243513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01fc0053c0 0x7f01fc00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.274+0000 7f0213e00640 1 -- 172.21.15.119:0/2152243513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc010680 con 0x7f01fc0053c0 2024-04-25T10:07:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.274+0000 7f0213e00640 1 -- 172.21.15.119:0/2152243513 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc010830 con 0x7f01fc003a00 2024-04-25T10:07:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.274+0000 7f0213e00640 1 -- 172.21.15.119:0/2152243513 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc010a30 con 0x7f01fc004800 2024-04-25T10:07:42.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.274+0000 7f021a200640 1 --2- 172.21.15.119:0/2152243513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01fc0053c0 0x7f01fc00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.275+0000 7f0218e00640 1 --2- 172.21.15.119:0/2152243513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01fc004800 0x7f01fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.275+0000 7f021a200640 1 -- 172.21.15.119:0/2152243513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01fc004800 msgr2=0x7f01fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:42.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.275+0000 7f021a200640 1 --2- 172.21.15.119:0/2152243513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f01fc004800 0x7f01fc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.275+0000 7f021a200640 1 -- 172.21.15.119:0/2152243513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01fc003a00 msgr2=0x7f01fc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:42.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.275+0000 7f021a200640 1 --2- 172.21.15.119:0/2152243513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01fc003a00 0x7f01fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.275+0000 7f021a200640 1 -- 172.21.15.119:0/2152243513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc003520 con 0x7f01fc0053c0 2024-04-25T10:07:42.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.275+0000 7f021a200640 1 -- 172.21.15.119:0/2152243513 reap_dead start 2024-04-25T10:07:42.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.276+0000 7f021a200640 1 --2- 172.21.15.119:0/2152243513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01fc0053c0 0x7f01fc00fdd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f020c001e50 tx=0x7f01fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c123a94b140f595 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:42.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.276+0000 7f0213e00640 1 -- 172.21.15.119:0/2152243513 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c001a70 con 0x7f01fc0053c0 2024-04-25T10:07:42.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.276+0000 7f0213e00640 1 -- 172.21.15.119:0/2152243513 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020c003970 con 0x7f01fc0053c0 2024-04-25T10:07:42.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.276+0000 7f0213e00640 1 -- 172.21.15.119:0/2152243513 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c003d60 con 0x7f01fc0053c0 2024-04-25T10:07:42.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.276+0000 7f021ac00640 1 -- 172.21.15.119:0/2152243513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01fc0053c0 msgr2=0x7f01fc00fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.276+0000 7f021ac00640 1 --2- 172.21.15.119:0/2152243513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01fc0053c0 0x7f01fc00fdd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f020c001e50 tx=0x7f01fc005830 comp rx=0 tx=0).stop 2024-04-25T10:07:42.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.276+0000 7f021ac00640 1 -- 172.21.15.119:0/2152243513 shutdown_connections 2024-04-25T10:07:42.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.276+0000 7f021ac00640 1 --2- 172.21.15.119:0/2152243513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01fc0053c0 0x7f01fc00fdd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.276+0000 7f021ac00640 1 -- 172.21.15.119:0/2152243513 >> 172.21.15.119:0/2152243513 conn(0x7f0214066ff0 msgr2=0x7f02140b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:42.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.276+0000 7f021ac00640 1 -- 172.21.15.119:0/2152243513 shutdown_connections 2024-04-25T10:07:42.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.277+0000 7f021ac00640 1 -- 172.21.15.119:0/2152243513 wait complete. 2024-04-25T10:07:42.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.278+0000 7f021ac00640 1 Processor -- start 2024-04-25T10:07:42.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.278+0000 7f021ac00640 1 -- start start 2024-04-25T10:07:42.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.279+0000 7f021ac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01fc0053c0 0x7f021415b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.279+0000 7f021ac00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f021413e1a0 0x7f021413e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.279+0000 7f021ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02140af610 0x7f02140afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.279+0000 7f021ac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02140ccde0 con 0x7f01fc0053c0 2024-04-25T10:07:42.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.279+0000 7f021ac00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02140ccc60 con 0x7f021413e1a0 2024-04-25T10:07:42.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.279+0000 7f021ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02140cc2d0 con 0x7f02140af610 2024-04-25T10:07:42.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f0219800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01fc0053c0 0x7f021415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f0219800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01fc0053c0 0x7f021415b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58912/0 (socket says 172.21.15.119:58912) 2024-04-25T10:07:42.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f0219800640 1 -- 172.21.15.119:0/3851077987 learned_addr learned my addr 172.21.15.119:0/3851077987 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:42.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f0218e00640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f021413e1a0 0x7f021413e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f021a200640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02140af610 0x7f02140afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f0219800640 1 -- 172.21.15.119:0/3851077987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02140af610 msgr2=0x7f02140afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f0219800640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02140af610 0x7f02140afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f0219800640 1 -- 172.21.15.119:0/3851077987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f021413e1a0 msgr2=0x7f021413e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f0219800640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f021413e1a0 0x7f021413e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f0219800640 1 -- 172.21.15.119:0/3851077987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021413adb0 con 0x7f01fc0053c0 2024-04-25T10:07:42.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f0218e00640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f021413e1a0 0x7f021413e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:42.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f0219800640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01fc0053c0 0x7f021415b090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f020800c730 tx=0x7f020800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:42.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f0211600640 1 -- 172.21.15.119:0/3851077987 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208018070 con 0x7f01fc0053c0 2024-04-25T10:07:42.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f0211600640 1 -- 172.21.15.119:0/3851077987 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02080072d0 con 0x7f01fc0053c0 2024-04-25T10:07:42.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f0211600640 1 -- 172.21.15.119:0/3851077987 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02080047e0 con 0x7f01fc0053c0 2024-04-25T10:07:42.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.280+0000 7f021ac00640 1 -- 172.21.15.119:0/3851077987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02140b03b0 con 0x7f01fc0053c0 2024-04-25T10:07:42.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.281+0000 7f021ac00640 1 -- 172.21.15.119:0/3851077987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021413b050 con 0x7f01fc0053c0 2024-04-25T10:07:42.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.282+0000 7f021ac00640 1 -- 172.21.15.119:0/3851077987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02140749b0 con 0x7f01fc0053c0 2024-04-25T10:07:42.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.283+0000 7f0211600640 1 -- 172.21.15.119:0/3851077987 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0208004980 con 0x7f01fc0053c0 2024-04-25T10:07:42.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.284+0000 7f0211600640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f01f406b4d0 0x7f01f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.284+0000 7f0211600640 1 -- 172.21.15.119:0/3851077987 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02080895e0 con 0x7f01fc0053c0 2024-04-25T10:07:42.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.284+0000 7f0218e00640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f01f406b4d0 0x7f01f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.285+0000 7f0218e00640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f01f406b4d0 0x7f01f406d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f021406b0a0 tx=0x7f0200008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:42.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.288+0000 7f0211600640 1 -- 172.21.15.119:0/3851077987 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0208004c70 con 0x7f01fc0053c0 2024-04-25T10:07:42.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.469+0000 7f021ac00640 1 -- 172.21.15.119:0/3851077987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f021406adf0 con 0x7f01fc0053c0 2024-04-25T10:07:42.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.470+0000 7f0211600640 1 -- 172.21.15.119:0/3851077987 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0208058450 con 0x7f01fc0053c0 2024-04-25T10:07:42.476 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:07:42.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.472+0000 7f021ac00640 1 -- 172.21.15.119:0/3851077987 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f01f406b4d0 msgr2=0x7f01f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.472+0000 7f021ac00640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f01f406b4d0 0x7f01f406d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f021406b0a0 tx=0x7f0200008040 comp rx=0 tx=0).stop 2024-04-25T10:07:42.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.472+0000 7f021ac00640 1 -- 172.21.15.119:0/3851077987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01fc0053c0 msgr2=0x7f021415b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.472+0000 7f021ac00640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01fc0053c0 0x7f021415b090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f020800c730 tx=0x7f020800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.472+0000 7f021ac00640 1 -- 172.21.15.119:0/3851077987 shutdown_connections 2024-04-25T10:07:42.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.472+0000 7f021ac00640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f01f406b4d0 0x7f01f406d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.472+0000 7f021ac00640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02140af610 0x7f02140afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.472+0000 7f021ac00640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f021413e1a0 0x7f021413e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.473+0000 7f021ac00640 1 --2- 172.21.15.119:0/3851077987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01fc0053c0 0x7f021415b090 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.473+0000 7f021ac00640 1 -- 172.21.15.119:0/3851077987 >> 172.21.15.119:0/3851077987 conn(0x7f0214066ff0 msgr2=0x7f021406eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:42.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.473+0000 7f021ac00640 1 -- 172.21.15.119:0/3851077987 shutdown_connections 2024-04-25T10:07:42.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.473+0000 7f021ac00640 1 -- 172.21.15.119:0/3851077987 wait complete. 2024-04-25T10:07:42.496 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:42.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.686+0000 7fad87e00640 1 Processor -- start 2024-04-25T10:07:42.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.686+0000 7fad87e00640 1 -- start start 2024-04-25T10:07:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.688+0000 7fad87e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad88144040 0x7fad880cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.688+0000 7fad87e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fad8815acd0 0x7fad8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.688+0000 7fad87e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fad880cce40 0x7fad8806e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.688+0000 7fad87e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad88144420 con 0x7fad880cce40 2024-04-25T10:07:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.688+0000 7fad87e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad880b3d80 con 0x7fad8815acd0 2024-04-25T10:07:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.688+0000 7fad87e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad880b4060 con 0x7fad88144040 2024-04-25T10:07:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.688+0000 7fad87400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fad880cce40 0x7fad8806e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.688+0000 7fad87400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fad880cce40 0x7fad8806e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58926/0 (socket says 172.21.15.119:58926) 2024-04-25T10:07:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.688+0000 7fad87400640 1 -- 172.21.15.119:0/2535017732 learned_addr learned my addr 172.21.15.119:0/2535017732 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.688+0000 7fad86a00640 1 --2- 172.21.15.119:0/2535017732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad88144040 0x7fad880cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.688+0000 7fad86000640 1 --2- 172.21.15.119:0/2535017732 >> v2:172.21.15.132:3300/0 conn(0x7fad8815acd0 0x7fad8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.689+0000 7fad87400640 1 -- 172.21.15.119:0/2535017732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad88144040 msgr2=0x7fad880cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:42.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.689+0000 7fad87400640 1 --2- 172.21.15.119:0/2535017732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad88144040 0x7fad880cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.689+0000 7fad87400640 1 -- 172.21.15.119:0/2535017732 >> v2:172.21.15.132:3300/0 conn(0x7fad8815acd0 msgr2=0x7fad8806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:42.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.689+0000 7fad87400640 1 --2- 172.21.15.119:0/2535017732 >> v2:172.21.15.132:3300/0 conn(0x7fad8815acd0 0x7fad8806e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.689+0000 7fad87400640 1 -- 172.21.15.119:0/2535017732 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad880b4340 con 0x7fad880cce40 2024-04-25T10:07:42.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.689+0000 7fad86a00640 1 --2- 172.21.15.119:0/2535017732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad88144040 0x7fad880cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:42.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.689+0000 7fad86000640 1 --2- 172.21.15.119:0/2535017732 >> v2:172.21.15.132:3300/0 conn(0x7fad8815acd0 0x7fad8806e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:42.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.690+0000 7fad87400640 1 --2- 172.21.15.119:0/2535017732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad880cce40 0x7fad8806e860 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fad8806ef60 tx=0x7fad8002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fac2d09630c0d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:42.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.690+0000 7fad85600640 1 -- 172.21.15.119:0/2535017732 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80037070 con 0x7fad880cce40 2024-04-25T10:07:42.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.690+0000 7fad85600640 1 -- 172.21.15.119:0/2535017732 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad80032440 con 0x7fad880cce40 2024-04-25T10:07:42.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.690+0000 7fad85600640 1 -- 172.21.15.119:0/2535017732 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80032810 con 0x7fad880cce40 2024-04-25T10:07:42.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.691+0000 7fad87e00640 1 -- 172.21.15.119:0/2535017732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad880cce40 msgr2=0x7fad8806e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.691+0000 7fad87e00640 1 --2- 172.21.15.119:0/2535017732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad880cce40 0x7fad8806e860 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fad8806ef60 tx=0x7fad8002a140 comp rx=0 tx=0).stop 2024-04-25T10:07:42.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.691+0000 7fad87e00640 1 -- 172.21.15.119:0/2535017732 shutdown_connections 2024-04-25T10:07:42.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.691+0000 7fad87e00640 1 --2- 172.21.15.119:0/2535017732 >> v2:172.21.15.132:3300/0 conn(0x7fad8815acd0 0x7fad8806e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.691+0000 7fad87e00640 1 --2- 172.21.15.119:0/2535017732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad880cce40 0x7fad8806e860 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.691+0000 7fad87e00640 1 --2- 172.21.15.119:0/2535017732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad88144040 0x7fad880cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.691+0000 7fad87e00640 1 -- 172.21.15.119:0/2535017732 >> 172.21.15.119:0/2535017732 conn(0x7fad8806b1d0 msgr2=0x7fad880b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:42.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.691+0000 7fad87e00640 1 -- 172.21.15.119:0/2535017732 shutdown_connections 2024-04-25T10:07:42.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.691+0000 7fad87e00640 1 -- 172.21.15.119:0/2535017732 wait complete. 2024-04-25T10:07:42.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.693+0000 7fad87e00640 1 Processor -- start 2024-04-25T10:07:42.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.693+0000 7fad87e00640 1 -- start start 2024-04-25T10:07:42.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.694+0000 7fad87e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad880cce40 0x7fad8815a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.694+0000 7fad87e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad88144040 0x7fad881582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.694+0000 7fad87e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8815acd0 0x7fad8813ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.694+0000 7fad87e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad880b5180 con 0x7fad880cce40 2024-04-25T10:07:42.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.694+0000 7fad87e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad880b4900 con 0x7fad88144040 2024-04-25T10:07:42.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.694+0000 7fad87e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad880b5300 con 0x7fad8815acd0 2024-04-25T10:07:42.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.694+0000 7fad86a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad880cce40 0x7fad8815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.695+0000 7fad86a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad880cce40 0x7fad8815a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58932/0 (socket says 172.21.15.119:58932) 2024-04-25T10:07:42.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.695+0000 7fad86a00640 1 -- 172.21.15.119:0/1769317290 learned_addr learned my addr 172.21.15.119:0/1769317290 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:42.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.695+0000 7fad86000640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad88144040 0x7fad881582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.695+0000 7fad87400640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8815acd0 0x7fad8813ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.695+0000 7fad86a00640 1 -- 172.21.15.119:0/1769317290 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8815acd0 msgr2=0x7fad8813ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.695+0000 7fad86a00640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8815acd0 0x7fad8813ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.695+0000 7fad86a00640 1 -- 172.21.15.119:0/1769317290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad88144040 msgr2=0x7fad881582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.695+0000 7fad86a00640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad88144040 0x7fad881582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.695+0000 7fad86a00640 1 -- 172.21.15.119:0/1769317290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad88142140 con 0x7fad880cce40 2024-04-25T10:07:42.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.695+0000 7fad87400640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8815acd0 0x7fad8813ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:42.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.695+0000 7fad86a00640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad880cce40 0x7fad8815a030 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fad7800ce00 tx=0x7fad7800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:42.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.696+0000 7fad7ea00640 1 -- 172.21.15.119:0/1769317290 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7800f680 con 0x7fad880cce40 2024-04-25T10:07:42.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.696+0000 7fad7ea00640 1 -- 172.21.15.119:0/1769317290 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7800f820 con 0x7fad880cce40 2024-04-25T10:07:42.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.696+0000 7fad7ea00640 1 -- 172.21.15.119:0/1769317290 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7800fc30 con 0x7fad880cce40 2024-04-25T10:07:42.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.696+0000 7fad87e00640 1 -- 172.21.15.119:0/1769317290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad88142420 con 0x7fad880cce40 2024-04-25T10:07:42.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.696+0000 7fad87e00640 1 -- 172.21.15.119:0/1769317290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad88158830 con 0x7fad880cce40 2024-04-25T10:07:42.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.697+0000 7fad87e00640 1 -- 172.21.15.119:0/1769317290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8806bb30 con 0x7fad880cce40 2024-04-25T10:07:42.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.698+0000 7fad7ea00640 1 -- 172.21.15.119:0/1769317290 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad78018400 con 0x7fad880cce40 2024-04-25T10:07:42.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.699+0000 7fad7ea00640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fad6406b4d0 0x7fad6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.699+0000 7fad7ea00640 1 -- 172.21.15.119:0/1769317290 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad7808af80 con 0x7fad880cce40 2024-04-25T10:07:42.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.699+0000 7fad86000640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fad6406b4d0 0x7fad6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.700+0000 7fad86000640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fad6406b4d0 0x7fad6406d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fad880cb9a0 tx=0x7fad70008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:42.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.703+0000 7fad7ea00640 1 -- 172.21.15.119:0/1769317290 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad78014030 con 0x7fad880cce40 2024-04-25T10:07:42.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.886+0000 7fad87e00640 1 -- 172.21.15.119:0/1769317290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad88142940 con 0x7fad880cce40 2024-04-25T10:07:42.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.888+0000 7fad7ea00640 1 -- 172.21.15.119:0/1769317290 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fad780597b0 con 0x7fad880cce40 2024-04-25T10:07:42.895 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:42.895 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:08.319104+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:42.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.890+0000 7fad87e00640 1 -- 172.21.15.119:0/1769317290 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fad6406b4d0 msgr2=0x7fad6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.890+0000 7fad87e00640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fad6406b4d0 0x7fad6406d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fad880cb9a0 tx=0x7fad70008040 comp rx=0 tx=0).stop 2024-04-25T10:07:42.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.890+0000 7fad87e00640 1 -- 172.21.15.119:0/1769317290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad880cce40 msgr2=0x7fad8815a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.890+0000 7fad87e00640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad880cce40 0x7fad8815a030 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fad7800ce00 tx=0x7fad7800a650 comp rx=0 tx=0).stop 2024-04-25T10:07:42.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.891+0000 7fad87e00640 1 -- 172.21.15.119:0/1769317290 shutdown_connections 2024-04-25T10:07:42.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.891+0000 7fad87e00640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fad6406b4d0 0x7fad6406d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.891+0000 7fad87e00640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8815acd0 0x7fad8813ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.891+0000 7fad87e00640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad88144040 0x7fad881582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.891+0000 7fad87e00640 1 --2- 172.21.15.119:0/1769317290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad880cce40 0x7fad8815a030 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.891+0000 7fad87e00640 1 -- 172.21.15.119:0/1769317290 >> 172.21.15.119:0/1769317290 conn(0x7fad8806b1d0 msgr2=0x7fad8813d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:42.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.891+0000 7fad87e00640 1 -- 172.21.15.119:0/1769317290 shutdown_connections 2024-04-25T10:07:42.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:42.891+0000 7fad87e00640 1 -- 172.21.15.119:0/1769317290 wait complete. 2024-04-25T10:07:42.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:05.135832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:05.768083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:42.915 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:43.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.108+0000 7fee7ca00640 1 Processor -- start 2024-04-25T10:07:43.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.108+0000 7fee7ca00640 1 -- start start 2024-04-25T10:07:43.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.110+0000 7fee7ca00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fee780ce860 0x7fee780cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.110+0000 7fee7ca00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fee780d00a0 0x7fee780d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.110+0000 7fee7ca00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee780d0a50 con 0x7fee7815b7e0 2024-04-25T10:07:43.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.110+0000 7fee7ca00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee780b5710 con 0x7fee780ce860 2024-04-25T10:07:43.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.110+0000 7fee7ca00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee780b59f0 con 0x7fee780d00a0 2024-04-25T10:07:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.110+0000 7fee76a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fee780ce860 0x7fee780cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.110+0000 7fee77400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fee7815b7e0 0x7fee7815e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36670/0 (socket says 172.21.15.119:36670) 2024-04-25T10:07:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.110+0000 7fee77400640 1 -- 172.21.15.119:0/2443916655 learned_addr learned my addr 172.21.15.119:0/2443916655 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.111+0000 7fee77e00640 1 --2- 172.21.15.119:0/2443916655 >> v2:172.21.15.132:3300/0 conn(0x7fee780d00a0 0x7fee780d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.111+0000 7fee76a00640 1 -- 172.21.15.119:0/2443916655 >> v2:172.21.15.132:3300/0 conn(0x7fee780d00a0 msgr2=0x7fee780d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.111+0000 7fee76a00640 1 --2- 172.21.15.119:0/2443916655 >> v2:172.21.15.132:3300/0 conn(0x7fee780d00a0 0x7fee780d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.111+0000 7fee76a00640 1 -- 172.21.15.119:0/2443916655 >> v1:172.21.15.119:6789/0 conn(0x7fee7815b7e0 legacy=0x7fee7815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.111+0000 7fee76a00640 1 -- 172.21.15.119:0/2443916655 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee780b5cd0 con 0x7fee780ce860 2024-04-25T10:07:43.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.111+0000 7fee76000640 1 -- 172.21.15.119:0/2443916655 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041384888 0 0) 0x7fee780d0a50 con 0x7fee7815b7e0 2024-04-25T10:07:43.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.112+0000 7fee76a00640 1 --2- 172.21.15.119:0/2443916655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee780ce860 0x7fee780cecf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fee7808bf40 tx=0x7fee6002a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334f54d06d6dc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.112+0000 7fee76000640 1 -- 172.21.15.119:0/2443916655 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60007740 con 0x7fee780ce860 2024-04-25T10:07:43.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.112+0000 7fee76000640 1 -- 172.21.15.119:0/2443916655 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee600078e0 con 0x7fee780ce860 2024-04-25T10:07:43.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.112+0000 7fee76000640 1 -- 172.21.15.119:0/2443916655 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60007cb0 con 0x7fee780ce860 2024-04-25T10:07:43.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.113+0000 7fee7ca00640 1 -- 172.21.15.119:0/2443916655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee780ce860 msgr2=0x7fee780cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.113+0000 7fee7ca00640 1 --2- 172.21.15.119:0/2443916655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee780ce860 0x7fee780cecf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fee7808bf40 tx=0x7fee6002a560 comp rx=0 tx=0).stop 2024-04-25T10:07:43.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.113+0000 7fee7ca00640 1 -- 172.21.15.119:0/2443916655 shutdown_connections 2024-04-25T10:07:43.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.113+0000 7fee7ca00640 1 --2- 172.21.15.119:0/2443916655 >> v2:172.21.15.132:3300/0 conn(0x7fee780d00a0 0x7fee780d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.113+0000 7fee7ca00640 1 --2- 172.21.15.119:0/2443916655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee780ce860 0x7fee780cecf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.113+0000 7fee7ca00640 1 -- 172.21.15.119:0/2443916655 >> 172.21.15.119:0/2443916655 conn(0x7fee780722d0 msgr2=0x7fee780726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:43.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.113+0000 7fee7ca00640 1 -- 172.21.15.119:0/2443916655 shutdown_connections 2024-04-25T10:07:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.113+0000 7fee7ca00640 1 -- 172.21.15.119:0/2443916655 wait complete. 2024-04-25T10:07:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.114+0000 7fee7ca00640 1 Processor -- start 2024-04-25T10:07:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.115+0000 7fee7ca00640 1 -- start start 2024-04-25T10:07:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.115+0000 7fee7ca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780ce860 0x7fee78151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.115+0000 7fee7ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee780d00a0 0x7fee780cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.115+0000 7fee7ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815b7e0 0x7fee7815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.115+0000 7fee7ca00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee78161960 con 0x7fee7815b7e0 2024-04-25T10:07:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.115+0000 7fee7ca00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee780b53c0 con 0x7fee780ce860 2024-04-25T10:07:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.115+0000 7fee7ca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee78161ae0 con 0x7fee780d00a0 2024-04-25T10:07:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.116+0000 7fee77e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815b7e0 0x7fee7815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.116+0000 7fee77e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815b7e0 0x7fee7815a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58944/0 (socket says 172.21.15.119:58944) 2024-04-25T10:07:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.116+0000 7fee77e00640 1 -- 172.21.15.119:0/1188280096 learned_addr learned my addr 172.21.15.119:0/1188280096 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.116+0000 7fee76a00640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee780d00a0 0x7fee780cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.116+0000 7fee77e00640 1 -- 172.21.15.119:0/1188280096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee780d00a0 msgr2=0x7fee780cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.116+0000 7fee77e00640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee780d00a0 0x7fee780cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.116+0000 7fee77e00640 1 -- 172.21.15.119:0/1188280096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780ce860 msgr2=0x7fee78151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.116+0000 7fee77400640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780ce860 0x7fee78151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.116+0000 7fee77e00640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780ce860 0x7fee78151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.116+0000 7fee77e00640 1 -- 172.21.15.119:0/1188280096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee780c8790 con 0x7fee7815b7e0 2024-04-25T10:07:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.116+0000 7fee77400640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780ce860 0x7fee78151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.117+0000 7fee77e00640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815b7e0 0x7fee7815a570 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fee7000c7f0 tx=0x7fee7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.117+0000 7fee6f400640 1 -- 172.21.15.119:0/1188280096 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70015070 con 0x7fee7815b7e0 2024-04-25T10:07:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.117+0000 7fee6f400640 1 -- 172.21.15.119:0/1188280096 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee70004220 con 0x7fee7815b7e0 2024-04-25T10:07:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.117+0000 7fee6f400640 1 -- 172.21.15.119:0/1188280096 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70007620 con 0x7fee7815b7e0 2024-04-25T10:07:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.117+0000 7fee7ca00640 1 -- 172.21.15.119:0/1188280096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee78152060 con 0x7fee7815b7e0 2024-04-25T10:07:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.117+0000 7fee7ca00640 1 -- 172.21.15.119:0/1188280096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee781f7080 con 0x7fee7815b7e0 2024-04-25T10:07:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.118+0000 7fee7ca00640 1 -- 172.21.15.119:0/1188280096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee780749b0 con 0x7fee7815b7e0 2024-04-25T10:07:43.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.119+0000 7fee6f400640 1 -- 172.21.15.119:0/1188280096 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee700079d0 con 0x7fee7815b7e0 2024-04-25T10:07:43.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.120+0000 7fee6f400640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fee5406b4d0 0x7fee5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.120+0000 7fee6f400640 1 -- 172.21.15.119:0/1188280096 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee7008a2c0 con 0x7fee7815b7e0 2024-04-25T10:07:43.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.120+0000 7fee77400640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fee5406b4d0 0x7fee5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.121+0000 7fee77400640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fee5406b4d0 0x7fee5406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fee68000f60 tx=0x7fee68009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.124+0000 7fee6f400640 1 -- 172.21.15.119:0/1188280096 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee780c8790 con 0x7fee7815b7e0 2024-04-25T10:07:43.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.317+0000 7fee7ca00640 1 -- 172.21.15.119:0/1188280096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee780d45c0 con 0x7fee7815b7e0 2024-04-25T10:07:43.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.318+0000 7fee6f400640 1 -- 172.21.15.119:0/1188280096 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee700590a0 con 0x7fee7815b7e0 2024-04-25T10:07:43.324 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:07:43.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.320+0000 7fee7ca00640 1 -- 172.21.15.119:0/1188280096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fee5406b4d0 msgr2=0x7fee5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.320+0000 7fee7ca00640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fee5406b4d0 0x7fee5406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fee68000f60 tx=0x7fee68009040 comp rx=0 tx=0).stop 2024-04-25T10:07:43.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.320+0000 7fee7ca00640 1 -- 172.21.15.119:0/1188280096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815b7e0 msgr2=0x7fee7815a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.320+0000 7fee7ca00640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815b7e0 0x7fee7815a570 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fee7000c7f0 tx=0x7fee7000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.320+0000 7fee7ca00640 1 -- 172.21.15.119:0/1188280096 shutdown_connections 2024-04-25T10:07:43.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.320+0000 7fee7ca00640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fee5406b4d0 0x7fee5406d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.320+0000 7fee7ca00640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815b7e0 0x7fee7815a570 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.320+0000 7fee7ca00640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee780d00a0 0x7fee780cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.320+0000 7fee7ca00640 1 --2- 172.21.15.119:0/1188280096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee780ce860 0x7fee78151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.320+0000 7fee7ca00640 1 -- 172.21.15.119:0/1188280096 >> 172.21.15.119:0/1188280096 conn(0x7fee780722d0 msgr2=0x7fee780d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:43.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.321+0000 7fee7ca00640 1 -- 172.21.15.119:0/1188280096 shutdown_connections 2024-04-25T10:07:43.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.321+0000 7fee7ca00640 1 -- 172.21.15.119:0/1188280096 wait complete. 2024-04-25T10:07:43.344 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:43.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.532+0000 7f0f1e600640 1 Processor -- start 2024-04-25T10:07:43.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.532+0000 7f0f1e600640 1 -- start start 2024-04-25T10:07:43.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.534+0000 7f0f1e600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0f180b42c0 0x7f0f1815ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.534+0000 7f0f1e600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f180b46a0 0x7f0f181549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.534+0000 7f0f1e600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f1815b370 con 0x7f0f1815aef0 2024-04-25T10:07:43.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.534+0000 7f0f1e600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f180d05d0 con 0x7f0f180b42c0 2024-04-25T10:07:43.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.534+0000 7f0f1e600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1815b650 con 0x7f0f180b46a0 2024-04-25T10:07:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.534+0000 7f0f17400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0f180b42c0 0x7f0f1815ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.534+0000 7f0f17400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0f180b42c0 0x7f0f1815ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58954/0 (socket says 172.21.15.119:58954) 2024-04-25T10:07:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.534+0000 7f0f16a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0f1815aef0 0x7f0f180cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36686/0 (socket says 172.21.15.119:36686) 2024-04-25T10:07:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.534+0000 7f0f17400640 1 -- 172.21.15.119:0/3142685540 learned_addr learned my addr 172.21.15.119:0/3142685540 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.534+0000 7f0f17400640 1 -- 172.21.15.119:0/3142685540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f180b46a0 msgr2=0x7f0f181549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.534+0000 7f0f17400640 1 --2- 172.21.15.119:0/3142685540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f180b46a0 0x7f0f181549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.534+0000 7f0f17400640 1 -- 172.21.15.119:0/3142685540 >> v1:172.21.15.119:6789/0 conn(0x7f0f1815aef0 legacy=0x7f0f180cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.534+0000 7f0f17400640 1 -- 172.21.15.119:0/3142685540 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1815b930 con 0x7f0f180b42c0 2024-04-25T10:07:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.534+0000 7f0f16000640 1 -- 172.21.15.119:0/3142685540 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434969417 0 0) 0x7f0f1815b370 con 0x7f0f1815aef0 2024-04-25T10:07:43.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.535+0000 7f0f17400640 1 --2- 172.21.15.119:0/3142685540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f180b42c0 0x7f0f1815ef40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009870 tx=0x7f0f0802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8a0c10501551bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.536+0000 7f0f16000640 1 -- 172.21.15.119:0/3142685540 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08037070 con 0x7f0f180b42c0 2024-04-25T10:07:43.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.536+0000 7f0f16000640 1 -- 172.21.15.119:0/3142685540 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f08032440 con 0x7f0f180b42c0 2024-04-25T10:07:43.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.536+0000 7f0f16000640 1 -- 172.21.15.119:0/3142685540 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08032810 con 0x7f0f180b42c0 2024-04-25T10:07:43.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.536+0000 7f0f1e600640 1 -- 172.21.15.119:0/3142685540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f180b42c0 msgr2=0x7f0f1815ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.536+0000 7f0f1e600640 1 --2- 172.21.15.119:0/3142685540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f180b42c0 0x7f0f1815ef40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009870 tx=0x7f0f0802a130 comp rx=0 tx=0).stop 2024-04-25T10:07:43.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.536+0000 7f0f1e600640 1 -- 172.21.15.119:0/3142685540 shutdown_connections 2024-04-25T10:07:43.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.536+0000 7f0f1e600640 1 --2- 172.21.15.119:0/3142685540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f180b46a0 0x7f0f181549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.536+0000 7f0f1e600640 1 --2- 172.21.15.119:0/3142685540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f180b42c0 0x7f0f1815ef40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.536+0000 7f0f1e600640 1 -- 172.21.15.119:0/3142685540 >> 172.21.15.119:0/3142685540 conn(0x7f0f1806aba0 msgr2=0x7f0f1806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:43.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.536+0000 7f0f1e600640 1 -- 172.21.15.119:0/3142685540 shutdown_connections 2024-04-25T10:07:43.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.536+0000 7f0f1e600640 1 -- 172.21.15.119:0/3142685540 wait complete. 2024-04-25T10:07:43.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.538+0000 7f0f1e600640 1 Processor -- start 2024-04-25T10:07:43.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.538+0000 7f0f1e600640 1 -- start start 2024-04-25T10:07:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.539+0000 7f0f1e600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f180b42c0 0x7f0f1814d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.539+0000 7f0f1e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f180b46a0 0x7f0f180b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.539+0000 7f0f1e600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f1815aef0 0x7f0f180b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.539+0000 7f0f1e600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1806ec60 con 0x7f0f180b46a0 2024-04-25T10:07:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.539+0000 7f0f1e600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1806eae0 con 0x7f0f180b42c0 2024-04-25T10:07:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.539+0000 7f0f1e600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f18154f10 con 0x7f0f1815aef0 2024-04-25T10:07:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.540+0000 7f0f16a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f180b46a0 0x7f0f180b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.540+0000 7f0f16a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f180b46a0 0x7f0f180b2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58962/0 (socket says 172.21.15.119:58962) 2024-04-25T10:07:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.540+0000 7f0f16a00640 1 -- 172.21.15.119:0/1175918443 learned_addr learned my addr 172.21.15.119:0/1175918443 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.540+0000 7f0f17e00640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f1815aef0 0x7f0f180b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.540+0000 7f0f17400640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f180b42c0 0x7f0f1814d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.540+0000 7f0f16a00640 1 -- 172.21.15.119:0/1175918443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f1815aef0 msgr2=0x7f0f180b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.540+0000 7f0f16a00640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f1815aef0 0x7f0f180b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.540+0000 7f0f16a00640 1 -- 172.21.15.119:0/1175918443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f180b42c0 msgr2=0x7f0f1814d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.540+0000 7f0f16a00640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f180b42c0 0x7f0f1814d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.540+0000 7f0f16a00640 1 -- 172.21.15.119:0/1175918443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f180ca1b0 con 0x7f0f180b46a0 2024-04-25T10:07:43.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.540+0000 7f0f17400640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f180b42c0 0x7f0f1814d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:43.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.540+0000 7f0f16a00640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f180b46a0 0x7f0f180b2a10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400c410 tx=0x7f0f0400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.541+0000 7f0f0f400640 1 -- 172.21.15.119:0/1175918443 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f040074e0 con 0x7f0f180b46a0 2024-04-25T10:07:43.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.541+0000 7f0f0f400640 1 -- 172.21.15.119:0/1175918443 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f040047e0 con 0x7f0f180b46a0 2024-04-25T10:07:43.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.541+0000 7f0f0f400640 1 -- 172.21.15.119:0/1175918443 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04004bd0 con 0x7f0f180b46a0 2024-04-25T10:07:43.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.541+0000 7f0f1e600640 1 -- 172.21.15.119:0/1175918443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f180ca490 con 0x7f0f180b46a0 2024-04-25T10:07:43.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.541+0000 7f0f1e600640 1 -- 172.21.15.119:0/1175918443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f180caa50 con 0x7f0f180b46a0 2024-04-25T10:07:43.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.542+0000 7f0f1e600640 1 -- 172.21.15.119:0/1175918443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f180750b0 con 0x7f0f180b46a0 2024-04-25T10:07:43.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.543+0000 7f0f0f400640 1 -- 172.21.15.119:0/1175918443 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f04011070 con 0x7f0f180b46a0 2024-04-25T10:07:43.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.544+0000 7f0f0f400640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0ef806b4d0 0x7f0ef806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.544+0000 7f0f0f400640 1 -- 172.21.15.119:0/1175918443 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f04089e60 con 0x7f0f180b46a0 2024-04-25T10:07:43.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.544+0000 7f0f17400640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0ef806b4d0 0x7f0ef806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.545+0000 7f0f17400640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0ef806b4d0 0x7f0ef806d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0f08034000 tx=0x7f0f0802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.548+0000 7f0f0f400640 1 -- 172.21.15.119:0/1175918443 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f04058cd0 con 0x7f0f180b46a0 2024-04-25T10:07:43.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.726+0000 7f0f1e600640 1 -- 172.21.15.119:0/1175918443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0f180cac20 con 0x7f0f180b46a0 2024-04-25T10:07:43.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.727+0000 7f0f0f400640 1 -- 172.21.15.119:0/1175918443 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0f040585d0 con 0x7f0f180b46a0 2024-04-25T10:07:43.733 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:07:43.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.729+0000 7f0f1e600640 1 -- 172.21.15.119:0/1175918443 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0ef806b4d0 msgr2=0x7f0ef806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.729+0000 7f0f1e600640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0ef806b4d0 0x7f0ef806d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0f08034000 tx=0x7f0f0802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:43.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.729+0000 7f0f1e600640 1 -- 172.21.15.119:0/1175918443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f180b46a0 msgr2=0x7f0f180b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.729+0000 7f0f1e600640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f180b46a0 0x7f0f180b2a10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400c410 tx=0x7f0f0400c8d0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.729+0000 7f0f1e600640 1 -- 172.21.15.119:0/1175918443 shutdown_connections 2024-04-25T10:07:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.729+0000 7f0f1e600640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0ef806b4d0 0x7f0ef806d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.729+0000 7f0f1e600640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f1815aef0 0x7f0f180b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.729+0000 7f0f1e600640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f180b46a0 0x7f0f180b2a10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.729+0000 7f0f1e600640 1 --2- 172.21.15.119:0/1175918443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f180b42c0 0x7f0f1814d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.729+0000 7f0f1e600640 1 -- 172.21.15.119:0/1175918443 >> 172.21.15.119:0/1175918443 conn(0x7f0f1806aba0 msgr2=0x7f0f180b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.729+0000 7f0f1e600640 1 -- 172.21.15.119:0/1175918443 shutdown_connections 2024-04-25T10:07:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.730+0000 7f0f1e600640 1 -- 172.21.15.119:0/1175918443 wait complete. 2024-04-25T10:07:43.752 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:43.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.947+0000 7f8005200640 1 Processor -- start 2024-04-25T10:07:43.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.948+0000 7f8005200640 1 -- start start 2024-04-25T10:07:43.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.949+0000 7f8005200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f80000ce9d0 0x7f80000cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.949+0000 7f8005200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800006dcb0 con 0x7f8000144e70 2024-04-25T10:07:43.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.949+0000 7f8005200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800006df90 con 0x7f80000cc300 2024-04-25T10:07:43.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.949+0000 7f8005200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f800015bfe0 con 0x7f80000ce9d0 2024-04-25T10:07:43.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.949+0000 7f7ffde00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8000144e70 0x7f800006d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36690/0 (socket says 172.21.15.119:36690) 2024-04-25T10:07:43.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.949+0000 7f7ffde00640 1 -- 172.21.15.119:0/1147187249 learned_addr learned my addr 172.21.15.119:0/1147187249 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.950+0000 7f7ffd400640 1 -- 172.21.15.119:0/1147187249 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465379482 0 0) 0x7f800006dcb0 con 0x7f8000144e70 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.950+0000 7f7ffd400640 1 -- 172.21.15.119:0/1147187249 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fe4003520 con 0x7f8000144e70 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.950+0000 7f7ff5400640 1 --2- 172.21.15.119:0/1147187249 >> v2:172.21.15.132:3300/0 conn(0x7f80000ce9d0 0x7f80000cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.950+0000 7f7ffd400640 1 -- 172.21.15.119:0/1147187249 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3606527379 0 0) 0x7f7fe4003520 con 0x7f8000144e70 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.950+0000 7f7ffd400640 1 -- 172.21.15.119:0/1147187249 >> v2:172.21.15.132:3300/0 conn(0x7f80000ce9d0 msgr2=0x7f80000cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.950+0000 7f7ffd400640 1 --2- 172.21.15.119:0/1147187249 >> v2:172.21.15.132:3300/0 conn(0x7f80000ce9d0 0x7f80000cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.950+0000 7f7ffd400640 1 -- 172.21.15.119:0/1147187249 >> v1:172.21.15.132:6789/0 conn(0x7f80000cc300 legacy=0x7f80000cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.950+0000 7f7ffd400640 1 -- 172.21.15.119:0/1147187249 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800006e270 con 0x7f8000144e70 2024-04-25T10:07:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.950+0000 7f7ffd400640 1 -- 172.21.15.119:0/1147187249 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f7ff0001670 con 0x7f8000144e70 2024-04-25T10:07:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.950+0000 7f7ffd400640 1 -- 172.21.15.119:0/1147187249 >> v1:172.21.15.119:6789/0 conn(0x7f8000144e70 legacy=0x7f800006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.950+0000 7f7ffd400640 1 --2- 172.21.15.119:0/1147187249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe40039b0 0x7f7fe4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.950+0000 7f7ffd400640 1 --2- 172.21.15.119:0/1147187249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7fe4004800 0x7f7fe4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.950+0000 7f7ffd400640 1 --2- 172.21.15.119:0/1147187249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7fe4005210 0x7f7fe400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.951+0000 7f7ffd400640 1 -- 172.21.15.119:0/1147187249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800015bfe0 con 0x7f7fe40039b0 2024-04-25T10:07:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.951+0000 7f7ffd400640 1 -- 172.21.15.119:0/1147187249 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe400c140 con 0x7f7fe4004800 2024-04-25T10:07:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.951+0000 7f7ffd400640 1 -- 172.21.15.119:0/1147187249 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe400c340 con 0x7f7fe4005210 2024-04-25T10:07:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.951+0000 7f7ffde00640 1 --2- 172.21.15.119:0/1147187249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe40039b0 0x7f7fe4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.951+0000 7f7ffde00640 1 -- 172.21.15.119:0/1147187249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7fe4005210 msgr2=0x7f7fe400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.951+0000 7f7ffde00640 1 --2- 172.21.15.119:0/1147187249 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7fe4005210 0x7f7fe400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.951+0000 7f7ffde00640 1 -- 172.21.15.119:0/1147187249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7fe4004800 msgr2=0x7f7fe4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.951+0000 7f7ff5400640 1 --2- 172.21.15.119:0/1147187249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7fe4004800 0x7f7fe4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.951+0000 7f7ffde00640 1 --2- 172.21.15.119:0/1147187249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7fe4004800 0x7f7fe4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.951+0000 7f7ffde00640 1 -- 172.21.15.119:0/1147187249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe4003520 con 0x7f7fe40039b0 2024-04-25T10:07:43.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.951+0000 7f7ffe800640 1 -- 172.21.15.119:0/1147187249 reap_dead start 2024-04-25T10:07:43.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.952+0000 7f7ffde00640 1 --2- 172.21.15.119:0/1147187249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe40039b0 0x7f7fe4003df0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000c240 tx=0x7f7fe4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c297de33769ded7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.952+0000 7f7ffd400640 1 -- 172.21.15.119:0/1147187249 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0007d10 con 0x7f7fe40039b0 2024-04-25T10:07:43.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.952+0000 7f7ffd400640 1 -- 172.21.15.119:0/1147187249 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff0005ad0 con 0x7f7fe40039b0 2024-04-25T10:07:43.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.952+0000 7f7ffd400640 1 -- 172.21.15.119:0/1147187249 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0007070 con 0x7f7fe40039b0 2024-04-25T10:07:43.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.953+0000 7f8005200640 1 -- 172.21.15.119:0/1147187249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe40039b0 msgr2=0x7f7fe4003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.953+0000 7f8005200640 1 --2- 172.21.15.119:0/1147187249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe40039b0 0x7f7fe4003df0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000c240 tx=0x7f7fe4005830 comp rx=0 tx=0).stop 2024-04-25T10:07:43.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.953+0000 7f8005200640 1 -- 172.21.15.119:0/1147187249 shutdown_connections 2024-04-25T10:07:43.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.953+0000 7f8005200640 1 --2- 172.21.15.119:0/1147187249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fe40039b0 0x7f7fe4003df0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.953+0000 7f8005200640 1 -- 172.21.15.119:0/1147187249 >> 172.21.15.119:0/1147187249 conn(0x7f800006aba0 msgr2=0x7f800006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:43.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.953+0000 7f8005200640 1 -- 172.21.15.119:0/1147187249 shutdown_connections 2024-04-25T10:07:43.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.953+0000 7f8005200640 1 -- 172.21.15.119:0/1147187249 wait complete. 2024-04-25T10:07:43.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.955+0000 7f8005200640 1 Processor -- start 2024-04-25T10:07:43.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.955+0000 7f8005200640 1 -- start start 2024-04-25T10:07:43.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.956+0000 7f8005200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7fe40039b0 0x7f80001efd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.956+0000 7f8005200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8000143660 0x7f80000afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.956+0000 7f8005200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f800013bc70 0x7f800013c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.956+0000 7f8005200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8000145630 con 0x7f800013bc70 2024-04-25T10:07:43.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.956+0000 7f8005200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80001454b0 con 0x7f7fe40039b0 2024-04-25T10:07:43.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.956+0000 7f8005200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80000cee60 con 0x7f8000143660 2024-04-25T10:07:43.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.956+0000 7f7ffe800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f800013bc70 0x7f800013c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.956+0000 7f7ffe800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f800013bc70 0x7f800013c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58994/0 (socket says 172.21.15.119:58994) 2024-04-25T10:07:43.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.956+0000 7f7ffe800640 1 -- 172.21.15.119:0/687633500 learned_addr learned my addr 172.21.15.119:0/687633500 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:43.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.957+0000 7f7ffe800640 1 -- 172.21.15.119:0/687633500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8000143660 msgr2=0x7f80000afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.957+0000 7f7ffe800640 1 --2- 172.21.15.119:0/687633500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8000143660 0x7f80000afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.957+0000 7f7ffde00640 1 --2- 172.21.15.119:0/687633500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7fe40039b0 0x7f80001efd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.957+0000 7f7ffe800640 1 -- 172.21.15.119:0/687633500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7fe40039b0 msgr2=0x7f80001efd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:43.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.957+0000 7f7ffe800640 1 --2- 172.21.15.119:0/687633500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7fe40039b0 0x7f80001efd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:43.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.957+0000 7f7ffe800640 1 -- 172.21.15.119:0/687633500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80001f02b0 con 0x7f800013bc70 2024-04-25T10:07:43.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.957+0000 7f7ffe800640 1 --2- 172.21.15.119:0/687633500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f800013bc70 0x7f800013c120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f800006f690 tx=0x7f7ff800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.957+0000 7f7ff6a00640 1 -- 172.21.15.119:0/687633500 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8017070 con 0x7f800013bc70 2024-04-25T10:07:43.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.957+0000 7f7ff6a00640 1 -- 172.21.15.119:0/687633500 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff8005d40 con 0x7f800013bc70 2024-04-25T10:07:43.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.957+0000 7f7ff6a00640 1 -- 172.21.15.119:0/687633500 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800a830 con 0x7f800013bc70 2024-04-25T10:07:43.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.957+0000 7f8005200640 1 -- 172.21.15.119:0/687633500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80001425c0 con 0x7f800013bc70 2024-04-25T10:07:43.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.957+0000 7f8005200640 1 -- 172.21.15.119:0/687633500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8000142b80 con 0x7f800013bc70 2024-04-25T10:07:43.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.958+0000 7f8005200640 1 -- 172.21.15.119:0/687633500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80000750b0 con 0x7f800013bc70 2024-04-25T10:07:43.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.959+0000 7f7ff6a00640 1 -- 172.21.15.119:0/687633500 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ff800a9d0 con 0x7f800013bc70 2024-04-25T10:07:43.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.960+0000 7f7ff6a00640 1 --2- 172.21.15.119:0/687633500 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7fdc06b4d0 0x7f7fdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:43.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.960+0000 7f7ff6a00640 1 -- 172.21.15.119:0/687633500 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ff8089240 con 0x7f800013bc70 2024-04-25T10:07:43.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.960+0000 7f7ffde00640 1 --2- 172.21.15.119:0/687633500 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7fdc06b4d0 0x7f7fdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:43.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.961+0000 7f7ffde00640 1 --2- 172.21.15.119:0/687633500 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7fdc06b4d0 0x7f7fdc06d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0033000 tx=0x7f7ff002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:43.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:43.964+0000 7f7ff6a00640 1 -- 172.21.15.119:0/687633500 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f80001f02b0 con 0x7f800013bc70 2024-04-25T10:07:44.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.160+0000 7f8005200640 1 -- 172.21.15.119:0/687633500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80000aebe0 con 0x7f800013bc70 2024-04-25T10:07:44.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.160+0000 7f7ff6a00640 1 -- 172.21.15.119:0/687633500 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ff8058020 con 0x7f800013bc70 2024-04-25T10:07:44.166 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:07:44.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.163+0000 7f8005200640 1 -- 172.21.15.119:0/687633500 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7fdc06b4d0 msgr2=0x7f7fdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.163+0000 7f8005200640 1 --2- 172.21.15.119:0/687633500 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7fdc06b4d0 0x7f7fdc06d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0033000 tx=0x7f7ff002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:44.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.163+0000 7f8005200640 1 -- 172.21.15.119:0/687633500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f800013bc70 msgr2=0x7f800013c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.163+0000 7f8005200640 1 --2- 172.21.15.119:0/687633500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f800013bc70 0x7f800013c120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f800006f690 tx=0x7f7ff800cb90 comp rx=0 tx=0).stop 2024-04-25T10:07:44.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.163+0000 7f8005200640 1 -- 172.21.15.119:0/687633500 shutdown_connections 2024-04-25T10:07:44.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.163+0000 7f8005200640 1 --2- 172.21.15.119:0/687633500 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7fdc06b4d0 0x7f7fdc06d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.163+0000 7f8005200640 1 --2- 172.21.15.119:0/687633500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f800013bc70 0x7f800013c120 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.163+0000 7f8005200640 1 --2- 172.21.15.119:0/687633500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8000143660 0x7f80000afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.163+0000 7f8005200640 1 --2- 172.21.15.119:0/687633500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7fe40039b0 0x7f80001efd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.163+0000 7f8005200640 1 -- 172.21.15.119:0/687633500 >> 172.21.15.119:0/687633500 conn(0x7f800006aba0 msgr2=0x7f800015ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:44.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.164+0000 7f8005200640 1 -- 172.21.15.119:0/687633500 shutdown_connections 2024-04-25T10:07:44.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.164+0000 7f8005200640 1 -- 172.21.15.119:0/687633500 wait complete. 2024-04-25T10:07:44.195 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:07:44.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:44.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.385+0000 7f1801a00640 1 Processor -- start 2024-04-25T10:07:44.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.386+0000 7f1801a00640 1 -- start start 2024-04-25T10:07:44.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.387+0000 7f1801a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f17fc06ea30 0x7f17fc1653e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.387+0000 7f1801a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17fc144420 con 0x7f17fc15acd0 2024-04-25T10:07:44.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.387+0000 7f1801a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17fc0b3d80 con 0x7f17fc144040 2024-04-25T10:07:44.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.387+0000 7f1801a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17fc0b36d0 con 0x7f17fc06ea30 2024-04-25T10:07:44.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.387+0000 7f17f3e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f17fc15acd0 0x7f17fc06e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36706/0 (socket says 172.21.15.119:36706) 2024-04-25T10:07:44.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.387+0000 7f17f3e00640 1 -- 172.21.15.119:0/3858906376 learned_addr learned my addr 172.21.15.119:0/3858906376 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.388+0000 7f1801000640 1 --2- 172.21.15.119:0/3858906376 >> v2:172.21.15.132:3300/0 conn(0x7f17fc06ea30 0x7f17fc1653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.388+0000 7f17fb400640 1 -- 172.21.15.119:0/3858906376 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399875664 0 0) 0x7f17fc144420 con 0x7f17fc15acd0 2024-04-25T10:07:44.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.388+0000 7f17fb400640 1 -- 172.21.15.119:0/3858906376 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17e0003520 con 0x7f17fc15acd0 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.388+0000 7f17fb400640 1 -- 172.21.15.119:0/3858906376 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1806684783 0 0) 0x7f17e0003520 con 0x7f17fc15acd0 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.388+0000 7f17fb400640 1 -- 172.21.15.119:0/3858906376 >> v2:172.21.15.132:3300/0 conn(0x7f17fc06ea30 msgr2=0x7f17fc1653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.388+0000 7f17fb400640 1 --2- 172.21.15.119:0/3858906376 >> v2:172.21.15.132:3300/0 conn(0x7f17fc06ea30 0x7f17fc1653e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.388+0000 7f17fb400640 1 -- 172.21.15.119:0/3858906376 >> v1:172.21.15.132:6789/0 conn(0x7f17fc144040 legacy=0x7f17fc0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.388+0000 7f17fb400640 1 -- 172.21.15.119:0/3858906376 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc0b4060 con 0x7f17fc15acd0 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.388+0000 7f17fb400640 1 -- 172.21.15.119:0/3858906376 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f17e4001670 con 0x7f17fc15acd0 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.388+0000 7f17fb400640 1 -- 172.21.15.119:0/3858906376 >> v1:172.21.15.119:6789/0 conn(0x7f17fc15acd0 legacy=0x7f17fc06e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f17fb400640 1 --2- 172.21.15.119:0/3858906376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e0003a70 0x7f17e0003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f17fb400640 1 --2- 172.21.15.119:0/3858906376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e0004800 0x7f17e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f17fb400640 1 --2- 172.21.15.119:0/3858906376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17e00052e0 0x7f17e000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f17fb400640 1 -- 172.21.15.119:0/3858906376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc0b36d0 con 0x7f17e0004800 2024-04-25T10:07:44.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f17fb400640 1 -- 172.21.15.119:0/3858906376 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e000e3c0 con 0x7f17e00052e0 2024-04-25T10:07:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f17fb400640 1 -- 172.21.15.119:0/3858906376 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e000e5c0 con 0x7f17e0003a70 2024-04-25T10:07:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f17f3e00640 1 --2- 172.21.15.119:0/3858906376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e0004800 0x7f17e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f17f3e00640 1 -- 172.21.15.119:0/3858906376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e0003a70 msgr2=0x7f17e0003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f17fbe00640 1 --2- 172.21.15.119:0/3858906376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e0003a70 0x7f17e0003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f17f3e00640 1 --2- 172.21.15.119:0/3858906376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e0003a70 0x7f17e0003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f17f3e00640 1 -- 172.21.15.119:0/3858906376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17e00052e0 msgr2=0x7f17e000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f17f3e00640 1 --2- 172.21.15.119:0/3858906376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17e00052e0 0x7f17e000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f17f3e00640 1 -- 172.21.15.119:0/3858906376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e0003520 con 0x7f17e0004800 2024-04-25T10:07:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f1801000640 1 -- 172.21.15.119:0/3858906376 reap_dead start 2024-04-25T10:07:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.389+0000 7f17fbe00640 1 --2- 172.21.15.119:0/3858906376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e0003a70 0x7f17e0003e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.390+0000 7f17f3e00640 1 --2- 172.21.15.119:0/3858906376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e0004800 0x7f17e0004c80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f17e400c110 tx=0x7f17e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4178dcebc6c7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.391+0000 7f17fb400640 1 -- 172.21.15.119:0/3858906376 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e4007c40 con 0x7f17e0004800 2024-04-25T10:07:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.391+0000 7f17fb400640 1 -- 172.21.15.119:0/3858906376 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e4005af0 con 0x7f17e0004800 2024-04-25T10:07:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.391+0000 7f17fb400640 1 -- 172.21.15.119:0/3858906376 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e4007070 con 0x7f17e0004800 2024-04-25T10:07:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.391+0000 7f1801a00640 1 -- 172.21.15.119:0/3858906376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e0004800 msgr2=0x7f17e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.391+0000 7f1801a00640 1 --2- 172.21.15.119:0/3858906376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e0004800 0x7f17e0004c80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f17e400c110 tx=0x7f17e0005830 comp rx=0 tx=0).stop 2024-04-25T10:07:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.391+0000 7f1801a00640 1 -- 172.21.15.119:0/3858906376 shutdown_connections 2024-04-25T10:07:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.391+0000 7f1801a00640 1 --2- 172.21.15.119:0/3858906376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e0004800 0x7f17e0004c80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.391+0000 7f1801a00640 1 -- 172.21.15.119:0/3858906376 >> 172.21.15.119:0/3858906376 conn(0x7f17fc06b1d0 msgr2=0x7f17fc0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.391+0000 7f1801a00640 1 -- 172.21.15.119:0/3858906376 shutdown_connections 2024-04-25T10:07:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.391+0000 7f1801a00640 1 -- 172.21.15.119:0/3858906376 wait complete. 2024-04-25T10:07:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.393+0000 7f1801a00640 1 Processor -- start 2024-04-25T10:07:44.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.393+0000 7f1801a00640 1 -- start start 2024-04-25T10:07:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.394+0000 7f1801a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e0004800 0x7f17fc13e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.394+0000 7f1801a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17fc138e90 0x7f17fc0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.394+0000 7f1801a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17fc15cd10 0x7f17fc15d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.394+0000 7f1801a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc0b4900 con 0x7f17fc138e90 2024-04-25T10:07:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.394+0000 7f1801a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc0b4620 con 0x7f17fc15cd10 2024-04-25T10:07:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.394+0000 7f1801a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc0b5180 con 0x7f17e0004800 2024-04-25T10:07:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.395+0000 7f17f3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17fc138e90 0x7f17fc0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.395+0000 7f17f3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17fc138e90 0x7f17fc0ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59014/0 (socket says 172.21.15.119:59014) 2024-04-25T10:07:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.395+0000 7f17f3e00640 1 -- 172.21.15.119:0/3153070542 learned_addr learned my addr 172.21.15.119:0/3153070542 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.395+0000 7f1801000640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17fc15cd10 0x7f17fc15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.395+0000 7f17fbe00640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e0004800 0x7f17fc13e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.395+0000 7f17f3e00640 1 -- 172.21.15.119:0/3153070542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e0004800 msgr2=0x7f17fc13e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.395+0000 7f17f3e00640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e0004800 0x7f17fc13e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.395+0000 7f17f3e00640 1 -- 172.21.15.119:0/3153070542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17fc15cd10 msgr2=0x7f17fc15d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.395+0000 7f17f3e00640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17fc15cd10 0x7f17fc15d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.395+0000 7f17f3e00640 1 -- 172.21.15.119:0/3153070542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc0acdf0 con 0x7f17fc138e90 2024-04-25T10:07:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.395+0000 7f17fbe00640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e0004800 0x7f17fc13e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.395+0000 7f17f3e00640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17fc138e90 0x7f17fc0ac8b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f17e400c230 tx=0x7f17fc0b4230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.396+0000 7f17f8c00640 1 -- 172.21.15.119:0/3153070542 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e4037040 con 0x7f17fc138e90 2024-04-25T10:07:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.396+0000 7f17f8c00640 1 -- 172.21.15.119:0/3153070542 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e4041070 con 0x7f17fc138e90 2024-04-25T10:07:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.396+0000 7f17f8c00640 1 -- 172.21.15.119:0/3153070542 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e40047e0 con 0x7f17fc138e90 2024-04-25T10:07:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.396+0000 7f1801a00640 1 -- 172.21.15.119:0/3153070542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17fc0ad090 con 0x7f17fc138e90 2024-04-25T10:07:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.396+0000 7f1801a00640 1 -- 172.21.15.119:0/3153070542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17fc0ad650 con 0x7f17fc138e90 2024-04-25T10:07:44.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.397+0000 7f1801a00640 1 -- 172.21.15.119:0/3153070542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17fc06bb30 con 0x7f17fc138e90 2024-04-25T10:07:44.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.398+0000 7f17f8c00640 1 -- 172.21.15.119:0/3153070542 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17e40049a0 con 0x7f17fc138e90 2024-04-25T10:07:44.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.399+0000 7f17f8c00640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f17d806b500 0x7f17d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.399+0000 7f17f8c00640 1 -- 172.21.15.119:0/3153070542 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17e40aa090 con 0x7f17fc138e90 2024-04-25T10:07:44.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.399+0000 7f17fbe00640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f17d806b500 0x7f17d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.400+0000 7f17fbe00640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f17d806b500 0x7f17d806d9c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f17ec000c00 tx=0x7f17ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.403+0000 7f17f8c00640 1 -- 172.21.15.119:0/3153070542 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17e4078df0 con 0x7f17fc138e90 2024-04-25T10:07:44.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.599+0000 7f1801a00640 1 -- 172.21.15.119:0/3153070542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17fc0b17b0 con 0x7f17fc138e90 2024-04-25T10:07:44.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.602+0000 7f17f8c00640 1 -- 172.21.15.119:0/3153070542 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f17e40786f0 con 0x7f17fc138e90 2024-04-25T10:07:44.608 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:44.608 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:08.319104+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:44.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.604+0000 7f1801a00640 1 -- 172.21.15.119:0/3153070542 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f17d806b500 msgr2=0x7f17d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.604+0000 7f1801a00640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f17d806b500 0x7f17d806d9c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f17ec000c00 tx=0x7f17ec009040 comp rx=0 tx=0).stop 2024-04-25T10:07:44.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.604+0000 7f1801a00640 1 -- 172.21.15.119:0/3153070542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17fc138e90 msgr2=0x7f17fc0ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.604+0000 7f1801a00640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17fc138e90 0x7f17fc0ac8b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f17e400c230 tx=0x7f17fc0b4230 comp rx=0 tx=0).stop 2024-04-25T10:07:44.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.605+0000 7f1801a00640 1 -- 172.21.15.119:0/3153070542 shutdown_connections 2024-04-25T10:07:44.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.605+0000 7f1801a00640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f17d806b500 0x7f17d806d9c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.605+0000 7f1801a00640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17fc15cd10 0x7f17fc15d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.605+0000 7f1801a00640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17fc138e90 0x7f17fc0ac8b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.605+0000 7f1801a00640 1 --2- 172.21.15.119:0/3153070542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e0004800 0x7f17fc13e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.605+0000 7f1801a00640 1 -- 172.21.15.119:0/3153070542 >> 172.21.15.119:0/3153070542 conn(0x7f17fc06b1d0 msgr2=0x7f17fc0b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:44.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.605+0000 7f1801a00640 1 -- 172.21.15.119:0/3153070542 shutdown_connections 2024-04-25T10:07:44.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.605+0000 7f1801a00640 1 -- 172.21.15.119:0/3153070542 wait complete. 2024-04-25T10:07:44.629 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:05.135832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:05.768083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:44.630 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:44.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.815+0000 7f583a800640 1 Processor -- start 2024-04-25T10:07:44.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.816+0000 7f583a800640 1 -- start start 2024-04-25T10:07:44.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.817+0000 7f583a800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f583415b7e0 0x7f583415e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.817+0000 7f583a800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f58340ce860 0x7f58340cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.817+0000 7f583a800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58340b5710 con 0x7f58340d00a0 2024-04-25T10:07:44.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.817+0000 7f583a800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58340b59f0 con 0x7f583415b7e0 2024-04-25T10:07:44.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.817+0000 7f583a800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58340b5cd0 con 0x7f58340ce860 2024-04-25T10:07:44.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.817+0000 7f5839400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f583415b7e0 0x7f583415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.818+0000 7f5839e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f58340d00a0 0x7f58340d0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36710/0 (socket says 172.21.15.119:36710) 2024-04-25T10:07:44.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.818+0000 7f5839e00640 1 -- 172.21.15.119:0/503762068 learned_addr learned my addr 172.21.15.119:0/503762068 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.818+0000 7f5838a00640 1 --2- 172.21.15.119:0/503762068 >> v2:172.21.15.132:3300/0 conn(0x7f58340ce860 0x7f58340cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.818+0000 7f5833e00640 1 -- 172.21.15.119:0/503762068 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042944191 0 0) 0x7f58340b5710 con 0x7f58340d00a0 2024-04-25T10:07:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.818+0000 7f5839400640 1 -- 172.21.15.119:0/503762068 >> v2:172.21.15.132:3300/0 conn(0x7f58340ce860 msgr2=0x7f58340cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.818+0000 7f5839400640 1 --2- 172.21.15.119:0/503762068 >> v2:172.21.15.132:3300/0 conn(0x7f58340ce860 0x7f58340cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.818+0000 7f5839400640 1 -- 172.21.15.119:0/503762068 >> v1:172.21.15.119:6789/0 conn(0x7f58340d00a0 legacy=0x7f58340d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.818+0000 7f5839400640 1 -- 172.21.15.119:0/503762068 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58340b5fb0 con 0x7f583415b7e0 2024-04-25T10:07:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.819+0000 7f5839400640 1 --2- 172.21.15.119:0/503762068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f583415b7e0 0x7f583415e000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f58340b64d0 tx=0x7f582802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66cfb6ee02df7ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.819+0000 7f5833e00640 1 -- 172.21.15.119:0/503762068 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828037070 con 0x7f583415b7e0 2024-04-25T10:07:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.819+0000 7f5833e00640 1 -- 172.21.15.119:0/503762068 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582802add0 con 0x7f583415b7e0 2024-04-25T10:07:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.819+0000 7f5833e00640 1 -- 172.21.15.119:0/503762068 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828032660 con 0x7f583415b7e0 2024-04-25T10:07:44.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.820+0000 7f583a800640 1 -- 172.21.15.119:0/503762068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f583415b7e0 msgr2=0x7f583415e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.820+0000 7f583a800640 1 --2- 172.21.15.119:0/503762068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f583415b7e0 0x7f583415e000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f58340b64d0 tx=0x7f582802a080 comp rx=0 tx=0).stop 2024-04-25T10:07:44.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.820+0000 7f583a800640 1 -- 172.21.15.119:0/503762068 shutdown_connections 2024-04-25T10:07:44.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.820+0000 7f583a800640 1 --2- 172.21.15.119:0/503762068 >> v2:172.21.15.132:3300/0 conn(0x7f58340ce860 0x7f58340cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.820+0000 7f583a800640 1 --2- 172.21.15.119:0/503762068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f583415b7e0 0x7f583415e000 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.820+0000 7f583a800640 1 -- 172.21.15.119:0/503762068 >> 172.21.15.119:0/503762068 conn(0x7f58340722d0 msgr2=0x7f58340726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:44.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.820+0000 7f583a800640 1 -- 172.21.15.119:0/503762068 shutdown_connections 2024-04-25T10:07:44.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.820+0000 7f583a800640 1 -- 172.21.15.119:0/503762068 wait complete. 2024-04-25T10:07:44.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.822+0000 7f583a800640 1 Processor -- start 2024-04-25T10:07:44.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.822+0000 7f583a800640 1 -- start start 2024-04-25T10:07:44.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.823+0000 7f583a800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58340ce860 0x7f58340ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.823+0000 7f583a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58340d00a0 0x7f58341f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.823+0000 7f583a800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f583415b7e0 0x7f58340cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.823+0000 7f583a800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5834161960 con 0x7f58340d00a0 2024-04-25T10:07:44.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.823+0000 7f583a800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58340b53c0 con 0x7f58340ce860 2024-04-25T10:07:44.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.823+0000 7f583a800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5834161ae0 con 0x7f583415b7e0 2024-04-25T10:07:44.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.823+0000 7f5838a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58340d00a0 0x7f58341f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.823+0000 7f5838a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58340d00a0 0x7f58341f6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59028/0 (socket says 172.21.15.119:59028) 2024-04-25T10:07:44.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.823+0000 7f5838a00640 1 -- 172.21.15.119:0/904008423 learned_addr learned my addr 172.21.15.119:0/904008423 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:44.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.824+0000 7f5838a00640 1 -- 172.21.15.119:0/904008423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f583415b7e0 msgr2=0x7f58340cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.824+0000 7f5838a00640 1 --2- 172.21.15.119:0/904008423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f583415b7e0 0x7f58340cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.824+0000 7f5839400640 1 --2- 172.21.15.119:0/904008423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58340ce860 0x7f58340ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.824+0000 7f5838a00640 1 -- 172.21.15.119:0/904008423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58340ce860 msgr2=0x7f58340ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.824+0000 7f5838a00640 1 --2- 172.21.15.119:0/904008423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58340ce860 0x7f58340ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.824+0000 7f5838a00640 1 -- 172.21.15.119:0/904008423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58340cc570 con 0x7f58340d00a0 2024-04-25T10:07:44.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.824+0000 7f5839400640 1 --2- 172.21.15.119:0/904008423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58340ce860 0x7f58340ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:44.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.824+0000 7f5838a00640 1 --2- 172.21.15.119:0/904008423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58340d00a0 0x7f58341f6ea0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f582000d800 tx=0x7f582000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.824+0000 7f5831600640 1 -- 172.21.15.119:0/904008423 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000b6c0 con 0x7f58340d00a0 2024-04-25T10:07:44.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.824+0000 7f5831600640 1 -- 172.21.15.119:0/904008423 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5820019070 con 0x7f58340d00a0 2024-04-25T10:07:44.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.824+0000 7f5831600640 1 -- 172.21.15.119:0/904008423 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000ba80 con 0x7f58340d00a0 2024-04-25T10:07:44.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.824+0000 7f583a800640 1 -- 172.21.15.119:0/904008423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58341f7880 con 0x7f58340d00a0 2024-04-25T10:07:44.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.824+0000 7f583a800640 1 -- 172.21.15.119:0/904008423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58340cc810 con 0x7f58340d00a0 2024-04-25T10:07:44.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.825+0000 7f583a800640 1 -- 172.21.15.119:0/904008423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58341f7da0 con 0x7f58340d00a0 2024-04-25T10:07:44.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.826+0000 7f5831600640 1 -- 172.21.15.119:0/904008423 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f582000bc20 con 0x7f58340d00a0 2024-04-25T10:07:44.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.827+0000 7f5831600640 1 --2- 172.21.15.119:0/904008423 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f581406b4d0 0x7f581406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.827+0000 7f5831600640 1 -- 172.21.15.119:0/904008423 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f582008a620 con 0x7f58340d00a0 2024-04-25T10:07:44.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.828+0000 7f5839400640 1 --2- 172.21.15.119:0/904008423 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f581406b4d0 0x7f581406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.828+0000 7f5839400640 1 --2- 172.21.15.119:0/904008423 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f581406b4d0 0x7f581406d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5828034000 tx=0x7f582802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:44.831+0000 7f5831600640 1 -- 172.21.15.119:0/904008423 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58341f7da0 con 0x7f58340d00a0 2024-04-25T10:07:45.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.028+0000 7f583a800640 1 -- 172.21.15.119:0/904008423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58340d27d0 con 0x7f58340d00a0 2024-04-25T10:07:45.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.029+0000 7f5831600640 1 -- 172.21.15.119:0/904008423 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5820059530 con 0x7f58340d00a0 2024-04-25T10:07:45.035 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:07:45.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.032+0000 7f583a800640 1 -- 172.21.15.119:0/904008423 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f581406b4d0 msgr2=0x7f581406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.032+0000 7f583a800640 1 --2- 172.21.15.119:0/904008423 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f581406b4d0 0x7f581406d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5828034000 tx=0x7f582802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:45.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.032+0000 7f583a800640 1 -- 172.21.15.119:0/904008423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58340d00a0 msgr2=0x7f58341f6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.032+0000 7f583a800640 1 --2- 172.21.15.119:0/904008423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58340d00a0 0x7f58341f6ea0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f582000d800 tx=0x7f582000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.032+0000 7f583a800640 1 -- 172.21.15.119:0/904008423 shutdown_connections 2024-04-25T10:07:45.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.032+0000 7f583a800640 1 --2- 172.21.15.119:0/904008423 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f581406b4d0 0x7f581406d990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.032+0000 7f583a800640 1 --2- 172.21.15.119:0/904008423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f583415b7e0 0x7f58340cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.032+0000 7f583a800640 1 --2- 172.21.15.119:0/904008423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58340d00a0 0x7f58341f6ea0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.032+0000 7f583a800640 1 --2- 172.21.15.119:0/904008423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58340ce860 0x7f58340ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.032+0000 7f583a800640 1 -- 172.21.15.119:0/904008423 >> 172.21.15.119:0/904008423 conn(0x7f58340722d0 msgr2=0x7f5834074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:45.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.032+0000 7f583a800640 1 -- 172.21.15.119:0/904008423 shutdown_connections 2024-04-25T10:07:45.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.032+0000 7f583a800640 1 -- 172.21.15.119:0/904008423 wait complete. 2024-04-25T10:07:45.057 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:45.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.257+0000 7f4fe8a00640 1 Processor -- start 2024-04-25T10:07:45.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.258+0000 7f4fe8a00640 1 -- start start 2024-04-25T10:07:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.259+0000 7f4fe8a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4fe415b7e0 0x7f4fe415e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.259+0000 7f4fe8a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4fe40d07b0 0x7f4fe40cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.259+0000 7f4fe8a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe40b5710 con 0x7f4fe40ce860 2024-04-25T10:07:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.259+0000 7f4fe8a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe40b59f0 con 0x7f4fe415b7e0 2024-04-25T10:07:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.259+0000 7f4fe8a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe40b5cd0 con 0x7f4fe40d07b0 2024-04-25T10:07:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.259+0000 7f4fe3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4fe415b7e0 0x7f4fe415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.259+0000 7f4fe3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4fe415b7e0 0x7f4fe415e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59034/0 (socket says 172.21.15.119:59034) 2024-04-25T10:07:45.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.259+0000 7f4fe3400640 1 -- 172.21.15.119:0/1422015437 learned_addr learned my addr 172.21.15.119:0/1422015437 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:45.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.260+0000 7f4fe3400640 1 -- 172.21.15.119:0/1422015437 >> v2:172.21.15.132:3300/0 conn(0x7f4fe40d07b0 msgr2=0x7f4fe40cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:45.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.260+0000 7f4fe3e00640 1 --2- 172.21.15.119:0/1422015437 >> v2:172.21.15.132:3300/0 conn(0x7f4fe40d07b0 0x7f4fe40cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.260+0000 7f4fe3400640 1 --2- 172.21.15.119:0/1422015437 >> v2:172.21.15.132:3300/0 conn(0x7f4fe40d07b0 0x7f4fe40cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.260+0000 7f4fe3400640 1 -- 172.21.15.119:0/1422015437 >> v1:172.21.15.132:6789/0 conn(0x7f4fe40ce860 legacy=0x7f4fe40d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:45.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.260+0000 7f4fe3400640 1 -- 172.21.15.119:0/1422015437 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe40b5fb0 con 0x7f4fe415b7e0 2024-04-25T10:07:45.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.260+0000 7f4fe3e00640 1 --2- 172.21.15.119:0/1422015437 >> v2:172.21.15.132:3300/0 conn(0x7f4fe40d07b0 0x7f4fe40cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:45.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.261+0000 7f4fe3400640 1 --2- 172.21.15.119:0/1422015437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe415b7e0 0x7f4fe415e000 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4fe408bf40 tx=0x7f4fd402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a6fa9f2c857b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:45.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.262+0000 7f4fe2000640 1 -- 172.21.15.119:0/1422015437 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4037070 con 0x7f4fe415b7e0 2024-04-25T10:07:45.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.262+0000 7f4fe2000640 1 -- 172.21.15.119:0/1422015437 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd402ae20 con 0x7f4fe415b7e0 2024-04-25T10:07:45.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.262+0000 7f4fe2000640 1 -- 172.21.15.119:0/1422015437 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4032660 con 0x7f4fe415b7e0 2024-04-25T10:07:45.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.262+0000 7f4fe8a00640 1 -- 172.21.15.119:0/1422015437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe415b7e0 msgr2=0x7f4fe415e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.262+0000 7f4fe8a00640 1 --2- 172.21.15.119:0/1422015437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe415b7e0 0x7f4fe415e000 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4fe408bf40 tx=0x7f4fd402a110 comp rx=0 tx=0).stop 2024-04-25T10:07:45.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.262+0000 7f4fe8a00640 1 -- 172.21.15.119:0/1422015437 shutdown_connections 2024-04-25T10:07:45.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.262+0000 7f4fe8a00640 1 --2- 172.21.15.119:0/1422015437 >> v2:172.21.15.132:3300/0 conn(0x7f4fe40d07b0 0x7f4fe40cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.262+0000 7f4fe8a00640 1 --2- 172.21.15.119:0/1422015437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe415b7e0 0x7f4fe415e000 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.262+0000 7f4fe8a00640 1 -- 172.21.15.119:0/1422015437 >> 172.21.15.119:0/1422015437 conn(0x7f4fe40722d0 msgr2=0x7f4fe40726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:45.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.263+0000 7f4fe8a00640 1 -- 172.21.15.119:0/1422015437 shutdown_connections 2024-04-25T10:07:45.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.263+0000 7f4fe8a00640 1 -- 172.21.15.119:0/1422015437 wait complete. 2024-04-25T10:07:45.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.264+0000 7f4fe8a00640 1 Processor -- start 2024-04-25T10:07:45.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.265+0000 7f4fe8a00640 1 -- start start 2024-04-25T10:07:45.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.265+0000 7f4fe8a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fe40ce860 0x7f4fe40ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.265+0000 7f4fe8a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe40d07b0 0x7f4fe40cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.265+0000 7f4fe8a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fe415b7e0 0x7f4fe40cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.265+0000 7f4fe8a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4161960 con 0x7f4fe40d07b0 2024-04-25T10:07:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.265+0000 7f4fe8a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40b53c0 con 0x7f4fe415b7e0 2024-04-25T10:07:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.265+0000 7f4fe8a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4161ae0 con 0x7f4fe40ce860 2024-04-25T10:07:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.266+0000 7f4fe2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe40d07b0 0x7f4fe40cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.266+0000 7f4fe2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe40d07b0 0x7f4fe40cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59046/0 (socket says 172.21.15.119:59046) 2024-04-25T10:07:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.266+0000 7f4fe2a00640 1 -- 172.21.15.119:0/567177022 learned_addr learned my addr 172.21.15.119:0/567177022 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.266+0000 7f4fe3e00640 1 --2- 172.21.15.119:0/567177022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fe415b7e0 0x7f4fe40cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.266+0000 7f4fe3400640 1 --2- 172.21.15.119:0/567177022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fe40ce860 0x7f4fe40ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.266+0000 7f4fe2a00640 1 -- 172.21.15.119:0/567177022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fe40ce860 msgr2=0x7f4fe40ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.266+0000 7f4fe2a00640 1 --2- 172.21.15.119:0/567177022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fe40ce860 0x7f4fe40ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.266+0000 7f4fe2a00640 1 -- 172.21.15.119:0/567177022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fe415b7e0 msgr2=0x7f4fe40cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.266+0000 7f4fe2a00640 1 --2- 172.21.15.119:0/567177022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fe415b7e0 0x7f4fe40cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.266+0000 7f4fe2a00640 1 -- 172.21.15.119:0/567177022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe415fbb0 con 0x7f4fe40d07b0 2024-04-25T10:07:45.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.267+0000 7f4fe2a00640 1 --2- 172.21.15.119:0/567177022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe40d07b0 0x7f4fe40cbd90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000c430 tx=0x7f4fd000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:45.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.267+0000 7f4fdb400640 1 -- 172.21.15.119:0/567177022 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0011790 con 0x7f4fe40d07b0 2024-04-25T10:07:45.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.267+0000 7f4fdb400640 1 -- 172.21.15.119:0/567177022 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd0017070 con 0x7f4fe40d07b0 2024-04-25T10:07:45.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.267+0000 7f4fdb400640 1 -- 172.21.15.119:0/567177022 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0011b70 con 0x7f4fe40d07b0 2024-04-25T10:07:45.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.267+0000 7f4fe8a00640 1 -- 172.21.15.119:0/567177022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe415fe90 con 0x7f4fe40d07b0 2024-04-25T10:07:45.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.267+0000 7f4fe8a00640 1 -- 172.21.15.119:0/567177022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe4160450 con 0x7f4fe40d07b0 2024-04-25T10:07:45.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.268+0000 7f4fe8a00640 1 -- 172.21.15.119:0/567177022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe40749b0 con 0x7f4fe40d07b0 2024-04-25T10:07:45.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.269+0000 7f4fdb400640 1 -- 172.21.15.119:0/567177022 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fd0011d10 con 0x7f4fe40d07b0 2024-04-25T10:07:45.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.270+0000 7f4fdb400640 1 --2- 172.21.15.119:0/567177022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fc406b4d0 0x7f4fc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.270+0000 7f4fdb400640 1 -- 172.21.15.119:0/567177022 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fd008a160 con 0x7f4fe40d07b0 2024-04-25T10:07:45.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.270+0000 7f4fe3400640 1 --2- 172.21.15.119:0/567177022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fc406b4d0 0x7f4fc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.271+0000 7f4fe3400640 1 --2- 172.21.15.119:0/567177022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fc406b4d0 0x7f4fc406d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4034000 tx=0x7f4fd402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:45.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.274+0000 7f4fdb400640 1 -- 172.21.15.119:0/567177022 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fd0058f40 con 0x7f4fe40d07b0 2024-04-25T10:07:45.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.470+0000 7f4fe8a00640 1 -- 172.21.15.119:0/567177022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4fe40b8ad0 con 0x7f4fe40d07b0 2024-04-25T10:07:45.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.470+0000 7f4fdb400640 1 -- 172.21.15.119:0/567177022 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4fd0058840 con 0x7f4fe40d07b0 2024-04-25T10:07:45.477 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:07:45.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.473+0000 7f4fe8a00640 1 -- 172.21.15.119:0/567177022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fc406b4d0 msgr2=0x7f4fc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.473+0000 7f4fe8a00640 1 --2- 172.21.15.119:0/567177022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fc406b4d0 0x7f4fc406d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4034000 tx=0x7f4fd402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.473+0000 7f4fe8a00640 1 -- 172.21.15.119:0/567177022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe40d07b0 msgr2=0x7f4fe40cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.473+0000 7f4fe8a00640 1 --2- 172.21.15.119:0/567177022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe40d07b0 0x7f4fe40cbd90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000c430 tx=0x7f4fd000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.473+0000 7f4fe8a00640 1 -- 172.21.15.119:0/567177022 shutdown_connections 2024-04-25T10:07:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.473+0000 7f4fe8a00640 1 --2- 172.21.15.119:0/567177022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fc406b4d0 0x7f4fc406d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.473+0000 7f4fe8a00640 1 --2- 172.21.15.119:0/567177022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fe415b7e0 0x7f4fe40cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.473+0000 7f4fe8a00640 1 --2- 172.21.15.119:0/567177022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe40d07b0 0x7f4fe40cbd90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.473+0000 7f4fe8a00640 1 --2- 172.21.15.119:0/567177022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fe40ce860 0x7f4fe40ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.474+0000 7f4fe8a00640 1 -- 172.21.15.119:0/567177022 >> 172.21.15.119:0/567177022 conn(0x7f4fe40722d0 msgr2=0x7f4fe4150980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.474+0000 7f4fe8a00640 1 -- 172.21.15.119:0/567177022 shutdown_connections 2024-04-25T10:07:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.474+0000 7f4fe8a00640 1 -- 172.21.15.119:0/567177022 wait complete. 2024-04-25T10:07:45.498 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:45.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.695+0000 7f5d88c00640 1 Processor -- start 2024-04-25T10:07:45.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.695+0000 7f5d88c00640 1 -- start start 2024-04-25T10:07:45.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.697+0000 7f5d88c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5d840aed40 0x7f5d840cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.697+0000 7f5d88c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d840cfa90 0x7f5d840acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.697+0000 7f5d88c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8415e360 con 0x7f5d8415b720 2024-04-25T10:07:45.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.697+0000 7f5d88c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8415e5d0 con 0x7f5d840aed40 2024-04-25T10:07:45.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.697+0000 7f5d88c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8415e8b0 con 0x7f5d840cfa90 2024-04-25T10:07:45.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.697+0000 7f5d81400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5d840aed40 0x7f5d840cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.697+0000 7f5d81400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5d840aed40 0x7f5d840cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59060/0 (socket says 172.21.15.119:59060) 2024-04-25T10:07:45.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.697+0000 7f5d81400640 1 -- 172.21.15.119:0/1260283002 learned_addr learned my addr 172.21.15.119:0/1260283002 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:45.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.698+0000 7f5d81400640 1 -- 172.21.15.119:0/1260283002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d840cfa90 msgr2=0x7f5d840acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:45.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.698+0000 7f5d81400640 1 --2- 172.21.15.119:0/1260283002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d840cfa90 0x7f5d840acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.698+0000 7f5d81400640 1 -- 172.21.15.119:0/1260283002 >> v1:172.21.15.132:6789/0 conn(0x7f5d8415b720 legacy=0x7f5d840ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:45.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.698+0000 7f5d81400640 1 -- 172.21.15.119:0/1260283002 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8415eb90 con 0x7f5d840aed40 2024-04-25T10:07:45.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.699+0000 7f5d81400640 1 --2- 172.21.15.119:0/1260283002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d840aed40 0x7f5d840cf550 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009870 tx=0x7f5d7002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4390e62aaf3951 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:45.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.699+0000 7f5d80a00640 1 -- 172.21.15.119:0/1260283002 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70037070 con 0x7f5d840aed40 2024-04-25T10:07:45.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.699+0000 7f5d80a00640 1 -- 172.21.15.119:0/1260283002 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d70032440 con 0x7f5d840aed40 2024-04-25T10:07:45.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.699+0000 7f5d80a00640 1 -- 172.21.15.119:0/1260283002 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70032810 con 0x7f5d840aed40 2024-04-25T10:07:45.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.700+0000 7f5d88c00640 1 -- 172.21.15.119:0/1260283002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d840aed40 msgr2=0x7f5d840cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.700+0000 7f5d88c00640 1 --2- 172.21.15.119:0/1260283002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d840aed40 0x7f5d840cf550 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009870 tx=0x7f5d7002a210 comp rx=0 tx=0).stop 2024-04-25T10:07:45.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.700+0000 7f5d88c00640 1 -- 172.21.15.119:0/1260283002 shutdown_connections 2024-04-25T10:07:45.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.700+0000 7f5d88c00640 1 --2- 172.21.15.119:0/1260283002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d840cfa90 0x7f5d840acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.700+0000 7f5d88c00640 1 --2- 172.21.15.119:0/1260283002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d840aed40 0x7f5d840cf550 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.700+0000 7f5d88c00640 1 -- 172.21.15.119:0/1260283002 >> 172.21.15.119:0/1260283002 conn(0x7f5d8406aba0 msgr2=0x7f5d8406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:45.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.700+0000 7f5d88c00640 1 -- 172.21.15.119:0/1260283002 shutdown_connections 2024-04-25T10:07:45.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.700+0000 7f5d88c00640 1 -- 172.21.15.119:0/1260283002 wait complete. 2024-04-25T10:07:45.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.701+0000 7f5d88c00640 1 Processor -- start 2024-04-25T10:07:45.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.702+0000 7f5d88c00640 1 -- start start 2024-04-25T10:07:45.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.702+0000 7f5d88c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d840aed40 0x7f5d841f7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.702+0000 7f5d88c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d840cfa90 0x7f5d84158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.702+0000 7f5d88c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d8415b720 0x7f5d8413e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.703+0000 7f5d88c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d840cfed0 con 0x7f5d840aed40 2024-04-25T10:07:45.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.703+0000 7f5d88c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8415bb00 con 0x7f5d840cfa90 2024-04-25T10:07:45.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.703+0000 7f5d88c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d840d0050 con 0x7f5d8415b720 2024-04-25T10:07:45.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.703+0000 7f5d81e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d840aed40 0x7f5d841f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.703+0000 7f5d81e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d840aed40 0x7f5d841f7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59076/0 (socket says 172.21.15.119:59076) 2024-04-25T10:07:45.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.703+0000 7f5d81e00640 1 -- 172.21.15.119:0/2062152348 learned_addr learned my addr 172.21.15.119:0/2062152348 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:45.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.703+0000 7f5d82800640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d8415b720 0x7f5d8413e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.703+0000 7f5d81400640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d840cfa90 0x7f5d84158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.703+0000 7f5d81e00640 1 -- 172.21.15.119:0/2062152348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d8415b720 msgr2=0x7f5d8413e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.703+0000 7f5d81e00640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d8415b720 0x7f5d8413e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.703+0000 7f5d81e00640 1 -- 172.21.15.119:0/2062152348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d840cfa90 msgr2=0x7f5d84158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.703+0000 7f5d81e00640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d840cfa90 0x7f5d84158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.703+0000 7f5d81e00640 1 -- 172.21.15.119:0/2062152348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d84159460 con 0x7f5d840aed40 2024-04-25T10:07:45.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.703+0000 7f5d81400640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d840cfa90 0x7f5d84158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:45.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.704+0000 7f5d81e00640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d840aed40 0x7f5d841f7810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400b420 tx=0x7f5d7400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:45.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.704+0000 7f5d7a000640 1 -- 172.21.15.119:0/2062152348 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74011830 con 0x7f5d840aed40 2024-04-25T10:07:45.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.704+0000 7f5d7a000640 1 -- 172.21.15.119:0/2062152348 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d74016070 con 0x7f5d840aed40 2024-04-25T10:07:45.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.704+0000 7f5d88c00640 1 -- 172.21.15.119:0/2062152348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d84159740 con 0x7f5d840aed40 2024-04-25T10:07:45.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.704+0000 7f5d7a000640 1 -- 172.21.15.119:0/2062152348 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74011c10 con 0x7f5d840aed40 2024-04-25T10:07:45.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.704+0000 7f5d88c00640 1 -- 172.21.15.119:0/2062152348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8413e690 con 0x7f5d840aed40 2024-04-25T10:07:45.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.705+0000 7f5d88c00640 1 -- 172.21.15.119:0/2062152348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d840750b0 con 0x7f5d840aed40 2024-04-25T10:07:45.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.706+0000 7f5d7a000640 1 -- 172.21.15.119:0/2062152348 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d7401b460 con 0x7f5d840aed40 2024-04-25T10:07:45.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.707+0000 7f5d7a000640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5d6406b500 0x7f5d6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:45.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.707+0000 7f5d7a000640 1 -- 172.21.15.119:0/2062152348 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d74089d10 con 0x7f5d840aed40 2024-04-25T10:07:45.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.707+0000 7f5d81400640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5d6406b500 0x7f5d6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:45.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.708+0000 7f5d81400640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5d6406b500 0x7f5d6406d9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d70034000 tx=0x7f5d7002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:45.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.711+0000 7f5d7a000640 1 -- 172.21.15.119:0/2062152348 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d7408f020 con 0x7f5d840aed40 2024-04-25T10:07:45.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.907+0000 7f5d88c00640 1 -- 172.21.15.119:0/2062152348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d840d0300 con 0x7f5d840aed40 2024-04-25T10:07:45.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.907+0000 7f5d7a000640 1 -- 172.21.15.119:0/2062152348 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5d74020020 con 0x7f5d840aed40 2024-04-25T10:07:45.913 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:07:45.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.910+0000 7f5d88c00640 1 -- 172.21.15.119:0/2062152348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5d6406b500 msgr2=0x7f5d6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.910+0000 7f5d88c00640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5d6406b500 0x7f5d6406d9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d70034000 tx=0x7f5d7002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:45.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.910+0000 7f5d88c00640 1 -- 172.21.15.119:0/2062152348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d840aed40 msgr2=0x7f5d841f7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:45.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.910+0000 7f5d88c00640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d840aed40 0x7f5d841f7810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400b420 tx=0x7f5d7400b8e0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.910+0000 7f5d88c00640 1 -- 172.21.15.119:0/2062152348 shutdown_connections 2024-04-25T10:07:45.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.910+0000 7f5d88c00640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5d6406b500 0x7f5d6406d9c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.910+0000 7f5d88c00640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5d8415b720 0x7f5d8413e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.910+0000 7f5d88c00640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d840cfa90 0x7f5d84158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.910+0000 7f5d88c00640 1 --2- 172.21.15.119:0/2062152348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d840aed40 0x7f5d841f7810 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:45.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.910+0000 7f5d88c00640 1 -- 172.21.15.119:0/2062152348 >> 172.21.15.119:0/2062152348 conn(0x7f5d8406aba0 msgr2=0x7f5d8415bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:45.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.911+0000 7f5d88c00640 1 -- 172.21.15.119:0/2062152348 shutdown_connections 2024-04-25T10:07:45.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:45.911+0000 7f5d88c00640 1 -- 172.21.15.119:0/2062152348 wait complete. 2024-04-25T10:07:45.936 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T10:07:46.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.125+0000 7fe41f800640 1 Processor -- start 2024-04-25T10:07:46.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.125+0000 7fe41f800640 1 -- start start 2024-04-25T10:07:46.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.127+0000 7fe41f800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe41815a870 0x7fe4180cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.127+0000 7fe41f800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4180cdb10 con 0x7fe4180cfc40 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.127+0000 7fe41f800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4180cddf0 con 0x7fe41806e2e0 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.127+0000 7fe41f800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4180ce720 con 0x7fe41815a870 2024-04-25T10:07:46.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.127+0000 7fe41d800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe4180cfc40 0x7fe41815bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36714/0 (socket says 172.21.15.119:36714) 2024-04-25T10:07:46.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.127+0000 7fe41d800640 1 -- 172.21.15.119:0/2360990753 learned_addr learned my addr 172.21.15.119:0/2360990753 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:46.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe41ce00640 1 --2- 172.21.15.119:0/2360990753 >> v2:172.21.15.132:3300/0 conn(0x7fe41815a870 0x7fe4180cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 -- 172.21.15.119:0/2360990753 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157707039 0 0) 0x7fe4180cdb10 con 0x7fe4180cfc40 2024-04-25T10:07:46.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 -- 172.21.15.119:0/2360990753 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe400003520 con 0x7fe4180cfc40 2024-04-25T10:07:46.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 -- 172.21.15.119:0/2360990753 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 28737651 0 0) 0x7fe400003520 con 0x7fe4180cfc40 2024-04-25T10:07:46.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 -- 172.21.15.119:0/2360990753 >> v2:172.21.15.132:3300/0 conn(0x7fe41815a870 msgr2=0x7fe4180cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:46.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 --2- 172.21.15.119:0/2360990753 >> v2:172.21.15.132:3300/0 conn(0x7fe41815a870 0x7fe4180cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 -- 172.21.15.119:0/2360990753 >> v1:172.21.15.132:6789/0 conn(0x7fe41806e2e0 legacy=0x7fe4180cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 -- 172.21.15.119:0/2360990753 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4180ce0d0 con 0x7fe4180cfc40 2024-04-25T10:07:46.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 -- 172.21.15.119:0/2360990753 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fe4100019f0 con 0x7fe4180cfc40 2024-04-25T10:07:46.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 -- 172.21.15.119:0/2360990753 >> v1:172.21.15.119:6789/0 conn(0x7fe4180cfc40 legacy=0x7fe41815bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 --2- 172.21.15.119:0/2360990753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4000039b0 0x7fe400003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 --2- 172.21.15.119:0/2360990753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe400004800 0x7fe400004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 --2- 172.21.15.119:0/2360990753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4000051c0 0x7fe40000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 -- 172.21.15.119:0/2360990753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180ce720 con 0x7fe4000039b0 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 -- 172.21.15.119:0/2360990753 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40000c060 con 0x7fe4000051c0 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe417400640 1 -- 172.21.15.119:0/2360990753 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40000c260 con 0x7fe400004800 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.128+0000 7fe41ce00640 1 --2- 172.21.15.119:0/2360990753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4000039b0 0x7fe400003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.129+0000 7fe41ce00640 1 -- 172.21.15.119:0/2360990753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe400004800 msgr2=0x7fe400004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.129+0000 7fe417e00640 1 --2- 172.21.15.119:0/2360990753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe400004800 0x7fe400004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.129+0000 7fe41ce00640 1 --2- 172.21.15.119:0/2360990753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe400004800 0x7fe400004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.129+0000 7fe41d800640 1 --2- 172.21.15.119:0/2360990753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4000051c0 0x7fe40000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.129+0000 7fe41ce00640 1 -- 172.21.15.119:0/2360990753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4000051c0 msgr2=0x7fe40000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.129+0000 7fe41ce00640 1 --2- 172.21.15.119:0/2360990753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4000051c0 0x7fe40000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.129+0000 7fe41d800640 1 -- 172.21.15.119:0/2360990753 reap_dead start 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.129+0000 7fe41ce00640 1 -- 172.21.15.119:0/2360990753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe400003520 con 0x7fe4000039b0 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.129+0000 7fe417e00640 1 --2- 172.21.15.119:0/2360990753 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe400004800 0x7fe400004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:46.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.130+0000 7fe41ce00640 1 --2- 172.21.15.119:0/2360990753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4000039b0 0x7fe400003dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009ac0 tx=0x7fe400005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10fbc3f7be44fca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:46.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.130+0000 7fe417400640 1 -- 172.21.15.119:0/2360990753 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c036070 con 0x7fe4000039b0 2024-04-25T10:07:46.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.130+0000 7fe417400640 1 -- 172.21.15.119:0/2360990753 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40c031400 con 0x7fe4000039b0 2024-04-25T10:07:46.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.130+0000 7fe417400640 1 -- 172.21.15.119:0/2360990753 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c009690 con 0x7fe4000039b0 2024-04-25T10:07:46.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.131+0000 7fe41f800640 1 -- 172.21.15.119:0/2360990753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4000039b0 msgr2=0x7fe400003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.131+0000 7fe41f800640 1 --2- 172.21.15.119:0/2360990753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4000039b0 0x7fe400003dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009ac0 tx=0x7fe400005830 comp rx=0 tx=0).stop 2024-04-25T10:07:46.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.131+0000 7fe41f800640 1 -- 172.21.15.119:0/2360990753 shutdown_connections 2024-04-25T10:07:46.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.131+0000 7fe41f800640 1 --2- 172.21.15.119:0/2360990753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4000039b0 0x7fe400003dd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.131+0000 7fe41f800640 1 -- 172.21.15.119:0/2360990753 >> 172.21.15.119:0/2360990753 conn(0x7fe41806aba0 msgr2=0x7fe41806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:46.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.131+0000 7fe41f800640 1 -- 172.21.15.119:0/2360990753 shutdown_connections 2024-04-25T10:07:46.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.131+0000 7fe41f800640 1 -- 172.21.15.119:0/2360990753 wait complete. 2024-04-25T10:07:46.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.133+0000 7fe41f800640 1 Processor -- start 2024-04-25T10:07:46.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.133+0000 7fe41f800640 1 -- start start 2024-04-25T10:07:46.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.134+0000 7fe41f800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4000039b0 0x7fe418142010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.134+0000 7fe41f800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe418155390 0x7fe4181557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.134+0000 7fe41f800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4181e2fe0 0x7fe4181e3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.134+0000 7fe41f800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180cd700 con 0x7fe418155390 2024-04-25T10:07:46.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.134+0000 7fe41f800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180cd580 con 0x7fe4000039b0 2024-04-25T10:07:46.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.134+0000 7fe41f800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180cd880 con 0x7fe4181e2fe0 2024-04-25T10:07:46.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.134+0000 7fe417e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe418155390 0x7fe4181557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.134+0000 7fe417e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe418155390 0x7fe4181557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59096/0 (socket says 172.21.15.119:59096) 2024-04-25T10:07:46.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.134+0000 7fe417e00640 1 -- 172.21.15.119:0/1138969401 learned_addr learned my addr 172.21.15.119:0/1138969401 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:46.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.135+0000 7fe417e00640 1 -- 172.21.15.119:0/1138969401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4181e2fe0 msgr2=0x7fe4181e3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.135+0000 7fe41d800640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4181e2fe0 0x7fe4181e3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.135+0000 7fe41ce00640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4000039b0 0x7fe418142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.135+0000 7fe417e00640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4181e2fe0 0x7fe4181e3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.135+0000 7fe417e00640 1 -- 172.21.15.119:0/1138969401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4000039b0 msgr2=0x7fe418142010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.135+0000 7fe417e00640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4000039b0 0x7fe418142010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.135+0000 7fe417e00640 1 -- 172.21.15.119:0/1138969401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe418142550 con 0x7fe418155390 2024-04-25T10:07:46.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.135+0000 7fe41d800640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4181e2fe0 0x7fe4181e3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:46.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.135+0000 7fe417e00640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe418155390 0x7fe4181557f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe40400d4b0 tx=0x7fe40400d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:46.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.135+0000 7fe414c00640 1 -- 172.21.15.119:0/1138969401 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe404005480 con 0x7fe418155390 2024-04-25T10:07:46.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.135+0000 7fe414c00640 1 -- 172.21.15.119:0/1138969401 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe404017070 con 0x7fe418155390 2024-04-25T10:07:46.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.135+0000 7fe414c00640 1 -- 172.21.15.119:0/1138969401 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe404003830 con 0x7fe418155390 2024-04-25T10:07:46.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.135+0000 7fe41f800640 1 -- 172.21.15.119:0/1138969401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe418152ae0 con 0x7fe418155390 2024-04-25T10:07:46.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.136+0000 7fe41f800640 1 -- 172.21.15.119:0/1138969401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4181e3ed0 con 0x7fe418155390 2024-04-25T10:07:46.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.137+0000 7fe41f800640 1 -- 172.21.15.119:0/1138969401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4180749b0 con 0x7fe418155390 2024-04-25T10:07:46.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.138+0000 7fe414c00640 1 -- 172.21.15.119:0/1138969401 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe4040039d0 con 0x7fe418155390 2024-04-25T10:07:46.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.139+0000 7fe414c00640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe3f806b4d0 0x7fe3f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.139+0000 7fe414c00640 1 -- 172.21.15.119:0/1138969401 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe404089f90 con 0x7fe418155390 2024-04-25T10:07:46.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.139+0000 7fe41ce00640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe3f806b4d0 0x7fe3f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.140+0000 7fe41ce00640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe3f806b4d0 0x7fe3f806d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe40c033000 tx=0x7fe40c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:46.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.143+0000 7fe414c00640 1 -- 172.21.15.119:0/1138969401 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe418142550 con 0x7fe418155390 2024-04-25T10:07:46.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.411+0000 7fe41f800640 1 -- 172.21.15.119:0/1138969401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fe4180b0fb0 con 0x7fe418155390 2024-04-25T10:07:46.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.864+0000 7fe414c00640 1 -- 172.21.15.119:0/1138969401 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v62) v1 ==== 145+0+0 (secure 0 0 0) 0x7fe404058ec0 con 0x7fe418155390 2024-04-25T10:07:46.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.867+0000 7fe41f800640 1 -- 172.21.15.119:0/1138969401 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe3f806b4d0 msgr2=0x7fe3f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.867+0000 7fe41f800640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe3f806b4d0 0x7fe3f806d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe40c033000 tx=0x7fe40c008030 comp rx=0 tx=0).stop 2024-04-25T10:07:46.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.867+0000 7fe41f800640 1 -- 172.21.15.119:0/1138969401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe418155390 msgr2=0x7fe4181557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.867+0000 7fe41f800640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe418155390 0x7fe4181557f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe40400d4b0 tx=0x7fe40400d970 comp rx=0 tx=0).stop 2024-04-25T10:07:46.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.867+0000 7fe41f800640 1 -- 172.21.15.119:0/1138969401 shutdown_connections 2024-04-25T10:07:46.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.867+0000 7fe41f800640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe3f806b4d0 0x7fe3f806d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.867+0000 7fe41f800640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4181e2fe0 0x7fe4181e3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.867+0000 7fe41f800640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe418155390 0x7fe4181557f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.867+0000 7fe41f800640 1 --2- 172.21.15.119:0/1138969401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4000039b0 0x7fe418142010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.867+0000 7fe41f800640 1 -- 172.21.15.119:0/1138969401 >> 172.21.15.119:0/1138969401 conn(0x7fe41806aba0 msgr2=0x7fe4180b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:46.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.868+0000 7fe41f800640 1 -- 172.21.15.119:0/1138969401 shutdown_connections 2024-04-25T10:07:46.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:46.868+0000 7fe41f800640 1 -- 172.21.15.119:0/1138969401 wait complete. 2024-04-25T10:07:46.896 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:07:47.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.075+0000 7f30cbe00640 1 Processor -- start 2024-04-25T10:07:47.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.075+0000 7f30cbe00640 1 -- start start 2024-04-25T10:07:47.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.077+0000 7f30cbe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f30cc15b7e0 0x7f30cc15e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.077+0000 7f30cbe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f30cc0ce860 0x7f30cc0cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.077+0000 7f30cbe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30cc0b5710 con 0x7f30cc0d00a0 2024-04-25T10:07:47.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.077+0000 7f30cbe00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30cc0b59f0 con 0x7f30cc0ce860 2024-04-25T10:07:47.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.077+0000 7f30cbe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30cc0b5cd0 con 0x7f30cc15b7e0 2024-04-25T10:07:47.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.077+0000 7f30c2000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f30cc0ce860 0x7f30cc0cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.077+0000 7f30c2000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f30cc0ce860 0x7f30cc0cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59102/0 (socket says 172.21.15.119:59102) 2024-04-25T10:07:47.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.077+0000 7f30c2000640 1 -- 172.21.15.119:0/384620002 learned_addr learned my addr 172.21.15.119:0/384620002 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:47.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.078+0000 7f30c2000640 1 -- 172.21.15.119:0/384620002 >> v2:172.21.15.132:3300/0 conn(0x7f30cc15b7e0 msgr2=0x7f30cc15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.078+0000 7f30caa00640 1 --2- 172.21.15.119:0/384620002 >> v2:172.21.15.132:3300/0 conn(0x7f30cc15b7e0 0x7f30cc15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.078+0000 7f30c2000640 1 --2- 172.21.15.119:0/384620002 >> v2:172.21.15.132:3300/0 conn(0x7f30cc15b7e0 0x7f30cc15e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.078+0000 7f30c2000640 1 -- 172.21.15.119:0/384620002 >> v1:172.21.15.132:6789/0 conn(0x7f30cc0d00a0 legacy=0x7f30cc0d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.078+0000 7f30c2000640 1 -- 172.21.15.119:0/384620002 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30cc0b5fb0 con 0x7f30cc0ce860 2024-04-25T10:07:47.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.078+0000 7f30caa00640 1 --2- 172.21.15.119:0/384620002 >> v2:172.21.15.132:3300/0 conn(0x7f30cc15b7e0 0x7f30cc15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:47.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.079+0000 7f30c2000640 1 --2- 172.21.15.119:0/384620002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc0ce860 0x7f30cc0cecf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f30cc08bac0 tx=0x7f30b402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61647d64936f398 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.079+0000 7f30ca000640 1 -- 172.21.15.119:0/384620002 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4037070 con 0x7f30cc0ce860 2024-04-25T10:07:47.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.079+0000 7f30ca000640 1 -- 172.21.15.119:0/384620002 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b402ae20 con 0x7f30cc0ce860 2024-04-25T10:07:47.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.079+0000 7f30ca000640 1 -- 172.21.15.119:0/384620002 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4032660 con 0x7f30cc0ce860 2024-04-25T10:07:47.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.080+0000 7f30cbe00640 1 -- 172.21.15.119:0/384620002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc0ce860 msgr2=0x7f30cc0cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.080+0000 7f30cbe00640 1 --2- 172.21.15.119:0/384620002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc0ce860 0x7f30cc0cecf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f30cc08bac0 tx=0x7f30b402a110 comp rx=0 tx=0).stop 2024-04-25T10:07:47.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.080+0000 7f30cbe00640 1 -- 172.21.15.119:0/384620002 shutdown_connections 2024-04-25T10:07:47.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.080+0000 7f30cbe00640 1 --2- 172.21.15.119:0/384620002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc0ce860 0x7f30cc0cecf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.080+0000 7f30cbe00640 1 --2- 172.21.15.119:0/384620002 >> v2:172.21.15.132:3300/0 conn(0x7f30cc15b7e0 0x7f30cc15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.080+0000 7f30cbe00640 1 -- 172.21.15.119:0/384620002 >> 172.21.15.119:0/384620002 conn(0x7f30cc0722d0 msgr2=0x7f30cc0726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.080+0000 7f30cbe00640 1 -- 172.21.15.119:0/384620002 shutdown_connections 2024-04-25T10:07:47.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.080+0000 7f30cbe00640 1 -- 172.21.15.119:0/384620002 wait complete. 2024-04-25T10:07:47.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.082+0000 7f30cbe00640 1 Processor -- start 2024-04-25T10:07:47.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.082+0000 7f30cbe00640 1 -- start start 2024-04-25T10:07:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30cbe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30cc0ce860 0x7f30cc1f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30cbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc0d00a0 0x7f30cc1f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30cbe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30cc15b7e0 0x7f30cc0b7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30cbe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30cc161960 con 0x7f30cc0d00a0 2024-04-25T10:07:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30cbe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30cc0b53c0 con 0x7f30cc15b7e0 2024-04-25T10:07:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30cbe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30cc161ae0 con 0x7f30cc0ce860 2024-04-25T10:07:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30c2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc0d00a0 0x7f30cc1f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30c2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc0d00a0 0x7f30cc1f6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59116/0 (socket says 172.21.15.119:59116) 2024-04-25T10:07:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30c2000640 1 -- 172.21.15.119:0/2099833363 learned_addr learned my addr 172.21.15.119:0/2099833363 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30caa00640 1 --2- 172.21.15.119:0/2099833363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30cc0ce860 0x7f30cc1f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30c2000640 1 -- 172.21.15.119:0/2099833363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30cc0ce860 msgr2=0x7f30cc1f40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30c2000640 1 --2- 172.21.15.119:0/2099833363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30cc0ce860 0x7f30cc1f40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30c2000640 1 -- 172.21.15.119:0/2099833363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30cc15b7e0 msgr2=0x7f30cc0b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30c2000640 1 --2- 172.21.15.119:0/2099833363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30cc15b7e0 0x7f30cc0b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.083+0000 7f30c2000640 1 -- 172.21.15.119:0/2099833363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30cc0cd5d0 con 0x7f30cc0d00a0 2024-04-25T10:07:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.084+0000 7f30c2000640 1 --2- 172.21.15.119:0/2099833363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc0d00a0 0x7f30cc1f6df0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f30b402a150 tx=0x7f30b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.084+0000 7f30c3400640 1 -- 172.21.15.119:0/2099833363 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4041070 con 0x7f30cc0d00a0 2024-04-25T10:07:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.084+0000 7f30c3400640 1 -- 172.21.15.119:0/2099833363 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b403c670 con 0x7f30cc0d00a0 2024-04-25T10:07:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.084+0000 7f30c3400640 1 -- 172.21.15.119:0/2099833363 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4037040 con 0x7f30cc0d00a0 2024-04-25T10:07:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.084+0000 7f30cbe00640 1 -- 172.21.15.119:0/2099833363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30cc157ab0 con 0x7f30cc0d00a0 2024-04-25T10:07:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.084+0000 7f30cbe00640 1 -- 172.21.15.119:0/2099833363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30cc158070 con 0x7f30cc0d00a0 2024-04-25T10:07:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.085+0000 7f30cbe00640 1 -- 172.21.15.119:0/2099833363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30cc15dc40 con 0x7f30cc0d00a0 2024-04-25T10:07:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.086+0000 7f30c3400640 1 -- 172.21.15.119:0/2099833363 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30b4044070 con 0x7f30cc0d00a0 2024-04-25T10:07:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.087+0000 7f30c3400640 1 --2- 172.21.15.119:0/2099833363 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f30a806b4d0 0x7f30a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.087+0000 7f30c3400640 1 -- 172.21.15.119:0/2099833363 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30b40aab10 con 0x7f30cc0d00a0 2024-04-25T10:07:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.087+0000 7f30caa00640 1 --2- 172.21.15.119:0/2099833363 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f30a806b4d0 0x7f30a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.088+0000 7f30caa00640 1 --2- 172.21.15.119:0/2099833363 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f30a806b4d0 0x7f30a806d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00b840 tx=0x7f30bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.095+0000 7f30c3400640 1 -- 172.21.15.119:0/2099833363 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30b4079a10 con 0x7f30cc0d00a0 2024-04-25T10:07:47.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.289+0000 7f30cbe00640 1 -- 172.21.15.119:0/2099833363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f30cc0d3220 con 0x7f30cc0d00a0 2024-04-25T10:07:47.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.295+0000 7f30c3400640 1 -- 172.21.15.119:0/2099833363 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f30b4079310 con 0x7f30cc0d00a0 2024-04-25T10:07:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.297+0000 7f30cbe00640 1 -- 172.21.15.119:0/2099833363 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f30a806b4d0 msgr2=0x7f30a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.297+0000 7f30cbe00640 1 --2- 172.21.15.119:0/2099833363 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f30a806b4d0 0x7f30a806d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00b840 tx=0x7f30bc008040 comp rx=0 tx=0).stop 2024-04-25T10:07:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.297+0000 7f30cbe00640 1 -- 172.21.15.119:0/2099833363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc0d00a0 msgr2=0x7f30cc1f6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.297+0000 7f30cbe00640 1 --2- 172.21.15.119:0/2099833363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc0d00a0 0x7f30cc1f6df0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f30b402a150 tx=0x7f30b40046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.298+0000 7f30cbe00640 1 -- 172.21.15.119:0/2099833363 shutdown_connections 2024-04-25T10:07:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.298+0000 7f30cbe00640 1 --2- 172.21.15.119:0/2099833363 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f30a806b4d0 0x7f30a806d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.298+0000 7f30cbe00640 1 --2- 172.21.15.119:0/2099833363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30cc15b7e0 0x7f30cc0b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.298+0000 7f30cbe00640 1 --2- 172.21.15.119:0/2099833363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc0d00a0 0x7f30cc1f6df0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.298+0000 7f30cbe00640 1 --2- 172.21.15.119:0/2099833363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30cc0ce860 0x7f30cc1f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.298+0000 7f30cbe00640 1 -- 172.21.15.119:0/2099833363 >> 172.21.15.119:0/2099833363 conn(0x7f30cc0722d0 msgr2=0x7f30cc0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.298+0000 7f30cbe00640 1 -- 172.21.15.119:0/2099833363 shutdown_connections 2024-04-25T10:07:47.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.298+0000 7f30cbe00640 1 -- 172.21.15.119:0/2099833363 wait complete. 2024-04-25T10:07:47.321 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:07:47.321 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:47.417 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:47.525 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:07:47.525 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:07:47.525 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:47.577 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:07:47.577 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:47.625 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:07:47.625 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:47.717 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:47.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.926+0000 7f592d600640 1 Processor -- start 2024-04-25T10:07:47.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.926+0000 7f592d600640 1 -- start start 2024-04-25T10:07:47.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.927+0000 7f592d600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f592815c9f0 0x7f59281664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.928+0000 7f592d600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59280ce6c0 con 0x7f592806e8f0 2024-04-25T10:07:47.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.928+0000 7f592d600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59280cc4e0 con 0x7f592815bf00 2024-04-25T10:07:47.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.928+0000 7f592d600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59281455e0 con 0x7f592815c9f0 2024-04-25T10:07:47.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.928+0000 7f5926000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f592806e8f0 0x7f592815c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36726/0 (socket says 172.21.15.119:36726) 2024-04-25T10:07:47.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.928+0000 7f5926000640 1 -- 172.21.15.119:0/263612479 learned_addr learned my addr 172.21.15.119:0/263612479 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:47.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.928+0000 7f5927400640 1 --2- 172.21.15.119:0/263612479 >> v2:172.21.15.119:3300/0 conn(0x7f592815c9f0 0x7f59281664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.928+0000 7f5925600640 1 -- 172.21.15.119:0/263612479 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037640337 0 0) 0x7f5914001110 con 0x7f592806e8f0 2024-04-25T10:07:47.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.928+0000 7f5925600640 1 -- 172.21.15.119:0/263612479 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f590c003520 con 0x7f592806e8f0 2024-04-25T10:07:47.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.928+0000 7f5927400640 1 -- 172.21.15.119:0/263612479 >> v1:172.21.15.132:6789/0 conn(0x7f592815bf00 legacy=0x7f59280cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.928+0000 7f5927400640 1 -- 172.21.15.119:0/263612479 >> v1:172.21.15.119:6789/0 conn(0x7f592806e8f0 legacy=0x7f592815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.928+0000 7f5927400640 1 -- 172.21.15.119:0/263612479 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59280cc7b0 con 0x7f592815c9f0 2024-04-25T10:07:47.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.929+0000 7f5927400640 1 --2- 172.21.15.119:0/263612479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f592815c9f0 0x7f59281664c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5920009d90 tx=0x7f592002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6714945b540c891 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.930+0000 7f5925600640 1 -- 172.21.15.119:0/263612479 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920007740 con 0x7f592815c9f0 2024-04-25T10:07:47.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.930+0000 7f5925600640 1 -- 172.21.15.119:0/263612479 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59200078e0 con 0x7f592815c9f0 2024-04-25T10:07:47.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.930+0000 7f5925600640 1 -- 172.21.15.119:0/263612479 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920007c90 con 0x7f592815c9f0 2024-04-25T10:07:47.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.930+0000 7f592d600640 1 -- 172.21.15.119:0/263612479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f592815c9f0 msgr2=0x7f59281664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.930+0000 7f592d600640 1 --2- 172.21.15.119:0/263612479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f592815c9f0 0x7f59281664c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5920009d90 tx=0x7f592002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.930+0000 7f592d600640 1 -- 172.21.15.119:0/263612479 shutdown_connections 2024-04-25T10:07:47.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.930+0000 7f592d600640 1 --2- 172.21.15.119:0/263612479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f592815c9f0 0x7f59281664c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.930+0000 7f592d600640 1 -- 172.21.15.119:0/263612479 >> 172.21.15.119:0/263612479 conn(0x7f592806aba0 msgr2=0x7f592806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.931+0000 7f592d600640 1 -- 172.21.15.119:0/263612479 shutdown_connections 2024-04-25T10:07:47.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.931+0000 7f592d600640 1 -- 172.21.15.119:0/263612479 wait complete. 2024-04-25T10:07:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.932+0000 7f592d600640 1 Processor -- start 2024-04-25T10:07:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.932+0000 7f592d600640 1 -- start start 2024-04-25T10:07:47.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.933+0000 7f592d600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f592806e8f0 0x7f5928158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.933+0000 7f592d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f592815bf00 0x7f5928143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.933+0000 7f592d600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f592815c9f0 0x7f592813e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.933+0000 7f592d600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59280ccd70 con 0x7f592815bf00 2024-04-25T10:07:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.933+0000 7f592d600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592806d450 con 0x7f592815c9f0 2024-04-25T10:07:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.933+0000 7f592d600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59280cd050 con 0x7f592806e8f0 2024-04-25T10:07:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.933+0000 7f5926000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f592815bf00 0x7f5928143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.934+0000 7f5926000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f592815bf00 0x7f5928143a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59136/0 (socket says 172.21.15.119:59136) 2024-04-25T10:07:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.934+0000 7f5926000640 1 -- 172.21.15.119:0/706476029 learned_addr learned my addr 172.21.15.119:0/706476029 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.934+0000 7f5927400640 1 --2- 172.21.15.119:0/706476029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f592815c9f0 0x7f592813e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.934+0000 7f5926a00640 1 --2- 172.21.15.119:0/706476029 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f592806e8f0 0x7f5928158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.934+0000 7f5926000640 1 -- 172.21.15.119:0/706476029 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f592806e8f0 msgr2=0x7f5928158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.934+0000 7f5926000640 1 --2- 172.21.15.119:0/706476029 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f592806e8f0 0x7f5928158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.934+0000 7f5926000640 1 -- 172.21.15.119:0/706476029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f592815c9f0 msgr2=0x7f592813e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.934+0000 7f5926000640 1 --2- 172.21.15.119:0/706476029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f592815c9f0 0x7f592813e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.934+0000 7f5926000640 1 -- 172.21.15.119:0/706476029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592813efa0 con 0x7f592815bf00 2024-04-25T10:07:47.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.934+0000 7f5926000640 1 --2- 172.21.15.119:0/706476029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f592815bf00 0x7f5928143a80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f591400c730 tx=0x7f591400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.934+0000 7f591ea00640 1 -- 172.21.15.119:0/706476029 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914017070 con 0x7f592815bf00 2024-04-25T10:07:47.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.935+0000 7f591ea00640 1 -- 172.21.15.119:0/706476029 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59140058b0 con 0x7f592815bf00 2024-04-25T10:07:47.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.935+0000 7f592d600640 1 -- 172.21.15.119:0/706476029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5928142d00 con 0x7f592815bf00 2024-04-25T10:07:47.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.935+0000 7f591ea00640 1 -- 172.21.15.119:0/706476029 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914005ce0 con 0x7f592815bf00 2024-04-25T10:07:47.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.935+0000 7f592d600640 1 -- 172.21.15.119:0/706476029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5928143560 con 0x7f592815bf00 2024-04-25T10:07:47.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.936+0000 7f592d600640 1 -- 172.21.15.119:0/706476029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5928143760 con 0x7f592815bf00 2024-04-25T10:07:47.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.937+0000 7f591ea00640 1 -- 172.21.15.119:0/706476029 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f591401a070 con 0x7f592815bf00 2024-04-25T10:07:47.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.938+0000 7f591ea00640 1 --2- 172.21.15.119:0/706476029 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f590806b4d0 0x7f590806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.938+0000 7f591ea00640 1 -- 172.21.15.119:0/706476029 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59140897a0 con 0x7f592815bf00 2024-04-25T10:07:47.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.938+0000 7f5926a00640 1 --2- 172.21.15.119:0/706476029 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f590806b4d0 0x7f590806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.938+0000 7f5926a00640 1 --2- 172.21.15.119:0/706476029 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f590806b4d0 0x7f590806d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5918003d90 tx=0x7f5918009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:47.942+0000 7f591ea00640 1 -- 172.21.15.119:0/706476029 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5928143760 con 0x7f592815bf00 2024-04-25T10:07:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.135+0000 7f592d600640 1 -- 172.21.15.119:0/706476029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5928141700 con 0x7f592815bf00 2024-04-25T10:07:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.137+0000 7f591ea00640 1 -- 172.21.15.119:0/706476029 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f59280ce6c0 con 0x7f592815bf00 2024-04-25T10:07:48.143 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:48.143 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:08.319104+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.139+0000 7f592d600640 1 -- 172.21.15.119:0/706476029 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f590806b4d0 msgr2=0x7f590806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.139+0000 7f592d600640 1 --2- 172.21.15.119:0/706476029 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f590806b4d0 0x7f590806d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5918003d90 tx=0x7f5918009040 comp rx=0 tx=0).stop 2024-04-25T10:07:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.139+0000 7f592d600640 1 -- 172.21.15.119:0/706476029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f592815bf00 msgr2=0x7f5928143a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.139+0000 7f592d600640 1 --2- 172.21.15.119:0/706476029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f592815bf00 0x7f5928143a80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f591400c730 tx=0x7f591400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.140+0000 7f592d600640 1 -- 172.21.15.119:0/706476029 shutdown_connections 2024-04-25T10:07:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.140+0000 7f592d600640 1 --2- 172.21.15.119:0/706476029 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f590806b4d0 0x7f590806d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.140+0000 7f592d600640 1 --2- 172.21.15.119:0/706476029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f592815c9f0 0x7f592813e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.140+0000 7f592d600640 1 --2- 172.21.15.119:0/706476029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f592815bf00 0x7f5928143a80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.140+0000 7f592d600640 1 --2- 172.21.15.119:0/706476029 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f592806e8f0 0x7f5928158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.140+0000 7f592d600640 1 -- 172.21.15.119:0/706476029 >> 172.21.15.119:0/706476029 conn(0x7f592806aba0 msgr2=0x7f592815ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.140+0000 7f592d600640 1 -- 172.21.15.119:0/706476029 shutdown_connections 2024-04-25T10:07:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.140+0000 7f592d600640 1 -- 172.21.15.119:0/706476029 wait complete. 2024-04-25T10:07:48.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:05.135832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:05.768083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:48.164 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.349+0000 7fa908c00640 1 Processor -- start 2024-04-25T10:07:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.350+0000 7fa908c00640 1 -- start start 2024-04-25T10:07:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.351+0000 7fa908c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa90415dc80 0x7fa904161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.351+0000 7fa908c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9041627a0 con 0x7fa9040b97c0 2024-04-25T10:07:48.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.351+0000 7fa908c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9041621e0 con 0x7fa9040b5c00 2024-04-25T10:07:48.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.351+0000 7fa908c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9041624c0 con 0x7fa90415dc80 2024-04-25T10:07:48.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.351+0000 7fa901400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa9040b97c0 0x7fa9040b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36740/0 (socket says 172.21.15.119:36740) 2024-04-25T10:07:48.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.351+0000 7fa901400640 1 -- 172.21.15.119:0/3655782731 learned_addr learned my addr 172.21.15.119:0/3655782731 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.351+0000 7fa900a00640 1 -- 172.21.15.119:0/3655782731 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186877549 0 0) 0x7fa9041627a0 con 0x7fa9040b97c0 2024-04-25T10:07:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa901e00640 1 --2- 172.21.15.119:0/3655782731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa90415dc80 0x7fa904161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa900a00640 1 -- 172.21.15.119:0/3655782731 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8e8003520 con 0x7fa9040b97c0 2024-04-25T10:07:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa900a00640 1 -- 172.21.15.119:0/3655782731 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 17522846 0 0) 0x7fa8e8003520 con 0x7fa9040b97c0 2024-04-25T10:07:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa900a00640 1 -- 172.21.15.119:0/3655782731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa90415dc80 msgr2=0x7fa904161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa900a00640 1 --2- 172.21.15.119:0/3655782731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa90415dc80 0x7fa904161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa900a00640 1 -- 172.21.15.119:0/3655782731 >> v1:172.21.15.132:6789/0 conn(0x7fa9040b5c00 legacy=0x7fa9040d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa900a00640 1 -- 172.21.15.119:0/3655782731 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040d1c70 con 0x7fa9040b97c0 2024-04-25T10:07:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa900a00640 1 -- 172.21.15.119:0/3655782731 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fa8ec001670 con 0x7fa9040b97c0 2024-04-25T10:07:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa900a00640 1 -- 172.21.15.119:0/3655782731 >> v1:172.21.15.119:6789/0 conn(0x7fa9040b97c0 legacy=0x7fa9040b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa900a00640 1 --2- 172.21.15.119:0/3655782731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8e80039e0 0x7fa8e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa900a00640 1 --2- 172.21.15.119:0/3655782731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa90415dc80 0x7fa8e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa900a00640 1 --2- 172.21.15.119:0/3655782731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8e8004d40 0x7fa8e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa900a00640 1 -- 172.21.15.119:0/3655782731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9041624c0 con 0x7fa8e80039e0 2024-04-25T10:07:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa900a00640 1 -- 172.21.15.119:0/3655782731 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e800bb50 con 0x7fa8e8004d40 2024-04-25T10:07:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa900a00640 1 -- 172.21.15.119:0/3655782731 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e800bd20 con 0x7fa90415dc80 2024-04-25T10:07:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.352+0000 7fa901e00640 1 --2- 172.21.15.119:0/3655782731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8e80039e0 0x7fa8e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.353+0000 7fa901400640 1 --2- 172.21.15.119:0/3655782731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa90415dc80 0x7fa8e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.353+0000 7fa902800640 1 --2- 172.21.15.119:0/3655782731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8e8004d40 0x7fa8e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.353+0000 7fa901e00640 1 -- 172.21.15.119:0/3655782731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa90415dc80 msgr2=0x7fa8e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.353+0000 7fa901e00640 1 --2- 172.21.15.119:0/3655782731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa90415dc80 0x7fa8e8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.353+0000 7fa901e00640 1 -- 172.21.15.119:0/3655782731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8e8004d40 msgr2=0x7fa8e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.353+0000 7fa901e00640 1 --2- 172.21.15.119:0/3655782731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8e8004d40 0x7fa8e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.353+0000 7fa901e00640 1 -- 172.21.15.119:0/3655782731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e8003520 con 0x7fa8e80039e0 2024-04-25T10:07:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.353+0000 7fa902800640 1 --2- 172.21.15.119:0/3655782731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8e8004d40 0x7fa8e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.354+0000 7fa901e00640 1 --2- 172.21.15.119:0/3655782731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8e80039e0 0x7fa8e8003e00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400ba60 tx=0x7fa8e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c994e523764ec606 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.354+0000 7fa900a00640 1 -- 172.21.15.119:0/3655782731 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4030070 con 0x7fa8e80039e0 2024-04-25T10:07:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.354+0000 7fa900a00640 1 -- 172.21.15.119:0/3655782731 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f402ae10 con 0x7fa8e80039e0 2024-04-25T10:07:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.354+0000 7fa900a00640 1 -- 172.21.15.119:0/3655782731 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f400b600 con 0x7fa8e80039e0 2024-04-25T10:07:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.355+0000 7fa908c00640 1 -- 172.21.15.119:0/3655782731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8e80039e0 msgr2=0x7fa8e8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.355+0000 7fa908c00640 1 --2- 172.21.15.119:0/3655782731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8e80039e0 0x7fa8e8003e00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400ba60 tx=0x7fa8e8005830 comp rx=0 tx=0).stop 2024-04-25T10:07:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.355+0000 7fa902800640 1 -- 172.21.15.119:0/3655782731 reap_dead start 2024-04-25T10:07:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.355+0000 7fa908c00640 1 -- 172.21.15.119:0/3655782731 shutdown_connections 2024-04-25T10:07:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.355+0000 7fa908c00640 1 -- 172.21.15.119:0/3655782731 >> 172.21.15.119:0/3655782731 conn(0x7fa9040d04f0 msgr2=0x7fa9040d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.355+0000 7fa908c00640 1 -- 172.21.15.119:0/3655782731 shutdown_connections 2024-04-25T10:07:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.355+0000 7fa908c00640 1 -- 172.21.15.119:0/3655782731 wait complete. 2024-04-25T10:07:48.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.357+0000 7fa908c00640 1 Processor -- start 2024-04-25T10:07:48.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.357+0000 7fa908c00640 1 -- start start 2024-04-25T10:07:48.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.358+0000 7fa908c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8e80039e0 0x7fa9040ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.358+0000 7fa908c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa904160b10 0x7fa9040c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.358+0000 7fa908c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa904160f90 0x7fa9040c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.358+0000 7fa908c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9041619f0 con 0x7fa8e80039e0 2024-04-25T10:07:48.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.358+0000 7fa908c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa904161870 con 0x7fa904160f90 2024-04-25T10:07:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.358+0000 7fa908c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa904161b70 con 0x7fa904160b10 2024-04-25T10:07:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.358+0000 7fa901e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8e80039e0 0x7fa9040ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.358+0000 7fa901e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8e80039e0 0x7fa9040ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59156/0 (socket says 172.21.15.119:59156) 2024-04-25T10:07:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.358+0000 7fa901e00640 1 -- 172.21.15.119:0/2716630427 learned_addr learned my addr 172.21.15.119:0/2716630427 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.358+0000 7fa901e00640 1 -- 172.21.15.119:0/2716630427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa904160b10 msgr2=0x7fa9040c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.358+0000 7fa901e00640 1 --2- 172.21.15.119:0/2716630427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa904160b10 0x7fa9040c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.358+0000 7fa901e00640 1 -- 172.21.15.119:0/2716630427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa904160f90 msgr2=0x7fa9040c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.358+0000 7fa901e00640 1 --2- 172.21.15.119:0/2716630427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa904160f90 0x7fa9040c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.358+0000 7fa901e00640 1 -- 172.21.15.119:0/2716630427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040cd2e0 con 0x7fa8e80039e0 2024-04-25T10:07:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.359+0000 7fa901e00640 1 --2- 172.21.15.119:0/2716630427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8e80039e0 0x7fa9040ccda0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa8f402bfd0 tx=0x7fa8f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.359+0000 7fa8fa000640 1 -- 172.21.15.119:0/2716630427 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4043020 con 0x7fa8e80039e0 2024-04-25T10:07:48.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.359+0000 7fa8fa000640 1 -- 172.21.15.119:0/2716630427 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f403f680 con 0x7fa8e80039e0 2024-04-25T10:07:48.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.359+0000 7fa908c00640 1 -- 172.21.15.119:0/2716630427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9040caf00 con 0x7fa8e80039e0 2024-04-25T10:07:48.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.359+0000 7fa8fa000640 1 -- 172.21.15.119:0/2716630427 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f400b600 con 0x7fa8e80039e0 2024-04-25T10:07:48.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.359+0000 7fa908c00640 1 -- 172.21.15.119:0/2716630427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9040cb4c0 con 0x7fa8e80039e0 2024-04-25T10:07:48.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.360+0000 7fa908c00640 1 -- 172.21.15.119:0/2716630427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9040750b0 con 0x7fa8e80039e0 2024-04-25T10:07:48.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.361+0000 7fa8fa000640 1 -- 172.21.15.119:0/2716630427 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8f4046020 con 0x7fa8e80039e0 2024-04-25T10:07:48.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.362+0000 7fa8fa000640 1 --2- 172.21.15.119:0/2716630427 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8e006b4d0 0x7fa8e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.362+0000 7fa8fa000640 1 -- 172.21.15.119:0/2716630427 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8f40aade0 con 0x7fa8e80039e0 2024-04-25T10:07:48.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.362+0000 7fa901400640 1 --2- 172.21.15.119:0/2716630427 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8e006b4d0 0x7fa8e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.363+0000 7fa901400640 1 --2- 172.21.15.119:0/2716630427 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8e006b4d0 0x7fa8e006d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa9040cbea0 tx=0x7fa8ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.366+0000 7fa8fa000640 1 -- 172.21.15.119:0/2716630427 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8f404d030 con 0x7fa8e80039e0 2024-04-25T10:07:48.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.558+0000 7fa908c00640 1 -- 172.21.15.119:0/2716630427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9040d1780 con 0x7fa8e80039e0 2024-04-25T10:07:48.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.559+0000 7fa8fa000640 1 -- 172.21.15.119:0/2716630427 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa8f403be20 con 0x7fa8e80039e0 2024-04-25T10:07:48.564 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:07:48.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.561+0000 7fa908c00640 1 -- 172.21.15.119:0/2716630427 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8e006b4d0 msgr2=0x7fa8e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.561+0000 7fa908c00640 1 --2- 172.21.15.119:0/2716630427 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8e006b4d0 0x7fa8e006d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa9040cbea0 tx=0x7fa8ec009040 comp rx=0 tx=0).stop 2024-04-25T10:07:48.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.561+0000 7fa908c00640 1 -- 172.21.15.119:0/2716630427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8e80039e0 msgr2=0x7fa9040ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.561+0000 7fa908c00640 1 --2- 172.21.15.119:0/2716630427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8e80039e0 0x7fa9040ccda0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa8f402bfd0 tx=0x7fa8f4004730 comp rx=0 tx=0).stop 2024-04-25T10:07:48.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.561+0000 7fa908c00640 1 -- 172.21.15.119:0/2716630427 shutdown_connections 2024-04-25T10:07:48.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.561+0000 7fa908c00640 1 --2- 172.21.15.119:0/2716630427 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8e006b4d0 0x7fa8e006d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.561+0000 7fa908c00640 1 --2- 172.21.15.119:0/2716630427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa904160f90 0x7fa9040c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.561+0000 7fa908c00640 1 --2- 172.21.15.119:0/2716630427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa904160b10 0x7fa9040c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.561+0000 7fa908c00640 1 --2- 172.21.15.119:0/2716630427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8e80039e0 0x7fa9040ccda0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.561+0000 7fa908c00640 1 -- 172.21.15.119:0/2716630427 >> 172.21.15.119:0/2716630427 conn(0x7fa9040d04f0 msgr2=0x7fa9040b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.562+0000 7fa908c00640 1 -- 172.21.15.119:0/2716630427 shutdown_connections 2024-04-25T10:07:48.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.562+0000 7fa908c00640 1 -- 172.21.15.119:0/2716630427 wait complete. 2024-04-25T10:07:48.585 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:48.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.777+0000 7fa04a400640 1 Processor -- start 2024-04-25T10:07:48.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.777+0000 7fa04a400640 1 -- start start 2024-04-25T10:07:48.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.778+0000 7fa04a400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa0441450e0 0x7fa04406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.778+0000 7fa04a400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440cc9e0 0x7fa0440b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.778+0000 7fa04a400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa04415f400 con 0x7fa04406b0e0 2024-04-25T10:07:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.778+0000 7fa04a400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0441454c0 con 0x7fa0441450e0 2024-04-25T10:07:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.778+0000 7fa04a400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04406b540 con 0x7fa0440cc9e0 2024-04-25T10:07:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.779+0000 7fa043e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa04406b0e0 0x7fa0440cc2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36754/0 (socket says 172.21.15.119:36754) 2024-04-25T10:07:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.779+0000 7fa043e00640 1 -- 172.21.15.119:0/632323102 learned_addr learned my addr 172.21.15.119:0/632323102 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.779+0000 7fa043400640 1 -- 172.21.15.119:0/632323102 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518992745 0 0) 0x7fa04415f400 con 0x7fa04406b0e0 2024-04-25T10:07:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.779+0000 7fa043400640 1 -- 172.21.15.119:0/632323102 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa02c003520 con 0x7fa04406b0e0 2024-04-25T10:07:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.779+0000 7fa049000640 1 --2- 172.21.15.119:0/632323102 >> v2:172.21.15.132:3300/0 conn(0x7fa0441450e0 0x7fa04406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.779+0000 7fa049a00640 1 --2- 172.21.15.119:0/632323102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440cc9e0 0x7fa0440b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.779+0000 7fa043400640 1 -- 172.21.15.119:0/632323102 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4279841779 0 0) 0x7fa02c003520 con 0x7fa04406b0e0 2024-04-25T10:07:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.779+0000 7fa043400640 1 -- 172.21.15.119:0/632323102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440cc9e0 msgr2=0x7fa0440b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.779+0000 7fa043400640 1 --2- 172.21.15.119:0/632323102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440cc9e0 0x7fa0440b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.779+0000 7fa043400640 1 -- 172.21.15.119:0/632323102 >> v2:172.21.15.132:3300/0 conn(0x7fa0441450e0 msgr2=0x7fa04406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.779+0000 7fa043400640 1 --2- 172.21.15.119:0/632323102 >> v2:172.21.15.132:3300/0 conn(0x7fa0441450e0 0x7fa04406aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.779+0000 7fa043400640 1 -- 172.21.15.119:0/632323102 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04415f6e0 con 0x7fa04406b0e0 2024-04-25T10:07:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.779+0000 7fa043400640 1 -- 172.21.15.119:0/632323102 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fa030001690 con 0x7fa04406b0e0 2024-04-25T10:07:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.779+0000 7fa043400640 1 -- 172.21.15.119:0/632323102 >> v1:172.21.15.119:6789/0 conn(0x7fa04406b0e0 legacy=0x7fa0440cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.780+0000 7fa043400640 1 --2- 172.21.15.119:0/632323102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440cc9e0 0x7fa02c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.780+0000 7fa043400640 1 --2- 172.21.15.119:0/632323102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa02c003fe0 0x7fa02c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.780+0000 7fa043400640 1 --2- 172.21.15.119:0/632323102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa02c004d40 0x7fa02c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.780+0000 7fa043400640 1 -- 172.21.15.119:0/632323102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c00bb80 con 0x7fa02c004d40 2024-04-25T10:07:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.780+0000 7fa043400640 1 -- 172.21.15.119:0/632323102 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c00bd30 con 0x7fa02c003fe0 2024-04-25T10:07:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.780+0000 7fa043400640 1 -- 172.21.15.119:0/632323102 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c00bf30 con 0x7fa0440cc9e0 2024-04-25T10:07:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.780+0000 7fa049a00640 1 --2- 172.21.15.119:0/632323102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa02c004d40 0x7fa02c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.780+0000 7fa049000640 1 --2- 172.21.15.119:0/632323102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440cc9e0 0x7fa02c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.780+0000 7fa043e00640 1 --2- 172.21.15.119:0/632323102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa02c003fe0 0x7fa02c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.780+0000 7fa049a00640 1 -- 172.21.15.119:0/632323102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440cc9e0 msgr2=0x7fa02c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.780+0000 7fa049a00640 1 --2- 172.21.15.119:0/632323102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440cc9e0 0x7fa02c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.780+0000 7fa049a00640 1 -- 172.21.15.119:0/632323102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa02c003fe0 msgr2=0x7fa02c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.780+0000 7fa049a00640 1 --2- 172.21.15.119:0/632323102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa02c003fe0 0x7fa02c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.780+0000 7fa049a00640 1 -- 172.21.15.119:0/632323102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02c003520 con 0x7fa02c004d40 2024-04-25T10:07:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.781+0000 7fa049a00640 1 --2- 172.21.15.119:0/632323102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa02c004d40 0x7fa02c0051c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa03c007360 tx=0x7fa02c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f961568be5707a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.782+0000 7fa043400640 1 -- 172.21.15.119:0/632323102 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c009be0 con 0x7fa02c004d40 2024-04-25T10:07:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.782+0000 7fa043400640 1 -- 172.21.15.119:0/632323102 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c036600 con 0x7fa02c004d40 2024-04-25T10:07:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.782+0000 7fa043400640 1 -- 172.21.15.119:0/632323102 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c0369d0 con 0x7fa02c004d40 2024-04-25T10:07:48.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.782+0000 7fa04a400640 1 -- 172.21.15.119:0/632323102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa02c004d40 msgr2=0x7fa02c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.782+0000 7fa04a400640 1 --2- 172.21.15.119:0/632323102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa02c004d40 0x7fa02c0051c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa03c007360 tx=0x7fa02c005830 comp rx=0 tx=0).stop 2024-04-25T10:07:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.782+0000 7fa049a00640 1 -- 172.21.15.119:0/632323102 reap_dead start 2024-04-25T10:07:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.782+0000 7fa04a400640 1 -- 172.21.15.119:0/632323102 shutdown_connections 2024-04-25T10:07:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.782+0000 7fa04a400640 1 -- 172.21.15.119:0/632323102 >> 172.21.15.119:0/632323102 conn(0x7fa044066ff0 msgr2=0x7fa0440b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.782+0000 7fa04a400640 1 -- 172.21.15.119:0/632323102 shutdown_connections 2024-04-25T10:07:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.782+0000 7fa04a400640 1 -- 172.21.15.119:0/632323102 wait complete. 2024-04-25T10:07:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.784+0000 7fa04a400640 1 Processor -- start 2024-04-25T10:07:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.784+0000 7fa04a400640 1 -- start start 2024-04-25T10:07:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.785+0000 7fa04a400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa02c004d40 0x7fa0440c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.785+0000 7fa04a400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0440add90 0x7fa0440c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.785+0000 7fa04a400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04415ac90 0x7fa04415b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.785+0000 7fa04a400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0440b2e80 con 0x7fa04415ac90 2024-04-25T10:07:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.785+0000 7fa04a400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04415fca0 con 0x7fa0440add90 2024-04-25T10:07:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.785+0000 7fa04a400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0440b3000 con 0x7fa02c004d40 2024-04-25T10:07:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.785+0000 7fa049a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04415ac90 0x7fa04415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.786+0000 7fa049a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04415ac90 0x7fa04415b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59180/0 (socket says 172.21.15.119:59180) 2024-04-25T10:07:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.786+0000 7fa049a00640 1 -- 172.21.15.119:0/1517975053 learned_addr learned my addr 172.21.15.119:0/1517975053 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.786+0000 7fa049000640 1 --2- 172.21.15.119:0/1517975053 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa02c004d40 0x7fa0440c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.786+0000 7fa043e00640 1 --2- 172.21.15.119:0/1517975053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0440add90 0x7fa0440c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.786+0000 7fa049a00640 1 -- 172.21.15.119:0/1517975053 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa02c004d40 msgr2=0x7fa0440c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.786+0000 7fa049a00640 1 --2- 172.21.15.119:0/1517975053 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa02c004d40 0x7fa0440c43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.786+0000 7fa049a00640 1 -- 172.21.15.119:0/1517975053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0440add90 msgr2=0x7fa0440c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.786+0000 7fa049a00640 1 --2- 172.21.15.119:0/1517975053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0440add90 0x7fa0440c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.786+0000 7fa049a00640 1 -- 172.21.15.119:0/1517975053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0440ae220 con 0x7fa04415ac90 2024-04-25T10:07:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.786+0000 7fa049a00640 1 --2- 172.21.15.119:0/1517975053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04415ac90 0x7fa04415b140 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa03c007480 tx=0x7fa03c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.787+0000 7fa040c00640 1 -- 172.21.15.119:0/1517975053 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c009be0 con 0x7fa04415ac90 2024-04-25T10:07:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.787+0000 7fa040c00640 1 -- 172.21.15.119:0/1517975053 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c036400 con 0x7fa04415ac90 2024-04-25T10:07:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.787+0000 7fa040c00640 1 -- 172.21.15.119:0/1517975053 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c038500 con 0x7fa04415ac90 2024-04-25T10:07:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.787+0000 7fa04a400640 1 -- 172.21.15.119:0/1517975053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0440ae500 con 0x7fa04415ac90 2024-04-25T10:07:48.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.787+0000 7fa04a400640 1 -- 172.21.15.119:0/1517975053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0440aeac0 con 0x7fa04415ac90 2024-04-25T10:07:48.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.788+0000 7fa04a400640 1 -- 172.21.15.119:0/1517975053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0440749b0 con 0x7fa04415ac90 2024-04-25T10:07:48.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.789+0000 7fa040c00640 1 -- 172.21.15.119:0/1517975053 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa03c02e620 con 0x7fa04415ac90 2024-04-25T10:07:48.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.790+0000 7fa040c00640 1 --2- 172.21.15.119:0/1517975053 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa02406b4d0 0x7fa02406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.790+0000 7fa040c00640 1 -- 172.21.15.119:0/1517975053 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa03c0aa500 con 0x7fa04415ac90 2024-04-25T10:07:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.790+0000 7fa049000640 1 --2- 172.21.15.119:0/1517975053 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa02406b4d0 0x7fa02406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.791+0000 7fa049000640 1 --2- 172.21.15.119:0/1517975053 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa02406b4d0 0x7fa02406d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa038006fd0 tx=0x7fa038008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:48.794+0000 7fa040c00640 1 -- 172.21.15.119:0/1517975053 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa03c032030 con 0x7fa04415ac90 2024-04-25T10:07:49.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.000+0000 7fa04a400640 1 -- 172.21.15.119:0/1517975053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa04406cf80 con 0x7fa04415ac90 2024-04-25T10:07:49.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.001+0000 7fa040c00640 1 -- 172.21.15.119:0/1517975053 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa03c079250 con 0x7fa04415ac90 2024-04-25T10:07:49.008 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:07:49.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.004+0000 7fa04a400640 1 -- 172.21.15.119:0/1517975053 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa02406b4d0 msgr2=0x7fa02406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.004+0000 7fa04a400640 1 --2- 172.21.15.119:0/1517975053 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa02406b4d0 0x7fa02406d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa038006fd0 tx=0x7fa038008040 comp rx=0 tx=0).stop 2024-04-25T10:07:49.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.004+0000 7fa04a400640 1 -- 172.21.15.119:0/1517975053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04415ac90 msgr2=0x7fa04415b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.004+0000 7fa04a400640 1 --2- 172.21.15.119:0/1517975053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04415ac90 0x7fa04415b140 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa03c007480 tx=0x7fa03c004ab0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.004+0000 7fa04a400640 1 -- 172.21.15.119:0/1517975053 shutdown_connections 2024-04-25T10:07:49.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.004+0000 7fa04a400640 1 --2- 172.21.15.119:0/1517975053 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa02406b4d0 0x7fa02406d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.004+0000 7fa04a400640 1 --2- 172.21.15.119:0/1517975053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa04415ac90 0x7fa04415b140 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.004+0000 7fa04a400640 1 --2- 172.21.15.119:0/1517975053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0440add90 0x7fa0440c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.004+0000 7fa04a400640 1 --2- 172.21.15.119:0/1517975053 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa02c004d40 0x7fa0440c43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.004+0000 7fa04a400640 1 -- 172.21.15.119:0/1517975053 >> 172.21.15.119:0/1517975053 conn(0x7fa044066ff0 msgr2=0x7fa0440aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:49.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.005+0000 7fa04a400640 1 -- 172.21.15.119:0/1517975053 shutdown_connections 2024-04-25T10:07:49.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.005+0000 7fa04a400640 1 -- 172.21.15.119:0/1517975053 wait complete. 2024-04-25T10:07:49.030 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:49.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.228+0000 7f545be00640 1 Processor -- start 2024-04-25T10:07:49.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.228+0000 7f545be00640 1 -- start start 2024-04-25T10:07:49.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.230+0000 7f545be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f545c0ada20 0x7f545c0acb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.230+0000 7f545be00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c15e120 con 0x7f545c06aba0 2024-04-25T10:07:49.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.230+0000 7f545be00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c15e400 con 0x7f545c0b36a0 2024-04-25T10:07:49.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.230+0000 7f545be00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545c15dec0 con 0x7f545c0ada20 2024-04-25T10:07:49.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.230+0000 7f545aa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f545c0ada20 0x7f545c0acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.230+0000 7f545a000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f545c06aba0 0x7f545c0b2f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36758/0 (socket says 172.21.15.119:36758) 2024-04-25T10:07:49.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.230+0000 7f545a000640 1 -- 172.21.15.119:0/185030077 learned_addr learned my addr 172.21.15.119:0/185030077 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:49.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.231+0000 7f545aa00640 1 -- 172.21.15.119:0/185030077 >> v1:172.21.15.132:6789/0 conn(0x7f545c0b36a0 legacy=0x7f545c157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:49.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.231+0000 7f545aa00640 1 -- 172.21.15.119:0/185030077 >> v1:172.21.15.119:6789/0 conn(0x7f545c06aba0 legacy=0x7f545c0b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.231+0000 7f545aa00640 1 -- 172.21.15.119:0/185030077 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c15e6e0 con 0x7f545c0ada20 2024-04-25T10:07:49.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.231+0000 7f5459600640 1 -- 172.21.15.119:0/185030077 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990384044 0 0) 0x7f545c15e120 con 0x7f545c06aba0 2024-04-25T10:07:49.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.232+0000 7f545aa00640 1 --2- 172.21.15.119:0/185030077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c0ada20 0x7f545c0acb20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f544c0097a0 tx=0x7f544c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f1b9b38be7d3a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:49.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.232+0000 7f5459600640 1 -- 172.21.15.119:0/185030077 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c037070 con 0x7f545c0ada20 2024-04-25T10:07:49.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.232+0000 7f5459600640 1 -- 172.21.15.119:0/185030077 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c032440 con 0x7f545c0ada20 2024-04-25T10:07:49.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.232+0000 7f5459600640 1 -- 172.21.15.119:0/185030077 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c032810 con 0x7f545c0ada20 2024-04-25T10:07:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.233+0000 7f545be00640 1 -- 172.21.15.119:0/185030077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c0ada20 msgr2=0x7f545c0acb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.233+0000 7f545be00640 1 --2- 172.21.15.119:0/185030077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c0ada20 0x7f545c0acb20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f544c0097a0 tx=0x7f544c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.233+0000 7f545be00640 1 -- 172.21.15.119:0/185030077 shutdown_connections 2024-04-25T10:07:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.233+0000 7f545be00640 1 --2- 172.21.15.119:0/185030077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c0ada20 0x7f545c0acb20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.233+0000 7f545be00640 1 -- 172.21.15.119:0/185030077 >> 172.21.15.119:0/185030077 conn(0x7f545c066ff0 msgr2=0x7f545c0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.233+0000 7f545be00640 1 -- 172.21.15.119:0/185030077 shutdown_connections 2024-04-25T10:07:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.233+0000 7f545be00640 1 -- 172.21.15.119:0/185030077 wait complete. 2024-04-25T10:07:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.234+0000 7f545be00640 1 Processor -- start 2024-04-25T10:07:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.235+0000 7f545be00640 1 -- start start 2024-04-25T10:07:49.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.236+0000 7f545be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c06aba0 0x7f545c1508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.236+0000 7f545be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0ada20 0x7f545c0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.236+0000 7f545be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f545c0b36a0 0x7f545c0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.236+0000 7f545be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0ade00 con 0x7f545c06aba0 2024-04-25T10:07:49.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.236+0000 7f545be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c15eca0 con 0x7f545c0b36a0 2024-04-25T10:07:49.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.236+0000 7f545be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c06b050 con 0x7f545c0ada20 2024-04-25T10:07:49.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.236+0000 7f545aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c06aba0 0x7f545c1508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.236+0000 7f545aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c06aba0 0x7f545c1508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59210/0 (socket says 172.21.15.119:59210) 2024-04-25T10:07:49.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.236+0000 7f545aa00640 1 -- 172.21.15.119:0/1413300977 learned_addr learned my addr 172.21.15.119:0/1413300977 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:49.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.236+0000 7f545aa00640 1 -- 172.21.15.119:0/1413300977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0ada20 msgr2=0x7f545c0c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.236+0000 7f545aa00640 1 --2- 172.21.15.119:0/1413300977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0ada20 0x7f545c0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.236+0000 7f545aa00640 1 -- 172.21.15.119:0/1413300977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f545c0b36a0 msgr2=0x7f545c0c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.236+0000 7f545aa00640 1 --2- 172.21.15.119:0/1413300977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f545c0b36a0 0x7f545c0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.236+0000 7f545aa00640 1 -- 172.21.15.119:0/1413300977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c0c5f80 con 0x7f545c06aba0 2024-04-25T10:07:49.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.237+0000 7f545aa00640 1 --2- 172.21.15.119:0/1413300977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c06aba0 0x7f545c1508e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f544c0097a0 tx=0x7f544c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:49.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.237+0000 7f5452a00640 1 -- 172.21.15.119:0/1413300977 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c037040 con 0x7f545c06aba0 2024-04-25T10:07:49.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.237+0000 7f5452a00640 1 -- 172.21.15.119:0/1413300977 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c004250 con 0x7f545c06aba0 2024-04-25T10:07:49.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.237+0000 7f5452a00640 1 -- 172.21.15.119:0/1413300977 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c032e20 con 0x7f545c06aba0 2024-04-25T10:07:49.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.237+0000 7f545be00640 1 -- 172.21.15.119:0/1413300977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c0c6970 con 0x7f545c06aba0 2024-04-25T10:07:49.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.237+0000 7f545be00640 1 -- 172.21.15.119:0/1413300977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545c0c6f30 con 0x7f545c06aba0 2024-04-25T10:07:49.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.238+0000 7f545be00640 1 -- 172.21.15.119:0/1413300977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545c0749b0 con 0x7f545c06aba0 2024-04-25T10:07:49.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.239+0000 7f5452a00640 1 -- 172.21.15.119:0/1413300977 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f544c030770 con 0x7f545c06aba0 2024-04-25T10:07:49.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.240+0000 7f5452a00640 1 --2- 172.21.15.119:0/1413300977 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f543c06b4d0 0x7f543c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.240+0000 7f5452a00640 1 -- 172.21.15.119:0/1413300977 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f544c0ab5b0 con 0x7f545c06aba0 2024-04-25T10:07:49.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.240+0000 7f545a000640 1 --2- 172.21.15.119:0/1413300977 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f543c06b4d0 0x7f543c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.241+0000 7f545a000640 1 --2- 172.21.15.119:0/1413300977 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f543c06b4d0 0x7f543c06d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5448007fd0 tx=0x7f5448007920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:49.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.245+0000 7f5452a00640 1 -- 172.21.15.119:0/1413300977 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f544c07a450 con 0x7f545c06aba0 2024-04-25T10:07:49.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.435+0000 7f545be00640 1 -- 172.21.15.119:0/1413300977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f545c15d700 con 0x7f545c06aba0 2024-04-25T10:07:49.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.436+0000 7f5452a00640 1 -- 172.21.15.119:0/1413300977 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f544c079d50 con 0x7f545c06aba0 2024-04-25T10:07:49.441 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:07:49.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.438+0000 7f545be00640 1 -- 172.21.15.119:0/1413300977 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f543c06b4d0 msgr2=0x7f543c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.438+0000 7f545be00640 1 --2- 172.21.15.119:0/1413300977 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f543c06b4d0 0x7f543c06d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5448007fd0 tx=0x7f5448007920 comp rx=0 tx=0).stop 2024-04-25T10:07:49.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.438+0000 7f545be00640 1 -- 172.21.15.119:0/1413300977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c06aba0 msgr2=0x7f545c1508e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.438+0000 7f545be00640 1 --2- 172.21.15.119:0/1413300977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c06aba0 0x7f545c1508e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f544c0097a0 tx=0x7f544c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.438+0000 7f545be00640 1 -- 172.21.15.119:0/1413300977 shutdown_connections 2024-04-25T10:07:49.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.438+0000 7f545be00640 1 --2- 172.21.15.119:0/1413300977 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f543c06b4d0 0x7f543c06d990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.438+0000 7f545be00640 1 --2- 172.21.15.119:0/1413300977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f545c0b36a0 0x7f545c0c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.438+0000 7f545be00640 1 --2- 172.21.15.119:0/1413300977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0ada20 0x7f545c0c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.438+0000 7f545be00640 1 --2- 172.21.15.119:0/1413300977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c06aba0 0x7f545c1508e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.438+0000 7f545be00640 1 -- 172.21.15.119:0/1413300977 >> 172.21.15.119:0/1413300977 conn(0x7f545c066ff0 msgr2=0x7f545c0af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:49.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.438+0000 7f545be00640 1 -- 172.21.15.119:0/1413300977 shutdown_connections 2024-04-25T10:07:49.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.438+0000 7f545be00640 1 -- 172.21.15.119:0/1413300977 wait complete. 2024-04-25T10:07:49.462 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:49.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.655+0000 7f9bcce00640 1 Processor -- start 2024-04-25T10:07:49.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.656+0000 7f9bcce00640 1 -- start start 2024-04-25T10:07:49.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.657+0000 7f9bcce00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9bc815bca0 0x7f9bc80d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.657+0000 7f9bcce00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc815f980 con 0x7f9bc806b2b0 2024-04-25T10:07:49.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.657+0000 7f9bcce00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc815fc60 con 0x7f9bc80d1430 2024-04-25T10:07:49.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.657+0000 7f9bcce00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc80cc480 con 0x7f9bc815bca0 2024-04-25T10:07:49.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.658+0000 7f9bc6800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9bc806b2b0 0x7f9bc80cc8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36772/0 (socket says 172.21.15.119:36772) 2024-04-25T10:07:49.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.658+0000 7f9bc6800640 1 -- 172.21.15.119:0/3300693045 learned_addr learned my addr 172.21.15.119:0/3300693045 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:49.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.658+0000 7f9bc5e00640 1 --2- 172.21.15.119:0/3300693045 >> v2:172.21.15.132:3300/0 conn(0x7f9bc815bca0 0x7f9bc80d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.658+0000 7f9bc4a00640 1 -- 172.21.15.119:0/3300693045 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273586496 0 0) 0x7f9bc815f980 con 0x7f9bc806b2b0 2024-04-25T10:07:49.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.658+0000 7f9bc4a00640 1 -- 172.21.15.119:0/3300693045 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bac003520 con 0x7f9bc806b2b0 2024-04-25T10:07:49.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.658+0000 7f9bc4a00640 1 -- 172.21.15.119:0/3300693045 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3209963877 0 0) 0x7f9bac003520 con 0x7f9bc806b2b0 2024-04-25T10:07:49.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.658+0000 7f9bc4a00640 1 -- 172.21.15.119:0/3300693045 >> v2:172.21.15.132:3300/0 conn(0x7f9bc815bca0 msgr2=0x7f9bc80d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:49.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.658+0000 7f9bc4a00640 1 --2- 172.21.15.119:0/3300693045 >> v2:172.21.15.132:3300/0 conn(0x7f9bc815bca0 0x7f9bc80d0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.658+0000 7f9bc4a00640 1 -- 172.21.15.119:0/3300693045 >> v1:172.21.15.132:6789/0 conn(0x7f9bc80d1430 legacy=0x7f9bc806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc4a00640 1 -- 172.21.15.119:0/3300693045 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc815ff40 con 0x7f9bc806b2b0 2024-04-25T10:07:49.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc4a00640 1 -- 172.21.15.119:0/3300693045 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f9bc00019f0 con 0x7f9bc806b2b0 2024-04-25T10:07:49.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc4a00640 1 -- 172.21.15.119:0/3300693045 >> v1:172.21.15.119:6789/0 conn(0x7f9bc806b2b0 legacy=0x7f9bc80cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc4a00640 1 --2- 172.21.15.119:0/3300693045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bac003a00 0x7f9bac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc4a00640 1 --2- 172.21.15.119:0/3300693045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bac004800 0x7f9bac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc4a00640 1 --2- 172.21.15.119:0/3300693045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bac005210 0x7f9bac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc4a00640 1 -- 172.21.15.119:0/3300693045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac00c130 con 0x7f9bac005210 2024-04-25T10:07:49.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc4a00640 1 -- 172.21.15.119:0/3300693045 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac00c2e0 con 0x7f9bac004800 2024-04-25T10:07:49.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc4a00640 1 -- 172.21.15.119:0/3300693045 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac00c4e0 con 0x7f9bac003a00 2024-04-25T10:07:49.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc6800640 1 --2- 172.21.15.119:0/3300693045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bac005210 0x7f9bac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc5e00640 1 --2- 172.21.15.119:0/3300693045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bac003a00 0x7f9bac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc5400640 1 --2- 172.21.15.119:0/3300693045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bac004800 0x7f9bac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc6800640 1 -- 172.21.15.119:0/3300693045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bac003a00 msgr2=0x7f9bac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc6800640 1 --2- 172.21.15.119:0/3300693045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bac003a00 0x7f9bac003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc6800640 1 -- 172.21.15.119:0/3300693045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bac004800 msgr2=0x7f9bac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc6800640 1 --2- 172.21.15.119:0/3300693045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bac004800 0x7f9bac004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.659+0000 7f9bc6800640 1 -- 172.21.15.119:0/3300693045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac003520 con 0x7f9bac005210 2024-04-25T10:07:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.660+0000 7f9bc6800640 1 -- 172.21.15.119:0/3300693045 reap_dead start 2024-04-25T10:07:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.660+0000 7f9bc6800640 1 --2- 172.21.15.119:0/3300693045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bac005210 0x7f9bac00b9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8071980 tx=0x7f9bac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9b94ea2c2d6b4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.661+0000 7f9bc4a00640 1 -- 172.21.15.119:0/3300693045 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000a3d0 con 0x7f9bac005210 2024-04-25T10:07:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.661+0000 7f9bc4a00640 1 -- 172.21.15.119:0/3300693045 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc002f070 con 0x7f9bac005210 2024-04-25T10:07:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.661+0000 7f9bc4a00640 1 -- 172.21.15.119:0/3300693045 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc002bba0 con 0x7f9bac005210 2024-04-25T10:07:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.661+0000 7f9bcce00640 1 -- 172.21.15.119:0/3300693045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bac005210 msgr2=0x7f9bac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.661+0000 7f9bcce00640 1 --2- 172.21.15.119:0/3300693045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bac005210 0x7f9bac00b9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8071980 tx=0x7f9bac005830 comp rx=0 tx=0).stop 2024-04-25T10:07:49.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.661+0000 7f9bcce00640 1 -- 172.21.15.119:0/3300693045 shutdown_connections 2024-04-25T10:07:49.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.661+0000 7f9bcce00640 1 --2- 172.21.15.119:0/3300693045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bac005210 0x7f9bac00b9c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.661+0000 7f9bcce00640 1 -- 172.21.15.119:0/3300693045 >> 172.21.15.119:0/3300693045 conn(0x7f9bc8099a40 msgr2=0x7f9bc80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:49.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.661+0000 7f9bcce00640 1 -- 172.21.15.119:0/3300693045 shutdown_connections 2024-04-25T10:07:49.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.661+0000 7f9bcce00640 1 -- 172.21.15.119:0/3300693045 wait complete. 2024-04-25T10:07:49.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.663+0000 7f9bcce00640 1 Processor -- start 2024-04-25T10:07:49.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.663+0000 7f9bcce00640 1 -- start start 2024-04-25T10:07:49.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.664+0000 7f9bcce00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bac005210 0x7f9bc815e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.664+0000 7f9bcce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bc815d160 0x7f9bc80b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.664+0000 7f9bcce00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bc80b39c0 0x7f9bc80b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.664+0000 7f9bcce00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80d0ef0 con 0x7f9bc815d160 2024-04-25T10:07:49.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.664+0000 7f9bcce00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc815f610 con 0x7f9bac005210 2024-04-25T10:07:49.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.664+0000 7f9bcce00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80d1070 con 0x7f9bc80b39c0 2024-04-25T10:07:49.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.664+0000 7f9bc5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bc815d160 0x7f9bc80b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.664+0000 7f9bc5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bc815d160 0x7f9bc80b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59224/0 (socket says 172.21.15.119:59224) 2024-04-25T10:07:49.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.664+0000 7f9bc5400640 1 -- 172.21.15.119:0/1003549909 learned_addr learned my addr 172.21.15.119:0/1003549909 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:49.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.665+0000 7f9bc6800640 1 --2- 172.21.15.119:0/1003549909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bc80b39c0 0x7f9bc80b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.665+0000 7f9bc5e00640 1 --2- 172.21.15.119:0/1003549909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bac005210 0x7f9bc815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.665+0000 7f9bc5400640 1 -- 172.21.15.119:0/1003549909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bc80b39c0 msgr2=0x7f9bc80b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.665+0000 7f9bc5400640 1 --2- 172.21.15.119:0/1003549909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bc80b39c0 0x7f9bc80b5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.665+0000 7f9bc5400640 1 -- 172.21.15.119:0/1003549909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bac005210 msgr2=0x7f9bc815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.665+0000 7f9bc5400640 1 --2- 172.21.15.119:0/1003549909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bac005210 0x7f9bc815e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.665+0000 7f9bc5400640 1 -- 172.21.15.119:0/1003549909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc815d690 con 0x7f9bc815d160 2024-04-25T10:07:49.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.665+0000 7f9bc5400640 1 --2- 172.21.15.119:0/1003549909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bc815d160 0x7f9bc80b3480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400d570 tx=0x7f9bb400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:49.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.665+0000 7f9bbe000640 1 -- 172.21.15.119:0/1003549909 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4003610 con 0x7f9bc815d160 2024-04-25T10:07:49.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.665+0000 7f9bbe000640 1 -- 172.21.15.119:0/1003549909 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb4017070 con 0x7f9bc815d160 2024-04-25T10:07:49.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.665+0000 7f9bbe000640 1 -- 172.21.15.119:0/1003549909 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40039f0 con 0x7f9bc815d160 2024-04-25T10:07:49.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.665+0000 7f9bcce00640 1 -- 172.21.15.119:0/1003549909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc815d970 con 0x7f9bc815d160 2024-04-25T10:07:49.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.666+0000 7f9bcce00640 1 -- 172.21.15.119:0/1003549909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc815df30 con 0x7f9bc815d160 2024-04-25T10:07:49.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.667+0000 7f9bcce00640 1 -- 172.21.15.119:0/1003549909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc80750b0 con 0x7f9bc815d160 2024-04-25T10:07:49.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.667+0000 7f9bbe000640 1 -- 172.21.15.119:0/1003549909 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bb4003b90 con 0x7f9bc815d160 2024-04-25T10:07:49.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.668+0000 7f9bbe000640 1 --2- 172.21.15.119:0/1003549909 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ba806b4d0 0x7f9ba806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:49.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.668+0000 7f9bbe000640 1 -- 172.21.15.119:0/1003549909 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f9bb408a160 con 0x7f9bc815d160 2024-04-25T10:07:49.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.669+0000 7f9bc5e00640 1 --2- 172.21.15.119:0/1003549909 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ba806b4d0 0x7f9ba806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:49.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.669+0000 7f9bc5e00640 1 --2- 172.21.15.119:0/1003549909 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ba806b4d0 0x7f9ba806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009910 tx=0x7f9bb8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:49.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.673+0000 7f9bbe000640 1 -- 172.21.15.119:0/1003549909 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9bb4058fa0 con 0x7f9bc815d160 2024-04-25T10:07:49.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.886+0000 7f9bcce00640 1 -- 172.21.15.119:0/1003549909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9bc80cb6f0 con 0x7f9bc815d160 2024-04-25T10:07:49.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.891+0000 7f9bbe000640 1 -- 172.21.15.119:0/1003549909 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9bb40588a0 con 0x7f9bc815d160 2024-04-25T10:07:49.897 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2024-04-25T10:07:49.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.893+0000 7f9bcce00640 1 -- 172.21.15.119:0/1003549909 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ba806b4d0 msgr2=0x7f9ba806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.893+0000 7f9bcce00640 1 --2- 172.21.15.119:0/1003549909 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ba806b4d0 0x7f9ba806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009910 tx=0x7f9bb8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:49.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.893+0000 7f9bcce00640 1 -- 172.21.15.119:0/1003549909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bc815d160 msgr2=0x7f9bc80b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:49.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.893+0000 7f9bcce00640 1 --2- 172.21.15.119:0/1003549909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bc815d160 0x7f9bc80b3480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400d570 tx=0x7f9bb400da30 comp rx=0 tx=0).stop 2024-04-25T10:07:49.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.894+0000 7f9bcce00640 1 -- 172.21.15.119:0/1003549909 shutdown_connections 2024-04-25T10:07:49.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.894+0000 7f9bcce00640 1 --2- 172.21.15.119:0/1003549909 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ba806b4d0 0x7f9ba806d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.894+0000 7f9bcce00640 1 --2- 172.21.15.119:0/1003549909 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9bc80b39c0 0x7f9bc80b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.894+0000 7f9bcce00640 1 --2- 172.21.15.119:0/1003549909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bc815d160 0x7f9bc80b3480 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.894+0000 7f9bcce00640 1 --2- 172.21.15.119:0/1003549909 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bac005210 0x7f9bc815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:49.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.894+0000 7f9bcce00640 1 -- 172.21.15.119:0/1003549909 >> 172.21.15.119:0/1003549909 conn(0x7f9bc8099a40 msgr2=0x7f9bc80b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:49.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.894+0000 7f9bcce00640 1 -- 172.21.15.119:0/1003549909 shutdown_connections 2024-04-25T10:07:49.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:49.894+0000 7f9bcce00640 1 -- 172.21.15.119:0/1003549909 wait complete. 2024-04-25T10:07:49.921 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:07:49.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:49.921 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T10:07:50.008 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:07:50.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:50.008 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:07:50.075 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:07:50.075 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:50.135 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:07:50.135 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:50.190 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:07:50.190 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:50.240 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:50.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.439+0000 7fcedf200640 1 Processor -- start 2024-04-25T10:07:50.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.439+0000 7fcedf200640 1 -- start start 2024-04-25T10:07:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.440+0000 7fcedf200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcee0144470 0x7fcee015fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.440+0000 7fcedf200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcee00b5050 0x7fcee015bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.441+0000 7fcedf200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee00cd440 con 0x7fcee006e430 2024-04-25T10:07:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.441+0000 7fcedf200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee0145340 con 0x7fcee00b5050 2024-04-25T10:07:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.441+0000 7fcedf200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee00cd720 con 0x7fcee0144470 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.441+0000 7fcede800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcee00b5050 0x7fcee015bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.441+0000 7fcede800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcee00b5050 0x7fcee015bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59230/0 (socket says 172.21.15.119:59230) 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.441+0000 7fcede800640 1 -- 172.21.15.119:0/2328670429 learned_addr learned my addr 172.21.15.119:0/2328670429 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.441+0000 7fcede800640 1 -- 172.21.15.119:0/2328670429 >> v2:172.21.15.132:3300/0 conn(0x7fcee0144470 msgr2=0x7fcee015fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.441+0000 7fcede800640 1 --2- 172.21.15.119:0/2328670429 >> v2:172.21.15.132:3300/0 conn(0x7fcee0144470 0x7fcee015fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.441+0000 7fcede800640 1 -- 172.21.15.119:0/2328670429 >> v1:172.21.15.132:6789/0 conn(0x7fcee006e430 legacy=0x7fcee00b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.441+0000 7fcede800640 1 -- 172.21.15.119:0/2328670429 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee00cda00 con 0x7fcee00b5050 2024-04-25T10:07:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.442+0000 7fcede800640 1 --2- 172.21.15.119:0/2328670429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee00b5050 0x7fcee015bcf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcee00b5490 tx=0x7fced802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedf4ce770e98265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.443+0000 7fcedd400640 1 -- 172.21.15.119:0/2328670429 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8037070 con 0x7fcee00b5050 2024-04-25T10:07:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.443+0000 7fcedd400640 1 -- 172.21.15.119:0/2328670429 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced8032440 con 0x7fcee00b5050 2024-04-25T10:07:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.443+0000 7fcedd400640 1 -- 172.21.15.119:0/2328670429 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8032810 con 0x7fcee00b5050 2024-04-25T10:07:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.443+0000 7fcedf200640 1 -- 172.21.15.119:0/2328670429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee00b5050 msgr2=0x7fcee015bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.443+0000 7fcedf200640 1 --2- 172.21.15.119:0/2328670429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee00b5050 0x7fcee015bcf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcee00b5490 tx=0x7fced802a230 comp rx=0 tx=0).stop 2024-04-25T10:07:50.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.443+0000 7fcedf200640 1 -- 172.21.15.119:0/2328670429 shutdown_connections 2024-04-25T10:07:50.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.443+0000 7fcedf200640 1 --2- 172.21.15.119:0/2328670429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee00b5050 0x7fcee015bcf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.443+0000 7fcedf200640 1 --2- 172.21.15.119:0/2328670429 >> v2:172.21.15.132:3300/0 conn(0x7fcee0144470 0x7fcee015fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.443+0000 7fcedf200640 1 -- 172.21.15.119:0/2328670429 >> 172.21.15.119:0/2328670429 conn(0x7fcee006b1d0 msgr2=0x7fcee00b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:50.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.444+0000 7fcedf200640 1 -- 172.21.15.119:0/2328670429 shutdown_connections 2024-04-25T10:07:50.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.444+0000 7fcedf200640 1 -- 172.21.15.119:0/2328670429 wait complete. 2024-04-25T10:07:50.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.445+0000 7fcedf200640 1 Processor -- start 2024-04-25T10:07:50.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.445+0000 7fcedf200640 1 -- start start 2024-04-25T10:07:50.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.446+0000 7fcedf200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee006e430 0x7fcee00aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.446+0000 7fcedf200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee00b5050 0x7fcee013f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.446+0000 7fcedf200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee0144470 0x7fcee013fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.446+0000 7fcedf200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee00ce200 con 0x7fcee006e430 2024-04-25T10:07:50.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.446+0000 7fcedf200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee0144850 con 0x7fcee0144470 2024-04-25T10:07:50.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.446+0000 7fcedf200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee015c230 con 0x7fcee00b5050 2024-04-25T10:07:50.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.447+0000 7fcedde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee006e430 0x7fcee00aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.447+0000 7fcedde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee006e430 0x7fcee00aaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59242/0 (socket says 172.21.15.119:59242) 2024-04-25T10:07:50.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.447+0000 7fcedde00640 1 -- 172.21.15.119:0/326163263 learned_addr learned my addr 172.21.15.119:0/326163263 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:50.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.447+0000 7fced5400640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee00b5050 0x7fcee013f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.447+0000 7fcedde00640 1 -- 172.21.15.119:0/326163263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee00b5050 msgr2=0x7fcee013f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.447+0000 7fcedde00640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee00b5050 0x7fcee013f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.447+0000 7fcedde00640 1 -- 172.21.15.119:0/326163263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee0144470 msgr2=0x7fcee013fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.447+0000 7fcede800640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee0144470 0x7fcee013fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.447+0000 7fcedde00640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee0144470 0x7fcee013fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.447+0000 7fcedde00640 1 -- 172.21.15.119:0/326163263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee01e6000 con 0x7fcee006e430 2024-04-25T10:07:50.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.447+0000 7fced5400640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee00b5050 0x7fcee013f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:50.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.447+0000 7fcedde00640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee006e430 0x7fcee00aaab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fced000ba50 tx=0x7fced000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:50.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.448+0000 7fced6a00640 1 -- 172.21.15.119:0/326163263 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0019070 con 0x7fcee006e430 2024-04-25T10:07:50.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.448+0000 7fced6a00640 1 -- 172.21.15.119:0/326163263 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced000cc00 con 0x7fcee006e430 2024-04-25T10:07:50.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.448+0000 7fced6a00640 1 -- 172.21.15.119:0/326163263 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0014440 con 0x7fcee006e430 2024-04-25T10:07:50.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.448+0000 7fcedf200640 1 -- 172.21.15.119:0/326163263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee00a9590 con 0x7fcee006e430 2024-04-25T10:07:50.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.448+0000 7fcedf200640 1 -- 172.21.15.119:0/326163263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee00a9b50 con 0x7fcee006e430 2024-04-25T10:07:50.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.449+0000 7fcedf200640 1 -- 172.21.15.119:0/326163263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee00749b0 con 0x7fcee006e430 2024-04-25T10:07:50.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.450+0000 7fced6a00640 1 -- 172.21.15.119:0/326163263 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fced001c070 con 0x7fcee006e430 2024-04-25T10:07:50.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.451+0000 7fced6a00640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcebc06b4d0 0x7fcebc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.451+0000 7fced6a00640 1 -- 172.21.15.119:0/326163263 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fced0089fc0 con 0x7fcee006e430 2024-04-25T10:07:50.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.451+0000 7fced5400640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcebc06b4d0 0x7fcebc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.452+0000 7fced5400640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcebc06b4d0 0x7fcebc06d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcec8007fd0 tx=0x7fcec8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:50.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.455+0000 7fced6a00640 1 -- 172.21.15.119:0/326163263 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fced0058dd0 con 0x7fcee006e430 2024-04-25T10:07:50.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.655+0000 7fcedf200640 1 -- 172.21.15.119:0/326163263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcee00b2090 con 0x7fcee006e430 2024-04-25T10:07:50.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.656+0000 7fced6a00640 1 -- 172.21.15.119:0/326163263 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16861 (secure 0 0 0) 0x7fced00586d0 con 0x7fcee006e430 2024-04-25T10:07:50.663 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:50.663 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":22,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:49.883606+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.659+0000 7fcedf200640 1 -- 172.21.15.119:0/326163263 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcebc06b4d0 msgr2=0x7fcebc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.659+0000 7fcedf200640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcebc06b4d0 0x7fcebc06d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcec8007fd0 tx=0x7fcec8009040 comp rx=0 tx=0).stop 2024-04-25T10:07:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.659+0000 7fcedf200640 1 -- 172.21.15.119:0/326163263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee006e430 msgr2=0x7fcee00aaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.659+0000 7fcedf200640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee006e430 0x7fcee00aaab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fced000ba50 tx=0x7fced000bf10 comp rx=0 tx=0).stop 2024-04-25T10:07:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.660+0000 7fcedf200640 1 -- 172.21.15.119:0/326163263 shutdown_connections 2024-04-25T10:07:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.660+0000 7fcedf200640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcebc06b4d0 0x7fcebc06d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.660+0000 7fcedf200640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee0144470 0x7fcee013fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.660+0000 7fcedf200640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcee00b5050 0x7fcee013f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.660+0000 7fcedf200640 1 --2- 172.21.15.119:0/326163263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcee006e430 0x7fcee00aaab0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.660+0000 7fcedf200640 1 -- 172.21.15.119:0/326163263 >> 172.21.15.119:0/326163263 conn(0x7fcee006b1d0 msgr2=0x7fcee006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.660+0000 7fcedf200640 1 -- 172.21.15.119:0/326163263 shutdown_connections 2024-04-25T10:07:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.660+0000 7fcedf200640 1 -- 172.21.15.119:0/326163263 wait complete. 2024-04-25T10:07:50.684 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:05.135832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:05.768083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:50.685 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:50.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.880+0000 7f2037e00640 1 Processor -- start 2024-04-25T10:07:50.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.880+0000 7f2037e00640 1 -- start start 2024-04-25T10:07:50.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.881+0000 7f2037e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f20380b3e90 0x7f20380cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.881+0000 7f2037e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203815f400 con 0x7f203806b2b0 2024-04-25T10:07:50.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.881+0000 7f2037e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203815f6e0 con 0x7f20381450e0 2024-04-25T10:07:50.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.881+0000 7f2037e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20381454c0 con 0x7f20380b3e90 2024-04-25T10:07:50.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.882+0000 7f2036000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f203806b2b0 0x7f20380cc2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34806/0 (socket says 172.21.15.119:34806) 2024-04-25T10:07:50.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.882+0000 7f2036000640 1 -- 172.21.15.119:0/1565604901 learned_addr learned my addr 172.21.15.119:0/1565604901 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:50.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.882+0000 7f2037400640 1 --2- 172.21.15.119:0/1565604901 >> v2:172.21.15.132:3300/0 conn(0x7f20380b3e90 0x7f20380cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.883+0000 7f2035600640 1 -- 172.21.15.119:0/1565604901 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061784131 0 0) 0x7f203815f400 con 0x7f203806b2b0 2024-04-25T10:07:50.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.883+0000 7f2035600640 1 -- 172.21.15.119:0/1565604901 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f201c003520 con 0x7f203806b2b0 2024-04-25T10:07:50.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.883+0000 7f2037400640 1 -- 172.21.15.119:0/1565604901 >> v1:172.21.15.132:6789/0 conn(0x7f20381450e0 legacy=0x7f203806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.883+0000 7f2037400640 1 -- 172.21.15.119:0/1565604901 >> v1:172.21.15.119:6789/0 conn(0x7f203806b2b0 legacy=0x7f20380cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.883+0000 7f2037400640 1 -- 172.21.15.119:0/1565604901 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203815f9c0 con 0x7f20380b3e90 2024-04-25T10:07:50.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.884+0000 7f2037400640 1 --2- 172.21.15.119:0/1565604901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20380b3e90 0x7f20380cc9e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2030009d90 tx=0x7f203002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2259b4b3d566bceb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:50.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.885+0000 7f2035600640 1 -- 172.21.15.119:0/1565604901 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030031600 con 0x7f20380b3e90 2024-04-25T10:07:50.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.885+0000 7f2035600640 1 -- 172.21.15.119:0/1565604901 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20300317a0 con 0x7f20380b3e90 2024-04-25T10:07:50.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.885+0000 7f2035600640 1 -- 172.21.15.119:0/1565604901 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030031b70 con 0x7f20380b3e90 2024-04-25T10:07:50.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.886+0000 7f2037e00640 1 -- 172.21.15.119:0/1565604901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20380b3e90 msgr2=0x7f20380cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.886+0000 7f2037e00640 1 --2- 172.21.15.119:0/1565604901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20380b3e90 0x7f20380cc9e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2030009d90 tx=0x7f203002a5d0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.886+0000 7f2037e00640 1 -- 172.21.15.119:0/1565604901 shutdown_connections 2024-04-25T10:07:50.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.886+0000 7f2037e00640 1 --2- 172.21.15.119:0/1565604901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20380b3e90 0x7f20380cc9e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.886+0000 7f2037e00640 1 -- 172.21.15.119:0/1565604901 >> 172.21.15.119:0/1565604901 conn(0x7f2038066ff0 msgr2=0x7f20380b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:50.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.886+0000 7f2037e00640 1 -- 172.21.15.119:0/1565604901 shutdown_connections 2024-04-25T10:07:50.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.886+0000 7f2037e00640 1 -- 172.21.15.119:0/1565604901 wait complete. 2024-04-25T10:07:50.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.888+0000 7f2037e00640 1 Processor -- start 2024-04-25T10:07:50.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.888+0000 7f2037e00640 1 -- start start 2024-04-25T10:07:50.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.890+0000 7f2037e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f203806b2b0 0x7f203815aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.890+0000 7f2037e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20380b3e90 0x7f203813db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.890+0000 7f2037e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20381450e0 0x7f2038141260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.890+0000 7f2037e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20381666c0 con 0x7f20380b3e90 2024-04-25T10:07:50.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.890+0000 7f2037e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2038166540 con 0x7f203806b2b0 2024-04-25T10:07:50.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.890+0000 7f2037e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2038166840 con 0x7f20381450e0 2024-04-25T10:07:50.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.890+0000 7f2036000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20380b3e90 0x7f203813db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.890+0000 7f2036000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20380b3e90 0x7f203813db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48904/0 (socket says 172.21.15.119:48904) 2024-04-25T10:07:50.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.890+0000 7f2036000640 1 -- 172.21.15.119:0/1873038441 learned_addr learned my addr 172.21.15.119:0/1873038441 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:50.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.890+0000 7f2036a00640 1 --2- 172.21.15.119:0/1873038441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f203806b2b0 0x7f203815aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.891+0000 7f2036000640 1 -- 172.21.15.119:0/1873038441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20381450e0 msgr2=0x7f2038141260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.891+0000 7f2036000640 1 --2- 172.21.15.119:0/1873038441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20381450e0 0x7f2038141260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.891+0000 7f2036000640 1 -- 172.21.15.119:0/1873038441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f203806b2b0 msgr2=0x7f203815aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:50.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.891+0000 7f2036000640 1 --2- 172.21.15.119:0/1873038441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f203806b2b0 0x7f203815aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:50.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.891+0000 7f2036000640 1 -- 172.21.15.119:0/1873038441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2038140d80 con 0x7f20380b3e90 2024-04-25T10:07:50.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.891+0000 7f2036000640 1 --2- 172.21.15.119:0/1873038441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20380b3e90 0x7f203813db70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f202400c500 tx=0x7f202400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:50.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.891+0000 7f202ea00640 1 -- 172.21.15.119:0/1873038441 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024017070 con 0x7f20380b3e90 2024-04-25T10:07:50.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.891+0000 7f202ea00640 1 -- 172.21.15.119:0/1873038441 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2024005ce0 con 0x7f20380b3e90 2024-04-25T10:07:50.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.891+0000 7f2037e00640 1 -- 172.21.15.119:0/1873038441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2038166200 con 0x7f20380b3e90 2024-04-25T10:07:50.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.891+0000 7f202ea00640 1 -- 172.21.15.119:0/1873038441 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024007330 con 0x7f20380b3e90 2024-04-25T10:07:50.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.891+0000 7f2037e00640 1 -- 172.21.15.119:0/1873038441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2038165420 con 0x7f20380b3e90 2024-04-25T10:07:50.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.893+0000 7f2037e00640 1 -- 172.21.15.119:0/1873038441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2038140f50 con 0x7f20380b3e90 2024-04-25T10:07:50.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.893+0000 7f202ea00640 1 -- 172.21.15.119:0/1873038441 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f202401a070 con 0x7f20380b3e90 2024-04-25T10:07:50.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.894+0000 7f202ea00640 1 --2- 172.21.15.119:0/1873038441 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f201806b500 0x7f201806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:50.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.894+0000 7f202ea00640 1 -- 172.21.15.119:0/1873038441 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f2024089650 con 0x7f20380b3e90 2024-04-25T10:07:50.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.895+0000 7f2036a00640 1 --2- 172.21.15.119:0/1873038441 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f201806b500 0x7f201806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:50.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.896+0000 7f2036a00640 1 --2- 172.21.15.119:0/1873038441 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f201806b500 0x7f201806d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2028007fd0 tx=0x7f2028009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:50.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:50.899+0000 7f202ea00640 1 -- 172.21.15.119:0/1873038441 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20240585c0 con 0x7f20380b3e90 2024-04-25T10:07:51.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.095+0000 7f2037e00640 1 -- 172.21.15.119:0/1873038441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2038142fe0 con 0x7f20380b3e90 2024-04-25T10:07:51.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.096+0000 7f202ea00640 1 -- 172.21.15.119:0/1873038441 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2024057ec0 con 0x7f20380b3e90 2024-04-25T10:07:51.102 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:07:51.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.099+0000 7f2037e00640 1 -- 172.21.15.119:0/1873038441 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f201806b500 msgr2=0x7f201806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.099+0000 7f2037e00640 1 --2- 172.21.15.119:0/1873038441 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f201806b500 0x7f201806d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2028007fd0 tx=0x7f2028009040 comp rx=0 tx=0).stop 2024-04-25T10:07:51.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.099+0000 7f2037e00640 1 -- 172.21.15.119:0/1873038441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20380b3e90 msgr2=0x7f203813db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.099+0000 7f2037e00640 1 --2- 172.21.15.119:0/1873038441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20380b3e90 0x7f203813db70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f202400c500 tx=0x7f202400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.099+0000 7f2037e00640 1 -- 172.21.15.119:0/1873038441 shutdown_connections 2024-04-25T10:07:51.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.099+0000 7f2037e00640 1 --2- 172.21.15.119:0/1873038441 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f201806b500 0x7f201806d9c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.099+0000 7f2037e00640 1 --2- 172.21.15.119:0/1873038441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20381450e0 0x7f2038141260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.099+0000 7f2037e00640 1 --2- 172.21.15.119:0/1873038441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20380b3e90 0x7f203813db70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.099+0000 7f2037e00640 1 --2- 172.21.15.119:0/1873038441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f203806b2b0 0x7f203815aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.099+0000 7f2037e00640 1 -- 172.21.15.119:0/1873038441 >> 172.21.15.119:0/1873038441 conn(0x7f2038066ff0 msgr2=0x7f203813e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:51.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.099+0000 7f2037e00640 1 -- 172.21.15.119:0/1873038441 shutdown_connections 2024-04-25T10:07:51.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.099+0000 7f2037e00640 1 -- 172.21.15.119:0/1873038441 wait complete. 2024-04-25T10:07:51.123 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:51.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.321+0000 7f5924e00640 1 Processor -- start 2024-04-25T10:07:51.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.321+0000 7f5924e00640 1 -- start start 2024-04-25T10:07:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.323+0000 7f5924e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f592015dc80 0x7f5920161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.323+0000 7f5924e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59200b5c00 0x7f59200d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.323+0000 7f5924e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59201627a0 con 0x7f59200b97c0 2024-04-25T10:07:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.323+0000 7f5924e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59201621e0 con 0x7f59200b5c00 2024-04-25T10:07:51.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.323+0000 7f5924e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59201624c0 con 0x7f592015dc80 2024-04-25T10:07:51.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.323+0000 7f591e800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59200b5c00 0x7f59200d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.323+0000 7f591e800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59200b5c00 0x7f59200d2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48906/0 (socket says 172.21.15.119:48906) 2024-04-25T10:07:51.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.323+0000 7f591e800640 1 -- 172.21.15.119:0/3004310981 learned_addr learned my addr 172.21.15.119:0/3004310981 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:51.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.323+0000 7f591de00640 1 --2- 172.21.15.119:0/3004310981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f592015dc80 0x7f5920161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.323+0000 7f591e800640 1 -- 172.21.15.119:0/3004310981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f592015dc80 msgr2=0x7f5920161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:51.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.323+0000 7f591e800640 1 --2- 172.21.15.119:0/3004310981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f592015dc80 0x7f5920161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.323+0000 7f591e800640 1 -- 172.21.15.119:0/3004310981 >> v1:172.21.15.132:6789/0 conn(0x7f59200b97c0 legacy=0x7f59200b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:51.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.323+0000 7f591e800640 1 -- 172.21.15.119:0/3004310981 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59200d1c70 con 0x7f59200b5c00 2024-04-25T10:07:51.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.324+0000 7f591e800640 1 --2- 172.21.15.119:0/3004310981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59200b5c00 0x7f59200d2470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5918009870 tx=0x7f591802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60b852aa47c996b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:51.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.325+0000 7f591ca00640 1 -- 172.21.15.119:0/3004310981 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918037070 con 0x7f59200b5c00 2024-04-25T10:07:51.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.325+0000 7f591ca00640 1 -- 172.21.15.119:0/3004310981 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5918032440 con 0x7f59200b5c00 2024-04-25T10:07:51.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.325+0000 7f591ca00640 1 -- 172.21.15.119:0/3004310981 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918032810 con 0x7f59200b5c00 2024-04-25T10:07:51.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.325+0000 7f5924e00640 1 -- 172.21.15.119:0/3004310981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59200b5c00 msgr2=0x7f59200d2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.325+0000 7f5924e00640 1 --2- 172.21.15.119:0/3004310981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59200b5c00 0x7f59200d2470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5918009870 tx=0x7f591802a210 comp rx=0 tx=0).stop 2024-04-25T10:07:51.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.326+0000 7f5924e00640 1 -- 172.21.15.119:0/3004310981 shutdown_connections 2024-04-25T10:07:51.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.326+0000 7f5924e00640 1 --2- 172.21.15.119:0/3004310981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59200b5c00 0x7f59200d2470 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.326+0000 7f5924e00640 1 --2- 172.21.15.119:0/3004310981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f592015dc80 0x7f5920161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.326+0000 7f5924e00640 1 -- 172.21.15.119:0/3004310981 >> 172.21.15.119:0/3004310981 conn(0x7f59200d04f0 msgr2=0x7f59200d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:51.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.326+0000 7f5924e00640 1 -- 172.21.15.119:0/3004310981 shutdown_connections 2024-04-25T10:07:51.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.326+0000 7f5924e00640 1 -- 172.21.15.119:0/3004310981 wait complete. 2024-04-25T10:07:51.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.327+0000 7f5924e00640 1 Processor -- start 2024-04-25T10:07:51.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.328+0000 7f5924e00640 1 -- start start 2024-04-25T10:07:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.328+0000 7f5924e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59200b5c00 0x7f59200cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.328+0000 7f5924e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59200b97c0 0x7f59200b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.328+0000 7f5924e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f592015dc80 0x7f592014f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.328+0000 7f5924e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5920161f30 con 0x7f59200b5c00 2024-04-25T10:07:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.328+0000 7f5924e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5920161db0 con 0x7f592015dc80 2024-04-25T10:07:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.328+0000 7f5924e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59200d29b0 con 0x7f59200b97c0 2024-04-25T10:07:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.329+0000 7f591de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59200b5c00 0x7f59200cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.329+0000 7f591de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59200b5c00 0x7f59200cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48918/0 (socket says 172.21.15.119:48918) 2024-04-25T10:07:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.329+0000 7f591de00640 1 -- 172.21.15.119:0/513139971 learned_addr learned my addr 172.21.15.119:0/513139971 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.329+0000 7f591e800640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f592015dc80 0x7f592014f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.329+0000 7f591d400640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59200b97c0 0x7f59200b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.329+0000 7f591de00640 1 -- 172.21.15.119:0/513139971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59200b97c0 msgr2=0x7f59200b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.329+0000 7f591de00640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59200b97c0 0x7f59200b22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.329+0000 7f591de00640 1 -- 172.21.15.119:0/513139971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f592015dc80 msgr2=0x7f592014f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.329+0000 7f591de00640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f592015dc80 0x7f592014f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.329+0000 7f591de00640 1 -- 172.21.15.119:0/513139971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592014fc30 con 0x7f59200b5c00 2024-04-25T10:07:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.329+0000 7f591d400640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59200b97c0 0x7f59200b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.330+0000 7f591de00640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59200b5c00 0x7f59200cab40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f591000ccd0 tx=0x7f591000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.330+0000 7f5916000640 1 -- 172.21.15.119:0/513139971 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910003990 con 0x7f59200b5c00 2024-04-25T10:07:51.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.330+0000 7f5916000640 1 -- 172.21.15.119:0/513139971 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5910003b30 con 0x7f59200b5c00 2024-04-25T10:07:51.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.330+0000 7f5916000640 1 -- 172.21.15.119:0/513139971 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910004020 con 0x7f59200b5c00 2024-04-25T10:07:51.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.330+0000 7f5924e00640 1 -- 172.21.15.119:0/513139971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59200cb190 con 0x7f59200b5c00 2024-04-25T10:07:51.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.330+0000 7f5924e00640 1 -- 172.21.15.119:0/513139971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59200cb750 con 0x7f59200b5c00 2024-04-25T10:07:51.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.331+0000 7f5924e00640 1 -- 172.21.15.119:0/513139971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59200750b0 con 0x7f59200b5c00 2024-04-25T10:07:51.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.332+0000 7f5916000640 1 -- 172.21.15.119:0/513139971 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59100041c0 con 0x7f59200b5c00 2024-04-25T10:07:51.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.333+0000 7f5916000640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f58fc06b4d0 0x7f58fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.333+0000 7f5916000640 1 -- 172.21.15.119:0/513139971 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f591008a7e0 con 0x7f59200b5c00 2024-04-25T10:07:51.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.333+0000 7f591d400640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f58fc06b4d0 0x7f58fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.334+0000 7f591d400640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f58fc06b4d0 0x7f58fc06d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f59201f7110 tx=0x7f5908009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:51.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.337+0000 7f5916000640 1 -- 172.21.15.119:0/513139971 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5910014030 con 0x7f59200b5c00 2024-04-25T10:07:51.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.533+0000 7f5924e00640 1 -- 172.21.15.119:0/513139971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59200d1780 con 0x7f59200b5c00 2024-04-25T10:07:51.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.533+0000 7f5916000640 1 -- 172.21.15.119:0/513139971 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5910059560 con 0x7f59200b5c00 2024-04-25T10:07:51.540 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:07:51.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.536+0000 7f5924e00640 1 -- 172.21.15.119:0/513139971 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f58fc06b4d0 msgr2=0x7f58fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.536+0000 7f5924e00640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f58fc06b4d0 0x7f58fc06d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f59201f7110 tx=0x7f5908009040 comp rx=0 tx=0).stop 2024-04-25T10:07:51.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.536+0000 7f5924e00640 1 -- 172.21.15.119:0/513139971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59200b5c00 msgr2=0x7f59200cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.536+0000 7f5924e00640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59200b5c00 0x7f59200cab40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f591000ccd0 tx=0x7f591000a5c0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.536+0000 7f5924e00640 1 -- 172.21.15.119:0/513139971 shutdown_connections 2024-04-25T10:07:51.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.536+0000 7f5924e00640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f58fc06b4d0 0x7f58fc06d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.536+0000 7f5924e00640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f592015dc80 0x7f592014f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.536+0000 7f5924e00640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59200b97c0 0x7f59200b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.536+0000 7f5924e00640 1 --2- 172.21.15.119:0/513139971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59200b5c00 0x7f59200cab40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.536+0000 7f5924e00640 1 -- 172.21.15.119:0/513139971 >> 172.21.15.119:0/513139971 conn(0x7f59200d04f0 msgr2=0x7f5920160360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:51.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.536+0000 7f5924e00640 1 -- 172.21.15.119:0/513139971 shutdown_connections 2024-04-25T10:07:51.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.536+0000 7f5924e00640 1 -- 172.21.15.119:0/513139971 wait complete. 2024-04-25T10:07:51.562 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:51.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.767+0000 7fbffb800640 1 Processor -- start 2024-04-25T10:07:51.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.767+0000 7fbffb800640 1 -- start start 2024-04-25T10:07:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.769+0000 7fbffb800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbff406aba0 0x7fbff40b1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.769+0000 7fbffb800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbff40ce3a0 0x7fbff406af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.769+0000 7fbffb800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff406b4c0 con 0x7fbff4161930 2024-04-25T10:07:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.769+0000 7fbffb800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff40b0130 con 0x7fbff40ce3a0 2024-04-25T10:07:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.769+0000 7fbffb800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff40b09d0 con 0x7fbff406aba0 2024-04-25T10:07:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.769+0000 7fbff3e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbff40ce3a0 0x7fbff406af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.769+0000 7fbff9800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbff4161930 0x7fbff4161dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34808/0 (socket says 172.21.15.119:34808) 2024-04-25T10:07:51.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.769+0000 7fbff9800640 1 -- 172.21.15.119:0/2458826294 learned_addr learned my addr 172.21.15.119:0/2458826294 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:51.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.770+0000 7fbff8e00640 1 --2- 172.21.15.119:0/2458826294 >> v2:172.21.15.132:3300/0 conn(0x7fbff406aba0 0x7fbff40b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.770+0000 7fbff3e00640 1 -- 172.21.15.119:0/2458826294 >> v2:172.21.15.132:3300/0 conn(0x7fbff406aba0 msgr2=0x7fbff40b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:51.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.770+0000 7fbff3e00640 1 --2- 172.21.15.119:0/2458826294 >> v2:172.21.15.132:3300/0 conn(0x7fbff406aba0 0x7fbff40b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.770+0000 7fbff3e00640 1 -- 172.21.15.119:0/2458826294 >> v1:172.21.15.119:6789/0 conn(0x7fbff4161930 legacy=0x7fbff4161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.770+0000 7fbff3e00640 1 -- 172.21.15.119:0/2458826294 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40b1f60 con 0x7fbff40ce3a0 2024-04-25T10:07:51.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.770+0000 7fbff3400640 1 -- 172.21.15.119:0/2458826294 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211704885 0 0) 0x7fbff406b4c0 con 0x7fbff4161930 2024-04-25T10:07:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.771+0000 7fbff3e00640 1 --2- 172.21.15.119:0/2458826294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff40ce3a0 0x7fbff406af80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00097a0 tx=0x7fbfe002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da18131cdcc0cd95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.771+0000 7fbff3400640 1 -- 172.21.15.119:0/2458826294 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0037070 con 0x7fbff40ce3a0 2024-04-25T10:07:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.771+0000 7fbff3400640 1 -- 172.21.15.119:0/2458826294 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe0032440 con 0x7fbff40ce3a0 2024-04-25T10:07:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.771+0000 7fbff3400640 1 -- 172.21.15.119:0/2458826294 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0032810 con 0x7fbff40ce3a0 2024-04-25T10:07:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.772+0000 7fbffb800640 1 -- 172.21.15.119:0/2458826294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff40ce3a0 msgr2=0x7fbff406af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.772+0000 7fbffb800640 1 --2- 172.21.15.119:0/2458826294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff40ce3a0 0x7fbff406af80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00097a0 tx=0x7fbfe002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.772+0000 7fbffb800640 1 -- 172.21.15.119:0/2458826294 shutdown_connections 2024-04-25T10:07:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.772+0000 7fbffb800640 1 --2- 172.21.15.119:0/2458826294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff40ce3a0 0x7fbff406af80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.772+0000 7fbffb800640 1 --2- 172.21.15.119:0/2458826294 >> v2:172.21.15.132:3300/0 conn(0x7fbff406aba0 0x7fbff40b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.772+0000 7fbffb800640 1 -- 172.21.15.119:0/2458826294 >> 172.21.15.119:0/2458826294 conn(0x7fbff40d07d0 msgr2=0x7fbff40d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.772+0000 7fbffb800640 1 -- 172.21.15.119:0/2458826294 shutdown_connections 2024-04-25T10:07:51.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.772+0000 7fbffb800640 1 -- 172.21.15.119:0/2458826294 wait complete. 2024-04-25T10:07:51.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.774+0000 7fbffb800640 1 Processor -- start 2024-04-25T10:07:51.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.774+0000 7fbffb800640 1 -- start start 2024-04-25T10:07:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.775+0000 7fbffb800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbff406aba0 0x7fbff41f3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.775+0000 7fbffb800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbff40ce3a0 0x7fbff40b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.775+0000 7fbffb800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff4161930 0x7fbff406f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.775+0000 7fbffb800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff406b880 con 0x7fbff4161930 2024-04-25T10:07:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.775+0000 7fbffb800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40b0660 con 0x7fbff40ce3a0 2024-04-25T10:07:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.775+0000 7fbffb800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40b0e60 con 0x7fbff406aba0 2024-04-25T10:07:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.775+0000 7fbff9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff4161930 0x7fbff406f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.775+0000 7fbff9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff4161930 0x7fbff406f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48932/0 (socket says 172.21.15.119:48932) 2024-04-25T10:07:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.775+0000 7fbff9800640 1 -- 172.21.15.119:0/3266354821 learned_addr learned my addr 172.21.15.119:0/3266354821 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.775+0000 7fbff3e00640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbff40ce3a0 0x7fbff40b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.775+0000 7fbff8e00640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbff406aba0 0x7fbff41f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.776+0000 7fbff9800640 1 -- 172.21.15.119:0/3266354821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbff406aba0 msgr2=0x7fbff41f3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.776+0000 7fbff9800640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbff406aba0 0x7fbff41f3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.776+0000 7fbff9800640 1 -- 172.21.15.119:0/3266354821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbff40ce3a0 msgr2=0x7fbff40b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.776+0000 7fbff9800640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbff40ce3a0 0x7fbff40b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.776+0000 7fbff9800640 1 -- 172.21.15.119:0/3266354821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40b3d10 con 0x7fbff4161930 2024-04-25T10:07:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.776+0000 7fbff8e00640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbff406aba0 0x7fbff41f3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.776+0000 7fbff9800640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff4161930 0x7fbff406f420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00c410 tx=0x7fbfec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.776+0000 7fbff0c00640 1 -- 172.21.15.119:0/3266354821 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0074e0 con 0x7fbff4161930 2024-04-25T10:07:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.776+0000 7fbff0c00640 1 -- 172.21.15.119:0/3266354821 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec0047e0 con 0x7fbff4161930 2024-04-25T10:07:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.776+0000 7fbff0c00640 1 -- 172.21.15.119:0/3266354821 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec004bd0 con 0x7fbff4161930 2024-04-25T10:07:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.776+0000 7fbffb800640 1 -- 172.21.15.119:0/3266354821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff40b3ff0 con 0x7fbff4161930 2024-04-25T10:07:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.776+0000 7fbffb800640 1 -- 172.21.15.119:0/3266354821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff40c4550 con 0x7fbff4161930 2024-04-25T10:07:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.778+0000 7fbffb800640 1 -- 172.21.15.119:0/3266354821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff40750b0 con 0x7fbff4161930 2024-04-25T10:07:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.779+0000 7fbff0c00640 1 -- 172.21.15.119:0/3266354821 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbfec011070 con 0x7fbff4161930 2024-04-25T10:07:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.779+0000 7fbff0c00640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbfd406b4d0 0x7fbfd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.780+0000 7fbff0c00640 1 -- 172.21.15.119:0/3266354821 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fbfec089ea0 con 0x7fbff4161930 2024-04-25T10:07:51.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.780+0000 7fbff8e00640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbfd406b4d0 0x7fbfd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:51.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.780+0000 7fbff8e00640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbfd406b4d0 0x7fbfd406d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4007330 tx=0x7fbfe400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:51.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.784+0000 7fbff0c00640 1 -- 172.21.15.119:0/3266354821 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbfec01a2b0 con 0x7fbff4161930 2024-04-25T10:07:51.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.979+0000 7fbffb800640 1 -- 172.21.15.119:0/3266354821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbff4072f40 con 0x7fbff4161930 2024-04-25T10:07:51.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.980+0000 7fbff0c00640 1 -- 172.21.15.119:0/3266354821 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbfec058c20 con 0x7fbff4161930 2024-04-25T10:07:51.986 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:07:51.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.982+0000 7fbffb800640 1 -- 172.21.15.119:0/3266354821 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbfd406b4d0 msgr2=0x7fbfd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.982+0000 7fbffb800640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbfd406b4d0 0x7fbfd406d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4007330 tx=0x7fbfe400b040 comp rx=0 tx=0).stop 2024-04-25T10:07:51.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.982+0000 7fbffb800640 1 -- 172.21.15.119:0/3266354821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff4161930 msgr2=0x7fbff406f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:51.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.982+0000 7fbffb800640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff4161930 0x7fbff406f420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00c410 tx=0x7fbfec00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.983+0000 7fbffb800640 1 -- 172.21.15.119:0/3266354821 shutdown_connections 2024-04-25T10:07:51.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.983+0000 7fbffb800640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbfd406b4d0 0x7fbfd406d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.983+0000 7fbffb800640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff4161930 0x7fbff406f420 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.983+0000 7fbffb800640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbff40ce3a0 0x7fbff40b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.983+0000 7fbffb800640 1 --2- 172.21.15.119:0/3266354821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbff406aba0 0x7fbff41f3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:51.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.983+0000 7fbffb800640 1 -- 172.21.15.119:0/3266354821 >> 172.21.15.119:0/3266354821 conn(0x7fbff40d07d0 msgr2=0x7fbff40cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:51.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.983+0000 7fbffb800640 1 -- 172.21.15.119:0/3266354821 shutdown_connections 2024-04-25T10:07:51.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:51.983+0000 7fbffb800640 1 -- 172.21.15.119:0/3266354821 wait complete. 2024-04-25T10:07:52.008 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:52.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.197+0000 7f2062000640 1 Processor -- start 2024-04-25T10:07:52.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.197+0000 7f2062000640 1 -- start start 2024-04-25T10:07:52.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.199+0000 7f2062000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f205c071200 0x7f205c071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.199+0000 7f2062000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c0cc9a0 con 0x7f205c145260 2024-04-25T10:07:52.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.199+0000 7f2062000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c0ccc80 con 0x7f205c0cee90 2024-04-25T10:07:52.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.199+0000 7f2062000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c071ba0 con 0x7f205c071200 2024-04-25T10:07:52.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.199+0000 7f205b400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f205c145260 0x7f205c0cbf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34824/0 (socket says 172.21.15.119:34824) 2024-04-25T10:07:52.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.199+0000 7f205b400640 1 -- 172.21.15.119:0/1467745684 learned_addr learned my addr 172.21.15.119:0/1467745684 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:52.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205aa00640 1 --2- 172.21.15.119:0/1467745684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f205c071200 0x7f205c071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 -- 172.21.15.119:0/1467745684 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881040328 0 0) 0x7f205c0cc9a0 con 0x7f205c145260 2024-04-25T10:07:52.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 -- 172.21.15.119:0/1467745684 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2040003520 con 0x7f205c145260 2024-04-25T10:07:52.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 -- 172.21.15.119:0/1467745684 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2542836390 0 0) 0x7f2040003520 con 0x7f205c145260 2024-04-25T10:07:52.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 -- 172.21.15.119:0/1467745684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f205c071200 msgr2=0x7f205c071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 --2- 172.21.15.119:0/1467745684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f205c071200 0x7f205c071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 -- 172.21.15.119:0/1467745684 >> v1:172.21.15.132:6789/0 conn(0x7f205c0cee90 legacy=0x7f205c0cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 -- 172.21.15.119:0/1467745684 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c0ccf60 con 0x7f205c145260 2024-04-25T10:07:52.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 -- 172.21.15.119:0/1467745684 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f204c001670 con 0x7f205c145260 2024-04-25T10:07:52.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 -- 172.21.15.119:0/1467745684 >> v1:172.21.15.119:6789/0 conn(0x7f205c145260 legacy=0x7f205c0cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 --2- 172.21.15.119:0/1467745684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20400039e0 0x7f2040003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 --2- 172.21.15.119:0/1467745684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f205c071200 0x7f2040004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 --2- 172.21.15.119:0/1467745684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2040004d40 0x7f20400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 -- 172.21.15.119:0/1467745684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204000ba70 con 0x7f20400039e0 2024-04-25T10:07:52.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 -- 172.21.15.119:0/1467745684 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204000bbf0 con 0x7f2040004d40 2024-04-25T10:07:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.200+0000 7f205a000640 1 -- 172.21.15.119:0/1467745684 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204000bdf0 con 0x7f205c071200 2024-04-25T10:07:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.201+0000 7f205b400640 1 --2- 172.21.15.119:0/1467745684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20400039e0 0x7f2040003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.201+0000 7f205b400640 1 -- 172.21.15.119:0/1467745684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f205c071200 msgr2=0x7f2040004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.201+0000 7f205b400640 1 --2- 172.21.15.119:0/1467745684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f205c071200 0x7f2040004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.201+0000 7f205b400640 1 -- 172.21.15.119:0/1467745684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2040004d40 msgr2=0x7f20400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.201+0000 7f205b400640 1 --2- 172.21.15.119:0/1467745684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2040004d40 0x7f20400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.201+0000 7f205b400640 1 -- 172.21.15.119:0/1467745684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2040003520 con 0x7f20400039e0 2024-04-25T10:07:52.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.202+0000 7f205b400640 1 --2- 172.21.15.119:0/1467745684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20400039e0 0x7f2040003e00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f204c00c210 tx=0x7f2040005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f150d1f69cd19e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.202+0000 7f205a000640 1 -- 172.21.15.119:0/1467745684 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c007cf0 con 0x7f20400039e0 2024-04-25T10:07:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.202+0000 7f205a000640 1 -- 172.21.15.119:0/1467745684 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c005ad0 con 0x7f20400039e0 2024-04-25T10:07:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.202+0000 7f205a000640 1 -- 172.21.15.119:0/1467745684 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c007070 con 0x7f20400039e0 2024-04-25T10:07:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.203+0000 7f2062000640 1 -- 172.21.15.119:0/1467745684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20400039e0 msgr2=0x7f2040003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.203+0000 7f2062000640 1 --2- 172.21.15.119:0/1467745684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20400039e0 0x7f2040003e00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f204c00c210 tx=0x7f2040005830 comp rx=0 tx=0).stop 2024-04-25T10:07:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.203+0000 7f205be00640 1 -- 172.21.15.119:0/1467745684 reap_dead start 2024-04-25T10:07:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.203+0000 7f2062000640 1 -- 172.21.15.119:0/1467745684 shutdown_connections 2024-04-25T10:07:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.203+0000 7f2062000640 1 -- 172.21.15.119:0/1467745684 >> 172.21.15.119:0/1467745684 conn(0x7f205c06aba0 msgr2=0x7f205c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.203+0000 7f2062000640 1 -- 172.21.15.119:0/1467745684 shutdown_connections 2024-04-25T10:07:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.203+0000 7f2062000640 1 -- 172.21.15.119:0/1467745684 wait complete. 2024-04-25T10:07:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.205+0000 7f2062000640 1 Processor -- start 2024-04-25T10:07:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.205+0000 7f2062000640 1 -- start start 2024-04-25T10:07:52.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.206+0000 7f2062000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20400039e0 0x7f205c13d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.206+0000 7f2062000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f205c0b1fc0 0x7f205c0a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.206+0000 7f2062000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f205c0a89e0 0x7f205c0b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.206+0000 7f2062000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c0cc660 con 0x7f205c0b1fc0 2024-04-25T10:07:52.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.206+0000 7f2062000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c0cea00 con 0x7f205c0a89e0 2024-04-25T10:07:52.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.206+0000 7f2062000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c145640 con 0x7f20400039e0 2024-04-25T10:07:52.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.206+0000 7f205aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f205c0b1fc0 0x7f205c0a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.206+0000 7f205aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f205c0b1fc0 0x7f205c0a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48942/0 (socket says 172.21.15.119:48942) 2024-04-25T10:07:52.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.206+0000 7f205aa00640 1 -- 172.21.15.119:0/316820160 learned_addr learned my addr 172.21.15.119:0/316820160 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:52.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.207+0000 7f205b400640 1 --2- 172.21.15.119:0/316820160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20400039e0 0x7f205c13d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.207+0000 7f205be00640 1 --2- 172.21.15.119:0/316820160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f205c0a89e0 0x7f205c0b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.207+0000 7f205aa00640 1 -- 172.21.15.119:0/316820160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20400039e0 msgr2=0x7f205c13d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.207+0000 7f205aa00640 1 --2- 172.21.15.119:0/316820160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20400039e0 0x7f205c13d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.207+0000 7f205aa00640 1 -- 172.21.15.119:0/316820160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f205c0a89e0 msgr2=0x7f205c0b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.207+0000 7f205aa00640 1 --2- 172.21.15.119:0/316820160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f205c0a89e0 0x7f205c0b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.207+0000 7f205aa00640 1 -- 172.21.15.119:0/316820160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c0b0f20 con 0x7f205c0b1fc0 2024-04-25T10:07:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.207+0000 7f205aa00640 1 --2- 172.21.15.119:0/316820160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f205c0b1fc0 0x7f205c0a84a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f204800e7f0 tx=0x7f204800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.207+0000 7f2053400640 1 -- 172.21.15.119:0/316820160 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048017070 con 0x7f205c0b1fc0 2024-04-25T10:07:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.207+0000 7f2053400640 1 -- 172.21.15.119:0/316820160 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204800c940 con 0x7f205c0b1fc0 2024-04-25T10:07:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.207+0000 7f2053400640 1 -- 172.21.15.119:0/316820160 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204800cd10 con 0x7f205c0b1fc0 2024-04-25T10:07:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.207+0000 7f2062000640 1 -- 172.21.15.119:0/316820160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c0b1200 con 0x7f205c0b1fc0 2024-04-25T10:07:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.208+0000 7f2062000640 1 -- 172.21.15.119:0/316820160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c0b17c0 con 0x7f205c0b1fc0 2024-04-25T10:07:52.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.209+0000 7f2062000640 1 -- 172.21.15.119:0/316820160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205c0a8df0 con 0x7f205c0b1fc0 2024-04-25T10:07:52.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.210+0000 7f2053400640 1 -- 172.21.15.119:0/316820160 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f204801a070 con 0x7f205c0b1fc0 2024-04-25T10:07:52.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.211+0000 7f2053400640 1 --2- 172.21.15.119:0/316820160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f203c06b4d0 0x7f203c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.211+0000 7f2053400640 1 -- 172.21.15.119:0/316820160 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f204808a4c0 con 0x7f205c0b1fc0 2024-04-25T10:07:52.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.211+0000 7f205b400640 1 --2- 172.21.15.119:0/316820160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f203c06b4d0 0x7f203c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.211+0000 7f205b400640 1 --2- 172.21.15.119:0/316820160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f203c06b4d0 0x7f203c06d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f204c032000 tx=0x7f204c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.215+0000 7f2053400640 1 -- 172.21.15.119:0/316820160 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f205c0a8df0 con 0x7f205c0b1fc0 2024-04-25T10:07:52.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.427+0000 7f2062000640 1 -- 172.21.15.119:0/316820160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f205c0750b0 con 0x7f205c0b1fc0 2024-04-25T10:07:52.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.428+0000 7f2053400640 1 -- 172.21.15.119:0/316820160 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2048059240 con 0x7f205c0b1fc0 2024-04-25T10:07:52.434 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2024-04-25T10:07:52.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.431+0000 7f2062000640 1 -- 172.21.15.119:0/316820160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f203c06b4d0 msgr2=0x7f203c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.431+0000 7f2062000640 1 --2- 172.21.15.119:0/316820160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f203c06b4d0 0x7f203c06d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f204c032000 tx=0x7f204c009030 comp rx=0 tx=0).stop 2024-04-25T10:07:52.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.431+0000 7f2062000640 1 -- 172.21.15.119:0/316820160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f205c0b1fc0 msgr2=0x7f205c0a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.431+0000 7f2062000640 1 --2- 172.21.15.119:0/316820160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f205c0b1fc0 0x7f205c0a84a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f204800e7f0 tx=0x7f204800ecb0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.431+0000 7f2062000640 1 -- 172.21.15.119:0/316820160 shutdown_connections 2024-04-25T10:07:52.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.431+0000 7f2062000640 1 --2- 172.21.15.119:0/316820160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f203c06b4d0 0x7f203c06d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.431+0000 7f2062000640 1 --2- 172.21.15.119:0/316820160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f205c0a89e0 0x7f205c0b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.431+0000 7f2062000640 1 --2- 172.21.15.119:0/316820160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f205c0b1fc0 0x7f205c0a84a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.431+0000 7f2062000640 1 --2- 172.21.15.119:0/316820160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20400039e0 0x7f205c13d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.431+0000 7f2062000640 1 -- 172.21.15.119:0/316820160 >> 172.21.15.119:0/316820160 conn(0x7f205c06aba0 msgr2=0x7f205c0715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:52.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.431+0000 7f2062000640 1 -- 172.21.15.119:0/316820160 shutdown_connections 2024-04-25T10:07:52.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.431+0000 7f2062000640 1 -- 172.21.15.119:0/316820160 wait complete. 2024-04-25T10:07:52.455 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T10:07:52.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:52.456 DEBUG:teuthology.orchestra.run.smithi132:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T10:07:52.555 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2024-04-25T10:07:52.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:52.555 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:07:52.611 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:07:52.611 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:52.669 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:07:52.669 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:52.724 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:07:52.724 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:52.775 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:52.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.972+0000 7fa13e000640 1 Processor -- start 2024-04-25T10:07:52.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.972+0000 7fa13e000640 1 -- start start 2024-04-25T10:07:52.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.973+0000 7fa13e000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa1380aecf0 0x7fa1380cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.973+0000 7fa13e000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa13815ed70 0x7fa1380cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.973+0000 7fa13e000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1380712c0 con 0x7fa13815d980 2024-04-25T10:07:52.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.973+0000 7fa13e000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa138071ed0 con 0x7fa13815ed70 2024-04-25T10:07:52.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.973+0000 7fa13e000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1380715a0 con 0x7fa1380aecf0 2024-04-25T10:07:52.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.974+0000 7fa137400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa13815d980 0x7fa1380ae5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34840/0 (socket says 172.21.15.119:34840) 2024-04-25T10:07:52.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.974+0000 7fa137400640 1 -- 172.21.15.119:0/3471253032 learned_addr learned my addr 172.21.15.119:0/3471253032 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:52.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.974+0000 7fa137e00640 1 --2- 172.21.15.119:0/3471253032 >> v2:172.21.15.119:3300/0 conn(0x7fa13815ed70 0x7fa1380cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.974+0000 7fa12ea00640 1 --2- 172.21.15.119:0/3471253032 >> v2:172.21.15.132:3300/0 conn(0x7fa1380aecf0 0x7fa1380cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.974+0000 7fa136a00640 1 -- 172.21.15.119:0/3471253032 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233855477 0 0) 0x7fa1380712c0 con 0x7fa13815d980 2024-04-25T10:07:52.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.974+0000 7fa136a00640 1 -- 172.21.15.119:0/3471253032 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa11c003520 con 0x7fa13815d980 2024-04-25T10:07:52.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.974+0000 7fa137e00640 1 -- 172.21.15.119:0/3471253032 >> v2:172.21.15.132:3300/0 conn(0x7fa1380aecf0 msgr2=0x7fa1380cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:52.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.974+0000 7fa137e00640 1 --2- 172.21.15.119:0/3471253032 >> v2:172.21.15.132:3300/0 conn(0x7fa1380aecf0 0x7fa1380cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.974+0000 7fa137e00640 1 -- 172.21.15.119:0/3471253032 >> v1:172.21.15.119:6789/0 conn(0x7fa13815d980 legacy=0x7fa1380ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.974+0000 7fa137e00640 1 -- 172.21.15.119:0/3471253032 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa138071880 con 0x7fa13815ed70 2024-04-25T10:07:52.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.975+0000 7fa137e00640 1 --2- 172.21.15.119:0/3471253032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13815ed70 0x7fa1380cf7c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa138072050 tx=0x7fa13002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c59e2ed111d3b04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.976+0000 7fa136a00640 1 -- 172.21.15.119:0/3471253032 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa130031600 con 0x7fa13815ed70 2024-04-25T10:07:52.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.976+0000 7fa136a00640 1 -- 172.21.15.119:0/3471253032 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1300317a0 con 0x7fa13815ed70 2024-04-25T10:07:52.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.976+0000 7fa136a00640 1 -- 172.21.15.119:0/3471253032 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa130031b70 con 0x7fa13815ed70 2024-04-25T10:07:52.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.976+0000 7fa13e000640 1 -- 172.21.15.119:0/3471253032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13815ed70 msgr2=0x7fa1380cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.976+0000 7fa13e000640 1 --2- 172.21.15.119:0/3471253032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13815ed70 0x7fa1380cf7c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa138072050 tx=0x7fa13002a5d0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.976+0000 7fa13e000640 1 -- 172.21.15.119:0/3471253032 shutdown_connections 2024-04-25T10:07:52.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.976+0000 7fa13e000640 1 --2- 172.21.15.119:0/3471253032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13815ed70 0x7fa1380cf7c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.976+0000 7fa13e000640 1 --2- 172.21.15.119:0/3471253032 >> v2:172.21.15.132:3300/0 conn(0x7fa1380aecf0 0x7fa1380cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.976+0000 7fa13e000640 1 -- 172.21.15.119:0/3471253032 >> 172.21.15.119:0/3471253032 conn(0x7fa13806aba0 msgr2=0x7fa13806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:52.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.977+0000 7fa13e000640 1 -- 172.21.15.119:0/3471253032 shutdown_connections 2024-04-25T10:07:52.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.977+0000 7fa13e000640 1 -- 172.21.15.119:0/3471253032 wait complete. 2024-04-25T10:07:52.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.978+0000 7fa13e000640 1 Processor -- start 2024-04-25T10:07:52.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.978+0000 7fa13e000640 1 -- start start 2024-04-25T10:07:52.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.979+0000 7fa13e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13815d980 0x7fa1381450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.979+0000 7fa13e000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1380cc480 0x7fa1380cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.979+0000 7fa13e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa138147940 0x7fa138147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.979+0000 7fa13e000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1380ad0f0 con 0x7fa13815d980 2024-04-25T10:07:52.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.979+0000 7fa13e000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1380acf70 con 0x7fa1380cc480 2024-04-25T10:07:52.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.979+0000 7fa13e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1380ad270 con 0x7fa138147940 2024-04-25T10:07:52.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.979+0000 7fa137400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13815d980 0x7fa1381450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.980+0000 7fa137400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13815d980 0x7fa1381450d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48972/0 (socket says 172.21.15.119:48972) 2024-04-25T10:07:52.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.980+0000 7fa137400640 1 -- 172.21.15.119:0/4057850457 learned_addr learned my addr 172.21.15.119:0/4057850457 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:52.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.980+0000 7fa12ea00640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1380cc480 0x7fa1380cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.980+0000 7fa137e00640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa138147940 0x7fa138147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.980+0000 7fa137400640 1 -- 172.21.15.119:0/4057850457 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa138147940 msgr2=0x7fa138147dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.980+0000 7fa137400640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa138147940 0x7fa138147dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.980+0000 7fa137400640 1 -- 172.21.15.119:0/4057850457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1380cc480 msgr2=0x7fa1380cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:52.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.980+0000 7fa137400640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1380cc480 0x7fa1380cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:52.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.980+0000 7fa137400640 1 -- 172.21.15.119:0/4057850457 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa138145610 con 0x7fa13815d980 2024-04-25T10:07:52.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.980+0000 7fa12ea00640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1380cc480 0x7fa1380cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:52.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.980+0000 7fa137400640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13815d980 0x7fa1381450d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c430 tx=0x7fa12800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.980+0000 7fa12fe00640 1 -- 172.21.15.119:0/4057850457 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128017070 con 0x7fa13815d980 2024-04-25T10:07:52.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.981+0000 7fa12fe00640 1 -- 172.21.15.119:0/4057850457 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa128005ce0 con 0x7fa13815d980 2024-04-25T10:07:52.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.981+0000 7fa12fe00640 1 -- 172.21.15.119:0/4057850457 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1280072f0 con 0x7fa13815d980 2024-04-25T10:07:52.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.981+0000 7fa13e000640 1 -- 172.21.15.119:0/4057850457 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1380cd240 con 0x7fa13815d980 2024-04-25T10:07:52.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.981+0000 7fa13e000640 1 -- 172.21.15.119:0/4057850457 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa138159b90 con 0x7fa13815d980 2024-04-25T10:07:52.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.982+0000 7fa13e000640 1 -- 172.21.15.119:0/4057850457 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1380749b0 con 0x7fa13815d980 2024-04-25T10:07:52.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.983+0000 7fa12fe00640 1 -- 172.21.15.119:0/4057850457 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa12801a070 con 0x7fa13815d980 2024-04-25T10:07:52.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.984+0000 7fa12fe00640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa11406b500 0x7fa11406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:52.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.984+0000 7fa12fe00640 1 -- 172.21.15.119:0/4057850457 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fa128089680 con 0x7fa13815d980 2024-04-25T10:07:52.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.984+0000 7fa12ea00640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa11406b500 0x7fa11406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:52.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.985+0000 7fa12ea00640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa11406b500 0x7fa11406d9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa120006fd0 tx=0x7fa120008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:52.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:52.989+0000 7fa12fe00640 1 -- 172.21.15.119:0/4057850457 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa138145610 con 0x7fa13815d980 2024-04-25T10:07:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.183+0000 7fa13e000640 1 -- 172.21.15.119:0/4057850457 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa13806b260 con 0x7fa13815d980 2024-04-25T10:07:53.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.185+0000 7fa12fe00640 1 -- 172.21.15.119:0/4057850457 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16861 (secure 0 0 0) 0x7fa1280584a0 con 0x7fa13815d980 2024-04-25T10:07:53.191 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:53.191 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":22,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:49.883606+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:53.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.187+0000 7fa13e000640 1 -- 172.21.15.119:0/4057850457 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa11406b500 msgr2=0x7fa11406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.187+0000 7fa13e000640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa11406b500 0x7fa11406d9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa120006fd0 tx=0x7fa120008040 comp rx=0 tx=0).stop 2024-04-25T10:07:53.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.187+0000 7fa13e000640 1 -- 172.21.15.119:0/4057850457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13815d980 msgr2=0x7fa1381450d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.187+0000 7fa13e000640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13815d980 0x7fa1381450d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c430 tx=0x7fa12800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.187+0000 7fa13e000640 1 -- 172.21.15.119:0/4057850457 shutdown_connections 2024-04-25T10:07:53.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.187+0000 7fa13e000640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa11406b500 0x7fa11406d9c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.187+0000 7fa13e000640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa138147940 0x7fa138147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.187+0000 7fa13e000640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1380cc480 0x7fa1380cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.187+0000 7fa13e000640 1 --2- 172.21.15.119:0/4057850457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa13815d980 0x7fa1381450d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.187+0000 7fa13e000640 1 -- 172.21.15.119:0/4057850457 >> 172.21.15.119:0/4057850457 conn(0x7fa13806aba0 msgr2=0x7fa13806dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.188+0000 7fa13e000640 1 -- 172.21.15.119:0/4057850457 shutdown_connections 2024-04-25T10:07:53.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.188+0000 7fa13e000640 1 -- 172.21.15.119:0/4057850457 wait complete. 2024-04-25T10:07:53.212 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:05.135832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:05.768083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:53.213 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.414+0000 7f4d68e00640 1 Processor -- start 2024-04-25T10:07:53.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.415+0000 7f4d68e00640 1 -- start start 2024-04-25T10:07:53.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.416+0000 7f4d68e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b2740 0x7f4d6415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.416+0000 7f4d68e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4d640ce1a0 0x7f4d640ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.416+0000 7f4d68e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6406c150 con 0x7f4d640d0490 2024-04-25T10:07:53.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.416+0000 7f4d68e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d640d2000 con 0x7f4d640ce1a0 2024-04-25T10:07:53.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.416+0000 7f4d68e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6406cd60 con 0x7f4d640b2740 2024-04-25T10:07:53.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.416+0000 7f4d62800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4d640ce1a0 0x7f4d640ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.417+0000 7f4d62800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4d640ce1a0 0x7f4d640ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48980/0 (socket says 172.21.15.119:48980) 2024-04-25T10:07:53.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.417+0000 7f4d62800640 1 -- 172.21.15.119:0/849008462 learned_addr learned my addr 172.21.15.119:0/849008462 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.417+0000 7f4d61400640 1 --2- 172.21.15.119:0/849008462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b2740 0x7f4d6415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.417+0000 7f4d62800640 1 -- 172.21.15.119:0/849008462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b2740 msgr2=0x7f4d6415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.417+0000 7f4d62800640 1 --2- 172.21.15.119:0/849008462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b2740 0x7f4d6415ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.417+0000 7f4d62800640 1 -- 172.21.15.119:0/849008462 >> v1:172.21.15.132:6789/0 conn(0x7f4d640d0490 legacy=0x7f4d640d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.417+0000 7f4d62800640 1 -- 172.21.15.119:0/849008462 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6406c430 con 0x7f4d640ce1a0 2024-04-25T10:07:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.417+0000 7f4d61400640 1 --2- 172.21.15.119:0/849008462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b2740 0x7f4d6415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.418+0000 7f4d62800640 1 --2- 172.21.15.119:0/849008462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d640ce1a0 0x7f4d640ce630 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0098a0 tx=0x7f4d5c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b8ad5a0818eb03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.418+0000 7f4d60a00640 1 -- 172.21.15.119:0/849008462 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c037070 con 0x7f4d640ce1a0 2024-04-25T10:07:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.418+0000 7f4d60a00640 1 -- 172.21.15.119:0/849008462 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c032440 con 0x7f4d640ce1a0 2024-04-25T10:07:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.418+0000 7f4d60a00640 1 -- 172.21.15.119:0/849008462 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c032810 con 0x7f4d640ce1a0 2024-04-25T10:07:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.419+0000 7f4d68e00640 1 -- 172.21.15.119:0/849008462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d640ce1a0 msgr2=0x7f4d640ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.419+0000 7f4d68e00640 1 --2- 172.21.15.119:0/849008462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d640ce1a0 0x7f4d640ce630 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0098a0 tx=0x7f4d5c02a200 comp rx=0 tx=0).stop 2024-04-25T10:07:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.419+0000 7f4d68e00640 1 -- 172.21.15.119:0/849008462 shutdown_connections 2024-04-25T10:07:53.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.419+0000 7f4d68e00640 1 --2- 172.21.15.119:0/849008462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d640ce1a0 0x7f4d640ce630 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.419+0000 7f4d68e00640 1 --2- 172.21.15.119:0/849008462 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b2740 0x7f4d6415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.419+0000 7f4d68e00640 1 -- 172.21.15.119:0/849008462 >> 172.21.15.119:0/849008462 conn(0x7f4d640b5660 msgr2=0x7f4d640b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.419+0000 7f4d68e00640 1 -- 172.21.15.119:0/849008462 shutdown_connections 2024-04-25T10:07:53.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.419+0000 7f4d68e00640 1 -- 172.21.15.119:0/849008462 wait complete. 2024-04-25T10:07:53.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.421+0000 7f4d68e00640 1 Processor -- start 2024-04-25T10:07:53.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.421+0000 7f4d68e00640 1 -- start start 2024-04-25T10:07:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.422+0000 7f4d68e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b2740 0x7f4d6415a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.422+0000 7f4d68e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d640ce1a0 0x7f4d64157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.422+0000 7f4d68e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d640d0490 0x7f4d640d2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.422+0000 7f4d68e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6406c9f0 con 0x7f4d640d0490 2024-04-25T10:07:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.422+0000 7f4d68e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6415f1d0 con 0x7f4d640ce1a0 2024-04-25T10:07:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.422+0000 7f4d68e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6415e020 con 0x7f4d640b2740 2024-04-25T10:07:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.422+0000 7f4d62800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d640d0490 0x7f4d640d2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.423+0000 7f4d62800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d640d0490 0x7f4d640d2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48994/0 (socket says 172.21.15.119:48994) 2024-04-25T10:07:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.423+0000 7f4d62800640 1 -- 172.21.15.119:0/4170816317 learned_addr learned my addr 172.21.15.119:0/4170816317 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.423+0000 7f4d61e00640 1 --2- 172.21.15.119:0/4170816317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b2740 0x7f4d6415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.423+0000 7f4d61400640 1 --2- 172.21.15.119:0/4170816317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d640ce1a0 0x7f4d64157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.423+0000 7f4d62800640 1 -- 172.21.15.119:0/4170816317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b2740 msgr2=0x7f4d6415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.423+0000 7f4d62800640 1 --2- 172.21.15.119:0/4170816317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b2740 0x7f4d6415a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.423+0000 7f4d62800640 1 -- 172.21.15.119:0/4170816317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d640ce1a0 msgr2=0x7f4d64157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.423+0000 7f4d62800640 1 --2- 172.21.15.119:0/4170816317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d640ce1a0 0x7f4d64157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.423+0000 7f4d62800640 1 -- 172.21.15.119:0/4170816317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d64149fc0 con 0x7f4d640d0490 2024-04-25T10:07:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.423+0000 7f4d62800640 1 --2- 172.21.15.119:0/4170816317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d640d0490 0x7f4d640d2e10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0090b0 tx=0x7f4d5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.424+0000 7f4d5a000640 1 -- 172.21.15.119:0/4170816317 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c041020 con 0x7f4d640d0490 2024-04-25T10:07:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.424+0000 7f4d5a000640 1 -- 172.21.15.119:0/4170816317 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c030840 con 0x7f4d640d0490 2024-04-25T10:07:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.424+0000 7f4d5a000640 1 -- 172.21.15.119:0/4170816317 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c037040 con 0x7f4d640d0490 2024-04-25T10:07:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.424+0000 7f4d68e00640 1 -- 172.21.15.119:0/4170816317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d64149ce0 con 0x7f4d640d0490 2024-04-25T10:07:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.424+0000 7f4d68e00640 1 -- 172.21.15.119:0/4170816317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6415a000 con 0x7f4d640d0490 2024-04-25T10:07:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.425+0000 7f4d68e00640 1 -- 172.21.15.119:0/4170816317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d640750b0 con 0x7f4d640d0490 2024-04-25T10:07:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.426+0000 7f4d5a000640 1 -- 172.21.15.119:0/4170816317 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d5c044020 con 0x7f4d640d0490 2024-04-25T10:07:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.427+0000 7f4d5a000640 1 --2- 172.21.15.119:0/4170816317 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d4006b4d0 0x7f4d4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.427+0000 7f4d5a000640 1 -- 172.21.15.119:0/4170816317 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f4d5c0aaab0 con 0x7f4d640d0490 2024-04-25T10:07:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.427+0000 7f4d61e00640 1 --2- 172.21.15.119:0/4170816317 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d4006b4d0 0x7f4d4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.428+0000 7f4d61e00640 1 --2- 172.21.15.119:0/4170816317 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d4006b4d0 0x7f4d4006d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4d540016e0 tx=0x7f4d54009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.431+0000 7f4d5a000640 1 -- 172.21.15.119:0/4170816317 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d5c04b030 con 0x7f4d640d0490 2024-04-25T10:07:53.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.627+0000 7f4d68e00640 1 -- 172.21.15.119:0/4170816317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d640d18d0 con 0x7f4d640d0490 2024-04-25T10:07:53.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.628+0000 7f4d5a000640 1 -- 172.21.15.119:0/4170816317 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4d5c0791f0 con 0x7f4d640d0490 2024-04-25T10:07:53.634 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:07:53.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.631+0000 7f4d68e00640 1 -- 172.21.15.119:0/4170816317 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d4006b4d0 msgr2=0x7f4d4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.631+0000 7f4d68e00640 1 --2- 172.21.15.119:0/4170816317 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d4006b4d0 0x7f4d4006d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4d540016e0 tx=0x7f4d54009040 comp rx=0 tx=0).stop 2024-04-25T10:07:53.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.631+0000 7f4d68e00640 1 -- 172.21.15.119:0/4170816317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d640d0490 msgr2=0x7f4d640d2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.631+0000 7f4d68e00640 1 --2- 172.21.15.119:0/4170816317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d640d0490 0x7f4d640d2e10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0090b0 tx=0x7f4d5c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.631+0000 7f4d68e00640 1 -- 172.21.15.119:0/4170816317 shutdown_connections 2024-04-25T10:07:53.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.631+0000 7f4d68e00640 1 --2- 172.21.15.119:0/4170816317 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d4006b4d0 0x7f4d4006d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.631+0000 7f4d68e00640 1 --2- 172.21.15.119:0/4170816317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d640d0490 0x7f4d640d2e10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.631+0000 7f4d68e00640 1 --2- 172.21.15.119:0/4170816317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d640ce1a0 0x7f4d64157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.631+0000 7f4d68e00640 1 --2- 172.21.15.119:0/4170816317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d640b2740 0x7f4d6415a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.631+0000 7f4d68e00640 1 -- 172.21.15.119:0/4170816317 >> 172.21.15.119:0/4170816317 conn(0x7f4d640b5660 msgr2=0x7f4d640d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.631+0000 7f4d68e00640 1 -- 172.21.15.119:0/4170816317 shutdown_connections 2024-04-25T10:07:53.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.631+0000 7f4d68e00640 1 -- 172.21.15.119:0/4170816317 wait complete. 2024-04-25T10:07:53.655 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:53.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.845+0000 7f3947000640 1 Processor -- start 2024-04-25T10:07:53.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.845+0000 7f3947000640 1 -- start start 2024-04-25T10:07:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.846+0000 7f3947000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f394006aba0 0x7f394006af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.846+0000 7f3947000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f394006f040 0x7f394015e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.847+0000 7f3947000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f39400cec60 0x7f394015e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.847+0000 7f3947000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394006b4c0 con 0x7f39400cec60 2024-04-25T10:07:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.847+0000 7f3947000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394006d080 con 0x7f394006aba0 2024-04-25T10:07:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.847+0000 7f3947000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394006d360 con 0x7f394006f040 2024-04-25T10:07:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.847+0000 7f3946600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f39400cec60 0x7f394015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.847+0000 7f3946600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f39400cec60 0x7f394015e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48996/0 (socket says 172.21.15.119:48996) 2024-04-25T10:07:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.847+0000 7f3946600640 1 -- 172.21.15.119:0/1779412504 learned_addr learned my addr 172.21.15.119:0/1779412504 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:53.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.847+0000 7f3945c00640 1 --2- 172.21.15.119:0/1779412504 >> v2:172.21.15.132:3300/0 conn(0x7f394006aba0 0x7f394006af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.847+0000 7f3945200640 1 --2- 172.21.15.119:0/1779412504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f394006f040 0x7f394015e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.847+0000 7f3946600640 1 -- 172.21.15.119:0/1779412504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f394006f040 msgr2=0x7f394015e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:53.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.847+0000 7f3946600640 1 --2- 172.21.15.119:0/1779412504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f394006f040 0x7f394015e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.847+0000 7f3946600640 1 -- 172.21.15.119:0/1779412504 >> v2:172.21.15.132:3300/0 conn(0x7f394006aba0 msgr2=0x7f394006af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:53.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.847+0000 7f3946600640 1 --2- 172.21.15.119:0/1779412504 >> v2:172.21.15.132:3300/0 conn(0x7f394006aba0 0x7f394006af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.847+0000 7f3946600640 1 -- 172.21.15.119:0/1779412504 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394006d640 con 0x7f39400cec60 2024-04-25T10:07:53.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.848+0000 7f3946600640 1 --2- 172.21.15.119:0/1779412504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39400cec60 0x7f394015e720 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f39400b1ef0 tx=0x7f393802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f0c15e548b9a00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.849+0000 7f393fe00640 1 -- 172.21.15.119:0/1779412504 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938036070 con 0x7f39400cec60 2024-04-25T10:07:53.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.849+0000 7f393fe00640 1 -- 172.21.15.119:0/1779412504 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3938031600 con 0x7f39400cec60 2024-04-25T10:07:53.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.849+0000 7f393fe00640 1 -- 172.21.15.119:0/1779412504 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39380319d0 con 0x7f39400cec60 2024-04-25T10:07:53.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.849+0000 7f3947000640 1 -- 172.21.15.119:0/1779412504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39400cec60 msgr2=0x7f394015e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.849+0000 7f3947000640 1 --2- 172.21.15.119:0/1779412504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39400cec60 0x7f394015e720 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f39400b1ef0 tx=0x7f393802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.850+0000 7f3947000640 1 -- 172.21.15.119:0/1779412504 shutdown_connections 2024-04-25T10:07:53.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.850+0000 7f3947000640 1 --2- 172.21.15.119:0/1779412504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39400cec60 0x7f394015e720 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.850+0000 7f3947000640 1 --2- 172.21.15.119:0/1779412504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f394006f040 0x7f394015e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.850+0000 7f3947000640 1 --2- 172.21.15.119:0/1779412504 >> v2:172.21.15.132:3300/0 conn(0x7f394006aba0 0x7f394006af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.850+0000 7f3947000640 1 -- 172.21.15.119:0/1779412504 >> 172.21.15.119:0/1779412504 conn(0x7f3940066ff0 msgr2=0x7f39400b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:53.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.850+0000 7f3947000640 1 -- 172.21.15.119:0/1779412504 shutdown_connections 2024-04-25T10:07:53.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.850+0000 7f3947000640 1 -- 172.21.15.119:0/1779412504 wait complete. 2024-04-25T10:07:53.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.851+0000 7f3947000640 1 Processor -- start 2024-04-25T10:07:53.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.852+0000 7f3947000640 1 -- start start 2024-04-25T10:07:53.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.852+0000 7f3947000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f394006aba0 0x7f3940156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.852+0000 7f3947000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f394006f040 0x7f3940158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.852+0000 7f3947000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39400cec60 0x7f39400ad500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.853+0000 7f3947000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394015a980 con 0x7f39400cec60 2024-04-25T10:07:53.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.853+0000 7f3947000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394006dc00 con 0x7f394006aba0 2024-04-25T10:07:53.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.853+0000 7f3947000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39400aed40 con 0x7f394006f040 2024-04-25T10:07:53.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.853+0000 7f3946600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39400cec60 0x7f39400ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.853+0000 7f3946600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39400cec60 0x7f39400ad500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49012/0 (socket says 172.21.15.119:49012) 2024-04-25T10:07:53.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.853+0000 7f3946600640 1 -- 172.21.15.119:0/2391338144 learned_addr learned my addr 172.21.15.119:0/2391338144 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:53.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.853+0000 7f3945c00640 1 --2- 172.21.15.119:0/2391338144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f394006aba0 0x7f3940156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.853+0000 7f3945200640 1 --2- 172.21.15.119:0/2391338144 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f394006f040 0x7f3940158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.853+0000 7f3946600640 1 -- 172.21.15.119:0/2391338144 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f394006f040 msgr2=0x7f3940158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.853+0000 7f3946600640 1 --2- 172.21.15.119:0/2391338144 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f394006f040 0x7f3940158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.853+0000 7f3946600640 1 -- 172.21.15.119:0/2391338144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f394006aba0 msgr2=0x7f3940156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:53.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.853+0000 7f3946600640 1 --2- 172.21.15.119:0/2391338144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f394006aba0 0x7f3940156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:53.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.853+0000 7f3946600640 1 -- 172.21.15.119:0/2391338144 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3940159d50 con 0x7f39400cec60 2024-04-25T10:07:53.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.854+0000 7f3946600640 1 --2- 172.21.15.119:0/2391338144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39400cec60 0x7f39400ad500 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3938009f40 tx=0x7f3938004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.854+0000 7f393d600640 1 -- 172.21.15.119:0/2391338144 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938041020 con 0x7f39400cec60 2024-04-25T10:07:53.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.854+0000 7f393d600640 1 -- 172.21.15.119:0/2391338144 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3938007780 con 0x7f39400cec60 2024-04-25T10:07:53.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.854+0000 7f393d600640 1 -- 172.21.15.119:0/2391338144 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938036040 con 0x7f39400cec60 2024-04-25T10:07:53.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.854+0000 7f3947000640 1 -- 172.21.15.119:0/2391338144 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394015a030 con 0x7f39400cec60 2024-04-25T10:07:53.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.854+0000 7f3947000640 1 -- 172.21.15.119:0/2391338144 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3940157f70 con 0x7f39400cec60 2024-04-25T10:07:53.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.855+0000 7f3947000640 1 -- 172.21.15.119:0/2391338144 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39400749b0 con 0x7f39400cec60 2024-04-25T10:07:53.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.856+0000 7f393d600640 1 -- 172.21.15.119:0/2391338144 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3938044020 con 0x7f39400cec60 2024-04-25T10:07:53.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.857+0000 7f393d600640 1 --2- 172.21.15.119:0/2391338144 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f392006b4d0 0x7f392006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:53.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.857+0000 7f393d600640 1 -- 172.21.15.119:0/2391338144 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f39380aaf70 con 0x7f39400cec60 2024-04-25T10:07:53.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.857+0000 7f3945c00640 1 --2- 172.21.15.119:0/2391338144 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f392006b4d0 0x7f392006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:53.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.858+0000 7f3945c00640 1 --2- 172.21.15.119:0/2391338144 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f392006b4d0 0x7f392006d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3934006fd0 tx=0x7f3934008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:53.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:53.861+0000 7f393d600640 1 -- 172.21.15.119:0/2391338144 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f393804b030 con 0x7f39400cec60 2024-04-25T10:07:54.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.055+0000 7f3947000640 1 -- 172.21.15.119:0/2391338144 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f394015e030 con 0x7f39400cec60 2024-04-25T10:07:54.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.056+0000 7f393d600640 1 -- 172.21.15.119:0/2391338144 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3938079d80 con 0x7f39400cec60 2024-04-25T10:07:54.062 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:07:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.058+0000 7f3947000640 1 -- 172.21.15.119:0/2391338144 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f392006b4d0 msgr2=0x7f392006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.058+0000 7f3947000640 1 --2- 172.21.15.119:0/2391338144 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f392006b4d0 0x7f392006d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3934006fd0 tx=0x7f3934008040 comp rx=0 tx=0).stop 2024-04-25T10:07:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.058+0000 7f3947000640 1 -- 172.21.15.119:0/2391338144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39400cec60 msgr2=0x7f39400ad500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.058+0000 7f3947000640 1 --2- 172.21.15.119:0/2391338144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39400cec60 0x7f39400ad500 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3938009f40 tx=0x7f3938004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.059+0000 7f3947000640 1 -- 172.21.15.119:0/2391338144 shutdown_connections 2024-04-25T10:07:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.059+0000 7f3947000640 1 --2- 172.21.15.119:0/2391338144 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f392006b4d0 0x7f392006d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.059+0000 7f3947000640 1 --2- 172.21.15.119:0/2391338144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39400cec60 0x7f39400ad500 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.059+0000 7f3947000640 1 --2- 172.21.15.119:0/2391338144 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f394006f040 0x7f3940158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.059+0000 7f3947000640 1 --2- 172.21.15.119:0/2391338144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f394006aba0 0x7f3940156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.059+0000 7f3947000640 1 -- 172.21.15.119:0/2391338144 >> 172.21.15.119:0/2391338144 conn(0x7f3940066ff0 msgr2=0x7f39400b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.059+0000 7f3947000640 1 -- 172.21.15.119:0/2391338144 shutdown_connections 2024-04-25T10:07:54.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.059+0000 7f3947000640 1 -- 172.21.15.119:0/2391338144 wait complete. 2024-04-25T10:07:54.083 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:54.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.285+0000 7f8d93200640 1 Processor -- start 2024-04-25T10:07:54.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.285+0000 7f8d93200640 1 -- start start 2024-04-25T10:07:54.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.287+0000 7f8d93200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8d8c0ce9d0 0x7f8d8c0cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.287+0000 7f8d93200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8d8c0cc300 0x7f8d8c0cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.287+0000 7f8d93200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d8c06dcb0 con 0x7f8d8c144e70 2024-04-25T10:07:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.287+0000 7f8d93200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8c15bfe0 con 0x7f8d8c0ce9d0 2024-04-25T10:07:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.287+0000 7f8d93200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8c06df90 con 0x7f8d8c0cc300 2024-04-25T10:07:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.287+0000 7f8d8b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8d8c0ce9d0 0x7f8d8c0cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.287+0000 7f8d8b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8d8c0ce9d0 0x7f8d8c0cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49020/0 (socket says 172.21.15.119:49020) 2024-04-25T10:07:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.287+0000 7f8d8b400640 1 -- 172.21.15.119:0/4084894728 learned_addr learned my addr 172.21.15.119:0/4084894728 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.287+0000 7f8d91200640 1 --2- 172.21.15.119:0/4084894728 >> v2:172.21.15.132:3300/0 conn(0x7f8d8c0cc300 0x7f8d8c0cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.288+0000 7f8d8b400640 1 -- 172.21.15.119:0/4084894728 >> v2:172.21.15.132:3300/0 conn(0x7f8d8c0cc300 msgr2=0x7f8d8c0cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.288+0000 7f8d8b400640 1 --2- 172.21.15.119:0/4084894728 >> v2:172.21.15.132:3300/0 conn(0x7f8d8c0cc300 0x7f8d8c0cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.288+0000 7f8d8b400640 1 -- 172.21.15.119:0/4084894728 >> v1:172.21.15.132:6789/0 conn(0x7f8d8c144e70 legacy=0x7f8d8c06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:54.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.288+0000 7f8d8b400640 1 -- 172.21.15.119:0/4084894728 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c06e270 con 0x7f8d8c0ce9d0 2024-04-25T10:07:54.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.289+0000 7f8d8b400640 1 --2- 172.21.15.119:0/4084894728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d8c0ce9d0 0x7f8d8c0cee60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c1458f0 tx=0x7f8d7402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb720c262b45503 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.289+0000 7f8d8aa00640 1 -- 172.21.15.119:0/4084894728 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74037070 con 0x7f8d8c0ce9d0 2024-04-25T10:07:54.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.289+0000 7f8d8aa00640 1 -- 172.21.15.119:0/4084894728 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d74032440 con 0x7f8d8c0ce9d0 2024-04-25T10:07:54.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.289+0000 7f8d8aa00640 1 -- 172.21.15.119:0/4084894728 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74032810 con 0x7f8d8c0ce9d0 2024-04-25T10:07:54.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.290+0000 7f8d93200640 1 -- 172.21.15.119:0/4084894728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d8c0ce9d0 msgr2=0x7f8d8c0cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.290+0000 7f8d93200640 1 --2- 172.21.15.119:0/4084894728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d8c0ce9d0 0x7f8d8c0cee60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c1458f0 tx=0x7f8d7402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.290+0000 7f8d93200640 1 -- 172.21.15.119:0/4084894728 shutdown_connections 2024-04-25T10:07:54.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.290+0000 7f8d93200640 1 --2- 172.21.15.119:0/4084894728 >> v2:172.21.15.132:3300/0 conn(0x7f8d8c0cc300 0x7f8d8c0cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.290+0000 7f8d93200640 1 --2- 172.21.15.119:0/4084894728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d8c0ce9d0 0x7f8d8c0cee60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.290+0000 7f8d93200640 1 -- 172.21.15.119:0/4084894728 >> 172.21.15.119:0/4084894728 conn(0x7f8d8c06aba0 msgr2=0x7f8d8c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.290+0000 7f8d93200640 1 -- 172.21.15.119:0/4084894728 shutdown_connections 2024-04-25T10:07:54.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.290+0000 7f8d93200640 1 -- 172.21.15.119:0/4084894728 wait complete. 2024-04-25T10:07:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.292+0000 7f8d93200640 1 Processor -- start 2024-04-25T10:07:54.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.292+0000 7f8d93200640 1 -- start start 2024-04-25T10:07:54.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d93200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d8c0cc300 0x7f8d8c13eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d93200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d8c0ce9d0 0x7f8d8c13bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d93200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d8c144e70 0x7f8d8c13c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d93200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c06d970 con 0x7f8d8c144e70 2024-04-25T10:07:54.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d93200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c0cf3a0 con 0x7f8d8c0ce9d0 2024-04-25T10:07:54.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d93200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c1454b0 con 0x7f8d8c0cc300 2024-04-25T10:07:54.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d91200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d8c144e70 0x7f8d8c13c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d91200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d8c144e70 0x7f8d8c13c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49030/0 (socket says 172.21.15.119:49030) 2024-04-25T10:07:54.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d91200640 1 -- 172.21.15.119:0/3804426359 learned_addr learned my addr 172.21.15.119:0/3804426359 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:54.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d8b400640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d8c0ce9d0 0x7f8d8c13bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d8be00640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d8c0cc300 0x7f8d8c13eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d91200640 1 -- 172.21.15.119:0/3804426359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d8c0cc300 msgr2=0x7f8d8c13eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d91200640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d8c0cc300 0x7f8d8c13eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d91200640 1 -- 172.21.15.119:0/3804426359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d8c0ce9d0 msgr2=0x7f8d8c13bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d91200640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d8c0ce9d0 0x7f8d8c13bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d91200640 1 -- 172.21.15.119:0/3804426359 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c0b17c0 con 0x7f8d8c144e70 2024-04-25T10:07:54.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.293+0000 7f8d8be00640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d8c0cc300 0x7f8d8c13eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:54.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.294+0000 7f8d91200640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d8c144e70 0x7f8d8c13c2d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8d84007e40 tx=0x7f8d8400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.294+0000 7f8d83e00640 1 -- 172.21.15.119:0/3804426359 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84014620 con 0x7f8d8c144e70 2024-04-25T10:07:54.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.294+0000 7f8d83e00640 1 -- 172.21.15.119:0/3804426359 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d84019070 con 0x7f8d8c144e70 2024-04-25T10:07:54.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.294+0000 7f8d83e00640 1 -- 172.21.15.119:0/3804426359 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84014a20 con 0x7f8d8c144e70 2024-04-25T10:07:54.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.294+0000 7f8d93200640 1 -- 172.21.15.119:0/3804426359 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8c0b0c40 con 0x7f8d8c144e70 2024-04-25T10:07:54.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.294+0000 7f8d93200640 1 -- 172.21.15.119:0/3804426359 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8c0b1200 con 0x7f8d8c144e70 2024-04-25T10:07:54.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.295+0000 7f8d93200640 1 -- 172.21.15.119:0/3804426359 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8c0750b0 con 0x7f8d8c144e70 2024-04-25T10:07:54.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.296+0000 7f8d83e00640 1 -- 172.21.15.119:0/3804426359 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d84014bc0 con 0x7f8d8c144e70 2024-04-25T10:07:54.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.297+0000 7f8d83e00640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8d6806b4d0 0x7f8d6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.297+0000 7f8d83e00640 1 -- 172.21.15.119:0/3804426359 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f8d8408ac40 con 0x7f8d8c144e70 2024-04-25T10:07:54.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.298+0000 7f8d8be00640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8d6806b4d0 0x7f8d6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.298+0000 7f8d8be00640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8d6806b4d0 0x7f8d6806d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00a830 tx=0x7f8d7c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.301+0000 7f8d83e00640 1 -- 172.21.15.119:0/3804426359 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d840599c0 con 0x7f8d8c144e70 2024-04-25T10:07:54.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.497+0000 7f8d93200640 1 -- 172.21.15.119:0/3804426359 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8d8c0aebe0 con 0x7f8d8c144e70 2024-04-25T10:07:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.498+0000 7f8d83e00640 1 -- 172.21.15.119:0/3804426359 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d840592c0 con 0x7f8d8c144e70 2024-04-25T10:07:54.504 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:07:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.500+0000 7f8d93200640 1 -- 172.21.15.119:0/3804426359 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8d6806b4d0 msgr2=0x7f8d6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.500+0000 7f8d93200640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8d6806b4d0 0x7f8d6806d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00a830 tx=0x7f8d7c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.500+0000 7f8d93200640 1 -- 172.21.15.119:0/3804426359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d8c144e70 msgr2=0x7f8d8c13c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.500+0000 7f8d93200640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d8c144e70 0x7f8d8c13c2d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8d84007e40 tx=0x7f8d8400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.501+0000 7f8d93200640 1 -- 172.21.15.119:0/3804426359 shutdown_connections 2024-04-25T10:07:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.501+0000 7f8d93200640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8d6806b4d0 0x7f8d6806d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.501+0000 7f8d93200640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d8c144e70 0x7f8d8c13c2d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.501+0000 7f8d93200640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d8c0ce9d0 0x7f8d8c13bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.501+0000 7f8d93200640 1 --2- 172.21.15.119:0/3804426359 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d8c0cc300 0x7f8d8c13eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.501+0000 7f8d93200640 1 -- 172.21.15.119:0/3804426359 >> 172.21.15.119:0/3804426359 conn(0x7f8d8c06aba0 msgr2=0x7f8d8c0cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.501+0000 7f8d93200640 1 -- 172.21.15.119:0/3804426359 shutdown_connections 2024-04-25T10:07:54.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.501+0000 7f8d93200640 1 -- 172.21.15.119:0/3804426359 wait complete. 2024-04-25T10:07:54.525 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:54.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.720+0000 7f4842600640 1 Processor -- start 2024-04-25T10:07:54.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.720+0000 7f4842600640 1 -- start start 2024-04-25T10:07:54.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.721+0000 7f4842600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f483c15d2d0 0x7f483c099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.721+0000 7f4842600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f483c099f80 0x7f483c0cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.721+0000 7f4842600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483c0cf250 con 0x7f483c15cc10 2024-04-25T10:07:54.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.721+0000 7f4842600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f483c0ccca0 con 0x7f483c15d2d0 2024-04-25T10:07:54.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.721+0000 7f4842600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483c0ccf80 con 0x7f483c099f80 2024-04-25T10:07:54.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.722+0000 7f4841c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f483c099f80 0x7f483c0cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.722+0000 7f4841c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f483c099f80 0x7f483c0cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:48414/0 (socket says 172.21.15.119:48414) 2024-04-25T10:07:54.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.722+0000 7f4841c00640 1 -- 172.21.15.119:0/3873503459 learned_addr learned my addr 172.21.15.119:0/3873503459 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:54.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.722+0000 7f483be00640 1 --2- 172.21.15.119:0/3873503459 >> v2:172.21.15.132:3300/0 conn(0x7f483c15d2d0 0x7f483c099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.723+0000 7f4841c00640 1 -- 172.21.15.119:0/3873503459 >> v2:172.21.15.132:3300/0 conn(0x7f483c15d2d0 msgr2=0x7f483c099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:54.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.723+0000 7f4841c00640 1 --2- 172.21.15.119:0/3873503459 >> v2:172.21.15.132:3300/0 conn(0x7f483c15d2d0 0x7f483c099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.723+0000 7f4841c00640 1 -- 172.21.15.119:0/3873503459 >> v1:172.21.15.132:6789/0 conn(0x7f483c15cc10 legacy=0x7f483c0cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.723+0000 7f4841c00640 1 -- 172.21.15.119:0/3873503459 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c0cd260 con 0x7f483c099f80 2024-04-25T10:07:54.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.724+0000 7f4841c00640 1 --2- 172.21.15.119:0/3873503459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f483c099f80 0x7f483c0cf730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f48340098a0 tx=0x7f483402a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56593e6d20bf31ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.724+0000 7f483b400640 1 -- 172.21.15.119:0/3873503459 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834037070 con 0x7f483c099f80 2024-04-25T10:07:54.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.725+0000 7f483b400640 1 -- 172.21.15.119:0/3873503459 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4834032440 con 0x7f483c099f80 2024-04-25T10:07:54.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.725+0000 7f483b400640 1 -- 172.21.15.119:0/3873503459 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834032810 con 0x7f483c099f80 2024-04-25T10:07:54.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.725+0000 7f4842600640 1 -- 172.21.15.119:0/3873503459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f483c099f80 msgr2=0x7f483c0cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.725+0000 7f4842600640 1 --2- 172.21.15.119:0/3873503459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f483c099f80 0x7f483c0cf730 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f48340098a0 tx=0x7f483402a290 comp rx=0 tx=0).stop 2024-04-25T10:07:54.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.726+0000 7f4842600640 1 -- 172.21.15.119:0/3873503459 shutdown_connections 2024-04-25T10:07:54.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.726+0000 7f4842600640 1 --2- 172.21.15.119:0/3873503459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f483c099f80 0x7f483c0cf730 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.726+0000 7f4842600640 1 --2- 172.21.15.119:0/3873503459 >> v2:172.21.15.132:3300/0 conn(0x7f483c15d2d0 0x7f483c099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.726+0000 7f4842600640 1 -- 172.21.15.119:0/3873503459 >> 172.21.15.119:0/3873503459 conn(0x7f483c0b30d0 msgr2=0x7f483c0b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.726+0000 7f4842600640 1 -- 172.21.15.119:0/3873503459 shutdown_connections 2024-04-25T10:07:54.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.726+0000 7f4842600640 1 -- 172.21.15.119:0/3873503459 wait complete. 2024-04-25T10:07:54.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.728+0000 7f4842600640 1 Processor -- start 2024-04-25T10:07:54.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.728+0000 7f4842600640 1 -- start start 2024-04-25T10:07:54.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.729+0000 7f4842600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f483c099f80 0x7f483c06cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.729+0000 7f4842600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f483c15cc10 0x7f483c0b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.729+0000 7f4842600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f483c15d2d0 0x7f483c15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.729+0000 7f4842600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483c0cc950 con 0x7f483c15d2d0 2024-04-25T10:07:54.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.729+0000 7f4842600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483c09a9c0 con 0x7f483c15cc10 2024-04-25T10:07:54.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.729+0000 7f4842600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483c0cfc70 con 0x7f483c099f80 2024-04-25T10:07:54.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.729+0000 7f4841c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f483c15d2d0 0x7f483c15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.729+0000 7f4841c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f483c15d2d0 0x7f483c15ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49038/0 (socket says 172.21.15.119:49038) 2024-04-25T10:07:54.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.729+0000 7f4841c00640 1 -- 172.21.15.119:0/1776378823 learned_addr learned my addr 172.21.15.119:0/1776378823 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:54.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.730+0000 7f4841200640 1 --2- 172.21.15.119:0/1776378823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f483c099f80 0x7f483c06cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.730+0000 7f483be00640 1 --2- 172.21.15.119:0/1776378823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f483c15cc10 0x7f483c0b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.730+0000 7f4841c00640 1 -- 172.21.15.119:0/1776378823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f483c099f80 msgr2=0x7f483c06cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.730+0000 7f4841c00640 1 --2- 172.21.15.119:0/1776378823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f483c099f80 0x7f483c06cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.730+0000 7f4841c00640 1 -- 172.21.15.119:0/1776378823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f483c15cc10 msgr2=0x7f483c0b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.730+0000 7f4841c00640 1 --2- 172.21.15.119:0/1776378823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f483c15cc10 0x7f483c0b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.730+0000 7f4841c00640 1 -- 172.21.15.119:0/1776378823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c06d110 con 0x7f483c15d2d0 2024-04-25T10:07:54.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.730+0000 7f4841c00640 1 --2- 172.21.15.119:0/1776378823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f483c15d2d0 0x7f483c15ba20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f48340090b0 tx=0x7f48340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.730+0000 7f4838c00640 1 -- 172.21.15.119:0/1776378823 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834037040 con 0x7f483c15d2d0 2024-04-25T10:07:54.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.731+0000 7f4838c00640 1 -- 172.21.15.119:0/1776378823 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4834042070 con 0x7f483c15d2d0 2024-04-25T10:07:54.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.731+0000 7f4838c00640 1 -- 172.21.15.119:0/1776378823 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483403b950 con 0x7f483c15d2d0 2024-04-25T10:07:54.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.731+0000 7f4842600640 1 -- 172.21.15.119:0/1776378823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483c06d3b0 con 0x7f483c15d2d0 2024-04-25T10:07:54.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.731+0000 7f4842600640 1 -- 172.21.15.119:0/1776378823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483c155390 con 0x7f483c15d2d0 2024-04-25T10:07:54.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.732+0000 7f4842600640 1 -- 172.21.15.119:0/1776378823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f483c0749b0 con 0x7f483c15d2d0 2024-04-25T10:07:54.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.733+0000 7f4838c00640 1 -- 172.21.15.119:0/1776378823 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f483403baf0 con 0x7f483c15d2d0 2024-04-25T10:07:54.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.734+0000 7f4838c00640 1 --2- 172.21.15.119:0/1776378823 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f481806b4d0 0x7f481806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:54.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.734+0000 7f4838c00640 1 -- 172.21.15.119:0/1776378823 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f48340aab10 con 0x7f483c15d2d0 2024-04-25T10:07:54.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.734+0000 7f4841200640 1 --2- 172.21.15.119:0/1776378823 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f481806b4d0 0x7f481806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:54.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.735+0000 7f4841200640 1 --2- 172.21.15.119:0/1776378823 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f481806b4d0 0x7f481806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f483c067060 tx=0x7f482c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:54.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.739+0000 7f4838c00640 1 -- 172.21.15.119:0/1776378823 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4834079930 con 0x7f483c15d2d0 2024-04-25T10:07:54.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.945+0000 7f4842600640 1 -- 172.21.15.119:0/1776378823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f483c09a570 con 0x7f483c15d2d0 2024-04-25T10:07:54.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.946+0000 7f4838c00640 1 -- 172.21.15.119:0/1776378823 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4834079230 con 0x7f483c15d2d0 2024-04-25T10:07:54.952 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2024-04-25T10:07:54.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.949+0000 7f4842600640 1 -- 172.21.15.119:0/1776378823 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f481806b4d0 msgr2=0x7f481806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.949+0000 7f4842600640 1 --2- 172.21.15.119:0/1776378823 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f481806b4d0 0x7f481806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f483c067060 tx=0x7f482c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:54.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.949+0000 7f4842600640 1 -- 172.21.15.119:0/1776378823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f483c15d2d0 msgr2=0x7f483c15ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:54.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.949+0000 7f4842600640 1 --2- 172.21.15.119:0/1776378823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f483c15d2d0 0x7f483c15ba20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f48340090b0 tx=0x7f48340046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.949+0000 7f4842600640 1 -- 172.21.15.119:0/1776378823 shutdown_connections 2024-04-25T10:07:54.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.949+0000 7f4842600640 1 --2- 172.21.15.119:0/1776378823 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f481806b4d0 0x7f481806d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.949+0000 7f4842600640 1 --2- 172.21.15.119:0/1776378823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f483c15d2d0 0x7f483c15ba20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.949+0000 7f4842600640 1 --2- 172.21.15.119:0/1776378823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f483c15cc10 0x7f483c0b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.949+0000 7f4842600640 1 --2- 172.21.15.119:0/1776378823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f483c099f80 0x7f483c06cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:54.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.949+0000 7f4842600640 1 -- 172.21.15.119:0/1776378823 >> 172.21.15.119:0/1776378823 conn(0x7f483c0b30d0 msgr2=0x7f483c06fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:54.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.949+0000 7f4842600640 1 -- 172.21.15.119:0/1776378823 shutdown_connections 2024-04-25T10:07:54.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:54.949+0000 7f4842600640 1 -- 172.21.15.119:0/1776378823 wait complete. 2024-04-25T10:07:54.975 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T10:07:54.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:54.975 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T10:07:55.060 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:07:55.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:55.061 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:07:55.112 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:07:55.112 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:55.171 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:07:55.171 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:55.226 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:07:55.227 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:55.279 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:55.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.487+0000 7f87f6600640 1 Processor -- start 2024-04-25T10:07:55.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.487+0000 7f87f6600640 1 -- start start 2024-04-25T10:07:55.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.489+0000 7f87f6600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f006e8f0 0x7f87f015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.489+0000 7f87f6600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f87f015c820 0x7f87f01654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.489+0000 7f87f6600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f00ce6c0 con 0x7f87f015bf00 2024-04-25T10:07:55.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.489+0000 7f87f6600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f01455e0 con 0x7f87f015c820 2024-04-25T10:07:55.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.489+0000 7f87f6600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f006d450 con 0x7f87f006e8f0 2024-04-25T10:07:55.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.490+0000 7f87e6a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f006e8f0 0x7f87f015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.490+0000 7f87e6a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f006e8f0 0x7f87f015c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:48436/0 (socket says 172.21.15.119:48436) 2024-04-25T10:07:55.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.490+0000 7f87e6a00640 1 -- 172.21.15.119:0/4069209265 learned_addr learned my addr 172.21.15.119:0/4069209265 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:55.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.490+0000 7f87efe00640 1 --2- 172.21.15.119:0/4069209265 >> v2:172.21.15.132:3300/0 conn(0x7f87f015c820 0x7f87f01654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.490+0000 7f87e6a00640 1 -- 172.21.15.119:0/4069209265 >> v2:172.21.15.132:3300/0 conn(0x7f87f015c820 msgr2=0x7f87f01654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.490+0000 7f87e6a00640 1 --2- 172.21.15.119:0/4069209265 >> v2:172.21.15.132:3300/0 conn(0x7f87f015c820 0x7f87f01654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.490+0000 7f87e6a00640 1 -- 172.21.15.119:0/4069209265 >> v1:172.21.15.132:6789/0 conn(0x7f87f015bf00 legacy=0x7f87f00cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.490+0000 7f87e6a00640 1 -- 172.21.15.119:0/4069209265 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f00cc4e0 con 0x7f87f006e8f0 2024-04-25T10:07:55.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.492+0000 7f87e6a00640 1 --2- 172.21.15.119:0/4069209265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f006e8f0 0x7f87f015c2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009870 tx=0x7f87d802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfb1b597994ebbe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.492+0000 7f87eea00640 1 -- 172.21.15.119:0/4069209265 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8037070 con 0x7f87f006e8f0 2024-04-25T10:07:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.492+0000 7f87eea00640 1 -- 172.21.15.119:0/4069209265 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d8032440 con 0x7f87f006e8f0 2024-04-25T10:07:55.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.492+0000 7f87eea00640 1 -- 172.21.15.119:0/4069209265 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8032810 con 0x7f87f006e8f0 2024-04-25T10:07:55.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.493+0000 7f87f6600640 1 -- 172.21.15.119:0/4069209265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f006e8f0 msgr2=0x7f87f015c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.493+0000 7f87f6600640 1 --2- 172.21.15.119:0/4069209265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f006e8f0 0x7f87f015c2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009870 tx=0x7f87d802a270 comp rx=0 tx=0).stop 2024-04-25T10:07:55.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.493+0000 7f87f6600640 1 -- 172.21.15.119:0/4069209265 shutdown_connections 2024-04-25T10:07:55.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.493+0000 7f87f6600640 1 --2- 172.21.15.119:0/4069209265 >> v2:172.21.15.132:3300/0 conn(0x7f87f015c820 0x7f87f01654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.493+0000 7f87f6600640 1 --2- 172.21.15.119:0/4069209265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f006e8f0 0x7f87f015c2e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.493+0000 7f87f6600640 1 -- 172.21.15.119:0/4069209265 >> 172.21.15.119:0/4069209265 conn(0x7f87f006aba0 msgr2=0x7f87f006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:55.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.493+0000 7f87f6600640 1 -- 172.21.15.119:0/4069209265 shutdown_connections 2024-04-25T10:07:55.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.493+0000 7f87f6600640 1 -- 172.21.15.119:0/4069209265 wait complete. 2024-04-25T10:07:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.495+0000 7f87f6600640 1 Processor -- start 2024-04-25T10:07:55.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.495+0000 7f87f6600640 1 -- start start 2024-04-25T10:07:55.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.496+0000 7f87f6600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f006e8f0 0x7f87f013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.496+0000 7f87f6600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87f015bf00 0x7f87f00ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.496+0000 7f87f6600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87f015c820 0x7f87f00cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.496+0000 7f87f6600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f00ccd70 con 0x7f87f015bf00 2024-04-25T10:07:55.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.496+0000 7f87f6600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f00cca90 con 0x7f87f015c820 2024-04-25T10:07:55.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.496+0000 7f87f6600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f00cd050 con 0x7f87f006e8f0 2024-04-25T10:07:55.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.497+0000 7f87e6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87f015bf00 0x7f87f00ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.497+0000 7f87e6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87f015bf00 0x7f87f00ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49040/0 (socket says 172.21.15.119:49040) 2024-04-25T10:07:55.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.497+0000 7f87e6a00640 1 -- 172.21.15.119:0/1486220776 learned_addr learned my addr 172.21.15.119:0/1486220776 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:55.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.497+0000 7f87ef400640 1 --2- 172.21.15.119:0/1486220776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f006e8f0 0x7f87f013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.497+0000 7f87efe00640 1 --2- 172.21.15.119:0/1486220776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87f015c820 0x7f87f00cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.497+0000 7f87e6a00640 1 -- 172.21.15.119:0/1486220776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f006e8f0 msgr2=0x7f87f013ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.497+0000 7f87e6a00640 1 --2- 172.21.15.119:0/1486220776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f006e8f0 0x7f87f013ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.497+0000 7f87e6a00640 1 -- 172.21.15.119:0/1486220776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87f015c820 msgr2=0x7f87f00cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.497+0000 7f87e6a00640 1 --2- 172.21.15.119:0/1486220776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87f015c820 0x7f87f00cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.497+0000 7f87e6a00640 1 -- 172.21.15.119:0/1486220776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f00cb170 con 0x7f87f015bf00 2024-04-25T10:07:55.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.498+0000 7f87e6a00640 1 --2- 172.21.15.119:0/1486220776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87f015bf00 0x7f87f00ca5f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f87d802a2b0 tx=0x7f87d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.498+0000 7f87e7e00640 1 -- 172.21.15.119:0/1486220776 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8037040 con 0x7f87f015bf00 2024-04-25T10:07:55.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.498+0000 7f87e7e00640 1 -- 172.21.15.119:0/1486220776 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d8042070 con 0x7f87f015bf00 2024-04-25T10:07:55.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.498+0000 7f87e7e00640 1 -- 172.21.15.119:0/1486220776 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8030880 con 0x7f87f015bf00 2024-04-25T10:07:55.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.498+0000 7f87f6600640 1 -- 172.21.15.119:0/1486220776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f00cb450 con 0x7f87f015bf00 2024-04-25T10:07:55.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.498+0000 7f87f6600640 1 -- 172.21.15.119:0/1486220776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f013b390 con 0x7f87f015bf00 2024-04-25T10:07:55.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.499+0000 7f87f6600640 1 -- 172.21.15.119:0/1486220776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f006bb30 con 0x7f87f015bf00 2024-04-25T10:07:55.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.500+0000 7f87e7e00640 1 -- 172.21.15.119:0/1486220776 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87d8030a20 con 0x7f87f015bf00 2024-04-25T10:07:55.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.501+0000 7f87e7e00640 1 --2- 172.21.15.119:0/1486220776 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87cc06b4d0 0x7f87cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.502+0000 7f87e7e00640 1 -- 172.21.15.119:0/1486220776 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f87d80aaab0 con 0x7f87f015bf00 2024-04-25T10:07:55.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.502+0000 7f87ef400640 1 --2- 172.21.15.119:0/1486220776 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87cc06b4d0 0x7f87cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.502+0000 7f87ef400640 1 --2- 172.21.15.119:0/1486220776 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87cc06b4d0 0x7f87cc06d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f87e0001d40 tx=0x7f87e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.506+0000 7f87e7e00640 1 -- 172.21.15.119:0/1486220776 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87d80798c0 con 0x7f87f015bf00 2024-04-25T10:07:55.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.703+0000 7f87f6600640 1 -- 172.21.15.119:0/1486220776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87f006af80 con 0x7f87f015bf00 2024-04-25T10:07:55.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.705+0000 7f87e7e00640 1 -- 172.21.15.119:0/1486220776 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16861 (secure 0 0 0) 0x7f87d80791c0 con 0x7f87f015bf00 2024-04-25T10:07:55.711 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:55.712 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":22,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:49.883606+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:55.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.708+0000 7f87f6600640 1 -- 172.21.15.119:0/1486220776 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87cc06b4d0 msgr2=0x7f87cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.708+0000 7f87f6600640 1 --2- 172.21.15.119:0/1486220776 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87cc06b4d0 0x7f87cc06d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f87e0001d40 tx=0x7f87e0009040 comp rx=0 tx=0).stop 2024-04-25T10:07:55.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.708+0000 7f87f6600640 1 -- 172.21.15.119:0/1486220776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87f015bf00 msgr2=0x7f87f00ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.708+0000 7f87f6600640 1 --2- 172.21.15.119:0/1486220776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87f015bf00 0x7f87f00ca5f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f87d802a2b0 tx=0x7f87d80046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.708+0000 7f87f6600640 1 -- 172.21.15.119:0/1486220776 shutdown_connections 2024-04-25T10:07:55.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.708+0000 7f87f6600640 1 --2- 172.21.15.119:0/1486220776 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87cc06b4d0 0x7f87cc06d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.708+0000 7f87f6600640 1 --2- 172.21.15.119:0/1486220776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87f015c820 0x7f87f00cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.708+0000 7f87f6600640 1 --2- 172.21.15.119:0/1486220776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87f015bf00 0x7f87f00ca5f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.708+0000 7f87f6600640 1 --2- 172.21.15.119:0/1486220776 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87f006e8f0 0x7f87f013ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.708+0000 7f87f6600640 1 -- 172.21.15.119:0/1486220776 >> 172.21.15.119:0/1486220776 conn(0x7f87f006aba0 msgr2=0x7f87f00b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:55.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.708+0000 7f87f6600640 1 -- 172.21.15.119:0/1486220776 shutdown_connections 2024-04-25T10:07:55.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.708+0000 7f87f6600640 1 -- 172.21.15.119:0/1486220776 wait complete. 2024-04-25T10:07:55.733 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:05.135832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:05.768083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:55.733 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:55.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.927+0000 7f574f000640 1 Processor -- start 2024-04-25T10:07:55.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.928+0000 7f574f000640 1 -- start start 2024-04-25T10:07:55.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.929+0000 7f574f000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57480b97c0 0x7f57480b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.929+0000 7f574f000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57480b5a30 0x7f57480d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.929+0000 7f574f000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57481627a0 con 0x7f574815dc80 2024-04-25T10:07:55.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.929+0000 7f574f000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5748161f80 con 0x7f57480b97c0 2024-04-25T10:07:55.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.929+0000 7f574f000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57480b5e60 con 0x7f57480b5a30 2024-04-25T10:07:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.930+0000 7f5747400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57480b97c0 0x7f57480b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.930+0000 7f5747400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57480b97c0 0x7f57480b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49052/0 (socket says 172.21.15.119:49052) 2024-04-25T10:07:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.930+0000 7f5747400640 1 -- 172.21.15.119:0/3748421200 learned_addr learned my addr 172.21.15.119:0/3748421200 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.930+0000 7f574d000640 1 --2- 172.21.15.119:0/3748421200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57480b5a30 0x7f57480d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.930+0000 7f5747400640 1 -- 172.21.15.119:0/3748421200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57480b5a30 msgr2=0x7f57480d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.930+0000 7f5747400640 1 --2- 172.21.15.119:0/3748421200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57480b5a30 0x7f57480d2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.930+0000 7f5747400640 1 -- 172.21.15.119:0/3748421200 >> v1:172.21.15.119:6789/0 conn(0x7f574815dc80 legacy=0x7f5748161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.930+0000 7f5747400640 1 -- 172.21.15.119:0/3748421200 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57481621e0 con 0x7f57480b97c0 2024-04-25T10:07:55.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.931+0000 7f5747400640 1 --2- 172.21.15.119:0/3748421200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57480b97c0 0x7f57480b54f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f57340098a0 tx=0x7f573402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3473358a217056 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.931+0000 7f5746a00640 1 -- 172.21.15.119:0/3748421200 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734037070 con 0x7f57480b97c0 2024-04-25T10:07:55.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.932+0000 7f5746a00640 1 -- 172.21.15.119:0/3748421200 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5734032440 con 0x7f57480b97c0 2024-04-25T10:07:55.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.932+0000 7f5746a00640 1 -- 172.21.15.119:0/3748421200 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734032810 con 0x7f57480b97c0 2024-04-25T10:07:55.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.932+0000 7f574f000640 1 -- 172.21.15.119:0/3748421200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57480b97c0 msgr2=0x7f57480b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.932+0000 7f574f000640 1 --2- 172.21.15.119:0/3748421200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57480b97c0 0x7f57480b54f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f57340098a0 tx=0x7f573402a290 comp rx=0 tx=0).stop 2024-04-25T10:07:55.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.932+0000 7f574f000640 1 -- 172.21.15.119:0/3748421200 shutdown_connections 2024-04-25T10:07:55.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.932+0000 7f574f000640 1 --2- 172.21.15.119:0/3748421200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57480b5a30 0x7f57480d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.932+0000 7f574f000640 1 --2- 172.21.15.119:0/3748421200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57480b97c0 0x7f57480b54f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.932+0000 7f574f000640 1 -- 172.21.15.119:0/3748421200 >> 172.21.15.119:0/3748421200 conn(0x7f57480d04f0 msgr2=0x7f57480d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:55.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.932+0000 7f574f000640 1 -- 172.21.15.119:0/3748421200 shutdown_connections 2024-04-25T10:07:55.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.933+0000 7f574f000640 1 -- 172.21.15.119:0/3748421200 wait complete. 2024-04-25T10:07:55.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.934+0000 7f574f000640 1 Processor -- start 2024-04-25T10:07:55.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.934+0000 7f574f000640 1 -- start start 2024-04-25T10:07:55.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.935+0000 7f574f000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57480b5a30 0x7f5748151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.935+0000 7f574f000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57480b97c0 0x7f57480c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.935+0000 7f574f000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f574815dc80 0x7f57480cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.935+0000 7f574f000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574815e060 con 0x7f57480b5a30 2024-04-25T10:07:55.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.935+0000 7f574f000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57480d1c70 con 0x7f57480b97c0 2024-04-25T10:07:55.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.935+0000 7f574f000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57480d29b0 con 0x7f574815dc80 2024-04-25T10:07:55.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.936+0000 7f5747e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57480b5a30 0x7f5748151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.936+0000 7f5747e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57480b5a30 0x7f5748151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49054/0 (socket says 172.21.15.119:49054) 2024-04-25T10:07:55.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.936+0000 7f5747e00640 1 -- 172.21.15.119:0/2608362107 learned_addr learned my addr 172.21.15.119:0/2608362107 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:55.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.936+0000 7f5747e00640 1 -- 172.21.15.119:0/2608362107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f574815dc80 msgr2=0x7f57480cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.936+0000 7f5747e00640 1 --2- 172.21.15.119:0/2608362107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f574815dc80 0x7f57480cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.936+0000 7f5747e00640 1 -- 172.21.15.119:0/2608362107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57480b97c0 msgr2=0x7f57480c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:55.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.936+0000 7f5747e00640 1 --2- 172.21.15.119:0/2608362107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57480b97c0 0x7f57480c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:55.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.936+0000 7f5747e00640 1 -- 172.21.15.119:0/2608362107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57480c84f0 con 0x7f57480b5a30 2024-04-25T10:07:55.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.936+0000 7f5747e00640 1 --2- 172.21.15.119:0/2608362107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57480b5a30 0x7f5748151070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f573800c3c0 tx=0x7f573800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.937+0000 7f573fe00640 1 -- 172.21.15.119:0/2608362107 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57380052d0 con 0x7f57480b5a30 2024-04-25T10:07:55.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.937+0000 7f573fe00640 1 -- 172.21.15.119:0/2608362107 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5738005490 con 0x7f57480b5a30 2024-04-25T10:07:55.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.937+0000 7f573fe00640 1 -- 172.21.15.119:0/2608362107 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738007260 con 0x7f57480b5a30 2024-04-25T10:07:55.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.937+0000 7f574f000640 1 -- 172.21.15.119:0/2608362107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57481f3e70 con 0x7f57480b5a30 2024-04-25T10:07:55.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.937+0000 7f574f000640 1 -- 172.21.15.119:0/2608362107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57481f4430 con 0x7f57480b5a30 2024-04-25T10:07:55.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.938+0000 7f574f000640 1 -- 172.21.15.119:0/2608362107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57480750b0 con 0x7f57480b5a30 2024-04-25T10:07:55.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.939+0000 7f573fe00640 1 -- 172.21.15.119:0/2608362107 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5738007400 con 0x7f57480b5a30 2024-04-25T10:07:55.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.940+0000 7f573fe00640 1 --2- 172.21.15.119:0/2608362107 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f572806b4d0 0x7f572806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:55.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.940+0000 7f573fe00640 1 -- 172.21.15.119:0/2608362107 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f57380897a0 con 0x7f57480b5a30 2024-04-25T10:07:55.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.940+0000 7f5747400640 1 --2- 172.21.15.119:0/2608362107 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f572806b4d0 0x7f572806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:55.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.941+0000 7f5747400640 1 --2- 172.21.15.119:0/2608362107 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f572806b4d0 0x7f572806d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5734034000 tx=0x7f573402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:55.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:55.944+0000 7f573fe00640 1 -- 172.21.15.119:0/2608362107 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57480c84f0 con 0x7f57480b5a30 2024-04-25T10:07:56.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.143+0000 7f574f000640 1 -- 172.21.15.119:0/2608362107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57480d1780 con 0x7f57480b5a30 2024-04-25T10:07:56.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.144+0000 7f573fe00640 1 -- 172.21.15.119:0/2608362107 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5738058520 con 0x7f57480b5a30 2024-04-25T10:07:56.150 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:07:56.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.146+0000 7f574f000640 1 -- 172.21.15.119:0/2608362107 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f572806b4d0 msgr2=0x7f572806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.146+0000 7f574f000640 1 --2- 172.21.15.119:0/2608362107 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f572806b4d0 0x7f572806d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5734034000 tx=0x7f573402c000 comp rx=0 tx=0).stop 2024-04-25T10:07:56.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.146+0000 7f574f000640 1 -- 172.21.15.119:0/2608362107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57480b5a30 msgr2=0x7f5748151070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.146+0000 7f574f000640 1 --2- 172.21.15.119:0/2608362107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57480b5a30 0x7f5748151070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f573800c3c0 tx=0x7f573800c880 comp rx=0 tx=0).stop 2024-04-25T10:07:56.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.147+0000 7f574f000640 1 -- 172.21.15.119:0/2608362107 shutdown_connections 2024-04-25T10:07:56.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.147+0000 7f574f000640 1 --2- 172.21.15.119:0/2608362107 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f572806b4d0 0x7f572806d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.147+0000 7f574f000640 1 --2- 172.21.15.119:0/2608362107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f574815dc80 0x7f57480cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.147+0000 7f574f000640 1 --2- 172.21.15.119:0/2608362107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57480b97c0 0x7f57480c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.147+0000 7f574f000640 1 --2- 172.21.15.119:0/2608362107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57480b5a30 0x7f5748151070 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.147+0000 7f574f000640 1 -- 172.21.15.119:0/2608362107 >> 172.21.15.119:0/2608362107 conn(0x7f57480d04f0 msgr2=0x7f574815b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:56.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.147+0000 7f574f000640 1 -- 172.21.15.119:0/2608362107 shutdown_connections 2024-04-25T10:07:56.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.147+0000 7f574f000640 1 -- 172.21.15.119:0/2608362107 wait complete. 2024-04-25T10:07:56.170 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:56.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.365+0000 7fef71e00640 1 Processor -- start 2024-04-25T10:07:56.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.365+0000 7fef71e00640 1 -- start start 2024-04-25T10:07:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.367+0000 7fef71e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fef6c144e70 0x7fef6c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.367+0000 7fef71e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef6c0cf0e0 0x7fef6c0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.367+0000 7fef71e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c06dcb0 con 0x7fef6c06d7a0 2024-04-25T10:07:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.367+0000 7fef71e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6c15bfe0 con 0x7fef6c0cf0e0 2024-04-25T10:07:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.367+0000 7fef71e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6c06df90 con 0x7fef6c144e70 2024-04-25T10:07:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.367+0000 7fef6b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef6c0cf0e0 0x7fef6c0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.367+0000 7fef6b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef6c0cf0e0 0x7fef6c0cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49064/0 (socket says 172.21.15.119:49064) 2024-04-25T10:07:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.367+0000 7fef6b400640 1 -- 172.21.15.119:0/4073985526 learned_addr learned my addr 172.21.15.119:0/4073985526 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.367+0000 7fef6aa00640 1 --2- 172.21.15.119:0/4073985526 >> v2:172.21.15.132:3300/0 conn(0x7fef6c144e70 0x7fef6c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.368+0000 7fef6b400640 1 -- 172.21.15.119:0/4073985526 >> v2:172.21.15.132:3300/0 conn(0x7fef6c144e70 msgr2=0x7fef6c06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.368+0000 7fef6b400640 1 --2- 172.21.15.119:0/4073985526 >> v2:172.21.15.132:3300/0 conn(0x7fef6c144e70 0x7fef6c06d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.368+0000 7fef6b400640 1 -- 172.21.15.119:0/4073985526 >> v1:172.21.15.119:6789/0 conn(0x7fef6c06d7a0 legacy=0x7fef6c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.368+0000 7fef6b400640 1 -- 172.21.15.119:0/4073985526 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c06e270 con 0x7fef6c0cf0e0 2024-04-25T10:07:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.369+0000 7fef6b400640 1 --2- 172.21.15.119:0/4073985526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef6c0cf0e0 0x7fef6c0cc300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0b4a20 tx=0x7fef6402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0e3d85919cebb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.369+0000 7fef6a000640 1 -- 172.21.15.119:0/4073985526 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64036070 con 0x7fef6c0cf0e0 2024-04-25T10:07:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.369+0000 7fef6a000640 1 -- 172.21.15.119:0/4073985526 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef64031600 con 0x7fef6c0cf0e0 2024-04-25T10:07:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.369+0000 7fef6a000640 1 -- 172.21.15.119:0/4073985526 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef640319d0 con 0x7fef6c0cf0e0 2024-04-25T10:07:56.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.369+0000 7fef71e00640 1 -- 172.21.15.119:0/4073985526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef6c0cf0e0 msgr2=0x7fef6c0cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.369+0000 7fef71e00640 1 --2- 172.21.15.119:0/4073985526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef6c0cf0e0 0x7fef6c0cc300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0b4a20 tx=0x7fef6402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.370+0000 7fef71e00640 1 -- 172.21.15.119:0/4073985526 shutdown_connections 2024-04-25T10:07:56.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.370+0000 7fef71e00640 1 --2- 172.21.15.119:0/4073985526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef6c0cf0e0 0x7fef6c0cc300 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.370+0000 7fef71e00640 1 --2- 172.21.15.119:0/4073985526 >> v2:172.21.15.132:3300/0 conn(0x7fef6c144e70 0x7fef6c06d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.370+0000 7fef71e00640 1 -- 172.21.15.119:0/4073985526 >> 172.21.15.119:0/4073985526 conn(0x7fef6c06aba0 msgr2=0x7fef6c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:56.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.370+0000 7fef71e00640 1 -- 172.21.15.119:0/4073985526 shutdown_connections 2024-04-25T10:07:56.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.370+0000 7fef71e00640 1 -- 172.21.15.119:0/4073985526 wait complete. 2024-04-25T10:07:56.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.371+0000 7fef71e00640 1 Processor -- start 2024-04-25T10:07:56.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.371+0000 7fef71e00640 1 -- start start 2024-04-25T10:07:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.372+0000 7fef71e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef6c06d7a0 0x7fef6c14fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.372+0000 7fef71e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef6c0cf0e0 0x7fef6c0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.372+0000 7fef71e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef6c144e70 0x7fef6c13bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.372+0000 7fef71e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c145630 con 0x7fef6c144e70 2024-04-25T10:07:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.372+0000 7fef71e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c1454b0 con 0x7fef6c06d7a0 2024-04-25T10:07:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.372+0000 7fef71e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c0cc840 con 0x7fef6c0cf0e0 2024-04-25T10:07:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.373+0000 7fef6b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef6c144e70 0x7fef6c13bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.373+0000 7fef6b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef6c144e70 0x7fef6c13bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49080/0 (socket says 172.21.15.119:49080) 2024-04-25T10:07:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.373+0000 7fef6b400640 1 -- 172.21.15.119:0/3558492267 learned_addr learned my addr 172.21.15.119:0/3558492267 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.373+0000 7fef6b400640 1 -- 172.21.15.119:0/3558492267 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef6c0cf0e0 msgr2=0x7fef6c0b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.373+0000 7fef6b400640 1 --2- 172.21.15.119:0/3558492267 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef6c0cf0e0 0x7fef6c0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.373+0000 7fef6b400640 1 -- 172.21.15.119:0/3558492267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef6c06d7a0 msgr2=0x7fef6c14fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.373+0000 7fef6b400640 1 --2- 172.21.15.119:0/3558492267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef6c06d7a0 0x7fef6c14fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.373+0000 7fef6b400640 1 -- 172.21.15.119:0/3558492267 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c0af290 con 0x7fef6c144e70 2024-04-25T10:07:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.373+0000 7fef6b400640 1 --2- 172.21.15.119:0/3558492267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef6c144e70 0x7fef6c13bbd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fef64009f40 tx=0x7fef64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.374+0000 7fef63e00640 1 -- 172.21.15.119:0/3558492267 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64036050 con 0x7fef6c144e70 2024-04-25T10:07:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.374+0000 7fef63e00640 1 -- 172.21.15.119:0/3558492267 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef64042070 con 0x7fef6c144e70 2024-04-25T10:07:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.374+0000 7fef71e00640 1 -- 172.21.15.119:0/3558492267 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c0af570 con 0x7fef6c144e70 2024-04-25T10:07:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.374+0000 7fef63e00640 1 -- 172.21.15.119:0/3558492267 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64009510 con 0x7fef6c144e70 2024-04-25T10:07:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.374+0000 7fef71e00640 1 -- 172.21.15.119:0/3558492267 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c0afb30 con 0x7fef6c144e70 2024-04-25T10:07:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.375+0000 7fef71e00640 1 -- 172.21.15.119:0/3558492267 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c0750b0 con 0x7fef6c144e70 2024-04-25T10:07:56.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.376+0000 7fef63e00640 1 -- 172.21.15.119:0/3558492267 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef64007bd0 con 0x7fef6c144e70 2024-04-25T10:07:56.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.378+0000 7fef63e00640 1 --2- 172.21.15.119:0/3558492267 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fef4806b4d0 0x7fef4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.378+0000 7fef63e00640 1 -- 172.21.15.119:0/3558492267 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fef640ab1f0 con 0x7fef6c144e70 2024-04-25T10:07:56.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.378+0000 7fef6aa00640 1 --2- 172.21.15.119:0/3558492267 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fef4806b4d0 0x7fef4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.378+0000 7fef6aa00640 1 --2- 172.21.15.119:0/3558492267 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fef4806b4d0 0x7fef4806d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fef5c006fd0 tx=0x7fef5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.382+0000 7fef63e00640 1 -- 172.21.15.119:0/3558492267 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef64079f70 con 0x7fef6c144e70 2024-04-25T10:07:56.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.577+0000 7fef71e00640 1 -- 172.21.15.119:0/3558492267 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef6c144480 con 0x7fef6c144e70 2024-04-25T10:07:56.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.577+0000 7fef63e00640 1 -- 172.21.15.119:0/3558492267 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef64079870 con 0x7fef6c144e70 2024-04-25T10:07:56.583 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:07:56.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.580+0000 7fef71e00640 1 -- 172.21.15.119:0/3558492267 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fef4806b4d0 msgr2=0x7fef4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.580+0000 7fef71e00640 1 --2- 172.21.15.119:0/3558492267 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fef4806b4d0 0x7fef4806d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fef5c006fd0 tx=0x7fef5c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:56.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.580+0000 7fef71e00640 1 -- 172.21.15.119:0/3558492267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef6c144e70 msgr2=0x7fef6c13bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.580+0000 7fef71e00640 1 --2- 172.21.15.119:0/3558492267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef6c144e70 0x7fef6c13bbd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fef64009f40 tx=0x7fef64004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.580+0000 7fef71e00640 1 -- 172.21.15.119:0/3558492267 shutdown_connections 2024-04-25T10:07:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.580+0000 7fef71e00640 1 --2- 172.21.15.119:0/3558492267 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fef4806b4d0 0x7fef4806d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.580+0000 7fef71e00640 1 --2- 172.21.15.119:0/3558492267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef6c144e70 0x7fef6c13bbd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.580+0000 7fef71e00640 1 --2- 172.21.15.119:0/3558492267 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef6c0cf0e0 0x7fef6c0b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.580+0000 7fef71e00640 1 --2- 172.21.15.119:0/3558492267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef6c06d7a0 0x7fef6c14fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.580+0000 7fef71e00640 1 -- 172.21.15.119:0/3558492267 >> 172.21.15.119:0/3558492267 conn(0x7fef6c06aba0 msgr2=0x7fef6c0cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.580+0000 7fef71e00640 1 -- 172.21.15.119:0/3558492267 shutdown_connections 2024-04-25T10:07:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.581+0000 7fef71e00640 1 -- 172.21.15.119:0/3558492267 wait complete. 2024-04-25T10:07:56.604 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.790+0000 7f7ab6400640 1 Processor -- start 2024-04-25T10:07:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.791+0000 7f7ab6400640 1 -- start start 2024-04-25T10:07:56.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.792+0000 7f7ab6400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ab00d05b0 0x7f7ab015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.792+0000 7f7ab6400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab01625f0 con 0x7f7ab00d01d0 2024-04-25T10:07:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.792+0000 7f7ab6400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab00cf200 con 0x7f7ab00d1b90 2024-04-25T10:07:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.792+0000 7f7ab6400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab0162160 con 0x7f7ab00d05b0 2024-04-25T10:07:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.792+0000 7f7aafe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ab00d05b0 0x7f7ab015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.792+0000 7f7aafe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ab00d05b0 0x7f7ab015ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49096/0 (socket says 172.21.15.119:49096) 2024-04-25T10:07:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.792+0000 7f7aafe00640 1 -- 172.21.15.119:0/24484823 learned_addr learned my addr 172.21.15.119:0/24484823 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.793+0000 7f7aafe00640 1 -- 172.21.15.119:0/24484823 >> v1:172.21.15.132:6789/0 conn(0x7f7ab00d1b90 legacy=0x7f7ab00d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.793+0000 7f7aafe00640 1 -- 172.21.15.119:0/24484823 >> v1:172.21.15.119:6789/0 conn(0x7f7ab00d01d0 legacy=0x7f7ab00cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.793+0000 7f7aafe00640 1 -- 172.21.15.119:0/24484823 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab0161830 con 0x7f7ab00d05b0 2024-04-25T10:07:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.794+0000 7f7aafe00640 1 --2- 172.21.15.119:0/24484823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab00d05b0 0x7f7ab015ea00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009870 tx=0x7f7a9c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51569cedd55880d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.794+0000 7f7aaf400640 1 -- 172.21.15.119:0/24484823 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c037070 con 0x7f7ab00d05b0 2024-04-25T10:07:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.794+0000 7f7aaf400640 1 -- 172.21.15.119:0/24484823 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c032440 con 0x7f7ab00d05b0 2024-04-25T10:07:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.794+0000 7f7aaf400640 1 -- 172.21.15.119:0/24484823 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c032810 con 0x7f7ab00d05b0 2024-04-25T10:07:56.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.795+0000 7f7ab6400640 1 -- 172.21.15.119:0/24484823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab00d05b0 msgr2=0x7f7ab015ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.795+0000 7f7ab6400640 1 --2- 172.21.15.119:0/24484823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab00d05b0 0x7f7ab015ea00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009870 tx=0x7f7a9c02a210 comp rx=0 tx=0).stop 2024-04-25T10:07:56.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.795+0000 7f7ab6400640 1 -- 172.21.15.119:0/24484823 shutdown_connections 2024-04-25T10:07:56.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.795+0000 7f7ab6400640 1 --2- 172.21.15.119:0/24484823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab00d05b0 0x7f7ab015ea00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.795+0000 7f7ab6400640 1 -- 172.21.15.119:0/24484823 >> 172.21.15.119:0/24484823 conn(0x7f7ab006bed0 msgr2=0x7f7ab006c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:56.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.795+0000 7f7ab6400640 1 -- 172.21.15.119:0/24484823 shutdown_connections 2024-04-25T10:07:56.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.795+0000 7f7ab6400640 1 -- 172.21.15.119:0/24484823 wait complete. 2024-04-25T10:07:56.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.797+0000 7f7ab6400640 1 Processor -- start 2024-04-25T10:07:56.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.797+0000 7f7ab6400640 1 -- start start 2024-04-25T10:07:56.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.798+0000 7f7ab6400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab00d01d0 0x7f7ab00cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.798+0000 7f7ab6400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab00d05b0 0x7f7ab00c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.798+0000 7f7ab6400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ab00d1b90 0x7f7ab00b7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.798+0000 7f7ab6400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab015ef40 con 0x7f7ab00d05b0 2024-04-25T10:07:56.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.798+0000 7f7ab6400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab0161df0 con 0x7f7ab00d01d0 2024-04-25T10:07:56.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.798+0000 7f7ab6400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab015f0c0 con 0x7f7ab00d1b90 2024-04-25T10:07:56.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.798+0000 7f7aafe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab00d05b0 0x7f7ab00c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.798+0000 7f7aafe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab00d05b0 0x7f7ab00c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49102/0 (socket says 172.21.15.119:49102) 2024-04-25T10:07:56.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.798+0000 7f7aafe00640 1 -- 172.21.15.119:0/33823165 learned_addr learned my addr 172.21.15.119:0/33823165 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:56.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.798+0000 7f7aafe00640 1 -- 172.21.15.119:0/33823165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ab00d1b90 msgr2=0x7f7ab00b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.798+0000 7f7aafe00640 1 --2- 172.21.15.119:0/33823165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ab00d1b90 0x7f7ab00b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.798+0000 7f7aafe00640 1 -- 172.21.15.119:0/33823165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab00d01d0 msgr2=0x7f7ab00cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:56.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.798+0000 7f7aafe00640 1 --2- 172.21.15.119:0/33823165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab00d01d0 0x7f7ab00cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:56.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.798+0000 7f7aafe00640 1 -- 172.21.15.119:0/33823165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab00caa90 con 0x7f7ab00d05b0 2024-04-25T10:07:56.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.799+0000 7f7aafe00640 1 --2- 172.21.15.119:0/33823165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab00d05b0 0x7f7ab00c7f00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c000c00 tx=0x7f7a9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.799+0000 7f7aacc00640 1 -- 172.21.15.119:0/33823165 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c037040 con 0x7f7ab00d05b0 2024-04-25T10:07:56.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.799+0000 7f7aacc00640 1 -- 172.21.15.119:0/33823165 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c042070 con 0x7f7ab00d05b0 2024-04-25T10:07:56.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.799+0000 7f7aacc00640 1 -- 172.21.15.119:0/33823165 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c030bd0 con 0x7f7ab00d05b0 2024-04-25T10:07:56.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.799+0000 7f7ab6400640 1 -- 172.21.15.119:0/33823165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab00cad70 con 0x7f7ab00d05b0 2024-04-25T10:07:56.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.799+0000 7f7ab6400640 1 -- 172.21.15.119:0/33823165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab00cb330 con 0x7f7ab00d05b0 2024-04-25T10:07:56.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.800+0000 7f7ab6400640 1 -- 172.21.15.119:0/33823165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab00749b0 con 0x7f7ab00d05b0 2024-04-25T10:07:56.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.801+0000 7f7aacc00640 1 -- 172.21.15.119:0/33823165 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a9c047460 con 0x7f7ab00d05b0 2024-04-25T10:07:56.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.802+0000 7f7aacc00640 1 --2- 172.21.15.119:0/33823165 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a9006b4d0 0x7f7a9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:56.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.802+0000 7f7aacc00640 1 -- 172.21.15.119:0/33823165 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f7a9c0aaf10 con 0x7f7ab00d05b0 2024-04-25T10:07:56.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.802+0000 7f7ab5000640 1 --2- 172.21.15.119:0/33823165 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a9006b4d0 0x7f7a9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:56.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.803+0000 7f7ab5000640 1 --2- 172.21.15.119:0/33823165 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a9006b4d0 0x7f7a9006d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0000f60 tx=0x7f7aa00079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:56.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:56.807+0000 7f7aacc00640 1 -- 172.21.15.119:0/33823165 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a9c045360 con 0x7f7ab00d05b0 2024-04-25T10:07:57.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.002+0000 7f7ab6400640 1 -- 172.21.15.119:0/33823165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7ab00b8dd0 con 0x7f7ab00d05b0 2024-04-25T10:07:57.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.003+0000 7f7aacc00640 1 -- 172.21.15.119:0/33823165 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7a9c079d20 con 0x7f7ab00d05b0 2024-04-25T10:07:57.009 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:07:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.006+0000 7f7ab6400640 1 -- 172.21.15.119:0/33823165 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a9006b4d0 msgr2=0x7f7a9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.006+0000 7f7ab6400640 1 --2- 172.21.15.119:0/33823165 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a9006b4d0 0x7f7a9006d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0000f60 tx=0x7f7aa00079d0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.006+0000 7f7ab6400640 1 -- 172.21.15.119:0/33823165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab00d05b0 msgr2=0x7f7ab00c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.006+0000 7f7ab6400640 1 --2- 172.21.15.119:0/33823165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab00d05b0 0x7f7ab00c7f00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c000c00 tx=0x7f7a9c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.006+0000 7f7ab6400640 1 -- 172.21.15.119:0/33823165 shutdown_connections 2024-04-25T10:07:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.006+0000 7f7ab6400640 1 --2- 172.21.15.119:0/33823165 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a9006b4d0 0x7f7a9006d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.006+0000 7f7ab6400640 1 --2- 172.21.15.119:0/33823165 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7ab00d1b90 0x7f7ab00b7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.006+0000 7f7ab6400640 1 --2- 172.21.15.119:0/33823165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab00d05b0 0x7f7ab00c7f00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.006+0000 7f7ab6400640 1 --2- 172.21.15.119:0/33823165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab00d01d0 0x7f7ab00cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.006+0000 7f7ab6400640 1 -- 172.21.15.119:0/33823165 >> 172.21.15.119:0/33823165 conn(0x7f7ab006bed0 msgr2=0x7f7ab00cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.006+0000 7f7ab6400640 1 -- 172.21.15.119:0/33823165 shutdown_connections 2024-04-25T10:07:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.006+0000 7f7ab6400640 1 -- 172.21.15.119:0/33823165 wait complete. 2024-04-25T10:07:57.032 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:57.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.227+0000 7f4a22800640 1 Processor -- start 2024-04-25T10:07:57.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.227+0000 7f4a22800640 1 -- start start 2024-04-25T10:07:57.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.228+0000 7f4a22800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a1c0b4900 0x7f4a1c1654c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.228+0000 7f4a22800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c06da40 con 0x7f4a1c0aedc0 2024-04-25T10:07:57.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.228+0000 7f4a22800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c06dd20 con 0x7f4a1c15ae20 2024-04-25T10:07:57.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.228+0000 7f4a22800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c15c050 con 0x7f4a1c0b4900 2024-04-25T10:07:57.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.229+0000 7f4a1be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a1c0b4900 0x7f4a1c1654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.229+0000 7f4a1be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a1c0b4900 0x7f4a1c1654c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49114/0 (socket says 172.21.15.119:49114) 2024-04-25T10:07:57.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.229+0000 7f4a1be00640 1 -- 172.21.15.119:0/1707782128 learned_addr learned my addr 172.21.15.119:0/1707782128 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:57.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.229+0000 7f4a1be00640 1 -- 172.21.15.119:0/1707782128 >> v1:172.21.15.132:6789/0 conn(0x7f4a1c15ae20 legacy=0x7f4a1c0b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:57.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.229+0000 7f4a1be00640 1 -- 172.21.15.119:0/1707782128 >> v1:172.21.15.119:6789/0 conn(0x7f4a1c0aedc0 legacy=0x7f4a1c06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.229+0000 7f4a1be00640 1 -- 172.21.15.119:0/1707782128 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c06e000 con 0x7f4a1c0b4900 2024-04-25T10:07:57.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.230+0000 7f4a1be00640 1 --2- 172.21.15.119:0/1707782128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c0b4900 0x7f4a1c1654c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009e50 tx=0x7f4a1402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94dbf57b966d73d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.230+0000 7f4a1aa00640 1 -- 172.21.15.119:0/1707782128 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14036070 con 0x7f4a1c0b4900 2024-04-25T10:07:57.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.231+0000 7f4a1aa00640 1 -- 172.21.15.119:0/1707782128 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a14031600 con 0x7f4a1c0b4900 2024-04-25T10:07:57.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.231+0000 7f4a1aa00640 1 -- 172.21.15.119:0/1707782128 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140319d0 con 0x7f4a1c0b4900 2024-04-25T10:07:57.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.231+0000 7f4a22800640 1 -- 172.21.15.119:0/1707782128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c0b4900 msgr2=0x7f4a1c1654c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.231+0000 7f4a22800640 1 --2- 172.21.15.119:0/1707782128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c0b4900 0x7f4a1c1654c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009e50 tx=0x7f4a1402a6d0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.231+0000 7f4a22800640 1 -- 172.21.15.119:0/1707782128 shutdown_connections 2024-04-25T10:07:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.231+0000 7f4a22800640 1 --2- 172.21.15.119:0/1707782128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c0b4900 0x7f4a1c1654c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.231+0000 7f4a22800640 1 -- 172.21.15.119:0/1707782128 >> 172.21.15.119:0/1707782128 conn(0x7f4a1c06aba0 msgr2=0x7f4a1c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.232+0000 7f4a22800640 1 -- 172.21.15.119:0/1707782128 shutdown_connections 2024-04-25T10:07:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.232+0000 7f4a22800640 1 -- 172.21.15.119:0/1707782128 wait complete. 2024-04-25T10:07:57.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.233+0000 7f4a22800640 1 Processor -- start 2024-04-25T10:07:57.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.234+0000 7f4a22800640 1 -- start start 2024-04-25T10:07:57.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.234+0000 7f4a22800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a1c0aedc0 0x7f4a1c0ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.234+0000 7f4a22800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a1c0b4900 0x7f4a1c0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.234+0000 7f4a22800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c15ae20 0x7f4a1c1f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.234+0000 7f4a22800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c15d7d0 con 0x7f4a1c15ae20 2024-04-25T10:07:57.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.234+0000 7f4a22800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0af1a0 con 0x7f4a1c0b4900 2024-04-25T10:07:57.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.234+0000 7f4a22800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c15d950 con 0x7f4a1c0aedc0 2024-04-25T10:07:57.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.235+0000 7f4a1be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c15ae20 0x7f4a1c1f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.235+0000 7f4a1be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c15ae20 0x7f4a1c1f39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49128/0 (socket says 172.21.15.119:49128) 2024-04-25T10:07:57.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.235+0000 7f4a1be00640 1 -- 172.21.15.119:0/3339656690 learned_addr learned my addr 172.21.15.119:0/3339656690 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.235+0000 7f4a1be00640 1 -- 172.21.15.119:0/3339656690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a1c0aedc0 msgr2=0x7f4a1c0ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.235+0000 7f4a13e00640 1 --2- 172.21.15.119:0/3339656690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a1c0b4900 0x7f4a1c0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.235+0000 7f4a1b400640 1 --2- 172.21.15.119:0/3339656690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a1c0aedc0 0x7f4a1c0ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.235+0000 7f4a1be00640 1 --2- 172.21.15.119:0/3339656690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a1c0aedc0 0x7f4a1c0ad8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.235+0000 7f4a1be00640 1 -- 172.21.15.119:0/3339656690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a1c0b4900 msgr2=0x7f4a1c0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.235+0000 7f4a1be00640 1 --2- 172.21.15.119:0/3339656690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a1c0b4900 0x7f4a1c0ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.235+0000 7f4a1be00640 1 -- 172.21.15.119:0/3339656690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c159f10 con 0x7f4a1c15ae20 2024-04-25T10:07:57.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.236+0000 7f4a1be00640 1 --2- 172.21.15.119:0/3339656690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c15ae20 0x7f4a1c1f39d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009e50 tx=0x7f4a14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.236+0000 7f4a13400640 1 -- 172.21.15.119:0/3339656690 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14036050 con 0x7f4a1c15ae20 2024-04-25T10:07:57.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.236+0000 7f4a13400640 1 -- 172.21.15.119:0/3339656690 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a14042070 con 0x7f4a1c15ae20 2024-04-25T10:07:57.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.236+0000 7f4a13400640 1 -- 172.21.15.119:0/3339656690 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14009510 con 0x7f4a1c15ae20 2024-04-25T10:07:57.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.236+0000 7f4a22800640 1 -- 172.21.15.119:0/3339656690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c1f48c0 con 0x7f4a1c15ae20 2024-04-25T10:07:57.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.236+0000 7f4a22800640 1 -- 172.21.15.119:0/3339656690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c1f70d0 con 0x7f4a1c15ae20 2024-04-25T10:07:57.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.237+0000 7f4a22800640 1 -- 172.21.15.119:0/3339656690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c0750b0 con 0x7f4a1c15ae20 2024-04-25T10:07:57.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.238+0000 7f4a13400640 1 -- 172.21.15.119:0/3339656690 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a14007bd0 con 0x7f4a1c15ae20 2024-04-25T10:07:57.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.239+0000 7f4a13400640 1 --2- 172.21.15.119:0/3339656690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f49f806b4d0 0x7f49f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.239+0000 7f4a13400640 1 -- 172.21.15.119:0/3339656690 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f4a140ab230 con 0x7f4a1c15ae20 2024-04-25T10:07:57.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.240+0000 7f4a1b400640 1 --2- 172.21.15.119:0/3339656690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f49f806b4d0 0x7f49f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.240+0000 7f4a1b400640 1 --2- 172.21.15.119:0/3339656690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f49f806b4d0 0x7f49f806d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c15a180 tx=0x7f4a0c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.247+0000 7f4a13400640 1 -- 172.21.15.119:0/3339656690 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a1407a040 con 0x7f4a1c15ae20 2024-04-25T10:07:57.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.456+0000 7f4a22800640 1 -- 172.21.15.119:0/3339656690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4a1c06c220 con 0x7f4a1c15ae20 2024-04-25T10:07:57.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.457+0000 7f4a13400640 1 -- 172.21.15.119:0/3339656690 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4a14079940 con 0x7f4a1c15ae20 2024-04-25T10:07:57.462 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2024-04-25T10:07:57.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.459+0000 7f4a22800640 1 -- 172.21.15.119:0/3339656690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f49f806b4d0 msgr2=0x7f49f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.459+0000 7f4a22800640 1 --2- 172.21.15.119:0/3339656690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f49f806b4d0 0x7f49f806d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c15a180 tx=0x7f4a0c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:57.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.459+0000 7f4a22800640 1 -- 172.21.15.119:0/3339656690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c15ae20 msgr2=0x7f4a1c1f39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.459+0000 7f4a22800640 1 --2- 172.21.15.119:0/3339656690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c15ae20 0x7f4a1c1f39d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009e50 tx=0x7f4a14004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:57.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.460+0000 7f4a22800640 1 -- 172.21.15.119:0/3339656690 shutdown_connections 2024-04-25T10:07:57.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.460+0000 7f4a22800640 1 --2- 172.21.15.119:0/3339656690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f49f806b4d0 0x7f49f806d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.460+0000 7f4a22800640 1 --2- 172.21.15.119:0/3339656690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a1c15ae20 0x7f4a1c1f39d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.460+0000 7f4a22800640 1 --2- 172.21.15.119:0/3339656690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a1c0b4900 0x7f4a1c0ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.460+0000 7f4a22800640 1 --2- 172.21.15.119:0/3339656690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a1c0aedc0 0x7f4a1c0ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.460+0000 7f4a22800640 1 -- 172.21.15.119:0/3339656690 >> 172.21.15.119:0/3339656690 conn(0x7f4a1c06aba0 msgr2=0x7f4a1c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:57.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.460+0000 7f4a22800640 1 -- 172.21.15.119:0/3339656690 shutdown_connections 2024-04-25T10:07:57.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.460+0000 7f4a22800640 1 -- 172.21.15.119:0/3339656690 wait complete. 2024-04-25T10:07:57.484 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T10:07:57.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:57.485 DEBUG:teuthology.orchestra.run.smithi132:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T10:07:57.590 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2024-04-25T10:07:57.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:57.591 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:07:57.638 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:07:57.638 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:57.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.840+0000 7f570e200640 1 Processor -- start 2024-04-25T10:07:57.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.841+0000 7f570e200640 1 -- start start 2024-04-25T10:07:57.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.842+0000 7f570e200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f570815bf00 0x7f57080cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.842+0000 7f570e200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57080cc310 0x7f570815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.842+0000 7f570e200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57080ce6c0 con 0x7f570815c820 2024-04-25T10:07:57.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.842+0000 7f570e200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57081455e0 con 0x7f570815bf00 2024-04-25T10:07:57.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.842+0000 7f570e200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570806d450 con 0x7f57080cc310 2024-04-25T10:07:57.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.843+0000 7f5707400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f570815bf00 0x7f57080cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.843+0000 7f5707400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f570815bf00 0x7f57080cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49136/0 (socket says 172.21.15.119:49136) 2024-04-25T10:07:57.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.843+0000 7f5707e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f570815c820 0x7f570806d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34884/0 (socket says 172.21.15.119:34884) 2024-04-25T10:07:57.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.843+0000 7f5707400640 1 -- 172.21.15.119:0/543952662 learned_addr learned my addr 172.21.15.119:0/543952662 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:57.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.843+0000 7f56ffe00640 1 --2- 172.21.15.119:0/543952662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57080cc310 0x7f570815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.843+0000 7f5707400640 1 -- 172.21.15.119:0/543952662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57080cc310 msgr2=0x7f570815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:57.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.843+0000 7f5707400640 1 --2- 172.21.15.119:0/543952662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57080cc310 0x7f570815c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.843+0000 7f5707400640 1 -- 172.21.15.119:0/543952662 >> v1:172.21.15.119:6789/0 conn(0x7f570815c820 legacy=0x7f570806d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.843+0000 7f5707400640 1 -- 172.21.15.119:0/543952662 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57080cc7b0 con 0x7f570815bf00 2024-04-25T10:07:57.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.843+0000 7f5706a00640 1 -- 172.21.15.119:0/543952662 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222769144 0 0) 0x7f5700001490 con 0x7f570815c820 2024-04-25T10:07:57.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.845+0000 7f5707400640 1 --2- 172.21.15.119:0/543952662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570815bf00 0x7f57080cbdd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009870 tx=0x7f56f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4df067dc901611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.845+0000 7f5706a00640 1 -- 172.21.15.119:0/543952662 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8037070 con 0x7f570815bf00 2024-04-25T10:07:57.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.845+0000 7f5706a00640 1 -- 172.21.15.119:0/543952662 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f8032440 con 0x7f570815bf00 2024-04-25T10:07:57.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.845+0000 7f5706a00640 1 -- 172.21.15.119:0/543952662 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8032810 con 0x7f570815bf00 2024-04-25T10:07:57.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.846+0000 7f570e200640 1 -- 172.21.15.119:0/543952662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570815bf00 msgr2=0x7f57080cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.846+0000 7f570e200640 1 --2- 172.21.15.119:0/543952662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570815bf00 0x7f57080cbdd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009870 tx=0x7f56f802a270 comp rx=0 tx=0).stop 2024-04-25T10:07:57.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.846+0000 7f570e200640 1 -- 172.21.15.119:0/543952662 shutdown_connections 2024-04-25T10:07:57.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.846+0000 7f570e200640 1 --2- 172.21.15.119:0/543952662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57080cc310 0x7f570815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.846+0000 7f570e200640 1 --2- 172.21.15.119:0/543952662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570815bf00 0x7f57080cbdd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.846+0000 7f570e200640 1 -- 172.21.15.119:0/543952662 >> 172.21.15.119:0/543952662 conn(0x7f570806aba0 msgr2=0x7f570806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:57.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.846+0000 7f570e200640 1 -- 172.21.15.119:0/543952662 shutdown_connections 2024-04-25T10:07:57.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.846+0000 7f570e200640 1 -- 172.21.15.119:0/543952662 wait complete. 2024-04-25T10:07:57.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.848+0000 7f570e200640 1 Processor -- start 2024-04-25T10:07:57.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.848+0000 7f570e200640 1 -- start start 2024-04-25T10:07:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.849+0000 7f570e200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57080cc310 0x7f57081f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.849+0000 7f570e200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570815bf00 0x7f57080c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.849+0000 7f570e200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f570815c820 0x7f57080c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.849+0000 7f570e200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57080cd050 con 0x7f570815bf00 2024-04-25T10:07:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.849+0000 7f570e200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57080ccd70 con 0x7f57080cc310 2024-04-25T10:07:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.849+0000 7f570e200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570815cca0 con 0x7f570815c820 2024-04-25T10:07:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.850+0000 7f56ffe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570815bf00 0x7f57080c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.850+0000 7f56ffe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570815bf00 0x7f57080c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49152/0 (socket says 172.21.15.119:49152) 2024-04-25T10:07:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.850+0000 7f56ffe00640 1 -- 172.21.15.119:0/1302420532 learned_addr learned my addr 172.21.15.119:0/1302420532 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.850+0000 7f5707e00640 1 --2- 172.21.15.119:0/1302420532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f570815c820 0x7f57080c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.850+0000 7f5707400640 1 --2- 172.21.15.119:0/1302420532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57080cc310 0x7f57081f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.850+0000 7f56ffe00640 1 -- 172.21.15.119:0/1302420532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f570815c820 msgr2=0x7f57080c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.850+0000 7f56ffe00640 1 --2- 172.21.15.119:0/1302420532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f570815c820 0x7f57080c7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.850+0000 7f56ffe00640 1 -- 172.21.15.119:0/1302420532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57080cc310 msgr2=0x7f57081f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.850+0000 7f56ffe00640 1 --2- 172.21.15.119:0/1302420532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57080cc310 0x7f57081f3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.850+0000 7f56ffe00640 1 -- 172.21.15.119:0/1302420532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57081f3660 con 0x7f570815bf00 2024-04-25T10:07:57.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.851+0000 7f56ffe00640 1 --2- 172.21.15.119:0/1302420532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570815bf00 0x7f57080c6ef0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f56f000d800 tx=0x7f56f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.851+0000 7f56ff400640 1 -- 172.21.15.119:0/1302420532 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f000b6c0 con 0x7f570815bf00 2024-04-25T10:07:57.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.851+0000 7f56ff400640 1 -- 172.21.15.119:0/1302420532 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f0019070 con 0x7f570815bf00 2024-04-25T10:07:57.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.851+0000 7f570e200640 1 -- 172.21.15.119:0/1302420532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5708143660 con 0x7f570815bf00 2024-04-25T10:07:57.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.851+0000 7f56ff400640 1 -- 172.21.15.119:0/1302420532 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f000ba80 con 0x7f570815bf00 2024-04-25T10:07:57.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.851+0000 7f570e200640 1 -- 172.21.15.119:0/1302420532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5708143c20 con 0x7f570815bf00 2024-04-25T10:07:57.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.852+0000 7f570e200640 1 -- 172.21.15.119:0/1302420532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57080749b0 con 0x7f570815bf00 2024-04-25T10:07:57.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.853+0000 7f56ff400640 1 -- 172.21.15.119:0/1302420532 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56f000bc20 con 0x7f570815bf00 2024-04-25T10:07:57.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.854+0000 7f56ff400640 1 --2- 172.21.15.119:0/1302420532 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f56e406b4d0 0x7f56e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:57.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.854+0000 7f56ff400640 1 -- 172.21.15.119:0/1302420532 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f56f008a760 con 0x7f570815bf00 2024-04-25T10:07:57.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.855+0000 7f5707400640 1 --2- 172.21.15.119:0/1302420532 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f56e406b4d0 0x7f56e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:57.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.855+0000 7f5707400640 1 --2- 172.21.15.119:0/1302420532 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f56e406b4d0 0x7f56e406d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f56f8034000 tx=0x7f56f802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:57.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:57.860+0000 7f56ff400640 1 -- 172.21.15.119:0/1302420532 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57081f3660 con 0x7f570815bf00 2024-04-25T10:07:58.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.104+0000 7f570e200640 1 -- 172.21.15.119:0/1302420532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57081f4220 con 0x7f570815bf00 2024-04-25T10:07:58.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.105+0000 7f56ff400640 1 -- 172.21.15.119:0/1302420532 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4941 (secure 0 0 0) 0x7f56f001c360 con 0x7f570815bf00 2024-04-25T10:07:58.111 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:58.112 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1385207078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1385207078},{"type":"v1","addr":"172.21.15.132:6834","nonce":1385207078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1580923823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1580923823},{"type":"v1","addr":"172.21.15.119:6836","nonce":1580923823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4260,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/954053568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":954053568},{"type":"v1","addr":"172.21.15.132:6835","nonce":954053568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:06.743184+0000","modified":"2024-04-25T10:07:10.118630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1632024048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1632024048},{"type":"v1","addr":"172.21.15.119:6837","nonce":1632024048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:07:58.112 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2024-04-25T10:07:58.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.108+0000 7f570e200640 1 -- 172.21.15.119:0/1302420532 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f56e406b4d0 msgr2=0x7f56e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.108+0000 7f570e200640 1 --2- 172.21.15.119:0/1302420532 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f56e406b4d0 0x7f56e406d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f56f8034000 tx=0x7f56f802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:58.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.108+0000 7f570e200640 1 -- 172.21.15.119:0/1302420532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570815bf00 msgr2=0x7f57080c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.108+0000 7f570e200640 1 --2- 172.21.15.119:0/1302420532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570815bf00 0x7f57080c6ef0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f56f000d800 tx=0x7f56f000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.108+0000 7f570e200640 1 -- 172.21.15.119:0/1302420532 shutdown_connections 2024-04-25T10:07:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.108+0000 7f570e200640 1 --2- 172.21.15.119:0/1302420532 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f56e406b4d0 0x7f56e406d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.108+0000 7f570e200640 1 --2- 172.21.15.119:0/1302420532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f570815c820 0x7f57080c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.108+0000 7f570e200640 1 --2- 172.21.15.119:0/1302420532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f570815bf00 0x7f57080c6ef0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.108+0000 7f570e200640 1 --2- 172.21.15.119:0/1302420532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57080cc310 0x7f57081f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.108+0000 7f570e200640 1 -- 172.21.15.119:0/1302420532 >> 172.21.15.119:0/1302420532 conn(0x7f570806aba0 msgr2=0x7f570815ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.109+0000 7f570e200640 1 -- 172.21.15.119:0/1302420532 shutdown_connections 2024-04-25T10:07:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.109+0000 7f570e200640 1 -- 172.21.15.119:0/1302420532 wait complete. 2024-04-25T10:07:58.133 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:58.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.326+0000 7f06dbe00640 1 Processor -- start 2024-04-25T10:07:58.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.326+0000 7f06dbe00640 1 -- start start 2024-04-25T10:07:58.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.328+0000 7f06dbe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f06dc0b4410 0x7f06dc15e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.328+0000 7f06dbe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06dc15eb30 0x7f06dc0acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.328+0000 7f06dbe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06dc0af000 con 0x7f06dc147270 2024-04-25T10:07:58.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.328+0000 7f06dbe00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06dc0b4a80 con 0x7f06dc0b4410 2024-04-25T10:07:58.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.328+0000 7f06dbe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc0b4d60 con 0x7f06dc15eb30 2024-04-25T10:07:58.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.328+0000 7f06d2000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f06dc0b4410 0x7f06dc15e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.328+0000 7f06d2000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f06dc0b4410 0x7f06dc15e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49168/0 (socket says 172.21.15.119:49168) 2024-04-25T10:07:58.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.328+0000 7f06d2000640 1 -- 172.21.15.119:0/357908234 learned_addr learned my addr 172.21.15.119:0/357908234 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:58.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.328+0000 7f06d2000640 1 -- 172.21.15.119:0/357908234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06dc15eb30 msgr2=0x7f06dc0acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.329+0000 7f06db400640 1 --2- 172.21.15.119:0/357908234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06dc15eb30 0x7f06dc0acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.329+0000 7f06d2000640 1 --2- 172.21.15.119:0/357908234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06dc15eb30 0x7f06dc0acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.329+0000 7f06d2000640 1 -- 172.21.15.119:0/357908234 >> v1:172.21.15.132:6789/0 conn(0x7f06dc147270 legacy=0x7f06dc15dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.329+0000 7f06d2000640 1 -- 172.21.15.119:0/357908234 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc0b5040 con 0x7f06dc0b4410 2024-04-25T10:07:58.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.330+0000 7f06d2000640 1 --2- 172.21.15.119:0/357908234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06dc0b4410 0x7f06dc15e5f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f06dc06f500 tx=0x7f06c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69195a965ae1a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.330+0000 7f06da000640 1 -- 172.21.15.119:0/357908234 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4037070 con 0x7f06dc0b4410 2024-04-25T10:07:58.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.330+0000 7f06da000640 1 -- 172.21.15.119:0/357908234 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c4032440 con 0x7f06dc0b4410 2024-04-25T10:07:58.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.330+0000 7f06da000640 1 -- 172.21.15.119:0/357908234 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4032810 con 0x7f06dc0b4410 2024-04-25T10:07:58.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.331+0000 7f06dbe00640 1 -- 172.21.15.119:0/357908234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06dc0b4410 msgr2=0x7f06dc15e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.331+0000 7f06dbe00640 1 --2- 172.21.15.119:0/357908234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06dc0b4410 0x7f06dc15e5f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f06dc06f500 tx=0x7f06c402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.331+0000 7f06dbe00640 1 -- 172.21.15.119:0/357908234 shutdown_connections 2024-04-25T10:07:58.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.331+0000 7f06dbe00640 1 --2- 172.21.15.119:0/357908234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06dc15eb30 0x7f06dc0acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.331+0000 7f06dbe00640 1 --2- 172.21.15.119:0/357908234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06dc0b4410 0x7f06dc15e5f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.331+0000 7f06dbe00640 1 -- 172.21.15.119:0/357908234 >> 172.21.15.119:0/357908234 conn(0x7f06dc06b1d0 msgr2=0x7f06dc0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.331+0000 7f06dbe00640 1 -- 172.21.15.119:0/357908234 shutdown_connections 2024-04-25T10:07:58.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.331+0000 7f06dbe00640 1 -- 172.21.15.119:0/357908234 wait complete. 2024-04-25T10:07:58.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.333+0000 7f06dbe00640 1 Processor -- start 2024-04-25T10:07:58.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.333+0000 7f06dbe00640 1 -- start start 2024-04-25T10:07:58.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06dbe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06dc0b4410 0x7f06dc143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06dbe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06dc147270 0x7f06dc0cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06dbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06dc15eb30 0x7f06dc0cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06dbe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc0ad4c0 con 0x7f06dc15eb30 2024-04-25T10:07:58.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06dbe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc0b5600 con 0x7f06dc0b4410 2024-04-25T10:07:58.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06dbe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc0ad640 con 0x7f06dc147270 2024-04-25T10:07:58.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06db400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06dc15eb30 0x7f06dc0cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06db400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06dc15eb30 0x7f06dc0cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49180/0 (socket says 172.21.15.119:49180) 2024-04-25T10:07:58.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06db400640 1 -- 172.21.15.119:0/2482849307 learned_addr learned my addr 172.21.15.119:0/2482849307 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:58.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06daa00640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06dc0b4410 0x7f06dc143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06d2000640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06dc147270 0x7f06dc0cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06db400640 1 -- 172.21.15.119:0/2482849307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06dc147270 msgr2=0x7f06dc0cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06db400640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06dc147270 0x7f06dc0cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06db400640 1 -- 172.21.15.119:0/2482849307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06dc0b4410 msgr2=0x7f06dc143e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06db400640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06dc0b4410 0x7f06dc143e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.334+0000 7f06db400640 1 -- 172.21.15.119:0/2482849307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc0cc300 con 0x7f06dc15eb30 2024-04-25T10:07:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.335+0000 7f06daa00640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06dc0b4410 0x7f06dc143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.335+0000 7f06db400640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06dc15eb30 0x7f06dc0cd910 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f06d400cc30 tx=0x7f06d400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.335+0000 7f06d3400640 1 -- 172.21.15.119:0/2482849307 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d400ce40 con 0x7f06dc15eb30 2024-04-25T10:07:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.335+0000 7f06d3400640 1 -- 172.21.15.119:0/2482849307 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d4003990 con 0x7f06dc15eb30 2024-04-25T10:07:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.335+0000 7f06d3400640 1 -- 172.21.15.119:0/2482849307 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4003d40 con 0x7f06dc15eb30 2024-04-25T10:07:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.335+0000 7f06dbe00640 1 -- 172.21.15.119:0/2482849307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06dc0cc5e0 con 0x7f06dc15eb30 2024-04-25T10:07:58.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.335+0000 7f06dbe00640 1 -- 172.21.15.119:0/2482849307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06dc1f37b0 con 0x7f06dc15eb30 2024-04-25T10:07:58.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.336+0000 7f06dbe00640 1 -- 172.21.15.119:0/2482849307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06dc0749b0 con 0x7f06dc15eb30 2024-04-25T10:07:58.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.337+0000 7f06d3400640 1 -- 172.21.15.119:0/2482849307 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06d4003ee0 con 0x7f06dc15eb30 2024-04-25T10:07:58.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.339+0000 7f06d3400640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f06b806b4d0 0x7f06b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.339+0000 7f06d3400640 1 -- 172.21.15.119:0/2482849307 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f06d408a710 con 0x7f06dc15eb30 2024-04-25T10:07:58.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.339+0000 7f06daa00640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f06b806b4d0 0x7f06b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.340+0000 7f06daa00640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f06b806b4d0 0x7f06b806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f06cc003eb0 tx=0x7f06cc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.343+0000 7f06d3400640 1 -- 172.21.15.119:0/2482849307 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06d4014030 con 0x7f06dc15eb30 2024-04-25T10:07:58.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.537+0000 7f06dbe00640 1 -- 172.21.15.119:0/2482849307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06dc0cb220 con 0x7f06dc15eb30 2024-04-25T10:07:58.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.539+0000 7f06d3400640 1 -- 172.21.15.119:0/2482849307 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16861 (secure 0 0 0) 0x7f06d4059520 con 0x7f06dc15eb30 2024-04-25T10:07:58.546 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:07:58.546 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":22,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:49.883606+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:58.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.541+0000 7f06dbe00640 1 -- 172.21.15.119:0/2482849307 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f06b806b4d0 msgr2=0x7f06b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.541+0000 7f06dbe00640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f06b806b4d0 0x7f06b806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f06cc003eb0 tx=0x7f06cc009040 comp rx=0 tx=0).stop 2024-04-25T10:07:58.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.541+0000 7f06dbe00640 1 -- 172.21.15.119:0/2482849307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06dc15eb30 msgr2=0x7f06dc0cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.541+0000 7f06dbe00640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06dc15eb30 0x7f06dc0cd910 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f06d400cc30 tx=0x7f06d400a5c0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.542+0000 7f06dbe00640 1 -- 172.21.15.119:0/2482849307 shutdown_connections 2024-04-25T10:07:58.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.542+0000 7f06dbe00640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f06b806b4d0 0x7f06b806d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.542+0000 7f06dbe00640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06dc15eb30 0x7f06dc0cd910 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.542+0000 7f06dbe00640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06dc147270 0x7f06dc0cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.542+0000 7f06dbe00640 1 --2- 172.21.15.119:0/2482849307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06dc0b4410 0x7f06dc143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.542+0000 7f06dbe00640 1 -- 172.21.15.119:0/2482849307 >> 172.21.15.119:0/2482849307 conn(0x7f06dc06b1d0 msgr2=0x7f06dc144d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.542+0000 7f06dbe00640 1 -- 172.21.15.119:0/2482849307 shutdown_connections 2024-04-25T10:07:58.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.542+0000 7f06dbe00640 1 -- 172.21.15.119:0/2482849307 wait complete. 2024-04-25T10:07:58.568 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:05.135832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:05.768083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:58.569 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:58.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.769+0000 7f2cc7e00640 1 Processor -- start 2024-04-25T10:07:58.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.769+0000 7f2cc7e00640 1 -- start start 2024-04-25T10:07:58.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.770+0000 7f2cc7e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2cc806aba0 0x7f2cc806af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.770+0000 7f2cc7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cc806f040 0x7f2cc815e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.770+0000 7f2cc7e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2cc80cec60 0x7f2cc815e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.771+0000 7f2cc7e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc806b4c0 con 0x7f2cc806aba0 2024-04-25T10:07:58.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.771+0000 7f2cc7e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc806d080 con 0x7f2cc80cec60 2024-04-25T10:07:58.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.771+0000 7f2cc7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc806d360 con 0x7f2cc806f040 2024-04-25T10:07:58.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.771+0000 7f2cc6a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2cc806aba0 0x7f2cc806af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.771+0000 7f2cc6a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2cc806aba0 0x7f2cc806af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49194/0 (socket says 172.21.15.119:49194) 2024-04-25T10:07:58.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.771+0000 7f2cc6a00640 1 -- 172.21.15.119:0/3035951778 learned_addr learned my addr 172.21.15.119:0/3035951778 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:58.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.771+0000 7f2cc7400640 1 --2- 172.21.15.119:0/3035951778 >> v2:172.21.15.132:3300/0 conn(0x7f2cc80cec60 0x7f2cc815e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.771+0000 7f2cc6000640 1 --2- 172.21.15.119:0/3035951778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cc806f040 0x7f2cc815e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.771+0000 7f2cc6a00640 1 -- 172.21.15.119:0/3035951778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cc806f040 msgr2=0x7f2cc815e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.771+0000 7f2cc6a00640 1 --2- 172.21.15.119:0/3035951778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cc806f040 0x7f2cc815e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.771+0000 7f2cc6a00640 1 -- 172.21.15.119:0/3035951778 >> v2:172.21.15.132:3300/0 conn(0x7f2cc80cec60 msgr2=0x7f2cc815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.771+0000 7f2cc6a00640 1 --2- 172.21.15.119:0/3035951778 >> v2:172.21.15.132:3300/0 conn(0x7f2cc80cec60 0x7f2cc815e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.771+0000 7f2cc6a00640 1 -- 172.21.15.119:0/3035951778 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc806d640 con 0x7f2cc806aba0 2024-04-25T10:07:58.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.772+0000 7f2cc6a00640 1 --2- 172.21.15.119:0/3035951778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc806aba0 0x7f2cc806af80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2cc806ede0 tx=0x7f2cb802c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f9832a86426ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.773+0000 7f2cc5600640 1 -- 172.21.15.119:0/3035951778 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8038070 con 0x7f2cc806aba0 2024-04-25T10:07:58.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.773+0000 7f2cc5600640 1 -- 172.21.15.119:0/3035951778 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb8031710 con 0x7f2cc806aba0 2024-04-25T10:07:58.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.773+0000 7f2cc5600640 1 -- 172.21.15.119:0/3035951778 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8031ae0 con 0x7f2cc806aba0 2024-04-25T10:07:58.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.773+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3035951778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc806aba0 msgr2=0x7f2cc806af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.773+0000 7f2cc7e00640 1 --2- 172.21.15.119:0/3035951778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc806aba0 0x7f2cc806af80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2cc806ede0 tx=0x7f2cb802c3f0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.773+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3035951778 shutdown_connections 2024-04-25T10:07:58.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.773+0000 7f2cc7e00640 1 --2- 172.21.15.119:0/3035951778 >> v2:172.21.15.132:3300/0 conn(0x7f2cc80cec60 0x7f2cc815e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.773+0000 7f2cc7e00640 1 --2- 172.21.15.119:0/3035951778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cc806f040 0x7f2cc815e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.773+0000 7f2cc7e00640 1 --2- 172.21.15.119:0/3035951778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc806aba0 0x7f2cc806af80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.773+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3035951778 >> 172.21.15.119:0/3035951778 conn(0x7f2cc8066ff0 msgr2=0x7f2cc80b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.773+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3035951778 shutdown_connections 2024-04-25T10:07:58.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.774+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3035951778 wait complete. 2024-04-25T10:07:58.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.775+0000 7f2cc7e00640 1 Processor -- start 2024-04-25T10:07:58.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.775+0000 7f2cc7e00640 1 -- start start 2024-04-25T10:07:58.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.776+0000 7f2cc7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cc806aba0 0x7f2cc81f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.776+0000 7f2cc7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc806f040 0x7f2cc80adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.776+0000 7f2cc7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc80cec60 0x7f2cc815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.776+0000 7f2cc7e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc815a980 con 0x7f2cc80cec60 2024-04-25T10:07:58.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.776+0000 7f2cc7e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc806dc00 con 0x7f2cc806f040 2024-04-25T10:07:58.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.776+0000 7f2cc7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc80aed40 con 0x7f2cc806aba0 2024-04-25T10:07:58.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.777+0000 7f2cc7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc80cec60 0x7f2cc815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.777+0000 7f2cc7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc80cec60 0x7f2cc815d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49208/0 (socket says 172.21.15.119:49208) 2024-04-25T10:07:58.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.777+0000 7f2cc7400640 1 -- 172.21.15.119:0/3910962123 learned_addr learned my addr 172.21.15.119:0/3910962123 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:58.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.777+0000 7f2cc6000640 1 --2- 172.21.15.119:0/3910962123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc806f040 0x7f2cc80adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.777+0000 7f2cc6a00640 1 --2- 172.21.15.119:0/3910962123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cc806aba0 0x7f2cc81f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.777+0000 7f2cc7400640 1 -- 172.21.15.119:0/3910962123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cc806aba0 msgr2=0x7f2cc81f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.777+0000 7f2cc7400640 1 --2- 172.21.15.119:0/3910962123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cc806aba0 0x7f2cc81f3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.777+0000 7f2cc7400640 1 -- 172.21.15.119:0/3910962123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc806f040 msgr2=0x7f2cc80adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.777+0000 7f2cc7400640 1 --2- 172.21.15.119:0/3910962123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc806f040 0x7f2cc80adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.777+0000 7f2cc7400640 1 -- 172.21.15.119:0/3910962123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc81f3e80 con 0x7f2cc80cec60 2024-04-25T10:07:58.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.777+0000 7f2cc7400640 1 --2- 172.21.15.119:0/3910962123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc80cec60 0x7f2cc815d3c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0007e40 tx=0x7f2cc000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.778+0000 7f2cbea00640 1 -- 172.21.15.119:0/3910962123 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0014620 con 0x7f2cc80cec60 2024-04-25T10:07:58.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.778+0000 7f2cbea00640 1 -- 172.21.15.119:0/3910962123 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc0019070 con 0x7f2cc80cec60 2024-04-25T10:07:58.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.778+0000 7f2cbea00640 1 -- 172.21.15.119:0/3910962123 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0014a20 con 0x7f2cc80cec60 2024-04-25T10:07:58.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.778+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3910962123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc80ae320 con 0x7f2cc80cec60 2024-04-25T10:07:58.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.778+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3910962123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc80ad300 con 0x7f2cc80cec60 2024-04-25T10:07:58.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.779+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3910962123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc80749b0 con 0x7f2cc80cec60 2024-04-25T10:07:58.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.780+0000 7f2cbea00640 1 -- 172.21.15.119:0/3910962123 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2cc0014bc0 con 0x7f2cc80cec60 2024-04-25T10:07:58.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.781+0000 7f2cbea00640 1 --2- 172.21.15.119:0/3910962123 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ca806b4d0 0x7f2ca806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.781+0000 7f2cbea00640 1 -- 172.21.15.119:0/3910962123 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f2cc008ac40 con 0x7f2cc80cec60 2024-04-25T10:07:58.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.781+0000 7f2cc6a00640 1 --2- 172.21.15.119:0/3910962123 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ca806b4d0 0x7f2ca806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.782+0000 7f2cc6a00640 1 --2- 172.21.15.119:0/3910962123 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ca806b4d0 0x7f2ca806d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8035000 tx=0x7f2cb8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.786+0000 7f2cbea00640 1 -- 172.21.15.119:0/3910962123 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2cc81f3e80 con 0x7f2cc80cec60 2024-04-25T10:07:58.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.980+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3910962123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cc80cf1b0 con 0x7f2cc80cec60 2024-04-25T10:07:58.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.981+0000 7f2cbea00640 1 -- 172.21.15.119:0/3910962123 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cc00599c0 con 0x7f2cc80cec60 2024-04-25T10:07:58.987 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:07:58.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.983+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3910962123 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ca806b4d0 msgr2=0x7f2ca806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.983+0000 7f2cc7e00640 1 --2- 172.21.15.119:0/3910962123 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ca806b4d0 0x7f2ca806d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8035000 tx=0x7f2cb8008030 comp rx=0 tx=0).stop 2024-04-25T10:07:58.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.983+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3910962123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc80cec60 msgr2=0x7f2cc815d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.983+0000 7f2cc7e00640 1 --2- 172.21.15.119:0/3910962123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc80cec60 0x7f2cc815d3c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0007e40 tx=0x7f2cc000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.983+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3910962123 shutdown_connections 2024-04-25T10:07:58.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.983+0000 7f2cc7e00640 1 --2- 172.21.15.119:0/3910962123 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ca806b4d0 0x7f2ca806d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.983+0000 7f2cc7e00640 1 --2- 172.21.15.119:0/3910962123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc80cec60 0x7f2cc815d3c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.983+0000 7f2cc7e00640 1 --2- 172.21.15.119:0/3910962123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cc806f040 0x7f2cc80adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.983+0000 7f2cc7e00640 1 --2- 172.21.15.119:0/3910962123 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2cc806aba0 0x7f2cc81f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.983+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3910962123 >> 172.21.15.119:0/3910962123 conn(0x7f2cc8066ff0 msgr2=0x7f2cc80b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.983+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3910962123 shutdown_connections 2024-04-25T10:07:58.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:58.984+0000 7f2cc7e00640 1 -- 172.21.15.119:0/3910962123 wait complete. 2024-04-25T10:07:59.008 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.195+0000 7f04a3800640 1 Processor -- start 2024-04-25T10:07:59.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.195+0000 7f04a3800640 1 -- start start 2024-04-25T10:07:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.197+0000 7f04a3800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f049c144e70 0x7f049c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.197+0000 7f04a3800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f049c06dcb0 con 0x7f049c06d7a0 2024-04-25T10:07:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.197+0000 7f04a3800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f049c06df90 con 0x7f049c0cf0e0 2024-04-25T10:07:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.197+0000 7f04a3800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f049c15bfe0 con 0x7f049c144e70 2024-04-25T10:07:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.197+0000 7f04a0e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f049c144e70 0x7f049c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.197+0000 7f04a0e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f049c144e70 0x7f049c06d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49214/0 (socket says 172.21.15.119:49214) 2024-04-25T10:07:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.197+0000 7f04a0e00640 1 -- 172.21.15.119:0/1197400730 learned_addr learned my addr 172.21.15.119:0/1197400730 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:59.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.197+0000 7f049be00640 1 --1- 172.21.15.119:0/1197400730 >> v1:172.21.15.119:6789/0 conn(0x7f049c06d7a0 0x7f049c0ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34888/0 (socket says 172.21.15.119:34888) 2024-04-25T10:07:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.198+0000 7f04a0e00640 1 -- 172.21.15.119:0/1197400730 >> v1:172.21.15.132:6789/0 conn(0x7f049c0cf0e0 legacy=0x7f049c0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.198+0000 7f04a0e00640 1 -- 172.21.15.119:0/1197400730 >> v1:172.21.15.119:6789/0 conn(0x7f049c06d7a0 legacy=0x7f049c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.198+0000 7f04a0e00640 1 -- 172.21.15.119:0/1197400730 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049c06e270 con 0x7f049c144e70 2024-04-25T10:07:59.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.199+0000 7f04a0e00640 1 --2- 172.21.15.119:0/1197400730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c144e70 0x7f049c06d260 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f049c15c160 tx=0x7f049002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b613fdecdcba7577 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.199+0000 7f049b400640 1 -- 172.21.15.119:0/1197400730 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0490037070 con 0x7f049c144e70 2024-04-25T10:07:59.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.199+0000 7f049b400640 1 -- 172.21.15.119:0/1197400730 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0490032440 con 0x7f049c144e70 2024-04-25T10:07:59.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.199+0000 7f049b400640 1 -- 172.21.15.119:0/1197400730 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0490032810 con 0x7f049c144e70 2024-04-25T10:07:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.200+0000 7f04a3800640 1 -- 172.21.15.119:0/1197400730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c144e70 msgr2=0x7f049c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.200+0000 7f04a3800640 1 --2- 172.21.15.119:0/1197400730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c144e70 0x7f049c06d260 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f049c15c160 tx=0x7f049002a210 comp rx=0 tx=0).stop 2024-04-25T10:07:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.200+0000 7f04a3800640 1 -- 172.21.15.119:0/1197400730 shutdown_connections 2024-04-25T10:07:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.200+0000 7f04a3800640 1 --2- 172.21.15.119:0/1197400730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c144e70 0x7f049c06d260 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.200+0000 7f04a3800640 1 -- 172.21.15.119:0/1197400730 >> 172.21.15.119:0/1197400730 conn(0x7f049c06aba0 msgr2=0x7f049c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.200+0000 7f04a3800640 1 -- 172.21.15.119:0/1197400730 shutdown_connections 2024-04-25T10:07:59.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.200+0000 7f04a3800640 1 -- 172.21.15.119:0/1197400730 wait complete. 2024-04-25T10:07:59.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.202+0000 7f04a3800640 1 Processor -- start 2024-04-25T10:07:59.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.202+0000 7f04a3800640 1 -- start start 2024-04-25T10:07:59.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.203+0000 7f04a3800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f049c06d7a0 0x7f049c132ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.203+0000 7f04a3800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f049c0cf0e0 0x7f049c0b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.203+0000 7f04a3800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c144e70 0x7f049c158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.203+0000 7f04a3800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049c145630 con 0x7f049c144e70 2024-04-25T10:07:59.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.203+0000 7f04a3800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049c1454b0 con 0x7f049c0cf0e0 2024-04-25T10:07:59.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.203+0000 7f04a3800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049c0cca10 con 0x7f049c06d7a0 2024-04-25T10:07:59.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.204+0000 7f04a1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c144e70 0x7f049c158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.204+0000 7f04a1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c144e70 0x7f049c158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49222/0 (socket says 172.21.15.119:49222) 2024-04-25T10:07:59.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.204+0000 7f04a1800640 1 -- 172.21.15.119:0/4030842593 learned_addr learned my addr 172.21.15.119:0/4030842593 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:59.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.204+0000 7f049be00640 1 --2- 172.21.15.119:0/4030842593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f049c0cf0e0 0x7f049c0b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.204+0000 7f04a0e00640 1 --2- 172.21.15.119:0/4030842593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f049c06d7a0 0x7f049c132ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.204+0000 7f04a1800640 1 -- 172.21.15.119:0/4030842593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f049c06d7a0 msgr2=0x7f049c132ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.204+0000 7f04a1800640 1 --2- 172.21.15.119:0/4030842593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f049c06d7a0 0x7f049c132ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.204+0000 7f04a1800640 1 -- 172.21.15.119:0/4030842593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f049c0cf0e0 msgr2=0x7f049c0b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.204+0000 7f04a1800640 1 --2- 172.21.15.119:0/4030842593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f049c0cf0e0 0x7f049c0b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.204+0000 7f04a1800640 1 -- 172.21.15.119:0/4030842593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049c0af6a0 con 0x7f049c144e70 2024-04-25T10:07:59.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.205+0000 7f04a1800640 1 --2- 172.21.15.119:0/4030842593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c144e70 0x7f049c158d30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f049400c780 tx=0x7f049400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.205+0000 7f0498c00640 1 -- 172.21.15.119:0/4030842593 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494004b60 con 0x7f049c144e70 2024-04-25T10:07:59.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.205+0000 7f0498c00640 1 -- 172.21.15.119:0/4030842593 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0494004d00 con 0x7f049c144e70 2024-04-25T10:07:59.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.205+0000 7f0498c00640 1 -- 172.21.15.119:0/4030842593 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049400aa90 con 0x7f049c144e70 2024-04-25T10:07:59.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.205+0000 7f04a3800640 1 -- 172.21.15.119:0/4030842593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049c0af980 con 0x7f049c144e70 2024-04-25T10:07:59.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.205+0000 7f04a3800640 1 -- 172.21.15.119:0/4030842593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049c0b0220 con 0x7f049c144e70 2024-04-25T10:07:59.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.206+0000 7f04a3800640 1 -- 172.21.15.119:0/4030842593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049c0750b0 con 0x7f049c144e70 2024-04-25T10:07:59.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.207+0000 7f0498c00640 1 -- 172.21.15.119:0/4030842593 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f049400ac30 con 0x7f049c144e70 2024-04-25T10:07:59.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.208+0000 7f0498c00640 1 --2- 172.21.15.119:0/4030842593 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f047c06b4d0 0x7f047c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.208+0000 7f0498c00640 1 -- 172.21.15.119:0/4030842593 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f049408a5e0 con 0x7f049c144e70 2024-04-25T10:07:59.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.209+0000 7f04a0e00640 1 --2- 172.21.15.119:0/4030842593 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f047c06b4d0 0x7f047c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.209+0000 7f04a0e00640 1 --2- 172.21.15.119:0/4030842593 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f047c06b4d0 0x7f047c06d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0490034000 tx=0x7f049002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.213+0000 7f0498c00640 1 -- 172.21.15.119:0/4030842593 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0494020070 con 0x7f049c144e70 2024-04-25T10:07:59.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.406+0000 7f04a3800640 1 -- 172.21.15.119:0/4030842593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f049c0b03f0 con 0x7f049c144e70 2024-04-25T10:07:59.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.406+0000 7f0498c00640 1 -- 172.21.15.119:0/4030842593 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0494059360 con 0x7f049c144e70 2024-04-25T10:07:59.412 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:07:59.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.409+0000 7f04a3800640 1 -- 172.21.15.119:0/4030842593 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f047c06b4d0 msgr2=0x7f047c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.409+0000 7f04a3800640 1 --2- 172.21.15.119:0/4030842593 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f047c06b4d0 0x7f047c06d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0490034000 tx=0x7f049002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:59.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.409+0000 7f04a3800640 1 -- 172.21.15.119:0/4030842593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c144e70 msgr2=0x7f049c158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.409+0000 7f04a3800640 1 --2- 172.21.15.119:0/4030842593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c144e70 0x7f049c158d30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f049400c780 tx=0x7f049400cc40 comp rx=0 tx=0).stop 2024-04-25T10:07:59.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.409+0000 7f04a3800640 1 -- 172.21.15.119:0/4030842593 shutdown_connections 2024-04-25T10:07:59.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.409+0000 7f04a3800640 1 --2- 172.21.15.119:0/4030842593 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f047c06b4d0 0x7f047c06d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.409+0000 7f04a3800640 1 --2- 172.21.15.119:0/4030842593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f049c144e70 0x7f049c158d30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.409+0000 7f04a3800640 1 --2- 172.21.15.119:0/4030842593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f049c0cf0e0 0x7f049c0b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.409+0000 7f04a3800640 1 --2- 172.21.15.119:0/4030842593 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f049c06d7a0 0x7f049c132ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.409+0000 7f04a3800640 1 -- 172.21.15.119:0/4030842593 >> 172.21.15.119:0/4030842593 conn(0x7f049c06aba0 msgr2=0x7f049c0b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.409+0000 7f04a3800640 1 -- 172.21.15.119:0/4030842593 shutdown_connections 2024-04-25T10:07:59.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.410+0000 7f04a3800640 1 -- 172.21.15.119:0/4030842593 wait complete. 2024-04-25T10:07:59.434 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:59.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.633+0000 7f7b8f200640 1 Processor -- start 2024-04-25T10:07:59.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.633+0000 7f7b8f200640 1 -- start start 2024-04-25T10:07:59.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.635+0000 7f7b8f200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7b900ce860 0x7f7b900cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.635+0000 7f7b8f200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b900b5710 con 0x7f7b900d00a0 2024-04-25T10:07:59.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.635+0000 7f7b8f200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b900b59f0 con 0x7f7b9015b7e0 2024-04-25T10:07:59.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.635+0000 7f7b8f200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b900b5cd0 con 0x7f7b900ce860 2024-04-25T10:07:59.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.635+0000 7f7b8e800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7b900d00a0 0x7f7b900d0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34894/0 (socket says 172.21.15.119:34894) 2024-04-25T10:07:59.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.635+0000 7f7b8e800640 1 -- 172.21.15.119:0/2460838743 learned_addr learned my addr 172.21.15.119:0/2460838743 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:07:59.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b87e00640 1 --2- 172.21.15.119:0/2460838743 >> v2:172.21.15.132:3300/0 conn(0x7f7b900ce860 0x7f7b900cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 -- 172.21.15.119:0/2460838743 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486661724 0 0) 0x7f7b900b5710 con 0x7f7b900d00a0 2024-04-25T10:07:59.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 -- 172.21.15.119:0/2460838743 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b74003520 con 0x7f7b900d00a0 2024-04-25T10:07:59.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 -- 172.21.15.119:0/2460838743 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1941551442 0 0) 0x7f7b74003520 con 0x7f7b900d00a0 2024-04-25T10:07:59.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 -- 172.21.15.119:0/2460838743 >> v2:172.21.15.132:3300/0 conn(0x7f7b900ce860 msgr2=0x7f7b900cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 --2- 172.21.15.119:0/2460838743 >> v2:172.21.15.132:3300/0 conn(0x7f7b900ce860 0x7f7b900cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 -- 172.21.15.119:0/2460838743 >> v1:172.21.15.132:6789/0 conn(0x7f7b9015b7e0 legacy=0x7f7b9015e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 -- 172.21.15.119:0/2460838743 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900b5fb0 con 0x7f7b900d00a0 2024-04-25T10:07:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 -- 172.21.15.119:0/2460838743 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f7b880019f0 con 0x7f7b900d00a0 2024-04-25T10:07:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 -- 172.21.15.119:0/2460838743 >> v1:172.21.15.119:6789/0 conn(0x7f7b900d00a0 legacy=0x7f7b900d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 --2- 172.21.15.119:0/2460838743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b740039b0 0x7f7b74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 --2- 172.21.15.119:0/2460838743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b74004800 0x7f7b74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 --2- 172.21.15.119:0/2460838743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b740053c0 0x7f7b7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 -- 172.21.15.119:0/2460838743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900b5cd0 con 0x7f7b740039b0 2024-04-25T10:07:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 -- 172.21.15.119:0/2460838743 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7400c250 con 0x7f7b74004800 2024-04-25T10:07:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.636+0000 7f7b8d400640 1 -- 172.21.15.119:0/2460838743 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7400c450 con 0x7f7b740053c0 2024-04-25T10:07:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.637+0000 7f7b8de00640 1 --2- 172.21.15.119:0/2460838743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b740039b0 0x7f7b74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.637+0000 7f7b87e00640 1 --2- 172.21.15.119:0/2460838743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b74004800 0x7f7b74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.637+0000 7f7b8e800640 1 --2- 172.21.15.119:0/2460838743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b740053c0 0x7f7b7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.637+0000 7f7b8de00640 1 -- 172.21.15.119:0/2460838743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b740053c0 msgr2=0x7f7b7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.637+0000 7f7b8de00640 1 --2- 172.21.15.119:0/2460838743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b740053c0 0x7f7b7400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.637+0000 7f7b8de00640 1 -- 172.21.15.119:0/2460838743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b74004800 msgr2=0x7f7b74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.637+0000 7f7b8de00640 1 --2- 172.21.15.119:0/2460838743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b74004800 0x7f7b74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.637+0000 7f7b8de00640 1 -- 172.21.15.119:0/2460838743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b74003520 con 0x7f7b740039b0 2024-04-25T10:07:59.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.637+0000 7f7b8e800640 1 -- 172.21.15.119:0/2460838743 reap_dead start 2024-04-25T10:07:59.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.638+0000 7f7b8de00640 1 --2- 172.21.15.119:0/2460838743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b740039b0 0x7f7b74003dd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7b80001750 tx=0x7f7b74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efb02fe471870ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.638+0000 7f7b8d400640 1 -- 172.21.15.119:0/2460838743 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80001370 con 0x7f7b740039b0 2024-04-25T10:07:59.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.638+0000 7f7b8d400640 1 -- 172.21.15.119:0/2460838743 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b800052d0 con 0x7f7b740039b0 2024-04-25T10:07:59.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.638+0000 7f7b8d400640 1 -- 172.21.15.119:0/2460838743 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8002abe0 con 0x7f7b740039b0 2024-04-25T10:07:59.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.639+0000 7f7b8f200640 1 -- 172.21.15.119:0/2460838743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b740039b0 msgr2=0x7f7b74003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.639+0000 7f7b8f200640 1 --2- 172.21.15.119:0/2460838743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b740039b0 0x7f7b74003dd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7b80001750 tx=0x7f7b74005830 comp rx=0 tx=0).stop 2024-04-25T10:07:59.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.639+0000 7f7b8f200640 1 -- 172.21.15.119:0/2460838743 shutdown_connections 2024-04-25T10:07:59.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.639+0000 7f7b8f200640 1 --2- 172.21.15.119:0/2460838743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b740039b0 0x7f7b74003dd0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.639+0000 7f7b8f200640 1 -- 172.21.15.119:0/2460838743 >> 172.21.15.119:0/2460838743 conn(0x7f7b900722d0 msgr2=0x7f7b900726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.639+0000 7f7b8f200640 1 -- 172.21.15.119:0/2460838743 shutdown_connections 2024-04-25T10:07:59.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.639+0000 7f7b8f200640 1 -- 172.21.15.119:0/2460838743 wait complete. 2024-04-25T10:07:59.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.641+0000 7f7b8f200640 1 Processor -- start 2024-04-25T10:07:59.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.641+0000 7f7b8f200640 1 -- start start 2024-04-25T10:07:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.642+0000 7f7b8f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b740039b0 0x7f7b90159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.642+0000 7f7b8f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b900c86c0 0x7f7b900c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.642+0000 7f7b8f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b900b7b10 0x7f7b900b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.642+0000 7f7b8f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900cee70 con 0x7f7b740039b0 2024-04-25T10:07:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.642+0000 7f7b8f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900cecf0 con 0x7f7b900c86c0 2024-04-25T10:07:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.642+0000 7f7b8f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900ceff0 con 0x7f7b900b7b10 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.642+0000 7f7b8de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b740039b0 0x7f7b90159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.642+0000 7f7b8de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b740039b0 0x7f7b90159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49240/0 (socket says 172.21.15.119:49240) 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.642+0000 7f7b8de00640 1 -- 172.21.15.119:0/1934584579 learned_addr learned my addr 172.21.15.119:0/1934584579 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.642+0000 7f7b87e00640 1 --2- 172.21.15.119:0/1934584579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b900c86c0 0x7f7b900c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.642+0000 7f7b8e800640 1 --2- 172.21.15.119:0/1934584579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b900b7b10 0x7f7b900b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.643+0000 7f7b8de00640 1 -- 172.21.15.119:0/1934584579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b900b7b10 msgr2=0x7f7b900b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.643+0000 7f7b8de00640 1 --2- 172.21.15.119:0/1934584579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b900b7b10 0x7f7b900b7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.643+0000 7f7b8de00640 1 -- 172.21.15.119:0/1934584579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b900c86c0 msgr2=0x7f7b900c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.643+0000 7f7b8de00640 1 --2- 172.21.15.119:0/1934584579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b900c86c0 0x7f7b900c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.643+0000 7f7b8de00640 1 -- 172.21.15.119:0/1934584579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900c9160 con 0x7f7b740039b0 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.643+0000 7f7b8de00640 1 --2- 172.21.15.119:0/1934584579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b740039b0 0x7f7b90159360 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7b80005040 tx=0x7f7b80001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.643+0000 7f7b86000640 1 -- 172.21.15.119:0/1934584579 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8003f020 con 0x7f7b740039b0 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.643+0000 7f7b86000640 1 -- 172.21.15.119:0/1934584579 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b80005480 con 0x7f7b740039b0 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.643+0000 7f7b86000640 1 -- 172.21.15.119:0/1934584579 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80007a30 con 0x7f7b740039b0 2024-04-25T10:07:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.643+0000 7f7b8f200640 1 -- 172.21.15.119:0/1934584579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b900c7510 con 0x7f7b740039b0 2024-04-25T10:07:59.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.643+0000 7f7b8f200640 1 -- 172.21.15.119:0/1934584579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b900c7ad0 con 0x7f7b740039b0 2024-04-25T10:07:59.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.645+0000 7f7b8f200640 1 -- 172.21.15.119:0/1934584579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b900749b0 con 0x7f7b740039b0 2024-04-25T10:07:59.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.646+0000 7f7b86000640 1 -- 172.21.15.119:0/1934584579 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b80042020 con 0x7f7b740039b0 2024-04-25T10:07:59.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.646+0000 7f7b86000640 1 --2- 172.21.15.119:0/1934584579 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7b6c06b500 0x7f7b6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.647+0000 7f7b86000640 1 -- 172.21.15.119:0/1934584579 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f7b800a9e40 con 0x7f7b740039b0 2024-04-25T10:07:59.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.647+0000 7f7b87e00640 1 --2- 172.21.15.119:0/1934584579 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7b6c06b500 0x7f7b6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.647+0000 7f7b87e00640 1 --2- 172.21.15.119:0/1934584579 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7b6c06b500 0x7f7b6c06d9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7b900cc260 tx=0x7f7b78008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.651+0000 7f7b86000640 1 -- 172.21.15.119:0/1934584579 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b80049030 con 0x7f7b740039b0 2024-04-25T10:07:59.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.828+0000 7f7b8f200640 1 -- 172.21.15.119:0/1934584579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b900d1650 con 0x7f7b740039b0 2024-04-25T10:07:59.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.829+0000 7f7b86000640 1 -- 172.21.15.119:0/1934584579 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b80078bc0 con 0x7f7b740039b0 2024-04-25T10:07:59.835 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:07:59.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.831+0000 7f7b8f200640 1 -- 172.21.15.119:0/1934584579 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7b6c06b500 msgr2=0x7f7b6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.831+0000 7f7b8f200640 1 --2- 172.21.15.119:0/1934584579 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7b6c06b500 0x7f7b6c06d9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7b900cc260 tx=0x7f7b78008040 comp rx=0 tx=0).stop 2024-04-25T10:07:59.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.832+0000 7f7b8f200640 1 -- 172.21.15.119:0/1934584579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b740039b0 msgr2=0x7f7b90159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.832+0000 7f7b8f200640 1 --2- 172.21.15.119:0/1934584579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b740039b0 0x7f7b90159360 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7b80005040 tx=0x7f7b80001120 comp rx=0 tx=0).stop 2024-04-25T10:07:59.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.832+0000 7f7b8f200640 1 -- 172.21.15.119:0/1934584579 shutdown_connections 2024-04-25T10:07:59.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.832+0000 7f7b8f200640 1 --2- 172.21.15.119:0/1934584579 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7b6c06b500 0x7f7b6c06d9c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.832+0000 7f7b8f200640 1 --2- 172.21.15.119:0/1934584579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b900b7b10 0x7f7b900b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.832+0000 7f7b8f200640 1 --2- 172.21.15.119:0/1934584579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b900c86c0 0x7f7b900c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.832+0000 7f7b8f200640 1 --2- 172.21.15.119:0/1934584579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b740039b0 0x7f7b90159360 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.832+0000 7f7b8f200640 1 -- 172.21.15.119:0/1934584579 >> 172.21.15.119:0/1934584579 conn(0x7f7b900722d0 msgr2=0x7f7b900d1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.832+0000 7f7b8f200640 1 -- 172.21.15.119:0/1934584579 shutdown_connections 2024-04-25T10:07:59.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:07:59.832+0000 7f7b8f200640 1 -- 172.21.15.119:0/1934584579 wait complete. 2024-04-25T10:07:59.856 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:00.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.042+0000 7f7db3200640 1 Processor -- start 2024-04-25T10:08:00.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.042+0000 7f7db3200640 1 -- start start 2024-04-25T10:08:00.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.043+0000 7f7db3200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7db40b5470 0x7f7db4159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.044+0000 7f7db3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7db40b1a30 0x7f7db415d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.044+0000 7f7db3200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7db40b1e90 0x7f7db415de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.044+0000 7f7db3200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db40cec30 con 0x7f7db40b1e90 2024-04-25T10:08:00.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.044+0000 7f7db3200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db40cef10 con 0x7f7db40b5470 2024-04-25T10:08:00.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.044+0000 7f7db3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db40cf1f0 con 0x7f7db40b1a30 2024-04-25T10:08:00.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.044+0000 7f7db2800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7db40b1e90 0x7f7db415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.044+0000 7f7db2800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7db40b1e90 0x7f7db415de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49244/0 (socket says 172.21.15.119:49244) 2024-04-25T10:08:00.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.044+0000 7f7db2800640 1 -- 172.21.15.119:0/3579299028 learned_addr learned my addr 172.21.15.119:0/3579299028 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:00.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.044+0000 7f7db1e00640 1 --2- 172.21.15.119:0/3579299028 >> v2:172.21.15.132:3300/0 conn(0x7f7db40b5470 0x7f7db4159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.044+0000 7f7db1400640 1 --2- 172.21.15.119:0/3579299028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7db40b1a30 0x7f7db415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.045+0000 7f7db2800640 1 -- 172.21.15.119:0/3579299028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7db40b1a30 msgr2=0x7f7db415d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.045+0000 7f7db2800640 1 --2- 172.21.15.119:0/3579299028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7db40b1a30 0x7f7db415d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.045+0000 7f7db2800640 1 -- 172.21.15.119:0/3579299028 >> v2:172.21.15.132:3300/0 conn(0x7f7db40b5470 msgr2=0x7f7db4159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.045+0000 7f7db2800640 1 --2- 172.21.15.119:0/3579299028 >> v2:172.21.15.132:3300/0 conn(0x7f7db40b5470 0x7f7db4159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.045+0000 7f7db2800640 1 -- 172.21.15.119:0/3579299028 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db406d420 con 0x7f7db40b1e90 2024-04-25T10:08:00.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.046+0000 7f7db2800640 1 --2- 172.21.15.119:0/3579299028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40b1e90 0x7f7db415de50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7db406edb0 tx=0x7f7dac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e886bc9ba979d819 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.046+0000 7f7db0a00640 1 -- 172.21.15.119:0/3579299028 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac036070 con 0x7f7db40b1e90 2024-04-25T10:08:00.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.046+0000 7f7db0a00640 1 -- 172.21.15.119:0/3579299028 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dac031600 con 0x7f7db40b1e90 2024-04-25T10:08:00.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.046+0000 7f7db0a00640 1 -- 172.21.15.119:0/3579299028 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac0319d0 con 0x7f7db40b1e90 2024-04-25T10:08:00.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.047+0000 7f7db3200640 1 -- 172.21.15.119:0/3579299028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40b1e90 msgr2=0x7f7db415de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.047+0000 7f7db3200640 1 --2- 172.21.15.119:0/3579299028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40b1e90 0x7f7db415de50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7db406edb0 tx=0x7f7dac02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.047+0000 7f7db3200640 1 -- 172.21.15.119:0/3579299028 shutdown_connections 2024-04-25T10:08:00.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.047+0000 7f7db3200640 1 --2- 172.21.15.119:0/3579299028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40b1e90 0x7f7db415de50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.047+0000 7f7db3200640 1 --2- 172.21.15.119:0/3579299028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7db40b1a30 0x7f7db415d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.047+0000 7f7db3200640 1 --2- 172.21.15.119:0/3579299028 >> v2:172.21.15.132:3300/0 conn(0x7f7db40b5470 0x7f7db4159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.047+0000 7f7db3200640 1 -- 172.21.15.119:0/3579299028 >> 172.21.15.119:0/3579299028 conn(0x7f7db406b1d0 msgr2=0x7f7db40b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.047+0000 7f7db3200640 1 -- 172.21.15.119:0/3579299028 shutdown_connections 2024-04-25T10:08:00.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.047+0000 7f7db3200640 1 -- 172.21.15.119:0/3579299028 wait complete. 2024-04-25T10:08:00.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.049+0000 7f7db3200640 1 Processor -- start 2024-04-25T10:08:00.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.049+0000 7f7db3200640 1 -- start start 2024-04-25T10:08:00.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.050+0000 7f7db3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40b1a30 0x7f7db415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.050+0000 7f7db3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7db40b1e90 0x7f7db4154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.050+0000 7f7db3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7db40b5470 0x7f7db41474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.050+0000 7f7db3200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db40ce9f0 con 0x7f7db40b1a30 2024-04-25T10:08:00.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.050+0000 7f7db3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db40ce870 con 0x7f7db40b5470 2024-04-25T10:08:00.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.050+0000 7f7db3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db415e390 con 0x7f7db40b1e90 2024-04-25T10:08:00.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.050+0000 7f7db1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40b1a30 0x7f7db415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.050+0000 7f7db1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40b1a30 0x7f7db415a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49254/0 (socket says 172.21.15.119:49254) 2024-04-25T10:08:00.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.050+0000 7f7db1e00640 1 -- 172.21.15.119:0/417952076 learned_addr learned my addr 172.21.15.119:0/417952076 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:00.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.050+0000 7f7db1e00640 1 -- 172.21.15.119:0/417952076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7db40b1e90 msgr2=0x7f7db4154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.050+0000 7f7db1e00640 1 --2- 172.21.15.119:0/417952076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7db40b1e90 0x7f7db4154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.050+0000 7f7db1e00640 1 -- 172.21.15.119:0/417952076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7db40b5470 msgr2=0x7f7db41474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.050+0000 7f7db1e00640 1 --2- 172.21.15.119:0/417952076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7db40b5470 0x7f7db41474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.050+0000 7f7db1e00640 1 -- 172.21.15.119:0/417952076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db4154c40 con 0x7f7db40b1a30 2024-04-25T10:08:00.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.051+0000 7f7db1e00640 1 --2- 172.21.15.119:0/417952076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40b1a30 0x7f7db415a3f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00d620 tx=0x7f7d9c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.051+0000 7f7daa000640 1 -- 172.21.15.119:0/417952076 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00b620 con 0x7f7db40b1a30 2024-04-25T10:08:00.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.051+0000 7f7daa000640 1 -- 172.21.15.119:0/417952076 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c019070 con 0x7f7db40b1a30 2024-04-25T10:08:00.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.051+0000 7f7daa000640 1 -- 172.21.15.119:0/417952076 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00ba00 con 0x7f7db40b1a30 2024-04-25T10:08:00.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.051+0000 7f7db3200640 1 -- 172.21.15.119:0/417952076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db4149de0 con 0x7f7db40b1a30 2024-04-25T10:08:00.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.051+0000 7f7db3200640 1 -- 172.21.15.119:0/417952076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db414a3a0 con 0x7f7db40b1a30 2024-04-25T10:08:00.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.052+0000 7f7db3200640 1 -- 172.21.15.119:0/417952076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db40749b0 con 0x7f7db40b1a30 2024-04-25T10:08:00.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.053+0000 7f7daa000640 1 -- 172.21.15.119:0/417952076 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d9c00bba0 con 0x7f7db40b1a30 2024-04-25T10:08:00.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.054+0000 7f7daa000640 1 --2- 172.21.15.119:0/417952076 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d9006b4d0 0x7f7d9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.054+0000 7f7daa000640 1 -- 172.21.15.119:0/417952076 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f7d9c08a5f0 con 0x7f7db40b1a30 2024-04-25T10:08:00.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.054+0000 7f7db1400640 1 --2- 172.21.15.119:0/417952076 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d9006b4d0 0x7f7d9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.055+0000 7f7db1400640 1 --2- 172.21.15.119:0/417952076 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d9006b4d0 0x7f7d9006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7db4149900 tx=0x7f7da4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.058+0000 7f7daa000640 1 -- 172.21.15.119:0/417952076 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d9c059400 con 0x7f7db40b1a30 2024-04-25T10:08:00.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:00.204 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:00.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.294+0000 7f7db3200640 1 -- 172.21.15.119:0/417952076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7db415db50 con 0x7f7db40b1a30 2024-04-25T10:08:00.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.294+0000 7f7daa000640 1 -- 172.21.15.119:0/417952076 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4941 (secure 0 0 0) 0x7f7d9c058d00 con 0x7f7db40b1a30 2024-04-25T10:08:00.300 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:00.301 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1385207078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1385207078},{"type":"v1","addr":"172.21.15.132:6834","nonce":1385207078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1580923823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1580923823},{"type":"v1","addr":"172.21.15.119:6836","nonce":1580923823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4260,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/954053568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":954053568},{"type":"v1","addr":"172.21.15.132:6835","nonce":954053568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:06.743184+0000","modified":"2024-04-25T10:07:10.118630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1632024048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1632024048},{"type":"v1","addr":"172.21.15.119:6837","nonce":1632024048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:08:00.301 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2024-04-25T10:08:00.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.297+0000 7f7db3200640 1 -- 172.21.15.119:0/417952076 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d9006b4d0 msgr2=0x7f7d9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.297+0000 7f7db3200640 1 --2- 172.21.15.119:0/417952076 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d9006b4d0 0x7f7d9006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7db4149900 tx=0x7f7da4008040 comp rx=0 tx=0).stop 2024-04-25T10:08:00.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.297+0000 7f7db3200640 1 -- 172.21.15.119:0/417952076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40b1a30 msgr2=0x7f7db415a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.297+0000 7f7db3200640 1 --2- 172.21.15.119:0/417952076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40b1a30 0x7f7db415a3f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00d620 tx=0x7f7d9c00dae0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.297+0000 7f7db3200640 1 -- 172.21.15.119:0/417952076 shutdown_connections 2024-04-25T10:08:00.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.297+0000 7f7db3200640 1 --2- 172.21.15.119:0/417952076 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d9006b4d0 0x7f7d9006d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.297+0000 7f7db3200640 1 --2- 172.21.15.119:0/417952076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7db40b5470 0x7f7db41474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.297+0000 7f7db3200640 1 --2- 172.21.15.119:0/417952076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7db40b1e90 0x7f7db4154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.297+0000 7f7db3200640 1 --2- 172.21.15.119:0/417952076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db40b1a30 0x7f7db415a3f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.297+0000 7f7db3200640 1 -- 172.21.15.119:0/417952076 >> 172.21.15.119:0/417952076 conn(0x7f7db406b1d0 msgr2=0x7f7db41572f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.297+0000 7f7db3200640 1 -- 172.21.15.119:0/417952076 shutdown_connections 2024-04-25T10:08:00.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.297+0000 7f7db3200640 1 -- 172.21.15.119:0/417952076 wait complete. 2024-04-25T10:08:00.321 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:00.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.550+0000 7fa90f200640 1 Processor -- start 2024-04-25T10:08:00.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.550+0000 7fa90f200640 1 -- start start 2024-04-25T10:08:00.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.552+0000 7fa90f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9080d3f80 0x7fa9080d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.552+0000 7fa90f200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa9080b9270 0x7fa9080d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.552+0000 7fa90f200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90815ef10 con 0x7fa90815db60 2024-04-25T10:08:00.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.552+0000 7fa90f200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa90815e390 con 0x7fa9080b9270 2024-04-25T10:08:00.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.552+0000 7fa90f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90815e670 con 0x7fa9080d3f80 2024-04-25T10:08:00.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.552+0000 7fa90e800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa9080b9270 0x7fa9080d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.552+0000 7fa90e800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa9080b9270 0x7fa9080d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49258/0 (socket says 172.21.15.119:49258) 2024-04-25T10:08:00.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.552+0000 7fa90e800640 1 -- 172.21.15.119:0/1769715569 learned_addr learned my addr 172.21.15.119:0/1769715569 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:00.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.553+0000 7fa90d400640 1 --2- 172.21.15.119:0/1769715569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9080d3f80 0x7fa9080d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.553+0000 7fa90e800640 1 -- 172.21.15.119:0/1769715569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9080d3f80 msgr2=0x7fa9080d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.553+0000 7fa90e800640 1 --2- 172.21.15.119:0/1769715569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9080d3f80 0x7fa9080d1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.553+0000 7fa90e800640 1 -- 172.21.15.119:0/1769715569 >> v1:172.21.15.132:6789/0 conn(0x7fa90815db60 legacy=0x7fa9080d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.553+0000 7fa90e800640 1 -- 172.21.15.119:0/1769715569 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90815e950 con 0x7fa9080b9270 2024-04-25T10:08:00.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.553+0000 7fa90d400640 1 --2- 172.21.15.119:0/1769715569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9080d3f80 0x7fa9080d1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:00.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.554+0000 7fa90e800640 1 --2- 172.21.15.119:0/1769715569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9080b9270 0x7fa9080d24d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa904009e50 tx=0x7fa90402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8665f0e5072cf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.554+0000 7fa90ca00640 1 -- 172.21.15.119:0/1769715569 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904036070 con 0x7fa9080b9270 2024-04-25T10:08:00.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.554+0000 7fa90ca00640 1 -- 172.21.15.119:0/1769715569 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa904031600 con 0x7fa9080b9270 2024-04-25T10:08:00.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.554+0000 7fa90ca00640 1 -- 172.21.15.119:0/1769715569 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9040319d0 con 0x7fa9080b9270 2024-04-25T10:08:00.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.555+0000 7fa90f200640 1 -- 172.21.15.119:0/1769715569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9080b9270 msgr2=0x7fa9080d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.555+0000 7fa90f200640 1 --2- 172.21.15.119:0/1769715569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9080b9270 0x7fa9080d24d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa904009e50 tx=0x7fa90402a710 comp rx=0 tx=0).stop 2024-04-25T10:08:00.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.555+0000 7fa90f200640 1 -- 172.21.15.119:0/1769715569 shutdown_connections 2024-04-25T10:08:00.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.555+0000 7fa90f200640 1 --2- 172.21.15.119:0/1769715569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9080b9270 0x7fa9080d24d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.555+0000 7fa90f200640 1 --2- 172.21.15.119:0/1769715569 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9080d3f80 0x7fa9080d1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.555+0000 7fa90f200640 1 -- 172.21.15.119:0/1769715569 >> 172.21.15.119:0/1769715569 conn(0x7fa9080726c0 msgr2=0x7fa908072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.555+0000 7fa90f200640 1 -- 172.21.15.119:0/1769715569 shutdown_connections 2024-04-25T10:08:00.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.555+0000 7fa90f200640 1 -- 172.21.15.119:0/1769715569 wait complete. 2024-04-25T10:08:00.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.557+0000 7fa90f200640 1 Processor -- start 2024-04-25T10:08:00.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.557+0000 7fa90f200640 1 -- start start 2024-04-25T10:08:00.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.558+0000 7fa90f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9080b9270 0x7fa9081e5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.558+0000 7fa90f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9080d3f80 0x7fa908146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.558+0000 7fa90f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90815db60 0x7fa908147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.558+0000 7fa90f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080b3120 con 0x7fa90815db60 2024-04-25T10:08:00.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.558+0000 7fa90f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080b2fa0 con 0x7fa9080b9270 2024-04-25T10:08:00.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.558+0000 7fa90f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90815df40 con 0x7fa9080d3f80 2024-04-25T10:08:00.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.558+0000 7fa90e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90815db60 0x7fa908147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.558+0000 7fa90e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90815db60 0x7fa908147330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49272/0 (socket says 172.21.15.119:49272) 2024-04-25T10:08:00.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.558+0000 7fa90e800640 1 -- 172.21.15.119:0/277749193 learned_addr learned my addr 172.21.15.119:0/277749193 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:00.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.558+0000 7fa90de00640 1 --2- 172.21.15.119:0/277749193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9080b9270 0x7fa9081e5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.558+0000 7fa90e800640 1 -- 172.21.15.119:0/277749193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9080d3f80 msgr2=0x7fa908146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.558+0000 7fa90e800640 1 --2- 172.21.15.119:0/277749193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9080d3f80 0x7fa908146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.558+0000 7fa90e800640 1 -- 172.21.15.119:0/277749193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9080b9270 msgr2=0x7fa9081e5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.559+0000 7fa90e800640 1 --2- 172.21.15.119:0/277749193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9080b9270 0x7fa9081e5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.559+0000 7fa90e800640 1 -- 172.21.15.119:0/277749193 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9080cb0e0 con 0x7fa90815db60 2024-04-25T10:08:00.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.559+0000 7fa90de00640 1 --2- 172.21.15.119:0/277749193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9080b9270 0x7fa9081e5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:00.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.559+0000 7fa90e800640 1 --2- 172.21.15.119:0/277749193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90815db60 0x7fa908147330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa904009f70 tx=0x7fa904004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.559+0000 7fa902000640 1 -- 172.21.15.119:0/277749193 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90403f020 con 0x7fa90815db60 2024-04-25T10:08:00.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.559+0000 7fa902000640 1 -- 172.21.15.119:0/277749193 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa904037040 con 0x7fa90815db60 2024-04-25T10:08:00.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.559+0000 7fa902000640 1 -- 172.21.15.119:0/277749193 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904036070 con 0x7fa90815db60 2024-04-25T10:08:00.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.559+0000 7fa90f200640 1 -- 172.21.15.119:0/277749193 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9080cae00 con 0x7fa90815db60 2024-04-25T10:08:00.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.559+0000 7fa90f200640 1 -- 172.21.15.119:0/277749193 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9081e5d20 con 0x7fa90815db60 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.560+0000 7fa90f200640 1 -- 172.21.15.119:0/277749193 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9080749b0 con 0x7fa90815db60 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.561+0000 7fa902000640 1 -- 172.21.15.119:0/277749193 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa904042020 con 0x7fa90815db60 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.562+0000 7fa902000640 1 --2- 172.21.15.119:0/277749193 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8ec06b4d0 0x7fa8ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.562+0000 7fa902000640 1 -- 172.21.15.119:0/277749193 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fa9040aa8f0 con 0x7fa90815db60 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.563+0000 7fa90de00640 1 --2- 172.21.15.119:0/277749193 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8ec06b4d0 0x7fa8ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.563+0000 7fa90de00640 1 --2- 172.21.15.119:0/277749193 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8ec06b4d0 0x7fa8ec06d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc000f60 tx=0x7fa8fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.567+0000 7fa902000640 1 -- 172.21.15.119:0/277749193 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa904049030 con 0x7fa90815db60 2024-04-25T10:08:00.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.763+0000 7fa90f200640 1 -- 172.21.15.119:0/277749193 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9080d1a60 con 0x7fa90815db60 2024-04-25T10:08:00.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.765+0000 7fa902000640 1 -- 172.21.15.119:0/277749193 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16861 (secure 0 0 0) 0x7fa9080cb0e0 con 0x7fa90815db60 2024-04-25T10:08:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:00.772 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":22,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:49.883606+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:00.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.768+0000 7fa90f200640 1 -- 172.21.15.119:0/277749193 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8ec06b4d0 msgr2=0x7fa8ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.768+0000 7fa90f200640 1 --2- 172.21.15.119:0/277749193 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8ec06b4d0 0x7fa8ec06d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc000f60 tx=0x7fa8fc009040 comp rx=0 tx=0).stop 2024-04-25T10:08:00.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.768+0000 7fa90f200640 1 -- 172.21.15.119:0/277749193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90815db60 msgr2=0x7fa908147330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.768+0000 7fa90f200640 1 --2- 172.21.15.119:0/277749193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90815db60 0x7fa908147330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa904009f70 tx=0x7fa904004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:00.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.768+0000 7fa90f200640 1 -- 172.21.15.119:0/277749193 shutdown_connections 2024-04-25T10:08:00.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.768+0000 7fa90f200640 1 --2- 172.21.15.119:0/277749193 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8ec06b4d0 0x7fa8ec06d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.768+0000 7fa90f200640 1 --2- 172.21.15.119:0/277749193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90815db60 0x7fa908147330 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.768+0000 7fa90f200640 1 --2- 172.21.15.119:0/277749193 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9080d3f80 0x7fa908146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.768+0000 7fa90f200640 1 --2- 172.21.15.119:0/277749193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9080b9270 0x7fa9081e5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.768+0000 7fa90f200640 1 -- 172.21.15.119:0/277749193 >> 172.21.15.119:0/277749193 conn(0x7fa9080726c0 msgr2=0x7fa9080c7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.768+0000 7fa90f200640 1 -- 172.21.15.119:0/277749193 shutdown_connections 2024-04-25T10:08:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.768+0000 7fa90f200640 1 -- 172.21.15.119:0/277749193 wait complete. 2024-04-25T10:08:00.793 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:08:00.793 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:00.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.988+0000 7f4af1800640 1 Processor -- start 2024-04-25T10:08:00.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.988+0000 7f4af1800640 1 -- start start 2024-04-25T10:08:00.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.990+0000 7f4af1800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0d35b0 0x7f4aec0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.990+0000 7f4af1800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4aec0b5990 0x7f4aec0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.990+0000 7f4af1800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aec15e0c0 con 0x7f4aec15dce0 2024-04-25T10:08:00.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.990+0000 7f4af1800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aec0b5de0 con 0x7f4aec0b5990 2024-04-25T10:08:00.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.990+0000 7f4af1800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec0d2510 con 0x7f4aec0d35b0 2024-04-25T10:08:00.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.990+0000 7f4aeaa00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4aec15dce0 0x7f4aec0d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60924/0 (socket says 172.21.15.119:60924) 2024-04-25T10:08:00.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.990+0000 7f4aeaa00640 1 -- 172.21.15.119:0/1929772420 learned_addr learned my addr 172.21.15.119:0/1929772420 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:00.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.990+0000 7f4aeb400640 1 --2- 172.21.15.119:0/1929772420 >> v2:172.21.15.119:3300/0 conn(0x7f4aec0b5990 0x7f4aec0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.991+0000 7f4aea000640 1 --2- 172.21.15.119:0/1929772420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0d35b0 0x7f4aec0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.991+0000 7f4ae9600640 1 -- 172.21.15.119:0/1929772420 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206136200 0 0) 0x7f4aec15e0c0 con 0x7f4aec15dce0 2024-04-25T10:08:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.991+0000 7f4ae9600640 1 -- 172.21.15.119:0/1929772420 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4acc003520 con 0x7f4aec15dce0 2024-04-25T10:08:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.991+0000 7f4aeb400640 1 -- 172.21.15.119:0/1929772420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0d35b0 msgr2=0x7f4aec0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.991+0000 7f4aeb400640 1 --2- 172.21.15.119:0/1929772420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0d35b0 0x7f4aec0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.991+0000 7f4aeb400640 1 -- 172.21.15.119:0/1929772420 >> v1:172.21.15.119:6789/0 conn(0x7f4aec15dce0 legacy=0x7f4aec0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.991+0000 7f4aeb400640 1 -- 172.21.15.119:0/1929772420 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec0d27f0 con 0x7f4aec0b5990 2024-04-25T10:08:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.991+0000 7f4aea000640 1 --2- 172.21.15.119:0/1929772420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0d35b0 0x7f4aec0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:00.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.992+0000 7f4aeb400640 1 --2- 172.21.15.119:0/1929772420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aec0b5990 0x7f4aec0b6b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40097a0 tx=0x7f4ae402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1e4d87ee715a88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.992+0000 7f4ae9600640 1 -- 172.21.15.119:0/1929772420 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4037070 con 0x7f4aec0b5990 2024-04-25T10:08:00.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.993+0000 7f4ae9600640 1 -- 172.21.15.119:0/1929772420 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae4032440 con 0x7f4aec0b5990 2024-04-25T10:08:00.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.993+0000 7f4ae9600640 1 -- 172.21.15.119:0/1929772420 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4032810 con 0x7f4aec0b5990 2024-04-25T10:08:00.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.993+0000 7f4af1800640 1 -- 172.21.15.119:0/1929772420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aec0b5990 msgr2=0x7f4aec0b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.993+0000 7f4af1800640 1 --2- 172.21.15.119:0/1929772420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aec0b5990 0x7f4aec0b6b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40097a0 tx=0x7f4ae402a180 comp rx=0 tx=0).stop 2024-04-25T10:08:01.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.993+0000 7f4af1800640 1 -- 172.21.15.119:0/1929772420 shutdown_connections 2024-04-25T10:08:01.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.993+0000 7f4af1800640 1 --2- 172.21.15.119:0/1929772420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aec0b5990 0x7f4aec0b6b00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.993+0000 7f4af1800640 1 --2- 172.21.15.119:0/1929772420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0d35b0 0x7f4aec0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.993+0000 7f4af1800640 1 -- 172.21.15.119:0/1929772420 >> 172.21.15.119:0/1929772420 conn(0x7f4aec072900 msgr2=0x7f4aec0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:01.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.993+0000 7f4af1800640 1 -- 172.21.15.119:0/1929772420 shutdown_connections 2024-04-25T10:08:01.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.993+0000 7f4af1800640 1 -- 172.21.15.119:0/1929772420 wait complete. 2024-04-25T10:08:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.995+0000 7f4af1800640 1 Processor -- start 2024-04-25T10:08:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.995+0000 7f4af1800640 1 -- start start 2024-04-25T10:08:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.996+0000 7f4af1800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aec0b5990 0x7f4aec0cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.996+0000 7f4af1800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0d35b0 0x7f4aec0cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.996+0000 7f4af1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aec15dce0 0x7f4aec0c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.996+0000 7f4af1800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec0d3090 con 0x7f4aec15dce0 2024-04-25T10:08:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.996+0000 7f4af1800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec0d2db0 con 0x7f4aec0b5990 2024-04-25T10:08:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.996+0000 7f4af1800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec0d2160 con 0x7f4aec0d35b0 2024-04-25T10:08:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.996+0000 7f4aeb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aec15dce0 0x7f4aec0c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.996+0000 7f4aeb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aec15dce0 0x7f4aec0c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45172/0 (socket says 172.21.15.119:45172) 2024-04-25T10:08:01.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.996+0000 7f4aeb400640 1 -- 172.21.15.119:0/4102494820 learned_addr learned my addr 172.21.15.119:0/4102494820 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:01.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.997+0000 7f4aeb400640 1 -- 172.21.15.119:0/4102494820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0d35b0 msgr2=0x7f4aec0cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.997+0000 7f4aeb400640 1 --2- 172.21.15.119:0/4102494820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0d35b0 0x7f4aec0cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.997+0000 7f4aeb400640 1 -- 172.21.15.119:0/4102494820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aec0b5990 msgr2=0x7f4aec0cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.997+0000 7f4aeb400640 1 --2- 172.21.15.119:0/4102494820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aec0b5990 0x7f4aec0cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.997+0000 7f4aeb400640 1 -- 172.21.15.119:0/4102494820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec155540 con 0x7f4aec15dce0 2024-04-25T10:08:01.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.997+0000 7f4aeb400640 1 --2- 172.21.15.119:0/4102494820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aec15dce0 0x7f4aec0c8f80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4035040 tx=0x7f4ae40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.997+0000 7f4ae2a00640 1 -- 172.21.15.119:0/4102494820 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4037040 con 0x7f4aec15dce0 2024-04-25T10:08:01.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.997+0000 7f4ae2a00640 1 -- 172.21.15.119:0/4102494820 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae4042070 con 0x7f4aec15dce0 2024-04-25T10:08:01.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.997+0000 7f4af1800640 1 -- 172.21.15.119:0/4102494820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aec0cb4b0 con 0x7f4aec15dce0 2024-04-25T10:08:01.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.998+0000 7f4ae2a00640 1 -- 172.21.15.119:0/4102494820 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40308a0 con 0x7f4aec15dce0 2024-04-25T10:08:01.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.998+0000 7f4af1800640 1 -- 172.21.15.119:0/4102494820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aec0cba30 con 0x7f4aec15dce0 2024-04-25T10:08:01.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:00.999+0000 7f4af1800640 1 -- 172.21.15.119:0/4102494820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aec0749b0 con 0x7f4aec15dce0 2024-04-25T10:08:01.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.000+0000 7f4ae2a00640 1 -- 172.21.15.119:0/4102494820 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ae4030a40 con 0x7f4aec15dce0 2024-04-25T10:08:01.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.001+0000 7f4ae2a00640 1 --2- 172.21.15.119:0/4102494820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ac806b4d0 0x7f4ac806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.001+0000 7f4ae2a00640 1 -- 172.21.15.119:0/4102494820 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f4ae40aaa60 con 0x7f4aec15dce0 2024-04-25T10:08:01.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.001+0000 7f4aeaa00640 1 --2- 172.21.15.119:0/4102494820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ac806b4d0 0x7f4ac806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.002+0000 7f4aeaa00640 1 --2- 172.21.15.119:0/4102494820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ac806b4d0 0x7f4ac806d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4adc000f60 tx=0x7f4adc0078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.005+0000 7f4ae2a00640 1 -- 172.21.15.119:0/4102494820 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ae4079910 con 0x7f4aec15dce0 2024-04-25T10:08:01.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.235+0000 7f4af1800640 1 -- 172.21.15.119:0/4102494820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4aec06b080 con 0x7f4aec15dce0 2024-04-25T10:08:01.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.237+0000 7f4ae2a00640 1 -- 172.21.15.119:0/4102494820 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4941 (secure 0 0 0) 0x7f4ae4079210 con 0x7f4aec15dce0 2024-04-25T10:08:01.243 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:01.243 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1385207078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1385207078},{"type":"v1","addr":"172.21.15.132:6834","nonce":1385207078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1580923823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1580923823},{"type":"v1","addr":"172.21.15.119:6836","nonce":1580923823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4260,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/954053568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":954053568},{"type":"v1","addr":"172.21.15.132:6835","nonce":954053568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:06.743184+0000","modified":"2024-04-25T10:07:10.118630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1632024048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1632024048},{"type":"v1","addr":"172.21.15.119:6837","nonce":1632024048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:08:01.243 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2024-04-25T10:08:01.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.239+0000 7f4af1800640 1 -- 172.21.15.119:0/4102494820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ac806b4d0 msgr2=0x7f4ac806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.239+0000 7f4af1800640 1 --2- 172.21.15.119:0/4102494820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ac806b4d0 0x7f4ac806d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4adc000f60 tx=0x7f4adc0078f0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.239+0000 7f4af1800640 1 -- 172.21.15.119:0/4102494820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aec15dce0 msgr2=0x7f4aec0c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.239+0000 7f4af1800640 1 --2- 172.21.15.119:0/4102494820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aec15dce0 0x7f4aec0c8f80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4035040 tx=0x7f4ae40046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.240+0000 7f4af1800640 1 -- 172.21.15.119:0/4102494820 shutdown_connections 2024-04-25T10:08:01.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.240+0000 7f4af1800640 1 --2- 172.21.15.119:0/4102494820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ac806b4d0 0x7f4ac806d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.240+0000 7f4af1800640 1 --2- 172.21.15.119:0/4102494820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4aec15dce0 0x7f4aec0c8f80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.240+0000 7f4af1800640 1 --2- 172.21.15.119:0/4102494820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4aec0d35b0 0x7f4aec0cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.240+0000 7f4af1800640 1 --2- 172.21.15.119:0/4102494820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4aec0b5990 0x7f4aec0cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.240+0000 7f4af1800640 1 -- 172.21.15.119:0/4102494820 >> 172.21.15.119:0/4102494820 conn(0x7f4aec072900 msgr2=0x7f4aec0ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:01.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.240+0000 7f4af1800640 1 -- 172.21.15.119:0/4102494820 shutdown_connections 2024-04-25T10:08:01.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.240+0000 7f4af1800640 1 -- 172.21.15.119:0/4102494820 wait complete. 2024-04-25T10:08:01.265 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:01.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.456+0000 7f4706400640 1 Processor -- start 2024-04-25T10:08:01.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.456+0000 7f4706400640 1 -- start start 2024-04-25T10:08:01.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.458+0000 7f4706400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f470015bf00 0x7f47000cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.458+0000 7f4706400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f470006e8f0 0x7f47001654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.458+0000 7f4706400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47000ce6c0 con 0x7f47000cc310 2024-04-25T10:08:01.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.458+0000 7f4706400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47001455e0 con 0x7f470015bf00 2024-04-25T10:08:01.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.458+0000 7f4706400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470006d450 con 0x7f470006e8f0 2024-04-25T10:08:01.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.458+0000 7f46fea00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f47000cc310 0x7f470015c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60940/0 (socket says 172.21.15.119:60940) 2024-04-25T10:08:01.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.458+0000 7f46fea00640 1 -- 172.21.15.119:0/508225349 learned_addr learned my addr 172.21.15.119:0/508225349 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:01.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.458+0000 7f46fe000640 1 -- 172.21.15.119:0/508225349 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074557422 0 0) 0x7f46e8001110 con 0x7f47000cc310 2024-04-25T10:08:01.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.458+0000 7f46ff400640 1 --2- 172.21.15.119:0/508225349 >> v2:172.21.15.132:3300/0 conn(0x7f470015bf00 0x7f47000cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.458+0000 7f46ffe00640 1 --2- 172.21.15.119:0/508225349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f470006e8f0 0x7f47001654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 -- 172.21.15.119:0/508225349 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46e4003520 con 0x7f47000cc310 2024-04-25T10:08:01.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 -- 172.21.15.119:0/508225349 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 400824896 0 0) 0x7f46e4003520 con 0x7f47000cc310 2024-04-25T10:08:01.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 -- 172.21.15.119:0/508225349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f470006e8f0 msgr2=0x7f47001654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 --2- 172.21.15.119:0/508225349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f470006e8f0 0x7f47001654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 -- 172.21.15.119:0/508225349 >> v2:172.21.15.132:3300/0 conn(0x7f470015bf00 msgr2=0x7f47000cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 --2- 172.21.15.119:0/508225349 >> v2:172.21.15.132:3300/0 conn(0x7f470015bf00 0x7f47000cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 -- 172.21.15.119:0/508225349 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47000cc7b0 con 0x7f47000cc310 2024-04-25T10:08:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 -- 172.21.15.119:0/508225349 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f46e8001840 con 0x7f47000cc310 2024-04-25T10:08:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 -- 172.21.15.119:0/508225349 >> v1:172.21.15.119:6789/0 conn(0x7f47000cc310 legacy=0x7f470015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 --2- 172.21.15.119:0/508225349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e40039b0 0x7f46e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 --2- 172.21.15.119:0/508225349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46e4004800 0x7f46e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 --2- 172.21.15.119:0/508225349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f470006e8f0 0x7f46e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 -- 172.21.15.119:0/508225349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47001455e0 con 0x7f46e40039b0 2024-04-25T10:08:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 -- 172.21.15.119:0/508225349 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470006d450 con 0x7f46e4004800 2024-04-25T10:08:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46fe000640 1 -- 172.21.15.119:0/508225349 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e400ba40 con 0x7f470006e8f0 2024-04-25T10:08:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.459+0000 7f46ff400640 1 --2- 172.21.15.119:0/508225349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e40039b0 0x7f46e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.460+0000 7f46fea00640 1 --2- 172.21.15.119:0/508225349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46e4004800 0x7f46e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.460+0000 7f46ff400640 1 -- 172.21.15.119:0/508225349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f470006e8f0 msgr2=0x7f46e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.460+0000 7f46ff400640 1 --2- 172.21.15.119:0/508225349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f470006e8f0 0x7f46e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.460+0000 7f46ff400640 1 -- 172.21.15.119:0/508225349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46e4004800 msgr2=0x7f46e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.460+0000 7f46ff400640 1 --2- 172.21.15.119:0/508225349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46e4004800 0x7f46e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.460+0000 7f46ff400640 1 -- 172.21.15.119:0/508225349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e4003520 con 0x7f46e40039b0 2024-04-25T10:08:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.460+0000 7f46fea00640 1 --2- 172.21.15.119:0/508225349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46e4004800 0x7f46e4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.461+0000 7f46ff400640 1 --2- 172.21.15.119:0/508225349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e40039b0 0x7f46e4003dd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009ac0 tx=0x7f46e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b889ff7ff5ad5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.461+0000 7f46fe000640 1 -- 172.21.15.119:0/508225349 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0036070 con 0x7f46e40039b0 2024-04-25T10:08:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.461+0000 7f46fe000640 1 -- 172.21.15.119:0/508225349 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f0031400 con 0x7f46e40039b0 2024-04-25T10:08:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.461+0000 7f46fe000640 1 -- 172.21.15.119:0/508225349 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0009690 con 0x7f46e40039b0 2024-04-25T10:08:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.462+0000 7f4706400640 1 -- 172.21.15.119:0/508225349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e40039b0 msgr2=0x7f46e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.462+0000 7f4706400640 1 --2- 172.21.15.119:0/508225349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46e40039b0 0x7f46e4003dd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009ac0 tx=0x7f46e4005830 comp rx=0 tx=0).stop 2024-04-25T10:08:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.462+0000 7f46ffe00640 1 -- 172.21.15.119:0/508225349 reap_dead start 2024-04-25T10:08:01.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.462+0000 7f4706400640 1 -- 172.21.15.119:0/508225349 shutdown_connections 2024-04-25T10:08:01.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.462+0000 7f4706400640 1 -- 172.21.15.119:0/508225349 >> 172.21.15.119:0/508225349 conn(0x7f470006aba0 msgr2=0x7f470006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:01.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.462+0000 7f4706400640 1 -- 172.21.15.119:0/508225349 shutdown_connections 2024-04-25T10:08:01.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.462+0000 7f4706400640 1 -- 172.21.15.119:0/508225349 wait complete. 2024-04-25T10:08:01.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.464+0000 7f4706400640 1 Processor -- start 2024-04-25T10:08:01.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.464+0000 7f4706400640 1 -- start start 2024-04-25T10:08:01.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.465+0000 7f4706400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46e40039b0 0x7f47000b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.465+0000 7f4706400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f470015a4b0 0x7f47000b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.465+0000 7f4706400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470015a910 0x7f4700140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.465+0000 7f4706400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47000cd050 con 0x7f470015a910 2024-04-25T10:08:01.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.465+0000 7f4706400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47000ccd70 con 0x7f470015a4b0 2024-04-25T10:08:01.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.465+0000 7f4706400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470006ee60 con 0x7f46e40039b0 2024-04-25T10:08:01.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.465+0000 7f46ffe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470015a910 0x7f4700140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.465+0000 7f46ffe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470015a910 0x7f4700140ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45190/0 (socket says 172.21.15.119:45190) 2024-04-25T10:08:01.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.465+0000 7f46ffe00640 1 -- 172.21.15.119:0/3819042633 learned_addr learned my addr 172.21.15.119:0/3819042633 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.465+0000 7f46ffe00640 1 -- 172.21.15.119:0/3819042633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46e40039b0 msgr2=0x7f47000b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.465+0000 7f46ffe00640 1 --2- 172.21.15.119:0/3819042633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46e40039b0 0x7f47000b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.465+0000 7f46ffe00640 1 -- 172.21.15.119:0/3819042633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f470015a4b0 msgr2=0x7f47000b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.465+0000 7f46ffe00640 1 --2- 172.21.15.119:0/3819042633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f470015a4b0 0x7f47000b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.465+0000 7f46ffe00640 1 -- 172.21.15.119:0/3819042633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47000b44f0 con 0x7f470015a910 2024-04-25T10:08:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.466+0000 7f46ffe00640 1 --2- 172.21.15.119:0/3819042633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470015a910 0x7f4700140ca0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f47000d0190 tx=0x7f46f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.466+0000 7f46f7400640 1 -- 172.21.15.119:0/3819042633 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f800ebc0 con 0x7f470015a910 2024-04-25T10:08:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.466+0000 7f46f7400640 1 -- 172.21.15.119:0/3819042633 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f800ed80 con 0x7f470015a910 2024-04-25T10:08:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.466+0000 7f46f7400640 1 -- 172.21.15.119:0/3819042633 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8003bd0 con 0x7f470015a910 2024-04-25T10:08:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.466+0000 7f4706400640 1 -- 172.21.15.119:0/3819042633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47000b2880 con 0x7f470015a910 2024-04-25T10:08:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.466+0000 7f4706400640 1 -- 172.21.15.119:0/3819042633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47000b2e40 con 0x7f470015a910 2024-04-25T10:08:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.467+0000 7f4706400640 1 -- 172.21.15.119:0/3819042633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47000b2670 con 0x7f470015a910 2024-04-25T10:08:01.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.468+0000 7f46f7400640 1 -- 172.21.15.119:0/3819042633 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f46f801b020 con 0x7f470015a910 2024-04-25T10:08:01.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.469+0000 7f46f7400640 1 --2- 172.21.15.119:0/3819042633 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f46dc06b4d0 0x7f46dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.469+0000 7f46f7400640 1 -- 172.21.15.119:0/3819042633 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f46f808a100 con 0x7f470015a910 2024-04-25T10:08:01.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.469+0000 7f46ff400640 1 --2- 172.21.15.119:0/3819042633 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f46dc06b4d0 0x7f46dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.470+0000 7f46ff400640 1 --2- 172.21.15.119:0/3819042633 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f46dc06b4d0 0x7f46dc06d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f46f0033000 tx=0x7f46f002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.473+0000 7f46f7400640 1 -- 172.21.15.119:0/3819042633 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f46f8058e80 con 0x7f470015a910 2024-04-25T10:08:01.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.692+0000 7f4706400640 1 -- 172.21.15.119:0/3819042633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47000b3010 con 0x7f470015a910 2024-04-25T10:08:01.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.694+0000 7f46f7400640 1 -- 172.21.15.119:0/3819042633 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4941 (secure 0 0 0) 0x7f46f8058780 con 0x7f470015a910 2024-04-25T10:08:01.700 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:01.700 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1385207078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1385207078},{"type":"v1","addr":"172.21.15.132:6834","nonce":1385207078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1580923823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1580923823},{"type":"v1","addr":"172.21.15.119:6836","nonce":1580923823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4260,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/954053568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":954053568},{"type":"v1","addr":"172.21.15.132:6835","nonce":954053568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:06.743184+0000","modified":"2024-04-25T10:07:10.118630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1632024048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1632024048},{"type":"v1","addr":"172.21.15.119:6837","nonce":1632024048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:08:01.700 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2024-04-25T10:08:01.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.696+0000 7f4706400640 1 -- 172.21.15.119:0/3819042633 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f46dc06b4d0 msgr2=0x7f46dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.696+0000 7f4706400640 1 --2- 172.21.15.119:0/3819042633 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f46dc06b4d0 0x7f46dc06d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f46f0033000 tx=0x7f46f002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:01.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.696+0000 7f4706400640 1 -- 172.21.15.119:0/3819042633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470015a910 msgr2=0x7f4700140ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.696+0000 7f4706400640 1 --2- 172.21.15.119:0/3819042633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470015a910 0x7f4700140ca0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f47000d0190 tx=0x7f46f800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.696+0000 7f4706400640 1 -- 172.21.15.119:0/3819042633 shutdown_connections 2024-04-25T10:08:01.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.696+0000 7f4706400640 1 --2- 172.21.15.119:0/3819042633 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f46dc06b4d0 0x7f46dc06d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.696+0000 7f4706400640 1 --2- 172.21.15.119:0/3819042633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f470015a910 0x7f4700140ca0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.696+0000 7f4706400640 1 --2- 172.21.15.119:0/3819042633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f470015a4b0 0x7f47000b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.696+0000 7f4706400640 1 --2- 172.21.15.119:0/3819042633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46e40039b0 0x7f47000b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.696+0000 7f4706400640 1 -- 172.21.15.119:0/3819042633 >> 172.21.15.119:0/3819042633 conn(0x7f470006aba0 msgr2=0x7f4700143a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:01.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.696+0000 7f4706400640 1 -- 172.21.15.119:0/3819042633 shutdown_connections 2024-04-25T10:08:01.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.696+0000 7f4706400640 1 -- 172.21.15.119:0/3819042633 wait complete. 2024-04-25T10:08:01.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:01.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.906+0000 7f073f400640 1 Processor -- start 2024-04-25T10:08:01.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.906+0000 7f073f400640 1 -- start start 2024-04-25T10:08:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.908+0000 7f073f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f073815bf00 0x7f07380cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.908+0000 7f073f400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f07380cc310 0x7f073815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.908+0000 7f073f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f073815c820 0x7f073806d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.908+0000 7f073f400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07381455e0 con 0x7f073815c820 2024-04-25T10:08:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.908+0000 7f073f400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073806d450 con 0x7f07380cc310 2024-04-25T10:08:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.908+0000 7f073f400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07380ce6c0 con 0x7f073815bf00 2024-04-25T10:08:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.908+0000 7f073d400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f073815c820 0x7f073806d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.908+0000 7f073d400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f073815c820 0x7f073806d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45198/0 (socket says 172.21.15.119:45198) 2024-04-25T10:08:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.908+0000 7f073d400640 1 -- 172.21.15.119:0/2775439364 learned_addr learned my addr 172.21.15.119:0/2775439364 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.909+0000 7f0737e00640 1 --2- 172.21.15.119:0/2775439364 >> v2:172.21.15.132:3300/0 conn(0x7f07380cc310 0x7f073815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.909+0000 7f073d400640 1 -- 172.21.15.119:0/2775439364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f073815bf00 msgr2=0x7f07380cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.909+0000 7f073ca00640 1 --2- 172.21.15.119:0/2775439364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f073815bf00 0x7f07380cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.909+0000 7f073d400640 1 --2- 172.21.15.119:0/2775439364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f073815bf00 0x7f07380cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.909+0000 7f073d400640 1 -- 172.21.15.119:0/2775439364 >> v2:172.21.15.132:3300/0 conn(0x7f07380cc310 msgr2=0x7f073815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.909+0000 7f073d400640 1 --2- 172.21.15.119:0/2775439364 >> v2:172.21.15.132:3300/0 conn(0x7f07380cc310 0x7f073815c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.909+0000 7f073d400640 1 -- 172.21.15.119:0/2775439364 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07380cc7b0 con 0x7f073815c820 2024-04-25T10:08:01.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.910+0000 7f073d400640 1 --2- 172.21.15.119:0/2775439364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f073815c820 0x7f073806d8e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f073806ba40 tx=0x7f073002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fbf513cb3f1d71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.910+0000 7f0737400640 1 -- 172.21.15.119:0/2775439364 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730036070 con 0x7f073815c820 2024-04-25T10:08:01.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.910+0000 7f0737400640 1 -- 172.21.15.119:0/2775439364 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0730031600 con 0x7f073815c820 2024-04-25T10:08:01.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.910+0000 7f0737400640 1 -- 172.21.15.119:0/2775439364 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07300319d0 con 0x7f073815c820 2024-04-25T10:08:01.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.911+0000 7f073f400640 1 -- 172.21.15.119:0/2775439364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f073815c820 msgr2=0x7f073806d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.911+0000 7f073f400640 1 --2- 172.21.15.119:0/2775439364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f073815c820 0x7f073806d8e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f073806ba40 tx=0x7f073002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.911+0000 7f073f400640 1 -- 172.21.15.119:0/2775439364 shutdown_connections 2024-04-25T10:08:01.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.911+0000 7f073f400640 1 --2- 172.21.15.119:0/2775439364 >> v2:172.21.15.132:3300/0 conn(0x7f07380cc310 0x7f073815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.911+0000 7f073f400640 1 --2- 172.21.15.119:0/2775439364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f073815c820 0x7f073806d8e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.911+0000 7f073f400640 1 --2- 172.21.15.119:0/2775439364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f073815bf00 0x7f07380cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.911+0000 7f073f400640 1 -- 172.21.15.119:0/2775439364 >> 172.21.15.119:0/2775439364 conn(0x7f073806aba0 msgr2=0x7f073806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:01.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.911+0000 7f073f400640 1 -- 172.21.15.119:0/2775439364 shutdown_connections 2024-04-25T10:08:01.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.911+0000 7f073f400640 1 -- 172.21.15.119:0/2775439364 wait complete. 2024-04-25T10:08:01.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.913+0000 7f073f400640 1 Processor -- start 2024-04-25T10:08:01.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.913+0000 7f073f400640 1 -- start start 2024-04-25T10:08:01.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.914+0000 7f073f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07380cc310 0x7f073812bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.914+0000 7f073f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f073815bf00 0x7f073813f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.914+0000 7f073f400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f073815c820 0x7f073815e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.914+0000 7f073f400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380cd050 con 0x7f07380cc310 2024-04-25T10:08:01.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.914+0000 7f073f400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380ccd70 con 0x7f073815c820 2024-04-25T10:08:01.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.914+0000 7f073f400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073815cca0 con 0x7f073815bf00 2024-04-25T10:08:01.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.914+0000 7f073ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07380cc310 0x7f073812bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.914+0000 7f073ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07380cc310 0x7f073812bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45214/0 (socket says 172.21.15.119:45214) 2024-04-25T10:08:01.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.914+0000 7f073ca00640 1 -- 172.21.15.119:0/3375327585 learned_addr learned my addr 172.21.15.119:0/3375327585 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:01.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.914+0000 7f073ca00640 1 -- 172.21.15.119:0/3375327585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f073815bf00 msgr2=0x7f073813f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.914+0000 7f073ca00640 1 --2- 172.21.15.119:0/3375327585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f073815bf00 0x7f073813f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.914+0000 7f073ca00640 1 -- 172.21.15.119:0/3375327585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f073815c820 msgr2=0x7f073815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.914+0000 7f073ca00640 1 --2- 172.21.15.119:0/3375327585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f073815c820 0x7f073815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.914+0000 7f073ca00640 1 -- 172.21.15.119:0/3375327585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07381407f0 con 0x7f07380cc310 2024-04-25T10:08:01.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.915+0000 7f073ca00640 1 --2- 172.21.15.119:0/3375327585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07380cc310 0x7f073812bba0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f072c00c820 tx=0x7f072c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.915+0000 7f0734c00640 1 -- 172.21.15.119:0/3375327585 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c015070 con 0x7f07380cc310 2024-04-25T10:08:01.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.915+0000 7f0734c00640 1 -- 172.21.15.119:0/3375327585 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c004240 con 0x7f07380cc310 2024-04-25T10:08:01.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.915+0000 7f0734c00640 1 -- 172.21.15.119:0/3375327585 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c00ce10 con 0x7f07380cc310 2024-04-25T10:08:01.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.915+0000 7f073f400640 1 -- 172.21.15.119:0/3375327585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073813ff50 con 0x7f07380cc310 2024-04-25T10:08:01.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.915+0000 7f073f400640 1 -- 172.21.15.119:0/3375327585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0738140510 con 0x7f07380cc310 2024-04-25T10:08:01.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.916+0000 7f073f400640 1 -- 172.21.15.119:0/3375327585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073806bb30 con 0x7f07380cc310 2024-04-25T10:08:01.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.917+0000 7f0734c00640 1 -- 172.21.15.119:0/3375327585 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f072c007830 con 0x7f07380cc310 2024-04-25T10:08:01.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.918+0000 7f0734c00640 1 --2- 172.21.15.119:0/3375327585 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f071806b4d0 0x7f071806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.918+0000 7f0734c00640 1 -- 172.21.15.119:0/3375327585 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f072c08a080 con 0x7f07380cc310 2024-04-25T10:08:01.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.918+0000 7f0737e00640 1 --2- 172.21.15.119:0/3375327585 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f071806b4d0 0x7f071806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.919+0000 7f0737e00640 1 --2- 172.21.15.119:0/3375327585 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f071806b4d0 0x7f071806d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f07381f4ab0 tx=0x7f0724008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:01.922+0000 7f0734c00640 1 -- 172.21.15.119:0/3375327585 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f072c058ea0 con 0x7f07380cc310 2024-04-25T10:08:02.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.118+0000 7f073f400640 1 -- 172.21.15.119:0/3375327585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07380b0fe0 con 0x7f07380cc310 2024-04-25T10:08:02.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.121+0000 7f0734c00640 1 -- 172.21.15.119:0/3375327585 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16861 (secure 0 0 0) 0x7f072c0587a0 con 0x7f07380cc310 2024-04-25T10:08:02.127 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:02.128 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":22,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:07:49.883606+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:05.135832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:05.768083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:02.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.123+0000 7f073f400640 1 -- 172.21.15.119:0/3375327585 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f071806b4d0 msgr2=0x7f071806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.124+0000 7f073f400640 1 --2- 172.21.15.119:0/3375327585 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f071806b4d0 0x7f071806d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f07381f4ab0 tx=0x7f0724008040 comp rx=0 tx=0).stop 2024-04-25T10:08:02.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.124+0000 7f073f400640 1 -- 172.21.15.119:0/3375327585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07380cc310 msgr2=0x7f073812bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.124+0000 7f073f400640 1 --2- 172.21.15.119:0/3375327585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07380cc310 0x7f073812bba0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f072c00c820 tx=0x7f072c00cce0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.124+0000 7f073f400640 1 -- 172.21.15.119:0/3375327585 shutdown_connections 2024-04-25T10:08:02.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.124+0000 7f073f400640 1 --2- 172.21.15.119:0/3375327585 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f071806b4d0 0x7f071806d990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.124+0000 7f073f400640 1 --2- 172.21.15.119:0/3375327585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f073815c820 0x7f073815e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.124+0000 7f073f400640 1 --2- 172.21.15.119:0/3375327585 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f073815bf00 0x7f073813f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.124+0000 7f073f400640 1 --2- 172.21.15.119:0/3375327585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07380cc310 0x7f073812bba0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.124+0000 7f073f400640 1 -- 172.21.15.119:0/3375327585 >> 172.21.15.119:0/3375327585 conn(0x7f073806aba0 msgr2=0x7f073815d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.124+0000 7f073f400640 1 -- 172.21.15.119:0/3375327585 shutdown_connections 2024-04-25T10:08:02.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.124+0000 7f073f400640 1 -- 172.21.15.119:0/3375327585 wait complete. 2024-04-25T10:08:02.149 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:08:02.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.327+0000 7fc180a00640 1 Processor -- start 2024-04-25T10:08:02.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.327+0000 7fc180a00640 1 -- start start 2024-04-25T10:08:02.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.329+0000 7fc180a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc17c06b2b0 0x7fc17c0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.329+0000 7fc180a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17c0c5740 con 0x7fc17c0b0e00 2024-04-25T10:08:02.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.329+0000 7fc180a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17c0b11e0 con 0x7fc17c0b34a0 2024-04-25T10:08:02.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.329+0000 7fc180a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17c0ce140 con 0x7fc17c06b2b0 2024-04-25T10:08:02.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.329+0000 7fc179e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc17c06b2b0 0x7fc17c0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.329+0000 7fc17a800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc17c0b0e00 0x7fc17c06aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60956/0 (socket says 172.21.15.119:60956) 2024-04-25T10:08:02.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.329+0000 7fc17a800640 1 -- 172.21.15.119:0/3268481653 learned_addr learned my addr 172.21.15.119:0/3268481653 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:02.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.329+0000 7fc179e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc17c06b2b0 0x7fc17c0b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45220/0 (socket says 172.21.15.119:45220) 2024-04-25T10:08:02.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.329+0000 7fc179e00640 1 -- 172.21.15.119:0/3268481653 >> v1:172.21.15.132:6789/0 conn(0x7fc17c0b34a0 legacy=0x7fc17c1615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.329+0000 7fc179e00640 1 -- 172.21.15.119:0/3268481653 >> v1:172.21.15.119:6789/0 conn(0x7fc17c0b0e00 legacy=0x7fc17c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.330+0000 7fc179e00640 1 -- 172.21.15.119:0/3268481653 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c0c7320 con 0x7fc17c06b2b0 2024-04-25T10:08:02.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.330+0000 7fc179e00640 1 --2- 172.21.15.119:0/3268481653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17c06b2b0 0x7fc17c0b2cc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc1680097a0 tx=0x7fc16802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c5c8779ec42569 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.331+0000 7fc179400640 1 -- 172.21.15.119:0/3268481653 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc168037070 con 0x7fc17c06b2b0 2024-04-25T10:08:02.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.331+0000 7fc179400640 1 -- 172.21.15.119:0/3268481653 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16802ae50 con 0x7fc17c06b2b0 2024-04-25T10:08:02.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.331+0000 7fc179400640 1 -- 172.21.15.119:0/3268481653 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc168032660 con 0x7fc17c06b2b0 2024-04-25T10:08:02.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.331+0000 7fc180a00640 1 -- 172.21.15.119:0/3268481653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17c06b2b0 msgr2=0x7fc17c0b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.331+0000 7fc180a00640 1 --2- 172.21.15.119:0/3268481653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17c06b2b0 0x7fc17c0b2cc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc1680097a0 tx=0x7fc16802a140 comp rx=0 tx=0).stop 2024-04-25T10:08:02.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.331+0000 7fc180a00640 1 -- 172.21.15.119:0/3268481653 shutdown_connections 2024-04-25T10:08:02.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.331+0000 7fc180a00640 1 --2- 172.21.15.119:0/3268481653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17c06b2b0 0x7fc17c0b2cc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.331+0000 7fc180a00640 1 -- 172.21.15.119:0/3268481653 >> 172.21.15.119:0/3268481653 conn(0x7fc17c0b21a0 msgr2=0x7fc17c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.332+0000 7fc180a00640 1 -- 172.21.15.119:0/3268481653 shutdown_connections 2024-04-25T10:08:02.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.332+0000 7fc180a00640 1 -- 172.21.15.119:0/3268481653 wait complete. 2024-04-25T10:08:02.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.333+0000 7fc180a00640 1 Processor -- start 2024-04-25T10:08:02.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.333+0000 7fc180a00640 1 -- start start 2024-04-25T10:08:02.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.334+0000 7fc180a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc17c06b2b0 0x7fc17c14e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.334+0000 7fc180a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc17c0b0e00 0x7fc17c0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.334+0000 7fc180a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17c0b34a0 0x7fc17c0c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.334+0000 7fc180a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c0ce9c0 con 0x7fc17c0b34a0 2024-04-25T10:08:02.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.334+0000 7fc180a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c0ce6e0 con 0x7fc17c06b2b0 2024-04-25T10:08:02.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.334+0000 7fc180a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c0c6c60 con 0x7fc17c0b0e00 2024-04-25T10:08:02.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.334+0000 7fc17b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17c0b34a0 0x7fc17c0c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.334+0000 7fc17b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17c0b34a0 0x7fc17c0c9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45232/0 (socket says 172.21.15.119:45232) 2024-04-25T10:08:02.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.335+0000 7fc17b200640 1 -- 172.21.15.119:0/2511567937 learned_addr learned my addr 172.21.15.119:0/2511567937 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:02.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.335+0000 7fc17b200640 1 -- 172.21.15.119:0/2511567937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc17c0b0e00 msgr2=0x7fc17c0ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.335+0000 7fc17b200640 1 --2- 172.21.15.119:0/2511567937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc17c0b0e00 0x7fc17c0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.335+0000 7fc17b200640 1 -- 172.21.15.119:0/2511567937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc17c06b2b0 msgr2=0x7fc17c14e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.335+0000 7fc17b200640 1 --2- 172.21.15.119:0/2511567937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc17c06b2b0 0x7fc17c14e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.335+0000 7fc17b200640 1 -- 172.21.15.119:0/2511567937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c0ade20 con 0x7fc17c0b34a0 2024-04-25T10:08:02.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.335+0000 7fc17b200640 1 --2- 172.21.15.119:0/2511567937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17c0b34a0 0x7fc17c0c9140 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc17400b9c0 tx=0x7fc17400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.335+0000 7fc172a00640 1 -- 172.21.15.119:0/2511567937 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174019070 con 0x7fc17c0b34a0 2024-04-25T10:08:02.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.335+0000 7fc172a00640 1 -- 172.21.15.119:0/2511567937 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17400cba0 con 0x7fc17c0b34a0 2024-04-25T10:08:02.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.335+0000 7fc180a00640 1 -- 172.21.15.119:0/2511567937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17c0ae100 con 0x7fc17c0b34a0 2024-04-25T10:08:02.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.335+0000 7fc172a00640 1 -- 172.21.15.119:0/2511567937 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174014440 con 0x7fc17c0b34a0 2024-04-25T10:08:02.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.336+0000 7fc180a00640 1 -- 172.21.15.119:0/2511567937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17c0ae6c0 con 0x7fc17c0b34a0 2024-04-25T10:08:02.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.337+0000 7fc180a00640 1 -- 172.21.15.119:0/2511567937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17c0c6560 con 0x7fc17c0b34a0 2024-04-25T10:08:02.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.337+0000 7fc172a00640 1 -- 172.21.15.119:0/2511567937 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc17401c070 con 0x7fc17c0b34a0 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.338+0000 7fc172a00640 1 --2- 172.21.15.119:0/2511567937 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc15c06b4d0 0x7fc15c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.338+0000 7fc172a00640 1 -- 172.21.15.119:0/2511567937 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5804+0+0 (secure 0 0 0) 0x7fc17408a0b0 con 0x7fc17c0b34a0 2024-04-25T10:08:02.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.339+0000 7fc17a800640 1 --2- 172.21.15.119:0/2511567937 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc15c06b4d0 0x7fc15c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.339+0000 7fc17a800640 1 --2- 172.21.15.119:0/2511567937 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc15c06b4d0 0x7fc15c06d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc16c000f60 tx=0x7fc16c007a80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.343+0000 7fc172a00640 1 -- 172.21.15.119:0/2511567937 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc17c0c6560 con 0x7fc17c0b34a0 2024-04-25T10:08:02.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.556+0000 7fc180a00640 1 -- 172.21.15.119:0/2511567937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc17c0749b0 con 0x7fc17c0b34a0 2024-04-25T10:08:02.563 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:08:02.557+0000 7f8c64600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:08:02.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.561+0000 7fc172a00640 1 -- 172.21.15.119:0/2511567937 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc174058e30 con 0x7fc17c0b34a0 2024-04-25T10:08:02.567 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:08:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.563+0000 7fc180a00640 1 -- 172.21.15.119:0/2511567937 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc15c06b4d0 msgr2=0x7fc15c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.563+0000 7fc180a00640 1 --2- 172.21.15.119:0/2511567937 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc15c06b4d0 0x7fc15c06d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc16c000f60 tx=0x7fc16c007a80 comp rx=0 tx=0).stop 2024-04-25T10:08:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.563+0000 7fc180a00640 1 -- 172.21.15.119:0/2511567937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17c0b34a0 msgr2=0x7fc17c0c9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.563+0000 7fc180a00640 1 --2- 172.21.15.119:0/2511567937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17c0b34a0 0x7fc17c0c9140 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc17400b9c0 tx=0x7fc17400be80 comp rx=0 tx=0).stop 2024-04-25T10:08:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.564+0000 7fc180a00640 1 -- 172.21.15.119:0/2511567937 shutdown_connections 2024-04-25T10:08:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.564+0000 7fc180a00640 1 --2- 172.21.15.119:0/2511567937 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc15c06b4d0 0x7fc15c06d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.564+0000 7fc180a00640 1 --2- 172.21.15.119:0/2511567937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17c0b34a0 0x7fc17c0c9140 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.564+0000 7fc180a00640 1 --2- 172.21.15.119:0/2511567937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc17c0b0e00 0x7fc17c0ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.564+0000 7fc180a00640 1 --2- 172.21.15.119:0/2511567937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc17c06b2b0 0x7fc17c14e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.564+0000 7fc180a00640 1 -- 172.21.15.119:0/2511567937 >> 172.21.15.119:0/2511567937 conn(0x7fc17c0b21a0 msgr2=0x7fc17c06e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.564+0000 7fc180a00640 1 -- 172.21.15.119:0/2511567937 shutdown_connections 2024-04-25T10:08:02.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.564+0000 7fc180a00640 1 -- 172.21.15.119:0/2511567937 wait complete. 2024-04-25T10:08:02.590 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:08:02.606 INFO:tasks.ceph.mds.c.smithi119.stderr: -7770> 2024-04-25T10:07:08.260+0000 7f09d7800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:08:02.694 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2024-04-25T10:08:02.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.760+0000 7fbe3be00640 1 Processor -- start 2024-04-25T10:08:02.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.760+0000 7fbe3be00640 1 -- start start 2024-04-25T10:08:02.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.762+0000 7fbe3be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbe3c15bcb0 0x7fbe3c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.762+0000 7fbe3be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbe3c0cea40 0x7fbe3c06df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.762+0000 7fbe3be00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c0712c0 con 0x7fbe3c0c6150 2024-04-25T10:08:02.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.762+0000 7fbe3be00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3c06e790 con 0x7fbe3c0cea40 2024-04-25T10:08:02.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.762+0000 7fbe3be00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3c0715a0 con 0x7fbe3c15bcb0 2024-04-25T10:08:02.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.762+0000 7fbe3b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbe3c0cea40 0x7fbe3c06df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.762+0000 7fbe3b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbe3c0cea40 0x7fbe3c06df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45246/0 (socket says 172.21.15.119:45246) 2024-04-25T10:08:02.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.762+0000 7fbe3b400640 1 -- 172.21.15.119:0/3389220540 learned_addr learned my addr 172.21.15.119:0/3389220540 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:02.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.763+0000 7fbe32000640 1 --2- 172.21.15.119:0/3389220540 >> v2:172.21.15.132:3300/0 conn(0x7fbe3c15bcb0 0x7fbe3c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.763+0000 7fbe3b400640 1 -- 172.21.15.119:0/3389220540 >> v2:172.21.15.132:3300/0 conn(0x7fbe3c15bcb0 msgr2=0x7fbe3c06da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.763+0000 7fbe3b400640 1 --2- 172.21.15.119:0/3389220540 >> v2:172.21.15.132:3300/0 conn(0x7fbe3c15bcb0 0x7fbe3c06da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.763+0000 7fbe3b400640 1 -- 172.21.15.119:0/3389220540 >> v1:172.21.15.132:6789/0 conn(0x7fbe3c0c6150 legacy=0x7fbe3c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.763+0000 7fbe3b400640 1 -- 172.21.15.119:0/3389220540 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c071880 con 0x7fbe3c0cea40 2024-04-25T10:08:02.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.764+0000 7fbe3b400640 1 --2- 172.21.15.119:0/3389220540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0cea40 0x7fbe3c06df50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0cf580 tx=0x7fbe3402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437135afed339f01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.764+0000 7fbe3a000640 1 -- 172.21.15.119:0/3389220540 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34036070 con 0x7fbe3c0cea40 2024-04-25T10:08:02.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.764+0000 7fbe3a000640 1 -- 172.21.15.119:0/3389220540 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe34031600 con 0x7fbe3c0cea40 2024-04-25T10:08:02.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.764+0000 7fbe3a000640 1 -- 172.21.15.119:0/3389220540 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe340319d0 con 0x7fbe3c0cea40 2024-04-25T10:08:02.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.765+0000 7fbe3be00640 1 -- 172.21.15.119:0/3389220540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0cea40 msgr2=0x7fbe3c06df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.765+0000 7fbe3be00640 1 --2- 172.21.15.119:0/3389220540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0cea40 0x7fbe3c06df50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0cf580 tx=0x7fbe3402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.765+0000 7fbe3be00640 1 -- 172.21.15.119:0/3389220540 shutdown_connections 2024-04-25T10:08:02.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.765+0000 7fbe3be00640 1 --2- 172.21.15.119:0/3389220540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0cea40 0x7fbe3c06df50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.765+0000 7fbe3be00640 1 --2- 172.21.15.119:0/3389220540 >> v2:172.21.15.132:3300/0 conn(0x7fbe3c15bcb0 0x7fbe3c06da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.765+0000 7fbe3be00640 1 -- 172.21.15.119:0/3389220540 >> 172.21.15.119:0/3389220540 conn(0x7fbe3c06aba0 msgr2=0x7fbe3c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.765+0000 7fbe3be00640 1 -- 172.21.15.119:0/3389220540 shutdown_connections 2024-04-25T10:08:02.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.765+0000 7fbe3be00640 1 -- 172.21.15.119:0/3389220540 wait complete. 2024-04-25T10:08:02.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.767+0000 7fbe3be00640 1 Processor -- start 2024-04-25T10:08:02.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.767+0000 7fbe3be00640 1 -- start start 2024-04-25T10:08:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.768+0000 7fbe3be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe3c0c6150 0x7fbe3c156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.768+0000 7fbe3be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0cea40 0x7fbe3c143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.768+0000 7fbe3be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0b3c30 0x7fbe3c154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.768+0000 7fbe3be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0c6530 con 0x7fbe3c0b3c30 2024-04-25T10:08:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.768+0000 7fbe3be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c06e490 con 0x7fbe3c0c6150 2024-04-25T10:08:02.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.768+0000 7fbe3be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c15f2f0 con 0x7fbe3c0cea40 2024-04-25T10:08:02.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.768+0000 7fbe3b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0b3c30 0x7fbe3c154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.768+0000 7fbe3b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0b3c30 0x7fbe3c154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45248/0 (socket says 172.21.15.119:45248) 2024-04-25T10:08:02.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.768+0000 7fbe3b400640 1 -- 172.21.15.119:0/3972648334 learned_addr learned my addr 172.21.15.119:0/3972648334 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:02.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.769+0000 7fbe3b400640 1 -- 172.21.15.119:0/3972648334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0cea40 msgr2=0x7fbe3c143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.769+0000 7fbe32000640 1 --2- 172.21.15.119:0/3972648334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0cea40 0x7fbe3c143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.769+0000 7fbe3aa00640 1 --2- 172.21.15.119:0/3972648334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe3c0c6150 0x7fbe3c156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.769+0000 7fbe3b400640 1 --2- 172.21.15.119:0/3972648334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0cea40 0x7fbe3c143fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.769+0000 7fbe3b400640 1 -- 172.21.15.119:0/3972648334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe3c0c6150 msgr2=0x7fbe3c156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.769+0000 7fbe3b400640 1 --2- 172.21.15.119:0/3972648334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe3c0c6150 0x7fbe3c156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.769+0000 7fbe3b400640 1 -- 172.21.15.119:0/3972648334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c0b40e0 con 0x7fbe3c0b3c30 2024-04-25T10:08:02.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.769+0000 7fbe3b400640 1 --2- 172.21.15.119:0/3972648334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0b3c30 0x7fbe3c154fd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009f40 tx=0x7fbe34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.769+0000 7fbe33e00640 1 -- 172.21.15.119:0/3972648334 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34036050 con 0x7fbe3c0b3c30 2024-04-25T10:08:02.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.769+0000 7fbe33e00640 1 -- 172.21.15.119:0/3972648334 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe34042070 con 0x7fbe3c0b3c30 2024-04-25T10:08:02.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.769+0000 7fbe33e00640 1 -- 172.21.15.119:0/3972648334 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34009510 con 0x7fbe3c0b3c30 2024-04-25T10:08:02.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.769+0000 7fbe3be00640 1 -- 172.21.15.119:0/3972648334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c1f72e0 con 0x7fbe3c0b3c30 2024-04-25T10:08:02.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.770+0000 7fbe3be00640 1 -- 172.21.15.119:0/3972648334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c1f78a0 con 0x7fbe3c0b3c30 2024-04-25T10:08:02.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.771+0000 7fbe3be00640 1 -- 172.21.15.119:0/3972648334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3c0749b0 con 0x7fbe3c0b3c30 2024-04-25T10:08:02.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.772+0000 7fbe33e00640 1 -- 172.21.15.119:0/3972648334 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe34007bd0 con 0x7fbe3c0b3c30 2024-04-25T10:08:02.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.773+0000 7fbe33e00640 1 --2- 172.21.15.119:0/3972648334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe1806b4d0 0x7fbe1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.773+0000 7fbe33e00640 1 -- 172.21.15.119:0/3972648334 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5890+0+0 (secure 0 0 0) 0x7fbe340ab360 con 0x7fbe3c0b3c30 2024-04-25T10:08:02.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.773+0000 7fbe3aa00640 1 --2- 172.21.15.119:0/3972648334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe1806b4d0 0x7fbe1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.774+0000 7fbe3aa00640 1 --2- 172.21.15.119:0/3972648334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe1806b4d0 0x7fbe1806d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0015d0 tx=0x7fbe2c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.777+0000 7fbe33e00640 1 -- 172.21.15.119:0/3972648334 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe3407a080 con 0x7fbe3c0b3c30 2024-04-25T10:08:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:02.994+0000 7fbe3be00640 1 -- 172.21.15.119:0/3972648334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbe3c0cd260 con 0x7fbe3c0b3c30 2024-04-25T10:08:03.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.564+0000 7fbe33e00640 1 -- 172.21.15.119:0/3972648334 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbe34079980 con 0x7fbe3c0b3c30 2024-04-25T10:08:03.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.568+0000 7fbe3be00640 1 -- 172.21.15.119:0/3972648334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe1806b4d0 msgr2=0x7fbe1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.568+0000 7fbe3be00640 1 --2- 172.21.15.119:0/3972648334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe1806b4d0 0x7fbe1806d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0015d0 tx=0x7fbe2c009040 comp rx=0 tx=0).stop 2024-04-25T10:08:03.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.568+0000 7fbe3be00640 1 -- 172.21.15.119:0/3972648334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0b3c30 msgr2=0x7fbe3c154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.568+0000 7fbe3be00640 1 --2- 172.21.15.119:0/3972648334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0b3c30 0x7fbe3c154fd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009f40 tx=0x7fbe34004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:03.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.568+0000 7fbe3be00640 1 -- 172.21.15.119:0/3972648334 shutdown_connections 2024-04-25T10:08:03.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.568+0000 7fbe3be00640 1 --2- 172.21.15.119:0/3972648334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe1806b4d0 0x7fbe1806d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.568+0000 7fbe3be00640 1 --2- 172.21.15.119:0/3972648334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0b3c30 0x7fbe3c154fd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.568+0000 7fbe3be00640 1 --2- 172.21.15.119:0/3972648334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe3c0cea40 0x7fbe3c143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.568+0000 7fbe3be00640 1 --2- 172.21.15.119:0/3972648334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe3c0c6150 0x7fbe3c156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.568+0000 7fbe3be00640 1 -- 172.21.15.119:0/3972648334 >> 172.21.15.119:0/3972648334 conn(0x7fbe3c06aba0 msgr2=0x7fbe3c15eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.568+0000 7fbe3be00640 1 -- 172.21.15.119:0/3972648334 shutdown_connections 2024-04-25T10:08:03.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.568+0000 7fbe3be00640 1 -- 172.21.15.119:0/3972648334 wait complete. 2024-04-25T10:08:03.595 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:08:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.831+0000 7f3994000640 1 Processor -- start 2024-04-25T10:08:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.831+0000 7f3994000640 1 -- start start 2024-04-25T10:08:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.832+0000 7f3994000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f398c0aedc0 0x7f398c15ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.832+0000 7f3994000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c06da40 con 0x7f398c0b4900 2024-04-25T10:08:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.832+0000 7f3994000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c06dd20 con 0x7f398c06d260 2024-04-25T10:08:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.832+0000 7f3994000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c15c050 con 0x7f398c0aedc0 2024-04-25T10:08:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.832+0000 7f3991600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f398c0aedc0 0x7f398c15ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.832+0000 7f3992000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f398c0b4900 0x7f398c15c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60960/0 (socket says 172.21.15.119:60960) 2024-04-25T10:08:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.833+0000 7f3991600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f398c0aedc0 0x7f398c15ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45252/0 (socket says 172.21.15.119:45252) 2024-04-25T10:08:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.833+0000 7f3992000640 1 -- 172.21.15.119:0/4057331059 learned_addr learned my addr 172.21.15.119:0/4057331059 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.833+0000 7f3991600640 1 -- 172.21.15.119:0/4057331059 >> v1:172.21.15.132:6789/0 conn(0x7f398c06d260 legacy=0x7f398c0b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.833+0000 7f3991600640 1 -- 172.21.15.119:0/4057331059 >> v1:172.21.15.119:6789/0 conn(0x7f398c0b4900 legacy=0x7f398c15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.833+0000 7f3991600640 1 -- 172.21.15.119:0/4057331059 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c06e000 con 0x7f398c0aedc0 2024-04-25T10:08:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.834+0000 7f3991600640 1 --2- 172.21.15.119:0/4057331059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f398c0aedc0 0x7f398c15ae20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f398c15c1d0 tx=0x7f398002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b985a8ae05f697c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.834+0000 7f398be00640 1 -- 172.21.15.119:0/4057331059 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980037070 con 0x7f398c0aedc0 2024-04-25T10:08:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.834+0000 7f398be00640 1 -- 172.21.15.119:0/4057331059 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398002ae40 con 0x7f398c0aedc0 2024-04-25T10:08:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.834+0000 7f398be00640 1 -- 172.21.15.119:0/4057331059 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980032660 con 0x7f398c0aedc0 2024-04-25T10:08:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.835+0000 7f3994000640 1 -- 172.21.15.119:0/4057331059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f398c0aedc0 msgr2=0x7f398c15ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.835+0000 7f3994000640 1 --2- 172.21.15.119:0/4057331059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f398c0aedc0 0x7f398c15ae20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f398c15c1d0 tx=0x7f398002a130 comp rx=0 tx=0).stop 2024-04-25T10:08:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.835+0000 7f3994000640 1 -- 172.21.15.119:0/4057331059 shutdown_connections 2024-04-25T10:08:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.835+0000 7f3994000640 1 --2- 172.21.15.119:0/4057331059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f398c0aedc0 0x7f398c15ae20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.835+0000 7f3994000640 1 -- 172.21.15.119:0/4057331059 >> 172.21.15.119:0/4057331059 conn(0x7f398c06aba0 msgr2=0x7f398c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.835+0000 7f3994000640 1 -- 172.21.15.119:0/4057331059 shutdown_connections 2024-04-25T10:08:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.835+0000 7f3994000640 1 -- 172.21.15.119:0/4057331059 wait complete. 2024-04-25T10:08:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.837+0000 7f3994000640 1 Processor -- start 2024-04-25T10:08:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.837+0000 7f3994000640 1 -- start start 2024-04-25T10:08:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.838+0000 7f3994000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f398c06d260 0x7f398c158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.838+0000 7f3994000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f398c0aedc0 0x7f398c0ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.838+0000 7f3994000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f398c0b4900 0x7f398c0adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.838+0000 7f3994000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c06d6f0 con 0x7f398c0b4900 2024-04-25T10:08:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.838+0000 7f3994000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0af1a0 con 0x7f398c0aedc0 2024-04-25T10:08:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.838+0000 7f3994000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c15ced0 con 0x7f398c06d260 2024-04-25T10:08:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.838+0000 7f3992000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f398c0b4900 0x7f398c0adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.838+0000 7f3992000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f398c0b4900 0x7f398c0adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45256/0 (socket says 172.21.15.119:45256) 2024-04-25T10:08:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.838+0000 7f3992000640 1 -- 172.21.15.119:0/2239326788 learned_addr learned my addr 172.21.15.119:0/2239326788 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.839+0000 7f3990c00640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f398c0aedc0 0x7f398c0ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.839+0000 7f3991600640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f398c06d260 0x7f398c158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.839+0000 7f3992000640 1 -- 172.21.15.119:0/2239326788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f398c06d260 msgr2=0x7f398c158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.839+0000 7f3992000640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f398c06d260 0x7f398c158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.839+0000 7f3992000640 1 -- 172.21.15.119:0/2239326788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f398c0aedc0 msgr2=0x7f398c0ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.839+0000 7f3992000640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f398c0aedc0 0x7f398c0ad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.839+0000 7f3992000640 1 -- 172.21.15.119:0/2239326788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c0c4660 con 0x7f398c0b4900 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.839+0000 7f3990c00640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f398c0aedc0 0x7f398c0ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.839+0000 7f3992000640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f398c0b4900 0x7f398c0adb80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f398400ca80 tx=0x7f398400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.839+0000 7f3989600640 1 -- 172.21.15.119:0/2239326788 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984017070 con 0x7f398c0b4900 2024-04-25T10:08:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.839+0000 7f3989600640 1 -- 172.21.15.119:0/2239326788 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398400ac30 con 0x7f398c0b4900 2024-04-25T10:08:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.839+0000 7f3994000640 1 -- 172.21.15.119:0/2239326788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c0ab450 con 0x7f398c0b4900 2024-04-25T10:08:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.839+0000 7f3989600640 1 -- 172.21.15.119:0/2239326788 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984012400 con 0x7f398c0b4900 2024-04-25T10:08:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.839+0000 7f3994000640 1 -- 172.21.15.119:0/2239326788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c0ae490 con 0x7f398c0b4900 2024-04-25T10:08:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.840+0000 7f3994000640 1 -- 172.21.15.119:0/2239326788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c0750b0 con 0x7f398c0b4900 2024-04-25T10:08:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.841+0000 7f3989600640 1 -- 172.21.15.119:0/2239326788 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f398401a070 con 0x7f398c0b4900 2024-04-25T10:08:03.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.842+0000 7f3989600640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f396c06b4d0 0x7f396c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.842+0000 7f3989600640 1 -- 172.21.15.119:0/2239326788 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f3984089eb0 con 0x7f398c0b4900 2024-04-25T10:08:03.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.843+0000 7f3991600640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f396c06b4d0 0x7f396c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.843+0000 7f3991600640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f396c06b4d0 0x7f396c06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3980034000 tx=0x7f398002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:03.847+0000 7f3989600640 1 -- 172.21.15.119:0/2239326788 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3984058580 con 0x7f398c0b4900 2024-04-25T10:08:04.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:04.028+0000 7f3994000640 1 -- 172.21.15.119:0/2239326788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f398c158580 con 0x7f398c0b4900 2024-04-25T10:08:04.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:04.985+0000 7f3989600640 1 -- 172.21.15.119:0/2239326788 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f398405c620 con 0x7f398c0b4900 2024-04-25T10:08:04.991 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:08:04.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:04.989+0000 7f3994000640 1 -- 172.21.15.119:0/2239326788 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f396c06b4d0 msgr2=0x7f396c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:04.989+0000 7f3994000640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f396c06b4d0 0x7f396c06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3980034000 tx=0x7f398002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:04.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:04.989+0000 7f3994000640 1 -- 172.21.15.119:0/2239326788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f398c0b4900 msgr2=0x7f398c0adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:04.989+0000 7f3994000640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f398c0b4900 0x7f398c0adb80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f398400ca80 tx=0x7f398400cf40 comp rx=0 tx=0).stop 2024-04-25T10:08:04.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:04.990+0000 7f3994000640 1 -- 172.21.15.119:0/2239326788 shutdown_connections 2024-04-25T10:08:04.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:04.990+0000 7f3994000640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f396c06b4d0 0x7f396c06d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:04.990+0000 7f3994000640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f398c0b4900 0x7f398c0adb80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:04.990+0000 7f3994000640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f398c0aedc0 0x7f398c0ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:04.990+0000 7f3994000640 1 --2- 172.21.15.119:0/2239326788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f398c06d260 0x7f398c158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:04.990+0000 7f3994000640 1 -- 172.21.15.119:0/2239326788 >> 172.21.15.119:0/2239326788 conn(0x7f398c06aba0 msgr2=0x7f398c15c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:04.990+0000 7f3994000640 1 -- 172.21.15.119:0/2239326788 shutdown_connections 2024-04-25T10:08:04.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:04.990+0000 7f3994000640 1 -- 172.21.15.119:0/2239326788 wait complete. 2024-04-25T10:08:05.018 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:08:05.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.210+0000 7f5618600640 1 Processor -- start 2024-04-25T10:08:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.211+0000 7f5618600640 1 -- start start 2024-04-25T10:08:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.212+0000 7f5618600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561015ae20 0x7f56100b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.212+0000 7f5618600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f56100b4730 0x7f561015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.212+0000 7f5618600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561006da40 con 0x7f56100aedc0 2024-04-25T10:08:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.212+0000 7f5618600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f561015b280 con 0x7f56100b4730 2024-04-25T10:08:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.212+0000 7f5618600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561015c050 con 0x7f561015ae20 2024-04-25T10:08:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.213+0000 7f5616600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f56100b4730 0x7f561015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.213+0000 7f5616600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f56100b4730 0x7f561015c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45268/0 (socket says 172.21.15.119:45268) 2024-04-25T10:08:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.213+0000 7f5616600640 1 -- 172.21.15.119:0/910487769 learned_addr learned my addr 172.21.15.119:0/910487769 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.213+0000 7f5615c00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f56100aedc0 0x7f561006d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60968/0 (socket says 172.21.15.119:60968) 2024-04-25T10:08:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.213+0000 7f5615200640 1 --2- 172.21.15.119:0/910487769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561015ae20 0x7f56100b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.213+0000 7f5616600640 1 -- 172.21.15.119:0/910487769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561015ae20 msgr2=0x7f56100b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.213+0000 7f5616600640 1 --2- 172.21.15.119:0/910487769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561015ae20 0x7f56100b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.213+0000 7f5616600640 1 -- 172.21.15.119:0/910487769 >> v1:172.21.15.119:6789/0 conn(0x7f56100aedc0 legacy=0x7f561006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.213+0000 7f5616600640 1 -- 172.21.15.119:0/910487769 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561006dd20 con 0x7f56100b4730 2024-04-25T10:08:05.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.214+0000 7f5616600640 1 --2- 172.21.15.119:0/910487769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100b4730 0x7f561015c7c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5608009e50 tx=0x7f560802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c386be79786e94ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.214+0000 7f560fe00640 1 -- 172.21.15.119:0/910487769 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608036070 con 0x7f56100b4730 2024-04-25T10:08:05.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.215+0000 7f560fe00640 1 -- 172.21.15.119:0/910487769 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5608031600 con 0x7f56100b4730 2024-04-25T10:08:05.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.215+0000 7f560fe00640 1 -- 172.21.15.119:0/910487769 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56080319d0 con 0x7f56100b4730 2024-04-25T10:08:05.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.215+0000 7f5618600640 1 -- 172.21.15.119:0/910487769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100b4730 msgr2=0x7f561015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.215+0000 7f5618600640 1 --2- 172.21.15.119:0/910487769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100b4730 0x7f561015c7c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5608009e50 tx=0x7f560802a680 comp rx=0 tx=0).stop 2024-04-25T10:08:05.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.215+0000 7f5618600640 1 -- 172.21.15.119:0/910487769 shutdown_connections 2024-04-25T10:08:05.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.215+0000 7f5618600640 1 --2- 172.21.15.119:0/910487769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100b4730 0x7f561015c7c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.215+0000 7f5618600640 1 --2- 172.21.15.119:0/910487769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561015ae20 0x7f56100b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.215+0000 7f5618600640 1 -- 172.21.15.119:0/910487769 >> 172.21.15.119:0/910487769 conn(0x7f561006aba0 msgr2=0x7f561006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.215+0000 7f5618600640 1 -- 172.21.15.119:0/910487769 shutdown_connections 2024-04-25T10:08:05.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.215+0000 7f5618600640 1 -- 172.21.15.119:0/910487769 wait complete. 2024-04-25T10:08:05.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.217+0000 7f5618600640 1 Processor -- start 2024-04-25T10:08:05.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.217+0000 7f5618600640 1 -- start start 2024-04-25T10:08:05.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.218+0000 7f5618600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100aedc0 0x7f561013f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.218+0000 7f5618600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56100b4730 0x7f5610158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.218+0000 7f5618600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f561015ae20 0x7f5610159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.218+0000 7f5618600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561015c2a0 con 0x7f56100aedc0 2024-04-25T10:08:05.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.218+0000 7f5618600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56100af1a0 con 0x7f561015ae20 2024-04-25T10:08:05.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.218+0000 7f5618600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561015cd00 con 0x7f56100b4730 2024-04-25T10:08:05.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.218+0000 7f5615c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100aedc0 0x7f561013f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.218+0000 7f5615c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100aedc0 0x7f561013f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45278/0 (socket says 172.21.15.119:45278) 2024-04-25T10:08:05.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.218+0000 7f5615c00640 1 -- 172.21.15.119:0/2779437980 learned_addr learned my addr 172.21.15.119:0/2779437980 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:05.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.219+0000 7f5616600640 1 --2- 172.21.15.119:0/2779437980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f561015ae20 0x7f5610159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.219+0000 7f5615c00640 1 -- 172.21.15.119:0/2779437980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56100b4730 msgr2=0x7f5610158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.219+0000 7f5615c00640 1 --2- 172.21.15.119:0/2779437980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56100b4730 0x7f5610158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.219+0000 7f5615c00640 1 -- 172.21.15.119:0/2779437980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f561015ae20 msgr2=0x7f5610159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.219+0000 7f5615c00640 1 --2- 172.21.15.119:0/2779437980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f561015ae20 0x7f5610159230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.219+0000 7f5615c00640 1 -- 172.21.15.119:0/2779437980 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561013fac0 con 0x7f56100aedc0 2024-04-25T10:08:05.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.219+0000 7f5615c00640 1 --2- 172.21.15.119:0/2779437980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100aedc0 0x7f561013f090 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f560400c400 tx=0x7f560400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.219+0000 7f560d600640 1 -- 172.21.15.119:0/2779437980 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56040074e0 con 0x7f56100aedc0 2024-04-25T10:08:05.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.219+0000 7f560d600640 1 -- 172.21.15.119:0/2779437980 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5604007680 con 0x7f56100aedc0 2024-04-25T10:08:05.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.219+0000 7f560d600640 1 -- 172.21.15.119:0/2779437980 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604004a20 con 0x7f56100aedc0 2024-04-25T10:08:05.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.219+0000 7f5618600640 1 -- 172.21.15.119:0/2779437980 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561013fda0 con 0x7f56100aedc0 2024-04-25T10:08:05.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.220+0000 7f5618600640 1 -- 172.21.15.119:0/2779437980 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561013f5d0 con 0x7f56100aedc0 2024-04-25T10:08:05.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.221+0000 7f5618600640 1 -- 172.21.15.119:0/2779437980 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56100cc8a0 con 0x7f56100aedc0 2024-04-25T10:08:05.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.222+0000 7f560d600640 1 -- 172.21.15.119:0/2779437980 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f560401a020 con 0x7f56100aedc0 2024-04-25T10:08:05.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.223+0000 7f560d600640 1 --2- 172.21.15.119:0/2779437980 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55f406b500 0x7f55f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.223+0000 7f560d600640 1 -- 172.21.15.119:0/2779437980 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5452+0+0 (secure 0 0 0) 0x7f5604089890 con 0x7f56100aedc0 2024-04-25T10:08:05.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.223+0000 7f5615200640 1 --2- 172.21.15.119:0/2779437980 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55f406b500 0x7f55f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.224+0000 7f5615200640 1 --2- 172.21.15.119:0/2779437980 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55f406b500 0x7f55f406d9c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5600006fd0 tx=0x7f5600008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.227+0000 7f560d600640 1 -- 172.21.15.119:0/2779437980 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56100cc8a0 con 0x7f56100aedc0 2024-04-25T10:08:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.423+0000 7f5618600640 1 -- 172.21.15.119:0/2779437980 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f561015c7c0 con 0x7f56100aedc0 2024-04-25T10:08:06.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:05.995+0000 7f560d600640 1 -- 172.21.15.119:0/2779437980 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f561015c7c0 con 0x7f56100aedc0 2024-04-25T10:08:06.001 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2024-04-25T10:08:06.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.001+0000 7f5618600640 1 -- 172.21.15.119:0/2779437980 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55f406b500 msgr2=0x7f55f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.001+0000 7f5618600640 1 --2- 172.21.15.119:0/2779437980 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55f406b500 0x7f55f406d9c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5600006fd0 tx=0x7f5600008040 comp rx=0 tx=0).stop 2024-04-25T10:08:06.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.001+0000 7f5618600640 1 -- 172.21.15.119:0/2779437980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100aedc0 msgr2=0x7f561013f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.001+0000 7f5618600640 1 --2- 172.21.15.119:0/2779437980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100aedc0 0x7f561013f090 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f560400c400 tx=0x7f560400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.003+0000 7f5618600640 1 -- 172.21.15.119:0/2779437980 shutdown_connections 2024-04-25T10:08:06.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.003+0000 7f5618600640 1 --2- 172.21.15.119:0/2779437980 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55f406b500 0x7f55f406d9c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.003+0000 7f5618600640 1 --2- 172.21.15.119:0/2779437980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f561015ae20 0x7f5610159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.003+0000 7f5618600640 1 --2- 172.21.15.119:0/2779437980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56100b4730 0x7f5610158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.003+0000 7f5618600640 1 --2- 172.21.15.119:0/2779437980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56100aedc0 0x7f561013f090 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.003+0000 7f5618600640 1 -- 172.21.15.119:0/2779437980 >> 172.21.15.119:0/2779437980 conn(0x7f561006aba0 msgr2=0x7f56100ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:06.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.004+0000 7f5618600640 1 -- 172.21.15.119:0/2779437980 shutdown_connections 2024-04-25T10:08:06.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.004+0000 7f5618600640 1 -- 172.21.15.119:0/2779437980 wait complete. 2024-04-25T10:08:06.039 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:08:06.040 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:08:06.040 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:08:06.040 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:06.040 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:08:06.040 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:08:06.040 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:08:06.041 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:06.041 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:08:06.041 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:08:06.041 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:08:06.041 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:06.041 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:08:06.041 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:08:06.042 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:08:06.042 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:06.060 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:08:06.060 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:08:06.061 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:08:06.061 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:08:06.062 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:08:06.062 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:08:06.063 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:08:06.063 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:08:06.105 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:08:06.105 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:06.107 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:08:06.108 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:06.108 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:08:06.108 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:06.112 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:08:06.112 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:06.112 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:08:06.246 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:08:06.247 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2024-04-25T10:08:06.249 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-04-25T10:08:06.249 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-04-25T10:08:06.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.283+0000 7f8249e00640 1 Processor -- start 2024-04-25T10:08:06.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.283+0000 7f8249e00640 1 -- start start 2024-04-25T10:08:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.284+0000 7f8249e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f824415ae20 0x7f82440b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.284+0000 7f8249e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82440b4730 0x7f824415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.284+0000 7f8249e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f824406da40 con 0x7f82440aedc0 2024-04-25T10:08:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.284+0000 7f8249e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f824415c050 con 0x7f824415ae20 2024-04-25T10:08:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.284+0000 7f8249e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82440af1a0 con 0x7f82440b4730 2024-04-25T10:08:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.285+0000 7f8242000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f824415ae20 0x7f82440b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.285+0000 7f8242000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f824415ae20 0x7f82440b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45320/0 (socket says 172.21.15.119:45320) 2024-04-25T10:08:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.285+0000 7f8242000640 1 -- 172.21.15.119:0/2476727263 learned_addr learned my addr 172.21.15.119:0/2476727263 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.285+0000 7f8243400640 1 --2- 172.21.15.119:0/2476727263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82440b4730 0x7f824415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.285+0000 7f8242000640 1 -- 172.21.15.119:0/2476727263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82440b4730 msgr2=0x7f824415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:06.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.285+0000 7f8242000640 1 --2- 172.21.15.119:0/2476727263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82440b4730 0x7f824415c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.285+0000 7f8242000640 1 -- 172.21.15.119:0/2476727263 >> v1:172.21.15.119:6789/0 conn(0x7f82440aedc0 legacy=0x7f824406d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.285+0000 7f8242000640 1 -- 172.21.15.119:0/2476727263 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824406dd20 con 0x7f824415ae20 2024-04-25T10:08:06.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.285+0000 7f8241600640 1 -- 172.21.15.119:0/2476727263 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120356511 0 0) 0x7f824406da40 con 0x7f82440aedc0 2024-04-25T10:08:06.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.286+0000 7f8242000640 1 --2- 172.21.15.119:0/2476727263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f824415ae20 0x7f82440b41f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8230009870 tx=0x7f823002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07033644abdb8aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:06.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.287+0000 7f8241600640 1 -- 172.21.15.119:0/2476727263 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230037070 con 0x7f824415ae20 2024-04-25T10:08:06.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.287+0000 7f8241600640 1 -- 172.21.15.119:0/2476727263 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8230032440 con 0x7f824415ae20 2024-04-25T10:08:06.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.287+0000 7f8241600640 1 -- 172.21.15.119:0/2476727263 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230032810 con 0x7f824415ae20 2024-04-25T10:08:06.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.287+0000 7f8249e00640 1 -- 172.21.15.119:0/2476727263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f824415ae20 msgr2=0x7f82440b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.287+0000 7f8249e00640 1 --2- 172.21.15.119:0/2476727263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f824415ae20 0x7f82440b41f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8230009870 tx=0x7f823002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.287+0000 7f8249e00640 1 -- 172.21.15.119:0/2476727263 shutdown_connections 2024-04-25T10:08:06.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.287+0000 7f8249e00640 1 --2- 172.21.15.119:0/2476727263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82440b4730 0x7f824415c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.287+0000 7f8249e00640 1 --2- 172.21.15.119:0/2476727263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f824415ae20 0x7f82440b41f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.287+0000 7f8249e00640 1 -- 172.21.15.119:0/2476727263 >> 172.21.15.119:0/2476727263 conn(0x7f824406aba0 msgr2=0x7f824406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:06.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.288+0000 7f8249e00640 1 -- 172.21.15.119:0/2476727263 shutdown_connections 2024-04-25T10:08:06.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.288+0000 7f8249e00640 1 -- 172.21.15.119:0/2476727263 wait complete. 2024-04-25T10:08:06.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.289+0000 7f8249e00640 1 Processor -- start 2024-04-25T10:08:06.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.289+0000 7f8249e00640 1 -- start start 2024-04-25T10:08:06.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.290+0000 7f8249e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82440aedc0 0x7f8244144570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.290+0000 7f8249e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82440b4730 0x7f82441477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.290+0000 7f8249e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f824415ae20 0x7f8244157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.290+0000 7f8249e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824415cd00 con 0x7f82440aedc0 2024-04-25T10:08:06.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.290+0000 7f8249e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824415c2a0 con 0x7f82440b4730 2024-04-25T10:08:06.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.290+0000 7f8249e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824415ce80 con 0x7f824415ae20 2024-04-25T10:08:06.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.291+0000 7f8242a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82440aedc0 0x7f8244144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.291+0000 7f8242a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82440aedc0 0x7f8244144570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45322/0 (socket says 172.21.15.119:45322) 2024-04-25T10:08:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.291+0000 7f8242a00640 1 -- 172.21.15.119:0/3228731412 learned_addr learned my addr 172.21.15.119:0/3228731412 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.291+0000 7f8242000640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82440b4730 0x7f82441477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.291+0000 7f8243400640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f824415ae20 0x7f8244157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.291+0000 7f8242a00640 1 -- 172.21.15.119:0/3228731412 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f824415ae20 msgr2=0x7f8244157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.291+0000 7f8242a00640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f824415ae20 0x7f8244157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.291+0000 7f8242a00640 1 -- 172.21.15.119:0/3228731412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82440b4730 msgr2=0x7f82441477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.291+0000 7f8242a00640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82440b4730 0x7f82441477d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.291+0000 7f8242a00640 1 -- 172.21.15.119:0/3228731412 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8244147d10 con 0x7f82440aedc0 2024-04-25T10:08:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.291+0000 7f8243400640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f824415ae20 0x7f8244157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.292+0000 7f8242a00640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82440aedc0 0x7f8244144570 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f823400c520 tx=0x7f823400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.292+0000 7f823b400640 1 -- 172.21.15.119:0/3228731412 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234017070 con 0x7f82440aedc0 2024-04-25T10:08:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.292+0000 7f823b400640 1 -- 172.21.15.119:0/3228731412 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82340052d0 con 0x7f82440aedc0 2024-04-25T10:08:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.292+0000 7f823b400640 1 -- 172.21.15.119:0/3228731412 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234012870 con 0x7f82440aedc0 2024-04-25T10:08:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.292+0000 7f8249e00640 1 -- 172.21.15.119:0/3228731412 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82440ad190 con 0x7f82440aedc0 2024-04-25T10:08:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.292+0000 7f8249e00640 1 -- 172.21.15.119:0/3228731412 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82440ad750 con 0x7f82440aedc0 2024-04-25T10:08:06.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.293+0000 7f8249e00640 1 -- 172.21.15.119:0/3228731412 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82440750b0 con 0x7f82440aedc0 2024-04-25T10:08:06.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.294+0000 7f823b400640 1 -- 172.21.15.119:0/3228731412 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f823401a070 con 0x7f82440aedc0 2024-04-25T10:08:06.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.295+0000 7f823b400640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f822406b550 0x7f822406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.295+0000 7f823b400640 1 -- 172.21.15.119:0/3228731412 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8234089210 con 0x7f82440aedc0 2024-04-25T10:08:06.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.295+0000 7f8242000640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f822406b550 0x7f822406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.296+0000 7f8242000640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f822406b550 0x7f822406da10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8230034000 tx=0x7f823002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:06.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.299+0000 7f823b400640 1 -- 172.21.15.119:0/3228731412 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8234057d30 con 0x7f82440aedc0 2024-04-25T10:08:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:06.507+0000 7f8249e00640 1 -- 172.21.15.119:0/3228731412 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8244159d60 con 0x7f82440aedc0 2024-04-25T10:08:07.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.018+0000 7f823b400640 1 -- 172.21.15.119:0/3228731412 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7f823405bdd0 con 0x7f82440aedc0 2024-04-25T10:08:07.024 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2024-04-25T10:08:07.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.022+0000 7f8249e00640 1 -- 172.21.15.119:0/3228731412 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f822406b550 msgr2=0x7f822406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.022+0000 7f8249e00640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f822406b550 0x7f822406da10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8230034000 tx=0x7f823002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:07.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.022+0000 7f8249e00640 1 -- 172.21.15.119:0/3228731412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82440aedc0 msgr2=0x7f8244144570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.022+0000 7f8249e00640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82440aedc0 0x7f8244144570 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f823400c520 tx=0x7f823400c9e0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.022+0000 7f8249e00640 1 -- 172.21.15.119:0/3228731412 shutdown_connections 2024-04-25T10:08:07.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.022+0000 7f8249e00640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f822406b550 0x7f822406da10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.022+0000 7f8249e00640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f824415ae20 0x7f8244157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.022+0000 7f8249e00640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82440b4730 0x7f82441477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.022+0000 7f8249e00640 1 --2- 172.21.15.119:0/3228731412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82440aedc0 0x7f8244144570 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.022+0000 7f8249e00640 1 -- 172.21.15.119:0/3228731412 >> 172.21.15.119:0/3228731412 conn(0x7f824406aba0 msgr2=0x7f8244159650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:07.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.022+0000 7f8249e00640 1 -- 172.21.15.119:0/3228731412 shutdown_connections 2024-04-25T10:08:07.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.023+0000 7f8249e00640 1 -- 172.21.15.119:0/3228731412 wait complete. 2024-04-25T10:08:07.051 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:08:07.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.242+0000 7f3bef000640 1 Processor -- start 2024-04-25T10:08:07.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.242+0000 7f3bef000640 1 -- start start 2024-04-25T10:08:07.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.243+0000 7f3bef000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be80b11e0 0x7f3be815d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.244+0000 7f3bef000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be815d5d0 con 0x7f3be806dba0 2024-04-25T10:08:07.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.244+0000 7f3bef000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be80ca4a0 con 0x7f3be80b15c0 2024-04-25T10:08:07.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.244+0000 7f3bef000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80ca250 con 0x7f3be80b11e0 2024-04-25T10:08:07.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.244+0000 7f3bed000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3be806dba0 0x7f3be80cb260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:32768/0 (socket says 172.21.15.119:32768) 2024-04-25T10:08:07.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.244+0000 7f3bed000640 1 -- 172.21.15.119:0/723885163 learned_addr learned my addr 172.21.15.119:0/723885163 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.244+0000 7f3be7e00640 1 --2- 172.21.15.119:0/723885163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be80b11e0 0x7f3be815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.244+0000 7f3be6a00640 1 -- 172.21.15.119:0/723885163 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931780381 0 0) 0x7f3be815d5d0 con 0x7f3be806dba0 2024-04-25T10:08:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.244+0000 7f3be6a00640 1 -- 172.21.15.119:0/723885163 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bcc003520 con 0x7f3be806dba0 2024-04-25T10:08:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.245+0000 7f3be6a00640 1 -- 172.21.15.119:0/723885163 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 360127616 0 0) 0x7f3bcc003520 con 0x7f3be806dba0 2024-04-25T10:08:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.245+0000 7f3be6a00640 1 -- 172.21.15.119:0/723885163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be80b11e0 msgr2=0x7f3be815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.245+0000 7f3be6a00640 1 --2- 172.21.15.119:0/723885163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be80b11e0 0x7f3be815d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.245+0000 7f3be6a00640 1 -- 172.21.15.119:0/723885163 >> v1:172.21.15.132:6789/0 conn(0x7f3be80b15c0 legacy=0x7f3be806d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.245+0000 7f3be6a00640 1 -- 172.21.15.119:0/723885163 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be80ca780 con 0x7f3be806dba0 2024-04-25T10:08:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.245+0000 7f3be6a00640 1 -- 172.21.15.119:0/723885163 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f3be00019f0 con 0x7f3be806dba0 2024-04-25T10:08:07.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.245+0000 7f3be6a00640 1 -- 172.21.15.119:0/723885163 >> v1:172.21.15.119:6789/0 conn(0x7f3be806dba0 legacy=0x7f3be80cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.245+0000 7f3be6a00640 1 --2- 172.21.15.119:0/723885163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be80b11e0 0x7f3bcc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.245+0000 7f3be6a00640 1 --2- 172.21.15.119:0/723885163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bcc003f60 0x7f3bcc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.245+0000 7f3be6a00640 1 --2- 172.21.15.119:0/723885163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bcc004d40 0x7f3bcc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.245+0000 7f3be6a00640 1 -- 172.21.15.119:0/723885163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80ca250 con 0x7f3bcc003f60 2024-04-25T10:08:07.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.245+0000 7f3be6a00640 1 -- 172.21.15.119:0/723885163 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc00baa0 con 0x7f3bcc004d40 2024-04-25T10:08:07.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.245+0000 7f3be6a00640 1 -- 172.21.15.119:0/723885163 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc00bca0 con 0x7f3be80b11e0 2024-04-25T10:08:07.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.245+0000 7f3be7400640 1 --2- 172.21.15.119:0/723885163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bcc003f60 0x7f3bcc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.246+0000 7f3be7e00640 1 --2- 172.21.15.119:0/723885163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be80b11e0 0x7f3bcc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.246+0000 7f3bed000640 1 --2- 172.21.15.119:0/723885163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bcc004d40 0x7f3bcc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.246+0000 7f3be7400640 1 -- 172.21.15.119:0/723885163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be80b11e0 msgr2=0x7f3bcc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:07.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.246+0000 7f3be7400640 1 --2- 172.21.15.119:0/723885163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be80b11e0 0x7f3bcc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.246+0000 7f3be7400640 1 -- 172.21.15.119:0/723885163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bcc004d40 msgr2=0x7f3bcc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:07.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.246+0000 7f3be7400640 1 --2- 172.21.15.119:0/723885163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bcc004d40 0x7f3bcc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.246+0000 7f3be7400640 1 -- 172.21.15.119:0/723885163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc003520 con 0x7f3bcc003f60 2024-04-25T10:08:07.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.246+0000 7f3be7e00640 1 --2- 172.21.15.119:0/723885163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be80b11e0 0x7f3bcc0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:07.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.246+0000 7f3bed000640 1 --2- 172.21.15.119:0/723885163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bcc004d40 0x7f3bcc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:07.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.247+0000 7f3be7400640 1 --2- 172.21.15.119:0/723885163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bcc003f60 0x7f3bcc004800 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0001750 tx=0x7f3bcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46faa350c63bcb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.247+0000 7f3be6a00640 1 -- 172.21.15.119:0/723885163 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0001370 con 0x7f3bcc003f60 2024-04-25T10:08:07.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.247+0000 7f3be6a00640 1 -- 172.21.15.119:0/723885163 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd00052d0 con 0x7f3bcc003f60 2024-04-25T10:08:07.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.247+0000 7f3be6a00640 1 -- 172.21.15.119:0/723885163 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd002abe0 con 0x7f3bcc003f60 2024-04-25T10:08:07.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.248+0000 7f3bef000640 1 -- 172.21.15.119:0/723885163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bcc003f60 msgr2=0x7f3bcc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.248+0000 7f3bef000640 1 --2- 172.21.15.119:0/723885163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bcc003f60 0x7f3bcc004800 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0001750 tx=0x7f3bcc005830 comp rx=0 tx=0).stop 2024-04-25T10:08:07.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.248+0000 7f3bed000640 1 -- 172.21.15.119:0/723885163 reap_dead start 2024-04-25T10:08:07.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.248+0000 7f3bef000640 1 -- 172.21.15.119:0/723885163 shutdown_connections 2024-04-25T10:08:07.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.248+0000 7f3bef000640 1 -- 172.21.15.119:0/723885163 >> 172.21.15.119:0/723885163 conn(0x7f3be80cf5a0 msgr2=0x7f3be80cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:07.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.248+0000 7f3bef000640 1 -- 172.21.15.119:0/723885163 shutdown_connections 2024-04-25T10:08:07.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.248+0000 7f3bef000640 1 -- 172.21.15.119:0/723885163 wait complete. 2024-04-25T10:08:07.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.250+0000 7f3bef000640 1 Processor -- start 2024-04-25T10:08:07.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.250+0000 7f3bef000640 1 -- start start 2024-04-25T10:08:07.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.251+0000 7f3bef000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bcc003f60 0x7f3be813eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.251+0000 7f3bef000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be81516e0 0x7f3be80cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.251+0000 7f3bef000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be8151b60 0x7f3be80cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.251+0000 7f3bef000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80cad40 con 0x7f3bcc003f60 2024-04-25T10:08:07.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.251+0000 7f3bef000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be806dfd0 con 0x7f3be8151b60 2024-04-25T10:08:07.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.251+0000 7f3bef000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80b1a00 con 0x7f3be81516e0 2024-04-25T10:08:07.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.251+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bcc003f60 0x7f3be813eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.251+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bcc003f60 0x7f3be813eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45346/0 (socket says 172.21.15.119:45346) 2024-04-25T10:08:07.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.251+0000 7f3be7e00640 1 -- 172.21.15.119:0/2764213499 learned_addr learned my addr 172.21.15.119:0/2764213499 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:07.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.251+0000 7f3be7e00640 1 -- 172.21.15.119:0/2764213499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be81516e0 msgr2=0x7f3be80cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.251+0000 7f3be7e00640 1 --2- 172.21.15.119:0/2764213499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be81516e0 0x7f3be80cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.251+0000 7f3be7e00640 1 -- 172.21.15.119:0/2764213499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be8151b60 msgr2=0x7f3be80cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.251+0000 7f3be7e00640 1 --2- 172.21.15.119:0/2764213499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be8151b60 0x7f3be80cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.251+0000 7f3be7e00640 1 -- 172.21.15.119:0/2764213499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be813f0b0 con 0x7f3bcc003f60 2024-04-25T10:08:07.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.252+0000 7f3be7e00640 1 --2- 172.21.15.119:0/2764213499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bcc003f60 0x7f3be813eb70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80108b0 tx=0x7f3bd8010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.252+0000 7f3bdfe00640 1 -- 172.21.15.119:0/2764213499 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd800f040 con 0x7f3bcc003f60 2024-04-25T10:08:07.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.252+0000 7f3bdfe00640 1 -- 172.21.15.119:0/2764213499 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd800eb60 con 0x7f3bcc003f60 2024-04-25T10:08:07.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.252+0000 7f3bdfe00640 1 -- 172.21.15.119:0/2764213499 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd801c400 con 0x7f3bcc003f60 2024-04-25T10:08:07.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.252+0000 7f3bef000640 1 -- 172.21.15.119:0/2764213499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be81422e0 con 0x7f3bcc003f60 2024-04-25T10:08:07.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.252+0000 7f3bef000640 1 -- 172.21.15.119:0/2764213499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be8150c00 con 0x7f3bcc003f60 2024-04-25T10:08:07.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.253+0000 7f3bef000640 1 -- 172.21.15.119:0/2764213499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be80749b0 con 0x7f3bcc003f60 2024-04-25T10:08:07.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.254+0000 7f3bdfe00640 1 -- 172.21.15.119:0/2764213499 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bd801c5a0 con 0x7f3bcc003f60 2024-04-25T10:08:07.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.255+0000 7f3bdfe00640 1 --2- 172.21.15.119:0/2764213499 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bc406b4d0 0x7f3bc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.255+0000 7f3bdfe00640 1 -- 172.21.15.119:0/2764213499 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3bd808a660 con 0x7f3bcc003f60 2024-04-25T10:08:07.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.256+0000 7f3be7400640 1 --2- 172.21.15.119:0/2764213499 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bc406b4d0 0x7f3bc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.256+0000 7f3be7400640 1 --2- 172.21.15.119:0/2764213499 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bc406b4d0 0x7f3bc406d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0032000 tx=0x7f3bd002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.260+0000 7f3bdfe00640 1 -- 172.21.15.119:0/2764213499 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3be813f0b0 con 0x7f3bcc003f60 2024-04-25T10:08:07.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.527+0000 7f3bef000640 1 -- 172.21.15.119:0/2764213499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3be815e670 con 0x7f3bcc003f60 2024-04-25T10:08:07.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.528+0000 7f3bdfe00640 1 -- 172.21.15.119:0/2764213499 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f3bd8018030 con 0x7f3bcc003f60 2024-04-25T10:08:07.535 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:07.535 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:08:07.535 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2024-04-25T10:08:07.535 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.535 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2024-04-25T10:08:07.535 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmFDz9FBAAdC/N+8UnkYkK5wrDlgpkgA==", 2024-04-25T10:08:07.535 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.535 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:07.535 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:08:07.535 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:08:07.535 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.535 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmmh7lLhAAkzsFzTL6Ko/J/0ZJOpvR4A==", 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2024-04-25T10:08:07.536 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmojotIhAAdSIhFOXoYZN+01nbCs3e+Q==", 2024-04-25T10:08:07.537 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.537 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:07.537 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:08:07.537 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:08:07.537 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.537 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.537 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.537 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.537 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2024-04-25T10:08:07.537 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypm1gp7ORAAvtG28IuhGtR1/Vj6JCPq5A==", 2024-04-25T10:08:07.537 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.537 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:07.537 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAPKypmOI9YMRAAmMrg0xRUG+GG+waJ12llJQ==", 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.539 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2024-04-25T10:08:07.539 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQATKypmHV5GDxAAW0XzpAxF7oxgVBKy5yzavw==", 2024-04-25T10:08:07.539 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.539 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.539 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.539 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.539 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.539 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.539 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.539 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2024-04-25T10:08:07.539 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAWKypmXTiRGhAAZxct/Ck3A3DMBjqNW3HXrg==", 2024-04-25T10:08:07.539 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.539 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAZKypmEeGSMhAAwVNTny5OmP/KONYhIXzrrQ==", 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.540 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.541 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2024-04-25T10:08:07.541 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAgKypmh7gPBRAApx1VM3WpYW7hIQzu7+6J7w==", 2024-04-25T10:08:07.541 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.541 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.541 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.541 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.541 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.541 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.541 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.541 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2024-04-25T10:08:07.541 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAjKypmQ2EaEBAAoFYMlnh5/ar0pAlynwjYOA==", 2024-04-25T10:08:07.541 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.542 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.542 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.542 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.542 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.542 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.542 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2024-04-25T10:08:07.542 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAmKypmDATkKRAAsIDViOBeDrd5N91wf8zGLA==", 2024-04-25T10:08:07.542 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.542 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.542 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.542 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.542 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.543 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2024-04-25T10:08:07.543 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAqKypmNskbCBAAQIYhJ9lfHHr9EP/n5QJoRw==", 2024-04-25T10:08:07.543 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.543 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:07.543 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:07.543 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.543 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2024-04-25T10:08:07.543 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA==", 2024-04-25T10:08:07.543 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.544 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:07.544 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:08:07.544 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:08:07.544 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:08:07.544 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.544 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.544 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.544 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2024-04-25T10:08:07.544 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmm0DLGhAApkxEcc1jUU5T/9ErkTkT4A==", 2024-04-25T10:08:07.544 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.544 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:07.544 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:08:07.544 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:08:07.544 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:08:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2024-04-25T10:08:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmMZ89ERAAICfWm/ZgOLQMVSv5xG0Crg==", 2024-04-25T10:08:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:08:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:08:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:08:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.545 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2024-04-25T10:08:07.546 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmrokCIxAAwi8FtvYSVhMguvZ0OyYt+g==", 2024-04-25T10:08:07.546 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.546 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:07.546 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:08:07.546 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:08:07.546 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:08:07.546 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.546 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.546 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.546 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2024-04-25T10:08:07.546 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAKKypmN9WFGRAAi+stjuG2ZDBjafeR9Ygrug==", 2024-04-25T10:08:07.546 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-mds", 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAyKypmuazaMxAAi7IFF+8ljoPBarzYNdJ/mQ==", 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.547 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T10:08:07.548 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAyKypmJ8vaMxAAz9zo3DdImfsP0WXmicTR7g==", 2024-04-25T10:08:07.548 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.548 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T10:08:07.548 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.548 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-osd", 2024-04-25T10:08:07.548 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAyKypmJ+XaMxAAKZeMRqg9D3194Z/7S1bkEQ==", 2024-04-25T10:08:07.548 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.548 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T10:08:07.548 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.548 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.548 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.548 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAyKypmVv7aMxAAGdBm8kI4qvalCzM0HkDBgw==", 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAyKypm9RvbMxAAizYj+1T6MWWfzOW9O0puRQ==", 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T10:08:07.549 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAyKypmlTbbMxAA80ZlH4FpktTU31XY7sO8/Q==", 2024-04-25T10:08:07.550 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.550 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T10:08:07.550 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.550 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2024-04-25T10:08:07.550 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypm/KyKDBAA646fW6qrcXB31usxeIApaA==", 2024-04-25T10:08:07.550 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.550 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:08:07.550 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2024-04-25T10:08:07.550 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.550 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.550 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:07.550 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:07.551 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2024-04-25T10:08:07.551 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmr8SqBBAA8l4CjUEidXz1obWJFM4DHA==", 2024-04-25T10:08:07.551 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:07.551 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:08:07.551 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2024-04-25T10:08:07.551 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:07.551 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.551 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:07.551 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-25T10:08:07.551 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:08:07.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.531+0000 7f3bef000640 1 -- 172.21.15.119:0/2764213499 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bc406b4d0 msgr2=0x7f3bc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.531+0000 7f3bef000640 1 --2- 172.21.15.119:0/2764213499 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bc406b4d0 0x7f3bc406d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0032000 tx=0x7f3bd002c000 comp rx=0 tx=0).stop 2024-04-25T10:08:07.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.531+0000 7f3bef000640 1 -- 172.21.15.119:0/2764213499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bcc003f60 msgr2=0x7f3be813eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.531+0000 7f3bef000640 1 --2- 172.21.15.119:0/2764213499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bcc003f60 0x7f3be813eb70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80108b0 tx=0x7f3bd8010d70 comp rx=0 tx=0).stop 2024-04-25T10:08:07.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.531+0000 7f3bef000640 1 -- 172.21.15.119:0/2764213499 shutdown_connections 2024-04-25T10:08:07.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.531+0000 7f3bef000640 1 --2- 172.21.15.119:0/2764213499 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bc406b4d0 0x7f3bc406d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.531+0000 7f3bef000640 1 --2- 172.21.15.119:0/2764213499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be8151b60 0x7f3be80cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.531+0000 7f3bef000640 1 --2- 172.21.15.119:0/2764213499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be81516e0 0x7f3be80cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.531+0000 7f3bef000640 1 --2- 172.21.15.119:0/2764213499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bcc003f60 0x7f3be813eb70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.531+0000 7f3bef000640 1 -- 172.21.15.119:0/2764213499 >> 172.21.15.119:0/2764213499 conn(0x7f3be80cf5a0 msgr2=0x7f3be815d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:07.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.531+0000 7f3bef000640 1 -- 172.21.15.119:0/2764213499 shutdown_connections 2024-04-25T10:08:07.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.531+0000 7f3bef000640 1 -- 172.21.15.119:0/2764213499 wait complete. 2024-04-25T10:08:07.557 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T10:08:07.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.746+0000 7f3f0be00640 1 Processor -- start 2024-04-25T10:08:07.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.746+0000 7f3f0be00640 1 -- start start 2024-04-25T10:08:07.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.748+0000 7f3f0be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f0c1471b0 0x7f3f0c06dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.748+0000 7f3f0be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3f0c15df20 0x7f3f0c15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.748+0000 7f3f0be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3f0c0710b0 0x7f3f0c071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.748+0000 7f3f0be00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c071a40 con 0x7f3f0c0710b0 2024-04-25T10:08:07.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.748+0000 7f3f0be00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c06d740 con 0x7f3f0c15df20 2024-04-25T10:08:07.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.748+0000 7f3f0be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c147590 con 0x7f3f0c1471b0 2024-04-25T10:08:07.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.748+0000 7f3f0b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3f0c0710b0 0x7f3f0c071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.748+0000 7f3f0b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3f0c0710b0 0x7f3f0c071500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45358/0 (socket says 172.21.15.119:45358) 2024-04-25T10:08:07.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.748+0000 7f3f0b400640 1 -- 172.21.15.119:0/3581282912 learned_addr learned my addr 172.21.15.119:0/3581282912 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:07.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.749+0000 7f3f0aa00640 1 --2- 172.21.15.119:0/3581282912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f0c1471b0 0x7f3f0c06dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.749+0000 7f3f03e00640 1 --2- 172.21.15.119:0/3581282912 >> v2:172.21.15.132:3300/0 conn(0x7f3f0c15df20 0x7f3f0c15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.749+0000 7f3f0b400640 1 -- 172.21.15.119:0/3581282912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f0c1471b0 msgr2=0x7f3f0c06dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:07.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.749+0000 7f3f0b400640 1 --2- 172.21.15.119:0/3581282912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f0c1471b0 0x7f3f0c06dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.749+0000 7f3f0b400640 1 -- 172.21.15.119:0/3581282912 >> v2:172.21.15.132:3300/0 conn(0x7f3f0c15df20 msgr2=0x7f3f0c15e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:07.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.749+0000 7f3f0b400640 1 --2- 172.21.15.119:0/3581282912 >> v2:172.21.15.132:3300/0 conn(0x7f3f0c15df20 0x7f3f0c15e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.749+0000 7f3f0b400640 1 -- 172.21.15.119:0/3581282912 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c06d110 con 0x7f3f0c0710b0 2024-04-25T10:08:07.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.749+0000 7f3f03e00640 1 --2- 172.21.15.119:0/3581282912 >> v2:172.21.15.132:3300/0 conn(0x7f3f0c15df20 0x7f3f0c15e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:07.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.750+0000 7f3f0b400640 1 --2- 172.21.15.119:0/3581282912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f0c0710b0 0x7f3f0c071500 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c06f260 tx=0x7f3f0402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e126a7f55d22a4e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.750+0000 7f3f0a000640 1 -- 172.21.15.119:0/3581282912 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04036070 con 0x7f3f0c0710b0 2024-04-25T10:08:07.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.750+0000 7f3f0a000640 1 -- 172.21.15.119:0/3581282912 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f04031600 con 0x7f3f0c0710b0 2024-04-25T10:08:07.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.751+0000 7f3f0a000640 1 -- 172.21.15.119:0/3581282912 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f040319d0 con 0x7f3f0c0710b0 2024-04-25T10:08:07.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.751+0000 7f3f0be00640 1 -- 172.21.15.119:0/3581282912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f0c0710b0 msgr2=0x7f3f0c071500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.751+0000 7f3f0be00640 1 --2- 172.21.15.119:0/3581282912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f0c0710b0 0x7f3f0c071500 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c06f260 tx=0x7f3f0402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.751+0000 7f3f0be00640 1 -- 172.21.15.119:0/3581282912 shutdown_connections 2024-04-25T10:08:07.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.751+0000 7f3f0be00640 1 --2- 172.21.15.119:0/3581282912 >> v2:172.21.15.132:3300/0 conn(0x7f3f0c15df20 0x7f3f0c15e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.751+0000 7f3f0be00640 1 --2- 172.21.15.119:0/3581282912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f0c0710b0 0x7f3f0c071500 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.751+0000 7f3f0be00640 1 --2- 172.21.15.119:0/3581282912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f0c1471b0 0x7f3f0c06dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.751+0000 7f3f0be00640 1 -- 172.21.15.119:0/3581282912 >> 172.21.15.119:0/3581282912 conn(0x7f3f0c0b12e0 msgr2=0x7f3f0c0b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:07.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.751+0000 7f3f0be00640 1 -- 172.21.15.119:0/3581282912 shutdown_connections 2024-04-25T10:08:07.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.751+0000 7f3f0be00640 1 -- 172.21.15.119:0/3581282912 wait complete. 2024-04-25T10:08:07.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.753+0000 7f3f0be00640 1 Processor -- start 2024-04-25T10:08:07.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.754+0000 7f3f0be00640 1 -- start start 2024-04-25T10:08:07.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.755+0000 7f3f0be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f0c0710b0 0x7f3f0c0b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.755+0000 7f3f0be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f0c1471b0 0x7f3f0c15aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.755+0000 7f3f0be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f0c15df20 0x7f3f0c139110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.755+0000 7f3f0be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c06d3d0 con 0x7f3f0c1471b0 2024-04-25T10:08:07.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.755+0000 7f3f0be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c15ed10 con 0x7f3f0c15df20 2024-04-25T10:08:07.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.755+0000 7f3f0be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c06e7e0 con 0x7f3f0c0710b0 2024-04-25T10:08:07.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.755+0000 7f3f03e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f0c1471b0 0x7f3f0c15aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.755+0000 7f3f03e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f0c1471b0 0x7f3f0c15aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45366/0 (socket says 172.21.15.119:45366) 2024-04-25T10:08:07.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.755+0000 7f3f03e00640 1 -- 172.21.15.119:0/3625549558 learned_addr learned my addr 172.21.15.119:0/3625549558 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:07.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.755+0000 7f3f0aa00640 1 --2- 172.21.15.119:0/3625549558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f0c0710b0 0x7f3f0c0b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.756+0000 7f3f03e00640 1 -- 172.21.15.119:0/3625549558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f0c0710b0 msgr2=0x7f3f0c0b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.756+0000 7f3f03e00640 1 --2- 172.21.15.119:0/3625549558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f0c0710b0 0x7f3f0c0b0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.756+0000 7f3f03e00640 1 -- 172.21.15.119:0/3625549558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f0c15df20 msgr2=0x7f3f0c139110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.756+0000 7f3f03e00640 1 --2- 172.21.15.119:0/3625549558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f0c15df20 0x7f3f0c139110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.756+0000 7f3f03e00640 1 -- 172.21.15.119:0/3625549558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c139a40 con 0x7f3f0c1471b0 2024-04-25T10:08:07.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.756+0000 7f3f03e00640 1 --2- 172.21.15.119:0/3625549558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f0c1471b0 0x7f3f0c15aed0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400c820 tx=0x7f3ef400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.756+0000 7f3f02a00640 1 -- 172.21.15.119:0/3625549558 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4015070 con 0x7f3f0c1471b0 2024-04-25T10:08:07.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.756+0000 7f3f02a00640 1 -- 172.21.15.119:0/3625549558 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef4007a30 con 0x7f3f0c1471b0 2024-04-25T10:08:07.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.756+0000 7f3f0be00640 1 -- 172.21.15.119:0/3625549558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c139d20 con 0x7f3f0c1471b0 2024-04-25T10:08:07.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.756+0000 7f3f02a00640 1 -- 172.21.15.119:0/3625549558 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4007e00 con 0x7f3f0c1471b0 2024-04-25T10:08:07.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.756+0000 7f3f0be00640 1 -- 172.21.15.119:0/3625549558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c13a2e0 con 0x7f3f0c1471b0 2024-04-25T10:08:07.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.757+0000 7f3f0be00640 1 -- 172.21.15.119:0/3625549558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c0749b0 con 0x7f3f0c1471b0 2024-04-25T10:08:07.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.758+0000 7f3f02a00640 1 -- 172.21.15.119:0/3625549558 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ef4018070 con 0x7f3f0c1471b0 2024-04-25T10:08:07.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.759+0000 7f3f02a00640 1 --2- 172.21.15.119:0/3625549558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3ee806b4d0 0x7f3ee806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:07.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.759+0000 7f3f02a00640 1 -- 172.21.15.119:0/3625549558 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3ef4089df0 con 0x7f3f0c1471b0 2024-04-25T10:08:07.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.760+0000 7f3f0aa00640 1 --2- 172.21.15.119:0/3625549558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3ee806b4d0 0x7f3ee806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:07.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.760+0000 7f3f0aa00640 1 --2- 172.21.15.119:0/3625549558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3ee806b4d0 0x7f3ee806d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00b7e0 tx=0x7f3efc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:07.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:07.763+0000 7f3f02a00640 1 -- 172.21.15.119:0/3625549558 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ef4058fc0 con 0x7f3f0c1471b0 2024-04-25T10:08:08.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.034+0000 7f3f0be00640 1 -- 172.21.15.119:0/3625549558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f3f0c072360 con 0x7f3f0c1471b0 2024-04-25T10:08:08.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.038+0000 7f3f02a00640 1 -- 172.21.15.119:0/3625549558 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3ef40588c0 con 0x7f3f0c1471b0 2024-04-25T10:08:08.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.040+0000 7f3f0be00640 1 -- 172.21.15.119:0/3625549558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3ee806b4d0 msgr2=0x7f3ee806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.040+0000 7f3f0be00640 1 --2- 172.21.15.119:0/3625549558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3ee806b4d0 0x7f3ee806d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00b7e0 tx=0x7f3efc008040 comp rx=0 tx=0).stop 2024-04-25T10:08:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.040+0000 7f3f0be00640 1 -- 172.21.15.119:0/3625549558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f0c1471b0 msgr2=0x7f3f0c15aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.040+0000 7f3f0be00640 1 --2- 172.21.15.119:0/3625549558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f0c1471b0 0x7f3f0c15aed0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400c820 tx=0x7f3ef400cc20 comp rx=0 tx=0).stop 2024-04-25T10:08:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.040+0000 7f3f0be00640 1 -- 172.21.15.119:0/3625549558 shutdown_connections 2024-04-25T10:08:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.040+0000 7f3f0be00640 1 --2- 172.21.15.119:0/3625549558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3ee806b4d0 0x7f3ee806d990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.040+0000 7f3f0be00640 1 --2- 172.21.15.119:0/3625549558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f0c15df20 0x7f3f0c139110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.040+0000 7f3f0be00640 1 --2- 172.21.15.119:0/3625549558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f0c1471b0 0x7f3f0c15aed0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.040+0000 7f3f0be00640 1 --2- 172.21.15.119:0/3625549558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f0c0710b0 0x7f3f0c0b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.040+0000 7f3f0be00640 1 -- 172.21.15.119:0/3625549558 >> 172.21.15.119:0/3625549558 conn(0x7f3f0c0b12e0 msgr2=0x7f3f0c15f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.041+0000 7f3f0be00640 1 -- 172.21.15.119:0/3625549558 shutdown_connections 2024-04-25T10:08:08.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.041+0000 7f3f0be00640 1 -- 172.21.15.119:0/3625549558 wait complete. 2024-04-25T10:08:08.065 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T10:08:08.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.244+0000 7fa8b3200640 1 Processor -- start 2024-04-25T10:08:08.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.245+0000 7fa8b3200640 1 -- start start 2024-04-25T10:08:08.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.246+0000 7fa8b3200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa8b415dc20 0x7fa8b40b88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.246+0000 7fa8b3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8b40b2570 0x7fa8b40b9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.246+0000 7fa8b3200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8b40b2de0 con 0x7fa8b40d0850 2024-04-25T10:08:08.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.246+0000 7fa8b3200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8b40b9840 con 0x7fa8b415dc20 2024-04-25T10:08:08.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.246+0000 7fa8b3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b40b7b00 con 0x7fa8b40b2570 2024-04-25T10:08:08.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.246+0000 7fa8b1e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa8b415dc20 0x7fa8b40b88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.246+0000 7fa8b1e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa8b415dc20 0x7fa8b40b88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45368/0 (socket says 172.21.15.119:45368) 2024-04-25T10:08:08.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.246+0000 7fa8b1e00640 1 -- 172.21.15.119:0/3838588676 learned_addr learned my addr 172.21.15.119:0/3838588676 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:08.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.247+0000 7fa8b1e00640 1 -- 172.21.15.119:0/3838588676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8b40b2570 msgr2=0x7fa8b40b9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.247+0000 7fa8b1400640 1 --2- 172.21.15.119:0/3838588676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8b40b2570 0x7fa8b40b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.247+0000 7fa8b1e00640 1 --2- 172.21.15.119:0/3838588676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8b40b2570 0x7fa8b40b9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.247+0000 7fa8b1e00640 1 -- 172.21.15.119:0/3838588676 >> v1:172.21.15.132:6789/0 conn(0x7fa8b40d0850 legacy=0x7fa8b40d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.247+0000 7fa8b1e00640 1 -- 172.21.15.119:0/3838588676 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b40b7de0 con 0x7fa8b415dc20 2024-04-25T10:08:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.248+0000 7fa8b1e00640 1 --2- 172.21.15.119:0/3838588676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b415dc20 0x7fa8b40b88c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009e50 tx=0x7fa8a402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6358ed16b908ba27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.248+0000 7fa8b0a00640 1 -- 172.21.15.119:0/3838588676 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4036070 con 0x7fa8b415dc20 2024-04-25T10:08:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.248+0000 7fa8b0a00640 1 -- 172.21.15.119:0/3838588676 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a4031600 con 0x7fa8b415dc20 2024-04-25T10:08:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.248+0000 7fa8b0a00640 1 -- 172.21.15.119:0/3838588676 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a40319d0 con 0x7fa8b415dc20 2024-04-25T10:08:08.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.249+0000 7fa8b3200640 1 -- 172.21.15.119:0/3838588676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b415dc20 msgr2=0x7fa8b40b88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.249+0000 7fa8b3200640 1 --2- 172.21.15.119:0/3838588676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b415dc20 0x7fa8b40b88c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009e50 tx=0x7fa8a402a630 comp rx=0 tx=0).stop 2024-04-25T10:08:08.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.249+0000 7fa8b3200640 1 -- 172.21.15.119:0/3838588676 shutdown_connections 2024-04-25T10:08:08.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.249+0000 7fa8b3200640 1 --2- 172.21.15.119:0/3838588676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8b40b2570 0x7fa8b40b9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.249+0000 7fa8b3200640 1 --2- 172.21.15.119:0/3838588676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b415dc20 0x7fa8b40b88c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.249+0000 7fa8b3200640 1 -- 172.21.15.119:0/3838588676 >> 172.21.15.119:0/3838588676 conn(0x7fa8b40726c0 msgr2=0x7fa8b4072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.249+0000 7fa8b3200640 1 -- 172.21.15.119:0/3838588676 shutdown_connections 2024-04-25T10:08:08.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.249+0000 7fa8b3200640 1 -- 172.21.15.119:0/3838588676 wait complete. 2024-04-25T10:08:08.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.251+0000 7fa8b3200640 1 Processor -- start 2024-04-25T10:08:08.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.251+0000 7fa8b3200640 1 -- start start 2024-04-25T10:08:08.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.252+0000 7fa8b3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8b40b2570 0x7fa8b41f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.252+0000 7fa8b3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8b40d0850 0x7fa8b4143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.252+0000 7fa8b3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b415dc20 0x7fa8b40c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.252+0000 7fa8b3200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b40d0ca0 con 0x7fa8b415dc20 2024-04-25T10:08:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.252+0000 7fa8b3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b40b83a0 con 0x7fa8b40d0850 2024-04-25T10:08:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.252+0000 7fa8b3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b40d0e20 con 0x7fa8b40b2570 2024-04-25T10:08:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.252+0000 7fa8b2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b415dc20 0x7fa8b40c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.252+0000 7fa8b2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b415dc20 0x7fa8b40c7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45384/0 (socket says 172.21.15.119:45384) 2024-04-25T10:08:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.252+0000 7fa8b2800640 1 -- 172.21.15.119:0/2129289020 learned_addr learned my addr 172.21.15.119:0/2129289020 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.252+0000 7fa8b2800640 1 -- 172.21.15.119:0/2129289020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8b40b2570 msgr2=0x7fa8b41f3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.253+0000 7fa8b2800640 1 --2- 172.21.15.119:0/2129289020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8b40b2570 0x7fa8b41f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.253+0000 7fa8b2800640 1 -- 172.21.15.119:0/2129289020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8b40d0850 msgr2=0x7fa8b4143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.253+0000 7fa8b1400640 1 --2- 172.21.15.119:0/2129289020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8b40d0850 0x7fa8b4143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.253+0000 7fa8b2800640 1 --2- 172.21.15.119:0/2129289020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8b40d0850 0x7fa8b4143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.253+0000 7fa8b2800640 1 -- 172.21.15.119:0/2129289020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b41f3840 con 0x7fa8b415dc20 2024-04-25T10:08:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.253+0000 7fa8b2800640 1 --2- 172.21.15.119:0/2129289020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b415dc20 0x7fa8b40c7de0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00c440 tx=0x7fa8ac00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.253+0000 7fa8aa000640 1 -- 172.21.15.119:0/2129289020 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac011790 con 0x7fa8b415dc20 2024-04-25T10:08:08.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.253+0000 7fa8aa000640 1 -- 172.21.15.119:0/2129289020 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ac017070 con 0x7fa8b415dc20 2024-04-25T10:08:08.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.253+0000 7fa8aa000640 1 -- 172.21.15.119:0/2129289020 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac011b70 con 0x7fa8b415dc20 2024-04-25T10:08:08.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.253+0000 7fa8b3200640 1 -- 172.21.15.119:0/2129289020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b40c87c0 con 0x7fa8b415dc20 2024-04-25T10:08:08.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.253+0000 7fa8b3200640 1 -- 172.21.15.119:0/2129289020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b4150880 con 0x7fa8b415dc20 2024-04-25T10:08:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.254+0000 7fa8b3200640 1 -- 172.21.15.119:0/2129289020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b40749b0 con 0x7fa8b415dc20 2024-04-25T10:08:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.255+0000 7fa8aa000640 1 -- 172.21.15.119:0/2129289020 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8ac011d10 con 0x7fa8b415dc20 2024-04-25T10:08:08.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.256+0000 7fa8aa000640 1 --2- 172.21.15.119:0/2129289020 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa89006b500 0x7fa89006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.256+0000 7fa8aa000640 1 -- 172.21.15.119:0/2129289020 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa8ac089d60 con 0x7fa8b415dc20 2024-04-25T10:08:08.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.256+0000 7fa8b1e00640 1 --2- 172.21.15.119:0/2129289020 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa89006b500 0x7fa89006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.257+0000 7fa8b1e00640 1 --2- 172.21.15.119:0/2129289020 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa89006b500 0x7fa89006d9c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4033000 tx=0x7fa8a402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.260+0000 7fa8aa000640 1 -- 172.21.15.119:0/2129289020 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8ac058e00 con 0x7fa8b415dc20 2024-04-25T10:08:08.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.517+0000 7fa8b3200640 1 -- 172.21.15.119:0/2129289020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fa8b40b3540 con 0x7fa8b415dc20 2024-04-25T10:08:08.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.522+0000 7fa8aa000640 1 -- 172.21.15.119:0/2129289020 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa8ac058700 con 0x7fa8b415dc20 2024-04-25T10:08:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.524+0000 7fa8b3200640 1 -- 172.21.15.119:0/2129289020 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa89006b500 msgr2=0x7fa89006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.524+0000 7fa8b3200640 1 --2- 172.21.15.119:0/2129289020 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa89006b500 0x7fa89006d9c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4033000 tx=0x7fa8a402e000 comp rx=0 tx=0).stop 2024-04-25T10:08:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.524+0000 7fa8b3200640 1 -- 172.21.15.119:0/2129289020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b415dc20 msgr2=0x7fa8b40c7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.524+0000 7fa8b3200640 1 --2- 172.21.15.119:0/2129289020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b415dc20 0x7fa8b40c7de0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00c440 tx=0x7fa8ac00c900 comp rx=0 tx=0).stop 2024-04-25T10:08:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.525+0000 7fa8b3200640 1 -- 172.21.15.119:0/2129289020 shutdown_connections 2024-04-25T10:08:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.525+0000 7fa8b3200640 1 --2- 172.21.15.119:0/2129289020 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa89006b500 0x7fa89006d9c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.525+0000 7fa8b3200640 1 --2- 172.21.15.119:0/2129289020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8b415dc20 0x7fa8b40c7de0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.525+0000 7fa8b3200640 1 --2- 172.21.15.119:0/2129289020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8b40d0850 0x7fa8b4143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.525+0000 7fa8b3200640 1 --2- 172.21.15.119:0/2129289020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8b40b2570 0x7fa8b41f3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.525+0000 7fa8b3200640 1 -- 172.21.15.119:0/2129289020 >> 172.21.15.119:0/2129289020 conn(0x7fa8b40726c0 msgr2=0x7fa8b415bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.525+0000 7fa8b3200640 1 -- 172.21.15.119:0/2129289020 shutdown_connections 2024-04-25T10:08:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.525+0000 7fa8b3200640 1 -- 172.21.15.119:0/2129289020 wait complete. 2024-04-25T10:08:08.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T10:08:08.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.737+0000 7f5cf8e00640 1 Processor -- start 2024-04-25T10:08:08.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.737+0000 7f5cf8e00640 1 -- start start 2024-04-25T10:08:08.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.739+0000 7f5cf8e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5cf406e2e0 0x7f5cf40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.739+0000 7f5cf8e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5cf40cfc40 0x7f5cf415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.739+0000 7f5cf8e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf40cdb10 con 0x7f5cf415a870 2024-04-25T10:08:08.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.739+0000 7f5cf8e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf40ce720 con 0x7f5cf40cfc40 2024-04-25T10:08:08.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.739+0000 7f5cf8e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf40cddf0 con 0x7f5cf406e2e0 2024-04-25T10:08:08.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.739+0000 7f5cf1e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5cf415a870 0x7f5cf40cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:32784/0 (socket says 172.21.15.119:32784) 2024-04-25T10:08:08.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.739+0000 7f5cf1e00640 1 -- 172.21.15.119:0/1456458275 learned_addr learned my addr 172.21.15.119:0/1456458275 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:08.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.739+0000 7f5cf2800640 1 --2- 172.21.15.119:0/1456458275 >> v2:172.21.15.119:3300/0 conn(0x7f5cf40cfc40 0x7f5cf415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.739+0000 7f5cf2800640 1 -- 172.21.15.119:0/1456458275 >> v2:172.21.15.132:3300/0 conn(0x7f5cf406e2e0 msgr2=0x7f5cf40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.739+0000 7f5cebe00640 1 --2- 172.21.15.119:0/1456458275 >> v2:172.21.15.132:3300/0 conn(0x7f5cf406e2e0 0x7f5cf40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.740+0000 7f5cf1400640 1 -- 172.21.15.119:0/1456458275 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569714286 0 0) 0x7f5cf40cdb10 con 0x7f5cf415a870 2024-04-25T10:08:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.740+0000 7f5cf2800640 1 --2- 172.21.15.119:0/1456458275 >> v2:172.21.15.132:3300/0 conn(0x7f5cf406e2e0 0x7f5cf40cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.740+0000 7f5cf2800640 1 -- 172.21.15.119:0/1456458275 >> v1:172.21.15.119:6789/0 conn(0x7f5cf415a870 legacy=0x7f5cf40cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.740+0000 7f5cf2800640 1 -- 172.21.15.119:0/1456458275 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40ce0d0 con 0x7f5cf40cfc40 2024-04-25T10:08:08.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.741+0000 7f5cf2800640 1 --2- 172.21.15.119:0/1456458275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cf40cfc40 0x7f5cf415ac50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009d90 tx=0x7f5cec02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d57493a25b580c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.741+0000 7f5cf1400640 1 -- 172.21.15.119:0/1456458275 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec007740 con 0x7f5cf40cfc40 2024-04-25T10:08:08.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.741+0000 7f5cf1400640 1 -- 172.21.15.119:0/1456458275 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec0078e0 con 0x7f5cf40cfc40 2024-04-25T10:08:08.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.741+0000 7f5cf1400640 1 -- 172.21.15.119:0/1456458275 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec007cb0 con 0x7f5cf40cfc40 2024-04-25T10:08:08.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.742+0000 7f5cf8e00640 1 -- 172.21.15.119:0/1456458275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cf40cfc40 msgr2=0x7f5cf415ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.742+0000 7f5cf8e00640 1 --2- 172.21.15.119:0/1456458275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cf40cfc40 0x7f5cf415ac50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009d90 tx=0x7f5cec02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.742+0000 7f5cf8e00640 1 -- 172.21.15.119:0/1456458275 shutdown_connections 2024-04-25T10:08:08.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.742+0000 7f5cf8e00640 1 --2- 172.21.15.119:0/1456458275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cf40cfc40 0x7f5cf415ac50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.742+0000 7f5cf8e00640 1 --2- 172.21.15.119:0/1456458275 >> v2:172.21.15.132:3300/0 conn(0x7f5cf406e2e0 0x7f5cf40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.742+0000 7f5cf8e00640 1 -- 172.21.15.119:0/1456458275 >> 172.21.15.119:0/1456458275 conn(0x7f5cf406aba0 msgr2=0x7f5cf406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.742+0000 7f5cf8e00640 1 -- 172.21.15.119:0/1456458275 shutdown_connections 2024-04-25T10:08:08.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.742+0000 7f5cf8e00640 1 -- 172.21.15.119:0/1456458275 wait complete. 2024-04-25T10:08:08.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.744+0000 7f5cf8e00640 1 Processor -- start 2024-04-25T10:08:08.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.744+0000 7f5cf8e00640 1 -- start start 2024-04-25T10:08:08.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.745+0000 7f5cf8e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf406e2e0 0x7f5cf4155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.745+0000 7f5cf8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cf40cfc40 0x7f5cf415c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.745+0000 7f5cf8e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf415a870 0x7f5cf415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.745+0000 7f5cf8e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40cd700 con 0x7f5cf40cfc40 2024-04-25T10:08:08.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.745+0000 7f5cf8e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40cd580 con 0x7f5cf406e2e0 2024-04-25T10:08:08.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.745+0000 7f5cf8e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40cd880 con 0x7f5cf415a870 2024-04-25T10:08:08.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.746+0000 7f5cebe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cf40cfc40 0x7f5cf415c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.746+0000 7f5cebe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cf40cfc40 0x7f5cf415c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45408/0 (socket says 172.21.15.119:45408) 2024-04-25T10:08:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.746+0000 7f5cebe00640 1 -- 172.21.15.119:0/2684721945 learned_addr learned my addr 172.21.15.119:0/2684721945 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.746+0000 7f5cf1e00640 1 --2- 172.21.15.119:0/2684721945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf406e2e0 0x7f5cf4155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.746+0000 7f5cf2800640 1 --2- 172.21.15.119:0/2684721945 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf415a870 0x7f5cf415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.746+0000 7f5cebe00640 1 -- 172.21.15.119:0/2684721945 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf415a870 msgr2=0x7f5cf415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.746+0000 7f5cebe00640 1 --2- 172.21.15.119:0/2684721945 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf415a870 0x7f5cf415ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.746+0000 7f5cebe00640 1 -- 172.21.15.119:0/2684721945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf406e2e0 msgr2=0x7f5cf4155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.746+0000 7f5cebe00640 1 --2- 172.21.15.119:0/2684721945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf406e2e0 0x7f5cf4155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.746+0000 7f5cebe00640 1 -- 172.21.15.119:0/2684721945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40c4800 con 0x7f5cf40cfc40 2024-04-25T10:08:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.747+0000 7f5cebe00640 1 --2- 172.21.15.119:0/2684721945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cf40cfc40 0x7f5cf415c990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00c7f0 tx=0x7f5cdc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.747+0000 7f5cea000640 1 -- 172.21.15.119:0/2684721945 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc015070 con 0x7f5cf40cfc40 2024-04-25T10:08:08.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.747+0000 7f5cea000640 1 -- 172.21.15.119:0/2684721945 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc004240 con 0x7f5cf40cfc40 2024-04-25T10:08:08.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.747+0000 7f5cf8e00640 1 -- 172.21.15.119:0/2684721945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf4156e30 con 0x7f5cf40cfc40 2024-04-25T10:08:08.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.747+0000 7f5cea000640 1 -- 172.21.15.119:0/2684721945 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc007620 con 0x7f5cf40cfc40 2024-04-25T10:08:08.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.747+0000 7f5cf8e00640 1 -- 172.21.15.119:0/2684721945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf41573f0 con 0x7f5cf40cfc40 2024-04-25T10:08:08.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.748+0000 7f5cf8e00640 1 -- 172.21.15.119:0/2684721945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf40749b0 con 0x7f5cf40cfc40 2024-04-25T10:08:08.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.749+0000 7f5cea000640 1 -- 172.21.15.119:0/2684721945 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5cdc0079d0 con 0x7f5cf40cfc40 2024-04-25T10:08:08.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.750+0000 7f5cea000640 1 --2- 172.21.15.119:0/2684721945 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5cd006b4d0 0x7f5cd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.750+0000 7f5cea000640 1 -- 172.21.15.119:0/2684721945 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5cdc089e80 con 0x7f5cf40cfc40 2024-04-25T10:08:08.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.750+0000 7f5cf1e00640 1 --2- 172.21.15.119:0/2684721945 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5cd006b4d0 0x7f5cd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.751+0000 7f5cf1e00640 1 --2- 172.21.15.119:0/2684721945 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5cd006b4d0 0x7f5cd006d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4000f60 tx=0x7f5ce4007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:08.754+0000 7f5cea000640 1 -- 172.21.15.119:0/2684721945 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5cdc0590e0 con 0x7f5cf40cfc40 2024-04-25T10:08:09.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.027+0000 7f5cf8e00640 1 -- 172.21.15.119:0/2684721945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f5cf40aeb20 con 0x7f5cf40cfc40 2024-04-25T10:08:09.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.035+0000 7f5cea000640 1 -- 172.21.15.119:0/2684721945 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5cdc0589e0 con 0x7f5cf40cfc40 2024-04-25T10:08:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.037+0000 7f5cf8e00640 1 -- 172.21.15.119:0/2684721945 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5cd006b4d0 msgr2=0x7f5cd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.037+0000 7f5cf8e00640 1 --2- 172.21.15.119:0/2684721945 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5cd006b4d0 0x7f5cd006d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4000f60 tx=0x7f5ce4007c00 comp rx=0 tx=0).stop 2024-04-25T10:08:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.037+0000 7f5cf8e00640 1 -- 172.21.15.119:0/2684721945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cf40cfc40 msgr2=0x7f5cf415c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.037+0000 7f5cf8e00640 1 --2- 172.21.15.119:0/2684721945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cf40cfc40 0x7f5cf415c990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00c7f0 tx=0x7f5cdc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.038+0000 7f5cf8e00640 1 -- 172.21.15.119:0/2684721945 shutdown_connections 2024-04-25T10:08:09.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.038+0000 7f5cf8e00640 1 --2- 172.21.15.119:0/2684721945 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5cd006b4d0 0x7f5cd006d990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.038+0000 7f5cf8e00640 1 --2- 172.21.15.119:0/2684721945 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5cf415a870 0x7f5cf415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.038+0000 7f5cf8e00640 1 --2- 172.21.15.119:0/2684721945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cf40cfc40 0x7f5cf415c990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.038+0000 7f5cf8e00640 1 --2- 172.21.15.119:0/2684721945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf406e2e0 0x7f5cf4155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.038+0000 7f5cf8e00640 1 -- 172.21.15.119:0/2684721945 >> 172.21.15.119:0/2684721945 conn(0x7f5cf406aba0 msgr2=0x7f5cf415d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.038+0000 7f5cf8e00640 1 -- 172.21.15.119:0/2684721945 shutdown_connections 2024-04-25T10:08:09.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.038+0000 7f5cf8e00640 1 -- 172.21.15.119:0/2684721945 wait complete. 2024-04-25T10:08:09.063 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T10:08:09.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.255+0000 7f75fb200640 1 Processor -- start 2024-04-25T10:08:09.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.255+0000 7f75fb200640 1 -- start start 2024-04-25T10:08:09.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.257+0000 7f75fb200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f75fc0717c0 0x7f75fc06e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.257+0000 7f75fb200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75fc06dfb0 con 0x7f75fc06db20 2024-04-25T10:08:09.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.257+0000 7f75fb200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75fc0cd250 con 0x7f75fc144540 2024-04-25T10:08:09.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.257+0000 7f75fb200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75fc0cd530 con 0x7f75fc0717c0 2024-04-25T10:08:09.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.257+0000 7f75fa800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f75fc06db20 0x7f75fc15bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:32798/0 (socket says 172.21.15.119:32798) 2024-04-25T10:08:09.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.257+0000 7f75fa800640 1 -- 172.21.15.119:0/348071096 learned_addr learned my addr 172.21.15.119:0/348071096 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:09.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f1400640 1 --2- 172.21.15.119:0/348071096 >> v2:172.21.15.132:3300/0 conn(0x7f75fc0717c0 0x7f75fc06e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 -- 172.21.15.119:0/348071096 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559479725 0 0) 0x7f75f4001490 con 0x7f75fc06db20 2024-04-25T10:08:09.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 -- 172.21.15.119:0/348071096 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e0003520 con 0x7f75fc06db20 2024-04-25T10:08:09.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 -- 172.21.15.119:0/348071096 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2340127637 0 0) 0x7f75e0003520 con 0x7f75fc06db20 2024-04-25T10:08:09.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 -- 172.21.15.119:0/348071096 >> v2:172.21.15.132:3300/0 conn(0x7f75fc0717c0 msgr2=0x7f75fc06e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 --2- 172.21.15.119:0/348071096 >> v2:172.21.15.132:3300/0 conn(0x7f75fc0717c0 0x7f75fc06e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 -- 172.21.15.119:0/348071096 >> v1:172.21.15.132:6789/0 conn(0x7f75fc144540 legacy=0x7f75fc0710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 -- 172.21.15.119:0/348071096 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc0cd810 con 0x7f75fc06db20 2024-04-25T10:08:09.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 -- 172.21.15.119:0/348071096 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f75f4001bc0 con 0x7f75fc06db20 2024-04-25T10:08:09.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 -- 172.21.15.119:0/348071096 >> v1:172.21.15.119:6789/0 conn(0x7f75fc06db20 legacy=0x7f75fc15bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 --2- 172.21.15.119:0/348071096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75e00039b0 0x7f75e0003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 --2- 172.21.15.119:0/348071096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e0004800 0x7f75e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 --2- 172.21.15.119:0/348071096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e00052e0 0x7f75e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 -- 172.21.15.119:0/348071096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc0cd530 con 0x7f75e00052e0 2024-04-25T10:08:09.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 -- 172.21.15.119:0/348071096 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e000c190 con 0x7f75e0004800 2024-04-25T10:08:09.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f9400640 1 -- 172.21.15.119:0/348071096 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e000c390 con 0x7f75e00039b0 2024-04-25T10:08:09.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.258+0000 7f75f1400640 1 --2- 172.21.15.119:0/348071096 >> v2:172.21.15.132:3300/0 conn(0x7f75fc0717c0 0x7f75fc06e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:09.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.259+0000 7f75fa800640 1 --2- 172.21.15.119:0/348071096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e00052e0 0x7f75e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.259+0000 7f75f9e00640 1 --2- 172.21.15.119:0/348071096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75e00039b0 0x7f75e0003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.259+0000 7f75fa800640 1 -- 172.21.15.119:0/348071096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75e00039b0 msgr2=0x7f75e0003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.259+0000 7f75f1400640 1 --2- 172.21.15.119:0/348071096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e0004800 0x7f75e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.259+0000 7f75fa800640 1 --2- 172.21.15.119:0/348071096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75e00039b0 0x7f75e0003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.259+0000 7f75fa800640 1 -- 172.21.15.119:0/348071096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e0004800 msgr2=0x7f75e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.259+0000 7f75fa800640 1 --2- 172.21.15.119:0/348071096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e0004800 0x7f75e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.259+0000 7f75fa800640 1 -- 172.21.15.119:0/348071096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e0003520 con 0x7f75e00052e0 2024-04-25T10:08:09.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.259+0000 7f75f1400640 1 --2- 172.21.15.119:0/348071096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75e0004800 0x7f75e0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:09.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.259+0000 7f75f9e00640 1 --2- 172.21.15.119:0/348071096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75e00039b0 0x7f75e0003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:09.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.260+0000 7f75fa800640 1 -- 172.21.15.119:0/348071096 reap_dead start 2024-04-25T10:08:09.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.260+0000 7f75fa800640 1 --2- 172.21.15.119:0/348071096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e00052e0 0x7f75e000b9c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f75f400aa00 tx=0x7f75e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d49d1d6fdf5360 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.260+0000 7f75f9400640 1 -- 172.21.15.119:0/348071096 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f400a650 con 0x7f75e00052e0 2024-04-25T10:08:09.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.260+0000 7f75f9400640 1 -- 172.21.15.119:0/348071096 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f402f070 con 0x7f75e00052e0 2024-04-25T10:08:09.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.260+0000 7f75f9400640 1 -- 172.21.15.119:0/348071096 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f402be60 con 0x7f75e00052e0 2024-04-25T10:08:09.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.261+0000 7f75fb200640 1 -- 172.21.15.119:0/348071096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e00052e0 msgr2=0x7f75e000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.261+0000 7f75fb200640 1 --2- 172.21.15.119:0/348071096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e00052e0 0x7f75e000b9c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f75f400aa00 tx=0x7f75e0005830 comp rx=0 tx=0).stop 2024-04-25T10:08:09.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.261+0000 7f75fb200640 1 -- 172.21.15.119:0/348071096 shutdown_connections 2024-04-25T10:08:09.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.261+0000 7f75fb200640 1 --2- 172.21.15.119:0/348071096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e00052e0 0x7f75e000b9c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.261+0000 7f75fb200640 1 -- 172.21.15.119:0/348071096 >> 172.21.15.119:0/348071096 conn(0x7f75fc0b12e0 msgr2=0x7f75fc0b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.261+0000 7f75fb200640 1 -- 172.21.15.119:0/348071096 shutdown_connections 2024-04-25T10:08:09.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.261+0000 7f75fb200640 1 -- 172.21.15.119:0/348071096 wait complete. 2024-04-25T10:08:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.263+0000 7f75fb200640 1 Processor -- start 2024-04-25T10:08:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.263+0000 7f75fb200640 1 -- start start 2024-04-25T10:08:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.264+0000 7f75fb200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75e00052e0 0x7f75fc14cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.264+0000 7f75fb200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75fc140870 0x7f75fc13c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.264+0000 7f75fb200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75fc140cf0 0x7f75fc13c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.264+0000 7f75fb200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc06e520 con 0x7f75fc140cf0 2024-04-25T10:08:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.264+0000 7f75fb200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc06e3a0 con 0x7f75fc140870 2024-04-25T10:08:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.264+0000 7f75fb200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc06e6a0 con 0x7f75e00052e0 2024-04-25T10:08:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.264+0000 7f75fa800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75fc140cf0 0x7f75fc13c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.264+0000 7f75fa800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75fc140cf0 0x7f75fc13c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45416/0 (socket says 172.21.15.119:45416) 2024-04-25T10:08:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.264+0000 7f75fa800640 1 -- 172.21.15.119:0/1898918912 learned_addr learned my addr 172.21.15.119:0/1898918912 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.265+0000 7f75fa800640 1 -- 172.21.15.119:0/1898918912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75e00052e0 msgr2=0x7f75fc14cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.265+0000 7f75fa800640 1 --2- 172.21.15.119:0/1898918912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75e00052e0 0x7f75fc14cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.265+0000 7f75fa800640 1 -- 172.21.15.119:0/1898918912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75fc140870 msgr2=0x7f75fc13c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.265+0000 7f75f1400640 1 --2- 172.21.15.119:0/1898918912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75fc140870 0x7f75fc13c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.265+0000 7f75fa800640 1 --2- 172.21.15.119:0/1898918912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75fc140870 0x7f75fc13c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.265+0000 7f75fa800640 1 -- 172.21.15.119:0/1898918912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc13f7d0 con 0x7f75fc140cf0 2024-04-25T10:08:09.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.265+0000 7f75fa800640 1 --2- 172.21.15.119:0/1898918912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75fc140cf0 0x7f75fc13c5a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f75f400ab20 tx=0x7f75fc0cd420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.265+0000 7f75f2a00640 1 -- 172.21.15.119:0/1898918912 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f400a650 con 0x7f75fc140cf0 2024-04-25T10:08:09.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.265+0000 7f75f2a00640 1 -- 172.21.15.119:0/1898918912 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f402ba00 con 0x7f75fc140cf0 2024-04-25T10:08:09.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.265+0000 7f75f2a00640 1 -- 172.21.15.119:0/1898918912 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f402bdf0 con 0x7f75fc140cf0 2024-04-25T10:08:09.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.265+0000 7f75fb200640 1 -- 172.21.15.119:0/1898918912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75fc13fab0 con 0x7f75fc140cf0 2024-04-25T10:08:09.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.265+0000 7f75fb200640 1 -- 172.21.15.119:0/1898918912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75fc140070 con 0x7f75fc140cf0 2024-04-25T10:08:09.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.266+0000 7f75fb200640 1 -- 172.21.15.119:0/1898918912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75fc0749b0 con 0x7f75fc140cf0 2024-04-25T10:08:09.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.267+0000 7f75f2a00640 1 -- 172.21.15.119:0/1898918912 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75f4034470 con 0x7f75fc140cf0 2024-04-25T10:08:09.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.268+0000 7f75f2a00640 1 --2- 172.21.15.119:0/1898918912 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f75d806b4d0 0x7f75d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.268+0000 7f75f2a00640 1 -- 172.21.15.119:0/1898918912 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f75f40a97e0 con 0x7f75fc140cf0 2024-04-25T10:08:09.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.268+0000 7f75f9e00640 1 --2- 172.21.15.119:0/1898918912 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f75d806b4d0 0x7f75d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.269+0000 7f75f9e00640 1 --2- 172.21.15.119:0/1898918912 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f75d806b4d0 0x7f75d806d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f75ec003db0 tx=0x7f75ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.272+0000 7f75f2a00640 1 -- 172.21.15.119:0/1898918912 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75f4046030 con 0x7f75fc140cf0 2024-04-25T10:08:09.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.537+0000 7f75fb200640 1 -- 172.21.15.119:0/1898918912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f75fc144cf0 con 0x7f75fc140cf0 2024-04-25T10:08:09.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.541+0000 7f75f2a00640 1 -- 172.21.15.119:0/1898918912 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f75fc06dfb0 con 0x7f75fc140cf0 2024-04-25T10:08:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.543+0000 7f75fb200640 1 -- 172.21.15.119:0/1898918912 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f75d806b4d0 msgr2=0x7f75d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.543+0000 7f75fb200640 1 --2- 172.21.15.119:0/1898918912 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f75d806b4d0 0x7f75d806d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f75ec003db0 tx=0x7f75ec009040 comp rx=0 tx=0).stop 2024-04-25T10:08:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.543+0000 7f75fb200640 1 -- 172.21.15.119:0/1898918912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75fc140cf0 msgr2=0x7f75fc13c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.543+0000 7f75fb200640 1 --2- 172.21.15.119:0/1898918912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75fc140cf0 0x7f75fc13c5a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f75f400ab20 tx=0x7f75fc0cd420 comp rx=0 tx=0).stop 2024-04-25T10:08:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.544+0000 7f75fb200640 1 -- 172.21.15.119:0/1898918912 shutdown_connections 2024-04-25T10:08:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.544+0000 7f75fb200640 1 --2- 172.21.15.119:0/1898918912 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f75d806b4d0 0x7f75d806d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.544+0000 7f75fb200640 1 --2- 172.21.15.119:0/1898918912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75fc140cf0 0x7f75fc13c5a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.544+0000 7f75fb200640 1 --2- 172.21.15.119:0/1898918912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75fc140870 0x7f75fc13c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.544+0000 7f75fb200640 1 --2- 172.21.15.119:0/1898918912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75e00052e0 0x7f75fc14cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.544+0000 7f75fb200640 1 -- 172.21.15.119:0/1898918912 >> 172.21.15.119:0/1898918912 conn(0x7f75fc0b12e0 msgr2=0x7f75fc15ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.544+0000 7f75fb200640 1 -- 172.21.15.119:0/1898918912 shutdown_connections 2024-04-25T10:08:09.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.544+0000 7f75fb200640 1 -- 172.21.15.119:0/1898918912 wait complete. 2024-04-25T10:08:09.569 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T10:08:09.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.751+0000 7fa12bc00640 1 Processor -- start 2024-04-25T10:08:09.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.752+0000 7fa12bc00640 1 -- start start 2024-04-25T10:08:09.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.753+0000 7fa12bc00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1241610b0 0x7fa1240b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.753+0000 7fa12bc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa1241618d0 0x7fa1240ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.753+0000 7fa12bc00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa124162040 con 0x7fa1240ced70 2024-04-25T10:08:09.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.753+0000 7fa12bc00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1240cf1c0 con 0x7fa1241610b0 2024-04-25T10:08:09.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.753+0000 7fa12bc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1240d1ea0 con 0x7fa1241618d0 2024-04-25T10:08:09.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.753+0000 7fa129200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1241610b0 0x7fa1240b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.753+0000 7fa129c00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa1240ced70 0x7fa12406c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:32808/0 (socket says 172.21.15.119:32808) 2024-04-25T10:08:09.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.753+0000 7fa129200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1241610b0 0x7fa1240b9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45418/0 (socket says 172.21.15.119:45418) 2024-04-25T10:08:09.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.753+0000 7fa129c00640 1 -- 172.21.15.119:0/154083443 learned_addr learned my addr 172.21.15.119:0/154083443 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:09.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.754+0000 7fa11be00640 1 --2- 172.21.15.119:0/154083443 >> v2:172.21.15.132:3300/0 conn(0x7fa1241618d0 0x7fa1240ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.754+0000 7fa129200640 1 -- 172.21.15.119:0/154083443 >> v2:172.21.15.132:3300/0 conn(0x7fa1241618d0 msgr2=0x7fa1240ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.754+0000 7fa129200640 1 --2- 172.21.15.119:0/154083443 >> v2:172.21.15.132:3300/0 conn(0x7fa1241618d0 0x7fa1240ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.754+0000 7fa129200640 1 -- 172.21.15.119:0/154083443 >> v1:172.21.15.119:6789/0 conn(0x7fa1240ced70 legacy=0x7fa12406c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.754+0000 7fa129200640 1 -- 172.21.15.119:0/154083443 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa124162320 con 0x7fa1241610b0 2024-04-25T10:08:09.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.754+0000 7fa11be00640 1 --2- 172.21.15.119:0/154083443 >> v2:172.21.15.132:3300/0 conn(0x7fa1241618d0 0x7fa1240ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:09.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.754+0000 7fa123e00640 1 -- 172.21.15.119:0/154083443 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679147707 0 0) 0x7fa124162040 con 0x7fa1240ced70 2024-04-25T10:08:09.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.755+0000 7fa129200640 1 --2- 172.21.15.119:0/154083443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1241610b0 0x7fa1240b9580 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa12406d320 tx=0x7fa11402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429a6060bff2884f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.755+0000 7fa123e00640 1 -- 172.21.15.119:0/154083443 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114037070 con 0x7fa1241610b0 2024-04-25T10:08:09.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.755+0000 7fa123e00640 1 -- 172.21.15.119:0/154083443 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa114032440 con 0x7fa1241610b0 2024-04-25T10:08:09.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.755+0000 7fa123e00640 1 -- 172.21.15.119:0/154083443 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11402ae60 con 0x7fa1241610b0 2024-04-25T10:08:09.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.756+0000 7fa12bc00640 1 -- 172.21.15.119:0/154083443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1241610b0 msgr2=0x7fa1240b9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.756+0000 7fa12bc00640 1 --2- 172.21.15.119:0/154083443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1241610b0 0x7fa1240b9580 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa12406d320 tx=0x7fa11402a110 comp rx=0 tx=0).stop 2024-04-25T10:08:09.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.756+0000 7fa12bc00640 1 -- 172.21.15.119:0/154083443 shutdown_connections 2024-04-25T10:08:09.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.756+0000 7fa12bc00640 1 --2- 172.21.15.119:0/154083443 >> v2:172.21.15.132:3300/0 conn(0x7fa1241618d0 0x7fa1240ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.756+0000 7fa12bc00640 1 --2- 172.21.15.119:0/154083443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1241610b0 0x7fa1240b9580 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.756+0000 7fa12bc00640 1 -- 172.21.15.119:0/154083443 >> 172.21.15.119:0/154083443 conn(0x7fa1240b5570 msgr2=0x7fa1240b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.756+0000 7fa12bc00640 1 -- 172.21.15.119:0/154083443 shutdown_connections 2024-04-25T10:08:09.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.756+0000 7fa12bc00640 1 -- 172.21.15.119:0/154083443 wait complete. 2024-04-25T10:08:09.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.758+0000 7fa12bc00640 1 Processor -- start 2024-04-25T10:08:09.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.758+0000 7fa12bc00640 1 -- start start 2024-04-25T10:08:09.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa12bc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1240ced70 0x7fa1240c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa12bc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1241610b0 0x7fa124157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa12bc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1241618d0 0x7fa1241580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa12bc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1240b6c00 con 0x7fa1241610b0 2024-04-25T10:08:09.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa12bc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1240d20f0 con 0x7fa1241618d0 2024-04-25T10:08:09.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa12bc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa124161d10 con 0x7fa1240ced70 2024-04-25T10:08:09.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa11be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1241610b0 0x7fa124157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa11be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1241610b0 0x7fa124157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45434/0 (socket says 172.21.15.119:45434) 2024-04-25T10:08:09.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa11be00640 1 -- 172.21.15.119:0/1701853174 learned_addr learned my addr 172.21.15.119:0/1701853174 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:09.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa129200640 1 --2- 172.21.15.119:0/1701853174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1240ced70 0x7fa1240c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa11be00640 1 -- 172.21.15.119:0/1701853174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1240ced70 msgr2=0x7fa1240c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa129c00640 1 --2- 172.21.15.119:0/1701853174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1241618d0 0x7fa1241580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa11be00640 1 --2- 172.21.15.119:0/1701853174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1240ced70 0x7fa1240c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa11be00640 1 -- 172.21.15.119:0/1701853174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1241618d0 msgr2=0x7fa1241580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa11be00640 1 --2- 172.21.15.119:0/1701853174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1241618d0 0x7fa1241580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.759+0000 7fa11be00640 1 -- 172.21.15.119:0/1701853174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa124142be0 con 0x7fa1241610b0 2024-04-25T10:08:09.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.760+0000 7fa11be00640 1 --2- 172.21.15.119:0/1701853174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1241610b0 0x7fa124157ba0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00c7f0 tx=0x7fa10c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.760+0000 7fa121600640 1 -- 172.21.15.119:0/1701853174 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c015070 con 0x7fa1241610b0 2024-04-25T10:08:09.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.760+0000 7fa121600640 1 -- 172.21.15.119:0/1701853174 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c007820 con 0x7fa1241610b0 2024-04-25T10:08:09.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.760+0000 7fa12bc00640 1 -- 172.21.15.119:0/1701853174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1240cc180 con 0x7fa1241610b0 2024-04-25T10:08:09.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.760+0000 7fa121600640 1 -- 172.21.15.119:0/1701853174 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c007bd0 con 0x7fa1241610b0 2024-04-25T10:08:09.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.760+0000 7fa12bc00640 1 -- 172.21.15.119:0/1701853174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa124142e80 con 0x7fa1241610b0 2024-04-25T10:08:09.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.761+0000 7fa12bc00640 1 -- 172.21.15.119:0/1701853174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1240750b0 con 0x7fa1241610b0 2024-04-25T10:08:09.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.762+0000 7fa121600640 1 -- 172.21.15.119:0/1701853174 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa10c018070 con 0x7fa1241610b0 2024-04-25T10:08:09.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.763+0000 7fa121600640 1 --2- 172.21.15.119:0/1701853174 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa10006b4d0 0x7fa10006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.763+0000 7fa121600640 1 -- 172.21.15.119:0/1701853174 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa10c08a080 con 0x7fa1241610b0 2024-04-25T10:08:09.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.763+0000 7fa129200640 1 --2- 172.21.15.119:0/1701853174 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa10006b4d0 0x7fa10006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.764+0000 7fa129200640 1 --2- 172.21.15.119:0/1701853174 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa10006b4d0 0x7fa10006d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa114034000 tx=0x7fa11402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:09.767+0000 7fa121600640 1 -- 172.21.15.119:0/1701853174 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa10c0591b0 con 0x7fa1241610b0 2024-04-25T10:08:10.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.033+0000 7fa12bc00640 1 -- 172.21.15.119:0/1701853174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fa1240cd920 con 0x7fa1241610b0 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.039+0000 7fa121600640 1 -- 172.21.15.119:0/1701853174 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa10c058ab0 con 0x7fa1241610b0 2024-04-25T10:08:10.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.041+0000 7fa12bc00640 1 -- 172.21.15.119:0/1701853174 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa10006b4d0 msgr2=0x7fa10006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.041+0000 7fa12bc00640 1 --2- 172.21.15.119:0/1701853174 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa10006b4d0 0x7fa10006d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa114034000 tx=0x7fa11402e000 comp rx=0 tx=0).stop 2024-04-25T10:08:10.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.041+0000 7fa12bc00640 1 -- 172.21.15.119:0/1701853174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1241610b0 msgr2=0x7fa124157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.041+0000 7fa12bc00640 1 --2- 172.21.15.119:0/1701853174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1241610b0 0x7fa124157ba0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00c7f0 tx=0x7fa10c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.041+0000 7fa12bc00640 1 -- 172.21.15.119:0/1701853174 shutdown_connections 2024-04-25T10:08:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.042+0000 7fa12bc00640 1 --2- 172.21.15.119:0/1701853174 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa10006b4d0 0x7fa10006d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.042+0000 7fa12bc00640 1 --2- 172.21.15.119:0/1701853174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1241618d0 0x7fa1241580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.042+0000 7fa12bc00640 1 --2- 172.21.15.119:0/1701853174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1241610b0 0x7fa124157ba0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.042+0000 7fa12bc00640 1 --2- 172.21.15.119:0/1701853174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1240ced70 0x7fa1240c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.042+0000 7fa12bc00640 1 -- 172.21.15.119:0/1701853174 >> 172.21.15.119:0/1701853174 conn(0x7fa1240b5570 msgr2=0x7fa1240d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.042+0000 7fa12bc00640 1 -- 172.21.15.119:0/1701853174 shutdown_connections 2024-04-25T10:08:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.042+0000 7fa12bc00640 1 -- 172.21.15.119:0/1701853174 wait complete. 2024-04-25T10:08:10.065 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T10:08:10.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.253+0000 7fba25200640 1 Processor -- start 2024-04-25T10:08:10.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.253+0000 7fba25200640 1 -- start start 2024-04-25T10:08:10.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.254+0000 7fba25200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fba201471b0 0x7fba2006dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.254+0000 7fba25200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fba2015df20 0x7fba2015e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.254+0000 7fba25200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba20147590 con 0x7fba200710b0 2024-04-25T10:08:10.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.255+0000 7fba25200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2006d740 con 0x7fba201471b0 2024-04-25T10:08:10.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.255+0000 7fba25200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2006d110 con 0x7fba2015df20 2024-04-25T10:08:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.255+0000 7fba1f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fba201471b0 0x7fba2006dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.255+0000 7fba1f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fba201471b0 0x7fba2006dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45440/0 (socket says 172.21.15.119:45440) 2024-04-25T10:08:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.255+0000 7fba1f400640 1 -- 172.21.15.119:0/1561527902 learned_addr learned my addr 172.21.15.119:0/1561527902 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.255+0000 7fba1f400640 1 -- 172.21.15.119:0/1561527902 >> v2:172.21.15.132:3300/0 conn(0x7fba2015df20 msgr2=0x7fba2015e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.255+0000 7fba1f400640 1 --2- 172.21.15.119:0/1561527902 >> v2:172.21.15.132:3300/0 conn(0x7fba2015df20 0x7fba2015e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.255+0000 7fba1f400640 1 -- 172.21.15.119:0/1561527902 >> v1:172.21.15.132:6789/0 conn(0x7fba200710b0 legacy=0x7fba20071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.255+0000 7fba1f400640 1 -- 172.21.15.119:0/1561527902 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2015ea30 con 0x7fba201471b0 2024-04-25T10:08:10.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.256+0000 7fba1f400640 1 --2- 172.21.15.119:0/1561527902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba201471b0 0x7fba2006dbd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fba200cf160 tx=0x7fba1002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d507eb88a32a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.256+0000 7fba1e000640 1 -- 172.21.15.119:0/1561527902 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10037070 con 0x7fba201471b0 2024-04-25T10:08:10.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.257+0000 7fba1e000640 1 -- 172.21.15.119:0/1561527902 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba10032440 con 0x7fba201471b0 2024-04-25T10:08:10.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.257+0000 7fba1e000640 1 -- 172.21.15.119:0/1561527902 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10032810 con 0x7fba201471b0 2024-04-25T10:08:10.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.257+0000 7fba25200640 1 -- 172.21.15.119:0/1561527902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba201471b0 msgr2=0x7fba2006dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.257+0000 7fba25200640 1 --2- 172.21.15.119:0/1561527902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba201471b0 0x7fba2006dbd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fba200cf160 tx=0x7fba1002a230 comp rx=0 tx=0).stop 2024-04-25T10:08:10.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.257+0000 7fba25200640 1 -- 172.21.15.119:0/1561527902 shutdown_connections 2024-04-25T10:08:10.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.257+0000 7fba25200640 1 --2- 172.21.15.119:0/1561527902 >> v2:172.21.15.132:3300/0 conn(0x7fba2015df20 0x7fba2015e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.257+0000 7fba25200640 1 --2- 172.21.15.119:0/1561527902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba201471b0 0x7fba2006dbd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.257+0000 7fba25200640 1 -- 172.21.15.119:0/1561527902 >> 172.21.15.119:0/1561527902 conn(0x7fba200b12e0 msgr2=0x7fba200b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.257+0000 7fba25200640 1 -- 172.21.15.119:0/1561527902 shutdown_connections 2024-04-25T10:08:10.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.257+0000 7fba25200640 1 -- 172.21.15.119:0/1561527902 wait complete. 2024-04-25T10:08:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.259+0000 7fba25200640 1 Processor -- start 2024-04-25T10:08:10.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.259+0000 7fba25200640 1 -- start start 2024-04-25T10:08:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.260+0000 7fba25200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba200710b0 0x7fba2012ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.260+0000 7fba25200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba201471b0 0x7fba201406a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.260+0000 7fba25200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba2015df20 0x7fba20142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.260+0000 7fba25200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2006e7e0 con 0x7fba200710b0 2024-04-25T10:08:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.260+0000 7fba25200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2006d3d0 con 0x7fba2015df20 2024-04-25T10:08:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.260+0000 7fba25200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2006e110 con 0x7fba201471b0 2024-04-25T10:08:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.260+0000 7fba1f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba200710b0 0x7fba2012ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.261+0000 7fba1f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba200710b0 0x7fba2012ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45454/0 (socket says 172.21.15.119:45454) 2024-04-25T10:08:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.261+0000 7fba1f400640 1 -- 172.21.15.119:0/4190150884 learned_addr learned my addr 172.21.15.119:0/4190150884 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:10.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.261+0000 7fba1fe00640 1 --2- 172.21.15.119:0/4190150884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba2015df20 0x7fba20142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.261+0000 7fba1f400640 1 -- 172.21.15.119:0/4190150884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba201471b0 msgr2=0x7fba201406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.261+0000 7fba1f400640 1 --2- 172.21.15.119:0/4190150884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba201471b0 0x7fba201406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.261+0000 7fba1f400640 1 -- 172.21.15.119:0/4190150884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba2015df20 msgr2=0x7fba20142190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.261+0000 7fba1f400640 1 --2- 172.21.15.119:0/4190150884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba2015df20 0x7fba20142190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.261+0000 7fba1f400640 1 -- 172.21.15.119:0/4190150884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba201410f0 con 0x7fba200710b0 2024-04-25T10:08:10.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.261+0000 7fba1f400640 1 --2- 172.21.15.119:0/4190150884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba200710b0 0x7fba2012ddd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba10000c00 tx=0x7fba100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.262+0000 7fba17400640 1 -- 172.21.15.119:0/4190150884 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10041070 con 0x7fba200710b0 2024-04-25T10:08:10.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.262+0000 7fba17400640 1 -- 172.21.15.119:0/4190150884 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1003c870 con 0x7fba200710b0 2024-04-25T10:08:10.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.262+0000 7fba25200640 1 -- 172.21.15.119:0/4190150884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba201413d0 con 0x7fba200710b0 2024-04-25T10:08:10.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.262+0000 7fba17400640 1 -- 172.21.15.119:0/4190150884 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10037040 con 0x7fba200710b0 2024-04-25T10:08:10.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.262+0000 7fba25200640 1 -- 172.21.15.119:0/4190150884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba20141990 con 0x7fba200710b0 2024-04-25T10:08:10.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.263+0000 7fba25200640 1 -- 172.21.15.119:0/4190150884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba20141dd0 con 0x7fba200710b0 2024-04-25T10:08:10.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.264+0000 7fba17400640 1 -- 172.21.15.119:0/4190150884 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba10044070 con 0x7fba200710b0 2024-04-25T10:08:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.265+0000 7fba17400640 1 --2- 172.21.15.119:0/4190150884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb9fc06b4d0 0x7fb9fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.265+0000 7fba17400640 1 -- 172.21.15.119:0/4190150884 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fba100aab20 con 0x7fba200710b0 2024-04-25T10:08:10.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.265+0000 7fba1ea00640 1 --2- 172.21.15.119:0/4190150884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb9fc06b4d0 0x7fb9fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.266+0000 7fba1ea00640 1 --2- 172.21.15.119:0/4190150884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb9fc06b4d0 0x7fb9fc06d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fba2013f570 tx=0x7fba08006bf0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.270+0000 7fba17400640 1 -- 172.21.15.119:0/4190150884 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba20147590 con 0x7fba200710b0 2024-04-25T10:08:10.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.534+0000 7fba25200640 1 -- 172.21.15.119:0/4190150884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fba200749b0 con 0x7fba200710b0 2024-04-25T10:08:10.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.538+0000 7fba17400640 1 -- 172.21.15.119:0/4190150884 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fba10079c50 con 0x7fba200710b0 2024-04-25T10:08:10.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.541+0000 7fba25200640 1 -- 172.21.15.119:0/4190150884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb9fc06b4d0 msgr2=0x7fb9fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.541+0000 7fba25200640 1 --2- 172.21.15.119:0/4190150884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb9fc06b4d0 0x7fb9fc06d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fba2013f570 tx=0x7fba08006bf0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.541+0000 7fba25200640 1 -- 172.21.15.119:0/4190150884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba200710b0 msgr2=0x7fba2012ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.541+0000 7fba25200640 1 --2- 172.21.15.119:0/4190150884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba200710b0 0x7fba2012ddd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba10000c00 tx=0x7fba100046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.541+0000 7fba25200640 1 -- 172.21.15.119:0/4190150884 shutdown_connections 2024-04-25T10:08:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.541+0000 7fba25200640 1 --2- 172.21.15.119:0/4190150884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb9fc06b4d0 0x7fb9fc06d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.541+0000 7fba25200640 1 --2- 172.21.15.119:0/4190150884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba2015df20 0x7fba20142190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.541+0000 7fba25200640 1 --2- 172.21.15.119:0/4190150884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba201471b0 0x7fba201406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.541+0000 7fba25200640 1 --2- 172.21.15.119:0/4190150884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba200710b0 0x7fba2012ddd0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.541+0000 7fba25200640 1 -- 172.21.15.119:0/4190150884 >> 172.21.15.119:0/4190150884 conn(0x7fba200b12e0 msgr2=0x7fba20160150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.542+0000 7fba25200640 1 -- 172.21.15.119:0/4190150884 shutdown_connections 2024-04-25T10:08:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.542+0000 7fba25200640 1 -- 172.21.15.119:0/4190150884 wait complete. 2024-04-25T10:08:10.567 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:10.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.750+0000 7fc4b6e00640 1 Processor -- start 2024-04-25T10:08:10.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.750+0000 7fc4b6e00640 1 -- start start 2024-04-25T10:08:10.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.753+0000 7fc4b6e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4b009a150 0x7fc4b006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.753+0000 7fc4b6e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4b006c1f0 0x7fc4b00cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.753+0000 7fc4b6e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b006c680 con 0x7fc4b015ba10 2024-04-25T10:08:10.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.753+0000 7fc4b6e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b00cbe50 con 0x7fc4b006c1f0 2024-04-25T10:08:10.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.753+0000 7fc4b6e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b00cfe80 con 0x7fc4b009a150 2024-04-25T10:08:10.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.753+0000 7fc4b4e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4b006c1f0 0x7fc4b00cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.753+0000 7fc4b4e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4b006c1f0 0x7fc4b00cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45460/0 (socket says 172.21.15.119:45460) 2024-04-25T10:08:10.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.753+0000 7fc4b4e00640 1 -- 172.21.15.119:0/1371489653 learned_addr learned my addr 172.21.15.119:0/1371489653 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:10.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.753+0000 7fc4a7e00640 1 --2- 172.21.15.119:0/1371489653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4b009a150 0x7fc4b006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.753+0000 7fc4b4e00640 1 -- 172.21.15.119:0/1371489653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4b009a150 msgr2=0x7fc4b006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.753+0000 7fc4b4e00640 1 --2- 172.21.15.119:0/1371489653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4b009a150 0x7fc4b006bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.753+0000 7fc4b4e00640 1 -- 172.21.15.119:0/1371489653 >> v1:172.21.15.132:6789/0 conn(0x7fc4b015ba10 legacy=0x7fc4b0099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.753+0000 7fc4b4e00640 1 -- 172.21.15.119:0/1371489653 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b006c900 con 0x7fc4b006c1f0 2024-04-25T10:08:10.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.754+0000 7fc4b4e00640 1 --2- 172.21.15.119:0/1371489653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4b006c1f0 0x7fc4b00cd220 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009e50 tx=0x7fc4a802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe6df05dcdfffe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.754+0000 7fc4af400640 1 -- 172.21.15.119:0/1371489653 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8036070 con 0x7fc4b006c1f0 2024-04-25T10:08:10.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.754+0000 7fc4af400640 1 -- 172.21.15.119:0/1371489653 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a8031600 con 0x7fc4b006c1f0 2024-04-25T10:08:10.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.754+0000 7fc4af400640 1 -- 172.21.15.119:0/1371489653 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80319d0 con 0x7fc4b006c1f0 2024-04-25T10:08:10.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.755+0000 7fc4b6e00640 1 -- 172.21.15.119:0/1371489653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4b006c1f0 msgr2=0x7fc4b00cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.755+0000 7fc4b6e00640 1 --2- 172.21.15.119:0/1371489653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4b006c1f0 0x7fc4b00cd220 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009e50 tx=0x7fc4a802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.755+0000 7fc4b6e00640 1 -- 172.21.15.119:0/1371489653 shutdown_connections 2024-04-25T10:08:10.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.755+0000 7fc4b6e00640 1 --2- 172.21.15.119:0/1371489653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4b006c1f0 0x7fc4b00cd220 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.755+0000 7fc4b6e00640 1 --2- 172.21.15.119:0/1371489653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4b009a150 0x7fc4b006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.755+0000 7fc4b6e00640 1 -- 172.21.15.119:0/1371489653 >> 172.21.15.119:0/1371489653 conn(0x7fc4b00b32f0 msgr2=0x7fc4b00b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.755+0000 7fc4b6e00640 1 -- 172.21.15.119:0/1371489653 shutdown_connections 2024-04-25T10:08:10.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.755+0000 7fc4b6e00640 1 -- 172.21.15.119:0/1371489653 wait complete. 2024-04-25T10:08:10.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.757+0000 7fc4b6e00640 1 Processor -- start 2024-04-25T10:08:10.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.757+0000 7fc4b6e00640 1 -- start start 2024-04-25T10:08:10.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.758+0000 7fc4b6e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4b006c1f0 0x7fc4b00af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.758+0000 7fc4b6e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4b009a150 0x7fc4b01589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.758+0000 7fc4b6e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4b015ba10 0x7fc4b0160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.758+0000 7fc4b6e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b00cd760 con 0x7fc4b009a150 2024-04-25T10:08:10.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.758+0000 7fc4b6e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b006cec0 con 0x7fc4b006c1f0 2024-04-25T10:08:10.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.758+0000 7fc4b6e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b00cd8e0 con 0x7fc4b015ba10 2024-04-25T10:08:10.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.758+0000 7fc4a7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4b009a150 0x7fc4b01589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.758+0000 7fc4a7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4b009a150 0x7fc4b01589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45466/0 (socket says 172.21.15.119:45466) 2024-04-25T10:08:10.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.758+0000 7fc4a7e00640 1 -- 172.21.15.119:0/235027618 learned_addr learned my addr 172.21.15.119:0/235027618 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:10.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.759+0000 7fc4afe00640 1 --2- 172.21.15.119:0/235027618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4b006c1f0 0x7fc4b00af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.759+0000 7fc4a7e00640 1 -- 172.21.15.119:0/235027618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4b015ba10 msgr2=0x7fc4b0160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.759+0000 7fc4a7e00640 1 --2- 172.21.15.119:0/235027618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4b015ba10 0x7fc4b0160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.759+0000 7fc4a7e00640 1 -- 172.21.15.119:0/235027618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4b006c1f0 msgr2=0x7fc4b00af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.759+0000 7fc4a7e00640 1 --2- 172.21.15.119:0/235027618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4b006c1f0 0x7fc4b00af6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.759+0000 7fc4a7e00640 1 -- 172.21.15.119:0/235027618 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b01f85c0 con 0x7fc4b009a150 2024-04-25T10:08:10.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.759+0000 7fc4afe00640 1 --2- 172.21.15.119:0/235027618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4b006c1f0 0x7fc4b00af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:10.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.759+0000 7fc4a7e00640 1 --2- 172.21.15.119:0/235027618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4b009a150 0x7fc4b01589e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc49800d800 tx=0x7fc49800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.759+0000 7fc4acc00640 1 -- 172.21.15.119:0/235027618 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49800b6c0 con 0x7fc4b009a150 2024-04-25T10:08:10.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.759+0000 7fc4acc00640 1 -- 172.21.15.119:0/235027618 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc498019070 con 0x7fc4b009a150 2024-04-25T10:08:10.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.759+0000 7fc4acc00640 1 -- 172.21.15.119:0/235027618 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49800ba80 con 0x7fc4b009a150 2024-04-25T10:08:10.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.759+0000 7fc4b6e00640 1 -- 172.21.15.119:0/235027618 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b00afc10 con 0x7fc4b009a150 2024-04-25T10:08:10.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.759+0000 7fc4b6e00640 1 -- 172.21.15.119:0/235027618 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b01f8000 con 0x7fc4b009a150 2024-04-25T10:08:10.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.760+0000 7fc4b6e00640 1 -- 172.21.15.119:0/235027618 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b00749b0 con 0x7fc4b009a150 2024-04-25T10:08:10.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.761+0000 7fc4acc00640 1 -- 172.21.15.119:0/235027618 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc49800bc20 con 0x7fc4b009a150 2024-04-25T10:08:10.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.762+0000 7fc4acc00640 1 --2- 172.21.15.119:0/235027618 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc48c06b4d0 0x7fc48c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.762+0000 7fc4acc00640 1 -- 172.21.15.119:0/235027618 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc49808a400 con 0x7fc4b009a150 2024-04-25T10:08:10.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.763+0000 7fc4afe00640 1 --2- 172.21.15.119:0/235027618 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc48c06b4d0 0x7fc48c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.763+0000 7fc4afe00640 1 --2- 172.21.15.119:0/235027618 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc48c06b4d0 0x7fc48c06d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000a830 tx=0x7fc4a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.766+0000 7fc4acc00640 1 -- 172.21.15.119:0/235027618 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc498059530 con 0x7fc4b009a150 2024-04-25T10:08:10.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.949+0000 7fc4b6e00640 1 -- 172.21.15.119:0/235027618 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4b009a570 con 0x7fc4b009a150 2024-04-25T10:08:10.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.951+0000 7fc4acc00640 1 -- 172.21.15.119:0/235027618 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12927 (secure 0 0 0) 0x7fc498058e30 con 0x7fc4b009a150 2024-04-25T10:08:10.957 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:10.957 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":29,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:07.032373+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:10.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.953+0000 7fc4b6e00640 1 -- 172.21.15.119:0/235027618 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc48c06b4d0 msgr2=0x7fc48c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.953+0000 7fc4b6e00640 1 --2- 172.21.15.119:0/235027618 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc48c06b4d0 0x7fc48c06d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000a830 tx=0x7fc4a0008040 comp rx=0 tx=0).stop 2024-04-25T10:08:10.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.953+0000 7fc4b6e00640 1 -- 172.21.15.119:0/235027618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4b009a150 msgr2=0x7fc4b01589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.953+0000 7fc4b6e00640 1 --2- 172.21.15.119:0/235027618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4b009a150 0x7fc4b01589e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc49800d800 tx=0x7fc49800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.953+0000 7fc4b6e00640 1 -- 172.21.15.119:0/235027618 shutdown_connections 2024-04-25T10:08:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.953+0000 7fc4b6e00640 1 --2- 172.21.15.119:0/235027618 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc48c06b4d0 0x7fc48c06d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.953+0000 7fc4b6e00640 1 --2- 172.21.15.119:0/235027618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4b015ba10 0x7fc4b0160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.953+0000 7fc4b6e00640 1 --2- 172.21.15.119:0/235027618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4b009a150 0x7fc4b01589e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.953+0000 7fc4b6e00640 1 --2- 172.21.15.119:0/235027618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4b006c1f0 0x7fc4b00af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.953+0000 7fc4b6e00640 1 -- 172.21.15.119:0/235027618 >> 172.21.15.119:0/235027618 conn(0x7fc4b00b32f0 msgr2=0x7fc4b015acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.954+0000 7fc4b6e00640 1 -- 172.21.15.119:0/235027618 shutdown_connections 2024-04-25T10:08:10.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:10.954+0000 7fc4b6e00640 1 -- 172.21.15.119:0/235027618 wait complete. 2024-04-25T10:08:10.976 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:10.977 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:11.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.168+0000 7fe4a4800640 1 Processor -- start 2024-04-25T10:08:11.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.168+0000 7fe4a4800640 1 -- start start 2024-04-25T10:08:11.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.170+0000 7fe4a4800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe49c06ec20 0x7fe49c0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.170+0000 7fe4a4800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe49c06aba0 0x7fe49c0cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.170+0000 7fe4a4800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49c147a30 con 0x7fe49c147650 2024-04-25T10:08:11.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.170+0000 7fe4a4800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c06d780 con 0x7fe49c06ec20 2024-04-25T10:08:11.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.170+0000 7fe4a4800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c06b110 con 0x7fe49c06aba0 2024-04-25T10:08:11.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.170+0000 7fe4a1e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe49c147650 0x7fe49c0720e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39672/0 (socket says 172.21.15.119:39672) 2024-04-25T10:08:11.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.170+0000 7fe4a1e00640 1 -- 172.21.15.119:0/1588521338 learned_addr learned my addr 172.21.15.119:0/1588521338 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:11.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a1400640 1 --2- 172.21.15.119:0/1588521338 >> v2:172.21.15.132:3300/0 conn(0x7fe49c06ec20 0x7fe49c0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a2800640 1 --2- 172.21.15.119:0/1588521338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe49c06aba0 0x7fe49c0cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 -- 172.21.15.119:0/1588521338 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290922486 0 0) 0x7fe49c147a30 con 0x7fe49c147650 2024-04-25T10:08:11.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 -- 172.21.15.119:0/1588521338 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe484003520 con 0x7fe49c147650 2024-04-25T10:08:11.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 -- 172.21.15.119:0/1588521338 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3252374238 0 0) 0x7fe484003520 con 0x7fe49c147650 2024-04-25T10:08:11.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 -- 172.21.15.119:0/1588521338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe49c06aba0 msgr2=0x7fe49c0cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 --2- 172.21.15.119:0/1588521338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe49c06aba0 0x7fe49c0cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 -- 172.21.15.119:0/1588521338 >> v2:172.21.15.132:3300/0 conn(0x7fe49c06ec20 msgr2=0x7fe49c0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 --2- 172.21.15.119:0/1588521338 >> v2:172.21.15.132:3300/0 conn(0x7fe49c06ec20 0x7fe49c0cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 -- 172.21.15.119:0/1588521338 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c06b3f0 con 0x7fe49c147650 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 -- 172.21.15.119:0/1588521338 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fe490001670 con 0x7fe49c147650 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 -- 172.21.15.119:0/1588521338 >> v1:172.21.15.119:6789/0 conn(0x7fe49c147650 legacy=0x7fe49c0720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 --2- 172.21.15.119:0/1588521338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4840039b0 0x7fe484003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 --2- 172.21.15.119:0/1588521338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe484004800 0x7fe484004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 --2- 172.21.15.119:0/1588521338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe49c06aba0 0x7fe4840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 -- 172.21.15.119:0/1588521338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c06d780 con 0x7fe484004800 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 -- 172.21.15.119:0/1588521338 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c06b110 con 0x7fe4840039b0 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.171+0000 7fe4a0a00640 1 -- 172.21.15.119:0/1588521338 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe48400b9f0 con 0x7fe49c06aba0 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.172+0000 7fe4a1400640 1 --2- 172.21.15.119:0/1588521338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe484004800 0x7fe484004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.172+0000 7fe4a1e00640 1 --2- 172.21.15.119:0/1588521338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4840039b0 0x7fe484003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.172+0000 7fe4a1400640 1 -- 172.21.15.119:0/1588521338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe49c06aba0 msgr2=0x7fe4840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.172+0000 7fe4a1400640 1 --2- 172.21.15.119:0/1588521338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe49c06aba0 0x7fe4840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.172+0000 7fe4a1400640 1 -- 172.21.15.119:0/1588521338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4840039b0 msgr2=0x7fe484003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.172+0000 7fe4a1400640 1 --2- 172.21.15.119:0/1588521338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4840039b0 0x7fe484003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.172+0000 7fe4a1400640 1 -- 172.21.15.119:0/1588521338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe484003520 con 0x7fe484004800 2024-04-25T10:08:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.173+0000 7fe4a1400640 1 --2- 172.21.15.119:0/1588521338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe484004800 0x7fe484004c80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009a90 tx=0x7fe484005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6303d3af9472c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.173+0000 7fe4a0a00640 1 -- 172.21.15.119:0/1588521338 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c036070 con 0x7fe484004800 2024-04-25T10:08:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.173+0000 7fe4a0a00640 1 -- 172.21.15.119:0/1588521338 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c031400 con 0x7fe484004800 2024-04-25T10:08:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.173+0000 7fe4a0a00640 1 -- 172.21.15.119:0/1588521338 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c009690 con 0x7fe484004800 2024-04-25T10:08:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.174+0000 7fe4a4800640 1 -- 172.21.15.119:0/1588521338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe484004800 msgr2=0x7fe484004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.174+0000 7fe4a4800640 1 --2- 172.21.15.119:0/1588521338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe484004800 0x7fe484004c80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009a90 tx=0x7fe484005830 comp rx=0 tx=0).stop 2024-04-25T10:08:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.174+0000 7fe4a2800640 1 -- 172.21.15.119:0/1588521338 reap_dead start 2024-04-25T10:08:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.174+0000 7fe4a4800640 1 -- 172.21.15.119:0/1588521338 shutdown_connections 2024-04-25T10:08:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.174+0000 7fe4a4800640 1 -- 172.21.15.119:0/1588521338 >> 172.21.15.119:0/1588521338 conn(0x7fe49c0cc5e0 msgr2=0x7fe49c0cc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.174+0000 7fe4a4800640 1 -- 172.21.15.119:0/1588521338 shutdown_connections 2024-04-25T10:08:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.174+0000 7fe4a4800640 1 -- 172.21.15.119:0/1588521338 wait complete. 2024-04-25T10:08:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.176+0000 7fe4a4800640 1 Processor -- start 2024-04-25T10:08:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.176+0000 7fe4a4800640 1 -- start start 2024-04-25T10:08:11.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.177+0000 7fe4a4800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe484004800 0x7fe49c141810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.177+0000 7fe4a4800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe49c0b00a0 0x7fe49c0b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.177+0000 7fe4a4800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe49c0b4280 0x7fe49c0b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.177+0000 7fe4a4800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0cdc50 con 0x7fe484004800 2024-04-25T10:08:11.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.177+0000 7fe4a4800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c06b9b0 con 0x7fe49c0b00a0 2024-04-25T10:08:11.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.177+0000 7fe4a4800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0cddd0 con 0x7fe49c0b4280 2024-04-25T10:08:11.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.177+0000 7fe4a1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe484004800 0x7fe49c141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.177+0000 7fe4a1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe484004800 0x7fe49c141810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34262/0 (socket says 172.21.15.119:34262) 2024-04-25T10:08:11.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.177+0000 7fe4a1e00640 1 -- 172.21.15.119:0/4036288607 learned_addr learned my addr 172.21.15.119:0/4036288607 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:11.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.177+0000 7fe4a1e00640 1 -- 172.21.15.119:0/4036288607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe49c0b4280 msgr2=0x7fe49c0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.177+0000 7fe4a1e00640 1 --2- 172.21.15.119:0/4036288607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe49c0b4280 0x7fe49c0b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.177+0000 7fe4a1e00640 1 -- 172.21.15.119:0/4036288607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe49c0b00a0 msgr2=0x7fe49c0b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.177+0000 7fe4a1e00640 1 --2- 172.21.15.119:0/4036288607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe49c0b00a0 0x7fe49c0b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.177+0000 7fe4a1e00640 1 -- 172.21.15.119:0/4036288607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c0b4c40 con 0x7fe484004800 2024-04-25T10:08:11.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.178+0000 7fe4a1e00640 1 --2- 172.21.15.119:0/4036288607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe484004800 0x7fe49c141810 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe49000e840 tx=0x7fe49000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.178+0000 7fe496000640 1 -- 172.21.15.119:0/4036288607 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49000c850 con 0x7fe484004800 2024-04-25T10:08:11.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.178+0000 7fe496000640 1 -- 172.21.15.119:0/4036288607 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe490018070 con 0x7fe484004800 2024-04-25T10:08:11.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.178+0000 7fe4a4800640 1 -- 172.21.15.119:0/4036288607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49c0b2a20 con 0x7fe484004800 2024-04-25T10:08:11.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.178+0000 7fe496000640 1 -- 172.21.15.119:0/4036288607 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49000cc30 con 0x7fe484004800 2024-04-25T10:08:11.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.178+0000 7fe4a4800640 1 -- 172.21.15.119:0/4036288607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c0b2fe0 con 0x7fe484004800 2024-04-25T10:08:11.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.179+0000 7fe4a4800640 1 -- 172.21.15.119:0/4036288607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49c0750b0 con 0x7fe484004800 2024-04-25T10:08:11.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.180+0000 7fe496000640 1 -- 172.21.15.119:0/4036288607 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe490013420 con 0x7fe484004800 2024-04-25T10:08:11.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.181+0000 7fe496000640 1 --2- 172.21.15.119:0/4036288607 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe48006b500 0x7fe48006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.181+0000 7fe496000640 1 -- 172.21.15.119:0/4036288607 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe490089a50 con 0x7fe484004800 2024-04-25T10:08:11.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.181+0000 7fe4a1400640 1 --2- 172.21.15.119:0/4036288607 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe48006b500 0x7fe48006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.182+0000 7fe4a1400640 1 --2- 172.21.15.119:0/4036288607 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe48006b500 0x7fe48006d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe48c033000 tx=0x7fe48c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.186+0000 7fe496000640 1 -- 172.21.15.119:0/4036288607 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe490058b80 con 0x7fe484004800 2024-04-25T10:08:11.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.378+0000 7fe4a4800640 1 -- 172.21.15.119:0/4036288607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe49c0cfe00 con 0x7fe484004800 2024-04-25T10:08:11.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.378+0000 7fe496000640 1 -- 172.21.15.119:0/4036288607 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe490058480 con 0x7fe484004800 2024-04-25T10:08:11.384 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:08:11.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.381+0000 7fe4a4800640 1 -- 172.21.15.119:0/4036288607 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe48006b500 msgr2=0x7fe48006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.381+0000 7fe4a4800640 1 --2- 172.21.15.119:0/4036288607 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe48006b500 0x7fe48006d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe48c033000 tx=0x7fe48c02e000 comp rx=0 tx=0).stop 2024-04-25T10:08:11.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.381+0000 7fe4a4800640 1 -- 172.21.15.119:0/4036288607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe484004800 msgr2=0x7fe49c141810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.381+0000 7fe4a4800640 1 --2- 172.21.15.119:0/4036288607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe484004800 0x7fe49c141810 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe49000e840 tx=0x7fe49000ed00 comp rx=0 tx=0).stop 2024-04-25T10:08:11.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.381+0000 7fe4a4800640 1 -- 172.21.15.119:0/4036288607 shutdown_connections 2024-04-25T10:08:11.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.381+0000 7fe4a4800640 1 --2- 172.21.15.119:0/4036288607 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe48006b500 0x7fe48006d9c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.381+0000 7fe4a4800640 1 --2- 172.21.15.119:0/4036288607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe49c0b4280 0x7fe49c0b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.381+0000 7fe4a4800640 1 --2- 172.21.15.119:0/4036288607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe49c0b00a0 0x7fe49c0b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.381+0000 7fe4a4800640 1 --2- 172.21.15.119:0/4036288607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe484004800 0x7fe49c141810 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.381+0000 7fe4a4800640 1 -- 172.21.15.119:0/4036288607 >> 172.21.15.119:0/4036288607 conn(0x7fe49c0cc5e0 msgr2=0x7fe49c144530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.381+0000 7fe4a4800640 1 -- 172.21.15.119:0/4036288607 shutdown_connections 2024-04-25T10:08:11.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.381+0000 7fe4a4800640 1 -- 172.21.15.119:0/4036288607 wait complete. 2024-04-25T10:08:11.403 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:08:11.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.583+0000 7f97f0a00640 1 Processor -- start 2024-04-25T10:08:11.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.583+0000 7f97f0a00640 1 -- start start 2024-04-25T10:08:11.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.584+0000 7f97f0a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f97ec155b90 0x7f97ec0b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.584+0000 7f97f0a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ec155f70 0x7f97ec0ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.584+0000 7f97f0a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec141010 con 0x7f97ec06d060 2024-04-25T10:08:11.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.584+0000 7f97f0a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ec1412f0 con 0x7f97ec155b90 2024-04-25T10:08:11.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.584+0000 7f97f0a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec1415d0 con 0x7f97ec155f70 2024-04-25T10:08:11.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.585+0000 7f97ea800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f97ec155b90 0x7f97ec0b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.585+0000 7f97ea800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f97ec155b90 0x7f97ec0b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:51606/0 (socket says 172.21.15.119:51606) 2024-04-25T10:08:11.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.585+0000 7f97ea800640 1 -- 172.21.15.119:0/2763933228 learned_addr learned my addr 172.21.15.119:0/2763933228 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:11.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.585+0000 7f97eb200640 1 --2- 172.21.15.119:0/2763933228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ec155f70 0x7f97ec0ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.585+0000 7f97ea800640 1 -- 172.21.15.119:0/2763933228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ec155f70 msgr2=0x7f97ec0ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.585+0000 7f97ea800640 1 --2- 172.21.15.119:0/2763933228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ec155f70 0x7f97ec0ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.585+0000 7f97ea800640 1 -- 172.21.15.119:0/2763933228 >> v1:172.21.15.132:6789/0 conn(0x7f97ec06d060 legacy=0x7f97ec0cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.585+0000 7f97ea800640 1 -- 172.21.15.119:0/2763933228 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec1418b0 con 0x7f97ec155b90 2024-04-25T10:08:11.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.587+0000 7f97ea800640 1 --2- 172.21.15.119:0/2763933228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97ec155b90 0x7f97ec0b1100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009870 tx=0x7f97dc02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deb9754ec8a6cac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.587+0000 7f97e9400640 1 -- 172.21.15.119:0/2763933228 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc037070 con 0x7f97ec155b90 2024-04-25T10:08:11.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.587+0000 7f97e9400640 1 -- 172.21.15.119:0/2763933228 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97dc032440 con 0x7f97ec155b90 2024-04-25T10:08:11.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.587+0000 7f97e9400640 1 -- 172.21.15.119:0/2763933228 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc032810 con 0x7f97ec155b90 2024-04-25T10:08:11.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.588+0000 7f97f0a00640 1 -- 172.21.15.119:0/2763933228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97ec155b90 msgr2=0x7f97ec0b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.588+0000 7f97f0a00640 1 --2- 172.21.15.119:0/2763933228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97ec155b90 0x7f97ec0b1100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009870 tx=0x7f97dc02a130 comp rx=0 tx=0).stop 2024-04-25T10:08:11.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.588+0000 7f97f0a00640 1 -- 172.21.15.119:0/2763933228 shutdown_connections 2024-04-25T10:08:11.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.588+0000 7f97f0a00640 1 --2- 172.21.15.119:0/2763933228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ec155f70 0x7f97ec0ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.588+0000 7f97f0a00640 1 --2- 172.21.15.119:0/2763933228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97ec155b90 0x7f97ec0b1100 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.588+0000 7f97f0a00640 1 -- 172.21.15.119:0/2763933228 >> 172.21.15.119:0/2763933228 conn(0x7f97ec0b5070 msgr2=0x7f97ec0cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.588+0000 7f97f0a00640 1 -- 172.21.15.119:0/2763933228 shutdown_connections 2024-04-25T10:08:11.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.588+0000 7f97f0a00640 1 -- 172.21.15.119:0/2763933228 wait complete. 2024-04-25T10:08:11.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.590+0000 7f97f0a00640 1 Processor -- start 2024-04-25T10:08:11.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.591+0000 7f97f0a00640 1 -- start start 2024-04-25T10:08:11.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.592+0000 7f97f0a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97ec06d060 0x7f97ec1e4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.592+0000 7f97f0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ec155b90 0x7f97ec15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.592+0000 7f97f0a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ec155f70 0x7f97ec1f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.592+0000 7f97f0a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0b17c0 con 0x7f97ec155b90 2024-04-25T10:08:11.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.592+0000 7f97f0a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0b1640 con 0x7f97ec06d060 2024-04-25T10:08:11.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.592+0000 7f97f0a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0b1940 con 0x7f97ec155f70 2024-04-25T10:08:11.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.592+0000 7f97e9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ec155b90 0x7f97ec15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.592+0000 7f97e9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ec155b90 0x7f97ec15d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34278/0 (socket says 172.21.15.119:34278) 2024-04-25T10:08:11.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.592+0000 7f97e9e00640 1 -- 172.21.15.119:0/2054797595 learned_addr learned my addr 172.21.15.119:0/2054797595 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:11.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.592+0000 7f97ea800640 1 --2- 172.21.15.119:0/2054797595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97ec06d060 0x7f97ec1e4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.593+0000 7f97eb200640 1 --2- 172.21.15.119:0/2054797595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ec155f70 0x7f97ec1f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.593+0000 7f97e9e00640 1 -- 172.21.15.119:0/2054797595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ec155f70 msgr2=0x7f97ec1f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.593+0000 7f97e9e00640 1 --2- 172.21.15.119:0/2054797595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ec155f70 0x7f97ec1f6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.593+0000 7f97e9e00640 1 -- 172.21.15.119:0/2054797595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97ec06d060 msgr2=0x7f97ec1e4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.593+0000 7f97e9e00640 1 --2- 172.21.15.119:0/2054797595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97ec06d060 0x7f97ec1e4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.593+0000 7f97e9e00640 1 -- 172.21.15.119:0/2054797595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec1e4660 con 0x7f97ec155b90 2024-04-25T10:08:11.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.593+0000 7f97e9e00640 1 --2- 172.21.15.119:0/2054797595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ec155b90 0x7f97ec15d9a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f97d400c440 tx=0x7f97d400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.593+0000 7f97e2a00640 1 -- 172.21.15.119:0/2054797595 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d40052d0 con 0x7f97ec155b90 2024-04-25T10:08:11.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.593+0000 7f97e2a00640 1 -- 172.21.15.119:0/2054797595 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d4005470 con 0x7f97ec155b90 2024-04-25T10:08:11.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.593+0000 7f97e2a00640 1 -- 172.21.15.119:0/2054797595 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4007260 con 0x7f97ec155b90 2024-04-25T10:08:11.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.593+0000 7f97f0a00640 1 -- 172.21.15.119:0/2054797595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ec1e4940 con 0x7f97ec155b90 2024-04-25T10:08:11.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.593+0000 7f97f0a00640 1 -- 172.21.15.119:0/2054797595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec1e4f00 con 0x7f97ec155b90 2024-04-25T10:08:11.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.595+0000 7f97f0a00640 1 -- 172.21.15.119:0/2054797595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ec0749b0 con 0x7f97ec155b90 2024-04-25T10:08:11.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.595+0000 7f97e2a00640 1 -- 172.21.15.119:0/2054797595 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97d4007400 con 0x7f97ec155b90 2024-04-25T10:08:11.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.596+0000 7f97e2a00640 1 --2- 172.21.15.119:0/2054797595 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f97c806b500 0x7f97c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.596+0000 7f97e2a00640 1 -- 172.21.15.119:0/2054797595 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f97d4089360 con 0x7f97ec155b90 2024-04-25T10:08:11.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.596+0000 7f97ea800640 1 --2- 172.21.15.119:0/2054797595 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f97c806b500 0x7f97c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.597+0000 7f97ea800640 1 --2- 172.21.15.119:0/2054797595 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f97c806b500 0x7f97c806d9c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f97dc034000 tx=0x7f97dc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.600+0000 7f97e2a00640 1 -- 172.21.15.119:0/2054797595 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97d4058490 con 0x7f97ec155b90 2024-04-25T10:08:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.795+0000 7f97f0a00640 1 -- 172.21.15.119:0/2054797595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f97ec0b5750 con 0x7f97ec155b90 2024-04-25T10:08:11.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.796+0000 7f97e2a00640 1 -- 172.21.15.119:0/2054797595 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f97d4057d90 con 0x7f97ec155b90 2024-04-25T10:08:11.802 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:11.802 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2024-04-25T10:08:11.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:11.802 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2024-04-25T10:08:11.802 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2024-04-25T10:08:11.803 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:11.803 INFO:teuthology.orchestra.run.smithi119.stdout:] 2024-04-25T10:08:11.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.798+0000 7f97f0a00640 1 -- 172.21.15.119:0/2054797595 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f97c806b500 msgr2=0x7f97c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.798+0000 7f97f0a00640 1 --2- 172.21.15.119:0/2054797595 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f97c806b500 0x7f97c806d9c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f97dc034000 tx=0x7f97dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:08:11.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.798+0000 7f97f0a00640 1 -- 172.21.15.119:0/2054797595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ec155b90 msgr2=0x7f97ec15d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.798+0000 7f97f0a00640 1 --2- 172.21.15.119:0/2054797595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ec155b90 0x7f97ec15d9a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f97d400c440 tx=0x7f97d400c900 comp rx=0 tx=0).stop 2024-04-25T10:08:11.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.799+0000 7f97f0a00640 1 -- 172.21.15.119:0/2054797595 shutdown_connections 2024-04-25T10:08:11.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.799+0000 7f97f0a00640 1 --2- 172.21.15.119:0/2054797595 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f97c806b500 0x7f97c806d9c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.799+0000 7f97f0a00640 1 --2- 172.21.15.119:0/2054797595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97ec155f70 0x7f97ec1f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.799+0000 7f97f0a00640 1 --2- 172.21.15.119:0/2054797595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97ec155b90 0x7f97ec15d9a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.799+0000 7f97f0a00640 1 --2- 172.21.15.119:0/2054797595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97ec06d060 0x7f97ec1e4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.799+0000 7f97f0a00640 1 -- 172.21.15.119:0/2054797595 >> 172.21.15.119:0/2054797595 conn(0x7f97ec0b5070 msgr2=0x7f97ec0af0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.799+0000 7f97f0a00640 1 -- 172.21.15.119:0/2054797595 shutdown_connections 2024-04-25T10:08:11.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:11.799+0000 7f97f0a00640 1 -- 172.21.15.119:0/2054797595 wait complete. 2024-04-25T10:08:11.822 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:08:11.822 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:08:12.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.018+0000 7f19f5000640 1 Processor -- start 2024-04-25T10:08:12.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.018+0000 7f19f5000640 1 -- start start 2024-04-25T10:08:12.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.019+0000 7f19f5000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f19f0144fd0 0x7f19f015f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.019+0000 7f19f5000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19f00cf180 0x7f19f00ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.019+0000 7f19f5000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f015bdf0 con 0x7f19f006e670 2024-04-25T10:08:12.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.019+0000 7f19f5000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f006d4e0 con 0x7f19f0144fd0 2024-04-25T10:08:12.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.019+0000 7f19f5000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f006d7c0 con 0x7f19f00cf180 2024-04-25T10:08:12.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.019+0000 7f19ef400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f19f0144fd0 0x7f19f015f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.019+0000 7f19efe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f19f006e670 0x7f19f015c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39686/0 (socket says 172.21.15.119:39686) 2024-04-25T10:08:12.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.019+0000 7f19efe00640 1 -- 172.21.15.119:0/1225334794 learned_addr learned my addr 172.21.15.119:0/1225334794 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:12.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.019+0000 7f19ef400640 1 -- 172.21.15.119:0/1225334794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19f00cf180 msgr2=0x7f19f00ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:12.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.019+0000 7f19ef400640 1 --2- 172.21.15.119:0/1225334794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19f00cf180 0x7f19f00ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.019+0000 7f19ef400640 1 -- 172.21.15.119:0/1225334794 >> v1:172.21.15.119:6789/0 conn(0x7f19f006e670 legacy=0x7f19f015c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.019+0000 7f19ef400640 1 -- 172.21.15.119:0/1225334794 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f006daa0 con 0x7f19f0144fd0 2024-04-25T10:08:12.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.019+0000 7f19eea00640 1 -- 172.21.15.119:0/1225334794 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093486070 0 0) 0x7f19f015bdf0 con 0x7f19f006e670 2024-04-25T10:08:12.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.020+0000 7f19ef400640 1 --2- 172.21.15.119:0/1225334794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19f0144fd0 0x7f19f015f400 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f19e00097a0 tx=0x7f19e002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af16faf35f12276 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.021+0000 7f19eea00640 1 -- 172.21.15.119:0/1225334794 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0037070 con 0x7f19f0144fd0 2024-04-25T10:08:12.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.021+0000 7f19eea00640 1 -- 172.21.15.119:0/1225334794 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e002ae50 con 0x7f19f0144fd0 2024-04-25T10:08:12.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.021+0000 7f19eea00640 1 -- 172.21.15.119:0/1225334794 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0032660 con 0x7f19f0144fd0 2024-04-25T10:08:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.021+0000 7f19f5000640 1 -- 172.21.15.119:0/1225334794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19f0144fd0 msgr2=0x7f19f015f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.021+0000 7f19f5000640 1 --2- 172.21.15.119:0/1225334794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19f0144fd0 0x7f19f015f400 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f19e00097a0 tx=0x7f19e002a100 comp rx=0 tx=0).stop 2024-04-25T10:08:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.022+0000 7f19f5000640 1 -- 172.21.15.119:0/1225334794 shutdown_connections 2024-04-25T10:08:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.022+0000 7f19f5000640 1 --2- 172.21.15.119:0/1225334794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19f00cf180 0x7f19f00ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.022+0000 7f19f5000640 1 --2- 172.21.15.119:0/1225334794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19f0144fd0 0x7f19f015f400 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.022+0000 7f19f5000640 1 -- 172.21.15.119:0/1225334794 >> 172.21.15.119:0/1225334794 conn(0x7f19f006aba0 msgr2=0x7f19f006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.022+0000 7f19f5000640 1 -- 172.21.15.119:0/1225334794 shutdown_connections 2024-04-25T10:08:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.022+0000 7f19f5000640 1 -- 172.21.15.119:0/1225334794 wait complete. 2024-04-25T10:08:12.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.023+0000 7f19f5000640 1 Processor -- start 2024-04-25T10:08:12.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.023+0000 7f19f5000640 1 -- start start 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.024+0000 7f19f5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19f006e670 0x7f19f01e71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.024+0000 7f19f5000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19f00cf180 0x7f19f00cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.024+0000 7f19f5000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19f0144fd0 0x7f19f015b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.024+0000 7f19f5000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f015f940 con 0x7f19f006e670 2024-04-25T10:08:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.024+0000 7f19f5000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f006d190 con 0x7f19f0144fd0 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.024+0000 7f19f5000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f01453b0 con 0x7f19f00cf180 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.024+0000 7f19ef400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19f006e670 0x7f19f01e71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.025+0000 7f19ef400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19f006e670 0x7f19f01e71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34298/0 (socket says 172.21.15.119:34298) 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.025+0000 7f19ef400640 1 -- 172.21.15.119:0/1133694011 learned_addr learned my addr 172.21.15.119:0/1133694011 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.025+0000 7f19ef400640 1 -- 172.21.15.119:0/1133694011 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19f00cf180 msgr2=0x7f19f00cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.025+0000 7f19ef400640 1 --2- 172.21.15.119:0/1133694011 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19f00cf180 0x7f19f00cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.025+0000 7f19ef400640 1 -- 172.21.15.119:0/1133694011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19f0144fd0 msgr2=0x7f19f015b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.025+0000 7f19ef400640 1 --2- 172.21.15.119:0/1133694011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19f0144fd0 0x7f19f015b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.025+0000 7f19ef400640 1 -- 172.21.15.119:0/1133694011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f00cda50 con 0x7f19f006e670 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.025+0000 7f19ef400640 1 --2- 172.21.15.119:0/1133694011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19f006e670 0x7f19f01e71b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f19e00097a0 tx=0x7f19e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.025+0000 7f19e7400640 1 -- 172.21.15.119:0/1133694011 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0042070 con 0x7f19f006e670 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.025+0000 7f19e7400640 1 -- 172.21.15.119:0/1133694011 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e003e670 con 0x7f19f006e670 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.025+0000 7f19f5000640 1 -- 172.21.15.119:0/1133694011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f01e49b0 con 0x7f19f006e670 2024-04-25T10:08:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.025+0000 7f19e7400640 1 -- 172.21.15.119:0/1133694011 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0037040 con 0x7f19f006e670 2024-04-25T10:08:12.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.026+0000 7f19f5000640 1 -- 172.21.15.119:0/1133694011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f00ccd10 con 0x7f19f006e670 2024-04-25T10:08:12.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.027+0000 7f19f5000640 1 -- 172.21.15.119:0/1133694011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f00ccb30 con 0x7f19f006e670 2024-04-25T10:08:12.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.028+0000 7f19e7400640 1 -- 172.21.15.119:0/1133694011 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19e0045070 con 0x7f19f006e670 2024-04-25T10:08:12.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.028+0000 7f19e7400640 1 --2- 172.21.15.119:0/1133694011 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f19cc06b4d0 0x7f19cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.029+0000 7f19e7400640 1 -- 172.21.15.119:0/1133694011 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f19e00aa630 con 0x7f19f006e670 2024-04-25T10:08:12.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.029+0000 7f19e7e00640 1 --2- 172.21.15.119:0/1133694011 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f19cc06b4d0 0x7f19cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.029+0000 7f19e7e00640 1 --2- 172.21.15.119:0/1133694011 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f19cc06b4d0 0x7f19cc06d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f19f00ce090 tx=0x7f19d8006c90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.033+0000 7f19e7400640 1 -- 172.21.15.119:0/1133694011 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19f00ccb30 con 0x7f19f006e670 2024-04-25T10:08:12.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:12.218+0000 7f19f5000640 1 -- 172.21.15.119:0/1133694011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f19f00b59c0 con 0x7f19f006e670 2024-04-25T10:08:13.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.062+0000 7f19e7400640 1 -- 172.21.15.119:0/1133694011 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f19f00b59c0 con 0x7f19f006e670 2024-04-25T10:08:13.067 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2024-04-25T10:08:13.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.065+0000 7f19f5000640 1 -- 172.21.15.119:0/1133694011 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f19cc06b4d0 msgr2=0x7f19cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.065+0000 7f19f5000640 1 --2- 172.21.15.119:0/1133694011 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f19cc06b4d0 0x7f19cc06d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f19f00ce090 tx=0x7f19d8006c90 comp rx=0 tx=0).stop 2024-04-25T10:08:13.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.065+0000 7f19f5000640 1 -- 172.21.15.119:0/1133694011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19f006e670 msgr2=0x7f19f01e71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.065+0000 7f19f5000640 1 --2- 172.21.15.119:0/1133694011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19f006e670 0x7f19f01e71b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f19e00097a0 tx=0x7f19e00046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.066+0000 7f19f5000640 1 -- 172.21.15.119:0/1133694011 shutdown_connections 2024-04-25T10:08:13.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.066+0000 7f19f5000640 1 --2- 172.21.15.119:0/1133694011 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f19cc06b4d0 0x7f19cc06d990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.066+0000 7f19f5000640 1 --2- 172.21.15.119:0/1133694011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19f0144fd0 0x7f19f015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.066+0000 7f19f5000640 1 --2- 172.21.15.119:0/1133694011 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19f00cf180 0x7f19f00cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.066+0000 7f19f5000640 1 --2- 172.21.15.119:0/1133694011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19f006e670 0x7f19f01e71b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.066+0000 7f19f5000640 1 -- 172.21.15.119:0/1133694011 >> 172.21.15.119:0/1133694011 conn(0x7f19f006aba0 msgr2=0x7f19f0140680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.066+0000 7f19f5000640 1 -- 172.21.15.119:0/1133694011 shutdown_connections 2024-04-25T10:08:13.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.067+0000 7f19f5000640 1 -- 172.21.15.119:0/1133694011 wait complete. 2024-04-25T10:08:13.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:08:13.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.284+0000 7f5785c00640 1 Processor -- start 2024-04-25T10:08:13.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.284+0000 7f5785c00640 1 -- start start 2024-04-25T10:08:13.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.285+0000 7f5785c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f57800aed80 0x7f5780162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.285+0000 7f5785c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f578015dcd0 0x7f578015e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.285+0000 7f5785c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57800ae1d0 con 0x7f578015d8f0 2024-04-25T10:08:13.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.285+0000 7f5785c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57800715c0 con 0x7f57800aed80 2024-04-25T10:08:13.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.285+0000 7f5785c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57800718a0 con 0x7f578015dcd0 2024-04-25T10:08:13.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.286+0000 7f5776000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f57800aed80 0x7f5780162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.286+0000 7f5776000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f57800aed80 0x7f5780162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:51644/0 (socket says 172.21.15.119:51644) 2024-04-25T10:08:13.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.286+0000 7f5776000640 1 -- 172.21.15.119:0/4293129129 learned_addr learned my addr 172.21.15.119:0/4293129129 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:13.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.286+0000 7f577f400640 1 --2- 172.21.15.119:0/4293129129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f578015dcd0 0x7f578015e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.287+0000 7f577f400640 1 -- 172.21.15.119:0/4293129129 >> v2:172.21.15.132:3300/0 conn(0x7f57800aed80 msgr2=0x7f5780162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.287+0000 7f577f400640 1 --2- 172.21.15.119:0/4293129129 >> v2:172.21.15.132:3300/0 conn(0x7f57800aed80 0x7f5780162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.287+0000 7f577f400640 1 -- 172.21.15.119:0/4293129129 >> v1:172.21.15.132:6789/0 conn(0x7f578015d8f0 legacy=0x7f57800ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.287+0000 7f577f400640 1 -- 172.21.15.119:0/4293129129 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5780071b80 con 0x7f578015dcd0 2024-04-25T10:08:13.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.289+0000 7f577f400640 1 --2- 172.21.15.119:0/4293129129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f578015dcd0 0x7f578015e140 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f57780097a0 tx=0x7f577802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=176fc63b78c01964 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.289+0000 7f577e000640 1 -- 172.21.15.119:0/4293129129 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778037070 con 0x7f578015dcd0 2024-04-25T10:08:13.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.290+0000 7f577e000640 1 -- 172.21.15.119:0/4293129129 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5778032440 con 0x7f578015dcd0 2024-04-25T10:08:13.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.290+0000 7f577e000640 1 -- 172.21.15.119:0/4293129129 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778032810 con 0x7f578015dcd0 2024-04-25T10:08:13.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.290+0000 7f5785c00640 1 -- 172.21.15.119:0/4293129129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f578015dcd0 msgr2=0x7f578015e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.290+0000 7f5785c00640 1 --2- 172.21.15.119:0/4293129129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f578015dcd0 0x7f578015e140 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f57780097a0 tx=0x7f577802a180 comp rx=0 tx=0).stop 2024-04-25T10:08:13.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.290+0000 7f5785c00640 1 -- 172.21.15.119:0/4293129129 shutdown_connections 2024-04-25T10:08:13.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.290+0000 7f5785c00640 1 --2- 172.21.15.119:0/4293129129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f578015dcd0 0x7f578015e140 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.290+0000 7f5785c00640 1 --2- 172.21.15.119:0/4293129129 >> v2:172.21.15.132:3300/0 conn(0x7f57800aed80 0x7f5780162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.290+0000 7f5785c00640 1 -- 172.21.15.119:0/4293129129 >> 172.21.15.119:0/4293129129 conn(0x7f578006aba0 msgr2=0x7f578006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.290+0000 7f5785c00640 1 -- 172.21.15.119:0/4293129129 shutdown_connections 2024-04-25T10:08:13.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.291+0000 7f5785c00640 1 -- 172.21.15.119:0/4293129129 wait complete. 2024-04-25T10:08:13.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.292+0000 7f5785c00640 1 Processor -- start 2024-04-25T10:08:13.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.292+0000 7f5785c00640 1 -- start start 2024-04-25T10:08:13.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.293+0000 7f5785c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57800aed80 0x7f5780141290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.293+0000 7f5785c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578015d8f0 0x7f57800cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.293+0000 7f5785c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f578015dcd0 0x7f578015c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.293+0000 7f5785c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578006dc70 con 0x7f578015d8f0 2024-04-25T10:08:13.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.293+0000 7f5785c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5780072140 con 0x7f57800aed80 2024-04-25T10:08:13.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.293+0000 7f5785c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5780071220 con 0x7f578015dcd0 2024-04-25T10:08:13.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.294+0000 7f5776000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578015d8f0 0x7f57800cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.294+0000 7f5776000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578015d8f0 0x7f57800cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34300/0 (socket says 172.21.15.119:34300) 2024-04-25T10:08:13.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.294+0000 7f5776000640 1 -- 172.21.15.119:0/1402826968 learned_addr learned my addr 172.21.15.119:0/1402826968 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:13.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.294+0000 7f577ea00640 1 --2- 172.21.15.119:0/1402826968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57800aed80 0x7f5780141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.294+0000 7f577f400640 1 --2- 172.21.15.119:0/1402826968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f578015dcd0 0x7f578015c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.294+0000 7f5776000640 1 -- 172.21.15.119:0/1402826968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f578015dcd0 msgr2=0x7f578015c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.294+0000 7f5776000640 1 --2- 172.21.15.119:0/1402826968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f578015dcd0 0x7f578015c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.294+0000 7f5776000640 1 -- 172.21.15.119:0/1402826968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57800aed80 msgr2=0x7f5780141290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.294+0000 7f5776000640 1 --2- 172.21.15.119:0/1402826968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57800aed80 0x7f5780141290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.294+0000 7f5776000640 1 -- 172.21.15.119:0/1402826968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578015b970 con 0x7f578015d8f0 2024-04-25T10:08:13.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.295+0000 7f5776000640 1 --2- 172.21.15.119:0/1402826968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578015d8f0 0x7f57800cb840 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f576800e870 tx=0x7f576800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.295+0000 7f5777400640 1 -- 172.21.15.119:0/1402826968 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768017070 con 0x7f578015d8f0 2024-04-25T10:08:13.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.295+0000 7f5777400640 1 -- 172.21.15.119:0/1402826968 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576800c8e0 con 0x7f578015d8f0 2024-04-25T10:08:13.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.295+0000 7f5785c00640 1 -- 172.21.15.119:0/1402826968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578015bc10 con 0x7f578015d8f0 2024-04-25T10:08:13.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.295+0000 7f5777400640 1 -- 172.21.15.119:0/1402826968 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576800cc90 con 0x7f578015d8f0 2024-04-25T10:08:13.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.295+0000 7f5785c00640 1 -- 172.21.15.119:0/1402826968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578015c1d0 con 0x7f578015d8f0 2024-04-25T10:08:13.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.296+0000 7f5785c00640 1 -- 172.21.15.119:0/1402826968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57800749b0 con 0x7f578015d8f0 2024-04-25T10:08:13.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.298+0000 7f5777400640 1 -- 172.21.15.119:0/1402826968 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f576801a070 con 0x7f578015d8f0 2024-04-25T10:08:13.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.299+0000 7f5777400640 1 --2- 172.21.15.119:0/1402826968 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f575c06b4d0 0x7f575c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.299+0000 7f5777400640 1 -- 172.21.15.119:0/1402826968 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f576808a450 con 0x7f578015d8f0 2024-04-25T10:08:13.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.299+0000 7f577ea00640 1 --2- 172.21.15.119:0/1402826968 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f575c06b4d0 0x7f575c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.299+0000 7f577ea00640 1 --2- 172.21.15.119:0/1402826968 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f575c06b4d0 0x7f575c06d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5770001d40 tx=0x7f5770009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.304+0000 7f5777400640 1 -- 172.21.15.119:0/1402826968 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5768059390 con 0x7f578015d8f0 2024-04-25T10:08:13.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:13.497+0000 7f5785c00640 1 -- 172.21.15.119:0/1402826968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5780158290 con 0x7f578015d8f0 2024-04-25T10:08:14.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.077+0000 7f5777400640 1 -- 172.21.15.119:0/1402826968 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5768058c90 con 0x7f578015d8f0 2024-04-25T10:08:14.084 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2024-04-25T10:08:14.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.083+0000 7f5785c00640 1 -- 172.21.15.119:0/1402826968 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f575c06b4d0 msgr2=0x7f575c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.083+0000 7f5785c00640 1 --2- 172.21.15.119:0/1402826968 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f575c06b4d0 0x7f575c06d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5770001d40 tx=0x7f5770009040 comp rx=0 tx=0).stop 2024-04-25T10:08:14.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.083+0000 7f5785c00640 1 -- 172.21.15.119:0/1402826968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578015d8f0 msgr2=0x7f57800cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.083+0000 7f5785c00640 1 --2- 172.21.15.119:0/1402826968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578015d8f0 0x7f57800cb840 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f576800e870 tx=0x7f576800ed30 comp rx=0 tx=0).stop 2024-04-25T10:08:14.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.083+0000 7f5785c00640 1 -- 172.21.15.119:0/1402826968 shutdown_connections 2024-04-25T10:08:14.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.083+0000 7f5785c00640 1 --2- 172.21.15.119:0/1402826968 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f575c06b4d0 0x7f575c06d990 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5770001d40 tx=0x7f5770009040 comp rx=0 tx=0).stop 2024-04-25T10:08:14.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.083+0000 7f5785c00640 1 --2- 172.21.15.119:0/1402826968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f578015dcd0 0x7f578015c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.083+0000 7f5785c00640 1 --2- 172.21.15.119:0/1402826968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578015d8f0 0x7f57800cb840 secure :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f576800e870 tx=0x7f576800ed30 comp rx=0 tx=0).stop 2024-04-25T10:08:14.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.083+0000 7f5785c00640 1 --2- 172.21.15.119:0/1402826968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57800aed80 0x7f5780141290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.083+0000 7f5785c00640 1 -- 172.21.15.119:0/1402826968 >> 172.21.15.119:0/1402826968 conn(0x7f578006aba0 msgr2=0x7f57801615b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:14.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.084+0000 7f5785c00640 1 -- 172.21.15.119:0/1402826968 shutdown_connections 2024-04-25T10:08:14.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.085+0000 7f5785c00640 1 -- 172.21.15.119:0/1402826968 wait complete. 2024-04-25T10:08:14.129 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:08:14.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.300+0000 7fdb79200640 1 Processor -- start 2024-04-25T10:08:14.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.300+0000 7fdb79200640 1 -- start start 2024-04-25T10:08:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.301+0000 7fdb79200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdb741450e0 0x7fdb7406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.301+0000 7fdb79200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb740cc9e0 0x7fdb740b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.301+0000 7fdb79200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7415f400 con 0x7fdb7406b0e0 2024-04-25T10:08:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.301+0000 7fdb79200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb741454c0 con 0x7fdb741450e0 2024-04-25T10:08:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.301+0000 7fdb79200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7406b540 con 0x7fdb740cc9e0 2024-04-25T10:08:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.302+0000 7fdb72a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdb7406b0e0 0x7fdb740cc2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39692/0 (socket says 172.21.15.119:39692) 2024-04-25T10:08:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.302+0000 7fdb72a00640 1 -- 172.21.15.119:0/371187458 learned_addr learned my addr 172.21.15.119:0/371187458 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:14.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.302+0000 7fdb73e00640 1 --2- 172.21.15.119:0/371187458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb740cc9e0 0x7fdb740b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.302+0000 7fdb73400640 1 --2- 172.21.15.119:0/371187458 >> v2:172.21.15.132:3300/0 conn(0x7fdb741450e0 0x7fdb7406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.302+0000 7fdb72000640 1 -- 172.21.15.119:0/371187458 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247865518 0 0) 0x7fdb7415f400 con 0x7fdb7406b0e0 2024-04-25T10:08:14.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.302+0000 7fdb72000640 1 -- 172.21.15.119:0/371187458 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb58003520 con 0x7fdb7406b0e0 2024-04-25T10:08:14.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb72000640 1 -- 172.21.15.119:0/371187458 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2778474223 0 0) 0x7fdb58003520 con 0x7fdb7406b0e0 2024-04-25T10:08:14.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb72000640 1 -- 172.21.15.119:0/371187458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb740cc9e0 msgr2=0x7fdb740b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:14.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb72000640 1 --2- 172.21.15.119:0/371187458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb740cc9e0 0x7fdb740b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb72000640 1 -- 172.21.15.119:0/371187458 >> v2:172.21.15.132:3300/0 conn(0x7fdb741450e0 msgr2=0x7fdb7406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb72000640 1 --2- 172.21.15.119:0/371187458 >> v2:172.21.15.132:3300/0 conn(0x7fdb741450e0 0x7fdb7406aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb72000640 1 -- 172.21.15.119:0/371187458 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7415f6e0 con 0x7fdb7406b0e0 2024-04-25T10:08:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb72000640 1 -- 172.21.15.119:0/371187458 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fdb60001690 con 0x7fdb7406b0e0 2024-04-25T10:08:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb72000640 1 -- 172.21.15.119:0/371187458 >> v1:172.21.15.119:6789/0 conn(0x7fdb7406b0e0 legacy=0x7fdb740cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb72000640 1 --2- 172.21.15.119:0/371187458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb580039b0 0x7fdb58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb72000640 1 --2- 172.21.15.119:0/371187458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb58004800 0x7fdb58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb72000640 1 --2- 172.21.15.119:0/371187458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb740cc9e0 0x7fdb580051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb72000640 1 -- 172.21.15.119:0/371187458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5800baa0 con 0x7fdb580039b0 2024-04-25T10:08:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb72000640 1 -- 172.21.15.119:0/371187458 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5800bc50 con 0x7fdb58004800 2024-04-25T10:08:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb72000640 1 -- 172.21.15.119:0/371187458 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5800be50 con 0x7fdb740cc9e0 2024-04-25T10:08:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.303+0000 7fdb73400640 1 --2- 172.21.15.119:0/371187458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb580039b0 0x7fdb58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.304+0000 7fdb72a00640 1 --2- 172.21.15.119:0/371187458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb58004800 0x7fdb58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.304+0000 7fdb73400640 1 -- 172.21.15.119:0/371187458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb740cc9e0 msgr2=0x7fdb580051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.304+0000 7fdb73400640 1 --2- 172.21.15.119:0/371187458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb740cc9e0 0x7fdb580051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.304+0000 7fdb73400640 1 -- 172.21.15.119:0/371187458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb58004800 msgr2=0x7fdb58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:14.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.304+0000 7fdb73400640 1 --2- 172.21.15.119:0/371187458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb58004800 0x7fdb58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.304+0000 7fdb73400640 1 -- 172.21.15.119:0/371187458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb58003520 con 0x7fdb580039b0 2024-04-25T10:08:14.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.305+0000 7fdb73400640 1 --2- 172.21.15.119:0/371187458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb580039b0 0x7fdb58003dd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400ba30 tx=0x7fdb58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cb217d3ede5291 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.305+0000 7fdb72000640 1 -- 172.21.15.119:0/371187458 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64030070 con 0x7fdb580039b0 2024-04-25T10:08:14.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.305+0000 7fdb72000640 1 -- 172.21.15.119:0/371187458 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6402ae10 con 0x7fdb580039b0 2024-04-25T10:08:14.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.305+0000 7fdb72000640 1 -- 172.21.15.119:0/371187458 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6400b600 con 0x7fdb580039b0 2024-04-25T10:08:14.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.306+0000 7fdb79200640 1 -- 172.21.15.119:0/371187458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb580039b0 msgr2=0x7fdb58003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.306+0000 7fdb79200640 1 --2- 172.21.15.119:0/371187458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb580039b0 0x7fdb58003dd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400ba30 tx=0x7fdb58005830 comp rx=0 tx=0).stop 2024-04-25T10:08:14.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.306+0000 7fdb73e00640 1 -- 172.21.15.119:0/371187458 reap_dead start 2024-04-25T10:08:14.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.306+0000 7fdb79200640 1 -- 172.21.15.119:0/371187458 shutdown_connections 2024-04-25T10:08:14.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.306+0000 7fdb79200640 1 -- 172.21.15.119:0/371187458 >> 172.21.15.119:0/371187458 conn(0x7fdb74066ff0 msgr2=0x7fdb740b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:14.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.306+0000 7fdb79200640 1 -- 172.21.15.119:0/371187458 shutdown_connections 2024-04-25T10:08:14.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.306+0000 7fdb79200640 1 -- 172.21.15.119:0/371187458 wait complete. 2024-04-25T10:08:14.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.308+0000 7fdb79200640 1 Processor -- start 2024-04-25T10:08:14.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.308+0000 7fdb79200640 1 -- start start 2024-04-25T10:08:14.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.309+0000 7fdb79200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb580039b0 0x7fdb7413be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.309+0000 7fdb79200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb74142390 0x7fdb74140570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.309+0000 7fdb79200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb74140ab0 0x7fdb7413cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.309+0000 7fdb79200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb740b2e80 con 0x7fdb74140ab0 2024-04-25T10:08:14.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.309+0000 7fdb79200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7415fca0 con 0x7fdb580039b0 2024-04-25T10:08:14.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.309+0000 7fdb79200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb740b3000 con 0x7fdb74142390 2024-04-25T10:08:14.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.309+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb74140ab0 0x7fdb7413cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.309+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb74140ab0 0x7fdb7413cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34316/0 (socket says 172.21.15.119:34316) 2024-04-25T10:08:14.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.309+0000 7fdb73e00640 1 -- 172.21.15.119:0/3437860957 learned_addr learned my addr 172.21.15.119:0/3437860957 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:14.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.310+0000 7fdb72a00640 1 --2- 172.21.15.119:0/3437860957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb74142390 0x7fdb74140570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.310+0000 7fdb73400640 1 --2- 172.21.15.119:0/3437860957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb580039b0 0x7fdb7413be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.310+0000 7fdb73e00640 1 -- 172.21.15.119:0/3437860957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb74142390 msgr2=0x7fdb74140570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.310+0000 7fdb73e00640 1 --2- 172.21.15.119:0/3437860957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb74142390 0x7fdb74140570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.310+0000 7fdb73e00640 1 -- 172.21.15.119:0/3437860957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb580039b0 msgr2=0x7fdb7413be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.310+0000 7fdb73e00640 1 --2- 172.21.15.119:0/3437860957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb580039b0 0x7fdb7413be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.310+0000 7fdb73e00640 1 -- 172.21.15.119:0/3437860957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7415bee0 con 0x7fdb74140ab0 2024-04-25T10:08:14.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.310+0000 7fdb73e00640 1 --2- 172.21.15.119:0/3437860957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb74140ab0 0x7fdb7413cfc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdb740af070 tx=0x7fdb6c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.310+0000 7fdb6b400640 1 -- 172.21.15.119:0/3437860957 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c018020 con 0x7fdb74140ab0 2024-04-25T10:08:14.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.310+0000 7fdb6b400640 1 -- 172.21.15.119:0/3437860957 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6c00bce0 con 0x7fdb74140ab0 2024-04-25T10:08:14.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.310+0000 7fdb6b400640 1 -- 172.21.15.119:0/3437860957 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c014640 con 0x7fdb74140ab0 2024-04-25T10:08:14.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.310+0000 7fdb79200640 1 -- 172.21.15.119:0/3437860957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7415c1c0 con 0x7fdb74140ab0 2024-04-25T10:08:14.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.311+0000 7fdb79200640 1 -- 172.21.15.119:0/3437860957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7415c780 con 0x7fdb74140ab0 2024-04-25T10:08:14.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.312+0000 7fdb79200640 1 -- 172.21.15.119:0/3437860957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb740749b0 con 0x7fdb74140ab0 2024-04-25T10:08:14.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.313+0000 7fdb6b400640 1 -- 172.21.15.119:0/3437860957 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb6c005d10 con 0x7fdb74140ab0 2024-04-25T10:08:14.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.314+0000 7fdb6b400640 1 --2- 172.21.15.119:0/3437860957 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb5406b4d0 0x7fdb5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.314+0000 7fdb6b400640 1 -- 172.21.15.119:0/3437860957 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fdb6c08a980 con 0x7fdb74140ab0 2024-04-25T10:08:14.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.314+0000 7fdb73400640 1 --2- 172.21.15.119:0/3437860957 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb5406b4d0 0x7fdb5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.315+0000 7fdb73400640 1 --2- 172.21.15.119:0/3437860957 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb5406b4d0 0x7fdb5406d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400ba00 tx=0x7fdb64030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.318+0000 7fdb6b400640 1 -- 172.21.15.119:0/3437860957 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb6c022030 con 0x7fdb74140ab0 2024-04-25T10:08:14.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:14.551+0000 7fdb79200640 1 -- 172.21.15.119:0/3437860957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdb7406ae60 con 0x7fdb74140ab0 2024-04-25T10:08:15.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.095+0000 7fdb6b400640 1 -- 172.21.15.119:0/3437860957 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdb6c059850 con 0x7fdb74140ab0 2024-04-25T10:08:15.102 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T10:08:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.106+0000 7fdb79200640 1 -- 172.21.15.119:0/3437860957 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb5406b4d0 msgr2=0x7fdb5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.106+0000 7fdb79200640 1 --2- 172.21.15.119:0/3437860957 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb5406b4d0 0x7fdb5406d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400ba00 tx=0x7fdb64030000 comp rx=0 tx=0).stop 2024-04-25T10:08:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.106+0000 7fdb79200640 1 -- 172.21.15.119:0/3437860957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb74140ab0 msgr2=0x7fdb7413cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.106+0000 7fdb79200640 1 --2- 172.21.15.119:0/3437860957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb74140ab0 0x7fdb7413cfc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdb740af070 tx=0x7fdb6c00b480 comp rx=0 tx=0).stop 2024-04-25T10:08:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.107+0000 7fdb79200640 1 -- 172.21.15.119:0/3437860957 shutdown_connections 2024-04-25T10:08:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.107+0000 7fdb79200640 1 --2- 172.21.15.119:0/3437860957 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb5406b4d0 0x7fdb5406d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.107+0000 7fdb79200640 1 --2- 172.21.15.119:0/3437860957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb74140ab0 0x7fdb7413cfc0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.107+0000 7fdb79200640 1 --2- 172.21.15.119:0/3437860957 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb74142390 0x7fdb74140570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.107+0000 7fdb79200640 1 --2- 172.21.15.119:0/3437860957 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb580039b0 0x7fdb7413be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.107+0000 7fdb79200640 1 -- 172.21.15.119:0/3437860957 >> 172.21.15.119:0/3437860957 conn(0x7fdb74066ff0 msgr2=0x7fdb740cfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:15.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.108+0000 7fdb79200640 1 -- 172.21.15.119:0/3437860957 shutdown_connections 2024-04-25T10:08:15.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.109+0000 7fdb79200640 1 -- 172.21.15.119:0/3437860957 wait complete. 2024-04-25T10:08:15.151 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.314+0000 7f357b800640 1 Processor -- start 2024-04-25T10:08:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.314+0000 7f357b800640 1 -- start start 2024-04-25T10:08:15.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.315+0000 7f357b800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f357415d980 0x7f35740ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.316+0000 7f357b800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35740712c0 con 0x7f35740aeb20 2024-04-25T10:08:15.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.316+0000 7f357b800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35740715a0 con 0x7f357415ed70 2024-04-25T10:08:15.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.316+0000 7f357b800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3574071ed0 con 0x7f357415d980 2024-04-25T10:08:15.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.316+0000 7f3578e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f357415d980 0x7f35740ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.316+0000 7f3573e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f35740aeb20 0x7f35740cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39696/0 (socket says 172.21.15.119:39696) 2024-04-25T10:08:15.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.316+0000 7f3573e00640 1 -- 172.21.15.119:0/910360859 learned_addr learned my addr 172.21.15.119:0/910360859 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:15.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.316+0000 7f3578e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f357415d980 0x7f35740ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34318/0 (socket says 172.21.15.119:34318) 2024-04-25T10:08:15.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.316+0000 7f3578e00640 1 -- 172.21.15.119:0/910360859 >> v1:172.21.15.132:6789/0 conn(0x7f357415ed70 legacy=0x7f35740acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:15.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.316+0000 7f3578e00640 1 -- 172.21.15.119:0/910360859 >> v1:172.21.15.119:6789/0 conn(0x7f35740aeb20 legacy=0x7f35740cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.317+0000 7f3578e00640 1 -- 172.21.15.119:0/910360859 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3574071880 con 0x7f357415d980 2024-04-25T10:08:15.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.318+0000 7f3578e00640 1 --2- 172.21.15.119:0/910360859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f357415d980 0x7f35740ae5e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3574072050 tx=0x7f356802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd2f1a686a4dc81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.318+0000 7f3573400640 1 -- 172.21.15.119:0/910360859 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568037070 con 0x7f357415d980 2024-04-25T10:08:15.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.318+0000 7f3573400640 1 -- 172.21.15.119:0/910360859 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356802ae40 con 0x7f357415d980 2024-04-25T10:08:15.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.318+0000 7f3573400640 1 -- 172.21.15.119:0/910360859 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568032660 con 0x7f357415d980 2024-04-25T10:08:15.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.319+0000 7f357b800640 1 -- 172.21.15.119:0/910360859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f357415d980 msgr2=0x7f35740ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.319+0000 7f357b800640 1 --2- 172.21.15.119:0/910360859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f357415d980 0x7f35740ae5e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3574072050 tx=0x7f356802a130 comp rx=0 tx=0).stop 2024-04-25T10:08:15.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.319+0000 7f357b800640 1 -- 172.21.15.119:0/910360859 shutdown_connections 2024-04-25T10:08:15.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.319+0000 7f357b800640 1 --2- 172.21.15.119:0/910360859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f357415d980 0x7f35740ae5e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.319+0000 7f357b800640 1 -- 172.21.15.119:0/910360859 >> 172.21.15.119:0/910360859 conn(0x7f357406aba0 msgr2=0x7f357406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:15.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.319+0000 7f357b800640 1 -- 172.21.15.119:0/910360859 shutdown_connections 2024-04-25T10:08:15.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.319+0000 7f357b800640 1 -- 172.21.15.119:0/910360859 wait complete. 2024-04-25T10:08:15.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.320+0000 7f357b800640 1 Processor -- start 2024-04-25T10:08:15.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.321+0000 7f357b800640 1 -- start start 2024-04-25T10:08:15.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.321+0000 7f357b800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35740aeb20 0x7f3574150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.321+0000 7f357b800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f357415d980 0x7f357415bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.321+0000 7f357b800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f357414fa50 0x7f3574152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.322+0000 7f357b800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35740cfb10 con 0x7f357415d980 2024-04-25T10:08:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.322+0000 7f357b800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35740cf990 con 0x7f35740aeb20 2024-04-25T10:08:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.322+0000 7f357b800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35740ad680 con 0x7f357414fa50 2024-04-25T10:08:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.322+0000 7f3573e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f357415d980 0x7f357415bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.322+0000 7f3573e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f357415d980 0x7f357415bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34320/0 (socket says 172.21.15.119:34320) 2024-04-25T10:08:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.322+0000 7f3573e00640 1 -- 172.21.15.119:0/1384719543 learned_addr learned my addr 172.21.15.119:0/1384719543 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.322+0000 7f3579800640 1 --2- 172.21.15.119:0/1384719543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f357414fa50 0x7f3574152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.322+0000 7f3578e00640 1 --2- 172.21.15.119:0/1384719543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35740aeb20 0x7f3574150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.322+0000 7f3573e00640 1 -- 172.21.15.119:0/1384719543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f357414fa50 msgr2=0x7f3574152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.322+0000 7f3573e00640 1 --2- 172.21.15.119:0/1384719543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f357414fa50 0x7f3574152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.322+0000 7f3573e00640 1 -- 172.21.15.119:0/1384719543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35740aeb20 msgr2=0x7f3574150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.322+0000 7f3573e00640 1 --2- 172.21.15.119:0/1384719543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35740aeb20 0x7f3574150dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.322+0000 7f3573e00640 1 -- 172.21.15.119:0/1384719543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35741514b0 con 0x7f357415d980 2024-04-25T10:08:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.323+0000 7f3573e00640 1 --2- 172.21.15.119:0/1384719543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f357415d980 0x7f357415bd30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f356400c400 tx=0x7f356400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.323+0000 7f3570c00640 1 -- 172.21.15.119:0/1384719543 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3564017070 con 0x7f357415d980 2024-04-25T10:08:15.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.323+0000 7f3570c00640 1 -- 172.21.15.119:0/1384719543 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3564005ce0 con 0x7f357415d980 2024-04-25T10:08:15.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.323+0000 7f3570c00640 1 -- 172.21.15.119:0/1384719543 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35640072f0 con 0x7f357415d980 2024-04-25T10:08:15.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.323+0000 7f357b800640 1 -- 172.21.15.119:0/1384719543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3574151790 con 0x7f357415d980 2024-04-25T10:08:15.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.323+0000 7f357b800640 1 -- 172.21.15.119:0/1384719543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3574151d50 con 0x7f357415d980 2024-04-25T10:08:15.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.324+0000 7f357b800640 1 -- 172.21.15.119:0/1384719543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35740749b0 con 0x7f357415d980 2024-04-25T10:08:15.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.326+0000 7f3570c00640 1 -- 172.21.15.119:0/1384719543 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f356401a070 con 0x7f357415d980 2024-04-25T10:08:15.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.327+0000 7f3570c00640 1 --2- 172.21.15.119:0/1384719543 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f355806b4d0 0x7f355806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.327+0000 7f3570c00640 1 -- 172.21.15.119:0/1384719543 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35640896b0 con 0x7f357415d980 2024-04-25T10:08:15.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.327+0000 7f3578e00640 1 --2- 172.21.15.119:0/1384719543 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f355806b4d0 0x7f355806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.328+0000 7f3578e00640 1 --2- 172.21.15.119:0/1384719543 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f355806b4d0 0x7f355806d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3568034000 tx=0x7f356802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.331+0000 7f3570c00640 1 -- 172.21.15.119:0/1384719543 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3564058490 con 0x7f357415d980 2024-04-25T10:08:15.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.520+0000 7f357b800640 1 -- 172.21.15.119:0/1384719543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f357406d4c0 con 0x7f357415d980 2024-04-25T10:08:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.523+0000 7f3570c00640 1 -- 172.21.15.119:0/1384719543 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16718 (secure 0 0 0) 0x7f3564057d90 con 0x7f357415d980 2024-04-25T10:08:15.530 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:15.531 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":32,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:14.553107+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:12.268457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:13.546259+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.526+0000 7f357b800640 1 -- 172.21.15.119:0/1384719543 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f355806b4d0 msgr2=0x7f355806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.526+0000 7f357b800640 1 --2- 172.21.15.119:0/1384719543 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f355806b4d0 0x7f355806d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3568034000 tx=0x7f356802e000 comp rx=0 tx=0).stop 2024-04-25T10:08:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.526+0000 7f357b800640 1 -- 172.21.15.119:0/1384719543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f357415d980 msgr2=0x7f357415bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.526+0000 7f357b800640 1 --2- 172.21.15.119:0/1384719543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f357415d980 0x7f357415bd30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f356400c400 tx=0x7f356400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.526+0000 7f357b800640 1 -- 172.21.15.119:0/1384719543 shutdown_connections 2024-04-25T10:08:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.526+0000 7f357b800640 1 --2- 172.21.15.119:0/1384719543 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f355806b4d0 0x7f355806d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.526+0000 7f357b800640 1 --2- 172.21.15.119:0/1384719543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f357414fa50 0x7f3574152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.526+0000 7f357b800640 1 --2- 172.21.15.119:0/1384719543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f357415d980 0x7f357415bd30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.526+0000 7f357b800640 1 --2- 172.21.15.119:0/1384719543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35740aeb20 0x7f3574150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.526+0000 7f357b800640 1 -- 172.21.15.119:0/1384719543 >> 172.21.15.119:0/1384719543 conn(0x7f357406aba0 msgr2=0x7f3574160810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.526+0000 7f357b800640 1 -- 172.21.15.119:0/1384719543 shutdown_connections 2024-04-25T10:08:15.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.527+0000 7f357b800640 1 -- 172.21.15.119:0/1384719543 wait complete. 2024-04-25T10:08:15.551 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:15.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.733+0000 7fbcdee00640 1 Processor -- start 2024-04-25T10:08:15.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.733+0000 7fbcdee00640 1 -- start start 2024-04-25T10:08:15.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.734+0000 7fbcdee00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbcd8144470 0x7fbcd815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.734+0000 7fbcdee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcd806e430 0x7fbcd80b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.734+0000 7fbcdee00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd80cd440 con 0x7fbcd80b4e80 2024-04-25T10:08:15.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.734+0000 7fbcdee00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd8145340 con 0x7fbcd8144470 2024-04-25T10:08:15.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.734+0000 7fbcdee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd80cd720 con 0x7fbcd806e430 2024-04-25T10:08:15.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.735+0000 7fbcde400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbcd80b4e80 0x7fbcd806d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39698/0 (socket says 172.21.15.119:39698) 2024-04-25T10:08:15.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.735+0000 7fbcde400640 1 -- 172.21.15.119:0/3058449624 learned_addr learned my addr 172.21.15.119:0/3058449624 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:15.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.735+0000 7fbcdda00640 1 --2- 172.21.15.119:0/3058449624 >> v2:172.21.15.132:3300/0 conn(0x7fbcd8144470 0x7fbcd815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.735+0000 7fbcdd000640 1 --2- 172.21.15.119:0/3058449624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcd806e430 0x7fbcd80b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.735+0000 7fbcd7e00640 1 -- 172.21.15.119:0/3058449624 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874954636 0 0) 0x7fbcd80cd440 con 0x7fbcd80b4e80 2024-04-25T10:08:15.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.735+0000 7fbcd7e00640 1 -- 172.21.15.119:0/3058449624 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcc0003520 con 0x7fbcd80b4e80 2024-04-25T10:08:15.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.735+0000 7fbcd7e00640 1 -- 172.21.15.119:0/3058449624 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2534200120 0 0) 0x7fbcc0003520 con 0x7fbcd80b4e80 2024-04-25T10:08:15.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.735+0000 7fbcd7e00640 1 -- 172.21.15.119:0/3058449624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcd806e430 msgr2=0x7fbcd80b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:15.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.735+0000 7fbcd7e00640 1 --2- 172.21.15.119:0/3058449624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcd806e430 0x7fbcd80b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.735+0000 7fbcd7e00640 1 -- 172.21.15.119:0/3058449624 >> v2:172.21.15.132:3300/0 conn(0x7fbcd8144470 msgr2=0x7fbcd815fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:15.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.735+0000 7fbcd7e00640 1 --2- 172.21.15.119:0/3058449624 >> v2:172.21.15.132:3300/0 conn(0x7fbcd8144470 0x7fbcd815fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.735+0000 7fbcd7e00640 1 -- 172.21.15.119:0/3058449624 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd80cda00 con 0x7fbcd80b4e80 2024-04-25T10:08:15.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.735+0000 7fbcd7e00640 1 -- 172.21.15.119:0/3058449624 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fbcd00019f0 con 0x7fbcd80b4e80 2024-04-25T10:08:15.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.736+0000 7fbcd7e00640 1 -- 172.21.15.119:0/3058449624 >> v1:172.21.15.119:6789/0 conn(0x7fbcd80b4e80 legacy=0x7fbcd806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.736+0000 7fbcd7e00640 1 --2- 172.21.15.119:0/3058449624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a20 0x7fbcc0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.736+0000 7fbcd7e00640 1 --2- 172.21.15.119:0/3058449624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcd806e430 0x7fbcc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.736+0000 7fbcd7e00640 1 --2- 172.21.15.119:0/3058449624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcc0004d40 0x7fbcc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.736+0000 7fbcd7e00640 1 -- 172.21.15.119:0/3058449624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd80cd720 con 0x7fbcc0003a20 2024-04-25T10:08:15.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.736+0000 7fbcd7e00640 1 -- 172.21.15.119:0/3058449624 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc000b9c0 con 0x7fbcc0004d40 2024-04-25T10:08:15.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.736+0000 7fbcd7e00640 1 -- 172.21.15.119:0/3058449624 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc000bb40 con 0x7fbcd806e430 2024-04-25T10:08:15.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.736+0000 7fbcdda00640 1 --2- 172.21.15.119:0/3058449624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a20 0x7fbcc0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.736+0000 7fbcdda00640 1 -- 172.21.15.119:0/3058449624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcd806e430 msgr2=0x7fbcc0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:15.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.736+0000 7fbcdda00640 1 --2- 172.21.15.119:0/3058449624 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcd806e430 0x7fbcc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.736+0000 7fbcdda00640 1 -- 172.21.15.119:0/3058449624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcc0004d40 msgr2=0x7fbcc00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:15.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.736+0000 7fbcdda00640 1 --2- 172.21.15.119:0/3058449624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcc0004d40 0x7fbcc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.736+0000 7fbcdda00640 1 -- 172.21.15.119:0/3058449624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc0003520 con 0x7fbcc0003a20 2024-04-25T10:08:15.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.737+0000 7fbcdda00640 1 --2- 172.21.15.119:0/3058449624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a20 0x7fbcc0003e40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009ac0 tx=0x7fbcc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ed205b9a1a830a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.737+0000 7fbcd7e00640 1 -- 172.21.15.119:0/3058449624 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc036070 con 0x7fbcc0003a20 2024-04-25T10:08:15.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.737+0000 7fbcd7e00640 1 -- 172.21.15.119:0/3058449624 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc031400 con 0x7fbcc0003a20 2024-04-25T10:08:15.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.737+0000 7fbcd7e00640 1 -- 172.21.15.119:0/3058449624 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc009690 con 0x7fbcc0003a20 2024-04-25T10:08:15.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.738+0000 7fbcdee00640 1 -- 172.21.15.119:0/3058449624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a20 msgr2=0x7fbcc0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.738+0000 7fbcdee00640 1 --2- 172.21.15.119:0/3058449624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a20 0x7fbcc0003e40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009ac0 tx=0x7fbcc0005830 comp rx=0 tx=0).stop 2024-04-25T10:08:15.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.738+0000 7fbcde400640 1 -- 172.21.15.119:0/3058449624 reap_dead start 2024-04-25T10:08:15.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.738+0000 7fbcdee00640 1 -- 172.21.15.119:0/3058449624 shutdown_connections 2024-04-25T10:08:15.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.738+0000 7fbcdee00640 1 -- 172.21.15.119:0/3058449624 >> 172.21.15.119:0/3058449624 conn(0x7fbcd806b1d0 msgr2=0x7fbcd80b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:15.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.738+0000 7fbcdee00640 1 -- 172.21.15.119:0/3058449624 shutdown_connections 2024-04-25T10:08:15.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.738+0000 7fbcdee00640 1 -- 172.21.15.119:0/3058449624 wait complete. 2024-04-25T10:08:15.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.740+0000 7fbcdee00640 1 Processor -- start 2024-04-25T10:08:15.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.740+0000 7fbcdee00640 1 -- start start 2024-04-25T10:08:15.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.741+0000 7fbcdee00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a20 0x7fbcd81f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.741+0000 7fbcdee00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcd80aca90 0x7fbcd81414b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.741+0000 7fbcdee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcd8152720 0x7fbcd8152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.741+0000 7fbcdee00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd80ce200 con 0x7fbcc0003a20 2024-04-25T10:08:15.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.741+0000 7fbcdee00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd8144850 con 0x7fbcd80aca90 2024-04-25T10:08:15.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.741+0000 7fbcdee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd815cd00 con 0x7fbcd8152720 2024-04-25T10:08:15.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.741+0000 7fbcdda00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a20 0x7fbcd81f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.741+0000 7fbcdda00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a20 0x7fbcd81f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34332/0 (socket says 172.21.15.119:34332) 2024-04-25T10:08:15.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.741+0000 7fbcdda00640 1 -- 172.21.15.119:0/4031327629 learned_addr learned my addr 172.21.15.119:0/4031327629 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:15.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.742+0000 7fbcdd000640 1 --2- 172.21.15.119:0/4031327629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcd80aca90 0x7fbcd81414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.742+0000 7fbcdda00640 1 -- 172.21.15.119:0/4031327629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcd8152720 msgr2=0x7fbcd8152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.742+0000 7fbcdda00640 1 --2- 172.21.15.119:0/4031327629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcd8152720 0x7fbcd8152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.742+0000 7fbcdda00640 1 -- 172.21.15.119:0/4031327629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcd80aca90 msgr2=0x7fbcd81414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.742+0000 7fbcdda00640 1 --2- 172.21.15.119:0/4031327629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcd80aca90 0x7fbcd81414b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.742+0000 7fbcdda00640 1 -- 172.21.15.119:0/4031327629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd81403e0 con 0x7fbcc0003a20 2024-04-25T10:08:15.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.742+0000 7fbcdda00640 1 --2- 172.21.15.119:0/4031327629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a20 0x7fbcd81f78c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbccc034040 tx=0x7fbccc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.742+0000 7fbcd5600640 1 -- 172.21.15.119:0/4031327629 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc040020 con 0x7fbcc0003a20 2024-04-25T10:08:15.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.742+0000 7fbcd5600640 1 -- 172.21.15.119:0/4031327629 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc003b40 con 0x7fbcc0003a20 2024-04-25T10:08:15.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.742+0000 7fbcd5600640 1 -- 172.21.15.119:0/4031327629 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc036040 con 0x7fbcc0003a20 2024-04-25T10:08:15.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.742+0000 7fbcdee00640 1 -- 172.21.15.119:0/4031327629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd80b2e50 con 0x7fbcc0003a20 2024-04-25T10:08:15.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.742+0000 7fbcdee00640 1 -- 172.21.15.119:0/4031327629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd8140960 con 0x7fbcc0003a20 2024-04-25T10:08:15.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.744+0000 7fbcdee00640 1 -- 172.21.15.119:0/4031327629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd80749b0 con 0x7fbcc0003a20 2024-04-25T10:08:15.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.745+0000 7fbcd5600640 1 -- 172.21.15.119:0/4031327629 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbccc043020 con 0x7fbcc0003a20 2024-04-25T10:08:15.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.745+0000 7fbcd5600640 1 --2- 172.21.15.119:0/4031327629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbcb806b500 0x7fbcb806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.746+0000 7fbcd5600640 1 -- 172.21.15.119:0/4031327629 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbccc0aa430 con 0x7fbcc0003a20 2024-04-25T10:08:15.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.746+0000 7fbcdd000640 1 --2- 172.21.15.119:0/4031327629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbcb806b500 0x7fbcb806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.746+0000 7fbcdd000640 1 --2- 172.21.15.119:0/4031327629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbcb806b500 0x7fbcb806d9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80b2da0 tx=0x7fbcc4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.750+0000 7fbcd5600640 1 -- 172.21.15.119:0/4031327629 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbccc04a070 con 0x7fbcc0003a20 2024-04-25T10:08:15.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.929+0000 7fbcdee00640 1 -- 172.21.15.119:0/4031327629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcd80b2090 con 0x7fbcc0003a20 2024-04-25T10:08:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.932+0000 7fbcd5600640 1 -- 172.21.15.119:0/4031327629 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16718 (secure 0 0 0) 0x7fbccc0044e0 con 0x7fbcc0003a20 2024-04-25T10:08:15.938 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":32,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:14.553107+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:12.268457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:13.546259+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.934+0000 7fbcdee00640 1 -- 172.21.15.119:0/4031327629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbcb806b500 msgr2=0x7fbcb806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.934+0000 7fbcdee00640 1 --2- 172.21.15.119:0/4031327629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbcb806b500 0x7fbcb806d9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80b2da0 tx=0x7fbcc4008040 comp rx=0 tx=0).stop 2024-04-25T10:08:15.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.934+0000 7fbcdee00640 1 -- 172.21.15.119:0/4031327629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a20 msgr2=0x7fbcd81f78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.934+0000 7fbcdee00640 1 --2- 172.21.15.119:0/4031327629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a20 0x7fbcd81f78c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbccc034040 tx=0x7fbccc004730 comp rx=0 tx=0).stop 2024-04-25T10:08:15.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.934+0000 7fbcdee00640 1 -- 172.21.15.119:0/4031327629 shutdown_connections 2024-04-25T10:08:15.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.934+0000 7fbcdee00640 1 --2- 172.21.15.119:0/4031327629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbcb806b500 0x7fbcb806d9c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.934+0000 7fbcdee00640 1 --2- 172.21.15.119:0/4031327629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcd8152720 0x7fbcd8152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.934+0000 7fbcdee00640 1 --2- 172.21.15.119:0/4031327629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcd80aca90 0x7fbcd81414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.934+0000 7fbcdee00640 1 --2- 172.21.15.119:0/4031327629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcc0003a20 0x7fbcd81f78c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.934+0000 7fbcdee00640 1 -- 172.21.15.119:0/4031327629 >> 172.21.15.119:0/4031327629 conn(0x7fbcd806b1d0 msgr2=0x7fbcd806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.934+0000 7fbcdee00640 1 -- 172.21.15.119:0/4031327629 shutdown_connections 2024-04-25T10:08:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:15.934+0000 7fbcdee00640 1 -- 172.21.15.119:0/4031327629 wait complete. 2024-04-25T10:08:15.960 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:08:16.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.157+0000 7fd613200640 1 Processor -- start 2024-04-25T10:08:16.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.157+0000 7fd613200640 1 -- start start 2024-04-25T10:08:16.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.159+0000 7fd613200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6140ce860 0x7fd6140cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.159+0000 7fd613200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd6140d00a0 0x7fd6140d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.159+0000 7fd613200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6140d0a70 con 0x7fd61415b7e0 2024-04-25T10:08:16.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.159+0000 7fd613200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6140b5710 con 0x7fd6140d00a0 2024-04-25T10:08:16.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.159+0000 7fd613200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140b59f0 con 0x7fd6140ce860 2024-04-25T10:08:16.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.159+0000 7fd612800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd6140d00a0 0x7fd6140d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.159+0000 7fd612800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd6140d00a0 0x7fd6140d0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34338/0 (socket says 172.21.15.119:34338) 2024-04-25T10:08:16.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.159+0000 7fd612800640 1 -- 172.21.15.119:0/2556014427 learned_addr learned my addr 172.21.15.119:0/2556014427 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:16.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.159+0000 7fd611400640 1 --2- 172.21.15.119:0/2556014427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6140ce860 0x7fd6140cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.159+0000 7fd612800640 1 -- 172.21.15.119:0/2556014427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6140ce860 msgr2=0x7fd6140cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:16.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.159+0000 7fd612800640 1 --2- 172.21.15.119:0/2556014427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6140ce860 0x7fd6140cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.159+0000 7fd612800640 1 -- 172.21.15.119:0/2556014427 >> v1:172.21.15.132:6789/0 conn(0x7fd61415b7e0 legacy=0x7fd61415e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:16.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.160+0000 7fd612800640 1 -- 172.21.15.119:0/2556014427 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6140b5cd0 con 0x7fd6140d00a0 2024-04-25T10:08:16.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.161+0000 7fd612800640 1 --2- 172.21.15.119:0/2556014427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6140d00a0 0x7fd6140d0530 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009e50 tx=0x7fd60c02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4a032b5705935d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:16.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.161+0000 7fd610a00640 1 -- 172.21.15.119:0/2556014427 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c036070 con 0x7fd6140d00a0 2024-04-25T10:08:16.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.161+0000 7fd610a00640 1 -- 172.21.15.119:0/2556014427 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c031600 con 0x7fd6140d00a0 2024-04-25T10:08:16.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.161+0000 7fd610a00640 1 -- 172.21.15.119:0/2556014427 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0319d0 con 0x7fd6140d00a0 2024-04-25T10:08:16.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.161+0000 7fd613200640 1 -- 172.21.15.119:0/2556014427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6140d00a0 msgr2=0x7fd6140d0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.161+0000 7fd613200640 1 --2- 172.21.15.119:0/2556014427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6140d00a0 0x7fd6140d0530 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009e50 tx=0x7fd60c02a5b0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.162+0000 7fd613200640 1 -- 172.21.15.119:0/2556014427 shutdown_connections 2024-04-25T10:08:16.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.162+0000 7fd613200640 1 --2- 172.21.15.119:0/2556014427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6140d00a0 0x7fd6140d0530 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.162+0000 7fd613200640 1 --2- 172.21.15.119:0/2556014427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6140ce860 0x7fd6140cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.162+0000 7fd613200640 1 -- 172.21.15.119:0/2556014427 >> 172.21.15.119:0/2556014427 conn(0x7fd6140722d0 msgr2=0x7fd6140726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:16.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.162+0000 7fd613200640 1 -- 172.21.15.119:0/2556014427 shutdown_connections 2024-04-25T10:08:16.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.162+0000 7fd613200640 1 -- 172.21.15.119:0/2556014427 wait complete. 2024-04-25T10:08:16.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.163+0000 7fd613200640 1 Processor -- start 2024-04-25T10:08:16.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.163+0000 7fd613200640 1 -- start start 2024-04-25T10:08:16.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.164+0000 7fd613200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6140ce860 0x7fd6140caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.164+0000 7fd613200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6140d00a0 0x7fd6141f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.164+0000 7fd613200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd61415b7e0 0x7fd6140cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.164+0000 7fd613200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd614161960 con 0x7fd61415b7e0 2024-04-25T10:08:16.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.164+0000 7fd613200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140b53c0 con 0x7fd6140d00a0 2024-04-25T10:08:16.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.164+0000 7fd613200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd614161ae0 con 0x7fd6140ce860 2024-04-25T10:08:16.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.164+0000 7fd612800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd61415b7e0 0x7fd6140cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.165+0000 7fd612800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd61415b7e0 0x7fd6140cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34352/0 (socket says 172.21.15.119:34352) 2024-04-25T10:08:16.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.165+0000 7fd612800640 1 -- 172.21.15.119:0/2345986331 learned_addr learned my addr 172.21.15.119:0/2345986331 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:16.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.165+0000 7fd611400640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6140d00a0 0x7fd6141f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.165+0000 7fd611e00640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6140ce860 0x7fd6140caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.165+0000 7fd612800640 1 -- 172.21.15.119:0/2345986331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6140ce860 msgr2=0x7fd6140caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.165+0000 7fd612800640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6140ce860 0x7fd6140caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.165+0000 7fd612800640 1 -- 172.21.15.119:0/2345986331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6140d00a0 msgr2=0x7fd6141f7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.165+0000 7fd612800640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6140d00a0 0x7fd6141f7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.165+0000 7fd612800640 1 -- 172.21.15.119:0/2345986331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6141f81d0 con 0x7fd61415b7e0 2024-04-25T10:08:16.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.165+0000 7fd611400640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6140d00a0 0x7fd6141f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:16.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.165+0000 7fd612800640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd61415b7e0 0x7fd6140cd2b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd60c034040 tx=0x7fd60c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:16.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.166+0000 7fd60a000640 1 -- 172.21.15.119:0/2345986331 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c041020 con 0x7fd61415b7e0 2024-04-25T10:08:16.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.166+0000 7fd60a000640 1 -- 172.21.15.119:0/2345986331 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c0077a0 con 0x7fd61415b7e0 2024-04-25T10:08:16.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.166+0000 7fd60a000640 1 -- 172.21.15.119:0/2345986331 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c036040 con 0x7fd61415b7e0 2024-04-25T10:08:16.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.166+0000 7fd613200640 1 -- 172.21.15.119:0/2345986331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61415e500 con 0x7fd61415b7e0 2024-04-25T10:08:16.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.166+0000 7fd613200640 1 -- 172.21.15.119:0/2345986331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61415e220 con 0x7fd61415b7e0 2024-04-25T10:08:16.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.167+0000 7fd613200640 1 -- 172.21.15.119:0/2345986331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6140749b0 con 0x7fd61415b7e0 2024-04-25T10:08:16.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.168+0000 7fd60a000640 1 -- 172.21.15.119:0/2345986331 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd60c044020 con 0x7fd61415b7e0 2024-04-25T10:08:16.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.169+0000 7fd60a000640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd5f006b4d0 0x7fd5f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.169+0000 7fd60a000640 1 -- 172.21.15.119:0/2345986331 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd60c0ab050 con 0x7fd61415b7e0 2024-04-25T10:08:16.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.169+0000 7fd611e00640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd5f006b4d0 0x7fd5f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.170+0000 7fd611e00640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd5f006b4d0 0x7fd5f006d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd60400a750 tx=0x7fd604008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:16.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.173+0000 7fd60a000640 1 -- 172.21.15.119:0/2345986331 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd60c04b030 con 0x7fd61415b7e0 2024-04-25T10:08:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:16.400+0000 7fd613200640 1 -- 172.21.15.119:0/2345986331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd61415b020 con 0x7fd61415b7e0 2024-04-25T10:08:17.025 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:08:17.019+0000 7fe4e1e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:08:17.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.111+0000 7fd60a000640 1 -- 172.21.15.119:0/2345986331 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd6141f81d0 con 0x7fd61415b7e0 2024-04-25T10:08:17.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.114+0000 7fd613200640 1 -- 172.21.15.119:0/2345986331 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd5f006b4d0 msgr2=0x7fd5f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.114+0000 7fd613200640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd5f006b4d0 0x7fd5f006d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd60400a750 tx=0x7fd604008040 comp rx=0 tx=0).stop 2024-04-25T10:08:17.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.115+0000 7fd613200640 1 -- 172.21.15.119:0/2345986331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd61415b7e0 msgr2=0x7fd6140cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.115+0000 7fd613200640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd61415b7e0 0x7fd6140cd2b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd60c034040 tx=0x7fd60c004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:17.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.115+0000 7fd613200640 1 -- 172.21.15.119:0/2345986331 shutdown_connections 2024-04-25T10:08:17.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.115+0000 7fd613200640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd5f006b4d0 0x7fd5f006d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.115+0000 7fd613200640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd61415b7e0 0x7fd6140cd2b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.115+0000 7fd613200640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6140d00a0 0x7fd6141f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.115+0000 7fd613200640 1 --2- 172.21.15.119:0/2345986331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd6140ce860 0x7fd6140caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.115+0000 7fd613200640 1 -- 172.21.15.119:0/2345986331 >> 172.21.15.119:0/2345986331 conn(0x7fd6140722d0 msgr2=0x7fd6140b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:17.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.115+0000 7fd613200640 1 -- 172.21.15.119:0/2345986331 shutdown_connections 2024-04-25T10:08:17.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.115+0000 7fd613200640 1 -- 172.21.15.119:0/2345986331 wait complete. 2024-04-25T10:08:17.141 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:08:17.141 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:08:17.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.329+0000 7f87d0a00640 1 Processor -- start 2024-04-25T10:08:17.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.329+0000 7f87d0a00640 1 -- start start 2024-04-25T10:08:17.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.331+0000 7f87d0a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87cc144e70 0x7f87cc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.331+0000 7f87d0a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f87cc0cf0e0 0x7f87cc0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.331+0000 7f87d0a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc06dcb0 con 0x7f87cc06d7a0 2024-04-25T10:08:17.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.331+0000 7f87d0a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87cc15bfe0 con 0x7f87cc144e70 2024-04-25T10:08:17.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.331+0000 7f87d0a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87cc06df90 con 0x7f87cc0cf0e0 2024-04-25T10:08:17.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.331+0000 7f87c9e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87cc144e70 0x7f87cc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.331+0000 7f87c1400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f87cc06d7a0 0x7f87cc0ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39712/0 (socket says 172.21.15.119:39712) 2024-04-25T10:08:17.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.331+0000 7f87c9e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87cc144e70 0x7f87cc06d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34362/0 (socket says 172.21.15.119:34362) 2024-04-25T10:08:17.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.331+0000 7f87c1400640 1 -- 172.21.15.119:0/916937499 learned_addr learned my addr 172.21.15.119:0/916937499 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:17.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.331+0000 7f87c9e00640 1 -- 172.21.15.119:0/916937499 >> v2:172.21.15.132:3300/0 conn(0x7f87cc0cf0e0 msgr2=0x7f87cc0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:17.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.332+0000 7f87c9e00640 1 --2- 172.21.15.119:0/916937499 >> v2:172.21.15.132:3300/0 conn(0x7f87cc0cf0e0 0x7f87cc0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.332+0000 7f87c9e00640 1 -- 172.21.15.119:0/916937499 >> v1:172.21.15.119:6789/0 conn(0x7f87cc06d7a0 legacy=0x7f87cc0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.332+0000 7f87c9e00640 1 -- 172.21.15.119:0/916937499 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc06e270 con 0x7f87cc144e70 2024-04-25T10:08:17.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.333+0000 7f87c9e00640 1 --2- 172.21.15.119:0/916937499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87cc144e70 0x7f87cc06d260 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f87cc1458f0 tx=0x7f87bc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129639553ec3a7ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:17.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.333+0000 7f87c9400640 1 -- 172.21.15.119:0/916937499 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc007740 con 0x7f87cc144e70 2024-04-25T10:08:17.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.333+0000 7f87c9400640 1 -- 172.21.15.119:0/916937499 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc0078e0 con 0x7f87cc144e70 2024-04-25T10:08:17.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.333+0000 7f87c9400640 1 -- 172.21.15.119:0/916937499 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc007cb0 con 0x7f87cc144e70 2024-04-25T10:08:17.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.333+0000 7f87d0a00640 1 -- 172.21.15.119:0/916937499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87cc144e70 msgr2=0x7f87cc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.333+0000 7f87d0a00640 1 --2- 172.21.15.119:0/916937499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87cc144e70 0x7f87cc06d260 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f87cc1458f0 tx=0x7f87bc02a5f0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.334+0000 7f87d0a00640 1 -- 172.21.15.119:0/916937499 shutdown_connections 2024-04-25T10:08:17.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.334+0000 7f87d0a00640 1 --2- 172.21.15.119:0/916937499 >> v2:172.21.15.132:3300/0 conn(0x7f87cc0cf0e0 0x7f87cc0cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.334+0000 7f87d0a00640 1 --2- 172.21.15.119:0/916937499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87cc144e70 0x7f87cc06d260 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.334+0000 7f87d0a00640 1 -- 172.21.15.119:0/916937499 >> 172.21.15.119:0/916937499 conn(0x7f87cc06aba0 msgr2=0x7f87cc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:17.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.334+0000 7f87d0a00640 1 -- 172.21.15.119:0/916937499 shutdown_connections 2024-04-25T10:08:17.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.334+0000 7f87d0a00640 1 -- 172.21.15.119:0/916937499 wait complete. 2024-04-25T10:08:17.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.336+0000 7f87d0a00640 1 Processor -- start 2024-04-25T10:08:17.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.336+0000 7f87d0a00640 1 -- start start 2024-04-25T10:08:17.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87d0a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87cc06d7a0 0x7f87cc1f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87d0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87cc0cf0e0 0x7f87cc1f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87d0a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87cc144e70 0x7f87cc0af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87d0a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc145630 con 0x7f87cc0cf0e0 2024-04-25T10:08:17.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87d0a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1454b0 con 0x7f87cc06d7a0 2024-04-25T10:08:17.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87d0a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc0cc840 con 0x7f87cc144e70 2024-04-25T10:08:17.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87c1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87cc0cf0e0 0x7f87cc1f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87c1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87cc0cf0e0 0x7f87cc1f6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34376/0 (socket says 172.21.15.119:34376) 2024-04-25T10:08:17.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87c1400640 1 -- 172.21.15.119:0/1736347337 learned_addr learned my addr 172.21.15.119:0/1736347337 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:17.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87c9e00640 1 --2- 172.21.15.119:0/1736347337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87cc06d7a0 0x7f87cc1f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87c1400640 1 -- 172.21.15.119:0/1736347337 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87cc144e70 msgr2=0x7f87cc0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87c1400640 1 --2- 172.21.15.119:0/1736347337 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87cc144e70 0x7f87cc0af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87c1400640 1 -- 172.21.15.119:0/1736347337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87cc06d7a0 msgr2=0x7f87cc1f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87c1400640 1 --2- 172.21.15.119:0/1736347337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87cc06d7a0 0x7f87cc1f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87c1400640 1 -- 172.21.15.119:0/1736347337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc1428e0 con 0x7f87cc0cf0e0 2024-04-25T10:08:17.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.337+0000 7f87c9e00640 1 --2- 172.21.15.119:0/1736347337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87cc06d7a0 0x7f87cc1f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:17.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.338+0000 7f87c1400640 1 --2- 172.21.15.119:0/1736347337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87cc0cf0e0 0x7f87cc1f6c40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f87b400c400 tx=0x7f87b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:17.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.338+0000 7f87c2a00640 1 -- 172.21.15.119:0/1736347337 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4017070 con 0x7f87cc0cf0e0 2024-04-25T10:08:17.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.338+0000 7f87c2a00640 1 -- 172.21.15.119:0/1736347337 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b4005ce0 con 0x7f87cc0cf0e0 2024-04-25T10:08:17.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.338+0000 7f87d0a00640 1 -- 172.21.15.119:0/1736347337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc142bc0 con 0x7f87cc0cf0e0 2024-04-25T10:08:17.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.338+0000 7f87c2a00640 1 -- 172.21.15.119:0/1736347337 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b40072f0 con 0x7f87cc0cf0e0 2024-04-25T10:08:17.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.338+0000 7f87d0a00640 1 -- 172.21.15.119:0/1736347337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc143180 con 0x7f87cc0cf0e0 2024-04-25T10:08:17.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.339+0000 7f87d0a00640 1 -- 172.21.15.119:0/1736347337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87cc0750b0 con 0x7f87cc0cf0e0 2024-04-25T10:08:17.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.340+0000 7f87c2a00640 1 -- 172.21.15.119:0/1736347337 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87b401a070 con 0x7f87cc0cf0e0 2024-04-25T10:08:17.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.341+0000 7f87c2a00640 1 --2- 172.21.15.119:0/1736347337 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87a806b4d0 0x7f87a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.341+0000 7f87c2a00640 1 -- 172.21.15.119:0/1736347337 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87b4089650 con 0x7f87cc0cf0e0 2024-04-25T10:08:17.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.341+0000 7f87c9e00640 1 --2- 172.21.15.119:0/1736347337 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87a806b4d0 0x7f87a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.342+0000 7f87c9e00640 1 --2- 172.21.15.119:0/1736347337 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87a806b4d0 0x7f87a806d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f87bc034000 tx=0x7f87bc033040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:17.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.345+0000 7f87c2a00640 1 -- 172.21.15.119:0/1736347337 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87b40584c0 con 0x7f87cc0cf0e0 2024-04-25T10:08:17.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:17.584+0000 7f87d0a00640 1 -- 172.21.15.119:0/1736347337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f87cc155820 con 0x7f87cc0cf0e0 2024-04-25T10:08:18.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.121+0000 7f87c2a00640 1 -- 172.21.15.119:0/1736347337 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f87b4057dc0 con 0x7f87cc0cf0e0 2024-04-25T10:08:18.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.126+0000 7f87d0a00640 1 -- 172.21.15.119:0/1736347337 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87a806b4d0 msgr2=0x7f87a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.126+0000 7f87d0a00640 1 --2- 172.21.15.119:0/1736347337 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87a806b4d0 0x7f87a806d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f87bc034000 tx=0x7f87bc033040 comp rx=0 tx=0).stop 2024-04-25T10:08:18.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.126+0000 7f87d0a00640 1 -- 172.21.15.119:0/1736347337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87cc0cf0e0 msgr2=0x7f87cc1f6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.126+0000 7f87d0a00640 1 --2- 172.21.15.119:0/1736347337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87cc0cf0e0 0x7f87cc1f6c40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f87b400c400 tx=0x7f87b400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.126+0000 7f87d0a00640 1 -- 172.21.15.119:0/1736347337 shutdown_connections 2024-04-25T10:08:18.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.126+0000 7f87d0a00640 1 --2- 172.21.15.119:0/1736347337 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87a806b4d0 0x7f87a806d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.126+0000 7f87d0a00640 1 --2- 172.21.15.119:0/1736347337 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87cc144e70 0x7f87cc0af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.126+0000 7f87d0a00640 1 --2- 172.21.15.119:0/1736347337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87cc0cf0e0 0x7f87cc1f6c40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.126+0000 7f87d0a00640 1 --2- 172.21.15.119:0/1736347337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87cc06d7a0 0x7f87cc1f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.126+0000 7f87d0a00640 1 -- 172.21.15.119:0/1736347337 >> 172.21.15.119:0/1736347337 conn(0x7f87cc06aba0 msgr2=0x7f87cc0cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:18.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.126+0000 7f87d0a00640 1 -- 172.21.15.119:0/1736347337 shutdown_connections 2024-04-25T10:08:18.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.126+0000 7f87d0a00640 1 -- 172.21.15.119:0/1736347337 wait complete. 2024-04-25T10:08:18.153 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:08:18.154 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:18.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.347+0000 7f8686600640 1 Processor -- start 2024-04-25T10:08:18.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.348+0000 7f8686600640 1 -- start start 2024-04-25T10:08:18.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.349+0000 7f8686600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f868015bf00 0x7f86800cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.349+0000 7f8686600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86800cc310 0x7f868015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.349+0000 7f8686600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86800ce6c0 con 0x7f868015c820 2024-04-25T10:08:18.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.349+0000 7f8686600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86801455e0 con 0x7f868015bf00 2024-04-25T10:08:18.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.349+0000 7f8686600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868006d450 con 0x7f86800cc310 2024-04-25T10:08:18.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.350+0000 7f867f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f868015bf00 0x7f86800cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.350+0000 7f867f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f868015bf00 0x7f86800cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34378/0 (socket says 172.21.15.119:34378) 2024-04-25T10:08:18.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.350+0000 7f867f400640 1 -- 172.21.15.119:0/2551625025 learned_addr learned my addr 172.21.15.119:0/2551625025 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:18.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.350+0000 7f867f400640 1 -- 172.21.15.119:0/2551625025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86800cc310 msgr2=0x7f868015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.350+0000 7f867f400640 1 --2- 172.21.15.119:0/2551625025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86800cc310 0x7f868015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.350+0000 7f867f400640 1 -- 172.21.15.119:0/2551625025 >> v1:172.21.15.132:6789/0 conn(0x7f868015c820 legacy=0x7f868006d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.350+0000 7f867f400640 1 -- 172.21.15.119:0/2551625025 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86800cc7b0 con 0x7f868015bf00 2024-04-25T10:08:18.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.351+0000 7f867f400640 1 --2- 172.21.15.119:0/2551625025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868015bf00 0x7f86800cbdd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8670009870 tx=0x7f867002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61808c2d76e53843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.351+0000 7f867ea00640 1 -- 172.21.15.119:0/2551625025 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670037070 con 0x7f868015bf00 2024-04-25T10:08:18.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.351+0000 7f867ea00640 1 -- 172.21.15.119:0/2551625025 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8670032440 con 0x7f868015bf00 2024-04-25T10:08:18.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.351+0000 7f867ea00640 1 -- 172.21.15.119:0/2551625025 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670032810 con 0x7f868015bf00 2024-04-25T10:08:18.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.352+0000 7f8686600640 1 -- 172.21.15.119:0/2551625025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868015bf00 msgr2=0x7f86800cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.352+0000 7f8686600640 1 --2- 172.21.15.119:0/2551625025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868015bf00 0x7f86800cbdd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8670009870 tx=0x7f867002a210 comp rx=0 tx=0).stop 2024-04-25T10:08:18.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.352+0000 7f8686600640 1 -- 172.21.15.119:0/2551625025 shutdown_connections 2024-04-25T10:08:18.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.352+0000 7f8686600640 1 --2- 172.21.15.119:0/2551625025 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86800cc310 0x7f868015c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.352+0000 7f8686600640 1 --2- 172.21.15.119:0/2551625025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868015bf00 0x7f86800cbdd0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.352+0000 7f8686600640 1 -- 172.21.15.119:0/2551625025 >> 172.21.15.119:0/2551625025 conn(0x7f868006aba0 msgr2=0x7f868006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:18.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.352+0000 7f8686600640 1 -- 172.21.15.119:0/2551625025 shutdown_connections 2024-04-25T10:08:18.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.352+0000 7f8686600640 1 -- 172.21.15.119:0/2551625025 wait complete. 2024-04-25T10:08:18.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.355+0000 7f8686600640 1 Processor -- start 2024-04-25T10:08:18.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.355+0000 7f8686600640 1 -- start start 2024-04-25T10:08:18.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.356+0000 7f8686600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86800cc310 0x7f8680156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.356+0000 7f8686600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868015bf00 0x7f86801f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.356+0000 7f8686600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f868015c820 0x7f8680143c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.356+0000 7f8686600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86800cd050 con 0x7f868015bf00 2024-04-25T10:08:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.356+0000 7f8686600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86800ccd70 con 0x7f86800cc310 2024-04-25T10:08:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.356+0000 7f8686600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868015cca0 con 0x7f868015c820 2024-04-25T10:08:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.356+0000 7f8677e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868015bf00 0x7f86801f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.356+0000 7f8677e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868015bf00 0x7f86801f69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34382/0 (socket says 172.21.15.119:34382) 2024-04-25T10:08:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.356+0000 7f8677e00640 1 -- 172.21.15.119:0/195833626 learned_addr learned my addr 172.21.15.119:0/195833626 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.357+0000 7f867f400640 1 --2- 172.21.15.119:0/195833626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86800cc310 0x7f8680156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.357+0000 7f867fe00640 1 --2- 172.21.15.119:0/195833626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f868015c820 0x7f8680143c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.357+0000 7f8677e00640 1 -- 172.21.15.119:0/195833626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f868015c820 msgr2=0x7f8680143c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.357+0000 7f8677e00640 1 --2- 172.21.15.119:0/195833626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f868015c820 0x7f8680143c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.357+0000 7f8677e00640 1 -- 172.21.15.119:0/195833626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86800cc310 msgr2=0x7f8680156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.357+0000 7f8677e00640 1 --2- 172.21.15.119:0/195833626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86800cc310 0x7f8680156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.357+0000 7f8677e00640 1 -- 172.21.15.119:0/195833626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86801f5930 con 0x7f868015bf00 2024-04-25T10:08:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.357+0000 7f8677e00640 1 --2- 172.21.15.119:0/195833626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868015bf00 0x7f86801f69d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f866800b450 tx=0x7f866800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.357+0000 7f8677400640 1 -- 172.21.15.119:0/195833626 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668015020 con 0x7f868015bf00 2024-04-25T10:08:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.358+0000 7f8677400640 1 -- 172.21.15.119:0/195833626 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8668011890 con 0x7f868015bf00 2024-04-25T10:08:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.358+0000 7f8677400640 1 -- 172.21.15.119:0/195833626 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668011c80 con 0x7f868015bf00 2024-04-25T10:08:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.358+0000 7f8686600640 1 -- 172.21.15.119:0/195833626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86801f5c10 con 0x7f868015bf00 2024-04-25T10:08:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.358+0000 7f8686600640 1 -- 172.21.15.119:0/195833626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86801f61d0 con 0x7f868015bf00 2024-04-25T10:08:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.359+0000 7f8686600640 1 -- 172.21.15.119:0/195833626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86800749b0 con 0x7f868015bf00 2024-04-25T10:08:18.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.360+0000 7f8677400640 1 -- 172.21.15.119:0/195833626 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8668018020 con 0x7f868015bf00 2024-04-25T10:08:18.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.361+0000 7f8677400640 1 --2- 172.21.15.119:0/195833626 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f865c06b500 0x7f865c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.361+0000 7f8677400640 1 -- 172.21.15.119:0/195833626 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f866808a410 con 0x7f868015bf00 2024-04-25T10:08:18.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.361+0000 7f867f400640 1 --2- 172.21.15.119:0/195833626 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f865c06b500 0x7f865c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.362+0000 7f867f400640 1 --2- 172.21.15.119:0/195833626 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f865c06b500 0x7f865c06d9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8670034000 tx=0x7f867002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.365+0000 7f8677400640 1 -- 172.21.15.119:0/195833626 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f866801f020 con 0x7f868015bf00 2024-04-25T10:08:18.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.598+0000 7f8686600640 1 -- 172.21.15.119:0/195833626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f868015c380 con 0x7f868015bf00 2024-04-25T10:08:18.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.599+0000 7f8677400640 1 -- 172.21.15.119:0/195833626 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f866800d040 con 0x7f868015bf00 2024-04-25T10:08:18.606 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:18.606 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1554372269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1554372269},{"type":"v1","addr":"172.21.15.119:6837","nonce":1554372269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4699,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4250304148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4250304148},{"type":"v1","addr":"172.21.15.132:6834","nonce":4250304148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/637229574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":637229574},{"type":"v1","addr":"172.21.15.132:6835","nonce":637229574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:14.553437+0000","modified":"2024-04-25T10:08:17.586270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6836/3585315292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3585315292},{"type":"v1","addr":"172.21.15.119:6836","nonce":3585315292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:08:18.606 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 16 2024-04-25T10:08:18.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.602+0000 7f8686600640 1 -- 172.21.15.119:0/195833626 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f865c06b500 msgr2=0x7f865c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.602+0000 7f8686600640 1 --2- 172.21.15.119:0/195833626 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f865c06b500 0x7f865c06d9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8670034000 tx=0x7f867002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:18.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.602+0000 7f8686600640 1 -- 172.21.15.119:0/195833626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868015bf00 msgr2=0x7f86801f69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.602+0000 7f8686600640 1 --2- 172.21.15.119:0/195833626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868015bf00 0x7f86801f69d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f866800b450 tx=0x7f866800b910 comp rx=0 tx=0).stop 2024-04-25T10:08:18.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.602+0000 7f8686600640 1 -- 172.21.15.119:0/195833626 shutdown_connections 2024-04-25T10:08:18.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.602+0000 7f8686600640 1 --2- 172.21.15.119:0/195833626 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f865c06b500 0x7f865c06d9c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.602+0000 7f8686600640 1 --2- 172.21.15.119:0/195833626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f868015c820 0x7f8680143c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.602+0000 7f8686600640 1 --2- 172.21.15.119:0/195833626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f868015bf00 0x7f86801f69d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.602+0000 7f8686600640 1 --2- 172.21.15.119:0/195833626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86800cc310 0x7f8680156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.602+0000 7f8686600640 1 -- 172.21.15.119:0/195833626 >> 172.21.15.119:0/195833626 conn(0x7f868006aba0 msgr2=0x7f86800cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:18.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.602+0000 7f8686600640 1 -- 172.21.15.119:0/195833626 shutdown_connections 2024-04-25T10:08:18.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.602+0000 7f8686600640 1 -- 172.21.15.119:0/195833626 wait complete. 2024-04-25T10:08:18.627 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:18.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.810+0000 7fbd60800640 1 Processor -- start 2024-04-25T10:08:18.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.810+0000 7fbd60800640 1 -- start start 2024-04-25T10:08:18.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.812+0000 7fbd60800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbd5815c9f0 0x7fbd581664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.812+0000 7fbd60800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd580ce6c0 con 0x7fbd5806e8f0 2024-04-25T10:08:18.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.812+0000 7fbd60800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd580cc4e0 con 0x7fbd5815bf00 2024-04-25T10:08:18.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.812+0000 7fbd60800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd581455e0 con 0x7fbd5815c9f0 2024-04-25T10:08:18.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.812+0000 7fbd5d400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbd5806e8f0 0x7fbd5815c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39728/0 (socket says 172.21.15.119:39728) 2024-04-25T10:08:18.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.812+0000 7fbd5d400640 1 -- 172.21.15.119:0/3125609182 learned_addr learned my addr 172.21.15.119:0/3125609182 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:18.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.813+0000 7fbd5e800640 1 --2- 172.21.15.119:0/3125609182 >> v2:172.21.15.132:3300/0 conn(0x7fbd5815c9f0 0x7fbd581664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.813+0000 7fbd5ca00640 1 -- 172.21.15.119:0/3125609182 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3675980085 0 0) 0x7fbd48001110 con 0x7fbd5806e8f0 2024-04-25T10:08:18.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.813+0000 7fbd5ca00640 1 -- 172.21.15.119:0/3125609182 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd44003520 con 0x7fbd5806e8f0 2024-04-25T10:08:18.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.813+0000 7fbd5ca00640 1 -- 172.21.15.119:0/3125609182 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2985782882 0 0) 0x7fbd44003520 con 0x7fbd5806e8f0 2024-04-25T10:08:18.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.813+0000 7fbd5ca00640 1 -- 172.21.15.119:0/3125609182 >> v2:172.21.15.132:3300/0 conn(0x7fbd5815c9f0 msgr2=0x7fbd581664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.813+0000 7fbd5ca00640 1 --2- 172.21.15.119:0/3125609182 >> v2:172.21.15.132:3300/0 conn(0x7fbd5815c9f0 0x7fbd581664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.813+0000 7fbd5ca00640 1 -- 172.21.15.119:0/3125609182 >> v1:172.21.15.132:6789/0 conn(0x7fbd5815bf00 legacy=0x7fbd580cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.813+0000 7fbd5ca00640 1 -- 172.21.15.119:0/3125609182 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd580cc7b0 con 0x7fbd5806e8f0 2024-04-25T10:08:18.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.813+0000 7fbd5ca00640 1 -- 172.21.15.119:0/3125609182 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fbd48001840 con 0x7fbd5806e8f0 2024-04-25T10:08:18.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.813+0000 7fbd5ca00640 1 -- 172.21.15.119:0/3125609182 >> v1:172.21.15.119:6789/0 conn(0x7fbd5806e8f0 legacy=0x7fbd5815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.813+0000 7fbd5ca00640 1 --2- 172.21.15.119:0/3125609182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd440039b0 0x7fbd44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.813+0000 7fbd5ca00640 1 --2- 172.21.15.119:0/3125609182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd44004800 0x7fbd44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.813+0000 7fbd5ca00640 1 --2- 172.21.15.119:0/3125609182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440051c0 0x7fbd4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.814+0000 7fbd5ca00640 1 -- 172.21.15.119:0/3125609182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd581455e0 con 0x7fbd440039b0 2024-04-25T10:08:18.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.814+0000 7fbd5ca00640 1 -- 172.21.15.119:0/3125609182 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4400c060 con 0x7fbd440051c0 2024-04-25T10:08:18.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.814+0000 7fbd5ca00640 1 -- 172.21.15.119:0/3125609182 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4400c260 con 0x7fbd44004800 2024-04-25T10:08:18.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.814+0000 7fbd5de00640 1 --2- 172.21.15.119:0/3125609182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd440039b0 0x7fbd44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.814+0000 7fbd5d400640 1 --2- 172.21.15.119:0/3125609182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd44004800 0x7fbd44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.814+0000 7fbd5e800640 1 --2- 172.21.15.119:0/3125609182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440051c0 0x7fbd4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.814+0000 7fbd5de00640 1 -- 172.21.15.119:0/3125609182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd44004800 msgr2=0x7fbd44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.814+0000 7fbd5de00640 1 --2- 172.21.15.119:0/3125609182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd44004800 0x7fbd44004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.814+0000 7fbd5de00640 1 -- 172.21.15.119:0/3125609182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440051c0 msgr2=0x7fbd4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.814+0000 7fbd5de00640 1 --2- 172.21.15.119:0/3125609182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440051c0 0x7fbd4400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.814+0000 7fbd5de00640 1 -- 172.21.15.119:0/3125609182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd44003520 con 0x7fbd440039b0 2024-04-25T10:08:18.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.814+0000 7fbd5e800640 1 --2- 172.21.15.119:0/3125609182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440051c0 0x7fbd4400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:18.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.814+0000 7fbd5e800640 1 -- 172.21.15.119:0/3125609182 reap_dead start 2024-04-25T10:08:18.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.815+0000 7fbd5de00640 1 --2- 172.21.15.119:0/3125609182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd440039b0 0x7fbd44003dd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbd50001e50 tx=0x7fbd44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f370160839b083cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.815+0000 7fbd5ca00640 1 -- 172.21.15.119:0/3125609182 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50001a70 con 0x7fbd440039b0 2024-04-25T10:08:18.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.815+0000 7fbd5ca00640 1 -- 172.21.15.119:0/3125609182 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5002ae40 con 0x7fbd440039b0 2024-04-25T10:08:18.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.815+0000 7fbd5ca00640 1 -- 172.21.15.119:0/3125609182 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50003bb0 con 0x7fbd440039b0 2024-04-25T10:08:18.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.816+0000 7fbd60800640 1 -- 172.21.15.119:0/3125609182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd440039b0 msgr2=0x7fbd44003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.816+0000 7fbd60800640 1 --2- 172.21.15.119:0/3125609182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd440039b0 0x7fbd44003dd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbd50001e50 tx=0x7fbd44005830 comp rx=0 tx=0).stop 2024-04-25T10:08:18.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.816+0000 7fbd60800640 1 -- 172.21.15.119:0/3125609182 shutdown_connections 2024-04-25T10:08:18.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.816+0000 7fbd60800640 1 --2- 172.21.15.119:0/3125609182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd440039b0 0x7fbd44003dd0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.816+0000 7fbd60800640 1 -- 172.21.15.119:0/3125609182 >> 172.21.15.119:0/3125609182 conn(0x7fbd5806aba0 msgr2=0x7fbd5806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:18.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.816+0000 7fbd60800640 1 -- 172.21.15.119:0/3125609182 shutdown_connections 2024-04-25T10:08:18.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.816+0000 7fbd60800640 1 -- 172.21.15.119:0/3125609182 wait complete. 2024-04-25T10:08:18.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.818+0000 7fbd60800640 1 Processor -- start 2024-04-25T10:08:18.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.818+0000 7fbd60800640 1 -- start start 2024-04-25T10:08:18.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.819+0000 7fbd60800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd440039b0 0x7fbd58141590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.819+0000 7fbd60800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd580cb1d0 0x7fbd58144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.819+0000 7fbd60800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd581efbc0 0x7fbd58141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.819+0000 7fbd60800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580ccd70 con 0x7fbd581efbc0 2024-04-25T10:08:18.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.819+0000 7fbd60800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5806d450 con 0x7fbd580cb1d0 2024-04-25T10:08:18.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.819+0000 7fbd60800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580cd050 con 0x7fbd440039b0 2024-04-25T10:08:18.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.819+0000 7fbd5e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd581efbc0 0x7fbd58141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.819+0000 7fbd5e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd581efbc0 0x7fbd58141e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34394/0 (socket says 172.21.15.119:34394) 2024-04-25T10:08:18.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.819+0000 7fbd5e800640 1 -- 172.21.15.119:0/3766790007 learned_addr learned my addr 172.21.15.119:0/3766790007 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:18.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.820+0000 7fbd5d400640 1 --2- 172.21.15.119:0/3766790007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd580cb1d0 0x7fbd58144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.820+0000 7fbd5e800640 1 -- 172.21.15.119:0/3766790007 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd440039b0 msgr2=0x7fbd58141590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.820+0000 7fbd5e800640 1 --2- 172.21.15.119:0/3766790007 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd440039b0 0x7fbd58141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.820+0000 7fbd5e800640 1 -- 172.21.15.119:0/3766790007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd580cb1d0 msgr2=0x7fbd58144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.820+0000 7fbd5e800640 1 --2- 172.21.15.119:0/3766790007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd580cb1d0 0x7fbd58144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.820+0000 7fbd5e800640 1 -- 172.21.15.119:0/3766790007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd580cb650 con 0x7fbd581efbc0 2024-04-25T10:08:18.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.820+0000 7fbd5d400640 1 --2- 172.21.15.119:0/3766790007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd580cb1d0 0x7fbd58144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:18.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.820+0000 7fbd5e800640 1 --2- 172.21.15.119:0/3766790007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd581efbc0 0x7fbd58141e00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbd580cdb90 tx=0x7fbd54010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.820+0000 7fbd4e000640 1 -- 172.21.15.119:0/3766790007 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54018070 con 0x7fbd581efbc0 2024-04-25T10:08:18.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.820+0000 7fbd4e000640 1 -- 172.21.15.119:0/3766790007 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd54004230 con 0x7fbd581efbc0 2024-04-25T10:08:18.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.820+0000 7fbd4e000640 1 -- 172.21.15.119:0/3766790007 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5400f040 con 0x7fbd581efbc0 2024-04-25T10:08:18.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.820+0000 7fbd60800640 1 -- 172.21.15.119:0/3766790007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd58144750 con 0x7fbd581efbc0 2024-04-25T10:08:18.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.820+0000 7fbd60800640 1 -- 172.21.15.119:0/3766790007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5813cf30 con 0x7fbd581efbc0 2024-04-25T10:08:18.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.822+0000 7fbd60800640 1 -- 172.21.15.119:0/3766790007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5806bb30 con 0x7fbd581efbc0 2024-04-25T10:08:18.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.822+0000 7fbd4e000640 1 -- 172.21.15.119:0/3766790007 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd5401b070 con 0x7fbd581efbc0 2024-04-25T10:08:18.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.823+0000 7fbd4e000640 1 --2- 172.21.15.119:0/3766790007 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd3c06b500 0x7fbd3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.823+0000 7fbd4e000640 1 -- 172.21.15.119:0/3766790007 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd54089fe0 con 0x7fbd581efbc0 2024-04-25T10:08:18.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.824+0000 7fbd5de00640 1 --2- 172.21.15.119:0/3766790007 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd3c06b500 0x7fbd3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.824+0000 7fbd5de00640 1 --2- 172.21.15.119:0/3766790007 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd3c06b500 0x7fbd3c06d9c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbd50032000 tx=0x7fbd5002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:18.828+0000 7fbd4e000640 1 -- 172.21.15.119:0/3766790007 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd54058de0 con 0x7fbd581efbc0 2024-04-25T10:08:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.024+0000 7fbd60800640 1 -- 172.21.15.119:0/3766790007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd580749b0 con 0x7fbd581efbc0 2024-04-25T10:08:19.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.027+0000 7fbd4e000640 1 -- 172.21.15.119:0/3766790007 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16705 (secure 0 0 0) 0x7fbd540586e0 con 0x7fbd581efbc0 2024-04-25T10:08:19.033 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:19.034 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":33,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:16.095956+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:12.268457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:13.546259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:19.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.029+0000 7fbd60800640 1 -- 172.21.15.119:0/3766790007 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd3c06b500 msgr2=0x7fbd3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.029+0000 7fbd60800640 1 --2- 172.21.15.119:0/3766790007 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd3c06b500 0x7fbd3c06d9c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbd50032000 tx=0x7fbd5002c000 comp rx=0 tx=0).stop 2024-04-25T10:08:19.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.029+0000 7fbd60800640 1 -- 172.21.15.119:0/3766790007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd581efbc0 msgr2=0x7fbd58141e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.029+0000 7fbd60800640 1 --2- 172.21.15.119:0/3766790007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd581efbc0 0x7fbd58141e00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbd580cdb90 tx=0x7fbd54010c20 comp rx=0 tx=0).stop 2024-04-25T10:08:19.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.030+0000 7fbd60800640 1 -- 172.21.15.119:0/3766790007 shutdown_connections 2024-04-25T10:08:19.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.030+0000 7fbd60800640 1 --2- 172.21.15.119:0/3766790007 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd3c06b500 0x7fbd3c06d9c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.030+0000 7fbd60800640 1 --2- 172.21.15.119:0/3766790007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd581efbc0 0x7fbd58141e00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.030+0000 7fbd60800640 1 --2- 172.21.15.119:0/3766790007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd580cb1d0 0x7fbd58144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.030+0000 7fbd60800640 1 --2- 172.21.15.119:0/3766790007 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd440039b0 0x7fbd58141590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.030+0000 7fbd60800640 1 -- 172.21.15.119:0/3766790007 >> 172.21.15.119:0/3766790007 conn(0x7fbd5806aba0 msgr2=0x7fbd58071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:19.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.030+0000 7fbd60800640 1 -- 172.21.15.119:0/3766790007 shutdown_connections 2024-04-25T10:08:19.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.030+0000 7fbd60800640 1 -- 172.21.15.119:0/3766790007 wait complete. 2024-04-25T10:08:19.055 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:08:19.055 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:19.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.245+0000 7fa8fbe00640 1 Processor -- start 2024-04-25T10:08:19.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.246+0000 7fa8fbe00640 1 -- start start 2024-04-25T10:08:19.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.247+0000 7fa8fbe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa8fc144470 0x7fa8fc15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.247+0000 7fa8fbe00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8fc0cd440 con 0x7fa8fc0b5050 2024-04-25T10:08:19.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.247+0000 7fa8fbe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8fc0cd720 con 0x7fa8fc06e430 2024-04-25T10:08:19.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.247+0000 7fa8fbe00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8fc145340 con 0x7fa8fc144470 2024-04-25T10:08:19.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.248+0000 7fa8faa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa8fc144470 0x7fa8fc15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.248+0000 7fa8fb400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa8fc0b5050 0x7fa8fc15bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39736/0 (socket says 172.21.15.119:39736) 2024-04-25T10:08:19.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.248+0000 7fa8fb400640 1 -- 172.21.15.119:0/3546497180 learned_addr learned my addr 172.21.15.119:0/3546497180 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:19.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.248+0000 7fa8faa00640 1 -- 172.21.15.119:0/3546497180 >> v1:172.21.15.132:6789/0 conn(0x7fa8fc06e430 legacy=0x7fa8fc0b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:19.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.248+0000 7fa8faa00640 1 -- 172.21.15.119:0/3546497180 >> v1:172.21.15.119:6789/0 conn(0x7fa8fc0b5050 legacy=0x7fa8fc15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.248+0000 7fa8faa00640 1 -- 172.21.15.119:0/3546497180 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc0cda00 con 0x7fa8fc144470 2024-04-25T10:08:19.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.248+0000 7fa8fa000640 1 -- 172.21.15.119:0/3546497180 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248880241 0 0) 0x7fa8fc0cd440 con 0x7fa8fc0b5050 2024-04-25T10:08:19.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.249+0000 7fa8faa00640 1 --2- 172.21.15.119:0/3546497180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc144470 0x7fa8fc15fc90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0097a0 tx=0x7fa8ec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79610037b95c8ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.249+0000 7fa8fa000640 1 -- 172.21.15.119:0/3546497180 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec037070 con 0x7fa8fc144470 2024-04-25T10:08:19.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.249+0000 7fa8fa000640 1 -- 172.21.15.119:0/3546497180 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ec032440 con 0x7fa8fc144470 2024-04-25T10:08:19.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.249+0000 7fa8fa000640 1 -- 172.21.15.119:0/3546497180 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec032810 con 0x7fa8fc144470 2024-04-25T10:08:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.250+0000 7fa8fbe00640 1 -- 172.21.15.119:0/3546497180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc144470 msgr2=0x7fa8fc15fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.250+0000 7fa8fbe00640 1 --2- 172.21.15.119:0/3546497180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc144470 0x7fa8fc15fc90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0097a0 tx=0x7fa8ec02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.250+0000 7fa8fbe00640 1 -- 172.21.15.119:0/3546497180 shutdown_connections 2024-04-25T10:08:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.250+0000 7fa8fbe00640 1 --2- 172.21.15.119:0/3546497180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc144470 0x7fa8fc15fc90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.250+0000 7fa8fbe00640 1 -- 172.21.15.119:0/3546497180 >> 172.21.15.119:0/3546497180 conn(0x7fa8fc06b1d0 msgr2=0x7fa8fc0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.250+0000 7fa8fbe00640 1 -- 172.21.15.119:0/3546497180 shutdown_connections 2024-04-25T10:08:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.250+0000 7fa8fbe00640 1 -- 172.21.15.119:0/3546497180 wait complete. 2024-04-25T10:08:19.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.252+0000 7fa8fbe00640 1 Processor -- start 2024-04-25T10:08:19.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.252+0000 7fa8fbe00640 1 -- start start 2024-04-25T10:08:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.253+0000 7fa8fbe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8fc06e430 0x7fa8fc15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.253+0000 7fa8fbe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8fc0b5050 0x7fa8fc141760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.253+0000 7fa8fbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc144470 0x7fa8fc0a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.253+0000 7fa8fbe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc0ce200 con 0x7fa8fc144470 2024-04-25T10:08:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.253+0000 7fa8fbe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc144850 con 0x7fa8fc06e430 2024-04-25T10:08:19.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.253+0000 7fa8fbe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc15c400 con 0x7fa8fc0b5050 2024-04-25T10:08:19.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.253+0000 7fa8fb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc144470 0x7fa8fc0a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.253+0000 7fa8fb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc144470 0x7fa8fc0a9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34404/0 (socket says 172.21.15.119:34404) 2024-04-25T10:08:19.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.253+0000 7fa8fb400640 1 -- 172.21.15.119:0/575507570 learned_addr learned my addr 172.21.15.119:0/575507570 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.254+0000 7fa8faa00640 1 --2- 172.21.15.119:0/575507570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8fc06e430 0x7fa8fc15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.254+0000 7fa8f2000640 1 --2- 172.21.15.119:0/575507570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8fc0b5050 0x7fa8fc141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.254+0000 7fa8fb400640 1 -- 172.21.15.119:0/575507570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8fc0b5050 msgr2=0x7fa8fc141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.254+0000 7fa8fb400640 1 --2- 172.21.15.119:0/575507570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8fc0b5050 0x7fa8fc141760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.254+0000 7fa8fb400640 1 -- 172.21.15.119:0/575507570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8fc06e430 msgr2=0x7fa8fc15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.254+0000 7fa8fb400640 1 --2- 172.21.15.119:0/575507570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8fc06e430 0x7fa8fc15b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.254+0000 7fa8fb400640 1 -- 172.21.15.119:0/575507570 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc1406c0 con 0x7fa8fc144470 2024-04-25T10:08:19.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.254+0000 7fa8fb400640 1 --2- 172.21.15.119:0/575507570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc144470 0x7fa8fc0a9fa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400c730 tx=0x7fa8f400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.254+0000 7fa8f3e00640 1 -- 172.21.15.119:0/575507570 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4018070 con 0x7fa8fc144470 2024-04-25T10:08:19.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.255+0000 7fa8f3e00640 1 -- 172.21.15.119:0/575507570 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f400ce40 con 0x7fa8fc144470 2024-04-25T10:08:19.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.255+0000 7fa8f3e00640 1 -- 172.21.15.119:0/575507570 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f400a860 con 0x7fa8fc144470 2024-04-25T10:08:19.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.255+0000 7fa8fbe00640 1 -- 172.21.15.119:0/575507570 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8fc1409a0 con 0x7fa8fc144470 2024-04-25T10:08:19.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.255+0000 7fa8fbe00640 1 -- 172.21.15.119:0/575507570 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8fc140f60 con 0x7fa8fc144470 2024-04-25T10:08:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.256+0000 7fa8fbe00640 1 -- 172.21.15.119:0/575507570 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8fc0749b0 con 0x7fa8fc144470 2024-04-25T10:08:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.257+0000 7fa8f3e00640 1 -- 172.21.15.119:0/575507570 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8f401b070 con 0x7fa8fc144470 2024-04-25T10:08:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.258+0000 7fa8f3e00640 1 --2- 172.21.15.119:0/575507570 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8d806b4d0 0x7fa8d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.258+0000 7fa8f3e00640 1 -- 172.21.15.119:0/575507570 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8f40897c0 con 0x7fa8fc144470 2024-04-25T10:08:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.258+0000 7fa8faa00640 1 --2- 172.21.15.119:0/575507570 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8d806b4d0 0x7fa8d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.259+0000 7fa8faa00640 1 --2- 172.21.15.119:0/575507570 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8d806b4d0 0x7fa8d806d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec034000 tx=0x7fa8ec02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.262+0000 7fa8f3e00640 1 -- 172.21.15.119:0/575507570 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8f4058630 con 0x7fa8fc144470 2024-04-25T10:08:19.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.453+0000 7fa8fbe00640 1 -- 172.21.15.119:0/575507570 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8fc0b2090 con 0x7fa8d806b4d0 2024-04-25T10:08:19.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.463+0000 7fa8f3e00640 1 -- 172.21.15.119:0/575507570 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251882 (secure 0 0 0) 0x7fa8fc0b2090 con 0x7fa8d806b4d0 2024-04-25T10:08:19.473 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:19.474 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:08:19.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.468+0000 7fa8fbe00640 1 -- 172.21.15.119:0/575507570 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8d806b4d0 msgr2=0x7fa8d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.468+0000 7fa8fbe00640 1 --2- 172.21.15.119:0/575507570 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8d806b4d0 0x7fa8d806d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec034000 tx=0x7fa8ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:08:19.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.469+0000 7fa8fbe00640 1 -- 172.21.15.119:0/575507570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc144470 msgr2=0x7fa8fc0a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.469+0000 7fa8fbe00640 1 --2- 172.21.15.119:0/575507570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc144470 0x7fa8fc0a9fa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400c730 tx=0x7fa8f400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.469+0000 7fa8fbe00640 1 -- 172.21.15.119:0/575507570 shutdown_connections 2024-04-25T10:08:19.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.469+0000 7fa8fbe00640 1 --2- 172.21.15.119:0/575507570 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8d806b4d0 0x7fa8d806d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.469+0000 7fa8fbe00640 1 --2- 172.21.15.119:0/575507570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8fc144470 0x7fa8fc0a9fa0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.469+0000 7fa8fbe00640 1 --2- 172.21.15.119:0/575507570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa8fc0b5050 0x7fa8fc141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.469+0000 7fa8fbe00640 1 --2- 172.21.15.119:0/575507570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8fc06e430 0x7fa8fc15b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.469+0000 7fa8fbe00640 1 -- 172.21.15.119:0/575507570 >> 172.21.15.119:0/575507570 conn(0x7fa8fc06b1d0 msgr2=0x7fa8fc15c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:19.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.469+0000 7fa8fbe00640 1 -- 172.21.15.119:0/575507570 shutdown_connections 2024-04-25T10:08:19.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.469+0000 7fa8fbe00640 1 -- 172.21.15.119:0/575507570 wait complete. 2024-04-25T10:08:19.492 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-25T10:08:17.947595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220228,"kb_used_data":4020,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754492,"statfs":{"total":773094113280,"available":772868599808,"internally_reserved":0,"allocated":4116480,"data_stored":2300101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.884118"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.104304+0000","last_change":"2024-04-25T10:08:15.100083+0000","last_active":"2024-04-25T10:08:15.104304+0000","last_peered":"2024-04-25T10:08:15.104304+0000","last_clean":"2024-04-25T10:08:15.104304+0000","last_became_active":"2024-04-25T10:08:15.099636+0000","last_became_peered":"2024-04-25T10:08:15.099636+0000","last_unstale":"2024-04-25T10:08:15.104304+0000","last_undegraded":"2024-04-25T10:08:15.104304+0000","last_fullsized":"2024-04-25T10:08:15.104304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:43.448784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.117707+0000","last_change":"2024-04-25T10:08:15.110342+0000","last_active":"2024-04-25T10:08:15.117707+0000","last_peered":"2024-04-25T10:08:15.117707+0000","last_clean":"2024-04-25T10:08:15.117707+0000","last_became_active":"2024-04-25T10:08:15.108964+0000","last_became_peered":"2024-04-25T10:08:15.108964+0000","last_unstale":"2024-04-25T10:08:15.117707+0000","last_undegraded":"2024-04-25T10:08:15.117707+0000","last_fullsized":"2024-04-25T10:08:15.117707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:04.742692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.137746+0000","last_change":"2024-04-25T10:08:15.131503+0000","last_active":"2024-04-25T10:08:15.137746+0000","last_peered":"2024-04-25T10:08:15.137746+0000","last_clean":"2024-04-25T10:08:15.137746+0000","last_became_active":"2024-04-25T10:08:15.131249+0000","last_became_peered":"2024-04-25T10:08:15.131249+0000","last_unstale":"2024-04-25T10:08:15.137746+0000","last_undegraded":"2024-04-25T10:08:15.137746+0000","last_fullsized":"2024-04-25T10:08:15.137746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:00.375372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.135167+0000","last_change":"2024-04-25T10:08:15.124953+0000","last_active":"2024-04-25T10:08:15.135167+0000","last_peered":"2024-04-25T10:08:15.135167+0000","last_clean":"2024-04-25T10:08:15.135167+0000","last_became_active":"2024-04-25T10:08:15.122813+0000","last_became_peered":"2024-04-25T10:08:15.122813+0000","last_unstale":"2024-04-25T10:08:15.135167+0000","last_undegraded":"2024-04-25T10:08:15.135167+0000","last_fullsized":"2024-04-25T10:08:15.135167+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:48.317640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.114107+0000","last_change":"2024-04-25T10:08:15.107948+0000","last_active":"2024-04-25T10:08:15.114107+0000","last_peered":"2024-04-25T10:08:15.114107+0000","last_clean":"2024-04-25T10:08:15.114107+0000","last_became_active":"2024-04-25T10:08:15.107617+0000","last_became_peered":"2024-04-25T10:08:15.107617+0000","last_unstale":"2024-04-25T10:08:15.114107+0000","last_undegraded":"2024-04-25T10:08:15.114107+0000","last_fullsized":"2024-04-25T10:08:15.114107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:14.743414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.100615+0000","last_change":"2024-04-25T10:08:15.095553+0000","last_active":"2024-04-25T10:08:15.100615+0000","last_peered":"2024-04-25T10:08:15.100615+0000","last_clean":"2024-04-25T10:08:15.100615+0000","last_became_active":"2024-04-25T10:08:15.095089+0000","last_became_peered":"2024-04-25T10:08:15.095089+0000","last_unstale":"2024-04-25T10:08:15.100615+0000","last_undegraded":"2024-04-25T10:08:15.100615+0000","last_fullsized":"2024-04-25T10:08:15.100615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:14.806267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.119487+0000","last_change":"2024-04-25T10:08:15.113757+0000","last_active":"2024-04-25T10:08:15.119487+0000","last_peered":"2024-04-25T10:08:15.119487+0000","last_clean":"2024-04-25T10:08:15.119487+0000","last_became_active":"2024-04-25T10:08:15.113530+0000","last_became_peered":"2024-04-25T10:08:15.113530+0000","last_unstale":"2024-04-25T10:08:15.119487+0000","last_undegraded":"2024-04-25T10:08:15.119487+0000","last_fullsized":"2024-04-25T10:08:15.119487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:14.806267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.113098+0000","last_change":"2024-04-25T10:08:15.108497+0000","last_active":"2024-04-25T10:08:15.113098+0000","last_peered":"2024-04-25T10:08:15.113098+0000","last_clean":"2024-04-25T10:08:15.113098+0000","last_became_active":"2024-04-25T10:08:15.107919+0000","last_became_peered":"2024-04-25T10:08:15.107919+0000","last_unstale":"2024-04-25T10:08:15.113098+0000","last_undegraded":"2024-04-25T10:08:15.113098+0000","last_fullsized":"2024-04-25T10:08:15.113098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:51.105227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.133099+0000","last_change":"2024-04-25T10:08:15.129184+0000","last_active":"2024-04-25T10:08:15.133099+0000","last_peered":"2024-04-25T10:08:15.133099+0000","last_clean":"2024-04-25T10:08:15.133099+0000","last_became_active":"2024-04-25T10:08:15.128910+0000","last_became_peered":"2024-04-25T10:08:15.128910+0000","last_unstale":"2024-04-25T10:08:15.133099+0000","last_undegraded":"2024-04-25T10:08:15.133099+0000","last_fullsized":"2024-04-25T10:08:15.133099+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.483430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.132808+0000","last_change":"2024-04-25T10:08:15.127981+0000","last_active":"2024-04-25T10:08:15.132808+0000","last_peered":"2024-04-25T10:08:15.132808+0000","last_clean":"2024-04-25T10:08:15.132808+0000","last_became_active":"2024-04-25T10:08:15.127713+0000","last_became_peered":"2024-04-25T10:08:15.127713+0000","last_unstale":"2024-04-25T10:08:15.132808+0000","last_undegraded":"2024-04-25T10:08:15.132808+0000","last_fullsized":"2024-04-25T10:08:15.132808+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:15.934629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.137757+0000","last_change":"2024-04-25T10:08:15.131791+0000","last_active":"2024-04-25T10:08:15.137757+0000","last_peered":"2024-04-25T10:08:15.137757+0000","last_clean":"2024-04-25T10:08:15.137757+0000","last_became_active":"2024-04-25T10:08:15.131478+0000","last_became_peered":"2024-04-25T10:08:15.131478+0000","last_unstale":"2024-04-25T10:08:15.137757+0000","last_undegraded":"2024-04-25T10:08:15.137757+0000","last_fullsized":"2024-04-25T10:08:15.137757+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:00.375372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.114245+0000","last_change":"2024-04-25T10:08:15.101166+0000","last_active":"2024-04-25T10:08:15.114245+0000","last_peered":"2024-04-25T10:08:15.114245+0000","last_clean":"2024-04-25T10:08:15.114245+0000","last_became_active":"2024-04-25T10:08:15.100782+0000","last_became_peered":"2024-04-25T10:08:15.100782+0000","last_unstale":"2024-04-25T10:08:15.114245+0000","last_undegraded":"2024-04-25T10:08:15.114245+0000","last_fullsized":"2024-04-25T10:08:15.114245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:15.934629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.133058+0000","last_change":"2024-04-25T10:08:15.129100+0000","last_active":"2024-04-25T10:08:15.133058+0000","last_peered":"2024-04-25T10:08:15.133058+0000","last_clean":"2024-04-25T10:08:15.133058+0000","last_became_active":"2024-04-25T10:08:15.128746+0000","last_became_peered":"2024-04-25T10:08:15.128746+0000","last_unstale":"2024-04-25T10:08:15.133058+0000","last_undegraded":"2024-04-25T10:08:15.133058+0000","last_fullsized":"2024-04-25T10:08:15.133058+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:14.698665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.115066+0000","last_change":"2024-04-25T10:08:15.108123+0000","last_active":"2024-04-25T10:08:15.115066+0000","last_peered":"2024-04-25T10:08:15.115066+0000","last_clean":"2024-04-25T10:08:15.115066+0000","last_became_active":"2024-04-25T10:08:15.105973+0000","last_became_peered":"2024-04-25T10:08:15.105973+0000","last_unstale":"2024-04-25T10:08:15.115066+0000","last_undegraded":"2024-04-25T10:08:15.115066+0000","last_fullsized":"2024-04-25T10:08:15.115066+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:01.431948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.111156+0000","last_change":"2024-04-25T10:08:15.100006+0000","last_active":"2024-04-25T10:08:15.111156+0000","last_peered":"2024-04-25T10:08:15.111156+0000","last_clean":"2024-04-25T10:08:15.111156+0000","last_became_active":"2024-04-25T10:08:15.099703+0000","last_became_peered":"2024-04-25T10:08:15.099703+0000","last_unstale":"2024-04-25T10:08:15.111156+0000","last_undegraded":"2024-04-25T10:08:15.111156+0000","last_fullsized":"2024-04-25T10:08:15.111156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:48.317640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.113076+0000","last_change":"2024-04-25T10:08:15.108210+0000","last_active":"2024-04-25T10:08:15.113076+0000","last_peered":"2024-04-25T10:08:15.113076+0000","last_clean":"2024-04-25T10:08:15.113076+0000","last_became_active":"2024-04-25T10:08:15.107547+0000","last_became_peered":"2024-04-25T10:08:15.107547+0000","last_unstale":"2024-04-25T10:08:15.113076+0000","last_undegraded":"2024-04-25T10:08:15.113076+0000","last_fullsized":"2024-04-25T10:08:15.113076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:19.536090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.119747+0000","last_change":"2024-04-25T10:08:15.115863+0000","last_active":"2024-04-25T10:08:15.119747+0000","last_peered":"2024-04-25T10:08:15.119747+0000","last_clean":"2024-04-25T10:08:15.119747+0000","last_became_active":"2024-04-25T10:08:15.115478+0000","last_became_peered":"2024-04-25T10:08:15.115478+0000","last_unstale":"2024-04-25T10:08:15.119747+0000","last_undegraded":"2024-04-25T10:08:15.119747+0000","last_fullsized":"2024-04-25T10:08:15.119747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:01.431948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.133615+0000","last_change":"2024-04-25T10:08:15.129416+0000","last_active":"2024-04-25T10:08:15.133615+0000","last_peered":"2024-04-25T10:08:15.133615+0000","last_clean":"2024-04-25T10:08:15.133615+0000","last_became_active":"2024-04-25T10:08:15.129118+0000","last_became_peered":"2024-04-25T10:08:15.129118+0000","last_unstale":"2024-04-25T10:08:15.133615+0000","last_undegraded":"2024-04-25T10:08:15.133615+0000","last_fullsized":"2024-04-25T10:08:15.133615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:01.055962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.136534+0000","last_change":"2024-04-25T10:08:15.135479+0000","last_active":"2024-04-25T10:08:15.136534+0000","last_peered":"2024-04-25T10:08:15.136534+0000","last_clean":"2024-04-25T10:08:15.136534+0000","last_became_active":"2024-04-25T10:08:15.135119+0000","last_became_peered":"2024-04-25T10:08:15.135119+0000","last_unstale":"2024-04-25T10:08:15.136534+0000","last_undegraded":"2024-04-25T10:08:15.136534+0000","last_fullsized":"2024-04-25T10:08:15.136534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:50.400597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.119884+0000","last_change":"2024-04-25T10:08:15.115513+0000","last_active":"2024-04-25T10:08:15.119884+0000","last_peered":"2024-04-25T10:08:15.119884+0000","last_clean":"2024-04-25T10:08:15.119884+0000","last_became_active":"2024-04-25T10:08:15.115169+0000","last_became_peered":"2024-04-25T10:08:15.115169+0000","last_unstale":"2024-04-25T10:08:15.119884+0000","last_undegraded":"2024-04-25T10:08:15.119884+0000","last_fullsized":"2024-04-25T10:08:15.119884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:55.173693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.110836+0000","last_change":"2024-04-25T10:08:15.104432+0000","last_active":"2024-04-25T10:08:15.110836+0000","last_peered":"2024-04-25T10:08:15.110836+0000","last_clean":"2024-04-25T10:08:15.110836+0000","last_became_active":"2024-04-25T10:08:15.103725+0000","last_became_peered":"2024-04-25T10:08:15.103725+0000","last_unstale":"2024-04-25T10:08:15.110836+0000","last_undegraded":"2024-04-25T10:08:15.110836+0000","last_fullsized":"2024-04-25T10:08:15.110836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:30.708311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.111545+0000","last_change":"2024-04-25T10:08:15.103997+0000","last_active":"2024-04-25T10:08:15.111545+0000","last_peered":"2024-04-25T10:08:15.111545+0000","last_clean":"2024-04-25T10:08:15.111545+0000","last_became_active":"2024-04-25T10:08:15.103659+0000","last_became_peered":"2024-04-25T10:08:15.103659+0000","last_unstale":"2024-04-25T10:08:15.111545+0000","last_undegraded":"2024-04-25T10:08:15.111545+0000","last_fullsized":"2024-04-25T10:08:15.111545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:36.513896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.117755+0000","last_change":"2024-04-25T10:08:15.111560+0000","last_active":"2024-04-25T10:08:15.117755+0000","last_peered":"2024-04-25T10:08:15.117755+0000","last_clean":"2024-04-25T10:08:15.117755+0000","last_became_active":"2024-04-25T10:08:15.111027+0000","last_became_peered":"2024-04-25T10:08:15.111027+0000","last_unstale":"2024-04-25T10:08:15.117755+0000","last_undegraded":"2024-04-25T10:08:15.117755+0000","last_fullsized":"2024-04-25T10:08:15.117755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:19.536090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.112894+0000","last_change":"2024-04-25T10:08:15.103160+0000","last_active":"2024-04-25T10:08:15.112894+0000","last_peered":"2024-04-25T10:08:15.112894+0000","last_clean":"2024-04-25T10:08:15.112894+0000","last_became_active":"2024-04-25T10:08:15.102831+0000","last_became_peered":"2024-04-25T10:08:15.102831+0000","last_unstale":"2024-04-25T10:08:15.112894+0000","last_undegraded":"2024-04-25T10:08:15.112894+0000","last_fullsized":"2024-04-25T10:08:15.112894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:14.806267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.132931+0000","last_change":"2024-04-25T10:08:15.128121+0000","last_active":"2024-04-25T10:08:15.132931+0000","last_peered":"2024-04-25T10:08:15.132931+0000","last_clean":"2024-04-25T10:08:15.132931+0000","last_became_active":"2024-04-25T10:08:15.127687+0000","last_became_peered":"2024-04-25T10:08:15.127687+0000","last_unstale":"2024-04-25T10:08:15.132931+0000","last_undegraded":"2024-04-25T10:08:15.132931+0000","last_fullsized":"2024-04-25T10:08:15.132931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:14.806267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.118920+0000","last_change":"2024-04-25T10:08:15.113185+0000","last_active":"2024-04-25T10:08:15.118920+0000","last_peered":"2024-04-25T10:08:15.118920+0000","last_clean":"2024-04-25T10:08:15.118920+0000","last_became_active":"2024-04-25T10:08:15.112881+0000","last_became_peered":"2024-04-25T10:08:15.112881+0000","last_unstale":"2024-04-25T10:08:15.118920+0000","last_undegraded":"2024-04-25T10:08:15.118920+0000","last_fullsized":"2024-04-25T10:08:15.118920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:00.375372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.137056+0000","last_change":"2024-04-25T10:08:15.129665+0000","last_active":"2024-04-25T10:08:15.137056+0000","last_peered":"2024-04-25T10:08:15.137056+0000","last_clean":"2024-04-25T10:08:15.137056+0000","last_became_active":"2024-04-25T10:08:15.126698+0000","last_became_peered":"2024-04-25T10:08:15.126698+0000","last_unstale":"2024-04-25T10:08:15.137056+0000","last_undegraded":"2024-04-25T10:08:15.137056+0000","last_fullsized":"2024-04-25T10:08:15.137056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:11.098185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.118140+0000","last_change":"2024-04-25T10:08:15.114347+0000","last_active":"2024-04-25T10:08:15.118140+0000","last_peered":"2024-04-25T10:08:15.118140+0000","last_clean":"2024-04-25T10:08:15.118140+0000","last_became_active":"2024-04-25T10:08:15.114100+0000","last_became_peered":"2024-04-25T10:08:15.114100+0000","last_unstale":"2024-04-25T10:08:15.118140+0000","last_undegraded":"2024-04-25T10:08:15.118140+0000","last_fullsized":"2024-04-25T10:08:15.118140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:01.431948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.110860+0000","last_change":"2024-04-25T10:08:15.104681+0000","last_active":"2024-04-25T10:08:15.110860+0000","last_peered":"2024-04-25T10:08:15.110860+0000","last_clean":"2024-04-25T10:08:15.110860+0000","last_became_active":"2024-04-25T10:08:15.104149+0000","last_became_peered":"2024-04-25T10:08:15.104149+0000","last_unstale":"2024-04-25T10:08:15.110860+0000","last_undegraded":"2024-04-25T10:08:15.110860+0000","last_fullsized":"2024-04-25T10:08:15.110860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:33.683021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.119873+0000","last_change":"2024-04-25T10:08:15.115405+0000","last_active":"2024-04-25T10:08:15.119873+0000","last_peered":"2024-04-25T10:08:15.119873+0000","last_clean":"2024-04-25T10:08:15.119873+0000","last_became_active":"2024-04-25T10:08:15.115030+0000","last_became_peered":"2024-04-25T10:08:15.115030+0000","last_unstale":"2024-04-25T10:08:15.119873+0000","last_undegraded":"2024-04-25T10:08:15.119873+0000","last_fullsized":"2024-04-25T10:08:15.119873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:11.098185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.111877+0000","last_change":"2024-04-25T10:08:15.106880+0000","last_active":"2024-04-25T10:08:15.111877+0000","last_peered":"2024-04-25T10:08:15.111877+0000","last_clean":"2024-04-25T10:08:15.111877+0000","last_became_active":"2024-04-25T10:08:15.106505+0000","last_became_peered":"2024-04-25T10:08:15.106505+0000","last_unstale":"2024-04-25T10:08:15.111877+0000","last_undegraded":"2024-04-25T10:08:15.111877+0000","last_fullsized":"2024-04-25T10:08:15.111877+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:57.719111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.117506+0000","last_change":"2024-04-25T10:08:15.111345+0000","last_active":"2024-04-25T10:08:15.117506+0000","last_peered":"2024-04-25T10:08:15.117506+0000","last_clean":"2024-04-25T10:08:15.117506+0000","last_became_active":"2024-04-25T10:08:15.110841+0000","last_became_peered":"2024-04-25T10:08:15.110841+0000","last_unstale":"2024-04-25T10:08:15.117506+0000","last_undegraded":"2024-04-25T10:08:15.117506+0000","last_fullsized":"2024-04-25T10:08:15.117506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:48.317640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.120301+0000","last_change":"2024-04-25T10:08:15.114843+0000","last_active":"2024-04-25T10:08:15.120301+0000","last_peered":"2024-04-25T10:08:15.120301+0000","last_clean":"2024-04-25T10:08:15.120301+0000","last_became_active":"2024-04-25T10:08:15.114502+0000","last_became_peered":"2024-04-25T10:08:15.114502+0000","last_unstale":"2024-04-25T10:08:15.120301+0000","last_undegraded":"2024-04-25T10:08:15.120301+0000","last_fullsized":"2024-04-25T10:08:15.120301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:51.105227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.126336+0000","last_change":"2024-04-25T10:08:14.122353+0000","last_active":"2024-04-25T10:08:15.126336+0000","last_peered":"2024-04-25T10:08:15.126336+0000","last_clean":"2024-04-25T10:08:15.126336+0000","last_became_active":"2024-04-25T10:08:14.121782+0000","last_became_peered":"2024-04-25T10:08:14.121782+0000","last_unstale":"2024-04-25T10:08:15.126336+0000","last_undegraded":"2024-04-25T10:08:15.126336+0000","last_fullsized":"2024-04-25T10:08:15.126336+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:33.368825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.106932+0000","last_change":"2024-04-25T10:08:15.101949+0000","last_active":"2024-04-25T10:08:15.106932+0000","last_peered":"2024-04-25T10:08:15.106932+0000","last_clean":"2024-04-25T10:08:15.106932+0000","last_became_active":"2024-04-25T10:08:15.101574+0000","last_became_peered":"2024-04-25T10:08:15.101574+0000","last_unstale":"2024-04-25T10:08:15.106932+0000","last_undegraded":"2024-04-25T10:08:15.106932+0000","last_fullsized":"2024-04-25T10:08:15.106932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:48.538126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.127618+0000","last_change":"2024-04-25T10:08:14.114580+0000","last_active":"2024-04-25T10:08:15.127618+0000","last_peered":"2024-04-25T10:08:15.127618+0000","last_clean":"2024-04-25T10:08:15.127618+0000","last_became_active":"2024-04-25T10:08:14.113993+0000","last_became_peered":"2024-04-25T10:08:14.113993+0000","last_unstale":"2024-04-25T10:08:15.127618+0000","last_undegraded":"2024-04-25T10:08:15.127618+0000","last_fullsized":"2024-04-25T10:08:15.127618+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:47.523609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.110503+0000","last_change":"2024-04-25T10:08:15.103415+0000","last_active":"2024-04-25T10:08:15.110503+0000","last_peered":"2024-04-25T10:08:15.110503+0000","last_clean":"2024-04-25T10:08:15.110503+0000","last_became_active":"2024-04-25T10:08:15.103054+0000","last_became_peered":"2024-04-25T10:08:15.103054+0000","last_unstale":"2024-04-25T10:08:15.110503+0000","last_undegraded":"2024-04-25T10:08:15.110503+0000","last_fullsized":"2024-04-25T10:08:15.110503+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:58.893166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.139849+0000","last_change":"2024-04-25T10:08:14.105538+0000","last_active":"2024-04-25T10:08:15.139849+0000","last_peered":"2024-04-25T10:08:15.139849+0000","last_clean":"2024-04-25T10:08:15.139849+0000","last_became_active":"2024-04-25T10:08:14.103635+0000","last_became_peered":"2024-04-25T10:08:14.103635+0000","last_unstale":"2024-04-25T10:08:15.139849+0000","last_undegraded":"2024-04-25T10:08:15.139849+0000","last_fullsized":"2024-04-25T10:08:15.139849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:49.468913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.137077+0000","last_change":"2024-04-25T10:08:15.127007+0000","last_active":"2024-04-25T10:08:15.137077+0000","last_peered":"2024-04-25T10:08:15.137077+0000","last_clean":"2024-04-25T10:08:15.137077+0000","last_became_active":"2024-04-25T10:08:15.125909+0000","last_became_peered":"2024-04-25T10:08:15.125909+0000","last_unstale":"2024-04-25T10:08:15.137077+0000","last_undegraded":"2024-04-25T10:08:15.137077+0000","last_fullsized":"2024-04-25T10:08:15.137077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:00.996884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.135764+0000","last_change":"2024-04-25T10:08:14.104539+0000","last_active":"2024-04-25T10:08:15.135764+0000","last_peered":"2024-04-25T10:08:15.135764+0000","last_clean":"2024-04-25T10:08:15.135764+0000","last_became_active":"2024-04-25T10:08:14.103497+0000","last_became_peered":"2024-04-25T10:08:14.103497+0000","last_unstale":"2024-04-25T10:08:15.135764+0000","last_undegraded":"2024-04-25T10:08:15.135764+0000","last_fullsized":"2024-04-25T10:08:15.135764+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:47.170269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.122022+0000","last_change":"2024-04-25T10:08:15.115969+0000","last_active":"2024-04-25T10:08:15.122022+0000","last_peered":"2024-04-25T10:08:15.122022+0000","last_clean":"2024-04-25T10:08:15.122022+0000","last_became_active":"2024-04-25T10:08:15.115616+0000","last_became_peered":"2024-04-25T10:08:15.115616+0000","last_unstale":"2024-04-25T10:08:15.122022+0000","last_undegraded":"2024-04-25T10:08:15.122022+0000","last_fullsized":"2024-04-25T10:08:15.122022+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:04.742692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.125220+0000","last_change":"2024-04-25T10:08:14.114127+0000","last_active":"2024-04-25T10:08:15.125220+0000","last_peered":"2024-04-25T10:08:15.125220+0000","last_clean":"2024-04-25T10:08:15.125220+0000","last_became_active":"2024-04-25T10:08:14.110197+0000","last_became_peered":"2024-04-25T10:08:14.110197+0000","last_unstale":"2024-04-25T10:08:15.125220+0000","last_undegraded":"2024-04-25T10:08:15.125220+0000","last_fullsized":"2024-04-25T10:08:15.125220+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:03.932590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.118034+0000","last_change":"2024-04-25T10:08:15.113196+0000","last_active":"2024-04-25T10:08:15.118034+0000","last_peered":"2024-04-25T10:08:15.118034+0000","last_clean":"2024-04-25T10:08:15.118034+0000","last_became_active":"2024-04-25T10:08:15.112481+0000","last_became_peered":"2024-04-25T10:08:15.112481+0000","last_unstale":"2024-04-25T10:08:15.118034+0000","last_undegraded":"2024-04-25T10:08:15.118034+0000","last_fullsized":"2024-04-25T10:08:15.118034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:11.098185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.125362+0000","last_change":"2024-04-25T10:08:14.110738+0000","last_active":"2024-04-25T10:08:15.125362+0000","last_peered":"2024-04-25T10:08:15.125362+0000","last_clean":"2024-04-25T10:08:15.125362+0000","last_became_active":"2024-04-25T10:08:14.110333+0000","last_became_peered":"2024-04-25T10:08:14.110333+0000","last_unstale":"2024-04-25T10:08:15.125362+0000","last_undegraded":"2024-04-25T10:08:15.125362+0000","last_fullsized":"2024-04-25T10:08:15.125362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:14.915348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.111693+0000","last_change":"2024-04-25T10:08:15.106221+0000","last_active":"2024-04-25T10:08:15.111693+0000","last_peered":"2024-04-25T10:08:15.111693+0000","last_clean":"2024-04-25T10:08:15.111693+0000","last_became_active":"2024-04-25T10:08:15.105934+0000","last_became_peered":"2024-04-25T10:08:15.105934+0000","last_unstale":"2024-04-25T10:08:15.111693+0000","last_undegraded":"2024-04-25T10:08:15.111693+0000","last_fullsized":"2024-04-25T10:08:15.111693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:51.105227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.077671+0000","last_change":"2024-04-25T10:08:14.104359+0000","last_active":"2024-04-25T10:08:15.077671+0000","last_peered":"2024-04-25T10:08:15.077671+0000","last_clean":"2024-04-25T10:08:15.077671+0000","last_became_active":"2024-04-25T10:08:14.103990+0000","last_became_peered":"2024-04-25T10:08:14.103990+0000","last_unstale":"2024-04-25T10:08:15.077671+0000","last_undegraded":"2024-04-25T10:08:15.077671+0000","last_fullsized":"2024-04-25T10:08:15.077671+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:32.668504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.129554+0000","last_change":"2024-04-25T10:08:15.122103+0000","last_active":"2024-04-25T10:08:15.129554+0000","last_peered":"2024-04-25T10:08:15.129554+0000","last_clean":"2024-04-25T10:08:15.129554+0000","last_became_active":"2024-04-25T10:08:15.121481+0000","last_became_peered":"2024-04-25T10:08:15.121481+0000","last_unstale":"2024-04-25T10:08:15.129554+0000","last_undegraded":"2024-04-25T10:08:15.129554+0000","last_fullsized":"2024-04-25T10:08:15.129554+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:19.536090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.083406+0000","last_change":"2024-04-25T10:08:14.116504+0000","last_active":"2024-04-25T10:08:15.083406+0000","last_peered":"2024-04-25T10:08:15.083406+0000","last_clean":"2024-04-25T10:08:15.083406+0000","last_became_active":"2024-04-25T10:08:14.116208+0000","last_became_peered":"2024-04-25T10:08:14.116208+0000","last_unstale":"2024-04-25T10:08:15.083406+0000","last_undegraded":"2024-04-25T10:08:15.083406+0000","last_fullsized":"2024-04-25T10:08:15.083406+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:13.684148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.112187+0000","last_change":"2024-04-25T10:08:15.100447+0000","last_active":"2024-04-25T10:08:15.112187+0000","last_peered":"2024-04-25T10:08:15.112187+0000","last_clean":"2024-04-25T10:08:15.112187+0000","last_became_active":"2024-04-25T10:08:15.100142+0000","last_became_peered":"2024-04-25T10:08:15.100142+0000","last_unstale":"2024-04-25T10:08:15.112187+0000","last_undegraded":"2024-04-25T10:08:15.112187+0000","last_fullsized":"2024-04-25T10:08:15.112187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:00.996884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.079865+0000","last_change":"2024-04-25T10:08:14.113674+0000","last_active":"2024-04-25T10:08:15.079865+0000","last_peered":"2024-04-25T10:08:15.079865+0000","last_clean":"2024-04-25T10:08:15.079865+0000","last_became_active":"2024-04-25T10:08:14.113375+0000","last_became_peered":"2024-04-25T10:08:14.113375+0000","last_unstale":"2024-04-25T10:08:15.079865+0000","last_undegraded":"2024-04-25T10:08:15.079865+0000","last_fullsized":"2024-04-25T10:08:15.079865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:14.920112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.133651+0000","last_change":"2024-04-25T10:08:15.129477+0000","last_active":"2024-04-25T10:08:15.133651+0000","last_peered":"2024-04-25T10:08:15.133651+0000","last_clean":"2024-04-25T10:08:15.133651+0000","last_became_active":"2024-04-25T10:08:15.129162+0000","last_became_peered":"2024-04-25T10:08:15.129162+0000","last_unstale":"2024-04-25T10:08:15.133651+0000","last_undegraded":"2024-04-25T10:08:15.133651+0000","last_fullsized":"2024-04-25T10:08:15.133651+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:18.282295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.098002+0000","last_change":"2024-04-25T10:08:14.107555+0000","last_active":"2024-04-25T10:08:15.098002+0000","last_peered":"2024-04-25T10:08:15.098002+0000","last_clean":"2024-04-25T10:08:15.098002+0000","last_became_active":"2024-04-25T10:08:14.106517+0000","last_became_peered":"2024-04-25T10:08:14.106517+0000","last_unstale":"2024-04-25T10:08:15.098002+0000","last_undegraded":"2024-04-25T10:08:15.098002+0000","last_fullsized":"2024-04-25T10:08:15.098002+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:56.704594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.118840+0000","last_change":"2024-04-25T10:08:15.113349+0000","last_active":"2024-04-25T10:08:15.118840+0000","last_peered":"2024-04-25T10:08:15.118840+0000","last_clean":"2024-04-25T10:08:15.118840+0000","last_became_active":"2024-04-25T10:08:15.112970+0000","last_became_peered":"2024-04-25T10:08:15.112970+0000","last_unstale":"2024-04-25T10:08:15.118840+0000","last_undegraded":"2024-04-25T10:08:15.118840+0000","last_fullsized":"2024-04-25T10:08:15.118840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:35.553951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.138052+0000","last_change":"2024-04-25T10:08:14.113960+0000","last_active":"2024-04-25T10:08:15.138052+0000","last_peered":"2024-04-25T10:08:15.138052+0000","last_clean":"2024-04-25T10:08:15.138052+0000","last_became_active":"2024-04-25T10:08:14.113642+0000","last_became_peered":"2024-04-25T10:08:14.113642+0000","last_unstale":"2024-04-25T10:08:15.138052+0000","last_undegraded":"2024-04-25T10:08:15.138052+0000","last_fullsized":"2024-04-25T10:08:15.138052+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:18.521573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.137769+0000","last_change":"2024-04-25T10:08:15.131535+0000","last_active":"2024-04-25T10:08:15.137769+0000","last_peered":"2024-04-25T10:08:15.137769+0000","last_clean":"2024-04-25T10:08:15.137769+0000","last_became_active":"2024-04-25T10:08:15.131252+0000","last_became_peered":"2024-04-25T10:08:15.131252+0000","last_unstale":"2024-04-25T10:08:15.137769+0000","last_undegraded":"2024-04-25T10:08:15.137769+0000","last_fullsized":"2024-04-25T10:08:15.137769+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:35.553951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.143108+0000","last_change":"2024-04-25T10:08:14.103534+0000","last_active":"2024-04-25T10:08:15.143108+0000","last_peered":"2024-04-25T10:08:15.143108+0000","last_clean":"2024-04-25T10:08:15.143108+0000","last_became_active":"2024-04-25T10:08:14.103098+0000","last_became_peered":"2024-04-25T10:08:14.103098+0000","last_unstale":"2024-04-25T10:08:15.143108+0000","last_undegraded":"2024-04-25T10:08:15.143108+0000","last_fullsized":"2024-04-25T10:08:15.143108+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:50.090710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.129668+0000","last_change":"2024-04-25T10:08:15.123389+0000","last_active":"2024-04-25T10:08:15.129668+0000","last_peered":"2024-04-25T10:08:15.129668+0000","last_clean":"2024-04-25T10:08:15.129668+0000","last_became_active":"2024-04-25T10:08:15.120955+0000","last_became_peered":"2024-04-25T10:08:15.120955+0000","last_unstale":"2024-04-25T10:08:15.129668+0000","last_undegraded":"2024-04-25T10:08:15.129668+0000","last_fullsized":"2024-04-25T10:08:15.129668+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.483430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.126581+0000","last_change":"2024-04-25T10:08:14.119402+0000","last_active":"2024-04-25T10:08:15.126581+0000","last_peered":"2024-04-25T10:08:15.126581+0000","last_clean":"2024-04-25T10:08:15.126581+0000","last_became_active":"2024-04-25T10:08:14.119057+0000","last_became_peered":"2024-04-25T10:08:14.119057+0000","last_unstale":"2024-04-25T10:08:15.126581+0000","last_undegraded":"2024-04-25T10:08:15.126581+0000","last_fullsized":"2024-04-25T10:08:15.126581+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:33.368825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.108811+0000","last_change":"2024-04-25T10:08:15.105485+0000","last_active":"2024-04-25T10:08:15.108811+0000","last_peered":"2024-04-25T10:08:15.108811+0000","last_clean":"2024-04-25T10:08:15.108811+0000","last_became_active":"2024-04-25T10:08:15.104908+0000","last_became_peered":"2024-04-25T10:08:15.104908+0000","last_unstale":"2024-04-25T10:08:15.108811+0000","last_undegraded":"2024-04-25T10:08:15.108811+0000","last_fullsized":"2024-04-25T10:08:15.108811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:11.098185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.115666+0000","last_change":"2024-04-25T10:08:14.116721+0000","last_active":"2024-04-25T10:08:15.115666+0000","last_peered":"2024-04-25T10:08:15.115666+0000","last_clean":"2024-04-25T10:08:15.115666+0000","last_became_active":"2024-04-25T10:08:14.116415+0000","last_became_peered":"2024-04-25T10:08:14.116415+0000","last_unstale":"2024-04-25T10:08:15.115666+0000","last_undegraded":"2024-04-25T10:08:15.115666+0000","last_fullsized":"2024-04-25T10:08:15.115666+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:56.704594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.117481+0000","last_change":"2024-04-25T10:08:15.111293+0000","last_active":"2024-04-25T10:08:15.117481+0000","last_peered":"2024-04-25T10:08:15.117481+0000","last_clean":"2024-04-25T10:08:15.117481+0000","last_became_active":"2024-04-25T10:08:15.110962+0000","last_became_peered":"2024-04-25T10:08:15.110962+0000","last_unstale":"2024-04-25T10:08:15.117481+0000","last_undegraded":"2024-04-25T10:08:15.117481+0000","last_fullsized":"2024-04-25T10:08:15.117481+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.483430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.078110+0000","last_change":"2024-04-25T10:08:14.118362+0000","last_active":"2024-04-25T10:08:15.078110+0000","last_peered":"2024-04-25T10:08:15.078110+0000","last_clean":"2024-04-25T10:08:15.078110+0000","last_became_active":"2024-04-25T10:08:14.118153+0000","last_became_peered":"2024-04-25T10:08:14.118153+0000","last_unstale":"2024-04-25T10:08:15.078110+0000","last_undegraded":"2024-04-25T10:08:15.078110+0000","last_fullsized":"2024-04-25T10:08:15.078110+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:18.521573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.111902+0000","last_change":"2024-04-25T10:08:15.107802+0000","last_active":"2024-04-25T10:08:15.111902+0000","last_peered":"2024-04-25T10:08:15.111902+0000","last_clean":"2024-04-25T10:08:15.111902+0000","last_became_active":"2024-04-25T10:08:15.107591+0000","last_became_peered":"2024-04-25T10:08:15.107591+0000","last_unstale":"2024-04-25T10:08:15.111902+0000","last_undegraded":"2024-04-25T10:08:15.111902+0000","last_fullsized":"2024-04-25T10:08:15.111902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:58.893166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.142483+0000","last_change":"2024-04-25T10:08:14.104238+0000","last_active":"2024-04-25T10:08:15.142483+0000","last_peered":"2024-04-25T10:08:15.142483+0000","last_clean":"2024-04-25T10:08:15.142483+0000","last_became_active":"2024-04-25T10:08:14.103941+0000","last_became_peered":"2024-04-25T10:08:14.103941+0000","last_unstale":"2024-04-25T10:08:15.142483+0000","last_undegraded":"2024-04-25T10:08:15.142483+0000","last_fullsized":"2024-04-25T10:08:15.142483+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:47.303123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.098064+0000","last_change":"2024-04-25T10:08:15.095804+0000","last_active":"2024-04-25T10:08:15.098064+0000","last_peered":"2024-04-25T10:08:15.098064+0000","last_clean":"2024-04-25T10:08:15.098064+0000","last_became_active":"2024-04-25T10:08:15.095242+0000","last_became_peered":"2024-04-25T10:08:15.095242+0000","last_unstale":"2024-04-25T10:08:15.098064+0000","last_undegraded":"2024-04-25T10:08:15.098064+0000","last_fullsized":"2024-04-25T10:08:15.098064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:57.719111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.132780+0000","last_change":"2024-04-25T10:08:14.107761+0000","last_active":"2024-04-25T10:08:15.132780+0000","last_peered":"2024-04-25T10:08:15.132780+0000","last_clean":"2024-04-25T10:08:15.132780+0000","last_became_active":"2024-04-25T10:08:14.106877+0000","last_became_peered":"2024-04-25T10:08:14.106877+0000","last_unstale":"2024-04-25T10:08:15.132780+0000","last_undegraded":"2024-04-25T10:08:15.132780+0000","last_fullsized":"2024-04-25T10:08:15.132780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:35.499379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.107664+0000","last_change":"2024-04-25T10:08:15.095635+0000","last_active":"2024-04-25T10:08:15.107664+0000","last_peered":"2024-04-25T10:08:15.107664+0000","last_clean":"2024-04-25T10:08:15.107664+0000","last_became_active":"2024-04-25T10:08:15.095285+0000","last_became_peered":"2024-04-25T10:08:15.095285+0000","last_unstale":"2024-04-25T10:08:15.107664+0000","last_undegraded":"2024-04-25T10:08:15.107664+0000","last_fullsized":"2024-04-25T10:08:15.107664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.483430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.126852+0000","last_change":"2024-04-25T10:08:14.116454+0000","last_active":"2024-04-25T10:08:15.126852+0000","last_peered":"2024-04-25T10:08:15.126852+0000","last_clean":"2024-04-25T10:08:15.126852+0000","last_became_active":"2024-04-25T10:08:14.115808+0000","last_became_peered":"2024-04-25T10:08:14.115808+0000","last_unstale":"2024-04-25T10:08:15.126852+0000","last_undegraded":"2024-04-25T10:08:15.126852+0000","last_fullsized":"2024-04-25T10:08:15.126852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:39.203427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.135146+0000","last_change":"2024-04-25T10:08:15.123291+0000","last_active":"2024-04-25T10:08:15.135146+0000","last_peered":"2024-04-25T10:08:15.135146+0000","last_clean":"2024-04-25T10:08:15.135146+0000","last_became_active":"2024-04-25T10:08:15.121418+0000","last_became_peered":"2024-04-25T10:08:15.121418+0000","last_unstale":"2024-04-25T10:08:15.135146+0000","last_undegraded":"2024-04-25T10:08:15.135146+0000","last_fullsized":"2024-04-25T10:08:15.135146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:51.105227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.126598+0000","last_change":"2024-04-25T10:08:14.108022+0000","last_active":"2024-04-25T10:08:15.126598+0000","last_peered":"2024-04-25T10:08:15.126598+0000","last_clean":"2024-04-25T10:08:15.126598+0000","last_became_active":"2024-04-25T10:08:14.107708+0000","last_became_peered":"2024-04-25T10:08:14.107708+0000","last_unstale":"2024-04-25T10:08:15.126598+0000","last_undegraded":"2024-04-25T10:08:15.126598+0000","last_fullsized":"2024-04-25T10:08:15.126598+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:42.434267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.120269+0000","last_change":"2024-04-25T10:08:15.116799+0000","last_active":"2024-04-25T10:08:15.120269+0000","last_peered":"2024-04-25T10:08:15.120269+0000","last_clean":"2024-04-25T10:08:15.120269+0000","last_became_active":"2024-04-25T10:08:15.116533+0000","last_became_peered":"2024-04-25T10:08:15.116533+0000","last_unstale":"2024-04-25T10:08:15.120269+0000","last_undegraded":"2024-04-25T10:08:15.120269+0000","last_fullsized":"2024-04-25T10:08:15.120269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:40.217944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.087362+0000","last_change":"2024-04-25T10:08:14.111048+0000","last_active":"2024-04-25T10:08:15.087362+0000","last_peered":"2024-04-25T10:08:15.087362+0000","last_clean":"2024-04-25T10:08:15.087362+0000","last_became_active":"2024-04-25T10:08:14.110739+0000","last_became_peered":"2024-04-25T10:08:14.110739+0000","last_unstale":"2024-04-25T10:08:15.087362+0000","last_undegraded":"2024-04-25T10:08:15.087362+0000","last_fullsized":"2024-04-25T10:08:15.087362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:47.523609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.120317+0000","last_change":"2024-04-25T10:08:15.118046+0000","last_active":"2024-04-25T10:08:15.120317+0000","last_peered":"2024-04-25T10:08:15.120317+0000","last_clean":"2024-04-25T10:08:15.120317+0000","last_became_active":"2024-04-25T10:08:15.115942+0000","last_became_peered":"2024-04-25T10:08:15.115942+0000","last_unstale":"2024-04-25T10:08:15.120317+0000","last_undegraded":"2024-04-25T10:08:15.120317+0000","last_fullsized":"2024-04-25T10:08:15.120317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:15.929865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.134748+0000","last_change":"2024-04-25T10:08:14.103980+0000","last_active":"2024-04-25T10:08:15.134748+0000","last_peered":"2024-04-25T10:08:15.134748+0000","last_clean":"2024-04-25T10:08:15.134748+0000","last_became_active":"2024-04-25T10:08:14.102938+0000","last_became_peered":"2024-04-25T10:08:14.102938+0000","last_unstale":"2024-04-25T10:08:15.134748+0000","last_undegraded":"2024-04-25T10:08:15.134748+0000","last_fullsized":"2024-04-25T10:08:15.134748+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:35.499379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.115987+0000","last_change":"2024-04-25T10:08:15.113005+0000","last_active":"2024-04-25T10:08:15.115987+0000","last_peered":"2024-04-25T10:08:15.115987+0000","last_clean":"2024-04-25T10:08:15.115987+0000","last_became_active":"2024-04-25T10:08:15.112210+0000","last_became_peered":"2024-04-25T10:08:15.112210+0000","last_unstale":"2024-04-25T10:08:15.115987+0000","last_undegraded":"2024-04-25T10:08:15.115987+0000","last_fullsized":"2024-04-25T10:08:15.115987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:56.791167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.091724+0000","last_change":"2024-04-25T10:08:14.121667+0000","last_active":"2024-04-25T10:08:15.091724+0000","last_peered":"2024-04-25T10:08:15.091724+0000","last_clean":"2024-04-25T10:08:15.091724+0000","last_became_active":"2024-04-25T10:08:14.121369+0000","last_became_peered":"2024-04-25T10:08:14.121369+0000","last_unstale":"2024-04-25T10:08:15.091724+0000","last_undegraded":"2024-04-25T10:08:15.091724+0000","last_fullsized":"2024-04-25T10:08:15.091724+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:59.337836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.106064+0000","last_change":"2024-04-25T10:08:15.102655+0000","last_active":"2024-04-25T10:08:15.106064+0000","last_peered":"2024-04-25T10:08:15.106064+0000","last_clean":"2024-04-25T10:08:15.106064+0000","last_became_active":"2024-04-25T10:08:15.102313+0000","last_became_peered":"2024-04-25T10:08:15.102313+0000","last_unstale":"2024-04-25T10:08:15.106064+0000","last_undegraded":"2024-04-25T10:08:15.106064+0000","last_fullsized":"2024-04-25T10:08:15.106064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:51.196612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.083645+0000","last_change":"2024-04-25T10:08:14.116705+0000","last_active":"2024-04-25T10:08:15.083645+0000","last_peered":"2024-04-25T10:08:15.083645+0000","last_clean":"2024-04-25T10:08:15.083645+0000","last_became_active":"2024-04-25T10:08:14.116246+0000","last_became_peered":"2024-04-25T10:08:14.116246+0000","last_unstale":"2024-04-25T10:08:15.083645+0000","last_undegraded":"2024-04-25T10:08:15.083645+0000","last_fullsized":"2024-04-25T10:08:15.083645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:38.540741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.119459+0000","last_change":"2024-04-25T10:08:15.113280+0000","last_active":"2024-04-25T10:08:15.119459+0000","last_peered":"2024-04-25T10:08:15.119459+0000","last_clean":"2024-04-25T10:08:15.119459+0000","last_became_active":"2024-04-25T10:08:15.112707+0000","last_became_peered":"2024-04-25T10:08:15.112707+0000","last_unstale":"2024-04-25T10:08:15.119459+0000","last_undegraded":"2024-04-25T10:08:15.119459+0000","last_fullsized":"2024-04-25T10:08:15.119459+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:15.934629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.086800+0000","last_change":"2024-04-25T10:08:14.115096+0000","last_active":"2024-04-25T10:08:15.086800+0000","last_peered":"2024-04-25T10:08:15.086800+0000","last_clean":"2024-04-25T10:08:15.086800+0000","last_became_active":"2024-04-25T10:08:14.114760+0000","last_became_peered":"2024-04-25T10:08:14.114760+0000","last_unstale":"2024-04-25T10:08:15.086800+0000","last_undegraded":"2024-04-25T10:08:15.086800+0000","last_fullsized":"2024-04-25T10:08:15.086800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:50.090710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.117361+0000","last_change":"2024-04-25T10:08:15.113919+0000","last_active":"2024-04-25T10:08:15.117361+0000","last_peered":"2024-04-25T10:08:15.117361+0000","last_clean":"2024-04-25T10:08:15.117361+0000","last_became_active":"2024-04-25T10:08:15.113594+0000","last_became_peered":"2024-04-25T10:08:15.113594+0000","last_unstale":"2024-04-25T10:08:15.117361+0000","last_undegraded":"2024-04-25T10:08:15.117361+0000","last_fullsized":"2024-04-25T10:08:15.117361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:22.586940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.132880+0000","last_change":"2024-04-25T10:08:14.107878+0000","last_active":"2024-04-25T10:08:15.132880+0000","last_peered":"2024-04-25T10:08:15.132880+0000","last_clean":"2024-04-25T10:08:15.132880+0000","last_became_active":"2024-04-25T10:08:14.107443+0000","last_became_peered":"2024-04-25T10:08:14.107443+0000","last_unstale":"2024-04-25T10:08:15.132880+0000","last_undegraded":"2024-04-25T10:08:15.132880+0000","last_fullsized":"2024-04-25T10:08:15.132880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:29.693794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.132969+0000","last_change":"2024-04-25T10:08:15.128961+0000","last_active":"2024-04-25T10:08:15.132969+0000","last_peered":"2024-04-25T10:08:15.132969+0000","last_clean":"2024-04-25T10:08:15.132969+0000","last_became_active":"2024-04-25T10:08:15.128642+0000","last_became_peered":"2024-04-25T10:08:15.128642+0000","last_unstale":"2024-04-25T10:08:15.132969+0000","last_undegraded":"2024-04-25T10:08:15.132969+0000","last_fullsized":"2024-04-25T10:08:15.132969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:04.742692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.098220+0000","last_change":"2024-04-25T10:08:14.113405+0000","last_active":"2024-04-25T10:08:15.098220+0000","last_peered":"2024-04-25T10:08:15.098220+0000","last_clean":"2024-04-25T10:08:15.098220+0000","last_became_active":"2024-04-25T10:08:14.112916+0000","last_became_peered":"2024-04-25T10:08:14.112916+0000","last_unstale":"2024-04-25T10:08:15.098220+0000","last_undegraded":"2024-04-25T10:08:15.098220+0000","last_fullsized":"2024-04-25T10:08:15.098220+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:57.878649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.133883+0000","last_change":"2024-04-25T10:08:15.132178+0000","last_active":"2024-04-25T10:08:15.133883+0000","last_peered":"2024-04-25T10:08:15.133883+0000","last_clean":"2024-04-25T10:08:15.133883+0000","last_became_active":"2024-04-25T10:08:15.131428+0000","last_became_peered":"2024-04-25T10:08:15.131428+0000","last_unstale":"2024-04-25T10:08:15.133883+0000","last_undegraded":"2024-04-25T10:08:15.133883+0000","last_fullsized":"2024-04-25T10:08:15.133883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.483430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.119183+0000","last_change":"2024-04-25T10:08:14.108366+0000","last_active":"2024-04-25T10:08:15.119183+0000","last_peered":"2024-04-25T10:08:15.119183+0000","last_clean":"2024-04-25T10:08:15.119183+0000","last_became_active":"2024-04-25T10:08:14.103224+0000","last_became_peered":"2024-04-25T10:08:14.103224+0000","last_unstale":"2024-04-25T10:08:15.119183+0000","last_undegraded":"2024-04-25T10:08:15.119183+0000","last_fullsized":"2024-04-25T10:08:15.119183+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:32.668504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.112474+0000","last_change":"2024-04-25T10:08:15.110587+0000","last_active":"2024-04-25T10:08:15.112474+0000","last_peered":"2024-04-25T10:08:15.112474+0000","last_clean":"2024-04-25T10:08:15.112474+0000","last_became_active":"2024-04-25T10:08:15.110013+0000","last_became_peered":"2024-04-25T10:08:15.110013+0000","last_unstale":"2024-04-25T10:08:15.112474+0000","last_undegraded":"2024-04-25T10:08:15.112474+0000","last_fullsized":"2024-04-25T10:08:15.112474+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:36.513896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.098694+0000","last_change":"2024-04-25T10:08:14.104838+0000","last_active":"2024-04-25T10:08:15.098694+0000","last_peered":"2024-04-25T10:08:15.098694+0000","last_clean":"2024-04-25T10:08:15.098694+0000","last_became_active":"2024-04-25T10:08:14.104537+0000","last_became_peered":"2024-04-25T10:08:14.104537+0000","last_unstale":"2024-04-25T10:08:15.098694+0000","last_undegraded":"2024-04-25T10:08:15.098694+0000","last_fullsized":"2024-04-25T10:08:15.098694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:29.693794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.111454+0000","last_change":"2024-04-25T10:08:15.103258+0000","last_active":"2024-04-25T10:08:15.111454+0000","last_peered":"2024-04-25T10:08:15.111454+0000","last_clean":"2024-04-25T10:08:15.111454+0000","last_became_active":"2024-04-25T10:08:15.102838+0000","last_became_peered":"2024-04-25T10:08:15.102838+0000","last_unstale":"2024-04-25T10:08:15.111454+0000","last_undegraded":"2024-04-25T10:08:15.111454+0000","last_fullsized":"2024-04-25T10:08:15.111454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:33.683021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.115800+0000","last_change":"2024-04-25T10:08:14.107630+0000","last_active":"2024-04-25T10:08:15.115800+0000","last_peered":"2024-04-25T10:08:15.115800+0000","last_clean":"2024-04-25T10:08:15.115800+0000","last_became_active":"2024-04-25T10:08:14.107342+0000","last_became_peered":"2024-04-25T10:08:14.107342+0000","last_unstale":"2024-04-25T10:08:15.115800+0000","last_undegraded":"2024-04-25T10:08:15.115800+0000","last_fullsized":"2024-04-25T10:08:15.115800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:57.878649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.121009+0000","last_change":"2024-04-25T10:08:15.120102+0000","last_active":"2024-04-25T10:08:15.121009+0000","last_peered":"2024-04-25T10:08:15.121009+0000","last_clean":"2024-04-25T10:08:15.121009+0000","last_became_active":"2024-04-25T10:08:15.119752+0000","last_became_peered":"2024-04-25T10:08:15.119752+0000","last_unstale":"2024-04-25T10:08:15.121009+0000","last_undegraded":"2024-04-25T10:08:15.121009+0000","last_fullsized":"2024-04-25T10:08:15.121009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:41.876481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.083680+0000","last_change":"2024-04-25T10:08:14.113420+0000","last_active":"2024-04-25T10:08:15.083680+0000","last_peered":"2024-04-25T10:08:15.083680+0000","last_clean":"2024-04-25T10:08:15.083680+0000","last_became_active":"2024-04-25T10:08:14.113110+0000","last_became_peered":"2024-04-25T10:08:14.113110+0000","last_unstale":"2024-04-25T10:08:15.083680+0000","last_undegraded":"2024-04-25T10:08:15.083680+0000","last_fullsized":"2024-04-25T10:08:15.083680+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:59.337836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.110923+0000","last_change":"2024-04-25T10:08:15.103423+0000","last_active":"2024-04-25T10:08:15.110923+0000","last_peered":"2024-04-25T10:08:15.110923+0000","last_clean":"2024-04-25T10:08:15.110923+0000","last_became_active":"2024-04-25T10:08:15.102956+0000","last_became_peered":"2024-04-25T10:08:15.102956+0000","last_unstale":"2024-04-25T10:08:15.110923+0000","last_undegraded":"2024-04-25T10:08:15.110923+0000","last_fullsized":"2024-04-25T10:08:15.110923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:19.536090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.078970+0000","last_change":"2024-04-25T10:08:14.115663+0000","last_active":"2024-04-25T10:08:15.078970+0000","last_peered":"2024-04-25T10:08:15.078970+0000","last_clean":"2024-04-25T10:08:15.078970+0000","last_became_active":"2024-04-25T10:08:14.115375+0000","last_became_peered":"2024-04-25T10:08:14.115375+0000","last_unstale":"2024-04-25T10:08:15.078970+0000","last_undegraded":"2024-04-25T10:08:15.078970+0000","last_fullsized":"2024-04-25T10:08:15.078970+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:59.982367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.118725+0000","last_change":"2024-04-25T10:08:15.112405+0000","last_active":"2024-04-25T10:08:15.118725+0000","last_peered":"2024-04-25T10:08:15.118725+0000","last_clean":"2024-04-25T10:08:15.118725+0000","last_became_active":"2024-04-25T10:08:15.112083+0000","last_became_peered":"2024-04-25T10:08:15.112083+0000","last_unstale":"2024-04-25T10:08:15.118725+0000","last_undegraded":"2024-04-25T10:08:15.118725+0000","last_fullsized":"2024-04-25T10:08:15.118725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:00.996884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.092037+0000","last_change":"2024-04-25T10:08:14.124287+0000","last_active":"2024-04-25T10:08:15.092037+0000","last_peered":"2024-04-25T10:08:15.092037+0000","last_clean":"2024-04-25T10:08:15.092037+0000","last_became_active":"2024-04-25T10:08:14.123910+0000","last_became_peered":"2024-04-25T10:08:14.123910+0000","last_unstale":"2024-04-25T10:08:15.092037+0000","last_undegraded":"2024-04-25T10:08:15.092037+0000","last_fullsized":"2024-04-25T10:08:15.092037+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:03.932590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.098539+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:08:15.098539+0000","last_peered":"2024-04-25T10:08:15.098539+0000","last_clean":"2024-04-25T10:08:15.098539+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:08:15.098539+0000","last_undegraded":"2024-04-25T10:08:15.098539+0000","last_fullsized":"2024-04-25T10:08:15.098539+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":958464,"data_stored":730636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":140794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":140850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":348160,"data_stored":133403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":141812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":140248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":141812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":954368,"data_stored":730546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:19.498 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:19.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.730+0000 7f1c7a200640 1 Processor -- start 2024-04-25T10:08:19.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.730+0000 7f1c7a200640 1 -- start start 2024-04-25T10:08:19.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.732+0000 7f1c7a200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1c740d3f80 0x7f1c740d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.732+0000 7f1c7a200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c740b9270 0x7f1c740d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.732+0000 7f1c7a200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7415ef10 con 0x7f1c7415db60 2024-04-25T10:08:19.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.732+0000 7f1c7a200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7415e390 con 0x7f1c740d3f80 2024-04-25T10:08:19.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.732+0000 7f1c7a200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7415e670 con 0x7f1c740b9270 2024-04-25T10:08:19.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.732+0000 7f1c78e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1c7415db60 0x7f1c740d3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39748/0 (socket says 172.21.15.119:39748) 2024-04-25T10:08:19.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.732+0000 7f1c78e00640 1 -- 172.21.15.119:0/427641076 learned_addr learned my addr 172.21.15.119:0/427641076 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:19.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 -- 172.21.15.119:0/427641076 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218183995 0 0) 0x7f1c7415ef10 con 0x7f1c7415db60 2024-04-25T10:08:19.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c79800640 1 --2- 172.21.15.119:0/427641076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c740b9270 0x7f1c740d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 -- 172.21.15.119:0/427641076 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c58003520 con 0x7f1c7415db60 2024-04-25T10:08:19.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c6be00640 1 --2- 172.21.15.119:0/427641076 >> v2:172.21.15.132:3300/0 conn(0x7f1c740d3f80 0x7f1c740d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 -- 172.21.15.119:0/427641076 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3266588039 0 0) 0x7f1c58003520 con 0x7f1c7415db60 2024-04-25T10:08:19.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 -- 172.21.15.119:0/427641076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c740b9270 msgr2=0x7f1c740d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:19.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 --2- 172.21.15.119:0/427641076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c740b9270 0x7f1c740d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 -- 172.21.15.119:0/427641076 >> v2:172.21.15.132:3300/0 conn(0x7f1c740d3f80 msgr2=0x7f1c740d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:19.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 --2- 172.21.15.119:0/427641076 >> v2:172.21.15.132:3300/0 conn(0x7f1c740d3f80 0x7f1c740d1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 -- 172.21.15.119:0/427641076 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7415e950 con 0x7f1c7415db60 2024-04-25T10:08:19.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 -- 172.21.15.119:0/427641076 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f1c64001690 con 0x7f1c7415db60 2024-04-25T10:08:19.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 -- 172.21.15.119:0/427641076 >> v1:172.21.15.119:6789/0 conn(0x7f1c7415db60 legacy=0x7f1c740d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 --2- 172.21.15.119:0/427641076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c58003a20 0x7f1c58003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 --2- 172.21.15.119:0/427641076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c740b9270 0x7f1c58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 --2- 172.21.15.119:0/427641076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58004d40 0x7f1c580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 -- 172.21.15.119:0/427641076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7415e390 con 0x7f1c58004d40 2024-04-25T10:08:19.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 -- 172.21.15.119:0/427641076 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7415e670 con 0x7f1c58003a20 2024-04-25T10:08:19.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.733+0000 7f1c73e00640 1 -- 172.21.15.119:0/427641076 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5800b9c0 con 0x7f1c740b9270 2024-04-25T10:08:19.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.734+0000 7f1c79800640 1 --2- 172.21.15.119:0/427641076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58004d40 0x7f1c580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.734+0000 7f1c78e00640 1 --2- 172.21.15.119:0/427641076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c58003a20 0x7f1c58003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.734+0000 7f1c6be00640 1 --2- 172.21.15.119:0/427641076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c740b9270 0x7f1c58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.734+0000 7f1c79800640 1 -- 172.21.15.119:0/427641076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c740b9270 msgr2=0x7f1c58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:19.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.734+0000 7f1c79800640 1 --2- 172.21.15.119:0/427641076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c740b9270 0x7f1c58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.734+0000 7f1c79800640 1 -- 172.21.15.119:0/427641076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c58003a20 msgr2=0x7f1c58003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:19.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.734+0000 7f1c79800640 1 --2- 172.21.15.119:0/427641076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c58003a20 0x7f1c58003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.734+0000 7f1c79800640 1 -- 172.21.15.119:0/427641076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58003520 con 0x7f1c58004d40 2024-04-25T10:08:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.735+0000 7f1c79800640 1 --2- 172.21.15.119:0/427641076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58004d40 0x7f1c580051c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009fc0 tx=0x7f1c58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d212321409066d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.735+0000 7f1c73e00640 1 -- 172.21.15.119:0/427641076 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c009c10 con 0x7f1c58004d40 2024-04-25T10:08:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.735+0000 7f1c73e00640 1 -- 172.21.15.119:0/427641076 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c036070 con 0x7f1c58004d40 2024-04-25T10:08:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.735+0000 7f1c73e00640 1 -- 172.21.15.119:0/427641076 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c031880 con 0x7f1c58004d40 2024-04-25T10:08:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.736+0000 7f1c7a200640 1 -- 172.21.15.119:0/427641076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58004d40 msgr2=0x7f1c580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.736+0000 7f1c7a200640 1 --2- 172.21.15.119:0/427641076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c58004d40 0x7f1c580051c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009fc0 tx=0x7f1c58005830 comp rx=0 tx=0).stop 2024-04-25T10:08:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.736+0000 7f1c79800640 1 -- 172.21.15.119:0/427641076 reap_dead start 2024-04-25T10:08:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.736+0000 7f1c7a200640 1 -- 172.21.15.119:0/427641076 shutdown_connections 2024-04-25T10:08:19.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.736+0000 7f1c7a200640 1 -- 172.21.15.119:0/427641076 >> 172.21.15.119:0/427641076 conn(0x7f1c740726c0 msgr2=0x7f1c74072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:19.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.736+0000 7f1c7a200640 1 -- 172.21.15.119:0/427641076 shutdown_connections 2024-04-25T10:08:19.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.736+0000 7f1c7a200640 1 -- 172.21.15.119:0/427641076 wait complete. 2024-04-25T10:08:19.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.738+0000 7f1c7a200640 1 Processor -- start 2024-04-25T10:08:19.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.738+0000 7f1c7a200640 1 -- start start 2024-04-25T10:08:19.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.739+0000 7f1c7a200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c58004d40 0x7f1c741f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.739+0000 7f1c7a200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740b57a0 0x7f1c740b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.739+0000 7f1c7a200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c740b5c20 0x7f1c7415b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.739+0000 7f1c7a200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740b3120 con 0x7f1c740b5c20 2024-04-25T10:08:19.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.739+0000 7f1c7a200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740b2fa0 con 0x7f1c740b57a0 2024-04-25T10:08:19.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.739+0000 7f1c7a200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7415df40 con 0x7f1c58004d40 2024-04-25T10:08:19.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.739+0000 7f1c79800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c740b5c20 0x7f1c7415b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.739+0000 7f1c79800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c740b5c20 0x7f1c7415b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34422/0 (socket says 172.21.15.119:34422) 2024-04-25T10:08:19.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.739+0000 7f1c79800640 1 -- 172.21.15.119:0/1033323516 learned_addr learned my addr 172.21.15.119:0/1033323516 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:19.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.740+0000 7f1c78e00640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c58004d40 0x7f1c741f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.740+0000 7f1c6be00640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740b57a0 0x7f1c740b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.740+0000 7f1c79800640 1 -- 172.21.15.119:0/1033323516 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c58004d40 msgr2=0x7f1c741f32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.740+0000 7f1c79800640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c58004d40 0x7f1c741f32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.740+0000 7f1c79800640 1 -- 172.21.15.119:0/1033323516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740b57a0 msgr2=0x7f1c740b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.740+0000 7f1c79800640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740b57a0 0x7f1c740b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.740+0000 7f1c79800640 1 -- 172.21.15.119:0/1033323516 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c740b3d60 con 0x7f1c740b5c20 2024-04-25T10:08:19.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.740+0000 7f1c6be00640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740b57a0 0x7f1c740b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:19.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.740+0000 7f1c79800640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c740b5c20 0x7f1c7415b750 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00e230 tx=0x7f1c6c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.741+0000 7f1c71600640 1 -- 172.21.15.119:0/1033323516 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c009c10 con 0x7f1c740b5c20 2024-04-25T10:08:19.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.741+0000 7f1c71600640 1 -- 172.21.15.119:0/1033323516 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c031dd0 con 0x7f1c740b5c20 2024-04-25T10:08:19.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.741+0000 7f1c71600640 1 -- 172.21.15.119:0/1033323516 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c003bd0 con 0x7f1c740b5c20 2024-04-25T10:08:19.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.741+0000 7f1c7a200640 1 -- 172.21.15.119:0/1033323516 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c740b4040 con 0x7f1c740b5c20 2024-04-25T10:08:19.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.741+0000 7f1c7a200640 1 -- 172.21.15.119:0/1033323516 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c740b5280 con 0x7f1c740b5c20 2024-04-25T10:08:19.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.742+0000 7f1c7a200640 1 -- 172.21.15.119:0/1033323516 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c740749b0 con 0x7f1c740b5c20 2024-04-25T10:08:19.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.743+0000 7f1c71600640 1 -- 172.21.15.119:0/1033323516 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c6c005d10 con 0x7f1c740b5c20 2024-04-25T10:08:19.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.744+0000 7f1c71600640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1c5006b4d0 0x7f1c5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.744+0000 7f1c71600640 1 -- 172.21.15.119:0/1033323516 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c6c0ab520 con 0x7f1c740b5c20 2024-04-25T10:08:19.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.744+0000 7f1c78e00640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1c5006b4d0 0x7f1c5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.745+0000 7f1c78e00640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1c5006b4d0 0x7f1c5006d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1c740d37c0 tx=0x7f1c64000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.748+0000 7f1c71600640 1 -- 172.21.15.119:0/1033323516 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c6c07a4b0 con 0x7f1c740b5c20 2024-04-25T10:08:19.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.938+0000 7f1c7a200640 1 -- 172.21.15.119:0/1033323516 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1c740b5450 con 0x7f1c5006b4d0 2024-04-25T10:08:19.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.946+0000 7f1c71600640 1 -- 172.21.15.119:0/1033323516 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251882 (secure 0 0 0) 0x7f1c740b5450 con 0x7f1c5006b4d0 2024-04-25T10:08:19.955 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:19.957 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:08:19.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.951+0000 7f1c7a200640 1 -- 172.21.15.119:0/1033323516 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1c5006b4d0 msgr2=0x7f1c5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.951+0000 7f1c7a200640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1c5006b4d0 0x7f1c5006d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1c740d37c0 tx=0x7f1c64000ed0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.951+0000 7f1c7a200640 1 -- 172.21.15.119:0/1033323516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c740b5c20 msgr2=0x7f1c7415b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.951+0000 7f1c7a200640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c740b5c20 0x7f1c7415b750 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00e230 tx=0x7f1c6c009750 comp rx=0 tx=0).stop 2024-04-25T10:08:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.951+0000 7f1c7a200640 1 -- 172.21.15.119:0/1033323516 shutdown_connections 2024-04-25T10:08:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.951+0000 7f1c7a200640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1c5006b4d0 0x7f1c5006d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.951+0000 7f1c7a200640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c740b5c20 0x7f1c7415b750 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.951+0000 7f1c7a200640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c740b57a0 0x7f1c740b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.951+0000 7f1c7a200640 1 --2- 172.21.15.119:0/1033323516 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c58004d40 0x7f1c741f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.951+0000 7f1c7a200640 1 -- 172.21.15.119:0/1033323516 >> 172.21.15.119:0/1033323516 conn(0x7f1c740726c0 msgr2=0x7f1c740b7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.951+0000 7f1c7a200640 1 -- 172.21.15.119:0/1033323516 shutdown_connections 2024-04-25T10:08:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:19.951+0000 7f1c7a200640 1 -- 172.21.15.119:0/1033323516 wait complete. 2024-04-25T10:08:19.976 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-25T10:08:17.947595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220228,"kb_used_data":4020,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754492,"statfs":{"total":773094113280,"available":772868599808,"internally_reserved":0,"allocated":4116480,"data_stored":2300101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.884118"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.104304+0000","last_change":"2024-04-25T10:08:15.100083+0000","last_active":"2024-04-25T10:08:15.104304+0000","last_peered":"2024-04-25T10:08:15.104304+0000","last_clean":"2024-04-25T10:08:15.104304+0000","last_became_active":"2024-04-25T10:08:15.099636+0000","last_became_peered":"2024-04-25T10:08:15.099636+0000","last_unstale":"2024-04-25T10:08:15.104304+0000","last_undegraded":"2024-04-25T10:08:15.104304+0000","last_fullsized":"2024-04-25T10:08:15.104304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:43.448784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.117707+0000","last_change":"2024-04-25T10:08:15.110342+0000","last_active":"2024-04-25T10:08:15.117707+0000","last_peered":"2024-04-25T10:08:15.117707+0000","last_clean":"2024-04-25T10:08:15.117707+0000","last_became_active":"2024-04-25T10:08:15.108964+0000","last_became_peered":"2024-04-25T10:08:15.108964+0000","last_unstale":"2024-04-25T10:08:15.117707+0000","last_undegraded":"2024-04-25T10:08:15.117707+0000","last_fullsized":"2024-04-25T10:08:15.117707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:04.742692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.137746+0000","last_change":"2024-04-25T10:08:15.131503+0000","last_active":"2024-04-25T10:08:15.137746+0000","last_peered":"2024-04-25T10:08:15.137746+0000","last_clean":"2024-04-25T10:08:15.137746+0000","last_became_active":"2024-04-25T10:08:15.131249+0000","last_became_peered":"2024-04-25T10:08:15.131249+0000","last_unstale":"2024-04-25T10:08:15.137746+0000","last_undegraded":"2024-04-25T10:08:15.137746+0000","last_fullsized":"2024-04-25T10:08:15.137746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:00.375372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.135167+0000","last_change":"2024-04-25T10:08:15.124953+0000","last_active":"2024-04-25T10:08:15.135167+0000","last_peered":"2024-04-25T10:08:15.135167+0000","last_clean":"2024-04-25T10:08:15.135167+0000","last_became_active":"2024-04-25T10:08:15.122813+0000","last_became_peered":"2024-04-25T10:08:15.122813+0000","last_unstale":"2024-04-25T10:08:15.135167+0000","last_undegraded":"2024-04-25T10:08:15.135167+0000","last_fullsized":"2024-04-25T10:08:15.135167+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:48.317640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.114107+0000","last_change":"2024-04-25T10:08:15.107948+0000","last_active":"2024-04-25T10:08:15.114107+0000","last_peered":"2024-04-25T10:08:15.114107+0000","last_clean":"2024-04-25T10:08:15.114107+0000","last_became_active":"2024-04-25T10:08:15.107617+0000","last_became_peered":"2024-04-25T10:08:15.107617+0000","last_unstale":"2024-04-25T10:08:15.114107+0000","last_undegraded":"2024-04-25T10:08:15.114107+0000","last_fullsized":"2024-04-25T10:08:15.114107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:14.743414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.100615+0000","last_change":"2024-04-25T10:08:15.095553+0000","last_active":"2024-04-25T10:08:15.100615+0000","last_peered":"2024-04-25T10:08:15.100615+0000","last_clean":"2024-04-25T10:08:15.100615+0000","last_became_active":"2024-04-25T10:08:15.095089+0000","last_became_peered":"2024-04-25T10:08:15.095089+0000","last_unstale":"2024-04-25T10:08:15.100615+0000","last_undegraded":"2024-04-25T10:08:15.100615+0000","last_fullsized":"2024-04-25T10:08:15.100615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:14.806267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.119487+0000","last_change":"2024-04-25T10:08:15.113757+0000","last_active":"2024-04-25T10:08:15.119487+0000","last_peered":"2024-04-25T10:08:15.119487+0000","last_clean":"2024-04-25T10:08:15.119487+0000","last_became_active":"2024-04-25T10:08:15.113530+0000","last_became_peered":"2024-04-25T10:08:15.113530+0000","last_unstale":"2024-04-25T10:08:15.119487+0000","last_undegraded":"2024-04-25T10:08:15.119487+0000","last_fullsized":"2024-04-25T10:08:15.119487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:14.806267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.113098+0000","last_change":"2024-04-25T10:08:15.108497+0000","last_active":"2024-04-25T10:08:15.113098+0000","last_peered":"2024-04-25T10:08:15.113098+0000","last_clean":"2024-04-25T10:08:15.113098+0000","last_became_active":"2024-04-25T10:08:15.107919+0000","last_became_peered":"2024-04-25T10:08:15.107919+0000","last_unstale":"2024-04-25T10:08:15.113098+0000","last_undegraded":"2024-04-25T10:08:15.113098+0000","last_fullsized":"2024-04-25T10:08:15.113098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:51.105227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.133099+0000","last_change":"2024-04-25T10:08:15.129184+0000","last_active":"2024-04-25T10:08:15.133099+0000","last_peered":"2024-04-25T10:08:15.133099+0000","last_clean":"2024-04-25T10:08:15.133099+0000","last_became_active":"2024-04-25T10:08:15.128910+0000","last_became_peered":"2024-04-25T10:08:15.128910+0000","last_unstale":"2024-04-25T10:08:15.133099+0000","last_undegraded":"2024-04-25T10:08:15.133099+0000","last_fullsized":"2024-04-25T10:08:15.133099+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.483430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.132808+0000","last_change":"2024-04-25T10:08:15.127981+0000","last_active":"2024-04-25T10:08:15.132808+0000","last_peered":"2024-04-25T10:08:15.132808+0000","last_clean":"2024-04-25T10:08:15.132808+0000","last_became_active":"2024-04-25T10:08:15.127713+0000","last_became_peered":"2024-04-25T10:08:15.127713+0000","last_unstale":"2024-04-25T10:08:15.132808+0000","last_undegraded":"2024-04-25T10:08:15.132808+0000","last_fullsized":"2024-04-25T10:08:15.132808+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:15.934629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.137757+0000","last_change":"2024-04-25T10:08:15.131791+0000","last_active":"2024-04-25T10:08:15.137757+0000","last_peered":"2024-04-25T10:08:15.137757+0000","last_clean":"2024-04-25T10:08:15.137757+0000","last_became_active":"2024-04-25T10:08:15.131478+0000","last_became_peered":"2024-04-25T10:08:15.131478+0000","last_unstale":"2024-04-25T10:08:15.137757+0000","last_undegraded":"2024-04-25T10:08:15.137757+0000","last_fullsized":"2024-04-25T10:08:15.137757+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:00.375372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.114245+0000","last_change":"2024-04-25T10:08:15.101166+0000","last_active":"2024-04-25T10:08:15.114245+0000","last_peered":"2024-04-25T10:08:15.114245+0000","last_clean":"2024-04-25T10:08:15.114245+0000","last_became_active":"2024-04-25T10:08:15.100782+0000","last_became_peered":"2024-04-25T10:08:15.100782+0000","last_unstale":"2024-04-25T10:08:15.114245+0000","last_undegraded":"2024-04-25T10:08:15.114245+0000","last_fullsized":"2024-04-25T10:08:15.114245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:15.934629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.133058+0000","last_change":"2024-04-25T10:08:15.129100+0000","last_active":"2024-04-25T10:08:15.133058+0000","last_peered":"2024-04-25T10:08:15.133058+0000","last_clean":"2024-04-25T10:08:15.133058+0000","last_became_active":"2024-04-25T10:08:15.128746+0000","last_became_peered":"2024-04-25T10:08:15.128746+0000","last_unstale":"2024-04-25T10:08:15.133058+0000","last_undegraded":"2024-04-25T10:08:15.133058+0000","last_fullsized":"2024-04-25T10:08:15.133058+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:14.698665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.115066+0000","last_change":"2024-04-25T10:08:15.108123+0000","last_active":"2024-04-25T10:08:15.115066+0000","last_peered":"2024-04-25T10:08:15.115066+0000","last_clean":"2024-04-25T10:08:15.115066+0000","last_became_active":"2024-04-25T10:08:15.105973+0000","last_became_peered":"2024-04-25T10:08:15.105973+0000","last_unstale":"2024-04-25T10:08:15.115066+0000","last_undegraded":"2024-04-25T10:08:15.115066+0000","last_fullsized":"2024-04-25T10:08:15.115066+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:01.431948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.111156+0000","last_change":"2024-04-25T10:08:15.100006+0000","last_active":"2024-04-25T10:08:15.111156+0000","last_peered":"2024-04-25T10:08:15.111156+0000","last_clean":"2024-04-25T10:08:15.111156+0000","last_became_active":"2024-04-25T10:08:15.099703+0000","last_became_peered":"2024-04-25T10:08:15.099703+0000","last_unstale":"2024-04-25T10:08:15.111156+0000","last_undegraded":"2024-04-25T10:08:15.111156+0000","last_fullsized":"2024-04-25T10:08:15.111156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:48.317640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.113076+0000","last_change":"2024-04-25T10:08:15.108210+0000","last_active":"2024-04-25T10:08:15.113076+0000","last_peered":"2024-04-25T10:08:15.113076+0000","last_clean":"2024-04-25T10:08:15.113076+0000","last_became_active":"2024-04-25T10:08:15.107547+0000","last_became_peered":"2024-04-25T10:08:15.107547+0000","last_unstale":"2024-04-25T10:08:15.113076+0000","last_undegraded":"2024-04-25T10:08:15.113076+0000","last_fullsized":"2024-04-25T10:08:15.113076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:19.536090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.119747+0000","last_change":"2024-04-25T10:08:15.115863+0000","last_active":"2024-04-25T10:08:15.119747+0000","last_peered":"2024-04-25T10:08:15.119747+0000","last_clean":"2024-04-25T10:08:15.119747+0000","last_became_active":"2024-04-25T10:08:15.115478+0000","last_became_peered":"2024-04-25T10:08:15.115478+0000","last_unstale":"2024-04-25T10:08:15.119747+0000","last_undegraded":"2024-04-25T10:08:15.119747+0000","last_fullsized":"2024-04-25T10:08:15.119747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:01.431948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.133615+0000","last_change":"2024-04-25T10:08:15.129416+0000","last_active":"2024-04-25T10:08:15.133615+0000","last_peered":"2024-04-25T10:08:15.133615+0000","last_clean":"2024-04-25T10:08:15.133615+0000","last_became_active":"2024-04-25T10:08:15.129118+0000","last_became_peered":"2024-04-25T10:08:15.129118+0000","last_unstale":"2024-04-25T10:08:15.133615+0000","last_undegraded":"2024-04-25T10:08:15.133615+0000","last_fullsized":"2024-04-25T10:08:15.133615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:01.055962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.136534+0000","last_change":"2024-04-25T10:08:15.135479+0000","last_active":"2024-04-25T10:08:15.136534+0000","last_peered":"2024-04-25T10:08:15.136534+0000","last_clean":"2024-04-25T10:08:15.136534+0000","last_became_active":"2024-04-25T10:08:15.135119+0000","last_became_peered":"2024-04-25T10:08:15.135119+0000","last_unstale":"2024-04-25T10:08:15.136534+0000","last_undegraded":"2024-04-25T10:08:15.136534+0000","last_fullsized":"2024-04-25T10:08:15.136534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:50.400597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.119884+0000","last_change":"2024-04-25T10:08:15.115513+0000","last_active":"2024-04-25T10:08:15.119884+0000","last_peered":"2024-04-25T10:08:15.119884+0000","last_clean":"2024-04-25T10:08:15.119884+0000","last_became_active":"2024-04-25T10:08:15.115169+0000","last_became_peered":"2024-04-25T10:08:15.115169+0000","last_unstale":"2024-04-25T10:08:15.119884+0000","last_undegraded":"2024-04-25T10:08:15.119884+0000","last_fullsized":"2024-04-25T10:08:15.119884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:55.173693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.110836+0000","last_change":"2024-04-25T10:08:15.104432+0000","last_active":"2024-04-25T10:08:15.110836+0000","last_peered":"2024-04-25T10:08:15.110836+0000","last_clean":"2024-04-25T10:08:15.110836+0000","last_became_active":"2024-04-25T10:08:15.103725+0000","last_became_peered":"2024-04-25T10:08:15.103725+0000","last_unstale":"2024-04-25T10:08:15.110836+0000","last_undegraded":"2024-04-25T10:08:15.110836+0000","last_fullsized":"2024-04-25T10:08:15.110836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:30.708311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.111545+0000","last_change":"2024-04-25T10:08:15.103997+0000","last_active":"2024-04-25T10:08:15.111545+0000","last_peered":"2024-04-25T10:08:15.111545+0000","last_clean":"2024-04-25T10:08:15.111545+0000","last_became_active":"2024-04-25T10:08:15.103659+0000","last_became_peered":"2024-04-25T10:08:15.103659+0000","last_unstale":"2024-04-25T10:08:15.111545+0000","last_undegraded":"2024-04-25T10:08:15.111545+0000","last_fullsized":"2024-04-25T10:08:15.111545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:36.513896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.117755+0000","last_change":"2024-04-25T10:08:15.111560+0000","last_active":"2024-04-25T10:08:15.117755+0000","last_peered":"2024-04-25T10:08:15.117755+0000","last_clean":"2024-04-25T10:08:15.117755+0000","last_became_active":"2024-04-25T10:08:15.111027+0000","last_became_peered":"2024-04-25T10:08:15.111027+0000","last_unstale":"2024-04-25T10:08:15.117755+0000","last_undegraded":"2024-04-25T10:08:15.117755+0000","last_fullsized":"2024-04-25T10:08:15.117755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:19.536090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.112894+0000","last_change":"2024-04-25T10:08:15.103160+0000","last_active":"2024-04-25T10:08:15.112894+0000","last_peered":"2024-04-25T10:08:15.112894+0000","last_clean":"2024-04-25T10:08:15.112894+0000","last_became_active":"2024-04-25T10:08:15.102831+0000","last_became_peered":"2024-04-25T10:08:15.102831+0000","last_unstale":"2024-04-25T10:08:15.112894+0000","last_undegraded":"2024-04-25T10:08:15.112894+0000","last_fullsized":"2024-04-25T10:08:15.112894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:14.806267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.132931+0000","last_change":"2024-04-25T10:08:15.128121+0000","last_active":"2024-04-25T10:08:15.132931+0000","last_peered":"2024-04-25T10:08:15.132931+0000","last_clean":"2024-04-25T10:08:15.132931+0000","last_became_active":"2024-04-25T10:08:15.127687+0000","last_became_peered":"2024-04-25T10:08:15.127687+0000","last_unstale":"2024-04-25T10:08:15.132931+0000","last_undegraded":"2024-04-25T10:08:15.132931+0000","last_fullsized":"2024-04-25T10:08:15.132931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:14.806267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.118920+0000","last_change":"2024-04-25T10:08:15.113185+0000","last_active":"2024-04-25T10:08:15.118920+0000","last_peered":"2024-04-25T10:08:15.118920+0000","last_clean":"2024-04-25T10:08:15.118920+0000","last_became_active":"2024-04-25T10:08:15.112881+0000","last_became_peered":"2024-04-25T10:08:15.112881+0000","last_unstale":"2024-04-25T10:08:15.118920+0000","last_undegraded":"2024-04-25T10:08:15.118920+0000","last_fullsized":"2024-04-25T10:08:15.118920+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:00.375372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.137056+0000","last_change":"2024-04-25T10:08:15.129665+0000","last_active":"2024-04-25T10:08:15.137056+0000","last_peered":"2024-04-25T10:08:15.137056+0000","last_clean":"2024-04-25T10:08:15.137056+0000","last_became_active":"2024-04-25T10:08:15.126698+0000","last_became_peered":"2024-04-25T10:08:15.126698+0000","last_unstale":"2024-04-25T10:08:15.137056+0000","last_undegraded":"2024-04-25T10:08:15.137056+0000","last_fullsized":"2024-04-25T10:08:15.137056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:11.098185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.118140+0000","last_change":"2024-04-25T10:08:15.114347+0000","last_active":"2024-04-25T10:08:15.118140+0000","last_peered":"2024-04-25T10:08:15.118140+0000","last_clean":"2024-04-25T10:08:15.118140+0000","last_became_active":"2024-04-25T10:08:15.114100+0000","last_became_peered":"2024-04-25T10:08:15.114100+0000","last_unstale":"2024-04-25T10:08:15.118140+0000","last_undegraded":"2024-04-25T10:08:15.118140+0000","last_fullsized":"2024-04-25T10:08:15.118140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:01.431948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.110860+0000","last_change":"2024-04-25T10:08:15.104681+0000","last_active":"2024-04-25T10:08:15.110860+0000","last_peered":"2024-04-25T10:08:15.110860+0000","last_clean":"2024-04-25T10:08:15.110860+0000","last_became_active":"2024-04-25T10:08:15.104149+0000","last_became_peered":"2024-04-25T10:08:15.104149+0000","last_unstale":"2024-04-25T10:08:15.110860+0000","last_undegraded":"2024-04-25T10:08:15.110860+0000","last_fullsized":"2024-04-25T10:08:15.110860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:33.683021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.119873+0000","last_change":"2024-04-25T10:08:15.115405+0000","last_active":"2024-04-25T10:08:15.119873+0000","last_peered":"2024-04-25T10:08:15.119873+0000","last_clean":"2024-04-25T10:08:15.119873+0000","last_became_active":"2024-04-25T10:08:15.115030+0000","last_became_peered":"2024-04-25T10:08:15.115030+0000","last_unstale":"2024-04-25T10:08:15.119873+0000","last_undegraded":"2024-04-25T10:08:15.119873+0000","last_fullsized":"2024-04-25T10:08:15.119873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:11.098185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.111877+0000","last_change":"2024-04-25T10:08:15.106880+0000","last_active":"2024-04-25T10:08:15.111877+0000","last_peered":"2024-04-25T10:08:15.111877+0000","last_clean":"2024-04-25T10:08:15.111877+0000","last_became_active":"2024-04-25T10:08:15.106505+0000","last_became_peered":"2024-04-25T10:08:15.106505+0000","last_unstale":"2024-04-25T10:08:15.111877+0000","last_undegraded":"2024-04-25T10:08:15.111877+0000","last_fullsized":"2024-04-25T10:08:15.111877+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:57.719111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.117506+0000","last_change":"2024-04-25T10:08:15.111345+0000","last_active":"2024-04-25T10:08:15.117506+0000","last_peered":"2024-04-25T10:08:15.117506+0000","last_clean":"2024-04-25T10:08:15.117506+0000","last_became_active":"2024-04-25T10:08:15.110841+0000","last_became_peered":"2024-04-25T10:08:15.110841+0000","last_unstale":"2024-04-25T10:08:15.117506+0000","last_undegraded":"2024-04-25T10:08:15.117506+0000","last_fullsized":"2024-04-25T10:08:15.117506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:48.317640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.120301+0000","last_change":"2024-04-25T10:08:15.114843+0000","last_active":"2024-04-25T10:08:15.120301+0000","last_peered":"2024-04-25T10:08:15.120301+0000","last_clean":"2024-04-25T10:08:15.120301+0000","last_became_active":"2024-04-25T10:08:15.114502+0000","last_became_peered":"2024-04-25T10:08:15.114502+0000","last_unstale":"2024-04-25T10:08:15.120301+0000","last_undegraded":"2024-04-25T10:08:15.120301+0000","last_fullsized":"2024-04-25T10:08:15.120301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:51.105227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.126336+0000","last_change":"2024-04-25T10:08:14.122353+0000","last_active":"2024-04-25T10:08:15.126336+0000","last_peered":"2024-04-25T10:08:15.126336+0000","last_clean":"2024-04-25T10:08:15.126336+0000","last_became_active":"2024-04-25T10:08:14.121782+0000","last_became_peered":"2024-04-25T10:08:14.121782+0000","last_unstale":"2024-04-25T10:08:15.126336+0000","last_undegraded":"2024-04-25T10:08:15.126336+0000","last_fullsized":"2024-04-25T10:08:15.126336+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:33.368825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.106932+0000","last_change":"2024-04-25T10:08:15.101949+0000","last_active":"2024-04-25T10:08:15.106932+0000","last_peered":"2024-04-25T10:08:15.106932+0000","last_clean":"2024-04-25T10:08:15.106932+0000","last_became_active":"2024-04-25T10:08:15.101574+0000","last_became_peered":"2024-04-25T10:08:15.101574+0000","last_unstale":"2024-04-25T10:08:15.106932+0000","last_undegraded":"2024-04-25T10:08:15.106932+0000","last_fullsized":"2024-04-25T10:08:15.106932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:48.538126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.127618+0000","last_change":"2024-04-25T10:08:14.114580+0000","last_active":"2024-04-25T10:08:15.127618+0000","last_peered":"2024-04-25T10:08:15.127618+0000","last_clean":"2024-04-25T10:08:15.127618+0000","last_became_active":"2024-04-25T10:08:14.113993+0000","last_became_peered":"2024-04-25T10:08:14.113993+0000","last_unstale":"2024-04-25T10:08:15.127618+0000","last_undegraded":"2024-04-25T10:08:15.127618+0000","last_fullsized":"2024-04-25T10:08:15.127618+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:47.523609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.110503+0000","last_change":"2024-04-25T10:08:15.103415+0000","last_active":"2024-04-25T10:08:15.110503+0000","last_peered":"2024-04-25T10:08:15.110503+0000","last_clean":"2024-04-25T10:08:15.110503+0000","last_became_active":"2024-04-25T10:08:15.103054+0000","last_became_peered":"2024-04-25T10:08:15.103054+0000","last_unstale":"2024-04-25T10:08:15.110503+0000","last_undegraded":"2024-04-25T10:08:15.110503+0000","last_fullsized":"2024-04-25T10:08:15.110503+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:58.893166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.139849+0000","last_change":"2024-04-25T10:08:14.105538+0000","last_active":"2024-04-25T10:08:15.139849+0000","last_peered":"2024-04-25T10:08:15.139849+0000","last_clean":"2024-04-25T10:08:15.139849+0000","last_became_active":"2024-04-25T10:08:14.103635+0000","last_became_peered":"2024-04-25T10:08:14.103635+0000","last_unstale":"2024-04-25T10:08:15.139849+0000","last_undegraded":"2024-04-25T10:08:15.139849+0000","last_fullsized":"2024-04-25T10:08:15.139849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:49.468913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.137077+0000","last_change":"2024-04-25T10:08:15.127007+0000","last_active":"2024-04-25T10:08:15.137077+0000","last_peered":"2024-04-25T10:08:15.137077+0000","last_clean":"2024-04-25T10:08:15.137077+0000","last_became_active":"2024-04-25T10:08:15.125909+0000","last_became_peered":"2024-04-25T10:08:15.125909+0000","last_unstale":"2024-04-25T10:08:15.137077+0000","last_undegraded":"2024-04-25T10:08:15.137077+0000","last_fullsized":"2024-04-25T10:08:15.137077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:00.996884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.135764+0000","last_change":"2024-04-25T10:08:14.104539+0000","last_active":"2024-04-25T10:08:15.135764+0000","last_peered":"2024-04-25T10:08:15.135764+0000","last_clean":"2024-04-25T10:08:15.135764+0000","last_became_active":"2024-04-25T10:08:14.103497+0000","last_became_peered":"2024-04-25T10:08:14.103497+0000","last_unstale":"2024-04-25T10:08:15.135764+0000","last_undegraded":"2024-04-25T10:08:15.135764+0000","last_fullsized":"2024-04-25T10:08:15.135764+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:47.170269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.122022+0000","last_change":"2024-04-25T10:08:15.115969+0000","last_active":"2024-04-25T10:08:15.122022+0000","last_peered":"2024-04-25T10:08:15.122022+0000","last_clean":"2024-04-25T10:08:15.122022+0000","last_became_active":"2024-04-25T10:08:15.115616+0000","last_became_peered":"2024-04-25T10:08:15.115616+0000","last_unstale":"2024-04-25T10:08:15.122022+0000","last_undegraded":"2024-04-25T10:08:15.122022+0000","last_fullsized":"2024-04-25T10:08:15.122022+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:04.742692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.125220+0000","last_change":"2024-04-25T10:08:14.114127+0000","last_active":"2024-04-25T10:08:15.125220+0000","last_peered":"2024-04-25T10:08:15.125220+0000","last_clean":"2024-04-25T10:08:15.125220+0000","last_became_active":"2024-04-25T10:08:14.110197+0000","last_became_peered":"2024-04-25T10:08:14.110197+0000","last_unstale":"2024-04-25T10:08:15.125220+0000","last_undegraded":"2024-04-25T10:08:15.125220+0000","last_fullsized":"2024-04-25T10:08:15.125220+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:03.932590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.118034+0000","last_change":"2024-04-25T10:08:15.113196+0000","last_active":"2024-04-25T10:08:15.118034+0000","last_peered":"2024-04-25T10:08:15.118034+0000","last_clean":"2024-04-25T10:08:15.118034+0000","last_became_active":"2024-04-25T10:08:15.112481+0000","last_became_peered":"2024-04-25T10:08:15.112481+0000","last_unstale":"2024-04-25T10:08:15.118034+0000","last_undegraded":"2024-04-25T10:08:15.118034+0000","last_fullsized":"2024-04-25T10:08:15.118034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:11.098185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.125362+0000","last_change":"2024-04-25T10:08:14.110738+0000","last_active":"2024-04-25T10:08:15.125362+0000","last_peered":"2024-04-25T10:08:15.125362+0000","last_clean":"2024-04-25T10:08:15.125362+0000","last_became_active":"2024-04-25T10:08:14.110333+0000","last_became_peered":"2024-04-25T10:08:14.110333+0000","last_unstale":"2024-04-25T10:08:15.125362+0000","last_undegraded":"2024-04-25T10:08:15.125362+0000","last_fullsized":"2024-04-25T10:08:15.125362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:14.915348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.111693+0000","last_change":"2024-04-25T10:08:15.106221+0000","last_active":"2024-04-25T10:08:15.111693+0000","last_peered":"2024-04-25T10:08:15.111693+0000","last_clean":"2024-04-25T10:08:15.111693+0000","last_became_active":"2024-04-25T10:08:15.105934+0000","last_became_peered":"2024-04-25T10:08:15.105934+0000","last_unstale":"2024-04-25T10:08:15.111693+0000","last_undegraded":"2024-04-25T10:08:15.111693+0000","last_fullsized":"2024-04-25T10:08:15.111693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:51.105227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.077671+0000","last_change":"2024-04-25T10:08:14.104359+0000","last_active":"2024-04-25T10:08:15.077671+0000","last_peered":"2024-04-25T10:08:15.077671+0000","last_clean":"2024-04-25T10:08:15.077671+0000","last_became_active":"2024-04-25T10:08:14.103990+0000","last_became_peered":"2024-04-25T10:08:14.103990+0000","last_unstale":"2024-04-25T10:08:15.077671+0000","last_undegraded":"2024-04-25T10:08:15.077671+0000","last_fullsized":"2024-04-25T10:08:15.077671+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:32.668504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.129554+0000","last_change":"2024-04-25T10:08:15.122103+0000","last_active":"2024-04-25T10:08:15.129554+0000","last_peered":"2024-04-25T10:08:15.129554+0000","last_clean":"2024-04-25T10:08:15.129554+0000","last_became_active":"2024-04-25T10:08:15.121481+0000","last_became_peered":"2024-04-25T10:08:15.121481+0000","last_unstale":"2024-04-25T10:08:15.129554+0000","last_undegraded":"2024-04-25T10:08:15.129554+0000","last_fullsized":"2024-04-25T10:08:15.129554+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:19.536090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.083406+0000","last_change":"2024-04-25T10:08:14.116504+0000","last_active":"2024-04-25T10:08:15.083406+0000","last_peered":"2024-04-25T10:08:15.083406+0000","last_clean":"2024-04-25T10:08:15.083406+0000","last_became_active":"2024-04-25T10:08:14.116208+0000","last_became_peered":"2024-04-25T10:08:14.116208+0000","last_unstale":"2024-04-25T10:08:15.083406+0000","last_undegraded":"2024-04-25T10:08:15.083406+0000","last_fullsized":"2024-04-25T10:08:15.083406+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:13.684148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.112187+0000","last_change":"2024-04-25T10:08:15.100447+0000","last_active":"2024-04-25T10:08:15.112187+0000","last_peered":"2024-04-25T10:08:15.112187+0000","last_clean":"2024-04-25T10:08:15.112187+0000","last_became_active":"2024-04-25T10:08:15.100142+0000","last_became_peered":"2024-04-25T10:08:15.100142+0000","last_unstale":"2024-04-25T10:08:15.112187+0000","last_undegraded":"2024-04-25T10:08:15.112187+0000","last_fullsized":"2024-04-25T10:08:15.112187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:00.996884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.079865+0000","last_change":"2024-04-25T10:08:14.113674+0000","last_active":"2024-04-25T10:08:15.079865+0000","last_peered":"2024-04-25T10:08:15.079865+0000","last_clean":"2024-04-25T10:08:15.079865+0000","last_became_active":"2024-04-25T10:08:14.113375+0000","last_became_peered":"2024-04-25T10:08:14.113375+0000","last_unstale":"2024-04-25T10:08:15.079865+0000","last_undegraded":"2024-04-25T10:08:15.079865+0000","last_fullsized":"2024-04-25T10:08:15.079865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:14.920112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.133651+0000","last_change":"2024-04-25T10:08:15.129477+0000","last_active":"2024-04-25T10:08:15.133651+0000","last_peered":"2024-04-25T10:08:15.133651+0000","last_clean":"2024-04-25T10:08:15.133651+0000","last_became_active":"2024-04-25T10:08:15.129162+0000","last_became_peered":"2024-04-25T10:08:15.129162+0000","last_unstale":"2024-04-25T10:08:15.133651+0000","last_undegraded":"2024-04-25T10:08:15.133651+0000","last_fullsized":"2024-04-25T10:08:15.133651+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:18.282295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.098002+0000","last_change":"2024-04-25T10:08:14.107555+0000","last_active":"2024-04-25T10:08:15.098002+0000","last_peered":"2024-04-25T10:08:15.098002+0000","last_clean":"2024-04-25T10:08:15.098002+0000","last_became_active":"2024-04-25T10:08:14.106517+0000","last_became_peered":"2024-04-25T10:08:14.106517+0000","last_unstale":"2024-04-25T10:08:15.098002+0000","last_undegraded":"2024-04-25T10:08:15.098002+0000","last_fullsized":"2024-04-25T10:08:15.098002+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:56.704594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.118840+0000","last_change":"2024-04-25T10:08:15.113349+0000","last_active":"2024-04-25T10:08:15.118840+0000","last_peered":"2024-04-25T10:08:15.118840+0000","last_clean":"2024-04-25T10:08:15.118840+0000","last_became_active":"2024-04-25T10:08:15.112970+0000","last_became_peered":"2024-04-25T10:08:15.112970+0000","last_unstale":"2024-04-25T10:08:15.118840+0000","last_undegraded":"2024-04-25T10:08:15.118840+0000","last_fullsized":"2024-04-25T10:08:15.118840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:35.553951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.138052+0000","last_change":"2024-04-25T10:08:14.113960+0000","last_active":"2024-04-25T10:08:15.138052+0000","last_peered":"2024-04-25T10:08:15.138052+0000","last_clean":"2024-04-25T10:08:15.138052+0000","last_became_active":"2024-04-25T10:08:14.113642+0000","last_became_peered":"2024-04-25T10:08:14.113642+0000","last_unstale":"2024-04-25T10:08:15.138052+0000","last_undegraded":"2024-04-25T10:08:15.138052+0000","last_fullsized":"2024-04-25T10:08:15.138052+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:18.521573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.137769+0000","last_change":"2024-04-25T10:08:15.131535+0000","last_active":"2024-04-25T10:08:15.137769+0000","last_peered":"2024-04-25T10:08:15.137769+0000","last_clean":"2024-04-25T10:08:15.137769+0000","last_became_active":"2024-04-25T10:08:15.131252+0000","last_became_peered":"2024-04-25T10:08:15.131252+0000","last_unstale":"2024-04-25T10:08:15.137769+0000","last_undegraded":"2024-04-25T10:08:15.137769+0000","last_fullsized":"2024-04-25T10:08:15.137769+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:35.553951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.143108+0000","last_change":"2024-04-25T10:08:14.103534+0000","last_active":"2024-04-25T10:08:15.143108+0000","last_peered":"2024-04-25T10:08:15.143108+0000","last_clean":"2024-04-25T10:08:15.143108+0000","last_became_active":"2024-04-25T10:08:14.103098+0000","last_became_peered":"2024-04-25T10:08:14.103098+0000","last_unstale":"2024-04-25T10:08:15.143108+0000","last_undegraded":"2024-04-25T10:08:15.143108+0000","last_fullsized":"2024-04-25T10:08:15.143108+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:50.090710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.129668+0000","last_change":"2024-04-25T10:08:15.123389+0000","last_active":"2024-04-25T10:08:15.129668+0000","last_peered":"2024-04-25T10:08:15.129668+0000","last_clean":"2024-04-25T10:08:15.129668+0000","last_became_active":"2024-04-25T10:08:15.120955+0000","last_became_peered":"2024-04-25T10:08:15.120955+0000","last_unstale":"2024-04-25T10:08:15.129668+0000","last_undegraded":"2024-04-25T10:08:15.129668+0000","last_fullsized":"2024-04-25T10:08:15.129668+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.483430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.126581+0000","last_change":"2024-04-25T10:08:14.119402+0000","last_active":"2024-04-25T10:08:15.126581+0000","last_peered":"2024-04-25T10:08:15.126581+0000","last_clean":"2024-04-25T10:08:15.126581+0000","last_became_active":"2024-04-25T10:08:14.119057+0000","last_became_peered":"2024-04-25T10:08:14.119057+0000","last_unstale":"2024-04-25T10:08:15.126581+0000","last_undegraded":"2024-04-25T10:08:15.126581+0000","last_fullsized":"2024-04-25T10:08:15.126581+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:33.368825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.108811+0000","last_change":"2024-04-25T10:08:15.105485+0000","last_active":"2024-04-25T10:08:15.108811+0000","last_peered":"2024-04-25T10:08:15.108811+0000","last_clean":"2024-04-25T10:08:15.108811+0000","last_became_active":"2024-04-25T10:08:15.104908+0000","last_became_peered":"2024-04-25T10:08:15.104908+0000","last_unstale":"2024-04-25T10:08:15.108811+0000","last_undegraded":"2024-04-25T10:08:15.108811+0000","last_fullsized":"2024-04-25T10:08:15.108811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:11.098185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.115666+0000","last_change":"2024-04-25T10:08:14.116721+0000","last_active":"2024-04-25T10:08:15.115666+0000","last_peered":"2024-04-25T10:08:15.115666+0000","last_clean":"2024-04-25T10:08:15.115666+0000","last_became_active":"2024-04-25T10:08:14.116415+0000","last_became_peered":"2024-04-25T10:08:14.116415+0000","last_unstale":"2024-04-25T10:08:15.115666+0000","last_undegraded":"2024-04-25T10:08:15.115666+0000","last_fullsized":"2024-04-25T10:08:15.115666+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:56.704594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.117481+0000","last_change":"2024-04-25T10:08:15.111293+0000","last_active":"2024-04-25T10:08:15.117481+0000","last_peered":"2024-04-25T10:08:15.117481+0000","last_clean":"2024-04-25T10:08:15.117481+0000","last_became_active":"2024-04-25T10:08:15.110962+0000","last_became_peered":"2024-04-25T10:08:15.110962+0000","last_unstale":"2024-04-25T10:08:15.117481+0000","last_undegraded":"2024-04-25T10:08:15.117481+0000","last_fullsized":"2024-04-25T10:08:15.117481+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.483430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.078110+0000","last_change":"2024-04-25T10:08:14.118362+0000","last_active":"2024-04-25T10:08:15.078110+0000","last_peered":"2024-04-25T10:08:15.078110+0000","last_clean":"2024-04-25T10:08:15.078110+0000","last_became_active":"2024-04-25T10:08:14.118153+0000","last_became_peered":"2024-04-25T10:08:14.118153+0000","last_unstale":"2024-04-25T10:08:15.078110+0000","last_undegraded":"2024-04-25T10:08:15.078110+0000","last_fullsized":"2024-04-25T10:08:15.078110+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:18.521573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.111902+0000","last_change":"2024-04-25T10:08:15.107802+0000","last_active":"2024-04-25T10:08:15.111902+0000","last_peered":"2024-04-25T10:08:15.111902+0000","last_clean":"2024-04-25T10:08:15.111902+0000","last_became_active":"2024-04-25T10:08:15.107591+0000","last_became_peered":"2024-04-25T10:08:15.107591+0000","last_unstale":"2024-04-25T10:08:15.111902+0000","last_undegraded":"2024-04-25T10:08:15.111902+0000","last_fullsized":"2024-04-25T10:08:15.111902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:58.893166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.142483+0000","last_change":"2024-04-25T10:08:14.104238+0000","last_active":"2024-04-25T10:08:15.142483+0000","last_peered":"2024-04-25T10:08:15.142483+0000","last_clean":"2024-04-25T10:08:15.142483+0000","last_became_active":"2024-04-25T10:08:14.103941+0000","last_became_peered":"2024-04-25T10:08:14.103941+0000","last_unstale":"2024-04-25T10:08:15.142483+0000","last_undegraded":"2024-04-25T10:08:15.142483+0000","last_fullsized":"2024-04-25T10:08:15.142483+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:47.303123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.098064+0000","last_change":"2024-04-25T10:08:15.095804+0000","last_active":"2024-04-25T10:08:15.098064+0000","last_peered":"2024-04-25T10:08:15.098064+0000","last_clean":"2024-04-25T10:08:15.098064+0000","last_became_active":"2024-04-25T10:08:15.095242+0000","last_became_peered":"2024-04-25T10:08:15.095242+0000","last_unstale":"2024-04-25T10:08:15.098064+0000","last_undegraded":"2024-04-25T10:08:15.098064+0000","last_fullsized":"2024-04-25T10:08:15.098064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:57.719111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.132780+0000","last_change":"2024-04-25T10:08:14.107761+0000","last_active":"2024-04-25T10:08:15.132780+0000","last_peered":"2024-04-25T10:08:15.132780+0000","last_clean":"2024-04-25T10:08:15.132780+0000","last_became_active":"2024-04-25T10:08:14.106877+0000","last_became_peered":"2024-04-25T10:08:14.106877+0000","last_unstale":"2024-04-25T10:08:15.132780+0000","last_undegraded":"2024-04-25T10:08:15.132780+0000","last_fullsized":"2024-04-25T10:08:15.132780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:35.499379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.107664+0000","last_change":"2024-04-25T10:08:15.095635+0000","last_active":"2024-04-25T10:08:15.107664+0000","last_peered":"2024-04-25T10:08:15.107664+0000","last_clean":"2024-04-25T10:08:15.107664+0000","last_became_active":"2024-04-25T10:08:15.095285+0000","last_became_peered":"2024-04-25T10:08:15.095285+0000","last_unstale":"2024-04-25T10:08:15.107664+0000","last_undegraded":"2024-04-25T10:08:15.107664+0000","last_fullsized":"2024-04-25T10:08:15.107664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.483430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.126852+0000","last_change":"2024-04-25T10:08:14.116454+0000","last_active":"2024-04-25T10:08:15.126852+0000","last_peered":"2024-04-25T10:08:15.126852+0000","last_clean":"2024-04-25T10:08:15.126852+0000","last_became_active":"2024-04-25T10:08:14.115808+0000","last_became_peered":"2024-04-25T10:08:14.115808+0000","last_unstale":"2024-04-25T10:08:15.126852+0000","last_undegraded":"2024-04-25T10:08:15.126852+0000","last_fullsized":"2024-04-25T10:08:15.126852+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:39.203427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.135146+0000","last_change":"2024-04-25T10:08:15.123291+0000","last_active":"2024-04-25T10:08:15.135146+0000","last_peered":"2024-04-25T10:08:15.135146+0000","last_clean":"2024-04-25T10:08:15.135146+0000","last_became_active":"2024-04-25T10:08:15.121418+0000","last_became_peered":"2024-04-25T10:08:15.121418+0000","last_unstale":"2024-04-25T10:08:15.135146+0000","last_undegraded":"2024-04-25T10:08:15.135146+0000","last_fullsized":"2024-04-25T10:08:15.135146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:51.105227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.126598+0000","last_change":"2024-04-25T10:08:14.108022+0000","last_active":"2024-04-25T10:08:15.126598+0000","last_peered":"2024-04-25T10:08:15.126598+0000","last_clean":"2024-04-25T10:08:15.126598+0000","last_became_active":"2024-04-25T10:08:14.107708+0000","last_became_peered":"2024-04-25T10:08:14.107708+0000","last_unstale":"2024-04-25T10:08:15.126598+0000","last_undegraded":"2024-04-25T10:08:15.126598+0000","last_fullsized":"2024-04-25T10:08:15.126598+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:42.434267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.120269+0000","last_change":"2024-04-25T10:08:15.116799+0000","last_active":"2024-04-25T10:08:15.120269+0000","last_peered":"2024-04-25T10:08:15.120269+0000","last_clean":"2024-04-25T10:08:15.120269+0000","last_became_active":"2024-04-25T10:08:15.116533+0000","last_became_peered":"2024-04-25T10:08:15.116533+0000","last_unstale":"2024-04-25T10:08:15.120269+0000","last_undegraded":"2024-04-25T10:08:15.120269+0000","last_fullsized":"2024-04-25T10:08:15.120269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:40.217944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.087362+0000","last_change":"2024-04-25T10:08:14.111048+0000","last_active":"2024-04-25T10:08:15.087362+0000","last_peered":"2024-04-25T10:08:15.087362+0000","last_clean":"2024-04-25T10:08:15.087362+0000","last_became_active":"2024-04-25T10:08:14.110739+0000","last_became_peered":"2024-04-25T10:08:14.110739+0000","last_unstale":"2024-04-25T10:08:15.087362+0000","last_undegraded":"2024-04-25T10:08:15.087362+0000","last_fullsized":"2024-04-25T10:08:15.087362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:47.523609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.120317+0000","last_change":"2024-04-25T10:08:15.118046+0000","last_active":"2024-04-25T10:08:15.120317+0000","last_peered":"2024-04-25T10:08:15.120317+0000","last_clean":"2024-04-25T10:08:15.120317+0000","last_became_active":"2024-04-25T10:08:15.115942+0000","last_became_peered":"2024-04-25T10:08:15.115942+0000","last_unstale":"2024-04-25T10:08:15.120317+0000","last_undegraded":"2024-04-25T10:08:15.120317+0000","last_fullsized":"2024-04-25T10:08:15.120317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:15.929865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.134748+0000","last_change":"2024-04-25T10:08:14.103980+0000","last_active":"2024-04-25T10:08:15.134748+0000","last_peered":"2024-04-25T10:08:15.134748+0000","last_clean":"2024-04-25T10:08:15.134748+0000","last_became_active":"2024-04-25T10:08:14.102938+0000","last_became_peered":"2024-04-25T10:08:14.102938+0000","last_unstale":"2024-04-25T10:08:15.134748+0000","last_undegraded":"2024-04-25T10:08:15.134748+0000","last_fullsized":"2024-04-25T10:08:15.134748+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:35.499379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.115987+0000","last_change":"2024-04-25T10:08:15.113005+0000","last_active":"2024-04-25T10:08:15.115987+0000","last_peered":"2024-04-25T10:08:15.115987+0000","last_clean":"2024-04-25T10:08:15.115987+0000","last_became_active":"2024-04-25T10:08:15.112210+0000","last_became_peered":"2024-04-25T10:08:15.112210+0000","last_unstale":"2024-04-25T10:08:15.115987+0000","last_undegraded":"2024-04-25T10:08:15.115987+0000","last_fullsized":"2024-04-25T10:08:15.115987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:56.791167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.091724+0000","last_change":"2024-04-25T10:08:14.121667+0000","last_active":"2024-04-25T10:08:15.091724+0000","last_peered":"2024-04-25T10:08:15.091724+0000","last_clean":"2024-04-25T10:08:15.091724+0000","last_became_active":"2024-04-25T10:08:14.121369+0000","last_became_peered":"2024-04-25T10:08:14.121369+0000","last_unstale":"2024-04-25T10:08:15.091724+0000","last_undegraded":"2024-04-25T10:08:15.091724+0000","last_fullsized":"2024-04-25T10:08:15.091724+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:59.337836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.106064+0000","last_change":"2024-04-25T10:08:15.102655+0000","last_active":"2024-04-25T10:08:15.106064+0000","last_peered":"2024-04-25T10:08:15.106064+0000","last_clean":"2024-04-25T10:08:15.106064+0000","last_became_active":"2024-04-25T10:08:15.102313+0000","last_became_peered":"2024-04-25T10:08:15.102313+0000","last_unstale":"2024-04-25T10:08:15.106064+0000","last_undegraded":"2024-04-25T10:08:15.106064+0000","last_fullsized":"2024-04-25T10:08:15.106064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:51.196612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.083645+0000","last_change":"2024-04-25T10:08:14.116705+0000","last_active":"2024-04-25T10:08:15.083645+0000","last_peered":"2024-04-25T10:08:15.083645+0000","last_clean":"2024-04-25T10:08:15.083645+0000","last_became_active":"2024-04-25T10:08:14.116246+0000","last_became_peered":"2024-04-25T10:08:14.116246+0000","last_unstale":"2024-04-25T10:08:15.083645+0000","last_undegraded":"2024-04-25T10:08:15.083645+0000","last_fullsized":"2024-04-25T10:08:15.083645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:38.540741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.119459+0000","last_change":"2024-04-25T10:08:15.113280+0000","last_active":"2024-04-25T10:08:15.119459+0000","last_peered":"2024-04-25T10:08:15.119459+0000","last_clean":"2024-04-25T10:08:15.119459+0000","last_became_active":"2024-04-25T10:08:15.112707+0000","last_became_peered":"2024-04-25T10:08:15.112707+0000","last_unstale":"2024-04-25T10:08:15.119459+0000","last_undegraded":"2024-04-25T10:08:15.119459+0000","last_fullsized":"2024-04-25T10:08:15.119459+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:15.934629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.086800+0000","last_change":"2024-04-25T10:08:14.115096+0000","last_active":"2024-04-25T10:08:15.086800+0000","last_peered":"2024-04-25T10:08:15.086800+0000","last_clean":"2024-04-25T10:08:15.086800+0000","last_became_active":"2024-04-25T10:08:14.114760+0000","last_became_peered":"2024-04-25T10:08:14.114760+0000","last_unstale":"2024-04-25T10:08:15.086800+0000","last_undegraded":"2024-04-25T10:08:15.086800+0000","last_fullsized":"2024-04-25T10:08:15.086800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:50.090710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.117361+0000","last_change":"2024-04-25T10:08:15.113919+0000","last_active":"2024-04-25T10:08:15.117361+0000","last_peered":"2024-04-25T10:08:15.117361+0000","last_clean":"2024-04-25T10:08:15.117361+0000","last_became_active":"2024-04-25T10:08:15.113594+0000","last_became_peered":"2024-04-25T10:08:15.113594+0000","last_unstale":"2024-04-25T10:08:15.117361+0000","last_undegraded":"2024-04-25T10:08:15.117361+0000","last_fullsized":"2024-04-25T10:08:15.117361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:22.586940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.132880+0000","last_change":"2024-04-25T10:08:14.107878+0000","last_active":"2024-04-25T10:08:15.132880+0000","last_peered":"2024-04-25T10:08:15.132880+0000","last_clean":"2024-04-25T10:08:15.132880+0000","last_became_active":"2024-04-25T10:08:14.107443+0000","last_became_peered":"2024-04-25T10:08:14.107443+0000","last_unstale":"2024-04-25T10:08:15.132880+0000","last_undegraded":"2024-04-25T10:08:15.132880+0000","last_fullsized":"2024-04-25T10:08:15.132880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:29.693794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.132969+0000","last_change":"2024-04-25T10:08:15.128961+0000","last_active":"2024-04-25T10:08:15.132969+0000","last_peered":"2024-04-25T10:08:15.132969+0000","last_clean":"2024-04-25T10:08:15.132969+0000","last_became_active":"2024-04-25T10:08:15.128642+0000","last_became_peered":"2024-04-25T10:08:15.128642+0000","last_unstale":"2024-04-25T10:08:15.132969+0000","last_undegraded":"2024-04-25T10:08:15.132969+0000","last_fullsized":"2024-04-25T10:08:15.132969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:04.742692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.098220+0000","last_change":"2024-04-25T10:08:14.113405+0000","last_active":"2024-04-25T10:08:15.098220+0000","last_peered":"2024-04-25T10:08:15.098220+0000","last_clean":"2024-04-25T10:08:15.098220+0000","last_became_active":"2024-04-25T10:08:14.112916+0000","last_became_peered":"2024-04-25T10:08:14.112916+0000","last_unstale":"2024-04-25T10:08:15.098220+0000","last_undegraded":"2024-04-25T10:08:15.098220+0000","last_fullsized":"2024-04-25T10:08:15.098220+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:57.878649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.133883+0000","last_change":"2024-04-25T10:08:15.132178+0000","last_active":"2024-04-25T10:08:15.133883+0000","last_peered":"2024-04-25T10:08:15.133883+0000","last_clean":"2024-04-25T10:08:15.133883+0000","last_became_active":"2024-04-25T10:08:15.131428+0000","last_became_peered":"2024-04-25T10:08:15.131428+0000","last_unstale":"2024-04-25T10:08:15.133883+0000","last_undegraded":"2024-04-25T10:08:15.133883+0000","last_fullsized":"2024-04-25T10:08:15.133883+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:50.483430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.119183+0000","last_change":"2024-04-25T10:08:14.108366+0000","last_active":"2024-04-25T10:08:15.119183+0000","last_peered":"2024-04-25T10:08:15.119183+0000","last_clean":"2024-04-25T10:08:15.119183+0000","last_became_active":"2024-04-25T10:08:14.103224+0000","last_became_peered":"2024-04-25T10:08:14.103224+0000","last_unstale":"2024-04-25T10:08:15.119183+0000","last_undegraded":"2024-04-25T10:08:15.119183+0000","last_fullsized":"2024-04-25T10:08:15.119183+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:32.668504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.112474+0000","last_change":"2024-04-25T10:08:15.110587+0000","last_active":"2024-04-25T10:08:15.112474+0000","last_peered":"2024-04-25T10:08:15.112474+0000","last_clean":"2024-04-25T10:08:15.112474+0000","last_became_active":"2024-04-25T10:08:15.110013+0000","last_became_peered":"2024-04-25T10:08:15.110013+0000","last_unstale":"2024-04-25T10:08:15.112474+0000","last_undegraded":"2024-04-25T10:08:15.112474+0000","last_fullsized":"2024-04-25T10:08:15.112474+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:36.513896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.098694+0000","last_change":"2024-04-25T10:08:14.104838+0000","last_active":"2024-04-25T10:08:15.098694+0000","last_peered":"2024-04-25T10:08:15.098694+0000","last_clean":"2024-04-25T10:08:15.098694+0000","last_became_active":"2024-04-25T10:08:14.104537+0000","last_became_peered":"2024-04-25T10:08:14.104537+0000","last_unstale":"2024-04-25T10:08:15.098694+0000","last_undegraded":"2024-04-25T10:08:15.098694+0000","last_fullsized":"2024-04-25T10:08:15.098694+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:29.693794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.111454+0000","last_change":"2024-04-25T10:08:15.103258+0000","last_active":"2024-04-25T10:08:15.111454+0000","last_peered":"2024-04-25T10:08:15.111454+0000","last_clean":"2024-04-25T10:08:15.111454+0000","last_became_active":"2024-04-25T10:08:15.102838+0000","last_became_peered":"2024-04-25T10:08:15.102838+0000","last_unstale":"2024-04-25T10:08:15.111454+0000","last_undegraded":"2024-04-25T10:08:15.111454+0000","last_fullsized":"2024-04-25T10:08:15.111454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:33.683021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.115800+0000","last_change":"2024-04-25T10:08:14.107630+0000","last_active":"2024-04-25T10:08:15.115800+0000","last_peered":"2024-04-25T10:08:15.115800+0000","last_clean":"2024-04-25T10:08:15.115800+0000","last_became_active":"2024-04-25T10:08:14.107342+0000","last_became_peered":"2024-04-25T10:08:14.107342+0000","last_unstale":"2024-04-25T10:08:15.115800+0000","last_undegraded":"2024-04-25T10:08:15.115800+0000","last_fullsized":"2024-04-25T10:08:15.115800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:57.878649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.121009+0000","last_change":"2024-04-25T10:08:15.120102+0000","last_active":"2024-04-25T10:08:15.121009+0000","last_peered":"2024-04-25T10:08:15.121009+0000","last_clean":"2024-04-25T10:08:15.121009+0000","last_became_active":"2024-04-25T10:08:15.119752+0000","last_became_peered":"2024-04-25T10:08:15.119752+0000","last_unstale":"2024-04-25T10:08:15.121009+0000","last_undegraded":"2024-04-25T10:08:15.121009+0000","last_fullsized":"2024-04-25T10:08:15.121009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:41.876481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.083680+0000","last_change":"2024-04-25T10:08:14.113420+0000","last_active":"2024-04-25T10:08:15.083680+0000","last_peered":"2024-04-25T10:08:15.083680+0000","last_clean":"2024-04-25T10:08:15.083680+0000","last_became_active":"2024-04-25T10:08:14.113110+0000","last_became_peered":"2024-04-25T10:08:14.113110+0000","last_unstale":"2024-04-25T10:08:15.083680+0000","last_undegraded":"2024-04-25T10:08:15.083680+0000","last_fullsized":"2024-04-25T10:08:15.083680+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:59.337836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.110923+0000","last_change":"2024-04-25T10:08:15.103423+0000","last_active":"2024-04-25T10:08:15.110923+0000","last_peered":"2024-04-25T10:08:15.110923+0000","last_clean":"2024-04-25T10:08:15.110923+0000","last_became_active":"2024-04-25T10:08:15.102956+0000","last_became_peered":"2024-04-25T10:08:15.102956+0000","last_unstale":"2024-04-25T10:08:15.110923+0000","last_undegraded":"2024-04-25T10:08:15.110923+0000","last_fullsized":"2024-04-25T10:08:15.110923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:19.536090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.078970+0000","last_change":"2024-04-25T10:08:14.115663+0000","last_active":"2024-04-25T10:08:15.078970+0000","last_peered":"2024-04-25T10:08:15.078970+0000","last_clean":"2024-04-25T10:08:15.078970+0000","last_became_active":"2024-04-25T10:08:14.115375+0000","last_became_peered":"2024-04-25T10:08:14.115375+0000","last_unstale":"2024-04-25T10:08:15.078970+0000","last_undegraded":"2024-04-25T10:08:15.078970+0000","last_fullsized":"2024-04-25T10:08:15.078970+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:59.982367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.118725+0000","last_change":"2024-04-25T10:08:15.112405+0000","last_active":"2024-04-25T10:08:15.118725+0000","last_peered":"2024-04-25T10:08:15.118725+0000","last_clean":"2024-04-25T10:08:15.118725+0000","last_became_active":"2024-04-25T10:08:15.112083+0000","last_became_peered":"2024-04-25T10:08:15.112083+0000","last_unstale":"2024-04-25T10:08:15.118725+0000","last_undegraded":"2024-04-25T10:08:15.118725+0000","last_fullsized":"2024-04-25T10:08:15.118725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:14.062803+0000","last_clean_scrub_stamp":"2024-04-25T10:08:14.062803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:00.996884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.092037+0000","last_change":"2024-04-25T10:08:14.124287+0000","last_active":"2024-04-25T10:08:15.092037+0000","last_peered":"2024-04-25T10:08:15.092037+0000","last_clean":"2024-04-25T10:08:15.092037+0000","last_became_active":"2024-04-25T10:08:14.123910+0000","last_became_peered":"2024-04-25T10:08:14.123910+0000","last_unstale":"2024-04-25T10:08:15.092037+0000","last_undegraded":"2024-04-25T10:08:15.092037+0000","last_fullsized":"2024-04-25T10:08:15.092037+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:13.048286+0000","last_clean_scrub_stamp":"2024-04-25T10:08:13.048286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:03.932590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T10:08:15.098539+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:08:15.098539+0000","last_peered":"2024-04-25T10:08:15.098539+0000","last_clean":"2024-04-25T10:08:15.098539+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:08:15.098539+0000","last_undegraded":"2024-04-25T10:08:15.098539+0000","last_fullsized":"2024-04-25T10:08:15.098539+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":958464,"data_stored":730636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":140794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":140850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":348160,"data_stored":133403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":141812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":140248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":141812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":954368,"data_stored":730546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:19.980 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:08:19.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:08:20.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.221+0000 7f1abb200640 1 Processor -- start 2024-04-25T10:08:20.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.221+0000 7f1abb200640 1 -- start start 2024-04-25T10:08:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.223+0000 7f1abb200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1ab40b6ab0 0x7f1ab40d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.223+0000 7f1abb200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab40d09b0 0x7f1ab40d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.223+0000 7f1abb200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab40b2e60 con 0x7f1ab415daf0 2024-04-25T10:08:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.223+0000 7f1abb200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab40d0e40 con 0x7f1ab40b6ab0 2024-04-25T10:08:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.223+0000 7f1abb200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab40d17b0 con 0x7f1ab40d09b0 2024-04-25T10:08:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.224+0000 7f1ab9400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1ab40b6ab0 0x7f1ab40d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.224+0000 7f1aba800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab40d09b0 0x7f1ab40d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.224+0000 7f1aba800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab40d09b0 0x7f1ab40d1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:39304/0 (socket says 172.21.15.119:39304) 2024-04-25T10:08:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.224+0000 7f1ab9400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1ab40b6ab0 0x7f1ab40d0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:51842/0 (socket says 172.21.15.119:51842) 2024-04-25T10:08:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.224+0000 7f1aba800640 1 -- 172.21.15.119:0/3103969327 learned_addr learned my addr 172.21.15.119:0/3103969327 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:20.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.225+0000 7f1aba800640 1 -- 172.21.15.119:0/3103969327 >> v2:172.21.15.132:3300/0 conn(0x7f1ab40b6ab0 msgr2=0x7f1ab40d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.225+0000 7f1aba800640 1 --2- 172.21.15.119:0/3103969327 >> v2:172.21.15.132:3300/0 conn(0x7f1ab40b6ab0 0x7f1ab40d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.225+0000 7f1aba800640 1 -- 172.21.15.119:0/3103969327 >> v1:172.21.15.132:6789/0 conn(0x7f1ab415daf0 legacy=0x7f1ab40b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.225+0000 7f1aba800640 1 -- 172.21.15.119:0/3103969327 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab40ce760 con 0x7f1ab40d09b0 2024-04-25T10:08:20.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.226+0000 7f1aba800640 1 --2- 172.21.15.119:0/3103969327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab40d09b0 0x7f1ab40d1c40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009e50 tx=0x7f1ab002a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a73207fcee9a441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.226+0000 7f1ab8a00640 1 -- 172.21.15.119:0/3103969327 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0036070 con 0x7f1ab40d09b0 2024-04-25T10:08:20.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.226+0000 7f1ab8a00640 1 -- 172.21.15.119:0/3103969327 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab0031600 con 0x7f1ab40d09b0 2024-04-25T10:08:20.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.226+0000 7f1ab8a00640 1 -- 172.21.15.119:0/3103969327 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00319d0 con 0x7f1ab40d09b0 2024-04-25T10:08:20.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.227+0000 7f1abb200640 1 -- 172.21.15.119:0/3103969327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab40d09b0 msgr2=0x7f1ab40d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.227+0000 7f1abb200640 1 --2- 172.21.15.119:0/3103969327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab40d09b0 0x7f1ab40d1c40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009e50 tx=0x7f1ab002a7a0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.227+0000 7f1abb200640 1 -- 172.21.15.119:0/3103969327 shutdown_connections 2024-04-25T10:08:20.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.227+0000 7f1abb200640 1 --2- 172.21.15.119:0/3103969327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab40d09b0 0x7f1ab40d1c40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.227+0000 7f1abb200640 1 --2- 172.21.15.119:0/3103969327 >> v2:172.21.15.132:3300/0 conn(0x7f1ab40b6ab0 0x7f1ab40d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.227+0000 7f1abb200640 1 -- 172.21.15.119:0/3103969327 >> 172.21.15.119:0/3103969327 conn(0x7f1ab40b55c0 msgr2=0x7f1ab40b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.227+0000 7f1abb200640 1 -- 172.21.15.119:0/3103969327 shutdown_connections 2024-04-25T10:08:20.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.227+0000 7f1abb200640 1 -- 172.21.15.119:0/3103969327 wait complete. 2024-04-25T10:08:20.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.229+0000 7f1abb200640 1 Processor -- start 2024-04-25T10:08:20.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.230+0000 7f1abb200640 1 -- start start 2024-04-25T10:08:20.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.231+0000 7f1abb200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab40b6ab0 0x7f1ab415e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.231+0000 7f1abb200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab40d09b0 0x7f1ab40cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.231+0000 7f1abb200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ab415daf0 0x7f1ab40b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.231+0000 7f1abb200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab415ded0 con 0x7f1ab40d09b0 2024-04-25T10:08:20.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.231+0000 7f1abb200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab40ceda0 con 0x7f1ab415daf0 2024-04-25T10:08:20.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.231+0000 7f1abb200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab40cf000 con 0x7f1ab40b6ab0 2024-04-25T10:08:20.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.231+0000 7f1ab9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab40d09b0 0x7f1ab40cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.231+0000 7f1ab9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab40d09b0 0x7f1ab40cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34430/0 (socket says 172.21.15.119:34430) 2024-04-25T10:08:20.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.231+0000 7f1ab9400640 1 -- 172.21.15.119:0/3984188259 learned_addr learned my addr 172.21.15.119:0/3984188259 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:20.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.232+0000 7f1ab9e00640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab40b6ab0 0x7f1ab415e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.232+0000 7f1aba800640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ab415daf0 0x7f1ab40b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.232+0000 7f1ab9400640 1 -- 172.21.15.119:0/3984188259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab40b6ab0 msgr2=0x7f1ab415e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.232+0000 7f1ab9400640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab40b6ab0 0x7f1ab415e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.232+0000 7f1ab9400640 1 -- 172.21.15.119:0/3984188259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ab415daf0 msgr2=0x7f1ab40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.232+0000 7f1ab9400640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ab415daf0 0x7f1ab40b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.232+0000 7f1ab9400640 1 -- 172.21.15.119:0/3984188259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab41554f0 con 0x7f1ab40d09b0 2024-04-25T10:08:20.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.232+0000 7f1ab9e00640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab40b6ab0 0x7f1ab415e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:20.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.232+0000 7f1ab9400640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab40d09b0 0x7f1ab40cb560 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400e870 tx=0x7f1aa400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.233+0000 7f1aae000640 1 -- 172.21.15.119:0/3984188259 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4017070 con 0x7f1ab40d09b0 2024-04-25T10:08:20.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.233+0000 7f1aae000640 1 -- 172.21.15.119:0/3984188259 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa400c940 con 0x7f1ab40d09b0 2024-04-25T10:08:20.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.233+0000 7f1aae000640 1 -- 172.21.15.119:0/3984188259 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa400cd30 con 0x7f1ab40d09b0 2024-04-25T10:08:20.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.233+0000 7f1abb200640 1 -- 172.21.15.119:0/3984188259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab415e7c0 con 0x7f1ab40d09b0 2024-04-25T10:08:20.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.233+0000 7f1abb200640 1 -- 172.21.15.119:0/3984188259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab415ed80 con 0x7f1ab40d09b0 2024-04-25T10:08:20.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.234+0000 7f1abb200640 1 -- 172.21.15.119:0/3984188259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab40749b0 con 0x7f1ab40d09b0 2024-04-25T10:08:20.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.235+0000 7f1aae000640 1 -- 172.21.15.119:0/3984188259 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1aa401a070 con 0x7f1ab40d09b0 2024-04-25T10:08:20.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.236+0000 7f1aae000640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a9806b4d0 0x7f1a9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.236+0000 7f1aae000640 1 -- 172.21.15.119:0/3984188259 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1aa408a520 con 0x7f1ab40d09b0 2024-04-25T10:08:20.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.236+0000 7f1ab9e00640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a9806b4d0 0x7f1a9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.237+0000 7f1ab9e00640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a9806b4d0 0x7f1a9806d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ab415d500 tx=0x7f1aa8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.240+0000 7f1aae000640 1 -- 172.21.15.119:0/3984188259 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ab41554f0 con 0x7f1ab40d09b0 2024-04-25T10:08:20.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.504+0000 7f1abb200640 1 -- 172.21.15.119:0/3984188259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1ab415a250 con 0x7f1ab40d09b0 2024-04-25T10:08:20.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.509+0000 7f1aae000640 1 -- 172.21.15.119:0/3984188259 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1aa4059300 con 0x7f1ab40d09b0 2024-04-25T10:08:20.515 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-04-25T10:08:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA== 2024-04-25T10:08:20.515 INFO:teuthology.orchestra.run.smithi119.stdout: caps mds = "allow" 2024-04-25T10:08:20.516 INFO:teuthology.orchestra.run.smithi119.stdout: caps mon = "allow r" 2024-04-25T10:08:20.516 INFO:teuthology.orchestra.run.smithi119.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:08:20.516 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2024-04-25T10:08:20.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.512+0000 7f1abb200640 1 -- 172.21.15.119:0/3984188259 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a9806b4d0 msgr2=0x7f1a9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.512+0000 7f1abb200640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a9806b4d0 0x7f1a9806d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ab415d500 tx=0x7f1aa8009040 comp rx=0 tx=0).stop 2024-04-25T10:08:20.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.512+0000 7f1abb200640 1 -- 172.21.15.119:0/3984188259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab40d09b0 msgr2=0x7f1ab40cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.512+0000 7f1abb200640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab40d09b0 0x7f1ab40cb560 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400e870 tx=0x7f1aa400ed30 comp rx=0 tx=0).stop 2024-04-25T10:08:20.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.513+0000 7f1abb200640 1 -- 172.21.15.119:0/3984188259 shutdown_connections 2024-04-25T10:08:20.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.513+0000 7f1abb200640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a9806b4d0 0x7f1a9806d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.513+0000 7f1abb200640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ab415daf0 0x7f1ab40b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.513+0000 7f1abb200640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab40d09b0 0x7f1ab40cb560 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.513+0000 7f1abb200640 1 --2- 172.21.15.119:0/3984188259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab40b6ab0 0x7f1ab415e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.513+0000 7f1abb200640 1 -- 172.21.15.119:0/3984188259 >> 172.21.15.119:0/3984188259 conn(0x7f1ab40b55c0 msgr2=0x7f1ab40b59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.513+0000 7f1abb200640 1 -- 172.21.15.119:0/3984188259 shutdown_connections 2024-04-25T10:08:20.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.513+0000 7f1abb200640 1 -- 172.21.15.119:0/3984188259 wait complete. 2024-04-25T10:08:20.540 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:20.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.734+0000 7fce53a00640 1 Processor -- start 2024-04-25T10:08:20.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.734+0000 7fce53a00640 1 -- start start 2024-04-25T10:08:20.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.736+0000 7fce53a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fce4c15dce0 0x7fce4c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.736+0000 7fce53a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce4c0d35b0 0x7fce4c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.736+0000 7fce53a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c15e0c0 con 0x7fce4c0d1f90 2024-04-25T10:08:20.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.736+0000 7fce53a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4c0b7b10 con 0x7fce4c0d35b0 2024-04-25T10:08:20.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.736+0000 7fce53a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4c0d2510 con 0x7fce4c15dce0 2024-04-25T10:08:20.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.736+0000 7fce4be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce4c0d35b0 0x7fce4c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.736+0000 7fce4be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce4c0d35b0 0x7fce4c0b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34432/0 (socket says 172.21.15.119:34432) 2024-04-25T10:08:20.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.736+0000 7fce4be00640 1 -- 172.21.15.119:0/3061276423 learned_addr learned my addr 172.21.15.119:0/3061276423 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:20.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.737+0000 7fce51000640 1 --2- 172.21.15.119:0/3061276423 >> v2:172.21.15.132:3300/0 conn(0x7fce4c15dce0 0x7fce4c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.737+0000 7fce4be00640 1 -- 172.21.15.119:0/3061276423 >> v2:172.21.15.132:3300/0 conn(0x7fce4c15dce0 msgr2=0x7fce4c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.737+0000 7fce4be00640 1 --2- 172.21.15.119:0/3061276423 >> v2:172.21.15.132:3300/0 conn(0x7fce4c15dce0 0x7fce4c0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.737+0000 7fce4be00640 1 -- 172.21.15.119:0/3061276423 >> v1:172.21.15.132:6789/0 conn(0x7fce4c0d1f90 legacy=0x7fce4c06bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.737+0000 7fce4be00640 1 -- 172.21.15.119:0/3061276423 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c0d27f0 con 0x7fce4c0d35b0 2024-04-25T10:08:20.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.738+0000 7fce4be00640 1 --2- 172.21.15.119:0/3061276423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce4c0d35b0 0x7fce4c0b5450 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0ce220 tx=0x7fce3802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b9deb224b02fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.738+0000 7fce4b400640 1 -- 172.21.15.119:0/3061276423 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38037070 con 0x7fce4c0d35b0 2024-04-25T10:08:20.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.738+0000 7fce4b400640 1 -- 172.21.15.119:0/3061276423 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce38032440 con 0x7fce4c0d35b0 2024-04-25T10:08:20.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.738+0000 7fce4b400640 1 -- 172.21.15.119:0/3061276423 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38032810 con 0x7fce4c0d35b0 2024-04-25T10:08:20.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.739+0000 7fce53a00640 1 -- 172.21.15.119:0/3061276423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce4c0d35b0 msgr2=0x7fce4c0b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.739+0000 7fce53a00640 1 --2- 172.21.15.119:0/3061276423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce4c0d35b0 0x7fce4c0b5450 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0ce220 tx=0x7fce3802a230 comp rx=0 tx=0).stop 2024-04-25T10:08:20.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.739+0000 7fce53a00640 1 -- 172.21.15.119:0/3061276423 shutdown_connections 2024-04-25T10:08:20.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.739+0000 7fce53a00640 1 --2- 172.21.15.119:0/3061276423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce4c0d35b0 0x7fce4c0b5450 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.739+0000 7fce53a00640 1 --2- 172.21.15.119:0/3061276423 >> v2:172.21.15.132:3300/0 conn(0x7fce4c15dce0 0x7fce4c0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.739+0000 7fce53a00640 1 -- 172.21.15.119:0/3061276423 >> 172.21.15.119:0/3061276423 conn(0x7fce4c072900 msgr2=0x7fce4c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.739+0000 7fce53a00640 1 -- 172.21.15.119:0/3061276423 shutdown_connections 2024-04-25T10:08:20.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.739+0000 7fce53a00640 1 -- 172.21.15.119:0/3061276423 wait complete. 2024-04-25T10:08:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.741+0000 7fce53a00640 1 Processor -- start 2024-04-25T10:08:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.741+0000 7fce53a00640 1 -- start start 2024-04-25T10:08:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.742+0000 7fce53a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce4c0d1f90 0x7fce4c15f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.742+0000 7fce53a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce4c0d35b0 0x7fce4c0c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.742+0000 7fce53a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce4c15dce0 0x7fce4c1f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.742+0000 7fce53a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0d3090 con 0x7fce4c0d1f90 2024-04-25T10:08:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.742+0000 7fce53a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0d2db0 con 0x7fce4c0d35b0 2024-04-25T10:08:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.742+0000 7fce53a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0b9820 con 0x7fce4c15dce0 2024-04-25T10:08:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.742+0000 7fce51000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce4c0d1f90 0x7fce4c15f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.742+0000 7fce51000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce4c0d1f90 0x7fce4c15f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34448/0 (socket says 172.21.15.119:34448) 2024-04-25T10:08:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.742+0000 7fce51000640 1 -- 172.21.15.119:0/3473158135 learned_addr learned my addr 172.21.15.119:0/3473158135 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.743+0000 7fce4be00640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce4c0d35b0 0x7fce4c0c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.743+0000 7fce51a00640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce4c15dce0 0x7fce4c1f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.743+0000 7fce51000640 1 -- 172.21.15.119:0/3473158135 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce4c15dce0 msgr2=0x7fce4c1f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.743+0000 7fce51000640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce4c15dce0 0x7fce4c1f7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.743+0000 7fce51000640 1 -- 172.21.15.119:0/3473158135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce4c0d35b0 msgr2=0x7fce4c0c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.743+0000 7fce51000640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce4c0d35b0 0x7fce4c0c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.743+0000 7fce51000640 1 -- 172.21.15.119:0/3473158135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c15f8a0 con 0x7fce4c0d1f90 2024-04-25T10:08:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.743+0000 7fce51a00640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce4c15dce0 0x7fce4c1f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.743+0000 7fce51000640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce4c0d1f90 0x7fce4c15f210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00c7f0 tx=0x7fce3c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.743+0000 7fce48c00640 1 -- 172.21.15.119:0/3473158135 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c015070 con 0x7fce4c0d1f90 2024-04-25T10:08:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.743+0000 7fce48c00640 1 -- 172.21.15.119:0/3473158135 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c004220 con 0x7fce4c0d1f90 2024-04-25T10:08:20.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.743+0000 7fce48c00640 1 -- 172.21.15.119:0/3473158135 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c007620 con 0x7fce4c0d1f90 2024-04-25T10:08:20.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.743+0000 7fce53a00640 1 -- 172.21.15.119:0/3473158135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c15fb80 con 0x7fce4c0d1f90 2024-04-25T10:08:20.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.744+0000 7fce53a00640 1 -- 172.21.15.119:0/3473158135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c160140 con 0x7fce4c0d1f90 2024-04-25T10:08:20.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.745+0000 7fce53a00640 1 -- 172.21.15.119:0/3473158135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4c0749b0 con 0x7fce4c0d1f90 2024-04-25T10:08:20.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.746+0000 7fce48c00640 1 -- 172.21.15.119:0/3473158135 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce3c0079d0 con 0x7fce4c0d1f90 2024-04-25T10:08:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.747+0000 7fce48c00640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fce2c06b4d0 0x7fce2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.747+0000 7fce48c00640 1 -- 172.21.15.119:0/3473158135 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce3c08a660 con 0x7fce4c0d1f90 2024-04-25T10:08:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.747+0000 7fce4be00640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fce2c06b4d0 0x7fce2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.748+0000 7fce4be00640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fce2c06b4d0 0x7fce2c06d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fce38034000 tx=0x7fce3802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.752+0000 7fce48c00640 1 -- 172.21.15.119:0/3473158135 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce3c059440 con 0x7fce4c0d1f90 2024-04-25T10:08:20.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.986+0000 7fce53a00640 1 -- 172.21.15.119:0/3473158135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce4c153460 con 0x7fce4c0d1f90 2024-04-25T10:08:20.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.987+0000 7fce48c00640 1 -- 172.21.15.119:0/3473158135 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fce3c058d40 con 0x7fce4c0d1f90 2024-04-25T10:08:20.993 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:20.993 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1554372269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1554372269},{"type":"v1","addr":"172.21.15.119:6837","nonce":1554372269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4699,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4250304148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4250304148},{"type":"v1","addr":"172.21.15.132:6834","nonce":4250304148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/637229574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":637229574},{"type":"v1","addr":"172.21.15.132:6835","nonce":637229574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:14.553437+0000","modified":"2024-04-25T10:08:17.586270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6836/3585315292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3585315292},{"type":"v1","addr":"172.21.15.119:6836","nonce":3585315292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:08:20.994 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 16 2024-04-25T10:08:20.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.990+0000 7fce53a00640 1 -- 172.21.15.119:0/3473158135 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fce2c06b4d0 msgr2=0x7fce2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.990+0000 7fce53a00640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fce2c06b4d0 0x7fce2c06d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fce38034000 tx=0x7fce3802e000 comp rx=0 tx=0).stop 2024-04-25T10:08:20.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.990+0000 7fce53a00640 1 -- 172.21.15.119:0/3473158135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce4c0d1f90 msgr2=0x7fce4c15f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.990+0000 7fce53a00640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce4c0d1f90 0x7fce4c15f210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00c7f0 tx=0x7fce3c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.990+0000 7fce53a00640 1 -- 172.21.15.119:0/3473158135 shutdown_connections 2024-04-25T10:08:20.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.990+0000 7fce53a00640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fce2c06b4d0 0x7fce2c06d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.990+0000 7fce53a00640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce4c15dce0 0x7fce4c1f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.990+0000 7fce53a00640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce4c0d35b0 0x7fce4c0c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.990+0000 7fce53a00640 1 --2- 172.21.15.119:0/3473158135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce4c0d1f90 0x7fce4c15f210 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.991+0000 7fce53a00640 1 -- 172.21.15.119:0/3473158135 >> 172.21.15.119:0/3473158135 conn(0x7fce4c072900 msgr2=0x7fce4c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.991+0000 7fce53a00640 1 -- 172.21.15.119:0/3473158135 shutdown_connections 2024-04-25T10:08:20.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:20.991+0000 7fce53a00640 1 -- 172.21.15.119:0/3473158135 wait complete. 2024-04-25T10:08:21.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:14.553437+0000', 'modified': '2024-04-25T10:08:17.586270+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5049}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5049': {'gid': 5049, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.119:6836/3585315292', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 3585315292}, {'type': 'v1', 'addr': '172.21.15.119:6836', 'nonce': 3585315292}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:08:21.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:08:21.420 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:08:21.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.626+0000 7f02b5a00640 1 Processor -- start 2024-04-25T10:08:21.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.627+0000 7f02b5a00640 1 -- start start 2024-04-25T10:08:21.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.628+0000 7f02b5a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f02b0071200 0x7f02b0071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.628+0000 7f02b5a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b00cee90 0x7f02b00cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.628+0000 7f02b5a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b00cc9a0 con 0x7f02b0145260 2024-04-25T10:08:21.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.628+0000 7f02b5a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b0071ba0 con 0x7f02b0071200 2024-04-25T10:08:21.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.628+0000 7f02b5a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b00cea00 con 0x7f02b00cee90 2024-04-25T10:08:21.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.628+0000 7f02aea00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f02b0145260 0x7f02b00cbf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41780/0 (socket says 172.21.15.119:41780) 2024-04-25T10:08:21.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.628+0000 7f02aea00640 1 -- 172.21.15.119:0/3048835136 learned_addr learned my addr 172.21.15.119:0/3048835136 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:21.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.628+0000 7f02ae000640 1 --2- 172.21.15.119:0/3048835136 >> v2:172.21.15.119:3300/0 conn(0x7f02b0071200 0x7f02b0071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.629+0000 7f02af400640 1 --2- 172.21.15.119:0/3048835136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b00cee90 0x7f02b00cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.629+0000 7f02ae000640 1 -- 172.21.15.119:0/3048835136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b00cee90 msgr2=0x7f02b00cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.629+0000 7f02ad600640 1 -- 172.21.15.119:0/3048835136 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715808766 0 0) 0x7f02b00cc9a0 con 0x7f02b0145260 2024-04-25T10:08:21.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.629+0000 7f02ae000640 1 --2- 172.21.15.119:0/3048835136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b00cee90 0x7f02b00cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.629+0000 7f02ae000640 1 -- 172.21.15.119:0/3048835136 >> v1:172.21.15.119:6789/0 conn(0x7f02b0145260 legacy=0x7f02b00cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.629+0000 7f02ae000640 1 -- 172.21.15.119:0/3048835136 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b00ccc80 con 0x7f02b0071200 2024-04-25T10:08:21.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.630+0000 7f02ae000640 1 --2- 172.21.15.119:0/3048835136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0071200 0x7f02b0071660 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f02980097a0 tx=0x7f029802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a39029ac0d2bd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.630+0000 7f02ad600640 1 -- 172.21.15.119:0/3048835136 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0298037070 con 0x7f02b0071200 2024-04-25T10:08:21.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.631+0000 7f02ad600640 1 -- 172.21.15.119:0/3048835136 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0298032440 con 0x7f02b0071200 2024-04-25T10:08:21.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.631+0000 7f02ad600640 1 -- 172.21.15.119:0/3048835136 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0298032810 con 0x7f02b0071200 2024-04-25T10:08:21.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.631+0000 7f02b5a00640 1 -- 172.21.15.119:0/3048835136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0071200 msgr2=0x7f02b0071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.631+0000 7f02b5a00640 1 --2- 172.21.15.119:0/3048835136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0071200 0x7f02b0071660 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f02980097a0 tx=0x7f029802a180 comp rx=0 tx=0).stop 2024-04-25T10:08:21.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.631+0000 7f02b5a00640 1 -- 172.21.15.119:0/3048835136 shutdown_connections 2024-04-25T10:08:21.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.631+0000 7f02b5a00640 1 --2- 172.21.15.119:0/3048835136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b00cee90 0x7f02b00cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.631+0000 7f02b5a00640 1 --2- 172.21.15.119:0/3048835136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0071200 0x7f02b0071660 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.631+0000 7f02b5a00640 1 -- 172.21.15.119:0/3048835136 >> 172.21.15.119:0/3048835136 conn(0x7f02b006aba0 msgr2=0x7f02b006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:21.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.631+0000 7f02b5a00640 1 -- 172.21.15.119:0/3048835136 shutdown_connections 2024-04-25T10:08:21.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.631+0000 7f02b5a00640 1 -- 172.21.15.119:0/3048835136 wait complete. 2024-04-25T10:08:21.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.633+0000 7f02b5a00640 1 Processor -- start 2024-04-25T10:08:21.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.633+0000 7f02b5a00640 1 -- start start 2024-04-25T10:08:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.634+0000 7f02b5a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b0071200 0x7f02b0138000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.634+0000 7f02b5a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02b00cee90 0x7f02b00b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.634+0000 7f02b5a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0145260 0x7f02b01f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.634+0000 7f02b5a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b00cc660 con 0x7f02b0145260 2024-04-25T10:08:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.634+0000 7f02b5a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b00cd240 con 0x7f02b00cee90 2024-04-25T10:08:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.634+0000 7f02b5a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b0145640 con 0x7f02b0071200 2024-04-25T10:08:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.635+0000 7f02af400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0145260 0x7f02b01f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.635+0000 7f02af400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0145260 0x7f02b01f3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56776/0 (socket says 172.21.15.119:56776) 2024-04-25T10:08:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.635+0000 7f02af400640 1 -- 172.21.15.119:0/2913855789 learned_addr learned my addr 172.21.15.119:0/2913855789 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.635+0000 7f02aea00640 1 --2- 172.21.15.119:0/2913855789 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b0071200 0x7f02b0138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.635+0000 7f02ae000640 1 --2- 172.21.15.119:0/2913855789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02b00cee90 0x7f02b00b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.635+0000 7f02af400640 1 -- 172.21.15.119:0/2913855789 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b0071200 msgr2=0x7f02b0138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.635+0000 7f02af400640 1 --2- 172.21.15.119:0/2913855789 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b0071200 0x7f02b0138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.635+0000 7f02af400640 1 -- 172.21.15.119:0/2913855789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02b00cee90 msgr2=0x7f02b00b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.635+0000 7f02af400640 1 --2- 172.21.15.119:0/2913855789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02b00cee90 0x7f02b00b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.635+0000 7f02af400640 1 -- 172.21.15.119:0/2913855789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b00b20c0 con 0x7f02b0145260 2024-04-25T10:08:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.636+0000 7f02af400640 1 --2- 172.21.15.119:0/2913855789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0145260 0x7f02b01f3880 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f02a8007e90 tx=0x7f02a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.636+0000 7f02a6a00640 1 -- 172.21.15.119:0/2913855789 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a8014420 con 0x7f02b0145260 2024-04-25T10:08:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.636+0000 7f02a6a00640 1 -- 172.21.15.119:0/2913855789 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a8019070 con 0x7f02b0145260 2024-04-25T10:08:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.636+0000 7f02b5a00640 1 -- 172.21.15.119:0/2913855789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b00b23a0 con 0x7f02b0145260 2024-04-25T10:08:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.636+0000 7f02a6a00640 1 -- 172.21.15.119:0/2913855789 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a8014800 con 0x7f02b0145260 2024-04-25T10:08:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.636+0000 7f02b5a00640 1 -- 172.21.15.119:0/2913855789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b00b2960 con 0x7f02b0145260 2024-04-25T10:08:21.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.638+0000 7f02a6a00640 1 -- 172.21.15.119:0/2913855789 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02a80149a0 con 0x7f02b0145260 2024-04-25T10:08:21.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.639+0000 7f02a6a00640 1 --2- 172.21.15.119:0/2913855789 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f028c06b500 0x7f028c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.639+0000 7f02a6a00640 1 -- 172.21.15.119:0/2913855789 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02a808a9c0 con 0x7f02b0145260 2024-04-25T10:08:21.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.639+0000 7f02aea00640 1 --2- 172.21.15.119:0/2913855789 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f028c06b500 0x7f028c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.640+0000 7f02b87e7740 1 Processor -- start 2024-04-25T10:08:21.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.640+0000 7f02b87e7740 1 -- start start 2024-04-25T10:08:21.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.640+0000 7f02aea00640 1 --2- 172.21.15.119:0/2913855789 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f028c06b500 0x7f028c06d9c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f02a0000f60 tx=0x7f02a0007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.641+0000 7f02b87e7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f432950 0x55bf0f45a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.641+0000 7f02b87e7740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f45acd0 0x55bf0f432d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.641+0000 7f02b87e7740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f47b5c0 0x55bf0f431110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.641+0000 7f02b87e7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0f396620 con 0x55bf0f432950 2024-04-25T10:08:21.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.641+0000 7f02b87e7740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0f33fca0 con 0x55bf0f45acd0 2024-04-25T10:08:21.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.641+0000 7f02b87e7740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf0f3926c0 con 0x55bf0f47b5c0 2024-04-25T10:08:21.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.641+0000 7f02aea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f432950 0x55bf0f45a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.641+0000 7f02aea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f432950 0x55bf0f45a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56778/0 (socket says 172.21.15.119:56778) 2024-04-25T10:08:21.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.641+0000 7f02aea00640 1 -- 172.21.15.119:0/2150336129 learned_addr learned my addr 172.21.15.119:0/2150336129 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:21.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.641+0000 7f02af400640 1 --2- 172.21.15.119:0/2150336129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f47b5c0 0x55bf0f431110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.641+0000 7f02aea00640 1 -- 172.21.15.119:0/2150336129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f47b5c0 msgr2=0x55bf0f431110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.642+0000 7f02aea00640 1 --2- 172.21.15.119:0/2150336129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f47b5c0 0x55bf0f431110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.642+0000 7f02ae000640 1 --2- 172.21.15.119:0/2150336129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f45acd0 0x55bf0f432d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.642+0000 7f02aea00640 1 -- 172.21.15.119:0/2150336129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f45acd0 msgr2=0x55bf0f432d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.642+0000 7f02aea00640 1 --2- 172.21.15.119:0/2150336129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f45acd0 0x55bf0f432d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.642+0000 7f02aea00640 1 -- 172.21.15.119:0/2150336129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf0f346f40 con 0x55bf0f432950 2024-04-25T10:08:21.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.642+0000 7f02aea00640 1 --2- 172.21.15.119:0/2150336129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f432950 0x55bf0f45a790 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f02a0005d80 tx=0x7f02a0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939ea050d7e62d61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.642+0000 7f029f400640 1 -- 172.21.15.119:0/2150336129 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a00071a0 con 0x55bf0f432950 2024-04-25T10:08:21.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.642+0000 7f029f400640 1 -- 172.21.15.119:0/2150336129 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a0015110 con 0x55bf0f432950 2024-04-25T10:08:21.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.642+0000 7f029f400640 1 -- 172.21.15.119:0/2150336129 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a0015bc0 con 0x55bf0f432950 2024-04-25T10:08:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.643+0000 7f02b87e7740 1 -- 172.21.15.119:0/2150336129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f432950 msgr2=0x55bf0f45a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.643+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2150336129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f432950 0x55bf0f45a790 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f02a0005d80 tx=0x7f02a0012000 comp rx=0 tx=0).stop 2024-04-25T10:08:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.643+0000 7f02b87e7740 1 -- 172.21.15.119:0/2150336129 shutdown_connections 2024-04-25T10:08:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.643+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2150336129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f47b5c0 0x55bf0f431110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.643+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2150336129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f45acd0 0x55bf0f432d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.643+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2150336129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f432950 0x55bf0f45a790 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.643+0000 7f02b87e7740 1 -- 172.21.15.119:0/2150336129 >> 172.21.15.119:0/2150336129 conn(0x55bf0f42fb70 msgr2=0x55bf0f478550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.643+0000 7f02b87e7740 1 -- 172.21.15.119:0/2150336129 shutdown_connections 2024-04-25T10:08:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.643+0000 7f02b87e7740 1 -- 172.21.15.119:0/2150336129 wait complete. 2024-04-25T10:08:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.644+0000 7f02b87e7740 1 -- start start 2024-04-25T10:08:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.644+0000 7f02b87e7740 1 Processor -- start 2024-04-25T10:08:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.645+0000 7f02b87e7740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f45acd0 0x55bf0f451140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.645+0000 7f02b87e7740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f47b5c0 0x55bf0f488aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.645+0000 7f02b87e7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f4594d0 0x55bf0f44ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.645+0000 7f02b87e7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0f45a300 con 0x55bf0f4594d0 2024-04-25T10:08:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.645+0000 7f02b87e7740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0f3351f0 con 0x55bf0f45acd0 2024-04-25T10:08:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.645+0000 7f02b87e7740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf0f459cf0 con 0x55bf0f47b5c0 2024-04-25T10:08:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.645+0000 7f02b87e7740 10 client.0 _ll_register_callbacks cb 0x55bf0f352ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.645+0000 7f02af400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f4594d0 0x55bf0f44ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.645+0000 7f02af400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f4594d0 0x55bf0f44ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56782/0 (socket says 172.21.15.119:56782) 2024-04-25T10:08:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.645+0000 7f02af400640 1 -- 172.21.15.119:0/2773321117 learned_addr learned my addr 172.21.15.119:0/2773321117 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.646+0000 7f02af400640 1 -- 172.21.15.119:0/2773321117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f47b5c0 msgr2=0x55bf0f488aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.646+0000 7f02af400640 1 --2- 172.21.15.119:0/2773321117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f47b5c0 0x55bf0f488aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.646+0000 7f02af400640 1 -- 172.21.15.119:0/2773321117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f45acd0 msgr2=0x55bf0f451140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.646+0000 7f02af400640 1 --2- 172.21.15.119:0/2773321117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f45acd0 0x55bf0f451140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.646+0000 7f02af400640 1 -- 172.21.15.119:0/2773321117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf0f354c50 con 0x55bf0f4594d0 2024-04-25T10:08:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.646+0000 7f02af400640 1 --2- 172.21.15.119:0/2773321117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f4594d0 0x55bf0f44ed90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f02a8061310 tx=0x7f02a80591f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796c7d459d78cd08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.646+0000 7f0293e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:08:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.646+0000 7f0293e00640 1 -- 172.21.15.119:0/2773321117 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a80557a0 con 0x55bf0f4594d0 2024-04-25T10:08:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.646+0000 7f0293e00640 1 -- 172.21.15.119:0/2773321117 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02a80613b0 con 0x55bf0f4594d0 2024-04-25T10:08:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.646+0000 7f02b87e7740 1 -- 172.21.15.119:0/2773321117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02a0015110 con 0x55bf0f4594d0 2024-04-25T10:08:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.647+0000 7f0293e00640 1 -- 172.21.15.119:0/2773321117 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02a8055940 con 0x55bf0f4594d0 2024-04-25T10:08:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.647+0000 7f02b87e7740 10 client.5229 fetch_fsmap learned FSMap version 16 2024-04-25T10:08:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.647+0000 7f02b87e7740 1 -- 172.21.15.119:0/2773321117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55bf0f0ab2d0 con 0x55bf0f4594d0 2024-04-25T10:08:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.647+0000 7f0293e00640 1 -- 172.21.15.119:0/2773321117 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a0015110 con 0x55bf0f4594d0 2024-04-25T10:08:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.647+0000 7f0293e00640 1 -- 172.21.15.119:0/2773321117 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f02a8058580 con 0x55bf0f4594d0 2024-04-25T10:08:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.647+0000 7f02b87e7740 10 client.5229 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T10:08:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.647+0000 7f02b87e7740 10 client.5229 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:08:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.647+0000 7f02b87e7740 20 client.5229 populate_metadata read hostname 'smithi119' 2024-04-25T10:08:21.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.647+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2773321117 >> [v2:172.21.15.119:6834/3585315292,v1:172.21.15.119:6836/3585315292] conn(0x55bf0f456980 0x55bf0f456d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.647+0000 7f02b87e7740 4 client.5229 mds_command: new command op to 5049 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:21.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.647+0000 7f02b87e7740 1 -- 172.21.15.119:0/2773321117 --> [v2:172.21.15.119:6834/3585315292,v1:172.21.15.119:6836/3585315292] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf0f3a3b60 con 0x55bf0f456980 2024-04-25T10:08:21.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.647+0000 7f02aea00640 1 --2- 172.21.15.119:0/2773321117 >> [v2:172.21.15.119:6834/3585315292,v1:172.21.15.119:6836/3585315292] conn(0x55bf0f456980 0x55bf0f456d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.648+0000 7f02aea00640 1 --2- 172.21.15.119:0/2773321117 >> [v2:172.21.15.119:6834/3585315292,v1:172.21.15.119:6836/3585315292] conn(0x55bf0f456980 0x55bf0f456d60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3120fa001376f9b1 server_cookie=2db9d21fce3dcf74 in_seq=0 out_seq=0 2024-04-25T10:08:21.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.648+0000 7f0293e00640 10 client.5229 ms_handle_connect on v2:172.21.15.119:6834/3585315292 2024-04-25T10:08:21.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f0293e00640 1 -- 172.21.15.119:0/2773321117 <== mds.0 v2:172.21.15.119:6834/3585315292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf0f3a3b60 con 0x55bf0f456980 2024-04-25T10:08:21.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f0293e00640 10 client.5229 handle_command_reply: tid=0 2024-04-25T10:08:21.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f02b87e7740 1 client.5229 shutdown 2024-04-25T10:08:21.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f02b87e7740 1 -- 172.21.15.119:0/2773321117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f4594d0 msgr2=0x55bf0f44ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2773321117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f4594d0 0x55bf0f44ed90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f02a8061310 tx=0x7f02a80591f0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f02b87e7740 1 -- 172.21.15.119:0/2773321117 shutdown_connections 2024-04-25T10:08:21.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2773321117 >> [v2:172.21.15.119:6834/3585315292,v1:172.21.15.119:6836/3585315292] conn(0x55bf0f456980 0x55bf0f456d60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2773321117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f4594d0 0x55bf0f44ed90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2773321117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f47b5c0 0x55bf0f488aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2773321117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f45acd0 0x55bf0f451140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f0293e00640 0 client.5229 ms_handle_reset on v2:172.21.15.119:6834/3585315292 2024-04-25T10:08:21.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f02b87e7740 1 -- 172.21.15.119:0/2773321117 >> 172.21.15.119:0/2773321117 conn(0x55bf0f432950 msgr2=0x55bf0f478350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:21.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f02b87e7740 1 -- 172.21.15.119:0/2773321117 shutdown_connections 2024-04-25T10:08:21.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f02b87e7740 1 -- 172.21.15.119:0/2773321117 wait complete. 2024-04-25T10:08:21.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.649+0000 7f02b87e7740 20 client.5229 trim_cache size 0 max 16384 2024-04-25T10:08:21.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.666+0000 7f02b87e7740 1 Processor -- start 2024-04-25T10:08:21.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.667+0000 7f02b87e7740 1 -- start start 2024-04-25T10:08:21.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.667+0000 7f02b87e7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f4594d0 0x55bf0f431110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.667+0000 7f02b87e7740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f45acd0 0x55bf0f45b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.668+0000 7f02b87e7740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f47b5c0 0x55bf0f44ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.668+0000 7f02b87e7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b01fee40 con 0x55bf0f4594d0 2024-04-25T10:08:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.668+0000 7f02b87e7740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b01fe240 con 0x55bf0f45acd0 2024-04-25T10:08:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.668+0000 7f02b87e7740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b01fe840 con 0x55bf0f47b5c0 2024-04-25T10:08:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.668+0000 7f02aea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f4594d0 0x55bf0f431110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.668+0000 7f02aea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f4594d0 0x55bf0f431110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56792/0 (socket says 172.21.15.119:56792) 2024-04-25T10:08:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.668+0000 7f02aea00640 1 -- 172.21.15.119:0/1299273182 learned_addr learned my addr 172.21.15.119:0/1299273182 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.668+0000 7f02aea00640 1 -- 172.21.15.119:0/1299273182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f47b5c0 msgr2=0x55bf0f44ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.668+0000 7f02aea00640 1 --2- 172.21.15.119:0/1299273182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f47b5c0 0x55bf0f44ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.668+0000 7f02aea00640 1 -- 172.21.15.119:0/1299273182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f45acd0 msgr2=0x55bf0f45b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.668+0000 7f02aea00640 1 --2- 172.21.15.119:0/1299273182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f45acd0 0x55bf0f45b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.668+0000 7f02aea00640 1 -- 172.21.15.119:0/1299273182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf0f1756b0 con 0x55bf0f4594d0 2024-04-25T10:08:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.668+0000 7f02aea00640 1 --2- 172.21.15.119:0/1299273182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f4594d0 0x55bf0f431110 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f02a00011e0 tx=0x7f02a0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c126c5d8af9cd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.669+0000 7f029ea00640 1 -- 172.21.15.119:0/1299273182 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a002e070 con 0x55bf0f4594d0 2024-04-25T10:08:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.669+0000 7f029ea00640 1 -- 172.21.15.119:0/1299273182 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a0015290 con 0x55bf0f4594d0 2024-04-25T10:08:21.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.669+0000 7f029ea00640 1 -- 172.21.15.119:0/1299273182 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a0010040 con 0x55bf0f4594d0 2024-04-25T10:08:21.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.669+0000 7f02b87e7740 1 -- 172.21.15.119:0/1299273182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f4594d0 msgr2=0x55bf0f431110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.669+0000 7f02b87e7740 1 --2- 172.21.15.119:0/1299273182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f4594d0 0x55bf0f431110 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f02a00011e0 tx=0x7f02a0013000 comp rx=0 tx=0).stop 2024-04-25T10:08:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.669+0000 7f02b87e7740 1 -- 172.21.15.119:0/1299273182 shutdown_connections 2024-04-25T10:08:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.669+0000 7f02b87e7740 1 --2- 172.21.15.119:0/1299273182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f47b5c0 0x55bf0f44ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.669+0000 7f02b87e7740 1 --2- 172.21.15.119:0/1299273182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f45acd0 0x55bf0f45b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.669+0000 7f02b87e7740 1 --2- 172.21.15.119:0/1299273182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f4594d0 0x55bf0f431110 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.669+0000 7f02b87e7740 1 -- 172.21.15.119:0/1299273182 >> 172.21.15.119:0/1299273182 conn(0x55bf0f432950 msgr2=0x55bf0f492770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.670+0000 7f02b87e7740 1 -- 172.21.15.119:0/1299273182 shutdown_connections 2024-04-25T10:08:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.670+0000 7f02b87e7740 1 -- 172.21.15.119:0/1299273182 wait complete. 2024-04-25T10:08:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.670+0000 7f02b87e7740 1 -- start start 2024-04-25T10:08:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.670+0000 7f02b87e7740 1 Processor -- start 2024-04-25T10:08:21.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.671+0000 7f02b87e7740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f4594d0 0x55bf0f4e24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.671+0000 7f02b87e7740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f45acd0 0x55bf0f44ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.671+0000 7f02b87e7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f47b5c0 0x55bf0f44f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.671+0000 7f02b87e7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0f47c620 con 0x55bf0f47b5c0 2024-04-25T10:08:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.671+0000 7f02b87e7740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf0f451d90 con 0x55bf0f4594d0 2024-04-25T10:08:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.671+0000 7f02b87e7740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf0f450490 con 0x55bf0f45acd0 2024-04-25T10:08:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.671+0000 7f02b87e7740 10 client.0 _ll_register_callbacks cb 0x7f02a80553d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.672+0000 7f02af400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f47b5c0 0x55bf0f44f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.672+0000 7f02af400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f47b5c0 0x55bf0f44f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56806/0 (socket says 172.21.15.119:56806) 2024-04-25T10:08:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.672+0000 7f02af400640 1 -- 172.21.15.119:0/2337509940 learned_addr learned my addr 172.21.15.119:0/2337509940 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.672+0000 7f02aea00640 1 --2- 172.21.15.119:0/2337509940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f4594d0 0x55bf0f4e24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.672+0000 7f02ae000640 1 --2- 172.21.15.119:0/2337509940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f45acd0 0x55bf0f44ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.672+0000 7f02af400640 1 -- 172.21.15.119:0/2337509940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f45acd0 msgr2=0x55bf0f44ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.672+0000 7f02af400640 1 --2- 172.21.15.119:0/2337509940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f45acd0 0x55bf0f44ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.672+0000 7f02af400640 1 -- 172.21.15.119:0/2337509940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f4594d0 msgr2=0x55bf0f4e24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.672+0000 7f02af400640 1 --2- 172.21.15.119:0/2337509940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f4594d0 0x55bf0f4e24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.672+0000 7f02af400640 1 -- 172.21.15.119:0/2337509940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf0f451050 con 0x55bf0f47b5c0 2024-04-25T10:08:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.672+0000 7f02ae000640 1 --2- 172.21.15.119:0/2337509940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f45acd0 0x55bf0f44ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.672+0000 7f02af400640 1 --2- 172.21.15.119:0/2337509940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f47b5c0 0x55bf0f44f2d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f02a800c040 tx=0x7f02a8088db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9261fe7476f5a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.672+0000 7f0293e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:08:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.673+0000 7f0293e00640 1 -- 172.21.15.119:0/2337509940 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a8087070 con 0x55bf0f47b5c0 2024-04-25T10:08:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.673+0000 7f02b87e7740 1 -- 172.21.15.119:0/2337509940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02a0015290 con 0x55bf0f47b5c0 2024-04-25T10:08:21.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.673+0000 7f0293e00640 1 -- 172.21.15.119:0/2337509940 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02a805f410 con 0x55bf0f47b5c0 2024-04-25T10:08:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.673+0000 7f0293e00640 1 -- 172.21.15.119:0/2337509940 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02a80557a0 con 0x55bf0f47b5c0 2024-04-25T10:08:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.673+0000 7f02b87e7740 10 client.5235 fetch_fsmap learned FSMap version 16 2024-04-25T10:08:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.673+0000 7f02b87e7740 1 -- 172.21.15.119:0/2337509940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55bf0f45de00 con 0x55bf0f47b5c0 2024-04-25T10:08:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.673+0000 7f0293e00640 1 -- 172.21.15.119:0/2337509940 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a0015290 con 0x55bf0f47b5c0 2024-04-25T10:08:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.673+0000 7f0293e00640 1 -- 172.21.15.119:0/2337509940 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f02a80822f0 con 0x55bf0f47b5c0 2024-04-25T10:08:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.673+0000 7f02b87e7740 10 client.5235 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T10:08:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.673+0000 7f02b87e7740 10 client.5235 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:08:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.673+0000 7f02b87e7740 20 client.5235 populate_metadata read hostname 'smithi119' 2024-04-25T10:08:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.673+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2337509940 >> [v2:172.21.15.119:6834/3585315292,v1:172.21.15.119:6836/3585315292] conn(0x55bf0f4e6880 0x55bf0f42c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.673+0000 7f02b87e7740 4 client.5235 mds_command: new command op to 5049 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:08:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.673+0000 7f02b87e7740 1 -- 172.21.15.119:0/2337509940 --> [v2:172.21.15.119:6834/3585315292,v1:172.21.15.119:6836/3585315292] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bf0f39deb0 con 0x55bf0f4e6880 2024-04-25T10:08:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.674+0000 7f02aea00640 1 --2- 172.21.15.119:0/2337509940 >> [v2:172.21.15.119:6834/3585315292,v1:172.21.15.119:6836/3585315292] conn(0x55bf0f4e6880 0x55bf0f42c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.674+0000 7f02aea00640 1 --2- 172.21.15.119:0/2337509940 >> [v2:172.21.15.119:6834/3585315292,v1:172.21.15.119:6836/3585315292] conn(0x55bf0f4e6880 0x55bf0f42c8b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff254857d4e8a93 server_cookie=93d544f6ddf2b795 in_seq=0 out_seq=0 2024-04-25T10:08:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.674+0000 7f0293e00640 10 client.5235 ms_handle_connect on v2:172.21.15.119:6834/3585315292 2024-04-25T10:08:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.674+0000 7f0293e00640 1 -- 172.21.15.119:0/2337509940 <== mds.0 v2:172.21.15.119:6834/3585315292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55bf0f39deb0 con 0x55bf0f4e6880 2024-04-25T10:08:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.674+0000 7f0293e00640 10 client.5235 handle_command_reply: tid=0 2024-04-25T10:08:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.674+0000 7f02b87e7740 1 client.5235 shutdown 2024-04-25T10:08:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.674+0000 7f02b87e7740 1 -- 172.21.15.119:0/2337509940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f47b5c0 msgr2=0x55bf0f44f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.674+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2337509940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f47b5c0 0x55bf0f44f2d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f02a800c040 tx=0x7f02a8088db0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.675+0000 7f02b87e7740 1 -- 172.21.15.119:0/2337509940 shutdown_connections 2024-04-25T10:08:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.675+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2337509940 >> [v2:172.21.15.119:6834/3585315292,v1:172.21.15.119:6836/3585315292] conn(0x55bf0f4e6880 0x55bf0f42c8b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.675+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2337509940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bf0f47b5c0 0x55bf0f44f2d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.675+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2337509940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55bf0f45acd0 0x55bf0f44ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.675+0000 7f02b87e7740 1 --2- 172.21.15.119:0/2337509940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bf0f4594d0 0x55bf0f4e24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.675+0000 7f0293e00640 0 client.5235 ms_handle_reset on v2:172.21.15.119:6834/3585315292 2024-04-25T10:08:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.675+0000 7f02b87e7740 1 -- 172.21.15.119:0/2337509940 >> 172.21.15.119:0/2337509940 conn(0x55bf0f432950 msgr2=0x55bf0f457140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.675+0000 7f02b87e7740 1 -- 172.21.15.119:0/2337509940 shutdown_connections 2024-04-25T10:08:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.675+0000 7f02b87e7740 1 -- 172.21.15.119:0/2337509940 wait complete. 2024-04-25T10:08:21.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.675+0000 7f02b87e7740 20 client.5235 trim_cache size 0 max 16384 2024-04-25T10:08:21.687 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:08:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "e0b0f428-b441-4237-a4d1-ed53321f87a5", 2024-04-25T10:08:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-04-25T10:08:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 5049, 2024-04-25T10:08:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-04-25T10:08:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-04-25T10:08:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-04-25T10:08:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.5725382459999997, 2024-04-25T10:08:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 16, 2024-04-25T10:08:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 33, 2024-04-25T10:08:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 33, 2024-04-25T10:08:21.688 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 15.429526174999999 2024-04-25T10:08:21.689 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:08:21.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.675+0000 7f02b5a00640 1 -- 172.21.15.119:0/2913855789 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f028c06b500 msgr2=0x7f028c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.675+0000 7f02b5a00640 1 --2- 172.21.15.119:0/2913855789 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f028c06b500 0x7f028c06d9c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f02a0000f60 tx=0x7f02a0007b50 comp rx=0 tx=0).stop 2024-04-25T10:08:21.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.675+0000 7f02b5a00640 1 -- 172.21.15.119:0/2913855789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0145260 msgr2=0x7f02b01f3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.675+0000 7f02b5a00640 1 --2- 172.21.15.119:0/2913855789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0145260 0x7f02b01f3880 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f02a8007e90 tx=0x7f02a800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.676+0000 7f02b5a00640 1 -- 172.21.15.119:0/2913855789 shutdown_connections 2024-04-25T10:08:21.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.676+0000 7f02b5a00640 1 --2- 172.21.15.119:0/2913855789 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f028c06b500 0x7f028c06d9c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.676+0000 7f02b5a00640 1 --2- 172.21.15.119:0/2913855789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02b0145260 0x7f02b01f3880 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.676+0000 7f02b5a00640 1 --2- 172.21.15.119:0/2913855789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02b00cee90 0x7f02b00b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.676+0000 7f02b5a00640 1 --2- 172.21.15.119:0/2913855789 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02b0071200 0x7f02b0138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.676+0000 7f02b5a00640 1 -- 172.21.15.119:0/2913855789 >> 172.21.15.119:0/2913855789 conn(0x7f02b006aba0 msgr2=0x7f02b0071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:21.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.676+0000 7f02b5a00640 1 -- 172.21.15.119:0/2913855789 shutdown_connections 2024-04-25T10:08:21.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.676+0000 7f02b5a00640 1 -- 172.21.15.119:0/2913855789 wait complete. 2024-04-25T10:08:21.698 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:08:21.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.884+0000 7fd83bc00640 1 Processor -- start 2024-04-25T10:08:21.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.884+0000 7fd83bc00640 1 -- start start 2024-04-25T10:08:21.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.886+0000 7fd83bc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8340ae9a0 0x7fd8340b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.886+0000 7fd83bc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd8340b2c60 0x7fd8340b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.886+0000 7fd83bc00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83406d550 con 0x7fd834071190 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.886+0000 7fd83bc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83406d830 con 0x7fd8340b2c60 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.886+0000 7fd83bc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83406db10 con 0x7fd8340ae9a0 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.886+0000 7fd839200640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd834071190 0x7fd834071570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41790/0 (socket says 172.21.15.119:41790) 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.886+0000 7fd839200640 1 -- 172.21.15.119:0/2216697094 learned_addr learned my addr 172.21.15.119:0/2216697094 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.886+0000 7fd833400640 1 -- 172.21.15.119:0/2216697094 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400777842 0 0) 0x7fd83406d550 con 0x7fd834071190 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.886+0000 7fd833400640 1 -- 172.21.15.119:0/2216697094 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd81c003520 con 0x7fd834071190 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833e00640 1 --2- 172.21.15.119:0/2216697094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8340ae9a0 0x7fd8340b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd839c00640 1 --2- 172.21.15.119:0/2216697094 >> v2:172.21.15.132:3300/0 conn(0x7fd8340b2c60 0x7fd8340b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833400640 1 -- 172.21.15.119:0/2216697094 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2950270917 0 0) 0x7fd81c003520 con 0x7fd834071190 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833400640 1 -- 172.21.15.119:0/2216697094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8340ae9a0 msgr2=0x7fd8340b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833400640 1 --2- 172.21.15.119:0/2216697094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8340ae9a0 0x7fd8340b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833400640 1 -- 172.21.15.119:0/2216697094 >> v2:172.21.15.132:3300/0 conn(0x7fd8340b2c60 msgr2=0x7fd8340b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833400640 1 --2- 172.21.15.119:0/2216697094 >> v2:172.21.15.132:3300/0 conn(0x7fd8340b2c60 0x7fd8340b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833400640 1 -- 172.21.15.119:0/2216697094 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83406ddf0 con 0x7fd834071190 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833400640 1 -- 172.21.15.119:0/2216697094 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fd828001670 con 0x7fd834071190 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833400640 1 -- 172.21.15.119:0/2216697094 >> v1:172.21.15.119:6789/0 conn(0x7fd834071190 legacy=0x7fd834071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833400640 1 --2- 172.21.15.119:0/2216697094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd81c0039b0 0x7fd81c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833400640 1 --2- 172.21.15.119:0/2216697094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd81c004800 0x7fd81c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833400640 1 --2- 172.21.15.119:0/2216697094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8340ae9a0 0x7fd81c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833400640 1 -- 172.21.15.119:0/2216697094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83406d830 con 0x7fd81c004800 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833400640 1 -- 172.21.15.119:0/2216697094 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83406db10 con 0x7fd81c0039b0 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833400640 1 -- 172.21.15.119:0/2216697094 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c00baf0 con 0x7fd8340ae9a0 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.887+0000 7fd833e00640 1 --2- 172.21.15.119:0/2216697094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd81c004800 0x7fd81c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.888+0000 7fd839c00640 1 --2- 172.21.15.119:0/2216697094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8340ae9a0 0x7fd81c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.888+0000 7fd833e00640 1 -- 172.21.15.119:0/2216697094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8340ae9a0 msgr2=0x7fd81c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.888+0000 7fd833e00640 1 --2- 172.21.15.119:0/2216697094 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8340ae9a0 0x7fd81c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.888+0000 7fd833e00640 1 -- 172.21.15.119:0/2216697094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd81c0039b0 msgr2=0x7fd81c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.888+0000 7fd833e00640 1 --2- 172.21.15.119:0/2216697094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd81c0039b0 0x7fd81c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.888+0000 7fd833e00640 1 -- 172.21.15.119:0/2216697094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c003520 con 0x7fd81c004800 2024-04-25T10:08:21.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.889+0000 7fd833e00640 1 --2- 172.21.15.119:0/2216697094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd81c004800 0x7fd81c004c80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd82000dd10 tx=0x7fd81c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1607eb286ba00b75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.889+0000 7fd833400640 1 -- 172.21.15.119:0/2216697094 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820033070 con 0x7fd81c004800 2024-04-25T10:08:21.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.889+0000 7fd833400640 1 -- 172.21.15.119:0/2216697094 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82002d900 con 0x7fd81c004800 2024-04-25T10:08:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.889+0000 7fd833400640 1 -- 172.21.15.119:0/2216697094 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82000d910 con 0x7fd81c004800 2024-04-25T10:08:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.890+0000 7fd83bc00640 1 -- 172.21.15.119:0/2216697094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd81c004800 msgr2=0x7fd81c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.890+0000 7fd83bc00640 1 --2- 172.21.15.119:0/2216697094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd81c004800 0x7fd81c004c80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd82000dd10 tx=0x7fd81c005830 comp rx=0 tx=0).stop 2024-04-25T10:08:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.890+0000 7fd839c00640 1 -- 172.21.15.119:0/2216697094 reap_dead start 2024-04-25T10:08:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.890+0000 7fd83bc00640 1 -- 172.21.15.119:0/2216697094 shutdown_connections 2024-04-25T10:08:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.890+0000 7fd83bc00640 1 -- 172.21.15.119:0/2216697094 >> 172.21.15.119:0/2216697094 conn(0x7fd83406aba0 msgr2=0x7fd83406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.890+0000 7fd83bc00640 1 -- 172.21.15.119:0/2216697094 shutdown_connections 2024-04-25T10:08:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.890+0000 7fd83bc00640 1 -- 172.21.15.119:0/2216697094 wait complete. 2024-04-25T10:08:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.892+0000 7fd83bc00640 1 Processor -- start 2024-04-25T10:08:21.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.892+0000 7fd83bc00640 1 -- start start 2024-04-25T10:08:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.893+0000 7fd83bc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd81c004800 0x7fd834143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.893+0000 7fd83bc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd83414c5a0 0x7fd83415ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.893+0000 7fd83bc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd83414d010 0x7fd83414d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.893+0000 7fd83bc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83406e8b0 con 0x7fd83414d010 2024-04-25T10:08:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.893+0000 7fd83bc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8340aee00 con 0x7fd83414c5a0 2024-04-25T10:08:21.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.893+0000 7fd83bc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83406d1b0 con 0x7fd81c004800 2024-04-25T10:08:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.893+0000 7fd839c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd83414d010 0x7fd83414d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.893+0000 7fd839c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd83414d010 0x7fd83414d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56824/0 (socket says 172.21.15.119:56824) 2024-04-25T10:08:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.893+0000 7fd839c00640 1 -- 172.21.15.119:0/2930479027 learned_addr learned my addr 172.21.15.119:0/2930479027 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.893+0000 7fd839200640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd81c004800 0x7fd834143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.893+0000 7fd833e00640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd83414c5a0 0x7fd83415ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.894+0000 7fd839c00640 1 -- 172.21.15.119:0/2930479027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd81c004800 msgr2=0x7fd834143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.894+0000 7fd839c00640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd81c004800 0x7fd834143df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.894+0000 7fd839c00640 1 -- 172.21.15.119:0/2930479027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd83414c5a0 msgr2=0x7fd83415ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.894+0000 7fd839c00640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd83414c5a0 0x7fd83415ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.894+0000 7fd839c00640 1 -- 172.21.15.119:0/2930479027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd834153080 con 0x7fd83414d010 2024-04-25T10:08:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.894+0000 7fd833e00640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd83414c5a0 0x7fd83415ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:21.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.894+0000 7fd839c00640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd83414d010 0x7fd83414d490 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd834071e20 tx=0x7fd82c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.894+0000 7fd830c00640 1 -- 172.21.15.119:0/2930479027 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c019070 con 0x7fd83414d010 2024-04-25T10:08:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.894+0000 7fd830c00640 1 -- 172.21.15.119:0/2930479027 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c007ce0 con 0x7fd83414d010 2024-04-25T10:08:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.894+0000 7fd830c00640 1 -- 172.21.15.119:0/2930479027 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c014640 con 0x7fd83414d010 2024-04-25T10:08:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.894+0000 7fd83bc00640 1 -- 172.21.15.119:0/2930479027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd834153360 con 0x7fd83414d010 2024-04-25T10:08:21.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.894+0000 7fd83bc00640 1 -- 172.21.15.119:0/2930479027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83415d380 con 0x7fd83414d010 2024-04-25T10:08:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.896+0000 7fd83bc00640 1 -- 172.21.15.119:0/2930479027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd834152e10 con 0x7fd83414d010 2024-04-25T10:08:21.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.897+0000 7fd830c00640 1 -- 172.21.15.119:0/2930479027 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd82c01c070 con 0x7fd83414d010 2024-04-25T10:08:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.898+0000 7fd830c00640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd81406b500 0x7fd81406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.898+0000 7fd830c00640 1 -- 172.21.15.119:0/2930479027 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd82c089d70 con 0x7fd83414d010 2024-04-25T10:08:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.898+0000 7fd839200640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd81406b500 0x7fd81406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.899+0000 7fd839200640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd81406b500 0x7fd81406d9c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd828005a70 tx=0x7fd8280016e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:21.903+0000 7fd830c00640 1 -- 172.21.15.119:0/2930479027 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd82c058b50 con 0x7fd83414d010 2024-04-25T10:08:22.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:22.135+0000 7fd83bc00640 1 -- 172.21.15.119:0/2930479027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd83414d9d0 con 0x7fd83414d010 2024-04-25T10:08:22.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:22.136+0000 7fd830c00640 1 -- 172.21.15.119:0/2930479027 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd82c058450 con 0x7fd83414d010 2024-04-25T10:08:22.142 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:08:22.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:22.138+0000 7fd83bc00640 1 -- 172.21.15.119:0/2930479027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd81406b500 msgr2=0x7fd81406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:22.138+0000 7fd83bc00640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd81406b500 0x7fd81406d9c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd828005a70 tx=0x7fd8280016e0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:22.138+0000 7fd83bc00640 1 -- 172.21.15.119:0/2930479027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd83414d010 msgr2=0x7fd83414d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:22.138+0000 7fd83bc00640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd83414d010 0x7fd83414d490 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd834071e20 tx=0x7fd82c009da0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:22.138+0000 7fd83bc00640 1 -- 172.21.15.119:0/2930479027 shutdown_connections 2024-04-25T10:08:22.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:22.138+0000 7fd83bc00640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd81406b500 0x7fd81406d9c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:22.138+0000 7fd83bc00640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd83414d010 0x7fd83414d490 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:22.138+0000 7fd83bc00640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd83414c5a0 0x7fd83415ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:22.138+0000 7fd83bc00640 1 --2- 172.21.15.119:0/2930479027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd81c004800 0x7fd834143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:22.138+0000 7fd83bc00640 1 -- 172.21.15.119:0/2930479027 >> 172.21.15.119:0/2930479027 conn(0x7fd83406aba0 msgr2=0x7fd83414e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:22.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:22.139+0000 7fd83bc00640 1 -- 172.21.15.119:0/2930479027 shutdown_connections 2024-04-25T10:08:22.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:22.139+0000 7fd83bc00640 1 -- 172.21.15.119:0/2930479027 wait complete. 2024-04-25T10:08:22.162 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:08:22.162 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:08:22.163 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:08:22.163 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:08:22.163 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:22.163 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:08:22.163 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:08:22.163 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:08:22.163 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:22.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.163 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:08:22.206 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:08:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:08:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:08:22.206 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:08:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:22.207 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:08:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:08:22.207 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:08:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:22.207 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:08:22.207 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:08:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:22.208 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:08:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:08:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42587sec preferred_lft 42587sec 2024-04-25T10:08:22.208 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:08:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:22.209 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:08:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:08:22.209 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:08:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:22.210 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:08:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:08:22.210 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:22.211 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:08:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:08:22.211 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:22.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.212 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:08:22.323 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:08:22.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.327 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:08:22.379 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:08:22.379 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:08:22.379 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:08:22.379 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:08:22.379 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:08:22.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.380 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:08:22.380 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:08:22.380 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:08:22.380 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:08:22.380 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:08:22.380 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:08:22.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.531 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:08:22.579 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:08:22.579 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:08:22.579 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:22.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.579 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:08:22.650 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:08:22.650 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:22.699 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:22.700 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:08:22.700 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:22.748 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:08:22.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.748 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:22.872 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:22.872 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:08:22.873 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:08:22.893 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:22.893 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:08:22.893 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:08:22.894 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:08:22.894 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:08:22.918 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:22.918 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:08:22.919 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:08:22.919 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:08:22.919 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:08:22.919 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:08:22.919 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:08:22.919 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:08:22.920 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:08:22.921 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:08:22.921 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:08:22.921 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:08:22.921 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:08:22.925 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:08:22.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.925 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:22.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:22.998 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:08:23.069 INFO:teuthology.orchestra.run.smithi119.stdout:client5244 2024-04-25T10:08:23.069 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:08:23.069 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:23.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:23.122 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-25T10:08:23.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:23.181 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-25T10:08:23.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:23.236 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-25T10:08:23.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:23.295 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-25T10:08:23.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:23.357 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-25T10:08:23.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:23.410 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:08:23.466 INFO:teuthology.orchestra.run.smithi119.stdout:./ 2024-04-25T10:08:23.467 INFO:teuthology.orchestra.run.smithi119.stdout:./rfile 2024-04-25T10:08:23.467 INFO:teuthology.orchestra.run.smithi119.stdout:./parentdir 2024-04-25T10:08:23.467 INFO:teuthology.orchestra.run.smithi119.stdout:./parentdir/pfile 2024-04-25T10:08:23.467 INFO:teuthology.orchestra.run.smithi119.stdout:./parentdir/childdir 2024-04-25T10:08:23.467 INFO:teuthology.orchestra.run.smithi119.stdout:./parentdir/childdir/cfile 2024-04-25T10:08:23.469 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:08:23.469 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:08:23.469 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:08:23.469 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:08:23.469 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:08:23.469 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:08:23.596 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:08:23.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:08:23.609 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:08:23.609 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:08:23.610 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:08:23.610 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:08:23.610 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-25T10:08:23.610 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:08:23.794 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627778 2024-04-25T10:08:23.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:08:23.808 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:08:23.808 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:08:23.808 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:08:23.808 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:08:23.808 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-25T10:08:23.808 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:08:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627776 2024-04-25T10:08:23.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:08:24.004 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:08:24.004 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:08:24.004 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:08:24.004 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:08:24.005 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-25T10:08:24.005 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:08:24.185 INFO:teuthology.orchestra.run.smithi119.stdout:1099511628279 2024-04-25T10:08:24.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:08:24.197 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:08:24.198 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:08:24.198 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:08:24.198 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:08:24.198 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-25T10:08:24.198 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:08:24.382 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627777 2024-04-25T10:08:24.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:08:24.395 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:08:24.395 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:08:24.395 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:08:24.395 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:08:24.395 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-25T10:08:24.395 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:08:24.579 INFO:teuthology.orchestra.run.smithi119.stdout:1099511628280 2024-04-25T10:08:24.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:08:24.593 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:08:24.593 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:24.691 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:08:24.691 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:24.741 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:24.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.984+0000 7f38e3e00640 1 Processor -- start 2024-04-25T10:08:24.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.984+0000 7f38e3e00640 1 -- start start 2024-04-25T10:08:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.986+0000 7f38e3e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f38e4147270 0x7f38e40b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.986+0000 7f38e3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38e415ea60 0x7f38e40acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.986+0000 7f38e3e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e40af000 con 0x7f38e415dee0 2024-04-25T10:08:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.986+0000 7f38e3e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e40b4a80 con 0x7f38e4147270 2024-04-25T10:08:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.986+0000 7f38e3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40b4d60 con 0x7f38e415ea60 2024-04-25T10:08:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.986+0000 7f38e2a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f38e4147270 0x7f38e40b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.986+0000 7f38e2a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f38e4147270 0x7f38e40b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56832/0 (socket says 172.21.15.119:56832) 2024-04-25T10:08:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.986+0000 7f38e2a00640 1 -- 172.21.15.119:0/2304123367 learned_addr learned my addr 172.21.15.119:0/2304123367 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.986+0000 7f38e2a00640 1 -- 172.21.15.119:0/2304123367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38e415ea60 msgr2=0x7f38e40acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.986+0000 7f38e2a00640 1 --2- 172.21.15.119:0/2304123367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38e415ea60 0x7f38e40acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.986+0000 7f38e2a00640 1 -- 172.21.15.119:0/2304123367 >> v1:172.21.15.132:6789/0 conn(0x7f38e415dee0 legacy=0x7f38e415e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.986+0000 7f38e2a00640 1 -- 172.21.15.119:0/2304123367 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e40b5040 con 0x7f38e4147270 2024-04-25T10:08:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.988+0000 7f38e2a00640 1 --2- 172.21.15.119:0/2304123367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38e4147270 0x7f38e40b4410 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009870 tx=0x7f38d402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8744b597457b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.988+0000 7f38e2000640 1 -- 172.21.15.119:0/2304123367 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4037070 con 0x7f38e4147270 2024-04-25T10:08:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.988+0000 7f38e2000640 1 -- 172.21.15.119:0/2304123367 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d4032440 con 0x7f38e4147270 2024-04-25T10:08:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.988+0000 7f38e2000640 1 -- 172.21.15.119:0/2304123367 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4032810 con 0x7f38e4147270 2024-04-25T10:08:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.988+0000 7f38e3e00640 1 -- 172.21.15.119:0/2304123367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38e4147270 msgr2=0x7f38e40b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.988+0000 7f38e3e00640 1 --2- 172.21.15.119:0/2304123367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38e4147270 0x7f38e40b4410 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009870 tx=0x7f38d402a180 comp rx=0 tx=0).stop 2024-04-25T10:08:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.989+0000 7f38e3e00640 1 -- 172.21.15.119:0/2304123367 shutdown_connections 2024-04-25T10:08:24.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.989+0000 7f38e3e00640 1 --2- 172.21.15.119:0/2304123367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38e415ea60 0x7f38e40acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.989+0000 7f38e3e00640 1 --2- 172.21.15.119:0/2304123367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38e4147270 0x7f38e40b4410 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.989+0000 7f38e3e00640 1 -- 172.21.15.119:0/2304123367 >> 172.21.15.119:0/2304123367 conn(0x7f38e406b1d0 msgr2=0x7f38e40b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.989+0000 7f38e3e00640 1 -- 172.21.15.119:0/2304123367 shutdown_connections 2024-04-25T10:08:24.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.989+0000 7f38e3e00640 1 -- 172.21.15.119:0/2304123367 wait complete. 2024-04-25T10:08:24.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.990+0000 7f38e3e00640 1 Processor -- start 2024-04-25T10:08:24.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.991+0000 7f38e3e00640 1 -- start start 2024-04-25T10:08:24.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.991+0000 7f38e3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38e4147270 0x7f38e413cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.991+0000 7f38e3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38e415dee0 0x7f38e41e7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.991+0000 7f38e3e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38e415ea60 0x7f38e41e7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.992+0000 7f38e3e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40ad4c0 con 0x7f38e4147270 2024-04-25T10:08:24.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.992+0000 7f38e3e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40b5600 con 0x7f38e415ea60 2024-04-25T10:08:24.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.992+0000 7f38e3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40ad640 con 0x7f38e415dee0 2024-04-25T10:08:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.992+0000 7f38e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38e4147270 0x7f38e413cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.992+0000 7f38e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38e4147270 0x7f38e413cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56834/0 (socket says 172.21.15.119:56834) 2024-04-25T10:08:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.992+0000 7f38e2a00640 1 -- 172.21.15.119:0/3221674006 learned_addr learned my addr 172.21.15.119:0/3221674006 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.992+0000 7f38da000640 1 --2- 172.21.15.119:0/3221674006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38e415dee0 0x7f38e41e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.992+0000 7f38e2a00640 1 -- 172.21.15.119:0/3221674006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38e415dee0 msgr2=0x7f38e41e7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.992+0000 7f38e2a00640 1 --2- 172.21.15.119:0/3221674006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38e415dee0 0x7f38e41e7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.992+0000 7f38e3400640 1 --2- 172.21.15.119:0/3221674006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38e415ea60 0x7f38e41e7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.992+0000 7f38e2a00640 1 -- 172.21.15.119:0/3221674006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38e415ea60 msgr2=0x7f38e41e7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.992+0000 7f38e2a00640 1 --2- 172.21.15.119:0/3221674006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38e415ea60 0x7f38e41e7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.992+0000 7f38e2a00640 1 -- 172.21.15.119:0/3221674006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e413e8e0 con 0x7f38e4147270 2024-04-25T10:08:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.993+0000 7f38e2a00640 1 --2- 172.21.15.119:0/3221674006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38e4147270 0x7f38e413cf40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009870 tx=0x7f38d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.993+0000 7f38db400640 1 -- 172.21.15.119:0/3221674006 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4041070 con 0x7f38e4147270 2024-04-25T10:08:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.993+0000 7f38db400640 1 -- 172.21.15.119:0/3221674006 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d403c870 con 0x7f38e4147270 2024-04-25T10:08:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.993+0000 7f38db400640 1 -- 172.21.15.119:0/3221674006 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4037040 con 0x7f38e4147270 2024-04-25T10:08:25.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.993+0000 7f38e3e00640 1 -- 172.21.15.119:0/3221674006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e41e4f50 con 0x7f38e4147270 2024-04-25T10:08:25.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.993+0000 7f38e3e00640 1 -- 172.21.15.119:0/3221674006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e41e57f0 con 0x7f38e4147270 2024-04-25T10:08:25.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.994+0000 7f38e3e00640 1 -- 172.21.15.119:0/3221674006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e40749b0 con 0x7f38e4147270 2024-04-25T10:08:25.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.995+0000 7f38db400640 1 -- 172.21.15.119:0/3221674006 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38d4044070 con 0x7f38e4147270 2024-04-25T10:08:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.996+0000 7f38db400640 1 --2- 172.21.15.119:0/3221674006 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f38c006b4d0 0x7f38c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.996+0000 7f38db400640 1 -- 172.21.15.119:0/3221674006 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38d40aade0 con 0x7f38e4147270 2024-04-25T10:08:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.996+0000 7f38da000640 1 --2- 172.21.15.119:0/3221674006 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f38c006b4d0 0x7f38c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:24.997+0000 7f38da000640 1 --2- 172.21.15.119:0/3221674006 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f38c006b4d0 0x7f38c006d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f38e41f7120 tx=0x7f38cc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.001+0000 7f38db400640 1 -- 172.21.15.119:0/3221674006 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38d4079bc0 con 0x7f38e4147270 2024-04-25T10:08:25.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.197+0000 7f38e3e00640 1 -- 172.21.15.119:0/3221674006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38e40b2180 con 0x7f38e4147270 2024-04-25T10:08:25.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.200+0000 7f38db400640 1 -- 172.21.15.119:0/3221674006 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16705 (secure 0 0 0) 0x7f38d40794c0 con 0x7f38e4147270 2024-04-25T10:08:25.207 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:25.207 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":33,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:16.095956+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:12.268457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:13.546259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:25.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.203+0000 7f38e3e00640 1 -- 172.21.15.119:0/3221674006 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f38c006b4d0 msgr2=0x7f38c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.203+0000 7f38e3e00640 1 --2- 172.21.15.119:0/3221674006 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f38c006b4d0 0x7f38c006d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f38e41f7120 tx=0x7f38cc009040 comp rx=0 tx=0).stop 2024-04-25T10:08:25.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.203+0000 7f38e3e00640 1 -- 172.21.15.119:0/3221674006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38e4147270 msgr2=0x7f38e413cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.203+0000 7f38e3e00640 1 --2- 172.21.15.119:0/3221674006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38e4147270 0x7f38e413cf40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009870 tx=0x7f38d40046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.203+0000 7f38e3e00640 1 -- 172.21.15.119:0/3221674006 shutdown_connections 2024-04-25T10:08:25.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.203+0000 7f38e3e00640 1 --2- 172.21.15.119:0/3221674006 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f38c006b4d0 0x7f38c006d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.203+0000 7f38e3e00640 1 --2- 172.21.15.119:0/3221674006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38e415ea60 0x7f38e41e7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.203+0000 7f38e3e00640 1 --2- 172.21.15.119:0/3221674006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38e415dee0 0x7f38e41e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.203+0000 7f38e3e00640 1 --2- 172.21.15.119:0/3221674006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f38e4147270 0x7f38e413cf40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.203+0000 7f38e3e00640 1 -- 172.21.15.119:0/3221674006 >> 172.21.15.119:0/3221674006 conn(0x7f38e406b1d0 msgr2=0x7f38e406d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:25.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.203+0000 7f38e3e00640 1 -- 172.21.15.119:0/3221674006 shutdown_connections 2024-04-25T10:08:25.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.203+0000 7f38e3e00640 1 -- 172.21.15.119:0/3221674006 wait complete. 2024-04-25T10:08:25.229 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:12.268457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:13.546259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:25.230 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:25.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.431+0000 7fba5ea00640 1 Processor -- start 2024-04-25T10:08:25.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.431+0000 7fba5ea00640 1 -- start start 2024-04-25T10:08:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.432+0000 7fba5ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba5806b2b0 0x7fba580cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.432+0000 7fba5ea00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5815f400 con 0x7fba580cc810 2024-04-25T10:08:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.432+0000 7fba5ea00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5815f6e0 con 0x7fba581450e0 2024-04-25T10:08:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.432+0000 7fba5ea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba581454c0 con 0x7fba5806b2b0 2024-04-25T10:08:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.433+0000 7fba5e000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fba580cc810 0x7fba580b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41806/0 (socket says 172.21.15.119:41806) 2024-04-25T10:08:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.433+0000 7fba5e000640 1 -- 172.21.15.119:0/760233992 learned_addr learned my addr 172.21.15.119:0/760233992 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.433+0000 7fba5cc00640 1 --2- 172.21.15.119:0/760233992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba5806b2b0 0x7fba580cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.433+0000 7fba57e00640 1 -- 172.21.15.119:0/760233992 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759118652 0 0) 0x7fba5815f400 con 0x7fba580cc810 2024-04-25T10:08:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.433+0000 7fba57e00640 1 -- 172.21.15.119:0/760233992 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba40003520 con 0x7fba580cc810 2024-04-25T10:08:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.434+0000 7fba57e00640 1 -- 172.21.15.119:0/760233992 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1611581936 0 0) 0x7fba40003520 con 0x7fba580cc810 2024-04-25T10:08:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.434+0000 7fba57e00640 1 -- 172.21.15.119:0/760233992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba5806b2b0 msgr2=0x7fba580cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.434+0000 7fba57e00640 1 --2- 172.21.15.119:0/760233992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba5806b2b0 0x7fba580cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.434+0000 7fba57e00640 1 -- 172.21.15.119:0/760233992 >> v1:172.21.15.132:6789/0 conn(0x7fba581450e0 legacy=0x7fba5806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.434+0000 7fba57e00640 1 -- 172.21.15.119:0/760233992 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5815f9c0 con 0x7fba580cc810 2024-04-25T10:08:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.434+0000 7fba57e00640 1 -- 172.21.15.119:0/760233992 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fba500019f0 con 0x7fba580cc810 2024-04-25T10:08:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.434+0000 7fba57e00640 1 -- 172.21.15.119:0/760233992 >> v1:172.21.15.119:6789/0 conn(0x7fba580cc810 legacy=0x7fba580b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.434+0000 7fba57e00640 1 --2- 172.21.15.119:0/760233992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba400039e0 0x7fba40003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.434+0000 7fba57e00640 1 --2- 172.21.15.119:0/760233992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba40004800 0x7fba40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.434+0000 7fba57e00640 1 --2- 172.21.15.119:0/760233992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba5806b2b0 0x7fba4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.434+0000 7fba57e00640 1 -- 172.21.15.119:0/760233992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba400054f0 con 0x7fba400039e0 2024-04-25T10:08:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.434+0000 7fba57e00640 1 -- 172.21.15.119:0/760233992 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4000bf00 con 0x7fba40004800 2024-04-25T10:08:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.434+0000 7fba57e00640 1 -- 172.21.15.119:0/760233992 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4000c080 con 0x7fba5806b2b0 2024-04-25T10:08:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.434+0000 7fba5d600640 1 --2- 172.21.15.119:0/760233992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba400039e0 0x7fba40003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.435+0000 7fba5cc00640 1 --2- 172.21.15.119:0/760233992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba40004800 0x7fba40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.435+0000 7fba5e000640 1 --2- 172.21.15.119:0/760233992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba5806b2b0 0x7fba4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.435+0000 7fba5d600640 1 -- 172.21.15.119:0/760233992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba5806b2b0 msgr2=0x7fba4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.435+0000 7fba5d600640 1 --2- 172.21.15.119:0/760233992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba5806b2b0 0x7fba4000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.435+0000 7fba5d600640 1 -- 172.21.15.119:0/760233992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba40004800 msgr2=0x7fba40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:25.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.435+0000 7fba5d600640 1 --2- 172.21.15.119:0/760233992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba40004800 0x7fba40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.435+0000 7fba5d600640 1 -- 172.21.15.119:0/760233992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba40003520 con 0x7fba400039e0 2024-04-25T10:08:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.436+0000 7fba5d600640 1 --2- 172.21.15.119:0/760233992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba400039e0 0x7fba40003e00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fba4c001750 tx=0x7fba40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a97b75a72b18f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.437+0000 7fba57e00640 1 -- 172.21.15.119:0/760233992 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c001370 con 0x7fba400039e0 2024-04-25T10:08:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.437+0000 7fba57e00640 1 -- 172.21.15.119:0/760233992 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c0052d0 con 0x7fba400039e0 2024-04-25T10:08:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.437+0000 7fba57e00640 1 -- 172.21.15.119:0/760233992 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c02abe0 con 0x7fba400039e0 2024-04-25T10:08:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.437+0000 7fba5ea00640 1 -- 172.21.15.119:0/760233992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba400039e0 msgr2=0x7fba40003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.437+0000 7fba5ea00640 1 --2- 172.21.15.119:0/760233992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba400039e0 0x7fba40003e00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fba4c001750 tx=0x7fba40005830 comp rx=0 tx=0).stop 2024-04-25T10:08:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.437+0000 7fba5e000640 1 -- 172.21.15.119:0/760233992 reap_dead start 2024-04-25T10:08:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.437+0000 7fba5ea00640 1 -- 172.21.15.119:0/760233992 shutdown_connections 2024-04-25T10:08:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.437+0000 7fba5ea00640 1 -- 172.21.15.119:0/760233992 >> 172.21.15.119:0/760233992 conn(0x7fba58066ff0 msgr2=0x7fba580b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.437+0000 7fba5ea00640 1 -- 172.21.15.119:0/760233992 shutdown_connections 2024-04-25T10:08:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.437+0000 7fba5ea00640 1 -- 172.21.15.119:0/760233992 wait complete. 2024-04-25T10:08:25.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.439+0000 7fba5ea00640 1 Processor -- start 2024-04-25T10:08:25.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.439+0000 7fba5ea00640 1 -- start start 2024-04-25T10:08:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.440+0000 7fba5ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba5815b4e0 0x7fba5813df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.440+0000 7fba5ea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba5813da70 0x7fba5815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.440+0000 7fba5ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba5815e7f0 0x7fba5815c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.440+0000 7fba5ea00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba580cce00 con 0x7fba5815b4e0 2024-04-25T10:08:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.440+0000 7fba5ea00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba580ccc80 con 0x7fba5813da70 2024-04-25T10:08:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.440+0000 7fba5ea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba580cc2d0 con 0x7fba5815e7f0 2024-04-25T10:08:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.440+0000 7fba5d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba5815b4e0 0x7fba5813df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.440+0000 7fba5d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba5815b4e0 0x7fba5813df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56852/0 (socket says 172.21.15.119:56852) 2024-04-25T10:08:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.440+0000 7fba5d600640 1 -- 172.21.15.119:0/2957069489 learned_addr learned my addr 172.21.15.119:0/2957069489 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.441+0000 7fba5e000640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba5815e7f0 0x7fba5815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.441+0000 7fba5cc00640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba5813da70 0x7fba5815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.441+0000 7fba5d600640 1 -- 172.21.15.119:0/2957069489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba5815e7f0 msgr2=0x7fba5815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.441+0000 7fba5d600640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba5815e7f0 0x7fba5815c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.441+0000 7fba5d600640 1 -- 172.21.15.119:0/2957069489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba5813da70 msgr2=0x7fba5815e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.441+0000 7fba5d600640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba5813da70 0x7fba5815e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.441+0000 7fba5d600640 1 -- 172.21.15.119:0/2957069489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba58143860 con 0x7fba5815b4e0 2024-04-25T10:08:25.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.441+0000 7fba5cc00640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba5813da70 0x7fba5815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:25.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.441+0000 7fba5d600640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba5815b4e0 0x7fba5813df60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fba4c02a510 tx=0x7fba4c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.442+0000 7fba55600640 1 -- 172.21.15.119:0/2957069489 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c007a30 con 0x7fba5815b4e0 2024-04-25T10:08:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.442+0000 7fba55600640 1 -- 172.21.15.119:0/2957069489 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c040070 con 0x7fba5815b4e0 2024-04-25T10:08:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.442+0000 7fba55600640 1 -- 172.21.15.119:0/2957069489 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c005480 con 0x7fba5815b4e0 2024-04-25T10:08:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.442+0000 7fba5ea00640 1 -- 172.21.15.119:0/2957069489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba581427c0 con 0x7fba5815b4e0 2024-04-25T10:08:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.442+0000 7fba5ea00640 1 -- 172.21.15.119:0/2957069489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba58142d80 con 0x7fba5815b4e0 2024-04-25T10:08:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.443+0000 7fba5ea00640 1 -- 172.21.15.119:0/2957069489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba580749b0 con 0x7fba5815b4e0 2024-04-25T10:08:25.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.444+0000 7fba55600640 1 -- 172.21.15.119:0/2957069489 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba4c03a070 con 0x7fba5815b4e0 2024-04-25T10:08:25.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.445+0000 7fba55600640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fba3806b4d0 0x7fba3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.445+0000 7fba55600640 1 -- 172.21.15.119:0/2957069489 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba4c0a9be0 con 0x7fba5815b4e0 2024-04-25T10:08:25.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.445+0000 7fba5cc00640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fba3806b4d0 0x7fba3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.446+0000 7fba5cc00640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fba3806b4d0 0x7fba3806d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fba5813da20 tx=0x7fba44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.449+0000 7fba55600640 1 -- 172.21.15.119:0/2957069489 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba4c0789c0 con 0x7fba5815b4e0 2024-04-25T10:08:25.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.642+0000 7fba5ea00640 1 -- 172.21.15.119:0/2957069489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba5815c250 con 0x7fba5815b4e0 2024-04-25T10:08:25.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.643+0000 7fba55600640 1 -- 172.21.15.119:0/2957069489 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba4c0782c0 con 0x7fba5815b4e0 2024-04-25T10:08:25.650 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:08:25.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.645+0000 7fba5ea00640 1 -- 172.21.15.119:0/2957069489 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fba3806b4d0 msgr2=0x7fba3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.645+0000 7fba5ea00640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fba3806b4d0 0x7fba3806d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fba5813da20 tx=0x7fba44008040 comp rx=0 tx=0).stop 2024-04-25T10:08:25.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.645+0000 7fba5ea00640 1 -- 172.21.15.119:0/2957069489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba5815b4e0 msgr2=0x7fba5813df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.645+0000 7fba5ea00640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba5815b4e0 0x7fba5813df60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fba4c02a510 tx=0x7fba4c001120 comp rx=0 tx=0).stop 2024-04-25T10:08:25.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.645+0000 7fba5ea00640 1 -- 172.21.15.119:0/2957069489 shutdown_connections 2024-04-25T10:08:25.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.645+0000 7fba5ea00640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fba3806b4d0 0x7fba3806d990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.645+0000 7fba5ea00640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba5815e7f0 0x7fba5815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.645+0000 7fba5ea00640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba5813da70 0x7fba5815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.645+0000 7fba5ea00640 1 --2- 172.21.15.119:0/2957069489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba5815b4e0 0x7fba5813df60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.645+0000 7fba5ea00640 1 -- 172.21.15.119:0/2957069489 >> 172.21.15.119:0/2957069489 conn(0x7fba58066ff0 msgr2=0x7fba580ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:25.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.646+0000 7fba5ea00640 1 -- 172.21.15.119:0/2957069489 shutdown_connections 2024-04-25T10:08:25.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.646+0000 7fba5ea00640 1 -- 172.21.15.119:0/2957069489 wait complete. 2024-04-25T10:08:25.671 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:08:25.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.867+0000 7f4b36a00640 1 Processor -- start 2024-04-25T10:08:25.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.868+0000 7f4b36a00640 1 -- start start 2024-04-25T10:08:25.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.869+0000 7f4b36a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b300aed40 0x7f4b300cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.869+0000 7f4b36a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3015e360 con 0x7f4b300cfa90 2024-04-25T10:08:25.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.869+0000 7f4b36a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3015e5d0 con 0x7f4b3015b720 2024-04-25T10:08:25.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.869+0000 7f4b36a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3015e8b0 con 0x7f4b300aed40 2024-04-25T10:08:25.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.870+0000 7f4b34a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4b300cfa90 0x7f4b300acfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41818/0 (socket says 172.21.15.119:41818) 2024-04-25T10:08:25.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.870+0000 7f4b34a00640 1 -- 172.21.15.119:0/816253918 learned_addr learned my addr 172.21.15.119:0/816253918 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:25.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.870+0000 7f4b27400640 1 --2- 172.21.15.119:0/816253918 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b300aed40 0x7f4b300cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.870+0000 7f4b2f400640 1 -- 172.21.15.119:0/816253918 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3748083831 0 0) 0x7f4b3015e360 con 0x7f4b300cfa90 2024-04-25T10:08:25.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.870+0000 7f4b2f400640 1 -- 172.21.15.119:0/816253918 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b14003520 con 0x7f4b300cfa90 2024-04-25T10:08:25.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.871+0000 7f4b27400640 1 -- 172.21.15.119:0/816253918 >> v1:172.21.15.132:6789/0 conn(0x7f4b3015b720 legacy=0x7f4b300ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.871+0000 7f4b27400640 1 -- 172.21.15.119:0/816253918 >> v1:172.21.15.119:6789/0 conn(0x7f4b300cfa90 legacy=0x7f4b300acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.871+0000 7f4b27400640 1 -- 172.21.15.119:0/816253918 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3015eb90 con 0x7f4b300aed40 2024-04-25T10:08:25.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.872+0000 7f4b27400640 1 --2- 172.21.15.119:0/816253918 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b300aed40 0x7f4b300cf550 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4b180097d0 tx=0x7f4b1802a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb3b39e3cdc1e9af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.872+0000 7f4b2f400640 1 -- 172.21.15.119:0/816253918 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18037070 con 0x7f4b300aed40 2024-04-25T10:08:25.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.873+0000 7f4b2f400640 1 -- 172.21.15.119:0/816253918 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b18032440 con 0x7f4b300aed40 2024-04-25T10:08:25.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.873+0000 7f4b2f400640 1 -- 172.21.15.119:0/816253918 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18032810 con 0x7f4b300aed40 2024-04-25T10:08:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.873+0000 7f4b36a00640 1 -- 172.21.15.119:0/816253918 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b300aed40 msgr2=0x7f4b300cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.873+0000 7f4b36a00640 1 --2- 172.21.15.119:0/816253918 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b300aed40 0x7f4b300cf550 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4b180097d0 tx=0x7f4b1802a200 comp rx=0 tx=0).stop 2024-04-25T10:08:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.873+0000 7f4b36a00640 1 -- 172.21.15.119:0/816253918 shutdown_connections 2024-04-25T10:08:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.873+0000 7f4b36a00640 1 --2- 172.21.15.119:0/816253918 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b300aed40 0x7f4b300cf550 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.873+0000 7f4b36a00640 1 -- 172.21.15.119:0/816253918 >> 172.21.15.119:0/816253918 conn(0x7f4b3006aba0 msgr2=0x7f4b3006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.873+0000 7f4b36a00640 1 -- 172.21.15.119:0/816253918 shutdown_connections 2024-04-25T10:08:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.873+0000 7f4b36a00640 1 -- 172.21.15.119:0/816253918 wait complete. 2024-04-25T10:08:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.875+0000 7f4b36a00640 1 Processor -- start 2024-04-25T10:08:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.875+0000 7f4b36a00640 1 -- start start 2024-04-25T10:08:25.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.876+0000 7f4b36a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b300aed40 0x7f4b301518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.876+0000 7f4b36a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b300cfa90 0x7f4b3014c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.876+0000 7f4b36a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b3015b720 0x7f4b3014cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.876+0000 7f4b36a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300cff00 con 0x7f4b3015b720 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.876+0000 7f4b36a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3015bb00 con 0x7f4b300aed40 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.876+0000 7f4b36a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300d0080 con 0x7f4b300cfa90 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.877+0000 7f4b34a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b3015b720 0x7f4b3014cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.877+0000 7f4b34a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b3015b720 0x7f4b3014cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56854/0 (socket says 172.21.15.119:56854) 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.877+0000 7f4b34a00640 1 -- 172.21.15.119:0/2560095978 learned_addr learned my addr 172.21.15.119:0/2560095978 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.877+0000 7f4b2fe00640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b300aed40 0x7f4b301518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.877+0000 7f4b27400640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b300cfa90 0x7f4b3014c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.877+0000 7f4b34a00640 1 -- 172.21.15.119:0/2560095978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b300cfa90 msgr2=0x7f4b3014c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.877+0000 7f4b34a00640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b300cfa90 0x7f4b3014c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.877+0000 7f4b34a00640 1 -- 172.21.15.119:0/2560095978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b300aed40 msgr2=0x7f4b301518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.877+0000 7f4b34a00640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b300aed40 0x7f4b301518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.877+0000 7f4b34a00640 1 -- 172.21.15.119:0/2560095978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3014d5b0 con 0x7f4b3015b720 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.877+0000 7f4b27400640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b300cfa90 0x7f4b3014c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.878+0000 7f4b34a00640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b3015b720 0x7f4b3014cf30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800cb70 tx=0x7f4b2800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.878+0000 7f4b2cc00640 1 -- 172.21.15.119:0/2560095978 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b28017070 con 0x7f4b3015b720 2024-04-25T10:08:25.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.878+0000 7f4b2cc00640 1 -- 172.21.15.119:0/2560095978 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2800ad40 con 0x7f4b3015b720 2024-04-25T10:08:25.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.878+0000 7f4b36a00640 1 -- 172.21.15.119:0/2560095978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3014d890 con 0x7f4b3015b720 2024-04-25T10:08:25.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.878+0000 7f4b2cc00640 1 -- 172.21.15.119:0/2560095978 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b28012620 con 0x7f4b3015b720 2024-04-25T10:08:25.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.878+0000 7f4b36a00640 1 -- 172.21.15.119:0/2560095978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3014de50 con 0x7f4b3015b720 2024-04-25T10:08:25.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.879+0000 7f4b36a00640 1 -- 172.21.15.119:0/2560095978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b300750b0 con 0x7f4b3015b720 2024-04-25T10:08:25.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.880+0000 7f4b2cc00640 1 -- 172.21.15.119:0/2560095978 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b2801a070 con 0x7f4b3015b720 2024-04-25T10:08:25.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.881+0000 7f4b2cc00640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b0c06b4d0 0x7f4b0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.881+0000 7f4b2cc00640 1 -- 172.21.15.119:0/2560095978 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b2808a180 con 0x7f4b3015b720 2024-04-25T10:08:25.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.881+0000 7f4b2fe00640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b0c06b4d0 0x7f4b0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.882+0000 7f4b2fe00640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b0c06b4d0 0x7f4b0c06d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4b3015cfe0 tx=0x7f4b20009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:25.885+0000 7f4b2cc00640 1 -- 172.21.15.119:0/2560095978 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b28058f60 con 0x7f4b3015b720 2024-04-25T10:08:26.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.080+0000 7f4b36a00640 1 -- 172.21.15.119:0/2560095978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4b300b3dc0 con 0x7f4b3015b720 2024-04-25T10:08:26.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.081+0000 7f4b2cc00640 1 -- 172.21.15.119:0/2560095978 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4b28058860 con 0x7f4b3015b720 2024-04-25T10:08:26.087 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:08:26.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.084+0000 7f4b36a00640 1 -- 172.21.15.119:0/2560095978 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b0c06b4d0 msgr2=0x7f4b0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.084+0000 7f4b36a00640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b0c06b4d0 0x7f4b0c06d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4b3015cfe0 tx=0x7f4b20009040 comp rx=0 tx=0).stop 2024-04-25T10:08:26.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.084+0000 7f4b36a00640 1 -- 172.21.15.119:0/2560095978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b3015b720 msgr2=0x7f4b3014cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.084+0000 7f4b36a00640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b3015b720 0x7f4b3014cf30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800cb70 tx=0x7f4b2800a430 comp rx=0 tx=0).stop 2024-04-25T10:08:26.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.084+0000 7f4b36a00640 1 -- 172.21.15.119:0/2560095978 shutdown_connections 2024-04-25T10:08:26.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.084+0000 7f4b36a00640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b0c06b4d0 0x7f4b0c06d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.084+0000 7f4b36a00640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b3015b720 0x7f4b3014cf30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.084+0000 7f4b36a00640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b300cfa90 0x7f4b3014c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.084+0000 7f4b36a00640 1 --2- 172.21.15.119:0/2560095978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b300aed40 0x7f4b301518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.084+0000 7f4b36a00640 1 -- 172.21.15.119:0/2560095978 >> 172.21.15.119:0/2560095978 conn(0x7f4b3006aba0 msgr2=0x7f4b3015bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:26.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.084+0000 7f4b36a00640 1 -- 172.21.15.119:0/2560095978 shutdown_connections 2024-04-25T10:08:26.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.084+0000 7f4b36a00640 1 -- 172.21.15.119:0/2560095978 wait complete. 2024-04-25T10:08:26.109 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:08:26.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.305+0000 7f5e40e00640 1 Processor -- start 2024-04-25T10:08:26.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.305+0000 7f5e40e00640 1 -- start start 2024-04-25T10:08:26.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.306+0000 7f5e40e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5e3c144e70 0x7f5e3c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.306+0000 7f5e40e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e3c0cf0e0 0x7f5e3c0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.306+0000 7f5e40e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c06dcb0 con 0x7f5e3c06d7a0 2024-04-25T10:08:26.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.307+0000 7f5e40e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c15bfe0 con 0x7f5e3c0cf0e0 2024-04-25T10:08:26.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.307+0000 7f5e40e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c06df90 con 0x7f5e3c144e70 2024-04-25T10:08:26.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.307+0000 7f5e3a800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e3c0cf0e0 0x7f5e3c0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.307+0000 7f5e3a800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e3c0cf0e0 0x7f5e3c0cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56856/0 (socket says 172.21.15.119:56856) 2024-04-25T10:08:26.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.307+0000 7f5e3a800640 1 -- 172.21.15.119:0/218026704 learned_addr learned my addr 172.21.15.119:0/218026704 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:26.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.307+0000 7f5e39e00640 1 --2- 172.21.15.119:0/218026704 >> v2:172.21.15.132:3300/0 conn(0x7f5e3c144e70 0x7f5e3c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.307+0000 7f5e3a800640 1 -- 172.21.15.119:0/218026704 >> v2:172.21.15.132:3300/0 conn(0x7f5e3c144e70 msgr2=0x7f5e3c06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.307+0000 7f5e3a800640 1 --2- 172.21.15.119:0/218026704 >> v2:172.21.15.132:3300/0 conn(0x7f5e3c144e70 0x7f5e3c06d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.307+0000 7f5e3a800640 1 -- 172.21.15.119:0/218026704 >> v1:172.21.15.132:6789/0 conn(0x7f5e3c06d7a0 legacy=0x7f5e3c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.307+0000 7f5e3a800640 1 -- 172.21.15.119:0/218026704 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c06e270 con 0x7f5e3c0cf0e0 2024-04-25T10:08:26.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.308+0000 7f5e3a800640 1 --2- 172.21.15.119:0/218026704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e3c0cf0e0 0x7f5e3c0cc300 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0b4a20 tx=0x7f5e3402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20091b1aeb810951 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.309+0000 7f5e38a00640 1 -- 172.21.15.119:0/218026704 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34037070 con 0x7f5e3c0cf0e0 2024-04-25T10:08:26.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.309+0000 7f5e38a00640 1 -- 172.21.15.119:0/218026704 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e34032440 con 0x7f5e3c0cf0e0 2024-04-25T10:08:26.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.309+0000 7f5e38a00640 1 -- 172.21.15.119:0/218026704 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34032810 con 0x7f5e3c0cf0e0 2024-04-25T10:08:26.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.309+0000 7f5e40e00640 1 -- 172.21.15.119:0/218026704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e3c0cf0e0 msgr2=0x7f5e3c0cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.309+0000 7f5e40e00640 1 --2- 172.21.15.119:0/218026704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e3c0cf0e0 0x7f5e3c0cc300 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0b4a20 tx=0x7f5e3402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.310+0000 7f5e40e00640 1 -- 172.21.15.119:0/218026704 shutdown_connections 2024-04-25T10:08:26.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.310+0000 7f5e40e00640 1 --2- 172.21.15.119:0/218026704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e3c0cf0e0 0x7f5e3c0cc300 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.310+0000 7f5e40e00640 1 --2- 172.21.15.119:0/218026704 >> v2:172.21.15.132:3300/0 conn(0x7f5e3c144e70 0x7f5e3c06d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.310+0000 7f5e40e00640 1 -- 172.21.15.119:0/218026704 >> 172.21.15.119:0/218026704 conn(0x7f5e3c06aba0 msgr2=0x7f5e3c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:26.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.310+0000 7f5e40e00640 1 -- 172.21.15.119:0/218026704 shutdown_connections 2024-04-25T10:08:26.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.310+0000 7f5e40e00640 1 -- 172.21.15.119:0/218026704 wait complete. 2024-04-25T10:08:26.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.311+0000 7f5e40e00640 1 Processor -- start 2024-04-25T10:08:26.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.312+0000 7f5e40e00640 1 -- start start 2024-04-25T10:08:26.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.312+0000 7f5e40e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e3c06d7a0 0x7f5e3c0b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e40e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e3c0cf0e0 0x7f5e3c1e6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e40e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e3c144e70 0x7f5e3c13e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e40e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c145630 con 0x7f5e3c06d7a0 2024-04-25T10:08:26.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e40e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c1454b0 con 0x7f5e3c144e70 2024-04-25T10:08:26.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e40e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0cc840 con 0x7f5e3c0cf0e0 2024-04-25T10:08:26.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e39e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e3c06d7a0 0x7f5e3c0b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e39e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e3c06d7a0 0x7f5e3c0b0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56868/0 (socket says 172.21.15.119:56868) 2024-04-25T10:08:26.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e39e00640 1 -- 172.21.15.119:0/2507049334 learned_addr learned my addr 172.21.15.119:0/2507049334 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:26.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e39400640 1 --2- 172.21.15.119:0/2507049334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e3c0cf0e0 0x7f5e3c1e6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e3a800640 1 --2- 172.21.15.119:0/2507049334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e3c144e70 0x7f5e3c13e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e39e00640 1 -- 172.21.15.119:0/2507049334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e3c0cf0e0 msgr2=0x7f5e3c1e6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e39e00640 1 --2- 172.21.15.119:0/2507049334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e3c0cf0e0 0x7f5e3c1e6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e39e00640 1 -- 172.21.15.119:0/2507049334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e3c144e70 msgr2=0x7f5e3c13e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e39e00640 1 --2- 172.21.15.119:0/2507049334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e3c144e70 0x7f5e3c13e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.313+0000 7f5e39e00640 1 -- 172.21.15.119:0/2507049334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c0b1160 con 0x7f5e3c06d7a0 2024-04-25T10:08:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.314+0000 7f5e39e00640 1 --2- 172.21.15.119:0/2507049334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e3c06d7a0 0x7f5e3c0b0830 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00d5f0 tx=0x7f5e2c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.314+0000 7f5e32000640 1 -- 172.21.15.119:0/2507049334 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c0125f0 con 0x7f5e3c06d7a0 2024-04-25T10:08:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.314+0000 7f5e32000640 1 -- 172.21.15.119:0/2507049334 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c019070 con 0x7f5e3c06d7a0 2024-04-25T10:08:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.314+0000 7f5e32000640 1 -- 172.21.15.119:0/2507049334 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c0129d0 con 0x7f5e3c06d7a0 2024-04-25T10:08:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.314+0000 7f5e40e00640 1 -- 172.21.15.119:0/2507049334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c0b1440 con 0x7f5e3c06d7a0 2024-04-25T10:08:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.314+0000 7f5e40e00640 1 -- 172.21.15.119:0/2507049334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c0a9780 con 0x7f5e3c06d7a0 2024-04-25T10:08:26.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.315+0000 7f5e40e00640 1 -- 172.21.15.119:0/2507049334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c0b1610 con 0x7f5e3c06d7a0 2024-04-25T10:08:26.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.316+0000 7f5e32000640 1 -- 172.21.15.119:0/2507049334 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e2c012b70 con 0x7f5e3c06d7a0 2024-04-25T10:08:26.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.317+0000 7f5e32000640 1 --2- 172.21.15.119:0/2507049334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e1c06b500 0x7f5e1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.317+0000 7f5e32000640 1 -- 172.21.15.119:0/2507049334 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e2c08a530 con 0x7f5e3c06d7a0 2024-04-25T10:08:26.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.318+0000 7f5e39400640 1 --2- 172.21.15.119:0/2507049334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e1c06b500 0x7f5e1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.318+0000 7f5e39400640 1 --2- 172.21.15.119:0/2507049334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e1c06b500 0x7f5e1c06d9c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c1e4750 tx=0x7f5e28009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.322+0000 7f5e32000640 1 -- 172.21.15.119:0/2507049334 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e2c059440 con 0x7f5e3c06d7a0 2024-04-25T10:08:26.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.516+0000 7f5e40e00640 1 -- 172.21.15.119:0/2507049334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e3c0750b0 con 0x7f5e3c06d7a0 2024-04-25T10:08:26.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.516+0000 7f5e32000640 1 -- 172.21.15.119:0/2507049334 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e2c058d40 con 0x7f5e3c06d7a0 2024-04-25T10:08:26.523 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:08:26.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.519+0000 7f5e40e00640 1 -- 172.21.15.119:0/2507049334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e1c06b500 msgr2=0x7f5e1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.519+0000 7f5e40e00640 1 --2- 172.21.15.119:0/2507049334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e1c06b500 0x7f5e1c06d9c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c1e4750 tx=0x7f5e28009040 comp rx=0 tx=0).stop 2024-04-25T10:08:26.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.519+0000 7f5e40e00640 1 -- 172.21.15.119:0/2507049334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e3c06d7a0 msgr2=0x7f5e3c0b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.519+0000 7f5e40e00640 1 --2- 172.21.15.119:0/2507049334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e3c06d7a0 0x7f5e3c0b0830 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00d5f0 tx=0x7f5e2c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.519+0000 7f5e40e00640 1 -- 172.21.15.119:0/2507049334 shutdown_connections 2024-04-25T10:08:26.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.519+0000 7f5e40e00640 1 --2- 172.21.15.119:0/2507049334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e1c06b500 0x7f5e1c06d9c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.519+0000 7f5e40e00640 1 --2- 172.21.15.119:0/2507049334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e3c144e70 0x7f5e3c13e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.519+0000 7f5e40e00640 1 --2- 172.21.15.119:0/2507049334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e3c0cf0e0 0x7f5e3c1e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.519+0000 7f5e40e00640 1 --2- 172.21.15.119:0/2507049334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e3c06d7a0 0x7f5e3c0b0830 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.519+0000 7f5e40e00640 1 -- 172.21.15.119:0/2507049334 >> 172.21.15.119:0/2507049334 conn(0x7f5e3c06aba0 msgr2=0x7f5e3c0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:26.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.519+0000 7f5e40e00640 1 -- 172.21.15.119:0/2507049334 shutdown_connections 2024-04-25T10:08:26.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.519+0000 7f5e40e00640 1 -- 172.21.15.119:0/2507049334 wait complete. 2024-04-25T10:08:26.545 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:08:26.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.739+0000 7f6965c00640 1 Processor -- start 2024-04-25T10:08:26.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.739+0000 7f6965c00640 1 -- start start 2024-04-25T10:08:26.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.740+0000 7f6965c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69600aedc0 0x7f696015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.740+0000 7f6965c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69600b4900 0x7f696015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.740+0000 7f6965c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f696006da40 con 0x7f696006d260 2024-04-25T10:08:26.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.740+0000 7f6965c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696015c050 con 0x7f69600aedc0 2024-04-25T10:08:26.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.740+0000 7f6965c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69600af1a0 con 0x7f69600b4900 2024-04-25T10:08:26.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.740+0000 7f695ea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69600aedc0 0x7f696015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.740+0000 7f695ea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69600aedc0 0x7f696015ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56870/0 (socket says 172.21.15.119:56870) 2024-04-25T10:08:26.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.740+0000 7f695ea00640 1 -- 172.21.15.119:0/1964152234 learned_addr learned my addr 172.21.15.119:0/1964152234 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:26.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.741+0000 7f695f400640 1 --2- 172.21.15.119:0/1964152234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69600b4900 0x7f696015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.741+0000 7f695ea00640 1 -- 172.21.15.119:0/1964152234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69600b4900 msgr2=0x7f696015c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.741+0000 7f695ea00640 1 --2- 172.21.15.119:0/1964152234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69600b4900 0x7f696015c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.741+0000 7f695ea00640 1 -- 172.21.15.119:0/1964152234 >> v1:172.21.15.132:6789/0 conn(0x7f696006d260 legacy=0x7f69600b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.741+0000 7f695ea00640 1 -- 172.21.15.119:0/1964152234 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696006dd20 con 0x7f69600aedc0 2024-04-25T10:08:26.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.741+0000 7f695f400640 1 --2- 172.21.15.119:0/1964152234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69600b4900 0x7f696015c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:26.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.742+0000 7f695ea00640 1 --2- 172.21.15.119:0/1964152234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69600aedc0 0x7f696015ae20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f695000bb10 tx=0x7f695002c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36405c739dea8ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.742+0000 7f695e000640 1 -- 172.21.15.119:0/1964152234 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695003a070 con 0x7f69600aedc0 2024-04-25T10:08:26.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.742+0000 7f695e000640 1 -- 172.21.15.119:0/1964152234 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69500326f0 con 0x7f69600aedc0 2024-04-25T10:08:26.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.742+0000 7f695e000640 1 -- 172.21.15.119:0/1964152234 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950032ac0 con 0x7f69600aedc0 2024-04-25T10:08:26.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.743+0000 7f6965c00640 1 -- 172.21.15.119:0/1964152234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69600aedc0 msgr2=0x7f696015ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.743+0000 7f6965c00640 1 --2- 172.21.15.119:0/1964152234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69600aedc0 0x7f696015ae20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f695000bb10 tx=0x7f695002c830 comp rx=0 tx=0).stop 2024-04-25T10:08:26.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.743+0000 7f6965c00640 1 -- 172.21.15.119:0/1964152234 shutdown_connections 2024-04-25T10:08:26.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.743+0000 7f6965c00640 1 --2- 172.21.15.119:0/1964152234 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69600b4900 0x7f696015c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.743+0000 7f6965c00640 1 --2- 172.21.15.119:0/1964152234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69600aedc0 0x7f696015ae20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.743+0000 7f6965c00640 1 -- 172.21.15.119:0/1964152234 >> 172.21.15.119:0/1964152234 conn(0x7f696006aba0 msgr2=0x7f696006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:26.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.743+0000 7f6965c00640 1 -- 172.21.15.119:0/1964152234 shutdown_connections 2024-04-25T10:08:26.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.743+0000 7f6965c00640 1 -- 172.21.15.119:0/1964152234 wait complete. 2024-04-25T10:08:26.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.745+0000 7f6965c00640 1 Processor -- start 2024-04-25T10:08:26.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.745+0000 7f6965c00640 1 -- start start 2024-04-25T10:08:26.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.746+0000 7f6965c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f696006d260 0x7f69600cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.746+0000 7f6965c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600aedc0 0x7f69600b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.746+0000 7f6965c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69600b4900 0x7f69600cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.746+0000 7f6965c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696006d6f0 con 0x7f69600b4900 2024-04-25T10:08:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.746+0000 7f6965c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696015c2a0 con 0x7f69600aedc0 2024-04-25T10:08:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.746+0000 7f6965c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696015cd00 con 0x7f696006d260 2024-04-25T10:08:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.746+0000 7f695f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69600b4900 0x7f69600cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.746+0000 7f695f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69600b4900 0x7f69600cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56876/0 (socket says 172.21.15.119:56876) 2024-04-25T10:08:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.746+0000 7f695f400640 1 -- 172.21.15.119:0/215722582 learned_addr learned my addr 172.21.15.119:0/215722582 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.747+0000 7f695ea00640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f696006d260 0x7f69600cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.747+0000 7f6957e00640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600aedc0 0x7f69600b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.747+0000 7f695f400640 1 -- 172.21.15.119:0/215722582 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f696006d260 msgr2=0x7f69600cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.747+0000 7f695f400640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f696006d260 0x7f69600cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.747+0000 7f695f400640 1 -- 172.21.15.119:0/215722582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600aedc0 msgr2=0x7f69600b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.747+0000 7f695f400640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600aedc0 0x7f69600b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.747+0000 7f695f400640 1 -- 172.21.15.119:0/215722582 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69600b2320 con 0x7f69600b4900 2024-04-25T10:08:26.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.747+0000 7f6957e00640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600aedc0 0x7f69600b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.747+0000 7f695f400640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69600b4900 0x7f69600cda70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f695800cd50 tx=0x7f695800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.747+0000 7f6956a00640 1 -- 172.21.15.119:0/215722582 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695800f680 con 0x7f69600b4900 2024-04-25T10:08:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.747+0000 7f6956a00640 1 -- 172.21.15.119:0/215722582 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695800f840 con 0x7f69600b4900 2024-04-25T10:08:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.747+0000 7f6956a00640 1 -- 172.21.15.119:0/215722582 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695800fc30 con 0x7f69600b4900 2024-04-25T10:08:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.747+0000 7f6965c00640 1 -- 172.21.15.119:0/215722582 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69600b25a0 con 0x7f69600b4900 2024-04-25T10:08:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.748+0000 7f6965c00640 1 -- 172.21.15.119:0/215722582 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69600b2b60 con 0x7f69600b4900 2024-04-25T10:08:26.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.749+0000 7f6965c00640 1 -- 172.21.15.119:0/215722582 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69600750b0 con 0x7f69600b4900 2024-04-25T10:08:26.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.750+0000 7f6956a00640 1 -- 172.21.15.119:0/215722582 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69580213f0 con 0x7f69600b4900 2024-04-25T10:08:26.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.751+0000 7f6956a00640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f693c06b4d0 0x7f693c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.751+0000 7f6956a00640 1 -- 172.21.15.119:0/215722582 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f695808afe0 con 0x7f69600b4900 2024-04-25T10:08:26.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.751+0000 7f695ea00640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f693c06b4d0 0x7f693c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.752+0000 7f695ea00640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f693c06b4d0 0x7f693c06d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6950037000 tx=0x7f6950008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.755+0000 7f6956a00640 1 -- 172.21.15.119:0/215722582 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69600b2320 con 0x7f69600b4900 2024-04-25T10:08:26.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.963+0000 7f6965c00640 1 -- 172.21.15.119:0/215722582 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f696015add0 con 0x7f69600b4900 2024-04-25T10:08:26.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.964+0000 7f6956a00640 1 -- 172.21.15.119:0/215722582 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6958014030 con 0x7f69600b4900 2024-04-25T10:08:26.970 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2024-04-25T10:08:26.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.966+0000 7f6965c00640 1 -- 172.21.15.119:0/215722582 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f693c06b4d0 msgr2=0x7f693c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.966+0000 7f6965c00640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f693c06b4d0 0x7f693c06d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6950037000 tx=0x7f6950008040 comp rx=0 tx=0).stop 2024-04-25T10:08:26.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.966+0000 7f6965c00640 1 -- 172.21.15.119:0/215722582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69600b4900 msgr2=0x7f69600cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.966+0000 7f6965c00640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69600b4900 0x7f69600cda70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f695800cd50 tx=0x7f695800a650 comp rx=0 tx=0).stop 2024-04-25T10:08:26.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.967+0000 7f6965c00640 1 -- 172.21.15.119:0/215722582 shutdown_connections 2024-04-25T10:08:26.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.967+0000 7f6965c00640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f693c06b4d0 0x7f693c06d990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.967+0000 7f6965c00640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69600b4900 0x7f69600cda70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.967+0000 7f6965c00640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69600aedc0 0x7f69600b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.967+0000 7f6965c00640 1 --2- 172.21.15.119:0/215722582 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f696006d260 0x7f69600cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.967+0000 7f6965c00640 1 -- 172.21.15.119:0/215722582 >> 172.21.15.119:0/215722582 conn(0x7f696006aba0 msgr2=0x7f69600ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:26.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.967+0000 7f6965c00640 1 -- 172.21.15.119:0/215722582 shutdown_connections 2024-04-25T10:08:26.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:26.967+0000 7f6965c00640 1 -- 172.21.15.119:0/215722582 wait complete. 2024-04-25T10:08:26.992 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:08:26.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:26.992 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:27.114 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:08:27.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:27.114 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:08:27.176 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:27.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.340+0000 7f96ba200640 1 Processor -- start 2024-04-25T10:08:27.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.340+0000 7f96ba200640 1 -- start start 2024-04-25T10:08:27.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.342+0000 7f96ba200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f96b415bf00 0x7f96b40cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:27.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.342+0000 7f96ba200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96b40cc310 0x7f96b415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:27.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.342+0000 7f96ba200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96b415c820 0x7f96b406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:27.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.342+0000 7f96ba200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b41455e0 con 0x7f96b415c820 2024-04-25T10:08:27.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.342+0000 7f96ba200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b406d450 con 0x7f96b415bf00 2024-04-25T10:08:27.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.342+0000 7f96ba200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40ce6c0 con 0x7f96b40cc310 2024-04-25T10:08:27.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.342+0000 7f96b3e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96b415c820 0x7f96b406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:27.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.342+0000 7f96b3e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96b415c820 0x7f96b406d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56892/0 (socket says 172.21.15.119:56892) 2024-04-25T10:08:27.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.342+0000 7f96b3e00640 1 -- 172.21.15.119:0/2464117509 learned_addr learned my addr 172.21.15.119:0/2464117509 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:27.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.342+0000 7f96b3400640 1 --2- 172.21.15.119:0/2464117509 >> v2:172.21.15.132:3300/0 conn(0x7f96b415bf00 0x7f96b40cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:27.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.343+0000 7f96aaa00640 1 --2- 172.21.15.119:0/2464117509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96b40cc310 0x7f96b415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.343+0000 7f96b3e00640 1 -- 172.21.15.119:0/2464117509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96b40cc310 msgr2=0x7f96b415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.343+0000 7f96b3e00640 1 --2- 172.21.15.119:0/2464117509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96b40cc310 0x7f96b415c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.343+0000 7f96b3e00640 1 -- 172.21.15.119:0/2464117509 >> v2:172.21.15.132:3300/0 conn(0x7f96b415bf00 msgr2=0x7f96b40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.343+0000 7f96b3e00640 1 --2- 172.21.15.119:0/2464117509 >> v2:172.21.15.132:3300/0 conn(0x7f96b415bf00 0x7f96b40cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.343+0000 7f96b3e00640 1 -- 172.21.15.119:0/2464117509 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b40cc7b0 con 0x7f96b415c820 2024-04-25T10:08:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.344+0000 7f96b3e00640 1 --2- 172.21.15.119:0/2464117509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b415c820 0x7f96b406d8e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f96b406ba10 tx=0x7f96ac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc83449cea4d1d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:27.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.344+0000 7f96b2a00640 1 -- 172.21.15.119:0/2464117509 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac036070 con 0x7f96b415c820 2024-04-25T10:08:27.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.344+0000 7f96b2a00640 1 -- 172.21.15.119:0/2464117509 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac031600 con 0x7f96b415c820 2024-04-25T10:08:27.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.344+0000 7f96b2a00640 1 -- 172.21.15.119:0/2464117509 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac0319d0 con 0x7f96b415c820 2024-04-25T10:08:27.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.345+0000 7f96ba200640 1 -- 172.21.15.119:0/2464117509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b415c820 msgr2=0x7f96b406d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:27.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.345+0000 7f96ba200640 1 --2- 172.21.15.119:0/2464117509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b415c820 0x7f96b406d8e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f96b406ba10 tx=0x7f96ac02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:08:27.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.345+0000 7f96ba200640 1 -- 172.21.15.119:0/2464117509 shutdown_connections 2024-04-25T10:08:27.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.345+0000 7f96ba200640 1 --2- 172.21.15.119:0/2464117509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b415c820 0x7f96b406d8e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:27.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.345+0000 7f96ba200640 1 --2- 172.21.15.119:0/2464117509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96b40cc310 0x7f96b415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:27.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.345+0000 7f96ba200640 1 --2- 172.21.15.119:0/2464117509 >> v2:172.21.15.132:3300/0 conn(0x7f96b415bf00 0x7f96b40cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:27.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.345+0000 7f96ba200640 1 -- 172.21.15.119:0/2464117509 >> 172.21.15.119:0/2464117509 conn(0x7f96b406aba0 msgr2=0x7f96b406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:27.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.345+0000 7f96ba200640 1 -- 172.21.15.119:0/2464117509 shutdown_connections 2024-04-25T10:08:27.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.345+0000 7f96ba200640 1 -- 172.21.15.119:0/2464117509 wait complete. 2024-04-25T10:08:27.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.347+0000 7f96ba200640 1 Processor -- start 2024-04-25T10:08:27.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.347+0000 7f96ba200640 1 -- start start 2024-04-25T10:08:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.348+0000 7f96ba200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96b40cc310 0x7f96b415a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.348+0000 7f96ba200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96b415bf00 0x7f96b41f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.348+0000 7f96ba200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b415c820 0x7f96b41f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.348+0000 7f96ba200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40cd050 con 0x7f96b415c820 2024-04-25T10:08:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.348+0000 7f96ba200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40ccd70 con 0x7f96b415bf00 2024-04-25T10:08:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.348+0000 7f96ba200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b415cca0 con 0x7f96b40cc310 2024-04-25T10:08:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.348+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b415c820 0x7f96b41f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.348+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b415c820 0x7f96b41f52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56894/0 (socket says 172.21.15.119:56894) 2024-04-25T10:08:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.349+0000 7f96b3e00640 1 -- 172.21.15.119:0/1724766627 learned_addr learned my addr 172.21.15.119:0/1724766627 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.349+0000 7f96aaa00640 1 --2- 172.21.15.119:0/1724766627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96b415bf00 0x7f96b41f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.349+0000 7f96b3e00640 1 -- 172.21.15.119:0/1724766627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96b40cc310 msgr2=0x7f96b415a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.349+0000 7f96b3e00640 1 --2- 172.21.15.119:0/1724766627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96b40cc310 0x7f96b415a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.349+0000 7f96b3e00640 1 -- 172.21.15.119:0/1724766627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96b415bf00 msgr2=0x7f96b41f4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.349+0000 7f96b3e00640 1 --2- 172.21.15.119:0/1724766627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96b415bf00 0x7f96b41f4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.349+0000 7f96b3e00640 1 -- 172.21.15.119:0/1724766627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b413c960 con 0x7f96b415c820 2024-04-25T10:08:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.349+0000 7f96aaa00640 1 --2- 172.21.15.119:0/1724766627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96b415bf00 0x7f96b41f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.349+0000 7f96b3e00640 1 --2- 172.21.15.119:0/1724766627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b415c820 0x7f96b41f52e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009f40 tx=0x7f96ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.349+0000 7f96abe00640 1 -- 172.21.15.119:0/1724766627 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac036050 con 0x7f96b415c820 2024-04-25T10:08:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.350+0000 7f96abe00640 1 -- 172.21.15.119:0/1724766627 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac042070 con 0x7f96b415c820 2024-04-25T10:08:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.350+0000 7f96abe00640 1 -- 172.21.15.119:0/1724766627 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac009510 con 0x7f96b415c820 2024-04-25T10:08:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.350+0000 7f96ba200640 1 -- 172.21.15.119:0/1724766627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b4138cf0 con 0x7f96b415c820 2024-04-25T10:08:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.350+0000 7f96ba200640 1 -- 172.21.15.119:0/1724766627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b41392b0 con 0x7f96b415c820 2024-04-25T10:08:27.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.351+0000 7f96ba200640 1 -- 172.21.15.119:0/1724766627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b406bb30 con 0x7f96b415c820 2024-04-25T10:08:27.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.352+0000 7f96abe00640 1 -- 172.21.15.119:0/1724766627 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96ac007bd0 con 0x7f96b415c820 2024-04-25T10:08:27.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.353+0000 7f96abe00640 1 --2- 172.21.15.119:0/1724766627 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f969006b4d0 0x7f969006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:27.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.353+0000 7f96abe00640 1 -- 172.21.15.119:0/1724766627 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96ac0ab160 con 0x7f96b415c820 2024-04-25T10:08:27.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.353+0000 7f96b3400640 1 --2- 172.21.15.119:0/1724766627 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f969006b4d0 0x7f969006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:27.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.353+0000 7f96b3400640 1 --2- 172.21.15.119:0/1724766627 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f969006b4d0 0x7f969006d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f96b41f4d00 tx=0x7f96a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:27.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.357+0000 7f96abe00640 1 -- 172.21.15.119:0/1724766627 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96ac079fd0 con 0x7f96b415c820 2024-04-25T10:08:27.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.585+0000 7f96ba200640 1 -- 172.21.15.119:0/1724766627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96b40749b0 con 0x7f96b415c820 2024-04-25T10:08:27.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.586+0000 7f96abe00640 1 -- 172.21.15.119:0/1724766627 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f96ac0798d0 con 0x7f96b415c820 2024-04-25T10:08:27.592 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:27.592 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1554372269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1554372269},{"type":"v1","addr":"172.21.15.119:6837","nonce":1554372269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4699,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4250304148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4250304148},{"type":"v1","addr":"172.21.15.132:6834","nonce":4250304148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/637229574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":637229574},{"type":"v1","addr":"172.21.15.132:6835","nonce":637229574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:14.553437+0000","modified":"2024-04-25T10:08:17.586270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6836/3585315292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3585315292},{"type":"v1","addr":"172.21.15.119:6836","nonce":3585315292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:08:27.592 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 16 2024-04-25T10:08:27.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.589+0000 7f96ba200640 1 -- 172.21.15.119:0/1724766627 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f969006b4d0 msgr2=0x7f969006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:27.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.589+0000 7f96ba200640 1 --2- 172.21.15.119:0/1724766627 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f969006b4d0 0x7f969006d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f96b41f4d00 tx=0x7f96a4008040 comp rx=0 tx=0).stop 2024-04-25T10:08:27.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.589+0000 7f96ba200640 1 -- 172.21.15.119:0/1724766627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b415c820 msgr2=0x7f96b41f52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:27.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.589+0000 7f96ba200640 1 --2- 172.21.15.119:0/1724766627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b415c820 0x7f96b41f52e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009f40 tx=0x7f96ac004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:27.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.589+0000 7f96ba200640 1 -- 172.21.15.119:0/1724766627 shutdown_connections 2024-04-25T10:08:27.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.589+0000 7f96ba200640 1 --2- 172.21.15.119:0/1724766627 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f969006b4d0 0x7f969006d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:27.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.589+0000 7f96ba200640 1 --2- 172.21.15.119:0/1724766627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b415c820 0x7f96b41f52e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:27.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.589+0000 7f96ba200640 1 --2- 172.21.15.119:0/1724766627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96b415bf00 0x7f96b41f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:27.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.589+0000 7f96ba200640 1 --2- 172.21.15.119:0/1724766627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96b40cc310 0x7f96b415a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:27.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.589+0000 7f96ba200640 1 -- 172.21.15.119:0/1724766627 >> 172.21.15.119:0/1724766627 conn(0x7f96b406aba0 msgr2=0x7f96b415d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:27.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.589+0000 7f96ba200640 1 -- 172.21.15.119:0/1724766627 shutdown_connections 2024-04-25T10:08:27.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:27.589+0000 7f96ba200640 1 -- 172.21.15.119:0/1724766627 wait complete. 2024-04-25T10:08:27.613 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:08:27.850 INFO:teuthology.orchestra.run.smithi119.stdout:{"message":"","return_code":0} 2024-04-25T10:08:27.850 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:08:27.850 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:28.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.036+0000 7f342d400640 1 Processor -- start 2024-04-25T10:08:28.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.036+0000 7f342d400640 1 -- start start 2024-04-25T10:08:28.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.037+0000 7f342d400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3428144470 0x7f342815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:28.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.037+0000 7f342d400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f342806e430 0x7f34280b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:28.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.037+0000 7f342d400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34280cd440 con 0x7f34280b4e80 2024-04-25T10:08:28.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.037+0000 7f342d400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3428145340 con 0x7f342806e430 2024-04-25T10:08:28.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.038+0000 7f342d400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34280cd720 con 0x7f3428144470 2024-04-25T10:08:28.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.038+0000 7f341fe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f342806e430 0x7f34280b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:28.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.038+0000 7f341fe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f342806e430 0x7f34280b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56908/0 (socket says 172.21.15.119:56908) 2024-04-25T10:08:28.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.038+0000 7f342ca00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f34280b4e80 0x7f342806d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41822/0 (socket says 172.21.15.119:41822) 2024-04-25T10:08:28.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.038+0000 7f341fe00640 1 -- 172.21.15.119:0/3954479706 learned_addr learned my addr 172.21.15.119:0/3954479706 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:28.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.038+0000 7f341fe00640 1 -- 172.21.15.119:0/3954479706 >> v2:172.21.15.132:3300/0 conn(0x7f3428144470 msgr2=0x7f342815fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:28.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.038+0000 7f3427e00640 1 --2- 172.21.15.119:0/3954479706 >> v2:172.21.15.132:3300/0 conn(0x7f3428144470 0x7f342815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:28.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.038+0000 7f341fe00640 1 --2- 172.21.15.119:0/3954479706 >> v2:172.21.15.132:3300/0 conn(0x7f3428144470 0x7f342815fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:28.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.038+0000 7f341fe00640 1 -- 172.21.15.119:0/3954479706 >> v1:172.21.15.119:6789/0 conn(0x7f34280b4e80 legacy=0x7f342806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:28.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.038+0000 7f341fe00640 1 -- 172.21.15.119:0/3954479706 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34280cda00 con 0x7f342806e430 2024-04-25T10:08:28.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.038+0000 7f3427400640 1 -- 172.21.15.119:0/3954479706 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094498124 0 0) 0x7f34280cd440 con 0x7f34280b4e80 2024-04-25T10:08:28.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.039+0000 7f341fe00640 1 --2- 172.21.15.119:0/3954479706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342806e430 0x7f34280b4940 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f34280b5490 tx=0x7f341002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f86b47d6b53614 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:28.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.040+0000 7f3427400640 1 -- 172.21.15.119:0/3954479706 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410037070 con 0x7f342806e430 2024-04-25T10:08:28.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.040+0000 7f3427400640 1 -- 172.21.15.119:0/3954479706 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3410032440 con 0x7f342806e430 2024-04-25T10:08:28.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.040+0000 7f3427400640 1 -- 172.21.15.119:0/3954479706 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410032810 con 0x7f342806e430 2024-04-25T10:08:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.040+0000 7f342d400640 1 -- 172.21.15.119:0/3954479706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342806e430 msgr2=0x7f34280b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.040+0000 7f342d400640 1 --2- 172.21.15.119:0/3954479706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342806e430 0x7f34280b4940 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f34280b5490 tx=0x7f341002a270 comp rx=0 tx=0).stop 2024-04-25T10:08:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.040+0000 7f342d400640 1 -- 172.21.15.119:0/3954479706 shutdown_connections 2024-04-25T10:08:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.040+0000 7f342d400640 1 --2- 172.21.15.119:0/3954479706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342806e430 0x7f34280b4940 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.040+0000 7f342d400640 1 --2- 172.21.15.119:0/3954479706 >> v2:172.21.15.132:3300/0 conn(0x7f3428144470 0x7f342815fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.040+0000 7f342d400640 1 -- 172.21.15.119:0/3954479706 >> 172.21.15.119:0/3954479706 conn(0x7f342806b1d0 msgr2=0x7f34280b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.040+0000 7f342d400640 1 -- 172.21.15.119:0/3954479706 shutdown_connections 2024-04-25T10:08:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.040+0000 7f342d400640 1 -- 172.21.15.119:0/3954479706 wait complete. 2024-04-25T10:08:28.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.042+0000 7f342d400640 1 Processor -- start 2024-04-25T10:08:28.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.042+0000 7f342d400640 1 -- start start 2024-04-25T10:08:28.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.043+0000 7f342d400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f342806e430 0x7f34281f1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:28.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.043+0000 7f342d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34280b4e80 0x7f342815ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:28.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.043+0000 7f342d400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3428144470 0x7f34281f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:28.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.043+0000 7f342d400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34280ce200 con 0x7f34280b4e80 2024-04-25T10:08:28.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.043+0000 7f342d400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3428144850 con 0x7f3428144470 2024-04-25T10:08:28.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.043+0000 7f342d400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342815cd00 con 0x7f342806e430 2024-04-25T10:08:28.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.043+0000 7f341fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34280b4e80 0x7f342815ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:28.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.043+0000 7f341fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34280b4e80 0x7f342815ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56910/0 (socket says 172.21.15.119:56910) 2024-04-25T10:08:28.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.043+0000 7f341fe00640 1 -- 172.21.15.119:0/737508130 learned_addr learned my addr 172.21.15.119:0/737508130 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.044+0000 7f342ca00640 1 --2- 172.21.15.119:0/737508130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3428144470 0x7f34281f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.044+0000 7f341fe00640 1 -- 172.21.15.119:0/737508130 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f342806e430 msgr2=0x7f34281f1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.044+0000 7f341fe00640 1 --2- 172.21.15.119:0/737508130 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f342806e430 0x7f34281f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.044+0000 7f341fe00640 1 -- 172.21.15.119:0/737508130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3428144470 msgr2=0x7f34281f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.044+0000 7f341fe00640 1 --2- 172.21.15.119:0/737508130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3428144470 0x7f34281f45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.044+0000 7f341fe00640 1 -- 172.21.15.119:0/737508130 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34281f50b0 con 0x7f34280b4e80 2024-04-25T10:08:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.044+0000 7f341fe00640 1 --2- 172.21.15.119:0/737508130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34280b4e80 0x7f342815ac80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f341002a2b0 tx=0x7f34100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.044+0000 7f3424c00640 1 -- 172.21.15.119:0/737508130 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410037040 con 0x7f34280b4e80 2024-04-25T10:08:28.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.044+0000 7f3424c00640 1 -- 172.21.15.119:0/737508130 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3410042070 con 0x7f34280b4e80 2024-04-25T10:08:28.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.044+0000 7f3424c00640 1 -- 172.21.15.119:0/737508130 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410030bb0 con 0x7f34280b4e80 2024-04-25T10:08:28.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.045+0000 7f342d400640 1 -- 172.21.15.119:0/737508130 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34281f5390 con 0x7f34280b4e80 2024-04-25T10:08:28.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.045+0000 7f342d400640 1 -- 172.21.15.119:0/737508130 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342813aab0 con 0x7f34280b4e80 2024-04-25T10:08:28.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.046+0000 7f341d600640 1 -- 172.21.15.119:0/737508130 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342813a5c0 con 0x7f34280b4e80 2024-04-25T10:08:28.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.047+0000 7f3424c00640 1 -- 172.21.15.119:0/737508130 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3410047460 con 0x7f34280b4e80 2024-04-25T10:08:28.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.048+0000 7f3424c00640 1 --2- 172.21.15.119:0/737508130 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f340406b4d0 0x7f340406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:28.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.048+0000 7f3424c00640 1 -- 172.21.15.119:0/737508130 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34100aaec0 con 0x7f34280b4e80 2024-04-25T10:08:28.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.048+0000 7f3427e00640 1 --2- 172.21.15.119:0/737508130 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f340406b4d0 0x7f340406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:28.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.049+0000 7f3427e00640 1 --2- 172.21.15.119:0/737508130 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f340406b4d0 0x7f340406d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f341800a810 tx=0x7f3418008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:28.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.053+0000 7f3424c00640 1 -- 172.21.15.119:0/737508130 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3410045360 con 0x7f34280b4e80 2024-04-25T10:08:28.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.287+0000 7f341d600640 1 -- 172.21.15.119:0/737508130 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34280749b0 con 0x7f34280b4e80 2024-04-25T10:08:28.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.288+0000 7f3424c00640 1 -- 172.21.15.119:0/737508130 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f3410079d30 con 0x7f34280b4e80 2024-04-25T10:08:28.294 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:28.295 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1554372269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1554372269},{"type":"v1","addr":"172.21.15.119:6837","nonce":1554372269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4699,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4250304148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4250304148},{"type":"v1","addr":"172.21.15.132:6834","nonce":4250304148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/637229574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":637229574},{"type":"v1","addr":"172.21.15.132:6835","nonce":637229574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:14.553437+0000","modified":"2024-04-25T10:08:17.586270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6836/3585315292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3585315292},{"type":"v1","addr":"172.21.15.119:6836","nonce":3585315292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:08:28.295 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 16 2024-04-25T10:08:28.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.291+0000 7f341d600640 1 -- 172.21.15.119:0/737508130 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f340406b4d0 msgr2=0x7f340406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:28.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.291+0000 7f341d600640 1 --2- 172.21.15.119:0/737508130 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f340406b4d0 0x7f340406d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f341800a810 tx=0x7f3418008040 comp rx=0 tx=0).stop 2024-04-25T10:08:28.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.291+0000 7f341d600640 1 -- 172.21.15.119:0/737508130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34280b4e80 msgr2=0x7f342815ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:28.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.291+0000 7f341d600640 1 --2- 172.21.15.119:0/737508130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34280b4e80 0x7f342815ac80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f341002a2b0 tx=0x7f34100046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:28.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.291+0000 7f341d600640 1 -- 172.21.15.119:0/737508130 shutdown_connections 2024-04-25T10:08:28.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.291+0000 7f341d600640 1 --2- 172.21.15.119:0/737508130 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f340406b4d0 0x7f340406d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:28.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.291+0000 7f341d600640 1 --2- 172.21.15.119:0/737508130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3428144470 0x7f34281f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:28.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.291+0000 7f341d600640 1 --2- 172.21.15.119:0/737508130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34280b4e80 0x7f342815ac80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:28.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.291+0000 7f341d600640 1 --2- 172.21.15.119:0/737508130 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f342806e430 0x7f34281f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:28.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.291+0000 7f341d600640 1 -- 172.21.15.119:0/737508130 >> 172.21.15.119:0/737508130 conn(0x7f342806b1d0 msgr2=0x7f34280cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:28.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.291+0000 7f341d600640 1 -- 172.21.15.119:0/737508130 shutdown_connections 2024-04-25T10:08:28.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:28.291+0000 7f341d600640 1 -- 172.21.15.119:0/737508130 wait complete. 2024-04-25T10:08:28.315 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parentdir mytag 2024-04-25T10:08:28.536 INFO:teuthology.orchestra.run.smithi119.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-25T10:08:28.537 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-25T10:08:30.331 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:30.340 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:38.538 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-25T10:08:38.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.651+0000 7ff5f4b56f40 1 Processor -- start 2024-04-25T10:08:38.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.651+0000 7ff5f4b56f40 1 -- start start 2024-04-25T10:08:38.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.651+0000 7ff5f4b56f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fab58b7220 0x55fab598fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.651+0000 7ff5f4b56f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab5902080 0x55fab59046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.651+0000 7ff5f4b56f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fab598ec70 0x55fab5985dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.651+0000 7ff5f4b56f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab58ea2f0 con 0x55fab598ec70 2024-04-25T10:08:38.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.651+0000 7ff5f4b56f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab585edd0 con 0x55fab5902080 2024-04-25T10:08:38.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.651+0000 7ff5f4b56f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fab58b7960 con 0x55fab58b7220 2024-04-25T10:08:38.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.652+0000 7ff5f2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fab598ec70 0x55fab5985dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.652+0000 7ff5f2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fab598ec70 0x55fab5985dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42554/0 (socket says 172.21.15.119:42554) 2024-04-25T10:08:38.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.652+0000 7ff5f2800640 1 -- 172.21.15.119:0/2644659066 learned_addr learned my addr 172.21.15.119:0/2644659066 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:38.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.652+0000 7ff5f2800640 1 -- 172.21.15.119:0/2644659066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fab58b7220 msgr2=0x55fab598fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.652+0000 7ff5f2800640 1 --2- 172.21.15.119:0/2644659066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fab58b7220 0x55fab598fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.652+0000 7ff5f2800640 1 -- 172.21.15.119:0/2644659066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab5902080 msgr2=0x55fab59046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.652+0000 7ff5f1400640 1 --2- 172.21.15.119:0/2644659066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab5902080 0x55fab59046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.652+0000 7ff5f2800640 1 --2- 172.21.15.119:0/2644659066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab5902080 0x55fab59046d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.652+0000 7ff5f2800640 1 -- 172.21.15.119:0/2644659066 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fab58e6100 con 0x55fab598ec70 2024-04-25T10:08:38.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.653+0000 7ff5f2800640 1 --2- 172.21.15.119:0/2644659066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fab598ec70 0x55fab5985dd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec002d90 tx=0x7ff5ec00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44babd436cd54b76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.653+0000 7ff5ebe00640 1 -- 172.21.15.119:0/2644659066 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec00f400 con 0x55fab598ec70 2024-04-25T10:08:38.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.653+0000 7ff5ebe00640 1 -- 172.21.15.119:0/2644659066 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec014070 con 0x55fab598ec70 2024-04-25T10:08:38.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.653+0000 7ff5ebe00640 1 -- 172.21.15.119:0/2644659066 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec00f800 con 0x55fab598ec70 2024-04-25T10:08:38.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.653+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2644659066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fab598ec70 msgr2=0x55fab5985dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.653+0000 7ff5f4b56f40 1 --2- 172.21.15.119:0/2644659066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fab598ec70 0x55fab5985dd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec002d90 tx=0x7ff5ec00b5b0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.653+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2644659066 shutdown_connections 2024-04-25T10:08:38.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.653+0000 7ff5f4b56f40 1 --2- 172.21.15.119:0/2644659066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fab598ec70 0x55fab5985dd0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.653+0000 7ff5f4b56f40 1 --2- 172.21.15.119:0/2644659066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab5902080 0x55fab59046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.653+0000 7ff5f4b56f40 1 --2- 172.21.15.119:0/2644659066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fab58b7220 0x55fab598fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.653+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2644659066 >> 172.21.15.119:0/2644659066 conn(0x55fab5905070 msgr2=0x55fab58b64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.654+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2644659066 shutdown_connections 2024-04-25T10:08:38.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.654+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2644659066 wait complete. 2024-04-25T10:08:38.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.654+0000 7ff5f4b56f40 1 Processor -- start 2024-04-25T10:08:38.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.655+0000 7ff5f4b56f40 1 -- start start 2024-04-25T10:08:38.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.655+0000 7ff5f4b56f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab58b7220 0x55fab597cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.655+0000 7ff5f4b56f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fab5902080 0x55fab58b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.655+0000 7ff5f4b56f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fab598ec70 0x55fab58b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.655+0000 7ff5f4b56f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab598a690 con 0x55fab598ec70 2024-04-25T10:08:38.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.655+0000 7ff5f4b56f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab5a1d7a0 con 0x55fab58b7220 2024-04-25T10:08:38.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.655+0000 7ff5f4b56f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fab598a970 con 0x55fab5902080 2024-04-25T10:08:38.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.656+0000 7ff5f2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fab598ec70 0x55fab58b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.656+0000 7ff5f2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fab598ec70 0x55fab58b66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42566/0 (socket says 172.21.15.119:42566) 2024-04-25T10:08:38.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.656+0000 7ff5f2800640 1 -- 172.21.15.119:0/2993870696 learned_addr learned my addr 172.21.15.119:0/2993870696 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:38.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.656+0000 7ff5f1e00640 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab58b7220 0x55fab597cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.656+0000 7ff5f1400640 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fab5902080 0x55fab58b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.656+0000 7ff5f2800640 1 -- 172.21.15.119:0/2993870696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fab5902080 msgr2=0x55fab58b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.656+0000 7ff5f2800640 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fab5902080 0x55fab58b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.656+0000 7ff5f2800640 1 -- 172.21.15.119:0/2993870696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab58b7220 msgr2=0x55fab597cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.656+0000 7ff5f2800640 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fab58b7220 0x55fab597cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.656+0000 7ff5f2800640 1 -- 172.21.15.119:0/2993870696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fab58e4bb0 con 0x55fab598ec70 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.656+0000 7ff5f1400640 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fab5902080 0x55fab58b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.657+0000 7ff5f2800640 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fab598ec70 0x55fab58b66b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00bab0 tx=0x7ff5ec001fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.657+0000 7ff5eaa00640 1 -- 172.21.15.119:0/2993870696 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec028070 con 0x55fab598ec70 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.657+0000 7ff5eaa00640 1 -- 172.21.15.119:0/2993870696 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec01be40 con 0x55fab598ec70 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.657+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2993870696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fab58e4e90 con 0x55fab598ec70 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.657+0000 7ff5eaa00640 1 -- 172.21.15.119:0/2993870696 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec0196c0 con 0x55fab598ec70 2024-04-25T10:08:38.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.657+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2993870696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fab58e5450 con 0x55fab598ec70 2024-04-25T10:08:38.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.659+0000 7ff5eaa00640 1 -- 172.21.15.119:0/2993870696 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5ec02b070 con 0x55fab598ec70 2024-04-25T10:08:38.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.660+0000 7ff5eaa00640 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff5d0069eb0 0x7ff5d006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.660+0000 7ff5eaa00640 1 -- 172.21.15.119:0/2993870696 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5ec08a3c0 con 0x55fab598ec70 2024-04-25T10:08:38.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.660+0000 7ff5f1e00640 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff5d0069eb0 0x7ff5d006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.661+0000 7ff5f4b56f40 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55fab5986c20 0x55fab5987000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.661+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2993870696 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v9 -- 0x55fab5987540 con 0x55fab5986c20 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.661+0000 7ff5f1400640 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55fab5986c20 0x55fab5987000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.661+0000 7ff5f1e00640 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff5d0069eb0 0x7ff5d006c370 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4003420 tx=0x7ff5e4007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.662+0000 7ff5f1400640 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55fab5986c20 0x55fab5987000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.662+0000 7ff5f1400640 1 -- 172.21.15.119:0/2993870696 <== osd.3 v2:172.21.15.119:6816/1142971786 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7ff5dc054870 con 0x55fab5986c20 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.662+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2993870696 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55fab5986c20 msgr2=0x55fab5987000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.662+0000 7ff5f4b56f40 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55fab5986c20 0x55fab5987000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.662+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2993870696 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff5d0069eb0 msgr2=0x7ff5d006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.662+0000 7ff5f4b56f40 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff5d0069eb0 0x7ff5d006c370 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4003420 tx=0x7ff5e4007040 comp rx=0 tx=0).stop 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.662+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2993870696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fab598ec70 msgr2=0x55fab58b66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.662+0000 7ff5f4b56f40 1 --2- 172.21.15.119:0/2993870696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fab598ec70 0x55fab58b66b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00bab0 tx=0x7ff5ec001fe0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.662+0000 7ff5f2800640 1 -- 172.21.15.119:0/2993870696 reap_dead start 2024-04-25T10:08:38.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.662+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2993870696 shutdown_connections 2024-04-25T10:08:38.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.662+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2993870696 >> 172.21.15.119:0/2993870696 conn(0x55fab5905070 msgr2=0x55fab5983510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.663+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2993870696 shutdown_connections 2024-04-25T10:08:38.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.663+0000 7ff5f4b56f40 1 -- 172.21.15.119:0/2993870696 wait complete. 2024-04-25T10:08:38.674 INFO:teuthology.orchestra.run.smithi119.stdout:mytag 2024-04-25T10:08:38.674 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-25T10:08:38.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.759+0000 7fa94d7e6f40 1 Processor -- start 2024-04-25T10:08:38.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.759+0000 7fa94d7e6f40 1 -- start start 2024-04-25T10:08:38.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.760+0000 7fa94d7e6f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555794c94220 0x555794c93990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.760+0000 7fa94d7e6f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555794cdf080 0x555794d686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.760+0000 7fa94d7e6f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555794d6bc70 0x555794ce1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.760+0000 7fa94d7e6f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555794c946c0 con 0x555794d6bc70 2024-04-25T10:08:38.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.760+0000 7fa94d7e6f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555794c94960 con 0x555794c94220 2024-04-25T10:08:38.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.760+0000 7fa94d7e6f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555794cc61c0 con 0x555794cdf080 2024-04-25T10:08:38.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.760+0000 7fa94b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555794d6bc70 0x555794ce1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.760+0000 7fa94b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555794d6bc70 0x555794ce1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42584/0 (socket says 172.21.15.119:42584) 2024-04-25T10:08:38.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.760+0000 7fa94b400640 1 -- 172.21.15.119:0/2668991417 learned_addr learned my addr 172.21.15.119:0/2668991417 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:38.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.761+0000 7fa94aa00640 1 --2- 172.21.15.119:0/2668991417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555794c94220 0x555794c93990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.761+0000 7fa94b400640 1 -- 172.21.15.119:0/2668991417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555794cdf080 msgr2=0x555794d686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.761+0000 7fa94b400640 1 --2- 172.21.15.119:0/2668991417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555794cdf080 0x555794d686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.761+0000 7fa94b400640 1 -- 172.21.15.119:0/2668991417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555794c94220 msgr2=0x555794c93990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.761+0000 7fa94b400640 1 --2- 172.21.15.119:0/2668991417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555794c94220 0x555794c93990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.761+0000 7fa94b400640 1 -- 172.21.15.119:0/2668991417 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555794c013b0 con 0x555794d6bc70 2024-04-25T10:08:38.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.761+0000 7fa94b400640 1 --2- 172.21.15.119:0/2668991417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555794d6bc70 0x555794ce1310 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa944009570 tx=0x7fa944009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c91cd7a1e80685 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.761+0000 7fa949600640 1 -- 172.21.15.119:0/2668991417 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944017070 con 0x555794d6bc70 2024-04-25T10:08:38.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.761+0000 7fa949600640 1 -- 172.21.15.119:0/2668991417 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa944006600 con 0x555794d6bc70 2024-04-25T10:08:38.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.761+0000 7fa949600640 1 -- 172.21.15.119:0/2668991417 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440069f0 con 0x555794d6bc70 2024-04-25T10:08:38.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.762+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2668991417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555794d6bc70 msgr2=0x555794ce1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.762+0000 7fa94d7e6f40 1 --2- 172.21.15.119:0/2668991417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555794d6bc70 0x555794ce1310 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa944009570 tx=0x7fa944009a30 comp rx=0 tx=0).stop 2024-04-25T10:08:38.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.762+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2668991417 shutdown_connections 2024-04-25T10:08:38.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.762+0000 7fa94d7e6f40 1 --2- 172.21.15.119:0/2668991417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555794d6bc70 0x555794ce1310 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.762+0000 7fa94d7e6f40 1 --2- 172.21.15.119:0/2668991417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555794cdf080 0x555794d686e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.762+0000 7fa94d7e6f40 1 --2- 172.21.15.119:0/2668991417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555794c94220 0x555794c93990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.762+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2668991417 >> 172.21.15.119:0/2668991417 conn(0x555794ce2070 msgr2=0x555794ce2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.762+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2668991417 shutdown_connections 2024-04-25T10:08:38.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.762+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2668991417 wait complete. 2024-04-25T10:08:38.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.763+0000 7fa94d7e6f40 1 Processor -- start 2024-04-25T10:08:38.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.763+0000 7fa94d7e6f40 1 -- start start 2024-04-25T10:08:38.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.764+0000 7fa94d7e6f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555794c94220 0x555794dfa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.764+0000 7fa94d7e6f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555794cdf080 0x555794d5a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.764+0000 7fa94d7e6f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555794d6bc70 0x555794c91e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.764+0000 7fa94d7e6f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555794d6c2a0 con 0x555794c94220 2024-04-25T10:08:38.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.764+0000 7fa94d7e6f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555794d6caa0 con 0x555794cdf080 2024-04-25T10:08:38.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.764+0000 7fa94d7e6f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555794d6c580 con 0x555794d6bc70 2024-04-25T10:08:38.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.764+0000 7fa94aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555794c94220 0x555794dfa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.764+0000 7fa94aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555794c94220 0x555794dfa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42588/0 (socket says 172.21.15.119:42588) 2024-04-25T10:08:38.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.764+0000 7fa94aa00640 1 -- 172.21.15.119:0/2649775513 learned_addr learned my addr 172.21.15.119:0/2649775513 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:38.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.765+0000 7fa94b400640 1 --2- 172.21.15.119:0/2649775513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555794d6bc70 0x555794c91e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.765+0000 7fa942000640 1 --2- 172.21.15.119:0/2649775513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555794cdf080 0x555794d5a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.765+0000 7fa94aa00640 1 -- 172.21.15.119:0/2649775513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555794d6bc70 msgr2=0x555794c91e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.765+0000 7fa94aa00640 1 --2- 172.21.15.119:0/2649775513 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555794d6bc70 0x555794c91e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.765+0000 7fa94aa00640 1 -- 172.21.15.119:0/2649775513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555794cdf080 msgr2=0x555794d5a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.765+0000 7fa94aa00640 1 --2- 172.21.15.119:0/2649775513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555794cdf080 0x555794d5a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.765+0000 7fa94aa00640 1 -- 172.21.15.119:0/2649775513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555794d5a8c0 con 0x555794c94220 2024-04-25T10:08:38.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.765+0000 7fa94aa00640 1 --2- 172.21.15.119:0/2649775513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555794c94220 0x555794dfa170 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa93c008420 tx=0x7fa93c0088e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.765+0000 7fa943e00640 1 -- 172.21.15.119:0/2649775513 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c016030 con 0x555794c94220 2024-04-25T10:08:38.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.765+0000 7fa943e00640 1 -- 172.21.15.119:0/2649775513 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c012410 con 0x555794c94220 2024-04-25T10:08:38.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.765+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2649775513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555794d5e0e0 con 0x555794c94220 2024-04-25T10:08:38.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.765+0000 7fa943e00640 1 -- 172.21.15.119:0/2649775513 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00e040 con 0x555794c94220 2024-04-25T10:08:38.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.766+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2649775513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555794d5e6a0 con 0x555794c94220 2024-04-25T10:08:38.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.768+0000 7fa943e00640 1 -- 172.21.15.119:0/2649775513 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa93c019070 con 0x555794c94220 2024-04-25T10:08:38.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.768+0000 7fa943e00640 1 --2- 172.21.15.119:0/2649775513 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa928069eb0 0x7fa92806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.769+0000 7fa943e00640 1 -- 172.21.15.119:0/2649775513 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa93c088a60 con 0x555794c94220 2024-04-25T10:08:38.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.769+0000 7fa942000640 1 --2- 172.21.15.119:0/2649775513 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa928069eb0 0x7fa92806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.769+0000 7fa94d7e6f40 1 --2- 172.21.15.119:0/2649775513 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x555794d5cc90 0x555794d5d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.769+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2649775513 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:1 4.7 4:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v9 -- 0x555794d5d650 con 0x555794d5cc90 2024-04-25T10:08:38.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.769+0000 7fa94b400640 1 --2- 172.21.15.119:0/2649775513 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x555794d5cc90 0x555794d5d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.770+0000 7fa942000640 1 --2- 172.21.15.119:0/2649775513 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa928069eb0 0x7fa92806c370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa934056000 tx=0x7fa93404e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.770+0000 7fa94b400640 1 --2- 172.21.15.119:0/2649775513 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x555794d5cc90 0x555794d5d110 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.770+0000 7fa94b400640 1 -- 172.21.15.119:0/2649775513 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fa944006a60 con 0x555794d5cc90 2024-04-25T10:08:38.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.770+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2649775513 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x555794d5cc90 msgr2=0x555794d5d110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.770+0000 7fa94d7e6f40 1 --2- 172.21.15.119:0/2649775513 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x555794d5cc90 0x555794d5d110 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.770+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2649775513 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa928069eb0 msgr2=0x7fa92806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.770+0000 7fa94d7e6f40 1 --2- 172.21.15.119:0/2649775513 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa928069eb0 0x7fa92806c370 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa934056000 tx=0x7fa93404e000 comp rx=0 tx=0).stop 2024-04-25T10:08:38.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.770+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2649775513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555794c94220 msgr2=0x555794dfa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.770+0000 7fa94d7e6f40 1 --2- 172.21.15.119:0/2649775513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555794c94220 0x555794dfa170 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa93c008420 tx=0x7fa93c0088e0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.771+0000 7fa94b400640 1 -- 172.21.15.119:0/2649775513 reap_dead start 2024-04-25T10:08:38.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.771+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2649775513 shutdown_connections 2024-04-25T10:08:38.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.771+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2649775513 >> 172.21.15.119:0/2649775513 conn(0x555794ce2070 msgr2=0x555794d6aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.771+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2649775513 shutdown_connections 2024-04-25T10:08:38.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.771+0000 7fa94d7e6f40 1 -- 172.21.15.119:0/2649775513 wait complete. 2024-04-25T10:08:38.782 INFO:teuthology.orchestra.run.smithi119.stdout:mytag 2024-04-25T10:08:38.782 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-25T10:08:38.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.875+0000 7fda48885f40 1 Processor -- start 2024-04-25T10:08:38.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.875+0000 7fda48885f40 1 -- start start 2024-04-25T10:08:38.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda48885f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559d6f169220 0x559d6f1b6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda48885f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559d6f1b4080 0x559d6f167f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda48885f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d6f240c70 0x559d6f242d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda48885f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6f19c2f0 con 0x559d6f240c70 2024-04-25T10:08:38.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda48885f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6f110dd0 con 0x559d6f169220 2024-04-25T10:08:38.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda48885f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d6f169960 con 0x559d6f1b4080 2024-04-25T10:08:38.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda46400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d6f240c70 0x559d6f242d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda46400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d6f240c70 0x559d6f242d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42614/0 (socket says 172.21.15.119:42614) 2024-04-25T10:08:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda46400640 1 -- 172.21.15.119:0/1964208085 learned_addr learned my addr 172.21.15.119:0/1964208085 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda46400640 1 -- 172.21.15.119:0/1964208085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559d6f1b4080 msgr2=0x559d6f167f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda46400640 1 --2- 172.21.15.119:0/1964208085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559d6f1b4080 0x559d6f167f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda45a00640 1 --2- 172.21.15.119:0/1964208085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559d6f169220 0x559d6f1b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda46400640 1 -- 172.21.15.119:0/1964208085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559d6f169220 msgr2=0x559d6f1b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda46400640 1 --2- 172.21.15.119:0/1964208085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559d6f169220 0x559d6f1b6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.876+0000 7fda46400640 1 -- 172.21.15.119:0/1964208085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d6f241050 con 0x559d6f240c70 2024-04-25T10:08:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.877+0000 7fda46400640 1 --2- 172.21.15.119:0/1964208085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d6f240c70 0x559d6f242d30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fda400095d0 tx=0x7fda40009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e313d7b24b202a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.877+0000 7fda3f400640 1 -- 172.21.15.119:0/1964208085 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40017070 con 0x559d6f240c70 2024-04-25T10:08:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.877+0000 7fda3f400640 1 -- 172.21.15.119:0/1964208085 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda40006680 con 0x559d6f240c70 2024-04-25T10:08:38.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.877+0000 7fda3f400640 1 -- 172.21.15.119:0/1964208085 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40006a70 con 0x559d6f240c70 2024-04-25T10:08:38.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.878+0000 7fda48885f40 1 -- 172.21.15.119:0/1964208085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d6f240c70 msgr2=0x559d6f242d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.878+0000 7fda48885f40 1 --2- 172.21.15.119:0/1964208085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d6f240c70 0x559d6f242d30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fda400095d0 tx=0x7fda40009a90 comp rx=0 tx=0).stop 2024-04-25T10:08:38.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.878+0000 7fda48885f40 1 -- 172.21.15.119:0/1964208085 shutdown_connections 2024-04-25T10:08:38.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.878+0000 7fda48885f40 1 --2- 172.21.15.119:0/1964208085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d6f240c70 0x559d6f242d30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.878+0000 7fda48885f40 1 --2- 172.21.15.119:0/1964208085 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559d6f1b4080 0x559d6f167f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.878+0000 7fda48885f40 1 --2- 172.21.15.119:0/1964208085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559d6f169220 0x559d6f1b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.878+0000 7fda48885f40 1 -- 172.21.15.119:0/1964208085 >> 172.21.15.119:0/1964208085 conn(0x559d6f1b7070 msgr2=0x559d6f1b7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.878+0000 7fda48885f40 1 -- 172.21.15.119:0/1964208085 shutdown_connections 2024-04-25T10:08:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.878+0000 7fda48885f40 1 -- 172.21.15.119:0/1964208085 wait complete. 2024-04-25T10:08:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.879+0000 7fda48885f40 1 Processor -- start 2024-04-25T10:08:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.879+0000 7fda48885f40 1 -- start start 2024-04-25T10:08:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda48885f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559d6f169220 0x559d6f23fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda48885f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d6f1b4080 0x559d6f2266b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda48885f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559d6f240c70 0x559d6f1b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda48885f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6f168c10 con 0x559d6f1b4080 2024-04-25T10:08:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda48885f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6f168e60 con 0x559d6f240c70 2024-04-25T10:08:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda48885f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d6f241dc0 con 0x559d6f169220 2024-04-25T10:08:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda45000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d6f1b4080 0x559d6f2266b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda45000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d6f1b4080 0x559d6f2266b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42630/0 (socket says 172.21.15.119:42630) 2024-04-25T10:08:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda45000640 1 -- 172.21.15.119:0/1316097527 learned_addr learned my addr 172.21.15.119:0/1316097527 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda45000640 1 -- 172.21.15.119:0/1316097527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559d6f169220 msgr2=0x559d6f23fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda45a00640 1 --2- 172.21.15.119:0/1316097527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559d6f169220 0x559d6f23fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda45000640 1 --2- 172.21.15.119:0/1316097527 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559d6f169220 0x559d6f23fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda45000640 1 -- 172.21.15.119:0/1316097527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559d6f240c70 msgr2=0x559d6f1b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda45000640 1 --2- 172.21.15.119:0/1316097527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559d6f240c70 0x559d6f1b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.880+0000 7fda45000640 1 -- 172.21.15.119:0/1316097527 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d6f227010 con 0x559d6f1b4080 2024-04-25T10:08:38.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.881+0000 7fda45000640 1 --2- 172.21.15.119:0/1316097527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d6f1b4080 0x559d6f2266b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x559d6f2283a0 tx=0x7fda3000bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.881+0000 7fda3e000640 1 -- 172.21.15.119:0/1316097527 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30013070 con 0x559d6f1b4080 2024-04-25T10:08:38.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.881+0000 7fda3e000640 1 -- 172.21.15.119:0/1316097527 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda30009cd0 con 0x559d6f1b4080 2024-04-25T10:08:38.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.881+0000 7fda48885f40 1 -- 172.21.15.119:0/1316097527 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559d6f23f1b0 con 0x559d6f1b4080 2024-04-25T10:08:38.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.881+0000 7fda3e000640 1 -- 172.21.15.119:0/1316097527 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3000f640 con 0x559d6f1b4080 2024-04-25T10:08:38.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.881+0000 7fda48885f40 1 -- 172.21.15.119:0/1316097527 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d6f23f770 con 0x559d6f1b4080 2024-04-25T10:08:38.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.883+0000 7fda3e000640 1 -- 172.21.15.119:0/1316097527 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda30016070 con 0x559d6f1b4080 2024-04-25T10:08:38.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.884+0000 7fda3e000640 1 --2- 172.21.15.119:0/1316097527 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fda24069eb0 0x7fda2406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.884+0000 7fda3e000640 1 -- 172.21.15.119:0/1316097527 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda30088740 con 0x559d6f1b4080 2024-04-25T10:08:38.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.884+0000 7fda45a00640 1 --2- 172.21.15.119:0/1316097527 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fda24069eb0 0x7fda2406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.885+0000 7fda48885f40 1 --2- 172.21.15.119:0/1316097527 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x559d6f2cf940 0x559d6f2cc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.885+0000 7fda48885f40 1 -- 172.21.15.119:0/1316097527 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:1 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v9 -- 0x559d6f1b6c70 con 0x559d6f2cf940 2024-04-25T10:08:38.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.885+0000 7fda46400640 1 --2- 172.21.15.119:0/1316097527 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x559d6f2cf940 0x559d6f2cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.885+0000 7fda45a00640 1 --2- 172.21.15.119:0/1316097527 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fda24069eb0 0x7fda2406c370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fda38056000 tx=0x7fda38007030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.886+0000 7fda46400640 1 --2- 172.21.15.119:0/1316097527 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x559d6f2cf940 0x559d6f2cc050 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.886+0000 7fda46400640 1 -- 172.21.15.119:0/1316097527 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fda4001a040 con 0x559d6f2cf940 2024-04-25T10:08:38.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.886+0000 7fda48885f40 1 -- 172.21.15.119:0/1316097527 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x559d6f2cf940 msgr2=0x559d6f2cc050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.886+0000 7fda48885f40 1 --2- 172.21.15.119:0/1316097527 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x559d6f2cf940 0x559d6f2cc050 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.886+0000 7fda48885f40 1 -- 172.21.15.119:0/1316097527 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fda24069eb0 msgr2=0x7fda2406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.886+0000 7fda48885f40 1 --2- 172.21.15.119:0/1316097527 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fda24069eb0 0x7fda2406c370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fda38056000 tx=0x7fda38007030 comp rx=0 tx=0).stop 2024-04-25T10:08:38.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.886+0000 7fda48885f40 1 -- 172.21.15.119:0/1316097527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d6f1b4080 msgr2=0x559d6f2266b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.886+0000 7fda48885f40 1 --2- 172.21.15.119:0/1316097527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d6f1b4080 0x559d6f2266b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x559d6f2283a0 tx=0x7fda3000bef0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.886+0000 7fda46400640 1 -- 172.21.15.119:0/1316097527 reap_dead start 2024-04-25T10:08:38.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.886+0000 7fda48885f40 1 -- 172.21.15.119:0/1316097527 shutdown_connections 2024-04-25T10:08:38.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.886+0000 7fda48885f40 1 -- 172.21.15.119:0/1316097527 >> 172.21.15.119:0/1316097527 conn(0x559d6f1b7070 msgr2=0x559d6f242b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.886+0000 7fda48885f40 1 -- 172.21.15.119:0/1316097527 shutdown_connections 2024-04-25T10:08:38.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.886+0000 7fda48885f40 1 -- 172.21.15.119:0/1316097527 wait complete. 2024-04-25T10:08:38.897 INFO:teuthology.orchestra.run.smithi119.stdout:mytag 2024-04-25T10:08:38.897 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-25T10:08:38.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.984+0000 7f42c700cf40 1 Processor -- start 2024-04-25T10:08:38.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.984+0000 7f42c700cf40 1 -- start start 2024-04-25T10:08:38.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.985+0000 7f42c700cf40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55782eb23220 0x55782ebf28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.985+0000 7f42c700cf40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55782eb6e080 0x55782eb70590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.985+0000 7f42c700cf40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55782ebfac70 0x55782ebfc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.985+0000 7f42c700cf40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782eb562f0 con 0x55782eb6e080 2024-04-25T10:08:38.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.985+0000 7f42c700cf40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782eacadd0 con 0x55782ebfac70 2024-04-25T10:08:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.985+0000 7f42c700cf40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55782eb23960 con 0x55782eb23220 2024-04-25T10:08:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.985+0000 7f42bac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55782eb6e080 0x55782eb70590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.985+0000 7f42bac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55782eb6e080 0x55782eb70590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42632/0 (socket says 172.21.15.119:42632) 2024-04-25T10:08:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.985+0000 7f42bac00640 1 -- 172.21.15.119:0/2905096903 learned_addr learned my addr 172.21.15.119:0/2905096903 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.985+0000 7f42c4000640 1 --2- 172.21.15.119:0/2905096903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55782ebfac70 0x55782ebfc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.985+0000 7f42c3600640 1 --2- 172.21.15.119:0/2905096903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55782eb23220 0x55782ebf28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.986+0000 7f42bac00640 1 -- 172.21.15.119:0/2905096903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55782eb23220 msgr2=0x55782ebf28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.986+0000 7f42bac00640 1 --2- 172.21.15.119:0/2905096903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55782eb23220 0x55782ebf28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.986+0000 7f42bac00640 1 -- 172.21.15.119:0/2905096903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55782ebfac70 msgr2=0x55782ebfc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.986+0000 7f42bac00640 1 --2- 172.21.15.119:0/2905096903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55782ebfac70 0x55782ebfc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.986+0000 7f42bac00640 1 -- 172.21.15.119:0/2905096903 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55782ebfa2e0 con 0x55782eb6e080 2024-04-25T10:08:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.986+0000 7f42bac00640 1 --2- 172.21.15.119:0/2905096903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55782eb6e080 0x55782eb70590 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f42ac001dd0 tx=0x7f42ac00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a09df62142a14b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.986+0000 7f42c2200640 1 -- 172.21.15.119:0/2905096903 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00bd80 con 0x55782eb6e080 2024-04-25T10:08:38.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.986+0000 7f42c2200640 1 -- 172.21.15.119:0/2905096903 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac00f400 con 0x55782eb6e080 2024-04-25T10:08:38.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.986+0000 7f42c2200640 1 -- 172.21.15.119:0/2905096903 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00f830 con 0x55782eb6e080 2024-04-25T10:08:38.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.987+0000 7f42c700cf40 1 -- 172.21.15.119:0/2905096903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55782eb6e080 msgr2=0x55782eb70590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.987+0000 7f42c700cf40 1 --2- 172.21.15.119:0/2905096903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55782eb6e080 0x55782eb70590 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f42ac001dd0 tx=0x7f42ac00b5b0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.987+0000 7f42c700cf40 1 -- 172.21.15.119:0/2905096903 shutdown_connections 2024-04-25T10:08:38.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.987+0000 7f42c700cf40 1 --2- 172.21.15.119:0/2905096903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55782ebfac70 0x55782ebfc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.987+0000 7f42c700cf40 1 --2- 172.21.15.119:0/2905096903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55782eb6e080 0x55782eb70590 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.987+0000 7f42c700cf40 1 --2- 172.21.15.119:0/2905096903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55782eb23220 0x55782ebf28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.987+0000 7f42c700cf40 1 -- 172.21.15.119:0/2905096903 >> 172.21.15.119:0/2905096903 conn(0x55782eb71070 msgr2=0x55782eb71450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.987+0000 7f42c700cf40 1 -- 172.21.15.119:0/2905096903 shutdown_connections 2024-04-25T10:08:38.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.987+0000 7f42c700cf40 1 -- 172.21.15.119:0/2905096903 wait complete. 2024-04-25T10:08:38.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.988+0000 7f42c700cf40 1 Processor -- start 2024-04-25T10:08:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.988+0000 7f42c700cf40 1 -- start start 2024-04-25T10:08:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c700cf40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55782eb23220 0x55782ebf9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c700cf40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55782eb6e080 0x55782ec90f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c700cf40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55782ebfac70 0x55782ebfcb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c700cf40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782ebfbc90 con 0x55782ebfac70 2024-04-25T10:08:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c700cf40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782ebfddb0 con 0x55782eb23220 2024-04-25T10:08:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c700cf40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55782ebfbee0 con 0x55782eb6e080 2024-04-25T10:08:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c4000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55782ebfac70 0x55782ebfcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c4000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55782ebfac70 0x55782ebfcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42644/0 (socket says 172.21.15.119:42644) 2024-04-25T10:08:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c4000640 1 -- 172.21.15.119:0/2860113921 learned_addr learned my addr 172.21.15.119:0/2860113921 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42bac00640 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55782eb6e080 0x55782ec90f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c3600640 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55782eb23220 0x55782ebf9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c4000640 1 -- 172.21.15.119:0/2860113921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55782eb6e080 msgr2=0x55782ec90f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c4000640 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55782eb6e080 0x55782ec90f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c4000640 1 -- 172.21.15.119:0/2860113921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55782eb23220 msgr2=0x55782ebf9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c4000640 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55782eb23220 0x55782ebf9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c4000640 1 -- 172.21.15.119:0/2860113921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55782ebefd30 con 0x55782ebfac70 2024-04-25T10:08:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.989+0000 7f42c3600640 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55782eb23220 0x55782ebf9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.990+0000 7f42c4000640 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55782ebfac70 0x55782ebfcb80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f42bc008430 tx=0x7f42bc008520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.990+0000 7f42c0e00640 1 -- 172.21.15.119:0/2860113921 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc008e60 con 0x55782ebfac70 2024-04-25T10:08:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.990+0000 7f42c0e00640 1 -- 172.21.15.119:0/2860113921 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc019070 con 0x55782ebfac70 2024-04-25T10:08:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.990+0000 7f42c0e00640 1 -- 172.21.15.119:0/2860113921 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc014850 con 0x55782ebfac70 2024-04-25T10:08:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.990+0000 7f42c700cf40 1 -- 172.21.15.119:0/2860113921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55782ebf0010 con 0x55782ebfac70 2024-04-25T10:08:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.990+0000 7f42c700cf40 1 -- 172.21.15.119:0/2860113921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55782ebf05d0 con 0x55782ebfac70 2024-04-25T10:08:39.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.992+0000 7f42c0e00640 1 -- 172.21.15.119:0/2860113921 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42bc0149f0 con 0x55782ebfac70 2024-04-25T10:08:39.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.993+0000 7f42c0e00640 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42a0069eb0 0x7f42a006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.993+0000 7f42c0e00640 1 -- 172.21.15.119:0/2860113921 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42bc088f00 con 0x55782ebfac70 2024-04-25T10:08:39.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.993+0000 7f42c3600640 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42a0069eb0 0x7f42a006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.993+0000 7f42c700cf40 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55782ebf34d0 0x55782ebdd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.993+0000 7f42c700cf40 1 -- 172.21.15.119:0/2860113921 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:1 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v9 -- 0x55782ebef8c0 con 0x55782ebf34d0 2024-04-25T10:08:39.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.994+0000 7f42bac00640 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55782ebf34d0 0x55782ebdd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.994+0000 7f42c3600640 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42a0069eb0 0x7f42a006c370 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f42b4056000 tx=0x7f42b405d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.994+0000 7f42bac00640 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55782ebf34d0 0x55782ebdd8a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.995+0000 7f42bac00640 1 -- 172.21.15.119:0/2860113921 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f42ac00f6c0 con 0x55782ebf34d0 2024-04-25T10:08:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.995+0000 7f42c700cf40 1 -- 172.21.15.119:0/2860113921 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55782ebf34d0 msgr2=0x55782ebdd8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.995+0000 7f42c700cf40 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55782ebf34d0 0x55782ebdd8a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.995+0000 7f42c700cf40 1 -- 172.21.15.119:0/2860113921 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42a0069eb0 msgr2=0x7f42a006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.995+0000 7f42c700cf40 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42a0069eb0 0x7f42a006c370 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f42b4056000 tx=0x7f42b405d040 comp rx=0 tx=0).stop 2024-04-25T10:08:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.995+0000 7f42c700cf40 1 -- 172.21.15.119:0/2860113921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55782ebfac70 msgr2=0x55782ebfcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.995+0000 7f42c700cf40 1 --2- 172.21.15.119:0/2860113921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55782ebfac70 0x55782ebfcb80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f42bc008430 tx=0x7f42bc008520 comp rx=0 tx=0).stop 2024-04-25T10:08:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.995+0000 7f42c4000640 1 -- 172.21.15.119:0/2860113921 reap_dead start 2024-04-25T10:08:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.995+0000 7f42c700cf40 1 -- 172.21.15.119:0/2860113921 shutdown_connections 2024-04-25T10:08:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.995+0000 7f42c700cf40 1 -- 172.21.15.119:0/2860113921 >> 172.21.15.119:0/2860113921 conn(0x55782eb71070 msgr2=0x55782ebfb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.995+0000 7f42c700cf40 1 -- 172.21.15.119:0/2860113921 shutdown_connections 2024-04-25T10:08:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:38.995+0000 7f42c700cf40 1 -- 172.21.15.119:0/2860113921 wait complete. 2024-04-25T10:08:39.007 INFO:teuthology.orchestra.run.smithi119.stdout:mytag 2024-04-25T10:08:39.008 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T10:08:39.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.090+0000 7f190171af40 1 Processor -- start 2024-04-25T10:08:39.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.090+0000 7f190171af40 1 -- start start 2024-04-25T10:08:39.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.091+0000 7f190171af40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5597175ca220 0x559717691160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.091+0000 7f190171af40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559717615080 0x55971769c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.091+0000 7f190171af40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597176a1c70 0x55971769ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.091+0000 7f190171af40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559717571dd0 con 0x5597176a1c70 2024-04-25T10:08:39.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.091+0000 7f190171af40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559717575d30 con 0x559717615080 2024-04-25T10:08:39.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.091+0000 7f190171af40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5597175fd2b0 con 0x5597175ca220 2024-04-25T10:08:39.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.091+0000 7f18fe800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597176a1c70 0x55971769ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.091+0000 7f18fe800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597176a1c70 0x55971769ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42658/0 (socket says 172.21.15.119:42658) 2024-04-25T10:08:39.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.091+0000 7f18fe800640 1 -- 172.21.15.119:0/1398885261 learned_addr learned my addr 172.21.15.119:0/1398885261 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:39.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.092+0000 7f18fd400640 1 --2- 172.21.15.119:0/1398885261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559717615080 0x55971769c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.092+0000 7f18fde00640 1 --2- 172.21.15.119:0/1398885261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5597175ca220 0x559717691160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.092+0000 7f18fe800640 1 -- 172.21.15.119:0/1398885261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5597175ca220 msgr2=0x559717691160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:39.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.092+0000 7f18fe800640 1 --2- 172.21.15.119:0/1398885261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5597175ca220 0x559717691160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.092+0000 7f18fe800640 1 -- 172.21.15.119:0/1398885261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559717615080 msgr2=0x55971769c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:39.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.092+0000 7f18fe800640 1 --2- 172.21.15.119:0/1398885261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559717615080 0x55971769c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.092+0000 7f18fe800640 1 -- 172.21.15.119:0/1398885261 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597176916a0 con 0x5597176a1c70 2024-04-25T10:08:39.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.092+0000 7f18fd400640 1 --2- 172.21.15.119:0/1398885261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559717615080 0x55971769c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:39.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.092+0000 7f18fde00640 1 --2- 172.21.15.119:0/1398885261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5597175ca220 0x559717691160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:39.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.092+0000 7f18fe800640 1 --2- 172.21.15.119:0/1398885261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597176a1c70 0x55971769ca70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f18f8005fd0 tx=0x7f18f8050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9428efe6469aafa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.092+0000 7f18f7e00640 1 -- 172.21.15.119:0/1398885261 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8067070 con 0x5597176a1c70 2024-04-25T10:08:39.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.093+0000 7f18f7e00640 1 -- 172.21.15.119:0/1398885261 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f804fc50 con 0x5597176a1c70 2024-04-25T10:08:39.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.093+0000 7f18f7e00640 1 -- 172.21.15.119:0/1398885261 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f804f6c0 con 0x5597176a1c70 2024-04-25T10:08:39.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.093+0000 7f190171af40 1 -- 172.21.15.119:0/1398885261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597176a1c70 msgr2=0x55971769ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.093+0000 7f190171af40 1 --2- 172.21.15.119:0/1398885261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597176a1c70 0x55971769ca70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f18f8005fd0 tx=0x7f18f8050000 comp rx=0 tx=0).stop 2024-04-25T10:08:39.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.093+0000 7f190171af40 1 -- 172.21.15.119:0/1398885261 shutdown_connections 2024-04-25T10:08:39.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.093+0000 7f190171af40 1 --2- 172.21.15.119:0/1398885261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597176a1c70 0x55971769ca70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.093+0000 7f190171af40 1 --2- 172.21.15.119:0/1398885261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559717615080 0x55971769c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.093+0000 7f190171af40 1 --2- 172.21.15.119:0/1398885261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5597175ca220 0x559717691160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.093+0000 7f190171af40 1 -- 172.21.15.119:0/1398885261 >> 172.21.15.119:0/1398885261 conn(0x559717618070 msgr2=0x5597175f84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:39.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.093+0000 7f190171af40 1 -- 172.21.15.119:0/1398885261 shutdown_connections 2024-04-25T10:08:39.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.093+0000 7f190171af40 1 -- 172.21.15.119:0/1398885261 wait complete. 2024-04-25T10:08:39.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.094+0000 7f190171af40 1 Processor -- start 2024-04-25T10:08:39.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.094+0000 7f190171af40 1 -- start start 2024-04-25T10:08:39.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.095+0000 7f190171af40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597175ca220 0x559717731830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.095+0000 7f190171af40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559717615080 0x5597175f8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.095+0000 7f190171af40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5597176a1c70 0x5597176a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.095+0000 7f190171af40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597175c9300 con 0x5597175ca220 2024-04-25T10:08:39.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.095+0000 7f190171af40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55971769c0c0 con 0x559717615080 2024-04-25T10:08:39.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.095+0000 7f190171af40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5597175c9ad0 con 0x5597176a1c70 2024-04-25T10:08:39.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.095+0000 7f18fde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597175ca220 0x559717731830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.095+0000 7f18fde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597175ca220 0x559717731830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42670/0 (socket says 172.21.15.119:42670) 2024-04-25T10:08:39.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.095+0000 7f18fde00640 1 -- 172.21.15.119:0/134327027 learned_addr learned my addr 172.21.15.119:0/134327027 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:39.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.096+0000 7f18fde00640 1 -- 172.21.15.119:0/134327027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5597176a1c70 msgr2=0x5597176a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.096+0000 7f18fe800640 1 --2- 172.21.15.119:0/134327027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5597176a1c70 0x5597176a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.096+0000 7f18fde00640 1 --2- 172.21.15.119:0/134327027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5597176a1c70 0x5597176a04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.096+0000 7f18fde00640 1 -- 172.21.15.119:0/134327027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559717615080 msgr2=0x5597175f8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.096+0000 7f18fd400640 1 --2- 172.21.15.119:0/134327027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559717615080 0x5597175f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.096+0000 7f18fde00640 1 --2- 172.21.15.119:0/134327027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559717615080 0x5597175f8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.096+0000 7f18fde00640 1 -- 172.21.15.119:0/134327027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55971768d860 con 0x5597175ca220 2024-04-25T10:08:39.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.096+0000 7f18fde00640 1 --2- 172.21.15.119:0/134327027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597175ca220 0x559717731830 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18f0001ea0 tx=0x7f18f000a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.096+0000 7f18f6a00640 1 -- 172.21.15.119:0/134327027 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0010470 con 0x5597175ca220 2024-04-25T10:08:39.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.096+0000 7f18f6a00640 1 -- 172.21.15.119:0/134327027 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f0010610 con 0x5597175ca220 2024-04-25T10:08:39.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.096+0000 7f18f6a00640 1 -- 172.21.15.119:0/134327027 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f00109c0 con 0x5597175ca220 2024-04-25T10:08:39.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.096+0000 7f190171af40 1 -- 172.21.15.119:0/134327027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55971768db40 con 0x5597175ca220 2024-04-25T10:08:39.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.097+0000 7f190171af40 1 -- 172.21.15.119:0/134327027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55971768de20 con 0x5597175ca220 2024-04-25T10:08:39.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.098+0000 7f18f6a00640 1 -- 172.21.15.119:0/134327027 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f18f000e040 con 0x5597175ca220 2024-04-25T10:08:39.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.099+0000 7f18f6a00640 1 --2- 172.21.15.119:0/134327027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f18e0069eb0 0x7f18e006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.099+0000 7f18f6a00640 1 -- 172.21.15.119:0/134327027 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18f0088c50 con 0x5597175ca220 2024-04-25T10:08:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.100+0000 7f18fd400640 1 --2- 172.21.15.119:0/134327027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f18e0069eb0 0x7f18e006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.100+0000 7f190171af40 1 --2- 172.21.15.119:0/134327027 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5597175f8840 0x55971773bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.100+0000 7f190171af40 1 -- 172.21.15.119:0/134327027 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v9 -- 0x5597176a0f00 con 0x5597175f8840 2024-04-25T10:08:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.100+0000 7f18fe800640 1 --2- 172.21.15.119:0/134327027 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5597175f8840 0x55971773bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.100+0000 7f18fd400640 1 --2- 172.21.15.119:0/134327027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f18e0069eb0 0x7f18e006c370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5597175c7ea0 tx=0x7f18ec0014d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.101+0000 7f18fe800640 1 --2- 172.21.15.119:0/134327027 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5597175f8840 0x55971773bb00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.101+0000 7f18fe800640 1 -- 172.21.15.119:0/134327027 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f18f8060070 con 0x5597175f8840 2024-04-25T10:08:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-04-25T10:08:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.101+0000 7f190171af40 1 -- 172.21.15.119:0/134327027 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5597175f8840 msgr2=0x55971773bb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.101+0000 7f190171af40 1 --2- 172.21.15.119:0/134327027 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5597175f8840 0x55971773bb00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.101+0000 7f190171af40 1 -- 172.21.15.119:0/134327027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f18e0069eb0 msgr2=0x7f18e006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.101+0000 7f190171af40 1 --2- 172.21.15.119:0/134327027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f18e0069eb0 0x7f18e006c370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5597175c7ea0 tx=0x7f18ec0014d0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.101+0000 7f190171af40 1 -- 172.21.15.119:0/134327027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597175ca220 msgr2=0x559717731830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.101+0000 7f190171af40 1 --2- 172.21.15.119:0/134327027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5597175ca220 0x559717731830 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18f0001ea0 tx=0x7f18f000a5a0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.101+0000 7f18fe800640 1 -- 172.21.15.119:0/134327027 reap_dead start 2024-04-25T10:08:39.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.102+0000 7f190171af40 1 -- 172.21.15.119:0/134327027 shutdown_connections 2024-04-25T10:08:39.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.102+0000 7f190171af40 1 -- 172.21.15.119:0/134327027 >> 172.21.15.119:0/134327027 conn(0x559717618070 msgr2=0x5597176172d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:39.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.102+0000 7f190171af40 1 -- 172.21.15.119:0/134327027 shutdown_connections 2024-04-25T10:08:39.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.102+0000 7f190171af40 1 -- 172.21.15.119:0/134327027 wait complete. 2024-04-25T10:08:39.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:08:39.115 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:08:39.116 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:39.216 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:39.320 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:08:39.321 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:08:39.321 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:39.379 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:39.379 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:08:39.379 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:39.438 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:39.438 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:08:39.438 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:39.498 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:39.498 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:08:39.498 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:39.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:39.555 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:39.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.758+0000 7f8f85000640 1 Processor -- start 2024-04-25T10:08:39.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.759+0000 7f8f85000640 1 -- start start 2024-04-25T10:08:39.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.760+0000 7f8f85000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8f8015ed70 0x7f8f80168a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.760+0000 7f8f85000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f800712c0 con 0x7f8f8015d980 2024-04-25T10:08:39.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.760+0000 7f8f85000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f800715a0 con 0x7f8f800aecf0 2024-04-25T10:08:39.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.760+0000 7f8f85000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f80071ed0 con 0x7f8f8015ed70 2024-04-25T10:08:39.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.760+0000 7f8f7de00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8f8015d980 0x7f8f800ae5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:49766/0 (socket says 172.21.15.119:49766) 2024-04-25T10:08:39.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.760+0000 7f8f7de00640 1 -- 172.21.15.119:0/418181321 learned_addr learned my addr 172.21.15.119:0/418181321 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:39.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.761+0000 7f8f7d400640 1 -- 172.21.15.119:0/418181321 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162957344 0 0) 0x7f8f800712c0 con 0x7f8f8015d980 2024-04-25T10:08:39.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.761+0000 7f8f7d400640 1 -- 172.21.15.119:0/418181321 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f64003520 con 0x7f8f8015d980 2024-04-25T10:08:39.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.761+0000 7f8f7e800640 1 --2- 172.21.15.119:0/418181321 >> v2:172.21.15.132:3300/0 conn(0x7f8f8015ed70 0x7f8f80168a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.761+0000 7f8f7d400640 1 -- 172.21.15.119:0/418181321 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3899432057 0 0) 0x7f8f64003520 con 0x7f8f8015d980 2024-04-25T10:08:39.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.761+0000 7f8f7d400640 1 -- 172.21.15.119:0/418181321 >> v2:172.21.15.132:3300/0 conn(0x7f8f8015ed70 msgr2=0x7f8f80168a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:39.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.761+0000 7f8f7d400640 1 --2- 172.21.15.119:0/418181321 >> v2:172.21.15.132:3300/0 conn(0x7f8f8015ed70 0x7f8f80168a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.761+0000 7f8f7d400640 1 -- 172.21.15.119:0/418181321 >> v1:172.21.15.132:6789/0 conn(0x7f8f800aecf0 legacy=0x7f8f800cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:39.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.761+0000 7f8f7d400640 1 -- 172.21.15.119:0/418181321 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f80071880 con 0x7f8f8015d980 2024-04-25T10:08:39.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.761+0000 7f8f7d400640 1 -- 172.21.15.119:0/418181321 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f8f70001670 con 0x7f8f8015d980 2024-04-25T10:08:39.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.761+0000 7f8f7d400640 1 -- 172.21.15.119:0/418181321 >> v1:172.21.15.119:6789/0 conn(0x7f8f8015d980 legacy=0x7f8f800ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.761+0000 7f8f7d400640 1 --2- 172.21.15.119:0/418181321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f640039b0 0x7f8f64003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.761+0000 7f8f7d400640 1 --2- 172.21.15.119:0/418181321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f64004800 0x7f8f64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.761+0000 7f8f7d400640 1 --2- 172.21.15.119:0/418181321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f640052e0 0x7f8f6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.762+0000 7f8f7d400640 1 -- 172.21.15.119:0/418181321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f80071ed0 con 0x7f8f640039b0 2024-04-25T10:08:39.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.762+0000 7f8f7d400640 1 -- 172.21.15.119:0/418181321 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6400c190 con 0x7f8f64004800 2024-04-25T10:08:39.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.762+0000 7f8f7d400640 1 -- 172.21.15.119:0/418181321 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6400c390 con 0x7f8f640052e0 2024-04-25T10:08:39.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.762+0000 7f8f7de00640 1 --2- 172.21.15.119:0/418181321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f640039b0 0x7f8f64003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.762+0000 7f8f7de00640 1 -- 172.21.15.119:0/418181321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f640052e0 msgr2=0x7f8f6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:39.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.762+0000 7f8f7de00640 1 --2- 172.21.15.119:0/418181321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f640052e0 0x7f8f6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.762+0000 7f8f7de00640 1 -- 172.21.15.119:0/418181321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f64004800 msgr2=0x7f8f64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:39.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.762+0000 7f8f7de00640 1 --2- 172.21.15.119:0/418181321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f64004800 0x7f8f64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.762+0000 7f8f7de00640 1 -- 172.21.15.119:0/418181321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f64003520 con 0x7f8f640039b0 2024-04-25T10:08:39.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.762+0000 7f8f7e800640 1 -- 172.21.15.119:0/418181321 reap_dead start 2024-04-25T10:08:39.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.763+0000 7f8f7de00640 1 --2- 172.21.15.119:0/418181321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f640039b0 0x7f8f64003db0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000c210 tx=0x7f8f64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3983a15cfb9ffa2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.763+0000 7f8f7d400640 1 -- 172.21.15.119:0/418181321 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f70007d10 con 0x7f8f640039b0 2024-04-25T10:08:39.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.763+0000 7f8f7d400640 1 -- 172.21.15.119:0/418181321 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f70005ad0 con 0x7f8f640039b0 2024-04-25T10:08:39.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.763+0000 7f8f7d400640 1 -- 172.21.15.119:0/418181321 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f70007070 con 0x7f8f640039b0 2024-04-25T10:08:39.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.764+0000 7f8f85000640 1 -- 172.21.15.119:0/418181321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f640039b0 msgr2=0x7f8f64003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.764+0000 7f8f85000640 1 --2- 172.21.15.119:0/418181321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f640039b0 0x7f8f64003db0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000c210 tx=0x7f8f64005830 comp rx=0 tx=0).stop 2024-04-25T10:08:39.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.764+0000 7f8f85000640 1 -- 172.21.15.119:0/418181321 shutdown_connections 2024-04-25T10:08:39.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.764+0000 7f8f85000640 1 --2- 172.21.15.119:0/418181321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f640039b0 0x7f8f64003db0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.764+0000 7f8f85000640 1 -- 172.21.15.119:0/418181321 >> 172.21.15.119:0/418181321 conn(0x7f8f8006aba0 msgr2=0x7f8f8006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:39.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.764+0000 7f8f85000640 1 -- 172.21.15.119:0/418181321 shutdown_connections 2024-04-25T10:08:39.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.764+0000 7f8f85000640 1 -- 172.21.15.119:0/418181321 wait complete. 2024-04-25T10:08:39.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.766+0000 7f8f85000640 1 Processor -- start 2024-04-25T10:08:39.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.766+0000 7f8f85000640 1 -- start start 2024-04-25T10:08:39.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.767+0000 7f8f85000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f640039b0 0x7f8f8015a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.767+0000 7f8f85000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f8015b150 0x7f8f8015fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.767+0000 7f8f85000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f8014f500 0x7f8f80160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.767+0000 7f8f85000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f800ae760 con 0x7f8f8015b150 2024-04-25T10:08:39.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.767+0000 7f8f85000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f800ae5e0 con 0x7f8f8014f500 2024-04-25T10:08:39.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.767+0000 7f8f85000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f800ae8e0 con 0x7f8f640039b0 2024-04-25T10:08:39.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.767+0000 7f8f75400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f8015b150 0x7f8f8015fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.767+0000 7f8f75400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f8015b150 0x7f8f8015fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42688/0 (socket says 172.21.15.119:42688) 2024-04-25T10:08:39.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.767+0000 7f8f75400640 1 -- 172.21.15.119:0/4144510572 learned_addr learned my addr 172.21.15.119:0/4144510572 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:39.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.768+0000 7f8f7e800640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f8014f500 0x7f8f80160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.768+0000 7f8f7de00640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f640039b0 0x7f8f8015a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.768+0000 7f8f75400640 1 -- 172.21.15.119:0/4144510572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f640039b0 msgr2=0x7f8f8015a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.768+0000 7f8f75400640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f640039b0 0x7f8f8015a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.768+0000 7f8f75400640 1 -- 172.21.15.119:0/4144510572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f8014f500 msgr2=0x7f8f80160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.768+0000 7f8f75400640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f8014f500 0x7f8f80160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.768+0000 7f8f75400640 1 -- 172.21.15.119:0/4144510572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800c4f40 con 0x7f8f8015b150 2024-04-25T10:08:39.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.768+0000 7f8f7de00640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f640039b0 0x7f8f8015a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:39.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.768+0000 7f8f75400640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f8015b150 0x7f8f8015fe50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800c3f0 tx=0x7f8f6800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.768+0000 7f8f76a00640 1 -- 172.21.15.119:0/4144510572 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68011790 con 0x7f8f8015b150 2024-04-25T10:08:39.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.768+0000 7f8f76a00640 1 -- 172.21.15.119:0/4144510572 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f68017070 con 0x7f8f8015b150 2024-04-25T10:08:39.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.768+0000 7f8f85000640 1 -- 172.21.15.119:0/4144510572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f800c51e0 con 0x7f8f8015b150 2024-04-25T10:08:39.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.768+0000 7f8f76a00640 1 -- 172.21.15.119:0/4144510572 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68011b70 con 0x7f8f8015b150 2024-04-25T10:08:39.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.769+0000 7f8f85000640 1 -- 172.21.15.119:0/4144510572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8015bc70 con 0x7f8f8015b150 2024-04-25T10:08:39.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.770+0000 7f8f85000640 1 -- 172.21.15.119:0/4144510572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f800afa70 con 0x7f8f8015b150 2024-04-25T10:08:39.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.770+0000 7f8f76a00640 1 -- 172.21.15.119:0/4144510572 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f68011d10 con 0x7f8f8015b150 2024-04-25T10:08:39.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.771+0000 7f8f76a00640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8f5c06b500 0x7f8f5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.771+0000 7f8f76a00640 1 -- 172.21.15.119:0/4144510572 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f68089eb0 con 0x7f8f8015b150 2024-04-25T10:08:39.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.772+0000 7f8f7de00640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8f5c06b500 0x7f8f5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.773+0000 7f8f7de00640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8f5c06b500 0x7f8f5c06d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8f70032000 tx=0x7f8f70034040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:39.778+0000 7f8f76a00640 1 -- 172.21.15.119:0/4144510572 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f68058e80 con 0x7f8f8015b150 2024-04-25T10:08:40.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.012+0000 7f8f85000640 1 -- 172.21.15.119:0/4144510572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f800749b0 con 0x7f8f8015b150 2024-04-25T10:08:40.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.013+0000 7f8f76a00640 1 -- 172.21.15.119:0/4144510572 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f8f68058780 con 0x7f8f8015b150 2024-04-25T10:08:40.019 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:40.019 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1554372269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1554372269},{"type":"v1","addr":"172.21.15.119:6837","nonce":1554372269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4699,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4250304148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4250304148},{"type":"v1","addr":"172.21.15.132:6834","nonce":4250304148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/637229574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":637229574},{"type":"v1","addr":"172.21.15.132:6835","nonce":637229574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:14.553437+0000","modified":"2024-04-25T10:08:17.586270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6836/3585315292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3585315292},{"type":"v1","addr":"172.21.15.119:6836","nonce":3585315292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:08:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 16 2024-04-25T10:08:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.015+0000 7f8f85000640 1 -- 172.21.15.119:0/4144510572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8f5c06b500 msgr2=0x7f8f5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.015+0000 7f8f85000640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8f5c06b500 0x7f8f5c06d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8f70032000 tx=0x7f8f70034040 comp rx=0 tx=0).stop 2024-04-25T10:08:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.015+0000 7f8f85000640 1 -- 172.21.15.119:0/4144510572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f8015b150 msgr2=0x7f8f8015fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.015+0000 7f8f85000640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f8015b150 0x7f8f8015fe50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800c3f0 tx=0x7f8f6800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.015+0000 7f8f85000640 1 -- 172.21.15.119:0/4144510572 shutdown_connections 2024-04-25T10:08:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.015+0000 7f8f85000640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8f5c06b500 0x7f8f5c06d9c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.015+0000 7f8f85000640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f8014f500 0x7f8f80160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.015+0000 7f8f85000640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f8015b150 0x7f8f8015fe50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.015+0000 7f8f85000640 1 --2- 172.21.15.119:0/4144510572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f640039b0 0x7f8f8015a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.015+0000 7f8f85000640 1 -- 172.21.15.119:0/4144510572 >> 172.21.15.119:0/4144510572 conn(0x7f8f8006aba0 msgr2=0x7f8f800af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.016+0000 7f8f85000640 1 -- 172.21.15.119:0/4144510572 shutdown_connections 2024-04-25T10:08:40.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.016+0000 7f8f85000640 1 -- 172.21.15.119:0/4144510572 wait complete. 2024-04-25T10:08:40.041 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:40.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.235+0000 7f755b200640 1 Processor -- start 2024-04-25T10:08:40.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.235+0000 7f755b200640 1 -- start start 2024-04-25T10:08:40.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.237+0000 7f755b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75540cf090 0x7f75540d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.237+0000 7f755b200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f75540d09c0 0x7f75540cd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.237+0000 7f755b200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75540d0e50 con 0x7f755415a6c0 2024-04-25T10:08:40.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.237+0000 7f755b200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75540cf510 con 0x7f75540d09c0 2024-04-25T10:08:40.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.237+0000 7f755b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7554070240 con 0x7f75540cf090 2024-04-25T10:08:40.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.237+0000 7f7559e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f755415a6c0 0x7f755406f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:49780/0 (socket says 172.21.15.119:49780) 2024-04-25T10:08:40.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.237+0000 7f7559e00640 1 -- 172.21.15.119:0/3079157905 learned_addr learned my addr 172.21.15.119:0/3079157905 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:40.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.237+0000 7f7559400640 1 --2- 172.21.15.119:0/3079157905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75540cf090 0x7f75540d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.238+0000 7f755a800640 1 --2- 172.21.15.119:0/3079157905 >> v2:172.21.15.132:3300/0 conn(0x7f75540d09c0 0x7f75540cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.238+0000 7f7559400640 1 -- 172.21.15.119:0/3079157905 >> v2:172.21.15.132:3300/0 conn(0x7f75540d09c0 msgr2=0x7f75540cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:40.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.238+0000 7f7559400640 1 --2- 172.21.15.119:0/3079157905 >> v2:172.21.15.132:3300/0 conn(0x7f75540d09c0 0x7f75540cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.238+0000 7f7559400640 1 -- 172.21.15.119:0/3079157905 >> v1:172.21.15.119:6789/0 conn(0x7f755415a6c0 legacy=0x7f755406f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.238+0000 7f7559400640 1 -- 172.21.15.119:0/3079157905 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755406fc80 con 0x7f75540cf090 2024-04-25T10:08:40.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.240+0000 7f7559400640 1 --2- 172.21.15.119:0/3079157905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75540cf090 0x7f75540d0480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f75480097a0 tx=0x7f754802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdd493d8471da80e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.242+0000 7f7558a00640 1 -- 172.21.15.119:0/3079157905 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548037070 con 0x7f75540cf090 2024-04-25T10:08:40.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.242+0000 7f7558a00640 1 -- 172.21.15.119:0/3079157905 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7548032440 con 0x7f75540cf090 2024-04-25T10:08:40.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.242+0000 7f7558a00640 1 -- 172.21.15.119:0/3079157905 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548032810 con 0x7f75540cf090 2024-04-25T10:08:40.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.242+0000 7f755b200640 1 -- 172.21.15.119:0/3079157905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75540cf090 msgr2=0x7f75540d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.242+0000 7f755b200640 1 --2- 172.21.15.119:0/3079157905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75540cf090 0x7f75540d0480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f75480097a0 tx=0x7f754802a180 comp rx=0 tx=0).stop 2024-04-25T10:08:40.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.242+0000 7f755b200640 1 -- 172.21.15.119:0/3079157905 shutdown_connections 2024-04-25T10:08:40.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.242+0000 7f755b200640 1 --2- 172.21.15.119:0/3079157905 >> v2:172.21.15.132:3300/0 conn(0x7f75540d09c0 0x7f75540cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.242+0000 7f755b200640 1 --2- 172.21.15.119:0/3079157905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75540cf090 0x7f75540d0480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.242+0000 7f755b200640 1 -- 172.21.15.119:0/3079157905 >> 172.21.15.119:0/3079157905 conn(0x7f75540726b0 msgr2=0x7f7554099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:40.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.243+0000 7f755b200640 1 -- 172.21.15.119:0/3079157905 shutdown_connections 2024-04-25T10:08:40.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.243+0000 7f755b200640 1 -- 172.21.15.119:0/3079157905 wait complete. 2024-04-25T10:08:40.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.244+0000 7f755b200640 1 Processor -- start 2024-04-25T10:08:40.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.244+0000 7f755b200640 1 -- start start 2024-04-25T10:08:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.245+0000 7f755b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75540cf090 0x7f75540b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.245+0000 7f755b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75540d09c0 0x7f7554157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.245+0000 7f755b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f755415a6c0 0x7f7554157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.245+0000 7f755b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75540ad700 con 0x7f75540d09c0 2024-04-25T10:08:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.245+0000 7f755b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75540ad420 con 0x7f755415a6c0 2024-04-25T10:08:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.245+0000 7f755b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755415aaa0 con 0x7f75540cf090 2024-04-25T10:08:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.246+0000 7f7559400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75540d09c0 0x7f7554157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.246+0000 7f7559400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75540d09c0 0x7f7554157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42700/0 (socket says 172.21.15.119:42700) 2024-04-25T10:08:40.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.246+0000 7f7559400640 1 -- 172.21.15.119:0/1134234731 learned_addr learned my addr 172.21.15.119:0/1134234731 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:40.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.246+0000 7f7559e00640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75540cf090 0x7f75540b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.246+0000 7f755a800640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f755415a6c0 0x7f7554157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.246+0000 7f7559400640 1 -- 172.21.15.119:0/1134234731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75540cf090 msgr2=0x7f75540b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.246+0000 7f7559400640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75540cf090 0x7f75540b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.246+0000 7f7559400640 1 -- 172.21.15.119:0/1134234731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f755415a6c0 msgr2=0x7f7554157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.246+0000 7f7559400640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f755415a6c0 0x7f7554157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.246+0000 7f7559400640 1 -- 172.21.15.119:0/1134234731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75540c7740 con 0x7f75540d09c0 2024-04-25T10:08:40.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.246+0000 7f755a800640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f755415a6c0 0x7f7554157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.247+0000 7f7559400640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75540d09c0 0x7f7554157970 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f754802a1c0 tx=0x7f75480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.247+0000 7f754e000640 1 -- 172.21.15.119:0/1134234731 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548041020 con 0x7f75540d09c0 2024-04-25T10:08:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.247+0000 7f754e000640 1 -- 172.21.15.119:0/1134234731 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7548030700 con 0x7f75540d09c0 2024-04-25T10:08:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.247+0000 7f755b200640 1 -- 172.21.15.119:0/1134234731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75540c79b0 con 0x7f75540d09c0 2024-04-25T10:08:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.247+0000 7f754e000640 1 -- 172.21.15.119:0/1134234731 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548037040 con 0x7f75540d09c0 2024-04-25T10:08:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.247+0000 7f755b200640 1 -- 172.21.15.119:0/1134234731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75540c7f70 con 0x7f75540d09c0 2024-04-25T10:08:40.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.248+0000 7f755b200640 1 -- 172.21.15.119:0/1134234731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75540c8460 con 0x7f75540d09c0 2024-04-25T10:08:40.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.249+0000 7f754e000640 1 -- 172.21.15.119:0/1134234731 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7548044020 con 0x7f75540d09c0 2024-04-25T10:08:40.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.250+0000 7f754e000640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f753406b4d0 0x7f753406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.250+0000 7f754e000640 1 -- 172.21.15.119:0/1134234731 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75480aaa10 con 0x7f75540d09c0 2024-04-25T10:08:40.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.250+0000 7f7559e00640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f753406b4d0 0x7f753406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.251+0000 7f7559e00640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f753406b4d0 0x7f753406d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7540000f60 tx=0x7f7540009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.254+0000 7f754e000640 1 -- 172.21.15.119:0/1134234731 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f754804b030 con 0x7f75540d09c0 2024-04-25T10:08:40.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.454+0000 7f755b200640 1 -- 172.21.15.119:0/1134234731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7554067850 con 0x7f75540d09c0 2024-04-25T10:08:40.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.457+0000 7f754e000640 1 -- 172.21.15.119:0/1134234731 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16705 (secure 0 0 0) 0x7f7548079880 con 0x7f75540d09c0 2024-04-25T10:08:40.463 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:40.463 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":33,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:16.095956+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:12.268457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:13.546259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.460+0000 7f755b200640 1 -- 172.21.15.119:0/1134234731 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f753406b4d0 msgr2=0x7f753406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.460+0000 7f755b200640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f753406b4d0 0x7f753406d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7540000f60 tx=0x7f7540009040 comp rx=0 tx=0).stop 2024-04-25T10:08:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.460+0000 7f755b200640 1 -- 172.21.15.119:0/1134234731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75540d09c0 msgr2=0x7f7554157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.460+0000 7f755b200640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75540d09c0 0x7f7554157970 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f754802a1c0 tx=0x7f75480046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.460+0000 7f755b200640 1 -- 172.21.15.119:0/1134234731 shutdown_connections 2024-04-25T10:08:40.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.460+0000 7f755b200640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f753406b4d0 0x7f753406d990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.460+0000 7f755b200640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f755415a6c0 0x7f7554157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.460+0000 7f755b200640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75540d09c0 0x7f7554157970 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.460+0000 7f755b200640 1 --2- 172.21.15.119:0/1134234731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75540cf090 0x7f75540b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.460+0000 7f755b200640 1 -- 172.21.15.119:0/1134234731 >> 172.21.15.119:0/1134234731 conn(0x7f75540726b0 msgr2=0x7f7554159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:40.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.460+0000 7f755b200640 1 -- 172.21.15.119:0/1134234731 shutdown_connections 2024-04-25T10:08:40.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.460+0000 7f755b200640 1 -- 172.21.15.119:0/1134234731 wait complete. 2024-04-25T10:08:40.486 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:12.268457+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:13.546259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:40.486 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:40.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.687+0000 7fa268a00640 1 Processor -- start 2024-04-25T10:08:40.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.687+0000 7fa268a00640 1 -- start start 2024-04-25T10:08:40.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.689+0000 7fa268a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2640cf900 0x7fa2640b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.689+0000 7fa268a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa2640b2b90 0x7fa264162f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.689+0000 7fa268a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa264071350 con 0x7fa2640cfce0 2024-04-25T10:08:40.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.689+0000 7fa268a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa264071630 con 0x7fa2640b2b90 2024-04-25T10:08:40.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.689+0000 7fa268a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa264071910 con 0x7fa2640cf900 2024-04-25T10:08:40.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.689+0000 7fa262800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa2640b2b90 0x7fa264162f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.689+0000 7fa262800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa2640b2b90 0x7fa264162f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42712/0 (socket says 172.21.15.119:42712) 2024-04-25T10:08:40.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.689+0000 7fa262800640 1 -- 172.21.15.119:0/3826890550 learned_addr learned my addr 172.21.15.119:0/3826890550 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:40.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.690+0000 7fa261e00640 1 --2- 172.21.15.119:0/3826890550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2640cf900 0x7fa2640b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.690+0000 7fa262800640 1 -- 172.21.15.119:0/3826890550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2640cf900 msgr2=0x7fa2640b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:40.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.690+0000 7fa262800640 1 --2- 172.21.15.119:0/3826890550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2640cf900 0x7fa2640b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.690+0000 7fa262800640 1 -- 172.21.15.119:0/3826890550 >> v1:172.21.15.132:6789/0 conn(0x7fa2640cfce0 legacy=0x7fa264162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.690+0000 7fa262800640 1 -- 172.21.15.119:0/3826890550 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa264071bf0 con 0x7fa2640b2b90 2024-04-25T10:08:40.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.691+0000 7fa262800640 1 --2- 172.21.15.119:0/3826890550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2640b2b90 0x7fa264162f90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009e50 tx=0x7fa25c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7f5c80bf272e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.692+0000 7fa261400640 1 -- 172.21.15.119:0/3826890550 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c036070 con 0x7fa2640b2b90 2024-04-25T10:08:40.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.692+0000 7fa261400640 1 -- 172.21.15.119:0/3826890550 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25c031600 con 0x7fa2640b2b90 2024-04-25T10:08:40.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.692+0000 7fa261400640 1 -- 172.21.15.119:0/3826890550 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0319d0 con 0x7fa2640b2b90 2024-04-25T10:08:40.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.692+0000 7fa268a00640 1 -- 172.21.15.119:0/3826890550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2640b2b90 msgr2=0x7fa264162f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.692+0000 7fa268a00640 1 --2- 172.21.15.119:0/3826890550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2640b2b90 0x7fa264162f90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009e50 tx=0x7fa25c02a680 comp rx=0 tx=0).stop 2024-04-25T10:08:40.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.692+0000 7fa268a00640 1 -- 172.21.15.119:0/3826890550 shutdown_connections 2024-04-25T10:08:40.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.692+0000 7fa268a00640 1 --2- 172.21.15.119:0/3826890550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2640b2b90 0x7fa264162f90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.692+0000 7fa268a00640 1 --2- 172.21.15.119:0/3826890550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2640cf900 0x7fa2640b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.692+0000 7fa268a00640 1 -- 172.21.15.119:0/3826890550 >> 172.21.15.119:0/3826890550 conn(0x7fa26406aba0 msgr2=0x7fa26406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:40.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.692+0000 7fa268a00640 1 -- 172.21.15.119:0/3826890550 shutdown_connections 2024-04-25T10:08:40.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.693+0000 7fa268a00640 1 -- 172.21.15.119:0/3826890550 wait complete. 2024-04-25T10:08:40.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.694+0000 7fa268a00640 1 Processor -- start 2024-04-25T10:08:40.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.694+0000 7fa268a00640 1 -- start start 2024-04-25T10:08:40.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.695+0000 7fa268a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2640b2b90 0x7fa264155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.695+0000 7fa268a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2640cf900 0x7fa264159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.695+0000 7fa268a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2640cfce0 0x7fa26415cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.695+0000 7fa268a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa264165c30 con 0x7fa2640cfce0 2024-04-25T10:08:40.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.695+0000 7fa268a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2640b3e00 con 0x7fa2640b2b90 2024-04-25T10:08:40.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.695+0000 7fa268a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa264165db0 con 0x7fa2640cf900 2024-04-25T10:08:40.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.696+0000 7fa262800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2640cfce0 0x7fa26415cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.696+0000 7fa262800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2640cfce0 0x7fa26415cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42722/0 (socket says 172.21.15.119:42722) 2024-04-25T10:08:40.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.696+0000 7fa262800640 1 -- 172.21.15.119:0/3134044954 learned_addr learned my addr 172.21.15.119:0/3134044954 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:40.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.696+0000 7fa261e00640 1 --2- 172.21.15.119:0/3134044954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2640b2b90 0x7fa264155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.696+0000 7fa25be00640 1 --2- 172.21.15.119:0/3134044954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2640cf900 0x7fa264159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.696+0000 7fa262800640 1 -- 172.21.15.119:0/3134044954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2640cf900 msgr2=0x7fa264159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.696+0000 7fa262800640 1 --2- 172.21.15.119:0/3134044954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2640cf900 0x7fa264159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.696+0000 7fa262800640 1 -- 172.21.15.119:0/3134044954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2640b2b90 msgr2=0x7fa264155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.696+0000 7fa262800640 1 --2- 172.21.15.119:0/3134044954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2640b2b90 0x7fa264155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.696+0000 7fa262800640 1 -- 172.21.15.119:0/3134044954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2641eafb0 con 0x7fa2640cfce0 2024-04-25T10:08:40.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.697+0000 7fa262800640 1 --2- 172.21.15.119:0/3134044954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2640cfce0 0x7fa26415cdf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009e50 tx=0x7fa25c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.697+0000 7fa25a000640 1 -- 172.21.15.119:0/3134044954 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c036050 con 0x7fa2640cfce0 2024-04-25T10:08:40.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.697+0000 7fa25a000640 1 -- 172.21.15.119:0/3134044954 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25c042070 con 0x7fa2640cfce0 2024-04-25T10:08:40.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.697+0000 7fa268a00640 1 -- 172.21.15.119:0/3134044954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2641eb290 con 0x7fa2640cfce0 2024-04-25T10:08:40.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.697+0000 7fa25a000640 1 -- 172.21.15.119:0/3134044954 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c009510 con 0x7fa2640cfce0 2024-04-25T10:08:40.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.697+0000 7fa268a00640 1 -- 172.21.15.119:0/3134044954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2641eb850 con 0x7fa2640cfce0 2024-04-25T10:08:40.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.698+0000 7fa268a00640 1 -- 172.21.15.119:0/3134044954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2640750b0 con 0x7fa2640cfce0 2024-04-25T10:08:40.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.699+0000 7fa25a000640 1 -- 172.21.15.119:0/3134044954 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa25c007bd0 con 0x7fa2640cfce0 2024-04-25T10:08:40.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.700+0000 7fa25a000640 1 --2- 172.21.15.119:0/3134044954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa24006b4d0 0x7fa24006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.700+0000 7fa25a000640 1 -- 172.21.15.119:0/3134044954 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa25c0ab040 con 0x7fa2640cfce0 2024-04-25T10:08:40.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.701+0000 7fa261e00640 1 --2- 172.21.15.119:0/3134044954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa24006b4d0 0x7fa24006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.701+0000 7fa261e00640 1 --2- 172.21.15.119:0/3134044954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa24006b4d0 0x7fa24006d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa254006fd0 tx=0x7fa254008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.704+0000 7fa25a000640 1 -- 172.21.15.119:0/3134044954 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa25c079f50 con 0x7fa2640cfce0 2024-04-25T10:08:40.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.902+0000 7fa268a00640 1 -- 172.21.15.119:0/3134044954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2641eba20 con 0x7fa2640cfce0 2024-04-25T10:08:40.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.903+0000 7fa25a000640 1 -- 172.21.15.119:0/3134044954 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa25c079850 con 0x7fa2640cfce0 2024-04-25T10:08:40.909 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:08:40.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.905+0000 7fa268a00640 1 -- 172.21.15.119:0/3134044954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa24006b4d0 msgr2=0x7fa24006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.905+0000 7fa268a00640 1 --2- 172.21.15.119:0/3134044954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa24006b4d0 0x7fa24006d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa254006fd0 tx=0x7fa254008040 comp rx=0 tx=0).stop 2024-04-25T10:08:40.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.905+0000 7fa268a00640 1 -- 172.21.15.119:0/3134044954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2640cfce0 msgr2=0x7fa26415cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.905+0000 7fa268a00640 1 --2- 172.21.15.119:0/3134044954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2640cfce0 0x7fa26415cdf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009e50 tx=0x7fa25c004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:40.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.905+0000 7fa268a00640 1 -- 172.21.15.119:0/3134044954 shutdown_connections 2024-04-25T10:08:40.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.905+0000 7fa268a00640 1 --2- 172.21.15.119:0/3134044954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa24006b4d0 0x7fa24006d990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.905+0000 7fa268a00640 1 --2- 172.21.15.119:0/3134044954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2640cfce0 0x7fa26415cdf0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.905+0000 7fa268a00640 1 --2- 172.21.15.119:0/3134044954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2640cf900 0x7fa264159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.905+0000 7fa268a00640 1 --2- 172.21.15.119:0/3134044954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2640b2b90 0x7fa264155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.905+0000 7fa268a00640 1 -- 172.21.15.119:0/3134044954 >> 172.21.15.119:0/3134044954 conn(0x7fa26406aba0 msgr2=0x7fa264157ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:40.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.906+0000 7fa268a00640 1 -- 172.21.15.119:0/3134044954 shutdown_connections 2024-04-25T10:08:40.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:40.906+0000 7fa268a00640 1 -- 172.21.15.119:0/3134044954 wait complete. 2024-04-25T10:08:40.931 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:08:41.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.124+0000 7fa1d1000640 1 Processor -- start 2024-04-25T10:08:41.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.125+0000 7fa1d1000640 1 -- start start 2024-04-25T10:08:41.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.126+0000 7fa1d1000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa1cc0cc480 0x7fa1cc0b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.126+0000 7fa1d1000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1cc0b3b40 0x7fa1cc0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.126+0000 7fa1d1000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1cc0cb6c0 con 0x7fa1cc15ab70 2024-04-25T10:08:41.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.126+0000 7fa1d1000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1cc15af50 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.126+0000 7fa1d1000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1cc0cbff0 con 0x7fa1cc0cc480 2024-04-25T10:08:41.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.127+0000 7fa1ca800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1cc0b3b40 0x7fa1cc0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.127+0000 7fa1ca800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1cc0b3b40 0x7fa1cc0b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51422/0 (socket says 172.21.15.119:51422) 2024-04-25T10:08:41.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.127+0000 7fa1ca800640 1 -- 172.21.15.119:0/3236828462 learned_addr learned my addr 172.21.15.119:0/3236828462 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:41.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.127+0000 7fa1c9400640 1 --2- 172.21.15.119:0/3236828462 >> v2:172.21.15.132:3300/0 conn(0x7fa1cc0cc480 0x7fa1cc0b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.127+0000 7fa1ca800640 1 -- 172.21.15.119:0/3236828462 >> v2:172.21.15.132:3300/0 conn(0x7fa1cc0cc480 msgr2=0x7fa1cc0b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:41.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.127+0000 7fa1ca800640 1 --2- 172.21.15.119:0/3236828462 >> v2:172.21.15.132:3300/0 conn(0x7fa1cc0cc480 0x7fa1cc0b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.127+0000 7fa1ca800640 1 -- 172.21.15.119:0/3236828462 >> v1:172.21.15.132:6789/0 conn(0x7fa1cc15ab70 legacy=0x7fa1cc0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.127+0000 7fa1ca800640 1 -- 172.21.15.119:0/3236828462 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc0cb9a0 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.127+0000 7fa1c9400640 1 --2- 172.21.15.119:0/3236828462 >> v2:172.21.15.132:3300/0 conn(0x7fa1cc0cc480 0x7fa1cc0b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.128+0000 7fa1ca800640 1 --2- 172.21.15.119:0/3236828462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1cc0b3b40 0x7fa1cc0b3fd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0cfe70 tx=0x7fa1c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace6d8f81bc45e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.129+0000 7fa1c8a00640 1 -- 172.21.15.119:0/3236828462 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4037070 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.129+0000 7fa1c8a00640 1 -- 172.21.15.119:0/3236828462 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c4032440 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.129+0000 7fa1c8a00640 1 -- 172.21.15.119:0/3236828462 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4032810 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.129+0000 7fa1d1000640 1 -- 172.21.15.119:0/3236828462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1cc0b3b40 msgr2=0x7fa1cc0b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.129+0000 7fa1d1000640 1 --2- 172.21.15.119:0/3236828462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1cc0b3b40 0x7fa1cc0b3fd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0cfe70 tx=0x7fa1c402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.129+0000 7fa1d1000640 1 -- 172.21.15.119:0/3236828462 shutdown_connections 2024-04-25T10:08:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.129+0000 7fa1d1000640 1 --2- 172.21.15.119:0/3236828462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1cc0b3b40 0x7fa1cc0b3fd0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.129+0000 7fa1d1000640 1 --2- 172.21.15.119:0/3236828462 >> v2:172.21.15.132:3300/0 conn(0x7fa1cc0cc480 0x7fa1cc0b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.129+0000 7fa1d1000640 1 -- 172.21.15.119:0/3236828462 >> 172.21.15.119:0/3236828462 conn(0x7fa1cc0d07d0 msgr2=0x7fa1cc0d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.130+0000 7fa1d1000640 1 -- 172.21.15.119:0/3236828462 shutdown_connections 2024-04-25T10:08:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.130+0000 7fa1d1000640 1 -- 172.21.15.119:0/3236828462 wait complete. 2024-04-25T10:08:41.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.132+0000 7fa1d1000640 1 Processor -- start 2024-04-25T10:08:41.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.132+0000 7fa1d1000640 1 -- start start 2024-04-25T10:08:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.133+0000 7fa1d1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1cc0b3b40 0x7fa1cc154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.133+0000 7fa1d1000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1cc0cc480 0x7fa1cc1f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.133+0000 7fa1d1000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1cc15ab70 0x7fa1cc0c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.133+0000 7fa1d1000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0ce7e0 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.133+0000 7fa1d1000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0cb370 con 0x7fa1cc0cc480 2024-04-25T10:08:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.133+0000 7fa1d1000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0ce960 con 0x7fa1cc15ab70 2024-04-25T10:08:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.133+0000 7fa1c9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1cc0b3b40 0x7fa1cc154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.134+0000 7fa1c9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1cc0b3b40 0x7fa1cc154630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51434/0 (socket says 172.21.15.119:51434) 2024-04-25T10:08:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.134+0000 7fa1c9e00640 1 -- 172.21.15.119:0/3900648391 learned_addr learned my addr 172.21.15.119:0/3900648391 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.134+0000 7fa1c9400640 1 --2- 172.21.15.119:0/3900648391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1cc0cc480 0x7fa1cc1f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.134+0000 7fa1ca800640 1 --2- 172.21.15.119:0/3900648391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1cc15ab70 0x7fa1cc0c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.134+0000 7fa1c9e00640 1 -- 172.21.15.119:0/3900648391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1cc15ab70 msgr2=0x7fa1cc0c33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.134+0000 7fa1c9e00640 1 --2- 172.21.15.119:0/3900648391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1cc15ab70 0x7fa1cc0c33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.134+0000 7fa1c9e00640 1 -- 172.21.15.119:0/3900648391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1cc0cc480 msgr2=0x7fa1cc1f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.134+0000 7fa1c9e00640 1 --2- 172.21.15.119:0/3900648391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1cc0cc480 0x7fa1cc1f7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.134+0000 7fa1c9e00640 1 -- 172.21.15.119:0/3900648391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc1f81f0 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.134+0000 7fa1c9e00640 1 --2- 172.21.15.119:0/3900648391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1cc0b3b40 0x7fa1cc154630 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00c3f0 tx=0x7fa1bc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.135+0000 7fa1c2000640 1 -- 172.21.15.119:0/3900648391 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0052d0 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.135+0000 7fa1c2000640 1 -- 172.21.15.119:0/3900648391 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc005490 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.135+0000 7fa1c2000640 1 -- 172.21.15.119:0/3900648391 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc007260 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.135+0000 7fa1d1000640 1 -- 172.21.15.119:0/3900648391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1cc15cb90 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.135+0000 7fa1d1000640 1 -- 172.21.15.119:0/3900648391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1cc1f74b0 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.136+0000 7fa1d1000640 1 -- 172.21.15.119:0/3900648391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1cc06cc40 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.137+0000 7fa1c2000640 1 -- 172.21.15.119:0/3900648391 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1bc007400 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.138+0000 7fa1c2000640 1 --2- 172.21.15.119:0/3900648391 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa1a806b4d0 0x7fa1a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.138+0000 7fa1c2000640 1 -- 172.21.15.119:0/3900648391 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1bc089780 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.138+0000 7fa1c9400640 1 --2- 172.21.15.119:0/3900648391 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa1a806b4d0 0x7fa1a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.139+0000 7fa1c9400640 1 --2- 172.21.15.119:0/3900648391 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa1a806b4d0 0x7fa1a806d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc15c650 tx=0x7fa1b40084d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.143+0000 7fa1c2000640 1 -- 172.21.15.119:0/3900648391 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1bc058560 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.341+0000 7fa1d1000640 1 -- 172.21.15.119:0/3900648391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1cc0750b0 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.342+0000 7fa1c2000640 1 -- 172.21.15.119:0/3900648391 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1bc057e60 con 0x7fa1cc0b3b40 2024-04-25T10:08:41.348 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:08:41.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.344+0000 7fa1d1000640 1 -- 172.21.15.119:0/3900648391 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa1a806b4d0 msgr2=0x7fa1a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.344+0000 7fa1d1000640 1 --2- 172.21.15.119:0/3900648391 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa1a806b4d0 0x7fa1a806d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc15c650 tx=0x7fa1b40084d0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.344+0000 7fa1d1000640 1 -- 172.21.15.119:0/3900648391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1cc0b3b40 msgr2=0x7fa1cc154630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.344+0000 7fa1d1000640 1 --2- 172.21.15.119:0/3900648391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1cc0b3b40 0x7fa1cc154630 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00c3f0 tx=0x7fa1bc00c8b0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.344+0000 7fa1d1000640 1 -- 172.21.15.119:0/3900648391 shutdown_connections 2024-04-25T10:08:41.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.344+0000 7fa1d1000640 1 --2- 172.21.15.119:0/3900648391 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa1a806b4d0 0x7fa1a806d990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.344+0000 7fa1d1000640 1 --2- 172.21.15.119:0/3900648391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1cc15ab70 0x7fa1cc0c33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.344+0000 7fa1d1000640 1 --2- 172.21.15.119:0/3900648391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1cc0cc480 0x7fa1cc1f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.344+0000 7fa1d1000640 1 --2- 172.21.15.119:0/3900648391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1cc0b3b40 0x7fa1cc154630 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.344+0000 7fa1d1000640 1 -- 172.21.15.119:0/3900648391 >> 172.21.15.119:0/3900648391 conn(0x7fa1cc0d07d0 msgr2=0x7fa1cc06eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:41.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.345+0000 7fa1d1000640 1 -- 172.21.15.119:0/3900648391 shutdown_connections 2024-04-25T10:08:41.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.345+0000 7fa1d1000640 1 -- 172.21.15.119:0/3900648391 wait complete. 2024-04-25T10:08:41.368 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:08:41.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.559+0000 7f22d9800640 1 Processor -- start 2024-04-25T10:08:41.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.559+0000 7f22d9800640 1 -- start start 2024-04-25T10:08:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.561+0000 7f22d9800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22d4144e70 0x7f22d406d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.561+0000 7f22d9800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d406dcb0 con 0x7f22d40cf0e0 2024-04-25T10:08:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.561+0000 7f22d9800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d406df90 con 0x7f22d406d7a0 2024-04-25T10:08:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.561+0000 7f22d9800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d415bfe0 con 0x7f22d4144e70 2024-04-25T10:08:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.561+0000 7f22d2a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22d4144e70 0x7f22d406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.561+0000 7f22d3400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f22d40cf0e0 0x7f22d40cc300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53784/0 (socket says 172.21.15.119:53784) 2024-04-25T10:08:41.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.561+0000 7f22d3400640 1 -- 172.21.15.119:0/724995314 learned_addr learned my addr 172.21.15.119:0/724995314 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:41.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.562+0000 7f22d2a00640 1 -- 172.21.15.119:0/724995314 >> v1:172.21.15.132:6789/0 conn(0x7f22d406d7a0 legacy=0x7f22d40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:41.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.562+0000 7f22d2a00640 1 -- 172.21.15.119:0/724995314 >> v1:172.21.15.119:6789/0 conn(0x7f22d40cf0e0 legacy=0x7f22d40cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.562+0000 7f22d2a00640 1 -- 172.21.15.119:0/724995314 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d406e270 con 0x7f22d4144e70 2024-04-25T10:08:41.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.562+0000 7f22d1600640 1 -- 172.21.15.119:0/724995314 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100592000 0 0) 0x7f22d406dcb0 con 0x7f22d40cf0e0 2024-04-25T10:08:41.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.563+0000 7f22d2a00640 1 --2- 172.21.15.119:0/724995314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d4144e70 0x7f22d406d260 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f22d415c160 tx=0x7f22c402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63bff79c115bd53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.563+0000 7f22d1600640 1 -- 172.21.15.119:0/724995314 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4037070 con 0x7f22d4144e70 2024-04-25T10:08:41.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.563+0000 7f22d1600640 1 -- 172.21.15.119:0/724995314 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c402ae40 con 0x7f22d4144e70 2024-04-25T10:08:41.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.563+0000 7f22d1600640 1 -- 172.21.15.119:0/724995314 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4032660 con 0x7f22d4144e70 2024-04-25T10:08:41.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.563+0000 7f22d9800640 1 -- 172.21.15.119:0/724995314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d4144e70 msgr2=0x7f22d406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.563+0000 7f22d9800640 1 --2- 172.21.15.119:0/724995314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d4144e70 0x7f22d406d260 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f22d415c160 tx=0x7f22c402a130 comp rx=0 tx=0).stop 2024-04-25T10:08:41.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.564+0000 7f22d9800640 1 -- 172.21.15.119:0/724995314 shutdown_connections 2024-04-25T10:08:41.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.564+0000 7f22d9800640 1 --2- 172.21.15.119:0/724995314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d4144e70 0x7f22d406d260 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.564+0000 7f22d9800640 1 -- 172.21.15.119:0/724995314 >> 172.21.15.119:0/724995314 conn(0x7f22d406aba0 msgr2=0x7f22d406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:41.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.564+0000 7f22d9800640 1 -- 172.21.15.119:0/724995314 shutdown_connections 2024-04-25T10:08:41.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.564+0000 7f22d9800640 1 -- 172.21.15.119:0/724995314 wait complete. 2024-04-25T10:08:41.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.565+0000 7f22d9800640 1 Processor -- start 2024-04-25T10:08:41.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.565+0000 7f22d9800640 1 -- start start 2024-04-25T10:08:41.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.566+0000 7f22d9800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22d406d7a0 0x7f22d4155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.566+0000 7f22d9800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22d40cf0e0 0x7f22d41f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.566+0000 7f22d9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d4144e70 0x7f22d413c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.566+0000 7f22d9800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4145630 con 0x7f22d4144e70 2024-04-25T10:08:41.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.566+0000 7f22d9800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d41454b0 con 0x7f22d40cf0e0 2024-04-25T10:08:41.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.566+0000 7f22d9800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d40cca10 con 0x7f22d406d7a0 2024-04-25T10:08:41.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.567+0000 7f22d3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d4144e70 0x7f22d413c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.567+0000 7f22d3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d4144e70 0x7f22d413c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51456/0 (socket says 172.21.15.119:51456) 2024-04-25T10:08:41.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.567+0000 7f22d3400640 1 -- 172.21.15.119:0/2151757354 learned_addr learned my addr 172.21.15.119:0/2151757354 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:41.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.567+0000 7f22d2a00640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22d406d7a0 0x7f22d4155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.567+0000 7f22d2000640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22d40cf0e0 0x7f22d41f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.567+0000 7f22d3400640 1 -- 172.21.15.119:0/2151757354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22d406d7a0 msgr2=0x7f22d4155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.567+0000 7f22d3400640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22d406d7a0 0x7f22d4155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.567+0000 7f22d3400640 1 -- 172.21.15.119:0/2151757354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22d40cf0e0 msgr2=0x7f22d41f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.567+0000 7f22d3400640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22d40cf0e0 0x7f22d41f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.567+0000 7f22d3400640 1 -- 172.21.15.119:0/2151757354 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d41f3fb0 con 0x7f22d4144e70 2024-04-25T10:08:41.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.567+0000 7f22d2000640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22d40cf0e0 0x7f22d41f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:41.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.567+0000 7f22d3400640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d4144e70 0x7f22d413c110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00c7d0 tx=0x7f22cc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.568+0000 7f22caa00640 1 -- 172.21.15.119:0/2151757354 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc018070 con 0x7f22d4144e70 2024-04-25T10:08:41.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.568+0000 7f22caa00640 1 -- 172.21.15.119:0/2151757354 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc00a640 con 0x7f22d4144e70 2024-04-25T10:08:41.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.568+0000 7f22caa00640 1 -- 172.21.15.119:0/2151757354 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00aa70 con 0x7f22d4144e70 2024-04-25T10:08:41.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.568+0000 7f22d9800640 1 -- 172.21.15.119:0/2151757354 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d41f4290 con 0x7f22d4144e70 2024-04-25T10:08:41.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.568+0000 7f22d9800640 1 -- 172.21.15.119:0/2151757354 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d41f4850 con 0x7f22d4144e70 2024-04-25T10:08:41.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.569+0000 7f22d9800640 1 -- 172.21.15.119:0/2151757354 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d40750b0 con 0x7f22d4144e70 2024-04-25T10:08:41.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.570+0000 7f22caa00640 1 -- 172.21.15.119:0/2151757354 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22cc00ac10 con 0x7f22d4144e70 2024-04-25T10:08:41.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.571+0000 7f22caa00640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f22b406b4d0 0x7f22b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.572+0000 7f22caa00640 1 -- 172.21.15.119:0/2151757354 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22cc089860 con 0x7f22d4144e70 2024-04-25T10:08:41.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.572+0000 7f22d2a00640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f22b406b4d0 0x7f22b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.572+0000 7f22d2a00640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f22b406b4d0 0x7f22b406d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f22c4034000 tx=0x7f22c402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.576+0000 7f22caa00640 1 -- 172.21.15.119:0/2151757354 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22cc0586e0 con 0x7f22d4144e70 2024-04-25T10:08:41.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.769+0000 7f22d9800640 1 -- 172.21.15.119:0/2151757354 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f22d40aebe0 con 0x7f22d4144e70 2024-04-25T10:08:41.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.770+0000 7f22caa00640 1 -- 172.21.15.119:0/2151757354 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f22cc089030 con 0x7f22d4144e70 2024-04-25T10:08:41.777 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:08:41.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.772+0000 7f22d9800640 1 -- 172.21.15.119:0/2151757354 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f22b406b4d0 msgr2=0x7f22b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.772+0000 7f22d9800640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f22b406b4d0 0x7f22b406d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f22c4034000 tx=0x7f22c402e000 comp rx=0 tx=0).stop 2024-04-25T10:08:41.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.772+0000 7f22d9800640 1 -- 172.21.15.119:0/2151757354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d4144e70 msgr2=0x7f22d413c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.772+0000 7f22d9800640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d4144e70 0x7f22d413c110 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00c7d0 tx=0x7f22cc00cc90 comp rx=0 tx=0).stop 2024-04-25T10:08:41.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.773+0000 7f22d9800640 1 -- 172.21.15.119:0/2151757354 shutdown_connections 2024-04-25T10:08:41.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.773+0000 7f22d9800640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f22b406b4d0 0x7f22b406d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.773+0000 7f22d9800640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d4144e70 0x7f22d413c110 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.773+0000 7f22d9800640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22d40cf0e0 0x7f22d41f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.773+0000 7f22d9800640 1 --2- 172.21.15.119:0/2151757354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22d406d7a0 0x7f22d4155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.773+0000 7f22d9800640 1 -- 172.21.15.119:0/2151757354 >> 172.21.15.119:0/2151757354 conn(0x7f22d406aba0 msgr2=0x7f22d4144090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:41.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.773+0000 7f22d9800640 1 -- 172.21.15.119:0/2151757354 shutdown_connections 2024-04-25T10:08:41.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.773+0000 7f22d9800640 1 -- 172.21.15.119:0/2151757354 wait complete. 2024-04-25T10:08:41.796 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:41.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.989+0000 7f8fc5200640 1 Processor -- start 2024-04-25T10:08:41.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.989+0000 7f8fc5200640 1 -- start start 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.990+0000 7f8fc5200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8fc006e8f0 0x7f8fc015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.990+0000 7f8fc5200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc015c820 0x7f8fc01654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.990+0000 7f8fc5200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc00ce6c0 con 0x7f8fc015bf00 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.990+0000 7f8fc5200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc01455e0 con 0x7f8fc006e8f0 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.990+0000 7f8fc5200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc006d450 con 0x7f8fc015c820 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.991+0000 7f8fbd400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8fc006e8f0 0x7f8fc015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.991+0000 7f8fbd400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8fc006e8f0 0x7f8fc015c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:50764/0 (socket says 172.21.15.119:50764) 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.991+0000 7f8fbd400640 1 -- 172.21.15.119:0/2803860118 learned_addr learned my addr 172.21.15.119:0/2803860118 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.991+0000 7f8fbe800640 1 --2- 172.21.15.119:0/2803860118 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc015c820 0x7f8fc01654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.992+0000 7f8fbe800640 1 -- 172.21.15.119:0/2803860118 >> v2:172.21.15.132:3300/0 conn(0x7f8fc006e8f0 msgr2=0x7f8fc015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:41.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.992+0000 7f8fbe800640 1 --2- 172.21.15.119:0/2803860118 >> v2:172.21.15.132:3300/0 conn(0x7f8fc006e8f0 0x7f8fc015c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.992+0000 7f8fbe800640 1 -- 172.21.15.119:0/2803860118 >> v1:172.21.15.132:6789/0 conn(0x7f8fc015bf00 legacy=0x7f8fc00cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.992+0000 7f8fbe800640 1 -- 172.21.15.119:0/2803860118 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00cc4e0 con 0x7f8fc015c820 2024-04-25T10:08:41.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.993+0000 7f8fbe800640 1 --2- 172.21.15.119:0/2803860118 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc015c820 0x7f8fc01654b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009d90 tx=0x7f8fb802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79e2e826d864d58b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.993+0000 7f8fbca00640 1 -- 172.21.15.119:0/2803860118 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8007740 con 0x7f8fc015c820 2024-04-25T10:08:41.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.993+0000 7f8fbca00640 1 -- 172.21.15.119:0/2803860118 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb80078e0 con 0x7f8fc015c820 2024-04-25T10:08:41.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.993+0000 7f8fbca00640 1 -- 172.21.15.119:0/2803860118 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8007cb0 con 0x7f8fc015c820 2024-04-25T10:08:42.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.994+0000 7f8fc5200640 1 -- 172.21.15.119:0/2803860118 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc015c820 msgr2=0x7f8fc01654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.994+0000 7f8fc5200640 1 --2- 172.21.15.119:0/2803860118 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc015c820 0x7f8fc01654b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009d90 tx=0x7f8fb802a660 comp rx=0 tx=0).stop 2024-04-25T10:08:42.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.994+0000 7f8fc5200640 1 -- 172.21.15.119:0/2803860118 shutdown_connections 2024-04-25T10:08:42.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.994+0000 7f8fc5200640 1 --2- 172.21.15.119:0/2803860118 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc015c820 0x7f8fc01654b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.994+0000 7f8fc5200640 1 --2- 172.21.15.119:0/2803860118 >> v2:172.21.15.132:3300/0 conn(0x7f8fc006e8f0 0x7f8fc015c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.994+0000 7f8fc5200640 1 -- 172.21.15.119:0/2803860118 >> 172.21.15.119:0/2803860118 conn(0x7f8fc006aba0 msgr2=0x7f8fc006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:42.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.994+0000 7f8fc5200640 1 -- 172.21.15.119:0/2803860118 shutdown_connections 2024-04-25T10:08:42.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.994+0000 7f8fc5200640 1 -- 172.21.15.119:0/2803860118 wait complete. 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.995+0000 7f8fc5200640 1 Processor -- start 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.996+0000 7f8fc5200640 1 -- start start 2024-04-25T10:08:42.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.996+0000 7f8fc5200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc006e8f0 0x7f8fc0153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.996+0000 7f8fc5200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc015bf00 0x7f8fc00c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.996+0000 7f8fc5200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc015c820 0x7f8fc00c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.996+0000 7f8fc5200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00ccd70 con 0x7f8fc015c820 2024-04-25T10:08:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.997+0000 7f8fc5200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00cca90 con 0x7f8fc006e8f0 2024-04-25T10:08:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.997+0000 7f8fc5200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00cd050 con 0x7f8fc015bf00 2024-04-25T10:08:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.997+0000 7f8fbe800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc015c820 0x7f8fc00c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.997+0000 7f8fbe800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc015c820 0x7f8fc00c9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51470/0 (socket says 172.21.15.119:51470) 2024-04-25T10:08:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.997+0000 7f8fbe800640 1 -- 172.21.15.119:0/1499539478 learned_addr learned my addr 172.21.15.119:0/1499539478 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.997+0000 7f8fbd400640 1 --2- 172.21.15.119:0/1499539478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc015bf00 0x7f8fc00c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.997+0000 7f8fbde00640 1 --2- 172.21.15.119:0/1499539478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc006e8f0 0x7f8fc0153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.997+0000 7f8fbe800640 1 -- 172.21.15.119:0/1499539478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc015bf00 msgr2=0x7f8fc00c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.997+0000 7f8fbe800640 1 --2- 172.21.15.119:0/1499539478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc015bf00 0x7f8fc00c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.997+0000 7f8fbe800640 1 -- 172.21.15.119:0/1499539478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc006e8f0 msgr2=0x7f8fc0153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.997+0000 7f8fbe800640 1 --2- 172.21.15.119:0/1499539478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc006e8f0 0x7f8fc0153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.997+0000 7f8fbe800640 1 -- 172.21.15.119:0/1499539478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00b4290 con 0x7f8fc015c820 2024-04-25T10:08:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.998+0000 7f8fbe800640 1 --2- 172.21.15.119:0/1499539478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc015c820 0x7f8fc00c9690 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009eb0 tx=0x7f8fb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.998+0000 7f8fb6000640 1 -- 172.21.15.119:0/1499539478 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb803f070 con 0x7f8fc015c820 2024-04-25T10:08:42.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.998+0000 7f8fb6000640 1 -- 172.21.15.119:0/1499539478 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb8031c60 con 0x7f8fc015c820 2024-04-25T10:08:42.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.998+0000 7f8fb6000640 1 -- 172.21.15.119:0/1499539478 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8009450 con 0x7f8fc015c820 2024-04-25T10:08:42.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.998+0000 7f8fc5200640 1 -- 172.21.15.119:0/1499539478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc013d270 con 0x7f8fc015c820 2024-04-25T10:08:42.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.998+0000 7f8fc5200640 1 -- 172.21.15.119:0/1499539478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc01544b0 con 0x7f8fc015c820 2024-04-25T10:08:42.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:41.999+0000 7f8fc5200640 1 -- 172.21.15.119:0/1499539478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc015ccb0 con 0x7f8fc015c820 2024-04-25T10:08:42.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.000+0000 7f8fb6000640 1 -- 172.21.15.119:0/1499539478 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fb8043650 con 0x7f8fc015c820 2024-04-25T10:08:42.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.001+0000 7f8fb6000640 1 --2- 172.21.15.119:0/1499539478 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fa006b4d0 0x7f8fa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.001+0000 7f8fb6000640 1 -- 172.21.15.119:0/1499539478 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fb80ab960 con 0x7f8fc015c820 2024-04-25T10:08:42.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.002+0000 7f8fbde00640 1 --2- 172.21.15.119:0/1499539478 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fa006b4d0 0x7f8fa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.002+0000 7f8fbde00640 1 --2- 172.21.15.119:0/1499539478 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fa006b4d0 0x7f8fa006d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0001640 tx=0x7f8fb0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.005+0000 7f8fb6000640 1 -- 172.21.15.119:0/1499539478 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8fc015ccb0 con 0x7f8fc015c820 2024-04-25T10:08:42.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.234+0000 7f8fc5200640 1 -- 172.21.15.119:0/1499539478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fc00749b0 con 0x7f8fc015c820 2024-04-25T10:08:42.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.235+0000 7f8fb6000640 1 -- 172.21.15.119:0/1499539478 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f8fb807a970 con 0x7f8fc015c820 2024-04-25T10:08:42.241 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:42.242 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1554372269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1554372269},{"type":"v1","addr":"172.21.15.119:6837","nonce":1554372269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4699,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4250304148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4250304148},{"type":"v1","addr":"172.21.15.132:6834","nonce":4250304148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/637229574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":637229574},{"type":"v1","addr":"172.21.15.132:6835","nonce":637229574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:14.553437+0000","modified":"2024-04-25T10:08:17.586270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6836/3585315292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3585315292},{"type":"v1","addr":"172.21.15.119:6836","nonce":3585315292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:08:42.242 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 16 2024-04-25T10:08:42.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.238+0000 7f8fc5200640 1 -- 172.21.15.119:0/1499539478 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fa006b4d0 msgr2=0x7f8fa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.238+0000 7f8fc5200640 1 --2- 172.21.15.119:0/1499539478 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fa006b4d0 0x7f8fa006d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0001640 tx=0x7f8fb0009040 comp rx=0 tx=0).stop 2024-04-25T10:08:42.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.238+0000 7f8fc5200640 1 -- 172.21.15.119:0/1499539478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc015c820 msgr2=0x7f8fc00c9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.238+0000 7f8fc5200640 1 --2- 172.21.15.119:0/1499539478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc015c820 0x7f8fc00c9690 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009eb0 tx=0x7f8fb8004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:42.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.238+0000 7f8fc5200640 1 -- 172.21.15.119:0/1499539478 shutdown_connections 2024-04-25T10:08:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.238+0000 7f8fc5200640 1 --2- 172.21.15.119:0/1499539478 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fa006b4d0 0x7f8fa006d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.238+0000 7f8fc5200640 1 --2- 172.21.15.119:0/1499539478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc015c820 0x7f8fc00c9690 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.238+0000 7f8fc5200640 1 --2- 172.21.15.119:0/1499539478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fc015bf00 0x7f8fc00c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.238+0000 7f8fc5200640 1 --2- 172.21.15.119:0/1499539478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fc006e8f0 0x7f8fc0153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.238+0000 7f8fc5200640 1 -- 172.21.15.119:0/1499539478 >> 172.21.15.119:0/1499539478 conn(0x7f8fc006aba0 msgr2=0x7f8fc0144190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.238+0000 7f8fc5200640 1 -- 172.21.15.119:0/1499539478 shutdown_connections 2024-04-25T10:08:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.238+0000 7f8fc5200640 1 -- 172.21.15.119:0/1499539478 wait complete. 2024-04-25T10:08:42.263 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:42.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.452+0000 7fb534c00640 1 Processor -- start 2024-04-25T10:08:42.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.452+0000 7fb534c00640 1 -- start start 2024-04-25T10:08:42.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.454+0000 7fb534c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5300b3040 0x7fb5300b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.454+0000 7fb534c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb53006aba0 0x7fb53006b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.454+0000 7fb534c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb53006e6c0 con 0x7fb53015ec10 2024-04-25T10:08:42.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.454+0000 7fb534c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb53006f2d0 con 0x7fb5300b3040 2024-04-25T10:08:42.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.454+0000 7fb534c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53006e9a0 con 0x7fb53006aba0 2024-04-25T10:08:42.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.454+0000 7fb525400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5300b3040 0x7fb5300b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.454+0000 7fb525400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5300b3040 0x7fb5300b34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51482/0 (socket says 172.21.15.119:51482) 2024-04-25T10:08:42.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.454+0000 7fb525400640 1 -- 172.21.15.119:0/1189720662 learned_addr learned my addr 172.21.15.119:0/1189720662 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:42.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.455+0000 7fb525400640 1 -- 172.21.15.119:0/1189720662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb53006aba0 msgr2=0x7fb53006b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:42.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.455+0000 7fb525400640 1 --2- 172.21.15.119:0/1189720662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb53006aba0 0x7fb53006b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.455+0000 7fb525400640 1 -- 172.21.15.119:0/1189720662 >> v1:172.21.15.132:6789/0 conn(0x7fb53015ec10 legacy=0x7fb53006dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:42.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.455+0000 7fb525400640 1 -- 172.21.15.119:0/1189720662 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53006ec80 con 0x7fb5300b3040 2024-04-25T10:08:42.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.456+0000 7fb525400640 1 --2- 172.21.15.119:0/1189720662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5300b3040 0x7fb5300b34f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb5180098a0 tx=0x7fb51802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132b65443f5da89e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.456+0000 7fb52d400640 1 -- 172.21.15.119:0/1189720662 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518037070 con 0x7fb5300b3040 2024-04-25T10:08:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.456+0000 7fb52d400640 1 -- 172.21.15.119:0/1189720662 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51802ae20 con 0x7fb5300b3040 2024-04-25T10:08:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.456+0000 7fb52d400640 1 -- 172.21.15.119:0/1189720662 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518032660 con 0x7fb5300b3040 2024-04-25T10:08:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.457+0000 7fb534c00640 1 -- 172.21.15.119:0/1189720662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5300b3040 msgr2=0x7fb5300b34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.457+0000 7fb534c00640 1 --2- 172.21.15.119:0/1189720662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5300b3040 0x7fb5300b34f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb5180098a0 tx=0x7fb51802a110 comp rx=0 tx=0).stop 2024-04-25T10:08:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.457+0000 7fb534c00640 1 -- 172.21.15.119:0/1189720662 shutdown_connections 2024-04-25T10:08:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.457+0000 7fb534c00640 1 --2- 172.21.15.119:0/1189720662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb53006aba0 0x7fb53006b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.457+0000 7fb534c00640 1 --2- 172.21.15.119:0/1189720662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5300b3040 0x7fb5300b34f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.457+0000 7fb534c00640 1 -- 172.21.15.119:0/1189720662 >> 172.21.15.119:0/1189720662 conn(0x7fb5300b2220 msgr2=0x7fb5300cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.457+0000 7fb534c00640 1 -- 172.21.15.119:0/1189720662 shutdown_connections 2024-04-25T10:08:42.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.457+0000 7fb534c00640 1 -- 172.21.15.119:0/1189720662 wait complete. 2024-04-25T10:08:42.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.459+0000 7fb534c00640 1 Processor -- start 2024-04-25T10:08:42.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.459+0000 7fb534c00640 1 -- start start 2024-04-25T10:08:42.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.460+0000 7fb534c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb53006aba0 0x7fb5301435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb534c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5300b3040 0x7fb53014ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb534c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb53015ec10 0x7fb5300ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb534c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53006e380 con 0x7fb53006aba0 2024-04-25T10:08:42.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb534c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53015eff0 con 0x7fb53015ec10 2024-04-25T10:08:42.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb534c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53015e2e0 con 0x7fb5300b3040 2024-04-25T10:08:42.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb52de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb53006aba0 0x7fb5301435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb52de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb53006aba0 0x7fb5301435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51498/0 (socket says 172.21.15.119:51498) 2024-04-25T10:08:42.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb52de00640 1 -- 172.21.15.119:0/3910438200 learned_addr learned my addr 172.21.15.119:0/3910438200 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:42.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb52de00640 1 -- 172.21.15.119:0/3910438200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5300b3040 msgr2=0x7fb53014ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb525400640 1 --2- 172.21.15.119:0/3910438200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5300b3040 0x7fb53014ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb52de00640 1 --2- 172.21.15.119:0/3910438200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5300b3040 0x7fb53014ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb52de00640 1 -- 172.21.15.119:0/3910438200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb53015ec10 msgr2=0x7fb5300ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb52e800640 1 --2- 172.21.15.119:0/3910438200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb53015ec10 0x7fb5300ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb52de00640 1 --2- 172.21.15.119:0/3910438200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb53015ec10 0x7fb5300ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.461+0000 7fb52de00640 1 -- 172.21.15.119:0/3910438200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53014e0b0 con 0x7fb53006aba0 2024-04-25T10:08:42.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.462+0000 7fb52de00640 1 --2- 172.21.15.119:0/3910438200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb53006aba0 0x7fb5301435c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb52000c410 tx=0x7fb52000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.462+0000 7fb526a00640 1 -- 172.21.15.119:0/3910438200 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520017070 con 0x7fb53006aba0 2024-04-25T10:08:42.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.462+0000 7fb526a00640 1 -- 172.21.15.119:0/3910438200 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5200035f0 con 0x7fb53006aba0 2024-04-25T10:08:42.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.462+0000 7fb526a00640 1 -- 172.21.15.119:0/3910438200 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520003a20 con 0x7fb53006aba0 2024-04-25T10:08:42.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.462+0000 7fb534c00640 1 -- 172.21.15.119:0/3910438200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53014e390 con 0x7fb53006aba0 2024-04-25T10:08:42.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.462+0000 7fb534c00640 1 -- 172.21.15.119:0/3910438200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53014e950 con 0x7fb53006aba0 2024-04-25T10:08:42.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.463+0000 7fb534c00640 1 -- 172.21.15.119:0/3910438200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5300cc210 con 0x7fb53006aba0 2024-04-25T10:08:42.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.464+0000 7fb526a00640 1 -- 172.21.15.119:0/3910438200 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb520003bc0 con 0x7fb53006aba0 2024-04-25T10:08:42.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.465+0000 7fb526a00640 1 --2- 172.21.15.119:0/3910438200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb50c06b4d0 0x7fb50c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.465+0000 7fb526a00640 1 -- 172.21.15.119:0/3910438200 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5200895b0 con 0x7fb53006aba0 2024-04-25T10:08:42.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.465+0000 7fb525400640 1 --2- 172.21.15.119:0/3910438200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb50c06b4d0 0x7fb50c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.466+0000 7fb525400640 1 --2- 172.21.15.119:0/3910438200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb50c06b4d0 0x7fb50c06d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb518034000 tx=0x7fb51802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.470+0000 7fb526a00640 1 -- 172.21.15.119:0/3910438200 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb520058390 con 0x7fb53006aba0 2024-04-25T10:08:42.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.669+0000 7fb534c00640 1 -- 172.21.15.119:0/3910438200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5300749b0 con 0x7fb53006aba0 2024-04-25T10:08:42.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.671+0000 7fb526a00640 1 -- 172.21.15.119:0/3910438200 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16705 (secure 0 0 0) 0x7fb520057c90 con 0x7fb53006aba0 2024-04-25T10:08:42.678 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:42.678 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":33,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:16.095956+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:12.268457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:13.546259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:42.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.674+0000 7fb534c00640 1 -- 172.21.15.119:0/3910438200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb50c06b4d0 msgr2=0x7fb50c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.674+0000 7fb534c00640 1 --2- 172.21.15.119:0/3910438200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb50c06b4d0 0x7fb50c06d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb518034000 tx=0x7fb51802e000 comp rx=0 tx=0).stop 2024-04-25T10:08:42.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.674+0000 7fb534c00640 1 -- 172.21.15.119:0/3910438200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb53006aba0 msgr2=0x7fb5301435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.674+0000 7fb534c00640 1 --2- 172.21.15.119:0/3910438200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb53006aba0 0x7fb5301435c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb52000c410 tx=0x7fb52000c8d0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.674+0000 7fb534c00640 1 -- 172.21.15.119:0/3910438200 shutdown_connections 2024-04-25T10:08:42.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.674+0000 7fb534c00640 1 --2- 172.21.15.119:0/3910438200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb50c06b4d0 0x7fb50c06d990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.674+0000 7fb534c00640 1 --2- 172.21.15.119:0/3910438200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb53015ec10 0x7fb5300ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.674+0000 7fb534c00640 1 --2- 172.21.15.119:0/3910438200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5300b3040 0x7fb53014ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.674+0000 7fb534c00640 1 --2- 172.21.15.119:0/3910438200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb53006aba0 0x7fb5301435c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.674+0000 7fb534c00640 1 -- 172.21.15.119:0/3910438200 >> 172.21.15.119:0/3910438200 conn(0x7fb5300b2220 msgr2=0x7fb5301570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:42.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.674+0000 7fb534c00640 1 -- 172.21.15.119:0/3910438200 shutdown_connections 2024-04-25T10:08:42.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.674+0000 7fb534c00640 1 -- 172.21.15.119:0/3910438200 wait complete. 2024-04-25T10:08:42.699 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:08:42.700 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:42.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.896+0000 7f2f52000640 1 Processor -- start 2024-04-25T10:08:42.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.896+0000 7f2f52000640 1 -- start start 2024-04-25T10:08:42.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f52000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2f4c144040 0x7f2f4c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f52000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2f4c15acd0 0x7f2f4c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f52000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c144420 con 0x7f2f4c0cce40 2024-04-25T10:08:42.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f52000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4c0b36d0 con 0x7f2f4c144040 2024-04-25T10:08:42.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f52000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4c0b3d80 con 0x7f2f4c15acd0 2024-04-25T10:08:42.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f50c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2f4c144040 0x7f2f4c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f51600640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2f4c0cce40 0x7f2f4c15e2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53796/0 (socket says 172.21.15.119:53796) 2024-04-25T10:08:42.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f51600640 1 -- 172.21.15.119:0/3881151201 learned_addr learned my addr 172.21.15.119:0/3881151201 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:42.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f50c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2f4c144040 0x7f2f4c0cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51506/0 (socket says 172.21.15.119:51506) 2024-04-25T10:08:42.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f50c00640 1 -- 172.21.15.119:0/3881151201 >> v2:172.21.15.132:3300/0 conn(0x7f2f4c15acd0 msgr2=0x7f2f4c06e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:42.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f43e00640 1 --2- 172.21.15.119:0/3881151201 >> v2:172.21.15.132:3300/0 conn(0x7f2f4c15acd0 0x7f2f4c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f50c00640 1 --2- 172.21.15.119:0/3881151201 >> v2:172.21.15.132:3300/0 conn(0x7f2f4c15acd0 0x7f2f4c06e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f4be00640 1 -- 172.21.15.119:0/3881151201 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423870357 0 0) 0x7f2f4c144420 con 0x7f2f4c0cce40 2024-04-25T10:08:42.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f50c00640 1 -- 172.21.15.119:0/3881151201 >> v1:172.21.15.119:6789/0 conn(0x7f2f4c0cce40 legacy=0x7f2f4c15e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.898+0000 7f2f50c00640 1 -- 172.21.15.119:0/3881151201 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c0b4060 con 0x7f2f4c144040 2024-04-25T10:08:42.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.900+0000 7f2f50c00640 1 --2- 172.21.15.119:0/3881151201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f4c144040 0x7f2f4c0cc900 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0b3850 tx=0x7f2f3c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b43aa6381c088e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.900+0000 7f2f4be00640 1 -- 172.21.15.119:0/3881151201 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c037070 con 0x7f2f4c144040 2024-04-25T10:08:42.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.900+0000 7f2f4be00640 1 -- 172.21.15.119:0/3881151201 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c032440 con 0x7f2f4c144040 2024-04-25T10:08:42.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.900+0000 7f2f4be00640 1 -- 172.21.15.119:0/3881151201 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c02ae60 con 0x7f2f4c144040 2024-04-25T10:08:42.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.900+0000 7f2f52000640 1 -- 172.21.15.119:0/3881151201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f4c144040 msgr2=0x7f2f4c0cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.900+0000 7f2f52000640 1 --2- 172.21.15.119:0/3881151201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f4c144040 0x7f2f4c0cc900 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0b3850 tx=0x7f2f3c02a110 comp rx=0 tx=0).stop 2024-04-25T10:08:42.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.901+0000 7f2f52000640 1 -- 172.21.15.119:0/3881151201 shutdown_connections 2024-04-25T10:08:42.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.901+0000 7f2f52000640 1 --2- 172.21.15.119:0/3881151201 >> v2:172.21.15.132:3300/0 conn(0x7f2f4c15acd0 0x7f2f4c06e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.901+0000 7f2f52000640 1 --2- 172.21.15.119:0/3881151201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f4c144040 0x7f2f4c0cc900 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.901+0000 7f2f52000640 1 -- 172.21.15.119:0/3881151201 >> 172.21.15.119:0/3881151201 conn(0x7f2f4c06b1d0 msgr2=0x7f2f4c0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:42.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.901+0000 7f2f52000640 1 -- 172.21.15.119:0/3881151201 shutdown_connections 2024-04-25T10:08:42.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.901+0000 7f2f52000640 1 -- 172.21.15.119:0/3881151201 wait complete. 2024-04-25T10:08:42.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.902+0000 7f2f52000640 1 Processor -- start 2024-04-25T10:08:42.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.903+0000 7f2f52000640 1 -- start start 2024-04-25T10:08:42.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.903+0000 7f2f52000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f4c0cce40 0x7f2f4c1f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.903+0000 7f2f52000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f4c144040 0x7f2f4c15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.903+0000 7f2f52000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c15acd0 0x7f2f4c1f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.904+0000 7f2f52000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0b4900 con 0x7f2f4c144040 2024-04-25T10:08:42.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.904+0000 7f2f52000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0b4620 con 0x7f2f4c15acd0 2024-04-25T10:08:42.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.904+0000 7f2f52000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0b5180 con 0x7f2f4c0cce40 2024-04-25T10:08:42.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.904+0000 7f2f43e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f4c144040 0x7f2f4c15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.904+0000 7f2f43e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f4c144040 0x7f2f4c15a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51520/0 (socket says 172.21.15.119:51520) 2024-04-25T10:08:42.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.904+0000 7f2f43e00640 1 -- 172.21.15.119:0/827136256 learned_addr learned my addr 172.21.15.119:0/827136256 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:42.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.904+0000 7f2f43e00640 1 -- 172.21.15.119:0/827136256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f4c0cce40 msgr2=0x7f2f4c1f6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.904+0000 7f2f43e00640 1 --2- 172.21.15.119:0/827136256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f4c0cce40 0x7f2f4c1f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.904+0000 7f2f43e00640 1 -- 172.21.15.119:0/827136256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c15acd0 msgr2=0x7f2f4c1f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.904+0000 7f2f51600640 1 --2- 172.21.15.119:0/827136256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c15acd0 0x7f2f4c1f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.904+0000 7f2f43e00640 1 --2- 172.21.15.119:0/827136256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c15acd0 0x7f2f4c1f3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.904+0000 7f2f43e00640 1 -- 172.21.15.119:0/827136256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c1f7190 con 0x7f2f4c144040 2024-04-25T10:08:42.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.905+0000 7f2f43e00640 1 --2- 172.21.15.119:0/827136256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f4c144040 0x7f2f4c15a1a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400c820 tx=0x7f2f3400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.905+0000 7f2f49600640 1 -- 172.21.15.119:0/827136256 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34015070 con 0x7f2f4c144040 2024-04-25T10:08:42.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.905+0000 7f2f49600640 1 -- 172.21.15.119:0/827136256 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f34004020 con 0x7f2f4c144040 2024-04-25T10:08:42.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.905+0000 7f2f52000640 1 -- 172.21.15.119:0/827136256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4c1f7470 con 0x7f2f4c144040 2024-04-25T10:08:42.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.905+0000 7f2f49600640 1 -- 172.21.15.119:0/827136256 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f340043f0 con 0x7f2f4c144040 2024-04-25T10:08:42.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.905+0000 7f2f52000640 1 -- 172.21.15.119:0/827136256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4c142090 con 0x7f2f4c144040 2024-04-25T10:08:42.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.906+0000 7f2f52000640 1 -- 172.21.15.119:0/827136256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4c06bb30 con 0x7f2f4c144040 2024-04-25T10:08:42.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.907+0000 7f2f49600640 1 -- 172.21.15.119:0/827136256 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f34007830 con 0x7f2f4c144040 2024-04-25T10:08:42.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.908+0000 7f2f49600640 1 --2- 172.21.15.119:0/827136256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2f2806b4d0 0x7f2f2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.908+0000 7f2f49600640 1 -- 172.21.15.119:0/827136256 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f34089ff0 con 0x7f2f4c144040 2024-04-25T10:08:42.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.908+0000 7f2f50c00640 1 --2- 172.21.15.119:0/827136256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2f2806b4d0 0x7f2f2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.909+0000 7f2f50c00640 1 --2- 172.21.15.119:0/827136256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2f2806b4d0 0x7f2f2806d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c034000 tx=0x7f2f3c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:42.913+0000 7f2f49600640 1 -- 172.21.15.119:0/827136256 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f34058ea0 con 0x7f2f4c144040 2024-04-25T10:08:43.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.145+0000 7f2f52000640 1 -- 172.21.15.119:0/827136256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f4c0749b0 con 0x7f2f4c144040 2024-04-25T10:08:43.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.147+0000 7f2f49600640 1 -- 172.21.15.119:0/827136256 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f2f340587a0 con 0x7f2f4c144040 2024-04-25T10:08:43.153 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:43.153 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1554372269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1554372269},{"type":"v1","addr":"172.21.15.119:6837","nonce":1554372269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4699,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4250304148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4250304148},{"type":"v1","addr":"172.21.15.132:6834","nonce":4250304148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/637229574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":637229574},{"type":"v1","addr":"172.21.15.132:6835","nonce":637229574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:14.553437+0000","modified":"2024-04-25T10:08:17.586270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6836/3585315292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3585315292},{"type":"v1","addr":"172.21.15.119:6836","nonce":3585315292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:08:43.153 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 16 2024-04-25T10:08:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.149+0000 7f2f52000640 1 -- 172.21.15.119:0/827136256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2f2806b4d0 msgr2=0x7f2f2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.149+0000 7f2f52000640 1 --2- 172.21.15.119:0/827136256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2f2806b4d0 0x7f2f2806d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c034000 tx=0x7f2f3c02c000 comp rx=0 tx=0).stop 2024-04-25T10:08:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.149+0000 7f2f52000640 1 -- 172.21.15.119:0/827136256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f4c144040 msgr2=0x7f2f4c15a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.149+0000 7f2f52000640 1 --2- 172.21.15.119:0/827136256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f4c144040 0x7f2f4c15a1a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400c820 tx=0x7f2f3400cce0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.149+0000 7f2f52000640 1 -- 172.21.15.119:0/827136256 shutdown_connections 2024-04-25T10:08:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.149+0000 7f2f52000640 1 --2- 172.21.15.119:0/827136256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2f2806b4d0 0x7f2f2806d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.149+0000 7f2f52000640 1 --2- 172.21.15.119:0/827136256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c15acd0 0x7f2f4c1f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.149+0000 7f2f52000640 1 --2- 172.21.15.119:0/827136256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f4c144040 0x7f2f4c15a1a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.149+0000 7f2f52000640 1 --2- 172.21.15.119:0/827136256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f4c0cce40 0x7f2f4c1f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.149+0000 7f2f52000640 1 -- 172.21.15.119:0/827136256 >> 172.21.15.119:0/827136256 conn(0x7f2f4c06b1d0 msgr2=0x7f2f4c06e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.150+0000 7f2f52000640 1 -- 172.21.15.119:0/827136256 shutdown_connections 2024-04-25T10:08:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.150+0000 7f2f52000640 1 -- 172.21.15.119:0/827136256 wait complete. 2024-04-25T10:08:43.174 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:43.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.364+0000 7fd85ee00640 1 Processor -- start 2024-04-25T10:08:43.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.364+0000 7fd85ee00640 1 -- start start 2024-04-25T10:08:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.365+0000 7fd85ee00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd85815acd0 0x7fd85806e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.365+0000 7fd85ee00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd85806e860 0x7fd8580b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.365+0000 7fd85ee00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85806ec70 con 0x7fd858144040 2024-04-25T10:08:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.365+0000 7fd85ee00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd858144420 con 0x7fd85815acd0 2024-04-25T10:08:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.365+0000 7fd85ee00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8580b3d80 con 0x7fd85806e860 2024-04-25T10:08:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.366+0000 7fd85d000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd85815acd0 0x7fd85806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.366+0000 7fd85d000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd85815acd0 0x7fd85806e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51528/0 (socket says 172.21.15.119:51528) 2024-04-25T10:08:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.366+0000 7fd85d000640 1 -- 172.21.15.119:0/849855708 learned_addr learned my addr 172.21.15.119:0/849855708 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.366+0000 7fd85e400640 1 --2- 172.21.15.119:0/849855708 >> v2:172.21.15.132:3300/0 conn(0x7fd85806e860 0x7fd8580b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.366+0000 7fd85d000640 1 -- 172.21.15.119:0/849855708 >> v2:172.21.15.132:3300/0 conn(0x7fd85806e860 msgr2=0x7fd8580b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:43.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.366+0000 7fd85d000640 1 --2- 172.21.15.119:0/849855708 >> v2:172.21.15.132:3300/0 conn(0x7fd85806e860 0x7fd8580b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.367+0000 7fd85d000640 1 -- 172.21.15.119:0/849855708 >> v1:172.21.15.132:6789/0 conn(0x7fd858144040 legacy=0x7fd8580cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.367+0000 7fd85d000640 1 -- 172.21.15.119:0/849855708 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8580b4060 con 0x7fd85815acd0 2024-04-25T10:08:43.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.367+0000 7fd85e400640 1 --2- 172.21.15.119:0/849855708 >> v2:172.21.15.132:3300/0 conn(0x7fd85806e860 0x7fd8580b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:43.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.368+0000 7fd85d000640 1 --2- 172.21.15.119:0/849855708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85815acd0 0x7fd85806e320 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd8580b58d0 tx=0x7fd84402c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdb02b909b425c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.368+0000 7fd857e00640 1 -- 172.21.15.119:0/849855708 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844033640 con 0x7fd85815acd0 2024-04-25T10:08:43.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.368+0000 7fd857e00640 1 -- 172.21.15.119:0/849855708 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd844038070 con 0x7fd85815acd0 2024-04-25T10:08:43.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.368+0000 7fd857e00640 1 -- 172.21.15.119:0/849855708 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844033a00 con 0x7fd85815acd0 2024-04-25T10:08:43.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.369+0000 7fd85ee00640 1 -- 172.21.15.119:0/849855708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85815acd0 msgr2=0x7fd85806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.369+0000 7fd85ee00640 1 --2- 172.21.15.119:0/849855708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85815acd0 0x7fd85806e320 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd8580b58d0 tx=0x7fd84402c8b0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.369+0000 7fd85ee00640 1 -- 172.21.15.119:0/849855708 shutdown_connections 2024-04-25T10:08:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.369+0000 7fd85ee00640 1 --2- 172.21.15.119:0/849855708 >> v2:172.21.15.132:3300/0 conn(0x7fd85806e860 0x7fd8580b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.369+0000 7fd85ee00640 1 --2- 172.21.15.119:0/849855708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85815acd0 0x7fd85806e320 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.369+0000 7fd85ee00640 1 -- 172.21.15.119:0/849855708 >> 172.21.15.119:0/849855708 conn(0x7fd85806b1d0 msgr2=0x7fd8580b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.369+0000 7fd85ee00640 1 -- 172.21.15.119:0/849855708 shutdown_connections 2024-04-25T10:08:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.369+0000 7fd85ee00640 1 -- 172.21.15.119:0/849855708 wait complete. 2024-04-25T10:08:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.371+0000 7fd85ee00640 1 Processor -- start 2024-04-25T10:08:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.371+0000 7fd85ee00640 1 -- start start 2024-04-25T10:08:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.372+0000 7fd85ee00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85806e860 0x7fd8581f8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.372+0000 7fd85ee00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd858144040 0x7fd85815c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.372+0000 7fd85ee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85815acd0 0x7fd85813c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.372+0000 7fd85ee00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580b4900 con 0x7fd85806e860 2024-04-25T10:08:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.372+0000 7fd85ee00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580b4620 con 0x7fd858144040 2024-04-25T10:08:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.372+0000 7fd85ee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580b5180 con 0x7fd85815acd0 2024-04-25T10:08:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.373+0000 7fd85da00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85806e860 0x7fd8581f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.373+0000 7fd85da00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85806e860 0x7fd8581f8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51530/0 (socket says 172.21.15.119:51530) 2024-04-25T10:08:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.373+0000 7fd85da00640 1 -- 172.21.15.119:0/1039711869 learned_addr learned my addr 172.21.15.119:0/1039711869 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.373+0000 7fd85d000640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd858144040 0x7fd85815c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.373+0000 7fd85e400640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85815acd0 0x7fd85813c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.373+0000 7fd85da00640 1 -- 172.21.15.119:0/1039711869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85815acd0 msgr2=0x7fd85813c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.373+0000 7fd85da00640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85815acd0 0x7fd85813c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.373+0000 7fd85da00640 1 -- 172.21.15.119:0/1039711869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd858144040 msgr2=0x7fd85815c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.373+0000 7fd85da00640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd858144040 0x7fd85815c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.373+0000 7fd85da00640 1 -- 172.21.15.119:0/1039711869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85815c830 con 0x7fd85806e860 2024-04-25T10:08:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.373+0000 7fd85e400640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85815acd0 0x7fd85813c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.373+0000 7fd85da00640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85806e860 0x7fd8581f8550 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00c430 tx=0x7fd84c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.374+0000 7fd855600640 1 -- 172.21.15.119:0/1039711869 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c0052d0 con 0x7fd85806e860 2024-04-25T10:08:43.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.374+0000 7fd855600640 1 -- 172.21.15.119:0/1039711869 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c005490 con 0x7fd85806e860 2024-04-25T10:08:43.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.374+0000 7fd85ee00640 1 -- 172.21.15.119:0/1039711869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85815cb10 con 0x7fd85806e860 2024-04-25T10:08:43.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.374+0000 7fd855600640 1 -- 172.21.15.119:0/1039711869 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c007260 con 0x7fd85806e860 2024-04-25T10:08:43.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.374+0000 7fd85ee00640 1 -- 172.21.15.119:0/1039711869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8581f7d50 con 0x7fd85806e860 2024-04-25T10:08:43.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.375+0000 7fd85ee00640 1 -- 172.21.15.119:0/1039711869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85806bb30 con 0x7fd85806e860 2024-04-25T10:08:43.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.376+0000 7fd855600640 1 -- 172.21.15.119:0/1039711869 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd84c007400 con 0x7fd85806e860 2024-04-25T10:08:43.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.377+0000 7fd855600640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd83806b4d0 0x7fd83806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.377+0000 7fd855600640 1 -- 172.21.15.119:0/1039711869 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd84c089660 con 0x7fd85806e860 2024-04-25T10:08:43.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.377+0000 7fd85d000640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd83806b4d0 0x7fd83806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.378+0000 7fd85d000640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd83806b4d0 0x7fd83806d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd844037000 tx=0x7fd844008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.382+0000 7fd855600640 1 -- 172.21.15.119:0/1039711869 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd85806ec70 con 0x7fd85806e860 2024-04-25T10:08:43.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.609+0000 7fd85ee00640 1 -- 172.21.15.119:0/1039711869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8580749b0 con 0x7fd85806e860 2024-04-25T10:08:43.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.610+0000 7fd855600640 1 -- 172.21.15.119:0/1039711869 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fd84c058570 con 0x7fd85806e860 2024-04-25T10:08:43.616 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:43.616 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1554372269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1554372269},{"type":"v1","addr":"172.21.15.119:6837","nonce":1554372269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4699,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4250304148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4250304148},{"type":"v1","addr":"172.21.15.132:6834","nonce":4250304148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/637229574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":637229574},{"type":"v1","addr":"172.21.15.132:6835","nonce":637229574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:14.553437+0000","modified":"2024-04-25T10:08:17.586270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5049":{"gid":5049,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6836/3585315292","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3585315292},{"type":"v1","addr":"172.21.15.119:6836","nonce":3585315292}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:08:43.616 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 16 2024-04-25T10:08:43.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.613+0000 7fd85ee00640 1 -- 172.21.15.119:0/1039711869 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd83806b4d0 msgr2=0x7fd83806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.613+0000 7fd85ee00640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd83806b4d0 0x7fd83806d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd844037000 tx=0x7fd844008040 comp rx=0 tx=0).stop 2024-04-25T10:08:43.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.613+0000 7fd85ee00640 1 -- 172.21.15.119:0/1039711869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85806e860 msgr2=0x7fd8581f8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.613+0000 7fd85ee00640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85806e860 0x7fd8581f8550 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00c430 tx=0x7fd84c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.613+0000 7fd85ee00640 1 -- 172.21.15.119:0/1039711869 shutdown_connections 2024-04-25T10:08:43.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.613+0000 7fd85ee00640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd83806b4d0 0x7fd83806d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.613+0000 7fd85ee00640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd85815acd0 0x7fd85813c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.613+0000 7fd85ee00640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd858144040 0x7fd85815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.613+0000 7fd85ee00640 1 --2- 172.21.15.119:0/1039711869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85806e860 0x7fd8581f8550 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.613+0000 7fd85ee00640 1 -- 172.21.15.119:0/1039711869 >> 172.21.15.119:0/1039711869 conn(0x7fd85806b1d0 msgr2=0x7fd8580b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.613+0000 7fd85ee00640 1 -- 172.21.15.119:0/1039711869 shutdown_connections 2024-04-25T10:08:43.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.613+0000 7fd85ee00640 1 -- 172.21.15.119:0/1039711869 wait complete. 2024-04-25T10:08:43.638 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.830+0000 7f94a5800640 1 Processor -- start 2024-04-25T10:08:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.830+0000 7f94a5800640 1 -- start start 2024-04-25T10:08:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.831+0000 7f94a5800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94a015ed70 0x7f94a0168a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.831+0000 7f94a5800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a00712c0 con 0x7f94a015d980 2024-04-25T10:08:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.831+0000 7f94a5800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a00715a0 con 0x7f94a00aecf0 2024-04-25T10:08:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.831+0000 7f94a5800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0071ed0 con 0x7f94a015ed70 2024-04-25T10:08:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.832+0000 7f949ea00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f94a015d980 0x7f94a00ae5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53804/0 (socket says 172.21.15.119:53804) 2024-04-25T10:08:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.832+0000 7f949ea00640 1 -- 172.21.15.119:0/3121267587 learned_addr learned my addr 172.21.15.119:0/3121267587 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:43.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.832+0000 7f949f400640 1 --2- 172.21.15.119:0/3121267587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94a015ed70 0x7f94a0168a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.832+0000 7f949d600640 1 -- 172.21.15.119:0/3121267587 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346962689 0 0) 0x7f94a00712c0 con 0x7f94a015d980 2024-04-25T10:08:43.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.832+0000 7f949d600640 1 -- 172.21.15.119:0/3121267587 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9484003520 con 0x7f94a015d980 2024-04-25T10:08:43.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949d600640 1 -- 172.21.15.119:0/3121267587 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3494300134 0 0) 0x7f9484003520 con 0x7f94a015d980 2024-04-25T10:08:43.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949d600640 1 -- 172.21.15.119:0/3121267587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94a015ed70 msgr2=0x7f94a0168a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:43.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949d600640 1 --2- 172.21.15.119:0/3121267587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94a015ed70 0x7f94a0168a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949d600640 1 -- 172.21.15.119:0/3121267587 >> v1:172.21.15.132:6789/0 conn(0x7f94a00aecf0 legacy=0x7f94a00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949d600640 1 -- 172.21.15.119:0/3121267587 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a0071880 con 0x7f94a015d980 2024-04-25T10:08:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949d600640 1 -- 172.21.15.119:0/3121267587 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f9490001670 con 0x7f94a015d980 2024-04-25T10:08:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949d600640 1 -- 172.21.15.119:0/3121267587 >> v1:172.21.15.119:6789/0 conn(0x7f94a015d980 legacy=0x7f94a00ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949d600640 1 --2- 172.21.15.119:0/3121267587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94840039e0 0x7f9484003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949d600640 1 --2- 172.21.15.119:0/3121267587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9484004800 0x7f9484004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949d600640 1 --2- 172.21.15.119:0/3121267587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9484005180 0x7f948400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949d600640 1 -- 172.21.15.119:0/3121267587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0071ed0 con 0x7f94840039e0 2024-04-25T10:08:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949d600640 1 -- 172.21.15.119:0/3121267587 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948400bf00 con 0x7f9484005180 2024-04-25T10:08:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949d600640 1 -- 172.21.15.119:0/3121267587 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948400c100 con 0x7f9484004800 2024-04-25T10:08:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949ea00640 1 --2- 172.21.15.119:0/3121267587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94840039e0 0x7f9484003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949ea00640 1 -- 172.21.15.119:0/3121267587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9484004800 msgr2=0x7f9484004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:43.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949ea00640 1 --2- 172.21.15.119:0/3121267587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9484004800 0x7f9484004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949ea00640 1 -- 172.21.15.119:0/3121267587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9484005180 msgr2=0x7f948400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:43.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.833+0000 7f949ea00640 1 --2- 172.21.15.119:0/3121267587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9484005180 0x7f948400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.834+0000 7f949ea00640 1 -- 172.21.15.119:0/3121267587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9484003520 con 0x7f94840039e0 2024-04-25T10:08:43.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.835+0000 7f949ea00640 1 --2- 172.21.15.119:0/3121267587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94840039e0 0x7f9484003e20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f949000c210 tx=0x7f9484005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e692b1aab755596e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.835+0000 7f949d600640 1 -- 172.21.15.119:0/3121267587 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490007cf0 con 0x7f94840039e0 2024-04-25T10:08:43.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.835+0000 7f949d600640 1 -- 172.21.15.119:0/3121267587 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9490005af0 con 0x7f94840039e0 2024-04-25T10:08:43.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.835+0000 7f949d600640 1 -- 172.21.15.119:0/3121267587 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490007070 con 0x7f94840039e0 2024-04-25T10:08:43.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.835+0000 7f94a5800640 1 -- 172.21.15.119:0/3121267587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94840039e0 msgr2=0x7f9484003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.835+0000 7f94a5800640 1 --2- 172.21.15.119:0/3121267587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94840039e0 0x7f9484003e20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f949000c210 tx=0x7f9484005830 comp rx=0 tx=0).stop 2024-04-25T10:08:43.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.835+0000 7f949f400640 1 -- 172.21.15.119:0/3121267587 reap_dead start 2024-04-25T10:08:43.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.835+0000 7f94a5800640 1 -- 172.21.15.119:0/3121267587 shutdown_connections 2024-04-25T10:08:43.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.835+0000 7f94a5800640 1 -- 172.21.15.119:0/3121267587 >> 172.21.15.119:0/3121267587 conn(0x7f94a006aba0 msgr2=0x7f94a006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.836+0000 7f94a5800640 1 -- 172.21.15.119:0/3121267587 shutdown_connections 2024-04-25T10:08:43.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.836+0000 7f94a5800640 1 -- 172.21.15.119:0/3121267587 wait complete. 2024-04-25T10:08:43.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.837+0000 7f94a5800640 1 Processor -- start 2024-04-25T10:08:43.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.837+0000 7f94a5800640 1 -- start start 2024-04-25T10:08:43.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.838+0000 7f94a5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94840039e0 0x7f94a0154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.838+0000 7f94a5800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a0152e20 0x7f94a014ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.838+0000 7f94a5800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94a015fd70 0x7f94a01601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.838+0000 7f94a5800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00cfb10 con 0x7f94840039e0 2024-04-25T10:08:43.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.838+0000 7f94a5800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00cf990 con 0x7f94a0152e20 2024-04-25T10:08:43.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.838+0000 7f94a5800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00adf80 con 0x7f94a015fd70 2024-04-25T10:08:43.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.839+0000 7f949ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94840039e0 0x7f94a0154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.839+0000 7f949ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94840039e0 0x7f94a0154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51542/0 (socket says 172.21.15.119:51542) 2024-04-25T10:08:43.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.839+0000 7f949ea00640 1 -- 172.21.15.119:0/999080132 learned_addr learned my addr 172.21.15.119:0/999080132 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:43.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.839+0000 7f949e000640 1 --2- 172.21.15.119:0/999080132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a0152e20 0x7f94a014ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.839+0000 7f949ea00640 1 -- 172.21.15.119:0/999080132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94a015fd70 msgr2=0x7f94a01601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.839+0000 7f949ea00640 1 --2- 172.21.15.119:0/999080132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94a015fd70 0x7f94a01601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.839+0000 7f949ea00640 1 -- 172.21.15.119:0/999080132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a0152e20 msgr2=0x7f94a014ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.839+0000 7f949ea00640 1 --2- 172.21.15.119:0/999080132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a0152e20 0x7f94a014ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.839+0000 7f949ea00640 1 -- 172.21.15.119:0/999080132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a0155190 con 0x7f94840039e0 2024-04-25T10:08:43.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.839+0000 7f949e000640 1 --2- 172.21.15.119:0/999080132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a0152e20 0x7f94a014ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:43.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.839+0000 7f949ea00640 1 --2- 172.21.15.119:0/999080132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94840039e0 0x7f94a0154ad0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f949002add0 tx=0x7f94a0071770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.840+0000 7f9496a00640 1 -- 172.21.15.119:0/999080132 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490041020 con 0x7f94840039e0 2024-04-25T10:08:43.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.840+0000 7f9496a00640 1 -- 172.21.15.119:0/999080132 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9490038040 con 0x7f94840039e0 2024-04-25T10:08:43.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.840+0000 7f9496a00640 1 -- 172.21.15.119:0/999080132 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94900072b0 con 0x7f94840039e0 2024-04-25T10:08:43.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.840+0000 7f94a5800640 1 -- 172.21.15.119:0/999080132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a0151040 con 0x7f94840039e0 2024-04-25T10:08:43.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.840+0000 7f94a5800640 1 -- 172.21.15.119:0/999080132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a0151600 con 0x7f94840039e0 2024-04-25T10:08:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.841+0000 7f94a5800640 1 -- 172.21.15.119:0/999080132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a00749b0 con 0x7f94840039e0 2024-04-25T10:08:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.842+0000 7f9496a00640 1 -- 172.21.15.119:0/999080132 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9490044020 con 0x7f94840039e0 2024-04-25T10:08:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.843+0000 7f9496a00640 1 --2- 172.21.15.119:0/999080132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f948006b4d0 0x7f948006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.843+0000 7f9496a00640 1 -- 172.21.15.119:0/999080132 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94900aa460 con 0x7f94840039e0 2024-04-25T10:08:43.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.844+0000 7f949e000640 1 --2- 172.21.15.119:0/999080132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f948006b4d0 0x7f948006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.844+0000 7f949e000640 1 --2- 172.21.15.119:0/999080132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f948006b4d0 0x7f948006d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f94a0152a10 tx=0x7f948c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:43.848+0000 7f9496a00640 1 -- 172.21.15.119:0/999080132 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9490045e50 con 0x7f94840039e0 2024-04-25T10:08:44.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.045+0000 7f94a5800640 1 -- 172.21.15.119:0/999080132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94a006ebc0 con 0x7f94840039e0 2024-04-25T10:08:44.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.047+0000 7f9496a00640 1 -- 172.21.15.119:0/999080132 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16705 (secure 0 0 0) 0x7f9490079240 con 0x7f94840039e0 2024-04-25T10:08:44.053 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:44.054 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":33,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:16.095956+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:12.268457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:13.546259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.050+0000 7f94a5800640 1 -- 172.21.15.119:0/999080132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f948006b4d0 msgr2=0x7f948006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.050+0000 7f94a5800640 1 --2- 172.21.15.119:0/999080132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f948006b4d0 0x7f948006d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f94a0152a10 tx=0x7f948c009040 comp rx=0 tx=0).stop 2024-04-25T10:08:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.050+0000 7f94a5800640 1 -- 172.21.15.119:0/999080132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94840039e0 msgr2=0x7f94a0154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.050+0000 7f94a5800640 1 --2- 172.21.15.119:0/999080132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94840039e0 0x7f94a0154ad0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f949002add0 tx=0x7f94a0071770 comp rx=0 tx=0).stop 2024-04-25T10:08:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.050+0000 7f94a5800640 1 -- 172.21.15.119:0/999080132 shutdown_connections 2024-04-25T10:08:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.050+0000 7f94a5800640 1 --2- 172.21.15.119:0/999080132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f948006b4d0 0x7f948006d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.050+0000 7f94a5800640 1 --2- 172.21.15.119:0/999080132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94a015fd70 0x7f94a01601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.050+0000 7f94a5800640 1 --2- 172.21.15.119:0/999080132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a0152e20 0x7f94a014ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.050+0000 7f94a5800640 1 --2- 172.21.15.119:0/999080132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94840039e0 0x7f94a0154ad0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.050+0000 7f94a5800640 1 -- 172.21.15.119:0/999080132 >> 172.21.15.119:0/999080132 conn(0x7f94a006aba0 msgr2=0x7f94a00acdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.050+0000 7f94a5800640 1 -- 172.21.15.119:0/999080132 shutdown_connections 2024-04-25T10:08:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.050+0000 7f94a5800640 1 -- 172.21.15.119:0/999080132 wait complete. 2024-04-25T10:08:44.077 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:08:44.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.269+0000 7f803f200640 1 Processor -- start 2024-04-25T10:08:44.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.269+0000 7f803f200640 1 -- start start 2024-04-25T10:08:44.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.271+0000 7f803f200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f804015db10 0x7f80400b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.271+0000 7f803f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80400ad130 0x7f804006d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.271+0000 7f803f200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f804006dea0 0x7f804006bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.271+0000 7f803f200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80400b3410 con 0x7f804015db10 2024-04-25T10:08:44.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.271+0000 7f803f200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80400b36f0 con 0x7f804006dea0 2024-04-25T10:08:44.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.271+0000 7f803f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80400b39d0 con 0x7f80400ad130 2024-04-25T10:08:44.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.271+0000 7f803de00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f804015db10 0x7f80400b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.271+0000 7f803de00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f804015db10 0x7f80400b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51544/0 (socket says 172.21.15.119:51544) 2024-04-25T10:08:44.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.271+0000 7f803de00640 1 -- 172.21.15.119:0/1453799415 learned_addr learned my addr 172.21.15.119:0/1453799415 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:44.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.272+0000 7f803de00640 1 -- 172.21.15.119:0/1453799415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80400ad130 msgr2=0x7f804006d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:44.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.272+0000 7f803de00640 1 --2- 172.21.15.119:0/1453799415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80400ad130 0x7f804006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.272+0000 7f803de00640 1 -- 172.21.15.119:0/1453799415 >> v2:172.21.15.132:3300/0 conn(0x7f804006dea0 msgr2=0x7f804006bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:44.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.272+0000 7f803de00640 1 --2- 172.21.15.119:0/1453799415 >> v2:172.21.15.132:3300/0 conn(0x7f804006dea0 0x7f804006bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.272+0000 7f803de00640 1 -- 172.21.15.119:0/1453799415 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80400b3cb0 con 0x7f804015db10 2024-04-25T10:08:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.273+0000 7f803de00640 1 --2- 172.21.15.119:0/1453799415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f804015db10 0x7f80400b2bc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8040072d80 tx=0x7f803002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a06f026d30c54c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.273+0000 7f803d400640 1 -- 172.21.15.119:0/1453799415 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030037070 con 0x7f804015db10 2024-04-25T10:08:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.273+0000 7f803d400640 1 -- 172.21.15.119:0/1453799415 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8030032440 con 0x7f804015db10 2024-04-25T10:08:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.273+0000 7f803d400640 1 -- 172.21.15.119:0/1453799415 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030032810 con 0x7f804015db10 2024-04-25T10:08:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.274+0000 7f803f200640 1 -- 172.21.15.119:0/1453799415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f804015db10 msgr2=0x7f80400b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.274+0000 7f803f200640 1 --2- 172.21.15.119:0/1453799415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f804015db10 0x7f80400b2bc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8040072d80 tx=0x7f803002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.274+0000 7f803f200640 1 -- 172.21.15.119:0/1453799415 shutdown_connections 2024-04-25T10:08:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.274+0000 7f803f200640 1 --2- 172.21.15.119:0/1453799415 >> v2:172.21.15.132:3300/0 conn(0x7f804006dea0 0x7f804006bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.274+0000 7f803f200640 1 --2- 172.21.15.119:0/1453799415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80400ad130 0x7f804006d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.274+0000 7f803f200640 1 --2- 172.21.15.119:0/1453799415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f804015db10 0x7f80400b2bc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.274+0000 7f803f200640 1 -- 172.21.15.119:0/1453799415 >> 172.21.15.119:0/1453799415 conn(0x7f8040071900 msgr2=0x7f8040071d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:44.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.274+0000 7f803f200640 1 -- 172.21.15.119:0/1453799415 shutdown_connections 2024-04-25T10:08:44.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.274+0000 7f803f200640 1 -- 172.21.15.119:0/1453799415 wait complete. 2024-04-25T10:08:44.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.276+0000 7f803f200640 1 Processor -- start 2024-04-25T10:08:44.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.276+0000 7f803f200640 1 -- start start 2024-04-25T10:08:44.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f804006dea0 0x7f804015f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80400ad130 0x7f80400cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f804015db10 0x7f804006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804015def0 con 0x7f804015db10 2024-04-25T10:08:44.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80400b3100 con 0x7f804006dea0 2024-04-25T10:08:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804006c0f0 con 0x7f80400ad130 2024-04-25T10:08:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f804015db10 0x7f804006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f804015db10 0x7f804006ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51560/0 (socket says 172.21.15.119:51560) 2024-04-25T10:08:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803e800640 1 -- 172.21.15.119:0/1861471420 learned_addr learned my addr 172.21.15.119:0/1861471420 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803de00640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f804006dea0 0x7f804015f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f8035400640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80400ad130 0x7f80400cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803e800640 1 -- 172.21.15.119:0/1861471420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80400ad130 msgr2=0x7f80400cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803e800640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80400ad130 0x7f80400cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803e800640 1 -- 172.21.15.119:0/1861471420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f804006dea0 msgr2=0x7f804015f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803e800640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f804006dea0 0x7f804015f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.277+0000 7f803e800640 1 -- 172.21.15.119:0/1861471420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804014d810 con 0x7f804015db10 2024-04-25T10:08:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.278+0000 7f803de00640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f804006dea0 0x7f804015f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.278+0000 7f8035400640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80400ad130 0x7f80400cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.278+0000 7f803e800640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f804015db10 0x7f804006ec10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f803800b970 tx=0x7f803800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.278+0000 7f8036a00640 1 -- 172.21.15.119:0/1861471420 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8038019070 con 0x7f804015db10 2024-04-25T10:08:44.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.278+0000 7f8036a00640 1 -- 172.21.15.119:0/1861471420 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803800cb60 con 0x7f804015db10 2024-04-25T10:08:44.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.278+0000 7f8036a00640 1 -- 172.21.15.119:0/1861471420 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8038014440 con 0x7f804015db10 2024-04-25T10:08:44.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.278+0000 7f803f200640 1 -- 172.21.15.119:0/1861471420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80401f3ae0 con 0x7f804015db10 2024-04-25T10:08:44.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.278+0000 7f803f200640 1 -- 172.21.15.119:0/1861471420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80401f40a0 con 0x7f804015db10 2024-04-25T10:08:44.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.279+0000 7f803f200640 1 -- 172.21.15.119:0/1861471420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80400749b0 con 0x7f804015db10 2024-04-25T10:08:44.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.280+0000 7f8036a00640 1 -- 172.21.15.119:0/1861471420 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f803801c070 con 0x7f804015db10 2024-04-25T10:08:44.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.281+0000 7f8036a00640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f801c06b4d0 0x7f801c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.281+0000 7f8036a00640 1 -- 172.21.15.119:0/1861471420 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8038089f00 con 0x7f804015db10 2024-04-25T10:08:44.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.281+0000 7f803de00640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f801c06b4d0 0x7f801c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.282+0000 7f803de00640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f801c06b4d0 0x7f801c06d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8030034000 tx=0x7f803002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.286+0000 7f8036a00640 1 -- 172.21.15.119:0/1861471420 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8038058d70 con 0x7f804015db10 2024-04-25T10:08:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.517+0000 7f803f200640 1 -- 172.21.15.119:0/1861471420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f80400b1260 con 0x7f804015db10 2024-04-25T10:08:44.523 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:08:44.518+0000 7f8c64600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:08:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.521+0000 7f8036a00640 1 -- 172.21.15.119:0/1861471420 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8038058670 con 0x7f804015db10 2024-04-25T10:08:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:08:44.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.524+0000 7f803f200640 1 -- 172.21.15.119:0/1861471420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f801c06b4d0 msgr2=0x7f801c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.524+0000 7f803f200640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f801c06b4d0 0x7f801c06d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8030034000 tx=0x7f803002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:44.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.524+0000 7f803f200640 1 -- 172.21.15.119:0/1861471420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f804015db10 msgr2=0x7f804006ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.524+0000 7f803f200640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f804015db10 0x7f804006ec10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f803800b970 tx=0x7f803800be30 comp rx=0 tx=0).stop 2024-04-25T10:08:44.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.524+0000 7f803f200640 1 -- 172.21.15.119:0/1861471420 shutdown_connections 2024-04-25T10:08:44.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.524+0000 7f803f200640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f801c06b4d0 0x7f801c06d990 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8030034000 tx=0x7f803002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:44.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.524+0000 7f803f200640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f804015db10 0x7f804006ec10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.524+0000 7f803f200640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f80400ad130 0x7f80400cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.524+0000 7f803f200640 1 --2- 172.21.15.119:0/1861471420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f804006dea0 0x7f804015f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.524+0000 7f803f200640 1 -- 172.21.15.119:0/1861471420 >> 172.21.15.119:0/1861471420 conn(0x7f8040071900 msgr2=0x7f80400af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:44.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.525+0000 7f803f200640 1 -- 172.21.15.119:0/1861471420 shutdown_connections 2024-04-25T10:08:44.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.525+0000 7f803f200640 1 -- 172.21.15.119:0/1861471420 wait complete. 2024-04-25T10:08:44.542 INFO:tasks.ceph.mds.a.smithi119.stderr: -6669> 2024-04-25T10:08:17.019+0000 7fe4e1e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:08:44.551 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:08:44.604 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:08:44.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.719+0000 7f5093e00640 1 Processor -- start 2024-04-25T10:08:44.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.719+0000 7f5093e00640 1 -- start start 2024-04-25T10:08:44.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.720+0000 7f5093e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f509415bcb0 0x7f509406da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.720+0000 7f5093e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50940712c0 con 0x7f50940cea40 2024-04-25T10:08:44.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.721+0000 7f5093e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50940715a0 con 0x7f50940c6150 2024-04-25T10:08:44.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.721+0000 7f5093e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509406e790 con 0x7f509415bcb0 2024-04-25T10:08:44.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.721+0000 7f5093400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f50940cea40 0x7f509406df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53816/0 (socket says 172.21.15.119:53816) 2024-04-25T10:08:44.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.721+0000 7f5093400640 1 -- 172.21.15.119:0/2558776892 learned_addr learned my addr 172.21.15.119:0/2558776892 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:44.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.721+0000 7f508be00640 1 --2- 172.21.15.119:0/2558776892 >> v2:172.21.15.132:3300/0 conn(0x7f509415bcb0 0x7f509406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.721+0000 7f5092000640 1 -- 172.21.15.119:0/2558776892 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706949529 0 0) 0x7f50940712c0 con 0x7f50940cea40 2024-04-25T10:08:44.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.721+0000 7f5092000640 1 -- 172.21.15.119:0/2558776892 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5078003520 con 0x7f50940cea40 2024-04-25T10:08:44.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f5092000640 1 -- 172.21.15.119:0/2558776892 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 8188547 0 0) 0x7f5078003520 con 0x7f50940cea40 2024-04-25T10:08:44.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f5092000640 1 -- 172.21.15.119:0/2558776892 >> v2:172.21.15.132:3300/0 conn(0x7f509415bcb0 msgr2=0x7f509406da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:44.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f5092000640 1 --2- 172.21.15.119:0/2558776892 >> v2:172.21.15.132:3300/0 conn(0x7f509415bcb0 0x7f509406da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f5092000640 1 -- 172.21.15.119:0/2558776892 >> v1:172.21.15.132:6789/0 conn(0x7f50940c6150 legacy=0x7f50940ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f5092000640 1 -- 172.21.15.119:0/2558776892 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5094071880 con 0x7f50940cea40 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f5092000640 1 -- 172.21.15.119:0/2558776892 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f508c001a10 con 0x7f50940cea40 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f5092000640 1 -- 172.21.15.119:0/2558776892 >> v1:172.21.15.119:6789/0 conn(0x7f50940cea40 legacy=0x7f509406df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f5092000640 1 --2- 172.21.15.119:0/2558776892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5078003a00 0x7f5078003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f5092000640 1 --2- 172.21.15.119:0/2558776892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078004800 0x7f5078004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f5092000640 1 --2- 172.21.15.119:0/2558776892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50780051c0 0x7f507800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f5092000640 1 -- 172.21.15.119:0/2558776892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507800c090 con 0x7f5078004800 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f5092000640 1 -- 172.21.15.119:0/2558776892 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507800c240 con 0x7f5078003a00 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f5092000640 1 -- 172.21.15.119:0/2558776892 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507800c440 con 0x7f50780051c0 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f508be00640 1 --2- 172.21.15.119:0/2558776892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078004800 0x7f5078004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f508be00640 1 -- 172.21.15.119:0/2558776892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50780051c0 msgr2=0x7f507800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f508be00640 1 --2- 172.21.15.119:0/2558776892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50780051c0 0x7f507800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f5092a00640 1 --2- 172.21.15.119:0/2558776892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5078003a00 0x7f5078003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.722+0000 7f508be00640 1 -- 172.21.15.119:0/2558776892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5078003a00 msgr2=0x7f5078003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.723+0000 7f508be00640 1 --2- 172.21.15.119:0/2558776892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5078003a00 0x7f5078003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.723+0000 7f508be00640 1 -- 172.21.15.119:0/2558776892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5078003520 con 0x7f5078004800 2024-04-25T10:08:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.723+0000 7f5093400640 1 -- 172.21.15.119:0/2558776892 reap_dead start 2024-04-25T10:08:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.724+0000 7f508be00640 1 --2- 172.21.15.119:0/2558776892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078004800 0x7f5078004c80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f507c009ac0 tx=0x7f5078005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d955697b8b0a7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.724+0000 7f5092000640 1 -- 172.21.15.119:0/2558776892 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c036070 con 0x7f5078004800 2024-04-25T10:08:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.724+0000 7f5092000640 1 -- 172.21.15.119:0/2558776892 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c031400 con 0x7f5078004800 2024-04-25T10:08:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.724+0000 7f5092000640 1 -- 172.21.15.119:0/2558776892 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c009690 con 0x7f5078004800 2024-04-25T10:08:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.724+0000 7f5093e00640 1 -- 172.21.15.119:0/2558776892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078004800 msgr2=0x7f5078004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.724+0000 7f5093e00640 1 --2- 172.21.15.119:0/2558776892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078004800 0x7f5078004c80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f507c009ac0 tx=0x7f5078005830 comp rx=0 tx=0).stop 2024-04-25T10:08:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.724+0000 7f5093e00640 1 -- 172.21.15.119:0/2558776892 shutdown_connections 2024-04-25T10:08:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.724+0000 7f5093e00640 1 --2- 172.21.15.119:0/2558776892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078004800 0x7f5078004c80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.724+0000 7f5093e00640 1 -- 172.21.15.119:0/2558776892 >> 172.21.15.119:0/2558776892 conn(0x7f509406aba0 msgr2=0x7f509406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.725+0000 7f5093e00640 1 -- 172.21.15.119:0/2558776892 shutdown_connections 2024-04-25T10:08:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.725+0000 7f5093e00640 1 -- 172.21.15.119:0/2558776892 wait complete. 2024-04-25T10:08:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.726+0000 7f5093e00640 1 Processor -- start 2024-04-25T10:08:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.726+0000 7f5093e00640 1 -- start start 2024-04-25T10:08:44.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.727+0000 7f5093e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5078004800 0x7f50941534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.727+0000 7f5093e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5094154c10 0x7f509414efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.727+0000 7f5093e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5094138a50 0x7f509414c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.727+0000 7f5093e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50940ce330 con 0x7f5094154c10 2024-04-25T10:08:44.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.727+0000 7f5093e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50940c6530 con 0x7f5078004800 2024-04-25T10:08:44.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.727+0000 7f5093e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50940ce4b0 con 0x7f5094138a50 2024-04-25T10:08:44.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.727+0000 7f508be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5094154c10 0x7f509414efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.728+0000 7f508be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5094154c10 0x7f509414efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51592/0 (socket says 172.21.15.119:51592) 2024-04-25T10:08:44.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.728+0000 7f508be00640 1 -- 172.21.15.119:0/2944702186 learned_addr learned my addr 172.21.15.119:0/2944702186 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:44.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.728+0000 7f5092a00640 1 --2- 172.21.15.119:0/2944702186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5078004800 0x7f50941534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.728+0000 7f508be00640 1 -- 172.21.15.119:0/2944702186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5094138a50 msgr2=0x7f509414c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.728+0000 7f508be00640 1 --2- 172.21.15.119:0/2944702186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5094138a50 0x7f509414c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.728+0000 7f508be00640 1 -- 172.21.15.119:0/2944702186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5078004800 msgr2=0x7f50941534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.728+0000 7f508be00640 1 --2- 172.21.15.119:0/2944702186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5078004800 0x7f50941534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.728+0000 7f508be00640 1 -- 172.21.15.119:0/2944702186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509414f520 con 0x7f5094154c10 2024-04-25T10:08:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.728+0000 7f508be00640 1 --2- 172.21.15.119:0/2944702186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5094154c10 0x7f509414efe0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f507c009be0 tx=0x7f507c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.728+0000 7f508aa00640 1 -- 172.21.15.119:0/2944702186 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c03f020 con 0x7f5094154c10 2024-04-25T10:08:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.729+0000 7f508aa00640 1 -- 172.21.15.119:0/2944702186 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c037040 con 0x7f5094154c10 2024-04-25T10:08:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.729+0000 7f5093e00640 1 -- 172.21.15.119:0/2944702186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5094153b70 con 0x7f5094154c10 2024-04-25T10:08:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.729+0000 7f508aa00640 1 -- 172.21.15.119:0/2944702186 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c036070 con 0x7f5094154c10 2024-04-25T10:08:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.729+0000 7f5093e00640 1 -- 172.21.15.119:0/2944702186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5094154130 con 0x7f5094154c10 2024-04-25T10:08:44.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.730+0000 7f5093e00640 1 -- 172.21.15.119:0/2944702186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50940749b0 con 0x7f5094154c10 2024-04-25T10:08:44.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.731+0000 7f508aa00640 1 -- 172.21.15.119:0/2944702186 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f507c042020 con 0x7f5094154c10 2024-04-25T10:08:44.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.732+0000 7f508aa00640 1 --2- 172.21.15.119:0/2944702186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f507006b500 0x7f507006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.732+0000 7f508aa00640 1 -- 172.21.15.119:0/2944702186 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f507c0aa240 con 0x7f5094154c10 2024-04-25T10:08:44.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.733+0000 7f5092a00640 1 --2- 172.21.15.119:0/2944702186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f507006b500 0x7f507006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.734+0000 7f5092a00640 1 --2- 172.21.15.119:0/2944702186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f507006b500 0x7f507006d9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f509406d2b0 tx=0x7f5084009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.737+0000 7f508aa00640 1 -- 172.21.15.119:0/2944702186 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f507c049030 con 0x7f5094154c10 2024-04-25T10:08:44.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:44.971+0000 7f5093e00640 1 -- 172.21.15.119:0/2944702186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5094152530 con 0x7f5094154c10 2024-04-25T10:08:45.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.524+0000 7f508aa00640 1 -- 172.21.15.119:0/2944702186 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f507c0790f0 con 0x7f5094154c10 2024-04-25T10:08:45.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.528+0000 7f5093e00640 1 -- 172.21.15.119:0/2944702186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f507006b500 msgr2=0x7f507006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.528+0000 7f5093e00640 1 --2- 172.21.15.119:0/2944702186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f507006b500 0x7f507006d9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f509406d2b0 tx=0x7f5084009040 comp rx=0 tx=0).stop 2024-04-25T10:08:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.528+0000 7f5093e00640 1 -- 172.21.15.119:0/2944702186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5094154c10 msgr2=0x7f509414efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.528+0000 7f5093e00640 1 --2- 172.21.15.119:0/2944702186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5094154c10 0x7f509414efe0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f507c009be0 tx=0x7f507c004730 comp rx=0 tx=0).stop 2024-04-25T10:08:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.528+0000 7f5093e00640 1 -- 172.21.15.119:0/2944702186 shutdown_connections 2024-04-25T10:08:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.528+0000 7f5093e00640 1 --2- 172.21.15.119:0/2944702186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f507006b500 0x7f507006d9c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.528+0000 7f5093e00640 1 --2- 172.21.15.119:0/2944702186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5094138a50 0x7f509414c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.528+0000 7f5093e00640 1 --2- 172.21.15.119:0/2944702186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5094154c10 0x7f509414efe0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.528+0000 7f5093e00640 1 --2- 172.21.15.119:0/2944702186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5078004800 0x7f50941534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.528+0000 7f5093e00640 1 -- 172.21.15.119:0/2944702186 >> 172.21.15.119:0/2944702186 conn(0x7f509406aba0 msgr2=0x7f509406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.529+0000 7f5093e00640 1 -- 172.21.15.119:0/2944702186 shutdown_connections 2024-04-25T10:08:45.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.529+0000 7f5093e00640 1 -- 172.21.15.119:0/2944702186 wait complete. 2024-04-25T10:08:45.556 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:08:45.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.745+0000 7fbc33c00640 1 Processor -- start 2024-04-25T10:08:45.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.745+0000 7fbc33c00640 1 -- start start 2024-04-25T10:08:45.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.747+0000 7fbc33c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc2c15dbe0 0x7fbc2c071c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.747+0000 7fbc33c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc2c0cfad0 0x7fbc2c06d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.747+0000 7fbc33c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2c0ace70 con 0x7fbc2c0aedc0 2024-04-25T10:08:45.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.747+0000 7fbc33c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2c072150 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.747+0000 7fbc33c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0ad110 con 0x7fbc2c15dbe0 2024-04-25T10:08:45.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.747+0000 7fbc31c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc2c0cfad0 0x7fbc2c06d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.747+0000 7fbc23e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbc2c0aedc0 0x7fbc2c0cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53828/0 (socket says 172.21.15.119:53828) 2024-04-25T10:08:45.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.747+0000 7fbc23e00640 1 -- 172.21.15.119:0/3542956182 learned_addr learned my addr 172.21.15.119:0/3542956182 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:45.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.748+0000 7fbc31200640 1 --2- 172.21.15.119:0/3542956182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc2c15dbe0 0x7fbc2c071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.748+0000 7fbc31c00640 1 -- 172.21.15.119:0/3542956182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc2c15dbe0 msgr2=0x7fbc2c071c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:45.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.748+0000 7fbc31c00640 1 --2- 172.21.15.119:0/3542956182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc2c15dbe0 0x7fbc2c071c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.748+0000 7fbc31c00640 1 -- 172.21.15.119:0/3542956182 >> v1:172.21.15.119:6789/0 conn(0x7fbc2c0aedc0 legacy=0x7fbc2c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.748+0000 7fbc31c00640 1 -- 172.21.15.119:0/3542956182 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c0ad3f0 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.749+0000 7fbc31c00640 1 --2- 172.21.15.119:0/3542956182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c0cfad0 0x7fbc2c06d9a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbc240097d0 tx=0x7fbc2402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218b1a6be0bd16e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.749+0000 7fbc2be00640 1 -- 172.21.15.119:0/3542956182 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24037070 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.749+0000 7fbc2be00640 1 -- 172.21.15.119:0/3542956182 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc24032440 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.749+0000 7fbc2be00640 1 -- 172.21.15.119:0/3542956182 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24032810 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.750+0000 7fbc33c00640 1 -- 172.21.15.119:0/3542956182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c0cfad0 msgr2=0x7fbc2c06d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.750+0000 7fbc33c00640 1 --2- 172.21.15.119:0/3542956182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c0cfad0 0x7fbc2c06d9a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbc240097d0 tx=0x7fbc2402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.750+0000 7fbc33c00640 1 -- 172.21.15.119:0/3542956182 shutdown_connections 2024-04-25T10:08:45.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.750+0000 7fbc33c00640 1 --2- 172.21.15.119:0/3542956182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c0cfad0 0x7fbc2c06d9a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.750+0000 7fbc33c00640 1 --2- 172.21.15.119:0/3542956182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc2c15dbe0 0x7fbc2c071c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.750+0000 7fbc33c00640 1 -- 172.21.15.119:0/3542956182 >> 172.21.15.119:0/3542956182 conn(0x7fbc2c06aba0 msgr2=0x7fbc2c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:45.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.750+0000 7fbc33c00640 1 -- 172.21.15.119:0/3542956182 shutdown_connections 2024-04-25T10:08:45.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.750+0000 7fbc33c00640 1 -- 172.21.15.119:0/3542956182 wait complete. 2024-04-25T10:08:45.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.752+0000 7fbc33c00640 1 Processor -- start 2024-04-25T10:08:45.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.752+0000 7fbc33c00640 1 -- start start 2024-04-25T10:08:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.753+0000 7fbc33c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc2c0aedc0 0x7fbc2c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.753+0000 7fbc33c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c0cfad0 0x7fbc2c0af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.753+0000 7fbc33c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc2c15dbe0 0x7fbc2c1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.753+0000 7fbc33c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0adc90 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.753+0000 7fbc33c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0ad9b0 con 0x7fbc2c0aedc0 2024-04-25T10:08:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.753+0000 7fbc33c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c15dfc0 con 0x7fbc2c15dbe0 2024-04-25T10:08:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.753+0000 7fbc23e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c0cfad0 0x7fbc2c0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.753+0000 7fbc23e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c0cfad0 0x7fbc2c0af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51608/0 (socket says 172.21.15.119:51608) 2024-04-25T10:08:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.753+0000 7fbc23e00640 1 -- 172.21.15.119:0/4057375824 learned_addr learned my addr 172.21.15.119:0/4057375824 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:45.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.753+0000 7fbc31200640 1 --2- 172.21.15.119:0/4057375824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc2c0aedc0 0x7fbc2c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.754+0000 7fbc31c00640 1 --2- 172.21.15.119:0/4057375824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc2c15dbe0 0x7fbc2c1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.754+0000 7fbc23e00640 1 -- 172.21.15.119:0/4057375824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc2c15dbe0 msgr2=0x7fbc2c1552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.754+0000 7fbc23e00640 1 --2- 172.21.15.119:0/4057375824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc2c15dbe0 0x7fbc2c1552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.754+0000 7fbc23e00640 1 -- 172.21.15.119:0/4057375824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc2c0aedc0 msgr2=0x7fbc2c0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.754+0000 7fbc23e00640 1 --2- 172.21.15.119:0/4057375824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc2c0aedc0 0x7fbc2c0b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.754+0000 7fbc23e00640 1 -- 172.21.15.119:0/4057375824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c0af770 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.754+0000 7fbc23e00640 1 --2- 172.21.15.119:0/4057375824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c0cfad0 0x7fbc2c0af230 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400c440 tx=0x7fbc14001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.754+0000 7fbc29600640 1 -- 172.21.15.119:0/4057375824 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14017070 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.754+0000 7fbc29600640 1 -- 172.21.15.119:0/4057375824 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc14005d00 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.754+0000 7fbc33c00640 1 -- 172.21.15.119:0/4057375824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2c1557f0 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.754+0000 7fbc29600640 1 -- 172.21.15.119:0/4057375824 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14007310 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.755+0000 7fbc33c00640 1 -- 172.21.15.119:0/4057375824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c1f7260 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.756+0000 7fbc33c00640 1 -- 172.21.15.119:0/4057375824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2c0750b0 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.757+0000 7fbc29600640 1 -- 172.21.15.119:0/4057375824 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc1401a070 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.758+0000 7fbc29600640 1 --2- 172.21.15.119:0/4057375824 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbc0806b500 0x7fbc0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.758+0000 7fbc29600640 1 -- 172.21.15.119:0/4057375824 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbc140895a0 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.758+0000 7fbc31200640 1 --2- 172.21.15.119:0/4057375824 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbc0806b500 0x7fbc0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.758+0000 7fbc31200640 1 --2- 172.21.15.119:0/4057375824 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbc0806b500 0x7fbc0806d9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0099c0 tx=0x7fbc1c003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.764+0000 7fbc29600640 1 -- 172.21.15.119:0/4057375824 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc2c1557f0 con 0x7fbc2c0cfad0 2024-04-25T10:08:45.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:45.946+0000 7fbc33c00640 1 -- 172.21.15.119:0/4057375824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbc2c158870 con 0x7fbc2c0cfad0 2024-04-25T10:08:46.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.534+0000 7fbc29600640 1 -- 172.21.15.119:0/4057375824 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbc2c0af770 con 0x7fbc2c0cfad0 2024-04-25T10:08:46.541 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:08:46.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.538+0000 7fbc33c00640 1 -- 172.21.15.119:0/4057375824 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbc0806b500 msgr2=0x7fbc0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.538+0000 7fbc33c00640 1 --2- 172.21.15.119:0/4057375824 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbc0806b500 0x7fbc0806d9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0099c0 tx=0x7fbc1c003990 comp rx=0 tx=0).stop 2024-04-25T10:08:46.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.538+0000 7fbc33c00640 1 -- 172.21.15.119:0/4057375824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c0cfad0 msgr2=0x7fbc2c0af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.538+0000 7fbc33c00640 1 --2- 172.21.15.119:0/4057375824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c0cfad0 0x7fbc2c0af230 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400c440 tx=0x7fbc14001e70 comp rx=0 tx=0).stop 2024-04-25T10:08:46.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.538+0000 7fbc33c00640 1 -- 172.21.15.119:0/4057375824 shutdown_connections 2024-04-25T10:08:46.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.538+0000 7fbc33c00640 1 --2- 172.21.15.119:0/4057375824 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbc0806b500 0x7fbc0806d9c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.538+0000 7fbc33c00640 1 --2- 172.21.15.119:0/4057375824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc2c15dbe0 0x7fbc2c1552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.538+0000 7fbc33c00640 1 --2- 172.21.15.119:0/4057375824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc2c0cfad0 0x7fbc2c0af230 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.538+0000 7fbc33c00640 1 --2- 172.21.15.119:0/4057375824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc2c0aedc0 0x7fbc2c0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.539+0000 7fbc33c00640 1 -- 172.21.15.119:0/4057375824 >> 172.21.15.119:0/4057375824 conn(0x7fbc2c06aba0 msgr2=0x7fbc2c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.539+0000 7fbc33c00640 1 -- 172.21.15.119:0/4057375824 shutdown_connections 2024-04-25T10:08:46.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.539+0000 7fbc33c00640 1 -- 172.21.15.119:0/4057375824 wait complete. 2024-04-25T10:08:46.565 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:08:46.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.749+0000 7f4c0f200640 1 Processor -- start 2024-04-25T10:08:46.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.749+0000 7f4c0f200640 1 -- start start 2024-04-25T10:08:46.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.751+0000 7f4c0f200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4c08146410 0x7f4c080cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.751+0000 7f4c0f200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c080c9c40 con 0x7f4c08146f00 2024-04-25T10:08:46.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.751+0000 7f4c0f200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c080c9f20 con 0x7f4c080cb960 2024-04-25T10:08:46.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.751+0000 7f4c0f200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c080cab30 con 0x7f4c08146410 2024-04-25T10:08:46.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.751+0000 7f4c0e800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4c08146f00 0x7f4c0806d190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53832/0 (socket says 172.21.15.119:53832) 2024-04-25T10:08:46.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.751+0000 7f4c0e800640 1 -- 172.21.15.119:0/402926734 learned_addr learned my addr 172.21.15.119:0/402926734 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:46.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0de00640 1 --2- 172.21.15.119:0/402926734 >> v2:172.21.15.132:3300/0 conn(0x7f4c08146410 0x7f4c080cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 -- 172.21.15.119:0/402926734 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523562187 0 0) 0x7f4c080c9c40 con 0x7f4c08146f00 2024-04-25T10:08:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 -- 172.21.15.119:0/402926734 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bf0003520 con 0x7f4c08146f00 2024-04-25T10:08:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 -- 172.21.15.119:0/402926734 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3899042355 0 0) 0x7f4bf0003520 con 0x7f4c08146f00 2024-04-25T10:08:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 -- 172.21.15.119:0/402926734 >> v2:172.21.15.132:3300/0 conn(0x7f4c08146410 msgr2=0x7f4c080cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 --2- 172.21.15.119:0/402926734 >> v2:172.21.15.132:3300/0 conn(0x7f4c08146410 0x7f4c080cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 -- 172.21.15.119:0/402926734 >> v1:172.21.15.132:6789/0 conn(0x7f4c080cb960 legacy=0x7f4c081467f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 -- 172.21.15.119:0/402926734 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c080ca200 con 0x7f4c08146f00 2024-04-25T10:08:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 -- 172.21.15.119:0/402926734 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f4c040019f0 con 0x7f4c08146f00 2024-04-25T10:08:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 -- 172.21.15.119:0/402926734 >> v1:172.21.15.119:6789/0 conn(0x7f4c08146f00 legacy=0x7f4c0806d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 --2- 172.21.15.119:0/402926734 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf00039b0 0x7f4bf0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 --2- 172.21.15.119:0/402926734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 --2- 172.21.15.119:0/402926734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bf00051c0 0x7f4bf000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 -- 172.21.15.119:0/402926734 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080cab30 con 0x7f4bf00051c0 2024-04-25T10:08:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 -- 172.21.15.119:0/402926734 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf000c090 con 0x7f4bf0004800 2024-04-25T10:08:46.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.752+0000 7f4c0ca00640 1 -- 172.21.15.119:0/402926734 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf000c290 con 0x7f4bf00039b0 2024-04-25T10:08:46.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.753+0000 7f4c0e800640 1 --2- 172.21.15.119:0/402926734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bf00051c0 0x7f4bf000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.753+0000 7f4c0de00640 1 --2- 172.21.15.119:0/402926734 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf00039b0 0x7f4bf0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.753+0000 7f4c0d400640 1 --2- 172.21.15.119:0/402926734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.753+0000 7f4c0e800640 1 -- 172.21.15.119:0/402926734 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf00039b0 msgr2=0x7f4bf0003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.753+0000 7f4c0e800640 1 --2- 172.21.15.119:0/402926734 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bf00039b0 0x7f4bf0003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.753+0000 7f4c0e800640 1 -- 172.21.15.119:0/402926734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf0004800 msgr2=0x7f4bf0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.753+0000 7f4c0e800640 1 --2- 172.21.15.119:0/402926734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.753+0000 7f4c0e800640 1 -- 172.21.15.119:0/402926734 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf0003520 con 0x7f4bf00051c0 2024-04-25T10:08:46.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.754+0000 7f4c0e800640 1 -- 172.21.15.119:0/402926734 reap_dead start 2024-04-25T10:08:46.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.754+0000 7f4c0e800640 1 --2- 172.21.15.119:0/402926734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bf00051c0 0x7f4bf000b9c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400a840 tx=0x7f4bf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4de98eb70bcb2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.754+0000 7f4c0ca00640 1 -- 172.21.15.119:0/402926734 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0400a460 con 0x7f4bf00051c0 2024-04-25T10:08:46.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.754+0000 7f4c0ca00640 1 -- 172.21.15.119:0/402926734 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0402f070 con 0x7f4bf00051c0 2024-04-25T10:08:46.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.754+0000 7f4c0ca00640 1 -- 172.21.15.119:0/402926734 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0402bcb0 con 0x7f4bf00051c0 2024-04-25T10:08:46.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.755+0000 7f4c0f200640 1 -- 172.21.15.119:0/402926734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bf00051c0 msgr2=0x7f4bf000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.755+0000 7f4c0f200640 1 --2- 172.21.15.119:0/402926734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bf00051c0 0x7f4bf000b9c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400a840 tx=0x7f4bf0005830 comp rx=0 tx=0).stop 2024-04-25T10:08:46.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.755+0000 7f4c0f200640 1 -- 172.21.15.119:0/402926734 shutdown_connections 2024-04-25T10:08:46.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.755+0000 7f4c0f200640 1 --2- 172.21.15.119:0/402926734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bf00051c0 0x7f4bf000b9c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.755+0000 7f4c0f200640 1 -- 172.21.15.119:0/402926734 >> 172.21.15.119:0/402926734 conn(0x7f4c0806aba0 msgr2=0x7f4c0806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.755+0000 7f4c0f200640 1 -- 172.21.15.119:0/402926734 shutdown_connections 2024-04-25T10:08:46.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.755+0000 7f4c0f200640 1 -- 172.21.15.119:0/402926734 wait complete. 2024-04-25T10:08:46.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.757+0000 7f4c0f200640 1 Processor -- start 2024-04-25T10:08:46.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.757+0000 7f4c0f200640 1 -- start start 2024-04-25T10:08:46.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.758+0000 7f4c0f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf00051c0 0x7f4c0815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.758+0000 7f4c0f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c08138ac0 0x7f4c08138f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.758+0000 7f4c0f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c081f3aa0 0x7f4c081f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.758+0000 7f4c0f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080cafc0 con 0x7f4c081f3aa0 2024-04-25T10:08:46.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.758+0000 7f4c0f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080ca7c0 con 0x7f4bf00051c0 2024-04-25T10:08:46.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.758+0000 7f4c0f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c08147350 con 0x7f4c08138ac0 2024-04-25T10:08:46.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.758+0000 7f4c0e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c081f3aa0 0x7f4c081f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.758+0000 7f4c0e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c081f3aa0 0x7f4c081f3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51626/0 (socket says 172.21.15.119:51626) 2024-04-25T10:08:46.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.758+0000 7f4c0e800640 1 -- 172.21.15.119:0/1904894963 learned_addr learned my addr 172.21.15.119:0/1904894963 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:46.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.759+0000 7f4c0e800640 1 -- 172.21.15.119:0/1904894963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c08138ac0 msgr2=0x7f4c08138f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.759+0000 7f4c0e800640 1 --2- 172.21.15.119:0/1904894963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c08138ac0 0x7f4c08138f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.759+0000 7f4c0de00640 1 --2- 172.21.15.119:0/1904894963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf00051c0 0x7f4c0815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.759+0000 7f4c0e800640 1 -- 172.21.15.119:0/1904894963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf00051c0 msgr2=0x7f4c0815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.759+0000 7f4c0e800640 1 --2- 172.21.15.119:0/1904894963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf00051c0 0x7f4c0815ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.759+0000 7f4c0e800640 1 -- 172.21.15.119:0/1904894963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0813b0f0 con 0x7f4c081f3aa0 2024-04-25T10:08:46.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.759+0000 7f4c0e800640 1 --2- 172.21.15.119:0/1904894963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c081f3aa0 0x7f4c081f3f30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4c040012e0 tx=0x7f4c080c9e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.759+0000 7f4c02000640 1 -- 172.21.15.119:0/1904894963 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0400a460 con 0x7f4c081f3aa0 2024-04-25T10:08:46.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.759+0000 7f4c02000640 1 -- 172.21.15.119:0/1904894963 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c04034670 con 0x7f4c081f3aa0 2024-04-25T10:08:46.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.759+0000 7f4c02000640 1 -- 172.21.15.119:0/1904894963 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04001fb0 con 0x7f4c081f3aa0 2024-04-25T10:08:46.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.759+0000 7f4c0f200640 1 -- 172.21.15.119:0/1904894963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0813b3d0 con 0x7f4c081f3aa0 2024-04-25T10:08:46.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.759+0000 7f4c0f200640 1 -- 172.21.15.119:0/1904894963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0813b990 con 0x7f4c081f3aa0 2024-04-25T10:08:46.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.761+0000 7f4c0f200640 1 -- 172.21.15.119:0/1904894963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c08139480 con 0x7f4c081f3aa0 2024-04-25T10:08:46.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.762+0000 7f4c02000640 1 -- 172.21.15.119:0/1904894963 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c04034a20 con 0x7f4c081f3aa0 2024-04-25T10:08:46.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.762+0000 7f4c02000640 1 --2- 172.21.15.119:0/1904894963 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bec06b500 0x7f4bec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.763+0000 7f4c02000640 1 -- 172.21.15.119:0/1904894963 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f4c040aa320 con 0x7f4c081f3aa0 2024-04-25T10:08:46.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.763+0000 7f4c0de00640 1 --2- 172.21.15.119:0/1904894963 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bec06b500 0x7f4bec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.763+0000 7f4c0de00640 1 --2- 172.21.15.119:0/1904894963 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bec06b500 0x7f4bec06d9c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c08145260 tx=0x7f4bfc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.766+0000 7f4c02000640 1 -- 172.21.15.119:0/1904894963 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c08139480 con 0x7f4c081f3aa0 2024-04-25T10:08:46.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:46.960+0000 7f4c0f200640 1 -- 172.21.15.119:0/1904894963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4c080749b0 con 0x7f4c081f3aa0 2024-04-25T10:08:47.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.546+0000 7f4c02000640 1 -- 172.21.15.119:0/1904894963 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4c04046060 con 0x7f4c081f3aa0 2024-04-25T10:08:47.554 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2024-04-25T10:08:47.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.551+0000 7f4c0f200640 1 -- 172.21.15.119:0/1904894963 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bec06b500 msgr2=0x7f4bec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.551+0000 7f4c0f200640 1 --2- 172.21.15.119:0/1904894963 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bec06b500 0x7f4bec06d9c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c08145260 tx=0x7f4bfc008040 comp rx=0 tx=0).stop 2024-04-25T10:08:47.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.551+0000 7f4c0f200640 1 -- 172.21.15.119:0/1904894963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c081f3aa0 msgr2=0x7f4c081f3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.551+0000 7f4c0f200640 1 --2- 172.21.15.119:0/1904894963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c081f3aa0 0x7f4c081f3f30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4c040012e0 tx=0x7f4c080c9e10 comp rx=0 tx=0).stop 2024-04-25T10:08:47.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.552+0000 7f4c0f200640 1 -- 172.21.15.119:0/1904894963 shutdown_connections 2024-04-25T10:08:47.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.552+0000 7f4c0f200640 1 --2- 172.21.15.119:0/1904894963 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bec06b500 0x7f4bec06d9c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.552+0000 7f4c0f200640 1 --2- 172.21.15.119:0/1904894963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c081f3aa0 0x7f4c081f3f30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.552+0000 7f4c0f200640 1 --2- 172.21.15.119:0/1904894963 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4c08138ac0 0x7f4c08138f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.552+0000 7f4c0f200640 1 --2- 172.21.15.119:0/1904894963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf00051c0 0x7f4c0815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.552+0000 7f4c0f200640 1 -- 172.21.15.119:0/1904894963 >> 172.21.15.119:0/1904894963 conn(0x7f4c0806aba0 msgr2=0x7f4c080ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:47.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.553+0000 7f4c0f200640 1 -- 172.21.15.119:0/1904894963 shutdown_connections 2024-04-25T10:08:47.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.553+0000 7f4c0f200640 1 -- 172.21.15.119:0/1904894963 wait complete. 2024-04-25T10:08:47.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:08:47.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.772+0000 7fe045800640 1 Processor -- start 2024-04-25T10:08:47.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.772+0000 7fe045800640 1 -- start start 2024-04-25T10:08:47.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.773+0000 7fe045800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe040071830 0x7fe04015b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.774+0000 7fe045800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04013d9a0 con 0x7fe04013dce0 2024-04-25T10:08:47.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.774+0000 7fe045800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04015e650 con 0x7fe04013d5c0 2024-04-25T10:08:47.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.774+0000 7fe045800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04015abf0 con 0x7fe040071830 2024-04-25T10:08:47.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.774+0000 7fe037400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe04013dce0 0x7fe040071120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53838/0 (socket says 172.21.15.119:53838) 2024-04-25T10:08:47.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.774+0000 7fe037400640 1 -- 172.21.15.119:0/521662882 learned_addr learned my addr 172.21.15.119:0/521662882 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:47.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.774+0000 7fe044e00640 1 --2- 172.21.15.119:0/521662882 >> v2:172.21.15.119:3300/0 conn(0x7fe040071830 0x7fe04015b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.774+0000 7fe03f400640 1 -- 172.21.15.119:0/521662882 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680023977 0 0) 0x7fe04013d9a0 con 0x7fe04013dce0 2024-04-25T10:08:47.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.774+0000 7fe03f400640 1 -- 172.21.15.119:0/521662882 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe024003520 con 0x7fe04013dce0 2024-04-25T10:08:47.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.775+0000 7fe044e00640 1 -- 172.21.15.119:0/521662882 >> v1:172.21.15.132:6789/0 conn(0x7fe04013d5c0 legacy=0x7fe0401407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:47.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.775+0000 7fe044e00640 1 -- 172.21.15.119:0/521662882 >> v1:172.21.15.119:6789/0 conn(0x7fe04013dce0 legacy=0x7fe040071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.775+0000 7fe044e00640 1 -- 172.21.15.119:0/521662882 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04015e930 con 0x7fe040071830 2024-04-25T10:08:47.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.776+0000 7fe044e00640 1 --2- 172.21.15.119:0/521662882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe040071830 0x7fe04015b090 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe038009d90 tx=0x7fe03802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588f6b9bfd875088 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:47.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.776+0000 7fe03f400640 1 -- 172.21.15.119:0/521662882 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038007740 con 0x7fe040071830 2024-04-25T10:08:47.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.776+0000 7fe03f400640 1 -- 172.21.15.119:0/521662882 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0380078e0 con 0x7fe040071830 2024-04-25T10:08:47.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.776+0000 7fe03f400640 1 -- 172.21.15.119:0/521662882 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038007cb0 con 0x7fe040071830 2024-04-25T10:08:47.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.777+0000 7fe045800640 1 -- 172.21.15.119:0/521662882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe040071830 msgr2=0x7fe04015b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.777+0000 7fe045800640 1 --2- 172.21.15.119:0/521662882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe040071830 0x7fe04015b090 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe038009d90 tx=0x7fe03802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.777+0000 7fe045800640 1 -- 172.21.15.119:0/521662882 shutdown_connections 2024-04-25T10:08:47.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.777+0000 7fe045800640 1 --2- 172.21.15.119:0/521662882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe040071830 0x7fe04015b090 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.777+0000 7fe045800640 1 -- 172.21.15.119:0/521662882 >> 172.21.15.119:0/521662882 conn(0x7fe04006aba0 msgr2=0x7fe04006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:47.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.777+0000 7fe045800640 1 -- 172.21.15.119:0/521662882 shutdown_connections 2024-04-25T10:08:47.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.777+0000 7fe045800640 1 -- 172.21.15.119:0/521662882 wait complete. 2024-04-25T10:08:47.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.778+0000 7fe045800640 1 Processor -- start 2024-04-25T10:08:47.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.779+0000 7fe045800640 1 -- start start 2024-04-25T10:08:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.779+0000 7fe045800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe040071830 0x7fe04015c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.779+0000 7fe045800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe04013d5c0 0x7fe040159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.780+0000 7fe045800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe04013dce0 0x7fe0401369e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.780+0000 7fe045800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0400b1980 con 0x7fe04013dce0 2024-04-25T10:08:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.780+0000 7fe045800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04015eef0 con 0x7fe040071830 2024-04-25T10:08:47.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.780+0000 7fe045800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04013e150 con 0x7fe04013d5c0 2024-04-25T10:08:47.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.780+0000 7fe044e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe04013dce0 0x7fe0401369e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.780+0000 7fe044e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe04013dce0 0x7fe0401369e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51644/0 (socket says 172.21.15.119:51644) 2024-04-25T10:08:47.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.780+0000 7fe044e00640 1 -- 172.21.15.119:0/2594010811 learned_addr learned my addr 172.21.15.119:0/2594010811 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:47.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.780+0000 7fe03fe00640 1 --2- 172.21.15.119:0/2594010811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe040071830 0x7fe04015c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.780+0000 7fe037400640 1 --2- 172.21.15.119:0/2594010811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe04013d5c0 0x7fe040159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.780+0000 7fe044e00640 1 -- 172.21.15.119:0/2594010811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe04013d5c0 msgr2=0x7fe040159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.780+0000 7fe044e00640 1 --2- 172.21.15.119:0/2594010811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe04013d5c0 0x7fe040159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.780+0000 7fe044e00640 1 -- 172.21.15.119:0/2594010811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe040071830 msgr2=0x7fe04015c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.780+0000 7fe044e00640 1 --2- 172.21.15.119:0/2594010811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe040071830 0x7fe04015c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.780+0000 7fe044e00640 1 -- 172.21.15.119:0/2594010811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe040154260 con 0x7fe04013dce0 2024-04-25T10:08:47.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.781+0000 7fe044e00640 1 --2- 172.21.15.119:0/2594010811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe04013dce0 0x7fe0401369e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe038009d90 tx=0x7fe038004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.781+0000 7fe03cc00640 1 -- 172.21.15.119:0/2594010811 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03803f070 con 0x7fe04013dce0 2024-04-25T10:08:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.781+0000 7fe03cc00640 1 -- 172.21.15.119:0/2594010811 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe038031bc0 con 0x7fe04013dce0 2024-04-25T10:08:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.781+0000 7fe03cc00640 1 -- 172.21.15.119:0/2594010811 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038009450 con 0x7fe04013dce0 2024-04-25T10:08:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.781+0000 7fe045800640 1 -- 172.21.15.119:0/2594010811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe040154540 con 0x7fe04013dce0 2024-04-25T10:08:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.781+0000 7fe045800640 1 -- 172.21.15.119:0/2594010811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe040154b00 con 0x7fe04013dce0 2024-04-25T10:08:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.782+0000 7fe045800640 1 -- 172.21.15.119:0/2594010811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04013ac60 con 0x7fe04013dce0 2024-04-25T10:08:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.783+0000 7fe03cc00640 1 -- 172.21.15.119:0/2594010811 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe03803b410 con 0x7fe04013dce0 2024-04-25T10:08:47.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.784+0000 7fe03cc00640 1 --2- 172.21.15.119:0/2594010811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe01c06b4d0 0x7fe01c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.784+0000 7fe03cc00640 1 -- 172.21.15.119:0/2594010811 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe0380aafe0 con 0x7fe04013dce0 2024-04-25T10:08:47.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.784+0000 7fe03fe00640 1 --2- 172.21.15.119:0/2594010811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe01c06b4d0 0x7fe01c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.785+0000 7fe03fe00640 1 --2- 172.21.15.119:0/2594010811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe01c06b4d0 0x7fe01c06d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe030003d90 tx=0x7fe030009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:47.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:47.789+0000 7fe03cc00640 1 -- 172.21.15.119:0/2594010811 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe03807a0e0 con 0x7fe04013dce0 2024-04-25T10:08:48.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.004+0000 7fe045800640 1 -- 172.21.15.119:0/2594010811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe0400749b0 con 0x7fe04013dce0 2024-04-25T10:08:48.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.010+0000 7fe03cc00640 1 -- 172.21.15.119:0/2594010811 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe0380799e0 con 0x7fe04013dce0 2024-04-25T10:08:48.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.013+0000 7fe045800640 1 -- 172.21.15.119:0/2594010811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe01c06b4d0 msgr2=0x7fe01c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.013+0000 7fe045800640 1 --2- 172.21.15.119:0/2594010811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe01c06b4d0 0x7fe01c06d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe030003d90 tx=0x7fe030009040 comp rx=0 tx=0).stop 2024-04-25T10:08:48.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.013+0000 7fe045800640 1 -- 172.21.15.119:0/2594010811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe04013dce0 msgr2=0x7fe0401369e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.013+0000 7fe045800640 1 --2- 172.21.15.119:0/2594010811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe04013dce0 0x7fe0401369e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe038009d90 tx=0x7fe038004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:48.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.013+0000 7fe045800640 1 -- 172.21.15.119:0/2594010811 shutdown_connections 2024-04-25T10:08:48.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.013+0000 7fe045800640 1 --2- 172.21.15.119:0/2594010811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe01c06b4d0 0x7fe01c06d990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.013+0000 7fe045800640 1 --2- 172.21.15.119:0/2594010811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe04013dce0 0x7fe0401369e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.013+0000 7fe045800640 1 --2- 172.21.15.119:0/2594010811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe04013d5c0 0x7fe040159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.013+0000 7fe045800640 1 --2- 172.21.15.119:0/2594010811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe040071830 0x7fe04015c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.013+0000 7fe045800640 1 -- 172.21.15.119:0/2594010811 >> 172.21.15.119:0/2594010811 conn(0x7fe04006aba0 msgr2=0x7fe040137690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:48.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.014+0000 7fe045800640 1 -- 172.21.15.119:0/2594010811 shutdown_connections 2024-04-25T10:08:48.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.014+0000 7fe045800640 1 -- 172.21.15.119:0/2594010811 wait complete. 2024-04-25T10:08:48.039 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T10:08:48.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.227+0000 7f60d1400640 1 Processor -- start 2024-04-25T10:08:48.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.227+0000 7f60d1400640 1 -- start start 2024-04-25T10:08:48.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.228+0000 7f60d1400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f60cc15a870 0x7f60cc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.229+0000 7f60d1400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60cc0cfc40 0x7f60cc15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.229+0000 7f60d1400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60cc0cdb10 con 0x7f60cc06e2e0 2024-04-25T10:08:48.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.229+0000 7f60d1400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60cc0ce720 con 0x7f60cc15a870 2024-04-25T10:08:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.229+0000 7f60d1400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc0cddf0 con 0x7f60cc0cfc40 2024-04-25T10:08:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.229+0000 7f60ca800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f60cc15a870 0x7f60cc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.229+0000 7f60c3e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f60cc06e2e0 0x7f60cc0cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53850/0 (socket says 172.21.15.119:53850) 2024-04-25T10:08:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.229+0000 7f60ca800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f60cc15a870 0x7f60cc0cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51660/0 (socket says 172.21.15.119:51660) 2024-04-25T10:08:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.229+0000 7f60c3e00640 1 -- 172.21.15.119:0/665335724 learned_addr learned my addr 172.21.15.119:0/665335724 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.229+0000 7f60ca800640 1 -- 172.21.15.119:0/665335724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60cc0cfc40 msgr2=0x7f60cc15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.229+0000 7f60ca800640 1 --2- 172.21.15.119:0/665335724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60cc0cfc40 0x7f60cc15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.229+0000 7f60ca800640 1 -- 172.21.15.119:0/665335724 >> v1:172.21.15.119:6789/0 conn(0x7f60cc06e2e0 legacy=0x7f60cc0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.229+0000 7f60ca800640 1 -- 172.21.15.119:0/665335724 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60cc0ce0d0 con 0x7f60cc15a870 2024-04-25T10:08:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.229+0000 7f60c9e00640 1 -- 172.21.15.119:0/665335724 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961294968 0 0) 0x7f60cc0cdb10 con 0x7f60cc06e2e0 2024-04-25T10:08:48.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.230+0000 7f60ca800640 1 --2- 172.21.15.119:0/665335724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60cc15a870 0x7f60cc0cebb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0097a0 tx=0x7f60bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854dd4cd5c66be89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.231+0000 7f60c9e00640 1 -- 172.21.15.119:0/665335724 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc037070 con 0x7f60cc15a870 2024-04-25T10:08:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.231+0000 7f60c9e00640 1 -- 172.21.15.119:0/665335724 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60bc032440 con 0x7f60cc15a870 2024-04-25T10:08:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.231+0000 7f60c9e00640 1 -- 172.21.15.119:0/665335724 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc032810 con 0x7f60cc15a870 2024-04-25T10:08:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.231+0000 7f60d1400640 1 -- 172.21.15.119:0/665335724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60cc15a870 msgr2=0x7f60cc0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.231+0000 7f60d1400640 1 --2- 172.21.15.119:0/665335724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60cc15a870 0x7f60cc0cebb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0097a0 tx=0x7f60bc02a180 comp rx=0 tx=0).stop 2024-04-25T10:08:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.232+0000 7f60d1400640 1 -- 172.21.15.119:0/665335724 shutdown_connections 2024-04-25T10:08:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.232+0000 7f60d1400640 1 --2- 172.21.15.119:0/665335724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60cc0cfc40 0x7f60cc15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.232+0000 7f60d1400640 1 --2- 172.21.15.119:0/665335724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60cc15a870 0x7f60cc0cebb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.232+0000 7f60d1400640 1 -- 172.21.15.119:0/665335724 >> 172.21.15.119:0/665335724 conn(0x7f60cc06aba0 msgr2=0x7f60cc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.232+0000 7f60d1400640 1 -- 172.21.15.119:0/665335724 shutdown_connections 2024-04-25T10:08:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.232+0000 7f60d1400640 1 -- 172.21.15.119:0/665335724 wait complete. 2024-04-25T10:08:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.233+0000 7f60d1400640 1 Processor -- start 2024-04-25T10:08:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.233+0000 7f60d1400640 1 -- start start 2024-04-25T10:08:48.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.234+0000 7f60d1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60cc06e2e0 0x7f60cc0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.234+0000 7f60d1400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60cc0cfc40 0x7f60cc1511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.234+0000 7f60d1400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60cc15a870 0x7f60cc071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.234+0000 7f60d1400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc0cd700 con 0x7f60cc06e2e0 2024-04-25T10:08:48.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.234+0000 7f60d1400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc0cd580 con 0x7f60cc15a870 2024-04-25T10:08:48.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.234+0000 7f60d1400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc0cd880 con 0x7f60cc0cfc40 2024-04-25T10:08:48.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.235+0000 7f60ca800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60cc06e2e0 0x7f60cc0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.235+0000 7f60ca800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60cc06e2e0 0x7f60cc0adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51668/0 (socket says 172.21.15.119:51668) 2024-04-25T10:08:48.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.235+0000 7f60ca800640 1 -- 172.21.15.119:0/1170018849 learned_addr learned my addr 172.21.15.119:0/1170018849 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:48.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.235+0000 7f60ca800640 1 -- 172.21.15.119:0/1170018849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60cc0cfc40 msgr2=0x7f60cc1511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.235+0000 7f60ca800640 1 --2- 172.21.15.119:0/1170018849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60cc0cfc40 0x7f60cc1511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.235+0000 7f60ca800640 1 -- 172.21.15.119:0/1170018849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60cc15a870 msgr2=0x7f60cc071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.235+0000 7f60ca800640 1 --2- 172.21.15.119:0/1170018849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60cc15a870 0x7f60cc071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.235+0000 7f60ca800640 1 -- 172.21.15.119:0/1170018849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60cc151720 con 0x7f60cc06e2e0 2024-04-25T10:08:48.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.235+0000 7f60ca800640 1 --2- 172.21.15.119:0/1170018849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60cc06e2e0 0x7f60cc0adfb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0098c0 tx=0x7f60bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:48.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.236+0000 7f60c2a00640 1 -- 172.21.15.119:0/1170018849 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc041070 con 0x7f60cc06e2e0 2024-04-25T10:08:48.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.236+0000 7f60c2a00640 1 -- 172.21.15.119:0/1170018849 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60bc03c870 con 0x7f60cc06e2e0 2024-04-25T10:08:48.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.236+0000 7f60c2a00640 1 -- 172.21.15.119:0/1170018849 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc037040 con 0x7f60cc06e2e0 2024-04-25T10:08:48.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.236+0000 7f60d1400640 1 -- 172.21.15.119:0/1170018849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60cc0c4d50 con 0x7f60cc06e2e0 2024-04-25T10:08:48.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.236+0000 7f60d1400640 1 -- 172.21.15.119:0/1170018849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60cc0c55f0 con 0x7f60cc06e2e0 2024-04-25T10:08:48.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.237+0000 7f60d1400640 1 -- 172.21.15.119:0/1170018849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60cc0749b0 con 0x7f60cc06e2e0 2024-04-25T10:08:48.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.238+0000 7f60c2a00640 1 -- 172.21.15.119:0/1170018849 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60bc044070 con 0x7f60cc06e2e0 2024-04-25T10:08:48.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.239+0000 7f60c2a00640 1 --2- 172.21.15.119:0/1170018849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f60a806b4d0 0x7f60a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.239+0000 7f60c2a00640 1 -- 172.21.15.119:0/1170018849 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f60bc0aac00 con 0x7f60cc06e2e0 2024-04-25T10:08:48.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.239+0000 7f60c3e00640 1 --2- 172.21.15.119:0/1170018849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f60a806b4d0 0x7f60a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.240+0000 7f60c3e00640 1 --2- 172.21.15.119:0/1170018849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f60a806b4d0 0x7f60a806d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f60b4007fd0 tx=0x7f60b4007960 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:48.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.243+0000 7f60c2a00640 1 -- 172.21.15.119:0/1170018849 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60bc079e00 con 0x7f60cc06e2e0 2024-04-25T10:08:48.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.517+0000 7f60d1400640 1 -- 172.21.15.119:0/1170018849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f60cc0cbb40 con 0x7f60cc06e2e0 2024-04-25T10:08:48.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.559+0000 7f60c2a00640 1 -- 172.21.15.119:0/1170018849 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v118) v1 ==== 142+0+0 (secure 0 0 0) 0x7f60bc079700 con 0x7f60cc06e2e0 2024-04-25T10:08:48.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.562+0000 7f60d1400640 1 -- 172.21.15.119:0/1170018849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f60a806b4d0 msgr2=0x7f60a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.562+0000 7f60d1400640 1 --2- 172.21.15.119:0/1170018849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f60a806b4d0 0x7f60a806d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f60b4007fd0 tx=0x7f60b4007960 comp rx=0 tx=0).stop 2024-04-25T10:08:48.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.562+0000 7f60d1400640 1 -- 172.21.15.119:0/1170018849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60cc06e2e0 msgr2=0x7f60cc0adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.562+0000 7f60d1400640 1 --2- 172.21.15.119:0/1170018849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60cc06e2e0 0x7f60cc0adfb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0098c0 tx=0x7f60bc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.562+0000 7f60d1400640 1 -- 172.21.15.119:0/1170018849 shutdown_connections 2024-04-25T10:08:48.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.562+0000 7f60d1400640 1 --2- 172.21.15.119:0/1170018849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f60a806b4d0 0x7f60a806d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.562+0000 7f60d1400640 1 --2- 172.21.15.119:0/1170018849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60cc15a870 0x7f60cc071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.562+0000 7f60d1400640 1 --2- 172.21.15.119:0/1170018849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f60cc0cfc40 0x7f60cc1511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.562+0000 7f60d1400640 1 --2- 172.21.15.119:0/1170018849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60cc06e2e0 0x7f60cc0adfb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.562+0000 7f60d1400640 1 -- 172.21.15.119:0/1170018849 >> 172.21.15.119:0/1170018849 conn(0x7f60cc06aba0 msgr2=0x7f60cc159a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:48.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.563+0000 7f60d1400640 1 -- 172.21.15.119:0/1170018849 shutdown_connections 2024-04-25T10:08:48.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.563+0000 7f60d1400640 1 -- 172.21.15.119:0/1170018849 wait complete. 2024-04-25T10:08:48.588 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:08:48.588 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-25T10:08:48.588 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:08:48.589 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.781+0000 7faf35200640 1 Processor -- start 2024-04-25T10:08:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.781+0000 7faf35200640 1 -- start start 2024-04-25T10:08:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.783+0000 7faf35200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf3015db60 0x7faf300d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.783+0000 7faf35200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faf300d2fa0 0x7faf300d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.783+0000 7faf35200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3015ef10 con 0x7faf300b9270 2024-04-25T10:08:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.783+0000 7faf35200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3015e390 con 0x7faf300d2fa0 2024-04-25T10:08:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.783+0000 7faf35200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3015e670 con 0x7faf3015db60 2024-04-25T10:08:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.784+0000 7faf2fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faf300d2fa0 0x7faf300d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.784+0000 7faf2f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf3015db60 0x7faf300d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.784+0000 7faf2fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faf300d2fa0 0x7faf300d3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:50956/0 (socket says 172.21.15.119:50956) 2024-04-25T10:08:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.784+0000 7faf2f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf3015db60 0x7faf300d3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:42880/0 (socket says 172.21.15.119:42880) 2024-04-25T10:08:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.784+0000 7faf2fe00640 1 -- 172.21.15.119:0/332122265 learned_addr learned my addr 172.21.15.119:0/332122265 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.784+0000 7faf27e00640 1 --1- 172.21.15.119:0/332122265 >> v1:172.21.15.132:6789/0 conn(0x7faf300b9270 0x7faf300b5360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:42378/0 (socket says 172.21.15.119:42378) 2024-04-25T10:08:48.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.785+0000 7faf2ea00640 1 -- 172.21.15.119:0/332122265 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877671746 0 0) 0x7faf3015ef10 con 0x7faf300b9270 2024-04-25T10:08:48.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.785+0000 7faf2ea00640 1 -- 172.21.15.119:0/332122265 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf14003520 con 0x7faf300b9270 2024-04-25T10:08:48.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.785+0000 7faf2f400640 1 -- 172.21.15.119:0/332122265 >> v2:172.21.15.132:3300/0 conn(0x7faf300d2fa0 msgr2=0x7faf300d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:48.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.785+0000 7faf2f400640 1 --2- 172.21.15.119:0/332122265 >> v2:172.21.15.132:3300/0 conn(0x7faf300d2fa0 0x7faf300d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.785+0000 7faf2f400640 1 -- 172.21.15.119:0/332122265 >> v1:172.21.15.132:6789/0 conn(0x7faf300b9270 legacy=0x7faf300b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.785+0000 7faf2f400640 1 -- 172.21.15.119:0/332122265 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3015e950 con 0x7faf3015db60 2024-04-25T10:08:48.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.786+0000 7faf2f400640 1 --2- 172.21.15.119:0/332122265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf3015db60 0x7faf300d3870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faf200097a0 tx=0x7faf2002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b7ac58ca4955422 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:48.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.786+0000 7faf2ea00640 1 -- 172.21.15.119:0/332122265 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20037070 con 0x7faf3015db60 2024-04-25T10:08:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.787+0000 7faf2ea00640 1 -- 172.21.15.119:0/332122265 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf20032440 con 0x7faf3015db60 2024-04-25T10:08:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.787+0000 7faf2ea00640 1 -- 172.21.15.119:0/332122265 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20032810 con 0x7faf3015db60 2024-04-25T10:08:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.787+0000 7faf35200640 1 -- 172.21.15.119:0/332122265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf3015db60 msgr2=0x7faf300d3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.787+0000 7faf35200640 1 --2- 172.21.15.119:0/332122265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf3015db60 0x7faf300d3870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faf200097a0 tx=0x7faf2002a180 comp rx=0 tx=0).stop 2024-04-25T10:08:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.787+0000 7faf35200640 1 -- 172.21.15.119:0/332122265 shutdown_connections 2024-04-25T10:08:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.787+0000 7faf35200640 1 --2- 172.21.15.119:0/332122265 >> v2:172.21.15.132:3300/0 conn(0x7faf300d2fa0 0x7faf300d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.787+0000 7faf35200640 1 --2- 172.21.15.119:0/332122265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf3015db60 0x7faf300d3870 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.787+0000 7faf35200640 1 -- 172.21.15.119:0/332122265 >> 172.21.15.119:0/332122265 conn(0x7faf300726c0 msgr2=0x7faf30072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.787+0000 7faf35200640 1 -- 172.21.15.119:0/332122265 shutdown_connections 2024-04-25T10:08:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.787+0000 7faf35200640 1 -- 172.21.15.119:0/332122265 wait complete. 2024-04-25T10:08:48.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.789+0000 7faf35200640 1 Processor -- start 2024-04-25T10:08:48.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.789+0000 7faf35200640 1 -- start start 2024-04-25T10:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.790+0000 7faf35200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf300b9270 0x7faf30159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.790+0000 7faf35200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf300d2fa0 0x7faf30154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.790+0000 7faf35200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3015db60 0x7faf300cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.790+0000 7faf35200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf300b3120 con 0x7faf3015db60 2024-04-25T10:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.790+0000 7faf35200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf300b2fa0 con 0x7faf300d2fa0 2024-04-25T10:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.790+0000 7faf35200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3015df40 con 0x7faf300b9270 2024-04-25T10:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.790+0000 7faf2fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3015db60 0x7faf300cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.790+0000 7faf2fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3015db60 0x7faf300cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51682/0 (socket says 172.21.15.119:51682) 2024-04-25T10:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.790+0000 7faf2fe00640 1 -- 172.21.15.119:0/2307160115 learned_addr learned my addr 172.21.15.119:0/2307160115 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.791+0000 7faf27e00640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf300d2fa0 0x7faf30154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.791+0000 7faf2f400640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf300b9270 0x7faf30159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.791+0000 7faf2fe00640 1 -- 172.21.15.119:0/2307160115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf300b9270 msgr2=0x7faf30159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.791+0000 7faf2fe00640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf300b9270 0x7faf30159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.791+0000 7faf2fe00640 1 -- 172.21.15.119:0/2307160115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf300d2fa0 msgr2=0x7faf30154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.791+0000 7faf2fe00640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf300d2fa0 0x7faf30154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.791+0000 7faf2fe00640 1 -- 172.21.15.119:0/2307160115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf30155260 con 0x7faf3015db60 2024-04-25T10:08:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.791+0000 7faf2f400640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf300b9270 0x7faf30159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.791+0000 7faf2fe00640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3015db60 0x7faf300cb7a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7faf2800efc0 tx=0x7faf2800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.791+0000 7faf27400640 1 -- 172.21.15.119:0/2307160115 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2800cd20 con 0x7faf3015db60 2024-04-25T10:08:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.791+0000 7faf27400640 1 -- 172.21.15.119:0/2307160115 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf28003990 con 0x7faf3015db60 2024-04-25T10:08:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.791+0000 7faf35200640 1 -- 172.21.15.119:0/2307160115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf30159910 con 0x7faf3015db60 2024-04-25T10:08:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.792+0000 7faf27400640 1 -- 172.21.15.119:0/2307160115 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28003d80 con 0x7faf3015db60 2024-04-25T10:08:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.792+0000 7faf35200640 1 -- 172.21.15.119:0/2307160115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf30159ed0 con 0x7faf3015db60 2024-04-25T10:08:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.793+0000 7faf35200640 1 -- 172.21.15.119:0/2307160115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf300749b0 con 0x7faf3015db60 2024-04-25T10:08:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.794+0000 7faf27400640 1 -- 172.21.15.119:0/2307160115 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf28005d10 con 0x7faf3015db60 2024-04-25T10:08:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.795+0000 7faf27400640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf0c06b4d0 0x7faf0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.795+0000 7faf27400640 1 -- 172.21.15.119:0/2307160115 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7faf2808a520 con 0x7faf3015db60 2024-04-25T10:08:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.795+0000 7faf2f400640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf0c06b4d0 0x7faf0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.796+0000 7faf2f400640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf0c06b4d0 0x7faf0c06d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faf20034000 tx=0x7faf2002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:48.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.800+0000 7faf27400640 1 -- 172.21.15.119:0/2307160115 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faf28022020 con 0x7faf3015db60 2024-04-25T10:08:49.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.997+0000 7faf35200640 1 -- 172.21.15.119:0/2307160115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf300d1a60 con 0x7faf3015db60 2024-04-25T10:08:49.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:48.999+0000 7faf27400640 1 -- 172.21.15.119:0/2307160115 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+13058 (secure 0 0 0) 0x7faf28010040 con 0x7faf3015db60 2024-04-25T10:08:49.005 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:49.005 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":36,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:47.536437+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6836/3585315292":"2024-04-26T10:08:44.518755+0000","172.21.15.119:6834/3585315292":"2024-04-26T10:08:44.518755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:49.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.002+0000 7faf35200640 1 -- 172.21.15.119:0/2307160115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf0c06b4d0 msgr2=0x7faf0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.002+0000 7faf35200640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf0c06b4d0 0x7faf0c06d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faf20034000 tx=0x7faf2002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:49.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.002+0000 7faf35200640 1 -- 172.21.15.119:0/2307160115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3015db60 msgr2=0x7faf300cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.002+0000 7faf35200640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3015db60 0x7faf300cb7a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7faf2800efc0 tx=0x7faf2800c490 comp rx=0 tx=0).stop 2024-04-25T10:08:49.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.002+0000 7faf35200640 1 -- 172.21.15.119:0/2307160115 shutdown_connections 2024-04-25T10:08:49.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.002+0000 7faf35200640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf0c06b4d0 0x7faf0c06d990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.002+0000 7faf35200640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf3015db60 0x7faf300cb7a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.002+0000 7faf35200640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf300d2fa0 0x7faf30154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.002+0000 7faf35200640 1 --2- 172.21.15.119:0/2307160115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf300b9270 0x7faf30159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.002+0000 7faf35200640 1 -- 172.21.15.119:0/2307160115 >> 172.21.15.119:0/2307160115 conn(0x7faf300726c0 msgr2=0x7faf300c82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:49.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.002+0000 7faf35200640 1 -- 172.21.15.119:0/2307160115 shutdown_connections 2024-04-25T10:08:49.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.002+0000 7faf35200640 1 -- 172.21.15.119:0/2307160115 wait complete. 2024-04-25T10:08:49.027 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:49.028 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:49.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.224+0000 7f9183000640 1 Processor -- start 2024-04-25T10:08:49.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.224+0000 7f9183000640 1 -- start start 2024-04-25T10:08:49.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.225+0000 7f9183000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f917c15dca0 0x7f917c0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.226+0000 7f9183000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c0d3b90 0x7f917c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.226+0000 7f9183000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f917c0cee80 0x7f917c0d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.226+0000 7f9183000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f917c0b6b20 con 0x7f917c15dca0 2024-04-25T10:08:49.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.226+0000 7f9183000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f917c0d4090 con 0x7f917c0cee80 2024-04-25T10:08:49.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.226+0000 7f9183000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0d4370 con 0x7f917c0d3b90 2024-04-25T10:08:49.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.226+0000 7f917be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f917c15dca0 0x7f917c0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.226+0000 7f917be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f917c15dca0 0x7f917c0ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51684/0 (socket says 172.21.15.119:51684) 2024-04-25T10:08:49.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.226+0000 7f917be00640 1 -- 172.21.15.119:0/1461992911 learned_addr learned my addr 172.21.15.119:0/1461992911 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:49.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.226+0000 7f917be00640 1 -- 172.21.15.119:0/1461992911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c0d3b90 msgr2=0x7f917c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:49.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.226+0000 7f917be00640 1 --2- 172.21.15.119:0/1461992911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c0d3b90 0x7f917c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.226+0000 7f917be00640 1 -- 172.21.15.119:0/1461992911 >> v2:172.21.15.132:3300/0 conn(0x7f917c0cee80 msgr2=0x7f917c0d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:49.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.226+0000 7f917be00640 1 --2- 172.21.15.119:0/1461992911 >> v2:172.21.15.132:3300/0 conn(0x7f917c0cee80 0x7f917c0d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.226+0000 7f917be00640 1 -- 172.21.15.119:0/1461992911 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c0d4650 con 0x7f917c15dca0 2024-04-25T10:08:49.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.227+0000 7f917be00640 1 --2- 172.21.15.119:0/1461992911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f917c15dca0 0x7f917c0ce940 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f917c0d04c0 tx=0x7f916c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec0bc1a83f8505b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:49.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.228+0000 7f917aa00640 1 -- 172.21.15.119:0/1461992911 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c036070 con 0x7f917c15dca0 2024-04-25T10:08:49.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.228+0000 7f917aa00640 1 -- 172.21.15.119:0/1461992911 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c031600 con 0x7f917c15dca0 2024-04-25T10:08:49.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.228+0000 7f917aa00640 1 -- 172.21.15.119:0/1461992911 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c0319d0 con 0x7f917c15dca0 2024-04-25T10:08:49.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.228+0000 7f9183000640 1 -- 172.21.15.119:0/1461992911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f917c15dca0 msgr2=0x7f917c0ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.228+0000 7f9183000640 1 --2- 172.21.15.119:0/1461992911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f917c15dca0 0x7f917c0ce940 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f917c0d04c0 tx=0x7f916c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.228+0000 7f9183000640 1 -- 172.21.15.119:0/1461992911 shutdown_connections 2024-04-25T10:08:49.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.228+0000 7f9183000640 1 --2- 172.21.15.119:0/1461992911 >> v2:172.21.15.132:3300/0 conn(0x7f917c0cee80 0x7f917c0d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.228+0000 7f9183000640 1 --2- 172.21.15.119:0/1461992911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c0d3b90 0x7f917c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.228+0000 7f9183000640 1 --2- 172.21.15.119:0/1461992911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f917c15dca0 0x7f917c0ce940 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.228+0000 7f9183000640 1 -- 172.21.15.119:0/1461992911 >> 172.21.15.119:0/1461992911 conn(0x7f917c0726c0 msgr2=0x7f917c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:49.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.229+0000 7f9183000640 1 -- 172.21.15.119:0/1461992911 shutdown_connections 2024-04-25T10:08:49.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.229+0000 7f9183000640 1 -- 172.21.15.119:0/1461992911 wait complete. 2024-04-25T10:08:49.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.230+0000 7f9183000640 1 Processor -- start 2024-04-25T10:08:49.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.230+0000 7f9183000640 1 -- start start 2024-04-25T10:08:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.231+0000 7f9183000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f917c0cee80 0x7f917c1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.231+0000 7f9183000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f917c0d3b90 0x7f917c15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.231+0000 7f9183000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c15dca0 0x7f917c0cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.231+0000 7f9183000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c15e200 con 0x7f917c0d3b90 2024-04-25T10:08:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.231+0000 7f9183000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c15e080 con 0x7f917c0cee80 2024-04-25T10:08:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.231+0000 7f9183000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0b2500 con 0x7f917c15dca0 2024-04-25T10:08:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.232+0000 7f917b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f917c0d3b90 0x7f917c15aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.232+0000 7f917b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f917c0d3b90 0x7f917c15aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51700/0 (socket says 172.21.15.119:51700) 2024-04-25T10:08:49.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.232+0000 7f917b400640 1 -- 172.21.15.119:0/3881371008 learned_addr learned my addr 172.21.15.119:0/3881371008 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.232+0000 7f917be00640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f917c0cee80 0x7f917c1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.232+0000 7f917b400640 1 -- 172.21.15.119:0/3881371008 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c15dca0 msgr2=0x7f917c0cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.232+0000 7f9181000640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c15dca0 0x7f917c0cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.232+0000 7f917b400640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c15dca0 0x7f917c0cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.232+0000 7f917b400640 1 -- 172.21.15.119:0/3881371008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f917c0cee80 msgr2=0x7f917c1550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.232+0000 7f917b400640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f917c0cee80 0x7f917c1550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.232+0000 7f917b400640 1 -- 172.21.15.119:0/3881371008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c0cc320 con 0x7f917c0d3b90 2024-04-25T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.232+0000 7f917be00640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f917c0cee80 0x7f917c1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.232+0000 7f917b400640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f917c0d3b90 0x7f917c15aab0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f916400b450 tx=0x7f916400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.233+0000 7f9173e00640 1 -- 172.21.15.119:0/3881371008 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164015020 con 0x7f917c0d3b90 2024-04-25T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.233+0000 7f9173e00640 1 -- 172.21.15.119:0/3881371008 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9164011890 con 0x7f917c0d3b90 2024-04-25T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.233+0000 7f9173e00640 1 -- 172.21.15.119:0/3881371008 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164011c80 con 0x7f917c0d3b90 2024-04-25T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.233+0000 7f9183000640 1 -- 172.21.15.119:0/3881371008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917c0cc8e0 con 0x7f917c0d3b90 2024-04-25T10:08:49.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.233+0000 7f9183000640 1 -- 172.21.15.119:0/3881371008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917c0ccea0 con 0x7f917c0d3b90 2024-04-25T10:08:49.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.234+0000 7f9183000640 1 -- 172.21.15.119:0/3881371008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f917c0750b0 con 0x7f917c0d3b90 2024-04-25T10:08:49.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.235+0000 7f9173e00640 1 -- 172.21.15.119:0/3881371008 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9164018020 con 0x7f917c0d3b90 2024-04-25T10:08:49.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.236+0000 7f9173e00640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f915806b4d0 0x7f915806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.236+0000 7f9173e00640 1 -- 172.21.15.119:0/3881371008 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f916408a100 con 0x7f917c0d3b90 2024-04-25T10:08:49.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.236+0000 7f917be00640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f915806b4d0 0x7f915806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.237+0000 7f917be00640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f915806b4d0 0x7f915806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f916c033000 tx=0x7f916c009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:49.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.240+0000 7f9173e00640 1 -- 172.21.15.119:0/3881371008 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f916401f020 con 0x7f917c0d3b90 2024-04-25T10:08:49.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.432+0000 7f9183000640 1 -- 172.21.15.119:0/3881371008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f917c0d33f0 con 0x7f917c0d3b90 2024-04-25T10:08:49.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.433+0000 7f9173e00640 1 -- 172.21.15.119:0/3881371008 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f916400d040 con 0x7f917c0d3b90 2024-04-25T10:08:49.439 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:08:49.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.436+0000 7f9183000640 1 -- 172.21.15.119:0/3881371008 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f915806b4d0 msgr2=0x7f915806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.436+0000 7f9183000640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f915806b4d0 0x7f915806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f916c033000 tx=0x7f916c009000 comp rx=0 tx=0).stop 2024-04-25T10:08:49.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.436+0000 7f9183000640 1 -- 172.21.15.119:0/3881371008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f917c0d3b90 msgr2=0x7f917c15aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.436+0000 7f9183000640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f917c0d3b90 0x7f917c15aab0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f916400b450 tx=0x7f916400b910 comp rx=0 tx=0).stop 2024-04-25T10:08:49.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.436+0000 7f9183000640 1 -- 172.21.15.119:0/3881371008 shutdown_connections 2024-04-25T10:08:49.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.436+0000 7f9183000640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f915806b4d0 0x7f915806d990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.436+0000 7f9183000640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f917c15dca0 0x7f917c0cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.436+0000 7f9183000640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f917c0d3b90 0x7f917c15aab0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.436+0000 7f9183000640 1 --2- 172.21.15.119:0/3881371008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f917c0cee80 0x7f917c1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.436+0000 7f9183000640 1 -- 172.21.15.119:0/3881371008 >> 172.21.15.119:0/3881371008 conn(0x7f917c0726c0 msgr2=0x7f917c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:49.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.436+0000 7f9183000640 1 -- 172.21.15.119:0/3881371008 shutdown_connections 2024-04-25T10:08:49.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.437+0000 7f9183000640 1 -- 172.21.15.119:0/3881371008 wait complete. 2024-04-25T10:08:49.463 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T10:08:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.662+0000 7fbd8f400640 1 Processor -- start 2024-04-25T10:08:49.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.662+0000 7fbd8f400640 1 -- start start 2024-04-25T10:08:49.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.663+0000 7fbd8f400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbd8806e8f0 0x7fbd8815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.663+0000 7fbd8f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd8815c820 0x7fbd881654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.663+0000 7fbd8f400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd880ce6c0 con 0x7fbd8815bf00 2024-04-25T10:08:49.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.663+0000 7fbd8f400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd881455e0 con 0x7fbd8806e8f0 2024-04-25T10:08:49.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.663+0000 7fbd8f400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8806d450 con 0x7fbd8815c820 2024-04-25T10:08:49.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.664+0000 7fbd8d400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd8815c820 0x7fbd881654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.664+0000 7fbd87e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbd8806e8f0 0x7fbd8815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.664+0000 7fbd8d400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd8815c820 0x7fbd881654b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:42902/0 (socket says 172.21.15.119:42902) 2024-04-25T10:08:49.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.664+0000 7fbd8d400640 1 -- 172.21.15.119:0/75459278 learned_addr learned my addr 172.21.15.119:0/75459278 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:49.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.664+0000 7fbd87e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbd8806e8f0 0x7fbd8815c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:51004/0 (socket says 172.21.15.119:51004) 2024-04-25T10:08:49.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.665+0000 7fbd8d400640 1 -- 172.21.15.119:0/75459278 >> v2:172.21.15.132:3300/0 conn(0x7fbd8806e8f0 msgr2=0x7fbd8815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:49.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.665+0000 7fbd8d400640 1 --2- 172.21.15.119:0/75459278 >> v2:172.21.15.132:3300/0 conn(0x7fbd8806e8f0 0x7fbd8815c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.665+0000 7fbd8d400640 1 -- 172.21.15.119:0/75459278 >> v1:172.21.15.132:6789/0 conn(0x7fbd8815bf00 legacy=0x7fbd880cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.665+0000 7fbd8d400640 1 -- 172.21.15.119:0/75459278 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd880cc4e0 con 0x7fbd8815c820 2024-04-25T10:08:49.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.666+0000 7fbd8d400640 1 --2- 172.21.15.119:0/75459278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd8815c820 0x7fbd881654b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009e50 tx=0x7fbd8002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59015f94ee5041f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:49.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.667+0000 7fbd87400640 1 -- 172.21.15.119:0/75459278 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80036070 con 0x7fbd8815c820 2024-04-25T10:08:49.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.667+0000 7fbd87400640 1 -- 172.21.15.119:0/75459278 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd80031600 con 0x7fbd8815c820 2024-04-25T10:08:49.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.667+0000 7fbd87400640 1 -- 172.21.15.119:0/75459278 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd800319d0 con 0x7fbd8815c820 2024-04-25T10:08:49.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.668+0000 7fbd8f400640 1 -- 172.21.15.119:0/75459278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd8815c820 msgr2=0x7fbd881654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.668+0000 7fbd8f400640 1 --2- 172.21.15.119:0/75459278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd8815c820 0x7fbd881654b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009e50 tx=0x7fbd8002a710 comp rx=0 tx=0).stop 2024-04-25T10:08:49.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.668+0000 7fbd8f400640 1 -- 172.21.15.119:0/75459278 shutdown_connections 2024-04-25T10:08:49.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.668+0000 7fbd8f400640 1 --2- 172.21.15.119:0/75459278 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd8815c820 0x7fbd881654b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.668+0000 7fbd8f400640 1 --2- 172.21.15.119:0/75459278 >> v2:172.21.15.132:3300/0 conn(0x7fbd8806e8f0 0x7fbd8815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.668+0000 7fbd8f400640 1 -- 172.21.15.119:0/75459278 >> 172.21.15.119:0/75459278 conn(0x7fbd8806aba0 msgr2=0x7fbd8806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:49.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.668+0000 7fbd8f400640 1 -- 172.21.15.119:0/75459278 shutdown_connections 2024-04-25T10:08:49.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.668+0000 7fbd8f400640 1 -- 172.21.15.119:0/75459278 wait complete. 2024-04-25T10:08:49.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.670+0000 7fbd8f400640 1 Processor -- start 2024-04-25T10:08:49.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.671+0000 7fbd8f400640 1 -- start start 2024-04-25T10:08:49.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.672+0000 7fbd8f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd8806e8f0 0x7fbd881f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.672+0000 7fbd8f400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd8815bf00 0x7fbd88138b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.672+0000 7fbd8f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8815c820 0x7fbd88139090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.672+0000 7fbd8f400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd880ccd70 con 0x7fbd8815c820 2024-04-25T10:08:49.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.672+0000 7fbd8f400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd880cca90 con 0x7fbd8815bf00 2024-04-25T10:08:49.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.672+0000 7fbd8f400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd880cd050 con 0x7fbd8806e8f0 2024-04-25T10:08:49.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.672+0000 7fbd8d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8815c820 0x7fbd88139090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.672+0000 7fbd8d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8815c820 0x7fbd88139090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51712/0 (socket says 172.21.15.119:51712) 2024-04-25T10:08:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.672+0000 7fbd8d400640 1 -- 172.21.15.119:0/2472189171 learned_addr learned my addr 172.21.15.119:0/2472189171 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.672+0000 7fbd8ca00640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd8806e8f0 0x7fbd881f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.673+0000 7fbd87e00640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd8815bf00 0x7fbd88138b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.673+0000 7fbd8d400640 1 -- 172.21.15.119:0/2472189171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd8806e8f0 msgr2=0x7fbd881f64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.673+0000 7fbd8d400640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd8806e8f0 0x7fbd881f64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.673+0000 7fbd8d400640 1 -- 172.21.15.119:0/2472189171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd8815bf00 msgr2=0x7fbd88138b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.673+0000 7fbd8d400640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd8815bf00 0x7fbd88138b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.673+0000 7fbd8d400640 1 -- 172.21.15.119:0/2472189171 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd880c93f0 con 0x7fbd8815c820 2024-04-25T10:08:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.673+0000 7fbd8ca00640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd8806e8f0 0x7fbd881f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.673+0000 7fbd8d400640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8815c820 0x7fbd88139090 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbd80034040 tx=0x7fbd80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.674+0000 7fbd84c00640 1 -- 172.21.15.119:0/2472189171 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8003f020 con 0x7fbd8815c820 2024-04-25T10:08:49.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.674+0000 7fbd84c00640 1 -- 172.21.15.119:0/2472189171 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd80037040 con 0x7fbd8815c820 2024-04-25T10:08:49.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.674+0000 7fbd84c00640 1 -- 172.21.15.119:0/2472189171 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80036070 con 0x7fbd8815c820 2024-04-25T10:08:49.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.674+0000 7fbd8f400640 1 -- 172.21.15.119:0/2472189171 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd880c96d0 con 0x7fbd8815c820 2024-04-25T10:08:49.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.674+0000 7fbd8f400640 1 -- 172.21.15.119:0/2472189171 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd880c9c90 con 0x7fbd8815c820 2024-04-25T10:08:49.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.675+0000 7fbd8f400640 1 -- 172.21.15.119:0/2472189171 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd880749b0 con 0x7fbd8815c820 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.676+0000 7fbd84c00640 1 -- 172.21.15.119:0/2472189171 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd80042020 con 0x7fbd8815c820 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.677+0000 7fbd84c00640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd6806b4d0 0x7fbd6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.677+0000 7fbd84c00640 1 -- 172.21.15.119:0/2472189171 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbd800aa3c0 con 0x7fbd8815c820 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.677+0000 7fbd8ca00640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd6806b4d0 0x7fbd6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.678+0000 7fbd8ca00640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd6806b4d0 0x7fbd6806d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbd78007fd0 tx=0x7fbd78009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:49.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.681+0000 7fbd84c00640 1 -- 172.21.15.119:0/2472189171 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd80049030 con 0x7fbd8815c820 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:49.948+0000 7fbd8f400640 1 -- 172.21.15.119:0/2472189171 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fbd8806c170 con 0x7fbd8815c820 2024-04-25T10:08:50.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.572+0000 7fbd84c00640 1 -- 172.21.15.119:0/2472189171 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v120) v1 ==== 152+0+0 (secure 0 0 0) 0x7fbd80078e50 con 0x7fbd8815c820 2024-04-25T10:08:50.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.576+0000 7fbd8f400640 1 -- 172.21.15.119:0/2472189171 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd6806b4d0 msgr2=0x7fbd6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.576+0000 7fbd8f400640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd6806b4d0 0x7fbd6806d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbd78007fd0 tx=0x7fbd78009040 comp rx=0 tx=0).stop 2024-04-25T10:08:50.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.576+0000 7fbd8f400640 1 -- 172.21.15.119:0/2472189171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8815c820 msgr2=0x7fbd88139090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.576+0000 7fbd8f400640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8815c820 0x7fbd88139090 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbd80034040 tx=0x7fbd80004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:50.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.577+0000 7fbd8f400640 1 -- 172.21.15.119:0/2472189171 shutdown_connections 2024-04-25T10:08:50.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.577+0000 7fbd8f400640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd6806b4d0 0x7fbd6806d990 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.577+0000 7fbd8f400640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8815c820 0x7fbd88139090 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.577+0000 7fbd8f400640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd8815bf00 0x7fbd88138b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.577+0000 7fbd8f400640 1 --2- 172.21.15.119:0/2472189171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd8806e8f0 0x7fbd881f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.577+0000 7fbd8f400640 1 -- 172.21.15.119:0/2472189171 >> 172.21.15.119:0/2472189171 conn(0x7fbd8806aba0 msgr2=0x7fbd8815d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:50.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.577+0000 7fbd8f400640 1 -- 172.21.15.119:0/2472189171 shutdown_connections 2024-04-25T10:08:50.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.577+0000 7fbd8f400640 1 -- 172.21.15.119:0/2472189171 wait complete. 2024-04-25T10:08:50.604 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:08:50.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.802+0000 7fa983e00640 1 Processor -- start 2024-04-25T10:08:50.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.803+0000 7fa983e00640 1 -- start start 2024-04-25T10:08:50.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.804+0000 7fa983e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa98406b2b0 0x7fa9840acb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.804+0000 7fa983e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa9840b2f90 0x7fa9840b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.804+0000 7fa983e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa98415e120 con 0x7fa9840ada20 2024-04-25T10:08:50.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.804+0000 7fa983e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98415dec0 con 0x7fa98406b2b0 2024-04-25T10:08:50.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.804+0000 7fa983e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98415e400 con 0x7fa9840b2f90 2024-04-25T10:08:50.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.804+0000 7fa97a000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa98406b2b0 0x7fa9840acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.805+0000 7fa97a000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa98406b2b0 0x7fa9840acb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51714/0 (socket says 172.21.15.119:51714) 2024-04-25T10:08:50.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.805+0000 7fa97a000640 1 -- 172.21.15.119:0/1480718130 learned_addr learned my addr 172.21.15.119:0/1480718130 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:50.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.805+0000 7fa983400640 1 --2- 172.21.15.119:0/1480718130 >> v2:172.21.15.132:3300/0 conn(0x7fa9840b2f90 0x7fa9840b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.805+0000 7fa97a000640 1 -- 172.21.15.119:0/1480718130 >> v2:172.21.15.132:3300/0 conn(0x7fa9840b2f90 msgr2=0x7fa9840b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:50.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.805+0000 7fa97a000640 1 --2- 172.21.15.119:0/1480718130 >> v2:172.21.15.132:3300/0 conn(0x7fa9840b2f90 0x7fa9840b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.805+0000 7fa97a000640 1 -- 172.21.15.119:0/1480718130 >> v1:172.21.15.132:6789/0 conn(0x7fa9840ada20 legacy=0x7fa98406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:50.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.805+0000 7fa97a000640 1 -- 172.21.15.119:0/1480718130 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98415e6e0 con 0x7fa98406b2b0 2024-04-25T10:08:50.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.805+0000 7fa983400640 1 --2- 172.21.15.119:0/1480718130 >> v2:172.21.15.132:3300/0 conn(0x7fa9840b2f90 0x7fa9840b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:50.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.806+0000 7fa97a000640 1 --2- 172.21.15.119:0/1480718130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa98406b2b0 0x7fa9840acb20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa98406d770 tx=0x7fa96c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e094034ca9ed12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.806+0000 7fa982000640 1 -- 172.21.15.119:0/1480718130 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c037070 con 0x7fa98406b2b0 2024-04-25T10:08:50.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.807+0000 7fa982000640 1 -- 172.21.15.119:0/1480718130 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96c032440 con 0x7fa98406b2b0 2024-04-25T10:08:50.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.807+0000 7fa982000640 1 -- 172.21.15.119:0/1480718130 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c032810 con 0x7fa98406b2b0 2024-04-25T10:08:50.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.807+0000 7fa983e00640 1 -- 172.21.15.119:0/1480718130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa98406b2b0 msgr2=0x7fa9840acb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.807+0000 7fa983e00640 1 --2- 172.21.15.119:0/1480718130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa98406b2b0 0x7fa9840acb20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa98406d770 tx=0x7fa96c02a230 comp rx=0 tx=0).stop 2024-04-25T10:08:50.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.807+0000 7fa983e00640 1 -- 172.21.15.119:0/1480718130 shutdown_connections 2024-04-25T10:08:50.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.807+0000 7fa983e00640 1 --2- 172.21.15.119:0/1480718130 >> v2:172.21.15.132:3300/0 conn(0x7fa9840b2f90 0x7fa9840b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.807+0000 7fa983e00640 1 --2- 172.21.15.119:0/1480718130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa98406b2b0 0x7fa9840acb20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.807+0000 7fa983e00640 1 -- 172.21.15.119:0/1480718130 >> 172.21.15.119:0/1480718130 conn(0x7fa984066ff0 msgr2=0x7fa9840b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:50.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.807+0000 7fa983e00640 1 -- 172.21.15.119:0/1480718130 shutdown_connections 2024-04-25T10:08:50.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.807+0000 7fa983e00640 1 -- 172.21.15.119:0/1480718130 wait complete. 2024-04-25T10:08:50.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.809+0000 7fa983e00640 1 Processor -- start 2024-04-25T10:08:50.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.809+0000 7fa983e00640 1 -- start start 2024-04-25T10:08:50.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.810+0000 7fa983e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa98406b2b0 0x7fa984144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.810+0000 7fa983e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9840ada20 0x7fa9841f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.810+0000 7fa983e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9840b2f90 0x7fa984160100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.810+0000 7fa983e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9840b38e0 con 0x7fa9840ada20 2024-04-25T10:08:50.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.810+0000 7fa983e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98415eca0 con 0x7fa98406b2b0 2024-04-25T10:08:50.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.810+0000 7fa983e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9840ade00 con 0x7fa9840b2f90 2024-04-25T10:08:50.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.811+0000 7fa97a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9840ada20 0x7fa9841f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.811+0000 7fa97a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9840ada20 0x7fa9841f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51724/0 (socket says 172.21.15.119:51724) 2024-04-25T10:08:50.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.811+0000 7fa97a000640 1 -- 172.21.15.119:0/402581151 learned_addr learned my addr 172.21.15.119:0/402581151 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:50.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.811+0000 7fa97a000640 1 -- 172.21.15.119:0/402581151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9840b2f90 msgr2=0x7fa984160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.811+0000 7fa97a000640 1 --2- 172.21.15.119:0/402581151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9840b2f90 0x7fa984160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.811+0000 7fa97a000640 1 -- 172.21.15.119:0/402581151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa98406b2b0 msgr2=0x7fa984144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.811+0000 7fa97a000640 1 --2- 172.21.15.119:0/402581151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa98406b2b0 0x7fa984144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.811+0000 7fa97a000640 1 -- 172.21.15.119:0/402581151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9841448e0 con 0x7fa9840ada20 2024-04-25T10:08:50.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.811+0000 7fa97a000640 1 --2- 172.21.15.119:0/402581151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9840ada20 0x7fa9841f2fb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa96c000c00 tx=0x7fa96c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.812+0000 7fa97be00640 1 -- 172.21.15.119:0/402581151 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c037040 con 0x7fa9840ada20 2024-04-25T10:08:50.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.812+0000 7fa97be00640 1 -- 172.21.15.119:0/402581151 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96c042070 con 0x7fa9840ada20 2024-04-25T10:08:50.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.812+0000 7fa983e00640 1 -- 172.21.15.119:0/402581151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa984144bc0 con 0x7fa9840ada20 2024-04-25T10:08:50.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.812+0000 7fa97be00640 1 -- 172.21.15.119:0/402581151 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c030bb0 con 0x7fa9840ada20 2024-04-25T10:08:50.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.812+0000 7fa983e00640 1 -- 172.21.15.119:0/402581151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa984145180 con 0x7fa9840ada20 2024-04-25T10:08:50.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.813+0000 7fa983e00640 1 -- 172.21.15.119:0/402581151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9840749b0 con 0x7fa9840ada20 2024-04-25T10:08:50.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.814+0000 7fa97be00640 1 -- 172.21.15.119:0/402581151 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa96c047460 con 0x7fa9840ada20 2024-04-25T10:08:50.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.815+0000 7fa97be00640 1 --2- 172.21.15.119:0/402581151 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa96006b4d0 0x7fa96006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.815+0000 7fa97be00640 1 -- 172.21.15.119:0/402581151 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa96c0aac80 con 0x7fa9840ada20 2024-04-25T10:08:50.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.816+0000 7fa982a00640 1 --2- 172.21.15.119:0/402581151 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa96006b4d0 0x7fa96006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.816+0000 7fa982a00640 1 --2- 172.21.15.119:0/402581151 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa96006b4d0 0x7fa96006d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9740043c0 tx=0x7fa974007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:50.820+0000 7fa97be00640 1 -- 172.21.15.119:0/402581151 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa96c045360 con 0x7fa9840ada20 2024-04-25T10:08:51.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.018+0000 7fa983e00640 1 -- 172.21.15.119:0/402581151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa98406d7c0 con 0x7fa9840ada20 2024-04-25T10:08:51.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.025+0000 7fa97be00640 1 -- 172.21.15.119:0/402581151 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa96c079cd0 con 0x7fa9840ada20 2024-04-25T10:08:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.027+0000 7fa983e00640 1 -- 172.21.15.119:0/402581151 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa96006b4d0 msgr2=0x7fa96006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.027+0000 7fa983e00640 1 --2- 172.21.15.119:0/402581151 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa96006b4d0 0x7fa96006d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9740043c0 tx=0x7fa974007d30 comp rx=0 tx=0).stop 2024-04-25T10:08:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.027+0000 7fa983e00640 1 -- 172.21.15.119:0/402581151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9840ada20 msgr2=0x7fa9841f2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.027+0000 7fa983e00640 1 --2- 172.21.15.119:0/402581151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9840ada20 0x7fa9841f2fb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa96c000c00 tx=0x7fa96c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.028+0000 7fa983e00640 1 -- 172.21.15.119:0/402581151 shutdown_connections 2024-04-25T10:08:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.028+0000 7fa983e00640 1 --2- 172.21.15.119:0/402581151 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa96006b4d0 0x7fa96006d990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.028+0000 7fa983e00640 1 --2- 172.21.15.119:0/402581151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9840b2f90 0x7fa984160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.028+0000 7fa983e00640 1 --2- 172.21.15.119:0/402581151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9840ada20 0x7fa9841f2fb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.028+0000 7fa983e00640 1 --2- 172.21.15.119:0/402581151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa98406b2b0 0x7fa984144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.028+0000 7fa983e00640 1 -- 172.21.15.119:0/402581151 >> 172.21.15.119:0/402581151 conn(0x7fa984066ff0 msgr2=0x7fa98406aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.028+0000 7fa983e00640 1 -- 172.21.15.119:0/402581151 shutdown_connections 2024-04-25T10:08:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.028+0000 7fa983e00640 1 -- 172.21.15.119:0/402581151 wait complete. 2024-04-25T10:08:51.052 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:08:51.053 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:51.162 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:51.276 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:08:51.276 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:08:51.276 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:51.334 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:51.335 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:08:51.335 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:51.392 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:51.392 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:08:51.392 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:51.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:51.449 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:08:51.449 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:51.507 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:51.507 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:08:51.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:51.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.708+0000 7fc784200640 1 Processor -- start 2024-04-25T10:08:51.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.708+0000 7fc784200640 1 -- start start 2024-04-25T10:08:51.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.710+0000 7fc784200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc77c15a870 0x7fc77c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.710+0000 7fc784200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc77c06e2e0 0x7fc77c15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.710+0000 7fc784200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77c0cdb10 con 0x7fc77c0cd580 2024-04-25T10:08:51.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.710+0000 7fc784200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77c0ce720 con 0x7fc77c06e2e0 2024-04-25T10:08:51.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.710+0000 7fc784200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0cddf0 con 0x7fc77c15a870 2024-04-25T10:08:51.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.710+0000 7fc782200640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc77c0cd580 0x7fc77c0cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45700/0 (socket says 172.21.15.119:45700) 2024-04-25T10:08:51.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.710+0000 7fc782200640 1 -- 172.21.15.119:0/2858804185 learned_addr learned my addr 172.21.15.119:0/2858804185 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:51.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.710+0000 7fc781800640 1 --2- 172.21.15.119:0/2858804185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc77c15a870 0x7fc77c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.710+0000 7fc77be00640 1 -- 172.21.15.119:0/2858804185 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3979691539 0 0) 0x7fc77c0cdb10 con 0x7fc77c0cd580 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 -- 172.21.15.119:0/2858804185 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc764003520 con 0x7fc77c0cd580 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc780e00640 1 --2- 172.21.15.119:0/2858804185 >> v2:172.21.15.132:3300/0 conn(0x7fc77c06e2e0 0x7fc77c15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 -- 172.21.15.119:0/2858804185 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 441328000 0 0) 0x7fc764003520 con 0x7fc77c0cd580 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 -- 172.21.15.119:0/2858804185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc77c15a870 msgr2=0x7fc77c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 --2- 172.21.15.119:0/2858804185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc77c15a870 0x7fc77c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 -- 172.21.15.119:0/2858804185 >> v2:172.21.15.132:3300/0 conn(0x7fc77c06e2e0 msgr2=0x7fc77c15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 --2- 172.21.15.119:0/2858804185 >> v2:172.21.15.132:3300/0 conn(0x7fc77c06e2e0 0x7fc77c15ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 -- 172.21.15.119:0/2858804185 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c0ce0d0 con 0x7fc77c0cd580 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 -- 172.21.15.119:0/2858804185 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fc7740019f0 con 0x7fc77c0cd580 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 -- 172.21.15.119:0/2858804185 >> v1:172.21.15.119:6789/0 conn(0x7fc77c0cd580 legacy=0x7fc77c0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 --2- 172.21.15.119:0/2858804185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc77c15a870 0x7fc7640039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 --2- 172.21.15.119:0/2858804185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc764004030 0x7fc764004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 --2- 172.21.15.119:0/2858804185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc764004d40 0x7fc7640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 -- 172.21.15.119:0/2858804185 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0ce720 con 0x7fc764004030 2024-04-25T10:08:51.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 -- 172.21.15.119:0/2858804185 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0cddf0 con 0x7fc764004d40 2024-04-25T10:08:51.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc77be00640 1 -- 172.21.15.119:0/2858804185 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76400bb20 con 0x7fc77c15a870 2024-04-25T10:08:51.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.711+0000 7fc780e00640 1 --2- 172.21.15.119:0/2858804185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc764004030 0x7fc764004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.712+0000 7fc781800640 1 --2- 172.21.15.119:0/2858804185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc77c15a870 0x7fc7640039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.712+0000 7fc780e00640 1 -- 172.21.15.119:0/2858804185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc77c15a870 msgr2=0x7fc7640039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:51.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.712+0000 7fc780e00640 1 --2- 172.21.15.119:0/2858804185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc77c15a870 0x7fc7640039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.712+0000 7fc780e00640 1 -- 172.21.15.119:0/2858804185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc764004d40 msgr2=0x7fc7640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:51.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.712+0000 7fc782200640 1 --2- 172.21.15.119:0/2858804185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc764004d40 0x7fc7640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.712+0000 7fc780e00640 1 --2- 172.21.15.119:0/2858804185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc764004d40 0x7fc7640051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.712+0000 7fc780e00640 1 -- 172.21.15.119:0/2858804185 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc764003520 con 0x7fc764004030 2024-04-25T10:08:51.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.712+0000 7fc781800640 1 --2- 172.21.15.119:0/2858804185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc77c15a870 0x7fc7640039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:51.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.713+0000 7fc780e00640 1 --2- 172.21.15.119:0/2858804185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc764004030 0x7fc764004800 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc76800ba30 tx=0x7fc764005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46acb57acecb7400 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:51.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.713+0000 7fc77be00640 1 -- 172.21.15.119:0/2858804185 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768030070 con 0x7fc764004030 2024-04-25T10:08:51.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.713+0000 7fc77be00640 1 -- 172.21.15.119:0/2858804185 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76802ae10 con 0x7fc764004030 2024-04-25T10:08:51.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.713+0000 7fc77be00640 1 -- 172.21.15.119:0/2858804185 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76800b600 con 0x7fc764004030 2024-04-25T10:08:51.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.714+0000 7fc784200640 1 -- 172.21.15.119:0/2858804185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc764004030 msgr2=0x7fc764004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.714+0000 7fc784200640 1 --2- 172.21.15.119:0/2858804185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc764004030 0x7fc764004800 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc76800ba30 tx=0x7fc764005830 comp rx=0 tx=0).stop 2024-04-25T10:08:51.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.714+0000 7fc782200640 1 -- 172.21.15.119:0/2858804185 reap_dead start 2024-04-25T10:08:51.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.714+0000 7fc784200640 1 -- 172.21.15.119:0/2858804185 shutdown_connections 2024-04-25T10:08:51.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.714+0000 7fc784200640 1 -- 172.21.15.119:0/2858804185 >> 172.21.15.119:0/2858804185 conn(0x7fc77c06aba0 msgr2=0x7fc77c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:51.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.714+0000 7fc784200640 1 -- 172.21.15.119:0/2858804185 shutdown_connections 2024-04-25T10:08:51.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.714+0000 7fc784200640 1 -- 172.21.15.119:0/2858804185 wait complete. 2024-04-25T10:08:51.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.716+0000 7fc784200640 1 Processor -- start 2024-04-25T10:08:51.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.716+0000 7fc784200640 1 -- start start 2024-04-25T10:08:51.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.717+0000 7fc784200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc764004030 0x7fc77c154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.717+0000 7fc784200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc77c146f90 0x7fc77c147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.717+0000 7fc784200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc77c0ad530 0x7fc77c0ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.717+0000 7fc784200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0cfdc0 con 0x7fc77c0ad530 2024-04-25T10:08:51.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.717+0000 7fc784200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0cfc40 con 0x7fc77c146f90 2024-04-25T10:08:51.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.717+0000 7fc784200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0cebb0 con 0x7fc764004030 2024-04-25T10:08:51.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.717+0000 7fc782200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc77c0ad530 0x7fc77c0ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.718+0000 7fc782200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc77c0ad530 0x7fc77c0ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38274/0 (socket says 172.21.15.119:38274) 2024-04-25T10:08:51.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.718+0000 7fc782200640 1 -- 172.21.15.119:0/2169381812 learned_addr learned my addr 172.21.15.119:0/2169381812 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:51.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.718+0000 7fc780e00640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc77c146f90 0x7fc77c147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.718+0000 7fc781800640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc764004030 0x7fc77c154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.718+0000 7fc782200640 1 -- 172.21.15.119:0/2169381812 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc764004030 msgr2=0x7fc77c154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.718+0000 7fc782200640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc764004030 0x7fc77c154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.718+0000 7fc782200640 1 -- 172.21.15.119:0/2169381812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc77c146f90 msgr2=0x7fc77c147410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.718+0000 7fc782200640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc77c146f90 0x7fc77c147410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.718+0000 7fc782200640 1 -- 172.21.15.119:0/2169381812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c147950 con 0x7fc77c0ad530 2024-04-25T10:08:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.718+0000 7fc781800640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc764004030 0x7fc77c154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.718+0000 7fc782200640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc77c0ad530 0x7fc77c0ad9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc774001370 tx=0x7fc77400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.719+0000 7fc779600640 1 -- 172.21.15.119:0/2169381812 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774004d60 con 0x7fc77c0ad530 2024-04-25T10:08:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.719+0000 7fc779600640 1 -- 172.21.15.119:0/2169381812 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77400bb80 con 0x7fc77c0ad530 2024-04-25T10:08:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.719+0000 7fc784200640 1 -- 172.21.15.119:0/2169381812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c0ae2a0 con 0x7fc77c0ad530 2024-04-25T10:08:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.719+0000 7fc779600640 1 -- 172.21.15.119:0/2169381812 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774007110 con 0x7fc77c0ad530 2024-04-25T10:08:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.719+0000 7fc784200640 1 -- 172.21.15.119:0/2169381812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c0ae860 con 0x7fc77c0ad530 2024-04-25T10:08:51.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.720+0000 7fc784200640 1 -- 172.21.15.119:0/2169381812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c0749b0 con 0x7fc77c0ad530 2024-04-25T10:08:51.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.721+0000 7fc779600640 1 -- 172.21.15.119:0/2169381812 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc77401b020 con 0x7fc77c0ad530 2024-04-25T10:08:51.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.722+0000 7fc779600640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc75c06b4d0 0x7fc75c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.722+0000 7fc779600640 1 -- 172.21.15.119:0/2169381812 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc77408a050 con 0x7fc77c0ad530 2024-04-25T10:08:51.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.722+0000 7fc781800640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc75c06b4d0 0x7fc75c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.723+0000 7fc781800640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc75c06b4d0 0x7fc75c06d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc77c15d8c0 tx=0x7fc770008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:51.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.726+0000 7fc779600640 1 -- 172.21.15.119:0/2169381812 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc774058c10 con 0x7fc77c0ad530 2024-04-25T10:08:51.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.958+0000 7fc784200640 1 -- 172.21.15.119:0/2169381812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc77c0b00a0 con 0x7fc77c0ad530 2024-04-25T10:08:51.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.959+0000 7fc779600640 1 -- 172.21.15.119:0/2169381812 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fc77405ccb0 con 0x7fc77c0ad530 2024-04-25T10:08:51.965 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:51.966 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1554372269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1554372269},{"type":"v1","addr":"172.21.15.119:6837","nonce":1554372269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4699,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4250304148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":4250304148},{"type":"v1","addr":"172.21.15.132:6834","nonce":4250304148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/637229574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":637229574},{"type":"v1","addr":"172.21.15.132:6835","nonce":637229574}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1766900703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1766900703},{"type":"v1","addr":"172.21.15.119:6836","nonce":1766900703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[]} 2024-04-25T10:08:51.966 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 18 2024-04-25T10:08:51.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.962+0000 7fc784200640 1 -- 172.21.15.119:0/2169381812 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc75c06b4d0 msgr2=0x7fc75c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.962+0000 7fc784200640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc75c06b4d0 0x7fc75c06d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc77c15d8c0 tx=0x7fc770008040 comp rx=0 tx=0).stop 2024-04-25T10:08:51.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.962+0000 7fc784200640 1 -- 172.21.15.119:0/2169381812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc77c0ad530 msgr2=0x7fc77c0ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.962+0000 7fc784200640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc77c0ad530 0x7fc77c0ad9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc774001370 tx=0x7fc77400b4a0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.962+0000 7fc784200640 1 -- 172.21.15.119:0/2169381812 shutdown_connections 2024-04-25T10:08:51.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.962+0000 7fc784200640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc75c06b4d0 0x7fc75c06d990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.962+0000 7fc784200640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc77c0ad530 0x7fc77c0ad9c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.962+0000 7fc784200640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc77c146f90 0x7fc77c147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.962+0000 7fc784200640 1 --2- 172.21.15.119:0/2169381812 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc764004030 0x7fc77c154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.962+0000 7fc784200640 1 -- 172.21.15.119:0/2169381812 >> 172.21.15.119:0/2169381812 conn(0x7fc77c06aba0 msgr2=0x7fc77c159aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.962+0000 7fc784200640 1 -- 172.21.15.119:0/2169381812 shutdown_connections 2024-04-25T10:08:51.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:51.962+0000 7fc784200640 1 -- 172.21.15.119:0/2169381812 wait complete. 2024-04-25T10:08:51.987 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:08:51.987 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:08:51.988 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:08:51.988 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:51.988 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:08:51.988 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:08:51.988 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:08:51.988 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:51.988 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:08:51.989 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:08:51.989 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:08:51.989 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:51.989 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:08:51.989 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:08:51.989 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:08:51.989 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:52.005 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:08:52.005 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:08:52.006 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:08:52.006 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:08:52.009 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:08:52.009 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:08:52.023 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:08:52.023 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:08:52.050 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:08:52.051 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:52.053 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:08:52.053 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:52.070 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:08:52.071 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:52.077 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:08:52.077 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:52.077 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:08:52.186 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2024-04-25T10:08:52.187 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:08:52.225 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-04-25T10:08:52.235 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-04-25T10:08:52.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.245+0000 7f7771000640 1 Processor -- start 2024-04-25T10:08:52.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.246+0000 7f7771000640 1 -- start start 2024-04-25T10:08:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.247+0000 7f7771000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f776c1610b0 0x7f776c0b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.247+0000 7f7771000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f776c1618d0 0x7f776c0ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.247+0000 7f7771000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f776c162040 con 0x7f776c0ced70 2024-04-25T10:08:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.247+0000 7f7771000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776c0cf1c0 con 0x7f776c1618d0 2024-04-25T10:08:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.247+0000 7f7771000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776c0d1ea0 con 0x7f776c1610b0 2024-04-25T10:08:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.247+0000 7f7769400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f776c1618d0 0x7f776c0ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.247+0000 7f7769400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f776c1618d0 0x7f776c0ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38308/0 (socket says 172.21.15.119:38308) 2024-04-25T10:08:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.247+0000 7f7769400640 1 -- 172.21.15.119:0/3634104438 learned_addr learned my addr 172.21.15.119:0/3634104438 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.247+0000 7f7769e00640 1 --2- 172.21.15.119:0/3634104438 >> v2:172.21.15.132:3300/0 conn(0x7f776c1610b0 0x7f776c0b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.247+0000 7f7769400640 1 -- 172.21.15.119:0/3634104438 >> v2:172.21.15.132:3300/0 conn(0x7f776c1610b0 msgr2=0x7f776c0b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.247+0000 7f7769400640 1 --2- 172.21.15.119:0/3634104438 >> v2:172.21.15.132:3300/0 conn(0x7f776c1610b0 0x7f776c0b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.247+0000 7f7769400640 1 -- 172.21.15.119:0/3634104438 >> v1:172.21.15.132:6789/0 conn(0x7f776c0ced70 legacy=0x7f776c06c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.247+0000 7f7769400640 1 -- 172.21.15.119:0/3634104438 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c162320 con 0x7f776c1618d0 2024-04-25T10:08:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.248+0000 7f7769400640 1 --2- 172.21.15.119:0/3634104438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f776c1618d0 0x7f776c0ce830 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f776c08c410 tx=0x7f775802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331b037b9897ebeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.249+0000 7f7768a00640 1 -- 172.21.15.119:0/3634104438 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758037070 con 0x7f776c1618d0 2024-04-25T10:08:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.249+0000 7f7768a00640 1 -- 172.21.15.119:0/3634104438 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7758032440 con 0x7f776c1618d0 2024-04-25T10:08:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.249+0000 7f7768a00640 1 -- 172.21.15.119:0/3634104438 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758032810 con 0x7f776c1618d0 2024-04-25T10:08:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.249+0000 7f7771000640 1 -- 172.21.15.119:0/3634104438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f776c1618d0 msgr2=0x7f776c0ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.249+0000 7f7771000640 1 --2- 172.21.15.119:0/3634104438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f776c1618d0 0x7f776c0ce830 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f776c08c410 tx=0x7f775802a200 comp rx=0 tx=0).stop 2024-04-25T10:08:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.249+0000 7f7771000640 1 -- 172.21.15.119:0/3634104438 shutdown_connections 2024-04-25T10:08:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.249+0000 7f7771000640 1 --2- 172.21.15.119:0/3634104438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f776c1618d0 0x7f776c0ce830 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.249+0000 7f7771000640 1 --2- 172.21.15.119:0/3634104438 >> v2:172.21.15.132:3300/0 conn(0x7f776c1610b0 0x7f776c0b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.249+0000 7f7771000640 1 -- 172.21.15.119:0/3634104438 >> 172.21.15.119:0/3634104438 conn(0x7f776c0b5570 msgr2=0x7f776c0b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.249+0000 7f7771000640 1 -- 172.21.15.119:0/3634104438 shutdown_connections 2024-04-25T10:08:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.249+0000 7f7771000640 1 -- 172.21.15.119:0/3634104438 wait complete. 2024-04-25T10:08:52.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.251+0000 7f7771000640 1 Processor -- start 2024-04-25T10:08:52.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.251+0000 7f7771000640 1 -- start start 2024-04-25T10:08:52.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.252+0000 7f7771000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f776c0ced70 0x7f776c14bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.252+0000 7f7771000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f776c1610b0 0x7f776c15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.252+0000 7f7771000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f776c1618d0 0x7f776c1601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.252+0000 7f7771000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c0b6c00 con 0x7f776c1618d0 2024-04-25T10:08:52.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.252+0000 7f7771000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c0d20f0 con 0x7f776c1610b0 2024-04-25T10:08:52.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.252+0000 7f7771000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c161d10 con 0x7f776c0ced70 2024-04-25T10:08:52.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.252+0000 7f776a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f776c1618d0 0x7f776c1601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.252+0000 7f776a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f776c1618d0 0x7f776c1601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38320/0 (socket says 172.21.15.119:38320) 2024-04-25T10:08:52.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.252+0000 7f776a800640 1 -- 172.21.15.119:0/3237024406 learned_addr learned my addr 172.21.15.119:0/3237024406 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:52.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.252+0000 7f7769e00640 1 --2- 172.21.15.119:0/3237024406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f776c0ced70 0x7f776c14bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.253+0000 7f7769400640 1 --2- 172.21.15.119:0/3237024406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f776c1610b0 0x7f776c15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.253+0000 7f776a800640 1 -- 172.21.15.119:0/3237024406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f776c0ced70 msgr2=0x7f776c14bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.253+0000 7f776a800640 1 --2- 172.21.15.119:0/3237024406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f776c0ced70 0x7f776c14bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.253+0000 7f776a800640 1 -- 172.21.15.119:0/3237024406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f776c1610b0 msgr2=0x7f776c15fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.253+0000 7f776a800640 1 --2- 172.21.15.119:0/3237024406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f776c1610b0 0x7f776c15fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.253+0000 7f776a800640 1 -- 172.21.15.119:0/3237024406 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c14c000 con 0x7f776c1618d0 2024-04-25T10:08:52.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.253+0000 7f776a800640 1 --2- 172.21.15.119:0/3237024406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f776c1618d0 0x7f776c1601d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7764001eb0 tx=0x7f776400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.253+0000 7f7762000640 1 -- 172.21.15.119:0/3237024406 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764017070 con 0x7f776c1618d0 2024-04-25T10:08:52.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.253+0000 7f7762000640 1 -- 172.21.15.119:0/3237024406 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77640052d0 con 0x7f776c1618d0 2024-04-25T10:08:52.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.253+0000 7f7762000640 1 -- 172.21.15.119:0/3237024406 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764003810 con 0x7f776c1618d0 2024-04-25T10:08:52.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.253+0000 7f7771000640 1 -- 172.21.15.119:0/3237024406 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776c1505a0 con 0x7f776c1618d0 2024-04-25T10:08:52.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.253+0000 7f7771000640 1 -- 172.21.15.119:0/3237024406 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776c150b60 con 0x7f776c1618d0 2024-04-25T10:08:52.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.254+0000 7f7771000640 1 -- 172.21.15.119:0/3237024406 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f776c067f90 con 0x7f776c1618d0 2024-04-25T10:08:52.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.255+0000 7f7762000640 1 -- 172.21.15.119:0/3237024406 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f776401a070 con 0x7f776c1618d0 2024-04-25T10:08:52.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.256+0000 7f7762000640 1 --2- 172.21.15.119:0/3237024406 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f774c06b4d0 0x7f774c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.256+0000 7f7762000640 1 -- 172.21.15.119:0/3237024406 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7764089320 con 0x7f776c1618d0 2024-04-25T10:08:52.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.257+0000 7f7769e00640 1 --2- 172.21.15.119:0/3237024406 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f774c06b4d0 0x7f774c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.257+0000 7f7769e00640 1 --2- 172.21.15.119:0/3237024406 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f774c06b4d0 0x7f774c06d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f775c009a60 tx=0x7f775c003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.260+0000 7f7762000640 1 -- 172.21.15.119:0/3237024406 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f776c067f90 con 0x7f776c1618d0 2024-04-25T10:08:52.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.467+0000 7f7771000640 1 -- 172.21.15.119:0/3237024406 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f776c0750b0 con 0x7f776c1618d0 2024-04-25T10:08:52.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.600+0000 7f7762000640 1 -- 172.21.15.119:0/3237024406 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7764058400 con 0x7f776c1618d0 2024-04-25T10:08:52.606 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2024-04-25T10:08:52.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.603+0000 7f7771000640 1 -- 172.21.15.119:0/3237024406 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f774c06b4d0 msgr2=0x7f774c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.603+0000 7f7771000640 1 --2- 172.21.15.119:0/3237024406 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f774c06b4d0 0x7f774c06d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f775c009a60 tx=0x7f775c003990 comp rx=0 tx=0).stop 2024-04-25T10:08:52.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.603+0000 7f7771000640 1 -- 172.21.15.119:0/3237024406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f776c1618d0 msgr2=0x7f776c1601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.603+0000 7f7771000640 1 --2- 172.21.15.119:0/3237024406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f776c1618d0 0x7f776c1601d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7764001eb0 tx=0x7f776400c890 comp rx=0 tx=0).stop 2024-04-25T10:08:52.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.603+0000 7f7771000640 1 -- 172.21.15.119:0/3237024406 shutdown_connections 2024-04-25T10:08:52.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.603+0000 7f7771000640 1 --2- 172.21.15.119:0/3237024406 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f774c06b4d0 0x7f774c06d990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.603+0000 7f7771000640 1 --2- 172.21.15.119:0/3237024406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f776c1618d0 0x7f776c1601d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.603+0000 7f7771000640 1 --2- 172.21.15.119:0/3237024406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f776c1610b0 0x7f776c15fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.603+0000 7f7771000640 1 --2- 172.21.15.119:0/3237024406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f776c0ced70 0x7f776c14bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.603+0000 7f7771000640 1 -- 172.21.15.119:0/3237024406 >> 172.21.15.119:0/3237024406 conn(0x7f776c0b5570 msgr2=0x7f776c0b6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:52.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.603+0000 7f7771000640 1 -- 172.21.15.119:0/3237024406 shutdown_connections 2024-04-25T10:08:52.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.603+0000 7f7771000640 1 -- 172.21.15.119:0/3237024406 wait complete. 2024-04-25T10:08:52.632 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:08:52.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.819+0000 7f962cc00640 1 Processor -- start 2024-04-25T10:08:52.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.819+0000 7f962cc00640 1 -- start start 2024-04-25T10:08:52.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.819+0000 7f962cc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f962806ebd0 0x7f96280b1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.819+0000 7f962cc00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962815b140 con 0x7f96280b44c0 2024-04-25T10:08:52.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.820+0000 7f962cc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96280b2850 con 0x7f96280b13e0 2024-04-25T10:08:52.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.820+0000 7f962cc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96280b33d0 con 0x7f962806ebd0 2024-04-25T10:08:52.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.820+0000 7f9627400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f96280b44c0 0x7f962815aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45728/0 (socket says 172.21.15.119:45728) 2024-04-25T10:08:52.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.820+0000 7f9627400640 1 -- 172.21.15.119:0/3152392349 learned_addr learned my addr 172.21.15.119:0/3152392349 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:52.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.820+0000 7f9626a00640 1 -- 172.21.15.119:0/3152392349 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749425767 0 0) 0x7f962815b140 con 0x7f96280b44c0 2024-04-25T10:08:52.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.820+0000 7f9626a00640 1 -- 172.21.15.119:0/3152392349 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f960c003520 con 0x7f96280b44c0 2024-04-25T10:08:52.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9627e00640 1 --2- 172.21.15.119:0/3152392349 >> v2:172.21.15.132:3300/0 conn(0x7f962806ebd0 0x7f96280b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9626a00640 1 -- 172.21.15.119:0/3152392349 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3033728492 0 0) 0x7f960c003520 con 0x7f96280b44c0 2024-04-25T10:08:52.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9626a00640 1 -- 172.21.15.119:0/3152392349 >> v2:172.21.15.132:3300/0 conn(0x7f962806ebd0 msgr2=0x7f96280b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:52.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9626a00640 1 --2- 172.21.15.119:0/3152392349 >> v2:172.21.15.132:3300/0 conn(0x7f962806ebd0 0x7f96280b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9626a00640 1 -- 172.21.15.119:0/3152392349 >> v1:172.21.15.132:6789/0 conn(0x7f96280b13e0 legacy=0x7f962806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:52.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9626a00640 1 -- 172.21.15.119:0/3152392349 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96280b2b30 con 0x7f96280b44c0 2024-04-25T10:08:52.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9626a00640 1 -- 172.21.15.119:0/3152392349 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f9618001670 con 0x7f96280b44c0 2024-04-25T10:08:52.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9626a00640 1 -- 172.21.15.119:0/3152392349 >> v1:172.21.15.119:6789/0 conn(0x7f96280b44c0 legacy=0x7f962815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9626a00640 1 --2- 172.21.15.119:0/3152392349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f960c003a00 0x7f960c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9626a00640 1 --2- 172.21.15.119:0/3152392349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f960c004800 0x7f960c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9626a00640 1 --2- 172.21.15.119:0/3152392349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f960c005290 0x7f960c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9626a00640 1 -- 172.21.15.119:0/3152392349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96280b33d0 con 0x7f960c003a00 2024-04-25T10:08:52.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9626a00640 1 -- 172.21.15.119:0/3152392349 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c00c1e0 con 0x7f960c004800 2024-04-25T10:08:52.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9626a00640 1 -- 172.21.15.119:0/3152392349 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c00c3e0 con 0x7f960c005290 2024-04-25T10:08:52.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.821+0000 7f9627400640 1 --2- 172.21.15.119:0/3152392349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f960c003a00 0x7f960c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.822+0000 7f9627400640 1 -- 172.21.15.119:0/3152392349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f960c005290 msgr2=0x7f960c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:52.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.822+0000 7f9627400640 1 --2- 172.21.15.119:0/3152392349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f960c005290 0x7f960c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.822+0000 7f961ea00640 1 --2- 172.21.15.119:0/3152392349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f960c004800 0x7f960c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.822+0000 7f9627400640 1 -- 172.21.15.119:0/3152392349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f960c004800 msgr2=0x7f960c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:52.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.822+0000 7f9627400640 1 --2- 172.21.15.119:0/3152392349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f960c004800 0x7f960c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.822+0000 7f9627400640 1 -- 172.21.15.119:0/3152392349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c003520 con 0x7f960c003a00 2024-04-25T10:08:52.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.822+0000 7f9627e00640 1 -- 172.21.15.119:0/3152392349 reap_dead start 2024-04-25T10:08:52.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.822+0000 7f961ea00640 1 --2- 172.21.15.119:0/3152392349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f960c004800 0x7f960c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:52.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.823+0000 7f9627400640 1 --2- 172.21.15.119:0/3152392349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f960c003a00 0x7f960c003e00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9618007c20 tx=0x7f960c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5d9cf78e64639e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.823+0000 7f9626a00640 1 -- 172.21.15.119:0/3152392349 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618007c60 con 0x7f960c003a00 2024-04-25T10:08:52.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.823+0000 7f9626a00640 1 -- 172.21.15.119:0/3152392349 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9618036070 con 0x7f960c003a00 2024-04-25T10:08:52.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.823+0000 7f9626a00640 1 -- 172.21.15.119:0/3152392349 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618005d50 con 0x7f960c003a00 2024-04-25T10:08:52.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.823+0000 7f962cc00640 1 -- 172.21.15.119:0/3152392349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f960c003a00 msgr2=0x7f960c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.823+0000 7f962cc00640 1 --2- 172.21.15.119:0/3152392349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f960c003a00 0x7f960c003e00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9618007c20 tx=0x7f960c005830 comp rx=0 tx=0).stop 2024-04-25T10:08:52.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.824+0000 7f962cc00640 1 -- 172.21.15.119:0/3152392349 shutdown_connections 2024-04-25T10:08:52.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.824+0000 7f962cc00640 1 --2- 172.21.15.119:0/3152392349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f960c003a00 0x7f960c003e00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.824+0000 7f962cc00640 1 -- 172.21.15.119:0/3152392349 >> 172.21.15.119:0/3152392349 conn(0x7f96280b5070 msgr2=0x7f96280cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:52.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.824+0000 7f962cc00640 1 -- 172.21.15.119:0/3152392349 shutdown_connections 2024-04-25T10:08:52.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.824+0000 7f962cc00640 1 -- 172.21.15.119:0/3152392349 wait complete. 2024-04-25T10:08:52.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.825+0000 7f962cc00640 1 Processor -- start 2024-04-25T10:08:52.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.826+0000 7f962cc00640 1 -- start start 2024-04-25T10:08:52.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.826+0000 7f962cc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f960c003a00 0x7f962814ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.826+0000 7f962cc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96280ad600 0x7f96281f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.826+0000 7f962cc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962806ae00 0x7f962806b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.826+0000 7f962cc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96280b2470 con 0x7f960c003a00 2024-04-25T10:08:52.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.826+0000 7f962cc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96280b30f0 con 0x7f96280ad600 2024-04-25T10:08:52.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.826+0000 7f962cc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96280b25f0 con 0x7f962806ae00 2024-04-25T10:08:52.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.827+0000 7f9627400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f960c003a00 0x7f962814ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.827+0000 7f9627400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f960c003a00 0x7f962814ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38338/0 (socket says 172.21.15.119:38338) 2024-04-25T10:08:52.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.827+0000 7f9627400640 1 -- 172.21.15.119:0/466482421 learned_addr learned my addr 172.21.15.119:0/466482421 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:52.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.827+0000 7f961ea00640 1 --2- 172.21.15.119:0/466482421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96280ad600 0x7f96281f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.827+0000 7f9627400640 1 -- 172.21.15.119:0/466482421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962806ae00 msgr2=0x7f962806b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.827+0000 7f9627400640 1 --2- 172.21.15.119:0/466482421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962806ae00 0x7f962806b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.827+0000 7f9627400640 1 -- 172.21.15.119:0/466482421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96280ad600 msgr2=0x7f96281f78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.827+0000 7f9627400640 1 --2- 172.21.15.119:0/466482421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96280ad600 0x7f96281f78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.827+0000 7f9627400640 1 -- 172.21.15.119:0/466482421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96281f7e10 con 0x7f960c003a00 2024-04-25T10:08:52.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.827+0000 7f9627400640 1 --2- 172.21.15.119:0/466482421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f960c003a00 0x7f962814ce80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f961802cf90 tx=0x7f96280b2d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.828+0000 7f961fe00640 1 -- 172.21.15.119:0/466482421 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618007c60 con 0x7f960c003a00 2024-04-25T10:08:52.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.828+0000 7f961fe00640 1 -- 172.21.15.119:0/466482421 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961803f070 con 0x7f960c003a00 2024-04-25T10:08:52.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.828+0000 7f961fe00640 1 -- 172.21.15.119:0/466482421 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618004e60 con 0x7f960c003a00 2024-04-25T10:08:52.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.828+0000 7f962cc00640 1 -- 172.21.15.119:0/466482421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96281537c0 con 0x7f960c003a00 2024-04-25T10:08:52.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.828+0000 7f962cc00640 1 -- 172.21.15.119:0/466482421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9628153d80 con 0x7f960c003a00 2024-04-25T10:08:52.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.829+0000 7f962cc00640 1 -- 172.21.15.119:0/466482421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96280ccbe0 con 0x7f960c003a00 2024-04-25T10:08:52.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.830+0000 7f961fe00640 1 -- 172.21.15.119:0/466482421 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9618044470 con 0x7f960c003a00 2024-04-25T10:08:52.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.831+0000 7f961fe00640 1 --2- 172.21.15.119:0/466482421 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f960406b4d0 0x7f960406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.831+0000 7f961fe00640 1 -- 172.21.15.119:0/466482421 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f96180aae00 con 0x7f960c003a00 2024-04-25T10:08:52.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.831+0000 7f961ea00640 1 --2- 172.21.15.119:0/466482421 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f960406b4d0 0x7f960406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.832+0000 7f961ea00640 1 --2- 172.21.15.119:0/466482421 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f960406b4d0 0x7f960406d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9628075f30 tx=0x7f9610009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:52.836+0000 7f961fe00640 1 -- 172.21.15.119:0/466482421 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96280ccbe0 con 0x7f960c003a00 2024-04-25T10:08:53.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.104+0000 7f962cc00640 1 -- 172.21.15.119:0/466482421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f96280749b0 con 0x7f960c003a00 2024-04-25T10:08:53.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.105+0000 7f961fe00640 1 -- 172.21.15.119:0/466482421 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9618049060 con 0x7f960c003a00 2024-04-25T10:08:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:53.111 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:08:53.111 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2024-04-25T10:08:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2024-04-25T10:08:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmFDz9FBAAdC/N+8UnkYkK5wrDlgpkgA==", 2024-04-25T10:08:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:08:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:08:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2024-04-25T10:08:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmmh7lLhAAkzsFzTL6Ko/J/0ZJOpvR4A==", 2024-04-25T10:08:53.112 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:08:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:08:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2024-04-25T10:08:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmojotIhAAdSIhFOXoYZN+01nbCs3e+Q==", 2024-04-25T10:08:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.113 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:08:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:08:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2024-04-25T10:08:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypm1gp7ORAAvtG28IuhGtR1/Vj6JCPq5A==", 2024-04-25T10:08:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:08:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:08:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.114 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2024-04-25T10:08:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAPKypmOI9YMRAAmMrg0xRUG+GG+waJ12llJQ==", 2024-04-25T10:08:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2024-04-25T10:08:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQATKypmHV5GDxAAW0XzpAxF7oxgVBKy5yzavw==", 2024-04-25T10:08:53.115 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAWKypmXTiRGhAAZxct/Ck3A3DMBjqNW3HXrg==", 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.116 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2024-04-25T10:08:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAZKypmEeGSMhAAwVNTny5OmP/KONYhIXzrrQ==", 2024-04-25T10:08:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2024-04-25T10:08:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAgKypmh7gPBRAApx1VM3WpYW7hIQzu7+6J7w==", 2024-04-25T10:08:53.117 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2024-04-25T10:08:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAjKypmQ2EaEBAAoFYMlnh5/ar0pAlynwjYOA==", 2024-04-25T10:08:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.118 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2024-04-25T10:08:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAmKypmDATkKRAAsIDViOBeDrd5N91wf8zGLA==", 2024-04-25T10:08:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2024-04-25T10:08:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAqKypmNskbCBAAQIYhJ9lfHHr9EP/n5QJoRw==", 2024-04-25T10:08:53.119 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA==", 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.120 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2024-04-25T10:08:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmm0DLGhAApkxEcc1jUU5T/9ErkTkT4A==", 2024-04-25T10:08:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:08:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:08:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:08:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2024-04-25T10:08:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmMZ89ERAAICfWm/ZgOLQMVSv5xG0Crg==", 2024-04-25T10:08:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:53.121 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:08:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:08:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:08:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2024-04-25T10:08:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmrokCIxAAwi8FtvYSVhMguvZ0OyYt+g==", 2024-04-25T10:08:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:08:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:08:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:08:53.122 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:08:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2024-04-25T10:08:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAKKypmN9WFGRAAi+stjuG2ZDBjafeR9Ygrug==", 2024-04-25T10:08:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:08:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:08:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:08:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.123 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2024-04-25T10:08:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypm/KyKDBAA646fW6qrcXB31usxeIApaA==", 2024-04-25T10:08:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:08:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2024-04-25T10:08:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:08:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2024-04-25T10:08:53.124 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmr8SqBBAA8l4CjUEidXz1obWJFM4DHA==", 2024-04-25T10:08:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:08:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:08:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2024-04-25T10:08:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:08:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:53.125 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-25T10:08:53.125 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:08:53.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.109+0000 7f962cc00640 1 -- 172.21.15.119:0/466482421 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f960406b4d0 msgr2=0x7f960406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.109+0000 7f962cc00640 1 --2- 172.21.15.119:0/466482421 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f960406b4d0 0x7f960406d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9628075f30 tx=0x7f9610009040 comp rx=0 tx=0).stop 2024-04-25T10:08:53.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.109+0000 7f962cc00640 1 -- 172.21.15.119:0/466482421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f960c003a00 msgr2=0x7f962814ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.109+0000 7f962cc00640 1 --2- 172.21.15.119:0/466482421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f960c003a00 0x7f962814ce80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f961802cf90 tx=0x7f96280b2d00 comp rx=0 tx=0).stop 2024-04-25T10:08:53.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.109+0000 7f962cc00640 1 -- 172.21.15.119:0/466482421 shutdown_connections 2024-04-25T10:08:53.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.109+0000 7f962cc00640 1 --2- 172.21.15.119:0/466482421 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f960406b4d0 0x7f960406d990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.109+0000 7f962cc00640 1 --2- 172.21.15.119:0/466482421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962806ae00 0x7f962806b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.109+0000 7f962cc00640 1 --2- 172.21.15.119:0/466482421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96280ad600 0x7f96281f78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.109+0000 7f962cc00640 1 --2- 172.21.15.119:0/466482421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f960c003a00 0x7f962814ce80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.109+0000 7f962cc00640 1 -- 172.21.15.119:0/466482421 >> 172.21.15.119:0/466482421 conn(0x7f96280b5070 msgr2=0x7f9628159f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:53.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.109+0000 7f962cc00640 1 -- 172.21.15.119:0/466482421 shutdown_connections 2024-04-25T10:08:53.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.109+0000 7f962cc00640 1 -- 172.21.15.119:0/466482421 wait complete. 2024-04-25T10:08:53.132 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:53.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.328+0000 7f1da8e00640 1 Processor -- start 2024-04-25T10:08:53.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.328+0000 7f1da8e00640 1 -- start start 2024-04-25T10:08:53.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.330+0000 7f1da8e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1da406e8f0 0x7f1da415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.330+0000 7f1da8e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1da415c820 0x7f1da41654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.330+0000 7f1da8e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da40ce6c0 con 0x7f1da415bf00 2024-04-25T10:08:53.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.330+0000 7f1da8e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da41455e0 con 0x7f1da415c820 2024-04-25T10:08:53.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.330+0000 7f1da8e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da406d450 con 0x7f1da406e8f0 2024-04-25T10:08:53.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.331+0000 7f1da2800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1da415c820 0x7f1da41654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.331+0000 7f1da1e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f1da415bf00 0x7f1da40cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:60010/0 (socket says 172.21.15.119:60010) 2024-04-25T10:08:53.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.331+0000 7f1da2800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1da415c820 0x7f1da41654b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:37884/0 (socket says 172.21.15.119:37884) 2024-04-25T10:08:53.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.331+0000 7f1da1e00640 1 -- 172.21.15.119:0/4082280357 learned_addr learned my addr 172.21.15.119:0/4082280357 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:53.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.331+0000 7f1da1400640 1 --2- 172.21.15.119:0/4082280357 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1da406e8f0 0x7f1da415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.331+0000 7f1da2800640 1 -- 172.21.15.119:0/4082280357 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1da406e8f0 msgr2=0x7f1da415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:53.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.331+0000 7f1da2800640 1 --2- 172.21.15.119:0/4082280357 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1da406e8f0 0x7f1da415c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.331+0000 7f1da2800640 1 -- 172.21.15.119:0/4082280357 >> v1:172.21.15.132:6789/0 conn(0x7f1da415bf00 legacy=0x7f1da40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.331+0000 7f1da2800640 1 -- 172.21.15.119:0/4082280357 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da40cc4e0 con 0x7f1da415c820 2024-04-25T10:08:53.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.333+0000 7f1da2800640 1 --2- 172.21.15.119:0/4082280357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1da415c820 0x7f1da41654b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0097a0 tx=0x7f1d9c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=627a67751a547414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.333+0000 7f1da0a00640 1 -- 172.21.15.119:0/4082280357 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c037070 con 0x7f1da415c820 2024-04-25T10:08:53.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.333+0000 7f1da0a00640 1 -- 172.21.15.119:0/4082280357 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c032440 con 0x7f1da415c820 2024-04-25T10:08:53.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.333+0000 7f1da0a00640 1 -- 172.21.15.119:0/4082280357 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c032810 con 0x7f1da415c820 2024-04-25T10:08:53.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.334+0000 7f1da8e00640 1 -- 172.21.15.119:0/4082280357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1da415c820 msgr2=0x7f1da41654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.334+0000 7f1da8e00640 1 --2- 172.21.15.119:0/4082280357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1da415c820 0x7f1da41654b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0097a0 tx=0x7f1d9c02a180 comp rx=0 tx=0).stop 2024-04-25T10:08:53.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.334+0000 7f1da8e00640 1 -- 172.21.15.119:0/4082280357 shutdown_connections 2024-04-25T10:08:53.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.334+0000 7f1da8e00640 1 --2- 172.21.15.119:0/4082280357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1da415c820 0x7f1da41654b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.334+0000 7f1da8e00640 1 --2- 172.21.15.119:0/4082280357 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1da406e8f0 0x7f1da415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.334+0000 7f1da8e00640 1 -- 172.21.15.119:0/4082280357 >> 172.21.15.119:0/4082280357 conn(0x7f1da406aba0 msgr2=0x7f1da406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:53.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.334+0000 7f1da8e00640 1 -- 172.21.15.119:0/4082280357 shutdown_connections 2024-04-25T10:08:53.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.334+0000 7f1da8e00640 1 -- 172.21.15.119:0/4082280357 wait complete. 2024-04-25T10:08:53.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.336+0000 7f1da8e00640 1 Processor -- start 2024-04-25T10:08:53.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.336+0000 7f1da8e00640 1 -- start start 2024-04-25T10:08:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.337+0000 7f1da8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1da406e8f0 0x7f1da4137fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.337+0000 7f1da8e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1da415bf00 0x7f1da413b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.337+0000 7f1da8e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1da415c820 0x7f1da4071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.337+0000 7f1da8e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da40ccd70 con 0x7f1da406e8f0 2024-04-25T10:08:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.337+0000 7f1da8e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da40cca90 con 0x7f1da415c820 2024-04-25T10:08:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.337+0000 7f1da8e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da40cd050 con 0x7f1da415bf00 2024-04-25T10:08:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.337+0000 7f1da1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1da406e8f0 0x7f1da4137fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.337+0000 7f1da1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1da406e8f0 0x7f1da4137fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38340/0 (socket says 172.21.15.119:38340) 2024-04-25T10:08:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.337+0000 7f1da1e00640 1 -- 172.21.15.119:0/3031137222 learned_addr learned my addr 172.21.15.119:0/3031137222 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.338+0000 7f1da1400640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1da415bf00 0x7f1da413b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.338+0000 7f1da2800640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1da415c820 0x7f1da4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.338+0000 7f1da1e00640 1 -- 172.21.15.119:0/3031137222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1da415bf00 msgr2=0x7f1da413b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.338+0000 7f1da1e00640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1da415bf00 0x7f1da413b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.338+0000 7f1da1e00640 1 -- 172.21.15.119:0/3031137222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1da415c820 msgr2=0x7f1da4071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.338+0000 7f1da1e00640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1da415c820 0x7f1da4071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.338+0000 7f1da1e00640 1 -- 172.21.15.119:0/3031137222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da40b4430 con 0x7f1da406e8f0 2024-04-25T10:08:53.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.338+0000 7f1da2800640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1da415c820 0x7f1da4071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:53.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.338+0000 7f1da1e00640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1da406e8f0 0x7f1da4137fe0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400cb10 tx=0x7f1d9400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.338+0000 7f1d9a000640 1 -- 172.21.15.119:0/3031137222 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94017070 con 0x7f1da406e8f0 2024-04-25T10:08:53.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.338+0000 7f1d9a000640 1 -- 172.21.15.119:0/3031137222 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9400ac70 con 0x7f1da406e8f0 2024-04-25T10:08:53.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.338+0000 7f1d9a000640 1 -- 172.21.15.119:0/3031137222 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94012620 con 0x7f1da406e8f0 2024-04-25T10:08:53.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.338+0000 7f1da8e00640 1 -- 172.21.15.119:0/3031137222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da413a730 con 0x7f1da406e8f0 2024-04-25T10:08:53.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.339+0000 7f1da8e00640 1 -- 172.21.15.119:0/3031137222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da413acf0 con 0x7f1da406e8f0 2024-04-25T10:08:53.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.340+0000 7f1da8e00640 1 -- 172.21.15.119:0/3031137222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da406bb30 con 0x7f1da406e8f0 2024-04-25T10:08:53.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.341+0000 7f1d9a000640 1 -- 172.21.15.119:0/3031137222 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d9401a070 con 0x7f1da406e8f0 2024-04-25T10:08:53.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.341+0000 7f1d9a000640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d8006b4d0 0x7f1d8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.342+0000 7f1d9a000640 1 -- 172.21.15.119:0/3031137222 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1d94089d20 con 0x7f1da406e8f0 2024-04-25T10:08:53.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.342+0000 7f1da1400640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d8006b4d0 0x7f1d8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.342+0000 7f1da1400640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d8006b4d0 0x7f1d8006d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1da415b890 tx=0x7f1d8c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.346+0000 7f1d9a000640 1 -- 172.21.15.119:0/3031137222 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1da40ce6c0 con 0x7f1da406e8f0 2024-04-25T10:08:53.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.541+0000 7f1da8e00640 1 -- 172.21.15.119:0/3031137222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1da40749b0 con 0x7f1da406e8f0 2024-04-25T10:08:53.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.543+0000 7f1d9a000640 1 -- 172.21.15.119:0/3031137222 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+12927 (secure 0 0 0) 0x7f1d94058e50 con 0x7f1da406e8f0 2024-04-25T10:08:53.549 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:53.549 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":40,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:52.614826+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:53.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.545+0000 7f1da8e00640 1 -- 172.21.15.119:0/3031137222 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d8006b4d0 msgr2=0x7f1d8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.545+0000 7f1da8e00640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d8006b4d0 0x7f1d8006d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1da415b890 tx=0x7f1d8c008040 comp rx=0 tx=0).stop 2024-04-25T10:08:53.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.545+0000 7f1da8e00640 1 -- 172.21.15.119:0/3031137222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1da406e8f0 msgr2=0x7f1da4137fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.545+0000 7f1da8e00640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1da406e8f0 0x7f1da4137fe0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400cb10 tx=0x7f1d9400a060 comp rx=0 tx=0).stop 2024-04-25T10:08:53.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.546+0000 7f1da8e00640 1 -- 172.21.15.119:0/3031137222 shutdown_connections 2024-04-25T10:08:53.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.546+0000 7f1da8e00640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d8006b4d0 0x7f1d8006d990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.546+0000 7f1da8e00640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1da415c820 0x7f1da4071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.546+0000 7f1da8e00640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1da415bf00 0x7f1da413b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.546+0000 7f1da8e00640 1 --2- 172.21.15.119:0/3031137222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1da406e8f0 0x7f1da4137fe0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.546+0000 7f1da8e00640 1 -- 172.21.15.119:0/3031137222 >> 172.21.15.119:0/3031137222 conn(0x7f1da406aba0 msgr2=0x7f1da413f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:53.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.546+0000 7f1da8e00640 1 -- 172.21.15.119:0/3031137222 shutdown_connections 2024-04-25T10:08:53.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.546+0000 7f1da8e00640 1 -- 172.21.15.119:0/3031137222 wait complete. 2024-04-25T10:08:53.571 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:53.571 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:53.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.769+0000 7fd5c4800640 1 Processor -- start 2024-04-25T10:08:53.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.769+0000 7fd5c4800640 1 -- start start 2024-04-25T10:08:53.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.771+0000 7fd5c4800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5bc15dbb0 0x7fd5bc0b9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.771+0000 7fd5c4800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5bc0d1e20 0x7fd5bc0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.771+0000 7fd5c4800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc0b7b80 con 0x7fd5bc0b5aa0 2024-04-25T10:08:53.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.771+0000 7fd5c4800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5bc0d2670 con 0x7fd5bc0d1e20 2024-04-25T10:08:53.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.771+0000 7fd5c4800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0d2950 con 0x7fd5bc15dbb0 2024-04-25T10:08:53.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.771+0000 7fd5c1400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5bc0d1e20 0x7fd5bc0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.771+0000 7fd5c1400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5bc0d1e20 0x7fd5bc0b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38350/0 (socket says 172.21.15.119:38350) 2024-04-25T10:08:53.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.771+0000 7fd5c1400640 1 -- 172.21.15.119:0/1574170168 learned_addr learned my addr 172.21.15.119:0/1574170168 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:53.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.772+0000 7fd5c1e00640 1 --2- 172.21.15.119:0/1574170168 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5bc15dbb0 0x7fd5bc0b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.772+0000 7fd5c1400640 1 -- 172.21.15.119:0/1574170168 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5bc15dbb0 msgr2=0x7fd5bc0b9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:53.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.772+0000 7fd5c1400640 1 --2- 172.21.15.119:0/1574170168 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5bc15dbb0 0x7fd5bc0b9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.772+0000 7fd5c1400640 1 -- 172.21.15.119:0/1574170168 >> v1:172.21.15.132:6789/0 conn(0x7fd5bc0b5aa0 legacy=0x7fd5bc06c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:53.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.772+0000 7fd5c1400640 1 -- 172.21.15.119:0/1574170168 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc0d2c30 con 0x7fd5bc0d1e20 2024-04-25T10:08:53.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.773+0000 7fd5c1400640 1 --2- 172.21.15.119:0/1574170168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5bc0d1e20 0x7fd5bc0b5560 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009870 tx=0x7fd5b002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0573de81e12af2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.773+0000 7fd5c0a00640 1 -- 172.21.15.119:0/1574170168 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0037070 con 0x7fd5bc0d1e20 2024-04-25T10:08:53.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.773+0000 7fd5c0a00640 1 -- 172.21.15.119:0/1574170168 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b0032440 con 0x7fd5bc0d1e20 2024-04-25T10:08:53.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.773+0000 7fd5c0a00640 1 -- 172.21.15.119:0/1574170168 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0032810 con 0x7fd5bc0d1e20 2024-04-25T10:08:53.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.774+0000 7fd5c4800640 1 -- 172.21.15.119:0/1574170168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5bc0d1e20 msgr2=0x7fd5bc0b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.774+0000 7fd5c4800640 1 --2- 172.21.15.119:0/1574170168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5bc0d1e20 0x7fd5bc0b5560 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009870 tx=0x7fd5b002a210 comp rx=0 tx=0).stop 2024-04-25T10:08:53.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.774+0000 7fd5c4800640 1 -- 172.21.15.119:0/1574170168 shutdown_connections 2024-04-25T10:08:53.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.774+0000 7fd5c4800640 1 --2- 172.21.15.119:0/1574170168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5bc0d1e20 0x7fd5bc0b5560 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.774+0000 7fd5c4800640 1 --2- 172.21.15.119:0/1574170168 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5bc15dbb0 0x7fd5bc0b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.774+0000 7fd5c4800640 1 -- 172.21.15.119:0/1574170168 >> 172.21.15.119:0/1574170168 conn(0x7fd5bc0d0560 msgr2=0x7fd5bc0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:53.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.774+0000 7fd5c4800640 1 -- 172.21.15.119:0/1574170168 shutdown_connections 2024-04-25T10:08:53.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.774+0000 7fd5c4800640 1 -- 172.21.15.119:0/1574170168 wait complete. 2024-04-25T10:08:53.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.776+0000 7fd5c4800640 1 Processor -- start 2024-04-25T10:08:53.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.776+0000 7fd5c4800640 1 -- start start 2024-04-25T10:08:53.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.777+0000 7fd5c4800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5bc0b5aa0 0x7fd5bc1554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.777+0000 7fd5c4800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5bc0d1e20 0x7fd5bc158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.777+0000 7fd5c4800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5bc15dbb0 0x7fd5bc159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.777+0000 7fd5c4800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc15df90 con 0x7fd5bc0b5aa0 2024-04-25T10:08:53.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.777+0000 7fd5c4800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0d31f0 con 0x7fd5bc15dbb0 2024-04-25T10:08:53.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.777+0000 7fd5c4800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc15e110 con 0x7fd5bc0d1e20 2024-04-25T10:08:53.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.777+0000 7fd5c1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5bc0b5aa0 0x7fd5bc1554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.777+0000 7fd5c1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5bc0b5aa0 0x7fd5bc1554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38352/0 (socket says 172.21.15.119:38352) 2024-04-25T10:08:53.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.777+0000 7fd5c1e00640 1 -- 172.21.15.119:0/865382382 learned_addr learned my addr 172.21.15.119:0/865382382 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.778+0000 7fd5c2800640 1 --2- 172.21.15.119:0/865382382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5bc15dbb0 0x7fd5bc159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.778+0000 7fd5c1400640 1 --2- 172.21.15.119:0/865382382 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5bc0d1e20 0x7fd5bc158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.778+0000 7fd5c1e00640 1 -- 172.21.15.119:0/865382382 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5bc0d1e20 msgr2=0x7fd5bc158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.778+0000 7fd5c1e00640 1 --2- 172.21.15.119:0/865382382 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5bc0d1e20 0x7fd5bc158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.778+0000 7fd5c1e00640 1 -- 172.21.15.119:0/865382382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5bc15dbb0 msgr2=0x7fd5bc159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.778+0000 7fd5c1e00640 1 --2- 172.21.15.119:0/865382382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5bc15dbb0 0x7fd5bc159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.778+0000 7fd5c1e00640 1 -- 172.21.15.119:0/865382382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc155a20 con 0x7fd5bc0b5aa0 2024-04-25T10:08:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.778+0000 7fd5c1e00640 1 --2- 172.21.15.119:0/865382382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5bc0b5aa0 0x7fd5bc1554e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800c7f0 tx=0x7fd5a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.778+0000 7fd5b6000640 1 -- 172.21.15.119:0/865382382 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8015070 con 0x7fd5bc0b5aa0 2024-04-25T10:08:53.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.778+0000 7fd5b6000640 1 -- 172.21.15.119:0/865382382 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a8004220 con 0x7fd5bc0b5aa0 2024-04-25T10:08:53.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.778+0000 7fd5b6000640 1 -- 172.21.15.119:0/865382382 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8007620 con 0x7fd5bc0b5aa0 2024-04-25T10:08:53.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.779+0000 7fd5c4800640 1 -- 172.21.15.119:0/865382382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc1e4780 con 0x7fd5bc0b5aa0 2024-04-25T10:08:53.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.779+0000 7fd5c4800640 1 -- 172.21.15.119:0/865382382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc1e4d40 con 0x7fd5bc0b5aa0 2024-04-25T10:08:53.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.780+0000 7fd5c4800640 1 -- 172.21.15.119:0/865382382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5bc075090 con 0x7fd5bc0b5aa0 2024-04-25T10:08:53.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.781+0000 7fd5b6000640 1 -- 172.21.15.119:0/865382382 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5a80079d0 con 0x7fd5bc0b5aa0 2024-04-25T10:08:53.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.782+0000 7fd5b6000640 1 --2- 172.21.15.119:0/865382382 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd59c06b4d0 0x7fd59c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.782+0000 7fd5b6000640 1 -- 172.21.15.119:0/865382382 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd5a808a150 con 0x7fd5bc0b5aa0 2024-04-25T10:08:53.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.782+0000 7fd5c1400640 1 --2- 172.21.15.119:0/865382382 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd59c06b4d0 0x7fd59c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.783+0000 7fd5c1400640 1 --2- 172.21.15.119:0/865382382 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd59c06b4d0 0x7fd59c06d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0034000 tx=0x7fd5b002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.786+0000 7fd5b6000640 1 -- 172.21.15.119:0/865382382 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5a80593b0 con 0x7fd5bc0b5aa0 2024-04-25T10:08:53.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.983+0000 7fd5c4800640 1 -- 172.21.15.119:0/865382382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5bc0d43b0 con 0x7fd5bc0b5aa0 2024-04-25T10:08:53.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.983+0000 7fd5b6000640 1 -- 172.21.15.119:0/865382382 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5a8058cb0 con 0x7fd5bc0b5aa0 2024-04-25T10:08:53.989 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:08:53.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.986+0000 7fd5c4800640 1 -- 172.21.15.119:0/865382382 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd59c06b4d0 msgr2=0x7fd59c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.986+0000 7fd5c4800640 1 --2- 172.21.15.119:0/865382382 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd59c06b4d0 0x7fd59c06d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0034000 tx=0x7fd5b002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.986+0000 7fd5c4800640 1 -- 172.21.15.119:0/865382382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5bc0b5aa0 msgr2=0x7fd5bc1554e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.986+0000 7fd5c4800640 1 --2- 172.21.15.119:0/865382382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5bc0b5aa0 0x7fd5bc1554e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800c7f0 tx=0x7fd5a800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.986+0000 7fd5c4800640 1 -- 172.21.15.119:0/865382382 shutdown_connections 2024-04-25T10:08:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.986+0000 7fd5c4800640 1 --2- 172.21.15.119:0/865382382 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd59c06b4d0 0x7fd59c06d990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.986+0000 7fd5c4800640 1 --2- 172.21.15.119:0/865382382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5bc15dbb0 0x7fd5bc159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.986+0000 7fd5c4800640 1 --2- 172.21.15.119:0/865382382 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5bc0d1e20 0x7fd5bc158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.986+0000 7fd5c4800640 1 --2- 172.21.15.119:0/865382382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5bc0b5aa0 0x7fd5bc1554e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.986+0000 7fd5c4800640 1 -- 172.21.15.119:0/865382382 >> 172.21.15.119:0/865382382 conn(0x7fd5bc0d0560 msgr2=0x7fd5bc0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.987+0000 7fd5c4800640 1 -- 172.21.15.119:0/865382382 shutdown_connections 2024-04-25T10:08:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:53.987+0000 7fd5c4800640 1 -- 172.21.15.119:0/865382382 wait complete. 2024-04-25T10:08:54.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:08:54.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.203+0000 7f4dea800640 1 Processor -- start 2024-04-25T10:08:54.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.203+0000 7f4dea800640 1 -- start start 2024-04-25T10:08:54.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.205+0000 7f4dea800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4de40b11e0 0x7f4de415d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.205+0000 7f4dea800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4de40b15c0 0x7f4de406d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.205+0000 7f4dea800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4de415d5d0 con 0x7f4de406d9d0 2024-04-25T10:08:54.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.205+0000 7f4dea800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de40ca250 con 0x7f4de40b11e0 2024-04-25T10:08:54.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.205+0000 7f4dea800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de40ca4a0 con 0x7f4de40b15c0 2024-04-25T10:08:54.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.205+0000 7f4de3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4de40b11e0 0x7f4de415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.205+0000 7f4de3e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4de406d9d0 0x7f4de40cb260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45736/0 (socket says 172.21.15.119:45736) 2024-04-25T10:08:54.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.205+0000 7f4de3e00640 1 -- 172.21.15.119:0/3435716544 learned_addr learned my addr 172.21.15.119:0/3435716544 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:54.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.205+0000 7f4de3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4de40b11e0 0x7f4de415d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38364/0 (socket says 172.21.15.119:38364) 2024-04-25T10:08:54.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.206+0000 7f4de2a00640 1 --2- 172.21.15.119:0/3435716544 >> v2:172.21.15.132:3300/0 conn(0x7f4de40b15c0 0x7f4de406d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.206+0000 7f4de3400640 1 -- 172.21.15.119:0/3435716544 >> v2:172.21.15.132:3300/0 conn(0x7f4de40b15c0 msgr2=0x7f4de406d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.206+0000 7f4de3400640 1 --2- 172.21.15.119:0/3435716544 >> v2:172.21.15.132:3300/0 conn(0x7f4de40b15c0 0x7f4de406d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.206+0000 7f4de3400640 1 -- 172.21.15.119:0/3435716544 >> v1:172.21.15.119:6789/0 conn(0x7f4de406d9d0 legacy=0x7f4de40cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.206+0000 7f4de3400640 1 -- 172.21.15.119:0/3435716544 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de40ca780 con 0x7f4de40b11e0 2024-04-25T10:08:54.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.206+0000 7f4de2a00640 1 --2- 172.21.15.119:0/3435716544 >> v2:172.21.15.132:3300/0 conn(0x7f4de40b15c0 0x7f4de406d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:54.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.207+0000 7f4de3400640 1 --2- 172.21.15.119:0/3435716544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de40b11e0 0x7f4de415d090 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4de40ca3d0 tx=0x7f4dd402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58b60c6ccd39c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.207+0000 7f4de2000640 1 -- 172.21.15.119:0/3435716544 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4037070 con 0x7f4de40b11e0 2024-04-25T10:08:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.208+0000 7f4de2000640 1 -- 172.21.15.119:0/3435716544 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd4032440 con 0x7f4de40b11e0 2024-04-25T10:08:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.208+0000 7f4de2000640 1 -- 172.21.15.119:0/3435716544 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd402ae60 con 0x7f4de40b11e0 2024-04-25T10:08:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.208+0000 7f4dea800640 1 -- 172.21.15.119:0/3435716544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de40b11e0 msgr2=0x7f4de415d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.208+0000 7f4dea800640 1 --2- 172.21.15.119:0/3435716544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de40b11e0 0x7f4de415d090 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4de40ca3d0 tx=0x7f4dd402a110 comp rx=0 tx=0).stop 2024-04-25T10:08:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.208+0000 7f4dea800640 1 -- 172.21.15.119:0/3435716544 shutdown_connections 2024-04-25T10:08:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.208+0000 7f4dea800640 1 --2- 172.21.15.119:0/3435716544 >> v2:172.21.15.132:3300/0 conn(0x7f4de40b15c0 0x7f4de406d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.208+0000 7f4dea800640 1 --2- 172.21.15.119:0/3435716544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de40b11e0 0x7f4de415d090 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.208+0000 7f4dea800640 1 -- 172.21.15.119:0/3435716544 >> 172.21.15.119:0/3435716544 conn(0x7f4de40cf5a0 msgr2=0x7f4de40cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.209+0000 7f4dea800640 1 -- 172.21.15.119:0/3435716544 shutdown_connections 2024-04-25T10:08:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.209+0000 7f4dea800640 1 -- 172.21.15.119:0/3435716544 wait complete. 2024-04-25T10:08:54.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.210+0000 7f4dea800640 1 Processor -- start 2024-04-25T10:08:54.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.210+0000 7f4dea800640 1 -- start start 2024-04-25T10:08:54.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.211+0000 7f4dea800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de406d9d0 0x7f4de4154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.211+0000 7f4dea800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4de40b11e0 0x7f4de41e6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.211+0000 7f4dea800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de40b15c0 0x7f4de41e7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.211+0000 7f4dea800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de40b1a30 con 0x7f4de40b15c0 2024-04-25T10:08:54.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.211+0000 7f4dea800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de40cad40 con 0x7f4de406d9d0 2024-04-25T10:08:54.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.211+0000 7f4dea800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de41597b0 con 0x7f4de40b11e0 2024-04-25T10:08:54.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.212+0000 7f4de3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de40b15c0 0x7f4de41e7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.212+0000 7f4de3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de40b15c0 0x7f4de41e7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38378/0 (socket says 172.21.15.119:38378) 2024-04-25T10:08:54.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.212+0000 7f4de3e00640 1 -- 172.21.15.119:0/171506265 learned_addr learned my addr 172.21.15.119:0/171506265 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:54.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.212+0000 7f4de3400640 1 --2- 172.21.15.119:0/171506265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de406d9d0 0x7f4de4154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.212+0000 7f4de3e00640 1 -- 172.21.15.119:0/171506265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4de40b11e0 msgr2=0x7f4de41e6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.212+0000 7f4de3e00640 1 --2- 172.21.15.119:0/171506265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4de40b11e0 0x7f4de41e6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.212+0000 7f4de3e00640 1 -- 172.21.15.119:0/171506265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de406d9d0 msgr2=0x7f4de4154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.212+0000 7f4de3e00640 1 --2- 172.21.15.119:0/171506265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de406d9d0 0x7f4de4154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.212+0000 7f4de3e00640 1 -- 172.21.15.119:0/171506265 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de41e85f0 con 0x7f4de40b15c0 2024-04-25T10:08:54.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.213+0000 7f4de3e00640 1 --2- 172.21.15.119:0/171506265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de40b15c0 0x7f4de41e7400 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00caa0 tx=0x7f4ddc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.213+0000 7f4ddb400640 1 -- 172.21.15.119:0/171506265 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc017070 con 0x7f4de40b15c0 2024-04-25T10:08:54.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.213+0000 7f4ddb400640 1 -- 172.21.15.119:0/171506265 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ddc00ac00 con 0x7f4de40b15c0 2024-04-25T10:08:54.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.213+0000 7f4ddb400640 1 -- 172.21.15.119:0/171506265 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc012620 con 0x7f4de40b15c0 2024-04-25T10:08:54.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.213+0000 7f4dea800640 1 -- 172.21.15.119:0/171506265 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de4157e40 con 0x7f4de40b15c0 2024-04-25T10:08:54.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.213+0000 7f4dea800640 1 -- 172.21.15.119:0/171506265 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de4158400 con 0x7f4de40b15c0 2024-04-25T10:08:54.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.214+0000 7f4dea800640 1 -- 172.21.15.119:0/171506265 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de40aed80 con 0x7f4de40b15c0 2024-04-25T10:08:54.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.215+0000 7f4ddb400640 1 -- 172.21.15.119:0/171506265 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ddc01a070 con 0x7f4de40b15c0 2024-04-25T10:08:54.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.216+0000 7f4ddb400640 1 --2- 172.21.15.119:0/171506265 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4dc406b500 0x7f4dc406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.217+0000 7f4ddb400640 1 -- 172.21.15.119:0/171506265 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4ddc089bd0 con 0x7f4de40b15c0 2024-04-25T10:08:54.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.217+0000 7f4de3400640 1 --2- 172.21.15.119:0/171506265 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4dc406b500 0x7f4dc406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.217+0000 7f4de3400640 1 --2- 172.21.15.119:0/171506265 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4dc406b500 0x7f4dc406d9c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4034000 tx=0x7f4dd402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.221+0000 7f4ddb400640 1 -- 172.21.15.119:0/171506265 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ddc058d90 con 0x7f4de40b15c0 2024-04-25T10:08:54.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.419+0000 7f4dea800640 1 -- 172.21.15.119:0/171506265 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4de40749b0 con 0x7f4de40b15c0 2024-04-25T10:08:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.419+0000 7f4ddb400640 1 -- 172.21.15.119:0/171506265 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4ddc058690 con 0x7f4de40b15c0 2024-04-25T10:08:54.425 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:54.425 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2024-04-25T10:08:54.425 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:08:54.426 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2024-04-25T10:08:54.426 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2024-04-25T10:08:54.426 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:08:54.426 INFO:teuthology.orchestra.run.smithi119.stdout:] 2024-04-25T10:08:54.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.422+0000 7f4dea800640 1 -- 172.21.15.119:0/171506265 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4dc406b500 msgr2=0x7f4dc406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.422+0000 7f4dea800640 1 --2- 172.21.15.119:0/171506265 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4dc406b500 0x7f4dc406d9c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4034000 tx=0x7f4dd402e000 comp rx=0 tx=0).stop 2024-04-25T10:08:54.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.422+0000 7f4dea800640 1 -- 172.21.15.119:0/171506265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de40b15c0 msgr2=0x7f4de41e7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.422+0000 7f4dea800640 1 --2- 172.21.15.119:0/171506265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de40b15c0 0x7f4de41e7400 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00caa0 tx=0x7f4ddc00cf60 comp rx=0 tx=0).stop 2024-04-25T10:08:54.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.422+0000 7f4dea800640 1 -- 172.21.15.119:0/171506265 shutdown_connections 2024-04-25T10:08:54.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.422+0000 7f4dea800640 1 --2- 172.21.15.119:0/171506265 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4dc406b500 0x7f4dc406d9c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.422+0000 7f4dea800640 1 --2- 172.21.15.119:0/171506265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de40b15c0 0x7f4de41e7400 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.422+0000 7f4dea800640 1 --2- 172.21.15.119:0/171506265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4de40b11e0 0x7f4de41e6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.422+0000 7f4dea800640 1 --2- 172.21.15.119:0/171506265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de406d9d0 0x7f4de4154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.422+0000 7f4dea800640 1 -- 172.21.15.119:0/171506265 >> 172.21.15.119:0/171506265 conn(0x7f4de40cf5a0 msgr2=0x7f4de40cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.423+0000 7f4dea800640 1 -- 172.21.15.119:0/171506265 shutdown_connections 2024-04-25T10:08:54.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.423+0000 7f4dea800640 1 -- 172.21.15.119:0/171506265 wait complete. 2024-04-25T10:08:54.447 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:08:54.447 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:08:54.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.651+0000 7f14ee400640 1 Processor -- start 2024-04-25T10:08:54.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.651+0000 7f14ee400640 1 -- start start 2024-04-25T10:08:54.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.652+0000 7f14ee400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e8144fd0 0x7f14e815f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.652+0000 7f14ee400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f14e80cf180 0x7f14e80ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.653+0000 7f14ee400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e815bdf0 con 0x7f14e806e670 2024-04-25T10:08:54.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.653+0000 7f14ee400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e806d4e0 con 0x7f14e80cf180 2024-04-25T10:08:54.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.653+0000 7f14ee400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e806d7c0 con 0x7f14e8144fd0 2024-04-25T10:08:54.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.653+0000 7f14ed000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e8144fd0 0x7f14e815f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.654+0000 7f14ed000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e8144fd0 0x7f14e815f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:40310/0 (socket says 172.21.15.119:40310) 2024-04-25T10:08:54.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.654+0000 7f14ed000640 1 -- 172.21.15.119:0/4286892461 learned_addr learned my addr 172.21.15.119:0/4286892461 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:54.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.654+0000 7f14eda00640 1 --1- 172.21.15.119:0/4286892461 >> v1:172.21.15.132:6789/0 conn(0x7f14e806e670 0x7f14e815c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:60036/0 (socket says 172.21.15.119:60036) 2024-04-25T10:08:54.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.654+0000 7f14e7e00640 1 --2- 172.21.15.119:0/4286892461 >> v2:172.21.15.132:3300/0 conn(0x7f14e80cf180 0x7f14e80ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.654+0000 7f14e7400640 1 -- 172.21.15.119:0/4286892461 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506953002 0 0) 0x7f14e815bdf0 con 0x7f14e806e670 2024-04-25T10:08:54.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.654+0000 7f14e7400640 1 -- 172.21.15.119:0/4286892461 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14d0003520 con 0x7f14e806e670 2024-04-25T10:08:54.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.654+0000 7f14ed000640 1 -- 172.21.15.119:0/4286892461 >> v2:172.21.15.132:3300/0 conn(0x7f14e80cf180 msgr2=0x7f14e80ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.654+0000 7f14ed000640 1 --2- 172.21.15.119:0/4286892461 >> v2:172.21.15.132:3300/0 conn(0x7f14e80cf180 0x7f14e80ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.654+0000 7f14ed000640 1 -- 172.21.15.119:0/4286892461 >> v1:172.21.15.132:6789/0 conn(0x7f14e806e670 legacy=0x7f14e815c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.654+0000 7f14ed000640 1 -- 172.21.15.119:0/4286892461 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e806daa0 con 0x7f14e8144fd0 2024-04-25T10:08:54.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.654+0000 7f14e7e00640 1 --2- 172.21.15.119:0/4286892461 >> v2:172.21.15.132:3300/0 conn(0x7f14e80cf180 0x7f14e80ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:54.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.656+0000 7f14ed000640 1 --2- 172.21.15.119:0/4286892461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e8144fd0 0x7f14e815f400 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009870 tx=0x7f14dc02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2434be6e74b23fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.656+0000 7f14e7400640 1 -- 172.21.15.119:0/4286892461 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc037070 con 0x7f14e8144fd0 2024-04-25T10:08:54.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.657+0000 7f14e7400640 1 -- 172.21.15.119:0/4286892461 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc032440 con 0x7f14e8144fd0 2024-04-25T10:08:54.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.657+0000 7f14e7400640 1 -- 172.21.15.119:0/4286892461 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0327f0 con 0x7f14e8144fd0 2024-04-25T10:08:54.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.657+0000 7f14ee400640 1 -- 172.21.15.119:0/4286892461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e8144fd0 msgr2=0x7f14e815f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.657+0000 7f14ee400640 1 --2- 172.21.15.119:0/4286892461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e8144fd0 0x7f14e815f400 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009870 tx=0x7f14dc02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.657+0000 7f14ee400640 1 -- 172.21.15.119:0/4286892461 shutdown_connections 2024-04-25T10:08:54.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.657+0000 7f14ee400640 1 --2- 172.21.15.119:0/4286892461 >> v2:172.21.15.132:3300/0 conn(0x7f14e80cf180 0x7f14e80ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.657+0000 7f14ee400640 1 --2- 172.21.15.119:0/4286892461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e8144fd0 0x7f14e815f400 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.657+0000 7f14ee400640 1 -- 172.21.15.119:0/4286892461 >> 172.21.15.119:0/4286892461 conn(0x7f14e806aba0 msgr2=0x7f14e806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.657+0000 7f14ee400640 1 -- 172.21.15.119:0/4286892461 shutdown_connections 2024-04-25T10:08:54.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.658+0000 7f14ee400640 1 -- 172.21.15.119:0/4286892461 wait complete. 2024-04-25T10:08:54.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.659+0000 7f14ee400640 1 Processor -- start 2024-04-25T10:08:54.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.659+0000 7f14ee400640 1 -- start start 2024-04-25T10:08:54.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.660+0000 7f14ee400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e806e670 0x7f14e813b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.660+0000 7f14ee400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e80cf180 0x7f14e81f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.660+0000 7f14ee400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14e8144fd0 0x7f14e815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.660+0000 7f14ee400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e815f940 con 0x7f14e80cf180 2024-04-25T10:08:54.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.660+0000 7f14ee400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e806d190 con 0x7f14e8144fd0 2024-04-25T10:08:54.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.660+0000 7f14ee400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e81453b0 con 0x7f14e806e670 2024-04-25T10:08:54.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.661+0000 7f14e7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e80cf180 0x7f14e81f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.661+0000 7f14e7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e80cf180 0x7f14e81f1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38388/0 (socket says 172.21.15.119:38388) 2024-04-25T10:08:54.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.661+0000 7f14e7e00640 1 -- 172.21.15.119:0/305157537 learned_addr learned my addr 172.21.15.119:0/305157537 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:54.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.661+0000 7f14ed000640 1 --2- 172.21.15.119:0/305157537 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e806e670 0x7f14e813b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.661+0000 7f14eda00640 1 --2- 172.21.15.119:0/305157537 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14e8144fd0 0x7f14e815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.661+0000 7f14e7e00640 1 -- 172.21.15.119:0/305157537 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e806e670 msgr2=0x7f14e813b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.661+0000 7f14e7e00640 1 --2- 172.21.15.119:0/305157537 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e806e670 0x7f14e813b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.661+0000 7f14e7e00640 1 -- 172.21.15.119:0/305157537 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14e8144fd0 msgr2=0x7f14e815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.661+0000 7f14e7e00640 1 --2- 172.21.15.119:0/305157537 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14e8144fd0 0x7f14e815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.661+0000 7f14e7e00640 1 -- 172.21.15.119:0/305157537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e815bc00 con 0x7f14e80cf180 2024-04-25T10:08:54.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.662+0000 7f14e7e00640 1 --2- 172.21.15.119:0/305157537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e80cf180 0x7f14e81f1d90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f14d400e730 tx=0x7f14d400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.662+0000 7f14e4c00640 1 -- 172.21.15.119:0/305157537 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400f040 con 0x7f14e80cf180 2024-04-25T10:08:54.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.662+0000 7f14e4c00640 1 -- 172.21.15.119:0/305157537 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d4004230 con 0x7f14e80cf180 2024-04-25T10:08:54.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.662+0000 7f14e4c00640 1 -- 172.21.15.119:0/305157537 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400ee40 con 0x7f14e80cf180 2024-04-25T10:08:54.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.662+0000 7f14ee400640 1 -- 172.21.15.119:0/305157537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e81e4870 con 0x7f14e80cf180 2024-04-25T10:08:54.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.662+0000 7f14ee400640 1 -- 172.21.15.119:0/305157537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e813c620 con 0x7f14e80cf180 2024-04-25T10:08:54.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.663+0000 7f14ee400640 1 -- 172.21.15.119:0/305157537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e80749b0 con 0x7f14e80cf180 2024-04-25T10:08:54.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.664+0000 7f14e4c00640 1 -- 172.21.15.119:0/305157537 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14d400c6d0 con 0x7f14e80cf180 2024-04-25T10:08:54.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.665+0000 7f14e4c00640 1 --2- 172.21.15.119:0/305157537 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f14c806b500 0x7f14c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.665+0000 7f14e4c00640 1 -- 172.21.15.119:0/305157537 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f14d4089b90 con 0x7f14e80cf180 2024-04-25T10:08:54.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.665+0000 7f14ed000640 1 --2- 172.21.15.119:0/305157537 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f14c806b500 0x7f14c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.666+0000 7f14ed000640 1 --2- 172.21.15.119:0/305157537 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f14c806b500 0x7f14c806d9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f14dc034000 tx=0x7f14dc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.669+0000 7f14e4c00640 1 -- 172.21.15.119:0/305157537 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14d4017020 con 0x7f14e80cf180 2024-04-25T10:08:54.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:54.866+0000 7f14ee400640 1 -- 172.21.15.119:0/305157537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f14e80cef60 con 0x7f14e80cf180 2024-04-25T10:08:55.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.628+0000 7f14e4c00640 1 -- 172.21.15.119:0/305157537 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7f14d4058d50 con 0x7f14e80cf180 2024-04-25T10:08:55.635 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2024-04-25T10:08:55.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.633+0000 7f14ee400640 1 -- 172.21.15.119:0/305157537 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f14c806b500 msgr2=0x7f14c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.633+0000 7f14ee400640 1 --2- 172.21.15.119:0/305157537 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f14c806b500 0x7f14c806d9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f14dc034000 tx=0x7f14dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:08:55.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.633+0000 7f14ee400640 1 -- 172.21.15.119:0/305157537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e80cf180 msgr2=0x7f14e81f1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.633+0000 7f14ee400640 1 --2- 172.21.15.119:0/305157537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e80cf180 0x7f14e81f1d90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f14d400e730 tx=0x7f14d400ebf0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.633+0000 7f14ee400640 1 -- 172.21.15.119:0/305157537 shutdown_connections 2024-04-25T10:08:55.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.633+0000 7f14ee400640 1 --2- 172.21.15.119:0/305157537 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f14c806b500 0x7f14c806d9c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.633+0000 7f14ee400640 1 --2- 172.21.15.119:0/305157537 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14e8144fd0 0x7f14e815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.633+0000 7f14ee400640 1 --2- 172.21.15.119:0/305157537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14e80cf180 0x7f14e81f1d90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.633+0000 7f14ee400640 1 --2- 172.21.15.119:0/305157537 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14e806e670 0x7f14e813b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.633+0000 7f14ee400640 1 -- 172.21.15.119:0/305157537 >> 172.21.15.119:0/305157537 conn(0x7f14e806aba0 msgr2=0x7f14e8144140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.634+0000 7f14ee400640 1 -- 172.21.15.119:0/305157537 shutdown_connections 2024-04-25T10:08:55.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.634+0000 7f14ee400640 1 -- 172.21.15.119:0/305157537 wait complete. 2024-04-25T10:08:55.669 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:08:55.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.858+0000 7f23d5c00640 1 Processor -- start 2024-04-25T10:08:55.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.858+0000 7f23d5c00640 1 -- start start 2024-04-25T10:08:55.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.859+0000 7f23d5c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f23d00cf5b0 0x7f23d00c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.859+0000 7f23d5c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d009a650 0x7f23d01638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.859+0000 7f23d5c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d00c4fa0 con 0x7f23d00c4680 2024-04-25T10:08:55.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.859+0000 7f23d5c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d009a1c0 con 0x7f23d00cf5b0 2024-04-25T10:08:55.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.859+0000 7f23d5c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d00b2b10 con 0x7f23d009a650 2024-04-25T10:08:55.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.860+0000 7f23cfe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f23d00c4680 0x7f23d0162fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45742/0 (socket says 172.21.15.119:45742) 2024-04-25T10:08:55.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.860+0000 7f23cfe00640 1 -- 172.21.15.119:0/1610860557 learned_addr learned my addr 172.21.15.119:0/1610860557 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:55.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.860+0000 7f23d5200640 1 --2- 172.21.15.119:0/1610860557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d009a650 0x7f23d01638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.860+0000 7f23cf400640 1 --2- 172.21.15.119:0/1610860557 >> v2:172.21.15.132:3300/0 conn(0x7f23d00cf5b0 0x7f23d00c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.860+0000 7f23cea00640 1 -- 172.21.15.119:0/1610860557 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748525956 0 0) 0x7f23d00c4fa0 con 0x7f23d00c4680 2024-04-25T10:08:55.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.860+0000 7f23cea00640 1 -- 172.21.15.119:0/1610860557 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23b0003520 con 0x7f23d00c4680 2024-04-25T10:08:55.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23cea00640 1 -- 172.21.15.119:0/1610860557 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4027193663 0 0) 0x7f23b0003520 con 0x7f23d00c4680 2024-04-25T10:08:55.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23cea00640 1 -- 172.21.15.119:0/1610860557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d009a650 msgr2=0x7f23d01638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23cea00640 1 --2- 172.21.15.119:0/1610860557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d009a650 0x7f23d01638f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23cea00640 1 -- 172.21.15.119:0/1610860557 >> v2:172.21.15.132:3300/0 conn(0x7f23d00cf5b0 msgr2=0x7f23d00c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23cea00640 1 --2- 172.21.15.119:0/1610860557 >> v2:172.21.15.132:3300/0 conn(0x7f23d00cf5b0 0x7f23d00c4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23cea00640 1 -- 172.21.15.119:0/1610860557 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d00cfab0 con 0x7f23d00c4680 2024-04-25T10:08:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23cea00640 1 -- 172.21.15.119:0/1610860557 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f23c0001670 con 0x7f23d00c4680 2024-04-25T10:08:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23cea00640 1 -- 172.21.15.119:0/1610860557 >> v1:172.21.15.119:6789/0 conn(0x7f23d00c4680 legacy=0x7f23d0162fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23cea00640 1 --2- 172.21.15.119:0/1610860557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d009a650 0x7f23b00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23cea00640 1 --2- 172.21.15.119:0/1610860557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23b0003fb0 0x7f23b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23cea00640 1 --2- 172.21.15.119:0/1610860557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b0004d40 0x7f23b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23cea00640 1 -- 172.21.15.119:0/1610860557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d00b2b10 con 0x7f23b0004d40 2024-04-25T10:08:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23cea00640 1 -- 172.21.15.119:0/1610860557 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b000ba20 con 0x7f23b0003fb0 2024-04-25T10:08:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23cea00640 1 -- 172.21.15.119:0/1610860557 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b000bc20 con 0x7f23d009a650 2024-04-25T10:08:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.861+0000 7f23d5200640 1 --2- 172.21.15.119:0/1610860557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b0004d40 0x7f23b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.862+0000 7f23cfe00640 1 --2- 172.21.15.119:0/1610860557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d009a650 0x7f23b00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.862+0000 7f23d5200640 1 -- 172.21.15.119:0/1610860557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d009a650 msgr2=0x7f23b00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.862+0000 7f23d5200640 1 --2- 172.21.15.119:0/1610860557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d009a650 0x7f23b00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.862+0000 7f23d5200640 1 -- 172.21.15.119:0/1610860557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23b0003fb0 msgr2=0x7f23b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.862+0000 7f23d5200640 1 --2- 172.21.15.119:0/1610860557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23b0003fb0 0x7f23b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.862+0000 7f23d5200640 1 -- 172.21.15.119:0/1610860557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b0003520 con 0x7f23b0004d40 2024-04-25T10:08:55.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.862+0000 7f23cfe00640 1 --2- 172.21.15.119:0/1610860557 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d009a650 0x7f23b00039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:55.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.863+0000 7f23d5200640 1 --2- 172.21.15.119:0/1610860557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b0004d40 0x7f23b00051c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009ac0 tx=0x7f23b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcb96516ac5da55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.863+0000 7f23cea00640 1 -- 172.21.15.119:0/1610860557 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8036070 con 0x7f23b0004d40 2024-04-25T10:08:55.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.863+0000 7f23cea00640 1 -- 172.21.15.119:0/1610860557 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c8031400 con 0x7f23b0004d40 2024-04-25T10:08:55.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.863+0000 7f23cea00640 1 -- 172.21.15.119:0/1610860557 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8009690 con 0x7f23b0004d40 2024-04-25T10:08:55.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.864+0000 7f23d5c00640 1 -- 172.21.15.119:0/1610860557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b0004d40 msgr2=0x7f23b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.864+0000 7f23d5c00640 1 --2- 172.21.15.119:0/1610860557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23b0004d40 0x7f23b00051c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009ac0 tx=0x7f23b0005830 comp rx=0 tx=0).stop 2024-04-25T10:08:55.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.864+0000 7f23d5200640 1 -- 172.21.15.119:0/1610860557 reap_dead start 2024-04-25T10:08:55.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.864+0000 7f23d5c00640 1 -- 172.21.15.119:0/1610860557 shutdown_connections 2024-04-25T10:08:55.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.864+0000 7f23d5c00640 1 -- 172.21.15.119:0/1610860557 >> 172.21.15.119:0/1610860557 conn(0x7f23d00b31a0 msgr2=0x7f23d00b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.864+0000 7f23d5c00640 1 -- 172.21.15.119:0/1610860557 shutdown_connections 2024-04-25T10:08:55.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.864+0000 7f23d5c00640 1 -- 172.21.15.119:0/1610860557 wait complete. 2024-04-25T10:08:55.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.866+0000 7f23d5c00640 1 Processor -- start 2024-04-25T10:08:55.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.866+0000 7f23d5c00640 1 -- start start 2024-04-25T10:08:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.867+0000 7f23d5c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23b0004d40 0x7f23d014d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.867+0000 7f23d5c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d00b42d0 0x7f23d00b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.867+0000 7f23d5c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d00c5190 0x7f23d00c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.867+0000 7f23d5c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d00cfd90 con 0x7f23d00b42d0 2024-04-25T10:08:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.867+0000 7f23d5c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0099e50 con 0x7f23b0004d40 2024-04-25T10:08:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.867+0000 7f23d5c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d00b1e50 con 0x7f23d00c5190 2024-04-25T10:08:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.867+0000 7f23cf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d00b42d0 0x7f23d00b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.867+0000 7f23cf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d00b42d0 0x7f23d00b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38406/0 (socket says 172.21.15.119:38406) 2024-04-25T10:08:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.867+0000 7f23cf400640 1 -- 172.21.15.119:0/3065855826 learned_addr learned my addr 172.21.15.119:0/3065855826 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.868+0000 7f23d5200640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d00c5190 0x7f23d00c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.868+0000 7f23cfe00640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23b0004d40 0x7f23d014d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.868+0000 7f23cf400640 1 -- 172.21.15.119:0/3065855826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d00c5190 msgr2=0x7f23d00c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.868+0000 7f23cf400640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d00c5190 0x7f23d00c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.868+0000 7f23cf400640 1 -- 172.21.15.119:0/3065855826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23b0004d40 msgr2=0x7f23d014d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.868+0000 7f23cf400640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23b0004d40 0x7f23d014d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.868+0000 7f23cf400640 1 -- 172.21.15.119:0/3065855826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d00b4dd0 con 0x7f23d00b42d0 2024-04-25T10:08:55.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.868+0000 7f23d5200640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d00c5190 0x7f23d00c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:55.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.868+0000 7f23cf400640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d00b42d0 0x7f23d00b4730 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009d20 tx=0x7f23b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.868+0000 7f23c7e00640 1 -- 172.21.15.119:0/3065855826 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8003990 con 0x7f23d00b42d0 2024-04-25T10:08:55.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.868+0000 7f23c7e00640 1 -- 172.21.15.119:0/3065855826 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b8019070 con 0x7f23d00b42d0 2024-04-25T10:08:55.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.868+0000 7f23c7e00640 1 -- 172.21.15.119:0/3065855826 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8003d90 con 0x7f23d00b42d0 2024-04-25T10:08:55.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.868+0000 7f23d5c00640 1 -- 172.21.15.119:0/3065855826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d00b50b0 con 0x7f23d00b42d0 2024-04-25T10:08:55.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.869+0000 7f23d5c00640 1 -- 172.21.15.119:0/3065855826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d00b5670 con 0x7f23d00b42d0 2024-04-25T10:08:55.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.870+0000 7f23d5c00640 1 -- 172.21.15.119:0/3065855826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d00749b0 con 0x7f23d00b42d0 2024-04-25T10:08:55.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.871+0000 7f23c7e00640 1 -- 172.21.15.119:0/3065855826 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23b8005d10 con 0x7f23d00b42d0 2024-04-25T10:08:55.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.873+0000 7f23c7e00640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f23ac06b4d0 0x7f23ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.873+0000 7f23c7e00640 1 -- 172.21.15.119:0/3065855826 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f23b808b980 con 0x7f23d00b42d0 2024-04-25T10:08:55.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.873+0000 7f23cfe00640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f23ac06b4d0 0x7f23ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.874+0000 7f23cfe00640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f23ac06b4d0 0x7f23ac06d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f23c00079d0 tx=0x7f23c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:55.877+0000 7f23c7e00640 1 -- 172.21.15.119:0/3065855826 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f23b8023020 con 0x7f23d00b42d0 2024-04-25T10:08:56.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.075+0000 7f23d5c00640 1 -- 172.21.15.119:0/3065855826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f23d00b40b0 con 0x7f23d00b42d0 2024-04-25T10:08:56.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.644+0000 7f23c7e00640 1 -- 172.21.15.119:0/3065855826 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v42) v1 ==== 170+0+0 (secure 0 0 0) 0x7f23b8010040 con 0x7f23d00b42d0 2024-04-25T10:08:56.651 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2024-04-25T10:08:56.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.648+0000 7f23d5c00640 1 -- 172.21.15.119:0/3065855826 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f23ac06b4d0 msgr2=0x7f23ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.648+0000 7f23d5c00640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f23ac06b4d0 0x7f23ac06d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f23c00079d0 tx=0x7f23c0009040 comp rx=0 tx=0).stop 2024-04-25T10:08:56.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.648+0000 7f23d5c00640 1 -- 172.21.15.119:0/3065855826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d00b42d0 msgr2=0x7f23d00b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.648+0000 7f23d5c00640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d00b42d0 0x7f23d00b4730 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009d20 tx=0x7f23b800c490 comp rx=0 tx=0).stop 2024-04-25T10:08:56.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.648+0000 7f23d5c00640 1 -- 172.21.15.119:0/3065855826 shutdown_connections 2024-04-25T10:08:56.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.648+0000 7f23d5c00640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f23ac06b4d0 0x7f23ac06d990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.648+0000 7f23d5c00640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d00c5190 0x7f23d00c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.648+0000 7f23d5c00640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d00b42d0 0x7f23d00b4730 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.648+0000 7f23d5c00640 1 --2- 172.21.15.119:0/3065855826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23b0004d40 0x7f23d014d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.649+0000 7f23d5c00640 1 -- 172.21.15.119:0/3065855826 >> 172.21.15.119:0/3065855826 conn(0x7f23d00b31a0 msgr2=0x7f23d0160980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:56.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.649+0000 7f23d5c00640 1 -- 172.21.15.119:0/3065855826 shutdown_connections 2024-04-25T10:08:56.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.649+0000 7f23d5c00640 1 -- 172.21.15.119:0/3065855826 wait complete. 2024-04-25T10:08:56.690 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:08:56.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.859+0000 7f279ee00640 1 Processor -- start 2024-04-25T10:08:56.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.859+0000 7f279ee00640 1 -- start start 2024-04-25T10:08:56.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.861+0000 7f279ee00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f27980cf900 0x7f27980b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.861+0000 7f279ee00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f27980b2b90 0x7f2798162f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.861+0000 7f279ee00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2798071350 con 0x7f27980cfce0 2024-04-25T10:08:56.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.861+0000 7f279ee00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2798071630 con 0x7f27980cf900 2024-04-25T10:08:56.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.861+0000 7f279ee00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2798071910 con 0x7f27980b2b90 2024-04-25T10:08:56.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.861+0000 7f2797e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f27980cf900 0x7f27980b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.861+0000 7f2797e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f27980cf900 0x7f27980b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38412/0 (socket says 172.21.15.119:38412) 2024-04-25T10:08:56.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.861+0000 7f278f400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f27980cfce0 0x7f2798162880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45748/0 (socket says 172.21.15.119:45748) 2024-04-25T10:08:56.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.861+0000 7f2797e00640 1 -- 172.21.15.119:0/2021727534 learned_addr learned my addr 172.21.15.119:0/2021727534 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:56.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.862+0000 7f279ce00640 1 --2- 172.21.15.119:0/2021727534 >> v2:172.21.15.132:3300/0 conn(0x7f27980b2b90 0x7f2798162f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.862+0000 7f2797e00640 1 -- 172.21.15.119:0/2021727534 >> v2:172.21.15.132:3300/0 conn(0x7f27980b2b90 msgr2=0x7f2798162f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:56.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.862+0000 7f2797e00640 1 --2- 172.21.15.119:0/2021727534 >> v2:172.21.15.132:3300/0 conn(0x7f27980b2b90 0x7f2798162f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.862+0000 7f2797e00640 1 -- 172.21.15.119:0/2021727534 >> v1:172.21.15.119:6789/0 conn(0x7f27980cfce0 legacy=0x7f2798162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.862+0000 7f2797e00640 1 -- 172.21.15.119:0/2021727534 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2798071bf0 con 0x7f27980cf900 2024-04-25T10:08:56.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.862+0000 7f2797400640 1 -- 172.21.15.119:0/2021727534 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288064104 0 0) 0x7f2798071350 con 0x7f27980cfce0 2024-04-25T10:08:56.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.862+0000 7f279ce00640 1 --2- 172.21.15.119:0/2021727534 >> v2:172.21.15.132:3300/0 conn(0x7f27980b2b90 0x7f2798162f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:56.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.863+0000 7f2797e00640 1 --2- 172.21.15.119:0/2021727534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27980cf900 0x7f27980b2650 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f27980d0450 tx=0x7f278802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820d12fcc881036e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:56.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.863+0000 7f2797400640 1 -- 172.21.15.119:0/2021727534 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788037070 con 0x7f27980cf900 2024-04-25T10:08:56.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.863+0000 7f2797400640 1 -- 172.21.15.119:0/2021727534 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2788032440 con 0x7f27980cf900 2024-04-25T10:08:56.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.863+0000 7f2797400640 1 -- 172.21.15.119:0/2021727534 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788032810 con 0x7f27980cf900 2024-04-25T10:08:56.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.864+0000 7f279ee00640 1 -- 172.21.15.119:0/2021727534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27980cf900 msgr2=0x7f27980b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.864+0000 7f279ee00640 1 --2- 172.21.15.119:0/2021727534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27980cf900 0x7f27980b2650 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f27980d0450 tx=0x7f278802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.864+0000 7f279ee00640 1 -- 172.21.15.119:0/2021727534 shutdown_connections 2024-04-25T10:08:56.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.864+0000 7f279ee00640 1 --2- 172.21.15.119:0/2021727534 >> v2:172.21.15.132:3300/0 conn(0x7f27980b2b90 0x7f2798162f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.864+0000 7f279ee00640 1 --2- 172.21.15.119:0/2021727534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27980cf900 0x7f27980b2650 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.864+0000 7f279ee00640 1 -- 172.21.15.119:0/2021727534 >> 172.21.15.119:0/2021727534 conn(0x7f279806aba0 msgr2=0x7f279806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:56.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.864+0000 7f279ee00640 1 -- 172.21.15.119:0/2021727534 shutdown_connections 2024-04-25T10:08:56.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.864+0000 7f279ee00640 1 -- 172.21.15.119:0/2021727534 wait complete. 2024-04-25T10:08:56.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.866+0000 7f279ee00640 1 Processor -- start 2024-04-25T10:08:56.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.866+0000 7f279ee00640 1 -- start start 2024-04-25T10:08:56.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f279ee00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27980b2b90 0x7f27980ad050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f279ee00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27980cf900 0x7f2798152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f279ee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27980cfce0 0x7f2798152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f279ee00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2798165c30 con 0x7f27980b2b90 2024-04-25T10:08:56.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f279ee00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980b3e00 con 0x7f27980cf900 2024-04-25T10:08:56.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f279ee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2798165db0 con 0x7f27980cfce0 2024-04-25T10:08:56.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f2797e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27980b2b90 0x7f27980ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f2797e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27980b2b90 0x7f27980ad050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38428/0 (socket says 172.21.15.119:38428) 2024-04-25T10:08:56.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f2797e00640 1 -- 172.21.15.119:0/2638952051 learned_addr learned my addr 172.21.15.119:0/2638952051 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:56.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f278f400640 1 --2- 172.21.15.119:0/2638952051 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27980cf900 0x7f2798152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f2797e00640 1 -- 172.21.15.119:0/2638952051 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27980cfce0 msgr2=0x7f2798152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f2797e00640 1 --2- 172.21.15.119:0/2638952051 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27980cfce0 0x7f2798152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f2797e00640 1 -- 172.21.15.119:0/2638952051 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27980cf900 msgr2=0x7f2798152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f2797e00640 1 --2- 172.21.15.119:0/2638952051 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27980cf900 0x7f2798152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f2797e00640 1 -- 172.21.15.119:0/2638952051 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27980a9420 con 0x7f27980b2b90 2024-04-25T10:08:56.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.867+0000 7f278f400640 1 --2- 172.21.15.119:0/2638952051 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27980cf900 0x7f2798152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:56.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.868+0000 7f2797e00640 1 --2- 172.21.15.119:0/2638952051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27980b2b90 0x7f27980ad050 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f278802a220 tx=0x7f27880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:56.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.868+0000 7f2794c00640 1 -- 172.21.15.119:0/2638952051 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788037040 con 0x7f27980b2b90 2024-04-25T10:08:56.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.868+0000 7f2794c00640 1 -- 172.21.15.119:0/2638952051 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2788042070 con 0x7f27980b2b90 2024-04-25T10:08:56.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.868+0000 7f2794c00640 1 -- 172.21.15.119:0/2638952051 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788030bb0 con 0x7f27980b2b90 2024-04-25T10:08:56.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.868+0000 7f279ee00640 1 -- 172.21.15.119:0/2638952051 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27980a9700 con 0x7f27980b2b90 2024-04-25T10:08:56.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.868+0000 7f279ee00640 1 -- 172.21.15.119:0/2638952051 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27980a9cc0 con 0x7f27980b2b90 2024-04-25T10:08:56.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.870+0000 7f279ee00640 1 -- 172.21.15.119:0/2638952051 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2798067640 con 0x7f27980b2b90 2024-04-25T10:08:56.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.871+0000 7f2794c00640 1 -- 172.21.15.119:0/2638952051 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2788047460 con 0x7f27980b2b90 2024-04-25T10:08:56.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.872+0000 7f2794c00640 1 --2- 172.21.15.119:0/2638952051 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f277406b500 0x7f277406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.872+0000 7f2794c00640 1 -- 172.21.15.119:0/2638952051 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f27880aaec0 con 0x7f27980b2b90 2024-04-25T10:08:56.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.872+0000 7f278f400640 1 --2- 172.21.15.119:0/2638952051 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f277406b500 0x7f277406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.873+0000 7f278f400640 1 --2- 172.21.15.119:0/2638952051 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f277406b500 0x7f277406d9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2798165390 tx=0x7f2780009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:56.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:56.876+0000 7f2794c00640 1 -- 172.21.15.119:0/2638952051 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2798067640 con 0x7f27980b2b90 2024-04-25T10:08:57.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.118+0000 7f279ee00640 1 -- 172.21.15.119:0/2638952051 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f27980750b0 con 0x7f27980b2b90 2024-04-25T10:08:57.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.658+0000 7f2794c00640 1 -- 172.21.15.119:0/2638952051 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2788045360 con 0x7f27980b2b90 2024-04-25T10:08:57.667 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 6 and data pool 7 2024-04-25T10:08:57.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.670+0000 7f279ee00640 1 -- 172.21.15.119:0/2638952051 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f277406b500 msgr2=0x7f277406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.670+0000 7f279ee00640 1 --2- 172.21.15.119:0/2638952051 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f277406b500 0x7f277406d9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2798165390 tx=0x7f2780009040 comp rx=0 tx=0).stop 2024-04-25T10:08:57.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.670+0000 7f279ee00640 1 -- 172.21.15.119:0/2638952051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27980b2b90 msgr2=0x7f27980ad050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.670+0000 7f279ee00640 1 --2- 172.21.15.119:0/2638952051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27980b2b90 0x7f27980ad050 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f278802a220 tx=0x7f27880046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.674+0000 7f279ee00640 1 -- 172.21.15.119:0/2638952051 shutdown_connections 2024-04-25T10:08:57.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.674+0000 7f279ee00640 1 --2- 172.21.15.119:0/2638952051 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f277406b500 0x7f277406d9c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.674+0000 7f279ee00640 1 --2- 172.21.15.119:0/2638952051 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27980cfce0 0x7f2798152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.674+0000 7f279ee00640 1 --2- 172.21.15.119:0/2638952051 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27980cf900 0x7f2798152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.674+0000 7f279ee00640 1 --2- 172.21.15.119:0/2638952051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27980b2b90 0x7f27980ad050 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.674+0000 7f279ee00640 1 -- 172.21.15.119:0/2638952051 >> 172.21.15.119:0/2638952051 conn(0x7f279806aba0 msgr2=0x7f27980cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:57.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.676+0000 7f279ee00640 1 -- 172.21.15.119:0/2638952051 shutdown_connections 2024-04-25T10:08:57.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.676+0000 7f279ee00640 1 -- 172.21.15.119:0/2638952051 wait complete. 2024-04-25T10:08:57.714 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:57.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.874+0000 7fa452a00640 1 Processor -- start 2024-04-25T10:08:57.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.874+0000 7fa452a00640 1 -- start start 2024-04-25T10:08:57.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.876+0000 7fa452a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa44c06e8f0 0x7fa44c15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.876+0000 7fa452a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa44c15c820 0x7fa44c1654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.876+0000 7fa452a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44c0ce6c0 con 0x7fa44c15bf00 2024-04-25T10:08:57.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.876+0000 7fa452a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44c1455e0 con 0x7fa44c15c820 2024-04-25T10:08:57.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.876+0000 7fa452a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44c06d450 con 0x7fa44c06e8f0 2024-04-25T10:08:57.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.876+0000 7fa44be00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa44c15bf00 0x7fa44c0cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45762/0 (socket says 172.21.15.119:45762) 2024-04-25T10:08:57.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.876+0000 7fa44be00640 1 -- 172.21.15.119:0/1862460475 learned_addr learned my addr 172.21.15.119:0/1862460475 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:57.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.876+0000 7fa450a00640 1 --2- 172.21.15.119:0/1862460475 >> v2:172.21.15.119:3300/0 conn(0x7fa44c15c820 0x7fa44c1654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.877+0000 7fa44b400640 1 --2- 172.21.15.119:0/1862460475 >> v2:172.21.15.132:3300/0 conn(0x7fa44c06e8f0 0x7fa44c15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.877+0000 7fa44aa00640 1 -- 172.21.15.119:0/1862460475 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096877765 0 0) 0x7fa43c001110 con 0x7fa44c15bf00 2024-04-25T10:08:57.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.877+0000 7fa44aa00640 1 -- 172.21.15.119:0/1862460475 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa430003520 con 0x7fa44c15bf00 2024-04-25T10:08:57.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.877+0000 7fa450a00640 1 -- 172.21.15.119:0/1862460475 >> v2:172.21.15.132:3300/0 conn(0x7fa44c06e8f0 msgr2=0x7fa44c15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:57.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.877+0000 7fa450a00640 1 --2- 172.21.15.119:0/1862460475 >> v2:172.21.15.132:3300/0 conn(0x7fa44c06e8f0 0x7fa44c15c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.877+0000 7fa450a00640 1 -- 172.21.15.119:0/1862460475 >> v1:172.21.15.119:6789/0 conn(0x7fa44c15bf00 legacy=0x7fa44c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.877+0000 7fa450a00640 1 -- 172.21.15.119:0/1862460475 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c0cc4e0 con 0x7fa44c15c820 2024-04-25T10:08:57.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.878+0000 7fa450a00640 1 --2- 172.21.15.119:0/1862460475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c15c820 0x7fa44c1654b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa44c06b900 tx=0x7fa44402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631a3ece13bb3f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:57.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.878+0000 7fa44aa00640 1 -- 172.21.15.119:0/1862460475 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444007740 con 0x7fa44c15c820 2024-04-25T10:08:57.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.878+0000 7fa44aa00640 1 -- 172.21.15.119:0/1862460475 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4440078e0 con 0x7fa44c15c820 2024-04-25T10:08:57.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.878+0000 7fa44aa00640 1 -- 172.21.15.119:0/1862460475 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444007cb0 con 0x7fa44c15c820 2024-04-25T10:08:57.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.879+0000 7fa452a00640 1 -- 172.21.15.119:0/1862460475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c15c820 msgr2=0x7fa44c1654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.879+0000 7fa452a00640 1 --2- 172.21.15.119:0/1862460475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c15c820 0x7fa44c1654b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa44c06b900 tx=0x7fa44402a680 comp rx=0 tx=0).stop 2024-04-25T10:08:57.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.879+0000 7fa452a00640 1 -- 172.21.15.119:0/1862460475 shutdown_connections 2024-04-25T10:08:57.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.879+0000 7fa452a00640 1 --2- 172.21.15.119:0/1862460475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c15c820 0x7fa44c1654b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.879+0000 7fa452a00640 1 --2- 172.21.15.119:0/1862460475 >> v2:172.21.15.132:3300/0 conn(0x7fa44c06e8f0 0x7fa44c15c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.879+0000 7fa452a00640 1 -- 172.21.15.119:0/1862460475 >> 172.21.15.119:0/1862460475 conn(0x7fa44c06aba0 msgr2=0x7fa44c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:57.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.879+0000 7fa452a00640 1 -- 172.21.15.119:0/1862460475 shutdown_connections 2024-04-25T10:08:57.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.879+0000 7fa452a00640 1 -- 172.21.15.119:0/1862460475 wait complete. 2024-04-25T10:08:57.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.881+0000 7fa452a00640 1 Processor -- start 2024-04-25T10:08:57.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.881+0000 7fa452a00640 1 -- start start 2024-04-25T10:08:57.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.882+0000 7fa452a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa44c06e8f0 0x7fa44c158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.882+0000 7fa452a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c15bf00 0x7fa44c1f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.882+0000 7fa452a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44c15c820 0x7fa44c13dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.882+0000 7fa452a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0ccd70 con 0x7fa44c15bf00 2024-04-25T10:08:57.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.882+0000 7fa452a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0cca90 con 0x7fa44c06e8f0 2024-04-25T10:08:57.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.882+0000 7fa452a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0cd050 con 0x7fa44c15c820 2024-04-25T10:08:57.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.882+0000 7fa44b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c15bf00 0x7fa44c1f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.882+0000 7fa44b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c15bf00 0x7fa44c1f2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38446/0 (socket says 172.21.15.119:38446) 2024-04-25T10:08:57.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.882+0000 7fa44b400640 1 -- 172.21.15.119:0/4292073288 learned_addr learned my addr 172.21.15.119:0/4292073288 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:57.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.882+0000 7fa44be00640 1 --2- 172.21.15.119:0/4292073288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa44c06e8f0 0x7fa44c158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.883+0000 7fa44b400640 1 -- 172.21.15.119:0/4292073288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44c15c820 msgr2=0x7fa44c13dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.883+0000 7fa44b400640 1 --2- 172.21.15.119:0/4292073288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44c15c820 0x7fa44c13dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.883+0000 7fa44b400640 1 -- 172.21.15.119:0/4292073288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa44c06e8f0 msgr2=0x7fa44c158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.883+0000 7fa44b400640 1 --2- 172.21.15.119:0/4292073288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa44c06e8f0 0x7fa44c158290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.883+0000 7fa44b400640 1 -- 172.21.15.119:0/4292073288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c1f2940 con 0x7fa44c15bf00 2024-04-25T10:08:57.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.883+0000 7fa44b400640 1 --2- 172.21.15.119:0/4292073288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c15bf00 0x7fa44c1f2400 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa43400d7d0 tx=0x7fa43400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:57.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.883+0000 7fa443e00640 1 -- 172.21.15.119:0/4292073288 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434004020 con 0x7fa44c15bf00 2024-04-25T10:08:57.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.883+0000 7fa443e00640 1 -- 172.21.15.119:0/4292073288 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4340041e0 con 0x7fa44c15bf00 2024-04-25T10:08:57.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.883+0000 7fa452a00640 1 -- 172.21.15.119:0/4292073288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44c141740 con 0x7fa44c15bf00 2024-04-25T10:08:57.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.883+0000 7fa443e00640 1 -- 172.21.15.119:0/4292073288 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4340043d0 con 0x7fa44c15bf00 2024-04-25T10:08:57.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.883+0000 7fa452a00640 1 -- 172.21.15.119:0/4292073288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c141d00 con 0x7fa44c15bf00 2024-04-25T10:08:57.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.884+0000 7fa452a00640 1 -- 172.21.15.119:0/4292073288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44c06bb30 con 0x7fa44c15bf00 2024-04-25T10:08:57.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.885+0000 7fa443e00640 1 -- 172.21.15.119:0/4292073288 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa43401b020 con 0x7fa44c15bf00 2024-04-25T10:08:57.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.886+0000 7fa443e00640 1 --2- 172.21.15.119:0/4292073288 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa42806b500 0x7fa42806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.886+0000 7fa443e00640 1 -- 172.21.15.119:0/4292073288 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa43408a0b0 con 0x7fa44c15bf00 2024-04-25T10:08:57.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.887+0000 7fa44be00640 1 --2- 172.21.15.119:0/4292073288 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa42806b500 0x7fa42806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.887+0000 7fa44be00640 1 --2- 172.21.15.119:0/4292073288 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa42806b500 0x7fa42806d9c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0012b0 tx=0x7fa43c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:57.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:57.891+0000 7fa443e00640 1 -- 172.21.15.119:0/4292073288 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa434058e90 con 0x7fa44c15bf00 2024-04-25T10:08:58.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.080+0000 7fa452a00640 1 -- 172.21.15.119:0/4292073288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa44c0749b0 con 0x7fa44c15bf00 2024-04-25T10:08:58.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.081+0000 7fa443e00640 1 -- 172.21.15.119:0/4292073288 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16662 (secure 0 0 0) 0x7fa434058790 con 0x7fa44c15bf00 2024-04-25T10:08:58.088 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:58.088 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":43,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:57.120107+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:54.917692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:56.128440+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:58.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.084+0000 7fa452a00640 1 -- 172.21.15.119:0/4292073288 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa42806b500 msgr2=0x7fa42806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.084+0000 7fa452a00640 1 --2- 172.21.15.119:0/4292073288 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa42806b500 0x7fa42806d9c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0012b0 tx=0x7fa43c009040 comp rx=0 tx=0).stop 2024-04-25T10:08:58.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.084+0000 7fa452a00640 1 -- 172.21.15.119:0/4292073288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c15bf00 msgr2=0x7fa44c1f2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.084+0000 7fa452a00640 1 --2- 172.21.15.119:0/4292073288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c15bf00 0x7fa44c1f2400 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa43400d7d0 tx=0x7fa43400dc90 comp rx=0 tx=0).stop 2024-04-25T10:08:58.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.084+0000 7fa452a00640 1 -- 172.21.15.119:0/4292073288 shutdown_connections 2024-04-25T10:08:58.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.084+0000 7fa452a00640 1 --2- 172.21.15.119:0/4292073288 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa42806b500 0x7fa42806d9c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.084+0000 7fa452a00640 1 --2- 172.21.15.119:0/4292073288 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44c15c820 0x7fa44c13dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.084+0000 7fa452a00640 1 --2- 172.21.15.119:0/4292073288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa44c15bf00 0x7fa44c1f2400 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.084+0000 7fa452a00640 1 --2- 172.21.15.119:0/4292073288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa44c06e8f0 0x7fa44c158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.084+0000 7fa452a00640 1 -- 172.21.15.119:0/4292073288 >> 172.21.15.119:0/4292073288 conn(0x7fa44c06aba0 msgr2=0x7fa44c15ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:58.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.084+0000 7fa452a00640 1 -- 172.21.15.119:0/4292073288 shutdown_connections 2024-04-25T10:08:58.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.084+0000 7fa452a00640 1 -- 172.21.15.119:0/4292073288 wait complete. 2024-04-25T10:08:58.107 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:58.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.291+0000 7fa5e0c00640 1 Processor -- start 2024-04-25T10:08:58.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.292+0000 7fa5e0c00640 1 -- start start 2024-04-25T10:08:58.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.293+0000 7fa5e0c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa5dc0cfc40 0x7fa5dc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.293+0000 7fa5e0c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5dc0cdb10 con 0x7fa5dc06e2e0 2024-04-25T10:08:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.293+0000 7fa5e0c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5dc0cddf0 con 0x7fa5dc15a870 2024-04-25T10:08:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.293+0000 7fa5e0c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5dc0ce720 con 0x7fa5dc0cfc40 2024-04-25T10:08:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.294+0000 7fa5d1400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa5dc06e2e0 0x7fa5dc15bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45768/0 (socket says 172.21.15.119:45768) 2024-04-25T10:08:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.294+0000 7fa5d1400640 1 -- 172.21.15.119:0/3347435916 learned_addr learned my addr 172.21.15.119:0/3347435916 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:08:58.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.294+0000 7fa5d9400640 1 -- 172.21.15.119:0/3347435916 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128791192 0 0) 0x7fa5dc0cdb10 con 0x7fa5dc06e2e0 2024-04-25T10:08:58.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.294+0000 7fa5d9400640 1 -- 172.21.15.119:0/3347435916 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5c0003520 con 0x7fa5dc06e2e0 2024-04-25T10:08:58.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.294+0000 7fa5da800640 1 --2- 172.21.15.119:0/3347435916 >> v2:172.21.15.132:3300/0 conn(0x7fa5dc0cfc40 0x7fa5dc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.294+0000 7fa5d9400640 1 -- 172.21.15.119:0/3347435916 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3833709979 0 0) 0x7fa5c0003520 con 0x7fa5dc06e2e0 2024-04-25T10:08:58.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.294+0000 7fa5d9400640 1 -- 172.21.15.119:0/3347435916 >> v2:172.21.15.132:3300/0 conn(0x7fa5dc0cfc40 msgr2=0x7fa5dc0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:58.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.294+0000 7fa5d9400640 1 --2- 172.21.15.119:0/3347435916 >> v2:172.21.15.132:3300/0 conn(0x7fa5dc0cfc40 0x7fa5dc0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.294+0000 7fa5d9400640 1 -- 172.21.15.119:0/3347435916 >> v1:172.21.15.132:6789/0 conn(0x7fa5dc15a870 legacy=0x7fa5dc0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.294+0000 7fa5d9400640 1 -- 172.21.15.119:0/3347435916 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc0ce0d0 con 0x7fa5dc06e2e0 2024-04-25T10:08:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.294+0000 7fa5d9400640 1 -- 172.21.15.119:0/3347435916 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fa5c4001670 con 0x7fa5dc06e2e0 2024-04-25T10:08:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.294+0000 7fa5d9400640 1 -- 172.21.15.119:0/3347435916 >> v1:172.21.15.119:6789/0 conn(0x7fa5dc06e2e0 legacy=0x7fa5dc15bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.294+0000 7fa5d9400640 1 --2- 172.21.15.119:0/3347435916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c0003a30 0x7fa5c0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.295+0000 7fa5d9400640 1 --2- 172.21.15.119:0/3347435916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5c0004800 0x7fa5c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.295+0000 7fa5d9400640 1 --2- 172.21.15.119:0/3347435916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5c0005230 0x7fa5c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.295+0000 7fa5d9400640 1 -- 172.21.15.119:0/3347435916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0ce720 con 0x7fa5c0003a30 2024-04-25T10:08:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.295+0000 7fa5d9400640 1 -- 172.21.15.119:0/3347435916 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c000c190 con 0x7fa5c0004800 2024-04-25T10:08:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.295+0000 7fa5d9400640 1 -- 172.21.15.119:0/3347435916 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c000e5a0 con 0x7fa5c0005230 2024-04-25T10:08:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.295+0000 7fa5d9e00640 1 --2- 172.21.15.119:0/3347435916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c0003a30 0x7fa5c0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.295+0000 7fa5d1400640 1 --2- 172.21.15.119:0/3347435916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5c0004800 0x7fa5c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.295+0000 7fa5d9e00640 1 -- 172.21.15.119:0/3347435916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5c0005230 msgr2=0x7fa5c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.295+0000 7fa5d9e00640 1 --2- 172.21.15.119:0/3347435916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5c0005230 0x7fa5c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.295+0000 7fa5d9e00640 1 -- 172.21.15.119:0/3347435916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5c0004800 msgr2=0x7fa5c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:58.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.295+0000 7fa5d9e00640 1 --2- 172.21.15.119:0/3347435916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5c0004800 0x7fa5c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.295+0000 7fa5d9e00640 1 -- 172.21.15.119:0/3347435916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c0003520 con 0x7fa5c0003a30 2024-04-25T10:08:58.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.295+0000 7fa5da800640 1 -- 172.21.15.119:0/3347435916 reap_dead start 2024-04-25T10:08:58.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.295+0000 7fa5d1400640 1 --2- 172.21.15.119:0/3347435916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5c0004800 0x7fa5c0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:58.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.296+0000 7fa5d9e00640 1 --2- 172.21.15.119:0/3347435916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c0003a30 0x7fa5c0003e30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc003db0 tx=0x7fa5c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e035e7e46541a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:58.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.296+0000 7fa5d9400640 1 -- 172.21.15.119:0/3347435916 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc037070 con 0x7fa5c0003a30 2024-04-25T10:08:58.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.296+0000 7fa5d9400640 1 -- 172.21.15.119:0/3347435916 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc02a940 con 0x7fa5c0003a30 2024-04-25T10:08:58.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.297+0000 7fa5d9400640 1 -- 172.21.15.119:0/3347435916 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc003980 con 0x7fa5c0003a30 2024-04-25T10:08:58.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.297+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3347435916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c0003a30 msgr2=0x7fa5c0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.297+0000 7fa5e0c00640 1 --2- 172.21.15.119:0/3347435916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c0003a30 0x7fa5c0003e30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc003db0 tx=0x7fa5c0005830 comp rx=0 tx=0).stop 2024-04-25T10:08:58.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.297+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3347435916 shutdown_connections 2024-04-25T10:08:58.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.297+0000 7fa5e0c00640 1 --2- 172.21.15.119:0/3347435916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c0003a30 0x7fa5c0003e30 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.297+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3347435916 >> 172.21.15.119:0/3347435916 conn(0x7fa5dc06aba0 msgr2=0x7fa5dc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.297+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3347435916 shutdown_connections 2024-04-25T10:08:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.297+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3347435916 wait complete. 2024-04-25T10:08:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.299+0000 7fa5e0c00640 1 Processor -- start 2024-04-25T10:08:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.299+0000 7fa5e0c00640 1 -- start start 2024-04-25T10:08:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.300+0000 7fa5e0c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c0003a30 0x7fa5dc14fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.300+0000 7fa5e0c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5dc150210 0x7fa5dc15d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.300+0000 7fa5e0c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc151340 0x7fa5dc15c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.300+0000 7fa5e0c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0cd700 con 0x7fa5c0003a30 2024-04-25T10:08:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.300+0000 7fa5e0c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0cd580 con 0x7fa5dc150210 2024-04-25T10:08:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.300+0000 7fa5e0c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0cd880 con 0x7fa5dc151340 2024-04-25T10:08:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.300+0000 7fa5d9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c0003a30 0x7fa5dc14fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.300+0000 7fa5d9e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c0003a30 0x7fa5dc14fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38458/0 (socket says 172.21.15.119:38458) 2024-04-25T10:08:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.300+0000 7fa5d9e00640 1 -- 172.21.15.119:0/3322519979 learned_addr learned my addr 172.21.15.119:0/3322519979 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.301+0000 7fa5d9e00640 1 -- 172.21.15.119:0/3322519979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc151340 msgr2=0x7fa5dc15c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.301+0000 7fa5d1400640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5dc150210 0x7fa5dc15d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.301+0000 7fa5da800640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc151340 0x7fa5dc15c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.301+0000 7fa5d9e00640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc151340 0x7fa5dc15c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.301+0000 7fa5d9e00640 1 -- 172.21.15.119:0/3322519979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5dc150210 msgr2=0x7fa5dc15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.301+0000 7fa5d9e00640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5dc150210 0x7fa5dc15d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.301+0000 7fa5d9e00640 1 -- 172.21.15.119:0/3322519979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc15ba20 con 0x7fa5c0003a30 2024-04-25T10:08:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.301+0000 7fa5da800640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc151340 0x7fa5dc15c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.301+0000 7fa5d9e00640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c0003a30 0x7fa5dc14fa50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc035040 tx=0x7fa5cc0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.301+0000 7fa5d2a00640 1 -- 172.21.15.119:0/3322519979 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc037050 con 0x7fa5c0003a30 2024-04-25T10:08:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.301+0000 7fa5d2a00640 1 -- 172.21.15.119:0/3322519979 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc038040 con 0x7fa5c0003a30 2024-04-25T10:08:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.301+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3322519979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5dc14ff90 con 0x7fa5c0003a30 2024-04-25T10:08:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.301+0000 7fa5d2a00640 1 -- 172.21.15.119:0/3322519979 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc007a70 con 0x7fa5c0003a30 2024-04-25T10:08:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.302+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3322519979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5dc1599c0 con 0x7fa5c0003a30 2024-04-25T10:08:58.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.303+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3322519979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5dc159d50 con 0x7fa5c0003a30 2024-04-25T10:08:58.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.303+0000 7fa5d2a00640 1 -- 172.21.15.119:0/3322519979 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5cc045020 con 0x7fa5c0003a30 2024-04-25T10:08:58.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.304+0000 7fa5d2a00640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa5b806b500 0x7fa5b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.304+0000 7fa5d2a00640 1 -- 172.21.15.119:0/3322519979 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5cc0a9e50 con 0x7fa5c0003a30 2024-04-25T10:08:58.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.305+0000 7fa5d1400640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa5b806b500 0x7fa5b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.305+0000 7fa5d1400640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa5b806b500 0x7fa5b806d9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40017e0 tx=0x7fa5c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:58.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.309+0000 7fa5d2a00640 1 -- 172.21.15.119:0/3322519979 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa5cc078cc0 con 0x7fa5c0003a30 2024-04-25T10:08:58.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.501+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3322519979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5dc0749b0 con 0x7fa5c0003a30 2024-04-25T10:08:58.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.504+0000 7fa5d2a00640 1 -- 172.21.15.119:0/3322519979 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16662 (secure 0 0 0) 0x7fa5cc0785c0 con 0x7fa5c0003a30 2024-04-25T10:08:58.511 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:08:58.511 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":43,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:57.120107+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:54.917692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:56.128440+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:58.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.507+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3322519979 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa5b806b500 msgr2=0x7fa5b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.507+0000 7fa5e0c00640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa5b806b500 0x7fa5b806d9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40017e0 tx=0x7fa5c4009040 comp rx=0 tx=0).stop 2024-04-25T10:08:58.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.507+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3322519979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c0003a30 msgr2=0x7fa5dc14fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.507+0000 7fa5e0c00640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c0003a30 0x7fa5dc14fa50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc035040 tx=0x7fa5cc0036f0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.507+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3322519979 shutdown_connections 2024-04-25T10:08:58.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.507+0000 7fa5e0c00640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa5b806b500 0x7fa5b806d9c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.507+0000 7fa5e0c00640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc151340 0x7fa5dc15c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.507+0000 7fa5e0c00640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5dc150210 0x7fa5dc15d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.507+0000 7fa5e0c00640 1 --2- 172.21.15.119:0/3322519979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c0003a30 0x7fa5dc14fa50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.507+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3322519979 >> 172.21.15.119:0/3322519979 conn(0x7fa5dc06aba0 msgr2=0x7fa5dc0ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:58.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.508+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3322519979 shutdown_connections 2024-04-25T10:08:58.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.508+0000 7fa5e0c00640 1 -- 172.21.15.119:0/3322519979 wait complete. 2024-04-25T10:08:58.533 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:08:58.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.737+0000 7fbf5ca00640 1 Processor -- start 2024-04-25T10:08:58.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.738+0000 7fbf5ca00640 1 -- start start 2024-04-25T10:08:58.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.739+0000 7fbf5ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf580aed40 0x7fbf580cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.739+0000 7fbf5ca00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf580cfa90 0x7fbf580acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.739+0000 7fbf5ca00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5815e360 con 0x7fbf5815b720 2024-04-25T10:08:58.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.739+0000 7fbf5ca00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5815e5d0 con 0x7fbf580cfa90 2024-04-25T10:08:58.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.739+0000 7fbf5ca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5815e8b0 con 0x7fbf580aed40 2024-04-25T10:08:58.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.740+0000 7fbf56800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf580cfa90 0x7fbf580acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.740+0000 7fbf56800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf580cfa90 0x7fbf580acfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38462/0 (socket says 172.21.15.119:38462) 2024-04-25T10:08:58.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.740+0000 7fbf56800640 1 -- 172.21.15.119:0/437847253 learned_addr learned my addr 172.21.15.119:0/437847253 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:58.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.740+0000 7fbf4d400640 1 --2- 172.21.15.119:0/437847253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf580aed40 0x7fbf580cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.740+0000 7fbf56800640 1 -- 172.21.15.119:0/437847253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf580aed40 msgr2=0x7fbf580cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:58.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.740+0000 7fbf56800640 1 --2- 172.21.15.119:0/437847253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf580aed40 0x7fbf580cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.740+0000 7fbf56800640 1 -- 172.21.15.119:0/437847253 >> v1:172.21.15.132:6789/0 conn(0x7fbf5815b720 legacy=0x7fbf580ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.740+0000 7fbf56800640 1 -- 172.21.15.119:0/437847253 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5815eb90 con 0x7fbf580cfa90 2024-04-25T10:08:58.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.741+0000 7fbf56800640 1 --2- 172.21.15.119:0/437847253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf580cfa90 0x7fbf580acfc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009e50 tx=0x7fbf5002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d136927be6bf32ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:58.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.742+0000 7fbf55400640 1 -- 172.21.15.119:0/437847253 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50036070 con 0x7fbf580cfa90 2024-04-25T10:08:58.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.742+0000 7fbf55400640 1 -- 172.21.15.119:0/437847253 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf50031600 con 0x7fbf580cfa90 2024-04-25T10:08:58.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.742+0000 7fbf55400640 1 -- 172.21.15.119:0/437847253 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf500319d0 con 0x7fbf580cfa90 2024-04-25T10:08:58.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.742+0000 7fbf5ca00640 1 -- 172.21.15.119:0/437847253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf580cfa90 msgr2=0x7fbf580acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.742+0000 7fbf5ca00640 1 --2- 172.21.15.119:0/437847253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf580cfa90 0x7fbf580acfc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009e50 tx=0x7fbf5002a710 comp rx=0 tx=0).stop 2024-04-25T10:08:58.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.742+0000 7fbf5ca00640 1 -- 172.21.15.119:0/437847253 shutdown_connections 2024-04-25T10:08:58.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.742+0000 7fbf5ca00640 1 --2- 172.21.15.119:0/437847253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf580cfa90 0x7fbf580acfc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.742+0000 7fbf5ca00640 1 --2- 172.21.15.119:0/437847253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf580aed40 0x7fbf580cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.742+0000 7fbf5ca00640 1 -- 172.21.15.119:0/437847253 >> 172.21.15.119:0/437847253 conn(0x7fbf5806aba0 msgr2=0x7fbf5806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:58.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.742+0000 7fbf5ca00640 1 -- 172.21.15.119:0/437847253 shutdown_connections 2024-04-25T10:08:58.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.743+0000 7fbf5ca00640 1 -- 172.21.15.119:0/437847253 wait complete. 2024-04-25T10:08:58.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.744+0000 7fbf5ca00640 1 Processor -- start 2024-04-25T10:08:58.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.744+0000 7fbf5ca00640 1 -- start start 2024-04-25T10:08:58.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.745+0000 7fbf5ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf580aed40 0x7fbf580ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.745+0000 7fbf5ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf580cfa90 0x7fbf5814b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.745+0000 7fbf5ca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5815b720 0x7fbf580cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.745+0000 7fbf5ca00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580cff00 con 0x7fbf580aed40 2024-04-25T10:08:58.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.745+0000 7fbf5ca00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5815bb00 con 0x7fbf5815b720 2024-04-25T10:08:58.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.745+0000 7fbf5ca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580d0080 con 0x7fbf580cfa90 2024-04-25T10:08:58.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.746+0000 7fbf55e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf580aed40 0x7fbf580ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.746+0000 7fbf55e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf580aed40 0x7fbf580ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38478/0 (socket says 172.21.15.119:38478) 2024-04-25T10:08:58.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.746+0000 7fbf55e00640 1 -- 172.21.15.119:0/2451660314 learned_addr learned my addr 172.21.15.119:0/2451660314 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:58.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.746+0000 7fbf4d400640 1 --2- 172.21.15.119:0/2451660314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf580cfa90 0x7fbf5814b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.746+0000 7fbf55e00640 1 -- 172.21.15.119:0/2451660314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf580cfa90 msgr2=0x7fbf5814b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.746+0000 7fbf55e00640 1 --2- 172.21.15.119:0/2451660314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf580cfa90 0x7fbf5814b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.746+0000 7fbf55e00640 1 -- 172.21.15.119:0/2451660314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5815b720 msgr2=0x7fbf580cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.746+0000 7fbf55e00640 1 --2- 172.21.15.119:0/2451660314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5815b720 0x7fbf580cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.746+0000 7fbf55e00640 1 -- 172.21.15.119:0/2451660314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf580cc690 con 0x7fbf580aed40 2024-04-25T10:08:58.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.746+0000 7fbf55e00640 1 --2- 172.21.15.119:0/2451660314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf580aed40 0x7fbf580ceb90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800c3f0 tx=0x7fbf4800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:58.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.747+0000 7fbf4ea00640 1 -- 172.21.15.119:0/2451660314 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf480052d0 con 0x7fbf580aed40 2024-04-25T10:08:58.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.747+0000 7fbf4ea00640 1 -- 172.21.15.119:0/2451660314 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf48005490 con 0x7fbf580aed40 2024-04-25T10:08:58.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.747+0000 7fbf4ea00640 1 -- 172.21.15.119:0/2451660314 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48007260 con 0x7fbf580aed40 2024-04-25T10:08:58.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.747+0000 7fbf5ca00640 1 -- 172.21.15.119:0/2451660314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf580cc930 con 0x7fbf580aed40 2024-04-25T10:08:58.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.747+0000 7fbf5ca00640 1 -- 172.21.15.119:0/2451660314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf580ccef0 con 0x7fbf580aed40 2024-04-25T10:08:58.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.748+0000 7fbf5ca00640 1 -- 172.21.15.119:0/2451660314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf580750b0 con 0x7fbf580aed40 2024-04-25T10:08:58.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.749+0000 7fbf4ea00640 1 -- 172.21.15.119:0/2451660314 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf48007400 con 0x7fbf580aed40 2024-04-25T10:08:58.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.750+0000 7fbf4ea00640 1 --2- 172.21.15.119:0/2451660314 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbf3406b4d0 0x7fbf3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.750+0000 7fbf4ea00640 1 -- 172.21.15.119:0/2451660314 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf480896f0 con 0x7fbf580aed40 2024-04-25T10:08:58.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.750+0000 7fbf4d400640 1 --2- 172.21.15.119:0/2451660314 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbf3406b4d0 0x7fbf3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.751+0000 7fbf4d400640 1 --2- 172.21.15.119:0/2451660314 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbf3406b4d0 0x7fbf3406d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbf58152050 tx=0x7fbf40008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:58.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.754+0000 7fbf4ea00640 1 -- 172.21.15.119:0/2451660314 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf48058560 con 0x7fbf580aed40 2024-04-25T10:08:58.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:58.975+0000 7fbf5ca00640 1 -- 172.21.15.119:0/2451660314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbf5806f0f0 con 0x7fbf580aed40 2024-04-25T10:08:59.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.675+0000 7fbf4ea00640 1 -- 172.21.15.119:0/2451660314 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbf48057e60 con 0x7fbf580aed40 2024-04-25T10:08:59.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.679+0000 7fbf5ca00640 1 -- 172.21.15.119:0/2451660314 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbf3406b4d0 msgr2=0x7fbf3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.679+0000 7fbf5ca00640 1 --2- 172.21.15.119:0/2451660314 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbf3406b4d0 0x7fbf3406d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbf58152050 tx=0x7fbf40008040 comp rx=0 tx=0).stop 2024-04-25T10:08:59.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.679+0000 7fbf5ca00640 1 -- 172.21.15.119:0/2451660314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf580aed40 msgr2=0x7fbf580ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.679+0000 7fbf5ca00640 1 --2- 172.21.15.119:0/2451660314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf580aed40 0x7fbf580ceb90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800c3f0 tx=0x7fbf4800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.679+0000 7fbf5ca00640 1 -- 172.21.15.119:0/2451660314 shutdown_connections 2024-04-25T10:08:59.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.679+0000 7fbf5ca00640 1 --2- 172.21.15.119:0/2451660314 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbf3406b4d0 0x7fbf3406d990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.679+0000 7fbf5ca00640 1 --2- 172.21.15.119:0/2451660314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf5815b720 0x7fbf580cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.679+0000 7fbf5ca00640 1 --2- 172.21.15.119:0/2451660314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf580cfa90 0x7fbf5814b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.679+0000 7fbf5ca00640 1 --2- 172.21.15.119:0/2451660314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf580aed40 0x7fbf580ceb90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.679+0000 7fbf5ca00640 1 -- 172.21.15.119:0/2451660314 >> 172.21.15.119:0/2451660314 conn(0x7fbf5806aba0 msgr2=0x7fbf580ad770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:59.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.679+0000 7fbf5ca00640 1 -- 172.21.15.119:0/2451660314 shutdown_connections 2024-04-25T10:08:59.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.679+0000 7fbf5ca00640 1 -- 172.21.15.119:0/2451660314 wait complete. 2024-04-25T10:08:59.705 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:08:59.705 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:08:59.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.906+0000 7f2785a00640 1 Processor -- start 2024-04-25T10:08:59.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.906+0000 7f2785a00640 1 -- start start 2024-04-25T10:08:59.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.907+0000 7f2785a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f278015b7e0 0x7f278015e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.907+0000 7f2785a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27800ce860 0x7f27800cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.907+0000 7f2785a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27800b5710 con 0x7f27800d00a0 2024-04-25T10:08:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.907+0000 7f2785a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27800b59f0 con 0x7f278015b7e0 2024-04-25T10:08:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.907+0000 7f2785a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27800b5cd0 con 0x7f27800ce860 2024-04-25T10:08:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.908+0000 7f277fe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f278015b7e0 0x7f278015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.908+0000 7f277fe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f278015b7e0 0x7f278015e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38490/0 (socket says 172.21.15.119:38490) 2024-04-25T10:08:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.908+0000 7f277fe00640 1 -- 172.21.15.119:0/3958739663 learned_addr learned my addr 172.21.15.119:0/3958739663 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.908+0000 7f277f400640 1 --2- 172.21.15.119:0/3958739663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27800ce860 0x7f27800cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.908+0000 7f277fe00640 1 -- 172.21.15.119:0/3958739663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27800ce860 msgr2=0x7f27800cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.908+0000 7f277fe00640 1 --2- 172.21.15.119:0/3958739663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27800ce860 0x7f27800cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.908+0000 7f277fe00640 1 -- 172.21.15.119:0/3958739663 >> v1:172.21.15.119:6789/0 conn(0x7f27800d00a0 legacy=0x7f27800d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.908+0000 7f277ea00640 1 -- 172.21.15.119:0/3958739663 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4063075558 0 0) 0x7f27800b5710 con 0x7f27800d00a0 2024-04-25T10:08:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.908+0000 7f277fe00640 1 -- 172.21.15.119:0/3958739663 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27800b5fb0 con 0x7f278015b7e0 2024-04-25T10:08:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.908+0000 7f277f400640 1 --2- 172.21.15.119:0/3958739663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27800ce860 0x7f27800cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:59.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.909+0000 7f277fe00640 1 --2- 172.21.15.119:0/3958739663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f278015b7e0 0x7f278015e000 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2770009870 tx=0x7f277002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a6405a0cfdf1f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.910+0000 7f277ea00640 1 -- 172.21.15.119:0/3958739663 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770037070 con 0x7f278015b7e0 2024-04-25T10:08:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.910+0000 7f277ea00640 1 -- 172.21.15.119:0/3958739663 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2770032440 con 0x7f278015b7e0 2024-04-25T10:08:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.910+0000 7f277ea00640 1 -- 172.21.15.119:0/3958739663 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770032810 con 0x7f278015b7e0 2024-04-25T10:08:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.910+0000 7f2785a00640 1 -- 172.21.15.119:0/3958739663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f278015b7e0 msgr2=0x7f278015e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.910+0000 7f2785a00640 1 --2- 172.21.15.119:0/3958739663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f278015b7e0 0x7f278015e000 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2770009870 tx=0x7f277002a130 comp rx=0 tx=0).stop 2024-04-25T10:08:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.911+0000 7f2785a00640 1 -- 172.21.15.119:0/3958739663 shutdown_connections 2024-04-25T10:08:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.911+0000 7f2785a00640 1 --2- 172.21.15.119:0/3958739663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27800ce860 0x7f27800cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.911+0000 7f2785a00640 1 --2- 172.21.15.119:0/3958739663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f278015b7e0 0x7f278015e000 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.911+0000 7f2785a00640 1 -- 172.21.15.119:0/3958739663 >> 172.21.15.119:0/3958739663 conn(0x7f27800722d0 msgr2=0x7f27800726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.911+0000 7f2785a00640 1 -- 172.21.15.119:0/3958739663 shutdown_connections 2024-04-25T10:08:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.911+0000 7f2785a00640 1 -- 172.21.15.119:0/3958739663 wait complete. 2024-04-25T10:08:59.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.912+0000 7f2785a00640 1 Processor -- start 2024-04-25T10:08:59.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.913+0000 7f2785a00640 1 -- start start 2024-04-25T10:08:59.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.913+0000 7f2785a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27800ce860 0x7f27801517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.913+0000 7f2785a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27800d00a0 0x7f27800caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.914+0000 7f2785a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f278015b7e0 0x7f278015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.914+0000 7f2785a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2780161960 con 0x7f27800ce860 2024-04-25T10:08:59.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.914+0000 7f2785a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27800b53c0 con 0x7f27800d00a0 2024-04-25T10:08:59.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.914+0000 7f2785a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2780161ae0 con 0x7f278015b7e0 2024-04-25T10:08:59.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.914+0000 7f277fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27800ce860 0x7f27801517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.914+0000 7f277fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27800ce860 0x7f27801517e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38504/0 (socket says 172.21.15.119:38504) 2024-04-25T10:08:59.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.914+0000 7f277fe00640 1 -- 172.21.15.119:0/458041996 learned_addr learned my addr 172.21.15.119:0/458041996 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:08:59.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.914+0000 7f277fe00640 1 -- 172.21.15.119:0/458041996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f278015b7e0 msgr2=0x7f278015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.914+0000 7f277fe00640 1 --2- 172.21.15.119:0/458041996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f278015b7e0 0x7f278015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.914+0000 7f277fe00640 1 -- 172.21.15.119:0/458041996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27800d00a0 msgr2=0x7f27800caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.914+0000 7f277fe00640 1 --2- 172.21.15.119:0/458041996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27800d00a0 0x7f27800caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.914+0000 7f277fe00640 1 -- 172.21.15.119:0/458041996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27800b6f50 con 0x7f27800ce860 2024-04-25T10:08:59.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.915+0000 7f277fe00640 1 --2- 172.21.15.119:0/458041996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27800ce860 0x7f27801517e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2770000c00 tx=0x7f27700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.915+0000 7f2777e00640 1 -- 172.21.15.119:0/458041996 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770037040 con 0x7f27800ce860 2024-04-25T10:08:59.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.915+0000 7f2777e00640 1 -- 172.21.15.119:0/458041996 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2770042070 con 0x7f27800ce860 2024-04-25T10:08:59.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.915+0000 7f2785a00640 1 -- 172.21.15.119:0/458041996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27800b7230 con 0x7f27800ce860 2024-04-25T10:08:59.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.915+0000 7f2777e00640 1 -- 172.21.15.119:0/458041996 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770030bd0 con 0x7f27800ce860 2024-04-25T10:08:59.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.915+0000 7f2785a00640 1 -- 172.21.15.119:0/458041996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27800b77f0 con 0x7f27800ce860 2024-04-25T10:08:59.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.916+0000 7f2785a00640 1 -- 172.21.15.119:0/458041996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27800749b0 con 0x7f27800ce860 2024-04-25T10:08:59.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.917+0000 7f2777e00640 1 -- 172.21.15.119:0/458041996 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2770047460 con 0x7f27800ce860 2024-04-25T10:08:59.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.918+0000 7f2777e00640 1 --2- 172.21.15.119:0/458041996 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f275c06b500 0x7f275c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.918+0000 7f2777e00640 1 -- 172.21.15.119:0/458041996 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27700aad90 con 0x7f27800ce860 2024-04-25T10:08:59.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.918+0000 7f277f400640 1 --2- 172.21.15.119:0/458041996 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f275c06b500 0x7f275c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.919+0000 7f277f400640 1 --2- 172.21.15.119:0/458041996 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f275c06b500 0x7f275c06d9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f278015ac40 tx=0x7f27680083d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:08:59.923+0000 7f2777e00640 1 -- 172.21.15.119:0/458041996 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2770045360 con 0x7f27800ce860 2024-04-25T10:09:00.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.155+0000 7f2785a00640 1 -- 172.21.15.119:0/458041996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f27800d1650 con 0x7f27800ce860 2024-04-25T10:09:00.460 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:00.469 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:00.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.679+0000 7f2777e00640 1 -- 172.21.15.119:0/458041996 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2770079ca0 con 0x7f27800ce860 2024-04-25T10:09:00.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.682+0000 7f2785a00640 1 -- 172.21.15.119:0/458041996 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f275c06b500 msgr2=0x7f275c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.682+0000 7f2785a00640 1 --2- 172.21.15.119:0/458041996 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f275c06b500 0x7f275c06d9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f278015ac40 tx=0x7f27680083d0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.682+0000 7f2785a00640 1 -- 172.21.15.119:0/458041996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27800ce860 msgr2=0x7f27801517e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.682+0000 7f2785a00640 1 --2- 172.21.15.119:0/458041996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27800ce860 0x7f27801517e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2770000c00 tx=0x7f27700046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.682+0000 7f2785a00640 1 -- 172.21.15.119:0/458041996 shutdown_connections 2024-04-25T10:09:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.682+0000 7f2785a00640 1 --2- 172.21.15.119:0/458041996 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f275c06b500 0x7f275c06d9c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.682+0000 7f2785a00640 1 --2- 172.21.15.119:0/458041996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f278015b7e0 0x7f278015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.682+0000 7f2785a00640 1 --2- 172.21.15.119:0/458041996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27800d00a0 0x7f27800caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.682+0000 7f2785a00640 1 --2- 172.21.15.119:0/458041996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27800ce860 0x7f27801517e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.682+0000 7f2785a00640 1 -- 172.21.15.119:0/458041996 >> 172.21.15.119:0/458041996 conn(0x7f27800722d0 msgr2=0x7f27800d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.683+0000 7f2785a00640 1 -- 172.21.15.119:0/458041996 shutdown_connections 2024-04-25T10:09:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.683+0000 7f2785a00640 1 -- 172.21.15.119:0/458041996 wait complete. 2024-04-25T10:09:00.709 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:09:00.710 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:00.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.904+0000 7f5ad3200640 1 Processor -- start 2024-04-25T10:09:00.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.905+0000 7f5ad3200640 1 -- start start 2024-04-25T10:09:00.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.906+0000 7f5ad3200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5acc15bf00 0x7f5acc0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.906+0000 7f5ad3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0cc310 0x7f5acc15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.906+0000 7f5ad3200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5acc0ce6c0 con 0x7f5acc15c820 2024-04-25T10:09:00.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.907+0000 7f5ad3200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5acc1455e0 con 0x7f5acc15bf00 2024-04-25T10:09:00.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.907+0000 7f5ad3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc06d450 con 0x7f5acc0cc310 2024-04-25T10:09:00.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.907+0000 7f5acb400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0cc310 0x7f5acc15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.907+0000 7f5acb400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0cc310 0x7f5acc15c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:57740/0 (socket says 172.21.15.119:57740) 2024-04-25T10:09:00.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.907+0000 7f5acb400640 1 -- 172.21.15.119:0/1564410107 learned_addr learned my addr 172.21.15.119:0/1564410107 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:00.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.907+0000 7f5acbe00640 1 --2- 172.21.15.119:0/1564410107 >> v2:172.21.15.132:3300/0 conn(0x7f5acc15bf00 0x7f5acc0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.908+0000 7f5acb400640 1 -- 172.21.15.119:0/1564410107 >> v2:172.21.15.132:3300/0 conn(0x7f5acc15bf00 msgr2=0x7f5acc0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:00.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.908+0000 7f5acb400640 1 --2- 172.21.15.119:0/1564410107 >> v2:172.21.15.132:3300/0 conn(0x7f5acc15bf00 0x7f5acc0cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.908+0000 7f5acb400640 1 -- 172.21.15.119:0/1564410107 >> v1:172.21.15.132:6789/0 conn(0x7f5acc15c820 legacy=0x7f5acc06d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.908+0000 7f5acb400640 1 -- 172.21.15.119:0/1564410107 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc0cc7b0 con 0x7f5acc0cc310 2024-04-25T10:09:00.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.909+0000 7f5acb400640 1 --2- 172.21.15.119:0/1564410107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0cc310 0x7f5acc15c2e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009870 tx=0x7f5ab402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84bce49007921553 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:00.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.910+0000 7f5acaa00640 1 -- 172.21.15.119:0/1564410107 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4037070 con 0x7f5acc0cc310 2024-04-25T10:09:00.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.910+0000 7f5acaa00640 1 -- 172.21.15.119:0/1564410107 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab4032440 con 0x7f5acc0cc310 2024-04-25T10:09:00.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.910+0000 7f5acaa00640 1 -- 172.21.15.119:0/1564410107 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4032810 con 0x7f5acc0cc310 2024-04-25T10:09:00.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.911+0000 7f5ad3200640 1 -- 172.21.15.119:0/1564410107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0cc310 msgr2=0x7f5acc15c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.911+0000 7f5ad3200640 1 --2- 172.21.15.119:0/1564410107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0cc310 0x7f5acc15c2e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009870 tx=0x7f5ab402a270 comp rx=0 tx=0).stop 2024-04-25T10:09:00.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.911+0000 7f5ad3200640 1 -- 172.21.15.119:0/1564410107 shutdown_connections 2024-04-25T10:09:00.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.911+0000 7f5ad3200640 1 --2- 172.21.15.119:0/1564410107 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0cc310 0x7f5acc15c2e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.911+0000 7f5ad3200640 1 --2- 172.21.15.119:0/1564410107 >> v2:172.21.15.132:3300/0 conn(0x7f5acc15bf00 0x7f5acc0cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.911+0000 7f5ad3200640 1 -- 172.21.15.119:0/1564410107 >> 172.21.15.119:0/1564410107 conn(0x7f5acc06aba0 msgr2=0x7f5acc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:00.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.911+0000 7f5ad3200640 1 -- 172.21.15.119:0/1564410107 shutdown_connections 2024-04-25T10:09:00.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.911+0000 7f5ad3200640 1 -- 172.21.15.119:0/1564410107 wait complete. 2024-04-25T10:09:00.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.913+0000 7f5ad3200640 1 Processor -- start 2024-04-25T10:09:00.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.913+0000 7f5ad3200640 1 -- start start 2024-04-25T10:09:00.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.914+0000 7f5ad3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0cc310 0x7f5acc14db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.914+0000 7f5ad3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5acc15bf00 0x7f5acc1e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.914+0000 7f5ad3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc15c820 0x7f5acc140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.914+0000 7f5ad3200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0cd050 con 0x7f5acc15bf00 2024-04-25T10:09:00.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.914+0000 7f5ad3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0ccd70 con 0x7f5acc15c820 2024-04-25T10:09:00.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.914+0000 7f5ad3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc15cca0 con 0x7f5acc0cc310 2024-04-25T10:09:00.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.914+0000 7f5acb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5acc15bf00 0x7f5acc1e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.915+0000 7f5acb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5acc15bf00 0x7f5acc1e49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38206/0 (socket says 172.21.15.119:38206) 2024-04-25T10:09:00.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.915+0000 7f5acb400640 1 -- 172.21.15.119:0/931343824 learned_addr learned my addr 172.21.15.119:0/931343824 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:00.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.915+0000 7f5acbe00640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0cc310 0x7f5acc14db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.915+0000 7f5acb400640 1 -- 172.21.15.119:0/931343824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0cc310 msgr2=0x7f5acc14db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.915+0000 7f5acb400640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0cc310 0x7f5acc14db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.915+0000 7f5ad1200640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc15c820 0x7f5acc140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.915+0000 7f5acb400640 1 -- 172.21.15.119:0/931343824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc15c820 msgr2=0x7f5acc140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.915+0000 7f5acb400640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc15c820 0x7f5acc140820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.915+0000 7f5acb400640 1 -- 172.21.15.119:0/931343824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc1f3690 con 0x7f5acc15bf00 2024-04-25T10:09:00.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.915+0000 7f5ad1200640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc15c820 0x7f5acc140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:00.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.915+0000 7f5acb400640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5acc15bf00 0x7f5acc1e49c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4000c00 tx=0x7f5ab40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:00.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.916+0000 7f5ac3e00640 1 -- 172.21.15.119:0/931343824 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4037040 con 0x7f5acc15bf00 2024-04-25T10:09:00.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.916+0000 7f5ac3e00640 1 -- 172.21.15.119:0/931343824 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab4042070 con 0x7f5acc15bf00 2024-04-25T10:09:00.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.916+0000 7f5ad3200640 1 -- 172.21.15.119:0/931343824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5acc13f780 con 0x7f5acc15bf00 2024-04-25T10:09:00.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.916+0000 7f5ac3e00640 1 -- 172.21.15.119:0/931343824 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4030880 con 0x7f5acc15bf00 2024-04-25T10:09:00.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.916+0000 7f5ad3200640 1 -- 172.21.15.119:0/931343824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5acc140020 con 0x7f5acc15bf00 2024-04-25T10:09:00.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.917+0000 7f5ad3200640 1 -- 172.21.15.119:0/931343824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5acc06bb30 con 0x7f5acc15bf00 2024-04-25T10:09:00.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.918+0000 7f5ac3e00640 1 -- 172.21.15.119:0/931343824 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ab4030a20 con 0x7f5acc15bf00 2024-04-25T10:09:00.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.919+0000 7f5ac3e00640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5aa806b4d0 0x7f5aa806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.919+0000 7f5ac3e00640 1 -- 172.21.15.119:0/931343824 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ab40aaab0 con 0x7f5acc15bf00 2024-04-25T10:09:00.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.919+0000 7f5acbe00640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5aa806b4d0 0x7f5aa806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.920+0000 7f5acbe00640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5aa806b4d0 0x7f5aa806d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009a10 tx=0x7f5abc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:00.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:00.924+0000 7f5ac3e00640 1 -- 172.21.15.119:0/931343824 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5acc1f3690 con 0x7f5acc15bf00 2024-04-25T10:09:01.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.157+0000 7f5ad3200640 1 -- 172.21.15.119:0/931343824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5acc0cea20 con 0x7f5acc15bf00 2024-04-25T10:09:01.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.158+0000 7f5ac3e00640 1 -- 172.21.15.119:0/931343824 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f5ab4079920 con 0x7f5acc15bf00 2024-04-25T10:09:01.165 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:01.165 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3317744182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3317744182},{"type":"v1","addr":"172.21.15.132:6833","nonce":3317744182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/700427287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":700427287},{"type":"v1","addr":"172.21.15.132:6835","nonce":700427287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2207174972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2207174972},{"type":"v1","addr":"172.21.15.119:6836","nonce":2207174972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:57.120451+0000","modified":"2024-04-25T10:09:00.157396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5478":{"gid":5478,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/270766821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":270766821},{"type":"v1","addr":"172.21.15.119:6837","nonce":270766821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:09:01.165 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 25 2024-04-25T10:09:01.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.161+0000 7f5ad3200640 1 -- 172.21.15.119:0/931343824 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5aa806b4d0 msgr2=0x7f5aa806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.161+0000 7f5ad3200640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5aa806b4d0 0x7f5aa806d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009a10 tx=0x7f5abc008040 comp rx=0 tx=0).stop 2024-04-25T10:09:01.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.161+0000 7f5ad3200640 1 -- 172.21.15.119:0/931343824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5acc15bf00 msgr2=0x7f5acc1e49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.161+0000 7f5ad3200640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5acc15bf00 0x7f5acc1e49c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4000c00 tx=0x7f5ab40046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.161+0000 7f5ad3200640 1 -- 172.21.15.119:0/931343824 shutdown_connections 2024-04-25T10:09:01.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.161+0000 7f5ad3200640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5aa806b4d0 0x7f5aa806d990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.161+0000 7f5ad3200640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc15c820 0x7f5acc140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.161+0000 7f5ad3200640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5acc15bf00 0x7f5acc1e49c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.161+0000 7f5ad3200640 1 --2- 172.21.15.119:0/931343824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5acc0cc310 0x7f5acc14db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.161+0000 7f5ad3200640 1 -- 172.21.15.119:0/931343824 >> 172.21.15.119:0/931343824 conn(0x7f5acc06aba0 msgr2=0x7f5acc15e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:01.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.161+0000 7f5ad3200640 1 -- 172.21.15.119:0/931343824 shutdown_connections 2024-04-25T10:09:01.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.162+0000 7f5ad3200640 1 -- 172.21.15.119:0/931343824 wait complete. 2024-04-25T10:09:01.187 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:01.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.378+0000 7f0edf400640 1 Processor -- start 2024-04-25T10:09:01.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.378+0000 7f0edf400640 1 -- start start 2024-04-25T10:09:01.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.380+0000 7f0edf400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ed815bf00 0x7f0ed80cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.380+0000 7f0edf400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed80ce6c0 con 0x7f0ed80cc310 2024-04-25T10:09:01.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.380+0000 7f0edf400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed80cc7b0 con 0x7f0ed806e8f0 2024-04-25T10:09:01.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.380+0000 7f0edf400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed81455e0 con 0x7f0ed815bf00 2024-04-25T10:09:01.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.380+0000 7f0ecfe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0ed80cc310 0x7f0ed815c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42090/0 (socket says 172.21.15.119:42090) 2024-04-25T10:09:01.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.380+0000 7f0ecfe00640 1 -- 172.21.15.119:0/2163019902 learned_addr learned my addr 172.21.15.119:0/2163019902 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:01.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.380+0000 7f0edca00640 1 --2- 172.21.15.119:0/2163019902 >> v2:172.21.15.119:3300/0 conn(0x7f0ed815bf00 0x7f0ed80cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.381+0000 7f0edca00640 1 -- 172.21.15.119:0/2163019902 >> v1:172.21.15.132:6789/0 conn(0x7f0ed806e8f0 legacy=0x7f0ed81654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:01.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.381+0000 7f0edca00640 1 -- 172.21.15.119:0/2163019902 >> v1:172.21.15.119:6789/0 conn(0x7f0ed80cc310 legacy=0x7f0ed815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.381+0000 7f0edca00640 1 -- 172.21.15.119:0/2163019902 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80cca90 con 0x7f0ed815bf00 2024-04-25T10:09:01.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.381+0000 7f0ed7e00640 1 -- 172.21.15.119:0/2163019902 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470880123 0 0) 0x7f0ec0001110 con 0x7f0ed80cc310 2024-04-25T10:09:01.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.382+0000 7f0edca00640 1 --2- 172.21.15.119:0/2163019902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed815bf00 0x7f0ed80cbdd0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80097a0 tx=0x7f0ec802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b249dc2b563cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:01.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.382+0000 7f0ed7e00640 1 -- 172.21.15.119:0/2163019902 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8037070 con 0x7f0ed815bf00 2024-04-25T10:09:01.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.382+0000 7f0ed7e00640 1 -- 172.21.15.119:0/2163019902 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec8032440 con 0x7f0ed815bf00 2024-04-25T10:09:01.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.382+0000 7f0ed7e00640 1 -- 172.21.15.119:0/2163019902 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8032810 con 0x7f0ed815bf00 2024-04-25T10:09:01.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.383+0000 7f0edf400640 1 -- 172.21.15.119:0/2163019902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed815bf00 msgr2=0x7f0ed80cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.383+0000 7f0edf400640 1 --2- 172.21.15.119:0/2163019902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed815bf00 0x7f0ed80cbdd0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80097a0 tx=0x7f0ec802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.383+0000 7f0edf400640 1 -- 172.21.15.119:0/2163019902 shutdown_connections 2024-04-25T10:09:01.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.383+0000 7f0edf400640 1 --2- 172.21.15.119:0/2163019902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed815bf00 0x7f0ed80cbdd0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.383+0000 7f0edf400640 1 -- 172.21.15.119:0/2163019902 >> 172.21.15.119:0/2163019902 conn(0x7f0ed806aba0 msgr2=0x7f0ed806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:01.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.383+0000 7f0edf400640 1 -- 172.21.15.119:0/2163019902 shutdown_connections 2024-04-25T10:09:01.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.383+0000 7f0edf400640 1 -- 172.21.15.119:0/2163019902 wait complete. 2024-04-25T10:09:01.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.385+0000 7f0edf400640 1 Processor -- start 2024-04-25T10:09:01.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.385+0000 7f0edf400640 1 -- start start 2024-04-25T10:09:01.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.386+0000 7f0edf400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ed806e8f0 0x7f0ed80cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.386+0000 7f0edf400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ed80cc310 0x7f0ed81f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.386+0000 7f0edf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed815bf00 0x7f0ed81f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.386+0000 7f0edf400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80cd050 con 0x7f0ed815bf00 2024-04-25T10:09:01.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.386+0000 7f0edf400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed806d450 con 0x7f0ed80cc310 2024-04-25T10:09:01.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.386+0000 7f0edf400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed806ee60 con 0x7f0ed806e8f0 2024-04-25T10:09:01.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.386+0000 7f0edd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed815bf00 0x7f0ed81f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.386+0000 7f0edd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed815bf00 0x7f0ed81f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38222/0 (socket says 172.21.15.119:38222) 2024-04-25T10:09:01.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.386+0000 7f0edd400640 1 -- 172.21.15.119:0/4137578954 learned_addr learned my addr 172.21.15.119:0/4137578954 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:01.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.386+0000 7f0edca00640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ed806e8f0 0x7f0ed80cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.386+0000 7f0edd400640 1 -- 172.21.15.119:0/4137578954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ed806e8f0 msgr2=0x7f0ed80cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.387+0000 7f0edd400640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ed806e8f0 0x7f0ed80cf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.387+0000 7f0edd400640 1 -- 172.21.15.119:0/4137578954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ed80cc310 msgr2=0x7f0ed81f16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.387+0000 7f0ecfe00640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ed80cc310 0x7f0ed81f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.387+0000 7f0edd400640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ed80cc310 0x7f0ed81f16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.387+0000 7f0edd400640 1 -- 172.21.15.119:0/4137578954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed81f5750 con 0x7f0ed815bf00 2024-04-25T10:09:01.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.387+0000 7f0ecfe00640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ed80cc310 0x7f0ed81f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:01.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.387+0000 7f0edd400640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed815bf00 0x7f0ed81f4c90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000c780 tx=0x7f0ed000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:01.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.387+0000 7f0ed5600640 1 -- 172.21.15.119:0/4137578954 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0004b60 con 0x7f0ed815bf00 2024-04-25T10:09:01.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.387+0000 7f0ed5600640 1 -- 172.21.15.119:0/4137578954 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed0004d00 con 0x7f0ed815bf00 2024-04-25T10:09:01.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.387+0000 7f0ed5600640 1 -- 172.21.15.119:0/4137578954 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed000aa90 con 0x7f0ed815bf00 2024-04-25T10:09:01.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.387+0000 7f0edf400640 1 -- 172.21.15.119:0/4137578954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed81f5a30 con 0x7f0ed815bf00 2024-04-25T10:09:01.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.387+0000 7f0edf400640 1 -- 172.21.15.119:0/4137578954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed81e4630 con 0x7f0ed815bf00 2024-04-25T10:09:01.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.388+0000 7f0edf400640 1 -- 172.21.15.119:0/4137578954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed806bb30 con 0x7f0ed815bf00 2024-04-25T10:09:01.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.389+0000 7f0ed5600640 1 -- 172.21.15.119:0/4137578954 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ed000ac30 con 0x7f0ed815bf00 2024-04-25T10:09:01.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.390+0000 7f0ed5600640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0eb406b4d0 0x7f0eb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.390+0000 7f0ed5600640 1 -- 172.21.15.119:0/4137578954 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ed008a580 con 0x7f0ed815bf00 2024-04-25T10:09:01.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.391+0000 7f0edca00640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0eb406b4d0 0x7f0eb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.391+0000 7f0edca00640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0eb406b4d0 0x7f0eb406d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8034000 tx=0x7f0ec802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:01.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.395+0000 7f0ed5600640 1 -- 172.21.15.119:0/4137578954 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ed0020070 con 0x7f0ed815bf00 2024-04-25T10:09:01.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.589+0000 7f0edf400640 1 -- 172.21.15.119:0/4137578954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ed81e4800 con 0x7f0ed815bf00 2024-04-25T10:09:01.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.591+0000 7f0ed5600640 1 -- 172.21.15.119:0/4137578954 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16649 (secure 0 0 0) 0x7f0ed0059360 con 0x7f0ed815bf00 2024-04-25T10:09:01.600 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:01.601 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":44,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:58.637281+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:54.917692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:56.128440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:01.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.593+0000 7f0edf400640 1 -- 172.21.15.119:0/4137578954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0eb406b4d0 msgr2=0x7f0eb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.593+0000 7f0edf400640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0eb406b4d0 0x7f0eb406d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8034000 tx=0x7f0ec802e000 comp rx=0 tx=0).stop 2024-04-25T10:09:01.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.593+0000 7f0edf400640 1 -- 172.21.15.119:0/4137578954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed815bf00 msgr2=0x7f0ed81f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.593+0000 7f0edf400640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed815bf00 0x7f0ed81f4c90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000c780 tx=0x7f0ed000cc40 comp rx=0 tx=0).stop 2024-04-25T10:09:01.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.594+0000 7f0edf400640 1 -- 172.21.15.119:0/4137578954 shutdown_connections 2024-04-25T10:09:01.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.594+0000 7f0edf400640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0eb406b4d0 0x7f0eb406d990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.594+0000 7f0edf400640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed815bf00 0x7f0ed81f4c90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.594+0000 7f0edf400640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ed80cc310 0x7f0ed81f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.594+0000 7f0edf400640 1 --2- 172.21.15.119:0/4137578954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ed806e8f0 0x7f0ed80cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.594+0000 7f0edf400640 1 -- 172.21.15.119:0/4137578954 >> 172.21.15.119:0/4137578954 conn(0x7f0ed806aba0 msgr2=0x7f0ed80ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:01.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.594+0000 7f0edf400640 1 -- 172.21.15.119:0/4137578954 shutdown_connections 2024-04-25T10:09:01.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.594+0000 7f0edf400640 1 -- 172.21.15.119:0/4137578954 wait complete. 2024-04-25T10:09:01.619 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:09:01.619 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:09:01.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.816+0000 7fa9c6e00640 1 Processor -- start 2024-04-25T10:09:01.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.816+0000 7fa9c6e00640 1 -- start start 2024-04-25T10:09:01.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.817+0000 7fa9c6e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa9c00b5050 0x7fa9c015fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.817+0000 7fa9c6e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa9c006e430 0x7fa9c015bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.818+0000 7fa9c6e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c00cd440 con 0x7fa9c0144470 2024-04-25T10:09:01.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.818+0000 7fa9c6e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9c0145340 con 0x7fa9c006e430 2024-04-25T10:09:01.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.818+0000 7fa9c6e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9c00cd720 con 0x7fa9c00b5050 2024-04-25T10:09:01.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.818+0000 7fa9c5a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa9c0144470 0x7fa9c00b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42096/0 (socket says 172.21.15.119:42096) 2024-04-25T10:09:01.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.818+0000 7fa9c5a00640 1 -- 172.21.15.119:0/358991091 learned_addr learned my addr 172.21.15.119:0/358991091 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:01.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.818+0000 7fa9c6400640 1 --2- 172.21.15.119:0/358991091 >> v2:172.21.15.119:3300/0 conn(0x7fa9c006e430 0x7fa9c015bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.818+0000 7fa9c5000640 1 --2- 172.21.15.119:0/358991091 >> v2:172.21.15.132:3300/0 conn(0x7fa9c00b5050 0x7fa9c015fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.818+0000 7fa9bfe00640 1 -- 172.21.15.119:0/358991091 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289452316 0 0) 0x7fa9c00cd440 con 0x7fa9c0144470 2024-04-25T10:09:01.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.818+0000 7fa9bfe00640 1 -- 172.21.15.119:0/358991091 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9a8003520 con 0x7fa9c0144470 2024-04-25T10:09:01.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.818+0000 7fa9c6400640 1 -- 172.21.15.119:0/358991091 >> v2:172.21.15.132:3300/0 conn(0x7fa9c00b5050 msgr2=0x7fa9c015fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:01.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.818+0000 7fa9c6400640 1 --2- 172.21.15.119:0/358991091 >> v2:172.21.15.132:3300/0 conn(0x7fa9c00b5050 0x7fa9c015fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.818+0000 7fa9c6400640 1 -- 172.21.15.119:0/358991091 >> v1:172.21.15.119:6789/0 conn(0x7fa9c0144470 legacy=0x7fa9c00b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.818+0000 7fa9c6400640 1 -- 172.21.15.119:0/358991091 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c00cda00 con 0x7fa9c006e430 2024-04-25T10:09:01.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.819+0000 7fa9c6400640 1 --2- 172.21.15.119:0/358991091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c006e430 0x7fa9c015bcf0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa9c006f500 tx=0x7fa9b802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2974d8844692cffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:01.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.820+0000 7fa9bfe00640 1 -- 172.21.15.119:0/358991091 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8007740 con 0x7fa9c006e430 2024-04-25T10:09:01.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.820+0000 7fa9bfe00640 1 -- 172.21.15.119:0/358991091 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b80078e0 con 0x7fa9c006e430 2024-04-25T10:09:01.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.820+0000 7fa9bfe00640 1 -- 172.21.15.119:0/358991091 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8007cb0 con 0x7fa9c006e430 2024-04-25T10:09:01.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.820+0000 7fa9c6e00640 1 -- 172.21.15.119:0/358991091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c006e430 msgr2=0x7fa9c015bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.820+0000 7fa9c6e00640 1 --2- 172.21.15.119:0/358991091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c006e430 0x7fa9c015bcf0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa9c006f500 tx=0x7fa9b802a680 comp rx=0 tx=0).stop 2024-04-25T10:09:01.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.820+0000 7fa9c6e00640 1 -- 172.21.15.119:0/358991091 shutdown_connections 2024-04-25T10:09:01.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.820+0000 7fa9c6e00640 1 --2- 172.21.15.119:0/358991091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c006e430 0x7fa9c015bcf0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.820+0000 7fa9c6e00640 1 --2- 172.21.15.119:0/358991091 >> v2:172.21.15.132:3300/0 conn(0x7fa9c00b5050 0x7fa9c015fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.820+0000 7fa9c6e00640 1 -- 172.21.15.119:0/358991091 >> 172.21.15.119:0/358991091 conn(0x7fa9c006b1d0 msgr2=0x7fa9c00b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:01.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.821+0000 7fa9c6e00640 1 -- 172.21.15.119:0/358991091 shutdown_connections 2024-04-25T10:09:01.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.821+0000 7fa9c6e00640 1 -- 172.21.15.119:0/358991091 wait complete. 2024-04-25T10:09:01.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.822+0000 7fa9c6e00640 1 Processor -- start 2024-04-25T10:09:01.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.822+0000 7fa9c6e00640 1 -- start start 2024-04-25T10:09:01.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.823+0000 7fa9c6e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c006e430 0x7fa9c01f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.823+0000 7fa9c6e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c00b5050 0x7fa9c00a9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.823+0000 7fa9c6e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9c0144470 0x7fa9c01f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.823+0000 7fa9c6e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c00ce200 con 0x7fa9c00b5050 2024-04-25T10:09:01.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.823+0000 7fa9c6e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c0144850 con 0x7fa9c006e430 2024-04-25T10:09:01.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.823+0000 7fa9c6e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c015c230 con 0x7fa9c0144470 2024-04-25T10:09:01.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.824+0000 7fa9c5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c00b5050 0x7fa9c00a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.824+0000 7fa9c5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c00b5050 0x7fa9c00a9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38252/0 (socket says 172.21.15.119:38252) 2024-04-25T10:09:01.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.824+0000 7fa9c5000640 1 -- 172.21.15.119:0/959000833 learned_addr learned my addr 172.21.15.119:0/959000833 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:01.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.824+0000 7fa9c5000640 1 -- 172.21.15.119:0/959000833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9c0144470 msgr2=0x7fa9c01f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.824+0000 7fa9c6400640 1 --2- 172.21.15.119:0/959000833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9c0144470 0x7fa9c01f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.824+0000 7fa9c5000640 1 --2- 172.21.15.119:0/959000833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9c0144470 0x7fa9c01f2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.824+0000 7fa9c5a00640 1 --2- 172.21.15.119:0/959000833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c006e430 0x7fa9c01f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.824+0000 7fa9c5000640 1 -- 172.21.15.119:0/959000833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c006e430 msgr2=0x7fa9c01f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.824+0000 7fa9c5000640 1 --2- 172.21.15.119:0/959000833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c006e430 0x7fa9c01f1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.824+0000 7fa9c5000640 1 -- 172.21.15.119:0/959000833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c015e7f0 con 0x7fa9c00b5050 2024-04-25T10:09:01.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.824+0000 7fa9c5000640 1 --2- 172.21.15.119:0/959000833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c00b5050 0x7fa9c00a9210 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00d830 tx=0x7fa9ac00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:01.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.825+0000 7fa9bd600640 1 -- 172.21.15.119:0/959000833 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00b6c0 con 0x7fa9c00b5050 2024-04-25T10:09:01.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.825+0000 7fa9bd600640 1 -- 172.21.15.119:0/959000833 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac019070 con 0x7fa9c00b5050 2024-04-25T10:09:01.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.825+0000 7fa9bd600640 1 -- 172.21.15.119:0/959000833 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00ba80 con 0x7fa9c00b5050 2024-04-25T10:09:01.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.825+0000 7fa9c6e00640 1 -- 172.21.15.119:0/959000833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c00a8770 con 0x7fa9c00b5050 2024-04-25T10:09:01.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.825+0000 7fa9c6e00640 1 -- 172.21.15.119:0/959000833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c00a8cf0 con 0x7fa9c00b5050 2024-04-25T10:09:01.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.826+0000 7fa9c6e00640 1 -- 172.21.15.119:0/959000833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c00749b0 con 0x7fa9c00b5050 2024-04-25T10:09:01.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.827+0000 7fa9bd600640 1 -- 172.21.15.119:0/959000833 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9ac01e460 con 0x7fa9c00b5050 2024-04-25T10:09:01.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.828+0000 7fa9bd600640 1 --2- 172.21.15.119:0/959000833 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa9a006b4d0 0x7fa9a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.828+0000 7fa9bd600640 1 -- 172.21.15.119:0/959000833 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9ac08a8c0 con 0x7fa9c00b5050 2024-04-25T10:09:01.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.828+0000 7fa9c5a00640 1 --2- 172.21.15.119:0/959000833 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa9a006b4d0 0x7fa9a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.829+0000 7fa9c5a00640 1 --2- 172.21.15.119:0/959000833 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa9a006b4d0 0x7fa9a006d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4000f60 tx=0x7fa9b4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:01.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:01.832+0000 7fa9bd600640 1 -- 172.21.15.119:0/959000833 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9ac059730 con 0x7fa9c00b5050 2024-04-25T10:09:02.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.026+0000 7fa9c6e00640 1 -- 172.21.15.119:0/959000833 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa9c00b2090 con 0x7fa9a006b4d0 2024-04-25T10:09:02.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.037+0000 7fa9bd600640 1 -- 172.21.15.119:0/959000833 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251959 (secure 0 0 0) 0x7fa9c00b2090 con 0x7fa9a006b4d0 2024-04-25T10:09:02.046 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:02.048 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:09:02.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.041+0000 7fa9c6e00640 1 -- 172.21.15.119:0/959000833 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa9a006b4d0 msgr2=0x7fa9a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.041+0000 7fa9c6e00640 1 --2- 172.21.15.119:0/959000833 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa9a006b4d0 0x7fa9a006d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4000f60 tx=0x7fa9b4009040 comp rx=0 tx=0).stop 2024-04-25T10:09:02.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.041+0000 7fa9c6e00640 1 -- 172.21.15.119:0/959000833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c00b5050 msgr2=0x7fa9c00a9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.041+0000 7fa9c6e00640 1 --2- 172.21.15.119:0/959000833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c00b5050 0x7fa9c00a9210 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00d830 tx=0x7fa9ac00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.042+0000 7fa9c6e00640 1 -- 172.21.15.119:0/959000833 shutdown_connections 2024-04-25T10:09:02.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.042+0000 7fa9c6e00640 1 --2- 172.21.15.119:0/959000833 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa9a006b4d0 0x7fa9a006d990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.042+0000 7fa9c6e00640 1 --2- 172.21.15.119:0/959000833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9c0144470 0x7fa9c01f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.042+0000 7fa9c6e00640 1 --2- 172.21.15.119:0/959000833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c00b5050 0x7fa9c00a9210 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.042+0000 7fa9c6e00640 1 --2- 172.21.15.119:0/959000833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c006e430 0x7fa9c01f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.042+0000 7fa9c6e00640 1 -- 172.21.15.119:0/959000833 >> 172.21.15.119:0/959000833 conn(0x7fa9c006b1d0 msgr2=0x7fa9c013ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:02.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.042+0000 7fa9c6e00640 1 -- 172.21.15.119:0/959000833 shutdown_connections 2024-04-25T10:09:02.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.042+0000 7fa9c6e00640 1 -- 172.21.15.119:0/959000833 wait complete. 2024-04-25T10:09:02.067 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-25T10:09:01.960562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225624,"kb_used_data":5256,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749096,"statfs":{"total":773094113280,"available":772863074304,"internally_reserved":0,"allocated":5382144,"data_stored":2862916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001629"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.651254+0000","last_change":"2024-04-25T10:08:57.675804+0000","last_active":"2024-04-25T10:08:58.651254+0000","last_peered":"2024-04-25T10:08:58.651254+0000","last_clean":"2024-04-25T10:08:58.651254+0000","last_became_active":"2024-04-25T10:08:57.675475+0000","last_became_peered":"2024-04-25T10:08:57.675475+0000","last_unstale":"2024-04-25T10:08:58.651254+0000","last_undegraded":"2024-04-25T10:08:58.651254+0000","last_fullsized":"2024-04-25T10:08:58.651254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:18.005213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652124+0000","last_change":"2024-04-25T10:08:57.669569+0000","last_active":"2024-04-25T10:08:58.652124+0000","last_peered":"2024-04-25T10:08:58.652124+0000","last_clean":"2024-04-25T10:08:58.652124+0000","last_became_active":"2024-04-25T10:08:57.667496+0000","last_became_peered":"2024-04-25T10:08:57.667496+0000","last_unstale":"2024-04-25T10:08:58.652124+0000","last_undegraded":"2024-04-25T10:08:58.652124+0000","last_fullsized":"2024-04-25T10:08:58.652124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:12.575391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696600+0000","last_change":"2024-04-25T10:08:57.673417+0000","last_active":"2024-04-25T10:08:58.696600+0000","last_peered":"2024-04-25T10:08:58.696600+0000","last_clean":"2024-04-25T10:08:58.696600+0000","last_became_active":"2024-04-25T10:08:57.672943+0000","last_became_peered":"2024-04-25T10:08:57.672943+0000","last_unstale":"2024-04-25T10:08:58.696600+0000","last_undegraded":"2024-04-25T10:08:58.696600+0000","last_fullsized":"2024-04-25T10:08:58.696600+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:16.708303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786408+0000","last_change":"2024-04-25T10:08:57.685694+0000","last_active":"2024-04-25T10:08:58.786408+0000","last_peered":"2024-04-25T10:08:58.786408+0000","last_clean":"2024-04-25T10:08:58.786408+0000","last_became_active":"2024-04-25T10:08:57.685407+0000","last_became_peered":"2024-04-25T10:08:57.685407+0000","last_unstale":"2024-04-25T10:08:58.786408+0000","last_undegraded":"2024-04-25T10:08:58.786408+0000","last_fullsized":"2024-04-25T10:08:58.786408+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:51.183347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652092+0000","last_change":"2024-04-25T10:08:57.672287+0000","last_active":"2024-04-25T10:08:58.652092+0000","last_peered":"2024-04-25T10:08:58.652092+0000","last_clean":"2024-04-25T10:08:58.652092+0000","last_became_active":"2024-04-25T10:08:57.667472+0000","last_became_peered":"2024-04-25T10:08:57.667472+0000","last_unstale":"2024-04-25T10:08:58.652092+0000","last_undegraded":"2024-04-25T10:08:58.652092+0000","last_fullsized":"2024-04-25T10:08:58.652092+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.688922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689358+0000","last_change":"2024-04-25T10:08:57.676771+0000","last_active":"2024-04-25T10:08:58.689358+0000","last_peered":"2024-04-25T10:08:58.689358+0000","last_clean":"2024-04-25T10:08:58.689358+0000","last_became_active":"2024-04-25T10:08:57.676470+0000","last_became_peered":"2024-04-25T10:08:57.676470+0000","last_unstale":"2024-04-25T10:08:58.689358+0000","last_undegraded":"2024-04-25T10:08:58.689358+0000","last_fullsized":"2024-04-25T10:08:58.689358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:56.856980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.650034+0000","last_change":"2024-04-25T10:08:57.681918+0000","last_active":"2024-04-25T10:08:58.650034+0000","last_peered":"2024-04-25T10:08:58.650034+0000","last_clean":"2024-04-25T10:08:58.650034+0000","last_became_active":"2024-04-25T10:08:57.680201+0000","last_became_peered":"2024-04-25T10:08:57.680201+0000","last_unstale":"2024-04-25T10:08:58.650034+0000","last_undegraded":"2024-04-25T10:08:58.650034+0000","last_fullsized":"2024-04-25T10:08:58.650034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:51.183347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785623+0000","last_change":"2024-04-25T10:08:57.683699+0000","last_active":"2024-04-25T10:08:58.785623+0000","last_peered":"2024-04-25T10:08:58.785623+0000","last_clean":"2024-04-25T10:08:58.785623+0000","last_became_active":"2024-04-25T10:08:57.683204+0000","last_became_peered":"2024-04-25T10:08:57.683204+0000","last_unstale":"2024-04-25T10:08:58.785623+0000","last_undegraded":"2024-04-25T10:08:58.785623+0000","last_fullsized":"2024-04-25T10:08:58.785623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:16.708303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787506+0000","last_change":"2024-04-25T10:08:57.691808+0000","last_active":"2024-04-25T10:08:58.787506+0000","last_peered":"2024-04-25T10:08:58.787506+0000","last_clean":"2024-04-25T10:08:58.787506+0000","last_became_active":"2024-04-25T10:08:57.691530+0000","last_became_peered":"2024-04-25T10:08:57.691530+0000","last_unstale":"2024-04-25T10:08:58.787506+0000","last_undegraded":"2024-04-25T10:08:58.787506+0000","last_fullsized":"2024-04-25T10:08:58.787506+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:34.555824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787129+0000","last_change":"2024-04-25T10:08:57.686575+0000","last_active":"2024-04-25T10:08:58.787129+0000","last_peered":"2024-04-25T10:08:58.787129+0000","last_clean":"2024-04-25T10:08:58.787129+0000","last_became_active":"2024-04-25T10:08:57.686330+0000","last_became_peered":"2024-04-25T10:08:57.686330+0000","last_unstale":"2024-04-25T10:08:58.787129+0000","last_undegraded":"2024-04-25T10:08:58.787129+0000","last_fullsized":"2024-04-25T10:08:58.787129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:28.606869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787118+0000","last_change":"2024-04-25T10:08:57.673701+0000","last_active":"2024-04-25T10:08:58.787118+0000","last_peered":"2024-04-25T10:08:58.787118+0000","last_clean":"2024-04-25T10:08:58.787118+0000","last_became_active":"2024-04-25T10:08:57.673385+0000","last_became_peered":"2024-04-25T10:08:57.673385+0000","last_unstale":"2024-04-25T10:08:58.787118+0000","last_undegraded":"2024-04-25T10:08:58.787118+0000","last_fullsized":"2024-04-25T10:08:58.787118+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:18.005213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652090+0000","last_change":"2024-04-25T10:08:57.666858+0000","last_active":"2024-04-25T10:08:58.652090+0000","last_peered":"2024-04-25T10:08:58.652090+0000","last_clean":"2024-04-25T10:08:58.652090+0000","last_became_active":"2024-04-25T10:08:57.666495+0000","last_became_peered":"2024-04-25T10:08:57.666495+0000","last_unstale":"2024-04-25T10:08:58.652090+0000","last_undegraded":"2024-04-25T10:08:58.652090+0000","last_fullsized":"2024-04-25T10:08:58.652090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:04.281461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648854+0000","last_change":"2024-04-25T10:08:57.674287+0000","last_active":"2024-04-25T10:08:58.648854+0000","last_peered":"2024-04-25T10:08:58.648854+0000","last_clean":"2024-04-25T10:08:58.648854+0000","last_became_active":"2024-04-25T10:08:57.673879+0000","last_became_peered":"2024-04-25T10:08:57.673879+0000","last_unstale":"2024-04-25T10:08:58.648854+0000","last_undegraded":"2024-04-25T10:08:58.648854+0000","last_fullsized":"2024-04-25T10:08:58.648854+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:35.341277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786378+0000","last_change":"2024-04-25T10:08:57.689718+0000","last_active":"2024-04-25T10:08:58.786378+0000","last_peered":"2024-04-25T10:08:58.786378+0000","last_clean":"2024-04-25T10:08:58.786378+0000","last_became_active":"2024-04-25T10:08:57.689378+0000","last_became_peered":"2024-04-25T10:08:57.689378+0000","last_unstale":"2024-04-25T10:08:58.786378+0000","last_undegraded":"2024-04-25T10:08:58.786378+0000","last_fullsized":"2024-04-25T10:08:58.786378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:12.575391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689289+0000","last_change":"2024-04-25T10:08:57.671577+0000","last_active":"2024-04-25T10:08:58.689289+0000","last_peered":"2024-04-25T10:08:58.689289+0000","last_clean":"2024-04-25T10:08:58.689289+0000","last_became_active":"2024-04-25T10:08:57.671239+0000","last_became_peered":"2024-04-25T10:08:57.671239+0000","last_unstale":"2024-04-25T10:08:58.689289+0000","last_undegraded":"2024-04-25T10:08:58.689289+0000","last_fullsized":"2024-04-25T10:08:58.689289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:56.579721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787288+0000","last_change":"2024-04-25T10:08:57.684648+0000","last_active":"2024-04-25T10:08:58.787288+0000","last_peered":"2024-04-25T10:08:58.787288+0000","last_clean":"2024-04-25T10:08:58.787288+0000","last_became_active":"2024-04-25T10:08:57.684289+0000","last_became_peered":"2024-04-25T10:08:57.684289+0000","last_unstale":"2024-04-25T10:08:58.787288+0000","last_undegraded":"2024-04-25T10:08:58.787288+0000","last_fullsized":"2024-04-25T10:08:58.787288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:52.909262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696722+0000","last_change":"2024-04-25T10:08:57.657002+0000","last_active":"2024-04-25T10:08:58.696722+0000","last_peered":"2024-04-25T10:08:58.696722+0000","last_clean":"2024-04-25T10:08:58.696722+0000","last_became_active":"2024-04-25T10:08:57.656456+0000","last_became_peered":"2024-04-25T10:08:57.656456+0000","last_unstale":"2024-04-25T10:08:58.696722+0000","last_undegraded":"2024-04-25T10:08:58.696722+0000","last_fullsized":"2024-04-25T10:08:58.696722+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.688922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696448+0000","last_change":"2024-04-25T10:08:57.671155+0000","last_active":"2024-04-25T10:08:58.696448+0000","last_peered":"2024-04-25T10:08:58.696448+0000","last_clean":"2024-04-25T10:08:58.696448+0000","last_became_active":"2024-04-25T10:08:57.669482+0000","last_became_peered":"2024-04-25T10:08:57.669482+0000","last_unstale":"2024-04-25T10:08:58.696448+0000","last_undegraded":"2024-04-25T10:08:58.696448+0000","last_fullsized":"2024-04-25T10:08:58.696448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:35.541551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.788305+0000","last_change":"2024-04-25T10:08:57.683670+0000","last_active":"2024-04-25T10:08:58.788305+0000","last_peered":"2024-04-25T10:08:58.788305+0000","last_clean":"2024-04-25T10:08:58.788305+0000","last_became_active":"2024-04-25T10:08:57.682257+0000","last_became_peered":"2024-04-25T10:08:57.682257+0000","last_unstale":"2024-04-25T10:08:58.788305+0000","last_undegraded":"2024-04-25T10:08:58.788305+0000","last_fullsized":"2024-04-25T10:08:58.788305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:42.114199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648846+0000","last_change":"2024-04-25T10:08:57.673783+0000","last_active":"2024-04-25T10:08:58.648846+0000","last_peered":"2024-04-25T10:08:58.648846+0000","last_clean":"2024-04-25T10:08:58.648846+0000","last_became_active":"2024-04-25T10:08:57.672014+0000","last_became_peered":"2024-04-25T10:08:57.672014+0000","last_unstale":"2024-04-25T10:08:58.648846+0000","last_undegraded":"2024-04-25T10:08:58.648846+0000","last_fullsized":"2024-04-25T10:08:58.648846+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:56.474422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689758+0000","last_change":"2024-04-25T10:08:57.677623+0000","last_active":"2024-04-25T10:08:58.689758+0000","last_peered":"2024-04-25T10:08:58.689758+0000","last_clean":"2024-04-25T10:08:58.689758+0000","last_became_active":"2024-04-25T10:08:57.677331+0000","last_became_peered":"2024-04-25T10:08:57.677331+0000","last_unstale":"2024-04-25T10:08:58.689758+0000","last_undegraded":"2024-04-25T10:08:58.689758+0000","last_fullsized":"2024-04-25T10:08:58.689758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:49.880631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786686+0000","last_change":"2024-04-25T10:08:57.690217+0000","last_active":"2024-04-25T10:08:58.786686+0000","last_peered":"2024-04-25T10:08:58.786686+0000","last_clean":"2024-04-25T10:08:58.786686+0000","last_became_active":"2024-04-25T10:08:57.689116+0000","last_became_peered":"2024-04-25T10:08:57.689116+0000","last_unstale":"2024-04-25T10:08:58.786686+0000","last_undegraded":"2024-04-25T10:08:58.786686+0000","last_fullsized":"2024-04-25T10:08:58.786686+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:28.762976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787952+0000","last_change":"2024-04-25T10:08:57.678443+0000","last_active":"2024-04-25T10:08:58.787952+0000","last_peered":"2024-04-25T10:08:58.787952+0000","last_clean":"2024-04-25T10:08:58.787952+0000","last_became_active":"2024-04-25T10:08:57.678109+0000","last_became_peered":"2024-04-25T10:08:57.678109+0000","last_unstale":"2024-04-25T10:08:58.787952+0000","last_undegraded":"2024-04-25T10:08:58.787952+0000","last_fullsized":"2024-04-25T10:08:58.787952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:20.623071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787638+0000","last_change":"2024-04-25T10:08:57.678264+0000","last_active":"2024-04-25T10:08:58.787638+0000","last_peered":"2024-04-25T10:08:58.787638+0000","last_clean":"2024-04-25T10:08:58.787638+0000","last_became_active":"2024-04-25T10:08:57.677956+0000","last_became_peered":"2024-04-25T10:08:57.677956+0000","last_unstale":"2024-04-25T10:08:58.787638+0000","last_undegraded":"2024-04-25T10:08:58.787638+0000","last_fullsized":"2024-04-25T10:08:58.787638+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:53.652726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.788312+0000","last_change":"2024-04-25T10:08:57.681165+0000","last_active":"2024-04-25T10:08:58.788312+0000","last_peered":"2024-04-25T10:08:58.788312+0000","last_clean":"2024-04-25T10:08:58.788312+0000","last_became_active":"2024-04-25T10:08:57.680790+0000","last_became_peered":"2024-04-25T10:08:57.680790+0000","last_unstale":"2024-04-25T10:08:58.788312+0000","last_undegraded":"2024-04-25T10:08:58.788312+0000","last_fullsized":"2024-04-25T10:08:58.788312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:20.444343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648545+0000","last_change":"2024-04-25T10:08:57.674615+0000","last_active":"2024-04-25T10:08:58.648545+0000","last_peered":"2024-04-25T10:08:58.648545+0000","last_clean":"2024-04-25T10:08:58.648545+0000","last_became_active":"2024-04-25T10:08:57.673956+0000","last_became_peered":"2024-04-25T10:08:57.673956+0000","last_unstale":"2024-04-25T10:08:58.648545+0000","last_undegraded":"2024-04-25T10:08:58.648545+0000","last_fullsized":"2024-04-25T10:08:58.648545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:50.730066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696388+0000","last_change":"2024-04-25T10:08:57.657413+0000","last_active":"2024-04-25T10:08:58.696388+0000","last_peered":"2024-04-25T10:08:58.696388+0000","last_clean":"2024-04-25T10:08:58.696388+0000","last_became_active":"2024-04-25T10:08:57.656482+0000","last_became_peered":"2024-04-25T10:08:57.656482+0000","last_unstale":"2024-04-25T10:08:58.696388+0000","last_undegraded":"2024-04-25T10:08:58.696388+0000","last_fullsized":"2024-04-25T10:08:58.696388+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:48.484066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652559+0000","last_change":"2024-04-25T10:08:57.663001+0000","last_active":"2024-04-25T10:08:58.652559+0000","last_peered":"2024-04-25T10:08:58.652559+0000","last_clean":"2024-04-25T10:08:58.652559+0000","last_became_active":"2024-04-25T10:08:57.662678+0000","last_became_peered":"2024-04-25T10:08:57.662678+0000","last_unstale":"2024-04-25T10:08:58.652559+0000","last_undegraded":"2024-04-25T10:08:58.652559+0000","last_fullsized":"2024-04-25T10:08:58.652559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:23.905676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787480+0000","last_change":"2024-04-25T10:08:57.684275+0000","last_active":"2024-04-25T10:08:58.787480+0000","last_peered":"2024-04-25T10:08:58.787480+0000","last_clean":"2024-04-25T10:08:58.787480+0000","last_became_active":"2024-04-25T10:08:57.681876+0000","last_became_peered":"2024-04-25T10:08:57.681876+0000","last_unstale":"2024-04-25T10:08:58.787480+0000","last_undegraded":"2024-04-25T10:08:58.787480+0000","last_fullsized":"2024-04-25T10:08:58.787480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:33.253927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696100+0000","last_change":"2024-04-25T10:08:57.679953+0000","last_active":"2024-04-25T10:08:58.696100+0000","last_peered":"2024-04-25T10:08:58.696100+0000","last_clean":"2024-04-25T10:08:58.696100+0000","last_became_active":"2024-04-25T10:08:57.679274+0000","last_became_peered":"2024-04-25T10:08:57.679274+0000","last_unstale":"2024-04-25T10:08:58.696100+0000","last_undegraded":"2024-04-25T10:08:58.696100+0000","last_fullsized":"2024-04-25T10:08:58.696100+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:12.146272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689349+0000","last_change":"2024-04-25T10:08:57.671918+0000","last_active":"2024-04-25T10:08:58.689349+0000","last_peered":"2024-04-25T10:08:58.689349+0000","last_clean":"2024-04-25T10:08:58.689349+0000","last_became_active":"2024-04-25T10:08:57.670912+0000","last_became_peered":"2024-04-25T10:08:57.670912+0000","last_unstale":"2024-04-25T10:08:58.689349+0000","last_undegraded":"2024-04-25T10:08:58.689349+0000","last_fullsized":"2024-04-25T10:08:58.689349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:38.628073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648285+0000","last_change":"2024-04-25T10:08:57.679998+0000","last_active":"2024-04-25T10:08:58.648285+0000","last_peered":"2024-04-25T10:08:58.648285+0000","last_clean":"2024-04-25T10:08:58.648285+0000","last_became_active":"2024-04-25T10:08:57.679588+0000","last_became_peered":"2024-04-25T10:08:57.679588+0000","last_unstale":"2024-04-25T10:08:58.648285+0000","last_undegraded":"2024-04-25T10:08:58.648285+0000","last_fullsized":"2024-04-25T10:08:58.648285+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:12.818134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786482+0000","last_change":"2024-04-25T10:08:57.679167+0000","last_active":"2024-04-25T10:08:58.786482+0000","last_peered":"2024-04-25T10:08:58.786482+0000","last_clean":"2024-04-25T10:08:58.786482+0000","last_became_active":"2024-04-25T10:08:57.678468+0000","last_became_peered":"2024-04-25T10:08:57.678468+0000","last_unstale":"2024-04-25T10:08:58.786482+0000","last_undegraded":"2024-04-25T10:08:58.786482+0000","last_fullsized":"2024-04-25T10:08:58.786482+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:20.623071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.690310+0000","last_change":"2024-04-25T10:08:56.681068+0000","last_active":"2024-04-25T10:08:58.690310+0000","last_peered":"2024-04-25T10:08:58.690310+0000","last_clean":"2024-04-25T10:08:58.690310+0000","last_became_active":"2024-04-25T10:08:56.680598+0000","last_became_peered":"2024-04-25T10:08:56.680598+0000","last_unstale":"2024-04-25T10:08:58.690310+0000","last_undegraded":"2024-04-25T10:08:58.690310+0000","last_fullsized":"2024-04-25T10:08:58.690310+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:11.560129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.788324+0000","last_change":"2024-04-25T10:08:57.680540+0000","last_active":"2024-04-25T10:08:58.788324+0000","last_peered":"2024-04-25T10:08:58.788324+0000","last_clean":"2024-04-25T10:08:58.788324+0000","last_became_active":"2024-04-25T10:08:57.680271+0000","last_became_peered":"2024-04-25T10:08:57.680271+0000","last_unstale":"2024-04-25T10:08:58.788324+0000","last_undegraded":"2024-04-25T10:08:58.788324+0000","last_fullsized":"2024-04-25T10:08:58.788324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:12.146272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.697034+0000","last_change":"2024-04-25T10:08:56.687541+0000","last_active":"2024-04-25T10:08:58.697034+0000","last_peered":"2024-04-25T10:08:58.697034+0000","last_clean":"2024-04-25T10:08:58.697034+0000","last_became_active":"2024-04-25T10:08:56.685968+0000","last_became_peered":"2024-04-25T10:08:56.685968+0000","last_unstale":"2024-04-25T10:08:58.697034+0000","last_undegraded":"2024-04-25T10:08:58.697034+0000","last_fullsized":"2024-04-25T10:08:58.697034+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:49.147091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652439+0000","last_change":"2024-04-25T10:08:57.675214+0000","last_active":"2024-04-25T10:08:58.652439+0000","last_peered":"2024-04-25T10:08:58.652439+0000","last_clean":"2024-04-25T10:08:58.652439+0000","last_became_active":"2024-04-25T10:08:57.674848+0000","last_became_peered":"2024-04-25T10:08:57.674848+0000","last_unstale":"2024-04-25T10:08:58.652439+0000","last_undegraded":"2024-04-25T10:08:58.652439+0000","last_fullsized":"2024-04-25T10:08:58.652439+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:51.183347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786107+0000","last_change":"2024-04-25T10:08:56.677307+0000","last_active":"2024-04-25T10:08:58.786107+0000","last_peered":"2024-04-25T10:08:58.786107+0000","last_clean":"2024-04-25T10:08:58.786107+0000","last_became_active":"2024-04-25T10:08:56.676965+0000","last_became_peered":"2024-04-25T10:08:56.676965+0000","last_unstale":"2024-04-25T10:08:58.786107+0000","last_undegraded":"2024-04-25T10:08:58.786107+0000","last_fullsized":"2024-04-25T10:08:58.786107+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:49.147091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786025+0000","last_change":"2024-04-25T10:08:57.678144+0000","last_active":"2024-04-25T10:08:58.786025+0000","last_peered":"2024-04-25T10:08:58.786025+0000","last_clean":"2024-04-25T10:08:58.786025+0000","last_became_active":"2024-04-25T10:08:57.677858+0000","last_became_peered":"2024-04-25T10:08:57.677858+0000","last_unstale":"2024-04-25T10:08:58.786025+0000","last_undegraded":"2024-04-25T10:08:58.786025+0000","last_fullsized":"2024-04-25T10:08:58.786025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:34.131794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.649788+0000","last_change":"2024-04-25T10:08:56.681797+0000","last_active":"2024-04-25T10:08:58.649788+0000","last_peered":"2024-04-25T10:08:58.649788+0000","last_clean":"2024-04-25T10:08:58.649788+0000","last_became_active":"2024-04-25T10:08:56.681518+0000","last_became_peered":"2024-04-25T10:08:56.681518+0000","last_unstale":"2024-04-25T10:08:58.649788+0000","last_undegraded":"2024-04-25T10:08:58.649788+0000","last_fullsized":"2024-04-25T10:08:58.649788+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:43.317333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696239+0000","last_change":"2024-04-25T10:08:57.678092+0000","last_active":"2024-04-25T10:08:58.696239+0000","last_peered":"2024-04-25T10:08:58.696239+0000","last_clean":"2024-04-25T10:08:58.696239+0000","last_became_active":"2024-04-25T10:08:57.677130+0000","last_became_peered":"2024-04-25T10:08:57.677130+0000","last_unstale":"2024-04-25T10:08:58.696239+0000","last_undegraded":"2024-04-25T10:08:58.696239+0000","last_fullsized":"2024-04-25T10:08:58.696239+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:46.333691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689843+0000","last_change":"2024-04-25T10:08:56.683994+0000","last_active":"2024-04-25T10:08:58.689843+0000","last_peered":"2024-04-25T10:08:58.689843+0000","last_clean":"2024-04-25T10:08:58.689843+0000","last_became_active":"2024-04-25T10:08:56.683686+0000","last_became_peered":"2024-04-25T10:08:56.683686+0000","last_unstale":"2024-04-25T10:08:58.689843+0000","last_undegraded":"2024-04-25T10:08:58.689843+0000","last_fullsized":"2024-04-25T10:08:58.689843+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:25.673661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786201+0000","last_change":"2024-04-25T10:08:57.674573+0000","last_active":"2024-04-25T10:08:58.786201+0000","last_peered":"2024-04-25T10:08:58.786201+0000","last_clean":"2024-04-25T10:08:58.786201+0000","last_became_active":"2024-04-25T10:08:57.674280+0000","last_became_peered":"2024-04-25T10:08:57.674280+0000","last_unstale":"2024-04-25T10:08:58.786201+0000","last_undegraded":"2024-04-25T10:08:58.786201+0000","last_fullsized":"2024-04-25T10:08:58.786201+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:04.281461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696252+0000","last_change":"2024-04-25T10:08:56.682809+0000","last_active":"2024-04-25T10:08:58.696252+0000","last_peered":"2024-04-25T10:08:58.696252+0000","last_clean":"2024-04-25T10:08:58.696252+0000","last_became_active":"2024-04-25T10:08:56.682522+0000","last_became_peered":"2024-04-25T10:08:56.682522+0000","last_unstale":"2024-04-25T10:08:58.696252+0000","last_undegraded":"2024-04-25T10:08:58.696252+0000","last_fullsized":"2024-04-25T10:08:58.696252+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:12.104648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785876+0000","last_change":"2024-04-25T10:08:57.683466+0000","last_active":"2024-04-25T10:08:58.785876+0000","last_peered":"2024-04-25T10:08:58.785876+0000","last_clean":"2024-04-25T10:08:58.785876+0000","last_became_active":"2024-04-25T10:08:57.682853+0000","last_became_peered":"2024-04-25T10:08:57.682853+0000","last_unstale":"2024-04-25T10:08:58.785876+0000","last_undegraded":"2024-04-25T10:08:58.785876+0000","last_fullsized":"2024-04-25T10:08:58.785876+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.688922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.651454+0000","last_change":"2024-04-25T10:08:56.671637+0000","last_active":"2024-04-25T10:08:58.651454+0000","last_peered":"2024-04-25T10:08:58.651454+0000","last_clean":"2024-04-25T10:08:58.651454+0000","last_became_active":"2024-04-25T10:08:56.671141+0000","last_became_peered":"2024-04-25T10:08:56.671141+0000","last_unstale":"2024-04-25T10:08:58.651454+0000","last_undegraded":"2024-04-25T10:08:58.651454+0000","last_fullsized":"2024-04-25T10:08:58.651454+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:33.116533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787164+0000","last_change":"2024-04-25T10:08:57.678769+0000","last_active":"2024-04-25T10:08:58.787164+0000","last_peered":"2024-04-25T10:08:58.787164+0000","last_clean":"2024-04-25T10:08:58.787164+0000","last_became_active":"2024-04-25T10:08:57.678505+0000","last_became_peered":"2024-04-25T10:08:57.678505+0000","last_unstale":"2024-04-25T10:08:58.787164+0000","last_undegraded":"2024-04-25T10:08:58.787164+0000","last_fullsized":"2024-04-25T10:08:58.787164+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:31.382081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785730+0000","last_change":"2024-04-25T10:08:56.674412+0000","last_active":"2024-04-25T10:08:58.785730+0000","last_peered":"2024-04-25T10:08:58.785730+0000","last_clean":"2024-04-25T10:08:58.785730+0000","last_became_active":"2024-04-25T10:08:56.674105+0000","last_became_peered":"2024-04-25T10:08:56.674105+0000","last_unstale":"2024-04-25T10:08:58.785730+0000","last_undegraded":"2024-04-25T10:08:58.785730+0000","last_fullsized":"2024-04-25T10:08:58.785730+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:12.104648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696866+0000","last_change":"2024-04-25T10:08:57.670926+0000","last_active":"2024-04-25T10:08:58.696866+0000","last_peered":"2024-04-25T10:08:58.696866+0000","last_clean":"2024-04-25T10:08:58.696866+0000","last_became_active":"2024-04-25T10:08:57.668694+0000","last_became_peered":"2024-04-25T10:08:57.668694+0000","last_unstale":"2024-04-25T10:08:58.696866+0000","last_undegraded":"2024-04-25T10:08:58.696866+0000","last_fullsized":"2024-04-25T10:08:58.696866+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:04.281461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.697039+0000","last_change":"2024-04-25T10:08:56.685708+0000","last_active":"2024-04-25T10:08:58.697039+0000","last_peered":"2024-04-25T10:08:58.697039+0000","last_clean":"2024-04-25T10:08:58.697039+0000","last_became_active":"2024-04-25T10:08:56.685324+0000","last_became_peered":"2024-04-25T10:08:56.685324+0000","last_unstale":"2024-04-25T10:08:58.697039+0000","last_undegraded":"2024-04-25T10:08:58.697039+0000","last_fullsized":"2024-04-25T10:08:58.697039+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:53.946213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652081+0000","last_change":"2024-04-25T10:08:57.676105+0000","last_active":"2024-04-25T10:08:58.652081+0000","last_peered":"2024-04-25T10:08:58.652081+0000","last_clean":"2024-04-25T10:08:58.652081+0000","last_became_active":"2024-04-25T10:08:57.675476+0000","last_became_peered":"2024-04-25T10:08:57.675476+0000","last_unstale":"2024-04-25T10:08:58.652081+0000","last_undegraded":"2024-04-25T10:08:58.652081+0000","last_fullsized":"2024-04-25T10:08:58.652081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:28.606869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787334+0000","last_change":"2024-04-25T10:08:56.681228+0000","last_active":"2024-04-25T10:08:58.787334+0000","last_peered":"2024-04-25T10:08:58.787334+0000","last_clean":"2024-04-25T10:08:58.787334+0000","last_became_active":"2024-04-25T10:08:56.680351+0000","last_became_peered":"2024-04-25T10:08:56.680351+0000","last_unstale":"2024-04-25T10:08:58.787334+0000","last_undegraded":"2024-04-25T10:08:58.787334+0000","last_fullsized":"2024-04-25T10:08:58.787334+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:30.366820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.690254+0000","last_change":"2024-04-25T10:08:57.689932+0000","last_active":"2024-04-25T10:08:58.690254+0000","last_peered":"2024-04-25T10:08:58.690254+0000","last_clean":"2024-04-25T10:08:58.690254+0000","last_became_active":"2024-04-25T10:08:57.689667+0000","last_became_peered":"2024-04-25T10:08:57.689667+0000","last_unstale":"2024-04-25T10:08:58.690254+0000","last_undegraded":"2024-04-25T10:08:58.690254+0000","last_fullsized":"2024-04-25T10:08:58.690254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:57.472901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.650810+0000","last_change":"2024-04-25T10:08:56.684316+0000","last_active":"2024-04-25T10:08:58.650810+0000","last_peered":"2024-04-25T10:08:58.650810+0000","last_clean":"2024-04-25T10:08:58.650810+0000","last_became_active":"2024-04-25T10:08:56.683994+0000","last_became_peered":"2024-04-25T10:08:56.683994+0000","last_unstale":"2024-04-25T10:08:58.650810+0000","last_undegraded":"2024-04-25T10:08:58.650810+0000","last_fullsized":"2024-04-25T10:08:58.650810+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:19.429082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.650195+0000","last_change":"2024-04-25T10:08:57.680662+0000","last_active":"2024-04-25T10:08:58.650195+0000","last_peered":"2024-04-25T10:08:58.650195+0000","last_clean":"2024-04-25T10:08:58.650195+0000","last_became_active":"2024-04-25T10:08:57.680321+0000","last_became_peered":"2024-04-25T10:08:57.680321+0000","last_unstale":"2024-04-25T10:08:58.650195+0000","last_undegraded":"2024-04-25T10:08:58.650195+0000","last_fullsized":"2024-04-25T10:08:58.650195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:23.905676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648754+0000","last_change":"2024-04-25T10:08:56.686139+0000","last_active":"2024-04-25T10:08:58.648754+0000","last_peered":"2024-04-25T10:08:58.648754+0000","last_clean":"2024-04-25T10:08:58.648754+0000","last_became_active":"2024-04-25T10:08:56.685879+0000","last_became_peered":"2024-04-25T10:08:56.685879+0000","last_unstale":"2024-04-25T10:08:58.648754+0000","last_undegraded":"2024-04-25T10:08:58.648754+0000","last_fullsized":"2024-04-25T10:08:58.648754+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:31.953062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787406+0000","last_change":"2024-04-25T10:08:57.686439+0000","last_active":"2024-04-25T10:08:58.787406+0000","last_peered":"2024-04-25T10:08:58.787406+0000","last_clean":"2024-04-25T10:08:58.787406+0000","last_became_active":"2024-04-25T10:08:57.683649+0000","last_became_peered":"2024-04-25T10:08:57.683649+0000","last_unstale":"2024-04-25T10:08:58.787406+0000","last_undegraded":"2024-04-25T10:08:58.787406+0000","last_fullsized":"2024-04-25T10:08:58.787406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:37.096413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.690108+0000","last_change":"2024-04-25T10:08:56.680711+0000","last_active":"2024-04-25T10:08:58.690108+0000","last_peered":"2024-04-25T10:08:58.690108+0000","last_clean":"2024-04-25T10:08:58.690108+0000","last_became_active":"2024-04-25T10:08:56.678963+0000","last_became_peered":"2024-04-25T10:08:56.678963+0000","last_unstale":"2024-04-25T10:08:58.690108+0000","last_undegraded":"2024-04-25T10:08:58.690108+0000","last_fullsized":"2024-04-25T10:08:58.690108+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:03.266200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786416+0000","last_change":"2024-04-25T10:08:57.695716+0000","last_active":"2024-04-25T10:08:58.786416+0000","last_peered":"2024-04-25T10:08:58.786416+0000","last_clean":"2024-04-25T10:08:58.786416+0000","last_became_active":"2024-04-25T10:08:57.695436+0000","last_became_peered":"2024-04-25T10:08:57.695436+0000","last_unstale":"2024-04-25T10:08:58.786416+0000","last_undegraded":"2024-04-25T10:08:58.786416+0000","last_fullsized":"2024-04-25T10:08:58.786416+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:23.905676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.650715+0000","last_change":"2024-04-25T10:08:56.681239+0000","last_active":"2024-04-25T10:08:58.650715+0000","last_peered":"2024-04-25T10:08:58.650715+0000","last_clean":"2024-04-25T10:08:58.650715+0000","last_became_active":"2024-04-25T10:08:56.680924+0000","last_became_peered":"2024-04-25T10:08:56.680924+0000","last_unstale":"2024-04-25T10:08:58.650715+0000","last_undegraded":"2024-04-25T10:08:58.650715+0000","last_fullsized":"2024-04-25T10:08:58.650715+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:06.422241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689451+0000","last_change":"2024-04-25T10:08:57.674724+0000","last_active":"2024-04-25T10:08:58.689451+0000","last_peered":"2024-04-25T10:08:58.689451+0000","last_clean":"2024-04-25T10:08:58.689451+0000","last_became_active":"2024-04-25T10:08:57.674437+0000","last_became_peered":"2024-04-25T10:08:57.674437+0000","last_unstale":"2024-04-25T10:08:58.689451+0000","last_undegraded":"2024-04-25T10:08:58.689451+0000","last_fullsized":"2024-04-25T10:08:58.689451+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:52.858454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689378+0000","last_change":"2024-04-25T10:08:56.673870+0000","last_active":"2024-04-25T10:08:58.689378+0000","last_peered":"2024-04-25T10:08:58.689378+0000","last_clean":"2024-04-25T10:08:58.689378+0000","last_became_active":"2024-04-25T10:08:56.673578+0000","last_became_peered":"2024-04-25T10:08:56.673578+0000","last_unstale":"2024-04-25T10:08:58.689378+0000","last_undegraded":"2024-04-25T10:08:58.689378+0000","last_fullsized":"2024-04-25T10:08:58.689378+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:47.468805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.650214+0000","last_change":"2024-04-25T10:08:57.682613+0000","last_active":"2024-04-25T10:08:58.650214+0000","last_peered":"2024-04-25T10:08:58.650214+0000","last_clean":"2024-04-25T10:08:58.650214+0000","last_became_active":"2024-04-25T10:08:57.682327+0000","last_became_peered":"2024-04-25T10:08:57.682327+0000","last_unstale":"2024-04-25T10:08:58.650214+0000","last_undegraded":"2024-04-25T10:08:58.650214+0000","last_fullsized":"2024-04-25T10:08:58.650214+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:28.762976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":22,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785990+0000","last_change":"2024-04-25T10:08:56.680341+0000","last_active":"2024-04-25T10:08:58.785990+0000","last_peered":"2024-04-25T10:08:58.785990+0000","last_clean":"2024-04-25T10:08:58.785990+0000","last_became_active":"2024-04-25T10:08:56.680060+0000","last_became_peered":"2024-04-25T10:08:56.680060+0000","last_unstale":"2024-04-25T10:08:58.785990+0000","last_undegraded":"2024-04-25T10:08:58.785990+0000","last_fullsized":"2024-04-25T10:08:58.785990+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:53.946213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648929+0000","last_change":"2024-04-25T10:08:57.656725+0000","last_active":"2024-04-25T10:08:58.648929+0000","last_peered":"2024-04-25T10:08:58.648929+0000","last_clean":"2024-04-25T10:08:58.648929+0000","last_became_active":"2024-04-25T10:08:57.655783+0000","last_became_peered":"2024-04-25T10:08:57.655783+0000","last_unstale":"2024-04-25T10:08:58.648929+0000","last_undegraded":"2024-04-25T10:08:58.648929+0000","last_fullsized":"2024-04-25T10:08:58.648929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:13.181139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648537+0000","last_change":"2024-04-25T10:08:56.673276+0000","last_active":"2024-04-25T10:08:58.648537+0000","last_peered":"2024-04-25T10:08:58.648537+0000","last_clean":"2024-04-25T10:08:58.648537+0000","last_became_active":"2024-04-25T10:08:56.672940+0000","last_became_peered":"2024-04-25T10:08:56.672940+0000","last_unstale":"2024-04-25T10:08:58.648537+0000","last_undegraded":"2024-04-25T10:08:58.648537+0000","last_fullsized":"2024-04-25T10:08:58.648537+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:23.719316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787564+0000","last_change":"2024-04-25T10:08:57.681846+0000","last_active":"2024-04-25T10:08:58.787564+0000","last_peered":"2024-04-25T10:08:58.787564+0000","last_clean":"2024-04-25T10:08:58.787564+0000","last_became_active":"2024-04-25T10:08:57.681577+0000","last_became_peered":"2024-04-25T10:08:57.681577+0000","last_unstale":"2024-04-25T10:08:58.787564+0000","last_undegraded":"2024-04-25T10:08:58.787564+0000","last_fullsized":"2024-04-25T10:08:58.787564+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:45.096249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.697054+0000","last_change":"2024-04-25T10:08:56.688492+0000","last_active":"2024-04-25T10:08:58.697054+0000","last_peered":"2024-04-25T10:08:58.697054+0000","last_clean":"2024-04-25T10:08:58.697054+0000","last_became_active":"2024-04-25T10:08:56.688245+0000","last_became_peered":"2024-04-25T10:08:56.688245+0000","last_unstale":"2024-04-25T10:08:58.697054+0000","last_undegraded":"2024-04-25T10:08:58.697054+0000","last_fullsized":"2024-04-25T10:08:58.697054+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:30.366820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787660+0000","last_change":"2024-04-25T10:08:57.675803+0000","last_active":"2024-04-25T10:08:58.787660+0000","last_peered":"2024-04-25T10:08:58.787660+0000","last_clean":"2024-04-25T10:08:58.787660+0000","last_became_active":"2024-04-25T10:08:57.675512+0000","last_became_peered":"2024-04-25T10:08:57.675512+0000","last_unstale":"2024-04-25T10:08:58.787660+0000","last_undegraded":"2024-04-25T10:08:58.787660+0000","last_fullsized":"2024-04-25T10:08:58.787660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:34.131794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652189+0000","last_change":"2024-04-25T10:08:56.670169+0000","last_active":"2024-04-25T10:08:58.652189+0000","last_peered":"2024-04-25T10:08:58.652189+0000","last_clean":"2024-04-25T10:08:58.652189+0000","last_became_active":"2024-04-25T10:08:56.669659+0000","last_became_peered":"2024-04-25T10:08:56.669659+0000","last_unstale":"2024-04-25T10:08:58.652189+0000","last_undegraded":"2024-04-25T10:08:58.652189+0000","last_fullsized":"2024-04-25T10:08:58.652189+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:43.317333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.788000+0000","last_change":"2024-04-25T10:08:57.684029+0000","last_active":"2024-04-25T10:08:58.788000+0000","last_peered":"2024-04-25T10:08:58.788000+0000","last_clean":"2024-04-25T10:08:58.788000+0000","last_became_active":"2024-04-25T10:08:57.681955+0000","last_became_peered":"2024-04-25T10:08:57.681955+0000","last_unstale":"2024-04-25T10:08:58.788000+0000","last_undegraded":"2024-04-25T10:08:58.788000+0000","last_fullsized":"2024-04-25T10:08:58.788000+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:44.332595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.690116+0000","last_change":"2024-04-25T10:08:56.680963+0000","last_active":"2024-04-25T10:08:58.690116+0000","last_peered":"2024-04-25T10:08:58.690116+0000","last_clean":"2024-04-25T10:08:58.690116+0000","last_became_active":"2024-04-25T10:08:56.680666+0000","last_became_peered":"2024-04-25T10:08:56.680666+0000","last_unstale":"2024-04-25T10:08:58.690116+0000","last_undegraded":"2024-04-25T10:08:58.690116+0000","last_fullsized":"2024-04-25T10:08:58.690116+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:34.526289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689601+0000","last_change":"2024-04-25T10:08:57.675117+0000","last_active":"2024-04-25T10:08:58.689601+0000","last_peered":"2024-04-25T10:08:58.689601+0000","last_clean":"2024-04-25T10:08:58.689601+0000","last_became_active":"2024-04-25T10:08:57.674871+0000","last_became_peered":"2024-04-25T10:08:57.674871+0000","last_unstale":"2024-04-25T10:08:58.689601+0000","last_undegraded":"2024-04-25T10:08:58.689601+0000","last_fullsized":"2024-04-25T10:08:58.689601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:19.093937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.690351+0000","last_change":"2024-04-25T10:08:56.683965+0000","last_active":"2024-04-25T10:08:58.690351+0000","last_peered":"2024-04-25T10:08:58.690351+0000","last_clean":"2024-04-25T10:08:58.690351+0000","last_became_active":"2024-04-25T10:08:56.683697+0000","last_became_peered":"2024-04-25T10:08:56.683697+0000","last_unstale":"2024-04-25T10:08:58.690351+0000","last_undegraded":"2024-04-25T10:08:58.690351+0000","last_fullsized":"2024-04-25T10:08:58.690351+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:50.168086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785604+0000","last_change":"2024-04-25T10:08:57.680867+0000","last_active":"2024-04-25T10:08:58.785604+0000","last_peered":"2024-04-25T10:08:58.785604+0000","last_clean":"2024-04-25T10:08:58.785604+0000","last_became_active":"2024-04-25T10:08:57.680450+0000","last_became_peered":"2024-04-25T10:08:57.680450+0000","last_unstale":"2024-04-25T10:08:58.785604+0000","last_undegraded":"2024-04-25T10:08:58.785604+0000","last_fullsized":"2024-04-25T10:08:58.785604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:53.652726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.651528+0000","last_change":"2024-04-25T10:08:56.680574+0000","last_active":"2024-04-25T10:08:58.651528+0000","last_peered":"2024-04-25T10:08:58.651528+0000","last_clean":"2024-04-25T10:08:58.651528+0000","last_became_active":"2024-04-25T10:08:56.680200+0000","last_became_peered":"2024-04-25T10:08:56.680200+0000","last_unstale":"2024-04-25T10:08:58.651528+0000","last_undegraded":"2024-04-25T10:08:58.651528+0000","last_fullsized":"2024-04-25T10:08:58.651528+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:19.607810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648323+0000","last_change":"2024-04-25T10:08:57.676997+0000","last_active":"2024-04-25T10:08:58.648323+0000","last_peered":"2024-04-25T10:08:58.648323+0000","last_clean":"2024-04-25T10:08:58.648323+0000","last_became_active":"2024-04-25T10:08:57.676317+0000","last_became_peered":"2024-04-25T10:08:57.676317+0000","last_unstale":"2024-04-25T10:08:58.648323+0000","last_undegraded":"2024-04-25T10:08:58.648323+0000","last_fullsized":"2024-04-25T10:08:58.648323+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.510935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785857+0000","last_change":"2024-04-25T10:08:56.678727+0000","last_active":"2024-04-25T10:08:58.785857+0000","last_peered":"2024-04-25T10:08:58.785857+0000","last_clean":"2024-04-25T10:08:58.785857+0000","last_became_active":"2024-04-25T10:08:56.678451+0000","last_became_peered":"2024-04-25T10:08:56.678451+0000","last_unstale":"2024-04-25T10:08:58.785857+0000","last_undegraded":"2024-04-25T10:08:58.785857+0000","last_fullsized":"2024-04-25T10:08:58.785857+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:12.165878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785604+0000","last_change":"2024-04-25T10:08:57.680196+0000","last_active":"2024-04-25T10:08:58.785604+0000","last_peered":"2024-04-25T10:08:58.785604+0000","last_clean":"2024-04-25T10:08:58.785604+0000","last_became_active":"2024-04-25T10:08:57.679865+0000","last_became_peered":"2024-04-25T10:08:57.679865+0000","last_unstale":"2024-04-25T10:08:58.785604+0000","last_undegraded":"2024-04-25T10:08:58.785604+0000","last_fullsized":"2024-04-25T10:08:58.785604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:35.541551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.649498+0000","last_change":"2024-04-25T10:08:56.680808+0000","last_active":"2024-04-25T10:08:58.649498+0000","last_peered":"2024-04-25T10:08:58.649498+0000","last_clean":"2024-04-25T10:08:58.649498+0000","last_became_active":"2024-04-25T10:08:56.680518+0000","last_became_peered":"2024-04-25T10:08:56.680518+0000","last_unstale":"2024-04-25T10:08:58.649498+0000","last_undegraded":"2024-04-25T10:08:58.649498+0000","last_fullsized":"2024-04-25T10:08:58.649498+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:19.607810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.788552+0000","last_change":"2024-04-25T10:08:57.682048+0000","last_active":"2024-04-25T10:08:58.788552+0000","last_peered":"2024-04-25T10:08:58.788552+0000","last_clean":"2024-04-25T10:08:58.788552+0000","last_became_active":"2024-04-25T10:08:57.681653+0000","last_became_peered":"2024-04-25T10:08:57.681653+0000","last_unstale":"2024-04-25T10:08:58.788552+0000","last_undegraded":"2024-04-25T10:08:58.788552+0000","last_fullsized":"2024-04-25T10:08:58.788552+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:07.437502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.690217+0000","last_change":"2024-04-25T10:08:56.685894+0000","last_active":"2024-04-25T10:08:58.690217+0000","last_peered":"2024-04-25T10:08:58.690217+0000","last_clean":"2024-04-25T10:08:58.690217+0000","last_became_active":"2024-04-25T10:08:56.685605+0000","last_became_peered":"2024-04-25T10:08:56.685605+0000","last_unstale":"2024-04-25T10:08:58.690217+0000","last_undegraded":"2024-04-25T10:08:58.690217+0000","last_fullsized":"2024-04-25T10:08:58.690217+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:15.693042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787099+0000","last_change":"2024-04-25T10:08:57.670495+0000","last_active":"2024-04-25T10:08:58.787099+0000","last_peered":"2024-04-25T10:08:58.787099+0000","last_clean":"2024-04-25T10:08:58.787099+0000","last_became_active":"2024-04-25T10:08:57.670174+0000","last_became_peered":"2024-04-25T10:08:57.670174+0000","last_unstale":"2024-04-25T10:08:58.787099+0000","last_undegraded":"2024-04-25T10:08:58.787099+0000","last_fullsized":"2024-04-25T10:08:58.787099+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:45.063754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.651122+0000","last_change":"2024-04-25T10:08:56.684331+0000","last_active":"2024-04-25T10:08:58.651122+0000","last_peered":"2024-04-25T10:08:58.651122+0000","last_clean":"2024-04-25T10:08:58.651122+0000","last_became_active":"2024-04-25T10:08:56.684083+0000","last_became_peered":"2024-04-25T10:08:56.684083+0000","last_unstale":"2024-04-25T10:08:58.651122+0000","last_undegraded":"2024-04-25T10:08:58.651122+0000","last_fullsized":"2024-04-25T10:08:58.651122+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.098938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.650676+0000","last_change":"2024-04-25T10:08:57.676739+0000","last_active":"2024-04-25T10:08:58.650676+0000","last_peered":"2024-04-25T10:08:58.650676+0000","last_clean":"2024-04-25T10:08:58.650676+0000","last_became_active":"2024-04-25T10:08:57.676444+0000","last_became_peered":"2024-04-25T10:08:57.676444+0000","last_unstale":"2024-04-25T10:08:58.650676+0000","last_undegraded":"2024-04-25T10:08:58.650676+0000","last_fullsized":"2024-04-25T10:08:58.650676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:45.063754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785169+0000","last_change":"2024-04-25T10:08:56.670086+0000","last_active":"2024-04-25T10:08:58.785169+0000","last_peered":"2024-04-25T10:08:58.785169+0000","last_clean":"2024-04-25T10:08:58.785169+0000","last_became_active":"2024-04-25T10:08:56.669777+0000","last_became_peered":"2024-04-25T10:08:56.669777+0000","last_unstale":"2024-04-25T10:08:58.785169+0000","last_undegraded":"2024-04-25T10:08:58.785169+0000","last_fullsized":"2024-04-25T10:08:58.785169+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:17.639165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689744+0000","last_change":"2024-04-25T10:08:57.674964+0000","last_active":"2024-04-25T10:08:58.689744+0000","last_peered":"2024-04-25T10:08:58.689744+0000","last_clean":"2024-04-25T10:08:58.689744+0000","last_became_active":"2024-04-25T10:08:57.673115+0000","last_became_peered":"2024-04-25T10:08:57.673115+0000","last_unstale":"2024-04-25T10:08:58.689744+0000","last_undegraded":"2024-04-25T10:08:58.689744+0000","last_fullsized":"2024-04-25T10:08:58.689744+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:33.480358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.650703+0000","last_change":"2024-04-25T10:08:56.685961+0000","last_active":"2024-04-25T10:08:58.650703+0000","last_peered":"2024-04-25T10:08:58.650703+0000","last_clean":"2024-04-25T10:08:58.650703+0000","last_became_active":"2024-04-25T10:08:56.685257+0000","last_became_peered":"2024-04-25T10:08:56.685257+0000","last_unstale":"2024-04-25T10:08:58.650703+0000","last_undegraded":"2024-04-25T10:08:58.650703+0000","last_fullsized":"2024-04-25T10:08:58.650703+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:11.131011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689867+0000","last_change":"2024-04-25T10:08:57.674696+0000","last_active":"2024-04-25T10:08:58.689867+0000","last_peered":"2024-04-25T10:08:58.689867+0000","last_clean":"2024-04-25T10:08:58.689867+0000","last_became_active":"2024-04-25T10:08:57.673931+0000","last_became_peered":"2024-04-25T10:08:57.673931+0000","last_unstale":"2024-04-25T10:08:58.689867+0000","last_undegraded":"2024-04-25T10:08:58.689867+0000","last_fullsized":"2024-04-25T10:08:58.689867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:56.595202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696354+0000","last_change":"2024-04-25T10:08:56.682120+0000","last_active":"2024-04-25T10:08:58.696354+0000","last_peered":"2024-04-25T10:08:58.696354+0000","last_clean":"2024-04-25T10:08:58.696354+0000","last_became_active":"2024-04-25T10:08:56.681640+0000","last_became_peered":"2024-04-25T10:08:56.681640+0000","last_unstale":"2024-04-25T10:08:58.696354+0000","last_undegraded":"2024-04-25T10:08:58.696354+0000","last_fullsized":"2024-04-25T10:08:58.696354+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:17.639165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.651487+0000","last_change":"2024-04-25T10:08:57.673691+0000","last_active":"2024-04-25T10:08:58.651487+0000","last_peered":"2024-04-25T10:08:58.651487+0000","last_clean":"2024-04-25T10:08:58.651487+0000","last_became_active":"2024-04-25T10:08:57.673403+0000","last_became_peered":"2024-04-25T10:08:57.673403+0000","last_unstale":"2024-04-25T10:08:58.651487+0000","last_undegraded":"2024-04-25T10:08:58.651487+0000","last_fullsized":"2024-04-25T10:08:58.651487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:16.708303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785427+0000","last_change":"2024-04-25T10:08:56.681231+0000","last_active":"2024-04-25T10:08:58.785427+0000","last_peered":"2024-04-25T10:08:58.785427+0000","last_clean":"2024-04-25T10:08:58.785427+0000","last_became_active":"2024-04-25T10:08:56.680897+0000","last_became_peered":"2024-04-25T10:08:56.680897+0000","last_unstale":"2024-04-25T10:08:58.785427+0000","last_undegraded":"2024-04-25T10:08:58.785427+0000","last_fullsized":"2024-04-25T10:08:58.785427+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:30.366820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696109+0000","last_change":"2024-04-25T10:08:57.672456+0000","last_active":"2024-04-25T10:08:58.696109+0000","last_peered":"2024-04-25T10:08:58.696109+0000","last_clean":"2024-04-25T10:08:58.696109+0000","last_became_active":"2024-04-25T10:08:57.672181+0000","last_became_peered":"2024-04-25T10:08:57.672181+0000","last_unstale":"2024-04-25T10:08:58.696109+0000","last_undegraded":"2024-04-25T10:08:58.696109+0000","last_fullsized":"2024-04-25T10:08:58.696109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:07.437502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785980+0000","last_change":"2024-04-25T10:08:56.673731+0000","last_active":"2024-04-25T10:08:58.785980+0000","last_peered":"2024-04-25T10:08:58.785980+0000","last_clean":"2024-04-25T10:08:58.785980+0000","last_became_active":"2024-04-25T10:08:56.673457+0000","last_became_peered":"2024-04-25T10:08:56.673457+0000","last_unstale":"2024-04-25T10:08:58.785980+0000","last_undegraded":"2024-04-25T10:08:58.785980+0000","last_fullsized":"2024-04-25T10:08:58.785980+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:17.639165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787246+0000","last_change":"2024-04-25T10:08:57.693156+0000","last_active":"2024-04-25T10:08:58.787246+0000","last_peered":"2024-04-25T10:08:58.787246+0000","last_clean":"2024-04-25T10:08:58.787246+0000","last_became_active":"2024-04-25T10:08:57.692874+0000","last_became_peered":"2024-04-25T10:08:57.692874+0000","last_unstale":"2024-04-25T10:08:58.787246+0000","last_undegraded":"2024-04-25T10:08:58.787246+0000","last_fullsized":"2024-04-25T10:08:58.787246+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:23.774482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652469+0000","last_change":"2024-04-25T10:08:56.678464+0000","last_active":"2024-04-25T10:08:58.652469+0000","last_peered":"2024-04-25T10:08:58.652469+0000","last_clean":"2024-04-25T10:08:58.652469+0000","last_became_active":"2024-04-25T10:08:56.678121+0000","last_became_peered":"2024-04-25T10:08:56.678121+0000","last_unstale":"2024-04-25T10:08:58.652469+0000","last_undegraded":"2024-04-25T10:08:58.652469+0000","last_fullsized":"2024-04-25T10:08:58.652469+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:19.429082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.649937+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:08:58.649937+0000","last_peered":"2024-04-25T10:08:58.649937+0000","last_clean":"2024-04-25T10:08:58.649937+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:08:58.649937+0000","last_undegraded":"2024-04-25T10:08:58.649937+0000","last_fullsized":"2024-04-25T10:08:58.649937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":800128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":524288,"data_stored":210342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":209740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":211282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":210308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":211304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":209740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":800072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:09:02.072 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:09:02.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.305+0000 7fba8a000640 1 Processor -- start 2024-04-25T10:09:02.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.305+0000 7fba8a000640 1 -- start start 2024-04-25T10:09:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.306+0000 7fba8a000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fba8406e2e0 0x7fba840cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.306+0000 7fba8a000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fba840cfc40 0x7fba8415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.306+0000 7fba8a000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba840cdb10 con 0x7fba8415a870 2024-04-25T10:09:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.306+0000 7fba8a000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba840ce720 con 0x7fba8406e2e0 2024-04-25T10:09:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.306+0000 7fba8a000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba840cddf0 con 0x7fba840cfc40 2024-04-25T10:09:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.307+0000 7fba82a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fba8406e2e0 0x7fba840cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.307+0000 7fba83400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fba8415a870 0x7fba840cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42102/0 (socket says 172.21.15.119:42102) 2024-04-25T10:09:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.307+0000 7fba83400640 1 -- 172.21.15.119:0/954609268 learned_addr learned my addr 172.21.15.119:0/954609268 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.307+0000 7fba82a00640 1 -- 172.21.15.119:0/954609268 >> v2:172.21.15.132:3300/0 conn(0x7fba840cfc40 msgr2=0x7fba8415ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.307+0000 7fba83e00640 1 --2- 172.21.15.119:0/954609268 >> v2:172.21.15.132:3300/0 conn(0x7fba840cfc40 0x7fba8415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.307+0000 7fba82000640 1 -- 172.21.15.119:0/954609268 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448871449 0 0) 0x7fba840cdb10 con 0x7fba8415a870 2024-04-25T10:09:02.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.307+0000 7fba82a00640 1 --2- 172.21.15.119:0/954609268 >> v2:172.21.15.132:3300/0 conn(0x7fba840cfc40 0x7fba8415ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.307+0000 7fba82a00640 1 -- 172.21.15.119:0/954609268 >> v1:172.21.15.119:6789/0 conn(0x7fba8415a870 legacy=0x7fba840cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.307+0000 7fba82a00640 1 -- 172.21.15.119:0/954609268 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba840ce0d0 con 0x7fba8406e2e0 2024-04-25T10:09:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.308+0000 7fba82a00640 1 --2- 172.21.15.119:0/954609268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba8406e2e0 0x7fba840cebb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fba700097a0 tx=0x7fba7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f4d8aae1a9810d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.308+0000 7fba82000640 1 -- 172.21.15.119:0/954609268 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70037070 con 0x7fba8406e2e0 2024-04-25T10:09:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.309+0000 7fba82000640 1 -- 172.21.15.119:0/954609268 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba70032440 con 0x7fba8406e2e0 2024-04-25T10:09:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.309+0000 7fba82000640 1 -- 172.21.15.119:0/954609268 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70032810 con 0x7fba8406e2e0 2024-04-25T10:09:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.309+0000 7fba8a000640 1 -- 172.21.15.119:0/954609268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba8406e2e0 msgr2=0x7fba840cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.309+0000 7fba8a000640 1 --2- 172.21.15.119:0/954609268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba8406e2e0 0x7fba840cebb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fba700097a0 tx=0x7fba7002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.309+0000 7fba8a000640 1 -- 172.21.15.119:0/954609268 shutdown_connections 2024-04-25T10:09:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.309+0000 7fba8a000640 1 --2- 172.21.15.119:0/954609268 >> v2:172.21.15.132:3300/0 conn(0x7fba840cfc40 0x7fba8415ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.309+0000 7fba8a000640 1 --2- 172.21.15.119:0/954609268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba8406e2e0 0x7fba840cebb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.309+0000 7fba8a000640 1 -- 172.21.15.119:0/954609268 >> 172.21.15.119:0/954609268 conn(0x7fba8406aba0 msgr2=0x7fba8406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.309+0000 7fba8a000640 1 -- 172.21.15.119:0/954609268 shutdown_connections 2024-04-25T10:09:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.309+0000 7fba8a000640 1 -- 172.21.15.119:0/954609268 wait complete. 2024-04-25T10:09:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.311+0000 7fba8a000640 1 Processor -- start 2024-04-25T10:09:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.311+0000 7fba8a000640 1 -- start start 2024-04-25T10:09:02.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.312+0000 7fba8a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba8406e2e0 0x7fba8414aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.312+0000 7fba8a000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba840cfc40 0x7fba84158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.312+0000 7fba8a000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba8415a870 0x7fba840c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.312+0000 7fba8a000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba840cd700 con 0x7fba8406e2e0 2024-04-25T10:09:02.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.312+0000 7fba8a000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba840cd580 con 0x7fba8415a870 2024-04-25T10:09:02.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.312+0000 7fba8a000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba840cd880 con 0x7fba840cfc40 2024-04-25T10:09:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.313+0000 7fba83400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba8406e2e0 0x7fba8414aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.313+0000 7fba83400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba8406e2e0 0x7fba8414aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38272/0 (socket says 172.21.15.119:38272) 2024-04-25T10:09:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.313+0000 7fba83400640 1 -- 172.21.15.119:0/4164035900 learned_addr learned my addr 172.21.15.119:0/4164035900 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.313+0000 7fba83e00640 1 --2- 172.21.15.119:0/4164035900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba8415a870 0x7fba840c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.313+0000 7fba82a00640 1 --2- 172.21.15.119:0/4164035900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba840cfc40 0x7fba84158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.313+0000 7fba83400640 1 -- 172.21.15.119:0/4164035900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba840cfc40 msgr2=0x7fba84158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.313+0000 7fba83400640 1 --2- 172.21.15.119:0/4164035900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba840cfc40 0x7fba84158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.313+0000 7fba83400640 1 -- 172.21.15.119:0/4164035900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba8415a870 msgr2=0x7fba840c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.313+0000 7fba83400640 1 --2- 172.21.15.119:0/4164035900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba8415a870 0x7fba840c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.313+0000 7fba83400640 1 -- 172.21.15.119:0/4164035900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba840c44e0 con 0x7fba8406e2e0 2024-04-25T10:09:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.313+0000 7fba83400640 1 --2- 172.21.15.119:0/4164035900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba8406e2e0 0x7fba8414aee0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fba7400c430 tx=0x7fba7400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.314+0000 7fba7b400640 1 -- 172.21.15.119:0/4164035900 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba740074e0 con 0x7fba8406e2e0 2024-04-25T10:09:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.314+0000 7fba7b400640 1 -- 172.21.15.119:0/4164035900 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba740047e0 con 0x7fba8406e2e0 2024-04-25T10:09:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.314+0000 7fba7b400640 1 -- 172.21.15.119:0/4164035900 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74004bd0 con 0x7fba8406e2e0 2024-04-25T10:09:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.314+0000 7fba8a000640 1 -- 172.21.15.119:0/4164035900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba840c9440 con 0x7fba8406e2e0 2024-04-25T10:09:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.314+0000 7fba8a000640 1 -- 172.21.15.119:0/4164035900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba840c9a00 con 0x7fba8406e2e0 2024-04-25T10:09:02.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.315+0000 7fba8a000640 1 -- 172.21.15.119:0/4164035900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba840749b0 con 0x7fba8406e2e0 2024-04-25T10:09:02.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.316+0000 7fba7b400640 1 -- 172.21.15.119:0/4164035900 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba74011070 con 0x7fba8406e2e0 2024-04-25T10:09:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.317+0000 7fba7b400640 1 --2- 172.21.15.119:0/4164035900 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.317+0000 7fba7b400640 1 -- 172.21.15.119:0/4164035900 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba74089f10 con 0x7fba8406e2e0 2024-04-25T10:09:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.317+0000 7fba82a00640 1 --2- 172.21.15.119:0/4164035900 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.318+0000 7fba82a00640 1 --2- 172.21.15.119:0/4164035900 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fba6406b4d0 0x7fba6406d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fba70034000 tx=0x7fba7002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.321+0000 7fba7b400640 1 -- 172.21.15.119:0/4164035900 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba74058cf0 con 0x7fba8406e2e0 2024-04-25T10:09:02.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.511+0000 7fba8a000640 1 -- 172.21.15.119:0/4164035900 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fba8415c230 con 0x7fba6406b4d0 2024-04-25T10:09:02.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.529+0000 7fba7b400640 1 -- 172.21.15.119:0/4164035900 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251959 (secure 0 0 0) 0x7fba8415c230 con 0x7fba6406b4d0 2024-04-25T10:09:02.536 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:02.540 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:09:02.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.535+0000 7fba8a000640 1 -- 172.21.15.119:0/4164035900 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fba6406b4d0 msgr2=0x7fba6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.535+0000 7fba8a000640 1 --2- 172.21.15.119:0/4164035900 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fba6406b4d0 0x7fba6406d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fba70034000 tx=0x7fba7002e000 comp rx=0 tx=0).stop 2024-04-25T10:09:02.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.535+0000 7fba8a000640 1 -- 172.21.15.119:0/4164035900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba8406e2e0 msgr2=0x7fba8414aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.535+0000 7fba8a000640 1 --2- 172.21.15.119:0/4164035900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba8406e2e0 0x7fba8414aee0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fba7400c430 tx=0x7fba7400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.535+0000 7fba8a000640 1 -- 172.21.15.119:0/4164035900 shutdown_connections 2024-04-25T10:09:02.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.535+0000 7fba8a000640 1 --2- 172.21.15.119:0/4164035900 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.535+0000 7fba8a000640 1 --2- 172.21.15.119:0/4164035900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba8415a870 0x7fba840c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.535+0000 7fba8a000640 1 --2- 172.21.15.119:0/4164035900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba840cfc40 0x7fba84158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.535+0000 7fba8a000640 1 --2- 172.21.15.119:0/4164035900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba8406e2e0 0x7fba8414aee0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.535+0000 7fba8a000640 1 -- 172.21.15.119:0/4164035900 >> 172.21.15.119:0/4164035900 conn(0x7fba8406aba0 msgr2=0x7fba8415d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:02.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.535+0000 7fba8a000640 1 -- 172.21.15.119:0/4164035900 shutdown_connections 2024-04-25T10:09:02.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.535+0000 7fba8a000640 1 -- 172.21.15.119:0/4164035900 wait complete. 2024-04-25T10:09:02.560 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-25T10:09:01.960562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225624,"kb_used_data":5256,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749096,"statfs":{"total":773094113280,"available":772863074304,"internally_reserved":0,"allocated":5382144,"data_stored":2862916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001629"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.651254+0000","last_change":"2024-04-25T10:08:57.675804+0000","last_active":"2024-04-25T10:08:58.651254+0000","last_peered":"2024-04-25T10:08:58.651254+0000","last_clean":"2024-04-25T10:08:58.651254+0000","last_became_active":"2024-04-25T10:08:57.675475+0000","last_became_peered":"2024-04-25T10:08:57.675475+0000","last_unstale":"2024-04-25T10:08:58.651254+0000","last_undegraded":"2024-04-25T10:08:58.651254+0000","last_fullsized":"2024-04-25T10:08:58.651254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:18.005213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652124+0000","last_change":"2024-04-25T10:08:57.669569+0000","last_active":"2024-04-25T10:08:58.652124+0000","last_peered":"2024-04-25T10:08:58.652124+0000","last_clean":"2024-04-25T10:08:58.652124+0000","last_became_active":"2024-04-25T10:08:57.667496+0000","last_became_peered":"2024-04-25T10:08:57.667496+0000","last_unstale":"2024-04-25T10:08:58.652124+0000","last_undegraded":"2024-04-25T10:08:58.652124+0000","last_fullsized":"2024-04-25T10:08:58.652124+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:12.575391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696600+0000","last_change":"2024-04-25T10:08:57.673417+0000","last_active":"2024-04-25T10:08:58.696600+0000","last_peered":"2024-04-25T10:08:58.696600+0000","last_clean":"2024-04-25T10:08:58.696600+0000","last_became_active":"2024-04-25T10:08:57.672943+0000","last_became_peered":"2024-04-25T10:08:57.672943+0000","last_unstale":"2024-04-25T10:08:58.696600+0000","last_undegraded":"2024-04-25T10:08:58.696600+0000","last_fullsized":"2024-04-25T10:08:58.696600+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:16.708303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786408+0000","last_change":"2024-04-25T10:08:57.685694+0000","last_active":"2024-04-25T10:08:58.786408+0000","last_peered":"2024-04-25T10:08:58.786408+0000","last_clean":"2024-04-25T10:08:58.786408+0000","last_became_active":"2024-04-25T10:08:57.685407+0000","last_became_peered":"2024-04-25T10:08:57.685407+0000","last_unstale":"2024-04-25T10:08:58.786408+0000","last_undegraded":"2024-04-25T10:08:58.786408+0000","last_fullsized":"2024-04-25T10:08:58.786408+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:51.183347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652092+0000","last_change":"2024-04-25T10:08:57.672287+0000","last_active":"2024-04-25T10:08:58.652092+0000","last_peered":"2024-04-25T10:08:58.652092+0000","last_clean":"2024-04-25T10:08:58.652092+0000","last_became_active":"2024-04-25T10:08:57.667472+0000","last_became_peered":"2024-04-25T10:08:57.667472+0000","last_unstale":"2024-04-25T10:08:58.652092+0000","last_undegraded":"2024-04-25T10:08:58.652092+0000","last_fullsized":"2024-04-25T10:08:58.652092+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.688922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689358+0000","last_change":"2024-04-25T10:08:57.676771+0000","last_active":"2024-04-25T10:08:58.689358+0000","last_peered":"2024-04-25T10:08:58.689358+0000","last_clean":"2024-04-25T10:08:58.689358+0000","last_became_active":"2024-04-25T10:08:57.676470+0000","last_became_peered":"2024-04-25T10:08:57.676470+0000","last_unstale":"2024-04-25T10:08:58.689358+0000","last_undegraded":"2024-04-25T10:08:58.689358+0000","last_fullsized":"2024-04-25T10:08:58.689358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:56.856980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.650034+0000","last_change":"2024-04-25T10:08:57.681918+0000","last_active":"2024-04-25T10:08:58.650034+0000","last_peered":"2024-04-25T10:08:58.650034+0000","last_clean":"2024-04-25T10:08:58.650034+0000","last_became_active":"2024-04-25T10:08:57.680201+0000","last_became_peered":"2024-04-25T10:08:57.680201+0000","last_unstale":"2024-04-25T10:08:58.650034+0000","last_undegraded":"2024-04-25T10:08:58.650034+0000","last_fullsized":"2024-04-25T10:08:58.650034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:51.183347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785623+0000","last_change":"2024-04-25T10:08:57.683699+0000","last_active":"2024-04-25T10:08:58.785623+0000","last_peered":"2024-04-25T10:08:58.785623+0000","last_clean":"2024-04-25T10:08:58.785623+0000","last_became_active":"2024-04-25T10:08:57.683204+0000","last_became_peered":"2024-04-25T10:08:57.683204+0000","last_unstale":"2024-04-25T10:08:58.785623+0000","last_undegraded":"2024-04-25T10:08:58.785623+0000","last_fullsized":"2024-04-25T10:08:58.785623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:16.708303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787506+0000","last_change":"2024-04-25T10:08:57.691808+0000","last_active":"2024-04-25T10:08:58.787506+0000","last_peered":"2024-04-25T10:08:58.787506+0000","last_clean":"2024-04-25T10:08:58.787506+0000","last_became_active":"2024-04-25T10:08:57.691530+0000","last_became_peered":"2024-04-25T10:08:57.691530+0000","last_unstale":"2024-04-25T10:08:58.787506+0000","last_undegraded":"2024-04-25T10:08:58.787506+0000","last_fullsized":"2024-04-25T10:08:58.787506+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:34.555824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787129+0000","last_change":"2024-04-25T10:08:57.686575+0000","last_active":"2024-04-25T10:08:58.787129+0000","last_peered":"2024-04-25T10:08:58.787129+0000","last_clean":"2024-04-25T10:08:58.787129+0000","last_became_active":"2024-04-25T10:08:57.686330+0000","last_became_peered":"2024-04-25T10:08:57.686330+0000","last_unstale":"2024-04-25T10:08:58.787129+0000","last_undegraded":"2024-04-25T10:08:58.787129+0000","last_fullsized":"2024-04-25T10:08:58.787129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:28.606869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787118+0000","last_change":"2024-04-25T10:08:57.673701+0000","last_active":"2024-04-25T10:08:58.787118+0000","last_peered":"2024-04-25T10:08:58.787118+0000","last_clean":"2024-04-25T10:08:58.787118+0000","last_became_active":"2024-04-25T10:08:57.673385+0000","last_became_peered":"2024-04-25T10:08:57.673385+0000","last_unstale":"2024-04-25T10:08:58.787118+0000","last_undegraded":"2024-04-25T10:08:58.787118+0000","last_fullsized":"2024-04-25T10:08:58.787118+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:18.005213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652090+0000","last_change":"2024-04-25T10:08:57.666858+0000","last_active":"2024-04-25T10:08:58.652090+0000","last_peered":"2024-04-25T10:08:58.652090+0000","last_clean":"2024-04-25T10:08:58.652090+0000","last_became_active":"2024-04-25T10:08:57.666495+0000","last_became_peered":"2024-04-25T10:08:57.666495+0000","last_unstale":"2024-04-25T10:08:58.652090+0000","last_undegraded":"2024-04-25T10:08:58.652090+0000","last_fullsized":"2024-04-25T10:08:58.652090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:04.281461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648854+0000","last_change":"2024-04-25T10:08:57.674287+0000","last_active":"2024-04-25T10:08:58.648854+0000","last_peered":"2024-04-25T10:08:58.648854+0000","last_clean":"2024-04-25T10:08:58.648854+0000","last_became_active":"2024-04-25T10:08:57.673879+0000","last_became_peered":"2024-04-25T10:08:57.673879+0000","last_unstale":"2024-04-25T10:08:58.648854+0000","last_undegraded":"2024-04-25T10:08:58.648854+0000","last_fullsized":"2024-04-25T10:08:58.648854+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:35.341277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786378+0000","last_change":"2024-04-25T10:08:57.689718+0000","last_active":"2024-04-25T10:08:58.786378+0000","last_peered":"2024-04-25T10:08:58.786378+0000","last_clean":"2024-04-25T10:08:58.786378+0000","last_became_active":"2024-04-25T10:08:57.689378+0000","last_became_peered":"2024-04-25T10:08:57.689378+0000","last_unstale":"2024-04-25T10:08:58.786378+0000","last_undegraded":"2024-04-25T10:08:58.786378+0000","last_fullsized":"2024-04-25T10:08:58.786378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:12.575391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689289+0000","last_change":"2024-04-25T10:08:57.671577+0000","last_active":"2024-04-25T10:08:58.689289+0000","last_peered":"2024-04-25T10:08:58.689289+0000","last_clean":"2024-04-25T10:08:58.689289+0000","last_became_active":"2024-04-25T10:08:57.671239+0000","last_became_peered":"2024-04-25T10:08:57.671239+0000","last_unstale":"2024-04-25T10:08:58.689289+0000","last_undegraded":"2024-04-25T10:08:58.689289+0000","last_fullsized":"2024-04-25T10:08:58.689289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:56.579721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787288+0000","last_change":"2024-04-25T10:08:57.684648+0000","last_active":"2024-04-25T10:08:58.787288+0000","last_peered":"2024-04-25T10:08:58.787288+0000","last_clean":"2024-04-25T10:08:58.787288+0000","last_became_active":"2024-04-25T10:08:57.684289+0000","last_became_peered":"2024-04-25T10:08:57.684289+0000","last_unstale":"2024-04-25T10:08:58.787288+0000","last_undegraded":"2024-04-25T10:08:58.787288+0000","last_fullsized":"2024-04-25T10:08:58.787288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:52.909262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696722+0000","last_change":"2024-04-25T10:08:57.657002+0000","last_active":"2024-04-25T10:08:58.696722+0000","last_peered":"2024-04-25T10:08:58.696722+0000","last_clean":"2024-04-25T10:08:58.696722+0000","last_became_active":"2024-04-25T10:08:57.656456+0000","last_became_peered":"2024-04-25T10:08:57.656456+0000","last_unstale":"2024-04-25T10:08:58.696722+0000","last_undegraded":"2024-04-25T10:08:58.696722+0000","last_fullsized":"2024-04-25T10:08:58.696722+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.688922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696448+0000","last_change":"2024-04-25T10:08:57.671155+0000","last_active":"2024-04-25T10:08:58.696448+0000","last_peered":"2024-04-25T10:08:58.696448+0000","last_clean":"2024-04-25T10:08:58.696448+0000","last_became_active":"2024-04-25T10:08:57.669482+0000","last_became_peered":"2024-04-25T10:08:57.669482+0000","last_unstale":"2024-04-25T10:08:58.696448+0000","last_undegraded":"2024-04-25T10:08:58.696448+0000","last_fullsized":"2024-04-25T10:08:58.696448+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:35.541551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.788305+0000","last_change":"2024-04-25T10:08:57.683670+0000","last_active":"2024-04-25T10:08:58.788305+0000","last_peered":"2024-04-25T10:08:58.788305+0000","last_clean":"2024-04-25T10:08:58.788305+0000","last_became_active":"2024-04-25T10:08:57.682257+0000","last_became_peered":"2024-04-25T10:08:57.682257+0000","last_unstale":"2024-04-25T10:08:58.788305+0000","last_undegraded":"2024-04-25T10:08:58.788305+0000","last_fullsized":"2024-04-25T10:08:58.788305+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:42.114199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648846+0000","last_change":"2024-04-25T10:08:57.673783+0000","last_active":"2024-04-25T10:08:58.648846+0000","last_peered":"2024-04-25T10:08:58.648846+0000","last_clean":"2024-04-25T10:08:58.648846+0000","last_became_active":"2024-04-25T10:08:57.672014+0000","last_became_peered":"2024-04-25T10:08:57.672014+0000","last_unstale":"2024-04-25T10:08:58.648846+0000","last_undegraded":"2024-04-25T10:08:58.648846+0000","last_fullsized":"2024-04-25T10:08:58.648846+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:56.474422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689758+0000","last_change":"2024-04-25T10:08:57.677623+0000","last_active":"2024-04-25T10:08:58.689758+0000","last_peered":"2024-04-25T10:08:58.689758+0000","last_clean":"2024-04-25T10:08:58.689758+0000","last_became_active":"2024-04-25T10:08:57.677331+0000","last_became_peered":"2024-04-25T10:08:57.677331+0000","last_unstale":"2024-04-25T10:08:58.689758+0000","last_undegraded":"2024-04-25T10:08:58.689758+0000","last_fullsized":"2024-04-25T10:08:58.689758+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:49.880631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786686+0000","last_change":"2024-04-25T10:08:57.690217+0000","last_active":"2024-04-25T10:08:58.786686+0000","last_peered":"2024-04-25T10:08:58.786686+0000","last_clean":"2024-04-25T10:08:58.786686+0000","last_became_active":"2024-04-25T10:08:57.689116+0000","last_became_peered":"2024-04-25T10:08:57.689116+0000","last_unstale":"2024-04-25T10:08:58.786686+0000","last_undegraded":"2024-04-25T10:08:58.786686+0000","last_fullsized":"2024-04-25T10:08:58.786686+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:28.762976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787952+0000","last_change":"2024-04-25T10:08:57.678443+0000","last_active":"2024-04-25T10:08:58.787952+0000","last_peered":"2024-04-25T10:08:58.787952+0000","last_clean":"2024-04-25T10:08:58.787952+0000","last_became_active":"2024-04-25T10:08:57.678109+0000","last_became_peered":"2024-04-25T10:08:57.678109+0000","last_unstale":"2024-04-25T10:08:58.787952+0000","last_undegraded":"2024-04-25T10:08:58.787952+0000","last_fullsized":"2024-04-25T10:08:58.787952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:20.623071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787638+0000","last_change":"2024-04-25T10:08:57.678264+0000","last_active":"2024-04-25T10:08:58.787638+0000","last_peered":"2024-04-25T10:08:58.787638+0000","last_clean":"2024-04-25T10:08:58.787638+0000","last_became_active":"2024-04-25T10:08:57.677956+0000","last_became_peered":"2024-04-25T10:08:57.677956+0000","last_unstale":"2024-04-25T10:08:58.787638+0000","last_undegraded":"2024-04-25T10:08:58.787638+0000","last_fullsized":"2024-04-25T10:08:58.787638+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:53.652726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.788312+0000","last_change":"2024-04-25T10:08:57.681165+0000","last_active":"2024-04-25T10:08:58.788312+0000","last_peered":"2024-04-25T10:08:58.788312+0000","last_clean":"2024-04-25T10:08:58.788312+0000","last_became_active":"2024-04-25T10:08:57.680790+0000","last_became_peered":"2024-04-25T10:08:57.680790+0000","last_unstale":"2024-04-25T10:08:58.788312+0000","last_undegraded":"2024-04-25T10:08:58.788312+0000","last_fullsized":"2024-04-25T10:08:58.788312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:20.444343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648545+0000","last_change":"2024-04-25T10:08:57.674615+0000","last_active":"2024-04-25T10:08:58.648545+0000","last_peered":"2024-04-25T10:08:58.648545+0000","last_clean":"2024-04-25T10:08:58.648545+0000","last_became_active":"2024-04-25T10:08:57.673956+0000","last_became_peered":"2024-04-25T10:08:57.673956+0000","last_unstale":"2024-04-25T10:08:58.648545+0000","last_undegraded":"2024-04-25T10:08:58.648545+0000","last_fullsized":"2024-04-25T10:08:58.648545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:50.730066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696388+0000","last_change":"2024-04-25T10:08:57.657413+0000","last_active":"2024-04-25T10:08:58.696388+0000","last_peered":"2024-04-25T10:08:58.696388+0000","last_clean":"2024-04-25T10:08:58.696388+0000","last_became_active":"2024-04-25T10:08:57.656482+0000","last_became_peered":"2024-04-25T10:08:57.656482+0000","last_unstale":"2024-04-25T10:08:58.696388+0000","last_undegraded":"2024-04-25T10:08:58.696388+0000","last_fullsized":"2024-04-25T10:08:58.696388+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:48.484066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652559+0000","last_change":"2024-04-25T10:08:57.663001+0000","last_active":"2024-04-25T10:08:58.652559+0000","last_peered":"2024-04-25T10:08:58.652559+0000","last_clean":"2024-04-25T10:08:58.652559+0000","last_became_active":"2024-04-25T10:08:57.662678+0000","last_became_peered":"2024-04-25T10:08:57.662678+0000","last_unstale":"2024-04-25T10:08:58.652559+0000","last_undegraded":"2024-04-25T10:08:58.652559+0000","last_fullsized":"2024-04-25T10:08:58.652559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:23.905676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787480+0000","last_change":"2024-04-25T10:08:57.684275+0000","last_active":"2024-04-25T10:08:58.787480+0000","last_peered":"2024-04-25T10:08:58.787480+0000","last_clean":"2024-04-25T10:08:58.787480+0000","last_became_active":"2024-04-25T10:08:57.681876+0000","last_became_peered":"2024-04-25T10:08:57.681876+0000","last_unstale":"2024-04-25T10:08:58.787480+0000","last_undegraded":"2024-04-25T10:08:58.787480+0000","last_fullsized":"2024-04-25T10:08:58.787480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:33.253927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696100+0000","last_change":"2024-04-25T10:08:57.679953+0000","last_active":"2024-04-25T10:08:58.696100+0000","last_peered":"2024-04-25T10:08:58.696100+0000","last_clean":"2024-04-25T10:08:58.696100+0000","last_became_active":"2024-04-25T10:08:57.679274+0000","last_became_peered":"2024-04-25T10:08:57.679274+0000","last_unstale":"2024-04-25T10:08:58.696100+0000","last_undegraded":"2024-04-25T10:08:58.696100+0000","last_fullsized":"2024-04-25T10:08:58.696100+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:12.146272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689349+0000","last_change":"2024-04-25T10:08:57.671918+0000","last_active":"2024-04-25T10:08:58.689349+0000","last_peered":"2024-04-25T10:08:58.689349+0000","last_clean":"2024-04-25T10:08:58.689349+0000","last_became_active":"2024-04-25T10:08:57.670912+0000","last_became_peered":"2024-04-25T10:08:57.670912+0000","last_unstale":"2024-04-25T10:08:58.689349+0000","last_undegraded":"2024-04-25T10:08:58.689349+0000","last_fullsized":"2024-04-25T10:08:58.689349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:38.628073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648285+0000","last_change":"2024-04-25T10:08:57.679998+0000","last_active":"2024-04-25T10:08:58.648285+0000","last_peered":"2024-04-25T10:08:58.648285+0000","last_clean":"2024-04-25T10:08:58.648285+0000","last_became_active":"2024-04-25T10:08:57.679588+0000","last_became_peered":"2024-04-25T10:08:57.679588+0000","last_unstale":"2024-04-25T10:08:58.648285+0000","last_undegraded":"2024-04-25T10:08:58.648285+0000","last_fullsized":"2024-04-25T10:08:58.648285+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:12.818134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786482+0000","last_change":"2024-04-25T10:08:57.679167+0000","last_active":"2024-04-25T10:08:58.786482+0000","last_peered":"2024-04-25T10:08:58.786482+0000","last_clean":"2024-04-25T10:08:58.786482+0000","last_became_active":"2024-04-25T10:08:57.678468+0000","last_became_peered":"2024-04-25T10:08:57.678468+0000","last_unstale":"2024-04-25T10:08:58.786482+0000","last_undegraded":"2024-04-25T10:08:58.786482+0000","last_fullsized":"2024-04-25T10:08:58.786482+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:20.623071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.690310+0000","last_change":"2024-04-25T10:08:56.681068+0000","last_active":"2024-04-25T10:08:58.690310+0000","last_peered":"2024-04-25T10:08:58.690310+0000","last_clean":"2024-04-25T10:08:58.690310+0000","last_became_active":"2024-04-25T10:08:56.680598+0000","last_became_peered":"2024-04-25T10:08:56.680598+0000","last_unstale":"2024-04-25T10:08:58.690310+0000","last_undegraded":"2024-04-25T10:08:58.690310+0000","last_fullsized":"2024-04-25T10:08:58.690310+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:11.560129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.788324+0000","last_change":"2024-04-25T10:08:57.680540+0000","last_active":"2024-04-25T10:08:58.788324+0000","last_peered":"2024-04-25T10:08:58.788324+0000","last_clean":"2024-04-25T10:08:58.788324+0000","last_became_active":"2024-04-25T10:08:57.680271+0000","last_became_peered":"2024-04-25T10:08:57.680271+0000","last_unstale":"2024-04-25T10:08:58.788324+0000","last_undegraded":"2024-04-25T10:08:58.788324+0000","last_fullsized":"2024-04-25T10:08:58.788324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:12.146272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.697034+0000","last_change":"2024-04-25T10:08:56.687541+0000","last_active":"2024-04-25T10:08:58.697034+0000","last_peered":"2024-04-25T10:08:58.697034+0000","last_clean":"2024-04-25T10:08:58.697034+0000","last_became_active":"2024-04-25T10:08:56.685968+0000","last_became_peered":"2024-04-25T10:08:56.685968+0000","last_unstale":"2024-04-25T10:08:58.697034+0000","last_undegraded":"2024-04-25T10:08:58.697034+0000","last_fullsized":"2024-04-25T10:08:58.697034+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:49.147091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652439+0000","last_change":"2024-04-25T10:08:57.675214+0000","last_active":"2024-04-25T10:08:58.652439+0000","last_peered":"2024-04-25T10:08:58.652439+0000","last_clean":"2024-04-25T10:08:58.652439+0000","last_became_active":"2024-04-25T10:08:57.674848+0000","last_became_peered":"2024-04-25T10:08:57.674848+0000","last_unstale":"2024-04-25T10:08:58.652439+0000","last_undegraded":"2024-04-25T10:08:58.652439+0000","last_fullsized":"2024-04-25T10:08:58.652439+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:51.183347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786107+0000","last_change":"2024-04-25T10:08:56.677307+0000","last_active":"2024-04-25T10:08:58.786107+0000","last_peered":"2024-04-25T10:08:58.786107+0000","last_clean":"2024-04-25T10:08:58.786107+0000","last_became_active":"2024-04-25T10:08:56.676965+0000","last_became_peered":"2024-04-25T10:08:56.676965+0000","last_unstale":"2024-04-25T10:08:58.786107+0000","last_undegraded":"2024-04-25T10:08:58.786107+0000","last_fullsized":"2024-04-25T10:08:58.786107+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:49.147091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786025+0000","last_change":"2024-04-25T10:08:57.678144+0000","last_active":"2024-04-25T10:08:58.786025+0000","last_peered":"2024-04-25T10:08:58.786025+0000","last_clean":"2024-04-25T10:08:58.786025+0000","last_became_active":"2024-04-25T10:08:57.677858+0000","last_became_peered":"2024-04-25T10:08:57.677858+0000","last_unstale":"2024-04-25T10:08:58.786025+0000","last_undegraded":"2024-04-25T10:08:58.786025+0000","last_fullsized":"2024-04-25T10:08:58.786025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:34.131794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.649788+0000","last_change":"2024-04-25T10:08:56.681797+0000","last_active":"2024-04-25T10:08:58.649788+0000","last_peered":"2024-04-25T10:08:58.649788+0000","last_clean":"2024-04-25T10:08:58.649788+0000","last_became_active":"2024-04-25T10:08:56.681518+0000","last_became_peered":"2024-04-25T10:08:56.681518+0000","last_unstale":"2024-04-25T10:08:58.649788+0000","last_undegraded":"2024-04-25T10:08:58.649788+0000","last_fullsized":"2024-04-25T10:08:58.649788+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:43.317333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696239+0000","last_change":"2024-04-25T10:08:57.678092+0000","last_active":"2024-04-25T10:08:58.696239+0000","last_peered":"2024-04-25T10:08:58.696239+0000","last_clean":"2024-04-25T10:08:58.696239+0000","last_became_active":"2024-04-25T10:08:57.677130+0000","last_became_peered":"2024-04-25T10:08:57.677130+0000","last_unstale":"2024-04-25T10:08:58.696239+0000","last_undegraded":"2024-04-25T10:08:58.696239+0000","last_fullsized":"2024-04-25T10:08:58.696239+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:46.333691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689843+0000","last_change":"2024-04-25T10:08:56.683994+0000","last_active":"2024-04-25T10:08:58.689843+0000","last_peered":"2024-04-25T10:08:58.689843+0000","last_clean":"2024-04-25T10:08:58.689843+0000","last_became_active":"2024-04-25T10:08:56.683686+0000","last_became_peered":"2024-04-25T10:08:56.683686+0000","last_unstale":"2024-04-25T10:08:58.689843+0000","last_undegraded":"2024-04-25T10:08:58.689843+0000","last_fullsized":"2024-04-25T10:08:58.689843+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:25.673661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786201+0000","last_change":"2024-04-25T10:08:57.674573+0000","last_active":"2024-04-25T10:08:58.786201+0000","last_peered":"2024-04-25T10:08:58.786201+0000","last_clean":"2024-04-25T10:08:58.786201+0000","last_became_active":"2024-04-25T10:08:57.674280+0000","last_became_peered":"2024-04-25T10:08:57.674280+0000","last_unstale":"2024-04-25T10:08:58.786201+0000","last_undegraded":"2024-04-25T10:08:58.786201+0000","last_fullsized":"2024-04-25T10:08:58.786201+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:04.281461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696252+0000","last_change":"2024-04-25T10:08:56.682809+0000","last_active":"2024-04-25T10:08:58.696252+0000","last_peered":"2024-04-25T10:08:58.696252+0000","last_clean":"2024-04-25T10:08:58.696252+0000","last_became_active":"2024-04-25T10:08:56.682522+0000","last_became_peered":"2024-04-25T10:08:56.682522+0000","last_unstale":"2024-04-25T10:08:58.696252+0000","last_undegraded":"2024-04-25T10:08:58.696252+0000","last_fullsized":"2024-04-25T10:08:58.696252+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:12.104648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785876+0000","last_change":"2024-04-25T10:08:57.683466+0000","last_active":"2024-04-25T10:08:58.785876+0000","last_peered":"2024-04-25T10:08:58.785876+0000","last_clean":"2024-04-25T10:08:58.785876+0000","last_became_active":"2024-04-25T10:08:57.682853+0000","last_became_peered":"2024-04-25T10:08:57.682853+0000","last_unstale":"2024-04-25T10:08:58.785876+0000","last_undegraded":"2024-04-25T10:08:58.785876+0000","last_fullsized":"2024-04-25T10:08:58.785876+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.688922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.651454+0000","last_change":"2024-04-25T10:08:56.671637+0000","last_active":"2024-04-25T10:08:58.651454+0000","last_peered":"2024-04-25T10:08:58.651454+0000","last_clean":"2024-04-25T10:08:58.651454+0000","last_became_active":"2024-04-25T10:08:56.671141+0000","last_became_peered":"2024-04-25T10:08:56.671141+0000","last_unstale":"2024-04-25T10:08:58.651454+0000","last_undegraded":"2024-04-25T10:08:58.651454+0000","last_fullsized":"2024-04-25T10:08:58.651454+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:33.116533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787164+0000","last_change":"2024-04-25T10:08:57.678769+0000","last_active":"2024-04-25T10:08:58.787164+0000","last_peered":"2024-04-25T10:08:58.787164+0000","last_clean":"2024-04-25T10:08:58.787164+0000","last_became_active":"2024-04-25T10:08:57.678505+0000","last_became_peered":"2024-04-25T10:08:57.678505+0000","last_unstale":"2024-04-25T10:08:58.787164+0000","last_undegraded":"2024-04-25T10:08:58.787164+0000","last_fullsized":"2024-04-25T10:08:58.787164+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:31.382081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785730+0000","last_change":"2024-04-25T10:08:56.674412+0000","last_active":"2024-04-25T10:08:58.785730+0000","last_peered":"2024-04-25T10:08:58.785730+0000","last_clean":"2024-04-25T10:08:58.785730+0000","last_became_active":"2024-04-25T10:08:56.674105+0000","last_became_peered":"2024-04-25T10:08:56.674105+0000","last_unstale":"2024-04-25T10:08:58.785730+0000","last_undegraded":"2024-04-25T10:08:58.785730+0000","last_fullsized":"2024-04-25T10:08:58.785730+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:12.104648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696866+0000","last_change":"2024-04-25T10:08:57.670926+0000","last_active":"2024-04-25T10:08:58.696866+0000","last_peered":"2024-04-25T10:08:58.696866+0000","last_clean":"2024-04-25T10:08:58.696866+0000","last_became_active":"2024-04-25T10:08:57.668694+0000","last_became_peered":"2024-04-25T10:08:57.668694+0000","last_unstale":"2024-04-25T10:08:58.696866+0000","last_undegraded":"2024-04-25T10:08:58.696866+0000","last_fullsized":"2024-04-25T10:08:58.696866+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:04.281461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.697039+0000","last_change":"2024-04-25T10:08:56.685708+0000","last_active":"2024-04-25T10:08:58.697039+0000","last_peered":"2024-04-25T10:08:58.697039+0000","last_clean":"2024-04-25T10:08:58.697039+0000","last_became_active":"2024-04-25T10:08:56.685324+0000","last_became_peered":"2024-04-25T10:08:56.685324+0000","last_unstale":"2024-04-25T10:08:58.697039+0000","last_undegraded":"2024-04-25T10:08:58.697039+0000","last_fullsized":"2024-04-25T10:08:58.697039+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:53.946213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652081+0000","last_change":"2024-04-25T10:08:57.676105+0000","last_active":"2024-04-25T10:08:58.652081+0000","last_peered":"2024-04-25T10:08:58.652081+0000","last_clean":"2024-04-25T10:08:58.652081+0000","last_became_active":"2024-04-25T10:08:57.675476+0000","last_became_peered":"2024-04-25T10:08:57.675476+0000","last_unstale":"2024-04-25T10:08:58.652081+0000","last_undegraded":"2024-04-25T10:08:58.652081+0000","last_fullsized":"2024-04-25T10:08:58.652081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:28.606869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787334+0000","last_change":"2024-04-25T10:08:56.681228+0000","last_active":"2024-04-25T10:08:58.787334+0000","last_peered":"2024-04-25T10:08:58.787334+0000","last_clean":"2024-04-25T10:08:58.787334+0000","last_became_active":"2024-04-25T10:08:56.680351+0000","last_became_peered":"2024-04-25T10:08:56.680351+0000","last_unstale":"2024-04-25T10:08:58.787334+0000","last_undegraded":"2024-04-25T10:08:58.787334+0000","last_fullsized":"2024-04-25T10:08:58.787334+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:30.366820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.690254+0000","last_change":"2024-04-25T10:08:57.689932+0000","last_active":"2024-04-25T10:08:58.690254+0000","last_peered":"2024-04-25T10:08:58.690254+0000","last_clean":"2024-04-25T10:08:58.690254+0000","last_became_active":"2024-04-25T10:08:57.689667+0000","last_became_peered":"2024-04-25T10:08:57.689667+0000","last_unstale":"2024-04-25T10:08:58.690254+0000","last_undegraded":"2024-04-25T10:08:58.690254+0000","last_fullsized":"2024-04-25T10:08:58.690254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:57.472901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.650810+0000","last_change":"2024-04-25T10:08:56.684316+0000","last_active":"2024-04-25T10:08:58.650810+0000","last_peered":"2024-04-25T10:08:58.650810+0000","last_clean":"2024-04-25T10:08:58.650810+0000","last_became_active":"2024-04-25T10:08:56.683994+0000","last_became_peered":"2024-04-25T10:08:56.683994+0000","last_unstale":"2024-04-25T10:08:58.650810+0000","last_undegraded":"2024-04-25T10:08:58.650810+0000","last_fullsized":"2024-04-25T10:08:58.650810+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:19.429082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.650195+0000","last_change":"2024-04-25T10:08:57.680662+0000","last_active":"2024-04-25T10:08:58.650195+0000","last_peered":"2024-04-25T10:08:58.650195+0000","last_clean":"2024-04-25T10:08:58.650195+0000","last_became_active":"2024-04-25T10:08:57.680321+0000","last_became_peered":"2024-04-25T10:08:57.680321+0000","last_unstale":"2024-04-25T10:08:58.650195+0000","last_undegraded":"2024-04-25T10:08:58.650195+0000","last_fullsized":"2024-04-25T10:08:58.650195+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:23.905676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648754+0000","last_change":"2024-04-25T10:08:56.686139+0000","last_active":"2024-04-25T10:08:58.648754+0000","last_peered":"2024-04-25T10:08:58.648754+0000","last_clean":"2024-04-25T10:08:58.648754+0000","last_became_active":"2024-04-25T10:08:56.685879+0000","last_became_peered":"2024-04-25T10:08:56.685879+0000","last_unstale":"2024-04-25T10:08:58.648754+0000","last_undegraded":"2024-04-25T10:08:58.648754+0000","last_fullsized":"2024-04-25T10:08:58.648754+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:31.953062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787406+0000","last_change":"2024-04-25T10:08:57.686439+0000","last_active":"2024-04-25T10:08:58.787406+0000","last_peered":"2024-04-25T10:08:58.787406+0000","last_clean":"2024-04-25T10:08:58.787406+0000","last_became_active":"2024-04-25T10:08:57.683649+0000","last_became_peered":"2024-04-25T10:08:57.683649+0000","last_unstale":"2024-04-25T10:08:58.787406+0000","last_undegraded":"2024-04-25T10:08:58.787406+0000","last_fullsized":"2024-04-25T10:08:58.787406+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:37.096413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.690108+0000","last_change":"2024-04-25T10:08:56.680711+0000","last_active":"2024-04-25T10:08:58.690108+0000","last_peered":"2024-04-25T10:08:58.690108+0000","last_clean":"2024-04-25T10:08:58.690108+0000","last_became_active":"2024-04-25T10:08:56.678963+0000","last_became_peered":"2024-04-25T10:08:56.678963+0000","last_unstale":"2024-04-25T10:08:58.690108+0000","last_undegraded":"2024-04-25T10:08:58.690108+0000","last_fullsized":"2024-04-25T10:08:58.690108+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:03.266200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.786416+0000","last_change":"2024-04-25T10:08:57.695716+0000","last_active":"2024-04-25T10:08:58.786416+0000","last_peered":"2024-04-25T10:08:58.786416+0000","last_clean":"2024-04-25T10:08:58.786416+0000","last_became_active":"2024-04-25T10:08:57.695436+0000","last_became_peered":"2024-04-25T10:08:57.695436+0000","last_unstale":"2024-04-25T10:08:58.786416+0000","last_undegraded":"2024-04-25T10:08:58.786416+0000","last_fullsized":"2024-04-25T10:08:58.786416+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:23.905676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.650715+0000","last_change":"2024-04-25T10:08:56.681239+0000","last_active":"2024-04-25T10:08:58.650715+0000","last_peered":"2024-04-25T10:08:58.650715+0000","last_clean":"2024-04-25T10:08:58.650715+0000","last_became_active":"2024-04-25T10:08:56.680924+0000","last_became_peered":"2024-04-25T10:08:56.680924+0000","last_unstale":"2024-04-25T10:08:58.650715+0000","last_undegraded":"2024-04-25T10:08:58.650715+0000","last_fullsized":"2024-04-25T10:08:58.650715+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:06.422241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689451+0000","last_change":"2024-04-25T10:08:57.674724+0000","last_active":"2024-04-25T10:08:58.689451+0000","last_peered":"2024-04-25T10:08:58.689451+0000","last_clean":"2024-04-25T10:08:58.689451+0000","last_became_active":"2024-04-25T10:08:57.674437+0000","last_became_peered":"2024-04-25T10:08:57.674437+0000","last_unstale":"2024-04-25T10:08:58.689451+0000","last_undegraded":"2024-04-25T10:08:58.689451+0000","last_fullsized":"2024-04-25T10:08:58.689451+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:52.858454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689378+0000","last_change":"2024-04-25T10:08:56.673870+0000","last_active":"2024-04-25T10:08:58.689378+0000","last_peered":"2024-04-25T10:08:58.689378+0000","last_clean":"2024-04-25T10:08:58.689378+0000","last_became_active":"2024-04-25T10:08:56.673578+0000","last_became_peered":"2024-04-25T10:08:56.673578+0000","last_unstale":"2024-04-25T10:08:58.689378+0000","last_undegraded":"2024-04-25T10:08:58.689378+0000","last_fullsized":"2024-04-25T10:08:58.689378+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:47.468805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.650214+0000","last_change":"2024-04-25T10:08:57.682613+0000","last_active":"2024-04-25T10:08:58.650214+0000","last_peered":"2024-04-25T10:08:58.650214+0000","last_clean":"2024-04-25T10:08:58.650214+0000","last_became_active":"2024-04-25T10:08:57.682327+0000","last_became_peered":"2024-04-25T10:08:57.682327+0000","last_unstale":"2024-04-25T10:08:58.650214+0000","last_undegraded":"2024-04-25T10:08:58.650214+0000","last_fullsized":"2024-04-25T10:08:58.650214+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:28.762976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":22,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785990+0000","last_change":"2024-04-25T10:08:56.680341+0000","last_active":"2024-04-25T10:08:58.785990+0000","last_peered":"2024-04-25T10:08:58.785990+0000","last_clean":"2024-04-25T10:08:58.785990+0000","last_became_active":"2024-04-25T10:08:56.680060+0000","last_became_peered":"2024-04-25T10:08:56.680060+0000","last_unstale":"2024-04-25T10:08:58.785990+0000","last_undegraded":"2024-04-25T10:08:58.785990+0000","last_fullsized":"2024-04-25T10:08:58.785990+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:53.946213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648929+0000","last_change":"2024-04-25T10:08:57.656725+0000","last_active":"2024-04-25T10:08:58.648929+0000","last_peered":"2024-04-25T10:08:58.648929+0000","last_clean":"2024-04-25T10:08:58.648929+0000","last_became_active":"2024-04-25T10:08:57.655783+0000","last_became_peered":"2024-04-25T10:08:57.655783+0000","last_unstale":"2024-04-25T10:08:58.648929+0000","last_undegraded":"2024-04-25T10:08:58.648929+0000","last_fullsized":"2024-04-25T10:08:58.648929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:13.181139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648537+0000","last_change":"2024-04-25T10:08:56.673276+0000","last_active":"2024-04-25T10:08:58.648537+0000","last_peered":"2024-04-25T10:08:58.648537+0000","last_clean":"2024-04-25T10:08:58.648537+0000","last_became_active":"2024-04-25T10:08:56.672940+0000","last_became_peered":"2024-04-25T10:08:56.672940+0000","last_unstale":"2024-04-25T10:08:58.648537+0000","last_undegraded":"2024-04-25T10:08:58.648537+0000","last_fullsized":"2024-04-25T10:08:58.648537+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:23.719316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787564+0000","last_change":"2024-04-25T10:08:57.681846+0000","last_active":"2024-04-25T10:08:58.787564+0000","last_peered":"2024-04-25T10:08:58.787564+0000","last_clean":"2024-04-25T10:08:58.787564+0000","last_became_active":"2024-04-25T10:08:57.681577+0000","last_became_peered":"2024-04-25T10:08:57.681577+0000","last_unstale":"2024-04-25T10:08:58.787564+0000","last_undegraded":"2024-04-25T10:08:58.787564+0000","last_fullsized":"2024-04-25T10:08:58.787564+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:45.096249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.697054+0000","last_change":"2024-04-25T10:08:56.688492+0000","last_active":"2024-04-25T10:08:58.697054+0000","last_peered":"2024-04-25T10:08:58.697054+0000","last_clean":"2024-04-25T10:08:58.697054+0000","last_became_active":"2024-04-25T10:08:56.688245+0000","last_became_peered":"2024-04-25T10:08:56.688245+0000","last_unstale":"2024-04-25T10:08:58.697054+0000","last_undegraded":"2024-04-25T10:08:58.697054+0000","last_fullsized":"2024-04-25T10:08:58.697054+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:30.366820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787660+0000","last_change":"2024-04-25T10:08:57.675803+0000","last_active":"2024-04-25T10:08:58.787660+0000","last_peered":"2024-04-25T10:08:58.787660+0000","last_clean":"2024-04-25T10:08:58.787660+0000","last_became_active":"2024-04-25T10:08:57.675512+0000","last_became_peered":"2024-04-25T10:08:57.675512+0000","last_unstale":"2024-04-25T10:08:58.787660+0000","last_undegraded":"2024-04-25T10:08:58.787660+0000","last_fullsized":"2024-04-25T10:08:58.787660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:34.131794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652189+0000","last_change":"2024-04-25T10:08:56.670169+0000","last_active":"2024-04-25T10:08:58.652189+0000","last_peered":"2024-04-25T10:08:58.652189+0000","last_clean":"2024-04-25T10:08:58.652189+0000","last_became_active":"2024-04-25T10:08:56.669659+0000","last_became_peered":"2024-04-25T10:08:56.669659+0000","last_unstale":"2024-04-25T10:08:58.652189+0000","last_undegraded":"2024-04-25T10:08:58.652189+0000","last_fullsized":"2024-04-25T10:08:58.652189+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:43.317333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.788000+0000","last_change":"2024-04-25T10:08:57.684029+0000","last_active":"2024-04-25T10:08:58.788000+0000","last_peered":"2024-04-25T10:08:58.788000+0000","last_clean":"2024-04-25T10:08:58.788000+0000","last_became_active":"2024-04-25T10:08:57.681955+0000","last_became_peered":"2024-04-25T10:08:57.681955+0000","last_unstale":"2024-04-25T10:08:58.788000+0000","last_undegraded":"2024-04-25T10:08:58.788000+0000","last_fullsized":"2024-04-25T10:08:58.788000+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:44.332595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.690116+0000","last_change":"2024-04-25T10:08:56.680963+0000","last_active":"2024-04-25T10:08:58.690116+0000","last_peered":"2024-04-25T10:08:58.690116+0000","last_clean":"2024-04-25T10:08:58.690116+0000","last_became_active":"2024-04-25T10:08:56.680666+0000","last_became_peered":"2024-04-25T10:08:56.680666+0000","last_unstale":"2024-04-25T10:08:58.690116+0000","last_undegraded":"2024-04-25T10:08:58.690116+0000","last_fullsized":"2024-04-25T10:08:58.690116+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:34.526289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689601+0000","last_change":"2024-04-25T10:08:57.675117+0000","last_active":"2024-04-25T10:08:58.689601+0000","last_peered":"2024-04-25T10:08:58.689601+0000","last_clean":"2024-04-25T10:08:58.689601+0000","last_became_active":"2024-04-25T10:08:57.674871+0000","last_became_peered":"2024-04-25T10:08:57.674871+0000","last_unstale":"2024-04-25T10:08:58.689601+0000","last_undegraded":"2024-04-25T10:08:58.689601+0000","last_fullsized":"2024-04-25T10:08:58.689601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:19.093937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.690351+0000","last_change":"2024-04-25T10:08:56.683965+0000","last_active":"2024-04-25T10:08:58.690351+0000","last_peered":"2024-04-25T10:08:58.690351+0000","last_clean":"2024-04-25T10:08:58.690351+0000","last_became_active":"2024-04-25T10:08:56.683697+0000","last_became_peered":"2024-04-25T10:08:56.683697+0000","last_unstale":"2024-04-25T10:08:58.690351+0000","last_undegraded":"2024-04-25T10:08:58.690351+0000","last_fullsized":"2024-04-25T10:08:58.690351+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:50.168086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785604+0000","last_change":"2024-04-25T10:08:57.680867+0000","last_active":"2024-04-25T10:08:58.785604+0000","last_peered":"2024-04-25T10:08:58.785604+0000","last_clean":"2024-04-25T10:08:58.785604+0000","last_became_active":"2024-04-25T10:08:57.680450+0000","last_became_peered":"2024-04-25T10:08:57.680450+0000","last_unstale":"2024-04-25T10:08:58.785604+0000","last_undegraded":"2024-04-25T10:08:58.785604+0000","last_fullsized":"2024-04-25T10:08:58.785604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:53.652726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.651528+0000","last_change":"2024-04-25T10:08:56.680574+0000","last_active":"2024-04-25T10:08:58.651528+0000","last_peered":"2024-04-25T10:08:58.651528+0000","last_clean":"2024-04-25T10:08:58.651528+0000","last_became_active":"2024-04-25T10:08:56.680200+0000","last_became_peered":"2024-04-25T10:08:56.680200+0000","last_unstale":"2024-04-25T10:08:58.651528+0000","last_undegraded":"2024-04-25T10:08:58.651528+0000","last_fullsized":"2024-04-25T10:08:58.651528+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:19.607810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.648323+0000","last_change":"2024-04-25T10:08:57.676997+0000","last_active":"2024-04-25T10:08:58.648323+0000","last_peered":"2024-04-25T10:08:58.648323+0000","last_clean":"2024-04-25T10:08:58.648323+0000","last_became_active":"2024-04-25T10:08:57.676317+0000","last_became_peered":"2024-04-25T10:08:57.676317+0000","last_unstale":"2024-04-25T10:08:58.648323+0000","last_undegraded":"2024-04-25T10:08:58.648323+0000","last_fullsized":"2024-04-25T10:08:58.648323+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:50.510935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785857+0000","last_change":"2024-04-25T10:08:56.678727+0000","last_active":"2024-04-25T10:08:58.785857+0000","last_peered":"2024-04-25T10:08:58.785857+0000","last_clean":"2024-04-25T10:08:58.785857+0000","last_became_active":"2024-04-25T10:08:56.678451+0000","last_became_peered":"2024-04-25T10:08:56.678451+0000","last_unstale":"2024-04-25T10:08:58.785857+0000","last_undegraded":"2024-04-25T10:08:58.785857+0000","last_fullsized":"2024-04-25T10:08:58.785857+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:12.165878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785604+0000","last_change":"2024-04-25T10:08:57.680196+0000","last_active":"2024-04-25T10:08:58.785604+0000","last_peered":"2024-04-25T10:08:58.785604+0000","last_clean":"2024-04-25T10:08:58.785604+0000","last_became_active":"2024-04-25T10:08:57.679865+0000","last_became_peered":"2024-04-25T10:08:57.679865+0000","last_unstale":"2024-04-25T10:08:58.785604+0000","last_undegraded":"2024-04-25T10:08:58.785604+0000","last_fullsized":"2024-04-25T10:08:58.785604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:35.541551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.649498+0000","last_change":"2024-04-25T10:08:56.680808+0000","last_active":"2024-04-25T10:08:58.649498+0000","last_peered":"2024-04-25T10:08:58.649498+0000","last_clean":"2024-04-25T10:08:58.649498+0000","last_became_active":"2024-04-25T10:08:56.680518+0000","last_became_peered":"2024-04-25T10:08:56.680518+0000","last_unstale":"2024-04-25T10:08:58.649498+0000","last_undegraded":"2024-04-25T10:08:58.649498+0000","last_fullsized":"2024-04-25T10:08:58.649498+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:19.607810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.788552+0000","last_change":"2024-04-25T10:08:57.682048+0000","last_active":"2024-04-25T10:08:58.788552+0000","last_peered":"2024-04-25T10:08:58.788552+0000","last_clean":"2024-04-25T10:08:58.788552+0000","last_became_active":"2024-04-25T10:08:57.681653+0000","last_became_peered":"2024-04-25T10:08:57.681653+0000","last_unstale":"2024-04-25T10:08:58.788552+0000","last_undegraded":"2024-04-25T10:08:58.788552+0000","last_fullsized":"2024-04-25T10:08:58.788552+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:07.437502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.690217+0000","last_change":"2024-04-25T10:08:56.685894+0000","last_active":"2024-04-25T10:08:58.690217+0000","last_peered":"2024-04-25T10:08:58.690217+0000","last_clean":"2024-04-25T10:08:58.690217+0000","last_became_active":"2024-04-25T10:08:56.685605+0000","last_became_peered":"2024-04-25T10:08:56.685605+0000","last_unstale":"2024-04-25T10:08:58.690217+0000","last_undegraded":"2024-04-25T10:08:58.690217+0000","last_fullsized":"2024-04-25T10:08:58.690217+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:15.693042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787099+0000","last_change":"2024-04-25T10:08:57.670495+0000","last_active":"2024-04-25T10:08:58.787099+0000","last_peered":"2024-04-25T10:08:58.787099+0000","last_clean":"2024-04-25T10:08:58.787099+0000","last_became_active":"2024-04-25T10:08:57.670174+0000","last_became_peered":"2024-04-25T10:08:57.670174+0000","last_unstale":"2024-04-25T10:08:58.787099+0000","last_undegraded":"2024-04-25T10:08:58.787099+0000","last_fullsized":"2024-04-25T10:08:58.787099+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:45.063754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.651122+0000","last_change":"2024-04-25T10:08:56.684331+0000","last_active":"2024-04-25T10:08:58.651122+0000","last_peered":"2024-04-25T10:08:58.651122+0000","last_clean":"2024-04-25T10:08:58.651122+0000","last_became_active":"2024-04-25T10:08:56.684083+0000","last_became_peered":"2024-04-25T10:08:56.684083+0000","last_unstale":"2024-04-25T10:08:58.651122+0000","last_undegraded":"2024-04-25T10:08:58.651122+0000","last_fullsized":"2024-04-25T10:08:58.651122+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.098938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.650676+0000","last_change":"2024-04-25T10:08:57.676739+0000","last_active":"2024-04-25T10:08:58.650676+0000","last_peered":"2024-04-25T10:08:58.650676+0000","last_clean":"2024-04-25T10:08:58.650676+0000","last_became_active":"2024-04-25T10:08:57.676444+0000","last_became_peered":"2024-04-25T10:08:57.676444+0000","last_unstale":"2024-04-25T10:08:58.650676+0000","last_undegraded":"2024-04-25T10:08:58.650676+0000","last_fullsized":"2024-04-25T10:08:58.650676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:45.063754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785169+0000","last_change":"2024-04-25T10:08:56.670086+0000","last_active":"2024-04-25T10:08:58.785169+0000","last_peered":"2024-04-25T10:08:58.785169+0000","last_clean":"2024-04-25T10:08:58.785169+0000","last_became_active":"2024-04-25T10:08:56.669777+0000","last_became_peered":"2024-04-25T10:08:56.669777+0000","last_unstale":"2024-04-25T10:08:58.785169+0000","last_undegraded":"2024-04-25T10:08:58.785169+0000","last_fullsized":"2024-04-25T10:08:58.785169+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:17.639165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689744+0000","last_change":"2024-04-25T10:08:57.674964+0000","last_active":"2024-04-25T10:08:58.689744+0000","last_peered":"2024-04-25T10:08:58.689744+0000","last_clean":"2024-04-25T10:08:58.689744+0000","last_became_active":"2024-04-25T10:08:57.673115+0000","last_became_peered":"2024-04-25T10:08:57.673115+0000","last_unstale":"2024-04-25T10:08:58.689744+0000","last_undegraded":"2024-04-25T10:08:58.689744+0000","last_fullsized":"2024-04-25T10:08:58.689744+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:33.480358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.650703+0000","last_change":"2024-04-25T10:08:56.685961+0000","last_active":"2024-04-25T10:08:58.650703+0000","last_peered":"2024-04-25T10:08:58.650703+0000","last_clean":"2024-04-25T10:08:58.650703+0000","last_became_active":"2024-04-25T10:08:56.685257+0000","last_became_peered":"2024-04-25T10:08:56.685257+0000","last_unstale":"2024-04-25T10:08:58.650703+0000","last_undegraded":"2024-04-25T10:08:58.650703+0000","last_fullsized":"2024-04-25T10:08:58.650703+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:11.131011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.689867+0000","last_change":"2024-04-25T10:08:57.674696+0000","last_active":"2024-04-25T10:08:58.689867+0000","last_peered":"2024-04-25T10:08:58.689867+0000","last_clean":"2024-04-25T10:08:58.689867+0000","last_became_active":"2024-04-25T10:08:57.673931+0000","last_became_peered":"2024-04-25T10:08:57.673931+0000","last_unstale":"2024-04-25T10:08:58.689867+0000","last_undegraded":"2024-04-25T10:08:58.689867+0000","last_fullsized":"2024-04-25T10:08:58.689867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:56.595202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696354+0000","last_change":"2024-04-25T10:08:56.682120+0000","last_active":"2024-04-25T10:08:58.696354+0000","last_peered":"2024-04-25T10:08:58.696354+0000","last_clean":"2024-04-25T10:08:58.696354+0000","last_became_active":"2024-04-25T10:08:56.681640+0000","last_became_peered":"2024-04-25T10:08:56.681640+0000","last_unstale":"2024-04-25T10:08:58.696354+0000","last_undegraded":"2024-04-25T10:08:58.696354+0000","last_fullsized":"2024-04-25T10:08:58.696354+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:17.639165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.651487+0000","last_change":"2024-04-25T10:08:57.673691+0000","last_active":"2024-04-25T10:08:58.651487+0000","last_peered":"2024-04-25T10:08:58.651487+0000","last_clean":"2024-04-25T10:08:58.651487+0000","last_became_active":"2024-04-25T10:08:57.673403+0000","last_became_peered":"2024-04-25T10:08:57.673403+0000","last_unstale":"2024-04-25T10:08:58.651487+0000","last_undegraded":"2024-04-25T10:08:58.651487+0000","last_fullsized":"2024-04-25T10:08:58.651487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:16.708303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785427+0000","last_change":"2024-04-25T10:08:56.681231+0000","last_active":"2024-04-25T10:08:58.785427+0000","last_peered":"2024-04-25T10:08:58.785427+0000","last_clean":"2024-04-25T10:08:58.785427+0000","last_became_active":"2024-04-25T10:08:56.680897+0000","last_became_peered":"2024-04-25T10:08:56.680897+0000","last_unstale":"2024-04-25T10:08:58.785427+0000","last_undegraded":"2024-04-25T10:08:58.785427+0000","last_fullsized":"2024-04-25T10:08:58.785427+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:30.366820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.696109+0000","last_change":"2024-04-25T10:08:57.672456+0000","last_active":"2024-04-25T10:08:58.696109+0000","last_peered":"2024-04-25T10:08:58.696109+0000","last_clean":"2024-04-25T10:08:58.696109+0000","last_became_active":"2024-04-25T10:08:57.672181+0000","last_became_peered":"2024-04-25T10:08:57.672181+0000","last_unstale":"2024-04-25T10:08:58.696109+0000","last_undegraded":"2024-04-25T10:08:58.696109+0000","last_fullsized":"2024-04-25T10:08:58.696109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:07.437502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.785980+0000","last_change":"2024-04-25T10:08:56.673731+0000","last_active":"2024-04-25T10:08:58.785980+0000","last_peered":"2024-04-25T10:08:58.785980+0000","last_clean":"2024-04-25T10:08:58.785980+0000","last_became_active":"2024-04-25T10:08:56.673457+0000","last_became_peered":"2024-04-25T10:08:56.673457+0000","last_unstale":"2024-04-25T10:08:58.785980+0000","last_undegraded":"2024-04-25T10:08:58.785980+0000","last_fullsized":"2024-04-25T10:08:58.785980+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:17.639165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.787246+0000","last_change":"2024-04-25T10:08:57.693156+0000","last_active":"2024-04-25T10:08:58.787246+0000","last_peered":"2024-04-25T10:08:58.787246+0000","last_clean":"2024-04-25T10:08:58.787246+0000","last_became_active":"2024-04-25T10:08:57.692874+0000","last_became_peered":"2024-04-25T10:08:57.692874+0000","last_unstale":"2024-04-25T10:08:58.787246+0000","last_undegraded":"2024-04-25T10:08:58.787246+0000","last_fullsized":"2024-04-25T10:08:58.787246+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:56.630529+0000","last_clean_scrub_stamp":"2024-04-25T10:08:56.630529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:23.774482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.652469+0000","last_change":"2024-04-25T10:08:56.678464+0000","last_active":"2024-04-25T10:08:58.652469+0000","last_peered":"2024-04-25T10:08:58.652469+0000","last_clean":"2024-04-25T10:08:58.652469+0000","last_became_active":"2024-04-25T10:08:56.678121+0000","last_became_peered":"2024-04-25T10:08:56.678121+0000","last_unstale":"2024-04-25T10:08:58.652469+0000","last_undegraded":"2024-04-25T10:08:58.652469+0000","last_fullsized":"2024-04-25T10:08:58.652469+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:55.615268+0000","last_clean_scrub_stamp":"2024-04-25T10:08:55.615268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:19.429082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:08:58.649937+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:08:58.649937+0000","last_peered":"2024-04-25T10:08:58.649937+0000","last_clean":"2024-04-25T10:08:58.649937+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:08:58.649937+0000","last_undegraded":"2024-04-25T10:08:58.649937+0000","last_fullsized":"2024-04-25T10:08:58.649937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":800128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":524288,"data_stored":210342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":209740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":211282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":210308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":211304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":209740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":800072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:09:02.564 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:09:02.564 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:09:02.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.812+0000 7f851ac00640 1 Processor -- start 2024-04-25T10:09:02.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.812+0000 7f851ac00640 1 -- start start 2024-04-25T10:09:02.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.813+0000 7f851ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8514144fc0 0x7f851406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.814+0000 7f851ac00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85140ccb50 0x7f851406b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.814+0000 7f851ac00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851415f4f0 con 0x7f85140b2c90 2024-04-25T10:09:02.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.814+0000 7f851ac00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8514160100 con 0x7f85140ccb50 2024-04-25T10:09:02.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.814+0000 7f851ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851415f7d0 con 0x7f8514144fc0 2024-04-25T10:09:02.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.814+0000 7f851a200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85140ccb50 0x7f851406b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.814+0000 7f851a200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85140ccb50 0x7f851406b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38280/0 (socket says 172.21.15.119:38280) 2024-04-25T10:09:02.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.814+0000 7f851a200640 1 -- 172.21.15.119:0/2361865354 learned_addr learned my addr 172.21.15.119:0/2361865354 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:02.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.814+0000 7f8519800640 1 --2- 172.21.15.119:0/2361865354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8514144fc0 0x7f851406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.814+0000 7f851a200640 1 -- 172.21.15.119:0/2361865354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8514144fc0 msgr2=0x7f851406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:02.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.814+0000 7f851a200640 1 --2- 172.21.15.119:0/2361865354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8514144fc0 0x7f851406aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.814+0000 7f851a200640 1 -- 172.21.15.119:0/2361865354 >> v1:172.21.15.132:6789/0 conn(0x7f85140b2c90 legacy=0x7f85140cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:02.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.814+0000 7f851a200640 1 -- 172.21.15.119:0/2361865354 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851415fab0 con 0x7f85140ccb50 2024-04-25T10:09:02.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.815+0000 7f851a200640 1 --2- 172.21.15.119:0/2361865354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85140ccb50 0x7f851406b0e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f850c009e50 tx=0x7f850c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fef188594e0aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.816+0000 7f8513e00640 1 -- 172.21.15.119:0/2361865354 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c036070 con 0x7f85140ccb50 2024-04-25T10:09:02.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.816+0000 7f8513e00640 1 -- 172.21.15.119:0/2361865354 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c031600 con 0x7f85140ccb50 2024-04-25T10:09:02.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.816+0000 7f8513e00640 1 -- 172.21.15.119:0/2361865354 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c0319d0 con 0x7f85140ccb50 2024-04-25T10:09:02.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.816+0000 7f851ac00640 1 -- 172.21.15.119:0/2361865354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85140ccb50 msgr2=0x7f851406b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.816+0000 7f851ac00640 1 --2- 172.21.15.119:0/2361865354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85140ccb50 0x7f851406b0e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f850c009e50 tx=0x7f850c02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.817+0000 7f851ac00640 1 -- 172.21.15.119:0/2361865354 shutdown_connections 2024-04-25T10:09:02.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.817+0000 7f851ac00640 1 --2- 172.21.15.119:0/2361865354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85140ccb50 0x7f851406b0e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.817+0000 7f851ac00640 1 --2- 172.21.15.119:0/2361865354 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8514144fc0 0x7f851406aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.817+0000 7f851ac00640 1 -- 172.21.15.119:0/2361865354 >> 172.21.15.119:0/2361865354 conn(0x7f8514067040 msgr2=0x7f85140b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:02.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.817+0000 7f851ac00640 1 -- 172.21.15.119:0/2361865354 shutdown_connections 2024-04-25T10:09:02.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.817+0000 7f851ac00640 1 -- 172.21.15.119:0/2361865354 wait complete. 2024-04-25T10:09:02.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.818+0000 7f851ac00640 1 Processor -- start 2024-04-25T10:09:02.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.819+0000 7f851ac00640 1 -- start start 2024-04-25T10:09:02.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.819+0000 7f851ac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85140b2c90 0x7f8514136d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.819+0000 7f851ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85140ccb50 0x7f85140ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f851ac00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514144fc0 0x7f85140adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f851ac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851415bef0 con 0x7f85140b2c90 2024-04-25T10:09:02.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f851ac00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85141453a0 con 0x7f8514144fc0 2024-04-25T10:09:02.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f851ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851415c070 con 0x7f85140ccb50 2024-04-25T10:09:02.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f8519800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85140b2c90 0x7f8514136d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f8519800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85140b2c90 0x7f8514136d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38292/0 (socket says 172.21.15.119:38292) 2024-04-25T10:09:02.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f8519800640 1 -- 172.21.15.119:0/551525938 learned_addr learned my addr 172.21.15.119:0/551525938 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:02.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f851a200640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514144fc0 0x7f85140adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f8518e00640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85140ccb50 0x7f85140ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f8519800640 1 -- 172.21.15.119:0/551525938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85140ccb50 msgr2=0x7f85140ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f8519800640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85140ccb50 0x7f85140ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f8519800640 1 -- 172.21.15.119:0/551525938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514144fc0 msgr2=0x7f85140adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f8519800640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514144fc0 0x7f85140adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f8519800640 1 -- 172.21.15.119:0/551525938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85141e5330 con 0x7f85140b2c90 2024-04-25T10:09:02.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.820+0000 7f851a200640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514144fc0 0x7f85140adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:02.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.821+0000 7f8519800640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85140b2c90 0x7f8514136d50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f850800c7f0 tx=0x7f850800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.821+0000 7f8511600640 1 -- 172.21.15.119:0/551525938 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508015070 con 0x7f85140b2c90 2024-04-25T10:09:02.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.821+0000 7f8511600640 1 -- 172.21.15.119:0/551525938 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8508004220 con 0x7f85140b2c90 2024-04-25T10:09:02.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.821+0000 7f851ac00640 1 -- 172.21.15.119:0/551525938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85141e5610 con 0x7f85140b2c90 2024-04-25T10:09:02.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.821+0000 7f8511600640 1 -- 172.21.15.119:0/551525938 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508007620 con 0x7f85140b2c90 2024-04-25T10:09:02.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.821+0000 7f851ac00640 1 -- 172.21.15.119:0/551525938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85141e5bd0 con 0x7f85140b2c90 2024-04-25T10:09:02.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.822+0000 7f851ac00640 1 -- 172.21.15.119:0/551525938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85140749b0 con 0x7f85140b2c90 2024-04-25T10:09:02.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.823+0000 7f8511600640 1 -- 172.21.15.119:0/551525938 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85080079d0 con 0x7f85140b2c90 2024-04-25T10:09:02.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.824+0000 7f8511600640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f84f406b4d0 0x7f84f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.824+0000 7f8511600640 1 -- 172.21.15.119:0/551525938 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f850808a6f0 con 0x7f85140b2c90 2024-04-25T10:09:02.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.824+0000 7f8518e00640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f84f406b4d0 0x7f84f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.825+0000 7f8518e00640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f84f406b4d0 0x7f84f406d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f85141e67c0 tx=0x7f8500009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:02.828+0000 7f8511600640 1 -- 172.21.15.119:0/551525938 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8508059440 con 0x7f85140b2c90 2024-04-25T10:09:03.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.095+0000 7f851ac00640 1 -- 172.21.15.119:0/551525938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f85141e5da0 con 0x7f85140b2c90 2024-04-25T10:09:03.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.101+0000 7f8511600640 1 -- 172.21.15.119:0/551525938 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8508058d40 con 0x7f85140b2c90 2024-04-25T10:09:03.107 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-04-25T10:09:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA== 2024-04-25T10:09:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: caps mds = "allow" 2024-04-25T10:09:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: caps mon = "allow r" 2024-04-25T10:09:03.107 INFO:teuthology.orchestra.run.smithi119.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:09:03.107 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2024-04-25T10:09:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.103+0000 7f851ac00640 1 -- 172.21.15.119:0/551525938 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f84f406b4d0 msgr2=0x7f84f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.103+0000 7f851ac00640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f84f406b4d0 0x7f84f406d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f85141e67c0 tx=0x7f8500009040 comp rx=0 tx=0).stop 2024-04-25T10:09:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.104+0000 7f851ac00640 1 -- 172.21.15.119:0/551525938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85140b2c90 msgr2=0x7f8514136d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.104+0000 7f851ac00640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85140b2c90 0x7f8514136d50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f850800c7f0 tx=0x7f850800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.104+0000 7f851ac00640 1 -- 172.21.15.119:0/551525938 shutdown_connections 2024-04-25T10:09:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.104+0000 7f851ac00640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f84f406b4d0 0x7f84f406d990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.104+0000 7f851ac00640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8514144fc0 0x7f85140adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.104+0000 7f851ac00640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85140ccb50 0x7f85140ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.104+0000 7f851ac00640 1 --2- 172.21.15.119:0/551525938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85140b2c90 0x7f8514136d50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.104+0000 7f851ac00640 1 -- 172.21.15.119:0/551525938 >> 172.21.15.119:0/551525938 conn(0x7f8514067040 msgr2=0x7f85140cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:03.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.104+0000 7f851ac00640 1 -- 172.21.15.119:0/551525938 shutdown_connections 2024-04-25T10:09:03.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.104+0000 7f851ac00640 1 -- 172.21.15.119:0/551525938 wait complete. 2024-04-25T10:09:03.129 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:03.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.319+0000 7f8ba2000640 1 Processor -- start 2024-04-25T10:09:03.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.320+0000 7f8ba2000640 1 -- start start 2024-04-25T10:09:03.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.321+0000 7f8ba2000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b9c0b4020 0x7f8b9c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.321+0000 7f8ba2000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8b9c0cdbb0 0x7f8b9c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.321+0000 7f8ba2000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b9c0cc020 con 0x7f8b9c15a730 2024-04-25T10:09:03.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.321+0000 7f8ba2000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b9c0b49a0 con 0x7f8b9c0cdbb0 2024-04-25T10:09:03.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.321+0000 7f8ba2000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c0b32b0 con 0x7f8b9c0b4020 2024-04-25T10:09:03.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.322+0000 7f8b9aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b9c0b4020 0x7f8b9c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.322+0000 7f8b9aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b9c0b4020 0x7f8b9c0b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:57788/0 (socket says 172.21.15.119:57788) 2024-04-25T10:09:03.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.322+0000 7f8b9aa00640 1 -- 172.21.15.119:0/3873862305 learned_addr learned my addr 172.21.15.119:0/3873862305 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:03.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.322+0000 7f8b9be00640 1 --2- 172.21.15.119:0/3873862305 >> v2:172.21.15.132:3300/0 conn(0x7f8b9c0cdbb0 0x7f8b9c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.322+0000 7f8b9aa00640 1 -- 172.21.15.119:0/3873862305 >> v2:172.21.15.132:3300/0 conn(0x7f8b9c0cdbb0 msgr2=0x7f8b9c0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:03.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.322+0000 7f8b9aa00640 1 --2- 172.21.15.119:0/3873862305 >> v2:172.21.15.132:3300/0 conn(0x7f8b9c0cdbb0 0x7f8b9c0ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.322+0000 7f8b9aa00640 1 -- 172.21.15.119:0/3873862305 >> v1:172.21.15.132:6789/0 conn(0x7f8b9c15a730 legacy=0x7f8b9c0ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.322+0000 7f8b9aa00640 1 -- 172.21.15.119:0/3873862305 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9c0b3590 con 0x7f8b9c0b4020 2024-04-25T10:09:03.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.322+0000 7f8b9be00640 1 --2- 172.21.15.119:0/3873862305 >> v2:172.21.15.132:3300/0 conn(0x7f8b9c0cdbb0 0x7f8b9c0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:03.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.324+0000 7f8b9aa00640 1 --2- 172.21.15.119:0/3873862305 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b9c0b4020 0x7f8b9c0b4460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0b4b20 tx=0x7f8b8402c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33b5e71e07d9e103 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.324+0000 7f8b9a000640 1 -- 172.21.15.119:0/3873862305 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b84036020 con 0x7f8b9c0b4020 2024-04-25T10:09:03.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.324+0000 7f8b9a000640 1 -- 172.21.15.119:0/3873862305 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b84032800 con 0x7f8b9c0b4020 2024-04-25T10:09:03.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.324+0000 7f8b9a000640 1 -- 172.21.15.119:0/3873862305 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b84032bd0 con 0x7f8b9c0b4020 2024-04-25T10:09:03.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.325+0000 7f8ba2000640 1 -- 172.21.15.119:0/3873862305 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b9c0b4020 msgr2=0x7f8b9c0b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.325+0000 7f8ba2000640 1 --2- 172.21.15.119:0/3873862305 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b9c0b4020 0x7f8b9c0b4460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0b4b20 tx=0x7f8b8402c900 comp rx=0 tx=0).stop 2024-04-25T10:09:03.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.325+0000 7f8ba2000640 1 -- 172.21.15.119:0/3873862305 shutdown_connections 2024-04-25T10:09:03.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.325+0000 7f8ba2000640 1 --2- 172.21.15.119:0/3873862305 >> v2:172.21.15.132:3300/0 conn(0x7f8b9c0cdbb0 0x7f8b9c0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.325+0000 7f8ba2000640 1 --2- 172.21.15.119:0/3873862305 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b9c0b4020 0x7f8b9c0b4460 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.325+0000 7f8ba2000640 1 -- 172.21.15.119:0/3873862305 >> 172.21.15.119:0/3873862305 conn(0x7f8b9c06aba0 msgr2=0x7f8b9c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:03.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.325+0000 7f8ba2000640 1 -- 172.21.15.119:0/3873862305 shutdown_connections 2024-04-25T10:09:03.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.325+0000 7f8ba2000640 1 -- 172.21.15.119:0/3873862305 wait complete. 2024-04-25T10:09:03.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.327+0000 7f8ba2000640 1 Processor -- start 2024-04-25T10:09:03.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.327+0000 7f8ba2000640 1 -- start start 2024-04-25T10:09:03.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8ba2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b9c0b4020 0x7f8b9c140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8ba2000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b9c0cdbb0 0x7f8b9c15ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8ba2000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b9c15a730 0x7f8b9c0aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8ba2000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c0cd620 con 0x7f8b9c0b4020 2024-04-25T10:09:03.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8ba2000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c0b3b50 con 0x7f8b9c0cdbb0 2024-04-25T10:09:03.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8ba2000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c0b2f30 con 0x7f8b9c15a730 2024-04-25T10:09:03.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8b9b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b9c0b4020 0x7f8b9c140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8b9b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b9c0b4020 0x7f8b9c140040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38302/0 (socket says 172.21.15.119:38302) 2024-04-25T10:09:03.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8b9b400640 1 -- 172.21.15.119:0/1332765788 learned_addr learned my addr 172.21.15.119:0/1332765788 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:03.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8b9aa00640 1 --2- 172.21.15.119:0/1332765788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b9c0cdbb0 0x7f8b9c15ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8b9be00640 1 --2- 172.21.15.119:0/1332765788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b9c15a730 0x7f8b9c0aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8b9b400640 1 -- 172.21.15.119:0/1332765788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b9c15a730 msgr2=0x7f8b9c0aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8b9b400640 1 --2- 172.21.15.119:0/1332765788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b9c15a730 0x7f8b9c0aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8b9b400640 1 -- 172.21.15.119:0/1332765788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b9c0cdbb0 msgr2=0x7f8b9c15ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8b9b400640 1 --2- 172.21.15.119:0/1332765788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b9c0cdbb0 0x7f8b9c15ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.328+0000 7f8b9b400640 1 -- 172.21.15.119:0/1332765788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9c15b360 con 0x7f8b9c0b4020 2024-04-25T10:09:03.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.329+0000 7f8b9b400640 1 --2- 172.21.15.119:0/1332765788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b9c0b4020 0x7f8b9c140040 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00c390 tx=0x7f8b8c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.329+0000 7f8b93400640 1 -- 172.21.15.119:0/1332765788 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0052d0 con 0x7f8b9c0b4020 2024-04-25T10:09:03.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.329+0000 7f8b93400640 1 -- 172.21.15.119:0/1332765788 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c005470 con 0x7f8b9c0b4020 2024-04-25T10:09:03.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.329+0000 7f8ba2000640 1 -- 172.21.15.119:0/1332765788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9c15b640 con 0x7f8b9c0b4020 2024-04-25T10:09:03.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.329+0000 7f8b93400640 1 -- 172.21.15.119:0/1332765788 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c011a70 con 0x7f8b9c0b4020 2024-04-25T10:09:03.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.329+0000 7f8ba2000640 1 -- 172.21.15.119:0/1332765788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9c15c0b0 con 0x7f8b9c0b4020 2024-04-25T10:09:03.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.330+0000 7f8ba2000640 1 -- 172.21.15.119:0/1332765788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9c0749b0 con 0x7f8b9c0b4020 2024-04-25T10:09:03.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.331+0000 7f8b93400640 1 -- 172.21.15.119:0/1332765788 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b8c011c10 con 0x7f8b9c0b4020 2024-04-25T10:09:03.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.332+0000 7f8b93400640 1 --2- 172.21.15.119:0/1332765788 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b7806b4d0 0x7f8b7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.332+0000 7f8b93400640 1 -- 172.21.15.119:0/1332765788 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b8c08dab0 con 0x7f8b9c0b4020 2024-04-25T10:09:03.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.332+0000 7f8b9aa00640 1 --2- 172.21.15.119:0/1332765788 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b7806b4d0 0x7f8b7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.333+0000 7f8b9aa00640 1 --2- 172.21.15.119:0/1332765788 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b7806b4d0 0x7f8b7806d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8b84037000 tx=0x7f8b84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.337+0000 7f8b93400640 1 -- 172.21.15.119:0/1332765788 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b8c05c2d0 con 0x7f8b9c0b4020 2024-04-25T10:09:03.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.561+0000 7f8ba2000640 1 -- 172.21.15.119:0/1332765788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b9c0b4760 con 0x7f8b9c0b4020 2024-04-25T10:09:03.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.563+0000 7f8b93400640 1 -- 172.21.15.119:0/1332765788 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f8b8c060370 con 0x7f8b9c0b4020 2024-04-25T10:09:03.569 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:03.569 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3317744182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3317744182},{"type":"v1","addr":"172.21.15.132:6833","nonce":3317744182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/700427287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":700427287},{"type":"v1","addr":"172.21.15.132:6835","nonce":700427287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2207174972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2207174972},{"type":"v1","addr":"172.21.15.119:6836","nonce":2207174972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:57.120451+0000","modified":"2024-04-25T10:09:00.157396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5478":{"gid":5478,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/270766821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":270766821},{"type":"v1","addr":"172.21.15.119:6837","nonce":270766821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:09:03.569 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 25 2024-04-25T10:09:03.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.565+0000 7f8ba2000640 1 -- 172.21.15.119:0/1332765788 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b7806b4d0 msgr2=0x7f8b7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.565+0000 7f8ba2000640 1 --2- 172.21.15.119:0/1332765788 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b7806b4d0 0x7f8b7806d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8b84037000 tx=0x7f8b84008040 comp rx=0 tx=0).stop 2024-04-25T10:09:03.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.565+0000 7f8ba2000640 1 -- 172.21.15.119:0/1332765788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b9c0b4020 msgr2=0x7f8b9c140040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.565+0000 7f8ba2000640 1 --2- 172.21.15.119:0/1332765788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b9c0b4020 0x7f8b9c140040 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00c390 tx=0x7f8b8c00c850 comp rx=0 tx=0).stop 2024-04-25T10:09:03.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.566+0000 7f8ba2000640 1 -- 172.21.15.119:0/1332765788 shutdown_connections 2024-04-25T10:09:03.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.566+0000 7f8ba2000640 1 --2- 172.21.15.119:0/1332765788 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8b7806b4d0 0x7f8b7806d990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.566+0000 7f8ba2000640 1 --2- 172.21.15.119:0/1332765788 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8b9c15a730 0x7f8b9c0aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.566+0000 7f8ba2000640 1 --2- 172.21.15.119:0/1332765788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b9c0cdbb0 0x7f8b9c15ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.566+0000 7f8ba2000640 1 --2- 172.21.15.119:0/1332765788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b9c0b4020 0x7f8b9c140040 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.566+0000 7f8ba2000640 1 -- 172.21.15.119:0/1332765788 >> 172.21.15.119:0/1332765788 conn(0x7f8b9c06aba0 msgr2=0x7f8b9c1560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:03.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.566+0000 7f8ba2000640 1 -- 172.21.15.119:0/1332765788 shutdown_connections 2024-04-25T10:09:03.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:03.566+0000 7f8ba2000640 1 -- 172.21.15.119:0/1332765788 wait complete. 2024-04-25T10:09:03.616 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:09:03.610+0000 7f5287600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:09:03.993 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:57.120451+0000', 'modified': '2024-04-25T10:09:00.157396+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5478': {'gid': 5478, 'name': 'a', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6837/270766821', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 270766821}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 270766821}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:09:03.993 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:09:03.993 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:09:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.194+0000 7f48e3e00640 1 Processor -- start 2024-04-25T10:09:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.195+0000 7f48e3e00640 1 -- start start 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.196+0000 7f48e3e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f48e40b3500 0x7f48e4166380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.196+0000 7f48e3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48e40cea30 0x7f48e406dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.196+0000 7f48e3e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e4162140 con 0x7f48e4161d60 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.196+0000 7f48e3e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e40b2470 con 0x7f48e40b3500 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.196+0000 7f48e3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e4165ef0 con 0x7f48e40cea30 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.197+0000 7f48e2a00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f48e4161d60 0x7f48e40b2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:45908/0 (socket says 172.21.15.119:45908) 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.197+0000 7f48e2a00640 1 -- 172.21.15.119:0/3480776830 learned_addr learned my addr 172.21.15.119:0/3480776830 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.197+0000 7f48e3400640 1 --2- 172.21.15.119:0/3480776830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48e40cea30 0x7f48e406dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.197+0000 7f48e2000640 1 --2- 172.21.15.119:0/3480776830 >> v2:172.21.15.132:3300/0 conn(0x7f48e40b3500 0x7f48e4166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.198+0000 7f48e3400640 1 -- 172.21.15.119:0/3480776830 >> v2:172.21.15.132:3300/0 conn(0x7f48e40b3500 msgr2=0x7f48e4166380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.198+0000 7f48e3400640 1 --2- 172.21.15.119:0/3480776830 >> v2:172.21.15.132:3300/0 conn(0x7f48e40b3500 0x7f48e4166380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.198+0000 7f48e3400640 1 -- 172.21.15.119:0/3480776830 >> v1:172.21.15.132:6789/0 conn(0x7f48e4161d60 legacy=0x7f48e40b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.198+0000 7f48e3400640 1 -- 172.21.15.119:0/3480776830 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e40cf210 con 0x7f48e40cea30 2024-04-25T10:09:04.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.199+0000 7f48e3400640 1 --2- 172.21.15.119:0/3480776830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48e40cea30 0x7f48e406dcf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009d90 tx=0x7f48dc02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b64c39f9b3b67873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.200+0000 7f48e1600640 1 -- 172.21.15.119:0/3480776830 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc031600 con 0x7f48e40cea30 2024-04-25T10:09:04.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.200+0000 7f48e1600640 1 -- 172.21.15.119:0/3480776830 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc0317a0 con 0x7f48e40cea30 2024-04-25T10:09:04.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.200+0000 7f48e1600640 1 -- 172.21.15.119:0/3480776830 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc031b70 con 0x7f48e40cea30 2024-04-25T10:09:04.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.201+0000 7f48e3e00640 1 -- 172.21.15.119:0/3480776830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48e40cea30 msgr2=0x7f48e406dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.201+0000 7f48e3e00640 1 --2- 172.21.15.119:0/3480776830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48e40cea30 0x7f48e406dcf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009d90 tx=0x7f48dc02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.201+0000 7f48e3e00640 1 -- 172.21.15.119:0/3480776830 shutdown_connections 2024-04-25T10:09:04.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.201+0000 7f48e3e00640 1 --2- 172.21.15.119:0/3480776830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48e40cea30 0x7f48e406dcf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.201+0000 7f48e3e00640 1 --2- 172.21.15.119:0/3480776830 >> v2:172.21.15.132:3300/0 conn(0x7f48e40b3500 0x7f48e4166380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.201+0000 7f48e3e00640 1 -- 172.21.15.119:0/3480776830 >> 172.21.15.119:0/3480776830 conn(0x7f48e4066ff0 msgr2=0x7f48e40b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.201+0000 7f48e3e00640 1 -- 172.21.15.119:0/3480776830 shutdown_connections 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.201+0000 7f48e3e00640 1 -- 172.21.15.119:0/3480776830 wait complete. 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.202+0000 7f48e3e00640 1 Processor -- start 2024-04-25T10:09:04.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.203+0000 7f48e3e00640 1 -- start start 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48e40b3500 0x7f48e41f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48e40cea30 0x7f48e4164b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e3e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48e4161d60 0x7f48e415e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e3e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e41658a0 con 0x7f48e40b3500 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e3e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e40b29a0 con 0x7f48e4161d60 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e4165b80 con 0x7f48e40cea30 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48e40b3500 0x7f48e41f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48e40b3500 0x7f48e41f8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38318/0 (socket says 172.21.15.119:38318) 2024-04-25T10:09:04.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e2a00640 1 -- 172.21.15.119:0/1786532765 learned_addr learned my addr 172.21.15.119:0/1786532765 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e2000640 1 --2- 172.21.15.119:0/1786532765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48e40cea30 0x7f48e4164b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e3400640 1 --2- 172.21.15.119:0/1786532765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48e4161d60 0x7f48e415e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e2a00640 1 -- 172.21.15.119:0/1786532765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48e40cea30 msgr2=0x7f48e4164b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e2a00640 1 --2- 172.21.15.119:0/1786532765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48e40cea30 0x7f48e4164b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e2a00640 1 -- 172.21.15.119:0/1786532765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48e4161d60 msgr2=0x7f48e415e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e2a00640 1 --2- 172.21.15.119:0/1786532765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48e4161d60 0x7f48e415e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.204+0000 7f48e2a00640 1 -- 172.21.15.119:0/1786532765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e4155a40 con 0x7f48e40b3500 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.205+0000 7f48e2a00640 1 --2- 172.21.15.119:0/1786532765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48e40b3500 0x7f48e41f8fd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f48d400c400 tx=0x7f48d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.205+0000 7f48db400640 1 -- 172.21.15.119:0/1786532765 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4017070 con 0x7f48e40b3500 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.205+0000 7f48db400640 1 -- 172.21.15.119:0/1786532765 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d4005ce0 con 0x7f48e40b3500 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.205+0000 7f48e3e00640 1 -- 172.21.15.119:0/1786532765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e4146ac0 con 0x7f48e40b3500 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.205+0000 7f48db400640 1 -- 172.21.15.119:0/1786532765 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4007310 con 0x7f48e40b3500 2024-04-25T10:09:04.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.205+0000 7f48e3e00640 1 -- 172.21.15.119:0/1786532765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e4147080 con 0x7f48e40b3500 2024-04-25T10:09:04.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.207+0000 7f48db400640 1 -- 172.21.15.119:0/1786532765 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48d401a070 con 0x7f48e40b3500 2024-04-25T10:09:04.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.208+0000 7f48db400640 1 --2- 172.21.15.119:0/1786532765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f48c406b500 0x7f48c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.208+0000 7f48db400640 1 -- 172.21.15.119:0/1786532765 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48d40895b0 con 0x7f48e40b3500 2024-04-25T10:09:04.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.208+0000 7f48e2000640 1 --2- 172.21.15.119:0/1786532765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f48c406b500 0x7f48c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.209+0000 7f48ecbf2740 1 Processor -- start 2024-04-25T10:09:04.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.209+0000 7f48ecbf2740 1 -- start start 2024-04-25T10:09:04.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.209+0000 7f48e2000640 1 --2- 172.21.15.119:0/1786532765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f48c406b500 0x7f48c406d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f48e41474c0 tx=0x7f48d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.210+0000 7f48ecbf2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f10e0b30 0x55e2f1108970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.210+0000 7f48ecbf2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1108eb0 0x55e2f10e0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.210+0000 7f48ecbf2740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 0x55e2f10df2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.210+0000 7f48ecbf2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2f1044620 con 0x55e2f10e0b30 2024-04-25T10:09:04.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.210+0000 7f48ecbf2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2f0fedca0 con 0x55e2f1108eb0 2024-04-25T10:09:04.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.210+0000 7f48ecbf2740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2f10406c0 con 0x55e2f11297a0 2024-04-25T10:09:04.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.210+0000 7f48e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f10e0b30 0x55e2f1108970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.210+0000 7f48e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f10e0b30 0x55e2f1108970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38336/0 (socket says 172.21.15.119:38336) 2024-04-25T10:09:04.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.210+0000 7f48e2a00640 1 -- 172.21.15.119:0/962066816 learned_addr learned my addr 172.21.15.119:0/962066816 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:04.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.210+0000 7f48e2a00640 1 -- 172.21.15.119:0/962066816 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 msgr2=0x55e2f10df2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.210+0000 7f48e2a00640 1 --2- 172.21.15.119:0/962066816 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 0x55e2f10df2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.210+0000 7f48e2a00640 1 -- 172.21.15.119:0/962066816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1108eb0 msgr2=0x55e2f10e0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.210+0000 7f48e2a00640 1 --2- 172.21.15.119:0/962066816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1108eb0 0x55e2f10e0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.210+0000 7f48e2a00640 1 -- 172.21.15.119:0/962066816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2f0ff4f40 con 0x55e2f10e0b30 2024-04-25T10:09:04.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.211+0000 7f48e2a00640 1 --2- 172.21.15.119:0/962066816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f10e0b30 0x55e2f1108970 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f48d405ff00 tx=0x7f48d4057d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73587e65a080bd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.211+0000 7f48cf400640 1 -- 172.21.15.119:0/962066816 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4055dc0 con 0x55e2f10e0b30 2024-04-25T10:09:04.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.211+0000 7f48cf400640 1 -- 172.21.15.119:0/962066816 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d4056020 con 0x55e2f10e0b30 2024-04-25T10:09:04.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.211+0000 7f48cf400640 1 -- 172.21.15.119:0/962066816 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d40074b0 con 0x55e2f10e0b30 2024-04-25T10:09:04.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.211+0000 7f48ecbf2740 1 -- 172.21.15.119:0/962066816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f10e0b30 msgr2=0x55e2f1108970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.211+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/962066816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f10e0b30 0x55e2f1108970 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f48d405ff00 tx=0x7f48d4057d20 comp rx=0 tx=0).stop 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.211+0000 7f48ecbf2740 1 -- 172.21.15.119:0/962066816 shutdown_connections 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.211+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/962066816 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 0x55e2f10df2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.211+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/962066816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1108eb0 0x55e2f10e0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.211+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/962066816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f10e0b30 0x55e2f1108970 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.211+0000 7f48ecbf2740 1 -- 172.21.15.119:0/962066816 >> 172.21.15.119:0/962066816 conn(0x55e2f10ddd50 msgr2=0x55e2f1126730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.211+0000 7f48ecbf2740 1 -- 172.21.15.119:0/962066816 shutdown_connections 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.212+0000 7f48ecbf2740 1 -- 172.21.15.119:0/962066816 wait complete. 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.212+0000 7f48ecbf2740 1 -- start start 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.212+0000 7f48ecbf2740 1 Processor -- start 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.213+0000 7f48ecbf2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f11043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.213+0000 7f48ecbf2740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 0x55e2f110b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.213+0000 7f48ecbf2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1129e20 0x55e2f112a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.213+0000 7f48ecbf2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2f11084e0 con 0x55e2f1108eb0 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.213+0000 7f48ecbf2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2f0fe31f0 con 0x55e2f1129e20 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.213+0000 7f48ecbf2740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2f1107ed0 con 0x55e2f11297a0 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.213+0000 7f48ecbf2740 10 client.0 _ll_register_callbacks cb 0x55e2f1000ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.214+0000 7f48e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f11043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.214+0000 7f48e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f11043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38354/0 (socket says 172.21.15.119:38354) 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.214+0000 7f48e2a00640 1 -- 172.21.15.119:0/306553121 learned_addr learned my addr 172.21.15.119:0/306553121 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.214+0000 7f48e2a00640 1 -- 172.21.15.119:0/306553121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 msgr2=0x55e2f110b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.214+0000 7f48e2000640 1 --2- 172.21.15.119:0/306553121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 0x55e2f110b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.214+0000 7f48e2a00640 1 --2- 172.21.15.119:0/306553121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 0x55e2f110b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.214+0000 7f48e2a00640 1 -- 172.21.15.119:0/306553121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1129e20 msgr2=0x55e2f112a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.214+0000 7f48e2a00640 1 --2- 172.21.15.119:0/306553121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1129e20 0x55e2f112a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.214+0000 7f48e2a00640 1 -- 172.21.15.119:0/306553121 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2f1002c50 con 0x55e2f1108eb0 2024-04-25T10:09:04.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.214+0000 7f48e2a00640 1 --2- 172.21.15.119:0/306553121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f11043a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f48d400cdc0 tx=0x7f48d401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3a1b264eb13e52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.214+0000 7f48c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.214+0000 7f48c3e00640 1 -- 172.21.15.119:0/306553121 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d40076b0 con 0x55e2f1108eb0 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.214+0000 7f48c3e00640 1 -- 172.21.15.119:0/306553121 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48d40890c0 con 0x55e2f1108eb0 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.215+0000 7f48ecbf2740 1 -- 172.21.15.119:0/306553121 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48d4056020 con 0x55e2f1108eb0 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.215+0000 7f48c3e00640 1 -- 172.21.15.119:0/306553121 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48d40892c0 con 0x55e2f1108eb0 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.215+0000 7f48ecbf2740 10 client.5616 fetch_fsmap learned FSMap version 25 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.215+0000 7f48ecbf2740 1 -- 172.21.15.119:0/306553121 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55e2f0d592d0 con 0x55e2f1108eb0 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.215+0000 7f48c3e00640 1 -- 172.21.15.119:0/306553121 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d4056020 con 0x55e2f1108eb0 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.215+0000 7f48c3e00640 1 -- 172.21.15.119:0/306553121 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f48d406bce0 con 0x55e2f1108eb0 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.215+0000 7f48ecbf2740 10 client.5616 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.215+0000 7f48ecbf2740 10 client.5616 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.215+0000 7f48ecbf2740 20 client.5616 populate_metadata read hostname 'smithi119' 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.215+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/306553121 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55e2f1136de0 0x55e2f11371c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.215+0000 7f48ecbf2740 4 client.5616 mds_command: new command op to 5478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:04.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.215+0000 7f48ecbf2740 1 -- 172.21.15.119:0/306553121 --> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2f1051b60 con 0x55e2f1136de0 2024-04-25T10:09:04.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.215+0000 7f48e2000640 1 --2- 172.21.15.119:0/306553121 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55e2f1136de0 0x55e2f11371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.216+0000 7f48e2000640 1 --2- 172.21.15.119:0/306553121 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55e2f1136de0 0x55e2f11371c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab322558a59dd1e0 server_cookie=3589afa26eada6b0 in_seq=0 out_seq=0 2024-04-25T10:09:04.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.216+0000 7f48c3e00640 10 client.5616 ms_handle_connect on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.216+0000 7f48c3e00640 1 -- 172.21.15.119:0/306553121 <== mds.0 v2:172.21.15.119:6835/270766821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e2f1051b60 con 0x55e2f1136de0 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.216+0000 7f48c3e00640 10 client.5616 handle_command_reply: tid=0 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.217+0000 7f48ecbf2740 1 client.5616 shutdown 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.217+0000 7f48ecbf2740 1 -- 172.21.15.119:0/306553121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 msgr2=0x55e2f11043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.217+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/306553121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f11043a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f48d400cdc0 tx=0x7f48d401e040 comp rx=0 tx=0).stop 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.217+0000 7f48ecbf2740 1 -- 172.21.15.119:0/306553121 shutdown_connections 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.217+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/306553121 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55e2f1136de0 0x55e2f11371c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.217+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/306553121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1129e20 0x55e2f112a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.217+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/306553121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 0x55e2f110b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.217+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/306553121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f11043a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.217+0000 7f48c3e00640 0 client.5616 ms_handle_reset on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.217+0000 7f48ecbf2740 1 -- 172.21.15.119:0/306553121 >> 172.21.15.119:0/306553121 conn(0x55e2f10e0b30 msgr2=0x55e2f1126530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.217+0000 7f48ecbf2740 1 -- 172.21.15.119:0/306553121 shutdown_connections 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.217+0000 7f48ecbf2740 1 -- 172.21.15.119:0/306553121 wait complete. 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.217+0000 7f48ecbf2740 20 client.5616 trim_cache size 0 max 16384 2024-04-25T10:09:04.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.234+0000 7f48ecbf2740 1 Processor -- start 2024-04-25T10:09:04.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.234+0000 7f48ecbf2740 1 -- start start 2024-04-25T10:09:04.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.235+0000 7f48ecbf2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f110cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.235+0000 7f48ecbf2740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 0x55e2f11371c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.235+0000 7f48ecbf2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1129e20 0x55e2f1137700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.235+0000 7f48ecbf2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e4156f20 con 0x55e2f1108eb0 2024-04-25T10:09:04.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.235+0000 7f48ecbf2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e40cc990 con 0x55e2f1129e20 2024-04-25T10:09:04.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.235+0000 7f48ecbf2740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e40ccf90 con 0x55e2f11297a0 2024-04-25T10:09:04.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.235+0000 7f48e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f110cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.235+0000 7f48e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f110cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38366/0 (socket says 172.21.15.119:38366) 2024-04-25T10:09:04.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.235+0000 7f48e2a00640 1 -- 172.21.15.119:0/4166609673 learned_addr learned my addr 172.21.15.119:0/4166609673 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:04.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.235+0000 7f48e2a00640 1 -- 172.21.15.119:0/4166609673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 msgr2=0x55e2f11371c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.235+0000 7f48e2a00640 1 --2- 172.21.15.119:0/4166609673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 0x55e2f11371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.236+0000 7f48e2a00640 1 -- 172.21.15.119:0/4166609673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1129e20 msgr2=0x55e2f1137700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.236+0000 7f48e2a00640 1 --2- 172.21.15.119:0/4166609673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1129e20 0x55e2f1137700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.236+0000 7f48e2a00640 1 -- 172.21.15.119:0/4166609673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2f0e236b0 con 0x55e2f1108eb0 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.236+0000 7f48e2a00640 1 --2- 172.21.15.119:0/4166609673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f110cf30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f48d40574a0 tx=0x7f48d4083070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58fbd2f8961563 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.236+0000 7f48cea00640 1 -- 172.21.15.119:0/4166609673 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d407b070 con 0x55e2f1108eb0 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.236+0000 7f48cea00640 1 -- 172.21.15.119:0/4166609673 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d4085030 con 0x55e2f1108eb0 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.236+0000 7f48cea00640 1 -- 172.21.15.119:0/4166609673 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d40074b0 con 0x55e2f1108eb0 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.237+0000 7f48ecbf2740 1 -- 172.21.15.119:0/4166609673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 msgr2=0x55e2f110cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.237+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/4166609673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f110cf30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f48d40574a0 tx=0x7f48d4083070 comp rx=0 tx=0).stop 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.237+0000 7f48ecbf2740 1 -- 172.21.15.119:0/4166609673 shutdown_connections 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.237+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/4166609673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1129e20 0x55e2f1137700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.237+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/4166609673 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 0x55e2f11371c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.237+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/4166609673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f110cf30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.237+0000 7f48ecbf2740 1 -- 172.21.15.119:0/4166609673 >> 172.21.15.119:0/4166609673 conn(0x55e2f10e0b30 msgr2=0x55e2f11076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.237+0000 7f48ecbf2740 1 -- 172.21.15.119:0/4166609673 shutdown_connections 2024-04-25T10:09:04.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.237+0000 7f48ecbf2740 1 -- 172.21.15.119:0/4166609673 wait complete. 2024-04-25T10:09:04.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.237+0000 7f48ecbf2740 1 -- start start 2024-04-25T10:09:04.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.238+0000 7f48ecbf2740 1 Processor -- start 2024-04-25T10:09:04.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48ecbf2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f11912e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48ecbf2740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 0x55e2f11061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48ecbf2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1129e20 0x55e2f1106700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48ecbf2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2f11084e0 con 0x55e2f1108eb0 2024-04-25T10:09:04.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48ecbf2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2f1107ed0 con 0x55e2f1129e20 2024-04-25T10:09:04.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48ecbf2740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2f0fe31f0 con 0x55e2f11297a0 2024-04-25T10:09:04.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48ecbf2740 10 client.0 _ll_register_callbacks cb 0x7f48d406e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:04.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f11912e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f11912e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38370/0 (socket says 172.21.15.119:38370) 2024-04-25T10:09:04.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48e2a00640 1 -- 172.21.15.119:0/201921889 learned_addr learned my addr 172.21.15.119:0/201921889 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:04.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48e2a00640 1 -- 172.21.15.119:0/201921889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 msgr2=0x55e2f11061c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48e2a00640 1 --2- 172.21.15.119:0/201921889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 0x55e2f11061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48e2a00640 1 -- 172.21.15.119:0/201921889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1129e20 msgr2=0x55e2f1106700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48e2a00640 1 --2- 172.21.15.119:0/201921889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1129e20 0x55e2f1106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.239+0000 7f48e2a00640 1 -- 172.21.15.119:0/201921889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2f1127420 con 0x55e2f1108eb0 2024-04-25T10:09:04.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48e2a00640 1 --2- 172.21.15.119:0/201921889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f11912e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f48d4055fd0 tx=0x7f48d4054040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504821b71e1bbcd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:09:04.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48c3e00640 1 -- 172.21.15.119:0/201921889 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d407b070 con 0x55e2f1108eb0 2024-04-25T10:09:04.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48c3e00640 1 -- 172.21.15.119:0/201921889 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48d4056020 con 0x55e2f1108eb0 2024-04-25T10:09:04.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48ecbf2740 1 -- 172.21.15.119:0/201921889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48d4085030 con 0x55e2f1108eb0 2024-04-25T10:09:04.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48c3e00640 1 -- 172.21.15.119:0/201921889 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48d40574a0 con 0x55e2f1108eb0 2024-04-25T10:09:04.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48ecbf2740 10 client.5622 fetch_fsmap learned FSMap version 25 2024-04-25T10:09:04.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48ecbf2740 1 -- 172.21.15.119:0/201921889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55e2f1127420 con 0x55e2f1108eb0 2024-04-25T10:09:04.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48c3e00640 1 -- 172.21.15.119:0/201921889 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d4085030 con 0x55e2f1108eb0 2024-04-25T10:09:04.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48c3e00640 1 -- 172.21.15.119:0/201921889 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f48d406bce0 con 0x55e2f1108eb0 2024-04-25T10:09:04.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48ecbf2740 10 client.5622 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:09:04.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48ecbf2740 10 client.5622 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:04.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48ecbf2740 20 client.5622 populate_metadata read hostname 'smithi119' 2024-04-25T10:09:04.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/201921889 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55e2f1134dd0 0x55e2f11351b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48ecbf2740 4 client.5622 mds_command: new command op to 5478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.240+0000 7f48ecbf2740 1 -- 172.21.15.119:0/201921889 --> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e2f10dd1a0 con 0x55e2f1134dd0 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.241+0000 7f48e2000640 1 --2- 172.21.15.119:0/201921889 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55e2f1134dd0 0x55e2f11351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.241+0000 7f48e2000640 1 --2- 172.21.15.119:0/201921889 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55e2f1134dd0 0x55e2f11351b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b516964867715166 server_cookie=ea0c526ff5c2e696 in_seq=0 out_seq=0 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.241+0000 7f48c3e00640 10 client.5622 ms_handle_connect on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.241+0000 7f48c3e00640 1 -- 172.21.15.119:0/201921889 <== mds.0 v2:172.21.15.119:6835/270766821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55e2f10dd1a0 con 0x55e2f1134dd0 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.241+0000 7f48c3e00640 10 client.5622 handle_command_reply: tid=0 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.241+0000 7f48ecbf2740 1 client.5622 shutdown 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.241+0000 7f48ecbf2740 1 -- 172.21.15.119:0/201921889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 msgr2=0x55e2f11912e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.241+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/201921889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f11912e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f48d4055fd0 tx=0x7f48d4054040 comp rx=0 tx=0).stop 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.242+0000 7f48ecbf2740 1 -- 172.21.15.119:0/201921889 shutdown_connections 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.242+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/201921889 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55e2f1134dd0 0x55e2f11351b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.242+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/201921889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e2f1129e20 0x55e2f1106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.242+0000 7f48c3e00640 0 client.5622 ms_handle_reset on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.242+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/201921889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e2f11297a0 0x55e2f11061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.242+0000 7f48ecbf2740 1 --2- 172.21.15.119:0/201921889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2f1108eb0 0x55e2f11912e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.242+0000 7f48ecbf2740 1 -- 172.21.15.119:0/201921889 >> 172.21.15.119:0/201921889 conn(0x55e2f10e0b30 msgr2=0x55e2f11360c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.242+0000 7f48ecbf2740 1 -- 172.21.15.119:0/201921889 shutdown_connections 2024-04-25T10:09:04.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.242+0000 7f48ecbf2740 1 -- 172.21.15.119:0/201921889 wait complete. 2024-04-25T10:09:04.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.242+0000 7f48ecbf2740 20 client.5622 trim_cache size 0 max 16384 2024-04-25T10:09:04.252 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:09:04.252 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "e0b0f428-b441-4237-a4d1-ed53321f87a5", 2024-04-25T10:09:04.252 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-04-25T10:09:04.252 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 5478, 2024-04-25T10:09:04.252 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-04-25T10:09:04.252 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-04-25T10:09:04.253 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-04-25T10:09:04.253 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.5801408319999997, 2024-04-25T10:09:04.253 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 25, 2024-04-25T10:09:04.253 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 44, 2024-04-25T10:09:04.253 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 44, 2024-04-25T10:09:04.253 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 12.05645097 2024-04-25T10:09:04.253 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:09:04.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.242+0000 7f48e3e00640 1 -- 172.21.15.119:0/1786532765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f48c406b500 msgr2=0x7f48c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.242+0000 7f48e3e00640 1 --2- 172.21.15.119:0/1786532765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f48c406b500 0x7f48c406d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f48e41474c0 tx=0x7f48d0008040 comp rx=0 tx=0).stop 2024-04-25T10:09:04.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.242+0000 7f48e3e00640 1 -- 172.21.15.119:0/1786532765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48e40b3500 msgr2=0x7f48e41f8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.242+0000 7f48e3e00640 1 --2- 172.21.15.119:0/1786532765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48e40b3500 0x7f48e41f8fd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f48d400c400 tx=0x7f48d400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.243+0000 7f48e3e00640 1 -- 172.21.15.119:0/1786532765 shutdown_connections 2024-04-25T10:09:04.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.243+0000 7f48e3e00640 1 --2- 172.21.15.119:0/1786532765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f48c406b500 0x7f48c406d9c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.243+0000 7f48e3e00640 1 --2- 172.21.15.119:0/1786532765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48e4161d60 0x7f48e415e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.243+0000 7f48e3e00640 1 --2- 172.21.15.119:0/1786532765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48e40cea30 0x7f48e4164b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.243+0000 7f48e3e00640 1 --2- 172.21.15.119:0/1786532765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48e40b3500 0x7f48e41f8fd0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.243+0000 7f48e3e00640 1 -- 172.21.15.119:0/1786532765 >> 172.21.15.119:0/1786532765 conn(0x7f48e4066ff0 msgr2=0x7f48e4160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.243+0000 7f48e3e00640 1 -- 172.21.15.119:0/1786532765 shutdown_connections 2024-04-25T10:09:04.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.243+0000 7f48e3e00640 1 -- 172.21.15.119:0/1786532765 wait complete. 2024-04-25T10:09:04.266 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:09:04.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.437+0000 7f56f5400640 1 Processor -- start 2024-04-25T10:09:04.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.437+0000 7f56f5400640 1 -- start start 2024-04-25T10:09:04.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.438+0000 7f56f5400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f56f0144040 0x7f56f00cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.438+0000 7f56f5400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56f015acd0 0x7f56f006e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.439+0000 7f56f5400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f0144420 con 0x7f56f00cce40 2024-04-25T10:09:04.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.439+0000 7f56f5400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f00b36d0 con 0x7f56f0144040 2024-04-25T10:09:04.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.439+0000 7f56f5400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00b3d80 con 0x7f56f015acd0 2024-04-25T10:09:04.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.439+0000 7f56efe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f56f0144040 0x7f56f00cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.439+0000 7f56efe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f56f0144040 0x7f56f00cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38374/0 (socket says 172.21.15.119:38374) 2024-04-25T10:09:04.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.439+0000 7f56f4a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f56f00cce40 0x7f56f015e2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42114/0 (socket says 172.21.15.119:42114) 2024-04-25T10:09:04.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.439+0000 7f56efe00640 1 -- 172.21.15.119:0/3181158156 learned_addr learned my addr 172.21.15.119:0/3181158156 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:04.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.439+0000 7f56ef400640 1 --2- 172.21.15.119:0/3181158156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56f015acd0 0x7f56f006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.439+0000 7f56efe00640 1 -- 172.21.15.119:0/3181158156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56f015acd0 msgr2=0x7f56f006e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.439+0000 7f56efe00640 1 --2- 172.21.15.119:0/3181158156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56f015acd0 0x7f56f006e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.439+0000 7f56efe00640 1 -- 172.21.15.119:0/3181158156 >> v1:172.21.15.119:6789/0 conn(0x7f56f00cce40 legacy=0x7f56f015e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.439+0000 7f56efe00640 1 -- 172.21.15.119:0/3181158156 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f00b4060 con 0x7f56f0144040 2024-04-25T10:09:04.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.440+0000 7f56efe00640 1 --2- 172.21.15.119:0/3181158156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f0144040 0x7f56f00cc900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009870 tx=0x7f56e002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b7ee0af9fbd0c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.441+0000 7f56eea00640 1 -- 172.21.15.119:0/3181158156 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0037070 con 0x7f56f0144040 2024-04-25T10:09:04.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.441+0000 7f56eea00640 1 -- 172.21.15.119:0/3181158156 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e0032440 con 0x7f56f0144040 2024-04-25T10:09:04.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.441+0000 7f56eea00640 1 -- 172.21.15.119:0/3181158156 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0032810 con 0x7f56f0144040 2024-04-25T10:09:04.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.441+0000 7f56f5400640 1 -- 172.21.15.119:0/3181158156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f0144040 msgr2=0x7f56f00cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.441+0000 7f56f5400640 1 --2- 172.21.15.119:0/3181158156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f0144040 0x7f56f00cc900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009870 tx=0x7f56e002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.442+0000 7f56f5400640 1 -- 172.21.15.119:0/3181158156 shutdown_connections 2024-04-25T10:09:04.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.442+0000 7f56f5400640 1 --2- 172.21.15.119:0/3181158156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56f015acd0 0x7f56f006e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.442+0000 7f56f5400640 1 --2- 172.21.15.119:0/3181158156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f0144040 0x7f56f00cc900 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.442+0000 7f56f5400640 1 -- 172.21.15.119:0/3181158156 >> 172.21.15.119:0/3181158156 conn(0x7f56f006b1d0 msgr2=0x7f56f00b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.442+0000 7f56f5400640 1 -- 172.21.15.119:0/3181158156 shutdown_connections 2024-04-25T10:09:04.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.442+0000 7f56f5400640 1 -- 172.21.15.119:0/3181158156 wait complete. 2024-04-25T10:09:04.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.443+0000 7f56f5400640 1 Processor -- start 2024-04-25T10:09:04.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.444+0000 7f56f5400640 1 -- start start 2024-04-25T10:09:04.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.444+0000 7f56f5400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56f00cce40 0x7f56f0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.444+0000 7f56f5400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56f0144040 0x7f56f013b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56f5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f015acd0 0x7f56f015d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56f5400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00b4900 con 0x7f56f015acd0 2024-04-25T10:09:04.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56f5400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00b4620 con 0x7f56f00cce40 2024-04-25T10:09:04.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56f5400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00b5180 con 0x7f56f0144040 2024-04-25T10:09:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56f4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f015acd0 0x7f56f015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56f4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f015acd0 0x7f56f015d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38376/0 (socket says 172.21.15.119:38376) 2024-04-25T10:09:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56f4a00640 1 -- 172.21.15.119:0/179654064 learned_addr learned my addr 172.21.15.119:0/179654064 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56ef400640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56f0144040 0x7f56f013b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56efe00640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56f00cce40 0x7f56f0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56f4a00640 1 -- 172.21.15.119:0/179654064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56f0144040 msgr2=0x7f56f013b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56f4a00640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56f0144040 0x7f56f013b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56f4a00640 1 -- 172.21.15.119:0/179654064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56f00cce40 msgr2=0x7f56f0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56f4a00640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56f00cce40 0x7f56f0155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56f4a00640 1 -- 172.21.15.119:0/179654064 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f013c4d0 con 0x7f56f015acd0 2024-04-25T10:09:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.445+0000 7f56efe00640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56f00cce40 0x7f56f0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.446+0000 7f56f4a00640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f015acd0 0x7f56f015d280 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c7b0 tx=0x7f56e800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.446+0000 7f56e7e00640 1 -- 172.21.15.119:0/179654064 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8017070 con 0x7f56f015acd0 2024-04-25T10:09:04.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.446+0000 7f56e7e00640 1 -- 172.21.15.119:0/179654064 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e800a640 con 0x7f56f015acd0 2024-04-25T10:09:04.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.446+0000 7f56f5400640 1 -- 172.21.15.119:0/179654064 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f013de50 con 0x7f56f015acd0 2024-04-25T10:09:04.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.446+0000 7f56e7e00640 1 -- 172.21.15.119:0/179654064 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e800a9f0 con 0x7f56f015acd0 2024-04-25T10:09:04.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.446+0000 7f56f5400640 1 -- 172.21.15.119:0/179654064 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f013c770 con 0x7f56f015acd0 2024-04-25T10:09:04.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.447+0000 7f56f5400640 1 -- 172.21.15.119:0/179654064 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f006bb30 con 0x7f56f015acd0 2024-04-25T10:09:04.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.448+0000 7f56e7e00640 1 -- 172.21.15.119:0/179654064 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56e801a070 con 0x7f56f015acd0 2024-04-25T10:09:04.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.449+0000 7f56e7e00640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.449+0000 7f56e7e00640 1 -- 172.21.15.119:0/179654064 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56e8089740 con 0x7f56f015acd0 2024-04-25T10:09:04.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.450+0000 7f56efe00640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.450+0000 7f56efe00640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f56cc06b4d0 0x7f56cc06d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f56e0034000 tx=0x7f56e002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.454+0000 7f56e7e00640 1 -- 172.21.15.119:0/179654064 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56e8057f10 con 0x7f56f015acd0 2024-04-25T10:09:04.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.689+0000 7f56f5400640 1 -- 172.21.15.119:0/179654064 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f56f0139c80 con 0x7f56f015acd0 2024-04-25T10:09:04.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.690+0000 7f56e7e00640 1 -- 172.21.15.119:0/179654064 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f56e805bfb0 con 0x7f56f015acd0 2024-04-25T10:09:04.696 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.693+0000 7f56f5400640 1 -- 172.21.15.119:0/179654064 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f56cc06b4d0 msgr2=0x7f56cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.693+0000 7f56f5400640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f56cc06b4d0 0x7f56cc06d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f56e0034000 tx=0x7f56e002e000 comp rx=0 tx=0).stop 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.693+0000 7f56f5400640 1 -- 172.21.15.119:0/179654064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f015acd0 msgr2=0x7f56f015d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.693+0000 7f56f5400640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f015acd0 0x7f56f015d280 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c7b0 tx=0x7f56e800cc70 comp rx=0 tx=0).stop 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.694+0000 7f56f5400640 1 -- 172.21.15.119:0/179654064 shutdown_connections 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.694+0000 7f56f5400640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.694+0000 7f56f5400640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56f015acd0 0x7f56f015d280 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.694+0000 7f56f5400640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56f0144040 0x7f56f013b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.694+0000 7f56f5400640 1 --2- 172.21.15.119:0/179654064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56f00cce40 0x7f56f0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.694+0000 7f56f5400640 1 -- 172.21.15.119:0/179654064 >> 172.21.15.119:0/179654064 conn(0x7f56f006b1d0 msgr2=0x7f56f013a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.694+0000 7f56f5400640 1 -- 172.21.15.119:0/179654064 shutdown_connections 2024-04-25T10:09:04.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:04.694+0000 7f56f5400640 1 -- 172.21.15.119:0/179654064 wait complete. 2024-04-25T10:09:04.723 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:09:04.723 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:09:04.723 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:09:04.724 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:09:04.724 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:04.724 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:09:04.724 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:09:04.724 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:09:04.724 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:04.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:04.725 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:09:04.774 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:09:04.775 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:09:04.775 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:09:04.775 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:04.775 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:09:04.775 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:04.775 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:04.775 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:04.775 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:09:04.775 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:04.776 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:04.776 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:09:04.776 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:04.776 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:04.776 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:09:04.776 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:04.776 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:09:04.776 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42545sec preferred_lft 42545sec 2024-04-25T10:09:04.776 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:09:04.776 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:04.776 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:09:04.776 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:04.776 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:09:04.776 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:04.777 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:09:04.777 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:04.777 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:09:04.777 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:04.777 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:09:04.777 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:04.777 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:09:04.777 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:04.777 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:09:04.777 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:04.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:04.778 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:09:04.847 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:09:04.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:04.850 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:09:04.896 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:09:04.897 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:09:04.897 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:09:04.897 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:09:04.897 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:09:04.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:04.898 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:09:04.898 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:09:04.898 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:09:04.898 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:09:04.898 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:09:04.898 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:09:05.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:05.075 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:09:05.124 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:09:05.124 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:09:05.124 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:05.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:05.125 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:09:05.210 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:09:05.210 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:05.263 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:05.263 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:09:05.264 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:05.316 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:09:05.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:05.316 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:05.444 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:05.444 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:05.444 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:09:05.465 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:05.465 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:05.465 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:09:05.465 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:09:05.465 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:09:05.492 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:05.492 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:05.492 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:09:05.493 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:09:05.493 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:09:05.493 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:09:05.493 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:09:05.493 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:09:05.495 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:09:05.495 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:09:05.495 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:09:05.495 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:09:05.495 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:09:05.500 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:09:05.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:05.500 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:05.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:05.572 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:09:05.641 INFO:teuthology.orchestra.run.smithi119.stdout:client4984 2024-04-25T10:09:05.641 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:09:05.641 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:05.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:05.696 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-25T10:09:05.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:05.751 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-25T10:09:05.799 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:09:05.799 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:09:05.799 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:09:05.799 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:09:05.799 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-25T10:09:05.800 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:09:05.921 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627777 2024-04-25T10:09:05.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:09:05.934 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:06.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.172+0000 7fcc06400640 1 Processor -- start 2024-04-25T10:09:06.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.172+0000 7fcc06400640 1 -- start start 2024-04-25T10:09:06.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.174+0000 7fcc06400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc000ae9a0 0x7fcc000b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.174+0000 7fcc06400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0006d550 con 0x7fcc000b2c60 2024-04-25T10:09:06.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.174+0000 7fcc06400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0006d830 con 0x7fcc00071190 2024-04-25T10:09:06.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.174+0000 7fcc06400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0006db10 con 0x7fcc000ae9a0 2024-04-25T10:09:06.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.174+0000 7fcbffe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcc000b2c60 0x7fcc000b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42128/0 (socket says 172.21.15.119:42128) 2024-04-25T10:09:06.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.174+0000 7fcbffe00640 1 -- 172.21.15.119:0/3711660045 learned_addr learned my addr 172.21.15.119:0/3711660045 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:06.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.174+0000 7fcbfe000640 1 -- 172.21.15.119:0/3711660045 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265660018 0 0) 0x7fcc0006d550 con 0x7fcc000b2c60 2024-04-25T10:09:06.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.174+0000 7fcbfe000640 1 -- 172.21.15.119:0/3711660045 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe4003440 con 0x7fcc000b2c60 2024-04-25T10:09:06.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbfea00640 1 --2- 172.21.15.119:0/3711660045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc000ae9a0 0x7fcc000b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbfe000640 1 -- 172.21.15.119:0/3711660045 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1071917776 0 0) 0x7fcbe4003440 con 0x7fcc000b2c60 2024-04-25T10:09:06.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbfe000640 1 -- 172.21.15.119:0/3711660045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc000ae9a0 msgr2=0x7fcc000b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbfe000640 1 --2- 172.21.15.119:0/3711660045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc000ae9a0 0x7fcc000b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbfe000640 1 -- 172.21.15.119:0/3711660045 >> v1:172.21.15.132:6789/0 conn(0x7fcc00071190 legacy=0x7fcc00071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbfe000640 1 -- 172.21.15.119:0/3711660045 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0006ddf0 con 0x7fcc000b2c60 2024-04-25T10:09:06.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbfe000640 1 -- 172.21.15.119:0/3711660045 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fcbf8001670 con 0x7fcc000b2c60 2024-04-25T10:09:06.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbfe000640 1 -- 172.21.15.119:0/3711660045 >> v1:172.21.15.119:6789/0 conn(0x7fcc000b2c60 legacy=0x7fcc000b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbfe000640 1 --2- 172.21.15.119:0/3711660045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbe4003ad0 0x7fcbe4003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbfe000640 1 --2- 172.21.15.119:0/3711660045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc000ae9a0 0x7fcbe400a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbfe000640 1 --2- 172.21.15.119:0/3711660045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe400b090 0x7fcbe400d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbfe000640 1 -- 172.21.15.119:0/3711660045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0006db10 con 0x7fcbe400b090 2024-04-25T10:09:06.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbfe000640 1 -- 172.21.15.119:0/3711660045 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe400dd30 con 0x7fcbe4003ad0 2024-04-25T10:09:06.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbfe000640 1 -- 172.21.15.119:0/3711660045 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe400df30 con 0x7fcc000ae9a0 2024-04-25T10:09:06.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.175+0000 7fcbffe00640 1 --2- 172.21.15.119:0/3711660045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe400b090 0x7fcbe400d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.176+0000 7fcbff400640 1 --2- 172.21.15.119:0/3711660045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbe4003ad0 0x7fcbe4003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.176+0000 7fcbfea00640 1 --2- 172.21.15.119:0/3711660045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc000ae9a0 0x7fcbe400a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.176+0000 7fcbffe00640 1 -- 172.21.15.119:0/3711660045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc000ae9a0 msgr2=0x7fcbe400a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.176+0000 7fcbffe00640 1 --2- 172.21.15.119:0/3711660045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc000ae9a0 0x7fcbe400a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.176+0000 7fcbffe00640 1 -- 172.21.15.119:0/3711660045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbe4003ad0 msgr2=0x7fcbe4003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.176+0000 7fcbffe00640 1 --2- 172.21.15.119:0/3711660045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbe4003ad0 0x7fcbe4003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.176+0000 7fcbffe00640 1 -- 172.21.15.119:0/3711660045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe4003440 con 0x7fcbe400b090 2024-04-25T10:09:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.177+0000 7fcbffe00640 1 --2- 172.21.15.119:0/3711660045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe400b090 0x7fcbe400d520 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800c240 tx=0x7fcbe4003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836f2d07622feb62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.177+0000 7fcbfe000640 1 -- 172.21.15.119:0/3711660045 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8007d10 con 0x7fcbe400b090 2024-04-25T10:09:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.177+0000 7fcbfe000640 1 -- 172.21.15.119:0/3711660045 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf8005ad0 con 0x7fcbe400b090 2024-04-25T10:09:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.177+0000 7fcbfe000640 1 -- 172.21.15.119:0/3711660045 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8007070 con 0x7fcbe400b090 2024-04-25T10:09:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.178+0000 7fcc06400640 1 -- 172.21.15.119:0/3711660045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe400b090 msgr2=0x7fcbe400d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.178+0000 7fcc06400640 1 --2- 172.21.15.119:0/3711660045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe400b090 0x7fcbe400d520 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800c240 tx=0x7fcbe4003730 comp rx=0 tx=0).stop 2024-04-25T10:09:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.178+0000 7fcbffe00640 1 -- 172.21.15.119:0/3711660045 reap_dead start 2024-04-25T10:09:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.178+0000 7fcc06400640 1 -- 172.21.15.119:0/3711660045 shutdown_connections 2024-04-25T10:09:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.178+0000 7fcc06400640 1 -- 172.21.15.119:0/3711660045 >> 172.21.15.119:0/3711660045 conn(0x7fcc0006aba0 msgr2=0x7fcc0006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.178+0000 7fcc06400640 1 -- 172.21.15.119:0/3711660045 shutdown_connections 2024-04-25T10:09:06.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.178+0000 7fcc06400640 1 -- 172.21.15.119:0/3711660045 wait complete. 2024-04-25T10:09:06.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.180+0000 7fcc06400640 1 Processor -- start 2024-04-25T10:09:06.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.180+0000 7fcc06400640 1 -- start start 2024-04-25T10:09:06.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.181+0000 7fcc06400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe400b090 0x7fcc00142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.181+0000 7fcc06400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc00143590 0x7fcc0013f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.181+0000 7fcc06400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc000cdd70 0x7fcc000ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.181+0000 7fcc06400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0006e8b0 con 0x7fcbe400b090 2024-04-25T10:09:06.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.181+0000 7fcc06400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc000aee00 con 0x7fcc00143590 2024-04-25T10:09:06.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.181+0000 7fcc06400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0006d1b0 con 0x7fcc000cdd70 2024-04-25T10:09:06.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.181+0000 7fcbff400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe400b090 0x7fcc00142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.181+0000 7fcbff400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe400b090 0x7fcc00142af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38396/0 (socket says 172.21.15.119:38396) 2024-04-25T10:09:06.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.181+0000 7fcbff400640 1 -- 172.21.15.119:0/2487291243 learned_addr learned my addr 172.21.15.119:0/2487291243 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:06.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.181+0000 7fcbff400640 1 -- 172.21.15.119:0/2487291243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc000cdd70 msgr2=0x7fcc000ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.181+0000 7fcbff400640 1 --2- 172.21.15.119:0/2487291243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc000cdd70 0x7fcc000ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.181+0000 7fcbff400640 1 -- 172.21.15.119:0/2487291243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc00143590 msgr2=0x7fcc0013f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.181+0000 7fcbff400640 1 --2- 172.21.15.119:0/2487291243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc00143590 0x7fcc0013f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.181+0000 7fcbff400640 1 -- 172.21.15.119:0/2487291243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc001400a0 con 0x7fcbe400b090 2024-04-25T10:09:06.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.182+0000 7fcbff400640 1 --2- 172.21.15.119:0/2487291243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe400b090 0x7fcc00142af0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000e8c0 tx=0x7fcbf000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.182+0000 7fcbf7400640 1 -- 172.21.15.119:0/2487291243 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0003b80 con 0x7fcbe400b090 2024-04-25T10:09:06.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.182+0000 7fcbf7400640 1 -- 172.21.15.119:0/2487291243 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf0018070 con 0x7fcbe400b090 2024-04-25T10:09:06.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.182+0000 7fcbf7400640 1 -- 172.21.15.119:0/2487291243 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00050d0 con 0x7fcbe400b090 2024-04-25T10:09:06.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.182+0000 7fcc06400640 1 -- 172.21.15.119:0/2487291243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc00143030 con 0x7fcbe400b090 2024-04-25T10:09:06.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.182+0000 7fcc06400640 1 -- 172.21.15.119:0/2487291243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc00140620 con 0x7fcbe400b090 2024-04-25T10:09:06.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.183+0000 7fcc06400640 1 -- 172.21.15.119:0/2487291243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc000ce730 con 0x7fcbe400b090 2024-04-25T10:09:06.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.184+0000 7fcbf7400640 1 -- 172.21.15.119:0/2487291243 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbf0005270 con 0x7fcbe400b090 2024-04-25T10:09:06.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.185+0000 7fcbf7400640 1 --2- 172.21.15.119:0/2487291243 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcbe006b550 0x7fcbe006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.185+0000 7fcbf7400640 1 -- 172.21.15.119:0/2487291243 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbf008a1d0 con 0x7fcbe400b090 2024-04-25T10:09:06.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.185+0000 7fcbfea00640 1 --2- 172.21.15.119:0/2487291243 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcbe006b550 0x7fcbe006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.186+0000 7fcbfea00640 1 --2- 172.21.15.119:0/2487291243 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcbe006b550 0x7fcbe006da10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcc000cbea0 tx=0x7fcbec00b2e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.189+0000 7fcbf7400640 1 -- 172.21.15.119:0/2487291243 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcbf001a310 con 0x7fcbe400b090 2024-04-25T10:09:06.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.422+0000 7fcc06400640 1 -- 172.21.15.119:0/2487291243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc0006e470 con 0x7fcbe400b090 2024-04-25T10:09:06.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.423+0000 7fcbf7400640 1 -- 172.21.15.119:0/2487291243 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fcbf00590c0 con 0x7fcbe400b090 2024-04-25T10:09:06.429 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:06.429 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3317744182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3317744182},{"type":"v1","addr":"172.21.15.132:6833","nonce":3317744182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/700427287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":700427287},{"type":"v1","addr":"172.21.15.132:6835","nonce":700427287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2207174972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2207174972},{"type":"v1","addr":"172.21.15.119:6836","nonce":2207174972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:57.120451+0000","modified":"2024-04-25T10:09:00.157396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5478":{"gid":5478,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/270766821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":270766821},{"type":"v1","addr":"172.21.15.119:6837","nonce":270766821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:09:06.429 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 25 2024-04-25T10:09:06.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.425+0000 7fcc06400640 1 -- 172.21.15.119:0/2487291243 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcbe006b550 msgr2=0x7fcbe006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.425+0000 7fcc06400640 1 --2- 172.21.15.119:0/2487291243 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcbe006b550 0x7fcbe006da10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcc000cbea0 tx=0x7fcbec00b2e0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.425+0000 7fcc06400640 1 -- 172.21.15.119:0/2487291243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe400b090 msgr2=0x7fcc00142af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.425+0000 7fcc06400640 1 --2- 172.21.15.119:0/2487291243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe400b090 0x7fcc00142af0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000e8c0 tx=0x7fcbf000ed80 comp rx=0 tx=0).stop 2024-04-25T10:09:06.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.426+0000 7fcc06400640 1 -- 172.21.15.119:0/2487291243 shutdown_connections 2024-04-25T10:09:06.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.426+0000 7fcc06400640 1 --2- 172.21.15.119:0/2487291243 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcbe006b550 0x7fcbe006da10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.426+0000 7fcc06400640 1 --2- 172.21.15.119:0/2487291243 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc000cdd70 0x7fcc000ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.426+0000 7fcc06400640 1 --2- 172.21.15.119:0/2487291243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc00143590 0x7fcc0013f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.426+0000 7fcc06400640 1 --2- 172.21.15.119:0/2487291243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbe400b090 0x7fcc00142af0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.426+0000 7fcc06400640 1 -- 172.21.15.119:0/2487291243 >> 172.21.15.119:0/2487291243 conn(0x7fcc0006aba0 msgr2=0x7fcc000afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.426+0000 7fcc06400640 1 -- 172.21.15.119:0/2487291243 shutdown_connections 2024-04-25T10:09:06.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.426+0000 7fcc06400640 1 -- 172.21.15.119:0/2487291243 wait complete. 2024-04-25T10:09:06.451 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:09:06.689 INFO:teuthology.orchestra.run.smithi119.stdout:{"message":"","return_code":0} 2024-04-25T10:09:06.690 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:09:06.690 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:09:06.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.787+0000 7f41480e7f40 1 Processor -- start 2024-04-25T10:09:06.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.787+0000 7f41480e7f40 1 -- start start 2024-04-25T10:09:06.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.788+0000 7f41480e7f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56424b539220 0x56424b6049b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.788+0000 7f41480e7f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56424b584080 0x56424b586310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.788+0000 7f41480e7f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56424b610c70 0x56424b586850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.788+0000 7f41480e7f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424b56c2f0 con 0x56424b610c70 2024-04-25T10:09:06.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.788+0000 7f41480e7f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424b4e0dd0 con 0x56424b539220 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.788+0000 7f41480e7f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56424b539960 con 0x56424b584080 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.788+0000 7f4145c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56424b610c70 0x56424b586850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.788+0000 7f4145c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56424b610c70 0x56424b586850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38416/0 (socket says 172.21.15.119:38416) 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.788+0000 7f4145c00640 1 -- 172.21.15.119:0/4103372214 learned_addr learned my addr 172.21.15.119:0/4103372214 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.789+0000 7f4145200640 1 --2- 172.21.15.119:0/4103372214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56424b539220 0x56424b6049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.789+0000 7f4145c00640 1 -- 172.21.15.119:0/4103372214 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56424b584080 msgr2=0x56424b586310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.789+0000 7f4145c00640 1 --2- 172.21.15.119:0/4103372214 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56424b584080 0x56424b586310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.789+0000 7f4145c00640 1 -- 172.21.15.119:0/4103372214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56424b539220 msgr2=0x56424b6049b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.789+0000 7f4145c00640 1 --2- 172.21.15.119:0/4103372214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56424b539220 0x56424b6049b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.789+0000 7f4145c00640 1 -- 172.21.15.119:0/4103372214 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56424b605630 con 0x56424b610c70 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.789+0000 7f4145c00640 1 --2- 172.21.15.119:0/4103372214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56424b610c70 0x56424b586850 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f413c006fb0 tx=0x7f413c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe3a7d956b385cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.789+0000 7f4143400640 1 -- 172.21.15.119:0/4103372214 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c004520 con 0x56424b610c70 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.789+0000 7f4143400640 1 -- 172.21.15.119:0/4103372214 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413c0046e0 con 0x56424b610c70 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.789+0000 7f4143400640 1 -- 172.21.15.119:0/4103372214 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c004ad0 con 0x56424b610c70 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.790+0000 7f41480e7f40 1 -- 172.21.15.119:0/4103372214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56424b610c70 msgr2=0x56424b586850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.790+0000 7f41480e7f40 1 --2- 172.21.15.119:0/4103372214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56424b610c70 0x56424b586850 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f413c006fb0 tx=0x7f413c001ae0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.790+0000 7f41480e7f40 1 -- 172.21.15.119:0/4103372214 shutdown_connections 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.790+0000 7f41480e7f40 1 --2- 172.21.15.119:0/4103372214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56424b610c70 0x56424b586850 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.790+0000 7f41480e7f40 1 --2- 172.21.15.119:0/4103372214 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56424b584080 0x56424b586310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.790+0000 7f41480e7f40 1 --2- 172.21.15.119:0/4103372214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56424b539220 0x56424b6049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.790+0000 7f41480e7f40 1 -- 172.21.15.119:0/4103372214 >> 172.21.15.119:0/4103372214 conn(0x56424b587070 msgr2=0x56424b587450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.790+0000 7f41480e7f40 1 -- 172.21.15.119:0/4103372214 shutdown_connections 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.790+0000 7f41480e7f40 1 -- 172.21.15.119:0/4103372214 wait complete. 2024-04-25T10:09:06.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.791+0000 7f41480e7f40 1 Processor -- start 2024-04-25T10:09:06.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.791+0000 7f41480e7f40 1 -- start start 2024-04-25T10:09:06.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.792+0000 7f41480e7f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56424b539220 0x56424b5f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.792+0000 7f41480e7f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56424b584080 0x56424b5f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.792+0000 7f41480e7f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56424b610c70 0x56424b538500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.792+0000 7f41480e7f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424b611b20 con 0x56424b584080 2024-04-25T10:09:06.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.792+0000 7f41480e7f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424b6118d0 con 0x56424b539220 2024-04-25T10:09:06.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.792+0000 7f41480e7f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56424b611e00 con 0x56424b610c70 2024-04-25T10:09:06.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.792+0000 7f4144800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56424b584080 0x56424b5f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.792+0000 7f4144800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56424b584080 0x56424b5f5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38432/0 (socket says 172.21.15.119:38432) 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.792+0000 7f4144800640 1 -- 172.21.15.119:0/1623350928 learned_addr learned my addr 172.21.15.119:0/1623350928 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.792+0000 7f4145200640 1 --2- 172.21.15.119:0/1623350928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56424b539220 0x56424b5f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.792+0000 7f4144800640 1 -- 172.21.15.119:0/1623350928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56424b610c70 msgr2=0x56424b538500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.792+0000 7f4144800640 1 --2- 172.21.15.119:0/1623350928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56424b610c70 0x56424b538500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.792+0000 7f4144800640 1 -- 172.21.15.119:0/1623350928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56424b539220 msgr2=0x56424b5f1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.792+0000 7f4144800640 1 --2- 172.21.15.119:0/1623350928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56424b539220 0x56424b5f1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.793+0000 7f4144800640 1 -- 172.21.15.119:0/1623350928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56424b583c30 con 0x56424b584080 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.793+0000 7f4144800640 1 --2- 172.21.15.119:0/1623350928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56424b584080 0x56424b5f5b20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4130008620 tx=0x7f4130008710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.793+0000 7f4142000640 1 -- 172.21.15.119:0/1623350928 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130012790 con 0x56424b584080 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.793+0000 7f4142000640 1 -- 172.21.15.119:0/1623350928 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4130018070 con 0x56424b584080 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.793+0000 7f4142000640 1 -- 172.21.15.119:0/1623350928 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130012b90 con 0x56424b584080 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.793+0000 7f41480e7f40 1 -- 172.21.15.119:0/1623350928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56424b566fa0 con 0x56424b584080 2024-04-25T10:09:06.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.793+0000 7f41480e7f40 1 -- 172.21.15.119:0/1623350928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56424b567230 con 0x56424b584080 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.795+0000 7f4142000640 1 -- 172.21.15.119:0/1623350928 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f413001d460 con 0x56424b584080 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.796+0000 7f4142000640 1 --2- 172.21.15.119:0/1623350928 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4124069ee0 0x7f412406c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.796+0000 7f4142000640 1 -- 172.21.15.119:0/1623350928 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41300890d0 con 0x56424b584080 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.796+0000 7f4145200640 1 --2- 172.21.15.119:0/1623350928 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4124069ee0 0x7f412406c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.796+0000 7f41480e7f40 1 --2- 172.21.15.119:0/1623350928 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x56424b6015f0 0x56424b6049b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.796+0000 7f41480e7f40 1 -- 172.21.15.119:0/1623350928 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v9 -- 0x56424b604ef0 con 0x56424b6015f0 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.797+0000 7f4145200640 1 --2- 172.21.15.119:0/1623350928 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4124069ee0 0x7f412406c3a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4134056000 tx=0x7f4134050000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.797+0000 7f4145c00640 1 --2- 172.21.15.119:0/1623350928 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x56424b6015f0 0x56424b6049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.798+0000 7f4145c00640 1 --2- 172.21.15.119:0/1623350928 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x56424b6015f0 0x56424b6049b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.799+0000 7f4145c00640 1 -- 172.21.15.119:0/1623350928 <== osd.6 v2:172.21.15.132:6806/353653608 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f413c00f030 con 0x56424b6015f0 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.799+0000 7f41480e7f40 1 -- 172.21.15.119:0/1623350928 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x56424b6015f0 msgr2=0x56424b6049b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.799+0000 7f41480e7f40 1 --2- 172.21.15.119:0/1623350928 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x56424b6015f0 0x56424b6049b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.799+0000 7f41480e7f40 1 -- 172.21.15.119:0/1623350928 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4124069ee0 msgr2=0x7f412406c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.799+0000 7f41480e7f40 1 --2- 172.21.15.119:0/1623350928 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4124069ee0 0x7f412406c3a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4134056000 tx=0x7f4134050000 comp rx=0 tx=0).stop 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.799+0000 7f41480e7f40 1 -- 172.21.15.119:0/1623350928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56424b584080 msgr2=0x56424b5f5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.799+0000 7f41480e7f40 1 --2- 172.21.15.119:0/1623350928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56424b584080 0x56424b5f5b20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4130008620 tx=0x7f4130008710 comp rx=0 tx=0).stop 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.799+0000 7f4145c00640 1 -- 172.21.15.119:0/1623350928 reap_dead start 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.799+0000 7f41480e7f40 1 -- 172.21.15.119:0/1623350928 shutdown_connections 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.799+0000 7f41480e7f40 1 -- 172.21.15.119:0/1623350928 >> 172.21.15.119:0/1623350928 conn(0x56424b587070 msgr2=0x56424b568160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.800+0000 7f41480e7f40 1 -- 172.21.15.119:0/1623350928 shutdown_connections 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:06.800+0000 7f41480e7f40 1 -- 172.21.15.119:0/1623350928 wait complete. 2024-04-25T10:09:06.810 INFO:teuthology.orchestra.run.smithi119.stdout:Yalphaparent_a 2024-04-25T10:09:06.810 DEBUG:teuthology.orchestra.run.smithi119:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:09:06.984 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:09:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 1099511627777, 2024-04-25T10:09:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "ancestors": [ 2024-04-25T10:09:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "dirino": 1099511627776, 2024-04-25T10:09:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "dname": "alpha", 2024-04-25T10:09:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "version": 2 2024-04-25T10:09:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "dirino": 1, 2024-04-25T10:09:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "dname": "parent_a", 2024-04-25T10:09:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: "version": 4 2024-04-25T10:09:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:06.985 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-04-25T10:09:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 7, 2024-04-25T10:09:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: "old_pools": [] 2024-04-25T10:09:06.986 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:09:06.986 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:06.995 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-25T10:09:07.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.103+0000 7f2ee41b7f40 1 Processor -- start 2024-04-25T10:09:07.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.104+0000 7f2ee41b7f40 1 -- start start 2024-04-25T10:09:07.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.104+0000 7f2ee41b7f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56018521a220 0x5601853807f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.104+0000 7f2ee41b7f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560185265080 0x5601852ec220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.104+0000 7f2ee41b7f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601852f1c70 0x5601852e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.104+0000 7f2ee41b7f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018524d2f0 con 0x56018521a220 2024-04-25T10:09:07.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.104+0000 7f2ee41b7f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601851c1dd0 con 0x5601852f1c70 2024-04-25T10:09:07.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.104+0000 7f2ee41b7f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56018521a960 con 0x560185265080 2024-04-25T10:09:07.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.105+0000 7f2ee1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56018521a220 0x5601853807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.105+0000 7f2ee1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56018521a220 0x5601853807f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38464/0 (socket says 172.21.15.119:38464) 2024-04-25T10:09:07.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.105+0000 7f2ee1400640 1 -- 172.21.15.119:0/767261023 learned_addr learned my addr 172.21.15.119:0/767261023 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:07.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.105+0000 7f2ee1400640 1 -- 172.21.15.119:0/767261023 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560185265080 msgr2=0x5601852ec220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.105+0000 7f2ee1400640 1 --2- 172.21.15.119:0/767261023 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560185265080 0x5601852ec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.105+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/767261023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601852f1c70 0x5601852e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.105+0000 7f2ee1400640 1 -- 172.21.15.119:0/767261023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601852f1c70 msgr2=0x5601852e4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.105+0000 7f2ee1400640 1 --2- 172.21.15.119:0/767261023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601852f1c70 0x5601852e4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.105+0000 7f2ee1400640 1 -- 172.21.15.119:0/767261023 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560185248e90 con 0x56018521a220 2024-04-25T10:09:07.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.105+0000 7f2ee1400640 1 --2- 172.21.15.119:0/767261023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56018521a220 0x5601853807f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400a470 tx=0x7f2ed404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dabc66cda3278c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.106+0000 7f2edb400640 1 -- 172.21.15.119:0/767261023 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4062070 con 0x56018521a220 2024-04-25T10:09:07.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.106+0000 7f2edb400640 1 -- 172.21.15.119:0/767261023 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed40549f0 con 0x56018521a220 2024-04-25T10:09:07.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.106+0000 7f2edb400640 1 -- 172.21.15.119:0/767261023 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4059040 con 0x56018521a220 2024-04-25T10:09:07.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.106+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/767261023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56018521a220 msgr2=0x5601853807f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.106+0000 7f2ee41b7f40 1 --2- 172.21.15.119:0/767261023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56018521a220 0x5601853807f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400a470 tx=0x7f2ed404e000 comp rx=0 tx=0).stop 2024-04-25T10:09:07.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.106+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/767261023 shutdown_connections 2024-04-25T10:09:07.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.106+0000 7f2ee41b7f40 1 --2- 172.21.15.119:0/767261023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601852f1c70 0x5601852e4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.106+0000 7f2ee41b7f40 1 --2- 172.21.15.119:0/767261023 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560185265080 0x5601852ec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.106+0000 7f2ee41b7f40 1 --2- 172.21.15.119:0/767261023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56018521a220 0x5601853807f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.106+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/767261023 >> 172.21.15.119:0/767261023 conn(0x560185268070 msgr2=0x560185267450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.106+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/767261023 shutdown_connections 2024-04-25T10:09:07.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.106+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/767261023 wait complete. 2024-04-25T10:09:07.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.107+0000 7f2ee41b7f40 1 Processor -- start 2024-04-25T10:09:07.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.107+0000 7f2ee41b7f40 1 -- start start 2024-04-25T10:09:07.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.108+0000 7f2ee41b7f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018521a220 0x5601852e8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.108+0000 7f2ee41b7f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560185265080 0x5601852ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.108+0000 7f2ee41b7f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852f1c70 0x5601852e73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.108+0000 7f2ee41b7f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560185249760 con 0x5601852f1c70 2024-04-25T10:09:07.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.108+0000 7f2ee41b7f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601852f1420 con 0x56018521a220 2024-04-25T10:09:07.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.108+0000 7f2ee41b7f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601852f0f00 con 0x560185265080 2024-04-25T10:09:07.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.109+0000 7f2ee1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852f1c70 0x5601852e73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.109+0000 7f2ee1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852f1c70 0x5601852e73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38472/0 (socket says 172.21.15.119:38472) 2024-04-25T10:09:07.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.109+0000 7f2ee1e00640 1 -- 172.21.15.119:0/3728032651 learned_addr learned my addr 172.21.15.119:0/3728032651 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:07.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.109+0000 7f2ee1400640 1 --2- 172.21.15.119:0/3728032651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018521a220 0x5601852e8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.109+0000 7f2ee0a00640 1 --2- 172.21.15.119:0/3728032651 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560185265080 0x5601852ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.109+0000 7f2ee1e00640 1 -- 172.21.15.119:0/3728032651 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560185265080 msgr2=0x5601852ef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.109+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/3728032651 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560185265080 0x5601852ef2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.109+0000 7f2ee1e00640 1 -- 172.21.15.119:0/3728032651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018521a220 msgr2=0x5601852e8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.109+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/3728032651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56018521a220 0x5601852e8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.109+0000 7f2ee1e00640 1 -- 172.21.15.119:0/3728032651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601852e89e0 con 0x5601852f1c70 2024-04-25T10:09:07.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.109+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/3728032651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852f1c70 0x5601852e73c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00a470 tx=0x7f2edc00a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.110+0000 7f2eda000640 1 -- 172.21.15.119:0/3728032651 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc015070 con 0x5601852f1c70 2024-04-25T10:09:07.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.110+0000 7f2eda000640 1 -- 172.21.15.119:0/3728032651 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc011870 con 0x5601852f1c70 2024-04-25T10:09:07.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.110+0000 7f2eda000640 1 -- 172.21.15.119:0/3728032651 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc011c60 con 0x5601852f1c70 2024-04-25T10:09:07.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.110+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/3728032651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5601852e40b0 con 0x5601852f1c70 2024-04-25T10:09:07.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.110+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/3728032651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601852e91c0 con 0x5601852f1c70 2024-04-25T10:09:07.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.112+0000 7f2eda000640 1 -- 172.21.15.119:0/3728032651 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2edc018070 con 0x5601852f1c70 2024-04-25T10:09:07.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.113+0000 7f2eda000640 1 --2- 172.21.15.119:0/3728032651 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ec4069eb0 0x7f2ec406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.113+0000 7f2eda000640 1 -- 172.21.15.119:0/3728032651 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2edc088f90 con 0x5601852f1c70 2024-04-25T10:09:07.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.113+0000 7f2ee1400640 1 --2- 172.21.15.119:0/3728032651 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ec4069eb0 0x7f2ec406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.113+0000 7f2ee41b7f40 1 --2- 172.21.15.119:0/3728032651 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5601852ef830 0x5601852e3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.113+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/3728032651 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) v9 -- 0x5601852ebfd0 con 0x5601852ef830 2024-04-25T10:09:07.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.114+0000 7f2ee1400640 1 --2- 172.21.15.119:0/3728032651 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ec4069eb0 0x7f2ec406c370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4002000 tx=0x7f2ed4051000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.114+0000 7f2ee0a00640 1 --2- 172.21.15.119:0/3728032651 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5601852ef830 0x5601852e3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.115+0000 7f2ee0a00640 1 --2- 172.21.15.119:0/3728032651 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5601852ef830 0x5601852e3060 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.118+0000 7f2ee0a00640 1 -- 172.21.15.119:0/3728032651 <== osd.6 v2:172.21.15.132:6806/353653608 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v44'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2ed000f450 con 0x5601852ef830 2024-04-25T10:09:07.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.118+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/3728032651 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5601852ef830 msgr2=0x5601852e3060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.118+0000 7f2ee41b7f40 1 --2- 172.21.15.119:0/3728032651 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5601852ef830 0x5601852e3060 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.118+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/3728032651 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ec4069eb0 msgr2=0x7f2ec406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.118+0000 7f2ee41b7f40 1 --2- 172.21.15.119:0/3728032651 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ec4069eb0 0x7f2ec406c370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4002000 tx=0x7f2ed4051000 comp rx=0 tx=0).stop 2024-04-25T10:09:07.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.118+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/3728032651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852f1c70 msgr2=0x5601852e73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.118+0000 7f2ee41b7f40 1 --2- 172.21.15.119:0/3728032651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5601852f1c70 0x5601852e73c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00a470 tx=0x7f2edc00a870 comp rx=0 tx=0).stop 2024-04-25T10:09:07.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.118+0000 7f2ee1e00640 1 -- 172.21.15.119:0/3728032651 reap_dead start 2024-04-25T10:09:07.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.118+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/3728032651 shutdown_connections 2024-04-25T10:09:07.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.118+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/3728032651 >> 172.21.15.119:0/3728032651 conn(0x560185268070 msgr2=0x5601852e4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.118+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/3728032651 shutdown_connections 2024-04-25T10:09:07.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.118+0000 7f2ee41b7f40 1 -- 172.21.15.119:0/3728032651 wait complete. 2024-04-25T10:09:07.131 DEBUG:teuthology.orchestra.run.smithi119:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:09:07.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:07.138 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-25T10:09:07.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.314+0000 7f4055c00640 1 Processor -- start 2024-04-25T10:09:07.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.314+0000 7f4055c00640 1 -- start start 2024-04-25T10:09:07.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.315+0000 7f4055c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f40500d0ef0 0x7f40500ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.316+0000 7f4055c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40500d15f0 con 0x7f40500cce80 2024-04-25T10:09:07.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.316+0000 7f4055c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40500d1330 con 0x7f40500cb250 2024-04-25T10:09:07.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.316+0000 7f4055c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405015bea0 con 0x7f40500d0ef0 2024-04-25T10:09:07.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.316+0000 7f404ea00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f40500cce80 0x7f40500cd260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42148/0 (socket says 172.21.15.119:42148) 2024-04-25T10:09:07.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.316+0000 7f404e000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f40500d0ef0 0x7f40500ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.316+0000 7f404ea00640 1 -- 172.21.15.119:0/3934559630 learned_addr learned my addr 172.21.15.119:0/3934559630 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:07.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.316+0000 7f404e000640 1 -- 172.21.15.119:0/3934559630 >> v1:172.21.15.132:6789/0 conn(0x7f40500cb250 legacy=0x7f40500cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.316+0000 7f404e000640 1 -- 172.21.15.119:0/3934559630 >> v1:172.21.15.119:6789/0 conn(0x7f40500cce80 legacy=0x7f40500cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.317+0000 7f404d600640 1 -- 172.21.15.119:0/3934559630 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424359995 0 0) 0x7f40500d15f0 con 0x7f40500cce80 2024-04-25T10:09:07.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.317+0000 7f404e000640 1 -- 172.21.15.119:0/3934559630 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405015b0e0 con 0x7f40500d0ef0 2024-04-25T10:09:07.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.318+0000 7f404e000640 1 --2- 172.21.15.119:0/3934559630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40500d0ef0 0x7f40500ce5e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f40380097a0 tx=0x7f403802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f31c7131d43fecb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.318+0000 7f404d600640 1 -- 172.21.15.119:0/3934559630 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038037070 con 0x7f40500d0ef0 2024-04-25T10:09:07.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.318+0000 7f404d600640 1 -- 172.21.15.119:0/3934559630 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4038032440 con 0x7f40500d0ef0 2024-04-25T10:09:07.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.318+0000 7f404d600640 1 -- 172.21.15.119:0/3934559630 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038032810 con 0x7f40500d0ef0 2024-04-25T10:09:07.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.319+0000 7f4055c00640 1 -- 172.21.15.119:0/3934559630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40500d0ef0 msgr2=0x7f40500ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.319+0000 7f4055c00640 1 --2- 172.21.15.119:0/3934559630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40500d0ef0 0x7f40500ce5e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f40380097a0 tx=0x7f403802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.319+0000 7f4055c00640 1 -- 172.21.15.119:0/3934559630 shutdown_connections 2024-04-25T10:09:07.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.319+0000 7f4055c00640 1 --2- 172.21.15.119:0/3934559630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40500d0ef0 0x7f40500ce5e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.319+0000 7f4055c00640 1 -- 172.21.15.119:0/3934559630 >> 172.21.15.119:0/3934559630 conn(0x7f4050099a40 msgr2=0x7f40500d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.319+0000 7f4055c00640 1 -- 172.21.15.119:0/3934559630 shutdown_connections 2024-04-25T10:09:07.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.319+0000 7f4055c00640 1 -- 172.21.15.119:0/3934559630 wait complete. 2024-04-25T10:09:07.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.321+0000 7f4055c00640 1 Processor -- start 2024-04-25T10:09:07.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.321+0000 7f4055c00640 1 -- start start 2024-04-25T10:09:07.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.322+0000 7f4055c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40500cb250 0x7f40501f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.322+0000 7f4055c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40500cce80 0x7f40501f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.322+0000 7f4055c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40500d0ef0 0x7f40500ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.322+0000 7f4055c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405015b6a0 con 0x7f40500cce80 2024-04-25T10:09:07.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.322+0000 7f4055c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405015b980 con 0x7f40500d0ef0 2024-04-25T10:09:07.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.322+0000 7f4055c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40500cbdf0 con 0x7f40500cb250 2024-04-25T10:09:07.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.322+0000 7f404e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40500cce80 0x7f40501f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.322+0000 7f404e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40500cce80 0x7f40501f4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38492/0 (socket says 172.21.15.119:38492) 2024-04-25T10:09:07.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.322+0000 7f404e000640 1 -- 172.21.15.119:0/3457582212 learned_addr learned my addr 172.21.15.119:0/3457582212 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:07.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.323+0000 7f404e000640 1 -- 172.21.15.119:0/3457582212 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40500cb250 msgr2=0x7f40501f3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.323+0000 7f404e000640 1 --2- 172.21.15.119:0/3457582212 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40500cb250 0x7f40501f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.323+0000 7f404e000640 1 -- 172.21.15.119:0/3457582212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40500d0ef0 msgr2=0x7f40500ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.323+0000 7f404e000640 1 --2- 172.21.15.119:0/3457582212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40500d0ef0 0x7f40500ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.323+0000 7f404e000640 1 -- 172.21.15.119:0/3457582212 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405015d030 con 0x7f40500cce80 2024-04-25T10:09:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.323+0000 7f404e000640 1 --2- 172.21.15.119:0/3457582212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40500cce80 0x7f40501f4f00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4038035040 tx=0x7f40380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.323+0000 7f4046a00640 1 -- 172.21.15.119:0/3457582212 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038037040 con 0x7f40500cce80 2024-04-25T10:09:07.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.323+0000 7f4046a00640 1 -- 172.21.15.119:0/3457582212 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4038004230 con 0x7f40500cce80 2024-04-25T10:09:07.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.323+0000 7f4055c00640 1 -- 172.21.15.119:0/3457582212 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40500b4780 con 0x7f40500cce80 2024-04-25T10:09:07.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.323+0000 7f4046a00640 1 -- 172.21.15.119:0/3457582212 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038032e40 con 0x7f40500cce80 2024-04-25T10:09:07.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.323+0000 7f4055c00640 1 -- 172.21.15.119:0/3457582212 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40500b4d40 con 0x7f40500cce80 2024-04-25T10:09:07.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.325+0000 7f4046a00640 1 -- 172.21.15.119:0/3457582212 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4038030770 con 0x7f40500cce80 2024-04-25T10:09:07.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.326+0000 7f4046a00640 1 --2- 172.21.15.119:0/3457582212 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f402c06b4d0 0x7f402c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.326+0000 7f4046a00640 1 -- 172.21.15.119:0/3457582212 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40380ab4c0 con 0x7f40500cce80 2024-04-25T10:09:07.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.327+0000 7f404ea00640 1 --2- 172.21.15.119:0/3457582212 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f402c06b4d0 0x7f402c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.327+0000 7f404ea00640 1 --2- 172.21.15.119:0/3457582212 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f402c06b4d0 0x7f402c06d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4040001560 tx=0x7f4040007ca0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.329+0000 7f405899c740 1 Processor -- start 2024-04-25T10:09:07.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.329+0000 7f405899c740 1 -- start start 2024-04-25T10:09:07.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.330+0000 7f405899c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41e861a0 0x55fe41e86580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.330+0000 7f405899c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41ea3a40 0x55fe41ea3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.330+0000 7f405899c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e5be80 0x55fe41ea2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.330+0000 7f405899c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe41dbe980 con 0x55fe41ea3a40 2024-04-25T10:09:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.330+0000 7f405899c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe41d67de0 con 0x55fe41e861a0 2024-04-25T10:09:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.330+0000 7f405899c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe41dbaa20 con 0x55fe41e5be80 2024-04-25T10:09:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.331+0000 7f404e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41ea3a40 0x55fe41ea3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.331+0000 7f404e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41ea3a40 0x55fe41ea3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38500/0 (socket says 172.21.15.119:38500) 2024-04-25T10:09:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.331+0000 7f404e000640 1 -- 172.21.15.119:0/3693584183 learned_addr learned my addr 172.21.15.119:0/3693584183 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.331+0000 7f404ea00640 1 --2- 172.21.15.119:0/3693584183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41e861a0 0x55fe41e86580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.331+0000 7f404f400640 1 --2- 172.21.15.119:0/3693584183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e5be80 0x55fe41ea2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.331+0000 7f404e000640 1 -- 172.21.15.119:0/3693584183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e5be80 msgr2=0x55fe41ea2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.331+0000 7f404e000640 1 --2- 172.21.15.119:0/3693584183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e5be80 0x55fe41ea2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.331+0000 7f404e000640 1 -- 172.21.15.119:0/3693584183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41e861a0 msgr2=0x55fe41e86580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.331+0000 7f404e000640 1 --2- 172.21.15.119:0/3693584183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41e861a0 0x55fe41e86580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.331+0000 7f404e000640 1 -- 172.21.15.119:0/3693584183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe41d85e30 con 0x55fe41ea3a40 2024-04-25T10:09:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.331+0000 7f404ea00640 1 --2- 172.21.15.119:0/3693584183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41e861a0 0x55fe41e86580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:07.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.331+0000 7f404f400640 1 --2- 172.21.15.119:0/3693584183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e5be80 0x55fe41ea2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:07.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.332+0000 7f404e000640 1 --2- 172.21.15.119:0/3693584183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41ea3a40 0x55fe41ea3f90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f40380761b0 tx=0x7f403807a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb85406a647ef26b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.332+0000 7f403f400640 1 -- 172.21.15.119:0/3693584183 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038091020 con 0x55fe41ea3a40 2024-04-25T10:09:07.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.332+0000 7f403f400640 1 -- 172.21.15.119:0/3693584183 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4038082460 con 0x55fe41ea3a40 2024-04-25T10:09:07.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.332+0000 7f403f400640 1 -- 172.21.15.119:0/3693584183 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40380800b0 con 0x55fe41ea3a40 2024-04-25T10:09:07.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.333+0000 7f405899c740 1 -- 172.21.15.119:0/3693584183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41ea3a40 msgr2=0x55fe41ea3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.333+0000 7f405899c740 1 --2- 172.21.15.119:0/3693584183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41ea3a40 0x55fe41ea3f90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f40380761b0 tx=0x7f403807a510 comp rx=0 tx=0).stop 2024-04-25T10:09:07.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.333+0000 7f405899c740 1 -- 172.21.15.119:0/3693584183 shutdown_connections 2024-04-25T10:09:07.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.333+0000 7f405899c740 1 --2- 172.21.15.119:0/3693584183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e5be80 0x55fe41ea2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.333+0000 7f405899c740 1 --2- 172.21.15.119:0/3693584183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41ea3a40 0x55fe41ea3f90 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.333+0000 7f405899c740 1 --2- 172.21.15.119:0/3693584183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41e861a0 0x55fe41e86580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.333+0000 7f405899c740 1 -- 172.21.15.119:0/3693584183 >> 172.21.15.119:0/3693584183 conn(0x55fe41e587b0 msgr2=0x55fe41ea0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.333+0000 7f405899c740 1 -- 172.21.15.119:0/3693584183 shutdown_connections 2024-04-25T10:09:07.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.333+0000 7f405899c740 1 -- 172.21.15.119:0/3693584183 wait complete. 2024-04-25T10:09:07.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.334+0000 7f405899c740 1 -- start start 2024-04-25T10:09:07.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.334+0000 7f405899c740 1 Processor -- start 2024-04-25T10:09:07.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.335+0000 7f405899c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e861a0 0x55fe41e7bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.335+0000 7f405899c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 0x55fe41eb4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.335+0000 7f405899c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e98c40 0x55fe41eb5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.335+0000 7f405899c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe41dbe980 con 0x55fe41e98c40 2024-04-25T10:09:07.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.335+0000 7f405899c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe41d5d3c0 con 0x55fe41ea3a40 2024-04-25T10:09:07.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.335+0000 7f405899c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe41e5cc20 con 0x55fe41e861a0 2024-04-25T10:09:07.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.335+0000 7f405899c740 10 client.0 _ll_register_callbacks cb 0x55fe41ad2c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:07.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.335+0000 7f404f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e98c40 0x55fe41eb5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.335+0000 7f404f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e98c40 0x55fe41eb5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38502/0 (socket says 172.21.15.119:38502) 2024-04-25T10:09:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.335+0000 7f404f400640 1 -- 172.21.15.119:0/2445947213 learned_addr learned my addr 172.21.15.119:0/2445947213 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.336+0000 7f404e000640 1 --2- 172.21.15.119:0/2445947213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 0x55fe41eb4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.336+0000 7f404ea00640 1 --2- 172.21.15.119:0/2445947213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e861a0 0x55fe41e7bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.336+0000 7f404f400640 1 -- 172.21.15.119:0/2445947213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e861a0 msgr2=0x55fe41e7bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.336+0000 7f404f400640 1 --2- 172.21.15.119:0/2445947213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e861a0 0x55fe41e7bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.336+0000 7f404f400640 1 -- 172.21.15.119:0/2445947213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 msgr2=0x55fe41eb4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.336+0000 7f404f400640 1 --2- 172.21.15.119:0/2445947213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 0x55fe41eb4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.336+0000 7f404f400640 1 -- 172.21.15.119:0/2445947213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe41b9b7e0 con 0x55fe41e98c40 2024-04-25T10:09:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.336+0000 7f404ea00640 1 --2- 172.21.15.119:0/2445947213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e861a0 0x55fe41e7bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.336+0000 7f404f400640 1 --2- 172.21.15.119:0/2445947213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e98c40 0x55fe41eb5080 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4048004550 tx=0x7f404800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aedb5793c93acd55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.336+0000 7f4033e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:09:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.337+0000 7f4033e00640 1 -- 172.21.15.119:0/2445947213 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048016020 con 0x55fe41e98c40 2024-04-25T10:09:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.337+0000 7f4033e00640 1 -- 172.21.15.119:0/2445947213 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40480125f0 con 0x55fe41e98c40 2024-04-25T10:09:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.337+0000 7f405899c740 1 -- 172.21.15.119:0/2445947213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4038082460 con 0x55fe41e98c40 2024-04-25T10:09:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.337+0000 7f4033e00640 1 -- 172.21.15.119:0/2445947213 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4048020050 con 0x55fe41e98c40 2024-04-25T10:09:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.337+0000 7f405899c740 10 client.5676 fetch_fsmap learned FSMap version 25 2024-04-25T10:09:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.337+0000 7f405899c740 1 -- 172.21.15.119:0/2445947213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55fe41bfbf70 con 0x55fe41e98c40 2024-04-25T10:09:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.337+0000 7f4033e00640 1 -- 172.21.15.119:0/2445947213 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4038082460 con 0x55fe41e98c40 2024-04-25T10:09:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.337+0000 7f4033e00640 1 -- 172.21.15.119:0/2445947213 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4048023020 con 0x55fe41e98c40 2024-04-25T10:09:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.338+0000 7f405899c740 10 client.5676 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:09:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.338+0000 7f405899c740 10 client.5676 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:09:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.338+0000 7f405899c740 20 client.5676 populate_metadata read hostname 'smithi119' 2024-04-25T10:09:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.338+0000 7f405899c740 1 --2- 172.21.15.119:0/2445947213 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55fe41e9aad0 0x55fe41e9aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.338+0000 7f405899c740 4 client.5676 mds_command: new command op to 5478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:07.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.338+0000 7f405899c740 1 -- 172.21.15.119:0/2445947213 --> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe41dcbec0 con 0x55fe41e9aad0 2024-04-25T10:09:07.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.338+0000 7f404ea00640 1 --2- 172.21.15.119:0/2445947213 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55fe41e9aad0 0x55fe41e9aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.338+0000 7f404ea00640 1 --2- 172.21.15.119:0/2445947213 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55fe41e9aad0 0x55fe41e9aeb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec5c1dbef5dd1acb server_cookie=21fd6a340099f532 in_seq=0 out_seq=0 2024-04-25T10:09:07.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.338+0000 7f4033e00640 10 client.5676 ms_handle_connect on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:07.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.339+0000 7f4033e00640 1 -- 172.21.15.119:0/2445947213 <== mds.0 v2:172.21.15.119:6835/270766821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe41dcbec0 con 0x55fe41e9aad0 2024-04-25T10:09:07.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.339+0000 7f4033e00640 10 client.5676 handle_command_reply: tid=0 2024-04-25T10:09:07.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.339+0000 7f405899c740 1 client.5676 shutdown 2024-04-25T10:09:07.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.339+0000 7f405899c740 1 -- 172.21.15.119:0/2445947213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e98c40 msgr2=0x55fe41eb5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.339+0000 7f405899c740 1 --2- 172.21.15.119:0/2445947213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e98c40 0x55fe41eb5080 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4048004550 tx=0x7f404800d520 comp rx=0 tx=0).stop 2024-04-25T10:09:07.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.339+0000 7f405899c740 1 -- 172.21.15.119:0/2445947213 shutdown_connections 2024-04-25T10:09:07.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.339+0000 7f405899c740 1 --2- 172.21.15.119:0/2445947213 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55fe41e9aad0 0x55fe41e9aeb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.339+0000 7f405899c740 1 --2- 172.21.15.119:0/2445947213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e98c40 0x55fe41eb5080 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.339+0000 7f405899c740 1 --2- 172.21.15.119:0/2445947213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 0x55fe41eb4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.339+0000 7f405899c740 1 --2- 172.21.15.119:0/2445947213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e861a0 0x55fe41e7bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.340+0000 7f405899c740 1 -- 172.21.15.119:0/2445947213 >> 172.21.15.119:0/2445947213 conn(0x55fe41e5be80 msgr2=0x55fe41ea4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.340+0000 7f4033e00640 0 client.5676 ms_handle_reset on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:07.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.340+0000 7f405899c740 1 -- 172.21.15.119:0/2445947213 shutdown_connections 2024-04-25T10:09:07.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.340+0000 7f405899c740 1 -- 172.21.15.119:0/2445947213 wait complete. 2024-04-25T10:09:07.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.340+0000 7f405899c740 20 client.5676 trim_cache size 0 max 16384 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.350+0000 7f4effa00640 1 Processor -- start 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.350+0000 7f4effa00640 1 -- start start 2024-04-25T10:09:07.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.351+0000 7f4effa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ef815eed0 0x7f4ef80b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.351+0000 7f4effa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4ef80ce850 0x7f4ef80cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.351+0000 7f4effa00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef815f6a0 con 0x7f4ef80b75c0 2024-04-25T10:09:07.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.351+0000 7f4effa00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef815f980 con 0x7f4ef80ce850 2024-04-25T10:09:07.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.351+0000 7f4effa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef815fc60 con 0x7f4ef815eed0 2024-04-25T10:09:07.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.352+0000 7f4efd000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ef815eed0 0x7f4ef80b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.352+0000 7f4ef7e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f4ef80b75c0 0x7f4ef80ce140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:45938/0 (socket says 172.21.15.119:45938) 2024-04-25T10:09:07.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.352+0000 7f4ef7e00640 1 -- 172.21.15.119:0/608671032 learned_addr learned my addr 172.21.15.119:0/608671032 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:07.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.352+0000 7f4efda00640 1 --2- 172.21.15.119:0/608671032 >> v2:172.21.15.132:3300/0 conn(0x7f4ef80ce850 0x7f4ef80cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.352+0000 7f4efd000640 1 -- 172.21.15.119:0/608671032 >> v2:172.21.15.132:3300/0 conn(0x7f4ef80ce850 msgr2=0x7f4ef80cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.352+0000 7f4efd000640 1 --2- 172.21.15.119:0/608671032 >> v2:172.21.15.132:3300/0 conn(0x7f4ef80ce850 0x7f4ef80cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.352+0000 7f4efd000640 1 -- 172.21.15.119:0/608671032 >> v1:172.21.15.132:6789/0 conn(0x7f4ef80b75c0 legacy=0x7f4ef80ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.352+0000 7f4efd000640 1 -- 172.21.15.119:0/608671032 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef815ff40 con 0x7f4ef815eed0 2024-04-25T10:09:07.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.354+0000 7f4efd000640 1 --2- 172.21.15.119:0/608671032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ef815eed0 0x7f4ef80b7080 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0097a0 tx=0x7f4eec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e592d6d08069b59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.354+0000 7f4ef7400640 1 -- 172.21.15.119:0/608671032 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec037070 con 0x7f4ef815eed0 2024-04-25T10:09:07.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.354+0000 7f4ef7400640 1 -- 172.21.15.119:0/608671032 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eec032440 con 0x7f4ef815eed0 2024-04-25T10:09:07.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.354+0000 7f4ef7400640 1 -- 172.21.15.119:0/608671032 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec032810 con 0x7f4ef815eed0 2024-04-25T10:09:07.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.355+0000 7f4effa00640 1 -- 172.21.15.119:0/608671032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ef815eed0 msgr2=0x7f4ef80b7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.355+0000 7f4effa00640 1 --2- 172.21.15.119:0/608671032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ef815eed0 0x7f4ef80b7080 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0097a0 tx=0x7f4eec02a180 comp rx=0 tx=0).stop 2024-04-25T10:09:07.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.355+0000 7f4effa00640 1 -- 172.21.15.119:0/608671032 shutdown_connections 2024-04-25T10:09:07.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.355+0000 7f4effa00640 1 --2- 172.21.15.119:0/608671032 >> v2:172.21.15.132:3300/0 conn(0x7f4ef80ce850 0x7f4ef80cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.355+0000 7f4effa00640 1 --2- 172.21.15.119:0/608671032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ef815eed0 0x7f4ef80b7080 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.355+0000 7f4effa00640 1 -- 172.21.15.119:0/608671032 >> 172.21.15.119:0/608671032 conn(0x7f4ef806f4a0 msgr2=0x7f4ef806f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.355+0000 7f4effa00640 1 -- 172.21.15.119:0/608671032 shutdown_connections 2024-04-25T10:09:07.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.355+0000 7f4effa00640 1 -- 172.21.15.119:0/608671032 wait complete. 2024-04-25T10:09:07.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.357+0000 7f4effa00640 1 Processor -- start 2024-04-25T10:09:07.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.357+0000 7f405899c740 1 Processor -- start 2024-04-25T10:09:07.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.357+0000 7f4effa00640 1 -- start start 2024-04-25T10:09:07.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.357+0000 7f405899c740 1 -- start start 2024-04-25T10:09:07.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f405899c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e861a0 0x55fe41e5c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f405899c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e98c40 0x55fe41ea1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f405899c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 0x55fe41ea2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f4effa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ef80b75c0 0x7f4ef80cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f405899c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40501ffd00 con 0x55fe41e98c40 2024-04-25T10:09:07.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f405899c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500c5c30 con 0x55fe41ea3a40 2024-04-25T10:09:07.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f405899c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40500b5e00 con 0x55fe41e861a0 2024-04-25T10:09:07.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f4effa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ef80ce850 0x7f4ef8151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f4effa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ef815eed0 0x7f4ef8151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f4effa00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef8160500 con 0x7f4ef815eed0 2024-04-25T10:09:07.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f4effa00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef8160220 con 0x7f4ef80ce850 2024-04-25T10:09:07.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f4effa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef815f2b0 con 0x7f4ef80b75c0 2024-04-25T10:09:07.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f4efda00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ef815eed0 0x7f4ef8151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f404e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e98c40 0x55fe41ea1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f4efda00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ef815eed0 0x7f4ef8151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38516/0 (socket says 172.21.15.119:38516) 2024-04-25T10:09:07.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f404e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e98c40 0x55fe41ea1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38514/0 (socket says 172.21.15.119:38514) 2024-04-25T10:09:07.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f404e000640 1 -- 172.21.15.119:0/1945761892 learned_addr learned my addr 172.21.15.119:0/1945761892 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:07.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.358+0000 7f4efda00640 1 -- 172.21.15.119:0/3627293494 learned_addr learned my addr 172.21.15.119:0/3627293494 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:07.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f404f400640 1 --2- 172.21.15.119:0/1945761892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 0x55fe41ea2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f4efd000640 1 --2- 172.21.15.119:0/3627293494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ef80b75c0 0x7f4ef80cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f404ea00640 1 --2- 172.21.15.119:0/1945761892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e861a0 0x55fe41e5c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f404e000640 1 -- 172.21.15.119:0/1945761892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e861a0 msgr2=0x55fe41e5c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f404e000640 1 --2- 172.21.15.119:0/1945761892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e861a0 0x55fe41e5c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f404e000640 1 -- 172.21.15.119:0/1945761892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 msgr2=0x55fe41ea2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f404e000640 1 --2- 172.21.15.119:0/1945761892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 0x55fe41ea2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f404e000640 1 -- 172.21.15.119:0/1945761892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe41a93170 con 0x55fe41e98c40 2024-04-25T10:09:07.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f4ef7e00640 1 --2- 172.21.15.119:0/3627293494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ef80ce850 0x7f4ef8151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f4efda00640 1 -- 172.21.15.119:0/3627293494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ef80b75c0 msgr2=0x7f4ef80cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f4efda00640 1 --2- 172.21.15.119:0/3627293494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ef80b75c0 0x7f4ef80cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f4efda00640 1 -- 172.21.15.119:0/3627293494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ef80ce850 msgr2=0x7f4ef8151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f4efda00640 1 --2- 172.21.15.119:0/3627293494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ef80ce850 0x7f4ef8151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f4efda00640 1 -- 172.21.15.119:0/3627293494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef81514c0 con 0x7f4ef815eed0 2024-04-25T10:09:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f4efd000640 1 --2- 172.21.15.119:0/3627293494 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ef80b75c0 0x7f4ef80cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.359+0000 7f404e000640 1 --2- 172.21.15.119:0/1945761892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e98c40 0x55fe41ea1fd0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4038079f10 tx=0x7f4038008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6976714e19c71c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.360+0000 7f4efda00640 1 --2- 172.21.15.119:0/3627293494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ef815eed0 0x7f4ef8151c60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000efc0 tx=0x7f4ef000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.360+0000 7f403ea00640 1 -- 172.21.15.119:0/1945761892 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038091040 con 0x55fe41e98c40 2024-04-25T10:09:07.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.360+0000 7f403ea00640 1 -- 172.21.15.119:0/1945761892 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4038082460 con 0x55fe41e98c40 2024-04-25T10:09:07.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.360+0000 7f403ea00640 1 -- 172.21.15.119:0/1945761892 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038075ff0 con 0x55fe41e98c40 2024-04-25T10:09:07.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.360+0000 7f4ef4c00640 1 -- 172.21.15.119:0/3627293494 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0018020 con 0x7f4ef815eed0 2024-04-25T10:09:07.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.360+0000 7f4ef4c00640 1 -- 172.21.15.119:0/3627293494 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef0014480 con 0x7f4ef815eed0 2024-04-25T10:09:07.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.360+0000 7f4ef4c00640 1 -- 172.21.15.119:0/3627293494 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0014870 con 0x7f4ef815eed0 2024-04-25T10:09:07.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.360+0000 7f4effa00640 1 -- 172.21.15.119:0/3627293494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef8150420 con 0x7f4ef815eed0 2024-04-25T10:09:07.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.360+0000 7f4effa00640 1 -- 172.21.15.119:0/3627293494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef81509e0 con 0x7f4ef815eed0 2024-04-25T10:09:07.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.361+0000 7f405899c740 1 -- 172.21.15.119:0/1945761892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e98c40 msgr2=0x55fe41ea1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.361+0000 7f405899c740 1 --2- 172.21.15.119:0/1945761892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e98c40 0x55fe41ea1fd0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4038079f10 tx=0x7f4038008000 comp rx=0 tx=0).stop 2024-04-25T10:09:07.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.361+0000 7f405899c740 1 -- 172.21.15.119:0/1945761892 shutdown_connections 2024-04-25T10:09:07.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.361+0000 7f405899c740 1 --2- 172.21.15.119:0/1945761892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 0x55fe41ea2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.361+0000 7f405899c740 1 --2- 172.21.15.119:0/1945761892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e98c40 0x55fe41ea1fd0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.361+0000 7f405899c740 1 --2- 172.21.15.119:0/1945761892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e861a0 0x55fe41e5c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.361+0000 7f405899c740 1 -- 172.21.15.119:0/1945761892 >> 172.21.15.119:0/1945761892 conn(0x55fe41e5be80 msgr2=0x55fe41e981f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.361+0000 7f4effa00640 1 -- 172.21.15.119:0/3627293494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f4ef80749b0 con 0x7f4ef815eed0 2024-04-25T10:09:07.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.361+0000 7f405899c740 1 -- 172.21.15.119:0/1945761892 shutdown_connections 2024-04-25T10:09:07.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.361+0000 7f405899c740 1 -- 172.21.15.119:0/1945761892 wait complete. 2024-04-25T10:09:07.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.362+0000 7f405899c740 1 -- start start 2024-04-25T10:09:07.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.362+0000 7f405899c740 1 Processor -- start 2024-04-25T10:09:07.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.362+0000 7f4ef4c00640 1 -- 172.21.15.119:0/3627293494 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ef0005d10 con 0x7f4ef815eed0 2024-04-25T10:09:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.363+0000 7f405899c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e861a0 0x55fe41eef300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.363+0000 7f405899c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 0x55fe41e87980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.363+0000 7f405899c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e87ec0 0x55fe41e88340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.363+0000 7f405899c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe41e969f0 con 0x55fe41e861a0 2024-04-25T10:09:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.363+0000 7f405899c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500c5630 con 0x55fe41ea3a40 2024-04-25T10:09:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.363+0000 7f405899c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe41e97940 con 0x55fe41e87ec0 2024-04-25T10:09:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.363+0000 7f405899c740 10 client.0 _ll_register_callbacks cb 0x55fe41e56bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.363+0000 7f404ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e861a0 0x55fe41eef300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.363+0000 7f404ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e861a0 0x55fe41eef300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38526/0 (socket says 172.21.15.119:38526) 2024-04-25T10:09:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.363+0000 7f404ea00640 1 -- 172.21.15.119:0/1735739744 learned_addr learned my addr 172.21.15.119:0/1735739744 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.363+0000 7f4ef4c00640 1 --2- 172.21.15.119:0/3627293494 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ed806b4d0 0x7f4ed806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.363+0000 7f4ef4c00640 1 -- 172.21.15.119:0/3627293494 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ef008b5a0 con 0x7f4ef815eed0 2024-04-25T10:09:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f404e000640 1 --2- 172.21.15.119:0/1735739744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 0x55fe41e87980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f404f400640 1 --2- 172.21.15.119:0/1735739744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e87ec0 0x55fe41e88340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f404ea00640 1 -- 172.21.15.119:0/1735739744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e87ec0 msgr2=0x55fe41e88340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f404ea00640 1 --2- 172.21.15.119:0/1735739744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e87ec0 0x55fe41e88340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f404ea00640 1 -- 172.21.15.119:0/1735739744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 msgr2=0x55fe41e87980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f404ea00640 1 --2- 172.21.15.119:0/1735739744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 0x55fe41e87980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f404ea00640 1 -- 172.21.15.119:0/1735739744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe41eb3d80 con 0x55fe41e861a0 2024-04-25T10:09:07.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f4efd000640 1 --2- 172.21.15.119:0/3627293494 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ed806b4d0 0x7f4ed806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f4ef4c00640 1 -- 172.21.15.119:0/3627293494 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+501 (secure 0 0 0) 0x7f4ef0022070 con 0x7f4ef815eed0 2024-04-25T10:09:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f4effa00640 1 -- 172.21.15.119:0/3627293494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f4ef80702d0 con 0x7f4ef815eed0 2024-04-25T10:09:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f404ea00640 1 --2- 172.21.15.119:0/1735739744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e861a0 0x55fe41eef300 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4040002230 tx=0x7f4040001470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dac65550bb0cdc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f4033e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:09:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f4033e00640 1 -- 172.21.15.119:0/1735739744 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040004e30 con 0x55fe41e861a0 2024-04-25T10:09:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f4033e00640 1 -- 172.21.15.119:0/1735739744 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4040015ae0 con 0x55fe41e861a0 2024-04-25T10:09:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.364+0000 7f405899c740 1 -- 172.21.15.119:0/1735739744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4038082460 con 0x55fe41e861a0 2024-04-25T10:09:07.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.365+0000 7f4efd000640 1 --2- 172.21.15.119:0/3627293494 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ed806b4d0 0x7f4ed806d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4eec034000 tx=0x7f4eec02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.365+0000 7f4033e00640 1 -- 172.21.15.119:0/1735739744 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4040015ce0 con 0x55fe41e861a0 2024-04-25T10:09:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.365+0000 7f405899c740 10 client.5685 fetch_fsmap learned FSMap version 25 2024-04-25T10:09:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.365+0000 7f405899c740 1 -- 172.21.15.119:0/1735739744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55fe41eb4900 con 0x55fe41e861a0 2024-04-25T10:09:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.365+0000 7f4033e00640 1 -- 172.21.15.119:0/1735739744 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4040026030 con 0x55fe41e861a0 2024-04-25T10:09:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.365+0000 7f4033e00640 1 -- 172.21.15.119:0/1735739744 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4040022470 con 0x55fe41e861a0 2024-04-25T10:09:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.365+0000 7f405899c740 10 client.5685 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:09:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.365+0000 7f405899c740 10 client.5685 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:09:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.365+0000 7f405899c740 20 client.5685 populate_metadata read hostname 'smithi119' 2024-04-25T10:09:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.365+0000 7f405899c740 1 --2- 172.21.15.119:0/1735739744 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55fe41e81400 0x55fe41e81810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.365+0000 7f405899c740 4 client.5685 mds_command: new command op to 5478 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:09:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.365+0000 7f405899c740 1 -- 172.21.15.119:0/1735739744 --> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55fe41dc6210 con 0x55fe41e81400 2024-04-25T10:09:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.365+0000 7f404e000640 1 --2- 172.21.15.119:0/1735739744 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55fe41e81400 0x55fe41e81810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.366+0000 7f404e000640 1 --2- 172.21.15.119:0/1735739744 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55fe41e81400 0x55fe41e81810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=451859de6f08234a server_cookie=38bc78c75849bdaa in_seq=0 out_seq=0 2024-04-25T10:09:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.366+0000 7f4033e00640 10 client.5685 ms_handle_connect on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.367+0000 7f4033e00640 1 -- 172.21.15.119:0/1735739744 <== mds.0 v2:172.21.15.119:6835/270766821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fe41dc6210 con 0x55fe41e81400 2024-04-25T10:09:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.367+0000 7f4033e00640 10 client.5685 handle_command_reply: tid=0 2024-04-25T10:09:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.367+0000 7f405899c740 1 client.5685 shutdown 2024-04-25T10:09:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.367+0000 7f405899c740 1 -- 172.21.15.119:0/1735739744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e861a0 msgr2=0x55fe41eef300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.367+0000 7f405899c740 1 --2- 172.21.15.119:0/1735739744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e861a0 0x55fe41eef300 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4040002230 tx=0x7f4040001470 comp rx=0 tx=0).stop 2024-04-25T10:09:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.367+0000 7f405899c740 1 -- 172.21.15.119:0/1735739744 shutdown_connections 2024-04-25T10:09:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.367+0000 7f405899c740 1 --2- 172.21.15.119:0/1735739744 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x55fe41e81400 0x55fe41e81810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.367+0000 7f405899c740 1 --2- 172.21.15.119:0/1735739744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe41e87ec0 0x55fe41e88340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.367+0000 7f405899c740 1 --2- 172.21.15.119:0/1735739744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe41ea3a40 0x55fe41e87980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.367+0000 7f405899c740 1 --2- 172.21.15.119:0/1735739744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe41e861a0 0x55fe41eef300 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.367+0000 7f405899c740 1 -- 172.21.15.119:0/1735739744 >> 172.21.15.119:0/1735739744 conn(0x55fe41e5be80 msgr2=0x55fe41e86e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.367+0000 7f4033e00640 0 client.5685 ms_handle_reset on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:07.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.367+0000 7f405899c740 1 -- 172.21.15.119:0/1735739744 shutdown_connections 2024-04-25T10:09:07.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.368+0000 7f405899c740 1 -- 172.21.15.119:0/1735739744 wait complete. 2024-04-25T10:09:07.385 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:09:07.368+0000 7f5282600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={4984=0-4194304@1} caps={4984=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=4984 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x560d9e018100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:09:07.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.368+0000 7f405899c740 20 client.5685 trim_cache size 0 max 16384 2024-04-25T10:09:07.386 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:09:07.386 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2024-04-25T10:09:07.386 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "79c62faf-99ba-409c-ac46-2dac41f5f2e6", 2024-04-25T10:09:07.386 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2024-04-25T10:09:07.386 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:09:07.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.368+0000 7f4055c00640 1 -- 172.21.15.119:0/3457582212 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f402c06b4d0 msgr2=0x7f402c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.368+0000 7f4055c00640 1 --2- 172.21.15.119:0/3457582212 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f402c06b4d0 0x7f402c06d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4040001560 tx=0x7f4040007ca0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.368+0000 7f4055c00640 1 -- 172.21.15.119:0/3457582212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40500cce80 msgr2=0x7f40501f4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.368+0000 7f4055c00640 1 --2- 172.21.15.119:0/3457582212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40500cce80 0x7f40501f4f00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4038035040 tx=0x7f40380046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.368+0000 7f4055c00640 1 -- 172.21.15.119:0/3457582212 shutdown_connections 2024-04-25T10:09:07.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.368+0000 7f4055c00640 1 --2- 172.21.15.119:0/3457582212 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f402c06b4d0 0x7f402c06d990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.368+0000 7f4055c00640 1 --2- 172.21.15.119:0/3457582212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40500d0ef0 0x7f40500ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.368+0000 7f4055c00640 1 --2- 172.21.15.119:0/3457582212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40500cce80 0x7f40501f4f00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.368+0000 7f4055c00640 1 --2- 172.21.15.119:0/3457582212 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40500cb250 0x7f40501f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.368+0000 7f4055c00640 1 -- 172.21.15.119:0/3457582212 >> 172.21.15.119:0/3457582212 conn(0x7f4050099a40 msgr2=0x7f4050074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.369+0000 7f4055c00640 1 -- 172.21.15.119:0/3457582212 shutdown_connections 2024-04-25T10:09:07.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.369+0000 7f4055c00640 1 -- 172.21.15.119:0/3457582212 wait complete. 2024-04-25T10:09:07.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:07.393 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:09:07.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.563+0000 7f5008200640 1 Processor -- start 2024-04-25T10:09:07.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.563+0000 7f5008200640 1 -- start start 2024-04-25T10:09:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.565+0000 7f5008200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f50000b42c0 0x7f500015ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.565+0000 7f5008200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f500015aef0 0x7f50000cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.565+0000 7f5008200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50000b46a0 0x7f50001549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.565+0000 7f5008200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50000d05d0 con 0x7f500015aef0 2024-04-25T10:09:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.565+0000 7f5008200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500015b370 con 0x7f50000b42c0 2024-04-25T10:09:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.565+0000 7f5008200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500015b650 con 0x7f50000b46a0 2024-04-25T10:09:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.565+0000 7f5004e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f500015aef0 0x7f50000cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.565+0000 7f5004e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f500015aef0 0x7f50000cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38532/0 (socket says 172.21.15.119:38532) 2024-04-25T10:09:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.565+0000 7f5004e00640 1 -- 172.21.15.119:0/1050170476 learned_addr learned my addr 172.21.15.119:0/1050170476 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.566+0000 7f5005800640 1 --2- 172.21.15.119:0/1050170476 >> v2:172.21.15.132:3300/0 conn(0x7f50000b42c0 0x7f500015ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.566+0000 7f5006200640 1 --2- 172.21.15.119:0/1050170476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50000b46a0 0x7f50001549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.566+0000 7f5004e00640 1 -- 172.21.15.119:0/1050170476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50000b46a0 msgr2=0x7f50001549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.566+0000 7f5004e00640 1 --2- 172.21.15.119:0/1050170476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50000b46a0 0x7f50001549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.566+0000 7f5004e00640 1 -- 172.21.15.119:0/1050170476 >> v2:172.21.15.132:3300/0 conn(0x7f50000b42c0 msgr2=0x7f500015ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.566+0000 7f5004e00640 1 --2- 172.21.15.119:0/1050170476 >> v2:172.21.15.132:3300/0 conn(0x7f50000b42c0 0x7f500015ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.566+0000 7f5004e00640 1 -- 172.21.15.119:0/1050170476 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500015b930 con 0x7f500015aef0 2024-04-25T10:09:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.567+0000 7f5004e00640 1 --2- 172.21.15.119:0/1050170476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f500015aef0 0x7f50000cf440 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f50000d07c0 tx=0x7f4fec02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f0d8e3ae9ee691 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.567+0000 7f4fffe00640 1 -- 172.21.15.119:0/1050170476 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec037070 con 0x7f500015aef0 2024-04-25T10:09:07.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.567+0000 7f4fffe00640 1 -- 172.21.15.119:0/1050170476 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec02ae00 con 0x7f500015aef0 2024-04-25T10:09:07.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.567+0000 7f4fffe00640 1 -- 172.21.15.119:0/1050170476 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec032660 con 0x7f500015aef0 2024-04-25T10:09:07.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.568+0000 7f5008200640 1 -- 172.21.15.119:0/1050170476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f500015aef0 msgr2=0x7f50000cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.568+0000 7f5008200640 1 --2- 172.21.15.119:0/1050170476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f500015aef0 0x7f50000cf440 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f50000d07c0 tx=0x7f4fec02a0b0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.568+0000 7f5008200640 1 -- 172.21.15.119:0/1050170476 shutdown_connections 2024-04-25T10:09:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.568+0000 7f5008200640 1 --2- 172.21.15.119:0/1050170476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50000b46a0 0x7f50001549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.568+0000 7f5008200640 1 --2- 172.21.15.119:0/1050170476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f500015aef0 0x7f50000cf440 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.568+0000 7f5008200640 1 --2- 172.21.15.119:0/1050170476 >> v2:172.21.15.132:3300/0 conn(0x7f50000b42c0 0x7f500015ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.568+0000 7f5008200640 1 -- 172.21.15.119:0/1050170476 >> 172.21.15.119:0/1050170476 conn(0x7f500006aba0 msgr2=0x7f500006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.568+0000 7f5008200640 1 -- 172.21.15.119:0/1050170476 shutdown_connections 2024-04-25T10:09:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.568+0000 7f5008200640 1 -- 172.21.15.119:0/1050170476 wait complete. 2024-04-25T10:09:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.570+0000 7f5008200640 1 Processor -- start 2024-04-25T10:09:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.570+0000 7f5008200640 1 -- start start 2024-04-25T10:09:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.571+0000 7f5008200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50000b42c0 0x7f500013ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.571+0000 7f5008200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50000b46a0 0x7f50001f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.571+0000 7f5008200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f500015aef0 0x7f50001f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.571+0000 7f5008200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500006eae0 con 0x7f50000b42c0 2024-04-25T10:09:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.571+0000 7f5008200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50000cf980 con 0x7f500015aef0 2024-04-25T10:09:07.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.571+0000 7f5008200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500006ec60 con 0x7f50000b46a0 2024-04-25T10:09:07.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.571+0000 7f5005800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50000b42c0 0x7f500013ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.571+0000 7f5005800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50000b42c0 0x7f500013ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38540/0 (socket says 172.21.15.119:38540) 2024-04-25T10:09:07.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.571+0000 7f5005800640 1 -- 172.21.15.119:0/3676580162 learned_addr learned my addr 172.21.15.119:0/3676580162 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:07.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.572+0000 7f5004e00640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50000b46a0 0x7f50001f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.572+0000 7f5006200640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f500015aef0 0x7f50001f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.572+0000 7f5005800640 1 -- 172.21.15.119:0/3676580162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50000b46a0 msgr2=0x7f50001f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.572+0000 7f5005800640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50000b46a0 0x7f50001f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.572+0000 7f5005800640 1 -- 172.21.15.119:0/3676580162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f500015aef0 msgr2=0x7f50001f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.572+0000 7f5005800640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f500015aef0 0x7f50001f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.572+0000 7f5005800640 1 -- 172.21.15.119:0/3676580162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5000151600 con 0x7f50000b42c0 2024-04-25T10:09:07.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.572+0000 7f5006200640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f500015aef0 0x7f50001f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:07.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.572+0000 7f5005800640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50000b42c0 0x7f500013ddd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4007ef0 tx=0x7f4ff400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.572+0000 7f4ffd600640 1 -- 172.21.15.119:0/3676580162 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4014620 con 0x7f50000b42c0 2024-04-25T10:09:07.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.573+0000 7f4ffd600640 1 -- 172.21.15.119:0/3676580162 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff4019070 con 0x7f50000b42c0 2024-04-25T10:09:07.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.573+0000 7f4ffd600640 1 -- 172.21.15.119:0/3676580162 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4014a00 con 0x7f50000b42c0 2024-04-25T10:09:07.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.573+0000 7f5008200640 1 -- 172.21.15.119:0/3676580162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50001518c0 con 0x7f50000b42c0 2024-04-25T10:09:07.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.573+0000 7f5008200640 1 -- 172.21.15.119:0/3676580162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5000151e80 con 0x7f50000b42c0 2024-04-25T10:09:07.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.575+0000 7f4ffd600640 1 -- 172.21.15.119:0/3676580162 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ff4014ba0 con 0x7f50000b42c0 2024-04-25T10:09:07.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.576+0000 7f4ffd600640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fe006b4d0 0x7f4fe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.576+0000 7f4ffd600640 1 -- 172.21.15.119:0/3676580162 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ff408ab50 con 0x7f50000b42c0 2024-04-25T10:09:07.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.576+0000 7f5004e00640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fe006b4d0 0x7f4fe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.576+0000 7f500b024740 1 Processor -- start 2024-04-25T10:09:07.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.576+0000 7f500b024740 1 -- start start 2024-04-25T10:09:07.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.577+0000 7f5004e00640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fe006b4d0 0x7f4fe006d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4fec034000 tx=0x7f4fec02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.577+0000 7f500b024740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6ac8bb0 0x5639e6acad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.577+0000 7f500b024740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6b05280 0x5639e6ac8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.577+0000 7f500b024740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6af3670 0x5639e6af3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.577+0000 7f500b024740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e6a2cc70 con 0x5639e6af3670 2024-04-25T10:09:07.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.577+0000 7f500b024740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e69d5cf0 con 0x5639e6ac8bb0 2024-04-25T10:09:07.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.577+0000 7f500b024740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639e6a28d10 con 0x5639e6b05280 2024-04-25T10:09:07.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.578+0000 7f5006200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6af3670 0x5639e6af3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.578+0000 7f5006200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6af3670 0x5639e6af3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38552/0 (socket says 172.21.15.119:38552) 2024-04-25T10:09:07.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.578+0000 7f5006200640 1 -- 172.21.15.119:0/1760113665 learned_addr learned my addr 172.21.15.119:0/1760113665 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:07.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.578+0000 7f5006200640 1 -- 172.21.15.119:0/1760113665 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6b05280 msgr2=0x5639e6ac8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.578+0000 7f5006200640 1 --2- 172.21.15.119:0/1760113665 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6b05280 0x5639e6ac8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.578+0000 7f5005800640 1 --2- 172.21.15.119:0/1760113665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6ac8bb0 0x5639e6acad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.578+0000 7f5006200640 1 -- 172.21.15.119:0/1760113665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6ac8bb0 msgr2=0x5639e6acad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.578+0000 7f5006200640 1 --2- 172.21.15.119:0/1760113665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6ac8bb0 0x5639e6acad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.578+0000 7f5006200640 1 -- 172.21.15.119:0/1760113665 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639e680b7d0 con 0x5639e6af3670 2024-04-25T10:09:07.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.578+0000 7f5006200640 1 --2- 172.21.15.119:0/1760113665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6af3670 0x5639e6af3af0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009920 tx=0x7f4ff8009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce073a1bc0d64245 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.579+0000 7f4ff2000640 1 -- 172.21.15.119:0/1760113665 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8017030 con 0x5639e6af3670 2024-04-25T10:09:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.579+0000 7f4ff2000640 1 -- 172.21.15.119:0/1760113665 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff8004020 con 0x5639e6af3670 2024-04-25T10:09:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.579+0000 7f4ff2000640 1 -- 172.21.15.119:0/1760113665 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff800f040 con 0x5639e6af3670 2024-04-25T10:09:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.579+0000 7f500b024740 1 -- 172.21.15.119:0/1760113665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6af3670 msgr2=0x5639e6af3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.579+0000 7f500b024740 1 --2- 172.21.15.119:0/1760113665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6af3670 0x5639e6af3af0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009920 tx=0x7f4ff8009de0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.579+0000 7f500b024740 1 -- 172.21.15.119:0/1760113665 shutdown_connections 2024-04-25T10:09:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.579+0000 7f500b024740 1 --2- 172.21.15.119:0/1760113665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6af3670 0x5639e6af3af0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.579+0000 7f500b024740 1 --2- 172.21.15.119:0/1760113665 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6b05280 0x5639e6ac8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.579+0000 7f500b024740 1 --2- 172.21.15.119:0/1760113665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6ac8bb0 0x5639e6acad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.579+0000 7f500b024740 1 -- 172.21.15.119:0/1760113665 >> 172.21.15.119:0/1760113665 conn(0x5639e6a90a30 msgr2=0x5639e6b0e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.579+0000 7f500b024740 1 -- 172.21.15.119:0/1760113665 shutdown_connections 2024-04-25T10:09:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.579+0000 7f500b024740 1 -- 172.21.15.119:0/1760113665 wait complete. 2024-04-25T10:09:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.580+0000 7f500b024740 1 -- start start 2024-04-25T10:09:07.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.580+0000 7f500b024740 1 Processor -- start 2024-04-25T10:09:07.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.581+0000 7f500b024740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6ac8bb0 0x5639e6b07620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.581+0000 7f500b024740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6af3670 0x5639e6b21b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.581+0000 7f500b024740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6b05280 0x5639e6ae1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.581+0000 7f500b024740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e69cb3a0 con 0x5639e6ac8bb0 2024-04-25T10:09:07.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.581+0000 7f500b024740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e6af11c0 con 0x5639e6af3670 2024-04-25T10:09:07.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.581+0000 7f500b024740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639e6af6cb0 con 0x5639e6b05280 2024-04-25T10:09:07.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.581+0000 7f500b024740 10 client.0 _ll_register_callbacks cb 0x5639e69e8b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:07.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.582+0000 7f5005800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6ac8bb0 0x5639e6b07620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.582+0000 7f5005800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6ac8bb0 0x5639e6b07620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38554/0 (socket says 172.21.15.119:38554) 2024-04-25T10:09:07.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.582+0000 7f5005800640 1 -- 172.21.15.119:0/3549722888 learned_addr learned my addr 172.21.15.119:0/3549722888 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:07.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.582+0000 7f5004e00640 1 --2- 172.21.15.119:0/3549722888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6af3670 0x5639e6b21b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.582+0000 7f5006200640 1 --2- 172.21.15.119:0/3549722888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6b05280 0x5639e6ae1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.582+0000 7f5005800640 1 -- 172.21.15.119:0/3549722888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6b05280 msgr2=0x5639e6ae1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.582+0000 7f5005800640 1 --2- 172.21.15.119:0/3549722888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6b05280 0x5639e6ae1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.582+0000 7f5005800640 1 -- 172.21.15.119:0/3549722888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6af3670 msgr2=0x5639e6b21b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.582+0000 7f5005800640 1 --2- 172.21.15.119:0/3549722888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6af3670 0x5639e6b21b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.582+0000 7f5005800640 1 -- 172.21.15.119:0/3549722888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639e67413f0 con 0x5639e6ac8bb0 2024-04-25T10:09:07.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.582+0000 7f5005800640 1 --2- 172.21.15.119:0/3549722888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6ac8bb0 0x5639e6b07620 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400c040 tx=0x7f4ff4059380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c7c52dcda802fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.582+0000 7f4fe6a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:09:07.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.583+0000 7f4fe6a00640 1 -- 172.21.15.119:0/3549722888 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4055930 con 0x5639e6ac8bb0 2024-04-25T10:09:07.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.583+0000 7f500b024740 1 -- 172.21.15.119:0/3549722888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ff8004020 con 0x5639e6ac8bb0 2024-04-25T10:09:07.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.583+0000 7f4fe6a00640 1 -- 172.21.15.119:0/3549722888 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ff4059520 con 0x5639e6ac8bb0 2024-04-25T10:09:07.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.583+0000 7f4fe6a00640 1 -- 172.21.15.119:0/3549722888 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ff4055ad0 con 0x5639e6ac8bb0 2024-04-25T10:09:07.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.583+0000 7f500b024740 10 client.5697 fetch_fsmap learned FSMap version 25 2024-04-25T10:09:07.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.583+0000 7f500b024740 1 -- 172.21.15.119:0/3549722888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5639e67010f0 con 0x5639e6ac8bb0 2024-04-25T10:09:07.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.583+0000 7f4fe6a00640 1 -- 172.21.15.119:0/3549722888 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff8004020 con 0x5639e6ac8bb0 2024-04-25T10:09:07.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.583+0000 7f4fe6a00640 1 -- 172.21.15.119:0/3549722888 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4ff4061540 con 0x5639e6ac8bb0 2024-04-25T10:09:07.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.583+0000 7f500b024740 10 client.5697 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:09:07.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.583+0000 7f500b024740 10 client.5697 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:09:07.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.583+0000 7f500b024740 20 client.5697 populate_metadata read hostname 'smithi119' 2024-04-25T10:09:07.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.583+0000 7f500b024740 1 --2- 172.21.15.119:0/3549722888 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x5639e6af3a70 0x5639e6af3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.583+0000 7f500b024740 4 client.5697 mds_command: new command op to 5478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:07.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.583+0000 7f500b024740 1 -- 172.21.15.119:0/3549722888 --> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639e6a3a1b0 con 0x5639e6af3a70 2024-04-25T10:09:07.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.584+0000 7f5004e00640 1 --2- 172.21.15.119:0/3549722888 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x5639e6af3a70 0x5639e6af3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.584+0000 7f5004e00640 1 --2- 172.21.15.119:0/3549722888 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x5639e6af3a70 0x5639e6af3e80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c43393a71b8cf3c server_cookie=ab1b5c97f64d48a4 in_seq=0 out_seq=0 2024-04-25T10:09:07.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.584+0000 7f4fe6a00640 10 client.5697 ms_handle_connect on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:07.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.585+0000 7f4fe6a00640 1 -- 172.21.15.119:0/3549722888 <== mds.0 v2:172.21.15.119:6835/270766821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639e6a3a1b0 con 0x5639e6af3a70 2024-04-25T10:09:07.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.585+0000 7f4fe6a00640 10 client.5697 handle_command_reply: tid=0 2024-04-25T10:09:07.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.585+0000 7f500b024740 1 client.5697 shutdown 2024-04-25T10:09:07.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.585+0000 7f500b024740 1 -- 172.21.15.119:0/3549722888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6ac8bb0 msgr2=0x5639e6b07620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.585+0000 7f500b024740 1 --2- 172.21.15.119:0/3549722888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6ac8bb0 0x5639e6b07620 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400c040 tx=0x7f4ff4059380 comp rx=0 tx=0).stop 2024-04-25T10:09:07.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.585+0000 7f500b024740 1 -- 172.21.15.119:0/3549722888 shutdown_connections 2024-04-25T10:09:07.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.585+0000 7f500b024740 1 --2- 172.21.15.119:0/3549722888 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x5639e6af3a70 0x5639e6af3e80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.585+0000 7f500b024740 1 --2- 172.21.15.119:0/3549722888 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6b05280 0x5639e6ae1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.585+0000 7f500b024740 1 --2- 172.21.15.119:0/3549722888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6af3670 0x5639e6b21b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.585+0000 7f500b024740 1 --2- 172.21.15.119:0/3549722888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6ac8bb0 0x5639e6b07620 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.585+0000 7f4fe6a00640 0 client.5697 ms_handle_reset on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:07.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.585+0000 7f500b024740 1 -- 172.21.15.119:0/3549722888 >> 172.21.15.119:0/3549722888 conn(0x5639e6a90a30 msgr2=0x5639e6af0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.585+0000 7f500b024740 1 -- 172.21.15.119:0/3549722888 shutdown_connections 2024-04-25T10:09:07.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.585+0000 7f500b024740 1 -- 172.21.15.119:0/3549722888 wait complete. 2024-04-25T10:09:07.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.586+0000 7f500b024740 20 client.5697 trim_cache size 0 max 16384 2024-04-25T10:09:07.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.602+0000 7f500b024740 1 Processor -- start 2024-04-25T10:09:07.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.602+0000 7f500b024740 1 -- start start 2024-04-25T10:09:07.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.603+0000 7f500b024740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6ac8bb0 0x5639e6b093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.603+0000 7f500b024740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6af3670 0x5639e6b09e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.603+0000 7f500b024740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6b05280 0x5639e6b056c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.603+0000 7f500b024740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50001fffc0 con 0x5639e6ac8bb0 2024-04-25T10:09:07.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.603+0000 7f500b024740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50000ca540 con 0x5639e6b05280 2024-04-25T10:09:07.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.603+0000 7f500b024740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5000140b80 con 0x5639e6af3670 2024-04-25T10:09:07.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.603+0000 7f5005800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6ac8bb0 0x5639e6b093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.603+0000 7f5005800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6ac8bb0 0x5639e6b093c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38562/0 (socket says 172.21.15.119:38562) 2024-04-25T10:09:07.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.603+0000 7f5005800640 1 -- 172.21.15.119:0/3007848695 learned_addr learned my addr 172.21.15.119:0/3007848695 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:07.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.604+0000 7f5004e00640 1 --2- 172.21.15.119:0/3007848695 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6af3670 0x5639e6b09e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.604+0000 7f5006200640 1 --2- 172.21.15.119:0/3007848695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6b05280 0x5639e6b056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.604+0000 7f5005800640 1 -- 172.21.15.119:0/3007848695 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6af3670 msgr2=0x5639e6b09e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.604+0000 7f5005800640 1 --2- 172.21.15.119:0/3007848695 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6af3670 0x5639e6b09e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.604+0000 7f5005800640 1 -- 172.21.15.119:0/3007848695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6b05280 msgr2=0x5639e6b056c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.604+0000 7f5005800640 1 --2- 172.21.15.119:0/3007848695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6b05280 0x5639e6b056c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.604+0000 7f5005800640 1 -- 172.21.15.119:0/3007848695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639e6b12a70 con 0x5639e6ac8bb0 2024-04-25T10:09:07.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.604+0000 7f5006200640 1 --2- 172.21.15.119:0/3007848695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6b05280 0x5639e6b056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:07.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.604+0000 7f5005800640 1 --2- 172.21.15.119:0/3007848695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6ac8bb0 0x5639e6b093c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4059250 tx=0x7f4ff405d590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3054005c7478251c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.604+0000 7f4ff1600640 1 -- 172.21.15.119:0/3007848695 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4055930 con 0x5639e6ac8bb0 2024-04-25T10:09:07.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.604+0000 7f4ff1600640 1 -- 172.21.15.119:0/3007848695 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff4059520 con 0x5639e6ac8bb0 2024-04-25T10:09:07.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.604+0000 7f4ff1600640 1 -- 172.21.15.119:0/3007848695 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4061540 con 0x5639e6ac8bb0 2024-04-25T10:09:07.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.605+0000 7f500b024740 1 -- 172.21.15.119:0/3007848695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6ac8bb0 msgr2=0x5639e6b093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.605+0000 7f500b024740 1 --2- 172.21.15.119:0/3007848695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6ac8bb0 0x5639e6b093c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4059250 tx=0x7f4ff405d590 comp rx=0 tx=0).stop 2024-04-25T10:09:07.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.605+0000 7f500b024740 1 -- 172.21.15.119:0/3007848695 shutdown_connections 2024-04-25T10:09:07.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.605+0000 7f500b024740 1 --2- 172.21.15.119:0/3007848695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6b05280 0x5639e6b056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.605+0000 7f500b024740 1 --2- 172.21.15.119:0/3007848695 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6af3670 0x5639e6b09e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.605+0000 7f500b024740 1 --2- 172.21.15.119:0/3007848695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6ac8bb0 0x5639e6b093c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.605+0000 7f500b024740 1 -- 172.21.15.119:0/3007848695 >> 172.21.15.119:0/3007848695 conn(0x5639e6a90a30 msgr2=0x5639e6af3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.605+0000 7f500b024740 1 -- 172.21.15.119:0/3007848695 shutdown_connections 2024-04-25T10:09:07.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.605+0000 7f500b024740 1 -- 172.21.15.119:0/3007848695 wait complete. 2024-04-25T10:09:07.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.606+0000 7f500b024740 1 -- start start 2024-04-25T10:09:07.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.606+0000 7f500b024740 1 Processor -- start 2024-04-25T10:09:07.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.607+0000 7f500b024740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6ac8bb0 0x5639e6b81c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.607+0000 7f500b024740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6af3670 0x5639e6b08af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.607+0000 7f500b024740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6b05280 0x5639e6b09030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.607+0000 7f500b024740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50000c9f40 con 0x5639e6b05280 2024-04-25T10:09:07.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.607+0000 7f500b024740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e6ac9780 con 0x5639e6ac8bb0 2024-04-25T10:09:07.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.607+0000 7f500b024740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639e6b7bdb0 con 0x5639e6af3670 2024-04-25T10:09:07.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.607+0000 7f500b024740 10 client.0 _ll_register_callbacks cb 0x7f4ff400eef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:07.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.607+0000 7f5006200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6b05280 0x5639e6b09030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.607+0000 7f5006200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6b05280 0x5639e6b09030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38564/0 (socket says 172.21.15.119:38564) 2024-04-25T10:09:07.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.607+0000 7f5006200640 1 -- 172.21.15.119:0/3138066314 learned_addr learned my addr 172.21.15.119:0/3138066314 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:07.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.608+0000 7f5005800640 1 --2- 172.21.15.119:0/3138066314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6ac8bb0 0x5639e6b81c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.608+0000 7f5006200640 1 -- 172.21.15.119:0/3138066314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6af3670 msgr2=0x5639e6b08af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.608+0000 7f5006200640 1 --2- 172.21.15.119:0/3138066314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6af3670 0x5639e6b08af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.608+0000 7f5006200640 1 -- 172.21.15.119:0/3138066314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6ac8bb0 msgr2=0x5639e6b81c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.608+0000 7f5006200640 1 --2- 172.21.15.119:0/3138066314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6ac8bb0 0x5639e6b81c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.608+0000 7f5006200640 1 -- 172.21.15.119:0/3138066314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639e6b66ba0 con 0x5639e6b05280 2024-04-25T10:09:07.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.608+0000 7f5006200640 1 --2- 172.21.15.119:0/3138066314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6b05280 0x5639e6b09030 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8016040 tx=0x7f4ff8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2226d4d03e209a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.608+0000 7f4fe6a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:09:07.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.608+0000 7f4fe6a00640 1 -- 172.21.15.119:0/3138066314 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff800e9b0 con 0x5639e6b05280 2024-04-25T10:09:07.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.608+0000 7f4fe6a00640 1 -- 172.21.15.119:0/3138066314 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ff800eb50 con 0x5639e6b05280 2024-04-25T10:09:07.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.608+0000 7f500b024740 1 -- 172.21.15.119:0/3138066314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ff4059520 con 0x5639e6b05280 2024-04-25T10:09:07.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.609+0000 7f4fe6a00640 1 -- 172.21.15.119:0/3138066314 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ff800ed70 con 0x5639e6b05280 2024-04-25T10:09:07.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.609+0000 7f500b024740 10 client.5703 fetch_fsmap learned FSMap version 25 2024-04-25T10:09:07.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.609+0000 7f500b024740 1 -- 172.21.15.119:0/3138066314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5639e6af65f0 con 0x5639e6b05280 2024-04-25T10:09:07.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.609+0000 7f4fe6a00640 1 -- 172.21.15.119:0/3138066314 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff4059520 con 0x5639e6b05280 2024-04-25T10:09:07.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.609+0000 7f4fe6a00640 1 -- 172.21.15.119:0/3138066314 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4ff802e050 con 0x5639e6b05280 2024-04-25T10:09:07.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.609+0000 7f500b024740 10 client.5703 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:09:07.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.609+0000 7f500b024740 10 client.5703 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:09:07.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.609+0000 7f500b024740 20 client.5703 populate_metadata read hostname 'smithi119' 2024-04-25T10:09:07.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.609+0000 7f500b024740 1 --2- 172.21.15.119:0/3138066314 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x5639e6b82150 0x5639e6af5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.609+0000 7f500b024740 4 client.5703 mds_command: new command op to 5478 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:09:07.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.609+0000 7f500b024740 1 -- 172.21.15.119:0/3138066314 --> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5639e6a34500 con 0x5639e6b82150 2024-04-25T10:09:07.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.609+0000 7f5005800640 1 --2- 172.21.15.119:0/3138066314 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x5639e6b82150 0x5639e6af5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.610+0000 7f5005800640 1 --2- 172.21.15.119:0/3138066314 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x5639e6b82150 0x5639e6af5620 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=988feb993f2c0ee8 server_cookie=21385dde7254618b in_seq=0 out_seq=0 2024-04-25T10:09:07.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.610+0000 7f4fe6a00640 10 client.5703 ms_handle_connect on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:07.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.610+0000 7f4fe6a00640 1 -- 172.21.15.119:0/3138066314 <== mds.0 v2:172.21.15.119:6835/270766821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x5639e6a34500 con 0x5639e6b82150 2024-04-25T10:09:07.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.610+0000 7f4fe6a00640 10 client.5703 handle_command_reply: tid=0 2024-04-25T10:09:07.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.610+0000 7f500b024740 1 client.5703 shutdown 2024-04-25T10:09:07.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.610+0000 7f500b024740 1 -- 172.21.15.119:0/3138066314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6b05280 msgr2=0x5639e6b09030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.610+0000 7f500b024740 1 --2- 172.21.15.119:0/3138066314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6b05280 0x5639e6b09030 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8016040 tx=0x7f4ff8004730 comp rx=0 tx=0).stop 2024-04-25T10:09:07.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.610+0000 7f500b024740 1 -- 172.21.15.119:0/3138066314 shutdown_connections 2024-04-25T10:09:07.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.610+0000 7f500b024740 1 --2- 172.21.15.119:0/3138066314 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x5639e6b82150 0x5639e6af5620 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.610+0000 7f500b024740 1 --2- 172.21.15.119:0/3138066314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639e6b05280 0x5639e6b09030 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.610+0000 7f500b024740 1 --2- 172.21.15.119:0/3138066314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639e6af3670 0x5639e6b08af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.610+0000 7f4fe6a00640 0 client.5703 ms_handle_reset on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:07.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.610+0000 7f500b024740 1 --2- 172.21.15.119:0/3138066314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639e6ac8bb0 0x5639e6b81c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.610+0000 7f500b024740 1 -- 172.21.15.119:0/3138066314 >> 172.21.15.119:0/3138066314 conn(0x5639e6a90a30 msgr2=0x5639e6b07850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.611+0000 7f500b024740 1 -- 172.21.15.119:0/3138066314 shutdown_connections 2024-04-25T10:09:07.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.611+0000 7f500b024740 1 -- 172.21.15.119:0/3138066314 wait complete. 2024-04-25T10:09:07.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.611+0000 7f500b024740 20 client.5703 trim_cache size 0 max 16384 2024-04-25T10:09:07.621 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:09:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:09:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2024-04-25T10:09:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: "79c62faf-99ba-409c-ac46-2dac41f5f2e6": { 2024-04-25T10:09:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/", 2024-04-25T10:09:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "79c62faf-99ba-409c-ac46-2dac41f5f2e6", 2024-04-25T10:09:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,repair" 2024-04-25T10:09:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:07.622 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:09:07.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.611+0000 7f5008200640 1 -- 172.21.15.119:0/3676580162 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fe006b4d0 msgr2=0x7f4fe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.611+0000 7f5008200640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fe006b4d0 0x7f4fe006d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4fec034000 tx=0x7f4fec02e000 comp rx=0 tx=0).stop 2024-04-25T10:09:07.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.611+0000 7f5008200640 1 -- 172.21.15.119:0/3676580162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50000b42c0 msgr2=0x7f500013ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.611+0000 7f5008200640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50000b42c0 0x7f500013ddd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4007ef0 tx=0x7f4ff400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.612+0000 7f5008200640 1 -- 172.21.15.119:0/3676580162 shutdown_connections 2024-04-25T10:09:07.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.612+0000 7f5008200640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fe006b4d0 0x7f4fe006d990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.612+0000 7f5008200640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f500015aef0 0x7f50001f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.612+0000 7f5008200640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50000b46a0 0x7f50001f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.612+0000 7f5008200640 1 --2- 172.21.15.119:0/3676580162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50000b42c0 0x7f500013ddd0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.612+0000 7f5008200640 1 -- 172.21.15.119:0/3676580162 >> 172.21.15.119:0/3676580162 conn(0x7f500006aba0 msgr2=0x7f500015a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.612+0000 7f5008200640 1 -- 172.21.15.119:0/3676580162 shutdown_connections 2024-04-25T10:09:07.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:07.612+0000 7f5008200640 1 -- 172.21.15.119:0/3676580162 wait complete. 2024-04-25T10:09:07.635 INFO:tasks.cephfs.filesystem:scrub status for tag:79c62faf-99ba-409c-ac46-2dac41f5f2e6 - {'path': '/', 'tag': '79c62faf-99ba-409c-ac46-2dac41f5f2e6', 'options': 'recursive,repair'} 2024-04-25T10:09:08.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:08.732+0000 7f4ef4c00640 1 -- 172.21.15.119:0/3627293494 <== mon.0 v2:172.21.15.119:3300/0 7 ==== log(6 entries from seq 1 at 2024-04-25T10:09:07.367696+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f4ef0057d20 con 0x7f4ef815eed0 2024-04-25T10:09:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: cluster: 2024-04-25T10:09:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: id: e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:09:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: health: HEALTH_OK 2024-04-25T10:09:08.739 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: services: 2024-04-25T10:09:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-04-25T10:09:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: mgr: y(active, since 2m), standbys: x 2024-04-25T10:09:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:09:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-04-25T10:09:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: data: 2024-04-25T10:09:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: volumes: 1/1 healthy 2024-04-25T10:09:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:09:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: objects: 24 objects, 579 KiB 2024-04-25T10:09:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: usage: 220 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:09:08.740 INFO:teuthology.orchestra.run.smithi119.stdout: pgs: 97 active+clean 2024-04-25T10:09:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: io: 2024-04-25T10:09:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: client: 1.7 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-04-25T10:09:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:08.741 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:08.741 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:09:07.367696+0000 mds.a [INF] scrub queued for path: / 2024-04-25T10:09:08.741 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:09:07.367699+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:09:08.741 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:09:07.367704+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-25T10:09:08.741 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:09:07.369102+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-25T10:09:08.741 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:09:07.369115+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-25T10:09:08.741 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:09:07.369494+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:09:13.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:13.748+0000 7f4ef4c00640 1 -- 172.21.15.119:0/3627293494 <== mon.0 v2:172.21.15.119:3300/0 8 ==== log(1 entries from seq 7 at 2024-04-25T10:09:12.190093+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f4ef0057f80 con 0x7f4ef815eed0 2024-04-25T10:09:13.756 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:09:12.190093+0000 mds.a [INF] scrub summary: idle 2024-04-25T10:09:30.595 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:30.604 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:37.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:37.636 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:09:37.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.848+0000 7f0ca6600640 1 Processor -- start 2024-04-25T10:09:37.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.848+0000 7f0ca6600640 1 -- start start 2024-04-25T10:09:37.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.850+0000 7f0ca6600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca0144e70 0x7f0ca006d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.850+0000 7f0ca6600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ca00cf0e0 0x7f0ca00cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.850+0000 7f0ca6600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca006dcb0 con 0x7f0ca006d7a0 2024-04-25T10:09:37.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.850+0000 7f0ca6600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca015bfe0 con 0x7f0ca00cf0e0 2024-04-25T10:09:37.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.850+0000 7f0ca6600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca006df90 con 0x7f0ca0144e70 2024-04-25T10:09:37.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.850+0000 7f0c9ea00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0ca006d7a0 0x7f0ca00ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36732/0 (socket says 172.21.15.119:36732) 2024-04-25T10:09:37.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.850+0000 7f0c9ea00640 1 -- 172.21.15.119:0/2280453149 learned_addr learned my addr 172.21.15.119:0/2280453149 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:37.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.850+0000 7f0c9fe00640 1 --2- 172.21.15.119:0/2280453149 >> v2:172.21.15.119:3300/0 conn(0x7f0ca00cf0e0 0x7f0ca00cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.851+0000 7f0c9f400640 1 --2- 172.21.15.119:0/2280453149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca0144e70 0x7f0ca006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.851+0000 7f0c9e000640 1 -- 172.21.15.119:0/2280453149 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199949188 0 0) 0x7f0ca006dcb0 con 0x7f0ca006d7a0 2024-04-25T10:09:37.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.851+0000 7f0c9e000640 1 -- 172.21.15.119:0/2280453149 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c84003520 con 0x7f0ca006d7a0 2024-04-25T10:09:37.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.851+0000 7f0c9fe00640 1 -- 172.21.15.119:0/2280453149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca0144e70 msgr2=0x7f0ca006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:37.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.851+0000 7f0c9fe00640 1 --2- 172.21.15.119:0/2280453149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca0144e70 0x7f0ca006d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.851+0000 7f0c9fe00640 1 -- 172.21.15.119:0/2280453149 >> v1:172.21.15.119:6789/0 conn(0x7f0ca006d7a0 legacy=0x7f0ca00ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.851+0000 7f0c9fe00640 1 -- 172.21.15.119:0/2280453149 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca006e270 con 0x7f0ca00cf0e0 2024-04-25T10:09:37.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.852+0000 7f0c9fe00640 1 --2- 172.21.15.119:0/2280453149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ca00cf0e0 0x7f0ca00cc300 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009d90 tx=0x7f0c9802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a442f375bdf58ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:37.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.852+0000 7f0c9e000640 1 -- 172.21.15.119:0/2280453149 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98031600 con 0x7f0ca00cf0e0 2024-04-25T10:09:37.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.852+0000 7f0c9e000640 1 -- 172.21.15.119:0/2280453149 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c980317a0 con 0x7f0ca00cf0e0 2024-04-25T10:09:37.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.852+0000 7f0c9e000640 1 -- 172.21.15.119:0/2280453149 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98031b50 con 0x7f0ca00cf0e0 2024-04-25T10:09:37.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.853+0000 7f0ca6600640 1 -- 172.21.15.119:0/2280453149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ca00cf0e0 msgr2=0x7f0ca00cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.853+0000 7f0ca6600640 1 --2- 172.21.15.119:0/2280453149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ca00cf0e0 0x7f0ca00cc300 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009d90 tx=0x7f0c9802a5d0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.853+0000 7f0ca6600640 1 -- 172.21.15.119:0/2280453149 shutdown_connections 2024-04-25T10:09:37.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.853+0000 7f0ca6600640 1 --2- 172.21.15.119:0/2280453149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ca00cf0e0 0x7f0ca00cc300 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.853+0000 7f0ca6600640 1 --2- 172.21.15.119:0/2280453149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca0144e70 0x7f0ca006d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.853+0000 7f0ca6600640 1 -- 172.21.15.119:0/2280453149 >> 172.21.15.119:0/2280453149 conn(0x7f0ca006aba0 msgr2=0x7f0ca006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:37.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.853+0000 7f0ca6600640 1 -- 172.21.15.119:0/2280453149 shutdown_connections 2024-04-25T10:09:37.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.853+0000 7f0ca6600640 1 -- 172.21.15.119:0/2280453149 wait complete. 2024-04-25T10:09:37.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.855+0000 7f0ca6600640 1 Processor -- start 2024-04-25T10:09:37.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.855+0000 7f0ca6600640 1 -- start start 2024-04-25T10:09:37.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.856+0000 7f0ca6600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ca006d7a0 0x7f0ca01436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.856+0000 7f0ca6600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ca00cf0e0 0x7f0ca00afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.856+0000 7f0ca6600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca0144e70 0x7f0ca013bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.856+0000 7f0ca6600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca0145630 con 0x7f0ca006d7a0 2024-04-25T10:09:37.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.856+0000 7f0ca6600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01454b0 con 0x7f0ca00cf0e0 2024-04-25T10:09:37.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.856+0000 7f0ca6600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca00cc840 con 0x7f0ca0144e70 2024-04-25T10:09:37.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.856+0000 7f0c9f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ca006d7a0 0x7f0ca01436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.856+0000 7f0c9f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ca006d7a0 0x7f0ca01436e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54516/0 (socket says 172.21.15.119:54516) 2024-04-25T10:09:37.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.856+0000 7f0c9f400640 1 -- 172.21.15.119:0/1862278892 learned_addr learned my addr 172.21.15.119:0/1862278892 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:37.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.857+0000 7f0c9f400640 1 -- 172.21.15.119:0/1862278892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca0144e70 msgr2=0x7f0ca013bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.857+0000 7f0c9f400640 1 --2- 172.21.15.119:0/1862278892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca0144e70 0x7f0ca013bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.857+0000 7f0c9f400640 1 -- 172.21.15.119:0/1862278892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ca00cf0e0 msgr2=0x7f0ca00afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.857+0000 7f0c9f400640 1 --2- 172.21.15.119:0/1862278892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ca00cf0e0 0x7f0ca00afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.857+0000 7f0c9f400640 1 -- 172.21.15.119:0/1862278892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca015eb30 con 0x7f0ca006d7a0 2024-04-25T10:09:37.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.857+0000 7f0c9f400640 1 --2- 172.21.15.119:0/1862278892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ca006d7a0 0x7f0ca01436e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000d620 tx=0x7f0c9000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:37.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.857+0000 7f0c97400640 1 -- 172.21.15.119:0/1862278892 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000b620 con 0x7f0ca006d7a0 2024-04-25T10:09:37.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.857+0000 7f0c97400640 1 -- 172.21.15.119:0/1862278892 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c90019070 con 0x7f0ca006d7a0 2024-04-25T10:09:37.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.857+0000 7f0ca6600640 1 -- 172.21.15.119:0/1862278892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca015ddf0 con 0x7f0ca006d7a0 2024-04-25T10:09:37.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.857+0000 7f0c97400640 1 -- 172.21.15.119:0/1862278892 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000ba00 con 0x7f0ca006d7a0 2024-04-25T10:09:37.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.858+0000 7f0ca6600640 1 -- 172.21.15.119:0/1862278892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca013ca10 con 0x7f0ca006d7a0 2024-04-25T10:09:37.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.860+0000 7f0c97400640 1 -- 172.21.15.119:0/1862278892 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c9000bba0 con 0x7f0ca006d7a0 2024-04-25T10:09:37.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.861+0000 7f0c97400640 1 --2- 172.21.15.119:0/1862278892 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0c7c06b4d0 0x7f0c7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.861+0000 7f0c97400640 1 -- 172.21.15.119:0/1862278892 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c9008a560 con 0x7f0ca006d7a0 2024-04-25T10:09:37.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.861+0000 7f0c9ea00640 1 --2- 172.21.15.119:0/1862278892 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0c7c06b4d0 0x7f0c7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.861+0000 7f0ca941b740 1 Processor -- start 2024-04-25T10:09:37.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.862+0000 7f0ca941b740 1 -- start start 2024-04-25T10:09:37.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.862+0000 7f0c9ea00640 1 --2- 172.21.15.119:0/1862278892 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0c7c06b4d0 0x7f0c7c06d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0c88000f60 tx=0x7f0c880078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:37.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.862+0000 7f0ca941b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429decdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.862+0000 7f0ca941b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429df08280 0x56429decbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.862+0000 7f0ca941b740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429def6670 0x56429def6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.862+0000 7f0ca941b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429de2fc70 con 0x56429decbbb0 2024-04-25T10:09:37.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.862+0000 7f0ca941b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429ddd8cf0 con 0x56429df08280 2024-04-25T10:09:37.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.862+0000 7f0ca941b740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56429de2bd10 con 0x56429def6670 2024-04-25T10:09:37.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.863+0000 7f0c9f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429decdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.863+0000 7f0c9f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429decdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54520/0 (socket says 172.21.15.119:54520) 2024-04-25T10:09:37.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.863+0000 7f0c9f400640 1 -- 172.21.15.119:0/2044493426 learned_addr learned my addr 172.21.15.119:0/2044493426 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:37.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.863+0000 7f0c9f400640 1 -- 172.21.15.119:0/2044493426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429def6670 msgr2=0x56429def6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:37.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.863+0000 7f0c9f400640 1 --2- 172.21.15.119:0/2044493426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429def6670 0x56429def6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.863+0000 7f0c9f400640 1 -- 172.21.15.119:0/2044493426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429df08280 msgr2=0x56429decbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:37.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.863+0000 7f0c9f400640 1 --2- 172.21.15.119:0/2044493426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429df08280 0x56429decbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.863+0000 7f0c9f400640 1 -- 172.21.15.119:0/2044493426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56429dc0e7d0 con 0x56429decbbb0 2024-04-25T10:09:37.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.863+0000 7f0c9f400640 1 --2- 172.21.15.119:0/2044493426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429decdd10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0c90088ac0 tx=0x7f0c90058c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11dd0dc94828bd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:37.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.864+0000 7f0c8fe00640 1 -- 172.21.15.119:0/2044493426 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90061480 con 0x56429decbbb0 2024-04-25T10:09:37.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.864+0000 7f0c8fe00640 1 -- 172.21.15.119:0/2044493426 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c90057070 con 0x56429decbbb0 2024-04-25T10:09:37.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.864+0000 7f0c8fe00640 1 -- 172.21.15.119:0/2044493426 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c900553e0 con 0x56429decbbb0 2024-04-25T10:09:37.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.864+0000 7f0ca941b740 1 -- 172.21.15.119:0/2044493426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 msgr2=0x56429decdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.864+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2044493426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429decdd10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0c90088ac0 tx=0x7f0c90058c70 comp rx=0 tx=0).stop 2024-04-25T10:09:37.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.864+0000 7f0ca941b740 1 -- 172.21.15.119:0/2044493426 shutdown_connections 2024-04-25T10:09:37.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.864+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2044493426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429def6670 0x56429def6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.864+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2044493426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429df08280 0x56429decbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.864+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2044493426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429decdd10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.864+0000 7f0ca941b740 1 -- 172.21.15.119:0/2044493426 >> 172.21.15.119:0/2044493426 conn(0x56429de93a30 msgr2=0x56429df11670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:37.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.865+0000 7f0ca941b740 1 -- 172.21.15.119:0/2044493426 shutdown_connections 2024-04-25T10:09:37.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.865+0000 7f0ca941b740 1 -- 172.21.15.119:0/2044493426 wait complete. 2024-04-25T10:09:37.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.865+0000 7f0ca941b740 1 -- start start 2024-04-25T10:09:37.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.865+0000 7f0ca941b740 1 Processor -- start 2024-04-25T10:09:37.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.866+0000 7f0ca941b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429df0bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.866+0000 7f0ca941b740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429def6670 0x56429df12b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.866+0000 7f0ca941b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429df08280 0x56429dee3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.866+0000 7f0ca941b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429ddce3a0 con 0x56429decbbb0 2024-04-25T10:09:37.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.866+0000 7f0ca941b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429def41c0 con 0x56429df08280 2024-04-25T10:09:37.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.866+0000 7f0ca941b740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56429def35c0 con 0x56429def6670 2024-04-25T10:09:37.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.866+0000 7f0ca941b740 10 client.0 _ll_register_callbacks cb 0x56429ddebb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:37.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.867+0000 7f0c9f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429df0bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.867+0000 7f0c9f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429df0bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54536/0 (socket says 172.21.15.119:54536) 2024-04-25T10:09:37.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.867+0000 7f0c9f400640 1 -- 172.21.15.119:0/2887775795 learned_addr learned my addr 172.21.15.119:0/2887775795 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:37.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.867+0000 7f0c9f400640 1 -- 172.21.15.119:0/2887775795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429def6670 msgr2=0x56429df12b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:37.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.867+0000 7f0c9ea00640 1 --2- 172.21.15.119:0/2887775795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429def6670 0x56429df12b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.867+0000 7f0c9fe00640 1 --2- 172.21.15.119:0/2887775795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429df08280 0x56429dee3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.867+0000 7f0c9f400640 1 --2- 172.21.15.119:0/2887775795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429def6670 0x56429df12b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.867+0000 7f0c9f400640 1 -- 172.21.15.119:0/2887775795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429df08280 msgr2=0x56429dee3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:37.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.867+0000 7f0c9f400640 1 --2- 172.21.15.119:0/2887775795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429df08280 0x56429dee3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.867+0000 7f0c9f400640 1 -- 172.21.15.119:0/2887775795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56429db443f0 con 0x56429decbbb0 2024-04-25T10:09:37.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.867+0000 7f0c9ea00640 1 --2- 172.21.15.119:0/2887775795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429def6670 0x56429df12b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:37.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.867+0000 7f0c9f400640 1 --2- 172.21.15.119:0/2887775795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429df0bf40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0c90088b70 tx=0x7f0c90000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbd14505010e627 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:37.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.867+0000 7f0c8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:09:37.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.868+0000 7f0c8cc00640 1 -- 172.21.15.119:0/2887775795 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90088050 con 0x56429decbbb0 2024-04-25T10:09:37.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.868+0000 7f0c8cc00640 1 -- 172.21.15.119:0/2887775795 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c90060ff0 con 0x56429decbbb0 2024-04-25T10:09:37.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.868+0000 7f0ca941b740 1 -- 172.21.15.119:0/2887775795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c90057070 con 0x56429decbbb0 2024-04-25T10:09:37.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.868+0000 7f0c8cc00640 1 -- 172.21.15.119:0/2887775795 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c90061230 con 0x56429decbbb0 2024-04-25T10:09:37.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.868+0000 7f0ca941b740 10 client.5718 fetch_fsmap learned FSMap version 25 2024-04-25T10:09:37.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.868+0000 7f0ca941b740 1 -- 172.21.15.119:0/2887775795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x56429db040f0 con 0x56429decbbb0 2024-04-25T10:09:37.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.868+0000 7f0c8cc00640 1 -- 172.21.15.119:0/2887775795 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c90057070 con 0x56429decbbb0 2024-04-25T10:09:37.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.868+0000 7f0c8cc00640 1 -- 172.21.15.119:0/2887775795 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0c9007a030 con 0x56429decbbb0 2024-04-25T10:09:37.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.868+0000 7f0ca941b740 10 client.5718 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:09:37.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.868+0000 7f0ca941b740 10 client.5718 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:09:37.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.868+0000 7f0ca941b740 20 client.5718 populate_metadata read hostname 'smithi119' 2024-04-25T10:09:37.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.868+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2887775795 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x56429dee2130 0x56429dee4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.868+0000 7f0ca941b740 4 client.5718 mds_command: new command op to 5478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:37.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.868+0000 7f0ca941b740 1 -- 172.21.15.119:0/2887775795 --> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56429de3d1b0 con 0x56429dee2130 2024-04-25T10:09:37.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.869+0000 7f0c9ea00640 1 --2- 172.21.15.119:0/2887775795 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x56429dee2130 0x56429dee4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.869+0000 7f0c9ea00640 1 --2- 172.21.15.119:0/2887775795 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x56429dee2130 0x56429dee4690 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40dbcf59baa6057f server_cookie=2f1bed4f6882fb1a in_seq=0 out_seq=0 2024-04-25T10:09:37.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.869+0000 7f0c8cc00640 10 client.5718 ms_handle_connect on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:37.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.870+0000 7f0c8cc00640 1 -- 172.21.15.119:0/2887775795 <== mds.0 v2:172.21.15.119:6835/270766821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56429de3d1b0 con 0x56429dee2130 2024-04-25T10:09:37.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.870+0000 7f0c8cc00640 10 client.5718 handle_command_reply: tid=0 2024-04-25T10:09:37.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.870+0000 7f0ca941b740 1 client.5718 shutdown 2024-04-25T10:09:37.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.870+0000 7f0ca941b740 1 -- 172.21.15.119:0/2887775795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 msgr2=0x56429df0bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.870+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2887775795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429df0bf40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0c90088b70 tx=0x7f0c90000bc0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.870+0000 7f0ca941b740 1 -- 172.21.15.119:0/2887775795 shutdown_connections 2024-04-25T10:09:37.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.870+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2887775795 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x56429dee2130 0x56429dee4690 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.870+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2887775795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429df08280 0x56429dee3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.870+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2887775795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429def6670 0x56429df12b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.870+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2887775795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429df0bf40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.870+0000 7f0c8cc00640 0 client.5718 ms_handle_reset on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:37.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.870+0000 7f0ca941b740 1 -- 172.21.15.119:0/2887775795 >> 172.21.15.119:0/2887775795 conn(0x56429de93a30 msgr2=0x56429def3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:37.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.870+0000 7f0ca941b740 1 -- 172.21.15.119:0/2887775795 shutdown_connections 2024-04-25T10:09:37.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.870+0000 7f0ca941b740 1 -- 172.21.15.119:0/2887775795 wait complete. 2024-04-25T10:09:37.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.871+0000 7f0ca941b740 20 client.5718 trim_cache size 0 max 16384 2024-04-25T10:09:37.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.887+0000 7f0ca941b740 1 Processor -- start 2024-04-25T10:09:37.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.887+0000 7f0ca941b740 1 -- start start 2024-04-25T10:09:37.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.888+0000 7f0ca941b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429def3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.888+0000 7f0ca941b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429def6670 0x56429df33dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.888+0000 7f0ca941b740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429df08280 0x56429def55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.888+0000 7f0ca941b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429df2b900 con 0x56429decbbb0 2024-04-25T10:09:37.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.888+0000 7f0ca941b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01ffcc0 con 0x56429def6670 2024-04-25T10:09:37.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.888+0000 7f0ca941b740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01f7b70 con 0x56429df08280 2024-04-25T10:09:37.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.888+0000 7f0c9f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429def3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.888+0000 7f0c9f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429def3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54540/0 (socket says 172.21.15.119:54540) 2024-04-25T10:09:37.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.888+0000 7f0c9f400640 1 -- 172.21.15.119:0/1495367342 learned_addr learned my addr 172.21.15.119:0/1495367342 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:37.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.888+0000 7f0c9f400640 1 -- 172.21.15.119:0/1495367342 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429df08280 msgr2=0x56429def55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:37.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.888+0000 7f0c9f400640 1 --2- 172.21.15.119:0/1495367342 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429df08280 0x56429def55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.888+0000 7f0c9f400640 1 -- 172.21.15.119:0/1495367342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429def6670 msgr2=0x56429df33dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:37.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.888+0000 7f0c9f400640 1 --2- 172.21.15.119:0/1495367342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429def6670 0x56429df33dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.888+0000 7f0c9f400640 1 -- 172.21.15.119:0/1495367342 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56429df0ab10 con 0x56429decbbb0 2024-04-25T10:09:37.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.889+0000 7f0c9f400640 1 --2- 172.21.15.119:0/1495367342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429def3360 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0c90058a70 tx=0x7f0c90084070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1064047c6b3553f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:37.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.889+0000 7f0c8f400640 1 -- 172.21.15.119:0/1495367342 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90057070 con 0x56429decbbb0 2024-04-25T10:09:37.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.889+0000 7f0c8f400640 1 -- 172.21.15.119:0/1495367342 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c90075070 con 0x56429decbbb0 2024-04-25T10:09:37.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.889+0000 7f0c8f400640 1 -- 172.21.15.119:0/1495367342 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c900553e0 con 0x56429decbbb0 2024-04-25T10:09:37.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.889+0000 7f0ca941b740 1 -- 172.21.15.119:0/1495367342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 msgr2=0x56429def3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.889+0000 7f0ca941b740 1 --2- 172.21.15.119:0/1495367342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429def3360 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0c90058a70 tx=0x7f0c90084070 comp rx=0 tx=0).stop 2024-04-25T10:09:37.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.890+0000 7f0ca941b740 1 -- 172.21.15.119:0/1495367342 shutdown_connections 2024-04-25T10:09:37.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.890+0000 7f0ca941b740 1 --2- 172.21.15.119:0/1495367342 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429df08280 0x56429def55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.890+0000 7f0ca941b740 1 --2- 172.21.15.119:0/1495367342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429def6670 0x56429df33dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.890+0000 7f0ca941b740 1 --2- 172.21.15.119:0/1495367342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429decbbb0 0x56429def3360 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.890+0000 7f0ca941b740 1 -- 172.21.15.119:0/1495367342 >> 172.21.15.119:0/1495367342 conn(0x56429de93a30 msgr2=0x56429dec9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:37.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.890+0000 7f0ca941b740 1 -- 172.21.15.119:0/1495367342 shutdown_connections 2024-04-25T10:09:37.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.890+0000 7f0ca941b740 1 -- 172.21.15.119:0/1495367342 wait complete. 2024-04-25T10:09:37.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.891+0000 7f0ca941b740 1 -- start start 2024-04-25T10:09:37.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.891+0000 7f0ca941b740 1 Processor -- start 2024-04-25T10:09:37.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.892+0000 7f0ca941b740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429decbbb0 0x56429df7b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.892+0000 7f0ca941b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429def6670 0x56429def4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.892+0000 7f0ca941b740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429df08280 0x56429def4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.892+0000 7f0ca941b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01f7570 con 0x56429def6670 2024-04-25T10:09:37.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.892+0000 7f0ca941b740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca01f6f70 con 0x56429df08280 2024-04-25T10:09:37.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.892+0000 7f0ca941b740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56429dec2430 con 0x56429decbbb0 2024-04-25T10:09:37.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.892+0000 7f0ca941b740 10 client.0 _ll_register_callbacks cb 0x7f0c900588b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:37.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.892+0000 7f0c9ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429def6670 0x56429def4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.892+0000 7f0c9ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429def6670 0x56429def4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54554/0 (socket says 172.21.15.119:54554) 2024-04-25T10:09:37.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.892+0000 7f0c9ea00640 1 -- 172.21.15.119:0/2067174614 learned_addr learned my addr 172.21.15.119:0/2067174614 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:37.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.893+0000 7f0c9ea00640 1 -- 172.21.15.119:0/2067174614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429decbbb0 msgr2=0x56429df7b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:37.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.893+0000 7f0c9ea00640 1 --2- 172.21.15.119:0/2067174614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429decbbb0 0x56429df7b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.893+0000 7f0c9ea00640 1 -- 172.21.15.119:0/2067174614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429df08280 msgr2=0x56429def4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:37.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.893+0000 7f0c9ea00640 1 --2- 172.21.15.119:0/2067174614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429df08280 0x56429def4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.893+0000 7f0c9ea00640 1 -- 172.21.15.119:0/2067174614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56429df0b4e0 con 0x56429def6670 2024-04-25T10:09:37.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.893+0000 7f0c9ea00640 1 --2- 172.21.15.119:0/2067174614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429def6670 0x56429def4a60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0ca006dbe0 tx=0x7f0c88005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a678fb1c0ac5cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:37.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.893+0000 7f0c8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:09:37.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.893+0000 7f0c8cc00640 1 -- 172.21.15.119:0/2067174614 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c880205c0 con 0x56429def6670 2024-04-25T10:09:37.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.893+0000 7f0c8cc00640 1 -- 172.21.15.119:0/2067174614 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c88020760 con 0x56429def6670 2024-04-25T10:09:37.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.893+0000 7f0ca941b740 1 -- 172.21.15.119:0/2067174614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c90075070 con 0x56429def6670 2024-04-25T10:09:37.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.893+0000 7f0c8cc00640 1 -- 172.21.15.119:0/2067174614 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c8802c440 con 0x56429def6670 2024-04-25T10:09:37.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.893+0000 7f0ca941b740 10 client.5724 fetch_fsmap learned FSMap version 25 2024-04-25T10:09:37.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.893+0000 7f0ca941b740 1 -- 172.21.15.119:0/2067174614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x56429df12f30 con 0x56429def6670 2024-04-25T10:09:37.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.894+0000 7f0c8cc00640 1 -- 172.21.15.119:0/2067174614 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c90075070 con 0x56429def6670 2024-04-25T10:09:37.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.894+0000 7f0c8cc00640 1 -- 172.21.15.119:0/2067174614 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0c88029060 con 0x56429def6670 2024-04-25T10:09:37.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.894+0000 7f0ca941b740 10 client.5724 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T10:09:37.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.894+0000 7f0ca941b740 10 client.5724 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:09:37.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.894+0000 7f0ca941b740 20 client.5724 populate_metadata read hostname 'smithi119' 2024-04-25T10:09:37.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.894+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2067174614 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x56429df0c030 0x56429dedfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:37.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.894+0000 7f0ca941b740 4 client.5724 mds_command: new command op to 5478 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:09:37.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.894+0000 7f0ca941b740 1 -- 172.21.15.119:0/2067174614 --> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56429de37500 con 0x56429df0c030 2024-04-25T10:09:37.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.894+0000 7f0c9f400640 1 --2- 172.21.15.119:0/2067174614 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x56429df0c030 0x56429dedfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:37.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0c9f400640 1 --2- 172.21.15.119:0/2067174614 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x56429df0c030 0x56429dedfc20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f989526be6e2e951 server_cookie=aa424abea36bd266 in_seq=0 out_seq=0 2024-04-25T10:09:37.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0c8cc00640 10 client.5724 ms_handle_connect on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:37.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0c8cc00640 1 -- 172.21.15.119:0/2067174614 <== mds.0 v2:172.21.15.119:6835/270766821 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56429de37500 con 0x56429df0c030 2024-04-25T10:09:37.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0c8cc00640 10 client.5724 handle_command_reply: tid=0 2024-04-25T10:09:37.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0ca941b740 1 client.5724 shutdown 2024-04-25T10:09:37.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0ca941b740 1 -- 172.21.15.119:0/2067174614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429def6670 msgr2=0x56429def4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2067174614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429def6670 0x56429def4a60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0ca006dbe0 tx=0x7f0c88005ae0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0ca941b740 1 -- 172.21.15.119:0/2067174614 shutdown_connections 2024-04-25T10:09:37.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2067174614 >> [v2:172.21.15.119:6835/270766821,v1:172.21.15.119:6837/270766821] conn(0x56429df0c030 0x56429dedfc20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2067174614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56429df08280 0x56429def4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2067174614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56429def6670 0x56429def4a60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0ca941b740 1 --2- 172.21.15.119:0/2067174614 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56429decbbb0 0x56429df7b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0ca941b740 1 -- 172.21.15.119:0/2067174614 >> 172.21.15.119:0/2067174614 conn(0x56429de93a30 msgr2=0x56429df11920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:37.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0c8cc00640 0 client.5724 ms_handle_reset on v2:172.21.15.119:6835/270766821 2024-04-25T10:09:37.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0ca941b740 1 -- 172.21.15.119:0/2067174614 shutdown_connections 2024-04-25T10:09:37.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0ca941b740 1 -- 172.21.15.119:0/2067174614 wait complete. 2024-04-25T10:09:37.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.895+0000 7f0ca941b740 20 client.5724 trim_cache size 0 max 16384 2024-04-25T10:09:37.905 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:09:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2024-04-25T10:09:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2024-04-25T10:09:37.906 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:09:37.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.896+0000 7f0ca6600640 1 -- 172.21.15.119:0/1862278892 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0c7c06b4d0 msgr2=0x7f0c7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.896+0000 7f0ca6600640 1 --2- 172.21.15.119:0/1862278892 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0c7c06b4d0 0x7f0c7c06d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0c88000f60 tx=0x7f0c880078f0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.896+0000 7f0ca6600640 1 -- 172.21.15.119:0/1862278892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ca006d7a0 msgr2=0x7f0ca01436e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:37.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.896+0000 7f0ca6600640 1 --2- 172.21.15.119:0/1862278892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ca006d7a0 0x7f0ca01436e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000d620 tx=0x7f0c9000dae0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.896+0000 7f0ca6600640 1 -- 172.21.15.119:0/1862278892 shutdown_connections 2024-04-25T10:09:37.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.896+0000 7f0ca6600640 1 --2- 172.21.15.119:0/1862278892 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0c7c06b4d0 0x7f0c7c06d990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.896+0000 7f0ca6600640 1 --2- 172.21.15.119:0/1862278892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ca0144e70 0x7f0ca013bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.896+0000 7f0ca6600640 1 --2- 172.21.15.119:0/1862278892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ca00cf0e0 0x7f0ca00afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.896+0000 7f0ca6600640 1 --2- 172.21.15.119:0/1862278892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ca006d7a0 0x7f0ca01436e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:37.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.896+0000 7f0ca6600640 1 -- 172.21.15.119:0/1862278892 >> 172.21.15.119:0/1862278892 conn(0x7f0ca006aba0 msgr2=0x7f0ca006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:37.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.896+0000 7f0ca6600640 1 -- 172.21.15.119:0/1862278892 shutdown_connections 2024-04-25T10:09:37.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:37.896+0000 7f0ca6600640 1 -- 172.21.15.119:0/1862278892 wait complete. 2024-04-25T10:09:37.923 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:09:38.024 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:38.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.224+0000 7f3975c00640 1 Processor -- start 2024-04-25T10:09:38.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.225+0000 7f3975c00640 1 -- start start 2024-04-25T10:09:38.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.226+0000 7f3975c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f397015dce0 0x7f39700d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.226+0000 7f3975c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397015e0c0 con 0x7f39700b5b60 2024-04-25T10:09:38.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.226+0000 7f3975c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39700d2510 con 0x7f39700d35b0 2024-04-25T10:09:38.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.226+0000 7f3975c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39700d27f0 con 0x7f397015dce0 2024-04-25T10:09:38.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.226+0000 7f396f400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f39700b5b60 0x7f39700b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36746/0 (socket says 172.21.15.119:36746) 2024-04-25T10:09:38.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.226+0000 7f396f400640 1 -- 172.21.15.119:0/1591356387 learned_addr learned my addr 172.21.15.119:0/1591356387 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:38.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.226+0000 7f396ea00640 1 --2- 172.21.15.119:0/1591356387 >> v2:172.21.15.132:3300/0 conn(0x7f397015dce0 0x7f39700d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 -- 172.21.15.119:0/1591356387 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629831151 0 0) 0x7f397015e0c0 con 0x7f39700b5b60 2024-04-25T10:09:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 -- 172.21.15.119:0/1591356387 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3954003520 con 0x7f39700b5b60 2024-04-25T10:09:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 -- 172.21.15.119:0/1591356387 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3603649040 0 0) 0x7f3954003520 con 0x7f39700b5b60 2024-04-25T10:09:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 -- 172.21.15.119:0/1591356387 >> v2:172.21.15.132:3300/0 conn(0x7f397015dce0 msgr2=0x7f39700d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 --2- 172.21.15.119:0/1591356387 >> v2:172.21.15.132:3300/0 conn(0x7f397015dce0 0x7f39700d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 -- 172.21.15.119:0/1591356387 >> v1:172.21.15.132:6789/0 conn(0x7f39700d35b0 legacy=0x7f39700b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 -- 172.21.15.119:0/1591356387 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39700d2ad0 con 0x7f39700b5b60 2024-04-25T10:09:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 -- 172.21.15.119:0/1591356387 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f39680019f0 con 0x7f39700b5b60 2024-04-25T10:09:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 -- 172.21.15.119:0/1591356387 >> v1:172.21.15.119:6789/0 conn(0x7f39700b5b60 legacy=0x7f39700b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 --2- 172.21.15.119:0/1591356387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39540039b0 0x7f3954003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 --2- 172.21.15.119:0/1591356387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3954004800 0x7f3954004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 --2- 172.21.15.119:0/1591356387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540051c0 0x7f395400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 -- 172.21.15.119:0/1591356387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39700d27f0 con 0x7f39540039b0 2024-04-25T10:09:38.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 -- 172.21.15.119:0/1591356387 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395400c060 con 0x7f39540051c0 2024-04-25T10:09:38.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.227+0000 7f396e000640 1 -- 172.21.15.119:0/1591356387 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395400c260 con 0x7f3954004800 2024-04-25T10:09:38.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.228+0000 7f396ea00640 1 --2- 172.21.15.119:0/1591356387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39540039b0 0x7f3954003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.228+0000 7f396f400640 1 --2- 172.21.15.119:0/1591356387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540051c0 0x7f395400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.228+0000 7f3966000640 1 --2- 172.21.15.119:0/1591356387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3954004800 0x7f3954004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.228+0000 7f396ea00640 1 -- 172.21.15.119:0/1591356387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3954004800 msgr2=0x7f3954004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:38.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.228+0000 7f396ea00640 1 --2- 172.21.15.119:0/1591356387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3954004800 0x7f3954004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.228+0000 7f396ea00640 1 -- 172.21.15.119:0/1591356387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540051c0 msgr2=0x7f395400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:38.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.228+0000 7f396ea00640 1 --2- 172.21.15.119:0/1591356387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540051c0 0x7f395400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.228+0000 7f396ea00640 1 -- 172.21.15.119:0/1591356387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3954003520 con 0x7f39540039b0 2024-04-25T10:09:38.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.228+0000 7f396f400640 1 -- 172.21.15.119:0/1591356387 reap_dead start 2024-04-25T10:09:38.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.228+0000 7f3966000640 1 --2- 172.21.15.119:0/1591356387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3954004800 0x7f3954004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.229+0000 7f396ea00640 1 --2- 172.21.15.119:0/1591356387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39540039b0 0x7f3954003dd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3960009ac0 tx=0x7f3954005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c562271449026b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.229+0000 7f396e000640 1 -- 172.21.15.119:0/1591356387 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960036070 con 0x7f39540039b0 2024-04-25T10:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.229+0000 7f396e000640 1 -- 172.21.15.119:0/1591356387 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3960031400 con 0x7f39540039b0 2024-04-25T10:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.229+0000 7f396e000640 1 -- 172.21.15.119:0/1591356387 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960009690 con 0x7f39540039b0 2024-04-25T10:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.230+0000 7f3975c00640 1 -- 172.21.15.119:0/1591356387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39540039b0 msgr2=0x7f3954003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.230+0000 7f3975c00640 1 --2- 172.21.15.119:0/1591356387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39540039b0 0x7f3954003dd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3960009ac0 tx=0x7f3954005830 comp rx=0 tx=0).stop 2024-04-25T10:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.230+0000 7f3975c00640 1 -- 172.21.15.119:0/1591356387 shutdown_connections 2024-04-25T10:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.230+0000 7f3975c00640 1 --2- 172.21.15.119:0/1591356387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39540039b0 0x7f3954003dd0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.230+0000 7f3975c00640 1 -- 172.21.15.119:0/1591356387 >> 172.21.15.119:0/1591356387 conn(0x7f3970072900 msgr2=0x7f39700d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.230+0000 7f3975c00640 1 -- 172.21.15.119:0/1591356387 shutdown_connections 2024-04-25T10:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.230+0000 7f3975c00640 1 -- 172.21.15.119:0/1591356387 wait complete. 2024-04-25T10:09:38.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.232+0000 7f3975c00640 1 Processor -- start 2024-04-25T10:09:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.232+0000 7f3975c00640 1 -- start start 2024-04-25T10:09:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.233+0000 7f3975c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540039b0 0x7f397015bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.233+0000 7f3975c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39701f4880 0x7f39701f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.233+0000 7f3975c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39700ab2a0 0x7f39700ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.233+0000 7f3975c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39700b9820 con 0x7f39700ab2a0 2024-04-25T10:09:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.233+0000 7f3975c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39700d3090 con 0x7f39540039b0 2024-04-25T10:09:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.233+0000 7f3975c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39700b99a0 con 0x7f39701f4880 2024-04-25T10:09:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.234+0000 7f396f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39700ab2a0 0x7f39700ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.234+0000 7f396f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39700ab2a0 0x7f39700ab750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54566/0 (socket says 172.21.15.119:54566) 2024-04-25T10:09:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.234+0000 7f396f400640 1 -- 172.21.15.119:0/104565884 learned_addr learned my addr 172.21.15.119:0/104565884 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.234+0000 7f396ea00640 1 --2- 172.21.15.119:0/104565884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540039b0 0x7f397015bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.234+0000 7f3966000640 1 --2- 172.21.15.119:0/104565884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39701f4880 0x7f39701f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.234+0000 7f396f400640 1 -- 172.21.15.119:0/104565884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39701f4880 msgr2=0x7f39701f4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.234+0000 7f396f400640 1 --2- 172.21.15.119:0/104565884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39701f4880 0x7f39701f4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.234+0000 7f396f400640 1 -- 172.21.15.119:0/104565884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540039b0 msgr2=0x7f397015bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.234+0000 7f396f400640 1 --2- 172.21.15.119:0/104565884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540039b0 0x7f397015bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.234+0000 7f396f400640 1 -- 172.21.15.119:0/104565884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39700aa4e0 con 0x7f39700ab2a0 2024-04-25T10:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.234+0000 7f396f400640 1 --2- 172.21.15.119:0/104565884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39700ab2a0 0x7f39700ab750 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f397008ce80 tx=0x7f3968007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.235+0000 7f3967400640 1 -- 172.21.15.119:0/104565884 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396800eb00 con 0x7f39700ab2a0 2024-04-25T10:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.235+0000 7f3967400640 1 -- 172.21.15.119:0/104565884 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3968018070 con 0x7f39700ab2a0 2024-04-25T10:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.235+0000 7f3967400640 1 -- 172.21.15.119:0/104565884 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968013420 con 0x7f39700ab2a0 2024-04-25T10:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.235+0000 7f3975c00640 1 -- 172.21.15.119:0/104565884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39700aa7c0 con 0x7f39700ab2a0 2024-04-25T10:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.235+0000 7f3975c00640 1 -- 172.21.15.119:0/104565884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39700aad80 con 0x7f39700ab2a0 2024-04-25T10:09:38.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.236+0000 7f3975c00640 1 -- 172.21.15.119:0/104565884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39700749b0 con 0x7f39700ab2a0 2024-04-25T10:09:38.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.237+0000 7f3967400640 1 -- 172.21.15.119:0/104565884 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39680135c0 con 0x7f39700ab2a0 2024-04-25T10:09:38.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.238+0000 7f3967400640 1 --2- 172.21.15.119:0/104565884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f394c06b550 0x7f394c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.238+0000 7f3967400640 1 -- 172.21.15.119:0/104565884 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f396808a3e0 con 0x7f39700ab2a0 2024-04-25T10:09:38.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.238+0000 7f396ea00640 1 --2- 172.21.15.119:0/104565884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f394c06b550 0x7f394c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.239+0000 7f396ea00640 1 --2- 172.21.15.119:0/104565884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f394c06b550 0x7f394c06da10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3960033000 tx=0x7f3960008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.243+0000 7f3967400640 1 -- 172.21.15.119:0/104565884 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f39680592d0 con 0x7f39700ab2a0 2024-04-25T10:09:38.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.474+0000 7f3975c00640 1 -- 172.21.15.119:0/104565884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39700b2de0 con 0x7f39700ab2a0 2024-04-25T10:09:38.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.475+0000 7f3967400640 1 -- 172.21.15.119:0/104565884 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f3968058bd0 con 0x7f39700ab2a0 2024-04-25T10:09:38.482 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:38.482 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3317744182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3317744182},{"type":"v1","addr":"172.21.15.132:6833","nonce":3317744182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/700427287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":700427287},{"type":"v1","addr":"172.21.15.132:6835","nonce":700427287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2207174972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2207174972},{"type":"v1","addr":"172.21.15.119:6836","nonce":2207174972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:57.120451+0000","modified":"2024-04-25T10:09:00.157396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5478":{"gid":5478,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/270766821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":270766821},{"type":"v1","addr":"172.21.15.119:6837","nonce":270766821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:09:38.482 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 25 2024-04-25T10:09:38.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.478+0000 7f3975c00640 1 -- 172.21.15.119:0/104565884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f394c06b550 msgr2=0x7f394c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.478+0000 7f3975c00640 1 --2- 172.21.15.119:0/104565884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f394c06b550 0x7f394c06da10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3960033000 tx=0x7f3960008030 comp rx=0 tx=0).stop 2024-04-25T10:09:38.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.478+0000 7f3975c00640 1 -- 172.21.15.119:0/104565884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39700ab2a0 msgr2=0x7f39700ab750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.478+0000 7f3975c00640 1 --2- 172.21.15.119:0/104565884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39700ab2a0 0x7f39700ab750 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f397008ce80 tx=0x7f3968007bc0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.478+0000 7f3975c00640 1 -- 172.21.15.119:0/104565884 shutdown_connections 2024-04-25T10:09:38.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.478+0000 7f3975c00640 1 --2- 172.21.15.119:0/104565884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f394c06b550 0x7f394c06da10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.478+0000 7f3975c00640 1 --2- 172.21.15.119:0/104565884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39700ab2a0 0x7f39700ab750 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.478+0000 7f3975c00640 1 --2- 172.21.15.119:0/104565884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39701f4880 0x7f39701f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.478+0000 7f3975c00640 1 --2- 172.21.15.119:0/104565884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39540039b0 0x7f397015bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.478+0000 7f3975c00640 1 -- 172.21.15.119:0/104565884 >> 172.21.15.119:0/104565884 conn(0x7f3970072900 msgr2=0x7f39700d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:38.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.478+0000 7f3975c00640 1 -- 172.21.15.119:0/104565884 shutdown_connections 2024-04-25T10:09:38.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.478+0000 7f3975c00640 1 -- 172.21.15.119:0/104565884 wait complete. 2024-04-25T10:09:38.503 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:09:38.737 INFO:teuthology.orchestra.run.smithi119.stdout:{"message":"","return_code":0} 2024-04-25T10:09:38.739 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:09:38.739 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:09:38.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.838+0000 7f871c434f40 1 Processor -- start 2024-04-25T10:09:38.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.839+0000 7f871c434f40 1 -- start start 2024-04-25T10:09:38.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.839+0000 7f871c434f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87000030f0 0x55dabe9a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.839+0000 7f871c434f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dabe99e820 0x55dabe8fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.839+0000 7f871c434f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabe998790 0x55dabe998c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.839+0000 7f871c434f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabe872dd0 con 0x55dabe99e820 2024-04-25T10:09:38.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.839+0000 7f871c434f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabe876d30 con 0x55dabe998790 2024-04-25T10:09:38.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.839+0000 7f871c434f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dabe9a21d0 con 0x7f87000030f0 2024-04-25T10:09:38.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.840+0000 7f8718c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dabe99e820 0x55dabe8fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.840+0000 7f8718c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dabe99e820 0x55dabe8fbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54580/0 (socket says 172.21.15.119:54580) 2024-04-25T10:09:38.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.840+0000 7f8718c00640 1 -- 172.21.15.119:0/3455649418 learned_addr learned my addr 172.21.15.119:0/3455649418 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:38.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.840+0000 7f8719600640 1 --2- 172.21.15.119:0/3455649418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87000030f0 0x55dabe9a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.840+0000 7f871a000640 1 --2- 172.21.15.119:0/3455649418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabe998790 0x55dabe998c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.840+0000 7f8718c00640 1 -- 172.21.15.119:0/3455649418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87000030f0 msgr2=0x55dabe9a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:38.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.840+0000 7f8718c00640 1 --2- 172.21.15.119:0/3455649418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87000030f0 0x55dabe9a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.840+0000 7f8718c00640 1 -- 172.21.15.119:0/3455649418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabe998790 msgr2=0x55dabe998c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:38.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.840+0000 7f8718c00640 1 --2- 172.21.15.119:0/3455649418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabe998790 0x55dabe998c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.840+0000 7f8718c00640 1 -- 172.21.15.119:0/3455649418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dabe8cd4a0 con 0x55dabe99e820 2024-04-25T10:09:38.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.841+0000 7f8718c00640 1 --2- 172.21.15.119:0/3455649418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dabe99e820 0x55dabe8fbde0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8704025780 tx=0x7f870404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7f7ebe6eb982df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.841+0000 7f8713400640 1 -- 172.21.15.119:0/3455649418 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704067070 con 0x55dabe99e820 2024-04-25T10:09:38.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.841+0000 7f8713400640 1 -- 172.21.15.119:0/3455649418 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8704054810 con 0x55dabe99e820 2024-04-25T10:09:38.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.841+0000 7f8713400640 1 -- 172.21.15.119:0/3455649418 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704059040 con 0x55dabe99e820 2024-04-25T10:09:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.842+0000 7f871c434f40 1 -- 172.21.15.119:0/3455649418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dabe99e820 msgr2=0x55dabe8fbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.842+0000 7f871c434f40 1 --2- 172.21.15.119:0/3455649418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dabe99e820 0x55dabe8fbde0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8704025780 tx=0x7f870404e000 comp rx=0 tx=0).stop 2024-04-25T10:09:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.842+0000 7f871c434f40 1 -- 172.21.15.119:0/3455649418 shutdown_connections 2024-04-25T10:09:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.842+0000 7f871c434f40 1 --2- 172.21.15.119:0/3455649418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabe998790 0x55dabe998c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.842+0000 7f871c434f40 1 --2- 172.21.15.119:0/3455649418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dabe99e820 0x55dabe8fbde0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.842+0000 7f871c434f40 1 --2- 172.21.15.119:0/3455649418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87000030f0 0x55dabe9a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.842+0000 7f871c434f40 1 -- 172.21.15.119:0/3455649418 >> 172.21.15.119:0/3455649418 conn(0x55dabe8ce980 msgr2=0x55dabe8fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.842+0000 7f871c434f40 1 -- 172.21.15.119:0/3455649418 shutdown_connections 2024-04-25T10:09:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.842+0000 7f871c434f40 1 -- 172.21.15.119:0/3455649418 wait complete. 2024-04-25T10:09:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.843+0000 7f871c434f40 1 Processor -- start 2024-04-25T10:09:38.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.843+0000 7f871c434f40 1 -- start start 2024-04-25T10:09:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.844+0000 7f871c434f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dabe998790 0x55dabea3d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.844+0000 7f871c434f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabe99e820 0x55dabea39760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.844+0000 7f871c434f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87000030f0 0x55dabea29b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.844+0000 7f871c434f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabe8cda40 con 0x7f87000030f0 2024-04-25T10:09:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.844+0000 7f871c434f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dabe8cd760 con 0x55dabe99e820 2024-04-25T10:09:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.844+0000 7f871c434f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dabe8cdd20 con 0x55dabe998790 2024-04-25T10:09:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.844+0000 7f871a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87000030f0 0x55dabea29b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.844+0000 7f871a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87000030f0 0x55dabea29b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54592/0 (socket says 172.21.15.119:54592) 2024-04-25T10:09:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.844+0000 7f871a000640 1 -- 172.21.15.119:0/649602086 learned_addr learned my addr 172.21.15.119:0/649602086 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.844+0000 7f8718c00640 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabe99e820 0x55dabea39760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.844+0000 7f8719600640 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dabe998790 0x55dabea3d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.844+0000 7f871a000640 1 -- 172.21.15.119:0/649602086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dabe998790 msgr2=0x55dabea3d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.844+0000 7f871a000640 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55dabe998790 0x55dabea3d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.844+0000 7f871a000640 1 -- 172.21.15.119:0/649602086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabe99e820 msgr2=0x55dabea39760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.845+0000 7f871a000640 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabe99e820 0x55dabea39760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.845+0000 7f871a000640 1 -- 172.21.15.119:0/649602086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dabea3a020 con 0x7f87000030f0 2024-04-25T10:09:38.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.845+0000 7f8718c00640 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dabe99e820 0x55dabea39760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:38.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.845+0000 7f871a000640 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87000030f0 0x55dabea29b50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55dabe916030 tx=0x7f871400b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.845+0000 7f8712000640 1 -- 172.21.15.119:0/649602086 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871400e640 con 0x7f87000030f0 2024-04-25T10:09:38.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.845+0000 7f8712000640 1 -- 172.21.15.119:0/649602086 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8714012030 con 0x7f87000030f0 2024-04-25T10:09:38.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.845+0000 7f871c434f40 1 -- 172.21.15.119:0/649602086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dabea3a300 con 0x7f87000030f0 2024-04-25T10:09:38.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.845+0000 7f8712000640 1 -- 172.21.15.119:0/649602086 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871400ea40 con 0x7f87000030f0 2024-04-25T10:09:38.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.845+0000 7f871c434f40 1 -- 172.21.15.119:0/649602086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dabea29370 con 0x7f87000030f0 2024-04-25T10:09:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.847+0000 7f8712000640 1 -- 172.21.15.119:0/649602086 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8714009040 con 0x7f87000030f0 2024-04-25T10:09:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.848+0000 7f8712000640 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f86f8069ee0 0x7f86f806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.848+0000 7f8712000640 1 -- 172.21.15.119:0/649602086 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8714088f00 con 0x7f87000030f0 2024-04-25T10:09:38.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.848+0000 7f8719600640 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f86f8069ee0 0x7f86f806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.849+0000 7f871c434f40 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55dabea27780 0x55dabea3a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:38.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.849+0000 7f871c434f40 1 -- 172.21.15.119:0/649602086 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v9 -- 0x55dabea3adb0 con 0x55dabea27780 2024-04-25T10:09:38.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.849+0000 7f8719600640 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f86f8069ee0 0x7f86f806c3a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f870c003420 tx=0x7f870c007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.849+0000 7f8718c00640 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55dabea27780 0x55dabea3a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:38.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.850+0000 7f8718c00640 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55dabea27780 0x55dabea3a870 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.851+0000 7f8718c00640 1 -- 172.21.15.119:0/649602086 <== osd.6 v2:172.21.15.132:6806/353653608 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f8704054c80 con 0x55dabea27780 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.851+0000 7f871c434f40 1 -- 172.21.15.119:0/649602086 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55dabea27780 msgr2=0x55dabea3a870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.851+0000 7f871c434f40 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55dabea27780 0x55dabea3a870 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.851+0000 7f871c434f40 1 -- 172.21.15.119:0/649602086 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f86f8069ee0 msgr2=0x7f86f806c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.851+0000 7f871c434f40 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f86f8069ee0 0x7f86f806c3a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f870c003420 tx=0x7f870c007040 comp rx=0 tx=0).stop 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.851+0000 7f871c434f40 1 -- 172.21.15.119:0/649602086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87000030f0 msgr2=0x55dabea29b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.851+0000 7f871c434f40 1 --2- 172.21.15.119:0/649602086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87000030f0 0x55dabea29b50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55dabe916030 tx=0x7f871400b720 comp rx=0 tx=0).stop 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.851+0000 7f871a000640 1 -- 172.21.15.119:0/649602086 reap_dead start 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.851+0000 7f871c434f40 1 -- 172.21.15.119:0/649602086 shutdown_connections 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.851+0000 7f871c434f40 1 -- 172.21.15.119:0/649602086 >> 172.21.15.119:0/649602086 conn(0x55dabe8ce980 msgr2=0x55dabe8fbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.851+0000 7f871c434f40 1 -- 172.21.15.119:0/649602086 shutdown_connections 2024-04-25T10:09:38.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:38.851+0000 7f871c434f40 1 -- 172.21.15.119:0/649602086 wait complete. 2024-04-25T10:09:38.862 INFO:teuthology.orchestra.run.smithi119.stdout:Yalphaparent_a 2024-04-25T10:09:38.863 DEBUG:teuthology.orchestra.run.smithi119:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:09:38.954 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:09:38.954 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 1099511627777, 2024-04-25T10:09:38.954 INFO:teuthology.orchestra.run.smithi119.stdout: "ancestors": [ 2024-04-25T10:09:38.954 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:38.954 INFO:teuthology.orchestra.run.smithi119.stdout: "dirino": 1099511627776, 2024-04-25T10:09:38.954 INFO:teuthology.orchestra.run.smithi119.stdout: "dname": "alpha", 2024-04-25T10:09:38.955 INFO:teuthology.orchestra.run.smithi119.stdout: "version": 2 2024-04-25T10:09:38.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:38.955 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:38.955 INFO:teuthology.orchestra.run.smithi119.stdout: "dirino": 1, 2024-04-25T10:09:38.955 INFO:teuthology.orchestra.run.smithi119.stdout: "dname": "parent_a", 2024-04-25T10:09:38.955 INFO:teuthology.orchestra.run.smithi119.stdout: "version": 4 2024-04-25T10:09:38.955 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:38.955 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-04-25T10:09:38.955 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 7, 2024-04-25T10:09:38.955 INFO:teuthology.orchestra.run.smithi119.stdout: "old_pools": [] 2024-04-25T10:09:38.955 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:09:38.955 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:38.961 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:09:38.962 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:09:39.057 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:09:39.182 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:09:39.182 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:09:39.182 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:39.240 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:09:39.240 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:39.292 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:39.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.492+0000 7fca85c00640 1 Processor -- start 2024-04-25T10:09:39.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.492+0000 7fca85c00640 1 -- start start 2024-04-25T10:09:39.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.494+0000 7fca85c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fca8015d980 0x7fca800ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.494+0000 7fca85c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca8015ed70 0x7fca800acf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.494+0000 7fca85c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca800712c0 con 0x7fca800aeb20 2024-04-25T10:09:39.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.494+0000 7fca85c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca80071ed0 con 0x7fca8015d980 2024-04-25T10:09:39.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.494+0000 7fca85c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca800715a0 con 0x7fca8015ed70 2024-04-25T10:09:39.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.494+0000 7fca7ea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fca8015d980 0x7fca800ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.494+0000 7fca7ea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fca8015d980 0x7fca800ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54594/0 (socket says 172.21.15.119:54594) 2024-04-25T10:09:39.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.494+0000 7fca7ea00640 1 -- 172.21.15.119:0/611580781 learned_addr learned my addr 172.21.15.119:0/611580781 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:39.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.494+0000 7fca7f400640 1 --2- 172.21.15.119:0/611580781 >> v2:172.21.15.132:3300/0 conn(0x7fca8015ed70 0x7fca800acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.495+0000 7fca7ea00640 1 -- 172.21.15.119:0/611580781 >> v2:172.21.15.132:3300/0 conn(0x7fca8015ed70 msgr2=0x7fca800acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:39.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.495+0000 7fca7ea00640 1 --2- 172.21.15.119:0/611580781 >> v2:172.21.15.132:3300/0 conn(0x7fca8015ed70 0x7fca800acf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.495+0000 7fca7ea00640 1 -- 172.21.15.119:0/611580781 >> v1:172.21.15.132:6789/0 conn(0x7fca800aeb20 legacy=0x7fca800cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:39.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.495+0000 7fca7ea00640 1 -- 172.21.15.119:0/611580781 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca80071880 con 0x7fca8015d980 2024-04-25T10:09:39.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.495+0000 7fca7f400640 1 --2- 172.21.15.119:0/611580781 >> v2:172.21.15.132:3300/0 conn(0x7fca8015ed70 0x7fca800acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.496+0000 7fca7ea00640 1 --2- 172.21.15.119:0/611580781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8015d980 0x7fca800ae5e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fca8006db40 tx=0x7fca7002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277704184f8022b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.496+0000 7fca7d600640 1 -- 172.21.15.119:0/611580781 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70037070 con 0x7fca8015d980 2024-04-25T10:09:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.496+0000 7fca7d600640 1 -- 172.21.15.119:0/611580781 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca70032440 con 0x7fca8015d980 2024-04-25T10:09:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.496+0000 7fca7d600640 1 -- 172.21.15.119:0/611580781 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70032810 con 0x7fca8015d980 2024-04-25T10:09:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.497+0000 7fca85c00640 1 -- 172.21.15.119:0/611580781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8015d980 msgr2=0x7fca800ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.497+0000 7fca85c00640 1 --2- 172.21.15.119:0/611580781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8015d980 0x7fca800ae5e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fca8006db40 tx=0x7fca7002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.497+0000 7fca85c00640 1 -- 172.21.15.119:0/611580781 shutdown_connections 2024-04-25T10:09:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.497+0000 7fca85c00640 1 --2- 172.21.15.119:0/611580781 >> v2:172.21.15.132:3300/0 conn(0x7fca8015ed70 0x7fca800acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.497+0000 7fca85c00640 1 --2- 172.21.15.119:0/611580781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8015d980 0x7fca800ae5e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.497+0000 7fca85c00640 1 -- 172.21.15.119:0/611580781 >> 172.21.15.119:0/611580781 conn(0x7fca8006aba0 msgr2=0x7fca8006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:39.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.497+0000 7fca85c00640 1 -- 172.21.15.119:0/611580781 shutdown_connections 2024-04-25T10:09:39.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.497+0000 7fca85c00640 1 -- 172.21.15.119:0/611580781 wait complete. 2024-04-25T10:09:39.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.498+0000 7fca85c00640 1 Processor -- start 2024-04-25T10:09:39.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.499+0000 7fca85c00640 1 -- start start 2024-04-25T10:09:39.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.499+0000 7fca85c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca800aeb20 0x7fca80143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.500+0000 7fca85c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8015d980 0x7fca800b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.500+0000 7fca85c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca800b3ba0 0x7fca8015b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.500+0000 7fca85c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca800cfb10 con 0x7fca800aeb20 2024-04-25T10:09:39.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.500+0000 7fca85c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca800cf990 con 0x7fca8015d980 2024-04-25T10:09:39.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.500+0000 7fca85c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca800ad4b0 con 0x7fca800b3ba0 2024-04-25T10:09:39.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.500+0000 7fca7ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca800aeb20 0x7fca80143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.500+0000 7fca7ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca800aeb20 0x7fca80143390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54598/0 (socket says 172.21.15.119:54598) 2024-04-25T10:09:39.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.500+0000 7fca7ea00640 1 -- 172.21.15.119:0/2623737090 learned_addr learned my addr 172.21.15.119:0/2623737090 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:39.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.500+0000 7fca7ea00640 1 -- 172.21.15.119:0/2623737090 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca800b3ba0 msgr2=0x7fca8015b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.500+0000 7fca7ea00640 1 --2- 172.21.15.119:0/2623737090 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca800b3ba0 0x7fca8015b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.500+0000 7fca7ea00640 1 -- 172.21.15.119:0/2623737090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8015d980 msgr2=0x7fca800b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.500+0000 7fca7ea00640 1 --2- 172.21.15.119:0/2623737090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8015d980 0x7fca800b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.500+0000 7fca7ea00640 1 -- 172.21.15.119:0/2623737090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8015c3a0 con 0x7fca800aeb20 2024-04-25T10:09:39.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.501+0000 7fca7ea00640 1 --2- 172.21.15.119:0/2623737090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca800aeb20 0x7fca80143390 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fca7002a1e0 tx=0x7fca700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.501+0000 7fca76a00640 1 -- 172.21.15.119:0/2623737090 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70037040 con 0x7fca800aeb20 2024-04-25T10:09:39.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.501+0000 7fca76a00640 1 -- 172.21.15.119:0/2623737090 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca70042070 con 0x7fca800aeb20 2024-04-25T10:09:39.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.501+0000 7fca85c00640 1 -- 172.21.15.119:0/2623737090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8015c680 con 0x7fca800aeb20 2024-04-25T10:09:39.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.501+0000 7fca76a00640 1 -- 172.21.15.119:0/2623737090 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70030bd0 con 0x7fca800aeb20 2024-04-25T10:09:39.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.501+0000 7fca85c00640 1 -- 172.21.15.119:0/2623737090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca80144030 con 0x7fca800aeb20 2024-04-25T10:09:39.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.502+0000 7fca85c00640 1 -- 172.21.15.119:0/2623737090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca800749b0 con 0x7fca800aeb20 2024-04-25T10:09:39.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.503+0000 7fca76a00640 1 -- 172.21.15.119:0/2623737090 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca70047460 con 0x7fca800aeb20 2024-04-25T10:09:39.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.504+0000 7fca76a00640 1 --2- 172.21.15.119:0/2623737090 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca6006b500 0x7fca6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.505+0000 7fca76a00640 1 -- 172.21.15.119:0/2623737090 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca700aaf80 con 0x7fca800aeb20 2024-04-25T10:09:39.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.505+0000 7fca7e000640 1 --2- 172.21.15.119:0/2623737090 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca6006b500 0x7fca6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.505+0000 7fca7e000640 1 --2- 172.21.15.119:0/2623737090 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca6006b500 0x7fca6006d9c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fca801f6c00 tx=0x7fca6c0079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.509+0000 7fca76a00640 1 -- 172.21.15.119:0/2623737090 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca70045360 con 0x7fca800aeb20 2024-04-25T10:09:39.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.704+0000 7fca85c00640 1 -- 172.21.15.119:0/2623737090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca8006d4c0 con 0x7fca800aeb20 2024-04-25T10:09:39.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.706+0000 7fca76a00640 1 -- 172.21.15.119:0/2623737090 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16649 (secure 0 0 0) 0x7fca70079d60 con 0x7fca800aeb20 2024-04-25T10:09:39.713 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:39.713 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":44,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:58.637281+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:54.917692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:56.128440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:39.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.709+0000 7fca85c00640 1 -- 172.21.15.119:0/2623737090 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca6006b500 msgr2=0x7fca6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.709+0000 7fca85c00640 1 --2- 172.21.15.119:0/2623737090 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca6006b500 0x7fca6006d9c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fca801f6c00 tx=0x7fca6c0079b0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.709+0000 7fca85c00640 1 -- 172.21.15.119:0/2623737090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca800aeb20 msgr2=0x7fca80143390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.709+0000 7fca85c00640 1 --2- 172.21.15.119:0/2623737090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca800aeb20 0x7fca80143390 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fca7002a1e0 tx=0x7fca700046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.709+0000 7fca85c00640 1 -- 172.21.15.119:0/2623737090 shutdown_connections 2024-04-25T10:09:39.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.709+0000 7fca85c00640 1 --2- 172.21.15.119:0/2623737090 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca6006b500 0x7fca6006d9c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.709+0000 7fca85c00640 1 --2- 172.21.15.119:0/2623737090 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca800b3ba0 0x7fca8015b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.709+0000 7fca85c00640 1 --2- 172.21.15.119:0/2623737090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8015d980 0x7fca800b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.709+0000 7fca85c00640 1 --2- 172.21.15.119:0/2623737090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca800aeb20 0x7fca80143390 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.709+0000 7fca85c00640 1 -- 172.21.15.119:0/2623737090 >> 172.21.15.119:0/2623737090 conn(0x7fca8006aba0 msgr2=0x7fca8015f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:39.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.709+0000 7fca85c00640 1 -- 172.21.15.119:0/2623737090 shutdown_connections 2024-04-25T10:09:39.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.709+0000 7fca85c00640 1 -- 172.21.15.119:0/2623737090 wait complete. 2024-04-25T10:09:39.735 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:54.917692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:56.128440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:39.735 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:39.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.926+0000 7f79ee000640 1 Processor -- start 2024-04-25T10:09:39.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.926+0000 7f79ee000640 1 -- start start 2024-04-25T10:09:39.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.927+0000 7f79ee000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f79e80aed40 0x7f79e80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.927+0000 7f79ee000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e815e360 con 0x7f79e80cfa90 2024-04-25T10:09:39.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.927+0000 7f79ee000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e815e5d0 con 0x7f79e815b720 2024-04-25T10:09:39.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.927+0000 7f79ee000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e815e8b0 con 0x7f79e80aed40 2024-04-25T10:09:39.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.927+0000 7f79e7e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f79e80cfa90 0x7f79e80acfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36756/0 (socket says 172.21.15.119:36756) 2024-04-25T10:09:39.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.928+0000 7f79e7e00640 1 -- 172.21.15.119:0/195868341 learned_addr learned my addr 172.21.15.119:0/195868341 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:39.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.928+0000 7f79e6a00640 1 --2- 172.21.15.119:0/195868341 >> v2:172.21.15.132:3300/0 conn(0x7f79e80aed40 0x7f79e80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.928+0000 7f79e6000640 1 -- 172.21.15.119:0/195868341 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939790177 0 0) 0x7f79e815e360 con 0x7f79e80cfa90 2024-04-25T10:09:39.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.928+0000 7f79e6000640 1 -- 172.21.15.119:0/195868341 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79cc003520 con 0x7f79e80cfa90 2024-04-25T10:09:39.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.928+0000 7f79e6000640 1 -- 172.21.15.119:0/195868341 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445826600 0 0) 0x7f79cc003520 con 0x7f79e80cfa90 2024-04-25T10:09:39.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.928+0000 7f79e6000640 1 -- 172.21.15.119:0/195868341 >> v2:172.21.15.132:3300/0 conn(0x7f79e80aed40 msgr2=0x7f79e80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:39.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.928+0000 7f79e6000640 1 --2- 172.21.15.119:0/195868341 >> v2:172.21.15.132:3300/0 conn(0x7f79e80aed40 0x7f79e80cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.928+0000 7f79e6000640 1 -- 172.21.15.119:0/195868341 >> v1:172.21.15.132:6789/0 conn(0x7f79e815b720 legacy=0x7f79e80ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.928+0000 7f79e6000640 1 -- 172.21.15.119:0/195868341 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e815eb90 con 0x7f79e80cfa90 2024-04-25T10:09:39.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.928+0000 7f79e6000640 1 -- 172.21.15.119:0/195868341 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f79e00019f0 con 0x7f79e80cfa90 2024-04-25T10:09:39.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.928+0000 7f79e6000640 1 -- 172.21.15.119:0/195868341 >> v1:172.21.15.119:6789/0 conn(0x7f79e80cfa90 legacy=0x7f79e80acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.928+0000 7f79e6000640 1 --2- 172.21.15.119:0/195868341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79cc0039b0 0x7f79cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.929+0000 7f79e6000640 1 --2- 172.21.15.119:0/195868341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79cc004800 0x7f79cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.929+0000 7f79e6000640 1 --2- 172.21.15.119:0/195868341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79cc0051c0 0x7f79cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.929+0000 7f79e6000640 1 -- 172.21.15.119:0/195868341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e815e8b0 con 0x7f79cc004800 2024-04-25T10:09:39.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.929+0000 7f79e6000640 1 -- 172.21.15.119:0/195868341 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc00c060 con 0x7f79cc0039b0 2024-04-25T10:09:39.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.929+0000 7f79e6000640 1 -- 172.21.15.119:0/195868341 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc00c260 con 0x7f79cc0051c0 2024-04-25T10:09:39.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.929+0000 7f79e6a00640 1 --2- 172.21.15.119:0/195868341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79cc004800 0x7f79cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.929+0000 7f79e7400640 1 --2- 172.21.15.119:0/195868341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79cc0039b0 0x7f79cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.929+0000 7f79e6a00640 1 -- 172.21.15.119:0/195868341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79cc0051c0 msgr2=0x7f79cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:39.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.929+0000 7f79e6a00640 1 --2- 172.21.15.119:0/195868341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79cc0051c0 0x7f79cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.929+0000 7f79e6a00640 1 -- 172.21.15.119:0/195868341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79cc0039b0 msgr2=0x7f79cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:39.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.929+0000 7f79e6a00640 1 --2- 172.21.15.119:0/195868341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79cc0039b0 0x7f79cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.929+0000 7f79e6a00640 1 -- 172.21.15.119:0/195868341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc003520 con 0x7f79cc004800 2024-04-25T10:09:39.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.929+0000 7f79e7400640 1 --2- 172.21.15.119:0/195868341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79cc0039b0 0x7f79cc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:39.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.929+0000 7f79e7e00640 1 -- 172.21.15.119:0/195868341 reap_dead start 2024-04-25T10:09:39.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.930+0000 7f79e6a00640 1 --2- 172.21.15.119:0/195868341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79cc004800 0x7f79cc004c80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009a90 tx=0x7f79cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b579fdd7e3ae66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.930+0000 7f79e6000640 1 -- 172.21.15.119:0/195868341 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0036070 con 0x7f79cc004800 2024-04-25T10:09:39.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.930+0000 7f79e6000640 1 -- 172.21.15.119:0/195868341 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d0031400 con 0x7f79cc004800 2024-04-25T10:09:39.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.931+0000 7f79e6000640 1 -- 172.21.15.119:0/195868341 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0009690 con 0x7f79cc004800 2024-04-25T10:09:39.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.931+0000 7f79ee000640 1 -- 172.21.15.119:0/195868341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79cc004800 msgr2=0x7f79cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.931+0000 7f79ee000640 1 --2- 172.21.15.119:0/195868341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79cc004800 0x7f79cc004c80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009a90 tx=0x7f79cc005830 comp rx=0 tx=0).stop 2024-04-25T10:09:39.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.931+0000 7f79ee000640 1 -- 172.21.15.119:0/195868341 shutdown_connections 2024-04-25T10:09:39.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.931+0000 7f79ee000640 1 --2- 172.21.15.119:0/195868341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79cc004800 0x7f79cc004c80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.931+0000 7f79ee000640 1 -- 172.21.15.119:0/195868341 >> 172.21.15.119:0/195868341 conn(0x7f79e806aba0 msgr2=0x7f79e806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:39.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.931+0000 7f79ee000640 1 -- 172.21.15.119:0/195868341 shutdown_connections 2024-04-25T10:09:39.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.931+0000 7f79ee000640 1 -- 172.21.15.119:0/195868341 wait complete. 2024-04-25T10:09:39.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.933+0000 7f79ee000640 1 Processor -- start 2024-04-25T10:09:39.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.933+0000 7f79ee000640 1 -- start start 2024-04-25T10:09:39.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.934+0000 7f79ee000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79cc004800 0x7f79e8139c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.934+0000 7f79ee000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e814d3a0 0x7f79e81f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.934+0000 7f79ee000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79e814d800 0x7f79e81f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.934+0000 7f79ee000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e80cfed0 con 0x7f79e814d3a0 2024-04-25T10:09:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.934+0000 7f79ee000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e815bb00 con 0x7f79e814d800 2024-04-25T10:09:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.934+0000 7f79ee000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e80d0050 con 0x7f79cc004800 2024-04-25T10:09:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.934+0000 7f79e6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e814d3a0 0x7f79e81f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.935+0000 7f79e6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e814d3a0 0x7f79e81f45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54616/0 (socket says 172.21.15.119:54616) 2024-04-25T10:09:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.935+0000 7f79e6a00640 1 -- 172.21.15.119:0/281502952 learned_addr learned my addr 172.21.15.119:0/281502952 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.935+0000 7f79e7e00640 1 --2- 172.21.15.119:0/281502952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79e814d800 0x7f79e81f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.935+0000 7f79e6a00640 1 -- 172.21.15.119:0/281502952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79cc004800 msgr2=0x7f79e8139c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.935+0000 7f79e6a00640 1 --2- 172.21.15.119:0/281502952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79cc004800 0x7f79e8139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.935+0000 7f79e6a00640 1 -- 172.21.15.119:0/281502952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79e814d800 msgr2=0x7f79e81f4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.935+0000 7f79e6a00640 1 --2- 172.21.15.119:0/281502952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79e814d800 0x7f79e81f4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.935+0000 7f79e6a00640 1 -- 172.21.15.119:0/281502952 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e80cd240 con 0x7f79e814d3a0 2024-04-25T10:09:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.935+0000 7f79e7e00640 1 --2- 172.21.15.119:0/281502952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79e814d800 0x7f79e81f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.935+0000 7f79e6a00640 1 --2- 172.21.15.119:0/281502952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e814d3a0 0x7f79e81f45d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009bb0 tx=0x7f79d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.936+0000 7f79df400640 1 -- 172.21.15.119:0/281502952 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d003f020 con 0x7f79e814d3a0 2024-04-25T10:09:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.936+0000 7f79df400640 1 -- 172.21.15.119:0/281502952 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d0037040 con 0x7f79e814d3a0 2024-04-25T10:09:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.936+0000 7f79df400640 1 -- 172.21.15.119:0/281502952 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0036070 con 0x7f79e814d3a0 2024-04-25T10:09:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.936+0000 7f79ee000640 1 -- 172.21.15.119:0/281502952 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e80cd4b0 con 0x7f79e814d3a0 2024-04-25T10:09:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.936+0000 7f79ee000640 1 -- 172.21.15.119:0/281502952 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e80cda70 con 0x7f79e814d3a0 2024-04-25T10:09:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.937+0000 7f79ee000640 1 -- 172.21.15.119:0/281502952 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e813a210 con 0x7f79e814d3a0 2024-04-25T10:09:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.938+0000 7f79df400640 1 -- 172.21.15.119:0/281502952 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79d0042020 con 0x7f79e814d3a0 2024-04-25T10:09:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.939+0000 7f79df400640 1 --2- 172.21.15.119:0/281502952 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f79c406b500 0x7f79c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.939+0000 7f79df400640 1 -- 172.21.15.119:0/281502952 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79d00aa2c0 con 0x7f79e814d3a0 2024-04-25T10:09:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.939+0000 7f79e7400640 1 --2- 172.21.15.119:0/281502952 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f79c406b500 0x7f79c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.940+0000 7f79e7400640 1 --2- 172.21.15.119:0/281502952 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f79c406b500 0x7f79c406d9c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f79d8001690 tx=0x7f79d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:39.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:39.943+0000 7f79df400640 1 -- 172.21.15.119:0/281502952 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79e813a210 con 0x7f79e814d3a0 2024-04-25T10:09:40.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.139+0000 7f79ee000640 1 -- 172.21.15.119:0/281502952 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79e80750b0 con 0x7f79e814d3a0 2024-04-25T10:09:40.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.140+0000 7f79df400640 1 -- 172.21.15.119:0/281502952 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79d0049030 con 0x7f79e814d3a0 2024-04-25T10:09:40.146 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:09:40.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.143+0000 7f79ee000640 1 -- 172.21.15.119:0/281502952 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f79c406b500 msgr2=0x7f79c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.143+0000 7f79ee000640 1 --2- 172.21.15.119:0/281502952 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f79c406b500 0x7f79c406d9c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f79d8001690 tx=0x7f79d8009040 comp rx=0 tx=0).stop 2024-04-25T10:09:40.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.143+0000 7f79ee000640 1 -- 172.21.15.119:0/281502952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e814d3a0 msgr2=0x7f79e81f45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.143+0000 7f79ee000640 1 --2- 172.21.15.119:0/281502952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e814d3a0 0x7f79e81f45d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009bb0 tx=0x7f79d0004730 comp rx=0 tx=0).stop 2024-04-25T10:09:40.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.143+0000 7f79ee000640 1 -- 172.21.15.119:0/281502952 shutdown_connections 2024-04-25T10:09:40.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.143+0000 7f79ee000640 1 --2- 172.21.15.119:0/281502952 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f79c406b500 0x7f79c406d9c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.143+0000 7f79ee000640 1 --2- 172.21.15.119:0/281502952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79e814d800 0x7f79e81f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.143+0000 7f79ee000640 1 --2- 172.21.15.119:0/281502952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e814d3a0 0x7f79e81f45d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.143+0000 7f79ee000640 1 --2- 172.21.15.119:0/281502952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79cc004800 0x7f79e8139c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.143+0000 7f79ee000640 1 -- 172.21.15.119:0/281502952 >> 172.21.15.119:0/281502952 conn(0x7f79e806aba0 msgr2=0x7f79e80ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.143+0000 7f79ee000640 1 -- 172.21.15.119:0/281502952 shutdown_connections 2024-04-25T10:09:40.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.143+0000 7f79ee000640 1 -- 172.21.15.119:0/281502952 wait complete. 2024-04-25T10:09:40.168 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:09:40.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.368+0000 7fc66e000640 1 Processor -- start 2024-04-25T10:09:40.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.369+0000 7fc66e000640 1 -- start start 2024-04-25T10:09:40.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.370+0000 7fc66e000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc66815dca0 0x7fc6680ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.370+0000 7fc66e000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6680d4090 con 0x7fc6680d1130 2024-04-25T10:09:40.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.370+0000 7fc66e000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6680d4370 con 0x7fc6680d3b90 2024-04-25T10:09:40.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.370+0000 7fc66e000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6680b6b20 con 0x7fc66815dca0 2024-04-25T10:09:40.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.370+0000 7fc667e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc6680d1130 0x7fc6680b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36762/0 (socket says 172.21.15.119:36762) 2024-04-25T10:09:40.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.370+0000 7fc667e00640 1 -- 172.21.15.119:0/4177432710 learned_addr learned my addr 172.21.15.119:0/4177432710 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:40.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc667400640 1 --2- 172.21.15.119:0/4177432710 >> v2:172.21.15.132:3300/0 conn(0x7fc66815dca0 0x7fc6680ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 -- 172.21.15.119:0/4177432710 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714991340 0 0) 0x7fc6680d4090 con 0x7fc6680d1130 2024-04-25T10:09:40.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 -- 172.21.15.119:0/4177432710 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc64c003520 con 0x7fc6680d1130 2024-04-25T10:09:40.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 -- 172.21.15.119:0/4177432710 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 268848705 0 0) 0x7fc64c003520 con 0x7fc6680d1130 2024-04-25T10:09:40.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 -- 172.21.15.119:0/4177432710 >> v2:172.21.15.132:3300/0 conn(0x7fc66815dca0 msgr2=0x7fc6680ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 --2- 172.21.15.119:0/4177432710 >> v2:172.21.15.132:3300/0 conn(0x7fc66815dca0 0x7fc6680ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 -- 172.21.15.119:0/4177432710 >> v1:172.21.15.132:6789/0 conn(0x7fc6680d3b90 legacy=0x7fc6680d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 -- 172.21.15.119:0/4177432710 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6680d4650 con 0x7fc6680d1130 2024-04-25T10:09:40.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 -- 172.21.15.119:0/4177432710 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fc660001670 con 0x7fc6680d1130 2024-04-25T10:09:40.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 -- 172.21.15.119:0/4177432710 >> v1:172.21.15.119:6789/0 conn(0x7fc6680d1130 legacy=0x7fc6680b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 --2- 172.21.15.119:0/4177432710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc64c0039b0 0x7fc64c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 --2- 172.21.15.119:0/4177432710 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc64c004800 0x7fc64c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 --2- 172.21.15.119:0/4177432710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c0051c0 0x7fc64c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 -- 172.21.15.119:0/4177432710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c00c030 con 0x7fc64c0039b0 2024-04-25T10:09:40.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 -- 172.21.15.119:0/4177432710 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c00c1e0 con 0x7fc64c0051c0 2024-04-25T10:09:40.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.371+0000 7fc666000640 1 -- 172.21.15.119:0/4177432710 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c00c3e0 con 0x7fc64c004800 2024-04-25T10:09:40.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.372+0000 7fc667400640 1 --2- 172.21.15.119:0/4177432710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc64c0039b0 0x7fc64c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.372+0000 7fc667400640 1 -- 172.21.15.119:0/4177432710 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc64c004800 msgr2=0x7fc64c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.372+0000 7fc667400640 1 --2- 172.21.15.119:0/4177432710 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc64c004800 0x7fc64c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.372+0000 7fc667400640 1 -- 172.21.15.119:0/4177432710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c0051c0 msgr2=0x7fc64c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.372+0000 7fc667400640 1 --2- 172.21.15.119:0/4177432710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c0051c0 0x7fc64c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.372+0000 7fc667400640 1 -- 172.21.15.119:0/4177432710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c003520 con 0x7fc64c0039b0 2024-04-25T10:09:40.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.372+0000 7fc667e00640 1 -- 172.21.15.119:0/4177432710 reap_dead start 2024-04-25T10:09:40.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.373+0000 7fc667400640 1 --2- 172.21.15.119:0/4177432710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc64c0039b0 0x7fc64c003dd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc658009fc0 tx=0x7fc64c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c76bca7fd137e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.373+0000 7fc666000640 1 -- 172.21.15.119:0/4177432710 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658009c10 con 0x7fc64c0039b0 2024-04-25T10:09:40.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.373+0000 7fc666000640 1 -- 172.21.15.119:0/4177432710 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc658036070 con 0x7fc64c0039b0 2024-04-25T10:09:40.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.373+0000 7fc666000640 1 -- 172.21.15.119:0/4177432710 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65802d070 con 0x7fc64c0039b0 2024-04-25T10:09:40.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.374+0000 7fc66e000640 1 -- 172.21.15.119:0/4177432710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc64c0039b0 msgr2=0x7fc64c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.374+0000 7fc66e000640 1 --2- 172.21.15.119:0/4177432710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc64c0039b0 0x7fc64c003dd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc658009fc0 tx=0x7fc64c005830 comp rx=0 tx=0).stop 2024-04-25T10:09:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.374+0000 7fc66e000640 1 -- 172.21.15.119:0/4177432710 shutdown_connections 2024-04-25T10:09:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.374+0000 7fc66e000640 1 --2- 172.21.15.119:0/4177432710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc64c0039b0 0x7fc64c003dd0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.374+0000 7fc66e000640 1 -- 172.21.15.119:0/4177432710 >> 172.21.15.119:0/4177432710 conn(0x7fc6680726c0 msgr2=0x7fc668072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.374+0000 7fc66e000640 1 -- 172.21.15.119:0/4177432710 shutdown_connections 2024-04-25T10:09:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.374+0000 7fc66e000640 1 -- 172.21.15.119:0/4177432710 wait complete. 2024-04-25T10:09:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.376+0000 7fc66e000640 1 Processor -- start 2024-04-25T10:09:40.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.376+0000 7fc66e000640 1 -- start start 2024-04-25T10:09:40.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.377+0000 7fc66e000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c0039b0 0x7fc66814dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.377+0000 7fc66e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6680ccfb0 0x7fc668159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.377+0000 7fc66e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc668155da0 0x7fc668156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.377+0000 7fc66e000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66815e200 con 0x7fc6680ccfb0 2024-04-25T10:09:40.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.377+0000 7fc66e000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66815e080 con 0x7fc64c0039b0 2024-04-25T10:09:40.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.377+0000 7fc66e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680d0a20 con 0x7fc668155da0 2024-04-25T10:09:40.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.377+0000 7fc666a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6680ccfb0 0x7fc668159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.377+0000 7fc666a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6680ccfb0 0x7fc668159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54638/0 (socket says 172.21.15.119:54638) 2024-04-25T10:09:40.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.377+0000 7fc666a00640 1 -- 172.21.15.119:0/973971826 learned_addr learned my addr 172.21.15.119:0/973971826 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:40.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.378+0000 7fc667400640 1 --2- 172.21.15.119:0/973971826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c0039b0 0x7fc66814dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.378+0000 7fc667e00640 1 --2- 172.21.15.119:0/973971826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc668155da0 0x7fc668156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.378+0000 7fc666a00640 1 -- 172.21.15.119:0/973971826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc668155da0 msgr2=0x7fc668156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.378+0000 7fc666a00640 1 --2- 172.21.15.119:0/973971826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc668155da0 0x7fc668156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.378+0000 7fc666a00640 1 -- 172.21.15.119:0/973971826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c0039b0 msgr2=0x7fc66814dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.378+0000 7fc666a00640 1 --2- 172.21.15.119:0/973971826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c0039b0 0x7fc66814dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.378+0000 7fc666a00640 1 -- 172.21.15.119:0/973971826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6680cd430 con 0x7fc6680ccfb0 2024-04-25T10:09:40.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.378+0000 7fc666a00640 1 --2- 172.21.15.119:0/973971826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6680ccfb0 0x7fc668159410 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc65000c420 tx=0x7fc65000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.378+0000 7fc65f400640 1 -- 172.21.15.119:0/973971826 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6500052d0 con 0x7fc6680ccfb0 2024-04-25T10:09:40.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.378+0000 7fc65f400640 1 -- 172.21.15.119:0/973971826 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc650005490 con 0x7fc6680ccfb0 2024-04-25T10:09:40.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.378+0000 7fc65f400640 1 -- 172.21.15.119:0/973971826 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650007260 con 0x7fc6680ccfb0 2024-04-25T10:09:40.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.378+0000 7fc66e000640 1 -- 172.21.15.119:0/973971826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6681545c0 con 0x7fc6680ccfb0 2024-04-25T10:09:40.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.379+0000 7fc66e000640 1 -- 172.21.15.119:0/973971826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc668154b20 con 0x7fc6680ccfb0 2024-04-25T10:09:40.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.380+0000 7fc66e000640 1 -- 172.21.15.119:0/973971826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6680750b0 con 0x7fc6680ccfb0 2024-04-25T10:09:40.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.381+0000 7fc65f400640 1 -- 172.21.15.119:0/973971826 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc650007400 con 0x7fc6680ccfb0 2024-04-25T10:09:40.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.382+0000 7fc65f400640 1 --2- 172.21.15.119:0/973971826 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc64406b4d0 0x7fc64406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.382+0000 7fc65f400640 1 -- 172.21.15.119:0/973971826 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6500895c0 con 0x7fc6680ccfb0 2024-04-25T10:09:40.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.382+0000 7fc667400640 1 --2- 172.21.15.119:0/973971826 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc64406b4d0 0x7fc64406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.383+0000 7fc667400640 1 --2- 172.21.15.119:0/973971826 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc64406b4d0 0x7fc64406d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc658035000 tx=0x7fc65802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.386+0000 7fc65f400640 1 -- 172.21.15.119:0/973971826 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6681545c0 con 0x7fc6680ccfb0 2024-04-25T10:09:40.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.585+0000 7fc66e000640 1 -- 172.21.15.119:0/973971826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6680cdb90 con 0x7fc6680ccfb0 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.586+0000 7fc65f400640 1 -- 172.21.15.119:0/973971826 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc650058500 con 0x7fc6680ccfb0 2024-04-25T10:09:40.592 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.588+0000 7fc66e000640 1 -- 172.21.15.119:0/973971826 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc64406b4d0 msgr2=0x7fc64406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.588+0000 7fc66e000640 1 --2- 172.21.15.119:0/973971826 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc64406b4d0 0x7fc64406d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc658035000 tx=0x7fc65802f000 comp rx=0 tx=0).stop 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.588+0000 7fc66e000640 1 -- 172.21.15.119:0/973971826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6680ccfb0 msgr2=0x7fc668159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.588+0000 7fc66e000640 1 --2- 172.21.15.119:0/973971826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6680ccfb0 0x7fc668159410 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc65000c420 tx=0x7fc65000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.588+0000 7fc66e000640 1 -- 172.21.15.119:0/973971826 shutdown_connections 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.588+0000 7fc66e000640 1 --2- 172.21.15.119:0/973971826 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc64406b4d0 0x7fc64406d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.588+0000 7fc66e000640 1 --2- 172.21.15.119:0/973971826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc668155da0 0x7fc668156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.588+0000 7fc66e000640 1 --2- 172.21.15.119:0/973971826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6680ccfb0 0x7fc668159410 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.589+0000 7fc66e000640 1 --2- 172.21.15.119:0/973971826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc64c0039b0 0x7fc66814dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.589+0000 7fc66e000640 1 -- 172.21.15.119:0/973971826 >> 172.21.15.119:0/973971826 conn(0x7fc6680726c0 msgr2=0x7fc6680b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.589+0000 7fc66e000640 1 -- 172.21.15.119:0/973971826 shutdown_connections 2024-04-25T10:09:40.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.589+0000 7fc66e000640 1 -- 172.21.15.119:0/973971826 wait complete. 2024-04-25T10:09:40.614 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:09:40.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.808+0000 7f484fe00640 1 Processor -- start 2024-04-25T10:09:40.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.808+0000 7f484fe00640 1 -- start start 2024-04-25T10:09:40.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.810+0000 7f484fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f48480cf900 0x7f48480b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.810+0000 7f484fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48480b2b90 0x7f4848162f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.810+0000 7f484fe00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4848071350 con 0x7f48480cfce0 2024-04-25T10:09:40.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.810+0000 7f484fe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4848071630 con 0x7f48480cf900 2024-04-25T10:09:40.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.810+0000 7f484fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4848071910 con 0x7f48480b2b90 2024-04-25T10:09:40.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.810+0000 7f484ca00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f48480cfce0 0x7f4848162880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36776/0 (socket says 172.21.15.119:36776) 2024-04-25T10:09:40.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.810+0000 7f484ca00640 1 -- 172.21.15.119:0/3865443204 learned_addr learned my addr 172.21.15.119:0/3865443204 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:40.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.811+0000 7f484d400640 1 --2- 172.21.15.119:0/3865443204 >> v2:172.21.15.132:3300/0 conn(0x7f48480cf900 0x7f48480b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.811+0000 7f484de00640 1 --2- 172.21.15.119:0/3865443204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48480b2b90 0x7f4848162f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.811+0000 7f4847e00640 1 -- 172.21.15.119:0/3865443204 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224524370 0 0) 0x7f4848071350 con 0x7f48480cfce0 2024-04-25T10:09:40.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.811+0000 7f4847e00640 1 -- 172.21.15.119:0/3865443204 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4830003520 con 0x7f48480cfce0 2024-04-25T10:09:40.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.811+0000 7f4847e00640 1 -- 172.21.15.119:0/3865443204 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3578455665 0 0) 0x7f4830003520 con 0x7f48480cfce0 2024-04-25T10:09:40.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.811+0000 7f4847e00640 1 -- 172.21.15.119:0/3865443204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48480b2b90 msgr2=0x7f4848162f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.811+0000 7f4847e00640 1 --2- 172.21.15.119:0/3865443204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48480b2b90 0x7f4848162f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.811+0000 7f4847e00640 1 -- 172.21.15.119:0/3865443204 >> v2:172.21.15.132:3300/0 conn(0x7f48480cf900 msgr2=0x7f48480b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.811+0000 7f4847e00640 1 --2- 172.21.15.119:0/3865443204 >> v2:172.21.15.132:3300/0 conn(0x7f48480cf900 0x7f48480b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.811+0000 7f4847e00640 1 -- 172.21.15.119:0/3865443204 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4848071bf0 con 0x7f48480cfce0 2024-04-25T10:09:40.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.811+0000 7f4847e00640 1 -- 172.21.15.119:0/3865443204 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f4834001690 con 0x7f48480cfce0 2024-04-25T10:09:40.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.811+0000 7f4847e00640 1 -- 172.21.15.119:0/3865443204 >> v1:172.21.15.119:6789/0 conn(0x7f48480cfce0 legacy=0x7f4848162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.811+0000 7f4847e00640 1 --2- 172.21.15.119:0/3865443204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48300039b0 0x7f4830003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.812+0000 7f4847e00640 1 --2- 172.21.15.119:0/3865443204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48480b2b90 0x7f4830004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.812+0000 7f4847e00640 1 --2- 172.21.15.119:0/3865443204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4830004d40 0x7f48300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.812+0000 7f4847e00640 1 -- 172.21.15.119:0/3865443204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4848071630 con 0x7f48300039b0 2024-04-25T10:09:40.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.812+0000 7f4847e00640 1 -- 172.21.15.119:0/3865443204 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4848071910 con 0x7f4830004d40 2024-04-25T10:09:40.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.812+0000 7f4847e00640 1 -- 172.21.15.119:0/3865443204 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483000b9f0 con 0x7f48480b2b90 2024-04-25T10:09:40.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.812+0000 7f484d400640 1 --2- 172.21.15.119:0/3865443204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48300039b0 0x7f4830003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.812+0000 7f484de00640 1 --2- 172.21.15.119:0/3865443204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4830004d40 0x7f48300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.812+0000 7f484d400640 1 -- 172.21.15.119:0/3865443204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48480b2b90 msgr2=0x7f4830004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.812+0000 7f484d400640 1 --2- 172.21.15.119:0/3865443204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48480b2b90 0x7f4830004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.812+0000 7f484d400640 1 -- 172.21.15.119:0/3865443204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4830004d40 msgr2=0x7f48300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:40.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.812+0000 7f484d400640 1 --2- 172.21.15.119:0/3865443204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4830004d40 0x7f48300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.812+0000 7f484d400640 1 -- 172.21.15.119:0/3865443204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4830003520 con 0x7f48300039b0 2024-04-25T10:09:40.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.812+0000 7f484de00640 1 --2- 172.21.15.119:0/3865443204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4830004d40 0x7f48300051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:40.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.813+0000 7f484d400640 1 --2- 172.21.15.119:0/3865443204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48300039b0 0x7f4830003dd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f483c009ac0 tx=0x7f4830005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04680b988446e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.814+0000 7f4847e00640 1 -- 172.21.15.119:0/3865443204 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c036070 con 0x7f48300039b0 2024-04-25T10:09:40.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.814+0000 7f4847e00640 1 -- 172.21.15.119:0/3865443204 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c031400 con 0x7f48300039b0 2024-04-25T10:09:40.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.814+0000 7f4847e00640 1 -- 172.21.15.119:0/3865443204 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c009690 con 0x7f48300039b0 2024-04-25T10:09:40.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.814+0000 7f484fe00640 1 -- 172.21.15.119:0/3865443204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48300039b0 msgr2=0x7f4830003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.814+0000 7f484fe00640 1 --2- 172.21.15.119:0/3865443204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48300039b0 0x7f4830003dd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f483c009ac0 tx=0x7f4830005830 comp rx=0 tx=0).stop 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.814+0000 7f484de00640 1 -- 172.21.15.119:0/3865443204 reap_dead start 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.814+0000 7f484fe00640 1 -- 172.21.15.119:0/3865443204 shutdown_connections 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.814+0000 7f484fe00640 1 -- 172.21.15.119:0/3865443204 >> 172.21.15.119:0/3865443204 conn(0x7f484806aba0 msgr2=0x7f484806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.814+0000 7f484fe00640 1 -- 172.21.15.119:0/3865443204 shutdown_connections 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.814+0000 7f484fe00640 1 -- 172.21.15.119:0/3865443204 wait complete. 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.816+0000 7f484fe00640 1 Processor -- start 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.816+0000 7f484fe00640 1 -- start start 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.817+0000 7f484fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48300039b0 0x7f4848153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.817+0000 7f484fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48480acf60 0x7f48481fba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.817+0000 7f484fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4848164910 0x7f4848164dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.817+0000 7f484fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4848163110 con 0x7f48480acf60 2024-04-25T10:09:40.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.817+0000 7f484fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4848162f90 con 0x7f4848164910 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.817+0000 7f484fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4848163290 con 0x7f48300039b0 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.818+0000 7f484ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48480acf60 0x7f48481fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.818+0000 7f484ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48480acf60 0x7f48481fba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54656/0 (socket says 172.21.15.119:54656) 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.818+0000 7f484ca00640 1 -- 172.21.15.119:0/1208174873 learned_addr learned my addr 172.21.15.119:0/1208174873 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.818+0000 7f484de00640 1 --2- 172.21.15.119:0/1208174873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4848164910 0x7f4848164dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.818+0000 7f484d400640 1 --2- 172.21.15.119:0/1208174873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48300039b0 0x7f4848153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.818+0000 7f484ca00640 1 -- 172.21.15.119:0/1208174873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48300039b0 msgr2=0x7f4848153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.818+0000 7f484ca00640 1 --2- 172.21.15.119:0/1208174873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48300039b0 0x7f4848153b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.818+0000 7f484ca00640 1 -- 172.21.15.119:0/1208174873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4848164910 msgr2=0x7f4848164dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.818+0000 7f484ca00640 1 --2- 172.21.15.119:0/1208174873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4848164910 0x7f4848164dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.818+0000 7f484ca00640 1 -- 172.21.15.119:0/1208174873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48481fbf60 con 0x7f48480acf60 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.818+0000 7f484ca00640 1 --2- 172.21.15.119:0/1208174873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48480acf60 0x7f48481fba20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f483400c730 tx=0x7f483400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.819+0000 7f4845600640 1 -- 172.21.15.119:0/1208174873 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834018070 con 0x7f48480acf60 2024-04-25T10:09:40.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.819+0000 7f4845600640 1 -- 172.21.15.119:0/1208174873 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48340072d0 con 0x7f48480acf60 2024-04-25T10:09:40.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.819+0000 7f484fe00640 1 -- 172.21.15.119:0/1208174873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48480ad400 con 0x7f48480acf60 2024-04-25T10:09:40.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.819+0000 7f4845600640 1 -- 172.21.15.119:0/1208174873 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48340047e0 con 0x7f48480acf60 2024-04-25T10:09:40.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.819+0000 7f484fe00640 1 -- 172.21.15.119:0/1208174873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48480ad9c0 con 0x7f48480acf60 2024-04-25T10:09:40.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.820+0000 7f484fe00640 1 -- 172.21.15.119:0/1208174873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48480750b0 con 0x7f48480acf60 2024-04-25T10:09:40.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.821+0000 7f4845600640 1 -- 172.21.15.119:0/1208174873 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f483401b070 con 0x7f48480acf60 2024-04-25T10:09:40.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.822+0000 7f4845600640 1 --2- 172.21.15.119:0/1208174873 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f482806b500 0x7f482806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:40.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.822+0000 7f4845600640 1 -- 172.21.15.119:0/1208174873 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4834089640 con 0x7f48480acf60 2024-04-25T10:09:40.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.822+0000 7f484d400640 1 --2- 172.21.15.119:0/1208174873 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f482806b500 0x7f482806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:40.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.823+0000 7f484d400640 1 --2- 172.21.15.119:0/1208174873 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f482806b500 0x7f482806d9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f483c033000 tx=0x7f483c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:40.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:40.826+0000 7f4845600640 1 -- 172.21.15.119:0/1208174873 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48340584b0 con 0x7f48480acf60 2024-04-25T10:09:41.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.022+0000 7f484fe00640 1 -- 172.21.15.119:0/1208174873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f484806b140 con 0x7f48480acf60 2024-04-25T10:09:41.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.023+0000 7f4845600640 1 -- 172.21.15.119:0/1208174873 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+11 (secure 0 0 0) 0x7f483405ce20 con 0x7f48480acf60 2024-04-25T10:09:41.030 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:09:41.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.026+0000 7f484fe00640 1 -- 172.21.15.119:0/1208174873 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f482806b500 msgr2=0x7f482806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.026+0000 7f484fe00640 1 --2- 172.21.15.119:0/1208174873 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f482806b500 0x7f482806d9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f483c033000 tx=0x7f483c008030 comp rx=0 tx=0).stop 2024-04-25T10:09:41.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.026+0000 7f484fe00640 1 -- 172.21.15.119:0/1208174873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48480acf60 msgr2=0x7f48481fba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.026+0000 7f484fe00640 1 --2- 172.21.15.119:0/1208174873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48480acf60 0x7f48481fba20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f483400c730 tx=0x7f483400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.026+0000 7f484fe00640 1 -- 172.21.15.119:0/1208174873 shutdown_connections 2024-04-25T10:09:41.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.026+0000 7f484fe00640 1 --2- 172.21.15.119:0/1208174873 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f482806b500 0x7f482806d9c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.026+0000 7f484fe00640 1 --2- 172.21.15.119:0/1208174873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4848164910 0x7f4848164dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.026+0000 7f484fe00640 1 --2- 172.21.15.119:0/1208174873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48480acf60 0x7f48481fba20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.026+0000 7f484fe00640 1 --2- 172.21.15.119:0/1208174873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48300039b0 0x7f4848153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.026+0000 7f484fe00640 1 -- 172.21.15.119:0/1208174873 >> 172.21.15.119:0/1208174873 conn(0x7f484806aba0 msgr2=0x7f4848162780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:41.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.026+0000 7f484fe00640 1 -- 172.21.15.119:0/1208174873 shutdown_connections 2024-04-25T10:09:41.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.027+0000 7f484fe00640 1 -- 172.21.15.119:0/1208174873 wait complete. 2024-04-25T10:09:41.051 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:09:41.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.245+0000 7fe60ac00640 1 Processor -- start 2024-04-25T10:09:41.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.246+0000 7fe60ac00640 1 -- start start 2024-04-25T10:09:41.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe60ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6040aedc0 0x7fe60415ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe60ac00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe60406d260 0x7fe6040b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe60ac00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe6040b4730 0x7fe6040b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe60ac00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6040b4ba0 con 0x7fe6040b4730 2024-04-25T10:09:41.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe60ac00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60415c050 con 0x7fe60406d260 2024-04-25T10:09:41.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe60ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6040af1a0 con 0x7fe6040aedc0 2024-04-25T10:09:41.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe608c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe6040b4730 0x7fe6040b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe608c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe6040b4730 0x7fe6040b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39142/0 (socket says 172.21.15.119:39142) 2024-04-25T10:09:41.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe608c00640 1 -- 172.21.15.119:0/320688142 learned_addr learned my addr 172.21.15.119:0/320688142 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:41.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe603e00640 1 --2- 172.21.15.119:0/320688142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6040aedc0 0x7fe60415ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe608c00640 1 -- 172.21.15.119:0/320688142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6040aedc0 msgr2=0x7fe60415ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:41.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe608c00640 1 --2- 172.21.15.119:0/320688142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6040aedc0 0x7fe60415ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe603400640 1 --2- 172.21.15.119:0/320688142 >> v2:172.21.15.132:3300/0 conn(0x7fe60406d260 0x7fe6040b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe608c00640 1 -- 172.21.15.119:0/320688142 >> v2:172.21.15.132:3300/0 conn(0x7fe60406d260 msgr2=0x7fe6040b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:41.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.248+0000 7fe608c00640 1 --2- 172.21.15.119:0/320688142 >> v2:172.21.15.132:3300/0 conn(0x7fe60406d260 0x7fe6040b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.249+0000 7fe608c00640 1 -- 172.21.15.119:0/320688142 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60406da40 con 0x7fe6040b4730 2024-04-25T10:09:41.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.249+0000 7fe603400640 1 --2- 172.21.15.119:0/320688142 >> v2:172.21.15.132:3300/0 conn(0x7fe60406d260 0x7fe6040b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:41.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.250+0000 7fe608c00640 1 --2- 172.21.15.119:0/320688142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6040b4730 0x7fe6040b26f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe60406d0e0 tx=0x7fe5fc02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70acd7140407239a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:41.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.250+0000 7fe602a00640 1 -- 172.21.15.119:0/320688142 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc036070 con 0x7fe6040b4730 2024-04-25T10:09:41.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.250+0000 7fe602a00640 1 -- 172.21.15.119:0/320688142 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc031600 con 0x7fe6040b4730 2024-04-25T10:09:41.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.250+0000 7fe602a00640 1 -- 172.21.15.119:0/320688142 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc0319d0 con 0x7fe6040b4730 2024-04-25T10:09:41.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.251+0000 7fe60ac00640 1 -- 172.21.15.119:0/320688142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6040b4730 msgr2=0x7fe6040b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.251+0000 7fe60ac00640 1 --2- 172.21.15.119:0/320688142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6040b4730 0x7fe6040b26f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe60406d0e0 tx=0x7fe5fc02a630 comp rx=0 tx=0).stop 2024-04-25T10:09:41.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.251+0000 7fe60ac00640 1 -- 172.21.15.119:0/320688142 shutdown_connections 2024-04-25T10:09:41.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.251+0000 7fe60ac00640 1 --2- 172.21.15.119:0/320688142 >> v2:172.21.15.132:3300/0 conn(0x7fe60406d260 0x7fe6040b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.251+0000 7fe60ac00640 1 --2- 172.21.15.119:0/320688142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6040b4730 0x7fe6040b26f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.251+0000 7fe60ac00640 1 --2- 172.21.15.119:0/320688142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6040aedc0 0x7fe60415ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.251+0000 7fe60ac00640 1 -- 172.21.15.119:0/320688142 >> 172.21.15.119:0/320688142 conn(0x7fe60406aba0 msgr2=0x7fe60406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:41.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.251+0000 7fe60ac00640 1 -- 172.21.15.119:0/320688142 shutdown_connections 2024-04-25T10:09:41.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.251+0000 7fe60ac00640 1 -- 172.21.15.119:0/320688142 wait complete. 2024-04-25T10:09:41.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.253+0000 7fe60ac00640 1 Processor -- start 2024-04-25T10:09:41.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.253+0000 7fe60ac00640 1 -- start start 2024-04-25T10:09:41.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.254+0000 7fe60ac00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe60406d260 0x7fe6040b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.254+0000 7fe60ac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6040aedc0 0x7fe60414c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.254+0000 7fe60ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6040b4730 0x7fe604158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.254+0000 7fe60ac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60415c2a0 con 0x7fe6040aedc0 2024-04-25T10:09:41.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.254+0000 7fe60ac00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60406e000 con 0x7fe60406d260 2024-04-25T10:09:41.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.254+0000 7fe60ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60406d6c0 con 0x7fe6040b4730 2024-04-25T10:09:41.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.255+0000 7fe603400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6040aedc0 0x7fe60414c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.255+0000 7fe603400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6040aedc0 0x7fe60414c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39144/0 (socket says 172.21.15.119:39144) 2024-04-25T10:09:41.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.255+0000 7fe603400640 1 -- 172.21.15.119:0/3346224808 learned_addr learned my addr 172.21.15.119:0/3346224808 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:41.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.255+0000 7fe603e00640 1 --2- 172.21.15.119:0/3346224808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe60406d260 0x7fe6040b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.255+0000 7fe608c00640 1 --2- 172.21.15.119:0/3346224808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6040b4730 0x7fe604158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.255+0000 7fe603400640 1 -- 172.21.15.119:0/3346224808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6040b4730 msgr2=0x7fe604158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.255+0000 7fe603400640 1 --2- 172.21.15.119:0/3346224808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6040b4730 0x7fe604158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.255+0000 7fe603400640 1 -- 172.21.15.119:0/3346224808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe60406d260 msgr2=0x7fe6040b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.255+0000 7fe603400640 1 --2- 172.21.15.119:0/3346224808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe60406d260 0x7fe6040b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.255+0000 7fe603400640 1 -- 172.21.15.119:0/3346224808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6040c4a40 con 0x7fe6040aedc0 2024-04-25T10:09:41.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.255+0000 7fe603400640 1 --2- 172.21.15.119:0/3346224808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6040aedc0 0x7fe60414c160 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000c7f0 tx=0x7fe5f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:41.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.256+0000 7fe5fbe00640 1 -- 172.21.15.119:0/3346224808 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0015070 con 0x7fe6040aedc0 2024-04-25T10:09:41.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.256+0000 7fe5fbe00640 1 -- 172.21.15.119:0/3346224808 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f0007a30 con 0x7fe6040aedc0 2024-04-25T10:09:41.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.256+0000 7fe5fbe00640 1 -- 172.21.15.119:0/3346224808 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0007e00 con 0x7fe6040aedc0 2024-04-25T10:09:41.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.256+0000 7fe60ac00640 1 -- 172.21.15.119:0/3346224808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6040b2510 con 0x7fe6040aedc0 2024-04-25T10:09:41.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.256+0000 7fe60ac00640 1 -- 172.21.15.119:0/3346224808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6040b2ad0 con 0x7fe6040aedc0 2024-04-25T10:09:41.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.257+0000 7fe60ac00640 1 -- 172.21.15.119:0/3346224808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6040750b0 con 0x7fe6040aedc0 2024-04-25T10:09:41.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.258+0000 7fe5fbe00640 1 -- 172.21.15.119:0/3346224808 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5f0018070 con 0x7fe6040aedc0 2024-04-25T10:09:41.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.259+0000 7fe5fbe00640 1 --2- 172.21.15.119:0/3346224808 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe5e406b4d0 0x7fe5e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:41.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.259+0000 7fe5fbe00640 1 -- 172.21.15.119:0/3346224808 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5f008a150 con 0x7fe6040aedc0 2024-04-25T10:09:41.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.259+0000 7fe603e00640 1 --2- 172.21.15.119:0/3346224808 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe5e406b4d0 0x7fe5e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:41.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.260+0000 7fe603e00640 1 --2- 172.21.15.119:0/3346224808 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe5e406b4d0 0x7fe5e406d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4006fd0 tx=0x7fe5f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:41.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.263+0000 7fe5fbe00640 1 -- 172.21.15.119:0/3346224808 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5f0058fc0 con 0x7fe6040aedc0 2024-04-25T10:09:41.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.469+0000 7fe60ac00640 1 -- 172.21.15.119:0/3346224808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe6040d0330 con 0x7fe6040aedc0 2024-04-25T10:09:41.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.470+0000 7fe5fbe00640 1 -- 172.21.15.119:0/3346224808 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v44) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe5f00588c0 con 0x7fe6040aedc0 2024-04-25T10:09:41.476 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2024-04-25T10:09:41.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.473+0000 7fe60ac00640 1 -- 172.21.15.119:0/3346224808 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe5e406b4d0 msgr2=0x7fe5e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.473+0000 7fe60ac00640 1 --2- 172.21.15.119:0/3346224808 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe5e406b4d0 0x7fe5e406d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4006fd0 tx=0x7fe5f4008040 comp rx=0 tx=0).stop 2024-04-25T10:09:41.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.473+0000 7fe60ac00640 1 -- 172.21.15.119:0/3346224808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6040aedc0 msgr2=0x7fe60414c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:41.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.473+0000 7fe60ac00640 1 --2- 172.21.15.119:0/3346224808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6040aedc0 0x7fe60414c160 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000c7f0 tx=0x7fe5f000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.473+0000 7fe60ac00640 1 -- 172.21.15.119:0/3346224808 shutdown_connections 2024-04-25T10:09:41.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.473+0000 7fe60ac00640 1 --2- 172.21.15.119:0/3346224808 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe5e406b4d0 0x7fe5e406d990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.473+0000 7fe60ac00640 1 --2- 172.21.15.119:0/3346224808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6040b4730 0x7fe604158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.473+0000 7fe60ac00640 1 --2- 172.21.15.119:0/3346224808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6040aedc0 0x7fe60414c160 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.473+0000 7fe60ac00640 1 --2- 172.21.15.119:0/3346224808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe60406d260 0x7fe6040b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:41.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.473+0000 7fe60ac00640 1 -- 172.21.15.119:0/3346224808 >> 172.21.15.119:0/3346224808 conn(0x7fe60406aba0 msgr2=0x7fe604071a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:41.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.473+0000 7fe60ac00640 1 -- 172.21.15.119:0/3346224808 shutdown_connections 2024-04-25T10:09:41.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:41.473+0000 7fe60ac00640 1 -- 172.21.15.119:0/3346224808 wait complete. 2024-04-25T10:09:41.499 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:09:41.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:41.500 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:41.644 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:09:41.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:41.644 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:09:41.696 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:09:41.696 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:41.737 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:09:41.737 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:09:41.738 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:41.778 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:09:41.778 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:09:41.778 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:41.826 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:09:41.826 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:42.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.020+0000 7f24c4800640 1 Processor -- start 2024-04-25T10:09:42.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.020+0000 7f24c4800640 1 -- start start 2024-04-25T10:09:42.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.021+0000 7f24c4800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24bc15bf00 0x7f24bc0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.021+0000 7f24c4800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f24bc0cc310 0x7f24bc15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.022+0000 7f24c4800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24bc0ce6c0 con 0x7f24bc15c820 2024-04-25T10:09:42.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.022+0000 7f24c4800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24bc1455e0 con 0x7f24bc0cc310 2024-04-25T10:09:42.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.022+0000 7f24c4800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc06d450 con 0x7f24bc15bf00 2024-04-25T10:09:42.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.022+0000 7f24c2800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f24bc15c820 0x7f24bc06d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58342/0 (socket says 172.21.15.119:58342) 2024-04-25T10:09:42.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.022+0000 7f24c2800640 1 -- 172.21.15.119:0/707876774 learned_addr learned my addr 172.21.15.119:0/707876774 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:42.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.022+0000 7f24c1400640 1 --2- 172.21.15.119:0/707876774 >> v2:172.21.15.119:3300/0 conn(0x7f24bc0cc310 0x7f24bc15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.022+0000 7f24c1e00640 1 --2- 172.21.15.119:0/707876774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24bc15bf00 0x7f24bc0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.022+0000 7f24c0a00640 1 -- 172.21.15.119:0/707876774 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997030246 0 0) 0x7f24b8001490 con 0x7f24bc15c820 2024-04-25T10:09:42.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.023+0000 7f24c0a00640 1 -- 172.21.15.119:0/707876774 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24a4003520 con 0x7f24bc15c820 2024-04-25T10:09:42.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.023+0000 7f24c1400640 1 -- 172.21.15.119:0/707876774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24bc15bf00 msgr2=0x7f24bc0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:42.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.023+0000 7f24c1400640 1 --2- 172.21.15.119:0/707876774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24bc15bf00 0x7f24bc0cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.023+0000 7f24c1400640 1 -- 172.21.15.119:0/707876774 >> v1:172.21.15.119:6789/0 conn(0x7f24bc15c820 legacy=0x7f24bc06d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.023+0000 7f24c1400640 1 -- 172.21.15.119:0/707876774 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24bc0cc7b0 con 0x7f24bc0cc310 2024-04-25T10:09:42.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.024+0000 7f24c1400640 1 --2- 172.21.15.119:0/707876774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24bc0cc310 0x7f24bc15c2e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0097a0 tx=0x7f24ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5685db558265bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:42.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.024+0000 7f24c0a00640 1 -- 172.21.15.119:0/707876774 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac037070 con 0x7f24bc0cc310 2024-04-25T10:09:42.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.024+0000 7f24c0a00640 1 -- 172.21.15.119:0/707876774 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ac032440 con 0x7f24bc0cc310 2024-04-25T10:09:42.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.024+0000 7f24c0a00640 1 -- 172.21.15.119:0/707876774 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac032810 con 0x7f24bc0cc310 2024-04-25T10:09:42.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.025+0000 7f24c4800640 1 -- 172.21.15.119:0/707876774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24bc0cc310 msgr2=0x7f24bc15c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.025+0000 7f24c4800640 1 --2- 172.21.15.119:0/707876774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24bc0cc310 0x7f24bc15c2e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0097a0 tx=0x7f24ac02a180 comp rx=0 tx=0).stop 2024-04-25T10:09:42.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.025+0000 7f24c4800640 1 -- 172.21.15.119:0/707876774 shutdown_connections 2024-04-25T10:09:42.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.025+0000 7f24c4800640 1 --2- 172.21.15.119:0/707876774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24bc0cc310 0x7f24bc15c2e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.025+0000 7f24c4800640 1 --2- 172.21.15.119:0/707876774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24bc15bf00 0x7f24bc0cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.025+0000 7f24c4800640 1 -- 172.21.15.119:0/707876774 >> 172.21.15.119:0/707876774 conn(0x7f24bc06aba0 msgr2=0x7f24bc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:42.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.025+0000 7f24c4800640 1 -- 172.21.15.119:0/707876774 shutdown_connections 2024-04-25T10:09:42.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.025+0000 7f24c4800640 1 -- 172.21.15.119:0/707876774 wait complete. 2024-04-25T10:09:42.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.026+0000 7f24c4800640 1 Processor -- start 2024-04-25T10:09:42.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.027+0000 7f24c4800640 1 -- start start 2024-04-25T10:09:42.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.027+0000 7f24c4800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24bc0cc310 0x7f24bc1e4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.028+0000 7f24c4800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24bc15bf00 0x7f24bc138050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.028+0000 7f24c4800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24bc15c820 0x7f24bc140900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.028+0000 7f24c4800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc0cd050 con 0x7f24bc0cc310 2024-04-25T10:09:42.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.028+0000 7f24c4800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc0ccd70 con 0x7f24bc15c820 2024-04-25T10:09:42.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.028+0000 7f24c4800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc15cca0 con 0x7f24bc15bf00 2024-04-25T10:09:42.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.028+0000 7f24c1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24bc0cc310 0x7f24bc1e4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.028+0000 7f24c1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24bc0cc310 0x7f24bc1e4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39176/0 (socket says 172.21.15.119:39176) 2024-04-25T10:09:42.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.028+0000 7f24c1e00640 1 -- 172.21.15.119:0/2918470631 learned_addr learned my addr 172.21.15.119:0/2918470631 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:42.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.028+0000 7f24c1e00640 1 -- 172.21.15.119:0/2918470631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24bc15bf00 msgr2=0x7f24bc138050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.028+0000 7f24c1e00640 1 --2- 172.21.15.119:0/2918470631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24bc15bf00 0x7f24bc138050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.028+0000 7f24c1e00640 1 -- 172.21.15.119:0/2918470631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24bc15c820 msgr2=0x7f24bc140900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.028+0000 7f24c1e00640 1 --2- 172.21.15.119:0/2918470631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24bc15c820 0x7f24bc140900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.028+0000 7f24c1e00640 1 -- 172.21.15.119:0/2918470631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24bc141150 con 0x7f24bc0cc310 2024-04-25T10:09:42.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.029+0000 7f24c1e00640 1 --2- 172.21.15.119:0/2918470631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24bc0cc310 0x7f24bc1e4190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f24b000e9e0 tx=0x7f24b000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:42.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.029+0000 7f24b6000640 1 -- 172.21.15.119:0/2918470631 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0017070 con 0x7f24bc0cc310 2024-04-25T10:09:42.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.029+0000 7f24b6000640 1 -- 172.21.15.119:0/2918470631 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b000c9d0 con 0x7f24bc0cc310 2024-04-25T10:09:42.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.029+0000 7f24b6000640 1 -- 172.21.15.119:0/2918470631 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b000cdc0 con 0x7f24bc0cc310 2024-04-25T10:09:42.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.029+0000 7f24c4800640 1 -- 172.21.15.119:0/2918470631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24bc141430 con 0x7f24bc0cc310 2024-04-25T10:09:42.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.029+0000 7f24c4800640 1 -- 172.21.15.119:0/2918470631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24bc1419f0 con 0x7f24bc0cc310 2024-04-25T10:09:42.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.030+0000 7f24c4800640 1 -- 172.21.15.119:0/2918470631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24bc0749b0 con 0x7f24bc0cc310 2024-04-25T10:09:42.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.031+0000 7f24b6000640 1 -- 172.21.15.119:0/2918470631 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24b001a070 con 0x7f24bc0cc310 2024-04-25T10:09:42.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.032+0000 7f24b6000640 1 --2- 172.21.15.119:0/2918470631 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f24a006b4d0 0x7f24a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.032+0000 7f24b6000640 1 -- 172.21.15.119:0/2918470631 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24b008a650 con 0x7f24bc0cc310 2024-04-25T10:09:42.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.032+0000 7f24c1400640 1 --2- 172.21.15.119:0/2918470631 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f24a006b4d0 0x7f24a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.033+0000 7f24c1400640 1 --2- 172.21.15.119:0/2918470631 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f24a006b4d0 0x7f24a006d990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f24ac034000 tx=0x7f24ac02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:42.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.036+0000 7f24b6000640 1 -- 172.21.15.119:0/2918470631 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f24b00594c0 con 0x7f24bc0cc310 2024-04-25T10:09:42.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.269+0000 7f24c4800640 1 -- 172.21.15.119:0/2918470631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24bc071ac0 con 0x7f24bc0cc310 2024-04-25T10:09:42.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.270+0000 7f24b6000640 1 -- 172.21.15.119:0/2918470631 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f24b0058dc0 con 0x7f24bc0cc310 2024-04-25T10:09:42.276 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:42.276 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3317744182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3317744182},{"type":"v1","addr":"172.21.15.132:6833","nonce":3317744182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/700427287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":700427287},{"type":"v1","addr":"172.21.15.132:6835","nonce":700427287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2207174972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2207174972},{"type":"v1","addr":"172.21.15.119:6836","nonce":2207174972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:57.120451+0000","modified":"2024-04-25T10:09:00.157396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5478":{"gid":5478,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/270766821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":270766821},{"type":"v1","addr":"172.21.15.119:6837","nonce":270766821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:09:42.277 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 25 2024-04-25T10:09:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.272+0000 7f24c4800640 1 -- 172.21.15.119:0/2918470631 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f24a006b4d0 msgr2=0x7f24a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.272+0000 7f24c4800640 1 --2- 172.21.15.119:0/2918470631 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f24a006b4d0 0x7f24a006d990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f24ac034000 tx=0x7f24ac02c000 comp rx=0 tx=0).stop 2024-04-25T10:09:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.273+0000 7f24c4800640 1 -- 172.21.15.119:0/2918470631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24bc0cc310 msgr2=0x7f24bc1e4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.273+0000 7f24c4800640 1 --2- 172.21.15.119:0/2918470631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24bc0cc310 0x7f24bc1e4190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f24b000e9e0 tx=0x7f24b000eea0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.273+0000 7f24c4800640 1 -- 172.21.15.119:0/2918470631 shutdown_connections 2024-04-25T10:09:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.273+0000 7f24c4800640 1 --2- 172.21.15.119:0/2918470631 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f24a006b4d0 0x7f24a006d990 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.273+0000 7f24c4800640 1 --2- 172.21.15.119:0/2918470631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24bc15c820 0x7f24bc140900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.273+0000 7f24c4800640 1 --2- 172.21.15.119:0/2918470631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24bc15bf00 0x7f24bc138050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.273+0000 7f24c4800640 1 --2- 172.21.15.119:0/2918470631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24bc0cc310 0x7f24bc1e4190 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.273+0000 7f24c4800640 1 -- 172.21.15.119:0/2918470631 >> 172.21.15.119:0/2918470631 conn(0x7f24bc06aba0 msgr2=0x7f24bc0cf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:42.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.273+0000 7f24c4800640 1 -- 172.21.15.119:0/2918470631 shutdown_connections 2024-04-25T10:09:42.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.273+0000 7f24c4800640 1 -- 172.21.15.119:0/2918470631 wait complete. 2024-04-25T10:09:42.299 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:42.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.490+0000 7fd37e000640 1 Processor -- start 2024-04-25T10:09:42.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.490+0000 7fd37e000640 1 -- start start 2024-04-25T10:09:42.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.492+0000 7fd37e000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd37815db60 0x7fd3780d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.492+0000 7fd37e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3780d2fa0 0x7fd3780d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.492+0000 7fd37e000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37815ef10 con 0x7fd3780b9270 2024-04-25T10:09:42.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.492+0000 7fd37e000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd37815e390 con 0x7fd37815db60 2024-04-25T10:09:42.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.492+0000 7fd37e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37815e670 con 0x7fd3780d2fa0 2024-04-25T10:09:42.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.492+0000 7fd36fe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd3780b9270 0x7fd3780b5360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58356/0 (socket says 172.21.15.119:58356) 2024-04-25T10:09:42.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.492+0000 7fd36fe00640 1 -- 172.21.15.119:0/95174353 learned_addr learned my addr 172.21.15.119:0/95174353 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:42.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd37cc00640 1 --2- 172.21.15.119:0/95174353 >> v2:172.21.15.132:3300/0 conn(0x7fd37815db60 0x7fd3780d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd37d600640 1 --2- 172.21.15.119:0/95174353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3780d2fa0 0x7fd3780d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 -- 172.21.15.119:0/95174353 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831845748 0 0) 0x7fd37815ef10 con 0x7fd3780b9270 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 -- 172.21.15.119:0/95174353 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd35c003520 con 0x7fd3780b9270 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 -- 172.21.15.119:0/95174353 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 492218973 0 0) 0x7fd35c003520 con 0x7fd3780b9270 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 -- 172.21.15.119:0/95174353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3780d2fa0 msgr2=0x7fd3780d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 --2- 172.21.15.119:0/95174353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3780d2fa0 0x7fd3780d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 -- 172.21.15.119:0/95174353 >> v2:172.21.15.132:3300/0 conn(0x7fd37815db60 msgr2=0x7fd3780d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 --2- 172.21.15.119:0/95174353 >> v2:172.21.15.132:3300/0 conn(0x7fd37815db60 0x7fd3780d3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 -- 172.21.15.119:0/95174353 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37815e950 con 0x7fd3780b9270 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 -- 172.21.15.119:0/95174353 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fd3600017f0 con 0x7fd3780b9270 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 -- 172.21.15.119:0/95174353 >> v1:172.21.15.119:6789/0 conn(0x7fd3780b9270 legacy=0x7fd3780b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 --2- 172.21.15.119:0/95174353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd35c003a20 0x7fd35c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 --2- 172.21.15.119:0/95174353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd35c004800 0x7fd35c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 --2- 172.21.15.119:0/95174353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3780d2fa0 0x7fd35c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 -- 172.21.15.119:0/95174353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37815e390 con 0x7fd35c003a20 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 -- 172.21.15.119:0/95174353 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37815e670 con 0x7fd35c004800 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.493+0000 7fd377e00640 1 -- 172.21.15.119:0/95174353 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c00b9c0 con 0x7fd3780d2fa0 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.494+0000 7fd37cc00640 1 --2- 172.21.15.119:0/95174353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd35c003a20 0x7fd35c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.494+0000 7fd37d600640 1 --2- 172.21.15.119:0/95174353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3780d2fa0 0x7fd35c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.494+0000 7fd37cc00640 1 -- 172.21.15.119:0/95174353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3780d2fa0 msgr2=0x7fd35c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.494+0000 7fd36fe00640 1 --2- 172.21.15.119:0/95174353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd35c004800 0x7fd35c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.494+0000 7fd37cc00640 1 --2- 172.21.15.119:0/95174353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3780d2fa0 0x7fd35c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.494+0000 7fd37cc00640 1 -- 172.21.15.119:0/95174353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd35c004800 msgr2=0x7fd35c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.494+0000 7fd37cc00640 1 --2- 172.21.15.119:0/95174353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd35c004800 0x7fd35c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.494+0000 7fd37cc00640 1 -- 172.21.15.119:0/95174353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c003520 con 0x7fd35c003a20 2024-04-25T10:09:42.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.494+0000 7fd37d600640 1 --2- 172.21.15.119:0/95174353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3780d2fa0 0x7fd35c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:42.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.495+0000 7fd37cc00640 1 --2- 172.21.15.119:0/95174353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd35c003a20 0x7fd35c003e40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd368009af0 tx=0x7fd35c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e70a8a572dfde67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:42.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.495+0000 7fd377e00640 1 -- 172.21.15.119:0/95174353 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368036070 con 0x7fd35c003a20 2024-04-25T10:09:42.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.495+0000 7fd377e00640 1 -- 172.21.15.119:0/95174353 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd368031400 con 0x7fd35c003a20 2024-04-25T10:09:42.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.495+0000 7fd377e00640 1 -- 172.21.15.119:0/95174353 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368009690 con 0x7fd35c003a20 2024-04-25T10:09:42.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.496+0000 7fd37e000640 1 -- 172.21.15.119:0/95174353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd35c003a20 msgr2=0x7fd35c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.496+0000 7fd37e000640 1 --2- 172.21.15.119:0/95174353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd35c003a20 0x7fd35c003e40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd368009af0 tx=0x7fd35c005830 comp rx=0 tx=0).stop 2024-04-25T10:09:42.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.496+0000 7fd37d600640 1 -- 172.21.15.119:0/95174353 reap_dead start 2024-04-25T10:09:42.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.496+0000 7fd37e000640 1 -- 172.21.15.119:0/95174353 shutdown_connections 2024-04-25T10:09:42.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.496+0000 7fd37e000640 1 -- 172.21.15.119:0/95174353 >> 172.21.15.119:0/95174353 conn(0x7fd3780726c0 msgr2=0x7fd378072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:42.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.496+0000 7fd37e000640 1 -- 172.21.15.119:0/95174353 shutdown_connections 2024-04-25T10:09:42.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.496+0000 7fd37e000640 1 -- 172.21.15.119:0/95174353 wait complete. 2024-04-25T10:09:42.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.498+0000 7fd37e000640 1 Processor -- start 2024-04-25T10:09:42.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.498+0000 7fd37e000640 1 -- start start 2024-04-25T10:09:42.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.499+0000 7fd37e000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd35c003a20 0x7fd37814f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.499+0000 7fd37e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3780cd330 0x7fd3780b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.499+0000 7fd37e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3780b5760 0x7fd3780d3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.499+0000 7fd37e000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780b3120 con 0x7fd3780cd330 2024-04-25T10:09:42.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.499+0000 7fd37e000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780b2fa0 con 0x7fd35c003a20 2024-04-25T10:09:42.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.499+0000 7fd37e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37815df40 con 0x7fd3780b5760 2024-04-25T10:09:42.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.499+0000 7fd36fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3780cd330 0x7fd3780b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.499+0000 7fd36fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3780cd330 0x7fd3780b5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39198/0 (socket says 172.21.15.119:39198) 2024-04-25T10:09:42.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.499+0000 7fd36fe00640 1 -- 172.21.15.119:0/326542463 learned_addr learned my addr 172.21.15.119:0/326542463 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.500+0000 7fd37d600640 1 --2- 172.21.15.119:0/326542463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3780b5760 0x7fd3780d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.500+0000 7fd37cc00640 1 --2- 172.21.15.119:0/326542463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd35c003a20 0x7fd37814f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.500+0000 7fd36fe00640 1 -- 172.21.15.119:0/326542463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3780b5760 msgr2=0x7fd3780d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.500+0000 7fd36fe00640 1 --2- 172.21.15.119:0/326542463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3780b5760 0x7fd3780d3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.500+0000 7fd36fe00640 1 -- 172.21.15.119:0/326542463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd35c003a20 msgr2=0x7fd37814f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.500+0000 7fd36fe00640 1 --2- 172.21.15.119:0/326542463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd35c003a20 0x7fd37814f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.500+0000 7fd36fe00640 1 -- 172.21.15.119:0/326542463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3781f70e0 con 0x7fd3780cd330 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.500+0000 7fd36fe00640 1 --2- 172.21.15.119:0/326542463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3780cd330 0x7fd3780b5220 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd36000d700 tx=0x7fd36000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.500+0000 7fd375600640 1 -- 172.21.15.119:0/326542463 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd360007270 con 0x7fd3780cd330 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.501+0000 7fd375600640 1 -- 172.21.15.119:0/326542463 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd360017070 con 0x7fd3780cd330 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.501+0000 7fd37e000640 1 -- 172.21.15.119:0/326542463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3781f73c0 con 0x7fd3780cd330 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.501+0000 7fd375600640 1 -- 172.21.15.119:0/326542463 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3600057a0 con 0x7fd3780cd330 2024-04-25T10:09:42.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.501+0000 7fd37e000640 1 -- 172.21.15.119:0/326542463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3781f7980 con 0x7fd3780cd330 2024-04-25T10:09:42.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.502+0000 7fd37e000640 1 -- 172.21.15.119:0/326542463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3780749b0 con 0x7fd3780cd330 2024-04-25T10:09:42.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.503+0000 7fd375600640 1 -- 172.21.15.119:0/326542463 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd360005940 con 0x7fd3780cd330 2024-04-25T10:09:42.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.504+0000 7fd375600640 1 --2- 172.21.15.119:0/326542463 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd35406b4d0 0x7fd35406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.504+0000 7fd375600640 1 -- 172.21.15.119:0/326542463 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd36008a1b0 con 0x7fd3780cd330 2024-04-25T10:09:42.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.504+0000 7fd37cc00640 1 --2- 172.21.15.119:0/326542463 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd35406b4d0 0x7fd35406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.505+0000 7fd37cc00640 1 --2- 172.21.15.119:0/326542463 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd35406b4d0 0x7fd35406d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd368033000 tx=0x7fd368008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:42.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.508+0000 7fd375600640 1 -- 172.21.15.119:0/326542463 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd360019310 con 0x7fd3780cd330 2024-04-25T10:09:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.688+0000 7fd37e000640 1 -- 172.21.15.119:0/326542463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3780d1a60 con 0x7fd3780cd330 2024-04-25T10:09:42.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.690+0000 7fd375600640 1 -- 172.21.15.119:0/326542463 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16649 (secure 0 0 0) 0x7fd360059020 con 0x7fd3780cd330 2024-04-25T10:09:42.697 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:42.697 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":44,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:58.637281+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:54.917692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:56.128440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:42.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.693+0000 7fd37e000640 1 -- 172.21.15.119:0/326542463 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd35406b4d0 msgr2=0x7fd35406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.693+0000 7fd37e000640 1 --2- 172.21.15.119:0/326542463 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd35406b4d0 0x7fd35406d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd368033000 tx=0x7fd368008030 comp rx=0 tx=0).stop 2024-04-25T10:09:42.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.693+0000 7fd37e000640 1 -- 172.21.15.119:0/326542463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3780cd330 msgr2=0x7fd3780b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.693+0000 7fd37e000640 1 --2- 172.21.15.119:0/326542463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3780cd330 0x7fd3780b5220 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd36000d700 tx=0x7fd36000dbc0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.693+0000 7fd37e000640 1 -- 172.21.15.119:0/326542463 shutdown_connections 2024-04-25T10:09:42.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.693+0000 7fd37e000640 1 --2- 172.21.15.119:0/326542463 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd35406b4d0 0x7fd35406d990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.693+0000 7fd37e000640 1 --2- 172.21.15.119:0/326542463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3780b5760 0x7fd3780d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.693+0000 7fd37e000640 1 --2- 172.21.15.119:0/326542463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3780cd330 0x7fd3780b5220 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.693+0000 7fd37e000640 1 --2- 172.21.15.119:0/326542463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd35c003a20 0x7fd37814f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.693+0000 7fd37e000640 1 -- 172.21.15.119:0/326542463 >> 172.21.15.119:0/326542463 conn(0x7fd3780726c0 msgr2=0x7fd3780c9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:42.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.693+0000 7fd37e000640 1 -- 172.21.15.119:0/326542463 shutdown_connections 2024-04-25T10:09:42.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.693+0000 7fd37e000640 1 -- 172.21.15.119:0/326542463 wait complete. 2024-04-25T10:09:42.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:54.917692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:56.128440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:42.717 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:42.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.907+0000 7ff252200640 1 Processor -- start 2024-04-25T10:09:42.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.907+0000 7ff252200640 1 -- start start 2024-04-25T10:09:42.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.908+0000 7ff252200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff24c144e70 0x7ff24c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.908+0000 7ff252200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff24c0cf0e0 0x7ff24c0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.908+0000 7ff252200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff24c06dcb0 con 0x7ff24c06d7a0 2024-04-25T10:09:42.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.908+0000 7ff252200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24c15bfe0 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.908+0000 7ff252200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c06df90 con 0x7ff24c144e70 2024-04-25T10:09:42.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.909+0000 7ff24be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff24c0cf0e0 0x7ff24c0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.909+0000 7ff24be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff24c0cf0e0 0x7ff24c0cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39200/0 (socket says 172.21.15.119:39200) 2024-04-25T10:09:42.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.909+0000 7ff24be00640 1 -- 172.21.15.119:0/3893780328 learned_addr learned my addr 172.21.15.119:0/3893780328 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:42.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.909+0000 7ff24b400640 1 --2- 172.21.15.119:0/3893780328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff24c144e70 0x7ff24c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.909+0000 7ff24be00640 1 -- 172.21.15.119:0/3893780328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff24c144e70 msgr2=0x7ff24c06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:42.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.909+0000 7ff24be00640 1 --2- 172.21.15.119:0/3893780328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff24c144e70 0x7ff24c06d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.909+0000 7ff24be00640 1 -- 172.21.15.119:0/3893780328 >> v1:172.21.15.132:6789/0 conn(0x7ff24c06d7a0 legacy=0x7ff24c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.909+0000 7ff24be00640 1 -- 172.21.15.119:0/3893780328 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c06e270 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.911+0000 7ff24be00640 1 --2- 172.21.15.119:0/3893780328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24c0cf0e0 0x7ff24c0cc300 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff244009e50 tx=0x7ff24402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41172fefae6809b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:42.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.911+0000 7ff24aa00640 1 -- 172.21.15.119:0/3893780328 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244036070 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.911+0000 7ff24aa00640 1 -- 172.21.15.119:0/3893780328 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff244031600 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.911+0000 7ff24aa00640 1 -- 172.21.15.119:0/3893780328 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2440319d0 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.911+0000 7ff252200640 1 -- 172.21.15.119:0/3893780328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24c0cf0e0 msgr2=0x7ff24c0cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.911+0000 7ff252200640 1 --2- 172.21.15.119:0/3893780328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24c0cf0e0 0x7ff24c0cc300 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff244009e50 tx=0x7ff24402a680 comp rx=0 tx=0).stop 2024-04-25T10:09:42.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.912+0000 7ff252200640 1 -- 172.21.15.119:0/3893780328 shutdown_connections 2024-04-25T10:09:42.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.912+0000 7ff252200640 1 --2- 172.21.15.119:0/3893780328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24c0cf0e0 0x7ff24c0cc300 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.912+0000 7ff252200640 1 --2- 172.21.15.119:0/3893780328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff24c144e70 0x7ff24c06d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.912+0000 7ff252200640 1 -- 172.21.15.119:0/3893780328 >> 172.21.15.119:0/3893780328 conn(0x7ff24c06aba0 msgr2=0x7ff24c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:42.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.912+0000 7ff252200640 1 -- 172.21.15.119:0/3893780328 shutdown_connections 2024-04-25T10:09:42.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.912+0000 7ff252200640 1 -- 172.21.15.119:0/3893780328 wait complete. 2024-04-25T10:09:42.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.913+0000 7ff252200640 1 Processor -- start 2024-04-25T10:09:42.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.914+0000 7ff252200640 1 -- start start 2024-04-25T10:09:42.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.914+0000 7ff252200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c06d7a0 0x7ff24c1f3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.914+0000 7ff252200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24c0cf0e0 0x7ff24c13b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.914+0000 7ff252200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff24c144e70 0x7ff24c13baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.915+0000 7ff252200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c145630 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.915+0000 7ff252200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c1454b0 con 0x7ff24c06d7a0 2024-04-25T10:09:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.915+0000 7ff252200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c0cc840 con 0x7ff24c144e70 2024-04-25T10:09:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.915+0000 7ff242a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24c0cf0e0 0x7ff24c13b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.915+0000 7ff242a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24c0cf0e0 0x7ff24c13b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39212/0 (socket says 172.21.15.119:39212) 2024-04-25T10:09:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.915+0000 7ff242a00640 1 -- 172.21.15.119:0/2111972038 learned_addr learned my addr 172.21.15.119:0/2111972038 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.915+0000 7ff24b400640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c06d7a0 0x7ff24c1f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.915+0000 7ff24be00640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff24c144e70 0x7ff24c13baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.915+0000 7ff242a00640 1 -- 172.21.15.119:0/2111972038 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff24c144e70 msgr2=0x7ff24c13baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.915+0000 7ff242a00640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff24c144e70 0x7ff24c13baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.915+0000 7ff242a00640 1 -- 172.21.15.119:0/2111972038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c06d7a0 msgr2=0x7ff24c1f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.915+0000 7ff242a00640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c06d7a0 0x7ff24c1f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.915+0000 7ff242a00640 1 -- 172.21.15.119:0/2111972038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c0b1160 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.915+0000 7ff24b400640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c06d7a0 0x7ff24c1f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:42.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.916+0000 7ff242a00640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24c0cf0e0 0x7ff24c13b5b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff23400c3c0 tx=0x7ff23400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:42.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.916+0000 7ff243e00640 1 -- 172.21.15.119:0/2111972038 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234017070 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.916+0000 7ff243e00640 1 -- 172.21.15.119:0/2111972038 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2340052f0 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.916+0000 7ff243e00640 1 -- 172.21.15.119:0/2111972038 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234003850 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.916+0000 7ff252200640 1 -- 172.21.15.119:0/2111972038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24c0b1440 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.916+0000 7ff252200640 1 -- 172.21.15.119:0/2111972038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24c0b1a00 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.917+0000 7ff252200640 1 -- 172.21.15.119:0/2111972038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff24c0750b0 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.918+0000 7ff243e00640 1 -- 172.21.15.119:0/2111972038 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff23401a070 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.919+0000 7ff243e00640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff22806b4d0 0x7ff22806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.919+0000 7ff243e00640 1 -- 172.21.15.119:0/2111972038 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2340895a0 con 0x7ff24c0cf0e0 2024-04-25T10:09:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.919+0000 7ff24b400640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff22806b4d0 0x7ff22806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.920+0000 7ff24b400640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff22806b4d0 0x7ff22806d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff23c006fd0 tx=0x7ff23c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:42.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:42.923+0000 7ff243e00640 1 -- 172.21.15.119:0/2111972038 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff234058380 con 0x7ff24c0cf0e0 2024-04-25T10:09:43.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.115+0000 7ff252200640 1 -- 172.21.15.119:0/2111972038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff24c0aebe0 con 0x7ff24c0cf0e0 2024-04-25T10:09:43.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.116+0000 7ff243e00640 1 -- 172.21.15.119:0/2111972038 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff234057c80 con 0x7ff24c0cf0e0 2024-04-25T10:09:43.122 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:09:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.119+0000 7ff252200640 1 -- 172.21.15.119:0/2111972038 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff22806b4d0 msgr2=0x7ff22806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.119+0000 7ff252200640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff22806b4d0 0x7ff22806d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff23c006fd0 tx=0x7ff23c008040 comp rx=0 tx=0).stop 2024-04-25T10:09:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.119+0000 7ff252200640 1 -- 172.21.15.119:0/2111972038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24c0cf0e0 msgr2=0x7ff24c13b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.119+0000 7ff252200640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24c0cf0e0 0x7ff24c13b5b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff23400c3c0 tx=0x7ff23400c880 comp rx=0 tx=0).stop 2024-04-25T10:09:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.119+0000 7ff252200640 1 -- 172.21.15.119:0/2111972038 shutdown_connections 2024-04-25T10:09:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.119+0000 7ff252200640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff22806b4d0 0x7ff22806d990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.119+0000 7ff252200640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff24c144e70 0x7ff24c13baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.119+0000 7ff252200640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24c0cf0e0 0x7ff24c13b5b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.119+0000 7ff252200640 1 --2- 172.21.15.119:0/2111972038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c06d7a0 0x7ff24c1f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.119+0000 7ff252200640 1 -- 172.21.15.119:0/2111972038 >> 172.21.15.119:0/2111972038 conn(0x7ff24c06aba0 msgr2=0x7ff24c143a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.119+0000 7ff252200640 1 -- 172.21.15.119:0/2111972038 shutdown_connections 2024-04-25T10:09:43.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.119+0000 7ff252200640 1 -- 172.21.15.119:0/2111972038 wait complete. 2024-04-25T10:09:43.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:09:43.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.321+0000 7f34d1a00640 1 Processor -- start 2024-04-25T10:09:43.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.322+0000 7f34d1a00640 1 -- start start 2024-04-25T10:09:43.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.323+0000 7f34d1a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc06aba0 0x7f34cc06af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.323+0000 7f34d1a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f34cc0ce2f0 0x7f34cc15e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.323+0000 7f34d1a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34cc06b4c0 con 0x7f34cc06f040 2024-04-25T10:09:43.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.323+0000 7f34d1a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34cc06d080 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.323+0000 7f34d1a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc06d360 con 0x7f34cc06aba0 2024-04-25T10:09:43.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.323+0000 7f34d1000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f34cc0ce2f0 0x7f34cc15e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.324+0000 7f34d1000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f34cc0ce2f0 0x7f34cc15e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39224/0 (socket says 172.21.15.119:39224) 2024-04-25T10:09:43.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.324+0000 7f34d1000640 1 -- 172.21.15.119:0/2515929587 learned_addr learned my addr 172.21.15.119:0/2515929587 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:43.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.324+0000 7f34cbe00640 1 --2- 172.21.15.119:0/2515929587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc06aba0 0x7f34cc06af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.324+0000 7f34d1000640 1 -- 172.21.15.119:0/2515929587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc06aba0 msgr2=0x7f34cc06af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:43.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.324+0000 7f34d1000640 1 --2- 172.21.15.119:0/2515929587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc06aba0 0x7f34cc06af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.324+0000 7f34d1000640 1 -- 172.21.15.119:0/2515929587 >> v1:172.21.15.132:6789/0 conn(0x7f34cc06f040 legacy=0x7f34cc15e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:43.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.324+0000 7f34d1000640 1 -- 172.21.15.119:0/2515929587 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc06d640 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.325+0000 7f34d1000640 1 --2- 172.21.15.119:0/2515929587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34cc0ce2f0 0x7f34cc15e8f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009870 tx=0x7f34c402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdcba344b1a23b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:43.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.325+0000 7f34caa00640 1 -- 172.21.15.119:0/2515929587 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4037070 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.325+0000 7f34caa00640 1 -- 172.21.15.119:0/2515929587 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c4032440 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.325+0000 7f34caa00640 1 -- 172.21.15.119:0/2515929587 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4032810 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.326+0000 7f34d1a00640 1 -- 172.21.15.119:0/2515929587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34cc0ce2f0 msgr2=0x7f34cc15e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.326+0000 7f34d1a00640 1 --2- 172.21.15.119:0/2515929587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34cc0ce2f0 0x7f34cc15e8f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009870 tx=0x7f34c402a210 comp rx=0 tx=0).stop 2024-04-25T10:09:43.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.326+0000 7f34d1a00640 1 -- 172.21.15.119:0/2515929587 shutdown_connections 2024-04-25T10:09:43.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.326+0000 7f34d1a00640 1 --2- 172.21.15.119:0/2515929587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34cc0ce2f0 0x7f34cc15e8f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.326+0000 7f34d1a00640 1 --2- 172.21.15.119:0/2515929587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc06aba0 0x7f34cc06af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.326+0000 7f34d1a00640 1 -- 172.21.15.119:0/2515929587 >> 172.21.15.119:0/2515929587 conn(0x7f34cc066ff0 msgr2=0x7f34cc0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:43.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.326+0000 7f34d1a00640 1 -- 172.21.15.119:0/2515929587 shutdown_connections 2024-04-25T10:09:43.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.326+0000 7f34d1a00640 1 -- 172.21.15.119:0/2515929587 wait complete. 2024-04-25T10:09:43.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.328+0000 7f34d1a00640 1 Processor -- start 2024-04-25T10:09:43.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.328+0000 7f34d1a00640 1 -- start start 2024-04-25T10:09:43.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34d1a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc06aba0 0x7f34cc15b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34d1a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc06f040 0x7f34cc1f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34d1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34cc0ce2f0 0x7f34cc1f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34d1a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc15a980 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34d1a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc06dc00 con 0x7f34cc06f040 2024-04-25T10:09:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34d1a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc0aed40 con 0x7f34cc06aba0 2024-04-25T10:09:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34d1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34cc0ce2f0 0x7f34cc1f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34d1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34cc0ce2f0 0x7f34cc1f7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39232/0 (socket says 172.21.15.119:39232) 2024-04-25T10:09:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34d1000640 1 -- 172.21.15.119:0/4066245992 learned_addr learned my addr 172.21.15.119:0/4066245992 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34cbe00640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc06aba0 0x7f34cc15b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34d1000640 1 -- 172.21.15.119:0/4066245992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc06aba0 msgr2=0x7f34cc15b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34d1000640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc06aba0 0x7f34cc15b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34d1000640 1 -- 172.21.15.119:0/4066245992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc06f040 msgr2=0x7f34cc1f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34cb400640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc06f040 0x7f34cc1f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.329+0000 7f34d1000640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc06f040 0x7f34cc1f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.330+0000 7f34d1000640 1 -- 172.21.15.119:0/4066245992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc15bb90 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.330+0000 7f34cb400640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc06f040 0x7f34cc1f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:43.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.330+0000 7f34d1000640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34cc0ce2f0 0x7f34cc1f7a90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f34c402a250 tx=0x7f34c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:43.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.330+0000 7f34c3e00640 1 -- 172.21.15.119:0/4066245992 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4041070 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.330+0000 7f34c3e00640 1 -- 172.21.15.119:0/4066245992 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c403c890 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.330+0000 7f34c3e00640 1 -- 172.21.15.119:0/4066245992 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4037040 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.330+0000 7f34d1a00640 1 -- 172.21.15.119:0/4066245992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34cc15be70 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.330+0000 7f34d1a00640 1 -- 172.21.15.119:0/4066245992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34cc15c430 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.331+0000 7f34d1a00640 1 -- 172.21.15.119:0/4066245992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34cc0749b0 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.332+0000 7f34c3e00640 1 -- 172.21.15.119:0/4066245992 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34c4044070 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.333+0000 7f34c3e00640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f34a806b4d0 0x7f34a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.333+0000 7f34c3e00640 1 -- 172.21.15.119:0/4066245992 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34c40aaf80 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.333+0000 7f34cbe00640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f34a806b4d0 0x7f34a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.334+0000 7f34cbe00640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f34a806b4d0 0x7f34a806d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f34bc006fd0 tx=0x7f34bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:43.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.337+0000 7f34c3e00640 1 -- 172.21.15.119:0/4066245992 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34c4079f10 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.531+0000 7f34d1a00640 1 -- 172.21.15.119:0/4066245992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34cc071d00 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.532+0000 7f34c3e00640 1 -- 172.21.15.119:0/4066245992 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+11 (secure 0 0 0) 0x7f34c4079810 con 0x7f34cc0ce2f0 2024-04-25T10:09:43.538 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:09:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.535+0000 7f34d1a00640 1 -- 172.21.15.119:0/4066245992 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f34a806b4d0 msgr2=0x7f34a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.535+0000 7f34d1a00640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f34a806b4d0 0x7f34a806d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f34bc006fd0 tx=0x7f34bc008040 comp rx=0 tx=0).stop 2024-04-25T10:09:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.535+0000 7f34d1a00640 1 -- 172.21.15.119:0/4066245992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34cc0ce2f0 msgr2=0x7f34cc1f7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.535+0000 7f34d1a00640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34cc0ce2f0 0x7f34cc1f7a90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f34c402a250 tx=0x7f34c40046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.535+0000 7f34d1a00640 1 -- 172.21.15.119:0/4066245992 shutdown_connections 2024-04-25T10:09:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.535+0000 7f34d1a00640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f34a806b4d0 0x7f34a806d990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.535+0000 7f34d1a00640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34cc0ce2f0 0x7f34cc1f7a90 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.535+0000 7f34d1a00640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc06f040 0x7f34cc1f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.535+0000 7f34d1a00640 1 --2- 172.21.15.119:0/4066245992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc06aba0 0x7f34cc15b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.535+0000 7f34d1a00640 1 -- 172.21.15.119:0/4066245992 >> 172.21.15.119:0/4066245992 conn(0x7f34cc066ff0 msgr2=0x7f34cc15cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:43.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.535+0000 7f34d1a00640 1 -- 172.21.15.119:0/4066245992 shutdown_connections 2024-04-25T10:09:43.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.535+0000 7f34d1a00640 1 -- 172.21.15.119:0/4066245992 wait complete. 2024-04-25T10:09:43.559 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:09:43.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.743+0000 7fb0d5200640 1 Processor -- start 2024-04-25T10:09:43.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.744+0000 7fb0d5200640 1 -- start start 2024-04-25T10:09:43.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.745+0000 7fb0d5200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb0d015b7e0 0x7fb0d015e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.745+0000 7fb0d5200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d00b5710 con 0x7fb0d00d07b0 2024-04-25T10:09:43.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.745+0000 7fb0d5200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d00b59f0 con 0x7fb0d00ce860 2024-04-25T10:09:43.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.745+0000 7fb0d5200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d00b5cd0 con 0x7fb0d015b7e0 2024-04-25T10:09:43.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.745+0000 7fb0cf400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb0d015b7e0 0x7fb0d015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.745+0000 7fb0cfe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb0d00d07b0 0x7fb0d0161960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58364/0 (socket says 172.21.15.119:58364) 2024-04-25T10:09:43.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.745+0000 7fb0cfe00640 1 -- 172.21.15.119:0/3203507154 learned_addr learned my addr 172.21.15.119:0/3203507154 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:43.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.746+0000 7fb0cf400640 1 -- 172.21.15.119:0/3203507154 >> v1:172.21.15.132:6789/0 conn(0x7fb0d00ce860 legacy=0x7fb0d00d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:43.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.746+0000 7fb0cf400640 1 -- 172.21.15.119:0/3203507154 >> v1:172.21.15.119:6789/0 conn(0x7fb0d00d07b0 legacy=0x7fb0d0161960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.746+0000 7fb0cf400640 1 -- 172.21.15.119:0/3203507154 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d00b5fb0 con 0x7fb0d015b7e0 2024-04-25T10:09:43.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.747+0000 7fb0cf400640 1 --2- 172.21.15.119:0/3203507154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d015b7e0 0x7fb0d015e000 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00097a0 tx=0x7fb0c002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9135195e2556cfa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:43.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.747+0000 7fb0ce000640 1 -- 172.21.15.119:0/3203507154 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0037070 con 0x7fb0d015b7e0 2024-04-25T10:09:43.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.747+0000 7fb0ce000640 1 -- 172.21.15.119:0/3203507154 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c002adc0 con 0x7fb0d015b7e0 2024-04-25T10:09:43.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.747+0000 7fb0ce000640 1 -- 172.21.15.119:0/3203507154 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0032660 con 0x7fb0d015b7e0 2024-04-25T10:09:43.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.748+0000 7fb0d5200640 1 -- 172.21.15.119:0/3203507154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d015b7e0 msgr2=0x7fb0d015e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.748+0000 7fb0d5200640 1 --2- 172.21.15.119:0/3203507154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d015b7e0 0x7fb0d015e000 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00097a0 tx=0x7fb0c002a0b0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.748+0000 7fb0d5200640 1 -- 172.21.15.119:0/3203507154 shutdown_connections 2024-04-25T10:09:43.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.748+0000 7fb0d5200640 1 --2- 172.21.15.119:0/3203507154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d015b7e0 0x7fb0d015e000 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.748+0000 7fb0d5200640 1 -- 172.21.15.119:0/3203507154 >> 172.21.15.119:0/3203507154 conn(0x7fb0d00722d0 msgr2=0x7fb0d00726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:43.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.748+0000 7fb0d5200640 1 -- 172.21.15.119:0/3203507154 shutdown_connections 2024-04-25T10:09:43.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.748+0000 7fb0d5200640 1 -- 172.21.15.119:0/3203507154 wait complete. 2024-04-25T10:09:43.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.750+0000 7fb0d5200640 1 Processor -- start 2024-04-25T10:09:43.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.750+0000 7fb0d5200640 1 -- start start 2024-04-25T10:09:43.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0d5200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0d00ce860 0x7fb0d0150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0d5200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d00d07b0 0x7fb0d00d2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0d5200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0d015b7e0 0x7fb0d00cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0d5200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d00cecf0 con 0x7fb0d00d07b0 2024-04-25T10:09:43.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0d5200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d00b53c0 con 0x7fb0d00ce860 2024-04-25T10:09:43.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0d5200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d00cee70 con 0x7fb0d015b7e0 2024-04-25T10:09:43.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0cea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d00d07b0 0x7fb0d00d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0cea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d00d07b0 0x7fb0d00d2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39248/0 (socket says 172.21.15.119:39248) 2024-04-25T10:09:43.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0cea00640 1 -- 172.21.15.119:0/2068084038 learned_addr learned my addr 172.21.15.119:0/2068084038 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:43.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0cf400640 1 --2- 172.21.15.119:0/2068084038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0d00ce860 0x7fb0d0150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0cea00640 1 -- 172.21.15.119:0/2068084038 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0d015b7e0 msgr2=0x7fb0d00cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0cea00640 1 --2- 172.21.15.119:0/2068084038 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0d015b7e0 0x7fb0d00cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0cea00640 1 -- 172.21.15.119:0/2068084038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0d00ce860 msgr2=0x7fb0d0150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0cea00640 1 --2- 172.21.15.119:0/2068084038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0d00ce860 0x7fb0d0150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.751+0000 7fb0cea00640 1 -- 172.21.15.119:0/2068084038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d00cbe20 con 0x7fb0d00d07b0 2024-04-25T10:09:43.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.752+0000 7fb0cea00640 1 --2- 172.21.15.119:0/2068084038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d00d07b0 0x7fb0d00d2120 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800c430 tx=0x7fb0b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:43.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.752+0000 7fb0c7400640 1 -- 172.21.15.119:0/2068084038 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8011790 con 0x7fb0d00d07b0 2024-04-25T10:09:43.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.752+0000 7fb0c7400640 1 -- 172.21.15.119:0/2068084038 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b8017070 con 0x7fb0d00d07b0 2024-04-25T10:09:43.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.752+0000 7fb0c7400640 1 -- 172.21.15.119:0/2068084038 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8011bb0 con 0x7fb0d00d07b0 2024-04-25T10:09:43.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.752+0000 7fb0d5200640 1 -- 172.21.15.119:0/2068084038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d00cc100 con 0x7fb0d00d07b0 2024-04-25T10:09:43.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.752+0000 7fb0d5200640 1 -- 172.21.15.119:0/2068084038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d00cc6c0 con 0x7fb0d00d07b0 2024-04-25T10:09:43.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.753+0000 7fb0d5200640 1 -- 172.21.15.119:0/2068084038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d00749b0 con 0x7fb0d00d07b0 2024-04-25T10:09:43.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.754+0000 7fb0c7400640 1 -- 172.21.15.119:0/2068084038 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0b801c460 con 0x7fb0d00d07b0 2024-04-25T10:09:43.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.755+0000 7fb0c7400640 1 --2- 172.21.15.119:0/2068084038 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb0ac06b4d0 0x7fb0ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:43.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.756+0000 7fb0c7400640 1 -- 172.21.15.119:0/2068084038 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0b808a220 con 0x7fb0d00d07b0 2024-04-25T10:09:43.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.756+0000 7fb0cf400640 1 --2- 172.21.15.119:0/2068084038 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb0ac06b4d0 0x7fb0ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:43.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.756+0000 7fb0cf400640 1 --2- 172.21.15.119:0/2068084038 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb0ac06b4d0 0x7fb0ac06d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0034000 tx=0x7fb0c002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:43.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.760+0000 7fb0c7400640 1 -- 172.21.15.119:0/2068084038 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0b8058f70 con 0x7fb0d00d07b0 2024-04-25T10:09:43.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.942+0000 7fb0d5200640 1 -- 172.21.15.119:0/2068084038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0d015ec90 con 0x7fb0d00d07b0 2024-04-25T10:09:43.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.943+0000 7fb0c7400640 1 -- 172.21.15.119:0/2068084038 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0b8058870 con 0x7fb0d00d07b0 2024-04-25T10:09:43.949 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:09:43.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.945+0000 7fb0d5200640 1 -- 172.21.15.119:0/2068084038 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb0ac06b4d0 msgr2=0x7fb0ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.945+0000 7fb0d5200640 1 --2- 172.21.15.119:0/2068084038 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb0ac06b4d0 0x7fb0ac06d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0034000 tx=0x7fb0c002e000 comp rx=0 tx=0).stop 2024-04-25T10:09:43.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.945+0000 7fb0d5200640 1 -- 172.21.15.119:0/2068084038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d00d07b0 msgr2=0x7fb0d00d2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:43.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.945+0000 7fb0d5200640 1 --2- 172.21.15.119:0/2068084038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d00d07b0 0x7fb0d00d2120 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800c430 tx=0x7fb0b800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.946+0000 7fb0d5200640 1 -- 172.21.15.119:0/2068084038 shutdown_connections 2024-04-25T10:09:43.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.946+0000 7fb0d5200640 1 --2- 172.21.15.119:0/2068084038 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb0ac06b4d0 0x7fb0ac06d990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.946+0000 7fb0d5200640 1 --2- 172.21.15.119:0/2068084038 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0d015b7e0 0x7fb0d00cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.946+0000 7fb0d5200640 1 --2- 172.21.15.119:0/2068084038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0d00d07b0 0x7fb0d00d2120 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.946+0000 7fb0d5200640 1 --2- 172.21.15.119:0/2068084038 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0d00ce860 0x7fb0d0150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:43.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.946+0000 7fb0d5200640 1 -- 172.21.15.119:0/2068084038 >> 172.21.15.119:0/2068084038 conn(0x7fb0d00722d0 msgr2=0x7fb0d00b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:43.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.946+0000 7fb0d5200640 1 -- 172.21.15.119:0/2068084038 shutdown_connections 2024-04-25T10:09:43.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:43.946+0000 7fb0d5200640 1 -- 172.21.15.119:0/2068084038 wait complete. 2024-04-25T10:09:43.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:44.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.160+0000 7f544cc00640 1 Processor -- start 2024-04-25T10:09:44.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.161+0000 7f544cc00640 1 -- start start 2024-04-25T10:09:44.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.162+0000 7f544cc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54480b5470 0x7f5448159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.162+0000 7f544cc00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f54480b1a30 0x7f544815d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.162+0000 7f544cc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f54480b1e90 0x7f544815de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.162+0000 7f544cc00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54480cec30 con 0x7f54480b1a30 2024-04-25T10:09:44.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.162+0000 7f544cc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54480cef10 con 0x7f54480b1e90 2024-04-25T10:09:44.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.162+0000 7f544cc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54480cf1f0 con 0x7f54480b5470 2024-04-25T10:09:44.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.163+0000 7f543fe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f54480b1a30 0x7f544815d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.163+0000 7f543fe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f54480b1a30 0x7f544815d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39264/0 (socket says 172.21.15.119:39264) 2024-04-25T10:09:44.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.163+0000 7f543fe00640 1 -- 172.21.15.119:0/2705667149 learned_addr learned my addr 172.21.15.119:0/2705667149 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:44.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.163+0000 7f5447400640 1 --2- 172.21.15.119:0/2705667149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54480b5470 0x7f5448159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.163+0000 7f5447e00640 1 --2- 172.21.15.119:0/2705667149 >> v2:172.21.15.132:3300/0 conn(0x7f54480b1e90 0x7f544815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.163+0000 7f543fe00640 1 -- 172.21.15.119:0/2705667149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54480b5470 msgr2=0x7f5448159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:44.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.163+0000 7f543fe00640 1 --2- 172.21.15.119:0/2705667149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54480b5470 0x7f5448159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.163+0000 7f543fe00640 1 -- 172.21.15.119:0/2705667149 >> v2:172.21.15.132:3300/0 conn(0x7f54480b1e90 msgr2=0x7f544815de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:44.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.163+0000 7f543fe00640 1 --2- 172.21.15.119:0/2705667149 >> v2:172.21.15.132:3300/0 conn(0x7f54480b1e90 0x7f544815de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.163+0000 7f543fe00640 1 -- 172.21.15.119:0/2705667149 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544806d420 con 0x7f54480b1a30 2024-04-25T10:09:44.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.164+0000 7f543fe00640 1 --2- 172.21.15.119:0/2705667149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54480b1a30 0x7f544815d910 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f54480af2c0 tx=0x7f543002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d730b3e423fd50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:44.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.165+0000 7f5446a00640 1 -- 172.21.15.119:0/2705667149 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430032790 con 0x7f54480b1a30 2024-04-25T10:09:44.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.165+0000 7f5446a00640 1 -- 172.21.15.119:0/2705667149 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5430038070 con 0x7f54480b1a30 2024-04-25T10:09:44.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.165+0000 7f5446a00640 1 -- 172.21.15.119:0/2705667149 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430032b50 con 0x7f54480b1a30 2024-04-25T10:09:44.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.165+0000 7f544cc00640 1 -- 172.21.15.119:0/2705667149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54480b1a30 msgr2=0x7f544815d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.165+0000 7f544cc00640 1 --2- 172.21.15.119:0/2705667149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54480b1a30 0x7f544815d910 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f54480af2c0 tx=0x7f543002c890 comp rx=0 tx=0).stop 2024-04-25T10:09:44.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.166+0000 7f544cc00640 1 -- 172.21.15.119:0/2705667149 shutdown_connections 2024-04-25T10:09:44.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.166+0000 7f544cc00640 1 --2- 172.21.15.119:0/2705667149 >> v2:172.21.15.132:3300/0 conn(0x7f54480b1e90 0x7f544815de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.166+0000 7f544cc00640 1 --2- 172.21.15.119:0/2705667149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54480b1a30 0x7f544815d910 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.166+0000 7f544cc00640 1 --2- 172.21.15.119:0/2705667149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54480b5470 0x7f5448159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.166+0000 7f544cc00640 1 -- 172.21.15.119:0/2705667149 >> 172.21.15.119:0/2705667149 conn(0x7f544806b1d0 msgr2=0x7f54480b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:44.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.166+0000 7f544cc00640 1 -- 172.21.15.119:0/2705667149 shutdown_connections 2024-04-25T10:09:44.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.166+0000 7f544cc00640 1 -- 172.21.15.119:0/2705667149 wait complete. 2024-04-25T10:09:44.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.167+0000 7f544cc00640 1 Processor -- start 2024-04-25T10:09:44.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.167+0000 7f544cc00640 1 -- start start 2024-04-25T10:09:44.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.168+0000 7f544cc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54480b1a30 0x7f544813df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.168+0000 7f544cc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54480b1e90 0x7f54481416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.168+0000 7f544cc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54480b5470 0x7f544813eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.168+0000 7f544cc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54480ce9f0 con 0x7f54480b1a30 2024-04-25T10:09:44.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.168+0000 7f544cc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54480ce870 con 0x7f54480b5470 2024-04-25T10:09:44.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.168+0000 7f544cc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544815e390 con 0x7f54480b1e90 2024-04-25T10:09:44.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.168+0000 7f5447400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54480b1a30 0x7f544813df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.169+0000 7f5447400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54480b1a30 0x7f544813df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39272/0 (socket says 172.21.15.119:39272) 2024-04-25T10:09:44.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.169+0000 7f5447400640 1 -- 172.21.15.119:0/567054999 learned_addr learned my addr 172.21.15.119:0/567054999 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:44.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.169+0000 7f5447400640 1 -- 172.21.15.119:0/567054999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54480b1e90 msgr2=0x7f54481416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.169+0000 7f5447400640 1 --2- 172.21.15.119:0/567054999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54480b1e90 0x7f54481416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.169+0000 7f5447400640 1 -- 172.21.15.119:0/567054999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54480b5470 msgr2=0x7f544813eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.169+0000 7f5447400640 1 --2- 172.21.15.119:0/567054999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54480b5470 0x7f544813eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.169+0000 7f5447400640 1 -- 172.21.15.119:0/567054999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54481457d0 con 0x7f54480b1a30 2024-04-25T10:09:44.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.169+0000 7f5447400640 1 --2- 172.21.15.119:0/567054999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54480b1a30 0x7f544813df30 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f543800d5f0 tx=0x7f543800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:44.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.169+0000 7f543f400640 1 -- 172.21.15.119:0/567054999 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380125f0 con 0x7f54480b1a30 2024-04-25T10:09:44.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.169+0000 7f543f400640 1 -- 172.21.15.119:0/567054999 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5438019070 con 0x7f54480b1a30 2024-04-25T10:09:44.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.170+0000 7f543f400640 1 -- 172.21.15.119:0/567054999 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380129d0 con 0x7f54480b1a30 2024-04-25T10:09:44.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.170+0000 7f544cc00640 1 -- 172.21.15.119:0/567054999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5448140640 con 0x7f54480b1a30 2024-04-25T10:09:44.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.170+0000 7f544cc00640 1 -- 172.21.15.119:0/567054999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5448140c00 con 0x7f54480b1a30 2024-04-25T10:09:44.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.171+0000 7f544cc00640 1 -- 172.21.15.119:0/567054999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54480749b0 con 0x7f54480b1a30 2024-04-25T10:09:44.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.172+0000 7f543f400640 1 -- 172.21.15.119:0/567054999 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5438012b70 con 0x7f54480b1a30 2024-04-25T10:09:44.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.172+0000 7f543f400640 1 --2- 172.21.15.119:0/567054999 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f542406b4d0 0x7f542406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.173+0000 7f543f400640 1 -- 172.21.15.119:0/567054999 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f543808a610 con 0x7f54480b1a30 2024-04-25T10:09:44.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.173+0000 7f543fe00640 1 --2- 172.21.15.119:0/567054999 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f542406b4d0 0x7f542406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.173+0000 7f543fe00640 1 --2- 172.21.15.119:0/567054999 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f542406b4d0 0x7f542406d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5430037000 tx=0x7f5430008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:44.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.177+0000 7f543f400640 1 -- 172.21.15.119:0/567054999 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f54481457d0 con 0x7f54480b1a30 2024-04-25T10:09:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.393+0000 7f544cc00640 1 -- 172.21.15.119:0/567054999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54480b2d20 con 0x7f54480b1a30 2024-04-25T10:09:44.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.394+0000 7f543f400640 1 -- 172.21.15.119:0/567054999 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f54380593f0 con 0x7f54480b1a30 2024-04-25T10:09:44.400 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:44.400 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3317744182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3317744182},{"type":"v1","addr":"172.21.15.132:6833","nonce":3317744182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/700427287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":700427287},{"type":"v1","addr":"172.21.15.132:6835","nonce":700427287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2207174972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2207174972},{"type":"v1","addr":"172.21.15.119:6836","nonce":2207174972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:57.120451+0000","modified":"2024-04-25T10:09:00.157396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5478":{"gid":5478,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/270766821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":270766821},{"type":"v1","addr":"172.21.15.119:6837","nonce":270766821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:09:44.401 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 25 2024-04-25T10:09:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.396+0000 7f544cc00640 1 -- 172.21.15.119:0/567054999 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f542406b4d0 msgr2=0x7f542406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.396+0000 7f544cc00640 1 --2- 172.21.15.119:0/567054999 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f542406b4d0 0x7f542406d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5430037000 tx=0x7f5430008040 comp rx=0 tx=0).stop 2024-04-25T10:09:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.396+0000 7f544cc00640 1 -- 172.21.15.119:0/567054999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54480b1a30 msgr2=0x7f544813df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.396+0000 7f544cc00640 1 --2- 172.21.15.119:0/567054999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54480b1a30 0x7f544813df30 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f543800d5f0 tx=0x7f543800dab0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.396+0000 7f544cc00640 1 -- 172.21.15.119:0/567054999 shutdown_connections 2024-04-25T10:09:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.396+0000 7f544cc00640 1 --2- 172.21.15.119:0/567054999 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f542406b4d0 0x7f542406d990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.396+0000 7f544cc00640 1 --2- 172.21.15.119:0/567054999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54480b5470 0x7f544813eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.396+0000 7f544cc00640 1 --2- 172.21.15.119:0/567054999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f54480b1e90 0x7f54481416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.396+0000 7f544cc00640 1 --2- 172.21.15.119:0/567054999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54480b1a30 0x7f544813df30 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.396+0000 7f544cc00640 1 -- 172.21.15.119:0/567054999 >> 172.21.15.119:0/567054999 conn(0x7f544806b1d0 msgr2=0x7f5448157320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.396+0000 7f544cc00640 1 -- 172.21.15.119:0/567054999 shutdown_connections 2024-04-25T10:09:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.397+0000 7f544cc00640 1 -- 172.21.15.119:0/567054999 wait complete. 2024-04-25T10:09:44.421 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:44.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.611+0000 7f6307e00640 1 Processor -- start 2024-04-25T10:09:44.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.611+0000 7f6307e00640 1 -- start start 2024-04-25T10:09:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.612+0000 7f6307e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f630815fc90 0x7f630806e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.612+0000 7f6307e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63080cd440 con 0x7f63080b5050 2024-04-25T10:09:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.612+0000 7f6307e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63080cd720 con 0x7f6308144470 2024-04-25T10:09:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.612+0000 7f6307e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6308145340 con 0x7f630815fc90 2024-04-25T10:09:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.613+0000 7f6306000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f63080b5050 0x7f630815bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58366/0 (socket says 172.21.15.119:58366) 2024-04-25T10:09:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.613+0000 7f6306000640 1 -- 172.21.15.119:0/3063830562 learned_addr learned my addr 172.21.15.119:0/3063830562 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.613+0000 7f6307400640 1 --2- 172.21.15.119:0/3063830562 >> v2:172.21.15.119:3300/0 conn(0x7f630815fc90 0x7f630806e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.613+0000 7f6305600640 1 -- 172.21.15.119:0/3063830562 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305454916 0 0) 0x7f63080cd440 con 0x7f63080b5050 2024-04-25T10:09:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.613+0000 7f6305600640 1 -- 172.21.15.119:0/3063830562 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62ec003520 con 0x7f63080b5050 2024-04-25T10:09:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.613+0000 7f6307400640 1 -- 172.21.15.119:0/3063830562 >> v1:172.21.15.132:6789/0 conn(0x7f6308144470 legacy=0x7f63080b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.613+0000 7f6307400640 1 -- 172.21.15.119:0/3063830562 >> v1:172.21.15.119:6789/0 conn(0x7f63080b5050 legacy=0x7f630815bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.614+0000 7f6307400640 1 -- 172.21.15.119:0/3063830562 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63080cda00 con 0x7f630815fc90 2024-04-25T10:09:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.615+0000 7f6307400640 1 --2- 172.21.15.119:0/3063830562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f630815fc90 0x7f630806e430 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6300009d90 tx=0x7f630002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5162692ce2464b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:44.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.615+0000 7f6305600640 1 -- 172.21.15.119:0/3063830562 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300007740 con 0x7f630815fc90 2024-04-25T10:09:44.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.615+0000 7f6305600640 1 -- 172.21.15.119:0/3063830562 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63000078e0 con 0x7f630815fc90 2024-04-25T10:09:44.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.615+0000 7f6305600640 1 -- 172.21.15.119:0/3063830562 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300007cb0 con 0x7f630815fc90 2024-04-25T10:09:44.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.616+0000 7f6307e00640 1 -- 172.21.15.119:0/3063830562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f630815fc90 msgr2=0x7f630806e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.616+0000 7f6307e00640 1 --2- 172.21.15.119:0/3063830562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f630815fc90 0x7f630806e430 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6300009d90 tx=0x7f630002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.616+0000 7f6307e00640 1 -- 172.21.15.119:0/3063830562 shutdown_connections 2024-04-25T10:09:44.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.616+0000 7f6307e00640 1 --2- 172.21.15.119:0/3063830562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f630815fc90 0x7f630806e430 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.616+0000 7f6307e00640 1 -- 172.21.15.119:0/3063830562 >> 172.21.15.119:0/3063830562 conn(0x7f630806b1d0 msgr2=0x7f63080b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:44.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.616+0000 7f6307e00640 1 -- 172.21.15.119:0/3063830562 shutdown_connections 2024-04-25T10:09:44.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.616+0000 7f6307e00640 1 -- 172.21.15.119:0/3063830562 wait complete. 2024-04-25T10:09:44.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.618+0000 7f6307e00640 1 Processor -- start 2024-04-25T10:09:44.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.618+0000 7f6307e00640 1 -- start start 2024-04-25T10:09:44.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.619+0000 7f6307e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63080b5050 0x7f63080aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.619+0000 7f6307e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6308144470 0x7f630813fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.619+0000 7f6307e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630815fc90 0x7f630815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.619+0000 7f6307e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63080ce200 con 0x7f6308144470 2024-04-25T10:09:44.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.619+0000 7f6307e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6308144850 con 0x7f63080b5050 2024-04-25T10:09:44.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.619+0000 7f6307e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630815c400 con 0x7f630815fc90 2024-04-25T10:09:44.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.619+0000 7f6306000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6308144470 0x7f630813fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.619+0000 7f6306000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6308144470 0x7f630813fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39294/0 (socket says 172.21.15.119:39294) 2024-04-25T10:09:44.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.619+0000 7f6306000640 1 -- 172.21.15.119:0/2995624808 learned_addr learned my addr 172.21.15.119:0/2995624808 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:44.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.619+0000 7f6306000640 1 -- 172.21.15.119:0/2995624808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630815fc90 msgr2=0x7f630815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.619+0000 7f6306a00640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63080b5050 0x7f63080aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.619+0000 7f6307400640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630815fc90 0x7f630815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.620+0000 7f6306000640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630815fc90 0x7f630815d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.620+0000 7f6306000640 1 -- 172.21.15.119:0/2995624808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63080b5050 msgr2=0x7f63080aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.620+0000 7f6306000640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63080b5050 0x7f63080aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.620+0000 7f6306000640 1 -- 172.21.15.119:0/2995624808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63080ad2c0 con 0x7f6308144470 2024-04-25T10:09:44.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.620+0000 7f6307400640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630815fc90 0x7f630815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:44.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.620+0000 7f6306a00640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63080b5050 0x7f63080aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:44.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.620+0000 7f6306000640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6308144470 0x7f630813fa00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f62f400c500 tx=0x7f62f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:44.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.620+0000 7f62ff400640 1 -- 172.21.15.119:0/2995624808 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4017070 con 0x7f6308144470 2024-04-25T10:09:44.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.620+0000 7f62ff400640 1 -- 172.21.15.119:0/2995624808 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f4005ce0 con 0x7f6308144470 2024-04-25T10:09:44.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.620+0000 7f62ff400640 1 -- 172.21.15.119:0/2995624808 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4007330 con 0x7f6308144470 2024-04-25T10:09:44.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.620+0000 7f6307e00640 1 -- 172.21.15.119:0/2995624808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63080ad5a0 con 0x7f6308144470 2024-04-25T10:09:44.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.620+0000 7f6307e00640 1 -- 172.21.15.119:0/2995624808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63080adb60 con 0x7f6308144470 2024-04-25T10:09:44.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.621+0000 7f6307e00640 1 -- 172.21.15.119:0/2995624808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6308142f70 con 0x7f6308144470 2024-04-25T10:09:44.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.622+0000 7f62ff400640 1 -- 172.21.15.119:0/2995624808 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62f401a070 con 0x7f6308144470 2024-04-25T10:09:44.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.623+0000 7f62ff400640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62e806b500 0x7f62e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:44.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.623+0000 7f62ff400640 1 -- 172.21.15.119:0/2995624808 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62f4089660 con 0x7f6308144470 2024-04-25T10:09:44.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.623+0000 7f6306a00640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62e806b500 0x7f62e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:44.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.624+0000 7f6306a00640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62e806b500 0x7f62e806d9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f62f8001680 tx=0x7f62f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:44.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.627+0000 7f62ff400640 1 -- 172.21.15.119:0/2995624808 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6308142f70 con 0x7f6308144470 2024-04-25T10:09:44.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.825+0000 7f6307e00640 1 -- 172.21.15.119:0/2995624808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63080b2090 con 0x7f6308144470 2024-04-25T10:09:44.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.827+0000 7f62ff400640 1 -- 172.21.15.119:0/2995624808 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16649 (secure 0 0 0) 0x7f62f4058630 con 0x7f6308144470 2024-04-25T10:09:44.833 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:44.833 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":44,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:58.637281+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:54.917692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:56.128440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.829+0000 7f6307e00640 1 -- 172.21.15.119:0/2995624808 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62e806b500 msgr2=0x7f62e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.829+0000 7f6307e00640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62e806b500 0x7f62e806d9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f62f8001680 tx=0x7f62f8009040 comp rx=0 tx=0).stop 2024-04-25T10:09:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.829+0000 7f6307e00640 1 -- 172.21.15.119:0/2995624808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6308144470 msgr2=0x7f630813fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.829+0000 7f6307e00640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6308144470 0x7f630813fa00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f62f400c500 tx=0x7f62f400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.829+0000 7f6307e00640 1 -- 172.21.15.119:0/2995624808 shutdown_connections 2024-04-25T10:09:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.829+0000 7f6307e00640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62e806b500 0x7f62e806d9c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.829+0000 7f6307e00640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f630815fc90 0x7f630815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.829+0000 7f6307e00640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6308144470 0x7f630813fa00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.829+0000 7f6307e00640 1 --2- 172.21.15.119:0/2995624808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63080b5050 0x7f63080aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.829+0000 7f6307e00640 1 -- 172.21.15.119:0/2995624808 >> 172.21.15.119:0/2995624808 conn(0x7f630806b1d0 msgr2=0x7f630815bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.829+0000 7f6307e00640 1 -- 172.21.15.119:0/2995624808 shutdown_connections 2024-04-25T10:09:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:44.830+0000 7f6307e00640 1 -- 172.21.15.119:0/2995624808 wait complete. 2024-04-25T10:09:44.855 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:09:44.855 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:45.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.034+0000 7f5780600640 1 Processor -- start 2024-04-25T10:09:45.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.034+0000 7f5780600640 1 -- start start 2024-04-25T10:09:45.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.036+0000 7f5780600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f577806e8f0 0x7f577815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.036+0000 7f5780600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57780ce6c0 con 0x7f577815bf00 2024-04-25T10:09:45.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.036+0000 7f5780600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57780cc4e0 con 0x7f577815c820 2024-04-25T10:09:45.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.036+0000 7f5780600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57781455e0 con 0x7f577806e8f0 2024-04-25T10:09:45.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.036+0000 7f577dc00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f577815bf00 0x7f57780cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58374/0 (socket says 172.21.15.119:58374) 2024-04-25T10:09:45.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.036+0000 7f577dc00640 1 -- 172.21.15.119:0/2675848987 learned_addr learned my addr 172.21.15.119:0/2675848987 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:45.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.036+0000 7f5777e00640 1 -- 172.21.15.119:0/2675848987 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562374010 0 0) 0x7f576c001110 con 0x7f577815bf00 2024-04-25T10:09:45.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.036+0000 7f5777e00640 1 -- 172.21.15.119:0/2675848987 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5760003520 con 0x7f577815bf00 2024-04-25T10:09:45.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f577d200640 1 --2- 172.21.15.119:0/2675848987 >> v2:172.21.15.132:3300/0 conn(0x7f577806e8f0 0x7f577815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f5777e00640 1 -- 172.21.15.119:0/2675848987 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2287294599 0 0) 0x7f5760003520 con 0x7f577815bf00 2024-04-25T10:09:45.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f5777e00640 1 -- 172.21.15.119:0/2675848987 >> v2:172.21.15.132:3300/0 conn(0x7f577806e8f0 msgr2=0x7f577815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:45.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f5777e00640 1 --2- 172.21.15.119:0/2675848987 >> v2:172.21.15.132:3300/0 conn(0x7f577806e8f0 0x7f577815c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f5777e00640 1 -- 172.21.15.119:0/2675848987 >> v1:172.21.15.132:6789/0 conn(0x7f577815c820 legacy=0x7f57781654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:45.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f5777e00640 1 -- 172.21.15.119:0/2675848987 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57780cc7b0 con 0x7f577815bf00 2024-04-25T10:09:45.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f5777e00640 1 -- 172.21.15.119:0/2675848987 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f576c001840 con 0x7f577815bf00 2024-04-25T10:09:45.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f5777e00640 1 -- 172.21.15.119:0/2675848987 >> v1:172.21.15.119:6789/0 conn(0x7f577815bf00 legacy=0x7f57780cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f5777e00640 1 --2- 172.21.15.119:0/2675848987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57600039b0 0x7f5760003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f5777e00640 1 --2- 172.21.15.119:0/2675848987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5760004800 0x7f5760004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f5777e00640 1 --2- 172.21.15.119:0/2675848987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57600051c0 0x7f576000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f5777e00640 1 -- 172.21.15.119:0/2675848987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57781455e0 con 0x7f57600051c0 2024-04-25T10:09:45.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f5777e00640 1 -- 172.21.15.119:0/2675848987 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576000c110 con 0x7f57600039b0 2024-04-25T10:09:45.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f5777e00640 1 -- 172.21.15.119:0/2675848987 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576000c310 con 0x7f5760004800 2024-04-25T10:09:45.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.037+0000 7f577e600640 1 --2- 172.21.15.119:0/2675848987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57600051c0 0x7f576000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.038+0000 7f577dc00640 1 --2- 172.21.15.119:0/2675848987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57600039b0 0x7f5760003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.038+0000 7f577e600640 1 -- 172.21.15.119:0/2675848987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5760004800 msgr2=0x7f5760004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:45.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.038+0000 7f577e600640 1 --2- 172.21.15.119:0/2675848987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5760004800 0x7f5760004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.038+0000 7f577e600640 1 -- 172.21.15.119:0/2675848987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57600039b0 msgr2=0x7f5760003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:45.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.038+0000 7f577e600640 1 --2- 172.21.15.119:0/2675848987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57600039b0 0x7f5760003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.038+0000 7f577e600640 1 -- 172.21.15.119:0/2675848987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5760003520 con 0x7f57600051c0 2024-04-25T10:09:45.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.039+0000 7f577e600640 1 -- 172.21.15.119:0/2675848987 reap_dead start 2024-04-25T10:09:45.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.039+0000 7f577e600640 1 --2- 172.21.15.119:0/2675848987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57600051c0 0x7f576000b9c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5770001e50 tx=0x7f5760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d100f1fbe61d4db8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:45.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.039+0000 7f5777e00640 1 -- 172.21.15.119:0/2675848987 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770001a70 con 0x7f57600051c0 2024-04-25T10:09:45.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.039+0000 7f5777e00640 1 -- 172.21.15.119:0/2675848987 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5770003970 con 0x7f57600051c0 2024-04-25T10:09:45.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.039+0000 7f5777e00640 1 -- 172.21.15.119:0/2675848987 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770003d60 con 0x7f57600051c0 2024-04-25T10:09:45.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.039+0000 7f5780600640 1 -- 172.21.15.119:0/2675848987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57600051c0 msgr2=0x7f576000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.039+0000 7f5780600640 1 --2- 172.21.15.119:0/2675848987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57600051c0 0x7f576000b9c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5770001e50 tx=0x7f5760005830 comp rx=0 tx=0).stop 2024-04-25T10:09:45.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.040+0000 7f5780600640 1 -- 172.21.15.119:0/2675848987 shutdown_connections 2024-04-25T10:09:45.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.040+0000 7f5780600640 1 --2- 172.21.15.119:0/2675848987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57600051c0 0x7f576000b9c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.040+0000 7f5780600640 1 -- 172.21.15.119:0/2675848987 >> 172.21.15.119:0/2675848987 conn(0x7f577806aba0 msgr2=0x7f577806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:45.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.040+0000 7f5780600640 1 -- 172.21.15.119:0/2675848987 shutdown_connections 2024-04-25T10:09:45.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.040+0000 7f5780600640 1 -- 172.21.15.119:0/2675848987 wait complete. 2024-04-25T10:09:45.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.041+0000 7f5780600640 1 Processor -- start 2024-04-25T10:09:45.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.042+0000 7f5780600640 1 -- start start 2024-04-25T10:09:45.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.042+0000 7f5780600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57600051c0 0x7f5778153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.042+0000 7f5780600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5778153940 0x7f5778144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f5780600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5778071190 0x7f5778071640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f5780600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57780ccd70 con 0x7f5778071190 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f5780600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577806d450 con 0x7f5778153940 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f5780600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57780cd050 con 0x7f57600051c0 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f577e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5778071190 0x7f5778071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f577e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5778071190 0x7f5778071640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39318/0 (socket says 172.21.15.119:39318) 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f577e600640 1 -- 172.21.15.119:0/3317074011 learned_addr learned my addr 172.21.15.119:0/3317074011 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f577dc00640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57600051c0 0x7f5778153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f577d200640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5778153940 0x7f5778144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f577e600640 1 -- 172.21.15.119:0/3317074011 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57600051c0 msgr2=0x7f5778153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f577e600640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57600051c0 0x7f5778153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f577e600640 1 -- 172.21.15.119:0/3317074011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5778153940 msgr2=0x7f5778144800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f577e600640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5778153940 0x7f5778144800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f577e600640 1 -- 172.21.15.119:0/3317074011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577813b3c0 con 0x7f5778071190 2024-04-25T10:09:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.043+0000 7f577dc00640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57600051c0 0x7f5778153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:45.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.044+0000 7f577e600640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5778071190 0x7f5778071640 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f577000a030 tx=0x7f57700014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:45.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.044+0000 7f5775600640 1 -- 172.21.15.119:0/3317074011 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770007db0 con 0x7f5778071190 2024-04-25T10:09:45.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.044+0000 7f5775600640 1 -- 172.21.15.119:0/3317074011 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5770006d60 con 0x7f5778071190 2024-04-25T10:09:45.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.044+0000 7f5780600640 1 -- 172.21.15.119:0/3317074011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5778153dd0 con 0x7f5778071190 2024-04-25T10:09:45.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.044+0000 7f5775600640 1 -- 172.21.15.119:0/3317074011 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770001a70 con 0x7f5778071190 2024-04-25T10:09:45.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.044+0000 7f5780600640 1 -- 172.21.15.119:0/3317074011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5778154390 con 0x7f5778071190 2024-04-25T10:09:45.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.045+0000 7f5780600640 1 -- 172.21.15.119:0/3317074011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577806bb30 con 0x7f5778071190 2024-04-25T10:09:45.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.046+0000 7f5775600640 1 -- 172.21.15.119:0/3317074011 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57700056e0 con 0x7f5778071190 2024-04-25T10:09:45.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.047+0000 7f5775600640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f575806b4d0 0x7f575806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.047+0000 7f5775600640 1 -- 172.21.15.119:0/3317074011 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57700a9830 con 0x7f5778071190 2024-04-25T10:09:45.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.047+0000 7f577dc00640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f575806b4d0 0x7f575806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.048+0000 7f577dc00640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f575806b4d0 0x7f575806d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f576c0012b0 tx=0x7f576c000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:45.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.051+0000 7f5775600640 1 -- 172.21.15.119:0/3317074011 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5770042060 con 0x7f5778071190 2024-04-25T10:09:45.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.282+0000 7f5780600640 1 -- 172.21.15.119:0/3317074011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57780749b0 con 0x7f5778071190 2024-04-25T10:09:45.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.283+0000 7f5775600640 1 -- 172.21.15.119:0/3317074011 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f57700786b0 con 0x7f5778071190 2024-04-25T10:09:45.289 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:45.289 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3317744182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3317744182},{"type":"v1","addr":"172.21.15.132:6833","nonce":3317744182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/700427287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":700427287},{"type":"v1","addr":"172.21.15.132:6835","nonce":700427287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2207174972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2207174972},{"type":"v1","addr":"172.21.15.119:6836","nonce":2207174972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:57.120451+0000","modified":"2024-04-25T10:09:00.157396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5478":{"gid":5478,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/270766821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":270766821},{"type":"v1","addr":"172.21.15.119:6837","nonce":270766821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:09:45.290 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 25 2024-04-25T10:09:45.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.285+0000 7f5780600640 1 -- 172.21.15.119:0/3317074011 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f575806b4d0 msgr2=0x7f575806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.285+0000 7f5780600640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f575806b4d0 0x7f575806d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f576c0012b0 tx=0x7f576c000ed0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.285+0000 7f5780600640 1 -- 172.21.15.119:0/3317074011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5778071190 msgr2=0x7f5778071640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.285+0000 7f5780600640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5778071190 0x7f5778071640 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f577000a030 tx=0x7f57700014a0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.285+0000 7f5780600640 1 -- 172.21.15.119:0/3317074011 shutdown_connections 2024-04-25T10:09:45.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.285+0000 7f5780600640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f575806b4d0 0x7f575806d990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.285+0000 7f5780600640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5778071190 0x7f5778071640 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.285+0000 7f5780600640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5778153940 0x7f5778144800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.285+0000 7f5780600640 1 --2- 172.21.15.119:0/3317074011 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57600051c0 0x7f5778153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.285+0000 7f5780600640 1 -- 172.21.15.119:0/3317074011 >> 172.21.15.119:0/3317074011 conn(0x7f577806aba0 msgr2=0x7f577813b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:45.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.286+0000 7f5780600640 1 -- 172.21.15.119:0/3317074011 shutdown_connections 2024-04-25T10:09:45.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.286+0000 7f5780600640 1 -- 172.21.15.119:0/3317074011 wait complete. 2024-04-25T10:09:45.311 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:45.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.509+0000 7ffb19c00640 1 Processor -- start 2024-04-25T10:09:45.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.509+0000 7ffb19c00640 1 -- start start 2024-04-25T10:09:45.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.510+0000 7ffb19c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffb1415bf00 0x7ffb140cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.510+0000 7ffb19c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb140cc310 0x7ffb1415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.510+0000 7ffb19c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffb1415c820 0x7ffb1406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.510+0000 7ffb19c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb141455e0 con 0x7ffb1415bf00 2024-04-25T10:09:45.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.510+0000 7ffb19c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb1406d450 con 0x7ffb1415c820 2024-04-25T10:09:45.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.510+0000 7ffb19c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140ce6c0 con 0x7ffb140cc310 2024-04-25T10:09:45.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.510+0000 7ffb12a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffb1415bf00 0x7ffb140cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.510+0000 7ffb12a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffb1415bf00 0x7ffb140cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39322/0 (socket says 172.21.15.119:39322) 2024-04-25T10:09:45.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.511+0000 7ffb12a00640 1 -- 172.21.15.119:0/539052999 learned_addr learned my addr 172.21.15.119:0/539052999 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:45.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.511+0000 7ffb13400640 1 --2- 172.21.15.119:0/539052999 >> v2:172.21.15.132:3300/0 conn(0x7ffb1415c820 0x7ffb1406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.511+0000 7ffb12a00640 1 -- 172.21.15.119:0/539052999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb140cc310 msgr2=0x7ffb1415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:45.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.511+0000 7ffb12000640 1 --2- 172.21.15.119:0/539052999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb140cc310 0x7ffb1415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.511+0000 7ffb12a00640 1 --2- 172.21.15.119:0/539052999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb140cc310 0x7ffb1415c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.511+0000 7ffb12a00640 1 -- 172.21.15.119:0/539052999 >> v2:172.21.15.132:3300/0 conn(0x7ffb1415c820 msgr2=0x7ffb1406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:45.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.511+0000 7ffb12a00640 1 --2- 172.21.15.119:0/539052999 >> v2:172.21.15.132:3300/0 conn(0x7ffb1415c820 0x7ffb1406d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.511+0000 7ffb12a00640 1 -- 172.21.15.119:0/539052999 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140cc7b0 con 0x7ffb1415bf00 2024-04-25T10:09:45.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.511+0000 7ffb13400640 1 --2- 172.21.15.119:0/539052999 >> v2:172.21.15.132:3300/0 conn(0x7ffb1415c820 0x7ffb1406d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:45.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.511+0000 7ffb12000640 1 --2- 172.21.15.119:0/539052999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb140cc310 0x7ffb1415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:45.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.512+0000 7ffb12a00640 1 --2- 172.21.15.119:0/539052999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb1415bf00 0x7ffb140cbdd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ffb140d0160 tx=0x7ffb0402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae514cfefde97eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:45.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.512+0000 7ffb11600640 1 -- 172.21.15.119:0/539052999 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04037070 con 0x7ffb1415bf00 2024-04-25T10:09:45.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.512+0000 7ffb11600640 1 -- 172.21.15.119:0/539052999 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb04032440 con 0x7ffb1415bf00 2024-04-25T10:09:45.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.512+0000 7ffb11600640 1 -- 172.21.15.119:0/539052999 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04032810 con 0x7ffb1415bf00 2024-04-25T10:09:45.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.513+0000 7ffb19c00640 1 -- 172.21.15.119:0/539052999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb1415bf00 msgr2=0x7ffb140cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.513+0000 7ffb19c00640 1 --2- 172.21.15.119:0/539052999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb1415bf00 0x7ffb140cbdd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ffb140d0160 tx=0x7ffb0402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.513+0000 7ffb19c00640 1 -- 172.21.15.119:0/539052999 shutdown_connections 2024-04-25T10:09:45.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.513+0000 7ffb19c00640 1 --2- 172.21.15.119:0/539052999 >> v2:172.21.15.132:3300/0 conn(0x7ffb1415c820 0x7ffb1406d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.513+0000 7ffb19c00640 1 --2- 172.21.15.119:0/539052999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb140cc310 0x7ffb1415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.513+0000 7ffb19c00640 1 --2- 172.21.15.119:0/539052999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb1415bf00 0x7ffb140cbdd0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.513+0000 7ffb19c00640 1 -- 172.21.15.119:0/539052999 >> 172.21.15.119:0/539052999 conn(0x7ffb1406aba0 msgr2=0x7ffb1406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:45.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.513+0000 7ffb19c00640 1 -- 172.21.15.119:0/539052999 shutdown_connections 2024-04-25T10:09:45.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.513+0000 7ffb19c00640 1 -- 172.21.15.119:0/539052999 wait complete. 2024-04-25T10:09:45.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.515+0000 7ffb19c00640 1 Processor -- start 2024-04-25T10:09:45.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.515+0000 7ffb19c00640 1 -- start start 2024-04-25T10:09:45.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.516+0000 7ffb19c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb140cc310 0x7ffb140c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.516+0000 7ffb19c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb1415bf00 0x7ffb1415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.516+0000 7ffb19c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb1415c820 0x7ffb140c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.516+0000 7ffb19c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140cd050 con 0x7ffb1415c820 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.516+0000 7ffb19c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140ccd70 con 0x7ffb140cc310 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.516+0000 7ffb19c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1415cca0 con 0x7ffb1415bf00 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.516+0000 7ffb13400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb1415c820 0x7ffb140c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.516+0000 7ffb13400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb1415c820 0x7ffb140c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39336/0 (socket says 172.21.15.119:39336) 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.516+0000 7ffb13400640 1 -- 172.21.15.119:0/3126654576 learned_addr learned my addr 172.21.15.119:0/3126654576 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.517+0000 7ffb12a00640 1 --2- 172.21.15.119:0/3126654576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb140cc310 0x7ffb140c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.517+0000 7ffb12000640 1 --2- 172.21.15.119:0/3126654576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb1415bf00 0x7ffb1415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.517+0000 7ffb13400640 1 -- 172.21.15.119:0/3126654576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb1415bf00 msgr2=0x7ffb1415a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.517+0000 7ffb13400640 1 --2- 172.21.15.119:0/3126654576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb1415bf00 0x7ffb1415a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.517+0000 7ffb13400640 1 -- 172.21.15.119:0/3126654576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb140cc310 msgr2=0x7ffb140c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.517+0000 7ffb13400640 1 --2- 172.21.15.119:0/3126654576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb140cc310 0x7ffb140c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.517+0000 7ffb13400640 1 -- 172.21.15.119:0/3126654576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1415a7d0 con 0x7ffb1415c820 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.517+0000 7ffb13400640 1 --2- 172.21.15.119:0/3126654576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb1415c820 0x7ffb140c5580 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00cd50 tx=0x7ffb0c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.517+0000 7ffb0aa00640 1 -- 172.21.15.119:0/3126654576 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c00f680 con 0x7ffb1415c820 2024-04-25T10:09:45.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.517+0000 7ffb0aa00640 1 -- 172.21.15.119:0/3126654576 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c00f840 con 0x7ffb1415c820 2024-04-25T10:09:45.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.517+0000 7ffb0aa00640 1 -- 172.21.15.119:0/3126654576 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c00fc30 con 0x7ffb1415c820 2024-04-25T10:09:45.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.517+0000 7ffb19c00640 1 -- 172.21.15.119:0/3126654576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb140c5f90 con 0x7ffb1415c820 2024-04-25T10:09:45.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.518+0000 7ffb19c00640 1 -- 172.21.15.119:0/3126654576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb14142a80 con 0x7ffb1415c820 2024-04-25T10:09:45.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.519+0000 7ffb19c00640 1 -- 172.21.15.119:0/3126654576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb140749b0 con 0x7ffb1415c820 2024-04-25T10:09:45.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.520+0000 7ffb0aa00640 1 -- 172.21.15.119:0/3126654576 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb0c00e3e0 con 0x7ffb1415c820 2024-04-25T10:09:45.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.521+0000 7ffb0aa00640 1 --2- 172.21.15.119:0/3126654576 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ffaf006b4d0 0x7ffaf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.521+0000 7ffb0aa00640 1 -- 172.21.15.119:0/3126654576 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb0c08af60 con 0x7ffb1415c820 2024-04-25T10:09:45.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.521+0000 7ffb12a00640 1 --2- 172.21.15.119:0/3126654576 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ffaf006b4d0 0x7ffaf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.522+0000 7ffb12a00640 1 --2- 172.21.15.119:0/3126654576 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ffaf006b4d0 0x7ffaf006d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffb04034000 tx=0x7ffb0402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:45.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.526+0000 7ffb0aa00640 1 -- 172.21.15.119:0/3126654576 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb1415a7d0 con 0x7ffb1415c820 2024-04-25T10:09:45.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.758+0000 7ffb19c00640 1 -- 172.21.15.119:0/3126654576 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb1406db60 con 0x7ffb1415c820 2024-04-25T10:09:45.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.759+0000 7ffb0aa00640 1 -- 172.21.15.119:0/3126654576 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7ffb0c014030 con 0x7ffb1415c820 2024-04-25T10:09:45.765 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:45.766 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3317744182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3317744182},{"type":"v1","addr":"172.21.15.132:6833","nonce":3317744182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/700427287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":700427287},{"type":"v1","addr":"172.21.15.132:6835","nonce":700427287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2207174972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2207174972},{"type":"v1","addr":"172.21.15.119:6836","nonce":2207174972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:57.120451+0000","modified":"2024-04-25T10:09:00.157396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5478":{"gid":5478,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/270766821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":270766821},{"type":"v1","addr":"172.21.15.119:6837","nonce":270766821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:09:45.766 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 25 2024-04-25T10:09:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.762+0000 7ffb19c00640 1 -- 172.21.15.119:0/3126654576 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ffaf006b4d0 msgr2=0x7ffaf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.762+0000 7ffb19c00640 1 --2- 172.21.15.119:0/3126654576 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ffaf006b4d0 0x7ffaf006d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffb04034000 tx=0x7ffb0402e000 comp rx=0 tx=0).stop 2024-04-25T10:09:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.762+0000 7ffb19c00640 1 -- 172.21.15.119:0/3126654576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb1415c820 msgr2=0x7ffb140c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.762+0000 7ffb19c00640 1 --2- 172.21.15.119:0/3126654576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb1415c820 0x7ffb140c5580 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00cd50 tx=0x7ffb0c00a650 comp rx=0 tx=0).stop 2024-04-25T10:09:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.762+0000 7ffb19c00640 1 -- 172.21.15.119:0/3126654576 shutdown_connections 2024-04-25T10:09:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.762+0000 7ffb19c00640 1 --2- 172.21.15.119:0/3126654576 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ffaf006b4d0 0x7ffaf006d990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.762+0000 7ffb19c00640 1 --2- 172.21.15.119:0/3126654576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb1415c820 0x7ffb140c5580 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.762+0000 7ffb19c00640 1 --2- 172.21.15.119:0/3126654576 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb1415bf00 0x7ffb1415a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.762+0000 7ffb19c00640 1 --2- 172.21.15.119:0/3126654576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb140cc310 0x7ffb140c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.762+0000 7ffb19c00640 1 -- 172.21.15.119:0/3126654576 >> 172.21.15.119:0/3126654576 conn(0x7ffb1406aba0 msgr2=0x7ffb1406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.762+0000 7ffb19c00640 1 -- 172.21.15.119:0/3126654576 shutdown_connections 2024-04-25T10:09:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.762+0000 7ffb19c00640 1 -- 172.21.15.119:0/3126654576 wait complete. 2024-04-25T10:09:45.787 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:45.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.960+0000 7fe8fe200640 1 Processor -- start 2024-04-25T10:09:45.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.960+0000 7fe8fe200640 1 -- start start 2024-04-25T10:09:45.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.962+0000 7fe8fe200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe8f8161db0 0x7fe8f806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.962+0000 7fe8fe200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8f80b5270 0x7fe8f80cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.962+0000 7fe8fe200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f8166770 con 0x7fe8f80cb820 2024-04-25T10:09:45.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.962+0000 7fe8fe200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f806e030 con 0x7fe8f8161db0 2024-04-25T10:09:45.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.962+0000 7fe8fe200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f81631a0 con 0x7fe8f80b5270 2024-04-25T10:09:45.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.962+0000 7fe8f7e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe8f80cb820 0x7fe8f80b4b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58384/0 (socket says 172.21.15.119:58384) 2024-04-25T10:09:45.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.962+0000 7fe8f7e00640 1 -- 172.21.15.119:0/1768750080 learned_addr learned my addr 172.21.15.119:0/1768750080 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:45.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8fce00640 1 --2- 172.21.15.119:0/1768750080 >> v2:172.21.15.132:3300/0 conn(0x7fe8f8161db0 0x7fe8f806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8fd800640 1 --2- 172.21.15.119:0/1768750080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8f80b5270 0x7fe8f80cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 -- 172.21.15.119:0/1768750080 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006948759 0 0) 0x7fe8f8166770 con 0x7fe8f80cb820 2024-04-25T10:09:45.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 -- 172.21.15.119:0/1768750080 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e0003520 con 0x7fe8f80cb820 2024-04-25T10:09:45.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 -- 172.21.15.119:0/1768750080 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3774123376 0 0) 0x7fe8e0003520 con 0x7fe8f80cb820 2024-04-25T10:09:45.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 -- 172.21.15.119:0/1768750080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8f80b5270 msgr2=0x7fe8f80cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:45.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 --2- 172.21.15.119:0/1768750080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8f80b5270 0x7fe8f80cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 -- 172.21.15.119:0/1768750080 >> v2:172.21.15.132:3300/0 conn(0x7fe8f8161db0 msgr2=0x7fe8f806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:45.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 --2- 172.21.15.119:0/1768750080 >> v2:172.21.15.132:3300/0 conn(0x7fe8f8161db0 0x7fe8f806e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 -- 172.21.15.119:0/1768750080 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f806d420 con 0x7fe8f80cb820 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 -- 172.21.15.119:0/1768750080 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fe8e4001670 con 0x7fe8f80cb820 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 -- 172.21.15.119:0/1768750080 >> v1:172.21.15.119:6789/0 conn(0x7fe8f80cb820 legacy=0x7fe8f80b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 --2- 172.21.15.119:0/1768750080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e0003a20 0x7fe8e0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 --2- 172.21.15.119:0/1768750080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8e0004800 0x7fe8e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 --2- 172.21.15.119:0/1768750080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8f80b5270 0x7fe8e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 -- 172.21.15.119:0/1768750080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f806e030 con 0x7fe8e0003a20 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 -- 172.21.15.119:0/1768750080 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f81631a0 con 0x7fe8e0004800 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.963+0000 7fe8f7400640 1 -- 172.21.15.119:0/1768750080 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e000b9f0 con 0x7fe8f80b5270 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.964+0000 7fe8fce00640 1 --2- 172.21.15.119:0/1768750080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e0003a20 0x7fe8e0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.964+0000 7fe8f7e00640 1 --2- 172.21.15.119:0/1768750080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8e0004800 0x7fe8e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.964+0000 7fe8fce00640 1 -- 172.21.15.119:0/1768750080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8f80b5270 msgr2=0x7fe8e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.964+0000 7fe8fce00640 1 --2- 172.21.15.119:0/1768750080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8f80b5270 0x7fe8e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.964+0000 7fe8fce00640 1 -- 172.21.15.119:0/1768750080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8e0004800 msgr2=0x7fe8e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.964+0000 7fe8fce00640 1 --2- 172.21.15.119:0/1768750080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8e0004800 0x7fe8e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.964+0000 7fe8fce00640 1 -- 172.21.15.119:0/1768750080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e0003520 con 0x7fe8e0003a20 2024-04-25T10:09:45.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.964+0000 7fe8f7e00640 1 --2- 172.21.15.119:0/1768750080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8e0004800 0x7fe8e0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:45.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.965+0000 7fe8fce00640 1 --2- 172.21.15.119:0/1768750080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e0003a20 0x7fe8e0003e40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009ac0 tx=0x7fe8e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f8aa7e0520188f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:45.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.965+0000 7fe8f7400640 1 -- 172.21.15.119:0/1768750080 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec036070 con 0x7fe8e0003a20 2024-04-25T10:09:45.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.965+0000 7fe8f7400640 1 -- 172.21.15.119:0/1768750080 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec031400 con 0x7fe8e0003a20 2024-04-25T10:09:45.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.965+0000 7fe8f7400640 1 -- 172.21.15.119:0/1768750080 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec009690 con 0x7fe8e0003a20 2024-04-25T10:09:45.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.966+0000 7fe8fe200640 1 -- 172.21.15.119:0/1768750080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e0003a20 msgr2=0x7fe8e0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.966+0000 7fe8fe200640 1 --2- 172.21.15.119:0/1768750080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e0003a20 0x7fe8e0003e40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009ac0 tx=0x7fe8e0005830 comp rx=0 tx=0).stop 2024-04-25T10:09:45.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.966+0000 7fe8fd800640 1 -- 172.21.15.119:0/1768750080 reap_dead start 2024-04-25T10:09:45.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.966+0000 7fe8fe200640 1 -- 172.21.15.119:0/1768750080 shutdown_connections 2024-04-25T10:09:45.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.966+0000 7fe8fe200640 1 -- 172.21.15.119:0/1768750080 >> 172.21.15.119:0/1768750080 conn(0x7fe8f806b1d0 msgr2=0x7fe8f80b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:45.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.966+0000 7fe8fe200640 1 -- 172.21.15.119:0/1768750080 shutdown_connections 2024-04-25T10:09:45.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.966+0000 7fe8fe200640 1 -- 172.21.15.119:0/1768750080 wait complete. 2024-04-25T10:09:45.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.968+0000 7fe8fe200640 1 Processor -- start 2024-04-25T10:09:45.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.968+0000 7fe8fe200640 1 -- start start 2024-04-25T10:09:45.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.969+0000 7fe8fe200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8e0003a20 0x7fe8f81fc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.969+0000 7fe8fe200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f81462a0 0x7fe8f8146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.969+0000 7fe8fe200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8f8142d40 0x7fe8f81431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.969+0000 7fe8fe200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f806dcc0 con 0x7fe8f81462a0 2024-04-25T10:09:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.969+0000 7fe8fe200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f806d9e0 con 0x7fe8f8142d40 2024-04-25T10:09:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.969+0000 7fe8fe200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f806ea00 con 0x7fe8e0003a20 2024-04-25T10:09:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.969+0000 7fe8f7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f81462a0 0x7fe8f8146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.969+0000 7fe8f7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f81462a0 0x7fe8f8146720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39346/0 (socket says 172.21.15.119:39346) 2024-04-25T10:09:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.969+0000 7fe8f7e00640 1 -- 172.21.15.119:0/1127080110 learned_addr learned my addr 172.21.15.119:0/1127080110 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.970+0000 7fe8fd800640 1 --2- 172.21.15.119:0/1127080110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8f8142d40 0x7fe8f81431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.970+0000 7fe8f7e00640 1 -- 172.21.15.119:0/1127080110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8e0003a20 msgr2=0x7fe8f81fc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.970+0000 7fe8f7e00640 1 --2- 172.21.15.119:0/1127080110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8e0003a20 0x7fe8f81fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.970+0000 7fe8f7e00640 1 -- 172.21.15.119:0/1127080110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8f8142d40 msgr2=0x7fe8f81431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.970+0000 7fe8f7e00640 1 --2- 172.21.15.119:0/1127080110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8f8142d40 0x7fe8f81431f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.970+0000 7fe8f7e00640 1 -- 172.21.15.119:0/1127080110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f81fc6b0 con 0x7fe8f81462a0 2024-04-25T10:09:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.970+0000 7fe8f7e00640 1 --2- 172.21.15.119:0/1127080110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f81462a0 0x7fe8f8146720 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400d800 tx=0x7fe8e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.970+0000 7fe8f4c00640 1 -- 172.21.15.119:0/1127080110 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4019070 con 0x7fe8f81462a0 2024-04-25T10:09:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.970+0000 7fe8f4c00640 1 -- 172.21.15.119:0/1127080110 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e40074e0 con 0x7fe8f81462a0 2024-04-25T10:09:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.970+0000 7fe8f4c00640 1 -- 172.21.15.119:0/1127080110 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e40049e0 con 0x7fe8f81462a0 2024-04-25T10:09:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.970+0000 7fe8fe200640 1 -- 172.21.15.119:0/1127080110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f8159640 con 0x7fe8f81462a0 2024-04-25T10:09:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.970+0000 7fe8fe200640 1 -- 172.21.15.119:0/1127080110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f8159c00 con 0x7fe8f81462a0 2024-04-25T10:09:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.971+0000 7fe8fe200640 1 -- 172.21.15.119:0/1127080110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f80749b0 con 0x7fe8f81462a0 2024-04-25T10:09:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.972+0000 7fe8f4c00640 1 -- 172.21.15.119:0/1127080110 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8e4004b80 con 0x7fe8f81462a0 2024-04-25T10:09:45.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.973+0000 7fe8f4c00640 1 --2- 172.21.15.119:0/1127080110 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe8d806b4d0 0x7fe8d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:45.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.973+0000 7fe8f4c00640 1 -- 172.21.15.119:0/1127080110 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8e4089ca0 con 0x7fe8f81462a0 2024-04-25T10:09:45.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.974+0000 7fe8fce00640 1 --2- 172.21.15.119:0/1127080110 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe8d806b4d0 0x7fe8d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:45.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.974+0000 7fe8fce00640 1 --2- 172.21.15.119:0/1127080110 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe8d806b4d0 0x7fe8d806d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec033000 tx=0x7fe8ec02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:45.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:45.977+0000 7fe8f4c00640 1 -- 172.21.15.119:0/1127080110 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8e4058b10 con 0x7fe8f81462a0 2024-04-25T10:09:46.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.175+0000 7fe8fe200640 1 -- 172.21.15.119:0/1127080110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8f80b2180 con 0x7fe8f81462a0 2024-04-25T10:09:46.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.176+0000 7fe8f4c00640 1 -- 172.21.15.119:0/1127080110 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16649 (secure 0 0 0) 0x7fe8e4058410 con 0x7fe8f81462a0 2024-04-25T10:09:46.183 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:46.183 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":44,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:08:58.637281+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:54.917692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:56.128440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:46.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.179+0000 7fe8fe200640 1 -- 172.21.15.119:0/1127080110 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe8d806b4d0 msgr2=0x7fe8d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.179+0000 7fe8fe200640 1 --2- 172.21.15.119:0/1127080110 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe8d806b4d0 0x7fe8d806d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec033000 tx=0x7fe8ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:09:46.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.179+0000 7fe8fe200640 1 -- 172.21.15.119:0/1127080110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f81462a0 msgr2=0x7fe8f8146720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.179+0000 7fe8fe200640 1 --2- 172.21.15.119:0/1127080110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f81462a0 0x7fe8f8146720 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400d800 tx=0x7fe8e400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.179+0000 7fe8fe200640 1 -- 172.21.15.119:0/1127080110 shutdown_connections 2024-04-25T10:09:46.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.179+0000 7fe8fe200640 1 --2- 172.21.15.119:0/1127080110 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe8d806b4d0 0x7fe8d806d990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.179+0000 7fe8fe200640 1 --2- 172.21.15.119:0/1127080110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8f8142d40 0x7fe8f81431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.179+0000 7fe8fe200640 1 --2- 172.21.15.119:0/1127080110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f81462a0 0x7fe8f8146720 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.179+0000 7fe8fe200640 1 --2- 172.21.15.119:0/1127080110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8e0003a20 0x7fe8f81fc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.179+0000 7fe8fe200640 1 -- 172.21.15.119:0/1127080110 >> 172.21.15.119:0/1127080110 conn(0x7fe8f806b1d0 msgr2=0x7fe8f80af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:46.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.179+0000 7fe8fe200640 1 -- 172.21.15.119:0/1127080110 shutdown_connections 2024-04-25T10:09:46.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.180+0000 7fe8fe200640 1 -- 172.21.15.119:0/1127080110 wait complete. 2024-04-25T10:09:46.205 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:09:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.385+0000 7f3e94400640 1 Processor -- start 2024-04-25T10:09:46.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.385+0000 7f3e94400640 1 -- start start 2024-04-25T10:09:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.387+0000 7f3e94400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3e8c0aedc0 0x7f3e8c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.387+0000 7f3e94400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3e8c0cf900 0x7f3e8c06d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.387+0000 7f3e94400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8c0ace70 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.387+0000 7f3e94400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e8c0cfd70 con 0x7f3e8c0cf900 2024-04-25T10:09:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.387+0000 7f3e94400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e8c0ad110 con 0x7f3e8c0aedc0 2024-04-25T10:09:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.387+0000 7f3e91a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3e8c15dbe0 0x7f3e8c071c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58400/0 (socket says 172.21.15.119:58400) 2024-04-25T10:09:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.387+0000 7f3e91a00640 1 -- 172.21.15.119:0/16060600 learned_addr learned my addr 172.21.15.119:0/16060600 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.387+0000 7f3e92400640 1 --2- 172.21.15.119:0/16060600 >> v2:172.21.15.119:3300/0 conn(0x7f3e8c0cf900 0x7f3e8c06d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.388+0000 7f3e91000640 1 --2- 172.21.15.119:0/16060600 >> v2:172.21.15.132:3300/0 conn(0x7f3e8c0aedc0 0x7f3e8c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.388+0000 7f3e8be00640 1 -- 172.21.15.119:0/16060600 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526330668 0 0) 0x7f3e8c0ace70 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.388+0000 7f3e8be00640 1 -- 172.21.15.119:0/16060600 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e74003520 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.388+0000 7f3e92400640 1 -- 172.21.15.119:0/16060600 >> v2:172.21.15.132:3300/0 conn(0x7f3e8c0aedc0 msgr2=0x7f3e8c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.388+0000 7f3e92400640 1 --2- 172.21.15.119:0/16060600 >> v2:172.21.15.132:3300/0 conn(0x7f3e8c0aedc0 0x7f3e8c0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.388+0000 7f3e92400640 1 -- 172.21.15.119:0/16060600 >> v1:172.21.15.119:6789/0 conn(0x7f3e8c15dbe0 legacy=0x7f3e8c071c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.388+0000 7f3e92400640 1 -- 172.21.15.119:0/16060600 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c0ad3f0 con 0x7f3e8c0cf900 2024-04-25T10:09:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.389+0000 7f3e92400640 1 --2- 172.21.15.119:0/16060600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c0cf900 0x7f3e8c06d9a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0b4de0 tx=0x7f3e8402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8706128d2341d9cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.390+0000 7f3e8be00640 1 -- 172.21.15.119:0/16060600 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84007740 con 0x7f3e8c0cf900 2024-04-25T10:09:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.390+0000 7f3e8be00640 1 -- 172.21.15.119:0/16060600 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e840078e0 con 0x7f3e8c0cf900 2024-04-25T10:09:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.390+0000 7f3e8be00640 1 -- 172.21.15.119:0/16060600 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84007c90 con 0x7f3e8c0cf900 2024-04-25T10:09:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.390+0000 7f3e94400640 1 -- 172.21.15.119:0/16060600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c0cf900 msgr2=0x7f3e8c06d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.390+0000 7f3e94400640 1 --2- 172.21.15.119:0/16060600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c0cf900 0x7f3e8c06d9a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0b4de0 tx=0x7f3e8402a680 comp rx=0 tx=0).stop 2024-04-25T10:09:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.390+0000 7f3e94400640 1 -- 172.21.15.119:0/16060600 shutdown_connections 2024-04-25T10:09:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.390+0000 7f3e94400640 1 --2- 172.21.15.119:0/16060600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c0cf900 0x7f3e8c06d9a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.390+0000 7f3e94400640 1 --2- 172.21.15.119:0/16060600 >> v2:172.21.15.132:3300/0 conn(0x7f3e8c0aedc0 0x7f3e8c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.390+0000 7f3e94400640 1 -- 172.21.15.119:0/16060600 >> 172.21.15.119:0/16060600 conn(0x7f3e8c06aba0 msgr2=0x7f3e8c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.391+0000 7f3e94400640 1 -- 172.21.15.119:0/16060600 shutdown_connections 2024-04-25T10:09:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.391+0000 7f3e94400640 1 -- 172.21.15.119:0/16060600 wait complete. 2024-04-25T10:09:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.392+0000 7f3e94400640 1 Processor -- start 2024-04-25T10:09:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.392+0000 7f3e94400640 1 -- start start 2024-04-25T10:09:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.393+0000 7f3e94400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e8c0aedc0 0x7f3e8c0af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.393+0000 7f3e94400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e8c0cf900 0x7f3e8c0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.393+0000 7f3e94400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c15dbe0 0x7f3e8c0ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.393+0000 7f3e94400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0adc90 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.393+0000 7f3e94400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0ad9b0 con 0x7f3e8c0cf900 2024-04-25T10:09:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.393+0000 7f3e94400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c15dfc0 con 0x7f3e8c0aedc0 2024-04-25T10:09:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.394+0000 7f3e92400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c15dbe0 0x7f3e8c0ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.394+0000 7f3e92400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c15dbe0 0x7f3e8c0ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39370/0 (socket says 172.21.15.119:39370) 2024-04-25T10:09:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.394+0000 7f3e92400640 1 -- 172.21.15.119:0/2227364997 learned_addr learned my addr 172.21.15.119:0/2227364997 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.394+0000 7f3e91000640 1 --2- 172.21.15.119:0/2227364997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e8c0cf900 0x7f3e8c0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.394+0000 7f3e92400640 1 -- 172.21.15.119:0/2227364997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e8c0aedc0 msgr2=0x7f3e8c0af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.394+0000 7f3e92400640 1 --2- 172.21.15.119:0/2227364997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e8c0aedc0 0x7f3e8c0af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.394+0000 7f3e92400640 1 -- 172.21.15.119:0/2227364997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e8c0cf900 msgr2=0x7f3e8c0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.394+0000 7f3e92400640 1 --2- 172.21.15.119:0/2227364997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e8c0cf900 0x7f3e8c0ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.394+0000 7f3e92400640 1 -- 172.21.15.119:0/2227364997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c0b0c90 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.394+0000 7f3e92400640 1 --2- 172.21.15.119:0/2227364997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c15dbe0 0x7f3e8c0ce8c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009e50 tx=0x7f3e84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.395+0000 7f3e89600640 1 -- 172.21.15.119:0/2227364997 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84040070 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.395+0000 7f3e89600640 1 -- 172.21.15.119:0/2227364997 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e84031c90 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.395+0000 7f3e89600640 1 -- 172.21.15.119:0/2227364997 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84009450 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.395+0000 7f3e94400640 1 -- 172.21.15.119:0/2227364997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8c0afbf0 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.395+0000 7f3e94400640 1 -- 172.21.15.119:0/2227364997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8c0b01b0 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.396+0000 7f3e94400640 1 -- 172.21.15.119:0/2227364997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8c0750b0 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.397+0000 7f3e89600640 1 -- 172.21.15.119:0/2227364997 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e84039040 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.398+0000 7f3e89600640 1 --2- 172.21.15.119:0/2227364997 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e6c06b4d0 0x7f3e6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.398+0000 7f3e89600640 1 -- 172.21.15.119:0/2227364997 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e840ab390 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.398+0000 7f3e91a00640 1 --2- 172.21.15.119:0/2227364997 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e6c06b4d0 0x7f3e6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.399+0000 7f3e91a00640 1 --2- 172.21.15.119:0/2227364997 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e6c06b4d0 0x7f3e6c06d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3e80000f60 tx=0x7f3e800078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:46.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.402+0000 7f3e89600640 1 -- 172.21.15.119:0/2227364997 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e84042310 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.621+0000 7f3e94400640 1 -- 172.21.15.119:0/2227364997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3e8c06c220 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.628 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:09:46.622+0000 7f8c64600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:09:46.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.626+0000 7f3e89600640 1 -- 172.21.15.119:0/2227364997 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3e8407a170 con 0x7f3e8c15dbe0 2024-04-25T10:09:46.632 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:09:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.629+0000 7f3e94400640 1 -- 172.21.15.119:0/2227364997 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e6c06b4d0 msgr2=0x7f3e6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.629+0000 7f3e94400640 1 --2- 172.21.15.119:0/2227364997 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e6c06b4d0 0x7f3e6c06d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3e80000f60 tx=0x7f3e800078f0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.629+0000 7f3e94400640 1 -- 172.21.15.119:0/2227364997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c15dbe0 msgr2=0x7f3e8c0ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.629+0000 7f3e94400640 1 --2- 172.21.15.119:0/2227364997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c15dbe0 0x7f3e8c0ce8c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009e50 tx=0x7f3e84004a40 comp rx=0 tx=0).stop 2024-04-25T10:09:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.629+0000 7f3e94400640 1 -- 172.21.15.119:0/2227364997 shutdown_connections 2024-04-25T10:09:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.629+0000 7f3e94400640 1 --2- 172.21.15.119:0/2227364997 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e6c06b4d0 0x7f3e6c06d990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.629+0000 7f3e94400640 1 --2- 172.21.15.119:0/2227364997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e8c15dbe0 0x7f3e8c0ce8c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.629+0000 7f3e94400640 1 --2- 172.21.15.119:0/2227364997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e8c0cf900 0x7f3e8c0ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.629+0000 7f3e94400640 1 --2- 172.21.15.119:0/2227364997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e8c0aedc0 0x7f3e8c0af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.629+0000 7f3e94400640 1 -- 172.21.15.119:0/2227364997 >> 172.21.15.119:0/2227364997 conn(0x7f3e8c06aba0 msgr2=0x7f3e8c158cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.630+0000 7f3e94400640 1 -- 172.21.15.119:0/2227364997 shutdown_connections 2024-04-25T10:09:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.631+0000 7f3e94400640 1 -- 172.21.15.119:0/2227364997 wait complete. 2024-04-25T10:09:46.650 INFO:tasks.ceph.mds.a.smithi119.stderr: -5854> 2024-04-25T10:09:03.610+0000 7f5287600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:09:46.657 INFO:tasks.ceph.mds.a.smithi119.stderr: -3227> 2024-04-25T10:09:07.368+0000 7f5282600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={4984=0-4194304@1} caps={4984=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=4984 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x560d9e018100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:09:46.660 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:09:46.707 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:09:46.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.833+0000 7fa026800640 1 Processor -- start 2024-04-25T10:09:46.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.834+0000 7fa026800640 1 -- start start 2024-04-25T10:09:46.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.835+0000 7fa026800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa0200b98b0 0x7fa02015eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.835+0000 7fa026800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0200b5580 0x7fa0200b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.835+0000 7fa026800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02015f080 con 0x7fa0200b24c0 2024-04-25T10:09:46.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.835+0000 7fa026800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa02015bb80 con 0x7fa0200b98b0 2024-04-25T10:09:46.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.835+0000 7fa026800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02006c4b0 con 0x7fa0200b5580 2024-04-25T10:09:46.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.836+0000 7fa01ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0200b5580 0x7fa0200b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.836+0000 7fa01ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0200b5580 0x7fa0200b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:40980/0 (socket says 172.21.15.119:40980) 2024-04-25T10:09:46.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.836+0000 7fa01ea00640 1 -- 172.21.15.119:0/1935303941 learned_addr learned my addr 172.21.15.119:0/1935303941 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:46.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.836+0000 7fa01f400640 1 --2- 172.21.15.119:0/1935303941 >> v2:172.21.15.132:3300/0 conn(0x7fa0200b98b0 0x7fa02015eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.836+0000 7fa01ea00640 1 -- 172.21.15.119:0/1935303941 >> v2:172.21.15.132:3300/0 conn(0x7fa0200b98b0 msgr2=0x7fa02015eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:46.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.836+0000 7fa01ea00640 1 --2- 172.21.15.119:0/1935303941 >> v2:172.21.15.132:3300/0 conn(0x7fa0200b98b0 0x7fa02015eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.836+0000 7fa01ea00640 1 -- 172.21.15.119:0/1935303941 >> v1:172.21.15.132:6789/0 conn(0x7fa0200b24c0 legacy=0x7fa0200b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.836+0000 7fa01ea00640 1 -- 172.21.15.119:0/1935303941 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02006c790 con 0x7fa0200b5580 2024-04-25T10:09:46.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.838+0000 7fa01ea00640 1 --2- 172.21.15.119:0/1935303941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0200b5580 0x7fa0200b5a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa010009e50 tx=0x7fa01002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e9e8da1fc8a74b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:46.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.838+0000 7fa01e000640 1 -- 172.21.15.119:0/1935303941 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010036070 con 0x7fa0200b5580 2024-04-25T10:09:46.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.839+0000 7fa01e000640 1 -- 172.21.15.119:0/1935303941 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa010031600 con 0x7fa0200b5580 2024-04-25T10:09:46.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.839+0000 7fa01e000640 1 -- 172.21.15.119:0/1935303941 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100319d0 con 0x7fa0200b5580 2024-04-25T10:09:46.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.839+0000 7fa026800640 1 -- 172.21.15.119:0/1935303941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0200b5580 msgr2=0x7fa0200b5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.839+0000 7fa026800640 1 --2- 172.21.15.119:0/1935303941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0200b5580 0x7fa0200b5a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa010009e50 tx=0x7fa01002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.839+0000 7fa026800640 1 -- 172.21.15.119:0/1935303941 shutdown_connections 2024-04-25T10:09:46.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.839+0000 7fa026800640 1 --2- 172.21.15.119:0/1935303941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0200b5580 0x7fa0200b5a00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.839+0000 7fa026800640 1 --2- 172.21.15.119:0/1935303941 >> v2:172.21.15.132:3300/0 conn(0x7fa0200b98b0 0x7fa02015eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.839+0000 7fa026800640 1 -- 172.21.15.119:0/1935303941 >> 172.21.15.119:0/1935303941 conn(0x7fa0200d0580 msgr2=0x7fa0200d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:46.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.839+0000 7fa026800640 1 -- 172.21.15.119:0/1935303941 shutdown_connections 2024-04-25T10:09:46.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.839+0000 7fa026800640 1 -- 172.21.15.119:0/1935303941 wait complete. 2024-04-25T10:09:46.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.841+0000 7fa026800640 1 Processor -- start 2024-04-25T10:09:46.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.841+0000 7fa026800640 1 -- start start 2024-04-25T10:09:46.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.842+0000 7fa026800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0200b24c0 0x7fa02014afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.842+0000 7fa026800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0200b5580 0x7fa0200c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.842+0000 7fa026800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0201f70d0 0x7fa0200b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.842+0000 7fa026800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02006c140 con 0x7fa0200b5580 2024-04-25T10:09:46.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.842+0000 7fa026800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02006cd50 con 0x7fa0201f70d0 2024-04-25T10:09:46.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.842+0000 7fa026800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0200cec40 con 0x7fa0200b24c0 2024-04-25T10:09:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.842+0000 7fa01ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0200b5580 0x7fa0200c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.842+0000 7fa01ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0200b5580 0x7fa0200c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39402/0 (socket says 172.21.15.119:39402) 2024-04-25T10:09:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.842+0000 7fa01ea00640 1 -- 172.21.15.119:0/801896020 learned_addr learned my addr 172.21.15.119:0/801896020 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.843+0000 7fa01f400640 1 --2- 172.21.15.119:0/801896020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0200b24c0 0x7fa02014afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.843+0000 7fa01fe00640 1 --2- 172.21.15.119:0/801896020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0201f70d0 0x7fa0200b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.843+0000 7fa01ea00640 1 -- 172.21.15.119:0/801896020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0200b24c0 msgr2=0x7fa02014afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.843+0000 7fa01ea00640 1 --2- 172.21.15.119:0/801896020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0200b24c0 0x7fa02014afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.843+0000 7fa01ea00640 1 -- 172.21.15.119:0/801896020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0201f70d0 msgr2=0x7fa0200b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.843+0000 7fa01ea00640 1 --2- 172.21.15.119:0/801896020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0201f70d0 0x7fa0200b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.843+0000 7fa01ea00640 1 -- 172.21.15.119:0/801896020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0201f7640 con 0x7fa0200b5580 2024-04-25T10:09:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.843+0000 7fa01ea00640 1 --2- 172.21.15.119:0/801896020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0200b5580 0x7fa0200c9e40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa01002abe0 tx=0x7fa010004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:46.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.844+0000 7fa017400640 1 -- 172.21.15.119:0/801896020 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01003f020 con 0x7fa0200b5580 2024-04-25T10:09:46.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.844+0000 7fa017400640 1 -- 172.21.15.119:0/801896020 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa010037040 con 0x7fa0200b5580 2024-04-25T10:09:46.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.844+0000 7fa017400640 1 -- 172.21.15.119:0/801896020 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010036070 con 0x7fa0200b5580 2024-04-25T10:09:46.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.844+0000 7fa026800640 1 -- 172.21.15.119:0/801896020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0201f7920 con 0x7fa0200b5580 2024-04-25T10:09:46.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.844+0000 7fa026800640 1 -- 172.21.15.119:0/801896020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0201f7ee0 con 0x7fa0200b5580 2024-04-25T10:09:46.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.845+0000 7fa026800640 1 -- 172.21.15.119:0/801896020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0200750b0 con 0x7fa0200b5580 2024-04-25T10:09:46.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.846+0000 7fa017400640 1 -- 172.21.15.119:0/801896020 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa010042020 con 0x7fa0200b5580 2024-04-25T10:09:46.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.847+0000 7fa017400640 1 --2- 172.21.15.119:0/801896020 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ffc06b500 0x7f9ffc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:46.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.847+0000 7fa017400640 1 -- 172.21.15.119:0/801896020 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa0100aa740 con 0x7fa0200b5580 2024-04-25T10:09:46.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.847+0000 7fa01f400640 1 --2- 172.21.15.119:0/801896020 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ffc06b500 0x7f9ffc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:46.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.848+0000 7fa01f400640 1 --2- 172.21.15.119:0/801896020 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ffc06b500 0x7f9ffc06d9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa008009ad0 tx=0x7fa008008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:46.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:46.851+0000 7fa017400640 1 -- 172.21.15.119:0/801896020 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa010049030 con 0x7fa0200b5580 2024-04-25T10:09:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.087+0000 7fa026800640 1 -- 172.21.15.119:0/801896020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa0200683b0 con 0x7fa0200b5580 2024-04-25T10:09:47.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.629+0000 7fa017400640 1 -- 172.21.15.119:0/801896020 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa0100794c0 con 0x7fa0200b5580 2024-04-25T10:09:47.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.633+0000 7fa026800640 1 -- 172.21.15.119:0/801896020 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ffc06b500 msgr2=0x7f9ffc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.633+0000 7fa026800640 1 --2- 172.21.15.119:0/801896020 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ffc06b500 0x7f9ffc06d9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa008009ad0 tx=0x7fa008008040 comp rx=0 tx=0).stop 2024-04-25T10:09:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.633+0000 7fa026800640 1 -- 172.21.15.119:0/801896020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0200b5580 msgr2=0x7fa0200c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.633+0000 7fa026800640 1 --2- 172.21.15.119:0/801896020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0200b5580 0x7fa0200c9e40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa01002abe0 tx=0x7fa010004a40 comp rx=0 tx=0).stop 2024-04-25T10:09:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.633+0000 7fa026800640 1 -- 172.21.15.119:0/801896020 shutdown_connections 2024-04-25T10:09:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.633+0000 7fa026800640 1 --2- 172.21.15.119:0/801896020 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ffc06b500 0x7f9ffc06d9c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.633+0000 7fa026800640 1 --2- 172.21.15.119:0/801896020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0201f70d0 0x7fa0200b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.633+0000 7fa026800640 1 --2- 172.21.15.119:0/801896020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0200b5580 0x7fa0200c9e40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.633+0000 7fa026800640 1 --2- 172.21.15.119:0/801896020 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0200b24c0 0x7fa02014afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.633+0000 7fa026800640 1 -- 172.21.15.119:0/801896020 >> 172.21.15.119:0/801896020 conn(0x7fa0200d0580 msgr2=0x7fa0200d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.633+0000 7fa026800640 1 -- 172.21.15.119:0/801896020 shutdown_connections 2024-04-25T10:09:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.633+0000 7fa026800640 1 -- 172.21.15.119:0/801896020 wait complete. 2024-04-25T10:09:47.661 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:09:47.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.859+0000 7f984fe00640 1 Processor -- start 2024-04-25T10:09:47.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.859+0000 7f984fe00640 1 -- start start 2024-04-25T10:09:47.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.860+0000 7f984fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9850144fd0 0x7f985015f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.860+0000 7f984fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f98500cf180 0x7f98500ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.860+0000 7f984fe00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985015bdf0 con 0x7f985006e670 2024-04-25T10:09:47.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.860+0000 7f984fe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985006d4e0 con 0x7f98500cf180 2024-04-25T10:09:47.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.860+0000 7f984fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985006d7c0 con 0x7f9850144fd0 2024-04-25T10:09:47.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.861+0000 7f984f400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f985006e670 0x7f985015c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58408/0 (socket says 172.21.15.119:58408) 2024-04-25T10:09:47.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.861+0000 7f984f400640 1 -- 172.21.15.119:0/4166458284 learned_addr learned my addr 172.21.15.119:0/4166458284 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.861+0000 7f984ea00640 1 --2- 172.21.15.119:0/4166458284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9850144fd0 0x7f985015f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.861+0000 7f9846000640 1 --2- 172.21.15.119:0/4166458284 >> v2:172.21.15.132:3300/0 conn(0x7f98500cf180 0x7f98500ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.861+0000 7f984e000640 1 -- 172.21.15.119:0/4166458284 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260327375 0 0) 0x7f985015bdf0 con 0x7f985006e670 2024-04-25T10:09:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.861+0000 7f984e000640 1 -- 172.21.15.119:0/4166458284 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9834003520 con 0x7f985006e670 2024-04-25T10:09:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984e000640 1 -- 172.21.15.119:0/4166458284 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3755834534 0 0) 0x7f9834003520 con 0x7f985006e670 2024-04-25T10:09:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984e000640 1 -- 172.21.15.119:0/4166458284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9850144fd0 msgr2=0x7f985015f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984e000640 1 --2- 172.21.15.119:0/4166458284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9850144fd0 0x7f985015f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984e000640 1 -- 172.21.15.119:0/4166458284 >> v2:172.21.15.132:3300/0 conn(0x7f98500cf180 msgr2=0x7f98500ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984e000640 1 --2- 172.21.15.119:0/4166458284 >> v2:172.21.15.132:3300/0 conn(0x7f98500cf180 0x7f98500ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984e000640 1 -- 172.21.15.119:0/4166458284 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985006daa0 con 0x7f985006e670 2024-04-25T10:09:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984e000640 1 -- 172.21.15.119:0/4166458284 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f9848001670 con 0x7f985006e670 2024-04-25T10:09:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984e000640 1 -- 172.21.15.119:0/4166458284 >> v1:172.21.15.119:6789/0 conn(0x7f985006e670 legacy=0x7f985015c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984e000640 1 --2- 172.21.15.119:0/4166458284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9850144fd0 0x7f98340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984e000640 1 --2- 172.21.15.119:0/4166458284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9834003f30 0x7f9834004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984e000640 1 --2- 172.21.15.119:0/4166458284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9834004d40 0x7f98340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984e000640 1 -- 172.21.15.119:0/4166458284 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985006d4e0 con 0x7f9834003f30 2024-04-25T10:09:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984e000640 1 -- 172.21.15.119:0/4166458284 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985006d7c0 con 0x7f9834004d40 2024-04-25T10:09:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984e000640 1 -- 172.21.15.119:0/4166458284 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983400ba10 con 0x7f9850144fd0 2024-04-25T10:09:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f9846000640 1 --2- 172.21.15.119:0/4166458284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9834003f30 0x7f9834004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.862+0000 7f984ea00640 1 --2- 172.21.15.119:0/4166458284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9850144fd0 0x7f98340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.863+0000 7f9846000640 1 -- 172.21.15.119:0/4166458284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9850144fd0 msgr2=0x7f98340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.863+0000 7f9846000640 1 --2- 172.21.15.119:0/4166458284 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9850144fd0 0x7f98340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.863+0000 7f9846000640 1 -- 172.21.15.119:0/4166458284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9834004d40 msgr2=0x7f98340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.863+0000 7f9846000640 1 --2- 172.21.15.119:0/4166458284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9834004d40 0x7f98340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.863+0000 7f9846000640 1 -- 172.21.15.119:0/4166458284 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9834003520 con 0x7f9834003f30 2024-04-25T10:09:47.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.864+0000 7f9846000640 1 --2- 172.21.15.119:0/4166458284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9834003f30 0x7f9834004800 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9838009af0 tx=0x7f9834005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4516e63e30719563 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:47.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.864+0000 7f984e000640 1 -- 172.21.15.119:0/4166458284 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838036070 con 0x7f9834003f30 2024-04-25T10:09:47.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.864+0000 7f984e000640 1 -- 172.21.15.119:0/4166458284 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9838031400 con 0x7f9834003f30 2024-04-25T10:09:47.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.864+0000 7f984e000640 1 -- 172.21.15.119:0/4166458284 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838009690 con 0x7f9834003f30 2024-04-25T10:09:47.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.864+0000 7f984fe00640 1 -- 172.21.15.119:0/4166458284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9834003f30 msgr2=0x7f9834004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.864+0000 7f984fe00640 1 --2- 172.21.15.119:0/4166458284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9834003f30 0x7f9834004800 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9838009af0 tx=0x7f9834005830 comp rx=0 tx=0).stop 2024-04-25T10:09:47.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.864+0000 7f984f400640 1 -- 172.21.15.119:0/4166458284 reap_dead start 2024-04-25T10:09:47.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.865+0000 7f984fe00640 1 -- 172.21.15.119:0/4166458284 shutdown_connections 2024-04-25T10:09:47.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.865+0000 7f984fe00640 1 -- 172.21.15.119:0/4166458284 >> 172.21.15.119:0/4166458284 conn(0x7f985006aba0 msgr2=0x7f985006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:47.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.865+0000 7f984fe00640 1 -- 172.21.15.119:0/4166458284 shutdown_connections 2024-04-25T10:09:47.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.865+0000 7f984fe00640 1 -- 172.21.15.119:0/4166458284 wait complete. 2024-04-25T10:09:47.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.866+0000 7f984fe00640 1 Processor -- start 2024-04-25T10:09:47.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.867+0000 7f984fe00640 1 -- start start 2024-04-25T10:09:47.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.867+0000 7f984fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9834003f30 0x7f9850142760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.867+0000 7f984fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98501274a0 0x7f98501f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.867+0000 7f984fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9850151a30 0x7f9850151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.868+0000 7f984fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98501453b0 con 0x7f9834003f30 2024-04-25T10:09:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.868+0000 7f984fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985006d190 con 0x7f9850151a30 2024-04-25T10:09:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.868+0000 7f984fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9850145530 con 0x7f98501274a0 2024-04-25T10:09:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.868+0000 7f984ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9834003f30 0x7f9850142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.868+0000 7f984ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9834003f30 0x7f9850142760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39426/0 (socket says 172.21.15.119:39426) 2024-04-25T10:09:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.868+0000 7f984ea00640 1 -- 172.21.15.119:0/3797220564 learned_addr learned my addr 172.21.15.119:0/3797220564 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.868+0000 7f9846000640 1 --2- 172.21.15.119:0/3797220564 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98501274a0 0x7f98501f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.868+0000 7f984ea00640 1 -- 172.21.15.119:0/3797220564 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98501274a0 msgr2=0x7f98501f32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.868+0000 7f984f400640 1 --2- 172.21.15.119:0/3797220564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9850151a30 0x7f9850151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.868+0000 7f984ea00640 1 --2- 172.21.15.119:0/3797220564 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98501274a0 0x7f98501f32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.868+0000 7f984ea00640 1 -- 172.21.15.119:0/3797220564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9850151a30 msgr2=0x7f9850151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.868+0000 7f984ea00640 1 --2- 172.21.15.119:0/3797220564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9850151a30 0x7f9850151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.868+0000 7f984ea00640 1 -- 172.21.15.119:0/3797220564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98501416c0 con 0x7f9834003f30 2024-04-25T10:09:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.869+0000 7f984ea00640 1 --2- 172.21.15.119:0/3797220564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9834003f30 0x7f9850142760 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9840007f30 tx=0x7f9840011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.869+0000 7f9847400640 1 -- 172.21.15.119:0/3797220564 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984001a020 con 0x7f9834003f30 2024-04-25T10:09:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.869+0000 7f9847400640 1 -- 172.21.15.119:0/3797220564 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9840016470 con 0x7f9834003f30 2024-04-25T10:09:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.869+0000 7f9847400640 1 -- 172.21.15.119:0/3797220564 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840016860 con 0x7f9834003f30 2024-04-25T10:09:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.869+0000 7f984fe00640 1 -- 172.21.15.119:0/3797220564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98501419a0 con 0x7f9834003f30 2024-04-25T10:09:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.869+0000 7f984fe00640 1 -- 172.21.15.119:0/3797220564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9850141f60 con 0x7f9834003f30 2024-04-25T10:09:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.870+0000 7f984fe00640 1 -- 172.21.15.119:0/3797220564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985006bb30 con 0x7f9834003f30 2024-04-25T10:09:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.871+0000 7f9847400640 1 -- 172.21.15.119:0/3797220564 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9840005d10 con 0x7f9834003f30 2024-04-25T10:09:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.872+0000 7f9847400640 1 --2- 172.21.15.119:0/3797220564 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f982c06b500 0x7f982c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.872+0000 7f9847400640 1 -- 172.21.15.119:0/3797220564 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f984008b120 con 0x7f9834003f30 2024-04-25T10:09:47.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.872+0000 7f9846000640 1 --2- 172.21.15.119:0/3797220564 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f982c06b500 0x7f982c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:47.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.873+0000 7f9846000640 1 --2- 172.21.15.119:0/3797220564 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f982c06b500 0x7f982c06d9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9838033000 tx=0x7f9838008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:47.877+0000 7f9847400640 1 -- 172.21.15.119:0/3797220564 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9840024030 con 0x7f9834003f30 2024-04-25T10:09:48.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:48.074+0000 7f984fe00640 1 -- 172.21.15.119:0/3797220564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f98500cef60 con 0x7f9834003f30 2024-04-25T10:09:48.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:48.904+0000 7f9847400640 1 -- 172.21.15.119:0/3797220564 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v46) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9840059f30 con 0x7f9834003f30 2024-04-25T10:09:48.910 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:09:48.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:48.909+0000 7f984fe00640 1 -- 172.21.15.119:0/3797220564 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f982c06b500 msgr2=0x7f982c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:48.909+0000 7f984fe00640 1 --2- 172.21.15.119:0/3797220564 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f982c06b500 0x7f982c06d9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9838033000 tx=0x7f9838008030 comp rx=0 tx=0).stop 2024-04-25T10:09:48.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:48.909+0000 7f984fe00640 1 -- 172.21.15.119:0/3797220564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9834003f30 msgr2=0x7f9850142760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:48.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:48.909+0000 7f984fe00640 1 --2- 172.21.15.119:0/3797220564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9834003f30 0x7f9850142760 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9840007f30 tx=0x7f9840011660 comp rx=0 tx=0).stop 2024-04-25T10:09:48.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:48.909+0000 7f984fe00640 1 -- 172.21.15.119:0/3797220564 shutdown_connections 2024-04-25T10:09:48.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:48.909+0000 7f984fe00640 1 --2- 172.21.15.119:0/3797220564 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f982c06b500 0x7f982c06d9c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:48.909+0000 7f984fe00640 1 --2- 172.21.15.119:0/3797220564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9850151a30 0x7f9850151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:48.909+0000 7f984fe00640 1 --2- 172.21.15.119:0/3797220564 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98501274a0 0x7f98501f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:48.909+0000 7f984fe00640 1 --2- 172.21.15.119:0/3797220564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9834003f30 0x7f9850142760 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:48.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:48.909+0000 7f984fe00640 1 -- 172.21.15.119:0/3797220564 >> 172.21.15.119:0/3797220564 conn(0x7f985006aba0 msgr2=0x7f98500cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:48.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:48.909+0000 7f984fe00640 1 -- 172.21.15.119:0/3797220564 shutdown_connections 2024-04-25T10:09:48.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:48.909+0000 7f984fe00640 1 -- 172.21.15.119:0/3797220564 wait complete. 2024-04-25T10:09:48.938 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:09:49.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.135+0000 7f2812000640 1 Processor -- start 2024-04-25T10:09:49.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.135+0000 7f2812000640 1 -- start start 2024-04-25T10:09:49.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.137+0000 7f2812000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f280c144fd0 0x7f280c15f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:49.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.137+0000 7f2812000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0cea40 0x7f280c15c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:49.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.137+0000 7f2812000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280c15bdf0 con 0x7f280c0cf180 2024-04-25T10:09:49.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.137+0000 7f2812000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280c06d4e0 con 0x7f280c144fd0 2024-04-25T10:09:49.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.137+0000 7f2812000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c06d7c0 con 0x7f280c0cea40 2024-04-25T10:09:49.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.137+0000 7f2810c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f280c144fd0 0x7f280c15f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:49.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.137+0000 7f2810c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f280c144fd0 0x7f280c15f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39434/0 (socket says 172.21.15.119:39434) 2024-04-25T10:09:49.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.137+0000 7f2810c00640 1 -- 172.21.15.119:0/4063320161 learned_addr learned my addr 172.21.15.119:0/4063320161 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:49.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.138+0000 7f2811600640 1 --2- 172.21.15.119:0/4063320161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0cea40 0x7f280c15c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:49.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.138+0000 7f2810c00640 1 -- 172.21.15.119:0/4063320161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0cea40 msgr2=0x7f280c15c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:49.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.138+0000 7f2810c00640 1 --2- 172.21.15.119:0/4063320161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0cea40 0x7f280c15c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.138+0000 7f2810c00640 1 -- 172.21.15.119:0/4063320161 >> v1:172.21.15.119:6789/0 conn(0x7f280c0cf180 legacy=0x7f280c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:49.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.138+0000 7f2810c00640 1 -- 172.21.15.119:0/4063320161 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c06daa0 con 0x7f280c144fd0 2024-04-25T10:09:49.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.139+0000 7f2810c00640 1 --2- 172.21.15.119:0/4063320161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c144fd0 0x7f280c15f400 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2800009870 tx=0x7f280002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9b49bb31a216ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:49.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.139+0000 7f280b400640 1 -- 172.21.15.119:0/4063320161 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800037070 con 0x7f280c144fd0 2024-04-25T10:09:49.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.139+0000 7f280b400640 1 -- 172.21.15.119:0/4063320161 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2800032440 con 0x7f280c144fd0 2024-04-25T10:09:49.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.139+0000 7f280b400640 1 -- 172.21.15.119:0/4063320161 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800032810 con 0x7f280c144fd0 2024-04-25T10:09:49.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.140+0000 7f2812000640 1 -- 172.21.15.119:0/4063320161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c144fd0 msgr2=0x7f280c15f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:49.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.140+0000 7f2812000640 1 --2- 172.21.15.119:0/4063320161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c144fd0 0x7f280c15f400 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2800009870 tx=0x7f280002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.140+0000 7f2812000640 1 -- 172.21.15.119:0/4063320161 shutdown_connections 2024-04-25T10:09:49.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.140+0000 7f2812000640 1 --2- 172.21.15.119:0/4063320161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0cea40 0x7f280c15c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.140+0000 7f2812000640 1 --2- 172.21.15.119:0/4063320161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c144fd0 0x7f280c15f400 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.140+0000 7f2812000640 1 -- 172.21.15.119:0/4063320161 >> 172.21.15.119:0/4063320161 conn(0x7f280c06aba0 msgr2=0x7f280c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:49.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.140+0000 7f2812000640 1 -- 172.21.15.119:0/4063320161 shutdown_connections 2024-04-25T10:09:49.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.140+0000 7f2812000640 1 -- 172.21.15.119:0/4063320161 wait complete. 2024-04-25T10:09:49.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.142+0000 7f2812000640 1 Processor -- start 2024-04-25T10:09:49.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.142+0000 7f2812000640 1 -- start start 2024-04-25T10:09:49.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.143+0000 7f2812000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0cea40 0x7f280c1f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:49.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.143+0000 7f2812000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f280c0cf180 0x7f280c13aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:49.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.143+0000 7f2812000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c144fd0 0x7f280c1f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:49.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.143+0000 7f2812000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c15f940 con 0x7f280c144fd0 2024-04-25T10:09:49.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.143+0000 7f2812000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c06d190 con 0x7f280c0cf180 2024-04-25T10:09:49.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.143+0000 7f2812000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c1453b0 con 0x7f280c0cea40 2024-04-25T10:09:49.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.144+0000 7f2811600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c144fd0 0x7f280c1f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:49.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.144+0000 7f2811600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c144fd0 0x7f280c1f4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39436/0 (socket says 172.21.15.119:39436) 2024-04-25T10:09:49.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.144+0000 7f2811600640 1 -- 172.21.15.119:0/214532444 learned_addr learned my addr 172.21.15.119:0/214532444 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:49.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.144+0000 7f2810c00640 1 --2- 172.21.15.119:0/214532444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0cea40 0x7f280c1f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:49.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.144+0000 7f280be00640 1 --2- 172.21.15.119:0/214532444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f280c0cf180 0x7f280c13aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:49.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.144+0000 7f2811600640 1 -- 172.21.15.119:0/214532444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0cea40 msgr2=0x7f280c1f6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:49.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.144+0000 7f2811600640 1 --2- 172.21.15.119:0/214532444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0cea40 0x7f280c1f6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.144+0000 7f2811600640 1 -- 172.21.15.119:0/214532444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f280c0cf180 msgr2=0x7f280c13aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:49.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.144+0000 7f2811600640 1 --2- 172.21.15.119:0/214532444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f280c0cf180 0x7f280c13aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.144+0000 7f2811600640 1 -- 172.21.15.119:0/214532444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c13b430 con 0x7f280c144fd0 2024-04-25T10:09:49.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.144+0000 7f2811600640 1 --2- 172.21.15.119:0/214532444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c144fd0 0x7f280c1f4110 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2804007e40 tx=0x7f280400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:49.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.145+0000 7f2808c00640 1 -- 172.21.15.119:0/214532444 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804014620 con 0x7f280c144fd0 2024-04-25T10:09:49.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.145+0000 7f2808c00640 1 -- 172.21.15.119:0/214532444 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2804019070 con 0x7f280c144fd0 2024-04-25T10:09:49.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.145+0000 7f2812000640 1 -- 172.21.15.119:0/214532444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280c13b710 con 0x7f280c144fd0 2024-04-25T10:09:49.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.145+0000 7f2808c00640 1 -- 172.21.15.119:0/214532444 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804014a20 con 0x7f280c144fd0 2024-04-25T10:09:49.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.145+0000 7f2812000640 1 -- 172.21.15.119:0/214532444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c13bcd0 con 0x7f280c144fd0 2024-04-25T10:09:49.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.146+0000 7f2812000640 1 -- 172.21.15.119:0/214532444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280c0749b0 con 0x7f280c144fd0 2024-04-25T10:09:49.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.147+0000 7f2808c00640 1 -- 172.21.15.119:0/214532444 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2804014bc0 con 0x7f280c144fd0 2024-04-25T10:09:49.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.148+0000 7f2808c00640 1 --2- 172.21.15.119:0/214532444 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f27ec06b4d0 0x7f27ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:49.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.148+0000 7f2808c00640 1 -- 172.21.15.119:0/214532444 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f280408aa10 con 0x7f280c144fd0 2024-04-25T10:09:49.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.148+0000 7f2810c00640 1 --2- 172.21.15.119:0/214532444 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f27ec06b4d0 0x7f27ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:49.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.149+0000 7f2810c00640 1 --2- 172.21.15.119:0/214532444 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f27ec06b4d0 0x7f27ec06d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2800034000 tx=0x7f280002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:49.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.152+0000 7f2808c00640 1 -- 172.21.15.119:0/214532444 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2804059950 con 0x7f280c144fd0 2024-04-25T10:09:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.351+0000 7f2812000640 1 -- 172.21.15.119:0/214532444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f280c0cef60 con 0x7f280c144fd0 2024-04-25T10:09:49.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.907+0000 7f2808c00640 1 -- 172.21.15.119:0/214532444 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v47) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2804059250 con 0x7f280c144fd0 2024-04-25T10:09:49.914 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2024-04-25T10:09:49.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.912+0000 7f2812000640 1 -- 172.21.15.119:0/214532444 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f27ec06b4d0 msgr2=0x7f27ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:49.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.912+0000 7f2812000640 1 --2- 172.21.15.119:0/214532444 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f27ec06b4d0 0x7f27ec06d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2800034000 tx=0x7f280002e000 comp rx=0 tx=0).stop 2024-04-25T10:09:49.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.912+0000 7f2812000640 1 -- 172.21.15.119:0/214532444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c144fd0 msgr2=0x7f280c1f4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:49.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.912+0000 7f2812000640 1 --2- 172.21.15.119:0/214532444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c144fd0 0x7f280c1f4110 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2804007e40 tx=0x7f280400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.912+0000 7f2812000640 1 -- 172.21.15.119:0/214532444 shutdown_connections 2024-04-25T10:09:49.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.912+0000 7f2812000640 1 --2- 172.21.15.119:0/214532444 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f27ec06b4d0 0x7f27ec06d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.912+0000 7f2812000640 1 --2- 172.21.15.119:0/214532444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c144fd0 0x7f280c1f4110 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.912+0000 7f2812000640 1 --2- 172.21.15.119:0/214532444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f280c0cf180 0x7f280c13aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.912+0000 7f2812000640 1 --2- 172.21.15.119:0/214532444 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0cea40 0x7f280c1f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:49.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.912+0000 7f2812000640 1 -- 172.21.15.119:0/214532444 >> 172.21.15.119:0/214532444 conn(0x7f280c06aba0 msgr2=0x7f280c15c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:49.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.913+0000 7f2812000640 1 -- 172.21.15.119:0/214532444 shutdown_connections 2024-04-25T10:09:49.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:49.913+0000 7f2812000640 1 -- 172.21.15.119:0/214532444 wait complete. 2024-04-25T10:09:49.947 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:09:50.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.136+0000 7f32c1800640 1 Processor -- start 2024-04-25T10:09:50.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.136+0000 7f32c1800640 1 -- start start 2024-04-25T10:09:50.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.137+0000 7f32c1800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32bc144e70 0x7f32bc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.137+0000 7f32c1800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f32bc06d7a0 0x7f32bc0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.137+0000 7f32c1800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc06dcb0 con 0x7f32bc0cef10 2024-04-25T10:09:50.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.137+0000 7f32c1800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32bc15bfe0 con 0x7f32bc06d7a0 2024-04-25T10:09:50.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.137+0000 7f32c1800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc06df90 con 0x7f32bc144e70 2024-04-25T10:09:50.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.138+0000 7f32bb400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f32bc0cef10 0x7f32bc071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58434/0 (socket says 172.21.15.119:58434) 2024-04-25T10:09:50.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.138+0000 7f32bb400640 1 -- 172.21.15.119:0/3185818529 learned_addr learned my addr 172.21.15.119:0/3185818529 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:50.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.138+0000 7f32ba000640 1 --2- 172.21.15.119:0/3185818529 >> v2:172.21.15.132:3300/0 conn(0x7f32bc06d7a0 0x7f32bc0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.138+0000 7f32baa00640 1 --2- 172.21.15.119:0/3185818529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32bc144e70 0x7f32bc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.138+0000 7f32b9600640 1 -- 172.21.15.119:0/3185818529 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492483926 0 0) 0x7f32bc06dcb0 con 0x7f32bc0cef10 2024-04-25T10:09:50.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.138+0000 7f32b9600640 1 -- 172.21.15.119:0/3185818529 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32a0003520 con 0x7f32bc0cef10 2024-04-25T10:09:50.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32b9600640 1 -- 172.21.15.119:0/3185818529 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1794332586 0 0) 0x7f32a0003520 con 0x7f32bc0cef10 2024-04-25T10:09:50.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32b9600640 1 -- 172.21.15.119:0/3185818529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32bc144e70 msgr2=0x7f32bc06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32b9600640 1 --2- 172.21.15.119:0/3185818529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32bc144e70 0x7f32bc06d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32b9600640 1 -- 172.21.15.119:0/3185818529 >> v2:172.21.15.132:3300/0 conn(0x7f32bc06d7a0 msgr2=0x7f32bc0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32b9600640 1 --2- 172.21.15.119:0/3185818529 >> v2:172.21.15.132:3300/0 conn(0x7f32bc06d7a0 0x7f32bc0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32b9600640 1 -- 172.21.15.119:0/3185818529 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc06e270 con 0x7f32bc0cef10 2024-04-25T10:09:50.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32b9600640 1 -- 172.21.15.119:0/3185818529 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f32b4001a10 con 0x7f32bc0cef10 2024-04-25T10:09:50.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32b9600640 1 -- 172.21.15.119:0/3185818529 >> v1:172.21.15.119:6789/0 conn(0x7f32bc0cef10 legacy=0x7f32bc071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32b9600640 1 --2- 172.21.15.119:0/3185818529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32bc144e70 0x7f32a00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32b9600640 1 --2- 172.21.15.119:0/3185818529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a0003f30 0x7f32a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32b9600640 1 --2- 172.21.15.119:0/3185818529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a0004d40 0x7f32a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32b9600640 1 -- 172.21.15.119:0/3185818529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc15bfe0 con 0x7f32a0003f30 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32b9600640 1 -- 172.21.15.119:0/3185818529 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc06df90 con 0x7f32a0004d40 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32b9600640 1 -- 172.21.15.119:0/3185818529 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a000ba10 con 0x7f32bc144e70 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.139+0000 7f32ba000640 1 --2- 172.21.15.119:0/3185818529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a0003f30 0x7f32a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.140+0000 7f32baa00640 1 --2- 172.21.15.119:0/3185818529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32bc144e70 0x7f32a00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.140+0000 7f32bb400640 1 --2- 172.21.15.119:0/3185818529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a0004d40 0x7f32a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.140+0000 7f32ba000640 1 -- 172.21.15.119:0/3185818529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32bc144e70 msgr2=0x7f32a00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.140+0000 7f32ba000640 1 --2- 172.21.15.119:0/3185818529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32bc144e70 0x7f32a00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.140+0000 7f32ba000640 1 -- 172.21.15.119:0/3185818529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a0004d40 msgr2=0x7f32a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.140+0000 7f32ba000640 1 --2- 172.21.15.119:0/3185818529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a0004d40 0x7f32a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.140+0000 7f32ba000640 1 -- 172.21.15.119:0/3185818529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a0003520 con 0x7f32a0003f30 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.140+0000 7f32bb400640 1 --2- 172.21.15.119:0/3185818529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a0004d40 0x7f32a00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:50.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.141+0000 7f32ba000640 1 --2- 172.21.15.119:0/3185818529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a0003f30 0x7f32a0004800 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009ac0 tx=0x7f32a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cf0b39289b7d80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.141+0000 7f32b9600640 1 -- 172.21.15.119:0/3185818529 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8036070 con 0x7f32a0003f30 2024-04-25T10:09:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.141+0000 7f32b9600640 1 -- 172.21.15.119:0/3185818529 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a8031400 con 0x7f32a0003f30 2024-04-25T10:09:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.141+0000 7f32b9600640 1 -- 172.21.15.119:0/3185818529 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8009690 con 0x7f32a0003f30 2024-04-25T10:09:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.142+0000 7f32c1800640 1 -- 172.21.15.119:0/3185818529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a0003f30 msgr2=0x7f32a0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.142+0000 7f32c1800640 1 --2- 172.21.15.119:0/3185818529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a0003f30 0x7f32a0004800 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009ac0 tx=0x7f32a0005830 comp rx=0 tx=0).stop 2024-04-25T10:09:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.142+0000 7f32bb400640 1 -- 172.21.15.119:0/3185818529 reap_dead start 2024-04-25T10:09:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.142+0000 7f32c1800640 1 -- 172.21.15.119:0/3185818529 shutdown_connections 2024-04-25T10:09:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.142+0000 7f32c1800640 1 -- 172.21.15.119:0/3185818529 >> 172.21.15.119:0/3185818529 conn(0x7f32bc06aba0 msgr2=0x7f32bc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.142+0000 7f32c1800640 1 -- 172.21.15.119:0/3185818529 shutdown_connections 2024-04-25T10:09:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.142+0000 7f32c1800640 1 -- 172.21.15.119:0/3185818529 wait complete. 2024-04-25T10:09:50.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.144+0000 7f32c1800640 1 Processor -- start 2024-04-25T10:09:50.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.144+0000 7f32c1800640 1 -- start start 2024-04-25T10:09:50.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.145+0000 7f32c1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a0003f30 0x7f32bc13a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.145+0000 7f32c1800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32bc0b1990 0x7f32bc13c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.145+0000 7f32c1800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32bc15ad10 0x7f32bc1ef780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.145+0000 7f32c1800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc1454b0 con 0x7f32a0003f30 2024-04-25T10:09:50.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.145+0000 7f32c1800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0cf380 con 0x7f32bc0b1990 2024-04-25T10:09:50.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.145+0000 7f32c1800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc145630 con 0x7f32bc15ad10 2024-04-25T10:09:50.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.145+0000 7f32baa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a0003f30 0x7f32bc13a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.145+0000 7f32baa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a0003f30 0x7f32bc13a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39450/0 (socket says 172.21.15.119:39450) 2024-04-25T10:09:50.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.145+0000 7f32baa00640 1 -- 172.21.15.119:0/3754431208 learned_addr learned my addr 172.21.15.119:0/3754431208 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:50.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.146+0000 7f32baa00640 1 -- 172.21.15.119:0/3754431208 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32bc15ad10 msgr2=0x7f32bc1ef780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.146+0000 7f32baa00640 1 --2- 172.21.15.119:0/3754431208 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32bc15ad10 0x7f32bc1ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.146+0000 7f32baa00640 1 -- 172.21.15.119:0/3754431208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32bc0b1990 msgr2=0x7f32bc13c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.146+0000 7f32baa00640 1 --2- 172.21.15.119:0/3754431208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32bc0b1990 0x7f32bc13c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.146+0000 7f32baa00640 1 -- 172.21.15.119:0/3754431208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc13c540 con 0x7f32a0003f30 2024-04-25T10:09:50.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.146+0000 7f32baa00640 1 --2- 172.21.15.119:0/3754431208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a0003f30 0x7f32bc13a710 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f32ac011600 tx=0x7f32ac011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.146+0000 7f32b2a00640 1 -- 172.21.15.119:0/3754431208 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00f920 con 0x7f32a0003f30 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.146+0000 7f32b2a00640 1 -- 172.21.15.119:0/3754431208 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac00fac0 con 0x7f32a0003f30 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.146+0000 7f32b2a00640 1 -- 172.21.15.119:0/3754431208 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00fe70 con 0x7f32a0003f30 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.146+0000 7f32c1800640 1 -- 172.21.15.119:0/3754431208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32bc0b1ec0 con 0x7f32a0003f30 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.146+0000 7f32c1800640 1 -- 172.21.15.119:0/3754431208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc0af6d0 con 0x7f32a0003f30 2024-04-25T10:09:50.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.148+0000 7f32c1800640 1 -- 172.21.15.119:0/3754431208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32bc0720b0 con 0x7f32a0003f30 2024-04-25T10:09:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.149+0000 7f32b2a00640 1 -- 172.21.15.119:0/3754431208 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32ac015460 con 0x7f32a0003f30 2024-04-25T10:09:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.149+0000 7f32b2a00640 1 --2- 172.21.15.119:0/3754431208 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f329c06b4d0 0x7f329c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.150+0000 7f32b2a00640 1 -- 172.21.15.119:0/3754431208 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f32ac08a7b0 con 0x7f32a0003f30 2024-04-25T10:09:50.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.150+0000 7f32ba000640 1 --2- 172.21.15.119:0/3754431208 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f329c06b4d0 0x7f329c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.151+0000 7f32ba000640 1 --2- 172.21.15.119:0/3754431208 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f329c06b4d0 0x7f329c06d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f32a8033000 tx=0x7f32a802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.154+0000 7f32b2a00640 1 -- 172.21.15.119:0/3754431208 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32bc0720b0 con 0x7f32a0003f30 2024-04-25T10:09:50.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.350+0000 7f32c1800640 1 -- 172.21.15.119:0/3754431208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f32bc0750b0 con 0x7f32a0003f30 2024-04-25T10:09:50.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.357+0000 7f32b2a00640 1 -- 172.21.15.119:0/3754431208 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f32ac019030 con 0x7f32a0003f30 2024-04-25T10:09:50.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.360+0000 7f32c1800640 1 -- 172.21.15.119:0/3754431208 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f329c06b4d0 msgr2=0x7f329c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.360+0000 7f32c1800640 1 --2- 172.21.15.119:0/3754431208 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f329c06b4d0 0x7f329c06d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f32a8033000 tx=0x7f32a802e000 comp rx=0 tx=0).stop 2024-04-25T10:09:50.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.360+0000 7f32c1800640 1 -- 172.21.15.119:0/3754431208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a0003f30 msgr2=0x7f32bc13a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.360+0000 7f32c1800640 1 --2- 172.21.15.119:0/3754431208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a0003f30 0x7f32bc13a710 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f32ac011600 tx=0x7f32ac011ac0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.360+0000 7f32c1800640 1 -- 172.21.15.119:0/3754431208 shutdown_connections 2024-04-25T10:09:50.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.360+0000 7f32c1800640 1 --2- 172.21.15.119:0/3754431208 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f329c06b4d0 0x7f329c06d990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.360+0000 7f32c1800640 1 --2- 172.21.15.119:0/3754431208 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32bc15ad10 0x7f32bc1ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.360+0000 7f32c1800640 1 --2- 172.21.15.119:0/3754431208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32bc0b1990 0x7f32bc13c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.360+0000 7f32c1800640 1 --2- 172.21.15.119:0/3754431208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a0003f30 0x7f32bc13a710 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.360+0000 7f32c1800640 1 -- 172.21.15.119:0/3754431208 >> 172.21.15.119:0/3754431208 conn(0x7f32bc06aba0 msgr2=0x7f32bc141bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.360+0000 7f32c1800640 1 -- 172.21.15.119:0/3754431208 shutdown_connections 2024-04-25T10:09:50.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.360+0000 7f32c1800640 1 -- 172.21.15.119:0/3754431208 wait complete. 2024-04-25T10:09:50.385 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T10:09:50.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.579+0000 7fe1aa800640 1 Processor -- start 2024-04-25T10:09:50.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.579+0000 7fe1aa800640 1 -- start start 2024-04-25T10:09:50.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.581+0000 7fe1aa800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe1a4144470 0x7fe1a415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.581+0000 7fe1aa800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe1a406e430 0x7fe1a40b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.581+0000 7fe1aa800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a40cd440 con 0x7fe1a40b4e80 2024-04-25T10:09:50.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.581+0000 7fe1aa800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a4145340 con 0x7fe1a4144470 2024-04-25T10:09:50.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.581+0000 7fe1aa800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a40cd720 con 0x7fe1a406e430 2024-04-25T10:09:50.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.581+0000 7fe1a9e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe1a40b4e80 0x7fe1a406d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58436/0 (socket says 172.21.15.119:58436) 2024-04-25T10:09:50.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.582+0000 7fe1a9e00640 1 -- 172.21.15.119:0/4188176814 learned_addr learned my addr 172.21.15.119:0/4188176814 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:50.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.582+0000 7fe1a9400640 1 --2- 172.21.15.119:0/4188176814 >> v2:172.21.15.119:3300/0 conn(0x7fe1a4144470 0x7fe1a415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.582+0000 7fe1a8a00640 1 --2- 172.21.15.119:0/4188176814 >> v2:172.21.15.132:3300/0 conn(0x7fe1a406e430 0x7fe1a40b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.582+0000 7fe1a3e00640 1 -- 172.21.15.119:0/4188176814 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239025435 0 0) 0x7fe1a40cd440 con 0x7fe1a40b4e80 2024-04-25T10:09:50.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.582+0000 7fe1a3e00640 1 -- 172.21.15.119:0/4188176814 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe18c003520 con 0x7fe1a40b4e80 2024-04-25T10:09:50.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.582+0000 7fe1a9400640 1 -- 172.21.15.119:0/4188176814 >> v2:172.21.15.132:3300/0 conn(0x7fe1a406e430 msgr2=0x7fe1a40b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.582+0000 7fe1a9400640 1 --2- 172.21.15.119:0/4188176814 >> v2:172.21.15.132:3300/0 conn(0x7fe1a406e430 0x7fe1a40b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.582+0000 7fe1a9400640 1 -- 172.21.15.119:0/4188176814 >> v1:172.21.15.119:6789/0 conn(0x7fe1a40b4e80 legacy=0x7fe1a406d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.582+0000 7fe1a9400640 1 -- 172.21.15.119:0/4188176814 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a40cda00 con 0x7fe1a4144470 2024-04-25T10:09:50.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.583+0000 7fe1a9400640 1 --2- 172.21.15.119:0/4188176814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a4144470 0x7fe1a415fc90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe1a406f500 tx=0x7fe19802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef044bb84a011424 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.583+0000 7fe1a3e00640 1 -- 172.21.15.119:0/4188176814 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198037070 con 0x7fe1a4144470 2024-04-25T10:09:50.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.583+0000 7fe1a3e00640 1 -- 172.21.15.119:0/4188176814 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe198032440 con 0x7fe1a4144470 2024-04-25T10:09:50.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.584+0000 7fe1a3e00640 1 -- 172.21.15.119:0/4188176814 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198032810 con 0x7fe1a4144470 2024-04-25T10:09:50.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.584+0000 7fe1aa800640 1 -- 172.21.15.119:0/4188176814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a4144470 msgr2=0x7fe1a415fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.584+0000 7fe1aa800640 1 --2- 172.21.15.119:0/4188176814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a4144470 0x7fe1a415fc90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe1a406f500 tx=0x7fe19802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.584+0000 7fe1aa800640 1 -- 172.21.15.119:0/4188176814 shutdown_connections 2024-04-25T10:09:50.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.584+0000 7fe1aa800640 1 --2- 172.21.15.119:0/4188176814 >> v2:172.21.15.132:3300/0 conn(0x7fe1a406e430 0x7fe1a40b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.584+0000 7fe1aa800640 1 --2- 172.21.15.119:0/4188176814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a4144470 0x7fe1a415fc90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.584+0000 7fe1aa800640 1 -- 172.21.15.119:0/4188176814 >> 172.21.15.119:0/4188176814 conn(0x7fe1a406b1d0 msgr2=0x7fe1a40b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.584+0000 7fe1aa800640 1 -- 172.21.15.119:0/4188176814 shutdown_connections 2024-04-25T10:09:50.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.584+0000 7fe1aa800640 1 -- 172.21.15.119:0/4188176814 wait complete. 2024-04-25T10:09:50.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.586+0000 7fe1aa800640 1 Processor -- start 2024-04-25T10:09:50.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.586+0000 7fe1aa800640 1 -- start start 2024-04-25T10:09:50.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.587+0000 7fe1aa800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1a406e430 0x7fe1a4137500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.587+0000 7fe1aa800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a40b4e80 0x7fe1a40af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.587+0000 7fe1aa800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1a4144470 0x7fe1a413d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.587+0000 7fe1aa800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a40ce200 con 0x7fe1a40b4e80 2024-04-25T10:09:50.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.587+0000 7fe1aa800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a4144850 con 0x7fe1a4144470 2024-04-25T10:09:50.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.587+0000 7fe1aa800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a415cd00 con 0x7fe1a406e430 2024-04-25T10:09:50.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.588+0000 7fe1a8a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a40b4e80 0x7fe1a40af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.588+0000 7fe1a8a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a40b4e80 0x7fe1a40af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39462/0 (socket says 172.21.15.119:39462) 2024-04-25T10:09:50.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.588+0000 7fe1a8a00640 1 -- 172.21.15.119:0/3523117027 learned_addr learned my addr 172.21.15.119:0/3523117027 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:50.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.588+0000 7fe1a9400640 1 --2- 172.21.15.119:0/3523117027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1a406e430 0x7fe1a4137500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.588+0000 7fe1a9e00640 1 --2- 172.21.15.119:0/3523117027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1a4144470 0x7fe1a413d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.588+0000 7fe1a8a00640 1 -- 172.21.15.119:0/3523117027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1a406e430 msgr2=0x7fe1a4137500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.588+0000 7fe1a8a00640 1 --2- 172.21.15.119:0/3523117027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1a406e430 0x7fe1a4137500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.588+0000 7fe1a8a00640 1 -- 172.21.15.119:0/3523117027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1a4144470 msgr2=0x7fe1a413d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.588+0000 7fe1a8a00640 1 --2- 172.21.15.119:0/3523117027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1a4144470 0x7fe1a413d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.588+0000 7fe1a8a00640 1 -- 172.21.15.119:0/3523117027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a413de60 con 0x7fe1a40b4e80 2024-04-25T10:09:50.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.588+0000 7fe1a8a00640 1 --2- 172.21.15.119:0/3523117027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a40b4e80 0x7fe1a40af090 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe19000d800 tx=0x7fe19000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.589+0000 7fe1a1600640 1 -- 172.21.15.119:0/3523117027 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190004020 con 0x7fe1a40b4e80 2024-04-25T10:09:50.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.589+0000 7fe1a1600640 1 -- 172.21.15.119:0/3523117027 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1900041c0 con 0x7fe1a40b4e80 2024-04-25T10:09:50.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.589+0000 7fe1a1600640 1 -- 172.21.15.119:0/3523117027 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190004390 con 0x7fe1a40b4e80 2024-04-25T10:09:50.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.589+0000 7fe1aa800640 1 -- 172.21.15.119:0/3523117027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a413e140 con 0x7fe1a40b4e80 2024-04-25T10:09:50.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.589+0000 7fe1aa800640 1 -- 172.21.15.119:0/3523117027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a413e700 con 0x7fe1a40b4e80 2024-04-25T10:09:50.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.590+0000 7fe1aa800640 1 -- 172.21.15.119:0/3523117027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a40749b0 con 0x7fe1a40b4e80 2024-04-25T10:09:50.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.591+0000 7fe1a1600640 1 -- 172.21.15.119:0/3523117027 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe19001b020 con 0x7fe1a40b4e80 2024-04-25T10:09:50.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.592+0000 7fe1a1600640 1 --2- 172.21.15.119:0/3523117027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe18406b4d0 0x7fe18406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.592+0000 7fe1a1600640 1 -- 172.21.15.119:0/3523117027 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe190089d10 con 0x7fe1a40b4e80 2024-04-25T10:09:50.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.592+0000 7fe1a9400640 1 --2- 172.21.15.119:0/3523117027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe18406b4d0 0x7fe18406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.593+0000 7fe1a9400640 1 --2- 172.21.15.119:0/3523117027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe18406b4d0 0x7fe18406d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe198034000 tx=0x7fe19802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.596+0000 7fe1a1600640 1 -- 172.21.15.119:0/3523117027 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe19008f020 con 0x7fe1a40b4e80 2024-04-25T10:09:50.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.886+0000 7fe1aa800640 1 -- 172.21.15.119:0/3523117027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fe1a406daa0 con 0x7fe1a40b4e80 2024-04-25T10:09:50.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.924+0000 7fe1a1600640 1 -- 172.21.15.119:0/3523117027 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v165) v1 ==== 149+0+0 (secure 0 0 0) 0x7fe190058e80 con 0x7fe1a40b4e80 2024-04-25T10:09:50.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.927+0000 7fe1aa800640 1 -- 172.21.15.119:0/3523117027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe18406b4d0 msgr2=0x7fe18406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.927+0000 7fe1aa800640 1 --2- 172.21.15.119:0/3523117027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe18406b4d0 0x7fe18406d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe198034000 tx=0x7fe19802e000 comp rx=0 tx=0).stop 2024-04-25T10:09:50.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.927+0000 7fe1aa800640 1 -- 172.21.15.119:0/3523117027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a40b4e80 msgr2=0x7fe1a40af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.927+0000 7fe1aa800640 1 --2- 172.21.15.119:0/3523117027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a40b4e80 0x7fe1a40af090 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe19000d800 tx=0x7fe19000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.927+0000 7fe1aa800640 1 -- 172.21.15.119:0/3523117027 shutdown_connections 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.927+0000 7fe1aa800640 1 --2- 172.21.15.119:0/3523117027 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe18406b4d0 0x7fe18406d990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.927+0000 7fe1aa800640 1 --2- 172.21.15.119:0/3523117027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1a4144470 0x7fe1a413d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.927+0000 7fe1aa800640 1 --2- 172.21.15.119:0/3523117027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a40b4e80 0x7fe1a40af090 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.927+0000 7fe1aa800640 1 --2- 172.21.15.119:0/3523117027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1a406e430 0x7fe1a4137500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.927+0000 7fe1aa800640 1 -- 172.21.15.119:0/3523117027 >> 172.21.15.119:0/3523117027 conn(0x7fe1a406b1d0 msgr2=0x7fe1a40b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.927+0000 7fe1aa800640 1 -- 172.21.15.119:0/3523117027 shutdown_connections 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:50.927+0000 7fe1aa800640 1 -- 172.21.15.119:0/3523117027 wait complete. 2024-04-25T10:09:50.952 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-25T10:09:50.952 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:09:50.952 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:09:50.952 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:51.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.119+0000 7f1ba2600640 1 Processor -- start 2024-04-25T10:09:51.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.120+0000 7f1ba2600640 1 -- start start 2024-04-25T10:09:51.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.121+0000 7f1ba2600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b9c0aecf0 0x7f1b9c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.121+0000 7f1ba2600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9c0712c0 con 0x7f1b9c15ed70 2024-04-25T10:09:51.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.121+0000 7f1ba2600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9c0715a0 con 0x7f1b9c15d980 2024-04-25T10:09:51.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.121+0000 7f1ba2600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c071ed0 con 0x7f1b9c0aecf0 2024-04-25T10:09:51.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.121+0000 7f1b9be00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1b9c15ed70 0x7f1b9c0acf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50094/0 (socket says 172.21.15.119:50094) 2024-04-25T10:09:51.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.121+0000 7f1b9be00640 1 -- 172.21.15.119:0/3117975490 learned_addr learned my addr 172.21.15.119:0/3117975490 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.122+0000 7f1b92a00640 1 --2- 172.21.15.119:0/3117975490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b9c0aecf0 0x7f1b9c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.122+0000 7f1b9aa00640 1 -- 172.21.15.119:0/3117975490 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727600095 0 0) 0x7f1b9c0712c0 con 0x7f1b9c15ed70 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.122+0000 7f1b9aa00640 1 -- 172.21.15.119:0/3117975490 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b80003520 con 0x7f1b9c15ed70 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.122+0000 7f1b9aa00640 1 -- 172.21.15.119:0/3117975490 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1005940636 0 0) 0x7f1b80003520 con 0x7f1b9c15ed70 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.122+0000 7f1b9aa00640 1 -- 172.21.15.119:0/3117975490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b9c0aecf0 msgr2=0x7f1b9c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.122+0000 7f1b9aa00640 1 --2- 172.21.15.119:0/3117975490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b9c0aecf0 0x7f1b9c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.122+0000 7f1b9aa00640 1 -- 172.21.15.119:0/3117975490 >> v1:172.21.15.132:6789/0 conn(0x7f1b9c15d980 legacy=0x7f1b9c0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.122+0000 7f1b9aa00640 1 -- 172.21.15.119:0/3117975490 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c071880 con 0x7f1b9c15ed70 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.122+0000 7f1b9aa00640 1 -- 172.21.15.119:0/3117975490 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f1b94001670 con 0x7f1b9c15ed70 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.122+0000 7f1b92a00640 1 --2- 172.21.15.119:0/3117975490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b9c0aecf0 0x7f1b9c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.122+0000 7f1b9aa00640 1 -- 172.21.15.119:0/3117975490 >> v1:172.21.15.119:6789/0 conn(0x7f1b9c15ed70 legacy=0x7f1b9c0acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.122+0000 7f1b9aa00640 1 --2- 172.21.15.119:0/3117975490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b800039e0 0x7f1b80003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.122+0000 7f1b9aa00640 1 --2- 172.21.15.119:0/3117975490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b80004800 0x7f1b80004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.123+0000 7f1b9aa00640 1 --2- 172.21.15.119:0/3117975490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b80005140 0x7f1b8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.123+0000 7f1b9aa00640 1 -- 172.21.15.119:0/3117975490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c071ed0 con 0x7f1b800039e0 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.123+0000 7f1b9aa00640 1 -- 172.21.15.119:0/3117975490 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8000bfb0 con 0x7f1b80005140 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.123+0000 7f1b9aa00640 1 -- 172.21.15.119:0/3117975490 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8000c1b0 con 0x7f1b80004800 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.123+0000 7f1b9b400640 1 --2- 172.21.15.119:0/3117975490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b800039e0 0x7f1b80003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.123+0000 7f1b92a00640 1 --2- 172.21.15.119:0/3117975490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b80004800 0x7f1b80004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.123+0000 7f1b9be00640 1 --2- 172.21.15.119:0/3117975490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b80005140 0x7f1b8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.123+0000 7f1b9b400640 1 -- 172.21.15.119:0/3117975490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b80004800 msgr2=0x7f1b80004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.123+0000 7f1b9b400640 1 --2- 172.21.15.119:0/3117975490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b80004800 0x7f1b80004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.123+0000 7f1b9b400640 1 -- 172.21.15.119:0/3117975490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b80005140 msgr2=0x7f1b8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.123+0000 7f1b9b400640 1 --2- 172.21.15.119:0/3117975490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b80005140 0x7f1b8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.123+0000 7f1b9b400640 1 -- 172.21.15.119:0/3117975490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80003520 con 0x7f1b800039e0 2024-04-25T10:09:51.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.124+0000 7f1b9b400640 1 --2- 172.21.15.119:0/3117975490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b800039e0 0x7f1b80003e00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c001e50 tx=0x7f1b80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e5fe9ea3cac868 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.124+0000 7f1b9aa00640 1 -- 172.21.15.119:0/3117975490 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c001a70 con 0x7f1b800039e0 2024-04-25T10:09:51.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.125+0000 7f1b9aa00640 1 -- 172.21.15.119:0/3117975490 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c003970 con 0x7f1b800039e0 2024-04-25T10:09:51.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.125+0000 7f1b9aa00640 1 -- 172.21.15.119:0/3117975490 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c003d60 con 0x7f1b800039e0 2024-04-25T10:09:51.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.125+0000 7f1ba2600640 1 -- 172.21.15.119:0/3117975490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b800039e0 msgr2=0x7f1b80003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.125+0000 7f1ba2600640 1 --2- 172.21.15.119:0/3117975490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b800039e0 0x7f1b80003e00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c001e50 tx=0x7f1b80005830 comp rx=0 tx=0).stop 2024-04-25T10:09:51.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.125+0000 7f1b9be00640 1 -- 172.21.15.119:0/3117975490 reap_dead start 2024-04-25T10:09:51.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.125+0000 7f1ba2600640 1 -- 172.21.15.119:0/3117975490 shutdown_connections 2024-04-25T10:09:51.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.125+0000 7f1ba2600640 1 -- 172.21.15.119:0/3117975490 >> 172.21.15.119:0/3117975490 conn(0x7f1b9c06aba0 msgr2=0x7f1b9c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.125+0000 7f1ba2600640 1 -- 172.21.15.119:0/3117975490 shutdown_connections 2024-04-25T10:09:51.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.125+0000 7f1ba2600640 1 -- 172.21.15.119:0/3117975490 wait complete. 2024-04-25T10:09:51.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.127+0000 7f1ba2600640 1 Processor -- start 2024-04-25T10:09:51.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.127+0000 7f1ba2600640 1 -- start start 2024-04-25T10:09:51.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.128+0000 7f1ba2600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b800039e0 0x7f1b9c15b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.128+0000 7f1ba2600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b9c143090 0x7f1b9c1f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.128+0000 7f1ba2600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b9c155950 0x7f1b9c155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.128+0000 7f1ba2600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0cf400 con 0x7f1b9c143090 2024-04-25T10:09:51.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.128+0000 7f1ba2600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0cf280 con 0x7f1b800039e0 2024-04-25T10:09:51.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.128+0000 7f1ba2600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0cf580 con 0x7f1b9c155950 2024-04-25T10:09:51.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.128+0000 7f1b92a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b9c143090 0x7f1b9c1f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.128+0000 7f1b92a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b9c143090 0x7f1b9c1f7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42558/0 (socket says 172.21.15.119:42558) 2024-04-25T10:09:51.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.128+0000 7f1b92a00640 1 -- 172.21.15.119:0/2474095458 learned_addr learned my addr 172.21.15.119:0/2474095458 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:51.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.128+0000 7f1b9be00640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b9c155950 0x7f1b9c155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.129+0000 7f1b92a00640 1 -- 172.21.15.119:0/2474095458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b9c155950 msgr2=0x7f1b9c155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.129+0000 7f1b92a00640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b9c155950 0x7f1b9c155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.129+0000 7f1b9b400640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b800039e0 0x7f1b9c15b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.129+0000 7f1b92a00640 1 -- 172.21.15.119:0/2474095458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b800039e0 msgr2=0x7f1b9c15b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.129+0000 7f1b92a00640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b800039e0 0x7f1b9c15b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.129+0000 7f1b92a00640 1 -- 172.21.15.119:0/2474095458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c1f7fb0 con 0x7f1b9c143090 2024-04-25T10:09:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.129+0000 7f1b9b400640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b800039e0 0x7f1b9c15b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.129+0000 7f1b92a00640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b9c143090 0x7f1b9c1f7a70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1b84010890 tx=0x7f1b84010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.129+0000 7f1b93e00640 1 -- 172.21.15.119:0/2474095458 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8401a070 con 0x7f1b9c143090 2024-04-25T10:09:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.129+0000 7f1b93e00640 1 -- 172.21.15.119:0/2474095458 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8400cae0 con 0x7f1b9c143090 2024-04-25T10:09:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.129+0000 7f1b93e00640 1 -- 172.21.15.119:0/2474095458 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8400f040 con 0x7f1b9c143090 2024-04-25T10:09:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.129+0000 7f1ba2600640 1 -- 172.21.15.119:0/2474095458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9c15a650 con 0x7f1b9c143090 2024-04-25T10:09:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.129+0000 7f1ba2600640 1 -- 172.21.15.119:0/2474095458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9c15ac10 con 0x7f1b9c143090 2024-04-25T10:09:51.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.130+0000 7f1ba2600640 1 -- 172.21.15.119:0/2474095458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9c0749b0 con 0x7f1b9c143090 2024-04-25T10:09:51.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.131+0000 7f1b93e00640 1 -- 172.21.15.119:0/2474095458 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b8401d070 con 0x7f1b9c143090 2024-04-25T10:09:51.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.132+0000 7f1b93e00640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b7806b500 0x7f1b7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.132+0000 7f1b93e00640 1 -- 172.21.15.119:0/2474095458 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1b8408a270 con 0x7f1b9c143090 2024-04-25T10:09:51.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.132+0000 7f1b9b400640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b7806b500 0x7f1b7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.133+0000 7f1b9b400640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b7806b500 0x7f1b7806d9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c032000 tx=0x7f1b8c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.136+0000 7f1b93e00640 1 -- 172.21.15.119:0/2474095458 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b8400ee60 con 0x7f1b9c143090 2024-04-25T10:09:51.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.336+0000 7f1ba2600640 1 -- 172.21.15.119:0/2474095458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b9c06d4c0 con 0x7f1b9c143090 2024-04-25T10:09:51.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.338+0000 7f1b93e00640 1 -- 172.21.15.119:0/2474095458 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+13056 (secure 0 0 0) 0x7f1b84088e20 con 0x7f1b9c143090 2024-04-25T10:09:51.345 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:51.345 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":47,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:09:49.904932+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/270766821":"2024-04-26T10:09:46.623341+0000","172.21.15.119:6835/270766821":"2024-04-26T10:09:46.623341+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:51.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.341+0000 7f1ba2600640 1 -- 172.21.15.119:0/2474095458 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b7806b500 msgr2=0x7f1b7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.341+0000 7f1ba2600640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b7806b500 0x7f1b7806d9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c032000 tx=0x7f1b8c02e000 comp rx=0 tx=0).stop 2024-04-25T10:09:51.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.341+0000 7f1ba2600640 1 -- 172.21.15.119:0/2474095458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b9c143090 msgr2=0x7f1b9c1f7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.341+0000 7f1ba2600640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b9c143090 0x7f1b9c1f7a70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1b84010890 tx=0x7f1b84010d50 comp rx=0 tx=0).stop 2024-04-25T10:09:51.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.341+0000 7f1ba2600640 1 -- 172.21.15.119:0/2474095458 shutdown_connections 2024-04-25T10:09:51.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.341+0000 7f1ba2600640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b7806b500 0x7f1b7806d9c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.341+0000 7f1ba2600640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b9c155950 0x7f1b9c155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.341+0000 7f1ba2600640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b9c143090 0x7f1b9c1f7a70 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.341+0000 7f1ba2600640 1 --2- 172.21.15.119:0/2474095458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b800039e0 0x7f1b9c15b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.341+0000 7f1ba2600640 1 -- 172.21.15.119:0/2474095458 >> 172.21.15.119:0/2474095458 conn(0x7f1b9c06aba0 msgr2=0x7f1b9c0b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.341+0000 7f1ba2600640 1 -- 172.21.15.119:0/2474095458 shutdown_connections 2024-04-25T10:09:51.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.341+0000 7f1ba2600640 1 -- 172.21.15.119:0/2474095458 wait complete. 2024-04-25T10:09:51.366 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:51.366 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:51.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.560+0000 7f235ac00640 1 Processor -- start 2024-04-25T10:09:51.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.560+0000 7f235ac00640 1 -- start start 2024-04-25T10:09:51.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.561+0000 7f235ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2354166c40 0x7f2354162f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.561+0000 7f235ac00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2354167090 con 0x7f23540cfce0 2024-04-25T10:09:51.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.561+0000 7f235ac00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2354071350 con 0x7f23540cf900 2024-04-25T10:09:51.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.561+0000 7f235ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2354071630 con 0x7f2354166c40 2024-04-25T10:09:51.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.561+0000 7f2353400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f23540cfce0 0x7f2354162880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50110/0 (socket says 172.21.15.119:50110) 2024-04-25T10:09:51.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.561+0000 7f2353400640 1 -- 172.21.15.119:0/3192562238 learned_addr learned my addr 172.21.15.119:0/3192562238 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:51.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2358c00640 1 --2- 172.21.15.119:0/3192562238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2354166c40 0x7f2354162f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 -- 172.21.15.119:0/3192562238 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159499378 0 0) 0x7f2354167090 con 0x7f23540cfce0 2024-04-25T10:09:51.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 -- 172.21.15.119:0/3192562238 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2338003520 con 0x7f23540cfce0 2024-04-25T10:09:51.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 -- 172.21.15.119:0/3192562238 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1820584429 0 0) 0x7f2338003520 con 0x7f23540cfce0 2024-04-25T10:09:51.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 -- 172.21.15.119:0/3192562238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2354166c40 msgr2=0x7f2354162f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 --2- 172.21.15.119:0/3192562238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2354166c40 0x7f2354162f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 -- 172.21.15.119:0/3192562238 >> v1:172.21.15.132:6789/0 conn(0x7f23540cf900 legacy=0x7f23540b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 -- 172.21.15.119:0/3192562238 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2354071910 con 0x7f23540cfce0 2024-04-25T10:09:51.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 -- 172.21.15.119:0/3192562238 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f2340001690 con 0x7f23540cfce0 2024-04-25T10:09:51.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 -- 172.21.15.119:0/3192562238 >> v1:172.21.15.119:6789/0 conn(0x7f23540cfce0 legacy=0x7f2354162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 --2- 172.21.15.119:0/3192562238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23380039e0 0x7f2338003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 --2- 172.21.15.119:0/3192562238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2354166c40 0x7f2338004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 --2- 172.21.15.119:0/3192562238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2338004d40 0x7f23380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 -- 172.21.15.119:0/3192562238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2354071630 con 0x7f23380039e0 2024-04-25T10:09:51.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 -- 172.21.15.119:0/3192562238 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233800baf0 con 0x7f2338004d40 2024-04-25T10:09:51.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.562+0000 7f2352a00640 1 -- 172.21.15.119:0/3192562238 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233800bcf0 con 0x7f2354166c40 2024-04-25T10:09:51.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.563+0000 7f2353e00640 1 --2- 172.21.15.119:0/3192562238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23380039e0 0x7f2338003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.563+0000 7f2358c00640 1 --2- 172.21.15.119:0/3192562238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2338004d40 0x7f23380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.563+0000 7f2353e00640 1 -- 172.21.15.119:0/3192562238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2354166c40 msgr2=0x7f2338004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.563+0000 7f2353400640 1 --2- 172.21.15.119:0/3192562238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2354166c40 0x7f2338004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.563+0000 7f2353e00640 1 --2- 172.21.15.119:0/3192562238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2354166c40 0x7f2338004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.563+0000 7f2353e00640 1 -- 172.21.15.119:0/3192562238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2338004d40 msgr2=0x7f23380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.563+0000 7f2353e00640 1 --2- 172.21.15.119:0/3192562238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2338004d40 0x7f23380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.563+0000 7f2353e00640 1 -- 172.21.15.119:0/3192562238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2338003520 con 0x7f23380039e0 2024-04-25T10:09:51.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.563+0000 7f2353400640 1 --2- 172.21.15.119:0/3192562238 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2354166c40 0x7f2338004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:51.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.563+0000 7f2358c00640 1 --2- 172.21.15.119:0/3192562238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2338004d40 0x7f23380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:51.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.564+0000 7f2353e00640 1 --2- 172.21.15.119:0/3192562238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23380039e0 0x7f2338003e00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2344001750 tx=0x7f2338005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234fc1a7052716cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.564+0000 7f2352a00640 1 -- 172.21.15.119:0/3192562238 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2344001370 con 0x7f23380039e0 2024-04-25T10:09:51.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.564+0000 7f2352a00640 1 -- 172.21.15.119:0/3192562238 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23440052d0 con 0x7f23380039e0 2024-04-25T10:09:51.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.564+0000 7f2352a00640 1 -- 172.21.15.119:0/3192562238 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234402ab20 con 0x7f23380039e0 2024-04-25T10:09:51.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.565+0000 7f235ac00640 1 -- 172.21.15.119:0/3192562238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23380039e0 msgr2=0x7f2338003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.565+0000 7f235ac00640 1 --2- 172.21.15.119:0/3192562238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23380039e0 0x7f2338003e00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2344001750 tx=0x7f2338005830 comp rx=0 tx=0).stop 2024-04-25T10:09:51.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.565+0000 7f2358c00640 1 -- 172.21.15.119:0/3192562238 reap_dead start 2024-04-25T10:09:51.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.565+0000 7f235ac00640 1 -- 172.21.15.119:0/3192562238 shutdown_connections 2024-04-25T10:09:51.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.565+0000 7f235ac00640 1 -- 172.21.15.119:0/3192562238 >> 172.21.15.119:0/3192562238 conn(0x7f235406aba0 msgr2=0x7f235406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.565+0000 7f235ac00640 1 -- 172.21.15.119:0/3192562238 shutdown_connections 2024-04-25T10:09:51.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.565+0000 7f235ac00640 1 -- 172.21.15.119:0/3192562238 wait complete. 2024-04-25T10:09:51.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.567+0000 7f235ac00640 1 Processor -- start 2024-04-25T10:09:51.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.567+0000 7f235ac00640 1 -- start start 2024-04-25T10:09:51.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.569+0000 7f235ac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23380039e0 0x7f235415a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.569+0000 7f235ac00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2354157160 0x7f23541558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.569+0000 7f235ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23541651d0 0x7f23540b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.569+0000 7f235ac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23540b2d60 con 0x7f23380039e0 2024-04-25T10:09:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.569+0000 7f235ac00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23540b3e00 con 0x7f2354157160 2024-04-25T10:09:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.569+0000 7f235ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23540b2ee0 con 0x7f23541651d0 2024-04-25T10:09:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.569+0000 7f2353e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23380039e0 0x7f235415a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.569+0000 7f2353e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23380039e0 0x7f235415a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42580/0 (socket says 172.21.15.119:42580) 2024-04-25T10:09:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.569+0000 7f2353e00640 1 -- 172.21.15.119:0/2491140755 learned_addr learned my addr 172.21.15.119:0/2491140755 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.569+0000 7f2353e00640 1 -- 172.21.15.119:0/2491140755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23541651d0 msgr2=0x7f23540b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.569+0000 7f2353e00640 1 --2- 172.21.15.119:0/2491140755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23541651d0 0x7f23540b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.569+0000 7f2353e00640 1 -- 172.21.15.119:0/2491140755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2354157160 msgr2=0x7f23541558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.569+0000 7f2353e00640 1 --2- 172.21.15.119:0/2491140755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2354157160 0x7f23541558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.569+0000 7f2353e00640 1 -- 172.21.15.119:0/2491140755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235415ac20 con 0x7f23380039e0 2024-04-25T10:09:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.570+0000 7f2353e00640 1 --2- 172.21.15.119:0/2491140755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23380039e0 0x7f235415a6e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f23440017b0 tx=0x7f2344001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.570+0000 7f234be00640 1 -- 172.21.15.119:0/2491140755 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2344037040 con 0x7f23380039e0 2024-04-25T10:09:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.570+0000 7f234be00640 1 -- 172.21.15.119:0/2491140755 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2344005480 con 0x7f23380039e0 2024-04-25T10:09:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.570+0000 7f234be00640 1 -- 172.21.15.119:0/2491140755 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23440073f0 con 0x7f23380039e0 2024-04-25T10:09:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.570+0000 7f235ac00640 1 -- 172.21.15.119:0/2491140755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2354155df0 con 0x7f23380039e0 2024-04-25T10:09:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.570+0000 7f235ac00640 1 -- 172.21.15.119:0/2491140755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235415b1b0 con 0x7f23380039e0 2024-04-25T10:09:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.571+0000 7f235ac00640 1 -- 172.21.15.119:0/2491140755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23540750b0 con 0x7f23380039e0 2024-04-25T10:09:51.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.572+0000 7f234be00640 1 -- 172.21.15.119:0/2491140755 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2344044020 con 0x7f23380039e0 2024-04-25T10:09:51.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.573+0000 7f234be00640 1 --2- 172.21.15.119:0/2491140755 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f233406b500 0x7f233406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.573+0000 7f234be00640 1 -- 172.21.15.119:0/2491140755 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f23440a9b10 con 0x7f23380039e0 2024-04-25T10:09:51.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.573+0000 7f2353400640 1 --2- 172.21.15.119:0/2491140755 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f233406b500 0x7f233406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.574+0000 7f2353400640 1 --2- 172.21.15.119:0/2491140755 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f233406b500 0x7f233406d9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f235415fff0 tx=0x7f2340009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.577+0000 7f234be00640 1 -- 172.21.15.119:0/2491140755 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2344078bf0 con 0x7f23380039e0 2024-04-25T10:09:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.772+0000 7f235ac00640 1 -- 172.21.15.119:0/2491140755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23540b2a60 con 0x7f23380039e0 2024-04-25T10:09:51.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.773+0000 7f234be00640 1 -- 172.21.15.119:0/2491140755 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23440784f0 con 0x7f23380039e0 2024-04-25T10:09:51.779 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:09:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.775+0000 7f235ac00640 1 -- 172.21.15.119:0/2491140755 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f233406b500 msgr2=0x7f233406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.776+0000 7f235ac00640 1 --2- 172.21.15.119:0/2491140755 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f233406b500 0x7f233406d9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f235415fff0 tx=0x7f2340009040 comp rx=0 tx=0).stop 2024-04-25T10:09:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.776+0000 7f235ac00640 1 -- 172.21.15.119:0/2491140755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23380039e0 msgr2=0x7f235415a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.776+0000 7f235ac00640 1 --2- 172.21.15.119:0/2491140755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23380039e0 0x7f235415a6e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f23440017b0 tx=0x7f2344001120 comp rx=0 tx=0).stop 2024-04-25T10:09:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.776+0000 7f235ac00640 1 -- 172.21.15.119:0/2491140755 shutdown_connections 2024-04-25T10:09:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.776+0000 7f235ac00640 1 --2- 172.21.15.119:0/2491140755 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f233406b500 0x7f233406d9c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.776+0000 7f235ac00640 1 --2- 172.21.15.119:0/2491140755 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23541651d0 0x7f23540b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.776+0000 7f235ac00640 1 --2- 172.21.15.119:0/2491140755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2354157160 0x7f23541558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.776+0000 7f235ac00640 1 --2- 172.21.15.119:0/2491140755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23380039e0 0x7f235415a6e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.776+0000 7f235ac00640 1 -- 172.21.15.119:0/2491140755 >> 172.21.15.119:0/2491140755 conn(0x7f235406aba0 msgr2=0x7f2354156850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.776+0000 7f235ac00640 1 -- 172.21.15.119:0/2491140755 shutdown_connections 2024-04-25T10:09:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.776+0000 7f235ac00640 1 -- 172.21.15.119:0/2491140755 wait complete. 2024-04-25T10:09:51.801 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T10:09:52.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.997+0000 7fbf70800640 1 Processor -- start 2024-04-25T10:09:52.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.998+0000 7fbf70800640 1 -- start start 2024-04-25T10:09:52.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.999+0000 7fbf70800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf6815dce0 0x7fbf680d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.999+0000 7fbf70800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6815e0c0 con 0x7fbf680b5b60 2024-04-25T10:09:52.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.999+0000 7fbf70800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf680d2510 con 0x7fbf680d35b0 2024-04-25T10:09:52.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.999+0000 7fbf70800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf680d27f0 con 0x7fbf6815dce0 2024-04-25T10:09:52.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.999+0000 7fbf6e800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbf680b5b60 0x7fbf680b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50124/0 (socket says 172.21.15.119:50124) 2024-04-25T10:09:52.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.999+0000 7fbf6e800640 1 -- 172.21.15.119:0/1311338488 learned_addr learned my addr 172.21.15.119:0/1311338488 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:52.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:51.999+0000 7fbf6de00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf6815dce0 0x7fbf680d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.000+0000 7fbf6ca00640 1 -- 172.21.15.119:0/1311338488 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309755095 0 0) 0x7fbf6815e0c0 con 0x7fbf680b5b60 2024-04-25T10:09:52.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.000+0000 7fbf6ca00640 1 -- 172.21.15.119:0/1311338488 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf54003520 con 0x7fbf680b5b60 2024-04-25T10:09:52.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.000+0000 7fbf6de00640 1 -- 172.21.15.119:0/1311338488 >> v1:172.21.15.132:6789/0 conn(0x7fbf680d35b0 legacy=0x7fbf680b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.000+0000 7fbf6de00640 1 -- 172.21.15.119:0/1311338488 >> v1:172.21.15.119:6789/0 conn(0x7fbf680b5b60 legacy=0x7fbf680b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.000+0000 7fbf6de00640 1 -- 172.21.15.119:0/1311338488 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf680d2ad0 con 0x7fbf6815dce0 2024-04-25T10:09:52.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.001+0000 7fbf6de00640 1 --2- 172.21.15.119:0/1311338488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf6815dce0 0x7fbf680d1a50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009d90 tx=0x7fbf6402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cd4663385d5668 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.001+0000 7fbf6ca00640 1 -- 172.21.15.119:0/1311338488 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf64007740 con 0x7fbf6815dce0 2024-04-25T10:09:52.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.001+0000 7fbf6ca00640 1 -- 172.21.15.119:0/1311338488 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf640078e0 con 0x7fbf6815dce0 2024-04-25T10:09:52.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.001+0000 7fbf6ca00640 1 -- 172.21.15.119:0/1311338488 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf64007cb0 con 0x7fbf6815dce0 2024-04-25T10:09:52.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.002+0000 7fbf70800640 1 -- 172.21.15.119:0/1311338488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf6815dce0 msgr2=0x7fbf680d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.002+0000 7fbf70800640 1 --2- 172.21.15.119:0/1311338488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf6815dce0 0x7fbf680d1a50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009d90 tx=0x7fbf6402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.002+0000 7fbf70800640 1 -- 172.21.15.119:0/1311338488 shutdown_connections 2024-04-25T10:09:52.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.002+0000 7fbf70800640 1 --2- 172.21.15.119:0/1311338488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf6815dce0 0x7fbf680d1a50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.002+0000 7fbf70800640 1 -- 172.21.15.119:0/1311338488 >> 172.21.15.119:0/1311338488 conn(0x7fbf68072900 msgr2=0x7fbf680d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.002+0000 7fbf70800640 1 -- 172.21.15.119:0/1311338488 shutdown_connections 2024-04-25T10:09:52.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.002+0000 7fbf70800640 1 -- 172.21.15.119:0/1311338488 wait complete. 2024-04-25T10:09:52.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.004+0000 7fbf70800640 1 Processor -- start 2024-04-25T10:09:52.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.004+0000 7fbf70800640 1 -- start start 2024-04-25T10:09:52.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.005+0000 7fbf70800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf680b5b60 0x7fbf680ce460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.005+0000 7fbf70800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf680d35b0 0x7fbf68157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.005+0000 7fbf70800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf6815dce0 0x7fbf68157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.005+0000 7fbf70800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680b9820 con 0x7fbf6815dce0 2024-04-25T10:09:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.005+0000 7fbf70800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680d3090 con 0x7fbf680d35b0 2024-04-25T10:09:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.005+0000 7fbf70800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680b99a0 con 0x7fbf680b5b60 2024-04-25T10:09:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.005+0000 7fbf6e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf6815dce0 0x7fbf68157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.005+0000 7fbf6e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf6815dce0 0x7fbf68157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42586/0 (socket says 172.21.15.119:42586) 2024-04-25T10:09:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.005+0000 7fbf6e800640 1 -- 172.21.15.119:0/789001810 learned_addr learned my addr 172.21.15.119:0/789001810 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.006+0000 7fbf6d400640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf680d35b0 0x7fbf68157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.006+0000 7fbf6e800640 1 -- 172.21.15.119:0/789001810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf680b5b60 msgr2=0x7fbf680ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.006+0000 7fbf6de00640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf680b5b60 0x7fbf680ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.006+0000 7fbf6e800640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf680b5b60 0x7fbf680ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.006+0000 7fbf6e800640 1 -- 172.21.15.119:0/789001810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf680d35b0 msgr2=0x7fbf68157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.006+0000 7fbf6e800640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf680d35b0 0x7fbf68157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.006+0000 7fbf6e800640 1 -- 172.21.15.119:0/789001810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf68157db0 con 0x7fbf6815dce0 2024-04-25T10:09:52.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.006+0000 7fbf6d400640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf680d35b0 0x7fbf68157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:52.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.006+0000 7fbf6e800640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf6815dce0 0x7fbf68157870 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000c430 tx=0x7fbf6000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.006+0000 7fbf5e000640 1 -- 172.21.15.119:0/789001810 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60017070 con 0x7fbf6815dce0 2024-04-25T10:09:52.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.006+0000 7fbf5e000640 1 -- 172.21.15.119:0/789001810 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf60005ce0 con 0x7fbf6815dce0 2024-04-25T10:09:52.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.006+0000 7fbf70800640 1 -- 172.21.15.119:0/789001810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf68146d20 con 0x7fbf6815dce0 2024-04-25T10:09:52.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.006+0000 7fbf5e000640 1 -- 172.21.15.119:0/789001810 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60007310 con 0x7fbf6815dce0 2024-04-25T10:09:52.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.006+0000 7fbf70800640 1 -- 172.21.15.119:0/789001810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf681f6f70 con 0x7fbf6815dce0 2024-04-25T10:09:52.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.008+0000 7fbf70800640 1 -- 172.21.15.119:0/789001810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf681f6aa0 con 0x7fbf6815dce0 2024-04-25T10:09:52.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.008+0000 7fbf5e000640 1 -- 172.21.15.119:0/789001810 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf6001a070 con 0x7fbf6815dce0 2024-04-25T10:09:52.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.009+0000 7fbf5e000640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.009+0000 7fbf5e000640 1 -- 172.21.15.119:0/789001810 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbf60089420 con 0x7fbf6815dce0 2024-04-25T10:09:52.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.010+0000 7fbf6de00640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.010+0000 7fbf6de00640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf64034000 tx=0x7fbf64033040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.014+0000 7fbf5e000640 1 -- 172.21.15.119:0/789001810 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf60058500 con 0x7fbf6815dce0 2024-04-25T10:09:52.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.280+0000 7fbf70800640 1 -- 172.21.15.119:0/789001810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fbf680d10d0 con 0x7fbf6815dce0 2024-04-25T10:09:52.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.937+0000 7fbf5e000640 1 -- 172.21.15.119:0/789001810 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v167) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbf60057e00 con 0x7fbf6815dce0 2024-04-25T10:09:52.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.941+0000 7fbf70800640 1 -- 172.21.15.119:0/789001810 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbf4c06b4d0 msgr2=0x7fbf4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.941+0000 7fbf70800640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf64034000 tx=0x7fbf64033040 comp rx=0 tx=0).stop 2024-04-25T10:09:52.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.941+0000 7fbf70800640 1 -- 172.21.15.119:0/789001810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf6815dce0 msgr2=0x7fbf68157870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.941+0000 7fbf70800640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf6815dce0 0x7fbf68157870 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000c430 tx=0x7fbf6000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.941+0000 7fbf70800640 1 -- 172.21.15.119:0/789001810 shutdown_connections 2024-04-25T10:09:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.941+0000 7fbf70800640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.941+0000 7fbf70800640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf6815dce0 0x7fbf68157870 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.941+0000 7fbf70800640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf680d35b0 0x7fbf68157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.941+0000 7fbf70800640 1 --2- 172.21.15.119:0/789001810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbf680b5b60 0x7fbf680ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.941+0000 7fbf70800640 1 -- 172.21.15.119:0/789001810 >> 172.21.15.119:0/789001810 conn(0x7fbf68072900 msgr2=0x7fbf680b7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.942+0000 7fbf70800640 1 -- 172.21.15.119:0/789001810 shutdown_connections 2024-04-25T10:09:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:52.942+0000 7fbf70800640 1 -- 172.21.15.119:0/789001810 wait complete. 2024-04-25T10:09:52.969 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:09:53.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.167+0000 7f4863e00640 1 Processor -- start 2024-04-25T10:09:53.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.167+0000 7f4863e00640 1 -- start start 2024-04-25T10:09:53.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.169+0000 7f4863e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f48640ada20 0x7f48640acb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.169+0000 7f4863e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486415e120 con 0x7f486406aba0 2024-04-25T10:09:53.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.169+0000 7f4863e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486415e400 con 0x7f48640b36a0 2024-04-25T10:09:53.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.169+0000 7f4863e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f486415dec0 con 0x7f48640ada20 2024-04-25T10:09:53.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.169+0000 7f4862a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f48640ada20 0x7f48640acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.169+0000 7f4862000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f486406aba0 0x7f48640b2f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50126/0 (socket says 172.21.15.119:50126) 2024-04-25T10:09:53.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.169+0000 7f4862000640 1 -- 172.21.15.119:0/830312314 learned_addr learned my addr 172.21.15.119:0/830312314 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:53.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.170+0000 7f4862a00640 1 -- 172.21.15.119:0/830312314 >> v1:172.21.15.132:6789/0 conn(0x7f48640b36a0 legacy=0x7f4864157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:53.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.170+0000 7f4862a00640 1 -- 172.21.15.119:0/830312314 >> v1:172.21.15.119:6789/0 conn(0x7f486406aba0 legacy=0x7f48640b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.170+0000 7f4862a00640 1 -- 172.21.15.119:0/830312314 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486415e6e0 con 0x7f48640ada20 2024-04-25T10:09:53.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.171+0000 7f4862a00640 1 --2- 172.21.15.119:0/830312314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48640ada20 0x7f48640acb20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4854009d90 tx=0x7f485402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251990dc4e851cf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.171+0000 7f4861600640 1 -- 172.21.15.119:0/830312314 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854007740 con 0x7f48640ada20 2024-04-25T10:09:53.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.171+0000 7f4861600640 1 -- 172.21.15.119:0/830312314 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48540078e0 con 0x7f48640ada20 2024-04-25T10:09:53.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.171+0000 7f4861600640 1 -- 172.21.15.119:0/830312314 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854007cb0 con 0x7f48640ada20 2024-04-25T10:09:53.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.172+0000 7f4863e00640 1 -- 172.21.15.119:0/830312314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48640ada20 msgr2=0x7f48640acb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.172+0000 7f4863e00640 1 --2- 172.21.15.119:0/830312314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48640ada20 0x7f48640acb20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4854009d90 tx=0x7f485402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.172+0000 7f4863e00640 1 -- 172.21.15.119:0/830312314 shutdown_connections 2024-04-25T10:09:53.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.172+0000 7f4863e00640 1 --2- 172.21.15.119:0/830312314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48640ada20 0x7f48640acb20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.172+0000 7f4863e00640 1 -- 172.21.15.119:0/830312314 >> 172.21.15.119:0/830312314 conn(0x7f4864066ff0 msgr2=0x7f48640b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.172+0000 7f4863e00640 1 -- 172.21.15.119:0/830312314 shutdown_connections 2024-04-25T10:09:53.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.172+0000 7f4863e00640 1 -- 172.21.15.119:0/830312314 wait complete. 2024-04-25T10:09:53.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.174+0000 7f4863e00640 1 Processor -- start 2024-04-25T10:09:53.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.174+0000 7f4863e00640 1 -- start start 2024-04-25T10:09:53.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.175+0000 7f4863e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f486406aba0 0x7f4864154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.175+0000 7f4863e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48640ada20 0x7f4864159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.175+0000 7f4863e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48640b36a0 0x7f486414f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.175+0000 7f4863e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48640ade00 con 0x7f48640ada20 2024-04-25T10:09:53.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.175+0000 7f4863e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486415eca0 con 0x7f486406aba0 2024-04-25T10:09:53.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.175+0000 7f4863e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486406b050 con 0x7f48640b36a0 2024-04-25T10:09:53.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.175+0000 7f4862000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48640ada20 0x7f4864159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.175+0000 7f4862000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48640ada20 0x7f4864159560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42608/0 (socket says 172.21.15.119:42608) 2024-04-25T10:09:53.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.175+0000 7f4862000640 1 -- 172.21.15.119:0/4265805282 learned_addr learned my addr 172.21.15.119:0/4265805282 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:53.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.175+0000 7f4862a00640 1 --2- 172.21.15.119:0/4265805282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f486406aba0 0x7f4864154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.175+0000 7f4862000640 1 -- 172.21.15.119:0/4265805282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48640b36a0 msgr2=0x7f486414f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.175+0000 7f4862000640 1 --2- 172.21.15.119:0/4265805282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48640b36a0 0x7f486414f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.175+0000 7f4862000640 1 -- 172.21.15.119:0/4265805282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f486406aba0 msgr2=0x7f4864154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.175+0000 7f4862000640 1 --2- 172.21.15.119:0/4265805282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f486406aba0 0x7f4864154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.176+0000 7f4862000640 1 -- 172.21.15.119:0/4265805282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4864159aa0 con 0x7f48640ada20 2024-04-25T10:09:53.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.176+0000 7f4862000640 1 --2- 172.21.15.119:0/4265805282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48640ada20 0x7f4864159560 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f484c00c400 tx=0x7f484c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.176+0000 7f485aa00640 1 -- 172.21.15.119:0/4265805282 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c017070 con 0x7f48640ada20 2024-04-25T10:09:53.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.176+0000 7f485aa00640 1 -- 172.21.15.119:0/4265805282 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484c005ce0 con 0x7f48640ada20 2024-04-25T10:09:53.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.176+0000 7f485aa00640 1 -- 172.21.15.119:0/4265805282 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c0072f0 con 0x7f48640ada20 2024-04-25T10:09:53.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.176+0000 7f4863e00640 1 -- 172.21.15.119:0/4265805282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4864154950 con 0x7f48640ada20 2024-04-25T10:09:53.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.176+0000 7f4863e00640 1 -- 172.21.15.119:0/4265805282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4864152fc0 con 0x7f48640ada20 2024-04-25T10:09:53.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.177+0000 7f4863e00640 1 -- 172.21.15.119:0/4265805282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f486406b3e0 con 0x7f48640ada20 2024-04-25T10:09:53.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.178+0000 7f485aa00640 1 -- 172.21.15.119:0/4265805282 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f484c01a070 con 0x7f48640ada20 2024-04-25T10:09:53.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.179+0000 7f485aa00640 1 --2- 172.21.15.119:0/4265805282 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f484006b4d0 0x7f484006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.179+0000 7f485aa00640 1 -- 172.21.15.119:0/4265805282 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f484c0893b0 con 0x7f48640ada20 2024-04-25T10:09:53.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.180+0000 7f4862a00640 1 --2- 172.21.15.119:0/4265805282 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f484006b4d0 0x7f484006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.180+0000 7f4862a00640 1 --2- 172.21.15.119:0/4265805282 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f484006b4d0 0x7f484006d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4854034000 tx=0x7f485402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.184+0000 7f485aa00640 1 -- 172.21.15.119:0/4265805282 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f486406b3e0 con 0x7f48640ada20 2024-04-25T10:09:53.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.381+0000 7f4863e00640 1 -- 172.21.15.119:0/4265805282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f48640749b0 con 0x7f48640ada20 2024-04-25T10:09:53.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.388+0000 7f485aa00640 1 -- 172.21.15.119:0/4265805282 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f484c058490 con 0x7f48640ada20 2024-04-25T10:09:53.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.390+0000 7f4863e00640 1 -- 172.21.15.119:0/4265805282 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f484006b4d0 msgr2=0x7f484006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.390+0000 7f4863e00640 1 --2- 172.21.15.119:0/4265805282 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f484006b4d0 0x7f484006d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4854034000 tx=0x7f485402f000 comp rx=0 tx=0).stop 2024-04-25T10:09:53.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.390+0000 7f4863e00640 1 -- 172.21.15.119:0/4265805282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48640ada20 msgr2=0x7f4864159560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.391+0000 7f4863e00640 1 --2- 172.21.15.119:0/4265805282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48640ada20 0x7f4864159560 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f484c00c400 tx=0x7f484c00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.391+0000 7f4863e00640 1 -- 172.21.15.119:0/4265805282 shutdown_connections 2024-04-25T10:09:53.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.391+0000 7f4863e00640 1 --2- 172.21.15.119:0/4265805282 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f484006b4d0 0x7f484006d990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.391+0000 7f4863e00640 1 --2- 172.21.15.119:0/4265805282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48640b36a0 0x7f486414f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.391+0000 7f4863e00640 1 --2- 172.21.15.119:0/4265805282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48640ada20 0x7f4864159560 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.391+0000 7f4863e00640 1 --2- 172.21.15.119:0/4265805282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f486406aba0 0x7f4864154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.391+0000 7f4863e00640 1 -- 172.21.15.119:0/4265805282 >> 172.21.15.119:0/4265805282 conn(0x7f4864066ff0 msgr2=0x7f48640ae570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.391+0000 7f4863e00640 1 -- 172.21.15.119:0/4265805282 shutdown_connections 2024-04-25T10:09:53.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:53.391+0000 7f4863e00640 1 -- 172.21.15.119:0/4265805282 wait complete. 2024-04-25T10:09:53.416 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:09:53.417 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:09:53.518 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:09:53.633 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:09:53.633 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:09:53.633 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:53.690 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:09:53.690 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:09:53.690 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:53.746 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:09:53.746 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:09:53.747 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:53.803 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:09:53.804 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:09:53.804 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:53.860 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:09:53.860 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:09:53.860 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:54.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.067+0000 7f7abe600640 1 Processor -- start 2024-04-25T10:09:54.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.067+0000 7f7abe600640 1 -- start start 2024-04-25T10:09:54.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.068+0000 7f7abe600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7ab8144470 0x7f7ab815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.068+0000 7f7abe600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab80cd440 con 0x7f7ab80b5050 2024-04-25T10:09:54.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.068+0000 7f7abe600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab80cd720 con 0x7f7ab806e430 2024-04-25T10:09:54.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.068+0000 7f7abe600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab8145340 con 0x7f7ab8144470 2024-04-25T10:09:54.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.069+0000 7f7abdc00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7ab80b5050 0x7f7ab815bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50140/0 (socket says 172.21.15.119:50140) 2024-04-25T10:09:54.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.069+0000 7f7abdc00640 1 -- 172.21.15.119:0/1256884686 learned_addr learned my addr 172.21.15.119:0/1256884686 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:54.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.069+0000 7f7ab7e00640 1 -- 172.21.15.119:0/1256884686 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895987571 0 0) 0x7f7ab80cd440 con 0x7f7ab80b5050 2024-04-25T10:09:54.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.069+0000 7f7ab7e00640 1 -- 172.21.15.119:0/1256884686 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a9c003520 con 0x7f7ab80b5050 2024-04-25T10:09:54.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.069+0000 7f7abd200640 1 --2- 172.21.15.119:0/1256884686 >> v2:172.21.15.132:3300/0 conn(0x7f7ab8144470 0x7f7ab815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.070+0000 7f7ab7e00640 1 -- 172.21.15.119:0/1256884686 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2871558452 0 0) 0x7f7a9c003520 con 0x7f7ab80b5050 2024-04-25T10:09:54.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.070+0000 7f7ab7e00640 1 -- 172.21.15.119:0/1256884686 >> v2:172.21.15.132:3300/0 conn(0x7f7ab8144470 msgr2=0x7f7ab815fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.070+0000 7f7ab7e00640 1 --2- 172.21.15.119:0/1256884686 >> v2:172.21.15.132:3300/0 conn(0x7f7ab8144470 0x7f7ab815fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.070+0000 7f7ab7e00640 1 -- 172.21.15.119:0/1256884686 >> v1:172.21.15.132:6789/0 conn(0x7f7ab806e430 legacy=0x7f7ab80b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.070+0000 7f7ab7e00640 1 -- 172.21.15.119:0/1256884686 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab80cda00 con 0x7f7ab80b5050 2024-04-25T10:09:54.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.070+0000 7f7ab7e00640 1 -- 172.21.15.119:0/1256884686 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f7ab00019f0 con 0x7f7ab80b5050 2024-04-25T10:09:54.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.070+0000 7f7ab7e00640 1 -- 172.21.15.119:0/1256884686 >> v1:172.21.15.119:6789/0 conn(0x7f7ab80b5050 legacy=0x7f7ab815bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.070+0000 7f7ab7e00640 1 --2- 172.21.15.119:0/1256884686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a9c003a70 0x7f7a9c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.070+0000 7f7ab7e00640 1 --2- 172.21.15.119:0/1256884686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a9c004800 0x7f7a9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.070+0000 7f7ab7e00640 1 --2- 172.21.15.119:0/1256884686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a9c0052e0 0x7f7a9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.070+0000 7f7ab7e00640 1 -- 172.21.15.119:0/1256884686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c00c1b0 con 0x7f7a9c0052e0 2024-04-25T10:09:54.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.070+0000 7f7ab7e00640 1 -- 172.21.15.119:0/1256884686 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c00c360 con 0x7f7a9c004800 2024-04-25T10:09:54.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.070+0000 7f7ab7e00640 1 -- 172.21.15.119:0/1256884686 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c00c560 con 0x7f7a9c003a70 2024-04-25T10:09:54.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.070+0000 7f7abdc00640 1 --2- 172.21.15.119:0/1256884686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a9c0052e0 0x7f7a9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.071+0000 7f7abd200640 1 --2- 172.21.15.119:0/1256884686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a9c003a70 0x7f7a9c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.071+0000 7f7aafe00640 1 --2- 172.21.15.119:0/1256884686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a9c004800 0x7f7a9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.071+0000 7f7abdc00640 1 -- 172.21.15.119:0/1256884686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a9c003a70 msgr2=0x7f7a9c003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.071+0000 7f7abdc00640 1 --2- 172.21.15.119:0/1256884686 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a9c003a70 0x7f7a9c003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.071+0000 7f7abdc00640 1 -- 172.21.15.119:0/1256884686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a9c004800 msgr2=0x7f7a9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.071+0000 7f7abdc00640 1 --2- 172.21.15.119:0/1256884686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a9c004800 0x7f7a9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.071+0000 7f7abdc00640 1 -- 172.21.15.119:0/1256884686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c003520 con 0x7f7a9c0052e0 2024-04-25T10:09:54.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.072+0000 7f7abdc00640 1 -- 172.21.15.119:0/1256884686 reap_dead start 2024-04-25T10:09:54.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.072+0000 7f7abdc00640 1 --2- 172.21.15.119:0/1256884686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a9c0052e0 0x7f7a9c00b9c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80b2700 tx=0x7f7a9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd2abb602c6209a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.072+0000 7f7ab7e00640 1 -- 172.21.15.119:0/1256884686 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab000a390 con 0x7f7a9c0052e0 2024-04-25T10:09:54.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.072+0000 7f7ab7e00640 1 -- 172.21.15.119:0/1256884686 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab002f070 con 0x7f7a9c0052e0 2024-04-25T10:09:54.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.072+0000 7f7ab7e00640 1 -- 172.21.15.119:0/1256884686 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab002bb50 con 0x7f7a9c0052e0 2024-04-25T10:09:54.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.073+0000 7f7abe600640 1 -- 172.21.15.119:0/1256884686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a9c0052e0 msgr2=0x7f7a9c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.073+0000 7f7abe600640 1 --2- 172.21.15.119:0/1256884686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a9c0052e0 0x7f7a9c00b9c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80b2700 tx=0x7f7a9c005830 comp rx=0 tx=0).stop 2024-04-25T10:09:54.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.073+0000 7f7abe600640 1 -- 172.21.15.119:0/1256884686 shutdown_connections 2024-04-25T10:09:54.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.073+0000 7f7abe600640 1 --2- 172.21.15.119:0/1256884686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a9c0052e0 0x7f7a9c00b9c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.073+0000 7f7abe600640 1 -- 172.21.15.119:0/1256884686 >> 172.21.15.119:0/1256884686 conn(0x7f7ab806b1d0 msgr2=0x7f7ab80b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:54.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.073+0000 7f7abe600640 1 -- 172.21.15.119:0/1256884686 shutdown_connections 2024-04-25T10:09:54.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.073+0000 7f7abe600640 1 -- 172.21.15.119:0/1256884686 wait complete. 2024-04-25T10:09:54.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.075+0000 7f7abe600640 1 Processor -- start 2024-04-25T10:09:54.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.076+0000 7f7abe600640 1 -- start start 2024-04-25T10:09:54.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7abe600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a9c0052e0 0x7f7ab813d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7abe600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab8153160 0x7f7ab81f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7abe600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab813a090 0x7f7ab815d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7abe600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab80ce200 con 0x7f7ab8153160 2024-04-25T10:09:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7abe600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab8144850 con 0x7f7ab813a090 2024-04-25T10:09:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7abe600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab815fc90 con 0x7f7a9c0052e0 2024-04-25T10:09:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7aafe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab8153160 0x7f7ab81f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7aafe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab8153160 0x7f7ab81f3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42624/0 (socket says 172.21.15.119:42624) 2024-04-25T10:09:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7aafe00640 1 -- 172.21.15.119:0/1322468810 learned_addr learned my addr 172.21.15.119:0/1322468810 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7abd200640 1 --2- 172.21.15.119:0/1322468810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a9c0052e0 0x7f7ab813d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7aafe00640 1 -- 172.21.15.119:0/1322468810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a9c0052e0 msgr2=0x7f7ab813d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7aafe00640 1 --2- 172.21.15.119:0/1322468810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a9c0052e0 0x7f7ab813d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7aafe00640 1 -- 172.21.15.119:0/1322468810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab813a090 msgr2=0x7f7ab815d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7aafe00640 1 --2- 172.21.15.119:0/1322468810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab813a090 0x7f7ab815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.077+0000 7f7aafe00640 1 -- 172.21.15.119:0/1322468810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab81535e0 con 0x7f7ab8153160 2024-04-25T10:09:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.078+0000 7f7aafe00640 1 --2- 172.21.15.119:0/1322468810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab8153160 0x7f7ab81f3900 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000e440 tx=0x7f7aa000e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.078+0000 7f7ab5600640 1 -- 172.21.15.119:0/1322468810 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00052e0 con 0x7f7ab8153160 2024-04-25T10:09:54.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.078+0000 7f7ab5600640 1 -- 172.21.15.119:0/1322468810 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa0018070 con 0x7f7ab8153160 2024-04-25T10:09:54.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.078+0000 7f7ab5600640 1 -- 172.21.15.119:0/1322468810 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0003830 con 0x7f7ab8153160 2024-04-25T10:09:54.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.078+0000 7f7abe600640 1 -- 172.21.15.119:0/1322468810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab813e070 con 0x7f7ab8153160 2024-04-25T10:09:54.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.078+0000 7f7abe600640 1 -- 172.21.15.119:0/1322468810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab813e630 con 0x7f7ab8153160 2024-04-25T10:09:54.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.080+0000 7f7abe600640 1 -- 172.21.15.119:0/1322468810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab80749b0 con 0x7f7ab8153160 2024-04-25T10:09:54.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.080+0000 7f7ab5600640 1 -- 172.21.15.119:0/1322468810 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7aa00039d0 con 0x7f7ab8153160 2024-04-25T10:09:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.082+0000 7f7ab5600640 1 --2- 172.21.15.119:0/1322468810 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a9406b500 0x7f7a9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.082+0000 7f7ab5600640 1 -- 172.21.15.119:0/1322468810 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7aa0089b00 con 0x7f7ab8153160 2024-04-25T10:09:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.082+0000 7f7abd200640 1 --2- 172.21.15.119:0/1322468810 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a9406b500 0x7f7a9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.083+0000 7f7abd200640 1 --2- 172.21.15.119:0/1322468810 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a9406b500 0x7f7a9406d9c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8006fd0 tx=0x7f7aa8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.088+0000 7f7ab5600640 1 -- 172.21.15.119:0/1322468810 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7aa0058be0 con 0x7f7ab8153160 2024-04-25T10:09:54.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.323+0000 7f7abe600640 1 -- 172.21.15.119:0/1322468810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ab8141af0 con 0x7f7ab8153160 2024-04-25T10:09:54.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.323+0000 7f7ab5600640 1 -- 172.21.15.119:0/1322468810 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f7aa00584e0 con 0x7f7ab8153160 2024-04-25T10:09:54.330 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:54.330 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3317744182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3317744182},{"type":"v1","addr":"172.21.15.132:6833","nonce":3317744182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/700427287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":700427287},{"type":"v1","addr":"172.21.15.132:6835","nonce":700427287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2207174972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2207174972},{"type":"v1","addr":"172.21.15.119:6836","nonce":2207174972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5853,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2455205373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2455205373},{"type":"v1","addr":"172.21.15.119:6837","nonce":2455205373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[]} 2024-04-25T10:09:54.330 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 27 2024-04-25T10:09:54.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.326+0000 7f7abe600640 1 -- 172.21.15.119:0/1322468810 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a9406b500 msgr2=0x7f7a9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.326+0000 7f7abe600640 1 --2- 172.21.15.119:0/1322468810 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a9406b500 0x7f7a9406d9c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8006fd0 tx=0x7f7aa8008040 comp rx=0 tx=0).stop 2024-04-25T10:09:54.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.326+0000 7f7abe600640 1 -- 172.21.15.119:0/1322468810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab8153160 msgr2=0x7f7ab81f3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.326+0000 7f7abe600640 1 --2- 172.21.15.119:0/1322468810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab8153160 0x7f7ab81f3900 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000e440 tx=0x7f7aa000e900 comp rx=0 tx=0).stop 2024-04-25T10:09:54.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.326+0000 7f7abe600640 1 -- 172.21.15.119:0/1322468810 shutdown_connections 2024-04-25T10:09:54.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.326+0000 7f7abe600640 1 --2- 172.21.15.119:0/1322468810 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a9406b500 0x7f7a9406d9c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.326+0000 7f7abe600640 1 --2- 172.21.15.119:0/1322468810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ab813a090 0x7f7ab815d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.326+0000 7f7abe600640 1 --2- 172.21.15.119:0/1322468810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab8153160 0x7f7ab81f3900 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.326+0000 7f7abe600640 1 --2- 172.21.15.119:0/1322468810 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a9c0052e0 0x7f7ab813d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.326+0000 7f7abe600640 1 -- 172.21.15.119:0/1322468810 >> 172.21.15.119:0/1322468810 conn(0x7f7ab806b1d0 msgr2=0x7f7ab806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:54.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.326+0000 7f7abe600640 1 -- 172.21.15.119:0/1322468810 shutdown_connections 2024-04-25T10:09:54.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.326+0000 7f7abe600640 1 -- 172.21.15.119:0/1322468810 wait complete. 2024-04-25T10:09:54.354 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:09:54.354 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:09:54.355 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:09:54.355 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:09:54.355 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:09:54.355 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:09:54.355 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:09:54.355 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:09:54.355 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:09:54.356 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:09:54.356 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:09:54.356 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:09:54.356 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:09:54.356 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:09:54.356 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:09:54.356 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:09:54.373 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:09:54.374 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:09:54.375 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:09:54.375 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:09:54.379 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:09:54.379 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:09:54.383 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:09:54.383 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:09:54.422 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:09:54.422 DEBUG:teuthology.parallel:result is None 2024-04-25T10:09:54.428 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:09:54.428 DEBUG:teuthology.parallel:result is None 2024-04-25T10:09:54.430 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:09:54.430 DEBUG:teuthology.parallel:result is None 2024-04-25T10:09:54.436 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:09:54.436 DEBUG:teuthology.parallel:result is None 2024-04-25T10:09:54.436 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:09:54.573 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2024-04-25T10:09:54.574 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:09:54.586 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-04-25T10:09:54.587 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-04-25T10:09:54.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.611+0000 7f97aa200640 1 Processor -- start 2024-04-25T10:09:54.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.612+0000 7f97aa200640 1 -- start start 2024-04-25T10:09:54.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.613+0000 7f97aa200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f97a415dbe0 0x7f97a4071c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.613+0000 7f97aa200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97a40aedc0 0x7f97a40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.613+0000 7f97aa200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a40ace70 con 0x7f97a40cf900 2024-04-25T10:09:54.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.613+0000 7f97aa200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a4072150 con 0x7f97a415dbe0 2024-04-25T10:09:54.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.613+0000 7f97aa200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40ad110 con 0x7f97a40aedc0 2024-04-25T10:09:54.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.614+0000 7f97a3400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f97a415dbe0 0x7f97a4071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.614+0000 7f97a3e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f97a40cf900 0x7f97a40b3070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:34266/0 (socket says 172.21.15.119:34266) 2024-04-25T10:09:54.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.614+0000 7f97a3e00640 1 -- 172.21.15.119:0/67519334 learned_addr learned my addr 172.21.15.119:0/67519334 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:54.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.614+0000 7f979be00640 1 --2- 172.21.15.119:0/67519334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97a40aedc0 0x7f97a40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.614+0000 7f97a3400640 1 -- 172.21.15.119:0/67519334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97a40aedc0 msgr2=0x7f97a40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.614+0000 7f97a3400640 1 --2- 172.21.15.119:0/67519334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97a40aedc0 0x7f97a40cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.614+0000 7f97a3400640 1 -- 172.21.15.119:0/67519334 >> v1:172.21.15.132:6789/0 conn(0x7f97a40cf900 legacy=0x7f97a40b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.614+0000 7f97a3400640 1 -- 172.21.15.119:0/67519334 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a40ad3f0 con 0x7f97a415dbe0 2024-04-25T10:09:54.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.614+0000 7f979be00640 1 --2- 172.21.15.119:0/67519334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97a40aedc0 0x7f97a40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:54.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.615+0000 7f97a3400640 1 --2- 172.21.15.119:0/67519334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97a415dbe0 0x7f97a4071c10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f97940097a0 tx=0x7f979402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e571278339f1325a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.615+0000 7f97a2a00640 1 -- 172.21.15.119:0/67519334 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794037070 con 0x7f97a415dbe0 2024-04-25T10:09:54.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.616+0000 7f97a2a00640 1 -- 172.21.15.119:0/67519334 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9794032440 con 0x7f97a415dbe0 2024-04-25T10:09:54.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.616+0000 7f97a2a00640 1 -- 172.21.15.119:0/67519334 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794032810 con 0x7f97a415dbe0 2024-04-25T10:09:54.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.616+0000 7f97aa200640 1 -- 172.21.15.119:0/67519334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97a415dbe0 msgr2=0x7f97a4071c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.616+0000 7f97aa200640 1 --2- 172.21.15.119:0/67519334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97a415dbe0 0x7f97a4071c10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f97940097a0 tx=0x7f979402a180 comp rx=0 tx=0).stop 2024-04-25T10:09:54.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.616+0000 7f97aa200640 1 -- 172.21.15.119:0/67519334 shutdown_connections 2024-04-25T10:09:54.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.616+0000 7f97aa200640 1 --2- 172.21.15.119:0/67519334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97a40aedc0 0x7f97a40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.616+0000 7f97aa200640 1 --2- 172.21.15.119:0/67519334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97a415dbe0 0x7f97a4071c10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.616+0000 7f97aa200640 1 -- 172.21.15.119:0/67519334 >> 172.21.15.119:0/67519334 conn(0x7f97a406aba0 msgr2=0x7f97a406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:54.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.617+0000 7f97aa200640 1 -- 172.21.15.119:0/67519334 shutdown_connections 2024-04-25T10:09:54.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.617+0000 7f97aa200640 1 -- 172.21.15.119:0/67519334 wait complete. 2024-04-25T10:09:54.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.618+0000 7f97aa200640 1 Processor -- start 2024-04-25T10:09:54.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.618+0000 7f97aa200640 1 -- start start 2024-04-25T10:09:54.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.619+0000 7f97aa200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a40aedc0 0x7f97a40c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.619+0000 7f97aa200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97a40cf900 0x7f97a414c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.619+0000 7f97aa200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97a415dbe0 0x7f97a414b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.619+0000 7f97aa200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40adc90 con 0x7f97a40aedc0 2024-04-25T10:09:54.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.619+0000 7f97aa200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40ad9b0 con 0x7f97a40cf900 2024-04-25T10:09:54.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.619+0000 7f97aa200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a415dfc0 con 0x7f97a415dbe0 2024-04-25T10:09:54.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.620+0000 7f97a3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a40aedc0 0x7f97a40c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.620+0000 7f97a3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a40aedc0 0x7f97a40c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42650/0 (socket says 172.21.15.119:42650) 2024-04-25T10:09:54.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.620+0000 7f97a3400640 1 -- 172.21.15.119:0/2933159321 learned_addr learned my addr 172.21.15.119:0/2933159321 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:54.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.620+0000 7f97a3e00640 1 --2- 172.21.15.119:0/2933159321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97a415dbe0 0x7f97a414b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.620+0000 7f979be00640 1 --2- 172.21.15.119:0/2933159321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97a40cf900 0x7f97a414c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.620+0000 7f97a3400640 1 -- 172.21.15.119:0/2933159321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97a415dbe0 msgr2=0x7f97a414b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.620+0000 7f97a3400640 1 --2- 172.21.15.119:0/2933159321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97a415dbe0 0x7f97a414b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.620+0000 7f97a3400640 1 -- 172.21.15.119:0/2933159321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97a40cf900 msgr2=0x7f97a414c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.620+0000 7f97a3400640 1 --2- 172.21.15.119:0/2933159321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97a40cf900 0x7f97a414c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.620+0000 7f97a3400640 1 -- 172.21.15.119:0/2933159321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a414be70 con 0x7f97a40aedc0 2024-04-25T10:09:54.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.620+0000 7f97a3400640 1 --2- 172.21.15.119:0/2933159321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a40aedc0 0x7f97a40c5070 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9794035040 tx=0x7f97940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.621+0000 7f979b400640 1 -- 172.21.15.119:0/2933159321 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794037040 con 0x7f97a40aedc0 2024-04-25T10:09:54.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.621+0000 7f979b400640 1 -- 172.21.15.119:0/2933159321 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9794030500 con 0x7f97a40aedc0 2024-04-25T10:09:54.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.621+0000 7f97aa200640 1 -- 172.21.15.119:0/2933159321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a40c5770 con 0x7f97a40aedc0 2024-04-25T10:09:54.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.621+0000 7f979b400640 1 -- 172.21.15.119:0/2933159321 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97940308d0 con 0x7f97a40aedc0 2024-04-25T10:09:54.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.621+0000 7f97aa200640 1 -- 172.21.15.119:0/2933159321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a40cd8a0 con 0x7f97a40aedc0 2024-04-25T10:09:54.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.622+0000 7f97aa200640 1 -- 172.21.15.119:0/2933159321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a4067c00 con 0x7f97a40aedc0 2024-04-25T10:09:54.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.623+0000 7f979b400640 1 -- 172.21.15.119:0/2933159321 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9794044020 con 0x7f97a40aedc0 2024-04-25T10:09:54.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.624+0000 7f979b400640 1 --2- 172.21.15.119:0/2933159321 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f978006b4d0 0x7f978006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.624+0000 7f979b400640 1 -- 172.21.15.119:0/2933159321 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f97940aa240 con 0x7f97a40aedc0 2024-04-25T10:09:54.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.624+0000 7f979be00640 1 --2- 172.21.15.119:0/2933159321 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f978006b4d0 0x7f978006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.625+0000 7f979be00640 1 --2- 172.21.15.119:0/2933159321 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f978006b4d0 0x7f978006d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f978c009560 tx=0x7f978c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.629+0000 7f979b400640 1 -- 172.21.15.119:0/2933159321 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97a4067c00 con 0x7f97a40aedc0 2024-04-25T10:09:54.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.836+0000 7f97aa200640 1 -- 172.21.15.119:0/2933159321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f97a40750b0 con 0x7f97a40aedc0 2024-04-25T10:09:54.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.962+0000 7f979b400640 1 -- 172.21.15.119:0/2933159321 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v49) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9794079320 con 0x7f97a40aedc0 2024-04-25T10:09:54.969 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2024-04-25T10:09:54.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.966+0000 7f97aa200640 1 -- 172.21.15.119:0/2933159321 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f978006b4d0 msgr2=0x7f978006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.966+0000 7f97aa200640 1 --2- 172.21.15.119:0/2933159321 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f978006b4d0 0x7f978006d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f978c009560 tx=0x7f978c008040 comp rx=0 tx=0).stop 2024-04-25T10:09:54.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.966+0000 7f97aa200640 1 -- 172.21.15.119:0/2933159321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a40aedc0 msgr2=0x7f97a40c5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.966+0000 7f97aa200640 1 --2- 172.21.15.119:0/2933159321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a40aedc0 0x7f97a40c5070 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9794035040 tx=0x7f97940046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.967+0000 7f97aa200640 1 -- 172.21.15.119:0/2933159321 shutdown_connections 2024-04-25T10:09:54.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.967+0000 7f97aa200640 1 --2- 172.21.15.119:0/2933159321 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f978006b4d0 0x7f978006d990 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.967+0000 7f97aa200640 1 --2- 172.21.15.119:0/2933159321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f97a415dbe0 0x7f97a414b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.967+0000 7f97aa200640 1 --2- 172.21.15.119:0/2933159321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97a40cf900 0x7f97a414c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.967+0000 7f97aa200640 1 --2- 172.21.15.119:0/2933159321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a40aedc0 0x7f97a40c5070 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.967+0000 7f97aa200640 1 -- 172.21.15.119:0/2933159321 >> 172.21.15.119:0/2933159321 conn(0x7f97a406aba0 msgr2=0x7f97a4160c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:54.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.967+0000 7f97aa200640 1 -- 172.21.15.119:0/2933159321 shutdown_connections 2024-04-25T10:09:54.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:54.968+0000 7f97aa200640 1 -- 172.21.15.119:0/2933159321 wait complete. 2024-04-25T10:09:54.997 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:09:55.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.175+0000 7f5f05c00640 1 Processor -- start 2024-04-25T10:09:55.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.175+0000 7f5f05c00640 1 -- start start 2024-04-25T10:09:55.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.176+0000 7f5f05c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f0006dba0 0x7f5f000cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.177+0000 7f5f05c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0006dfb0 con 0x7f5f000b11e0 2024-04-25T10:09:55.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.177+0000 7f5f05c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f000ca4a0 con 0x7f5f000b15c0 2024-04-25T10:09:55.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.177+0000 7f5f05c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000ca250 con 0x7f5f0006dba0 2024-04-25T10:09:55.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.177+0000 7f5efea00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5f000b11e0 0x7f5f0015d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50154/0 (socket says 172.21.15.119:50154) 2024-04-25T10:09:55.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.177+0000 7f5efea00640 1 -- 172.21.15.119:0/1382347893 learned_addr learned my addr 172.21.15.119:0/1382347893 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:55.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.177+0000 7f5efd600640 1 -- 172.21.15.119:0/1382347893 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197565225 0 0) 0x7f5ef0001110 con 0x7f5f000b11e0 2024-04-25T10:09:55.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.177+0000 7f5efd600640 1 -- 172.21.15.119:0/1382347893 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee4003520 con 0x7f5f000b11e0 2024-04-25T10:09:55.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5eff400640 1 --2- 172.21.15.119:0/1382347893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f0006dba0 0x7f5f000cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5efd600640 1 -- 172.21.15.119:0/1382347893 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2840659480 0 0) 0x7f5ee4003520 con 0x7f5f000b11e0 2024-04-25T10:09:55.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5efd600640 1 -- 172.21.15.119:0/1382347893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f0006dba0 msgr2=0x7f5f000cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5efd600640 1 --2- 172.21.15.119:0/1382347893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f0006dba0 0x7f5f000cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5efd600640 1 -- 172.21.15.119:0/1382347893 >> v1:172.21.15.132:6789/0 conn(0x7f5f000b15c0 legacy=0x7f5f0006d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5efd600640 1 -- 172.21.15.119:0/1382347893 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f000ca780 con 0x7f5f000b11e0 2024-04-25T10:09:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5efd600640 1 -- 172.21.15.119:0/1382347893 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f5ef0001840 con 0x7f5f000b11e0 2024-04-25T10:09:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5efd600640 1 -- 172.21.15.119:0/1382347893 >> v1:172.21.15.119:6789/0 conn(0x7f5f000b11e0 legacy=0x7f5f0015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5efd600640 1 --2- 172.21.15.119:0/1382347893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ee40039e0 0x7f5ee4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5efd600640 1 --2- 172.21.15.119:0/1382347893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f0006dba0 0x7f5ee4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5efd600640 1 --2- 172.21.15.119:0/1382347893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ee4004d40 0x7f5ee40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5efd600640 1 -- 172.21.15.119:0/1382347893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000ca250 con 0x7f5ee40039e0 2024-04-25T10:09:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5efd600640 1 -- 172.21.15.119:0/1382347893 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee400bb50 con 0x7f5ee4004d40 2024-04-25T10:09:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5efd600640 1 -- 172.21.15.119:0/1382347893 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee400bd50 con 0x7f5f0006dba0 2024-04-25T10:09:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.178+0000 7f5efea00640 1 --2- 172.21.15.119:0/1382347893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ee40039e0 0x7f5ee4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.179+0000 7f5efe000640 1 --2- 172.21.15.119:0/1382347893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f0006dba0 0x7f5ee4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.179+0000 7f5efea00640 1 -- 172.21.15.119:0/1382347893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f0006dba0 msgr2=0x7f5ee4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.179+0000 7f5efea00640 1 --2- 172.21.15.119:0/1382347893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f0006dba0 0x7f5ee4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.179+0000 7f5efea00640 1 -- 172.21.15.119:0/1382347893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ee4004d40 msgr2=0x7f5ee40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.179+0000 7f5efea00640 1 --2- 172.21.15.119:0/1382347893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ee4004d40 0x7f5ee40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.179+0000 7f5efea00640 1 -- 172.21.15.119:0/1382347893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee4003520 con 0x7f5ee40039e0 2024-04-25T10:09:55.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.180+0000 7f5efea00640 1 --2- 172.21.15.119:0/1382347893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ee40039e0 0x7f5ee4003de0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000c2f0 tx=0x7f5ee4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79756ec833a4644a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.180+0000 7f5efd600640 1 -- 172.21.15.119:0/1382347893 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0007e30 con 0x7f5ee40039e0 2024-04-25T10:09:55.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.180+0000 7f5efd600640 1 -- 172.21.15.119:0/1382347893 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef00058d0 con 0x7f5ee40039e0 2024-04-25T10:09:55.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.180+0000 7f5efd600640 1 -- 172.21.15.119:0/1382347893 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0005cc0 con 0x7f5ee40039e0 2024-04-25T10:09:55.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.181+0000 7f5f05c00640 1 -- 172.21.15.119:0/1382347893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ee40039e0 msgr2=0x7f5ee4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.181+0000 7f5f05c00640 1 --2- 172.21.15.119:0/1382347893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ee40039e0 0x7f5ee4003de0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000c2f0 tx=0x7f5ee4005830 comp rx=0 tx=0).stop 2024-04-25T10:09:55.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.181+0000 7f5eff400640 1 -- 172.21.15.119:0/1382347893 reap_dead start 2024-04-25T10:09:55.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.181+0000 7f5f05c00640 1 -- 172.21.15.119:0/1382347893 shutdown_connections 2024-04-25T10:09:55.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.181+0000 7f5f05c00640 1 -- 172.21.15.119:0/1382347893 >> 172.21.15.119:0/1382347893 conn(0x7f5f000cf5a0 msgr2=0x7f5f000cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.181+0000 7f5f05c00640 1 -- 172.21.15.119:0/1382347893 shutdown_connections 2024-04-25T10:09:55.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.181+0000 7f5f05c00640 1 -- 172.21.15.119:0/1382347893 wait complete. 2024-04-25T10:09:55.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.183+0000 7f5f05c00640 1 Processor -- start 2024-04-25T10:09:55.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.183+0000 7f5f05c00640 1 -- start start 2024-04-25T10:09:55.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.184+0000 7f5f05c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ee40039e0 0x7f5f001400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.184+0000 7f5f05c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f001543a0 0x7f5f00154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.184+0000 7f5f05c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0014dac0 0x7f5f0014df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.184+0000 7f5f05c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000b1a30 con 0x7f5f0014dac0 2024-04-25T10:09:55.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.184+0000 7f5f05c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000cad40 con 0x7f5f001543a0 2024-04-25T10:09:55.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.184+0000 7f5f05c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f001597b0 con 0x7f5ee40039e0 2024-04-25T10:09:55.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.184+0000 7f5eff400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0014dac0 0x7f5f0014df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.184+0000 7f5eff400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0014dac0 0x7f5f0014df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42668/0 (socket says 172.21.15.119:42668) 2024-04-25T10:09:55.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.184+0000 7f5eff400640 1 -- 172.21.15.119:0/1650277890 learned_addr learned my addr 172.21.15.119:0/1650277890 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:55.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.185+0000 7f5eff400640 1 -- 172.21.15.119:0/1650277890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ee40039e0 msgr2=0x7f5f001400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.185+0000 7f5eff400640 1 --2- 172.21.15.119:0/1650277890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ee40039e0 0x7f5f001400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.185+0000 7f5eff400640 1 -- 172.21.15.119:0/1650277890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f001543a0 msgr2=0x7f5f00154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.185+0000 7f5eff400640 1 --2- 172.21.15.119:0/1650277890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f001543a0 0x7f5f00154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.185+0000 7f5eff400640 1 -- 172.21.15.119:0/1650277890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f00154e60 con 0x7f5f0014dac0 2024-04-25T10:09:55.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.185+0000 7f5eff400640 1 --2- 172.21.15.119:0/1650277890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0014dac0 0x7f5f0014df40 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5f000cf430 tx=0x7f5ef800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.185+0000 7f5ef6a00640 1 -- 172.21.15.119:0/1650277890 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef800eb30 con 0x7f5f0014dac0 2024-04-25T10:09:55.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.185+0000 7f5ef6a00640 1 -- 172.21.15.119:0/1650277890 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef800ecf0 con 0x7f5f0014dac0 2024-04-25T10:09:55.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.185+0000 7f5ef6a00640 1 -- 172.21.15.119:0/1650277890 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef800eea0 con 0x7f5f0014dac0 2024-04-25T10:09:55.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.185+0000 7f5f05c00640 1 -- 172.21.15.119:0/1650277890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f00155140 con 0x7f5f0014dac0 2024-04-25T10:09:55.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.185+0000 7f5f05c00640 1 -- 172.21.15.119:0/1650277890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0014d420 con 0x7f5f0014dac0 2024-04-25T10:09:55.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.186+0000 7f5f05c00640 1 -- 172.21.15.119:0/1650277890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f000b55b0 con 0x7f5f0014dac0 2024-04-25T10:09:55.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.188+0000 7f5ef6a00640 1 -- 172.21.15.119:0/1650277890 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ef801b020 con 0x7f5f0014dac0 2024-04-25T10:09:55.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.189+0000 7f5ef6a00640 1 --2- 172.21.15.119:0/1650277890 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5ee006b4d0 0x7f5ee006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.189+0000 7f5ef6a00640 1 -- 172.21.15.119:0/1650277890 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5ef8089bc0 con 0x7f5f0014dac0 2024-04-25T10:09:55.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.189+0000 7f5efea00640 1 --2- 172.21.15.119:0/1650277890 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5ee006b4d0 0x7f5ee006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.190+0000 7f5efea00640 1 --2- 172.21.15.119:0/1650277890 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5ee006b4d0 0x7f5ee006d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0033000 tx=0x7f5ef0009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.193+0000 7f5ef6a00640 1 -- 172.21.15.119:0/1650277890 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f000b55b0 con 0x7f5f0014dac0 2024-04-25T10:09:55.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.457+0000 7f5f05c00640 1 -- 172.21.15.119:0/1650277890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5f00156bb0 con 0x7f5f0014dac0 2024-04-25T10:09:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.458+0000 7f5ef6a00640 1 -- 172.21.15.119:0/1650277890 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5ef8058cf0 con 0x7f5f0014dac0 2024-04-25T10:09:55.465 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:55.465 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:09:55.465 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2024-04-25T10:09:55.465 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.465 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2024-04-25T10:09:55.465 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmFDz9FBAAdC/N+8UnkYkK5wrDlgpkgA==", 2024-04-25T10:09:55.465 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.465 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:09:55.465 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:09:55.465 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:09:55.465 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.465 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.466 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2024-04-25T10:09:55.466 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmmh7lLhAAkzsFzTL6Ko/J/0ZJOpvR4A==", 2024-04-25T10:09:55.466 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.466 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:09:55.466 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:09:55.466 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:09:55.466 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.466 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.466 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.466 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.466 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmojotIhAAdSIhFOXoYZN+01nbCs3e+Q==", 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypm1gp7ORAAvtG28IuhGtR1/Vj6JCPq5A==", 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAPKypmOI9YMRAAmMrg0xRUG+GG+waJ12llJQ==", 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.468 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2024-04-25T10:09:55.469 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQATKypmHV5GDxAAW0XzpAxF7oxgVBKy5yzavw==", 2024-04-25T10:09:55.469 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.469 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:55.469 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:09:55.469 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.469 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.469 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.469 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.469 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2024-04-25T10:09:55.469 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAWKypmXTiRGhAAZxct/Ck3A3DMBjqNW3HXrg==", 2024-04-25T10:09:55.469 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.469 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:55.469 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:09:55.469 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAZKypmEeGSMhAAwVNTny5OmP/KONYhIXzrrQ==", 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAgKypmh7gPBRAApx1VM3WpYW7hIQzu7+6J7w==", 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAjKypmQ2EaEBAAoFYMlnh5/ar0pAlynwjYOA==", 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.472 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2024-04-25T10:09:55.472 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAmKypmDATkKRAAsIDViOBeDrd5N91wf8zGLA==", 2024-04-25T10:09:55.472 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.472 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:55.472 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:09:55.472 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.472 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.472 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.472 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.472 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2024-04-25T10:09:55.472 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAqKypmNskbCBAAQIYhJ9lfHHr9EP/n5QJoRw==", 2024-04-25T10:09:55.472 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.472 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:09:55.472 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA==", 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmm0DLGhAApkxEcc1jUU5T/9ErkTkT4A==", 2024-04-25T10:09:55.473 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.474 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:09:55.474 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:09:55.474 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:09:55.474 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:09:55.474 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.474 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.474 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.474 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2024-04-25T10:09:55.474 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmMZ89ERAAICfWm/ZgOLQMVSv5xG0Crg==", 2024-04-25T10:09:55.474 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.474 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:09:55.474 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:09:55.474 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:09:55.474 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:09:55.475 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.475 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2024-04-25T10:09:55.475 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmrokCIxAAwi8FtvYSVhMguvZ0OyYt+g==", 2024-04-25T10:09:55.475 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.475 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:09:55.475 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:09:55.475 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:09:55.475 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:09:55.475 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.475 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.475 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.475 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2024-04-25T10:09:55.476 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAKKypmN9WFGRAAi+stjuG2ZDBjafeR9Ygrug==", 2024-04-25T10:09:55.476 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.476 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:09:55.476 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:09:55.476 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:09:55.476 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.476 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.476 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.476 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.476 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2024-04-25T10:09:55.476 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypm/KyKDBAA646fW6qrcXB31usxeIApaA==", 2024-04-25T10:09:55.476 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.476 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:09:55.476 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2024-04-25T10:09:55.477 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.477 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:09:55.477 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:55.477 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2024-04-25T10:09:55.477 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmr8SqBBAA8l4CjUEidXz1obWJFM4DHA==", 2024-04-25T10:09:55.477 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:09:55.477 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:09:55.477 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2024-04-25T10:09:55.477 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:09:55.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.477 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:55.477 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-25T10:09:55.477 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:09:55.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.461+0000 7f5f05c00640 1 -- 172.21.15.119:0/1650277890 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5ee006b4d0 msgr2=0x7f5ee006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.461+0000 7f5f05c00640 1 --2- 172.21.15.119:0/1650277890 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5ee006b4d0 0x7f5ee006d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0033000 tx=0x7f5ef0009030 comp rx=0 tx=0).stop 2024-04-25T10:09:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.461+0000 7f5f05c00640 1 -- 172.21.15.119:0/1650277890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0014dac0 msgr2=0x7f5f0014df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.461+0000 7f5f05c00640 1 --2- 172.21.15.119:0/1650277890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0014dac0 0x7f5f0014df40 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5f000cf430 tx=0x7f5ef800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.461+0000 7f5f05c00640 1 -- 172.21.15.119:0/1650277890 shutdown_connections 2024-04-25T10:09:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.461+0000 7f5f05c00640 1 --2- 172.21.15.119:0/1650277890 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5ee006b4d0 0x7f5ee006d990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.461+0000 7f5f05c00640 1 --2- 172.21.15.119:0/1650277890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0014dac0 0x7f5f0014df40 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.461+0000 7f5f05c00640 1 --2- 172.21.15.119:0/1650277890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f001543a0 0x7f5f00154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.461+0000 7f5f05c00640 1 --2- 172.21.15.119:0/1650277890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ee40039e0 0x7f5f001400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.461+0000 7f5f05c00640 1 -- 172.21.15.119:0/1650277890 >> 172.21.15.119:0/1650277890 conn(0x7f5f000cf5a0 msgr2=0x7f5f000cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.461+0000 7f5f05c00640 1 -- 172.21.15.119:0/1650277890 shutdown_connections 2024-04-25T10:09:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.461+0000 7f5f05c00640 1 -- 172.21.15.119:0/1650277890 wait complete. 2024-04-25T10:09:55.486 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:55.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.678+0000 7fbe4e600640 1 Processor -- start 2024-04-25T10:09:55.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.678+0000 7fbe4e600640 1 -- start start 2024-04-25T10:09:55.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.680+0000 7fbe4e600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbe4815f120 0x7fbe480ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.680+0000 7fbe4e600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe480ce830 0x7fbe480cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.680+0000 7fbe4e600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe480cdfe0 con 0x7fbe480cfbe0 2024-04-25T10:09:55.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.680+0000 7fbe4e600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe480d09f0 con 0x7fbe4815f120 2024-04-25T10:09:55.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.680+0000 7fbe4e600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe480cec70 con 0x7fbe480ce830 2024-04-25T10:09:55.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.680+0000 7fbe47e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbe480cfbe0 0x7fbe48099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50160/0 (socket says 172.21.15.119:50160) 2024-04-25T10:09:55.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.680+0000 7fbe47e00640 1 -- 172.21.15.119:0/2761185300 learned_addr learned my addr 172.21.15.119:0/2761185300 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:55.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.680+0000 7fbe3ea00640 1 --2- 172.21.15.119:0/2761185300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe480ce830 0x7fbe480cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.680+0000 7fbe46a00640 1 -- 172.21.15.119:0/2761185300 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051029319 0 0) 0x7fbe480cdfe0 con 0x7fbe480cfbe0 2024-04-25T10:09:55.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe47400640 1 --2- 172.21.15.119:0/2761185300 >> v2:172.21.15.132:3300/0 conn(0x7fbe4815f120 0x7fbe480ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 -- 172.21.15.119:0/2761185300 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe2c003520 con 0x7fbe480cfbe0 2024-04-25T10:09:55.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 -- 172.21.15.119:0/2761185300 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1547826604 0 0) 0x7fbe2c003520 con 0x7fbe480cfbe0 2024-04-25T10:09:55.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 -- 172.21.15.119:0/2761185300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe480ce830 msgr2=0x7fbe480cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 --2- 172.21.15.119:0/2761185300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe480ce830 0x7fbe480cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 -- 172.21.15.119:0/2761185300 >> v2:172.21.15.132:3300/0 conn(0x7fbe4815f120 msgr2=0x7fbe480ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 --2- 172.21.15.119:0/2761185300 >> v2:172.21.15.132:3300/0 conn(0x7fbe4815f120 0x7fbe480ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 -- 172.21.15.119:0/2761185300 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe480d0430 con 0x7fbe480cfbe0 2024-04-25T10:09:55.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 -- 172.21.15.119:0/2761185300 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fbe400019f0 con 0x7fbe480cfbe0 2024-04-25T10:09:55.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 -- 172.21.15.119:0/2761185300 >> v1:172.21.15.119:6789/0 conn(0x7fbe480cfbe0 legacy=0x7fbe48099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 --2- 172.21.15.119:0/2761185300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe2c0039b0 0x7fbe2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 --2- 172.21.15.119:0/2761185300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe2c004800 0x7fbe2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 --2- 172.21.15.119:0/2761185300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe480ce830 0x7fbe2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 -- 172.21.15.119:0/2761185300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe480d09f0 con 0x7fbe2c0039b0 2024-04-25T10:09:55.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 -- 172.21.15.119:0/2761185300 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe480cec70 con 0x7fbe2c004800 2024-04-25T10:09:55.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe46a00640 1 -- 172.21.15.119:0/2761185300 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c00baa0 con 0x7fbe480ce830 2024-04-25T10:09:55.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.681+0000 7fbe47400640 1 --2- 172.21.15.119:0/2761185300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe2c0039b0 0x7fbe2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.682+0000 7fbe47400640 1 -- 172.21.15.119:0/2761185300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe480ce830 msgr2=0x7fbe2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.682+0000 7fbe47400640 1 --2- 172.21.15.119:0/2761185300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe480ce830 0x7fbe2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.682+0000 7fbe47400640 1 -- 172.21.15.119:0/2761185300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe2c004800 msgr2=0x7fbe2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.682+0000 7fbe47400640 1 --2- 172.21.15.119:0/2761185300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe2c004800 0x7fbe2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.682+0000 7fbe47400640 1 -- 172.21.15.119:0/2761185300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c003520 con 0x7fbe2c0039b0 2024-04-25T10:09:55.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.683+0000 7fbe47400640 1 --2- 172.21.15.119:0/2761185300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe2c0039b0 0x7fbe2c003dd0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800ba30 tx=0x7fbe2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f64c35c831a2646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.683+0000 7fbe46a00640 1 -- 172.21.15.119:0/2761185300 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38030070 con 0x7fbe2c0039b0 2024-04-25T10:09:55.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.683+0000 7fbe46a00640 1 -- 172.21.15.119:0/2761185300 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3802ae00 con 0x7fbe2c0039b0 2024-04-25T10:09:55.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.683+0000 7fbe46a00640 1 -- 172.21.15.119:0/2761185300 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3800b600 con 0x7fbe2c0039b0 2024-04-25T10:09:55.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.684+0000 7fbe4e600640 1 -- 172.21.15.119:0/2761185300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe2c0039b0 msgr2=0x7fbe2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.684+0000 7fbe4e600640 1 --2- 172.21.15.119:0/2761185300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe2c0039b0 0x7fbe2c003dd0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800ba30 tx=0x7fbe2c005830 comp rx=0 tx=0).stop 2024-04-25T10:09:55.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.684+0000 7fbe47e00640 1 -- 172.21.15.119:0/2761185300 reap_dead start 2024-04-25T10:09:55.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.684+0000 7fbe4e600640 1 -- 172.21.15.119:0/2761185300 shutdown_connections 2024-04-25T10:09:55.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.684+0000 7fbe4e600640 1 -- 172.21.15.119:0/2761185300 >> 172.21.15.119:0/2761185300 conn(0x7fbe480d1900 msgr2=0x7fbe480d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.684+0000 7fbe4e600640 1 -- 172.21.15.119:0/2761185300 shutdown_connections 2024-04-25T10:09:55.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.684+0000 7fbe4e600640 1 -- 172.21.15.119:0/2761185300 wait complete. 2024-04-25T10:09:55.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.686+0000 7fbe4e600640 1 Processor -- start 2024-04-25T10:09:55.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.686+0000 7fbe4e600640 1 -- start start 2024-04-25T10:09:55.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe4e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe2c0039b0 0x7fbe4814cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe4e600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe480b0310 0x7fbe480b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe4e600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe4814d680 0x7fbe4814db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe4e600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe480ccab0 con 0x7fbe2c0039b0 2024-04-25T10:09:55.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe4e600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe480cc7d0 con 0x7fbe4814d680 2024-04-25T10:09:55.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe4e600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe480cc480 con 0x7fbe480b0310 2024-04-25T10:09:55.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe47400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe2c0039b0 0x7fbe4814cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe47400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe2c0039b0 0x7fbe4814cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42694/0 (socket says 172.21.15.119:42694) 2024-04-25T10:09:55.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe47400640 1 -- 172.21.15.119:0/1579428783 learned_addr learned my addr 172.21.15.119:0/1579428783 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:55.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe3ea00640 1 --2- 172.21.15.119:0/1579428783 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe480b0310 0x7fbe480b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe47400640 1 -- 172.21.15.119:0/1579428783 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe480b0310 msgr2=0x7fbe480b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe47400640 1 --2- 172.21.15.119:0/1579428783 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe480b0310 0x7fbe480b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe47400640 1 -- 172.21.15.119:0/1579428783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe4814d680 msgr2=0x7fbe4814db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe47e00640 1 --2- 172.21.15.119:0/1579428783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe4814d680 0x7fbe4814db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe47400640 1 --2- 172.21.15.119:0/1579428783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe4814d680 0x7fbe4814db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.687+0000 7fbe47400640 1 -- 172.21.15.119:0/1579428783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4814d290 con 0x7fbe2c0039b0 2024-04-25T10:09:55.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.688+0000 7fbe47400640 1 --2- 172.21.15.119:0/1579428783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe2c0039b0 0x7fbe4814cd50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbe38034d80 tx=0x7fbe38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.688+0000 7fbe3fe00640 1 -- 172.21.15.119:0/1579428783 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38042030 con 0x7fbe2c0039b0 2024-04-25T10:09:55.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.688+0000 7fbe3fe00640 1 -- 172.21.15.119:0/1579428783 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3802ac00 con 0x7fbe2c0039b0 2024-04-25T10:09:55.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.688+0000 7fbe4e600640 1 -- 172.21.15.119:0/1579428783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe480af270 con 0x7fbe2c0039b0 2024-04-25T10:09:55.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.688+0000 7fbe3fe00640 1 -- 172.21.15.119:0/1579428783 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3800b600 con 0x7fbe2c0039b0 2024-04-25T10:09:55.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.688+0000 7fbe4e600640 1 -- 172.21.15.119:0/1579428783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe480af830 con 0x7fbe2c0039b0 2024-04-25T10:09:55.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.689+0000 7fbe4e600640 1 -- 172.21.15.119:0/1579428783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe48067850 con 0x7fbe2c0039b0 2024-04-25T10:09:55.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.690+0000 7fbe3fe00640 1 -- 172.21.15.119:0/1579428783 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe38045070 con 0x7fbe2c0039b0 2024-04-25T10:09:55.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.691+0000 7fbe3fe00640 1 --2- 172.21.15.119:0/1579428783 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe2406b4d0 0x7fbe2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.691+0000 7fbe3fe00640 1 -- 172.21.15.119:0/1579428783 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbe380aa090 con 0x7fbe2c0039b0 2024-04-25T10:09:55.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.692+0000 7fbe3ea00640 1 --2- 172.21.15.119:0/1579428783 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe2406b4d0 0x7fbe2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.692+0000 7fbe3ea00640 1 --2- 172.21.15.119:0/1579428783 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe2406b4d0 0x7fbe2406d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbe4815ca10 tx=0x7fbe30008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.696+0000 7fbe3fe00640 1 -- 172.21.15.119:0/1579428783 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe4814d290 con 0x7fbe2c0039b0 2024-04-25T10:09:55.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.890+0000 7fbe4e600640 1 -- 172.21.15.119:0/1579428783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe480749b0 con 0x7fbe2c0039b0 2024-04-25T10:09:55.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.892+0000 7fbe3fe00640 1 -- 172.21.15.119:0/1579428783 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+12927 (secure 0 0 0) 0x7fbe380333b0 con 0x7fbe2c0039b0 2024-04-25T10:09:55.898 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:55.899 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":51,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:09:54.975758+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.895+0000 7fbe4e600640 1 -- 172.21.15.119:0/1579428783 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe2406b4d0 msgr2=0x7fbe2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.895+0000 7fbe4e600640 1 --2- 172.21.15.119:0/1579428783 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe2406b4d0 0x7fbe2406d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbe4815ca10 tx=0x7fbe30008040 comp rx=0 tx=0).stop 2024-04-25T10:09:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.895+0000 7fbe4e600640 1 -- 172.21.15.119:0/1579428783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe2c0039b0 msgr2=0x7fbe4814cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.895+0000 7fbe4e600640 1 --2- 172.21.15.119:0/1579428783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe2c0039b0 0x7fbe4814cd50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbe38034d80 tx=0x7fbe38004730 comp rx=0 tx=0).stop 2024-04-25T10:09:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.895+0000 7fbe4e600640 1 -- 172.21.15.119:0/1579428783 shutdown_connections 2024-04-25T10:09:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.895+0000 7fbe4e600640 1 --2- 172.21.15.119:0/1579428783 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe2406b4d0 0x7fbe2406d990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.895+0000 7fbe4e600640 1 --2- 172.21.15.119:0/1579428783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe4814d680 0x7fbe4814db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.895+0000 7fbe4e600640 1 --2- 172.21.15.119:0/1579428783 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe480b0310 0x7fbe480b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.895+0000 7fbe4e600640 1 --2- 172.21.15.119:0/1579428783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe2c0039b0 0x7fbe4814cd50 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.895+0000 7fbe4e600640 1 -- 172.21.15.119:0/1579428783 >> 172.21.15.119:0/1579428783 conn(0x7fbe480d1900 msgr2=0x7fbe480cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.895+0000 7fbe4e600640 1 -- 172.21.15.119:0/1579428783 shutdown_connections 2024-04-25T10:09:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:55.895+0000 7fbe4e600640 1 -- 172.21.15.119:0/1579428783 wait complete. 2024-04-25T10:09:55.919 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:55.920 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:56.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.106+0000 7efdddc00640 1 Processor -- start 2024-04-25T10:09:56.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.106+0000 7efdddc00640 1 -- start start 2024-04-25T10:09:56.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.107+0000 7efdddc00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efdd81450e0 0x7efdd806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.107+0000 7efdddc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efdd806b0e0 0x7efdd80cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.107+0000 7efdddc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80cc810 0x7efdd8071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.107+0000 7efdddc00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd81454c0 con 0x7efdd81450e0 2024-04-25T10:09:56.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.107+0000 7efdddc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd806b540 con 0x7efdd806b0e0 2024-04-25T10:09:56.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.107+0000 7efdddc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd815f400 con 0x7efdd80cc810 2024-04-25T10:09:56.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.108+0000 7efdd7e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efdd81450e0 0x7efdd806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.108+0000 7efdd7e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efdd81450e0 0x7efdd806aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42710/0 (socket says 172.21.15.119:42710) 2024-04-25T10:09:56.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.108+0000 7efdd7e00640 1 -- 172.21.15.119:0/1684740533 learned_addr learned my addr 172.21.15.119:0/1684740533 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:56.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.108+0000 7efdcf400640 1 --2- 172.21.15.119:0/1684740533 >> v2:172.21.15.132:3300/0 conn(0x7efdd806b0e0 0x7efdd80cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.108+0000 7efdd7e00640 1 -- 172.21.15.119:0/1684740533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80cc810 msgr2=0x7efdd8071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:56.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.108+0000 7efdd7e00640 1 --2- 172.21.15.119:0/1684740533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80cc810 0x7efdd8071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.108+0000 7efdd7e00640 1 -- 172.21.15.119:0/1684740533 >> v2:172.21.15.132:3300/0 conn(0x7efdd806b0e0 msgr2=0x7efdd80cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:56.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.108+0000 7efdd7e00640 1 --2- 172.21.15.119:0/1684740533 >> v2:172.21.15.132:3300/0 conn(0x7efdd806b0e0 0x7efdd80cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.108+0000 7efdd7e00640 1 -- 172.21.15.119:0/1684740533 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd815f6e0 con 0x7efdd81450e0 2024-04-25T10:09:56.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.108+0000 7efdcf400640 1 --2- 172.21.15.119:0/1684740533 >> v2:172.21.15.132:3300/0 conn(0x7efdd806b0e0 0x7efdd80cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:56.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.109+0000 7efdd7e00640 1 --2- 172.21.15.119:0/1684740533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd81450e0 0x7efdd806aba0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7efdd80724f0 tx=0x7efdc802a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b1d73db28d47c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.109+0000 7efdd7400640 1 -- 172.21.15.119:0/1684740533 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8036070 con 0x7efdd81450e0 2024-04-25T10:09:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.110+0000 7efdd7400640 1 -- 172.21.15.119:0/1684740533 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc8031600 con 0x7efdd81450e0 2024-04-25T10:09:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.110+0000 7efdd7400640 1 -- 172.21.15.119:0/1684740533 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc80319d0 con 0x7efdd81450e0 2024-04-25T10:09:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.110+0000 7efdddc00640 1 -- 172.21.15.119:0/1684740533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd81450e0 msgr2=0x7efdd806aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.110+0000 7efdddc00640 1 --2- 172.21.15.119:0/1684740533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd81450e0 0x7efdd806aba0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7efdd80724f0 tx=0x7efdc802a570 comp rx=0 tx=0).stop 2024-04-25T10:09:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.110+0000 7efdddc00640 1 -- 172.21.15.119:0/1684740533 shutdown_connections 2024-04-25T10:09:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.110+0000 7efdddc00640 1 --2- 172.21.15.119:0/1684740533 >> v2:172.21.15.132:3300/0 conn(0x7efdd806b0e0 0x7efdd80cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.110+0000 7efdddc00640 1 --2- 172.21.15.119:0/1684740533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80cc810 0x7efdd8071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.110+0000 7efdddc00640 1 --2- 172.21.15.119:0/1684740533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd81450e0 0x7efdd806aba0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.110+0000 7efdddc00640 1 -- 172.21.15.119:0/1684740533 >> 172.21.15.119:0/1684740533 conn(0x7efdd8066ff0 msgr2=0x7efdd80b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.110+0000 7efdddc00640 1 -- 172.21.15.119:0/1684740533 shutdown_connections 2024-04-25T10:09:56.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.110+0000 7efdddc00640 1 -- 172.21.15.119:0/1684740533 wait complete. 2024-04-25T10:09:56.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.112+0000 7efdddc00640 1 Processor -- start 2024-04-25T10:09:56.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.112+0000 7efdddc00640 1 -- start start 2024-04-25T10:09:56.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.113+0000 7efdddc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd806b0e0 0x7efdd8140b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.113+0000 7efdddc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80cc810 0x7efdd81434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.113+0000 7efdddc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd81450e0 0x7efdd81340b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.113+0000 7efdddc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd80ccc80 con 0x7efdd81450e0 2024-04-25T10:09:56.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.113+0000 7efdddc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd815fca0 con 0x7efdd806b0e0 2024-04-25T10:09:56.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.113+0000 7efdddc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd80cce00 con 0x7efdd80cc810 2024-04-25T10:09:56.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.113+0000 7efddd200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd81450e0 0x7efdd81340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.114+0000 7efddd200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd81450e0 0x7efdd81340b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42716/0 (socket says 172.21.15.119:42716) 2024-04-25T10:09:56.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.114+0000 7efddd200640 1 -- 172.21.15.119:0/2284867183 learned_addr learned my addr 172.21.15.119:0/2284867183 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:56.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.114+0000 7efdd7e00640 1 --2- 172.21.15.119:0/2284867183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd806b0e0 0x7efdd8140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.114+0000 7efdcf400640 1 --2- 172.21.15.119:0/2284867183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80cc810 0x7efdd81434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.114+0000 7efddd200640 1 -- 172.21.15.119:0/2284867183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80cc810 msgr2=0x7efdd81434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.114+0000 7efddd200640 1 --2- 172.21.15.119:0/2284867183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80cc810 0x7efdd81434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.114+0000 7efddd200640 1 -- 172.21.15.119:0/2284867183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd806b0e0 msgr2=0x7efdd8140b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.114+0000 7efddd200640 1 --2- 172.21.15.119:0/2284867183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd806b0e0 0x7efdd8140b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.114+0000 7efddd200640 1 -- 172.21.15.119:0/2284867183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd8131f50 con 0x7efdd81450e0 2024-04-25T10:09:56.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.114+0000 7efddd200640 1 --2- 172.21.15.119:0/2284867183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd81450e0 0x7efdd81340b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009fd0 tx=0x7efdd000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.115+0000 7efdd4c00640 1 -- 172.21.15.119:0/2284867183 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0011830 con 0x7efdd81450e0 2024-04-25T10:09:56.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.115+0000 7efdd4c00640 1 -- 172.21.15.119:0/2284867183 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd0016070 con 0x7efdd81450e0 2024-04-25T10:09:56.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.115+0000 7efdd4c00640 1 -- 172.21.15.119:0/2284867183 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0011bf0 con 0x7efdd81450e0 2024-04-25T10:09:56.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.115+0000 7efdddc00640 1 -- 172.21.15.119:0/2284867183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd8132230 con 0x7efdd81450e0 2024-04-25T10:09:56.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.115+0000 7efdddc00640 1 -- 172.21.15.119:0/2284867183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd81327f0 con 0x7efdd81450e0 2024-04-25T10:09:56.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.116+0000 7efdddc00640 1 -- 172.21.15.119:0/2284867183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd80749b0 con 0x7efdd81450e0 2024-04-25T10:09:56.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.117+0000 7efdd4c00640 1 -- 172.21.15.119:0/2284867183 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efdd001b460 con 0x7efdd81450e0 2024-04-25T10:09:56.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.118+0000 7efdd4c00640 1 --2- 172.21.15.119:0/2284867183 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efdb406b4d0 0x7efdb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.119+0000 7efdd4c00640 1 -- 172.21.15.119:0/2284867183 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efdd0089b60 con 0x7efdd81450e0 2024-04-25T10:09:56.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.119+0000 7efdd7e00640 1 --2- 172.21.15.119:0/2284867183 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efdb406b4d0 0x7efdb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.120+0000 7efdd7e00640 1 --2- 172.21.15.119:0/2284867183 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efdb406b4d0 0x7efdb406d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efdc8033000 tx=0x7efdc8009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.123+0000 7efdd4c00640 1 -- 172.21.15.119:0/2284867183 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efdd0058d20 con 0x7efdd81450e0 2024-04-25T10:09:56.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.321+0000 7efdddc00640 1 -- 172.21.15.119:0/2284867183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efdd80afd20 con 0x7efdd81450e0 2024-04-25T10:09:56.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.322+0000 7efdd4c00640 1 -- 172.21.15.119:0/2284867183 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7efdd0058620 con 0x7efdd81450e0 2024-04-25T10:09:56.328 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:09:56.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.325+0000 7efdddc00640 1 -- 172.21.15.119:0/2284867183 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efdb406b4d0 msgr2=0x7efdb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.325+0000 7efdddc00640 1 --2- 172.21.15.119:0/2284867183 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efdb406b4d0 0x7efdb406d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efdc8033000 tx=0x7efdc8009000 comp rx=0 tx=0).stop 2024-04-25T10:09:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.325+0000 7efdddc00640 1 -- 172.21.15.119:0/2284867183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd81450e0 msgr2=0x7efdd81340b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.325+0000 7efdddc00640 1 --2- 172.21.15.119:0/2284867183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd81450e0 0x7efdd81340b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009fd0 tx=0x7efdd000b8e0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.325+0000 7efdddc00640 1 -- 172.21.15.119:0/2284867183 shutdown_connections 2024-04-25T10:09:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.325+0000 7efdddc00640 1 --2- 172.21.15.119:0/2284867183 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efdb406b4d0 0x7efdb406d990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.325+0000 7efdddc00640 1 --2- 172.21.15.119:0/2284867183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdd81450e0 0x7efdd81340b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.325+0000 7efdddc00640 1 --2- 172.21.15.119:0/2284867183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdd80cc810 0x7efdd81434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.325+0000 7efdddc00640 1 --2- 172.21.15.119:0/2284867183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd806b0e0 0x7efdd8140b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.325+0000 7efdddc00640 1 -- 172.21.15.119:0/2284867183 >> 172.21.15.119:0/2284867183 conn(0x7efdd8066ff0 msgr2=0x7efdd80af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.325+0000 7efdddc00640 1 -- 172.21.15.119:0/2284867183 shutdown_connections 2024-04-25T10:09:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.325+0000 7efdddc00640 1 -- 172.21.15.119:0/2284867183 wait complete. 2024-04-25T10:09:56.350 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:09:56.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.541+0000 7f7c79c00640 1 Processor -- start 2024-04-25T10:09:56.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.541+0000 7f7c79c00640 1 -- start start 2024-04-25T10:09:56.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.542+0000 7f7c79c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7c7415a850 0x7f7c740ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.542+0000 7f7c79c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c740cf070 0x7f7c740cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.542+0000 7f7c79c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c740cdbe0 0x7f7c740b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.542+0000 7f7c79c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c740cb150 con 0x7f7c740cf070 2024-04-25T10:09:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.542+0000 7f7c79c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c740cb420 con 0x7f7c7415a850 2024-04-25T10:09:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.542+0000 7f7c79c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740cb700 con 0x7f7c740cdbe0 2024-04-25T10:09:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.543+0000 7f7c6a000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c740cf070 0x7f7c740cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.543+0000 7f7c6a000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c740cf070 0x7f7c740cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42718/0 (socket says 172.21.15.119:42718) 2024-04-25T10:09:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.543+0000 7f7c6a000640 1 -- 172.21.15.119:0/3352143756 learned_addr learned my addr 172.21.15.119:0/3352143756 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.543+0000 7f7c72a00640 1 --2- 172.21.15.119:0/3352143756 >> v2:172.21.15.132:3300/0 conn(0x7f7c7415a850 0x7f7c740ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.543+0000 7f7c73400640 1 --2- 172.21.15.119:0/3352143756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c740cdbe0 0x7f7c740b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.543+0000 7f7c6a000640 1 -- 172.21.15.119:0/3352143756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c740cdbe0 msgr2=0x7f7c740b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.543+0000 7f7c6a000640 1 --2- 172.21.15.119:0/3352143756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c740cdbe0 0x7f7c740b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.543+0000 7f7c6a000640 1 -- 172.21.15.119:0/3352143756 >> v2:172.21.15.132:3300/0 conn(0x7f7c7415a850 msgr2=0x7f7c740ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.543+0000 7f7c6a000640 1 --2- 172.21.15.119:0/3352143756 >> v2:172.21.15.132:3300/0 conn(0x7f7c7415a850 0x7f7c740ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.543+0000 7f7c6a000640 1 -- 172.21.15.119:0/3352143756 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740cb9e0 con 0x7f7c740cf070 2024-04-25T10:09:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.544+0000 7f7c6a000640 1 --2- 172.21.15.119:0/3352143756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c740cf070 0x7f7c740cd6a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7c740729a0 tx=0x7f7c5c02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd614123829c0a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.545+0000 7f7c72000640 1 -- 172.21.15.119:0/3352143756 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c032790 con 0x7f7c740cf070 2024-04-25T10:09:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.545+0000 7f7c72000640 1 -- 172.21.15.119:0/3352143756 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c038070 con 0x7f7c740cf070 2024-04-25T10:09:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.545+0000 7f7c72000640 1 -- 172.21.15.119:0/3352143756 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c032b50 con 0x7f7c740cf070 2024-04-25T10:09:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.545+0000 7f7c79c00640 1 -- 172.21.15.119:0/3352143756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c740cf070 msgr2=0x7f7c740cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.545+0000 7f7c79c00640 1 --2- 172.21.15.119:0/3352143756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c740cf070 0x7f7c740cd6a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7c740729a0 tx=0x7f7c5c02c890 comp rx=0 tx=0).stop 2024-04-25T10:09:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.545+0000 7f7c79c00640 1 -- 172.21.15.119:0/3352143756 shutdown_connections 2024-04-25T10:09:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.545+0000 7f7c79c00640 1 --2- 172.21.15.119:0/3352143756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c740cdbe0 0x7f7c740b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.545+0000 7f7c79c00640 1 --2- 172.21.15.119:0/3352143756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c740cf070 0x7f7c740cd6a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.545+0000 7f7c79c00640 1 --2- 172.21.15.119:0/3352143756 >> v2:172.21.15.132:3300/0 conn(0x7f7c7415a850 0x7f7c740ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.545+0000 7f7c79c00640 1 -- 172.21.15.119:0/3352143756 >> 172.21.15.119:0/3352143756 conn(0x7f7c740cf5a0 msgr2=0x7f7c740cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:56.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.546+0000 7f7c79c00640 1 -- 172.21.15.119:0/3352143756 shutdown_connections 2024-04-25T10:09:56.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.546+0000 7f7c79c00640 1 -- 172.21.15.119:0/3352143756 wait complete. 2024-04-25T10:09:56.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.547+0000 7f7c79c00640 1 Processor -- start 2024-04-25T10:09:56.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.547+0000 7f7c79c00640 1 -- start start 2024-04-25T10:09:56.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.548+0000 7f7c79c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c740cdbe0 0x7f7c74152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.548+0000 7f7c79c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c740cf070 0x7f7c740b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.548+0000 7f7c79c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c7415a850 0x7f7c74151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.548+0000 7f7c79c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740b5210 con 0x7f7c740cf070 2024-04-25T10:09:56.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.548+0000 7f7c79c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740cbcc0 con 0x7f7c7415a850 2024-04-25T10:09:56.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.548+0000 7f7c79c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740b5390 con 0x7f7c740cdbe0 2024-04-25T10:09:56.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.549+0000 7f7c6a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c740cf070 0x7f7c740b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.549+0000 7f7c6a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c740cf070 0x7f7c740b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42728/0 (socket says 172.21.15.119:42728) 2024-04-25T10:09:56.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.549+0000 7f7c6a000640 1 -- 172.21.15.119:0/468096336 learned_addr learned my addr 172.21.15.119:0/468096336 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:56.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.549+0000 7f7c73400640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c7415a850 0x7f7c74151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.549+0000 7f7c72a00640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c740cdbe0 0x7f7c74152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.549+0000 7f7c6a000640 1 -- 172.21.15.119:0/468096336 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c740cdbe0 msgr2=0x7f7c74152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.549+0000 7f7c6a000640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c740cdbe0 0x7f7c74152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.549+0000 7f7c6a000640 1 -- 172.21.15.119:0/468096336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c7415a850 msgr2=0x7f7c74151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.549+0000 7f7c6a000640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c7415a850 0x7f7c74151760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.549+0000 7f7c6a000640 1 -- 172.21.15.119:0/468096336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740b3e40 con 0x7f7c740cf070 2024-04-25T10:09:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.549+0000 7f7c72a00640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c740cdbe0 0x7f7c74152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.550+0000 7f7c6a000640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c740cf070 0x7f7c740b3740 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c02c8d0 tx=0x7f7c5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.550+0000 7f7c6b400640 1 -- 172.21.15.119:0/468096336 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c043070 con 0x7f7c740cf070 2024-04-25T10:09:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.550+0000 7f7c6b400640 1 -- 172.21.15.119:0/468096336 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c03c620 con 0x7f7c740cf070 2024-04-25T10:09:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.550+0000 7f7c79c00640 1 -- 172.21.15.119:0/468096336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c74150c80 con 0x7f7c740cf070 2024-04-25T10:09:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.550+0000 7f7c6b400640 1 -- 172.21.15.119:0/468096336 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c03ca10 con 0x7f7c740cf070 2024-04-25T10:09:56.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.550+0000 7f7c79c00640 1 -- 172.21.15.119:0/468096336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c741508a0 con 0x7f7c740cf070 2024-04-25T10:09:56.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.551+0000 7f7c79c00640 1 -- 172.21.15.119:0/468096336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c740749b0 con 0x7f7c740cf070 2024-04-25T10:09:56.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.552+0000 7f7c6b400640 1 -- 172.21.15.119:0/468096336 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c5c046070 con 0x7f7c740cf070 2024-04-25T10:09:56.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.553+0000 7f7c6b400640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7c5006b4d0 0x7f7c5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.553+0000 7f7c6b400640 1 -- 172.21.15.119:0/468096336 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7c5c0aa5a0 con 0x7f7c740cf070 2024-04-25T10:09:56.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.554+0000 7f7c72a00640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7c5006b4d0 0x7f7c5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.554+0000 7f7c72a00640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7c5006b4d0 0x7f7c5006d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7c64006fd0 tx=0x7f7c64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.558+0000 7f7c6b400640 1 -- 172.21.15.119:0/468096336 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c5c079760 con 0x7f7c740cf070 2024-04-25T10:09:56.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.751+0000 7f7c79c00640 1 -- 172.21.15.119:0/468096336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7c740ce400 con 0x7f7c740cf070 2024-04-25T10:09:56.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.751+0000 7f7c6b400640 1 -- 172.21.15.119:0/468096336 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7c5c079060 con 0x7f7c740cf070 2024-04-25T10:09:56.758 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:09:56.758 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2024-04-25T10:09:56.758 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:09:56.758 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2024-04-25T10:09:56.758 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2024-04-25T10:09:56.758 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:09:56.758 INFO:teuthology.orchestra.run.smithi119.stdout:] 2024-04-25T10:09:56.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.754+0000 7f7c79c00640 1 -- 172.21.15.119:0/468096336 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7c5006b4d0 msgr2=0x7f7c5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.754+0000 7f7c79c00640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7c5006b4d0 0x7f7c5006d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7c64006fd0 tx=0x7f7c64008040 comp rx=0 tx=0).stop 2024-04-25T10:09:56.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.754+0000 7f7c79c00640 1 -- 172.21.15.119:0/468096336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c740cf070 msgr2=0x7f7c740b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.754+0000 7f7c79c00640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c740cf070 0x7f7c740b3740 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c02c8d0 tx=0x7f7c5c004730 comp rx=0 tx=0).stop 2024-04-25T10:09:56.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.754+0000 7f7c79c00640 1 -- 172.21.15.119:0/468096336 shutdown_connections 2024-04-25T10:09:56.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.754+0000 7f7c79c00640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7c5006b4d0 0x7f7c5006d990 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.754+0000 7f7c79c00640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c7415a850 0x7f7c74151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.754+0000 7f7c79c00640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c740cf070 0x7f7c740b3740 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.754+0000 7f7c79c00640 1 --2- 172.21.15.119:0/468096336 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c740cdbe0 0x7f7c74152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.754+0000 7f7c79c00640 1 -- 172.21.15.119:0/468096336 >> 172.21.15.119:0/468096336 conn(0x7f7c740cf5a0 msgr2=0x7f7c740cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:56.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.754+0000 7f7c79c00640 1 -- 172.21.15.119:0/468096336 shutdown_connections 2024-04-25T10:09:56.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.754+0000 7f7c79c00640 1 -- 172.21.15.119:0/468096336 wait complete. 2024-04-25T10:09:56.778 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:09:56.779 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:09:56.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.973+0000 7f3175000640 1 Processor -- start 2024-04-25T10:09:56.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.974+0000 7f3175000640 1 -- start start 2024-04-25T10:09:56.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.975+0000 7f3175000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f317015bca0 0x7f31700d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.975+0000 7f3175000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31700d1430 0x7f317006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.975+0000 7f3175000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f317006b0e0 0x7f31700ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.975+0000 7f3175000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31700cc480 con 0x7f317015bca0 2024-04-25T10:09:56.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.975+0000 7f3175000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f317015f980 con 0x7f317006b0e0 2024-04-25T10:09:56.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.975+0000 7f3175000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317015fc60 con 0x7f31700d1430 2024-04-25T10:09:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.975+0000 7f316de00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f317015bca0 0x7f31700d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.976+0000 7f316de00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f317015bca0 0x7f31700d0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42740/0 (socket says 172.21.15.119:42740) 2024-04-25T10:09:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.976+0000 7f316de00640 1 -- 172.21.15.119:0/21726971 learned_addr learned my addr 172.21.15.119:0/21726971 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.976+0000 7f3167e00640 1 --2- 172.21.15.119:0/21726971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31700d1430 0x7f317006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.976+0000 7f316e800640 1 --2- 172.21.15.119:0/21726971 >> v2:172.21.15.132:3300/0 conn(0x7f317006b0e0 0x7f31700ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.976+0000 7f316de00640 1 -- 172.21.15.119:0/21726971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31700d1430 msgr2=0x7f317006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.976+0000 7f316de00640 1 --2- 172.21.15.119:0/21726971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31700d1430 0x7f317006aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.976+0000 7f316de00640 1 -- 172.21.15.119:0/21726971 >> v2:172.21.15.132:3300/0 conn(0x7f317006b0e0 msgr2=0x7f31700ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.976+0000 7f316de00640 1 --2- 172.21.15.119:0/21726971 >> v2:172.21.15.132:3300/0 conn(0x7f317006b0e0 0x7f31700ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.976+0000 7f316de00640 1 -- 172.21.15.119:0/21726971 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317015ff40 con 0x7f317015bca0 2024-04-25T10:09:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.976+0000 7f3167e00640 1 --2- 172.21.15.119:0/21726971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31700d1430 0x7f317006aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.977+0000 7f316de00640 1 --2- 172.21.15.119:0/21726971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f317015bca0 0x7f31700d0ef0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f317006bc60 tx=0x7f316002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31554e5ae9c1ed35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.978+0000 7f316d400640 1 -- 172.21.15.119:0/21726971 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160037070 con 0x7f317015bca0 2024-04-25T10:09:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.978+0000 7f316d400640 1 -- 172.21.15.119:0/21726971 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3160032440 con 0x7f317015bca0 2024-04-25T10:09:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.978+0000 7f316d400640 1 -- 172.21.15.119:0/21726971 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160032810 con 0x7f317015bca0 2024-04-25T10:09:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.978+0000 7f3175000640 1 -- 172.21.15.119:0/21726971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f317015bca0 msgr2=0x7f31700d0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.978+0000 7f3175000640 1 --2- 172.21.15.119:0/21726971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f317015bca0 0x7f31700d0ef0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f317006bc60 tx=0x7f316002a140 comp rx=0 tx=0).stop 2024-04-25T10:09:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.978+0000 7f3175000640 1 -- 172.21.15.119:0/21726971 shutdown_connections 2024-04-25T10:09:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.978+0000 7f3175000640 1 --2- 172.21.15.119:0/21726971 >> v2:172.21.15.132:3300/0 conn(0x7f317006b0e0 0x7f31700ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.978+0000 7f3175000640 1 --2- 172.21.15.119:0/21726971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31700d1430 0x7f317006aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.978+0000 7f3175000640 1 --2- 172.21.15.119:0/21726971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f317015bca0 0x7f31700d0ef0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.978+0000 7f3175000640 1 -- 172.21.15.119:0/21726971 >> 172.21.15.119:0/21726971 conn(0x7f3170099a40 msgr2=0x7f31700d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.979+0000 7f3175000640 1 -- 172.21.15.119:0/21726971 shutdown_connections 2024-04-25T10:09:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.979+0000 7f3175000640 1 -- 172.21.15.119:0/21726971 wait complete. 2024-04-25T10:09:56.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.980+0000 7f3175000640 1 Processor -- start 2024-04-25T10:09:56.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.980+0000 7f3175000640 1 -- start start 2024-04-25T10:09:56.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.981+0000 7f3175000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f317006b0e0 0x7f317015c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.981+0000 7f3175000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31700d1430 0x7f317015e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.981+0000 7f3175000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f317015bca0 0x7f317015e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.981+0000 7f3175000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31700ce8f0 con 0x7f317015bca0 2024-04-25T10:09:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.981+0000 7f3175000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317015f610 con 0x7f31700d1430 2024-04-25T10:09:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.981+0000 7f3175000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31700cea70 con 0x7f317006b0e0 2024-04-25T10:09:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f316e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f317015bca0 0x7f317015e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f316e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f317015bca0 0x7f317015e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42744/0 (socket says 172.21.15.119:42744) 2024-04-25T10:09:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f316e800640 1 -- 172.21.15.119:0/2362034348 learned_addr learned my addr 172.21.15.119:0/2362034348 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f316de00640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f317006b0e0 0x7f317015c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f3167e00640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31700d1430 0x7f317015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f316e800640 1 -- 172.21.15.119:0/2362034348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f317006b0e0 msgr2=0x7f317015c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f316e800640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f317006b0e0 0x7f317015c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f316e800640 1 -- 172.21.15.119:0/2362034348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31700d1430 msgr2=0x7f317015e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f316e800640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31700d1430 0x7f317015e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f316e800640 1 -- 172.21.15.119:0/2362034348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317015eef0 con 0x7f317015bca0 2024-04-25T10:09:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f316de00640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f317006b0e0 0x7f317015c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f316e800640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f317015bca0 0x7f317015e790 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3168007e60 tx=0x7f316800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f3166000640 1 -- 172.21.15.119:0/2362034348 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168014620 con 0x7f317015bca0 2024-04-25T10:09:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f3166000640 1 -- 172.21.15.119:0/2362034348 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3168019070 con 0x7f317015bca0 2024-04-25T10:09:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f3166000640 1 -- 172.21.15.119:0/2362034348 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168014a20 con 0x7f317015bca0 2024-04-25T10:09:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.982+0000 7f3175000640 1 -- 172.21.15.119:0/2362034348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31700c5220 con 0x7f317015bca0 2024-04-25T10:09:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.983+0000 7f3175000640 1 -- 172.21.15.119:0/2362034348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31700c57e0 con 0x7f317015bca0 2024-04-25T10:09:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.984+0000 7f3175000640 1 -- 172.21.15.119:0/2362034348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31700750b0 con 0x7f317015bca0 2024-04-25T10:09:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.985+0000 7f3166000640 1 -- 172.21.15.119:0/2362034348 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3168014bc0 con 0x7f317015bca0 2024-04-25T10:09:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.986+0000 7f3166000640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f314c06b4d0 0x7f314c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.986+0000 7f3166000640 1 -- 172.21.15.119:0/2362034348 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f316808a810 con 0x7f317015bca0 2024-04-25T10:09:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.986+0000 7f316de00640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f314c06b4d0 0x7f314c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.987+0000 7f316de00640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f314c06b4d0 0x7f314c06d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3160034000 tx=0x7f316002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:56.990+0000 7f3166000640 1 -- 172.21.15.119:0/2362034348 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3168059940 con 0x7f317015bca0 2024-04-25T10:09:57.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:57.183+0000 7f3175000640 1 -- 172.21.15.119:0/2362034348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f31700b6760 con 0x7f317015bca0 2024-04-25T10:09:57.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:57.982+0000 7f3166000640 1 -- 172.21.15.119:0/2362034348 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v52) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3168059240 con 0x7f317015bca0 2024-04-25T10:09:57.989 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2024-04-25T10:09:57.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:57.986+0000 7f3175000640 1 -- 172.21.15.119:0/2362034348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f314c06b4d0 msgr2=0x7f314c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:57.986+0000 7f3175000640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f314c06b4d0 0x7f314c06d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3160034000 tx=0x7f316002e000 comp rx=0 tx=0).stop 2024-04-25T10:09:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:57.986+0000 7f3175000640 1 -- 172.21.15.119:0/2362034348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f317015bca0 msgr2=0x7f317015e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:57.986+0000 7f3175000640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f317015bca0 0x7f317015e790 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3168007e60 tx=0x7f316800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:57.986+0000 7f3175000640 1 -- 172.21.15.119:0/2362034348 shutdown_connections 2024-04-25T10:09:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:57.986+0000 7f3175000640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f314c06b4d0 0x7f314c06d990 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:57.986+0000 7f3175000640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f317015bca0 0x7f317015e790 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:57.986+0000 7f3175000640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31700d1430 0x7f317015e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:57.986+0000 7f3175000640 1 --2- 172.21.15.119:0/2362034348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f317006b0e0 0x7f317015c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:57.986+0000 7f3175000640 1 -- 172.21.15.119:0/2362034348 >> 172.21.15.119:0/2362034348 conn(0x7f3170099a40 msgr2=0x7f31700ada70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:57.986+0000 7f3175000640 1 -- 172.21.15.119:0/2362034348 shutdown_connections 2024-04-25T10:09:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:57.986+0000 7f3175000640 1 -- 172.21.15.119:0/2362034348 wait complete. 2024-04-25T10:09:58.019 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:09:58.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.209+0000 7f9e58400640 1 Processor -- start 2024-04-25T10:09:58.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.209+0000 7f9e58400640 1 -- start start 2024-04-25T10:09:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.211+0000 7f9e58400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e50165480 0x7f9e500b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.211+0000 7f9e58400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9e501636d0 0x7f9e500b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.211+0000 7f9e58400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e50165ce0 con 0x7f9e500b3610 2024-04-25T10:09:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.211+0000 7f9e58400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e500b3a50 con 0x7f9e501636d0 2024-04-25T10:09:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.211+0000 7f9e58400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e50165fc0 con 0x7f9e50165480 2024-04-25T10:09:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.211+0000 7f9e55000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9e501636d0 0x7f9e500b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:58.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.212+0000 7f9e55000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9e501636d0 0x7f9e500b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42758/0 (socket says 172.21.15.119:42758) 2024-04-25T10:09:58.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.212+0000 7f9e55000640 1 -- 172.21.15.119:0/1727421108 learned_addr learned my addr 172.21.15.119:0/1727421108 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:58.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.212+0000 7f9e55a00640 1 --2- 172.21.15.119:0/1727421108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e50165480 0x7f9e500b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:58.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.212+0000 7f9e55000640 1 -- 172.21.15.119:0/1727421108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e50165480 msgr2=0x7f9e500b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:58.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.212+0000 7f9e55000640 1 --2- 172.21.15.119:0/1727421108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e50165480 0x7f9e500b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.212+0000 7f9e55000640 1 -- 172.21.15.119:0/1727421108 >> v1:172.21.15.132:6789/0 conn(0x7f9e500b3610 legacy=0x7f9e50071300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.212+0000 7f9e55000640 1 -- 172.21.15.119:0/1727421108 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e501662a0 con 0x7f9e501636d0 2024-04-25T10:09:58.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.214+0000 7f9e55000640 1 --2- 172.21.15.119:0/1727421108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e501636d0 0x7f9e500b30d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009870 tx=0x7f9e3c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dffd2b0f4c5d75d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:58.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.214+0000 7f9e4fe00640 1 -- 172.21.15.119:0/1727421108 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c037070 con 0x7f9e501636d0 2024-04-25T10:09:58.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.214+0000 7f9e4fe00640 1 -- 172.21.15.119:0/1727421108 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c032440 con 0x7f9e501636d0 2024-04-25T10:09:58.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.214+0000 7f9e4fe00640 1 -- 172.21.15.119:0/1727421108 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c032810 con 0x7f9e501636d0 2024-04-25T10:09:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.215+0000 7f9e58400640 1 -- 172.21.15.119:0/1727421108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e501636d0 msgr2=0x7f9e500b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.215+0000 7f9e58400640 1 --2- 172.21.15.119:0/1727421108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e501636d0 0x7f9e500b30d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009870 tx=0x7f9e3c02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.215+0000 7f9e58400640 1 -- 172.21.15.119:0/1727421108 shutdown_connections 2024-04-25T10:09:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.215+0000 7f9e58400640 1 --2- 172.21.15.119:0/1727421108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e501636d0 0x7f9e500b30d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.215+0000 7f9e58400640 1 --2- 172.21.15.119:0/1727421108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e50165480 0x7f9e500b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.215+0000 7f9e58400640 1 -- 172.21.15.119:0/1727421108 >> 172.21.15.119:0/1727421108 conn(0x7f9e5006aba0 msgr2=0x7f9e5006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.215+0000 7f9e58400640 1 -- 172.21.15.119:0/1727421108 shutdown_connections 2024-04-25T10:09:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.215+0000 7f9e58400640 1 -- 172.21.15.119:0/1727421108 wait complete. 2024-04-25T10:09:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.217+0000 7f9e58400640 1 Processor -- start 2024-04-25T10:09:58.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.217+0000 7f9e58400640 1 -- start start 2024-04-25T10:09:58.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.218+0000 7f9e58400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e500b3610 0x7f9e500b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:58.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.218+0000 7f9e58400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e501636d0 0x7f9e501fc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:58.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.218+0000 7f9e58400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e50165480 0x7f9e501ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:58.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.218+0000 7f9e58400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e50165860 con 0x7f9e500b3610 2024-04-25T10:09:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.218+0000 7f9e58400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e50166860 con 0x7f9e50165480 2024-04-25T10:09:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.218+0000 7f9e58400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5006e980 con 0x7f9e501636d0 2024-04-25T10:09:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.218+0000 7f9e55a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e500b3610 0x7f9e500b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.218+0000 7f9e55a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e500b3610 0x7f9e500b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42774/0 (socket says 172.21.15.119:42774) 2024-04-25T10:09:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.218+0000 7f9e55a00640 1 -- 172.21.15.119:0/2788221427 learned_addr learned my addr 172.21.15.119:0/2788221427 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.219+0000 7f9e55000640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e501636d0 0x7f9e501fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.219+0000 7f9e55a00640 1 -- 172.21.15.119:0/2788221427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e501636d0 msgr2=0x7f9e501fc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.219+0000 7f9e55a00640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e501636d0 0x7f9e501fc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.219+0000 7f9e56400640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e50165480 0x7f9e501ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.219+0000 7f9e55a00640 1 -- 172.21.15.119:0/2788221427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e50165480 msgr2=0x7f9e501ee0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.219+0000 7f9e55a00640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e50165480 0x7f9e501ee0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.219+0000 7f9e55a00640 1 -- 172.21.15.119:0/2788221427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e501fc6b0 con 0x7f9e500b3610 2024-04-25T10:09:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.219+0000 7f9e55000640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e501636d0 0x7f9e501fc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.219+0000 7f9e56400640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e50165480 0x7f9e501ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.219+0000 7f9e55a00640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e500b3610 0x7f9e500b1350 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400e870 tx=0x7f9e4400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.219+0000 7f9e4d600640 1 -- 172.21.15.119:0/2788221427 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e44017070 con 0x7f9e500b3610 2024-04-25T10:09:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.219+0000 7f9e4d600640 1 -- 172.21.15.119:0/2788221427 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4400c940 con 0x7f9e500b3610 2024-04-25T10:09:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.219+0000 7f9e58400640 1 -- 172.21.15.119:0/2788221427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e500b1e80 con 0x7f9e500b3610 2024-04-25T10:09:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.219+0000 7f9e4d600640 1 -- 172.21.15.119:0/2788221427 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4400cd30 con 0x7f9e500b3610 2024-04-25T10:09:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.220+0000 7f9e58400640 1 -- 172.21.15.119:0/2788221427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5015c8d0 con 0x7f9e500b3610 2024-04-25T10:09:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.221+0000 7f9e58400640 1 -- 172.21.15.119:0/2788221427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e500749b0 con 0x7f9e500b3610 2024-04-25T10:09:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.222+0000 7f9e4d600640 1 -- 172.21.15.119:0/2788221427 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e4401a070 con 0x7f9e500b3610 2024-04-25T10:09:58.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.223+0000 7f9e4d600640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9e3006b4d0 0x7f9e3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:58.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.223+0000 7f9e4d600640 1 -- 172.21.15.119:0/2788221427 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9e44089ef0 con 0x7f9e500b3610 2024-04-25T10:09:58.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.223+0000 7f9e55000640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9e3006b4d0 0x7f9e3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:58.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.224+0000 7f9e55000640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9e3006b4d0 0x7f9e3006d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c034000 tx=0x7f9e3c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:58.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.228+0000 7f9e4d600640 1 -- 172.21.15.119:0/2788221427 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9e44058f40 con 0x7f9e500b3610 2024-04-25T10:09:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.422+0000 7f9e58400640 1 -- 172.21.15.119:0/2788221427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9e500af040 con 0x7f9e500b3610 2024-04-25T10:09:58.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.989+0000 7f9e4d600640 1 -- 172.21.15.119:0/2788221427 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v53) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9e44058840 con 0x7f9e500b3610 2024-04-25T10:09:58.995 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2024-04-25T10:09:58.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.993+0000 7f9e58400640 1 -- 172.21.15.119:0/2788221427 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9e3006b4d0 msgr2=0x7f9e3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.993+0000 7f9e58400640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9e3006b4d0 0x7f9e3006d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c034000 tx=0x7f9e3c02e000 comp rx=0 tx=0).stop 2024-04-25T10:09:58.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.993+0000 7f9e58400640 1 -- 172.21.15.119:0/2788221427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e500b3610 msgr2=0x7f9e500b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.993+0000 7f9e58400640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e500b3610 0x7f9e500b1350 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400e870 tx=0x7f9e4400ed30 comp rx=0 tx=0).stop 2024-04-25T10:09:58.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.993+0000 7f9e58400640 1 -- 172.21.15.119:0/2788221427 shutdown_connections 2024-04-25T10:09:58.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.993+0000 7f9e58400640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9e3006b4d0 0x7f9e3006d990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.993+0000 7f9e58400640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e50165480 0x7f9e501ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.993+0000 7f9e58400640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e501636d0 0x7f9e501fc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.993+0000 7f9e58400640 1 --2- 172.21.15.119:0/2788221427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e500b3610 0x7f9e500b1350 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.993+0000 7f9e58400640 1 -- 172.21.15.119:0/2788221427 >> 172.21.15.119:0/2788221427 conn(0x7f9e5006aba0 msgr2=0x7f9e500ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:59.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.993+0000 7f9e58400640 1 -- 172.21.15.119:0/2788221427 shutdown_connections 2024-04-25T10:09:59.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:58.993+0000 7f9e58400640 1 -- 172.21.15.119:0/2788221427 wait complete. 2024-04-25T10:09:59.034 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:09:59.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.203+0000 7f4bf2400640 1 Processor -- start 2024-04-25T10:09:59.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.203+0000 7f4bf2400640 1 -- start start 2024-04-25T10:09:59.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.205+0000 7f4bf2400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4bec144e70 0x7f4bec06d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.205+0000 7f4bf2400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bec06dcb0 con 0x7f4bec06d7a0 2024-04-25T10:09:59.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.205+0000 7f4bf2400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bec06df90 con 0x7f4bec0cf0e0 2024-04-25T10:09:59.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.205+0000 7f4bf2400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec15bfe0 con 0x7f4bec144e70 2024-04-25T10:09:59.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.205+0000 7f4beaa00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4bec06d7a0 0x7f4bec0ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50174/0 (socket says 172.21.15.119:50174) 2024-04-25T10:09:59.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.205+0000 7f4beaa00640 1 -- 172.21.15.119:0/1070147588 learned_addr learned my addr 172.21.15.119:0/1070147588 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:09:59.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.206+0000 7f4beb400640 1 --2- 172.21.15.119:0/1070147588 >> v2:172.21.15.132:3300/0 conn(0x7f4bec144e70 0x7f4bec06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.206+0000 7f4bea000640 1 -- 172.21.15.119:0/1070147588 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532312249 0 0) 0x7f4bec06dcb0 con 0x7f4bec06d7a0 2024-04-25T10:09:59.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.206+0000 7f4bea000640 1 -- 172.21.15.119:0/1070147588 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bd0003520 con 0x7f4bec06d7a0 2024-04-25T10:09:59.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.206+0000 7f4bea000640 1 -- 172.21.15.119:0/1070147588 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 404641950 0 0) 0x7f4bd0003520 con 0x7f4bec06d7a0 2024-04-25T10:09:59.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.206+0000 7f4bea000640 1 -- 172.21.15.119:0/1070147588 >> v2:172.21.15.132:3300/0 conn(0x7f4bec144e70 msgr2=0x7f4bec06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:59.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.206+0000 7f4bea000640 1 --2- 172.21.15.119:0/1070147588 >> v2:172.21.15.132:3300/0 conn(0x7f4bec144e70 0x7f4bec06d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.206+0000 7f4bea000640 1 -- 172.21.15.119:0/1070147588 >> v1:172.21.15.132:6789/0 conn(0x7f4bec0cf0e0 legacy=0x7f4bec0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:59.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.206+0000 7f4bea000640 1 -- 172.21.15.119:0/1070147588 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec06e270 con 0x7f4bec06d7a0 2024-04-25T10:09:59.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.206+0000 7f4bea000640 1 -- 172.21.15.119:0/1070147588 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f4bd8001670 con 0x7f4bec06d7a0 2024-04-25T10:09:59.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.206+0000 7f4bea000640 1 -- 172.21.15.119:0/1070147588 >> v1:172.21.15.119:6789/0 conn(0x7f4bec06d7a0 legacy=0x7f4bec0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:59.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.206+0000 7f4bea000640 1 --2- 172.21.15.119:0/1070147588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd00039b0 0x7f4bd0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.207+0000 7f4bea000640 1 --2- 172.21.15.119:0/1070147588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bd0004800 0x7f4bd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.207+0000 7f4bea000640 1 --2- 172.21.15.119:0/1070147588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bd0005210 0x7f4bd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.207+0000 7f4bea000640 1 -- 172.21.15.119:0/1070147588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec15bfe0 con 0x7f4bd00039b0 2024-04-25T10:09:59.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.207+0000 7f4bea000640 1 -- 172.21.15.119:0/1070147588 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd000c140 con 0x7f4bd0004800 2024-04-25T10:09:59.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.207+0000 7f4bea000640 1 -- 172.21.15.119:0/1070147588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd000c340 con 0x7f4bd0005210 2024-04-25T10:09:59.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.207+0000 7f4beb400640 1 --2- 172.21.15.119:0/1070147588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd00039b0 0x7f4bd0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.207+0000 7f4beaa00640 1 --2- 172.21.15.119:0/1070147588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bd0004800 0x7f4bd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.207+0000 7f4beb400640 1 -- 172.21.15.119:0/1070147588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bd0005210 msgr2=0x7f4bd000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.207+0000 7f4beb400640 1 --2- 172.21.15.119:0/1070147588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bd0005210 0x7f4bd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.207+0000 7f4beb400640 1 -- 172.21.15.119:0/1070147588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bd0004800 msgr2=0x7f4bd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.207+0000 7f4beb400640 1 --2- 172.21.15.119:0/1070147588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bd0004800 0x7f4bd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.207+0000 7f4beb400640 1 -- 172.21.15.119:0/1070147588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd0003520 con 0x7f4bd00039b0 2024-04-25T10:09:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.207+0000 7f4bebe00640 1 -- 172.21.15.119:0/1070147588 reap_dead start 2024-04-25T10:09:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.208+0000 7f4beb400640 1 --2- 172.21.15.119:0/1070147588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd00039b0 0x7f4bd0003df0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009ac0 tx=0x7f4bd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd529860fddd8505 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.209+0000 7f4bea000640 1 -- 172.21.15.119:0/1070147588 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc036070 con 0x7f4bd00039b0 2024-04-25T10:09:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.209+0000 7f4bea000640 1 -- 172.21.15.119:0/1070147588 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc031400 con 0x7f4bd00039b0 2024-04-25T10:09:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.209+0000 7f4bea000640 1 -- 172.21.15.119:0/1070147588 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc009690 con 0x7f4bd00039b0 2024-04-25T10:09:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.209+0000 7f4bf2400640 1 -- 172.21.15.119:0/1070147588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd00039b0 msgr2=0x7f4bd0003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.209+0000 7f4bf2400640 1 --2- 172.21.15.119:0/1070147588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd00039b0 0x7f4bd0003df0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009ac0 tx=0x7f4bd0005830 comp rx=0 tx=0).stop 2024-04-25T10:09:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.209+0000 7f4bf2400640 1 -- 172.21.15.119:0/1070147588 shutdown_connections 2024-04-25T10:09:59.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.209+0000 7f4bf2400640 1 --2- 172.21.15.119:0/1070147588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd00039b0 0x7f4bd0003df0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.209+0000 7f4bf2400640 1 -- 172.21.15.119:0/1070147588 >> 172.21.15.119:0/1070147588 conn(0x7f4bec06aba0 msgr2=0x7f4bec06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:59.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.209+0000 7f4bf2400640 1 -- 172.21.15.119:0/1070147588 shutdown_connections 2024-04-25T10:09:59.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.209+0000 7f4bf2400640 1 -- 172.21.15.119:0/1070147588 wait complete. 2024-04-25T10:09:59.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.211+0000 7f4bf2400640 1 Processor -- start 2024-04-25T10:09:59.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.211+0000 7f4bf2400640 1 -- start start 2024-04-25T10:09:59.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.212+0000 7f4bf2400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd00039b0 0x7f4bec13a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.212+0000 7f4bf2400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec0b04d0 0x7f4bec0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.212+0000 7f4bf2400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bec0acdf0 0x7f4bec0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.212+0000 7f4bf2400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec145630 con 0x7f4bd00039b0 2024-04-25T10:09:59.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.212+0000 7f4bf2400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec1454b0 con 0x7f4bec0b04d0 2024-04-25T10:09:59.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.212+0000 7f4bf2400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec06d260 con 0x7f4bec0acdf0 2024-04-25T10:09:59.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.212+0000 7f4beb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd00039b0 0x7f4bec13a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.212+0000 7f4beb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd00039b0 0x7f4bec13a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42790/0 (socket says 172.21.15.119:42790) 2024-04-25T10:09:59.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.212+0000 7f4beb400640 1 -- 172.21.15.119:0/3158786209 learned_addr learned my addr 172.21.15.119:0/3158786209 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:09:59.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.213+0000 7f4bebe00640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bec0acdf0 0x7f4bec0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.213+0000 7f4beb400640 1 -- 172.21.15.119:0/3158786209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bec0acdf0 msgr2=0x7f4bec0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:59.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.213+0000 7f4beaa00640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec0b04d0 0x7f4bec0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.213+0000 7f4beb400640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bec0acdf0 0x7f4bec0b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.213+0000 7f4beb400640 1 -- 172.21.15.119:0/3158786209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec0b04d0 msgr2=0x7f4bec0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:59.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.213+0000 7f4beb400640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec0b04d0 0x7f4bec0ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.213+0000 7f4beb400640 1 -- 172.21.15.119:0/3158786209 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec0b0960 con 0x7f4bd00039b0 2024-04-25T10:09:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.213+0000 7f4beaa00640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec0b04d0 0x7f4bec0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.213+0000 7f4beb400640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd00039b0 0x7f4bec13a180 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc034040 tx=0x7f4bdc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.213+0000 7f4be3400640 1 -- 172.21.15.119:0/3158786209 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc03f020 con 0x7f4bd00039b0 2024-04-25T10:09:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.214+0000 7f4be3400640 1 -- 172.21.15.119:0/3158786209 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc037040 con 0x7f4bd00039b0 2024-04-25T10:09:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.214+0000 7f4be3400640 1 -- 172.21.15.119:0/3158786209 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc036070 con 0x7f4bd00039b0 2024-04-25T10:09:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.214+0000 7f4bf2400640 1 -- 172.21.15.119:0/3158786209 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec0b0c40 con 0x7f4bd00039b0 2024-04-25T10:09:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.214+0000 7f4bf2400640 1 -- 172.21.15.119:0/3158786209 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bec0b1200 con 0x7f4bd00039b0 2024-04-25T10:09:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.215+0000 7f4bf2400640 1 -- 172.21.15.119:0/3158786209 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bec0750b0 con 0x7f4bd00039b0 2024-04-25T10:09:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.216+0000 7f4be3400640 1 -- 172.21.15.119:0/3158786209 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bdc042020 con 0x7f4bd00039b0 2024-04-25T10:09:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.218+0000 7f4be3400640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bcc06b4d0 0x7f4bcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.218+0000 7f4be3400640 1 -- 172.21.15.119:0/3158786209 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4bdc0aa680 con 0x7f4bd00039b0 2024-04-25T10:09:59.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.218+0000 7f4beaa00640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bcc06b4d0 0x7f4bcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.219+0000 7f4beaa00640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bcc06b4d0 0x7f4bcc06d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4bec13b3f0 tx=0x7f4bd8001d70 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:59.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.222+0000 7f4be3400640 1 -- 172.21.15.119:0/3158786209 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4bdc049030 con 0x7f4bd00039b0 2024-04-25T10:09:59.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:09:59.452+0000 7f4bf2400640 1 -- 172.21.15.119:0/3158786209 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4bec0aebe0 con 0x7f4bd00039b0 2024-04-25T10:10:00.052 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 8 and data pool 9 2024-04-25T10:10:00.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.040+0000 7f4be3400640 1 -- 172.21.15.119:0/3158786209 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v30) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4bdc079670 con 0x7f4bd00039b0 2024-04-25T10:10:00.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.055+0000 7f4bf2400640 1 -- 172.21.15.119:0/3158786209 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bcc06b4d0 msgr2=0x7f4bcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.055+0000 7f4bf2400640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bcc06b4d0 0x7f4bcc06d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4bec13b3f0 tx=0x7f4bd8001d70 comp rx=0 tx=0).stop 2024-04-25T10:10:00.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.055+0000 7f4bf2400640 1 -- 172.21.15.119:0/3158786209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd00039b0 msgr2=0x7f4bec13a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.055+0000 7f4bf2400640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd00039b0 0x7f4bec13a180 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc034040 tx=0x7f4bdc004730 comp rx=0 tx=0).stop 2024-04-25T10:10:00.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.056+0000 7f4bf2400640 1 -- 172.21.15.119:0/3158786209 shutdown_connections 2024-04-25T10:10:00.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.056+0000 7f4bf2400640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bcc06b4d0 0x7f4bcc06d990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.056+0000 7f4bf2400640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bec0acdf0 0x7f4bec0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.056+0000 7f4bf2400640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bec0b04d0 0x7f4bec0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.056+0000 7f4bf2400640 1 --2- 172.21.15.119:0/3158786209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bd00039b0 0x7f4bec13a180 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.056+0000 7f4bf2400640 1 -- 172.21.15.119:0/3158786209 >> 172.21.15.119:0/3158786209 conn(0x7f4bec06aba0 msgr2=0x7f4bec0cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:00.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.057+0000 7f4bf2400640 1 -- 172.21.15.119:0/3158786209 shutdown_connections 2024-04-25T10:10:00.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.058+0000 7f4bf2400640 1 -- 172.21.15.119:0/3158786209 wait complete. 2024-04-25T10:10:00.094 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:00.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.262+0000 7f87e9600640 1 Processor -- start 2024-04-25T10:10:00.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.262+0000 7f87e9600640 1 -- start start 2024-04-25T10:10:00.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.264+0000 7f87e9600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87e40d35b0 0x7f87e40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:00.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.264+0000 7f87e9600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87e40b5990 0x7f87e40b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:00.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.264+0000 7f87e9600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e415e0c0 con 0x7f87e415dce0 2024-04-25T10:10:00.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.264+0000 7f87e9600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e40b5de0 con 0x7f87e40b5990 2024-04-25T10:10:00.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.264+0000 7f87e9600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40d2510 con 0x7f87e40d35b0 2024-04-25T10:10:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.264+0000 7f87e2a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f87e415dce0 0x7f87e40d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50184/0 (socket says 172.21.15.119:50184) 2024-04-25T10:10:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.264+0000 7f87e2a00640 1 -- 172.21.15.119:0/1569751356 learned_addr learned my addr 172.21.15.119:0/1569751356 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:10:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.264+0000 7f87e3400640 1 --2- 172.21.15.119:0/1569751356 >> v2:172.21.15.119:3300/0 conn(0x7f87e40b5990 0x7f87e40b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.265+0000 7f87dbe00640 1 --2- 172.21.15.119:0/1569751356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87e40d35b0 0x7f87e40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.265+0000 7f87e3400640 1 -- 172.21.15.119:0/1569751356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87e40d35b0 msgr2=0x7f87e40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.265+0000 7f87e3400640 1 --2- 172.21.15.119:0/1569751356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87e40d35b0 0x7f87e40b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.265+0000 7f87e3400640 1 -- 172.21.15.119:0/1569751356 >> v1:172.21.15.119:6789/0 conn(0x7f87e415dce0 legacy=0x7f87e40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.265+0000 7f87e3400640 1 -- 172.21.15.119:0/1569751356 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e40d27f0 con 0x7f87e40b5990 2024-04-25T10:10:00.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.265+0000 7f87dbe00640 1 --2- 172.21.15.119:0/1569751356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87e40d35b0 0x7f87e40b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:00.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.266+0000 7f87e3400640 1 --2- 172.21.15.119:0/1569751356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e40b5990 0x7f87e40b6b00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009d90 tx=0x7f87dc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db24fb78c5630d41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:00.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.266+0000 7f87e2000640 1 -- 172.21.15.119:0/1569751356 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc007740 con 0x7f87e40b5990 2024-04-25T10:10:00.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.266+0000 7f87e2000640 1 -- 172.21.15.119:0/1569751356 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc0078e0 con 0x7f87e40b5990 2024-04-25T10:10:00.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.266+0000 7f87e2000640 1 -- 172.21.15.119:0/1569751356 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc007cb0 con 0x7f87e40b5990 2024-04-25T10:10:00.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.267+0000 7f87e9600640 1 -- 172.21.15.119:0/1569751356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e40b5990 msgr2=0x7f87e40b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.267+0000 7f87e9600640 1 --2- 172.21.15.119:0/1569751356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e40b5990 0x7f87e40b6b00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009d90 tx=0x7f87dc02a660 comp rx=0 tx=0).stop 2024-04-25T10:10:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.267+0000 7f87e9600640 1 -- 172.21.15.119:0/1569751356 shutdown_connections 2024-04-25T10:10:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.267+0000 7f87e9600640 1 --2- 172.21.15.119:0/1569751356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e40b5990 0x7f87e40b6b00 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.267+0000 7f87e9600640 1 --2- 172.21.15.119:0/1569751356 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87e40d35b0 0x7f87e40b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.267+0000 7f87e9600640 1 -- 172.21.15.119:0/1569751356 >> 172.21.15.119:0/1569751356 conn(0x7f87e4072900 msgr2=0x7f87e40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.267+0000 7f87e9600640 1 -- 172.21.15.119:0/1569751356 shutdown_connections 2024-04-25T10:10:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.267+0000 7f87e9600640 1 -- 172.21.15.119:0/1569751356 wait complete. 2024-04-25T10:10:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.269+0000 7f87e9600640 1 Processor -- start 2024-04-25T10:10:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.269+0000 7f87e9600640 1 -- start start 2024-04-25T10:10:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.270+0000 7f87e9600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e40b5990 0x7f87e40c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.270+0000 7f87e9600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87e40d35b0 0x7f87e4152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.270+0000 7f87e9600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87e415dce0 0x7f87e40cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.270+0000 7f87e9600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40d3090 con 0x7f87e40b5990 2024-04-25T10:10:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.270+0000 7f87e9600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40d2db0 con 0x7f87e40d35b0 2024-04-25T10:10:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.270+0000 7f87e9600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40d2160 con 0x7f87e415dce0 2024-04-25T10:10:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.270+0000 7f87e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e40b5990 0x7f87e40c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.270+0000 7f87e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e40b5990 0x7f87e40c7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42798/0 (socket says 172.21.15.119:42798) 2024-04-25T10:10:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.270+0000 7f87e2a00640 1 -- 172.21.15.119:0/1251635887 learned_addr learned my addr 172.21.15.119:0/1251635887 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.271+0000 7f87dbe00640 1 --2- 172.21.15.119:0/1251635887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87e40d35b0 0x7f87e4152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.271+0000 7f87e2a00640 1 -- 172.21.15.119:0/1251635887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87e415dce0 msgr2=0x7f87e40cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.271+0000 7f87e2a00640 1 --2- 172.21.15.119:0/1251635887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87e415dce0 0x7f87e40cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.271+0000 7f87e2a00640 1 -- 172.21.15.119:0/1251635887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87e40d35b0 msgr2=0x7f87e4152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.271+0000 7f87e2a00640 1 --2- 172.21.15.119:0/1251635887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87e40d35b0 0x7f87e4152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.271+0000 7f87e2a00640 1 -- 172.21.15.119:0/1251635887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e4154d00 con 0x7f87e40b5990 2024-04-25T10:10:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.271+0000 7f87dbe00640 1 --2- 172.21.15.119:0/1251635887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87e40d35b0 0x7f87e4152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:00.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.271+0000 7f87e2a00640 1 --2- 172.21.15.119:0/1251635887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e40b5990 0x7f87e40c7500 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f87d400c3c0 tx=0x7f87d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:00.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.271+0000 7f87daa00640 1 -- 172.21.15.119:0/1251635887 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4017070 con 0x7f87e40b5990 2024-04-25T10:10:00.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.271+0000 7f87daa00640 1 -- 172.21.15.119:0/1251635887 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d4005ce0 con 0x7f87e40b5990 2024-04-25T10:10:00.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.271+0000 7f87e9600640 1 -- 172.21.15.119:0/1251635887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e4154fe0 con 0x7f87e40b5990 2024-04-25T10:10:00.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.271+0000 7f87daa00640 1 -- 172.21.15.119:0/1251635887 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d40072f0 con 0x7f87e40b5990 2024-04-25T10:10:00.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.272+0000 7f87e9600640 1 -- 172.21.15.119:0/1251635887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e4155880 con 0x7f87e40b5990 2024-04-25T10:10:00.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.273+0000 7f87e9600640 1 -- 172.21.15.119:0/1251635887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e41551b0 con 0x7f87e40b5990 2024-04-25T10:10:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.274+0000 7f87daa00640 1 -- 172.21.15.119:0/1251635887 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87d401a070 con 0x7f87e40b5990 2024-04-25T10:10:00.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.275+0000 7f87daa00640 1 --2- 172.21.15.119:0/1251635887 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87c006b4d0 0x7f87c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:00.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.275+0000 7f87daa00640 1 -- 172.21.15.119:0/1251635887 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87d40895b0 con 0x7f87e40b5990 2024-04-25T10:10:00.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.275+0000 7f87dbe00640 1 --2- 172.21.15.119:0/1251635887 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87c006b4d0 0x7f87c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:00.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.276+0000 7f87dbe00640 1 --2- 172.21.15.119:0/1251635887 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87c006b4d0 0x7f87c006d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f87cc006fd0 tx=0x7f87cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:00.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.279+0000 7f87daa00640 1 -- 172.21.15.119:0/1251635887 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87d4058420 con 0x7f87e40b5990 2024-04-25T10:10:00.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.475+0000 7f87e9600640 1 -- 172.21.15.119:0/1251635887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87e40749b0 con 0x7f87e40b5990 2024-04-25T10:10:00.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.476+0000 7f87daa00640 1 -- 172.21.15.119:0/1251635887 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16714 (secure 0 0 0) 0x7f87d4057d20 con 0x7f87e40b5990 2024-04-25T10:10:00.482 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:10:00.482 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":54,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:09:59.454189+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:57.237546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:58.475727+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:00.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.478+0000 7f87e9600640 1 -- 172.21.15.119:0/1251635887 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87c006b4d0 msgr2=0x7f87c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.478+0000 7f87e9600640 1 --2- 172.21.15.119:0/1251635887 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87c006b4d0 0x7f87c006d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f87cc006fd0 tx=0x7f87cc008040 comp rx=0 tx=0).stop 2024-04-25T10:10:00.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.478+0000 7f87e9600640 1 -- 172.21.15.119:0/1251635887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e40b5990 msgr2=0x7f87e40c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.478+0000 7f87e9600640 1 --2- 172.21.15.119:0/1251635887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e40b5990 0x7f87e40c7500 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f87d400c3c0 tx=0x7f87d400c880 comp rx=0 tx=0).stop 2024-04-25T10:10:00.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.478+0000 7f87e9600640 1 -- 172.21.15.119:0/1251635887 shutdown_connections 2024-04-25T10:10:00.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.478+0000 7f87e9600640 1 --2- 172.21.15.119:0/1251635887 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87c006b4d0 0x7f87c006d990 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.478+0000 7f87e9600640 1 --2- 172.21.15.119:0/1251635887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87e415dce0 0x7f87e40cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.478+0000 7f87e9600640 1 --2- 172.21.15.119:0/1251635887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87e40d35b0 0x7f87e4152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.478+0000 7f87e9600640 1 --2- 172.21.15.119:0/1251635887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e40b5990 0x7f87e40c7500 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.478+0000 7f87e9600640 1 -- 172.21.15.119:0/1251635887 >> 172.21.15.119:0/1251635887 conn(0x7f87e4072900 msgr2=0x7f87e406bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:00.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.478+0000 7f87e9600640 1 -- 172.21.15.119:0/1251635887 shutdown_connections 2024-04-25T10:10:00.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.478+0000 7f87e9600640 1 -- 172.21.15.119:0/1251635887 wait complete. 2024-04-25T10:10:00.501 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:00.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.664+0000 7f67f5600640 1 Processor -- start 2024-04-25T10:10:00.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.664+0000 7f67f5600640 1 -- start start 2024-04-25T10:10:00.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.665+0000 7f67f5600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67f006e2e0 0x7f67f00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:00.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.665+0000 7f67f5600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f00cdb10 con 0x7f67f00cfc40 2024-04-25T10:10:00.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.665+0000 7f67f5600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f00cddf0 con 0x7f67f015a870 2024-04-25T10:10:00.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.665+0000 7f67f5600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f00ce720 con 0x7f67f006e2e0 2024-04-25T10:10:00.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.666+0000 7f67ef400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f67f00cfc40 0x7f67f015bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50188/0 (socket says 172.21.15.119:50188) 2024-04-25T10:10:00.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.666+0000 7f67ef400640 1 -- 172.21.15.119:0/453412046 learned_addr learned my addr 172.21.15.119:0/453412046 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:10:00.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.666+0000 7f67e7e00640 1 --2- 172.21.15.119:0/453412046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67f006e2e0 0x7f67f00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:00.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.666+0000 7f67ee000640 1 -- 172.21.15.119:0/453412046 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3353677284 0 0) 0x7f67f00cdb10 con 0x7f67f00cfc40 2024-04-25T10:10:00.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.666+0000 7f67ee000640 1 -- 172.21.15.119:0/453412046 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d4003520 con 0x7f67f00cfc40 2024-04-25T10:10:00.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67ee000640 1 -- 172.21.15.119:0/453412046 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1601047450 0 0) 0x7f67d4003520 con 0x7f67f00cfc40 2024-04-25T10:10:00.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67ee000640 1 -- 172.21.15.119:0/453412046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67f006e2e0 msgr2=0x7f67f00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:00.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67ee000640 1 --2- 172.21.15.119:0/453412046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67f006e2e0 0x7f67f00cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67ee000640 1 -- 172.21.15.119:0/453412046 >> v1:172.21.15.132:6789/0 conn(0x7f67f015a870 legacy=0x7f67f00cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67ee000640 1 -- 172.21.15.119:0/453412046 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f00ce0d0 con 0x7f67f00cfc40 2024-04-25T10:10:00.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67e7e00640 1 --2- 172.21.15.119:0/453412046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67f006e2e0 0x7f67f00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:00.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67ee000640 1 -- 172.21.15.119:0/453412046 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f67e80019f0 con 0x7f67f00cfc40 2024-04-25T10:10:00.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67ee000640 1 -- 172.21.15.119:0/453412046 >> v1:172.21.15.119:6789/0 conn(0x7f67f00cfc40 legacy=0x7f67f015bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67ee000640 1 --2- 172.21.15.119:0/453412046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67d40039e0 0x7f67d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:00.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67ee000640 1 --2- 172.21.15.119:0/453412046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67f006e2e0 0x7f67d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:00.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67ee000640 1 --2- 172.21.15.119:0/453412046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d4004d40 0x7f67d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:00.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67ee000640 1 -- 172.21.15.119:0/453412046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f00ce720 con 0x7f67d4004d40 2024-04-25T10:10:00.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67ee000640 1 -- 172.21.15.119:0/453412046 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d400bad0 con 0x7f67d40039e0 2024-04-25T10:10:00.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67ee000640 1 -- 172.21.15.119:0/453412046 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d400bcd0 con 0x7f67f006e2e0 2024-04-25T10:10:00.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67ef400640 1 --2- 172.21.15.119:0/453412046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d4004d40 0x7f67d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:00.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67eea00640 1 --2- 172.21.15.119:0/453412046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67d40039e0 0x7f67d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:00.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.667+0000 7f67e7e00640 1 --2- 172.21.15.119:0/453412046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67f006e2e0 0x7f67d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:00.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.668+0000 7f67ef400640 1 -- 172.21.15.119:0/453412046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67f006e2e0 msgr2=0x7f67d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:00.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.668+0000 7f67ef400640 1 --2- 172.21.15.119:0/453412046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67f006e2e0 0x7f67d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.668+0000 7f67ef400640 1 -- 172.21.15.119:0/453412046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67d40039e0 msgr2=0x7f67d4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:00.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.668+0000 7f67ef400640 1 --2- 172.21.15.119:0/453412046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67d40039e0 0x7f67d4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.668+0000 7f67ef400640 1 -- 172.21.15.119:0/453412046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d4003520 con 0x7f67d4004d40 2024-04-25T10:10:00.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.669+0000 7f67ef400640 1 --2- 172.21.15.119:0/453412046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d4004d40 0x7f67d40051c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f67f00d0050 tx=0x7f67d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe192f1e56a288f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:00.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.669+0000 7f67ee000640 1 -- 172.21.15.119:0/453412046 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e800a460 con 0x7f67d4004d40 2024-04-25T10:10:00.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.669+0000 7f67ee000640 1 -- 172.21.15.119:0/453412046 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e802f070 con 0x7f67d4004d40 2024-04-25T10:10:00.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.669+0000 7f67ee000640 1 -- 172.21.15.119:0/453412046 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e802bc50 con 0x7f67d4004d40 2024-04-25T10:10:00.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.670+0000 7f67f5600640 1 -- 172.21.15.119:0/453412046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d4004d40 msgr2=0x7f67d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.670+0000 7f67f5600640 1 --2- 172.21.15.119:0/453412046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d4004d40 0x7f67d40051c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f67f00d0050 tx=0x7f67d4005830 comp rx=0 tx=0).stop 2024-04-25T10:10:00.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.670+0000 7f67ef400640 1 -- 172.21.15.119:0/453412046 reap_dead start 2024-04-25T10:10:00.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.670+0000 7f67f5600640 1 -- 172.21.15.119:0/453412046 shutdown_connections 2024-04-25T10:10:00.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.670+0000 7f67f5600640 1 -- 172.21.15.119:0/453412046 >> 172.21.15.119:0/453412046 conn(0x7f67f006aba0 msgr2=0x7f67f006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:00.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.670+0000 7f67f5600640 1 -- 172.21.15.119:0/453412046 shutdown_connections 2024-04-25T10:10:00.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.670+0000 7f67f5600640 1 -- 172.21.15.119:0/453412046 wait complete. 2024-04-25T10:10:00.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.672+0000 7f67f5600640 1 Processor -- start 2024-04-25T10:10:00.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.672+0000 7f67f5600640 1 -- start start 2024-04-25T10:10:00.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.672+0000 7f67f5600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d4004d40 0x7f67f014bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:00.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.673+0000 7f67f5600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67f0156a60 0x7f67f014c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:00.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.673+0000 7f67f5600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67f01e53e0 0x7f67f01e5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:00.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.673+0000 7f67f5600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f00cd700 con 0x7f67d4004d40 2024-04-25T10:10:00.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.673+0000 7f67f5600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f00cd580 con 0x7f67f01e53e0 2024-04-25T10:10:00.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.673+0000 7f67f5600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f00cd880 con 0x7f67f0156a60 2024-04-25T10:10:00.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.673+0000 7f67eea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d4004d40 0x7f67f014bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:00.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.673+0000 7f67eea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d4004d40 0x7f67f014bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42820/0 (socket says 172.21.15.119:42820) 2024-04-25T10:10:00.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.673+0000 7f67eea00640 1 -- 172.21.15.119:0/2753224451 learned_addr learned my addr 172.21.15.119:0/2753224451 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:00.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.673+0000 7f67eea00640 1 -- 172.21.15.119:0/2753224451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67f0156a60 msgr2=0x7f67f014c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.673+0000 7f67eea00640 1 --2- 172.21.15.119:0/2753224451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67f0156a60 0x7f67f014c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.673+0000 7f67ef400640 1 --2- 172.21.15.119:0/2753224451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67f01e53e0 0x7f67f01e5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:00.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.673+0000 7f67eea00640 1 -- 172.21.15.119:0/2753224451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67f01e53e0 msgr2=0x7f67f01e5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.673+0000 7f67eea00640 1 --2- 172.21.15.119:0/2753224451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67f01e53e0 0x7f67f01e5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.673+0000 7f67eea00640 1 -- 172.21.15.119:0/2753224451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f0158880 con 0x7f67d4004d40 2024-04-25T10:10:00.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.674+0000 7f67eea00640 1 --2- 172.21.15.119:0/2753224451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d4004d40 0x7f67f014bc80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f67e000e580 tx=0x7f67e000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:00.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.674+0000 7f67e6a00640 1 -- 172.21.15.119:0/2753224451 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e00052e0 con 0x7f67d4004d40 2024-04-25T10:10:00.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.674+0000 7f67e6a00640 1 -- 172.21.15.119:0/2753224451 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e0018070 con 0x7f67d4004d40 2024-04-25T10:10:00.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.674+0000 7f67e6a00640 1 -- 172.21.15.119:0/2753224451 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0003810 con 0x7f67d4004d40 2024-04-25T10:10:00.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.674+0000 7f67f5600640 1 -- 172.21.15.119:0/2753224451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f0158b60 con 0x7f67d4004d40 2024-04-25T10:10:00.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.674+0000 7f67f5600640 1 -- 172.21.15.119:0/2753224451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f0159120 con 0x7f67d4004d40 2024-04-25T10:10:00.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.675+0000 7f67f5600640 1 -- 172.21.15.119:0/2753224451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f00749b0 con 0x7f67d4004d40 2024-04-25T10:10:00.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.676+0000 7f67e6a00640 1 -- 172.21.15.119:0/2753224451 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67e00039b0 con 0x7f67d4004d40 2024-04-25T10:10:00.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.677+0000 7f67e6a00640 1 --2- 172.21.15.119:0/2753224451 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f67cc06b500 0x7f67cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:00.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.677+0000 7f67e6a00640 1 -- 172.21.15.119:0/2753224451 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67e0089e10 con 0x7f67d4004d40 2024-04-25T10:10:00.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.677+0000 7f67e7e00640 1 --2- 172.21.15.119:0/2753224451 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f67cc06b500 0x7f67cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:00.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.678+0000 7f67e7e00640 1 --2- 172.21.15.119:0/2753224451 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f67cc06b500 0x7f67cc06d9c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f67d8006f80 tx=0x7f67d8006e30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:00.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.681+0000 7f67e6a00640 1 -- 172.21.15.119:0/2753224451 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67e008d030 con 0x7f67d4004d40 2024-04-25T10:10:00.732 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:00.741 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:00.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.870+0000 7f67f5600640 1 -- 172.21.15.119:0/2753224451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67f015c470 con 0x7f67d4004d40 2024-04-25T10:10:00.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.873+0000 7f67e6a00640 1 -- 172.21.15.119:0/2753224451 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16714 (secure 0 0 0) 0x7f67e0058c00 con 0x7f67d4004d40 2024-04-25T10:10:00.881 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:10:00.881 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":54,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:09:59.454189+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:57.237546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:58.475727+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:00.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.876+0000 7f67f5600640 1 -- 172.21.15.119:0/2753224451 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f67cc06b500 msgr2=0x7f67cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.876+0000 7f67f5600640 1 --2- 172.21.15.119:0/2753224451 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f67cc06b500 0x7f67cc06d9c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f67d8006f80 tx=0x7f67d8006e30 comp rx=0 tx=0).stop 2024-04-25T10:10:00.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.876+0000 7f67f5600640 1 -- 172.21.15.119:0/2753224451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d4004d40 msgr2=0x7f67f014bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:00.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.876+0000 7f67f5600640 1 --2- 172.21.15.119:0/2753224451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d4004d40 0x7f67f014bc80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f67e000e580 tx=0x7f67e000ea40 comp rx=0 tx=0).stop 2024-04-25T10:10:00.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.876+0000 7f67f5600640 1 -- 172.21.15.119:0/2753224451 shutdown_connections 2024-04-25T10:10:00.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.876+0000 7f67f5600640 1 --2- 172.21.15.119:0/2753224451 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f67cc06b500 0x7f67cc06d9c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.876+0000 7f67f5600640 1 --2- 172.21.15.119:0/2753224451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67f01e53e0 0x7f67f01e5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.876+0000 7f67f5600640 1 --2- 172.21.15.119:0/2753224451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67f0156a60 0x7f67f014c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.876+0000 7f67f5600640 1 --2- 172.21.15.119:0/2753224451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d4004d40 0x7f67f014bc80 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:00.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.876+0000 7f67f5600640 1 -- 172.21.15.119:0/2753224451 >> 172.21.15.119:0/2753224451 conn(0x7f67f006aba0 msgr2=0x7f67f00cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:00.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.876+0000 7f67f5600640 1 -- 172.21.15.119:0/2753224451 shutdown_connections 2024-04-25T10:10:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:00.876+0000 7f67f5600640 1 -- 172.21.15.119:0/2753224451 wait complete. 2024-04-25T10:10:00.900 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:10:01.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.092+0000 7f420c400640 1 Processor -- start 2024-04-25T10:10:01.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.092+0000 7f420c400640 1 -- start start 2024-04-25T10:10:01.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.094+0000 7f420c400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f42040d3b90 0x7f42040d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:01.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.094+0000 7f420c400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42040d0f60 0x7f42040b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:01.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.094+0000 7f420c400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42040d4090 con 0x7f420415dca0 2024-04-25T10:10:01.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.094+0000 7f420c400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42040b6b20 con 0x7f42040d3b90 2024-04-25T10:10:01.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.094+0000 7f420c400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42040d1390 con 0x7f42040d0f60 2024-04-25T10:10:01.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.094+0000 7f4209a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f420415dca0 0x7f42040ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51426/0 (socket says 172.21.15.119:51426) 2024-04-25T10:10:01.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.094+0000 7f4209a00640 1 -- 172.21.15.119:0/876305907 learned_addr learned my addr 172.21.15.119:0/876305907 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:10:01.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.094+0000 7f4209000640 1 --2- 172.21.15.119:0/876305907 >> v2:172.21.15.119:3300/0 conn(0x7f42040d3b90 0x7f42040d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:01.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.094+0000 7f4209000640 1 -- 172.21.15.119:0/876305907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42040d0f60 msgr2=0x7f42040b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:01.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.094+0000 7f4209000640 1 --2- 172.21.15.119:0/876305907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42040d0f60 0x7f42040b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.094+0000 7f4209000640 1 -- 172.21.15.119:0/876305907 >> v1:172.21.15.119:6789/0 conn(0x7f420415dca0 legacy=0x7f42040ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:01.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.094+0000 7f4203e00640 1 -- 172.21.15.119:0/876305907 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929227731 0 0) 0x7f42040d4090 con 0x7f420415dca0 2024-04-25T10:10:01.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.095+0000 7f4209000640 1 -- 172.21.15.119:0/876305907 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42040d4370 con 0x7f42040d3b90 2024-04-25T10:10:01.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.095+0000 7f4209000640 1 --2- 172.21.15.119:0/876305907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42040d3b90 0x7f42040d0a20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f41f00097d0 tx=0x7f41f002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f021ea52c4bfe36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:01.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.096+0000 7f4203e00640 1 -- 172.21.15.119:0/876305907 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0037070 con 0x7f42040d3b90 2024-04-25T10:10:01.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.096+0000 7f4203e00640 1 -- 172.21.15.119:0/876305907 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f0032440 con 0x7f42040d3b90 2024-04-25T10:10:01.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.096+0000 7f4203e00640 1 -- 172.21.15.119:0/876305907 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0032810 con 0x7f42040d3b90 2024-04-25T10:10:01.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.096+0000 7f420c400640 1 -- 172.21.15.119:0/876305907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42040d3b90 msgr2=0x7f42040d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:01.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.096+0000 7f420c400640 1 --2- 172.21.15.119:0/876305907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42040d3b90 0x7f42040d0a20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f41f00097d0 tx=0x7f41f002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.096+0000 7f420c400640 1 -- 172.21.15.119:0/876305907 shutdown_connections 2024-04-25T10:10:01.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.096+0000 7f420c400640 1 --2- 172.21.15.119:0/876305907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42040d0f60 0x7f42040b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.096+0000 7f420c400640 1 --2- 172.21.15.119:0/876305907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42040d3b90 0x7f42040d0a20 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.097+0000 7f420c400640 1 -- 172.21.15.119:0/876305907 >> 172.21.15.119:0/876305907 conn(0x7f42040726c0 msgr2=0x7f4204072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:01.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.097+0000 7f420c400640 1 -- 172.21.15.119:0/876305907 shutdown_connections 2024-04-25T10:10:01.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.097+0000 7f420c400640 1 -- 172.21.15.119:0/876305907 wait complete. 2024-04-25T10:10:01.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.098+0000 7f420c400640 1 Processor -- start 2024-04-25T10:10:01.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.098+0000 7f420c400640 1 -- start start 2024-04-25T10:10:01.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.099+0000 7f420c400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42040d0f60 0x7f42041526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:01.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.099+0000 7f420c400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42040d3b90 0x7f420415f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:01.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.099+0000 7f420c400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f420415dca0 0x7f4204160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:01.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.099+0000 7f420c400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42040cf050 con 0x7f42040d0f60 2024-04-25T10:10:01.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.099+0000 7f420c400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42040b6d50 con 0x7f420415dca0 2024-04-25T10:10:01.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.099+0000 7f420c400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420415e080 con 0x7f42040d3b90 2024-04-25T10:10:01.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.100+0000 7f4209a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42040d0f60 0x7f42041526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:01.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.100+0000 7f4209a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42040d0f60 0x7f42041526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39162/0 (socket says 172.21.15.119:39162) 2024-04-25T10:10:01.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.100+0000 7f4209a00640 1 -- 172.21.15.119:0/1433634418 learned_addr learned my addr 172.21.15.119:0/1433634418 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:01.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.100+0000 7f4209a00640 1 -- 172.21.15.119:0/1433634418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42040d3b90 msgr2=0x7f420415f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:01.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.100+0000 7f4209a00640 1 --2- 172.21.15.119:0/1433634418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42040d3b90 0x7f420415f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.100+0000 7f4209a00640 1 -- 172.21.15.119:0/1433634418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f420415dca0 msgr2=0x7f4204160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:01.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.100+0000 7f4209a00640 1 --2- 172.21.15.119:0/1433634418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f420415dca0 0x7f4204160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.100+0000 7f4209a00640 1 -- 172.21.15.119:0/1433634418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420415e4a0 con 0x7f42040d0f60 2024-04-25T10:10:01.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.100+0000 7f4209a00640 1 --2- 172.21.15.119:0/1433634418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42040d0f60 0x7f42041526f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f41f400c430 tx=0x7f41f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:01.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.100+0000 7f4201600640 1 -- 172.21.15.119:0/1433634418 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f40074e0 con 0x7f42040d0f60 2024-04-25T10:10:01.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.100+0000 7f4201600640 1 -- 172.21.15.119:0/1433634418 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f40047e0 con 0x7f42040d0f60 2024-04-25T10:10:01.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.100+0000 7f4201600640 1 -- 172.21.15.119:0/1433634418 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4004bd0 con 0x7f42040d0f60 2024-04-25T10:10:01.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.101+0000 7f420c400640 1 -- 172.21.15.119:0/1433634418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420415e780 con 0x7f42040d0f60 2024-04-25T10:10:01.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.101+0000 7f420c400640 1 -- 172.21.15.119:0/1433634418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420415ed40 con 0x7f42040d0f60 2024-04-25T10:10:01.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.102+0000 7f420c400640 1 -- 172.21.15.119:0/1433634418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42040b4800 con 0x7f42040d0f60 2024-04-25T10:10:01.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.103+0000 7f4201600640 1 -- 172.21.15.119:0/1433634418 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41f4011070 con 0x7f42040d0f60 2024-04-25T10:10:01.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.104+0000 7f4201600640 1 --2- 172.21.15.119:0/1433634418 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41e406b4d0 0x7f41e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:01.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.104+0000 7f4201600640 1 -- 172.21.15.119:0/1433634418 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41f4089f50 con 0x7f42040d0f60 2024-04-25T10:10:01.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.104+0000 7f4209000640 1 --2- 172.21.15.119:0/1433634418 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41e406b4d0 0x7f41e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:01.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.105+0000 7f4209000640 1 --2- 172.21.15.119:0/1433634418 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41e406b4d0 0x7f41e406d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f41f0034000 tx=0x7f41f002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:01.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.108+0000 7f4201600640 1 -- 172.21.15.119:0/1433634418 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42040b4800 con 0x7f42040d0f60 2024-04-25T10:10:01.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:01.327+0000 7f420c400640 1 -- 172.21.15.119:0/1433634418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f42040750b0 con 0x7f42040d0f60 2024-04-25T10:10:02.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.049+0000 7f4201600640 1 -- 172.21.15.119:0/1433634418 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7f41f4058d40 con 0x7f42040d0f60 2024-04-25T10:10:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.053+0000 7f420c400640 1 -- 172.21.15.119:0/1433634418 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41e406b4d0 msgr2=0x7f41e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.053+0000 7f420c400640 1 --2- 172.21.15.119:0/1433634418 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41e406b4d0 0x7f41e406d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f41f0034000 tx=0x7f41f002c000 comp rx=0 tx=0).stop 2024-04-25T10:10:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.053+0000 7f420c400640 1 -- 172.21.15.119:0/1433634418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42040d0f60 msgr2=0x7f42041526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.053+0000 7f420c400640 1 --2- 172.21.15.119:0/1433634418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42040d0f60 0x7f42041526f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f41f400c430 tx=0x7f41f400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.053+0000 7f420c400640 1 -- 172.21.15.119:0/1433634418 shutdown_connections 2024-04-25T10:10:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.053+0000 7f420c400640 1 --2- 172.21.15.119:0/1433634418 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41e406b4d0 0x7f41e406d990 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.053+0000 7f420c400640 1 --2- 172.21.15.119:0/1433634418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f420415dca0 0x7f4204160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.053+0000 7f420c400640 1 --2- 172.21.15.119:0/1433634418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42040d3b90 0x7f420415f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.053+0000 7f420c400640 1 --2- 172.21.15.119:0/1433634418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42040d0f60 0x7f42041526f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.053+0000 7f420c400640 1 -- 172.21.15.119:0/1433634418 >> 172.21.15.119:0/1433634418 conn(0x7f42040726c0 msgr2=0x7f42040b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.053+0000 7f420c400640 1 -- 172.21.15.119:0/1433634418 shutdown_connections 2024-04-25T10:10:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.053+0000 7f420c400640 1 -- 172.21.15.119:0/1433634418 wait complete. 2024-04-25T10:10:02.081 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:10:02.081 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:10:02.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.271+0000 7f3db7e00640 1 Processor -- start 2024-04-25T10:10:02.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.271+0000 7f3db7e00640 1 -- start start 2024-04-25T10:10:02.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.273+0000 7f3db7e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3db80ce860 0x7f3db80cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.273+0000 7f3db7e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db80b5710 con 0x7f3db815b7e0 2024-04-25T10:10:02.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.273+0000 7f3db7e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db80b59f0 con 0x7f3db80d00a0 2024-04-25T10:10:02.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.273+0000 7f3db7e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db80b5cd0 con 0x7f3db80ce860 2024-04-25T10:10:02.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.273+0000 7f3db6a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3db815b7e0 0x7f3db815e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51436/0 (socket says 172.21.15.119:51436) 2024-04-25T10:10:02.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.273+0000 7f3db6a00640 1 -- 172.21.15.119:0/1762531884 learned_addr learned my addr 172.21.15.119:0/1762531884 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:10:02.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 -- 172.21.15.119:0/1762531884 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453184617 0 0) 0x7f3db80b5710 con 0x7f3db815b7e0 2024-04-25T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 -- 172.21.15.119:0/1762531884 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d9c003520 con 0x7f3db815b7e0 2024-04-25T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3dae000640 1 --2- 172.21.15.119:0/1762531884 >> v2:172.21.15.132:3300/0 conn(0x7f3db80ce860 0x7f3db80cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 -- 172.21.15.119:0/1762531884 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 345795636 0 0) 0x7f3d9c003520 con 0x7f3db815b7e0 2024-04-25T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 -- 172.21.15.119:0/1762531884 >> v2:172.21.15.132:3300/0 conn(0x7f3db80ce860 msgr2=0x7f3db80cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 --2- 172.21.15.119:0/1762531884 >> v2:172.21.15.132:3300/0 conn(0x7f3db80ce860 0x7f3db80cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 -- 172.21.15.119:0/1762531884 >> v1:172.21.15.132:6789/0 conn(0x7f3db80d00a0 legacy=0x7f3db80d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 -- 172.21.15.119:0/1762531884 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db80b5fb0 con 0x7f3db815b7e0 2024-04-25T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 -- 172.21.15.119:0/1762531884 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f3da80019f0 con 0x7f3db815b7e0 2024-04-25T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 -- 172.21.15.119:0/1762531884 >> v1:172.21.15.119:6789/0 conn(0x7f3db815b7e0 legacy=0x7f3db815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 --2- 172.21.15.119:0/1762531884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d9c003a00 0x7f3d9c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 --2- 172.21.15.119:0/1762531884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d9c004800 0x7f3d9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 --2- 172.21.15.119:0/1762531884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d9c0053c0 0x7f3d9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 -- 172.21.15.119:0/1762531884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80b5cd0 con 0x7f3d9c003a00 2024-04-25T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 -- 172.21.15.119:0/1762531884 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c00c2a0 con 0x7f3d9c0053c0 2024-04-25T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.274+0000 7f3db6000640 1 -- 172.21.15.119:0/1762531884 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c00c4a0 con 0x7f3d9c004800 2024-04-25T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.275+0000 7f3db6a00640 1 --2- 172.21.15.119:0/1762531884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d9c003a00 0x7f3d9c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.275+0000 7f3db7400640 1 --2- 172.21.15.119:0/1762531884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d9c0053c0 0x7f3d9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.275+0000 7f3db6a00640 1 -- 172.21.15.119:0/1762531884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d9c004800 msgr2=0x7f3d9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.275+0000 7f3dae000640 1 --2- 172.21.15.119:0/1762531884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d9c004800 0x7f3d9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.275+0000 7f3db6a00640 1 --2- 172.21.15.119:0/1762531884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d9c004800 0x7f3d9c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.275+0000 7f3db6a00640 1 -- 172.21.15.119:0/1762531884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d9c0053c0 msgr2=0x7f3d9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.275+0000 7f3db6a00640 1 --2- 172.21.15.119:0/1762531884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d9c0053c0 0x7f3d9c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.275+0000 7f3db6a00640 1 -- 172.21.15.119:0/1762531884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c003520 con 0x7f3d9c003a00 2024-04-25T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.275+0000 7f3db7400640 1 --2- 172.21.15.119:0/1762531884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d9c0053c0 0x7f3d9c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:02.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.275+0000 7f3db7400640 1 -- 172.21.15.119:0/1762531884 reap_dead start 2024-04-25T10:10:02.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.276+0000 7f3db6a00640 1 --2- 172.21.15.119:0/1762531884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d9c003a00 0x7f3d9c003e20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3da800a7b0 tx=0x7f3d9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3625d65d83a28a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:02.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.276+0000 7f3db6000640 1 -- 172.21.15.119:0/1762531884 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da800a3d0 con 0x7f3d9c003a00 2024-04-25T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.276+0000 7f3db6000640 1 -- 172.21.15.119:0/1762531884 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da802f070 con 0x7f3d9c003a00 2024-04-25T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.276+0000 7f3db6000640 1 -- 172.21.15.119:0/1762531884 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da802bc80 con 0x7f3d9c003a00 2024-04-25T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.277+0000 7f3db7e00640 1 -- 172.21.15.119:0/1762531884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d9c003a00 msgr2=0x7f3d9c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.277+0000 7f3db7e00640 1 --2- 172.21.15.119:0/1762531884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d9c003a00 0x7f3d9c003e20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3da800a7b0 tx=0x7f3d9c005830 comp rx=0 tx=0).stop 2024-04-25T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.277+0000 7f3db7e00640 1 -- 172.21.15.119:0/1762531884 shutdown_connections 2024-04-25T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.277+0000 7f3db7e00640 1 --2- 172.21.15.119:0/1762531884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d9c003a00 0x7f3d9c003e20 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.277+0000 7f3db7e00640 1 -- 172.21.15.119:0/1762531884 >> 172.21.15.119:0/1762531884 conn(0x7f3db80722d0 msgr2=0x7f3db80726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.277+0000 7f3db7e00640 1 -- 172.21.15.119:0/1762531884 shutdown_connections 2024-04-25T10:10:02.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.277+0000 7f3db7e00640 1 -- 172.21.15.119:0/1762531884 wait complete. 2024-04-25T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.279+0000 7f3db7e00640 1 Processor -- start 2024-04-25T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.279+0000 7f3db7e00640 1 -- start start 2024-04-25T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.280+0000 7f3db7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d9c003a00 0x7f3db8148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.280+0000 7f3db7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db80caa10 0x7f3db80cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.280+0000 7f3db7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db80cd5b0 0x7f3db80d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.280+0000 7f3db7e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80cee70 con 0x7f3db80caa10 2024-04-25T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.280+0000 7f3db7e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80cecf0 con 0x7f3db80cd5b0 2024-04-25T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.280+0000 7f3db7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80ceff0 con 0x7f3d9c003a00 2024-04-25T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.280+0000 7f3dae000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db80caa10 0x7f3db80cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.281+0000 7f3dae000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db80caa10 0x7f3db80cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39178/0 (socket says 172.21.15.119:39178) 2024-04-25T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.281+0000 7f3dae000640 1 -- 172.21.15.119:0/195945413 learned_addr learned my addr 172.21.15.119:0/195945413 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:02.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.281+0000 7f3db6a00640 1 --2- 172.21.15.119:0/195945413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d9c003a00 0x7f3db8148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.281+0000 7f3db7400640 1 --2- 172.21.15.119:0/195945413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db80cd5b0 0x7f3db80d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.281+0000 7f3dae000640 1 -- 172.21.15.119:0/195945413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d9c003a00 msgr2=0x7f3db8148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.281+0000 7f3dae000640 1 --2- 172.21.15.119:0/195945413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d9c003a00 0x7f3db8148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.281+0000 7f3dae000640 1 -- 172.21.15.119:0/195945413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db80cd5b0 msgr2=0x7f3db80d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.281+0000 7f3dae000640 1 --2- 172.21.15.119:0/195945413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db80cd5b0 0x7f3db80d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.281+0000 7f3dae000640 1 -- 172.21.15.119:0/195945413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db80caea0 con 0x7f3db80caa10 2024-04-25T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.281+0000 7f3dae000640 1 --2- 172.21.15.119:0/195945413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db80caa10 0x7f3db80cd070 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3da000e7e0 tx=0x7f3da000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.282+0000 7f3daf400640 1 -- 172.21.15.119:0/195945413 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da0017070 con 0x7f3db80caa10 2024-04-25T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.282+0000 7f3daf400640 1 -- 172.21.15.119:0/195945413 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da000c6d0 con 0x7f3db80caa10 2024-04-25T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.282+0000 7f3db7e00640 1 -- 172.21.15.119:0/195945413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db80cb180 con 0x7f3db80caa10 2024-04-25T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.282+0000 7f3daf400640 1 -- 172.21.15.119:0/195945413 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da000f040 con 0x7f3db80caa10 2024-04-25T10:10:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.282+0000 7f3db7e00640 1 -- 172.21.15.119:0/195945413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db80cb740 con 0x7f3db80caa10 2024-04-25T10:10:02.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.283+0000 7f3db7e00640 1 -- 172.21.15.119:0/195945413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db80749b0 con 0x7f3db80caa10 2024-04-25T10:10:02.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.284+0000 7f3daf400640 1 -- 172.21.15.119:0/195945413 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3da001a070 con 0x7f3db80caa10 2024-04-25T10:10:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.285+0000 7f3daf400640 1 --2- 172.21.15.119:0/195945413 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d9406b4d0 0x7f3d9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.285+0000 7f3daf400640 1 -- 172.21.15.119:0/195945413 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3da0089df0 con 0x7f3db80caa10 2024-04-25T10:10:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.285+0000 7f3db6a00640 1 --2- 172.21.15.119:0/195945413 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d9406b4d0 0x7f3d9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.286+0000 7f3db6a00640 1 --2- 172.21.15.119:0/195945413 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d9406b4d0 0x7f3d9406d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3da800a780 tx=0x7f3da8038040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.289+0000 7f3daf400640 1 -- 172.21.15.119:0/195945413 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3da0058bd0 con 0x7f3db80caa10 2024-04-25T10:10:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:02.505+0000 7f3db7e00640 1 -- 172.21.15.119:0/195945413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3db80d1650 con 0x7f3db80caa10 2024-04-25T10:10:03.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.060+0000 7f3daf400640 1 -- 172.21.15.119:0/195945413 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3da00584d0 con 0x7f3db80caa10 2024-04-25T10:10:03.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.063+0000 7f3db7e00640 1 -- 172.21.15.119:0/195945413 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d9406b4d0 msgr2=0x7f3d9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.063+0000 7f3db7e00640 1 --2- 172.21.15.119:0/195945413 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d9406b4d0 0x7f3d9406d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3da800a780 tx=0x7f3da8038040 comp rx=0 tx=0).stop 2024-04-25T10:10:03.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.063+0000 7f3db7e00640 1 -- 172.21.15.119:0/195945413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db80caa10 msgr2=0x7f3db80cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.063+0000 7f3db7e00640 1 --2- 172.21.15.119:0/195945413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db80caa10 0x7f3db80cd070 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3da000e7e0 tx=0x7f3da000eca0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.063+0000 7f3db7e00640 1 -- 172.21.15.119:0/195945413 shutdown_connections 2024-04-25T10:10:03.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.063+0000 7f3db7e00640 1 --2- 172.21.15.119:0/195945413 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d9406b4d0 0x7f3d9406d990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.063+0000 7f3db7e00640 1 --2- 172.21.15.119:0/195945413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db80cd5b0 0x7f3db80d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.063+0000 7f3db7e00640 1 --2- 172.21.15.119:0/195945413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db80caa10 0x7f3db80cd070 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.063+0000 7f3db7e00640 1 --2- 172.21.15.119:0/195945413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d9c003a00 0x7f3db8148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.063+0000 7f3db7e00640 1 -- 172.21.15.119:0/195945413 >> 172.21.15.119:0/195945413 conn(0x7f3db80722d0 msgr2=0x7f3db815f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:03.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.063+0000 7f3db7e00640 1 -- 172.21.15.119:0/195945413 shutdown_connections 2024-04-25T10:10:03.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.064+0000 7f3db7e00640 1 -- 172.21.15.119:0/195945413 wait complete. 2024-04-25T10:10:03.090 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:10:03.090 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:03.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.284+0000 7f3a0fc00640 1 Processor -- start 2024-04-25T10:10:03.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.285+0000 7f3a0fc00640 1 -- start start 2024-04-25T10:10:03.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.286+0000 7f3a0fc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a0815bd40 0x7f3a0809a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.286+0000 7f3a0fc00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a08099e20 0x7f3a080cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.286+0000 7f3a0fc00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a080cfb90 con 0x7f3a08099a40 2024-04-25T10:10:03.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.286+0000 7f3a0fc00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0815cfd0 con 0x7f3a08099e20 2024-04-25T10:10:03.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.286+0000 7f3a0fc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0815c210 con 0x7f3a0815bd40 2024-04-25T10:10:03.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.286+0000 7f3a0dc00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a08099e20 0x7f3a080cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.286+0000 7f3a0d200640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3a08099a40 0x7f3a080cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51452/0 (socket says 172.21.15.119:51452) 2024-04-25T10:10:03.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.286+0000 7f3a0d200640 1 -- 172.21.15.119:0/3176798367 learned_addr learned my addr 172.21.15.119:0/3176798367 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:10:03.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.287+0000 7f3a0dc00640 1 -- 172.21.15.119:0/3176798367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a0815bd40 msgr2=0x7f3a0809a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.287+0000 7f3a0dc00640 1 --2- 172.21.15.119:0/3176798367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a0815bd40 0x7f3a0809a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.287+0000 7f3a0dc00640 1 -- 172.21.15.119:0/3176798367 >> v1:172.21.15.119:6789/0 conn(0x7f3a08099a40 legacy=0x7f3a080cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.287+0000 7f3a0dc00640 1 -- 172.21.15.119:0/3176798367 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0815c4f0 con 0x7f3a08099e20 2024-04-25T10:10:03.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.287+0000 7f3a07400640 1 -- 172.21.15.119:0/3176798367 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288070318 0 0) 0x7f3a080cfb90 con 0x7f3a08099a40 2024-04-25T10:10:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.288+0000 7f3a0dc00640 1 --2- 172.21.15.119:0/3176798367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a08099e20 0x7f3a080cc3f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009d90 tx=0x7f3a0002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4a3e430953f25b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.288+0000 7f3a07400640 1 -- 172.21.15.119:0/3176798367 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00031600 con 0x7f3a08099e20 2024-04-25T10:10:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.288+0000 7f3a07400640 1 -- 172.21.15.119:0/3176798367 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a000317a0 con 0x7f3a08099e20 2024-04-25T10:10:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.288+0000 7f3a07400640 1 -- 172.21.15.119:0/3176798367 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00031b70 con 0x7f3a08099e20 2024-04-25T10:10:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.289+0000 7f3a0fc00640 1 -- 172.21.15.119:0/3176798367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a08099e20 msgr2=0x7f3a080cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.289+0000 7f3a0fc00640 1 --2- 172.21.15.119:0/3176798367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a08099e20 0x7f3a080cc3f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009d90 tx=0x7f3a0002a5d0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.289+0000 7f3a0fc00640 1 -- 172.21.15.119:0/3176798367 shutdown_connections 2024-04-25T10:10:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.289+0000 7f3a0fc00640 1 --2- 172.21.15.119:0/3176798367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a08099e20 0x7f3a080cc3f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.289+0000 7f3a0fc00640 1 --2- 172.21.15.119:0/3176798367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a0815bd40 0x7f3a0809a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.289+0000 7f3a0fc00640 1 -- 172.21.15.119:0/3176798367 >> 172.21.15.119:0/3176798367 conn(0x7f3a080b32f0 msgr2=0x7f3a080b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.289+0000 7f3a0fc00640 1 -- 172.21.15.119:0/3176798367 shutdown_connections 2024-04-25T10:10:03.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.289+0000 7f3a0fc00640 1 -- 172.21.15.119:0/3176798367 wait complete. 2024-04-25T10:10:03.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.291+0000 7f3a0fc00640 1 Processor -- start 2024-04-25T10:10:03.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.291+0000 7f3a0fc00640 1 -- start start 2024-04-25T10:10:03.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.292+0000 7f3a0fc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a08099a40 0x7f3a080c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.292+0000 7f3a0fc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a08099e20 0x7f3a0815e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.292+0000 7f3a0fc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0815bd40 0x7f3a0815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.292+0000 7f3a0fc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0809a2b0 con 0x7f3a0815bd40 2024-04-25T10:10:03.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.292+0000 7f3a0fc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0815cab0 con 0x7f3a08099e20 2024-04-25T10:10:03.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.292+0000 7f3a0fc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a080cc930 con 0x7f3a08099a40 2024-04-25T10:10:03.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.292+0000 7f3a0dc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0815bd40 0x7f3a0815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.292+0000 7f3a0dc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0815bd40 0x7f3a0815e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39202/0 (socket says 172.21.15.119:39202) 2024-04-25T10:10:03.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.292+0000 7f3a0dc00640 1 -- 172.21.15.119:0/765477334 learned_addr learned my addr 172.21.15.119:0/765477334 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:03.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.293+0000 7f3a0d200640 1 --2- 172.21.15.119:0/765477334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a08099a40 0x7f3a080c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.293+0000 7f3a0dc00640 1 -- 172.21.15.119:0/765477334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a08099a40 msgr2=0x7f3a080c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.293+0000 7f3a07e00640 1 --2- 172.21.15.119:0/765477334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a08099e20 0x7f3a0815e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.293+0000 7f3a0dc00640 1 --2- 172.21.15.119:0/765477334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a08099a40 0x7f3a080c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.293+0000 7f3a0dc00640 1 -- 172.21.15.119:0/765477334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a08099e20 msgr2=0x7f3a0815e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.293+0000 7f3a0dc00640 1 --2- 172.21.15.119:0/765477334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a08099e20 0x7f3a0815e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.293+0000 7f3a0dc00640 1 -- 172.21.15.119:0/765477334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a080c6800 con 0x7f3a0815bd40 2024-04-25T10:10:03.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.293+0000 7f3a0dc00640 1 --2- 172.21.15.119:0/765477334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0815bd40 0x7f3a0815e6f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009d90 tx=0x7f3a00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.293+0000 7f3a04c00640 1 -- 172.21.15.119:0/765477334 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00040070 con 0x7f3a0815bd40 2024-04-25T10:10:03.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.293+0000 7f3a04c00640 1 -- 172.21.15.119:0/765477334 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a00030b60 con 0x7f3a0815bd40 2024-04-25T10:10:03.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.293+0000 7f3a0fc00640 1 -- 172.21.15.119:0/765477334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a080c6aa0 con 0x7f3a0815bd40 2024-04-25T10:10:03.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.293+0000 7f3a04c00640 1 -- 172.21.15.119:0/765477334 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0003c420 con 0x7f3a0815bd40 2024-04-25T10:10:03.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.294+0000 7f3a0fc00640 1 -- 172.21.15.119:0/765477334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a080c7060 con 0x7f3a0815bd40 2024-04-25T10:10:03.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.295+0000 7f3a0fc00640 1 -- 172.21.15.119:0/765477334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a080749b0 con 0x7f3a0815bd40 2024-04-25T10:10:03.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.295+0000 7f3a04c00640 1 -- 172.21.15.119:0/765477334 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a00039040 con 0x7f3a0815bd40 2024-04-25T10:10:03.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.296+0000 7f3a04c00640 1 --2- 172.21.15.119:0/765477334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f39e406b4d0 0x7f39e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.296+0000 7f3a04c00640 1 -- 172.21.15.119:0/765477334 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a000ab730 con 0x7f3a0815bd40 2024-04-25T10:10:03.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.297+0000 7f3a0d200640 1 --2- 172.21.15.119:0/765477334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f39e406b4d0 0x7f39e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.297+0000 7f3a0d200640 1 --2- 172.21.15.119:0/765477334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f39e406b4d0 0x7f39e406d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f39f8007fd0 tx=0x7f39f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.301+0000 7f3a04c00640 1 -- 172.21.15.119:0/765477334 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a00079f80 con 0x7f3a0815bd40 2024-04-25T10:10:03.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.534+0000 7f3a0fc00640 1 -- 172.21.15.119:0/765477334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a0809a570 con 0x7f3a0815bd40 2024-04-25T10:10:03.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.536+0000 7f3a04c00640 1 -- 172.21.15.119:0/765477334 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f3a0007e020 con 0x7f3a0815bd40 2024-04-25T10:10:03.542 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:10:03.542 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4159222235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4159222235},{"type":"v1","addr":"172.21.15.132:6834","nonce":4159222235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4223361610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":4223361610},{"type":"v1","addr":"172.21.15.119:6837","nonce":4223361610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2367324442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2367324442},{"type":"v1","addr":"172.21.15.132:6835","nonce":2367324442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:59.454941+0000","modified":"2024-04-25T10:10:02.506906+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6836/4129112885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4129112885},{"type":"v1","addr":"172.21.15.119:6836","nonce":4129112885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:10:03.542 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 34 2024-04-25T10:10:03.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.538+0000 7f3a0fc00640 1 -- 172.21.15.119:0/765477334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f39e406b4d0 msgr2=0x7f39e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.538+0000 7f3a0fc00640 1 --2- 172.21.15.119:0/765477334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f39e406b4d0 0x7f39e406d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f39f8007fd0 tx=0x7f39f8009040 comp rx=0 tx=0).stop 2024-04-25T10:10:03.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.538+0000 7f3a0fc00640 1 -- 172.21.15.119:0/765477334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0815bd40 msgr2=0x7f3a0815e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.538+0000 7f3a0fc00640 1 --2- 172.21.15.119:0/765477334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0815bd40 0x7f3a0815e6f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009d90 tx=0x7f3a00004a40 comp rx=0 tx=0).stop 2024-04-25T10:10:03.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.538+0000 7f3a0fc00640 1 -- 172.21.15.119:0/765477334 shutdown_connections 2024-04-25T10:10:03.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.538+0000 7f3a0fc00640 1 --2- 172.21.15.119:0/765477334 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f39e406b4d0 0x7f39e406d990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.538+0000 7f3a0fc00640 1 --2- 172.21.15.119:0/765477334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a0815bd40 0x7f3a0815e6f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.538+0000 7f3a0fc00640 1 --2- 172.21.15.119:0/765477334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a08099e20 0x7f3a0815e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.538+0000 7f3a0fc00640 1 --2- 172.21.15.119:0/765477334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a08099a40 0x7f3a080c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.538+0000 7f3a0fc00640 1 -- 172.21.15.119:0/765477334 >> 172.21.15.119:0/765477334 conn(0x7f3a080b32f0 msgr2=0x7f3a080b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:03.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.538+0000 7f3a0fc00640 1 -- 172.21.15.119:0/765477334 shutdown_connections 2024-04-25T10:10:03.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.538+0000 7f3a0fc00640 1 -- 172.21.15.119:0/765477334 wait complete. 2024-04-25T10:10:03.563 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:03.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.738+0000 7f525dc00640 1 Processor -- start 2024-04-25T10:10:03.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.739+0000 7f525dc00640 1 -- start start 2024-04-25T10:10:03.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.740+0000 7f525dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5258144040 0x7f52580cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.740+0000 7f525dc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f525815acd0 0x7f525806e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.740+0000 7f525dc00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f52580cce40 0x7f525806e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.740+0000 7f525dc00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5258144420 con 0x7f52580cce40 2024-04-25T10:10:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.740+0000 7f525dc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52580b3d80 con 0x7f525815acd0 2024-04-25T10:10:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.740+0000 7f525dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52580b4060 con 0x7f5258144040 2024-04-25T10:10:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.741+0000 7f525d200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f52580cce40 0x7f525806e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.741+0000 7f525d200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f52580cce40 0x7f525806e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39204/0 (socket says 172.21.15.119:39204) 2024-04-25T10:10:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.741+0000 7f525d200640 1 -- 172.21.15.119:0/2701088039 learned_addr learned my addr 172.21.15.119:0/2701088039 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:03.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.741+0000 7f524fe00640 1 --2- 172.21.15.119:0/2701088039 >> v2:172.21.15.132:3300/0 conn(0x7f525815acd0 0x7f525806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.741+0000 7f5257e00640 1 --2- 172.21.15.119:0/2701088039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5258144040 0x7f52580cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.741+0000 7f525d200640 1 -- 172.21.15.119:0/2701088039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5258144040 msgr2=0x7f52580cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.741+0000 7f525d200640 1 --2- 172.21.15.119:0/2701088039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5258144040 0x7f52580cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.741+0000 7f525d200640 1 -- 172.21.15.119:0/2701088039 >> v2:172.21.15.132:3300/0 conn(0x7f525815acd0 msgr2=0x7f525806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.741+0000 7f525d200640 1 --2- 172.21.15.119:0/2701088039 >> v2:172.21.15.132:3300/0 conn(0x7f525815acd0 0x7f525806e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.741+0000 7f525d200640 1 -- 172.21.15.119:0/2701088039 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52580b4340 con 0x7f52580cce40 2024-04-25T10:10:03.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.742+0000 7f525d200640 1 --2- 172.21.15.119:0/2701088039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52580cce40 0x7f525806e860 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f525806ef60 tx=0x7f525002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec3d3ad99ed9b82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.743+0000 7f5257400640 1 -- 172.21.15.119:0/2701088039 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250036070 con 0x7f52580cce40 2024-04-25T10:10:03.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.743+0000 7f5257400640 1 -- 172.21.15.119:0/2701088039 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5250031600 con 0x7f52580cce40 2024-04-25T10:10:03.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.743+0000 7f5257400640 1 -- 172.21.15.119:0/2701088039 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52500319d0 con 0x7f52580cce40 2024-04-25T10:10:03.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.743+0000 7f525dc00640 1 -- 172.21.15.119:0/2701088039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52580cce40 msgr2=0x7f525806e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.743+0000 7f525dc00640 1 --2- 172.21.15.119:0/2701088039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52580cce40 0x7f525806e860 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f525806ef60 tx=0x7f525002a610 comp rx=0 tx=0).stop 2024-04-25T10:10:03.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.743+0000 7f525dc00640 1 -- 172.21.15.119:0/2701088039 shutdown_connections 2024-04-25T10:10:03.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.743+0000 7f525dc00640 1 --2- 172.21.15.119:0/2701088039 >> v2:172.21.15.132:3300/0 conn(0x7f525815acd0 0x7f525806e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.743+0000 7f525dc00640 1 --2- 172.21.15.119:0/2701088039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52580cce40 0x7f525806e860 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.743+0000 7f525dc00640 1 --2- 172.21.15.119:0/2701088039 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5258144040 0x7f52580cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.743+0000 7f525dc00640 1 -- 172.21.15.119:0/2701088039 >> 172.21.15.119:0/2701088039 conn(0x7f525806b1d0 msgr2=0x7f52580b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:03.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.743+0000 7f525dc00640 1 -- 172.21.15.119:0/2701088039 shutdown_connections 2024-04-25T10:10:03.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.743+0000 7f525dc00640 1 -- 172.21.15.119:0/2701088039 wait complete. 2024-04-25T10:10:03.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.745+0000 7f525dc00640 1 Processor -- start 2024-04-25T10:10:03.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.745+0000 7f525dc00640 1 -- start start 2024-04-25T10:10:03.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.746+0000 7f525dc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52580cce40 0x7f52581ee970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.746+0000 7f525dc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5258144040 0x7f525815cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.746+0000 7f525dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f525815acd0 0x7f525815d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.746+0000 7f525dc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52580b5180 con 0x7f52580cce40 2024-04-25T10:10:03.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.746+0000 7f525dc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52580b4900 con 0x7f5258144040 2024-04-25T10:10:03.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.746+0000 7f525dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52580b5300 con 0x7f525815acd0 2024-04-25T10:10:03.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.746+0000 7f5257e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52580cce40 0x7f52581ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.746+0000 7f5257e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52580cce40 0x7f52581ee970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39206/0 (socket says 172.21.15.119:39206) 2024-04-25T10:10:03.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.746+0000 7f5257e00640 1 -- 172.21.15.119:0/1495740022 learned_addr learned my addr 172.21.15.119:0/1495740022 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:03.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.747+0000 7f5257e00640 1 -- 172.21.15.119:0/1495740022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f525815acd0 msgr2=0x7f525815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.747+0000 7f5257e00640 1 --2- 172.21.15.119:0/1495740022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f525815acd0 0x7f525815d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.747+0000 7f5257e00640 1 -- 172.21.15.119:0/1495740022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5258144040 msgr2=0x7f525815cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.747+0000 7f5257e00640 1 --2- 172.21.15.119:0/1495740022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5258144040 0x7f525815cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.747+0000 7f5257e00640 1 -- 172.21.15.119:0/1495740022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52581eeeb0 con 0x7f52580cce40 2024-04-25T10:10:03.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.747+0000 7f5257e00640 1 --2- 172.21.15.119:0/1495740022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52580cce40 0x7f52581ee970 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f524800c7f0 tx=0x7f524800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.747+0000 7f5254c00640 1 -- 172.21.15.119:0/1495740022 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248015070 con 0x7f52580cce40 2024-04-25T10:10:03.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.747+0000 7f5254c00640 1 -- 172.21.15.119:0/1495740022 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5248004220 con 0x7f52580cce40 2024-04-25T10:10:03.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.747+0000 7f525dc00640 1 -- 172.21.15.119:0/1495740022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525813e090 con 0x7f52580cce40 2024-04-25T10:10:03.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.747+0000 7f5254c00640 1 -- 172.21.15.119:0/1495740022 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248007620 con 0x7f52580cce40 2024-04-25T10:10:03.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.747+0000 7f525dc00640 1 -- 172.21.15.119:0/1495740022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525813e650 con 0x7f52580cce40 2024-04-25T10:10:03.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.748+0000 7f525dc00640 1 -- 172.21.15.119:0/1495740022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525806bb30 con 0x7f52580cce40 2024-04-25T10:10:03.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.749+0000 7f5254c00640 1 -- 172.21.15.119:0/1495740022 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f52480079d0 con 0x7f52580cce40 2024-04-25T10:10:03.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.750+0000 7f5254c00640 1 --2- 172.21.15.119:0/1495740022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f523406b4d0 0x7f523406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.750+0000 7f5254c00640 1 -- 172.21.15.119:0/1495740022 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f524808a5c0 con 0x7f52580cce40 2024-04-25T10:10:03.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.750+0000 7f524fe00640 1 --2- 172.21.15.119:0/1495740022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f523406b4d0 0x7f523406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.751+0000 7f524fe00640 1 --2- 172.21.15.119:0/1495740022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f523406b4d0 0x7f523406d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5240006b10 tx=0x7f5240008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.754+0000 7f5254c00640 1 -- 172.21.15.119:0/1495740022 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f52480593e0 con 0x7f52580cce40 2024-04-25T10:10:03.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.948+0000 7f525dc00640 1 -- 172.21.15.119:0/1495740022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f525813e850 con 0x7f52580cce40 2024-04-25T10:10:03.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.950+0000 7f5254c00640 1 -- 172.21.15.119:0/1495740022 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16701 (secure 0 0 0) 0x7f5248058ce0 con 0x7f52580cce40 2024-04-25T10:10:03.956 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:10:03.957 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":55,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:10:01.035989+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:57.237546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:58.475727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:03.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.952+0000 7f525dc00640 1 -- 172.21.15.119:0/1495740022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f523406b4d0 msgr2=0x7f523406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.952+0000 7f525dc00640 1 --2- 172.21.15.119:0/1495740022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f523406b4d0 0x7f523406d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5240006b10 tx=0x7f5240008040 comp rx=0 tx=0).stop 2024-04-25T10:10:03.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.952+0000 7f525dc00640 1 -- 172.21.15.119:0/1495740022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52580cce40 msgr2=0x7f52581ee970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.952+0000 7f525dc00640 1 --2- 172.21.15.119:0/1495740022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52580cce40 0x7f52581ee970 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f524800c7f0 tx=0x7f524800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.953+0000 7f525dc00640 1 -- 172.21.15.119:0/1495740022 shutdown_connections 2024-04-25T10:10:03.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.953+0000 7f525dc00640 1 --2- 172.21.15.119:0/1495740022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f523406b4d0 0x7f523406d990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.953+0000 7f525dc00640 1 --2- 172.21.15.119:0/1495740022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f525815acd0 0x7f525815d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.953+0000 7f525dc00640 1 --2- 172.21.15.119:0/1495740022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5258144040 0x7f525815cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.953+0000 7f525dc00640 1 --2- 172.21.15.119:0/1495740022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52580cce40 0x7f52581ee970 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.953+0000 7f525dc00640 1 -- 172.21.15.119:0/1495740022 >> 172.21.15.119:0/1495740022 conn(0x7f525806b1d0 msgr2=0x7f52580b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:03.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.953+0000 7f525dc00640 1 -- 172.21.15.119:0/1495740022 shutdown_connections 2024-04-25T10:10:03.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:03.953+0000 7f525dc00640 1 -- 172.21.15.119:0/1495740022 wait complete. 2024-04-25T10:10:03.979 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:10:03.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:10:04.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.173+0000 7f9116000640 1 Processor -- start 2024-04-25T10:10:04.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.173+0000 7f9116000640 1 -- start start 2024-04-25T10:10:04.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.175+0000 7f9116000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f911006e8f0 0x7f911015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.175+0000 7f9116000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f911015c820 0x7f91101654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.175+0000 7f9116000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91100ce6c0 con 0x7f911015bf00 2024-04-25T10:10:04.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.175+0000 7f9116000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91101455e0 con 0x7f911006e8f0 2024-04-25T10:10:04.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.175+0000 7f9116000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911006d450 con 0x7f911015c820 2024-04-25T10:10:04.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.175+0000 7f910ea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f911006e8f0 0x7f911015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.175+0000 7f910ea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f911006e8f0 0x7f911015c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39220/0 (socket says 172.21.15.119:39220) 2024-04-25T10:10:04.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.175+0000 7f910ea00640 1 -- 172.21.15.119:0/960328401 learned_addr learned my addr 172.21.15.119:0/960328401 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:04.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.175+0000 7f910ea00640 1 -- 172.21.15.119:0/960328401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f911015c820 msgr2=0x7f91101654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:04.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.175+0000 7f910ea00640 1 --2- 172.21.15.119:0/960328401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f911015c820 0x7f91101654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.175+0000 7f910ea00640 1 -- 172.21.15.119:0/960328401 >> v1:172.21.15.119:6789/0 conn(0x7f911015bf00 legacy=0x7f91100cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.175+0000 7f910ea00640 1 -- 172.21.15.119:0/960328401 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100cc4e0 con 0x7f911006e8f0 2024-04-25T10:10:04.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.176+0000 7f910ea00640 1 --2- 172.21.15.119:0/960328401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911006e8f0 0x7f911015c2e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009870 tx=0x7f90fc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3ff93231044659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.177+0000 7f910e000640 1 -- 172.21.15.119:0/960328401 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc037070 con 0x7f911006e8f0 2024-04-25T10:10:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.177+0000 7f910e000640 1 -- 172.21.15.119:0/960328401 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc032440 con 0x7f911006e8f0 2024-04-25T10:10:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.177+0000 7f910e000640 1 -- 172.21.15.119:0/960328401 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc032810 con 0x7f911006e8f0 2024-04-25T10:10:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.177+0000 7f9116000640 1 -- 172.21.15.119:0/960328401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911006e8f0 msgr2=0x7f911015c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.177+0000 7f9116000640 1 --2- 172.21.15.119:0/960328401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911006e8f0 0x7f911015c2e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009870 tx=0x7f90fc02a250 comp rx=0 tx=0).stop 2024-04-25T10:10:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.178+0000 7f9116000640 1 -- 172.21.15.119:0/960328401 shutdown_connections 2024-04-25T10:10:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.178+0000 7f9116000640 1 --2- 172.21.15.119:0/960328401 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f911015c820 0x7f91101654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.178+0000 7f9116000640 1 --2- 172.21.15.119:0/960328401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911006e8f0 0x7f911015c2e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.178+0000 7f9116000640 1 -- 172.21.15.119:0/960328401 >> 172.21.15.119:0/960328401 conn(0x7f911006aba0 msgr2=0x7f911006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.178+0000 7f9116000640 1 -- 172.21.15.119:0/960328401 shutdown_connections 2024-04-25T10:10:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.178+0000 7f9116000640 1 -- 172.21.15.119:0/960328401 wait complete. 2024-04-25T10:10:04.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.179+0000 7f9116000640 1 Processor -- start 2024-04-25T10:10:04.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.179+0000 7f9116000640 1 -- start start 2024-04-25T10:10:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.180+0000 7f9116000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f911006e8f0 0x7f911013efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.180+0000 7f9116000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f911015bf00 0x7f911014fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.180+0000 7f9116000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911015c820 0x7f911013a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.180+0000 7f9116000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100ccd70 con 0x7f911015c820 2024-04-25T10:10:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.180+0000 7f9116000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100cca90 con 0x7f911015bf00 2024-04-25T10:10:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.180+0000 7f9116000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91100cd050 con 0x7f911006e8f0 2024-04-25T10:10:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.181+0000 7f910fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911015c820 0x7f911013a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.181+0000 7f910fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911015c820 0x7f911013a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39222/0 (socket says 172.21.15.119:39222) 2024-04-25T10:10:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.181+0000 7f910fe00640 1 -- 172.21.15.119:0/1157466898 learned_addr learned my addr 172.21.15.119:0/1157466898 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.181+0000 7f910f400640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f911006e8f0 0x7f911013efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.181+0000 7f910ea00640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f911015bf00 0x7f911014fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.181+0000 7f910fe00640 1 -- 172.21.15.119:0/1157466898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f911006e8f0 msgr2=0x7f911013efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.181+0000 7f910fe00640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f911006e8f0 0x7f911013efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.181+0000 7f910fe00640 1 -- 172.21.15.119:0/1157466898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f911015bf00 msgr2=0x7f911014fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.181+0000 7f910fe00640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f911015bf00 0x7f911014fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.181+0000 7f910fe00640 1 -- 172.21.15.119:0/1157466898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9110139110 con 0x7f911015c820 2024-04-25T10:10:04.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.181+0000 7f910f400640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f911006e8f0 0x7f911013efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:04.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.182+0000 7f910fe00640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911015c820 0x7f911013a1b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f910800b970 tx=0x7f910800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.182+0000 7f9107400640 1 -- 172.21.15.119:0/1157466898 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108019070 con 0x7f911015c820 2024-04-25T10:10:04.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.182+0000 7f9107400640 1 -- 172.21.15.119:0/1157466898 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910800cb60 con 0x7f911015c820 2024-04-25T10:10:04.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.182+0000 7f9107400640 1 -- 172.21.15.119:0/1157466898 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108014440 con 0x7f911015c820 2024-04-25T10:10:04.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.182+0000 7f9116000640 1 -- 172.21.15.119:0/1157466898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91101393f0 con 0x7f911015c820 2024-04-25T10:10:04.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.182+0000 7f9116000640 1 -- 172.21.15.119:0/1157466898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91101399b0 con 0x7f911015c820 2024-04-25T10:10:04.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.183+0000 7f9116000640 1 -- 172.21.15.119:0/1157466898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911006bb30 con 0x7f911015c820 2024-04-25T10:10:04.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.184+0000 7f9107400640 1 -- 172.21.15.119:0/1157466898 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f910801c070 con 0x7f911015c820 2024-04-25T10:10:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.185+0000 7f9107400640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f90f006b4d0 0x7f90f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.185+0000 7f9107400640 1 -- 172.21.15.119:0/1157466898 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9108089fd0 con 0x7f911015c820 2024-04-25T10:10:04.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.185+0000 7f910f400640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f90f006b4d0 0x7f90f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.186+0000 7f910f400640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f90f006b4d0 0x7f90f006d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9100001700 tx=0x7f9100009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.190+0000 7f9107400640 1 -- 172.21.15.119:0/1157466898 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9108058dc0 con 0x7f911015c820 2024-04-25T10:10:04.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.364+0000 7f9116000640 1 -- 172.21.15.119:0/1157466898 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91100749b0 con 0x7f90f006b4d0 2024-04-25T10:10:04.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.384+0000 7f9107400640 1 -- 172.21.15.119:0/1157466898 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251887 (secure 0 0 0) 0x7f91100749b0 con 0x7f90f006b4d0 2024-04-25T10:10:04.393 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:10:04.395 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:10:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.388+0000 7f9116000640 1 -- 172.21.15.119:0/1157466898 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f90f006b4d0 msgr2=0x7f90f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.388+0000 7f9116000640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f90f006b4d0 0x7f90f006d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9100001700 tx=0x7f9100009040 comp rx=0 tx=0).stop 2024-04-25T10:10:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.388+0000 7f9116000640 1 -- 172.21.15.119:0/1157466898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911015c820 msgr2=0x7f911013a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.388+0000 7f9116000640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911015c820 0x7f911013a1b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f910800b970 tx=0x7f910800be30 comp rx=0 tx=0).stop 2024-04-25T10:10:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.388+0000 7f9116000640 1 -- 172.21.15.119:0/1157466898 shutdown_connections 2024-04-25T10:10:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.388+0000 7f9116000640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f90f006b4d0 0x7f90f006d990 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.388+0000 7f9116000640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f911015c820 0x7f911013a1b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.388+0000 7f9116000640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f911015bf00 0x7f911014fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.388+0000 7f9116000640 1 --2- 172.21.15.119:0/1157466898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f911006e8f0 0x7f911013efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.388+0000 7f9116000640 1 -- 172.21.15.119:0/1157466898 >> 172.21.15.119:0/1157466898 conn(0x7f911006aba0 msgr2=0x7f91100cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.389+0000 7f9116000640 1 -- 172.21.15.119:0/1157466898 shutdown_connections 2024-04-25T10:10:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.389+0000 7f9116000640 1 -- 172.21.15.119:0/1157466898 wait complete. 2024-04-25T10:10:04.412 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-04-25T10:10:03.981655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247400,"kb_used_data":6272,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727320,"statfs":{"total":773094113280,"available":772840775680,"internally_reserved":0,"allocated":6422528,"data_stored":3317892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":246795826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.996347"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.055229+0000","last_change":"2024-04-25T10:10:00.053457+0000","last_active":"2024-04-25T10:10:00.055229+0000","last_peered":"2024-04-25T10:10:00.055229+0000","last_clean":"2024-04-25T10:10:00.055229+0000","last_became_active":"2024-04-25T10:10:00.053133+0000","last_became_peered":"2024-04-25T10:10:00.053133+0000","last_unstale":"2024-04-25T10:10:00.055229+0000","last_undegraded":"2024-04-25T10:10:00.055229+0000","last_fullsized":"2024-04-25T10:10:00.055229+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:06.634329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.072124+0000","last_change":"2024-04-25T10:10:00.070765+0000","last_active":"2024-04-25T10:10:00.072124+0000","last_peered":"2024-04-25T10:10:00.072124+0000","last_clean":"2024-04-25T10:10:00.072124+0000","last_became_active":"2024-04-25T10:10:00.069850+0000","last_became_peered":"2024-04-25T10:10:00.069850+0000","last_unstale":"2024-04-25T10:10:00.072124+0000","last_undegraded":"2024-04-25T10:10:00.072124+0000","last_fullsized":"2024-04-25T10:10:00.072124+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:01.400797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.055725+0000","last_change":"2024-04-25T10:10:00.051148+0000","last_active":"2024-04-25T10:10:00.055725+0000","last_peered":"2024-04-25T10:10:00.055725+0000","last_clean":"2024-04-25T10:10:00.055725+0000","last_became_active":"2024-04-25T10:10:00.049877+0000","last_became_peered":"2024-04-25T10:10:00.049877+0000","last_unstale":"2024-04-25T10:10:00.055725+0000","last_undegraded":"2024-04-25T10:10:00.055725+0000","last_fullsized":"2024-04-25T10:10:00.055725+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:43.167665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.049554+0000","last_change":"2024-04-25T10:10:00.043935+0000","last_active":"2024-04-25T10:10:00.049554+0000","last_peered":"2024-04-25T10:10:00.049554+0000","last_clean":"2024-04-25T10:10:00.049554+0000","last_became_active":"2024-04-25T10:10:00.043586+0000","last_became_peered":"2024-04-25T10:10:00.043586+0000","last_unstale":"2024-04-25T10:10:00.049554+0000","last_undegraded":"2024-04-25T10:10:00.049554+0000","last_fullsized":"2024-04-25T10:10:00.049554+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:11.916038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.042127+0000","last_change":"2024-04-25T10:10:00.038561+0000","last_active":"2024-04-25T10:10:00.042127+0000","last_peered":"2024-04-25T10:10:00.042127+0000","last_clean":"2024-04-25T10:10:00.042127+0000","last_became_active":"2024-04-25T10:10:00.038268+0000","last_became_peered":"2024-04-25T10:10:00.038268+0000","last_unstale":"2024-04-25T10:10:00.042127+0000","last_undegraded":"2024-04-25T10:10:00.042127+0000","last_fullsized":"2024-04-25T10:10:00.042127+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:19.061172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.051316+0000","last_change":"2024-04-25T10:10:00.047240+0000","last_active":"2024-04-25T10:10:00.051316+0000","last_peered":"2024-04-25T10:10:00.051316+0000","last_clean":"2024-04-25T10:10:00.051316+0000","last_became_active":"2024-04-25T10:10:00.046844+0000","last_became_peered":"2024-04-25T10:10:00.046844+0000","last_unstale":"2024-04-25T10:10:00.051316+0000","last_undegraded":"2024-04-25T10:10:00.051316+0000","last_fullsized":"2024-04-25T10:10:00.051316+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:27.493217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.056543+0000","last_change":"2024-04-25T10:10:00.053162+0000","last_active":"2024-04-25T10:10:00.056543+0000","last_peered":"2024-04-25T10:10:00.056543+0000","last_clean":"2024-04-25T10:10:00.056543+0000","last_became_active":"2024-04-25T10:10:00.051836+0000","last_became_peered":"2024-04-25T10:10:00.051836+0000","last_unstale":"2024-04-25T10:10:00.056543+0000","last_undegraded":"2024-04-25T10:10:00.056543+0000","last_fullsized":"2024-04-25T10:10:00.056543+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:55.090376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.064489+0000","last_change":"2024-04-25T10:10:00.053626+0000","last_active":"2024-04-25T10:10:00.064489+0000","last_peered":"2024-04-25T10:10:00.064489+0000","last_clean":"2024-04-25T10:10:00.064489+0000","last_became_active":"2024-04-25T10:10:00.053333+0000","last_became_peered":"2024-04-25T10:10:00.053333+0000","last_unstale":"2024-04-25T10:10:00.064489+0000","last_undegraded":"2024-04-25T10:10:00.064489+0000","last_fullsized":"2024-04-25T10:10:00.064489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:06.399299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.055394+0000","last_change":"2024-04-25T10:10:00.050684+0000","last_active":"2024-04-25T10:10:00.055394+0000","last_peered":"2024-04-25T10:10:00.055394+0000","last_clean":"2024-04-25T10:10:00.055394+0000","last_became_active":"2024-04-25T10:10:00.049996+0000","last_became_peered":"2024-04-25T10:10:00.049996+0000","last_unstale":"2024-04-25T10:10:00.055394+0000","last_undegraded":"2024-04-25T10:10:00.055394+0000","last_fullsized":"2024-04-25T10:10:00.055394+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:47.304806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.068976+0000","last_change":"2024-04-25T10:10:00.062367+0000","last_active":"2024-04-25T10:10:00.068976+0000","last_peered":"2024-04-25T10:10:00.068976+0000","last_clean":"2024-04-25T10:10:00.068976+0000","last_became_active":"2024-04-25T10:10:00.062011+0000","last_became_peered":"2024-04-25T10:10:00.062011+0000","last_unstale":"2024-04-25T10:10:00.068976+0000","last_undegraded":"2024-04-25T10:10:00.068976+0000","last_fullsized":"2024-04-25T10:10:00.068976+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:55.211323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.073609+0000","last_change":"2024-04-25T10:10:00.067572+0000","last_active":"2024-04-25T10:10:00.073609+0000","last_peered":"2024-04-25T10:10:00.073609+0000","last_clean":"2024-04-25T10:10:00.073609+0000","last_became_active":"2024-04-25T10:10:00.067270+0000","last_became_peered":"2024-04-25T10:10:00.067270+0000","last_unstale":"2024-04-25T10:10:00.073609+0000","last_undegraded":"2024-04-25T10:10:00.073609+0000","last_fullsized":"2024-04-25T10:10:00.073609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:58.948070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.052839+0000","last_change":"2024-04-25T10:10:00.048602+0000","last_active":"2024-04-25T10:10:00.052839+0000","last_peered":"2024-04-25T10:10:00.052839+0000","last_clean":"2024-04-25T10:10:00.052839+0000","last_became_active":"2024-04-25T10:10:00.048139+0000","last_became_peered":"2024-04-25T10:10:00.048139+0000","last_unstale":"2024-04-25T10:10:00.052839+0000","last_undegraded":"2024-04-25T10:10:00.052839+0000","last_fullsized":"2024-04-25T10:10:00.052839+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:08.780623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.053281+0000","last_change":"2024-04-25T10:10:00.048588+0000","last_active":"2024-04-25T10:10:00.053281+0000","last_peered":"2024-04-25T10:10:00.053281+0000","last_clean":"2024-04-25T10:10:00.053281+0000","last_became_active":"2024-04-25T10:10:00.048253+0000","last_became_peered":"2024-04-25T10:10:00.048253+0000","last_unstale":"2024-04-25T10:10:00.053281+0000","last_undegraded":"2024-04-25T10:10:00.053281+0000","last_fullsized":"2024-04-25T10:10:00.053281+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:35.833226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.054099+0000","last_change":"2024-04-25T10:10:00.047721+0000","last_active":"2024-04-25T10:10:00.054099+0000","last_peered":"2024-04-25T10:10:00.054099+0000","last_clean":"2024-04-25T10:10:00.054099+0000","last_became_active":"2024-04-25T10:10:00.046897+0000","last_became_peered":"2024-04-25T10:10:00.046897+0000","last_unstale":"2024-04-25T10:10:00.054099+0000","last_undegraded":"2024-04-25T10:10:00.054099+0000","last_fullsized":"2024-04-25T10:10:00.054099+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:35.012581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.053129+0000","last_change":"2024-04-25T10:10:00.047661+0000","last_active":"2024-04-25T10:10:00.053129+0000","last_peered":"2024-04-25T10:10:00.053129+0000","last_clean":"2024-04-25T10:10:00.053129+0000","last_became_active":"2024-04-25T10:10:00.047272+0000","last_became_peered":"2024-04-25T10:10:00.047272+0000","last_unstale":"2024-04-25T10:10:00.053129+0000","last_undegraded":"2024-04-25T10:10:00.053129+0000","last_fullsized":"2024-04-25T10:10:00.053129+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:33.568858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.060113+0000","last_change":"2024-04-25T10:10:00.053674+0000","last_active":"2024-04-25T10:10:00.060113+0000","last_peered":"2024-04-25T10:10:00.060113+0000","last_clean":"2024-04-25T10:10:00.060113+0000","last_became_active":"2024-04-25T10:10:00.053306+0000","last_became_peered":"2024-04-25T10:10:00.053306+0000","last_unstale":"2024-04-25T10:10:00.060113+0000","last_undegraded":"2024-04-25T10:10:00.060113+0000","last_fullsized":"2024-04-25T10:10:00.060113+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:53.194568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.061577+0000","last_change":"2024-04-25T10:10:00.050551+0000","last_active":"2024-04-25T10:10:00.061577+0000","last_peered":"2024-04-25T10:10:00.061577+0000","last_clean":"2024-04-25T10:10:00.061577+0000","last_became_active":"2024-04-25T10:10:00.050257+0000","last_became_peered":"2024-04-25T10:10:00.050257+0000","last_unstale":"2024-04-25T10:10:00.061577+0000","last_undegraded":"2024-04-25T10:10:00.061577+0000","last_fullsized":"2024-04-25T10:10:00.061577+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:30.886789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.055719+0000","last_change":"2024-04-25T10:10:00.050184+0000","last_active":"2024-04-25T10:10:00.055719+0000","last_peered":"2024-04-25T10:10:00.055719+0000","last_clean":"2024-04-25T10:10:00.055719+0000","last_became_active":"2024-04-25T10:10:00.049289+0000","last_became_peered":"2024-04-25T10:10:00.049289+0000","last_unstale":"2024-04-25T10:10:00.055719+0000","last_undegraded":"2024-04-25T10:10:00.055719+0000","last_fullsized":"2024-04-25T10:10:00.055719+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:00.349263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.073676+0000","last_change":"2024-04-25T10:10:00.066161+0000","last_active":"2024-04-25T10:10:00.073676+0000","last_peered":"2024-04-25T10:10:00.073676+0000","last_clean":"2024-04-25T10:10:00.073676+0000","last_became_active":"2024-04-25T10:10:00.065307+0000","last_became_peered":"2024-04-25T10:10:00.065307+0000","last_unstale":"2024-04-25T10:10:00.073676+0000","last_undegraded":"2024-04-25T10:10:00.073676+0000","last_fullsized":"2024-04-25T10:10:00.073676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:53.536215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.063913+0000","last_change":"2024-04-25T10:10:00.050827+0000","last_active":"2024-04-25T10:10:00.063913+0000","last_peered":"2024-04-25T10:10:00.063913+0000","last_clean":"2024-04-25T10:10:00.063913+0000","last_became_active":"2024-04-25T10:10:00.050085+0000","last_became_peered":"2024-04-25T10:10:00.050085+0000","last_unstale":"2024-04-25T10:10:00.063913+0000","last_undegraded":"2024-04-25T10:10:00.063913+0000","last_fullsized":"2024-04-25T10:10:00.063913+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:17.085267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.046903+0000","last_change":"2024-04-25T10:10:00.037326+0000","last_active":"2024-04-25T10:10:00.046903+0000","last_peered":"2024-04-25T10:10:00.046903+0000","last_clean":"2024-04-25T10:10:00.046903+0000","last_became_active":"2024-04-25T10:10:00.036577+0000","last_became_peered":"2024-04-25T10:10:00.036577+0000","last_unstale":"2024-04-25T10:10:00.046903+0000","last_undegraded":"2024-04-25T10:10:00.046903+0000","last_fullsized":"2024-04-25T10:10:00.046903+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:20.130812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.065600+0000","last_change":"2024-04-25T10:10:00.061812+0000","last_active":"2024-04-25T10:10:00.065600+0000","last_peered":"2024-04-25T10:10:00.065600+0000","last_clean":"2024-04-25T10:10:00.065600+0000","last_became_active":"2024-04-25T10:10:00.054477+0000","last_became_peered":"2024-04-25T10:10:00.054477+0000","last_unstale":"2024-04-25T10:10:00.065600+0000","last_undegraded":"2024-04-25T10:10:00.065600+0000","last_fullsized":"2024-04-25T10:10:00.065600+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:49.348976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.067496+0000","last_change":"2024-04-25T10:10:00.066525+0000","last_active":"2024-04-25T10:10:00.067496+0000","last_peered":"2024-04-25T10:10:00.067496+0000","last_clean":"2024-04-25T10:10:00.067496+0000","last_became_active":"2024-04-25T10:10:00.066243+0000","last_became_peered":"2024-04-25T10:10:00.066243+0000","last_unstale":"2024-04-25T10:10:00.067496+0000","last_undegraded":"2024-04-25T10:10:00.067496+0000","last_fullsized":"2024-04-25T10:10:00.067496+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:27.493217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.057735+0000","last_change":"2024-04-25T10:10:00.055056+0000","last_active":"2024-04-25T10:10:00.057735+0000","last_peered":"2024-04-25T10:10:00.057735+0000","last_clean":"2024-04-25T10:10:00.057735+0000","last_became_active":"2024-04-25T10:10:00.054591+0000","last_became_peered":"2024-04-25T10:10:00.054591+0000","last_unstale":"2024-04-25T10:10:00.057735+0000","last_undegraded":"2024-04-25T10:10:00.057735+0000","last_fullsized":"2024-04-25T10:10:00.057735+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:58.948070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.064597+0000","last_change":"2024-04-25T10:10:00.063307+0000","last_active":"2024-04-25T10:10:00.064597+0000","last_peered":"2024-04-25T10:10:00.064597+0000","last_clean":"2024-04-25T10:10:00.064597+0000","last_became_active":"2024-04-25T10:10:00.062961+0000","last_became_peered":"2024-04-25T10:10:00.062961+0000","last_unstale":"2024-04-25T10:10:00.064597+0000","last_undegraded":"2024-04-25T10:10:00.064597+0000","last_fullsized":"2024-04-25T10:10:00.064597+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:14.928259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.051650+0000","last_change":"2024-04-25T10:10:00.046736+0000","last_active":"2024-04-25T10:10:00.051650+0000","last_peered":"2024-04-25T10:10:00.051650+0000","last_clean":"2024-04-25T10:10:00.051650+0000","last_became_active":"2024-04-25T10:10:00.046342+0000","last_became_peered":"2024-04-25T10:10:00.046342+0000","last_unstale":"2024-04-25T10:10:00.051650+0000","last_undegraded":"2024-04-25T10:10:00.051650+0000","last_fullsized":"2024-04-25T10:10:00.051650+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:43.167665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.052766+0000","last_change":"2024-04-25T10:10:00.048566+0000","last_active":"2024-04-25T10:10:00.052766+0000","last_peered":"2024-04-25T10:10:00.052766+0000","last_clean":"2024-04-25T10:10:00.052766+0000","last_became_active":"2024-04-25T10:10:00.048285+0000","last_became_peered":"2024-04-25T10:10:00.048285+0000","last_unstale":"2024-04-25T10:10:00.052766+0000","last_undegraded":"2024-04-25T10:10:00.052766+0000","last_fullsized":"2024-04-25T10:10:00.052766+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:03.609745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.053737+0000","last_change":"2024-04-25T10:10:00.037092+0000","last_active":"2024-04-25T10:10:00.053737+0000","last_peered":"2024-04-25T10:10:00.053737+0000","last_clean":"2024-04-25T10:10:00.053737+0000","last_became_active":"2024-04-25T10:10:00.036405+0000","last_became_peered":"2024-04-25T10:10:00.036405+0000","last_unstale":"2024-04-25T10:10:00.053737+0000","last_undegraded":"2024-04-25T10:10:00.053737+0000","last_fullsized":"2024-04-25T10:10:00.053737+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:37.894419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.063866+0000","last_change":"2024-04-25T10:10:00.051336+0000","last_active":"2024-04-25T10:10:00.063866+0000","last_peered":"2024-04-25T10:10:00.063866+0000","last_clean":"2024-04-25T10:10:00.063866+0000","last_became_active":"2024-04-25T10:10:00.050640+0000","last_became_peered":"2024-04-25T10:10:00.050640+0000","last_unstale":"2024-04-25T10:10:00.063866+0000","last_undegraded":"2024-04-25T10:10:00.063866+0000","last_fullsized":"2024-04-25T10:10:00.063866+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:17.212811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.055498+0000","last_change":"2024-04-25T10:10:00.050180+0000","last_active":"2024-04-25T10:10:00.055498+0000","last_peered":"2024-04-25T10:10:00.055498+0000","last_clean":"2024-04-25T10:10:00.055498+0000","last_became_active":"2024-04-25T10:10:00.049908+0000","last_became_peered":"2024-04-25T10:10:00.049908+0000","last_unstale":"2024-04-25T10:10:00.055498+0000","last_undegraded":"2024-04-25T10:10:00.055498+0000","last_fullsized":"2024-04-25T10:10:00.055498+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:47.405959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.049093+0000","last_change":"2024-04-25T10:10:00.046517+0000","last_active":"2024-04-25T10:10:00.049093+0000","last_peered":"2024-04-25T10:10:00.049093+0000","last_clean":"2024-04-25T10:10:00.049093+0000","last_became_active":"2024-04-25T10:10:00.046158+0000","last_became_peered":"2024-04-25T10:10:00.046158+0000","last_unstale":"2024-04-25T10:10:00.049093+0000","last_undegraded":"2024-04-25T10:10:00.049093+0000","last_fullsized":"2024-04-25T10:10:00.049093+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:29.041790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.062791+0000","last_change":"2024-04-25T10:10:00.060790+0000","last_active":"2024-04-25T10:10:00.062791+0000","last_peered":"2024-04-25T10:10:00.062791+0000","last_clean":"2024-04-25T10:10:00.062791+0000","last_became_active":"2024-04-25T10:10:00.060299+0000","last_became_peered":"2024-04-25T10:10:00.060299+0000","last_unstale":"2024-04-25T10:10:00.062791+0000","last_undegraded":"2024-04-25T10:10:00.062791+0000","last_fullsized":"2024-04-25T10:10:00.062791+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:36.133119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.057614+0000","last_change":"2024-04-25T10:10:00.054412+0000","last_active":"2024-04-25T10:10:00.057614+0000","last_peered":"2024-04-25T10:10:00.057614+0000","last_clean":"2024-04-25T10:10:00.057614+0000","last_became_active":"2024-04-25T10:10:00.053801+0000","last_became_peered":"2024-04-25T10:10:00.053801+0000","last_unstale":"2024-04-25T10:10:00.057614+0000","last_undegraded":"2024-04-25T10:10:00.057614+0000","last_fullsized":"2024-04-25T10:10:00.057614+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:42.883016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.081347+0000","last_change":"2024-04-25T10:09:59.019152+0000","last_active":"2024-04-25T10:10:00.081347+0000","last_peered":"2024-04-25T10:10:00.081347+0000","last_clean":"2024-04-25T10:10:00.081347+0000","last_became_active":"2024-04-25T10:09:59.018544+0000","last_became_peered":"2024-04-25T10:09:59.018544+0000","last_unstale":"2024-04-25T10:10:00.081347+0000","last_undegraded":"2024-04-25T10:10:00.081347+0000","last_fullsized":"2024-04-25T10:10:00.081347+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:10.902662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.069094+0000","last_change":"2024-04-25T10:10:00.062197+0000","last_active":"2024-04-25T10:10:00.069094+0000","last_peered":"2024-04-25T10:10:00.069094+0000","last_clean":"2024-04-25T10:10:00.069094+0000","last_became_active":"2024-04-25T10:10:00.061589+0000","last_became_peered":"2024-04-25T10:10:00.061589+0000","last_unstale":"2024-04-25T10:10:00.069094+0000","last_undegraded":"2024-04-25T10:10:00.069094+0000","last_fullsized":"2024-04-25T10:10:00.069094+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:59.209848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.080168+0000","last_change":"2024-04-25T10:09:59.023773+0000","last_active":"2024-04-25T10:10:00.080168+0000","last_peered":"2024-04-25T10:10:00.080168+0000","last_clean":"2024-04-25T10:10:00.080168+0000","last_became_active":"2024-04-25T10:09:59.023445+0000","last_became_peered":"2024-04-25T10:09:59.023445+0000","last_unstale":"2024-04-25T10:10:00.080168+0000","last_undegraded":"2024-04-25T10:10:00.080168+0000","last_fullsized":"2024-04-25T10:10:00.080168+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:07.057321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.058802+0000","last_change":"2024-04-25T10:10:00.057488+0000","last_active":"2024-04-25T10:10:00.058802+0000","last_peered":"2024-04-25T10:10:00.058802+0000","last_clean":"2024-04-25T10:10:00.058802+0000","last_became_active":"2024-04-25T10:10:00.057153+0000","last_became_peered":"2024-04-25T10:10:00.057153+0000","last_unstale":"2024-04-25T10:10:00.058802+0000","last_undegraded":"2024-04-25T10:10:00.058802+0000","last_fullsized":"2024-04-25T10:10:00.058802+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:30.044417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.084311+0000","last_change":"2024-04-25T10:09:59.021669+0000","last_active":"2024-04-25T10:10:00.084311+0000","last_peered":"2024-04-25T10:10:00.084311+0000","last_clean":"2024-04-25T10:10:00.084311+0000","last_became_active":"2024-04-25T10:09:59.021431+0000","last_became_peered":"2024-04-25T10:09:59.021431+0000","last_unstale":"2024-04-25T10:10:00.084311+0000","last_undegraded":"2024-04-25T10:10:00.084311+0000","last_fullsized":"2024-04-25T10:10:00.084311+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:14.130629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.051278+0000","last_change":"2024-04-25T10:10:00.047739+0000","last_active":"2024-04-25T10:10:00.051278+0000","last_peered":"2024-04-25T10:10:00.051278+0000","last_clean":"2024-04-25T10:10:00.051278+0000","last_became_active":"2024-04-25T10:10:00.045327+0000","last_became_peered":"2024-04-25T10:10:00.045327+0000","last_unstale":"2024-04-25T10:10:00.051278+0000","last_undegraded":"2024-04-25T10:10:00.051278+0000","last_fullsized":"2024-04-25T10:10:00.051278+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:46.705938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.080424+0000","last_change":"2024-04-25T10:09:59.024931+0000","last_active":"2024-04-25T10:10:00.080424+0000","last_peered":"2024-04-25T10:10:00.080424+0000","last_clean":"2024-04-25T10:10:00.080424+0000","last_became_active":"2024-04-25T10:09:59.024633+0000","last_became_peered":"2024-04-25T10:09:59.024633+0000","last_unstale":"2024-04-25T10:10:00.080424+0000","last_undegraded":"2024-04-25T10:10:00.080424+0000","last_fullsized":"2024-04-25T10:10:00.080424+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:46.291430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.061968+0000","last_change":"2024-04-25T10:10:00.055059+0000","last_active":"2024-04-25T10:10:00.061968+0000","last_peered":"2024-04-25T10:10:00.061968+0000","last_clean":"2024-04-25T10:10:00.061968+0000","last_became_active":"2024-04-25T10:10:00.054073+0000","last_became_peered":"2024-04-25T10:10:00.054073+0000","last_unstale":"2024-04-25T10:10:00.061968+0000","last_undegraded":"2024-04-25T10:10:00.061968+0000","last_fullsized":"2024-04-25T10:10:00.061968+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:53.241996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.079984+0000","last_change":"2024-04-25T10:09:59.022940+0000","last_active":"2024-04-25T10:10:00.079984+0000","last_peered":"2024-04-25T10:10:00.079984+0000","last_clean":"2024-04-25T10:10:00.079984+0000","last_became_active":"2024-04-25T10:09:59.022081+0000","last_became_peered":"2024-04-25T10:09:59.022081+0000","last_unstale":"2024-04-25T10:10:00.079984+0000","last_undegraded":"2024-04-25T10:10:00.079984+0000","last_fullsized":"2024-04-25T10:10:00.079984+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:38.922189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.063888+0000","last_change":"2024-04-25T10:10:00.051568+0000","last_active":"2024-04-25T10:10:00.063888+0000","last_peered":"2024-04-25T10:10:00.063888+0000","last_clean":"2024-04-25T10:10:00.063888+0000","last_became_active":"2024-04-25T10:10:00.050972+0000","last_became_peered":"2024-04-25T10:10:00.050972+0000","last_unstale":"2024-04-25T10:10:00.063888+0000","last_undegraded":"2024-04-25T10:10:00.063888+0000","last_fullsized":"2024-04-25T10:10:00.063888+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:13.553159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.079301+0000","last_change":"2024-04-25T10:09:59.023429+0000","last_active":"2024-04-25T10:10:00.079301+0000","last_peered":"2024-04-25T10:10:00.079301+0000","last_clean":"2024-04-25T10:10:00.079301+0000","last_became_active":"2024-04-25T10:09:59.023200+0000","last_became_peered":"2024-04-25T10:09:59.023200+0000","last_unstale":"2024-04-25T10:10:00.079301+0000","last_undegraded":"2024-04-25T10:10:00.079301+0000","last_fullsized":"2024-04-25T10:10:00.079301+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:07.767247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.055709+0000","last_change":"2024-04-25T10:10:00.050773+0000","last_active":"2024-04-25T10:10:00.055709+0000","last_peered":"2024-04-25T10:10:00.055709+0000","last_clean":"2024-04-25T10:10:00.055709+0000","last_became_active":"2024-04-25T10:10:00.050472+0000","last_became_peered":"2024-04-25T10:10:00.050472+0000","last_unstale":"2024-04-25T10:10:00.055709+0000","last_undegraded":"2024-04-25T10:10:00.055709+0000","last_fullsized":"2024-04-25T10:10:00.055709+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:52.233499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.025430+0000","last_change":"2024-04-25T10:09:59.019986+0000","last_active":"2024-04-25T10:10:00.025430+0000","last_peered":"2024-04-25T10:10:00.025430+0000","last_clean":"2024-04-25T10:10:00.025430+0000","last_became_active":"2024-04-25T10:09:59.019673+0000","last_became_peered":"2024-04-25T10:09:59.019673+0000","last_unstale":"2024-04-25T10:10:00.025430+0000","last_undegraded":"2024-04-25T10:10:00.025430+0000","last_fullsized":"2024-04-25T10:10:00.025430+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:19.117436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.066670+0000","last_change":"2024-04-25T10:10:00.063283+0000","last_active":"2024-04-25T10:10:00.066670+0000","last_peered":"2024-04-25T10:10:00.066670+0000","last_clean":"2024-04-25T10:10:00.066670+0000","last_became_active":"2024-04-25T10:10:00.062951+0000","last_became_peered":"2024-04-25T10:10:00.062951+0000","last_unstale":"2024-04-25T10:10:00.066670+0000","last_undegraded":"2024-04-25T10:10:00.066670+0000","last_fullsized":"2024-04-25T10:10:00.066670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:20.254391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.039867+0000","last_change":"2024-04-25T10:09:59.024899+0000","last_active":"2024-04-25T10:10:00.039867+0000","last_peered":"2024-04-25T10:10:00.039867+0000","last_clean":"2024-04-25T10:10:00.039867+0000","last_became_active":"2024-04-25T10:09:59.024563+0000","last_became_peered":"2024-04-25T10:09:59.024563+0000","last_unstale":"2024-04-25T10:10:00.039867+0000","last_undegraded":"2024-04-25T10:10:00.039867+0000","last_fullsized":"2024-04-25T10:10:00.039867+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:54.437913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.063901+0000","last_change":"2024-04-25T10:10:00.062134+0000","last_active":"2024-04-25T10:10:00.063901+0000","last_peered":"2024-04-25T10:10:00.063901+0000","last_clean":"2024-04-25T10:10:00.063901+0000","last_became_active":"2024-04-25T10:10:00.061773+0000","last_became_peered":"2024-04-25T10:10:00.061773+0000","last_unstale":"2024-04-25T10:10:00.063901+0000","last_undegraded":"2024-04-25T10:10:00.063901+0000","last_fullsized":"2024-04-25T10:10:00.063901+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:17.363219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.031216+0000","last_change":"2024-04-25T10:09:59.030128+0000","last_active":"2024-04-25T10:10:00.031216+0000","last_peered":"2024-04-25T10:10:00.031216+0000","last_clean":"2024-04-25T10:10:00.031216+0000","last_became_active":"2024-04-25T10:09:59.029234+0000","last_became_peered":"2024-04-25T10:09:59.029234+0000","last_unstale":"2024-04-25T10:10:00.031216+0000","last_undegraded":"2024-04-25T10:10:00.031216+0000","last_fullsized":"2024-04-25T10:10:00.031216+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:54.077000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.071315+0000","last_change":"2024-04-25T10:10:00.070101+0000","last_active":"2024-04-25T10:10:00.071315+0000","last_peered":"2024-04-25T10:10:00.071315+0000","last_clean":"2024-04-25T10:10:00.071315+0000","last_became_active":"2024-04-25T10:10:00.069652+0000","last_became_peered":"2024-04-25T10:10:00.069652+0000","last_unstale":"2024-04-25T10:10:00.071315+0000","last_undegraded":"2024-04-25T10:10:00.071315+0000","last_fullsized":"2024-04-25T10:10:00.071315+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:03.609745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.040082+0000","last_change":"2024-04-25T10:09:59.010343+0000","last_active":"2024-04-25T10:10:00.040082+0000","last_peered":"2024-04-25T10:10:00.040082+0000","last_clean":"2024-04-25T10:10:00.040082+0000","last_became_active":"2024-04-25T10:09:59.010088+0000","last_became_peered":"2024-04-25T10:09:59.010088+0000","last_unstale":"2024-04-25T10:10:00.040082+0000","last_undegraded":"2024-04-25T10:10:00.040082+0000","last_fullsized":"2024-04-25T10:10:00.040082+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:29.031041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.058788+0000","last_change":"2024-04-25T10:10:00.055963+0000","last_active":"2024-04-25T10:10:00.058788+0000","last_peered":"2024-04-25T10:10:00.058788+0000","last_clean":"2024-04-25T10:10:00.058788+0000","last_became_active":"2024-04-25T10:10:00.055642+0000","last_became_peered":"2024-04-25T10:10:00.055642+0000","last_unstale":"2024-04-25T10:10:00.058788+0000","last_undegraded":"2024-04-25T10:10:00.058788+0000","last_fullsized":"2024-04-25T10:10:00.058788+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:58.932589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.084357+0000","last_change":"2024-04-25T10:09:59.020756+0000","last_active":"2024-04-25T10:10:00.084357+0000","last_peered":"2024-04-25T10:10:00.084357+0000","last_clean":"2024-04-25T10:10:00.084357+0000","last_became_active":"2024-04-25T10:09:59.020428+0000","last_became_peered":"2024-04-25T10:09:59.020428+0000","last_unstale":"2024-04-25T10:10:00.084357+0000","last_undegraded":"2024-04-25T10:10:00.084357+0000","last_fullsized":"2024-04-25T10:10:00.084357+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:16.349844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.072050+0000","last_change":"2024-04-25T10:10:00.070507+0000","last_active":"2024-04-25T10:10:00.072050+0000","last_peered":"2024-04-25T10:10:00.072050+0000","last_clean":"2024-04-25T10:10:00.072050+0000","last_became_active":"2024-04-25T10:10:00.069786+0000","last_became_peered":"2024-04-25T10:10:00.069786+0000","last_unstale":"2024-04-25T10:10:00.072050+0000","last_undegraded":"2024-04-25T10:10:00.072050+0000","last_fullsized":"2024-04-25T10:10:00.072050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:52.968116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"54'6","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.087007+0000","last_change":"2024-04-25T10:09:59.028172+0000","last_active":"2024-04-25T10:10:00.087007+0000","last_peered":"2024-04-25T10:10:00.087007+0000","last_clean":"2024-04-25T10:10:00.087007+0000","last_became_active":"2024-04-25T10:09:59.027692+0000","last_became_peered":"2024-04-25T10:09:59.027692+0000","last_unstale":"2024-04-25T10:10:00.087007+0000","last_undegraded":"2024-04-25T10:10:00.087007+0000","last_fullsized":"2024-04-25T10:10:00.087007+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:21.748509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.064237+0000","last_change":"2024-04-25T10:10:00.063101+0000","last_active":"2024-04-25T10:10:00.064237+0000","last_peered":"2024-04-25T10:10:00.064237+0000","last_clean":"2024-04-25T10:10:00.064237+0000","last_became_active":"2024-04-25T10:10:00.062455+0000","last_became_peered":"2024-04-25T10:10:00.062455+0000","last_unstale":"2024-04-25T10:10:00.064237+0000","last_undegraded":"2024-04-25T10:10:00.064237+0000","last_fullsized":"2024-04-25T10:10:00.064237+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:27.493217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.084609+0000","last_change":"2024-04-25T10:09:59.021295+0000","last_active":"2024-04-25T10:10:00.084609+0000","last_peered":"2024-04-25T10:10:00.084609+0000","last_clean":"2024-04-25T10:10:00.084609+0000","last_became_active":"2024-04-25T10:09:59.020973+0000","last_became_peered":"2024-04-25T10:09:59.020973+0000","last_unstale":"2024-04-25T10:10:00.084609+0000","last_undegraded":"2024-04-25T10:10:00.084609+0000","last_fullsized":"2024-04-25T10:10:00.084609+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:52.181192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.050643+0000","last_change":"2024-04-25T10:10:00.046754+0000","last_active":"2024-04-25T10:10:00.050643+0000","last_peered":"2024-04-25T10:10:00.050643+0000","last_clean":"2024-04-25T10:10:00.050643+0000","last_became_active":"2024-04-25T10:10:00.046433+0000","last_became_peered":"2024-04-25T10:10:00.046433+0000","last_unstale":"2024-04-25T10:10:00.050643+0000","last_undegraded":"2024-04-25T10:10:00.050643+0000","last_fullsized":"2024-04-25T10:10:00.050643+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:11.035238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.039422+0000","last_change":"2024-04-25T10:09:59.027970+0000","last_active":"2024-04-25T10:10:00.039422+0000","last_peered":"2024-04-25T10:10:00.039422+0000","last_clean":"2024-04-25T10:10:00.039422+0000","last_became_active":"2024-04-25T10:09:59.027098+0000","last_became_peered":"2024-04-25T10:09:59.027098+0000","last_unstale":"2024-04-25T10:10:00.039422+0000","last_undegraded":"2024-04-25T10:10:00.039422+0000","last_fullsized":"2024-04-25T10:10:00.039422+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:58.196472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.055494+0000","last_change":"2024-04-25T10:10:00.051269+0000","last_active":"2024-04-25T10:10:00.055494+0000","last_peered":"2024-04-25T10:10:00.055494+0000","last_clean":"2024-04-25T10:10:00.055494+0000","last_became_active":"2024-04-25T10:10:00.050926+0000","last_became_peered":"2024-04-25T10:10:00.050926+0000","last_unstale":"2024-04-25T10:10:00.055494+0000","last_undegraded":"2024-04-25T10:10:00.055494+0000","last_fullsized":"2024-04-25T10:10:00.055494+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:12.479880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.025463+0000","last_change":"2024-04-25T10:09:59.025294+0000","last_active":"2024-04-25T10:10:00.025463+0000","last_peered":"2024-04-25T10:10:00.025463+0000","last_clean":"2024-04-25T10:10:00.025463+0000","last_became_active":"2024-04-25T10:09:59.024309+0000","last_became_peered":"2024-04-25T10:09:59.024309+0000","last_unstale":"2024-04-25T10:10:00.025463+0000","last_undegraded":"2024-04-25T10:10:00.025463+0000","last_fullsized":"2024-04-25T10:10:00.025463+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:10.021862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.067481+0000","last_change":"2024-04-25T10:10:00.066428+0000","last_active":"2024-04-25T10:10:00.067481+0000","last_peered":"2024-04-25T10:10:00.067481+0000","last_clean":"2024-04-25T10:10:00.067481+0000","last_became_active":"2024-04-25T10:10:00.066143+0000","last_became_peered":"2024-04-25T10:10:00.066143+0000","last_unstale":"2024-04-25T10:10:00.067481+0000","last_undegraded":"2024-04-25T10:10:00.067481+0000","last_fullsized":"2024-04-25T10:10:00.067481+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:15.144005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.081549+0000","last_change":"2024-04-25T10:09:59.029040+0000","last_active":"2024-04-25T10:10:00.081549+0000","last_peered":"2024-04-25T10:10:00.081549+0000","last_clean":"2024-04-25T10:10:00.081549+0000","last_became_active":"2024-04-25T10:09:59.027224+0000","last_became_peered":"2024-04-25T10:09:59.027224+0000","last_unstale":"2024-04-25T10:10:00.081549+0000","last_undegraded":"2024-04-25T10:10:00.081549+0000","last_fullsized":"2024-04-25T10:10:00.081549+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.466504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.056859+0000","last_change":"2024-04-25T10:10:00.050146+0000","last_active":"2024-04-25T10:10:00.056859+0000","last_peered":"2024-04-25T10:10:00.056859+0000","last_clean":"2024-04-25T10:10:00.056859+0000","last_became_active":"2024-04-25T10:10:00.049711+0000","last_became_peered":"2024-04-25T10:10:00.049711+0000","last_unstale":"2024-04-25T10:10:00.056859+0000","last_undegraded":"2024-04-25T10:10:00.056859+0000","last_fullsized":"2024-04-25T10:10:00.056859+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:11.916038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.084711+0000","last_change":"2024-04-25T10:09:59.015996+0000","last_active":"2024-04-25T10:10:00.084711+0000","last_peered":"2024-04-25T10:10:00.084711+0000","last_clean":"2024-04-25T10:10:00.084711+0000","last_became_active":"2024-04-25T10:09:59.015765+0000","last_became_peered":"2024-04-25T10:09:59.015765+0000","last_unstale":"2024-04-25T10:10:00.084711+0000","last_undegraded":"2024-04-25T10:10:00.084711+0000","last_fullsized":"2024-04-25T10:10:00.084711+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:07.021975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.071283+0000","last_change":"2024-04-25T10:10:00.070229+0000","last_active":"2024-04-25T10:10:00.071283+0000","last_peered":"2024-04-25T10:10:00.071283+0000","last_clean":"2024-04-25T10:10:00.071283+0000","last_became_active":"2024-04-25T10:10:00.069736+0000","last_became_peered":"2024-04-25T10:10:00.069736+0000","last_unstale":"2024-04-25T10:10:00.071283+0000","last_undegraded":"2024-04-25T10:10:00.071283+0000","last_fullsized":"2024-04-25T10:10:00.071283+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:46.705938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.082242+0000","last_change":"2024-04-25T10:09:59.012428+0000","last_active":"2024-04-25T10:10:00.082242+0000","last_peered":"2024-04-25T10:10:00.082242+0000","last_clean":"2024-04-25T10:10:00.082242+0000","last_became_active":"2024-04-25T10:09:59.012128+0000","last_became_peered":"2024-04-25T10:09:59.012128+0000","last_unstale":"2024-04-25T10:10:00.082242+0000","last_undegraded":"2024-04-25T10:10:00.082242+0000","last_fullsized":"2024-04-25T10:10:00.082242+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:46.392583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.063932+0000","last_change":"2024-04-25T10:10:00.051095+0000","last_active":"2024-04-25T10:10:00.063932+0000","last_peered":"2024-04-25T10:10:00.063932+0000","last_clean":"2024-04-25T10:10:00.063932+0000","last_became_active":"2024-04-25T10:10:00.050529+0000","last_became_peered":"2024-04-25T10:10:00.050529+0000","last_unstale":"2024-04-25T10:10:00.063932+0000","last_undegraded":"2024-04-25T10:10:00.063932+0000","last_fullsized":"2024-04-25T10:10:00.063932+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:31.813859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.080876+0000","last_change":"2024-04-25T10:09:59.026331+0000","last_active":"2024-04-25T10:10:00.080876+0000","last_peered":"2024-04-25T10:10:00.080876+0000","last_clean":"2024-04-25T10:10:00.080876+0000","last_became_active":"2024-04-25T10:09:59.026026+0000","last_became_peered":"2024-04-25T10:09:59.026026+0000","last_unstale":"2024-04-25T10:10:00.080876+0000","last_undegraded":"2024-04-25T10:10:00.080876+0000","last_fullsized":"2024-04-25T10:10:00.080876+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:16.349844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.069440+0000","last_change":"2024-04-25T10:10:00.068259+0000","last_active":"2024-04-25T10:10:00.069440+0000","last_peered":"2024-04-25T10:10:00.069440+0000","last_clean":"2024-04-25T10:10:00.069440+0000","last_became_active":"2024-04-25T10:10:00.067922+0000","last_became_peered":"2024-04-25T10:10:00.067922+0000","last_unstale":"2024-04-25T10:10:00.069440+0000","last_undegraded":"2024-04-25T10:10:00.069440+0000","last_fullsized":"2024-04-25T10:10:00.069440+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:46.705938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.030932+0000","last_change":"2024-04-25T10:09:59.018256+0000","last_active":"2024-04-25T10:10:00.030932+0000","last_peered":"2024-04-25T10:10:00.030932+0000","last_clean":"2024-04-25T10:10:00.030932+0000","last_became_active":"2024-04-25T10:09:59.016274+0000","last_became_peered":"2024-04-25T10:09:59.016274+0000","last_unstale":"2024-04-25T10:10:00.030932+0000","last_undegraded":"2024-04-25T10:10:00.030932+0000","last_fullsized":"2024-04-25T10:10:00.030932+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:42.154289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.060265+0000","last_change":"2024-04-25T10:10:00.059369+0000","last_active":"2024-04-25T10:10:00.060265+0000","last_peered":"2024-04-25T10:10:00.060265+0000","last_clean":"2024-04-25T10:10:00.060265+0000","last_became_active":"2024-04-25T10:10:00.059113+0000","last_became_peered":"2024-04-25T10:10:00.059113+0000","last_unstale":"2024-04-25T10:10:00.060265+0000","last_undegraded":"2024-04-25T10:10:00.060265+0000","last_fullsized":"2024-04-25T10:10:00.060265+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:15.144005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.084447+0000","last_change":"2024-04-25T10:09:59.020140+0000","last_active":"2024-04-25T10:10:00.084447+0000","last_peered":"2024-04-25T10:10:00.084447+0000","last_clean":"2024-04-25T10:10:00.084447+0000","last_became_active":"2024-04-25T10:09:59.019786+0000","last_became_peered":"2024-04-25T10:09:59.019786+0000","last_unstale":"2024-04-25T10:10:00.084447+0000","last_undegraded":"2024-04-25T10:10:00.084447+0000","last_fullsized":"2024-04-25T10:10:00.084447+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:41.869640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.052864+0000","last_change":"2024-04-25T10:10:00.049402+0000","last_active":"2024-04-25T10:10:00.052864+0000","last_peered":"2024-04-25T10:10:00.052864+0000","last_clean":"2024-04-25T10:10:00.052864+0000","last_became_active":"2024-04-25T10:10:00.045947+0000","last_became_peered":"2024-04-25T10:10:00.045947+0000","last_unstale":"2024-04-25T10:10:00.052864+0000","last_undegraded":"2024-04-25T10:10:00.052864+0000","last_fullsized":"2024-04-25T10:10:00.052864+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:46.655110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.065890+0000","last_change":"2024-04-25T10:09:59.015843+0000","last_active":"2024-04-25T10:10:00.065890+0000","last_peered":"2024-04-25T10:10:00.065890+0000","last_clean":"2024-04-25T10:10:00.065890+0000","last_became_active":"2024-04-25T10:09:59.015538+0000","last_became_peered":"2024-04-25T10:09:59.015538+0000","last_unstale":"2024-04-25T10:10:00.065890+0000","last_undegraded":"2024-04-25T10:10:00.065890+0000","last_fullsized":"2024-04-25T10:10:00.065890+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:51.220123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.052833+0000","last_change":"2024-04-25T10:10:00.049680+0000","last_active":"2024-04-25T10:10:00.052833+0000","last_peered":"2024-04-25T10:10:00.052833+0000","last_clean":"2024-04-25T10:10:00.052833+0000","last_became_active":"2024-04-25T10:10:00.049376+0000","last_became_peered":"2024-04-25T10:10:00.049376+0000","last_unstale":"2024-04-25T10:10:00.052833+0000","last_undegraded":"2024-04-25T10:10:00.052833+0000","last_fullsized":"2024-04-25T10:10:00.052833+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:45.607269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.024851+0000","last_change":"2024-04-25T10:09:59.021922+0000","last_active":"2024-04-25T10:10:00.024851+0000","last_peered":"2024-04-25T10:10:00.024851+0000","last_clean":"2024-04-25T10:10:00.024851+0000","last_became_active":"2024-04-25T10:09:59.021630+0000","last_became_peered":"2024-04-25T10:09:59.021630+0000","last_unstale":"2024-04-25T10:10:00.024851+0000","last_undegraded":"2024-04-25T10:10:00.024851+0000","last_fullsized":"2024-04-25T10:10:00.024851+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:54.077000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.049746+0000","last_change":"2024-04-25T10:10:00.045668+0000","last_active":"2024-04-25T10:10:00.049746+0000","last_peered":"2024-04-25T10:10:00.049746+0000","last_clean":"2024-04-25T10:10:00.049746+0000","last_became_active":"2024-04-25T10:10:00.045262+0000","last_became_peered":"2024-04-25T10:10:00.045262+0000","last_unstale":"2024-04-25T10:10:00.049746+0000","last_undegraded":"2024-04-25T10:10:00.049746+0000","last_fullsized":"2024-04-25T10:10:00.049746+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:47.405959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.085281+0000","last_change":"2024-04-25T10:09:59.022783+0000","last_active":"2024-04-25T10:10:00.085281+0000","last_peered":"2024-04-25T10:10:00.085281+0000","last_clean":"2024-04-25T10:10:00.085281+0000","last_became_active":"2024-04-25T10:09:59.022440+0000","last_became_peered":"2024-04-25T10:09:59.022440+0000","last_unstale":"2024-04-25T10:10:00.085281+0000","last_undegraded":"2024-04-25T10:10:00.085281+0000","last_fullsized":"2024-04-25T10:10:00.085281+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:58.812393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.052871+0000","last_change":"2024-04-25T10:10:00.048383+0000","last_active":"2024-04-25T10:10:00.052871+0000","last_peered":"2024-04-25T10:10:00.052871+0000","last_clean":"2024-04-25T10:10:00.052871+0000","last_became_active":"2024-04-25T10:10:00.048051+0000","last_became_peered":"2024-04-25T10:10:00.048051+0000","last_unstale":"2024-04-25T10:10:00.052871+0000","last_undegraded":"2024-04-25T10:10:00.052871+0000","last_fullsized":"2024-04-25T10:10:00.052871+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:00.349263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.031324+0000","last_change":"2024-04-25T10:09:59.022236+0000","last_active":"2024-04-25T10:10:00.031324+0000","last_peered":"2024-04-25T10:10:00.031324+0000","last_clean":"2024-04-25T10:10:00.031324+0000","last_became_active":"2024-04-25T10:09:59.020911+0000","last_became_peered":"2024-04-25T10:09:59.020911+0000","last_unstale":"2024-04-25T10:10:00.031324+0000","last_undegraded":"2024-04-25T10:10:00.031324+0000","last_fullsized":"2024-04-25T10:10:00.031324+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:59.335887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.058659+0000","last_change":"2024-04-25T10:10:00.055571+0000","last_active":"2024-04-25T10:10:00.058659+0000","last_peered":"2024-04-25T10:10:00.058659+0000","last_clean":"2024-04-25T10:10:00.058659+0000","last_became_active":"2024-04-25T10:10:00.054875+0000","last_became_peered":"2024-04-25T10:10:00.054875+0000","last_unstale":"2024-04-25T10:10:00.058659+0000","last_undegraded":"2024-04-25T10:10:00.058659+0000","last_fullsized":"2024-04-25T10:10:00.058659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:24.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.042728+0000","last_change":"2024-04-25T10:09:59.023024+0000","last_active":"2024-04-25T10:10:00.042728+0000","last_peered":"2024-04-25T10:10:00.042728+0000","last_clean":"2024-04-25T10:10:00.042728+0000","last_became_active":"2024-04-25T10:09:59.021790+0000","last_became_peered":"2024-04-25T10:09:59.021790+0000","last_unstale":"2024-04-25T10:10:00.042728+0000","last_undegraded":"2024-04-25T10:10:00.042728+0000","last_fullsized":"2024-04-25T10:10:00.042728+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:20.433429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.058162+0000","last_change":"2024-04-25T10:10:00.054747+0000","last_active":"2024-04-25T10:10:00.058162+0000","last_peered":"2024-04-25T10:10:00.058162+0000","last_clean":"2024-04-25T10:10:00.058162+0000","last_became_active":"2024-04-25T10:10:00.054446+0000","last_became_peered":"2024-04-25T10:10:00.054446+0000","last_unstale":"2024-04-25T10:10:00.058162+0000","last_undegraded":"2024-04-25T10:10:00.058162+0000","last_fullsized":"2024-04-25T10:10:00.058162+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:08.070697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.024185+0000","last_change":"2024-04-25T10:09:59.022659+0000","last_active":"2024-04-25T10:10:00.024185+0000","last_peered":"2024-04-25T10:10:00.024185+0000","last_clean":"2024-04-25T10:10:00.024185+0000","last_became_active":"2024-04-25T10:09:59.021237+0000","last_became_peered":"2024-04-25T10:09:59.021237+0000","last_unstale":"2024-04-25T10:10:00.024185+0000","last_undegraded":"2024-04-25T10:10:00.024185+0000","last_fullsized":"2024-04-25T10:10:00.024185+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.466504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.067827+0000","last_change":"2024-04-25T10:10:00.062854+0000","last_active":"2024-04-25T10:10:00.067827+0000","last_peered":"2024-04-25T10:10:00.067827+0000","last_clean":"2024-04-25T10:10:00.067827+0000","last_became_active":"2024-04-25T10:10:00.062538+0000","last_became_peered":"2024-04-25T10:10:00.062538+0000","last_unstale":"2024-04-25T10:10:00.067827+0000","last_undegraded":"2024-04-25T10:10:00.067827+0000","last_fullsized":"2024-04-25T10:10:00.067827+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:41.043177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.065427+0000","last_change":"2024-04-25T10:09:59.025029+0000","last_active":"2024-04-25T10:10:00.065427+0000","last_peered":"2024-04-25T10:10:00.065427+0000","last_clean":"2024-04-25T10:10:00.065427+0000","last_became_active":"2024-04-25T10:09:59.024404+0000","last_became_peered":"2024-04-25T10:09:59.024404+0000","last_unstale":"2024-04-25T10:10:00.065427+0000","last_undegraded":"2024-04-25T10:10:00.065427+0000","last_fullsized":"2024-04-25T10:10:00.065427+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:20.433429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.058534+0000","last_change":"2024-04-25T10:10:00.057319+0000","last_active":"2024-04-25T10:10:00.058534+0000","last_peered":"2024-04-25T10:10:00.058534+0000","last_clean":"2024-04-25T10:10:00.058534+0000","last_became_active":"2024-04-25T10:10:00.056981+0000","last_became_peered":"2024-04-25T10:10:00.056981+0000","last_unstale":"2024-04-25T10:10:00.058534+0000","last_undegraded":"2024-04-25T10:10:00.058534+0000","last_fullsized":"2024-04-25T10:10:00.058534+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:08.780623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.036623+0000","last_change":"2024-04-25T10:09:59.028406+0000","last_active":"2024-04-25T10:10:00.036623+0000","last_peered":"2024-04-25T10:10:00.036623+0000","last_clean":"2024-04-25T10:10:00.036623+0000","last_became_active":"2024-04-25T10:09:59.028103+0000","last_became_peered":"2024-04-25T10:09:59.028103+0000","last_unstale":"2024-04-25T10:10:00.036623+0000","last_undegraded":"2024-04-25T10:10:00.036623+0000","last_fullsized":"2024-04-25T10:10:00.036623+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:44.296043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.068836+0000","last_change":"2024-04-25T10:10:00.059857+0000","last_active":"2024-04-25T10:10:00.068836+0000","last_peered":"2024-04-25T10:10:00.068836+0000","last_clean":"2024-04-25T10:10:00.068836+0000","last_became_active":"2024-04-25T10:10:00.052617+0000","last_became_peered":"2024-04-25T10:10:00.052617+0000","last_unstale":"2024-04-25T10:10:00.068836+0000","last_undegraded":"2024-04-25T10:10:00.068836+0000","last_fullsized":"2024-04-25T10:10:00.068836+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:40.980941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.065479+0000","last_change":"2024-04-25T10:09:59.027922+0000","last_active":"2024-04-25T10:10:00.065479+0000","last_peered":"2024-04-25T10:10:00.065479+0000","last_clean":"2024-04-25T10:10:00.065479+0000","last_became_active":"2024-04-25T10:09:59.027613+0000","last_became_peered":"2024-04-25T10:09:59.027613+0000","last_unstale":"2024-04-25T10:10:00.065479+0000","last_undegraded":"2024-04-25T10:10:00.065479+0000","last_fullsized":"2024-04-25T10:10:00.065479+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:34.819850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.064414+0000","last_change":"2024-04-25T10:10:00.058385+0000","last_active":"2024-04-25T10:10:00.064414+0000","last_peered":"2024-04-25T10:10:00.064414+0000","last_clean":"2024-04-25T10:10:00.064414+0000","last_became_active":"2024-04-25T10:10:00.057823+0000","last_became_peered":"2024-04-25T10:10:00.057823+0000","last_unstale":"2024-04-25T10:10:00.064414+0000","last_undegraded":"2024-04-25T10:10:00.064414+0000","last_fullsized":"2024-04-25T10:10:00.064414+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:19.895762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.040164+0000","last_change":"2024-04-25T10:09:59.025945+0000","last_active":"2024-04-25T10:10:00.040164+0000","last_peered":"2024-04-25T10:10:00.040164+0000","last_clean":"2024-04-25T10:10:00.040164+0000","last_became_active":"2024-04-25T10:09:59.025319+0000","last_became_peered":"2024-04-25T10:09:59.025319+0000","last_unstale":"2024-04-25T10:10:00.040164+0000","last_undegraded":"2024-04-25T10:10:00.040164+0000","last_fullsized":"2024-04-25T10:10:00.040164+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:35.119743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.064512+0000","last_change":"2024-04-25T10:10:00.056971+0000","last_active":"2024-04-25T10:10:00.064512+0000","last_peered":"2024-04-25T10:10:00.064512+0000","last_clean":"2024-04-25T10:10:00.064512+0000","last_became_active":"2024-04-25T10:10:00.056679+0000","last_became_peered":"2024-04-25T10:10:00.056679+0000","last_unstale":"2024-04-25T10:10:00.064512+0000","last_undegraded":"2024-04-25T10:10:00.064512+0000","last_fullsized":"2024-04-25T10:10:00.064512+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:01.874569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.039553+0000","last_change":"2024-04-25T10:09:59.027207+0000","last_active":"2024-04-25T10:10:00.039553+0000","last_peered":"2024-04-25T10:10:00.039553+0000","last_clean":"2024-04-25T10:10:00.039553+0000","last_became_active":"2024-04-25T10:09:59.026386+0000","last_became_peered":"2024-04-25T10:09:59.026386+0000","last_unstale":"2024-04-25T10:10:00.039553+0000","last_undegraded":"2024-04-25T10:10:00.039553+0000","last_fullsized":"2024-04-25T10:10:00.039553+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:54.197947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.025484+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:10:00.025484+0000","last_peered":"2024-04-25T10:10:00.025484+0000","last_clean":"2024-04-25T10:10:00.025484+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:10:00.025484+0000","last_undegraded":"2024-04-25T10:10:00.025484+0000","last_fullsized":"2024-04-25T10:10:00.025484+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1241088,"data_stored":856930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":647168,"data_stored":266562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":655360,"data_stored":267214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":659456,"data_stored":268174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":655360,"data_stored":266618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":655360,"data_stored":266618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":651264,"data_stored":266652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":859124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:10:04.418 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:10:04.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.647+0000 7f62b9a00640 1 Processor -- start 2024-04-25T10:10:04.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.647+0000 7f62b9a00640 1 -- start start 2024-04-25T10:10:04.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.648+0000 7f62b9a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f62b415bf00 0x7f62b40cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.648+0000 7f62b9a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b40cc310 0x7f62b415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.649+0000 7f62b9a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b40ce6c0 con 0x7f62b415c820 2024-04-25T10:10:04.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.649+0000 7f62b9a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b41455e0 con 0x7f62b415bf00 2024-04-25T10:10:04.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.649+0000 7f62b9a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b406d450 con 0x7f62b40cc310 2024-04-25T10:10:04.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.649+0000 7f62abe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b40cc310 0x7f62b415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.649+0000 7f62abe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b40cc310 0x7f62b415c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:52470/0 (socket says 172.21.15.119:52470) 2024-04-25T10:10:04.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.650+0000 7f62abe00640 1 -- 172.21.15.119:0/1607088223 learned_addr learned my addr 172.21.15.119:0/1607088223 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:04.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.650+0000 7f62b2a00640 1 --2- 172.21.15.119:0/1607088223 >> v2:172.21.15.132:3300/0 conn(0x7f62b415bf00 0x7f62b40cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.650+0000 7f62abe00640 1 -- 172.21.15.119:0/1607088223 >> v2:172.21.15.132:3300/0 conn(0x7f62b415bf00 msgr2=0x7f62b40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:04.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.650+0000 7f62abe00640 1 --2- 172.21.15.119:0/1607088223 >> v2:172.21.15.132:3300/0 conn(0x7f62b415bf00 0x7f62b40cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.650+0000 7f62abe00640 1 -- 172.21.15.119:0/1607088223 >> v1:172.21.15.132:6789/0 conn(0x7f62b415c820 legacy=0x7f62b406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.650+0000 7f62abe00640 1 -- 172.21.15.119:0/1607088223 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b40cc7b0 con 0x7f62b40cc310 2024-04-25T10:10:04.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.652+0000 7f62abe00640 1 --2- 172.21.15.119:0/1607088223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b40cc310 0x7f62b415c2e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f629c009870 tx=0x7f629c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=359699920e1ed478 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.653+0000 7f62b2000640 1 -- 172.21.15.119:0/1607088223 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c037070 con 0x7f62b40cc310 2024-04-25T10:10:04.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.653+0000 7f62b2000640 1 -- 172.21.15.119:0/1607088223 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c032440 con 0x7f62b40cc310 2024-04-25T10:10:04.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.653+0000 7f62b2000640 1 -- 172.21.15.119:0/1607088223 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c032810 con 0x7f62b40cc310 2024-04-25T10:10:04.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.653+0000 7f62b9a00640 1 -- 172.21.15.119:0/1607088223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b40cc310 msgr2=0x7f62b415c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.653+0000 7f62b9a00640 1 --2- 172.21.15.119:0/1607088223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b40cc310 0x7f62b415c2e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f629c009870 tx=0x7f629c02a250 comp rx=0 tx=0).stop 2024-04-25T10:10:04.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.654+0000 7f62b9a00640 1 -- 172.21.15.119:0/1607088223 shutdown_connections 2024-04-25T10:10:04.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.654+0000 7f62b9a00640 1 --2- 172.21.15.119:0/1607088223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b40cc310 0x7f62b415c2e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.654+0000 7f62b9a00640 1 --2- 172.21.15.119:0/1607088223 >> v2:172.21.15.132:3300/0 conn(0x7f62b415bf00 0x7f62b40cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.654+0000 7f62b9a00640 1 -- 172.21.15.119:0/1607088223 >> 172.21.15.119:0/1607088223 conn(0x7f62b406aba0 msgr2=0x7f62b406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.654+0000 7f62b9a00640 1 -- 172.21.15.119:0/1607088223 shutdown_connections 2024-04-25T10:10:04.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.654+0000 7f62b9a00640 1 -- 172.21.15.119:0/1607088223 wait complete. 2024-04-25T10:10:04.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.656+0000 7f62b9a00640 1 Processor -- start 2024-04-25T10:10:04.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.656+0000 7f62b9a00640 1 -- start start 2024-04-25T10:10:04.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.657+0000 7f62b9a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b40cc310 0x7f62b41f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.657+0000 7f62b9a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b415bf00 0x7f62b414bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.657+0000 7f62b9a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b415c820 0x7f62b41e5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.657+0000 7f62b9a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b40cd050 con 0x7f62b415c820 2024-04-25T10:10:04.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.657+0000 7f62b9a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b40ccd70 con 0x7f62b415bf00 2024-04-25T10:10:04.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.657+0000 7f62b9a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b415cca0 con 0x7f62b40cc310 2024-04-25T10:10:04.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.657+0000 7f62b3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b415c820 0x7f62b41e5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.658+0000 7f62b3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b415c820 0x7f62b41e5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39236/0 (socket says 172.21.15.119:39236) 2024-04-25T10:10:04.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.658+0000 7f62b3400640 1 -- 172.21.15.119:0/4251384529 learned_addr learned my addr 172.21.15.119:0/4251384529 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:04.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.658+0000 7f62b2a00640 1 --2- 172.21.15.119:0/4251384529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b40cc310 0x7f62b41f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.658+0000 7f62abe00640 1 --2- 172.21.15.119:0/4251384529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b415bf00 0x7f62b414bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.658+0000 7f62b3400640 1 -- 172.21.15.119:0/4251384529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b40cc310 msgr2=0x7f62b41f18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.658+0000 7f62b3400640 1 --2- 172.21.15.119:0/4251384529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b40cc310 0x7f62b41f18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.658+0000 7f62b3400640 1 -- 172.21.15.119:0/4251384529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b415bf00 msgr2=0x7f62b414bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.658+0000 7f62b3400640 1 --2- 172.21.15.119:0/4251384529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b415bf00 0x7f62b414bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.658+0000 7f62b3400640 1 -- 172.21.15.119:0/4251384529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b414c450 con 0x7f62b415c820 2024-04-25T10:10:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.658+0000 7f62b3400640 1 --2- 172.21.15.119:0/4251384529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b415c820 0x7f62b41e5250 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00c6b0 tx=0x7f62ac00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.659+0000 7f62aaa00640 1 -- 172.21.15.119:0/4251384529 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac003b70 con 0x7f62b415c820 2024-04-25T10:10:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.659+0000 7f62aaa00640 1 -- 172.21.15.119:0/4251384529 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac003d30 con 0x7f62b415c820 2024-04-25T10:10:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.659+0000 7f62b9a00640 1 -- 172.21.15.119:0/4251384529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b41e5a80 con 0x7f62b415c820 2024-04-25T10:10:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.659+0000 7f62aaa00640 1 -- 172.21.15.119:0/4251384529 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac005310 con 0x7f62b415c820 2024-04-25T10:10:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.659+0000 7f62b9a00640 1 -- 172.21.15.119:0/4251384529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b41e6040 con 0x7f62b415c820 2024-04-25T10:10:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.660+0000 7f62b9a00640 1 -- 172.21.15.119:0/4251384529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b406bb30 con 0x7f62b415c820 2024-04-25T10:10:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.661+0000 7f62aaa00640 1 -- 172.21.15.119:0/4251384529 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62ac0054b0 con 0x7f62b415c820 2024-04-25T10:10:04.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.662+0000 7f62aaa00640 1 --2- 172.21.15.119:0/4251384529 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f629006b4d0 0x7f629006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.662+0000 7f62aaa00640 1 -- 172.21.15.119:0/4251384529 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62ac08a1b0 con 0x7f62b415c820 2024-04-25T10:10:04.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.663+0000 7f62b2a00640 1 --2- 172.21.15.119:0/4251384529 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f629006b4d0 0x7f629006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.663+0000 7f62b2a00640 1 --2- 172.21.15.119:0/4251384529 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f629006b4d0 0x7f629006d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009ad0 tx=0x7f62a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.667+0000 7f62aaa00640 1 -- 172.21.15.119:0/4251384529 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62b40ce6c0 con 0x7f62b415c820 2024-04-25T10:10:04.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.843+0000 7f62b9a00640 1 -- 172.21.15.119:0/4251384529 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62b415b9a0 con 0x7f629006b4d0 2024-04-25T10:10:04.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.856+0000 7f62aaa00640 1 -- 172.21.15.119:0/4251384529 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251887 (secure 0 0 0) 0x7f62b415b9a0 con 0x7f629006b4d0 2024-04-25T10:10:04.867 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:10:04.869 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:10:04.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.860+0000 7f62b9a00640 1 -- 172.21.15.119:0/4251384529 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f629006b4d0 msgr2=0x7f629006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.860+0000 7f62b9a00640 1 --2- 172.21.15.119:0/4251384529 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f629006b4d0 0x7f629006d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009ad0 tx=0x7f62a4008040 comp rx=0 tx=0).stop 2024-04-25T10:10:04.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.860+0000 7f62b9a00640 1 -- 172.21.15.119:0/4251384529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b415c820 msgr2=0x7f62b41e5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.860+0000 7f62b9a00640 1 --2- 172.21.15.119:0/4251384529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b415c820 0x7f62b41e5250 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00c6b0 tx=0x7f62ac00cb70 comp rx=0 tx=0).stop 2024-04-25T10:10:04.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.860+0000 7f62b9a00640 1 -- 172.21.15.119:0/4251384529 shutdown_connections 2024-04-25T10:10:04.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.860+0000 7f62b9a00640 1 --2- 172.21.15.119:0/4251384529 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f629006b4d0 0x7f629006d990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.860+0000 7f62b9a00640 1 --2- 172.21.15.119:0/4251384529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b415c820 0x7f62b41e5250 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.860+0000 7f62b9a00640 1 --2- 172.21.15.119:0/4251384529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b415bf00 0x7f62b414bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.860+0000 7f62b9a00640 1 --2- 172.21.15.119:0/4251384529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f62b40cc310 0x7f62b41f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.860+0000 7f62b9a00640 1 -- 172.21.15.119:0/4251384529 >> 172.21.15.119:0/4251384529 conn(0x7f62b406aba0 msgr2=0x7f62b406d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.861+0000 7f62b9a00640 1 -- 172.21.15.119:0/4251384529 shutdown_connections 2024-04-25T10:10:04.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:04.861+0000 7f62b9a00640 1 -- 172.21.15.119:0/4251384529 wait complete. 2024-04-25T10:10:04.885 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-04-25T10:10:03.981655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247400,"kb_used_data":6272,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727320,"statfs":{"total":773094113280,"available":772840775680,"internally_reserved":0,"allocated":6422528,"data_stored":3317892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":246795826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.996347"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.055229+0000","last_change":"2024-04-25T10:10:00.053457+0000","last_active":"2024-04-25T10:10:00.055229+0000","last_peered":"2024-04-25T10:10:00.055229+0000","last_clean":"2024-04-25T10:10:00.055229+0000","last_became_active":"2024-04-25T10:10:00.053133+0000","last_became_peered":"2024-04-25T10:10:00.053133+0000","last_unstale":"2024-04-25T10:10:00.055229+0000","last_undegraded":"2024-04-25T10:10:00.055229+0000","last_fullsized":"2024-04-25T10:10:00.055229+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:06.634329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.072124+0000","last_change":"2024-04-25T10:10:00.070765+0000","last_active":"2024-04-25T10:10:00.072124+0000","last_peered":"2024-04-25T10:10:00.072124+0000","last_clean":"2024-04-25T10:10:00.072124+0000","last_became_active":"2024-04-25T10:10:00.069850+0000","last_became_peered":"2024-04-25T10:10:00.069850+0000","last_unstale":"2024-04-25T10:10:00.072124+0000","last_undegraded":"2024-04-25T10:10:00.072124+0000","last_fullsized":"2024-04-25T10:10:00.072124+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:01.400797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.055725+0000","last_change":"2024-04-25T10:10:00.051148+0000","last_active":"2024-04-25T10:10:00.055725+0000","last_peered":"2024-04-25T10:10:00.055725+0000","last_clean":"2024-04-25T10:10:00.055725+0000","last_became_active":"2024-04-25T10:10:00.049877+0000","last_became_peered":"2024-04-25T10:10:00.049877+0000","last_unstale":"2024-04-25T10:10:00.055725+0000","last_undegraded":"2024-04-25T10:10:00.055725+0000","last_fullsized":"2024-04-25T10:10:00.055725+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:43.167665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.049554+0000","last_change":"2024-04-25T10:10:00.043935+0000","last_active":"2024-04-25T10:10:00.049554+0000","last_peered":"2024-04-25T10:10:00.049554+0000","last_clean":"2024-04-25T10:10:00.049554+0000","last_became_active":"2024-04-25T10:10:00.043586+0000","last_became_peered":"2024-04-25T10:10:00.043586+0000","last_unstale":"2024-04-25T10:10:00.049554+0000","last_undegraded":"2024-04-25T10:10:00.049554+0000","last_fullsized":"2024-04-25T10:10:00.049554+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:11.916038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.042127+0000","last_change":"2024-04-25T10:10:00.038561+0000","last_active":"2024-04-25T10:10:00.042127+0000","last_peered":"2024-04-25T10:10:00.042127+0000","last_clean":"2024-04-25T10:10:00.042127+0000","last_became_active":"2024-04-25T10:10:00.038268+0000","last_became_peered":"2024-04-25T10:10:00.038268+0000","last_unstale":"2024-04-25T10:10:00.042127+0000","last_undegraded":"2024-04-25T10:10:00.042127+0000","last_fullsized":"2024-04-25T10:10:00.042127+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:19.061172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.051316+0000","last_change":"2024-04-25T10:10:00.047240+0000","last_active":"2024-04-25T10:10:00.051316+0000","last_peered":"2024-04-25T10:10:00.051316+0000","last_clean":"2024-04-25T10:10:00.051316+0000","last_became_active":"2024-04-25T10:10:00.046844+0000","last_became_peered":"2024-04-25T10:10:00.046844+0000","last_unstale":"2024-04-25T10:10:00.051316+0000","last_undegraded":"2024-04-25T10:10:00.051316+0000","last_fullsized":"2024-04-25T10:10:00.051316+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:27.493217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.056543+0000","last_change":"2024-04-25T10:10:00.053162+0000","last_active":"2024-04-25T10:10:00.056543+0000","last_peered":"2024-04-25T10:10:00.056543+0000","last_clean":"2024-04-25T10:10:00.056543+0000","last_became_active":"2024-04-25T10:10:00.051836+0000","last_became_peered":"2024-04-25T10:10:00.051836+0000","last_unstale":"2024-04-25T10:10:00.056543+0000","last_undegraded":"2024-04-25T10:10:00.056543+0000","last_fullsized":"2024-04-25T10:10:00.056543+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:55.090376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.064489+0000","last_change":"2024-04-25T10:10:00.053626+0000","last_active":"2024-04-25T10:10:00.064489+0000","last_peered":"2024-04-25T10:10:00.064489+0000","last_clean":"2024-04-25T10:10:00.064489+0000","last_became_active":"2024-04-25T10:10:00.053333+0000","last_became_peered":"2024-04-25T10:10:00.053333+0000","last_unstale":"2024-04-25T10:10:00.064489+0000","last_undegraded":"2024-04-25T10:10:00.064489+0000","last_fullsized":"2024-04-25T10:10:00.064489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:06.399299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.055394+0000","last_change":"2024-04-25T10:10:00.050684+0000","last_active":"2024-04-25T10:10:00.055394+0000","last_peered":"2024-04-25T10:10:00.055394+0000","last_clean":"2024-04-25T10:10:00.055394+0000","last_became_active":"2024-04-25T10:10:00.049996+0000","last_became_peered":"2024-04-25T10:10:00.049996+0000","last_unstale":"2024-04-25T10:10:00.055394+0000","last_undegraded":"2024-04-25T10:10:00.055394+0000","last_fullsized":"2024-04-25T10:10:00.055394+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:47.304806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.068976+0000","last_change":"2024-04-25T10:10:00.062367+0000","last_active":"2024-04-25T10:10:00.068976+0000","last_peered":"2024-04-25T10:10:00.068976+0000","last_clean":"2024-04-25T10:10:00.068976+0000","last_became_active":"2024-04-25T10:10:00.062011+0000","last_became_peered":"2024-04-25T10:10:00.062011+0000","last_unstale":"2024-04-25T10:10:00.068976+0000","last_undegraded":"2024-04-25T10:10:00.068976+0000","last_fullsized":"2024-04-25T10:10:00.068976+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:55.211323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.073609+0000","last_change":"2024-04-25T10:10:00.067572+0000","last_active":"2024-04-25T10:10:00.073609+0000","last_peered":"2024-04-25T10:10:00.073609+0000","last_clean":"2024-04-25T10:10:00.073609+0000","last_became_active":"2024-04-25T10:10:00.067270+0000","last_became_peered":"2024-04-25T10:10:00.067270+0000","last_unstale":"2024-04-25T10:10:00.073609+0000","last_undegraded":"2024-04-25T10:10:00.073609+0000","last_fullsized":"2024-04-25T10:10:00.073609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:58.948070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.052839+0000","last_change":"2024-04-25T10:10:00.048602+0000","last_active":"2024-04-25T10:10:00.052839+0000","last_peered":"2024-04-25T10:10:00.052839+0000","last_clean":"2024-04-25T10:10:00.052839+0000","last_became_active":"2024-04-25T10:10:00.048139+0000","last_became_peered":"2024-04-25T10:10:00.048139+0000","last_unstale":"2024-04-25T10:10:00.052839+0000","last_undegraded":"2024-04-25T10:10:00.052839+0000","last_fullsized":"2024-04-25T10:10:00.052839+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:08.780623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.053281+0000","last_change":"2024-04-25T10:10:00.048588+0000","last_active":"2024-04-25T10:10:00.053281+0000","last_peered":"2024-04-25T10:10:00.053281+0000","last_clean":"2024-04-25T10:10:00.053281+0000","last_became_active":"2024-04-25T10:10:00.048253+0000","last_became_peered":"2024-04-25T10:10:00.048253+0000","last_unstale":"2024-04-25T10:10:00.053281+0000","last_undegraded":"2024-04-25T10:10:00.053281+0000","last_fullsized":"2024-04-25T10:10:00.053281+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:35.833226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.054099+0000","last_change":"2024-04-25T10:10:00.047721+0000","last_active":"2024-04-25T10:10:00.054099+0000","last_peered":"2024-04-25T10:10:00.054099+0000","last_clean":"2024-04-25T10:10:00.054099+0000","last_became_active":"2024-04-25T10:10:00.046897+0000","last_became_peered":"2024-04-25T10:10:00.046897+0000","last_unstale":"2024-04-25T10:10:00.054099+0000","last_undegraded":"2024-04-25T10:10:00.054099+0000","last_fullsized":"2024-04-25T10:10:00.054099+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:35.012581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.053129+0000","last_change":"2024-04-25T10:10:00.047661+0000","last_active":"2024-04-25T10:10:00.053129+0000","last_peered":"2024-04-25T10:10:00.053129+0000","last_clean":"2024-04-25T10:10:00.053129+0000","last_became_active":"2024-04-25T10:10:00.047272+0000","last_became_peered":"2024-04-25T10:10:00.047272+0000","last_unstale":"2024-04-25T10:10:00.053129+0000","last_undegraded":"2024-04-25T10:10:00.053129+0000","last_fullsized":"2024-04-25T10:10:00.053129+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:33.568858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.060113+0000","last_change":"2024-04-25T10:10:00.053674+0000","last_active":"2024-04-25T10:10:00.060113+0000","last_peered":"2024-04-25T10:10:00.060113+0000","last_clean":"2024-04-25T10:10:00.060113+0000","last_became_active":"2024-04-25T10:10:00.053306+0000","last_became_peered":"2024-04-25T10:10:00.053306+0000","last_unstale":"2024-04-25T10:10:00.060113+0000","last_undegraded":"2024-04-25T10:10:00.060113+0000","last_fullsized":"2024-04-25T10:10:00.060113+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:53.194568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.061577+0000","last_change":"2024-04-25T10:10:00.050551+0000","last_active":"2024-04-25T10:10:00.061577+0000","last_peered":"2024-04-25T10:10:00.061577+0000","last_clean":"2024-04-25T10:10:00.061577+0000","last_became_active":"2024-04-25T10:10:00.050257+0000","last_became_peered":"2024-04-25T10:10:00.050257+0000","last_unstale":"2024-04-25T10:10:00.061577+0000","last_undegraded":"2024-04-25T10:10:00.061577+0000","last_fullsized":"2024-04-25T10:10:00.061577+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:30.886789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.055719+0000","last_change":"2024-04-25T10:10:00.050184+0000","last_active":"2024-04-25T10:10:00.055719+0000","last_peered":"2024-04-25T10:10:00.055719+0000","last_clean":"2024-04-25T10:10:00.055719+0000","last_became_active":"2024-04-25T10:10:00.049289+0000","last_became_peered":"2024-04-25T10:10:00.049289+0000","last_unstale":"2024-04-25T10:10:00.055719+0000","last_undegraded":"2024-04-25T10:10:00.055719+0000","last_fullsized":"2024-04-25T10:10:00.055719+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:00.349263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.073676+0000","last_change":"2024-04-25T10:10:00.066161+0000","last_active":"2024-04-25T10:10:00.073676+0000","last_peered":"2024-04-25T10:10:00.073676+0000","last_clean":"2024-04-25T10:10:00.073676+0000","last_became_active":"2024-04-25T10:10:00.065307+0000","last_became_peered":"2024-04-25T10:10:00.065307+0000","last_unstale":"2024-04-25T10:10:00.073676+0000","last_undegraded":"2024-04-25T10:10:00.073676+0000","last_fullsized":"2024-04-25T10:10:00.073676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:53.536215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.063913+0000","last_change":"2024-04-25T10:10:00.050827+0000","last_active":"2024-04-25T10:10:00.063913+0000","last_peered":"2024-04-25T10:10:00.063913+0000","last_clean":"2024-04-25T10:10:00.063913+0000","last_became_active":"2024-04-25T10:10:00.050085+0000","last_became_peered":"2024-04-25T10:10:00.050085+0000","last_unstale":"2024-04-25T10:10:00.063913+0000","last_undegraded":"2024-04-25T10:10:00.063913+0000","last_fullsized":"2024-04-25T10:10:00.063913+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:17.085267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.046903+0000","last_change":"2024-04-25T10:10:00.037326+0000","last_active":"2024-04-25T10:10:00.046903+0000","last_peered":"2024-04-25T10:10:00.046903+0000","last_clean":"2024-04-25T10:10:00.046903+0000","last_became_active":"2024-04-25T10:10:00.036577+0000","last_became_peered":"2024-04-25T10:10:00.036577+0000","last_unstale":"2024-04-25T10:10:00.046903+0000","last_undegraded":"2024-04-25T10:10:00.046903+0000","last_fullsized":"2024-04-25T10:10:00.046903+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:20.130812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.065600+0000","last_change":"2024-04-25T10:10:00.061812+0000","last_active":"2024-04-25T10:10:00.065600+0000","last_peered":"2024-04-25T10:10:00.065600+0000","last_clean":"2024-04-25T10:10:00.065600+0000","last_became_active":"2024-04-25T10:10:00.054477+0000","last_became_peered":"2024-04-25T10:10:00.054477+0000","last_unstale":"2024-04-25T10:10:00.065600+0000","last_undegraded":"2024-04-25T10:10:00.065600+0000","last_fullsized":"2024-04-25T10:10:00.065600+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:49.348976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.067496+0000","last_change":"2024-04-25T10:10:00.066525+0000","last_active":"2024-04-25T10:10:00.067496+0000","last_peered":"2024-04-25T10:10:00.067496+0000","last_clean":"2024-04-25T10:10:00.067496+0000","last_became_active":"2024-04-25T10:10:00.066243+0000","last_became_peered":"2024-04-25T10:10:00.066243+0000","last_unstale":"2024-04-25T10:10:00.067496+0000","last_undegraded":"2024-04-25T10:10:00.067496+0000","last_fullsized":"2024-04-25T10:10:00.067496+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:27.493217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.057735+0000","last_change":"2024-04-25T10:10:00.055056+0000","last_active":"2024-04-25T10:10:00.057735+0000","last_peered":"2024-04-25T10:10:00.057735+0000","last_clean":"2024-04-25T10:10:00.057735+0000","last_became_active":"2024-04-25T10:10:00.054591+0000","last_became_peered":"2024-04-25T10:10:00.054591+0000","last_unstale":"2024-04-25T10:10:00.057735+0000","last_undegraded":"2024-04-25T10:10:00.057735+0000","last_fullsized":"2024-04-25T10:10:00.057735+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:58.948070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.064597+0000","last_change":"2024-04-25T10:10:00.063307+0000","last_active":"2024-04-25T10:10:00.064597+0000","last_peered":"2024-04-25T10:10:00.064597+0000","last_clean":"2024-04-25T10:10:00.064597+0000","last_became_active":"2024-04-25T10:10:00.062961+0000","last_became_peered":"2024-04-25T10:10:00.062961+0000","last_unstale":"2024-04-25T10:10:00.064597+0000","last_undegraded":"2024-04-25T10:10:00.064597+0000","last_fullsized":"2024-04-25T10:10:00.064597+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:14.928259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.051650+0000","last_change":"2024-04-25T10:10:00.046736+0000","last_active":"2024-04-25T10:10:00.051650+0000","last_peered":"2024-04-25T10:10:00.051650+0000","last_clean":"2024-04-25T10:10:00.051650+0000","last_became_active":"2024-04-25T10:10:00.046342+0000","last_became_peered":"2024-04-25T10:10:00.046342+0000","last_unstale":"2024-04-25T10:10:00.051650+0000","last_undegraded":"2024-04-25T10:10:00.051650+0000","last_fullsized":"2024-04-25T10:10:00.051650+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:43.167665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.052766+0000","last_change":"2024-04-25T10:10:00.048566+0000","last_active":"2024-04-25T10:10:00.052766+0000","last_peered":"2024-04-25T10:10:00.052766+0000","last_clean":"2024-04-25T10:10:00.052766+0000","last_became_active":"2024-04-25T10:10:00.048285+0000","last_became_peered":"2024-04-25T10:10:00.048285+0000","last_unstale":"2024-04-25T10:10:00.052766+0000","last_undegraded":"2024-04-25T10:10:00.052766+0000","last_fullsized":"2024-04-25T10:10:00.052766+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:03.609745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.053737+0000","last_change":"2024-04-25T10:10:00.037092+0000","last_active":"2024-04-25T10:10:00.053737+0000","last_peered":"2024-04-25T10:10:00.053737+0000","last_clean":"2024-04-25T10:10:00.053737+0000","last_became_active":"2024-04-25T10:10:00.036405+0000","last_became_peered":"2024-04-25T10:10:00.036405+0000","last_unstale":"2024-04-25T10:10:00.053737+0000","last_undegraded":"2024-04-25T10:10:00.053737+0000","last_fullsized":"2024-04-25T10:10:00.053737+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:37.894419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.063866+0000","last_change":"2024-04-25T10:10:00.051336+0000","last_active":"2024-04-25T10:10:00.063866+0000","last_peered":"2024-04-25T10:10:00.063866+0000","last_clean":"2024-04-25T10:10:00.063866+0000","last_became_active":"2024-04-25T10:10:00.050640+0000","last_became_peered":"2024-04-25T10:10:00.050640+0000","last_unstale":"2024-04-25T10:10:00.063866+0000","last_undegraded":"2024-04-25T10:10:00.063866+0000","last_fullsized":"2024-04-25T10:10:00.063866+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:17.212811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.055498+0000","last_change":"2024-04-25T10:10:00.050180+0000","last_active":"2024-04-25T10:10:00.055498+0000","last_peered":"2024-04-25T10:10:00.055498+0000","last_clean":"2024-04-25T10:10:00.055498+0000","last_became_active":"2024-04-25T10:10:00.049908+0000","last_became_peered":"2024-04-25T10:10:00.049908+0000","last_unstale":"2024-04-25T10:10:00.055498+0000","last_undegraded":"2024-04-25T10:10:00.055498+0000","last_fullsized":"2024-04-25T10:10:00.055498+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:47.405959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.049093+0000","last_change":"2024-04-25T10:10:00.046517+0000","last_active":"2024-04-25T10:10:00.049093+0000","last_peered":"2024-04-25T10:10:00.049093+0000","last_clean":"2024-04-25T10:10:00.049093+0000","last_became_active":"2024-04-25T10:10:00.046158+0000","last_became_peered":"2024-04-25T10:10:00.046158+0000","last_unstale":"2024-04-25T10:10:00.049093+0000","last_undegraded":"2024-04-25T10:10:00.049093+0000","last_fullsized":"2024-04-25T10:10:00.049093+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:29.041790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.062791+0000","last_change":"2024-04-25T10:10:00.060790+0000","last_active":"2024-04-25T10:10:00.062791+0000","last_peered":"2024-04-25T10:10:00.062791+0000","last_clean":"2024-04-25T10:10:00.062791+0000","last_became_active":"2024-04-25T10:10:00.060299+0000","last_became_peered":"2024-04-25T10:10:00.060299+0000","last_unstale":"2024-04-25T10:10:00.062791+0000","last_undegraded":"2024-04-25T10:10:00.062791+0000","last_fullsized":"2024-04-25T10:10:00.062791+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:36.133119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.057614+0000","last_change":"2024-04-25T10:10:00.054412+0000","last_active":"2024-04-25T10:10:00.057614+0000","last_peered":"2024-04-25T10:10:00.057614+0000","last_clean":"2024-04-25T10:10:00.057614+0000","last_became_active":"2024-04-25T10:10:00.053801+0000","last_became_peered":"2024-04-25T10:10:00.053801+0000","last_unstale":"2024-04-25T10:10:00.057614+0000","last_undegraded":"2024-04-25T10:10:00.057614+0000","last_fullsized":"2024-04-25T10:10:00.057614+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:42.883016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.081347+0000","last_change":"2024-04-25T10:09:59.019152+0000","last_active":"2024-04-25T10:10:00.081347+0000","last_peered":"2024-04-25T10:10:00.081347+0000","last_clean":"2024-04-25T10:10:00.081347+0000","last_became_active":"2024-04-25T10:09:59.018544+0000","last_became_peered":"2024-04-25T10:09:59.018544+0000","last_unstale":"2024-04-25T10:10:00.081347+0000","last_undegraded":"2024-04-25T10:10:00.081347+0000","last_fullsized":"2024-04-25T10:10:00.081347+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:10.902662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.069094+0000","last_change":"2024-04-25T10:10:00.062197+0000","last_active":"2024-04-25T10:10:00.069094+0000","last_peered":"2024-04-25T10:10:00.069094+0000","last_clean":"2024-04-25T10:10:00.069094+0000","last_became_active":"2024-04-25T10:10:00.061589+0000","last_became_peered":"2024-04-25T10:10:00.061589+0000","last_unstale":"2024-04-25T10:10:00.069094+0000","last_undegraded":"2024-04-25T10:10:00.069094+0000","last_fullsized":"2024-04-25T10:10:00.069094+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:59.209848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.080168+0000","last_change":"2024-04-25T10:09:59.023773+0000","last_active":"2024-04-25T10:10:00.080168+0000","last_peered":"2024-04-25T10:10:00.080168+0000","last_clean":"2024-04-25T10:10:00.080168+0000","last_became_active":"2024-04-25T10:09:59.023445+0000","last_became_peered":"2024-04-25T10:09:59.023445+0000","last_unstale":"2024-04-25T10:10:00.080168+0000","last_undegraded":"2024-04-25T10:10:00.080168+0000","last_fullsized":"2024-04-25T10:10:00.080168+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:07.057321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.058802+0000","last_change":"2024-04-25T10:10:00.057488+0000","last_active":"2024-04-25T10:10:00.058802+0000","last_peered":"2024-04-25T10:10:00.058802+0000","last_clean":"2024-04-25T10:10:00.058802+0000","last_became_active":"2024-04-25T10:10:00.057153+0000","last_became_peered":"2024-04-25T10:10:00.057153+0000","last_unstale":"2024-04-25T10:10:00.058802+0000","last_undegraded":"2024-04-25T10:10:00.058802+0000","last_fullsized":"2024-04-25T10:10:00.058802+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:30.044417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.084311+0000","last_change":"2024-04-25T10:09:59.021669+0000","last_active":"2024-04-25T10:10:00.084311+0000","last_peered":"2024-04-25T10:10:00.084311+0000","last_clean":"2024-04-25T10:10:00.084311+0000","last_became_active":"2024-04-25T10:09:59.021431+0000","last_became_peered":"2024-04-25T10:09:59.021431+0000","last_unstale":"2024-04-25T10:10:00.084311+0000","last_undegraded":"2024-04-25T10:10:00.084311+0000","last_fullsized":"2024-04-25T10:10:00.084311+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:14.130629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.051278+0000","last_change":"2024-04-25T10:10:00.047739+0000","last_active":"2024-04-25T10:10:00.051278+0000","last_peered":"2024-04-25T10:10:00.051278+0000","last_clean":"2024-04-25T10:10:00.051278+0000","last_became_active":"2024-04-25T10:10:00.045327+0000","last_became_peered":"2024-04-25T10:10:00.045327+0000","last_unstale":"2024-04-25T10:10:00.051278+0000","last_undegraded":"2024-04-25T10:10:00.051278+0000","last_fullsized":"2024-04-25T10:10:00.051278+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:46.705938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.080424+0000","last_change":"2024-04-25T10:09:59.024931+0000","last_active":"2024-04-25T10:10:00.080424+0000","last_peered":"2024-04-25T10:10:00.080424+0000","last_clean":"2024-04-25T10:10:00.080424+0000","last_became_active":"2024-04-25T10:09:59.024633+0000","last_became_peered":"2024-04-25T10:09:59.024633+0000","last_unstale":"2024-04-25T10:10:00.080424+0000","last_undegraded":"2024-04-25T10:10:00.080424+0000","last_fullsized":"2024-04-25T10:10:00.080424+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:46.291430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.061968+0000","last_change":"2024-04-25T10:10:00.055059+0000","last_active":"2024-04-25T10:10:00.061968+0000","last_peered":"2024-04-25T10:10:00.061968+0000","last_clean":"2024-04-25T10:10:00.061968+0000","last_became_active":"2024-04-25T10:10:00.054073+0000","last_became_peered":"2024-04-25T10:10:00.054073+0000","last_unstale":"2024-04-25T10:10:00.061968+0000","last_undegraded":"2024-04-25T10:10:00.061968+0000","last_fullsized":"2024-04-25T10:10:00.061968+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:53.241996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.079984+0000","last_change":"2024-04-25T10:09:59.022940+0000","last_active":"2024-04-25T10:10:00.079984+0000","last_peered":"2024-04-25T10:10:00.079984+0000","last_clean":"2024-04-25T10:10:00.079984+0000","last_became_active":"2024-04-25T10:09:59.022081+0000","last_became_peered":"2024-04-25T10:09:59.022081+0000","last_unstale":"2024-04-25T10:10:00.079984+0000","last_undegraded":"2024-04-25T10:10:00.079984+0000","last_fullsized":"2024-04-25T10:10:00.079984+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:38.922189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.063888+0000","last_change":"2024-04-25T10:10:00.051568+0000","last_active":"2024-04-25T10:10:00.063888+0000","last_peered":"2024-04-25T10:10:00.063888+0000","last_clean":"2024-04-25T10:10:00.063888+0000","last_became_active":"2024-04-25T10:10:00.050972+0000","last_became_peered":"2024-04-25T10:10:00.050972+0000","last_unstale":"2024-04-25T10:10:00.063888+0000","last_undegraded":"2024-04-25T10:10:00.063888+0000","last_fullsized":"2024-04-25T10:10:00.063888+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:13.553159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.079301+0000","last_change":"2024-04-25T10:09:59.023429+0000","last_active":"2024-04-25T10:10:00.079301+0000","last_peered":"2024-04-25T10:10:00.079301+0000","last_clean":"2024-04-25T10:10:00.079301+0000","last_became_active":"2024-04-25T10:09:59.023200+0000","last_became_peered":"2024-04-25T10:09:59.023200+0000","last_unstale":"2024-04-25T10:10:00.079301+0000","last_undegraded":"2024-04-25T10:10:00.079301+0000","last_fullsized":"2024-04-25T10:10:00.079301+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:07.767247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.055709+0000","last_change":"2024-04-25T10:10:00.050773+0000","last_active":"2024-04-25T10:10:00.055709+0000","last_peered":"2024-04-25T10:10:00.055709+0000","last_clean":"2024-04-25T10:10:00.055709+0000","last_became_active":"2024-04-25T10:10:00.050472+0000","last_became_peered":"2024-04-25T10:10:00.050472+0000","last_unstale":"2024-04-25T10:10:00.055709+0000","last_undegraded":"2024-04-25T10:10:00.055709+0000","last_fullsized":"2024-04-25T10:10:00.055709+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:52.233499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.025430+0000","last_change":"2024-04-25T10:09:59.019986+0000","last_active":"2024-04-25T10:10:00.025430+0000","last_peered":"2024-04-25T10:10:00.025430+0000","last_clean":"2024-04-25T10:10:00.025430+0000","last_became_active":"2024-04-25T10:09:59.019673+0000","last_became_peered":"2024-04-25T10:09:59.019673+0000","last_unstale":"2024-04-25T10:10:00.025430+0000","last_undegraded":"2024-04-25T10:10:00.025430+0000","last_fullsized":"2024-04-25T10:10:00.025430+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:19.117436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.066670+0000","last_change":"2024-04-25T10:10:00.063283+0000","last_active":"2024-04-25T10:10:00.066670+0000","last_peered":"2024-04-25T10:10:00.066670+0000","last_clean":"2024-04-25T10:10:00.066670+0000","last_became_active":"2024-04-25T10:10:00.062951+0000","last_became_peered":"2024-04-25T10:10:00.062951+0000","last_unstale":"2024-04-25T10:10:00.066670+0000","last_undegraded":"2024-04-25T10:10:00.066670+0000","last_fullsized":"2024-04-25T10:10:00.066670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:20.254391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.039867+0000","last_change":"2024-04-25T10:09:59.024899+0000","last_active":"2024-04-25T10:10:00.039867+0000","last_peered":"2024-04-25T10:10:00.039867+0000","last_clean":"2024-04-25T10:10:00.039867+0000","last_became_active":"2024-04-25T10:09:59.024563+0000","last_became_peered":"2024-04-25T10:09:59.024563+0000","last_unstale":"2024-04-25T10:10:00.039867+0000","last_undegraded":"2024-04-25T10:10:00.039867+0000","last_fullsized":"2024-04-25T10:10:00.039867+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:54.437913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.063901+0000","last_change":"2024-04-25T10:10:00.062134+0000","last_active":"2024-04-25T10:10:00.063901+0000","last_peered":"2024-04-25T10:10:00.063901+0000","last_clean":"2024-04-25T10:10:00.063901+0000","last_became_active":"2024-04-25T10:10:00.061773+0000","last_became_peered":"2024-04-25T10:10:00.061773+0000","last_unstale":"2024-04-25T10:10:00.063901+0000","last_undegraded":"2024-04-25T10:10:00.063901+0000","last_fullsized":"2024-04-25T10:10:00.063901+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:17.363219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.031216+0000","last_change":"2024-04-25T10:09:59.030128+0000","last_active":"2024-04-25T10:10:00.031216+0000","last_peered":"2024-04-25T10:10:00.031216+0000","last_clean":"2024-04-25T10:10:00.031216+0000","last_became_active":"2024-04-25T10:09:59.029234+0000","last_became_peered":"2024-04-25T10:09:59.029234+0000","last_unstale":"2024-04-25T10:10:00.031216+0000","last_undegraded":"2024-04-25T10:10:00.031216+0000","last_fullsized":"2024-04-25T10:10:00.031216+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:54.077000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.071315+0000","last_change":"2024-04-25T10:10:00.070101+0000","last_active":"2024-04-25T10:10:00.071315+0000","last_peered":"2024-04-25T10:10:00.071315+0000","last_clean":"2024-04-25T10:10:00.071315+0000","last_became_active":"2024-04-25T10:10:00.069652+0000","last_became_peered":"2024-04-25T10:10:00.069652+0000","last_unstale":"2024-04-25T10:10:00.071315+0000","last_undegraded":"2024-04-25T10:10:00.071315+0000","last_fullsized":"2024-04-25T10:10:00.071315+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:03.609745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.040082+0000","last_change":"2024-04-25T10:09:59.010343+0000","last_active":"2024-04-25T10:10:00.040082+0000","last_peered":"2024-04-25T10:10:00.040082+0000","last_clean":"2024-04-25T10:10:00.040082+0000","last_became_active":"2024-04-25T10:09:59.010088+0000","last_became_peered":"2024-04-25T10:09:59.010088+0000","last_unstale":"2024-04-25T10:10:00.040082+0000","last_undegraded":"2024-04-25T10:10:00.040082+0000","last_fullsized":"2024-04-25T10:10:00.040082+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:29.031041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.058788+0000","last_change":"2024-04-25T10:10:00.055963+0000","last_active":"2024-04-25T10:10:00.058788+0000","last_peered":"2024-04-25T10:10:00.058788+0000","last_clean":"2024-04-25T10:10:00.058788+0000","last_became_active":"2024-04-25T10:10:00.055642+0000","last_became_peered":"2024-04-25T10:10:00.055642+0000","last_unstale":"2024-04-25T10:10:00.058788+0000","last_undegraded":"2024-04-25T10:10:00.058788+0000","last_fullsized":"2024-04-25T10:10:00.058788+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:58.932589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.084357+0000","last_change":"2024-04-25T10:09:59.020756+0000","last_active":"2024-04-25T10:10:00.084357+0000","last_peered":"2024-04-25T10:10:00.084357+0000","last_clean":"2024-04-25T10:10:00.084357+0000","last_became_active":"2024-04-25T10:09:59.020428+0000","last_became_peered":"2024-04-25T10:09:59.020428+0000","last_unstale":"2024-04-25T10:10:00.084357+0000","last_undegraded":"2024-04-25T10:10:00.084357+0000","last_fullsized":"2024-04-25T10:10:00.084357+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:16.349844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.072050+0000","last_change":"2024-04-25T10:10:00.070507+0000","last_active":"2024-04-25T10:10:00.072050+0000","last_peered":"2024-04-25T10:10:00.072050+0000","last_clean":"2024-04-25T10:10:00.072050+0000","last_became_active":"2024-04-25T10:10:00.069786+0000","last_became_peered":"2024-04-25T10:10:00.069786+0000","last_unstale":"2024-04-25T10:10:00.072050+0000","last_undegraded":"2024-04-25T10:10:00.072050+0000","last_fullsized":"2024-04-25T10:10:00.072050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:52.968116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"54'6","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.087007+0000","last_change":"2024-04-25T10:09:59.028172+0000","last_active":"2024-04-25T10:10:00.087007+0000","last_peered":"2024-04-25T10:10:00.087007+0000","last_clean":"2024-04-25T10:10:00.087007+0000","last_became_active":"2024-04-25T10:09:59.027692+0000","last_became_peered":"2024-04-25T10:09:59.027692+0000","last_unstale":"2024-04-25T10:10:00.087007+0000","last_undegraded":"2024-04-25T10:10:00.087007+0000","last_fullsized":"2024-04-25T10:10:00.087007+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:21.748509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.064237+0000","last_change":"2024-04-25T10:10:00.063101+0000","last_active":"2024-04-25T10:10:00.064237+0000","last_peered":"2024-04-25T10:10:00.064237+0000","last_clean":"2024-04-25T10:10:00.064237+0000","last_became_active":"2024-04-25T10:10:00.062455+0000","last_became_peered":"2024-04-25T10:10:00.062455+0000","last_unstale":"2024-04-25T10:10:00.064237+0000","last_undegraded":"2024-04-25T10:10:00.064237+0000","last_fullsized":"2024-04-25T10:10:00.064237+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:27.493217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.084609+0000","last_change":"2024-04-25T10:09:59.021295+0000","last_active":"2024-04-25T10:10:00.084609+0000","last_peered":"2024-04-25T10:10:00.084609+0000","last_clean":"2024-04-25T10:10:00.084609+0000","last_became_active":"2024-04-25T10:09:59.020973+0000","last_became_peered":"2024-04-25T10:09:59.020973+0000","last_unstale":"2024-04-25T10:10:00.084609+0000","last_undegraded":"2024-04-25T10:10:00.084609+0000","last_fullsized":"2024-04-25T10:10:00.084609+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:52.181192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.050643+0000","last_change":"2024-04-25T10:10:00.046754+0000","last_active":"2024-04-25T10:10:00.050643+0000","last_peered":"2024-04-25T10:10:00.050643+0000","last_clean":"2024-04-25T10:10:00.050643+0000","last_became_active":"2024-04-25T10:10:00.046433+0000","last_became_peered":"2024-04-25T10:10:00.046433+0000","last_unstale":"2024-04-25T10:10:00.050643+0000","last_undegraded":"2024-04-25T10:10:00.050643+0000","last_fullsized":"2024-04-25T10:10:00.050643+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:11.035238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.039422+0000","last_change":"2024-04-25T10:09:59.027970+0000","last_active":"2024-04-25T10:10:00.039422+0000","last_peered":"2024-04-25T10:10:00.039422+0000","last_clean":"2024-04-25T10:10:00.039422+0000","last_became_active":"2024-04-25T10:09:59.027098+0000","last_became_peered":"2024-04-25T10:09:59.027098+0000","last_unstale":"2024-04-25T10:10:00.039422+0000","last_undegraded":"2024-04-25T10:10:00.039422+0000","last_fullsized":"2024-04-25T10:10:00.039422+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:58.196472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.055494+0000","last_change":"2024-04-25T10:10:00.051269+0000","last_active":"2024-04-25T10:10:00.055494+0000","last_peered":"2024-04-25T10:10:00.055494+0000","last_clean":"2024-04-25T10:10:00.055494+0000","last_became_active":"2024-04-25T10:10:00.050926+0000","last_became_peered":"2024-04-25T10:10:00.050926+0000","last_unstale":"2024-04-25T10:10:00.055494+0000","last_undegraded":"2024-04-25T10:10:00.055494+0000","last_fullsized":"2024-04-25T10:10:00.055494+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:12.479880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.025463+0000","last_change":"2024-04-25T10:09:59.025294+0000","last_active":"2024-04-25T10:10:00.025463+0000","last_peered":"2024-04-25T10:10:00.025463+0000","last_clean":"2024-04-25T10:10:00.025463+0000","last_became_active":"2024-04-25T10:09:59.024309+0000","last_became_peered":"2024-04-25T10:09:59.024309+0000","last_unstale":"2024-04-25T10:10:00.025463+0000","last_undegraded":"2024-04-25T10:10:00.025463+0000","last_fullsized":"2024-04-25T10:10:00.025463+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:10.021862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.067481+0000","last_change":"2024-04-25T10:10:00.066428+0000","last_active":"2024-04-25T10:10:00.067481+0000","last_peered":"2024-04-25T10:10:00.067481+0000","last_clean":"2024-04-25T10:10:00.067481+0000","last_became_active":"2024-04-25T10:10:00.066143+0000","last_became_peered":"2024-04-25T10:10:00.066143+0000","last_unstale":"2024-04-25T10:10:00.067481+0000","last_undegraded":"2024-04-25T10:10:00.067481+0000","last_fullsized":"2024-04-25T10:10:00.067481+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:15.144005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.081549+0000","last_change":"2024-04-25T10:09:59.029040+0000","last_active":"2024-04-25T10:10:00.081549+0000","last_peered":"2024-04-25T10:10:00.081549+0000","last_clean":"2024-04-25T10:10:00.081549+0000","last_became_active":"2024-04-25T10:09:59.027224+0000","last_became_peered":"2024-04-25T10:09:59.027224+0000","last_unstale":"2024-04-25T10:10:00.081549+0000","last_undegraded":"2024-04-25T10:10:00.081549+0000","last_fullsized":"2024-04-25T10:10:00.081549+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.466504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.056859+0000","last_change":"2024-04-25T10:10:00.050146+0000","last_active":"2024-04-25T10:10:00.056859+0000","last_peered":"2024-04-25T10:10:00.056859+0000","last_clean":"2024-04-25T10:10:00.056859+0000","last_became_active":"2024-04-25T10:10:00.049711+0000","last_became_peered":"2024-04-25T10:10:00.049711+0000","last_unstale":"2024-04-25T10:10:00.056859+0000","last_undegraded":"2024-04-25T10:10:00.056859+0000","last_fullsized":"2024-04-25T10:10:00.056859+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:11.916038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.084711+0000","last_change":"2024-04-25T10:09:59.015996+0000","last_active":"2024-04-25T10:10:00.084711+0000","last_peered":"2024-04-25T10:10:00.084711+0000","last_clean":"2024-04-25T10:10:00.084711+0000","last_became_active":"2024-04-25T10:09:59.015765+0000","last_became_peered":"2024-04-25T10:09:59.015765+0000","last_unstale":"2024-04-25T10:10:00.084711+0000","last_undegraded":"2024-04-25T10:10:00.084711+0000","last_fullsized":"2024-04-25T10:10:00.084711+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:07.021975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.071283+0000","last_change":"2024-04-25T10:10:00.070229+0000","last_active":"2024-04-25T10:10:00.071283+0000","last_peered":"2024-04-25T10:10:00.071283+0000","last_clean":"2024-04-25T10:10:00.071283+0000","last_became_active":"2024-04-25T10:10:00.069736+0000","last_became_peered":"2024-04-25T10:10:00.069736+0000","last_unstale":"2024-04-25T10:10:00.071283+0000","last_undegraded":"2024-04-25T10:10:00.071283+0000","last_fullsized":"2024-04-25T10:10:00.071283+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:46.705938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.082242+0000","last_change":"2024-04-25T10:09:59.012428+0000","last_active":"2024-04-25T10:10:00.082242+0000","last_peered":"2024-04-25T10:10:00.082242+0000","last_clean":"2024-04-25T10:10:00.082242+0000","last_became_active":"2024-04-25T10:09:59.012128+0000","last_became_peered":"2024-04-25T10:09:59.012128+0000","last_unstale":"2024-04-25T10:10:00.082242+0000","last_undegraded":"2024-04-25T10:10:00.082242+0000","last_fullsized":"2024-04-25T10:10:00.082242+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:46.392583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.063932+0000","last_change":"2024-04-25T10:10:00.051095+0000","last_active":"2024-04-25T10:10:00.063932+0000","last_peered":"2024-04-25T10:10:00.063932+0000","last_clean":"2024-04-25T10:10:00.063932+0000","last_became_active":"2024-04-25T10:10:00.050529+0000","last_became_peered":"2024-04-25T10:10:00.050529+0000","last_unstale":"2024-04-25T10:10:00.063932+0000","last_undegraded":"2024-04-25T10:10:00.063932+0000","last_fullsized":"2024-04-25T10:10:00.063932+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:31.813859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.080876+0000","last_change":"2024-04-25T10:09:59.026331+0000","last_active":"2024-04-25T10:10:00.080876+0000","last_peered":"2024-04-25T10:10:00.080876+0000","last_clean":"2024-04-25T10:10:00.080876+0000","last_became_active":"2024-04-25T10:09:59.026026+0000","last_became_peered":"2024-04-25T10:09:59.026026+0000","last_unstale":"2024-04-25T10:10:00.080876+0000","last_undegraded":"2024-04-25T10:10:00.080876+0000","last_fullsized":"2024-04-25T10:10:00.080876+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:16.349844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.069440+0000","last_change":"2024-04-25T10:10:00.068259+0000","last_active":"2024-04-25T10:10:00.069440+0000","last_peered":"2024-04-25T10:10:00.069440+0000","last_clean":"2024-04-25T10:10:00.069440+0000","last_became_active":"2024-04-25T10:10:00.067922+0000","last_became_peered":"2024-04-25T10:10:00.067922+0000","last_unstale":"2024-04-25T10:10:00.069440+0000","last_undegraded":"2024-04-25T10:10:00.069440+0000","last_fullsized":"2024-04-25T10:10:00.069440+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:46.705938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.030932+0000","last_change":"2024-04-25T10:09:59.018256+0000","last_active":"2024-04-25T10:10:00.030932+0000","last_peered":"2024-04-25T10:10:00.030932+0000","last_clean":"2024-04-25T10:10:00.030932+0000","last_became_active":"2024-04-25T10:09:59.016274+0000","last_became_peered":"2024-04-25T10:09:59.016274+0000","last_unstale":"2024-04-25T10:10:00.030932+0000","last_undegraded":"2024-04-25T10:10:00.030932+0000","last_fullsized":"2024-04-25T10:10:00.030932+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:42.154289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.060265+0000","last_change":"2024-04-25T10:10:00.059369+0000","last_active":"2024-04-25T10:10:00.060265+0000","last_peered":"2024-04-25T10:10:00.060265+0000","last_clean":"2024-04-25T10:10:00.060265+0000","last_became_active":"2024-04-25T10:10:00.059113+0000","last_became_peered":"2024-04-25T10:10:00.059113+0000","last_unstale":"2024-04-25T10:10:00.060265+0000","last_undegraded":"2024-04-25T10:10:00.060265+0000","last_fullsized":"2024-04-25T10:10:00.060265+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:15.144005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.084447+0000","last_change":"2024-04-25T10:09:59.020140+0000","last_active":"2024-04-25T10:10:00.084447+0000","last_peered":"2024-04-25T10:10:00.084447+0000","last_clean":"2024-04-25T10:10:00.084447+0000","last_became_active":"2024-04-25T10:09:59.019786+0000","last_became_peered":"2024-04-25T10:09:59.019786+0000","last_unstale":"2024-04-25T10:10:00.084447+0000","last_undegraded":"2024-04-25T10:10:00.084447+0000","last_fullsized":"2024-04-25T10:10:00.084447+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:41.869640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.052864+0000","last_change":"2024-04-25T10:10:00.049402+0000","last_active":"2024-04-25T10:10:00.052864+0000","last_peered":"2024-04-25T10:10:00.052864+0000","last_clean":"2024-04-25T10:10:00.052864+0000","last_became_active":"2024-04-25T10:10:00.045947+0000","last_became_peered":"2024-04-25T10:10:00.045947+0000","last_unstale":"2024-04-25T10:10:00.052864+0000","last_undegraded":"2024-04-25T10:10:00.052864+0000","last_fullsized":"2024-04-25T10:10:00.052864+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:46.655110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.065890+0000","last_change":"2024-04-25T10:09:59.015843+0000","last_active":"2024-04-25T10:10:00.065890+0000","last_peered":"2024-04-25T10:10:00.065890+0000","last_clean":"2024-04-25T10:10:00.065890+0000","last_became_active":"2024-04-25T10:09:59.015538+0000","last_became_peered":"2024-04-25T10:09:59.015538+0000","last_unstale":"2024-04-25T10:10:00.065890+0000","last_undegraded":"2024-04-25T10:10:00.065890+0000","last_fullsized":"2024-04-25T10:10:00.065890+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:51.220123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.052833+0000","last_change":"2024-04-25T10:10:00.049680+0000","last_active":"2024-04-25T10:10:00.052833+0000","last_peered":"2024-04-25T10:10:00.052833+0000","last_clean":"2024-04-25T10:10:00.052833+0000","last_became_active":"2024-04-25T10:10:00.049376+0000","last_became_peered":"2024-04-25T10:10:00.049376+0000","last_unstale":"2024-04-25T10:10:00.052833+0000","last_undegraded":"2024-04-25T10:10:00.052833+0000","last_fullsized":"2024-04-25T10:10:00.052833+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:45.607269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.024851+0000","last_change":"2024-04-25T10:09:59.021922+0000","last_active":"2024-04-25T10:10:00.024851+0000","last_peered":"2024-04-25T10:10:00.024851+0000","last_clean":"2024-04-25T10:10:00.024851+0000","last_became_active":"2024-04-25T10:09:59.021630+0000","last_became_peered":"2024-04-25T10:09:59.021630+0000","last_unstale":"2024-04-25T10:10:00.024851+0000","last_undegraded":"2024-04-25T10:10:00.024851+0000","last_fullsized":"2024-04-25T10:10:00.024851+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:54.077000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.049746+0000","last_change":"2024-04-25T10:10:00.045668+0000","last_active":"2024-04-25T10:10:00.049746+0000","last_peered":"2024-04-25T10:10:00.049746+0000","last_clean":"2024-04-25T10:10:00.049746+0000","last_became_active":"2024-04-25T10:10:00.045262+0000","last_became_peered":"2024-04-25T10:10:00.045262+0000","last_unstale":"2024-04-25T10:10:00.049746+0000","last_undegraded":"2024-04-25T10:10:00.049746+0000","last_fullsized":"2024-04-25T10:10:00.049746+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:47.405959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.085281+0000","last_change":"2024-04-25T10:09:59.022783+0000","last_active":"2024-04-25T10:10:00.085281+0000","last_peered":"2024-04-25T10:10:00.085281+0000","last_clean":"2024-04-25T10:10:00.085281+0000","last_became_active":"2024-04-25T10:09:59.022440+0000","last_became_peered":"2024-04-25T10:09:59.022440+0000","last_unstale":"2024-04-25T10:10:00.085281+0000","last_undegraded":"2024-04-25T10:10:00.085281+0000","last_fullsized":"2024-04-25T10:10:00.085281+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:58.812393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.052871+0000","last_change":"2024-04-25T10:10:00.048383+0000","last_active":"2024-04-25T10:10:00.052871+0000","last_peered":"2024-04-25T10:10:00.052871+0000","last_clean":"2024-04-25T10:10:00.052871+0000","last_became_active":"2024-04-25T10:10:00.048051+0000","last_became_peered":"2024-04-25T10:10:00.048051+0000","last_unstale":"2024-04-25T10:10:00.052871+0000","last_undegraded":"2024-04-25T10:10:00.052871+0000","last_fullsized":"2024-04-25T10:10:00.052871+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:00.349263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.031324+0000","last_change":"2024-04-25T10:09:59.022236+0000","last_active":"2024-04-25T10:10:00.031324+0000","last_peered":"2024-04-25T10:10:00.031324+0000","last_clean":"2024-04-25T10:10:00.031324+0000","last_became_active":"2024-04-25T10:09:59.020911+0000","last_became_peered":"2024-04-25T10:09:59.020911+0000","last_unstale":"2024-04-25T10:10:00.031324+0000","last_undegraded":"2024-04-25T10:10:00.031324+0000","last_fullsized":"2024-04-25T10:10:00.031324+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:59.335887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.058659+0000","last_change":"2024-04-25T10:10:00.055571+0000","last_active":"2024-04-25T10:10:00.058659+0000","last_peered":"2024-04-25T10:10:00.058659+0000","last_clean":"2024-04-25T10:10:00.058659+0000","last_became_active":"2024-04-25T10:10:00.054875+0000","last_became_peered":"2024-04-25T10:10:00.054875+0000","last_unstale":"2024-04-25T10:10:00.058659+0000","last_undegraded":"2024-04-25T10:10:00.058659+0000","last_fullsized":"2024-04-25T10:10:00.058659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:24.458296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.042728+0000","last_change":"2024-04-25T10:09:59.023024+0000","last_active":"2024-04-25T10:10:00.042728+0000","last_peered":"2024-04-25T10:10:00.042728+0000","last_clean":"2024-04-25T10:10:00.042728+0000","last_became_active":"2024-04-25T10:09:59.021790+0000","last_became_peered":"2024-04-25T10:09:59.021790+0000","last_unstale":"2024-04-25T10:10:00.042728+0000","last_undegraded":"2024-04-25T10:10:00.042728+0000","last_fullsized":"2024-04-25T10:10:00.042728+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:20.433429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.058162+0000","last_change":"2024-04-25T10:10:00.054747+0000","last_active":"2024-04-25T10:10:00.058162+0000","last_peered":"2024-04-25T10:10:00.058162+0000","last_clean":"2024-04-25T10:10:00.058162+0000","last_became_active":"2024-04-25T10:10:00.054446+0000","last_became_peered":"2024-04-25T10:10:00.054446+0000","last_unstale":"2024-04-25T10:10:00.058162+0000","last_undegraded":"2024-04-25T10:10:00.058162+0000","last_fullsized":"2024-04-25T10:10:00.058162+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:08.070697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.024185+0000","last_change":"2024-04-25T10:09:59.022659+0000","last_active":"2024-04-25T10:10:00.024185+0000","last_peered":"2024-04-25T10:10:00.024185+0000","last_clean":"2024-04-25T10:10:00.024185+0000","last_became_active":"2024-04-25T10:09:59.021237+0000","last_became_peered":"2024-04-25T10:09:59.021237+0000","last_unstale":"2024-04-25T10:10:00.024185+0000","last_undegraded":"2024-04-25T10:10:00.024185+0000","last_fullsized":"2024-04-25T10:10:00.024185+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.466504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.067827+0000","last_change":"2024-04-25T10:10:00.062854+0000","last_active":"2024-04-25T10:10:00.067827+0000","last_peered":"2024-04-25T10:10:00.067827+0000","last_clean":"2024-04-25T10:10:00.067827+0000","last_became_active":"2024-04-25T10:10:00.062538+0000","last_became_peered":"2024-04-25T10:10:00.062538+0000","last_unstale":"2024-04-25T10:10:00.067827+0000","last_undegraded":"2024-04-25T10:10:00.067827+0000","last_fullsized":"2024-04-25T10:10:00.067827+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:41.043177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.065427+0000","last_change":"2024-04-25T10:09:59.025029+0000","last_active":"2024-04-25T10:10:00.065427+0000","last_peered":"2024-04-25T10:10:00.065427+0000","last_clean":"2024-04-25T10:10:00.065427+0000","last_became_active":"2024-04-25T10:09:59.024404+0000","last_became_peered":"2024-04-25T10:09:59.024404+0000","last_unstale":"2024-04-25T10:10:00.065427+0000","last_undegraded":"2024-04-25T10:10:00.065427+0000","last_fullsized":"2024-04-25T10:10:00.065427+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:20.433429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.058534+0000","last_change":"2024-04-25T10:10:00.057319+0000","last_active":"2024-04-25T10:10:00.058534+0000","last_peered":"2024-04-25T10:10:00.058534+0000","last_clean":"2024-04-25T10:10:00.058534+0000","last_became_active":"2024-04-25T10:10:00.056981+0000","last_became_peered":"2024-04-25T10:10:00.056981+0000","last_unstale":"2024-04-25T10:10:00.058534+0000","last_undegraded":"2024-04-25T10:10:00.058534+0000","last_fullsized":"2024-04-25T10:10:00.058534+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:08.780623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.036623+0000","last_change":"2024-04-25T10:09:59.028406+0000","last_active":"2024-04-25T10:10:00.036623+0000","last_peered":"2024-04-25T10:10:00.036623+0000","last_clean":"2024-04-25T10:10:00.036623+0000","last_became_active":"2024-04-25T10:09:59.028103+0000","last_became_peered":"2024-04-25T10:09:59.028103+0000","last_unstale":"2024-04-25T10:10:00.036623+0000","last_undegraded":"2024-04-25T10:10:00.036623+0000","last_fullsized":"2024-04-25T10:10:00.036623+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:44.296043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.068836+0000","last_change":"2024-04-25T10:10:00.059857+0000","last_active":"2024-04-25T10:10:00.068836+0000","last_peered":"2024-04-25T10:10:00.068836+0000","last_clean":"2024-04-25T10:10:00.068836+0000","last_became_active":"2024-04-25T10:10:00.052617+0000","last_became_peered":"2024-04-25T10:10:00.052617+0000","last_unstale":"2024-04-25T10:10:00.068836+0000","last_undegraded":"2024-04-25T10:10:00.068836+0000","last_fullsized":"2024-04-25T10:10:00.068836+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:40.980941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.065479+0000","last_change":"2024-04-25T10:09:59.027922+0000","last_active":"2024-04-25T10:10:00.065479+0000","last_peered":"2024-04-25T10:10:00.065479+0000","last_clean":"2024-04-25T10:10:00.065479+0000","last_became_active":"2024-04-25T10:09:59.027613+0000","last_became_peered":"2024-04-25T10:09:59.027613+0000","last_unstale":"2024-04-25T10:10:00.065479+0000","last_undegraded":"2024-04-25T10:10:00.065479+0000","last_fullsized":"2024-04-25T10:10:00.065479+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:34.819850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.064414+0000","last_change":"2024-04-25T10:10:00.058385+0000","last_active":"2024-04-25T10:10:00.064414+0000","last_peered":"2024-04-25T10:10:00.064414+0000","last_clean":"2024-04-25T10:10:00.064414+0000","last_became_active":"2024-04-25T10:10:00.057823+0000","last_became_peered":"2024-04-25T10:10:00.057823+0000","last_unstale":"2024-04-25T10:10:00.064414+0000","last_undegraded":"2024-04-25T10:10:00.064414+0000","last_fullsized":"2024-04-25T10:10:00.064414+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:19.895762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.040164+0000","last_change":"2024-04-25T10:09:59.025945+0000","last_active":"2024-04-25T10:10:00.040164+0000","last_peered":"2024-04-25T10:10:00.040164+0000","last_clean":"2024-04-25T10:10:00.040164+0000","last_became_active":"2024-04-25T10:09:59.025319+0000","last_became_peered":"2024-04-25T10:09:59.025319+0000","last_unstale":"2024-04-25T10:10:00.040164+0000","last_undegraded":"2024-04-25T10:10:00.040164+0000","last_fullsized":"2024-04-25T10:10:00.040164+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:35.119743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.064512+0000","last_change":"2024-04-25T10:10:00.056971+0000","last_active":"2024-04-25T10:10:00.064512+0000","last_peered":"2024-04-25T10:10:00.064512+0000","last_clean":"2024-04-25T10:10:00.064512+0000","last_became_active":"2024-04-25T10:10:00.056679+0000","last_became_peered":"2024-04-25T10:10:00.056679+0000","last_unstale":"2024-04-25T10:10:00.064512+0000","last_undegraded":"2024-04-25T10:10:00.064512+0000","last_fullsized":"2024-04-25T10:10:00.064512+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:58.983397+0000","last_clean_scrub_stamp":"2024-04-25T10:09:58.983397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:01.874569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.039553+0000","last_change":"2024-04-25T10:09:59.027207+0000","last_active":"2024-04-25T10:10:00.039553+0000","last_peered":"2024-04-25T10:10:00.039553+0000","last_clean":"2024-04-25T10:10:00.039553+0000","last_became_active":"2024-04-25T10:09:59.026386+0000","last_became_peered":"2024-04-25T10:09:59.026386+0000","last_unstale":"2024-04-25T10:10:00.039553+0000","last_undegraded":"2024-04-25T10:10:00.039553+0000","last_fullsized":"2024-04-25T10:10:00.039553+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:09:57.970021+0000","last_clean_scrub_stamp":"2024-04-25T10:09:57.970021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:54.197947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":234,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:10:00.025484+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:10:00.025484+0000","last_peered":"2024-04-25T10:10:00.025484+0000","last_clean":"2024-04-25T10:10:00.025484+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:10:00.025484+0000","last_undegraded":"2024-04-25T10:10:00.025484+0000","last_fullsized":"2024-04-25T10:10:00.025484+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1241088,"data_stored":856930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":647168,"data_stored":266562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":655360,"data_stored":267214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":659456,"data_stored":268174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":655360,"data_stored":266618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":655360,"data_stored":266618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":651264,"data_stored":266652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":859124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:10:04.889 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:10:04.889 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:10:04.977 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:10:04.971+0000 7efdbf600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:10:05.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.133+0000 7fd630600640 1 Processor -- start 2024-04-25T10:10:05.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.133+0000 7fd630600640 1 -- start start 2024-04-25T10:10:05.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.135+0000 7fd630600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd628144d00 0x7fd62806d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:05.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.135+0000 7fd630600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62815fdd0 con 0x7fd6280cf620 2024-04-25T10:10:05.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.135+0000 7fd630600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6281600b0 con 0x7fd628144550 2024-04-25T10:10:05.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.135+0000 7fd630600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd628160390 con 0x7fd628144d00 2024-04-25T10:10:05.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.136+0000 7fd62e600640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd6280cf620 0x7fd6280cfa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51470/0 (socket says 172.21.15.119:51470) 2024-04-25T10:10:05.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.136+0000 7fd62e600640 1 -- 172.21.15.119:0/1823535414 learned_addr learned my addr 172.21.15.119:0/1823535414 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:10:05.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.136+0000 7fd62d200640 1 --2- 172.21.15.119:0/1823535414 >> v2:172.21.15.132:3300/0 conn(0x7fd628144d00 0x7fd62806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:05.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.136+0000 7fd627e00640 1 -- 172.21.15.119:0/1823535414 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048552557 0 0) 0x7fd62815fdd0 con 0x7fd6280cf620 2024-04-25T10:10:05.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.136+0000 7fd627e00640 1 -- 172.21.15.119:0/1823535414 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd610003520 con 0x7fd6280cf620 2024-04-25T10:10:05.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.136+0000 7fd627e00640 1 -- 172.21.15.119:0/1823535414 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2400561455 0 0) 0x7fd610003520 con 0x7fd6280cf620 2024-04-25T10:10:05.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.136+0000 7fd627e00640 1 -- 172.21.15.119:0/1823535414 >> v2:172.21.15.132:3300/0 conn(0x7fd628144d00 msgr2=0x7fd62806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:05.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.136+0000 7fd627e00640 1 --2- 172.21.15.119:0/1823535414 >> v2:172.21.15.132:3300/0 conn(0x7fd628144d00 0x7fd62806d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.136+0000 7fd627e00640 1 -- 172.21.15.119:0/1823535414 >> v1:172.21.15.132:6789/0 conn(0x7fd628144550 legacy=0x7fd62815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:05.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd627e00640 1 -- 172.21.15.119:0/1823535414 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6280cbeb0 con 0x7fd6280cf620 2024-04-25T10:10:05.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd627e00640 1 -- 172.21.15.119:0/1823535414 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fd6200019f0 con 0x7fd6280cf620 2024-04-25T10:10:05.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd627e00640 1 -- 172.21.15.119:0/1823535414 >> v1:172.21.15.119:6789/0 conn(0x7fd6280cf620 legacy=0x7fd6280cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:05.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd627e00640 1 --2- 172.21.15.119:0/1823535414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd610003a00 0x7fd610003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:05.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd627e00640 1 --2- 172.21.15.119:0/1823535414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd610004800 0x7fd610004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:05.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd627e00640 1 --2- 172.21.15.119:0/1823535414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd610005210 0x7fd61000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:05.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd627e00640 1 -- 172.21.15.119:0/1823535414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd628160390 con 0x7fd610003a00 2024-04-25T10:10:05.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd627e00640 1 -- 172.21.15.119:0/1823535414 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61000c160 con 0x7fd610005210 2024-04-25T10:10:05.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd627e00640 1 -- 172.21.15.119:0/1823535414 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61000c360 con 0x7fd610004800 2024-04-25T10:10:05.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd62dc00640 1 --2- 172.21.15.119:0/1823535414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd610003a00 0x7fd610003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:05.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd62d200640 1 --2- 172.21.15.119:0/1823535414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd610004800 0x7fd610004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:05.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd62dc00640 1 -- 172.21.15.119:0/1823535414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd610004800 msgr2=0x7fd610004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:05.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd62dc00640 1 --2- 172.21.15.119:0/1823535414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd610004800 0x7fd610004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd62e600640 1 --2- 172.21.15.119:0/1823535414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd610005210 0x7fd61000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:05.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd62dc00640 1 -- 172.21.15.119:0/1823535414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd610005210 msgr2=0x7fd61000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:05.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd62dc00640 1 --2- 172.21.15.119:0/1823535414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd610005210 0x7fd61000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd62dc00640 1 -- 172.21.15.119:0/1823535414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd610003520 con 0x7fd610003a00 2024-04-25T10:10:05.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.137+0000 7fd62e600640 1 -- 172.21.15.119:0/1823535414 reap_dead start 2024-04-25T10:10:05.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.138+0000 7fd62dc00640 1 --2- 172.21.15.119:0/1823535414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd610003a00 0x7fd610003e20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd61c001750 tx=0x7fd610005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9850f787f6148d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:05.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.139+0000 7fd627e00640 1 -- 172.21.15.119:0/1823535414 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c001370 con 0x7fd610003a00 2024-04-25T10:10:05.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.139+0000 7fd627e00640 1 -- 172.21.15.119:0/1823535414 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c0052d0 con 0x7fd610003a00 2024-04-25T10:10:05.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.139+0000 7fd627e00640 1 -- 172.21.15.119:0/1823535414 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c02abc0 con 0x7fd610003a00 2024-04-25T10:10:05.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.139+0000 7fd630600640 1 -- 172.21.15.119:0/1823535414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd610003a00 msgr2=0x7fd610003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:05.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.139+0000 7fd630600640 1 --2- 172.21.15.119:0/1823535414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd610003a00 0x7fd610003e20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd61c001750 tx=0x7fd610005830 comp rx=0 tx=0).stop 2024-04-25T10:10:05.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.139+0000 7fd630600640 1 -- 172.21.15.119:0/1823535414 shutdown_connections 2024-04-25T10:10:05.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.139+0000 7fd630600640 1 --2- 172.21.15.119:0/1823535414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd610003a00 0x7fd610003e20 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.139+0000 7fd630600640 1 -- 172.21.15.119:0/1823535414 >> 172.21.15.119:0/1823535414 conn(0x7fd62806aba0 msgr2=0x7fd62806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:05.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.139+0000 7fd630600640 1 -- 172.21.15.119:0/1823535414 shutdown_connections 2024-04-25T10:10:05.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.139+0000 7fd630600640 1 -- 172.21.15.119:0/1823535414 wait complete. 2024-04-25T10:10:05.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.141+0000 7fd630600640 1 Processor -- start 2024-04-25T10:10:05.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.141+0000 7fd630600640 1 -- start start 2024-04-25T10:10:05.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.142+0000 7fd630600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd610003a00 0x7fd62813c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:05.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.142+0000 7fd630600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6280b1390 0x7fd62815c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.142+0000 7fd630600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6280b1810 0x7fd62815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.142+0000 7fd630600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd628144ab0 con 0x7fd6280b1810 2024-04-25T10:10:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.142+0000 7fd630600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd628144930 con 0x7fd6280b1390 2024-04-25T10:10:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.142+0000 7fd630600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62806d590 con 0x7fd610003a00 2024-04-25T10:10:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.142+0000 7fd62e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6280b1810 0x7fd62815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.142+0000 7fd62e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6280b1810 0x7fd62815cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39258/0 (socket says 172.21.15.119:39258) 2024-04-25T10:10:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.142+0000 7fd62e600640 1 -- 172.21.15.119:0/1016769263 learned_addr learned my addr 172.21.15.119:0/1016769263 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.143+0000 7fd62dc00640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd610003a00 0x7fd62813c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.143+0000 7fd62d200640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6280b1390 0x7fd62815c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.143+0000 7fd62e600640 1 -- 172.21.15.119:0/1016769263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd610003a00 msgr2=0x7fd62813c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.143+0000 7fd62e600640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd610003a00 0x7fd62813c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.143+0000 7fd62e600640 1 -- 172.21.15.119:0/1016769263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6280b1390 msgr2=0x7fd62815c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.143+0000 7fd62e600640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6280b1390 0x7fd62815c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.143+0000 7fd62e600640 1 -- 172.21.15.119:0/1016769263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62815b570 con 0x7fd6280b1810 2024-04-25T10:10:05.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.143+0000 7fd62d200640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6280b1390 0x7fd62815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:05.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.143+0000 7fd62e600640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6280b1810 0x7fd62815cd30 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd620007c40 tx=0x7fd62000b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:05.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.143+0000 7fd625600640 1 -- 172.21.15.119:0/1016769263 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62000b8d0 con 0x7fd6280b1810 2024-04-25T10:10:05.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.143+0000 7fd625600640 1 -- 172.21.15.119:0/1016769263 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62000ba70 con 0x7fd6280b1810 2024-04-25T10:10:05.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.143+0000 7fd625600640 1 -- 172.21.15.119:0/1016769263 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62000be60 con 0x7fd6280b1810 2024-04-25T10:10:05.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.143+0000 7fd630600640 1 -- 172.21.15.119:0/1016769263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6280b05d0 con 0x7fd6280b1810 2024-04-25T10:10:05.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.144+0000 7fd630600640 1 -- 172.21.15.119:0/1016769263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6280b0b90 con 0x7fd6280b1810 2024-04-25T10:10:05.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.145+0000 7fd630600640 1 -- 172.21.15.119:0/1016769263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6280749b0 con 0x7fd6280b1810 2024-04-25T10:10:05.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.146+0000 7fd625600640 1 -- 172.21.15.119:0/1016769263 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd62001b060 con 0x7fd6280b1810 2024-04-25T10:10:05.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.147+0000 7fd625600640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd60806b4d0 0x7fd60806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:05.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.147+0000 7fd625600640 1 -- 172.21.15.119:0/1016769263 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd62008a140 con 0x7fd6280b1810 2024-04-25T10:10:05.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.147+0000 7fd62dc00640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd60806b4d0 0x7fd60806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:05.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.148+0000 7fd62dc00640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd60806b4d0 0x7fd60806d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd61c032000 tx=0x7fd61c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:05.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.151+0000 7fd625600640 1 -- 172.21.15.119:0/1016769263 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd62815b570 con 0x7fd6280b1810 2024-04-25T10:10:05.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.418+0000 7fd630600640 1 -- 172.21.15.119:0/1016769263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd62813d640 con 0x7fd6280b1810 2024-04-25T10:10:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.423+0000 7fd625600640 1 -- 172.21.15.119:0/1016769263 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd620058f20 con 0x7fd6280b1810 2024-04-25T10:10:05.430 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-04-25T10:10:05.430 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA== 2024-04-25T10:10:05.430 INFO:teuthology.orchestra.run.smithi119.stdout: caps mds = "allow" 2024-04-25T10:10:05.430 INFO:teuthology.orchestra.run.smithi119.stdout: caps mon = "allow r" 2024-04-25T10:10:05.430 INFO:teuthology.orchestra.run.smithi119.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:10:05.431 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2024-04-25T10:10:05.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.426+0000 7fd630600640 1 -- 172.21.15.119:0/1016769263 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd60806b4d0 msgr2=0x7fd60806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:05.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.426+0000 7fd630600640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd60806b4d0 0x7fd60806d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd61c032000 tx=0x7fd61c02e000 comp rx=0 tx=0).stop 2024-04-25T10:10:05.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.426+0000 7fd630600640 1 -- 172.21.15.119:0/1016769263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6280b1810 msgr2=0x7fd62815cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:05.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.426+0000 7fd630600640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6280b1810 0x7fd62815cd30 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd620007c40 tx=0x7fd62000b0d0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.426+0000 7fd630600640 1 -- 172.21.15.119:0/1016769263 shutdown_connections 2024-04-25T10:10:05.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.426+0000 7fd630600640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd60806b4d0 0x7fd60806d990 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.426+0000 7fd630600640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6280b1810 0x7fd62815cd30 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.426+0000 7fd630600640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6280b1390 0x7fd62815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.426+0000 7fd630600640 1 --2- 172.21.15.119:0/1016769263 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd610003a00 0x7fd62813c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.426+0000 7fd630600640 1 -- 172.21.15.119:0/1016769263 >> 172.21.15.119:0/1016769263 conn(0x7fd62806aba0 msgr2=0x7fd6281421e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:05.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.426+0000 7fd630600640 1 -- 172.21.15.119:0/1016769263 shutdown_connections 2024-04-25T10:10:05.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.426+0000 7fd630600640 1 -- 172.21.15.119:0/1016769263 wait complete. 2024-04-25T10:10:05.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.643+0000 7f64e2000640 1 Processor -- start 2024-04-25T10:10:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.643+0000 7f64e2000640 1 -- start start 2024-04-25T10:10:05.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.644+0000 7f64e2000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f64dc147270 0x7f64dc0b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:05.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.644+0000 7f64e2000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f64dc15dee0 0x7f64dc15e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:05.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.644+0000 7f64e2000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64dc0af000 con 0x7f64dc15e890 2024-04-25T10:10:05.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.644+0000 7f64e2000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64dc0b4a80 con 0x7f64dc147270 2024-04-25T10:10:05.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.644+0000 7f64e2000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64dc0b4d60 con 0x7f64dc15dee0 2024-04-25T10:10:05.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.645+0000 7f64e0c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f64dc147270 0x7f64dc0b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:05.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.645+0000 7f64e1600640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f64dc15e890 0x7f64dc1614a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51482/0 (socket says 172.21.15.119:51482) 2024-04-25T10:10:05.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.645+0000 7f64e0c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f64dc147270 0x7f64dc0b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39274/0 (socket says 172.21.15.119:39274) 2024-04-25T10:10:05.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.645+0000 7f64e1600640 1 -- 172.21.15.119:0/1546596723 learned_addr learned my addr 172.21.15.119:0/1546596723 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:10:05.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.645+0000 7f64dbe00640 1 --2- 172.21.15.119:0/1546596723 >> v2:172.21.15.132:3300/0 conn(0x7f64dc15dee0 0x7f64dc15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:05.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.645+0000 7f64e0c00640 1 -- 172.21.15.119:0/1546596723 >> v2:172.21.15.132:3300/0 conn(0x7f64dc15dee0 msgr2=0x7f64dc15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:05.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.645+0000 7f64e0c00640 1 --2- 172.21.15.119:0/1546596723 >> v2:172.21.15.132:3300/0 conn(0x7f64dc15dee0 0x7f64dc15e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.645+0000 7f64e0c00640 1 -- 172.21.15.119:0/1546596723 >> v1:172.21.15.119:6789/0 conn(0x7f64dc15e890 legacy=0x7f64dc1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:05.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.645+0000 7f64e0c00640 1 -- 172.21.15.119:0/1546596723 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc0b5040 con 0x7f64dc147270 2024-04-25T10:10:05.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.646+0000 7f64e0c00640 1 --2- 172.21.15.119:0/1546596723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64dc147270 0x7f64dc0b4410 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f64dc06f500 tx=0x7f64d002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b1d18d51a45b5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:05.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.647+0000 7f64db400640 1 -- 172.21.15.119:0/1546596723 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0037070 con 0x7f64dc147270 2024-04-25T10:10:05.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.647+0000 7f64db400640 1 -- 172.21.15.119:0/1546596723 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d0032440 con 0x7f64dc147270 2024-04-25T10:10:05.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.647+0000 7f64db400640 1 -- 172.21.15.119:0/1546596723 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d002ae60 con 0x7f64dc147270 2024-04-25T10:10:05.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.647+0000 7f64e2000640 1 -- 172.21.15.119:0/1546596723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64dc147270 msgr2=0x7f64dc0b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:05.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.647+0000 7f64e2000640 1 --2- 172.21.15.119:0/1546596723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64dc147270 0x7f64dc0b4410 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f64dc06f500 tx=0x7f64d002a110 comp rx=0 tx=0).stop 2024-04-25T10:10:05.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.647+0000 7f64e2000640 1 -- 172.21.15.119:0/1546596723 shutdown_connections 2024-04-25T10:10:05.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.647+0000 7f64e2000640 1 --2- 172.21.15.119:0/1546596723 >> v2:172.21.15.132:3300/0 conn(0x7f64dc15dee0 0x7f64dc15e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.647+0000 7f64e2000640 1 --2- 172.21.15.119:0/1546596723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64dc147270 0x7f64dc0b4410 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.647+0000 7f64e2000640 1 -- 172.21.15.119:0/1546596723 >> 172.21.15.119:0/1546596723 conn(0x7f64dc06b1d0 msgr2=0x7f64dc0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:05.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.648+0000 7f64e2000640 1 -- 172.21.15.119:0/1546596723 shutdown_connections 2024-04-25T10:10:05.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.648+0000 7f64e2000640 1 -- 172.21.15.119:0/1546596723 wait complete. 2024-04-25T10:10:05.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.649+0000 7f64e2000640 1 Processor -- start 2024-04-25T10:10:05.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.649+0000 7f64e2000640 1 -- start start 2024-04-25T10:10:05.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.650+0000 7f64e2000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64dc147270 0x7f64dc13bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:05.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.650+0000 7f64e2000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64dc15dee0 0x7f64dc1e4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:05.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.650+0000 7f64e2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64dc15e890 0x7f64dc0ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:05.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.650+0000 7f64e2000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc15ece0 con 0x7f64dc15e890 2024-04-25T10:10:05.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.650+0000 7f64e2000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc0b5600 con 0x7f64dc15dee0 2024-04-25T10:10:05.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.650+0000 7f64e2000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc0adf90 con 0x7f64dc147270 2024-04-25T10:10:05.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.651+0000 7f64e1600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64dc15e890 0x7f64dc0ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:05.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.651+0000 7f64e1600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64dc15e890 0x7f64dc0ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39284/0 (socket says 172.21.15.119:39284) 2024-04-25T10:10:05.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.651+0000 7f64e1600640 1 -- 172.21.15.119:0/3456853922 learned_addr learned my addr 172.21.15.119:0/3456853922 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:05.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.651+0000 7f64e0c00640 1 --2- 172.21.15.119:0/3456853922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64dc147270 0x7f64dc13bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:05.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.651+0000 7f64dbe00640 1 --2- 172.21.15.119:0/3456853922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64dc15dee0 0x7f64dc1e4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:05.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.651+0000 7f64e1600640 1 -- 172.21.15.119:0/3456853922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64dc147270 msgr2=0x7f64dc13bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:05.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.651+0000 7f64e1600640 1 --2- 172.21.15.119:0/3456853922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64dc147270 0x7f64dc13bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.651+0000 7f64e1600640 1 -- 172.21.15.119:0/3456853922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64dc15dee0 msgr2=0x7f64dc1e4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:05.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.651+0000 7f64e1600640 1 --2- 172.21.15.119:0/3456853922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64dc15dee0 0x7f64dc1e4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.651+0000 7f64e1600640 1 -- 172.21.15.119:0/3456853922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc13c470 con 0x7f64dc15e890 2024-04-25T10:10:05.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.651+0000 7f64e1600640 1 --2- 172.21.15.119:0/3456853922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64dc15e890 0x7f64dc0ce850 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f64d400ca80 tx=0x7f64d400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:05.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.652+0000 7f64d8c00640 1 -- 172.21.15.119:0/3456853922 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4017070 con 0x7f64dc15e890 2024-04-25T10:10:05.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.652+0000 7f64d8c00640 1 -- 172.21.15.119:0/3456853922 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d400ac30 con 0x7f64dc15e890 2024-04-25T10:10:05.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.652+0000 7f64d8c00640 1 -- 172.21.15.119:0/3456853922 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4012600 con 0x7f64dc15e890 2024-04-25T10:10:05.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.652+0000 7f64e2000640 1 -- 172.21.15.119:0/3456853922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64dc1f4d00 con 0x7f64dc15e890 2024-04-25T10:10:05.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.652+0000 7f64e2000640 1 -- 172.21.15.119:0/3456853922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64dc1e44c0 con 0x7f64dc15e890 2024-04-25T10:10:05.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.653+0000 7f64e2000640 1 -- 172.21.15.119:0/3456853922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64dc0749b0 con 0x7f64dc15e890 2024-04-25T10:10:05.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.654+0000 7f64d8c00640 1 -- 172.21.15.119:0/3456853922 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f64d401a070 con 0x7f64dc15e890 2024-04-25T10:10:05.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.655+0000 7f64d8c00640 1 --2- 172.21.15.119:0/3456853922 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f64bc06b4d0 0x7f64bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:05.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.655+0000 7f64d8c00640 1 -- 172.21.15.119:0/3456853922 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64d4089d60 con 0x7f64dc15e890 2024-04-25T10:10:05.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.655+0000 7f64e0c00640 1 --2- 172.21.15.119:0/3456853922 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f64bc06b4d0 0x7f64bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:05.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.656+0000 7f64e0c00640 1 --2- 172.21.15.119:0/3456853922 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f64bc06b4d0 0x7f64bc06d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64d0034000 tx=0x7f64d002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:05.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.659+0000 7f64d8c00640 1 -- 172.21.15.119:0/3456853922 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f64dc13c470 con 0x7f64dc15e890 2024-04-25T10:10:05.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.874+0000 7f64e2000640 1 -- 172.21.15.119:0/3456853922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64dc1e4700 con 0x7f64dc15e890 2024-04-25T10:10:05.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.874+0000 7f64d8c00640 1 -- 172.21.15.119:0/3456853922 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f64d400ae20 con 0x7f64dc15e890 2024-04-25T10:10:05.880 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 34 2024-04-25T10:10:05.881 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:10:05.881 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4159222235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4159222235},{"type":"v1","addr":"172.21.15.132:6834","nonce":4159222235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4223361610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":4223361610},{"type":"v1","addr":"172.21.15.119:6837","nonce":4223361610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2367324442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2367324442},{"type":"v1","addr":"172.21.15.132:6835","nonce":2367324442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:59.454941+0000","modified":"2024-04-25T10:10:02.506906+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6836/4129112885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4129112885},{"type":"v1","addr":"172.21.15.119:6836","nonce":4129112885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:10:05.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.877+0000 7f64e2000640 1 -- 172.21.15.119:0/3456853922 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f64bc06b4d0 msgr2=0x7f64bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:05.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.877+0000 7f64e2000640 1 --2- 172.21.15.119:0/3456853922 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f64bc06b4d0 0x7f64bc06d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64d0034000 tx=0x7f64d002e000 comp rx=0 tx=0).stop 2024-04-25T10:10:05.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.877+0000 7f64e2000640 1 -- 172.21.15.119:0/3456853922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64dc15e890 msgr2=0x7f64dc0ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:05.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.877+0000 7f64e2000640 1 --2- 172.21.15.119:0/3456853922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64dc15e890 0x7f64dc0ce850 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f64d400ca80 tx=0x7f64d400cf40 comp rx=0 tx=0).stop 2024-04-25T10:10:05.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.877+0000 7f64e2000640 1 -- 172.21.15.119:0/3456853922 shutdown_connections 2024-04-25T10:10:05.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.877+0000 7f64e2000640 1 --2- 172.21.15.119:0/3456853922 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f64bc06b4d0 0x7f64bc06d990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.877+0000 7f64e2000640 1 --2- 172.21.15.119:0/3456853922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64dc15e890 0x7f64dc0ce850 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.877+0000 7f64e2000640 1 --2- 172.21.15.119:0/3456853922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64dc15dee0 0x7f64dc1e4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.877+0000 7f64e2000640 1 --2- 172.21.15.119:0/3456853922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64dc147270 0x7f64dc13bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:05.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.877+0000 7f64e2000640 1 -- 172.21.15.119:0/3456853922 >> 172.21.15.119:0/3456853922 conn(0x7f64dc06b1d0 msgr2=0x7f64dc0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:05.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.877+0000 7f64e2000640 1 -- 172.21.15.119:0/3456853922 shutdown_connections 2024-04-25T10:10:05.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:05.877+0000 7f64e2000640 1 -- 172.21.15.119:0/3456853922 wait complete. 2024-04-25T10:10:06.303 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:09:59.454941+0000', 'modified': '2024-04-25T10:10:02.506906+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5931}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5931': {'gid': 5931, 'name': 'a', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6836/4129112885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 4129112885}, {'type': 'v1', 'addr': '172.21.15.119:6836', 'nonce': 4129112885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:10:06.304 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:10:06.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:10:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.509+0000 7fd488000640 1 Processor -- start 2024-04-25T10:10:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.509+0000 7fd488000640 1 -- start start 2024-04-25T10:10:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.511+0000 7fd488000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd48015dbe0 0x7fd480071c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.511+0000 7fd488000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800aedc0 0x7fd4800cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.511+0000 7fd488000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4800ace70 con 0x7fd4800cf900 2024-04-25T10:10:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.511+0000 7fd488000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd480072150 con 0x7fd48015dbe0 2024-04-25T10:10:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.511+0000 7fd488000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800ad110 con 0x7fd4800aedc0 2024-04-25T10:10:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.511+0000 7fd485600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd48015dbe0 0x7fd480071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.511+0000 7fd486000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd4800cf900 0x7fd4800b3070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51484/0 (socket says 172.21.15.119:51484) 2024-04-25T10:10:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.511+0000 7fd486000640 1 -- 172.21.15.119:0/1650011541 learned_addr learned my addr 172.21.15.119:0/1650011541 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:10:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.511+0000 7fd485600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd48015dbe0 0x7fd480071c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39286/0 (socket says 172.21.15.119:39286) 2024-04-25T10:10:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.512+0000 7fd484c00640 1 --2- 172.21.15.119:0/1650011541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800aedc0 0x7fd4800cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.512+0000 7fd485600640 1 -- 172.21.15.119:0/1650011541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800aedc0 msgr2=0x7fd4800cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.512+0000 7fd485600640 1 --2- 172.21.15.119:0/1650011541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800aedc0 0x7fd4800cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.512+0000 7fd485600640 1 -- 172.21.15.119:0/1650011541 >> v1:172.21.15.119:6789/0 conn(0x7fd4800cf900 legacy=0x7fd4800b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.512+0000 7fd485600640 1 -- 172.21.15.119:0/1650011541 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4800ad3f0 con 0x7fd48015dbe0 2024-04-25T10:10:06.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.513+0000 7fd485600640 1 --2- 172.21.15.119:0/1650011541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd48015dbe0 0x7fd480071c10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd4740097a0 tx=0x7fd47402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88518cfd22ba8f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.513+0000 7fd47fe00640 1 -- 172.21.15.119:0/1650011541 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474037070 con 0x7fd48015dbe0 2024-04-25T10:10:06.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.513+0000 7fd47fe00640 1 -- 172.21.15.119:0/1650011541 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd474032440 con 0x7fd48015dbe0 2024-04-25T10:10:06.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.513+0000 7fd47fe00640 1 -- 172.21.15.119:0/1650011541 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474032810 con 0x7fd48015dbe0 2024-04-25T10:10:06.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.514+0000 7fd488000640 1 -- 172.21.15.119:0/1650011541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd48015dbe0 msgr2=0x7fd480071c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.514+0000 7fd488000640 1 --2- 172.21.15.119:0/1650011541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd48015dbe0 0x7fd480071c10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd4740097a0 tx=0x7fd47402a180 comp rx=0 tx=0).stop 2024-04-25T10:10:06.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.514+0000 7fd488000640 1 -- 172.21.15.119:0/1650011541 shutdown_connections 2024-04-25T10:10:06.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.514+0000 7fd488000640 1 --2- 172.21.15.119:0/1650011541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800aedc0 0x7fd4800cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.514+0000 7fd488000640 1 --2- 172.21.15.119:0/1650011541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd48015dbe0 0x7fd480071c10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.514+0000 7fd488000640 1 -- 172.21.15.119:0/1650011541 >> 172.21.15.119:0/1650011541 conn(0x7fd48006aba0 msgr2=0x7fd48006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.514+0000 7fd488000640 1 -- 172.21.15.119:0/1650011541 shutdown_connections 2024-04-25T10:10:06.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.514+0000 7fd488000640 1 -- 172.21.15.119:0/1650011541 wait complete. 2024-04-25T10:10:06.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.516+0000 7fd488000640 1 Processor -- start 2024-04-25T10:10:06.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.516+0000 7fd488000640 1 -- start start 2024-04-25T10:10:06.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.517+0000 7fd488000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4800aedc0 0x7fd48014c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.517+0000 7fd488000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800cf900 0x7fd4800c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.517+0000 7fd488000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd48014f610 0x7fd48015bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.517+0000 7fd488000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800adc90 con 0x7fd4800aedc0 2024-04-25T10:10:06.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.517+0000 7fd488000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800ad9b0 con 0x7fd48014f610 2024-04-25T10:10:06.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.517+0000 7fd488000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48015dfc0 con 0x7fd4800cf900 2024-04-25T10:10:06.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.517+0000 7fd485600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4800aedc0 0x7fd48014c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.517+0000 7fd485600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4800aedc0 0x7fd48014c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39288/0 (socket says 172.21.15.119:39288) 2024-04-25T10:10:06.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.517+0000 7fd485600640 1 -- 172.21.15.119:0/3187625447 learned_addr learned my addr 172.21.15.119:0/3187625447 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:06.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.518+0000 7fd485600640 1 -- 172.21.15.119:0/3187625447 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800cf900 msgr2=0x7fd4800c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.518+0000 7fd485600640 1 --2- 172.21.15.119:0/3187625447 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800cf900 0x7fd4800c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.518+0000 7fd485600640 1 -- 172.21.15.119:0/3187625447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd48014f610 msgr2=0x7fd48015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.518+0000 7fd485600640 1 --2- 172.21.15.119:0/3187625447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd48014f610 0x7fd48015bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.518+0000 7fd485600640 1 -- 172.21.15.119:0/3187625447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48014c710 con 0x7fd4800aedc0 2024-04-25T10:10:06.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.518+0000 7fd485600640 1 --2- 172.21.15.119:0/3187625447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4800aedc0 0x7fd48014c1d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd4740098c0 tx=0x7fd4740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.518+0000 7fd47d600640 1 -- 172.21.15.119:0/3187625447 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474037040 con 0x7fd4800aedc0 2024-04-25T10:10:06.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.518+0000 7fd47d600640 1 -- 172.21.15.119:0/3187625447 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd474042070 con 0x7fd4800aedc0 2024-04-25T10:10:06.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.518+0000 7fd488000640 1 -- 172.21.15.119:0/3187625447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48014e5b0 con 0x7fd4800aedc0 2024-04-25T10:10:06.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.519+0000 7fd47d600640 1 -- 172.21.15.119:0/3187625447 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474006e10 con 0x7fd4800aedc0 2024-04-25T10:10:06.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.519+0000 7fd488000640 1 -- 172.21.15.119:0/3187625447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48014eb30 con 0x7fd4800aedc0 2024-04-25T10:10:06.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.521+0000 7fd47d600640 1 -- 172.21.15.119:0/3187625447 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd474030bb0 con 0x7fd4800aedc0 2024-04-25T10:10:06.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.522+0000 7fd47d600640 1 --2- 172.21.15.119:0/3187625447 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd46406b4d0 0x7fd46406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.522+0000 7fd47d600640 1 -- 172.21.15.119:0/3187625447 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4740aad80 con 0x7fd4800aedc0 2024-04-25T10:10:06.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.522+0000 7fd484c00640 1 --2- 172.21.15.119:0/3187625447 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd46406b4d0 0x7fd46406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.522+0000 7fd48af70740 1 Processor -- start 2024-04-25T10:10:06.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.523+0000 7fd48af70740 1 -- start start 2024-04-25T10:10:06.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.523+0000 7fd484c00640 1 --2- 172.21.15.119:0/3187625447 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd46406b4d0 0x7fd46406d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd48006f1f0 tx=0x7fd470008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.523+0000 7fd48af70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55657706c950 0x556577094790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.523+0000 7fd48af70740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 0x55657706cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.523+0000 7fd48af70740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565770b55c0 0x55657706b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.523+0000 7fd48af70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556576fd0620 con 0x55657706c950 2024-04-25T10:10:06.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.523+0000 7fd48af70740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556576f79ca0 con 0x556577094cd0 2024-04-25T10:10:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.523+0000 7fd48af70740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556576fcc6c0 con 0x5565770b55c0 2024-04-25T10:10:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.524+0000 7fd485600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55657706c950 0x556577094790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.524+0000 7fd485600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55657706c950 0x556577094790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39302/0 (socket says 172.21.15.119:39302) 2024-04-25T10:10:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.524+0000 7fd485600640 1 -- 172.21.15.119:0/2012190497 learned_addr learned my addr 172.21.15.119:0/2012190497 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.524+0000 7fd486000640 1 --2- 172.21.15.119:0/2012190497 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565770b55c0 0x55657706b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.524+0000 7fd485600640 1 -- 172.21.15.119:0/2012190497 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565770b55c0 msgr2=0x55657706b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.524+0000 7fd484c00640 1 --2- 172.21.15.119:0/2012190497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 0x55657706cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.524+0000 7fd485600640 1 --2- 172.21.15.119:0/2012190497 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565770b55c0 0x55657706b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.524+0000 7fd485600640 1 -- 172.21.15.119:0/2012190497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 msgr2=0x55657706cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.524+0000 7fd485600640 1 --2- 172.21.15.119:0/2012190497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 0x55657706cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.524+0000 7fd485600640 1 -- 172.21.15.119:0/2012190497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556576f80f40 con 0x55657706c950 2024-04-25T10:10:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.524+0000 7fd486000640 1 --2- 172.21.15.119:0/2012190497 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565770b55c0 0x55657706b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.524+0000 7fd484c00640 1 --2- 172.21.15.119:0/2012190497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 0x55657706cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.524+0000 7fd485600640 1 --2- 172.21.15.119:0/2012190497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55657706c950 0x556577094790 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd474081640 tx=0x7fd474079360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e468516b6b88a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.524+0000 7fd46dc00640 1 -- 172.21.15.119:0/2012190497 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474079b60 con 0x55657706c950 2024-04-25T10:10:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.525+0000 7fd46dc00640 1 -- 172.21.15.119:0/2012190497 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd474079460 con 0x55657706c950 2024-04-25T10:10:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.525+0000 7fd46dc00640 1 -- 172.21.15.119:0/2012190497 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47407d5b0 con 0x55657706c950 2024-04-25T10:10:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.525+0000 7fd48af70740 1 -- 172.21.15.119:0/2012190497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55657706c950 msgr2=0x556577094790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.525+0000 7fd48af70740 1 --2- 172.21.15.119:0/2012190497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55657706c950 0x556577094790 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd474081640 tx=0x7fd474079360 comp rx=0 tx=0).stop 2024-04-25T10:10:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.525+0000 7fd48af70740 1 -- 172.21.15.119:0/2012190497 shutdown_connections 2024-04-25T10:10:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.525+0000 7fd48af70740 1 --2- 172.21.15.119:0/2012190497 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565770b55c0 0x55657706b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.525+0000 7fd48af70740 1 --2- 172.21.15.119:0/2012190497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 0x55657706cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.525+0000 7fd48af70740 1 --2- 172.21.15.119:0/2012190497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55657706c950 0x556577094790 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.525+0000 7fd48af70740 1 -- 172.21.15.119:0/2012190497 >> 172.21.15.119:0/2012190497 conn(0x556577069b70 msgr2=0x5565770b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.525+0000 7fd48af70740 1 -- 172.21.15.119:0/2012190497 shutdown_connections 2024-04-25T10:10:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.525+0000 7fd48af70740 1 -- 172.21.15.119:0/2012190497 wait complete. 2024-04-25T10:10:06.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.526+0000 7fd48af70740 1 -- start start 2024-04-25T10:10:06.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.526+0000 7fd48af70740 1 Processor -- start 2024-04-25T10:10:06.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.527+0000 7fd48af70740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556577094cd0 0x556577099110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.527+0000 7fd48af70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565770b55c0 0x556577092a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.527+0000 7fd48af70740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577099650 0x556577092fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.527+0000 7fd48af70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556576fd0620 con 0x5565770b55c0 2024-04-25T10:10:06.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.527+0000 7fd48af70740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556576f6f1f0 con 0x556577099650 2024-04-25T10:10:06.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.527+0000 7fd48af70740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556577094300 con 0x556577094cd0 2024-04-25T10:10:06.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.527+0000 7fd48af70740 10 client.0 _ll_register_callbacks cb 0x556576f8cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:06.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.528+0000 7fd484c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565770b55c0 0x556577092a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.528+0000 7fd484c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565770b55c0 0x556577092a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39310/0 (socket says 172.21.15.119:39310) 2024-04-25T10:10:06.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.528+0000 7fd484c00640 1 -- 172.21.15.119:0/1152453711 learned_addr learned my addr 172.21.15.119:0/1152453711 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:06.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.528+0000 7fd485600640 1 --2- 172.21.15.119:0/1152453711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556577094cd0 0x556577099110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.528+0000 7fd486000640 1 --2- 172.21.15.119:0/1152453711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577099650 0x556577092fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.528+0000 7fd484c00640 1 -- 172.21.15.119:0/1152453711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556577094cd0 msgr2=0x556577099110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.528+0000 7fd484c00640 1 --2- 172.21.15.119:0/1152453711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556577094cd0 0x556577099110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.528+0000 7fd484c00640 1 -- 172.21.15.119:0/1152453711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577099650 msgr2=0x556577092fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.528+0000 7fd484c00640 1 --2- 172.21.15.119:0/1152453711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577099650 0x556577092fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.528+0000 7fd484c00640 1 -- 172.21.15.119:0/1152453711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556576f8ec50 con 0x5565770b55c0 2024-04-25T10:10:06.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.528+0000 7fd486000640 1 --2- 172.21.15.119:0/1152453711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577099650 0x556577092fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:06.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.528+0000 7fd484c00640 1 --2- 172.21.15.119:0/1152453711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565770b55c0 0x556577092a90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd470009290 tx=0x7fd470006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78edeab1b3e0b1be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.528+0000 7fd462000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:10:06.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.529+0000 7fd462000640 1 -- 172.21.15.119:0/1152453711 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47000de70 con 0x5565770b55c0 2024-04-25T10:10:06.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.529+0000 7fd48af70740 1 -- 172.21.15.119:0/1152453711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd474079460 con 0x5565770b55c0 2024-04-25T10:10:06.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.529+0000 7fd462000640 1 -- 172.21.15.119:0/1152453711 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4700125f0 con 0x5565770b55c0 2024-04-25T10:10:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.529+0000 7fd462000640 1 -- 172.21.15.119:0/1152453711 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd470005990 con 0x5565770b55c0 2024-04-25T10:10:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.529+0000 7fd48af70740 10 client.6078 fetch_fsmap learned FSMap version 34 2024-04-25T10:10:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.529+0000 7fd48af70740 1 -- 172.21.15.119:0/1152453711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x556576ce52d0 con 0x5565770b55c0 2024-04-25T10:10:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.529+0000 7fd462000640 1 -- 172.21.15.119:0/1152453711 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd474079460 con 0x5565770b55c0 2024-04-25T10:10:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.529+0000 7fd462000640 1 -- 172.21.15.119:0/1152453711 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd47002f080 con 0x5565770b55c0 2024-04-25T10:10:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.529+0000 7fd48af70740 10 client.6078 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T10:10:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.529+0000 7fd48af70740 10 client.6078 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:10:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.529+0000 7fd48af70740 20 client.6078 populate_metadata read hostname 'smithi119' 2024-04-25T10:10:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.529+0000 7fd48af70740 1 --2- 172.21.15.119:0/1152453711 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5565770955c0 0x5565770959c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.529+0000 7fd48af70740 4 client.6078 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.529+0000 7fd48af70740 1 -- 172.21.15.119:0/1152453711 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556576fddb60 con 0x5565770955c0 2024-04-25T10:10:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.530+0000 7fd485600640 1 --2- 172.21.15.119:0/1152453711 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5565770955c0 0x5565770959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.530+0000 7fd485600640 1 --2- 172.21.15.119:0/1152453711 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5565770955c0 0x5565770959c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=142c4b1fff3f6d98 server_cookie=35fde3e407024696 in_seq=0 out_seq=0 2024-04-25T10:10:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.530+0000 7fd462000640 10 client.6078 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.531+0000 7fd462000640 1 -- 172.21.15.119:0/1152453711 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556576fddb60 con 0x5565770955c0 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.531+0000 7fd462000640 10 client.6078 handle_command_reply: tid=0 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.531+0000 7fd48af70740 1 client.6078 shutdown 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.531+0000 7fd48af70740 1 -- 172.21.15.119:0/1152453711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565770b55c0 msgr2=0x556577092a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.531+0000 7fd48af70740 1 --2- 172.21.15.119:0/1152453711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565770b55c0 0x556577092a90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd470009290 tx=0x7fd470006e10 comp rx=0 tx=0).stop 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.531+0000 7fd48af70740 1 -- 172.21.15.119:0/1152453711 shutdown_connections 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.531+0000 7fd48af70740 1 --2- 172.21.15.119:0/1152453711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577099650 0x556577092fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.531+0000 7fd48af70740 1 --2- 172.21.15.119:0/1152453711 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5565770955c0 0x5565770959c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.531+0000 7fd48af70740 1 --2- 172.21.15.119:0/1152453711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565770b55c0 0x556577092a90 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.531+0000 7fd48af70740 1 --2- 172.21.15.119:0/1152453711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556577094cd0 0x556577099110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.531+0000 7fd462000640 0 client.6078 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.531+0000 7fd48af70740 1 -- 172.21.15.119:0/1152453711 >> 172.21.15.119:0/1152453711 conn(0x55657706c950 msgr2=0x5565770b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.532+0000 7fd48af70740 1 -- 172.21.15.119:0/1152453711 shutdown_connections 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.532+0000 7fd48af70740 1 -- 172.21.15.119:0/1152453711 wait complete. 2024-04-25T10:10:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.532+0000 7fd48af70740 20 client.6078 trim_cache size 0 max 16384 2024-04-25T10:10:06.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.549+0000 7fd48af70740 1 Processor -- start 2024-04-25T10:10:06.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.549+0000 7fd48af70740 1 -- start start 2024-04-25T10:10:06.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.549+0000 7fd48af70740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 0x556577066de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.549+0000 7fd48af70740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556577099650 0x556577099f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.549+0000 7fd48af70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565770b55c0 0x5565770959a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.549+0000 7fd48af70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4801ff650 con 0x5565770b55c0 2024-04-25T10:10:06.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.549+0000 7fd48af70740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4801f4340 con 0x556577094cd0 2024-04-25T10:10:06.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.549+0000 7fd48af70740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4801ff170 con 0x556577099650 2024-04-25T10:10:06.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.550+0000 7fd486000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565770b55c0 0x5565770959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.550+0000 7fd486000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565770b55c0 0x5565770959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39324/0 (socket says 172.21.15.119:39324) 2024-04-25T10:10:06.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.550+0000 7fd486000640 1 -- 172.21.15.119:0/2798940355 learned_addr learned my addr 172.21.15.119:0/2798940355 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:06.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.550+0000 7fd484c00640 1 --2- 172.21.15.119:0/2798940355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556577099650 0x556577099f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.550+0000 7fd485600640 1 --2- 172.21.15.119:0/2798940355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 0x556577066de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.550+0000 7fd486000640 1 -- 172.21.15.119:0/2798940355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556577099650 msgr2=0x556577099f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.550+0000 7fd486000640 1 --2- 172.21.15.119:0/2798940355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556577099650 0x556577099f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.550+0000 7fd486000640 1 -- 172.21.15.119:0/2798940355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 msgr2=0x556577066de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.550+0000 7fd486000640 1 --2- 172.21.15.119:0/2798940355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 0x556577066de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.550+0000 7fd486000640 1 -- 172.21.15.119:0/2798940355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556576daf6b0 con 0x5565770b55c0 2024-04-25T10:10:06.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.551+0000 7fd486000640 1 --2- 172.21.15.119:0/2798940355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565770b55c0 0x5565770959a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd478001fe0 tx=0x7fd47800e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f3f4ec3add4c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.551+0000 7fd46d200640 1 -- 172.21.15.119:0/2798940355 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780072d0 con 0x5565770b55c0 2024-04-25T10:10:06.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.551+0000 7fd46d200640 1 -- 172.21.15.119:0/2798940355 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd478007470 con 0x5565770b55c0 2024-04-25T10:10:06.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.551+0000 7fd46d200640 1 -- 172.21.15.119:0/2798940355 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478007860 con 0x5565770b55c0 2024-04-25T10:10:06.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.552+0000 7fd48af70740 1 -- 172.21.15.119:0/2798940355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565770b55c0 msgr2=0x5565770959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.552+0000 7fd48af70740 1 --2- 172.21.15.119:0/2798940355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565770b55c0 0x5565770959a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd478001fe0 tx=0x7fd47800e590 comp rx=0 tx=0).stop 2024-04-25T10:10:06.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.552+0000 7fd48af70740 1 -- 172.21.15.119:0/2798940355 shutdown_connections 2024-04-25T10:10:06.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.552+0000 7fd48af70740 1 --2- 172.21.15.119:0/2798940355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565770b55c0 0x5565770959a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.552+0000 7fd48af70740 1 --2- 172.21.15.119:0/2798940355 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556577099650 0x556577099f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.552+0000 7fd48af70740 1 --2- 172.21.15.119:0/2798940355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 0x556577066de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.552+0000 7fd48af70740 1 -- 172.21.15.119:0/2798940355 >> 172.21.15.119:0/2798940355 conn(0x55657706c950 msgr2=0x55657709af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.552+0000 7fd48af70740 1 -- 172.21.15.119:0/2798940355 shutdown_connections 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.552+0000 7fd48af70740 1 -- 172.21.15.119:0/2798940355 wait complete. 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.552+0000 7fd48af70740 1 -- start start 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.553+0000 7fd48af70740 1 Processor -- start 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd48af70740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 0x556577067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd48af70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556577099650 0x55657711c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd48af70740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565770b55c0 0x556577083b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd48af70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556577085000 con 0x556577099650 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd48af70740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565770c2ec0 con 0x556577094cd0 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd48af70740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565770c23d0 con 0x5565770b55c0 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd48af70740 10 client.0 _ll_register_callbacks cb 0x7fd470009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd484c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556577099650 0x55657711c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd484c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556577099650 0x55657711c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39338/0 (socket says 172.21.15.119:39338) 2024-04-25T10:10:06.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd484c00640 1 -- 172.21.15.119:0/929424778 learned_addr learned my addr 172.21.15.119:0/929424778 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:06.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd484c00640 1 -- 172.21.15.119:0/929424778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565770b55c0 msgr2=0x556577083b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd486000640 1 --2- 172.21.15.119:0/929424778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565770b55c0 0x556577083b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd484c00640 1 --2- 172.21.15.119:0/929424778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565770b55c0 0x556577083b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd485600640 1 --2- 172.21.15.119:0/929424778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 0x556577067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd484c00640 1 -- 172.21.15.119:0/929424778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 msgr2=0x556577067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd484c00640 1 --2- 172.21.15.119:0/929424778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 0x556577067a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.554+0000 7fd484c00640 1 -- 172.21.15.119:0/929424778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556577092ff0 con 0x556577099650 2024-04-25T10:10:06.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.555+0000 7fd484c00640 1 --2- 172.21.15.119:0/929424778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556577099650 0x55657711c1f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd47000d5a0 tx=0x7fd470014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac72972f7c7fc04f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.555+0000 7fd462000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:10:06.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.555+0000 7fd462000640 1 -- 172.21.15.119:0/929424778 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47003b060 con 0x556577099650 2024-04-25T10:10:06.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.555+0000 7fd462000640 1 -- 172.21.15.119:0/929424778 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd470005ef0 con 0x556577099650 2024-04-25T10:10:06.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.555+0000 7fd48af70740 1 -- 172.21.15.119:0/929424778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd478007470 con 0x556577099650 2024-04-25T10:10:06.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.555+0000 7fd462000640 1 -- 172.21.15.119:0/929424778 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd470041050 con 0x556577099650 2024-04-25T10:10:06.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.555+0000 7fd48af70740 10 client.6084 fetch_fsmap learned FSMap version 34 2024-04-25T10:10:06.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.555+0000 7fd48af70740 1 -- 172.21.15.119:0/929424778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55657708fda0 con 0x556577099650 2024-04-25T10:10:06.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.555+0000 7fd462000640 1 -- 172.21.15.119:0/929424778 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd478007470 con 0x556577099650 2024-04-25T10:10:06.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.556+0000 7fd462000640 1 -- 172.21.15.119:0/929424778 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd470044020 con 0x556577099650 2024-04-25T10:10:06.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.556+0000 7fd48af70740 10 client.6084 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T10:10:06.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.556+0000 7fd48af70740 10 client.6084 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:10:06.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.556+0000 7fd48af70740 20 client.6084 populate_metadata read hostname 'smithi119' 2024-04-25T10:10:06.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.556+0000 7fd48af70740 1 --2- 172.21.15.119:0/929424778 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5565770929e0 0x5565770959a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.556+0000 7fd48af70740 4 client.6084 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:10:06.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.556+0000 7fd48af70740 1 -- 172.21.15.119:0/929424778 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55657709ac60 con 0x5565770929e0 2024-04-25T10:10:06.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.556+0000 7fd485600640 1 --2- 172.21.15.119:0/929424778 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5565770929e0 0x5565770959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.556+0000 7fd485600640 1 --2- 172.21.15.119:0/929424778 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5565770929e0 0x5565770959a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f7f2f71900d2a0b server_cookie=cad2ede710b95a4f in_seq=0 out_seq=0 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.556+0000 7fd462000640 10 client.6084 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd462000640 1 -- 172.21.15.119:0/929424778 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55657709ac60 con 0x5565770929e0 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd462000640 10 client.6084 handle_command_reply: tid=0 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd48af70740 1 client.6084 shutdown 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd48af70740 1 -- 172.21.15.119:0/929424778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556577099650 msgr2=0x55657711c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd48af70740 1 --2- 172.21.15.119:0/929424778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556577099650 0x55657711c1f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd47000d5a0 tx=0x7fd470014000 comp rx=0 tx=0).stop 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd48af70740 1 -- 172.21.15.119:0/929424778 shutdown_connections 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd48af70740 1 --2- 172.21.15.119:0/929424778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565770b55c0 0x556577083b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd48af70740 1 --2- 172.21.15.119:0/929424778 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5565770929e0 0x5565770959a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd48af70740 1 --2- 172.21.15.119:0/929424778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556577099650 0x55657711c1f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd48af70740 1 --2- 172.21.15.119:0/929424778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556577094cd0 0x556577067a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd462000640 0 client.6084 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd48af70740 1 -- 172.21.15.119:0/929424778 >> 172.21.15.119:0/929424778 conn(0x55657706c950 msgr2=0x556577099f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd48af70740 1 -- 172.21.15.119:0/929424778 shutdown_connections 2024-04-25T10:10:06.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd48af70740 1 -- 172.21.15.119:0/929424778 wait complete. 2024-04-25T10:10:06.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.557+0000 7fd48af70740 20 client.6084 trim_cache size 0 max 16384 2024-04-25T10:10:06.568 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:10:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "e0b0f428-b441-4237-a4d1-ed53321f87a5", 2024-04-25T10:10:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-04-25T10:10:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 5931, 2024-04-25T10:10:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-04-25T10:10:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-04-25T10:10:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-04-25T10:10:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.5155928740000002, 2024-04-25T10:10:06.568 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 34, 2024-04-25T10:10:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 55, 2024-04-25T10:10:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 55, 2024-04-25T10:10:06.569 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 11.984694232000001 2024-04-25T10:10:06.569 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:10:06.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.558+0000 7fd488000640 1 -- 172.21.15.119:0/3187625447 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd46406b4d0 msgr2=0x7fd46406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.558+0000 7fd488000640 1 --2- 172.21.15.119:0/3187625447 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd46406b4d0 0x7fd46406d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd48006f1f0 tx=0x7fd470008040 comp rx=0 tx=0).stop 2024-04-25T10:10:06.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.558+0000 7fd488000640 1 -- 172.21.15.119:0/3187625447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4800aedc0 msgr2=0x7fd48014c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.558+0000 7fd488000640 1 --2- 172.21.15.119:0/3187625447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4800aedc0 0x7fd48014c1d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd4740098c0 tx=0x7fd4740046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.558+0000 7fd488000640 1 -- 172.21.15.119:0/3187625447 shutdown_connections 2024-04-25T10:10:06.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.558+0000 7fd488000640 1 --2- 172.21.15.119:0/3187625447 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd46406b4d0 0x7fd46406d990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.558+0000 7fd488000640 1 --2- 172.21.15.119:0/3187625447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd48014f610 0x7fd48015bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.558+0000 7fd488000640 1 --2- 172.21.15.119:0/3187625447 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800cf900 0x7fd4800c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.558+0000 7fd488000640 1 --2- 172.21.15.119:0/3187625447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4800aedc0 0x7fd48014c1d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.558+0000 7fd488000640 1 -- 172.21.15.119:0/3187625447 >> 172.21.15.119:0/3187625447 conn(0x7fd48006aba0 msgr2=0x7fd48015eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.558+0000 7fd488000640 1 -- 172.21.15.119:0/3187625447 shutdown_connections 2024-04-25T10:10:06.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.558+0000 7fd488000640 1 -- 172.21.15.119:0/3187625447 wait complete. 2024-04-25T10:10:06.582 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:10:06.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.777+0000 7fc4d2e00640 1 Processor -- start 2024-04-25T10:10:06.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.778+0000 7fc4d2e00640 1 -- start start 2024-04-25T10:10:06.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.779+0000 7fc4d2e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4cc15dce0 0x7fc4cc0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.779+0000 7fc4d2e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc4cc0d35b0 0x7fc4cc0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.779+0000 7fc4d2e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4cc0d1f90 0x7fc4cc0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.779+0000 7fc4d2e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4cc15e0c0 con 0x7fc4cc15dce0 2024-04-25T10:10:06.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.779+0000 7fc4d2e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4cc0d2510 con 0x7fc4cc0d35b0 2024-04-25T10:10:06.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.779+0000 7fc4d2e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc0d27f0 con 0x7fc4cc0d1f90 2024-04-25T10:10:06.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.779+0000 7fc4cbe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4cc15dce0 0x7fc4cc0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.779+0000 7fc4cbe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4cc15dce0 0x7fc4cc0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39342/0 (socket says 172.21.15.119:39342) 2024-04-25T10:10:06.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.779+0000 7fc4cbe00640 1 -- 172.21.15.119:0/726592578 learned_addr learned my addr 172.21.15.119:0/726592578 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:06.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.780+0000 7fc4c3400640 1 --2- 172.21.15.119:0/726592578 >> v2:172.21.15.132:3300/0 conn(0x7fc4cc0d35b0 0x7fc4cc0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.780+0000 7fc4d0e00640 1 --2- 172.21.15.119:0/726592578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4cc0d1f90 0x7fc4cc0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.780+0000 7fc4cbe00640 1 -- 172.21.15.119:0/726592578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4cc0d1f90 msgr2=0x7fc4cc0b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.780+0000 7fc4cbe00640 1 --2- 172.21.15.119:0/726592578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4cc0d1f90 0x7fc4cc0b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.780+0000 7fc4cbe00640 1 -- 172.21.15.119:0/726592578 >> v2:172.21.15.132:3300/0 conn(0x7fc4cc0d35b0 msgr2=0x7fc4cc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.780+0000 7fc4cbe00640 1 --2- 172.21.15.119:0/726592578 >> v2:172.21.15.132:3300/0 conn(0x7fc4cc0d35b0 0x7fc4cc0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.780+0000 7fc4cbe00640 1 -- 172.21.15.119:0/726592578 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc0d2ad0 con 0x7fc4cc15dce0 2024-04-25T10:10:06.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.780+0000 7fc4c3400640 1 --2- 172.21.15.119:0/726592578 >> v2:172.21.15.132:3300/0 conn(0x7fc4cc0d35b0 0x7fc4cc0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:06.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.781+0000 7fc4cbe00640 1 --2- 172.21.15.119:0/726592578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc15dce0 0x7fc4cc0d1a50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0cdfa0 tx=0x7fc4bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4057f9e51e4fb3cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.781+0000 7fc4cb400640 1 -- 172.21.15.119:0/726592578 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc037070 con 0x7fc4cc15dce0 2024-04-25T10:10:06.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.781+0000 7fc4cb400640 1 -- 172.21.15.119:0/726592578 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc032440 con 0x7fc4cc15dce0 2024-04-25T10:10:06.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.781+0000 7fc4cb400640 1 -- 172.21.15.119:0/726592578 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc032810 con 0x7fc4cc15dce0 2024-04-25T10:10:06.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.782+0000 7fc4d2e00640 1 -- 172.21.15.119:0/726592578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc15dce0 msgr2=0x7fc4cc0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.782+0000 7fc4d2e00640 1 --2- 172.21.15.119:0/726592578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc15dce0 0x7fc4cc0d1a50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0cdfa0 tx=0x7fc4bc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.782+0000 7fc4d2e00640 1 -- 172.21.15.119:0/726592578 shutdown_connections 2024-04-25T10:10:06.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.782+0000 7fc4d2e00640 1 --2- 172.21.15.119:0/726592578 >> v2:172.21.15.132:3300/0 conn(0x7fc4cc0d35b0 0x7fc4cc0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.782+0000 7fc4d2e00640 1 --2- 172.21.15.119:0/726592578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4cc0d1f90 0x7fc4cc0b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.782+0000 7fc4d2e00640 1 --2- 172.21.15.119:0/726592578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc15dce0 0x7fc4cc0d1a50 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.782+0000 7fc4d2e00640 1 -- 172.21.15.119:0/726592578 >> 172.21.15.119:0/726592578 conn(0x7fc4cc072900 msgr2=0x7fc4cc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.782+0000 7fc4d2e00640 1 -- 172.21.15.119:0/726592578 shutdown_connections 2024-04-25T10:10:06.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.782+0000 7fc4d2e00640 1 -- 172.21.15.119:0/726592578 wait complete. 2024-04-25T10:10:06.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.784+0000 7fc4d2e00640 1 Processor -- start 2024-04-25T10:10:06.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.784+0000 7fc4d2e00640 1 -- start start 2024-04-25T10:10:06.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.785+0000 7fc4d2e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4cc0d1f90 0x7fc4cc0b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.785+0000 7fc4d2e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc0d35b0 0x7fc4cc1f3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.785+0000 7fc4d2e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4cc15dce0 0x7fc4cc0aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.785+0000 7fc4d2e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc0b9820 con 0x7fc4cc0d35b0 2024-04-25T10:10:06.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.785+0000 7fc4d2e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc0d3090 con 0x7fc4cc15dce0 2024-04-25T10:10:06.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.785+0000 7fc4d2e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc0b99a0 con 0x7fc4cc0d1f90 2024-04-25T10:10:06.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.785+0000 7fc4c3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc0d35b0 0x7fc4cc1f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.785+0000 7fc4c3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc0d35b0 0x7fc4cc1f3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39350/0 (socket says 172.21.15.119:39350) 2024-04-25T10:10:06.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.785+0000 7fc4c3400640 1 -- 172.21.15.119:0/2633208378 learned_addr learned my addr 172.21.15.119:0/2633208378 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:06.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.786+0000 7fc4cbe00640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4cc0d1f90 0x7fc4cc0b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.786+0000 7fc4d0e00640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4cc15dce0 0x7fc4cc0aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.786+0000 7fc4c3400640 1 -- 172.21.15.119:0/2633208378 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4cc0d1f90 msgr2=0x7fc4cc0b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.786+0000 7fc4c3400640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4cc0d1f90 0x7fc4cc0b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.786+0000 7fc4c3400640 1 -- 172.21.15.119:0/2633208378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4cc15dce0 msgr2=0x7fc4cc0aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.786+0000 7fc4c3400640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4cc15dce0 0x7fc4cc0aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.786+0000 7fc4c3400640 1 -- 172.21.15.119:0/2633208378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc0b21a0 con 0x7fc4cc0d35b0 2024-04-25T10:10:06.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.786+0000 7fc4cbe00640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4cc0d1f90 0x7fc4cc0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:06.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.786+0000 7fc4d0e00640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4cc15dce0 0x7fc4cc0aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:06.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.786+0000 7fc4c3400640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc0d35b0 0x7fc4cc1f3860 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400c7f0 tx=0x7fc4b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.786+0000 7fc4c8c00640 1 -- 172.21.15.119:0/2633208378 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4015070 con 0x7fc4cc0d35b0 2024-04-25T10:10:06.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.786+0000 7fc4c8c00640 1 -- 172.21.15.119:0/2633208378 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b4007a30 con 0x7fc4cc0d35b0 2024-04-25T10:10:06.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.786+0000 7fc4d2e00640 1 -- 172.21.15.119:0/2633208378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4cc15e2e0 con 0x7fc4cc0d35b0 2024-04-25T10:10:06.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.786+0000 7fc4c8c00640 1 -- 172.21.15.119:0/2633208378 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4007e00 con 0x7fc4cc0d35b0 2024-04-25T10:10:06.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.787+0000 7fc4d2e00640 1 -- 172.21.15.119:0/2633208378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4cc15e8a0 con 0x7fc4cc0d35b0 2024-04-25T10:10:06.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.788+0000 7fc4d2e00640 1 -- 172.21.15.119:0/2633208378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4cc0749b0 con 0x7fc4cc0d35b0 2024-04-25T10:10:06.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.789+0000 7fc4c8c00640 1 -- 172.21.15.119:0/2633208378 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4b4018070 con 0x7fc4cc0d35b0 2024-04-25T10:10:06.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.790+0000 7fc4c8c00640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc4a806b4d0 0x7fc4a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.790+0000 7fc4c8c00640 1 -- 172.21.15.119:0/2633208378 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4b408a1c0 con 0x7fc4cc0d35b0 2024-04-25T10:10:06.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.790+0000 7fc4cbe00640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc4a806b4d0 0x7fc4a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.791+0000 7fc4cbe00640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc4a806b4d0 0x7fc4a806d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc034000 tx=0x7fc4bc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:06.795+0000 7fc4c8c00640 1 -- 172.21.15.119:0/2633208378 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4b4058fa0 con 0x7fc4cc0d35b0 2024-04-25T10:10:07.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:07.033+0000 7fc4d2e00640 1 -- 172.21.15.119:0/2633208378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc4cc15d220 con 0x7fc4cc0d35b0 2024-04-25T10:10:07.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:07.034+0000 7fc4c8c00640 1 -- 172.21.15.119:0/2633208378 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc4b40588a0 con 0x7fc4cc0d35b0 2024-04-25T10:10:07.040 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:10:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:07.037+0000 7fc4d2e00640 1 -- 172.21.15.119:0/2633208378 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc4a806b4d0 msgr2=0x7fc4a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:07.037+0000 7fc4d2e00640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc4a806b4d0 0x7fc4a806d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc034000 tx=0x7fc4bc02e000 comp rx=0 tx=0).stop 2024-04-25T10:10:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:07.037+0000 7fc4d2e00640 1 -- 172.21.15.119:0/2633208378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc0d35b0 msgr2=0x7fc4cc1f3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:07.037+0000 7fc4d2e00640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc0d35b0 0x7fc4cc1f3860 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400c7f0 tx=0x7fc4b400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:10:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:07.038+0000 7fc4d2e00640 1 -- 172.21.15.119:0/2633208378 shutdown_connections 2024-04-25T10:10:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:07.038+0000 7fc4d2e00640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc4a806b4d0 0x7fc4a806d990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:07.038+0000 7fc4d2e00640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4cc15dce0 0x7fc4cc0aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:07.038+0000 7fc4d2e00640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4cc0d35b0 0x7fc4cc1f3860 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:07.038+0000 7fc4d2e00640 1 --2- 172.21.15.119:0/2633208378 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4cc0d1f90 0x7fc4cc0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:07.038+0000 7fc4d2e00640 1 -- 172.21.15.119:0/2633208378 >> 172.21.15.119:0/2633208378 conn(0x7fc4cc072900 msgr2=0x7fc4cc06bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:07.038+0000 7fc4d2e00640 1 -- 172.21.15.119:0/2633208378 shutdown_connections 2024-04-25T10:10:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:07.038+0000 7fc4d2e00640 1 -- 172.21.15.119:0/2633208378 wait complete. 2024-04-25T10:10:07.061 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:10:07.061 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:10:07.061 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:10:07.061 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:10:07.061 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:07.061 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:10:07.061 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:10:07.061 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:10:07.061 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:07.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:07.062 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:10:07.106 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:10:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:10:07.106 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:10:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:10:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:07.107 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:10:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:10:07.107 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:10:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:10:07.107 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:10:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:07.107 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:10:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:07.107 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42482sec preferred_lft 42482sec 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:07.108 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:10:07.109 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:07.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:07.110 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:10:07.232 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:10:07.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:07.237 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:10:07.291 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:10:07.291 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:10:07.291 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:10:07.291 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:10:07.291 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:10:07.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:07.292 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:10:07.292 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:10:07.292 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:10:07.292 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:10:07.292 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:10:07.292 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:10:07.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:07.440 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:10:07.490 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:10:07.491 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:10:07.491 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:07.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:07.491 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:10:07.570 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:10:07.570 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:07.621 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:07.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:10:07.622 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:07.671 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:10:07.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:07.672 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:10:07.798 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:10:07.798 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:10:07.799 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:10:07.820 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:10:07.820 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:10:07.820 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:10:07.820 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:10:07.820 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:10:07.850 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:10:07.850 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:10:07.850 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:10:07.850 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:10:07.850 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:10:07.850 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:10:07.851 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:10:07.851 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:10:07.852 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:10:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:10:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:10:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:10:07.852 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:10:07.856 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:10:07.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:07.857 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:07.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:07.928 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:10:07.992 INFO:teuthology.orchestra.run.smithi119.stdout:client4880 2024-04-25T10:10:07.993 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:10:07.993 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:08.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:08.043 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-25T10:10:08.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:08.104 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-25T10:10:08.156 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:10:08.156 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:10:08.156 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:10:08.157 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:10:08.157 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-25T10:10:08.157 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:10:08.288 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627777 2024-04-25T10:10:08.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:10:08.301 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:08.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.541+0000 7f1c0b400640 1 Processor -- start 2024-04-25T10:10:08.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.541+0000 7f1c0b400640 1 -- start start 2024-04-25T10:10:08.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.543+0000 7f1c0b400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1c0406e8f0 0x7f1c0415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.543+0000 7f1c0b400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c040ce6c0 con 0x7f1c0415bf00 2024-04-25T10:10:08.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.543+0000 7f1c0b400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c040cc4e0 con 0x7f1c0415c820 2024-04-25T10:10:08.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.543+0000 7f1c0b400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c041455e0 con 0x7f1c0406e8f0 2024-04-25T10:10:08.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.543+0000 7f1c08a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1c0415bf00 0x7f1c040cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51488/0 (socket says 172.21.15.119:51488) 2024-04-25T10:10:08.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.543+0000 7f1c08a00640 1 -- 172.21.15.119:0/2345425387 learned_addr learned my addr 172.21.15.119:0/2345425387 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:10:08.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03e00640 1 --2- 172.21.15.119:0/2345425387 >> v2:172.21.15.132:3300/0 conn(0x7f1c0406e8f0 0x7f1c0415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 -- 172.21.15.119:0/2345425387 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924418609 0 0) 0x7f1bf8001110 con 0x7f1c0415bf00 2024-04-25T10:10:08.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 -- 172.21.15.119:0/2345425387 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bec003520 con 0x7f1c0415bf00 2024-04-25T10:10:08.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 -- 172.21.15.119:0/2345425387 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1049193928 0 0) 0x7f1bec003520 con 0x7f1c0415bf00 2024-04-25T10:10:08.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 -- 172.21.15.119:0/2345425387 >> v2:172.21.15.132:3300/0 conn(0x7f1c0406e8f0 msgr2=0x7f1c0415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 --2- 172.21.15.119:0/2345425387 >> v2:172.21.15.132:3300/0 conn(0x7f1c0406e8f0 0x7f1c0415c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 -- 172.21.15.119:0/2345425387 >> v1:172.21.15.132:6789/0 conn(0x7f1c0415c820 legacy=0x7f1c041654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 -- 172.21.15.119:0/2345425387 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c040cc7b0 con 0x7f1c0415bf00 2024-04-25T10:10:08.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 -- 172.21.15.119:0/2345425387 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f1bf8001840 con 0x7f1c0415bf00 2024-04-25T10:10:08.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 -- 172.21.15.119:0/2345425387 >> v1:172.21.15.119:6789/0 conn(0x7f1c0415bf00 legacy=0x7f1c040cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 --2- 172.21.15.119:0/2345425387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1bec0039b0 0x7f1bec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 --2- 172.21.15.119:0/2345425387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bec004800 0x7f1bec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 --2- 172.21.15.119:0/2345425387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bec0051c0 0x7f1bec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 -- 172.21.15.119:0/2345425387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c041455e0 con 0x7f1bec004800 2024-04-25T10:10:08.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 -- 172.21.15.119:0/2345425387 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec00c110 con 0x7f1bec0051c0 2024-04-25T10:10:08.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.544+0000 7f1c03400640 1 -- 172.21.15.119:0/2345425387 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec00c310 con 0x7f1bec0039b0 2024-04-25T10:10:08.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.545+0000 7f1c03e00640 1 --2- 172.21.15.119:0/2345425387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bec004800 0x7f1bec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.545+0000 7f1c08a00640 1 --2- 172.21.15.119:0/2345425387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1bec0039b0 0x7f1bec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.545+0000 7f1c03e00640 1 -- 172.21.15.119:0/2345425387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1bec0039b0 msgr2=0x7f1bec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.545+0000 7f1c03e00640 1 --2- 172.21.15.119:0/2345425387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1bec0039b0 0x7f1bec003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.545+0000 7f1c03e00640 1 -- 172.21.15.119:0/2345425387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bec0051c0 msgr2=0x7f1bec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:08.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.545+0000 7f1c09400640 1 --2- 172.21.15.119:0/2345425387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bec0051c0 0x7f1bec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.545+0000 7f1c03e00640 1 --2- 172.21.15.119:0/2345425387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bec0051c0 0x7f1bec00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.545+0000 7f1c03e00640 1 -- 172.21.15.119:0/2345425387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec003520 con 0x7f1bec004800 2024-04-25T10:10:08.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.545+0000 7f1c09400640 1 -- 172.21.15.119:0/2345425387 reap_dead start 2024-04-25T10:10:08.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.546+0000 7f1c03e00640 1 --2- 172.21.15.119:0/2345425387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bec004800 0x7f1bec004c80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000ba10 tx=0x7f1bec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e241e4881f016188 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:08.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.546+0000 7f1c03400640 1 -- 172.21.15.119:0/2345425387 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0030070 con 0x7f1bec004800 2024-04-25T10:10:08.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.546+0000 7f1c03400640 1 -- 172.21.15.119:0/2345425387 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf002adb0 con 0x7f1bec004800 2024-04-25T10:10:08.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.546+0000 7f1c03400640 1 -- 172.21.15.119:0/2345425387 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf000b610 con 0x7f1bec004800 2024-04-25T10:10:08.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.547+0000 7f1c0b400640 1 -- 172.21.15.119:0/2345425387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bec004800 msgr2=0x7f1bec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.547+0000 7f1c0b400640 1 --2- 172.21.15.119:0/2345425387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bec004800 0x7f1bec004c80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000ba10 tx=0x7f1bec005830 comp rx=0 tx=0).stop 2024-04-25T10:10:08.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.547+0000 7f1c0b400640 1 -- 172.21.15.119:0/2345425387 shutdown_connections 2024-04-25T10:10:08.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.547+0000 7f1c0b400640 1 --2- 172.21.15.119:0/2345425387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bec004800 0x7f1bec004c80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.547+0000 7f1c0b400640 1 -- 172.21.15.119:0/2345425387 >> 172.21.15.119:0/2345425387 conn(0x7f1c0406aba0 msgr2=0x7f1c0406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:08.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.547+0000 7f1c0b400640 1 -- 172.21.15.119:0/2345425387 shutdown_connections 2024-04-25T10:10:08.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.547+0000 7f1c0b400640 1 -- 172.21.15.119:0/2345425387 wait complete. 2024-04-25T10:10:08.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.549+0000 7f1c0b400640 1 Processor -- start 2024-04-25T10:10:08.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.549+0000 7f1c0b400640 1 -- start start 2024-04-25T10:10:08.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.550+0000 7f1c0b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bec004800 0x7f1c0414d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.550+0000 7f1c0b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c041f4e30 0x7f1c04138b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.550+0000 7f1c0b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c0413e830 0x7f1c0413ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.550+0000 7f1c0b400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040ccd70 con 0x7f1bec004800 2024-04-25T10:10:08.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.550+0000 7f1c0b400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0406d450 con 0x7f1c041f4e30 2024-04-25T10:10:08.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.550+0000 7f1c0b400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040cd050 con 0x7f1c0413e830 2024-04-25T10:10:08.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.550+0000 7f1c08a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bec004800 0x7f1c0414d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.551+0000 7f1c08a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bec004800 0x7f1c0414d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39360/0 (socket says 172.21.15.119:39360) 2024-04-25T10:10:08.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.551+0000 7f1c08a00640 1 -- 172.21.15.119:0/1418811306 learned_addr learned my addr 172.21.15.119:0/1418811306 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:08.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.551+0000 7f1c03e00640 1 --2- 172.21.15.119:0/1418811306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c041f4e30 0x7f1c04138b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.551+0000 7f1c08a00640 1 -- 172.21.15.119:0/1418811306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c0413e830 msgr2=0x7f1c0413ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.551+0000 7f1c08a00640 1 --2- 172.21.15.119:0/1418811306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c0413e830 0x7f1c0413ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.551+0000 7f1c08a00640 1 -- 172.21.15.119:0/1418811306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c041f4e30 msgr2=0x7f1c04138b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.551+0000 7f1c08a00640 1 --2- 172.21.15.119:0/1418811306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c041f4e30 0x7f1c04138b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.551+0000 7f1c08a00640 1 -- 172.21.15.119:0/1418811306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c041f52b0 con 0x7f1bec004800 2024-04-25T10:10:08.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.551+0000 7f1c03e00640 1 --2- 172.21.15.119:0/1418811306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c041f4e30 0x7f1c04138b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:08.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.551+0000 7f1c08a00640 1 --2- 172.21.15.119:0/1418811306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bec004800 0x7f1c0414d470 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800d800 tx=0x7f1bf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:08.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.551+0000 7f1c00c00640 1 -- 172.21.15.119:0/1418811306 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8019070 con 0x7f1bec004800 2024-04-25T10:10:08.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.552+0000 7f1c00c00640 1 -- 172.21.15.119:0/1418811306 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf80074e0 con 0x7f1bec004800 2024-04-25T10:10:08.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.552+0000 7f1c0b400640 1 -- 172.21.15.119:0/1418811306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c041392f0 con 0x7f1bec004800 2024-04-25T10:10:08.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.552+0000 7f1c00c00640 1 -- 172.21.15.119:0/1418811306 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80049e0 con 0x7f1bec004800 2024-04-25T10:10:08.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.552+0000 7f1c0b400640 1 -- 172.21.15.119:0/1418811306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c041398b0 con 0x7f1bec004800 2024-04-25T10:10:08.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.553+0000 7f1c0b400640 1 -- 172.21.15.119:0/1418811306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0406bb30 con 0x7f1bec004800 2024-04-25T10:10:08.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.554+0000 7f1c00c00640 1 -- 172.21.15.119:0/1418811306 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1bf801c070 con 0x7f1bec004800 2024-04-25T10:10:08.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.555+0000 7f1c00c00640 1 --2- 172.21.15.119:0/1418811306 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1be406b4d0 0x7f1be406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:08.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.555+0000 7f1c00c00640 1 -- 172.21.15.119:0/1418811306 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bf8089d40 con 0x7f1bec004800 2024-04-25T10:10:08.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.555+0000 7f1c03e00640 1 --2- 172.21.15.119:0/1418811306 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1be406b4d0 0x7f1be406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:08.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.556+0000 7f1c03e00640 1 --2- 172.21.15.119:0/1418811306 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1be406b4d0 0x7f1be406d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1c040d06c0 tx=0x7f1bf0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:08.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.560+0000 7f1c00c00640 1 -- 172.21.15.119:0/1418811306 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c040ce6c0 con 0x7f1bec004800 2024-04-25T10:10:08.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.790+0000 7f1c0b400640 1 -- 172.21.15.119:0/1418811306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c0413bed0 con 0x7f1bec004800 2024-04-25T10:10:08.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.791+0000 7f1c00c00640 1 -- 172.21.15.119:0/1418811306 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f1c0413bed0 con 0x7f1bec004800 2024-04-25T10:10:08.797 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:10:08.797 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4159222235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4159222235},{"type":"v1","addr":"172.21.15.132:6834","nonce":4159222235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4223361610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":4223361610},{"type":"v1","addr":"172.21.15.119:6837","nonce":4223361610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2367324442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2367324442},{"type":"v1","addr":"172.21.15.132:6835","nonce":2367324442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:59.454941+0000","modified":"2024-04-25T10:10:02.506906+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6836/4129112885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4129112885},{"type":"v1","addr":"172.21.15.119:6836","nonce":4129112885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:10:08.797 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 34 2024-04-25T10:10:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.793+0000 7f1c0b400640 1 -- 172.21.15.119:0/1418811306 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1be406b4d0 msgr2=0x7f1be406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.793+0000 7f1c0b400640 1 --2- 172.21.15.119:0/1418811306 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1be406b4d0 0x7f1be406d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1c040d06c0 tx=0x7f1bf0030000 comp rx=0 tx=0).stop 2024-04-25T10:10:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.793+0000 7f1c0b400640 1 -- 172.21.15.119:0/1418811306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bec004800 msgr2=0x7f1c0414d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.793+0000 7f1c0b400640 1 --2- 172.21.15.119:0/1418811306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bec004800 0x7f1c0414d470 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800d800 tx=0x7f1bf800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.794+0000 7f1c0b400640 1 -- 172.21.15.119:0/1418811306 shutdown_connections 2024-04-25T10:10:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.794+0000 7f1c0b400640 1 --2- 172.21.15.119:0/1418811306 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1be406b4d0 0x7f1be406d990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.794+0000 7f1c0b400640 1 --2- 172.21.15.119:0/1418811306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c0413e830 0x7f1c0413ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.794+0000 7f1c0b400640 1 --2- 172.21.15.119:0/1418811306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c041f4e30 0x7f1c04138b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.794+0000 7f1c0b400640 1 --2- 172.21.15.119:0/1418811306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bec004800 0x7f1c0414d470 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.794+0000 7f1c0b400640 1 -- 172.21.15.119:0/1418811306 >> 172.21.15.119:0/1418811306 conn(0x7f1c0406aba0 msgr2=0x7f1c040cf940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.794+0000 7f1c0b400640 1 -- 172.21.15.119:0/1418811306 shutdown_connections 2024-04-25T10:10:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:08.794+0000 7f1c0b400640 1 -- 172.21.15.119:0/1418811306 wait complete. 2024-04-25T10:10:08.817 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:10:09.063 INFO:teuthology.orchestra.run.smithi119.stdout:{"message":"","return_code":0} 2024-04-25T10:10:09.063 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:10:09.063 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:10:09.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.162+0000 7f0fb740bf40 1 Processor -- start 2024-04-25T10:10:09.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.162+0000 7f0fb740bf40 1 -- start start 2024-04-25T10:10:09.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.163+0000 7f0fb740bf40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561335226220 0x5613352f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.163+0000 7f0fb740bf40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561335271080 0x5613353009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.163+0000 7f0fb740bf40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613352fdc70 0x5613352f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.163+0000 7f0fb740bf40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613352592f0 con 0x5613352fdc70 2024-04-25T10:10:09.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.163+0000 7f0fb740bf40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613351cddd0 con 0x561335226220 2024-04-25T10:10:09.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.163+0000 7f0fb740bf40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561335226960 con 0x561335271080 2024-04-25T10:10:09.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.163+0000 7f0fb4400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613352fdc70 0x5613352f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.163+0000 7f0fb4400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613352fdc70 0x5613352f8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39376/0 (socket says 172.21.15.119:39376) 2024-04-25T10:10:09.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.163+0000 7f0fb4400640 1 -- 172.21.15.119:0/1051922938 learned_addr learned my addr 172.21.15.119:0/1051922938 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.163+0000 7f0fb3000640 1 --2- 172.21.15.119:0/1051922938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561335271080 0x5613353009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.164+0000 7f0fb3a00640 1 --2- 172.21.15.119:0/1051922938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561335226220 0x5613352f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.164+0000 7f0fb4400640 1 -- 172.21.15.119:0/1051922938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561335271080 msgr2=0x5613353009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.164+0000 7f0fb4400640 1 --2- 172.21.15.119:0/1051922938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561335271080 0x5613353009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.164+0000 7f0fb4400640 1 -- 172.21.15.119:0/1051922938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561335226220 msgr2=0x5613352f05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.164+0000 7f0fb4400640 1 --2- 172.21.15.119:0/1051922938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561335226220 0x5613352f05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.164+0000 7f0fb4400640 1 -- 172.21.15.119:0/1051922938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613352fe290 con 0x5613352fdc70 2024-04-25T10:10:09.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.164+0000 7f0fb4400640 1 --2- 172.21.15.119:0/1051922938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613352fdc70 0x5613352f8a30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0058c0 tx=0x7f0fac050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba603153194b8fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.164+0000 7f0fb1c00640 1 -- 172.21.15.119:0/1051922938 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac055030 con 0x5613352fdc70 2024-04-25T10:10:09.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.164+0000 7f0fb1c00640 1 -- 172.21.15.119:0/1051922938 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac051a60 con 0x5613352fdc70 2024-04-25T10:10:09.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.164+0000 7f0fb1c00640 1 -- 172.21.15.119:0/1051922938 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac051e10 con 0x5613352fdc70 2024-04-25T10:10:09.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.165+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1051922938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613352fdc70 msgr2=0x5613352f8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.165+0000 7f0fb740bf40 1 --2- 172.21.15.119:0/1051922938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613352fdc70 0x5613352f8a30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0058c0 tx=0x7f0fac050000 comp rx=0 tx=0).stop 2024-04-25T10:10:09.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.165+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1051922938 shutdown_connections 2024-04-25T10:10:09.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.165+0000 7f0fb740bf40 1 --2- 172.21.15.119:0/1051922938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5613352fdc70 0x5613352f8a30 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.165+0000 7f0fb740bf40 1 --2- 172.21.15.119:0/1051922938 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561335271080 0x5613353009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.165+0000 7f0fb740bf40 1 --2- 172.21.15.119:0/1051922938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561335226220 0x5613352f05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.165+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1051922938 >> 172.21.15.119:0/1051922938 conn(0x561335274070 msgr2=0x561335274450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.165+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1051922938 shutdown_connections 2024-04-25T10:10:09.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.165+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1051922938 wait complete. 2024-04-25T10:10:09.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.166+0000 7f0fb740bf40 1 Processor -- start 2024-04-25T10:10:09.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.166+0000 7f0fb740bf40 1 -- start start 2024-04-25T10:10:09.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.167+0000 7f0fb740bf40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561335226220 0x5613352ec060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.167+0000 7f0fb740bf40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561335271080 0x5613352739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.167+0000 7f0fb740bf40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5613352fdc70 0x5613352eac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.167+0000 7f0fb740bf40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613352f0fe0 con 0x561335226220 2024-04-25T10:10:09.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.167+0000 7f0fb740bf40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613352f0d00 con 0x561335271080 2024-04-25T10:10:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.167+0000 7f0fb740bf40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613352f12c0 con 0x5613352fdc70 2024-04-25T10:10:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.167+0000 7f0fb3a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561335226220 0x5613352ec060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.167+0000 7f0fb3a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561335226220 0x5613352ec060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39382/0 (socket says 172.21.15.119:39382) 2024-04-25T10:10:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.167+0000 7f0fb3a00640 1 -- 172.21.15.119:0/1581043268 learned_addr learned my addr 172.21.15.119:0/1581043268 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.168+0000 7f0fb4400640 1 --2- 172.21.15.119:0/1581043268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5613352fdc70 0x5613352eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.168+0000 7f0fb3a00640 1 -- 172.21.15.119:0/1581043268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5613352fdc70 msgr2=0x5613352eac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.168+0000 7f0fb3a00640 1 --2- 172.21.15.119:0/1581043268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5613352fdc70 0x5613352eac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.168+0000 7f0fb3a00640 1 -- 172.21.15.119:0/1581043268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561335271080 msgr2=0x5613352739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.168+0000 7f0fb3a00640 1 --2- 172.21.15.119:0/1581043268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561335271080 0x5613352739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.168+0000 7f0fb3a00640 1 -- 172.21.15.119:0/1581043268 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613352ec760 con 0x561335226220 2024-04-25T10:10:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.168+0000 7f0fb3a00640 1 --2- 172.21.15.119:0/1581043268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561335226220 0x5613352ec060 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009df0 tx=0x7f0fa4009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.168+0000 7f0fabe00640 1 -- 172.21.15.119:0/1581043268 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40143f0 con 0x561335226220 2024-04-25T10:10:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.168+0000 7f0fabe00640 1 -- 172.21.15.119:0/1581043268 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa4019070 con 0x561335226220 2024-04-25T10:10:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.168+0000 7f0fabe00640 1 -- 172.21.15.119:0/1581043268 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40147d0 con 0x561335226220 2024-04-25T10:10:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.168+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1581043268 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5613352eca40 con 0x561335226220 2024-04-25T10:10:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.168+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1581043268 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613352ed000 con 0x561335226220 2024-04-25T10:10:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.170+0000 7f0fabe00640 1 -- 172.21.15.119:0/1581043268 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fa4014970 con 0x561335226220 2024-04-25T10:10:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.171+0000 7f0fabe00640 1 --2- 172.21.15.119:0/1581043268 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f94069eb0 0x7f0f9406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.171+0000 7f0fabe00640 1 -- 172.21.15.119:0/1581043268 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fa4088d00 con 0x561335226220 2024-04-25T10:10:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.171+0000 7f0fb3000640 1 --2- 172.21.15.119:0/1581043268 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f94069eb0 0x7f0f9406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.172+0000 7f0fb740bf40 1 --2- 172.21.15.119:0/1581043268 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5613352fae10 0x5613352fb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.172+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1581043268 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55) v9 -- 0x5613352fb7b0 con 0x5613352fae10 2024-04-25T10:10:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.172+0000 7f0fb3000640 1 --2- 172.21.15.119:0/1581043268 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f94069eb0 0x7f0f9406c370 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x5613352f0690 tx=0x7f0fa00015f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.172+0000 7f0fb4400640 1 --2- 172.21.15.119:0/1581043268 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5613352fae10 0x5613352fb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.173+0000 7f0fb4400640 1 --2- 172.21.15.119:0/1581043268 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5613352fae10 0x5613352fb270 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.174+0000 7f0fb4400640 1 -- 172.21.15.119:0/1581043268 <== osd.5 v2:172.21.15.132:6815/2978152964 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f0fac065480 con 0x5613352fae10 2024-04-25T10:10:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.174+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1581043268 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5613352fae10 msgr2=0x5613352fb270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.174+0000 7f0fb740bf40 1 --2- 172.21.15.119:0/1581043268 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5613352fae10 0x5613352fb270 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.174+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1581043268 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f94069eb0 msgr2=0x7f0f9406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.174+0000 7f0fb740bf40 1 --2- 172.21.15.119:0/1581043268 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f94069eb0 0x7f0f9406c370 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x5613352f0690 tx=0x7f0fa00015f0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.174+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1581043268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561335226220 msgr2=0x5613352ec060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.174+0000 7f0fb740bf40 1 --2- 172.21.15.119:0/1581043268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561335226220 0x5613352ec060 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009df0 tx=0x7f0fa4009ee0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.174+0000 7f0fb4400640 1 -- 172.21.15.119:0/1581043268 reap_dead start 2024-04-25T10:10:09.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.174+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1581043268 shutdown_connections 2024-04-25T10:10:09.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.174+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1581043268 >> 172.21.15.119:0/1581043268 conn(0x561335274070 msgr2=0x561335274450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.174+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1581043268 shutdown_connections 2024-04-25T10:10:09.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.174+0000 7f0fb740bf40 1 -- 172.21.15.119:0/1581043268 wait complete. 2024-04-25T10:10:09.185 INFO:teuthology.orchestra.run.smithi119.stdout:Xfiledir_test 2024-04-25T10:10:09.186 DEBUG:teuthology.orchestra.run.smithi119:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:10:09.279 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:10:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 1099511627777, 2024-04-25T10:10:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "ancestors": [ 2024-04-25T10:10:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:10:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "dirino": 1099511627776, 2024-04-25T10:10:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "dname": "file", 2024-04-25T10:10:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: "version": 2 2024-04-25T10:10:09.279 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:10:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:10:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "dirino": 1, 2024-04-25T10:10:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "dname": "dir_test", 2024-04-25T10:10:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "version": 4 2024-04-25T10:10:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:10:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-04-25T10:10:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "pool": 9, 2024-04-25T10:10:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: "old_pools": [] 2024-04-25T10:10:09.280 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:10:09.280 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:10:09.285 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-25T10:10:09.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.385+0000 7fc1e7b1cf40 1 Processor -- start 2024-04-25T10:10:09.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.385+0000 7fc1e7b1cf40 1 -- start start 2024-04-25T10:10:09.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.386+0000 7fc1e7b1cf40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c8002850 0x556dba96e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.386+0000 7fc1e7b1cf40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556dba96e900 0x556dba8f0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.386+0000 7fc1e7b1cf40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dba8f1450 0x556dba97c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.386+0000 7fc1e7b1cf40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dba8d72f0 con 0x556dba8f1450 2024-04-25T10:10:09.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.386+0000 7fc1e7b1cf40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dba84bdd0 con 0x556dba96e900 2024-04-25T10:10:09.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.386+0000 7fc1e7b1cf40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556dba8a4960 con 0x7fc1c8002850 2024-04-25T10:10:09.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.386+0000 7fc1e4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dba8f1450 0x556dba97c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.386+0000 7fc1e4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dba8f1450 0x556dba97c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39410/0 (socket says 172.21.15.119:39410) 2024-04-25T10:10:09.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.386+0000 7fc1e4c00640 1 -- 172.21.15.119:0/3776072625 learned_addr learned my addr 172.21.15.119:0/3776072625 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.387+0000 7fc1e4200640 1 --2- 172.21.15.119:0/3776072625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c8002850 0x556dba96e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.387+0000 7fc1e4c00640 1 -- 172.21.15.119:0/3776072625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c8002850 msgr2=0x556dba96e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.387+0000 7fc1e4c00640 1 --2- 172.21.15.119:0/3776072625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c8002850 0x556dba96e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.387+0000 7fc1e4c00640 1 -- 172.21.15.119:0/3776072625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556dba96e900 msgr2=0x556dba8f0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.387+0000 7fc1e4c00640 1 --2- 172.21.15.119:0/3776072625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556dba96e900 0x556dba8f0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.387+0000 7fc1e4c00640 1 -- 172.21.15.119:0/3776072625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dba8f1be0 con 0x556dba8f1450 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.387+0000 7fc1e4c00640 1 --2- 172.21.15.119:0/3776072625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dba8f1450 0x556dba97c5a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x556dba8ab790 tx=0x7fc1dc002a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993e7ed3c6f98e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.387+0000 7fc1e2400640 1 -- 172.21.15.119:0/3776072625 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc016020 con 0x556dba8f1450 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.387+0000 7fc1e2400640 1 -- 172.21.15.119:0/3776072625 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1dc012680 con 0x556dba8f1450 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.387+0000 7fc1e2400640 1 -- 172.21.15.119:0/3776072625 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc012a50 con 0x556dba8f1450 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.388+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/3776072625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dba8f1450 msgr2=0x556dba97c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.388+0000 7fc1e7b1cf40 1 --2- 172.21.15.119:0/3776072625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dba8f1450 0x556dba97c5a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x556dba8ab790 tx=0x7fc1dc002a60 comp rx=0 tx=0).stop 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.388+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/3776072625 shutdown_connections 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.388+0000 7fc1e7b1cf40 1 --2- 172.21.15.119:0/3776072625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dba8f1450 0x556dba97c5a0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.388+0000 7fc1e7b1cf40 1 --2- 172.21.15.119:0/3776072625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556dba96e900 0x556dba8f0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.388+0000 7fc1e7b1cf40 1 --2- 172.21.15.119:0/3776072625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c8002850 0x556dba96e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.388+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/3776072625 >> 172.21.15.119:0/3776072625 conn(0x556dba8f2070 msgr2=0x556dba8f2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.388+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/3776072625 shutdown_connections 2024-04-25T10:10:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.388+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/3776072625 wait complete. 2024-04-25T10:10:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.389+0000 7fc1e7b1cf40 1 Processor -- start 2024-04-25T10:10:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.389+0000 7fc1e7b1cf40 1 -- start start 2024-04-25T10:10:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.390+0000 7fc1e7b1cf40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556dba8f1450 0x556dba96aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.390+0000 7fc1e7b1cf40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dba96e900 0x556dba96b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.390+0000 7fc1e7b1cf40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c8002850 0x556dba96bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.390+0000 7fc1e7b1cf40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dba977f30 con 0x556dba96e900 2024-04-25T10:10:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.390+0000 7fc1e7b1cf40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dba96f200 con 0x7fc1c8002850 2024-04-25T10:10:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.390+0000 7fc1e7b1cf40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556dba973d30 con 0x556dba8f1450 2024-04-25T10:10:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.390+0000 7fc1e3800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dba96e900 0x556dba96b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.390+0000 7fc1e3800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dba96e900 0x556dba96b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39414/0 (socket says 172.21.15.119:39414) 2024-04-25T10:10:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.390+0000 7fc1e3800640 1 -- 172.21.15.119:0/4208203326 learned_addr learned my addr 172.21.15.119:0/4208203326 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.391+0000 7fc1e4200640 1 --2- 172.21.15.119:0/4208203326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556dba8f1450 0x556dba96aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.391+0000 7fc1e4c00640 1 --2- 172.21.15.119:0/4208203326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c8002850 0x556dba96bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.391+0000 7fc1e3800640 1 -- 172.21.15.119:0/4208203326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556dba8f1450 msgr2=0x556dba96aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.391+0000 7fc1e3800640 1 --2- 172.21.15.119:0/4208203326 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556dba8f1450 0x556dba96aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.391+0000 7fc1e3800640 1 -- 172.21.15.119:0/4208203326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c8002850 msgr2=0x556dba96bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.391+0000 7fc1e3800640 1 --2- 172.21.15.119:0/4208203326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c8002850 0x556dba96bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.391+0000 7fc1e3800640 1 -- 172.21.15.119:0/4208203326 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dba974010 con 0x556dba96e900 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.391+0000 7fc1e3800640 1 --2- 172.21.15.119:0/4208203326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dba96e900 0x556dba96b880 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc026440 tx=0x7fc1cc01c2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.391+0000 7fc1e1000640 1 -- 172.21.15.119:0/4208203326 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc009720 con 0x556dba96e900 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.391+0000 7fc1e1000640 1 -- 172.21.15.119:0/4208203326 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1cc000ed0 con 0x556dba96e900 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.391+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/4208203326 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556dba9742f0 con 0x556dba96e900 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.391+0000 7fc1e1000640 1 -- 172.21.15.119:0/4208203326 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc065470 con 0x556dba96e900 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.392+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/4208203326 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556dba9748b0 con 0x556dba96e900 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.393+0000 7fc1e1000640 1 -- 172.21.15.119:0/4208203326 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1cc053030 con 0x556dba96e900 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.394+0000 7fc1e1000640 1 --2- 172.21.15.119:0/4208203326 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc1c0069eb0 0x7fc1c006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.394+0000 7fc1e1000640 1 -- 172.21.15.119:0/4208203326 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1cc0ccfe0 con 0x556dba96e900 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.395+0000 7fc1e4200640 1 --2- 172.21.15.119:0/4208203326 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc1c0069eb0 0x7fc1c006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.395+0000 7fc1e7b1cf40 1 --2- 172.21.15.119:0/4208203326 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x556dba8f0c20 0x556dbaa1f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.395+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/4208203326 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55) v9 -- 0x556dba8a3640 con 0x556dba8f0c20 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.395+0000 7fc1e4200640 1 --2- 172.21.15.119:0/4208203326 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc1c0069eb0 0x7fc1c006c370 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4005730 tx=0x7fc1d40056c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.396+0000 7fc1e4c00640 1 --2- 172.21.15.119:0/4208203326 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x556dba8f0c20 0x556dbaa1f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.396+0000 7fc1e4c00640 1 --2- 172.21.15.119:0/4208203326 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x556dba8f0c20 0x556dbaa1f3b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.400+0000 7fc1e4c00640 1 -- 172.21.15.119:0/4208203326 <== osd.5 v2:172.21.15.132:6815/2978152964 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v55'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fc1dc012c50 con 0x556dba8f0c20 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.400+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/4208203326 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x556dba8f0c20 msgr2=0x556dbaa1f3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.400+0000 7fc1e7b1cf40 1 --2- 172.21.15.119:0/4208203326 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x556dba8f0c20 0x556dbaa1f3b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.400+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/4208203326 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc1c0069eb0 msgr2=0x7fc1c006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.400+0000 7fc1e7b1cf40 1 --2- 172.21.15.119:0/4208203326 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc1c0069eb0 0x7fc1c006c370 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4005730 tx=0x7fc1d40056c0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.400+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/4208203326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dba96e900 msgr2=0x556dba96b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.400+0000 7fc1e7b1cf40 1 --2- 172.21.15.119:0/4208203326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556dba96e900 0x556dba96b880 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc026440 tx=0x7fc1cc01c2c0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.400+0000 7fc1e4c00640 1 -- 172.21.15.119:0/4208203326 reap_dead start 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.400+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/4208203326 shutdown_connections 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.400+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/4208203326 >> 172.21.15.119:0/4208203326 conn(0x556dba8f2070 msgr2=0x556dba8f2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.401+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/4208203326 shutdown_connections 2024-04-25T10:10:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.401+0000 7fc1e7b1cf40 1 -- 172.21.15.119:0/4208203326 wait complete. 2024-04-25T10:10:09.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:09.441 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-25T10:10:09.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.641+0000 7fd343a00640 1 Processor -- start 2024-04-25T10:10:09.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.641+0000 7fd343a00640 1 -- start start 2024-04-25T10:10:09.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.642+0000 7fd343a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd33c145260 0x7fd33c0cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.642+0000 7fd343a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd33c071910 0x7fd33c0cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.642+0000 7fd343a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd33c0cc9a0 con 0x7fd33c0cc490 2024-04-25T10:10:09.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.642+0000 7fd343a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd33c0cea00 con 0x7fd33c145260 2024-04-25T10:10:09.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.642+0000 7fd343a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c0ccc80 con 0x7fd33c071910 2024-04-25T10:10:09.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.643+0000 7fd341a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd33c071910 0x7fd33c0cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.643+0000 7fd341a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd33c071910 0x7fd33c0cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:52660/0 (socket says 172.21.15.119:52660) 2024-04-25T10:10:09.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.643+0000 7fd341a00640 1 -- 172.21.15.119:0/978666340 learned_addr learned my addr 172.21.15.119:0/978666340 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.644+0000 7fd341000640 1 --2- 172.21.15.119:0/978666340 >> v2:172.21.15.132:3300/0 conn(0x7fd33c145260 0x7fd33c0cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.644+0000 7fd341a00640 1 -- 172.21.15.119:0/978666340 >> v2:172.21.15.132:3300/0 conn(0x7fd33c145260 msgr2=0x7fd33c0cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.644+0000 7fd341a00640 1 --2- 172.21.15.119:0/978666340 >> v2:172.21.15.132:3300/0 conn(0x7fd33c145260 0x7fd33c0cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.644+0000 7fd341a00640 1 -- 172.21.15.119:0/978666340 >> v1:172.21.15.132:6789/0 conn(0x7fd33c0cc490 legacy=0x7fd33c071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.644+0000 7fd341a00640 1 -- 172.21.15.119:0/978666340 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c0ccf60 con 0x7fd33c071910 2024-04-25T10:10:09.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.645+0000 7fd341a00640 1 --2- 172.21.15.119:0/978666340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd33c071910 0x7fd33c0cee90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd334009e50 tx=0x7fd33402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3de2336a23b31f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.646+0000 7fd33b400640 1 -- 172.21.15.119:0/978666340 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334036070 con 0x7fd33c071910 2024-04-25T10:10:09.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.646+0000 7fd33b400640 1 -- 172.21.15.119:0/978666340 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd334031600 con 0x7fd33c071910 2024-04-25T10:10:09.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.646+0000 7fd33b400640 1 -- 172.21.15.119:0/978666340 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3340319d0 con 0x7fd33c071910 2024-04-25T10:10:09.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.647+0000 7fd343a00640 1 -- 172.21.15.119:0/978666340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd33c071910 msgr2=0x7fd33c0cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.647+0000 7fd343a00640 1 --2- 172.21.15.119:0/978666340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd33c071910 0x7fd33c0cee90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd334009e50 tx=0x7fd33402a710 comp rx=0 tx=0).stop 2024-04-25T10:10:09.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.647+0000 7fd343a00640 1 -- 172.21.15.119:0/978666340 shutdown_connections 2024-04-25T10:10:09.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.647+0000 7fd343a00640 1 --2- 172.21.15.119:0/978666340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd33c071910 0x7fd33c0cee90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.647+0000 7fd343a00640 1 --2- 172.21.15.119:0/978666340 >> v2:172.21.15.132:3300/0 conn(0x7fd33c145260 0x7fd33c0cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.647+0000 7fd343a00640 1 -- 172.21.15.119:0/978666340 >> 172.21.15.119:0/978666340 conn(0x7fd33c06aba0 msgr2=0x7fd33c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.647+0000 7fd343a00640 1 -- 172.21.15.119:0/978666340 shutdown_connections 2024-04-25T10:10:09.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.647+0000 7fd343a00640 1 -- 172.21.15.119:0/978666340 wait complete. 2024-04-25T10:10:09.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.649+0000 7fd343a00640 1 Processor -- start 2024-04-25T10:10:09.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.649+0000 7fd343a00640 1 -- start start 2024-04-25T10:10:09.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.650+0000 7fd343a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c071910 0x7fd33c143590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.650+0000 7fd343a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd33c0cc490 0x7fd33c13fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.650+0000 7fd343a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd33c145260 0x7fd33c140170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.650+0000 7fd343a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c15c270 con 0x7fd33c071910 2024-04-25T10:10:09.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.650+0000 7fd343a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c145640 con 0x7fd33c145260 2024-04-25T10:10:09.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.650+0000 7fd343a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c15c3f0 con 0x7fd33c0cc490 2024-04-25T10:10:09.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.650+0000 7fd341000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c071910 0x7fd33c143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.650+0000 7fd341000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c071910 0x7fd33c143590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39420/0 (socket says 172.21.15.119:39420) 2024-04-25T10:10:09.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.650+0000 7fd341000640 1 -- 172.21.15.119:0/1528529495 learned_addr learned my addr 172.21.15.119:0/1528529495 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.650+0000 7fd33be00640 1 --2- 172.21.15.119:0/1528529495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd33c0cc490 0x7fd33c13fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.650+0000 7fd341000640 1 -- 172.21.15.119:0/1528529495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd33c0cc490 msgr2=0x7fd33c13fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.650+0000 7fd341000640 1 --2- 172.21.15.119:0/1528529495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd33c0cc490 0x7fd33c13fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.651+0000 7fd341000640 1 -- 172.21.15.119:0/1528529495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd33c145260 msgr2=0x7fd33c140170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.651+0000 7fd341000640 1 --2- 172.21.15.119:0/1528529495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd33c145260 0x7fd33c140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.651+0000 7fd341000640 1 -- 172.21.15.119:0/1528529495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c143de0 con 0x7fd33c071910 2024-04-25T10:10:09.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.651+0000 7fd341000640 1 --2- 172.21.15.119:0/1528529495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c071910 0x7fd33c143590 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd33000d620 tx=0x7fd33000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.651+0000 7fd338c00640 1 -- 172.21.15.119:0/1528529495 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33000b620 con 0x7fd33c071910 2024-04-25T10:10:09.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.651+0000 7fd338c00640 1 -- 172.21.15.119:0/1528529495 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd330019070 con 0x7fd33c071910 2024-04-25T10:10:09.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.651+0000 7fd343a00640 1 -- 172.21.15.119:0/1528529495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33c1440c0 con 0x7fd33c071910 2024-04-25T10:10:09.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.651+0000 7fd338c00640 1 -- 172.21.15.119:0/1528529495 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33000ba00 con 0x7fd33c071910 2024-04-25T10:10:09.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.651+0000 7fd343a00640 1 -- 172.21.15.119:0/1528529495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33c143ad0 con 0x7fd33c071910 2024-04-25T10:10:09.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.654+0000 7fd338c00640 1 -- 172.21.15.119:0/1528529495 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd33000bba0 con 0x7fd33c071910 2024-04-25T10:10:09.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.655+0000 7fd338c00640 1 --2- 172.21.15.119:0/1528529495 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd31c06b4d0 0x7fd31c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.655+0000 7fd338c00640 1 -- 172.21.15.119:0/1528529495 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd33008a5e0 con 0x7fd33c071910 2024-04-25T10:10:09.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.655+0000 7fd33be00640 1 --2- 172.21.15.119:0/1528529495 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd31c06b4d0 0x7fd31c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.656+0000 7fd34699c740 1 Processor -- start 2024-04-25T10:10:09.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.656+0000 7fd33be00640 1 --2- 172.21.15.119:0/1528529495 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd31c06b4d0 0x7fd31c06d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0aece0 tx=0x7fd328009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.656+0000 7fd34699c740 1 -- start start 2024-04-25T10:10:09.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.657+0000 7fd34699c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b77e80 0x55c202b2a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.657+0000 7fd34699c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b5afa0 0x55c202b5a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.657+0000 7fd34699c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5a860 0x55c202b2f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.657+0000 7fd34699c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c202a8f920 con 0x55c202b5a860 2024-04-25T10:10:09.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.657+0000 7fd34699c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c202a38da0 con 0x55c202b77e80 2024-04-25T10:10:09.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.657+0000 7fd34699c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c202a8b9c0 con 0x55c202b5afa0 2024-04-25T10:10:09.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.657+0000 7fd341a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5a860 0x55c202b2f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.657+0000 7fd341a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5a860 0x55c202b2f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39424/0 (socket says 172.21.15.119:39424) 2024-04-25T10:10:09.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.657+0000 7fd341a00640 1 -- 172.21.15.119:0/2481041461 learned_addr learned my addr 172.21.15.119:0/2481041461 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.658+0000 7fd33be00640 1 --2- 172.21.15.119:0/2481041461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b5afa0 0x55c202b5a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.658+0000 7fd341a00640 1 -- 172.21.15.119:0/2481041461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b5afa0 msgr2=0x55c202b5a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.658+0000 7fd341a00640 1 --2- 172.21.15.119:0/2481041461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b5afa0 0x55c202b5a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.658+0000 7fd341a00640 1 -- 172.21.15.119:0/2481041461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b77e80 msgr2=0x55c202b2a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.658+0000 7fd341a00640 1 --2- 172.21.15.119:0/2481041461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b77e80 0x55c202b2a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.658+0000 7fd341a00640 1 -- 172.21.15.119:0/2481041461 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c202a45290 con 0x55c202b5a860 2024-04-25T10:10:09.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.658+0000 7fd341a00640 1 --2- 172.21.15.119:0/2481041461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5a860 0x55c202b2f670 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd334009f40 tx=0x7fd33402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490cbd07a26acc32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.658+0000 7fd32d600640 1 -- 172.21.15.119:0/2481041461 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334036070 con 0x55c202b5a860 2024-04-25T10:10:09.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.658+0000 7fd32d600640 1 -- 172.21.15.119:0/2481041461 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd334040070 con 0x55c202b5a860 2024-04-25T10:10:09.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.658+0000 7fd32d600640 1 -- 172.21.15.119:0/2481041461 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33403b840 con 0x55c202b5a860 2024-04-25T10:10:09.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.659+0000 7fd34699c740 1 -- 172.21.15.119:0/2481041461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5a860 msgr2=0x55c202b2f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.659+0000 7fd34699c740 1 --2- 172.21.15.119:0/2481041461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5a860 0x55c202b2f670 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd334009f40 tx=0x7fd33402c000 comp rx=0 tx=0).stop 2024-04-25T10:10:09.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.659+0000 7fd34699c740 1 -- 172.21.15.119:0/2481041461 shutdown_connections 2024-04-25T10:10:09.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.659+0000 7fd34699c740 1 --2- 172.21.15.119:0/2481041461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5a860 0x55c202b2f670 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.659+0000 7fd34699c740 1 --2- 172.21.15.119:0/2481041461 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b5afa0 0x55c202b5a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.659+0000 7fd34699c740 1 --2- 172.21.15.119:0/2481041461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b77e80 0x55c202b2a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.659+0000 7fd34699c740 1 -- 172.21.15.119:0/2481041461 >> 172.21.15.119:0/2481041461 conn(0x55c202b2ca80 msgr2=0x55c202b75320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.659+0000 7fd34699c740 1 -- 172.21.15.119:0/2481041461 shutdown_connections 2024-04-25T10:10:09.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.659+0000 7fd34699c740 1 -- 172.21.15.119:0/2481041461 wait complete. 2024-04-25T10:10:09.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.660+0000 7fd34699c740 1 -- start start 2024-04-25T10:10:09.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.660+0000 7fd34699c740 1 Processor -- start 2024-04-25T10:10:09.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.661+0000 7fd34699c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5afa0 0x55c202b5dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.661+0000 7fd34699c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 0x55c202b851b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.661+0000 7fd34699c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b856f0 0x55c202b85b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.661+0000 7fd34699c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c202b2ec30 con 0x55c202b5afa0 2024-04-25T10:10:09.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.661+0000 7fd34699c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c202a2e260 con 0x55c202b856f0 2024-04-25T10:10:09.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.661+0000 7fd34699c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c202b2e020 con 0x55c202b77e80 2024-04-25T10:10:09.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.661+0000 7fd34699c740 10 client.0 _ll_register_callbacks cb 0x55c202a4bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:09.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.661+0000 7fd341000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5afa0 0x55c202b5dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.661+0000 7fd341000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5afa0 0x55c202b5dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39430/0 (socket says 172.21.15.119:39430) 2024-04-25T10:10:09.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.661+0000 7fd341000640 1 -- 172.21.15.119:0/437741132 learned_addr learned my addr 172.21.15.119:0/437741132 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.662+0000 7fd341000640 1 -- 172.21.15.119:0/437741132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 msgr2=0x55c202b851b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.662+0000 7fd33be00640 1 --2- 172.21.15.119:0/437741132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 0x55c202b851b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.662+0000 7fd341a00640 1 --2- 172.21.15.119:0/437741132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b856f0 0x55c202b85b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.662+0000 7fd341000640 1 --2- 172.21.15.119:0/437741132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 0x55c202b851b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.662+0000 7fd341000640 1 -- 172.21.15.119:0/437741132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b856f0 msgr2=0x55c202b85b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.662+0000 7fd341000640 1 --2- 172.21.15.119:0/437741132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b856f0 0x55c202b85b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.662+0000 7fd341000640 1 -- 172.21.15.119:0/437741132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2028707b0 con 0x55c202b5afa0 2024-04-25T10:10:09.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.662+0000 7fd341000640 1 --2- 172.21.15.119:0/437741132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5afa0 0x55c202b5dba0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd330006de0 tx=0x7fd330059590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60131d228d70be44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.662+0000 7fd322000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:10:09.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.663+0000 7fd322000640 1 -- 172.21.15.119:0/437741132 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3300553c0 con 0x55c202b5afa0 2024-04-25T10:10:09.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.663+0000 7fd322000640 1 -- 172.21.15.119:0/437741132 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd330060fd0 con 0x55c202b5afa0 2024-04-25T10:10:09.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.663+0000 7fd34699c740 1 -- 172.21.15.119:0/437741132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd334040070 con 0x55c202b5afa0 2024-04-25T10:10:09.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.663+0000 7fd322000640 1 -- 172.21.15.119:0/437741132 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd330055560 con 0x55c202b5afa0 2024-04-25T10:10:09.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.663+0000 7fd34699c740 10 client.6120 fetch_fsmap learned FSMap version 34 2024-04-25T10:10:09.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.663+0000 7fd34699c740 1 -- 172.21.15.119:0/437741132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55c2028d0f40 con 0x55c202b5afa0 2024-04-25T10:10:09.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.663+0000 7fd322000640 1 -- 172.21.15.119:0/437741132 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd334040070 con 0x55c202b5afa0 2024-04-25T10:10:09.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.663+0000 7fd322000640 1 -- 172.21.15.119:0/437741132 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd330013070 con 0x55c202b5afa0 2024-04-25T10:10:09.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.663+0000 7fd34699c740 10 client.6120 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T10:10:09.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.663+0000 7fd34699c740 10 client.6120 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:10:09.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.663+0000 7fd34699c740 20 client.6120 populate_metadata read hostname 'smithi119' 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.663+0000 7fd34699c740 1 --2- 172.21.15.119:0/437741132 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55c202b5bde0 0x55c202b5c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.663+0000 7fd34699c740 4 client.6120 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.663+0000 7fd34699c740 1 -- 172.21.15.119:0/437741132 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c202a9ce60 con 0x55c202b5bde0 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.664+0000 7fd33be00640 1 --2- 172.21.15.119:0/437741132 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55c202b5bde0 0x55c202b5c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.664+0000 7fd33be00640 1 --2- 172.21.15.119:0/437741132 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55c202b5bde0 0x55c202b5c250 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14a6ee92e4568654 server_cookie=1de5e8b8ca8c50cc in_seq=0 out_seq=0 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.664+0000 7fd322000640 10 client.6120 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.665+0000 7fd322000640 1 -- 172.21.15.119:0/437741132 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c202a9ce60 con 0x55c202b5bde0 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.665+0000 7fd322000640 10 client.6120 handle_command_reply: tid=0 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.665+0000 7fd34699c740 1 client.6120 shutdown 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.665+0000 7fd34699c740 1 -- 172.21.15.119:0/437741132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5afa0 msgr2=0x55c202b5dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.665+0000 7fd34699c740 1 --2- 172.21.15.119:0/437741132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5afa0 0x55c202b5dba0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd330006de0 tx=0x7fd330059590 comp rx=0 tx=0).stop 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.665+0000 7fd34699c740 1 -- 172.21.15.119:0/437741132 shutdown_connections 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.665+0000 7fd34699c740 1 --2- 172.21.15.119:0/437741132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b856f0 0x55c202b85b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.665+0000 7fd34699c740 1 --2- 172.21.15.119:0/437741132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 0x55c202b851b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.665+0000 7fd34699c740 1 --2- 172.21.15.119:0/437741132 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55c202b5bde0 0x55c202b5c250 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.665+0000 7fd34699c740 1 --2- 172.21.15.119:0/437741132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5afa0 0x55c202b5dba0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.665+0000 7fd322000640 0 client.6120 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:09.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.665+0000 7fd34699c740 1 -- 172.21.15.119:0/437741132 >> 172.21.15.119:0/437741132 conn(0x55c202b5a860 msgr2=0x55c202b76450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.665+0000 7fd34699c740 1 -- 172.21.15.119:0/437741132 shutdown_connections 2024-04-25T10:10:09.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.665+0000 7fd34699c740 1 -- 172.21.15.119:0/437741132 wait complete. 2024-04-25T10:10:09.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.666+0000 7fd34699c740 20 client.6120 trim_cache size 0 max 16384 2024-04-25T10:10:09.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.682+0000 7fd34699c740 1 Processor -- start 2024-04-25T10:10:09.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.682+0000 7fd34699c740 1 -- start start 2024-04-25T10:10:09.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd34699c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b5afa0 0x55c202b2e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd34699c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 0x55c202b8dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd34699c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b856f0 0x55c202b551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd34699c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c0b4820 con 0x55c202b856f0 2024-04-25T10:10:09.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd34699c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c0b3a00 con 0x55c202b5afa0 2024-04-25T10:10:09.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd34699c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c13f890 con 0x55c202b77e80 2024-04-25T10:10:09.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd341a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b856f0 0x55c202b551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd341a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b856f0 0x55c202b551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39432/0 (socket says 172.21.15.119:39432) 2024-04-25T10:10:09.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd341a00640 1 -- 172.21.15.119:0/2850297604 learned_addr learned my addr 172.21.15.119:0/2850297604 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd33be00640 1 --2- 172.21.15.119:0/2850297604 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 0x55c202b8dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd341000640 1 --2- 172.21.15.119:0/2850297604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b5afa0 0x55c202b2e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd341a00640 1 -- 172.21.15.119:0/2850297604 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 msgr2=0x55c202b8dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd341a00640 1 --2- 172.21.15.119:0/2850297604 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 0x55c202b8dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd341a00640 1 -- 172.21.15.119:0/2850297604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b5afa0 msgr2=0x55c202b2e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd341a00640 1 --2- 172.21.15.119:0/2850297604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b5afa0 0x55c202b2e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd341a00640 1 -- 172.21.15.119:0/2850297604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c202b31d00 con 0x55c202b856f0 2024-04-25T10:10:09.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.683+0000 7fd33be00640 1 --2- 172.21.15.119:0/2850297604 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 0x55c202b8dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:09.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.684+0000 7fd341a00640 1 --2- 172.21.15.119:0/2850297604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b856f0 0x55c202b551f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd334009e50 tx=0x7fd334009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7325793775de2ee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.684+0000 7fd32cc00640 1 -- 172.21.15.119:0/2850297604 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334003c50 con 0x55c202b856f0 2024-04-25T10:10:09.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.684+0000 7fd32cc00640 1 -- 172.21.15.119:0/2850297604 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd334040040 con 0x55c202b856f0 2024-04-25T10:10:09.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.684+0000 7fd32cc00640 1 -- 172.21.15.119:0/2850297604 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334004040 con 0x55c202b856f0 2024-04-25T10:10:09.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.685+0000 7fd34699c740 1 -- 172.21.15.119:0/2850297604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b856f0 msgr2=0x55c202b551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.685+0000 7fd34699c740 1 --2- 172.21.15.119:0/2850297604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b856f0 0x55c202b551f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd334009e50 tx=0x7fd334009000 comp rx=0 tx=0).stop 2024-04-25T10:10:09.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.685+0000 7fd34699c740 1 -- 172.21.15.119:0/2850297604 shutdown_connections 2024-04-25T10:10:09.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.685+0000 7fd34699c740 1 --2- 172.21.15.119:0/2850297604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b856f0 0x55c202b551f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.685+0000 7fd34699c740 1 --2- 172.21.15.119:0/2850297604 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 0x55c202b8dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.685+0000 7fd34699c740 1 --2- 172.21.15.119:0/2850297604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b5afa0 0x55c202b2e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.685+0000 7fd34699c740 1 -- 172.21.15.119:0/2850297604 >> 172.21.15.119:0/2850297604 conn(0x55c202b5a860 msgr2=0x55c202b25c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.685+0000 7fd34699c740 1 -- 172.21.15.119:0/2850297604 shutdown_connections 2024-04-25T10:10:09.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.685+0000 7fd34699c740 1 -- 172.21.15.119:0/2850297604 wait complete. 2024-04-25T10:10:09.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.685+0000 7fd34699c740 1 -- start start 2024-04-25T10:10:09.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.686+0000 7fd34699c740 1 Processor -- start 2024-04-25T10:10:09.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd34699c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5afa0 0x55c202bdd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd34699c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 0x55c202be0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd34699c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b856f0 0x55c202b83a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd34699c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c13ae70 con 0x55c202b5afa0 2024-04-25T10:10:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd34699c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c202b5d2f0 con 0x55c202b856f0 2024-04-25T10:10:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd34699c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c202b84450 con 0x55c202b77e80 2024-04-25T10:10:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd34699c740 10 client.0 _ll_register_callbacks cb 0x7fd330006eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd341000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5afa0 0x55c202bdd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd341000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5afa0 0x55c202bdd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39438/0 (socket says 172.21.15.119:39438) 2024-04-25T10:10:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd341000640 1 -- 172.21.15.119:0/1859931319 learned_addr learned my addr 172.21.15.119:0/1859931319 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd341000640 1 -- 172.21.15.119:0/1859931319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 msgr2=0x55c202be0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd341000640 1 --2- 172.21.15.119:0/1859931319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 0x55c202be0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd341000640 1 -- 172.21.15.119:0/1859931319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b856f0 msgr2=0x55c202b83a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd341000640 1 --2- 172.21.15.119:0/1859931319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b856f0 0x55c202b83a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.687+0000 7fd341000640 1 -- 172.21.15.119:0/1859931319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c202b777e0 con 0x55c202b5afa0 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd341000640 1 --2- 172.21.15.119:0/1859931319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5afa0 0x55c202bdd620 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd33000b0e0 tx=0x7fd330020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a79a612673f38e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd322000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd322000640 1 -- 172.21.15.119:0/1859931319 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3300553c0 con 0x55c202b5afa0 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd322000640 1 -- 172.21.15.119:0/1859931319 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd330054cb0 con 0x55c202b5afa0 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd34699c740 1 -- 172.21.15.119:0/1859931319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd334040040 con 0x55c202b5afa0 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd322000640 1 -- 172.21.15.119:0/1859931319 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3300586b0 con 0x55c202b5afa0 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd34699c740 10 client.6126 fetch_fsmap learned FSMap version 34 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd34699c740 1 -- 172.21.15.119:0/1859931319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55c2027640a0 con 0x55c202b5afa0 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd322000640 1 -- 172.21.15.119:0/1859931319 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd334040040 con 0x55c202b5afa0 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd322000640 1 -- 172.21.15.119:0/1859931319 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd330070ae0 con 0x55c202b5afa0 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd34699c740 10 client.6126 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd34699c740 10 client.6126 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd34699c740 20 client.6126 populate_metadata read hostname 'smithi119' 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd34699c740 1 --2- 172.21.15.119:0/1859931319 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55c202b2e540 0x55c202b2e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd34699c740 4 client.6126 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.688+0000 7fd34699c740 1 -- 172.21.15.119:0/1859931319 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x55c202b6d4a0 con 0x55c202b2e540 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.689+0000 7fd33be00640 1 --2- 172.21.15.119:0/1859931319 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55c202b2e540 0x55c202b2e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.689+0000 7fd33be00640 1 --2- 172.21.15.119:0/1859931319 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55c202b2e540 0x55c202b2e950 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d6aa7bdf4e26a7 server_cookie=9370bcd64f3abc49 in_seq=0 out_seq=0 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.689+0000 7fd322000640 10 client.6126 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.690+0000 7fd322000640 1 -- 172.21.15.119:0/1859931319 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c202b6d4a0 con 0x55c202b2e540 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.690+0000 7fd322000640 10 client.6126 handle_command_reply: tid=0 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.690+0000 7fd34699c740 1 client.6126 shutdown 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.690+0000 7fd34699c740 1 -- 172.21.15.119:0/1859931319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5afa0 msgr2=0x55c202bdd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.690+0000 7fd34699c740 1 --2- 172.21.15.119:0/1859931319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5afa0 0x55c202bdd620 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd33000b0e0 tx=0x7fd330020070 comp rx=0 tx=0).stop 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.690+0000 7fd34699c740 1 -- 172.21.15.119:0/1859931319 shutdown_connections 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.690+0000 7fd34699c740 1 --2- 172.21.15.119:0/1859931319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c202b856f0 0x55c202b83a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.690+0000 7fd34699c740 1 --2- 172.21.15.119:0/1859931319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c202b77e80 0x55c202be0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.690+0000 7fd34699c740 1 --2- 172.21.15.119:0/1859931319 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55c202b2e540 0x55c202b2e950 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.690+0000 7fd34699c740 1 --2- 172.21.15.119:0/1859931319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c202b5afa0 0x55c202bdd620 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.690+0000 7fd34699c740 1 -- 172.21.15.119:0/1859931319 >> 172.21.15.119:0/1859931319 conn(0x55c202b5a860 msgr2=0x55c202b307a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.690+0000 7fd322000640 0 client.6126 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.691+0000 7fd34699c740 1 -- 172.21.15.119:0/1859931319 shutdown_connections 2024-04-25T10:10:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.691+0000 7fd34699c740 1 -- 172.21.15.119:0/1859931319 wait complete. 2024-04-25T10:10:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.691+0000 7fd34699c740 20 client.6126 trim_cache size 0 max 16384 2024-04-25T10:10:09.701 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:10:09.691+0000 7efdba600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={4880=0-4194304@1} caps={4880=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=4880 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x561f4681db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:10:09.701 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:10:09.701 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2024-04-25T10:10:09.702 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "60883a24-7f56-4761-8b27-43315998b266", 2024-04-25T10:10:09.702 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2024-04-25T10:10:09.702 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:10:09.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.691+0000 7fd343a00640 1 -- 172.21.15.119:0/1528529495 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd31c06b4d0 msgr2=0x7fd31c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.691+0000 7fd343a00640 1 --2- 172.21.15.119:0/1528529495 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd31c06b4d0 0x7fd31c06d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0aece0 tx=0x7fd328009040 comp rx=0 tx=0).stop 2024-04-25T10:10:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.691+0000 7fd343a00640 1 -- 172.21.15.119:0/1528529495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c071910 msgr2=0x7fd33c143590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.691+0000 7fd343a00640 1 --2- 172.21.15.119:0/1528529495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c071910 0x7fd33c143590 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd33000d620 tx=0x7fd33000dae0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.692+0000 7fd343a00640 1 -- 172.21.15.119:0/1528529495 shutdown_connections 2024-04-25T10:10:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.692+0000 7fd343a00640 1 --2- 172.21.15.119:0/1528529495 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd31c06b4d0 0x7fd31c06d990 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.692+0000 7fd343a00640 1 --2- 172.21.15.119:0/1528529495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd33c145260 0x7fd33c140170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.692+0000 7fd343a00640 1 --2- 172.21.15.119:0/1528529495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd33c0cc490 0x7fd33c13fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.692+0000 7fd343a00640 1 --2- 172.21.15.119:0/1528529495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd33c071910 0x7fd33c143590 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.692+0000 7fd343a00640 1 -- 172.21.15.119:0/1528529495 >> 172.21.15.119:0/1528529495 conn(0x7fd33c06aba0 msgr2=0x7fd33c071200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.692+0000 7fd343a00640 1 -- 172.21.15.119:0/1528529495 shutdown_connections 2024-04-25T10:10:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.692+0000 7fd343a00640 1 -- 172.21.15.119:0/1528529495 wait complete. 2024-04-25T10:10:09.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:09.714 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:10:09.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.899+0000 7f887be00640 1 Processor -- start 2024-04-25T10:10:09.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.900+0000 7f887be00640 1 -- start start 2024-04-25T10:10:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.901+0000 7f887be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f887c159d60 0x7f887c15a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.901+0000 7f887be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f887c0720c0 0x7f887c0ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.901+0000 7f887be00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887c0c4640 con 0x7f887c15e310 2024-04-25T10:10:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.901+0000 7f887be00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c0c4bc0 con 0x7f887c0720c0 2024-04-25T10:10:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.901+0000 7f887be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0c48e0 con 0x7f887c159d60 2024-04-25T10:10:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.901+0000 7f887b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f887c0720c0 0x7f887c0ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.901+0000 7f887b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f887c0720c0 0x7f887c0ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39448/0 (socket says 172.21.15.119:39448) 2024-04-25T10:10:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.901+0000 7f887b400640 1 -- 172.21.15.119:0/3964510121 learned_addr learned my addr 172.21.15.119:0/3964510121 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.901+0000 7f8872000640 1 --1- 172.21.15.119:0/3964510121 >> v1:172.21.15.119:6789/0 conn(0x7f887c15e310 0x7f887c0719b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51506/0 (socket says 172.21.15.119:51506) 2024-04-25T10:10:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.902+0000 7f887aa00640 1 --2- 172.21.15.119:0/3964510121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f887c159d60 0x7f887c15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.902+0000 7f887b400640 1 -- 172.21.15.119:0/3964510121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f887c159d60 msgr2=0x7f887c15a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.902+0000 7f887b400640 1 --2- 172.21.15.119:0/3964510121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f887c159d60 0x7f887c15a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.902+0000 7f887b400640 1 -- 172.21.15.119:0/3964510121 >> v1:172.21.15.119:6789/0 conn(0x7f887c15e310 legacy=0x7f887c0719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.902+0000 7f887b400640 1 -- 172.21.15.119:0/3964510121 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c0cb9f0 con 0x7f887c0720c0 2024-04-25T10:10:09.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.902+0000 7f887a000640 1 -- 172.21.15.119:0/3964510121 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635536745 0 0) 0x7f887c0c4640 con 0x7f887c15e310 2024-04-25T10:10:09.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.902+0000 7f887aa00640 1 --2- 172.21.15.119:0/3964510121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f887c159d60 0x7f887c15a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:09.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.903+0000 7f887b400640 1 --2- 172.21.15.119:0/3964510121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f887c0720c0 0x7f887c0ccf50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8874009870 tx=0x7f887402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abdda8b01b6fc52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.903+0000 7f887a000640 1 -- 172.21.15.119:0/3964510121 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874037070 con 0x7f887c0720c0 2024-04-25T10:10:09.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.903+0000 7f887a000640 1 -- 172.21.15.119:0/3964510121 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8874032440 con 0x7f887c0720c0 2024-04-25T10:10:09.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.903+0000 7f887a000640 1 -- 172.21.15.119:0/3964510121 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874032810 con 0x7f887c0720c0 2024-04-25T10:10:09.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.904+0000 7f887be00640 1 -- 172.21.15.119:0/3964510121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f887c0720c0 msgr2=0x7f887c0ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.904+0000 7f887be00640 1 --2- 172.21.15.119:0/3964510121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f887c0720c0 0x7f887c0ccf50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8874009870 tx=0x7f887402a250 comp rx=0 tx=0).stop 2024-04-25T10:10:09.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.904+0000 7f887be00640 1 -- 172.21.15.119:0/3964510121 shutdown_connections 2024-04-25T10:10:09.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.904+0000 7f887be00640 1 --2- 172.21.15.119:0/3964510121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f887c0720c0 0x7f887c0ccf50 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.904+0000 7f887be00640 1 --2- 172.21.15.119:0/3964510121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f887c159d60 0x7f887c15a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.904+0000 7f887be00640 1 -- 172.21.15.119:0/3964510121 >> 172.21.15.119:0/3964510121 conn(0x7f887c06aba0 msgr2=0x7f887c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.904+0000 7f887be00640 1 -- 172.21.15.119:0/3964510121 shutdown_connections 2024-04-25T10:10:09.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.904+0000 7f887be00640 1 -- 172.21.15.119:0/3964510121 wait complete. 2024-04-25T10:10:09.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.906+0000 7f887be00640 1 Processor -- start 2024-04-25T10:10:09.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.906+0000 7f887be00640 1 -- start start 2024-04-25T10:10:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.907+0000 7f887be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f887c0720c0 0x7f887c148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.907+0000 7f887be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f887c159d60 0x7f887c15aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.907+0000 7f887be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f887c15e310 0x7f887c150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.907+0000 7f887be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c15e730 con 0x7f887c0720c0 2024-04-25T10:10:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.907+0000 7f887be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0cbfb0 con 0x7f887c15e310 2024-04-25T10:10:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.907+0000 7f887be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c15e8b0 con 0x7f887c159d60 2024-04-25T10:10:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.907+0000 7f887aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f887c0720c0 0x7f887c148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.907+0000 7f887aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f887c0720c0 0x7f887c148080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39454/0 (socket says 172.21.15.119:39454) 2024-04-25T10:10:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.907+0000 7f887aa00640 1 -- 172.21.15.119:0/2704654757 learned_addr learned my addr 172.21.15.119:0/2704654757 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.908+0000 7f8872000640 1 --2- 172.21.15.119:0/2704654757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f887c159d60 0x7f887c15aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.908+0000 7f887aa00640 1 -- 172.21.15.119:0/2704654757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f887c159d60 msgr2=0x7f887c15aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.908+0000 7f887aa00640 1 --2- 172.21.15.119:0/2704654757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f887c159d60 0x7f887c15aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.908+0000 7f887aa00640 1 -- 172.21.15.119:0/2704654757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f887c15e310 msgr2=0x7f887c150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.908+0000 7f887aa00640 1 --2- 172.21.15.119:0/2704654757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f887c15e310 0x7f887c150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.908+0000 7f887aa00640 1 -- 172.21.15.119:0/2704654757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c15b420 con 0x7f887c0720c0 2024-04-25T10:10:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.908+0000 7f887aa00640 1 --2- 172.21.15.119:0/2704654757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f887c0720c0 0x7f887c148080 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f886c00cf90 tx=0x7f886c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.908+0000 7f8873e00640 1 -- 172.21.15.119:0/2704654757 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c01a070 con 0x7f887c0720c0 2024-04-25T10:10:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.908+0000 7f8873e00640 1 -- 172.21.15.119:0/2704654757 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c00d840 con 0x7f887c0720c0 2024-04-25T10:10:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.908+0000 7f8873e00640 1 -- 172.21.15.119:0/2704654757 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c00dc10 con 0x7f887c0720c0 2024-04-25T10:10:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.908+0000 7f887be00640 1 -- 172.21.15.119:0/2704654757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887c15c320 con 0x7f887c0720c0 2024-04-25T10:10:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.909+0000 7f887be00640 1 -- 172.21.15.119:0/2704654757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c166e80 con 0x7f887c0720c0 2024-04-25T10:10:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.911+0000 7f8873e00640 1 -- 172.21.15.119:0/2704654757 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f886c01d070 con 0x7f887c0720c0 2024-04-25T10:10:09.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.911+0000 7f8873e00640 1 --2- 172.21.15.119:0/2704654757 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f885806b4d0 0x7f885806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.912+0000 7f8873e00640 1 -- 172.21.15.119:0/2704654757 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f886c08aa50 con 0x7f887c0720c0 2024-04-25T10:10:09.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.912+0000 7f8872000640 1 --2- 172.21.15.119:0/2704654757 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f885806b4d0 0x7f885806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.912+0000 7f8884c4d740 1 Processor -- start 2024-04-25T10:10:09.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.912+0000 7f8872000640 1 --2- 172.21.15.119:0/2704654757 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f885806b4d0 0x7f885806d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f887c15d810 tx=0x7f8864009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.912+0000 7f8884c4d740 1 -- start start 2024-04-25T10:10:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.913+0000 7f8884c4d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a232069d70 0x55a23206bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.913+0000 7f8884c4d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2320a6440 0x55a23206a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.913+0000 7f8884c4d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a232094830 0x55a232094cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.913+0000 7f8884c4d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a231fcdc70 con 0x55a232069d70 2024-04-25T10:10:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.913+0000 7f8884c4d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a231f76cf0 con 0x55a232094830 2024-04-25T10:10:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.913+0000 7f8884c4d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a231fc9d10 con 0x55a2320a6440 2024-04-25T10:10:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.914+0000 7f887aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a232069d70 0x55a23206bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.914+0000 7f887aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a232069d70 0x55a23206bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39460/0 (socket says 172.21.15.119:39460) 2024-04-25T10:10:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.914+0000 7f887aa00640 1 -- 172.21.15.119:0/2058875971 learned_addr learned my addr 172.21.15.119:0/2058875971 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.914+0000 7f887aa00640 1 -- 172.21.15.119:0/2058875971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2320a6440 msgr2=0x55a23206a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.914+0000 7f887aa00640 1 --2- 172.21.15.119:0/2058875971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2320a6440 0x55a23206a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.914+0000 7f887aa00640 1 -- 172.21.15.119:0/2058875971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a232094830 msgr2=0x55a232094cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.914+0000 7f887aa00640 1 --2- 172.21.15.119:0/2058875971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a232094830 0x55a232094cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.914+0000 7f887aa00640 1 -- 172.21.15.119:0/2058875971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a231dac7d0 con 0x55a232069d70 2024-04-25T10:10:09.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.914+0000 7f887aa00640 1 --2- 172.21.15.119:0/2058875971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a232069d70 0x55a23206bed0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f886c061310 tx=0x7f886c059320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db6733693b3ea09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.914+0000 7f886b400640 1 -- 172.21.15.119:0/2058875971 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c00ddb0 con 0x55a232069d70 2024-04-25T10:10:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.914+0000 7f886b400640 1 -- 172.21.15.119:0/2058875971 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c0574e0 con 0x55a232069d70 2024-04-25T10:10:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.914+0000 7f886b400640 1 -- 172.21.15.119:0/2058875971 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c0613b0 con 0x55a232069d70 2024-04-25T10:10:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.915+0000 7f8884c4d740 1 -- 172.21.15.119:0/2058875971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a232069d70 msgr2=0x55a23206bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.915+0000 7f8884c4d740 1 --2- 172.21.15.119:0/2058875971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a232069d70 0x55a23206bed0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f886c061310 tx=0x7f886c059320 comp rx=0 tx=0).stop 2024-04-25T10:10:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.915+0000 7f8884c4d740 1 -- 172.21.15.119:0/2058875971 shutdown_connections 2024-04-25T10:10:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.915+0000 7f8884c4d740 1 --2- 172.21.15.119:0/2058875971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a232094830 0x55a232094cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.915+0000 7f8884c4d740 1 --2- 172.21.15.119:0/2058875971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a2320a6440 0x55a23206a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.915+0000 7f8884c4d740 1 --2- 172.21.15.119:0/2058875971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a232069d70 0x55a23206bed0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.915+0000 7f8884c4d740 1 -- 172.21.15.119:0/2058875971 >> 172.21.15.119:0/2058875971 conn(0x55a232031a30 msgr2=0x55a2320af830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.915+0000 7f8884c4d740 1 -- 172.21.15.119:0/2058875971 shutdown_connections 2024-04-25T10:10:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.915+0000 7f8884c4d740 1 -- 172.21.15.119:0/2058875971 wait complete. 2024-04-25T10:10:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.916+0000 7f8884c4d740 1 -- start start 2024-04-25T10:10:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.916+0000 7f8884c4d740 1 Processor -- start 2024-04-25T10:10:09.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.917+0000 7f8884c4d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 0x55a2320850d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.917+0000 7f8884c4d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a232094830 0x55a2320a8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.917+0000 7f8884c4d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2320a6440 0x55a232097510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.917+0000 7f8884c4d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a231f6c3a0 con 0x55a2320a6440 2024-04-25T10:10:09.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.917+0000 7f8884c4d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a232092380 con 0x55a232094830 2024-04-25T10:10:09.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.917+0000 7f8884c4d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a232085610 con 0x55a232069d70 2024-04-25T10:10:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.917+0000 7f8884c4d740 10 client.0 _ll_register_callbacks cb 0x55a231f89b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.918+0000 7f887b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2320a6440 0x55a232097510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.918+0000 7f887b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2320a6440 0x55a232097510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39466/0 (socket says 172.21.15.119:39466) 2024-04-25T10:10:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.918+0000 7f887b400640 1 -- 172.21.15.119:0/2249197091 learned_addr learned my addr 172.21.15.119:0/2249197091 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.918+0000 7f887aa00640 1 --2- 172.21.15.119:0/2249197091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 0x55a2320850d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.918+0000 7f887b400640 1 -- 172.21.15.119:0/2249197091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 msgr2=0x55a2320850d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.918+0000 7f887b400640 1 --2- 172.21.15.119:0/2249197091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 0x55a2320850d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.918+0000 7f887b400640 1 -- 172.21.15.119:0/2249197091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a232094830 msgr2=0x55a2320a8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.918+0000 7f8872000640 1 --2- 172.21.15.119:0/2249197091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a232094830 0x55a2320a8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.918+0000 7f887b400640 1 --2- 172.21.15.119:0/2249197091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a232094830 0x55a2320a8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.918+0000 7f887b400640 1 -- 172.21.15.119:0/2249197091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a231ce23f0 con 0x55a2320a6440 2024-04-25T10:10:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.918+0000 7f887aa00640 1 --2- 172.21.15.119:0/2249197091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 0x55a2320850d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.918+0000 7f887b400640 1 --2- 172.21.15.119:0/2249197091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2320a6440 0x55a232097510 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8874035040 tx=0x7f887402a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902831f5bd3170e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.918+0000 7f885fe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.919+0000 7f885fe00640 1 -- 172.21.15.119:0/2249197091 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874037040 con 0x55a2320a6440 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.919+0000 7f885fe00640 1 -- 172.21.15.119:0/2249197091 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8874003810 con 0x55a2320a6440 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.919+0000 7f8884c4d740 1 -- 172.21.15.119:0/2249197091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f886c0574e0 con 0x55a2320a6440 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.919+0000 7f885fe00640 1 -- 172.21.15.119:0/2249197091 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8874048050 con 0x55a2320a6440 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.919+0000 7f8884c4d740 10 client.6141 fetch_fsmap learned FSMap version 34 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.919+0000 7f8884c4d740 1 -- 172.21.15.119:0/2249197091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55a231ca20f0 con 0x55a2320a6440 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.919+0000 7f885fe00640 1 -- 172.21.15.119:0/2249197091 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c0574e0 con 0x55a2320a6440 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.919+0000 7f885fe00640 1 -- 172.21.15.119:0/2249197091 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f887404b020 con 0x55a2320a6440 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.919+0000 7f8884c4d740 10 client.6141 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.919+0000 7f8884c4d740 10 client.6141 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.919+0000 7f8884c4d740 20 client.6141 populate_metadata read hostname 'smithi119' 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.920+0000 7f8884c4d740 1 --2- 172.21.15.119:0/2249197091 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55a232097af0 0x55a2320bf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.920+0000 7f8884c4d740 4 client.6141 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.920+0000 7f8884c4d740 1 -- 172.21.15.119:0/2249197091 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a231fdb1b0 con 0x55a232097af0 2024-04-25T10:10:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.920+0000 7f887aa00640 1 --2- 172.21.15.119:0/2249197091 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55a232097af0 0x55a2320bf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.920+0000 7f887aa00640 1 --2- 172.21.15.119:0/2249197091 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55a232097af0 0x55a2320bf480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65b73cb92b0be90e server_cookie=e8b0acaeab72f6c2 in_seq=0 out_seq=0 2024-04-25T10:10:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.920+0000 7f885fe00640 10 client.6141 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.921+0000 7f885fe00640 1 -- 172.21.15.119:0/2249197091 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a231fdb1b0 con 0x55a232097af0 2024-04-25T10:10:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.921+0000 7f885fe00640 10 client.6141 handle_command_reply: tid=0 2024-04-25T10:10:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.921+0000 7f8884c4d740 1 client.6141 shutdown 2024-04-25T10:10:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.921+0000 7f8884c4d740 1 -- 172.21.15.119:0/2249197091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2320a6440 msgr2=0x55a232097510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.921+0000 7f8884c4d740 1 --2- 172.21.15.119:0/2249197091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2320a6440 0x55a232097510 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8874035040 tx=0x7f887402a770 comp rx=0 tx=0).stop 2024-04-25T10:10:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.921+0000 7f8884c4d740 1 -- 172.21.15.119:0/2249197091 shutdown_connections 2024-04-25T10:10:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.921+0000 7f8884c4d740 1 --2- 172.21.15.119:0/2249197091 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55a232097af0 0x55a2320bf480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.921+0000 7f8884c4d740 1 --2- 172.21.15.119:0/2249197091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2320a6440 0x55a232097510 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.921+0000 7f8884c4d740 1 --2- 172.21.15.119:0/2249197091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a232094830 0x55a2320a8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.921+0000 7f8884c4d740 1 --2- 172.21.15.119:0/2249197091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 0x55a2320850d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.921+0000 7f885fe00640 0 client.6141 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.921+0000 7f8884c4d740 1 -- 172.21.15.119:0/2249197091 >> 172.21.15.119:0/2249197091 conn(0x55a232031a30 msgr2=0x55a2320b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.921+0000 7f8884c4d740 1 -- 172.21.15.119:0/2249197091 shutdown_connections 2024-04-25T10:10:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.921+0000 7f8884c4d740 1 -- 172.21.15.119:0/2249197091 wait complete. 2024-04-25T10:10:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.922+0000 7f8884c4d740 20 client.6141 trim_cache size 0 max 16384 2024-04-25T10:10:09.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.938+0000 7f8884c4d740 1 Processor -- start 2024-04-25T10:10:09.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.938+0000 7f8884c4d740 1 -- start start 2024-04-25T10:10:09.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.939+0000 7f8884c4d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 0x55a232097500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.939+0000 7f8884c4d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a232094830 0x55a2320846c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.939+0000 7f8884c4d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2320a6440 0x55a2320601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.939+0000 7f8884c4d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c1ffb40 con 0x55a2320a6440 2024-04-25T10:10:09.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.939+0000 7f8884c4d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c1fed70 con 0x55a232094830 2024-04-25T10:10:09.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.939+0000 7f8884c4d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c1537f0 con 0x55a232069d70 2024-04-25T10:10:09.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.939+0000 7f887b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2320a6440 0x55a2320601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.939+0000 7f887b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2320a6440 0x55a2320601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39478/0 (socket says 172.21.15.119:39478) 2024-04-25T10:10:09.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.939+0000 7f887b400640 1 -- 172.21.15.119:0/233680712 learned_addr learned my addr 172.21.15.119:0/233680712 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.939+0000 7f887b400640 1 -- 172.21.15.119:0/233680712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 msgr2=0x55a232097500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.939+0000 7f887b400640 1 --2- 172.21.15.119:0/233680712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 0x55a232097500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.939+0000 7f887b400640 1 -- 172.21.15.119:0/233680712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a232094830 msgr2=0x55a2320846c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.939+0000 7f887b400640 1 --2- 172.21.15.119:0/233680712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a232094830 0x55a2320846c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.939+0000 7f887b400640 1 -- 172.21.15.119:0/233680712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a232000ca0 con 0x55a2320a6440 2024-04-25T10:10:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.940+0000 7f887b400640 1 --2- 172.21.15.119:0/233680712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2320a6440 0x55a2320601c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8874006da0 tx=0x7f8874038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3d4acaf0fe3162 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.940+0000 7f886aa00640 1 -- 172.21.15.119:0/233680712 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874037030 con 0x55a2320a6440 2024-04-25T10:10:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.940+0000 7f886aa00640 1 -- 172.21.15.119:0/233680712 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8874040070 con 0x55a2320a6440 2024-04-25T10:10:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.940+0000 7f886aa00640 1 -- 172.21.15.119:0/233680712 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874003610 con 0x55a2320a6440 2024-04-25T10:10:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.941+0000 7f8884c4d740 1 -- 172.21.15.119:0/233680712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2320a6440 msgr2=0x55a2320601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.941+0000 7f8884c4d740 1 --2- 172.21.15.119:0/233680712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2320a6440 0x55a2320601c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8874006da0 tx=0x7f8874038000 comp rx=0 tx=0).stop 2024-04-25T10:10:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.941+0000 7f8884c4d740 1 -- 172.21.15.119:0/233680712 shutdown_connections 2024-04-25T10:10:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.941+0000 7f8884c4d740 1 --2- 172.21.15.119:0/233680712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a2320a6440 0x55a2320601c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.941+0000 7f8884c4d740 1 --2- 172.21.15.119:0/233680712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a232094830 0x55a2320846c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.941+0000 7f8884c4d740 1 --2- 172.21.15.119:0/233680712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 0x55a232097500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.941+0000 7f8884c4d740 1 -- 172.21.15.119:0/233680712 >> 172.21.15.119:0/233680712 conn(0x55a232031a30 msgr2=0x55a2320a9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.941+0000 7f8884c4d740 1 -- 172.21.15.119:0/233680712 shutdown_connections 2024-04-25T10:10:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.941+0000 7f8884c4d740 1 -- 172.21.15.119:0/233680712 wait complete. 2024-04-25T10:10:09.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.941+0000 7f8884c4d740 1 -- start start 2024-04-25T10:10:09.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.942+0000 7f8884c4d740 1 Processor -- start 2024-04-25T10:10:09.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8884c4d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 0x55a2320a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8884c4d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a232094830 0x55a2320a9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8884c4d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2320a6440 0x55a23208f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8884c4d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a231f8b020 con 0x55a232094830 2024-04-25T10:10:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8884c4d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c1fe770 con 0x55a2320a6440 2024-04-25T10:10:09.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8884c4d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a232069750 con 0x55a232069d70 2024-04-25T10:10:09.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8884c4d740 10 client.0 _ll_register_callbacks cb 0x7f8874035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:09.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8872000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a232094830 0x55a2320a9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8872000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a232094830 0x55a2320a9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39492/0 (socket says 172.21.15.119:39492) 2024-04-25T10:10:09.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8872000640 1 -- 172.21.15.119:0/1843948824 learned_addr learned my addr 172.21.15.119:0/1843948824 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:09.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f887aa00640 1 --2- 172.21.15.119:0/1843948824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 0x55a2320a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8872000640 1 -- 172.21.15.119:0/1843948824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 msgr2=0x55a2320a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8872000640 1 --2- 172.21.15.119:0/1843948824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 0x55a2320a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f887b400640 1 --2- 172.21.15.119:0/1843948824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2320a6440 0x55a23208f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8872000640 1 -- 172.21.15.119:0/1843948824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2320a6440 msgr2=0x55a23208f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:09.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8872000640 1 --2- 172.21.15.119:0/1843948824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2320a6440 0x55a23208f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.943+0000 7f8872000640 1 -- 172.21.15.119:0/1843948824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a232068780 con 0x55a232094830 2024-04-25T10:10:09.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.944+0000 7f8872000640 1 --2- 172.21.15.119:0/1843948824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a232094830 0x55a2320a9180 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f887c168b20 tx=0x7f8864005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f173d94c13b1b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:09.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.944+0000 7f885fe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:10:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.944+0000 7f885fe00640 1 -- 172.21.15.119:0/1843948824 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886400ddf0 con 0x55a232094830 2024-04-25T10:10:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.944+0000 7f885fe00640 1 -- 172.21.15.119:0/1843948824 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8864020a80 con 0x55a232094830 2024-04-25T10:10:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.944+0000 7f8884c4d740 1 -- 172.21.15.119:0/1843948824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8874040070 con 0x55a232094830 2024-04-25T10:10:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.944+0000 7f885fe00640 1 -- 172.21.15.119:0/1843948824 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8864020c80 con 0x55a232094830 2024-04-25T10:10:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.944+0000 7f8884c4d740 10 client.6147 fetch_fsmap learned FSMap version 34 2024-04-25T10:10:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.944+0000 7f8884c4d740 1 -- 172.21.15.119:0/1843948824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55a232068da0 con 0x55a232094830 2024-04-25T10:10:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.944+0000 7f885fe00640 1 -- 172.21.15.119:0/1843948824 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8874040070 con 0x55a232094830 2024-04-25T10:10:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.945+0000 7f885fe00640 1 -- 172.21.15.119:0/1843948824 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8864020420 con 0x55a232094830 2024-04-25T10:10:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.945+0000 7f8884c4d740 10 client.6147 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T10:10:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.945+0000 7f8884c4d740 10 client.6147 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:10:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.945+0000 7f8884c4d740 20 client.6147 populate_metadata read hostname 'smithi119' 2024-04-25T10:10:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.945+0000 7f8884c4d740 1 --2- 172.21.15.119:0/1843948824 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55a232063ec0 0x55a2321192c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.945+0000 7f8884c4d740 4 client.6147 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:09.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.945+0000 7f8884c4d740 1 -- 172.21.15.119:0/1843948824 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a231fd5500 con 0x55a232063ec0 2024-04-25T10:10:09.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.945+0000 7f887aa00640 1 --2- 172.21.15.119:0/1843948824 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55a232063ec0 0x55a2321192c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.945+0000 7f887aa00640 1 --2- 172.21.15.119:0/1843948824 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55a232063ec0 0x55a2321192c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c0457cd5230eee3 server_cookie=5649bce82cfcf3d1 in_seq=0 out_seq=0 2024-04-25T10:10:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.945+0000 7f885fe00640 10 client.6147 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f885fe00640 1 -- 172.21.15.119:0/1843948824 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x55a231fd5500 con 0x55a232063ec0 2024-04-25T10:10:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f885fe00640 10 client.6147 handle_command_reply: tid=0 2024-04-25T10:10:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f8884c4d740 1 client.6147 shutdown 2024-04-25T10:10:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f8884c4d740 1 -- 172.21.15.119:0/1843948824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a232094830 msgr2=0x55a2320a9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f8884c4d740 1 --2- 172.21.15.119:0/1843948824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a232094830 0x55a2320a9180 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f887c168b20 tx=0x7f8864005f10 comp rx=0 tx=0).stop 2024-04-25T10:10:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f8884c4d740 1 -- 172.21.15.119:0/1843948824 shutdown_connections 2024-04-25T10:10:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f8884c4d740 1 --2- 172.21.15.119:0/1843948824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2320a6440 0x55a23208f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f8884c4d740 1 --2- 172.21.15.119:0/1843948824 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55a232063ec0 0x55a2321192c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f8884c4d740 1 --2- 172.21.15.119:0/1843948824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a232094830 0x55a2320a9180 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f8884c4d740 1 --2- 172.21.15.119:0/1843948824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a232069d70 0x55a2320a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f885fe00640 0 client.6147 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:09.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f8884c4d740 1 -- 172.21.15.119:0/1843948824 >> 172.21.15.119:0/1843948824 conn(0x55a232031a30 msgr2=0x55a2320b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f8884c4d740 1 -- 172.21.15.119:0/1843948824 shutdown_connections 2024-04-25T10:10:09.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f8884c4d740 1 -- 172.21.15.119:0/1843948824 wait complete. 2024-04-25T10:10:09.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.946+0000 7f8884c4d740 20 client.6147 trim_cache size 0 max 16384 2024-04-25T10:10:09.958 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:10:09.958 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:10:09.958 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2024-04-25T10:10:09.958 INFO:teuthology.orchestra.run.smithi119.stdout: "60883a24-7f56-4761-8b27-43315998b266": { 2024-04-25T10:10:09.959 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/", 2024-04-25T10:10:09.959 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "60883a24-7f56-4761-8b27-43315998b266", 2024-04-25T10:10:09.959 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive" 2024-04-25T10:10:09.959 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:10:09.959 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:10:09.959 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:10:09.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.947+0000 7f887be00640 1 -- 172.21.15.119:0/2704654757 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f885806b4d0 msgr2=0x7f885806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.947+0000 7f887be00640 1 --2- 172.21.15.119:0/2704654757 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f885806b4d0 0x7f885806d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f887c15d810 tx=0x7f8864009040 comp rx=0 tx=0).stop 2024-04-25T10:10:09.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.947+0000 7f887be00640 1 -- 172.21.15.119:0/2704654757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f887c0720c0 msgr2=0x7f887c148080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:09.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.947+0000 7f887be00640 1 --2- 172.21.15.119:0/2704654757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f887c0720c0 0x7f887c148080 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f886c00cf90 tx=0x7f886c00a7d0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.947+0000 7f887be00640 1 -- 172.21.15.119:0/2704654757 shutdown_connections 2024-04-25T10:10:09.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.947+0000 7f887be00640 1 --2- 172.21.15.119:0/2704654757 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f885806b4d0 0x7f885806d990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.947+0000 7f887be00640 1 --2- 172.21.15.119:0/2704654757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f887c15e310 0x7f887c150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.947+0000 7f887be00640 1 --2- 172.21.15.119:0/2704654757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f887c159d60 0x7f887c15aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.947+0000 7f887be00640 1 --2- 172.21.15.119:0/2704654757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f887c0720c0 0x7f887c148080 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:09.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.947+0000 7f887be00640 1 -- 172.21.15.119:0/2704654757 >> 172.21.15.119:0/2704654757 conn(0x7f887c06aba0 msgr2=0x7f887c158870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:09.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.947+0000 7f887be00640 1 -- 172.21.15.119:0/2704654757 shutdown_connections 2024-04-25T10:10:09.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:09.947+0000 7f887be00640 1 -- 172.21.15.119:0/2704654757 wait complete. 2024-04-25T10:10:09.971 INFO:tasks.cephfs.filesystem:scrub status for tag:60883a24-7f56-4761-8b27-43315998b266 - {'path': '/', 'tag': '60883a24-7f56-4761-8b27-43315998b266', 'options': 'recursive'} 2024-04-25T10:10:16.461 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:10:16.456+0000 7f8c67800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:10:30.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:30.871 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:39.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:39.972 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:10:40.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.180+0000 7f0abda00640 1 Processor -- start 2024-04-25T10:10:40.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.181+0000 7f0abda00640 1 -- start start 2024-04-25T10:10:40.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.182+0000 7f0abda00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ab8148290 0x7f0ab8071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.182+0000 7f0abda00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0ab8166050 0x7f0ab8071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.182+0000 7f0abda00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab81666b0 con 0x7f0ab81630f0 2024-04-25T10:10:40.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.182+0000 7f0abda00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab8166480 con 0x7f0ab8148290 2024-04-25T10:10:40.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.182+0000 7f0abda00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab8162610 con 0x7f0ab8166050 2024-04-25T10:10:40.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.182+0000 7f0aaf400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0ab81630f0 0x7f0ab8165940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46092/0 (socket says 172.21.15.119:46092) 2024-04-25T10:10:40.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.182+0000 7f0aaf400640 1 -- 172.21.15.119:0/1257917613 learned_addr learned my addr 172.21.15.119:0/1257917613 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:10:40.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.182+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/1257917613 >> v2:172.21.15.119:3300/0 conn(0x7f0ab8148290 0x7f0ab8071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.183+0000 7f0abd000640 1 --2- 172.21.15.119:0/1257917613 >> v2:172.21.15.132:3300/0 conn(0x7f0ab8166050 0x7f0ab8071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.183+0000 7f0ab7400640 1 -- 172.21.15.119:0/1257917613 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337626923 0 0) 0x7f0ab81666b0 con 0x7f0ab81630f0 2024-04-25T10:10:40.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.183+0000 7f0ab7400640 1 -- 172.21.15.119:0/1257917613 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a9c003520 con 0x7f0ab81630f0 2024-04-25T10:10:40.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.183+0000 7f0ab7e00640 1 -- 172.21.15.119:0/1257917613 >> v2:172.21.15.132:3300/0 conn(0x7f0ab8166050 msgr2=0x7f0ab8071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.183+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/1257917613 >> v2:172.21.15.132:3300/0 conn(0x7f0ab8166050 0x7f0ab8071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.183+0000 7f0ab7e00640 1 -- 172.21.15.119:0/1257917613 >> v1:172.21.15.119:6789/0 conn(0x7f0ab81630f0 legacy=0x7f0ab8165940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.183+0000 7f0ab7e00640 1 -- 172.21.15.119:0/1257917613 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab81628f0 con 0x7f0ab8148290 2024-04-25T10:10:40.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.185+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/1257917613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab8148290 0x7f0ab8071120 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0ab806e4c0 tx=0x7f0aa802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7ae75abfeae095 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.185+0000 7f0ab7400640 1 -- 172.21.15.119:0/1257917613 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8037070 con 0x7f0ab8148290 2024-04-25T10:10:40.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.185+0000 7f0ab7400640 1 -- 172.21.15.119:0/1257917613 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa8032440 con 0x7f0ab8148290 2024-04-25T10:10:40.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.185+0000 7f0ab7400640 1 -- 172.21.15.119:0/1257917613 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa802ae60 con 0x7f0ab8148290 2024-04-25T10:10:40.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.185+0000 7f0abda00640 1 -- 172.21.15.119:0/1257917613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab8148290 msgr2=0x7f0ab8071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.185+0000 7f0abda00640 1 --2- 172.21.15.119:0/1257917613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab8148290 0x7f0ab8071120 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0ab806e4c0 tx=0x7f0aa802a110 comp rx=0 tx=0).stop 2024-04-25T10:10:40.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.186+0000 7f0abda00640 1 -- 172.21.15.119:0/1257917613 shutdown_connections 2024-04-25T10:10:40.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.186+0000 7f0abda00640 1 --2- 172.21.15.119:0/1257917613 >> v2:172.21.15.132:3300/0 conn(0x7f0ab8166050 0x7f0ab8071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.186+0000 7f0abda00640 1 --2- 172.21.15.119:0/1257917613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab8148290 0x7f0ab8071120 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.186+0000 7f0abda00640 1 -- 172.21.15.119:0/1257917613 >> 172.21.15.119:0/1257917613 conn(0x7f0ab806aba0 msgr2=0x7f0ab806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.186+0000 7f0abda00640 1 -- 172.21.15.119:0/1257917613 shutdown_connections 2024-04-25T10:10:40.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.186+0000 7f0abda00640 1 -- 172.21.15.119:0/1257917613 wait complete. 2024-04-25T10:10:40.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.187+0000 7f0abda00640 1 Processor -- start 2024-04-25T10:10:40.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.188+0000 7f0abda00640 1 -- start start 2024-04-25T10:10:40.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0abda00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab8148290 0x7f0ab8146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0abda00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab81630f0 0x7f0ab8142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0abda00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab8166050 0x7f0ab814a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0abda00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80b1090 con 0x7f0ab8166050 2024-04-25T10:10:40.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0abda00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8162300 con 0x7f0ab8148290 2024-04-25T10:10:40.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0abda00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80b1210 con 0x7f0ab81630f0 2024-04-25T10:10:40.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0abd000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab8166050 0x7f0ab814a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0abd000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab8166050 0x7f0ab814a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39300/0 (socket says 172.21.15.119:39300) 2024-04-25T10:10:40.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0abd000640 1 -- 172.21.15.119:0/3098395703 learned_addr learned my addr 172.21.15.119:0/3098395703 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:40.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/3098395703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab8148290 0x7f0ab8146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0abd000640 1 -- 172.21.15.119:0/3098395703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab81630f0 msgr2=0x7f0ab8142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0abd000640 1 --2- 172.21.15.119:0/3098395703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab81630f0 0x7f0ab8142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0abd000640 1 -- 172.21.15.119:0/3098395703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab8148290 msgr2=0x7f0ab8146570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0abd000640 1 --2- 172.21.15.119:0/3098395703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab8148290 0x7f0ab8146570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0abd000640 1 -- 172.21.15.119:0/3098395703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab8143500 con 0x7f0ab8166050 2024-04-25T10:10:40.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.189+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/3098395703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab8148290 0x7f0ab8146570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:40.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.190+0000 7f0abd000640 1 --2- 172.21.15.119:0/3098395703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab8166050 0x7f0ab814a3b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0007e60 tx=0x7f0ab000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.190+0000 7f0ab4c00640 1 -- 172.21.15.119:0/3098395703 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0014420 con 0x7f0ab8166050 2024-04-25T10:10:40.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.190+0000 7f0ab4c00640 1 -- 172.21.15.119:0/3098395703 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab0019070 con 0x7f0ab8166050 2024-04-25T10:10:40.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.190+0000 7f0ab4c00640 1 -- 172.21.15.119:0/3098395703 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0014820 con 0x7f0ab8166050 2024-04-25T10:10:40.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.190+0000 7f0abda00640 1 -- 172.21.15.119:0/3098395703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab81437e0 con 0x7f0ab8166050 2024-04-25T10:10:40.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.190+0000 7f0abda00640 1 -- 172.21.15.119:0/3098395703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab813ed80 con 0x7f0ab8166050 2024-04-25T10:10:40.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.192+0000 7f0ab4c00640 1 -- 172.21.15.119:0/3098395703 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ab00149c0 con 0x7f0ab8166050 2024-04-25T10:10:40.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.193+0000 7f0ab4c00640 1 --2- 172.21.15.119:0/3098395703 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0a9406b4d0 0x7f0a9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.193+0000 7f0ab4c00640 1 -- 172.21.15.119:0/3098395703 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ab008aa70 con 0x7f0ab8166050 2024-04-25T10:10:40.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.194+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/3098395703 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0a9406b4d0 0x7f0a9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.194+0000 7f0ac1e6c740 1 Processor -- start 2024-04-25T10:10:40.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.194+0000 7f0ac1e6c740 1 -- start start 2024-04-25T10:10:40.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.194+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/3098395703 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0a9406b4d0 0x7f0a9406d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8034000 tx=0x7f0aa802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.195+0000 7f0ac1e6c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e846496d70 0x55e846498ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.195+0000 7f0ac1e6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8464d3440 0x55e846497150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.195+0000 7f0ac1e6c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8464c1830 0x55e8464c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.195+0000 7f0ac1e6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8463fac70 con 0x55e8464d3440 2024-04-25T10:10:40.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.195+0000 7f0ac1e6c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8463a3cf0 con 0x55e846496d70 2024-04-25T10:10:40.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.195+0000 7f0ac1e6c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8463f6d10 con 0x55e8464c1830 2024-04-25T10:10:40.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.196+0000 7f0aaf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8464d3440 0x55e846497150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.196+0000 7f0aaf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8464d3440 0x55e846497150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39312/0 (socket says 172.21.15.119:39312) 2024-04-25T10:10:40.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.196+0000 7f0aaf400640 1 -- 172.21.15.119:0/90876848 learned_addr learned my addr 172.21.15.119:0/90876848 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:40.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.196+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/90876848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e846496d70 0x55e846498ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.196+0000 7f0aaf400640 1 -- 172.21.15.119:0/90876848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8464c1830 msgr2=0x55e8464c1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.196+0000 7f0aaf400640 1 --2- 172.21.15.119:0/90876848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8464c1830 0x55e8464c1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.196+0000 7f0aaf400640 1 -- 172.21.15.119:0/90876848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e846496d70 msgr2=0x55e846498ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.196+0000 7f0aaf400640 1 --2- 172.21.15.119:0/90876848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e846496d70 0x55e846498ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.196+0000 7f0aaf400640 1 -- 172.21.15.119:0/90876848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8461d97d0 con 0x55e8464d3440 2024-04-25T10:10:40.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.196+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/90876848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e846496d70 0x55e846498ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:40.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.196+0000 7f0aaf400640 1 --2- 172.21.15.119:0/90876848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8464d3440 0x55e846497150 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0001650 tx=0x7f0aa0001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b336ae1cc92d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.197+0000 7f0aacc00640 1 -- 172.21.15.119:0/90876848 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0017070 con 0x55e8464d3440 2024-04-25T10:10:40.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.197+0000 7f0aacc00640 1 -- 172.21.15.119:0/90876848 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa0005ce0 con 0x55e8464d3440 2024-04-25T10:10:40.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.197+0000 7f0aacc00640 1 -- 172.21.15.119:0/90876848 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa00072d0 con 0x55e8464d3440 2024-04-25T10:10:40.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.197+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/90876848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8464d3440 msgr2=0x55e846497150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.197+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/90876848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8464d3440 0x55e846497150 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0001650 tx=0x7f0aa0001b10 comp rx=0 tx=0).stop 2024-04-25T10:10:40.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.197+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/90876848 shutdown_connections 2024-04-25T10:10:40.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.197+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/90876848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8464c1830 0x55e8464c1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.197+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/90876848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8464d3440 0x55e846497150 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.197+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/90876848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e846496d70 0x55e846498ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.197+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/90876848 >> 172.21.15.119:0/90876848 conn(0x55e84645ea30 msgr2=0x55e8464dc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.197+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/90876848 shutdown_connections 2024-04-25T10:10:40.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.197+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/90876848 wait complete. 2024-04-25T10:10:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.198+0000 7f0ac1e6c740 1 -- start start 2024-04-25T10:10:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.198+0000 7f0ac1e6c740 1 Processor -- start 2024-04-25T10:10:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.199+0000 7f0ac1e6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e846496d70 0x55e8464bc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.199+0000 7f0ac1e6c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464c1830 0x55e8464ece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.199+0000 7f0ac1e6c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8464d3440 0x55e8464c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.199+0000 7f0ac1e6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8463993a0 con 0x55e846496d70 2024-04-25T10:10:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.199+0000 7f0ac1e6c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8464bf380 con 0x55e8464c1830 2024-04-25T10:10:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.199+0000 7f0ac1e6c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8464d4cc0 con 0x55e8464d3440 2024-04-25T10:10:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.199+0000 7f0ac1e6c740 10 client.0 _ll_register_callbacks cb 0x55e8463b6b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.200+0000 7f0ab7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e846496d70 0x55e8464bc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.200+0000 7f0ab7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e846496d70 0x55e8464bc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39320/0 (socket says 172.21.15.119:39320) 2024-04-25T10:10:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.200+0000 7f0ab7e00640 1 -- 172.21.15.119:0/3177152942 learned_addr learned my addr 172.21.15.119:0/3177152942 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.200+0000 7f0aaf400640 1 --2- 172.21.15.119:0/3177152942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464c1830 0x55e8464ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.200+0000 7f0abd000640 1 --2- 172.21.15.119:0/3177152942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8464d3440 0x55e8464c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.200+0000 7f0ab7e00640 1 -- 172.21.15.119:0/3177152942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8464d3440 msgr2=0x55e8464c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.200+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/3177152942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8464d3440 0x55e8464c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.200+0000 7f0ab7e00640 1 -- 172.21.15.119:0/3177152942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464c1830 msgr2=0x55e8464ece80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.200+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/3177152942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464c1830 0x55e8464ece80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.200+0000 7f0ab7e00640 1 -- 172.21.15.119:0/3177152942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e84610f3f0 con 0x55e846496d70 2024-04-25T10:10:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.200+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/3177152942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e846496d70 0x55e8464bc4c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009860 tx=0x7f0aa8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0eaeedcbc26937d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.200+0000 7f0aa5600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:10:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.201+0000 7f0aa5600640 1 -- 172.21.15.119:0/3177152942 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8037040 con 0x55e846496d70 2024-04-25T10:10:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.201+0000 7f0aa5600640 1 -- 172.21.15.119:0/3177152942 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0aa803ca30 con 0x55e846496d70 2024-04-25T10:10:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.201+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/3177152942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0aa0005ce0 con 0x55e846496d70 2024-04-25T10:10:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.201+0000 7f0aa5600640 1 -- 172.21.15.119:0/3177152942 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0aa8054050 con 0x55e846496d70 2024-04-25T10:10:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.201+0000 7f0ac1e6c740 10 client.6162 fetch_fsmap learned FSMap version 35 2024-04-25T10:10:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.201+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/3177152942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55e8460cf0f0 con 0x55e846496d70 2024-04-25T10:10:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.201+0000 7f0aa5600640 1 -- 172.21.15.119:0/3177152942 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa0005ce0 con 0x55e846496d70 2024-04-25T10:10:40.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.201+0000 7f0aa5600640 1 -- 172.21.15.119:0/3177152942 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0aa8057020 con 0x55e846496d70 2024-04-25T10:10:40.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.201+0000 7f0ac1e6c740 10 client.6162 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T10:10:40.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.201+0000 7f0ac1e6c740 10 client.6162 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:10:40.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.201+0000 7f0ac1e6c740 20 client.6162 populate_metadata read hostname 'smithi119' 2024-04-25T10:10:40.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.201+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/3177152942 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55e8464d3bb0 0x55e8464d3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.201+0000 7f0ac1e6c740 4 client.6162 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:40.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.201+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/3177152942 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8464081b0 con 0x55e8464d3bb0 2024-04-25T10:10:40.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.202+0000 7f0aaf400640 1 --2- 172.21.15.119:0/3177152942 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55e8464d3bb0 0x55e8464d3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.202+0000 7f0aaf400640 1 --2- 172.21.15.119:0/3177152942 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55e8464d3bb0 0x55e8464d3fd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5785566387c143a2 server_cookie=6c4350653d9aa37d in_seq=0 out_seq=0 2024-04-25T10:10:40.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.202+0000 7f0aa5600640 10 client.6162 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:40.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.203+0000 7f0aa5600640 1 -- 172.21.15.119:0/3177152942 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e8464081b0 con 0x55e8464d3bb0 2024-04-25T10:10:40.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.203+0000 7f0aa5600640 10 client.6162 handle_command_reply: tid=0 2024-04-25T10:10:40.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.203+0000 7f0ac1e6c740 1 client.6162 shutdown 2024-04-25T10:10:40.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.203+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/3177152942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e846496d70 msgr2=0x55e8464bc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.203+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/3177152942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e846496d70 0x55e8464bc4c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009860 tx=0x7f0aa8004730 comp rx=0 tx=0).stop 2024-04-25T10:10:40.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.203+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/3177152942 shutdown_connections 2024-04-25T10:10:40.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.203+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/3177152942 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8464d3440 0x55e8464c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.203+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/3177152942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464c1830 0x55e8464ece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.203+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/3177152942 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55e8464d3bb0 0x55e8464d3fd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.203+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/3177152942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e846496d70 0x55e8464bc4c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.203+0000 7f0aa5600640 0 client.6162 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:40.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.203+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/3177152942 >> 172.21.15.119:0/3177152942 conn(0x55e84645ea30 msgr2=0x55e8464bca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.203+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/3177152942 shutdown_connections 2024-04-25T10:10:40.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.204+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/3177152942 wait complete. 2024-04-25T10:10:40.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.204+0000 7f0ac1e6c740 20 client.6162 trim_cache size 0 max 16384 2024-04-25T10:10:40.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.220+0000 7f0ac1e6c740 1 Processor -- start 2024-04-25T10:10:40.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.220+0000 7f0ac1e6c740 1 -- start start 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.221+0000 7f0ac1e6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e846496d70 0x55e8464d7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.221+0000 7f0ac1e6c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464c1830 0x55e8464d3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.221+0000 7f0ac1e6c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8464d3440 0x55e8464bbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.221+0000 7f0ac1e6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8203e10 con 0x55e846496d70 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.221+0000 7f0ac1e6c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8202ff0 con 0x55e8464c1830 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.221+0000 7f0ac1e6c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab813fee0 con 0x55e8464d3440 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.221+0000 7f0ab7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e846496d70 0x55e8464d7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.222+0000 7f0ab7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e846496d70 0x55e8464d7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39328/0 (socket says 172.21.15.119:39328) 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.222+0000 7f0ab7e00640 1 -- 172.21.15.119:0/816873637 learned_addr learned my addr 172.21.15.119:0/816873637 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.222+0000 7f0ab7e00640 1 -- 172.21.15.119:0/816873637 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8464d3440 msgr2=0x55e8464bbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.222+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/816873637 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8464d3440 0x55e8464bbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.222+0000 7f0ab7e00640 1 -- 172.21.15.119:0/816873637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464c1830 msgr2=0x55e8464d3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.222+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/816873637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464c1830 0x55e8464d3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.222+0000 7f0ab7e00640 1 -- 172.21.15.119:0/816873637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8464c2570 con 0x55e846496d70 2024-04-25T10:10:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.222+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/816873637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e846496d70 0x55e8464d7010 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8033fd0 tx=0x7f0aa803f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5174ceb15189edf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.222+0000 7f0aa7e00640 1 -- 172.21.15.119:0/816873637 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa805d070 con 0x55e846496d70 2024-04-25T10:10:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.222+0000 7f0aa7e00640 1 -- 172.21.15.119:0/816873637 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa804c070 con 0x55e846496d70 2024-04-25T10:10:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.222+0000 7f0aa7e00640 1 -- 172.21.15.119:0/816873637 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8037040 con 0x55e846496d70 2024-04-25T10:10:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.223+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/816873637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e846496d70 msgr2=0x55e8464d7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.223+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/816873637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e846496d70 0x55e8464d7010 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8033fd0 tx=0x7f0aa803f070 comp rx=0 tx=0).stop 2024-04-25T10:10:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.223+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/816873637 shutdown_connections 2024-04-25T10:10:40.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.223+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/816873637 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e8464d3440 0x55e8464bbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.223+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/816873637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464c1830 0x55e8464d3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.223+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/816873637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e846496d70 0x55e8464d7010 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.223+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/816873637 >> 172.21.15.119:0/816873637 conn(0x55e84645ea30 msgr2=0x55e8464c5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.223+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/816873637 shutdown_connections 2024-04-25T10:10:40.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.223+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/816873637 wait complete. 2024-04-25T10:10:40.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.224+0000 7f0ac1e6c740 1 -- start start 2024-04-25T10:10:40.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.224+0000 7f0ac1e6c740 1 Processor -- start 2024-04-25T10:10:40.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.225+0000 7f0ac1e6c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e846496d70 0x55e84652cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.225+0000 7f0ac1e6c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8464c1830 0x55e8464d9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.225+0000 7f0ac1e6c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464d3440 0x55e8464c3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.225+0000 7f0ac1e6c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8464ed9d0 con 0x55e8464c1830 2024-04-25T10:10:40.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.225+0000 7f0ac1e6c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab81404e0 con 0x55e8464d3440 2024-04-25T10:10:40.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.225+0000 7f0ac1e6c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8464d5940 con 0x55e846496d70 2024-04-25T10:10:40.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.225+0000 7f0ac1e6c740 10 client.0 _ll_register_callbacks cb 0x7f0aa8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:40.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.225+0000 7f0aaf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8464c1830 0x55e8464d9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.225+0000 7f0aaf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8464c1830 0x55e8464d9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39344/0 (socket says 172.21.15.119:39344) 2024-04-25T10:10:40.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.225+0000 7f0aaf400640 1 -- 172.21.15.119:0/310057305 learned_addr learned my addr 172.21.15.119:0/310057305 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:40.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.226+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/310057305 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e846496d70 0x55e84652cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.226+0000 7f0aaf400640 1 -- 172.21.15.119:0/310057305 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e846496d70 msgr2=0x55e84652cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.226+0000 7f0aaf400640 1 --2- 172.21.15.119:0/310057305 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e846496d70 0x55e84652cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.226+0000 7f0aaf400640 1 -- 172.21.15.119:0/310057305 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464d3440 msgr2=0x55e8464c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.226+0000 7f0abd000640 1 --2- 172.21.15.119:0/310057305 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464d3440 0x55e8464c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.226+0000 7f0aaf400640 1 --2- 172.21.15.119:0/310057305 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464d3440 0x55e8464c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.226+0000 7f0aaf400640 1 -- 172.21.15.119:0/310057305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e84654a790 con 0x55e8464c1830 2024-04-25T10:10:40.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.226+0000 7f0abd000640 1 --2- 172.21.15.119:0/310057305 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464d3440 0x55e8464c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:40.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.226+0000 7f0aaf400640 1 --2- 172.21.15.119:0/310057305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8464c1830 0x55e8464d9890 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0015040 tx=0x7f0aa0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38afb9e64f14a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.226+0000 7f0aa5600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:10:40.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.226+0000 7f0aa5600640 1 -- 172.21.15.119:0/310057305 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0017040 con 0x55e8464c1830 2024-04-25T10:10:40.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.226+0000 7f0aa5600640 1 -- 172.21.15.119:0/310057305 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0aa00073a0 con 0x55e8464c1830 2024-04-25T10:10:40.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.226+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/310057305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0aa804c070 con 0x55e8464c1830 2024-04-25T10:10:40.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.227+0000 7f0aa5600640 1 -- 172.21.15.119:0/310057305 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0aa0030050 con 0x55e8464c1830 2024-04-25T10:10:40.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.227+0000 7f0ac1e6c740 10 client.6168 fetch_fsmap learned FSMap version 35 2024-04-25T10:10:40.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.227+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/310057305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55e8464d9e30 con 0x55e8464c1830 2024-04-25T10:10:40.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.227+0000 7f0aa5600640 1 -- 172.21.15.119:0/310057305 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa804c070 con 0x55e8464c1830 2024-04-25T10:10:40.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.227+0000 7f0aa5600640 1 -- 172.21.15.119:0/310057305 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0aa0033020 con 0x55e8464c1830 2024-04-25T10:10:40.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.227+0000 7f0ac1e6c740 10 client.6168 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T10:10:40.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.227+0000 7f0ac1e6c740 10 client.6168 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:10:40.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.227+0000 7f0ac1e6c740 20 client.6168 populate_metadata read hostname 'smithi119' 2024-04-25T10:10:40.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.227+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/310057305 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55e8464bc2d0 0x55e8464c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.227+0000 7f0ac1e6c740 4 client.6168 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:40.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.227+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/310057305 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e846402500 con 0x55e8464bc2d0 2024-04-25T10:10:40.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.227+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/310057305 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55e8464bc2d0 0x55e8464c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0ab7e00640 1 --2- 172.21.15.119:0/310057305 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55e8464bc2d0 0x55e8464c3d80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6c0ce263d3cb767 server_cookie=dadf0871e5ce37f3 in_seq=0 out_seq=0 2024-04-25T10:10:40.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0aa5600640 10 client.6168 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:40.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0aa5600640 1 -- 172.21.15.119:0/310057305 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e846402500 con 0x55e8464bc2d0 2024-04-25T10:10:40.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0aa5600640 10 client.6168 handle_command_reply: tid=0 2024-04-25T10:10:40.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0ac1e6c740 1 client.6168 shutdown 2024-04-25T10:10:40.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/310057305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8464c1830 msgr2=0x55e8464d9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/310057305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8464c1830 0x55e8464d9890 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0015040 tx=0x7f0aa0000ed0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/310057305 shutdown_connections 2024-04-25T10:10:40.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/310057305 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e8464d3440 0x55e8464c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/310057305 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x55e8464bc2d0 0x55e8464c3d80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/310057305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e8464c1830 0x55e8464d9890 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0ac1e6c740 1 --2- 172.21.15.119:0/310057305 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e846496d70 0x55e84652cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0aa5600640 0 client.6168 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:40.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/310057305 >> 172.21.15.119:0/310057305 conn(0x55e84645ea30 msgr2=0x55e846498d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.228+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/310057305 shutdown_connections 2024-04-25T10:10:40.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.229+0000 7f0ac1e6c740 1 -- 172.21.15.119:0/310057305 wait complete. 2024-04-25T10:10:40.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.229+0000 7f0ac1e6c740 20 client.6168 trim_cache size 0 max 16384 2024-04-25T10:10:40.238 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:10:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2024-04-25T10:10:40.238 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2024-04-25T10:10:40.239 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:10:40.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.229+0000 7f0abda00640 1 -- 172.21.15.119:0/3098395703 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0a9406b4d0 msgr2=0x7f0a9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.229+0000 7f0abda00640 1 --2- 172.21.15.119:0/3098395703 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0a9406b4d0 0x7f0a9406d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8034000 tx=0x7f0aa802e000 comp rx=0 tx=0).stop 2024-04-25T10:10:40.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.229+0000 7f0abda00640 1 -- 172.21.15.119:0/3098395703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab8166050 msgr2=0x7f0ab814a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.229+0000 7f0abda00640 1 --2- 172.21.15.119:0/3098395703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab8166050 0x7f0ab814a3b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0007e60 tx=0x7f0ab000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.229+0000 7f0abda00640 1 -- 172.21.15.119:0/3098395703 shutdown_connections 2024-04-25T10:10:40.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.229+0000 7f0abda00640 1 --2- 172.21.15.119:0/3098395703 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0a9406b4d0 0x7f0a9406d990 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.229+0000 7f0abda00640 1 --2- 172.21.15.119:0/3098395703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab8166050 0x7f0ab814a3b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.229+0000 7f0abda00640 1 --2- 172.21.15.119:0/3098395703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab81630f0 0x7f0ab8142d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.229+0000 7f0abda00640 1 --2- 172.21.15.119:0/3098395703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab8148290 0x7f0ab8146570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.229+0000 7f0abda00640 1 -- 172.21.15.119:0/3098395703 >> 172.21.15.119:0/3098395703 conn(0x7f0ab806aba0 msgr2=0x7f0ab8165940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.229+0000 7f0abda00640 1 -- 172.21.15.119:0/3098395703 shutdown_connections 2024-04-25T10:10:40.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.230+0000 7f0abda00640 1 -- 172.21.15.119:0/3098395703 wait complete. 2024-04-25T10:10:40.252 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:10:40.253 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.444+0000 7fdf2d000640 1 Processor -- start 2024-04-25T10:10:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.444+0000 7fdf2d000640 1 -- start start 2024-04-25T10:10:40.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.446+0000 7fdf2d000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf2815bf00 0x7fdf280cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.446+0000 7fdf2d000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf280cc310 0x7fdf2815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.446+0000 7fdf2d000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf280ce6c0 con 0x7fdf2815c820 2024-04-25T10:10:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.446+0000 7fdf2d000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf281455e0 con 0x7fdf280cc310 2024-04-25T10:10:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.446+0000 7fdf2d000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2806d450 con 0x7fdf2815bf00 2024-04-25T10:10:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.446+0000 7fdf25400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf280cc310 0x7fdf2815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.446+0000 7fdf25400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf280cc310 0x7fdf2815c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39348/0 (socket says 172.21.15.119:39348) 2024-04-25T10:10:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.446+0000 7fdf25400640 1 -- 172.21.15.119:0/2334749679 learned_addr learned my addr 172.21.15.119:0/2334749679 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:40.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.447+0000 7fdf25e00640 1 --2- 172.21.15.119:0/2334749679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf2815bf00 0x7fdf280cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.447+0000 7fdf25400640 1 -- 172.21.15.119:0/2334749679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf2815bf00 msgr2=0x7fdf280cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.447+0000 7fdf25400640 1 --2- 172.21.15.119:0/2334749679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf2815bf00 0x7fdf280cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.447+0000 7fdf25400640 1 -- 172.21.15.119:0/2334749679 >> v1:172.21.15.132:6789/0 conn(0x7fdf2815c820 legacy=0x7fdf2806d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.447+0000 7fdf25400640 1 -- 172.21.15.119:0/2334749679 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf280cc7b0 con 0x7fdf280cc310 2024-04-25T10:10:40.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.447+0000 7fdf25e00640 1 --2- 172.21.15.119:0/2334749679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf2815bf00 0x7fdf280cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:40.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.448+0000 7fdf25400640 1 --2- 172.21.15.119:0/2334749679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf280cc310 0x7fdf2815c2e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009870 tx=0x7fdf1402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaffcdf35458e613 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.448+0000 7fdf24a00640 1 -- 172.21.15.119:0/2334749679 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14037070 con 0x7fdf280cc310 2024-04-25T10:10:40.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.448+0000 7fdf24a00640 1 -- 172.21.15.119:0/2334749679 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf14032440 con 0x7fdf280cc310 2024-04-25T10:10:40.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.448+0000 7fdf24a00640 1 -- 172.21.15.119:0/2334749679 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14032810 con 0x7fdf280cc310 2024-04-25T10:10:40.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.449+0000 7fdf2d000640 1 -- 172.21.15.119:0/2334749679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf280cc310 msgr2=0x7fdf2815c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.449+0000 7fdf2d000640 1 --2- 172.21.15.119:0/2334749679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf280cc310 0x7fdf2815c2e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009870 tx=0x7fdf1402a210 comp rx=0 tx=0).stop 2024-04-25T10:10:40.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.449+0000 7fdf2d000640 1 -- 172.21.15.119:0/2334749679 shutdown_connections 2024-04-25T10:10:40.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.449+0000 7fdf2d000640 1 --2- 172.21.15.119:0/2334749679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf280cc310 0x7fdf2815c2e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.449+0000 7fdf2d000640 1 --2- 172.21.15.119:0/2334749679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf2815bf00 0x7fdf280cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.449+0000 7fdf2d000640 1 -- 172.21.15.119:0/2334749679 >> 172.21.15.119:0/2334749679 conn(0x7fdf2806aba0 msgr2=0x7fdf2806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.449+0000 7fdf2d000640 1 -- 172.21.15.119:0/2334749679 shutdown_connections 2024-04-25T10:10:40.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.449+0000 7fdf2d000640 1 -- 172.21.15.119:0/2334749679 wait complete. 2024-04-25T10:10:40.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.451+0000 7fdf2d000640 1 Processor -- start 2024-04-25T10:10:40.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.451+0000 7fdf2d000640 1 -- start start 2024-04-25T10:10:40.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.452+0000 7fdf2d000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf280cc310 0x7fdf28137fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.452+0000 7fdf2d000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf2815bf00 0x7fdf28143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.452+0000 7fdf2d000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf2815c820 0x7fdf28144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.452+0000 7fdf2d000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf280cd050 con 0x7fdf280cc310 2024-04-25T10:10:40.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.452+0000 7fdf2d000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf280ccd70 con 0x7fdf2815c820 2024-04-25T10:10:40.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.452+0000 7fdf2d000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2815cca0 con 0x7fdf2815bf00 2024-04-25T10:10:40.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.452+0000 7fdf25e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf280cc310 0x7fdf28137fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.452+0000 7fdf25e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf280cc310 0x7fdf28137fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39352/0 (socket says 172.21.15.119:39352) 2024-04-25T10:10:40.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.452+0000 7fdf25e00640 1 -- 172.21.15.119:0/2386854149 learned_addr learned my addr 172.21.15.119:0/2386854149 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:40.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.453+0000 7fdf25400640 1 --2- 172.21.15.119:0/2386854149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf2815bf00 0x7fdf28143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.453+0000 7fdf26800640 1 --2- 172.21.15.119:0/2386854149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf2815c820 0x7fdf28144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.453+0000 7fdf25e00640 1 -- 172.21.15.119:0/2386854149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf2815bf00 msgr2=0x7fdf28143c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.453+0000 7fdf25e00640 1 --2- 172.21.15.119:0/2386854149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf2815bf00 0x7fdf28143c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.453+0000 7fdf25e00640 1 -- 172.21.15.119:0/2386854149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf2815c820 msgr2=0x7fdf28144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.453+0000 7fdf25e00640 1 --2- 172.21.15.119:0/2386854149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf2815c820 0x7fdf28144160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.453+0000 7fdf25e00640 1 -- 172.21.15.119:0/2386854149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2814d720 con 0x7fdf280cc310 2024-04-25T10:10:40.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.453+0000 7fdf25e00640 1 --2- 172.21.15.119:0/2386854149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf280cc310 0x7fdf28137fe0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800c820 tx=0x7fdf1800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.453+0000 7fdf1e000640 1 -- 172.21.15.119:0/2386854149 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18015070 con 0x7fdf280cc310 2024-04-25T10:10:40.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.453+0000 7fdf1e000640 1 -- 172.21.15.119:0/2386854149 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf18007a30 con 0x7fdf280cc310 2024-04-25T10:10:40.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.453+0000 7fdf1e000640 1 -- 172.21.15.119:0/2386854149 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18007e00 con 0x7fdf280cc310 2024-04-25T10:10:40.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.453+0000 7fdf2d000640 1 -- 172.21.15.119:0/2386854149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf281f4c90 con 0x7fdf280cc310 2024-04-25T10:10:40.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.454+0000 7fdf2d000640 1 -- 172.21.15.119:0/2386854149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf281f5250 con 0x7fdf280cc310 2024-04-25T10:10:40.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.455+0000 7fdf2d000640 1 -- 172.21.15.119:0/2386854149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2806bb30 con 0x7fdf280cc310 2024-04-25T10:10:40.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.455+0000 7fdf1e000640 1 -- 172.21.15.119:0/2386854149 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf18018070 con 0x7fdf280cc310 2024-04-25T10:10:40.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.456+0000 7fdf1e000640 1 --2- 172.21.15.119:0/2386854149 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdf0806b4d0 0x7fdf0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.456+0000 7fdf1e000640 1 -- 172.21.15.119:0/2386854149 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf1808a280 con 0x7fdf280cc310 2024-04-25T10:10:40.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.457+0000 7fdf25400640 1 --2- 172.21.15.119:0/2386854149 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdf0806b4d0 0x7fdf0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.457+0000 7fdf25400640 1 --2- 172.21.15.119:0/2386854149 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdf0806b4d0 0x7fdf0806d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdf14034000 tx=0x7fdf1402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.461+0000 7fdf1e000640 1 -- 172.21.15.119:0/2386854149 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf18059060 con 0x7fdf280cc310 2024-04-25T10:10:40.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.698+0000 7fdf2d000640 1 -- 172.21.15.119:0/2386854149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf280749b0 con 0x7fdf280cc310 2024-04-25T10:10:40.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.700+0000 7fdf1e000640 1 -- 172.21.15.119:0/2386854149 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fdf18058960 con 0x7fdf280cc310 2024-04-25T10:10:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:10:40.705 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4159222235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4159222235},{"type":"v1","addr":"172.21.15.132:6834","nonce":4159222235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4223361610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":4223361610},{"type":"v1","addr":"172.21.15.119:6837","nonce":4223361610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2367324442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2367324442},{"type":"v1","addr":"172.21.15.132:6835","nonce":2367324442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:59.454941+0000","modified":"2024-04-25T10:10:16.088495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":7,"addr":"172.21.15.119:6836/4129112885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4129112885},{"type":"v1","addr":"172.21.15.119:6836","nonce":4129112885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:10:40.706 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 35 2024-04-25T10:10:40.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.702+0000 7fdf2d000640 1 -- 172.21.15.119:0/2386854149 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdf0806b4d0 msgr2=0x7fdf0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.702+0000 7fdf2d000640 1 --2- 172.21.15.119:0/2386854149 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdf0806b4d0 0x7fdf0806d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdf14034000 tx=0x7fdf1402e000 comp rx=0 tx=0).stop 2024-04-25T10:10:40.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.702+0000 7fdf2d000640 1 -- 172.21.15.119:0/2386854149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf280cc310 msgr2=0x7fdf28137fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.702+0000 7fdf2d000640 1 --2- 172.21.15.119:0/2386854149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf280cc310 0x7fdf28137fe0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800c820 tx=0x7fdf1800cc20 comp rx=0 tx=0).stop 2024-04-25T10:10:40.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.702+0000 7fdf2d000640 1 -- 172.21.15.119:0/2386854149 shutdown_connections 2024-04-25T10:10:40.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.702+0000 7fdf2d000640 1 --2- 172.21.15.119:0/2386854149 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdf0806b4d0 0x7fdf0806d990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.702+0000 7fdf2d000640 1 --2- 172.21.15.119:0/2386854149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf2815c820 0x7fdf28144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.702+0000 7fdf2d000640 1 --2- 172.21.15.119:0/2386854149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf2815bf00 0x7fdf28143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.702+0000 7fdf2d000640 1 --2- 172.21.15.119:0/2386854149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf280cc310 0x7fdf28137fe0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.702+0000 7fdf2d000640 1 -- 172.21.15.119:0/2386854149 >> 172.21.15.119:0/2386854149 conn(0x7fdf2806aba0 msgr2=0x7fdf2815ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.702+0000 7fdf2d000640 1 -- 172.21.15.119:0/2386854149 shutdown_connections 2024-04-25T10:10:40.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.702+0000 7fdf2d000640 1 -- 172.21.15.119:0/2386854149 wait complete. 2024-04-25T10:10:40.726 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:10:40.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.924+0000 7f8e18a00640 1 Processor -- start 2024-04-25T10:10:40.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.925+0000 7f8e18a00640 1 -- start start 2024-04-25T10:10:40.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.926+0000 7f8e18a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e140c6150 0x7f8e140ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.926+0000 7f8e18a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e140712c0 con 0x7f8e1406e120 2024-04-25T10:10:40.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.926+0000 7f8e18a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e140715a0 con 0x7f8e1415bcb0 2024-04-25T10:10:40.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.926+0000 7f8e18a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1406e790 con 0x7f8e140c6150 2024-04-25T10:10:40.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.927+0000 7f8e13e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8e1406e120 0x7f8e1415f2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:49438/0 (socket says 172.21.15.119:49438) 2024-04-25T10:10:40.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.927+0000 7f8e13e00640 1 -- 172.21.15.119:0/1298924639 learned_addr learned my addr 172.21.15.119:0/1298924639 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:10:40.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.927+0000 7f8e13400640 1 --2- 172.21.15.119:0/1298924639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e140c6150 0x7f8e140ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.927+0000 7f8e12a00640 1 -- 172.21.15.119:0/1298924639 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265983233 0 0) 0x7f8e140712c0 con 0x7f8e1406e120 2024-04-25T10:10:40.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.927+0000 7f8e12a00640 1 -- 172.21.15.119:0/1298924639 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8df4003520 con 0x7f8e1406e120 2024-04-25T10:10:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.927+0000 7f8e12a00640 1 -- 172.21.15.119:0/1298924639 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 340092738 0 0) 0x7f8df4003520 con 0x7f8e1406e120 2024-04-25T10:10:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.927+0000 7f8e12a00640 1 -- 172.21.15.119:0/1298924639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e140c6150 msgr2=0x7f8e140ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e12a00640 1 --2- 172.21.15.119:0/1298924639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e140c6150 0x7f8e140ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e12a00640 1 -- 172.21.15.119:0/1298924639 >> v1:172.21.15.132:6789/0 conn(0x7f8e1415bcb0 legacy=0x7f8e1406da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e12a00640 1 -- 172.21.15.119:0/1298924639 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e14071880 con 0x7f8e1406e120 2024-04-25T10:10:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e12a00640 1 -- 172.21.15.119:0/1298924639 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f8e0c0019f0 con 0x7f8e1406e120 2024-04-25T10:10:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e12a00640 1 -- 172.21.15.119:0/1298924639 >> v1:172.21.15.119:6789/0 conn(0x7f8e1406e120 legacy=0x7f8e1415f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e12a00640 1 --2- 172.21.15.119:0/1298924639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df40039e0 0x7f8df4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e12a00640 1 --2- 172.21.15.119:0/1298924639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df4004800 0x7f8df4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e12a00640 1 --2- 172.21.15.119:0/1298924639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e140c6150 0x7f8df40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e12a00640 1 -- 172.21.15.119:0/1298924639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df400ba70 con 0x7f8df4004800 2024-04-25T10:10:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e12a00640 1 -- 172.21.15.119:0/1298924639 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df400bbf0 con 0x7f8df40039e0 2024-04-25T10:10:40.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e12a00640 1 -- 172.21.15.119:0/1298924639 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df400bdf0 con 0x7f8e140c6150 2024-04-25T10:10:40.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e0be00640 1 --2- 172.21.15.119:0/1298924639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df4004800 0x7f8df4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e13400640 1 --2- 172.21.15.119:0/1298924639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df40039e0 0x7f8df4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e0be00640 1 -- 172.21.15.119:0/1298924639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e140c6150 msgr2=0x7f8df40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e13e00640 1 --2- 172.21.15.119:0/1298924639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e140c6150 0x7f8df40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e0be00640 1 --2- 172.21.15.119:0/1298924639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e140c6150 0x7f8df40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e0be00640 1 -- 172.21.15.119:0/1298924639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df40039e0 msgr2=0x7f8df4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e0be00640 1 --2- 172.21.15.119:0/1298924639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df40039e0 0x7f8df4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.928+0000 7f8e0be00640 1 -- 172.21.15.119:0/1298924639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df4003520 con 0x7f8df4004800 2024-04-25T10:10:40.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.929+0000 7f8e13400640 1 --2- 172.21.15.119:0/1298924639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8df40039e0 0x7f8df4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:40.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.930+0000 7f8e0be00640 1 --2- 172.21.15.119:0/1298924639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df4004800 0x7f8df4004c80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc001720 tx=0x7f8df4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5657a227cd53ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.930+0000 7f8e12a00640 1 -- 172.21.15.119:0/1298924639 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc001370 con 0x7f8df4004800 2024-04-25T10:10:40.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.930+0000 7f8e12a00640 1 -- 172.21.15.119:0/1298924639 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc0052d0 con 0x7f8df4004800 2024-04-25T10:10:40.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.930+0000 7f8e12a00640 1 -- 172.21.15.119:0/1298924639 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc02ab90 con 0x7f8df4004800 2024-04-25T10:10:40.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.930+0000 7f8e18a00640 1 -- 172.21.15.119:0/1298924639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df4004800 msgr2=0x7f8df4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.930+0000 7f8e18a00640 1 --2- 172.21.15.119:0/1298924639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df4004800 0x7f8df4004c80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc001720 tx=0x7f8df4005830 comp rx=0 tx=0).stop 2024-04-25T10:10:40.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.930+0000 7f8e13e00640 1 -- 172.21.15.119:0/1298924639 reap_dead start 2024-04-25T10:10:40.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.930+0000 7f8e18a00640 1 -- 172.21.15.119:0/1298924639 shutdown_connections 2024-04-25T10:10:40.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.930+0000 7f8e18a00640 1 -- 172.21.15.119:0/1298924639 >> 172.21.15.119:0/1298924639 conn(0x7f8e1406aba0 msgr2=0x7f8e1406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.930+0000 7f8e18a00640 1 -- 172.21.15.119:0/1298924639 shutdown_connections 2024-04-25T10:10:40.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.930+0000 7f8e18a00640 1 -- 172.21.15.119:0/1298924639 wait complete. 2024-04-25T10:10:40.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.932+0000 7f8e18a00640 1 Processor -- start 2024-04-25T10:10:40.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.932+0000 7f8e18a00640 1 -- start start 2024-04-25T10:10:40.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.933+0000 7f8e18a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df4004800 0x7f8e14153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.933+0000 7f8e18a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e14154690 0x7f8e14154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.933+0000 7f8e18a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e141e5190 0x7f8e141e5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.933+0000 7f8e18a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1415fa00 con 0x7f8df4004800 2024-04-25T10:10:40.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.933+0000 7f8e18a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140c6530 con 0x7f8e14154690 2024-04-25T10:10:40.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.933+0000 7f8e18a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1415fb80 con 0x7f8e141e5190 2024-04-25T10:10:40.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.933+0000 7f8e13400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df4004800 0x7f8e14153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.933+0000 7f8e13400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df4004800 0x7f8e14153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43530/0 (socket says 172.21.15.119:43530) 2024-04-25T10:10:40.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.933+0000 7f8e13400640 1 -- 172.21.15.119:0/2130649717 learned_addr learned my addr 172.21.15.119:0/2130649717 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:40.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.934+0000 7f8e0be00640 1 --2- 172.21.15.119:0/2130649717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e14154690 0x7f8e14154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.934+0000 7f8e13e00640 1 --2- 172.21.15.119:0/2130649717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e141e5190 0x7f8e141e5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.934+0000 7f8e13400640 1 -- 172.21.15.119:0/2130649717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e141e5190 msgr2=0x7f8e141e5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.934+0000 7f8e13400640 1 --2- 172.21.15.119:0/2130649717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e141e5190 0x7f8e141e5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.934+0000 7f8e13400640 1 -- 172.21.15.119:0/2130649717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e14154690 msgr2=0x7f8e14154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.934+0000 7f8e13400640 1 --2- 172.21.15.119:0/2130649717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e14154690 0x7f8e14154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.934+0000 7f8e13400640 1 -- 172.21.15.119:0/2130649717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e14154010 con 0x7f8df4004800 2024-04-25T10:10:40.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.934+0000 7f8e13400640 1 --2- 172.21.15.119:0/2130649717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df4004800 0x7f8e14153ad0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8e04010910 tx=0x7f8e04010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.934+0000 7f8e0b400640 1 -- 172.21.15.119:0/2130649717 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0400f040 con 0x7f8df4004800 2024-04-25T10:10:40.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.935+0000 7f8e0b400640 1 -- 172.21.15.119:0/2130649717 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0400ebc0 con 0x7f8df4004800 2024-04-25T10:10:40.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.935+0000 7f8e0b400640 1 -- 172.21.15.119:0/2130649717 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0401c400 con 0x7f8df4004800 2024-04-25T10:10:40.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.935+0000 7f8e18a00640 1 -- 172.21.15.119:0/2130649717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e141e5f20 con 0x7f8df4004800 2024-04-25T10:10:40.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.935+0000 7f8e18a00640 1 -- 172.21.15.119:0/2130649717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e141e64e0 con 0x7f8df4004800 2024-04-25T10:10:40.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.937+0000 7f8e0b400640 1 -- 172.21.15.119:0/2130649717 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e0401c5a0 con 0x7f8df4004800 2024-04-25T10:10:40.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.938+0000 7f8e0b400640 1 --2- 172.21.15.119:0/2130649717 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8df006b4d0 0x7f8df006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.938+0000 7f8e0b400640 1 -- 172.21.15.119:0/2130649717 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e0408a950 con 0x7f8df4004800 2024-04-25T10:10:40.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.939+0000 7f8e0be00640 1 --2- 172.21.15.119:0/2130649717 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8df006b4d0 0x7f8df006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.939+0000 7f8e1cd8f740 1 Processor -- start 2024-04-25T10:10:40.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.939+0000 7f8e1cd8f740 1 -- start start 2024-04-25T10:10:40.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.939+0000 7f8e0be00640 1 --2- 172.21.15.119:0/2130649717 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8df006b4d0 0x7f8df006d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc032000 tx=0x7f8dfc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.940+0000 7f8e1cd8f740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5881b00 0x559bd585a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.940+0000 7f8e1cd8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd5881ee0 0x559bd5884360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.940+0000 7f8e1cd8f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd58848a0 0x559bd5858100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.940+0000 7f8e1cd8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd57bcae0 con 0x559bd5881ee0 2024-04-25T10:10:40.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.940+0000 7f8e1cd8f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd57657c0 con 0x559bd58848a0 2024-04-25T10:10:40.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.940+0000 7f8e1cd8f740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559bd57b8b80 con 0x559bd5881b00 2024-04-25T10:10:40.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.940+0000 7f8e0be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd5881ee0 0x559bd5884360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.940+0000 7f8e0be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd5881ee0 0x559bd5884360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43534/0 (socket says 172.21.15.119:43534) 2024-04-25T10:10:40.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.940+0000 7f8e0be00640 1 -- 172.21.15.119:0/2209962526 learned_addr learned my addr 172.21.15.119:0/2209962526 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:40.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.941+0000 7f8e13400640 1 --2- 172.21.15.119:0/2209962526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5881b00 0x559bd585a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.941+0000 7f8e13e00640 1 --2- 172.21.15.119:0/2209962526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd58848a0 0x559bd5858100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.941+0000 7f8e0be00640 1 -- 172.21.15.119:0/2209962526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5881b00 msgr2=0x559bd585a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.941+0000 7f8e0be00640 1 --2- 172.21.15.119:0/2209962526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5881b00 0x559bd585a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.941+0000 7f8e0be00640 1 -- 172.21.15.119:0/2209962526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd58848a0 msgr2=0x559bd5858100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.941+0000 7f8e0be00640 1 --2- 172.21.15.119:0/2209962526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd58848a0 0x559bd5858100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.941+0000 7f8e0be00640 1 -- 172.21.15.119:0/2209962526 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bd559d7c0 con 0x559bd5881ee0 2024-04-25T10:10:40.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.941+0000 7f8e13400640 1 --2- 172.21.15.119:0/2209962526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5881b00 0x559bd585a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:40.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.941+0000 7f8e13e00640 1 --2- 172.21.15.119:0/2209962526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd58848a0 0x559bd5858100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:40.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.941+0000 7f8e0be00640 1 --2- 172.21.15.119:0/2209962526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd5881ee0 0x559bd5884360 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc001810 tx=0x7f8dfc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f33fc0b3f0a6b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.942+0000 7f8e03400640 1 -- 172.21.15.119:0/2209962526 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0073c0 con 0x559bd5881ee0 2024-04-25T10:10:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.942+0000 7f8e03400640 1 -- 172.21.15.119:0/2209962526 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc007560 con 0x559bd5881ee0 2024-04-25T10:10:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.942+0000 7f8e03400640 1 -- 172.21.15.119:0/2209962526 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0069d0 con 0x559bd5881ee0 2024-04-25T10:10:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.942+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/2209962526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd5881ee0 msgr2=0x559bd5884360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.942+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/2209962526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd5881ee0 0x559bd5884360 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc001810 tx=0x7f8dfc02c000 comp rx=0 tx=0).stop 2024-04-25T10:10:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.942+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/2209962526 shutdown_connections 2024-04-25T10:10:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.942+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/2209962526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd58848a0 0x559bd5858100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.942+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/2209962526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd5881ee0 0x559bd5884360 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.942+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/2209962526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5881b00 0x559bd585a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.942+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/2209962526 >> 172.21.15.119:0/2209962526 conn(0x559bd5856b60 msgr2=0x559bd589f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.942+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/2209962526 shutdown_connections 2024-04-25T10:10:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.943+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/2209962526 wait complete. 2024-04-25T10:10:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.943+0000 7f8e1cd8f740 1 -- start start 2024-04-25T10:10:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.943+0000 7f8e1cd8f740 1 Processor -- start 2024-04-25T10:10:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.944+0000 7f8e1cd8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd5881ee0 0x559bd587fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.944+0000 7f8e1cd8f740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd58848a0 0x559bd5885710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.944+0000 7f8e1cd8f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5871e00 0x559bd589e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.944+0000 7f8e1cd8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd575ad40 con 0x559bd5881ee0 2024-04-25T10:10:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.944+0000 7f8e1cd8f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd589d060 con 0x559bd5871e00 2024-04-25T10:10:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.944+0000 7f8e1cd8f740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559bd5887380 con 0x559bd58848a0 2024-04-25T10:10:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.944+0000 7f8e1cd8f740 10 client.0 _ll_register_callbacks cb 0x559bd54d0ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.945+0000 7f8e13400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd5881ee0 0x559bd587fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.945+0000 7f8e13400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd5881ee0 0x559bd587fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43544/0 (socket says 172.21.15.119:43544) 2024-04-25T10:10:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.945+0000 7f8e13400640 1 -- 172.21.15.119:0/4216686295 learned_addr learned my addr 172.21.15.119:0/4216686295 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.945+0000 7f8e13400640 1 -- 172.21.15.119:0/4216686295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd58848a0 msgr2=0x559bd5885710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.945+0000 7f8e0be00640 1 --2- 172.21.15.119:0/4216686295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd58848a0 0x559bd5885710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.945+0000 7f8e13e00640 1 --2- 172.21.15.119:0/4216686295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5871e00 0x559bd589e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.945+0000 7f8e13400640 1 --2- 172.21.15.119:0/4216686295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd58848a0 0x559bd5885710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.945+0000 7f8e13400640 1 -- 172.21.15.119:0/4216686295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5871e00 msgr2=0x559bd589e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.945+0000 7f8e13400640 1 --2- 172.21.15.119:0/4216686295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5871e00 0x559bd589e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.945+0000 7f8e13400640 1 -- 172.21.15.119:0/4216686295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bd55fdf50 con 0x559bd5881ee0 2024-04-25T10:10:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.945+0000 7f8e13e00640 1 --2- 172.21.15.119:0/4216686295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5871e00 0x559bd589e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8e13400640 1 --2- 172.21.15.119:0/4216686295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd5881ee0 0x559bd587fcf0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8e04061210 tx=0x7f8e04059030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e255ce3b578b65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8dfbe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:10:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8dfbe00640 1 -- 172.21.15.119:0/4216686295 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e040612b0 con 0x559bd5881ee0 2024-04-25T10:10:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8dfbe00640 1 -- 172.21.15.119:0/4216686295 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e04061450 con 0x559bd5881ee0 2024-04-25T10:10:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4216686295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8dfc007560 con 0x559bd5881ee0 2024-04-25T10:10:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8dfbe00640 1 -- 172.21.15.119:0/4216686295 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e04077050 con 0x559bd5881ee0 2024-04-25T10:10:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8e1cd8f740 10 client.6186 fetch_fsmap learned FSMap version 35 2024-04-25T10:10:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4216686295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x559bd54910b0 con 0x559bd5881ee0 2024-04-25T10:10:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8dfbe00640 1 -- 172.21.15.119:0/4216686295 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc007560 con 0x559bd5881ee0 2024-04-25T10:10:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8dfbe00640 1 -- 172.21.15.119:0/4216686295 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8e0401bbd0 con 0x559bd5881ee0 2024-04-25T10:10:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8e1cd8f740 10 client.6186 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T10:10:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8e1cd8f740 10 client.6186 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:10:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8e1cd8f740 20 client.6186 populate_metadata read hostname 'smithi119' 2024-04-25T10:10:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/4216686295 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559bd5883550 0x559bd5886250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8e1cd8f740 4 client.6186 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.946+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4216686295 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bd581bb60 con 0x559bd5883550 2024-04-25T10:10:40.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.947+0000 7f8e0be00640 1 --2- 172.21.15.119:0/4216686295 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559bd5883550 0x559bd5886250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.947+0000 7f8e0be00640 1 --2- 172.21.15.119:0/4216686295 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559bd5883550 0x559bd5886250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=374836ca6eea320a server_cookie=61801bc1bd3db6b0 in_seq=0 out_seq=0 2024-04-25T10:10:40.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.947+0000 7f8dfbe00640 10 client.6186 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:40.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.948+0000 7f8dfbe00640 1 -- 172.21.15.119:0/4216686295 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559bd581bb60 con 0x559bd5883550 2024-04-25T10:10:40.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.948+0000 7f8dfbe00640 10 client.6186 handle_command_reply: tid=0 2024-04-25T10:10:40.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.948+0000 7f8e1cd8f740 1 client.6186 shutdown 2024-04-25T10:10:40.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.948+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4216686295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd5881ee0 msgr2=0x559bd587fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.948+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/4216686295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd5881ee0 0x559bd587fcf0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8e04061210 tx=0x7f8e04059030 comp rx=0 tx=0).stop 2024-04-25T10:10:40.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.948+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4216686295 shutdown_connections 2024-04-25T10:10:40.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.948+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/4216686295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5871e00 0x559bd589e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.948+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/4216686295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd58848a0 0x559bd5885710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.948+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/4216686295 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559bd5883550 0x559bd5886250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.948+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/4216686295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd5881ee0 0x559bd587fcf0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.948+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4216686295 >> 172.21.15.119:0/4216686295 conn(0x559bd5881b00 msgr2=0x559bd585a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.948+0000 7f8dfbe00640 0 client.6186 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:40.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.948+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4216686295 shutdown_connections 2024-04-25T10:10:40.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.948+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4216686295 wait complete. 2024-04-25T10:10:40.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.949+0000 7f8e1cd8f740 20 client.6186 trim_cache size 0 max 16384 2024-04-25T10:10:40.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.961+0000 7f8e1cd8f740 1 Processor -- start 2024-04-25T10:10:40.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.961+0000 7f8e1cd8f740 1 -- start start 2024-04-25T10:10:40.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.962+0000 7f8e1cd8f740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5871e00 0x559bd587faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.962+0000 7f8e1cd8f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5881ee0 0x559bd58861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.962+0000 7f8e1cd8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd58848a0 0x559bd5886710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.962+0000 7f8e1cd8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140cdb10 con 0x559bd58848a0 2024-04-25T10:10:40.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.962+0000 7f8e1cd8f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1413fdc0 con 0x559bd5881ee0 2024-04-25T10:10:40.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.962+0000 7f8e1cd8f740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e140cd510 con 0x559bd5871e00 2024-04-25T10:10:40.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.963+0000 7f8e13e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd58848a0 0x559bd5886710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.963+0000 7f8e13e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd58848a0 0x559bd5886710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43550/0 (socket says 172.21.15.119:43550) 2024-04-25T10:10:40.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.963+0000 7f8e13e00640 1 -- 172.21.15.119:0/418278490 learned_addr learned my addr 172.21.15.119:0/418278490 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:40.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.963+0000 7f8e13e00640 1 -- 172.21.15.119:0/418278490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5871e00 msgr2=0x559bd587faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.963+0000 7f8e13e00640 1 --2- 172.21.15.119:0/418278490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5871e00 0x559bd587faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.963+0000 7f8e13e00640 1 -- 172.21.15.119:0/418278490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5881ee0 msgr2=0x559bd58861d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.963+0000 7f8e13e00640 1 --2- 172.21.15.119:0/418278490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5881ee0 0x559bd58861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.963+0000 7f8e13e00640 1 -- 172.21.15.119:0/418278490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bd58c5eb0 con 0x559bd58848a0 2024-04-25T10:10:40.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.963+0000 7f8e13e00640 1 --2- 172.21.15.119:0/418278490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd58848a0 0x559bd5886710 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c000f80 tx=0x7f8e0c010670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb9915239040719 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.963+0000 7f8e02a00640 1 -- 172.21.15.119:0/418278490 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c004b60 con 0x559bd58848a0 2024-04-25T10:10:40.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.964+0000 7f8e02a00640 1 -- 172.21.15.119:0/418278490 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c004d00 con 0x559bd58848a0 2024-04-25T10:10:40.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.964+0000 7f8e02a00640 1 -- 172.21.15.119:0/418278490 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c007310 con 0x559bd58848a0 2024-04-25T10:10:40.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.964+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/418278490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd58848a0 msgr2=0x559bd5886710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.964+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/418278490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd58848a0 0x559bd5886710 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c000f80 tx=0x7f8e0c010670 comp rx=0 tx=0).stop 2024-04-25T10:10:40.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.964+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/418278490 shutdown_connections 2024-04-25T10:10:40.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.964+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/418278490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd58848a0 0x559bd5886710 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.964+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/418278490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5881ee0 0x559bd58861d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.964+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/418278490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5871e00 0x559bd587faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.964+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/418278490 >> 172.21.15.119:0/418278490 conn(0x559bd5881b00 msgr2=0x559bd585b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.964+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/418278490 shutdown_connections 2024-04-25T10:10:40.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.964+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/418278490 wait complete. 2024-04-25T10:10:40.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.965+0000 7f8e1cd8f740 1 -- start start 2024-04-25T10:10:40.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.965+0000 7f8e1cd8f740 1 Processor -- start 2024-04-25T10:10:40.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.966+0000 7f8e1cd8f740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5871e00 0x559bd58b5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.966+0000 7f8e1cd8f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5881ee0 0x559bd58b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.966+0000 7f8e1cd8f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd58848a0 0x559bd58bc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.966+0000 7f8e1cd8f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd58833d0 con 0x559bd58848a0 2024-04-25T10:10:40.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.966+0000 7f8e1cd8f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bd58870c0 con 0x559bd5881ee0 2024-04-25T10:10:40.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.966+0000 7f8e1cd8f740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559bd5883970 con 0x559bd5871e00 2024-04-25T10:10:40.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.966+0000 7f8e1cd8f740 10 client.0 _ll_register_callbacks cb 0x7f8df40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:40.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.966+0000 7f8e13e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd58848a0 0x559bd58bc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.967+0000 7f8e13e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd58848a0 0x559bd58bc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43552/0 (socket says 172.21.15.119:43552) 2024-04-25T10:10:40.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.967+0000 7f8e13e00640 1 -- 172.21.15.119:0/4014983009 learned_addr learned my addr 172.21.15.119:0/4014983009 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:40.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.967+0000 7f8e0be00640 1 --2- 172.21.15.119:0/4014983009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5881ee0 0x559bd58b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.967+0000 7f8e13400640 1 --2- 172.21.15.119:0/4014983009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5871e00 0x559bd58b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.967+0000 7f8e13e00640 1 -- 172.21.15.119:0/4014983009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5871e00 msgr2=0x559bd58b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.967+0000 7f8e13e00640 1 --2- 172.21.15.119:0/4014983009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5871e00 0x559bd58b5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.967+0000 7f8e13e00640 1 -- 172.21.15.119:0/4014983009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5881ee0 msgr2=0x559bd58b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.967+0000 7f8e13e00640 1 --2- 172.21.15.119:0/4014983009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5881ee0 0x559bd58b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.967+0000 7f8e13e00640 1 -- 172.21.15.119:0/4014983009 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bd58598b0 con 0x559bd58848a0 2024-04-25T10:10:40.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.967+0000 7f8e13e00640 1 --2- 172.21.15.119:0/4014983009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd58848a0 0x559bd58bc7a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8e140725c0 tx=0x7f8e14071770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ded9e81bf08366 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.967+0000 7f8dfbe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:10:40.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.967+0000 7f8dfbe00640 1 -- 172.21.15.119:0/4014983009 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c022070 con 0x559bd58848a0 2024-04-25T10:10:40.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.967+0000 7f8dfbe00640 1 -- 172.21.15.119:0/4014983009 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e0c013420 con 0x559bd58848a0 2024-04-25T10:10:40.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.967+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4014983009 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e0c004d00 con 0x559bd58848a0 2024-04-25T10:10:40.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.968+0000 7f8dfbe00640 1 -- 172.21.15.119:0/4014983009 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e0c004b60 con 0x559bd58848a0 2024-04-25T10:10:40.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.968+0000 7f8e1cd8f740 10 client.6192 fetch_fsmap learned FSMap version 35 2024-04-25T10:10:40.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.968+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4014983009 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x559bd58b5c10 con 0x559bd58848a0 2024-04-25T10:10:40.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.968+0000 7f8dfbe00640 1 -- 172.21.15.119:0/4014983009 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c004d00 con 0x559bd58848a0 2024-04-25T10:10:40.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.968+0000 7f8dfbe00640 1 -- 172.21.15.119:0/4014983009 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8e0c013620 con 0x559bd58848a0 2024-04-25T10:10:40.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.968+0000 7f8e1cd8f740 10 client.6192 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T10:10:40.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.968+0000 7f8e1cd8f740 10 client.6192 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:10:40.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.968+0000 7f8e1cd8f740 20 client.6192 populate_metadata read hostname 'smithi119' 2024-04-25T10:10:40.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.968+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/4014983009 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559bd5886040 0x559bd5886420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.968+0000 7f8e1cd8f740 4 client.6192 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:10:40.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.968+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4014983009 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x559bd57ca020 con 0x559bd5886040 2024-04-25T10:10:40.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.968+0000 7f8e13400640 1 --2- 172.21.15.119:0/4014983009 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559bd5886040 0x559bd5886420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.969+0000 7f8e13400640 1 --2- 172.21.15.119:0/4014983009 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559bd5886040 0x559bd5886420 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae6ffb09ba13f5 server_cookie=5b0736c6011e8b34 in_seq=0 out_seq=0 2024-04-25T10:10:40.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.969+0000 7f8dfbe00640 10 client.6192 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:40.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.969+0000 7f8dfbe00640 1 -- 172.21.15.119:0/4014983009 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+138 (crc 0 0 0) 0x559bd57ca020 con 0x559bd5886040 2024-04-25T10:10:40.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.969+0000 7f8dfbe00640 10 client.6192 handle_command_reply: tid=0 2024-04-25T10:10:40.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.969+0000 7f8e1cd8f740 1 client.6192 shutdown 2024-04-25T10:10:40.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.969+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4014983009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd58848a0 msgr2=0x559bd58bc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.969+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/4014983009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd58848a0 0x559bd58bc7a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8e140725c0 tx=0x7f8e14071770 comp rx=0 tx=0).stop 2024-04-25T10:10:40.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.969+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4014983009 shutdown_connections 2024-04-25T10:10:40.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.969+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/4014983009 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559bd5886040 0x559bd5886420 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.969+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/4014983009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559bd58848a0 0x559bd58bc7a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.969+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/4014983009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559bd5881ee0 0x559bd58b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.969+0000 7f8e1cd8f740 1 --2- 172.21.15.119:0/4014983009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559bd5871e00 0x559bd58b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.969+0000 7f8dfbe00640 0 client.6192 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:40.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.969+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4014983009 >> 172.21.15.119:0/4014983009 conn(0x559bd5881b00 msgr2=0x559bd585a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.970+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4014983009 shutdown_connections 2024-04-25T10:10:40.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.970+0000 7f8e1cd8f740 1 -- 172.21.15.119:0/4014983009 wait complete. 2024-04-25T10:10:40.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.970+0000 7f8e1cd8f740 20 client.6192 trim_cache size 0 max 16384 2024-04-25T10:10:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:10:40.981 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2024-04-25T10:10:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:10:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2024-04-25T10:10:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 8809705, 2024-04-25T10:10:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 1099511627777, 2024-04-25T10:10:40.981 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/dir_test/file" 2024-04-25T10:10:40.982 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:10:40.982 INFO:teuthology.orchestra.run.smithi119.stdout:] 2024-04-25T10:10:40.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.970+0000 7f8e18a00640 1 -- 172.21.15.119:0/2130649717 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8df006b4d0 msgr2=0x7f8df006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.970+0000 7f8e18a00640 1 --2- 172.21.15.119:0/2130649717 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8df006b4d0 0x7f8df006d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc032000 tx=0x7f8dfc02e000 comp rx=0 tx=0).stop 2024-04-25T10:10:40.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.970+0000 7f8e18a00640 1 -- 172.21.15.119:0/2130649717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df4004800 msgr2=0x7f8e14153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.970+0000 7f8e18a00640 1 --2- 172.21.15.119:0/2130649717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df4004800 0x7f8e14153ad0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8e04010910 tx=0x7f8e04010dd0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.971+0000 7f8e18a00640 1 -- 172.21.15.119:0/2130649717 shutdown_connections 2024-04-25T10:10:40.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.971+0000 7f8e18a00640 1 --2- 172.21.15.119:0/2130649717 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8df006b4d0 0x7f8df006d990 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.971+0000 7f8e18a00640 1 --2- 172.21.15.119:0/2130649717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8e141e5190 0x7f8e141e5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.971+0000 7f8e18a00640 1 --2- 172.21.15.119:0/2130649717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e14154690 0x7f8e14154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.971+0000 7f8e18a00640 1 --2- 172.21.15.119:0/2130649717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8df4004800 0x7f8e14153ad0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.971+0000 7f8e18a00640 1 -- 172.21.15.119:0/2130649717 >> 172.21.15.119:0/2130649717 conn(0x7f8e1406aba0 msgr2=0x7f8e14150bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.971+0000 7f8e18a00640 1 -- 172.21.15.119:0/2130649717 shutdown_connections 2024-04-25T10:10:40.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:40.971+0000 7f8e18a00640 1 -- 172.21.15.119:0/2130649717 wait complete. 2024-04-25T10:10:40.995 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:10:41.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.187+0000 7f8302600640 1 Processor -- start 2024-04-25T10:10:41.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.188+0000 7f8302600640 1 -- start start 2024-04-25T10:10:41.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.189+0000 7f8302600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f82fc0aecf0 0x7f82fc0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.189+0000 7f8302600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f82fc15ed70 0x7f82fc0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.189+0000 7f8302600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82fc0712c0 con 0x7f82fc15d980 2024-04-25T10:10:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.189+0000 7f8302600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82fc071ed0 con 0x7f82fc0aecf0 2024-04-25T10:10:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.189+0000 7f8302600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82fc0715a0 con 0x7f82fc15ed70 2024-04-25T10:10:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.190+0000 7f82f2a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f82fc0aecf0 0x7f82fc0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.190+0000 7f82f2a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f82fc0aecf0 0x7f82fc0cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43558/0 (socket says 172.21.15.119:43558) 2024-04-25T10:10:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.190+0000 7f82f2a00640 1 -- 172.21.15.119:0/3819287893 learned_addr learned my addr 172.21.15.119:0/3819287893 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.190+0000 7f82fb400640 1 --1- 172.21.15.119:0/3819287893 >> v1:172.21.15.119:6789/0 conn(0x7f82fc15d980 0x7f82fc0ae5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:49448/0 (socket says 172.21.15.119:49448) 2024-04-25T10:10:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.190+0000 7f82fbe00640 1 --2- 172.21.15.119:0/3819287893 >> v2:172.21.15.132:3300/0 conn(0x7f82fc15ed70 0x7f82fc0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.190+0000 7f82f2a00640 1 -- 172.21.15.119:0/3819287893 >> v2:172.21.15.132:3300/0 conn(0x7f82fc15ed70 msgr2=0x7f82fc0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.190+0000 7f82f2a00640 1 --2- 172.21.15.119:0/3819287893 >> v2:172.21.15.132:3300/0 conn(0x7f82fc15ed70 0x7f82fc0cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.190+0000 7f82f2a00640 1 -- 172.21.15.119:0/3819287893 >> v1:172.21.15.119:6789/0 conn(0x7f82fc15d980 legacy=0x7f82fc0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.190+0000 7f82f2a00640 1 -- 172.21.15.119:0/3819287893 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc071880 con 0x7f82fc0aecf0 2024-04-25T10:10:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.190+0000 7f82faa00640 1 -- 172.21.15.119:0/3819287893 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162814172 0 0) 0x7f82fc0712c0 con 0x7f82fc15d980 2024-04-25T10:10:41.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.190+0000 7f82fbe00640 1 --2- 172.21.15.119:0/3819287893 >> v2:172.21.15.132:3300/0 conn(0x7f82fc15ed70 0x7f82fc0cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.191+0000 7f82f2a00640 1 --2- 172.21.15.119:0/3819287893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc0aecf0 0x7f82fc0cf280 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f82fc072050 tx=0x7f82e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1106bc5d5e569ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.192+0000 7f82faa00640 1 -- 172.21.15.119:0/3819287893 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e4037070 con 0x7f82fc0aecf0 2024-04-25T10:10:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.192+0000 7f82faa00640 1 -- 172.21.15.119:0/3819287893 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e4032440 con 0x7f82fc0aecf0 2024-04-25T10:10:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.192+0000 7f82faa00640 1 -- 172.21.15.119:0/3819287893 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e4032810 con 0x7f82fc0aecf0 2024-04-25T10:10:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.192+0000 7f8302600640 1 -- 172.21.15.119:0/3819287893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc0aecf0 msgr2=0x7f82fc0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.192+0000 7f8302600640 1 --2- 172.21.15.119:0/3819287893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc0aecf0 0x7f82fc0cf280 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f82fc072050 tx=0x7f82e402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.192+0000 7f8302600640 1 -- 172.21.15.119:0/3819287893 shutdown_connections 2024-04-25T10:10:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.192+0000 7f8302600640 1 --2- 172.21.15.119:0/3819287893 >> v2:172.21.15.132:3300/0 conn(0x7f82fc15ed70 0x7f82fc0cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.192+0000 7f8302600640 1 --2- 172.21.15.119:0/3819287893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc0aecf0 0x7f82fc0cf280 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.192+0000 7f8302600640 1 -- 172.21.15.119:0/3819287893 >> 172.21.15.119:0/3819287893 conn(0x7f82fc06aba0 msgr2=0x7f82fc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.192+0000 7f8302600640 1 -- 172.21.15.119:0/3819287893 shutdown_connections 2024-04-25T10:10:41.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.192+0000 7f8302600640 1 -- 172.21.15.119:0/3819287893 wait complete. 2024-04-25T10:10:41.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.194+0000 7f8302600640 1 Processor -- start 2024-04-25T10:10:41.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.194+0000 7f8302600640 1 -- start start 2024-04-25T10:10:41.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.195+0000 7f8302600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82fc15d980 0x7f82fc14ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.195+0000 7f8302600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82fc1518d0 0x7f82fc151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.195+0000 7f8302600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc150130 0x7f82fc1505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.195+0000 7f8302600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0ad0f0 con 0x7f82fc150130 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.195+0000 7f8302600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0acf70 con 0x7f82fc15d980 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.195+0000 7f8302600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0ad270 con 0x7f82fc1518d0 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.196+0000 7f82fbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc150130 0x7f82fc1505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.196+0000 7f82fbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc150130 0x7f82fc1505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43568/0 (socket says 172.21.15.119:43568) 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.196+0000 7f82fbe00640 1 -- 172.21.15.119:0/3557558641 learned_addr learned my addr 172.21.15.119:0/3557558641 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.196+0000 7f82fb400640 1 --2- 172.21.15.119:0/3557558641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82fc15d980 0x7f82fc14ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.196+0000 7f82f2a00640 1 --2- 172.21.15.119:0/3557558641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82fc1518d0 0x7f82fc151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.196+0000 7f82fbe00640 1 -- 172.21.15.119:0/3557558641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82fc1518d0 msgr2=0x7f82fc151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.196+0000 7f82fbe00640 1 --2- 172.21.15.119:0/3557558641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82fc1518d0 0x7f82fc151d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.196+0000 7f82fbe00640 1 -- 172.21.15.119:0/3557558641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82fc15d980 msgr2=0x7f82fc14ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.196+0000 7f82fbe00640 1 --2- 172.21.15.119:0/3557558641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82fc15d980 0x7f82fc14ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.196+0000 7f82fbe00640 1 -- 172.21.15.119:0/3557558641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc154130 con 0x7f82fc150130 2024-04-25T10:10:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.196+0000 7f82fbe00640 1 --2- 172.21.15.119:0/3557558641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc150130 0x7f82fc1505b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f82f400cd40 tx=0x7f82f400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.197+0000 7f82f3e00640 1 -- 172.21.15.119:0/3557558641 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4003b90 con 0x7f82fc150130 2024-04-25T10:10:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.197+0000 7f82f3e00640 1 -- 172.21.15.119:0/3557558641 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f4003d50 con 0x7f82fc150130 2024-04-25T10:10:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.197+0000 7f82f3e00640 1 -- 172.21.15.119:0/3557558641 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4004260 con 0x7f82fc150130 2024-04-25T10:10:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.197+0000 7f8302600640 1 -- 172.21.15.119:0/3557558641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc152440 con 0x7f82fc150130 2024-04-25T10:10:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.197+0000 7f8302600640 1 -- 172.21.15.119:0/3557558641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc152a00 con 0x7f82fc150130 2024-04-25T10:10:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.198+0000 7f8302600640 1 -- 172.21.15.119:0/3557558641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc152c00 con 0x7f82fc150130 2024-04-25T10:10:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.199+0000 7f82f3e00640 1 -- 172.21.15.119:0/3557558641 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82f4004400 con 0x7f82fc150130 2024-04-25T10:10:41.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.200+0000 7f82f3e00640 1 --2- 172.21.15.119:0/3557558641 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f82d806b500 0x7f82d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.200+0000 7f82f3e00640 1 -- 172.21.15.119:0/3557558641 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82f408ac30 con 0x7f82fc150130 2024-04-25T10:10:41.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.200+0000 7f82fb400640 1 --2- 172.21.15.119:0/3557558641 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f82d806b500 0x7f82d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.201+0000 7f82fb400640 1 --2- 172.21.15.119:0/3557558641 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f82d806b500 0x7f82d806d9c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f82ec000f60 tx=0x7f82ec007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.205+0000 7f82f3e00640 1 -- 172.21.15.119:0/3557558641 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82fc154130 con 0x7f82fc150130 2024-04-25T10:10:41.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.432+0000 7f8302600640 1 -- 172.21.15.119:0/3557558641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f82fc0749b0 con 0x7f82fc150130 2024-04-25T10:10:41.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.432+0000 7f82f3e00640 1 -- 172.21.15.119:0/3557558641 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f82f4014030 con 0x7f82fc150130 2024-04-25T10:10:41.438 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:10:41.438 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:10:41.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.435+0000 7f8302600640 1 -- 172.21.15.119:0/3557558641 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f82d806b500 msgr2=0x7f82d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.435+0000 7f8302600640 1 --2- 172.21.15.119:0/3557558641 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f82d806b500 0x7f82d806d9c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f82ec000f60 tx=0x7f82ec007b50 comp rx=0 tx=0).stop 2024-04-25T10:10:41.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.435+0000 7f8302600640 1 -- 172.21.15.119:0/3557558641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc150130 msgr2=0x7f82fc1505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.435+0000 7f8302600640 1 --2- 172.21.15.119:0/3557558641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc150130 0x7f82fc1505b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f82f400cd40 tx=0x7f82f400a650 comp rx=0 tx=0).stop 2024-04-25T10:10:41.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.435+0000 7f8302600640 1 -- 172.21.15.119:0/3557558641 shutdown_connections 2024-04-25T10:10:41.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.435+0000 7f8302600640 1 --2- 172.21.15.119:0/3557558641 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f82d806b500 0x7f82d806d9c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.435+0000 7f8302600640 1 --2- 172.21.15.119:0/3557558641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82fc150130 0x7f82fc1505b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.435+0000 7f8302600640 1 --2- 172.21.15.119:0/3557558641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f82fc1518d0 0x7f82fc151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.435+0000 7f8302600640 1 --2- 172.21.15.119:0/3557558641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82fc15d980 0x7f82fc14ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.435+0000 7f8302600640 1 -- 172.21.15.119:0/3557558641 >> 172.21.15.119:0/3557558641 conn(0x7f82fc06aba0 msgr2=0x7f82fc15f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.435+0000 7f8302600640 1 -- 172.21.15.119:0/3557558641 shutdown_connections 2024-04-25T10:10:41.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.435+0000 7f8302600640 1 -- 172.21.15.119:0/3557558641 wait complete. 2024-04-25T10:10:41.459 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:10:41.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:41.459 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-25T10:10:41.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.652+0000 7f1fba200640 1 Processor -- start 2024-04-25T10:10:41.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.652+0000 7f1fba200640 1 -- start start 2024-04-25T10:10:41.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.654+0000 7f1fba200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1fb415d9c0 0x7f1fb40ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.654+0000 7f1fba200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb40b2a40 0x7f1fb40ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.654+0000 7f1fba200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb40b6b00 con 0x7f1fb40b2480 2024-04-25T10:10:41.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.654+0000 7f1fba200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb40cf020 con 0x7f1fb415d9c0 2024-04-25T10:10:41.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.654+0000 7f1fba200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40b7110 con 0x7f1fb40b2a40 2024-04-25T10:10:41.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.654+0000 7f1fb9800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1fb40b2480 0x7f1fb40b75a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:49456/0 (socket says 172.21.15.119:49456) 2024-04-25T10:10:41.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.654+0000 7f1fb9800640 1 -- 172.21.15.119:0/4215377683 learned_addr learned my addr 172.21.15.119:0/4215377683 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:10:41.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 -- 172.21.15.119:0/4215377683 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210399973 0 0) 0x7f1fb40b6b00 con 0x7f1fb40b2480 2024-04-25T10:10:41.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 -- 172.21.15.119:0/4215377683 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f98003520 con 0x7f1fb40b2480 2024-04-25T10:10:41.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fabe00640 1 --2- 172.21.15.119:0/4215377683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb40b2a40 0x7f1fb40ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb8e00640 1 --2- 172.21.15.119:0/4215377683 >> v2:172.21.15.132:3300/0 conn(0x7f1fb415d9c0 0x7f1fb40ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 -- 172.21.15.119:0/4215377683 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 767099669 0 0) 0x7f1f98003520 con 0x7f1fb40b2480 2024-04-25T10:10:41.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 -- 172.21.15.119:0/4215377683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb40b2a40 msgr2=0x7f1fb40ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 --2- 172.21.15.119:0/4215377683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb40b2a40 0x7f1fb40ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 -- 172.21.15.119:0/4215377683 >> v2:172.21.15.132:3300/0 conn(0x7f1fb415d9c0 msgr2=0x7f1fb40ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 --2- 172.21.15.119:0/4215377683 >> v2:172.21.15.132:3300/0 conn(0x7f1fb415d9c0 0x7f1fb40ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 -- 172.21.15.119:0/4215377683 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb40b9530 con 0x7f1fb40b2480 2024-04-25T10:10:41.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 -- 172.21.15.119:0/4215377683 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f1fac0019f0 con 0x7f1fb40b2480 2024-04-25T10:10:41.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 -- 172.21.15.119:0/4215377683 >> v1:172.21.15.119:6789/0 conn(0x7f1fb40b2480 legacy=0x7f1fb40b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 --2- 172.21.15.119:0/4215377683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb40b2a40 0x7f1f980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 --2- 172.21.15.119:0/4215377683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f98004030 0x7f1f98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 --2- 172.21.15.119:0/4215377683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f98004d40 0x7f1f980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 -- 172.21.15.119:0/4215377683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40cf020 con 0x7f1f98004030 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 -- 172.21.15.119:0/4215377683 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9800baa0 con 0x7f1f98004d40 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fb3e00640 1 -- 172.21.15.119:0/4215377683 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9800bca0 con 0x7f1fb40b2a40 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.655+0000 7f1fabe00640 1 --2- 172.21.15.119:0/4215377683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f98004030 0x7f1f98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.656+0000 7f1fb8e00640 1 --2- 172.21.15.119:0/4215377683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb40b2a40 0x7f1f980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.656+0000 7f1fabe00640 1 -- 172.21.15.119:0/4215377683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb40b2a40 msgr2=0x7f1f980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.656+0000 7f1fabe00640 1 --2- 172.21.15.119:0/4215377683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb40b2a40 0x7f1f980039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.656+0000 7f1fabe00640 1 -- 172.21.15.119:0/4215377683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f98004d40 msgr2=0x7f1f980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.656+0000 7f1fabe00640 1 --2- 172.21.15.119:0/4215377683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f98004d40 0x7f1f980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.656+0000 7f1fabe00640 1 -- 172.21.15.119:0/4215377683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98003520 con 0x7f1f98004030 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.657+0000 7f1fabe00640 1 --2- 172.21.15.119:0/4215377683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f98004030 0x7f1f98004800 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00ba00 tx=0x7f1f98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edcdd8e11396f02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.657+0000 7f1fb3e00640 1 -- 172.21.15.119:0/4215377683 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c030070 con 0x7f1f98004030 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.657+0000 7f1fb3e00640 1 -- 172.21.15.119:0/4215377683 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c02adb0 con 0x7f1f98004030 2024-04-25T10:10:41.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.657+0000 7f1fb3e00640 1 -- 172.21.15.119:0/4215377683 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c00b600 con 0x7f1f98004030 2024-04-25T10:10:41.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.657+0000 7f1fba200640 1 -- 172.21.15.119:0/4215377683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f98004030 msgr2=0x7f1f98004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.657+0000 7f1fba200640 1 --2- 172.21.15.119:0/4215377683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f98004030 0x7f1f98004800 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00ba00 tx=0x7f1f98005830 comp rx=0 tx=0).stop 2024-04-25T10:10:41.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.658+0000 7f1fb9800640 1 -- 172.21.15.119:0/4215377683 reap_dead start 2024-04-25T10:10:41.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.658+0000 7f1fba200640 1 -- 172.21.15.119:0/4215377683 shutdown_connections 2024-04-25T10:10:41.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.658+0000 7f1fba200640 1 -- 172.21.15.119:0/4215377683 >> 172.21.15.119:0/4215377683 conn(0x7f1fb40726c0 msgr2=0x7f1fb4072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.658+0000 7f1fba200640 1 -- 172.21.15.119:0/4215377683 shutdown_connections 2024-04-25T10:10:41.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.658+0000 7f1fba200640 1 -- 172.21.15.119:0/4215377683 wait complete. 2024-04-25T10:10:41.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.659+0000 7f1fba200640 1 Processor -- start 2024-04-25T10:10:41.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.660+0000 7f1fba200640 1 -- start start 2024-04-25T10:10:41.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.660+0000 7f1fba200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f98004030 0x7f1fb41469e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.660+0000 7f1fba200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fb41601b0 0x7f1fb40cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.660+0000 7f1fba200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb4160630 0x7f1fb4145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.660+0000 7f1fba200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb415dda0 con 0x7f1f98004030 2024-04-25T10:10:41.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.660+0000 7f1fba200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40b9810 con 0x7f1fb41601b0 2024-04-25T10:10:41.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.660+0000 7f1fba200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40b9150 con 0x7f1fb4160630 2024-04-25T10:10:41.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.661+0000 7f1fb8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f98004030 0x7f1fb41469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.661+0000 7f1fb8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f98004030 0x7f1fb41469e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43596/0 (socket says 172.21.15.119:43596) 2024-04-25T10:10:41.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.661+0000 7f1fb8e00640 1 -- 172.21.15.119:0/579184448 learned_addr learned my addr 172.21.15.119:0/579184448 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:41.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.661+0000 7f1fb8e00640 1 -- 172.21.15.119:0/579184448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb4160630 msgr2=0x7f1fb4145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.661+0000 7f1fb8e00640 1 --2- 172.21.15.119:0/579184448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb4160630 0x7f1fb4145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.661+0000 7f1fb8e00640 1 -- 172.21.15.119:0/579184448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fb41601b0 msgr2=0x7f1fb40cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.661+0000 7f1fb8e00640 1 --2- 172.21.15.119:0/579184448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fb41601b0 0x7f1fb40cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.661+0000 7f1fb8e00640 1 -- 172.21.15.119:0/579184448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb415f110 con 0x7f1f98004030 2024-04-25T10:10:41.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.661+0000 7f1fb8e00640 1 --2- 172.21.15.119:0/579184448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f98004030 0x7f1fb41469e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400e7e0 tx=0x7f1fa400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.662+0000 7f1fb1600640 1 -- 172.21.15.119:0/579184448 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4017070 con 0x7f1f98004030 2024-04-25T10:10:41.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.662+0000 7f1fb1600640 1 -- 172.21.15.119:0/579184448 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa400c6d0 con 0x7f1f98004030 2024-04-25T10:10:41.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.662+0000 7f1fba200640 1 -- 172.21.15.119:0/579184448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb415f3f0 con 0x7f1f98004030 2024-04-25T10:10:41.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.662+0000 7f1fb1600640 1 -- 172.21.15.119:0/579184448 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa400f040 con 0x7f1f98004030 2024-04-25T10:10:41.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.662+0000 7f1fba200640 1 -- 172.21.15.119:0/579184448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb415f9b0 con 0x7f1f98004030 2024-04-25T10:10:41.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.664+0000 7f1fb1600640 1 -- 172.21.15.119:0/579184448 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1fa401a070 con 0x7f1f98004030 2024-04-25T10:10:41.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.665+0000 7f1fb1600640 1 --2- 172.21.15.119:0/579184448 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f9006b500 0x7f1f9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.665+0000 7f1fb1600640 1 -- 172.21.15.119:0/579184448 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fa4089e40 con 0x7f1f98004030 2024-04-25T10:10:41.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.665+0000 7f1fabe00640 1 --2- 172.21.15.119:0/579184448 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f9006b500 0x7f1f9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.665+0000 7f1fbe78e740 1 Processor -- start 2024-04-25T10:10:41.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.666+0000 7f1fabe00640 1 --2- 172.21.15.119:0/579184448 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f9006b500 0x7f1f9006d9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40b2e80 tx=0x7f1f9c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.666+0000 7f1fbe78e740 1 -- start start 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.666+0000 7f1fbe78e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b62e40 0x564eb3b157d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.666+0000 7f1fbe78e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b45f60 0x564eb3b452e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.666+0000 7f1fbe78e740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b45820 0x564eb3b1a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.666+0000 7f1fbe78e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3a7e9f0 con 0x564eb3b45f60 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.666+0000 7f1fbe78e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3a27eb0 con 0x564eb3b62e40 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.666+0000 7f1fbe78e740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564eb3a7aa90 con 0x564eb3b45820 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.667+0000 7f1fabe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b45f60 0x564eb3b452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.667+0000 7f1fabe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b45f60 0x564eb3b452e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43612/0 (socket says 172.21.15.119:43612) 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.667+0000 7f1fabe00640 1 -- 172.21.15.119:0/3400454147 learned_addr learned my addr 172.21.15.119:0/3400454147 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.667+0000 7f1fb9800640 1 --2- 172.21.15.119:0/3400454147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b45820 0x564eb3b1a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.667+0000 7f1fb8e00640 1 --2- 172.21.15.119:0/3400454147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b62e40 0x564eb3b157d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.667+0000 7f1fabe00640 1 -- 172.21.15.119:0/3400454147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b45820 msgr2=0x564eb3b1a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.667+0000 7f1fabe00640 1 --2- 172.21.15.119:0/3400454147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b45820 0x564eb3b1a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.667+0000 7f1fabe00640 1 -- 172.21.15.119:0/3400454147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b62e40 msgr2=0x564eb3b157d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.667+0000 7f1fabe00640 1 --2- 172.21.15.119:0/3400454147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b62e40 0x564eb3b157d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.667+0000 7f1fabe00640 1 -- 172.21.15.119:0/3400454147 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eb385f940 con 0x564eb3b45f60 2024-04-25T10:10:41.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.667+0000 7f1fabe00640 1 --2- 172.21.15.119:0/3400454147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b45f60 0x564eb3b452e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00b200 tx=0x7f1f9c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b732e88401b58b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.668+0000 7f1fa9600640 1 -- 172.21.15.119:0/3400454147 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c03d940 con 0x564eb3b45f60 2024-04-25T10:10:41.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.668+0000 7f1fa9600640 1 -- 172.21.15.119:0/3400454147 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c04d070 con 0x564eb3b45f60 2024-04-25T10:10:41.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.668+0000 7f1fa9600640 1 -- 172.21.15.119:0/3400454147 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c03dce0 con 0x564eb3b45f60 2024-04-25T10:10:41.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.668+0000 7f1fbe78e740 1 -- 172.21.15.119:0/3400454147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b45f60 msgr2=0x564eb3b452e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.668+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/3400454147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b45f60 0x564eb3b452e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00b200 tx=0x7f1f9c02e000 comp rx=0 tx=0).stop 2024-04-25T10:10:41.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.668+0000 7f1fbe78e740 1 -- 172.21.15.119:0/3400454147 shutdown_connections 2024-04-25T10:10:41.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.668+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/3400454147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b45820 0x564eb3b1a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.668+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/3400454147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b45f60 0x564eb3b452e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.668+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/3400454147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b62e40 0x564eb3b157d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.668+0000 7f1fbe78e740 1 -- 172.21.15.119:0/3400454147 >> 172.21.15.119:0/3400454147 conn(0x564eb3b17a40 msgr2=0x564eb3b602e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.669+0000 7f1fbe78e740 1 -- 172.21.15.119:0/3400454147 shutdown_connections 2024-04-25T10:10:41.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.669+0000 7f1fbe78e740 1 -- 172.21.15.119:0/3400454147 wait complete. 2024-04-25T10:10:41.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.669+0000 7f1fbe78e740 1 -- start start 2024-04-25T10:10:41.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.669+0000 7f1fbe78e740 1 Processor -- start 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.670+0000 7f1fbe78e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b45f60 0x564eb3b606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.670+0000 7f1fbe78e740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b62e40 0x564eb3b61c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.670+0000 7f1fbe78e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b72960 0x564eb3b72de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.670+0000 7f1fbe78e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3b19bf0 con 0x564eb3b45f60 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.670+0000 7f1fbe78e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3a1d370 con 0x564eb3b72960 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.670+0000 7f1fbe78e740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564eb3b18fe0 con 0x564eb3b62e40 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.670+0000 7f1fbe78e740 10 client.0 _ll_register_callbacks cb 0x564eb3792d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:41.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.671+0000 7f1fb8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b45f60 0x564eb3b606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.671+0000 7f1fb8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b45f60 0x564eb3b606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43626/0 (socket says 172.21.15.119:43626) 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.671+0000 7f1fb8e00640 1 -- 172.21.15.119:0/41429357 learned_addr learned my addr 172.21.15.119:0/41429357 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.671+0000 7f1fabe00640 1 --2- 172.21.15.119:0/41429357 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b62e40 0x564eb3b61c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.671+0000 7f1fb8e00640 1 -- 172.21.15.119:0/41429357 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b62e40 msgr2=0x564eb3b61c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.671+0000 7f1fb8e00640 1 --2- 172.21.15.119:0/41429357 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b62e40 0x564eb3b61c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.671+0000 7f1fb8e00640 1 -- 172.21.15.119:0/41429357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b72960 msgr2=0x564eb3b72de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.671+0000 7f1fb8e00640 1 --2- 172.21.15.119:0/41429357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b72960 0x564eb3b72de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.671+0000 7f1fb8e00640 1 -- 172.21.15.119:0/41429357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564eb38c00d0 con 0x564eb3b45f60 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.671+0000 7f1fb8e00640 1 --2- 172.21.15.119:0/41429357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b45f60 0x564eb3b606e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4088270 tx=0x7f1fa405c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26641197cdee46f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.671+0000 7f1fa2000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.671+0000 7f1fa2000640 1 -- 172.21.15.119:0/41429357 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4021040 con 0x564eb3b45f60 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.671+0000 7f1fa2000640 1 -- 172.21.15.119:0/41429357 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fa405c370 con 0x564eb3b45f60 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.672+0000 7f1fbe78e740 1 -- 172.21.15.119:0/41429357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f9c04d070 con 0x564eb3b45f60 2024-04-25T10:10:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.672+0000 7f1fa2000640 1 -- 172.21.15.119:0/41429357 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fa4057970 con 0x564eb3b45f60 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.672+0000 7f1fbe78e740 10 client.6213 fetch_fsmap learned FSMap version 35 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.672+0000 7f1fbe78e740 1 -- 172.21.15.119:0/41429357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x564eb3753230 con 0x564eb3b45f60 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.672+0000 7f1fa2000640 1 -- 172.21.15.119:0/41429357 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c04d070 con 0x564eb3b45f60 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.672+0000 7f1fa2000640 1 -- 172.21.15.119:0/41429357 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1fa4012050 con 0x564eb3b45f60 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.672+0000 7f1fbe78e740 10 client.6213 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.672+0000 7f1fbe78e740 10 client.6213 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.672+0000 7f1fbe78e740 20 client.6213 populate_metadata read hostname 'smithi119' 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.672+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/41429357 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x564eb3b47970 0x564eb3b47de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.672+0000 7f1fbe78e740 4 client.6213 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.672+0000 7f1fbe78e740 1 -- 172.21.15.119:0/41429357 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564eb3a8bf30 con 0x564eb3b47970 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.672+0000 7f1fabe00640 1 --2- 172.21.15.119:0/41429357 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x564eb3b47970 0x564eb3b47de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.673+0000 7f1fa2000640 10 client.6213 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.673+0000 7f1fabe00640 1 --2- 172.21.15.119:0/41429357 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x564eb3b47970 0x564eb3b47de0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=542078baef2bf3f2 server_cookie=efabd7d3b721659d in_seq=0 out_seq=0 2024-04-25T10:10:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.673+0000 7f1fa2000640 1 -- 172.21.15.119:0/41429357 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564eb3a8bf30 con 0x564eb3b47970 2024-04-25T10:10:41.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.673+0000 7f1fa2000640 10 client.6213 handle_command_reply: tid=0 2024-04-25T10:10:41.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.674+0000 7f1fbe78e740 1 client.6213 shutdown 2024-04-25T10:10:41.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.674+0000 7f1fbe78e740 1 -- 172.21.15.119:0/41429357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b45f60 msgr2=0x564eb3b606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.674+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/41429357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b45f60 0x564eb3b606e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4088270 tx=0x7f1fa405c670 comp rx=0 tx=0).stop 2024-04-25T10:10:41.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.674+0000 7f1fbe78e740 1 -- 172.21.15.119:0/41429357 shutdown_connections 2024-04-25T10:10:41.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.674+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/41429357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b72960 0x564eb3b72de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.674+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/41429357 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b62e40 0x564eb3b61c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.674+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/41429357 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x564eb3b47970 0x564eb3b47de0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.674+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/41429357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b45f60 0x564eb3b606e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.674+0000 7f1fbe78e740 1 -- 172.21.15.119:0/41429357 >> 172.21.15.119:0/41429357 conn(0x564eb3b45820 msgr2=0x564eb3b42600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.674+0000 7f1fa2000640 0 client.6213 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:41.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.674+0000 7f1fbe78e740 1 -- 172.21.15.119:0/41429357 shutdown_connections 2024-04-25T10:10:41.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.674+0000 7f1fbe78e740 1 -- 172.21.15.119:0/41429357 wait complete. 2024-04-25T10:10:41.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.674+0000 7f1fbe78e740 20 client.6213 trim_cache size 0 max 16384 2024-04-25T10:10:41.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.690+0000 7f1fbe78e740 1 Processor -- start 2024-04-25T10:10:41.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.690+0000 7f1fbe78e740 1 -- start start 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.691+0000 7f1fbe78e740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b45f60 0x564eb3b42820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.691+0000 7f1fbe78e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b62e40 0x564eb3b5f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.691+0000 7f1fbe78e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b72960 0x564eb3b415b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.691+0000 7f1fbe78e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb42003a0 con 0x564eb3b72960 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.691+0000 7f1fbe78e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb41ff520 con 0x564eb3b62e40 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.691+0000 7f1fbe78e740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb41569a0 con 0x564eb3b45f60 2024-04-25T10:10:41.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.692+0000 7f1fb9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b72960 0x564eb3b415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.692+0000 7f1fb9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b72960 0x564eb3b415b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43632/0 (socket says 172.21.15.119:43632) 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.692+0000 7f1fb9800640 1 -- 172.21.15.119:0/1908496063 learned_addr learned my addr 172.21.15.119:0/1908496063 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.692+0000 7f1fabe00640 1 --2- 172.21.15.119:0/1908496063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b62e40 0x564eb3b5f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.692+0000 7f1fb9800640 1 -- 172.21.15.119:0/1908496063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b45f60 msgr2=0x564eb3b42820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.692+0000 7f1fb9800640 1 --2- 172.21.15.119:0/1908496063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b45f60 0x564eb3b42820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.692+0000 7f1fb9800640 1 -- 172.21.15.119:0/1908496063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b62e40 msgr2=0x564eb3b5f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.692+0000 7f1fb9800640 1 --2- 172.21.15.119:0/1908496063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b62e40 0x564eb3b5f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.692+0000 7f1fb9800640 1 -- 172.21.15.119:0/1908496063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eb3b600e0 con 0x564eb3b72960 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.692+0000 7f1fb9800640 1 --2- 172.21.15.119:0/1908496063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b72960 0x564eb3b415b0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1fac001e30 tx=0x7f1fac00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d761382e01dd34b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.693+0000 7f1fa8c00640 1 -- 172.21.15.119:0/1908496063 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac004d60 con 0x564eb3b72960 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.693+0000 7f1fa8c00640 1 -- 172.21.15.119:0/1908496063 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac00db50 con 0x564eb3b72960 2024-04-25T10:10:41.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.693+0000 7f1fa8c00640 1 -- 172.21.15.119:0/1908496063 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac007110 con 0x564eb3b72960 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.693+0000 7f1fbe78e740 1 -- 172.21.15.119:0/1908496063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b72960 msgr2=0x564eb3b415b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.693+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/1908496063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b72960 0x564eb3b415b0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1fac001e30 tx=0x7f1fac00d520 comp rx=0 tx=0).stop 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.693+0000 7f1fbe78e740 1 -- 172.21.15.119:0/1908496063 shutdown_connections 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.693+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/1908496063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b72960 0x564eb3b415b0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.693+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/1908496063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b62e40 0x564eb3b5f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.693+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/1908496063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b45f60 0x564eb3b42820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.693+0000 7f1fbe78e740 1 -- 172.21.15.119:0/1908496063 >> 172.21.15.119:0/1908496063 conn(0x564eb3b45820 msgr2=0x564eb3b59930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.693+0000 7f1fbe78e740 1 -- 172.21.15.119:0/1908496063 shutdown_connections 2024-04-25T10:10:41.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.693+0000 7f1fbe78e740 1 -- 172.21.15.119:0/1908496063 wait complete. 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.694+0000 7f1fbe78e740 1 -- start start 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.694+0000 7f1fbe78e740 1 Processor -- start 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.695+0000 7f1fbe78e740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b45f60 0x564eb3b56680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.695+0000 7f1fbe78e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b62e40 0x564eb3bce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.695+0000 7f1fbe78e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b72960 0x564eb3b714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.695+0000 7f1fbe78e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3b41af0 con 0x564eb3b62e40 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.695+0000 7f1fbe78e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb41fef20 con 0x564eb3b72960 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.695+0000 7f1fbe78e740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564eb3b482e0 con 0x564eb3b45f60 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.695+0000 7f1fbe78e740 10 client.0 _ll_register_callbacks cb 0x7f1f980036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.695+0000 7f1fabe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b62e40 0x564eb3bce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.695+0000 7f1fabe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b62e40 0x564eb3bce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43644/0 (socket says 172.21.15.119:43644) 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.695+0000 7f1fabe00640 1 -- 172.21.15.119:0/171491159 learned_addr learned my addr 172.21.15.119:0/171491159 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.696+0000 7f1fabe00640 1 -- 172.21.15.119:0/171491159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b45f60 msgr2=0x564eb3b56680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.696+0000 7f1fabe00640 1 --2- 172.21.15.119:0/171491159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b45f60 0x564eb3b56680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.696+0000 7f1fabe00640 1 -- 172.21.15.119:0/171491159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b72960 msgr2=0x564eb3b714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.696+0000 7f1fabe00640 1 --2- 172.21.15.119:0/171491159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b72960 0x564eb3b714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.696+0000 7f1fabe00640 1 -- 172.21.15.119:0/171491159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564eb3b61410 con 0x564eb3b62e40 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.696+0000 7f1fabe00640 1 --2- 172.21.15.119:0/171491159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b62e40 0x564eb3bce590 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c02a8a0 tx=0x7f1f9c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f308e15a4be63f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.696+0000 7f1fa2000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.696+0000 7f1fa2000640 1 -- 172.21.15.119:0/171491159 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c05d060 con 0x564eb3b62e40 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.696+0000 7f1fa2000640 1 -- 172.21.15.119:0/171491159 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f9c057d00 con 0x564eb3b62e40 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.696+0000 7f1fbe78e740 1 -- 172.21.15.119:0/171491159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fac00db50 con 0x564eb3b62e40 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.697+0000 7f1fa2000640 1 -- 172.21.15.119:0/171491159 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f9c00e000 con 0x564eb3b62e40 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.697+0000 7f1fbe78e740 10 client.6219 fetch_fsmap learned FSMap version 35 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.697+0000 7f1fbe78e740 1 -- 172.21.15.119:0/171491159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x564eb3b5f900 con 0x564eb3b62e40 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.697+0000 7f1fa2000640 1 -- 172.21.15.119:0/171491159 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac00db50 con 0x564eb3b62e40 2024-04-25T10:10:41.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.697+0000 7f1fa2000640 1 -- 172.21.15.119:0/171491159 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1f9c03dac0 con 0x564eb3b62e40 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.697+0000 7f1fbe78e740 10 client.6219 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.697+0000 7f1fbe78e740 10 client.6219 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.697+0000 7f1fbe78e740 20 client.6219 populate_metadata read hostname 'smithi119' 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.697+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/171491159 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x564eb3b7a050 0x564eb3b719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.697+0000 7f1fbe78e740 4 client.6219 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.697+0000 7f1fbe78e740 1 -- 172.21.15.119:0/171491159 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) v1 -- 0x564eb3b16e90 con 0x564eb3b7a050 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.697+0000 7f1fb8e00640 1 --2- 172.21.15.119:0/171491159 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x564eb3b7a050 0x564eb3b719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.698+0000 7f1fb8e00640 1 --2- 172.21.15.119:0/171491159 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x564eb3b7a050 0x564eb3b719e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ab045c7aac4ad2 server_cookie=fee790212db4a5ca in_seq=0 out_seq=0 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.698+0000 7f1fa2000640 10 client.6219 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.698+0000 7f1fa2000640 1 -- 172.21.15.119:0/171491159 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564eb3b16e90 con 0x564eb3b7a050 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.698+0000 7f1fa2000640 10 client.6219 handle_command_reply: tid=0 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.698+0000 7f1fbe78e740 1 client.6219 shutdown 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.699+0000 7f1fbe78e740 1 -- 172.21.15.119:0/171491159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b62e40 msgr2=0x564eb3bce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.699+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/171491159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b62e40 0x564eb3bce590 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c02a8a0 tx=0x7f1f9c02c070 comp rx=0 tx=0).stop 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.699+0000 7f1fbe78e740 1 -- 172.21.15.119:0/171491159 shutdown_connections 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.699+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/171491159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564eb3b72960 0x564eb3b714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.699+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/171491159 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x564eb3b7a050 0x564eb3b719e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.699+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/171491159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564eb3b62e40 0x564eb3bce590 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.699+0000 7f1fbe78e740 1 --2- 172.21.15.119:0/171491159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564eb3b45f60 0x564eb3b56680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.699+0000 7f1fbe78e740 1 -- 172.21.15.119:0/171491159 >> 172.21.15.119:0/171491159 conn(0x564eb3b45820 msgr2=0x564eb3b60600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.699+0000 7f1fa2000640 0 client.6219 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.699+0000 7f1fbe78e740 1 -- 172.21.15.119:0/171491159 shutdown_connections 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.699+0000 7f1fbe78e740 1 -- 172.21.15.119:0/171491159 wait complete. 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.699+0000 7f1fbe78e740 20 client.6219 trim_cache size 0 max 16384 2024-04-25T10:10:41.709 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:10:41.710 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2024-04-25T10:10:41.710 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "bda9564d-c2fc-40b3-8471-93323559c98a", 2024-04-25T10:10:41.710 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2024-04-25T10:10:41.710 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:10:41.710 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:10:41.699+0000 7efdba600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-25T10:10:08.149214+0000 1=1+0) (iversion lock) caps={4880=pAsLsXsFs/-@2},l=4880(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x561f4681db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:10:41.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.700+0000 7f1fba200640 1 -- 172.21.15.119:0/579184448 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f9006b500 msgr2=0x7f1f9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.700+0000 7f1fba200640 1 --2- 172.21.15.119:0/579184448 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f9006b500 0x7f1f9006d9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40b2e80 tx=0x7f1f9c02c000 comp rx=0 tx=0).stop 2024-04-25T10:10:41.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.700+0000 7f1fba200640 1 -- 172.21.15.119:0/579184448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f98004030 msgr2=0x7f1fb41469e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.700+0000 7f1fba200640 1 --2- 172.21.15.119:0/579184448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f98004030 0x7f1fb41469e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400e7e0 tx=0x7f1fa400eca0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.700+0000 7f1fba200640 1 -- 172.21.15.119:0/579184448 shutdown_connections 2024-04-25T10:10:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.700+0000 7f1fba200640 1 --2- 172.21.15.119:0/579184448 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f9006b500 0x7f1f9006d9c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.700+0000 7f1fba200640 1 --2- 172.21.15.119:0/579184448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1fb4160630 0x7f1fb4145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.700+0000 7f1fba200640 1 --2- 172.21.15.119:0/579184448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1fb41601b0 0x7f1fb40cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.700+0000 7f1fba200640 1 --2- 172.21.15.119:0/579184448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f98004030 0x7f1fb41469e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.700+0000 7f1fba200640 1 -- 172.21.15.119:0/579184448 >> 172.21.15.119:0/579184448 conn(0x7f1fb40726c0 msgr2=0x7f1fb40b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.700+0000 7f1fba200640 1 -- 172.21.15.119:0/579184448 shutdown_connections 2024-04-25T10:10:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.700+0000 7f1fba200640 1 -- 172.21.15.119:0/579184448 wait complete. 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:41.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:10:41.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.895+0000 7f3d2fe00640 1 Processor -- start 2024-04-25T10:10:41.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.896+0000 7f3d2fe00640 1 -- start start 2024-04-25T10:10:41.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.897+0000 7f3d2fe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3d3015bcb0 0x7f3d3006da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.897+0000 7f3d2fe00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d300712c0 con 0x7f3d300cea40 2024-04-25T10:10:41.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.897+0000 7f3d2fe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d300715a0 con 0x7f3d300c6150 2024-04-25T10:10:41.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.897+0000 7f3d2fe00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d3006e790 con 0x7f3d3015bcb0 2024-04-25T10:10:41.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.897+0000 7f3d26000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3d3015bcb0 0x7f3d3006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.897+0000 7f3d2f400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3d300cea40 0x7f3d3006df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:49462/0 (socket says 172.21.15.119:49462) 2024-04-25T10:10:41.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.897+0000 7f3d2f400640 1 -- 172.21.15.119:0/3960145442 learned_addr learned my addr 172.21.15.119:0/3960145442 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:10:41.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.898+0000 7f3d26000640 1 -- 172.21.15.119:0/3960145442 >> v1:172.21.15.132:6789/0 conn(0x7f3d300c6150 legacy=0x7f3d300ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.898+0000 7f3d26000640 1 -- 172.21.15.119:0/3960145442 >> v1:172.21.15.119:6789/0 conn(0x7f3d300cea40 legacy=0x7f3d3006df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.898+0000 7f3d2e000640 1 -- 172.21.15.119:0/3960145442 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499868087 0 0) 0x7f3d300712c0 con 0x7f3d300cea40 2024-04-25T10:10:41.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.898+0000 7f3d26000640 1 -- 172.21.15.119:0/3960145442 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d30071880 con 0x7f3d3015bcb0 2024-04-25T10:10:41.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.899+0000 7f3d26000640 1 --2- 172.21.15.119:0/3960145442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d3015bcb0 0x7f3d3006da10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3d180097a0 tx=0x7f3d1802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f376d55f2f8d1654 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.899+0000 7f3d2e000640 1 -- 172.21.15.119:0/3960145442 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18037070 con 0x7f3d3015bcb0 2024-04-25T10:10:41.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.899+0000 7f3d2e000640 1 -- 172.21.15.119:0/3960145442 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d18032440 con 0x7f3d3015bcb0 2024-04-25T10:10:41.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.899+0000 7f3d2e000640 1 -- 172.21.15.119:0/3960145442 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18032810 con 0x7f3d3015bcb0 2024-04-25T10:10:41.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.900+0000 7f3d2fe00640 1 -- 172.21.15.119:0/3960145442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d3015bcb0 msgr2=0x7f3d3006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.900+0000 7f3d2fe00640 1 --2- 172.21.15.119:0/3960145442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d3015bcb0 0x7f3d3006da10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3d180097a0 tx=0x7f3d1802a230 comp rx=0 tx=0).stop 2024-04-25T10:10:41.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.900+0000 7f3d2fe00640 1 -- 172.21.15.119:0/3960145442 shutdown_connections 2024-04-25T10:10:41.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.900+0000 7f3d2fe00640 1 --2- 172.21.15.119:0/3960145442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d3015bcb0 0x7f3d3006da10 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.900+0000 7f3d2fe00640 1 -- 172.21.15.119:0/3960145442 >> 172.21.15.119:0/3960145442 conn(0x7f3d3006aba0 msgr2=0x7f3d3006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.900+0000 7f3d2fe00640 1 -- 172.21.15.119:0/3960145442 shutdown_connections 2024-04-25T10:10:41.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.900+0000 7f3d2fe00640 1 -- 172.21.15.119:0/3960145442 wait complete. 2024-04-25T10:10:41.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.902+0000 7f3d2fe00640 1 Processor -- start 2024-04-25T10:10:41.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.902+0000 7f3d2fe00640 1 -- start start 2024-04-25T10:10:41.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.903+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d300c6150 0x7f3d30154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.903+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d300cea40 0x7f3d3013cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.903+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d3014b670 0x7f3d300cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.903+0000 7f3d2fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d30166470 con 0x7f3d3014b670 2024-04-25T10:10:41.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.903+0000 7f3d2fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d300c6530 con 0x7f3d300c6150 2024-04-25T10:10:41.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.903+0000 7f3d2fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d301665f0 con 0x7f3d300cea40 2024-04-25T10:10:41.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.903+0000 7f3d2f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d3014b670 0x7f3d300cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.903+0000 7f3d2f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d3014b670 0x7f3d300cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43664/0 (socket says 172.21.15.119:43664) 2024-04-25T10:10:41.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.903+0000 7f3d2f400640 1 -- 172.21.15.119:0/804847468 learned_addr learned my addr 172.21.15.119:0/804847468 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:41.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.904+0000 7f3d2f400640 1 -- 172.21.15.119:0/804847468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d300cea40 msgr2=0x7f3d3013cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.904+0000 7f3d26000640 1 --2- 172.21.15.119:0/804847468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d300cea40 0x7f3d3013cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.904+0000 7f3d2ea00640 1 --2- 172.21.15.119:0/804847468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d300c6150 0x7f3d30154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.904+0000 7f3d2f400640 1 --2- 172.21.15.119:0/804847468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d300cea40 0x7f3d3013cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.904+0000 7f3d2f400640 1 -- 172.21.15.119:0/804847468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d300c6150 msgr2=0x7f3d30154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.904+0000 7f3d2f400640 1 --2- 172.21.15.119:0/804847468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d300c6150 0x7f3d30154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.904+0000 7f3d2f400640 1 -- 172.21.15.119:0/804847468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d300ccaf0 con 0x7f3d3014b670 2024-04-25T10:10:41.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.904+0000 7f3d2f400640 1 --2- 172.21.15.119:0/804847468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d3014b670 0x7f3d300cdb90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800c480 tx=0x7f3d2800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.904+0000 7f3d27400640 1 -- 172.21.15.119:0/804847468 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28005ce0 con 0x7f3d3014b670 2024-04-25T10:10:41.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.904+0000 7f3d27400640 1 -- 172.21.15.119:0/804847468 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d28007070 con 0x7f3d3014b670 2024-04-25T10:10:41.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.904+0000 7f3d2fe00640 1 -- 172.21.15.119:0/804847468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d300ccdd0 con 0x7f3d3014b670 2024-04-25T10:10:41.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.904+0000 7f3d27400640 1 -- 172.21.15.119:0/804847468 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d280074e0 con 0x7f3d3014b670 2024-04-25T10:10:41.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.905+0000 7f3d2fe00640 1 -- 172.21.15.119:0/804847468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d300cd390 con 0x7f3d3014b670 2024-04-25T10:10:41.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.907+0000 7f3d27400640 1 -- 172.21.15.119:0/804847468 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d2801b400 con 0x7f3d3014b670 2024-04-25T10:10:41.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.908+0000 7f3d27400640 1 --2- 172.21.15.119:0/804847468 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d0c06b4d0 0x7f3d0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.908+0000 7f3d27400640 1 -- 172.21.15.119:0/804847468 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d28089e80 con 0x7f3d3014b670 2024-04-25T10:10:41.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.908+0000 7f3d2ea00640 1 --2- 172.21.15.119:0/804847468 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d0c06b4d0 0x7f3d0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.909+0000 7f3d2ea00640 1 --2- 172.21.15.119:0/804847468 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d0c06b4d0 0x7f3d0c06d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3d20007fd0 tx=0x7f3d20009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.909+0000 7f3d387d5740 1 Processor -- start 2024-04-25T10:10:41.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.909+0000 7f3d387d5740 1 -- start start 2024-04-25T10:10:41.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.910+0000 7f3d387d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c67932d70 0x559c67934ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.910+0000 7f3d387d5740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c6796f440 0x559c67933150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.910+0000 7f3d387d5740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6795d830 0x559c6795dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.910+0000 7f3d387d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c67896c70 con 0x559c67932d70 2024-04-25T10:10:41.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.910+0000 7f3d387d5740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6783fcf0 con 0x559c6796f440 2024-04-25T10:10:41.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.910+0000 7f3d387d5740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c67892d10 con 0x559c6795d830 2024-04-25T10:10:41.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.910+0000 7f3d2ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c67932d70 0x559c67934ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.910+0000 7f3d2ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c67932d70 0x559c67934ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43672/0 (socket says 172.21.15.119:43672) 2024-04-25T10:10:41.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.910+0000 7f3d2ea00640 1 -- 172.21.15.119:0/3294850022 learned_addr learned my addr 172.21.15.119:0/3294850022 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:41.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.911+0000 7f3d2ea00640 1 -- 172.21.15.119:0/3294850022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6795d830 msgr2=0x559c6795dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.911+0000 7f3d2ea00640 1 --2- 172.21.15.119:0/3294850022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6795d830 0x559c6795dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.911+0000 7f3d2ea00640 1 -- 172.21.15.119:0/3294850022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c6796f440 msgr2=0x559c67933150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.911+0000 7f3d2ea00640 1 --2- 172.21.15.119:0/3294850022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c6796f440 0x559c67933150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.911+0000 7f3d2ea00640 1 -- 172.21.15.119:0/3294850022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c676757d0 con 0x559c67932d70 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.911+0000 7f3d2ea00640 1 --2- 172.21.15.119:0/3294850022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c67932d70 0x559c67934ed0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3d20003e60 tx=0x7f3d20010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2155f4849f21642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.911+0000 7f3d1f400640 1 -- 172.21.15.119:0/3294850022 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2000dea0 con 0x559c67932d70 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.911+0000 7f3d1f400640 1 -- 172.21.15.119:0/3294850022 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d200055d0 con 0x559c67932d70 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.911+0000 7f3d1f400640 1 -- 172.21.15.119:0/3294850022 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d20021410 con 0x559c67932d70 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.912+0000 7f3d387d5740 1 -- 172.21.15.119:0/3294850022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c67932d70 msgr2=0x559c67934ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.912+0000 7f3d387d5740 1 --2- 172.21.15.119:0/3294850022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c67932d70 0x559c67934ed0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3d20003e60 tx=0x7f3d20010000 comp rx=0 tx=0).stop 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.912+0000 7f3d387d5740 1 -- 172.21.15.119:0/3294850022 shutdown_connections 2024-04-25T10:10:41.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.912+0000 7f3d387d5740 1 --2- 172.21.15.119:0/3294850022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6795d830 0x559c6795dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.912+0000 7f3d387d5740 1 --2- 172.21.15.119:0/3294850022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c6796f440 0x559c67933150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.912+0000 7f3d387d5740 1 --2- 172.21.15.119:0/3294850022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c67932d70 0x559c67934ed0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.912+0000 7f3d387d5740 1 -- 172.21.15.119:0/3294850022 >> 172.21.15.119:0/3294850022 conn(0x559c678faa30 msgr2=0x559c67978830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.912+0000 7f3d387d5740 1 -- 172.21.15.119:0/3294850022 shutdown_connections 2024-04-25T10:10:41.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.912+0000 7f3d387d5740 1 -- 172.21.15.119:0/3294850022 wait complete. 2024-04-25T10:10:41.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.913+0000 7f3d387d5740 1 -- start start 2024-04-25T10:10:41.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.913+0000 7f3d387d5740 1 Processor -- start 2024-04-25T10:10:41.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.914+0000 7f3d387d5740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c67932d70 0x559c6794a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.914+0000 7f3d387d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6795d830 0x559c6795f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.914+0000 7f3d387d5740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6796f440 0x559c6795fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.914+0000 7f3d387d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c678353a0 con 0x559c6795d830 2024-04-25T10:10:41.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.914+0000 7f3d387d5740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6795b380 con 0x559c67932d70 2024-04-25T10:10:41.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.914+0000 7f3d387d5740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c6794b260 con 0x559c6796f440 2024-04-25T10:10:41.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.914+0000 7f3d387d5740 10 client.0 _ll_register_callbacks cb 0x559c67852b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:41.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.914+0000 7f3d26000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6795d830 0x559c6795f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.914+0000 7f3d26000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6795d830 0x559c6795f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43676/0 (socket says 172.21.15.119:43676) 2024-04-25T10:10:41.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.914+0000 7f3d26000640 1 -- 172.21.15.119:0/265305475 learned_addr learned my addr 172.21.15.119:0/265305475 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:41.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.914+0000 7f3d26000640 1 -- 172.21.15.119:0/265305475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6796f440 msgr2=0x559c6795fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.915+0000 7f3d26000640 1 --2- 172.21.15.119:0/265305475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6796f440 0x559c6795fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.915+0000 7f3d26000640 1 -- 172.21.15.119:0/265305475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c67932d70 msgr2=0x559c6794a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.915+0000 7f3d26000640 1 --2- 172.21.15.119:0/265305475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c67932d70 0x559c6794a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.915+0000 7f3d26000640 1 -- 172.21.15.119:0/265305475 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c675ab3f0 con 0x559c6795d830 2024-04-25T10:10:41.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.915+0000 7f3d26000640 1 --2- 172.21.15.119:0/265305475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6795d830 0x559c6795f930 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009890 tx=0x7f3d180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3853e2673790b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.915+0000 7f3d13e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.915+0000 7f3d13e00640 1 -- 172.21.15.119:0/265305475 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18037040 con 0x559c6795d830 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.915+0000 7f3d13e00640 1 -- 172.21.15.119:0/265305475 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d1803b630 con 0x559c6795d830 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.915+0000 7f3d387d5740 1 -- 172.21.15.119:0/265305475 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d200055d0 con 0x559c6795d830 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.915+0000 7f3d13e00640 1 -- 172.21.15.119:0/265305475 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d18048050 con 0x559c6795d830 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.915+0000 7f3d387d5740 10 client.6234 fetch_fsmap learned FSMap version 35 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.915+0000 7f3d387d5740 1 -- 172.21.15.119:0/265305475 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x559c6756b0f0 con 0x559c6795d830 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.916+0000 7f3d13e00640 1 -- 172.21.15.119:0/265305475 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d200055d0 con 0x559c6795d830 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.916+0000 7f3d13e00640 1 -- 172.21.15.119:0/265305475 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3d1804b020 con 0x559c6795d830 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.916+0000 7f3d387d5740 10 client.6234 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.916+0000 7f3d387d5740 10 client.6234 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.916+0000 7f3d387d5740 20 client.6234 populate_metadata read hostname 'smithi119' 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.916+0000 7f3d387d5740 1 --2- 172.21.15.119:0/265305475 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559c6797ac40 0x559c6797c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.916+0000 7f3d387d5740 4 client.6234 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:41.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.916+0000 7f3d387d5740 1 -- 172.21.15.119:0/265305475 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c678a41b0 con 0x559c6797ac40 2024-04-25T10:10:41.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.916+0000 7f3d2ea00640 1 --2- 172.21.15.119:0/265305475 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559c6797ac40 0x559c6797c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.917+0000 7f3d13e00640 10 client.6234 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:41.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.917+0000 7f3d2ea00640 1 --2- 172.21.15.119:0/265305475 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559c6797ac40 0x559c6797c240 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f1744d0e52754d server_cookie=139fd87477f884f5 in_seq=0 out_seq=0 2024-04-25T10:10:41.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.917+0000 7f3d13e00640 1 -- 172.21.15.119:0/265305475 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559c678a41b0 con 0x559c6797ac40 2024-04-25T10:10:41.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.917+0000 7f3d13e00640 10 client.6234 handle_command_reply: tid=0 2024-04-25T10:10:41.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.917+0000 7f3d387d5740 1 client.6234 shutdown 2024-04-25T10:10:41.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.918+0000 7f3d387d5740 1 -- 172.21.15.119:0/265305475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6795d830 msgr2=0x559c6795f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.918+0000 7f3d387d5740 1 --2- 172.21.15.119:0/265305475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6795d830 0x559c6795f930 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009890 tx=0x7f3d180046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.918+0000 7f3d387d5740 1 -- 172.21.15.119:0/265305475 shutdown_connections 2024-04-25T10:10:41.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.918+0000 7f3d387d5740 1 --2- 172.21.15.119:0/265305475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6796f440 0x559c6795fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.918+0000 7f3d387d5740 1 --2- 172.21.15.119:0/265305475 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559c6797ac40 0x559c6797c240 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.918+0000 7f3d387d5740 1 --2- 172.21.15.119:0/265305475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6795d830 0x559c6795f930 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.918+0000 7f3d387d5740 1 --2- 172.21.15.119:0/265305475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c67932d70 0x559c6794a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.918+0000 7f3d13e00640 0 client.6234 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:41.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.918+0000 7f3d387d5740 1 -- 172.21.15.119:0/265305475 >> 172.21.15.119:0/265305475 conn(0x559c678faa30 msgr2=0x559c6795b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.918+0000 7f3d387d5740 1 -- 172.21.15.119:0/265305475 shutdown_connections 2024-04-25T10:10:41.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.918+0000 7f3d387d5740 1 -- 172.21.15.119:0/265305475 wait complete. 2024-04-25T10:10:41.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.918+0000 7f3d387d5740 20 client.6234 trim_cache size 0 max 16384 2024-04-25T10:10:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.934+0000 7f3d387d5740 1 Processor -- start 2024-04-25T10:10:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.934+0000 7f3d387d5740 1 -- start start 2024-04-25T10:10:41.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.935+0000 7f3d387d5740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c67932d70 0x559c67991e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.935+0000 7f3d387d5740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6795d830 0x559c6794a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.935+0000 7f3d387d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6796f440 0x559c6797c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.935+0000 7f3d387d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d301ffbf0 con 0x559c6796f440 2024-04-25T10:10:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.935+0000 7f3d387d5740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d301fee20 con 0x559c67932d70 2024-04-25T10:10:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.935+0000 7f3d387d5740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3014ca80 con 0x559c6795d830 2024-04-25T10:10:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.936+0000 7f3d2f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6796f440 0x559c6797c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.936+0000 7f3d2f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6796f440 0x559c6797c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43692/0 (socket says 172.21.15.119:43692) 2024-04-25T10:10:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.936+0000 7f3d2f400640 1 -- 172.21.15.119:0/1300994541 learned_addr learned my addr 172.21.15.119:0/1300994541 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.936+0000 7f3d2ea00640 1 --2- 172.21.15.119:0/1300994541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c67932d70 0x559c67991e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.936+0000 7f3d26000640 1 --2- 172.21.15.119:0/1300994541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6795d830 0x559c6794a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.936+0000 7f3d2f400640 1 -- 172.21.15.119:0/1300994541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6795d830 msgr2=0x559c6794a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.936+0000 7f3d2f400640 1 --2- 172.21.15.119:0/1300994541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6795d830 0x559c6794a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.936+0000 7f3d2f400640 1 -- 172.21.15.119:0/1300994541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c67932d70 msgr2=0x559c67991e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.936+0000 7f3d2f400640 1 --2- 172.21.15.119:0/1300994541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c67932d70 0x559c67991e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.936+0000 7f3d2f400640 1 -- 172.21.15.119:0/1300994541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c679923c0 con 0x559c6796f440 2024-04-25T10:10:41.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.936+0000 7f3d2f400640 1 --2- 172.21.15.119:0/1300994541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6796f440 0x559c6797c0e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3d280582f0 tx=0x7f3d28007f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8ff9ee1885658c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.937+0000 7f3d1ea00640 1 -- 172.21.15.119:0/1300994541 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28070020 con 0x559c6796f440 2024-04-25T10:10:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.937+0000 7f3d1ea00640 1 -- 172.21.15.119:0/1300994541 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d28060d00 con 0x559c6796f440 2024-04-25T10:10:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.937+0000 7f3d1ea00640 1 -- 172.21.15.119:0/1300994541 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d280568f0 con 0x559c6796f440 2024-04-25T10:10:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.937+0000 7f3d387d5740 1 -- 172.21.15.119:0/1300994541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6796f440 msgr2=0x559c6797c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.937+0000 7f3d387d5740 1 --2- 172.21.15.119:0/1300994541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6796f440 0x559c6797c0e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3d280582f0 tx=0x7f3d28007f80 comp rx=0 tx=0).stop 2024-04-25T10:10:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.937+0000 7f3d387d5740 1 -- 172.21.15.119:0/1300994541 shutdown_connections 2024-04-25T10:10:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.937+0000 7f3d387d5740 1 --2- 172.21.15.119:0/1300994541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6796f440 0x559c6797c0e0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.937+0000 7f3d387d5740 1 --2- 172.21.15.119:0/1300994541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6795d830 0x559c6794a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.937+0000 7f3d387d5740 1 --2- 172.21.15.119:0/1300994541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c67932d70 0x559c67991e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.937+0000 7f3d387d5740 1 -- 172.21.15.119:0/1300994541 >> 172.21.15.119:0/1300994541 conn(0x559c678faa30 msgr2=0x559c6792cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.937+0000 7f3d387d5740 1 -- 172.21.15.119:0/1300994541 shutdown_connections 2024-04-25T10:10:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.937+0000 7f3d387d5740 1 -- 172.21.15.119:0/1300994541 wait complete. 2024-04-25T10:10:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.938+0000 7f3d387d5740 1 -- start start 2024-04-25T10:10:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.938+0000 7f3d387d5740 1 Processor -- start 2024-04-25T10:10:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.939+0000 7f3d387d5740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c67932d70 0x559c679ef010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.939+0000 7f3d387d5740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6795d830 0x559c679610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d387d5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6796f440 0x559c67961600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d387d5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6795fef0 con 0x559c6796f440 2024-04-25T10:10:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d387d5740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d300b4d80 con 0x559c67932d70 2024-04-25T10:10:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d387d5740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c6792d7b0 con 0x559c6795d830 2024-04-25T10:10:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d387d5740 10 client.0 _ll_register_callbacks cb 0x7f3d18035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d2f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6796f440 0x559c67961600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d2f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6796f440 0x559c67961600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43700/0 (socket says 172.21.15.119:43700) 2024-04-25T10:10:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d2f400640 1 -- 172.21.15.119:0/1129225226 learned_addr learned my addr 172.21.15.119:0/1129225226 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:10:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d2ea00640 1 --2- 172.21.15.119:0/1129225226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c67932d70 0x559c679ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d26000640 1 --2- 172.21.15.119:0/1129225226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6795d830 0x559c679610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d2f400640 1 -- 172.21.15.119:0/1129225226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6795d830 msgr2=0x559c679610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d2f400640 1 --2- 172.21.15.119:0/1129225226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6795d830 0x559c679610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d2f400640 1 -- 172.21.15.119:0/1129225226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c67932d70 msgr2=0x559c679ef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d2f400640 1 --2- 172.21.15.119:0/1129225226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c67932d70 0x559c679ef010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d2f400640 1 -- 172.21.15.119:0/1129225226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c6798b480 con 0x559c6796f440 2024-04-25T10:10:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d2f400640 1 --2- 172.21.15.119:0/1129225226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6796f440 0x559c67961600 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3d280582f0 tx=0x7f3d28005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac505a98edbc9cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.940+0000 7f3d13e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:10:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.941+0000 7f3d13e00640 1 -- 172.21.15.119:0/1129225226 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28076060 con 0x559c6796f440 2024-04-25T10:10:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.941+0000 7f3d13e00640 1 -- 172.21.15.119:0/1129225226 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d28070020 con 0x559c6796f440 2024-04-25T10:10:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.941+0000 7f3d387d5740 1 -- 172.21.15.119:0/1129225226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d28060d00 con 0x559c6796f440 2024-04-25T10:10:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.941+0000 7f3d13e00640 1 -- 172.21.15.119:0/1129225226 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d28058850 con 0x559c6796f440 2024-04-25T10:10:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.941+0000 7f3d387d5740 10 client.6240 fetch_fsmap learned FSMap version 35 2024-04-25T10:10:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.941+0000 7f3d387d5740 1 -- 172.21.15.119:0/1129225226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x559c679eb3c0 con 0x559c6796f440 2024-04-25T10:10:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.941+0000 7f3d13e00640 1 -- 172.21.15.119:0/1129225226 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d28060d00 con 0x559c6796f440 2024-04-25T10:10:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.941+0000 7f3d13e00640 1 -- 172.21.15.119:0/1129225226 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3d28060870 con 0x559c6796f440 2024-04-25T10:10:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.941+0000 7f3d387d5740 10 client.6240 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T10:10:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.941+0000 7f3d387d5740 10 client.6240 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:10:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.941+0000 7f3d387d5740 20 client.6240 populate_metadata read hostname 'smithi119' 2024-04-25T10:10:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.941+0000 7f3d387d5740 1 --2- 172.21.15.119:0/1129225226 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559c6798a9d0 0x559c67958ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.941+0000 7f3d387d5740 4 client.6240 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.941+0000 7f3d387d5740 1 -- 172.21.15.119:0/1129225226 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559c6789e500 con 0x559c6798a9d0 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.942+0000 7f3d2ea00640 1 --2- 172.21.15.119:0/1129225226 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559c6798a9d0 0x559c67958ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.942+0000 7f3d13e00640 10 client.6240 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.942+0000 7f3d2ea00640 1 --2- 172.21.15.119:0/1129225226 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559c6798a9d0 0x559c67958ca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831e863e85871b9b server_cookie=7c118bb0d7920a9 in_seq=0 out_seq=0 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.942+0000 7f3d13e00640 1 -- 172.21.15.119:0/1129225226 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x559c6789e500 con 0x559c6798a9d0 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.942+0000 7f3d13e00640 10 client.6240 handle_command_reply: tid=0 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.942+0000 7f3d387d5740 1 client.6240 shutdown 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.943+0000 7f3d387d5740 1 -- 172.21.15.119:0/1129225226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6796f440 msgr2=0x559c67961600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.943+0000 7f3d387d5740 1 --2- 172.21.15.119:0/1129225226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6796f440 0x559c67961600 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3d280582f0 tx=0x7f3d28005e80 comp rx=0 tx=0).stop 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.943+0000 7f3d387d5740 1 -- 172.21.15.119:0/1129225226 shutdown_connections 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.943+0000 7f3d387d5740 1 --2- 172.21.15.119:0/1129225226 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x559c6798a9d0 0x559c67958ca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.943+0000 7f3d387d5740 1 --2- 172.21.15.119:0/1129225226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559c6796f440 0x559c67961600 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.943+0000 7f3d387d5740 1 --2- 172.21.15.119:0/1129225226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559c6795d830 0x559c679610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.943+0000 7f3d387d5740 1 --2- 172.21.15.119:0/1129225226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c67932d70 0x559c679ef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.943+0000 7f3d13e00640 0 client.6240 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:10:41.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.943+0000 7f3d387d5740 1 -- 172.21.15.119:0/1129225226 >> 172.21.15.119:0/1129225226 conn(0x559c678faa30 msgr2=0x559c67992590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.943+0000 7f3d387d5740 1 -- 172.21.15.119:0/1129225226 shutdown_connections 2024-04-25T10:10:41.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.943+0000 7f3d387d5740 1 -- 172.21.15.119:0/1129225226 wait complete. 2024-04-25T10:10:41.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.943+0000 7f3d387d5740 20 client.6240 trim_cache size 0 max 16384 2024-04-25T10:10:41.953 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:10:41.953 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:10:41.953 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2024-04-25T10:10:41.953 INFO:teuthology.orchestra.run.smithi119.stdout: "bda9564d-c2fc-40b3-8471-93323559c98a": { 2024-04-25T10:10:41.954 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/", 2024-04-25T10:10:41.954 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "bda9564d-c2fc-40b3-8471-93323559c98a", 2024-04-25T10:10:41.954 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,repair,force" 2024-04-25T10:10:41.954 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:10:41.954 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:10:41.954 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:10:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.944+0000 7f3d2fe00640 1 -- 172.21.15.119:0/804847468 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d0c06b4d0 msgr2=0x7f3d0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.944+0000 7f3d2fe00640 1 --2- 172.21.15.119:0/804847468 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d0c06b4d0 0x7f3d0c06d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3d20007fd0 tx=0x7f3d20009040 comp rx=0 tx=0).stop 2024-04-25T10:10:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.944+0000 7f3d2fe00640 1 -- 172.21.15.119:0/804847468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d3014b670 msgr2=0x7f3d300cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.944+0000 7f3d2fe00640 1 --2- 172.21.15.119:0/804847468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d3014b670 0x7f3d300cdb90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800c480 tx=0x7f3d2800c940 comp rx=0 tx=0).stop 2024-04-25T10:10:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.944+0000 7f3d2fe00640 1 -- 172.21.15.119:0/804847468 shutdown_connections 2024-04-25T10:10:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.944+0000 7f3d2fe00640 1 --2- 172.21.15.119:0/804847468 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d0c06b4d0 0x7f3d0c06d990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.944+0000 7f3d2fe00640 1 --2- 172.21.15.119:0/804847468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d3014b670 0x7f3d300cdb90 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.944+0000 7f3d2fe00640 1 --2- 172.21.15.119:0/804847468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d300cea40 0x7f3d3013cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.944+0000 7f3d2fe00640 1 --2- 172.21.15.119:0/804847468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d300c6150 0x7f3d30154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.944+0000 7f3d2fe00640 1 -- 172.21.15.119:0/804847468 >> 172.21.15.119:0/804847468 conn(0x7f3d3006aba0 msgr2=0x7f3d300b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.944+0000 7f3d2fe00640 1 -- 172.21.15.119:0/804847468 shutdown_connections 2024-04-25T10:10:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:10:41.944+0000 7f3d2fe00640 1 -- 172.21.15.119:0/804847468 wait complete. 2024-04-25T10:10:41.966 INFO:tasks.cephfs.filesystem:scrub status for tag:bda9564d-c2fc-40b3-8471-93323559c98a - {'path': '/', 'tag': 'bda9564d-c2fc-40b3-8471-93323559c98a', 'options': 'recursive,repair,force'} 2024-04-25T10:11:01.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:01.020 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:11.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:11.968 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:11:12.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.175+0000 7fe1d4200640 1 Processor -- start 2024-04-25T10:11:12.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.175+0000 7fe1d4200640 1 -- start start 2024-04-25T10:11:12.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.177+0000 7fe1d4200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1cc0b7030 0x7fe1cc0b7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.177+0000 7fe1d4200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1cc0cfcf0 con 0x7fe1cc0ce230 2024-04-25T10:11:12.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.177+0000 7fe1d4200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1cc0cffd0 con 0x7fe1cc15ba70 2024-04-25T10:11:12.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.177+0000 7fe1d4200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0d02b0 con 0x7fe1cc0b7030 2024-04-25T10:11:12.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.177+0000 7fe1d2200640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe1cc0ce230 0x7fe1cc0ce6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54878/0 (socket says 172.21.15.119:54878) 2024-04-25T10:11:12.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.177+0000 7fe1d2200640 1 -- 172.21.15.119:0/704987041 learned_addr learned my addr 172.21.15.119:0/704987041 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:12.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.177+0000 7fe1d0e00640 1 --2- 172.21.15.119:0/704987041 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1cc0b7030 0x7fe1cc0b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 -- 172.21.15.119:0/704987041 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216876731 0 0) 0x7fe1cc0cfcf0 con 0x7fe1cc0ce230 2024-04-25T10:11:12.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 -- 172.21.15.119:0/704987041 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1b4003520 con 0x7fe1cc0ce230 2024-04-25T10:11:12.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 -- 172.21.15.119:0/704987041 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1985149005 0 0) 0x7fe1b4003520 con 0x7fe1cc0ce230 2024-04-25T10:11:12.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 -- 172.21.15.119:0/704987041 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1cc0b7030 msgr2=0x7fe1cc0b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 --2- 172.21.15.119:0/704987041 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1cc0b7030 0x7fe1cc0b7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 -- 172.21.15.119:0/704987041 >> v1:172.21.15.132:6789/0 conn(0x7fe1cc15ba70 legacy=0x7fe1cc0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 -- 172.21.15.119:0/704987041 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc0d0590 con 0x7fe1cc0ce230 2024-04-25T10:11:12.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 -- 172.21.15.119:0/704987041 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fe1c40019f0 con 0x7fe1cc0ce230 2024-04-25T10:11:12.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 -- 172.21.15.119:0/704987041 >> v1:172.21.15.119:6789/0 conn(0x7fe1cc0ce230 legacy=0x7fe1cc0ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 --2- 172.21.15.119:0/704987041 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1cc0b7030 0x7fe1b40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 --2- 172.21.15.119:0/704987041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1b4003fb0 0x7fe1b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 --2- 172.21.15.119:0/704987041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1b4004d40 0x7fe1b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 -- 172.21.15.119:0/704987041 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0d02b0 con 0x7fe1b4004d40 2024-04-25T10:11:12.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 -- 172.21.15.119:0/704987041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b400bb00 con 0x7fe1b4003fb0 2024-04-25T10:11:12.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1cbe00640 1 -- 172.21.15.119:0/704987041 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b400bd00 con 0x7fe1cc0b7030 2024-04-25T10:11:12.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.178+0000 7fe1d2200640 1 --2- 172.21.15.119:0/704987041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1b4004d40 0x7fe1b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.179+0000 7fe1d1800640 1 --2- 172.21.15.119:0/704987041 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1cc0b7030 0x7fe1b40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.179+0000 7fe1d0e00640 1 --2- 172.21.15.119:0/704987041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1b4003fb0 0x7fe1b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.179+0000 7fe1d2200640 1 -- 172.21.15.119:0/704987041 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1cc0b7030 msgr2=0x7fe1b40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.179+0000 7fe1d2200640 1 --2- 172.21.15.119:0/704987041 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1cc0b7030 0x7fe1b40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.179+0000 7fe1d2200640 1 -- 172.21.15.119:0/704987041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1b4003fb0 msgr2=0x7fe1b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.179+0000 7fe1d2200640 1 --2- 172.21.15.119:0/704987041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1b4003fb0 0x7fe1b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.179+0000 7fe1d2200640 1 -- 172.21.15.119:0/704987041 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b4003520 con 0x7fe1b4004d40 2024-04-25T10:11:12.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.179+0000 7fe1d0e00640 1 --2- 172.21.15.119:0/704987041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1b4003fb0 0x7fe1b4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.180+0000 7fe1d2200640 1 --2- 172.21.15.119:0/704987041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1b4004d40 0x7fe1b40051c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400a770 tx=0x7fe1b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb8ec6f8e7a0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.180+0000 7fe1cbe00640 1 -- 172.21.15.119:0/704987041 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c400a390 con 0x7fe1b4004d40 2024-04-25T10:11:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.180+0000 7fe1cbe00640 1 -- 172.21.15.119:0/704987041 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c402f070 con 0x7fe1b4004d40 2024-04-25T10:11:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.180+0000 7fe1cbe00640 1 -- 172.21.15.119:0/704987041 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c402bc20 con 0x7fe1b4004d40 2024-04-25T10:11:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.181+0000 7fe1d4200640 1 -- 172.21.15.119:0/704987041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1b4004d40 msgr2=0x7fe1b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.181+0000 7fe1d4200640 1 --2- 172.21.15.119:0/704987041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1b4004d40 0x7fe1b40051c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400a770 tx=0x7fe1b4005830 comp rx=0 tx=0).stop 2024-04-25T10:11:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.181+0000 7fe1d2200640 1 -- 172.21.15.119:0/704987041 reap_dead start 2024-04-25T10:11:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.181+0000 7fe1d4200640 1 -- 172.21.15.119:0/704987041 shutdown_connections 2024-04-25T10:11:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.181+0000 7fe1d4200640 1 -- 172.21.15.119:0/704987041 >> 172.21.15.119:0/704987041 conn(0x7fe1cc06f4a0 msgr2=0x7fe1cc06f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.181+0000 7fe1d4200640 1 -- 172.21.15.119:0/704987041 shutdown_connections 2024-04-25T10:11:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.181+0000 7fe1d4200640 1 -- 172.21.15.119:0/704987041 wait complete. 2024-04-25T10:11:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.183+0000 7fe1d4200640 1 Processor -- start 2024-04-25T10:11:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.183+0000 7fe1d4200640 1 -- start start 2024-04-25T10:11:12.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.184+0000 7fe1d4200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1b4004d40 0x7fe1cc0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.184+0000 7fe1d4200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1cc0b0c40 0x7fe1cc0c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.184+0000 7fe1d4200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1cc0a8520 0x7fe1cc0a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.184+0000 7fe1d4200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc15be50 con 0x7fe1cc0b0c40 2024-04-25T10:11:12.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.184+0000 7fe1d4200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0cf980 con 0x7fe1cc0a8520 2024-04-25T10:11:12.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.184+0000 7fe1d4200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0c8710 con 0x7fe1b4004d40 2024-04-25T10:11:12.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.184+0000 7fe1d0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1cc0b0c40 0x7fe1cc0c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.184+0000 7fe1d0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1cc0b0c40 0x7fe1cc0c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43990/0 (socket says 172.21.15.119:43990) 2024-04-25T10:11:12.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.184+0000 7fe1d0e00640 1 -- 172.21.15.119:0/1603993022 learned_addr learned my addr 172.21.15.119:0/1603993022 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:12.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.184+0000 7fe1d1800640 1 --2- 172.21.15.119:0/1603993022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1b4004d40 0x7fe1cc0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.184+0000 7fe1d0e00640 1 -- 172.21.15.119:0/1603993022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1b4004d40 msgr2=0x7fe1cc0b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.185+0000 7fe1d0e00640 1 --2- 172.21.15.119:0/1603993022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1b4004d40 0x7fe1cc0b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.185+0000 7fe1d0e00640 1 -- 172.21.15.119:0/1603993022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1cc0a8520 msgr2=0x7fe1cc0a89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.185+0000 7fe1d0e00640 1 --2- 172.21.15.119:0/1603993022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1cc0a8520 0x7fe1cc0a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.185+0000 7fe1d0e00640 1 -- 172.21.15.119:0/1603993022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc0cace0 con 0x7fe1cc0b0c40 2024-04-25T10:11:12.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.185+0000 7fe1d1800640 1 --2- 172.21.15.119:0/1603993022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1b4004d40 0x7fe1cc0b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.185+0000 7fe1d0e00640 1 --2- 172.21.15.119:0/1603993022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1cc0b0c40 0x7fe1cc0c9cb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80108b0 tx=0x7fe1b8010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.185+0000 7fe1c9600640 1 -- 172.21.15.119:0/1603993022 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b800f040 con 0x7fe1cc0b0c40 2024-04-25T10:11:12.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.185+0000 7fe1c9600640 1 -- 172.21.15.119:0/1603993022 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b800eae0 con 0x7fe1cc0b0c40 2024-04-25T10:11:12.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.185+0000 7fe1c9600640 1 -- 172.21.15.119:0/1603993022 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b800ee90 con 0x7fe1cc0b0c40 2024-04-25T10:11:12.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.185+0000 7fe1d4200640 1 -- 172.21.15.119:0/1603993022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc0ca440 con 0x7fe1cc0b0c40 2024-04-25T10:11:12.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.185+0000 7fe1d4200640 1 -- 172.21.15.119:0/1603993022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc0caa00 con 0x7fe1cc0b0c40 2024-04-25T10:11:12.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.188+0000 7fe1c9600640 1 -- 172.21.15.119:0/1603993022 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1b801c400 con 0x7fe1cc0b0c40 2024-04-25T10:11:12.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.188+0000 7fe1c9600640 1 --2- 172.21.15.119:0/1603993022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe1ac06b6a0 0x7fe1ac06db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.189+0000 7fe1c9600640 1 -- 172.21.15.119:0/1603993022 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1b808a910 con 0x7fe1cc0b0c40 2024-04-25T10:11:12.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.189+0000 7fe1d1800640 1 --2- 172.21.15.119:0/1603993022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe1ac06b6a0 0x7fe1ac06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.189+0000 7fe1d7058740 1 Processor -- start 2024-04-25T10:11:12.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.189+0000 7fe1d7058740 1 -- start start 2024-04-25T10:11:12.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.189+0000 7fe1d1800640 1 --2- 172.21.15.119:0/1603993022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe1ac06b6a0 0x7fe1ac06db60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00016b0 tx=0x7fe1c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.190+0000 7fe1d7058740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 0x5606cfbb7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.190+0000 7fe1d7058740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbf2280 0x5606cfbb5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.190+0000 7fe1d7058740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbe0670 0x5606cfbe0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.190+0000 7fe1d7058740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cfb19c70 con 0x5606cfbf2280 2024-04-25T10:11:12.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.190+0000 7fe1d7058740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cfac2cf0 con 0x5606cfbb5bb0 2024-04-25T10:11:12.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.190+0000 7fe1d7058740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606cfb15d10 con 0x5606cfbe0670 2024-04-25T10:11:12.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.191+0000 7fe1d0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbf2280 0x5606cfbb5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.191+0000 7fe1d0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbf2280 0x5606cfbb5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44004/0 (socket says 172.21.15.119:44004) 2024-04-25T10:11:12.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.191+0000 7fe1d0e00640 1 -- 172.21.15.119:0/3342117112 learned_addr learned my addr 172.21.15.119:0/3342117112 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:12.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.191+0000 7fe1d0e00640 1 -- 172.21.15.119:0/3342117112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbe0670 msgr2=0x5606cfbe0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.191+0000 7fe1d2200640 1 --2- 172.21.15.119:0/3342117112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbe0670 0x5606cfbe0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.191+0000 7fe1d1800640 1 --2- 172.21.15.119:0/3342117112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 0x5606cfbb7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.191+0000 7fe1d0e00640 1 --2- 172.21.15.119:0/3342117112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbe0670 0x5606cfbe0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.191+0000 7fe1d0e00640 1 -- 172.21.15.119:0/3342117112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 msgr2=0x5606cfbb7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.191+0000 7fe1d0e00640 1 --2- 172.21.15.119:0/3342117112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 0x5606cfbb7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.191+0000 7fe1d0e00640 1 -- 172.21.15.119:0/3342117112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606cf8f87d0 con 0x5606cfbf2280 2024-04-25T10:11:12.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.191+0000 7fe1d0e00640 1 --2- 172.21.15.119:0/3342117112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbf2280 0x5606cfbb5f90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80611d0 tx=0x7fe1b8059130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418aaf87e9ea5d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.191+0000 7fe1be000640 1 -- 172.21.15.119:0/3342117112 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8087070 con 0x5606cfbf2280 2024-04-25T10:11:12.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.191+0000 7fe1be000640 1 -- 172.21.15.119:0/3342117112 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b8057090 con 0x5606cfbf2280 2024-04-25T10:11:12.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.191+0000 7fe1be000640 1 -- 172.21.15.119:0/3342117112 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8070070 con 0x5606cfbf2280 2024-04-25T10:11:12.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.192+0000 7fe1d7058740 1 -- 172.21.15.119:0/3342117112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbf2280 msgr2=0x5606cfbb5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.192+0000 7fe1d7058740 1 --2- 172.21.15.119:0/3342117112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbf2280 0x5606cfbb5f90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80611d0 tx=0x7fe1b8059130 comp rx=0 tx=0).stop 2024-04-25T10:11:12.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.192+0000 7fe1d7058740 1 -- 172.21.15.119:0/3342117112 shutdown_connections 2024-04-25T10:11:12.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.192+0000 7fe1d7058740 1 --2- 172.21.15.119:0/3342117112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbe0670 0x5606cfbe0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.192+0000 7fe1d7058740 1 --2- 172.21.15.119:0/3342117112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbf2280 0x5606cfbb5f90 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.192+0000 7fe1d7058740 1 --2- 172.21.15.119:0/3342117112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 0x5606cfbb7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.192+0000 7fe1d7058740 1 -- 172.21.15.119:0/3342117112 >> 172.21.15.119:0/3342117112 conn(0x5606cfb7da30 msgr2=0x5606cfbfb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.192+0000 7fe1d7058740 1 -- 172.21.15.119:0/3342117112 shutdown_connections 2024-04-25T10:11:12.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.192+0000 7fe1d7058740 1 -- 172.21.15.119:0/3342117112 wait complete. 2024-04-25T10:11:12.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.193+0000 7fe1d7058740 1 -- start start 2024-04-25T10:11:12.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.193+0000 7fe1d7058740 1 Processor -- start 2024-04-25T10:11:12.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.194+0000 7fe1d7058740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 0x5606cfbc6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.194+0000 7fe1d7058740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbe0670 0x5606cfbf2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.194+0000 7fe1d7058740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbf2280 0x5606cfbdcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.194+0000 7fe1d7058740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cfab83a0 con 0x5606cfbf2280 2024-04-25T10:11:12.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.194+0000 7fe1d7058740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cfbde1c0 con 0x5606cfbb5bb0 2024-04-25T10:11:12.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.194+0000 7fe1d7058740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606cfbc7c20 con 0x5606cfbe0670 2024-04-25T10:11:12.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.194+0000 7fe1d7058740 10 client.0 _ll_register_callbacks cb 0x5606cfad5b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:12.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.194+0000 7fe1d2200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbf2280 0x5606cfbdcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.195+0000 7fe1d2200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbf2280 0x5606cfbdcb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44012/0 (socket says 172.21.15.119:44012) 2024-04-25T10:11:12.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.195+0000 7fe1d2200640 1 -- 172.21.15.119:0/2012313943 learned_addr learned my addr 172.21.15.119:0/2012313943 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:12.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.195+0000 7fe1d1800640 1 --2- 172.21.15.119:0/2012313943 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 0x5606cfbc6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.195+0000 7fe1d0e00640 1 --2- 172.21.15.119:0/2012313943 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbe0670 0x5606cfbf2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.195+0000 7fe1d2200640 1 -- 172.21.15.119:0/2012313943 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbe0670 msgr2=0x5606cfbf2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.195+0000 7fe1d2200640 1 --2- 172.21.15.119:0/2012313943 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbe0670 0x5606cfbf2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.195+0000 7fe1d2200640 1 -- 172.21.15.119:0/2012313943 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 msgr2=0x5606cfbc6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.195+0000 7fe1d2200640 1 --2- 172.21.15.119:0/2012313943 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 0x5606cfbc6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.195+0000 7fe1d2200640 1 -- 172.21.15.119:0/2012313943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606cf82e3f0 con 0x5606cfbf2280 2024-04-25T10:11:12.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.195+0000 7fe1d2200640 1 --2- 172.21.15.119:0/2012313943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbf2280 0x5606cfbdcb50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe1c402b430 tx=0x7fe1cc0d0760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfea89b87e4ffe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.195+0000 7fe1b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:11:12.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.196+0000 7fe1b2a00640 1 -- 172.21.15.119:0/2012313943 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c402b990 con 0x5606cfbf2280 2024-04-25T10:11:12.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.196+0000 7fe1b2a00640 1 -- 172.21.15.119:0/2012313943 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1c402bb30 con 0x5606cfbf2280 2024-04-25T10:11:12.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.196+0000 7fe1d7058740 1 -- 172.21.15.119:0/2012313943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1b8057090 con 0x5606cfbf2280 2024-04-25T10:11:12.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.196+0000 7fe1b2a00640 1 -- 172.21.15.119:0/2012313943 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1c402bd30 con 0x5606cfbf2280 2024-04-25T10:11:12.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.196+0000 7fe1d7058740 10 client.6252 fetch_fsmap learned FSMap version 36 2024-04-25T10:11:12.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.196+0000 7fe1d7058740 1 -- 172.21.15.119:0/2012313943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x5606cf7ee0f0 con 0x5606cfbf2280 2024-04-25T10:11:12.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.196+0000 7fe1b2a00640 1 -- 172.21.15.119:0/2012313943 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b8057090 con 0x5606cfbf2280 2024-04-25T10:11:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.196+0000 7fe1b2a00640 1 -- 172.21.15.119:0/2012313943 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe1c4034470 con 0x5606cfbf2280 2024-04-25T10:11:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.196+0000 7fe1d7058740 10 client.6252 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T10:11:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.197+0000 7fe1d7058740 10 client.6252 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:11:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.197+0000 7fe1d7058740 20 client.6252 populate_metadata read hostname 'smithi119' 2024-04-25T10:11:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.197+0000 7fe1d7058740 1 --2- 172.21.15.119:0/2012313943 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5606cfbc8970 0x5606cfbc8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.197+0000 7fe1d7058740 4 client.6252 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.197+0000 7fe1d7058740 1 -- 172.21.15.119:0/2012313943 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606cfb271b0 con 0x5606cfbc8970 2024-04-25T10:11:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.197+0000 7fe1d1800640 1 --2- 172.21.15.119:0/2012313943 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5606cfbc8970 0x5606cfbc8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.197+0000 7fe1d1800640 1 --2- 172.21.15.119:0/2012313943 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5606cfbc8970 0x5606cfbc8da0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=264562a291885507 server_cookie=d33063ff4f2c0f79 in_seq=0 out_seq=0 2024-04-25T10:11:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.197+0000 7fe1b2a00640 10 client.6252 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:11:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.198+0000 7fe1b2a00640 1 -- 172.21.15.119:0/2012313943 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5606cfb271b0 con 0x5606cfbc8970 2024-04-25T10:11:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.198+0000 7fe1b2a00640 10 client.6252 handle_command_reply: tid=0 2024-04-25T10:11:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.198+0000 7fe1d7058740 1 client.6252 shutdown 2024-04-25T10:11:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.198+0000 7fe1d7058740 1 -- 172.21.15.119:0/2012313943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbf2280 msgr2=0x5606cfbdcb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.198+0000 7fe1d7058740 1 --2- 172.21.15.119:0/2012313943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbf2280 0x5606cfbdcb50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe1c402b430 tx=0x7fe1cc0d0760 comp rx=0 tx=0).stop 2024-04-25T10:11:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.198+0000 7fe1d7058740 1 -- 172.21.15.119:0/2012313943 shutdown_connections 2024-04-25T10:11:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.198+0000 7fe1d7058740 1 --2- 172.21.15.119:0/2012313943 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5606cfbc8970 0x5606cfbc8da0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.198+0000 7fe1d7058740 1 --2- 172.21.15.119:0/2012313943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbf2280 0x5606cfbdcb50 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.198+0000 7fe1d7058740 1 --2- 172.21.15.119:0/2012313943 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbe0670 0x5606cfbf2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.198+0000 7fe1d7058740 1 --2- 172.21.15.119:0/2012313943 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 0x5606cfbc6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.198+0000 7fe1b2a00640 0 client.6252 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:11:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.198+0000 7fe1d7058740 1 -- 172.21.15.119:0/2012313943 >> 172.21.15.119:0/2012313943 conn(0x5606cfb7da30 msgr2=0x5606cfbde3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.199+0000 7fe1d7058740 1 -- 172.21.15.119:0/2012313943 shutdown_connections 2024-04-25T10:11:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.199+0000 7fe1d7058740 1 -- 172.21.15.119:0/2012313943 wait complete. 2024-04-25T10:11:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.199+0000 7fe1d7058740 20 client.6252 trim_cache size 0 max 16384 2024-04-25T10:11:12.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.215+0000 7fe1d7058740 1 Processor -- start 2024-04-25T10:11:12.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.215+0000 7fe1d7058740 1 -- start start 2024-04-25T10:11:12.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.216+0000 7fe1d7058740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbb5bb0 0x5606cfbc44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.216+0000 7fe1d7058740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbe0670 0x5606cfbc49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.216+0000 7fe1d7058740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbf2280 0x5606cfbddf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.216+0000 7fe1d7058740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc1fc240 con 0x5606cfbb5bb0 2024-04-25T10:11:12.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.216+0000 7fe1d7058740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc1fb440 con 0x5606cfbe0670 2024-04-25T10:11:12.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.216+0000 7fe1d7058740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc148020 con 0x5606cfbf2280 2024-04-25T10:11:12.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.216+0000 7fe1d1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbb5bb0 0x5606cfbc44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.216+0000 7fe1d1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbb5bb0 0x5606cfbc44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44016/0 (socket says 172.21.15.119:44016) 2024-04-25T10:11:12.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.216+0000 7fe1d1800640 1 -- 172.21.15.119:0/1984259647 learned_addr learned my addr 172.21.15.119:0/1984259647 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:12.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.217+0000 7fe1d0e00640 1 --2- 172.21.15.119:0/1984259647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbe0670 0x5606cfbc49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.217+0000 7fe1d2200640 1 --2- 172.21.15.119:0/1984259647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbf2280 0x5606cfbddf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.217+0000 7fe1d1800640 1 -- 172.21.15.119:0/1984259647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbf2280 msgr2=0x5606cfbddf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.217+0000 7fe1d1800640 1 --2- 172.21.15.119:0/1984259647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbf2280 0x5606cfbddf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.217+0000 7fe1d1800640 1 -- 172.21.15.119:0/1984259647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbe0670 msgr2=0x5606cfbc49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.217+0000 7fe1d1800640 1 --2- 172.21.15.119:0/1984259647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbe0670 0x5606cfbc49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.217+0000 7fe1d1800640 1 -- 172.21.15.119:0/1984259647 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606cfbdd590 con 0x5606cfbb5bb0 2024-04-25T10:11:12.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.217+0000 7fe1d2200640 1 --2- 172.21.15.119:0/1984259647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbf2280 0x5606cfbddf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.217+0000 7fe1d1800640 1 --2- 172.21.15.119:0/1984259647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbb5bb0 0x5606cfbc44a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x5606cfbc9310 tx=0x7fe1c0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66a84f2bcd69624 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.217+0000 7fe1bd600640 1 -- 172.21.15.119:0/1984259647 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000fe20 con 0x5606cfbb5bb0 2024-04-25T10:11:12.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.217+0000 7fe1bd600640 1 -- 172.21.15.119:0/1984259647 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c001b040 con 0x5606cfbb5bb0 2024-04-25T10:11:12.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.217+0000 7fe1bd600640 1 -- 172.21.15.119:0/1984259647 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00249b0 con 0x5606cfbb5bb0 2024-04-25T10:11:12.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.218+0000 7fe1d7058740 1 -- 172.21.15.119:0/1984259647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbb5bb0 msgr2=0x5606cfbc44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.218+0000 7fe1d7058740 1 --2- 172.21.15.119:0/1984259647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbb5bb0 0x5606cfbc44a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x5606cfbc9310 tx=0x7fe1c0012000 comp rx=0 tx=0).stop 2024-04-25T10:11:12.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.218+0000 7fe1d7058740 1 -- 172.21.15.119:0/1984259647 shutdown_connections 2024-04-25T10:11:12.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.218+0000 7fe1d7058740 1 --2- 172.21.15.119:0/1984259647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbf2280 0x5606cfbddf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.218+0000 7fe1d7058740 1 --2- 172.21.15.119:0/1984259647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbe0670 0x5606cfbc49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.218+0000 7fe1d7058740 1 --2- 172.21.15.119:0/1984259647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbb5bb0 0x5606cfbc44a0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.218+0000 7fe1d7058740 1 -- 172.21.15.119:0/1984259647 >> 172.21.15.119:0/1984259647 conn(0x5606cfb7da30 msgr2=0x5606cfbc60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.218+0000 7fe1d7058740 1 -- 172.21.15.119:0/1984259647 shutdown_connections 2024-04-25T10:11:12.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.218+0000 7fe1d7058740 1 -- 172.21.15.119:0/1984259647 wait complete. 2024-04-25T10:11:12.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.219+0000 7fe1d7058740 1 -- start start 2024-04-25T10:11:12.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.219+0000 7fe1d7058740 1 Processor -- start 2024-04-25T10:11:12.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.220+0000 7fe1d7058740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 0x5606cfbdc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.220+0000 7fe1d7058740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbe0670 0x5606cfc5cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.220+0000 7fe1d7058740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbf2280 0x5606cfc66110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.220+0000 7fe1d7058740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606cfbb08a0 con 0x5606cfbe0670 2024-04-25T10:11:12.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.220+0000 7fe1d7058740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc1fae40 con 0x5606cfbb5bb0 2024-04-25T10:11:12.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.220+0000 7fe1d7058740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606cfbaca70 con 0x5606cfbf2280 2024-04-25T10:11:12.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.220+0000 7fe1d7058740 10 client.0 _ll_register_callbacks cb 0x7fe1b40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.220+0000 7fe1d0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbe0670 0x5606cfc5cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.220+0000 7fe1d0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbe0670 0x5606cfc5cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44024/0 (socket says 172.21.15.119:44024) 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.220+0000 7fe1d0e00640 1 -- 172.21.15.119:0/2816560925 learned_addr learned my addr 172.21.15.119:0/2816560925 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.221+0000 7fe1d1800640 1 --2- 172.21.15.119:0/2816560925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 0x5606cfbdc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.221+0000 7fe1d2200640 1 --2- 172.21.15.119:0/2816560925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbf2280 0x5606cfc66110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.221+0000 7fe1d0e00640 1 -- 172.21.15.119:0/2816560925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbf2280 msgr2=0x5606cfc66110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.221+0000 7fe1d0e00640 1 --2- 172.21.15.119:0/2816560925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbf2280 0x5606cfc66110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.221+0000 7fe1d0e00640 1 -- 172.21.15.119:0/2816560925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 msgr2=0x5606cfbdc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.221+0000 7fe1d0e00640 1 --2- 172.21.15.119:0/2816560925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 0x5606cfbdc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.221+0000 7fe1d0e00640 1 -- 172.21.15.119:0/2816560925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606cfbc6510 con 0x5606cfbe0670 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.221+0000 7fe1d1800640 1 --2- 172.21.15.119:0/2816560925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 0x5606cfbdc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.221+0000 7fe1d0e00640 1 --2- 172.21.15.119:0/2816560925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbe0670 0x5606cfc5cb20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80611d0 tx=0x7fe1b805b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b43f5150810f177 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.221+0000 7fe1b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.221+0000 7fe1b2a00640 1 -- 172.21.15.119:0/2816560925 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8087050 con 0x5606cfbe0670 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.221+0000 7fe1b2a00640 1 -- 172.21.15.119:0/2816560925 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1b8061470 con 0x5606cfbe0670 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.221+0000 7fe1d7058740 1 -- 172.21.15.119:0/2816560925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1c001b040 con 0x5606cfbe0670 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.222+0000 7fe1b2a00640 1 -- 172.21.15.119:0/2816560925 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1b8070040 con 0x5606cfbe0670 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.222+0000 7fe1d7058740 10 client.6258 fetch_fsmap learned FSMap version 36 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.222+0000 7fe1d7058740 1 -- 172.21.15.119:0/2816560925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x5606cfc66650 con 0x5606cfbe0670 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.222+0000 7fe1b2a00640 1 -- 172.21.15.119:0/2816560925 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c001b040 con 0x5606cfbe0670 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.222+0000 7fe1b2a00640 1 -- 172.21.15.119:0/2816560925 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe1b8058440 con 0x5606cfbe0670 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.222+0000 7fe1d7058740 10 client.6258 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.222+0000 7fe1d7058740 10 client.6258 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.222+0000 7fe1d7058740 20 client.6258 populate_metadata read hostname 'smithi119' 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.222+0000 7fe1d7058740 1 --2- 172.21.15.119:0/2816560925 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5606cfc60650 0x5606cfc0dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.222+0000 7fe1d7058740 4 client.6258 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.222+0000 7fe1d7058740 1 -- 172.21.15.119:0/2816560925 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5606cfb21500 con 0x5606cfc60650 2024-04-25T10:11:12.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.222+0000 7fe1d1800640 1 --2- 172.21.15.119:0/2816560925 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5606cfc60650 0x5606cfc0dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1d1800640 1 --2- 172.21.15.119:0/2816560925 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5606cfc60650 0x5606cfc0dc70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e976a96434fee484 server_cookie=1602a532374cf054 in_seq=0 out_seq=0 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1b2a00640 10 client.6258 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1b2a00640 1 -- 172.21.15.119:0/2816560925 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5606cfb21500 con 0x5606cfc60650 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1b2a00640 10 client.6258 handle_command_reply: tid=0 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1d7058740 1 client.6258 shutdown 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1d7058740 1 -- 172.21.15.119:0/2816560925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbe0670 msgr2=0x5606cfc5cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1d7058740 1 --2- 172.21.15.119:0/2816560925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbe0670 0x5606cfc5cb20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80611d0 tx=0x7fe1b805b000 comp rx=0 tx=0).stop 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1d7058740 1 -- 172.21.15.119:0/2816560925 shutdown_connections 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1d7058740 1 --2- 172.21.15.119:0/2816560925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5606cfbf2280 0x5606cfc66110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1d7058740 1 --2- 172.21.15.119:0/2816560925 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x5606cfc60650 0x5606cfc0dc70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1d7058740 1 --2- 172.21.15.119:0/2816560925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606cfbe0670 0x5606cfc5cb20 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1d7058740 1 --2- 172.21.15.119:0/2816560925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5606cfbb5bb0 0x5606cfbdc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1b2a00640 0 client.6258 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:11:12.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1d7058740 1 -- 172.21.15.119:0/2816560925 >> 172.21.15.119:0/2816560925 conn(0x5606cfb7da30 msgr2=0x5606cfbf3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1d7058740 1 -- 172.21.15.119:0/2816560925 shutdown_connections 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.223+0000 7fe1d7058740 1 -- 172.21.15.119:0/2816560925 wait complete. 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.224+0000 7fe1d7058740 20 client.6258 trim_cache size 0 max 16384 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2024-04-25T10:11:12.233 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:11:12.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.224+0000 7fe1d4200640 1 -- 172.21.15.119:0/1603993022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe1ac06b6a0 msgr2=0x7fe1ac06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.224+0000 7fe1d4200640 1 --2- 172.21.15.119:0/1603993022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe1ac06b6a0 0x7fe1ac06db60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00016b0 tx=0x7fe1c0009040 comp rx=0 tx=0).stop 2024-04-25T10:11:12.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.224+0000 7fe1d4200640 1 -- 172.21.15.119:0/1603993022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1cc0b0c40 msgr2=0x7fe1cc0c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.224+0000 7fe1d4200640 1 --2- 172.21.15.119:0/1603993022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1cc0b0c40 0x7fe1cc0c9cb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80108b0 tx=0x7fe1b8010d70 comp rx=0 tx=0).stop 2024-04-25T10:11:12.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.224+0000 7fe1d4200640 1 -- 172.21.15.119:0/1603993022 shutdown_connections 2024-04-25T10:11:12.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.224+0000 7fe1d4200640 1 --2- 172.21.15.119:0/1603993022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe1ac06b6a0 0x7fe1ac06db60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.224+0000 7fe1d4200640 1 --2- 172.21.15.119:0/1603993022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1cc0a8520 0x7fe1cc0a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.224+0000 7fe1d4200640 1 --2- 172.21.15.119:0/1603993022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1cc0b0c40 0x7fe1cc0c9cb0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.224+0000 7fe1d4200640 1 --2- 172.21.15.119:0/1603993022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1b4004d40 0x7fe1cc0b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.224+0000 7fe1d4200640 1 -- 172.21.15.119:0/1603993022 >> 172.21.15.119:0/1603993022 conn(0x7fe1cc06f4a0 msgr2=0x7fe1cc15ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.224+0000 7fe1d4200640 1 -- 172.21.15.119:0/1603993022 shutdown_connections 2024-04-25T10:11:12.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.224+0000 7fe1d4200640 1 -- 172.21.15.119:0/1603993022 wait complete. 2024-04-25T10:11:12.248 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:11:12.248 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:12.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.431+0000 7fb0bea00640 1 Processor -- start 2024-04-25T10:11:12.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.431+0000 7fb0bea00640 1 -- start start 2024-04-25T10:11:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.433+0000 7fb0bea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0b8144040 0x7fb0b80cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.433+0000 7fb0bea00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b8144420 con 0x7fb0b815acd0 2024-04-25T10:11:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.433+0000 7fb0bea00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b80b3d80 con 0x7fb0b806ea30 2024-04-25T10:11:12.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.433+0000 7fb0bea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80b4060 con 0x7fb0b8144040 2024-04-25T10:11:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.433+0000 7fb0bcc00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb0b815acd0 0x7fb0b806e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54884/0 (socket says 172.21.15.119:54884) 2024-04-25T10:11:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.433+0000 7fb0bcc00640 1 -- 172.21.15.119:0/440850615 learned_addr learned my addr 172.21.15.119:0/440850615 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.433+0000 7fb0bd600640 1 --2- 172.21.15.119:0/440850615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0b8144040 0x7fb0b80cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.433+0000 7fb0b7e00640 1 -- 172.21.15.119:0/440850615 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1131419459 0 0) 0x7fb0b8144420 con 0x7fb0b815acd0 2024-04-25T10:11:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0b7e00640 1 -- 172.21.15.119:0/440850615 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb09c003520 con 0x7fb0b815acd0 2024-04-25T10:11:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0b7e00640 1 -- 172.21.15.119:0/440850615 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4275477707 0 0) 0x7fb09c003520 con 0x7fb0b815acd0 2024-04-25T10:11:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0b7e00640 1 -- 172.21.15.119:0/440850615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0b8144040 msgr2=0x7fb0b80cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0b7e00640 1 --2- 172.21.15.119:0/440850615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0b8144040 0x7fb0b80cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0b7e00640 1 -- 172.21.15.119:0/440850615 >> v1:172.21.15.132:6789/0 conn(0x7fb0b806ea30 legacy=0x7fb0b80b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0b7e00640 1 -- 172.21.15.119:0/440850615 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b80b4340 con 0x7fb0b815acd0 2024-04-25T10:11:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0b7e00640 1 -- 172.21.15.119:0/440850615 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fb0a4001670 con 0x7fb0b815acd0 2024-04-25T10:11:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0b7e00640 1 -- 172.21.15.119:0/440850615 >> v1:172.21.15.119:6789/0 conn(0x7fb0b815acd0 legacy=0x7fb0b806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0b7e00640 1 --2- 172.21.15.119:0/440850615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c003a50 0x7fb09c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0b7e00640 1 --2- 172.21.15.119:0/440850615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0b8144040 0x7fb09c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0b7e00640 1 --2- 172.21.15.119:0/440850615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb09c004d40 0x7fb09c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0b7e00640 1 -- 172.21.15.119:0/440850615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80b4060 con 0x7fb09c003a50 2024-04-25T10:11:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0b7e00640 1 -- 172.21.15.119:0/440850615 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c00bb50 con 0x7fb09c004d40 2024-04-25T10:11:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0b7e00640 1 -- 172.21.15.119:0/440850615 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c00df60 con 0x7fb0b8144040 2024-04-25T10:11:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.434+0000 7fb0bd600640 1 --2- 172.21.15.119:0/440850615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c003a50 0x7fb09c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.435+0000 7fb0bcc00640 1 --2- 172.21.15.119:0/440850615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0b8144040 0x7fb09c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.435+0000 7fb0be000640 1 --2- 172.21.15.119:0/440850615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb09c004d40 0x7fb09c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.435+0000 7fb0bd600640 1 -- 172.21.15.119:0/440850615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0b8144040 msgr2=0x7fb09c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.435+0000 7fb0bd600640 1 --2- 172.21.15.119:0/440850615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0b8144040 0x7fb09c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.435+0000 7fb0bd600640 1 -- 172.21.15.119:0/440850615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb09c004d40 msgr2=0x7fb09c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.435+0000 7fb0bd600640 1 --2- 172.21.15.119:0/440850615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb09c004d40 0x7fb09c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.435+0000 7fb0bd600640 1 -- 172.21.15.119:0/440850615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09c003520 con 0x7fb09c003a50 2024-04-25T10:11:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.435+0000 7fb0be000640 1 --2- 172.21.15.119:0/440850615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb09c004d40 0x7fb09c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.436+0000 7fb0bd600640 1 --2- 172.21.15.119:0/440850615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c003a50 0x7fb09c003e70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009fc0 tx=0x7fb09c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2573a35703ed90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.436+0000 7fb0b7e00640 1 -- 172.21.15.119:0/440850615 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8009c10 con 0x7fb09c003a50 2024-04-25T10:11:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.436+0000 7fb0b7e00640 1 -- 172.21.15.119:0/440850615 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a8036070 con 0x7fb09c003a50 2024-04-25T10:11:12.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.436+0000 7fb0b7e00640 1 -- 172.21.15.119:0/440850615 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8031880 con 0x7fb09c003a50 2024-04-25T10:11:12.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.437+0000 7fb0bea00640 1 -- 172.21.15.119:0/440850615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c003a50 msgr2=0x7fb09c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.437+0000 7fb0bea00640 1 --2- 172.21.15.119:0/440850615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c003a50 0x7fb09c003e70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009fc0 tx=0x7fb09c005830 comp rx=0 tx=0).stop 2024-04-25T10:11:12.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.437+0000 7fb0be000640 1 -- 172.21.15.119:0/440850615 reap_dead start 2024-04-25T10:11:12.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.437+0000 7fb0bea00640 1 -- 172.21.15.119:0/440850615 shutdown_connections 2024-04-25T10:11:12.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.437+0000 7fb0bea00640 1 -- 172.21.15.119:0/440850615 >> 172.21.15.119:0/440850615 conn(0x7fb0b806b1d0 msgr2=0x7fb0b80b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.437+0000 7fb0bea00640 1 -- 172.21.15.119:0/440850615 shutdown_connections 2024-04-25T10:11:12.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.437+0000 7fb0bea00640 1 -- 172.21.15.119:0/440850615 wait complete. 2024-04-25T10:11:12.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.439+0000 7fb0bea00640 1 Processor -- start 2024-04-25T10:11:12.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.439+0000 7fb0bea00640 1 -- start start 2024-04-25T10:11:12.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0bea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb09c003a50 0x7fb0b8159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0bea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0b813d1b0 0x7fb0b813d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0bea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0b8141be0 0x7fb0b8142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0bea00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80b5180 con 0x7fb0b813d1b0 2024-04-25T10:11:12.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0bea00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80b4900 con 0x7fb09c003a50 2024-04-25T10:11:12.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0bea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80b5300 con 0x7fb0b8141be0 2024-04-25T10:11:12.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0bcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0b813d1b0 0x7fb0b813d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0bcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0b813d1b0 0x7fb0b813d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44040/0 (socket says 172.21.15.119:44040) 2024-04-25T10:11:12.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0bcc00640 1 -- 172.21.15.119:0/1572470361 learned_addr learned my addr 172.21.15.119:0/1572470361 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:12.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0bcc00640 1 -- 172.21.15.119:0/1572470361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0b8141be0 msgr2=0x7fb0b8142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0be000640 1 --2- 172.21.15.119:0/1572470361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0b8141be0 0x7fb0b8142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0bd600640 1 --2- 172.21.15.119:0/1572470361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb09c003a50 0x7fb0b8159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0bcc00640 1 --2- 172.21.15.119:0/1572470361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0b8141be0 0x7fb0b8142090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0bcc00640 1 -- 172.21.15.119:0/1572470361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb09c003a50 msgr2=0x7fb0b8159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.440+0000 7fb0bcc00640 1 --2- 172.21.15.119:0/1572470361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb09c003a50 0x7fb0b8159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.441+0000 7fb0bcc00640 1 -- 172.21.15.119:0/1572470361 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b815a4c0 con 0x7fb0b813d1b0 2024-04-25T10:11:12.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.441+0000 7fb0bcc00640 1 --2- 172.21.15.119:0/1572470361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0b813d1b0 0x7fb0b813d630 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400e680 tx=0x7fb0a400eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.441+0000 7fb0b5600640 1 -- 172.21.15.119:0/1572470361 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400c6c0 con 0x7fb0b813d1b0 2024-04-25T10:11:12.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.441+0000 7fb0b5600640 1 -- 172.21.15.119:0/1572470361 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a4018070 con 0x7fb0b813d1b0 2024-04-25T10:11:12.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.441+0000 7fb0b5600640 1 -- 172.21.15.119:0/1572470361 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400cac0 con 0x7fb0b813d1b0 2024-04-25T10:11:12.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.441+0000 7fb0bea00640 1 -- 172.21.15.119:0/1572470361 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b813dcc0 con 0x7fb0b813d1b0 2024-04-25T10:11:12.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.441+0000 7fb0bea00640 1 -- 172.21.15.119:0/1572470361 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b813e280 con 0x7fb0b813d1b0 2024-04-25T10:11:12.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.442+0000 7fb0bea00640 1 -- 172.21.15.119:0/1572470361 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b806bb30 con 0x7fb0b813d1b0 2024-04-25T10:11:12.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.443+0000 7fb0b5600640 1 -- 172.21.15.119:0/1572470361 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0a400cc60 con 0x7fb0b813d1b0 2024-04-25T10:11:12.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.444+0000 7fb0b5600640 1 --2- 172.21.15.119:0/1572470361 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb09806b500 0x7fb09806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.444+0000 7fb0b5600640 1 -- 172.21.15.119:0/1572470361 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0a4089ff0 con 0x7fb0b813d1b0 2024-04-25T10:11:12.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.444+0000 7fb0bd600640 1 --2- 172.21.15.119:0/1572470361 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb09806b500 0x7fb09806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.445+0000 7fb0bd600640 1 --2- 172.21.15.119:0/1572470361 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb09806b500 0x7fb09806d9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8035000 tx=0x7fb0a802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.449+0000 7fb0b5600640 1 -- 172.21.15.119:0/1572470361 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0a4058dd0 con 0x7fb0b813d1b0 2024-04-25T10:11:12.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.684+0000 7fb0bea00640 1 -- 172.21.15.119:0/1572470361 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0b80749b0 con 0x7fb0b813d1b0 2024-04-25T10:11:12.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.685+0000 7fb0b5600640 1 -- 172.21.15.119:0/1572470361 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4951 (secure 0 0 0) 0x7fb0a40586d0 con 0x7fb0b813d1b0 2024-04-25T10:11:12.691 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:12.691 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4159222235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4159222235},{"type":"v1","addr":"172.21.15.132:6834","nonce":4159222235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4223361610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":4223361610},{"type":"v1","addr":"172.21.15.119:6837","nonce":4223361610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2367324442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2367324442},{"type":"v1","addr":"172.21.15.132:6835","nonce":2367324442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:59.454941+0000","modified":"2024-04-25T10:10:48.089168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6836/4129112885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4129112885},{"type":"v1","addr":"172.21.15.119:6836","nonce":4129112885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:11:12.691 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 36 2024-04-25T10:11:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.688+0000 7fb0bea00640 1 -- 172.21.15.119:0/1572470361 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb09806b500 msgr2=0x7fb09806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.688+0000 7fb0bea00640 1 --2- 172.21.15.119:0/1572470361 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb09806b500 0x7fb09806d9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8035000 tx=0x7fb0a802f000 comp rx=0 tx=0).stop 2024-04-25T10:11:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.688+0000 7fb0bea00640 1 -- 172.21.15.119:0/1572470361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0b813d1b0 msgr2=0x7fb0b813d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.688+0000 7fb0bea00640 1 --2- 172.21.15.119:0/1572470361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0b813d1b0 0x7fb0b813d630 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400e680 tx=0x7fb0a400eb40 comp rx=0 tx=0).stop 2024-04-25T10:11:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.688+0000 7fb0bea00640 1 -- 172.21.15.119:0/1572470361 shutdown_connections 2024-04-25T10:11:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.688+0000 7fb0bea00640 1 --2- 172.21.15.119:0/1572470361 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb09806b500 0x7fb09806d9c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.688+0000 7fb0bea00640 1 --2- 172.21.15.119:0/1572470361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb0b8141be0 0x7fb0b8142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.688+0000 7fb0bea00640 1 --2- 172.21.15.119:0/1572470361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0b813d1b0 0x7fb0b813d630 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.688+0000 7fb0bea00640 1 --2- 172.21.15.119:0/1572470361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb09c003a50 0x7fb0b8159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.688+0000 7fb0bea00640 1 -- 172.21.15.119:0/1572470361 >> 172.21.15.119:0/1572470361 conn(0x7fb0b806b1d0 msgr2=0x7fb0b80b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.689+0000 7fb0bea00640 1 -- 172.21.15.119:0/1572470361 shutdown_connections 2024-04-25T10:11:12.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.689+0000 7fb0bea00640 1 -- 172.21.15.119:0/1572470361 wait complete. 2024-04-25T10:11:12.712 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:11:12.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.900+0000 7f2d2f600640 1 Processor -- start 2024-04-25T10:11:12.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.900+0000 7f2d2f600640 1 -- start start 2024-04-25T10:11:12.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.901+0000 7f2d2f600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2d28071930 0x7f2d2806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.901+0000 7f2d2f600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d2806ea30 0x7f2d280b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.901+0000 7f2d2f600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2815dc30 con 0x7f2d2815a260 2024-04-25T10:11:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.901+0000 7f2d2f600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2815df10 con 0x7f2d2806ea30 2024-04-25T10:11:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.901+0000 7f2d2f600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2815e1f0 con 0x7f2d28071930 2024-04-25T10:11:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.901+0000 7f2d2d600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d2806ea30 0x7f2d280b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.902+0000 7f2d2cc00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2d2815a260 0x7f2d28071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54900/0 (socket says 172.21.15.119:54900) 2024-04-25T10:11:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.902+0000 7f2d2cc00640 1 -- 172.21.15.119:0/2152944956 learned_addr learned my addr 172.21.15.119:0/2152944956 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.902+0000 7f2d1fe00640 1 --2- 172.21.15.119:0/2152944956 >> v2:172.21.15.132:3300/0 conn(0x7f2d28071930 0x7f2d2806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.902+0000 7f2d2d600640 1 -- 172.21.15.119:0/2152944956 >> v2:172.21.15.132:3300/0 conn(0x7f2d28071930 msgr2=0x7f2d2806e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.902+0000 7f2d2d600640 1 --2- 172.21.15.119:0/2152944956 >> v2:172.21.15.132:3300/0 conn(0x7f2d28071930 0x7f2d2806e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.902+0000 7f2d2d600640 1 -- 172.21.15.119:0/2152944956 >> v1:172.21.15.119:6789/0 conn(0x7f2d2815a260 legacy=0x7f2d28071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.902+0000 7f2d2d600640 1 -- 172.21.15.119:0/2152944956 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2815e4d0 con 0x7f2d2806ea30 2024-04-25T10:11:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.902+0000 7f2d27e00640 1 -- 172.21.15.119:0/2152944956 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928592872 0 0) 0x7f2d2815dc30 con 0x7f2d2815a260 2024-04-25T10:11:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.903+0000 7f2d2d600640 1 --2- 172.21.15.119:0/2152944956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2806ea30 0x7f2d280b2650 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2d280673d0 tx=0x7f2d2002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9efb9080b453f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.903+0000 7f2d27e00640 1 -- 172.21.15.119:0/2152944956 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20007740 con 0x7f2d2806ea30 2024-04-25T10:11:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.903+0000 7f2d27e00640 1 -- 172.21.15.119:0/2152944956 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d200078e0 con 0x7f2d2806ea30 2024-04-25T10:11:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.903+0000 7f2d27e00640 1 -- 172.21.15.119:0/2152944956 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20007cb0 con 0x7f2d2806ea30 2024-04-25T10:11:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.904+0000 7f2d2f600640 1 -- 172.21.15.119:0/2152944956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2806ea30 msgr2=0x7f2d280b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.904+0000 7f2d2f600640 1 --2- 172.21.15.119:0/2152944956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2806ea30 0x7f2d280b2650 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2d280673d0 tx=0x7f2d2002a680 comp rx=0 tx=0).stop 2024-04-25T10:11:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.904+0000 7f2d2f600640 1 -- 172.21.15.119:0/2152944956 shutdown_connections 2024-04-25T10:11:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.904+0000 7f2d2f600640 1 --2- 172.21.15.119:0/2152944956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2806ea30 0x7f2d280b2650 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.904+0000 7f2d2f600640 1 --2- 172.21.15.119:0/2152944956 >> v2:172.21.15.132:3300/0 conn(0x7f2d28071930 0x7f2d2806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.904+0000 7f2d2f600640 1 -- 172.21.15.119:0/2152944956 >> 172.21.15.119:0/2152944956 conn(0x7f2d2806aba0 msgr2=0x7f2d2806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.904+0000 7f2d2f600640 1 -- 172.21.15.119:0/2152944956 shutdown_connections 2024-04-25T10:11:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.904+0000 7f2d2f600640 1 -- 172.21.15.119:0/2152944956 wait complete. 2024-04-25T10:11:12.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.906+0000 7f2d2f600640 1 Processor -- start 2024-04-25T10:11:12.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.906+0000 7f2d2f600640 1 -- start start 2024-04-25T10:11:12.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.907+0000 7f2d2f600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2806ea30 0x7f2d28152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.907+0000 7f2d2f600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d28071930 0x7f2d281581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.907+0000 7f2d2f600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2815a260 0x7f2d281f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.907+0000 7f2d2f600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2806f040 con 0x7f2d2815a260 2024-04-25T10:11:12.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.907+0000 7f2d2f600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2806eec0 con 0x7f2d28071930 2024-04-25T10:11:12.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.907+0000 7f2d2f600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d280cf6f0 con 0x7f2d2806ea30 2024-04-25T10:11:12.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.907+0000 7f2d2d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2815a260 0x7f2d281f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.907+0000 7f2d2d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2815a260 0x7f2d281f3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44046/0 (socket says 172.21.15.119:44046) 2024-04-25T10:11:12.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.907+0000 7f2d2d600640 1 -- 172.21.15.119:0/3987064115 learned_addr learned my addr 172.21.15.119:0/3987064115 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:12.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.908+0000 7f2d2d600640 1 -- 172.21.15.119:0/3987064115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2806ea30 msgr2=0x7f2d28152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.908+0000 7f2d2d600640 1 --2- 172.21.15.119:0/3987064115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2806ea30 0x7f2d28152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.908+0000 7f2d2d600640 1 -- 172.21.15.119:0/3987064115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d28071930 msgr2=0x7f2d281581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.908+0000 7f2d1fe00640 1 --2- 172.21.15.119:0/3987064115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d28071930 0x7f2d281581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.908+0000 7f2d2d600640 1 --2- 172.21.15.119:0/3987064115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d28071930 0x7f2d281581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.908+0000 7f2d2d600640 1 -- 172.21.15.119:0/3987064115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d28152f50 con 0x7f2d2815a260 2024-04-25T10:11:12.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.908+0000 7f2d1fe00640 1 --2- 172.21.15.119:0/3987064115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d28071930 0x7f2d281581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.908+0000 7f2d2d600640 1 --2- 172.21.15.119:0/3987064115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2815a260 0x7f2d281f3b30 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2d2002ac70 tx=0x7f2d20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.908+0000 7f2d25600640 1 -- 172.21.15.119:0/3987064115 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20040070 con 0x7f2d2815a260 2024-04-25T10:11:12.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.908+0000 7f2d25600640 1 -- 172.21.15.119:0/3987064115 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d20031bc0 con 0x7f2d2815a260 2024-04-25T10:11:12.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.908+0000 7f2d25600640 1 -- 172.21.15.119:0/3987064115 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20045460 con 0x7f2d2815a260 2024-04-25T10:11:12.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.908+0000 7f2d2f600640 1 -- 172.21.15.119:0/3987064115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d281f45f0 con 0x7f2d2815a260 2024-04-25T10:11:12.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.909+0000 7f2d2f600640 1 -- 172.21.15.119:0/3987064115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d281579d0 con 0x7f2d2815a260 2024-04-25T10:11:12.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.911+0000 7f2d25600640 1 -- 172.21.15.119:0/3987064115 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d20039040 con 0x7f2d2815a260 2024-04-25T10:11:12.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.911+0000 7f2d25600640 1 --2- 172.21.15.119:0/3987064115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d0406b4d0 0x7f2d0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.912+0000 7f2d25600640 1 -- 172.21.15.119:0/3987064115 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d200abc60 con 0x7f2d2815a260 2024-04-25T10:11:12.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.912+0000 7f2d2cc00640 1 --2- 172.21.15.119:0/3987064115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d0406b4d0 0x7f2d0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.912+0000 7f2d323f1740 1 Processor -- start 2024-04-25T10:11:12.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.912+0000 7f2d323f1740 1 -- start start 2024-04-25T10:11:12.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.912+0000 7f2d2cc00640 1 --2- 172.21.15.119:0/3987064115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d0406b4d0 0x7f2d0406d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2d18007fd0 tx=0x7f2d18007960 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.913+0000 7f2d323f1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0dd7370 0x558dd0dd7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.913+0000 7f2d323f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0df5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.913+0000 7f2d323f1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dad050 0x558dd0df32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.913+0000 7f2d323f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd0d0fae0 con 0x558dd0df4c10 2024-04-25T10:11:12.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.913+0000 7f2d323f1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd0cb87c0 con 0x558dd0dd7370 2024-04-25T10:11:12.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.913+0000 7f2d323f1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558dd0d0bb80 con 0x558dd0dad050 2024-04-25T10:11:12.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.914+0000 7f2d1fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0df5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.914+0000 7f2d1fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0df5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44054/0 (socket says 172.21.15.119:44054) 2024-04-25T10:11:12.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.914+0000 7f2d1fe00640 1 -- 172.21.15.119:0/3750102010 learned_addr learned my addr 172.21.15.119:0/3750102010 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:12.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.914+0000 7f2d2cc00640 1 --2- 172.21.15.119:0/3750102010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0dd7370 0x558dd0dd7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.914+0000 7f2d2d600640 1 --2- 172.21.15.119:0/3750102010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dad050 0x558dd0df32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.914+0000 7f2d1fe00640 1 -- 172.21.15.119:0/3750102010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dad050 msgr2=0x558dd0df32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.914+0000 7f2d1fe00640 1 --2- 172.21.15.119:0/3750102010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dad050 0x558dd0df32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.914+0000 7f2d1fe00640 1 -- 172.21.15.119:0/3750102010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0dd7370 msgr2=0x558dd0dd7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.914+0000 7f2d1fe00640 1 --2- 172.21.15.119:0/3750102010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0dd7370 0x558dd0dd7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.914+0000 7f2d1fe00640 1 -- 172.21.15.119:0/3750102010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dd0af07c0 con 0x558dd0df4c10 2024-04-25T10:11:12.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.914+0000 7f2d1fe00640 1 --2- 172.21.15.119:0/3750102010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0df5160 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2d100099f0 tx=0x7f2d10009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d145d7133b52b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.915+0000 7f2d1d200640 1 -- 172.21.15.119:0/3750102010 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10017030 con 0x558dd0df4c10 2024-04-25T10:11:12.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.915+0000 7f2d1d200640 1 -- 172.21.15.119:0/3750102010 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d10004020 con 0x558dd0df4c10 2024-04-25T10:11:12.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.915+0000 7f2d1d200640 1 -- 172.21.15.119:0/3750102010 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1000f040 con 0x558dd0df4c10 2024-04-25T10:11:12.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.915+0000 7f2d323f1740 1 -- 172.21.15.119:0/3750102010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 msgr2=0x558dd0df5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.915+0000 7f2d323f1740 1 --2- 172.21.15.119:0/3750102010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0df5160 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2d100099f0 tx=0x7f2d10009eb0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.915+0000 7f2d323f1740 1 -- 172.21.15.119:0/3750102010 shutdown_connections 2024-04-25T10:11:12.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.915+0000 7f2d323f1740 1 --2- 172.21.15.119:0/3750102010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dad050 0x558dd0df32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.915+0000 7f2d323f1740 1 --2- 172.21.15.119:0/3750102010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0df5160 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.915+0000 7f2d323f1740 1 --2- 172.21.15.119:0/3750102010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0dd7370 0x558dd0dd7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.915+0000 7f2d323f1740 1 -- 172.21.15.119:0/3750102010 >> 172.21.15.119:0/3750102010 conn(0x558dd0da9980 msgr2=0x558dd0df2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.916+0000 7f2d323f1740 1 -- 172.21.15.119:0/3750102010 shutdown_connections 2024-04-25T10:11:12.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.916+0000 7f2d323f1740 1 -- 172.21.15.119:0/3750102010 wait complete. 2024-04-25T10:11:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.916+0000 7f2d323f1740 1 -- start start 2024-04-25T10:11:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.916+0000 7f2d323f1740 1 Processor -- start 2024-04-25T10:11:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.917+0000 7f2d323f1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dd7370 0x558dd0deb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.917+0000 7f2d323f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0de9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.917+0000 7f2d323f1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0dc86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.917+0000 7f2d323f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd0daddf0 con 0x558dd0df4c10 2024-04-25T10:11:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.917+0000 7f2d323f1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd0cadd40 con 0x558dd0debc90 2024-04-25T10:11:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.917+0000 7f2d323f1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558dd0da78b0 con 0x558dd0dd7370 2024-04-25T10:11:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.917+0000 7f2d323f1740 10 client.0 _ll_register_callbacks cb 0x558dd0a23ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.918+0000 7f2d1fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0de9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.918+0000 7f2d1fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0de9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44070/0 (socket says 172.21.15.119:44070) 2024-04-25T10:11:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.918+0000 7f2d1fe00640 1 -- 172.21.15.119:0/3189977740 learned_addr learned my addr 172.21.15.119:0/3189977740 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.918+0000 7f2d2cc00640 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dd7370 0x558dd0deb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.918+0000 7f2d2d600640 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0dc86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.918+0000 7f2d1fe00640 1 -- 172.21.15.119:0/3189977740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dd7370 msgr2=0x558dd0deb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.918+0000 7f2d1fe00640 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dd7370 0x558dd0deb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.918+0000 7f2d1fe00640 1 -- 172.21.15.119:0/3189977740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 msgr2=0x558dd0dc86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.918+0000 7f2d1fe00640 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0dc86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.918+0000 7f2d1fe00640 1 -- 172.21.15.119:0/3189977740 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dd0b50f50 con 0x558dd0df4c10 2024-04-25T10:11:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.918+0000 7f2d2d600640 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0dc86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.918+0000 7f2d2cc00640 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dd7370 0x558dd0deb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.918+0000 7f2d1fe00640 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0de9c40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2d10006e80 tx=0x7f2d10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250f84fba789cc83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.919+0000 7f2d15600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:11:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.919+0000 7f2d15600640 1 -- 172.21.15.119:0/3189977740 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d100095f0 con 0x558dd0df4c10 2024-04-25T10:11:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.919+0000 7f2d15600640 1 -- 172.21.15.119:0/3189977740 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d10003610 con 0x558dd0df4c10 2024-04-25T10:11:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.919+0000 7f2d323f1740 1 -- 172.21.15.119:0/3189977740 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d10004020 con 0x558dd0df4c10 2024-04-25T10:11:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.919+0000 7f2d15600640 1 -- 172.21.15.119:0/3189977740 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d1002f050 con 0x558dd0df4c10 2024-04-25T10:11:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.919+0000 7f2d323f1740 10 client.6279 fetch_fsmap learned FSMap version 36 2024-04-25T10:11:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.919+0000 7f2d323f1740 1 -- 172.21.15.119:0/3189977740 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x558dd09e40b0 con 0x558dd0df4c10 2024-04-25T10:11:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.919+0000 7f2d15600640 1 -- 172.21.15.119:0/3189977740 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d10004020 con 0x558dd0df4c10 2024-04-25T10:11:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.919+0000 7f2d15600640 1 -- 172.21.15.119:0/3189977740 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d10032020 con 0x558dd0df4c10 2024-04-25T10:11:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.919+0000 7f2d323f1740 10 client.6279 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T10:11:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.920+0000 7f2d323f1740 10 client.6279 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:11:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.920+0000 7f2d323f1740 20 client.6279 populate_metadata read hostname 'smithi119' 2024-04-25T10:11:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.920+0000 7f2d323f1740 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x558dd0e053c0 0x558dd0e057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.920+0000 7f2d323f1740 4 client.6279 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.920+0000 7f2d323f1740 1 -- 172.21.15.119:0/3189977740 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dd0d6eb60 con 0x558dd0e053c0 2024-04-25T10:11:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.920+0000 7f2d2cc00640 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x558dd0e053c0 0x558dd0e057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.920+0000 7f2d2cc00640 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x558dd0e053c0 0x558dd0e057a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c7ef510341aa727 server_cookie=b752dd9ce55e2f9e in_seq=0 out_seq=0 2024-04-25T10:11:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.920+0000 7f2d15600640 10 client.6279 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:11:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.921+0000 7f2d15600640 1 -- 172.21.15.119:0/3189977740 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558dd0d6eb60 con 0x558dd0e053c0 2024-04-25T10:11:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.921+0000 7f2d15600640 10 client.6279 handle_command_reply: tid=0 2024-04-25T10:11:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.921+0000 7f2d323f1740 1 client.6279 shutdown 2024-04-25T10:11:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.921+0000 7f2d323f1740 1 -- 172.21.15.119:0/3189977740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 msgr2=0x558dd0de9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.921+0000 7f2d323f1740 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0de9c40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2d10006e80 tx=0x7f2d10004730 comp rx=0 tx=0).stop 2024-04-25T10:11:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.921+0000 7f2d323f1740 1 -- 172.21.15.119:0/3189977740 shutdown_connections 2024-04-25T10:11:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.921+0000 7f2d323f1740 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0dc86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.921+0000 7f2d323f1740 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x558dd0e053c0 0x558dd0e057a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.921+0000 7f2d323f1740 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0de9c40 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.921+0000 7f2d323f1740 1 --2- 172.21.15.119:0/3189977740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dd7370 0x558dd0deb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.921+0000 7f2d15600640 0 client.6279 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:11:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.921+0000 7f2d323f1740 1 -- 172.21.15.119:0/3189977740 >> 172.21.15.119:0/3189977740 conn(0x558dd0dad050 msgr2=0x558dd0da6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.922+0000 7f2d323f1740 1 -- 172.21.15.119:0/3189977740 shutdown_connections 2024-04-25T10:11:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.922+0000 7f2d323f1740 1 -- 172.21.15.119:0/3189977740 wait complete. 2024-04-25T10:11:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.922+0000 7f2d323f1740 20 client.6279 trim_cache size 0 max 16384 2024-04-25T10:11:12.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.934+0000 7f2d323f1740 1 Processor -- start 2024-04-25T10:11:12.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.934+0000 7f2d323f1740 1 -- start start 2024-04-25T10:11:12.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.935+0000 7f2d323f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0dd7370 0x558dd0dad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.935+0000 7f2d323f1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0dabf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.935+0000 7f2d323f1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0df4c10 0x558dd0dac470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.935+0000 7f2d323f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d281f7d20 con 0x558dd0dd7370 2024-04-25T10:11:12.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.935+0000 7f2d323f1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d281f7120 con 0x558dd0debc90 2024-04-25T10:11:12.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.935+0000 7f2d323f1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d281f7720 con 0x558dd0df4c10 2024-04-25T10:11:12.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.936+0000 7f2d2cc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0dd7370 0x558dd0dad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.936+0000 7f2d2cc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0dd7370 0x558dd0dad620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44076/0 (socket says 172.21.15.119:44076) 2024-04-25T10:11:12.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.936+0000 7f2d2cc00640 1 -- 172.21.15.119:0/11301453 learned_addr learned my addr 172.21.15.119:0/11301453 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:12.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.936+0000 7f2d2cc00640 1 -- 172.21.15.119:0/11301453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0df4c10 msgr2=0x558dd0dac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.936+0000 7f2d1fe00640 1 --2- 172.21.15.119:0/11301453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0dabf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.936+0000 7f2d2d600640 1 --2- 172.21.15.119:0/11301453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0df4c10 0x558dd0dac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.936+0000 7f2d2cc00640 1 --2- 172.21.15.119:0/11301453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0df4c10 0x558dd0dac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.936+0000 7f2d2cc00640 1 -- 172.21.15.119:0/11301453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 msgr2=0x558dd0dabf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.936+0000 7f2d2cc00640 1 --2- 172.21.15.119:0/11301453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0dabf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.936+0000 7f2d2cc00640 1 -- 172.21.15.119:0/11301453 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dd0ded0e0 con 0x558dd0dd7370 2024-04-25T10:11:12.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.936+0000 7f2d1fe00640 1 --2- 172.21.15.119:0/11301453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0dabf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.936+0000 7f2d2cc00640 1 --2- 172.21.15.119:0/11301453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0dd7370 0x558dd0dad620 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2d180077b0 tx=0x7f2d1800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2209d9e0e53dd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.937+0000 7f2d17e00640 1 -- 172.21.15.119:0/11301453 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18005cc0 con 0x558dd0dd7370 2024-04-25T10:11:12.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.937+0000 7f2d17e00640 1 -- 172.21.15.119:0/11301453 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d180223d0 con 0x558dd0dd7370 2024-04-25T10:11:12.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.937+0000 7f2d17e00640 1 -- 172.21.15.119:0/11301453 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18005e60 con 0x558dd0dd7370 2024-04-25T10:11:12.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.937+0000 7f2d323f1740 1 -- 172.21.15.119:0/11301453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0dd7370 msgr2=0x558dd0dad620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.937+0000 7f2d323f1740 1 --2- 172.21.15.119:0/11301453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0dd7370 0x558dd0dad620 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2d180077b0 tx=0x7f2d1800f000 comp rx=0 tx=0).stop 2024-04-25T10:11:12.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.937+0000 7f2d323f1740 1 -- 172.21.15.119:0/11301453 shutdown_connections 2024-04-25T10:11:12.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.937+0000 7f2d323f1740 1 --2- 172.21.15.119:0/11301453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0df4c10 0x558dd0dac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.937+0000 7f2d323f1740 1 --2- 172.21.15.119:0/11301453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0dabf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.937+0000 7f2d323f1740 1 --2- 172.21.15.119:0/11301453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0dd7370 0x558dd0dad620 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.937+0000 7f2d323f1740 1 -- 172.21.15.119:0/11301453 >> 172.21.15.119:0/11301453 conn(0x558dd0dad050 msgr2=0x558dd0deb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.937+0000 7f2d323f1740 1 -- 172.21.15.119:0/11301453 shutdown_connections 2024-04-25T10:11:12.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.937+0000 7f2d323f1740 1 -- 172.21.15.119:0/11301453 wait complete. 2024-04-25T10:11:12.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.938+0000 7f2d323f1740 1 -- start start 2024-04-25T10:11:12.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.938+0000 7f2d323f1740 1 Processor -- start 2024-04-25T10:11:12.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.939+0000 7f2d323f1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dd7370 0x558dd0e59490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.939+0000 7f2d323f1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0e5f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.939+0000 7f2d323f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0dab020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.939+0000 7f2d323f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd0ded6c0 con 0x558dd0df4c10 2024-04-25T10:11:12.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.939+0000 7f2d323f1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dd0dc90d0 con 0x558dd0debc90 2024-04-25T10:11:12.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.939+0000 7f2d323f1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558dd0dd8f10 con 0x558dd0dd7370 2024-04-25T10:11:12.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.939+0000 7f2d323f1740 10 client.0 _ll_register_callbacks cb 0x558dd0da7d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:12.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.940+0000 7f2d2d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0dab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.940+0000 7f2d2d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0dab020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44088/0 (socket says 172.21.15.119:44088) 2024-04-25T10:11:12.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.940+0000 7f2d2d600640 1 -- 172.21.15.119:0/1029254132 learned_addr learned my addr 172.21.15.119:0/1029254132 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:12.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.940+0000 7f2d2cc00640 1 --2- 172.21.15.119:0/1029254132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dd7370 0x558dd0e59490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.940+0000 7f2d1fe00640 1 --2- 172.21.15.119:0/1029254132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0e5f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.940+0000 7f2d2d600640 1 -- 172.21.15.119:0/1029254132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dd7370 msgr2=0x558dd0e59490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.940+0000 7f2d2d600640 1 --2- 172.21.15.119:0/1029254132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dd7370 0x558dd0e59490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.940+0000 7f2d2d600640 1 -- 172.21.15.119:0/1029254132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 msgr2=0x558dd0e5f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.940+0000 7f2d2d600640 1 --2- 172.21.15.119:0/1029254132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0e5f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.940+0000 7f2d2d600640 1 -- 172.21.15.119:0/1029254132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558dd0e5c8d0 con 0x558dd0df4c10 2024-04-25T10:11:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.940+0000 7f2d1fe00640 1 --2- 172.21.15.119:0/1029254132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0e5f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.940+0000 7f2d2d600640 1 --2- 172.21.15.119:0/1029254132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0dab020 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2d200aa090 tx=0x7f2d200443c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a1453069072ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.940+0000 7f2d15600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:11:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.941+0000 7f2d15600640 1 -- 172.21.15.119:0/1029254132 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2004a050 con 0x558dd0df4c10 2024-04-25T10:11:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.941+0000 7f2d15600640 1 -- 172.21.15.119:0/1029254132 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d2007e0c0 con 0x558dd0df4c10 2024-04-25T10:11:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.941+0000 7f2d323f1740 1 -- 172.21.15.119:0/1029254132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d180223d0 con 0x558dd0df4c10 2024-04-25T10:11:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.941+0000 7f2d15600640 1 -- 172.21.15.119:0/1029254132 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d2007e490 con 0x558dd0df4c10 2024-04-25T10:11:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.941+0000 7f2d323f1740 10 client.6285 fetch_fsmap learned FSMap version 36 2024-04-25T10:11:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.941+0000 7f2d323f1740 1 -- 172.21.15.119:0/1029254132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x558dd0e5fa10 con 0x558dd0df4c10 2024-04-25T10:11:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.941+0000 7f2d15600640 1 -- 172.21.15.119:0/1029254132 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d180223d0 con 0x558dd0df4c10 2024-04-25T10:11:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.941+0000 7f2d15600640 1 -- 172.21.15.119:0/1029254132 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d2007a5a0 con 0x558dd0df4c10 2024-04-25T10:11:12.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.941+0000 7f2d323f1740 10 client.6285 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T10:11:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.941+0000 7f2d323f1740 10 client.6285 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:11:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.941+0000 7f2d323f1740 20 client.6285 populate_metadata read hostname 'smithi119' 2024-04-25T10:11:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.941+0000 7f2d323f1740 1 --2- 172.21.15.119:0/1029254132 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x558dd0dd6870 0x558dd0da66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.941+0000 7f2d323f1740 4 client.6285 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:11:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.941+0000 7f2d323f1740 1 -- 172.21.15.119:0/1029254132 --> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x558dd0d1d020 con 0x558dd0dd6870 2024-04-25T10:11:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.942+0000 7f2d2cc00640 1 --2- 172.21.15.119:0/1029254132 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x558dd0dd6870 0x558dd0da66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.942+0000 7f2d2cc00640 1 --2- 172.21.15.119:0/1029254132 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x558dd0dd6870 0x558dd0da66e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68473bb018240415 server_cookie=2ac23fb8c2c283fd in_seq=0 out_seq=0 2024-04-25T10:11:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.942+0000 7f2d15600640 10 client.6285 ms_handle_connect on v2:172.21.15.119:6835/4129112885 2024-04-25T10:11:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.942+0000 7f2d15600640 1 -- 172.21.15.119:0/1029254132 <== mds.0 v2:172.21.15.119:6835/4129112885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558dd0d1d020 con 0x558dd0dd6870 2024-04-25T10:11:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.942+0000 7f2d15600640 10 client.6285 handle_command_reply: tid=0 2024-04-25T10:11:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.942+0000 7f2d323f1740 1 client.6285 shutdown 2024-04-25T10:11:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.943+0000 7f2d323f1740 1 -- 172.21.15.119:0/1029254132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 msgr2=0x558dd0dab020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.943+0000 7f2d323f1740 1 --2- 172.21.15.119:0/1029254132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0dab020 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2d200aa090 tx=0x7f2d200443c0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.943+0000 7f2d323f1740 1 -- 172.21.15.119:0/1029254132 shutdown_connections 2024-04-25T10:11:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.943+0000 7f2d323f1740 1 --2- 172.21.15.119:0/1029254132 >> [v2:172.21.15.119:6835/4129112885,v1:172.21.15.119:6836/4129112885] conn(0x558dd0dd6870 0x558dd0da66e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.943+0000 7f2d323f1740 1 --2- 172.21.15.119:0/1029254132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558dd0df4c10 0x558dd0dab020 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.943+0000 7f2d323f1740 1 --2- 172.21.15.119:0/1029254132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558dd0debc90 0x558dd0e5f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.943+0000 7f2d323f1740 1 --2- 172.21.15.119:0/1029254132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558dd0dd7370 0x558dd0e59490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.943+0000 7f2d15600640 0 client.6285 ms_handle_reset on v2:172.21.15.119:6835/4129112885 2024-04-25T10:11:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.943+0000 7f2d323f1740 1 -- 172.21.15.119:0/1029254132 >> 172.21.15.119:0/1029254132 conn(0x558dd0dad050 msgr2=0x558dd0e0b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.943+0000 7f2d323f1740 1 -- 172.21.15.119:0/1029254132 shutdown_connections 2024-04-25T10:11:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.943+0000 7f2d323f1740 1 -- 172.21.15.119:0/1029254132 wait complete. 2024-04-25T10:11:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.943+0000 7f2d323f1740 20 client.6285 trim_cache size 0 max 16384 2024-04-25T10:11:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:12.953 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2024-04-25T10:11:12.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.944+0000 7f2d2f600640 1 -- 172.21.15.119:0/3987064115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d0406b4d0 msgr2=0x7f2d0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.944+0000 7f2d2f600640 1 --2- 172.21.15.119:0/3987064115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d0406b4d0 0x7f2d0406d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2d18007fd0 tx=0x7f2d18007960 comp rx=0 tx=0).stop 2024-04-25T10:11:12.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.944+0000 7f2d2f600640 1 -- 172.21.15.119:0/3987064115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2815a260 msgr2=0x7f2d281f3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.944+0000 7f2d2f600640 1 --2- 172.21.15.119:0/3987064115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2815a260 0x7f2d281f3b30 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2d2002ac70 tx=0x7f2d20004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:12.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.944+0000 7f2d2f600640 1 -- 172.21.15.119:0/3987064115 shutdown_connections 2024-04-25T10:11:12.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.944+0000 7f2d2f600640 1 --2- 172.21.15.119:0/3987064115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d0406b4d0 0x7f2d0406d990 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.944+0000 7f2d2f600640 1 --2- 172.21.15.119:0/3987064115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d2815a260 0x7f2d281f3b30 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.944+0000 7f2d2f600640 1 --2- 172.21.15.119:0/3987064115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d28071930 0x7f2d281581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.944+0000 7f2d2f600640 1 --2- 172.21.15.119:0/3987064115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2806ea30 0x7f2d28152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.944+0000 7f2d2f600640 1 -- 172.21.15.119:0/3987064115 >> 172.21.15.119:0/3987064115 conn(0x7f2d2806aba0 msgr2=0x7f2d2806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.944+0000 7f2d2f600640 1 -- 172.21.15.119:0/3987064115 shutdown_connections 2024-04-25T10:11:12.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:12.944+0000 7f2d2f600640 1 -- 172.21.15.119:0/3987064115 wait complete. 2024-04-25T10:11:12.968 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:11:13.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.151+0000 7f4620200640 1 Processor -- start 2024-04-25T10:11:13.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.152+0000 7f4620200640 1 -- start start 2024-04-25T10:11:13.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.153+0000 7f4620200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46180d35b0 0x7f46180b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.153+0000 7f4620200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f46180b5990 0x7f46180b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.153+0000 7f4620200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461815e0c0 con 0x7f461815dce0 2024-04-25T10:11:13.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.153+0000 7f4620200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46180b5de0 con 0x7f46180b5990 2024-04-25T10:11:13.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.153+0000 7f4620200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46180d2510 con 0x7f46180d35b0 2024-04-25T10:11:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.153+0000 7f461ce00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46180d35b0 0x7f46180b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.154+0000 7f461ce00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46180d35b0 0x7f46180b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:35770/0 (socket says 172.21.15.119:35770) 2024-04-25T10:11:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.154+0000 7f461ce00640 1 -- 172.21.15.119:0/4170423075 learned_addr learned my addr 172.21.15.119:0/4170423075 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.154+0000 7f461e200640 1 --2- 172.21.15.119:0/4170423075 >> v2:172.21.15.132:3300/0 conn(0x7f46180b5990 0x7f46180b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.154+0000 7f461ce00640 1 -- 172.21.15.119:0/4170423075 >> v2:172.21.15.132:3300/0 conn(0x7f46180b5990 msgr2=0x7f46180b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.154+0000 7f461ce00640 1 --2- 172.21.15.119:0/4170423075 >> v2:172.21.15.132:3300/0 conn(0x7f46180b5990 0x7f46180b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.154+0000 7f461ce00640 1 -- 172.21.15.119:0/4170423075 >> v1:172.21.15.132:6789/0 conn(0x7f461815dce0 legacy=0x7f46180d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.154+0000 7f461ce00640 1 -- 172.21.15.119:0/4170423075 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46180d27f0 con 0x7f46180d35b0 2024-04-25T10:11:13.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.156+0000 7f461ce00640 1 --2- 172.21.15.119:0/4170423075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46180d35b0 0x7f46180b5450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4604009870 tx=0x7f460402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c295a31151b62dff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.156+0000 7f4617e00640 1 -- 172.21.15.119:0/4170423075 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604037070 con 0x7f46180d35b0 2024-04-25T10:11:13.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.156+0000 7f4617e00640 1 -- 172.21.15.119:0/4170423075 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4604032440 con 0x7f46180d35b0 2024-04-25T10:11:13.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.156+0000 7f4617e00640 1 -- 172.21.15.119:0/4170423075 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604032810 con 0x7f46180d35b0 2024-04-25T10:11:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.157+0000 7f4620200640 1 -- 172.21.15.119:0/4170423075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46180d35b0 msgr2=0x7f46180b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.157+0000 7f4620200640 1 --2- 172.21.15.119:0/4170423075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46180d35b0 0x7f46180b5450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4604009870 tx=0x7f460402a250 comp rx=0 tx=0).stop 2024-04-25T10:11:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.157+0000 7f4620200640 1 -- 172.21.15.119:0/4170423075 shutdown_connections 2024-04-25T10:11:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.157+0000 7f4620200640 1 --2- 172.21.15.119:0/4170423075 >> v2:172.21.15.132:3300/0 conn(0x7f46180b5990 0x7f46180b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.157+0000 7f4620200640 1 --2- 172.21.15.119:0/4170423075 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46180d35b0 0x7f46180b5450 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.157+0000 7f4620200640 1 -- 172.21.15.119:0/4170423075 >> 172.21.15.119:0/4170423075 conn(0x7f4618072900 msgr2=0x7f46180d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.157+0000 7f4620200640 1 -- 172.21.15.119:0/4170423075 shutdown_connections 2024-04-25T10:11:13.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.157+0000 7f4620200640 1 -- 172.21.15.119:0/4170423075 wait complete. 2024-04-25T10:11:13.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.159+0000 7f4620200640 1 Processor -- start 2024-04-25T10:11:13.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.159+0000 7f4620200640 1 -- start start 2024-04-25T10:11:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.160+0000 7f4620200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46180b5990 0x7f4618151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.160+0000 7f4620200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46180d35b0 0x7f4618151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.160+0000 7f4620200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f461815dce0 0x7f46181598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.160+0000 7f4620200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46180d3090 con 0x7f46180d35b0 2024-04-25T10:11:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.160+0000 7f4620200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46180d2db0 con 0x7f461815dce0 2024-04-25T10:11:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.160+0000 7f4620200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46180d2160 con 0x7f46180b5990 2024-04-25T10:11:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.160+0000 7f461ce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46180d35b0 0x7f4618151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.160+0000 7f461ce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46180d35b0 0x7f4618151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44098/0 (socket says 172.21.15.119:44098) 2024-04-25T10:11:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.160+0000 7f461ce00640 1 -- 172.21.15.119:0/542357601 learned_addr learned my addr 172.21.15.119:0/542357601 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:13.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.160+0000 7f461d800640 1 --2- 172.21.15.119:0/542357601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46180b5990 0x7f4618151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.160+0000 7f461e200640 1 --2- 172.21.15.119:0/542357601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f461815dce0 0x7f46181598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.160+0000 7f461ce00640 1 -- 172.21.15.119:0/542357601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46180b5990 msgr2=0x7f4618151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.160+0000 7f461ce00640 1 --2- 172.21.15.119:0/542357601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46180b5990 0x7f4618151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.160+0000 7f461ce00640 1 -- 172.21.15.119:0/542357601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f461815dce0 msgr2=0x7f46181598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.161+0000 7f461ce00640 1 --2- 172.21.15.119:0/542357601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f461815dce0 0x7f46181598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.161+0000 7f461ce00640 1 -- 172.21.15.119:0/542357601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46181600f0 con 0x7f46180d35b0 2024-04-25T10:11:13.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.161+0000 7f461ce00640 1 --2- 172.21.15.119:0/542357601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46180d35b0 0x7f4618151560 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4604000c00 tx=0x7f46040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.161+0000 7f4615600640 1 -- 172.21.15.119:0/542357601 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604037040 con 0x7f46180d35b0 2024-04-25T10:11:13.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.161+0000 7f4615600640 1 -- 172.21.15.119:0/542357601 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4604042070 con 0x7f46180d35b0 2024-04-25T10:11:13.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.161+0000 7f4620200640 1 -- 172.21.15.119:0/542357601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4618160430 con 0x7f46180d35b0 2024-04-25T10:11:13.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.161+0000 7f4615600640 1 -- 172.21.15.119:0/542357601 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604030880 con 0x7f46180d35b0 2024-04-25T10:11:13.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.161+0000 7f4620200640 1 -- 172.21.15.119:0/542357601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4618160c70 con 0x7f46180d35b0 2024-04-25T10:11:13.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.163+0000 7f4620200640 1 -- 172.21.15.119:0/542357601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4618067490 con 0x7f46180d35b0 2024-04-25T10:11:13.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.164+0000 7f4615600640 1 -- 172.21.15.119:0/542357601 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4604030a20 con 0x7f46180d35b0 2024-04-25T10:11:13.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.164+0000 7f4615600640 1 --2- 172.21.15.119:0/542357601 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f45f806b500 0x7f45f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.165+0000 7f4615600640 1 -- 172.21.15.119:0/542357601 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46040aaaf0 con 0x7f46180d35b0 2024-04-25T10:11:13.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.165+0000 7f461d800640 1 --2- 172.21.15.119:0/542357601 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f45f806b500 0x7f45f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.165+0000 7f461d800640 1 --2- 172.21.15.119:0/542357601 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f45f806b500 0x7f45f806d9c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4608007fd0 tx=0x7f4608009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.170+0000 7f4615600640 1 -- 172.21.15.119:0/542357601 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4618067490 con 0x7f46180d35b0 2024-04-25T10:11:13.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.403+0000 7f4620200640 1 -- 172.21.15.119:0/542357601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f46180749b0 con 0x7f46180d35b0 2024-04-25T10:11:13.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.404+0000 7f4615600640 1 -- 172.21.15.119:0/542357601 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4618160430 con 0x7f46180d35b0 2024-04-25T10:11:13.410 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:13.410 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:11:13.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.406+0000 7f4620200640 1 -- 172.21.15.119:0/542357601 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f45f806b500 msgr2=0x7f45f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.406+0000 7f4620200640 1 --2- 172.21.15.119:0/542357601 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f45f806b500 0x7f45f806d9c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4608007fd0 tx=0x7f4608009040 comp rx=0 tx=0).stop 2024-04-25T10:11:13.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.406+0000 7f4620200640 1 -- 172.21.15.119:0/542357601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46180d35b0 msgr2=0x7f4618151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.406+0000 7f4620200640 1 --2- 172.21.15.119:0/542357601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46180d35b0 0x7f4618151560 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4604000c00 tx=0x7f46040046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.406+0000 7f4620200640 1 -- 172.21.15.119:0/542357601 shutdown_connections 2024-04-25T10:11:13.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.406+0000 7f4620200640 1 --2- 172.21.15.119:0/542357601 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f45f806b500 0x7f45f806d9c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.406+0000 7f4620200640 1 --2- 172.21.15.119:0/542357601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f461815dce0 0x7f46181598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.406+0000 7f4620200640 1 --2- 172.21.15.119:0/542357601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46180d35b0 0x7f4618151560 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.406+0000 7f4620200640 1 --2- 172.21.15.119:0/542357601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46180b5990 0x7f4618151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.406+0000 7f4620200640 1 -- 172.21.15.119:0/542357601 >> 172.21.15.119:0/542357601 conn(0x7f4618072900 msgr2=0x7f46180c6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.407+0000 7f4620200640 1 -- 172.21.15.119:0/542357601 shutdown_connections 2024-04-25T10:11:13.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.407+0000 7f4620200640 1 -- 172.21.15.119:0/542357601 wait complete. 2024-04-25T10:11:13.432 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:11:13.432 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:11:13.432 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:13.539 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:13.643 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:11:13.643 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:11:13.643 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:13.701 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:11:13.702 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:13.755 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:13.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.945+0000 7f8c9ac00640 1 Processor -- start 2024-04-25T10:11:13.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.945+0000 7f8c9ac00640 1 -- start start 2024-04-25T10:11:13.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.946+0000 7f8c9ac00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c9415bf00 0x7f8c940cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.947+0000 7f8c9ac00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8c940cc310 0x7f8c9415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.947+0000 7f8c9ac00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c940ce6c0 con 0x7f8c9415c820 2024-04-25T10:11:13.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.947+0000 7f8c9ac00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c941455e0 con 0x7f8c9415bf00 2024-04-25T10:11:13.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.947+0000 7f8c9ac00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9406d450 con 0x7f8c940cc310 2024-04-25T10:11:13.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.947+0000 7f8c93e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c9415bf00 0x7f8c940cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.947+0000 7f8c98c00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8c9415c820 0x7f8c9406d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54908/0 (socket says 172.21.15.119:54908) 2024-04-25T10:11:13.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.947+0000 7f8c98c00640 1 -- 172.21.15.119:0/2606020330 learned_addr learned my addr 172.21.15.119:0/2606020330 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:13.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.947+0000 7f8c93400640 1 --2- 172.21.15.119:0/2606020330 >> v2:172.21.15.132:3300/0 conn(0x7f8c940cc310 0x7f8c9415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.947+0000 7f8c93e00640 1 -- 172.21.15.119:0/2606020330 >> v2:172.21.15.132:3300/0 conn(0x7f8c940cc310 msgr2=0x7f8c9415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.948+0000 7f8c93e00640 1 --2- 172.21.15.119:0/2606020330 >> v2:172.21.15.132:3300/0 conn(0x7f8c940cc310 0x7f8c9415c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.948+0000 7f8c93e00640 1 -- 172.21.15.119:0/2606020330 >> v1:172.21.15.119:6789/0 conn(0x7f8c9415c820 legacy=0x7f8c9406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.948+0000 7f8c93e00640 1 -- 172.21.15.119:0/2606020330 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c940cc7b0 con 0x7f8c9415bf00 2024-04-25T10:11:13.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.948+0000 7f8c92a00640 1 -- 172.21.15.119:0/2606020330 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663362286 0 0) 0x7f8c8c001490 con 0x7f8c9415c820 2024-04-25T10:11:13.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.949+0000 7f8c93e00640 1 --2- 172.21.15.119:0/2606020330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c9415bf00 0x7f8c940cbdd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8c9406b900 tx=0x7f8c8402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf4ba659021993b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.949+0000 7f8c92a00640 1 -- 172.21.15.119:0/2606020330 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84037070 con 0x7f8c9415bf00 2024-04-25T10:11:13.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.949+0000 7f8c92a00640 1 -- 172.21.15.119:0/2606020330 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c84032440 con 0x7f8c9415bf00 2024-04-25T10:11:13.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.949+0000 7f8c92a00640 1 -- 172.21.15.119:0/2606020330 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84032810 con 0x7f8c9415bf00 2024-04-25T10:11:13.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.949+0000 7f8c9ac00640 1 -- 172.21.15.119:0/2606020330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c9415bf00 msgr2=0x7f8c940cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.949+0000 7f8c9ac00640 1 --2- 172.21.15.119:0/2606020330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c9415bf00 0x7f8c940cbdd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8c9406b900 tx=0x7f8c8402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.950+0000 7f8c9ac00640 1 -- 172.21.15.119:0/2606020330 shutdown_connections 2024-04-25T10:11:13.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.950+0000 7f8c9ac00640 1 --2- 172.21.15.119:0/2606020330 >> v2:172.21.15.132:3300/0 conn(0x7f8c940cc310 0x7f8c9415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.950+0000 7f8c9ac00640 1 --2- 172.21.15.119:0/2606020330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c9415bf00 0x7f8c940cbdd0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.950+0000 7f8c9ac00640 1 -- 172.21.15.119:0/2606020330 >> 172.21.15.119:0/2606020330 conn(0x7f8c9406aba0 msgr2=0x7f8c9406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.950+0000 7f8c9ac00640 1 -- 172.21.15.119:0/2606020330 shutdown_connections 2024-04-25T10:11:13.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.950+0000 7f8c9ac00640 1 -- 172.21.15.119:0/2606020330 wait complete. 2024-04-25T10:11:13.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.951+0000 7f8c9ac00640 1 Processor -- start 2024-04-25T10:11:13.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.952+0000 7f8c9ac00640 1 -- start start 2024-04-25T10:11:13.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.952+0000 7f8c9ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c940cc310 0x7f8c941e7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.952+0000 7f8c9ac00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9415bf00 0x7f8c940ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.952+0000 7f8c9ac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c9415c820 0x7f8c940caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.952+0000 7f8c9ac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940cd050 con 0x7f8c9415c820 2024-04-25T10:11:13.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.952+0000 7f8c9ac00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940ccd70 con 0x7f8c9415bf00 2024-04-25T10:11:13.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.952+0000 7f8c9ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9415ccd0 con 0x7f8c940cc310 2024-04-25T10:11:13.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.953+0000 7f8c98c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c9415c820 0x7f8c940caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.953+0000 7f8c98c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c9415c820 0x7f8c940caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44118/0 (socket says 172.21.15.119:44118) 2024-04-25T10:11:13.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.953+0000 7f8c98c00640 1 -- 172.21.15.119:0/1781640166 learned_addr learned my addr 172.21.15.119:0/1781640166 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:13.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.953+0000 7f8c93e00640 1 --2- 172.21.15.119:0/1781640166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c940cc310 0x7f8c941e7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.953+0000 7f8c98c00640 1 -- 172.21.15.119:0/1781640166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c940cc310 msgr2=0x7f8c941e7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.953+0000 7f8c98c00640 1 --2- 172.21.15.119:0/1781640166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c940cc310 0x7f8c941e7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.953+0000 7f8c93400640 1 --2- 172.21.15.119:0/1781640166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9415bf00 0x7f8c940ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.953+0000 7f8c98c00640 1 -- 172.21.15.119:0/1781640166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9415bf00 msgr2=0x7f8c940ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.953+0000 7f8c98c00640 1 --2- 172.21.15.119:0/1781640166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9415bf00 0x7f8c940ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.953+0000 7f8c98c00640 1 -- 172.21.15.119:0/1781640166 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c941e4740 con 0x7f8c9415c820 2024-04-25T10:11:13.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.954+0000 7f8c98c00640 1 --2- 172.21.15.119:0/1781640166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c9415c820 0x7f8c940caa80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00c730 tx=0x7f8c8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.954+0000 7f8c8be00640 1 -- 172.21.15.119:0/1781640166 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c018070 con 0x7f8c9415c820 2024-04-25T10:11:13.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.954+0000 7f8c8be00640 1 -- 172.21.15.119:0/1781640166 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c00ce40 con 0x7f8c9415c820 2024-04-25T10:11:13.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.954+0000 7f8c8be00640 1 -- 172.21.15.119:0/1781640166 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00a860 con 0x7f8c9415c820 2024-04-25T10:11:13.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.954+0000 7f8c9ac00640 1 -- 172.21.15.119:0/1781640166 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c941e4a20 con 0x7f8c9415c820 2024-04-25T10:11:13.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.954+0000 7f8c9ac00640 1 -- 172.21.15.119:0/1781640166 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c941e4fe0 con 0x7f8c9415c820 2024-04-25T10:11:13.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.955+0000 7f8c9ac00640 1 -- 172.21.15.119:0/1781640166 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c940749b0 con 0x7f8c9415c820 2024-04-25T10:11:13.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.956+0000 7f8c8be00640 1 -- 172.21.15.119:0/1781640166 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c8c00aa00 con 0x7f8c9415c820 2024-04-25T10:11:13.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.957+0000 7f8c8be00640 1 --2- 172.21.15.119:0/1781640166 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.957+0000 7f8c8be00640 1 -- 172.21.15.119:0/1781640166 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c8c089930 con 0x7f8c9415c820 2024-04-25T10:11:13.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.957+0000 7f8c93e00640 1 --2- 172.21.15.119:0/1781640166 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.958+0000 7f8c93e00640 1 --2- 172.21.15.119:0/1781640166 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8c7006b4d0 0x7f8c7006d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8c84034000 tx=0x7f8c8402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:13.961+0000 7f8c8be00640 1 -- 172.21.15.119:0/1781640166 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c940ce6c0 con 0x7f8c9415c820 2024-04-25T10:11:14.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.159+0000 7f8c9ac00640 1 -- 172.21.15.119:0/1781640166 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c9415ed60 con 0x7f8c9415c820 2024-04-25T10:11:14.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.162+0000 7f8c8be00640 1 -- 172.21.15.119:0/1781640166 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16701 (secure 0 0 0) 0x7f8c8c058680 con 0x7f8c9415c820 2024-04-25T10:11:14.168 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:14.168 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":55,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:10:01.035989+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:57.237546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:58.475727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:14.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.165+0000 7f8c9ac00640 1 -- 172.21.15.119:0/1781640166 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8c7006b4d0 msgr2=0x7f8c7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.165+0000 7f8c9ac00640 1 --2- 172.21.15.119:0/1781640166 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8c7006b4d0 0x7f8c7006d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8c84034000 tx=0x7f8c8402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:14.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.165+0000 7f8c9ac00640 1 -- 172.21.15.119:0/1781640166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c9415c820 msgr2=0x7f8c940caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.165+0000 7f8c9ac00640 1 --2- 172.21.15.119:0/1781640166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c9415c820 0x7f8c940caa80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00c730 tx=0x7f8c8c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.165+0000 7f8c9ac00640 1 -- 172.21.15.119:0/1781640166 shutdown_connections 2024-04-25T10:11:14.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.165+0000 7f8c9ac00640 1 --2- 172.21.15.119:0/1781640166 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.165+0000 7f8c9ac00640 1 --2- 172.21.15.119:0/1781640166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c9415c820 0x7f8c940caa80 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.165+0000 7f8c9ac00640 1 --2- 172.21.15.119:0/1781640166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c9415bf00 0x7f8c940ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.165+0000 7f8c9ac00640 1 --2- 172.21.15.119:0/1781640166 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c940cc310 0x7f8c941e7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.165+0000 7f8c9ac00640 1 -- 172.21.15.119:0/1781640166 >> 172.21.15.119:0/1781640166 conn(0x7f8c9406aba0 msgr2=0x7f8c9413b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.166+0000 7f8c9ac00640 1 -- 172.21.15.119:0/1781640166 shutdown_connections 2024-04-25T10:11:14.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.166+0000 7f8c9ac00640 1 -- 172.21.15.119:0/1781640166 wait complete. 2024-04-25T10:11:14.190 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:09:57.237546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:09:58.475727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:14.191 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:14.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.384+0000 7f7b9ca00640 1 Processor -- start 2024-04-25T10:11:14.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.384+0000 7f7b9ca00640 1 -- start start 2024-04-25T10:11:14.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.386+0000 7f7b9ca00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7b981450e0 0x7f7b9806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.386+0000 7f7b9ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b9806b0e0 0x7f7b980cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.386+0000 7f7b9ca00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7b980cc810 0x7f7b98071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.386+0000 7f7b9ca00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b981454c0 con 0x7f7b980cc810 2024-04-25T10:11:14.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.386+0000 7f7b9ca00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9806b540 con 0x7f7b981450e0 2024-04-25T10:11:14.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.386+0000 7f7b9ca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9815f400 con 0x7f7b9806b0e0 2024-04-25T10:11:14.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.386+0000 7f7b97200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7b980cc810 0x7f7b98071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.386+0000 7f7b97200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7b980cc810 0x7f7b98071050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44126/0 (socket says 172.21.15.119:44126) 2024-04-25T10:11:14.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.386+0000 7f7b97200640 1 -- 172.21.15.119:0/4121112204 learned_addr learned my addr 172.21.15.119:0/4121112204 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:14.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.387+0000 7f7b8fe00640 1 --2- 172.21.15.119:0/4121112204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b9806b0e0 0x7f7b980cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.387+0000 7f7b96800640 1 --2- 172.21.15.119:0/4121112204 >> v2:172.21.15.132:3300/0 conn(0x7f7b981450e0 0x7f7b9806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.387+0000 7f7b97200640 1 -- 172.21.15.119:0/4121112204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b9806b0e0 msgr2=0x7f7b980cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.387+0000 7f7b97200640 1 --2- 172.21.15.119:0/4121112204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b9806b0e0 0x7f7b980cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.387+0000 7f7b97200640 1 -- 172.21.15.119:0/4121112204 >> v2:172.21.15.132:3300/0 conn(0x7f7b981450e0 msgr2=0x7f7b9806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.387+0000 7f7b97200640 1 --2- 172.21.15.119:0/4121112204 >> v2:172.21.15.132:3300/0 conn(0x7f7b981450e0 0x7f7b9806aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.387+0000 7f7b97200640 1 -- 172.21.15.119:0/4121112204 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9815f6e0 con 0x7f7b980cc810 2024-04-25T10:11:14.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.388+0000 7f7b97200640 1 --2- 172.21.15.119:0/4121112204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b980cc810 0x7f7b98071050 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7b98072170 tx=0x7f7b9002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e47001c75ac027 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.388+0000 7f7b95e00640 1 -- 172.21.15.119:0/4121112204 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90036070 con 0x7f7b980cc810 2024-04-25T10:11:14.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.388+0000 7f7b95e00640 1 -- 172.21.15.119:0/4121112204 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b90031600 con 0x7f7b980cc810 2024-04-25T10:11:14.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.388+0000 7f7b95e00640 1 -- 172.21.15.119:0/4121112204 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b900319d0 con 0x7f7b980cc810 2024-04-25T10:11:14.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.389+0000 7f7b9ca00640 1 -- 172.21.15.119:0/4121112204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b980cc810 msgr2=0x7f7b98071050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.389+0000 7f7b9ca00640 1 --2- 172.21.15.119:0/4121112204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b980cc810 0x7f7b98071050 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7b98072170 tx=0x7f7b9002a570 comp rx=0 tx=0).stop 2024-04-25T10:11:14.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.389+0000 7f7b9ca00640 1 -- 172.21.15.119:0/4121112204 shutdown_connections 2024-04-25T10:11:14.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.389+0000 7f7b9ca00640 1 --2- 172.21.15.119:0/4121112204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b980cc810 0x7f7b98071050 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.389+0000 7f7b9ca00640 1 --2- 172.21.15.119:0/4121112204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b9806b0e0 0x7f7b980cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.389+0000 7f7b9ca00640 1 --2- 172.21.15.119:0/4121112204 >> v2:172.21.15.132:3300/0 conn(0x7f7b981450e0 0x7f7b9806aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.389+0000 7f7b9ca00640 1 -- 172.21.15.119:0/4121112204 >> 172.21.15.119:0/4121112204 conn(0x7f7b98066ff0 msgr2=0x7f7b980b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.389+0000 7f7b9ca00640 1 -- 172.21.15.119:0/4121112204 shutdown_connections 2024-04-25T10:11:14.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.389+0000 7f7b9ca00640 1 -- 172.21.15.119:0/4121112204 wait complete. 2024-04-25T10:11:14.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.391+0000 7f7b9ca00640 1 Processor -- start 2024-04-25T10:11:14.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.391+0000 7f7b9ca00640 1 -- start start 2024-04-25T10:11:14.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.392+0000 7f7b9ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b9806b0e0 0x7f7b9815b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.392+0000 7f7b9ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b980cc810 0x7f7b9815d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.392+0000 7f7b9ca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b981450e0 0x7f7b980c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.392+0000 7f7b9ca00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980ccca0 con 0x7f7b9806b0e0 2024-04-25T10:11:14.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.392+0000 7f7b9ca00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9815fca0 con 0x7f7b981450e0 2024-04-25T10:11:14.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.392+0000 7f7b9ca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9815df60 con 0x7f7b980cc810 2024-04-25T10:11:14.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.392+0000 7f7b96800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b9806b0e0 0x7f7b9815b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.392+0000 7f7b96800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b9806b0e0 0x7f7b9815b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44136/0 (socket says 172.21.15.119:44136) 2024-04-25T10:11:14.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.392+0000 7f7b96800640 1 -- 172.21.15.119:0/2944680413 learned_addr learned my addr 172.21.15.119:0/2944680413 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:14.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.393+0000 7f7b96800640 1 -- 172.21.15.119:0/2944680413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b980cc810 msgr2=0x7f7b9815d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.393+0000 7f7b96800640 1 --2- 172.21.15.119:0/2944680413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b980cc810 0x7f7b9815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.393+0000 7f7b96800640 1 -- 172.21.15.119:0/2944680413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b981450e0 msgr2=0x7f7b980c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.393+0000 7f7b96800640 1 --2- 172.21.15.119:0/2944680413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b981450e0 0x7f7b980c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.393+0000 7f7b96800640 1 -- 172.21.15.119:0/2944680413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9815e240 con 0x7f7b9806b0e0 2024-04-25T10:11:14.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.393+0000 7f7b96800640 1 --2- 172.21.15.119:0/2944680413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b9806b0e0 0x7f7b9815b490 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800c7f0 tx=0x7f7b8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.393+0000 7f7b8ea00640 1 -- 172.21.15.119:0/2944680413 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88015070 con 0x7f7b9806b0e0 2024-04-25T10:11:14.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.393+0000 7f7b8ea00640 1 -- 172.21.15.119:0/2944680413 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b88004240 con 0x7f7b9806b0e0 2024-04-25T10:11:14.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.393+0000 7f7b8ea00640 1 -- 172.21.15.119:0/2944680413 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88007620 con 0x7f7b9806b0e0 2024-04-25T10:11:14.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.393+0000 7f7b9ca00640 1 -- 172.21.15.119:0/2944680413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9815e520 con 0x7f7b9806b0e0 2024-04-25T10:11:14.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.393+0000 7f7b9ca00640 1 -- 172.21.15.119:0/2944680413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9815eae0 con 0x7f7b9806b0e0 2024-04-25T10:11:14.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.394+0000 7f7b9ca00640 1 -- 172.21.15.119:0/2944680413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b980749b0 con 0x7f7b9806b0e0 2024-04-25T10:11:14.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.396+0000 7f7b8ea00640 1 -- 172.21.15.119:0/2944680413 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b880079d0 con 0x7f7b9806b0e0 2024-04-25T10:11:14.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.397+0000 7f7b8ea00640 1 --2- 172.21.15.119:0/2944680413 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7b7406b500 0x7f7b7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.397+0000 7f7b8ea00640 1 -- 172.21.15.119:0/2944680413 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b8808a3a0 con 0x7f7b9806b0e0 2024-04-25T10:11:14.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.397+0000 7f7b8fe00640 1 --2- 172.21.15.119:0/2944680413 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7b7406b500 0x7f7b7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.398+0000 7f7b8fe00640 1 --2- 172.21.15.119:0/2944680413 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7b7406b500 0x7f7b7406d9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7b80006fd0 tx=0x7f7b80008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.401+0000 7f7b8ea00640 1 -- 172.21.15.119:0/2944680413 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b88059180 con 0x7f7b9806b0e0 2024-04-25T10:11:14.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.593+0000 7f7b9ca00640 1 -- 172.21.15.119:0/2944680413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b980cb710 con 0x7f7b9806b0e0 2024-04-25T10:11:14.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.594+0000 7f7b8ea00640 1 -- 172.21.15.119:0/2944680413 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b88058a80 con 0x7f7b9806b0e0 2024-04-25T10:11:14.600 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:11:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.598+0000 7f7b9ca00640 1 -- 172.21.15.119:0/2944680413 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7b7406b500 msgr2=0x7f7b7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.598+0000 7f7b9ca00640 1 --2- 172.21.15.119:0/2944680413 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7b7406b500 0x7f7b7406d9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7b80006fd0 tx=0x7f7b80008040 comp rx=0 tx=0).stop 2024-04-25T10:11:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.598+0000 7f7b9ca00640 1 -- 172.21.15.119:0/2944680413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b9806b0e0 msgr2=0x7f7b9815b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.598+0000 7f7b9ca00640 1 --2- 172.21.15.119:0/2944680413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b9806b0e0 0x7f7b9815b490 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800c7f0 tx=0x7f7b8800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.598+0000 7f7b9ca00640 1 -- 172.21.15.119:0/2944680413 shutdown_connections 2024-04-25T10:11:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.598+0000 7f7b9ca00640 1 --2- 172.21.15.119:0/2944680413 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7b7406b500 0x7f7b7406d9c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.598+0000 7f7b9ca00640 1 --2- 172.21.15.119:0/2944680413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b981450e0 0x7f7b980c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.598+0000 7f7b9ca00640 1 --2- 172.21.15.119:0/2944680413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b980cc810 0x7f7b9815d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.598+0000 7f7b9ca00640 1 --2- 172.21.15.119:0/2944680413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b9806b0e0 0x7f7b9815b490 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.598+0000 7f7b9ca00640 1 -- 172.21.15.119:0/2944680413 >> 172.21.15.119:0/2944680413 conn(0x7f7b98066ff0 msgr2=0x7f7b980b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.598+0000 7f7b9ca00640 1 -- 172.21.15.119:0/2944680413 shutdown_connections 2024-04-25T10:11:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.598+0000 7f7b9ca00640 1 -- 172.21.15.119:0/2944680413 wait complete. 2024-04-25T10:11:14.622 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:14.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.818+0000 7fe917200640 1 Processor -- start 2024-04-25T10:11:14.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.818+0000 7fe917200640 1 -- start start 2024-04-25T10:11:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.820+0000 7fe917200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe9181450e0 0x7fe91806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.820+0000 7fe917200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe91806b0e0 0x7fe9180cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.820+0000 7fe917200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9180cc810 0x7fe918071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.820+0000 7fe917200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9181454c0 con 0x7fe9180cc810 2024-04-25T10:11:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.820+0000 7fe917200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91806b540 con 0x7fe9181450e0 2024-04-25T10:11:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.820+0000 7fe917200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91815f400 con 0x7fe91806b0e0 2024-04-25T10:11:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.820+0000 7fe916800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9180cc810 0x7fe918071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.820+0000 7fe916800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9180cc810 0x7fe918071050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44148/0 (socket says 172.21.15.119:44148) 2024-04-25T10:11:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.820+0000 7fe916800640 1 -- 172.21.15.119:0/2672789393 learned_addr learned my addr 172.21.15.119:0/2672789393 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.820+0000 7fe915400640 1 --2- 172.21.15.119:0/2672789393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe91806b0e0 0x7fe9180cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.820+0000 7fe915e00640 1 --2- 172.21.15.119:0/2672789393 >> v2:172.21.15.132:3300/0 conn(0x7fe9181450e0 0x7fe91806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.821+0000 7fe916800640 1 -- 172.21.15.119:0/2672789393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe91806b0e0 msgr2=0x7fe9180cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.821+0000 7fe916800640 1 --2- 172.21.15.119:0/2672789393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe91806b0e0 0x7fe9180cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.821+0000 7fe916800640 1 -- 172.21.15.119:0/2672789393 >> v2:172.21.15.132:3300/0 conn(0x7fe9181450e0 msgr2=0x7fe91806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.821+0000 7fe916800640 1 --2- 172.21.15.119:0/2672789393 >> v2:172.21.15.132:3300/0 conn(0x7fe9181450e0 0x7fe91806aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.821+0000 7fe916800640 1 -- 172.21.15.119:0/2672789393 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91815f6e0 con 0x7fe9180cc810 2024-04-25T10:11:14.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.822+0000 7fe916800640 1 --2- 172.21.15.119:0/2672789393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9180cc810 0x7fe918071050 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe918072170 tx=0x7fe91002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1814b5dd8f45af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.822+0000 7fe914a00640 1 -- 172.21.15.119:0/2672789393 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910036070 con 0x7fe9180cc810 2024-04-25T10:11:14.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.822+0000 7fe914a00640 1 -- 172.21.15.119:0/2672789393 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe910031600 con 0x7fe9180cc810 2024-04-25T10:11:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.822+0000 7fe914a00640 1 -- 172.21.15.119:0/2672789393 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9100319d0 con 0x7fe9180cc810 2024-04-25T10:11:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.823+0000 7fe917200640 1 -- 172.21.15.119:0/2672789393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9180cc810 msgr2=0x7fe918071050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.823+0000 7fe917200640 1 --2- 172.21.15.119:0/2672789393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9180cc810 0x7fe918071050 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe918072170 tx=0x7fe91002a570 comp rx=0 tx=0).stop 2024-04-25T10:11:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.823+0000 7fe917200640 1 -- 172.21.15.119:0/2672789393 shutdown_connections 2024-04-25T10:11:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.823+0000 7fe917200640 1 --2- 172.21.15.119:0/2672789393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9180cc810 0x7fe918071050 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.823+0000 7fe917200640 1 --2- 172.21.15.119:0/2672789393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe91806b0e0 0x7fe9180cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.823+0000 7fe917200640 1 --2- 172.21.15.119:0/2672789393 >> v2:172.21.15.132:3300/0 conn(0x7fe9181450e0 0x7fe91806aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.823+0000 7fe917200640 1 -- 172.21.15.119:0/2672789393 >> 172.21.15.119:0/2672789393 conn(0x7fe918066ff0 msgr2=0x7fe9180b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.823+0000 7fe917200640 1 -- 172.21.15.119:0/2672789393 shutdown_connections 2024-04-25T10:11:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.823+0000 7fe917200640 1 -- 172.21.15.119:0/2672789393 wait complete. 2024-04-25T10:11:14.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.825+0000 7fe917200640 1 Processor -- start 2024-04-25T10:11:14.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.825+0000 7fe917200640 1 -- start start 2024-04-25T10:11:14.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe917200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe91806b0e0 0x7fe9180ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe917200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9180cc810 0x7fe91815b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe917200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9181450e0 0x7fe91815cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe917200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9180ccca0 con 0x7fe91806b0e0 2024-04-25T10:11:14.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe917200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91815fca0 con 0x7fe9180cc810 2024-04-25T10:11:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe917200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9181f79e0 con 0x7fe9181450e0 2024-04-25T10:11:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe915e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe91806b0e0 0x7fe9180ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe915e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe91806b0e0 0x7fe9180ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44154/0 (socket says 172.21.15.119:44154) 2024-04-25T10:11:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe915e00640 1 -- 172.21.15.119:0/1362400866 learned_addr learned my addr 172.21.15.119:0/1362400866 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe916800640 1 --2- 172.21.15.119:0/1362400866 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9181450e0 0x7fe91815cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe915e00640 1 -- 172.21.15.119:0/1362400866 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9181450e0 msgr2=0x7fe91815cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe915400640 1 --2- 172.21.15.119:0/1362400866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9180cc810 0x7fe91815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe915e00640 1 --2- 172.21.15.119:0/1362400866 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9181450e0 0x7fe91815cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe915e00640 1 -- 172.21.15.119:0/1362400866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9180cc810 msgr2=0x7fe91815b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe915e00640 1 --2- 172.21.15.119:0/1362400866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9180cc810 0x7fe91815b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.826+0000 7fe915e00640 1 -- 172.21.15.119:0/1362400866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9181f6e60 con 0x7fe91806b0e0 2024-04-25T10:11:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.827+0000 7fe915e00640 1 --2- 172.21.15.119:0/1362400866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe91806b0e0 0x7fe9180ae020 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe90800d800 tx=0x7fe90800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.827+0000 7fe90e000640 1 -- 172.21.15.119:0/1362400866 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90800b6c0 con 0x7fe91806b0e0 2024-04-25T10:11:14.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.827+0000 7fe90e000640 1 -- 172.21.15.119:0/1362400866 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe908019070 con 0x7fe91806b0e0 2024-04-25T10:11:14.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.827+0000 7fe90e000640 1 -- 172.21.15.119:0/1362400866 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90800ba80 con 0x7fe91806b0e0 2024-04-25T10:11:14.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.827+0000 7fe917200640 1 -- 172.21.15.119:0/1362400866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9181f7140 con 0x7fe91806b0e0 2024-04-25T10:11:14.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.827+0000 7fe917200640 1 -- 172.21.15.119:0/1362400866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9181f7700 con 0x7fe91806b0e0 2024-04-25T10:11:14.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.828+0000 7fe917200640 1 -- 172.21.15.119:0/1362400866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9180749b0 con 0x7fe91806b0e0 2024-04-25T10:11:14.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.829+0000 7fe90e000640 1 -- 172.21.15.119:0/1362400866 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe90800bc20 con 0x7fe91806b0e0 2024-04-25T10:11:14.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.830+0000 7fe90e000640 1 --2- 172.21.15.119:0/1362400866 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe8f806b500 0x7fe8f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.830+0000 7fe90e000640 1 -- 172.21.15.119:0/1362400866 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe90808a6d0 con 0x7fe91806b0e0 2024-04-25T10:11:14.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.830+0000 7fe915400640 1 --2- 172.21.15.119:0/1362400866 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe8f806b500 0x7fe8f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.831+0000 7fe915400640 1 --2- 172.21.15.119:0/1362400866 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe8f806b500 0x7fe8f806d9c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe904006fd0 tx=0x7fe904008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:14.834+0000 7fe90e000640 1 -- 172.21.15.119:0/1362400866 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe90801c360 con 0x7fe91806b0e0 2024-04-25T10:11:15.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.015+0000 7fe917200640 1 -- 172.21.15.119:0/1362400866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9180b12d0 con 0x7fe91806b0e0 2024-04-25T10:11:15.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.016+0000 7fe90e000640 1 -- 172.21.15.119:0/1362400866 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe908059540 con 0x7fe91806b0e0 2024-04-25T10:11:15.021 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:11:15.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.018+0000 7fe917200640 1 -- 172.21.15.119:0/1362400866 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe8f806b500 msgr2=0x7fe8f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.018+0000 7fe917200640 1 --2- 172.21.15.119:0/1362400866 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe8f806b500 0x7fe8f806d9c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe904006fd0 tx=0x7fe904008040 comp rx=0 tx=0).stop 2024-04-25T10:11:15.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.018+0000 7fe917200640 1 -- 172.21.15.119:0/1362400866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe91806b0e0 msgr2=0x7fe9180ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.018+0000 7fe917200640 1 --2- 172.21.15.119:0/1362400866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe91806b0e0 0x7fe9180ae020 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe90800d800 tx=0x7fe90800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.018+0000 7fe917200640 1 -- 172.21.15.119:0/1362400866 shutdown_connections 2024-04-25T10:11:15.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.018+0000 7fe917200640 1 --2- 172.21.15.119:0/1362400866 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe8f806b500 0x7fe8f806d9c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.018+0000 7fe917200640 1 --2- 172.21.15.119:0/1362400866 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe9181450e0 0x7fe91815cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.018+0000 7fe917200640 1 --2- 172.21.15.119:0/1362400866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9180cc810 0x7fe91815b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.018+0000 7fe917200640 1 --2- 172.21.15.119:0/1362400866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe91806b0e0 0x7fe9180ae020 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.018+0000 7fe917200640 1 -- 172.21.15.119:0/1362400866 >> 172.21.15.119:0/1362400866 conn(0x7fe918066ff0 msgr2=0x7fe91815eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.018+0000 7fe917200640 1 -- 172.21.15.119:0/1362400866 shutdown_connections 2024-04-25T10:11:15.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.019+0000 7fe917200640 1 -- 172.21.15.119:0/1362400866 wait complete. 2024-04-25T10:11:15.042 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:15.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.233+0000 7f4325800640 1 Processor -- start 2024-04-25T10:11:15.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.233+0000 7f4325800640 1 -- start start 2024-04-25T10:11:15.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.234+0000 7f4325800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4320144e70 0x7f432006d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.234+0000 7f4325800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f43200cf0e0 0x7f43200cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.234+0000 7f4325800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432006dcb0 con 0x7f432006d7a0 2024-04-25T10:11:15.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.234+0000 7f4325800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432015bfe0 con 0x7f43200cf0e0 2024-04-25T10:11:15.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.234+0000 7f4325800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432006df90 con 0x7f4320144e70 2024-04-25T10:11:15.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.235+0000 7f431f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f43200cf0e0 0x7f43200cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.235+0000 7f431f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f43200cf0e0 0x7f43200cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44170/0 (socket says 172.21.15.119:44170) 2024-04-25T10:11:15.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.235+0000 7f431f400640 1 -- 172.21.15.119:0/3246094982 learned_addr learned my addr 172.21.15.119:0/3246094982 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:15.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.235+0000 7f431ea00640 1 --2- 172.21.15.119:0/3246094982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4320144e70 0x7f432006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.235+0000 7f431f400640 1 -- 172.21.15.119:0/3246094982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4320144e70 msgr2=0x7f432006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.235+0000 7f431f400640 1 --2- 172.21.15.119:0/3246094982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4320144e70 0x7f432006d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.235+0000 7f431f400640 1 -- 172.21.15.119:0/3246094982 >> v1:172.21.15.132:6789/0 conn(0x7f432006d7a0 legacy=0x7f43200ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.235+0000 7f431f400640 1 -- 172.21.15.119:0/3246094982 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432006e270 con 0x7f43200cf0e0 2024-04-25T10:11:15.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.235+0000 7f431ea00640 1 --2- 172.21.15.119:0/3246094982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4320144e70 0x7f432006d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:15.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.236+0000 7f431f400640 1 --2- 172.21.15.119:0/3246094982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43200cf0e0 0x7f43200cc300 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f43200711b0 tx=0x7f431802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ec3b80b0bc9abd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.237+0000 7f431e000640 1 -- 172.21.15.119:0/3246094982 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318036070 con 0x7f43200cf0e0 2024-04-25T10:11:15.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.237+0000 7f431e000640 1 -- 172.21.15.119:0/3246094982 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4318031600 con 0x7f43200cf0e0 2024-04-25T10:11:15.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.237+0000 7f431e000640 1 -- 172.21.15.119:0/3246094982 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43180319d0 con 0x7f43200cf0e0 2024-04-25T10:11:15.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.237+0000 7f4325800640 1 -- 172.21.15.119:0/3246094982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43200cf0e0 msgr2=0x7f43200cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.237+0000 7f4325800640 1 --2- 172.21.15.119:0/3246094982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43200cf0e0 0x7f43200cc300 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f43200711b0 tx=0x7f431802a630 comp rx=0 tx=0).stop 2024-04-25T10:11:15.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.237+0000 7f4325800640 1 -- 172.21.15.119:0/3246094982 shutdown_connections 2024-04-25T10:11:15.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.237+0000 7f4325800640 1 --2- 172.21.15.119:0/3246094982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43200cf0e0 0x7f43200cc300 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.237+0000 7f4325800640 1 --2- 172.21.15.119:0/3246094982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4320144e70 0x7f432006d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.237+0000 7f4325800640 1 -- 172.21.15.119:0/3246094982 >> 172.21.15.119:0/3246094982 conn(0x7f432006aba0 msgr2=0x7f432006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.237+0000 7f4325800640 1 -- 172.21.15.119:0/3246094982 shutdown_connections 2024-04-25T10:11:15.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.237+0000 7f4325800640 1 -- 172.21.15.119:0/3246094982 wait complete. 2024-04-25T10:11:15.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.239+0000 7f4325800640 1 Processor -- start 2024-04-25T10:11:15.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.239+0000 7f4325800640 1 -- start start 2024-04-25T10:11:15.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.240+0000 7f4325800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f432006d7a0 0x7f432013dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.240+0000 7f4325800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43200cf0e0 0x7f43201f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.240+0000 7f4325800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4320144e70 0x7f43200aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.240+0000 7f4325800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4320145630 con 0x7f43200cf0e0 2024-04-25T10:11:15.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.240+0000 7f4325800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43201454b0 con 0x7f4320144e70 2024-04-25T10:11:15.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.240+0000 7f4325800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43200cc840 con 0x7f432006d7a0 2024-04-25T10:11:15.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.240+0000 7f4317e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43200cf0e0 0x7f43201f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.240+0000 7f4317e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43200cf0e0 0x7f43201f4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44174/0 (socket says 172.21.15.119:44174) 2024-04-25T10:11:15.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.240+0000 7f4317e00640 1 -- 172.21.15.119:0/447266115 learned_addr learned my addr 172.21.15.119:0/447266115 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:15.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.241+0000 7f4317e00640 1 -- 172.21.15.119:0/447266115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f432006d7a0 msgr2=0x7f432013dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.241+0000 7f4317e00640 1 --2- 172.21.15.119:0/447266115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f432006d7a0 0x7f432013dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.241+0000 7f4317e00640 1 -- 172.21.15.119:0/447266115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4320144e70 msgr2=0x7f43200aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.241+0000 7f4317e00640 1 --2- 172.21.15.119:0/447266115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4320144e70 0x7f43200aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.241+0000 7f4317e00640 1 -- 172.21.15.119:0/447266115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43201f4ba0 con 0x7f43200cf0e0 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.241+0000 7f4317e00640 1 --2- 172.21.15.119:0/447266115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43200cf0e0 0x7f43201f4660 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f430800c430 tx=0x7f430800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.241+0000 7f4316a00640 1 -- 172.21.15.119:0/447266115 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308011790 con 0x7f43200cf0e0 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.241+0000 7f4316a00640 1 -- 172.21.15.119:0/447266115 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4308017070 con 0x7f43200cf0e0 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.241+0000 7f4316a00640 1 -- 172.21.15.119:0/447266115 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308011b70 con 0x7f43200cf0e0 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.241+0000 7f4325800640 1 -- 172.21.15.119:0/447266115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43200af650 con 0x7f43200cf0e0 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.242+0000 7f4325800640 1 -- 172.21.15.119:0/447266115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43200afc10 con 0x7f43200cf0e0 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.243+0000 7f4325800640 1 -- 172.21.15.119:0/447266115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43200750b0 con 0x7f43200cf0e0 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.244+0000 7f4316a00640 1 -- 172.21.15.119:0/447266115 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4308011d10 con 0x7f43200cf0e0 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.244+0000 7f4316a00640 1 --2- 172.21.15.119:0/447266115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42fc06b4d0 0x7f42fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.245+0000 7f4316a00640 1 -- 172.21.15.119:0/447266115 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4308089fe0 con 0x7f43200cf0e0 2024-04-25T10:11:15.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.245+0000 7f431ea00640 1 --2- 172.21.15.119:0/447266115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42fc06b4d0 0x7f42fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.246+0000 7f431ea00640 1 --2- 172.21.15.119:0/447266115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42fc06b4d0 0x7f42fc06d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4310006fd0 tx=0x7f4310008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.249+0000 7f4316a00640 1 -- 172.21.15.119:0/447266115 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4308058ef0 con 0x7f43200cf0e0 2024-04-25T10:11:15.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.431+0000 7f4325800640 1 -- 172.21.15.119:0/447266115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f432006f0f0 con 0x7f43200cf0e0 2024-04-25T10:11:15.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.432+0000 7f4316a00640 1 -- 172.21.15.119:0/447266115 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43080587f0 con 0x7f43200cf0e0 2024-04-25T10:11:15.437 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:11:15.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.434+0000 7f4325800640 1 -- 172.21.15.119:0/447266115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42fc06b4d0 msgr2=0x7f42fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.434+0000 7f4325800640 1 --2- 172.21.15.119:0/447266115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42fc06b4d0 0x7f42fc06d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4310006fd0 tx=0x7f4310008040 comp rx=0 tx=0).stop 2024-04-25T10:11:15.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.434+0000 7f4325800640 1 -- 172.21.15.119:0/447266115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43200cf0e0 msgr2=0x7f43201f4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.434+0000 7f4325800640 1 --2- 172.21.15.119:0/447266115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43200cf0e0 0x7f43201f4660 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f430800c430 tx=0x7f430800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.434+0000 7f4325800640 1 -- 172.21.15.119:0/447266115 shutdown_connections 2024-04-25T10:11:15.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.434+0000 7f4325800640 1 --2- 172.21.15.119:0/447266115 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42fc06b4d0 0x7f42fc06d990 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.434+0000 7f4325800640 1 --2- 172.21.15.119:0/447266115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4320144e70 0x7f43200aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.434+0000 7f4325800640 1 --2- 172.21.15.119:0/447266115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43200cf0e0 0x7f43201f4660 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.435+0000 7f4325800640 1 --2- 172.21.15.119:0/447266115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f432006d7a0 0x7f432013dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.435+0000 7f4325800640 1 -- 172.21.15.119:0/447266115 >> 172.21.15.119:0/447266115 conn(0x7f432006aba0 msgr2=0x7f432006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.435+0000 7f4325800640 1 -- 172.21.15.119:0/447266115 shutdown_connections 2024-04-25T10:11:15.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.435+0000 7f4325800640 1 -- 172.21.15.119:0/447266115 wait complete. 2024-04-25T10:11:15.458 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:15.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.640+0000 7f4e95000640 1 Processor -- start 2024-04-25T10:11:15.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.640+0000 7f4e95000640 1 -- start start 2024-04-25T10:11:15.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.641+0000 7f4e95000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4e901450e0 0x7f4e9006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.641+0000 7f4e95000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e900cc9e0 0x7f4e900b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.641+0000 7f4e95000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9015f400 con 0x7f4e9006b0e0 2024-04-25T10:11:15.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.641+0000 7f4e95000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e901454c0 con 0x7f4e901450e0 2024-04-25T10:11:15.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.641+0000 7f4e95000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9006b540 con 0x7f4e900cc9e0 2024-04-25T10:11:15.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.642+0000 7f4e8fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e900cc9e0 0x7f4e900b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.642+0000 7f4e8f400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4e901450e0 0x7f4e9006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.642+0000 7f4e8fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e900cc9e0 0x7f4e900b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:35866/0 (socket says 172.21.15.119:35866) 2024-04-25T10:11:15.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.642+0000 7f4e8fe00640 1 -- 172.21.15.119:0/127791997 learned_addr learned my addr 172.21.15.119:0/127791997 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:15.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.643+0000 7f4e8fe00640 1 -- 172.21.15.119:0/127791997 >> v2:172.21.15.132:3300/0 conn(0x7f4e901450e0 msgr2=0x7f4e9006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.643+0000 7f4e8fe00640 1 --2- 172.21.15.119:0/127791997 >> v2:172.21.15.132:3300/0 conn(0x7f4e901450e0 0x7f4e9006aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.643+0000 7f4e8fe00640 1 -- 172.21.15.119:0/127791997 >> v1:172.21.15.132:6789/0 conn(0x7f4e9006b0e0 legacy=0x7f4e900cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.643+0000 7f4e8fe00640 1 -- 172.21.15.119:0/127791997 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9015f6e0 con 0x7f4e900cc9e0 2024-04-25T10:11:15.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.644+0000 7f4e8fe00640 1 --2- 172.21.15.119:0/127791997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e900cc9e0 0x7f4e900b2e80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009e50 tx=0x7f4e8802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93cad58703e8312a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.645+0000 7f4e8e000640 1 -- 172.21.15.119:0/127791997 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e88036070 con 0x7f4e900cc9e0 2024-04-25T10:11:15.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.645+0000 7f4e8e000640 1 -- 172.21.15.119:0/127791997 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e88031600 con 0x7f4e900cc9e0 2024-04-25T10:11:15.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.645+0000 7f4e8e000640 1 -- 172.21.15.119:0/127791997 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e880319d0 con 0x7f4e900cc9e0 2024-04-25T10:11:15.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.646+0000 7f4e95000640 1 -- 172.21.15.119:0/127791997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e900cc9e0 msgr2=0x7f4e900b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.646+0000 7f4e95000640 1 --2- 172.21.15.119:0/127791997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e900cc9e0 0x7f4e900b2e80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009e50 tx=0x7f4e8802a5f0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.646+0000 7f4e95000640 1 -- 172.21.15.119:0/127791997 shutdown_connections 2024-04-25T10:11:15.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.646+0000 7f4e95000640 1 --2- 172.21.15.119:0/127791997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e900cc9e0 0x7f4e900b2e80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.646+0000 7f4e95000640 1 --2- 172.21.15.119:0/127791997 >> v2:172.21.15.132:3300/0 conn(0x7f4e901450e0 0x7f4e9006aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.646+0000 7f4e95000640 1 -- 172.21.15.119:0/127791997 >> 172.21.15.119:0/127791997 conn(0x7f4e90066ff0 msgr2=0x7f4e900b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.646+0000 7f4e95000640 1 -- 172.21.15.119:0/127791997 shutdown_connections 2024-04-25T10:11:15.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.646+0000 7f4e95000640 1 -- 172.21.15.119:0/127791997 wait complete. 2024-04-25T10:11:15.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.648+0000 7f4e95000640 1 Processor -- start 2024-04-25T10:11:15.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.648+0000 7f4e95000640 1 -- start start 2024-04-25T10:11:15.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.649+0000 7f4e95000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9006b0e0 0x7f4e90139e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.649+0000 7f4e95000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e900cc9e0 0x7f4e9015e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.649+0000 7f4e95000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e901450e0 0x7f4e900c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.649+0000 7f4e95000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e900b33c0 con 0x7f4e900cc9e0 2024-04-25T10:11:15.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.649+0000 7f4e95000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9015fca0 con 0x7f4e901450e0 2024-04-25T10:11:15.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.649+0000 7f4e95000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e900b3540 con 0x7f4e9006b0e0 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.649+0000 7f4e8ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e900cc9e0 0x7f4e9015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.649+0000 7f4e8ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e900cc9e0 0x7f4e9015e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44180/0 (socket says 172.21.15.119:44180) 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.649+0000 7f4e8ea00640 1 -- 172.21.15.119:0/3341315071 learned_addr learned my addr 172.21.15.119:0/3341315071 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.649+0000 7f4e8ea00640 1 -- 172.21.15.119:0/3341315071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9006b0e0 msgr2=0x7f4e90139e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.649+0000 7f4e8ea00640 1 --2- 172.21.15.119:0/3341315071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9006b0e0 0x7f4e90139e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.649+0000 7f4e8ea00640 1 -- 172.21.15.119:0/3341315071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e901450e0 msgr2=0x7f4e900c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.649+0000 7f4e8ea00640 1 --2- 172.21.15.119:0/3341315071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e901450e0 0x7f4e900c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.649+0000 7f4e8ea00640 1 -- 172.21.15.119:0/3341315071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9015ca30 con 0x7f4e900cc9e0 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.650+0000 7f4e8ea00640 1 --2- 172.21.15.119:0/3341315071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e900cc9e0 0x7f4e9015e370 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c430 tx=0x7f4e7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.650+0000 7f4e87400640 1 -- 172.21.15.119:0/3341315071 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c017070 con 0x7f4e900cc9e0 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.650+0000 7f4e87400640 1 -- 172.21.15.119:0/3341315071 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c0052d0 con 0x7f4e900cc9e0 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.650+0000 7f4e95000640 1 -- 172.21.15.119:0/3341315071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9015cd10 con 0x7f4e900cc9e0 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.650+0000 7f4e87400640 1 -- 172.21.15.119:0/3341315071 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c003850 con 0x7f4e900cc9e0 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.650+0000 7f4e95000640 1 -- 172.21.15.119:0/3341315071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9015d2d0 con 0x7f4e900cc9e0 2024-04-25T10:11:15.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.651+0000 7f4e95000640 1 -- 172.21.15.119:0/3341315071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e900749b0 con 0x7f4e900cc9e0 2024-04-25T10:11:15.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.652+0000 7f4e87400640 1 -- 172.21.15.119:0/3341315071 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e7c01a070 con 0x7f4e900cc9e0 2024-04-25T10:11:15.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.653+0000 7f4e87400640 1 --2- 172.21.15.119:0/3341315071 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4e7006b500 0x7f4e7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.653+0000 7f4e87400640 1 -- 172.21.15.119:0/3341315071 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e7c089670 con 0x7f4e900cc9e0 2024-04-25T10:11:15.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.653+0000 7f4e8f400640 1 --2- 172.21.15.119:0/3341315071 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4e7006b500 0x7f4e7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.654+0000 7f4e8f400640 1 --2- 172.21.15.119:0/3341315071 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4e7006b500 0x7f4e7006d9c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4e80006a90 tx=0x7f4e80006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.657+0000 7f4e87400640 1 -- 172.21.15.119:0/3341315071 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e7c0583d0 con 0x7f4e900cc9e0 2024-04-25T10:11:15.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.865+0000 7f4e95000640 1 -- 172.21.15.119:0/3341315071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4e9015b840 con 0x7f4e900cc9e0 2024-04-25T10:11:15.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.866+0000 7f4e87400640 1 -- 172.21.15.119:0/3341315071 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4e7c05e0d0 con 0x7f4e900cc9e0 2024-04-25T10:11:15.871 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2024-04-25T10:11:15.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.868+0000 7f4e95000640 1 -- 172.21.15.119:0/3341315071 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4e7006b500 msgr2=0x7f4e7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.868+0000 7f4e95000640 1 --2- 172.21.15.119:0/3341315071 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4e7006b500 0x7f4e7006d9c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4e80006a90 tx=0x7f4e80006a20 comp rx=0 tx=0).stop 2024-04-25T10:11:15.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.868+0000 7f4e95000640 1 -- 172.21.15.119:0/3341315071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e900cc9e0 msgr2=0x7f4e9015e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.868+0000 7f4e95000640 1 --2- 172.21.15.119:0/3341315071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e900cc9e0 0x7f4e9015e370 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c430 tx=0x7f4e7c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.868+0000 7f4e95000640 1 -- 172.21.15.119:0/3341315071 shutdown_connections 2024-04-25T10:11:15.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.868+0000 7f4e95000640 1 --2- 172.21.15.119:0/3341315071 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4e7006b500 0x7f4e7006d9c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.868+0000 7f4e95000640 1 --2- 172.21.15.119:0/3341315071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e901450e0 0x7f4e900c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.868+0000 7f4e95000640 1 --2- 172.21.15.119:0/3341315071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e900cc9e0 0x7f4e9015e370 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.868+0000 7f4e95000640 1 --2- 172.21.15.119:0/3341315071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e9006b0e0 0x7f4e90139e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.868+0000 7f4e95000640 1 -- 172.21.15.119:0/3341315071 >> 172.21.15.119:0/3341315071 conn(0x7f4e90066ff0 msgr2=0x7f4e900b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.868+0000 7f4e95000640 1 -- 172.21.15.119:0/3341315071 shutdown_connections 2024-04-25T10:11:15.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:15.868+0000 7f4e95000640 1 -- 172.21.15.119:0/3341315071 wait complete. 2024-04-25T10:11:15.893 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:11:15.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:15.894 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:16.010 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:11:16.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:16.010 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:11:16.070 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:11:16.070 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:16.112 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:16.112 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:11:16.112 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:16.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:16.149 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:11:16.150 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:16.201 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:16.201 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.380+0000 7f3341000640 1 Processor -- start 2024-04-25T10:11:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.380+0000 7f3341000640 1 -- start start 2024-04-25T10:11:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.381+0000 7f3341000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f333c15bf00 0x7f333c0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.381+0000 7f3341000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333c0ce6c0 con 0x7f333c0cc310 2024-04-25T10:11:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.381+0000 7f3341000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333c0cc7b0 con 0x7f333c06e8f0 2024-04-25T10:11:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.381+0000 7f3341000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c1455e0 con 0x7f333c15bf00 2024-04-25T10:11:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.382+0000 7f3333e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f333c0cc310 0x7f333c15c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54914/0 (socket says 172.21.15.119:54914) 2024-04-25T10:11:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.382+0000 7f3333e00640 1 -- 172.21.15.119:0/736234740 learned_addr learned my addr 172.21.15.119:0/736234740 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.382+0000 7f3339e00640 1 --2- 172.21.15.119:0/736234740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f333c15bf00 0x7f333c0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.382+0000 7f3339400640 1 -- 172.21.15.119:0/736234740 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767919433 0 0) 0x7f3324001110 con 0x7f333c0cc310 2024-04-25T10:11:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.382+0000 7f3339400640 1 -- 172.21.15.119:0/736234740 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3320003520 con 0x7f333c0cc310 2024-04-25T10:11:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.383+0000 7f3339400640 1 -- 172.21.15.119:0/736234740 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1315190532 0 0) 0x7f3320003520 con 0x7f333c0cc310 2024-04-25T10:11:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.383+0000 7f3339400640 1 -- 172.21.15.119:0/736234740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f333c15bf00 msgr2=0x7f333c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.383+0000 7f3339400640 1 --2- 172.21.15.119:0/736234740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f333c15bf00 0x7f333c0cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.383+0000 7f3339400640 1 -- 172.21.15.119:0/736234740 >> v1:172.21.15.132:6789/0 conn(0x7f333c06e8f0 legacy=0x7f333c1654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.383+0000 7f3339400640 1 -- 172.21.15.119:0/736234740 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c0cca90 con 0x7f333c0cc310 2024-04-25T10:11:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.383+0000 7f3339400640 1 -- 172.21.15.119:0/736234740 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f3324001840 con 0x7f333c0cc310 2024-04-25T10:11:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.383+0000 7f3339400640 1 -- 172.21.15.119:0/736234740 >> v1:172.21.15.119:6789/0 conn(0x7f333c0cc310 legacy=0x7f333c15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.383+0000 7f3339400640 1 --2- 172.21.15.119:0/736234740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33200039e0 0x7f3320003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.383+0000 7f3339400640 1 --2- 172.21.15.119:0/736234740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320004800 0x7f3320004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.383+0000 7f3339400640 1 --2- 172.21.15.119:0/736234740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f333c15bf00 0x7f33200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.383+0000 7f3339400640 1 -- 172.21.15.119:0/736234740 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c1455e0 con 0x7f33200039e0 2024-04-25T10:11:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.383+0000 7f3339400640 1 -- 172.21.15.119:0/736234740 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332000ba70 con 0x7f3320004800 2024-04-25T10:11:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.383+0000 7f3339400640 1 -- 172.21.15.119:0/736234740 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332000bc70 con 0x7f333c15bf00 2024-04-25T10:11:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.383+0000 7f3339e00640 1 --2- 172.21.15.119:0/736234740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33200039e0 0x7f3320003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.384+0000 7f3339e00640 1 -- 172.21.15.119:0/736234740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f333c15bf00 msgr2=0x7f33200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.384+0000 7f3339e00640 1 --2- 172.21.15.119:0/736234740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f333c15bf00 0x7f33200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.384+0000 7f3339e00640 1 -- 172.21.15.119:0/736234740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320004800 msgr2=0x7f3320004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.384+0000 7f3339e00640 1 --2- 172.21.15.119:0/736234740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320004800 0x7f3320004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.384+0000 7f3339e00640 1 -- 172.21.15.119:0/736234740 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3320003520 con 0x7f33200039e0 2024-04-25T10:11:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.385+0000 7f3339e00640 1 --2- 172.21.15.119:0/736234740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33200039e0 0x7f3320003e00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f332c009a90 tx=0x7f3320005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39fdf29db6c17a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.385+0000 7f3339400640 1 -- 172.21.15.119:0/736234740 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c036070 con 0x7f33200039e0 2024-04-25T10:11:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.385+0000 7f3339400640 1 -- 172.21.15.119:0/736234740 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c031400 con 0x7f33200039e0 2024-04-25T10:11:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.385+0000 7f3339400640 1 -- 172.21.15.119:0/736234740 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c009690 con 0x7f33200039e0 2024-04-25T10:11:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.385+0000 7f3341000640 1 -- 172.21.15.119:0/736234740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33200039e0 msgr2=0x7f3320003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.385+0000 7f3341000640 1 --2- 172.21.15.119:0/736234740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33200039e0 0x7f3320003e00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f332c009a90 tx=0x7f3320005830 comp rx=0 tx=0).stop 2024-04-25T10:11:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.385+0000 7f333a800640 1 -- 172.21.15.119:0/736234740 reap_dead start 2024-04-25T10:11:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.385+0000 7f3341000640 1 -- 172.21.15.119:0/736234740 shutdown_connections 2024-04-25T10:11:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.385+0000 7f3341000640 1 -- 172.21.15.119:0/736234740 >> 172.21.15.119:0/736234740 conn(0x7f333c06aba0 msgr2=0x7f333c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.385+0000 7f3341000640 1 -- 172.21.15.119:0/736234740 shutdown_connections 2024-04-25T10:11:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.385+0000 7f3341000640 1 -- 172.21.15.119:0/736234740 wait complete. 2024-04-25T10:11:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.387+0000 7f3341000640 1 Processor -- start 2024-04-25T10:11:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.387+0000 7f3341000640 1 -- start start 2024-04-25T10:11:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.388+0000 7f3341000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33200039e0 0x7f333c1f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.388+0000 7f3341000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f333c1f2be0 0x7f333c1f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.388+0000 7f3341000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f333c13b3d0 0x7f333c13b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.388+0000 7f3341000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0cd050 con 0x7f333c1f2be0 2024-04-25T10:11:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.388+0000 7f3341000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c06d450 con 0x7f33200039e0 2024-04-25T10:11:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.388+0000 7f3341000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c06ee60 con 0x7f333c13b3d0 2024-04-25T10:11:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.388+0000 7f3333e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f333c1f2be0 0x7f333c1f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.389+0000 7f3333e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f333c1f2be0 0x7f333c1f3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44186/0 (socket says 172.21.15.119:44186) 2024-04-25T10:11:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.389+0000 7f3333e00640 1 -- 172.21.15.119:0/2519580720 learned_addr learned my addr 172.21.15.119:0/2519580720 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.389+0000 7f333a800640 1 --2- 172.21.15.119:0/2519580720 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f333c13b3d0 0x7f333c13b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.389+0000 7f3333e00640 1 -- 172.21.15.119:0/2519580720 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f333c13b3d0 msgr2=0x7f333c13b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.389+0000 7f3339e00640 1 --2- 172.21.15.119:0/2519580720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33200039e0 0x7f333c1f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.389+0000 7f3333e00640 1 --2- 172.21.15.119:0/2519580720 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f333c13b3d0 0x7f333c13b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.389+0000 7f3333e00640 1 -- 172.21.15.119:0/2519580720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33200039e0 msgr2=0x7f333c1f26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.389+0000 7f3333e00640 1 --2- 172.21.15.119:0/2519580720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33200039e0 0x7f333c1f26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.389+0000 7f3333e00640 1 -- 172.21.15.119:0/2519580720 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c13a930 con 0x7f333c1f2be0 2024-04-25T10:11:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.389+0000 7f3333e00640 1 --2- 172.21.15.119:0/2519580720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f333c1f2be0 0x7f333c1f3040 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f33240013b0 tx=0x7f332400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.390+0000 7f3332000640 1 -- 172.21.15.119:0/2519580720 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324018070 con 0x7f333c1f2be0 2024-04-25T10:11:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.390+0000 7f3332000640 1 -- 172.21.15.119:0/2519580720 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3324007500 con 0x7f333c1f2be0 2024-04-25T10:11:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.390+0000 7f3332000640 1 -- 172.21.15.119:0/2519580720 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324004a20 con 0x7f333c1f2be0 2024-04-25T10:11:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.390+0000 7f3341000640 1 -- 172.21.15.119:0/2519580720 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333c142bd0 con 0x7f333c1f2be0 2024-04-25T10:11:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.390+0000 7f3341000640 1 -- 172.21.15.119:0/2519580720 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333c13abd0 con 0x7f333c1f2be0 2024-04-25T10:11:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.391+0000 7f3341000640 1 -- 172.21.15.119:0/2519580720 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333c0749b0 con 0x7f333c1f2be0 2024-04-25T10:11:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.392+0000 7f3332000640 1 -- 172.21.15.119:0/2519580720 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3324004bc0 con 0x7f333c1f2be0 2024-04-25T10:11:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.393+0000 7f3332000640 1 --2- 172.21.15.119:0/2519580720 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f331806b4d0 0x7f331806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.393+0000 7f3332000640 1 -- 172.21.15.119:0/2519580720 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33240897f0 con 0x7f333c1f2be0 2024-04-25T10:11:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.393+0000 7f3339e00640 1 --2- 172.21.15.119:0/2519580720 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f331806b4d0 0x7f331806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.394+0000 7f3339e00640 1 --2- 172.21.15.119:0/2519580720 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f331806b4d0 0x7f331806d990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f332c033000 tx=0x7f332c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.397+0000 7f3332000640 1 -- 172.21.15.119:0/2519580720 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f333c0ce6c0 con 0x7f333c1f2be0 2024-04-25T10:11:16.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.631+0000 7f3341000640 1 -- 172.21.15.119:0/2519580720 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f333c15f1f0 con 0x7f333c1f2be0 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.633+0000 7f3332000640 1 -- 172.21.15.119:0/2519580720 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f3324058650 con 0x7f333c1f2be0 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4159222235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4159222235},{"type":"v1","addr":"172.21.15.132:6834","nonce":4159222235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4223361610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":4223361610},{"type":"v1","addr":"172.21.15.119:6837","nonce":4223361610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2367324442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2367324442},{"type":"v1","addr":"172.21.15.132:6835","nonce":2367324442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:59.454941+0000","modified":"2024-04-25T10:10:48.089168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6836/4129112885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4129112885},{"type":"v1","addr":"172.21.15.119:6836","nonce":4129112885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 36 2024-04-25T10:11:16.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.635+0000 7f3341000640 1 -- 172.21.15.119:0/2519580720 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f331806b4d0 msgr2=0x7f331806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.635+0000 7f3341000640 1 --2- 172.21.15.119:0/2519580720 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f331806b4d0 0x7f331806d990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f332c033000 tx=0x7f332c008030 comp rx=0 tx=0).stop 2024-04-25T10:11:16.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.635+0000 7f3341000640 1 -- 172.21.15.119:0/2519580720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f333c1f2be0 msgr2=0x7f333c1f3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.635+0000 7f3341000640 1 --2- 172.21.15.119:0/2519580720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f333c1f2be0 0x7f333c1f3040 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f33240013b0 tx=0x7f332400ccd0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.635+0000 7f3341000640 1 -- 172.21.15.119:0/2519580720 shutdown_connections 2024-04-25T10:11:16.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.635+0000 7f3341000640 1 --2- 172.21.15.119:0/2519580720 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f331806b4d0 0x7f331806d990 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.635+0000 7f3341000640 1 --2- 172.21.15.119:0/2519580720 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f333c13b3d0 0x7f333c13b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.635+0000 7f3341000640 1 --2- 172.21.15.119:0/2519580720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f333c1f2be0 0x7f333c1f3040 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.635+0000 7f3341000640 1 --2- 172.21.15.119:0/2519580720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33200039e0 0x7f333c1f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.635+0000 7f3341000640 1 -- 172.21.15.119:0/2519580720 >> 172.21.15.119:0/2519580720 conn(0x7f333c06aba0 msgr2=0x7f333c0ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.635+0000 7f3341000640 1 -- 172.21.15.119:0/2519580720 shutdown_connections 2024-04-25T10:11:16.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.636+0000 7f3341000640 1 -- 172.21.15.119:0/2519580720 wait complete. 2024-04-25T10:11:16.659 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:16.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.854+0000 7f7ca3200640 1 Processor -- start 2024-04-25T10:11:16.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.854+0000 7f7ca3200640 1 -- start start 2024-04-25T10:11:16.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.856+0000 7f7ca3200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7c9c15a870 0x7f7c9c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.856+0000 7f7ca3200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c9c0cdb10 con 0x7f7c9c0cfc40 2024-04-25T10:11:16.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.856+0000 7f7ca3200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c9c0cddf0 con 0x7f7c9c06e2e0 2024-04-25T10:11:16.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.856+0000 7f7ca3200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c9c0ce720 con 0x7f7c9c15a870 2024-04-25T10:11:16.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.856+0000 7f7ca1200640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7c9c0cfc40 0x7f7c9c15bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54926/0 (socket says 172.21.15.119:54926) 2024-04-25T10:11:16.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.856+0000 7f7ca1200640 1 -- 172.21.15.119:0/1683782407 learned_addr learned my addr 172.21.15.119:0/1683782407 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:16.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 -- 172.21.15.119:0/1683782407 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055463186 0 0) 0x7f7c9c0cdb10 con 0x7f7c9c0cfc40 2024-04-25T10:11:16.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9be00640 1 --2- 172.21.15.119:0/1683782407 >> v2:172.21.15.132:3300/0 conn(0x7f7c9c15a870 0x7f7c9c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 -- 172.21.15.119:0/1683782407 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c80005730 con 0x7f7c9c0cfc40 2024-04-25T10:11:16.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 -- 172.21.15.119:0/1683782407 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1200927440 0 0) 0x7f7c80005730 con 0x7f7c9c0cfc40 2024-04-25T10:11:16.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 -- 172.21.15.119:0/1683782407 >> v2:172.21.15.132:3300/0 conn(0x7f7c9c15a870 msgr2=0x7f7c9c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 --2- 172.21.15.119:0/1683782407 >> v2:172.21.15.132:3300/0 conn(0x7f7c9c15a870 0x7f7c9c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 -- 172.21.15.119:0/1683782407 >> v1:172.21.15.132:6789/0 conn(0x7f7c9c06e2e0 legacy=0x7f7c9c0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 -- 172.21.15.119:0/1683782407 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c0ce0d0 con 0x7f7c9c0cfc40 2024-04-25T10:11:16.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 -- 172.21.15.119:0/1683782407 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f7c940019f0 con 0x7f7c9c0cfc40 2024-04-25T10:11:16.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 -- 172.21.15.119:0/1683782407 >> v1:172.21.15.119:6789/0 conn(0x7f7c9c0cfc40 legacy=0x7f7c9c15bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 --2- 172.21.15.119:0/1683782407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c80005c10 0x7f7c80006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 --2- 172.21.15.119:0/1683782407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c80006a10 0x7f7c80006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 --2- 172.21.15.119:0/1683782407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80007420 0x7f7c8000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 -- 172.21.15.119:0/1683782407 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c0ce720 con 0x7f7c80007420 2024-04-25T10:11:16.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 -- 172.21.15.119:0/1683782407 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8000e370 con 0x7f7c80005c10 2024-04-25T10:11:16.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.857+0000 7f7c9b400640 1 -- 172.21.15.119:0/1683782407 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8000e570 con 0x7f7c80006a10 2024-04-25T10:11:16.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.858+0000 7f7ca1200640 1 --2- 172.21.15.119:0/1683782407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80007420 0x7f7c8000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.858+0000 7f7c93400640 1 --2- 172.21.15.119:0/1683782407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c80006a10 0x7f7c80006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.858+0000 7f7ca1200640 1 -- 172.21.15.119:0/1683782407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c80006a10 msgr2=0x7f7c80006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.858+0000 7f7ca1200640 1 --2- 172.21.15.119:0/1683782407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c80006a10 0x7f7c80006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.858+0000 7f7c9be00640 1 --2- 172.21.15.119:0/1683782407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c80005c10 0x7f7c80006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.858+0000 7f7ca1200640 1 -- 172.21.15.119:0/1683782407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c80005c10 msgr2=0x7f7c80006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.858+0000 7f7ca1200640 1 --2- 172.21.15.119:0/1683782407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c80005c10 0x7f7c80006030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.858+0000 7f7ca1200640 1 -- 172.21.15.119:0/1683782407 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c80005730 con 0x7f7c80007420 2024-04-25T10:11:16.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.858+0000 7f7c93400640 1 --2- 172.21.15.119:0/1683782407 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c80006a10 0x7f7c80006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:16.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.859+0000 7f7ca1200640 1 -- 172.21.15.119:0/1683782407 reap_dead start 2024-04-25T10:11:16.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.859+0000 7f7ca1200640 1 --2- 172.21.15.119:0/1683782407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80007420 0x7f7c8000dbd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c06bb80 tx=0x7f7c80007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ff0275b7f97367 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.859+0000 7f7c9b400640 1 -- 172.21.15.119:0/1683782407 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9400a390 con 0x7f7c80007420 2024-04-25T10:11:16.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.860+0000 7f7c9b400640 1 -- 172.21.15.119:0/1683782407 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9402f070 con 0x7f7c80007420 2024-04-25T10:11:16.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.860+0000 7f7c9b400640 1 -- 172.21.15.119:0/1683782407 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9402bb90 con 0x7f7c80007420 2024-04-25T10:11:16.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.860+0000 7f7ca3200640 1 -- 172.21.15.119:0/1683782407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80007420 msgr2=0x7f7c8000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.860+0000 7f7ca3200640 1 --2- 172.21.15.119:0/1683782407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80007420 0x7f7c8000dbd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c06bb80 tx=0x7f7c80007a40 comp rx=0 tx=0).stop 2024-04-25T10:11:16.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.860+0000 7f7ca3200640 1 -- 172.21.15.119:0/1683782407 shutdown_connections 2024-04-25T10:11:16.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.860+0000 7f7ca3200640 1 --2- 172.21.15.119:0/1683782407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80007420 0x7f7c8000dbd0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.860+0000 7f7ca3200640 1 -- 172.21.15.119:0/1683782407 >> 172.21.15.119:0/1683782407 conn(0x7f7c9c06aba0 msgr2=0x7f7c9c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.860+0000 7f7ca3200640 1 -- 172.21.15.119:0/1683782407 shutdown_connections 2024-04-25T10:11:16.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.860+0000 7f7ca3200640 1 -- 172.21.15.119:0/1683782407 wait complete. 2024-04-25T10:11:16.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.862+0000 7f7ca3200640 1 Processor -- start 2024-04-25T10:11:16.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.862+0000 7f7ca3200640 1 -- start start 2024-04-25T10:11:16.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.863+0000 7f7ca3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c80007420 0x7f7c9c14d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.863+0000 7f7ca3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c9c14d8e0 0x7f7c9c14b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.863+0000 7f7ca3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c9c0b33b0 0x7f7c9c0b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.863+0000 7f7ca3200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c0cd700 con 0x7f7c9c14d8e0 2024-04-25T10:11:16.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.863+0000 7f7ca3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c0cd580 con 0x7f7c80007420 2024-04-25T10:11:16.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.863+0000 7f7ca3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c0cd880 con 0x7f7c9c0b33b0 2024-04-25T10:11:16.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.863+0000 7f7c93400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c9c14d8e0 0x7f7c9c14b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.864+0000 7f7c93400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c9c14d8e0 0x7f7c9c14b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44198/0 (socket says 172.21.15.119:44198) 2024-04-25T10:11:16.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.864+0000 7f7c93400640 1 -- 172.21.15.119:0/1257776042 learned_addr learned my addr 172.21.15.119:0/1257776042 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:16.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.864+0000 7f7c9be00640 1 --2- 172.21.15.119:0/1257776042 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c80007420 0x7f7c9c14d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.864+0000 7f7c93400640 1 -- 172.21.15.119:0/1257776042 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c9c0b33b0 msgr2=0x7f7c9c0b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.864+0000 7f7c93400640 1 --2- 172.21.15.119:0/1257776042 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c9c0b33b0 0x7f7c9c0b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.864+0000 7f7c93400640 1 -- 172.21.15.119:0/1257776042 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c80007420 msgr2=0x7f7c9c14d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.864+0000 7f7c93400640 1 --2- 172.21.15.119:0/1257776042 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c80007420 0x7f7c9c14d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.864+0000 7f7c93400640 1 -- 172.21.15.119:0/1257776042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c14d640 con 0x7f7c9c14d8e0 2024-04-25T10:11:16.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.864+0000 7f7c93400640 1 --2- 172.21.15.119:0/1257776042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c9c14d8e0 0x7f7c9c14b0a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400d770 tx=0x7f7c8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.864+0000 7f7c98c00640 1 -- 172.21.15.119:0/1257776042 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84005270 con 0x7f7c9c14d8e0 2024-04-25T10:11:16.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.865+0000 7f7c98c00640 1 -- 172.21.15.119:0/1257776042 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c84017070 con 0x7f7c9c14d8e0 2024-04-25T10:11:16.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.865+0000 7f7c98c00640 1 -- 172.21.15.119:0/1257776042 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8400b6b0 con 0x7f7c9c14d8e0 2024-04-25T10:11:16.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.865+0000 7f7ca3200640 1 -- 172.21.15.119:0/1257776042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9c14de80 con 0x7f7c9c14d8e0 2024-04-25T10:11:16.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.865+0000 7f7ca3200640 1 -- 172.21.15.119:0/1257776042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9c14fb40 con 0x7f7c9c14d8e0 2024-04-25T10:11:16.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.866+0000 7f7ca3200640 1 -- 172.21.15.119:0/1257776042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c9c0749b0 con 0x7f7c9c14d8e0 2024-04-25T10:11:16.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.867+0000 7f7c98c00640 1 -- 172.21.15.119:0/1257776042 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c8400b850 con 0x7f7c9c14d8e0 2024-04-25T10:11:16.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.868+0000 7f7c98c00640 1 --2- 172.21.15.119:0/1257776042 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7c7806b500 0x7f7c7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.868+0000 7f7c98c00640 1 -- 172.21.15.119:0/1257776042 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c84089fd0 con 0x7f7c9c14d8e0 2024-04-25T10:11:16.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.868+0000 7f7c9be00640 1 --2- 172.21.15.119:0/1257776042 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7c7806b500 0x7f7c7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.869+0000 7f7c9be00640 1 --2- 172.21.15.119:0/1257776042 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7c7806b500 0x7f7c7806d9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c006fd0 tx=0x7f7c8c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:16.872+0000 7f7c98c00640 1 -- 172.21.15.119:0/1257776042 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c84019310 con 0x7f7c9c14d8e0 2024-04-25T10:11:17.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.067+0000 7f7ca3200640 1 -- 172.21.15.119:0/1257776042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c9c0aef30 con 0x7f7c9c14d8e0 2024-04-25T10:11:17.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.069+0000 7f7c98c00640 1 -- 172.21.15.119:0/1257776042 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16701 (secure 0 0 0) 0x7f7c84001230 con 0x7f7c9c14d8e0 2024-04-25T10:11:17.075 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:17.075 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":55,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:10:01.035989+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:57.237546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:58.475727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:17.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.072+0000 7f7ca3200640 1 -- 172.21.15.119:0/1257776042 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7c7806b500 msgr2=0x7f7c7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.072+0000 7f7ca3200640 1 --2- 172.21.15.119:0/1257776042 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7c7806b500 0x7f7c7806d9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c006fd0 tx=0x7f7c8c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:17.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.072+0000 7f7ca3200640 1 -- 172.21.15.119:0/1257776042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c9c14d8e0 msgr2=0x7f7c9c14b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.072+0000 7f7ca3200640 1 --2- 172.21.15.119:0/1257776042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c9c14d8e0 0x7f7c9c14b0a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400d770 tx=0x7f7c8400dc30 comp rx=0 tx=0).stop 2024-04-25T10:11:17.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.072+0000 7f7ca3200640 1 -- 172.21.15.119:0/1257776042 shutdown_connections 2024-04-25T10:11:17.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.072+0000 7f7ca3200640 1 --2- 172.21.15.119:0/1257776042 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7c7806b500 0x7f7c7806d9c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.072+0000 7f7ca3200640 1 --2- 172.21.15.119:0/1257776042 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7c9c0b33b0 0x7f7c9c0b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.072+0000 7f7ca3200640 1 --2- 172.21.15.119:0/1257776042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c9c14d8e0 0x7f7c9c14b0a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.072+0000 7f7ca3200640 1 --2- 172.21.15.119:0/1257776042 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c80007420 0x7f7c9c14d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.072+0000 7f7ca3200640 1 -- 172.21.15.119:0/1257776042 >> 172.21.15.119:0/1257776042 conn(0x7f7c9c06aba0 msgr2=0x7f7c9c0ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.072+0000 7f7ca3200640 1 -- 172.21.15.119:0/1257776042 shutdown_connections 2024-04-25T10:11:17.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.072+0000 7f7ca3200640 1 -- 172.21.15.119:0/1257776042 wait complete. 2024-04-25T10:11:17.097 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:09:57.237546+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:09:58.475727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:17.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:17.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.285+0000 7f39a3200640 1 Processor -- start 2024-04-25T10:11:17.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.286+0000 7f39a3200640 1 -- start start 2024-04-25T10:11:17.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.287+0000 7f39a3200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f39a4159ee0 0x7f39a415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.287+0000 7f39a3200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f39a40b3d80 0x7f39a4071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.287+0000 7f39a3200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a40b2850 con 0x7f39a40b38f0 2024-04-25T10:11:17.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.287+0000 7f39a3200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a415b300 con 0x7f39a40b3d80 2024-04-25T10:11:17.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.287+0000 7f39a3200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a40b4180 con 0x7f39a4159ee0 2024-04-25T10:11:17.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.288+0000 7f39a2800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f39a40b3d80 0x7f39a4071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.288+0000 7f39a2800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f39a40b3d80 0x7f39a4071050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44202/0 (socket says 172.21.15.119:44202) 2024-04-25T10:11:17.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.288+0000 7f39a2800640 1 -- 172.21.15.119:0/1932271618 learned_addr learned my addr 172.21.15.119:0/1932271618 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.288+0000 7f39a2800640 1 -- 172.21.15.119:0/1932271618 >> v2:172.21.15.132:3300/0 conn(0x7f39a4159ee0 msgr2=0x7f39a415adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.288+0000 7f39a2800640 1 --2- 172.21.15.119:0/1932271618 >> v2:172.21.15.132:3300/0 conn(0x7f39a4159ee0 0x7f39a415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.288+0000 7f39a2800640 1 -- 172.21.15.119:0/1932271618 >> v1:172.21.15.132:6789/0 conn(0x7f39a40b38f0 legacy=0x7f39a415b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.288+0000 7f39a2800640 1 -- 172.21.15.119:0/1932271618 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a40b2b30 con 0x7f39a40b3d80 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.289+0000 7f39a2800640 1 --2- 172.21.15.119:0/1932271618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a40b3d80 0x7f39a4071050 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f39a40b1f40 tx=0x7f399c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1632a7965666cd6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.290+0000 7f39a1400640 1 -- 172.21.15.119:0/1932271618 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c036070 con 0x7f39a40b3d80 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.290+0000 7f39a1400640 1 -- 172.21.15.119:0/1932271618 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c031600 con 0x7f39a40b3d80 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.290+0000 7f39a1400640 1 -- 172.21.15.119:0/1932271618 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c0319d0 con 0x7f39a40b3d80 2024-04-25T10:11:17.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.290+0000 7f39a3200640 1 -- 172.21.15.119:0/1932271618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a40b3d80 msgr2=0x7f39a4071050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.290+0000 7f39a3200640 1 --2- 172.21.15.119:0/1932271618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a40b3d80 0x7f39a4071050 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f39a40b1f40 tx=0x7f399c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.290+0000 7f39a3200640 1 -- 172.21.15.119:0/1932271618 shutdown_connections 2024-04-25T10:11:17.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.290+0000 7f39a3200640 1 --2- 172.21.15.119:0/1932271618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a40b3d80 0x7f39a4071050 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.290+0000 7f39a3200640 1 --2- 172.21.15.119:0/1932271618 >> v2:172.21.15.132:3300/0 conn(0x7f39a4159ee0 0x7f39a415adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.290+0000 7f39a3200640 1 -- 172.21.15.119:0/1932271618 >> 172.21.15.119:0/1932271618 conn(0x7f39a4066ff0 msgr2=0x7f39a40b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.290+0000 7f39a3200640 1 -- 172.21.15.119:0/1932271618 shutdown_connections 2024-04-25T10:11:17.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.290+0000 7f39a3200640 1 -- 172.21.15.119:0/1932271618 wait complete. 2024-04-25T10:11:17.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.292+0000 7f39a3200640 1 Processor -- start 2024-04-25T10:11:17.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.292+0000 7f39a3200640 1 -- start start 2024-04-25T10:11:17.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.293+0000 7f39a3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39a40b38f0 0x7f39a4153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.293+0000 7f39a3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a40b3d80 0x7f39a4154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.293+0000 7f39a3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a4159ee0 0x7f39a4154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.293+0000 7f39a3200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a406efc0 con 0x7f39a40b3d80 2024-04-25T10:11:17.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.293+0000 7f39a3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a40b30f0 con 0x7f39a4159ee0 2024-04-25T10:11:17.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.293+0000 7f39a3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a40b2470 con 0x7f39a40b38f0 2024-04-25T10:11:17.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.293+0000 7f399be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a40b3d80 0x7f39a4154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.294+0000 7f399be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a40b3d80 0x7f39a4154230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44204/0 (socket says 172.21.15.119:44204) 2024-04-25T10:11:17.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.294+0000 7f399be00640 1 -- 172.21.15.119:0/4195273024 learned_addr learned my addr 172.21.15.119:0/4195273024 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:17.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.294+0000 7f39a1e00640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39a40b38f0 0x7f39a4153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.294+0000 7f399be00640 1 -- 172.21.15.119:0/4195273024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39a40b38f0 msgr2=0x7f39a4153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.294+0000 7f39a2800640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a4159ee0 0x7f39a4154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.294+0000 7f399be00640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39a40b38f0 0x7f39a4153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.294+0000 7f399be00640 1 -- 172.21.15.119:0/4195273024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a4159ee0 msgr2=0x7f39a4154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.294+0000 7f399be00640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a4159ee0 0x7f39a4154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.294+0000 7f399be00640 1 -- 172.21.15.119:0/4195273024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a40b04d0 con 0x7f39a40b3d80 2024-04-25T10:11:17.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.294+0000 7f39a2800640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a4159ee0 0x7f39a4154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:17.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.294+0000 7f399be00640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a40b3d80 0x7f39a4154230 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f398c00c430 tx=0x7f398c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.295+0000 7f399a000640 1 -- 172.21.15.119:0/4195273024 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c011790 con 0x7f39a40b3d80 2024-04-25T10:11:17.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.295+0000 7f399a000640 1 -- 172.21.15.119:0/4195273024 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c017070 con 0x7f39a40b3d80 2024-04-25T10:11:17.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.295+0000 7f399a000640 1 -- 172.21.15.119:0/4195273024 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c011b70 con 0x7f39a40b3d80 2024-04-25T10:11:17.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.295+0000 7f39a3200640 1 -- 172.21.15.119:0/4195273024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a40b07b0 con 0x7f39a40b3d80 2024-04-25T10:11:17.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.295+0000 7f39a3200640 1 -- 172.21.15.119:0/4195273024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a4154770 con 0x7f39a40b3d80 2024-04-25T10:11:17.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.296+0000 7f39a3200640 1 -- 172.21.15.119:0/4195273024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a40749b0 con 0x7f39a40b3d80 2024-04-25T10:11:17.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.297+0000 7f399a000640 1 -- 172.21.15.119:0/4195273024 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f398c011d10 con 0x7f39a40b3d80 2024-04-25T10:11:17.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.298+0000 7f399a000640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f398006b4d0 0x7f398006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.298+0000 7f399a000640 1 -- 172.21.15.119:0/4195273024 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f398c08a0d0 con 0x7f39a40b3d80 2024-04-25T10:11:17.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.298+0000 7f39a1e00640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f398006b4d0 0x7f398006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.299+0000 7f39a1e00640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f398006b4d0 0x7f398006d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f399400a770 tx=0x7f3994008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.302+0000 7f399a000640 1 -- 172.21.15.119:0/4195273024 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f398c058f40 con 0x7f39a40b3d80 2024-04-25T10:11:17.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.498+0000 7f39a3200640 1 -- 172.21.15.119:0/4195273024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39a406eb50 con 0x7f39a40b3d80 2024-04-25T10:11:17.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.499+0000 7f399a000640 1 -- 172.21.15.119:0/4195273024 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f398c058840 con 0x7f39a40b3d80 2024-04-25T10:11:17.504 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:11:17.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.501+0000 7f39a3200640 1 -- 172.21.15.119:0/4195273024 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f398006b4d0 msgr2=0x7f398006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.501+0000 7f39a3200640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f398006b4d0 0x7f398006d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f399400a770 tx=0x7f3994008040 comp rx=0 tx=0).stop 2024-04-25T10:11:17.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.501+0000 7f39a3200640 1 -- 172.21.15.119:0/4195273024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a40b3d80 msgr2=0x7f39a4154230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.501+0000 7f39a3200640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a40b3d80 0x7f39a4154230 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f398c00c430 tx=0x7f398c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.501+0000 7f39a3200640 1 -- 172.21.15.119:0/4195273024 shutdown_connections 2024-04-25T10:11:17.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.501+0000 7f39a3200640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f398006b4d0 0x7f398006d990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.501+0000 7f39a3200640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a4159ee0 0x7f39a4154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.501+0000 7f39a3200640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39a40b3d80 0x7f39a4154230 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.501+0000 7f39a3200640 1 --2- 172.21.15.119:0/4195273024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39a40b38f0 0x7f39a4153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.501+0000 7f39a3200640 1 -- 172.21.15.119:0/4195273024 >> 172.21.15.119:0/4195273024 conn(0x7f39a4066ff0 msgr2=0x7f39a406d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.502+0000 7f39a3200640 1 -- 172.21.15.119:0/4195273024 shutdown_connections 2024-04-25T10:11:17.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.502+0000 7f39a3200640 1 -- 172.21.15.119:0/4195273024 wait complete. 2024-04-25T10:11:17.525 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:17.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.719+0000 7fce86e00640 1 Processor -- start 2024-04-25T10:11:17.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.719+0000 7fce86e00640 1 -- start start 2024-04-25T10:11:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.721+0000 7fce86e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce8006aba0 0x7fce8006af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.721+0000 7fce86e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce8006f040 0x7fce8015e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.721+0000 7fce86e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8006b4c0 con 0x7fce800cec60 2024-04-25T10:11:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.721+0000 7fce86e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8006d080 con 0x7fce8006f040 2024-04-25T10:11:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.721+0000 7fce86e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8006d360 con 0x7fce8006aba0 2024-04-25T10:11:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.721+0000 7fce85000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce8006f040 0x7fce8015e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.722+0000 7fce85000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce8006f040 0x7fce8015e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44210/0 (socket says 172.21.15.119:44210) 2024-04-25T10:11:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.722+0000 7fce85000640 1 -- 172.21.15.119:0/3758574850 learned_addr learned my addr 172.21.15.119:0/3758574850 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.722+0000 7fce85a00640 1 --2- 172.21.15.119:0/3758574850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce8006aba0 0x7fce8006af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.722+0000 7fce85000640 1 -- 172.21.15.119:0/3758574850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce8006aba0 msgr2=0x7fce8006af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.722+0000 7fce85000640 1 --2- 172.21.15.119:0/3758574850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce8006aba0 0x7fce8006af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.722+0000 7fce85000640 1 -- 172.21.15.119:0/3758574850 >> v1:172.21.15.119:6789/0 conn(0x7fce800cec60 legacy=0x7fce8015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.722+0000 7fce85000640 1 -- 172.21.15.119:0/3758574850 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8006d640 con 0x7fce8006f040 2024-04-25T10:11:17.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.723+0000 7fce85000640 1 --2- 172.21.15.119:0/3758574850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8006f040 0x7fce8015e1e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00bb10 tx=0x7fce6c02c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e94b60e3042c2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.723+0000 7fce7fe00640 1 -- 172.21.15.119:0/3758574850 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c036020 con 0x7fce8006f040 2024-04-25T10:11:17.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.724+0000 7fce7fe00640 1 -- 172.21.15.119:0/3758574850 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c032820 con 0x7fce8006f040 2024-04-25T10:11:17.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.724+0000 7fce7fe00640 1 -- 172.21.15.119:0/3758574850 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c032bf0 con 0x7fce8006f040 2024-04-25T10:11:17.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.724+0000 7fce86e00640 1 -- 172.21.15.119:0/3758574850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8006f040 msgr2=0x7fce8015e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.724+0000 7fce86e00640 1 --2- 172.21.15.119:0/3758574850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8006f040 0x7fce8015e1e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00bb10 tx=0x7fce6c02c920 comp rx=0 tx=0).stop 2024-04-25T10:11:17.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.724+0000 7fce86e00640 1 -- 172.21.15.119:0/3758574850 shutdown_connections 2024-04-25T10:11:17.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.724+0000 7fce86e00640 1 --2- 172.21.15.119:0/3758574850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8006f040 0x7fce8015e1e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.724+0000 7fce86e00640 1 --2- 172.21.15.119:0/3758574850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce8006aba0 0x7fce8006af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.724+0000 7fce86e00640 1 -- 172.21.15.119:0/3758574850 >> 172.21.15.119:0/3758574850 conn(0x7fce80066ff0 msgr2=0x7fce800b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.725+0000 7fce86e00640 1 -- 172.21.15.119:0/3758574850 shutdown_connections 2024-04-25T10:11:17.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.725+0000 7fce86e00640 1 -- 172.21.15.119:0/3758574850 wait complete. 2024-04-25T10:11:17.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.727+0000 7fce86e00640 1 Processor -- start 2024-04-25T10:11:17.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.727+0000 7fce86e00640 1 -- start start 2024-04-25T10:11:17.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.728+0000 7fce86e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce8006aba0 0x7fce8014a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.728+0000 7fce86e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8006f040 0x7fce800cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.728+0000 7fce86e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce800cec60 0x7fce800cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.728+0000 7fce86e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8015a980 con 0x7fce8006f040 2024-04-25T10:11:17.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.728+0000 7fce86e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8006dc00 con 0x7fce800cec60 2024-04-25T10:11:17.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.728+0000 7fce86e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce800aed40 con 0x7fce8006aba0 2024-04-25T10:11:17.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.728+0000 7fce85000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8006f040 0x7fce800cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.728+0000 7fce85000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8006f040 0x7fce800cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44222/0 (socket says 172.21.15.119:44222) 2024-04-25T10:11:17.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.728+0000 7fce85000640 1 -- 172.21.15.119:0/3137763131 learned_addr learned my addr 172.21.15.119:0/3137763131 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:17.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.729+0000 7fce85a00640 1 --2- 172.21.15.119:0/3137763131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce8006aba0 0x7fce8014a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.729+0000 7fce85000640 1 -- 172.21.15.119:0/3137763131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce8006aba0 msgr2=0x7fce8014a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.729+0000 7fce85000640 1 --2- 172.21.15.119:0/3137763131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce8006aba0 0x7fce8014a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.729+0000 7fce86400640 1 --2- 172.21.15.119:0/3137763131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce800cec60 0x7fce800cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.729+0000 7fce85000640 1 -- 172.21.15.119:0/3137763131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce800cec60 msgr2=0x7fce800cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.729+0000 7fce85000640 1 --2- 172.21.15.119:0/3137763131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce800cec60 0x7fce800cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.729+0000 7fce85000640 1 -- 172.21.15.119:0/3137763131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8014c2b0 con 0x7fce8006f040 2024-04-25T10:11:17.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.729+0000 7fce85000640 1 --2- 172.21.15.119:0/3137763131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8006f040 0x7fce800cd4a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fce6c000c00 tx=0x7fce6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.730+0000 7fce7d600640 1 -- 172.21.15.119:0/3137763131 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c032620 con 0x7fce8006f040 2024-04-25T10:11:17.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.730+0000 7fce7d600640 1 -- 172.21.15.119:0/3137763131 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c03bce0 con 0x7fce8006f040 2024-04-25T10:11:17.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.730+0000 7fce7d600640 1 -- 172.21.15.119:0/3137763131 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c044630 con 0x7fce8006f040 2024-04-25T10:11:17.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.730+0000 7fce86e00640 1 -- 172.21.15.119:0/3137763131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8015ad50 con 0x7fce8006f040 2024-04-25T10:11:17.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.730+0000 7fce86e00640 1 -- 172.21.15.119:0/3137763131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8015b310 con 0x7fce8006f040 2024-04-25T10:11:17.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.731+0000 7fce86e00640 1 -- 172.21.15.119:0/3137763131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce800749b0 con 0x7fce8006f040 2024-04-25T10:11:17.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.732+0000 7fce7d600640 1 -- 172.21.15.119:0/3137763131 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce6c039020 con 0x7fce8006f040 2024-04-25T10:11:17.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.733+0000 7fce7d600640 1 --2- 172.21.15.119:0/3137763131 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fce6006b4d0 0x7fce6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.733+0000 7fce7d600640 1 -- 172.21.15.119:0/3137763131 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce6c0ad5e0 con 0x7fce8006f040 2024-04-25T10:11:17.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.734+0000 7fce85a00640 1 --2- 172.21.15.119:0/3137763131 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fce6006b4d0 0x7fce6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.734+0000 7fce85a00640 1 --2- 172.21.15.119:0/3137763131 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fce6006b4d0 0x7fce6006d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fce74006fd0 tx=0x7fce74008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.738+0000 7fce7d600640 1 -- 172.21.15.119:0/3137763131 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce6c02f070 con 0x7fce8006f040 2024-04-25T10:11:17.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.931+0000 7fce86e00640 1 -- 172.21.15.119:0/3137763131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce800cf1b0 con 0x7fce8006f040 2024-04-25T10:11:17.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.931+0000 7fce7d600640 1 -- 172.21.15.119:0/3137763131 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce6c07c510 con 0x7fce8006f040 2024-04-25T10:11:17.937 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:11:17.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.934+0000 7fce86e00640 1 -- 172.21.15.119:0/3137763131 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fce6006b4d0 msgr2=0x7fce6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.934+0000 7fce86e00640 1 --2- 172.21.15.119:0/3137763131 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fce6006b4d0 0x7fce6006d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fce74006fd0 tx=0x7fce74008040 comp rx=0 tx=0).stop 2024-04-25T10:11:17.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.934+0000 7fce86e00640 1 -- 172.21.15.119:0/3137763131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8006f040 msgr2=0x7fce800cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.934+0000 7fce86e00640 1 --2- 172.21.15.119:0/3137763131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8006f040 0x7fce800cd4a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fce6c000c00 tx=0x7fce6c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.934+0000 7fce86e00640 1 -- 172.21.15.119:0/3137763131 shutdown_connections 2024-04-25T10:11:17.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.934+0000 7fce86e00640 1 --2- 172.21.15.119:0/3137763131 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fce6006b4d0 0x7fce6006d990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.934+0000 7fce86e00640 1 --2- 172.21.15.119:0/3137763131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce800cec60 0x7fce800cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.934+0000 7fce86e00640 1 --2- 172.21.15.119:0/3137763131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce8006f040 0x7fce800cd4a0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.934+0000 7fce86e00640 1 --2- 172.21.15.119:0/3137763131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce8006aba0 0x7fce8014a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.934+0000 7fce86e00640 1 -- 172.21.15.119:0/3137763131 >> 172.21.15.119:0/3137763131 conn(0x7fce80066ff0 msgr2=0x7fce8015e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.934+0000 7fce86e00640 1 -- 172.21.15.119:0/3137763131 shutdown_connections 2024-04-25T10:11:17.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:17.934+0000 7fce86e00640 1 -- 172.21.15.119:0/3137763131 wait complete. 2024-04-25T10:11:17.957 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:18.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.157+0000 7fc76de00640 1 Processor -- start 2024-04-25T10:11:18.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.158+0000 7fc76de00640 1 -- start start 2024-04-25T10:11:18.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.159+0000 7fc76de00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7680d07b0 0x7fc7680cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.159+0000 7fc76de00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7680b5710 con 0x7fc76815b7e0 2024-04-25T10:11:18.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.159+0000 7fc76de00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7680b59f0 con 0x7fc7680ce860 2024-04-25T10:11:18.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.159+0000 7fc76de00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7680b5cd0 con 0x7fc7680d07b0 2024-04-25T10:11:18.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.159+0000 7fc76d400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7680d07b0 0x7fc7680cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.159+0000 7fc76ca00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc76815b7e0 0x7fc76815e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54940/0 (socket says 172.21.15.119:54940) 2024-04-25T10:11:18.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.159+0000 7fc76ca00640 1 -- 172.21.15.119:0/628196933 learned_addr learned my addr 172.21.15.119:0/628196933 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:18.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.160+0000 7fc76d400640 1 -- 172.21.15.119:0/628196933 >> v1:172.21.15.132:6789/0 conn(0x7fc7680ce860 legacy=0x7fc7680d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:18.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.160+0000 7fc76d400640 1 -- 172.21.15.119:0/628196933 >> v1:172.21.15.119:6789/0 conn(0x7fc76815b7e0 legacy=0x7fc76815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.160+0000 7fc76d400640 1 -- 172.21.15.119:0/628196933 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7680b5fb0 con 0x7fc7680d07b0 2024-04-25T10:11:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.161+0000 7fc76d400640 1 --2- 172.21.15.119:0/628196933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7680d07b0 0x7fc7680cecf0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc760009d90 tx=0x7fc76002a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391f3614fbb31ef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.161+0000 7fc767400640 1 -- 172.21.15.119:0/628196933 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760007740 con 0x7fc7680d07b0 2024-04-25T10:11:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.161+0000 7fc767400640 1 -- 172.21.15.119:0/628196933 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7600078e0 con 0x7fc7680d07b0 2024-04-25T10:11:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.161+0000 7fc767400640 1 -- 172.21.15.119:0/628196933 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760007cb0 con 0x7fc7680d07b0 2024-04-25T10:11:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.162+0000 7fc76de00640 1 -- 172.21.15.119:0/628196933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7680d07b0 msgr2=0x7fc7680cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.162+0000 7fc76de00640 1 --2- 172.21.15.119:0/628196933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7680d07b0 0x7fc7680cecf0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc760009d90 tx=0x7fc76002a590 comp rx=0 tx=0).stop 2024-04-25T10:11:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.162+0000 7fc76de00640 1 -- 172.21.15.119:0/628196933 shutdown_connections 2024-04-25T10:11:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.162+0000 7fc76de00640 1 --2- 172.21.15.119:0/628196933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7680d07b0 0x7fc7680cecf0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.162+0000 7fc76de00640 1 -- 172.21.15.119:0/628196933 >> 172.21.15.119:0/628196933 conn(0x7fc7680722d0 msgr2=0x7fc7680726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.162+0000 7fc76de00640 1 -- 172.21.15.119:0/628196933 shutdown_connections 2024-04-25T10:11:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.162+0000 7fc76de00640 1 -- 172.21.15.119:0/628196933 wait complete. 2024-04-25T10:11:18.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.164+0000 7fc76de00640 1 Processor -- start 2024-04-25T10:11:18.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.164+0000 7fc76de00640 1 -- start start 2024-04-25T10:11:18.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.165+0000 7fc76de00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7680ce860 0x7fc7681f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.165+0000 7fc76de00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7680d07b0 0x7fc768148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.165+0000 7fc76de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc76815b7e0 0x7fc768160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.165+0000 7fc76de00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc768161ae0 con 0x7fc76815b7e0 2024-04-25T10:11:18.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.165+0000 7fc76de00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc768161960 con 0x7fc7680ce860 2024-04-25T10:11:18.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.165+0000 7fc76de00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc768161c60 con 0x7fc7680d07b0 2024-04-25T10:11:18.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.165+0000 7fc76d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc76815b7e0 0x7fc768160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.165+0000 7fc76d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc76815b7e0 0x7fc768160570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44236/0 (socket says 172.21.15.119:44236) 2024-04-25T10:11:18.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.165+0000 7fc76d400640 1 -- 172.21.15.119:0/441933622 learned_addr learned my addr 172.21.15.119:0/441933622 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:18.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc76ca00640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7680ce860 0x7fc7681f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc767e00640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7680d07b0 0x7fc768148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc76d400640 1 -- 172.21.15.119:0/441933622 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7680d07b0 msgr2=0x7fc768148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc76d400640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7680d07b0 0x7fc768148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc76d400640 1 -- 172.21.15.119:0/441933622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7680ce860 msgr2=0x7fc7681f7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc76d400640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7680ce860 0x7fc7681f7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc76d400640 1 -- 172.21.15.119:0/441933622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7681f83d0 con 0x7fc76815b7e0 2024-04-25T10:11:18.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc767e00640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7680d07b0 0x7fc768148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:18.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc76ca00640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7680ce860 0x7fc7681f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:18.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc76d400640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc76815b7e0 0x7fc768160570 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc760009d90 tx=0x7fc760004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc764c00640 1 -- 172.21.15.119:0/441933622 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760040070 con 0x7fc76815b7e0 2024-04-25T10:11:18.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc764c00640 1 -- 172.21.15.119:0/441933622 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc760031c90 con 0x7fc76815b7e0 2024-04-25T10:11:18.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc764c00640 1 -- 172.21.15.119:0/441933622 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76003c640 con 0x7fc76815b7e0 2024-04-25T10:11:18.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc76de00640 1 -- 172.21.15.119:0/441933622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc768149500 con 0x7fc76815b7e0 2024-04-25T10:11:18.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.166+0000 7fc76de00640 1 -- 172.21.15.119:0/441933622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76815f7b0 con 0x7fc76815b7e0 2024-04-25T10:11:18.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.167+0000 7fc76de00640 1 -- 172.21.15.119:0/441933622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7680cecd0 con 0x7fc76815b7e0 2024-04-25T10:11:18.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.169+0000 7fc764c00640 1 -- 172.21.15.119:0/441933622 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc760039040 con 0x7fc76815b7e0 2024-04-25T10:11:18.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.169+0000 7fc764c00640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc74806b500 0x7fc74806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.169+0000 7fc764c00640 1 -- 172.21.15.119:0/441933622 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7600ab910 con 0x7fc76815b7e0 2024-04-25T10:11:18.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.170+0000 7fc76ca00640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc74806b500 0x7fc74806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.170+0000 7fc76ca00640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc74806b500 0x7fc74806d9c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc758000f60 tx=0x7fc758009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.174+0000 7fc764c00640 1 -- 172.21.15.119:0/441933622 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc76004b340 con 0x7fc76815b7e0 2024-04-25T10:11:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.366+0000 7fc76de00640 1 -- 172.21.15.119:0/441933622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc7680749b0 con 0x7fc76815b7e0 2024-04-25T10:11:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.367+0000 7fc764c00640 1 -- 172.21.15.119:0/441933622 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc76007a7b0 con 0x7fc76815b7e0 2024-04-25T10:11:18.373 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:11:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.369+0000 7fc76de00640 1 -- 172.21.15.119:0/441933622 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc74806b500 msgr2=0x7fc74806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.369+0000 7fc76de00640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc74806b500 0x7fc74806d9c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc758000f60 tx=0x7fc758009040 comp rx=0 tx=0).stop 2024-04-25T10:11:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.369+0000 7fc76de00640 1 -- 172.21.15.119:0/441933622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc76815b7e0 msgr2=0x7fc768160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.369+0000 7fc76de00640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc76815b7e0 0x7fc768160570 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc760009d90 tx=0x7fc760004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.370+0000 7fc76de00640 1 -- 172.21.15.119:0/441933622 shutdown_connections 2024-04-25T10:11:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.370+0000 7fc76de00640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc74806b500 0x7fc74806d9c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.370+0000 7fc76de00640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc76815b7e0 0x7fc768160570 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.370+0000 7fc76de00640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7680d07b0 0x7fc768148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.370+0000 7fc76de00640 1 --2- 172.21.15.119:0/441933622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7680ce860 0x7fc7681f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.370+0000 7fc76de00640 1 -- 172.21.15.119:0/441933622 >> 172.21.15.119:0/441933622 conn(0x7fc7680722d0 msgr2=0x7fc76815ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.370+0000 7fc76de00640 1 -- 172.21.15.119:0/441933622 shutdown_connections 2024-04-25T10:11:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.370+0000 7fc76de00640 1 -- 172.21.15.119:0/441933622 wait complete. 2024-04-25T10:11:18.395 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:18.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.583+0000 7f93b9a00640 1 Processor -- start 2024-04-25T10:11:18.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.583+0000 7f93b9a00640 1 -- start start 2024-04-25T10:11:18.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.584+0000 7f93b9a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93b415dce0 0x7f93b40d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.584+0000 7f93b9a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93b40b5b60 0x7f93b40b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.584+0000 7f93b9a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b415e0c0 con 0x7f93b40d35b0 2024-04-25T10:11:18.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.584+0000 7f93b9a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b40d2510 con 0x7f93b40b5b60 2024-04-25T10:11:18.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.584+0000 7f93b9a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b40d27f0 con 0x7f93b415dce0 2024-04-25T10:11:18.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.585+0000 7f93b3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93b40b5b60 0x7f93b40b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.585+0000 7f93b3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93b40b5b60 0x7f93b40b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44242/0 (socket says 172.21.15.119:44242) 2024-04-25T10:11:18.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.585+0000 7f93b3400640 1 -- 172.21.15.119:0/1397564653 learned_addr learned my addr 172.21.15.119:0/1397564653 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:18.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.585+0000 7f93b2a00640 1 --2- 172.21.15.119:0/1397564653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93b415dce0 0x7f93b40d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.585+0000 7f93b3400640 1 -- 172.21.15.119:0/1397564653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93b415dce0 msgr2=0x7f93b40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:18.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.585+0000 7f93b3400640 1 --2- 172.21.15.119:0/1397564653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93b415dce0 0x7f93b40d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.585+0000 7f93b3400640 1 -- 172.21.15.119:0/1397564653 >> v1:172.21.15.132:6789/0 conn(0x7f93b40d35b0 legacy=0x7f93b40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:18.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.585+0000 7f93b3400640 1 -- 172.21.15.119:0/1397564653 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b40d2ad0 con 0x7f93b40b5b60 2024-04-25T10:11:18.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.586+0000 7f93b3400640 1 --2- 172.21.15.119:0/1397564653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b40b5b60 0x7f93b40b6b00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009e50 tx=0x7f93ac02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad1b558d06cbcf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.587+0000 7f93b2000640 1 -- 172.21.15.119:0/1397564653 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac036070 con 0x7f93b40b5b60 2024-04-25T10:11:18.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.587+0000 7f93b2000640 1 -- 172.21.15.119:0/1397564653 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ac031600 con 0x7f93b40b5b60 2024-04-25T10:11:18.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.587+0000 7f93b2000640 1 -- 172.21.15.119:0/1397564653 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0319d0 con 0x7f93b40b5b60 2024-04-25T10:11:18.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.587+0000 7f93b9a00640 1 -- 172.21.15.119:0/1397564653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b40b5b60 msgr2=0x7f93b40b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.587+0000 7f93b9a00640 1 --2- 172.21.15.119:0/1397564653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b40b5b60 0x7f93b40b6b00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009e50 tx=0x7f93ac02a710 comp rx=0 tx=0).stop 2024-04-25T10:11:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.587+0000 7f93b9a00640 1 -- 172.21.15.119:0/1397564653 shutdown_connections 2024-04-25T10:11:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.587+0000 7f93b9a00640 1 --2- 172.21.15.119:0/1397564653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b40b5b60 0x7f93b40b6b00 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.587+0000 7f93b9a00640 1 --2- 172.21.15.119:0/1397564653 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93b415dce0 0x7f93b40d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.587+0000 7f93b9a00640 1 -- 172.21.15.119:0/1397564653 >> 172.21.15.119:0/1397564653 conn(0x7f93b4072900 msgr2=0x7f93b40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.588+0000 7f93b9a00640 1 -- 172.21.15.119:0/1397564653 shutdown_connections 2024-04-25T10:11:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.588+0000 7f93b9a00640 1 -- 172.21.15.119:0/1397564653 wait complete. 2024-04-25T10:11:18.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.589+0000 7f93b9a00640 1 Processor -- start 2024-04-25T10:11:18.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.589+0000 7f93b9a00640 1 -- start start 2024-04-25T10:11:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.590+0000 7f93b9a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93b40b5b60 0x7f93b40cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.590+0000 7f93b9a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b40d35b0 0x7f93b4160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.590+0000 7f93b9a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93b415dce0 0x7f93b41f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.590+0000 7f93b9a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b40b9820 con 0x7f93b40d35b0 2024-04-25T10:11:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.590+0000 7f93b9a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b40d3090 con 0x7f93b415dce0 2024-04-25T10:11:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.590+0000 7f93b9a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b40b99a0 con 0x7f93b40b5b60 2024-04-25T10:11:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.591+0000 7f93aa000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b40d35b0 0x7f93b4160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.591+0000 7f93aa000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b40d35b0 0x7f93b4160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44248/0 (socket says 172.21.15.119:44248) 2024-04-25T10:11:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.591+0000 7f93aa000640 1 -- 172.21.15.119:0/627491157 learned_addr learned my addr 172.21.15.119:0/627491157 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:18.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.591+0000 7f93b2a00640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93b40b5b60 0x7f93b40cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.591+0000 7f93b3400640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93b415dce0 0x7f93b41f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.591+0000 7f93aa000640 1 -- 172.21.15.119:0/627491157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93b40b5b60 msgr2=0x7f93b40cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.591+0000 7f93aa000640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93b40b5b60 0x7f93b40cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.591+0000 7f93aa000640 1 -- 172.21.15.119:0/627491157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93b415dce0 msgr2=0x7f93b41f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.591+0000 7f93aa000640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93b415dce0 0x7f93b41f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.591+0000 7f93aa000640 1 -- 172.21.15.119:0/627491157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b40cbb90 con 0x7f93b40d35b0 2024-04-25T10:11:18.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.591+0000 7f93b3400640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93b415dce0 0x7f93b41f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:18.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.592+0000 7f93aa000640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b40d35b0 0x7f93b4160b40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f939c00c430 tx=0x7f939c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.592+0000 7f93ab400640 1 -- 172.21.15.119:0/627491157 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c0052d0 con 0x7f93b40d35b0 2024-04-25T10:11:18.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.592+0000 7f93ab400640 1 -- 172.21.15.119:0/627491157 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939c005490 con 0x7f93b40d35b0 2024-04-25T10:11:18.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.592+0000 7f93ab400640 1 -- 172.21.15.119:0/627491157 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c007260 con 0x7f93b40d35b0 2024-04-25T10:11:18.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.592+0000 7f93b9a00640 1 -- 172.21.15.119:0/627491157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b40cbe70 con 0x7f93b40d35b0 2024-04-25T10:11:18.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.592+0000 7f93b9a00640 1 -- 172.21.15.119:0/627491157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b41f3970 con 0x7f93b40d35b0 2024-04-25T10:11:18.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.593+0000 7f93b9a00640 1 -- 172.21.15.119:0/627491157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b40749b0 con 0x7f93b40d35b0 2024-04-25T10:11:18.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.594+0000 7f93ab400640 1 -- 172.21.15.119:0/627491157 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f939c007400 con 0x7f93b40d35b0 2024-04-25T10:11:18.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.595+0000 7f93ab400640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f939006b4d0 0x7f939006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.595+0000 7f93ab400640 1 -- 172.21.15.119:0/627491157 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f939c089710 con 0x7f93b40d35b0 2024-04-25T10:11:18.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.595+0000 7f93b2a00640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f939006b4d0 0x7f939006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.596+0000 7f93b2a00640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f939006b4d0 0x7f939006d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f93a4006fd0 tx=0x7f93a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.599+0000 7f93ab400640 1 -- 172.21.15.119:0/627491157 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f939c058570 con 0x7f93b40d35b0 2024-04-25T10:11:18.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.825+0000 7f93b9a00640 1 -- 172.21.15.119:0/627491157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93b415d4e0 con 0x7f93b40d35b0 2024-04-25T10:11:18.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.826+0000 7f93ab400640 1 -- 172.21.15.119:0/627491157 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f939c057e70 con 0x7f93b40d35b0 2024-04-25T10:11:18.832 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:18.832 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4159222235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4159222235},{"type":"v1","addr":"172.21.15.132:6834","nonce":4159222235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4223361610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":4223361610},{"type":"v1","addr":"172.21.15.119:6837","nonce":4223361610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2367324442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2367324442},{"type":"v1","addr":"172.21.15.132:6835","nonce":2367324442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:59.454941+0000","modified":"2024-04-25T10:10:48.089168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6836/4129112885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4129112885},{"type":"v1","addr":"172.21.15.119:6836","nonce":4129112885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:11:18.832 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 36 2024-04-25T10:11:18.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.829+0000 7f93b9a00640 1 -- 172.21.15.119:0/627491157 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f939006b4d0 msgr2=0x7f939006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.829+0000 7f93b9a00640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f939006b4d0 0x7f939006d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f93a4006fd0 tx=0x7f93a4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:18.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.829+0000 7f93b9a00640 1 -- 172.21.15.119:0/627491157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b40d35b0 msgr2=0x7f93b4160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.829+0000 7f93b9a00640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b40d35b0 0x7f93b4160b40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f939c00c430 tx=0x7f939c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.830+0000 7f93b9a00640 1 -- 172.21.15.119:0/627491157 shutdown_connections 2024-04-25T10:11:18.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.830+0000 7f93b9a00640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f939006b4d0 0x7f939006d990 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.830+0000 7f93b9a00640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93b415dce0 0x7f93b41f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.830+0000 7f93b9a00640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b40d35b0 0x7f93b4160b40 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.830+0000 7f93b9a00640 1 --2- 172.21.15.119:0/627491157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93b40b5b60 0x7f93b40cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.830+0000 7f93b9a00640 1 -- 172.21.15.119:0/627491157 >> 172.21.15.119:0/627491157 conn(0x7f93b4072900 msgr2=0x7f93b40b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:18.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.830+0000 7f93b9a00640 1 -- 172.21.15.119:0/627491157 shutdown_connections 2024-04-25T10:11:18.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:18.830+0000 7f93b9a00640 1 -- 172.21.15.119:0/627491157 wait complete. 2024-04-25T10:11:18.854 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:19.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.045+0000 7fc7dec00640 1 Processor -- start 2024-04-25T10:11:19.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.045+0000 7fc7dec00640 1 -- start start 2024-04-25T10:11:19.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.047+0000 7fc7dec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7d815ded0 0x7fc7d80cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.047+0000 7fc7dec00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc7d806fab0 0x7fc7d8070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.047+0000 7fc7dec00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d80c6540 con 0x7fc7d806ff10 2024-04-25T10:11:19.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.047+0000 7fc7dec00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d8072bc0 con 0x7fc7d806fab0 2024-04-25T10:11:19.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.047+0000 7fc7dec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80ccfc0 con 0x7fc7d815ded0 2024-04-25T10:11:19.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.047+0000 7fc7dcc00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc7d806ff10 0x7fc7d80cad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54950/0 (socket says 172.21.15.119:54950) 2024-04-25T10:11:19.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.047+0000 7fc7dcc00640 1 -- 172.21.15.119:0/1230182459 learned_addr learned my addr 172.21.15.119:0/1230182459 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:19.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d7e00640 1 --2- 172.21.15.119:0/1230182459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7d815ded0 0x7fc7d80cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d7400640 1 --2- 172.21.15.119:0/1230182459 >> v2:172.21.15.132:3300/0 conn(0x7fc7d806fab0 0x7fc7d8070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 -- 172.21.15.119:0/1230182459 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896396902 0 0) 0x7fc7d80c6540 con 0x7fc7d806ff10 2024-04-25T10:11:19.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 -- 172.21.15.119:0/1230182459 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7bc003520 con 0x7fc7d806ff10 2024-04-25T10:11:19.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 -- 172.21.15.119:0/1230182459 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3648004384 0 0) 0x7fc7bc003520 con 0x7fc7d806ff10 2024-04-25T10:11:19.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 -- 172.21.15.119:0/1230182459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7d815ded0 msgr2=0x7fc7d80cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 --2- 172.21.15.119:0/1230182459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7d815ded0 0x7fc7d80cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 -- 172.21.15.119:0/1230182459 >> v2:172.21.15.132:3300/0 conn(0x7fc7d806fab0 msgr2=0x7fc7d8070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 --2- 172.21.15.119:0/1230182459 >> v2:172.21.15.132:3300/0 conn(0x7fc7d806fab0 0x7fc7d8070320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 -- 172.21.15.119:0/1230182459 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d8072e10 con 0x7fc7d806ff10 2024-04-25T10:11:19.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 -- 172.21.15.119:0/1230182459 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fc7d00019f0 con 0x7fc7d806ff10 2024-04-25T10:11:19.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 -- 172.21.15.119:0/1230182459 >> v1:172.21.15.119:6789/0 conn(0x7fc7d806ff10 legacy=0x7fc7d80cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 --2- 172.21.15.119:0/1230182459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7bc0039b0 0x7fc7bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 --2- 172.21.15.119:0/1230182459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7bc004800 0x7fc7bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 --2- 172.21.15.119:0/1230182459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7d815ded0 0x7fc7bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 -- 172.21.15.119:0/1230182459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8072bc0 con 0x7fc7bc004800 2024-04-25T10:11:19.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 -- 172.21.15.119:0/1230182459 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80ccfc0 con 0x7fc7bc0039b0 2024-04-25T10:11:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d6a00640 1 -- 172.21.15.119:0/1230182459 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc00baa0 con 0x7fc7d815ded0 2024-04-25T10:11:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.048+0000 7fc7d7400640 1 --2- 172.21.15.119:0/1230182459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7bc004800 0x7fc7bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.049+0000 7fc7dcc00640 1 --2- 172.21.15.119:0/1230182459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7d815ded0 0x7fc7bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.049+0000 7fc7d7400640 1 -- 172.21.15.119:0/1230182459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7d815ded0 msgr2=0x7fc7bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.049+0000 7fc7d7400640 1 --2- 172.21.15.119:0/1230182459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7d815ded0 0x7fc7bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.049+0000 7fc7d7400640 1 -- 172.21.15.119:0/1230182459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7bc0039b0 msgr2=0x7fc7bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.049+0000 7fc7d7400640 1 --2- 172.21.15.119:0/1230182459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7bc0039b0 0x7fc7bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.049+0000 7fc7d7400640 1 -- 172.21.15.119:0/1230182459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc003520 con 0x7fc7bc004800 2024-04-25T10:11:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.049+0000 7fc7dcc00640 1 --2- 172.21.15.119:0/1230182459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7d815ded0 0x7fc7bc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.050+0000 7fc7d7400640 1 --2- 172.21.15.119:0/1230182459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7bc004800 0x7fc7bc004c80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400ba30 tx=0x7fc7bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4634676e9c14c00d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.050+0000 7fc7d6a00640 1 -- 172.21.15.119:0/1230182459 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4030070 con 0x7fc7bc004800 2024-04-25T10:11:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.050+0000 7fc7d6a00640 1 -- 172.21.15.119:0/1230182459 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c402ae10 con 0x7fc7bc004800 2024-04-25T10:11:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.050+0000 7fc7d6a00640 1 -- 172.21.15.119:0/1230182459 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c400b600 con 0x7fc7bc004800 2024-04-25T10:11:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.050+0000 7fc7dec00640 1 -- 172.21.15.119:0/1230182459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7bc004800 msgr2=0x7fc7bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.050+0000 7fc7dec00640 1 --2- 172.21.15.119:0/1230182459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7bc004800 0x7fc7bc004c80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400ba30 tx=0x7fc7bc005830 comp rx=0 tx=0).stop 2024-04-25T10:11:19.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.050+0000 7fc7dcc00640 1 -- 172.21.15.119:0/1230182459 reap_dead start 2024-04-25T10:11:19.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.051+0000 7fc7dec00640 1 -- 172.21.15.119:0/1230182459 shutdown_connections 2024-04-25T10:11:19.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.051+0000 7fc7dec00640 1 -- 172.21.15.119:0/1230182459 >> 172.21.15.119:0/1230182459 conn(0x7fc7d80b2370 msgr2=0x7fc7d80b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.051+0000 7fc7dec00640 1 -- 172.21.15.119:0/1230182459 shutdown_connections 2024-04-25T10:11:19.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.051+0000 7fc7dec00640 1 -- 172.21.15.119:0/1230182459 wait complete. 2024-04-25T10:11:19.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.052+0000 7fc7dec00640 1 Processor -- start 2024-04-25T10:11:19.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.052+0000 7fc7dec00640 1 -- start start 2024-04-25T10:11:19.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.053+0000 7fc7dec00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7bc004800 0x7fc7d80b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.053+0000 7fc7dec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7d80b3460 0x7fc7d81f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.053+0000 7fc7dec00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7d80b1210 0x7fc7d80c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.053+0000 7fc7dec00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80cd4f0 con 0x7fc7d80b1210 2024-04-25T10:11:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.053+0000 7fc7dec00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80cd210 con 0x7fc7bc004800 2024-04-25T10:11:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.053+0000 7fc7dec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80cb4a0 con 0x7fc7d80b3460 2024-04-25T10:11:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.054+0000 7fc7dcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7d80b1210 0x7fc7d80c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.054+0000 7fc7dcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7d80b1210 0x7fc7d80c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44264/0 (socket says 172.21.15.119:44264) 2024-04-25T10:11:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.054+0000 7fc7dcc00640 1 -- 172.21.15.119:0/2200220021 learned_addr learned my addr 172.21.15.119:0/2200220021 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.054+0000 7fc7d7400640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7d80b3460 0x7fc7d81f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.054+0000 7fc7d7e00640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7bc004800 0x7fc7d80b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.054+0000 7fc7dcc00640 1 -- 172.21.15.119:0/2200220021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7d80b3460 msgr2=0x7fc7d81f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.054+0000 7fc7dcc00640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7d80b3460 0x7fc7d81f77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.054+0000 7fc7dcc00640 1 -- 172.21.15.119:0/2200220021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7bc004800 msgr2=0x7fc7d80b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.054+0000 7fc7dcc00640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7bc004800 0x7fc7d80b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.054+0000 7fc7dcc00640 1 -- 172.21.15.119:0/2200220021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d81f6d20 con 0x7fc7d80b1210 2024-04-25T10:11:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.054+0000 7fc7d7e00640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7bc004800 0x7fc7d80b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:19.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.054+0000 7fc7dcc00640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7d80b1210 0x7fc7d80c7810 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80b31b0 tx=0x7fc7d000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.055+0000 7fc7cfe00640 1 -- 172.21.15.119:0/2200220021 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d000bb20 con 0x7fc7d80b1210 2024-04-25T10:11:19.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.055+0000 7fc7cfe00640 1 -- 172.21.15.119:0/2200220021 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d000bcc0 con 0x7fc7d80b1210 2024-04-25T10:11:19.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.055+0000 7fc7cfe00640 1 -- 172.21.15.119:0/2200220021 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0007290 con 0x7fc7d80b1210 2024-04-25T10:11:19.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.055+0000 7fc7dec00640 1 -- 172.21.15.119:0/2200220021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d8160ec0 con 0x7fc7d80b1210 2024-04-25T10:11:19.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.055+0000 7fc7dec00640 1 -- 172.21.15.119:0/2200220021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d81f72a0 con 0x7fc7d80b1210 2024-04-25T10:11:19.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.056+0000 7fc7dec00640 1 -- 172.21.15.119:0/2200220021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d80749b0 con 0x7fc7d80b1210 2024-04-25T10:11:19.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.057+0000 7fc7cfe00640 1 -- 172.21.15.119:0/2200220021 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7d001b020 con 0x7fc7d80b1210 2024-04-25T10:11:19.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.058+0000 7fc7cfe00640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc7b806b4d0 0x7fc7b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.058+0000 7fc7cfe00640 1 -- 172.21.15.119:0/2200220021 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7d008a3b0 con 0x7fc7d80b1210 2024-04-25T10:11:19.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.058+0000 7fc7d7e00640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc7b806b4d0 0x7fc7b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.059+0000 7fc7d7e00640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc7b806b4d0 0x7fc7b806d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8004770 tx=0x7fc7c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.062+0000 7fc7cfe00640 1 -- 172.21.15.119:0/2200220021 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7d0059230 con 0x7fc7d80b1210 2024-04-25T10:11:19.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.257+0000 7fc7dec00640 1 -- 172.21.15.119:0/2200220021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7d80b6740 con 0x7fc7d80b1210 2024-04-25T10:11:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.259+0000 7fc7cfe00640 1 -- 172.21.15.119:0/2200220021 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16701 (secure 0 0 0) 0x7fc7d0058b30 con 0x7fc7d80b1210 2024-04-25T10:11:19.264 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:19.265 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":55,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:10:01.035989+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:57.237546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:58.475727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.261+0000 7fc7dec00640 1 -- 172.21.15.119:0/2200220021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc7b806b4d0 msgr2=0x7fc7b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.261+0000 7fc7dec00640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc7b806b4d0 0x7fc7b806d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8004770 tx=0x7fc7c8008040 comp rx=0 tx=0).stop 2024-04-25T10:11:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.261+0000 7fc7dec00640 1 -- 172.21.15.119:0/2200220021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7d80b1210 msgr2=0x7fc7d80c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.261+0000 7fc7dec00640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7d80b1210 0x7fc7d80c7810 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80b31b0 tx=0x7fc7d000b4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.262+0000 7fc7dec00640 1 -- 172.21.15.119:0/2200220021 shutdown_connections 2024-04-25T10:11:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.262+0000 7fc7dec00640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc7b806b4d0 0x7fc7b806d990 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.262+0000 7fc7dec00640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7d80b1210 0x7fc7d80c7810 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.262+0000 7fc7dec00640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7d80b3460 0x7fc7d81f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.262+0000 7fc7dec00640 1 --2- 172.21.15.119:0/2200220021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7bc004800 0x7fc7d80b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.262+0000 7fc7dec00640 1 -- 172.21.15.119:0/2200220021 >> 172.21.15.119:0/2200220021 conn(0x7fc7d80b2370 msgr2=0x7fc7d806aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.262+0000 7fc7dec00640 1 -- 172.21.15.119:0/2200220021 shutdown_connections 2024-04-25T10:11:19.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.262+0000 7fc7dec00640 1 -- 172.21.15.119:0/2200220021 wait complete. 2024-04-25T10:11:19.286 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:11:19.287 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:19.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.484+0000 7f0906200640 1 Processor -- start 2024-04-25T10:11:19.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.485+0000 7f0906200640 1 -- start start 2024-04-25T10:11:19.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.486+0000 7f0906200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09000d3f80 0x7f09000d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.486+0000 7f0906200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090015ef10 con 0x7f090015db60 2024-04-25T10:11:19.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.486+0000 7f0906200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090015e390 con 0x7f09000b9270 2024-04-25T10:11:19.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.486+0000 7f0906200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090015e670 con 0x7f09000d3f80 2024-04-25T10:11:19.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.487+0000 7f08f7e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09000d3f80 0x7f09000d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.487+0000 7f08f7e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09000d3f80 0x7f09000d1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44278/0 (socket says 172.21.15.119:44278) 2024-04-25T10:11:19.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.487+0000 7f0904e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f090015db60 0x7f09000d3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54958/0 (socket says 172.21.15.119:54958) 2024-04-25T10:11:19.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.487+0000 7f08f7e00640 1 -- 172.21.15.119:0/1259323631 learned_addr learned my addr 172.21.15.119:0/1259323631 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:19.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.487+0000 7f08f7e00640 1 -- 172.21.15.119:0/1259323631 >> v1:172.21.15.132:6789/0 conn(0x7f09000b9270 legacy=0x7f09000d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.487+0000 7f08f7e00640 1 -- 172.21.15.119:0/1259323631 >> v1:172.21.15.119:6789/0 conn(0x7f090015db60 legacy=0x7f09000d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.487+0000 7f08f7e00640 1 -- 172.21.15.119:0/1259323631 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090015e950 con 0x7f09000d3f80 2024-04-25T10:11:19.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.488+0000 7f08f7e00640 1 --2- 172.21.15.119:0/1259323631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09000d3f80 0x7f09000d1f90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009850 tx=0x7f08e802a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286e23298bd51344 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.489+0000 7f08ffe00640 1 -- 172.21.15.119:0/1259323631 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8004230 con 0x7f09000d3f80 2024-04-25T10:11:19.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.489+0000 7f08ffe00640 1 -- 172.21.15.119:0/1259323631 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e80043d0 con 0x7f09000d3f80 2024-04-25T10:11:19.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.489+0000 7f08ffe00640 1 -- 172.21.15.119:0/1259323631 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8003810 con 0x7f09000d3f80 2024-04-25T10:11:19.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.489+0000 7f0906200640 1 -- 172.21.15.119:0/1259323631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09000d3f80 msgr2=0x7f09000d1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.489+0000 7f0906200640 1 --2- 172.21.15.119:0/1259323631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09000d3f80 0x7f09000d1f90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009850 tx=0x7f08e802a2e0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.489+0000 7f0906200640 1 -- 172.21.15.119:0/1259323631 shutdown_connections 2024-04-25T10:11:19.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.489+0000 7f0906200640 1 --2- 172.21.15.119:0/1259323631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09000d3f80 0x7f09000d1f90 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.489+0000 7f0906200640 1 -- 172.21.15.119:0/1259323631 >> 172.21.15.119:0/1259323631 conn(0x7f09000726c0 msgr2=0x7f0900072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.490+0000 7f0906200640 1 -- 172.21.15.119:0/1259323631 shutdown_connections 2024-04-25T10:11:19.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.490+0000 7f0906200640 1 -- 172.21.15.119:0/1259323631 wait complete. 2024-04-25T10:11:19.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.491+0000 7f0906200640 1 Processor -- start 2024-04-25T10:11:19.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.491+0000 7f0906200640 1 -- start start 2024-04-25T10:11:19.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.492+0000 7f0906200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09000b9270 0x7f09000cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.492+0000 7f0906200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09000d3f80 0x7f09001f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.492+0000 7f0906200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f090015db60 0x7f09001f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.492+0000 7f0906200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09000b3120 con 0x7f09000d3f80 2024-04-25T10:11:19.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.492+0000 7f0906200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09000b2fa0 con 0x7f090015db60 2024-04-25T10:11:19.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.492+0000 7f0906200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090015df40 con 0x7f09000b9270 2024-04-25T10:11:19.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.493+0000 7f08f7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09000d3f80 0x7f09001f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.493+0000 7f08f7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09000d3f80 0x7f09001f7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44282/0 (socket says 172.21.15.119:44282) 2024-04-25T10:11:19.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.493+0000 7f08f7e00640 1 -- 172.21.15.119:0/368395506 learned_addr learned my addr 172.21.15.119:0/368395506 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:19.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.493+0000 7f0904e00640 1 --2- 172.21.15.119:0/368395506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09000b9270 0x7f09000cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.493+0000 7f08f7e00640 1 -- 172.21.15.119:0/368395506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09000b9270 msgr2=0x7f09000cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.493+0000 7f08f7e00640 1 --2- 172.21.15.119:0/368395506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09000b9270 0x7f09000cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.493+0000 7f0905800640 1 --2- 172.21.15.119:0/368395506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f090015db60 0x7f09001f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.493+0000 7f08f7e00640 1 -- 172.21.15.119:0/368395506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f090015db60 msgr2=0x7f09001f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.493+0000 7f08f7e00640 1 --2- 172.21.15.119:0/368395506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f090015db60 0x7f09001f75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.493+0000 7f08f7e00640 1 -- 172.21.15.119:0/368395506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09000b4e40 con 0x7f09000d3f80 2024-04-25T10:11:19.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.494+0000 7f08f7e00640 1 --2- 172.21.15.119:0/368395506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09000d3f80 0x7f09001f7090 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f08e8034040 tx=0x7f08e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.494+0000 7f08fd600640 1 -- 172.21.15.119:0/368395506 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8041030 con 0x7f09000d3f80 2024-04-25T10:11:19.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.494+0000 7f08fd600640 1 -- 172.21.15.119:0/368395506 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e8003c50 con 0x7f09000d3f80 2024-04-25T10:11:19.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.494+0000 7f08fd600640 1 -- 172.21.15.119:0/368395506 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8003e20 con 0x7f09000d3f80 2024-04-25T10:11:19.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.494+0000 7f0906200640 1 -- 172.21.15.119:0/368395506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09000b5120 con 0x7f09000d3f80 2024-04-25T10:11:19.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.494+0000 7f0906200640 1 -- 172.21.15.119:0/368395506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09000b56e0 con 0x7f09000d3f80 2024-04-25T10:11:19.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.495+0000 7f0906200640 1 -- 172.21.15.119:0/368395506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09000749b0 con 0x7f09000d3f80 2024-04-25T10:11:19.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.496+0000 7f08fd600640 1 -- 172.21.15.119:0/368395506 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08e8044070 con 0x7f09000d3f80 2024-04-25T10:11:19.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.497+0000 7f08fd600640 1 --2- 172.21.15.119:0/368395506 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f08dc06b4d0 0x7f08dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.497+0000 7f08fd600640 1 -- 172.21.15.119:0/368395506 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08e80aae40 con 0x7f09000d3f80 2024-04-25T10:11:19.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.497+0000 7f0904e00640 1 --2- 172.21.15.119:0/368395506 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f08dc06b4d0 0x7f08dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.498+0000 7f0904e00640 1 --2- 172.21.15.119:0/368395506 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f08dc06b4d0 0x7f08dc06d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f08f0000f60 tx=0x7f08f0007c80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.501+0000 7f08fd600640 1 -- 172.21.15.119:0/368395506 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f08e8079d40 con 0x7f09000d3f80 2024-04-25T10:11:19.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.734+0000 7f0906200640 1 -- 172.21.15.119:0/368395506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09000d1a60 con 0x7f09000d3f80 2024-04-25T10:11:19.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.735+0000 7f08fd600640 1 -- 172.21.15.119:0/368395506 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f08e8079640 con 0x7f09000d3f80 2024-04-25T10:11:19.741 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:19.741 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4159222235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4159222235},{"type":"v1","addr":"172.21.15.132:6834","nonce":4159222235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4223361610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":4223361610},{"type":"v1","addr":"172.21.15.119:6837","nonce":4223361610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2367324442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2367324442},{"type":"v1","addr":"172.21.15.132:6835","nonce":2367324442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:59.454941+0000","modified":"2024-04-25T10:10:48.089168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6836/4129112885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4129112885},{"type":"v1","addr":"172.21.15.119:6836","nonce":4129112885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:11:19.741 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 36 2024-04-25T10:11:19.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.738+0000 7f0906200640 1 -- 172.21.15.119:0/368395506 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f08dc06b4d0 msgr2=0x7f08dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.738+0000 7f0906200640 1 --2- 172.21.15.119:0/368395506 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f08dc06b4d0 0x7f08dc06d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f08f0000f60 tx=0x7f08f0007c80 comp rx=0 tx=0).stop 2024-04-25T10:11:19.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.738+0000 7f0906200640 1 -- 172.21.15.119:0/368395506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09000d3f80 msgr2=0x7f09001f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.738+0000 7f0906200640 1 --2- 172.21.15.119:0/368395506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09000d3f80 0x7f09001f7090 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f08e8034040 tx=0x7f08e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.738+0000 7f0906200640 1 -- 172.21.15.119:0/368395506 shutdown_connections 2024-04-25T10:11:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.738+0000 7f0906200640 1 --2- 172.21.15.119:0/368395506 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f08dc06b4d0 0x7f08dc06d990 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.738+0000 7f0906200640 1 --2- 172.21.15.119:0/368395506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f090015db60 0x7f09001f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.738+0000 7f0906200640 1 --2- 172.21.15.119:0/368395506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09000d3f80 0x7f09001f7090 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.738+0000 7f0906200640 1 --2- 172.21.15.119:0/368395506 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09000b9270 0x7f09000cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.738+0000 7f0906200640 1 -- 172.21.15.119:0/368395506 >> 172.21.15.119:0/368395506 conn(0x7f09000726c0 msgr2=0x7f09000c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.738+0000 7f0906200640 1 -- 172.21.15.119:0/368395506 shutdown_connections 2024-04-25T10:11:19.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.738+0000 7f0906200640 1 -- 172.21.15.119:0/368395506 wait complete. 2024-04-25T10:11:19.763 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:19.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.960+0000 7f1f47200640 1 Processor -- start 2024-04-25T10:11:19.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.960+0000 7f1f47200640 1 -- start start 2024-04-25T10:11:19.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.962+0000 7f1f47200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f48147270 0x7f1f480b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.962+0000 7f1f47200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1f4815dee0 0x7f1f4815e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.962+0000 7f1f47200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1f4815e860 0x7f1f481614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.962+0000 7f1f47200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f480af000 con 0x7f1f4815dee0 2024-04-25T10:11:19.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.962+0000 7f1f47200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f480b4a80 con 0x7f1f4815e860 2024-04-25T10:11:19.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.962+0000 7f1f47200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f480b4d60 con 0x7f1f48147270 2024-04-25T10:11:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.962+0000 7f1f45400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1f4815dee0 0x7f1f4815e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.962+0000 7f1f45400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1f4815dee0 0x7f1f4815e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44288/0 (socket says 172.21.15.119:44288) 2024-04-25T10:11:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.962+0000 7f1f45400640 1 -- 172.21.15.119:0/1494101375 learned_addr learned my addr 172.21.15.119:0/1494101375 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.962+0000 7f1f46800640 1 --2- 172.21.15.119:0/1494101375 >> v2:172.21.15.132:3300/0 conn(0x7f1f4815e860 0x7f1f481614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.962+0000 7f1f45e00640 1 --2- 172.21.15.119:0/1494101375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f48147270 0x7f1f480b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.963+0000 7f1f45400640 1 -- 172.21.15.119:0/1494101375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f48147270 msgr2=0x7f1f480b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.963+0000 7f1f45400640 1 --2- 172.21.15.119:0/1494101375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f48147270 0x7f1f480b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.963+0000 7f1f45400640 1 -- 172.21.15.119:0/1494101375 >> v2:172.21.15.132:3300/0 conn(0x7f1f4815e860 msgr2=0x7f1f481614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.963+0000 7f1f45400640 1 --2- 172.21.15.119:0/1494101375 >> v2:172.21.15.132:3300/0 conn(0x7f1f4815e860 0x7f1f481614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.963+0000 7f1f45400640 1 -- 172.21.15.119:0/1494101375 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f480b5040 con 0x7f1f4815dee0 2024-04-25T10:11:19.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.964+0000 7f1f45400640 1 --2- 172.21.15.119:0/1494101375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f4815dee0 0x7f1f4815e320 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1f4815f270 tx=0x7f1f3402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f31f32db3e24df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.964+0000 7f1f44a00640 1 -- 172.21.15.119:0/1494101375 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34037070 con 0x7f1f4815dee0 2024-04-25T10:11:19.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.964+0000 7f1f44a00640 1 -- 172.21.15.119:0/1494101375 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f34032440 con 0x7f1f4815dee0 2024-04-25T10:11:19.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.964+0000 7f1f44a00640 1 -- 172.21.15.119:0/1494101375 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34032810 con 0x7f1f4815dee0 2024-04-25T10:11:19.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.965+0000 7f1f47200640 1 -- 172.21.15.119:0/1494101375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f4815dee0 msgr2=0x7f1f4815e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.965+0000 7f1f47200640 1 --2- 172.21.15.119:0/1494101375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f4815dee0 0x7f1f4815e320 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1f4815f270 tx=0x7f1f3402a140 comp rx=0 tx=0).stop 2024-04-25T10:11:19.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.965+0000 7f1f47200640 1 -- 172.21.15.119:0/1494101375 shutdown_connections 2024-04-25T10:11:19.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.965+0000 7f1f47200640 1 --2- 172.21.15.119:0/1494101375 >> v2:172.21.15.132:3300/0 conn(0x7f1f4815e860 0x7f1f481614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.965+0000 7f1f47200640 1 --2- 172.21.15.119:0/1494101375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f4815dee0 0x7f1f4815e320 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.965+0000 7f1f47200640 1 --2- 172.21.15.119:0/1494101375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f48147270 0x7f1f480b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.965+0000 7f1f47200640 1 -- 172.21.15.119:0/1494101375 >> 172.21.15.119:0/1494101375 conn(0x7f1f4806b1d0 msgr2=0x7f1f480b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.965+0000 7f1f47200640 1 -- 172.21.15.119:0/1494101375 shutdown_connections 2024-04-25T10:11:19.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.965+0000 7f1f47200640 1 -- 172.21.15.119:0/1494101375 wait complete. 2024-04-25T10:11:19.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.967+0000 7f1f47200640 1 Processor -- start 2024-04-25T10:11:19.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.967+0000 7f1f47200640 1 -- start start 2024-04-25T10:11:19.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f47200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f48147270 0x7f1f48134170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f47200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f4815dee0 0x7f1f4815f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f47200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4815e860 0x7f1f48137080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f47200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4815ecf0 con 0x7f1f4815dee0 2024-04-25T10:11:19.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f47200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f480b5600 con 0x7f1f48147270 2024-04-25T10:11:19.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f47200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f481619e0 con 0x7f1f4815e860 2024-04-25T10:11:19.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f45400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f4815dee0 0x7f1f4815f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f45400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f4815dee0 0x7f1f4815f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44290/0 (socket says 172.21.15.119:44290) 2024-04-25T10:11:19.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f45400640 1 -- 172.21.15.119:0/2299719954 learned_addr learned my addr 172.21.15.119:0/2299719954 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:19.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f46800640 1 --2- 172.21.15.119:0/2299719954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4815e860 0x7f1f48137080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f45e00640 1 --2- 172.21.15.119:0/2299719954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f48147270 0x7f1f48134170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f45400640 1 -- 172.21.15.119:0/2299719954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4815e860 msgr2=0x7f1f48137080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f45400640 1 --2- 172.21.15.119:0/2299719954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4815e860 0x7f1f48137080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f45400640 1 -- 172.21.15.119:0/2299719954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f48147270 msgr2=0x7f1f48134170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f45400640 1 --2- 172.21.15.119:0/2299719954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f48147270 0x7f1f48134170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.968+0000 7f1f45400640 1 -- 172.21.15.119:0/2299719954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f481346b0 con 0x7f1f4815dee0 2024-04-25T10:11:19.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.969+0000 7f1f45400640 1 --2- 172.21.15.119:0/2299719954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f4815dee0 0x7f1f4815f8a0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1f3402a180 tx=0x7f1f340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.969+0000 7f1f3e000640 1 -- 172.21.15.119:0/2299719954 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34041020 con 0x7f1f4815dee0 2024-04-25T10:11:19.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.969+0000 7f1f3e000640 1 -- 172.21.15.119:0/2299719954 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f340306c0 con 0x7f1f4815dee0 2024-04-25T10:11:19.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.969+0000 7f1f3e000640 1 -- 172.21.15.119:0/2299719954 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34037040 con 0x7f1f4815dee0 2024-04-25T10:11:19.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.969+0000 7f1f47200640 1 -- 172.21.15.119:0/2299719954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f480c4a60 con 0x7f1f4815dee0 2024-04-25T10:11:19.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.969+0000 7f1f47200640 1 -- 172.21.15.119:0/2299719954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f480c5300 con 0x7f1f4815dee0 2024-04-25T10:11:19.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.970+0000 7f1f47200640 1 -- 172.21.15.119:0/2299719954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f480749b0 con 0x7f1f4815dee0 2024-04-25T10:11:19.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.971+0000 7f1f3e000640 1 -- 172.21.15.119:0/2299719954 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f34044020 con 0x7f1f4815dee0 2024-04-25T10:11:19.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.972+0000 7f1f3e000640 1 --2- 172.21.15.119:0/2299719954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f2806b4d0 0x7f1f2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.972+0000 7f1f3e000640 1 -- 172.21.15.119:0/2299719954 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f340aaab0 con 0x7f1f4815dee0 2024-04-25T10:11:19.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.972+0000 7f1f45e00640 1 --2- 172.21.15.119:0/2299719954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f2806b4d0 0x7f1f2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.973+0000 7f1f45e00640 1 --2- 172.21.15.119:0/2299719954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f2806b4d0 0x7f1f2806d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1f38006fd0 tx=0x7f1f38008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:19.977+0000 7f1f3e000640 1 -- 172.21.15.119:0/2299719954 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f3404b030 con 0x7f1f4815dee0 2024-04-25T10:11:20.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.217+0000 7f1f47200640 1 -- 172.21.15.119:0/2299719954 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f480b2180 con 0x7f1f4815dee0 2024-04-25T10:11:20.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.219+0000 7f1f3e000640 1 -- 172.21.15.119:0/2299719954 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f1f481346b0 con 0x7f1f4815dee0 2024-04-25T10:11:20.225 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:20.225 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4159222235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4159222235},{"type":"v1","addr":"172.21.15.132:6834","nonce":4159222235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4223361610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":4223361610},{"type":"v1","addr":"172.21.15.119:6837","nonce":4223361610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2367324442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2367324442},{"type":"v1","addr":"172.21.15.132:6835","nonce":2367324442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:09:59.454941+0000","modified":"2024-04-25T10:10:48.089168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6836/4129112885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4129112885},{"type":"v1","addr":"172.21.15.119:6836","nonce":4129112885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:11:20.225 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 36 2024-04-25T10:11:20.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.221+0000 7f1f47200640 1 -- 172.21.15.119:0/2299719954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f2806b4d0 msgr2=0x7f1f2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.221+0000 7f1f47200640 1 --2- 172.21.15.119:0/2299719954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f2806b4d0 0x7f1f2806d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1f38006fd0 tx=0x7f1f38008040 comp rx=0 tx=0).stop 2024-04-25T10:11:20.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.221+0000 7f1f47200640 1 -- 172.21.15.119:0/2299719954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f4815dee0 msgr2=0x7f1f4815f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.221+0000 7f1f47200640 1 --2- 172.21.15.119:0/2299719954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f4815dee0 0x7f1f4815f8a0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1f3402a180 tx=0x7f1f340046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.221+0000 7f1f47200640 1 -- 172.21.15.119:0/2299719954 shutdown_connections 2024-04-25T10:11:20.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.221+0000 7f1f47200640 1 --2- 172.21.15.119:0/2299719954 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f2806b4d0 0x7f1f2806d990 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.221+0000 7f1f47200640 1 --2- 172.21.15.119:0/2299719954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4815e860 0x7f1f48137080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.221+0000 7f1f47200640 1 --2- 172.21.15.119:0/2299719954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f4815dee0 0x7f1f4815f8a0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.221+0000 7f1f47200640 1 --2- 172.21.15.119:0/2299719954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f48147270 0x7f1f48134170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.221+0000 7f1f47200640 1 -- 172.21.15.119:0/2299719954 >> 172.21.15.119:0/2299719954 conn(0x7f1f4806b1d0 msgr2=0x7f1f480ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.222+0000 7f1f47200640 1 -- 172.21.15.119:0/2299719954 shutdown_connections 2024-04-25T10:11:20.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.222+0000 7f1f47200640 1 -- 172.21.15.119:0/2299719954 wait complete. 2024-04-25T10:11:20.247 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:20.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.444+0000 7f2211a00640 1 Processor -- start 2024-04-25T10:11:20.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.444+0000 7f2211a00640 1 -- start start 2024-04-25T10:11:20.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.445+0000 7f2211a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f220c15a870 0x7f220c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.445+0000 7f2211a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220c0cfc40 0x7f220c15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.445+0000 7f2211a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220c0cdb10 con 0x7f220c06e2e0 2024-04-25T10:11:20.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.445+0000 7f2211a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220c0ce720 con 0x7f220c15a870 2024-04-25T10:11:20.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.445+0000 7f2211a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220c0cddf0 con 0x7f220c0cfc40 2024-04-25T10:11:20.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.446+0000 7f220b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220c0cfc40 0x7f220c15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.446+0000 7f220b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220c0cfc40 0x7f220c15ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:36024/0 (socket says 172.21.15.119:36024) 2024-04-25T10:11:20.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.446+0000 7f220b400640 1 -- 172.21.15.119:0/722560302 learned_addr learned my addr 172.21.15.119:0/722560302 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:20.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.446+0000 7f220aa00640 1 --2- 172.21.15.119:0/722560302 >> v2:172.21.15.132:3300/0 conn(0x7f220c15a870 0x7f220c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.447+0000 7f220b400640 1 -- 172.21.15.119:0/722560302 >> v2:172.21.15.132:3300/0 conn(0x7f220c15a870 msgr2=0x7f220c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.447+0000 7f220b400640 1 --2- 172.21.15.119:0/722560302 >> v2:172.21.15.132:3300/0 conn(0x7f220c15a870 0x7f220c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.447+0000 7f220b400640 1 -- 172.21.15.119:0/722560302 >> v1:172.21.15.132:6789/0 conn(0x7f220c06e2e0 legacy=0x7f220c0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.447+0000 7f220b400640 1 -- 172.21.15.119:0/722560302 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c0ce0d0 con 0x7f220c0cfc40 2024-04-25T10:11:20.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.448+0000 7f220b400640 1 --2- 172.21.15.119:0/722560302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220c0cfc40 0x7f220c15ac50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2204009e50 tx=0x7f220402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1ff7ab0afa592e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.449+0000 7f220a000640 1 -- 172.21.15.119:0/722560302 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204036070 con 0x7f220c0cfc40 2024-04-25T10:11:20.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.449+0000 7f220a000640 1 -- 172.21.15.119:0/722560302 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2204031600 con 0x7f220c0cfc40 2024-04-25T10:11:20.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.449+0000 7f220a000640 1 -- 172.21.15.119:0/722560302 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22040319d0 con 0x7f220c0cfc40 2024-04-25T10:11:20.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.450+0000 7f2211a00640 1 -- 172.21.15.119:0/722560302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220c0cfc40 msgr2=0x7f220c15ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.450+0000 7f2211a00640 1 --2- 172.21.15.119:0/722560302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220c0cfc40 0x7f220c15ac50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2204009e50 tx=0x7f220402a710 comp rx=0 tx=0).stop 2024-04-25T10:11:20.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.450+0000 7f2211a00640 1 -- 172.21.15.119:0/722560302 shutdown_connections 2024-04-25T10:11:20.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.450+0000 7f2211a00640 1 --2- 172.21.15.119:0/722560302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220c0cfc40 0x7f220c15ac50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.450+0000 7f2211a00640 1 --2- 172.21.15.119:0/722560302 >> v2:172.21.15.132:3300/0 conn(0x7f220c15a870 0x7f220c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.450+0000 7f2211a00640 1 -- 172.21.15.119:0/722560302 >> 172.21.15.119:0/722560302 conn(0x7f220c06aba0 msgr2=0x7f220c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.450+0000 7f2211a00640 1 -- 172.21.15.119:0/722560302 shutdown_connections 2024-04-25T10:11:20.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.450+0000 7f2211a00640 1 -- 172.21.15.119:0/722560302 wait complete. 2024-04-25T10:11:20.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.452+0000 7f2211a00640 1 Processor -- start 2024-04-25T10:11:20.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.452+0000 7f2211a00640 1 -- start start 2024-04-25T10:11:20.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.453+0000 7f2211a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f220c06e2e0 0x7f220c1443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.453+0000 7f2211a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220c0cfc40 0x7f220c1e3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.453+0000 7f2211a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f220c15a870 0x7f220c144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.453+0000 7f2211a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c0cd700 con 0x7f220c15a870 2024-04-25T10:11:20.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.453+0000 7f2211a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c0cd580 con 0x7f220c06e2e0 2024-04-25T10:11:20.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.453+0000 7f2211a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220c0cd880 con 0x7f220c0cfc40 2024-04-25T10:11:20.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.453+0000 7f220b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f220c15a870 0x7f220c144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.453+0000 7f220b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f220c15a870 0x7f220c144d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44296/0 (socket says 172.21.15.119:44296) 2024-04-25T10:11:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.453+0000 7f220b400640 1 -- 172.21.15.119:0/4241669594 learned_addr learned my addr 172.21.15.119:0/4241669594 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.454+0000 7f2203e00640 1 --2- 172.21.15.119:0/4241669594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220c0cfc40 0x7f220c1e3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.454+0000 7f220aa00640 1 --2- 172.21.15.119:0/4241669594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f220c06e2e0 0x7f220c1443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.454+0000 7f220b400640 1 -- 172.21.15.119:0/4241669594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220c0cfc40 msgr2=0x7f220c1e3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.454+0000 7f220b400640 1 --2- 172.21.15.119:0/4241669594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220c0cfc40 0x7f220c1e3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.454+0000 7f220b400640 1 -- 172.21.15.119:0/4241669594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f220c06e2e0 msgr2=0x7f220c1443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.454+0000 7f220b400640 1 --2- 172.21.15.119:0/4241669594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f220c06e2e0 0x7f220c1443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.454+0000 7f220b400640 1 -- 172.21.15.119:0/4241669594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c1e4820 con 0x7f220c15a870 2024-04-25T10:11:20.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.454+0000 7f220b400640 1 --2- 172.21.15.119:0/4241669594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f220c15a870 0x7f220c144d30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2204009f70 tx=0x7f2204004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.454+0000 7f2202a00640 1 -- 172.21.15.119:0/4241669594 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204041020 con 0x7f220c15a870 2024-04-25T10:11:20.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.455+0000 7f2202a00640 1 -- 172.21.15.119:0/4241669594 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2204008730 con 0x7f220c15a870 2024-04-25T10:11:20.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.455+0000 7f2202a00640 1 -- 172.21.15.119:0/4241669594 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204036040 con 0x7f220c15a870 2024-04-25T10:11:20.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.455+0000 7f2211a00640 1 -- 172.21.15.119:0/4241669594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f220c1e4b00 con 0x7f220c15a870 2024-04-25T10:11:20.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.455+0000 7f2211a00640 1 -- 172.21.15.119:0/4241669594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220c1e50c0 con 0x7f220c15a870 2024-04-25T10:11:20.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.456+0000 7f2211a00640 1 -- 172.21.15.119:0/4241669594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220c0cf1d0 con 0x7f220c15a870 2024-04-25T10:11:20.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.457+0000 7f2202a00640 1 -- 172.21.15.119:0/4241669594 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2204044020 con 0x7f220c15a870 2024-04-25T10:11:20.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.458+0000 7f2202a00640 1 --2- 172.21.15.119:0/4241669594 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f21e806b4d0 0x7f21e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.458+0000 7f2202a00640 1 -- 172.21.15.119:0/4241669594 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22040aae70 con 0x7f220c15a870 2024-04-25T10:11:20.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.458+0000 7f220aa00640 1 --2- 172.21.15.119:0/4241669594 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f21e806b4d0 0x7f21e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.459+0000 7f220aa00640 1 --2- 172.21.15.119:0/4241669594 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f21e806b4d0 0x7f21e806d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009ad0 tx=0x7f21fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.463+0000 7f2202a00640 1 -- 172.21.15.119:0/4241669594 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f220c0cf1d0 con 0x7f220c15a870 2024-04-25T10:11:20.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.655+0000 7f2211a00640 1 -- 172.21.15.119:0/4241669594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f220c0749b0 con 0x7f220c15a870 2024-04-25T10:11:20.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.658+0000 7f2202a00640 1 -- 172.21.15.119:0/4241669594 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16701 (secure 0 0 0) 0x7f220404b030 con 0x7f220c15a870 2024-04-25T10:11:20.664 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:20.664 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":55,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:10:01.035989+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:09:57.237546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:09:58.475727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:20.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.661+0000 7f2211a00640 1 -- 172.21.15.119:0/4241669594 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f21e806b4d0 msgr2=0x7f21e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.661+0000 7f2211a00640 1 --2- 172.21.15.119:0/4241669594 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f21e806b4d0 0x7f21e806d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009ad0 tx=0x7f21fc008040 comp rx=0 tx=0).stop 2024-04-25T10:11:20.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.661+0000 7f2211a00640 1 -- 172.21.15.119:0/4241669594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f220c15a870 msgr2=0x7f220c144d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.661+0000 7f2211a00640 1 --2- 172.21.15.119:0/4241669594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f220c15a870 0x7f220c144d30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2204009f70 tx=0x7f2204004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:20.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.661+0000 7f2211a00640 1 -- 172.21.15.119:0/4241669594 shutdown_connections 2024-04-25T10:11:20.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.661+0000 7f2211a00640 1 --2- 172.21.15.119:0/4241669594 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f21e806b4d0 0x7f21e806d990 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.661+0000 7f2211a00640 1 --2- 172.21.15.119:0/4241669594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f220c15a870 0x7f220c144d30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.661+0000 7f2211a00640 1 --2- 172.21.15.119:0/4241669594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220c0cfc40 0x7f220c1e3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.661+0000 7f2211a00640 1 --2- 172.21.15.119:0/4241669594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f220c06e2e0 0x7f220c1443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.661+0000 7f2211a00640 1 -- 172.21.15.119:0/4241669594 >> 172.21.15.119:0/4241669594 conn(0x7f220c06aba0 msgr2=0x7f220c0cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.661+0000 7f2211a00640 1 -- 172.21.15.119:0/4241669594 shutdown_connections 2024-04-25T10:11:20.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.662+0000 7f2211a00640 1 -- 172.21.15.119:0/4241669594 wait complete. 2024-04-25T10:11:20.686 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:11:20.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.882+0000 7f29ba000640 1 Processor -- start 2024-04-25T10:11:20.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.882+0000 7f29ba000640 1 -- start start 2024-04-25T10:11:20.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.884+0000 7f29ba000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f29b40aedc0 0x7f29b415ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.884+0000 7f29ba000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29b406d260 0x7f29b40b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.884+0000 7f29ba000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b406da40 con 0x7f29b40b4730 2024-04-25T10:11:20.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.884+0000 7f29ba000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b40b4ba0 con 0x7f29b40aedc0 2024-04-25T10:11:20.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.884+0000 7f29ba000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b415c050 con 0x7f29b406d260 2024-04-25T10:11:20.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.884+0000 7f29b3e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f29b40b4730 0x7f29b40b26f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39150/0 (socket says 172.21.15.119:39150) 2024-04-25T10:11:20.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.884+0000 7f29b3e00640 1 -- 172.21.15.119:0/1744661694 learned_addr learned my addr 172.21.15.119:0/1744661694 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:20.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.884+0000 7f29b3400640 1 --2- 172.21.15.119:0/1744661694 >> v2:172.21.15.119:3300/0 conn(0x7f29b40aedc0 0x7f29b415ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.884+0000 7f29b2a00640 1 --2- 172.21.15.119:0/1744661694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29b406d260 0x7f29b40b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.884+0000 7f29b2000640 1 -- 172.21.15.119:0/1744661694 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116111224 0 0) 0x7f29b406da40 con 0x7f29b40b4730 2024-04-25T10:11:20.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.885+0000 7f29b2000640 1 -- 172.21.15.119:0/1744661694 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2998003520 con 0x7f29b40b4730 2024-04-25T10:11:20.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.885+0000 7f29b3400640 1 -- 172.21.15.119:0/1744661694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29b406d260 msgr2=0x7f29b40b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.885+0000 7f29b3400640 1 --2- 172.21.15.119:0/1744661694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29b406d260 0x7f29b40b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.885+0000 7f29b3400640 1 -- 172.21.15.119:0/1744661694 >> v1:172.21.15.119:6789/0 conn(0x7f29b40b4730 legacy=0x7f29b40b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.885+0000 7f29b3400640 1 -- 172.21.15.119:0/1744661694 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b406dd20 con 0x7f29b40aedc0 2024-04-25T10:11:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.886+0000 7f29b3400640 1 --2- 172.21.15.119:0/1744661694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29b40aedc0 0x7f29b415ae20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f29a40097a0 tx=0x7f29a402a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86aa7a4a29dd8d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.886+0000 7f29b2000640 1 -- 172.21.15.119:0/1744661694 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4037070 con 0x7f29b40aedc0 2024-04-25T10:11:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.886+0000 7f29b2000640 1 -- 172.21.15.119:0/1744661694 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a402ae40 con 0x7f29b40aedc0 2024-04-25T10:11:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.886+0000 7f29b2000640 1 -- 172.21.15.119:0/1744661694 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4032660 con 0x7f29b40aedc0 2024-04-25T10:11:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.887+0000 7f29ba000640 1 -- 172.21.15.119:0/1744661694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29b40aedc0 msgr2=0x7f29b415ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.887+0000 7f29ba000640 1 --2- 172.21.15.119:0/1744661694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29b40aedc0 0x7f29b415ae20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f29a40097a0 tx=0x7f29a402a0f0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.887+0000 7f29ba000640 1 -- 172.21.15.119:0/1744661694 shutdown_connections 2024-04-25T10:11:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.887+0000 7f29ba000640 1 --2- 172.21.15.119:0/1744661694 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29b406d260 0x7f29b40b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.887+0000 7f29ba000640 1 --2- 172.21.15.119:0/1744661694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29b40aedc0 0x7f29b415ae20 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.887+0000 7f29ba000640 1 -- 172.21.15.119:0/1744661694 >> 172.21.15.119:0/1744661694 conn(0x7f29b406aba0 msgr2=0x7f29b406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.887+0000 7f29ba000640 1 -- 172.21.15.119:0/1744661694 shutdown_connections 2024-04-25T10:11:20.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.887+0000 7f29ba000640 1 -- 172.21.15.119:0/1744661694 wait complete. 2024-04-25T10:11:20.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.889+0000 7f29ba000640 1 Processor -- start 2024-04-25T10:11:20.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.889+0000 7f29ba000640 1 -- start start 2024-04-25T10:11:20.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.890+0000 7f29ba000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29b406d260 0x7f29b40c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.890+0000 7f29ba000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29b40aedc0 0x7f29b41548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.890+0000 7f29ba000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29b40b4730 0x7f29b41f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.890+0000 7f29ba000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b415c2a0 con 0x7f29b40b4730 2024-04-25T10:11:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.890+0000 7f29ba000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b40af1a0 con 0x7f29b40aedc0 2024-04-25T10:11:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.890+0000 7f29ba000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b406d6c0 con 0x7f29b406d260 2024-04-25T10:11:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.890+0000 7f29b3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29b40b4730 0x7f29b41f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.890+0000 7f29b3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29b40b4730 0x7f29b41f3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59186/0 (socket says 172.21.15.119:59186) 2024-04-25T10:11:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.890+0000 7f29b3e00640 1 -- 172.21.15.119:0/3946490744 learned_addr learned my addr 172.21.15.119:0/3946490744 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.890+0000 7f29b3400640 1 --2- 172.21.15.119:0/3946490744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29b406d260 0x7f29b40c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.890+0000 7f29b2a00640 1 --2- 172.21.15.119:0/3946490744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29b40aedc0 0x7f29b41548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.891+0000 7f29b3e00640 1 -- 172.21.15.119:0/3946490744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29b406d260 msgr2=0x7f29b40c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.891+0000 7f29b3e00640 1 --2- 172.21.15.119:0/3946490744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29b406d260 0x7f29b40c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.891+0000 7f29b3e00640 1 -- 172.21.15.119:0/3946490744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29b40aedc0 msgr2=0x7f29b41548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.891+0000 7f29b3e00640 1 --2- 172.21.15.119:0/3946490744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29b40aedc0 0x7f29b41548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.891+0000 7f29b3e00640 1 -- 172.21.15.119:0/3946490744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b40c4d40 con 0x7f29b40b4730 2024-04-25T10:11:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.891+0000 7f29b3e00640 1 --2- 172.21.15.119:0/3946490744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29b40b4730 0x7f29b41f3420 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00caa0 tx=0x7f29ac00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.891+0000 7f29ab400640 1 -- 172.21.15.119:0/3946490744 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac017070 con 0x7f29b40b4730 2024-04-25T10:11:20.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.891+0000 7f29ab400640 1 -- 172.21.15.119:0/3946490744 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac00aba0 con 0x7f29b40b4730 2024-04-25T10:11:20.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.891+0000 7f29ab400640 1 -- 172.21.15.119:0/3946490744 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac012400 con 0x7f29b40b4730 2024-04-25T10:11:20.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.891+0000 7f29ba000640 1 -- 172.21.15.119:0/3946490744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b40ac950 con 0x7f29b40b4730 2024-04-25T10:11:20.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.891+0000 7f29ba000640 1 -- 172.21.15.119:0/3946490744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b40acf10 con 0x7f29b40b4730 2024-04-25T10:11:20.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.892+0000 7f29ba000640 1 -- 172.21.15.119:0/3946490744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b40750b0 con 0x7f29b40b4730 2024-04-25T10:11:20.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.893+0000 7f29ab400640 1 -- 172.21.15.119:0/3946490744 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29ac01a070 con 0x7f29b40b4730 2024-04-25T10:11:20.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.894+0000 7f29ab400640 1 --2- 172.21.15.119:0/3946490744 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f299406b4d0 0x7f299406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.894+0000 7f29ab400640 1 -- 172.21.15.119:0/3946490744 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29ac089f80 con 0x7f29b40b4730 2024-04-25T10:11:20.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.895+0000 7f29b3400640 1 --2- 172.21.15.119:0/3946490744 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f299406b4d0 0x7f299406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.895+0000 7f29b3400640 1 --2- 172.21.15.119:0/3946490744 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f299406b4d0 0x7f299406d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f29a4034000 tx=0x7f29a402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:20.899+0000 7f29ab400640 1 -- 172.21.15.119:0/3946490744 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29ac058df0 con 0x7f29b40b4730 2024-04-25T10:11:21.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.131+0000 7f29ba000640 1 -- 172.21.15.119:0/3946490744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f29b406b190 con 0x7f29b40b4730 2024-04-25T10:11:21.195 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:11:21.132+0000 7f8c64600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:11:21.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.135+0000 7f29ab400640 1 -- 172.21.15.119:0/3946490744 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7f29ac0586f0 con 0x7f29b40b4730 2024-04-25T10:11:21.196 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:11:21.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.138+0000 7f29ba000640 1 -- 172.21.15.119:0/3946490744 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f299406b4d0 msgr2=0x7f299406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.138+0000 7f29ba000640 1 --2- 172.21.15.119:0/3946490744 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f299406b4d0 0x7f299406d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f29a4034000 tx=0x7f29a402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:21.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.138+0000 7f29ba000640 1 -- 172.21.15.119:0/3946490744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29b40b4730 msgr2=0x7f29b41f3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.138+0000 7f29ba000640 1 --2- 172.21.15.119:0/3946490744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29b40b4730 0x7f29b41f3420 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00caa0 tx=0x7f29ac00cf60 comp rx=0 tx=0).stop 2024-04-25T10:11:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.139+0000 7f29ba000640 1 -- 172.21.15.119:0/3946490744 shutdown_connections 2024-04-25T10:11:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.139+0000 7f29ba000640 1 --2- 172.21.15.119:0/3946490744 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f299406b4d0 0x7f299406d990 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.139+0000 7f29ba000640 1 --2- 172.21.15.119:0/3946490744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29b40b4730 0x7f29b41f3420 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.139+0000 7f29ba000640 1 --2- 172.21.15.119:0/3946490744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29b40aedc0 0x7f29b41548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.139+0000 7f29ba000640 1 --2- 172.21.15.119:0/3946490744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29b406d260 0x7f29b40c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.139+0000 7f29ba000640 1 -- 172.21.15.119:0/3946490744 >> 172.21.15.119:0/3946490744 conn(0x7f29b406aba0 msgr2=0x7f29b415c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.139+0000 7f29ba000640 1 -- 172.21.15.119:0/3946490744 shutdown_connections 2024-04-25T10:11:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.140+0000 7f29ba000640 1 -- 172.21.15.119:0/3946490744 wait complete. 2024-04-25T10:11:21.198 INFO:tasks.ceph.mds.a.smithi119.stderr: -7625> 2024-04-25T10:10:04.971+0000 7efdbf600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:21.199 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:11:21.200 INFO:tasks.ceph.mds.a.smithi119.stderr: -4993> 2024-04-25T10:10:09.691+0000 7efdba600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={4880=0-4194304@1} caps={4880=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=4880 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x561f4681db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:11:21.200 INFO:tasks.ceph.mds.a.smithi119.stderr: -3017> 2024-04-25T10:10:41.699+0000 7efdba600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-25T10:10:08.149214+0000 1=1+0) (iversion lock) caps={4880=pAsLsXsFs/-@2},l=4880(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x561f4681db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:11:21.230 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:11:21.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.376+0000 7f0d16000640 1 Processor -- start 2024-04-25T10:11:21.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.376+0000 7f0d16000640 1 -- start start 2024-04-25T10:11:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.377+0000 7f0d16000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d100c6150 0x7f0d100ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.377+0000 7f0d16000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0d1015bcb0 0x7f0d100ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.378+0000 7f0d16000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0d1006da10 0x7f0d1006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.378+0000 7f0d16000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1006e790 con 0x7f0d1015bcb0 2024-04-25T10:11:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.378+0000 7f0d16000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d100712c0 con 0x7f0d1006da10 2024-04-25T10:11:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.378+0000 7f0d16000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100715a0 con 0x7f0d100c6150 2024-04-25T10:11:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.378+0000 7f0d07e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0d1015bcb0 0x7f0d100ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.378+0000 7f0d07e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0d1015bcb0 0x7f0d100ce870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59204/0 (socket says 172.21.15.119:59204) 2024-04-25T10:11:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.378+0000 7f0d07e00640 1 -- 172.21.15.119:0/836053894 learned_addr learned my addr 172.21.15.119:0/836053894 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.378+0000 7f0d14c00640 1 --2- 172.21.15.119:0/836053894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d100c6150 0x7f0d100ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.378+0000 7f0d15600640 1 --2- 172.21.15.119:0/836053894 >> v2:172.21.15.132:3300/0 conn(0x7f0d1006da10 0x7f0d1006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.378+0000 7f0d07e00640 1 -- 172.21.15.119:0/836053894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d100c6150 msgr2=0x7f0d100ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.378+0000 7f0d07e00640 1 --2- 172.21.15.119:0/836053894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d100c6150 0x7f0d100ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.378+0000 7f0d07e00640 1 -- 172.21.15.119:0/836053894 >> v2:172.21.15.132:3300/0 conn(0x7f0d1006da10 msgr2=0x7f0d1006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.378+0000 7f0d07e00640 1 --2- 172.21.15.119:0/836053894 >> v2:172.21.15.132:3300/0 conn(0x7f0d1006da10 0x7f0d1006de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.378+0000 7f0d07e00640 1 -- 172.21.15.119:0/836053894 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d10071880 con 0x7f0d1015bcb0 2024-04-25T10:11:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.379+0000 7f0d07e00640 1 --2- 172.21.15.119:0/836053894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d1015bcb0 0x7f0d100ce870 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0d100b1b10 tx=0x7f0cf802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342989525547a227 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.380+0000 7f0d0fe00640 1 -- 172.21.15.119:0/836053894 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8037070 con 0x7f0d1015bcb0 2024-04-25T10:11:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.380+0000 7f0d0fe00640 1 -- 172.21.15.119:0/836053894 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf8032440 con 0x7f0d1015bcb0 2024-04-25T10:11:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.380+0000 7f0d0fe00640 1 -- 172.21.15.119:0/836053894 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8032810 con 0x7f0d1015bcb0 2024-04-25T10:11:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.380+0000 7f0d16000640 1 -- 172.21.15.119:0/836053894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d1015bcb0 msgr2=0x7f0d100ce870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.380+0000 7f0d16000640 1 --2- 172.21.15.119:0/836053894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d1015bcb0 0x7f0d100ce870 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0d100b1b10 tx=0x7f0cf802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.381+0000 7f0d16000640 1 -- 172.21.15.119:0/836053894 shutdown_connections 2024-04-25T10:11:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.381+0000 7f0d16000640 1 --2- 172.21.15.119:0/836053894 >> v2:172.21.15.132:3300/0 conn(0x7f0d1006da10 0x7f0d1006de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.381+0000 7f0d16000640 1 --2- 172.21.15.119:0/836053894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d1015bcb0 0x7f0d100ce870 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.381+0000 7f0d16000640 1 --2- 172.21.15.119:0/836053894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d100c6150 0x7f0d100ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.381+0000 7f0d16000640 1 -- 172.21.15.119:0/836053894 >> 172.21.15.119:0/836053894 conn(0x7f0d1006aba0 msgr2=0x7f0d1006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.381+0000 7f0d16000640 1 -- 172.21.15.119:0/836053894 shutdown_connections 2024-04-25T10:11:21.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.381+0000 7f0d16000640 1 -- 172.21.15.119:0/836053894 wait complete. 2024-04-25T10:11:21.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.382+0000 7f0d16000640 1 Processor -- start 2024-04-25T10:11:21.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.383+0000 7f0d16000640 1 -- start start 2024-04-25T10:11:21.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.383+0000 7f0d16000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d1006da10 0x7f0d10141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.383+0000 7f0d16000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d100c6150 0x7f0d10159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d16000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d10159840 0x7f0d100b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d16000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1006e3c0 con 0x7f0d10159840 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d16000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100c6530 con 0x7f0d100c6150 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d16000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1006e540 con 0x7f0d1006da10 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d15600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d10159840 0x7f0d100b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d15600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d10159840 0x7f0d100b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59220/0 (socket says 172.21.15.119:59220) 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d15600640 1 -- 172.21.15.119:0/791416639 learned_addr learned my addr 172.21.15.119:0/791416639 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d14c00640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d1006da10 0x7f0d10141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d07e00640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d100c6150 0x7f0d10159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d15600640 1 -- 172.21.15.119:0/791416639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d1006da10 msgr2=0x7f0d10141fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d15600640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d1006da10 0x7f0d10141fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d15600640 1 -- 172.21.15.119:0/791416639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d100c6150 msgr2=0x7f0d10159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d15600640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d100c6150 0x7f0d10159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d15600640 1 -- 172.21.15.119:0/791416639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d10159cf0 con 0x7f0d10159840 2024-04-25T10:11:21.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.384+0000 7f0d07e00640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d100c6150 0x7f0d10159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.385+0000 7f0d15600640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d10159840 0x7f0d100b3f20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0d08007e70 tx=0x7f0d0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.385+0000 7f0d0d600640 1 -- 172.21.15.119:0/791416639 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08014620 con 0x7f0d10159840 2024-04-25T10:11:21.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.385+0000 7f0d0d600640 1 -- 172.21.15.119:0/791416639 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d08019070 con 0x7f0d10159840 2024-04-25T10:11:21.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.385+0000 7f0d16000640 1 -- 172.21.15.119:0/791416639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d100b4be0 con 0x7f0d10159840 2024-04-25T10:11:21.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.385+0000 7f0d0d600640 1 -- 172.21.15.119:0/791416639 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08014a20 con 0x7f0d10159840 2024-04-25T10:11:21.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.385+0000 7f0d16000640 1 -- 172.21.15.119:0/791416639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d10142af0 con 0x7f0d10159840 2024-04-25T10:11:21.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.386+0000 7f0d16000640 1 -- 172.21.15.119:0/791416639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d100749b0 con 0x7f0d10159840 2024-04-25T10:11:21.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.387+0000 7f0d0d600640 1 -- 172.21.15.119:0/791416639 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d08014bc0 con 0x7f0d10159840 2024-04-25T10:11:21.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.388+0000 7f0d0d600640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0cec06b4d0 0x7f0cec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.388+0000 7f0d0d600640 1 -- 172.21.15.119:0/791416639 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0d0808aa80 con 0x7f0d10159840 2024-04-25T10:11:21.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.388+0000 7f0d14c00640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0cec06b4d0 0x7f0cec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.389+0000 7f0d14c00640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0cec06b4d0 0x7f0cec06d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0d00006fd0 tx=0x7f0d00008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.392+0000 7f0d0d600640 1 -- 172.21.15.119:0/791416639 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d08059930 con 0x7f0d10159840 2024-04-25T10:11:21.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:21.628+0000 7f0d16000640 1 -- 172.21.15.119:0/791416639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0d100ce8a0 con 0x7f0d10159840 2024-04-25T10:11:22.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.139+0000 7f0d0d600640 1 -- 172.21.15.119:0/791416639 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0d08059230 con 0x7f0d10159840 2024-04-25T10:11:22.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.143+0000 7f0d16000640 1 -- 172.21.15.119:0/791416639 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0cec06b4d0 msgr2=0x7f0cec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.143+0000 7f0d16000640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0cec06b4d0 0x7f0cec06d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0d00006fd0 tx=0x7f0d00008040 comp rx=0 tx=0).stop 2024-04-25T10:11:22.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.143+0000 7f0d16000640 1 -- 172.21.15.119:0/791416639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d10159840 msgr2=0x7f0d100b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.143+0000 7f0d16000640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d10159840 0x7f0d100b3f20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0d08007e70 tx=0x7f0d0800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.143+0000 7f0d16000640 1 -- 172.21.15.119:0/791416639 shutdown_connections 2024-04-25T10:11:22.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.143+0000 7f0d16000640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0cec06b4d0 0x7f0cec06d990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.143+0000 7f0d16000640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d10159840 0x7f0d100b3f20 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.143+0000 7f0d16000640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d100c6150 0x7f0d10159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.143+0000 7f0d16000640 1 --2- 172.21.15.119:0/791416639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0d1006da10 0x7f0d10141fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.143+0000 7f0d16000640 1 -- 172.21.15.119:0/791416639 >> 172.21.15.119:0/791416639 conn(0x7f0d1006aba0 msgr2=0x7f0d100cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:22.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.143+0000 7f0d16000640 1 -- 172.21.15.119:0/791416639 shutdown_connections 2024-04-25T10:11:22.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.143+0000 7f0d16000640 1 -- 172.21.15.119:0/791416639 wait complete. 2024-04-25T10:11:22.170 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:11:22.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.408+0000 7fad7f000640 1 Processor -- start 2024-04-25T10:11:22.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.408+0000 7fad7f000640 1 -- start start 2024-04-25T10:11:22.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.410+0000 7fad7f000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fad780b36f0 0x7fad78163540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.410+0000 7fad7f000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad780b3b40 con 0x7fad78071fc0 2024-04-25T10:11:22.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.410+0000 7fad7f000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7806e3f0 con 0x7fad780b3240 2024-04-25T10:11:22.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.410+0000 7fad7f000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad7806e6d0 con 0x7fad780b36f0 2024-04-25T10:11:22.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.410+0000 7fad77e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fad78071fc0 0x7fad7806d9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39158/0 (socket says 172.21.15.119:39158) 2024-04-25T10:11:22.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.410+0000 7fad77e00640 1 -- 172.21.15.119:0/2041339562 learned_addr learned my addr 172.21.15.119:0/2041339562 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:22.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 -- 172.21.15.119:0/2041339562 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87898051 0 0) 0x7fad780b3b40 con 0x7fad78071fc0 2024-04-25T10:11:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 -- 172.21.15.119:0/2041339562 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad5c003520 con 0x7fad78071fc0 2024-04-25T10:11:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad7d000640 1 --2- 172.21.15.119:0/2041339562 >> v2:172.21.15.132:3300/0 conn(0x7fad780b36f0 0x7fad78163540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 -- 172.21.15.119:0/2041339562 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2787005116 0 0) 0x7fad5c003520 con 0x7fad78071fc0 2024-04-25T10:11:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 -- 172.21.15.119:0/2041339562 >> v2:172.21.15.132:3300/0 conn(0x7fad780b36f0 msgr2=0x7fad78163540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 --2- 172.21.15.119:0/2041339562 >> v2:172.21.15.132:3300/0 conn(0x7fad780b36f0 0x7fad78163540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 -- 172.21.15.119:0/2041339562 >> v1:172.21.15.132:6789/0 conn(0x7fad780b3240 legacy=0x7fad78162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 -- 172.21.15.119:0/2041339562 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7806e9b0 con 0x7fad78071fc0 2024-04-25T10:11:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 -- 172.21.15.119:0/2041339562 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fad68001670 con 0x7fad78071fc0 2024-04-25T10:11:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 -- 172.21.15.119:0/2041339562 >> v1:172.21.15.119:6789/0 conn(0x7fad78071fc0 legacy=0x7fad7806d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 --2- 172.21.15.119:0/2041339562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad5c003a00 0x7fad5c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 --2- 172.21.15.119:0/2041339562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad5c004800 0x7fad5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 --2- 172.21.15.119:0/2041339562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad5c005210 0x7fad5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 -- 172.21.15.119:0/2041339562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7806e6d0 con 0x7fad5c004800 2024-04-25T10:11:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 -- 172.21.15.119:0/2041339562 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c00c160 con 0x7fad5c003a00 2024-04-25T10:11:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.411+0000 7fad76a00640 1 -- 172.21.15.119:0/2041339562 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c00c360 con 0x7fad5c005210 2024-04-25T10:11:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.412+0000 7fad77400640 1 --2- 172.21.15.119:0/2041339562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad5c004800 0x7fad5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.412+0000 7fad77e00640 1 --2- 172.21.15.119:0/2041339562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad5c003a00 0x7fad5c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.412+0000 7fad77400640 1 -- 172.21.15.119:0/2041339562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad5c005210 msgr2=0x7fad5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.412+0000 7fad77400640 1 --2- 172.21.15.119:0/2041339562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad5c005210 0x7fad5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.412+0000 7fad77400640 1 -- 172.21.15.119:0/2041339562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad5c003a00 msgr2=0x7fad5c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:22.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.412+0000 7fad77400640 1 --2- 172.21.15.119:0/2041339562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad5c003a00 0x7fad5c003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.412+0000 7fad77400640 1 -- 172.21.15.119:0/2041339562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c003520 con 0x7fad5c004800 2024-04-25T10:11:22.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.412+0000 7fad7d000640 1 -- 172.21.15.119:0/2041339562 reap_dead start 2024-04-25T10:11:22.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.413+0000 7fad77400640 1 --2- 172.21.15.119:0/2041339562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad5c004800 0x7fad5c004c80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fad64001750 tx=0x7fad5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583f6dc0f780dde8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.413+0000 7fad76a00640 1 -- 172.21.15.119:0/2041339562 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad64001370 con 0x7fad5c004800 2024-04-25T10:11:22.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.413+0000 7fad76a00640 1 -- 172.21.15.119:0/2041339562 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad640052d0 con 0x7fad5c004800 2024-04-25T10:11:22.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.413+0000 7fad76a00640 1 -- 172.21.15.119:0/2041339562 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6402abc0 con 0x7fad5c004800 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.414+0000 7fad7f000640 1 -- 172.21.15.119:0/2041339562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad5c004800 msgr2=0x7fad5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.414+0000 7fad7f000640 1 --2- 172.21.15.119:0/2041339562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad5c004800 0x7fad5c004c80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fad64001750 tx=0x7fad5c005830 comp rx=0 tx=0).stop 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.414+0000 7fad7f000640 1 -- 172.21.15.119:0/2041339562 shutdown_connections 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.414+0000 7fad7f000640 1 --2- 172.21.15.119:0/2041339562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad5c004800 0x7fad5c004c80 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.414+0000 7fad7f000640 1 -- 172.21.15.119:0/2041339562 >> 172.21.15.119:0/2041339562 conn(0x7fad7806aba0 msgr2=0x7fad7806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.414+0000 7fad7f000640 1 -- 172.21.15.119:0/2041339562 shutdown_connections 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.414+0000 7fad7f000640 1 -- 172.21.15.119:0/2041339562 wait complete. 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.416+0000 7fad7f000640 1 Processor -- start 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.416+0000 7fad7f000640 1 -- start start 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.417+0000 7fad7f000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad5c004800 0x7fad780cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.417+0000 7fad7f000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad780cc740 0x7fad780ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.417+0000 7fad7f000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad78163fc0 0x7fad78164440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.417+0000 7fad7f000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78162590 con 0x7fad780cc740 2024-04-25T10:11:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.417+0000 7fad7f000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad780723a0 con 0x7fad78163fc0 2024-04-25T10:11:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.417+0000 7fad7f000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad780713c0 con 0x7fad5c004800 2024-04-25T10:11:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.417+0000 7fad77400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad780cc740 0x7fad780ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.417+0000 7fad77400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad780cc740 0x7fad780ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59232/0 (socket says 172.21.15.119:59232) 2024-04-25T10:11:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.417+0000 7fad77400640 1 -- 172.21.15.119:0/4173134890 learned_addr learned my addr 172.21.15.119:0/4173134890 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.417+0000 7fad77400640 1 -- 172.21.15.119:0/4173134890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad5c004800 msgr2=0x7fad780cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.417+0000 7fad77400640 1 --2- 172.21.15.119:0/4173134890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad5c004800 0x7fad780cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.417+0000 7fad77400640 1 -- 172.21.15.119:0/4173134890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad78163fc0 msgr2=0x7fad78164440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.417+0000 7fad77400640 1 --2- 172.21.15.119:0/4173134890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad78163fc0 0x7fad78164440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.417+0000 7fad77400640 1 -- 172.21.15.119:0/4173134890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7815d540 con 0x7fad780cc740 2024-04-25T10:11:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.418+0000 7fad77400640 1 --2- 172.21.15.119:0/4173134890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad780cc740 0x7fad780ccbc0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fad64005040 tx=0x7fad64001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.418+0000 7fad6fe00640 1 -- 172.21.15.119:0/4173134890 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6403f020 con 0x7fad780cc740 2024-04-25T10:11:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.418+0000 7fad6fe00640 1 -- 172.21.15.119:0/4173134890 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad64005480 con 0x7fad780cc740 2024-04-25T10:11:22.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.418+0000 7fad6fe00640 1 -- 172.21.15.119:0/4173134890 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad64037040 con 0x7fad780cc740 2024-04-25T10:11:22.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.418+0000 7fad7f000640 1 -- 172.21.15.119:0/4173134890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad780cd2b0 con 0x7fad780cc740 2024-04-25T10:11:22.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.418+0000 7fad7f000640 1 -- 172.21.15.119:0/4173134890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad780cd870 con 0x7fad780cc740 2024-04-25T10:11:22.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.419+0000 7fad7f000640 1 -- 172.21.15.119:0/4173134890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad78164980 con 0x7fad780cc740 2024-04-25T10:11:22.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.420+0000 7fad6fe00640 1 -- 172.21.15.119:0/4173134890 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad64042020 con 0x7fad780cc740 2024-04-25T10:11:22.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.421+0000 7fad6fe00640 1 --2- 172.21.15.119:0/4173134890 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fad5806b500 0x7fad5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.421+0000 7fad6fe00640 1 -- 172.21.15.119:0/4173134890 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fad640a9de0 con 0x7fad780cc740 2024-04-25T10:11:22.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.421+0000 7fad77e00640 1 --2- 172.21.15.119:0/4173134890 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fad5806b500 0x7fad5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.422+0000 7fad77e00640 1 --2- 172.21.15.119:0/4173134890 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fad5806b500 0x7fad5806d9c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fad68001860 tx=0x7fad68009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.425+0000 7fad6fe00640 1 -- 172.21.15.119:0/4173134890 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad64049030 con 0x7fad780cc740 2024-04-25T10:11:22.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:22.624+0000 7fad7f000640 1 -- 172.21.15.119:0/4173134890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad780750b0 con 0x7fad780cc740 2024-04-25T10:11:23.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.143+0000 7fad6fe00640 1 -- 172.21.15.119:0/4173134890 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 181+0+0 (secure 0 0 0) 0x7fad64078b60 con 0x7fad780cc740 2024-04-25T10:11:23.148 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:11:23.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.147+0000 7fad7f000640 1 -- 172.21.15.119:0/4173134890 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fad5806b500 msgr2=0x7fad5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.147+0000 7fad7f000640 1 --2- 172.21.15.119:0/4173134890 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fad5806b500 0x7fad5806d9c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fad68001860 tx=0x7fad68009040 comp rx=0 tx=0).stop 2024-04-25T10:11:23.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.148+0000 7fad7f000640 1 -- 172.21.15.119:0/4173134890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad780cc740 msgr2=0x7fad780ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.148+0000 7fad7f000640 1 --2- 172.21.15.119:0/4173134890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad780cc740 0x7fad780ccbc0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fad64005040 tx=0x7fad64001120 comp rx=0 tx=0).stop 2024-04-25T10:11:23.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.149+0000 7fad7f000640 1 -- 172.21.15.119:0/4173134890 shutdown_connections 2024-04-25T10:11:23.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.149+0000 7fad7f000640 1 --2- 172.21.15.119:0/4173134890 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fad5806b500 0x7fad5806d9c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.149+0000 7fad7f000640 1 --2- 172.21.15.119:0/4173134890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad78163fc0 0x7fad78164440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.149+0000 7fad7f000640 1 --2- 172.21.15.119:0/4173134890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad780cc740 0x7fad780ccbc0 secure :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fad64005040 tx=0x7fad64001120 comp rx=0 tx=0).stop 2024-04-25T10:11:23.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.149+0000 7fad7f000640 1 --2- 172.21.15.119:0/4173134890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad5c004800 0x7fad780cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.149+0000 7fad7f000640 1 -- 172.21.15.119:0/4173134890 >> 172.21.15.119:0/4173134890 conn(0x7fad7806aba0 msgr2=0x7fad78160f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.150+0000 7fad7f000640 1 -- 172.21.15.119:0/4173134890 shutdown_connections 2024-04-25T10:11:23.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.152+0000 7fad7f000640 1 -- 172.21.15.119:0/4173134890 wait complete. 2024-04-25T10:11:23.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:11:23.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.370+0000 7f3291200640 1 Processor -- start 2024-04-25T10:11:23.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.371+0000 7f3291200640 1 -- start start 2024-04-25T10:11:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.372+0000 7f3291200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f328c15da80 0x7f328c0ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.372+0000 7f3291200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f328c0ce730 0x7f328c0d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.372+0000 7f3291200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328c0cf200 con 0x7f328c0b6ba0 2024-04-25T10:11:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.372+0000 7f3291200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328c0d22e0 con 0x7f328c0ce730 2024-04-25T10:11:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.372+0000 7f3291200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328c0cec40 con 0x7f328c15da80 2024-04-25T10:11:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.372+0000 7f3283e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f328c0ce730 0x7f328c0d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.372+0000 7f3283e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f328c0ce730 0x7f328c0d01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59246/0 (socket says 172.21.15.119:59246) 2024-04-25T10:11:23.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.372+0000 7f3283e00640 1 -- 172.21.15.119:0/562928898 learned_addr learned my addr 172.21.15.119:0/562928898 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:23.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.372+0000 7f328be00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f328c0b6ba0 0x7f328c0b5410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39166/0 (socket says 172.21.15.119:39166) 2024-04-25T10:11:23.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.373+0000 7f328b400640 1 --2- 172.21.15.119:0/562928898 >> v2:172.21.15.132:3300/0 conn(0x7f328c15da80 0x7f328c0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.373+0000 7f3283e00640 1 -- 172.21.15.119:0/562928898 >> v2:172.21.15.132:3300/0 conn(0x7f328c15da80 msgr2=0x7f328c0ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.373+0000 7f3283e00640 1 --2- 172.21.15.119:0/562928898 >> v2:172.21.15.132:3300/0 conn(0x7f328c15da80 0x7f328c0ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.373+0000 7f3283e00640 1 -- 172.21.15.119:0/562928898 >> v1:172.21.15.119:6789/0 conn(0x7f328c0b6ba0 legacy=0x7f328c0b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.373+0000 7f328aa00640 1 -- 172.21.15.119:0/562928898 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179529424 0 0) 0x7f3284001490 con 0x7f328c0b6ba0 2024-04-25T10:11:23.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.373+0000 7f3283e00640 1 -- 172.21.15.119:0/562928898 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c0cef20 con 0x7f328c0ce730 2024-04-25T10:11:23.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.373+0000 7f328b400640 1 --2- 172.21.15.119:0/562928898 >> v2:172.21.15.132:3300/0 conn(0x7f328c15da80 0x7f328c0ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:23.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.374+0000 7f3283e00640 1 --2- 172.21.15.119:0/562928898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f328c0ce730 0x7f328c0d01a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f328c06cf80 tx=0x7f327402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa8497c7242afbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.374+0000 7f328aa00640 1 -- 172.21.15.119:0/562928898 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274037070 con 0x7f328c0ce730 2024-04-25T10:11:23.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.374+0000 7f328aa00640 1 -- 172.21.15.119:0/562928898 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3274032440 con 0x7f328c0ce730 2024-04-25T10:11:23.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.374+0000 7f328aa00640 1 -- 172.21.15.119:0/562928898 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274032810 con 0x7f328c0ce730 2024-04-25T10:11:23.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.375+0000 7f3291200640 1 -- 172.21.15.119:0/562928898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f328c0ce730 msgr2=0x7f328c0d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.375+0000 7f3291200640 1 --2- 172.21.15.119:0/562928898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f328c0ce730 0x7f328c0d01a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f328c06cf80 tx=0x7f327402a270 comp rx=0 tx=0).stop 2024-04-25T10:11:23.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.375+0000 7f3291200640 1 -- 172.21.15.119:0/562928898 shutdown_connections 2024-04-25T10:11:23.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.375+0000 7f3291200640 1 --2- 172.21.15.119:0/562928898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f328c0ce730 0x7f328c0d01a0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.375+0000 7f3291200640 1 --2- 172.21.15.119:0/562928898 >> v2:172.21.15.132:3300/0 conn(0x7f328c15da80 0x7f328c0ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.375+0000 7f3291200640 1 -- 172.21.15.119:0/562928898 >> 172.21.15.119:0/562928898 conn(0x7f328c06be90 msgr2=0x7f328c06c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.375+0000 7f3291200640 1 -- 172.21.15.119:0/562928898 shutdown_connections 2024-04-25T10:11:23.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.375+0000 7f3291200640 1 -- 172.21.15.119:0/562928898 wait complete. 2024-04-25T10:11:23.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.377+0000 7f3291200640 1 Processor -- start 2024-04-25T10:11:23.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.377+0000 7f3291200640 1 -- start start 2024-04-25T10:11:23.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.378+0000 7f3291200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f328c0b6ba0 0x7f328c160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.378+0000 7f3291200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f328c0ce730 0x7f328c0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.378+0000 7f3291200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f328c15da80 0x7f328c15bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.378+0000 7f3291200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c161b40 con 0x7f328c0b6ba0 2024-04-25T10:11:23.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.378+0000 7f3291200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c1619c0 con 0x7f328c0ce730 2024-04-25T10:11:23.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.378+0000 7f3291200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c15de60 con 0x7f328c15da80 2024-04-25T10:11:23.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.378+0000 7f328b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f328c0b6ba0 0x7f328c160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.378+0000 7f328b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f328c0b6ba0 0x7f328c160e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59262/0 (socket says 172.21.15.119:59262) 2024-04-25T10:11:23.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.378+0000 7f328b400640 1 -- 172.21.15.119:0/3940577420 learned_addr learned my addr 172.21.15.119:0/3940577420 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:23.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.378+0000 7f328b400640 1 -- 172.21.15.119:0/3940577420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f328c15da80 msgr2=0x7f328c15bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.378+0000 7f328b400640 1 --2- 172.21.15.119:0/3940577420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f328c15da80 0x7f328c15bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.379+0000 7f328b400640 1 -- 172.21.15.119:0/3940577420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f328c0ce730 msgr2=0x7f328c0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.379+0000 7f328b400640 1 --2- 172.21.15.119:0/3940577420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f328c0ce730 0x7f328c0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.379+0000 7f328b400640 1 -- 172.21.15.119:0/3940577420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c0cafa0 con 0x7f328c0b6ba0 2024-04-25T10:11:23.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.379+0000 7f328b400640 1 --2- 172.21.15.119:0/3940577420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f328c0b6ba0 0x7f328c160e20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f327c00c820 tx=0x7f327c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.379+0000 7f3283400640 1 -- 172.21.15.119:0/3940577420 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c015070 con 0x7f328c0b6ba0 2024-04-25T10:11:23.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.379+0000 7f3283400640 1 -- 172.21.15.119:0/3940577420 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c007a30 con 0x7f328c0b6ba0 2024-04-25T10:11:23.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.379+0000 7f3291200640 1 -- 172.21.15.119:0/3940577420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328c0b4920 con 0x7f328c0b6ba0 2024-04-25T10:11:23.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.379+0000 7f3283400640 1 -- 172.21.15.119:0/3940577420 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c007de0 con 0x7f328c0b6ba0 2024-04-25T10:11:23.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.379+0000 7f3291200640 1 -- 172.21.15.119:0/3940577420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328c0b4ee0 con 0x7f328c0b6ba0 2024-04-25T10:11:23.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.380+0000 7f3291200640 1 -- 172.21.15.119:0/3940577420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328c0749b0 con 0x7f328c0b6ba0 2024-04-25T10:11:23.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.381+0000 7f3283400640 1 -- 172.21.15.119:0/3940577420 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f327c018070 con 0x7f328c0b6ba0 2024-04-25T10:11:23.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.383+0000 7f3283400640 1 --2- 172.21.15.119:0/3940577420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f326806b4d0 0x7f326806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.383+0000 7f3283400640 1 -- 172.21.15.119:0/3940577420 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f327c08a080 con 0x7f328c0b6ba0 2024-04-25T10:11:23.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.383+0000 7f3283e00640 1 --2- 172.21.15.119:0/3940577420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f326806b4d0 0x7f326806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.384+0000 7f3283e00640 1 --2- 172.21.15.119:0/3940577420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f326806b4d0 0x7f326806d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3274034000 tx=0x7f327402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.387+0000 7f3283400640 1 -- 172.21.15.119:0/3940577420 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f327c058f30 con 0x7f328c0b6ba0 2024-04-25T10:11:23.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:23.591+0000 7f3291200640 1 -- 172.21.15.119:0/3940577420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f328c0d16a0 con 0x7f328c0b6ba0 2024-04-25T10:11:24.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.146+0000 7f3283400640 1 -- 172.21.15.119:0/3940577420 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 169+0+0 (secure 0 0 0) 0x7f327c058830 con 0x7f328c0b6ba0 2024-04-25T10:11:24.152 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2024-04-25T10:11:24.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.150+0000 7f3291200640 1 -- 172.21.15.119:0/3940577420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f326806b4d0 msgr2=0x7f326806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.150+0000 7f3291200640 1 --2- 172.21.15.119:0/3940577420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f326806b4d0 0x7f326806d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3274034000 tx=0x7f327402c000 comp rx=0 tx=0).stop 2024-04-25T10:11:24.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.150+0000 7f3291200640 1 -- 172.21.15.119:0/3940577420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f328c0b6ba0 msgr2=0x7f328c160e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.150+0000 7f3291200640 1 --2- 172.21.15.119:0/3940577420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f328c0b6ba0 0x7f328c160e20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f327c00c820 tx=0x7f327c00cce0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.150+0000 7f3291200640 1 -- 172.21.15.119:0/3940577420 shutdown_connections 2024-04-25T10:11:24.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.151+0000 7f3291200640 1 --2- 172.21.15.119:0/3940577420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f326806b4d0 0x7f326806d990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.151+0000 7f3291200640 1 --2- 172.21.15.119:0/3940577420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f328c15da80 0x7f328c15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.151+0000 7f3291200640 1 --2- 172.21.15.119:0/3940577420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f328c0ce730 0x7f328c0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.151+0000 7f3291200640 1 --2- 172.21.15.119:0/3940577420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f328c0b6ba0 0x7f328c160e20 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.151+0000 7f3291200640 1 -- 172.21.15.119:0/3940577420 >> 172.21.15.119:0/3940577420 conn(0x7f328c06be90 msgr2=0x7f328c1e9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:24.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.151+0000 7f3291200640 1 -- 172.21.15.119:0/3940577420 shutdown_connections 2024-04-25T10:11:24.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.151+0000 7f3291200640 1 -- 172.21.15.119:0/3940577420 wait complete. 2024-04-25T10:11:24.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:11:24.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.367+0000 7f43cca00640 1 Processor -- start 2024-04-25T10:11:24.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.367+0000 7f43cca00640 1 -- start start 2024-04-25T10:11:24.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.368+0000 7f43cca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43c80cea40 0x7f43c806e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.368+0000 7f43cca00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c80712c0 con 0x7f43c80c6150 2024-04-25T10:11:24.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.369+0000 7f43cca00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c80715a0 con 0x7f43c815bcb0 2024-04-25T10:11:24.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.369+0000 7f43cca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c806e790 con 0x7f43c80cea40 2024-04-25T10:11:24.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.369+0000 7f43c7400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f43c80c6150 0x7f43c80ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39178/0 (socket says 172.21.15.119:39178) 2024-04-25T10:11:24.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.369+0000 7f43c7400640 1 -- 172.21.15.119:0/1389536490 learned_addr learned my addr 172.21.15.119:0/1389536490 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:24.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.369+0000 7f43c7e00640 1 --2- 172.21.15.119:0/1389536490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43c80cea40 0x7f43c806e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.369+0000 7f43c6a00640 1 -- 172.21.15.119:0/1389536490 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 505320537 0 0) 0x7f43c80712c0 con 0x7f43c80c6150 2024-04-25T10:11:24.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.369+0000 7f43c6a00640 1 -- 172.21.15.119:0/1389536490 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43ac003520 con 0x7f43c80c6150 2024-04-25T10:11:24.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.370+0000 7f43c6a00640 1 -- 172.21.15.119:0/1389536490 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2020471362 0 0) 0x7f43ac003520 con 0x7f43c80c6150 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.370+0000 7f43c6a00640 1 -- 172.21.15.119:0/1389536490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43c80cea40 msgr2=0x7f43c806e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.370+0000 7f43c6a00640 1 --2- 172.21.15.119:0/1389536490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43c80cea40 0x7f43c806e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.370+0000 7f43c6a00640 1 -- 172.21.15.119:0/1389536490 >> v1:172.21.15.132:6789/0 conn(0x7f43c815bcb0 legacy=0x7f43c806da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.370+0000 7f43c6a00640 1 -- 172.21.15.119:0/1389536490 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c8071880 con 0x7f43c80c6150 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.370+0000 7f43c6a00640 1 -- 172.21.15.119:0/1389536490 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f43b8001670 con 0x7f43c80c6150 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.370+0000 7f43c6a00640 1 -- 172.21.15.119:0/1389536490 >> v1:172.21.15.119:6789/0 conn(0x7f43c80c6150 legacy=0x7f43c80ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.370+0000 7f43c6a00640 1 --2- 172.21.15.119:0/1389536490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43ac0039e0 0x7f43ac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.370+0000 7f43c6a00640 1 --2- 172.21.15.119:0/1389536490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43c80cea40 0x7f43ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.370+0000 7f43c6a00640 1 --2- 172.21.15.119:0/1389536490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43ac004d40 0x7f43ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.370+0000 7f43c6a00640 1 -- 172.21.15.119:0/1389536490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac00bb50 con 0x7f43ac0039e0 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.370+0000 7f43c6a00640 1 -- 172.21.15.119:0/1389536490 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac00bd00 con 0x7f43ac004d40 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.370+0000 7f43c6a00640 1 -- 172.21.15.119:0/1389536490 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac00bf00 con 0x7f43c80cea40 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.370+0000 7f43c7400640 1 --2- 172.21.15.119:0/1389536490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43ac0039e0 0x7f43ac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.371+0000 7f43c7e00640 1 --2- 172.21.15.119:0/1389536490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43ac004d40 0x7f43ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.371+0000 7f43bfe00640 1 --2- 172.21.15.119:0/1389536490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43c80cea40 0x7f43ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.371+0000 7f43c7400640 1 -- 172.21.15.119:0/1389536490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43c80cea40 msgr2=0x7f43ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:24.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.371+0000 7f43c7400640 1 --2- 172.21.15.119:0/1389536490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43c80cea40 0x7f43ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.371+0000 7f43c7400640 1 -- 172.21.15.119:0/1389536490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43ac004d40 msgr2=0x7f43ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:24.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.371+0000 7f43c7400640 1 --2- 172.21.15.119:0/1389536490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43ac004d40 0x7f43ac0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.371+0000 7f43c7400640 1 -- 172.21.15.119:0/1389536490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac003520 con 0x7f43ac0039e0 2024-04-25T10:11:24.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.372+0000 7f43c7400640 1 --2- 172.21.15.119:0/1389536490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43ac0039e0 0x7f43ac003e00 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f43b800c0e0 tx=0x7f43ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73211ab8b84022d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.372+0000 7f43c6a00640 1 -- 172.21.15.119:0/1389536490 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8007c20 con 0x7f43ac0039e0 2024-04-25T10:11:24.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.372+0000 7f43c6a00640 1 -- 172.21.15.119:0/1389536490 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b8005af0 con 0x7f43ac0039e0 2024-04-25T10:11:24.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.372+0000 7f43c6a00640 1 -- 172.21.15.119:0/1389536490 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8007070 con 0x7f43ac0039e0 2024-04-25T10:11:24.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.373+0000 7f43cca00640 1 -- 172.21.15.119:0/1389536490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43ac0039e0 msgr2=0x7f43ac003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.373+0000 7f43cca00640 1 --2- 172.21.15.119:0/1389536490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43ac0039e0 0x7f43ac003e00 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f43b800c0e0 tx=0x7f43ac005830 comp rx=0 tx=0).stop 2024-04-25T10:11:24.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.373+0000 7f43c7e00640 1 -- 172.21.15.119:0/1389536490 reap_dead start 2024-04-25T10:11:24.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.373+0000 7f43cca00640 1 -- 172.21.15.119:0/1389536490 shutdown_connections 2024-04-25T10:11:24.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.373+0000 7f43cca00640 1 -- 172.21.15.119:0/1389536490 >> 172.21.15.119:0/1389536490 conn(0x7f43c806aba0 msgr2=0x7f43c806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:24.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.373+0000 7f43cca00640 1 -- 172.21.15.119:0/1389536490 shutdown_connections 2024-04-25T10:11:24.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.373+0000 7f43cca00640 1 -- 172.21.15.119:0/1389536490 wait complete. 2024-04-25T10:11:24.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.375+0000 7f43cca00640 1 Processor -- start 2024-04-25T10:11:24.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.375+0000 7f43cca00640 1 -- start start 2024-04-25T10:11:24.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.376+0000 7f43cca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43ac0039e0 0x7f43c81412b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.376+0000 7f43cca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43c8154400 0x7f43c8148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.376+0000 7f43cca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43c8154880 0x7f43c8149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.376+0000 7f43cca00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c806e120 con 0x7f43c8154880 2024-04-25T10:11:24.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.376+0000 7f43cca00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80c6530 con 0x7f43c8154400 2024-04-25T10:11:24.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.376+0000 7f43cca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c806e2a0 con 0x7f43ac0039e0 2024-04-25T10:11:24.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.376+0000 7f43c7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43c8154880 0x7f43c8149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.376+0000 7f43c7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43c8154880 0x7f43c8149360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59270/0 (socket says 172.21.15.119:59270) 2024-04-25T10:11:24.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.376+0000 7f43c7e00640 1 -- 172.21.15.119:0/3639069177 learned_addr learned my addr 172.21.15.119:0/3639069177 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:24.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.377+0000 7f43c7400640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43ac0039e0 0x7f43c81412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.377+0000 7f43bfe00640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43c8154400 0x7f43c8148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.377+0000 7f43c7e00640 1 -- 172.21.15.119:0/3639069177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43ac0039e0 msgr2=0x7f43c81412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.377+0000 7f43c7e00640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43ac0039e0 0x7f43c81412b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.377+0000 7f43c7e00640 1 -- 172.21.15.119:0/3639069177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43c8154400 msgr2=0x7f43c8148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.377+0000 7f43c7e00640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43c8154400 0x7f43c8148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.377+0000 7f43c7e00640 1 -- 172.21.15.119:0/3639069177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c8154dc0 con 0x7f43c8154880 2024-04-25T10:11:24.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.377+0000 7f43bfe00640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43c8154400 0x7f43c8148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:24.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.377+0000 7f43c7e00640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43c8154880 0x7f43c8149360 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f43c806f450 tx=0x7f43c000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.377+0000 7f43bf400640 1 -- 172.21.15.119:0/3639069177 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c001c070 con 0x7f43c8154880 2024-04-25T10:11:24.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.377+0000 7f43bf400640 1 -- 172.21.15.119:0/3639069177 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c0017400 con 0x7f43c8154880 2024-04-25T10:11:24.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.378+0000 7f43bf400640 1 -- 172.21.15.119:0/3639069177 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c00177f0 con 0x7f43c8154880 2024-04-25T10:11:24.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.378+0000 7f43cca00640 1 -- 172.21.15.119:0/3639069177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c81550a0 con 0x7f43c8154880 2024-04-25T10:11:24.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.378+0000 7f43cca00640 1 -- 172.21.15.119:0/3639069177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c8155660 con 0x7f43c8154880 2024-04-25T10:11:24.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.379+0000 7f43cca00640 1 -- 172.21.15.119:0/3639069177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c806b260 con 0x7f43c8154880 2024-04-25T10:11:24.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.380+0000 7f43bf400640 1 -- 172.21.15.119:0/3639069177 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43c001f070 con 0x7f43c8154880 2024-04-25T10:11:24.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.381+0000 7f43bf400640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43a406b500 0x7f43a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.381+0000 7f43bf400640 1 -- 172.21.15.119:0/3639069177 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f43c0089e40 con 0x7f43c8154880 2024-04-25T10:11:24.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.381+0000 7f43c7400640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43a406b500 0x7f43a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.382+0000 7f43c7400640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43a406b500 0x7f43a406d9c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f43b8032000 tx=0x7f43b8009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.385+0000 7f43bf400640 1 -- 172.21.15.119:0/3639069177 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43c0012070 con 0x7f43c8154880 2024-04-25T10:11:24.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.580+0000 7f43cca00640 1 -- 172.21.15.119:0/3639069177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f43c806af80 con 0x7f43c8154880 2024-04-25T10:11:24.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.586+0000 7f43bf400640 1 -- 172.21.15.119:0/3639069177 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f43c0058f20 con 0x7f43c8154880 2024-04-25T10:11:24.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.589+0000 7f43cca00640 1 -- 172.21.15.119:0/3639069177 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43a406b500 msgr2=0x7f43a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.589+0000 7f43cca00640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43a406b500 0x7f43a406d9c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f43b8032000 tx=0x7f43b8009030 comp rx=0 tx=0).stop 2024-04-25T10:11:24.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.589+0000 7f43cca00640 1 -- 172.21.15.119:0/3639069177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43c8154880 msgr2=0x7f43c8149360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.589+0000 7f43cca00640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43c8154880 0x7f43c8149360 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f43c806f450 tx=0x7f43c000f4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.589+0000 7f43cca00640 1 -- 172.21.15.119:0/3639069177 shutdown_connections 2024-04-25T10:11:24.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.589+0000 7f43cca00640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43a406b500 0x7f43a406d9c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.589+0000 7f43cca00640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43c8154880 0x7f43c8149360 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.589+0000 7f43cca00640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43c8154400 0x7f43c8148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.589+0000 7f43cca00640 1 --2- 172.21.15.119:0/3639069177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43ac0039e0 0x7f43c81412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.589+0000 7f43cca00640 1 -- 172.21.15.119:0/3639069177 >> 172.21.15.119:0/3639069177 conn(0x7f43c806aba0 msgr2=0x7f43c80b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:24.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.590+0000 7f43cca00640 1 -- 172.21.15.119:0/3639069177 shutdown_connections 2024-04-25T10:11:24.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.590+0000 7f43cca00640 1 -- 172.21.15.119:0/3639069177 wait complete. 2024-04-25T10:11:24.613 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T10:11:24.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.789+0000 7fdd8b000640 1 Processor -- start 2024-04-25T10:11:24.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.789+0000 7fdd8b000640 1 -- start start 2024-04-25T10:11:24.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.790+0000 7fdd8b000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdd84144470 0x7fdd8415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.790+0000 7fdd8b000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd8406e430 0x7fdd840b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.791+0000 7fdd8b000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd840cd440 con 0x7fdd840b4e80 2024-04-25T10:11:24.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.791+0000 7fdd8b000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd84145340 con 0x7fdd84144470 2024-04-25T10:11:24.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.791+0000 7fdd8b000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840cd720 con 0x7fdd8406e430 2024-04-25T10:11:24.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.791+0000 7fdd89c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdd84144470 0x7fdd8415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.791+0000 7fdd89c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdd84144470 0x7fdd8415fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59278/0 (socket says 172.21.15.119:59278) 2024-04-25T10:11:24.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.791+0000 7fdd89c00640 1 -- 172.21.15.119:0/2708605203 learned_addr learned my addr 172.21.15.119:0/2708605203 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:24.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.791+0000 7fdd89c00640 1 -- 172.21.15.119:0/2708605203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd8406e430 msgr2=0x7fdd840b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:24.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.791+0000 7fdd89c00640 1 --2- 172.21.15.119:0/2708605203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd8406e430 0x7fdd840b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.791+0000 7fdd89c00640 1 -- 172.21.15.119:0/2708605203 >> v1:172.21.15.132:6789/0 conn(0x7fdd840b4e80 legacy=0x7fdd8406d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:24.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.791+0000 7fdd89c00640 1 -- 172.21.15.119:0/2708605203 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd840cda00 con 0x7fdd84144470 2024-04-25T10:11:24.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.792+0000 7fdd89c00640 1 --2- 172.21.15.119:0/2708605203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84144470 0x7fdd8415fc90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009870 tx=0x7fdd7802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b2ddbbef2df01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.792+0000 7fdd83e00640 1 -- 172.21.15.119:0/2708605203 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78037070 con 0x7fdd84144470 2024-04-25T10:11:24.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.793+0000 7fdd83e00640 1 -- 172.21.15.119:0/2708605203 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd78032440 con 0x7fdd84144470 2024-04-25T10:11:24.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.793+0000 7fdd83e00640 1 -- 172.21.15.119:0/2708605203 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78032810 con 0x7fdd84144470 2024-04-25T10:11:24.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.793+0000 7fdd8b000640 1 -- 172.21.15.119:0/2708605203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84144470 msgr2=0x7fdd8415fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.793+0000 7fdd8b000640 1 --2- 172.21.15.119:0/2708605203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84144470 0x7fdd8415fc90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009870 tx=0x7fdd7802a210 comp rx=0 tx=0).stop 2024-04-25T10:11:24.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.793+0000 7fdd8b000640 1 -- 172.21.15.119:0/2708605203 shutdown_connections 2024-04-25T10:11:24.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.793+0000 7fdd8b000640 1 --2- 172.21.15.119:0/2708605203 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd8406e430 0x7fdd840b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.793+0000 7fdd8b000640 1 --2- 172.21.15.119:0/2708605203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84144470 0x7fdd8415fc90 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.793+0000 7fdd8b000640 1 -- 172.21.15.119:0/2708605203 >> 172.21.15.119:0/2708605203 conn(0x7fdd8406b1d0 msgr2=0x7fdd840b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:24.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.793+0000 7fdd8b000640 1 -- 172.21.15.119:0/2708605203 shutdown_connections 2024-04-25T10:11:24.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.793+0000 7fdd8b000640 1 -- 172.21.15.119:0/2708605203 wait complete. 2024-04-25T10:11:24.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.795+0000 7fdd8b000640 1 Processor -- start 2024-04-25T10:11:24.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.795+0000 7fdd8b000640 1 -- start start 2024-04-25T10:11:24.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.796+0000 7fdd8b000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd8406e430 0x7fdd8413afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.796+0000 7fdd8b000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd840b4e80 0x7fdd84140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.796+0000 7fdd8b000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84144470 0x7fdd841410a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.796+0000 7fdd8b000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840ce200 con 0x7fdd84144470 2024-04-25T10:11:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.796+0000 7fdd8b000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd84144850 con 0x7fdd840b4e80 2024-04-25T10:11:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.796+0000 7fdd8b000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8415cd00 con 0x7fdd8406e430 2024-04-25T10:11:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.796+0000 7fdd8a600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84144470 0x7fdd841410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.796+0000 7fdd8a600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84144470 0x7fdd841410a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59280/0 (socket says 172.21.15.119:59280) 2024-04-25T10:11:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.796+0000 7fdd8a600640 1 -- 172.21.15.119:0/539081200 learned_addr learned my addr 172.21.15.119:0/539081200 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.796+0000 7fdd89c00640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd8406e430 0x7fdd8413afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.796+0000 7fdd89200640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd840b4e80 0x7fdd84140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.796+0000 7fdd8a600640 1 -- 172.21.15.119:0/539081200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd8406e430 msgr2=0x7fdd8413afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.796+0000 7fdd8a600640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd8406e430 0x7fdd8413afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.797+0000 7fdd8a600640 1 -- 172.21.15.119:0/539081200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd840b4e80 msgr2=0x7fdd84140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.797+0000 7fdd8a600640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd840b4e80 0x7fdd84140b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.797+0000 7fdd8a600640 1 -- 172.21.15.119:0/539081200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8415dac0 con 0x7fdd84144470 2024-04-25T10:11:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.797+0000 7fdd89200640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd840b4e80 0x7fdd84140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.797+0000 7fdd8a600640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84144470 0x7fdd841410a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00c780 tx=0x7fdd7c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.797+0000 7fdd81600640 1 -- 172.21.15.119:0/539081200 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c004b60 con 0x7fdd84144470 2024-04-25T10:11:24.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.797+0000 7fdd81600640 1 -- 172.21.15.119:0/539081200 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7c004d00 con 0x7fdd84144470 2024-04-25T10:11:24.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.797+0000 7fdd8b000640 1 -- 172.21.15.119:0/539081200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8413e190 con 0x7fdd84144470 2024-04-25T10:11:24.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.797+0000 7fdd81600640 1 -- 172.21.15.119:0/539081200 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c00aa90 con 0x7fdd84144470 2024-04-25T10:11:24.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.797+0000 7fdd8b000640 1 -- 172.21.15.119:0/539081200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8415e040 con 0x7fdd84144470 2024-04-25T10:11:24.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.798+0000 7fdd8b000640 1 -- 172.21.15.119:0/539081200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd840749b0 con 0x7fdd84144470 2024-04-25T10:11:24.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.799+0000 7fdd81600640 1 -- 172.21.15.119:0/539081200 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd7c00ac30 con 0x7fdd84144470 2024-04-25T10:11:24.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.801+0000 7fdd81600640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdd6406b4d0 0x7fdd6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.801+0000 7fdd81600640 1 -- 172.21.15.119:0/539081200 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdd7c08a200 con 0x7fdd84144470 2024-04-25T10:11:24.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.801+0000 7fdd89c00640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdd6406b4d0 0x7fdd6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.802+0000 7fdd89c00640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdd6406b4d0 0x7fdd6406d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdd78034000 tx=0x7fdd7802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:24.805+0000 7fdd81600640 1 -- 172.21.15.119:0/539081200 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd7c020070 con 0x7fdd84144470 2024-04-25T10:11:25.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.070+0000 7fdd8b000640 1 -- 172.21.15.119:0/539081200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fdd840b24a0 con 0x7fdd84144470 2024-04-25T10:11:25.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.150+0000 7fdd81600640 1 -- 172.21.15.119:0/539081200 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v227) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdd7c0592e0 con 0x7fdd84144470 2024-04-25T10:11:25.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.153+0000 7fdd8b000640 1 -- 172.21.15.119:0/539081200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdd6406b4d0 msgr2=0x7fdd6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.153+0000 7fdd8b000640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdd6406b4d0 0x7fdd6406d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdd78034000 tx=0x7fdd7802e000 comp rx=0 tx=0).stop 2024-04-25T10:11:25.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.153+0000 7fdd8b000640 1 -- 172.21.15.119:0/539081200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84144470 msgr2=0x7fdd841410a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.153+0000 7fdd8b000640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84144470 0x7fdd841410a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00c780 tx=0x7fdd7c00cc40 comp rx=0 tx=0).stop 2024-04-25T10:11:25.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.153+0000 7fdd8b000640 1 -- 172.21.15.119:0/539081200 shutdown_connections 2024-04-25T10:11:25.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.153+0000 7fdd8b000640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdd6406b4d0 0x7fdd6406d990 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.153+0000 7fdd8b000640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84144470 0x7fdd841410a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.153+0000 7fdd8b000640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd840b4e80 0x7fdd84140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.153+0000 7fdd8b000640 1 --2- 172.21.15.119:0/539081200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd8406e430 0x7fdd8413afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.153+0000 7fdd8b000640 1 -- 172.21.15.119:0/539081200 >> 172.21.15.119:0/539081200 conn(0x7fdd8406b1d0 msgr2=0x7fdd840b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:25.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.153+0000 7fdd8b000640 1 -- 172.21.15.119:0/539081200 shutdown_connections 2024-04-25T10:11:25.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.153+0000 7fdd8b000640 1 -- 172.21.15.119:0/539081200 wait complete. 2024-04-25T10:11:25.178 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:11:25.178 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:11:25.178 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:11:25.179 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:25.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.365+0000 7f78fb200640 1 Processor -- start 2024-04-25T10:11:25.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.365+0000 7f78fb200640 1 -- start start 2024-04-25T10:11:25.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.366+0000 7f78fb200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f78fc0b1a30 0x7f78fc0b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.366+0000 7f78fb200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc0cec30 con 0x7f78fc0b5470 2024-04-25T10:11:25.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.366+0000 7f78fb200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc0cef10 con 0x7f78fc0b1e90 2024-04-25T10:11:25.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.366+0000 7f78fb200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc0cf1f0 con 0x7f78fc0b1a30 2024-04-25T10:11:25.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.367+0000 7f78f9e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f78fc0b5470 0x7f78fc159ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39192/0 (socket says 172.21.15.119:39192) 2024-04-25T10:11:25.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.367+0000 7f78f9e00640 1 -- 172.21.15.119:0/2338549198 learned_addr learned my addr 172.21.15.119:0/2338549198 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:25.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.367+0000 7f78f1400640 1 --2- 172.21.15.119:0/2338549198 >> v2:172.21.15.119:3300/0 conn(0x7f78fc0b1a30 0x7f78fc0b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.367+0000 7f78f1400640 1 -- 172.21.15.119:0/2338549198 >> v1:172.21.15.132:6789/0 conn(0x7f78fc0b1e90 legacy=0x7f78fc0b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:25.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.367+0000 7f78f1400640 1 -- 172.21.15.119:0/2338549198 >> v1:172.21.15.119:6789/0 conn(0x7f78fc0b5470 legacy=0x7f78fc159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.367+0000 7f78f1400640 1 -- 172.21.15.119:0/2338549198 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc06d420 con 0x7f78fc0b1a30 2024-04-25T10:11:25.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.368+0000 7f78f1400640 1 --2- 172.21.15.119:0/2338549198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78fc0b1a30 0x7f78fc0b3420 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f78e40097a0 tx=0x7f78e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f21a76dd88edac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.368+0000 7f78f9400640 1 -- 172.21.15.119:0/2338549198 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e4037070 con 0x7f78fc0b1a30 2024-04-25T10:11:25.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.368+0000 7f78f9400640 1 -- 172.21.15.119:0/2338549198 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e4032440 con 0x7f78fc0b1a30 2024-04-25T10:11:25.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.368+0000 7f78f9400640 1 -- 172.21.15.119:0/2338549198 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e4032810 con 0x7f78fc0b1a30 2024-04-25T10:11:25.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.369+0000 7f78fb200640 1 -- 172.21.15.119:0/2338549198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78fc0b1a30 msgr2=0x7f78fc0b3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.369+0000 7f78fb200640 1 --2- 172.21.15.119:0/2338549198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78fc0b1a30 0x7f78fc0b3420 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f78e40097a0 tx=0x7f78e402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.369+0000 7f78fb200640 1 -- 172.21.15.119:0/2338549198 shutdown_connections 2024-04-25T10:11:25.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.369+0000 7f78fb200640 1 --2- 172.21.15.119:0/2338549198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78fc0b1a30 0x7f78fc0b3420 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.369+0000 7f78fb200640 1 -- 172.21.15.119:0/2338549198 >> 172.21.15.119:0/2338549198 conn(0x7f78fc06b1d0 msgr2=0x7f78fc0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:25.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.369+0000 7f78fb200640 1 -- 172.21.15.119:0/2338549198 shutdown_connections 2024-04-25T10:11:25.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.369+0000 7f78fb200640 1 -- 172.21.15.119:0/2338549198 wait complete. 2024-04-25T10:11:25.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.371+0000 7f78fb200640 1 Processor -- start 2024-04-25T10:11:25.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.371+0000 7f78fb200640 1 -- start start 2024-04-25T10:11:25.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.372+0000 7f78fb200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78fc0b1a30 0x7f78fc140f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.372+0000 7f78fb200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78fc0b1e90 0x7f78fc152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.372+0000 7f78fb200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78fc0b5470 0x7f78fc15ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.372+0000 7f78fb200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0ce9f0 con 0x7f78fc0b5470 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.372+0000 7f78fb200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0ce870 con 0x7f78fc0b1a30 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.372+0000 7f78fb200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0b4070 con 0x7f78fc0b1e90 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.372+0000 7f78fa800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78fc0b5470 0x7f78fc15ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.372+0000 7f78fa800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78fc0b5470 0x7f78fc15ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59300/0 (socket says 172.21.15.119:59300) 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.372+0000 7f78fa800640 1 -- 172.21.15.119:0/3677599913 learned_addr learned my addr 172.21.15.119:0/3677599913 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.372+0000 7f78fa800640 1 -- 172.21.15.119:0/3677599913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78fc0b1e90 msgr2=0x7f78fc152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.372+0000 7f78fa800640 1 --2- 172.21.15.119:0/3677599913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78fc0b1e90 0x7f78fc152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.372+0000 7f78fa800640 1 -- 172.21.15.119:0/3677599913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78fc0b1a30 msgr2=0x7f78fc140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.372+0000 7f78fa800640 1 --2- 172.21.15.119:0/3677599913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78fc0b1a30 0x7f78fc140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.372+0000 7f78fa800640 1 -- 172.21.15.119:0/3677599913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc153420 con 0x7f78fc0b5470 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.373+0000 7f78fa800640 1 --2- 172.21.15.119:0/3677599913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78fc0b5470 0x7f78fc15ace0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f78f400c780 tx=0x7f78f400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.373+0000 7f78f2a00640 1 -- 172.21.15.119:0/3677599913 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4017070 con 0x7f78fc0b5470 2024-04-25T10:11:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.373+0000 7f78f2a00640 1 -- 172.21.15.119:0/3677599913 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f4003da0 con 0x7f78fc0b5470 2024-04-25T10:11:25.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.373+0000 7f78f2a00640 1 -- 172.21.15.119:0/3677599913 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f40052b0 con 0x7f78fc0b5470 2024-04-25T10:11:25.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.373+0000 7f78fb200640 1 -- 172.21.15.119:0/3677599913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc151e40 con 0x7f78fc0b5470 2024-04-25T10:11:25.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.373+0000 7f78fb200640 1 -- 172.21.15.119:0/3677599913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc152400 con 0x7f78fc0b5470 2024-04-25T10:11:25.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.374+0000 7f78fb200640 1 -- 172.21.15.119:0/3677599913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc0749b0 con 0x7f78fc0b5470 2024-04-25T10:11:25.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.375+0000 7f78f2a00640 1 -- 172.21.15.119:0/3677599913 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78f401a070 con 0x7f78fc0b5470 2024-04-25T10:11:25.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.376+0000 7f78f2a00640 1 --2- 172.21.15.119:0/3677599913 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f78d806b4d0 0x7f78d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.376+0000 7f78f2a00640 1 -- 172.21.15.119:0/3677599913 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f78f40894f0 con 0x7f78fc0b5470 2024-04-25T10:11:25.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.376+0000 7f78f9e00640 1 --2- 172.21.15.119:0/3677599913 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f78d806b4d0 0x7f78d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.377+0000 7f78f9e00640 1 --2- 172.21.15.119:0/3677599913 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f78d806b4d0 0x7f78d806d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f78ec000f60 tx=0x7f78ec007a80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.381+0000 7f78f2a00640 1 -- 172.21.15.119:0/3677599913 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78f4058700 con 0x7f78fc0b5470 2024-04-25T10:11:25.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.565+0000 7f78fb200640 1 -- 172.21.15.119:0/3677599913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78fc152ac0 con 0x7f78fc0b5470 2024-04-25T10:11:25.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.566+0000 7f78f2a00640 1 -- 172.21.15.119:0/3677599913 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+13058 (secure 0 0 0) 0x7f78f4058000 con 0x7f78fc0b5470 2024-04-25T10:11:25.572 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:25.572 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":58,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:11:24.143713+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6836/4129112885":"2024-04-26T10:11:21.133019+0000","172.21.15.119:6835/4129112885":"2024-04-26T10:11:21.133019+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:25.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.569+0000 7f78fb200640 1 -- 172.21.15.119:0/3677599913 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f78d806b4d0 msgr2=0x7f78d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.569+0000 7f78fb200640 1 --2- 172.21.15.119:0/3677599913 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f78d806b4d0 0x7f78d806d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f78ec000f60 tx=0x7f78ec007a80 comp rx=0 tx=0).stop 2024-04-25T10:11:25.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.569+0000 7f78fb200640 1 -- 172.21.15.119:0/3677599913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78fc0b5470 msgr2=0x7f78fc15ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.569+0000 7f78fb200640 1 --2- 172.21.15.119:0/3677599913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78fc0b5470 0x7f78fc15ace0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f78f400c780 tx=0x7f78f400cc40 comp rx=0 tx=0).stop 2024-04-25T10:11:25.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.569+0000 7f78fb200640 1 -- 172.21.15.119:0/3677599913 shutdown_connections 2024-04-25T10:11:25.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.569+0000 7f78fb200640 1 --2- 172.21.15.119:0/3677599913 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f78d806b4d0 0x7f78d806d990 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.569+0000 7f78fb200640 1 --2- 172.21.15.119:0/3677599913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78fc0b5470 0x7f78fc15ace0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.569+0000 7f78fb200640 1 --2- 172.21.15.119:0/3677599913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f78fc0b1e90 0x7f78fc152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.569+0000 7f78fb200640 1 --2- 172.21.15.119:0/3677599913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78fc0b1a30 0x7f78fc140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.569+0000 7f78fb200640 1 -- 172.21.15.119:0/3677599913 >> 172.21.15.119:0/3677599913 conn(0x7f78fc06b1d0 msgr2=0x7f78fc154480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:25.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.569+0000 7f78fb200640 1 -- 172.21.15.119:0/3677599913 shutdown_connections 2024-04-25T10:11:25.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.569+0000 7f78fb200640 1 -- 172.21.15.119:0/3677599913 wait complete. 2024-04-25T10:11:25.593 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:25.593 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:25.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.790+0000 7fc460e00640 1 Processor -- start 2024-04-25T10:11:25.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.791+0000 7fc460e00640 1 -- start start 2024-04-25T10:11:25.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.792+0000 7fc460e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc45c0d3b90 0x7fc45c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.792+0000 7fc460e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc45c0d0f60 0x7fc45c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.792+0000 7fc460e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc45c0d4090 con 0x7fc45c15dca0 2024-04-25T10:11:25.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.792+0000 7fc460e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45c0b6b20 con 0x7fc45c0d0f60 2024-04-25T10:11:25.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.792+0000 7fc460e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45c0d1390 con 0x7fc45c0d3b90 2024-04-25T10:11:25.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.792+0000 7fc45a800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc45c0d0f60 0x7fc45c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.792+0000 7fc45a800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc45c0d0f60 0x7fc45c0b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59308/0 (socket says 172.21.15.119:59308) 2024-04-25T10:11:25.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.792+0000 7fc45a800640 1 -- 172.21.15.119:0/2944624053 learned_addr learned my addr 172.21.15.119:0/2944624053 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:25.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.793+0000 7fc45a800640 1 -- 172.21.15.119:0/2944624053 >> v2:172.21.15.132:3300/0 conn(0x7fc45c0d3b90 msgr2=0x7fc45c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:25.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.793+0000 7fc45a800640 1 --2- 172.21.15.119:0/2944624053 >> v2:172.21.15.132:3300/0 conn(0x7fc45c0d3b90 0x7fc45c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.793+0000 7fc45a800640 1 -- 172.21.15.119:0/2944624053 >> v1:172.21.15.119:6789/0 conn(0x7fc45c15dca0 legacy=0x7fc45c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.793+0000 7fc45a800640 1 -- 172.21.15.119:0/2944624053 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c0d4370 con 0x7fc45c0d0f60 2024-04-25T10:11:25.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.794+0000 7fc45a800640 1 --2- 172.21.15.119:0/2944624053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc45c0d0f60 0x7fc45c0b2500 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0b6560 tx=0x7fc45402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ffcce92fcec637 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.794+0000 7fc458a00640 1 -- 172.21.15.119:0/2944624053 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454037070 con 0x7fc45c0d0f60 2024-04-25T10:11:25.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.794+0000 7fc458a00640 1 -- 172.21.15.119:0/2944624053 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc454032440 con 0x7fc45c0d0f60 2024-04-25T10:11:25.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.794+0000 7fc458a00640 1 -- 172.21.15.119:0/2944624053 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454032810 con 0x7fc45c0d0f60 2024-04-25T10:11:25.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.795+0000 7fc460e00640 1 -- 172.21.15.119:0/2944624053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc45c0d0f60 msgr2=0x7fc45c0b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.795+0000 7fc460e00640 1 --2- 172.21.15.119:0/2944624053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc45c0d0f60 0x7fc45c0b2500 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0b6560 tx=0x7fc45402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.795+0000 7fc460e00640 1 -- 172.21.15.119:0/2944624053 shutdown_connections 2024-04-25T10:11:25.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.795+0000 7fc460e00640 1 --2- 172.21.15.119:0/2944624053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc45c0d0f60 0x7fc45c0b2500 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.795+0000 7fc460e00640 1 --2- 172.21.15.119:0/2944624053 >> v2:172.21.15.132:3300/0 conn(0x7fc45c0d3b90 0x7fc45c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.795+0000 7fc460e00640 1 -- 172.21.15.119:0/2944624053 >> 172.21.15.119:0/2944624053 conn(0x7fc45c0726c0 msgr2=0x7fc45c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:25.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.795+0000 7fc460e00640 1 -- 172.21.15.119:0/2944624053 shutdown_connections 2024-04-25T10:11:25.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.795+0000 7fc460e00640 1 -- 172.21.15.119:0/2944624053 wait complete. 2024-04-25T10:11:25.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.796+0000 7fc460e00640 1 Processor -- start 2024-04-25T10:11:25.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.797+0000 7fc460e00640 1 -- start start 2024-04-25T10:11:25.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.797+0000 7fc460e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc45c0d0f60 0x7fc45c152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.797+0000 7fc460e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc45c0d3b90 0x7fc45c0aa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.797+0000 7fc460e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc45c15dca0 0x7fc45c0aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.797+0000 7fc460e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c0cf050 con 0x7fc45c0d3b90 2024-04-25T10:11:25.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.797+0000 7fc460e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c0b6d50 con 0x7fc45c15dca0 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.797+0000 7fc460e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c15e080 con 0x7fc45c0d0f60 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.798+0000 7fc459400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc45c0d3b90 0x7fc45c0aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.798+0000 7fc459400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc45c0d3b90 0x7fc45c0aa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59312/0 (socket says 172.21.15.119:59312) 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.798+0000 7fc459400640 1 -- 172.21.15.119:0/3767236884 learned_addr learned my addr 172.21.15.119:0/3767236884 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.798+0000 7fc459e00640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc45c0d0f60 0x7fc45c152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.798+0000 7fc45a800640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc45c15dca0 0x7fc45c0aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.798+0000 7fc459400640 1 -- 172.21.15.119:0/3767236884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc45c0d0f60 msgr2=0x7fc45c152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.798+0000 7fc459400640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc45c0d0f60 0x7fc45c152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.798+0000 7fc459400640 1 -- 172.21.15.119:0/3767236884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc45c15dca0 msgr2=0x7fc45c0aa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.798+0000 7fc459400640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc45c15dca0 0x7fc45c0aa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.798+0000 7fc459400640 1 -- 172.21.15.119:0/3767236884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c0c8130 con 0x7fc45c0d3b90 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.798+0000 7fc459e00640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc45c0d0f60 0x7fc45c152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.799+0000 7fc459400640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc45c0d3b90 0x7fc45c0aa480 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc44800b9f0 tx=0x7fc44800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.799+0000 7fc452000640 1 -- 172.21.15.119:0/3767236884 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448019070 con 0x7fc45c0d3b90 2024-04-25T10:11:25.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.799+0000 7fc452000640 1 -- 172.21.15.119:0/3767236884 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44800cbf0 con 0x7fc45c0d3b90 2024-04-25T10:11:25.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.799+0000 7fc460e00640 1 -- 172.21.15.119:0/3767236884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45c0c8410 con 0x7fc45c0d3b90 2024-04-25T10:11:25.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.799+0000 7fc452000640 1 -- 172.21.15.119:0/3767236884 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448014440 con 0x7fc45c0d3b90 2024-04-25T10:11:25.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.799+0000 7fc460e00640 1 -- 172.21.15.119:0/3767236884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45c150710 con 0x7fc45c0d3b90 2024-04-25T10:11:25.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.800+0000 7fc460e00640 1 -- 172.21.15.119:0/3767236884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45c0750b0 con 0x7fc45c0d3b90 2024-04-25T10:11:25.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.801+0000 7fc452000640 1 -- 172.21.15.119:0/3767236884 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc44801c070 con 0x7fc45c0d3b90 2024-04-25T10:11:25.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.803+0000 7fc452000640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc43c06b4d0 0x7fc43c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.803+0000 7fc452000640 1 -- 172.21.15.119:0/3767236884 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc448089be0 con 0x7fc45c0d3b90 2024-04-25T10:11:25.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.804+0000 7fc459e00640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc43c06b4d0 0x7fc43c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.804+0000 7fc459e00640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc43c06b4d0 0x7fc43c06d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0015e0 tx=0x7fc44c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.807+0000 7fc452000640 1 -- 172.21.15.119:0/3767236884 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc448058df0 con 0x7fc45c0d3b90 2024-04-25T10:11:26.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:25.999+0000 7fc460e00640 1 -- 172.21.15.119:0/3767236884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc45c0cdb90 con 0x7fc45c0d3b90 2024-04-25T10:11:26.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.000+0000 7fc452000640 1 -- 172.21.15.119:0/3767236884 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4480586f0 con 0x7fc45c0d3b90 2024-04-25T10:11:26.006 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:11:26.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.003+0000 7fc460e00640 1 -- 172.21.15.119:0/3767236884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc43c06b4d0 msgr2=0x7fc43c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.003+0000 7fc460e00640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc43c06b4d0 0x7fc43c06d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0015e0 tx=0x7fc44c009040 comp rx=0 tx=0).stop 2024-04-25T10:11:26.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.003+0000 7fc460e00640 1 -- 172.21.15.119:0/3767236884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc45c0d3b90 msgr2=0x7fc45c0aa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.003+0000 7fc460e00640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc45c0d3b90 0x7fc45c0aa480 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc44800b9f0 tx=0x7fc44800beb0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.003+0000 7fc460e00640 1 -- 172.21.15.119:0/3767236884 shutdown_connections 2024-04-25T10:11:26.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.003+0000 7fc460e00640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc43c06b4d0 0x7fc43c06d990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.003+0000 7fc460e00640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc45c15dca0 0x7fc45c0aa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.003+0000 7fc460e00640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc45c0d3b90 0x7fc45c0aa480 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.003+0000 7fc460e00640 1 --2- 172.21.15.119:0/3767236884 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc45c0d0f60 0x7fc45c152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.003+0000 7fc460e00640 1 -- 172.21.15.119:0/3767236884 >> 172.21.15.119:0/3767236884 conn(0x7fc45c0726c0 msgr2=0x7fc45c0b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:26.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.003+0000 7fc460e00640 1 -- 172.21.15.119:0/3767236884 shutdown_connections 2024-04-25T10:11:26.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.003+0000 7fc460e00640 1 -- 172.21.15.119:0/3767236884 wait complete. 2024-04-25T10:11:26.029 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T10:11:26.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.220+0000 7f41b7c00640 1 Processor -- start 2024-04-25T10:11:26.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.220+0000 7f41b7c00640 1 -- start start 2024-04-25T10:11:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.222+0000 7f41b7c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f41b015a870 0x7f41b00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.222+0000 7f41b7c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f41b00cfc40 0x7f41b015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.222+0000 7f41b7c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b00cdb10 con 0x7f41b006e2e0 2024-04-25T10:11:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.222+0000 7f41b7c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b00ce720 con 0x7f41b00cfc40 2024-04-25T10:11:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.222+0000 7f41b7c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b00cddf0 con 0x7f41b015a870 2024-04-25T10:11:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.222+0000 7f41b5c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f41b00cfc40 0x7f41b015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.222+0000 7f41b5c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f41b00cfc40 0x7f41b015ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59318/0 (socket says 172.21.15.119:59318) 2024-04-25T10:11:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.222+0000 7f41b5c00640 1 -- 172.21.15.119:0/1366851052 learned_addr learned my addr 172.21.15.119:0/1366851052 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.223+0000 7f41b5200640 1 --2- 172.21.15.119:0/1366851052 >> v2:172.21.15.132:3300/0 conn(0x7f41b015a870 0x7f41b00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.223+0000 7f41b5c00640 1 -- 172.21.15.119:0/1366851052 >> v2:172.21.15.132:3300/0 conn(0x7f41b015a870 msgr2=0x7f41b00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:26.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.223+0000 7f41b5c00640 1 --2- 172.21.15.119:0/1366851052 >> v2:172.21.15.132:3300/0 conn(0x7f41b015a870 0x7f41b00cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.223+0000 7f41b5c00640 1 -- 172.21.15.119:0/1366851052 >> v1:172.21.15.132:6789/0 conn(0x7f41b006e2e0 legacy=0x7f41b00cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:26.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.223+0000 7f41b5c00640 1 -- 172.21.15.119:0/1366851052 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b00ce0d0 con 0x7f41b00cfc40 2024-04-25T10:11:26.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.224+0000 7f41b5c00640 1 --2- 172.21.15.119:0/1366851052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41b00cfc40 0x7f41b015ac50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009e50 tx=0x7f41a802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b26609e7cc047a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.224+0000 7f41af400640 1 -- 172.21.15.119:0/1366851052 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a8036070 con 0x7f41b00cfc40 2024-04-25T10:11:26.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.224+0000 7f41af400640 1 -- 172.21.15.119:0/1366851052 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a8031600 con 0x7f41b00cfc40 2024-04-25T10:11:26.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.224+0000 7f41af400640 1 -- 172.21.15.119:0/1366851052 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a80319d0 con 0x7f41b00cfc40 2024-04-25T10:11:26.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.225+0000 7f41b7c00640 1 -- 172.21.15.119:0/1366851052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41b00cfc40 msgr2=0x7f41b015ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.225+0000 7f41b7c00640 1 --2- 172.21.15.119:0/1366851052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41b00cfc40 0x7f41b015ac50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009e50 tx=0x7f41a802a710 comp rx=0 tx=0).stop 2024-04-25T10:11:26.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.225+0000 7f41b7c00640 1 -- 172.21.15.119:0/1366851052 shutdown_connections 2024-04-25T10:11:26.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.225+0000 7f41b7c00640 1 --2- 172.21.15.119:0/1366851052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41b00cfc40 0x7f41b015ac50 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.225+0000 7f41b7c00640 1 --2- 172.21.15.119:0/1366851052 >> v2:172.21.15.132:3300/0 conn(0x7f41b015a870 0x7f41b00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.225+0000 7f41b7c00640 1 -- 172.21.15.119:0/1366851052 >> 172.21.15.119:0/1366851052 conn(0x7f41b006aba0 msgr2=0x7f41b006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:26.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.225+0000 7f41b7c00640 1 -- 172.21.15.119:0/1366851052 shutdown_connections 2024-04-25T10:11:26.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.225+0000 7f41b7c00640 1 -- 172.21.15.119:0/1366851052 wait complete. 2024-04-25T10:11:26.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.227+0000 7f41b7c00640 1 Processor -- start 2024-04-25T10:11:26.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.227+0000 7f41b7c00640 1 -- start start 2024-04-25T10:11:26.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.228+0000 7f41b7c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b006e2e0 0x7f41b014a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.228+0000 7f41b7c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41b00cfc40 0x7f41b00a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.228+0000 7f41b7c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b015a870 0x7f41b00aa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.228+0000 7f41b7c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b00cd700 con 0x7f41b00cfc40 2024-04-25T10:11:26.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.228+0000 7f41b7c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b00cd580 con 0x7f41b006e2e0 2024-04-25T10:11:26.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.228+0000 7f41b7c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b00cd880 con 0x7f41b015a870 2024-04-25T10:11:26.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.228+0000 7f41afe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41b00cfc40 0x7f41b00a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.228+0000 7f41afe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41b00cfc40 0x7f41b00a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59328/0 (socket says 172.21.15.119:59328) 2024-04-25T10:11:26.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.228+0000 7f41afe00640 1 -- 172.21.15.119:0/2684017119 learned_addr learned my addr 172.21.15.119:0/2684017119 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:26.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.229+0000 7f41b5200640 1 --2- 172.21.15.119:0/2684017119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b006e2e0 0x7f41b014a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.229+0000 7f41b5c00640 1 --2- 172.21.15.119:0/2684017119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b015a870 0x7f41b00aa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.229+0000 7f41afe00640 1 -- 172.21.15.119:0/2684017119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b015a870 msgr2=0x7f41b00aa0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.229+0000 7f41afe00640 1 --2- 172.21.15.119:0/2684017119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b015a870 0x7f41b00aa0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.229+0000 7f41afe00640 1 -- 172.21.15.119:0/2684017119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b006e2e0 msgr2=0x7f41b014a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.229+0000 7f41afe00640 1 --2- 172.21.15.119:0/2684017119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b006e2e0 0x7f41b014a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.229+0000 7f41afe00640 1 -- 172.21.15.119:0/2684017119 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b00a8f60 con 0x7f41b00cfc40 2024-04-25T10:11:26.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.229+0000 7f41afe00640 1 --2- 172.21.15.119:0/2684017119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41b00cfc40 0x7f41b00a84a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f419c00c460 tx=0x7f419c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.229+0000 7f41acc00640 1 -- 172.21.15.119:0/2684017119 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c016020 con 0x7f41b00cfc40 2024-04-25T10:11:26.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.229+0000 7f41acc00640 1 -- 172.21.15.119:0/2684017119 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419c012870 con 0x7f41b00cfc40 2024-04-25T10:11:26.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.229+0000 7f41acc00640 1 -- 172.21.15.119:0/2684017119 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c012c60 con 0x7f41b00cfc40 2024-04-25T10:11:26.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.229+0000 7f41b7c00640 1 -- 172.21.15.119:0/2684017119 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b00a9240 con 0x7f41b00cfc40 2024-04-25T10:11:26.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.229+0000 7f41b7c00640 1 -- 172.21.15.119:0/2684017119 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b00a9800 con 0x7f41b00cfc40 2024-04-25T10:11:26.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.231+0000 7f41b7c00640 1 -- 172.21.15.119:0/2684017119 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b015d3d0 con 0x7f41b00cfc40 2024-04-25T10:11:26.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.231+0000 7f41acc00640 1 -- 172.21.15.119:0/2684017119 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f419c019020 con 0x7f41b00cfc40 2024-04-25T10:11:26.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.232+0000 7f41acc00640 1 --2- 172.21.15.119:0/2684017119 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f419006b4d0 0x7f419006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.233+0000 7f41acc00640 1 -- 172.21.15.119:0/2684017119 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f419c089c30 con 0x7f41b00cfc40 2024-04-25T10:11:26.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.233+0000 7f41b5200640 1 --2- 172.21.15.119:0/2684017119 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f419006b4d0 0x7f419006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.233+0000 7f41b5200640 1 --2- 172.21.15.119:0/2684017119 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f419006b4d0 0x7f419006d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f41a400b7c0 tx=0x7f41a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.237+0000 7f41acc00640 1 -- 172.21.15.119:0/2684017119 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41b015d3d0 con 0x7f41b00cfc40 2024-04-25T10:11:26.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:26.502+0000 7f41b7c00640 1 -- 172.21.15.119:0/2684017119 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7f41b014a640 con 0x7f41b00cfc40 2024-04-25T10:11:27.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.165+0000 7f41acc00640 1 -- 172.21.15.119:0/2684017119 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v229) v1 ==== 169+0+0 (secure 0 0 0) 0x7f419c020030 con 0x7f41b00cfc40 2024-04-25T10:11:27.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.168+0000 7f41b7c00640 1 -- 172.21.15.119:0/2684017119 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f419006b4d0 msgr2=0x7f419006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.168+0000 7f41b7c00640 1 --2- 172.21.15.119:0/2684017119 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f419006b4d0 0x7f419006d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f41a400b7c0 tx=0x7f41a4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:27.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.168+0000 7f41b7c00640 1 -- 172.21.15.119:0/2684017119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41b00cfc40 msgr2=0x7f41b00a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.169+0000 7f41b7c00640 1 --2- 172.21.15.119:0/2684017119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41b00cfc40 0x7f41b00a84a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f419c00c460 tx=0x7f419c00c920 comp rx=0 tx=0).stop 2024-04-25T10:11:27.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.169+0000 7f41b7c00640 1 -- 172.21.15.119:0/2684017119 shutdown_connections 2024-04-25T10:11:27.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.169+0000 7f41b7c00640 1 --2- 172.21.15.119:0/2684017119 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f419006b4d0 0x7f419006d990 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.169+0000 7f41b7c00640 1 --2- 172.21.15.119:0/2684017119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41b015a870 0x7f41b00aa0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.169+0000 7f41b7c00640 1 --2- 172.21.15.119:0/2684017119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41b00cfc40 0x7f41b00a84a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.169+0000 7f41b7c00640 1 --2- 172.21.15.119:0/2684017119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b006e2e0 0x7f41b014a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.169+0000 7f41b7c00640 1 -- 172.21.15.119:0/2684017119 >> 172.21.15.119:0/2684017119 conn(0x7f41b006aba0 msgr2=0x7f41b006d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:27.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.169+0000 7f41b7c00640 1 -- 172.21.15.119:0/2684017119 shutdown_connections 2024-04-25T10:11:27.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.169+0000 7f41b7c00640 1 -- 172.21.15.119:0/2684017119 wait complete. 2024-04-25T10:11:27.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:11:27.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.395+0000 7f83e5c00640 1 Processor -- start 2024-04-25T10:11:27.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.396+0000 7f83e5c00640 1 -- start start 2024-04-25T10:11:27.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.397+0000 7f83e5c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83e01450e0 0x7f83e006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.397+0000 7f83e5c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f83e006b0e0 0x7f83e00cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.397+0000 7f83e5c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e015f400 con 0x7f83e00cc810 2024-04-25T10:11:27.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.397+0000 7f83e5c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e01454c0 con 0x7f83e006b0e0 2024-04-25T10:11:27.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.397+0000 7f83e5c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e006b540 con 0x7f83e01450e0 2024-04-25T10:11:27.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.398+0000 7f83e5200640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f83e00cc810 0x7f83e0071050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39210/0 (socket says 172.21.15.119:39210) 2024-04-25T10:11:27.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.398+0000 7f83e5200640 1 -- 172.21.15.119:0/2306978744 learned_addr learned my addr 172.21.15.119:0/2306978744 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:27.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.398+0000 7f83dfe00640 1 --2- 172.21.15.119:0/2306978744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83e01450e0 0x7f83e006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.398+0000 7f83d7400640 1 --2- 172.21.15.119:0/2306978744 >> v2:172.21.15.132:3300/0 conn(0x7f83e006b0e0 0x7f83e00cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.398+0000 7f83df400640 1 -- 172.21.15.119:0/2306978744 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3505936454 0 0) 0x7f83e015f400 con 0x7f83e00cc810 2024-04-25T10:11:27.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.398+0000 7f83df400640 1 -- 172.21.15.119:0/2306978744 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83c4003520 con 0x7f83e00cc810 2024-04-25T10:11:27.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83df400640 1 -- 172.21.15.119:0/2306978744 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 905208488 0 0) 0x7f83c4003520 con 0x7f83e00cc810 2024-04-25T10:11:27.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83df400640 1 -- 172.21.15.119:0/2306978744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83e01450e0 msgr2=0x7f83e006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:27.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83df400640 1 --2- 172.21.15.119:0/2306978744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83e01450e0 0x7f83e006aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83df400640 1 -- 172.21.15.119:0/2306978744 >> v2:172.21.15.132:3300/0 conn(0x7f83e006b0e0 msgr2=0x7f83e00cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:27.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83df400640 1 --2- 172.21.15.119:0/2306978744 >> v2:172.21.15.132:3300/0 conn(0x7f83e006b0e0 0x7f83e00cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83df400640 1 -- 172.21.15.119:0/2306978744 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e015f6e0 con 0x7f83e00cc810 2024-04-25T10:11:27.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83df400640 1 -- 172.21.15.119:0/2306978744 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f83d80019f0 con 0x7f83e00cc810 2024-04-25T10:11:27.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83df400640 1 -- 172.21.15.119:0/2306978744 >> v1:172.21.15.119:6789/0 conn(0x7f83e00cc810 legacy=0x7f83e0071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83df400640 1 --2- 172.21.15.119:0/2306978744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83c40039b0 0x7f83c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83df400640 1 --2- 172.21.15.119:0/2306978744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c4004800 0x7f83c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83df400640 1 --2- 172.21.15.119:0/2306978744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83e01450e0 0x7f83c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83df400640 1 -- 172.21.15.119:0/2306978744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c400ba70 con 0x7f83c4004800 2024-04-25T10:11:27.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83df400640 1 -- 172.21.15.119:0/2306978744 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c400bbf0 con 0x7f83c40039b0 2024-04-25T10:11:27.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83df400640 1 -- 172.21.15.119:0/2306978744 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c400bda0 con 0x7f83e01450e0 2024-04-25T10:11:27.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83d7400640 1 --2- 172.21.15.119:0/2306978744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c4004800 0x7f83c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83d7400640 1 -- 172.21.15.119:0/2306978744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83e01450e0 msgr2=0x7f83c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:27.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83d7400640 1 --2- 172.21.15.119:0/2306978744 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83e01450e0 0x7f83c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83d7400640 1 -- 172.21.15.119:0/2306978744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83c40039b0 msgr2=0x7f83c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:27.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83d7400640 1 --2- 172.21.15.119:0/2306978744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83c40039b0 0x7f83c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.399+0000 7f83d7400640 1 -- 172.21.15.119:0/2306978744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c4003520 con 0x7f83c4004800 2024-04-25T10:11:27.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.401+0000 7f83d7400640 1 --2- 172.21.15.119:0/2306978744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c4004800 0x7f83c4004c80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009a90 tx=0x7f83c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837e9c28cc842fa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.401+0000 7f83df400640 1 -- 172.21.15.119:0/2306978744 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c8036070 con 0x7f83c4004800 2024-04-25T10:11:27.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.401+0000 7f83df400640 1 -- 172.21.15.119:0/2306978744 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c8031400 con 0x7f83c4004800 2024-04-25T10:11:27.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.401+0000 7f83df400640 1 -- 172.21.15.119:0/2306978744 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c8009690 con 0x7f83c4004800 2024-04-25T10:11:27.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.401+0000 7f83e5c00640 1 -- 172.21.15.119:0/2306978744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c4004800 msgr2=0x7f83c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.401+0000 7f83e5c00640 1 --2- 172.21.15.119:0/2306978744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c4004800 0x7f83c4004c80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009a90 tx=0x7f83c4005830 comp rx=0 tx=0).stop 2024-04-25T10:11:27.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.401+0000 7f83e5200640 1 -- 172.21.15.119:0/2306978744 reap_dead start 2024-04-25T10:11:27.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.402+0000 7f83e5c00640 1 -- 172.21.15.119:0/2306978744 shutdown_connections 2024-04-25T10:11:27.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.402+0000 7f83e5c00640 1 -- 172.21.15.119:0/2306978744 >> 172.21.15.119:0/2306978744 conn(0x7f83e0066ff0 msgr2=0x7f83e00b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:27.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.402+0000 7f83e5c00640 1 -- 172.21.15.119:0/2306978744 shutdown_connections 2024-04-25T10:11:27.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.402+0000 7f83e5c00640 1 -- 172.21.15.119:0/2306978744 wait complete. 2024-04-25T10:11:27.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.403+0000 7f83e5c00640 1 Processor -- start 2024-04-25T10:11:27.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.404+0000 7f83e5c00640 1 -- start start 2024-04-25T10:11:27.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.404+0000 7f83e5c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c4004800 0x7f83e00adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.404+0000 7f83e5c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83e006d080 0x7f83e00c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.405+0000 7f83e5c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83e00c8c50 0x7f83e015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.405+0000 7f83e5c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e00ccca0 con 0x7f83c4004800 2024-04-25T10:11:27.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.405+0000 7f83e5c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e015fca0 con 0x7f83e006d080 2024-04-25T10:11:27.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.405+0000 7f83e5c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e006da60 con 0x7f83e00c8c50 2024-04-25T10:11:27.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.405+0000 7f83dfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c4004800 0x7f83e00adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.405+0000 7f83dfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c4004800 0x7f83e00adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59346/0 (socket says 172.21.15.119:59346) 2024-04-25T10:11:27.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.405+0000 7f83dfe00640 1 -- 172.21.15.119:0/3534763318 learned_addr learned my addr 172.21.15.119:0/3534763318 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:27.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.405+0000 7f83dfe00640 1 -- 172.21.15.119:0/3534763318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83e00c8c50 msgr2=0x7f83e015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.405+0000 7f83dfe00640 1 --2- 172.21.15.119:0/3534763318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83e00c8c50 0x7f83e015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.405+0000 7f83dfe00640 1 -- 172.21.15.119:0/3534763318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83e006d080 msgr2=0x7f83e00c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.405+0000 7f83dfe00640 1 --2- 172.21.15.119:0/3534763318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83e006d080 0x7f83e00c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.405+0000 7f83dfe00640 1 -- 172.21.15.119:0/3534763318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e006dd40 con 0x7f83c4004800 2024-04-25T10:11:27.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.405+0000 7f83dfe00640 1 --2- 172.21.15.119:0/3534763318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c4004800 0x7f83e00adfc0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f83d000e7a0 tx=0x7f83d000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.406+0000 7f83dcc00640 1 -- 172.21.15.119:0/3534763318 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0017070 con 0x7f83c4004800 2024-04-25T10:11:27.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.406+0000 7f83dcc00640 1 -- 172.21.15.119:0/3534763318 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d000c940 con 0x7f83c4004800 2024-04-25T10:11:27.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.406+0000 7f83e5c00640 1 -- 172.21.15.119:0/3534763318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e006e020 con 0x7f83c4004800 2024-04-25T10:11:27.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.406+0000 7f83dcc00640 1 -- 172.21.15.119:0/3534763318 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d000cd10 con 0x7f83c4004800 2024-04-25T10:11:27.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.406+0000 7f83e5c00640 1 -- 172.21.15.119:0/3534763318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e00c4c10 con 0x7f83c4004800 2024-04-25T10:11:27.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.407+0000 7f83e5c00640 1 -- 172.21.15.119:0/3534763318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e00749b0 con 0x7f83c4004800 2024-04-25T10:11:27.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.408+0000 7f83dcc00640 1 -- 172.21.15.119:0/3534763318 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83d001a070 con 0x7f83c4004800 2024-04-25T10:11:27.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.409+0000 7f83dcc00640 1 --2- 172.21.15.119:0/3534763318 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f83bc06b4d0 0x7f83bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.409+0000 7f83dcc00640 1 -- 172.21.15.119:0/3534763318 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f83d0089db0 con 0x7f83c4004800 2024-04-25T10:11:27.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.409+0000 7f83d7400640 1 --2- 172.21.15.119:0/3534763318 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f83bc06b4d0 0x7f83bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.410+0000 7f83d7400640 1 --2- 172.21.15.119:0/3534763318 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f83bc06b4d0 0x7f83bc06d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f83c8033000 tx=0x7f83c802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.413+0000 7f83dcc00640 1 -- 172.21.15.119:0/3534763318 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83d008e070 con 0x7f83c4004800 2024-04-25T10:11:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.614+0000 7f83e5c00640 1 -- 172.21.15.119:0/3534763318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f83e00af410 con 0x7f83c4004800 2024-04-25T10:11:27.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.621+0000 7f83dcc00640 1 -- 172.21.15.119:0/3534763318 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f83d0058e90 con 0x7f83c4004800 2024-04-25T10:11:27.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.624+0000 7f83e5c00640 1 -- 172.21.15.119:0/3534763318 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f83bc06b4d0 msgr2=0x7f83bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.624+0000 7f83e5c00640 1 --2- 172.21.15.119:0/3534763318 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f83bc06b4d0 0x7f83bc06d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f83c8033000 tx=0x7f83c802e000 comp rx=0 tx=0).stop 2024-04-25T10:11:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.624+0000 7f83e5c00640 1 -- 172.21.15.119:0/3534763318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c4004800 msgr2=0x7f83e00adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.624+0000 7f83e5c00640 1 --2- 172.21.15.119:0/3534763318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c4004800 0x7f83e00adfc0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f83d000e7a0 tx=0x7f83d000ec60 comp rx=0 tx=0).stop 2024-04-25T10:11:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.624+0000 7f83e5c00640 1 -- 172.21.15.119:0/3534763318 shutdown_connections 2024-04-25T10:11:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.624+0000 7f83e5c00640 1 --2- 172.21.15.119:0/3534763318 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f83bc06b4d0 0x7f83bc06d990 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.624+0000 7f83e5c00640 1 --2- 172.21.15.119:0/3534763318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f83e00c8c50 0x7f83e015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.624+0000 7f83e5c00640 1 --2- 172.21.15.119:0/3534763318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83e006d080 0x7f83e00c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.624+0000 7f83e5c00640 1 --2- 172.21.15.119:0/3534763318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83c4004800 0x7f83e00adfc0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.624+0000 7f83e5c00640 1 -- 172.21.15.119:0/3534763318 >> 172.21.15.119:0/3534763318 conn(0x7f83e0066ff0 msgr2=0x7f83e0144180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.624+0000 7f83e5c00640 1 -- 172.21.15.119:0/3534763318 shutdown_connections 2024-04-25T10:11:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:27.624+0000 7f83e5c00640 1 -- 172.21.15.119:0/3534763318 wait complete. 2024-04-25T10:11:27.649 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:11:27.649 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:27.760 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:27.874 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:11:27.874 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:11:27.874 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:27.931 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:27.931 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:11:27.932 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:27.987 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:27.988 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:11:27.988 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:28.046 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:28.046 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:11:28.046 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:28.102 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:28.103 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:11:28.103 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.307+0000 7fa44a200640 1 Processor -- start 2024-04-25T10:11:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.307+0000 7fa44a200640 1 -- start start 2024-04-25T10:11:28.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.309+0000 7fa44a200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa44415c9f0 0x7fa4441664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.309+0000 7fa44a200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4440ce6c0 con 0x7fa44406e8f0 2024-04-25T10:11:28.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.309+0000 7fa44a200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4440cc4e0 con 0x7fa44415bf00 2024-04-25T10:11:28.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.309+0000 7fa44a200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4441455e0 con 0x7fa44415c9f0 2024-04-25T10:11:28.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.309+0000 7fa442a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa44406e8f0 0x7fa44415c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39224/0 (socket says 172.21.15.119:39224) 2024-04-25T10:11:28.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.309+0000 7fa442a00640 1 -- 172.21.15.119:0/923054807 learned_addr learned my addr 172.21.15.119:0/923054807 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 -- 172.21.15.119:0/923054807 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919734993 0 0) 0x7fa430001110 con 0x7fa44406e8f0 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa443e00640 1 --2- 172.21.15.119:0/923054807 >> v2:172.21.15.132:3300/0 conn(0x7fa44415c9f0 0x7fa4441664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 -- 172.21.15.119:0/923054807 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa428003520 con 0x7fa44406e8f0 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 -- 172.21.15.119:0/923054807 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1367084737 0 0) 0x7fa428003520 con 0x7fa44406e8f0 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 -- 172.21.15.119:0/923054807 >> v2:172.21.15.132:3300/0 conn(0x7fa44415c9f0 msgr2=0x7fa4441664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 --2- 172.21.15.119:0/923054807 >> v2:172.21.15.132:3300/0 conn(0x7fa44415c9f0 0x7fa4441664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 -- 172.21.15.119:0/923054807 >> v1:172.21.15.132:6789/0 conn(0x7fa44415bf00 legacy=0x7fa4440cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 -- 172.21.15.119:0/923054807 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4440cc7b0 con 0x7fa44406e8f0 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 -- 172.21.15.119:0/923054807 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fa430001840 con 0x7fa44406e8f0 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 -- 172.21.15.119:0/923054807 >> v1:172.21.15.119:6789/0 conn(0x7fa44406e8f0 legacy=0x7fa44415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 --2- 172.21.15.119:0/923054807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4280039b0 0x7fa428003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 --2- 172.21.15.119:0/923054807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa428004800 0x7fa428004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 --2- 172.21.15.119:0/923054807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4280051c0 0x7fa42800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 -- 172.21.15.119:0/923054807 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4441455e0 con 0x7fa4280051c0 2024-04-25T10:11:28.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 -- 172.21.15.119:0/923054807 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42800c060 con 0x7fa4280039b0 2024-04-25T10:11:28.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.310+0000 7fa442000640 1 -- 172.21.15.119:0/923054807 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42800c260 con 0x7fa428004800 2024-04-25T10:11:28.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.311+0000 7fa443e00640 1 --2- 172.21.15.119:0/923054807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4280051c0 0x7fa42800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.311+0000 7fa442a00640 1 --2- 172.21.15.119:0/923054807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa428004800 0x7fa428004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.311+0000 7fa443e00640 1 -- 172.21.15.119:0/923054807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa428004800 msgr2=0x7fa428004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.311+0000 7fa443e00640 1 --2- 172.21.15.119:0/923054807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa428004800 0x7fa428004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.311+0000 7fa443e00640 1 -- 172.21.15.119:0/923054807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4280039b0 msgr2=0x7fa428003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.311+0000 7fa443e00640 1 --2- 172.21.15.119:0/923054807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4280039b0 0x7fa428003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.311+0000 7fa443e00640 1 -- 172.21.15.119:0/923054807 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa428003520 con 0x7fa4280051c0 2024-04-25T10:11:28.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.311+0000 7fa442a00640 1 --2- 172.21.15.119:0/923054807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa428004800 0x7fa428004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:28.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.312+0000 7fa443e00640 1 -- 172.21.15.119:0/923054807 reap_dead start 2024-04-25T10:11:28.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.312+0000 7fa443e00640 1 --2- 172.21.15.119:0/923054807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4280051c0 0x7fa42800b9c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009fc0 tx=0x7fa428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1aa460a1d340a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.312+0000 7fa442000640 1 -- 172.21.15.119:0/923054807 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c035020 con 0x7fa4280051c0 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.312+0000 7fa442000640 1 -- 172.21.15.119:0/923054807 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43c003b90 con 0x7fa4280051c0 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.312+0000 7fa442000640 1 -- 172.21.15.119:0/923054807 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c009c10 con 0x7fa4280051c0 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.313+0000 7fa44a200640 1 -- 172.21.15.119:0/923054807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4280051c0 msgr2=0x7fa42800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.313+0000 7fa44a200640 1 --2- 172.21.15.119:0/923054807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4280051c0 0x7fa42800b9c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009fc0 tx=0x7fa428005830 comp rx=0 tx=0).stop 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.313+0000 7fa44a200640 1 -- 172.21.15.119:0/923054807 shutdown_connections 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.313+0000 7fa44a200640 1 --2- 172.21.15.119:0/923054807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4280051c0 0x7fa42800b9c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.313+0000 7fa44a200640 1 -- 172.21.15.119:0/923054807 >> 172.21.15.119:0/923054807 conn(0x7fa44406aba0 msgr2=0x7fa44406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.313+0000 7fa44a200640 1 -- 172.21.15.119:0/923054807 shutdown_connections 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.313+0000 7fa44a200640 1 -- 172.21.15.119:0/923054807 wait complete. 2024-04-25T10:11:28.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.315+0000 7fa44a200640 1 Processor -- start 2024-04-25T10:11:28.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.315+0000 7fa44a200640 1 -- start start 2024-04-25T10:11:28.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.316+0000 7fa44a200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4280051c0 0x7fa44413bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.316+0000 7fa44a200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa444140aa0 0x7fa44413cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.316+0000 7fa44a200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44413d1c0 0x7fa444143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.316+0000 7fa44a200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4440ccd70 con 0x7fa444140aa0 2024-04-25T10:11:28.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.316+0000 7fa44a200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44406d450 con 0x7fa4280051c0 2024-04-25T10:11:28.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.316+0000 7fa44a200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4440cd050 con 0x7fa44413d1c0 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.316+0000 7fa442a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa444140aa0 0x7fa44413cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.316+0000 7fa442a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa444140aa0 0x7fa44413cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59370/0 (socket says 172.21.15.119:59370) 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.316+0000 7fa442a00640 1 -- 172.21.15.119:0/720907186 learned_addr learned my addr 172.21.15.119:0/720907186 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.317+0000 7fa443e00640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44413d1c0 0x7fa444143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.317+0000 7fa443400640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4280051c0 0x7fa44413bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.317+0000 7fa442a00640 1 -- 172.21.15.119:0/720907186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44413d1c0 msgr2=0x7fa444143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.317+0000 7fa442a00640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44413d1c0 0x7fa444143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.317+0000 7fa442a00640 1 -- 172.21.15.119:0/720907186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4280051c0 msgr2=0x7fa44413bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.317+0000 7fa442a00640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4280051c0 0x7fa44413bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.317+0000 7fa442a00640 1 -- 172.21.15.119:0/720907186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44413c290 con 0x7fa444140aa0 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.317+0000 7fa443e00640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44413d1c0 0x7fa444143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.317+0000 7fa442a00640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa444140aa0 0x7fa44413cc80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa43000d850 tx=0x7fa43000dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.317+0000 7fa43b400640 1 -- 172.21.15.119:0/720907186 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa430019070 con 0x7fa444140aa0 2024-04-25T10:11:28.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.318+0000 7fa43b400640 1 -- 172.21.15.119:0/720907186 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4300074e0 con 0x7fa444140aa0 2024-04-25T10:11:28.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.318+0000 7fa43b400640 1 -- 172.21.15.119:0/720907186 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4300049e0 con 0x7fa444140aa0 2024-04-25T10:11:28.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.318+0000 7fa44a200640 1 -- 172.21.15.119:0/720907186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44413fa20 con 0x7fa444140aa0 2024-04-25T10:11:28.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.318+0000 7fa44a200640 1 -- 172.21.15.119:0/720907186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44413ffe0 con 0x7fa444140aa0 2024-04-25T10:11:28.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.319+0000 7fa44a200640 1 -- 172.21.15.119:0/720907186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44406bb30 con 0x7fa444140aa0 2024-04-25T10:11:28.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.320+0000 7fa43b400640 1 -- 172.21.15.119:0/720907186 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa43001c070 con 0x7fa444140aa0 2024-04-25T10:11:28.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.321+0000 7fa43b400640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa42406b4a0 0x7fa42406d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.321+0000 7fa43b400640 1 -- 172.21.15.119:0/720907186 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa430089720 con 0x7fa444140aa0 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.321+0000 7fa443400640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa42406b4a0 0x7fa42406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.322+0000 7fa443400640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa42406b4a0 0x7fa42406d960 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa434007fd0 tx=0x7fa434009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.325+0000 7fa43b400640 1 -- 172.21.15.119:0/720907186 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa44413c290 con 0x7fa444140aa0 2024-04-25T10:11:28.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.556+0000 7fa44a200640 1 -- 172.21.15.119:0/720907186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4440749b0 con 0x7fa444140aa0 2024-04-25T10:11:28.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.557+0000 7fa43b400640 1 -- 172.21.15.119:0/720907186 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3711 (secure 0 0 0) 0x7fa4440ce6c0 con 0x7fa444140aa0 2024-04-25T10:11:28.563 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:28.563 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/4159222235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":4159222235},{"type":"v1","addr":"172.21.15.132:6834","nonce":4159222235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/4223361610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":4223361610},{"type":"v1","addr":"172.21.15.119:6837","nonce":4223361610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5119,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2367324442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2367324442},{"type":"v1","addr":"172.21.15.132:6835","nonce":2367324442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6399,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1193489991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1193489991},{"type":"v1","addr":"172.21.15.119:6836","nonce":1193489991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[]} 2024-04-25T10:11:28.563 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 38 2024-04-25T10:11:28.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.560+0000 7fa44a200640 1 -- 172.21.15.119:0/720907186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa42406b4a0 msgr2=0x7fa42406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.560+0000 7fa44a200640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa42406b4a0 0x7fa42406d960 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa434007fd0 tx=0x7fa434009040 comp rx=0 tx=0).stop 2024-04-25T10:11:28.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.560+0000 7fa44a200640 1 -- 172.21.15.119:0/720907186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa444140aa0 msgr2=0x7fa44413cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.560+0000 7fa44a200640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa444140aa0 0x7fa44413cc80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa43000d850 tx=0x7fa43000dc50 comp rx=0 tx=0).stop 2024-04-25T10:11:28.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.560+0000 7fa44a200640 1 -- 172.21.15.119:0/720907186 shutdown_connections 2024-04-25T10:11:28.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.560+0000 7fa44a200640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa42406b4a0 0x7fa42406d960 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.560+0000 7fa44a200640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa44413d1c0 0x7fa444143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.560+0000 7fa44a200640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa444140aa0 0x7fa44413cc80 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.560+0000 7fa44a200640 1 --2- 172.21.15.119:0/720907186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4280051c0 0x7fa44413bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.560+0000 7fa44a200640 1 -- 172.21.15.119:0/720907186 >> 172.21.15.119:0/720907186 conn(0x7fa44406aba0 msgr2=0x7fa44415e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:28.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.560+0000 7fa44a200640 1 -- 172.21.15.119:0/720907186 shutdown_connections 2024-04-25T10:11:28.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.560+0000 7fa44a200640 1 -- 172.21.15.119:0/720907186 wait complete. 2024-04-25T10:11:28.586 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:11:28.586 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:11:28.586 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:11:28.586 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:28.586 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:11:28.586 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:11:28.587 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:11:28.587 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:28.587 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:11:28.587 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:11:28.587 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:11:28.587 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:28.587 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:11:28.587 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:11:28.588 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:11:28.588 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:28.607 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:11:28.607 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:11:28.608 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:11:28.608 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:11:28.613 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:11:28.614 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:11:28.614 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:11:28.615 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:11:28.653 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:11:28.653 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:28.656 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:11:28.656 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:28.659 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:11:28.659 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:28.664 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:11:28.665 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:28.665 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:11:28.797 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:11:28.800 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2024-04-25T10:11:28.811 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-04-25T10:11:28.816 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-04-25T10:11:28.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.838+0000 7f949e600640 1 Processor -- start 2024-04-25T10:11:28.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.838+0000 7f949e600640 1 -- start start 2024-04-25T10:11:28.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.839+0000 7f949e600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9498161090 0x7f9498161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.839+0000 7f949e600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f94980b9720 0x7f94980ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.839+0000 7f949e600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94980cece0 0x7f949806c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.839+0000 7f949e600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94980cf130 con 0x7f94980b9720 2024-04-25T10:11:28.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.839+0000 7f949e600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94980d21a0 con 0x7f9498161090 2024-04-25T10:11:28.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.839+0000 7f949e600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9498161f80 con 0x7f94980cece0 2024-04-25T10:11:28.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.839+0000 7f9496a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f94980b9720 0x7f94980ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.839+0000 7f9496a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f94980b9720 0x7f94980ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59408/0 (socket says 172.21.15.119:59408) 2024-04-25T10:11:28.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.839+0000 7f9496a00640 1 -- 172.21.15.119:0/3783190732 learned_addr learned my addr 172.21.15.119:0/3783190732 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.840+0000 7f9497e00640 1 --2- 172.21.15.119:0/3783190732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94980cece0 0x7f949806c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.840+0000 7f9497400640 1 --2- 172.21.15.119:0/3783190732 >> v2:172.21.15.132:3300/0 conn(0x7f9498161090 0x7f9498161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.840+0000 7f9496a00640 1 -- 172.21.15.119:0/3783190732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94980cece0 msgr2=0x7f949806c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.840+0000 7f9496a00640 1 --2- 172.21.15.119:0/3783190732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94980cece0 0x7f949806c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.840+0000 7f9496a00640 1 -- 172.21.15.119:0/3783190732 >> v2:172.21.15.132:3300/0 conn(0x7f9498161090 msgr2=0x7f9498161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.840+0000 7f9496a00640 1 --2- 172.21.15.119:0/3783190732 >> v2:172.21.15.132:3300/0 conn(0x7f9498161090 0x7f9498161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.840+0000 7f9496a00640 1 -- 172.21.15.119:0/3783190732 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9498162260 con 0x7f94980b9720 2024-04-25T10:11:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.840+0000 7f9497e00640 1 --2- 172.21.15.119:0/3783190732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94980cece0 0x7f949806c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.840+0000 7f9497400640 1 --2- 172.21.15.119:0/3783190732 >> v2:172.21.15.132:3300/0 conn(0x7f9498161090 0x7f9498161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.841+0000 7f9496a00640 1 --2- 172.21.15.119:0/3783190732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94980b9720 0x7f94980ce7a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f94980d1210 tx=0x7f948402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4fc2ed2bcea044 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.841+0000 7f9496000640 1 -- 172.21.15.119:0/3783190732 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484037070 con 0x7f94980b9720 2024-04-25T10:11:28.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.841+0000 7f9496000640 1 -- 172.21.15.119:0/3783190732 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9484032440 con 0x7f94980b9720 2024-04-25T10:11:28.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.841+0000 7f9496000640 1 -- 172.21.15.119:0/3783190732 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484032810 con 0x7f94980b9720 2024-04-25T10:11:28.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.842+0000 7f949e600640 1 -- 172.21.15.119:0/3783190732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94980b9720 msgr2=0x7f94980ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.842+0000 7f949e600640 1 --2- 172.21.15.119:0/3783190732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94980b9720 0x7f94980ce7a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f94980d1210 tx=0x7f948402a140 comp rx=0 tx=0).stop 2024-04-25T10:11:28.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.842+0000 7f949e600640 1 -- 172.21.15.119:0/3783190732 shutdown_connections 2024-04-25T10:11:28.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.842+0000 7f949e600640 1 --2- 172.21.15.119:0/3783190732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94980cece0 0x7f949806c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.842+0000 7f949e600640 1 --2- 172.21.15.119:0/3783190732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94980b9720 0x7f94980ce7a0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.842+0000 7f949e600640 1 --2- 172.21.15.119:0/3783190732 >> v2:172.21.15.132:3300/0 conn(0x7f9498161090 0x7f9498161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.842+0000 7f949e600640 1 -- 172.21.15.119:0/3783190732 >> 172.21.15.119:0/3783190732 conn(0x7f94980b55e0 msgr2=0x7f94980b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:28.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.842+0000 7f949e600640 1 -- 172.21.15.119:0/3783190732 shutdown_connections 2024-04-25T10:11:28.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.842+0000 7f949e600640 1 -- 172.21.15.119:0/3783190732 wait complete. 2024-04-25T10:11:28.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.844+0000 7f949e600640 1 Processor -- start 2024-04-25T10:11:28.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.844+0000 7f949e600640 1 -- start start 2024-04-25T10:11:28.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f949e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94980cece0 0x7f949814bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f949e600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9498161090 0x7f949815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f949e600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949815f220 0x7f94980c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f949e600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9498161470 con 0x7f94980cece0 2024-04-25T10:11:28.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f949e600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94980d1e30 con 0x7f9498161090 2024-04-25T10:11:28.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f949e600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9498161c70 con 0x7f949815f220 2024-04-25T10:11:28.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f9497400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94980cece0 0x7f949814bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f9497400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94980cece0 0x7f949814bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59418/0 (socket says 172.21.15.119:59418) 2024-04-25T10:11:28.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f9497400640 1 -- 172.21.15.119:0/4151689392 learned_addr learned my addr 172.21.15.119:0/4151689392 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:28.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f9497e00640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949815f220 0x7f94980c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f9496a00640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9498161090 0x7f949815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f9497400640 1 -- 172.21.15.119:0/4151689392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949815f220 msgr2=0x7f94980c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f9497400640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949815f220 0x7f94980c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f9497400640 1 -- 172.21.15.119:0/4151689392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9498161090 msgr2=0x7f949815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f9497400640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9498161090 0x7f949815ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f9497400640 1 -- 172.21.15.119:0/4151689392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949815d370 con 0x7f94980cece0 2024-04-25T10:11:28.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.845+0000 7f9497e00640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949815f220 0x7f94980c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:28.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.846+0000 7f9497400640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94980cece0 0x7f949814bf60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f948800c850 tx=0x7f948800cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.846+0000 7f948f400640 1 -- 172.21.15.119:0/4151689392 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488015070 con 0x7f94980cece0 2024-04-25T10:11:28.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.846+0000 7f948f400640 1 -- 172.21.15.119:0/4151689392 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9488007830 con 0x7f94980cece0 2024-04-25T10:11:28.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.846+0000 7f949e600640 1 -- 172.21.15.119:0/4151689392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949815d650 con 0x7f94980cece0 2024-04-25T10:11:28.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.846+0000 7f948f400640 1 -- 172.21.15.119:0/4151689392 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488007be0 con 0x7f94980cece0 2024-04-25T10:11:28.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.846+0000 7f949e600640 1 -- 172.21.15.119:0/4151689392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949815dc10 con 0x7f94980cece0 2024-04-25T10:11:28.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.847+0000 7f949e600640 1 -- 172.21.15.119:0/4151689392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94980ca860 con 0x7f94980cece0 2024-04-25T10:11:28.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.848+0000 7f948f400640 1 -- 172.21.15.119:0/4151689392 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9488018070 con 0x7f94980cece0 2024-04-25T10:11:28.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.849+0000 7f948f400640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f947806b500 0x7f947806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.849+0000 7f948f400640 1 -- 172.21.15.119:0/4151689392 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9488089df0 con 0x7f94980cece0 2024-04-25T10:11:28.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.849+0000 7f9496a00640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f947806b500 0x7f947806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.850+0000 7f9496a00640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f947806b500 0x7f947806d9c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9484034000 tx=0x7f948402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:28.853+0000 7f948f400640 1 -- 172.21.15.119:0/4151689392 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94980ca860 con 0x7f94980cece0 2024-04-25T10:11:29.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:29.058+0000 7f949e600640 1 -- 172.21.15.119:0/4151689392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f94980750b0 con 0x7f94980cece0 2024-04-25T10:11:30.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.206+0000 7f948f400640 1 -- 172.21.15.119:0/4151689392 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) v1 ==== 97+0+0 (secure 0 0 0) 0x7f94880587f0 con 0x7f94980cece0 2024-04-25T10:11:30.212 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2024-04-25T10:11:30.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.209+0000 7f949e600640 1 -- 172.21.15.119:0/4151689392 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f947806b500 msgr2=0x7f947806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.209+0000 7f949e600640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f947806b500 0x7f947806d9c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9484034000 tx=0x7f948402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:30.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.209+0000 7f949e600640 1 -- 172.21.15.119:0/4151689392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94980cece0 msgr2=0x7f949814bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.210+0000 7f949e600640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94980cece0 0x7f949814bf60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f948800c850 tx=0x7f948800cd10 comp rx=0 tx=0).stop 2024-04-25T10:11:30.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.210+0000 7f949e600640 1 -- 172.21.15.119:0/4151689392 shutdown_connections 2024-04-25T10:11:30.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.210+0000 7f949e600640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f947806b500 0x7f947806d9c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.210+0000 7f949e600640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949815f220 0x7f94980c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.210+0000 7f949e600640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9498161090 0x7f949815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.210+0000 7f949e600640 1 --2- 172.21.15.119:0/4151689392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94980cece0 0x7f949814bf60 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.210+0000 7f949e600640 1 -- 172.21.15.119:0/4151689392 >> 172.21.15.119:0/4151689392 conn(0x7f94980b55e0 msgr2=0x7f94980c7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:30.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.210+0000 7f949e600640 1 -- 172.21.15.119:0/4151689392 shutdown_connections 2024-04-25T10:11:30.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.210+0000 7f949e600640 1 -- 172.21.15.119:0/4151689392 wait complete. 2024-04-25T10:11:30.236 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:11:30.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.431+0000 7f2e1b200640 1 Processor -- start 2024-04-25T10:11:30.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.432+0000 7f2e1b200640 1 -- start start 2024-04-25T10:11:30.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.433+0000 7f2e1b200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2e1c0b44c0 0x7f2e1c15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.433+0000 7f2e1b200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1c0b2850 con 0x7f2e1c06e4c0 2024-04-25T10:11:30.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.433+0000 7f2e1b200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1c0b2b30 con 0x7f2e1c15af70 2024-04-25T10:11:30.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.433+0000 7f2e1b200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e1c0b33d0 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.434+0000 7f2e1a800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2e1c06e4c0 0x7f2e1c15e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39252/0 (socket says 172.21.15.119:39252) 2024-04-25T10:11:30.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.434+0000 7f2e19e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2e1c0b44c0 0x7f2e1c15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.434+0000 7f2e1a800640 1 -- 172.21.15.119:0/1904073570 learned_addr learned my addr 172.21.15.119:0/1904073570 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:30.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.434+0000 7f2e19e00640 1 -- 172.21.15.119:0/1904073570 >> v1:172.21.15.132:6789/0 conn(0x7f2e1c15af70 legacy=0x7f2e1c0b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:30.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.434+0000 7f2e19e00640 1 -- 172.21.15.119:0/1904073570 >> v1:172.21.15.119:6789/0 conn(0x7f2e1c06e4c0 legacy=0x7f2e1c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.434+0000 7f2e19400640 1 -- 172.21.15.119:0/1904073570 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292950367 0 0) 0x7f2e1c0b2850 con 0x7f2e1c06e4c0 2024-04-25T10:11:30.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.434+0000 7f2e19e00640 1 -- 172.21.15.119:0/1904073570 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c0b2e10 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.435+0000 7f2e19e00640 1 --2- 172.21.15.119:0/1904073570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1c0b44c0 0x7f2e1c15aa30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0097a0 tx=0x7f2e0c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58eaba91d197ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.435+0000 7f2e19400640 1 -- 172.21.15.119:0/1904073570 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c037070 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.435+0000 7f2e19400640 1 -- 172.21.15.119:0/1904073570 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c02ae50 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.435+0000 7f2e19400640 1 -- 172.21.15.119:0/1904073570 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c032660 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.436+0000 7f2e1b200640 1 -- 172.21.15.119:0/1904073570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1c0b44c0 msgr2=0x7f2e1c15aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.436+0000 7f2e1b200640 1 --2- 172.21.15.119:0/1904073570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1c0b44c0 0x7f2e1c15aa30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0097a0 tx=0x7f2e0c02a140 comp rx=0 tx=0).stop 2024-04-25T10:11:30.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.436+0000 7f2e1b200640 1 -- 172.21.15.119:0/1904073570 shutdown_connections 2024-04-25T10:11:30.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.436+0000 7f2e1b200640 1 --2- 172.21.15.119:0/1904073570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1c0b44c0 0x7f2e1c15aa30 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.436+0000 7f2e1b200640 1 -- 172.21.15.119:0/1904073570 >> 172.21.15.119:0/1904073570 conn(0x7f2e1c0b5070 msgr2=0x7f2e1c0cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:30.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.436+0000 7f2e1b200640 1 -- 172.21.15.119:0/1904073570 shutdown_connections 2024-04-25T10:11:30.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.436+0000 7f2e1b200640 1 -- 172.21.15.119:0/1904073570 wait complete. 2024-04-25T10:11:30.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.438+0000 7f2e1b200640 1 Processor -- start 2024-04-25T10:11:30.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.438+0000 7f2e1b200640 1 -- start start 2024-04-25T10:11:30.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.439+0000 7f2e1b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e1c06e4c0 0x7f2e1c1491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.439+0000 7f2e1b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1c0b44c0 0x7f2e1c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.439+0000 7f2e1b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e1c15af70 0x7f2e1c15d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.439+0000 7f2e1b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0b2470 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.439+0000 7f2e1b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0b1af0 con 0x7f2e1c15af70 2024-04-25T10:11:30.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.439+0000 7f2e1b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0b25f0 con 0x7f2e1c06e4c0 2024-04-25T10:11:30.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.440+0000 7f2e11400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1c0b44c0 0x7f2e1c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.440+0000 7f2e11400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1c0b44c0 0x7f2e1c15d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59426/0 (socket says 172.21.15.119:59426) 2024-04-25T10:11:30.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.440+0000 7f2e11400640 1 -- 172.21.15.119:0/1737883840 learned_addr learned my addr 172.21.15.119:0/1737883840 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:30.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.440+0000 7f2e19e00640 1 --2- 172.21.15.119:0/1737883840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e1c06e4c0 0x7f2e1c1491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.440+0000 7f2e1a800640 1 --2- 172.21.15.119:0/1737883840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e1c15af70 0x7f2e1c15d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.440+0000 7f2e11400640 1 -- 172.21.15.119:0/1737883840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e1c06e4c0 msgr2=0x7f2e1c1491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.440+0000 7f2e11400640 1 --2- 172.21.15.119:0/1737883840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e1c06e4c0 0x7f2e1c1491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.440+0000 7f2e11400640 1 -- 172.21.15.119:0/1737883840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e1c15af70 msgr2=0x7f2e1c15d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.440+0000 7f2e11400640 1 --2- 172.21.15.119:0/1737883840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e1c15af70 0x7f2e1c15d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.440+0000 7f2e11400640 1 -- 172.21.15.119:0/1737883840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c0ac8b0 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.440+0000 7f2e11400640 1 --2- 172.21.15.119:0/1737883840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1c0b44c0 0x7f2e1c15d2a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400b3e0 tx=0x7f2e0400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.441+0000 7f2e12a00640 1 -- 172.21.15.119:0/1737883840 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04015020 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.441+0000 7f2e12a00640 1 -- 172.21.15.119:0/1737883840 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e04011830 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.441+0000 7f2e12a00640 1 -- 172.21.15.119:0/1737883840 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04011c20 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.441+0000 7f2e1b200640 1 -- 172.21.15.119:0/1737883840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1c0acb50 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.441+0000 7f2e1b200640 1 -- 172.21.15.119:0/1737883840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c0ad110 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.442+0000 7f2e1b200640 1 -- 172.21.15.119:0/1737883840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1c0749b0 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.443+0000 7f2e12a00640 1 -- 172.21.15.119:0/1737883840 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e04018020 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.444+0000 7f2e12a00640 1 --2- 172.21.15.119:0/1737883840 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2df806b500 0x7f2df806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.444+0000 7f2e12a00640 1 -- 172.21.15.119:0/1737883840 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2e04089dc0 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.444+0000 7f2e19e00640 1 --2- 172.21.15.119:0/1737883840 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2df806b500 0x7f2df806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.445+0000 7f2e19e00640 1 --2- 172.21.15.119:0/1737883840 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2df806b500 0x7f2df806d9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c034000 tx=0x7f2e0c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.448+0000 7f2e12a00640 1 -- 172.21.15.119:0/1737883840 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e0401f020 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.714+0000 7f2e1b200640 1 -- 172.21.15.119:0/1737883840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2e1c0cbd80 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.716+0000 7f2e12a00640 1 -- 172.21.15.119:0/1737883840 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2e04058ef0 con 0x7f2e1c0b44c0 2024-04-25T10:11:30.721 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:30.721 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:11:30.721 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2024-04-25T10:11:30.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.721 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2024-04-25T10:11:30.721 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmFDz9FBAAdC/N+8UnkYkK5wrDlgpkgA==", 2024-04-25T10:11:30.721 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.721 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:11:30.721 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:11:30.722 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:11:30.722 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.722 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2024-04-25T10:11:30.722 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmmh7lLhAAkzsFzTL6Ko/J/0ZJOpvR4A==", 2024-04-25T10:11:30.722 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.722 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:11:30.722 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:11:30.722 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:11:30.722 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.722 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmojotIhAAdSIhFOXoYZN+01nbCs3e+Q==", 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypm1gp7ORAAvtG28IuhGtR1/Vj6JCPq5A==", 2024-04-25T10:11:30.723 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAPKypmOI9YMRAAmMrg0xRUG+GG+waJ12llJQ==", 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.725 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2024-04-25T10:11:30.725 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQATKypmHV5GDxAAW0XzpAxF7oxgVBKy5yzavw==", 2024-04-25T10:11:30.725 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.725 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:30.725 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:11:30.725 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.725 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.725 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.725 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.725 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2024-04-25T10:11:30.725 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAWKypmXTiRGhAAZxct/Ck3A3DMBjqNW3HXrg==", 2024-04-25T10:11:30.725 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.726 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:30.726 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:11:30.726 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.726 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.726 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.726 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2024-04-25T10:11:30.726 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAZKypmEeGSMhAAwVNTny5OmP/KONYhIXzrrQ==", 2024-04-25T10:11:30.726 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.726 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:30.726 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:11:30.726 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.726 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.726 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.727 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2024-04-25T10:11:30.727 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAgKypmh7gPBRAApx1VM3WpYW7hIQzu7+6J7w==", 2024-04-25T10:11:30.727 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.727 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:30.727 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:11:30.727 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.727 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.727 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.727 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.727 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2024-04-25T10:11:30.727 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAjKypmQ2EaEBAAoFYMlnh5/ar0pAlynwjYOA==", 2024-04-25T10:11:30.727 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.727 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:30.728 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:11:30.728 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.728 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.728 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2024-04-25T10:11:30.728 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAmKypmDATkKRAAsIDViOBeDrd5N91wf8zGLA==", 2024-04-25T10:11:30.728 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.728 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:30.728 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:11:30.728 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.728 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.728 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.728 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.729 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2024-04-25T10:11:30.729 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAqKypmNskbCBAAQIYhJ9lfHHr9EP/n5QJoRw==", 2024-04-25T10:11:30.729 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.729 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:30.729 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:11:30.729 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.729 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.729 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.729 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.729 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2024-04-25T10:11:30.729 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA==", 2024-04-25T10:11:30.729 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.729 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:11:30.729 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2024-04-25T10:11:30.730 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:11:30.730 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.730 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2024-04-25T10:11:30.730 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmm0DLGhAApkxEcc1jUU5T/9ErkTkT4A==", 2024-04-25T10:11:30.730 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.730 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:11:30.730 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:11:30.730 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:11:30.730 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:11:30.730 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.730 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.731 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2024-04-25T10:11:30.731 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmMZ89ERAAICfWm/ZgOLQMVSv5xG0Crg==", 2024-04-25T10:11:30.731 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.731 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:11:30.731 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:11:30.731 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:11:30.731 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:11:30.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.731 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.731 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.731 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2024-04-25T10:11:30.731 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmrokCIxAAwi8FtvYSVhMguvZ0OyYt+g==", 2024-04-25T10:11:30.731 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.731 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:11:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:11:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:11:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:11:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2024-04-25T10:11:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAKKypmN9WFGRAAi+stjuG2ZDBjafeR9Ygrug==", 2024-04-25T10:11:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:11:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:11:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:11:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.733 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2024-04-25T10:11:30.733 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypm/KyKDBAA646fW6qrcXB31usxeIApaA==", 2024-04-25T10:11:30.733 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.733 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:11:30.733 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2024-04-25T10:11:30.733 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.733 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.733 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:11:30.733 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:30.733 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2024-04-25T10:11:30.733 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmr8SqBBAA8l4CjUEidXz1obWJFM4DHA==", 2024-04-25T10:11:30.733 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:11:30.734 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:11:30.734 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2024-04-25T10:11:30.734 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:11:30.734 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.734 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:30.734 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-25T10:11:30.734 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:11:30.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.718+0000 7f2e1b200640 1 -- 172.21.15.119:0/1737883840 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2df806b500 msgr2=0x7f2df806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.718+0000 7f2e1b200640 1 --2- 172.21.15.119:0/1737883840 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2df806b500 0x7f2df806d9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c034000 tx=0x7f2e0c02e000 comp rx=0 tx=0).stop 2024-04-25T10:11:30.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.718+0000 7f2e1b200640 1 -- 172.21.15.119:0/1737883840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1c0b44c0 msgr2=0x7f2e1c15d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.718+0000 7f2e1b200640 1 --2- 172.21.15.119:0/1737883840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1c0b44c0 0x7f2e1c15d2a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400b3e0 tx=0x7f2e0400b8a0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.718+0000 7f2e1b200640 1 -- 172.21.15.119:0/1737883840 shutdown_connections 2024-04-25T10:11:30.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.718+0000 7f2e1b200640 1 --2- 172.21.15.119:0/1737883840 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2df806b500 0x7f2df806d9c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.718+0000 7f2e1b200640 1 --2- 172.21.15.119:0/1737883840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e1c15af70 0x7f2e1c15d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.718+0000 7f2e1b200640 1 --2- 172.21.15.119:0/1737883840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e1c0b44c0 0x7f2e1c15d2a0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.718+0000 7f2e1b200640 1 --2- 172.21.15.119:0/1737883840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e1c06e4c0 0x7f2e1c1491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.718+0000 7f2e1b200640 1 -- 172.21.15.119:0/1737883840 >> 172.21.15.119:0/1737883840 conn(0x7f2e1c0b5070 msgr2=0x7f2e1c0ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:30.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.719+0000 7f2e1b200640 1 -- 172.21.15.119:0/1737883840 shutdown_connections 2024-04-25T10:11:30.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.719+0000 7f2e1b200640 1 -- 172.21.15.119:0/1737883840 wait complete. 2024-04-25T10:11:30.741 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:30.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.925+0000 7f3fdfe00640 1 Processor -- start 2024-04-25T10:11:30.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.925+0000 7f3fdfe00640 1 -- start start 2024-04-25T10:11:30.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.927+0000 7f3fdfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3fe0144470 0x7f3fe015fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.927+0000 7f3fdfe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3fe006e430 0x7f3fe00b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.927+0000 7f3fdfe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe00cd440 con 0x7f3fe00b4e80 2024-04-25T10:11:30.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.927+0000 7f3fdfe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe0145340 con 0x7f3fe006e430 2024-04-25T10:11:30.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.927+0000 7f3fdfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00cd720 con 0x7f3fe0144470 2024-04-25T10:11:30.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.927+0000 7f3fdea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3fe0144470 0x7f3fe015fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.928+0000 7f3fdf400640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f3fe00b4e80 0x7f3fe006d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:39578/0 (socket says 172.21.15.119:39578) 2024-04-25T10:11:30.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.928+0000 7f3fdea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3fe0144470 0x7f3fe015fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:40736/0 (socket says 172.21.15.119:40736) 2024-04-25T10:11:30.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.928+0000 7f3fdf400640 1 -- 172.21.15.119:0/2185524095 learned_addr learned my addr 172.21.15.119:0/2185524095 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:30.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.928+0000 7f3fde000640 1 --2- 172.21.15.119:0/2185524095 >> v2:172.21.15.132:3300/0 conn(0x7f3fe006e430 0x7f3fe00b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.928+0000 7f3fdea00640 1 -- 172.21.15.119:0/2185524095 >> v2:172.21.15.132:3300/0 conn(0x7f3fe006e430 msgr2=0x7f3fe00b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:30.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.928+0000 7f3fdea00640 1 --2- 172.21.15.119:0/2185524095 >> v2:172.21.15.132:3300/0 conn(0x7f3fe006e430 0x7f3fe00b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.928+0000 7f3fdea00640 1 -- 172.21.15.119:0/2185524095 >> v1:172.21.15.132:6789/0 conn(0x7f3fe00b4e80 legacy=0x7f3fe006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.928+0000 7f3fdea00640 1 -- 172.21.15.119:0/2185524095 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe00cda00 con 0x7f3fe0144470 2024-04-25T10:11:30.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.930+0000 7f3fdea00640 1 --2- 172.21.15.119:0/2185524095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3fe0144470 0x7f3fe015fc90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00097a0 tx=0x7f3fd002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44107dd89d2b767f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.930+0000 7f3fdd600640 1 -- 172.21.15.119:0/2185524095 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0037070 con 0x7f3fe0144470 2024-04-25T10:11:30.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.931+0000 7f3fdd600640 1 -- 172.21.15.119:0/2185524095 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd0032440 con 0x7f3fe0144470 2024-04-25T10:11:30.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.931+0000 7f3fdd600640 1 -- 172.21.15.119:0/2185524095 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0032810 con 0x7f3fe0144470 2024-04-25T10:11:30.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.931+0000 7f3fdfe00640 1 -- 172.21.15.119:0/2185524095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3fe0144470 msgr2=0x7f3fe015fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.931+0000 7f3fdfe00640 1 --2- 172.21.15.119:0/2185524095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3fe0144470 0x7f3fe015fc90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00097a0 tx=0x7f3fd002a180 comp rx=0 tx=0).stop 2024-04-25T10:11:30.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.931+0000 7f3fdfe00640 1 -- 172.21.15.119:0/2185524095 shutdown_connections 2024-04-25T10:11:30.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.931+0000 7f3fdfe00640 1 --2- 172.21.15.119:0/2185524095 >> v2:172.21.15.132:3300/0 conn(0x7f3fe006e430 0x7f3fe00b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.931+0000 7f3fdfe00640 1 --2- 172.21.15.119:0/2185524095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3fe0144470 0x7f3fe015fc90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.931+0000 7f3fdfe00640 1 -- 172.21.15.119:0/2185524095 >> 172.21.15.119:0/2185524095 conn(0x7f3fe006b1d0 msgr2=0x7f3fe00b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:30.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.931+0000 7f3fdfe00640 1 -- 172.21.15.119:0/2185524095 shutdown_connections 2024-04-25T10:11:30.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.931+0000 7f3fdfe00640 1 -- 172.21.15.119:0/2185524095 wait complete. 2024-04-25T10:11:30.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.933+0000 7f3fdfe00640 1 Processor -- start 2024-04-25T10:11:30.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.933+0000 7f3fdfe00640 1 -- start start 2024-04-25T10:11:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.934+0000 7f3fdfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3fe006e430 0x7f3fe01e64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.934+0000 7f3fdfe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe00b4e80 0x7f3fe014dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.934+0000 7f3fdfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe0144470 0x7f3fe0140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.934+0000 7f3fdfe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00ce200 con 0x7f3fe0144470 2024-04-25T10:11:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.934+0000 7f3fdfe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe0144850 con 0x7f3fe00b4e80 2024-04-25T10:11:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.934+0000 7f3fdfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe015cd00 con 0x7f3fe006e430 2024-04-25T10:11:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.934+0000 7f3fdf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe0144470 0x7f3fe0140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.935+0000 7f3fdf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe0144470 0x7f3fe0140fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42858/0 (socket says 172.21.15.119:42858) 2024-04-25T10:11:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.935+0000 7f3fdf400640 1 -- 172.21.15.119:0/3604006174 learned_addr learned my addr 172.21.15.119:0/3604006174 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.935+0000 7f3fdea00640 1 --2- 172.21.15.119:0/3604006174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3fe006e430 0x7f3fe01e64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.935+0000 7f3fde000640 1 --2- 172.21.15.119:0/3604006174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe00b4e80 0x7f3fe014dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.935+0000 7f3fdf400640 1 -- 172.21.15.119:0/3604006174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3fe006e430 msgr2=0x7f3fe01e64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.935+0000 7f3fdf400640 1 --2- 172.21.15.119:0/3604006174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3fe006e430 0x7f3fe01e64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.935+0000 7f3fdf400640 1 -- 172.21.15.119:0/3604006174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe00b4e80 msgr2=0x7f3fe014dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.935+0000 7f3fdf400640 1 --2- 172.21.15.119:0/3604006174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe00b4e80 0x7f3fe014dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.935+0000 7f3fdf400640 1 -- 172.21.15.119:0/3604006174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe01e5400 con 0x7f3fe0144470 2024-04-25T10:11:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.935+0000 7f3fdf400640 1 --2- 172.21.15.119:0/3604006174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe0144470 0x7f3fe0140fc0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800cbc0 tx=0x7f3fd800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.936+0000 7f3fd6a00640 1 -- 172.21.15.119:0/3604006174 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8017070 con 0x7f3fe0144470 2024-04-25T10:11:30.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.936+0000 7f3fd6a00640 1 -- 172.21.15.119:0/3604006174 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd800ad40 con 0x7f3fe0144470 2024-04-25T10:11:30.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.936+0000 7f3fd6a00640 1 -- 172.21.15.119:0/3604006174 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8012600 con 0x7f3fe0144470 2024-04-25T10:11:30.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.936+0000 7f3fdfe00640 1 -- 172.21.15.119:0/3604006174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe01e56e0 con 0x7f3fe0144470 2024-04-25T10:11:30.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.936+0000 7f3fdfe00640 1 -- 172.21.15.119:0/3604006174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe01e5ca0 con 0x7f3fe0144470 2024-04-25T10:11:30.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.937+0000 7f3fdfe00640 1 -- 172.21.15.119:0/3604006174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe00749b0 con 0x7f3fe0144470 2024-04-25T10:11:30.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.938+0000 7f3fd6a00640 1 -- 172.21.15.119:0/3604006174 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fd801a070 con 0x7f3fe0144470 2024-04-25T10:11:30.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.939+0000 7f3fd6a00640 1 --2- 172.21.15.119:0/3604006174 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3fc006b4d0 0x7f3fc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.939+0000 7f3fd6a00640 1 -- 172.21.15.119:0/3604006174 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3fd8089c00 con 0x7f3fe0144470 2024-04-25T10:11:30.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.939+0000 7f3fdea00640 1 --2- 172.21.15.119:0/3604006174 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3fc006b4d0 0x7f3fc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.940+0000 7f3fdea00640 1 --2- 172.21.15.119:0/3604006174 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3fc006b4d0 0x7f3fc006d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0034000 tx=0x7f3fd002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:30.943+0000 7f3fd6a00640 1 -- 172.21.15.119:0/3604006174 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3fd80586e0 con 0x7f3fe0144470 2024-04-25T10:11:31.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.143+0000 7f3fdfe00640 1 -- 172.21.15.119:0/3604006174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fe0144fe0 con 0x7f3fe0144470 2024-04-25T10:11:31.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.145+0000 7f3fd6a00640 1 -- 172.21.15.119:0/3604006174 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+12927 (secure 0 0 0) 0x7f3fd805c780 con 0x7f3fe0144470 2024-04-25T10:11:31.151 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:31.151 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":63,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:11:30.203779+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:31.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.148+0000 7f3fdfe00640 1 -- 172.21.15.119:0/3604006174 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3fc006b4d0 msgr2=0x7f3fc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.148+0000 7f3fdfe00640 1 --2- 172.21.15.119:0/3604006174 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3fc006b4d0 0x7f3fc006d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0034000 tx=0x7f3fd002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:31.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.148+0000 7f3fdfe00640 1 -- 172.21.15.119:0/3604006174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe0144470 msgr2=0x7f3fe0140fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.148+0000 7f3fdfe00640 1 --2- 172.21.15.119:0/3604006174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe0144470 0x7f3fe0140fc0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800cbc0 tx=0x7f3fd800a430 comp rx=0 tx=0).stop 2024-04-25T10:11:31.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.148+0000 7f3fdfe00640 1 -- 172.21.15.119:0/3604006174 shutdown_connections 2024-04-25T10:11:31.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.148+0000 7f3fdfe00640 1 --2- 172.21.15.119:0/3604006174 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3fc006b4d0 0x7f3fc006d990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.148+0000 7f3fdfe00640 1 --2- 172.21.15.119:0/3604006174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fe0144470 0x7f3fe0140fc0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.148+0000 7f3fdfe00640 1 --2- 172.21.15.119:0/3604006174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fe00b4e80 0x7f3fe014dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.148+0000 7f3fdfe00640 1 --2- 172.21.15.119:0/3604006174 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3fe006e430 0x7f3fe01e64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.148+0000 7f3fdfe00640 1 -- 172.21.15.119:0/3604006174 >> 172.21.15.119:0/3604006174 conn(0x7f3fe006b1d0 msgr2=0x7f3fe015d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.148+0000 7f3fdfe00640 1 -- 172.21.15.119:0/3604006174 shutdown_connections 2024-04-25T10:11:31.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.148+0000 7f3fdfe00640 1 -- 172.21.15.119:0/3604006174 wait complete. 2024-04-25T10:11:31.162 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:31.169 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:31.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:31.174 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:31.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.396+0000 7fe2c7e00640 1 Processor -- start 2024-04-25T10:11:31.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.396+0000 7fe2c7e00640 1 -- start start 2024-04-25T10:11:31.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.398+0000 7fe2c7e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe2c806b2b0 0x7fe2c80b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.398+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2c80b34a0 0x7fe2c80c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.398+0000 7fe2c7e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c80c5740 con 0x7fe2c80b0e00 2024-04-25T10:11:31.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.398+0000 7fe2c7e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c80b11e0 con 0x7fe2c806b2b0 2024-04-25T10:11:31.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.398+0000 7fe2c7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80ce140 con 0x7fe2c80b34a0 2024-04-25T10:11:31.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.398+0000 7fe2c6a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe2c80b0e00 0x7fe2c806aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52726/0 (socket says 172.21.15.119:52726) 2024-04-25T10:11:31.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.398+0000 7fe2c6a00640 1 -- 172.21.15.119:0/2723717577 learned_addr learned my addr 172.21.15.119:0/2723717577 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:31.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.398+0000 7fe2be000640 1 --2- 172.21.15.119:0/2723717577 >> v2:172.21.15.119:3300/0 conn(0x7fe2c806b2b0 0x7fe2c80b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.399+0000 7fe2c7400640 1 --2- 172.21.15.119:0/2723717577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2c80b34a0 0x7fe2c80c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.399+0000 7fe2c6000640 1 -- 172.21.15.119:0/2723717577 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718870874 0 0) 0x7fe2b8001490 con 0x7fe2c80b0e00 2024-04-25T10:11:31.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.399+0000 7fe2c6000640 1 -- 172.21.15.119:0/2723717577 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2ac003520 con 0x7fe2c80b0e00 2024-04-25T10:11:31.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.399+0000 7fe2be000640 1 -- 172.21.15.119:0/2723717577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2c80b34a0 msgr2=0x7fe2c80c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:31.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.399+0000 7fe2be000640 1 --2- 172.21.15.119:0/2723717577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2c80b34a0 0x7fe2c80c6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.399+0000 7fe2be000640 1 -- 172.21.15.119:0/2723717577 >> v1:172.21.15.119:6789/0 conn(0x7fe2c80b0e00 legacy=0x7fe2c806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.399+0000 7fe2be000640 1 -- 172.21.15.119:0/2723717577 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c80c7320 con 0x7fe2c806b2b0 2024-04-25T10:11:31.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.400+0000 7fe2be000640 1 --2- 172.21.15.119:0/2723717577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c806b2b0 0x7fe2c80b2cc0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00097a0 tx=0x7fe2b002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbf556d2e0887ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.400+0000 7fe2c6000640 1 -- 172.21.15.119:0/2723717577 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0037070 con 0x7fe2c806b2b0 2024-04-25T10:11:31.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.400+0000 7fe2c6000640 1 -- 172.21.15.119:0/2723717577 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b002ae50 con 0x7fe2c806b2b0 2024-04-25T10:11:31.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.400+0000 7fe2c6000640 1 -- 172.21.15.119:0/2723717577 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0032640 con 0x7fe2c806b2b0 2024-04-25T10:11:31.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.401+0000 7fe2c7e00640 1 -- 172.21.15.119:0/2723717577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c806b2b0 msgr2=0x7fe2c80b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.401+0000 7fe2c7e00640 1 --2- 172.21.15.119:0/2723717577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c806b2b0 0x7fe2c80b2cc0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00097a0 tx=0x7fe2b002a100 comp rx=0 tx=0).stop 2024-04-25T10:11:31.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.401+0000 7fe2c7e00640 1 -- 172.21.15.119:0/2723717577 shutdown_connections 2024-04-25T10:11:31.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.401+0000 7fe2c7e00640 1 --2- 172.21.15.119:0/2723717577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2c80b34a0 0x7fe2c80c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.401+0000 7fe2c7e00640 1 --2- 172.21.15.119:0/2723717577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c806b2b0 0x7fe2c80b2cc0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.401+0000 7fe2c7e00640 1 -- 172.21.15.119:0/2723717577 >> 172.21.15.119:0/2723717577 conn(0x7fe2c80b21a0 msgr2=0x7fe2c80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.401+0000 7fe2c7e00640 1 -- 172.21.15.119:0/2723717577 shutdown_connections 2024-04-25T10:11:31.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.401+0000 7fe2c7e00640 1 -- 172.21.15.119:0/2723717577 wait complete. 2024-04-25T10:11:31.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.403+0000 7fe2c7e00640 1 Processor -- start 2024-04-25T10:11:31.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.403+0000 7fe2c7e00640 1 -- start start 2024-04-25T10:11:31.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.404+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2c806b2b0 0x7fe2c81f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.404+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2c80b0e00 0x7fe2c80b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.404+0000 7fe2c7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c80b34a0 0x7fe2c80ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.404+0000 7fe2c7e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80ce9c0 con 0x7fe2c80b34a0 2024-04-25T10:11:31.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.404+0000 7fe2c7e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c80ce6e0 con 0x7fe2c806b2b0 2024-04-25T10:11:31.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.404+0000 7fe2c7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c81615e0 con 0x7fe2c80b0e00 2024-04-25T10:11:31.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.404+0000 7fe2c7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c80b34a0 0x7fe2c80ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.404+0000 7fe2c7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c80b34a0 0x7fe2c80ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42884/0 (socket says 172.21.15.119:42884) 2024-04-25T10:11:31.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.404+0000 7fe2c7400640 1 -- 172.21.15.119:0/1725993365 learned_addr learned my addr 172.21.15.119:0/1725993365 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:31.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.405+0000 7fe2be000640 1 --2- 172.21.15.119:0/1725993365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2c80b0e00 0x7fe2c80b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.405+0000 7fe2c6a00640 1 --2- 172.21.15.119:0/1725993365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2c806b2b0 0x7fe2c81f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.405+0000 7fe2c7400640 1 -- 172.21.15.119:0/1725993365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2c80b0e00 msgr2=0x7fe2c80b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.405+0000 7fe2c7400640 1 --2- 172.21.15.119:0/1725993365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2c80b0e00 0x7fe2c80b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.405+0000 7fe2c7400640 1 -- 172.21.15.119:0/1725993365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2c806b2b0 msgr2=0x7fe2c81f3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.405+0000 7fe2c7400640 1 --2- 172.21.15.119:0/1725993365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2c806b2b0 0x7fe2c81f3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.405+0000 7fe2c7400640 1 -- 172.21.15.119:0/1725993365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c81f7d50 con 0x7fe2c80b34a0 2024-04-25T10:11:31.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.405+0000 7fe2c7400640 1 --2- 172.21.15.119:0/1725993365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c80b34a0 0x7fe2c80ae130 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000d600 tx=0x7fe2c000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.406+0000 7fe2bf400640 1 -- 172.21.15.119:0/1725993365 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c00125f0 con 0x7fe2c80b34a0 2024-04-25T10:11:31.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.406+0000 7fe2bf400640 1 -- 172.21.15.119:0/1725993365 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c0019070 con 0x7fe2c80b34a0 2024-04-25T10:11:31.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.406+0000 7fe2bf400640 1 -- 172.21.15.119:0/1725993365 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c00129f0 con 0x7fe2c80b34a0 2024-04-25T10:11:31.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.406+0000 7fe2c7e00640 1 -- 172.21.15.119:0/1725993365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c81f3d70 con 0x7fe2c80b34a0 2024-04-25T10:11:31.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.406+0000 7fe2c7e00640 1 -- 172.21.15.119:0/1725993365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c80c8e20 con 0x7fe2c80b34a0 2024-04-25T10:11:31.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.407+0000 7fe2c7e00640 1 -- 172.21.15.119:0/1725993365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c806b0d0 con 0x7fe2c80b34a0 2024-04-25T10:11:31.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.408+0000 7fe2bf400640 1 -- 172.21.15.119:0/1725993365 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2c0012b90 con 0x7fe2c80b34a0 2024-04-25T10:11:31.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.409+0000 7fe2bf400640 1 --2- 172.21.15.119:0/1725993365 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2a406b4d0 0x7fe2a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.409+0000 7fe2bf400640 1 -- 172.21.15.119:0/1725993365 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe2c008a310 con 0x7fe2c80b34a0 2024-04-25T10:11:31.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.409+0000 7fe2c6a00640 1 --2- 172.21.15.119:0/1725993365 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2a406b4d0 0x7fe2a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.410+0000 7fe2c6a00640 1 --2- 172.21.15.119:0/1725993365 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2a406b4d0 0x7fe2a406d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8005c00 tx=0x7fe2b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.415+0000 7fe2bf400640 1 -- 172.21.15.119:0/1725993365 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2c806b0d0 con 0x7fe2c80b34a0 2024-04-25T10:11:31.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.611+0000 7fe2c7e00640 1 -- 172.21.15.119:0/1725993365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2c806caf0 con 0x7fe2c80b34a0 2024-04-25T10:11:31.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.612+0000 7fe2bf400640 1 -- 172.21.15.119:0/1725993365 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2c0059450 con 0x7fe2c80b34a0 2024-04-25T10:11:31.618 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:11:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.614+0000 7fe2c7e00640 1 -- 172.21.15.119:0/1725993365 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2a406b4d0 msgr2=0x7fe2a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.614+0000 7fe2c7e00640 1 --2- 172.21.15.119:0/1725993365 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2a406b4d0 0x7fe2a406d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8005c00 tx=0x7fe2b8009040 comp rx=0 tx=0).stop 2024-04-25T10:11:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.614+0000 7fe2c7e00640 1 -- 172.21.15.119:0/1725993365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c80b34a0 msgr2=0x7fe2c80ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.614+0000 7fe2c7e00640 1 --2- 172.21.15.119:0/1725993365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c80b34a0 0x7fe2c80ae130 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000d600 tx=0x7fe2c000dac0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.615+0000 7fe2c7e00640 1 -- 172.21.15.119:0/1725993365 shutdown_connections 2024-04-25T10:11:31.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.615+0000 7fe2c7e00640 1 --2- 172.21.15.119:0/1725993365 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2a406b4d0 0x7fe2a406d990 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.615+0000 7fe2c7e00640 1 --2- 172.21.15.119:0/1725993365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c80b34a0 0x7fe2c80ae130 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.615+0000 7fe2c7e00640 1 --2- 172.21.15.119:0/1725993365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2c80b0e00 0x7fe2c80b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.615+0000 7fe2c7e00640 1 --2- 172.21.15.119:0/1725993365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2c806b2b0 0x7fe2c81f3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.615+0000 7fe2c7e00640 1 -- 172.21.15.119:0/1725993365 >> 172.21.15.119:0/1725993365 conn(0x7fe2c80b21a0 msgr2=0x7fe2c806efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.615+0000 7fe2c7e00640 1 -- 172.21.15.119:0/1725993365 shutdown_connections 2024-04-25T10:11:31.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.615+0000 7fe2c7e00640 1 -- 172.21.15.119:0/1725993365 wait complete. 2024-04-25T10:11:31.640 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:11:31.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.815+0000 7f258b200640 1 Processor -- start 2024-04-25T10:11:31.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.815+0000 7f258b200640 1 -- start start 2024-04-25T10:11:31.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.816+0000 7f258b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f258415dbc0 0x7f25840b9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.816+0000 7f258b200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2584162340 0x7f25840b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.816+0000 7f258b200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258415e9f0 con 0x7f2584161ee0 2024-04-25T10:11:31.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.816+0000 7f258b200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2584161a50 con 0x7f2584162340 2024-04-25T10:11:31.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.816+0000 7f258b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258415ecd0 con 0x7f258415dbc0 2024-04-25T10:11:31.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.817+0000 7f2583e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f258415dbc0 0x7f25840b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.817+0000 7f2589200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2584162340 0x7f25840b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.817+0000 7f2583e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f258415dbc0 0x7f25840b9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:40776/0 (socket says 172.21.15.119:40776) 2024-04-25T10:11:31.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.817+0000 7f2583e00640 1 -- 172.21.15.119:0/2104834913 learned_addr learned my addr 172.21.15.119:0/2104834913 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:31.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.818+0000 7f2583e00640 1 -- 172.21.15.119:0/2104834913 >> v2:172.21.15.132:3300/0 conn(0x7f2584162340 msgr2=0x7f25840b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:31.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.818+0000 7f2583e00640 1 --2- 172.21.15.119:0/2104834913 >> v2:172.21.15.132:3300/0 conn(0x7f2584162340 0x7f25840b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.818+0000 7f2583e00640 1 -- 172.21.15.119:0/2104834913 >> v1:172.21.15.132:6789/0 conn(0x7f2584161ee0 legacy=0x7f258415dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.818+0000 7f2583e00640 1 -- 172.21.15.119:0/2104834913 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258415efb0 con 0x7f258415dbc0 2024-04-25T10:11:31.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.819+0000 7f2583e00640 1 --2- 172.21.15.119:0/2104834913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f258415dbc0 0x7f25840b9620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2574009870 tx=0x7f257402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d88469a8d85b746f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.820+0000 7f2582a00640 1 -- 172.21.15.119:0/2104834913 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574037070 con 0x7f258415dbc0 2024-04-25T10:11:31.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.820+0000 7f2582a00640 1 -- 172.21.15.119:0/2104834913 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2574032440 con 0x7f258415dbc0 2024-04-25T10:11:31.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.820+0000 7f2582a00640 1 -- 172.21.15.119:0/2104834913 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574032810 con 0x7f258415dbc0 2024-04-25T10:11:31.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.820+0000 7f258b200640 1 -- 172.21.15.119:0/2104834913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f258415dbc0 msgr2=0x7f25840b9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.820+0000 7f258b200640 1 --2- 172.21.15.119:0/2104834913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f258415dbc0 0x7f25840b9620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2574009870 tx=0x7f257402a250 comp rx=0 tx=0).stop 2024-04-25T10:11:31.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.821+0000 7f258b200640 1 -- 172.21.15.119:0/2104834913 shutdown_connections 2024-04-25T10:11:31.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.821+0000 7f258b200640 1 --2- 172.21.15.119:0/2104834913 >> v2:172.21.15.132:3300/0 conn(0x7f2584162340 0x7f25840b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.821+0000 7f258b200640 1 --2- 172.21.15.119:0/2104834913 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f258415dbc0 0x7f25840b9620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.821+0000 7f258b200640 1 -- 172.21.15.119:0/2104834913 >> 172.21.15.119:0/2104834913 conn(0x7f2584072900 msgr2=0x7f25840d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.821+0000 7f258b200640 1 -- 172.21.15.119:0/2104834913 shutdown_connections 2024-04-25T10:11:31.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.821+0000 7f258b200640 1 -- 172.21.15.119:0/2104834913 wait complete. 2024-04-25T10:11:31.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.823+0000 7f258b200640 1 Processor -- start 2024-04-25T10:11:31.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.823+0000 7f258b200640 1 -- start start 2024-04-25T10:11:31.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.825+0000 7f258b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f258415dbc0 0x7f25840abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.825+0000 7f258b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2584161ee0 0x7f25840cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.825+0000 7f258b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2584162340 0x7f258415ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.825+0000 7f258b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25840b5a00 con 0x7f2584162340 2024-04-25T10:11:31.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.825+0000 7f258b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258415e6b0 con 0x7f258415dbc0 2024-04-25T10:11:31.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.825+0000 7f258b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25840b5b80 con 0x7f2584161ee0 2024-04-25T10:11:31.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.825+0000 7f2589200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2584162340 0x7f258415ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.825+0000 7f2589200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2584162340 0x7f258415ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42894/0 (socket says 172.21.15.119:42894) 2024-04-25T10:11:31.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.825+0000 7f2589200640 1 -- 172.21.15.119:0/2348292889 learned_addr learned my addr 172.21.15.119:0/2348292889 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:31.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.826+0000 7f2583400640 1 --2- 172.21.15.119:0/2348292889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2584161ee0 0x7f25840cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.826+0000 7f2583e00640 1 --2- 172.21.15.119:0/2348292889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f258415dbc0 0x7f25840abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.826+0000 7f2589200640 1 -- 172.21.15.119:0/2348292889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2584161ee0 msgr2=0x7f25840cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.826+0000 7f2589200640 1 --2- 172.21.15.119:0/2348292889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2584161ee0 0x7f25840cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.826+0000 7f2589200640 1 -- 172.21.15.119:0/2348292889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f258415dbc0 msgr2=0x7f25840abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.826+0000 7f2589200640 1 --2- 172.21.15.119:0/2348292889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f258415dbc0 0x7f25840abe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.826+0000 7f2589200640 1 -- 172.21.15.119:0/2348292889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25840cce60 con 0x7f2584162340 2024-04-25T10:11:31.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.826+0000 7f2589200640 1 --2- 172.21.15.119:0/2348292889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2584162340 0x7f258415ad70 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f257c00efc0 tx=0x7f257c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.826+0000 7f257be00640 1 -- 172.21.15.119:0/2348292889 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c018020 con 0x7f2584162340 2024-04-25T10:11:31.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.827+0000 7f257be00640 1 -- 172.21.15.119:0/2348292889 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c014480 con 0x7f2584162340 2024-04-25T10:11:31.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.827+0000 7f258b200640 1 -- 172.21.15.119:0/2348292889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25840cd100 con 0x7f2584162340 2024-04-25T10:11:31.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.827+0000 7f257be00640 1 -- 172.21.15.119:0/2348292889 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c014870 con 0x7f2584162340 2024-04-25T10:11:31.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.827+0000 7f258b200640 1 -- 172.21.15.119:0/2348292889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25840aa520 con 0x7f2584162340 2024-04-25T10:11:31.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.828+0000 7f258b200640 1 -- 172.21.15.119:0/2348292889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25840749b0 con 0x7f2584162340 2024-04-25T10:11:31.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.829+0000 7f257be00640 1 -- 172.21.15.119:0/2348292889 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f257c005d10 con 0x7f2584162340 2024-04-25T10:11:31.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.830+0000 7f257be00640 1 --2- 172.21.15.119:0/2348292889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f256406b4d0 0x7f256406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.830+0000 7f257be00640 1 -- 172.21.15.119:0/2348292889 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f257c08a9b0 con 0x7f2584162340 2024-04-25T10:11:31.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.831+0000 7f2583e00640 1 --2- 172.21.15.119:0/2348292889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f256406b4d0 0x7f256406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.831+0000 7f2583e00640 1 --2- 172.21.15.119:0/2348292889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f256406b4d0 0x7f256406d990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2574034000 tx=0x7f257402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:31.834+0000 7f257be00640 1 -- 172.21.15.119:0/2348292889 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f257c022070 con 0x7f2584162340 2024-04-25T10:11:32.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.033+0000 7f258b200640 1 -- 172.21.15.119:0/2348292889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f25840d16e0 con 0x7f2584162340 2024-04-25T10:11:32.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.034+0000 7f257be00640 1 -- 172.21.15.119:0/2348292889 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) v1 ==== 84+0+65 (secure 0 0 0) 0x7f257c010040 con 0x7f2584162340 2024-04-25T10:11:32.039 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:32.039 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2024-04-25T10:11:32.039 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:11:32.040 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2024-04-25T10:11:32.040 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2024-04-25T10:11:32.040 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:32.040 INFO:teuthology.orchestra.run.smithi119.stdout:] 2024-04-25T10:11:32.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.038+0000 7f258b200640 1 -- 172.21.15.119:0/2348292889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f256406b4d0 msgr2=0x7f256406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.038+0000 7f258b200640 1 --2- 172.21.15.119:0/2348292889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f256406b4d0 0x7f256406d990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2574034000 tx=0x7f257402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:32.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.038+0000 7f258b200640 1 -- 172.21.15.119:0/2348292889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2584162340 msgr2=0x7f258415ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.038+0000 7f258b200640 1 --2- 172.21.15.119:0/2348292889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2584162340 0x7f258415ad70 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f257c00efc0 tx=0x7f257c00c490 comp rx=0 tx=0).stop 2024-04-25T10:11:32.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.038+0000 7f258b200640 1 -- 172.21.15.119:0/2348292889 shutdown_connections 2024-04-25T10:11:32.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.038+0000 7f258b200640 1 --2- 172.21.15.119:0/2348292889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f256406b4d0 0x7f256406d990 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.038+0000 7f258b200640 1 --2- 172.21.15.119:0/2348292889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2584162340 0x7f258415ad70 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.038+0000 7f258b200640 1 --2- 172.21.15.119:0/2348292889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2584161ee0 0x7f25840cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.038+0000 7f258b200640 1 --2- 172.21.15.119:0/2348292889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f258415dbc0 0x7f25840abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.038+0000 7f258b200640 1 -- 172.21.15.119:0/2348292889 >> 172.21.15.119:0/2348292889 conn(0x7f2584072900 msgr2=0x7f25840d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:32.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.038+0000 7f258b200640 1 -- 172.21.15.119:0/2348292889 shutdown_connections 2024-04-25T10:11:32.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.038+0000 7f258b200640 1 -- 172.21.15.119:0/2348292889 wait complete. 2024-04-25T10:11:32.064 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:11:32.064 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:11:32.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.244+0000 7fd19da00640 1 Processor -- start 2024-04-25T10:11:32.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.245+0000 7fd19da00640 1 -- start start 2024-04-25T10:11:32.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.246+0000 7fd19da00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1980b2a40 0x7fd198165550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.246+0000 7fd19da00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1980b85b0 0x7fd198167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.246+0000 7fd19da00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1980b6b00 con 0x7fd19815d9c0 2024-04-25T10:11:32.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.246+0000 7fd19da00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1980b7110 con 0x7fd1980b85b0 2024-04-25T10:11:32.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.246+0000 7fd19da00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980b9530 con 0x7fd1980b2a40 2024-04-25T10:11:32.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.247+0000 7fd197e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd19815d9c0 0x7fd1980ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52730/0 (socket says 172.21.15.119:52730) 2024-04-25T10:11:32.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.247+0000 7fd197e00640 1 -- 172.21.15.119:0/697510533 learned_addr learned my addr 172.21.15.119:0/697510533 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:32.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.247+0000 7fd19d000640 1 --2- 172.21.15.119:0/697510533 >> v2:172.21.15.119:3300/0 conn(0x7fd1980b85b0 0x7fd198167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.247+0000 7fd18f400640 1 --2- 172.21.15.119:0/697510533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1980b2a40 0x7fd198165550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.247+0000 7fd197400640 1 -- 172.21.15.119:0/697510533 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166133980 0 0) 0x7fd1980b6b00 con 0x7fd19815d9c0 2024-04-25T10:11:32.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.247+0000 7fd197400640 1 -- 172.21.15.119:0/697510533 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd17c003520 con 0x7fd19815d9c0 2024-04-25T10:11:32.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.247+0000 7fd19d000640 1 -- 172.21.15.119:0/697510533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1980b2a40 msgr2=0x7fd198165550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:32.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.247+0000 7fd19d000640 1 --2- 172.21.15.119:0/697510533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1980b2a40 0x7fd198165550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.247+0000 7fd19d000640 1 -- 172.21.15.119:0/697510533 >> v1:172.21.15.119:6789/0 conn(0x7fd19815d9c0 legacy=0x7fd1980ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.247+0000 7fd19d000640 1 -- 172.21.15.119:0/697510533 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1980b6da0 con 0x7fd1980b85b0 2024-04-25T10:11:32.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.248+0000 7fd19d000640 1 --2- 172.21.15.119:0/697510533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1980b85b0 0x7fd198167aa0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd1900097a0 tx=0x7fd19002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1454c04abee64d30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.249+0000 7fd197400640 1 -- 172.21.15.119:0/697510533 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190037070 con 0x7fd1980b85b0 2024-04-25T10:11:32.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.249+0000 7fd197400640 1 -- 172.21.15.119:0/697510533 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd190032440 con 0x7fd1980b85b0 2024-04-25T10:11:32.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.249+0000 7fd197400640 1 -- 172.21.15.119:0/697510533 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1900327f0 con 0x7fd1980b85b0 2024-04-25T10:11:32.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.249+0000 7fd19da00640 1 -- 172.21.15.119:0/697510533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1980b85b0 msgr2=0x7fd198167aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.249+0000 7fd19da00640 1 --2- 172.21.15.119:0/697510533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1980b85b0 0x7fd198167aa0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd1900097a0 tx=0x7fd19002a180 comp rx=0 tx=0).stop 2024-04-25T10:11:32.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.249+0000 7fd19da00640 1 -- 172.21.15.119:0/697510533 shutdown_connections 2024-04-25T10:11:32.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.249+0000 7fd19da00640 1 --2- 172.21.15.119:0/697510533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1980b85b0 0x7fd198167aa0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.249+0000 7fd19da00640 1 --2- 172.21.15.119:0/697510533 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1980b2a40 0x7fd198165550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.249+0000 7fd19da00640 1 -- 172.21.15.119:0/697510533 >> 172.21.15.119:0/697510533 conn(0x7fd1980726c0 msgr2=0x7fd198072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:32.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.249+0000 7fd19da00640 1 -- 172.21.15.119:0/697510533 shutdown_connections 2024-04-25T10:11:32.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.249+0000 7fd19da00640 1 -- 172.21.15.119:0/697510533 wait complete. 2024-04-25T10:11:32.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.251+0000 7fd19da00640 1 Processor -- start 2024-04-25T10:11:32.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.251+0000 7fd19da00640 1 -- start start 2024-04-25T10:11:32.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.252+0000 7fd19da00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1980b2a40 0x7fd1980cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.252+0000 7fd19da00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1980b85b0 0x7fd1981567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.252+0000 7fd19da00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd19815d9c0 0x7fd198153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.252+0000 7fd19da00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980b9150 con 0x7fd1980b85b0 2024-04-25T10:11:32.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.252+0000 7fd19da00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19815dda0 con 0x7fd1980b2a40 2024-04-25T10:11:32.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.252+0000 7fd19da00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980b92d0 con 0x7fd19815d9c0 2024-04-25T10:11:32.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.253+0000 7fd18f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1980b85b0 0x7fd1981567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.253+0000 7fd18f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1980b85b0 0x7fd1981567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42910/0 (socket says 172.21.15.119:42910) 2024-04-25T10:11:32.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.253+0000 7fd18f400640 1 -- 172.21.15.119:0/3196019223 learned_addr learned my addr 172.21.15.119:0/3196019223 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:32.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.253+0000 7fd19d000640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd19815d9c0 0x7fd198153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.253+0000 7fd197e00640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1980b2a40 0x7fd1980cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.253+0000 7fd18f400640 1 -- 172.21.15.119:0/3196019223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd19815d9c0 msgr2=0x7fd198153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.253+0000 7fd18f400640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd19815d9c0 0x7fd198153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.253+0000 7fd18f400640 1 -- 172.21.15.119:0/3196019223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1980b2a40 msgr2=0x7fd1980cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.253+0000 7fd18f400640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1980b2a40 0x7fd1980cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.253+0000 7fd18f400640 1 -- 172.21.15.119:0/3196019223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1980cbf30 con 0x7fd1980b85b0 2024-04-25T10:11:32.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.253+0000 7fd197e00640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1980b2a40 0x7fd1980cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:32.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.254+0000 7fd18f400640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1980b85b0 0x7fd1981567f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd18000d5c0 tx=0x7fd18000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.254+0000 7fd194c00640 1 -- 172.21.15.119:0/3196019223 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1800125f0 con 0x7fd1980b85b0 2024-04-25T10:11:32.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.254+0000 7fd194c00640 1 -- 172.21.15.119:0/3196019223 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd180019070 con 0x7fd1980b85b0 2024-04-25T10:11:32.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.254+0000 7fd194c00640 1 -- 172.21.15.119:0/3196019223 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1800129d0 con 0x7fd1980b85b0 2024-04-25T10:11:32.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.254+0000 7fd19da00640 1 -- 172.21.15.119:0/3196019223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1980cc200 con 0x7fd1980b85b0 2024-04-25T10:11:32.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.254+0000 7fd19da00640 1 -- 172.21.15.119:0/3196019223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1980cc7c0 con 0x7fd1980b85b0 2024-04-25T10:11:32.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.255+0000 7fd19da00640 1 -- 172.21.15.119:0/3196019223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1980749b0 con 0x7fd1980b85b0 2024-04-25T10:11:32.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.256+0000 7fd194c00640 1 -- 172.21.15.119:0/3196019223 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd180012b70 con 0x7fd1980b85b0 2024-04-25T10:11:32.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.257+0000 7fd194c00640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd17406b500 0x7fd17406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.258+0000 7fd194c00640 1 -- 172.21.15.119:0/3196019223 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd18008a280 con 0x7fd1980b85b0 2024-04-25T10:11:32.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.258+0000 7fd197e00640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd17406b500 0x7fd17406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.258+0000 7fd197e00640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd17406b500 0x7fd17406d9c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd188005c00 tx=0x7fd188009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.262+0000 7fd194c00640 1 -- 172.21.15.119:0/3196019223 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd180059440 con 0x7fd1980b85b0 2024-04-25T10:11:32.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:32.455+0000 7fd19da00640 1 -- 172.21.15.119:0/3196019223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd1980cece0 con 0x7fd1980b85b0 2024-04-25T10:11:33.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.233+0000 7fd194c00640 1 -- 172.21.15.119:0/3196019223 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd180058d40 con 0x7fd1980b85b0 2024-04-25T10:11:33.238 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2024-04-25T10:11:33.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.236+0000 7fd19da00640 1 -- 172.21.15.119:0/3196019223 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd17406b500 msgr2=0x7fd17406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.236+0000 7fd19da00640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd17406b500 0x7fd17406d9c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd188005c00 tx=0x7fd188009040 comp rx=0 tx=0).stop 2024-04-25T10:11:33.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.236+0000 7fd19da00640 1 -- 172.21.15.119:0/3196019223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1980b85b0 msgr2=0x7fd1981567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.236+0000 7fd19da00640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1980b85b0 0x7fd1981567f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd18000d5c0 tx=0x7fd18000da80 comp rx=0 tx=0).stop 2024-04-25T10:11:33.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.237+0000 7fd19da00640 1 -- 172.21.15.119:0/3196019223 shutdown_connections 2024-04-25T10:11:33.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.237+0000 7fd19da00640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd17406b500 0x7fd17406d9c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.237+0000 7fd19da00640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd19815d9c0 0x7fd198153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.237+0000 7fd19da00640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1980b85b0 0x7fd1981567f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.237+0000 7fd19da00640 1 --2- 172.21.15.119:0/3196019223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1980b2a40 0x7fd1980cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.237+0000 7fd19da00640 1 -- 172.21.15.119:0/3196019223 >> 172.21.15.119:0/3196019223 conn(0x7fd1980726c0 msgr2=0x7fd19806bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.237+0000 7fd19da00640 1 -- 172.21.15.119:0/3196019223 shutdown_connections 2024-04-25T10:11:33.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.237+0000 7fd19da00640 1 -- 172.21.15.119:0/3196019223 wait complete. 2024-04-25T10:11:33.266 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:11:33.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.451+0000 7f853d600640 1 Processor -- start 2024-04-25T10:11:33.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.451+0000 7f853d600640 1 -- start start 2024-04-25T10:11:33.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.453+0000 7f853d600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8538144d00 0x7f853806d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.453+0000 7f853d600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f85380cf620 0x7f85380cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.453+0000 7f853d600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853815fdd0 con 0x7f8538144550 2024-04-25T10:11:33.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.453+0000 7f853d600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85380cfff0 con 0x7f85380cf620 2024-04-25T10:11:33.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.453+0000 7f853d600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85381600b0 con 0x7f8538144d00 2024-04-25T10:11:33.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.454+0000 7f8536a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8538144550 0x7f853815f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52732/0 (socket says 172.21.15.119:52732) 2024-04-25T10:11:33.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.454+0000 7f8536a00640 1 -- 172.21.15.119:0/514512155 learned_addr learned my addr 172.21.15.119:0/514512155 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:33.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.454+0000 7f8537400640 1 --2- 172.21.15.119:0/514512155 >> v2:172.21.15.132:3300/0 conn(0x7f85380cf620 0x7f85380cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.454+0000 7f8536000640 1 --2- 172.21.15.119:0/514512155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8538144d00 0x7f853806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.454+0000 7f8535600640 1 -- 172.21.15.119:0/514512155 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452380466 0 0) 0x7f853815fdd0 con 0x7f8538144550 2024-04-25T10:11:33.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.454+0000 7f8535600640 1 -- 172.21.15.119:0/514512155 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f851c003520 con 0x7f8538144550 2024-04-25T10:11:33.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.454+0000 7f8535600640 1 -- 172.21.15.119:0/514512155 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2944326330 0 0) 0x7f851c003520 con 0x7f8538144550 2024-04-25T10:11:33.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.454+0000 7f8535600640 1 -- 172.21.15.119:0/514512155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8538144d00 msgr2=0x7f853806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.454+0000 7f8535600640 1 --2- 172.21.15.119:0/514512155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8538144d00 0x7f853806d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.454+0000 7f8535600640 1 -- 172.21.15.119:0/514512155 >> v2:172.21.15.132:3300/0 conn(0x7f85380cf620 msgr2=0x7f85380cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.454+0000 7f8535600640 1 --2- 172.21.15.119:0/514512155 >> v2:172.21.15.132:3300/0 conn(0x7f85380cf620 0x7f85380cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.454+0000 7f8535600640 1 -- 172.21.15.119:0/514512155 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8538160390 con 0x7f8538144550 2024-04-25T10:11:33.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.454+0000 7f8535600640 1 -- 172.21.15.119:0/514512155 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f8528001670 con 0x7f8538144550 2024-04-25T10:11:33.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.454+0000 7f8535600640 1 -- 172.21.15.119:0/514512155 >> v1:172.21.15.119:6789/0 conn(0x7f8538144550 legacy=0x7f853815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.455+0000 7f8535600640 1 --2- 172.21.15.119:0/514512155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8538144d00 0x7f851c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.455+0000 7f8535600640 1 --2- 172.21.15.119:0/514512155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f851c003fe0 0x7f851c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.455+0000 7f8535600640 1 --2- 172.21.15.119:0/514512155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851c004d40 0x7f851c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.455+0000 7f8535600640 1 -- 172.21.15.119:0/514512155 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85381600b0 con 0x7f851c004d40 2024-04-25T10:11:33.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.455+0000 7f8535600640 1 -- 172.21.15.119:0/514512155 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c00ba70 con 0x7f851c003fe0 2024-04-25T10:11:33.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.455+0000 7f8535600640 1 -- 172.21.15.119:0/514512155 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c00bc40 con 0x7f8538144d00 2024-04-25T10:11:33.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.455+0000 7f8537400640 1 --2- 172.21.15.119:0/514512155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851c004d40 0x7f851c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.455+0000 7f8536000640 1 --2- 172.21.15.119:0/514512155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f851c003fe0 0x7f851c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.455+0000 7f8536a00640 1 --2- 172.21.15.119:0/514512155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8538144d00 0x7f851c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.455+0000 7f8537400640 1 -- 172.21.15.119:0/514512155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8538144d00 msgr2=0x7f851c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.455+0000 7f8537400640 1 --2- 172.21.15.119:0/514512155 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8538144d00 0x7f851c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.455+0000 7f8537400640 1 -- 172.21.15.119:0/514512155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f851c003fe0 msgr2=0x7f851c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.455+0000 7f8537400640 1 --2- 172.21.15.119:0/514512155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f851c003fe0 0x7f851c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.455+0000 7f8537400640 1 -- 172.21.15.119:0/514512155 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c003520 con 0x7f851c004d40 2024-04-25T10:11:33.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.456+0000 7f8537400640 1 --2- 172.21.15.119:0/514512155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851c004d40 0x7f851c0051c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8530007360 tx=0x7f851c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4abda54ca89c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.457+0000 7f8535600640 1 -- 172.21.15.119:0/514512155 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530009be0 con 0x7f851c004d40 2024-04-25T10:11:33.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.457+0000 7f8535600640 1 -- 172.21.15.119:0/514512155 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853002e620 con 0x7f851c004d40 2024-04-25T10:11:33.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.457+0000 7f8535600640 1 -- 172.21.15.119:0/514512155 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853002e9f0 con 0x7f851c004d40 2024-04-25T10:11:33.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.457+0000 7f853d600640 1 -- 172.21.15.119:0/514512155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851c004d40 msgr2=0x7f851c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.457+0000 7f853d600640 1 --2- 172.21.15.119:0/514512155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851c004d40 0x7f851c0051c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8530007360 tx=0x7f851c005830 comp rx=0 tx=0).stop 2024-04-25T10:11:33.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.457+0000 7f8537400640 1 -- 172.21.15.119:0/514512155 reap_dead start 2024-04-25T10:11:33.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.457+0000 7f853d600640 1 -- 172.21.15.119:0/514512155 shutdown_connections 2024-04-25T10:11:33.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.457+0000 7f853d600640 1 -- 172.21.15.119:0/514512155 >> 172.21.15.119:0/514512155 conn(0x7f853806aba0 msgr2=0x7f853806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.458+0000 7f853d600640 1 -- 172.21.15.119:0/514512155 shutdown_connections 2024-04-25T10:11:33.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.458+0000 7f853d600640 1 -- 172.21.15.119:0/514512155 wait complete. 2024-04-25T10:11:33.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.459+0000 7f853d600640 1 Processor -- start 2024-04-25T10:11:33.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.460+0000 7f853d600640 1 -- start start 2024-04-25T10:11:33.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f853d600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f851c004d40 0x7f853813f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f853d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85380a84a0 0x7f85381f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f853d600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85381f3720 0x7f853806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f853d600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8538144930 con 0x7f85380a84a0 2024-04-25T10:11:33.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f853d600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85380cc190 con 0x7f85381f3720 2024-04-25T10:11:33.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f853d600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8538144ab0 con 0x7f851c004d40 2024-04-25T10:11:33.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f8536000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85380a84a0 0x7f85381f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f8536000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85380a84a0 0x7f85381f31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42928/0 (socket says 172.21.15.119:42928) 2024-04-25T10:11:33.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f8536000640 1 -- 172.21.15.119:0/1925069970 learned_addr learned my addr 172.21.15.119:0/1925069970 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:33.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f8536a00640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f851c004d40 0x7f853813f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f8537400640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85381f3720 0x7f853806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f8536000640 1 -- 172.21.15.119:0/1925069970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f851c004d40 msgr2=0x7f853813f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f8536000640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f851c004d40 0x7f853813f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f8536000640 1 -- 172.21.15.119:0/1925069970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85381f3720 msgr2=0x7f853806e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f8536000640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85381f3720 0x7f853806e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f8536000640 1 -- 172.21.15.119:0/1925069970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853815d450 con 0x7f85380a84a0 2024-04-25T10:11:33.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.461+0000 7f8536a00640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f851c004d40 0x7f853813f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:33.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.462+0000 7f8536000640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85380a84a0 0x7f85381f31e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f852400f580 tx=0x7f852400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.462+0000 7f852ea00640 1 -- 172.21.15.119:0/1925069970 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524013810 con 0x7f85380a84a0 2024-04-25T10:11:33.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.462+0000 7f852ea00640 1 -- 172.21.15.119:0/1925069970 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852401a070 con 0x7f85380a84a0 2024-04-25T10:11:33.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.462+0000 7f852ea00640 1 -- 172.21.15.119:0/1925069970 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524013bb0 con 0x7f85380a84a0 2024-04-25T10:11:33.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.462+0000 7f853d600640 1 -- 172.21.15.119:0/1925069970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853815d730 con 0x7f85380a84a0 2024-04-25T10:11:33.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.462+0000 7f853d600640 1 -- 172.21.15.119:0/1925069970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853815dcf0 con 0x7f85380a84a0 2024-04-25T10:11:33.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.463+0000 7f853d600640 1 -- 172.21.15.119:0/1925069970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85380749b0 con 0x7f85380a84a0 2024-04-25T10:11:33.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.465+0000 7f852ea00640 1 -- 172.21.15.119:0/1925069970 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8524012070 con 0x7f85380a84a0 2024-04-25T10:11:33.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.465+0000 7f852ea00640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f851806b4d0 0x7f851806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.466+0000 7f852ea00640 1 -- 172.21.15.119:0/1925069970 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f852408a0e0 con 0x7f85380a84a0 2024-04-25T10:11:33.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.466+0000 7f8536a00640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f851806b4d0 0x7f851806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.466+0000 7f8536a00640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f851806b4d0 0x7f851806d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f85280059f0 tx=0x7f8528001580 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.470+0000 7f852ea00640 1 -- 172.21.15.119:0/1925069970 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85240590b0 con 0x7f85380a84a0 2024-04-25T10:11:33.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:33.656+0000 7f853d600640 1 -- 172.21.15.119:0/1925069970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8538074530 con 0x7f85380a84a0 2024-04-25T10:11:34.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.248+0000 7f852ea00640 1 -- 172.21.15.119:0/1925069970 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) v1 ==== 170+0+0 (secure 0 0 0) 0x7f85240589b0 con 0x7f85380a84a0 2024-04-25T10:11:34.254 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2024-04-25T10:11:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.252+0000 7f853d600640 1 -- 172.21.15.119:0/1925069970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f851806b4d0 msgr2=0x7f851806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.252+0000 7f853d600640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f851806b4d0 0x7f851806d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f85280059f0 tx=0x7f8528001580 comp rx=0 tx=0).stop 2024-04-25T10:11:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.252+0000 7f853d600640 1 -- 172.21.15.119:0/1925069970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85380a84a0 msgr2=0x7f85381f31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.252+0000 7f853d600640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85380a84a0 0x7f85381f31e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f852400f580 tx=0x7f852400fa40 comp rx=0 tx=0).stop 2024-04-25T10:11:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.252+0000 7f853d600640 1 -- 172.21.15.119:0/1925069970 shutdown_connections 2024-04-25T10:11:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.252+0000 7f853d600640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f851806b4d0 0x7f851806d990 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.252+0000 7f853d600640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85381f3720 0x7f853806e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.252+0000 7f853d600640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85380a84a0 0x7f85381f31e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.252+0000 7f853d600640 1 --2- 172.21.15.119:0/1925069970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f851c004d40 0x7f853813f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.252+0000 7f853d600640 1 -- 172.21.15.119:0/1925069970 >> 172.21.15.119:0/1925069970 conn(0x7f853806aba0 msgr2=0x7f8538071220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:34.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.253+0000 7f853d600640 1 -- 172.21.15.119:0/1925069970 shutdown_connections 2024-04-25T10:11:34.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.253+0000 7f853d600640 1 -- 172.21.15.119:0/1925069970 wait complete. 2024-04-25T10:11:34.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:11:34.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.464+0000 7f53f7800640 1 Processor -- start 2024-04-25T10:11:34.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.464+0000 7f53f7800640 1 -- start start 2024-04-25T10:11:34.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.465+0000 7f53f7800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f00cec80 0x7f53f006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.465+0000 7f53f7800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f53f006b150 0x7f53f006dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.465+0000 7f53f7800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f006c500 con 0x7f53f015a190 2024-04-25T10:11:34.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.465+0000 7f53f7800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f006b980 con 0x7f53f006b150 2024-04-25T10:11:34.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.465+0000 7f53f7800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f006bc60 con 0x7f53f00cec80 2024-04-25T10:11:34.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.466+0000 7f53f4e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f53f015a190 0x7f53f00ce570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52738/0 (socket says 172.21.15.119:52738) 2024-04-25T10:11:34.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.466+0000 7f53f4e00640 1 -- 172.21.15.119:0/3368629319 learned_addr learned my addr 172.21.15.119:0/3368629319 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:34.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.466+0000 7f53f5800640 1 --2- 172.21.15.119:0/3368629319 >> v2:172.21.15.119:3300/0 conn(0x7f53f006b150 0x7f53f006dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.466+0000 7f53efe00640 1 --2- 172.21.15.119:0/3368629319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f00cec80 0x7f53f006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.466+0000 7f53ef400640 1 -- 172.21.15.119:0/3368629319 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250948205 0 0) 0x7f53f006c500 con 0x7f53f015a190 2024-04-25T10:11:34.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.466+0000 7f53ef400640 1 -- 172.21.15.119:0/3368629319 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53d8005730 con 0x7f53f015a190 2024-04-25T10:11:34.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.467+0000 7f53f5800640 1 -- 172.21.15.119:0/3368629319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f00cec80 msgr2=0x7f53f006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.467+0000 7f53f5800640 1 --2- 172.21.15.119:0/3368629319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f00cec80 0x7f53f006ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.467+0000 7f53f5800640 1 -- 172.21.15.119:0/3368629319 >> v1:172.21.15.119:6789/0 conn(0x7f53f015a190 legacy=0x7f53f00ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.467+0000 7f53f5800640 1 -- 172.21.15.119:0/3368629319 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f006bf40 con 0x7f53f006b150 2024-04-25T10:11:34.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.467+0000 7f53efe00640 1 --2- 172.21.15.119:0/3368629319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f00cec80 0x7f53f006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:34.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.468+0000 7f53f5800640 1 --2- 172.21.15.119:0/3368629319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006b150 0x7f53f006dee0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009d90 tx=0x7f53e802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375a5158b7bccace server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.468+0000 7f53ef400640 1 -- 172.21.15.119:0/3368629319 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8007740 con 0x7f53f006b150 2024-04-25T10:11:34.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.468+0000 7f53ef400640 1 -- 172.21.15.119:0/3368629319 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e80078e0 con 0x7f53f006b150 2024-04-25T10:11:34.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.468+0000 7f53ef400640 1 -- 172.21.15.119:0/3368629319 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8007cb0 con 0x7f53f006b150 2024-04-25T10:11:34.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.468+0000 7f53f7800640 1 -- 172.21.15.119:0/3368629319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006b150 msgr2=0x7f53f006dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.468+0000 7f53f7800640 1 --2- 172.21.15.119:0/3368629319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006b150 0x7f53f006dee0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009d90 tx=0x7f53e802a660 comp rx=0 tx=0).stop 2024-04-25T10:11:34.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.469+0000 7f53f7800640 1 -- 172.21.15.119:0/3368629319 shutdown_connections 2024-04-25T10:11:34.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.469+0000 7f53f7800640 1 --2- 172.21.15.119:0/3368629319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006b150 0x7f53f006dee0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.469+0000 7f53f7800640 1 --2- 172.21.15.119:0/3368629319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f00cec80 0x7f53f006ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.469+0000 7f53f7800640 1 -- 172.21.15.119:0/3368629319 >> 172.21.15.119:0/3368629319 conn(0x7f53f00cca00 msgr2=0x7f53f00cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:34.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.469+0000 7f53f7800640 1 -- 172.21.15.119:0/3368629319 shutdown_connections 2024-04-25T10:11:34.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.469+0000 7f53f7800640 1 -- 172.21.15.119:0/3368629319 wait complete. 2024-04-25T10:11:34.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.470+0000 7f53f7800640 1 Processor -- start 2024-04-25T10:11:34.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.471+0000 7f53f7800640 1 -- start start 2024-04-25T10:11:34.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.471+0000 7f53f7800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006b150 0x7f53f0150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.471+0000 7f53f7800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f00cec80 0x7f53f00afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.471+0000 7f53f7800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f015a190 0x7f53f00b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.471+0000 7f53f7800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f0066960 con 0x7f53f006b150 2024-04-25T10:11:34.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.471+0000 7f53f7800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f006b600 con 0x7f53f00cec80 2024-04-25T10:11:34.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.471+0000 7f53f7800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f0066ae0 con 0x7f53f015a190 2024-04-25T10:11:34.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.472+0000 7f53f4e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006b150 0x7f53f0150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.472+0000 7f53f4e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006b150 0x7f53f0150840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42936/0 (socket says 172.21.15.119:42936) 2024-04-25T10:11:34.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.472+0000 7f53f4e00640 1 -- 172.21.15.119:0/1394561098 learned_addr learned my addr 172.21.15.119:0/1394561098 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:34.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.472+0000 7f53efe00640 1 --2- 172.21.15.119:0/1394561098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f00cec80 0x7f53f00afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.472+0000 7f53f5800640 1 --2- 172.21.15.119:0/1394561098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f015a190 0x7f53f00b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.472+0000 7f53f4e00640 1 -- 172.21.15.119:0/1394561098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f015a190 msgr2=0x7f53f00b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.472+0000 7f53f4e00640 1 --2- 172.21.15.119:0/1394561098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f015a190 0x7f53f00b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.472+0000 7f53f4e00640 1 -- 172.21.15.119:0/1394561098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f00cec80 msgr2=0x7f53f00afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.472+0000 7f53f4e00640 1 --2- 172.21.15.119:0/1394561098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f00cec80 0x7f53f00afde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.472+0000 7f53f4e00640 1 -- 172.21.15.119:0/1394561098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f00c6360 con 0x7f53f006b150 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.472+0000 7f53f4e00640 1 --2- 172.21.15.119:0/1394561098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006b150 0x7f53f0150840 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f53e400c390 tx=0x7f53e400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.473+0000 7f53ecc00640 1 -- 172.21.15.119:0/1394561098 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4017070 con 0x7f53f006b150 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.473+0000 7f53ecc00640 1 -- 172.21.15.119:0/1394561098 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e4005ce0 con 0x7f53f006b150 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.473+0000 7f53f7800640 1 -- 172.21.15.119:0/1394561098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f00c6640 con 0x7f53f006b150 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.473+0000 7f53ecc00640 1 -- 172.21.15.119:0/1394561098 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e40072f0 con 0x7f53f006b150 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.473+0000 7f53f7800640 1 -- 172.21.15.119:0/1394561098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f015a780 con 0x7f53f006b150 2024-04-25T10:11:34.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.474+0000 7f53f7800640 1 -- 172.21.15.119:0/1394561098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f00750b0 con 0x7f53f006b150 2024-04-25T10:11:34.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.475+0000 7f53ecc00640 1 -- 172.21.15.119:0/1394561098 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53e401a070 con 0x7f53f006b150 2024-04-25T10:11:34.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.476+0000 7f53ecc00640 1 --2- 172.21.15.119:0/1394561098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f53d006b4d0 0x7f53d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.476+0000 7f53ecc00640 1 -- 172.21.15.119:0/1394561098 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f53e4089650 con 0x7f53f006b150 2024-04-25T10:11:34.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.476+0000 7f53efe00640 1 --2- 172.21.15.119:0/1394561098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f53d006b4d0 0x7f53d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.477+0000 7f53efe00640 1 --2- 172.21.15.119:0/1394561098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f53d006b4d0 0x7f53d006d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f53f0154370 tx=0x7f53dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.481+0000 7f53ecc00640 1 -- 172.21.15.119:0/1394561098 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53e4058490 con 0x7f53f006b150 2024-04-25T10:11:34.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:34.709+0000 7f53f7800640 1 -- 172.21.15.119:0/1394561098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f53f006ad20 con 0x7f53f006b150 2024-04-25T10:11:35.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.280+0000 7f53ecc00640 1 -- 172.21.15.119:0/1394561098 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v40) v1 ==== 174+0+0 (secure 0 0 0) 0x7f53e4057d90 con 0x7f53f006b150 2024-04-25T10:11:35.290 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 10 and data pool 11 2024-04-25T10:11:35.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.287+0000 7f53f7800640 1 -- 172.21.15.119:0/1394561098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f53d006b4d0 msgr2=0x7f53d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.287+0000 7f53f7800640 1 --2- 172.21.15.119:0/1394561098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f53d006b4d0 0x7f53d006d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f53f0154370 tx=0x7f53dc008040 comp rx=0 tx=0).stop 2024-04-25T10:11:35.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.287+0000 7f53f7800640 1 -- 172.21.15.119:0/1394561098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006b150 msgr2=0x7f53f0150840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.287+0000 7f53f7800640 1 --2- 172.21.15.119:0/1394561098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006b150 0x7f53f0150840 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f53e400c390 tx=0x7f53e400c850 comp rx=0 tx=0).stop 2024-04-25T10:11:35.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.288+0000 7f53f7800640 1 -- 172.21.15.119:0/1394561098 shutdown_connections 2024-04-25T10:11:35.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.288+0000 7f53f7800640 1 --2- 172.21.15.119:0/1394561098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f53d006b4d0 0x7f53d006d990 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.288+0000 7f53f7800640 1 --2- 172.21.15.119:0/1394561098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f015a190 0x7f53f00b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.288+0000 7f53f7800640 1 --2- 172.21.15.119:0/1394561098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f00cec80 0x7f53f00afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.288+0000 7f53f7800640 1 --2- 172.21.15.119:0/1394561098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006b150 0x7f53f0150840 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.288+0000 7f53f7800640 1 -- 172.21.15.119:0/1394561098 >> 172.21.15.119:0/1394561098 conn(0x7f53f00cca00 msgr2=0x7f53f00b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:35.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.288+0000 7f53f7800640 1 -- 172.21.15.119:0/1394561098 shutdown_connections 2024-04-25T10:11:35.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.290+0000 7f53f7800640 1 -- 172.21.15.119:0/1394561098 wait complete. 2024-04-25T10:11:35.327 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.494+0000 7fd482600640 1 Processor -- start 2024-04-25T10:11:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.494+0000 7fd482600640 1 -- start start 2024-04-25T10:11:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.494+0000 7fd482600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd47c0b2e30 0x7fd47c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.494+0000 7fd482600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47c06d550 con 0x7fd47c071190 2024-04-25T10:11:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.494+0000 7fd482600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47c06d830 con 0x7fd47c0ae9a0 2024-04-25T10:11:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.494+0000 7fd482600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd47c06db10 con 0x7fd47c0b2e30 2024-04-25T10:11:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.494+0000 7fd47b400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd47c071190 0x7fd47c071570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52752/0 (socket says 172.21.15.119:52752) 2024-04-25T10:11:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.494+0000 7fd47b400640 1 -- 172.21.15.119:0/1302426708 learned_addr learned my addr 172.21.15.119:0/1302426708 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.495+0000 7fd47be00640 1 --2- 172.21.15.119:0/1302426708 >> v2:172.21.15.119:3300/0 conn(0x7fd47c0b2e30 0x7fd47c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.495+0000 7fd47a000640 1 -- 172.21.15.119:0/1302426708 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709698946 0 0) 0x7fd47c06d550 con 0x7fd47c071190 2024-04-25T10:11:35.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.495+0000 7fd47a000640 1 -- 172.21.15.119:0/1302426708 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd460003520 con 0x7fd47c071190 2024-04-25T10:11:35.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.495+0000 7fd47be00640 1 -- 172.21.15.119:0/1302426708 >> v1:172.21.15.132:6789/0 conn(0x7fd47c0ae9a0 legacy=0x7fd47c0b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:35.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.495+0000 7fd47be00640 1 -- 172.21.15.119:0/1302426708 >> v1:172.21.15.119:6789/0 conn(0x7fd47c071190 legacy=0x7fd47c071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.495+0000 7fd47be00640 1 -- 172.21.15.119:0/1302426708 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c06ddf0 con 0x7fd47c0b2e30 2024-04-25T10:11:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.496+0000 7fd47be00640 1 --2- 172.21.15.119:0/1302426708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47c0b2e30 0x7fd47c0b32c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd474009dc0 tx=0x7fd47402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c45ba615a882356 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.497+0000 7fd47a000640 1 -- 172.21.15.119:0/1302426708 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474007740 con 0x7fd47c0b2e30 2024-04-25T10:11:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.497+0000 7fd47a000640 1 -- 172.21.15.119:0/1302426708 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4740078e0 con 0x7fd47c0b2e30 2024-04-25T10:11:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.497+0000 7fd47a000640 1 -- 172.21.15.119:0/1302426708 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474007cb0 con 0x7fd47c0b2e30 2024-04-25T10:11:35.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.497+0000 7fd482600640 1 -- 172.21.15.119:0/1302426708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47c0b2e30 msgr2=0x7fd47c0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.497+0000 7fd482600640 1 --2- 172.21.15.119:0/1302426708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47c0b2e30 0x7fd47c0b32c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd474009dc0 tx=0x7fd47402b7e0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.497+0000 7fd482600640 1 -- 172.21.15.119:0/1302426708 shutdown_connections 2024-04-25T10:11:35.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.497+0000 7fd482600640 1 --2- 172.21.15.119:0/1302426708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47c0b2e30 0x7fd47c0b32c0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.497+0000 7fd482600640 1 -- 172.21.15.119:0/1302426708 >> 172.21.15.119:0/1302426708 conn(0x7fd47c06aba0 msgr2=0x7fd47c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:35.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.497+0000 7fd482600640 1 -- 172.21.15.119:0/1302426708 shutdown_connections 2024-04-25T10:11:35.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.498+0000 7fd482600640 1 -- 172.21.15.119:0/1302426708 wait complete. 2024-04-25T10:11:35.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.499+0000 7fd482600640 1 Processor -- start 2024-04-25T10:11:35.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.499+0000 7fd482600640 1 -- start start 2024-04-25T10:11:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.500+0000 7fd482600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47c071190 0x7fd47c0b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.500+0000 7fd482600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd47c0ae9a0 0x7fd47c14c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.500+0000 7fd482600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd47c0b2e30 0x7fd47c152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.500+0000 7fd482600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c06e8b0 con 0x7fd47c071190 2024-04-25T10:11:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.500+0000 7fd482600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c0aee30 con 0x7fd47c0b2e30 2024-04-25T10:11:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.500+0000 7fd482600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c06d1b0 con 0x7fd47c0ae9a0 2024-04-25T10:11:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.500+0000 7fd47b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47c071190 0x7fd47c0b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.501+0000 7fd47b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47c071190 0x7fd47c0b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42956/0 (socket says 172.21.15.119:42956) 2024-04-25T10:11:35.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.501+0000 7fd47b400640 1 -- 172.21.15.119:0/2494253811 learned_addr learned my addr 172.21.15.119:0/2494253811 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:35.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.501+0000 7fd47be00640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd47c0b2e30 0x7fd47c152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.501+0000 7fd47aa00640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd47c0ae9a0 0x7fd47c14c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.501+0000 7fd47b400640 1 -- 172.21.15.119:0/2494253811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd47c0ae9a0 msgr2=0x7fd47c14c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.501+0000 7fd47b400640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd47c0ae9a0 0x7fd47c14c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.501+0000 7fd47b400640 1 -- 172.21.15.119:0/2494253811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd47c0b2e30 msgr2=0x7fd47c152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.501+0000 7fd47b400640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd47c0b2e30 0x7fd47c152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.501+0000 7fd47b400640 1 -- 172.21.15.119:0/2494253811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c153f60 con 0x7fd47c071190 2024-04-25T10:11:35.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.501+0000 7fd47be00640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd47c0b2e30 0x7fd47c152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:35.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.501+0000 7fd47b400640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47c071190 0x7fd47c0b1f90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00c430 tx=0x7fd46c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:35.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.502+0000 7fd473400640 1 -- 172.21.15.119:0/2494253811 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c017070 con 0x7fd47c071190 2024-04-25T10:11:35.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.502+0000 7fd473400640 1 -- 172.21.15.119:0/2494253811 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c005ce0 con 0x7fd47c071190 2024-04-25T10:11:35.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.502+0000 7fd473400640 1 -- 172.21.15.119:0/2494253811 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0072f0 con 0x7fd47c071190 2024-04-25T10:11:35.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.502+0000 7fd482600640 1 -- 172.21.15.119:0/2494253811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47c152ec0 con 0x7fd47c071190 2024-04-25T10:11:35.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.502+0000 7fd482600640 1 -- 172.21.15.119:0/2494253811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47c153480 con 0x7fd47c071190 2024-04-25T10:11:35.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.503+0000 7fd482600640 1 -- 172.21.15.119:0/2494253811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd47c0749b0 con 0x7fd47c071190 2024-04-25T10:11:35.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.504+0000 7fd473400640 1 -- 172.21.15.119:0/2494253811 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd46c01a070 con 0x7fd47c071190 2024-04-25T10:11:35.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.505+0000 7fd473400640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd45c06b4d0 0x7fd45c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.505+0000 7fd473400640 1 -- 172.21.15.119:0/2494253811 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd46c0896f0 con 0x7fd47c071190 2024-04-25T10:11:35.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.505+0000 7fd47aa00640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd45c06b4d0 0x7fd45c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.506+0000 7fd47aa00640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd45c06b4d0 0x7fd45c06d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd47c06b080 tx=0x7fd468009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:35.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.509+0000 7fd473400640 1 -- 172.21.15.119:0/2494253811 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd46c058560 con 0x7fd47c071190 2024-04-25T10:11:35.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.705+0000 7fd482600640 1 -- 172.21.15.119:0/2494253811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd47c0cbdc0 con 0x7fd47c071190 2024-04-25T10:11:35.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.707+0000 7fd473400640 1 -- 172.21.15.119:0/2494253811 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16717 (secure 0 0 0) 0x7fd46c057e60 con 0x7fd47c071190 2024-04-25T10:11:35.712 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:35.712 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":66,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:11:34.711452+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:32.500199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:33.708192+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:35.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.710+0000 7fd482600640 1 -- 172.21.15.119:0/2494253811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd45c06b4d0 msgr2=0x7fd45c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.710+0000 7fd482600640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd45c06b4d0 0x7fd45c06d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd47c06b080 tx=0x7fd468009040 comp rx=0 tx=0).stop 2024-04-25T10:11:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.710+0000 7fd482600640 1 -- 172.21.15.119:0/2494253811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47c071190 msgr2=0x7fd47c0b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.710+0000 7fd482600640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47c071190 0x7fd47c0b1f90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00c430 tx=0x7fd46c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.710+0000 7fd482600640 1 -- 172.21.15.119:0/2494253811 shutdown_connections 2024-04-25T10:11:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.710+0000 7fd482600640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd45c06b4d0 0x7fd45c06d990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.710+0000 7fd482600640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd47c0b2e30 0x7fd47c152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.710+0000 7fd482600640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd47c0ae9a0 0x7fd47c14c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.710+0000 7fd482600640 1 --2- 172.21.15.119:0/2494253811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd47c071190 0x7fd47c0b1f90 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.710+0000 7fd482600640 1 -- 172.21.15.119:0/2494253811 >> 172.21.15.119:0/2494253811 conn(0x7fd47c06aba0 msgr2=0x7fd47c15da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.710+0000 7fd482600640 1 -- 172.21.15.119:0/2494253811 shutdown_connections 2024-04-25T10:11:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.710+0000 7fd482600640 1 -- 172.21.15.119:0/2494253811 wait complete. 2024-04-25T10:11:35.733 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.925+0000 7f941ea00640 1 Processor -- start 2024-04-25T10:11:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.925+0000 7f941ea00640 1 -- start start 2024-04-25T10:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.926+0000 7f941ea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f94180b5b60 0x7f94181687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.926+0000 7f941ea00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941815e0c0 con 0x7f941815dce0 2024-04-25T10:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.926+0000 7f941ea00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94180d2510 con 0x7f94180d35b0 2024-04-25T10:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.926+0000 7f941ea00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94180b7b10 con 0x7f94180b5b60 2024-04-25T10:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.926+0000 7f9417e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f941815dce0 0x7f94180d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52764/0 (socket says 172.21.15.119:52764) 2024-04-25T10:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.926+0000 7f9417e00640 1 -- 172.21.15.119:0/2412439962 learned_addr learned my addr 172.21.15.119:0/2412439962 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.926+0000 7f9416a00640 1 -- 172.21.15.119:0/2412439962 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799164609 0 0) 0x7f941815e0c0 con 0x7f941815dce0 2024-04-25T10:11:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9416a00640 1 -- 172.21.15.119:0/2412439962 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93fc003520 con 0x7f941815dce0 2024-04-25T10:11:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f941ca00640 1 --2- 172.21.15.119:0/2412439962 >> v2:172.21.15.132:3300/0 conn(0x7f94180b5b60 0x7f94181687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9416a00640 1 -- 172.21.15.119:0/2412439962 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 554032666 0 0) 0x7f93fc003520 con 0x7f941815dce0 2024-04-25T10:11:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9416a00640 1 -- 172.21.15.119:0/2412439962 >> v2:172.21.15.132:3300/0 conn(0x7f94180b5b60 msgr2=0x7f94181687f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9416a00640 1 --2- 172.21.15.119:0/2412439962 >> v2:172.21.15.132:3300/0 conn(0x7f94180b5b60 0x7f94181687f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9416a00640 1 -- 172.21.15.119:0/2412439962 >> v1:172.21.15.132:6789/0 conn(0x7f94180d35b0 legacy=0x7f94180b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9416a00640 1 -- 172.21.15.119:0/2412439962 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180d27f0 con 0x7f941815dce0 2024-04-25T10:11:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9416a00640 1 -- 172.21.15.119:0/2412439962 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f9408001670 con 0x7f941815dce0 2024-04-25T10:11:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9416a00640 1 -- 172.21.15.119:0/2412439962 >> v1:172.21.15.119:6789/0 conn(0x7f941815dce0 legacy=0x7f94180d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9416a00640 1 --2- 172.21.15.119:0/2412439962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc003a00 0x7f93fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9416a00640 1 --2- 172.21.15.119:0/2412439962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93fc004800 0x7f93fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9416a00640 1 --2- 172.21.15.119:0/2412439962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93fc005210 0x7f93fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9416a00640 1 -- 172.21.15.119:0/2412439962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94180b7b10 con 0x7f93fc003a00 2024-04-25T10:11:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9416a00640 1 -- 172.21.15.119:0/2412439962 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc00c140 con 0x7f93fc004800 2024-04-25T10:11:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9416a00640 1 -- 172.21.15.119:0/2412439962 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc00c340 con 0x7f93fc005210 2024-04-25T10:11:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.927+0000 7f9417e00640 1 --2- 172.21.15.119:0/2412439962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc003a00 0x7f93fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.928+0000 7f941ca00640 1 --2- 172.21.15.119:0/2412439962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93fc005210 0x7f93fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.928+0000 7f9417400640 1 --2- 172.21.15.119:0/2412439962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93fc004800 0x7f93fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.928+0000 7f9417e00640 1 -- 172.21.15.119:0/2412439962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93fc005210 msgr2=0x7f93fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.928+0000 7f9417e00640 1 --2- 172.21.15.119:0/2412439962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93fc005210 0x7f93fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.928+0000 7f9417e00640 1 -- 172.21.15.119:0/2412439962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93fc004800 msgr2=0x7f93fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.928+0000 7f9417e00640 1 --2- 172.21.15.119:0/2412439962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93fc004800 0x7f93fc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.928+0000 7f941ca00640 1 -- 172.21.15.119:0/2412439962 reap_dead start 2024-04-25T10:11:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.928+0000 7f9417e00640 1 -- 172.21.15.119:0/2412439962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc003520 con 0x7f93fc003a00 2024-04-25T10:11:35.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.929+0000 7f9417e00640 1 --2- 172.21.15.119:0/2412439962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc003a00 0x7f93fc003e00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f940800c160 tx=0x7f93fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9c7c6d8eeb71e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:35.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.929+0000 7f9416a00640 1 -- 172.21.15.119:0/2412439962 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408007c60 con 0x7f93fc003a00 2024-04-25T10:11:35.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.929+0000 7f9416a00640 1 -- 172.21.15.119:0/2412439962 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9408005ad0 con 0x7f93fc003a00 2024-04-25T10:11:35.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.929+0000 7f9416a00640 1 -- 172.21.15.119:0/2412439962 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408007070 con 0x7f93fc003a00 2024-04-25T10:11:35.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.930+0000 7f941ea00640 1 -- 172.21.15.119:0/2412439962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc003a00 msgr2=0x7f93fc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.930+0000 7f941ea00640 1 --2- 172.21.15.119:0/2412439962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc003a00 0x7f93fc003e00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f940800c160 tx=0x7f93fc005830 comp rx=0 tx=0).stop 2024-04-25T10:11:35.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.930+0000 7f941ea00640 1 -- 172.21.15.119:0/2412439962 shutdown_connections 2024-04-25T10:11:35.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.930+0000 7f941ea00640 1 --2- 172.21.15.119:0/2412439962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93fc003a00 0x7f93fc003e00 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.930+0000 7f941ea00640 1 -- 172.21.15.119:0/2412439962 >> 172.21.15.119:0/2412439962 conn(0x7f9418072900 msgr2=0x7f94180d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:35.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.930+0000 7f941ea00640 1 -- 172.21.15.119:0/2412439962 shutdown_connections 2024-04-25T10:11:35.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.930+0000 7f941ea00640 1 -- 172.21.15.119:0/2412439962 wait complete. 2024-04-25T10:11:35.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.932+0000 7f941ea00640 1 Processor -- start 2024-04-25T10:11:35.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.932+0000 7f941ea00640 1 -- start start 2024-04-25T10:11:35.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.933+0000 7f941ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93fc003a00 0x7f9418158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.933+0000 7f941ea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94180b8410 0x7f941814ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.933+0000 7f941ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94180b8890 0x7f941814cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.933+0000 7f941ea00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94180d3090 con 0x7f94180b8890 2024-04-25T10:11:35.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.933+0000 7f941ea00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94180d2db0 con 0x7f94180b8410 2024-04-25T10:11:35.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.933+0000 7f941ea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94180b9820 con 0x7f93fc003a00 2024-04-25T10:11:35.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.933+0000 7f941ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94180b8890 0x7f941814cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.933+0000 7f941ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94180b8890 0x7f941814cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42980/0 (socket says 172.21.15.119:42980) 2024-04-25T10:11:35.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.933+0000 7f941ca00640 1 -- 172.21.15.119:0/2544143650 learned_addr learned my addr 172.21.15.119:0/2544143650 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:35.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.934+0000 7f9417400640 1 --2- 172.21.15.119:0/2544143650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94180b8410 0x7f941814ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.934+0000 7f9417e00640 1 --2- 172.21.15.119:0/2544143650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93fc003a00 0x7f9418158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.934+0000 7f941ca00640 1 -- 172.21.15.119:0/2544143650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93fc003a00 msgr2=0x7f9418158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.934+0000 7f941ca00640 1 --2- 172.21.15.119:0/2544143650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93fc003a00 0x7f9418158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.934+0000 7f941ca00640 1 -- 172.21.15.119:0/2544143650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94180b8410 msgr2=0x7f941814ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.934+0000 7f941ca00640 1 --2- 172.21.15.119:0/2544143650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94180b8410 0x7f941814ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.934+0000 7f941ca00640 1 -- 172.21.15.119:0/2544143650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941815efa0 con 0x7f94180b8890 2024-04-25T10:11:35.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.934+0000 7f941ca00640 1 --2- 172.21.15.119:0/2544143650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94180b8890 0x7f941814cfa0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9418161710 tx=0x7f9410007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:35.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.934+0000 7f940fe00640 1 -- 172.21.15.119:0/2544143650 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410014410 con 0x7f94180b8890 2024-04-25T10:11:35.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.934+0000 7f940fe00640 1 -- 172.21.15.119:0/2544143650 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9410019070 con 0x7f94180b8890 2024-04-25T10:11:35.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.934+0000 7f940fe00640 1 -- 172.21.15.119:0/2544143650 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410014810 con 0x7f94180b8890 2024-04-25T10:11:35.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.934+0000 7f941ea00640 1 -- 172.21.15.119:0/2544143650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9418157c40 con 0x7f94180b8890 2024-04-25T10:11:35.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.935+0000 7f941ea00640 1 -- 172.21.15.119:0/2544143650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9418158200 con 0x7f94180b8890 2024-04-25T10:11:35.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.936+0000 7f941ea00640 1 -- 172.21.15.119:0/2544143650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94180749b0 con 0x7f94180b8890 2024-04-25T10:11:35.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.937+0000 7f940fe00640 1 -- 172.21.15.119:0/2544143650 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94100149b0 con 0x7f94180b8890 2024-04-25T10:11:35.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.937+0000 7f940fe00640 1 --2- 172.21.15.119:0/2544143650 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f93f806b4d0 0x7f93f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.938+0000 7f940fe00640 1 -- 172.21.15.119:0/2544143650 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f941008a890 con 0x7f94180b8890 2024-04-25T10:11:35.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.938+0000 7f9417e00640 1 --2- 172.21.15.119:0/2544143650 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f93f806b4d0 0x7f93f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.939+0000 7f9417e00640 1 --2- 172.21.15.119:0/2544143650 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f93f806b4d0 0x7f93f806d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9408032000 tx=0x7f940802d000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:35.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:35.942+0000 7f940fe00640 1 -- 172.21.15.119:0/2544143650 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9410059670 con 0x7f94180b8890 2024-04-25T10:11:36.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.144+0000 7f941ea00640 1 -- 172.21.15.119:0/2544143650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94181583d0 con 0x7f94180b8890 2024-04-25T10:11:36.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.147+0000 7f940fe00640 1 -- 172.21.15.119:0/2544143650 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16717 (secure 0 0 0) 0x7f9410058f70 con 0x7f94180b8890 2024-04-25T10:11:36.152 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:36.152 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":66,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:11:34.711452+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:32.500199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:33.708192+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.149+0000 7f941ea00640 1 -- 172.21.15.119:0/2544143650 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f93f806b4d0 msgr2=0x7f93f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.149+0000 7f941ea00640 1 --2- 172.21.15.119:0/2544143650 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f93f806b4d0 0x7f93f806d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9408032000 tx=0x7f940802d000 comp rx=0 tx=0).stop 2024-04-25T10:11:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.149+0000 7f941ea00640 1 -- 172.21.15.119:0/2544143650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94180b8890 msgr2=0x7f941814cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.149+0000 7f941ea00640 1 --2- 172.21.15.119:0/2544143650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94180b8890 0x7f941814cfa0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9418161710 tx=0x7f9410007700 comp rx=0 tx=0).stop 2024-04-25T10:11:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.149+0000 7f941ea00640 1 -- 172.21.15.119:0/2544143650 shutdown_connections 2024-04-25T10:11:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.149+0000 7f941ea00640 1 --2- 172.21.15.119:0/2544143650 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f93f806b4d0 0x7f93f806d990 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.149+0000 7f941ea00640 1 --2- 172.21.15.119:0/2544143650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94180b8890 0x7f941814cfa0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.149+0000 7f941ea00640 1 --2- 172.21.15.119:0/2544143650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94180b8410 0x7f941814ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.149+0000 7f941ea00640 1 --2- 172.21.15.119:0/2544143650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93fc003a00 0x7f9418158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.149+0000 7f941ea00640 1 -- 172.21.15.119:0/2544143650 >> 172.21.15.119:0/2544143650 conn(0x7f9418072900 msgr2=0x7f94180b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:36.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.150+0000 7f941ea00640 1 -- 172.21.15.119:0/2544143650 shutdown_connections 2024-04-25T10:11:36.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.150+0000 7f941ea00640 1 -- 172.21.15.119:0/2544143650 wait complete. 2024-04-25T10:11:36.172 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:11:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.374+0000 7f3153e00640 1 Processor -- start 2024-04-25T10:11:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.374+0000 7f3153e00640 1 -- start start 2024-04-25T10:11:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.376+0000 7f3153e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f31540ada20 0x7f31540acb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.376+0000 7f3153e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f31540b36a0 0x7f315406b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.376+0000 7f3153e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f315415e120 con 0x7f315406aba0 2024-04-25T10:11:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.376+0000 7f3153e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f315415dec0 con 0x7f31540b36a0 2024-04-25T10:11:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.376+0000 7f3153e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f315415e400 con 0x7f31540ada20 2024-04-25T10:11:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.376+0000 7f314be00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f315406aba0 0x7f31540b2f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52768/0 (socket says 172.21.15.119:52768) 2024-04-25T10:11:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.376+0000 7f3153400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f31540b36a0 0x7f315406b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.376+0000 7f314be00640 1 -- 172.21.15.119:0/2088840803 learned_addr learned my addr 172.21.15.119:0/2088840803 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:36.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.376+0000 7f3152a00640 1 --2- 172.21.15.119:0/2088840803 >> v2:172.21.15.132:3300/0 conn(0x7f31540ada20 0x7f31540acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.377+0000 7f3153400640 1 -- 172.21.15.119:0/2088840803 >> v2:172.21.15.132:3300/0 conn(0x7f31540ada20 msgr2=0x7f31540acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:36.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.376+0000 7f3152000640 1 -- 172.21.15.119:0/2088840803 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191456224 0 0) 0x7f315415e120 con 0x7f315406aba0 2024-04-25T10:11:36.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.377+0000 7f3153400640 1 --2- 172.21.15.119:0/2088840803 >> v2:172.21.15.132:3300/0 conn(0x7f31540ada20 0x7f31540acb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.377+0000 7f3153400640 1 -- 172.21.15.119:0/2088840803 >> v1:172.21.15.119:6789/0 conn(0x7f315406aba0 legacy=0x7f31540b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.377+0000 7f3153400640 1 -- 172.21.15.119:0/2088840803 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315415e6e0 con 0x7f31540b36a0 2024-04-25T10:11:36.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.377+0000 7f3152a00640 1 --2- 172.21.15.119:0/2088840803 >> v2:172.21.15.132:3300/0 conn(0x7f31540ada20 0x7f31540acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:36.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.378+0000 7f3153400640 1 --2- 172.21.15.119:0/2088840803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31540b36a0 0x7f315406b050 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f31540b1ec0 tx=0x7f314c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596a8567db102a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:36.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.378+0000 7f3152000640 1 -- 172.21.15.119:0/2088840803 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c007740 con 0x7f31540b36a0 2024-04-25T10:11:36.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.378+0000 7f3152000640 1 -- 172.21.15.119:0/2088840803 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c0078e0 con 0x7f31540b36a0 2024-04-25T10:11:36.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.378+0000 7f3152000640 1 -- 172.21.15.119:0/2088840803 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c007cb0 con 0x7f31540b36a0 2024-04-25T10:11:36.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.379+0000 7f3153e00640 1 -- 172.21.15.119:0/2088840803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31540b36a0 msgr2=0x7f315406b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.379+0000 7f3153e00640 1 --2- 172.21.15.119:0/2088840803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31540b36a0 0x7f315406b050 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f31540b1ec0 tx=0x7f314c02a680 comp rx=0 tx=0).stop 2024-04-25T10:11:36.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.379+0000 7f3153e00640 1 -- 172.21.15.119:0/2088840803 shutdown_connections 2024-04-25T10:11:36.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.379+0000 7f3153e00640 1 --2- 172.21.15.119:0/2088840803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31540b36a0 0x7f315406b050 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.379+0000 7f3153e00640 1 --2- 172.21.15.119:0/2088840803 >> v2:172.21.15.132:3300/0 conn(0x7f31540ada20 0x7f31540acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.379+0000 7f3153e00640 1 -- 172.21.15.119:0/2088840803 >> 172.21.15.119:0/2088840803 conn(0x7f3154066ff0 msgr2=0x7f31540b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:36.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.379+0000 7f3153e00640 1 -- 172.21.15.119:0/2088840803 shutdown_connections 2024-04-25T10:11:36.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.379+0000 7f3153e00640 1 -- 172.21.15.119:0/2088840803 wait complete. 2024-04-25T10:11:36.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.381+0000 7f3153e00640 1 Processor -- start 2024-04-25T10:11:36.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.381+0000 7f3153e00640 1 -- start start 2024-04-25T10:11:36.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f3153e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f315406aba0 0x7f3154152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f3153e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31540ada20 0x7f3154143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f3153e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31540b36a0 0x7f315414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f3153e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31540ade00 con 0x7f31540ada20 2024-04-25T10:11:36.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f3153e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315415eca0 con 0x7f315406aba0 2024-04-25T10:11:36.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f3153e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3154157700 con 0x7f31540b36a0 2024-04-25T10:11:36.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f314be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31540ada20 0x7f3154143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f314be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31540ada20 0x7f3154143eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43000/0 (socket says 172.21.15.119:43000) 2024-04-25T10:11:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f314be00640 1 -- 172.21.15.119:0/3025576207 learned_addr learned my addr 172.21.15.119:0/3025576207 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f3152a00640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f315406aba0 0x7f3154152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f314be00640 1 -- 172.21.15.119:0/3025576207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31540b36a0 msgr2=0x7f315414fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f3153400640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31540b36a0 0x7f315414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f314be00640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31540b36a0 0x7f315414fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f314be00640 1 -- 172.21.15.119:0/3025576207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f315406aba0 msgr2=0x7f3154152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f314be00640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f315406aba0 0x7f3154152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f314be00640 1 -- 172.21.15.119:0/3025576207 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31541503d0 con 0x7f31540ada20 2024-04-25T10:11:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.382+0000 7f3152a00640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f315406aba0 0x7f3154152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.383+0000 7f314be00640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31540ada20 0x7f3154143eb0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f313c00c4d0 tx=0x7f313c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.383+0000 7f314b400640 1 -- 172.21.15.119:0/3025576207 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c018070 con 0x7f31540ada20 2024-04-25T10:11:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.383+0000 7f314b400640 1 -- 172.21.15.119:0/3025576207 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c007500 con 0x7f31540ada20 2024-04-25T10:11:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.383+0000 7f3153e00640 1 -- 172.21.15.119:0/3025576207 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3154151280 con 0x7f31540ada20 2024-04-25T10:11:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.383+0000 7f314b400640 1 -- 172.21.15.119:0/3025576207 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c004a20 con 0x7f31540ada20 2024-04-25T10:11:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.383+0000 7f3153e00640 1 -- 172.21.15.119:0/3025576207 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3154151840 con 0x7f31540ada20 2024-04-25T10:11:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.384+0000 7f3153e00640 1 -- 172.21.15.119:0/3025576207 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31540749b0 con 0x7f31540ada20 2024-04-25T10:11:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.385+0000 7f314b400640 1 -- 172.21.15.119:0/3025576207 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f313c01b070 con 0x7f31540ada20 2024-04-25T10:11:36.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.386+0000 7f314b400640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f313006b4d0 0x7f313006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.386+0000 7f314b400640 1 -- 172.21.15.119:0/3025576207 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f313c089520 con 0x7f31540ada20 2024-04-25T10:11:36.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.386+0000 7f3152a00640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f313006b4d0 0x7f313006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.387+0000 7f3152a00640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f313006b4d0 0x7f313006d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3144006fd0 tx=0x7f3144008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:36.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.390+0000 7f314b400640 1 -- 172.21.15.119:0/3025576207 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f313c058390 con 0x7f31540ada20 2024-04-25T10:11:36.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:36.624+0000 7f3153e00640 1 -- 172.21.15.119:0/3025576207 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3154157180 con 0x7f31540ada20 2024-04-25T10:11:37.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.300+0000 7f314b400640 1 -- 172.21.15.119:0/3025576207 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7f313c057c90 con 0x7f31540ada20 2024-04-25T10:11:37.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.304+0000 7f3153e00640 1 -- 172.21.15.119:0/3025576207 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f313006b4d0 msgr2=0x7f313006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.304+0000 7f3153e00640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f313006b4d0 0x7f313006d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3144006fd0 tx=0x7f3144008040 comp rx=0 tx=0).stop 2024-04-25T10:11:37.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.304+0000 7f3153e00640 1 -- 172.21.15.119:0/3025576207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31540ada20 msgr2=0x7f3154143eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.304+0000 7f3153e00640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31540ada20 0x7f3154143eb0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f313c00c4d0 tx=0x7f313c00c990 comp rx=0 tx=0).stop 2024-04-25T10:11:37.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.304+0000 7f3153e00640 1 -- 172.21.15.119:0/3025576207 shutdown_connections 2024-04-25T10:11:37.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.304+0000 7f3153e00640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f313006b4d0 0x7f313006d990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.304+0000 7f3153e00640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31540b36a0 0x7f315414fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.304+0000 7f3153e00640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31540ada20 0x7f3154143eb0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.304+0000 7f3153e00640 1 --2- 172.21.15.119:0/3025576207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f315406aba0 0x7f3154152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.304+0000 7f3153e00640 1 -- 172.21.15.119:0/3025576207 >> 172.21.15.119:0/3025576207 conn(0x7f3154066ff0 msgr2=0x7f31540cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:37.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.304+0000 7f3153e00640 1 -- 172.21.15.119:0/3025576207 shutdown_connections 2024-04-25T10:11:37.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.304+0000 7f3153e00640 1 -- 172.21.15.119:0/3025576207 wait complete. 2024-04-25T10:11:37.329 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:11:37.329 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:11:37.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.530+0000 7f47b2600640 1 Processor -- start 2024-04-25T10:11:37.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.531+0000 7f47b2600640 1 -- start start 2024-04-25T10:11:37.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.532+0000 7f47b2600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f47ac0b5c70 0x7f47ac168830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.532+0000 7f47b2600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ac0b7b80 con 0x7f47ac15dbb0 2024-04-25T10:11:37.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.532+0000 7f47b2600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ac0d2670 con 0x7f47ac0b9720 2024-04-25T10:11:37.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.532+0000 7f47b2600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47ac0d2950 con 0x7f47ac0b5c70 2024-04-25T10:11:37.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.532+0000 7f47ab400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f47ac15dbb0 0x7f47ac0d1e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52770/0 (socket says 172.21.15.119:52770) 2024-04-25T10:11:37.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.532+0000 7f47ab400640 1 -- 172.21.15.119:0/2252266927 learned_addr learned my addr 172.21.15.119:0/2252266927 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:37.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47abe00640 1 --2- 172.21.15.119:0/2252266927 >> v2:172.21.15.132:3300/0 conn(0x7f47ac0b5c70 0x7f47ac168830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 -- 172.21.15.119:0/2252266927 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640611992 0 0) 0x7f47ac0b7b80 con 0x7f47ac15dbb0 2024-04-25T10:11:37.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 -- 172.21.15.119:0/2252266927 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4790003520 con 0x7f47ac15dbb0 2024-04-25T10:11:37.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 -- 172.21.15.119:0/2252266927 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2855749075 0 0) 0x7f4790003520 con 0x7f47ac15dbb0 2024-04-25T10:11:37.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 -- 172.21.15.119:0/2252266927 >> v2:172.21.15.132:3300/0 conn(0x7f47ac0b5c70 msgr2=0x7f47ac168830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 --2- 172.21.15.119:0/2252266927 >> v2:172.21.15.132:3300/0 conn(0x7f47ac0b5c70 0x7f47ac168830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 -- 172.21.15.119:0/2252266927 >> v1:172.21.15.132:6789/0 conn(0x7f47ac0b9720 legacy=0x7f47ac0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 -- 172.21.15.119:0/2252266927 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac0d2c30 con 0x7f47ac15dbb0 2024-04-25T10:11:37.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 -- 172.21.15.119:0/2252266927 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f479c001670 con 0x7f47ac15dbb0 2024-04-25T10:11:37.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 -- 172.21.15.119:0/2252266927 >> v1:172.21.15.119:6789/0 conn(0x7f47ac15dbb0 legacy=0x7f47ac0d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 --2- 172.21.15.119:0/2252266927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47900039b0 0x7f4790003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 --2- 172.21.15.119:0/2252266927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4790004800 0x7f4790004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 --2- 172.21.15.119:0/2252266927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47900051c0 0x7f479000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 -- 172.21.15.119:0/2252266927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac0d2950 con 0x7f47900039b0 2024-04-25T10:11:37.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 -- 172.21.15.119:0/2252266927 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479000c160 con 0x7f47900051c0 2024-04-25T10:11:37.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.533+0000 7f47aa000640 1 -- 172.21.15.119:0/2252266927 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479000c360 con 0x7f4790004800 2024-04-25T10:11:37.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.534+0000 7f47ab400640 1 --2- 172.21.15.119:0/2252266927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47900039b0 0x7f4790003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.534+0000 7f47abe00640 1 --2- 172.21.15.119:0/2252266927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47900051c0 0x7f479000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.534+0000 7f47aaa00640 1 --2- 172.21.15.119:0/2252266927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4790004800 0x7f4790004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.534+0000 7f47ab400640 1 -- 172.21.15.119:0/2252266927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4790004800 msgr2=0x7f4790004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.534+0000 7f47ab400640 1 --2- 172.21.15.119:0/2252266927 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4790004800 0x7f4790004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.534+0000 7f47ab400640 1 -- 172.21.15.119:0/2252266927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47900051c0 msgr2=0x7f479000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.534+0000 7f47ab400640 1 --2- 172.21.15.119:0/2252266927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47900051c0 0x7f479000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.534+0000 7f47ab400640 1 -- 172.21.15.119:0/2252266927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4790003520 con 0x7f47900039b0 2024-04-25T10:11:37.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.534+0000 7f47abe00640 1 -- 172.21.15.119:0/2252266927 reap_dead start 2024-04-25T10:11:37.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.535+0000 7f47ab400640 1 --2- 172.21.15.119:0/2252266927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47900039b0 0x7f4790003dd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f479c00c240 tx=0x7f4790005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dd3986b4662d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.536+0000 7f47aa000640 1 -- 172.21.15.119:0/2252266927 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c007cf0 con 0x7f47900039b0 2024-04-25T10:11:37.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.536+0000 7f47aa000640 1 -- 172.21.15.119:0/2252266927 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c005ad0 con 0x7f47900039b0 2024-04-25T10:11:37.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.536+0000 7f47aa000640 1 -- 172.21.15.119:0/2252266927 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c007070 con 0x7f47900039b0 2024-04-25T10:11:37.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.536+0000 7f47b2600640 1 -- 172.21.15.119:0/2252266927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47900039b0 msgr2=0x7f4790003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.536+0000 7f47b2600640 1 --2- 172.21.15.119:0/2252266927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47900039b0 0x7f4790003dd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f479c00c240 tx=0x7f4790005830 comp rx=0 tx=0).stop 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.536+0000 7f47b2600640 1 -- 172.21.15.119:0/2252266927 shutdown_connections 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.536+0000 7f47b2600640 1 --2- 172.21.15.119:0/2252266927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47900039b0 0x7f4790003dd0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.536+0000 7f47b2600640 1 -- 172.21.15.119:0/2252266927 >> 172.21.15.119:0/2252266927 conn(0x7f47ac0d0560 msgr2=0x7f47ac0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.536+0000 7f47b2600640 1 -- 172.21.15.119:0/2252266927 shutdown_connections 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.536+0000 7f47b2600640 1 -- 172.21.15.119:0/2252266927 wait complete. 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.538+0000 7f47b2600640 1 Processor -- start 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.538+0000 7f47b2600640 1 -- start start 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.539+0000 7f47b2600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47900039b0 0x7f47ac147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.539+0000 7f47b2600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47ac1e3ce0 0x7f47ac0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.539+0000 7f47b2600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47ac1f7c30 0x7f47ac1f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.539+0000 7f47b2600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac15df90 con 0x7f47ac1e3ce0 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.539+0000 7f47b2600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac0d31f0 con 0x7f47ac1f7c30 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.539+0000 7f47b2600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac15e110 con 0x7f47900039b0 2024-04-25T10:11:37.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.540+0000 7f47aaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47ac1e3ce0 0x7f47ac0cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.540+0000 7f47aaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47ac1e3ce0 0x7f47ac0cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43026/0 (socket says 172.21.15.119:43026) 2024-04-25T10:11:37.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.540+0000 7f47aaa00640 1 -- 172.21.15.119:0/1050117711 learned_addr learned my addr 172.21.15.119:0/1050117711 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:37.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.540+0000 7f47ab400640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47900039b0 0x7f47ac147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.540+0000 7f47abe00640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47ac1f7c30 0x7f47ac1f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.540+0000 7f47aaa00640 1 -- 172.21.15.119:0/1050117711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47900039b0 msgr2=0x7f47ac147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.540+0000 7f47aaa00640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47900039b0 0x7f47ac147b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.540+0000 7f47aaa00640 1 -- 172.21.15.119:0/1050117711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47ac1f7c30 msgr2=0x7f47ac1f32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.540+0000 7f47aaa00640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47ac1f7c30 0x7f47ac1f32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.540+0000 7f47aaa00640 1 -- 172.21.15.119:0/1050117711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac144740 con 0x7f47ac1e3ce0 2024-04-25T10:11:37.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.540+0000 7f47abe00640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47ac1f7c30 0x7f47ac1f32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:37.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.540+0000 7f47aaa00640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47ac1e3ce0 0x7f47ac0cbe70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f479400e4b0 tx=0x7f479400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.541+0000 7f47a3400640 1 -- 172.21.15.119:0/1050117711 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47940052e0 con 0x7f47ac1e3ce0 2024-04-25T10:11:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.541+0000 7f47a3400640 1 -- 172.21.15.119:0/1050117711 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4794018070 con 0x7f47ac1e3ce0 2024-04-25T10:11:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.541+0000 7f47a3400640 1 -- 172.21.15.119:0/1050117711 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4794003830 con 0x7f47ac1e3ce0 2024-04-25T10:11:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.541+0000 7f47b2600640 1 -- 172.21.15.119:0/1050117711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ac144a20 con 0x7f47ac1e3ce0 2024-04-25T10:11:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.541+0000 7f47b2600640 1 -- 172.21.15.119:0/1050117711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac144fe0 con 0x7f47ac1e3ce0 2024-04-25T10:11:37.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.542+0000 7f47b2600640 1 -- 172.21.15.119:0/1050117711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ac0750b0 con 0x7f47ac1e3ce0 2024-04-25T10:11:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.543+0000 7f47a3400640 1 -- 172.21.15.119:0/1050117711 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47940039d0 con 0x7f47ac1e3ce0 2024-04-25T10:11:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.544+0000 7f47a3400640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f478806b4d0 0x7f478806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.544+0000 7f47a3400640 1 -- 172.21.15.119:0/1050117711 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4794089e30 con 0x7f47ac1e3ce0 2024-04-25T10:11:37.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.544+0000 7f47ab400640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f478806b4d0 0x7f478806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.545+0000 7f47ab400640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f478806b4d0 0x7f478806d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f479c032000 tx=0x7f479c034040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.548+0000 7f47a3400640 1 -- 172.21.15.119:0/1050117711 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4794058ca0 con 0x7f47ac1e3ce0 2024-04-25T10:11:37.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:37.785+0000 7f47b2600640 1 -- 172.21.15.119:0/1050117711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f47ac0b5920 con 0x7f47ac1e3ce0 2024-04-25T10:11:38.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.303+0000 7f47a3400640 1 -- 172.21.15.119:0/1050117711 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f47940585a0 con 0x7f47ac1e3ce0 2024-04-25T10:11:38.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.307+0000 7f47b2600640 1 -- 172.21.15.119:0/1050117711 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f478806b4d0 msgr2=0x7f478806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.307+0000 7f47b2600640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f478806b4d0 0x7f478806d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f479c032000 tx=0x7f479c034040 comp rx=0 tx=0).stop 2024-04-25T10:11:38.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.307+0000 7f47b2600640 1 -- 172.21.15.119:0/1050117711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47ac1e3ce0 msgr2=0x7f47ac0cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.307+0000 7f47b2600640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47ac1e3ce0 0x7f47ac0cbe70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f479400e4b0 tx=0x7f479400e970 comp rx=0 tx=0).stop 2024-04-25T10:11:38.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.307+0000 7f47b2600640 1 -- 172.21.15.119:0/1050117711 shutdown_connections 2024-04-25T10:11:38.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.307+0000 7f47b2600640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f478806b4d0 0x7f478806d990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.307+0000 7f47b2600640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47ac1f7c30 0x7f47ac1f32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.307+0000 7f47b2600640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47ac1e3ce0 0x7f47ac0cbe70 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.307+0000 7f47b2600640 1 --2- 172.21.15.119:0/1050117711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f47900039b0 0x7f47ac147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.307+0000 7f47b2600640 1 -- 172.21.15.119:0/1050117711 >> 172.21.15.119:0/1050117711 conn(0x7f47ac0d0560 msgr2=0x7f47ac0cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:38.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.308+0000 7f47b2600640 1 -- 172.21.15.119:0/1050117711 shutdown_connections 2024-04-25T10:11:38.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.308+0000 7f47b2600640 1 -- 172.21.15.119:0/1050117711 wait complete. 2024-04-25T10:11:38.335 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:11:38.335 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:38.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.526+0000 7f3e8b200640 1 Processor -- start 2024-04-25T10:11:38.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.526+0000 7f3e8b200640 1 -- start start 2024-04-25T10:11:38.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.528+0000 7f3e8b200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3e8415ed70 0x7f3e84168a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.528+0000 7f3e8b200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e840712c0 con 0x7f3e840aecf0 2024-04-25T10:11:38.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.528+0000 7f3e8b200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e840715a0 con 0x7f3e8415d980 2024-04-25T10:11:38.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.528+0000 7f3e8b200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e84071ed0 con 0x7f3e8415ed70 2024-04-25T10:11:38.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.528+0000 7f3e83400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3e840aecf0 0x7f3e840cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52784/0 (socket says 172.21.15.119:52784) 2024-04-25T10:11:38.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.528+0000 7f3e83400640 1 -- 172.21.15.119:0/1227275414 learned_addr learned my addr 172.21.15.119:0/1227275414 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:38.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.528+0000 7f3e82a00640 1 -- 172.21.15.119:0/1227275414 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777305218 0 0) 0x7f3e840712c0 con 0x7f3e840aecf0 2024-04-25T10:11:38.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.528+0000 7f3e89200640 1 --2- 172.21.15.119:0/1227275414 >> v2:172.21.15.132:3300/0 conn(0x7f3e8415ed70 0x7f3e84168a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e82a00640 1 -- 172.21.15.119:0/1227275414 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e68003520 con 0x7f3e840aecf0 2024-04-25T10:11:38.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e82a00640 1 -- 172.21.15.119:0/1227275414 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 478499254 0 0) 0x7f3e68003520 con 0x7f3e840aecf0 2024-04-25T10:11:38.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e82a00640 1 -- 172.21.15.119:0/1227275414 >> v2:172.21.15.132:3300/0 conn(0x7f3e8415ed70 msgr2=0x7f3e84168a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:38.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e82a00640 1 --2- 172.21.15.119:0/1227275414 >> v2:172.21.15.132:3300/0 conn(0x7f3e8415ed70 0x7f3e84168a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e82a00640 1 -- 172.21.15.119:0/1227275414 >> v1:172.21.15.132:6789/0 conn(0x7f3e8415d980 legacy=0x7f3e840ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e82a00640 1 -- 172.21.15.119:0/1227275414 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e84071880 con 0x7f3e840aecf0 2024-04-25T10:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e82a00640 1 -- 172.21.15.119:0/1227275414 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f3e70001670 con 0x7f3e840aecf0 2024-04-25T10:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e82a00640 1 -- 172.21.15.119:0/1227275414 >> v1:172.21.15.119:6789/0 conn(0x7f3e840aecf0 legacy=0x7f3e840cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e82a00640 1 --2- 172.21.15.119:0/1227275414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e680039b0 0x7f3e68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e82a00640 1 --2- 172.21.15.119:0/1227275414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e68004800 0x7f3e68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e82a00640 1 --2- 172.21.15.119:0/1227275414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e680051c0 0x7f3e6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e82a00640 1 -- 172.21.15.119:0/1227275414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e84071ed0 con 0x7f3e68004800 2024-04-25T10:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e82a00640 1 -- 172.21.15.119:0/1227275414 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6800c160 con 0x7f3e680039b0 2024-04-25T10:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e82a00640 1 -- 172.21.15.119:0/1227275414 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6800c360 con 0x7f3e680051c0 2024-04-25T10:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.529+0000 7f3e83400640 1 --2- 172.21.15.119:0/1227275414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e68004800 0x7f3e68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.530+0000 7f3e83e00640 1 --2- 172.21.15.119:0/1227275414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e680039b0 0x7f3e68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.530+0000 7f3e83400640 1 -- 172.21.15.119:0/1227275414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e680051c0 msgr2=0x7f3e6800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.530+0000 7f3e83400640 1 --2- 172.21.15.119:0/1227275414 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e680051c0 0x7f3e6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.530+0000 7f3e83400640 1 -- 172.21.15.119:0/1227275414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e680039b0 msgr2=0x7f3e68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.530+0000 7f3e83400640 1 --2- 172.21.15.119:0/1227275414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e680039b0 0x7f3e68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.530+0000 7f3e83400640 1 -- 172.21.15.119:0/1227275414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e68003520 con 0x7f3e68004800 2024-04-25T10:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.530+0000 7f3e83e00640 1 --2- 172.21.15.119:0/1227275414 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e680039b0 0x7f3e68003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.530+0000 7f3e89200640 1 -- 172.21.15.119:0/1227275414 reap_dead start 2024-04-25T10:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.531+0000 7f3e83400640 1 --2- 172.21.15.119:0/1227275414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e68004800 0x7f3e68004c80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3e84070f70 tx=0x7f3e68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec6245ea9e53ccf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.531+0000 7f3e82a00640 1 -- 172.21.15.119:0/1227275414 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70007cf0 con 0x7f3e68004800 2024-04-25T10:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.531+0000 7f3e82a00640 1 -- 172.21.15.119:0/1227275414 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e70005af0 con 0x7f3e68004800 2024-04-25T10:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.531+0000 7f3e82a00640 1 -- 172.21.15.119:0/1227275414 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70007070 con 0x7f3e68004800 2024-04-25T10:11:38.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.532+0000 7f3e8b200640 1 -- 172.21.15.119:0/1227275414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e68004800 msgr2=0x7f3e68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.532+0000 7f3e8b200640 1 --2- 172.21.15.119:0/1227275414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e68004800 0x7f3e68004c80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3e84070f70 tx=0x7f3e68005830 comp rx=0 tx=0).stop 2024-04-25T10:11:38.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.532+0000 7f3e8b200640 1 -- 172.21.15.119:0/1227275414 shutdown_connections 2024-04-25T10:11:38.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.532+0000 7f3e8b200640 1 --2- 172.21.15.119:0/1227275414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e68004800 0x7f3e68004c80 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.532+0000 7f3e8b200640 1 -- 172.21.15.119:0/1227275414 >> 172.21.15.119:0/1227275414 conn(0x7f3e8406aba0 msgr2=0x7f3e8406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:38.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.532+0000 7f3e8b200640 1 -- 172.21.15.119:0/1227275414 shutdown_connections 2024-04-25T10:11:38.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.532+0000 7f3e8b200640 1 -- 172.21.15.119:0/1227275414 wait complete. 2024-04-25T10:11:38.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.534+0000 7f3e8b200640 1 Processor -- start 2024-04-25T10:11:38.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.534+0000 7f3e8b200640 1 -- start start 2024-04-25T10:11:38.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e8b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e68004800 0x7f3e840cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e8b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e841553b0 0x7f3e8414fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e8b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e84155f30 0x7f3e84156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e8b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e840ae760 con 0x7f3e841553b0 2024-04-25T10:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e8b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e840ae5e0 con 0x7f3e68004800 2024-04-25T10:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e8b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e840ae8e0 con 0x7f3e84155f30 2024-04-25T10:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e83400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e841553b0 0x7f3e8414fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e83400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e841553b0 0x7f3e8414fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43044/0 (socket says 172.21.15.119:43044) 2024-04-25T10:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e83400640 1 -- 172.21.15.119:0/131983429 learned_addr learned my addr 172.21.15.119:0/131983429 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:38.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e89200640 1 --2- 172.21.15.119:0/131983429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e84155f30 0x7f3e84156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e83e00640 1 --2- 172.21.15.119:0/131983429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e68004800 0x7f3e840cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e83400640 1 -- 172.21.15.119:0/131983429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e84155f30 msgr2=0x7f3e84156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e83400640 1 --2- 172.21.15.119:0/131983429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e84155f30 0x7f3e84156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e83400640 1 -- 172.21.15.119:0/131983429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e68004800 msgr2=0x7f3e840cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e83400640 1 --2- 172.21.15.119:0/131983429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e68004800 0x7f3e840cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.535+0000 7f3e83400640 1 -- 172.21.15.119:0/131983429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e840cc2f0 con 0x7f3e841553b0 2024-04-25T10:11:38.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.536+0000 7f3e83400640 1 --2- 172.21.15.119:0/131983429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e841553b0 0x7f3e8414fd30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3e7002ad60 tx=0x7f3e84071a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:38.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.536+0000 7f3e7be00640 1 -- 172.21.15.119:0/131983429 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70040030 con 0x7f3e841553b0 2024-04-25T10:11:38.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.536+0000 7f3e7be00640 1 -- 172.21.15.119:0/131983429 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e70001470 con 0x7f3e841553b0 2024-04-25T10:11:38.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.536+0000 7f3e7be00640 1 -- 172.21.15.119:0/131983429 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70037040 con 0x7f3e841553b0 2024-04-25T10:11:38.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.536+0000 7f3e8b200640 1 -- 172.21.15.119:0/131983429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e840cc5d0 con 0x7f3e841553b0 2024-04-25T10:11:38.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.536+0000 7f3e8b200640 1 -- 172.21.15.119:0/131983429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e840ccb90 con 0x7f3e841553b0 2024-04-25T10:11:38.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.537+0000 7f3e8b200640 1 -- 172.21.15.119:0/131983429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e840749b0 con 0x7f3e841553b0 2024-04-25T10:11:38.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.538+0000 7f3e7be00640 1 -- 172.21.15.119:0/131983429 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e70043070 con 0x7f3e841553b0 2024-04-25T10:11:38.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.539+0000 7f3e7be00640 1 --2- 172.21.15.119:0/131983429 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e6406b500 0x7f3e6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.539+0000 7f3e7be00640 1 -- 172.21.15.119:0/131983429 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e700a9a60 con 0x7f3e841553b0 2024-04-25T10:11:38.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.539+0000 7f3e83e00640 1 --2- 172.21.15.119:0/131983429 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e6406b500 0x7f3e6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.540+0000 7f3e83e00640 1 --2- 172.21.15.119:0/131983429 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e6406b500 0x7f3e6406d9c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3e84154020 tx=0x7f3e74009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:38.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.544+0000 7f3e7be00640 1 -- 172.21.15.119:0/131983429 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e7004a070 con 0x7f3e841553b0 2024-04-25T10:11:38.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.780+0000 7f3e8b200640 1 -- 172.21.15.119:0/131983429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e8406d4c0 con 0x7f3e841553b0 2024-04-25T10:11:38.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.781+0000 7f3e7be00640 1 -- 172.21.15.119:0/131983429 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f3e7007e030 con 0x7f3e841553b0 2024-04-25T10:11:38.787 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:38.787 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2798722683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2798722683},{"type":"v1","addr":"172.21.15.132:6833","nonce":2798722683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2834911408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2834911408},{"type":"v1","addr":"172.21.15.132:6835","nonce":2834911408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/643040335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":643040335},{"type":"v1","addr":"172.21.15.119:6835","nonce":643040335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:34.712290+0000","modified":"2024-04-25T10:11:37.787344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6483},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6483":{"gid":6483,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/2673349178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2673349178},{"type":"v1","addr":"172.21.15.119:6837","nonce":2673349178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:38.787 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 44 2024-04-25T10:11:38.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.784+0000 7f3e8b200640 1 -- 172.21.15.119:0/131983429 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e6406b500 msgr2=0x7f3e6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.784+0000 7f3e8b200640 1 --2- 172.21.15.119:0/131983429 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e6406b500 0x7f3e6406d9c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3e84154020 tx=0x7f3e74009040 comp rx=0 tx=0).stop 2024-04-25T10:11:38.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.784+0000 7f3e8b200640 1 -- 172.21.15.119:0/131983429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e841553b0 msgr2=0x7f3e8414fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.784+0000 7f3e8b200640 1 --2- 172.21.15.119:0/131983429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e841553b0 0x7f3e8414fd30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3e7002ad60 tx=0x7f3e84071a50 comp rx=0 tx=0).stop 2024-04-25T10:11:38.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.784+0000 7f3e8b200640 1 -- 172.21.15.119:0/131983429 shutdown_connections 2024-04-25T10:11:38.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.784+0000 7f3e8b200640 1 --2- 172.21.15.119:0/131983429 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e6406b500 0x7f3e6406d9c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.784+0000 7f3e8b200640 1 --2- 172.21.15.119:0/131983429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e84155f30 0x7f3e84156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.784+0000 7f3e8b200640 1 --2- 172.21.15.119:0/131983429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e841553b0 0x7f3e8414fd30 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.784+0000 7f3e8b200640 1 --2- 172.21.15.119:0/131983429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e68004800 0x7f3e840cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.784+0000 7f3e8b200640 1 -- 172.21.15.119:0/131983429 >> 172.21.15.119:0/131983429 conn(0x7f3e8406aba0 msgr2=0x7f3e840ad160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:38.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.785+0000 7f3e8b200640 1 -- 172.21.15.119:0/131983429 shutdown_connections 2024-04-25T10:11:38.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:38.785+0000 7f3e8b200640 1 -- 172.21.15.119:0/131983429 wait complete. 2024-04-25T10:11:38.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:39.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.006+0000 7f63b6e00640 1 Processor -- start 2024-04-25T10:11:39.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.006+0000 7f63b6e00640 1 -- start start 2024-04-25T10:11:39.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.007+0000 7f63b6e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f63b015bf00 0x7f63b00cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.008+0000 7f63b6e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b00ce6c0 con 0x7f63b006e8f0 2024-04-25T10:11:39.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.008+0000 7f63b6e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b00cc7b0 con 0x7f63b00cc310 2024-04-25T10:11:39.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.008+0000 7f63b6e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b01455e0 con 0x7f63b015bf00 2024-04-25T10:11:39.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.008+0000 7f63b4e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f63b006e8f0 0x7f63b01654b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52800/0 (socket says 172.21.15.119:52800) 2024-04-25T10:11:39.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.008+0000 7f63b4e00640 1 -- 172.21.15.119:0/377670654 learned_addr learned my addr 172.21.15.119:0/377670654 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:39.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.008+0000 7f63afe00640 1 --2- 172.21.15.119:0/377670654 >> v2:172.21.15.119:3300/0 conn(0x7f63b015bf00 0x7f63b00cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 -- 172.21.15.119:0/377670654 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616007785 0 0) 0x7f63a8001490 con 0x7f63b006e8f0 2024-04-25T10:11:39.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 -- 172.21.15.119:0/377670654 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6394003520 con 0x7f63b006e8f0 2024-04-25T10:11:39.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 -- 172.21.15.119:0/377670654 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2536794213 0 0) 0x7f6394003520 con 0x7f63b006e8f0 2024-04-25T10:11:39.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 -- 172.21.15.119:0/377670654 >> v2:172.21.15.119:3300/0 conn(0x7f63b015bf00 msgr2=0x7f63b00cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 --2- 172.21.15.119:0/377670654 >> v2:172.21.15.119:3300/0 conn(0x7f63b015bf00 0x7f63b00cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 -- 172.21.15.119:0/377670654 >> v1:172.21.15.132:6789/0 conn(0x7f63b00cc310 legacy=0x7f63b015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 -- 172.21.15.119:0/377670654 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b00cca90 con 0x7f63b006e8f0 2024-04-25T10:11:39.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63afe00640 1 --2- 172.21.15.119:0/377670654 >> v2:172.21.15.119:3300/0 conn(0x7f63b015bf00 0x7f63b00cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:39.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 -- 172.21.15.119:0/377670654 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f63a8001bc0 con 0x7f63b006e8f0 2024-04-25T10:11:39.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 -- 172.21.15.119:0/377670654 >> v1:172.21.15.119:6789/0 conn(0x7f63b006e8f0 legacy=0x7f63b01654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 --2- 172.21.15.119:0/377670654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6394003a00 0x7f6394003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 --2- 172.21.15.119:0/377670654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6394004800 0x7f6394004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 --2- 172.21.15.119:0/377670654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63940051c0 0x7f639400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 -- 172.21.15.119:0/377670654 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b01455e0 con 0x7f6394004800 2024-04-25T10:11:39.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 -- 172.21.15.119:0/377670654 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639400c0c0 con 0x7f63940051c0 2024-04-25T10:11:39.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63aea00640 1 -- 172.21.15.119:0/377670654 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639400c2c0 con 0x7f6394003a00 2024-04-25T10:11:39.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.009+0000 7f63af400640 1 --2- 172.21.15.119:0/377670654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6394004800 0x7f6394004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.010+0000 7f63af400640 1 -- 172.21.15.119:0/377670654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6394003a00 msgr2=0x7f6394003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.010+0000 7f63af400640 1 --2- 172.21.15.119:0/377670654 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6394003a00 0x7f6394003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.010+0000 7f63af400640 1 -- 172.21.15.119:0/377670654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63940051c0 msgr2=0x7f639400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.010+0000 7f63af400640 1 --2- 172.21.15.119:0/377670654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63940051c0 0x7f639400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.010+0000 7f63af400640 1 -- 172.21.15.119:0/377670654 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6394003520 con 0x7f6394004800 2024-04-25T10:11:39.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.010+0000 7f63b4e00640 1 -- 172.21.15.119:0/377670654 reap_dead start 2024-04-25T10:11:39.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.011+0000 7f63af400640 1 --2- 172.21.15.119:0/377670654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6394004800 0x7f6394004c80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f639c001720 tx=0x7f6394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a526dbeb1fc78986 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.011+0000 7f63aea00640 1 -- 172.21.15.119:0/377670654 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c001370 con 0x7f6394004800 2024-04-25T10:11:39.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.011+0000 7f63aea00640 1 -- 172.21.15.119:0/377670654 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c0052d0 con 0x7f6394004800 2024-04-25T10:11:39.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.011+0000 7f63aea00640 1 -- 172.21.15.119:0/377670654 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c02ab90 con 0x7f6394004800 2024-04-25T10:11:39.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.012+0000 7f63b6e00640 1 -- 172.21.15.119:0/377670654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6394004800 msgr2=0x7f6394004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.012+0000 7f63b6e00640 1 --2- 172.21.15.119:0/377670654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6394004800 0x7f6394004c80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f639c001720 tx=0x7f6394005830 comp rx=0 tx=0).stop 2024-04-25T10:11:39.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.012+0000 7f63b6e00640 1 -- 172.21.15.119:0/377670654 shutdown_connections 2024-04-25T10:11:39.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.012+0000 7f63b6e00640 1 --2- 172.21.15.119:0/377670654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6394004800 0x7f6394004c80 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.012+0000 7f63b6e00640 1 -- 172.21.15.119:0/377670654 >> 172.21.15.119:0/377670654 conn(0x7f63b006aba0 msgr2=0x7f63b006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.012+0000 7f63b6e00640 1 -- 172.21.15.119:0/377670654 shutdown_connections 2024-04-25T10:11:39.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.012+0000 7f63b6e00640 1 -- 172.21.15.119:0/377670654 wait complete. 2024-04-25T10:11:39.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.014+0000 7f63b6e00640 1 Processor -- start 2024-04-25T10:11:39.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.014+0000 7f63b6e00640 1 -- start start 2024-04-25T10:11:39.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.015+0000 7f63b6e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6394004800 0x7f63b00c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.015+0000 7f63b6e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63b01332a0 0x7f63b00c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.015+0000 7f63b6e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63b015ab20 0x7f63b015e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.015+0000 7f63b6e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00cd050 con 0x7f6394004800 2024-04-25T10:11:39.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.015+0000 7f63b6e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b006d450 con 0x7f63b01332a0 2024-04-25T10:11:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.015+0000 7f63b6e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b006ee60 con 0x7f63b015ab20 2024-04-25T10:11:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.015+0000 7f63afe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6394004800 0x7f63b00c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.015+0000 7f63afe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6394004800 0x7f63b00c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43068/0 (socket says 172.21.15.119:43068) 2024-04-25T10:11:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.015+0000 7f63afe00640 1 -- 172.21.15.119:0/3273808540 learned_addr learned my addr 172.21.15.119:0/3273808540 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.015+0000 7f63afe00640 1 -- 172.21.15.119:0/3273808540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63b015ab20 msgr2=0x7f63b015e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.015+0000 7f63afe00640 1 --2- 172.21.15.119:0/3273808540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63b015ab20 0x7f63b015e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.015+0000 7f63afe00640 1 -- 172.21.15.119:0/3273808540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63b01332a0 msgr2=0x7f63b00c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.015+0000 7f63afe00640 1 --2- 172.21.15.119:0/3273808540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63b01332a0 0x7f63b00c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.015+0000 7f63afe00640 1 -- 172.21.15.119:0/3273808540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b00c9190 con 0x7f6394004800 2024-04-25T10:11:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.016+0000 7f63afe00640 1 --2- 172.21.15.119:0/3273808540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6394004800 0x7f63b00c8c50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f63a000e910 tx=0x7f63a000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.016+0000 7f63a7e00640 1 -- 172.21.15.119:0/3273808540 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0004030 con 0x7f6394004800 2024-04-25T10:11:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.016+0000 7f63a7e00640 1 -- 172.21.15.119:0/3273808540 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a00041f0 con 0x7f6394004800 2024-04-25T10:11:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.016+0000 7f63a7e00640 1 -- 172.21.15.119:0/3273808540 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a000c8e0 con 0x7f6394004800 2024-04-25T10:11:39.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.016+0000 7f63b6e00640 1 -- 172.21.15.119:0/3273808540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b00c4a20 con 0x7f6394004800 2024-04-25T10:11:39.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.016+0000 7f63b6e00640 1 -- 172.21.15.119:0/3273808540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b00c4fd0 con 0x7f6394004800 2024-04-25T10:11:39.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.017+0000 7f63b6e00640 1 -- 172.21.15.119:0/3273808540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b00749b0 con 0x7f6394004800 2024-04-25T10:11:39.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.018+0000 7f63a7e00640 1 -- 172.21.15.119:0/3273808540 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63a000ca80 con 0x7f6394004800 2024-04-25T10:11:39.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.019+0000 7f63a7e00640 1 --2- 172.21.15.119:0/3273808540 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f639006b500 0x7f639006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.019+0000 7f63a7e00640 1 -- 172.21.15.119:0/3273808540 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63a008a220 con 0x7f6394004800 2024-04-25T10:11:39.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.019+0000 7f63af400640 1 --2- 172.21.15.119:0/3273808540 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f639006b500 0x7f639006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.020+0000 7f63af400640 1 --2- 172.21.15.119:0/3273808540 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f639006b500 0x7f639006d9c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f639c032000 tx=0x7f639c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.023+0000 7f63a7e00640 1 -- 172.21.15.119:0/3273808540 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63a0017020 con 0x7f6394004800 2024-04-25T10:11:39.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.223+0000 7f63b6e00640 1 -- 172.21.15.119:0/3273808540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63b015d9e0 con 0x7f6394004800 2024-04-25T10:11:39.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.225+0000 7f63a7e00640 1 -- 172.21.15.119:0/3273808540 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16704 (secure 0 0 0) 0x7f63a0059000 con 0x7f6394004800 2024-04-25T10:11:39.231 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:39.231 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":67,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:11:36.274669+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:32.500199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:33.708192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:39.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.228+0000 7f63b6e00640 1 -- 172.21.15.119:0/3273808540 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f639006b500 msgr2=0x7f639006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.228+0000 7f63b6e00640 1 --2- 172.21.15.119:0/3273808540 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f639006b500 0x7f639006d9c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f639c032000 tx=0x7f639c02c000 comp rx=0 tx=0).stop 2024-04-25T10:11:39.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.228+0000 7f63b6e00640 1 -- 172.21.15.119:0/3273808540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6394004800 msgr2=0x7f63b00c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.228+0000 7f63b6e00640 1 --2- 172.21.15.119:0/3273808540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6394004800 0x7f63b00c8c50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f63a000e910 tx=0x7f63a000edd0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.228+0000 7f63b6e00640 1 -- 172.21.15.119:0/3273808540 shutdown_connections 2024-04-25T10:11:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.228+0000 7f63b6e00640 1 --2- 172.21.15.119:0/3273808540 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f639006b500 0x7f639006d9c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.228+0000 7f63b6e00640 1 --2- 172.21.15.119:0/3273808540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63b015ab20 0x7f63b015e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.228+0000 7f63b6e00640 1 --2- 172.21.15.119:0/3273808540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63b01332a0 0x7f63b00c44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.228+0000 7f63b6e00640 1 --2- 172.21.15.119:0/3273808540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6394004800 0x7f63b00c8c50 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.228+0000 7f63b6e00640 1 -- 172.21.15.119:0/3273808540 >> 172.21.15.119:0/3273808540 conn(0x7f63b006aba0 msgr2=0x7f63b015c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.228+0000 7f63b6e00640 1 -- 172.21.15.119:0/3273808540 shutdown_connections 2024-04-25T10:11:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.228+0000 7f63b6e00640 1 -- 172.21.15.119:0/3273808540 wait complete. 2024-04-25T10:11:39.253 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:11:39.253 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:39.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.447+0000 7fca41600640 1 Processor -- start 2024-04-25T10:11:39.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.447+0000 7fca41600640 1 -- start start 2024-04-25T10:11:39.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.449+0000 7fca41600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca3c0b4410 0x7fca3c15e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.449+0000 7fca41600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca3c15eb30 0x7fca3c0acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.449+0000 7fca41600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c0af000 con 0x7fca3c147270 2024-04-25T10:11:39.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.449+0000 7fca41600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3c0b4a80 con 0x7fca3c0b4410 2024-04-25T10:11:39.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.449+0000 7fca41600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0b4d60 con 0x7fca3c15eb30 2024-04-25T10:11:39.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.449+0000 7fca3be00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fca3c147270 0x7fca3c15dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52810/0 (socket says 172.21.15.119:52810) 2024-04-25T10:11:39.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.449+0000 7fca3be00640 1 -- 172.21.15.119:0/3675635891 learned_addr learned my addr 172.21.15.119:0/3675635891 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:39.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3b400640 1 --2- 172.21.15.119:0/3675635891 >> v2:172.21.15.132:3300/0 conn(0x7fca3c0b4410 0x7fca3c15e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 -- 172.21.15.119:0/3675635891 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299396689 0 0) 0x7fca3c0af000 con 0x7fca3c147270 2024-04-25T10:11:39.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 -- 172.21.15.119:0/3675635891 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca20003520 con 0x7fca3c147270 2024-04-25T10:11:39.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca40c00640 1 --2- 172.21.15.119:0/3675635891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca3c15eb30 0x7fca3c0acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 -- 172.21.15.119:0/3675635891 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1181552574 0 0) 0x7fca20003520 con 0x7fca3c147270 2024-04-25T10:11:39.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 -- 172.21.15.119:0/3675635891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca3c15eb30 msgr2=0x7fca3c0acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 --2- 172.21.15.119:0/3675635891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca3c15eb30 0x7fca3c0acf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 -- 172.21.15.119:0/3675635891 >> v2:172.21.15.132:3300/0 conn(0x7fca3c0b4410 msgr2=0x7fca3c15e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 --2- 172.21.15.119:0/3675635891 >> v2:172.21.15.132:3300/0 conn(0x7fca3c0b4410 0x7fca3c15e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 -- 172.21.15.119:0/3675635891 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0b5040 con 0x7fca3c147270 2024-04-25T10:11:39.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 -- 172.21.15.119:0/3675635891 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fca2c001690 con 0x7fca3c147270 2024-04-25T10:11:39.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 -- 172.21.15.119:0/3675635891 >> v1:172.21.15.119:6789/0 conn(0x7fca3c147270 legacy=0x7fca3c15dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 --2- 172.21.15.119:0/3675635891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca20003a20 0x7fca20003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 --2- 172.21.15.119:0/3675635891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca20004800 0x7fca20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 --2- 172.21.15.119:0/3675635891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca3c15eb30 0x7fca200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 -- 172.21.15.119:0/3675635891 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0b4a80 con 0x7fca20004800 2024-04-25T10:11:39.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 -- 172.21.15.119:0/3675635891 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0b4d60 con 0x7fca20003a20 2024-04-25T10:11:39.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.450+0000 7fca3aa00640 1 -- 172.21.15.119:0/3675635891 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2000bac0 con 0x7fca3c15eb30 2024-04-25T10:11:39.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.451+0000 7fca3b400640 1 --2- 172.21.15.119:0/3675635891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca20004800 0x7fca20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.451+0000 7fca3be00640 1 --2- 172.21.15.119:0/3675635891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca20003a20 0x7fca20003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.451+0000 7fca3b400640 1 -- 172.21.15.119:0/3675635891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca3c15eb30 msgr2=0x7fca200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.451+0000 7fca40c00640 1 --2- 172.21.15.119:0/3675635891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca3c15eb30 0x7fca200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.451+0000 7fca3b400640 1 --2- 172.21.15.119:0/3675635891 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca3c15eb30 0x7fca200051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.451+0000 7fca3b400640 1 -- 172.21.15.119:0/3675635891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca20003a20 msgr2=0x7fca20003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.451+0000 7fca3b400640 1 --2- 172.21.15.119:0/3675635891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca20003a20 0x7fca20003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.451+0000 7fca3b400640 1 -- 172.21.15.119:0/3675635891 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca20003520 con 0x7fca20004800 2024-04-25T10:11:39.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.451+0000 7fca3be00640 1 --2- 172.21.15.119:0/3675635891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca20003a20 0x7fca20003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:39.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.452+0000 7fca3b400640 1 --2- 172.21.15.119:0/3675635891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca20004800 0x7fca20004c80 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fca28009ac0 tx=0x7fca20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dca7afe39782947 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.452+0000 7fca3aa00640 1 -- 172.21.15.119:0/3675635891 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28036070 con 0x7fca20004800 2024-04-25T10:11:39.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.452+0000 7fca3aa00640 1 -- 172.21.15.119:0/3675635891 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca28031400 con 0x7fca20004800 2024-04-25T10:11:39.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.452+0000 7fca3aa00640 1 -- 172.21.15.119:0/3675635891 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28009690 con 0x7fca20004800 2024-04-25T10:11:39.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.453+0000 7fca41600640 1 -- 172.21.15.119:0/3675635891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca20004800 msgr2=0x7fca20004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.453+0000 7fca41600640 1 --2- 172.21.15.119:0/3675635891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca20004800 0x7fca20004c80 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fca28009ac0 tx=0x7fca20005830 comp rx=0 tx=0).stop 2024-04-25T10:11:39.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.453+0000 7fca40c00640 1 -- 172.21.15.119:0/3675635891 reap_dead start 2024-04-25T10:11:39.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.453+0000 7fca41600640 1 -- 172.21.15.119:0/3675635891 shutdown_connections 2024-04-25T10:11:39.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.453+0000 7fca41600640 1 -- 172.21.15.119:0/3675635891 >> 172.21.15.119:0/3675635891 conn(0x7fca3c06b1d0 msgr2=0x7fca3c0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.453+0000 7fca41600640 1 -- 172.21.15.119:0/3675635891 shutdown_connections 2024-04-25T10:11:39.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.453+0000 7fca41600640 1 -- 172.21.15.119:0/3675635891 wait complete. 2024-04-25T10:11:39.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.455+0000 7fca41600640 1 Processor -- start 2024-04-25T10:11:39.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.455+0000 7fca41600640 1 -- start start 2024-04-25T10:11:39.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.456+0000 7fca41600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca20004800 0x7fca3c13c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.456+0000 7fca41600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca3c13f6c0 0x7fca3c141400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.456+0000 7fca41600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca3c13fb40 0x7fca3c141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.456+0000 7fca41600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0acf80 con 0x7fca20004800 2024-04-25T10:11:39.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.456+0000 7fca41600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0b5600 con 0x7fca3c13fb40 2024-04-25T10:11:39.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.456+0000 7fca41600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0ad100 con 0x7fca3c13f6c0 2024-04-25T10:11:39.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.457+0000 7fca3be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca20004800 0x7fca3c13c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.457+0000 7fca3be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca20004800 0x7fca3c13c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43090/0 (socket says 172.21.15.119:43090) 2024-04-25T10:11:39.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.457+0000 7fca3be00640 1 -- 172.21.15.119:0/1118082079 learned_addr learned my addr 172.21.15.119:0/1118082079 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:39.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.457+0000 7fca3be00640 1 -- 172.21.15.119:0/1118082079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca3c13f6c0 msgr2=0x7fca3c141400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.457+0000 7fca3be00640 1 --2- 172.21.15.119:0/1118082079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca3c13f6c0 0x7fca3c141400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.457+0000 7fca3be00640 1 -- 172.21.15.119:0/1118082079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca3c13fb40 msgr2=0x7fca3c141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.457+0000 7fca3be00640 1 --2- 172.21.15.119:0/1118082079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca3c13fb40 0x7fca3c141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.457+0000 7fca3be00640 1 -- 172.21.15.119:0/1118082079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c13c850 con 0x7fca20004800 2024-04-25T10:11:39.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.457+0000 7fca3be00640 1 --2- 172.21.15.119:0/1118082079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca20004800 0x7fca3c13c310 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00d810 tx=0x7fca2c00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.458+0000 7fca33e00640 1 -- 172.21.15.119:0/1118082079 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c019070 con 0x7fca20004800 2024-04-25T10:11:39.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.458+0000 7fca33e00640 1 -- 172.21.15.119:0/1118082079 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c0074e0 con 0x7fca20004800 2024-04-25T10:11:39.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.458+0000 7fca33e00640 1 -- 172.21.15.119:0/1118082079 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c004a00 con 0x7fca20004800 2024-04-25T10:11:39.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.458+0000 7fca41600640 1 -- 172.21.15.119:0/1118082079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c13e620 con 0x7fca20004800 2024-04-25T10:11:39.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.458+0000 7fca41600640 1 -- 172.21.15.119:0/1118082079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c13ebe0 con 0x7fca20004800 2024-04-25T10:11:39.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.459+0000 7fca41600640 1 -- 172.21.15.119:0/1118082079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3c0749b0 con 0x7fca20004800 2024-04-25T10:11:39.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.460+0000 7fca33e00640 1 -- 172.21.15.119:0/1118082079 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca2c01c070 con 0x7fca20004800 2024-04-25T10:11:39.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.461+0000 7fca33e00640 1 --2- 172.21.15.119:0/1118082079 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca1c06b4d0 0x7fca1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.461+0000 7fca33e00640 1 -- 172.21.15.119:0/1118082079 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca2c089dd0 con 0x7fca20004800 2024-04-25T10:11:39.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.462+0000 7fca3b400640 1 --2- 172.21.15.119:0/1118082079 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca1c06b4d0 0x7fca1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.462+0000 7fca3b400640 1 --2- 172.21.15.119:0/1118082079 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca1c06b4d0 0x7fca1c06d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fca28033000 tx=0x7fca2802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.466+0000 7fca33e00640 1 -- 172.21.15.119:0/1118082079 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca2c058c70 con 0x7fca20004800 2024-04-25T10:11:39.644 INFO:tasks.ceph.mds.c.smithi119.stderr:2024-04-25T10:11:39.639+0000 7fec44c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:39.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.658+0000 7fca41600640 1 -- 172.21.15.119:0/1118082079 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca3c0b2180 con 0x7fca1c06b4d0 2024-04-25T10:11:39.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.666+0000 7fca33e00640 1 -- 172.21.15.119:0/1118082079 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252000 (secure 0 0 0) 0x7fca3c0b2180 con 0x7fca1c06b4d0 2024-04-25T10:11:39.675 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:39.677 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:11:39.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.671+0000 7fca41600640 1 -- 172.21.15.119:0/1118082079 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca1c06b4d0 msgr2=0x7fca1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.671+0000 7fca41600640 1 --2- 172.21.15.119:0/1118082079 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca1c06b4d0 0x7fca1c06d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fca28033000 tx=0x7fca2802e000 comp rx=0 tx=0).stop 2024-04-25T10:11:39.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.671+0000 7fca41600640 1 -- 172.21.15.119:0/1118082079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca20004800 msgr2=0x7fca3c13c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.671+0000 7fca41600640 1 --2- 172.21.15.119:0/1118082079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca20004800 0x7fca3c13c310 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00d810 tx=0x7fca2c00dcd0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.671+0000 7fca41600640 1 -- 172.21.15.119:0/1118082079 shutdown_connections 2024-04-25T10:11:39.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.671+0000 7fca41600640 1 --2- 172.21.15.119:0/1118082079 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca1c06b4d0 0x7fca1c06d990 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.671+0000 7fca41600640 1 --2- 172.21.15.119:0/1118082079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca3c13fb40 0x7fca3c141940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.671+0000 7fca41600640 1 --2- 172.21.15.119:0/1118082079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca3c13f6c0 0x7fca3c141400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.671+0000 7fca41600640 1 --2- 172.21.15.119:0/1118082079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca20004800 0x7fca3c13c310 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.671+0000 7fca41600640 1 -- 172.21.15.119:0/1118082079 >> 172.21.15.119:0/1118082079 conn(0x7fca3c06b1d0 msgr2=0x7fca3c070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.671+0000 7fca41600640 1 -- 172.21.15.119:0/1118082079 shutdown_connections 2024-04-25T10:11:39.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.671+0000 7fca41600640 1 -- 172.21.15.119:0/1118082079 wait complete. 2024-04-25T10:11:39.694 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-04-25T10:11:38.014651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256932,"kb_used_data":7444,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717788,"statfs":{"total":773094113280,"available":772831014912,"internally_reserved":0,"allocated":7622656,"data_stored":3849778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.777965"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.301513+0000","last_change":"2024-04-25T10:11:35.290899+0000","last_active":"2024-04-25T10:11:35.301513+0000","last_peered":"2024-04-25T10:11:35.301513+0000","last_clean":"2024-04-25T10:11:35.301513+0000","last_became_active":"2024-04-25T10:11:35.289495+0000","last_became_peered":"2024-04-25T10:11:35.289495+0000","last_unstale":"2024-04-25T10:11:35.301513+0000","last_undegraded":"2024-04-25T10:11:35.301513+0000","last_fullsized":"2024-04-25T10:11:35.301513+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:47.545189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.312658+0000","last_change":"2024-04-25T10:11:35.308246+0000","last_active":"2024-04-25T10:11:35.312658+0000","last_peered":"2024-04-25T10:11:35.312658+0000","last_clean":"2024-04-25T10:11:35.312658+0000","last_became_active":"2024-04-25T10:11:35.307668+0000","last_became_peered":"2024-04-25T10:11:35.307668+0000","last_unstale":"2024-04-25T10:11:35.312658+0000","last_undegraded":"2024-04-25T10:11:35.312658+0000","last_fullsized":"2024-04-25T10:11:35.312658+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:37.334486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.303945+0000","last_change":"2024-04-25T10:11:35.293708+0000","last_active":"2024-04-25T10:11:35.303945+0000","last_peered":"2024-04-25T10:11:35.303945+0000","last_clean":"2024-04-25T10:11:35.303945+0000","last_became_active":"2024-04-25T10:11:35.293060+0000","last_became_peered":"2024-04-25T10:11:35.293060+0000","last_unstale":"2024-04-25T10:11:35.303945+0000","last_undegraded":"2024-04-25T10:11:35.303945+0000","last_fullsized":"2024-04-25T10:11:35.303945+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:28.610072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.301647+0000","last_change":"2024-04-25T10:11:35.294833+0000","last_active":"2024-04-25T10:11:35.301647+0000","last_peered":"2024-04-25T10:11:35.301647+0000","last_clean":"2024-04-25T10:11:35.301647+0000","last_became_active":"2024-04-25T10:11:35.294483+0000","last_became_peered":"2024-04-25T10:11:35.294483+0000","last_unstale":"2024-04-25T10:11:35.301647+0000","last_undegraded":"2024-04-25T10:11:35.301647+0000","last_fullsized":"2024-04-25T10:11:35.301647+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:09.804191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.306202+0000","last_change":"2024-04-25T10:11:35.295072+0000","last_active":"2024-04-25T10:11:35.306202+0000","last_peered":"2024-04-25T10:11:35.306202+0000","last_clean":"2024-04-25T10:11:35.306202+0000","last_became_active":"2024-04-25T10:11:35.293393+0000","last_became_peered":"2024-04-25T10:11:35.293393+0000","last_unstale":"2024-04-25T10:11:35.306202+0000","last_undegraded":"2024-04-25T10:11:35.306202+0000","last_fullsized":"2024-04-25T10:11:35.306202+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:18.134328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304137+0000","last_change":"2024-04-25T10:11:35.294365+0000","last_active":"2024-04-25T10:11:35.304137+0000","last_peered":"2024-04-25T10:11:35.304137+0000","last_clean":"2024-04-25T10:11:35.304137+0000","last_became_active":"2024-04-25T10:11:35.294058+0000","last_became_peered":"2024-04-25T10:11:35.294058+0000","last_unstale":"2024-04-25T10:11:35.304137+0000","last_undegraded":"2024-04-25T10:11:35.304137+0000","last_fullsized":"2024-04-25T10:11:35.304137+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:41.623334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.306897+0000","last_change":"2024-04-25T10:11:35.299291+0000","last_active":"2024-04-25T10:11:35.306897+0000","last_peered":"2024-04-25T10:11:35.306897+0000","last_clean":"2024-04-25T10:11:35.306897+0000","last_became_active":"2024-04-25T10:11:35.298165+0000","last_became_peered":"2024-04-25T10:11:35.298165+0000","last_unstale":"2024-04-25T10:11:35.306897+0000","last_undegraded":"2024-04-25T10:11:35.306897+0000","last_fullsized":"2024-04-25T10:11:35.306897+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:47.545189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.312567+0000","last_change":"2024-04-25T10:11:35.307511+0000","last_active":"2024-04-25T10:11:35.312567+0000","last_peered":"2024-04-25T10:11:35.312567+0000","last_clean":"2024-04-25T10:11:35.312567+0000","last_became_active":"2024-04-25T10:11:35.305859+0000","last_became_peered":"2024-04-25T10:11:35.305859+0000","last_unstale":"2024-04-25T10:11:35.312567+0000","last_undegraded":"2024-04-25T10:11:35.312567+0000","last_fullsized":"2024-04-25T10:11:35.312567+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:26.509901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.308408+0000","last_change":"2024-04-25T10:11:35.296855+0000","last_active":"2024-04-25T10:11:35.308408+0000","last_peered":"2024-04-25T10:11:35.308408+0000","last_clean":"2024-04-25T10:11:35.308408+0000","last_became_active":"2024-04-25T10:11:35.296529+0000","last_became_peered":"2024-04-25T10:11:35.296529+0000","last_unstale":"2024-04-25T10:11:35.308408+0000","last_undegraded":"2024-04-25T10:11:35.308408+0000","last_fullsized":"2024-04-25T10:11:35.308408+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:05.295729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.292755+0000","last_change":"2024-04-25T10:11:35.286253+0000","last_active":"2024-04-25T10:11:35.292755+0000","last_peered":"2024-04-25T10:11:35.292755+0000","last_clean":"2024-04-25T10:11:35.292755+0000","last_became_active":"2024-04-25T10:11:35.285800+0000","last_became_peered":"2024-04-25T10:11:35.285800+0000","last_unstale":"2024-04-25T10:11:35.292755+0000","last_undegraded":"2024-04-25T10:11:35.292755+0000","last_fullsized":"2024-04-25T10:11:35.292755+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:57.570901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.309054+0000","last_change":"2024-04-25T10:11:35.302288+0000","last_active":"2024-04-25T10:11:35.309054+0000","last_peered":"2024-04-25T10:11:35.309054+0000","last_clean":"2024-04-25T10:11:35.309054+0000","last_became_active":"2024-04-25T10:11:35.301807+0000","last_became_peered":"2024-04-25T10:11:35.301807+0000","last_unstale":"2024-04-25T10:11:35.309054+0000","last_undegraded":"2024-04-25T10:11:35.309054+0000","last_fullsized":"2024-04-25T10:11:35.309054+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:00.087291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304247+0000","last_change":"2024-04-25T10:11:35.299438+0000","last_active":"2024-04-25T10:11:35.304247+0000","last_peered":"2024-04-25T10:11:35.304247+0000","last_clean":"2024-04-25T10:11:35.304247+0000","last_became_active":"2024-04-25T10:11:35.298771+0000","last_became_peered":"2024-04-25T10:11:35.298771+0000","last_unstale":"2024-04-25T10:11:35.304247+0000","last_undegraded":"2024-04-25T10:11:35.304247+0000","last_fullsized":"2024-04-25T10:11:35.304247+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:38.314011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.302934+0000","last_change":"2024-04-25T10:11:35.292222+0000","last_active":"2024-04-25T10:11:35.302934+0000","last_peered":"2024-04-25T10:11:35.302934+0000","last_clean":"2024-04-25T10:11:35.302934+0000","last_became_active":"2024-04-25T10:11:35.291816+0000","last_became_peered":"2024-04-25T10:11:35.291816+0000","last_unstale":"2024-04-25T10:11:35.302934+0000","last_undegraded":"2024-04-25T10:11:35.302934+0000","last_fullsized":"2024-04-25T10:11:35.302934+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:00.087291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304585+0000","last_change":"2024-04-25T10:11:35.291437+0000","last_active":"2024-04-25T10:11:35.304585+0000","last_peered":"2024-04-25T10:11:35.304585+0000","last_clean":"2024-04-25T10:11:35.304585+0000","last_became_active":"2024-04-25T10:11:35.291163+0000","last_became_peered":"2024-04-25T10:11:35.291163+0000","last_unstale":"2024-04-25T10:11:35.304585+0000","last_undegraded":"2024-04-25T10:11:35.304585+0000","last_fullsized":"2024-04-25T10:11:35.304585+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:20.858581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.289732+0000","last_change":"2024-04-25T10:11:35.285958+0000","last_active":"2024-04-25T10:11:35.289732+0000","last_peered":"2024-04-25T10:11:35.289732+0000","last_clean":"2024-04-25T10:11:35.289732+0000","last_became_active":"2024-04-25T10:11:35.285008+0000","last_became_peered":"2024-04-25T10:11:35.285008+0000","last_unstale":"2024-04-25T10:11:35.289732+0000","last_undegraded":"2024-04-25T10:11:35.289732+0000","last_fullsized":"2024-04-25T10:11:35.289732+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:34.745226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304929+0000","last_change":"2024-04-25T10:11:35.295932+0000","last_active":"2024-04-25T10:11:35.304929+0000","last_peered":"2024-04-25T10:11:35.304929+0000","last_clean":"2024-04-25T10:11:35.304929+0000","last_became_active":"2024-04-25T10:11:35.293823+0000","last_became_peered":"2024-04-25T10:11:35.293823+0000","last_unstale":"2024-04-25T10:11:35.304929+0000","last_undegraded":"2024-04-25T10:11:35.304929+0000","last_fullsized":"2024-04-25T10:11:35.304929+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:29.059664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.308432+0000","last_change":"2024-04-25T10:11:35.296917+0000","last_active":"2024-04-25T10:11:35.308432+0000","last_peered":"2024-04-25T10:11:35.308432+0000","last_clean":"2024-04-25T10:11:35.308432+0000","last_became_active":"2024-04-25T10:11:35.296736+0000","last_became_peered":"2024-04-25T10:11:35.296736+0000","last_unstale":"2024-04-25T10:11:35.308432+0000","last_undegraded":"2024-04-25T10:11:35.308432+0000","last_fullsized":"2024-04-25T10:11:35.308432+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:11.384430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.300446+0000","last_change":"2024-04-25T10:11:35.292313+0000","last_active":"2024-04-25T10:11:35.300446+0000","last_peered":"2024-04-25T10:11:35.300446+0000","last_clean":"2024-04-25T10:11:35.300446+0000","last_became_active":"2024-04-25T10:11:35.291955+0000","last_became_peered":"2024-04-25T10:11:35.291955+0000","last_unstale":"2024-04-25T10:11:35.300446+0000","last_undegraded":"2024-04-25T10:11:35.300446+0000","last_fullsized":"2024-04-25T10:11:35.300446+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:20.836377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.312549+0000","last_change":"2024-04-25T10:11:35.305679+0000","last_active":"2024-04-25T10:11:35.312549+0000","last_peered":"2024-04-25T10:11:35.312549+0000","last_clean":"2024-04-25T10:11:35.312549+0000","last_became_active":"2024-04-25T10:11:35.305384+0000","last_became_peered":"2024-04-25T10:11:35.305384+0000","last_unstale":"2024-04-25T10:11:35.312549+0000","last_undegraded":"2024-04-25T10:11:35.312549+0000","last_fullsized":"2024-04-25T10:11:35.312549+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:28.247206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.286844+0000","last_change":"2024-04-25T10:11:35.282460+0000","last_active":"2024-04-25T10:11:35.286844+0000","last_peered":"2024-04-25T10:11:35.286844+0000","last_clean":"2024-04-25T10:11:35.286844+0000","last_became_active":"2024-04-25T10:11:35.281151+0000","last_became_peered":"2024-04-25T10:11:35.281151+0000","last_unstale":"2024-04-25T10:11:35.286844+0000","last_undegraded":"2024-04-25T10:11:35.286844+0000","last_fullsized":"2024-04-25T10:11:35.286844+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:07.791553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.286356+0000","last_change":"2024-04-25T10:11:35.276678+0000","last_active":"2024-04-25T10:11:35.286356+0000","last_peered":"2024-04-25T10:11:35.286356+0000","last_clean":"2024-04-25T10:11:35.286356+0000","last_became_active":"2024-04-25T10:11:35.276030+0000","last_became_peered":"2024-04-25T10:11:35.276030+0000","last_unstale":"2024-04-25T10:11:35.286356+0000","last_undegraded":"2024-04-25T10:11:35.286356+0000","last_fullsized":"2024-04-25T10:11:35.286356+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:33.808111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.290939+0000","last_change":"2024-04-25T10:11:35.285768+0000","last_active":"2024-04-25T10:11:35.290939+0000","last_peered":"2024-04-25T10:11:35.290939+0000","last_clean":"2024-04-25T10:11:35.290939+0000","last_became_active":"2024-04-25T10:11:35.284589+0000","last_became_peered":"2024-04-25T10:11:35.284589+0000","last_unstale":"2024-04-25T10:11:35.290939+0000","last_undegraded":"2024-04-25T10:11:35.290939+0000","last_fullsized":"2024-04-25T10:11:35.290939+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:37.334486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.296574+0000","last_change":"2024-04-25T10:11:35.290247+0000","last_active":"2024-04-25T10:11:35.296574+0000","last_peered":"2024-04-25T10:11:35.296574+0000","last_clean":"2024-04-25T10:11:35.296574+0000","last_became_active":"2024-04-25T10:11:35.289914+0000","last_became_peered":"2024-04-25T10:11:35.289914+0000","last_unstale":"2024-04-25T10:11:35.296574+0000","last_undegraded":"2024-04-25T10:11:35.296574+0000","last_fullsized":"2024-04-25T10:11:35.296574+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:46.520578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304130+0000","last_change":"2024-04-25T10:11:35.298244+0000","last_active":"2024-04-25T10:11:35.304130+0000","last_peered":"2024-04-25T10:11:35.304130+0000","last_clean":"2024-04-25T10:11:35.304130+0000","last_became_active":"2024-04-25T10:11:35.297969+0000","last_became_peered":"2024-04-25T10:11:35.297969+0000","last_unstale":"2024-04-25T10:11:35.304130+0000","last_undegraded":"2024-04-25T10:11:35.304130+0000","last_fullsized":"2024-04-25T10:11:35.304130+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:32.327826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.303838+0000","last_change":"2024-04-25T10:11:35.296178+0000","last_active":"2024-04-25T10:11:35.303838+0000","last_peered":"2024-04-25T10:11:35.303838+0000","last_clean":"2024-04-25T10:11:35.303838+0000","last_became_active":"2024-04-25T10:11:35.295877+0000","last_became_peered":"2024-04-25T10:11:35.295877+0000","last_unstale":"2024-04-25T10:11:35.303838+0000","last_undegraded":"2024-04-25T10:11:35.303838+0000","last_fullsized":"2024-04-25T10:11:35.303838+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:19.216901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.298778+0000","last_change":"2024-04-25T10:11:35.295743+0000","last_active":"2024-04-25T10:11:35.298778+0000","last_peered":"2024-04-25T10:11:35.298778+0000","last_clean":"2024-04-25T10:11:35.298778+0000","last_became_active":"2024-04-25T10:11:35.295201+0000","last_became_peered":"2024-04-25T10:11:35.295201+0000","last_unstale":"2024-04-25T10:11:35.298778+0000","last_undegraded":"2024-04-25T10:11:35.298778+0000","last_fullsized":"2024-04-25T10:11:35.298778+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:46.520578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304096+0000","last_change":"2024-04-25T10:11:35.295755+0000","last_active":"2024-04-25T10:11:35.304096+0000","last_peered":"2024-04-25T10:11:35.304096+0000","last_clean":"2024-04-25T10:11:35.304096+0000","last_became_active":"2024-04-25T10:11:35.292401+0000","last_became_peered":"2024-04-25T10:11:35.292401+0000","last_unstale":"2024-04-25T10:11:35.304096+0000","last_undegraded":"2024-04-25T10:11:35.304096+0000","last_fullsized":"2024-04-25T10:11:35.304096+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:35.319360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.297320+0000","last_change":"2024-04-25T10:11:35.288725+0000","last_active":"2024-04-25T10:11:35.297320+0000","last_peered":"2024-04-25T10:11:35.297320+0000","last_clean":"2024-04-25T10:11:35.297320+0000","last_became_active":"2024-04-25T10:11:35.288408+0000","last_became_peered":"2024-04-25T10:11:35.288408+0000","last_unstale":"2024-04-25T10:11:35.297320+0000","last_undegraded":"2024-04-25T10:11:35.297320+0000","last_fullsized":"2024-04-25T10:11:35.297320+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:10.957813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.298360+0000","last_change":"2024-04-25T10:11:35.289236+0000","last_active":"2024-04-25T10:11:35.298360+0000","last_peered":"2024-04-25T10:11:35.298360+0000","last_clean":"2024-04-25T10:11:35.298360+0000","last_became_active":"2024-04-25T10:11:35.288910+0000","last_became_peered":"2024-04-25T10:11:35.288910+0000","last_unstale":"2024-04-25T10:11:35.298360+0000","last_undegraded":"2024-04-25T10:11:35.298360+0000","last_fullsized":"2024-04-25T10:11:35.298360+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:11.983167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.301062+0000","last_change":"2024-04-25T10:11:35.295458+0000","last_active":"2024-04-25T10:11:35.301062+0000","last_peered":"2024-04-25T10:11:35.301062+0000","last_clean":"2024-04-25T10:11:35.301062+0000","last_became_active":"2024-04-25T10:11:35.294990+0000","last_became_peered":"2024-04-25T10:11:35.294990+0000","last_unstale":"2024-04-25T10:11:35.301062+0000","last_undegraded":"2024-04-25T10:11:35.301062+0000","last_fullsized":"2024-04-25T10:11:35.301062+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:41.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.294575+0000","last_change":"2024-04-25T10:11:35.284590+0000","last_active":"2024-04-25T10:11:35.294575+0000","last_peered":"2024-04-25T10:11:35.294575+0000","last_clean":"2024-04-25T10:11:35.294575+0000","last_became_active":"2024-04-25T10:11:35.284305+0000","last_became_peered":"2024-04-25T10:11:35.284305+0000","last_unstale":"2024-04-25T10:11:35.294575+0000","last_undegraded":"2024-04-25T10:11:35.294575+0000","last_fullsized":"2024-04-25T10:11:35.294575+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:19.216901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.312134+0000","last_change":"2024-04-25T10:11:35.305947+0000","last_active":"2024-04-25T10:11:35.312134+0000","last_peered":"2024-04-25T10:11:35.312134+0000","last_clean":"2024-04-25T10:11:35.312134+0000","last_became_active":"2024-04-25T10:11:35.305374+0000","last_became_peered":"2024-04-25T10:11:35.305374+0000","last_unstale":"2024-04-25T10:11:35.312134+0000","last_undegraded":"2024-04-25T10:11:35.312134+0000","last_fullsized":"2024-04-25T10:11:35.312134+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:49.908492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.307989+0000","last_change":"2024-04-25T10:11:35.299624+0000","last_active":"2024-04-25T10:11:35.307989+0000","last_peered":"2024-04-25T10:11:35.307989+0000","last_clean":"2024-04-25T10:11:35.307989+0000","last_became_active":"2024-04-25T10:11:35.299324+0000","last_became_peered":"2024-04-25T10:11:35.299324+0000","last_unstale":"2024-04-25T10:11:35.307989+0000","last_undegraded":"2024-04-25T10:11:35.307989+0000","last_fullsized":"2024-04-25T10:11:35.307989+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:29.604949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"66'4","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.313680+0000","last_change":"2024-04-25T10:11:34.286746+0000","last_active":"2024-04-25T10:11:35.313680+0000","last_peered":"2024-04-25T10:11:35.313680+0000","last_clean":"2024-04-25T10:11:35.313680+0000","last_became_active":"2024-04-25T10:11:34.286428+0000","last_became_peered":"2024-04-25T10:11:34.286428+0000","last_unstale":"2024-04-25T10:11:35.313680+0000","last_undegraded":"2024-04-25T10:11:35.313680+0000","last_fullsized":"2024-04-25T10:11:35.313680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:46.688289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304182+0000","last_change":"2024-04-25T10:11:35.288648+0000","last_active":"2024-04-25T10:11:35.304182+0000","last_peered":"2024-04-25T10:11:35.304182+0000","last_clean":"2024-04-25T10:11:35.304182+0000","last_became_active":"2024-04-25T10:11:35.288365+0000","last_became_peered":"2024-04-25T10:11:35.288365+0000","last_unstale":"2024-04-25T10:11:35.304182+0000","last_undegraded":"2024-04-25T10:11:35.304182+0000","last_fullsized":"2024-04-25T10:11:35.304182+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:28.219428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.310724+0000","last_change":"2024-04-25T10:11:34.278951+0000","last_active":"2024-04-25T10:11:35.310724+0000","last_peered":"2024-04-25T10:11:35.310724+0000","last_clean":"2024-04-25T10:11:35.310724+0000","last_became_active":"2024-04-25T10:11:34.273919+0000","last_became_peered":"2024-04-25T10:11:34.273919+0000","last_unstale":"2024-04-25T10:11:35.310724+0000","last_undegraded":"2024-04-25T10:11:35.310724+0000","last_fullsized":"2024-04-25T10:11:35.310724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:46.529810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.297106+0000","last_change":"2024-04-25T10:11:35.293045+0000","last_active":"2024-04-25T10:11:35.297106+0000","last_peered":"2024-04-25T10:11:35.297106+0000","last_clean":"2024-04-25T10:11:35.297106+0000","last_became_active":"2024-04-25T10:11:35.292682+0000","last_became_peered":"2024-04-25T10:11:35.292682+0000","last_unstale":"2024-04-25T10:11:35.297106+0000","last_undegraded":"2024-04-25T10:11:35.297106+0000","last_fullsized":"2024-04-25T10:11:35.297106+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:02.236799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.308693+0000","last_change":"2024-04-25T10:11:34.284716+0000","last_active":"2024-04-25T10:11:35.308693+0000","last_peered":"2024-04-25T10:11:35.308693+0000","last_clean":"2024-04-25T10:11:35.308693+0000","last_became_active":"2024-04-25T10:11:34.284323+0000","last_became_peered":"2024-04-25T10:11:34.284323+0000","last_unstale":"2024-04-25T10:11:35.308693+0000","last_undegraded":"2024-04-25T10:11:35.308693+0000","last_fullsized":"2024-04-25T10:11:35.308693+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:48.893113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.294477+0000","last_change":"2024-04-25T10:11:35.289177+0000","last_active":"2024-04-25T10:11:35.294477+0000","last_peered":"2024-04-25T10:11:35.294477+0000","last_clean":"2024-04-25T10:11:35.294477+0000","last_became_active":"2024-04-25T10:11:35.288865+0000","last_became_peered":"2024-04-25T10:11:35.288865+0000","last_unstale":"2024-04-25T10:11:35.294477+0000","last_undegraded":"2024-04-25T10:11:35.294477+0000","last_fullsized":"2024-04-25T10:11:35.294477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:43.629406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.311323+0000","last_change":"2024-04-25T10:11:34.290037+0000","last_active":"2024-04-25T10:11:35.311323+0000","last_peered":"2024-04-25T10:11:35.311323+0000","last_clean":"2024-04-25T10:11:35.311323+0000","last_became_active":"2024-04-25T10:11:34.288487+0000","last_became_peered":"2024-04-25T10:11:34.288487+0000","last_unstale":"2024-04-25T10:11:35.311323+0000","last_undegraded":"2024-04-25T10:11:35.311323+0000","last_fullsized":"2024-04-25T10:11:35.311323+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:35.981626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.297862+0000","last_change":"2024-04-25T10:11:35.289434+0000","last_active":"2024-04-25T10:11:35.297862+0000","last_peered":"2024-04-25T10:11:35.297862+0000","last_clean":"2024-04-25T10:11:35.297862+0000","last_became_active":"2024-04-25T10:11:35.289151+0000","last_became_peered":"2024-04-25T10:11:35.289151+0000","last_unstale":"2024-04-25T10:11:35.297862+0000","last_undegraded":"2024-04-25T10:11:35.297862+0000","last_fullsized":"2024-04-25T10:11:35.297862+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:43.629406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.303062+0000","last_change":"2024-04-25T10:11:34.279711+0000","last_active":"2024-04-25T10:11:35.303062+0000","last_peered":"2024-04-25T10:11:35.303062+0000","last_clean":"2024-04-25T10:11:35.303062+0000","last_became_active":"2024-04-25T10:11:34.279394+0000","last_became_peered":"2024-04-25T10:11:34.279394+0000","last_unstale":"2024-04-25T10:11:35.303062+0000","last_undegraded":"2024-04-25T10:11:35.303062+0000","last_fullsized":"2024-04-25T10:11:35.303062+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:23.027335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.302088+0000","last_change":"2024-04-25T10:11:35.295035+0000","last_active":"2024-04-25T10:11:35.302088+0000","last_peered":"2024-04-25T10:11:35.302088+0000","last_clean":"2024-04-25T10:11:35.302088+0000","last_became_active":"2024-04-25T10:11:35.294737+0000","last_became_peered":"2024-04-25T10:11:35.294737+0000","last_unstale":"2024-04-25T10:11:35.302088+0000","last_undegraded":"2024-04-25T10:11:35.302088+0000","last_fullsized":"2024-04-25T10:11:35.302088+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:29.474273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.308156+0000","last_change":"2024-04-25T10:11:34.290766+0000","last_active":"2024-04-25T10:11:35.308156+0000","last_peered":"2024-04-25T10:11:35.308156+0000","last_clean":"2024-04-25T10:11:35.308156+0000","last_became_active":"2024-04-25T10:11:34.289236+0000","last_became_peered":"2024-04-25T10:11:34.289236+0000","last_unstale":"2024-04-25T10:11:35.308156+0000","last_undegraded":"2024-04-25T10:11:35.308156+0000","last_fullsized":"2024-04-25T10:11:35.308156+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:15.279109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.306233+0000","last_change":"2024-04-25T10:11:35.293321+0000","last_active":"2024-04-25T10:11:35.306233+0000","last_peered":"2024-04-25T10:11:35.306233+0000","last_clean":"2024-04-25T10:11:35.306233+0000","last_became_active":"2024-04-25T10:11:35.293032+0000","last_became_peered":"2024-04-25T10:11:35.293032+0000","last_unstale":"2024-04-25T10:11:35.306233+0000","last_undegraded":"2024-04-25T10:11:35.306233+0000","last_fullsized":"2024-04-25T10:11:35.306233+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:28.445879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.282415+0000","last_change":"2024-04-25T10:11:34.282281+0000","last_active":"2024-04-25T10:11:35.282415+0000","last_peered":"2024-04-25T10:11:35.282415+0000","last_clean":"2024-04-25T10:11:35.282415+0000","last_became_active":"2024-04-25T10:11:34.281983+0000","last_became_peered":"2024-04-25T10:11:34.281983+0000","last_unstale":"2024-04-25T10:11:35.282415+0000","last_undegraded":"2024-04-25T10:11:35.282415+0000","last_fullsized":"2024-04-25T10:11:35.282415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:03.701671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.301464+0000","last_change":"2024-04-25T10:11:35.297163+0000","last_active":"2024-04-25T10:11:35.301464+0000","last_peered":"2024-04-25T10:11:35.301464+0000","last_clean":"2024-04-25T10:11:35.301464+0000","last_became_active":"2024-04-25T10:11:35.294350+0000","last_became_peered":"2024-04-25T10:11:35.294350+0000","last_unstale":"2024-04-25T10:11:35.301464+0000","last_undegraded":"2024-04-25T10:11:35.301464+0000","last_fullsized":"2024-04-25T10:11:35.301464+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:21.346895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.279025+0000","last_change":"2024-04-25T10:11:34.281232+0000","last_active":"2024-04-25T10:11:35.279025+0000","last_peered":"2024-04-25T10:11:35.279025+0000","last_clean":"2024-04-25T10:11:35.279025+0000","last_became_active":"2024-04-25T10:11:34.280973+0000","last_became_peered":"2024-04-25T10:11:34.280973+0000","last_unstale":"2024-04-25T10:11:35.279025+0000","last_undegraded":"2024-04-25T10:11:35.279025+0000","last_fullsized":"2024-04-25T10:11:35.279025+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:49.379938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.298638+0000","last_change":"2024-04-25T10:11:35.291294+0000","last_active":"2024-04-25T10:11:35.298638+0000","last_peered":"2024-04-25T10:11:35.298638+0000","last_clean":"2024-04-25T10:11:35.298638+0000","last_became_active":"2024-04-25T10:11:35.290996+0000","last_became_peered":"2024-04-25T10:11:35.290996+0000","last_unstale":"2024-04-25T10:11:35.298638+0000","last_undegraded":"2024-04-25T10:11:35.298638+0000","last_fullsized":"2024-04-25T10:11:35.298638+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:26.632804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.269753+0000","last_change":"2024-04-25T10:11:34.276292+0000","last_active":"2024-04-25T10:11:35.269753+0000","last_peered":"2024-04-25T10:11:35.269753+0000","last_clean":"2024-04-25T10:11:35.269753+0000","last_became_active":"2024-04-25T10:11:34.275999+0000","last_became_peered":"2024-04-25T10:11:34.275999+0000","last_unstale":"2024-04-25T10:11:35.269753+0000","last_undegraded":"2024-04-25T10:11:35.269753+0000","last_fullsized":"2024-04-25T10:11:35.269753+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:25.494522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.309028+0000","last_change":"2024-04-25T10:11:35.301363+0000","last_active":"2024-04-25T10:11:35.309028+0000","last_peered":"2024-04-25T10:11:35.309028+0000","last_clean":"2024-04-25T10:11:35.309028+0000","last_became_active":"2024-04-25T10:11:35.300243+0000","last_became_peered":"2024-04-25T10:11:35.300243+0000","last_unstale":"2024-04-25T10:11:35.309028+0000","last_undegraded":"2024-04-25T10:11:35.309028+0000","last_fullsized":"2024-04-25T10:11:35.309028+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.995189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.288226+0000","last_change":"2024-04-25T10:11:34.289593+0000","last_active":"2024-04-25T10:11:35.288226+0000","last_peered":"2024-04-25T10:11:35.288226+0000","last_clean":"2024-04-25T10:11:35.288226+0000","last_became_active":"2024-04-25T10:11:34.288958+0000","last_became_peered":"2024-04-25T10:11:34.288958+0000","last_unstale":"2024-04-25T10:11:35.288226+0000","last_undegraded":"2024-04-25T10:11:35.288226+0000","last_fullsized":"2024-04-25T10:11:35.288226+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:03.701671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.312632+0000","last_change":"2024-04-25T10:11:35.307869+0000","last_active":"2024-04-25T10:11:35.312632+0000","last_peered":"2024-04-25T10:11:35.312632+0000","last_clean":"2024-04-25T10:11:35.312632+0000","last_became_active":"2024-04-25T10:11:35.307527+0000","last_became_peered":"2024-04-25T10:11:35.307527+0000","last_unstale":"2024-04-25T10:11:35.312632+0000","last_undegraded":"2024-04-25T10:11:35.312632+0000","last_fullsized":"2024-04-25T10:11:35.312632+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:33.808111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"66'3","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.307484+0000","last_change":"2024-04-25T10:11:34.282859+0000","last_active":"2024-04-25T10:11:35.307484+0000","last_peered":"2024-04-25T10:11:35.307484+0000","last_clean":"2024-04-25T10:11:35.307484+0000","last_became_active":"2024-04-25T10:11:34.282355+0000","last_became_peered":"2024-04-25T10:11:34.282355+0000","last_unstale":"2024-04-25T10:11:35.307484+0000","last_undegraded":"2024-04-25T10:11:35.307484+0000","last_fullsized":"2024-04-25T10:11:35.307484+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:24.494895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.308587+0000","last_change":"2024-04-25T10:11:35.301068+0000","last_active":"2024-04-25T10:11:35.308587+0000","last_peered":"2024-04-25T10:11:35.308587+0000","last_clean":"2024-04-25T10:11:35.308587+0000","last_became_active":"2024-04-25T10:11:35.300223+0000","last_became_peered":"2024-04-25T10:11:35.300223+0000","last_unstale":"2024-04-25T10:11:35.308587+0000","last_undegraded":"2024-04-25T10:11:35.308587+0000","last_fullsized":"2024-04-25T10:11:35.308587+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:43.286662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"66'6","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.315799+0000","last_change":"2024-04-25T10:11:34.285777+0000","last_active":"2024-04-25T10:11:35.315799+0000","last_peered":"2024-04-25T10:11:35.315799+0000","last_clean":"2024-04-25T10:11:35.315799+0000","last_became_active":"2024-04-25T10:11:34.285425+0000","last_became_peered":"2024-04-25T10:11:34.285425+0000","last_unstale":"2024-04-25T10:11:35.315799+0000","last_undegraded":"2024-04-25T10:11:35.315799+0000","last_fullsized":"2024-04-25T10:11:35.315799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:07.422965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.294540+0000","last_change":"2024-04-25T10:11:35.289614+0000","last_active":"2024-04-25T10:11:35.294540+0000","last_peered":"2024-04-25T10:11:35.294540+0000","last_clean":"2024-04-25T10:11:35.294540+0000","last_became_active":"2024-04-25T10:11:35.289234+0000","last_became_peered":"2024-04-25T10:11:35.289234+0000","last_unstale":"2024-04-25T10:11:35.294540+0000","last_undegraded":"2024-04-25T10:11:35.294540+0000","last_fullsized":"2024-04-25T10:11:35.294540+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:26.632804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.309651+0000","last_change":"2024-04-25T10:11:34.285343+0000","last_active":"2024-04-25T10:11:35.309651+0000","last_peered":"2024-04-25T10:11:35.309651+0000","last_clean":"2024-04-25T10:11:35.309651+0000","last_became_active":"2024-04-25T10:11:34.284947+0000","last_became_peered":"2024-04-25T10:11:34.284947+0000","last_unstale":"2024-04-25T10:11:35.309651+0000","last_undegraded":"2024-04-25T10:11:35.309651+0000","last_fullsized":"2024-04-25T10:11:35.309651+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:15.279109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.292650+0000","last_change":"2024-04-25T10:11:35.286959+0000","last_active":"2024-04-25T10:11:35.292650+0000","last_peered":"2024-04-25T10:11:35.292650+0000","last_clean":"2024-04-25T10:11:35.292650+0000","last_became_active":"2024-04-25T10:11:35.286141+0000","last_became_peered":"2024-04-25T10:11:35.286141+0000","last_unstale":"2024-04-25T10:11:35.292650+0000","last_undegraded":"2024-04-25T10:11:35.292650+0000","last_fullsized":"2024-04-25T10:11:35.292650+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:47.703668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.284776+0000","last_change":"2024-04-25T10:11:34.285246+0000","last_active":"2024-04-25T10:11:35.284776+0000","last_peered":"2024-04-25T10:11:35.284776+0000","last_clean":"2024-04-25T10:11:35.284776+0000","last_became_active":"2024-04-25T10:11:34.284969+0000","last_became_peered":"2024-04-25T10:11:34.284969+0000","last_unstale":"2024-04-25T10:11:35.284776+0000","last_undegraded":"2024-04-25T10:11:35.284776+0000","last_fullsized":"2024-04-25T10:11:35.284776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:35.636729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.298682+0000","last_change":"2024-04-25T10:11:35.290310+0000","last_active":"2024-04-25T10:11:35.298682+0000","last_peered":"2024-04-25T10:11:35.298682+0000","last_clean":"2024-04-25T10:11:35.298682+0000","last_became_active":"2024-04-25T10:11:35.289919+0000","last_became_peered":"2024-04-25T10:11:35.289919+0000","last_unstale":"2024-04-25T10:11:35.298682+0000","last_undegraded":"2024-04-25T10:11:35.298682+0000","last_fullsized":"2024-04-25T10:11:35.298682+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.995189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.265648+0000","last_change":"2024-04-25T10:11:34.268634+0000","last_active":"2024-04-25T10:11:35.265648+0000","last_peered":"2024-04-25T10:11:35.265648+0000","last_clean":"2024-04-25T10:11:35.265648+0000","last_became_active":"2024-04-25T10:11:34.267992+0000","last_became_peered":"2024-04-25T10:11:34.267992+0000","last_unstale":"2024-04-25T10:11:35.265648+0000","last_undegraded":"2024-04-25T10:11:35.265648+0000","last_fullsized":"2024-04-25T10:11:35.265648+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:45.371081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304291+0000","last_change":"2024-04-25T10:11:35.290684+0000","last_active":"2024-04-25T10:11:35.304291+0000","last_peered":"2024-04-25T10:11:35.304291+0000","last_clean":"2024-04-25T10:11:35.304291+0000","last_became_active":"2024-04-25T10:11:35.290355+0000","last_became_peered":"2024-04-25T10:11:35.290355+0000","last_unstale":"2024-04-25T10:11:35.304291+0000","last_undegraded":"2024-04-25T10:11:35.304291+0000","last_fullsized":"2024-04-25T10:11:35.304291+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:21.906422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"66'8","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.312885+0000","last_change":"2024-04-25T10:11:34.284924+0000","last_active":"2024-04-25T10:11:35.312885+0000","last_peered":"2024-04-25T10:11:35.312885+0000","last_clean":"2024-04-25T10:11:35.312885+0000","last_became_active":"2024-04-25T10:11:34.284529+0000","last_became_peered":"2024-04-25T10:11:34.284529+0000","last_unstale":"2024-04-25T10:11:35.312885+0000","last_undegraded":"2024-04-25T10:11:35.312885+0000","last_fullsized":"2024-04-25T10:11:35.312885+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:40.607955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.302002+0000","last_change":"2024-04-25T10:11:35.292886+0000","last_active":"2024-04-25T10:11:35.302002+0000","last_peered":"2024-04-25T10:11:35.302002+0000","last_clean":"2024-04-25T10:11:35.302002+0000","last_became_active":"2024-04-25T10:11:35.292559+0000","last_became_peered":"2024-04-25T10:11:35.292559+0000","last_unstale":"2024-04-25T10:11:35.302002+0000","last_undegraded":"2024-04-25T10:11:35.302002+0000","last_fullsized":"2024-04-25T10:11:35.302002+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:46.386460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.311927+0000","last_change":"2024-04-25T10:11:34.288990+0000","last_active":"2024-04-25T10:11:35.311927+0000","last_peered":"2024-04-25T10:11:35.311927+0000","last_clean":"2024-04-25T10:11:35.311927+0000","last_became_active":"2024-04-25T10:11:34.286484+0000","last_became_peered":"2024-04-25T10:11:34.286484+0000","last_unstale":"2024-04-25T10:11:35.311927+0000","last_undegraded":"2024-04-25T10:11:35.311927+0000","last_fullsized":"2024-04-25T10:11:35.311927+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:08.788812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.309002+0000","last_change":"2024-04-25T10:11:35.299962+0000","last_active":"2024-04-25T10:11:35.309002+0000","last_peered":"2024-04-25T10:11:35.309002+0000","last_clean":"2024-04-25T10:11:35.309002+0000","last_became_active":"2024-04-25T10:11:35.299597+0000","last_became_peered":"2024-04-25T10:11:35.299597+0000","last_unstale":"2024-04-25T10:11:35.309002+0000","last_undegraded":"2024-04-25T10:11:35.309002+0000","last_fullsized":"2024-04-25T10:11:35.309002+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:34.745226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.308883+0000","last_change":"2024-04-25T10:11:34.284619+0000","last_active":"2024-04-25T10:11:35.308883+0000","last_peered":"2024-04-25T10:11:35.308883+0000","last_clean":"2024-04-25T10:11:35.308883+0000","last_became_active":"2024-04-25T10:11:34.284300+0000","last_became_peered":"2024-04-25T10:11:34.284300+0000","last_unstale":"2024-04-25T10:11:35.308883+0000","last_undegraded":"2024-04-25T10:11:35.308883+0000","last_fullsized":"2024-04-25T10:11:35.308883+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:27.231827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304223+0000","last_change":"2024-04-25T10:11:35.299182+0000","last_active":"2024-04-25T10:11:35.304223+0000","last_peered":"2024-04-25T10:11:35.304223+0000","last_clean":"2024-04-25T10:11:35.304223+0000","last_became_active":"2024-04-25T10:11:35.297616+0000","last_became_peered":"2024-04-25T10:11:35.297616+0000","last_unstale":"2024-04-25T10:11:35.304223+0000","last_undegraded":"2024-04-25T10:11:35.304223+0000","last_fullsized":"2024-04-25T10:11:35.304223+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:36.997005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.307611+0000","last_change":"2024-04-25T10:11:34.290024+0000","last_active":"2024-04-25T10:11:35.307611+0000","last_peered":"2024-04-25T10:11:35.307611+0000","last_clean":"2024-04-25T10:11:35.307611+0000","last_became_active":"2024-04-25T10:11:34.288707+0000","last_became_peered":"2024-04-25T10:11:34.288707+0000","last_unstale":"2024-04-25T10:11:35.307611+0000","last_undegraded":"2024-04-25T10:11:35.307611+0000","last_fullsized":"2024-04-25T10:11:35.307611+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:19.820998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.288112+0000","last_change":"2024-04-25T10:11:35.284777+0000","last_active":"2024-04-25T10:11:35.288112+0000","last_peered":"2024-04-25T10:11:35.288112+0000","last_clean":"2024-04-25T10:11:35.288112+0000","last_became_active":"2024-04-25T10:11:35.284516+0000","last_became_peered":"2024-04-25T10:11:35.284516+0000","last_unstale":"2024-04-25T10:11:35.288112+0000","last_undegraded":"2024-04-25T10:11:35.288112+0000","last_fullsized":"2024-04-25T10:11:35.288112+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:38.861057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.277842+0000","last_change":"2024-04-25T10:11:34.289744+0000","last_active":"2024-04-25T10:11:35.277842+0000","last_peered":"2024-04-25T10:11:35.277842+0000","last_clean":"2024-04-25T10:11:35.277842+0000","last_became_active":"2024-04-25T10:11:34.289462+0000","last_became_peered":"2024-04-25T10:11:34.289462+0000","last_unstale":"2024-04-25T10:11:35.277842+0000","last_undegraded":"2024-04-25T10:11:35.277842+0000","last_fullsized":"2024-04-25T10:11:35.277842+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:18.201522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.305154+0000","last_change":"2024-04-25T10:11:35.297567+0000","last_active":"2024-04-25T10:11:35.305154+0000","last_peered":"2024-04-25T10:11:35.305154+0000","last_clean":"2024-04-25T10:11:35.305154+0000","last_became_active":"2024-04-25T10:11:35.297242+0000","last_became_peered":"2024-04-25T10:11:35.297242+0000","last_unstale":"2024-04-25T10:11:35.305154+0000","last_undegraded":"2024-04-25T10:11:35.305154+0000","last_fullsized":"2024-04-25T10:11:35.305154+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:30.558925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.287290+0000","last_change":"2024-04-25T10:11:34.289995+0000","last_active":"2024-04-25T10:11:35.287290+0000","last_peered":"2024-04-25T10:11:35.287290+0000","last_clean":"2024-04-25T10:11:35.287290+0000","last_became_active":"2024-04-25T10:11:34.289649+0000","last_became_peered":"2024-04-25T10:11:34.289649+0000","last_unstale":"2024-04-25T10:11:35.287290+0000","last_undegraded":"2024-04-25T10:11:35.287290+0000","last_fullsized":"2024-04-25T10:11:35.287290+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:07.422965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.298510+0000","last_change":"2024-04-25T10:11:35.290526+0000","last_active":"2024-04-25T10:11:35.298510+0000","last_peered":"2024-04-25T10:11:35.298510+0000","last_clean":"2024-04-25T10:11:35.298510+0000","last_became_active":"2024-04-25T10:11:35.290149+0000","last_became_peered":"2024-04-25T10:11:35.290149+0000","last_unstale":"2024-04-25T10:11:35.298510+0000","last_undegraded":"2024-04-25T10:11:35.298510+0000","last_fullsized":"2024-04-25T10:11:35.298510+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:23.616117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.267125+0000","last_change":"2024-04-25T10:11:34.278538+0000","last_active":"2024-04-25T10:11:35.267125+0000","last_peered":"2024-04-25T10:11:35.267125+0000","last_clean":"2024-04-25T10:11:35.267125+0000","last_became_active":"2024-04-25T10:11:34.277907+0000","last_became_peered":"2024-04-25T10:11:34.277907+0000","last_unstale":"2024-04-25T10:11:35.267125+0000","last_undegraded":"2024-04-25T10:11:35.267125+0000","last_fullsized":"2024-04-25T10:11:35.267125+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:28.458894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304448+0000","last_change":"2024-04-25T10:11:35.296618+0000","last_active":"2024-04-25T10:11:35.304448+0000","last_peered":"2024-04-25T10:11:35.304448+0000","last_clean":"2024-04-25T10:11:35.304448+0000","last_became_active":"2024-04-25T10:11:35.296322+0000","last_became_peered":"2024-04-25T10:11:35.296322+0000","last_unstale":"2024-04-25T10:11:35.304448+0000","last_undegraded":"2024-04-25T10:11:35.304448+0000","last_fullsized":"2024-04-25T10:11:35.304448+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:00.339202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.259134+0000","last_change":"2024-04-25T10:11:34.277383+0000","last_active":"2024-04-25T10:11:35.259134+0000","last_peered":"2024-04-25T10:11:35.259134+0000","last_clean":"2024-04-25T10:11:35.259134+0000","last_became_active":"2024-04-25T10:11:34.277085+0000","last_became_peered":"2024-04-25T10:11:34.277085+0000","last_unstale":"2024-04-25T10:11:35.259134+0000","last_undegraded":"2024-04-25T10:11:35.259134+0000","last_fullsized":"2024-04-25T10:11:35.259134+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:45.505199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.289515+0000","last_change":"2024-04-25T10:11:35.285357+0000","last_active":"2024-04-25T10:11:35.289515+0000","last_peered":"2024-04-25T10:11:35.289515+0000","last_clean":"2024-04-25T10:11:35.289515+0000","last_became_active":"2024-04-25T10:11:35.285018+0000","last_became_peered":"2024-04-25T10:11:35.285018+0000","last_unstale":"2024-04-25T10:11:35.289515+0000","last_undegraded":"2024-04-25T10:11:35.289515+0000","last_fullsized":"2024-04-25T10:11:35.289515+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:29.604949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.309887+0000","last_change":"2024-04-25T10:11:34.289251+0000","last_active":"2024-04-25T10:11:35.309887+0000","last_peered":"2024-04-25T10:11:35.309887+0000","last_clean":"2024-04-25T10:11:35.309887+0000","last_became_active":"2024-04-25T10:11:34.288582+0000","last_became_peered":"2024-04-25T10:11:34.288582+0000","last_unstale":"2024-04-25T10:11:35.309887+0000","last_undegraded":"2024-04-25T10:11:35.309887+0000","last_fullsized":"2024-04-25T10:11:35.309887+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:13.498898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.306871+0000","last_change":"2024-04-25T10:11:35.300735+0000","last_active":"2024-04-25T10:11:35.306871+0000","last_peered":"2024-04-25T10:11:35.306871+0000","last_clean":"2024-04-25T10:11:35.306871+0000","last_became_active":"2024-04-25T10:11:35.297691+0000","last_became_peered":"2024-04-25T10:11:35.297691+0000","last_unstale":"2024-04-25T10:11:35.306871+0000","last_undegraded":"2024-04-25T10:11:35.306871+0000","last_fullsized":"2024-04-25T10:11:35.306871+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:32.327826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.271734+0000","last_change":"2024-04-25T10:11:34.270969+0000","last_active":"2024-04-25T10:11:35.271734+0000","last_peered":"2024-04-25T10:11:35.271734+0000","last_clean":"2024-04-25T10:11:35.271734+0000","last_became_active":"2024-04-25T10:11:34.269806+0000","last_became_peered":"2024-04-25T10:11:34.269806+0000","last_unstale":"2024-04-25T10:11:35.271734+0000","last_undegraded":"2024-04-25T10:11:35.271734+0000","last_fullsized":"2024-04-25T10:11:35.271734+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:28.044285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.307742+0000","last_change":"2024-04-25T10:11:35.299259+0000","last_active":"2024-04-25T10:11:35.307742+0000","last_peered":"2024-04-25T10:11:35.307742+0000","last_clean":"2024-04-25T10:11:35.307742+0000","last_became_active":"2024-04-25T10:11:35.298960+0000","last_became_peered":"2024-04-25T10:11:35.298960+0000","last_unstale":"2024-04-25T10:11:35.307742+0000","last_undegraded":"2024-04-25T10:11:35.307742+0000","last_fullsized":"2024-04-25T10:11:35.307742+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:07.791553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.279191+0000","last_change":"2024-04-25T10:11:34.275045+0000","last_active":"2024-04-25T10:11:35.279191+0000","last_peered":"2024-04-25T10:11:35.279191+0000","last_clean":"2024-04-25T10:11:35.279191+0000","last_became_active":"2024-04-25T10:11:34.274675+0000","last_became_peered":"2024-04-25T10:11:34.274675+0000","last_unstale":"2024-04-25T10:11:35.279191+0000","last_undegraded":"2024-04-25T10:11:35.279191+0000","last_fullsized":"2024-04-25T10:11:35.279191+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:01.729149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.301861+0000","last_change":"2024-04-25T10:11:35.289935+0000","last_active":"2024-04-25T10:11:35.301861+0000","last_peered":"2024-04-25T10:11:35.301861+0000","last_clean":"2024-04-25T10:11:35.301861+0000","last_became_active":"2024-04-25T10:11:35.289524+0000","last_became_peered":"2024-04-25T10:11:35.289524+0000","last_unstale":"2024-04-25T10:11:35.301861+0000","last_undegraded":"2024-04-25T10:11:35.301861+0000","last_fullsized":"2024-04-25T10:11:35.301861+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:41.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.260211+0000","last_change":"2024-04-25T10:11:34.277305+0000","last_active":"2024-04-25T10:11:35.260211+0000","last_peered":"2024-04-25T10:11:35.260211+0000","last_clean":"2024-04-25T10:11:35.260211+0000","last_became_active":"2024-04-25T10:11:34.276945+0000","last_became_peered":"2024-04-25T10:11:34.276945+0000","last_unstale":"2024-04-25T10:11:35.260211+0000","last_undegraded":"2024-04-25T10:11:35.260211+0000","last_fullsized":"2024-04-25T10:11:35.260211+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:31.312447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.308248+0000","last_change":"2024-04-25T10:11:35.296112+0000","last_active":"2024-04-25T10:11:35.308248+0000","last_peered":"2024-04-25T10:11:35.308248+0000","last_clean":"2024-04-25T10:11:35.308248+0000","last_became_active":"2024-04-25T10:11:35.295803+0000","last_became_peered":"2024-04-25T10:11:35.295803+0000","last_unstale":"2024-04-25T10:11:35.308248+0000","last_undegraded":"2024-04-25T10:11:35.308248+0000","last_fullsized":"2024-04-25T10:11:35.308248+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:43.322009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.281696+0000","last_change":"2024-04-25T10:11:34.280684+0000","last_active":"2024-04-25T10:11:35.281696+0000","last_peered":"2024-04-25T10:11:35.281696+0000","last_clean":"2024-04-25T10:11:35.281696+0000","last_became_active":"2024-04-25T10:11:34.280258+0000","last_became_peered":"2024-04-25T10:11:34.280258+0000","last_unstale":"2024-04-25T10:11:35.281696+0000","last_undegraded":"2024-04-25T10:11:35.281696+0000","last_fullsized":"2024-04-25T10:11:35.281696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:20.941871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.301568+0000","last_change":"2024-04-25T10:11:35.292920+0000","last_active":"2024-04-25T10:11:35.301568+0000","last_peered":"2024-04-25T10:11:35.301568+0000","last_clean":"2024-04-25T10:11:35.301568+0000","last_became_active":"2024-04-25T10:11:35.292602+0000","last_became_peered":"2024-04-25T10:11:35.292602+0000","last_unstale":"2024-04-25T10:11:35.301568+0000","last_undegraded":"2024-04-25T10:11:35.301568+0000","last_fullsized":"2024-04-25T10:11:35.301568+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:38.314011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.275560+0000","last_change":"2024-04-25T10:11:34.287140+0000","last_active":"2024-04-25T10:11:35.275560+0000","last_peered":"2024-04-25T10:11:35.275560+0000","last_clean":"2024-04-25T10:11:35.275560+0000","last_became_active":"2024-04-25T10:11:34.286827+0000","last_became_peered":"2024-04-25T10:11:34.286827+0000","last_unstale":"2024-04-25T10:11:35.275560+0000","last_undegraded":"2024-04-25T10:11:35.275560+0000","last_fullsized":"2024-04-25T10:11:35.275560+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:35.636729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.300506+0000","last_change":"2024-04-25T10:11:35.296096+0000","last_active":"2024-04-25T10:11:35.300506+0000","last_peered":"2024-04-25T10:11:35.300506+0000","last_clean":"2024-04-25T10:11:35.300506+0000","last_became_active":"2024-04-25T10:11:35.295797+0000","last_became_peered":"2024-04-25T10:11:35.295797+0000","last_unstale":"2024-04-25T10:11:35.300506+0000","last_undegraded":"2024-04-25T10:11:35.300506+0000","last_fullsized":"2024-04-25T10:11:35.300506+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:14.514277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.264614+0000","last_change":"2024-04-25T10:11:34.282019+0000","last_active":"2024-04-25T10:11:35.264614+0000","last_peered":"2024-04-25T10:11:35.264614+0000","last_clean":"2024-04-25T10:11:35.264614+0000","last_became_active":"2024-04-25T10:11:34.281724+0000","last_became_peered":"2024-04-25T10:11:34.281724+0000","last_unstale":"2024-04-25T10:11:35.264614+0000","last_undegraded":"2024-04-25T10:11:35.264614+0000","last_fullsized":"2024-04-25T10:11:35.264614+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:31.283351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304332+0000","last_change":"2024-04-25T10:11:35.296507+0000","last_active":"2024-04-25T10:11:35.304332+0000","last_peered":"2024-04-25T10:11:35.304332+0000","last_clean":"2024-04-25T10:11:35.304332+0000","last_became_active":"2024-04-25T10:11:35.296251+0000","last_became_peered":"2024-04-25T10:11:35.296251+0000","last_unstale":"2024-04-25T10:11:35.304332+0000","last_undegraded":"2024-04-25T10:11:35.304332+0000","last_fullsized":"2024-04-25T10:11:35.304332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:32.298730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.282260+0000","last_change":"2024-04-25T10:11:34.288829+0000","last_active":"2024-04-25T10:11:35.282260+0000","last_peered":"2024-04-25T10:11:35.282260+0000","last_clean":"2024-04-25T10:11:35.282260+0000","last_became_active":"2024-04-25T10:11:34.286659+0000","last_became_peered":"2024-04-25T10:11:34.286659+0000","last_unstale":"2024-04-25T10:11:35.282260+0000","last_undegraded":"2024-04-25T10:11:35.282260+0000","last_fullsized":"2024-04-25T10:11:35.282260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:23.584908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.303851+0000","last_change":"2024-04-25T10:11:35.297109+0000","last_active":"2024-04-25T10:11:35.303851+0000","last_peered":"2024-04-25T10:11:35.303851+0000","last_clean":"2024-04-25T10:11:35.303851+0000","last_became_active":"2024-04-25T10:11:35.296802+0000","last_became_peered":"2024-04-25T10:11:35.296802+0000","last_unstale":"2024-04-25T10:11:35.303851+0000","last_undegraded":"2024-04-25T10:11:35.303851+0000","last_fullsized":"2024-04-25T10:11:35.303851+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:40.761823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.311072+0000","last_change":"2024-04-25T10:11:34.289411+0000","last_active":"2024-04-25T10:11:35.311072+0000","last_peered":"2024-04-25T10:11:35.311072+0000","last_clean":"2024-04-25T10:11:35.311072+0000","last_became_active":"2024-04-25T10:11:34.289084+0000","last_became_peered":"2024-04-25T10:11:34.289084+0000","last_unstale":"2024-04-25T10:11:35.311072+0000","last_undegraded":"2024-04-25T10:11:35.311072+0000","last_fullsized":"2024-04-25T10:11:35.311072+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:54.490324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":256,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.280466+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:11:35.280466+0000","last_peered":"2024-04-25T10:11:35.280466+0000","last_clean":"2024-04-25T10:11:35.280466+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:11:35.280466+0000","last_undegraded":"2024-04-25T10:11:35.280466+0000","last_fullsized":"2024-04-25T10:11:35.280466+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1384448,"data_stored":918670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":335169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":335167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":335147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":335189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":815104,"data_stored":336689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":790528,"data_stored":328302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":925445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:39.699 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:39.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.933+0000 7f0faec00640 1 Processor -- start 2024-04-25T10:11:39.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.933+0000 7f0faec00640 1 -- start start 2024-04-25T10:11:39.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.934+0000 7f0faec00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0fa806e8f0 0x7f0fa815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.934+0000 7f0faec00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0fa815c820 0x7f0fa81654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.934+0000 7f0faec00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa80ce6c0 con 0x7f0fa815bf00 2024-04-25T10:11:39.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.934+0000 7f0faec00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa81455e0 con 0x7f0fa806e8f0 2024-04-25T10:11:39.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.934+0000 7f0faec00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa806d450 con 0x7f0fa815c820 2024-04-25T10:11:39.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.935+0000 7f0fa7400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0fa806e8f0 0x7f0fa815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.935+0000 7f0fa7400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0fa806e8f0 0x7f0fa815c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43096/0 (socket says 172.21.15.119:43096) 2024-04-25T10:11:39.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.935+0000 7f0fa7400640 1 -- 172.21.15.119:0/610951521 learned_addr learned my addr 172.21.15.119:0/610951521 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:39.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.935+0000 7f0facc00640 1 --2- 172.21.15.119:0/610951521 >> v2:172.21.15.132:3300/0 conn(0x7f0fa815c820 0x7f0fa81654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.935+0000 7f0fa7400640 1 -- 172.21.15.119:0/610951521 >> v2:172.21.15.132:3300/0 conn(0x7f0fa815c820 msgr2=0x7f0fa81654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.935+0000 7f0fa7400640 1 --2- 172.21.15.119:0/610951521 >> v2:172.21.15.132:3300/0 conn(0x7f0fa815c820 0x7f0fa81654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.935+0000 7f0fa7400640 1 -- 172.21.15.119:0/610951521 >> v1:172.21.15.132:6789/0 conn(0x7f0fa815bf00 legacy=0x7f0fa80cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.935+0000 7f0fa7400640 1 -- 172.21.15.119:0/610951521 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa80cc4e0 con 0x7f0fa806e8f0 2024-04-25T10:11:39.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.937+0000 7f0fa7400640 1 --2- 172.21.15.119:0/610951521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa806e8f0 0x7f0fa815c2e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80cdf50 tx=0x7f0f9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecff37e21b629290 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.937+0000 7f0fa6a00640 1 -- 172.21.15.119:0/610951521 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f94037070 con 0x7f0fa806e8f0 2024-04-25T10:11:39.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.937+0000 7f0fa6a00640 1 -- 172.21.15.119:0/610951521 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f94032440 con 0x7f0fa806e8f0 2024-04-25T10:11:39.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.937+0000 7f0fa6a00640 1 -- 172.21.15.119:0/610951521 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f94032810 con 0x7f0fa806e8f0 2024-04-25T10:11:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.938+0000 7f0faec00640 1 -- 172.21.15.119:0/610951521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa806e8f0 msgr2=0x7f0fa815c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.938+0000 7f0faec00640 1 --2- 172.21.15.119:0/610951521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa806e8f0 0x7f0fa815c2e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80cdf50 tx=0x7f0f9402a270 comp rx=0 tx=0).stop 2024-04-25T10:11:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.938+0000 7f0faec00640 1 -- 172.21.15.119:0/610951521 shutdown_connections 2024-04-25T10:11:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.938+0000 7f0faec00640 1 --2- 172.21.15.119:0/610951521 >> v2:172.21.15.132:3300/0 conn(0x7f0fa815c820 0x7f0fa81654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.938+0000 7f0faec00640 1 --2- 172.21.15.119:0/610951521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa806e8f0 0x7f0fa815c2e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.938+0000 7f0faec00640 1 -- 172.21.15.119:0/610951521 >> 172.21.15.119:0/610951521 conn(0x7f0fa806aba0 msgr2=0x7f0fa806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.938+0000 7f0faec00640 1 -- 172.21.15.119:0/610951521 shutdown_connections 2024-04-25T10:11:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.938+0000 7f0faec00640 1 -- 172.21.15.119:0/610951521 wait complete. 2024-04-25T10:11:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.940+0000 7f0faec00640 1 Processor -- start 2024-04-25T10:11:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.940+0000 7f0faec00640 1 -- start start 2024-04-25T10:11:39.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.941+0000 7f0faec00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa806e8f0 0x7f0fa8158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.941+0000 7f0faec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fa815bf00 0x7f0fa815dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.941+0000 7f0faec00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa815c820 0x7f0fa815e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.941+0000 7f0faec00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa80ccd70 con 0x7f0fa815c820 2024-04-25T10:11:39.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.941+0000 7f0faec00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa80cca90 con 0x7f0fa806e8f0 2024-04-25T10:11:39.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.941+0000 7f0faec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa80cd050 con 0x7f0fa815bf00 2024-04-25T10:11:39.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.941+0000 7f0facc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa815c820 0x7f0fa815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.941+0000 7f0facc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa815c820 0x7f0fa815e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43110/0 (socket says 172.21.15.119:43110) 2024-04-25T10:11:39.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.941+0000 7f0facc00640 1 -- 172.21.15.119:0/2250308216 learned_addr learned my addr 172.21.15.119:0/2250308216 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:39.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.941+0000 7f0facc00640 1 -- 172.21.15.119:0/2250308216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fa815bf00 msgr2=0x7f0fa815dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.941+0000 7f0facc00640 1 --2- 172.21.15.119:0/2250308216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fa815bf00 0x7f0fa815dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.941+0000 7f0facc00640 1 -- 172.21.15.119:0/2250308216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa806e8f0 msgr2=0x7f0fa8158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.941+0000 7f0fa7e00640 1 --2- 172.21.15.119:0/2250308216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa806e8f0 0x7f0fa8158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.942+0000 7f0facc00640 1 --2- 172.21.15.119:0/2250308216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa806e8f0 0x7f0fa8158300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.942+0000 7f0facc00640 1 -- 172.21.15.119:0/2250308216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa80cb7c0 con 0x7f0fa815c820 2024-04-25T10:11:39.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.942+0000 7f0facc00640 1 --2- 172.21.15.119:0/2250308216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa815c820 0x7f0fa815e240 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0007e60 tx=0x7f0fa000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.942+0000 7f0f9fe00640 1 -- 172.21.15.119:0/2250308216 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0014420 con 0x7f0fa815c820 2024-04-25T10:11:39.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.942+0000 7f0f9fe00640 1 -- 172.21.15.119:0/2250308216 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa0019070 con 0x7f0fa815c820 2024-04-25T10:11:39.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.942+0000 7f0f9fe00640 1 -- 172.21.15.119:0/2250308216 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0014820 con 0x7f0fa815c820 2024-04-25T10:11:39.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.942+0000 7f0faec00640 1 -- 172.21.15.119:0/2250308216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa813f650 con 0x7f0fa815c820 2024-04-25T10:11:39.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.942+0000 7f0faec00640 1 -- 172.21.15.119:0/2250308216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa813fc10 con 0x7f0fa815c820 2024-04-25T10:11:39.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.943+0000 7f0faec00640 1 -- 172.21.15.119:0/2250308216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa80749b0 con 0x7f0fa815c820 2024-04-25T10:11:39.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.945+0000 7f0f9fe00640 1 -- 172.21.15.119:0/2250308216 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fa00149c0 con 0x7f0fa815c820 2024-04-25T10:11:39.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.946+0000 7f0f9fe00640 1 --2- 172.21.15.119:0/2250308216 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f8806b500 0x7f0f8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.946+0000 7f0f9fe00640 1 -- 172.21.15.119:0/2250308216 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fa008aad0 con 0x7f0fa815c820 2024-04-25T10:11:39.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.946+0000 7f0fa7e00640 1 --2- 172.21.15.119:0/2250308216 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f8806b500 0x7f0f8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.947+0000 7f0fa7e00640 1 --2- 172.21.15.119:0/2250308216 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f8806b500 0x7f0f8806d9c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0f980015a0 tx=0x7f0f98009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:39.950+0000 7f0f9fe00640 1 -- 172.21.15.119:0/2250308216 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0fa00598b0 con 0x7f0fa815c820 2024-04-25T10:11:40.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.140+0000 7f0faec00640 1 -- 172.21.15.119:0/2250308216 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0fa8158840 con 0x7f0f8806b500 2024-04-25T10:11:40.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.151+0000 7f0f9fe00640 1 -- 172.21.15.119:0/2250308216 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252000 (secure 0 0 0) 0x7f0fa8158840 con 0x7f0f8806b500 2024-04-25T10:11:40.160 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:40.161 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:11:40.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.157+0000 7f0faec00640 1 -- 172.21.15.119:0/2250308216 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f8806b500 msgr2=0x7f0f8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.157+0000 7f0faec00640 1 --2- 172.21.15.119:0/2250308216 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f8806b500 0x7f0f8806d9c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0f980015a0 tx=0x7f0f98009040 comp rx=0 tx=0).stop 2024-04-25T10:11:40.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.157+0000 7f0faec00640 1 -- 172.21.15.119:0/2250308216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa815c820 msgr2=0x7f0fa815e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.157+0000 7f0faec00640 1 --2- 172.21.15.119:0/2250308216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa815c820 0x7f0fa815e240 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0007e60 tx=0x7f0fa000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.157+0000 7f0faec00640 1 -- 172.21.15.119:0/2250308216 shutdown_connections 2024-04-25T10:11:40.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.157+0000 7f0faec00640 1 --2- 172.21.15.119:0/2250308216 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f8806b500 0x7f0f8806d9c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.157+0000 7f0faec00640 1 --2- 172.21.15.119:0/2250308216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa815c820 0x7f0fa815e240 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.157+0000 7f0faec00640 1 --2- 172.21.15.119:0/2250308216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fa815bf00 0x7f0fa815dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.157+0000 7f0faec00640 1 --2- 172.21.15.119:0/2250308216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa806e8f0 0x7f0fa8158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.157+0000 7f0faec00640 1 -- 172.21.15.119:0/2250308216 >> 172.21.15.119:0/2250308216 conn(0x7f0fa806aba0 msgr2=0x7f0fa8142e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:40.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.158+0000 7f0faec00640 1 -- 172.21.15.119:0/2250308216 shutdown_connections 2024-04-25T10:11:40.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.158+0000 7f0faec00640 1 -- 172.21.15.119:0/2250308216 wait complete. 2024-04-25T10:11:40.182 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-04-25T10:11:40.015265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256932,"kb_used_data":7444,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717788,"statfs":{"total":773094113280,"available":772831014912,"internally_reserved":0,"allocated":7622656,"data_stored":3849778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001646"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.301513+0000","last_change":"2024-04-25T10:11:35.290899+0000","last_active":"2024-04-25T10:11:35.301513+0000","last_peered":"2024-04-25T10:11:35.301513+0000","last_clean":"2024-04-25T10:11:35.301513+0000","last_became_active":"2024-04-25T10:11:35.289495+0000","last_became_peered":"2024-04-25T10:11:35.289495+0000","last_unstale":"2024-04-25T10:11:35.301513+0000","last_undegraded":"2024-04-25T10:11:35.301513+0000","last_fullsized":"2024-04-25T10:11:35.301513+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:47.545189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.312658+0000","last_change":"2024-04-25T10:11:35.308246+0000","last_active":"2024-04-25T10:11:35.312658+0000","last_peered":"2024-04-25T10:11:35.312658+0000","last_clean":"2024-04-25T10:11:35.312658+0000","last_became_active":"2024-04-25T10:11:35.307668+0000","last_became_peered":"2024-04-25T10:11:35.307668+0000","last_unstale":"2024-04-25T10:11:35.312658+0000","last_undegraded":"2024-04-25T10:11:35.312658+0000","last_fullsized":"2024-04-25T10:11:35.312658+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:37.334486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.303945+0000","last_change":"2024-04-25T10:11:35.293708+0000","last_active":"2024-04-25T10:11:35.303945+0000","last_peered":"2024-04-25T10:11:35.303945+0000","last_clean":"2024-04-25T10:11:35.303945+0000","last_became_active":"2024-04-25T10:11:35.293060+0000","last_became_peered":"2024-04-25T10:11:35.293060+0000","last_unstale":"2024-04-25T10:11:35.303945+0000","last_undegraded":"2024-04-25T10:11:35.303945+0000","last_fullsized":"2024-04-25T10:11:35.303945+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:28.610072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.301647+0000","last_change":"2024-04-25T10:11:35.294833+0000","last_active":"2024-04-25T10:11:35.301647+0000","last_peered":"2024-04-25T10:11:35.301647+0000","last_clean":"2024-04-25T10:11:35.301647+0000","last_became_active":"2024-04-25T10:11:35.294483+0000","last_became_peered":"2024-04-25T10:11:35.294483+0000","last_unstale":"2024-04-25T10:11:35.301647+0000","last_undegraded":"2024-04-25T10:11:35.301647+0000","last_fullsized":"2024-04-25T10:11:35.301647+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:09.804191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.306202+0000","last_change":"2024-04-25T10:11:35.295072+0000","last_active":"2024-04-25T10:11:35.306202+0000","last_peered":"2024-04-25T10:11:35.306202+0000","last_clean":"2024-04-25T10:11:35.306202+0000","last_became_active":"2024-04-25T10:11:35.293393+0000","last_became_peered":"2024-04-25T10:11:35.293393+0000","last_unstale":"2024-04-25T10:11:35.306202+0000","last_undegraded":"2024-04-25T10:11:35.306202+0000","last_fullsized":"2024-04-25T10:11:35.306202+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:18.134328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304137+0000","last_change":"2024-04-25T10:11:35.294365+0000","last_active":"2024-04-25T10:11:35.304137+0000","last_peered":"2024-04-25T10:11:35.304137+0000","last_clean":"2024-04-25T10:11:35.304137+0000","last_became_active":"2024-04-25T10:11:35.294058+0000","last_became_peered":"2024-04-25T10:11:35.294058+0000","last_unstale":"2024-04-25T10:11:35.304137+0000","last_undegraded":"2024-04-25T10:11:35.304137+0000","last_fullsized":"2024-04-25T10:11:35.304137+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:41.623334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.306897+0000","last_change":"2024-04-25T10:11:35.299291+0000","last_active":"2024-04-25T10:11:35.306897+0000","last_peered":"2024-04-25T10:11:35.306897+0000","last_clean":"2024-04-25T10:11:35.306897+0000","last_became_active":"2024-04-25T10:11:35.298165+0000","last_became_peered":"2024-04-25T10:11:35.298165+0000","last_unstale":"2024-04-25T10:11:35.306897+0000","last_undegraded":"2024-04-25T10:11:35.306897+0000","last_fullsized":"2024-04-25T10:11:35.306897+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:47.545189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.312567+0000","last_change":"2024-04-25T10:11:35.307511+0000","last_active":"2024-04-25T10:11:35.312567+0000","last_peered":"2024-04-25T10:11:35.312567+0000","last_clean":"2024-04-25T10:11:35.312567+0000","last_became_active":"2024-04-25T10:11:35.305859+0000","last_became_peered":"2024-04-25T10:11:35.305859+0000","last_unstale":"2024-04-25T10:11:35.312567+0000","last_undegraded":"2024-04-25T10:11:35.312567+0000","last_fullsized":"2024-04-25T10:11:35.312567+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:26.509901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.308408+0000","last_change":"2024-04-25T10:11:35.296855+0000","last_active":"2024-04-25T10:11:35.308408+0000","last_peered":"2024-04-25T10:11:35.308408+0000","last_clean":"2024-04-25T10:11:35.308408+0000","last_became_active":"2024-04-25T10:11:35.296529+0000","last_became_peered":"2024-04-25T10:11:35.296529+0000","last_unstale":"2024-04-25T10:11:35.308408+0000","last_undegraded":"2024-04-25T10:11:35.308408+0000","last_fullsized":"2024-04-25T10:11:35.308408+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:05.295729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.292755+0000","last_change":"2024-04-25T10:11:35.286253+0000","last_active":"2024-04-25T10:11:35.292755+0000","last_peered":"2024-04-25T10:11:35.292755+0000","last_clean":"2024-04-25T10:11:35.292755+0000","last_became_active":"2024-04-25T10:11:35.285800+0000","last_became_peered":"2024-04-25T10:11:35.285800+0000","last_unstale":"2024-04-25T10:11:35.292755+0000","last_undegraded":"2024-04-25T10:11:35.292755+0000","last_fullsized":"2024-04-25T10:11:35.292755+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:57.570901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.309054+0000","last_change":"2024-04-25T10:11:35.302288+0000","last_active":"2024-04-25T10:11:35.309054+0000","last_peered":"2024-04-25T10:11:35.309054+0000","last_clean":"2024-04-25T10:11:35.309054+0000","last_became_active":"2024-04-25T10:11:35.301807+0000","last_became_peered":"2024-04-25T10:11:35.301807+0000","last_unstale":"2024-04-25T10:11:35.309054+0000","last_undegraded":"2024-04-25T10:11:35.309054+0000","last_fullsized":"2024-04-25T10:11:35.309054+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:00.087291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304247+0000","last_change":"2024-04-25T10:11:35.299438+0000","last_active":"2024-04-25T10:11:35.304247+0000","last_peered":"2024-04-25T10:11:35.304247+0000","last_clean":"2024-04-25T10:11:35.304247+0000","last_became_active":"2024-04-25T10:11:35.298771+0000","last_became_peered":"2024-04-25T10:11:35.298771+0000","last_unstale":"2024-04-25T10:11:35.304247+0000","last_undegraded":"2024-04-25T10:11:35.304247+0000","last_fullsized":"2024-04-25T10:11:35.304247+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:38.314011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.302934+0000","last_change":"2024-04-25T10:11:35.292222+0000","last_active":"2024-04-25T10:11:35.302934+0000","last_peered":"2024-04-25T10:11:35.302934+0000","last_clean":"2024-04-25T10:11:35.302934+0000","last_became_active":"2024-04-25T10:11:35.291816+0000","last_became_peered":"2024-04-25T10:11:35.291816+0000","last_unstale":"2024-04-25T10:11:35.302934+0000","last_undegraded":"2024-04-25T10:11:35.302934+0000","last_fullsized":"2024-04-25T10:11:35.302934+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:00.087291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304585+0000","last_change":"2024-04-25T10:11:35.291437+0000","last_active":"2024-04-25T10:11:35.304585+0000","last_peered":"2024-04-25T10:11:35.304585+0000","last_clean":"2024-04-25T10:11:35.304585+0000","last_became_active":"2024-04-25T10:11:35.291163+0000","last_became_peered":"2024-04-25T10:11:35.291163+0000","last_unstale":"2024-04-25T10:11:35.304585+0000","last_undegraded":"2024-04-25T10:11:35.304585+0000","last_fullsized":"2024-04-25T10:11:35.304585+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:20.858581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.289732+0000","last_change":"2024-04-25T10:11:35.285958+0000","last_active":"2024-04-25T10:11:35.289732+0000","last_peered":"2024-04-25T10:11:35.289732+0000","last_clean":"2024-04-25T10:11:35.289732+0000","last_became_active":"2024-04-25T10:11:35.285008+0000","last_became_peered":"2024-04-25T10:11:35.285008+0000","last_unstale":"2024-04-25T10:11:35.289732+0000","last_undegraded":"2024-04-25T10:11:35.289732+0000","last_fullsized":"2024-04-25T10:11:35.289732+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:34.745226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304929+0000","last_change":"2024-04-25T10:11:35.295932+0000","last_active":"2024-04-25T10:11:35.304929+0000","last_peered":"2024-04-25T10:11:35.304929+0000","last_clean":"2024-04-25T10:11:35.304929+0000","last_became_active":"2024-04-25T10:11:35.293823+0000","last_became_peered":"2024-04-25T10:11:35.293823+0000","last_unstale":"2024-04-25T10:11:35.304929+0000","last_undegraded":"2024-04-25T10:11:35.304929+0000","last_fullsized":"2024-04-25T10:11:35.304929+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:29.059664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.308432+0000","last_change":"2024-04-25T10:11:35.296917+0000","last_active":"2024-04-25T10:11:35.308432+0000","last_peered":"2024-04-25T10:11:35.308432+0000","last_clean":"2024-04-25T10:11:35.308432+0000","last_became_active":"2024-04-25T10:11:35.296736+0000","last_became_peered":"2024-04-25T10:11:35.296736+0000","last_unstale":"2024-04-25T10:11:35.308432+0000","last_undegraded":"2024-04-25T10:11:35.308432+0000","last_fullsized":"2024-04-25T10:11:35.308432+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:11.384430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.300446+0000","last_change":"2024-04-25T10:11:35.292313+0000","last_active":"2024-04-25T10:11:35.300446+0000","last_peered":"2024-04-25T10:11:35.300446+0000","last_clean":"2024-04-25T10:11:35.300446+0000","last_became_active":"2024-04-25T10:11:35.291955+0000","last_became_peered":"2024-04-25T10:11:35.291955+0000","last_unstale":"2024-04-25T10:11:35.300446+0000","last_undegraded":"2024-04-25T10:11:35.300446+0000","last_fullsized":"2024-04-25T10:11:35.300446+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:20.836377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.312549+0000","last_change":"2024-04-25T10:11:35.305679+0000","last_active":"2024-04-25T10:11:35.312549+0000","last_peered":"2024-04-25T10:11:35.312549+0000","last_clean":"2024-04-25T10:11:35.312549+0000","last_became_active":"2024-04-25T10:11:35.305384+0000","last_became_peered":"2024-04-25T10:11:35.305384+0000","last_unstale":"2024-04-25T10:11:35.312549+0000","last_undegraded":"2024-04-25T10:11:35.312549+0000","last_fullsized":"2024-04-25T10:11:35.312549+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:28.247206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.286844+0000","last_change":"2024-04-25T10:11:35.282460+0000","last_active":"2024-04-25T10:11:35.286844+0000","last_peered":"2024-04-25T10:11:35.286844+0000","last_clean":"2024-04-25T10:11:35.286844+0000","last_became_active":"2024-04-25T10:11:35.281151+0000","last_became_peered":"2024-04-25T10:11:35.281151+0000","last_unstale":"2024-04-25T10:11:35.286844+0000","last_undegraded":"2024-04-25T10:11:35.286844+0000","last_fullsized":"2024-04-25T10:11:35.286844+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:07.791553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.286356+0000","last_change":"2024-04-25T10:11:35.276678+0000","last_active":"2024-04-25T10:11:35.286356+0000","last_peered":"2024-04-25T10:11:35.286356+0000","last_clean":"2024-04-25T10:11:35.286356+0000","last_became_active":"2024-04-25T10:11:35.276030+0000","last_became_peered":"2024-04-25T10:11:35.276030+0000","last_unstale":"2024-04-25T10:11:35.286356+0000","last_undegraded":"2024-04-25T10:11:35.286356+0000","last_fullsized":"2024-04-25T10:11:35.286356+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:33.808111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.290939+0000","last_change":"2024-04-25T10:11:35.285768+0000","last_active":"2024-04-25T10:11:35.290939+0000","last_peered":"2024-04-25T10:11:35.290939+0000","last_clean":"2024-04-25T10:11:35.290939+0000","last_became_active":"2024-04-25T10:11:35.284589+0000","last_became_peered":"2024-04-25T10:11:35.284589+0000","last_unstale":"2024-04-25T10:11:35.290939+0000","last_undegraded":"2024-04-25T10:11:35.290939+0000","last_fullsized":"2024-04-25T10:11:35.290939+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:37.334486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.296574+0000","last_change":"2024-04-25T10:11:35.290247+0000","last_active":"2024-04-25T10:11:35.296574+0000","last_peered":"2024-04-25T10:11:35.296574+0000","last_clean":"2024-04-25T10:11:35.296574+0000","last_became_active":"2024-04-25T10:11:35.289914+0000","last_became_peered":"2024-04-25T10:11:35.289914+0000","last_unstale":"2024-04-25T10:11:35.296574+0000","last_undegraded":"2024-04-25T10:11:35.296574+0000","last_fullsized":"2024-04-25T10:11:35.296574+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:46.520578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304130+0000","last_change":"2024-04-25T10:11:35.298244+0000","last_active":"2024-04-25T10:11:35.304130+0000","last_peered":"2024-04-25T10:11:35.304130+0000","last_clean":"2024-04-25T10:11:35.304130+0000","last_became_active":"2024-04-25T10:11:35.297969+0000","last_became_peered":"2024-04-25T10:11:35.297969+0000","last_unstale":"2024-04-25T10:11:35.304130+0000","last_undegraded":"2024-04-25T10:11:35.304130+0000","last_fullsized":"2024-04-25T10:11:35.304130+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:32.327826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.303838+0000","last_change":"2024-04-25T10:11:35.296178+0000","last_active":"2024-04-25T10:11:35.303838+0000","last_peered":"2024-04-25T10:11:35.303838+0000","last_clean":"2024-04-25T10:11:35.303838+0000","last_became_active":"2024-04-25T10:11:35.295877+0000","last_became_peered":"2024-04-25T10:11:35.295877+0000","last_unstale":"2024-04-25T10:11:35.303838+0000","last_undegraded":"2024-04-25T10:11:35.303838+0000","last_fullsized":"2024-04-25T10:11:35.303838+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:19.216901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.298778+0000","last_change":"2024-04-25T10:11:35.295743+0000","last_active":"2024-04-25T10:11:35.298778+0000","last_peered":"2024-04-25T10:11:35.298778+0000","last_clean":"2024-04-25T10:11:35.298778+0000","last_became_active":"2024-04-25T10:11:35.295201+0000","last_became_peered":"2024-04-25T10:11:35.295201+0000","last_unstale":"2024-04-25T10:11:35.298778+0000","last_undegraded":"2024-04-25T10:11:35.298778+0000","last_fullsized":"2024-04-25T10:11:35.298778+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:46.520578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304096+0000","last_change":"2024-04-25T10:11:35.295755+0000","last_active":"2024-04-25T10:11:35.304096+0000","last_peered":"2024-04-25T10:11:35.304096+0000","last_clean":"2024-04-25T10:11:35.304096+0000","last_became_active":"2024-04-25T10:11:35.292401+0000","last_became_peered":"2024-04-25T10:11:35.292401+0000","last_unstale":"2024-04-25T10:11:35.304096+0000","last_undegraded":"2024-04-25T10:11:35.304096+0000","last_fullsized":"2024-04-25T10:11:35.304096+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:35.319360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.297320+0000","last_change":"2024-04-25T10:11:35.288725+0000","last_active":"2024-04-25T10:11:35.297320+0000","last_peered":"2024-04-25T10:11:35.297320+0000","last_clean":"2024-04-25T10:11:35.297320+0000","last_became_active":"2024-04-25T10:11:35.288408+0000","last_became_peered":"2024-04-25T10:11:35.288408+0000","last_unstale":"2024-04-25T10:11:35.297320+0000","last_undegraded":"2024-04-25T10:11:35.297320+0000","last_fullsized":"2024-04-25T10:11:35.297320+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:10.957813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.298360+0000","last_change":"2024-04-25T10:11:35.289236+0000","last_active":"2024-04-25T10:11:35.298360+0000","last_peered":"2024-04-25T10:11:35.298360+0000","last_clean":"2024-04-25T10:11:35.298360+0000","last_became_active":"2024-04-25T10:11:35.288910+0000","last_became_peered":"2024-04-25T10:11:35.288910+0000","last_unstale":"2024-04-25T10:11:35.298360+0000","last_undegraded":"2024-04-25T10:11:35.298360+0000","last_fullsized":"2024-04-25T10:11:35.298360+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:11.983167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.301062+0000","last_change":"2024-04-25T10:11:35.295458+0000","last_active":"2024-04-25T10:11:35.301062+0000","last_peered":"2024-04-25T10:11:35.301062+0000","last_clean":"2024-04-25T10:11:35.301062+0000","last_became_active":"2024-04-25T10:11:35.294990+0000","last_became_peered":"2024-04-25T10:11:35.294990+0000","last_unstale":"2024-04-25T10:11:35.301062+0000","last_undegraded":"2024-04-25T10:11:35.301062+0000","last_fullsized":"2024-04-25T10:11:35.301062+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:41.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.294575+0000","last_change":"2024-04-25T10:11:35.284590+0000","last_active":"2024-04-25T10:11:35.294575+0000","last_peered":"2024-04-25T10:11:35.294575+0000","last_clean":"2024-04-25T10:11:35.294575+0000","last_became_active":"2024-04-25T10:11:35.284305+0000","last_became_peered":"2024-04-25T10:11:35.284305+0000","last_unstale":"2024-04-25T10:11:35.294575+0000","last_undegraded":"2024-04-25T10:11:35.294575+0000","last_fullsized":"2024-04-25T10:11:35.294575+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:19.216901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.312134+0000","last_change":"2024-04-25T10:11:35.305947+0000","last_active":"2024-04-25T10:11:35.312134+0000","last_peered":"2024-04-25T10:11:35.312134+0000","last_clean":"2024-04-25T10:11:35.312134+0000","last_became_active":"2024-04-25T10:11:35.305374+0000","last_became_peered":"2024-04-25T10:11:35.305374+0000","last_unstale":"2024-04-25T10:11:35.312134+0000","last_undegraded":"2024-04-25T10:11:35.312134+0000","last_fullsized":"2024-04-25T10:11:35.312134+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:49.908492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.307989+0000","last_change":"2024-04-25T10:11:35.299624+0000","last_active":"2024-04-25T10:11:35.307989+0000","last_peered":"2024-04-25T10:11:35.307989+0000","last_clean":"2024-04-25T10:11:35.307989+0000","last_became_active":"2024-04-25T10:11:35.299324+0000","last_became_peered":"2024-04-25T10:11:35.299324+0000","last_unstale":"2024-04-25T10:11:35.307989+0000","last_undegraded":"2024-04-25T10:11:35.307989+0000","last_fullsized":"2024-04-25T10:11:35.307989+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:29.604949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"66'4","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.313680+0000","last_change":"2024-04-25T10:11:34.286746+0000","last_active":"2024-04-25T10:11:35.313680+0000","last_peered":"2024-04-25T10:11:35.313680+0000","last_clean":"2024-04-25T10:11:35.313680+0000","last_became_active":"2024-04-25T10:11:34.286428+0000","last_became_peered":"2024-04-25T10:11:34.286428+0000","last_unstale":"2024-04-25T10:11:35.313680+0000","last_undegraded":"2024-04-25T10:11:35.313680+0000","last_fullsized":"2024-04-25T10:11:35.313680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:46.688289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304182+0000","last_change":"2024-04-25T10:11:35.288648+0000","last_active":"2024-04-25T10:11:35.304182+0000","last_peered":"2024-04-25T10:11:35.304182+0000","last_clean":"2024-04-25T10:11:35.304182+0000","last_became_active":"2024-04-25T10:11:35.288365+0000","last_became_peered":"2024-04-25T10:11:35.288365+0000","last_unstale":"2024-04-25T10:11:35.304182+0000","last_undegraded":"2024-04-25T10:11:35.304182+0000","last_fullsized":"2024-04-25T10:11:35.304182+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:28.219428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.310724+0000","last_change":"2024-04-25T10:11:34.278951+0000","last_active":"2024-04-25T10:11:35.310724+0000","last_peered":"2024-04-25T10:11:35.310724+0000","last_clean":"2024-04-25T10:11:35.310724+0000","last_became_active":"2024-04-25T10:11:34.273919+0000","last_became_peered":"2024-04-25T10:11:34.273919+0000","last_unstale":"2024-04-25T10:11:35.310724+0000","last_undegraded":"2024-04-25T10:11:35.310724+0000","last_fullsized":"2024-04-25T10:11:35.310724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:46.529810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.297106+0000","last_change":"2024-04-25T10:11:35.293045+0000","last_active":"2024-04-25T10:11:35.297106+0000","last_peered":"2024-04-25T10:11:35.297106+0000","last_clean":"2024-04-25T10:11:35.297106+0000","last_became_active":"2024-04-25T10:11:35.292682+0000","last_became_peered":"2024-04-25T10:11:35.292682+0000","last_unstale":"2024-04-25T10:11:35.297106+0000","last_undegraded":"2024-04-25T10:11:35.297106+0000","last_fullsized":"2024-04-25T10:11:35.297106+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:02.236799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.308693+0000","last_change":"2024-04-25T10:11:34.284716+0000","last_active":"2024-04-25T10:11:35.308693+0000","last_peered":"2024-04-25T10:11:35.308693+0000","last_clean":"2024-04-25T10:11:35.308693+0000","last_became_active":"2024-04-25T10:11:34.284323+0000","last_became_peered":"2024-04-25T10:11:34.284323+0000","last_unstale":"2024-04-25T10:11:35.308693+0000","last_undegraded":"2024-04-25T10:11:35.308693+0000","last_fullsized":"2024-04-25T10:11:35.308693+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:48.893113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.294477+0000","last_change":"2024-04-25T10:11:35.289177+0000","last_active":"2024-04-25T10:11:35.294477+0000","last_peered":"2024-04-25T10:11:35.294477+0000","last_clean":"2024-04-25T10:11:35.294477+0000","last_became_active":"2024-04-25T10:11:35.288865+0000","last_became_peered":"2024-04-25T10:11:35.288865+0000","last_unstale":"2024-04-25T10:11:35.294477+0000","last_undegraded":"2024-04-25T10:11:35.294477+0000","last_fullsized":"2024-04-25T10:11:35.294477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:43.629406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.311323+0000","last_change":"2024-04-25T10:11:34.290037+0000","last_active":"2024-04-25T10:11:35.311323+0000","last_peered":"2024-04-25T10:11:35.311323+0000","last_clean":"2024-04-25T10:11:35.311323+0000","last_became_active":"2024-04-25T10:11:34.288487+0000","last_became_peered":"2024-04-25T10:11:34.288487+0000","last_unstale":"2024-04-25T10:11:35.311323+0000","last_undegraded":"2024-04-25T10:11:35.311323+0000","last_fullsized":"2024-04-25T10:11:35.311323+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:35.981626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.297862+0000","last_change":"2024-04-25T10:11:35.289434+0000","last_active":"2024-04-25T10:11:35.297862+0000","last_peered":"2024-04-25T10:11:35.297862+0000","last_clean":"2024-04-25T10:11:35.297862+0000","last_became_active":"2024-04-25T10:11:35.289151+0000","last_became_peered":"2024-04-25T10:11:35.289151+0000","last_unstale":"2024-04-25T10:11:35.297862+0000","last_undegraded":"2024-04-25T10:11:35.297862+0000","last_fullsized":"2024-04-25T10:11:35.297862+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:43.629406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.303062+0000","last_change":"2024-04-25T10:11:34.279711+0000","last_active":"2024-04-25T10:11:35.303062+0000","last_peered":"2024-04-25T10:11:35.303062+0000","last_clean":"2024-04-25T10:11:35.303062+0000","last_became_active":"2024-04-25T10:11:34.279394+0000","last_became_peered":"2024-04-25T10:11:34.279394+0000","last_unstale":"2024-04-25T10:11:35.303062+0000","last_undegraded":"2024-04-25T10:11:35.303062+0000","last_fullsized":"2024-04-25T10:11:35.303062+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:23.027335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.302088+0000","last_change":"2024-04-25T10:11:35.295035+0000","last_active":"2024-04-25T10:11:35.302088+0000","last_peered":"2024-04-25T10:11:35.302088+0000","last_clean":"2024-04-25T10:11:35.302088+0000","last_became_active":"2024-04-25T10:11:35.294737+0000","last_became_peered":"2024-04-25T10:11:35.294737+0000","last_unstale":"2024-04-25T10:11:35.302088+0000","last_undegraded":"2024-04-25T10:11:35.302088+0000","last_fullsized":"2024-04-25T10:11:35.302088+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:29.474273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.308156+0000","last_change":"2024-04-25T10:11:34.290766+0000","last_active":"2024-04-25T10:11:35.308156+0000","last_peered":"2024-04-25T10:11:35.308156+0000","last_clean":"2024-04-25T10:11:35.308156+0000","last_became_active":"2024-04-25T10:11:34.289236+0000","last_became_peered":"2024-04-25T10:11:34.289236+0000","last_unstale":"2024-04-25T10:11:35.308156+0000","last_undegraded":"2024-04-25T10:11:35.308156+0000","last_fullsized":"2024-04-25T10:11:35.308156+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:15.279109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.306233+0000","last_change":"2024-04-25T10:11:35.293321+0000","last_active":"2024-04-25T10:11:35.306233+0000","last_peered":"2024-04-25T10:11:35.306233+0000","last_clean":"2024-04-25T10:11:35.306233+0000","last_became_active":"2024-04-25T10:11:35.293032+0000","last_became_peered":"2024-04-25T10:11:35.293032+0000","last_unstale":"2024-04-25T10:11:35.306233+0000","last_undegraded":"2024-04-25T10:11:35.306233+0000","last_fullsized":"2024-04-25T10:11:35.306233+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:28.445879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.282415+0000","last_change":"2024-04-25T10:11:34.282281+0000","last_active":"2024-04-25T10:11:35.282415+0000","last_peered":"2024-04-25T10:11:35.282415+0000","last_clean":"2024-04-25T10:11:35.282415+0000","last_became_active":"2024-04-25T10:11:34.281983+0000","last_became_peered":"2024-04-25T10:11:34.281983+0000","last_unstale":"2024-04-25T10:11:35.282415+0000","last_undegraded":"2024-04-25T10:11:35.282415+0000","last_fullsized":"2024-04-25T10:11:35.282415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:03.701671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.301464+0000","last_change":"2024-04-25T10:11:35.297163+0000","last_active":"2024-04-25T10:11:35.301464+0000","last_peered":"2024-04-25T10:11:35.301464+0000","last_clean":"2024-04-25T10:11:35.301464+0000","last_became_active":"2024-04-25T10:11:35.294350+0000","last_became_peered":"2024-04-25T10:11:35.294350+0000","last_unstale":"2024-04-25T10:11:35.301464+0000","last_undegraded":"2024-04-25T10:11:35.301464+0000","last_fullsized":"2024-04-25T10:11:35.301464+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:21.346895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.279025+0000","last_change":"2024-04-25T10:11:34.281232+0000","last_active":"2024-04-25T10:11:35.279025+0000","last_peered":"2024-04-25T10:11:35.279025+0000","last_clean":"2024-04-25T10:11:35.279025+0000","last_became_active":"2024-04-25T10:11:34.280973+0000","last_became_peered":"2024-04-25T10:11:34.280973+0000","last_unstale":"2024-04-25T10:11:35.279025+0000","last_undegraded":"2024-04-25T10:11:35.279025+0000","last_fullsized":"2024-04-25T10:11:35.279025+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:49.379938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.298638+0000","last_change":"2024-04-25T10:11:35.291294+0000","last_active":"2024-04-25T10:11:35.298638+0000","last_peered":"2024-04-25T10:11:35.298638+0000","last_clean":"2024-04-25T10:11:35.298638+0000","last_became_active":"2024-04-25T10:11:35.290996+0000","last_became_peered":"2024-04-25T10:11:35.290996+0000","last_unstale":"2024-04-25T10:11:35.298638+0000","last_undegraded":"2024-04-25T10:11:35.298638+0000","last_fullsized":"2024-04-25T10:11:35.298638+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:26.632804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.269753+0000","last_change":"2024-04-25T10:11:34.276292+0000","last_active":"2024-04-25T10:11:35.269753+0000","last_peered":"2024-04-25T10:11:35.269753+0000","last_clean":"2024-04-25T10:11:35.269753+0000","last_became_active":"2024-04-25T10:11:34.275999+0000","last_became_peered":"2024-04-25T10:11:34.275999+0000","last_unstale":"2024-04-25T10:11:35.269753+0000","last_undegraded":"2024-04-25T10:11:35.269753+0000","last_fullsized":"2024-04-25T10:11:35.269753+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:25.494522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.309028+0000","last_change":"2024-04-25T10:11:35.301363+0000","last_active":"2024-04-25T10:11:35.309028+0000","last_peered":"2024-04-25T10:11:35.309028+0000","last_clean":"2024-04-25T10:11:35.309028+0000","last_became_active":"2024-04-25T10:11:35.300243+0000","last_became_peered":"2024-04-25T10:11:35.300243+0000","last_unstale":"2024-04-25T10:11:35.309028+0000","last_undegraded":"2024-04-25T10:11:35.309028+0000","last_fullsized":"2024-04-25T10:11:35.309028+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.995189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.288226+0000","last_change":"2024-04-25T10:11:34.289593+0000","last_active":"2024-04-25T10:11:35.288226+0000","last_peered":"2024-04-25T10:11:35.288226+0000","last_clean":"2024-04-25T10:11:35.288226+0000","last_became_active":"2024-04-25T10:11:34.288958+0000","last_became_peered":"2024-04-25T10:11:34.288958+0000","last_unstale":"2024-04-25T10:11:35.288226+0000","last_undegraded":"2024-04-25T10:11:35.288226+0000","last_fullsized":"2024-04-25T10:11:35.288226+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:03.701671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.312632+0000","last_change":"2024-04-25T10:11:35.307869+0000","last_active":"2024-04-25T10:11:35.312632+0000","last_peered":"2024-04-25T10:11:35.312632+0000","last_clean":"2024-04-25T10:11:35.312632+0000","last_became_active":"2024-04-25T10:11:35.307527+0000","last_became_peered":"2024-04-25T10:11:35.307527+0000","last_unstale":"2024-04-25T10:11:35.312632+0000","last_undegraded":"2024-04-25T10:11:35.312632+0000","last_fullsized":"2024-04-25T10:11:35.312632+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:33.808111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"66'3","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.307484+0000","last_change":"2024-04-25T10:11:34.282859+0000","last_active":"2024-04-25T10:11:35.307484+0000","last_peered":"2024-04-25T10:11:35.307484+0000","last_clean":"2024-04-25T10:11:35.307484+0000","last_became_active":"2024-04-25T10:11:34.282355+0000","last_became_peered":"2024-04-25T10:11:34.282355+0000","last_unstale":"2024-04-25T10:11:35.307484+0000","last_undegraded":"2024-04-25T10:11:35.307484+0000","last_fullsized":"2024-04-25T10:11:35.307484+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:24.494895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.308587+0000","last_change":"2024-04-25T10:11:35.301068+0000","last_active":"2024-04-25T10:11:35.308587+0000","last_peered":"2024-04-25T10:11:35.308587+0000","last_clean":"2024-04-25T10:11:35.308587+0000","last_became_active":"2024-04-25T10:11:35.300223+0000","last_became_peered":"2024-04-25T10:11:35.300223+0000","last_unstale":"2024-04-25T10:11:35.308587+0000","last_undegraded":"2024-04-25T10:11:35.308587+0000","last_fullsized":"2024-04-25T10:11:35.308587+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:43.286662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"66'6","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.315799+0000","last_change":"2024-04-25T10:11:34.285777+0000","last_active":"2024-04-25T10:11:35.315799+0000","last_peered":"2024-04-25T10:11:35.315799+0000","last_clean":"2024-04-25T10:11:35.315799+0000","last_became_active":"2024-04-25T10:11:34.285425+0000","last_became_peered":"2024-04-25T10:11:34.285425+0000","last_unstale":"2024-04-25T10:11:35.315799+0000","last_undegraded":"2024-04-25T10:11:35.315799+0000","last_fullsized":"2024-04-25T10:11:35.315799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:07.422965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.294540+0000","last_change":"2024-04-25T10:11:35.289614+0000","last_active":"2024-04-25T10:11:35.294540+0000","last_peered":"2024-04-25T10:11:35.294540+0000","last_clean":"2024-04-25T10:11:35.294540+0000","last_became_active":"2024-04-25T10:11:35.289234+0000","last_became_peered":"2024-04-25T10:11:35.289234+0000","last_unstale":"2024-04-25T10:11:35.294540+0000","last_undegraded":"2024-04-25T10:11:35.294540+0000","last_fullsized":"2024-04-25T10:11:35.294540+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:26.632804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.309651+0000","last_change":"2024-04-25T10:11:34.285343+0000","last_active":"2024-04-25T10:11:35.309651+0000","last_peered":"2024-04-25T10:11:35.309651+0000","last_clean":"2024-04-25T10:11:35.309651+0000","last_became_active":"2024-04-25T10:11:34.284947+0000","last_became_peered":"2024-04-25T10:11:34.284947+0000","last_unstale":"2024-04-25T10:11:35.309651+0000","last_undegraded":"2024-04-25T10:11:35.309651+0000","last_fullsized":"2024-04-25T10:11:35.309651+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:15.279109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.292650+0000","last_change":"2024-04-25T10:11:35.286959+0000","last_active":"2024-04-25T10:11:35.292650+0000","last_peered":"2024-04-25T10:11:35.292650+0000","last_clean":"2024-04-25T10:11:35.292650+0000","last_became_active":"2024-04-25T10:11:35.286141+0000","last_became_peered":"2024-04-25T10:11:35.286141+0000","last_unstale":"2024-04-25T10:11:35.292650+0000","last_undegraded":"2024-04-25T10:11:35.292650+0000","last_fullsized":"2024-04-25T10:11:35.292650+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:47.703668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.284776+0000","last_change":"2024-04-25T10:11:34.285246+0000","last_active":"2024-04-25T10:11:35.284776+0000","last_peered":"2024-04-25T10:11:35.284776+0000","last_clean":"2024-04-25T10:11:35.284776+0000","last_became_active":"2024-04-25T10:11:34.284969+0000","last_became_peered":"2024-04-25T10:11:34.284969+0000","last_unstale":"2024-04-25T10:11:35.284776+0000","last_undegraded":"2024-04-25T10:11:35.284776+0000","last_fullsized":"2024-04-25T10:11:35.284776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:35.636729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.298682+0000","last_change":"2024-04-25T10:11:35.290310+0000","last_active":"2024-04-25T10:11:35.298682+0000","last_peered":"2024-04-25T10:11:35.298682+0000","last_clean":"2024-04-25T10:11:35.298682+0000","last_became_active":"2024-04-25T10:11:35.289919+0000","last_became_peered":"2024-04-25T10:11:35.289919+0000","last_unstale":"2024-04-25T10:11:35.298682+0000","last_undegraded":"2024-04-25T10:11:35.298682+0000","last_fullsized":"2024-04-25T10:11:35.298682+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:41.995189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.265648+0000","last_change":"2024-04-25T10:11:34.268634+0000","last_active":"2024-04-25T10:11:35.265648+0000","last_peered":"2024-04-25T10:11:35.265648+0000","last_clean":"2024-04-25T10:11:35.265648+0000","last_became_active":"2024-04-25T10:11:34.267992+0000","last_became_peered":"2024-04-25T10:11:34.267992+0000","last_unstale":"2024-04-25T10:11:35.265648+0000","last_undegraded":"2024-04-25T10:11:35.265648+0000","last_fullsized":"2024-04-25T10:11:35.265648+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:45.371081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304291+0000","last_change":"2024-04-25T10:11:35.290684+0000","last_active":"2024-04-25T10:11:35.304291+0000","last_peered":"2024-04-25T10:11:35.304291+0000","last_clean":"2024-04-25T10:11:35.304291+0000","last_became_active":"2024-04-25T10:11:35.290355+0000","last_became_peered":"2024-04-25T10:11:35.290355+0000","last_unstale":"2024-04-25T10:11:35.304291+0000","last_undegraded":"2024-04-25T10:11:35.304291+0000","last_fullsized":"2024-04-25T10:11:35.304291+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:21.906422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"66'8","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.312885+0000","last_change":"2024-04-25T10:11:34.284924+0000","last_active":"2024-04-25T10:11:35.312885+0000","last_peered":"2024-04-25T10:11:35.312885+0000","last_clean":"2024-04-25T10:11:35.312885+0000","last_became_active":"2024-04-25T10:11:34.284529+0000","last_became_peered":"2024-04-25T10:11:34.284529+0000","last_unstale":"2024-04-25T10:11:35.312885+0000","last_undegraded":"2024-04-25T10:11:35.312885+0000","last_fullsized":"2024-04-25T10:11:35.312885+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:40.607955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.302002+0000","last_change":"2024-04-25T10:11:35.292886+0000","last_active":"2024-04-25T10:11:35.302002+0000","last_peered":"2024-04-25T10:11:35.302002+0000","last_clean":"2024-04-25T10:11:35.302002+0000","last_became_active":"2024-04-25T10:11:35.292559+0000","last_became_peered":"2024-04-25T10:11:35.292559+0000","last_unstale":"2024-04-25T10:11:35.302002+0000","last_undegraded":"2024-04-25T10:11:35.302002+0000","last_fullsized":"2024-04-25T10:11:35.302002+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:46.386460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.311927+0000","last_change":"2024-04-25T10:11:34.288990+0000","last_active":"2024-04-25T10:11:35.311927+0000","last_peered":"2024-04-25T10:11:35.311927+0000","last_clean":"2024-04-25T10:11:35.311927+0000","last_became_active":"2024-04-25T10:11:34.286484+0000","last_became_peered":"2024-04-25T10:11:34.286484+0000","last_unstale":"2024-04-25T10:11:35.311927+0000","last_undegraded":"2024-04-25T10:11:35.311927+0000","last_fullsized":"2024-04-25T10:11:35.311927+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:08.788812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.309002+0000","last_change":"2024-04-25T10:11:35.299962+0000","last_active":"2024-04-25T10:11:35.309002+0000","last_peered":"2024-04-25T10:11:35.309002+0000","last_clean":"2024-04-25T10:11:35.309002+0000","last_became_active":"2024-04-25T10:11:35.299597+0000","last_became_peered":"2024-04-25T10:11:35.299597+0000","last_unstale":"2024-04-25T10:11:35.309002+0000","last_undegraded":"2024-04-25T10:11:35.309002+0000","last_fullsized":"2024-04-25T10:11:35.309002+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:34.745226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.308883+0000","last_change":"2024-04-25T10:11:34.284619+0000","last_active":"2024-04-25T10:11:35.308883+0000","last_peered":"2024-04-25T10:11:35.308883+0000","last_clean":"2024-04-25T10:11:35.308883+0000","last_became_active":"2024-04-25T10:11:34.284300+0000","last_became_peered":"2024-04-25T10:11:34.284300+0000","last_unstale":"2024-04-25T10:11:35.308883+0000","last_undegraded":"2024-04-25T10:11:35.308883+0000","last_fullsized":"2024-04-25T10:11:35.308883+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:27.231827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304223+0000","last_change":"2024-04-25T10:11:35.299182+0000","last_active":"2024-04-25T10:11:35.304223+0000","last_peered":"2024-04-25T10:11:35.304223+0000","last_clean":"2024-04-25T10:11:35.304223+0000","last_became_active":"2024-04-25T10:11:35.297616+0000","last_became_peered":"2024-04-25T10:11:35.297616+0000","last_unstale":"2024-04-25T10:11:35.304223+0000","last_undegraded":"2024-04-25T10:11:35.304223+0000","last_fullsized":"2024-04-25T10:11:35.304223+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:36.997005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.307611+0000","last_change":"2024-04-25T10:11:34.290024+0000","last_active":"2024-04-25T10:11:35.307611+0000","last_peered":"2024-04-25T10:11:35.307611+0000","last_clean":"2024-04-25T10:11:35.307611+0000","last_became_active":"2024-04-25T10:11:34.288707+0000","last_became_peered":"2024-04-25T10:11:34.288707+0000","last_unstale":"2024-04-25T10:11:35.307611+0000","last_undegraded":"2024-04-25T10:11:35.307611+0000","last_fullsized":"2024-04-25T10:11:35.307611+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:19.820998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.288112+0000","last_change":"2024-04-25T10:11:35.284777+0000","last_active":"2024-04-25T10:11:35.288112+0000","last_peered":"2024-04-25T10:11:35.288112+0000","last_clean":"2024-04-25T10:11:35.288112+0000","last_became_active":"2024-04-25T10:11:35.284516+0000","last_became_peered":"2024-04-25T10:11:35.284516+0000","last_unstale":"2024-04-25T10:11:35.288112+0000","last_undegraded":"2024-04-25T10:11:35.288112+0000","last_fullsized":"2024-04-25T10:11:35.288112+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:38.861057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.277842+0000","last_change":"2024-04-25T10:11:34.289744+0000","last_active":"2024-04-25T10:11:35.277842+0000","last_peered":"2024-04-25T10:11:35.277842+0000","last_clean":"2024-04-25T10:11:35.277842+0000","last_became_active":"2024-04-25T10:11:34.289462+0000","last_became_peered":"2024-04-25T10:11:34.289462+0000","last_unstale":"2024-04-25T10:11:35.277842+0000","last_undegraded":"2024-04-25T10:11:35.277842+0000","last_fullsized":"2024-04-25T10:11:35.277842+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:18.201522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.305154+0000","last_change":"2024-04-25T10:11:35.297567+0000","last_active":"2024-04-25T10:11:35.305154+0000","last_peered":"2024-04-25T10:11:35.305154+0000","last_clean":"2024-04-25T10:11:35.305154+0000","last_became_active":"2024-04-25T10:11:35.297242+0000","last_became_peered":"2024-04-25T10:11:35.297242+0000","last_unstale":"2024-04-25T10:11:35.305154+0000","last_undegraded":"2024-04-25T10:11:35.305154+0000","last_fullsized":"2024-04-25T10:11:35.305154+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:30.558925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.287290+0000","last_change":"2024-04-25T10:11:34.289995+0000","last_active":"2024-04-25T10:11:35.287290+0000","last_peered":"2024-04-25T10:11:35.287290+0000","last_clean":"2024-04-25T10:11:35.287290+0000","last_became_active":"2024-04-25T10:11:34.289649+0000","last_became_peered":"2024-04-25T10:11:34.289649+0000","last_unstale":"2024-04-25T10:11:35.287290+0000","last_undegraded":"2024-04-25T10:11:35.287290+0000","last_fullsized":"2024-04-25T10:11:35.287290+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:07.422965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.298510+0000","last_change":"2024-04-25T10:11:35.290526+0000","last_active":"2024-04-25T10:11:35.298510+0000","last_peered":"2024-04-25T10:11:35.298510+0000","last_clean":"2024-04-25T10:11:35.298510+0000","last_became_active":"2024-04-25T10:11:35.290149+0000","last_became_peered":"2024-04-25T10:11:35.290149+0000","last_unstale":"2024-04-25T10:11:35.298510+0000","last_undegraded":"2024-04-25T10:11:35.298510+0000","last_fullsized":"2024-04-25T10:11:35.298510+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:23.616117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.267125+0000","last_change":"2024-04-25T10:11:34.278538+0000","last_active":"2024-04-25T10:11:35.267125+0000","last_peered":"2024-04-25T10:11:35.267125+0000","last_clean":"2024-04-25T10:11:35.267125+0000","last_became_active":"2024-04-25T10:11:34.277907+0000","last_became_peered":"2024-04-25T10:11:34.277907+0000","last_unstale":"2024-04-25T10:11:35.267125+0000","last_undegraded":"2024-04-25T10:11:35.267125+0000","last_fullsized":"2024-04-25T10:11:35.267125+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:28.458894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304448+0000","last_change":"2024-04-25T10:11:35.296618+0000","last_active":"2024-04-25T10:11:35.304448+0000","last_peered":"2024-04-25T10:11:35.304448+0000","last_clean":"2024-04-25T10:11:35.304448+0000","last_became_active":"2024-04-25T10:11:35.296322+0000","last_became_peered":"2024-04-25T10:11:35.296322+0000","last_unstale":"2024-04-25T10:11:35.304448+0000","last_undegraded":"2024-04-25T10:11:35.304448+0000","last_fullsized":"2024-04-25T10:11:35.304448+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:00.339202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.259134+0000","last_change":"2024-04-25T10:11:34.277383+0000","last_active":"2024-04-25T10:11:35.259134+0000","last_peered":"2024-04-25T10:11:35.259134+0000","last_clean":"2024-04-25T10:11:35.259134+0000","last_became_active":"2024-04-25T10:11:34.277085+0000","last_became_peered":"2024-04-25T10:11:34.277085+0000","last_unstale":"2024-04-25T10:11:35.259134+0000","last_undegraded":"2024-04-25T10:11:35.259134+0000","last_fullsized":"2024-04-25T10:11:35.259134+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:45.505199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.289515+0000","last_change":"2024-04-25T10:11:35.285357+0000","last_active":"2024-04-25T10:11:35.289515+0000","last_peered":"2024-04-25T10:11:35.289515+0000","last_clean":"2024-04-25T10:11:35.289515+0000","last_became_active":"2024-04-25T10:11:35.285018+0000","last_became_peered":"2024-04-25T10:11:35.285018+0000","last_unstale":"2024-04-25T10:11:35.289515+0000","last_undegraded":"2024-04-25T10:11:35.289515+0000","last_fullsized":"2024-04-25T10:11:35.289515+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:29.604949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.309887+0000","last_change":"2024-04-25T10:11:34.289251+0000","last_active":"2024-04-25T10:11:35.309887+0000","last_peered":"2024-04-25T10:11:35.309887+0000","last_clean":"2024-04-25T10:11:35.309887+0000","last_became_active":"2024-04-25T10:11:34.288582+0000","last_became_peered":"2024-04-25T10:11:34.288582+0000","last_unstale":"2024-04-25T10:11:35.309887+0000","last_undegraded":"2024-04-25T10:11:35.309887+0000","last_fullsized":"2024-04-25T10:11:35.309887+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:13.498898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.306871+0000","last_change":"2024-04-25T10:11:35.300735+0000","last_active":"2024-04-25T10:11:35.306871+0000","last_peered":"2024-04-25T10:11:35.306871+0000","last_clean":"2024-04-25T10:11:35.306871+0000","last_became_active":"2024-04-25T10:11:35.297691+0000","last_became_peered":"2024-04-25T10:11:35.297691+0000","last_unstale":"2024-04-25T10:11:35.306871+0000","last_undegraded":"2024-04-25T10:11:35.306871+0000","last_fullsized":"2024-04-25T10:11:35.306871+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:32.327826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.271734+0000","last_change":"2024-04-25T10:11:34.270969+0000","last_active":"2024-04-25T10:11:35.271734+0000","last_peered":"2024-04-25T10:11:35.271734+0000","last_clean":"2024-04-25T10:11:35.271734+0000","last_became_active":"2024-04-25T10:11:34.269806+0000","last_became_peered":"2024-04-25T10:11:34.269806+0000","last_unstale":"2024-04-25T10:11:35.271734+0000","last_undegraded":"2024-04-25T10:11:35.271734+0000","last_fullsized":"2024-04-25T10:11:35.271734+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:28.044285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.307742+0000","last_change":"2024-04-25T10:11:35.299259+0000","last_active":"2024-04-25T10:11:35.307742+0000","last_peered":"2024-04-25T10:11:35.307742+0000","last_clean":"2024-04-25T10:11:35.307742+0000","last_became_active":"2024-04-25T10:11:35.298960+0000","last_became_peered":"2024-04-25T10:11:35.298960+0000","last_unstale":"2024-04-25T10:11:35.307742+0000","last_undegraded":"2024-04-25T10:11:35.307742+0000","last_fullsized":"2024-04-25T10:11:35.307742+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:07.791553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.279191+0000","last_change":"2024-04-25T10:11:34.275045+0000","last_active":"2024-04-25T10:11:35.279191+0000","last_peered":"2024-04-25T10:11:35.279191+0000","last_clean":"2024-04-25T10:11:35.279191+0000","last_became_active":"2024-04-25T10:11:34.274675+0000","last_became_peered":"2024-04-25T10:11:34.274675+0000","last_unstale":"2024-04-25T10:11:35.279191+0000","last_undegraded":"2024-04-25T10:11:35.279191+0000","last_fullsized":"2024-04-25T10:11:35.279191+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:01.729149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.301861+0000","last_change":"2024-04-25T10:11:35.289935+0000","last_active":"2024-04-25T10:11:35.301861+0000","last_peered":"2024-04-25T10:11:35.301861+0000","last_clean":"2024-04-25T10:11:35.301861+0000","last_became_active":"2024-04-25T10:11:35.289524+0000","last_became_peered":"2024-04-25T10:11:35.289524+0000","last_unstale":"2024-04-25T10:11:35.301861+0000","last_undegraded":"2024-04-25T10:11:35.301861+0000","last_fullsized":"2024-04-25T10:11:35.301861+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:41.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.260211+0000","last_change":"2024-04-25T10:11:34.277305+0000","last_active":"2024-04-25T10:11:35.260211+0000","last_peered":"2024-04-25T10:11:35.260211+0000","last_clean":"2024-04-25T10:11:35.260211+0000","last_became_active":"2024-04-25T10:11:34.276945+0000","last_became_peered":"2024-04-25T10:11:34.276945+0000","last_unstale":"2024-04-25T10:11:35.260211+0000","last_undegraded":"2024-04-25T10:11:35.260211+0000","last_fullsized":"2024-04-25T10:11:35.260211+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:31.312447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.308248+0000","last_change":"2024-04-25T10:11:35.296112+0000","last_active":"2024-04-25T10:11:35.308248+0000","last_peered":"2024-04-25T10:11:35.308248+0000","last_clean":"2024-04-25T10:11:35.308248+0000","last_became_active":"2024-04-25T10:11:35.295803+0000","last_became_peered":"2024-04-25T10:11:35.295803+0000","last_unstale":"2024-04-25T10:11:35.308248+0000","last_undegraded":"2024-04-25T10:11:35.308248+0000","last_fullsized":"2024-04-25T10:11:35.308248+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:43.322009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.281696+0000","last_change":"2024-04-25T10:11:34.280684+0000","last_active":"2024-04-25T10:11:35.281696+0000","last_peered":"2024-04-25T10:11:35.281696+0000","last_clean":"2024-04-25T10:11:35.281696+0000","last_became_active":"2024-04-25T10:11:34.280258+0000","last_became_peered":"2024-04-25T10:11:34.280258+0000","last_unstale":"2024-04-25T10:11:35.281696+0000","last_undegraded":"2024-04-25T10:11:35.281696+0000","last_fullsized":"2024-04-25T10:11:35.281696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:20.941871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.301568+0000","last_change":"2024-04-25T10:11:35.292920+0000","last_active":"2024-04-25T10:11:35.301568+0000","last_peered":"2024-04-25T10:11:35.301568+0000","last_clean":"2024-04-25T10:11:35.301568+0000","last_became_active":"2024-04-25T10:11:35.292602+0000","last_became_peered":"2024-04-25T10:11:35.292602+0000","last_unstale":"2024-04-25T10:11:35.301568+0000","last_undegraded":"2024-04-25T10:11:35.301568+0000","last_fullsized":"2024-04-25T10:11:35.301568+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:38.314011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.275560+0000","last_change":"2024-04-25T10:11:34.287140+0000","last_active":"2024-04-25T10:11:35.275560+0000","last_peered":"2024-04-25T10:11:35.275560+0000","last_clean":"2024-04-25T10:11:35.275560+0000","last_became_active":"2024-04-25T10:11:34.286827+0000","last_became_peered":"2024-04-25T10:11:34.286827+0000","last_unstale":"2024-04-25T10:11:35.275560+0000","last_undegraded":"2024-04-25T10:11:35.275560+0000","last_fullsized":"2024-04-25T10:11:35.275560+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:35.636729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.300506+0000","last_change":"2024-04-25T10:11:35.296096+0000","last_active":"2024-04-25T10:11:35.300506+0000","last_peered":"2024-04-25T10:11:35.300506+0000","last_clean":"2024-04-25T10:11:35.300506+0000","last_became_active":"2024-04-25T10:11:35.295797+0000","last_became_peered":"2024-04-25T10:11:35.295797+0000","last_unstale":"2024-04-25T10:11:35.300506+0000","last_undegraded":"2024-04-25T10:11:35.300506+0000","last_fullsized":"2024-04-25T10:11:35.300506+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:14.514277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.264614+0000","last_change":"2024-04-25T10:11:34.282019+0000","last_active":"2024-04-25T10:11:35.264614+0000","last_peered":"2024-04-25T10:11:35.264614+0000","last_clean":"2024-04-25T10:11:35.264614+0000","last_became_active":"2024-04-25T10:11:34.281724+0000","last_became_peered":"2024-04-25T10:11:34.281724+0000","last_unstale":"2024-04-25T10:11:35.264614+0000","last_undegraded":"2024-04-25T10:11:35.264614+0000","last_fullsized":"2024-04-25T10:11:35.264614+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:31.283351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.304332+0000","last_change":"2024-04-25T10:11:35.296507+0000","last_active":"2024-04-25T10:11:35.304332+0000","last_peered":"2024-04-25T10:11:35.304332+0000","last_clean":"2024-04-25T10:11:35.304332+0000","last_became_active":"2024-04-25T10:11:35.296251+0000","last_became_peered":"2024-04-25T10:11:35.296251+0000","last_unstale":"2024-04-25T10:11:35.304332+0000","last_undegraded":"2024-04-25T10:11:35.304332+0000","last_fullsized":"2024-04-25T10:11:35.304332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:32.298730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.282260+0000","last_change":"2024-04-25T10:11:34.288829+0000","last_active":"2024-04-25T10:11:35.282260+0000","last_peered":"2024-04-25T10:11:35.282260+0000","last_clean":"2024-04-25T10:11:35.282260+0000","last_became_active":"2024-04-25T10:11:34.286659+0000","last_became_peered":"2024-04-25T10:11:34.286659+0000","last_unstale":"2024-04-25T10:11:35.282260+0000","last_undegraded":"2024-04-25T10:11:35.282260+0000","last_fullsized":"2024-04-25T10:11:35.282260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:23.584908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.303851+0000","last_change":"2024-04-25T10:11:35.297109+0000","last_active":"2024-04-25T10:11:35.303851+0000","last_peered":"2024-04-25T10:11:35.303851+0000","last_clean":"2024-04-25T10:11:35.303851+0000","last_became_active":"2024-04-25T10:11:35.296802+0000","last_became_peered":"2024-04-25T10:11:35.296802+0000","last_unstale":"2024-04-25T10:11:35.303851+0000","last_undegraded":"2024-04-25T10:11:35.303851+0000","last_fullsized":"2024-04-25T10:11:35.303851+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:34.234708+0000","last_clean_scrub_stamp":"2024-04-25T10:11:34.234708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:40.761823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.311072+0000","last_change":"2024-04-25T10:11:34.289411+0000","last_active":"2024-04-25T10:11:35.311072+0000","last_peered":"2024-04-25T10:11:35.311072+0000","last_clean":"2024-04-25T10:11:35.311072+0000","last_became_active":"2024-04-25T10:11:34.289084+0000","last_became_peered":"2024-04-25T10:11:34.289084+0000","last_unstale":"2024-04-25T10:11:35.311072+0000","last_undegraded":"2024-04-25T10:11:35.311072+0000","last_fullsized":"2024-04-25T10:11:35.311072+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:33.219329+0000","last_clean_scrub_stamp":"2024-04-25T10:11:33.219329+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:54.490324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":256,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T10:11:35.280466+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:11:35.280466+0000","last_peered":"2024-04-25T10:11:35.280466+0000","last_clean":"2024-04-25T10:11:35.280466+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:11:35.280466+0000","last_undegraded":"2024-04-25T10:11:35.280466+0000","last_fullsized":"2024-04-25T10:11:35.280466+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1384448,"data_stored":918670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":335169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":335167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":335147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":335189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":815104,"data_stored":336689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":790528,"data_stored":328302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":925445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:40.186 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:11:40.187 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:11:40.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.435+0000 7feefae00640 1 Processor -- start 2024-04-25T10:11:40.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.435+0000 7feefae00640 1 -- start start 2024-04-25T10:11:40.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.436+0000 7feefae00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7feef415dc90 0x7feef40d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.436+0000 7feefae00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef40b9290 0x7feef40d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.436+0000 7feefae00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef40d30f0 con 0x7feef40b96f0 2024-04-25T10:11:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.436+0000 7feefae00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef415e070 con 0x7feef415dc90 2024-04-25T10:11:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.436+0000 7feefae00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef40d1b10 con 0x7feef40b9290 2024-04-25T10:11:40.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.437+0000 7feef3400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef40b9290 0x7feef40d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.437+0000 7feef3e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7feef415dc90 0x7feef40d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.437+0000 7feef3400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef40b9290 0x7feef40d0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:40994/0 (socket says 172.21.15.119:40994) 2024-04-25T10:11:40.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.437+0000 7feef3e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7feef415dc90 0x7feef40d2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:53702/0 (socket says 172.21.15.119:53702) 2024-04-25T10:11:40.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.437+0000 7feef3400640 1 -- 172.21.15.119:0/1547873520 learned_addr learned my addr 172.21.15.119:0/1547873520 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:40.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.437+0000 7feef8e00640 1 --1- 172.21.15.119:0/1547873520 >> v1:172.21.15.132:6789/0 conn(0x7feef40b96f0 0x7feef40b6aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:39646/0 (socket says 172.21.15.119:39646) 2024-04-25T10:11:40.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.438+0000 7feef2a00640 1 -- 172.21.15.119:0/1547873520 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343506716 0 0) 0x7feef40d30f0 con 0x7feef40b96f0 2024-04-25T10:11:40.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.438+0000 7feef2a00640 1 -- 172.21.15.119:0/1547873520 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feed4003520 con 0x7feef40b96f0 2024-04-25T10:11:40.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.438+0000 7feef3400640 1 -- 172.21.15.119:0/1547873520 >> v2:172.21.15.132:3300/0 conn(0x7feef415dc90 msgr2=0x7feef40d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:40.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.438+0000 7feef3400640 1 --2- 172.21.15.119:0/1547873520 >> v2:172.21.15.132:3300/0 conn(0x7feef415dc90 0x7feef40d2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.438+0000 7feef3400640 1 -- 172.21.15.119:0/1547873520 >> v1:172.21.15.132:6789/0 conn(0x7feef40b96f0 legacy=0x7feef40b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.438+0000 7feef3400640 1 -- 172.21.15.119:0/1547873520 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef40d1df0 con 0x7feef40b9290 2024-04-25T10:11:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.439+0000 7feef3400640 1 --2- 172.21.15.119:0/1547873520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef40b9290 0x7feef40d0420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feedc009e50 tx=0x7feedc02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd6f2369a1424a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.440+0000 7feef2a00640 1 -- 172.21.15.119:0/1547873520 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc036070 con 0x7feef40b9290 2024-04-25T10:11:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.440+0000 7feef2a00640 1 -- 172.21.15.119:0/1547873520 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc031600 con 0x7feef40b9290 2024-04-25T10:11:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.440+0000 7feef2a00640 1 -- 172.21.15.119:0/1547873520 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc0319d0 con 0x7feef40b9290 2024-04-25T10:11:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.441+0000 7feefae00640 1 -- 172.21.15.119:0/1547873520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef40b9290 msgr2=0x7feef40d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.441+0000 7feefae00640 1 --2- 172.21.15.119:0/1547873520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef40b9290 0x7feef40d0420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feedc009e50 tx=0x7feedc02a710 comp rx=0 tx=0).stop 2024-04-25T10:11:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.441+0000 7feefae00640 1 -- 172.21.15.119:0/1547873520 shutdown_connections 2024-04-25T10:11:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.441+0000 7feefae00640 1 --2- 172.21.15.119:0/1547873520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef40b9290 0x7feef40d0420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.441+0000 7feefae00640 1 --2- 172.21.15.119:0/1547873520 >> v2:172.21.15.132:3300/0 conn(0x7feef415dc90 0x7feef40d2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.441+0000 7feefae00640 1 -- 172.21.15.119:0/1547873520 >> 172.21.15.119:0/1547873520 conn(0x7feef4072900 msgr2=0x7feef406c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:40.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.441+0000 7feefae00640 1 -- 172.21.15.119:0/1547873520 shutdown_connections 2024-04-25T10:11:40.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.441+0000 7feefae00640 1 -- 172.21.15.119:0/1547873520 wait complete. 2024-04-25T10:11:40.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.442+0000 7feefae00640 1 Processor -- start 2024-04-25T10:11:40.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.443+0000 7feefae00640 1 -- start start 2024-04-25T10:11:40.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.443+0000 7feefae00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef40b9290 0x7feef40c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.443+0000 7feefae00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef40b96f0 0x7feef40cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.444+0000 7feefae00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feef415dc90 0x7feef40cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.444+0000 7feefae00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef40d2690 con 0x7feef415dc90 2024-04-25T10:11:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.444+0000 7feefae00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef40d23b0 con 0x7feef40b9290 2024-04-25T10:11:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.444+0000 7feefae00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef40d3480 con 0x7feef40b96f0 2024-04-25T10:11:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.444+0000 7feef8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feef415dc90 0x7feef40cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.444+0000 7feef8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feef415dc90 0x7feef40cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43124/0 (socket says 172.21.15.119:43124) 2024-04-25T10:11:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.444+0000 7feef8e00640 1 -- 172.21.15.119:0/1138046605 learned_addr learned my addr 172.21.15.119:0/1138046605 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.444+0000 7feef3400640 1 --2- 172.21.15.119:0/1138046605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef40b96f0 0x7feef40cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.444+0000 7feef3e00640 1 --2- 172.21.15.119:0/1138046605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef40b9290 0x7feef40c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.444+0000 7feef8e00640 1 -- 172.21.15.119:0/1138046605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef40b96f0 msgr2=0x7feef40cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.444+0000 7feef8e00640 1 --2- 172.21.15.119:0/1138046605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef40b96f0 0x7feef40cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.444+0000 7feef8e00640 1 -- 172.21.15.119:0/1138046605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef40b9290 msgr2=0x7feef40c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.444+0000 7feef8e00640 1 --2- 172.21.15.119:0/1138046605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef40b9290 0x7feef40c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.444+0000 7feef8e00640 1 -- 172.21.15.119:0/1138046605 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef4161180 con 0x7feef415dc90 2024-04-25T10:11:40.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.445+0000 7feef8e00640 1 --2- 172.21.15.119:0/1138046605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feef415dc90 0x7feef40cc180 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7feeec00c400 tx=0x7feeec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.445+0000 7feeebe00640 1 -- 172.21.15.119:0/1138046605 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0074e0 con 0x7feef415dc90 2024-04-25T10:11:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.445+0000 7feeebe00640 1 -- 172.21.15.119:0/1138046605 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec007680 con 0x7feef415dc90 2024-04-25T10:11:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.445+0000 7feefae00640 1 -- 172.21.15.119:0/1138046605 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef4160320 con 0x7feef415dc90 2024-04-25T10:11:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.445+0000 7feeebe00640 1 -- 172.21.15.119:0/1138046605 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec004a20 con 0x7feef415dc90 2024-04-25T10:11:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.445+0000 7feefae00640 1 -- 172.21.15.119:0/1138046605 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef4160bc0 con 0x7feef415dc90 2024-04-25T10:11:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.447+0000 7feefae00640 1 -- 172.21.15.119:0/1138046605 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef40749b0 con 0x7feef415dc90 2024-04-25T10:11:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.447+0000 7feeebe00640 1 -- 172.21.15.119:0/1138046605 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feeec01a020 con 0x7feef415dc90 2024-04-25T10:11:40.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.448+0000 7feeebe00640 1 --2- 172.21.15.119:0/1138046605 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7feed006b4d0 0x7feed006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.448+0000 7feeebe00640 1 -- 172.21.15.119:0/1138046605 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feeec089aa0 con 0x7feef415dc90 2024-04-25T10:11:40.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.449+0000 7feef3e00640 1 --2- 172.21.15.119:0/1138046605 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7feed006b4d0 0x7feed006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.449+0000 7feef3e00640 1 --2- 172.21.15.119:0/1138046605 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7feed006b4d0 0x7feed006d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feee4009a30 tx=0x7feee4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:40.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.453+0000 7feeebe00640 1 -- 172.21.15.119:0/1138046605 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feeec010300 con 0x7feef415dc90 2024-04-25T10:11:40.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.718+0000 7feefae00640 1 -- 172.21.15.119:0/1138046605 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7feef406c4b0 con 0x7feef415dc90 2024-04-25T10:11:40.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.722+0000 7feeebe00640 1 -- 172.21.15.119:0/1138046605 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7feeec058920 con 0x7feef415dc90 2024-04-25T10:11:40.727 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-04-25T10:11:40.727 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA== 2024-04-25T10:11:40.728 INFO:teuthology.orchestra.run.smithi119.stdout: caps mds = "allow" 2024-04-25T10:11:40.728 INFO:teuthology.orchestra.run.smithi119.stdout: caps mon = "allow r" 2024-04-25T10:11:40.728 INFO:teuthology.orchestra.run.smithi119.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:11:40.728 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2024-04-25T10:11:40.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.725+0000 7feefae00640 1 -- 172.21.15.119:0/1138046605 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7feed006b4d0 msgr2=0x7feed006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.725+0000 7feefae00640 1 --2- 172.21.15.119:0/1138046605 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7feed006b4d0 0x7feed006d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feee4009a30 tx=0x7feee4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:40.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.725+0000 7feefae00640 1 -- 172.21.15.119:0/1138046605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feef415dc90 msgr2=0x7feef40cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.725+0000 7feefae00640 1 --2- 172.21.15.119:0/1138046605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feef415dc90 0x7feef40cc180 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7feeec00c400 tx=0x7feeec00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.725+0000 7feefae00640 1 -- 172.21.15.119:0/1138046605 shutdown_connections 2024-04-25T10:11:40.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.725+0000 7feefae00640 1 --2- 172.21.15.119:0/1138046605 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7feed006b4d0 0x7feed006d990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.725+0000 7feefae00640 1 --2- 172.21.15.119:0/1138046605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feef415dc90 0x7feef40cc180 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.725+0000 7feefae00640 1 --2- 172.21.15.119:0/1138046605 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef40b96f0 0x7feef40cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.725+0000 7feefae00640 1 --2- 172.21.15.119:0/1138046605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef40b9290 0x7feef40c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.725+0000 7feefae00640 1 -- 172.21.15.119:0/1138046605 >> 172.21.15.119:0/1138046605 conn(0x7feef4072900 msgr2=0x7feef40d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:40.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.725+0000 7feefae00640 1 -- 172.21.15.119:0/1138046605 shutdown_connections 2024-04-25T10:11:40.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.726+0000 7feefae00640 1 -- 172.21.15.119:0/1138046605 wait complete. 2024-04-25T10:11:40.749 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:40.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.941+0000 7f2015600640 1 Processor -- start 2024-04-25T10:11:40.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.941+0000 7f2015600640 1 -- start start 2024-04-25T10:11:40.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.943+0000 7f2015600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f201015bf00 0x7f20100cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.943+0000 7f2015600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20100cc310 0x7f201015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.943+0000 7f2015600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f201015c820 0x7f201006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.943+0000 7f2015600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20101455e0 con 0x7f201015c820 2024-04-25T10:11:40.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.943+0000 7f2015600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f201006d450 con 0x7f201015bf00 2024-04-25T10:11:40.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.943+0000 7f2015600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20100ce6c0 con 0x7f20100cc310 2024-04-25T10:11:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.943+0000 7f200f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f201015c820 0x7f201006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.943+0000 7f200f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f201015c820 0x7f201006d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55606/0 (socket says 172.21.15.119:55606) 2024-04-25T10:11:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.943+0000 7f200f400640 1 -- 172.21.15.119:0/1273140617 learned_addr learned my addr 172.21.15.119:0/1273140617 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.943+0000 7f2007e00640 1 --2- 172.21.15.119:0/1273140617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20100cc310 0x7f201015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.943+0000 7f200ea00640 1 --2- 172.21.15.119:0/1273140617 >> v2:172.21.15.132:3300/0 conn(0x7f201015bf00 0x7f20100cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.944+0000 7f200f400640 1 -- 172.21.15.119:0/1273140617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20100cc310 msgr2=0x7f201015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.944+0000 7f200f400640 1 --2- 172.21.15.119:0/1273140617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20100cc310 0x7f201015c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.944+0000 7f200f400640 1 -- 172.21.15.119:0/1273140617 >> v2:172.21.15.132:3300/0 conn(0x7f201015bf00 msgr2=0x7f20100cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.944+0000 7f200f400640 1 --2- 172.21.15.119:0/1273140617 >> v2:172.21.15.132:3300/0 conn(0x7f201015bf00 0x7f20100cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.944+0000 7f200f400640 1 -- 172.21.15.119:0/1273140617 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20100cc7b0 con 0x7f201015c820 2024-04-25T10:11:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.945+0000 7f200f400640 1 --2- 172.21.15.119:0/1273140617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f201015c820 0x7f201006d8e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f201006ba10 tx=0x7f200802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54223d57d780774 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.945+0000 7f200e000640 1 -- 172.21.15.119:0/1273140617 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008036070 con 0x7f201015c820 2024-04-25T10:11:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.945+0000 7f200e000640 1 -- 172.21.15.119:0/1273140617 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2008031600 con 0x7f201015c820 2024-04-25T10:11:40.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.945+0000 7f200e000640 1 -- 172.21.15.119:0/1273140617 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080319d0 con 0x7f201015c820 2024-04-25T10:11:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.946+0000 7f2015600640 1 -- 172.21.15.119:0/1273140617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f201015c820 msgr2=0x7f201006d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.946+0000 7f2015600640 1 --2- 172.21.15.119:0/1273140617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f201015c820 0x7f201006d8e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f201006ba10 tx=0x7f200802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.946+0000 7f2015600640 1 -- 172.21.15.119:0/1273140617 shutdown_connections 2024-04-25T10:11:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.946+0000 7f2015600640 1 --2- 172.21.15.119:0/1273140617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f201015c820 0x7f201006d8e0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.946+0000 7f2015600640 1 --2- 172.21.15.119:0/1273140617 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20100cc310 0x7f201015c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.946+0000 7f2015600640 1 --2- 172.21.15.119:0/1273140617 >> v2:172.21.15.132:3300/0 conn(0x7f201015bf00 0x7f20100cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.946+0000 7f2015600640 1 -- 172.21.15.119:0/1273140617 >> 172.21.15.119:0/1273140617 conn(0x7f201006aba0 msgr2=0x7f201006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.946+0000 7f2015600640 1 -- 172.21.15.119:0/1273140617 shutdown_connections 2024-04-25T10:11:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.946+0000 7f2015600640 1 -- 172.21.15.119:0/1273140617 wait complete. 2024-04-25T10:11:40.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.948+0000 7f2015600640 1 Processor -- start 2024-04-25T10:11:40.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.948+0000 7f2015600640 1 -- start start 2024-04-25T10:11:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.949+0000 7f2015600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20100cc310 0x7f201013abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.949+0000 7f2015600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f201015bf00 0x7f2010140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.949+0000 7f2015600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f201015c820 0x7f2010142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.949+0000 7f2015600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20100cd050 con 0x7f201015bf00 2024-04-25T10:11:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.949+0000 7f2015600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20100ccd70 con 0x7f20100cc310 2024-04-25T10:11:40.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.949+0000 7f2015600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201015cca0 con 0x7f201015c820 2024-04-25T10:11:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.949+0000 7f2007e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f201015bf00 0x7f2010140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.949+0000 7f2007e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f201015bf00 0x7f2010140060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55622/0 (socket says 172.21.15.119:55622) 2024-04-25T10:11:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.949+0000 7f2007e00640 1 -- 172.21.15.119:0/3428796417 learned_addr learned my addr 172.21.15.119:0/3428796417 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.949+0000 7f2007e00640 1 -- 172.21.15.119:0/3428796417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f201015c820 msgr2=0x7f2010142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.949+0000 7f2007e00640 1 --2- 172.21.15.119:0/3428796417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f201015c820 0x7f2010142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.949+0000 7f2007e00640 1 -- 172.21.15.119:0/3428796417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20100cc310 msgr2=0x7f201013abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.949+0000 7f200ea00640 1 --2- 172.21.15.119:0/3428796417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20100cc310 0x7f201013abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.949+0000 7f2007e00640 1 --2- 172.21.15.119:0/3428796417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20100cc310 0x7f201013abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.950+0000 7f2007e00640 1 -- 172.21.15.119:0/3428796417 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2010140740 con 0x7f201015bf00 2024-04-25T10:11:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.950+0000 7f2007e00640 1 --2- 172.21.15.119:0/3428796417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f201015bf00 0x7f2010140060 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800c820 tx=0x7f1ff800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.950+0000 7f2006a00640 1 -- 172.21.15.119:0/3428796417 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8015070 con 0x7f201015bf00 2024-04-25T10:11:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.950+0000 7f2006a00640 1 -- 172.21.15.119:0/3428796417 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff8004220 con 0x7f201015bf00 2024-04-25T10:11:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.950+0000 7f2006a00640 1 -- 172.21.15.119:0/3428796417 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800ce10 con 0x7f201015bf00 2024-04-25T10:11:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.950+0000 7f2015600640 1 -- 172.21.15.119:0/3428796417 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2010140a20 con 0x7f201015bf00 2024-04-25T10:11:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.950+0000 7f2015600640 1 -- 172.21.15.119:0/3428796417 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2010140fe0 con 0x7f201015bf00 2024-04-25T10:11:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.952+0000 7f2015600640 1 -- 172.21.15.119:0/3428796417 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f201006bb30 con 0x7f201015bf00 2024-04-25T10:11:40.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.952+0000 7f2006a00640 1 -- 172.21.15.119:0/3428796417 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ff8007830 con 0x7f201015bf00 2024-04-25T10:11:40.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.953+0000 7f2006a00640 1 --2- 172.21.15.119:0/3428796417 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1fec06b4d0 0x7f1fec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:40.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.953+0000 7f2006a00640 1 -- 172.21.15.119:0/3428796417 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ff808a360 con 0x7f201015bf00 2024-04-25T10:11:40.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.954+0000 7f200ea00640 1 --2- 172.21.15.119:0/3428796417 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1fec06b4d0 0x7f1fec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:40.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.954+0000 7f200ea00640 1 --2- 172.21.15.119:0/3428796417 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1fec06b4d0 0x7f1fec06d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f200000b7e0 tx=0x7f2000008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:40.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:40.957+0000 7f2006a00640 1 -- 172.21.15.119:0/3428796417 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ff8059270 con 0x7f201015bf00 2024-04-25T10:11:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.193+0000 7f2015600640 1 -- 172.21.15.119:0/3428796417 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20100749b0 con 0x7f201015bf00 2024-04-25T10:11:41.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.194+0000 7f2006a00640 1 -- 172.21.15.119:0/3428796417 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f1ff8058b70 con 0x7f201015bf00 2024-04-25T10:11:41.200 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:41.200 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2798722683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2798722683},{"type":"v1","addr":"172.21.15.132:6833","nonce":2798722683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2834911408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2834911408},{"type":"v1","addr":"172.21.15.132:6835","nonce":2834911408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/643040335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":643040335},{"type":"v1","addr":"172.21.15.119:6835","nonce":643040335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:34.712290+0000","modified":"2024-04-25T10:11:37.787344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6483},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6483":{"gid":6483,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/2673349178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2673349178},{"type":"v1","addr":"172.21.15.119:6837","nonce":2673349178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:41.200 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 44 2024-04-25T10:11:41.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.197+0000 7f2015600640 1 -- 172.21.15.119:0/3428796417 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1fec06b4d0 msgr2=0x7f1fec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.197+0000 7f2015600640 1 --2- 172.21.15.119:0/3428796417 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1fec06b4d0 0x7f1fec06d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f200000b7e0 tx=0x7f2000008040 comp rx=0 tx=0).stop 2024-04-25T10:11:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.197+0000 7f2015600640 1 -- 172.21.15.119:0/3428796417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f201015bf00 msgr2=0x7f2010140060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.197+0000 7f2015600640 1 --2- 172.21.15.119:0/3428796417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f201015bf00 0x7f2010140060 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800c820 tx=0x7f1ff800cce0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.197+0000 7f2015600640 1 -- 172.21.15.119:0/3428796417 shutdown_connections 2024-04-25T10:11:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.197+0000 7f2015600640 1 --2- 172.21.15.119:0/3428796417 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1fec06b4d0 0x7f1fec06d990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.197+0000 7f2015600640 1 --2- 172.21.15.119:0/3428796417 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f201015c820 0x7f2010142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.197+0000 7f2015600640 1 --2- 172.21.15.119:0/3428796417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f201015bf00 0x7f2010140060 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.197+0000 7f2015600640 1 --2- 172.21.15.119:0/3428796417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20100cc310 0x7f201013abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.197+0000 7f2015600640 1 -- 172.21.15.119:0/3428796417 >> 172.21.15.119:0/3428796417 conn(0x7f201006aba0 msgr2=0x7f2010071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.197+0000 7f2015600640 1 -- 172.21.15.119:0/3428796417 shutdown_connections 2024-04-25T10:11:41.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.197+0000 7f2015600640 1 -- 172.21.15.119:0/3428796417 wait complete. 2024-04-25T10:11:41.625 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:34.712290+0000', 'modified': '2024-04-25T10:11:37.787344+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6483}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6483': {'gid': 6483, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6837/2673349178', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 2673349178}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2673349178}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:41.625 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:11:41.625 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:11:41.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.834+0000 7f053a600640 1 Processor -- start 2024-04-25T10:11:41.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.834+0000 7f053a600640 1 -- start start 2024-04-25T10:11:41.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.836+0000 7f053a600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f053415dbe0 0x7f0534071c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.836+0000 7f053a600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f05340cfad0 0x7f053406d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.836+0000 7f053a600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05340ace70 con 0x7f05340aedc0 2024-04-25T10:11:41.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.836+0000 7f053a600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0534072150 con 0x7f053415dbe0 2024-04-25T10:11:41.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.836+0000 7f053a600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05340ad110 con 0x7f05340cfad0 2024-04-25T10:11:41.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.836+0000 7f0532a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f05340aedc0 0x7f05340cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40206/0 (socket says 172.21.15.119:40206) 2024-04-25T10:11:41.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.836+0000 7f0533400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f053415dbe0 0x7f0534071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.836+0000 7f0532a00640 1 -- 172.21.15.119:0/1484748736 learned_addr learned my addr 172.21.15.119:0/1484748736 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:41.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.837+0000 7f0533e00640 1 --2- 172.21.15.119:0/1484748736 >> v2:172.21.15.132:3300/0 conn(0x7f05340cfad0 0x7f053406d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.837+0000 7f0532000640 1 -- 172.21.15.119:0/1484748736 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805271035 0 0) 0x7f05340ace70 con 0x7f05340aedc0 2024-04-25T10:11:41.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.837+0000 7f0532000640 1 -- 172.21.15.119:0/1484748736 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0518003520 con 0x7f05340aedc0 2024-04-25T10:11:41.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.837+0000 7f0533400640 1 -- 172.21.15.119:0/1484748736 >> v2:172.21.15.132:3300/0 conn(0x7f05340cfad0 msgr2=0x7f053406d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:41.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.837+0000 7f0533400640 1 --2- 172.21.15.119:0/1484748736 >> v2:172.21.15.132:3300/0 conn(0x7f05340cfad0 0x7f053406d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.837+0000 7f0533400640 1 -- 172.21.15.119:0/1484748736 >> v1:172.21.15.119:6789/0 conn(0x7f05340aedc0 legacy=0x7f05340cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.837+0000 7f0533400640 1 -- 172.21.15.119:0/1484748736 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340ad3f0 con 0x7f053415dbe0 2024-04-25T10:11:41.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.837+0000 7f0533e00640 1 --2- 172.21.15.119:0/1484748736 >> v2:172.21.15.132:3300/0 conn(0x7f05340cfad0 0x7f053406d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:41.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.838+0000 7f0533400640 1 --2- 172.21.15.119:0/1484748736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053415dbe0 0x7f0534071c10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f05340aec70 tx=0x7f052402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15bb67ab2e63206 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.839+0000 7f0532000640 1 -- 172.21.15.119:0/1484748736 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524037070 con 0x7f053415dbe0 2024-04-25T10:11:41.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.839+0000 7f0532000640 1 -- 172.21.15.119:0/1484748736 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0524032440 con 0x7f053415dbe0 2024-04-25T10:11:41.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.839+0000 7f0532000640 1 -- 172.21.15.119:0/1484748736 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524032810 con 0x7f053415dbe0 2024-04-25T10:11:41.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.839+0000 7f053a600640 1 -- 172.21.15.119:0/1484748736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053415dbe0 msgr2=0x7f0534071c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.839+0000 7f053a600640 1 --2- 172.21.15.119:0/1484748736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053415dbe0 0x7f0534071c10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f05340aec70 tx=0x7f052402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.839+0000 7f053a600640 1 -- 172.21.15.119:0/1484748736 shutdown_connections 2024-04-25T10:11:41.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.839+0000 7f053a600640 1 --2- 172.21.15.119:0/1484748736 >> v2:172.21.15.132:3300/0 conn(0x7f05340cfad0 0x7f053406d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.839+0000 7f053a600640 1 --2- 172.21.15.119:0/1484748736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053415dbe0 0x7f0534071c10 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.839+0000 7f053a600640 1 -- 172.21.15.119:0/1484748736 >> 172.21.15.119:0/1484748736 conn(0x7f053406aba0 msgr2=0x7f053406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:41.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.840+0000 7f053a600640 1 -- 172.21.15.119:0/1484748736 shutdown_connections 2024-04-25T10:11:41.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.840+0000 7f053a600640 1 -- 172.21.15.119:0/1484748736 wait complete. 2024-04-25T10:11:41.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.841+0000 7f053a600640 1 Processor -- start 2024-04-25T10:11:41.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.842+0000 7f053a600640 1 -- start start 2024-04-25T10:11:41.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.842+0000 7f053a600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05340aedc0 0x7f05340cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.842+0000 7f053a600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05340cfad0 0x7f05340b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.842+0000 7f053a600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053415dbe0 0x7f053406d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.843+0000 7f053a600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05340adc90 con 0x7f053415dbe0 2024-04-25T10:11:41.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.843+0000 7f053a600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05340ad9b0 con 0x7f05340aedc0 2024-04-25T10:11:41.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.843+0000 7f053a600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053415dfc0 con 0x7f05340cfad0 2024-04-25T10:11:41.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.843+0000 7f0533e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053415dbe0 0x7f053406d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.843+0000 7f0533e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053415dbe0 0x7f053406d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55640/0 (socket says 172.21.15.119:55640) 2024-04-25T10:11:41.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.843+0000 7f0533e00640 1 -- 172.21.15.119:0/3671914651 learned_addr learned my addr 172.21.15.119:0/3671914651 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:41.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.843+0000 7f0533e00640 1 -- 172.21.15.119:0/3671914651 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05340cfad0 msgr2=0x7f05340b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.843+0000 7f0533e00640 1 --2- 172.21.15.119:0/3671914651 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05340cfad0 0x7f05340b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.843+0000 7f0533e00640 1 -- 172.21.15.119:0/3671914651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05340aedc0 msgr2=0x7f05340cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.843+0000 7f0533400640 1 --2- 172.21.15.119:0/3671914651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05340aedc0 0x7f05340cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.843+0000 7f0533e00640 1 --2- 172.21.15.119:0/3671914651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05340aedc0 0x7f05340cafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.843+0000 7f0533e00640 1 -- 172.21.15.119:0/3671914651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340cb500 con 0x7f053415dbe0 2024-04-25T10:11:41.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.844+0000 7f0533e00640 1 --2- 172.21.15.119:0/3671914651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053415dbe0 0x7f053406d9a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f052c00cf20 tx=0x7f052c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.844+0000 7f052b400640 1 -- 172.21.15.119:0/3671914651 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c01a070 con 0x7f053415dbe0 2024-04-25T10:11:41.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.844+0000 7f052b400640 1 -- 172.21.15.119:0/3671914651 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c00d840 con 0x7f053415dbe0 2024-04-25T10:11:41.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.844+0000 7f052b400640 1 -- 172.21.15.119:0/3671914651 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c00dc30 con 0x7f053415dbe0 2024-04-25T10:11:41.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.844+0000 7f053a600640 1 -- 172.21.15.119:0/3671914651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05340b0ef0 con 0x7f053415dbe0 2024-04-25T10:11:41.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.844+0000 7f053a600640 1 -- 172.21.15.119:0/3671914651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05340b14b0 con 0x7f053415dbe0 2024-04-25T10:11:41.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.846+0000 7f052b400640 1 -- 172.21.15.119:0/3671914651 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f052c01d070 con 0x7f053415dbe0 2024-04-25T10:11:41.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.847+0000 7f052b400640 1 --2- 172.21.15.119:0/3671914651 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f051406b4d0 0x7f051406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.847+0000 7f052b400640 1 -- 172.21.15.119:0/3671914651 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f052c08aab0 con 0x7f053415dbe0 2024-04-25T10:11:41.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.847+0000 7f0533400640 1 --2- 172.21.15.119:0/3671914651 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f051406b4d0 0x7f051406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.848+0000 7f053d41a740 1 Processor -- start 2024-04-25T10:11:41.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.848+0000 7f0533400640 1 --2- 172.21.15.119:0/3671914651 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f051406b4d0 0x7f051406d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0524034000 tx=0x7f052402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.848+0000 7f053d41a740 1 -- start start 2024-04-25T10:11:41.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.849+0000 7f053d41a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d1929c8950 0x55d1929f0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.849+0000 7f053d41a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929f0cd0 0x55d1929c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.849+0000 7f053d41a740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d192a115c0 0x55d1929c7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.849+0000 7f053d41a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d19292c620 con 0x55d1929f0cd0 2024-04-25T10:11:41.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.849+0000 7f053d41a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1928d5ca0 con 0x55d1929c8950 2024-04-25T10:11:41.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.849+0000 7f053d41a740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1929286c0 con 0x55d192a115c0 2024-04-25T10:11:41.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.849+0000 7f0532a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929f0cd0 0x55d1929c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.849+0000 7f0532a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929f0cd0 0x55d1929c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55648/0 (socket says 172.21.15.119:55648) 2024-04-25T10:11:41.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.849+0000 7f0532a00640 1 -- 172.21.15.119:0/3411745426 learned_addr learned my addr 172.21.15.119:0/3411745426 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:41.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.850+0000 7f0533400640 1 --2- 172.21.15.119:0/3411745426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d1929c8950 0x55d1929f0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.850+0000 7f0533e00640 1 --2- 172.21.15.119:0/3411745426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d192a115c0 0x55d1929c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.850+0000 7f0532a00640 1 -- 172.21.15.119:0/3411745426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d192a115c0 msgr2=0x55d1929c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:41.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.850+0000 7f0532a00640 1 --2- 172.21.15.119:0/3411745426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d192a115c0 0x55d1929c7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.850+0000 7f0532a00640 1 -- 172.21.15.119:0/3411745426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d1929c8950 msgr2=0x55d1929f0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:41.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.850+0000 7f0532a00640 1 --2- 172.21.15.119:0/3411745426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d1929c8950 0x55d1929f0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.850+0000 7f0532a00640 1 -- 172.21.15.119:0/3411745426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1928dcf40 con 0x55d1929f0cd0 2024-04-25T10:11:41.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.850+0000 7f0532a00640 1 --2- 172.21.15.119:0/3411745426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929f0cd0 0x55d1929c8d30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0520001620 tx=0x7f0520001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9effb69f5060d73e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.850+0000 7f051fe00640 1 -- 172.21.15.119:0/3411745426 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520017070 con 0x55d1929f0cd0 2024-04-25T10:11:41.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.850+0000 7f051fe00640 1 -- 172.21.15.119:0/3411745426 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0520001d30 con 0x55d1929f0cd0 2024-04-25T10:11:41.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.850+0000 7f051fe00640 1 -- 172.21.15.119:0/3411745426 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05200070d0 con 0x55d1929f0cd0 2024-04-25T10:11:41.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.851+0000 7f053d41a740 1 -- 172.21.15.119:0/3411745426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929f0cd0 msgr2=0x55d1929c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.851+0000 7f053d41a740 1 --2- 172.21.15.119:0/3411745426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929f0cd0 0x55d1929c8d30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0520001620 tx=0x7f0520001ae0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.851+0000 7f053d41a740 1 -- 172.21.15.119:0/3411745426 shutdown_connections 2024-04-25T10:11:41.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.851+0000 7f053d41a740 1 --2- 172.21.15.119:0/3411745426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d192a115c0 0x55d1929c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.851+0000 7f053d41a740 1 --2- 172.21.15.119:0/3411745426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929f0cd0 0x55d1929c8d30 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.851+0000 7f053d41a740 1 --2- 172.21.15.119:0/3411745426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d1929c8950 0x55d1929f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.851+0000 7f053d41a740 1 -- 172.21.15.119:0/3411745426 >> 172.21.15.119:0/3411745426 conn(0x55d1929c5b70 msgr2=0x55d192a0e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:41.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.851+0000 7f053d41a740 1 -- 172.21.15.119:0/3411745426 shutdown_connections 2024-04-25T10:11:41.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.851+0000 7f053d41a740 1 -- 172.21.15.119:0/3411745426 wait complete. 2024-04-25T10:11:41.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.852+0000 7f053d41a740 1 -- start start 2024-04-25T10:11:41.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.852+0000 7f053d41a740 1 Processor -- start 2024-04-25T10:11:41.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.853+0000 7f053d41a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929f0cd0 0x55d192a1e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.853+0000 7f053d41a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d192a115c0 0x55d1929eedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.853+0000 7f053d41a740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d1929ed140 0x55d1929e7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.853+0000 7f053d41a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1929f0300 con 0x55d1929f0cd0 2024-04-25T10:11:41.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.853+0000 7f053d41a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1928cb1f0 con 0x55d192a115c0 2024-04-25T10:11:41.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.853+0000 7f053d41a740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1929efcf0 con 0x55d1929ed140 2024-04-25T10:11:41.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.853+0000 7f053d41a740 10 client.0 _ll_register_callbacks cb 0x55d1928e8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:41.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.853+0000 7f0533400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929f0cd0 0x55d192a1e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.853+0000 7f0533400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929f0cd0 0x55d192a1e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55656/0 (socket says 172.21.15.119:55656) 2024-04-25T10:11:41.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.853+0000 7f0533400640 1 -- 172.21.15.119:0/3946515751 learned_addr learned my addr 172.21.15.119:0/3946515751 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:41.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.854+0000 7f0532a00640 1 --2- 172.21.15.119:0/3946515751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d192a115c0 0x55d1929eedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.854+0000 7f0533400640 1 -- 172.21.15.119:0/3946515751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d1929ed140 msgr2=0x55d1929e7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:41.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.854+0000 7f0533400640 1 --2- 172.21.15.119:0/3946515751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d1929ed140 0x55d1929e7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.854+0000 7f0533400640 1 -- 172.21.15.119:0/3946515751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d192a115c0 msgr2=0x55d1929eedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:41.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.854+0000 7f0533400640 1 --2- 172.21.15.119:0/3946515751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d192a115c0 0x55d1929eedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.854+0000 7f0533400640 1 -- 172.21.15.119:0/3946515751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1928eac50 con 0x55d1929f0cd0 2024-04-25T10:11:41.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.854+0000 7f0533400640 1 --2- 172.21.15.119:0/3946515751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929f0cd0 0x55d192a1e850 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f052402a790 tx=0x7f0524004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdadb30756dded4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.854+0000 7f051cc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:11:41.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.854+0000 7f051cc00640 1 -- 172.21.15.119:0/3946515751 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052404c060 con 0x55d1929f0cd0 2024-04-25T10:11:41.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.854+0000 7f051cc00640 1 -- 172.21.15.119:0/3946515751 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f052403c660 con 0x55d1929f0cd0 2024-04-25T10:11:41.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.854+0000 7f053d41a740 1 -- 172.21.15.119:0/3946515751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0520001d30 con 0x55d1929f0cd0 2024-04-25T10:11:41.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.855+0000 7f051cc00640 1 -- 172.21.15.119:0/3946515751 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0524052050 con 0x55d1929f0cd0 2024-04-25T10:11:41.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.855+0000 7f053d41a740 10 client.6627 fetch_fsmap learned FSMap version 44 2024-04-25T10:11:41.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.855+0000 7f053d41a740 1 -- 172.21.15.119:0/3946515751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55d1926412d0 con 0x55d1929f0cd0 2024-04-25T10:11:41.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.855+0000 7f051cc00640 1 -- 172.21.15.119:0/3946515751 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0520001d30 con 0x55d1929f0cd0 2024-04-25T10:11:41.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.855+0000 7f051cc00640 1 -- 172.21.15.119:0/3946515751 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0524055020 con 0x55d1929f0cd0 2024-04-25T10:11:41.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.855+0000 7f053d41a740 10 client.6627 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T10:11:41.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.855+0000 7f053d41a740 10 client.6627 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:41.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.855+0000 7f053d41a740 20 client.6627 populate_metadata read hostname 'smithi119' 2024-04-25T10:11:41.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.855+0000 7f053d41a740 1 --2- 172.21.15.119:0/3946515751 >> [v2:172.21.15.119:6836/2673349178,v1:172.21.15.119:6837/2673349178] conn(0x55d1929e58f0 0x55d192a1f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.855+0000 7f053d41a740 4 client.6627 mds_command: new command op to 6483 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:41.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.855+0000 7f053d41a740 1 -- 172.21.15.119:0/3946515751 --> [v2:172.21.15.119:6836/2673349178,v1:172.21.15.119:6837/2673349178] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d192939b60 con 0x55d1929e58f0 2024-04-25T10:11:41.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.856+0000 7f0532a00640 1 --2- 172.21.15.119:0/3946515751 >> [v2:172.21.15.119:6836/2673349178,v1:172.21.15.119:6837/2673349178] conn(0x55d1929e58f0 0x55d192a1f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.856+0000 7f0532a00640 1 --2- 172.21.15.119:0/3946515751 >> [v2:172.21.15.119:6836/2673349178,v1:172.21.15.119:6837/2673349178] conn(0x55d1929e58f0 0x55d192a1f290 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c20456ee4dda6a7 server_cookie=c1d732b2272914c6 in_seq=0 out_seq=0 2024-04-25T10:11:41.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.856+0000 7f051cc00640 10 client.6627 ms_handle_connect on v2:172.21.15.119:6836/2673349178 2024-04-25T10:11:41.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.857+0000 7f051cc00640 1 -- 172.21.15.119:0/3946515751 <== mds.0 v2:172.21.15.119:6836/2673349178 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d192939b60 con 0x55d1929e58f0 2024-04-25T10:11:41.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.857+0000 7f051cc00640 10 client.6627 handle_command_reply: tid=0 2024-04-25T10:11:41.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.857+0000 7f053d41a740 1 client.6627 shutdown 2024-04-25T10:11:41.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.857+0000 7f053d41a740 1 -- 172.21.15.119:0/3946515751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929f0cd0 msgr2=0x55d192a1e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.857+0000 7f053d41a740 1 --2- 172.21.15.119:0/3946515751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929f0cd0 0x55d192a1e850 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f052402a790 tx=0x7f0524004730 comp rx=0 tx=0).stop 2024-04-25T10:11:41.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.857+0000 7f053d41a740 1 -- 172.21.15.119:0/3946515751 shutdown_connections 2024-04-25T10:11:41.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.857+0000 7f053d41a740 1 --2- 172.21.15.119:0/3946515751 >> [v2:172.21.15.119:6836/2673349178,v1:172.21.15.119:6837/2673349178] conn(0x55d1929e58f0 0x55d192a1f290 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.857+0000 7f053d41a740 1 --2- 172.21.15.119:0/3946515751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d1929ed140 0x55d1929e7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.857+0000 7f053d41a740 1 --2- 172.21.15.119:0/3946515751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d192a115c0 0x55d1929eedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.857+0000 7f053d41a740 1 --2- 172.21.15.119:0/3946515751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929f0cd0 0x55d192a1e850 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.857+0000 7f051cc00640 0 client.6627 ms_handle_reset on v2:172.21.15.119:6836/2673349178 2024-04-25T10:11:41.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.857+0000 7f053d41a740 1 -- 172.21.15.119:0/3946515751 >> 172.21.15.119:0/3946515751 conn(0x55d1929c8950 msgr2=0x55d1929c28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:41.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.858+0000 7f053d41a740 1 -- 172.21.15.119:0/3946515751 shutdown_connections 2024-04-25T10:11:41.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.858+0000 7f053d41a740 1 -- 172.21.15.119:0/3946515751 wait complete. 2024-04-25T10:11:41.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.858+0000 7f053d41a740 20 client.6627 trim_cache size 0 max 16384 2024-04-25T10:11:41.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.874+0000 7f053d41a740 1 Processor -- start 2024-04-25T10:11:41.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.875+0000 7f053d41a740 1 -- start start 2024-04-25T10:11:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.875+0000 7f053d41a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929ed140 0x55d192a286e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.875+0000 7f053d41a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d1929f0cd0 0x55d192a28c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.875+0000 7f053d41a740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d192a115c0 0x55d1929c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.875+0000 7f053d41a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05341ffaf0 con 0x55d1929ed140 2024-04-25T10:11:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.875+0000 7f053d41a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05341f4de0 con 0x55d1929f0cd0 2024-04-25T10:11:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.875+0000 7f053d41a740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053415fb60 con 0x55d192a115c0 2024-04-25T10:11:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.876+0000 7f0533400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929ed140 0x55d192a286e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.876+0000 7f0533400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929ed140 0x55d192a286e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55664/0 (socket says 172.21.15.119:55664) 2024-04-25T10:11:41.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.876+0000 7f0533400640 1 -- 172.21.15.119:0/3349445800 learned_addr learned my addr 172.21.15.119:0/3349445800 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:41.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.876+0000 7f0533400640 1 -- 172.21.15.119:0/3349445800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d192a115c0 msgr2=0x55d1929c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:41.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.876+0000 7f0533400640 1 --2- 172.21.15.119:0/3349445800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d192a115c0 0x55d1929c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.876+0000 7f0533400640 1 -- 172.21.15.119:0/3349445800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d1929f0cd0 msgr2=0x55d192a28c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:41.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.876+0000 7f0533400640 1 --2- 172.21.15.119:0/3349445800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d1929f0cd0 0x55d192a28c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.876+0000 7f0533400640 1 -- 172.21.15.119:0/3349445800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1929f0630 con 0x55d1929ed140 2024-04-25T10:11:41.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.876+0000 7f0533400640 1 --2- 172.21.15.119:0/3349445800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929ed140 0x55d192a286e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f052402a1e0 tx=0x7f0524044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09eac85e0298528 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.877+0000 7f051f400640 1 -- 172.21.15.119:0/3349445800 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524037040 con 0x55d1929ed140 2024-04-25T10:11:41.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.877+0000 7f051f400640 1 -- 172.21.15.119:0/3349445800 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052403c660 con 0x55d1929ed140 2024-04-25T10:11:41.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.877+0000 7f051f400640 1 -- 172.21.15.119:0/3349445800 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052403cd10 con 0x55d1929ed140 2024-04-25T10:11:41.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.877+0000 7f053d41a740 1 -- 172.21.15.119:0/3349445800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929ed140 msgr2=0x55d192a286e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.877+0000 7f053d41a740 1 --2- 172.21.15.119:0/3349445800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929ed140 0x55d192a286e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f052402a1e0 tx=0x7f0524044000 comp rx=0 tx=0).stop 2024-04-25T10:11:41.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.877+0000 7f053d41a740 1 -- 172.21.15.119:0/3349445800 shutdown_connections 2024-04-25T10:11:41.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.877+0000 7f053d41a740 1 --2- 172.21.15.119:0/3349445800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d192a115c0 0x55d1929c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.877+0000 7f053d41a740 1 --2- 172.21.15.119:0/3349445800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d1929f0cd0 0x55d192a28c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.877+0000 7f053d41a740 1 --2- 172.21.15.119:0/3349445800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d1929ed140 0x55d192a286e0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.877+0000 7f053d41a740 1 -- 172.21.15.119:0/3349445800 >> 172.21.15.119:0/3349445800 conn(0x55d1929c8950 msgr2=0x55d192a1e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.877+0000 7f053d41a740 1 -- 172.21.15.119:0/3349445800 shutdown_connections 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.877+0000 7f053d41a740 1 -- 172.21.15.119:0/3349445800 wait complete. 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.878+0000 7f053d41a740 1 -- start start 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.878+0000 7f053d41a740 1 Processor -- start 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.879+0000 7f053d41a740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d1929ed140 0x55d1929c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.879+0000 7f053d41a740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d1929f0cd0 0x55d1929c9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.879+0000 7f053d41a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d192a115c0 0x55d1929ead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.879+0000 7f053d41a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1929e5770 con 0x55d192a115c0 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.879+0000 7f053d41a740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05341f47e0 con 0x55d1929f0cd0 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.879+0000 7f053d41a740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1929e5ce0 con 0x55d1929ed140 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.879+0000 7f053d41a740 10 client.0 _ll_register_callbacks cb 0x7f0524035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.880+0000 7f0533e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d192a115c0 0x55d1929ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.880+0000 7f0533e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d192a115c0 0x55d1929ead90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55676/0 (socket says 172.21.15.119:55676) 2024-04-25T10:11:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.880+0000 7f0533e00640 1 -- 172.21.15.119:0/1705350418 learned_addr learned my addr 172.21.15.119:0/1705350418 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.880+0000 7f0533e00640 1 -- 172.21.15.119:0/1705350418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d1929ed140 msgr2=0x55d1929c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.880+0000 7f0533e00640 1 --2- 172.21.15.119:0/1705350418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d1929ed140 0x55d1929c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.880+0000 7f0533e00640 1 -- 172.21.15.119:0/1705350418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d1929f0cd0 msgr2=0x55d1929c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.880+0000 7f0533e00640 1 --2- 172.21.15.119:0/1705350418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d1929f0cd0 0x55d1929c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.880+0000 7f0533e00640 1 -- 172.21.15.119:0/1705350418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d19270b6b0 con 0x55d192a115c0 2024-04-25T10:11:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.880+0000 7f0533e00640 1 --2- 172.21.15.119:0/1705350418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d192a115c0 0x55d1929ead90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f052c055680 tx=0x7f052c0592e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c1c347bef10020 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.880+0000 7f051cc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.880+0000 7f051cc00640 1 -- 172.21.15.119:0/1705350418 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c05d170 con 0x55d192a115c0 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.880+0000 7f051cc00640 1 -- 172.21.15.119:0/1705350418 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f052c0614a0 con 0x55d192a115c0 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.880+0000 7f053d41a740 1 -- 172.21.15.119:0/1705350418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f052403c660 con 0x55d192a115c0 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.881+0000 7f051cc00640 1 -- 172.21.15.119:0/1705350418 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f052c0616c0 con 0x55d192a115c0 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.881+0000 7f053d41a740 10 client.6633 fetch_fsmap learned FSMap version 44 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.881+0000 7f053d41a740 1 -- 172.21.15.119:0/1705350418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55d1929c9a60 con 0x55d192a115c0 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.881+0000 7f051cc00640 1 -- 172.21.15.119:0/1705350418 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052403c660 con 0x55d192a115c0 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.881+0000 7f051cc00640 1 -- 172.21.15.119:0/1705350418 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f052c058670 con 0x55d192a115c0 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.881+0000 7f053d41a740 10 client.6633 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.881+0000 7f053d41a740 10 client.6633 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.881+0000 7f053d41a740 20 client.6633 populate_metadata read hostname 'smithi119' 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.881+0000 7f053d41a740 1 --2- 172.21.15.119:0/1705350418 >> [v2:172.21.15.119:6836/2673349178,v1:172.21.15.119:6837/2673349178] conn(0x55d192a78f10 0x55d192a792f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.881+0000 7f053d41a740 4 client.6633 mds_command: new command op to 6483 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:11:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.881+0000 7f053d41a740 1 -- 172.21.15.119:0/1705350418 --> [v2:172.21.15.119:6836/2673349178,v1:172.21.15.119:6837/2673349178] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d1929c62d0 con 0x55d192a78f10 2024-04-25T10:11:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.881+0000 7f0533400640 1 --2- 172.21.15.119:0/1705350418 >> [v2:172.21.15.119:6836/2673349178,v1:172.21.15.119:6837/2673349178] conn(0x55d192a78f10 0x55d192a792f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f0533400640 1 --2- 172.21.15.119:0/1705350418 >> [v2:172.21.15.119:6836/2673349178,v1:172.21.15.119:6837/2673349178] conn(0x55d192a78f10 0x55d192a792f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3afa37c416de6c19 server_cookie=d8c57e8eb8dd312e in_seq=0 out_seq=0 2024-04-25T10:11:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f051cc00640 10 client.6633 ms_handle_connect on v2:172.21.15.119:6836/2673349178 2024-04-25T10:11:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f051cc00640 1 -- 172.21.15.119:0/1705350418 <== mds.0 v2:172.21.15.119:6836/2673349178 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55d1929c62d0 con 0x55d192a78f10 2024-04-25T10:11:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f051cc00640 10 client.6633 handle_command_reply: tid=0 2024-04-25T10:11:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f053d41a740 1 client.6633 shutdown 2024-04-25T10:11:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f053d41a740 1 -- 172.21.15.119:0/1705350418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d192a115c0 msgr2=0x55d1929ead90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f053d41a740 1 --2- 172.21.15.119:0/1705350418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d192a115c0 0x55d1929ead90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f052c055680 tx=0x7f052c0592e0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f053d41a740 1 -- 172.21.15.119:0/1705350418 shutdown_connections 2024-04-25T10:11:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f053d41a740 1 --2- 172.21.15.119:0/1705350418 >> [v2:172.21.15.119:6836/2673349178,v1:172.21.15.119:6837/2673349178] conn(0x55d192a78f10 0x55d192a792f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f053d41a740 1 --2- 172.21.15.119:0/1705350418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d192a115c0 0x55d1929ead90 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f053d41a740 1 --2- 172.21.15.119:0/1705350418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d1929f0cd0 0x55d1929c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f053d41a740 1 --2- 172.21.15.119:0/1705350418 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d1929ed140 0x55d1929c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f051cc00640 0 client.6633 ms_handle_reset on v2:172.21.15.119:6836/2673349178 2024-04-25T10:11:41.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f053d41a740 1 -- 172.21.15.119:0/1705350418 >> 172.21.15.119:0/1705350418 conn(0x55d1929c8950 msgr2=0x55d192a10040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:41.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f053d41a740 1 -- 172.21.15.119:0/1705350418 shutdown_connections 2024-04-25T10:11:41.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.882+0000 7f053d41a740 1 -- 172.21.15.119:0/1705350418 wait complete. 2024-04-25T10:11:41.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.883+0000 7f053d41a740 20 client.6633 trim_cache size 0 max 16384 2024-04-25T10:11:41.894 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:11:41.894 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "e0b0f428-b441-4237-a4d1-ed53321f87a5", 2024-04-25T10:11:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-04-25T10:11:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 6483, 2024-04-25T10:11:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-04-25T10:11:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-04-25T10:11:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-04-25T10:11:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.6046896049999999, 2024-04-25T10:11:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 44, 2024-04-25T10:11:41.895 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 67, 2024-04-25T10:11:41.896 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 67, 2024-04-25T10:11:41.896 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 13.081738680999999 2024-04-25T10:11:41.896 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:11:41.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.883+0000 7f053a600640 1 -- 172.21.15.119:0/3671914651 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f051406b4d0 msgr2=0x7f051406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.883+0000 7f053a600640 1 --2- 172.21.15.119:0/3671914651 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f051406b4d0 0x7f051406d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0524034000 tx=0x7f052402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.883+0000 7f053a600640 1 -- 172.21.15.119:0/3671914651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053415dbe0 msgr2=0x7f053406d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.883+0000 7f053a600640 1 --2- 172.21.15.119:0/3671914651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053415dbe0 0x7f053406d9a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f052c00cf20 tx=0x7f052c00a7d0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.883+0000 7f053a600640 1 -- 172.21.15.119:0/3671914651 shutdown_connections 2024-04-25T10:11:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.883+0000 7f053a600640 1 --2- 172.21.15.119:0/3671914651 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f051406b4d0 0x7f051406d990 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.883+0000 7f053a600640 1 --2- 172.21.15.119:0/3671914651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f053415dbe0 0x7f053406d9a0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.883+0000 7f053a600640 1 --2- 172.21.15.119:0/3671914651 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05340cfad0 0x7f05340b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.883+0000 7f053a600640 1 --2- 172.21.15.119:0/3671914651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05340aedc0 0x7f05340cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.883+0000 7f053a600640 1 -- 172.21.15.119:0/3671914651 >> 172.21.15.119:0/3671914651 conn(0x7f053406aba0 msgr2=0x7f05340719b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.884+0000 7f053a600640 1 -- 172.21.15.119:0/3671914651 shutdown_connections 2024-04-25T10:11:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:41.884+0000 7f053a600640 1 -- 172.21.15.119:0/3671914651 wait complete. 2024-04-25T10:11:41.907 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:11:42.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.083+0000 7f1da1e00640 1 Processor -- start 2024-04-25T10:11:42.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.083+0000 7f1da1e00640 1 -- start start 2024-04-25T10:11:42.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.084+0000 7f1da1e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c15acd0 0x7f1d9c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.084+0000 7f1da1e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1d9c06e860 0x7f1d9c0b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.084+0000 7f1da1e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d9c144420 con 0x7f1d9c144040 2024-04-25T10:11:42.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.084+0000 7f1da1e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9c06ec90 con 0x7f1d9c06e860 2024-04-25T10:11:42.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.084+0000 7f1da1e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0b3d80 con 0x7f1d9c15acd0 2024-04-25T10:11:42.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.085+0000 7f1da0a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1d9c144040 0x7f1d9c0cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40210/0 (socket says 172.21.15.119:40210) 2024-04-25T10:11:42.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.085+0000 7f1da0a00640 1 -- 172.21.15.119:0/2905657870 learned_addr learned my addr 172.21.15.119:0/2905657870 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:42.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.085+0000 7f1d93e00640 1 --2- 172.21.15.119:0/2905657870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c15acd0 0x7f1d9c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.085+0000 7f1da1400640 1 --2- 172.21.15.119:0/2905657870 >> v2:172.21.15.132:3300/0 conn(0x7f1d9c06e860 0x7f1d9c0b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.085+0000 7f1d9be00640 1 -- 172.21.15.119:0/2905657870 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527378382 0 0) 0x7f1d9c144420 con 0x7f1d9c144040 2024-04-25T10:11:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.085+0000 7f1d9be00640 1 -- 172.21.15.119:0/2905657870 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d80003520 con 0x7f1d9c144040 2024-04-25T10:11:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1d9be00640 1 -- 172.21.15.119:0/2905657870 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2786550580 0 0) 0x7f1d80003520 con 0x7f1d9c144040 2024-04-25T10:11:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1d9be00640 1 -- 172.21.15.119:0/2905657870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c15acd0 msgr2=0x7f1d9c06e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1d9be00640 1 --2- 172.21.15.119:0/2905657870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c15acd0 0x7f1d9c06e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1d9be00640 1 -- 172.21.15.119:0/2905657870 >> v2:172.21.15.132:3300/0 conn(0x7f1d9c06e860 msgr2=0x7f1d9c0b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1d9be00640 1 --2- 172.21.15.119:0/2905657870 >> v2:172.21.15.132:3300/0 conn(0x7f1d9c06e860 0x7f1d9c0b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1d9be00640 1 -- 172.21.15.119:0/2905657870 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c0b4060 con 0x7f1d9c144040 2024-04-25T10:11:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1d9be00640 1 -- 172.21.15.119:0/2905657870 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f1d8c001670 con 0x7f1d9c144040 2024-04-25T10:11:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1d9be00640 1 -- 172.21.15.119:0/2905657870 >> v1:172.21.15.119:6789/0 conn(0x7f1d9c144040 legacy=0x7f1d9c0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1d9be00640 1 --2- 172.21.15.119:0/2905657870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d80003a20 0x7f1d80003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1d9be00640 1 --2- 172.21.15.119:0/2905657870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c15acd0 0x7f1d80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1d9be00640 1 --2- 172.21.15.119:0/2905657870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d80004d40 0x7f1d800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1d9be00640 1 -- 172.21.15.119:0/2905657870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c06ec90 con 0x7f1d80003a20 2024-04-25T10:11:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1d9be00640 1 -- 172.21.15.119:0/2905657870 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0b3d80 con 0x7f1d80004d40 2024-04-25T10:11:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1d9be00640 1 -- 172.21.15.119:0/2905657870 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8000dc80 con 0x7f1d9c15acd0 2024-04-25T10:11:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.086+0000 7f1da0a00640 1 --2- 172.21.15.119:0/2905657870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d80003a20 0x7f1d80003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.087+0000 7f1d93e00640 1 --2- 172.21.15.119:0/2905657870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c15acd0 0x7f1d80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.087+0000 7f1da1400640 1 --2- 172.21.15.119:0/2905657870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d80004d40 0x7f1d800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.087+0000 7f1da0a00640 1 -- 172.21.15.119:0/2905657870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c15acd0 msgr2=0x7f1d80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:42.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.087+0000 7f1da0a00640 1 --2- 172.21.15.119:0/2905657870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c15acd0 0x7f1d80004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.087+0000 7f1da0a00640 1 -- 172.21.15.119:0/2905657870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d80004d40 msgr2=0x7f1d800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:42.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.087+0000 7f1da0a00640 1 --2- 172.21.15.119:0/2905657870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d80004d40 0x7f1d800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.087+0000 7f1da0a00640 1 -- 172.21.15.119:0/2905657870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d80003520 con 0x7f1d80003a20 2024-04-25T10:11:42.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.087+0000 7f1da1400640 1 --2- 172.21.15.119:0/2905657870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d80004d40 0x7f1d800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:42.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.088+0000 7f1da0a00640 1 --2- 172.21.15.119:0/2905657870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d80003a20 0x7f1d80003e40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00c290 tx=0x7f1d80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5167907406dc77a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.088+0000 7f1d9be00640 1 -- 172.21.15.119:0/2905657870 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c007d50 con 0x7f1d80003a20 2024-04-25T10:11:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.088+0000 7f1d9be00640 1 -- 172.21.15.119:0/2905657870 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c005af0 con 0x7f1d80003a20 2024-04-25T10:11:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.088+0000 7f1d9be00640 1 -- 172.21.15.119:0/2905657870 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c007070 con 0x7f1d80003a20 2024-04-25T10:11:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.089+0000 7f1da1e00640 1 -- 172.21.15.119:0/2905657870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d80003a20 msgr2=0x7f1d80003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.089+0000 7f1da1e00640 1 --2- 172.21.15.119:0/2905657870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d80003a20 0x7f1d80003e40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00c290 tx=0x7f1d80005830 comp rx=0 tx=0).stop 2024-04-25T10:11:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.089+0000 7f1da1400640 1 -- 172.21.15.119:0/2905657870 reap_dead start 2024-04-25T10:11:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.089+0000 7f1da1e00640 1 -- 172.21.15.119:0/2905657870 shutdown_connections 2024-04-25T10:11:42.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.089+0000 7f1da1e00640 1 -- 172.21.15.119:0/2905657870 >> 172.21.15.119:0/2905657870 conn(0x7f1d9c06b1d0 msgr2=0x7f1d9c0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:42.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.089+0000 7f1da1e00640 1 -- 172.21.15.119:0/2905657870 shutdown_connections 2024-04-25T10:11:42.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.089+0000 7f1da1e00640 1 -- 172.21.15.119:0/2905657870 wait complete. 2024-04-25T10:11:42.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.091+0000 7f1da1e00640 1 Processor -- start 2024-04-25T10:11:42.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.091+0000 7f1da1e00640 1 -- start start 2024-04-25T10:11:42.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.092+0000 7f1da1e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d80003a20 0x7f1d9c13d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.092+0000 7f1da1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c13ca80 0x7f1d9c148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.092+0000 7f1da1e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c13f770 0x7f1d9c1f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.092+0000 7f1da1e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0b4900 con 0x7f1d9c13ca80 2024-04-25T10:11:42.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.092+0000 7f1da1e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0b4620 con 0x7f1d80003a20 2024-04-25T10:11:42.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.092+0000 7f1da1e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0b5180 con 0x7f1d9c13f770 2024-04-25T10:11:42.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.092+0000 7f1d93e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c13ca80 0x7f1d9c148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.092+0000 7f1d93e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c13ca80 0x7f1d9c148780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55708/0 (socket says 172.21.15.119:55708) 2024-04-25T10:11:42.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.092+0000 7f1d93e00640 1 -- 172.21.15.119:0/3684661340 learned_addr learned my addr 172.21.15.119:0/3684661340 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:42.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.093+0000 7f1d93e00640 1 -- 172.21.15.119:0/3684661340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c13f770 msgr2=0x7f1d9c1f6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.093+0000 7f1da1400640 1 --2- 172.21.15.119:0/3684661340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c13f770 0x7f1d9c1f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.093+0000 7f1da0a00640 1 --2- 172.21.15.119:0/3684661340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d80003a20 0x7f1d9c13d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.093+0000 7f1d93e00640 1 --2- 172.21.15.119:0/3684661340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c13f770 0x7f1d9c1f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.093+0000 7f1d93e00640 1 -- 172.21.15.119:0/3684661340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d80003a20 msgr2=0x7f1d9c13d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.093+0000 7f1d93e00640 1 --2- 172.21.15.119:0/3684661340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d80003a20 0x7f1d9c13d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.093+0000 7f1d93e00640 1 -- 172.21.15.119:0/3684661340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c148cc0 con 0x7f1d9c13ca80 2024-04-25T10:11:42.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.093+0000 7f1d93e00640 1 --2- 172.21.15.119:0/3684661340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c13ca80 0x7f1d9c148780 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1d84010890 tx=0x7f1d84010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:42.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.093+0000 7f1d99600640 1 -- 172.21.15.119:0/3684661340 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8400f040 con 0x7f1d9c13ca80 2024-04-25T10:11:42.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.093+0000 7f1d99600640 1 -- 172.21.15.119:0/3684661340 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8400eb60 con 0x7f1d9c13ca80 2024-04-25T10:11:42.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.093+0000 7f1d99600640 1 -- 172.21.15.119:0/3684661340 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8401c400 con 0x7f1d9c13ca80 2024-04-25T10:11:42.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.093+0000 7f1da1e00640 1 -- 172.21.15.119:0/3684661340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9c140070 con 0x7f1d9c13ca80 2024-04-25T10:11:42.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.093+0000 7f1da1e00640 1 -- 172.21.15.119:0/3684661340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c1e68d0 con 0x7f1d9c13ca80 2024-04-25T10:11:42.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.094+0000 7f1da1e00640 1 -- 172.21.15.119:0/3684661340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9c06bb30 con 0x7f1d9c13ca80 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.095+0000 7f1d99600640 1 -- 172.21.15.119:0/3684661340 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d8401c5a0 con 0x7f1d9c13ca80 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.096+0000 7f1d99600640 1 --2- 172.21.15.119:0/3684661340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d7806b4d0 0x7f1d7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.096+0000 7f1d99600640 1 -- 172.21.15.119:0/3684661340 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d8408a830 con 0x7f1d9c13ca80 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.097+0000 7f1da0a00640 1 --2- 172.21.15.119:0/3684661340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d7806b4d0 0x7f1d7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:42.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.098+0000 7f1da0a00640 1 --2- 172.21.15.119:0/3684661340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d7806b4d0 0x7f1d7806d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c033000 tx=0x7f1d8c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:42.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.101+0000 7f1d99600640 1 -- 172.21.15.119:0/3684661340 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d84018030 con 0x7f1d9c13ca80 2024-04-25T10:11:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.336+0000 7f1da1e00640 1 -- 172.21.15.119:0/3684661340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1d9c0749b0 con 0x7f1d9c13ca80 2024-04-25T10:11:42.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.337+0000 7f1d99600640 1 -- 172.21.15.119:0/3684661340 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1d84059050 con 0x7f1d9c13ca80 2024-04-25T10:11:42.342 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:11:42.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.339+0000 7f1da1e00640 1 -- 172.21.15.119:0/3684661340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d7806b4d0 msgr2=0x7f1d7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.339+0000 7f1da1e00640 1 --2- 172.21.15.119:0/3684661340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d7806b4d0 0x7f1d7806d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c033000 tx=0x7f1d8c009030 comp rx=0 tx=0).stop 2024-04-25T10:11:42.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.339+0000 7f1da1e00640 1 -- 172.21.15.119:0/3684661340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c13ca80 msgr2=0x7f1d9c148780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:42.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.339+0000 7f1da1e00640 1 --2- 172.21.15.119:0/3684661340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c13ca80 0x7f1d9c148780 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1d84010890 tx=0x7f1d84010d50 comp rx=0 tx=0).stop 2024-04-25T10:11:42.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.339+0000 7f1da1e00640 1 -- 172.21.15.119:0/3684661340 shutdown_connections 2024-04-25T10:11:42.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.339+0000 7f1da1e00640 1 --2- 172.21.15.119:0/3684661340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d7806b4d0 0x7f1d7806d990 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.339+0000 7f1da1e00640 1 --2- 172.21.15.119:0/3684661340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1d9c13f770 0x7f1d9c1f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.339+0000 7f1da1e00640 1 --2- 172.21.15.119:0/3684661340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c13ca80 0x7f1d9c148780 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.339+0000 7f1da1e00640 1 --2- 172.21.15.119:0/3684661340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d80003a20 0x7f1d9c13d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:42.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.339+0000 7f1da1e00640 1 -- 172.21.15.119:0/3684661340 >> 172.21.15.119:0/3684661340 conn(0x7f1d9c06b1d0 msgr2=0x7f1d9c0ad660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:42.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.339+0000 7f1da1e00640 1 -- 172.21.15.119:0/3684661340 shutdown_connections 2024-04-25T10:11:42.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:42.339+0000 7f1da1e00640 1 -- 172.21.15.119:0/3684661340 wait complete. 2024-04-25T10:11:42.360 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:11:42.361 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:11:42.361 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:11:42.361 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:11:42.361 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:42.361 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:11:42.361 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:11:42.361 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:11:42.361 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:42.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:42.361 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:11:42.411 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:11:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:11:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:11:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:11:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:42.412 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:11:42.412 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:42.412 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:11:42.413 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:42.413 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:42.414 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:11:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:11:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42387sec preferred_lft 42387sec 2024-04-25T10:11:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:11:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:42.414 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:11:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:11:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:42.414 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:11:42.415 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:42.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:42.416 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:11:42.485 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:11:42.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:42.489 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:11:42.539 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:11:42.539 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:11:42.539 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:11:42.539 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:11:42.539 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:11:42.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:42.540 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:11:42.540 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:11:42.540 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:11:42.540 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:11:42.540 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:11:42.540 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:11:42.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:42.685 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:11:42.735 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:11:42.735 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:11:42.736 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:42.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:42.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:11:42.814 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:42.814 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:42.857 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:42.857 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:42.858 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:42.949 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:11:42.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:42.949 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:43.075 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:43.076 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:43.076 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:11:43.098 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:43.098 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:43.098 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:11:43.098 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:11:43.098 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:11:43.122 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:43.122 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:43.122 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:11:43.122 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:11:43.123 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:11:43.123 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:11:43.123 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:11:43.123 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:11:43.124 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:11:43.125 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:11:43.125 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:11:43.125 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:11:43.125 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:11:43.128 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:11:43.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:43.128 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:43.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:43.196 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:11:43.256 INFO:teuthology.orchestra.run.smithi119.stdout:client5419 2024-04-25T10:11:43.257 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:11:43.257 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:43.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:43.310 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-25T10:11:43.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:43.367 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-25T10:11:43.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:43.418 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-25T10:11:43.465 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:11:43.465 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:11:43.465 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:11:43.465 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:11:43.466 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-25T10:11:43.466 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:11:43.651 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627776 2024-04-25T10:11:43.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:43.664 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:11:43.664 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:43.762 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:11:43.762 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:43.813 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:44.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.012+0000 7fcc96400640 1 Processor -- start 2024-04-25T10:11:44.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.012+0000 7fcc96400640 1 -- start start 2024-04-25T10:11:44.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.014+0000 7fcc96400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc9015acd0 0x7fcc9006e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.014+0000 7fcc96400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc9006e860 0x7fcc900b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.014+0000 7fcc96400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc90144420 con 0x7fcc90144040 2024-04-25T10:11:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.014+0000 7fcc96400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc9006ec90 con 0x7fcc9015acd0 2024-04-25T10:11:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.014+0000 7fcc96400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900b3d80 con 0x7fcc9006e860 2024-04-25T10:11:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.014+0000 7fcc8fe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc9015acd0 0x7fcc9006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.014+0000 7fcc95000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcc90144040 0x7fcc900cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40212/0 (socket says 172.21.15.119:40212) 2024-04-25T10:11:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.014+0000 7fcc95000640 1 -- 172.21.15.119:0/2974225832 learned_addr learned my addr 172.21.15.119:0/2974225832 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.015+0000 7fcc8fe00640 1 -- 172.21.15.119:0/2974225832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc9006e860 msgr2=0x7fcc900b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.015+0000 7fcc8fe00640 1 --2- 172.21.15.119:0/2974225832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc9006e860 0x7fcc900b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.015+0000 7fcc8fe00640 1 -- 172.21.15.119:0/2974225832 >> v1:172.21.15.119:6789/0 conn(0x7fcc90144040 legacy=0x7fcc900cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.015+0000 7fcc8fe00640 1 -- 172.21.15.119:0/2974225832 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc900b4060 con 0x7fcc9015acd0 2024-04-25T10:11:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.016+0000 7fcc8fe00640 1 --2- 172.21.15.119:0/2974225832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9015acd0 0x7fcc9006e320 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00ba40 tx=0x7fcc7c02c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f57bb3b542df047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.016+0000 7fcc8f400640 1 -- 172.21.15.119:0/2974225832 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c031710 con 0x7fcc9015acd0 2024-04-25T10:11:44.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.016+0000 7fcc8f400640 1 -- 172.21.15.119:0/2974225832 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c0318b0 con 0x7fcc9015acd0 2024-04-25T10:11:44.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.016+0000 7fcc8f400640 1 -- 172.21.15.119:0/2974225832 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c031c80 con 0x7fcc9015acd0 2024-04-25T10:11:44.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.017+0000 7fcc96400640 1 -- 172.21.15.119:0/2974225832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9015acd0 msgr2=0x7fcc9006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.017+0000 7fcc96400640 1 --2- 172.21.15.119:0/2974225832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9015acd0 0x7fcc9006e320 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00ba40 tx=0x7fcc7c02c7b0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.017+0000 7fcc96400640 1 -- 172.21.15.119:0/2974225832 shutdown_connections 2024-04-25T10:11:44.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.017+0000 7fcc96400640 1 --2- 172.21.15.119:0/2974225832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc9006e860 0x7fcc900b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.017+0000 7fcc96400640 1 --2- 172.21.15.119:0/2974225832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9015acd0 0x7fcc9006e320 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.017+0000 7fcc96400640 1 -- 172.21.15.119:0/2974225832 >> 172.21.15.119:0/2974225832 conn(0x7fcc9006b1d0 msgr2=0x7fcc900b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.017+0000 7fcc96400640 1 -- 172.21.15.119:0/2974225832 shutdown_connections 2024-04-25T10:11:44.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.017+0000 7fcc96400640 1 -- 172.21.15.119:0/2974225832 wait complete. 2024-04-25T10:11:44.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.019+0000 7fcc96400640 1 Processor -- start 2024-04-25T10:11:44.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.019+0000 7fcc96400640 1 -- start start 2024-04-25T10:11:44.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.020+0000 7fcc96400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc9006e860 0x7fcc901ee740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.020+0000 7fcc96400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc90144040 0x7fcc9015cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.020+0000 7fcc96400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9015acd0 0x7fcc90142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.020+0000 7fcc96400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900b4900 con 0x7fcc9015acd0 2024-04-25T10:11:44.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.020+0000 7fcc96400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900b4620 con 0x7fcc90144040 2024-04-25T10:11:44.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.020+0000 7fcc96400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900b5180 con 0x7fcc9006e860 2024-04-25T10:11:44.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.020+0000 7fcc95a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9015acd0 0x7fcc90142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.020+0000 7fcc95a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9015acd0 0x7fcc90142310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55716/0 (socket says 172.21.15.119:55716) 2024-04-25T10:11:44.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.020+0000 7fcc95a00640 1 -- 172.21.15.119:0/3201202885 learned_addr learned my addr 172.21.15.119:0/3201202885 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:44.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.020+0000 7fcc95a00640 1 -- 172.21.15.119:0/3201202885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc9006e860 msgr2=0x7fcc901ee740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.020+0000 7fcc95a00640 1 --2- 172.21.15.119:0/3201202885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc9006e860 0x7fcc901ee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.020+0000 7fcc95a00640 1 -- 172.21.15.119:0/3201202885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc90144040 msgr2=0x7fcc9015cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.020+0000 7fcc95a00640 1 --2- 172.21.15.119:0/3201202885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc90144040 0x7fcc9015cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.020+0000 7fcc95a00640 1 -- 172.21.15.119:0/3201202885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc90136db0 con 0x7fcc9015acd0 2024-04-25T10:11:44.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.021+0000 7fcc95a00640 1 --2- 172.21.15.119:0/3201202885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9015acd0 0x7fcc90142310 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800b970 tx=0x7fcc8800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.021+0000 7fcc8cc00640 1 -- 172.21.15.119:0/3201202885 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88019070 con 0x7fcc9015acd0 2024-04-25T10:11:44.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.021+0000 7fcc8cc00640 1 -- 172.21.15.119:0/3201202885 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8800cb60 con 0x7fcc9015acd0 2024-04-25T10:11:44.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.021+0000 7fcc8cc00640 1 -- 172.21.15.119:0/3201202885 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88014440 con 0x7fcc9015acd0 2024-04-25T10:11:44.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.021+0000 7fcc96400640 1 -- 172.21.15.119:0/3201202885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc90137090 con 0x7fcc9015acd0 2024-04-25T10:11:44.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.021+0000 7fcc96400640 1 -- 172.21.15.119:0/3201202885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc90137650 con 0x7fcc9015acd0 2024-04-25T10:11:44.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.022+0000 7fcc96400640 1 -- 172.21.15.119:0/3201202885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc9006bab0 con 0x7fcc9015acd0 2024-04-25T10:11:44.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.023+0000 7fcc8cc00640 1 -- 172.21.15.119:0/3201202885 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc8801c070 con 0x7fcc9015acd0 2024-04-25T10:11:44.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.025+0000 7fcc8cc00640 1 --2- 172.21.15.119:0/3201202885 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcc7006b4d0 0x7fcc7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.025+0000 7fcc8cc00640 1 -- 172.21.15.119:0/3201202885 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc88089fe0 con 0x7fcc9015acd0 2024-04-25T10:11:44.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.025+0000 7fcc95000640 1 --2- 172.21.15.119:0/3201202885 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcc7006b4d0 0x7fcc7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.026+0000 7fcc95000640 1 --2- 172.21.15.119:0/3201202885 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcc7006b4d0 0x7fcc7006d990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcc84000f60 tx=0x7fcc84007a80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.029+0000 7fcc8cc00640 1 -- 172.21.15.119:0/3201202885 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc88058dc0 con 0x7fcc9015acd0 2024-04-25T10:11:44.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.224+0000 7fcc96400640 1 -- 172.21.15.119:0/3201202885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc90137890 con 0x7fcc9015acd0 2024-04-25T10:11:44.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.226+0000 7fcc8cc00640 1 -- 172.21.15.119:0/3201202885 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16704 (secure 0 0 0) 0x7fcc880586c0 con 0x7fcc9015acd0 2024-04-25T10:11:44.232 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:44.232 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":67,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:11:36.274669+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:32.500199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:33.708192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.229+0000 7fcc96400640 1 -- 172.21.15.119:0/3201202885 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcc7006b4d0 msgr2=0x7fcc7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.229+0000 7fcc96400640 1 --2- 172.21.15.119:0/3201202885 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcc7006b4d0 0x7fcc7006d990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcc84000f60 tx=0x7fcc84007a80 comp rx=0 tx=0).stop 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.229+0000 7fcc96400640 1 -- 172.21.15.119:0/3201202885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9015acd0 msgr2=0x7fcc90142310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.229+0000 7fcc96400640 1 --2- 172.21.15.119:0/3201202885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9015acd0 0x7fcc90142310 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800b970 tx=0x7fcc8800be30 comp rx=0 tx=0).stop 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.229+0000 7fcc96400640 1 -- 172.21.15.119:0/3201202885 shutdown_connections 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.229+0000 7fcc96400640 1 --2- 172.21.15.119:0/3201202885 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcc7006b4d0 0x7fcc7006d990 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.229+0000 7fcc96400640 1 --2- 172.21.15.119:0/3201202885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc9015acd0 0x7fcc90142310 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.229+0000 7fcc96400640 1 --2- 172.21.15.119:0/3201202885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc90144040 0x7fcc9015cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.229+0000 7fcc96400640 1 --2- 172.21.15.119:0/3201202885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcc9006e860 0x7fcc901ee740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.229+0000 7fcc96400640 1 -- 172.21.15.119:0/3201202885 >> 172.21.15.119:0/3201202885 conn(0x7fcc9006b1d0 msgr2=0x7fcc90140ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.230+0000 7fcc96400640 1 -- 172.21.15.119:0/3201202885 shutdown_connections 2024-04-25T10:11:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.230+0000 7fcc96400640 1 -- 172.21.15.119:0/3201202885 wait complete. 2024-04-25T10:11:44.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:32.500199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:33.708192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:44.254 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.453+0000 7fab1b200640 1 Processor -- start 2024-04-25T10:11:44.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.454+0000 7fab1b200640 1 -- start start 2024-04-25T10:11:44.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.455+0000 7fab1b200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fab141450e0 0x7fab1406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.455+0000 7fab1b200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab1406b0e0 0x7fab140cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.456+0000 7fab1b200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1415f400 con 0x7fab140cc810 2024-04-25T10:11:44.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.456+0000 7fab1b200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab141454c0 con 0x7fab1406b0e0 2024-04-25T10:11:44.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.456+0000 7fab1b200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1406b540 con 0x7fab141450e0 2024-04-25T10:11:44.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.456+0000 7fab1a800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fab140cc810 0x7fab14071050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40226/0 (socket says 172.21.15.119:40226) 2024-04-25T10:11:44.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.456+0000 7fab1a800640 1 -- 172.21.15.119:0/692246231 learned_addr learned my addr 172.21.15.119:0/692246231 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:44.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.456+0000 7fab19400640 1 --2- 172.21.15.119:0/692246231 >> v2:172.21.15.119:3300/0 conn(0x7fab1406b0e0 0x7fab140cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.456+0000 7fab19e00640 1 --2- 172.21.15.119:0/692246231 >> v2:172.21.15.132:3300/0 conn(0x7fab141450e0 0x7fab1406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.457+0000 7fab19400640 1 -- 172.21.15.119:0/692246231 >> v2:172.21.15.132:3300/0 conn(0x7fab141450e0 msgr2=0x7fab1406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.457+0000 7fab19400640 1 --2- 172.21.15.119:0/692246231 >> v2:172.21.15.132:3300/0 conn(0x7fab141450e0 0x7fab1406aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.456+0000 7fab18a00640 1 -- 172.21.15.119:0/692246231 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2671222828 0 0) 0x7fab1415f400 con 0x7fab140cc810 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.457+0000 7fab19400640 1 -- 172.21.15.119:0/692246231 >> v1:172.21.15.119:6789/0 conn(0x7fab140cc810 legacy=0x7fab14071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.457+0000 7fab19400640 1 -- 172.21.15.119:0/692246231 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1415f6e0 con 0x7fab1406b0e0 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.458+0000 7fab19400640 1 --2- 172.21.15.119:0/692246231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab1406b0e0 0x7fab140cc2d0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fab14072170 tx=0x7fab0402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5096c96274df2d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.458+0000 7fab18a00640 1 -- 172.21.15.119:0/692246231 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04037070 con 0x7fab1406b0e0 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.458+0000 7fab18a00640 1 -- 172.21.15.119:0/692246231 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0402add0 con 0x7fab1406b0e0 2024-04-25T10:11:44.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.458+0000 7fab18a00640 1 -- 172.21.15.119:0/692246231 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04032660 con 0x7fab1406b0e0 2024-04-25T10:11:44.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.458+0000 7fab1b200640 1 -- 172.21.15.119:0/692246231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab1406b0e0 msgr2=0x7fab140cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.458+0000 7fab1b200640 1 --2- 172.21.15.119:0/692246231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab1406b0e0 0x7fab140cc2d0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fab14072170 tx=0x7fab0402a080 comp rx=0 tx=0).stop 2024-04-25T10:11:44.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.459+0000 7fab1b200640 1 -- 172.21.15.119:0/692246231 shutdown_connections 2024-04-25T10:11:44.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.459+0000 7fab1b200640 1 --2- 172.21.15.119:0/692246231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab1406b0e0 0x7fab140cc2d0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.459+0000 7fab1b200640 1 --2- 172.21.15.119:0/692246231 >> v2:172.21.15.132:3300/0 conn(0x7fab141450e0 0x7fab1406aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.459+0000 7fab1b200640 1 -- 172.21.15.119:0/692246231 >> 172.21.15.119:0/692246231 conn(0x7fab14066ff0 msgr2=0x7fab140b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.459+0000 7fab1b200640 1 -- 172.21.15.119:0/692246231 shutdown_connections 2024-04-25T10:11:44.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.459+0000 7fab1b200640 1 -- 172.21.15.119:0/692246231 wait complete. 2024-04-25T10:11:44.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.460+0000 7fab1b200640 1 Processor -- start 2024-04-25T10:11:44.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.461+0000 7fab1b200640 1 -- start start 2024-04-25T10:11:44.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.461+0000 7fab1b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab1406b0e0 0x7fab140adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.461+0000 7fab1b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab140cc810 0x7fab1415b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.461+0000 7fab1b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab141450e0 0x7fab140c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.462+0000 7fab1b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab140ccc30 con 0x7fab141450e0 2024-04-25T10:11:44.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.462+0000 7fab1b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1415fca0 con 0x7fab140cc810 2024-04-25T10:11:44.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.462+0000 7fab1b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab140ccdb0 con 0x7fab1406b0e0 2024-04-25T10:11:44.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.462+0000 7fab1a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab141450e0 0x7fab140c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.462+0000 7fab1a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab141450e0 0x7fab140c4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55740/0 (socket says 172.21.15.119:55740) 2024-04-25T10:11:44.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.462+0000 7fab1a800640 1 -- 172.21.15.119:0/1936690430 learned_addr learned my addr 172.21.15.119:0/1936690430 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:44.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.462+0000 7fab19e00640 1 --2- 172.21.15.119:0/1936690430 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab1406b0e0 0x7fab140adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.462+0000 7fab19400640 1 --2- 172.21.15.119:0/1936690430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab140cc810 0x7fab1415b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.462+0000 7fab1a800640 1 -- 172.21.15.119:0/1936690430 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab1406b0e0 msgr2=0x7fab140adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.462+0000 7fab1a800640 1 --2- 172.21.15.119:0/1936690430 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab1406b0e0 0x7fab140adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.462+0000 7fab1a800640 1 -- 172.21.15.119:0/1936690430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab140cc810 msgr2=0x7fab1415b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.462+0000 7fab1a800640 1 --2- 172.21.15.119:0/1936690430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab140cc810 0x7fab1415b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.462+0000 7fab1a800640 1 -- 172.21.15.119:0/1936690430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab140ae3e0 con 0x7fab141450e0 2024-04-25T10:11:44.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.463+0000 7fab1a800640 1 --2- 172.21.15.119:0/1936690430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab141450e0 0x7fab140c4610 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c7d0 tx=0x7fab1000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.463+0000 7fab0e000640 1 -- 172.21.15.119:0/1936690430 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10018070 con 0x7fab141450e0 2024-04-25T10:11:44.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.463+0000 7fab0e000640 1 -- 172.21.15.119:0/1936690430 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1000a640 con 0x7fab141450e0 2024-04-25T10:11:44.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.463+0000 7fab0e000640 1 -- 172.21.15.119:0/1936690430 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1000aa70 con 0x7fab141450e0 2024-04-25T10:11:44.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.463+0000 7fab1b200640 1 -- 172.21.15.119:0/1936690430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab140ace00 con 0x7fab141450e0 2024-04-25T10:11:44.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.463+0000 7fab1b200640 1 -- 172.21.15.119:0/1936690430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab140ad3c0 con 0x7fab141450e0 2024-04-25T10:11:44.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.464+0000 7fab1b200640 1 -- 172.21.15.119:0/1936690430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab140749b0 con 0x7fab141450e0 2024-04-25T10:11:44.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.465+0000 7fab0e000640 1 -- 172.21.15.119:0/1936690430 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab1001b070 con 0x7fab141450e0 2024-04-25T10:11:44.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.466+0000 7fab0e000640 1 --2- 172.21.15.119:0/1936690430 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faaf806b4d0 0x7faaf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.466+0000 7fab0e000640 1 -- 172.21.15.119:0/1936690430 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab10089880 con 0x7fab141450e0 2024-04-25T10:11:44.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.466+0000 7fab19e00640 1 --2- 172.21.15.119:0/1936690430 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faaf806b4d0 0x7faaf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.467+0000 7fab19e00640 1 --2- 172.21.15.119:0/1936690430 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faaf806b4d0 0x7faaf806d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fab08006fd0 tx=0x7fab08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.470+0000 7fab0e000640 1 -- 172.21.15.119:0/1936690430 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab10058660 con 0x7fab141450e0 2024-04-25T10:11:44.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.665+0000 7fab1b200640 1 -- 172.21.15.119:0/1936690430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab14141fb0 con 0x7fab141450e0 2024-04-25T10:11:44.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.666+0000 7fab0e000640 1 -- 172.21.15.119:0/1936690430 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab10057f60 con 0x7fab141450e0 2024-04-25T10:11:44.672 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:11:44.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.669+0000 7fab1b200640 1 -- 172.21.15.119:0/1936690430 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faaf806b4d0 msgr2=0x7faaf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.669+0000 7fab1b200640 1 --2- 172.21.15.119:0/1936690430 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faaf806b4d0 0x7faaf806d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fab08006fd0 tx=0x7fab08008040 comp rx=0 tx=0).stop 2024-04-25T10:11:44.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.669+0000 7fab1b200640 1 -- 172.21.15.119:0/1936690430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab141450e0 msgr2=0x7fab140c4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.669+0000 7fab1b200640 1 --2- 172.21.15.119:0/1936690430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab141450e0 0x7fab140c4610 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c7d0 tx=0x7fab1000cc90 comp rx=0 tx=0).stop 2024-04-25T10:11:44.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.669+0000 7fab1b200640 1 -- 172.21.15.119:0/1936690430 shutdown_connections 2024-04-25T10:11:44.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.669+0000 7fab1b200640 1 --2- 172.21.15.119:0/1936690430 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faaf806b4d0 0x7faaf806d990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.669+0000 7fab1b200640 1 --2- 172.21.15.119:0/1936690430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab141450e0 0x7fab140c4610 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.669+0000 7fab1b200640 1 --2- 172.21.15.119:0/1936690430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab140cc810 0x7fab1415b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.669+0000 7fab1b200640 1 --2- 172.21.15.119:0/1936690430 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab1406b0e0 0x7fab140adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.669+0000 7fab1b200640 1 -- 172.21.15.119:0/1936690430 >> 172.21.15.119:0/1936690430 conn(0x7fab14066ff0 msgr2=0x7fab140b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.669+0000 7fab1b200640 1 -- 172.21.15.119:0/1936690430 shutdown_connections 2024-04-25T10:11:44.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.669+0000 7fab1b200640 1 -- 172.21.15.119:0/1936690430 wait complete. 2024-04-25T10:11:44.693 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:44.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.888+0000 7f6c5c800640 1 Processor -- start 2024-04-25T10:11:44.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.889+0000 7f6c5c800640 1 -- start start 2024-04-25T10:11:44.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.890+0000 7f6c5c800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6c5415b7b0 0x7f6c540ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.890+0000 7f6c5c800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c540cf760 0x7f6c540cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.890+0000 7f6c5c800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5415f000 con 0x7f6c5415e110 2024-04-25T10:11:44.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.890+0000 7f6c5c800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c540d0100 con 0x7f6c5415b7b0 2024-04-25T10:11:44.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.890+0000 7f6c5c800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540ced00 con 0x7f6c540cf760 2024-04-25T10:11:44.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.891+0000 7f6c5a800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6c5415e110 0x7f6c540cc080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40242/0 (socket says 172.21.15.119:40242) 2024-04-25T10:11:44.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.891+0000 7f6c5a800640 1 -- 172.21.15.119:0/1045577719 learned_addr learned my addr 172.21.15.119:0/1045577719 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:44.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.891+0000 7f6c59e00640 1 --2- 172.21.15.119:0/1045577719 >> v2:172.21.15.132:3300/0 conn(0x7f6c5415b7b0 0x7f6c540ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.891+0000 7f6c59400640 1 --2- 172.21.15.119:0/1045577719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c540cf760 0x7f6c540cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.891+0000 7f6c58a00640 1 -- 172.21.15.119:0/1045577719 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798908606 0 0) 0x7f6c5415f000 con 0x7f6c5415e110 2024-04-25T10:11:44.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.891+0000 7f6c58a00640 1 -- 172.21.15.119:0/1045577719 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c3c003520 con 0x7f6c5415e110 2024-04-25T10:11:44.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c58a00640 1 -- 172.21.15.119:0/1045577719 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4034155978 0 0) 0x7f6c3c003520 con 0x7f6c5415e110 2024-04-25T10:11:44.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c58a00640 1 -- 172.21.15.119:0/1045577719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c540cf760 msgr2=0x7f6c540cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c58a00640 1 --2- 172.21.15.119:0/1045577719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c540cf760 0x7f6c540cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c58a00640 1 -- 172.21.15.119:0/1045577719 >> v2:172.21.15.132:3300/0 conn(0x7f6c5415b7b0 msgr2=0x7f6c540ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c58a00640 1 --2- 172.21.15.119:0/1045577719 >> v2:172.21.15.132:3300/0 conn(0x7f6c5415b7b0 0x7f6c540ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c58a00640 1 -- 172.21.15.119:0/1045577719 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5415ed20 con 0x7f6c5415e110 2024-04-25T10:11:44.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c58a00640 1 -- 172.21.15.119:0/1045577719 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f6c500019f0 con 0x7f6c5415e110 2024-04-25T10:11:44.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c58a00640 1 -- 172.21.15.119:0/1045577719 >> v1:172.21.15.119:6789/0 conn(0x7f6c5415e110 legacy=0x7f6c540cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c58a00640 1 --2- 172.21.15.119:0/1045577719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c3c0039b0 0x7f6c3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c58a00640 1 --2- 172.21.15.119:0/1045577719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c3c004800 0x7f6c3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c58a00640 1 --2- 172.21.15.119:0/1045577719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c540cf760 0x7f6c3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c58a00640 1 -- 172.21.15.119:0/1045577719 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540ced00 con 0x7f6c3c004800 2024-04-25T10:11:44.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c58a00640 1 -- 172.21.15.119:0/1045577719 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c00b9c0 con 0x7f6c3c0039b0 2024-04-25T10:11:44.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c58a00640 1 -- 172.21.15.119:0/1045577719 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c00bb40 con 0x7f6c540cf760 2024-04-25T10:11:44.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.892+0000 7f6c59400640 1 --2- 172.21.15.119:0/1045577719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c3c004800 0x7f6c3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.893+0000 7f6c59400640 1 -- 172.21.15.119:0/1045577719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c540cf760 msgr2=0x7f6c3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.893+0000 7f6c59400640 1 --2- 172.21.15.119:0/1045577719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c540cf760 0x7f6c3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.893+0000 7f6c59400640 1 -- 172.21.15.119:0/1045577719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c3c0039b0 msgr2=0x7f6c3c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:44.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.893+0000 7f6c59400640 1 --2- 172.21.15.119:0/1045577719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c3c0039b0 0x7f6c3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.893+0000 7f6c59400640 1 -- 172.21.15.119:0/1045577719 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c003520 con 0x7f6c3c004800 2024-04-25T10:11:44.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.894+0000 7f6c59400640 1 --2- 172.21.15.119:0/1045577719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c3c004800 0x7f6c3c004c80 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009ac0 tx=0x7f6c3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428329b658437b6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.894+0000 7f6c58a00640 1 -- 172.21.15.119:0/1045577719 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c44036070 con 0x7f6c3c004800 2024-04-25T10:11:44.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.894+0000 7f6c58a00640 1 -- 172.21.15.119:0/1045577719 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c44031400 con 0x7f6c3c004800 2024-04-25T10:11:44.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.894+0000 7f6c58a00640 1 -- 172.21.15.119:0/1045577719 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c44009690 con 0x7f6c3c004800 2024-04-25T10:11:44.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.895+0000 7f6c5c800640 1 -- 172.21.15.119:0/1045577719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c3c004800 msgr2=0x7f6c3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.895+0000 7f6c5c800640 1 --2- 172.21.15.119:0/1045577719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c3c004800 0x7f6c3c004c80 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009ac0 tx=0x7f6c3c005830 comp rx=0 tx=0).stop 2024-04-25T10:11:44.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.895+0000 7f6c5a800640 1 -- 172.21.15.119:0/1045577719 reap_dead start 2024-04-25T10:11:44.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.895+0000 7f6c5c800640 1 -- 172.21.15.119:0/1045577719 shutdown_connections 2024-04-25T10:11:44.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.895+0000 7f6c5c800640 1 -- 172.21.15.119:0/1045577719 >> 172.21.15.119:0/1045577719 conn(0x7f6c5406f4a0 msgr2=0x7f6c5406f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:44.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.895+0000 7f6c5c800640 1 -- 172.21.15.119:0/1045577719 shutdown_connections 2024-04-25T10:11:44.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.895+0000 7f6c5c800640 1 -- 172.21.15.119:0/1045577719 wait complete. 2024-04-25T10:11:44.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.897+0000 7f6c5c800640 1 Processor -- start 2024-04-25T10:11:44.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.897+0000 7f6c5c800640 1 -- start start 2024-04-25T10:11:44.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.898+0000 7f6c5c800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c3c004800 0x7f6c540c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.898+0000 7f6c5c800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c540b1e20 0x7f6c541589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.898+0000 7f6c5c800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c540b4470 0x7f6c540cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.898+0000 7f6c5c800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540cdfc0 con 0x7f6c3c004800 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.898+0000 7f6c5c800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540ce330 con 0x7f6c540b1e20 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.898+0000 7f6c5c800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541619f0 con 0x7f6c540b4470 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.898+0000 7f6c59e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c3c004800 0x7f6c540c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.898+0000 7f6c59e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c3c004800 0x7f6c540c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55754/0 (socket says 172.21.15.119:55754) 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.898+0000 7f6c59e00640 1 -- 172.21.15.119:0/906396426 learned_addr learned my addr 172.21.15.119:0/906396426 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.899+0000 7f6c5a800640 1 --2- 172.21.15.119:0/906396426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c540b4470 0x7f6c540cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.899+0000 7f6c59e00640 1 -- 172.21.15.119:0/906396426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c540b4470 msgr2=0x7f6c540cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.899+0000 7f6c59e00640 1 --2- 172.21.15.119:0/906396426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c540b4470 0x7f6c540cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.899+0000 7f6c59e00640 1 -- 172.21.15.119:0/906396426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c540b1e20 msgr2=0x7f6c541589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.899+0000 7f6c59e00640 1 --2- 172.21.15.119:0/906396426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c540b1e20 0x7f6c541589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.899+0000 7f6c59e00640 1 -- 172.21.15.119:0/906396426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c54158ef0 con 0x7f6c3c004800 2024-04-25T10:11:44.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.899+0000 7f6c59e00640 1 --2- 172.21.15.119:0/906396426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c3c004800 0x7f6c540c9540 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e860 tx=0x7f6c4800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.899+0000 7f6c4e000640 1 -- 172.21.15.119:0/906396426 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48017070 con 0x7f6c3c004800 2024-04-25T10:11:44.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.899+0000 7f6c4e000640 1 -- 172.21.15.119:0/906396426 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4800c940 con 0x7f6c3c004800 2024-04-25T10:11:44.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.899+0000 7f6c5c800640 1 -- 172.21.15.119:0/906396426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c540b2600 con 0x7f6c3c004800 2024-04-25T10:11:44.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.899+0000 7f6c4e000640 1 -- 172.21.15.119:0/906396426 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4800cd10 con 0x7f6c3c004800 2024-04-25T10:11:44.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.899+0000 7f6c5c800640 1 -- 172.21.15.119:0/906396426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c540b2bc0 con 0x7f6c3c004800 2024-04-25T10:11:44.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.900+0000 7f6c5c800640 1 -- 172.21.15.119:0/906396426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c540750b0 con 0x7f6c3c004800 2024-04-25T10:11:44.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.902+0000 7f6c4e000640 1 -- 172.21.15.119:0/906396426 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c4801a070 con 0x7f6c3c004800 2024-04-25T10:11:44.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.902+0000 7f6c4e000640 1 --2- 172.21.15.119:0/906396426 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6c3806b4d0 0x7f6c3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:44.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.903+0000 7f6c4e000640 1 -- 172.21.15.119:0/906396426 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c4808a2b0 con 0x7f6c3c004800 2024-04-25T10:11:44.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.903+0000 7f6c59400640 1 --2- 172.21.15.119:0/906396426 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6c3806b4d0 0x7f6c3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:44.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.904+0000 7f6c59400640 1 --2- 172.21.15.119:0/906396426 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6c3806b4d0 0x7f6c3806d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6c44033000 tx=0x7f6c4402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:44.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:44.907+0000 7f6c4e000640 1 -- 172.21.15.119:0/906396426 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c48059090 con 0x7f6c3c004800 2024-04-25T10:11:45.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.101+0000 7f6c5c800640 1 -- 172.21.15.119:0/906396426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c540b7870 con 0x7f6c3c004800 2024-04-25T10:11:45.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.102+0000 7f6c4e000640 1 -- 172.21.15.119:0/906396426 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c48058990 con 0x7f6c3c004800 2024-04-25T10:11:45.107 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:11:45.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.105+0000 7f6c5c800640 1 -- 172.21.15.119:0/906396426 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6c3806b4d0 msgr2=0x7f6c3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.105+0000 7f6c5c800640 1 --2- 172.21.15.119:0/906396426 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6c3806b4d0 0x7f6c3806d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6c44033000 tx=0x7f6c4402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:45.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.105+0000 7f6c5c800640 1 -- 172.21.15.119:0/906396426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c3c004800 msgr2=0x7f6c540c9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.105+0000 7f6c5c800640 1 --2- 172.21.15.119:0/906396426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c3c004800 0x7f6c540c9540 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e860 tx=0x7f6c4800ed20 comp rx=0 tx=0).stop 2024-04-25T10:11:45.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.105+0000 7f6c5c800640 1 -- 172.21.15.119:0/906396426 shutdown_connections 2024-04-25T10:11:45.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.105+0000 7f6c5c800640 1 --2- 172.21.15.119:0/906396426 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6c3806b4d0 0x7f6c3806d990 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.105+0000 7f6c5c800640 1 --2- 172.21.15.119:0/906396426 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6c540b4470 0x7f6c540cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.105+0000 7f6c5c800640 1 --2- 172.21.15.119:0/906396426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c540b1e20 0x7f6c541589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.105+0000 7f6c5c800640 1 --2- 172.21.15.119:0/906396426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c3c004800 0x7f6c540c9540 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.105+0000 7f6c5c800640 1 -- 172.21.15.119:0/906396426 >> 172.21.15.119:0/906396426 conn(0x7f6c5406f4a0 msgr2=0x7f6c5415c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:45.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.105+0000 7f6c5c800640 1 -- 172.21.15.119:0/906396426 shutdown_connections 2024-04-25T10:11:45.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.105+0000 7f6c5c800640 1 -- 172.21.15.119:0/906396426 wait complete. 2024-04-25T10:11:45.129 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:45.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.321+0000 7f5fe4200640 1 Processor -- start 2024-04-25T10:11:45.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.321+0000 7f5fe4200640 1 -- start start 2024-04-25T10:11:45.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.323+0000 7f5fe4200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5fdc144e70 0x7f5fdc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.323+0000 7f5fe4200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5fdc06d7a0 0x7f5fdc0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.323+0000 7f5fe4200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fdc06dcb0 con 0x7f5fdc0cef10 2024-04-25T10:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.323+0000 7f5fe4200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fdc15bfe0 con 0x7f5fdc144e70 2024-04-25T10:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.323+0000 7f5fe4200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fdc06df90 con 0x7f5fdc06d7a0 2024-04-25T10:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.323+0000 7f5fe2200640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5fdc0cef10 0x7f5fdc071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40254/0 (socket says 172.21.15.119:40254) 2024-04-25T10:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.323+0000 7f5fe2200640 1 -- 172.21.15.119:0/1384852129 learned_addr learned my addr 172.21.15.119:0/1384852129 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.323+0000 7f5fe1800640 1 --2- 172.21.15.119:0/1384852129 >> v2:172.21.15.119:3300/0 conn(0x7f5fdc144e70 0x7f5fdc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.323+0000 7f5fe0e00640 1 --2- 172.21.15.119:0/1384852129 >> v2:172.21.15.132:3300/0 conn(0x7f5fdc06d7a0 0x7f5fdc0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.324+0000 7f5fdbe00640 1 -- 172.21.15.119:0/1384852129 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438711632 0 0) 0x7f5fdc06dcb0 con 0x7f5fdc0cef10 2024-04-25T10:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.324+0000 7f5fdbe00640 1 -- 172.21.15.119:0/1384852129 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fc4003520 con 0x7f5fdc0cef10 2024-04-25T10:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.324+0000 7f5fe1800640 1 -- 172.21.15.119:0/1384852129 >> v2:172.21.15.132:3300/0 conn(0x7f5fdc06d7a0 msgr2=0x7f5fdc0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.324+0000 7f5fe1800640 1 --2- 172.21.15.119:0/1384852129 >> v2:172.21.15.132:3300/0 conn(0x7f5fdc06d7a0 0x7f5fdc0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.324+0000 7f5fe1800640 1 -- 172.21.15.119:0/1384852129 >> v1:172.21.15.119:6789/0 conn(0x7f5fdc0cef10 legacy=0x7f5fdc071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.324+0000 7f5fe1800640 1 -- 172.21.15.119:0/1384852129 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc06e270 con 0x7f5fdc144e70 2024-04-25T10:11:45.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.325+0000 7f5fe1800640 1 --2- 172.21.15.119:0/1384852129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fdc144e70 0x7f5fdc06d260 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc072560 tx=0x7f5fd002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3a606472371006 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.325+0000 7f5fdbe00640 1 -- 172.21.15.119:0/1384852129 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0037070 con 0x7f5fdc144e70 2024-04-25T10:11:45.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.325+0000 7f5fdbe00640 1 -- 172.21.15.119:0/1384852129 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd0032440 con 0x7f5fdc144e70 2024-04-25T10:11:45.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.325+0000 7f5fdbe00640 1 -- 172.21.15.119:0/1384852129 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd002ae60 con 0x7f5fdc144e70 2024-04-25T10:11:45.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.326+0000 7f5fe4200640 1 -- 172.21.15.119:0/1384852129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fdc144e70 msgr2=0x7f5fdc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.326+0000 7f5fe4200640 1 --2- 172.21.15.119:0/1384852129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fdc144e70 0x7f5fdc06d260 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc072560 tx=0x7f5fd002a110 comp rx=0 tx=0).stop 2024-04-25T10:11:45.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.326+0000 7f5fe4200640 1 -- 172.21.15.119:0/1384852129 shutdown_connections 2024-04-25T10:11:45.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.326+0000 7f5fe4200640 1 --2- 172.21.15.119:0/1384852129 >> v2:172.21.15.132:3300/0 conn(0x7f5fdc06d7a0 0x7f5fdc0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.326+0000 7f5fe4200640 1 --2- 172.21.15.119:0/1384852129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fdc144e70 0x7f5fdc06d260 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.326+0000 7f5fe4200640 1 -- 172.21.15.119:0/1384852129 >> 172.21.15.119:0/1384852129 conn(0x7f5fdc06aba0 msgr2=0x7f5fdc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:45.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.326+0000 7f5fe4200640 1 -- 172.21.15.119:0/1384852129 shutdown_connections 2024-04-25T10:11:45.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.326+0000 7f5fe4200640 1 -- 172.21.15.119:0/1384852129 wait complete. 2024-04-25T10:11:45.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.328+0000 7f5fe4200640 1 Processor -- start 2024-04-25T10:11:45.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.328+0000 7f5fe4200640 1 -- start start 2024-04-25T10:11:45.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.329+0000 7f5fe4200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5fdc06d7a0 0x7f5fdc1f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.329+0000 7f5fe4200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5fdc0cef10 0x7f5fdc0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.329+0000 7f5fe4200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fdc144e70 0x7f5fdc15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.329+0000 7f5fe4200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc1454b0 con 0x7f5fdc144e70 2024-04-25T10:11:45.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.329+0000 7f5fe4200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc0cf370 con 0x7f5fdc0cef10 2024-04-25T10:11:45.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.329+0000 7f5fe4200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc145630 con 0x7f5fdc06d7a0 2024-04-25T10:11:45.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.329+0000 7f5fe2200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fdc144e70 0x7f5fdc15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.329+0000 7f5fe2200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fdc144e70 0x7f5fdc15aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55772/0 (socket says 172.21.15.119:55772) 2024-04-25T10:11:45.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.329+0000 7f5fe2200640 1 -- 172.21.15.119:0/1222926436 learned_addr learned my addr 172.21.15.119:0/1222926436 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:45.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.330+0000 7f5fe0e00640 1 --2- 172.21.15.119:0/1222926436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5fdc0cef10 0x7f5fdc0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.330+0000 7f5fe2200640 1 -- 172.21.15.119:0/1222926436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5fdc06d7a0 msgr2=0x7f5fdc1f4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.330+0000 7f5fe2200640 1 --2- 172.21.15.119:0/1222926436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5fdc06d7a0 0x7f5fdc1f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.330+0000 7f5fe2200640 1 -- 172.21.15.119:0/1222926436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5fdc0cef10 msgr2=0x7f5fdc0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.330+0000 7f5fe2200640 1 --2- 172.21.15.119:0/1222926436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5fdc0cef10 0x7f5fdc0b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.330+0000 7f5fe2200640 1 -- 172.21.15.119:0/1222926436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc0c47d0 con 0x7f5fdc144e70 2024-04-25T10:11:45.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.330+0000 7f5fe2200640 1 --2- 172.21.15.119:0/1222926436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fdc144e70 0x7f5fdc15aca0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400cb40 tx=0x7f5fd400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.330+0000 7f5fd9600640 1 -- 172.21.15.119:0/1222926436 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4017070 con 0x7f5fdc144e70 2024-04-25T10:11:45.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.330+0000 7f5fd9600640 1 -- 172.21.15.119:0/1222926436 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd400aca0 con 0x7f5fdc144e70 2024-04-25T10:11:45.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.330+0000 7f5fd9600640 1 -- 172.21.15.119:0/1222926436 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4012620 con 0x7f5fdc144e70 2024-04-25T10:11:45.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.330+0000 7f5fe4200640 1 -- 172.21.15.119:0/1222926436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fdc0c4ab0 con 0x7f5fdc144e70 2024-04-25T10:11:45.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.330+0000 7f5fe4200640 1 -- 172.21.15.119:0/1222926436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fdc0c5070 con 0x7f5fdc144e70 2024-04-25T10:11:45.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.332+0000 7f5fe4200640 1 -- 172.21.15.119:0/1222926436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fdc0750b0 con 0x7f5fdc144e70 2024-04-25T10:11:45.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.332+0000 7f5fd9600640 1 -- 172.21.15.119:0/1222926436 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fd401a070 con 0x7f5fdc144e70 2024-04-25T10:11:45.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.333+0000 7f5fd9600640 1 --2- 172.21.15.119:0/1222926436 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5fbc06b500 0x7f5fbc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.334+0000 7f5fd9600640 1 -- 172.21.15.119:0/1222926436 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fd408a070 con 0x7f5fdc144e70 2024-04-25T10:11:45.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.334+0000 7f5fe1800640 1 --2- 172.21.15.119:0/1222926436 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5fbc06b500 0x7f5fbc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.334+0000 7f5fe1800640 1 --2- 172.21.15.119:0/1222926436 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5fbc06b500 0x7f5fbc06d9c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0034000 tx=0x7f5fd002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.338+0000 7f5fd9600640 1 -- 172.21.15.119:0/1222926436 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5fd4058ee0 con 0x7f5fdc144e70 2024-04-25T10:11:45.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.531+0000 7f5fe4200640 1 -- 172.21.15.119:0/1222926436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5fdc0cd770 con 0x7f5fdc144e70 2024-04-25T10:11:45.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.532+0000 7f5fd9600640 1 -- 172.21.15.119:0/1222926436 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5fd40587e0 con 0x7f5fdc144e70 2024-04-25T10:11:45.537 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:11:45.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.535+0000 7f5fe4200640 1 -- 172.21.15.119:0/1222926436 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5fbc06b500 msgr2=0x7f5fbc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.535+0000 7f5fe4200640 1 --2- 172.21.15.119:0/1222926436 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5fbc06b500 0x7f5fbc06d9c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0034000 tx=0x7f5fd002c000 comp rx=0 tx=0).stop 2024-04-25T10:11:45.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.535+0000 7f5fe4200640 1 -- 172.21.15.119:0/1222926436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fdc144e70 msgr2=0x7f5fdc15aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.535+0000 7f5fe4200640 1 --2- 172.21.15.119:0/1222926436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fdc144e70 0x7f5fdc15aca0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400cb40 tx=0x7f5fd400a060 comp rx=0 tx=0).stop 2024-04-25T10:11:45.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.535+0000 7f5fe4200640 1 -- 172.21.15.119:0/1222926436 shutdown_connections 2024-04-25T10:11:45.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.535+0000 7f5fe4200640 1 --2- 172.21.15.119:0/1222926436 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5fbc06b500 0x7f5fbc06d9c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.535+0000 7f5fe4200640 1 --2- 172.21.15.119:0/1222926436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fdc144e70 0x7f5fdc15aca0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.535+0000 7f5fe4200640 1 --2- 172.21.15.119:0/1222926436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5fdc0cef10 0x7f5fdc0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.535+0000 7f5fe4200640 1 --2- 172.21.15.119:0/1222926436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5fdc06d7a0 0x7f5fdc1f4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.535+0000 7f5fe4200640 1 -- 172.21.15.119:0/1222926436 >> 172.21.15.119:0/1222926436 conn(0x7f5fdc06aba0 msgr2=0x7f5fdc0cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:45.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.535+0000 7f5fe4200640 1 -- 172.21.15.119:0/1222926436 shutdown_connections 2024-04-25T10:11:45.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.535+0000 7f5fe4200640 1 -- 172.21.15.119:0/1222926436 wait complete. 2024-04-25T10:11:45.559 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:45.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.751+0000 7f366dc00640 1 Processor -- start 2024-04-25T10:11:45.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.752+0000 7f366dc00640 1 -- start start 2024-04-25T10:11:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.753+0000 7f366dc00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f36680d01d0 0x7f36680cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.753+0000 7f366dc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f36680d1b90 0x7f36680d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.753+0000 7f366dc00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36681625f0 con 0x7f36680d05b0 2024-04-25T10:11:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.753+0000 7f366dc00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3668162160 con 0x7f36680d01d0 2024-04-25T10:11:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.753+0000 7f366dc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36680cef60 con 0x7f36680d1b90 2024-04-25T10:11:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.753+0000 7f3667400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f36680d05b0 0x7f366815ea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40268/0 (socket says 172.21.15.119:40268) 2024-04-25T10:11:45.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.753+0000 7f3667400640 1 -- 172.21.15.119:0/2761936603 learned_addr learned my addr 172.21.15.119:0/2761936603 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:45.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.754+0000 7f3667e00640 1 --2- 172.21.15.119:0/2761936603 >> v2:172.21.15.119:3300/0 conn(0x7f36680d01d0 0x7f36680cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.754+0000 7f366d200640 1 --2- 172.21.15.119:0/2761936603 >> v2:172.21.15.132:3300/0 conn(0x7f36680d1b90 0x7f36680d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.754+0000 7f3666a00640 1 -- 172.21.15.119:0/2761936603 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555796056 0 0) 0x7f36681625f0 con 0x7f36680d05b0 2024-04-25T10:11:45.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.754+0000 7f3666a00640 1 -- 172.21.15.119:0/2761936603 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f364c003520 con 0x7f36680d05b0 2024-04-25T10:11:45.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.754+0000 7f3667e00640 1 -- 172.21.15.119:0/2761936603 >> v2:172.21.15.132:3300/0 conn(0x7f36680d1b90 msgr2=0x7f36680d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:45.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.754+0000 7f3667e00640 1 --2- 172.21.15.119:0/2761936603 >> v2:172.21.15.132:3300/0 conn(0x7f36680d1b90 0x7f36680d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.754+0000 7f3667e00640 1 -- 172.21.15.119:0/2761936603 >> v1:172.21.15.119:6789/0 conn(0x7f36680d05b0 legacy=0x7f366815ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.754+0000 7f3667e00640 1 -- 172.21.15.119:0/2761936603 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36680cf200 con 0x7f36680d01d0 2024-04-25T10:11:45.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.755+0000 7f3667e00640 1 --2- 172.21.15.119:0/2761936603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36680d01d0 0x7f36680cea20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f36680d16e0 tx=0x7f365002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e481de0c822e51a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.755+0000 7f3666a00640 1 -- 172.21.15.119:0/2761936603 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3650037070 con 0x7f36680d01d0 2024-04-25T10:11:45.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.755+0000 7f3666a00640 1 -- 172.21.15.119:0/2761936603 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3650032440 con 0x7f36680d01d0 2024-04-25T10:11:45.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.755+0000 7f3666a00640 1 -- 172.21.15.119:0/2761936603 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365002ae60 con 0x7f36680d01d0 2024-04-25T10:11:45.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.756+0000 7f366dc00640 1 -- 172.21.15.119:0/2761936603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36680d01d0 msgr2=0x7f36680cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.756+0000 7f366dc00640 1 --2- 172.21.15.119:0/2761936603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36680d01d0 0x7f36680cea20 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f36680d16e0 tx=0x7f365002a110 comp rx=0 tx=0).stop 2024-04-25T10:11:45.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.756+0000 7f366dc00640 1 -- 172.21.15.119:0/2761936603 shutdown_connections 2024-04-25T10:11:45.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.756+0000 7f366dc00640 1 --2- 172.21.15.119:0/2761936603 >> v2:172.21.15.132:3300/0 conn(0x7f36680d1b90 0x7f36680d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.756+0000 7f366dc00640 1 --2- 172.21.15.119:0/2761936603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36680d01d0 0x7f36680cea20 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.756+0000 7f366dc00640 1 -- 172.21.15.119:0/2761936603 >> 172.21.15.119:0/2761936603 conn(0x7f366806bed0 msgr2=0x7f366806c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:45.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.756+0000 7f366dc00640 1 -- 172.21.15.119:0/2761936603 shutdown_connections 2024-04-25T10:11:45.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.756+0000 7f366dc00640 1 -- 172.21.15.119:0/2761936603 wait complete. 2024-04-25T10:11:45.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.758+0000 7f366dc00640 1 Processor -- start 2024-04-25T10:11:45.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.758+0000 7f366dc00640 1 -- start start 2024-04-25T10:11:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.759+0000 7f366dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36680d01d0 0x7f36680cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.759+0000 7f366dc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36680d05b0 0x7f36680c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.759+0000 7f366dc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36680d1b90 0x7f3668145490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.759+0000 7f366dc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3668161df0 con 0x7f36680d05b0 2024-04-25T10:11:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.759+0000 7f366dc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3668161b10 con 0x7f36680d1b90 2024-04-25T10:11:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.759+0000 7f366dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36680d0a10 con 0x7f36680d01d0 2024-04-25T10:11:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.759+0000 7f3667400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36680d05b0 0x7f36680c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.759+0000 7f3667400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36680d05b0 0x7f36680c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55796/0 (socket says 172.21.15.119:55796) 2024-04-25T10:11:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.759+0000 7f3667400640 1 -- 172.21.15.119:0/603142110 learned_addr learned my addr 172.21.15.119:0/603142110 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.760+0000 7f366d200640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36680d1b90 0x7f3668145490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.760+0000 7f3667e00640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36680d01d0 0x7f36680cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.760+0000 7f3667400640 1 -- 172.21.15.119:0/603142110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36680d01d0 msgr2=0x7f36680cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.760+0000 7f3667400640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36680d01d0 0x7f36680cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.760+0000 7f3667400640 1 -- 172.21.15.119:0/603142110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36680d1b90 msgr2=0x7f3668145490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.760+0000 7f3667400640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36680d1b90 0x7f3668145490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.760+0000 7f3667400640 1 -- 172.21.15.119:0/603142110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36680cc590 con 0x7f36680d05b0 2024-04-25T10:11:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.760+0000 7f366d200640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36680d1b90 0x7f3668145490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.760+0000 7f3667400640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36680d05b0 0x7f36680c9d30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f365800cb40 tx=0x7f365800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.760+0000 7f365fe00640 1 -- 172.21.15.119:0/603142110 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658017070 con 0x7f36680d05b0 2024-04-25T10:11:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.760+0000 7f365fe00640 1 -- 172.21.15.119:0/603142110 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365800aca0 con 0x7f36680d05b0 2024-04-25T10:11:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.760+0000 7f365fe00640 1 -- 172.21.15.119:0/603142110 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658012620 con 0x7f36680d05b0 2024-04-25T10:11:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.760+0000 7f366dc00640 1 -- 172.21.15.119:0/603142110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3668158b70 con 0x7f36680d05b0 2024-04-25T10:11:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.760+0000 7f366dc00640 1 -- 172.21.15.119:0/603142110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36680ccb10 con 0x7f36680d05b0 2024-04-25T10:11:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.761+0000 7f366dc00640 1 -- 172.21.15.119:0/603142110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36680749b0 con 0x7f36680d05b0 2024-04-25T10:11:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.762+0000 7f365fe00640 1 -- 172.21.15.119:0/603142110 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f365801a070 con 0x7f36680d05b0 2024-04-25T10:11:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.764+0000 7f365fe00640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f364406b4d0 0x7f364406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.764+0000 7f365fe00640 1 -- 172.21.15.119:0/603142110 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f365808a100 con 0x7f36680d05b0 2024-04-25T10:11:45.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.764+0000 7f3667e00640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f364406b4d0 0x7f364406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:45.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.765+0000 7f3667e00640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f364406b4d0 0x7f364406d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3650034000 tx=0x7f365002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:45.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.768+0000 7f365fe00640 1 -- 172.21.15.119:0/603142110 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3658058ee0 con 0x7f36680d05b0 2024-04-25T10:11:45.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.978+0000 7f366dc00640 1 -- 172.21.15.119:0/603142110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f36680ce000 con 0x7f36680d05b0 2024-04-25T10:11:45.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.979+0000 7f365fe00640 1 -- 172.21.15.119:0/603142110 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) v1 ==== 80+0+0 (secure 0 0 0) 0x7f36580587e0 con 0x7f36680d05b0 2024-04-25T10:11:45.984 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2024-04-25T10:11:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.981+0000 7f366dc00640 1 -- 172.21.15.119:0/603142110 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f364406b4d0 msgr2=0x7f364406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.981+0000 7f366dc00640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f364406b4d0 0x7f364406d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3650034000 tx=0x7f365002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.981+0000 7f366dc00640 1 -- 172.21.15.119:0/603142110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36680d05b0 msgr2=0x7f36680c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.981+0000 7f366dc00640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36680d05b0 0x7f36680c9d30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f365800cb40 tx=0x7f365800a060 comp rx=0 tx=0).stop 2024-04-25T10:11:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.981+0000 7f366dc00640 1 -- 172.21.15.119:0/603142110 shutdown_connections 2024-04-25T10:11:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.981+0000 7f366dc00640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f364406b4d0 0x7f364406d990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.982+0000 7f366dc00640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36680d1b90 0x7f3668145490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.982+0000 7f366dc00640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36680d05b0 0x7f36680c9d30 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.982+0000 7f366dc00640 1 --2- 172.21.15.119:0/603142110 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f36680d01d0 0x7f36680cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.982+0000 7f366dc00640 1 -- 172.21.15.119:0/603142110 >> 172.21.15.119:0/603142110 conn(0x7f366806bed0 msgr2=0x7f3668156a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.982+0000 7f366dc00640 1 -- 172.21.15.119:0/603142110 shutdown_connections 2024-04-25T10:11:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:45.982+0000 7f366dc00640 1 -- 172.21.15.119:0/603142110 wait complete. 2024-04-25T10:11:46.004 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:11:46.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:46.005 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:46.130 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:11:46.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:46.130 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:11:46.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:46.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.385+0000 7f9e53e00640 1 Processor -- start 2024-04-25T10:11:46.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.385+0000 7f9e53e00640 1 -- start start 2024-04-25T10:11:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.386+0000 7f9e53e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e54144470 0x7f9e5415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.387+0000 7f9e53e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9e5406e430 0x7f9e540b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.387+0000 7f9e53e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e540cd440 con 0x7f9e540b4e80 2024-04-25T10:11:46.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.387+0000 7f9e53e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e54145340 con 0x7f9e5406e430 2024-04-25T10:11:46.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.387+0000 7f9e53e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e540cd720 con 0x7f9e54144470 2024-04-25T10:11:46.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.387+0000 7f9e52000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9e5406e430 0x7f9e540b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.387+0000 7f9e52a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e54144470 0x7f9e5415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.387+0000 7f9e52a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e54144470 0x7f9e5415fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:56974/0 (socket says 172.21.15.119:56974) 2024-04-25T10:11:46.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.388+0000 7f9e52a00640 1 -- 172.21.15.119:0/2194360983 learned_addr learned my addr 172.21.15.119:0/2194360983 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:46.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.388+0000 7f9e52000640 1 -- 172.21.15.119:0/2194360983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e54144470 msgr2=0x7f9e5415fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:46.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.388+0000 7f9e52000640 1 --2- 172.21.15.119:0/2194360983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e54144470 0x7f9e5415fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.388+0000 7f9e52000640 1 -- 172.21.15.119:0/2194360983 >> v1:172.21.15.132:6789/0 conn(0x7f9e540b4e80 legacy=0x7f9e5406d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.388+0000 7f9e52000640 1 -- 172.21.15.119:0/2194360983 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e540cda00 con 0x7f9e5406e430 2024-04-25T10:11:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.389+0000 7f9e52000640 1 --2- 172.21.15.119:0/2194360983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5406e430 0x7f9e540b4940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0097a0 tx=0x7f9e3c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22565ed31f7e0883 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.390+0000 7f9e51600640 1 -- 172.21.15.119:0/2194360983 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c037070 con 0x7f9e5406e430 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.390+0000 7f9e51600640 1 -- 172.21.15.119:0/2194360983 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c032440 con 0x7f9e5406e430 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.390+0000 7f9e51600640 1 -- 172.21.15.119:0/2194360983 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c032810 con 0x7f9e5406e430 2024-04-25T10:11:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.391+0000 7f9e53e00640 1 -- 172.21.15.119:0/2194360983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5406e430 msgr2=0x7f9e540b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.391+0000 7f9e53e00640 1 --2- 172.21.15.119:0/2194360983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5406e430 0x7f9e540b4940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0097a0 tx=0x7f9e3c02a180 comp rx=0 tx=0).stop 2024-04-25T10:11:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.391+0000 7f9e53e00640 1 -- 172.21.15.119:0/2194360983 shutdown_connections 2024-04-25T10:11:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.391+0000 7f9e53e00640 1 --2- 172.21.15.119:0/2194360983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5406e430 0x7f9e540b4940 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.391+0000 7f9e53e00640 1 --2- 172.21.15.119:0/2194360983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e54144470 0x7f9e5415fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.391+0000 7f9e53e00640 1 -- 172.21.15.119:0/2194360983 >> 172.21.15.119:0/2194360983 conn(0x7f9e5406b1d0 msgr2=0x7f9e540b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.391+0000 7f9e53e00640 1 -- 172.21.15.119:0/2194360983 shutdown_connections 2024-04-25T10:11:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.391+0000 7f9e53e00640 1 -- 172.21.15.119:0/2194360983 wait complete. 2024-04-25T10:11:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.392+0000 7f9e53e00640 1 Processor -- start 2024-04-25T10:11:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.393+0000 7f9e53e00640 1 -- start start 2024-04-25T10:11:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.393+0000 7f9e53e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5406e430 0x7f9e541f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.393+0000 7f9e53e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e540b4e80 0x7f9e541e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.393+0000 7f9e53e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e54144470 0x7f9e541e7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.393+0000 7f9e53e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e540ce200 con 0x7f9e540b4e80 2024-04-25T10:11:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.393+0000 7f9e53e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e54144850 con 0x7f9e5406e430 2024-04-25T10:11:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.393+0000 7f9e53e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5415cd00 con 0x7f9e54144470 2024-04-25T10:11:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.394+0000 7f9e52000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e540b4e80 0x7f9e541e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.394+0000 7f9e52000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e540b4e80 0x7f9e541e6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55804/0 (socket says 172.21.15.119:55804) 2024-04-25T10:11:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.394+0000 7f9e52000640 1 -- 172.21.15.119:0/607066958 learned_addr learned my addr 172.21.15.119:0/607066958 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.394+0000 7f9e52000640 1 -- 172.21.15.119:0/607066958 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e54144470 msgr2=0x7f9e541e7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.394+0000 7f9e53400640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e54144470 0x7f9e541e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.394+0000 7f9e52a00640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5406e430 0x7f9e541f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.394+0000 7f9e52000640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e54144470 0x7f9e541e7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.394+0000 7f9e52000640 1 -- 172.21.15.119:0/607066958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5406e430 msgr2=0x7f9e541f6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.394+0000 7f9e52000640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5406e430 0x7f9e541f6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.394+0000 7f9e52000640 1 -- 172.21.15.119:0/607066958 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e540b0980 con 0x7f9e540b4e80 2024-04-25T10:11:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.394+0000 7f9e53400640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e54144470 0x7f9e541e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.394+0000 7f9e52a00640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5406e430 0x7f9e541f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.394+0000 7f9e52000640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e540b4e80 0x7f9e541e6fd0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c02a1c0 tx=0x7f9e3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.395+0000 7f9e4b400640 1 -- 172.21.15.119:0/607066958 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c041020 con 0x7f9e540b4e80 2024-04-25T10:11:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.395+0000 7f9e4b400640 1 -- 172.21.15.119:0/607066958 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c030700 con 0x7f9e540b4e80 2024-04-25T10:11:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.395+0000 7f9e4b400640 1 -- 172.21.15.119:0/607066958 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c037040 con 0x7f9e540b4e80 2024-04-25T10:11:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.395+0000 7f9e53e00640 1 -- 172.21.15.119:0/607066958 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5412cd90 con 0x7f9e540b4e80 2024-04-25T10:11:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.395+0000 7f9e53e00640 1 -- 172.21.15.119:0/607066958 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e540afc60 con 0x7f9e540b4e80 2024-04-25T10:11:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.396+0000 7f9e53e00640 1 -- 172.21.15.119:0/607066958 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e540ce380 con 0x7f9e540b4e80 2024-04-25T10:11:46.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.397+0000 7f9e4b400640 1 -- 172.21.15.119:0/607066958 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e3c044020 con 0x7f9e540b4e80 2024-04-25T10:11:46.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.398+0000 7f9e4b400640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9e3006b4d0 0x7f9e3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.398+0000 7f9e4b400640 1 -- 172.21.15.119:0/607066958 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e3c0aaa80 con 0x7f9e540b4e80 2024-04-25T10:11:46.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.398+0000 7f9e52a00640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9e3006b4d0 0x7f9e3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.398+0000 7f9e52a00640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9e3006b4d0 0x7f9e3006d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009a20 tx=0x7f9e44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.402+0000 7f9e4b400640 1 -- 172.21.15.119:0/607066958 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9e540ce380 con 0x7f9e540b4e80 2024-04-25T10:11:46.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.631+0000 7f9e53e00640 1 -- 172.21.15.119:0/607066958 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e540749b0 con 0x7f9e540b4e80 2024-04-25T10:11:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.632+0000 7f9e4b400640 1 -- 172.21.15.119:0/607066958 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f9e3c0ace20 con 0x7f9e540b4e80 2024-04-25T10:11:46.637 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:46.637 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2798722683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2798722683},{"type":"v1","addr":"172.21.15.132:6833","nonce":2798722683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2834911408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2834911408},{"type":"v1","addr":"172.21.15.132:6835","nonce":2834911408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/643040335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":643040335},{"type":"v1","addr":"172.21.15.119:6835","nonce":643040335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:34.712290+0000","modified":"2024-04-25T10:11:37.787344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6483},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6483":{"gid":6483,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/2673349178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2673349178},{"type":"v1","addr":"172.21.15.119:6837","nonce":2673349178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:46.638 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 44 2024-04-25T10:11:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.634+0000 7f9e53e00640 1 -- 172.21.15.119:0/607066958 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9e3006b4d0 msgr2=0x7f9e3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.634+0000 7f9e53e00640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9e3006b4d0 0x7f9e3006d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009a20 tx=0x7f9e44008040 comp rx=0 tx=0).stop 2024-04-25T10:11:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.634+0000 7f9e53e00640 1 -- 172.21.15.119:0/607066958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e540b4e80 msgr2=0x7f9e541e6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.634+0000 7f9e53e00640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e540b4e80 0x7f9e541e6fd0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c02a1c0 tx=0x7f9e3c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.635+0000 7f9e53e00640 1 -- 172.21.15.119:0/607066958 shutdown_connections 2024-04-25T10:11:46.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.635+0000 7f9e53e00640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9e3006b4d0 0x7f9e3006d990 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.635+0000 7f9e53e00640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9e54144470 0x7f9e541e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.635+0000 7f9e53e00640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e540b4e80 0x7f9e541e6fd0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.635+0000 7f9e53e00640 1 --2- 172.21.15.119:0/607066958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5406e430 0x7f9e541f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.635+0000 7f9e53e00640 1 -- 172.21.15.119:0/607066958 >> 172.21.15.119:0/607066958 conn(0x7f9e5406b1d0 msgr2=0x7f9e540ae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:46.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.635+0000 7f9e53e00640 1 -- 172.21.15.119:0/607066958 shutdown_connections 2024-04-25T10:11:46.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:46.635+0000 7f9e53e00640 1 -- 172.21.15.119:0/607066958 wait complete. 2024-04-25T10:11:46.658 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:11:46.889 INFO:teuthology.orchestra.run.smithi119.stdout:{"message":"","return_code":0} 2024-04-25T10:11:46.889 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:11:46.889 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:11:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.086+0000 7fbef8a00640 1 Processor -- start 2024-04-25T10:11:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.087+0000 7fbef8a00640 1 -- start start 2024-04-25T10:11:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.088+0000 7fbef8a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbef415b7e0 0x7fbef415e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.088+0000 7fbef8a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef40b5710 con 0x7fbef40ce860 2024-04-25T10:11:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.088+0000 7fbef8a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef40b59f0 con 0x7fbef40d07b0 2024-04-25T10:11:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.088+0000 7fbef8a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef40b5cd0 con 0x7fbef415b7e0 2024-04-25T10:11:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.088+0000 7fbef1e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbef40ce860 0x7fbef40d00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40270/0 (socket says 172.21.15.119:40270) 2024-04-25T10:11:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.088+0000 7fbef1e00640 1 -- 172.21.15.119:0/1613472857 learned_addr learned my addr 172.21.15.119:0/1613472857 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 -- 172.21.15.119:0/1613472857 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930579976 0 0) 0x7fbef40b5710 con 0x7fbef40ce860 2024-04-25T10:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 -- 172.21.15.119:0/1613472857 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbed8003520 con 0x7fbef40ce860 2024-04-25T10:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef2800640 1 --2- 172.21.15.119:0/1613472857 >> v2:172.21.15.132:3300/0 conn(0x7fbef415b7e0 0x7fbef415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 -- 172.21.15.119:0/1613472857 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3734820700 0 0) 0x7fbed8003520 con 0x7fbef40ce860 2024-04-25T10:11:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 -- 172.21.15.119:0/1613472857 >> v2:172.21.15.132:3300/0 conn(0x7fbef415b7e0 msgr2=0x7fbef415e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 --2- 172.21.15.119:0/1613472857 >> v2:172.21.15.132:3300/0 conn(0x7fbef415b7e0 0x7fbef415e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 -- 172.21.15.119:0/1613472857 >> v1:172.21.15.132:6789/0 conn(0x7fbef40d07b0 legacy=0x7fbef4161960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 -- 172.21.15.119:0/1613472857 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40b5fb0 con 0x7fbef40ce860 2024-04-25T10:11:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 -- 172.21.15.119:0/1613472857 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fbee0001670 con 0x7fbef40ce860 2024-04-25T10:11:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 -- 172.21.15.119:0/1613472857 >> v1:172.21.15.119:6789/0 conn(0x7fbef40ce860 legacy=0x7fbef40d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 --2- 172.21.15.119:0/1613472857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbed80039b0 0x7fbed8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 --2- 172.21.15.119:0/1613472857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbed8004800 0x7fbed8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 --2- 172.21.15.119:0/1613472857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbed80052e0 0x7fbed800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 -- 172.21.15.119:0/1613472857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40b5cd0 con 0x7fbed80052e0 2024-04-25T10:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 -- 172.21.15.119:0/1613472857 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed800c230 con 0x7fbed8004800 2024-04-25T10:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef1400640 1 -- 172.21.15.119:0/1613472857 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed800c430 con 0x7fbed80039b0 2024-04-25T10:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.089+0000 7fbef3200640 1 --2- 172.21.15.119:0/1613472857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbed80052e0 0x7fbed800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.090+0000 7fbef1e00640 1 --2- 172.21.15.119:0/1613472857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbed8004800 0x7fbed8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.090+0000 7fbef2800640 1 --2- 172.21.15.119:0/1613472857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbed80039b0 0x7fbed8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.090+0000 7fbef3200640 1 -- 172.21.15.119:0/1613472857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbed80039b0 msgr2=0x7fbed8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.090+0000 7fbef3200640 1 --2- 172.21.15.119:0/1613472857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbed80039b0 0x7fbed8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.090+0000 7fbef3200640 1 -- 172.21.15.119:0/1613472857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbed8004800 msgr2=0x7fbed8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.090+0000 7fbef3200640 1 --2- 172.21.15.119:0/1613472857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbed8004800 0x7fbed8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.090+0000 7fbef3200640 1 -- 172.21.15.119:0/1613472857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed8003520 con 0x7fbed80052e0 2024-04-25T10:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.090+0000 7fbef1e00640 1 --2- 172.21.15.119:0/1613472857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbed8004800 0x7fbed8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:47.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.090+0000 7fbef2800640 1 --2- 172.21.15.119:0/1613472857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbed80039b0 0x7fbed8003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:47.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.091+0000 7fbef3200640 1 -- 172.21.15.119:0/1613472857 reap_dead start 2024-04-25T10:11:47.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.091+0000 7fbef3200640 1 --2- 172.21.15.119:0/1613472857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbed80052e0 0x7fbed800b9c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fbeec004480 tx=0x7fbed8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b929432fa354dad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.091+0000 7fbef1400640 1 -- 172.21.15.119:0/1613472857 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec004080 con 0x7fbed80052e0 2024-04-25T10:11:47.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.091+0000 7fbef1400640 1 -- 172.21.15.119:0/1613472857 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec02ad80 con 0x7fbed80052e0 2024-04-25T10:11:47.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.091+0000 7fbef1400640 1 -- 172.21.15.119:0/1613472857 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec006d90 con 0x7fbed80052e0 2024-04-25T10:11:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.092+0000 7fbef8a00640 1 -- 172.21.15.119:0/1613472857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbed80052e0 msgr2=0x7fbed800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.092+0000 7fbef8a00640 1 --2- 172.21.15.119:0/1613472857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbed80052e0 0x7fbed800b9c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fbeec004480 tx=0x7fbed8005830 comp rx=0 tx=0).stop 2024-04-25T10:11:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.092+0000 7fbef8a00640 1 -- 172.21.15.119:0/1613472857 shutdown_connections 2024-04-25T10:11:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.092+0000 7fbef8a00640 1 --2- 172.21.15.119:0/1613472857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbed80052e0 0x7fbed800b9c0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.092+0000 7fbef8a00640 1 -- 172.21.15.119:0/1613472857 >> 172.21.15.119:0/1613472857 conn(0x7fbef40722d0 msgr2=0x7fbef40726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.092+0000 7fbef8a00640 1 -- 172.21.15.119:0/1613472857 shutdown_connections 2024-04-25T10:11:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.092+0000 7fbef8a00640 1 -- 172.21.15.119:0/1613472857 wait complete. 2024-04-25T10:11:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.094+0000 7fbef8a00640 1 Processor -- start 2024-04-25T10:11:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.094+0000 7fbef8a00640 1 -- start start 2024-04-25T10:11:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.095+0000 7fbef8a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbed80052e0 0x7fbef41f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.095+0000 7fbef8a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef415a600 0x7fbef40cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.095+0000 7fbef8a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbef415aa60 0x7fbef40cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.095+0000 7fbef8a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40cee70 con 0x7fbef415aa60 2024-04-25T10:11:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.095+0000 7fbef8a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40cecf0 con 0x7fbed80052e0 2024-04-25T10:11:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.095+0000 7fbef8a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40ceff0 con 0x7fbef415a600 2024-04-25T10:11:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.095+0000 7fbef3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbef415aa60 0x7fbef40cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.095+0000 7fbef3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbef415aa60 0x7fbef40cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55828/0 (socket says 172.21.15.119:55828) 2024-04-25T10:11:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.095+0000 7fbef3200640 1 -- 172.21.15.119:0/929417693 learned_addr learned my addr 172.21.15.119:0/929417693 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.096+0000 7fbef1e00640 1 --2- 172.21.15.119:0/929417693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef415a600 0x7fbef40cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.096+0000 7fbef3200640 1 -- 172.21.15.119:0/929417693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef415a600 msgr2=0x7fbef40cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.096+0000 7fbef2800640 1 --2- 172.21.15.119:0/929417693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbed80052e0 0x7fbef41f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.096+0000 7fbef3200640 1 --2- 172.21.15.119:0/929417693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef415a600 0x7fbef40cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.096+0000 7fbef3200640 1 -- 172.21.15.119:0/929417693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbed80052e0 msgr2=0x7fbef41f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.096+0000 7fbef3200640 1 --2- 172.21.15.119:0/929417693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbed80052e0 0x7fbef41f7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.096+0000 7fbef3200640 1 -- 172.21.15.119:0/929417693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef41f7b90 con 0x7fbef415aa60 2024-04-25T10:11:47.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.096+0000 7fbef3200640 1 --2- 172.21.15.119:0/929417693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbef415aa60 0x7fbef40cc3b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fbeec000f80 tx=0x7fbeec003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.096+0000 7fbeeaa00640 1 -- 172.21.15.119:0/929417693 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec007df0 con 0x7fbef415aa60 2024-04-25T10:11:47.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.096+0000 7fbeeaa00640 1 -- 172.21.15.119:0/929417693 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec042070 con 0x7fbef415aa60 2024-04-25T10:11:47.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.096+0000 7fbef8a00640 1 -- 172.21.15.119:0/929417693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef4142ed0 con 0x7fbef415aa60 2024-04-25T10:11:47.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.096+0000 7fbeeaa00640 1 -- 172.21.15.119:0/929417693 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec007a20 con 0x7fbef415aa60 2024-04-25T10:11:47.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.096+0000 7fbef8a00640 1 -- 172.21.15.119:0/929417693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef4143490 con 0x7fbef415aa60 2024-04-25T10:11:47.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.098+0000 7fbef8a00640 1 -- 172.21.15.119:0/929417693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef40749b0 con 0x7fbef415aa60 2024-04-25T10:11:47.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.098+0000 7fbeeaa00640 1 -- 172.21.15.119:0/929417693 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbeec047460 con 0x7fbef415aa60 2024-04-25T10:11:47.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.099+0000 7fbeeaa00640 1 --2- 172.21.15.119:0/929417693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbed406b500 0x7fbed406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.099+0000 7fbeeaa00640 1 -- 172.21.15.119:0/929417693 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbeec0aa5b0 con 0x7fbef415aa60 2024-04-25T10:11:47.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.100+0000 7fbef2800640 1 --2- 172.21.15.119:0/929417693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbed406b500 0x7fbed406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.100+0000 7fbef2800640 1 --2- 172.21.15.119:0/929417693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbed406b500 0x7fbed406d9c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbee4006fd0 tx=0x7fbee4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.103+0000 7fbeeaa00640 1 -- 172.21.15.119:0/929417693 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbef41f7b90 con 0x7fbef415aa60 2024-04-25T10:11:47.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.335+0000 7fbef8a00640 1 -- 172.21.15.119:0/929417693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbef40d1650 con 0x7fbef415aa60 2024-04-25T10:11:47.341 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:11:47.336+0000 7f8c64600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:11:47.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.339+0000 7fbeeaa00640 1 -- 172.21.15.119:0/929417693 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbeec079420 con 0x7fbef415aa60 2024-04-25T10:11:47.345 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:11:47.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.342+0000 7fbef8a00640 1 -- 172.21.15.119:0/929417693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbed406b500 msgr2=0x7fbed406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.342+0000 7fbef8a00640 1 --2- 172.21.15.119:0/929417693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbed406b500 0x7fbed406d9c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbee4006fd0 tx=0x7fbee4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:47.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.342+0000 7fbef8a00640 1 -- 172.21.15.119:0/929417693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbef415aa60 msgr2=0x7fbef40cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.342+0000 7fbef8a00640 1 --2- 172.21.15.119:0/929417693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbef415aa60 0x7fbef40cc3b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fbeec000f80 tx=0x7fbeec003a70 comp rx=0 tx=0).stop 2024-04-25T10:11:47.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.343+0000 7fbef8a00640 1 -- 172.21.15.119:0/929417693 shutdown_connections 2024-04-25T10:11:47.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.343+0000 7fbef8a00640 1 --2- 172.21.15.119:0/929417693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbed406b500 0x7fbed406d9c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.343+0000 7fbef8a00640 1 --2- 172.21.15.119:0/929417693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbef415aa60 0x7fbef40cc3b0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.343+0000 7fbef8a00640 1 --2- 172.21.15.119:0/929417693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbef415a600 0x7fbef40cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.343+0000 7fbef8a00640 1 --2- 172.21.15.119:0/929417693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbed80052e0 0x7fbef41f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.343+0000 7fbef8a00640 1 -- 172.21.15.119:0/929417693 >> 172.21.15.119:0/929417693 conn(0x7fbef40722d0 msgr2=0x7fbef40d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:47.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.343+0000 7fbef8a00640 1 -- 172.21.15.119:0/929417693 shutdown_connections 2024-04-25T10:11:47.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.343+0000 7fbef8a00640 1 -- 172.21.15.119:0/929417693 wait complete. 2024-04-25T10:11:47.360 INFO:tasks.ceph.mds.c.smithi119.stderr: -4447> 2024-04-25T10:11:39.639+0000 7fec44c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:47.368 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T10:11:47.428 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2024-04-25T10:11:47.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.448+0000 7f0f42154f40 1 Processor -- start 2024-04-25T10:11:47.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.448+0000 7f0f42154f40 1 -- start start 2024-04-25T10:11:47.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.448+0000 7f0f42154f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f240030f0 0x5585ecfb4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.448+0000 7f0f42154f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ed07faf0 0x5585ed07efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.448+0000 7f0f42154f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5585ed003700 0x5585ed08f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.448+0000 7f0f42154f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ecf5cdd0 con 0x5585ed07faf0 2024-04-25T10:11:47.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.448+0000 7f0f42154f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ecf60d30 con 0x7f0f240030f0 2024-04-25T10:11:47.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.448+0000 7f0f42154f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585ecfe7db0 con 0x5585ed003700 2024-04-25T10:11:47.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.449+0000 7f0f3ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ed07faf0 0x5585ed07efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.449+0000 7f0f3ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ed07faf0 0x5585ed07efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55862/0 (socket says 172.21.15.119:55862) 2024-04-25T10:11:47.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.449+0000 7f0f3ea00640 1 -- 172.21.15.119:0/306059473 learned_addr learned my addr 172.21.15.119:0/306059473 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:47.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.449+0000 7f0f3ea00640 1 -- 172.21.15.119:0/306059473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5585ed003700 msgr2=0x5585ed08f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.449+0000 7f0f3ea00640 1 --2- 172.21.15.119:0/306059473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5585ed003700 0x5585ed08f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.449+0000 7f0f3f400640 1 --2- 172.21.15.119:0/306059473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f240030f0 0x5585ecfb4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.449+0000 7f0f3ea00640 1 -- 172.21.15.119:0/306059473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f240030f0 msgr2=0x5585ecfb4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.449+0000 7f0f3ea00640 1 --2- 172.21.15.119:0/306059473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f240030f0 0x5585ecfb4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.449+0000 7f0f3ea00640 1 -- 172.21.15.119:0/306059473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585ecfb2f80 con 0x5585ed07faf0 2024-04-25T10:11:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.449+0000 7f0f3ea00640 1 --2- 172.21.15.119:0/306059473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ed07faf0 0x5585ed07efe0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c001970 tx=0x7f0f2c04f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228428ab9b042ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.450+0000 7f0f3d600640 1 -- 172.21.15.119:0/306059473 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c001570 con 0x5585ed07faf0 2024-04-25T10:11:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.450+0000 7f0f3d600640 1 -- 172.21.15.119:0/306059473 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2c04d9e0 con 0x5585ed07faf0 2024-04-25T10:11:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.450+0000 7f0f3d600640 1 -- 172.21.15.119:0/306059473 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c04db90 con 0x5585ed07faf0 2024-04-25T10:11:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.450+0000 7f0f42154f40 1 -- 172.21.15.119:0/306059473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ed07faf0 msgr2=0x5585ed07efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.450+0000 7f0f42154f40 1 --2- 172.21.15.119:0/306059473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ed07faf0 0x5585ed07efe0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c001970 tx=0x7f0f2c04f000 comp rx=0 tx=0).stop 2024-04-25T10:11:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.450+0000 7f0f42154f40 1 -- 172.21.15.119:0/306059473 shutdown_connections 2024-04-25T10:11:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.450+0000 7f0f42154f40 1 --2- 172.21.15.119:0/306059473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5585ed003700 0x5585ed08f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.450+0000 7f0f42154f40 1 --2- 172.21.15.119:0/306059473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5585ed07faf0 0x5585ed07efe0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.450+0000 7f0f42154f40 1 --2- 172.21.15.119:0/306059473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f240030f0 0x5585ecfb4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.450+0000 7f0f42154f40 1 -- 172.21.15.119:0/306059473 >> 172.21.15.119:0/306059473 conn(0x5585ed003070 msgr2=0x5585ed088990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.450+0000 7f0f42154f40 1 -- 172.21.15.119:0/306059473 shutdown_connections 2024-04-25T10:11:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.450+0000 7f0f42154f40 1 -- 172.21.15.119:0/306059473 wait complete. 2024-04-25T10:11:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.451+0000 7f0f42154f40 1 Processor -- start 2024-04-25T10:11:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.451+0000 7f0f42154f40 1 -- start start 2024-04-25T10:11:47.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.452+0000 7f0f42154f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5585ed003700 0x5585ed08fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.452+0000 7f0f42154f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5585ed07faf0 0x5585ed072c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.452+0000 7f0f42154f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f240030f0 0x5585ed08d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.452+0000 7f0f42154f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ecfb35e0 con 0x7f0f240030f0 2024-04-25T10:11:47.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.452+0000 7f0f42154f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ed080120 con 0x5585ed003700 2024-04-25T10:11:47.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.452+0000 7f0f42154f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585ecfb4b60 con 0x5585ed07faf0 2024-04-25T10:11:47.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.452+0000 7f0f3fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f240030f0 0x5585ed08d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.452+0000 7f0f3fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f240030f0 0x5585ed08d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55868/0 (socket says 172.21.15.119:55868) 2024-04-25T10:11:47.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.452+0000 7f0f3fe00640 1 -- 172.21.15.119:0/2519803935 learned_addr learned my addr 172.21.15.119:0/2519803935 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.452+0000 7f0f3fe00640 1 -- 172.21.15.119:0/2519803935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5585ed07faf0 msgr2=0x5585ed072c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.452+0000 7f0f3fe00640 1 --2- 172.21.15.119:0/2519803935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5585ed07faf0 0x5585ed072c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.452+0000 7f0f3fe00640 1 -- 172.21.15.119:0/2519803935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5585ed003700 msgr2=0x5585ed08fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.452+0000 7f0f3fe00640 1 --2- 172.21.15.119:0/2519803935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5585ed003700 0x5585ed08fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.452+0000 7f0f3fe00640 1 -- 172.21.15.119:0/2519803935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585ed079c80 con 0x7f0f240030f0 2024-04-25T10:11:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.453+0000 7f0f3fe00640 1 --2- 172.21.15.119:0/2519803935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f240030f0 0x5585ed08d260 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x5585ecfe2680 tx=0x7f0f3800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.453+0000 7f0f37e00640 1 -- 172.21.15.119:0/2519803935 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38015020 con 0x7f0f240030f0 2024-04-25T10:11:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.453+0000 7f0f37e00640 1 -- 172.21.15.119:0/2519803935 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f380040f0 con 0x7f0f240030f0 2024-04-25T10:11:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.453+0000 7f0f42154f40 1 -- 172.21.15.119:0/2519803935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5585ed076ce0 con 0x7f0f240030f0 2024-04-25T10:11:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.453+0000 7f0f37e00640 1 -- 172.21.15.119:0/2519803935 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38011610 con 0x7f0f240030f0 2024-04-25T10:11:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.453+0000 7f0f42154f40 1 -- 172.21.15.119:0/2519803935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585ed0731d0 con 0x7f0f240030f0 2024-04-25T10:11:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.455+0000 7f0f37e00640 1 -- 172.21.15.119:0/2519803935 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f38018020 con 0x7f0f240030f0 2024-04-25T10:11:47.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.456+0000 7f0f37e00640 1 --2- 172.21.15.119:0/2519803935 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f20069eb0 0x7f0f2006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.456+0000 7f0f37e00640 1 -- 172.21.15.119:0/2519803935 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0f380888b0 con 0x7f0f240030f0 2024-04-25T10:11:47.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.456+0000 7f0f3f400640 1 --2- 172.21.15.119:0/2519803935 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f20069eb0 0x7f0f2006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.456+0000 7f0f42154f40 1 --2- 172.21.15.119:0/2519803935 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5585ed08e4d0 0x5585ed07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.456+0000 7f0f42154f40 1 -- 172.21.15.119:0/2519803935 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68) v9 -- 0x5585ed0797a0 con 0x5585ed08e4d0 2024-04-25T10:11:47.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.457+0000 7f0f3f400640 1 --2- 172.21.15.119:0/2519803935 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f20069eb0 0x7f0f2006c370 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0f30005380 tx=0x7f0f30005b80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.457+0000 7f0f3ea00640 1 --2- 172.21.15.119:0/2519803935 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5585ed08e4d0 0x5585ed07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.458+0000 7f0f3ea00640 1 --2- 172.21.15.119:0/2519803935 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5585ed08e4d0 0x5585ed07b2c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.467+0000 7f0f3ea00640 1 -- 172.21.15.119:0/2519803935 <== osd.4 v2:172.21.15.132:6800/1740646323 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v68'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0f2c05b070 con 0x5585ed08e4d0 2024-04-25T10:11:47.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.467+0000 7f0f42154f40 1 -- 172.21.15.119:0/2519803935 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5585ed08e4d0 msgr2=0x5585ed07b2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.467+0000 7f0f42154f40 1 --2- 172.21.15.119:0/2519803935 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5585ed08e4d0 0x5585ed07b2c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.467+0000 7f0f42154f40 1 -- 172.21.15.119:0/2519803935 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f20069eb0 msgr2=0x7f0f2006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.467+0000 7f0f42154f40 1 --2- 172.21.15.119:0/2519803935 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0f20069eb0 0x7f0f2006c370 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0f30005380 tx=0x7f0f30005b80 comp rx=0 tx=0).stop 2024-04-25T10:11:47.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.467+0000 7f0f42154f40 1 -- 172.21.15.119:0/2519803935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f240030f0 msgr2=0x5585ed08d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.467+0000 7f0f42154f40 1 --2- 172.21.15.119:0/2519803935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f240030f0 0x5585ed08d260 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x5585ecfe2680 tx=0x7f0f3800c520 comp rx=0 tx=0).stop 2024-04-25T10:11:47.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.467+0000 7f0f3fe00640 1 -- 172.21.15.119:0/2519803935 reap_dead start 2024-04-25T10:11:47.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.468+0000 7f0f42154f40 1 -- 172.21.15.119:0/2519803935 shutdown_connections 2024-04-25T10:11:47.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.468+0000 7f0f42154f40 1 -- 172.21.15.119:0/2519803935 >> 172.21.15.119:0/2519803935 conn(0x5585ed003070 msgr2=0x5585ed087650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:47.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.468+0000 7f0f42154f40 1 -- 172.21.15.119:0/2519803935 shutdown_connections 2024-04-25T10:11:47.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.468+0000 7f0f42154f40 1 -- 172.21.15.119:0/2519803935 wait complete. 2024-04-25T10:11:47.477 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:11:47.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.651+0000 7f545be00640 1 Processor -- start 2024-04-25T10:11:47.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.652+0000 7f545be00640 1 -- start start 2024-04-25T10:11:47.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.653+0000 7f545be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f545c0ce860 0x7f545c0cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.653+0000 7f545be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0d00a0 0x7f545c0d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.653+0000 7f545be00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c0d0a70 con 0x7f545c15b7e0 2024-04-25T10:11:47.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.653+0000 7f545be00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545c0b5710 con 0x7f545c0ce860 2024-04-25T10:11:47.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.653+0000 7f545be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0b59f0 con 0x7f545c0d00a0 2024-04-25T10:11:47.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.654+0000 7f5452000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f545c0ce860 0x7f545c0cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.654+0000 7f545b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0d00a0 0x7f545c0d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.654+0000 7f545b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0d00a0 0x7f545c0d0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:57048/0 (socket says 172.21.15.119:57048) 2024-04-25T10:11:47.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.654+0000 7f545b400640 1 -- 172.21.15.119:0/2233956091 learned_addr learned my addr 172.21.15.119:0/2233956091 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:47.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.654+0000 7f5452000640 1 -- 172.21.15.119:0/2233956091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0d00a0 msgr2=0x7f545c0d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:47.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.654+0000 7f5452000640 1 --2- 172.21.15.119:0/2233956091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0d00a0 0x7f545c0d0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.654+0000 7f5452000640 1 -- 172.21.15.119:0/2233956091 >> v1:172.21.15.132:6789/0 conn(0x7f545c15b7e0 legacy=0x7f545c15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.655+0000 7f5452000640 1 -- 172.21.15.119:0/2233956091 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c0b5cd0 con 0x7f545c0ce860 2024-04-25T10:11:47.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.656+0000 7f5452000640 1 --2- 172.21.15.119:0/2233956091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f545c0ce860 0x7f545c0cecf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f54440097a0 tx=0x7f544402a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=140eab10cdc77c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.657+0000 7f545a000640 1 -- 172.21.15.119:0/2233956091 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444037070 con 0x7f545c0ce860 2024-04-25T10:11:47.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.657+0000 7f545a000640 1 -- 172.21.15.119:0/2233956091 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544402adb0 con 0x7f545c0ce860 2024-04-25T10:11:47.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.657+0000 7f545a000640 1 -- 172.21.15.119:0/2233956091 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444032660 con 0x7f545c0ce860 2024-04-25T10:11:47.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.657+0000 7f545be00640 1 -- 172.21.15.119:0/2233956091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f545c0ce860 msgr2=0x7f545c0cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.657+0000 7f545be00640 1 --2- 172.21.15.119:0/2233956091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f545c0ce860 0x7f545c0cecf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f54440097a0 tx=0x7f544402a060 comp rx=0 tx=0).stop 2024-04-25T10:11:47.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.657+0000 7f545be00640 1 -- 172.21.15.119:0/2233956091 shutdown_connections 2024-04-25T10:11:47.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.657+0000 7f545be00640 1 --2- 172.21.15.119:0/2233956091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0d00a0 0x7f545c0d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.657+0000 7f545be00640 1 --2- 172.21.15.119:0/2233956091 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f545c0ce860 0x7f545c0cecf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.657+0000 7f545be00640 1 -- 172.21.15.119:0/2233956091 >> 172.21.15.119:0/2233956091 conn(0x7f545c0722d0 msgr2=0x7f545c0726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:47.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.658+0000 7f545be00640 1 -- 172.21.15.119:0/2233956091 shutdown_connections 2024-04-25T10:11:47.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.658+0000 7f545be00640 1 -- 172.21.15.119:0/2233956091 wait complete. 2024-04-25T10:11:47.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.659+0000 7f545be00640 1 Processor -- start 2024-04-25T10:11:47.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.660+0000 7f545be00640 1 -- start start 2024-04-25T10:11:47.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.660+0000 7f545be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0ce860 0x7f545c14d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.660+0000 7f545be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f545c0d00a0 0x7f545c1f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.661+0000 7f545be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c15b7e0 0x7f545c1f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.661+0000 7f545be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c161960 con 0x7f545c15b7e0 2024-04-25T10:11:47.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.661+0000 7f545be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0b53c0 con 0x7f545c0d00a0 2024-04-25T10:11:47.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.661+0000 7f545be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c161ae0 con 0x7f545c0ce860 2024-04-25T10:11:47.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.661+0000 7f545b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c15b7e0 0x7f545c1f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.661+0000 7f545b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c15b7e0 0x7f545c1f7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55872/0 (socket says 172.21.15.119:55872) 2024-04-25T10:11:47.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.661+0000 7f545b400640 1 -- 172.21.15.119:0/2552058376 learned_addr learned my addr 172.21.15.119:0/2552058376 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:47.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.661+0000 7f545aa00640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0ce860 0x7f545c14d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.661+0000 7f5452000640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f545c0d00a0 0x7f545c1f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.661+0000 7f545b400640 1 -- 172.21.15.119:0/2552058376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0ce860 msgr2=0x7f545c14d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.661+0000 7f545b400640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0ce860 0x7f545c14d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.661+0000 7f545b400640 1 -- 172.21.15.119:0/2552058376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f545c0d00a0 msgr2=0x7f545c1f6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.661+0000 7f545b400640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f545c0d00a0 0x7f545c1f6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.662+0000 7f545b400640 1 -- 172.21.15.119:0/2552058376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c14dee0 con 0x7f545c15b7e0 2024-04-25T10:11:47.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.662+0000 7f545aa00640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0ce860 0x7f545c14d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:47.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.662+0000 7f5452000640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f545c0d00a0 0x7f545c1f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:47.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.662+0000 7f545b400640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c15b7e0 0x7f545c1f7060 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f545400efc0 tx=0x7f545400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.662+0000 7f5453400640 1 -- 172.21.15.119:0/2552058376 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545400cd20 con 0x7f545c15b7e0 2024-04-25T10:11:47.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.662+0000 7f5453400640 1 -- 172.21.15.119:0/2552058376 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5454003990 con 0x7f545c15b7e0 2024-04-25T10:11:47.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.662+0000 7f545be00640 1 -- 172.21.15.119:0/2552058376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c15d080 con 0x7f545c15b7e0 2024-04-25T10:11:47.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.662+0000 7f5453400640 1 -- 172.21.15.119:0/2552058376 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454003d80 con 0x7f545c15b7e0 2024-04-25T10:11:47.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.662+0000 7f545be00640 1 -- 172.21.15.119:0/2552058376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545c15d640 con 0x7f545c15b7e0 2024-04-25T10:11:47.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.664+0000 7f545be00640 1 -- 172.21.15.119:0/2552058376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545c0749b0 con 0x7f545c15b7e0 2024-04-25T10:11:47.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.665+0000 7f5453400640 1 -- 172.21.15.119:0/2552058376 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5454012050 con 0x7f545c15b7e0 2024-04-25T10:11:47.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.665+0000 7f5453400640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f543806b4d0 0x7f543806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.666+0000 7f5453400640 1 -- 172.21.15.119:0/2552058376 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f545408a7c0 con 0x7f545c15b7e0 2024-04-25T10:11:47.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.666+0000 7f545aa00640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f543806b4d0 0x7f543806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.666+0000 7f545aa00640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f543806b4d0 0x7f543806d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f544c001660 tx=0x7f544c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.670+0000 7f5453400640 1 -- 172.21.15.119:0/2552058376 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5454010040 con 0x7f545c15b7e0 2024-04-25T10:11:47.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:47.891+0000 7f545be00640 1 -- 172.21.15.119:0/2552058376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f545c0d1e10 con 0x7f545c15b7e0 2024-04-25T10:11:48.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.375+0000 7f5453400640 1 -- 172.21.15.119:0/2552058376 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v46) v1 ==== 162+0+0 (secure 0 0 0) 0x7f54540595e0 con 0x7f545c15b7e0 2024-04-25T10:11:48.380 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:11:48.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.378+0000 7f545be00640 1 -- 172.21.15.119:0/2552058376 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f543806b4d0 msgr2=0x7f543806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.378+0000 7f545be00640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f543806b4d0 0x7f543806d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f544c001660 tx=0x7f544c009040 comp rx=0 tx=0).stop 2024-04-25T10:11:48.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.378+0000 7f545be00640 1 -- 172.21.15.119:0/2552058376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c15b7e0 msgr2=0x7f545c1f7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.378+0000 7f545be00640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c15b7e0 0x7f545c1f7060 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f545400efc0 tx=0x7f545400c490 comp rx=0 tx=0).stop 2024-04-25T10:11:48.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.378+0000 7f545be00640 1 -- 172.21.15.119:0/2552058376 shutdown_connections 2024-04-25T10:11:48.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.378+0000 7f545be00640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f543806b4d0 0x7f543806d990 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.378+0000 7f545be00640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f545c15b7e0 0x7f545c1f7060 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.378+0000 7f545be00640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f545c0d00a0 0x7f545c1f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.378+0000 7f545be00640 1 --2- 172.21.15.119:0/2552058376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f545c0ce860 0x7f545c14d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.378+0000 7f545be00640 1 -- 172.21.15.119:0/2552058376 >> 172.21.15.119:0/2552058376 conn(0x7f545c0722d0 msgr2=0x7f545c15a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:48.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.379+0000 7f545be00640 1 -- 172.21.15.119:0/2552058376 shutdown_connections 2024-04-25T10:11:48.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.379+0000 7f545be00640 1 -- 172.21.15.119:0/2552058376 wait complete. 2024-04-25T10:11:48.409 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:48.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.582+0000 7fa3f9a00640 1 Processor -- start 2024-04-25T10:11:48.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.582+0000 7fa3f9a00640 1 -- start start 2024-04-25T10:11:48.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.584+0000 7fa3f9a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa3f415bf00 0x7fa3f40cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.584+0000 7fa3f9a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3f406e8f0 0x7fa3f41654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.584+0000 7fa3f9a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f40ce6c0 con 0x7fa3f40cc310 2024-04-25T10:11:48.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.584+0000 7fa3f9a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f41455e0 con 0x7fa3f415bf00 2024-04-25T10:11:48.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.584+0000 7fa3f9a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f406d450 con 0x7fa3f406e8f0 2024-04-25T10:11:48.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.584+0000 7fa3f2000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa3f40cc310 0x7fa3f415c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40286/0 (socket says 172.21.15.119:40286) 2024-04-25T10:11:48.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.584+0000 7fa3f2000640 1 -- 172.21.15.119:0/2560492668 learned_addr learned my addr 172.21.15.119:0/2560492668 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:48.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.584+0000 7fa3f2a00640 1 --2- 172.21.15.119:0/2560492668 >> v2:172.21.15.132:3300/0 conn(0x7fa3f415bf00 0x7fa3f40cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f3400640 1 --2- 172.21.15.119:0/2560492668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3f406e8f0 0x7fa3f41654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 -- 172.21.15.119:0/2560492668 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210430853 0 0) 0x7fa3dc001110 con 0x7fa3f40cc310 2024-04-25T10:11:48.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 -- 172.21.15.119:0/2560492668 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3d8003520 con 0x7fa3f40cc310 2024-04-25T10:11:48.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 -- 172.21.15.119:0/2560492668 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1492589379 0 0) 0x7fa3d8003520 con 0x7fa3f40cc310 2024-04-25T10:11:48.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 -- 172.21.15.119:0/2560492668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3f406e8f0 msgr2=0x7fa3f41654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:48.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 --2- 172.21.15.119:0/2560492668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3f406e8f0 0x7fa3f41654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 -- 172.21.15.119:0/2560492668 >> v2:172.21.15.132:3300/0 conn(0x7fa3f415bf00 msgr2=0x7fa3f40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:48.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 --2- 172.21.15.119:0/2560492668 >> v2:172.21.15.132:3300/0 conn(0x7fa3f415bf00 0x7fa3f40cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 -- 172.21.15.119:0/2560492668 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f40cc7b0 con 0x7fa3f40cc310 2024-04-25T10:11:48.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 -- 172.21.15.119:0/2560492668 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fa3dc001840 con 0x7fa3f40cc310 2024-04-25T10:11:48.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 -- 172.21.15.119:0/2560492668 >> v1:172.21.15.119:6789/0 conn(0x7fa3f40cc310 legacy=0x7fa3f415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 --2- 172.21.15.119:0/2560492668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3f406e8f0 0x7fa3d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 --2- 172.21.15.119:0/2560492668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d8003f30 0x7fa3d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 --2- 172.21.15.119:0/2560492668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3d8004d40 0x7fa3d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 -- 172.21.15.119:0/2560492668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f41455e0 con 0x7fa3d8003f30 2024-04-25T10:11:48.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 -- 172.21.15.119:0/2560492668 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f406d450 con 0x7fa3d8004d40 2024-04-25T10:11:48.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.585+0000 7fa3f1600640 1 -- 172.21.15.119:0/2560492668 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d800b9c0 con 0x7fa3f406e8f0 2024-04-25T10:11:48.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.586+0000 7fa3f2000640 1 --2- 172.21.15.119:0/2560492668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d8003f30 0x7fa3d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.586+0000 7fa3f3400640 1 --2- 172.21.15.119:0/2560492668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3d8004d40 0x7fa3d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.586+0000 7fa3f2a00640 1 --2- 172.21.15.119:0/2560492668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3f406e8f0 0x7fa3d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.586+0000 7fa3f2000640 1 -- 172.21.15.119:0/2560492668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3f406e8f0 msgr2=0x7fa3d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:48.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.586+0000 7fa3f2000640 1 --2- 172.21.15.119:0/2560492668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3f406e8f0 0x7fa3d80039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.586+0000 7fa3f2000640 1 -- 172.21.15.119:0/2560492668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3d8004d40 msgr2=0x7fa3d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:48.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.586+0000 7fa3f2000640 1 --2- 172.21.15.119:0/2560492668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3d8004d40 0x7fa3d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.586+0000 7fa3f2000640 1 -- 172.21.15.119:0/2560492668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d8003520 con 0x7fa3d8003f30 2024-04-25T10:11:48.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.586+0000 7fa3f3400640 1 --2- 172.21.15.119:0/2560492668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3d8004d40 0x7fa3d80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:48.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.587+0000 7fa3f2000640 1 --2- 172.21.15.119:0/2560492668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d8003f30 0x7fa3d8004800 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40d06c0 tx=0x7fa3d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d5841f0658f56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.587+0000 7fa3f1600640 1 -- 172.21.15.119:0/2560492668 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc00c0e0 con 0x7fa3d8003f30 2024-04-25T10:11:48.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.587+0000 7fa3f1600640 1 -- 172.21.15.119:0/2560492668 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc034070 con 0x7fa3d8003f30 2024-04-25T10:11:48.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.587+0000 7fa3f1600640 1 -- 172.21.15.119:0/2560492668 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc005d10 con 0x7fa3d8003f30 2024-04-25T10:11:48.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.588+0000 7fa3f9a00640 1 -- 172.21.15.119:0/2560492668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d8003f30 msgr2=0x7fa3d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.588+0000 7fa3f9a00640 1 --2- 172.21.15.119:0/2560492668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3d8003f30 0x7fa3d8004800 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40d06c0 tx=0x7fa3d8005830 comp rx=0 tx=0).stop 2024-04-25T10:11:48.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.588+0000 7fa3f3400640 1 -- 172.21.15.119:0/2560492668 reap_dead start 2024-04-25T10:11:48.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.588+0000 7fa3f9a00640 1 -- 172.21.15.119:0/2560492668 shutdown_connections 2024-04-25T10:11:48.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.588+0000 7fa3f9a00640 1 -- 172.21.15.119:0/2560492668 >> 172.21.15.119:0/2560492668 conn(0x7fa3f406aba0 msgr2=0x7fa3f406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:48.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.588+0000 7fa3f9a00640 1 -- 172.21.15.119:0/2560492668 shutdown_connections 2024-04-25T10:11:48.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.588+0000 7fa3f9a00640 1 -- 172.21.15.119:0/2560492668 wait complete. 2024-04-25T10:11:48.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.590+0000 7fa3f9a00640 1 Processor -- start 2024-04-25T10:11:48.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.590+0000 7fa3f9a00640 1 -- start start 2024-04-25T10:11:48.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.591+0000 7fa3f9a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3d8003f30 0x7fa3f41e5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.591+0000 7fa3f9a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f41e5d60 0x7fa3f41f5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.591+0000 7fa3f9a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f41f57d0 0x7fa3f41f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.591+0000 7fa3f9a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40cd050 con 0x7fa3f41e5d60 2024-04-25T10:11:48.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.591+0000 7fa3f9a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40ccd70 con 0x7fa3f41f57d0 2024-04-25T10:11:48.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.591+0000 7fa3f9a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f406ee60 con 0x7fa3d8003f30 2024-04-25T10:11:48.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.591+0000 7fa3f2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f41e5d60 0x7fa3f41f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.591+0000 7fa3f2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f41e5d60 0x7fa3f41f5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55882/0 (socket says 172.21.15.119:55882) 2024-04-25T10:11:48.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.591+0000 7fa3f2000640 1 -- 172.21.15.119:0/3873385621 learned_addr learned my addr 172.21.15.119:0/3873385621 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:48.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.592+0000 7fa3f2a00640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3d8003f30 0x7fa3f41e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.592+0000 7fa3f2000640 1 -- 172.21.15.119:0/3873385621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3d8003f30 msgr2=0x7fa3f41e5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.592+0000 7fa3f3400640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f41f57d0 0x7fa3f41f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.592+0000 7fa3f2000640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3d8003f30 0x7fa3f41e5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.592+0000 7fa3f2000640 1 -- 172.21.15.119:0/3873385621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f41f57d0 msgr2=0x7fa3f41f19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.592+0000 7fa3f2000640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f41f57d0 0x7fa3f41f19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.592+0000 7fa3f2000640 1 -- 172.21.15.119:0/3873385621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f41e75b0 con 0x7fa3f41e5d60 2024-04-25T10:11:48.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.592+0000 7fa3f2a00640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3d8003f30 0x7fa3f41e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:48.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.592+0000 7fa3f2000640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f41e5d60 0x7fa3f41f5290 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc02afb0 tx=0x7fa3f40cc980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.592+0000 7fa3eaa00640 1 -- 172.21.15.119:0/3873385621 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc004cb0 con 0x7fa3f41e5d60 2024-04-25T10:11:48.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.592+0000 7fa3eaa00640 1 -- 172.21.15.119:0/3873385621 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc004e50 con 0x7fa3f41e5d60 2024-04-25T10:11:48.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.592+0000 7fa3f9a00640 1 -- 172.21.15.119:0/3873385621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f41e6510 con 0x7fa3f41e5d60 2024-04-25T10:11:48.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.592+0000 7fa3eaa00640 1 -- 172.21.15.119:0/3873385621 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc0072b0 con 0x7fa3f41e5d60 2024-04-25T10:11:48.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.592+0000 7fa3f9a00640 1 -- 172.21.15.119:0/3873385621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f41e6ad0 con 0x7fa3f41e5d60 2024-04-25T10:11:48.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.593+0000 7fa3f9a00640 1 -- 172.21.15.119:0/3873385621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f40749b0 con 0x7fa3f41e5d60 2024-04-25T10:11:48.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.594+0000 7fa3eaa00640 1 -- 172.21.15.119:0/3873385621 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3dc007450 con 0x7fa3f41e5d60 2024-04-25T10:11:48.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.595+0000 7fa3eaa00640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3d006b4d0 0x7fa3d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:48.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.595+0000 7fa3eaa00640 1 -- 172.21.15.119:0/3873385621 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa3dc0aa7a0 con 0x7fa3f41e5d60 2024-04-25T10:11:48.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.596+0000 7fa3f2a00640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3d006b4d0 0x7fa3d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:48.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.596+0000 7fa3f2a00640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3d006b4d0 0x7fa3d006d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4007460 tx=0x7fa3e400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:48.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.600+0000 7fa3eaa00640 1 -- 172.21.15.119:0/3873385621 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3dc049020 con 0x7fa3f41e5d60 2024-04-25T10:11:48.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.836+0000 7fa3f9a00640 1 -- 172.21.15.119:0/3873385621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3f406b3c0 con 0x7fa3f41e5d60 2024-04-25T10:11:48.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.837+0000 7fa3eaa00640 1 -- 172.21.15.119:0/3873385621 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fa3dc040070 con 0x7fa3f41e5d60 2024-04-25T10:11:48.843 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:48.843 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2798722683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2798722683},{"type":"v1","addr":"172.21.15.132:6833","nonce":2798722683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2834911408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2834911408},{"type":"v1","addr":"172.21.15.132:6835","nonce":2834911408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/643040335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":643040335},{"type":"v1","addr":"172.21.15.119:6835","nonce":643040335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:34.712290+0000","modified":"2024-04-25T10:11:48.380782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":47,"state":"up:replay","state_seq":1,"addr":"172.21.15.119:6837/2127292269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2127292269},{"type":"v1","addr":"172.21.15.119:6837","nonce":2127292269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:48.843 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 47 2024-04-25T10:11:48.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.840+0000 7fa3f9a00640 1 -- 172.21.15.119:0/3873385621 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3d006b4d0 msgr2=0x7fa3d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.840+0000 7fa3f9a00640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3d006b4d0 0x7fa3d006d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4007460 tx=0x7fa3e400b040 comp rx=0 tx=0).stop 2024-04-25T10:11:48.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.840+0000 7fa3f9a00640 1 -- 172.21.15.119:0/3873385621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f41e5d60 msgr2=0x7fa3f41f5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:48.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.840+0000 7fa3f9a00640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f41e5d60 0x7fa3f41f5290 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc02afb0 tx=0x7fa3f40cc980 comp rx=0 tx=0).stop 2024-04-25T10:11:48.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.840+0000 7fa3f9a00640 1 -- 172.21.15.119:0/3873385621 shutdown_connections 2024-04-25T10:11:48.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.840+0000 7fa3f9a00640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3d006b4d0 0x7fa3d006d990 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.840+0000 7fa3f9a00640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3f41f57d0 0x7fa3f41f19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.840+0000 7fa3f9a00640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f41e5d60 0x7fa3f41f5290 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.840+0000 7fa3f9a00640 1 --2- 172.21.15.119:0/3873385621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3d8003f30 0x7fa3f41e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:48.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.840+0000 7fa3f9a00640 1 -- 172.21.15.119:0/3873385621 >> 172.21.15.119:0/3873385621 conn(0x7fa3f406aba0 msgr2=0x7fa3f40cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:48.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.840+0000 7fa3f9a00640 1 -- 172.21.15.119:0/3873385621 shutdown_connections 2024-04-25T10:11:48.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:48.840+0000 7fa3f9a00640 1 -- 172.21.15.119:0/3873385621 wait complete. 2024-04-25T10:11:49.269 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:34.712290+0000', 'modified': '2024-04-25T10:11:48.380782+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6699}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6699': {'gid': 6699, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.119:6837/2127292269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 2127292269}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2127292269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:49.269 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6699:up:replay 2024-04-25T10:11:50.270 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:50.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.482+0000 7fc42a200640 1 Processor -- start 2024-04-25T10:11:50.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.483+0000 7fc42a200640 1 -- start start 2024-04-25T10:11:50.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.484+0000 7fc42a200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc42415bf00 0x7fc4240cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.484+0000 7fc42a200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4240cc310 0x7fc42415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.484+0000 7fc42a200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42415c820 0x7fc42406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.484+0000 7fc42a200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4241455e0 con 0x7fc4240cc310 2024-04-25T10:11:50.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.484+0000 7fc42a200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc42406d450 con 0x7fc42415bf00 2024-04-25T10:11:50.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.484+0000 7fc42a200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240ce6c0 con 0x7fc42415c820 2024-04-25T10:11:50.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.484+0000 7fc41aa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4240cc310 0x7fc42415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.485+0000 7fc41aa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4240cc310 0x7fc42415c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55892/0 (socket says 172.21.15.119:55892) 2024-04-25T10:11:50.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.485+0000 7fc41aa00640 1 -- 172.21.15.119:0/2122175633 learned_addr learned my addr 172.21.15.119:0/2122175633 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.485+0000 7fc423400640 1 --2- 172.21.15.119:0/2122175633 >> v2:172.21.15.132:3300/0 conn(0x7fc42415bf00 0x7fc4240cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.485+0000 7fc423e00640 1 --2- 172.21.15.119:0/2122175633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42415c820 0x7fc42406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.485+0000 7fc41aa00640 1 -- 172.21.15.119:0/2122175633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42415c820 msgr2=0x7fc42406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.485+0000 7fc41aa00640 1 --2- 172.21.15.119:0/2122175633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42415c820 0x7fc42406d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.485+0000 7fc41aa00640 1 -- 172.21.15.119:0/2122175633 >> v2:172.21.15.132:3300/0 conn(0x7fc42415bf00 msgr2=0x7fc4240cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.485+0000 7fc41aa00640 1 --2- 172.21.15.119:0/2122175633 >> v2:172.21.15.132:3300/0 conn(0x7fc42415bf00 0x7fc4240cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.485+0000 7fc41aa00640 1 -- 172.21.15.119:0/2122175633 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4240cc7b0 con 0x7fc4240cc310 2024-04-25T10:11:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.485+0000 7fc423e00640 1 --2- 172.21.15.119:0/2122175633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42415c820 0x7fc42406d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.486+0000 7fc41aa00640 1 --2- 172.21.15.119:0/2122175633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4240cc310 0x7fc42415c2e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc42406ba10 tx=0x7fc40c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b9aca25f0575a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.486+0000 7fc422a00640 1 -- 172.21.15.119:0/2122175633 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c037070 con 0x7fc4240cc310 2024-04-25T10:11:50.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.486+0000 7fc422a00640 1 -- 172.21.15.119:0/2122175633 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c032440 con 0x7fc4240cc310 2024-04-25T10:11:50.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.486+0000 7fc422a00640 1 -- 172.21.15.119:0/2122175633 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c032810 con 0x7fc4240cc310 2024-04-25T10:11:50.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.487+0000 7fc42a200640 1 -- 172.21.15.119:0/2122175633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4240cc310 msgr2=0x7fc42415c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.487+0000 7fc42a200640 1 --2- 172.21.15.119:0/2122175633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4240cc310 0x7fc42415c2e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc42406ba10 tx=0x7fc40c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.487+0000 7fc42a200640 1 -- 172.21.15.119:0/2122175633 shutdown_connections 2024-04-25T10:11:50.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.487+0000 7fc42a200640 1 --2- 172.21.15.119:0/2122175633 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc42415c820 0x7fc42406d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.487+0000 7fc42a200640 1 --2- 172.21.15.119:0/2122175633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4240cc310 0x7fc42415c2e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.487+0000 7fc42a200640 1 --2- 172.21.15.119:0/2122175633 >> v2:172.21.15.132:3300/0 conn(0x7fc42415bf00 0x7fc4240cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.487+0000 7fc42a200640 1 -- 172.21.15.119:0/2122175633 >> 172.21.15.119:0/2122175633 conn(0x7fc42406aba0 msgr2=0x7fc42406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.487+0000 7fc42a200640 1 -- 172.21.15.119:0/2122175633 shutdown_connections 2024-04-25T10:11:50.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.487+0000 7fc42a200640 1 -- 172.21.15.119:0/2122175633 wait complete. 2024-04-25T10:11:50.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.489+0000 7fc42a200640 1 Processor -- start 2024-04-25T10:11:50.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.489+0000 7fc42a200640 1 -- start start 2024-04-25T10:11:50.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.490+0000 7fc42a200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4240cc310 0x7fc424158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.490+0000 7fc42a200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc42415bf00 0x7fc4240ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.490+0000 7fc42a200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc42415c820 0x7fc42413b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.490+0000 7fc42a200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240cd050 con 0x7fc42415c820 2024-04-25T10:11:50.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.490+0000 7fc42a200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240ccd70 con 0x7fc42415bf00 2024-04-25T10:11:50.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.490+0000 7fc42a200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42415ccd0 con 0x7fc4240cc310 2024-04-25T10:11:50.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.491+0000 7fc423e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc42415c820 0x7fc42413b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.491+0000 7fc423e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc42415c820 0x7fc42413b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55908/0 (socket says 172.21.15.119:55908) 2024-04-25T10:11:50.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.491+0000 7fc423e00640 1 -- 172.21.15.119:0/3566106245 learned_addr learned my addr 172.21.15.119:0/3566106245 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:50.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.491+0000 7fc41aa00640 1 --2- 172.21.15.119:0/3566106245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc42415bf00 0x7fc4240ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.491+0000 7fc423400640 1 --2- 172.21.15.119:0/3566106245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4240cc310 0x7fc424158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.491+0000 7fc423e00640 1 -- 172.21.15.119:0/3566106245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4240cc310 msgr2=0x7fc424158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.491+0000 7fc423e00640 1 --2- 172.21.15.119:0/3566106245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4240cc310 0x7fc424158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.491+0000 7fc423e00640 1 -- 172.21.15.119:0/3566106245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc42415bf00 msgr2=0x7fc4240ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.491+0000 7fc423e00640 1 --2- 172.21.15.119:0/3566106245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc42415bf00 0x7fc4240ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.491+0000 7fc423e00640 1 -- 172.21.15.119:0/3566106245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42413b5e0 con 0x7fc42415c820 2024-04-25T10:11:50.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.492+0000 7fc423e00640 1 --2- 172.21.15.119:0/3566106245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc42415c820 0x7fc42413b0a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00cd50 tx=0x7fc41c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.492+0000 7fc41be00640 1 -- 172.21.15.119:0/3566106245 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00f680 con 0x7fc42415c820 2024-04-25T10:11:50.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.492+0000 7fc41be00640 1 -- 172.21.15.119:0/3566106245 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c00f840 con 0x7fc42415c820 2024-04-25T10:11:50.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.492+0000 7fc42a200640 1 -- 172.21.15.119:0/3566106245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc424159a90 con 0x7fc42415c820 2024-04-25T10:11:50.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.492+0000 7fc41be00640 1 -- 172.21.15.119:0/3566106245 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00fc30 con 0x7fc42415c820 2024-04-25T10:11:50.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.492+0000 7fc42a200640 1 -- 172.21.15.119:0/3566106245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4240c9910 con 0x7fc42415c820 2024-04-25T10:11:50.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.493+0000 7fc42a200640 1 -- 172.21.15.119:0/3566106245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42406bb30 con 0x7fc42415c820 2024-04-25T10:11:50.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.494+0000 7fc41be00640 1 -- 172.21.15.119:0/3566106245 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc41c0213f0 con 0x7fc42415c820 2024-04-25T10:11:50.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.495+0000 7fc41be00640 1 --2- 172.21.15.119:0/3566106245 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc40006b4d0 0x7fc40006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.495+0000 7fc41be00640 1 -- 172.21.15.119:0/3566106245 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc41c08af70 con 0x7fc42415c820 2024-04-25T10:11:50.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.495+0000 7fc423400640 1 --2- 172.21.15.119:0/3566106245 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc40006b4d0 0x7fc40006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.496+0000 7fc423400640 1 --2- 172.21.15.119:0/3566106245 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc40006b4d0 0x7fc40006d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc414006fd0 tx=0x7fc414008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.499+0000 7fc41be00640 1 -- 172.21.15.119:0/3566106245 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc42413b5e0 con 0x7fc42415c820 2024-04-25T10:11:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.730+0000 7fc42a200640 1 -- 172.21.15.119:0/3566106245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4240749b0 con 0x7fc42415c820 2024-04-25T10:11:50.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.731+0000 7fc41be00640 1 -- 172.21.15.119:0/3566106245 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fc41c014030 con 0x7fc42415c820 2024-04-25T10:11:50.736 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:50.736 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2798722683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2798722683},{"type":"v1","addr":"172.21.15.132:6833","nonce":2798722683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2834911408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2834911408},{"type":"v1","addr":"172.21.15.132:6835","nonce":2834911408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/643040335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":643040335},{"type":"v1","addr":"172.21.15.119:6835","nonce":643040335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:34.712290+0000","modified":"2024-04-25T10:11:49.395182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":47,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.119:6837/2127292269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2127292269},{"type":"v1","addr":"172.21.15.119:6837","nonce":2127292269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:50.737 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 49 2024-04-25T10:11:50.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.734+0000 7fc42a200640 1 -- 172.21.15.119:0/3566106245 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc40006b4d0 msgr2=0x7fc40006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.734+0000 7fc42a200640 1 --2- 172.21.15.119:0/3566106245 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc40006b4d0 0x7fc40006d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc414006fd0 tx=0x7fc414008040 comp rx=0 tx=0).stop 2024-04-25T10:11:50.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.734+0000 7fc42a200640 1 -- 172.21.15.119:0/3566106245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc42415c820 msgr2=0x7fc42413b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.734+0000 7fc42a200640 1 --2- 172.21.15.119:0/3566106245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc42415c820 0x7fc42413b0a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00cd50 tx=0x7fc41c00a650 comp rx=0 tx=0).stop 2024-04-25T10:11:50.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.734+0000 7fc42a200640 1 -- 172.21.15.119:0/3566106245 shutdown_connections 2024-04-25T10:11:50.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.734+0000 7fc42a200640 1 --2- 172.21.15.119:0/3566106245 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc40006b4d0 0x7fc40006d990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.734+0000 7fc42a200640 1 --2- 172.21.15.119:0/3566106245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc42415c820 0x7fc42413b0a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.734+0000 7fc42a200640 1 --2- 172.21.15.119:0/3566106245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc42415bf00 0x7fc4240ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.734+0000 7fc42a200640 1 --2- 172.21.15.119:0/3566106245 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc4240cc310 0x7fc424158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.734+0000 7fc42a200640 1 -- 172.21.15.119:0/3566106245 >> 172.21.15.119:0/3566106245 conn(0x7fc42406aba0 msgr2=0x7fc4240cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.734+0000 7fc42a200640 1 -- 172.21.15.119:0/3566106245 shutdown_connections 2024-04-25T10:11:50.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:50.734+0000 7fc42a200640 1 -- 172.21.15.119:0/3566106245 wait complete. 2024-04-25T10:11:51.161 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:34.712290+0000', 'modified': '2024-04-25T10:11:49.395182+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6699}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6699': {'gid': 6699, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.119:6837/2127292269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 2127292269}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2127292269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:51.161 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6699:up:rejoin 2024-04-25T10:11:52.162 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:52.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.362+0000 7f1b31600640 1 Processor -- start 2024-04-25T10:11:52.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.362+0000 7f1b31600640 1 -- start start 2024-04-25T10:11:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.363+0000 7f1b31600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1b2c144470 0x7f1b2c15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.363+0000 7f1b31600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b2c06e430 0x7f1b2c0b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.363+0000 7f1b31600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b2c0b4e80 0x7f1b2c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.363+0000 7f1b31600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b2c145340 con 0x7f1b2c0b4e80 2024-04-25T10:11:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.363+0000 7f1b31600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b2c0cd440 con 0x7f1b2c144470 2024-04-25T10:11:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.363+0000 7f1b31600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c0cd720 con 0x7f1b2c06e430 2024-04-25T10:11:52.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.364+0000 7f1b30c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b2c0b4e80 0x7f1b2c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.364+0000 7f1b30c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b2c0b4e80 0x7f1b2c06d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34920/0 (socket says 172.21.15.119:34920) 2024-04-25T10:11:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.364+0000 7f1b30c00640 1 -- 172.21.15.119:0/705486571 learned_addr learned my addr 172.21.15.119:0/705486571 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.364+0000 7f1b2b400640 1 --2- 172.21.15.119:0/705486571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b2c06e430 0x7f1b2c0b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.364+0000 7f1b30c00640 1 -- 172.21.15.119:0/705486571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b2c06e430 msgr2=0x7f1b2c0b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.364+0000 7f1b30c00640 1 --2- 172.21.15.119:0/705486571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b2c06e430 0x7f1b2c0b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.364+0000 7f1b30c00640 1 -- 172.21.15.119:0/705486571 >> v2:172.21.15.132:3300/0 conn(0x7f1b2c144470 msgr2=0x7f1b2c15fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.364+0000 7f1b30c00640 1 --2- 172.21.15.119:0/705486571 >> v2:172.21.15.132:3300/0 conn(0x7f1b2c144470 0x7f1b2c15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.364+0000 7f1b30c00640 1 -- 172.21.15.119:0/705486571 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2c0cda00 con 0x7f1b2c0b4e80 2024-04-25T10:11:52.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.365+0000 7f1b30c00640 1 --2- 172.21.15.119:0/705486571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b2c0b4e80 0x7f1b2c06d210 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0b5490 tx=0x7f1b2402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91de040ac6412d59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.366+0000 7f1b2aa00640 1 -- 172.21.15.119:0/705486571 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b24036070 con 0x7f1b2c0b4e80 2024-04-25T10:11:52.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.366+0000 7f1b2aa00640 1 -- 172.21.15.119:0/705486571 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b24031600 con 0x7f1b2c0b4e80 2024-04-25T10:11:52.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.366+0000 7f1b2aa00640 1 -- 172.21.15.119:0/705486571 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b240319d0 con 0x7f1b2c0b4e80 2024-04-25T10:11:52.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.366+0000 7f1b31600640 1 -- 172.21.15.119:0/705486571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b2c0b4e80 msgr2=0x7f1b2c06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.366+0000 7f1b31600640 1 --2- 172.21.15.119:0/705486571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b2c0b4e80 0x7f1b2c06d210 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0b5490 tx=0x7f1b2402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.366+0000 7f1b31600640 1 -- 172.21.15.119:0/705486571 shutdown_connections 2024-04-25T10:11:52.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.366+0000 7f1b31600640 1 --2- 172.21.15.119:0/705486571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b2c0b4e80 0x7f1b2c06d210 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.366+0000 7f1b31600640 1 --2- 172.21.15.119:0/705486571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b2c06e430 0x7f1b2c0b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.366+0000 7f1b31600640 1 --2- 172.21.15.119:0/705486571 >> v2:172.21.15.132:3300/0 conn(0x7f1b2c144470 0x7f1b2c15fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.366+0000 7f1b31600640 1 -- 172.21.15.119:0/705486571 >> 172.21.15.119:0/705486571 conn(0x7f1b2c06b1d0 msgr2=0x7f1b2c0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.367+0000 7f1b31600640 1 -- 172.21.15.119:0/705486571 shutdown_connections 2024-04-25T10:11:52.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.367+0000 7f1b31600640 1 -- 172.21.15.119:0/705486571 wait complete. 2024-04-25T10:11:52.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.368+0000 7f1b31600640 1 Processor -- start 2024-04-25T10:11:52.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.369+0000 7f1b31600640 1 -- start start 2024-04-25T10:11:52.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.369+0000 7f1b31600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b2c06e430 0x7f1b2c13ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.369+0000 7f1b31600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b2c0b4e80 0x7f1b2c135df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.369+0000 7f1b31600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b2c144470 0x7f1b2c139b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.369+0000 7f1b31600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c0ce200 con 0x7f1b2c144470 2024-04-25T10:11:52.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.369+0000 7f1b31600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c144850 con 0x7f1b2c0b4e80 2024-04-25T10:11:52.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.369+0000 7f1b31600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c06d750 con 0x7f1b2c06e430 2024-04-25T10:11:52.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.370+0000 7f1b30c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b2c144470 0x7f1b2c139b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.370+0000 7f1b30c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b2c144470 0x7f1b2c139b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34926/0 (socket says 172.21.15.119:34926) 2024-04-25T10:11:52.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.370+0000 7f1b30c00640 1 -- 172.21.15.119:0/2638550811 learned_addr learned my addr 172.21.15.119:0/2638550811 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:52.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.370+0000 7f1b2b400640 1 --2- 172.21.15.119:0/2638550811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b2c0b4e80 0x7f1b2c135df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.370+0000 7f1b2be00640 1 --2- 172.21.15.119:0/2638550811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b2c06e430 0x7f1b2c13ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.370+0000 7f1b30c00640 1 -- 172.21.15.119:0/2638550811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b2c06e430 msgr2=0x7f1b2c13ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.370+0000 7f1b30c00640 1 --2- 172.21.15.119:0/2638550811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b2c06e430 0x7f1b2c13ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.370+0000 7f1b30c00640 1 -- 172.21.15.119:0/2638550811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b2c0b4e80 msgr2=0x7f1b2c135df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.370+0000 7f1b30c00640 1 --2- 172.21.15.119:0/2638550811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b2c0b4e80 0x7f1b2c135df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.370+0000 7f1b30c00640 1 -- 172.21.15.119:0/2638550811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2c0acc30 con 0x7f1b2c144470 2024-04-25T10:11:52.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.371+0000 7f1b30c00640 1 --2- 172.21.15.119:0/2638550811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b2c144470 0x7f1b2c139b40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009f40 tx=0x7f1b24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.371+0000 7f1b23e00640 1 -- 172.21.15.119:0/2638550811 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b24041020 con 0x7f1b2c144470 2024-04-25T10:11:52.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.371+0000 7f1b23e00640 1 -- 172.21.15.119:0/2638550811 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b24007780 con 0x7f1b2c144470 2024-04-25T10:11:52.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.371+0000 7f1b23e00640 1 -- 172.21.15.119:0/2638550811 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b24036040 con 0x7f1b2c144470 2024-04-25T10:11:52.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.371+0000 7f1b31600640 1 -- 172.21.15.119:0/2638550811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2c0acf10 con 0x7f1b2c144470 2024-04-25T10:11:52.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.371+0000 7f1b31600640 1 -- 172.21.15.119:0/2638550811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2c0b3970 con 0x7f1b2c144470 2024-04-25T10:11:52.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.373+0000 7f1b31600640 1 -- 172.21.15.119:0/2638550811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2c0749b0 con 0x7f1b2c144470 2024-04-25T10:11:52.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.373+0000 7f1b23e00640 1 -- 172.21.15.119:0/2638550811 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b24044020 con 0x7f1b2c144470 2024-04-25T10:11:52.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.374+0000 7f1b23e00640 1 --2- 172.21.15.119:0/2638550811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b0c06b4d0 0x7f1b0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.374+0000 7f1b23e00640 1 -- 172.21.15.119:0/2638550811 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1b240aaf10 con 0x7f1b2c144470 2024-04-25T10:11:52.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.375+0000 7f1b2be00640 1 --2- 172.21.15.119:0/2638550811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b0c06b4d0 0x7f1b0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.375+0000 7f1b2be00640 1 --2- 172.21.15.119:0/2638550811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b0c06b4d0 0x7f1b0c06d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c13b490 tx=0x7f1b1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.379+0000 7f1b23e00640 1 -- 172.21.15.119:0/2638550811 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b2404b030 con 0x7f1b2c144470 2024-04-25T10:11:52.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.616+0000 7f1b31600640 1 -- 172.21.15.119:0/2638550811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b2c0b4720 con 0x7f1b2c144470 2024-04-25T10:11:52.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.617+0000 7f1b23e00640 1 -- 172.21.15.119:0/2638550811 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f1b24079e50 con 0x7f1b2c144470 2024-04-25T10:11:52.622 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:52.623 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2798722683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2798722683},{"type":"v1","addr":"172.21.15.132:6833","nonce":2798722683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2834911408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2834911408},{"type":"v1","addr":"172.21.15.132:6835","nonce":2834911408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/643040335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":643040335},{"type":"v1","addr":"172.21.15.119:6835","nonce":643040335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:34.712290+0000","modified":"2024-04-25T10:11:50.411448+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/2127292269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2127292269},{"type":"v1","addr":"172.21.15.119:6837","nonce":2127292269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:52.623 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 50 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.620+0000 7f1b31600640 1 -- 172.21.15.119:0/2638550811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b0c06b4d0 msgr2=0x7f1b0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.620+0000 7f1b31600640 1 --2- 172.21.15.119:0/2638550811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b0c06b4d0 0x7f1b0c06d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c13b490 tx=0x7f1b1c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:52.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.620+0000 7f1b31600640 1 -- 172.21.15.119:0/2638550811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b2c144470 msgr2=0x7f1b2c139b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.620+0000 7f1b31600640 1 --2- 172.21.15.119:0/2638550811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b2c144470 0x7f1b2c139b40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009f40 tx=0x7f1b24004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.620+0000 7f1b31600640 1 -- 172.21.15.119:0/2638550811 shutdown_connections 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.620+0000 7f1b31600640 1 --2- 172.21.15.119:0/2638550811 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b0c06b4d0 0x7f1b0c06d990 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.620+0000 7f1b31600640 1 --2- 172.21.15.119:0/2638550811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b2c144470 0x7f1b2c139b40 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.620+0000 7f1b31600640 1 --2- 172.21.15.119:0/2638550811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b2c0b4e80 0x7f1b2c135df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.620+0000 7f1b31600640 1 --2- 172.21.15.119:0/2638550811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b2c06e430 0x7f1b2c13ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.620+0000 7f1b31600640 1 -- 172.21.15.119:0/2638550811 >> 172.21.15.119:0/2638550811 conn(0x7f1b2c06b1d0 msgr2=0x7f1b2c0afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.620+0000 7f1b31600640 1 -- 172.21.15.119:0/2638550811 shutdown_connections 2024-04-25T10:11:52.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:52.620+0000 7f1b31600640 1 -- 172.21.15.119:0/2638550811 wait complete. 2024-04-25T10:11:53.047 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:34.712290+0000', 'modified': '2024-04-25T10:11:50.411448+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 68, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6699}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6699': {'gid': 6699, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.119:6837/2127292269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 2127292269}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2127292269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:53.047 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:11:53.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:11:53.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.251+0000 7f2ea8e00640 1 Processor -- start 2024-04-25T10:11:53.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.251+0000 7f2ea8e00640 1 -- start start 2024-04-25T10:11:53.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.253+0000 7f2ea8e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2ea41450e0 0x7f2ea406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.253+0000 7f2ea8e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea406b0e0 0x7f2ea40cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.253+0000 7f2ea8e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2ea40cc810 0x7f2ea4071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.253+0000 7f2ea8e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea41454c0 con 0x7f2ea41450e0 2024-04-25T10:11:53.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.253+0000 7f2ea8e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea406b540 con 0x7f2ea40cc810 2024-04-25T10:11:53.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.253+0000 7f2ea8e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea415f400 con 0x7f2ea406b0e0 2024-04-25T10:11:53.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.253+0000 7f2ea3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2ea41450e0 0x7f2ea406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.253+0000 7f2ea3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2ea41450e0 0x7f2ea406aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34928/0 (socket says 172.21.15.119:34928) 2024-04-25T10:11:53.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.253+0000 7f2ea3400640 1 -- 172.21.15.119:0/327045969 learned_addr learned my addr 172.21.15.119:0/327045969 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:53.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.253+0000 7f2ea3e00640 1 --2- 172.21.15.119:0/327045969 >> v2:172.21.15.132:3300/0 conn(0x7f2ea40cc810 0x7f2ea4071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.253+0000 7f2ea2a00640 1 --2- 172.21.15.119:0/327045969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea406b0e0 0x7f2ea40cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.254+0000 7f2ea3400640 1 -- 172.21.15.119:0/327045969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea406b0e0 msgr2=0x7f2ea40cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.254+0000 7f2ea3400640 1 --2- 172.21.15.119:0/327045969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea406b0e0 0x7f2ea40cc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.254+0000 7f2ea3400640 1 -- 172.21.15.119:0/327045969 >> v2:172.21.15.132:3300/0 conn(0x7f2ea40cc810 msgr2=0x7f2ea4071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.254+0000 7f2ea3400640 1 --2- 172.21.15.119:0/327045969 >> v2:172.21.15.132:3300/0 conn(0x7f2ea40cc810 0x7f2ea4071050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.254+0000 7f2ea3400640 1 -- 172.21.15.119:0/327045969 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea415f6e0 con 0x7f2ea41450e0 2024-04-25T10:11:53.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.255+0000 7f2ea3400640 1 --2- 172.21.15.119:0/327045969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea41450e0 0x7f2ea406aba0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40cbe90 tx=0x7f2e9402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60c9de83476b8b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.255+0000 7f2ea2000640 1 -- 172.21.15.119:0/327045969 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94037070 con 0x7f2ea41450e0 2024-04-25T10:11:53.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.255+0000 7f2ea2000640 1 -- 172.21.15.119:0/327045969 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9402ae00 con 0x7f2ea41450e0 2024-04-25T10:11:53.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.255+0000 7f2ea2000640 1 -- 172.21.15.119:0/327045969 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94032660 con 0x7f2ea41450e0 2024-04-25T10:11:53.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.256+0000 7f2ea8e00640 1 -- 172.21.15.119:0/327045969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea41450e0 msgr2=0x7f2ea406aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.256+0000 7f2ea8e00640 1 --2- 172.21.15.119:0/327045969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea41450e0 0x7f2ea406aba0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40cbe90 tx=0x7f2e9402a0b0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.256+0000 7f2ea8e00640 1 -- 172.21.15.119:0/327045969 shutdown_connections 2024-04-25T10:11:53.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.256+0000 7f2ea8e00640 1 --2- 172.21.15.119:0/327045969 >> v2:172.21.15.132:3300/0 conn(0x7f2ea40cc810 0x7f2ea4071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.256+0000 7f2ea8e00640 1 --2- 172.21.15.119:0/327045969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea406b0e0 0x7f2ea40cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.256+0000 7f2ea8e00640 1 --2- 172.21.15.119:0/327045969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea41450e0 0x7f2ea406aba0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.256+0000 7f2ea8e00640 1 -- 172.21.15.119:0/327045969 >> 172.21.15.119:0/327045969 conn(0x7f2ea4066ff0 msgr2=0x7f2ea40b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.256+0000 7f2ea8e00640 1 -- 172.21.15.119:0/327045969 shutdown_connections 2024-04-25T10:11:53.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.256+0000 7f2ea8e00640 1 -- 172.21.15.119:0/327045969 wait complete. 2024-04-25T10:11:53.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.258+0000 7f2ea8e00640 1 Processor -- start 2024-04-25T10:11:53.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.258+0000 7f2ea8e00640 1 -- start start 2024-04-25T10:11:53.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea8e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea406b0e0 0x7f2ea41e89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea40cc810 0x7f2ea41e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea8e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea41450e0 0x7f2ea4133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea8e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea40ccca0 con 0x7f2ea40cc810 2024-04-25T10:11:53.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea8e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea415fca0 con 0x7f2ea406b0e0 2024-04-25T10:11:53.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea8e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea41e6ce0 con 0x7f2ea41450e0 2024-04-25T10:11:53.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea40cc810 0x7f2ea41e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea40cc810 0x7f2ea41e7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34932/0 (socket says 172.21.15.119:34932) 2024-04-25T10:11:53.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea2a00640 1 -- 172.21.15.119:0/2577747768 learned_addr learned my addr 172.21.15.119:0/2577747768 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:53.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea3400640 1 --2- 172.21.15.119:0/2577747768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea406b0e0 0x7f2ea41e89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea3e00640 1 --2- 172.21.15.119:0/2577747768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea41450e0 0x7f2ea4133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea2a00640 1 -- 172.21.15.119:0/2577747768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea41450e0 msgr2=0x7f2ea4133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea2a00640 1 --2- 172.21.15.119:0/2577747768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea41450e0 0x7f2ea4133c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea2a00640 1 -- 172.21.15.119:0/2577747768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea406b0e0 msgr2=0x7f2ea41e89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea2a00640 1 --2- 172.21.15.119:0/2577747768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea406b0e0 0x7f2ea41e89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.259+0000 7f2ea2a00640 1 -- 172.21.15.119:0/2577747768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea41e6fc0 con 0x7f2ea40cc810 2024-04-25T10:11:53.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.260+0000 7f2ea2a00640 1 --2- 172.21.15.119:0/2577747768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea40cc810 0x7f2ea41e7d80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000c570 tx=0x7f2e9000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.260+0000 7f2e9b400640 1 -- 172.21.15.119:0/2577747768 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90015070 con 0x7f2ea40cc810 2024-04-25T10:11:53.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.260+0000 7f2e9b400640 1 -- 172.21.15.119:0/2577747768 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e90011800 con 0x7f2ea40cc810 2024-04-25T10:11:53.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.260+0000 7f2ea8e00640 1 -- 172.21.15.119:0/2577747768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea41e72a0 con 0x7f2ea40cc810 2024-04-25T10:11:53.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.260+0000 7f2e9b400640 1 -- 172.21.15.119:0/2577747768 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90011bf0 con 0x7f2ea40cc810 2024-04-25T10:11:53.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.260+0000 7f2ea8e00640 1 -- 172.21.15.119:0/2577747768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea41e7860 con 0x7f2ea40cc810 2024-04-25T10:11:53.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.262+0000 7f2e9b400640 1 -- 172.21.15.119:0/2577747768 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e90018070 con 0x7f2ea40cc810 2024-04-25T10:11:53.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.263+0000 7f2e9b400640 1 --2- 172.21.15.119:0/2577747768 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2e8406b4d0 0x7f2e8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.263+0000 7f2e9b400640 1 -- 172.21.15.119:0/2577747768 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2e9008a1c0 con 0x7f2ea40cc810 2024-04-25T10:11:53.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.263+0000 7f2ea3400640 1 --2- 172.21.15.119:0/2577747768 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2e8406b4d0 0x7f2e8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.264+0000 7f2ead3ae740 1 Processor -- start 2024-04-25T10:11:53.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.264+0000 7f2ead3ae740 1 -- start start 2024-04-25T10:11:53.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.264+0000 7f2ea3400640 1 --2- 172.21.15.119:0/2577747768 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2e8406b4d0 0x7f2e8406d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2e94034000 tx=0x7f2e9402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.265+0000 7f2ead3ae740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa560b30 0x557aaa588970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.265+0000 7f2ead3ae740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 0x557aaa560f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.265+0000 7f2ead3ae740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa5a97a0 0x557aaa55f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.265+0000 7f2ead3ae740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aaa4c4620 con 0x557aaa5a97a0 2024-04-25T10:11:53.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.265+0000 7f2ead3ae740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aaa46dca0 con 0x557aaa588eb0 2024-04-25T10:11:53.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.265+0000 7f2ead3ae740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557aaa4c06c0 con 0x557aaa560b30 2024-04-25T10:11:53.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.265+0000 7f2ea3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa5a97a0 0x557aaa55f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.265+0000 7f2ea3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa5a97a0 0x557aaa55f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34942/0 (socket says 172.21.15.119:34942) 2024-04-25T10:11:53.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.265+0000 7f2ea3e00640 1 -- 172.21.15.119:0/36317180 learned_addr learned my addr 172.21.15.119:0/36317180 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:53.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.266+0000 7f2ea2a00640 1 --2- 172.21.15.119:0/36317180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 0x557aaa560f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.266+0000 7f2ea3400640 1 --2- 172.21.15.119:0/36317180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa560b30 0x557aaa588970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.266+0000 7f2ea3e00640 1 -- 172.21.15.119:0/36317180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa560b30 msgr2=0x557aaa588970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.266+0000 7f2ea3e00640 1 --2- 172.21.15.119:0/36317180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa560b30 0x557aaa588970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.266+0000 7f2ea3e00640 1 -- 172.21.15.119:0/36317180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 msgr2=0x557aaa560f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.266+0000 7f2ea3e00640 1 --2- 172.21.15.119:0/36317180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 0x557aaa560f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.266+0000 7f2ea3e00640 1 -- 172.21.15.119:0/36317180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aaa474f40 con 0x557aaa5a97a0 2024-04-25T10:11:53.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.266+0000 7f2ea2a00640 1 --2- 172.21.15.119:0/36317180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 0x557aaa560f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:53.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.266+0000 7f2ea3e00640 1 --2- 172.21.15.119:0/36317180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa5a97a0 0x557aaa55f2f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c008f70 tx=0x7f2e9c00d810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa5ed178e4a330e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.266+0000 7f2e8fe00640 1 -- 172.21.15.119:0/36317180 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c003da0 con 0x557aaa5a97a0 2024-04-25T10:11:53.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.266+0000 7f2e8fe00640 1 -- 172.21.15.119:0/36317180 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c004030 con 0x557aaa5a97a0 2024-04-25T10:11:53.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.266+0000 7f2e8fe00640 1 -- 172.21.15.119:0/36317180 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c004420 con 0x557aaa5a97a0 2024-04-25T10:11:53.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.267+0000 7f2ead3ae740 1 -- 172.21.15.119:0/36317180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa5a97a0 msgr2=0x557aaa55f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.267+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/36317180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa5a97a0 0x557aaa55f2f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c008f70 tx=0x7f2e9c00d810 comp rx=0 tx=0).stop 2024-04-25T10:11:53.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.267+0000 7f2ead3ae740 1 -- 172.21.15.119:0/36317180 shutdown_connections 2024-04-25T10:11:53.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.267+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/36317180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa5a97a0 0x557aaa55f2f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.267+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/36317180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 0x557aaa560f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.267+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/36317180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa560b30 0x557aaa588970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.267+0000 7f2ead3ae740 1 -- 172.21.15.119:0/36317180 >> 172.21.15.119:0/36317180 conn(0x557aaa55dd50 msgr2=0x557aaa5a6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.267+0000 7f2ead3ae740 1 -- 172.21.15.119:0/36317180 shutdown_connections 2024-04-25T10:11:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.267+0000 7f2ead3ae740 1 -- 172.21.15.119:0/36317180 wait complete. 2024-04-25T10:11:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.268+0000 7f2ead3ae740 1 -- start start 2024-04-25T10:11:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.268+0000 7f2ead3ae740 1 Processor -- start 2024-04-25T10:11:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.269+0000 7f2ead3ae740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 0x557aaa58d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.269+0000 7f2ead3ae740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa5a97a0 0x557aaa587170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.269+0000 7f2ead3ae740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa58ddf0 0x557aaa5876b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.269+0000 7f2ead3ae740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aaa5884e0 con 0x557aaa58ddf0 2024-04-25T10:11:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.269+0000 7f2ead3ae740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aaa4631f0 con 0x557aaa588eb0 2024-04-25T10:11:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.269+0000 7f2ead3ae740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557aaa587ed0 con 0x557aaa5a97a0 2024-04-25T10:11:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.269+0000 7f2ead3ae740 10 client.0 _ll_register_callbacks cb 0x557aaa480ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.269+0000 7f2ea3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa58ddf0 0x557aaa5876b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.269+0000 7f2ea3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa58ddf0 0x557aaa5876b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34956/0 (socket says 172.21.15.119:34956) 2024-04-25T10:11:53.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.269+0000 7f2ea3e00640 1 -- 172.21.15.119:0/3639269924 learned_addr learned my addr 172.21.15.119:0/3639269924 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.270+0000 7f2ea3400640 1 --2- 172.21.15.119:0/3639269924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 0x557aaa58d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.270+0000 7f2ea2a00640 1 --2- 172.21.15.119:0/3639269924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa5a97a0 0x557aaa587170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.270+0000 7f2ea3e00640 1 -- 172.21.15.119:0/3639269924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa5a97a0 msgr2=0x557aaa587170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.270+0000 7f2ea3e00640 1 --2- 172.21.15.119:0/3639269924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa5a97a0 0x557aaa587170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.270+0000 7f2ea3e00640 1 -- 172.21.15.119:0/3639269924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 msgr2=0x557aaa58d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.270+0000 7f2ea3e00640 1 --2- 172.21.15.119:0/3639269924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 0x557aaa58d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.270+0000 7f2ea3e00640 1 -- 172.21.15.119:0/3639269924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557aaa482c50 con 0x557aaa58ddf0 2024-04-25T10:11:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.270+0000 7f2ea3400640 1 --2- 172.21.15.119:0/3639269924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 0x557aaa58d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.270+0000 7f2ea3e00640 1 --2- 172.21.15.119:0/3639269924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa58ddf0 0x557aaa5876b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00dd10 tx=0x7f2e9c010070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890915dc42d23db3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.270+0000 7f2e8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:11:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.270+0000 7f2e8cc00640 1 -- 172.21.15.119:0/3639269924 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c028060 con 0x557aaa58ddf0 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.270+0000 7f2e8cc00640 1 -- 172.21.15.119:0/3639269924 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2e9c0043f0 con 0x557aaa58ddf0 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.270+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3639269924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e9c004030 con 0x557aaa58ddf0 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.271+0000 7f2e8cc00640 1 -- 172.21.15.119:0/3639269924 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e9c02e050 con 0x557aaa58ddf0 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.271+0000 7f2ead3ae740 10 client.6741 fetch_fsmap learned FSMap version 50 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.271+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3639269924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x557aaa1d92d0 con 0x557aaa58ddf0 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.271+0000 7f2e8cc00640 1 -- 172.21.15.119:0/3639269924 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c004030 con 0x557aaa58ddf0 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.271+0000 7f2e8cc00640 1 -- 172.21.15.119:0/3639269924 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2e9c031020 con 0x557aaa58ddf0 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.271+0000 7f2ead3ae740 10 client.6741 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.271+0000 7f2ead3ae740 10 client.6741 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.271+0000 7f2ead3ae740 20 client.6741 populate_metadata read hostname 'smithi119' 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.271+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/3639269924 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x557aaa589840 0x557aaa589c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.271+0000 7f2ead3ae740 4 client.6741 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.271+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3639269924 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557aaa4d1b60 con 0x557aaa589840 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.271+0000 7f2ea3400640 1 --2- 172.21.15.119:0/3639269924 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x557aaa589840 0x557aaa589c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.272+0000 7f2ea3400640 1 --2- 172.21.15.119:0/3639269924 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x557aaa589840 0x557aaa589c20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f2d3e02c9d0298f server_cookie=f7157061e4b871e9 in_seq=0 out_seq=0 2024-04-25T10:11:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.272+0000 7f2e8cc00640 10 client.6741 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:11:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.273+0000 7f2e8cc00640 1 -- 172.21.15.119:0/3639269924 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557aaa4d1b60 con 0x557aaa589840 2024-04-25T10:11:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.273+0000 7f2e8cc00640 10 client.6741 handle_command_reply: tid=0 2024-04-25T10:11:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.273+0000 7f2ead3ae740 1 client.6741 shutdown 2024-04-25T10:11:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.273+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3639269924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa58ddf0 msgr2=0x557aaa5876b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.273+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/3639269924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa58ddf0 0x557aaa5876b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00dd10 tx=0x7f2e9c010070 comp rx=0 tx=0).stop 2024-04-25T10:11:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.273+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3639269924 shutdown_connections 2024-04-25T10:11:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.273+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/3639269924 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x557aaa589840 0x557aaa589c20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.273+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/3639269924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa58ddf0 0x557aaa5876b0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.273+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/3639269924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa5a97a0 0x557aaa587170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.273+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/3639269924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 0x557aaa58d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.273+0000 7f2e8cc00640 0 client.6741 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:11:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.273+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3639269924 >> 172.21.15.119:0/3639269924 conn(0x557aaa560b30 msgr2=0x557aaa5a6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.273+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3639269924 shutdown_connections 2024-04-25T10:11:53.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.273+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3639269924 wait complete. 2024-04-25T10:11:53.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.274+0000 7f2ead3ae740 20 client.6741 trim_cache size 0 max 16384 2024-04-25T10:11:53.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.290+0000 7f2ead3ae740 1 Processor -- start 2024-04-25T10:11:53.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.290+0000 7f2ead3ae740 1 -- start start 2024-04-25T10:11:53.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.291+0000 7f2ead3ae740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa588eb0 0x557aaa5a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.291+0000 7f2ead3ae740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa58ddf0 0x557aaa58d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.291+0000 7f2ead3ae740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa5a97a0 0x557aaa587e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.291+0000 7f2ead3ae740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea41ff1a0 con 0x557aaa58ddf0 2024-04-25T10:11:53.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.291+0000 7f2ead3ae740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea415dc90 con 0x557aaa5a97a0 2024-04-25T10:11:53.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.291+0000 7f2ead3ae740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea41e6830 con 0x557aaa588eb0 2024-04-25T10:11:53.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.292+0000 7f2ea3400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa588eb0 0x557aaa5a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.292+0000 7f2ea2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa58ddf0 0x557aaa58d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.292+0000 7f2ea2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa58ddf0 0x557aaa58d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34972/0 (socket says 172.21.15.119:34972) 2024-04-25T10:11:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.292+0000 7f2ea2a00640 1 -- 172.21.15.119:0/455047747 learned_addr learned my addr 172.21.15.119:0/455047747 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.292+0000 7f2ea3e00640 1 --2- 172.21.15.119:0/455047747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa5a97a0 0x557aaa587e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.292+0000 7f2ea2a00640 1 -- 172.21.15.119:0/455047747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa588eb0 msgr2=0x557aaa5a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.292+0000 7f2ea2a00640 1 --2- 172.21.15.119:0/455047747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa588eb0 0x557aaa5a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.292+0000 7f2ea2a00640 1 -- 172.21.15.119:0/455047747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa5a97a0 msgr2=0x557aaa587e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.292+0000 7f2ea2a00640 1 --2- 172.21.15.119:0/455047747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa5a97a0 0x557aaa587e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.292+0000 7f2ea2a00640 1 -- 172.21.15.119:0/455047747 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aaa2a36b0 con 0x557aaa58ddf0 2024-04-25T10:11:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.292+0000 7f2ea3400640 1 --2- 172.21.15.119:0/455047747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa588eb0 0x557aaa5a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.292+0000 7f2ea2a00640 1 --2- 172.21.15.119:0/455047747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa58ddf0 0x557aaa58d690 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2e90004770 tx=0x7f2e900583d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bbd35019720d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.293+0000 7f2e8f400640 1 -- 172.21.15.119:0/455047747 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90086070 con 0x557aaa58ddf0 2024-04-25T10:11:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.293+0000 7f2e8f400640 1 -- 172.21.15.119:0/455047747 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e90056a80 con 0x557aaa58ddf0 2024-04-25T10:11:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.293+0000 7f2e8f400640 1 -- 172.21.15.119:0/455047747 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90070070 con 0x557aaa58ddf0 2024-04-25T10:11:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.293+0000 7f2ead3ae740 1 -- 172.21.15.119:0/455047747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa58ddf0 msgr2=0x557aaa58d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.293+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/455047747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa58ddf0 0x557aaa58d690 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2e90004770 tx=0x7f2e900583d0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.293+0000 7f2ead3ae740 1 -- 172.21.15.119:0/455047747 shutdown_connections 2024-04-25T10:11:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.293+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/455047747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa5a97a0 0x557aaa587e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.293+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/455047747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa58ddf0 0x557aaa58d690 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.293+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/455047747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa588eb0 0x557aaa5a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.293+0000 7f2ead3ae740 1 -- 172.21.15.119:0/455047747 >> 172.21.15.119:0/455047747 conn(0x557aaa560b30 msgr2=0x557aaa5c9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.293+0000 7f2ead3ae740 1 -- 172.21.15.119:0/455047747 shutdown_connections 2024-04-25T10:11:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.293+0000 7f2ead3ae740 1 -- 172.21.15.119:0/455047747 wait complete. 2024-04-25T10:11:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.294+0000 7f2ead3ae740 1 -- start start 2024-04-25T10:11:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.294+0000 7f2ead3ae740 1 Processor -- start 2024-04-25T10:11:53.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.295+0000 7f2ead3ae740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 0x557aaa5c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.295+0000 7f2ead3ae740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa58ddf0 0x557aaa5bff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.295+0000 7f2ead3ae740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa5a97a0 0x557aaa5c0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.295+0000 7f2ead3ae740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aaa5843a0 con 0x557aaa5a97a0 2024-04-25T10:11:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.295+0000 7f2ead3ae740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aaa58b760 con 0x557aaa588eb0 2024-04-25T10:11:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.295+0000 7f2ead3ae740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557aaa58c580 con 0x557aaa58ddf0 2024-04-25T10:11:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.295+0000 7f2ead3ae740 10 client.0 _ll_register_callbacks cb 0x7f2e9c008dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.295+0000 7f2ea3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa5a97a0 0x557aaa5c0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.295+0000 7f2ea3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa5a97a0 0x557aaa5c0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34988/0 (socket says 172.21.15.119:34988) 2024-04-25T10:11:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.295+0000 7f2ea3e00640 1 -- 172.21.15.119:0/3716120914 learned_addr learned my addr 172.21.15.119:0/3716120914 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.296+0000 7f2ea3e00640 1 -- 172.21.15.119:0/3716120914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa58ddf0 msgr2=0x557aaa5bff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.296+0000 7f2ea3e00640 1 --2- 172.21.15.119:0/3716120914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa58ddf0 0x557aaa5bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.296+0000 7f2ea3e00640 1 -- 172.21.15.119:0/3716120914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 msgr2=0x557aaa5c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.296+0000 7f2ea3e00640 1 --2- 172.21.15.119:0/3716120914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 0x557aaa5c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.296+0000 7f2ea3e00640 1 -- 172.21.15.119:0/3716120914 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557aaa5b7970 con 0x557aaa5a97a0 2024-04-25T10:11:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.296+0000 7f2ea3e00640 1 --2- 172.21.15.119:0/3716120914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa5a97a0 0x557aaa5c0450 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c026040 tx=0x7f2e9c008860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def7bd04b959a7a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.296+0000 7f2e8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.296+0000 7f2e8cc00640 1 -- 172.21.15.119:0/3716120914 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c008990 con 0x557aaa5a97a0 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.296+0000 7f2e8cc00640 1 -- 172.21.15.119:0/3716120914 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2e9c008270 con 0x557aaa5a97a0 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.296+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3716120914 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e90056a80 con 0x557aaa5a97a0 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.297+0000 7f2e8cc00640 1 -- 172.21.15.119:0/3716120914 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e9c03e050 con 0x557aaa5a97a0 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.297+0000 7f2ead3ae740 10 client.6747 fetch_fsmap learned FSMap version 50 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.297+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3716120914 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x557aaa589dc0 con 0x557aaa5a97a0 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.297+0000 7f2e8cc00640 1 -- 172.21.15.119:0/3716120914 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e90056a80 con 0x557aaa5a97a0 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.297+0000 7f2e8cc00640 1 -- 172.21.15.119:0/3716120914 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2e9c041020 con 0x557aaa5a97a0 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.297+0000 7f2ead3ae740 10 client.6747 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.297+0000 7f2ead3ae740 10 client.6747 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.297+0000 7f2ead3ae740 20 client.6747 populate_metadata read hostname 'smithi119' 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.297+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/3716120914 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x557aaa5c6e50 0x557aaa55a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.297+0000 7f2ead3ae740 4 client.6747 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:11:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.297+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3716120914 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557aaa4cbeb0 con 0x557aaa5c6e50 2024-04-25T10:11:53.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.297+0000 7f2ea3400640 1 --2- 172.21.15.119:0/3716120914 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x557aaa5c6e50 0x557aaa55a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2ea3400640 1 --2- 172.21.15.119:0/3716120914 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x557aaa5c6e50 0x557aaa55a8b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b85e249f77fa546 server_cookie=3d60f4575b102a0d in_seq=0 out_seq=0 2024-04-25T10:11:53.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2e8cc00640 10 client.6747 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:11:53.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2e8cc00640 1 -- 172.21.15.119:0/3716120914 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x557aaa4cbeb0 con 0x557aaa5c6e50 2024-04-25T10:11:53.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2e8cc00640 10 client.6747 handle_command_reply: tid=0 2024-04-25T10:11:53.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2ead3ae740 1 client.6747 shutdown 2024-04-25T10:11:53.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3716120914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa5a97a0 msgr2=0x557aaa5c0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/3716120914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa5a97a0 0x557aaa5c0450 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c026040 tx=0x7f2e9c008860 comp rx=0 tx=0).stop 2024-04-25T10:11:53.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3716120914 shutdown_connections 2024-04-25T10:11:53.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/3716120914 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x557aaa5c6e50 0x557aaa55a8b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/3716120914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557aaa5a97a0 0x557aaa5c0450 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/3716120914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557aaa58ddf0 0x557aaa5bff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2e8cc00640 0 client.6747 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:11:53.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2ead3ae740 1 --2- 172.21.15.119:0/3716120914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557aaa588eb0 0x557aaa5c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3716120914 >> 172.21.15.119:0/3716120914 conn(0x557aaa560b30 msgr2=0x557aaa5a8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.298+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3716120914 shutdown_connections 2024-04-25T10:11:53.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.299+0000 7f2ead3ae740 1 -- 172.21.15.119:0/3716120914 wait complete. 2024-04-25T10:11:53.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.299+0000 7f2ead3ae740 20 client.6747 trim_cache size 0 max 16384 2024-04-25T10:11:53.307 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:11:53.308 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "e0b0f428-b441-4237-a4d1-ed53321f87a5", 2024-04-25T10:11:53.308 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-04-25T10:11:53.308 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 6699, 2024-04-25T10:11:53.308 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-04-25T10:11:53.308 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-04-25T10:11:53.308 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-04-25T10:11:53.308 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 4.9095832440000002, 2024-04-25T10:11:53.308 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 50, 2024-04-25T10:11:53.308 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 70, 2024-04-25T10:11:53.308 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 69, 2024-04-25T10:11:53.308 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 5.8712960660000002 2024-04-25T10:11:53.308 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:11:53.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.299+0000 7f2ea8e00640 1 -- 172.21.15.119:0/2577747768 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2e8406b4d0 msgr2=0x7f2e8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.299+0000 7f2ea8e00640 1 --2- 172.21.15.119:0/2577747768 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2e8406b4d0 0x7f2e8406d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2e94034000 tx=0x7f2e9402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:53.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.299+0000 7f2ea8e00640 1 -- 172.21.15.119:0/2577747768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea40cc810 msgr2=0x7f2ea41e7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.299+0000 7f2ea8e00640 1 --2- 172.21.15.119:0/2577747768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea40cc810 0x7f2ea41e7d80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000c570 tx=0x7f2e9000ca30 comp rx=0 tx=0).stop 2024-04-25T10:11:53.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.299+0000 7f2ea8e00640 1 -- 172.21.15.119:0/2577747768 shutdown_connections 2024-04-25T10:11:53.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.299+0000 7f2ea8e00640 1 --2- 172.21.15.119:0/2577747768 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2e8406b4d0 0x7f2e8406d990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.299+0000 7f2ea8e00640 1 --2- 172.21.15.119:0/2577747768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea41450e0 0x7f2ea4133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.299+0000 7f2ea8e00640 1 --2- 172.21.15.119:0/2577747768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea40cc810 0x7f2ea41e7d80 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.299+0000 7f2ea8e00640 1 --2- 172.21.15.119:0/2577747768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea406b0e0 0x7f2ea41e89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.299+0000 7f2ea8e00640 1 -- 172.21.15.119:0/2577747768 >> 172.21.15.119:0/2577747768 conn(0x7f2ea4066ff0 msgr2=0x7f2ea40cf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.299+0000 7f2ea8e00640 1 -- 172.21.15.119:0/2577747768 shutdown_connections 2024-04-25T10:11:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.299+0000 7f2ea8e00640 1 -- 172.21.15.119:0/2577747768 wait complete. 2024-04-25T10:11:53.322 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:11:53.396 INFO:tasks.ceph.mds.c.smithi119.stderr:2024-04-25T10:11:53.391+0000 7f4b2fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:53.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.499+0000 7f3aeba00640 1 Processor -- start 2024-04-25T10:11:53.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.499+0000 7f3aeba00640 1 -- start start 2024-04-25T10:11:53.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.501+0000 7f3aeba00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae415bf00 0x7f3ae40cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.501+0000 7f3aeba00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3ae406e8f0 0x7f3ae41654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.501+0000 7f3aeba00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae40ce6c0 con 0x7f3ae40cc310 2024-04-25T10:11:53.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.501+0000 7f3aeba00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae41455e0 con 0x7f3ae406e8f0 2024-04-25T10:11:53.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.501+0000 7f3aeba00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae406d450 con 0x7f3ae415bf00 2024-04-25T10:11:53.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.501+0000 7f3ae9a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3ae406e8f0 0x7f3ae41654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.501+0000 7f3adbe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3ae40cc310 0x7f3ae415c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37970/0 (socket says 172.21.15.119:37970) 2024-04-25T10:11:53.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.501+0000 7f3adbe00640 1 -- 172.21.15.119:0/275348070 learned_addr learned my addr 172.21.15.119:0/275348070 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:53.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.502+0000 7f3ae9000640 1 --2- 172.21.15.119:0/275348070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae415bf00 0x7f3ae40cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.502+0000 7f3ae9a00640 1 -- 172.21.15.119:0/275348070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae415bf00 msgr2=0x7f3ae40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.502+0000 7f3ae9a00640 1 --2- 172.21.15.119:0/275348070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae415bf00 0x7f3ae40cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.502+0000 7f3ae9a00640 1 -- 172.21.15.119:0/275348070 >> v1:172.21.15.119:6789/0 conn(0x7f3ae40cc310 legacy=0x7f3ae415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.502+0000 7f3ae9a00640 1 -- 172.21.15.119:0/275348070 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae40cc7b0 con 0x7f3ae406e8f0 2024-04-25T10:11:53.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.503+0000 7f3ae9a00640 1 --2- 172.21.15.119:0/275348070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ae406e8f0 0x7f3ae41654b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009d90 tx=0x7f3adc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f7ca309a1b662 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.503+0000 7f3ae3e00640 1 -- 172.21.15.119:0/275348070 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc007740 con 0x7f3ae406e8f0 2024-04-25T10:11:53.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.504+0000 7f3ae3e00640 1 -- 172.21.15.119:0/275348070 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3adc0078e0 con 0x7f3ae406e8f0 2024-04-25T10:11:53.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.504+0000 7f3ae3e00640 1 -- 172.21.15.119:0/275348070 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc007cb0 con 0x7f3ae406e8f0 2024-04-25T10:11:53.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.504+0000 7f3aeba00640 1 -- 172.21.15.119:0/275348070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ae406e8f0 msgr2=0x7f3ae41654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.504+0000 7f3aeba00640 1 --2- 172.21.15.119:0/275348070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ae406e8f0 0x7f3ae41654b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009d90 tx=0x7f3adc02a660 comp rx=0 tx=0).stop 2024-04-25T10:11:53.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.504+0000 7f3aeba00640 1 -- 172.21.15.119:0/275348070 shutdown_connections 2024-04-25T10:11:53.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.504+0000 7f3aeba00640 1 --2- 172.21.15.119:0/275348070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ae406e8f0 0x7f3ae41654b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.504+0000 7f3aeba00640 1 --2- 172.21.15.119:0/275348070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae415bf00 0x7f3ae40cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.504+0000 7f3aeba00640 1 -- 172.21.15.119:0/275348070 >> 172.21.15.119:0/275348070 conn(0x7f3ae406aba0 msgr2=0x7f3ae406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.504+0000 7f3aeba00640 1 -- 172.21.15.119:0/275348070 shutdown_connections 2024-04-25T10:11:53.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.504+0000 7f3aeba00640 1 -- 172.21.15.119:0/275348070 wait complete. 2024-04-25T10:11:53.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.506+0000 7f3aeba00640 1 Processor -- start 2024-04-25T10:11:53.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.506+0000 7f3aeba00640 1 -- start start 2024-04-25T10:11:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.507+0000 7f3aeba00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ae406e8f0 0x7f3ae415b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.507+0000 7f3aeba00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40cc310 0x7f3ae40c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.507+0000 7f3aeba00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ae415bf00 0x7f3ae415d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.507+0000 7f3aeba00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae40cd050 con 0x7f3ae415bf00 2024-04-25T10:11:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.507+0000 7f3aeba00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae40ccd70 con 0x7f3ae406e8f0 2024-04-25T10:11:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.507+0000 7f3aeba00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae406ee60 con 0x7f3ae40cc310 2024-04-25T10:11:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.507+0000 7f3ae9a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ae415bf00 0x7f3ae415d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.507+0000 7f3ae9a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ae415bf00 0x7f3ae415d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35008/0 (socket says 172.21.15.119:35008) 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.507+0000 7f3ae9a00640 1 -- 172.21.15.119:0/698493496 learned_addr learned my addr 172.21.15.119:0/698493496 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.508+0000 7f3ae9000640 1 --2- 172.21.15.119:0/698493496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ae406e8f0 0x7f3ae415b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.508+0000 7f3adbe00640 1 --2- 172.21.15.119:0/698493496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40cc310 0x7f3ae40c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.508+0000 7f3ae9a00640 1 -- 172.21.15.119:0/698493496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40cc310 msgr2=0x7f3ae40c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.508+0000 7f3ae9a00640 1 --2- 172.21.15.119:0/698493496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40cc310 0x7f3ae40c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.508+0000 7f3ae9a00640 1 -- 172.21.15.119:0/698493496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ae406e8f0 msgr2=0x7f3ae415b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.508+0000 7f3ae9a00640 1 --2- 172.21.15.119:0/698493496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ae406e8f0 0x7f3ae415b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.508+0000 7f3ae9a00640 1 -- 172.21.15.119:0/698493496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae413c0d0 con 0x7f3ae415bf00 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.508+0000 7f3ae9a00640 1 --2- 172.21.15.119:0/698493496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ae415bf00 0x7f3ae415d330 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009eb0 tx=0x7f3adc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.508+0000 7f3ae1600640 1 -- 172.21.15.119:0/698493496 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc040070 con 0x7f3ae415bf00 2024-04-25T10:11:53.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.509+0000 7f3ae1600640 1 -- 172.21.15.119:0/698493496 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3adc031bc0 con 0x7f3ae415bf00 2024-04-25T10:11:53.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.509+0000 7f3ae1600640 1 -- 172.21.15.119:0/698493496 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc045460 con 0x7f3ae415bf00 2024-04-25T10:11:53.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.509+0000 7f3aeba00640 1 -- 172.21.15.119:0/698493496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae413c3b0 con 0x7f3ae415bf00 2024-04-25T10:11:53.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.509+0000 7f3aeba00640 1 -- 172.21.15.119:0/698493496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae413c970 con 0x7f3ae415bf00 2024-04-25T10:11:53.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.510+0000 7f3aeba00640 1 -- 172.21.15.119:0/698493496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae406bb30 con 0x7f3ae415bf00 2024-04-25T10:11:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.511+0000 7f3ae1600640 1 -- 172.21.15.119:0/698493496 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3adc039040 con 0x7f3ae415bf00 2024-04-25T10:11:53.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.512+0000 7f3ae1600640 1 --2- 172.21.15.119:0/698493496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3ac006b4d0 0x7f3ac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.512+0000 7f3ae1600640 1 -- 172.21.15.119:0/698493496 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f3adc0abc20 con 0x7f3ae415bf00 2024-04-25T10:11:53.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.513+0000 7f3ae9000640 1 --2- 172.21.15.119:0/698493496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3ac006b4d0 0x7f3ac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.513+0000 7f3ae9000640 1 --2- 172.21.15.119:0/698493496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3ac006b4d0 0x7f3ac006d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4006fd0 tx=0x7f3ad4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.517+0000 7f3ae1600640 1 -- 172.21.15.119:0/698493496 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3adc07aa90 con 0x7f3ae415bf00 2024-04-25T10:11:53.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.749+0000 7f3aeba00640 1 -- 172.21.15.119:0/698493496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3ae40749b0 con 0x7f3ae415bf00 2024-04-25T10:11:53.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.750+0000 7f3ae1600640 1 -- 172.21.15.119:0/698493496 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3adc07a390 con 0x7f3ae415bf00 2024-04-25T10:11:53.755 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:11:53.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.753+0000 7f3aeba00640 1 -- 172.21.15.119:0/698493496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3ac006b4d0 msgr2=0x7f3ac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.753+0000 7f3aeba00640 1 --2- 172.21.15.119:0/698493496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3ac006b4d0 0x7f3ac006d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4006fd0 tx=0x7f3ad4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:53.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.753+0000 7f3aeba00640 1 -- 172.21.15.119:0/698493496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ae415bf00 msgr2=0x7f3ae415d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.753+0000 7f3aeba00640 1 --2- 172.21.15.119:0/698493496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ae415bf00 0x7f3ae415d330 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009eb0 tx=0x7f3adc004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:53.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.753+0000 7f3aeba00640 1 -- 172.21.15.119:0/698493496 shutdown_connections 2024-04-25T10:11:53.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.753+0000 7f3aeba00640 1 --2- 172.21.15.119:0/698493496 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3ac006b4d0 0x7f3ac006d990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.753+0000 7f3aeba00640 1 --2- 172.21.15.119:0/698493496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ae415bf00 0x7f3ae415d330 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.753+0000 7f3aeba00640 1 --2- 172.21.15.119:0/698493496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ae40cc310 0x7f3ae40c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.753+0000 7f3aeba00640 1 --2- 172.21.15.119:0/698493496 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ae406e8f0 0x7f3ae415b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.753+0000 7f3aeba00640 1 -- 172.21.15.119:0/698493496 >> 172.21.15.119:0/698493496 conn(0x7f3ae406aba0 msgr2=0x7f3ae4144330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.753+0000 7f3aeba00640 1 -- 172.21.15.119:0/698493496 shutdown_connections 2024-04-25T10:11:53.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:53.753+0000 7f3aeba00640 1 -- 172.21.15.119:0/698493496 wait complete. 2024-04-25T10:11:53.777 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:11:53.777 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:11:53.777 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:11:53.777 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:11:53.778 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:53.778 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:11:53.778 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:11:53.778 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:11:53.778 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:53.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:53.778 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:11:53.832 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:11:53.832 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:11:53.832 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:11:53.832 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:53.832 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:11:53.832 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:53.832 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:53.832 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:53.833 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:11:53.833 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:53.833 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:53.833 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:11:53.833 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:53.833 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:53.833 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:11:53.833 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:53.833 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:11:53.833 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42376sec preferred_lft 42376sec 2024-04-25T10:11:53.833 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:11:53.833 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:53.833 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:11:53.834 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:53.834 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:11:53.834 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:53.834 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:11:53.834 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:53.834 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:53.834 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:53.834 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:11:53.834 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:53.834 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:53.834 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:53.834 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:11:53.835 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:53.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:53.836 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:11:53.913 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:11:53.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:53.918 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:11:53.971 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:11:53.971 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:11:53.971 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:11:53.971 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:11:53.972 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:11:53.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:53.973 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:11:53.973 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:11:53.973 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:11:53.973 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:11:53.973 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:11:53.973 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:11:54.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:54.120 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:11:54.169 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:11:54.169 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:11:54.170 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:54.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:54.171 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:11:54.252 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:54.252 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:54.306 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:54.306 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:54.306 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:54.362 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:11:54.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:54.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:54.490 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:54.490 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:54.490 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:11:54.511 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:54.511 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:54.513 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:11:54.513 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:11:54.513 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:11:54.536 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:54.536 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:54.536 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:11:54.536 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:11:54.536 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:11:54.537 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:11:54.537 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:11:54.537 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:11:54.540 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:11:54.540 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:11:54.540 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:11:54.540 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:11:54.540 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:11:54.544 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:11:54.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:54.544 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:54.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:54.621 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:11:54.675 INFO:teuthology.orchestra.run.smithi119.stdout:client6756 2024-04-25T10:11:54.676 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:11:54.676 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:54.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:54.726 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-25T10:11:54.785 INFO:teuthology.orchestra.run.smithi119.stdout:file_undamaged 2024-04-25T10:11:54.787 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:11:54.787 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:11:54.788 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:11:54.788 DEBUG:teuthology.orchestra.run.smithi119:> import json 2024-04-25T10:11:54.788 DEBUG:teuthology.orchestra.run.smithi119:> import sys 2024-04-25T10:11:54.788 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:11:54.788 DEBUG:teuthology.orchestra.run.smithi119:> try: 2024-04-25T10:11:54.788 DEBUG:teuthology.orchestra.run.smithi119:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-25T10:11:54.788 DEBUG:teuthology.orchestra.run.smithi119:> except OSError as e: 2024-04-25T10:11:54.788 DEBUG:teuthology.orchestra.run.smithi119:> sys.exit(e.errno) 2024-04-25T10:11:54.788 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:11:54.788 DEBUG:teuthology.orchestra.run.smithi119:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:11:54.788 DEBUG:teuthology.orchestra.run.smithi119:> print(json.dumps( 2024-04-25T10:11:54.788 DEBUG:teuthology.orchestra.run.smithi119:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:11:54.788 DEBUG:teuthology.orchestra.run.smithi119:> indent=2)) 2024-04-25T10:11:54.788 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:11:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:54 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T10:11:54.942 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T10:11:54.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:54.942 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T10:11:55.042 INFO:teuthology.orchestra.run.smithi119.stdout:2 2024-04-25T10:11:55.043 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:11:55.043 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:55.093 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:11:55.094 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:55.144 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:55.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.338+0000 7f24b4a00640 1 Processor -- start 2024-04-25T10:11:55.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.339+0000 7f24b4a00640 1 -- start start 2024-04-25T10:11:55.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.340+0000 7f24b4a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f24b00b6a20 0x7f24b00b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.340+0000 7f24b4a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f24b00ce5c0 0x7f24b00ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.340+0000 7f24b4a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b00d1b10 con 0x7f24b015dd10 2024-04-25T10:11:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.340+0000 7f24b4a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b00cef30 con 0x7f24b00ce5c0 2024-04-25T10:11:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.340+0000 7f24b4a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b00d1df0 con 0x7f24b00b6a20 2024-04-25T10:11:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.340+0000 7f24af200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f24b00ce5c0 0x7f24b00ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.341+0000 7f24af200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f24b00ce5c0 0x7f24b00ce9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35020/0 (socket says 172.21.15.119:35020) 2024-04-25T10:11:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.341+0000 7f24af200640 1 -- 172.21.15.119:0/3101300906 learned_addr learned my addr 172.21.15.119:0/3101300906 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.341+0000 7f24af200640 1 -- 172.21.15.119:0/3101300906 >> v2:172.21.15.132:3300/0 conn(0x7f24b00b6a20 msgr2=0x7f24b00b6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.341+0000 7f24af200640 1 --2- 172.21.15.119:0/3101300906 >> v2:172.21.15.132:3300/0 conn(0x7f24b00b6a20 0x7f24b00b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.341+0000 7f24af200640 1 -- 172.21.15.119:0/3101300906 >> v1:172.21.15.132:6789/0 conn(0x7f24b015dd10 legacy=0x7f24b00b9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.341+0000 7f24af200640 1 -- 172.21.15.119:0/3101300906 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b00d20d0 con 0x7f24b00ce5c0 2024-04-25T10:11:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.342+0000 7f24af200640 1 --2- 172.21.15.119:0/3101300906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b00ce5c0 0x7f24b00ce9f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f24b00b5390 tx=0x7f24a802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372bdd77e5032894 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.342+0000 7f24ade00640 1 -- 172.21.15.119:0/3101300906 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8036070 con 0x7f24b00ce5c0 2024-04-25T10:11:55.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.342+0000 7f24ade00640 1 -- 172.21.15.119:0/3101300906 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a8031600 con 0x7f24b00ce5c0 2024-04-25T10:11:55.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.342+0000 7f24ade00640 1 -- 172.21.15.119:0/3101300906 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a80319d0 con 0x7f24b00ce5c0 2024-04-25T10:11:55.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.343+0000 7f24b4a00640 1 -- 172.21.15.119:0/3101300906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b00ce5c0 msgr2=0x7f24b00ce9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.343+0000 7f24b4a00640 1 --2- 172.21.15.119:0/3101300906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b00ce5c0 0x7f24b00ce9f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f24b00b5390 tx=0x7f24a802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.343+0000 7f24b4a00640 1 -- 172.21.15.119:0/3101300906 shutdown_connections 2024-04-25T10:11:55.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.343+0000 7f24b4a00640 1 --2- 172.21.15.119:0/3101300906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b00ce5c0 0x7f24b00ce9f0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.343+0000 7f24b4a00640 1 --2- 172.21.15.119:0/3101300906 >> v2:172.21.15.132:3300/0 conn(0x7f24b00b6a20 0x7f24b00b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.343+0000 7f24b4a00640 1 -- 172.21.15.119:0/3101300906 >> 172.21.15.119:0/3101300906 conn(0x7f24b00726f0 msgr2=0x7f24b0072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.343+0000 7f24b4a00640 1 -- 172.21.15.119:0/3101300906 shutdown_connections 2024-04-25T10:11:55.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.343+0000 7f24b4a00640 1 -- 172.21.15.119:0/3101300906 wait complete. 2024-04-25T10:11:55.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.345+0000 7f24b4a00640 1 Processor -- start 2024-04-25T10:11:55.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.345+0000 7f24b4a00640 1 -- start start 2024-04-25T10:11:55.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.346+0000 7f24b4a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24b00b6a20 0x7f24b0154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.346+0000 7f24b4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b00ce5c0 0x7f24b00c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.346+0000 7f24b4a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24b015dd10 0x7f24b0154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.346+0000 7f24b4a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b015e0f0 con 0x7f24b00ce5c0 2024-04-25T10:11:55.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.346+0000 7f24b4a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b00d2690 con 0x7f24b015dd10 2024-04-25T10:11:55.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.346+0000 7f24b4a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b00b2c90 con 0x7f24b00b6a20 2024-04-25T10:11:55.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.346+0000 7f24a5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b00ce5c0 0x7f24b00c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.346+0000 7f24a5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b00ce5c0 0x7f24b00c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35022/0 (socket says 172.21.15.119:35022) 2024-04-25T10:11:55.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.346+0000 7f24a5e00640 1 -- 172.21.15.119:0/4273964718 learned_addr learned my addr 172.21.15.119:0/4273964718 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:55.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.347+0000 7f24ae800640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24b00b6a20 0x7f24b0154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.347+0000 7f24af200640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24b015dd10 0x7f24b0154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.347+0000 7f24a5e00640 1 -- 172.21.15.119:0/4273964718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24b00b6a20 msgr2=0x7f24b0154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.347+0000 7f24a5e00640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24b00b6a20 0x7f24b0154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.347+0000 7f24a5e00640 1 -- 172.21.15.119:0/4273964718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24b015dd10 msgr2=0x7f24b0154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.347+0000 7f24a5e00640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24b015dd10 0x7f24b0154650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.347+0000 7f24a5e00640 1 -- 172.21.15.119:0/4273964718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b0155410 con 0x7f24b00ce5c0 2024-04-25T10:11:55.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.347+0000 7f24ae800640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24b00b6a20 0x7f24b0154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:55.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.347+0000 7f24a5e00640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b00ce5c0 0x7f24b00c9c10 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f249800b4a0 tx=0x7f249800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.347+0000 7f24a7400640 1 -- 172.21.15.119:0/4273964718 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498014070 con 0x7f24b00ce5c0 2024-04-25T10:11:55.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.347+0000 7f24a7400640 1 -- 172.21.15.119:0/4273964718 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2498010610 con 0x7f24b00ce5c0 2024-04-25T10:11:55.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.347+0000 7f24a7400640 1 -- 172.21.15.119:0/4273964718 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498010a00 con 0x7f24b00ce5c0 2024-04-25T10:11:55.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.347+0000 7f24b4a00640 1 -- 172.21.15.119:0/4273964718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b00c9130 con 0x7f24b00ce5c0 2024-04-25T10:11:55.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.348+0000 7f24b4a00640 1 -- 172.21.15.119:0/4273964718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b00ca150 con 0x7f24b00ce5c0 2024-04-25T10:11:55.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.349+0000 7f24b4a00640 1 -- 172.21.15.119:0/4273964718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b00c9900 con 0x7f24b00ce5c0 2024-04-25T10:11:55.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.350+0000 7f24a7400640 1 -- 172.21.15.119:0/4273964718 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2498017070 con 0x7f24b00ce5c0 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.351+0000 7f24a7400640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f248c06b4d0 0x7f248c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.351+0000 7f24a7400640 1 -- 172.21.15.119:0/4273964718 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2498089ce0 con 0x7f24b00ce5c0 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.351+0000 7f24ae800640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f248c06b4d0 0x7f248c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.352+0000 7f24ae800640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f248c06b4d0 0x7f248c06d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f24a000a810 tx=0x7f24a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.355+0000 7f24a7400640 1 -- 172.21.15.119:0/4273964718 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2498058ac0 con 0x7f24b00ce5c0 2024-04-25T10:11:55.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.553+0000 7f24b4a00640 1 -- 172.21.15.119:0/4273964718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24b00749b0 con 0x7f24b00ce5c0 2024-04-25T10:11:55.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.555+0000 7f24a7400640 1 -- 172.21.15.119:0/4273964718 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f24980583c0 con 0x7f24b00ce5c0 2024-04-25T10:11:55.560 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:11:55.560 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":70,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:11:51.402126+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:32.500199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:33.708192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.22","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/2673349178":"2024-04-26T10:11:47.336991+0000","172.21.15.119:6836/2673349178":"2024-04-26T10:11:47.336991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:55.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.558+0000 7f24b4a00640 1 -- 172.21.15.119:0/4273964718 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f248c06b4d0 msgr2=0x7f248c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.558+0000 7f24b4a00640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f248c06b4d0 0x7f248c06d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f24a000a810 tx=0x7f24a0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:55.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.558+0000 7f24b4a00640 1 -- 172.21.15.119:0/4273964718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b00ce5c0 msgr2=0x7f24b00c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.558+0000 7f24b4a00640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b00ce5c0 0x7f24b00c9c10 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f249800b4a0 tx=0x7f249800b960 comp rx=0 tx=0).stop 2024-04-25T10:11:55.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.558+0000 7f24b4a00640 1 -- 172.21.15.119:0/4273964718 shutdown_connections 2024-04-25T10:11:55.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.558+0000 7f24b4a00640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f248c06b4d0 0x7f248c06d990 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.558+0000 7f24b4a00640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24b015dd10 0x7f24b0154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.558+0000 7f24b4a00640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24b00ce5c0 0x7f24b00c9c10 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.558+0000 7f24b4a00640 1 --2- 172.21.15.119:0/4273964718 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24b00b6a20 0x7f24b0154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.558+0000 7f24b4a00640 1 -- 172.21.15.119:0/4273964718 >> 172.21.15.119:0/4273964718 conn(0x7f24b00726f0 msgr2=0x7f24b00b7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.558+0000 7f24b4a00640 1 -- 172.21.15.119:0/4273964718 shutdown_connections 2024-04-25T10:11:55.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.558+0000 7f24b4a00640 1 -- 172.21.15.119:0/4273964718 wait complete. 2024-04-25T10:11:55.581 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:32.500199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:33.708192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:55.581 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:55.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.782+0000 7fc5c3c00640 1 Processor -- start 2024-04-25T10:11:55.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.782+0000 7fc5c3c00640 1 -- start start 2024-04-25T10:11:55.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.784+0000 7fc5c3c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc5bc0b42c0 0x7fc5bc15ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.784+0000 7fc5c3c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc5bc0b46a0 0x7fc5bc1549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.784+0000 7fc5c3c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5bc15b370 con 0x7fc5bc15aef0 2024-04-25T10:11:55.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.784+0000 7fc5c3c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5bc0d05d0 con 0x7fc5bc0b46a0 2024-04-25T10:11:55.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.784+0000 7fc5c3c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5bc15b650 con 0x7fc5bc0b42c0 2024-04-25T10:11:55.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.785+0000 7fc5c1c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc5bc0b46a0 0x7fc5bc1549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.785+0000 7fc5c1c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc5bc0b46a0 0x7fc5bc1549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35038/0 (socket says 172.21.15.119:35038) 2024-04-25T10:11:55.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.785+0000 7fc5c1c00640 1 -- 172.21.15.119:0/2951784576 learned_addr learned my addr 172.21.15.119:0/2951784576 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:55.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.785+0000 7fc5bbe00640 1 --1- 172.21.15.119:0/2951784576 >> v1:172.21.15.119:6789/0 conn(0x7fc5bc15aef0 0x7fc5bc0cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37984/0 (socket says 172.21.15.119:37984) 2024-04-25T10:11:55.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.785+0000 7fc5c1200640 1 --2- 172.21.15.119:0/2951784576 >> v2:172.21.15.132:3300/0 conn(0x7fc5bc0b42c0 0x7fc5bc15ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.785+0000 7fc5c1c00640 1 -- 172.21.15.119:0/2951784576 >> v2:172.21.15.132:3300/0 conn(0x7fc5bc0b42c0 msgr2=0x7fc5bc15ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.785+0000 7fc5c1c00640 1 --2- 172.21.15.119:0/2951784576 >> v2:172.21.15.132:3300/0 conn(0x7fc5bc0b42c0 0x7fc5bc15ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.785+0000 7fc5c1c00640 1 -- 172.21.15.119:0/2951784576 >> v1:172.21.15.119:6789/0 conn(0x7fc5bc15aef0 legacy=0x7fc5bc0cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.785+0000 7fc5c1c00640 1 -- 172.21.15.119:0/2951784576 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc15b930 con 0x7fc5bc0b46a0 2024-04-25T10:11:55.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.785+0000 7fc5bb400640 1 -- 172.21.15.119:0/2951784576 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1603434210 0 0) 0x7fc5bc15b370 con 0x7fc5bc15aef0 2024-04-25T10:11:55.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.786+0000 7fc5c1c00640 1 --2- 172.21.15.119:0/2951784576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5bc0b46a0 0x7fc5bc1549d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0d07c0 tx=0x7fc5b402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6199ce71eb1b6daa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.786+0000 7fc5bb400640 1 -- 172.21.15.119:0/2951784576 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4037070 con 0x7fc5bc0b46a0 2024-04-25T10:11:55.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.787+0000 7fc5bb400640 1 -- 172.21.15.119:0/2951784576 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b4032440 con 0x7fc5bc0b46a0 2024-04-25T10:11:55.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.787+0000 7fc5bb400640 1 -- 172.21.15.119:0/2951784576 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4032810 con 0x7fc5bc0b46a0 2024-04-25T10:11:55.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.787+0000 7fc5c3c00640 1 -- 172.21.15.119:0/2951784576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5bc0b46a0 msgr2=0x7fc5bc1549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.787+0000 7fc5c3c00640 1 --2- 172.21.15.119:0/2951784576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5bc0b46a0 0x7fc5bc1549d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0d07c0 tx=0x7fc5b402a160 comp rx=0 tx=0).stop 2024-04-25T10:11:55.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.787+0000 7fc5c3c00640 1 -- 172.21.15.119:0/2951784576 shutdown_connections 2024-04-25T10:11:55.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.787+0000 7fc5c3c00640 1 --2- 172.21.15.119:0/2951784576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5bc0b46a0 0x7fc5bc1549d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.787+0000 7fc5c3c00640 1 --2- 172.21.15.119:0/2951784576 >> v2:172.21.15.132:3300/0 conn(0x7fc5bc0b42c0 0x7fc5bc15ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.787+0000 7fc5c3c00640 1 -- 172.21.15.119:0/2951784576 >> 172.21.15.119:0/2951784576 conn(0x7fc5bc06aba0 msgr2=0x7fc5bc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.787+0000 7fc5c3c00640 1 -- 172.21.15.119:0/2951784576 shutdown_connections 2024-04-25T10:11:55.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.787+0000 7fc5c3c00640 1 -- 172.21.15.119:0/2951784576 wait complete. 2024-04-25T10:11:55.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.789+0000 7fc5c3c00640 1 Processor -- start 2024-04-25T10:11:55.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.789+0000 7fc5c3c00640 1 -- start start 2024-04-25T10:11:55.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.790+0000 7fc5c3c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5bc0b42c0 0x7fc5bc1e6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.790+0000 7fc5c3c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5bc0b46a0 0x7fc5bc145f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.790+0000 7fc5c3c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5bc15aef0 0x7fc5bc0c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.790+0000 7fc5c3c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc06ec60 con 0x7fc5bc0b42c0 2024-04-25T10:11:55.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.790+0000 7fc5c3c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc06eae0 con 0x7fc5bc0b46a0 2024-04-25T10:11:55.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.790+0000 7fc5c3c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc154f10 con 0x7fc5bc15aef0 2024-04-25T10:11:55.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.790+0000 7fc5c1200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5bc0b42c0 0x7fc5bc1e6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.790+0000 7fc5c1200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5bc0b42c0 0x7fc5bc1e6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35054/0 (socket says 172.21.15.119:35054) 2024-04-25T10:11:55.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.790+0000 7fc5c1200640 1 -- 172.21.15.119:0/139967863 learned_addr learned my addr 172.21.15.119:0/139967863 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:55.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.791+0000 7fc5c1200640 1 -- 172.21.15.119:0/139967863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5bc15aef0 msgr2=0x7fc5bc0c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.791+0000 7fc5c1200640 1 --2- 172.21.15.119:0/139967863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5bc15aef0 0x7fc5bc0c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.791+0000 7fc5c1200640 1 -- 172.21.15.119:0/139967863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5bc0b46a0 msgr2=0x7fc5bc145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.791+0000 7fc5c1200640 1 --2- 172.21.15.119:0/139967863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5bc0b46a0 0x7fc5bc145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.791+0000 7fc5c1200640 1 -- 172.21.15.119:0/139967863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc1e7270 con 0x7fc5bc0b42c0 2024-04-25T10:11:55.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.791+0000 7fc5c1200640 1 --2- 172.21.15.119:0/139967863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5bc0b42c0 0x7fc5bc1e6c20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00d830 tx=0x7fc5ac00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.791+0000 7fc5b8c00640 1 -- 172.21.15.119:0/139967863 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00b6c0 con 0x7fc5bc0b42c0 2024-04-25T10:11:55.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.791+0000 7fc5b8c00640 1 -- 172.21.15.119:0/139967863 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ac019070 con 0x7fc5bc0b42c0 2024-04-25T10:11:55.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.791+0000 7fc5c3c00640 1 -- 172.21.15.119:0/139967863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc1e7550 con 0x7fc5bc0b42c0 2024-04-25T10:11:55.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.791+0000 7fc5b8c00640 1 -- 172.21.15.119:0/139967863 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00ba60 con 0x7fc5bc0b42c0 2024-04-25T10:11:55.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.792+0000 7fc5c3c00640 1 -- 172.21.15.119:0/139967863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc1e7b10 con 0x7fc5bc0b42c0 2024-04-25T10:11:55.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.793+0000 7fc5c3c00640 1 -- 172.21.15.119:0/139967863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5bc0750b0 con 0x7fc5bc0b42c0 2024-04-25T10:11:55.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.794+0000 7fc5b8c00640 1 -- 172.21.15.119:0/139967863 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5ac01e460 con 0x7fc5bc0b42c0 2024-04-25T10:11:55.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.794+0000 7fc5b8c00640 1 --2- 172.21.15.119:0/139967863 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc59806b500 0x7fc59806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.795+0000 7fc5b8c00640 1 -- 172.21.15.119:0/139967863 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc5ac08a950 con 0x7fc5bc0b42c0 2024-04-25T10:11:55.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.795+0000 7fc5bbe00640 1 --2- 172.21.15.119:0/139967863 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc59806b500 0x7fc59806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.795+0000 7fc5bbe00640 1 --2- 172.21.15.119:0/139967863 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc59806b500 0x7fc59806d9c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc15a380 tx=0x7fc5a4007d00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.799+0000 7fc5b8c00640 1 -- 172.21.15.119:0/139967863 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5ac059740 con 0x7fc5bc0b42c0 2024-04-25T10:11:56.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.996+0000 7fc5c3c00640 1 -- 172.21.15.119:0/139967863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5bc0b39b0 con 0x7fc5bc0b42c0 2024-04-25T10:11:56.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.997+0000 7fc5b8c00640 1 -- 172.21.15.119:0/139967863 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5ac059040 con 0x7fc5bc0b42c0 2024-04-25T10:11:56.002 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:11:56.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.999+0000 7fc5c3c00640 1 -- 172.21.15.119:0/139967863 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc59806b500 msgr2=0x7fc59806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.999+0000 7fc5c3c00640 1 --2- 172.21.15.119:0/139967863 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc59806b500 0x7fc59806d9c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc15a380 tx=0x7fc5a4007d00 comp rx=0 tx=0).stop 2024-04-25T10:11:56.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.999+0000 7fc5c3c00640 1 -- 172.21.15.119:0/139967863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5bc0b42c0 msgr2=0x7fc5bc1e6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.999+0000 7fc5c3c00640 1 --2- 172.21.15.119:0/139967863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5bc0b42c0 0x7fc5bc1e6c20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00d830 tx=0x7fc5ac00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.999+0000 7fc5c3c00640 1 -- 172.21.15.119:0/139967863 shutdown_connections 2024-04-25T10:11:56.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.999+0000 7fc5c3c00640 1 --2- 172.21.15.119:0/139967863 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc59806b500 0x7fc59806d9c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.999+0000 7fc5c3c00640 1 --2- 172.21.15.119:0/139967863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5bc15aef0 0x7fc5bc0c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.999+0000 7fc5c3c00640 1 --2- 172.21.15.119:0/139967863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5bc0b46a0 0x7fc5bc145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.999+0000 7fc5c3c00640 1 --2- 172.21.15.119:0/139967863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5bc0b42c0 0x7fc5bc1e6c20 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.999+0000 7fc5c3c00640 1 -- 172.21.15.119:0/139967863 >> 172.21.15.119:0/139967863 conn(0x7fc5bc06aba0 msgr2=0x7fc5bc1598c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.999+0000 7fc5c3c00640 1 -- 172.21.15.119:0/139967863 shutdown_connections 2024-04-25T10:11:56.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:55.999+0000 7fc5c3c00640 1 -- 172.21.15.119:0/139967863 wait complete. 2024-04-25T10:11:56.022 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:56.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.225+0000 7f2757200640 1 Processor -- start 2024-04-25T10:11:56.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.226+0000 7f2757200640 1 -- start start 2024-04-25T10:11:56.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.227+0000 7f2757200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f275015a260 0x7f27500cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.227+0000 7f2757200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2750071220 0x7f2750071680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.227+0000 7f2757200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275015dc30 con 0x7f275006e4f0 2024-04-25T10:11:56.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.227+0000 7f2757200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2750071bc0 con 0x7f275015a260 2024-04-25T10:11:56.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.227+0000 7f2757200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275015df10 con 0x7f2750071220 2024-04-25T10:11:56.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.227+0000 7f274fe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f275015a260 0x7f27500cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.228+0000 7f274fe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f275015a260 0x7f27500cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35058/0 (socket says 172.21.15.119:35058) 2024-04-25T10:11:56.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.228+0000 7f274fe00640 1 -- 172.21.15.119:0/2832169677 learned_addr learned my addr 172.21.15.119:0/2832169677 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:56.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.228+0000 7f2747400640 1 --2- 172.21.15.119:0/2832169677 >> v2:172.21.15.132:3300/0 conn(0x7f2750071220 0x7f2750071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.228+0000 7f274fe00640 1 -- 172.21.15.119:0/2832169677 >> v2:172.21.15.132:3300/0 conn(0x7f2750071220 msgr2=0x7f2750071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.228+0000 7f274fe00640 1 --2- 172.21.15.119:0/2832169677 >> v2:172.21.15.132:3300/0 conn(0x7f2750071220 0x7f2750071680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.228+0000 7f274fe00640 1 -- 172.21.15.119:0/2832169677 >> v1:172.21.15.119:6789/0 conn(0x7f275006e4f0 legacy=0x7f275006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.228+0000 7f274f400640 1 -- 172.21.15.119:0/2832169677 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318118247 0 0) 0x7f275015dc30 con 0x7f275006e4f0 2024-04-25T10:11:56.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.228+0000 7f274fe00640 1 -- 172.21.15.119:0/2832169677 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275015e1f0 con 0x7f275015a260 2024-04-25T10:11:56.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.228+0000 7f2747400640 1 --2- 172.21.15.119:0/2832169677 >> v2:172.21.15.132:3300/0 conn(0x7f2750071220 0x7f2750071680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:56.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.229+0000 7f274fe00640 1 --2- 172.21.15.119:0/2832169677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275015a260 0x7f27500cf6f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f27500673d0 tx=0x7f274002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3fb8aecff0c57e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.229+0000 7f274f400640 1 -- 172.21.15.119:0/2832169677 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740037070 con 0x7f275015a260 2024-04-25T10:11:56.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.230+0000 7f274f400640 1 -- 172.21.15.119:0/2832169677 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2740032440 con 0x7f275015a260 2024-04-25T10:11:56.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.230+0000 7f274f400640 1 -- 172.21.15.119:0/2832169677 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740032810 con 0x7f275015a260 2024-04-25T10:11:56.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.230+0000 7f2757200640 1 -- 172.21.15.119:0/2832169677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275015a260 msgr2=0x7f27500cf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.230+0000 7f2757200640 1 --2- 172.21.15.119:0/2832169677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275015a260 0x7f27500cf6f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f27500673d0 tx=0x7f274002a270 comp rx=0 tx=0).stop 2024-04-25T10:11:56.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.230+0000 7f2757200640 1 -- 172.21.15.119:0/2832169677 shutdown_connections 2024-04-25T10:11:56.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.230+0000 7f2757200640 1 --2- 172.21.15.119:0/2832169677 >> v2:172.21.15.132:3300/0 conn(0x7f2750071220 0x7f2750071680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.230+0000 7f2757200640 1 --2- 172.21.15.119:0/2832169677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f275015a260 0x7f27500cf6f0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.230+0000 7f2757200640 1 -- 172.21.15.119:0/2832169677 >> 172.21.15.119:0/2832169677 conn(0x7f275006aba0 msgr2=0x7f275006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.230+0000 7f2757200640 1 -- 172.21.15.119:0/2832169677 shutdown_connections 2024-04-25T10:11:56.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.230+0000 7f2757200640 1 -- 172.21.15.119:0/2832169677 wait complete. 2024-04-25T10:11:56.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.232+0000 7f2757200640 1 Processor -- start 2024-04-25T10:11:56.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.232+0000 7f2757200640 1 -- start start 2024-04-25T10:11:56.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.233+0000 7f2757200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f275006e4f0 0x7f2750159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.233+0000 7f2757200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2750071220 0x7f27501f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.233+0000 7f2757200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275015a260 0x7f275014cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.233+0000 7f2757200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27500b3760 con 0x7f2750071220 2024-04-25T10:11:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.233+0000 7f2757200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275015e7b0 con 0x7f275015a260 2024-04-25T10:11:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.233+0000 7f2757200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27500b38e0 con 0x7f275006e4f0 2024-04-25T10:11:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.233+0000 7f2747400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2750071220 0x7f27501f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.233+0000 7f2747400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2750071220 0x7f27501f2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35068/0 (socket says 172.21.15.119:35068) 2024-04-25T10:11:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.233+0000 7f2747400640 1 -- 172.21.15.119:0/1695175365 learned_addr learned my addr 172.21.15.119:0/1695175365 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.234+0000 7f274fe00640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f275006e4f0 0x7f2750159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.234+0000 7f2747400640 1 -- 172.21.15.119:0/1695175365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f275006e4f0 msgr2=0x7f2750159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.234+0000 7f2747400640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f275006e4f0 0x7f2750159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.234+0000 7f2747400640 1 -- 172.21.15.119:0/1695175365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275015a260 msgr2=0x7f275014cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.234+0000 7f2755200640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275015a260 0x7f275014cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.234+0000 7f2747400640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275015a260 0x7f275014cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.234+0000 7f2747400640 1 -- 172.21.15.119:0/1695175365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27501f2fc0 con 0x7f2750071220 2024-04-25T10:11:56.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.234+0000 7f2755200640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275015a260 0x7f275014cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:56.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.234+0000 7f2747400640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2750071220 0x7f27501f2a80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f273800c7f0 tx=0x7f273800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.234+0000 7f274cc00640 1 -- 172.21.15.119:0/1695175365 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738015070 con 0x7f2750071220 2024-04-25T10:11:56.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.235+0000 7f274cc00640 1 -- 172.21.15.119:0/1695175365 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2738007a30 con 0x7f2750071220 2024-04-25T10:11:56.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.235+0000 7f2757200640 1 -- 172.21.15.119:0/1695175365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275014b580 con 0x7f2750071220 2024-04-25T10:11:56.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.235+0000 7f274cc00640 1 -- 172.21.15.119:0/1695175365 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738007e00 con 0x7f2750071220 2024-04-25T10:11:56.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.235+0000 7f2757200640 1 -- 172.21.15.119:0/1695175365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275014bb40 con 0x7f2750071220 2024-04-25T10:11:56.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.236+0000 7f2757200640 1 -- 172.21.15.119:0/1695175365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2750071900 con 0x7f2750071220 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.237+0000 7f274cc00640 1 -- 172.21.15.119:0/1695175365 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2738018070 con 0x7f2750071220 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.238+0000 7f274cc00640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f272c06b4d0 0x7f272c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.238+0000 7f274cc00640 1 -- 172.21.15.119:0/1695175365 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f273808a250 con 0x7f2750071220 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.238+0000 7f274fe00640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f272c06b4d0 0x7f272c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.239+0000 7f274fe00640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f272c06b4d0 0x7f272c06d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2740034000 tx=0x7f274002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.242+0000 7f274cc00640 1 -- 172.21.15.119:0/1695175365 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2750071900 con 0x7f2750071220 2024-04-25T10:11:56.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.436+0000 7f2757200640 1 -- 172.21.15.119:0/1695175365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27500750b0 con 0x7f2750071220 2024-04-25T10:11:56.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.436+0000 7f274cc00640 1 -- 172.21.15.119:0/1695175365 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2738059030 con 0x7f2750071220 2024-04-25T10:11:56.441 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:11:56.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.439+0000 7f2757200640 1 -- 172.21.15.119:0/1695175365 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f272c06b4d0 msgr2=0x7f272c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.439+0000 7f2757200640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f272c06b4d0 0x7f272c06d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2740034000 tx=0x7f274002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:56.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.439+0000 7f2757200640 1 -- 172.21.15.119:0/1695175365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2750071220 msgr2=0x7f27501f2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.439+0000 7f2757200640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2750071220 0x7f27501f2a80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f273800c7f0 tx=0x7f273800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.439+0000 7f2757200640 1 -- 172.21.15.119:0/1695175365 shutdown_connections 2024-04-25T10:11:56.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.439+0000 7f2757200640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f272c06b4d0 0x7f272c06d990 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.439+0000 7f2757200640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f275015a260 0x7f275014cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.439+0000 7f2757200640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2750071220 0x7f27501f2a80 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.439+0000 7f2757200640 1 --2- 172.21.15.119:0/1695175365 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f275006e4f0 0x7f2750159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.439+0000 7f2757200640 1 -- 172.21.15.119:0/1695175365 >> 172.21.15.119:0/1695175365 conn(0x7f275006aba0 msgr2=0x7f27500cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.439+0000 7f2757200640 1 -- 172.21.15.119:0/1695175365 shutdown_connections 2024-04-25T10:11:56.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.439+0000 7f2757200640 1 -- 172.21.15.119:0/1695175365 wait complete. 2024-04-25T10:11:56.464 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:56.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.654+0000 7f5b72800640 1 Processor -- start 2024-04-25T10:11:56.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.654+0000 7f5b72800640 1 -- start start 2024-04-25T10:11:56.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.656+0000 7f5b72800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b6c0ce9d0 0x7f5b6c0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.656+0000 7f5b72800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5b6c0cc300 0x7f5b6c0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.656+0000 7f5b72800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b6c06dcb0 con 0x7f5b6c144e70 2024-04-25T10:11:56.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.656+0000 7f5b72800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b6c15bfe0 con 0x7f5b6c0cc300 2024-04-25T10:11:56.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.656+0000 7f5b72800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c06df90 con 0x7f5b6c0ce9d0 2024-04-25T10:11:56.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.656+0000 7f5b6b400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5b6c144e70 0x7f5b6c06d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38008/0 (socket says 172.21.15.119:38008) 2024-04-25T10:11:56.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.656+0000 7f5b6b400640 1 -- 172.21.15.119:0/1855068870 learned_addr learned my addr 172.21.15.119:0/1855068870 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:56.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.656+0000 7f5b6be00640 1 --2- 172.21.15.119:0/1855068870 >> v2:172.21.15.132:3300/0 conn(0x7f5b6c0cc300 0x7f5b6c0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.656+0000 7f5b6aa00640 1 --2- 172.21.15.119:0/1855068870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b6c0ce9d0 0x7f5b6c0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.656+0000 7f5b6a000640 1 -- 172.21.15.119:0/1855068870 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649716816 0 0) 0x7f5b6c06dcb0 con 0x7f5b6c144e70 2024-04-25T10:11:56.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 -- 172.21.15.119:0/1855068870 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b50003520 con 0x7f5b6c144e70 2024-04-25T10:11:56.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 -- 172.21.15.119:0/1855068870 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 282227731 0 0) 0x7f5b50003520 con 0x7f5b6c144e70 2024-04-25T10:11:56.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 -- 172.21.15.119:0/1855068870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b6c0ce9d0 msgr2=0x7f5b6c0cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 --2- 172.21.15.119:0/1855068870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b6c0ce9d0 0x7f5b6c0cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 -- 172.21.15.119:0/1855068870 >> v2:172.21.15.132:3300/0 conn(0x7f5b6c0cc300 msgr2=0x7f5b6c0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 --2- 172.21.15.119:0/1855068870 >> v2:172.21.15.132:3300/0 conn(0x7f5b6c0cc300 0x7f5b6c0cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 -- 172.21.15.119:0/1855068870 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6c06e270 con 0x7f5b6c144e70 2024-04-25T10:11:56.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 -- 172.21.15.119:0/1855068870 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f5b5c001690 con 0x7f5b6c144e70 2024-04-25T10:11:56.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 -- 172.21.15.119:0/1855068870 >> v1:172.21.15.119:6789/0 conn(0x7f5b6c144e70 legacy=0x7f5b6c06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 --2- 172.21.15.119:0/1855068870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b500039b0 0x7f5b50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 --2- 172.21.15.119:0/1855068870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b500043a0 0x7f5b50004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 --2- 172.21.15.119:0/1855068870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b50005040 0x7f5b5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 -- 172.21.15.119:0/1855068870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c15bfe0 con 0x7f5b500043a0 2024-04-25T10:11:56.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 -- 172.21.15.119:0/1855068870 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c06df90 con 0x7f5b50005040 2024-04-25T10:11:56.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6a000640 1 -- 172.21.15.119:0/1855068870 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5000bfe0 con 0x7f5b500039b0 2024-04-25T10:11:56.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.657+0000 7f5b6aa00640 1 --2- 172.21.15.119:0/1855068870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b6c0ce9d0 0x7f5b6c0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:56.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.658+0000 7f5b6aa00640 1 --2- 172.21.15.119:0/1855068870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b500043a0 0x7f5b50004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.658+0000 7f5b6be00640 1 --2- 172.21.15.119:0/1855068870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b50005040 0x7f5b5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.658+0000 7f5b6b400640 1 --2- 172.21.15.119:0/1855068870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b500039b0 0x7f5b50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.658+0000 7f5b6aa00640 1 -- 172.21.15.119:0/1855068870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b500039b0 msgr2=0x7f5b50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.658+0000 7f5b6aa00640 1 --2- 172.21.15.119:0/1855068870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b500039b0 0x7f5b50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.658+0000 7f5b6aa00640 1 -- 172.21.15.119:0/1855068870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b50005040 msgr2=0x7f5b5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.658+0000 7f5b6aa00640 1 --2- 172.21.15.119:0/1855068870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b50005040 0x7f5b5000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.658+0000 7f5b6aa00640 1 -- 172.21.15.119:0/1855068870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b50003520 con 0x7f5b500043a0 2024-04-25T10:11:56.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.659+0000 7f5b6aa00640 1 --2- 172.21.15.119:0/1855068870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b500043a0 0x7f5b50004880 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f5b58006e80 tx=0x7f5b50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af9ac114f3c853 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.659+0000 7f5b6a000640 1 -- 172.21.15.119:0/1855068870 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b58037070 con 0x7f5b500043a0 2024-04-25T10:11:56.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.659+0000 7f5b6a000640 1 -- 172.21.15.119:0/1855068870 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5802aae0 con 0x7f5b500043a0 2024-04-25T10:11:56.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.659+0000 7f5b6a000640 1 -- 172.21.15.119:0/1855068870 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5800f040 con 0x7f5b500043a0 2024-04-25T10:11:56.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.660+0000 7f5b72800640 1 -- 172.21.15.119:0/1855068870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b500043a0 msgr2=0x7f5b50004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.660+0000 7f5b72800640 1 --2- 172.21.15.119:0/1855068870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b500043a0 0x7f5b50004880 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f5b58006e80 tx=0x7f5b50005830 comp rx=0 tx=0).stop 2024-04-25T10:11:56.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.660+0000 7f5b6be00640 1 -- 172.21.15.119:0/1855068870 reap_dead start 2024-04-25T10:11:56.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.660+0000 7f5b72800640 1 -- 172.21.15.119:0/1855068870 shutdown_connections 2024-04-25T10:11:56.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.660+0000 7f5b72800640 1 -- 172.21.15.119:0/1855068870 >> 172.21.15.119:0/1855068870 conn(0x7f5b6c06aba0 msgr2=0x7f5b6c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.660+0000 7f5b72800640 1 -- 172.21.15.119:0/1855068870 shutdown_connections 2024-04-25T10:11:56.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.660+0000 7f5b72800640 1 -- 172.21.15.119:0/1855068870 wait complete. 2024-04-25T10:11:56.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.662+0000 7f5b72800640 1 Processor -- start 2024-04-25T10:11:56.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.662+0000 7f5b72800640 1 -- start start 2024-04-25T10:11:56.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.663+0000 7f5b72800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b500043a0 0x7f5b6c13c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.663+0000 7f5b72800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b6c156340 0x7f5b6c0afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.663+0000 7f5b72800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b6c15e330 0x7f5b6c15e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.663+0000 7f5b72800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c1454b0 con 0x7f5b500043a0 2024-04-25T10:11:56.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.663+0000 7f5b72800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c06d970 con 0x7f5b6c15e330 2024-04-25T10:11:56.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.663+0000 7f5b72800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c145630 con 0x7f5b6c156340 2024-04-25T10:11:56.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.663+0000 7f5b6b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b500043a0 0x7f5b6c13c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.663+0000 7f5b6b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b500043a0 0x7f5b6c13c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35078/0 (socket says 172.21.15.119:35078) 2024-04-25T10:11:56.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.663+0000 7f5b6b400640 1 -- 172.21.15.119:0/3180235956 learned_addr learned my addr 172.21.15.119:0/3180235956 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:56.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.664+0000 7f5b6aa00640 1 --2- 172.21.15.119:0/3180235956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b6c156340 0x7f5b6c0afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.664+0000 7f5b6b400640 1 -- 172.21.15.119:0/3180235956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b6c156340 msgr2=0x7f5b6c0afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.664+0000 7f5b6be00640 1 --2- 172.21.15.119:0/3180235956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b6c15e330 0x7f5b6c15e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.664+0000 7f5b6b400640 1 --2- 172.21.15.119:0/3180235956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b6c156340 0x7f5b6c0afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.664+0000 7f5b6b400640 1 -- 172.21.15.119:0/3180235956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b6c15e330 msgr2=0x7f5b6c15e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.664+0000 7f5b6b400640 1 --2- 172.21.15.119:0/3180235956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b6c15e330 0x7f5b6c15e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.664+0000 7f5b6b400640 1 -- 172.21.15.119:0/3180235956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6c0b05d0 con 0x7f5b500043a0 2024-04-25T10:11:56.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.664+0000 7f5b6b400640 1 --2- 172.21.15.119:0/3180235956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b500043a0 0x7f5b6c13c350 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00e7b0 tx=0x7f5b5c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.664+0000 7f5b63400640 1 -- 172.21.15.119:0/3180235956 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c00ee60 con 0x7f5b500043a0 2024-04-25T10:11:56.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.664+0000 7f5b63400640 1 -- 172.21.15.119:0/3180235956 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c018070 con 0x7f5b500043a0 2024-04-25T10:11:56.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.664+0000 7f5b63400640 1 -- 172.21.15.119:0/3180235956 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c00cb30 con 0x7f5b500043a0 2024-04-25T10:11:56.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.664+0000 7f5b72800640 1 -- 172.21.15.119:0/3180235956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6c0b08b0 con 0x7f5b500043a0 2024-04-25T10:11:56.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.665+0000 7f5b72800640 1 -- 172.21.15.119:0/3180235956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6c0b0e70 con 0x7f5b500043a0 2024-04-25T10:11:56.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.666+0000 7f5b72800640 1 -- 172.21.15.119:0/3180235956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6c0750b0 con 0x7f5b500043a0 2024-04-25T10:11:56.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.667+0000 7f5b63400640 1 -- 172.21.15.119:0/3180235956 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b5c00ccd0 con 0x7f5b500043a0 2024-04-25T10:11:56.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.667+0000 7f5b63400640 1 --2- 172.21.15.119:0/3180235956 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5b4c06b500 0x7f5b4c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.668+0000 7f5b63400640 1 -- 172.21.15.119:0/3180235956 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5b5c089e70 con 0x7f5b500043a0 2024-04-25T10:11:56.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.668+0000 7f5b6aa00640 1 --2- 172.21.15.119:0/3180235956 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5b4c06b500 0x7f5b4c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.668+0000 7f5b6aa00640 1 --2- 172.21.15.119:0/3180235956 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5b4c06b500 0x7f5b4c06d9c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5b58034000 tx=0x7f5b5802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.672+0000 7f5b63400640 1 -- 172.21.15.119:0/3180235956 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b5c058bc0 con 0x7f5b500043a0 2024-04-25T10:11:56.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.871+0000 7f5b72800640 1 -- 172.21.15.119:0/3180235956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b6c0aebe0 con 0x7f5b500043a0 2024-04-25T10:11:56.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.872+0000 7f5b63400640 1 -- 172.21.15.119:0/3180235956 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5b5c0584c0 con 0x7f5b500043a0 2024-04-25T10:11:56.877 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:11:56.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.875+0000 7f5b72800640 1 -- 172.21.15.119:0/3180235956 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5b4c06b500 msgr2=0x7f5b4c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.875+0000 7f5b72800640 1 --2- 172.21.15.119:0/3180235956 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5b4c06b500 0x7f5b4c06d9c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5b58034000 tx=0x7f5b5802e000 comp rx=0 tx=0).stop 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.875+0000 7f5b72800640 1 -- 172.21.15.119:0/3180235956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b500043a0 msgr2=0x7f5b6c13c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.875+0000 7f5b72800640 1 --2- 172.21.15.119:0/3180235956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b500043a0 0x7f5b6c13c350 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00e7b0 tx=0x7f5b5c00ec70 comp rx=0 tx=0).stop 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.875+0000 7f5b72800640 1 -- 172.21.15.119:0/3180235956 shutdown_connections 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.875+0000 7f5b72800640 1 --2- 172.21.15.119:0/3180235956 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5b4c06b500 0x7f5b4c06d9c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.875+0000 7f5b72800640 1 --2- 172.21.15.119:0/3180235956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b6c15e330 0x7f5b6c15e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.875+0000 7f5b72800640 1 --2- 172.21.15.119:0/3180235956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b6c156340 0x7f5b6c0afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.875+0000 7f5b72800640 1 --2- 172.21.15.119:0/3180235956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b500043a0 0x7f5b6c13c350 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.875+0000 7f5b72800640 1 -- 172.21.15.119:0/3180235956 >> 172.21.15.119:0/3180235956 conn(0x7f5b6c06aba0 msgr2=0x7f5b6c142270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.875+0000 7f5b72800640 1 -- 172.21.15.119:0/3180235956 shutdown_connections 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:56.875+0000 7f5b72800640 1 -- 172.21.15.119:0/3180235956 wait complete. 2024-04-25T10:11:56.900 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:57.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.097+0000 7fbdaea00640 1 Processor -- start 2024-04-25T10:11:57.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.097+0000 7fbdaea00640 1 -- start start 2024-04-25T10:11:57.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.099+0000 7fbdaea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbda81450e0 0x7fbda806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.099+0000 7fbdaea00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda815f400 con 0x7fbda80cc9e0 2024-04-25T10:11:57.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.099+0000 7fbdaea00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda815f6e0 con 0x7fbda806b0e0 2024-04-25T10:11:57.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.099+0000 7fbdaea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda81454c0 con 0x7fbda81450e0 2024-04-25T10:11:57.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.099+0000 7fbdae000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbda80cc9e0 0x7fbda80b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38022/0 (socket says 172.21.15.119:38022) 2024-04-25T10:11:57.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.099+0000 7fbdae000640 1 -- 172.21.15.119:0/4280369850 learned_addr learned my addr 172.21.15.119:0/4280369850 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:57.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbdad600640 1 --2- 172.21.15.119:0/4280369850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbda81450e0 0x7fbda806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 -- 172.21.15.119:0/4280369850 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291708372 0 0) 0x7fbda815f400 con 0x7fbda80cc9e0 2024-04-25T10:11:57.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 -- 172.21.15.119:0/4280369850 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd90003520 con 0x7fbda80cc9e0 2024-04-25T10:11:57.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 -- 172.21.15.119:0/4280369850 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1889495417 0 0) 0x7fbd90003520 con 0x7fbda80cc9e0 2024-04-25T10:11:57.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 -- 172.21.15.119:0/4280369850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbda81450e0 msgr2=0x7fbda806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 --2- 172.21.15.119:0/4280369850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbda81450e0 0x7fbda806aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 -- 172.21.15.119:0/4280369850 >> v1:172.21.15.132:6789/0 conn(0x7fbda806b0e0 legacy=0x7fbda80cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 -- 172.21.15.119:0/4280369850 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda815f9c0 con 0x7fbda80cc9e0 2024-04-25T10:11:57.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 -- 172.21.15.119:0/4280369850 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fbda00019f0 con 0x7fbda80cc9e0 2024-04-25T10:11:57.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 -- 172.21.15.119:0/4280369850 >> v1:172.21.15.119:6789/0 conn(0x7fbda80cc9e0 legacy=0x7fbda80b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 --2- 172.21.15.119:0/4280369850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd900039e0 0x7fbd90003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 --2- 172.21.15.119:0/4280369850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbda81450e0 0x7fbd90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 --2- 172.21.15.119:0/4280369850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd90004e60 0x7fbd9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 -- 172.21.15.119:0/4280369850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900055f0 con 0x7fbd900039e0 2024-04-25T10:11:57.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 -- 172.21.15.119:0/4280369850 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9000bf00 con 0x7fbd90004e60 2024-04-25T10:11:57.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.100+0000 7fbda7e00640 1 -- 172.21.15.119:0/4280369850 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9000c080 con 0x7fbda81450e0 2024-04-25T10:11:57.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.101+0000 7fbdad600640 1 --2- 172.21.15.119:0/4280369850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd900039e0 0x7fbd90003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.101+0000 7fbdad600640 1 -- 172.21.15.119:0/4280369850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbda81450e0 msgr2=0x7fbd90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.101+0000 7fbdad600640 1 --2- 172.21.15.119:0/4280369850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbda81450e0 0x7fbd90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.101+0000 7fbdad600640 1 -- 172.21.15.119:0/4280369850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd90004e60 msgr2=0x7fbd9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.101+0000 7fbdad600640 1 --2- 172.21.15.119:0/4280369850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd90004e60 0x7fbd9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.101+0000 7fbdad600640 1 -- 172.21.15.119:0/4280369850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd90003520 con 0x7fbd900039e0 2024-04-25T10:11:57.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.102+0000 7fbdad600640 1 --2- 172.21.15.119:0/4280369850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd900039e0 0x7fbd90003e00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00ba30 tx=0x7fbd90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0227dc010545b5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.102+0000 7fbda7e00640 1 -- 172.21.15.119:0/4280369850 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c030070 con 0x7fbd900039e0 2024-04-25T10:11:57.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.102+0000 7fbda7e00640 1 -- 172.21.15.119:0/4280369850 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c02ae00 con 0x7fbd900039e0 2024-04-25T10:11:57.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.102+0000 7fbda7e00640 1 -- 172.21.15.119:0/4280369850 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c00b600 con 0x7fbd900039e0 2024-04-25T10:11:57.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.103+0000 7fbdaea00640 1 -- 172.21.15.119:0/4280369850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd900039e0 msgr2=0x7fbd90003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.103+0000 7fbdaea00640 1 --2- 172.21.15.119:0/4280369850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd900039e0 0x7fbd90003e00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00ba30 tx=0x7fbd90005830 comp rx=0 tx=0).stop 2024-04-25T10:11:57.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.103+0000 7fbdae000640 1 -- 172.21.15.119:0/4280369850 reap_dead start 2024-04-25T10:11:57.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.103+0000 7fbdaea00640 1 -- 172.21.15.119:0/4280369850 shutdown_connections 2024-04-25T10:11:57.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.103+0000 7fbdaea00640 1 -- 172.21.15.119:0/4280369850 >> 172.21.15.119:0/4280369850 conn(0x7fbda8066ff0 msgr2=0x7fbda80b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.103+0000 7fbdaea00640 1 -- 172.21.15.119:0/4280369850 shutdown_connections 2024-04-25T10:11:57.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.103+0000 7fbdaea00640 1 -- 172.21.15.119:0/4280369850 wait complete. 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.105+0000 7fbdaea00640 1 Processor -- start 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.105+0000 7fbdaea00640 1 -- start start 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.106+0000 7fbdaea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd900039e0 0x7fbda80a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.106+0000 7fbdaea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbda8130490 0x7fbda8130910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.106+0000 7fbdaea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda8132ff0 0x7fbda81334a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.106+0000 7fbdaea00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda806aba0 con 0x7fbda8132ff0 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.106+0000 7fbdaea00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda806ad20 con 0x7fbd900039e0 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.106+0000 7fbdaea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda806aea0 con 0x7fbda8130490 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.106+0000 7fbdae000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda8132ff0 0x7fbda81334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.106+0000 7fbdae000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda8132ff0 0x7fbda81334a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35086/0 (socket says 172.21.15.119:35086) 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.106+0000 7fbdae000640 1 -- 172.21.15.119:0/3341253021 learned_addr learned my addr 172.21.15.119:0/3341253021 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.107+0000 7fbdad600640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd900039e0 0x7fbda80a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.107+0000 7fbdacc00640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbda8130490 0x7fbda8130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.107+0000 7fbdae000640 1 -- 172.21.15.119:0/3341253021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbda8130490 msgr2=0x7fbda8130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.107+0000 7fbdae000640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbda8130490 0x7fbda8130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.107+0000 7fbdae000640 1 -- 172.21.15.119:0/3341253021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd900039e0 msgr2=0x7fbda80a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.107+0000 7fbdae000640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd900039e0 0x7fbda80a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.107+0000 7fbdae000640 1 -- 172.21.15.119:0/3341253021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda80a8b30 con 0x7fbda8132ff0 2024-04-25T10:11:57.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.107+0000 7fbdad600640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd900039e0 0x7fbda80a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:57.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.107+0000 7fbdae000640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda8132ff0 0x7fbda81334a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbda80cbe60 tx=0x7fbda000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.107+0000 7fbda5600640 1 -- 172.21.15.119:0/3341253021 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda0007110 con 0x7fbda8132ff0 2024-04-25T10:11:57.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.108+0000 7fbda5600640 1 -- 172.21.15.119:0/3341253021 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda00072b0 con 0x7fbda8132ff0 2024-04-25T10:11:57.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.108+0000 7fbda5600640 1 -- 172.21.15.119:0/3341253021 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda00076a0 con 0x7fbda8132ff0 2024-04-25T10:11:57.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.108+0000 7fbdaea00640 1 -- 172.21.15.119:0/3341253021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda80a8e10 con 0x7fbda8132ff0 2024-04-25T10:11:57.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.108+0000 7fbdaea00640 1 -- 172.21.15.119:0/3341253021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda80a93d0 con 0x7fbda8132ff0 2024-04-25T10:11:57.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.109+0000 7fbdaea00640 1 -- 172.21.15.119:0/3341253021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda80a9700 con 0x7fbda8132ff0 2024-04-25T10:11:57.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.110+0000 7fbda5600640 1 -- 172.21.15.119:0/3341253021 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbda0007840 con 0x7fbda8132ff0 2024-04-25T10:11:57.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.111+0000 7fbda5600640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd8806b500 0x7fbd8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.111+0000 7fbda5600640 1 -- 172.21.15.119:0/3341253021 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fbda0089be0 con 0x7fbda8132ff0 2024-04-25T10:11:57.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.111+0000 7fbdad600640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd8806b500 0x7fbd8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.112+0000 7fbdad600640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd8806b500 0x7fbd8806d9c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00ba00 tx=0x7fbd9c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.116+0000 7fbda5600640 1 -- 172.21.15.119:0/3341253021 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbda001b020 con 0x7fbda8132ff0 2024-04-25T10:11:57.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.321+0000 7fbdaea00640 1 -- 172.21.15.119:0/3341253021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbda80749b0 con 0x7fbda8132ff0 2024-04-25T10:11:57.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.322+0000 7fbda5600640 1 -- 172.21.15.119:0/3341253021 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v70) v1 ==== 80+0+124 (secure 0 0 0) 0x7fbda00589c0 con 0x7fbda8132ff0 2024-04-25T10:11:57.327 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6837/2673349178 2024-04-26T10:11:47.336991+0000 2024-04-25T10:11:57.328 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6836/2673349178 2024-04-26T10:11:47.336991+0000 2024-04-25T10:11:57.328 INFO:teuthology.orchestra.run.smithi119.stderr:listed 2 entries 2024-04-25T10:11:57.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.325+0000 7fbdaea00640 1 -- 172.21.15.119:0/3341253021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd8806b500 msgr2=0x7fbd8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.325+0000 7fbdaea00640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd8806b500 0x7fbd8806d9c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00ba00 tx=0x7fbd9c030000 comp rx=0 tx=0).stop 2024-04-25T10:11:57.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.325+0000 7fbdaea00640 1 -- 172.21.15.119:0/3341253021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda8132ff0 msgr2=0x7fbda81334a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.325+0000 7fbdaea00640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda8132ff0 0x7fbda81334a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbda80cbe60 tx=0x7fbda000cc60 comp rx=0 tx=0).stop 2024-04-25T10:11:57.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.325+0000 7fbdaea00640 1 -- 172.21.15.119:0/3341253021 shutdown_connections 2024-04-25T10:11:57.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.325+0000 7fbdaea00640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd8806b500 0x7fbd8806d9c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.325+0000 7fbdaea00640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda8132ff0 0x7fbda81334a0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.325+0000 7fbdaea00640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbda8130490 0x7fbda8130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.325+0000 7fbdaea00640 1 --2- 172.21.15.119:0/3341253021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd900039e0 0x7fbda80a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.325+0000 7fbdaea00640 1 -- 172.21.15.119:0/3341253021 >> 172.21.15.119:0/3341253021 conn(0x7fbda8066ff0 msgr2=0x7fbda80aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.325+0000 7fbdaea00640 1 -- 172.21.15.119:0/3341253021 shutdown_connections 2024-04-25T10:11:57.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.326+0000 7fbdaea00640 1 -- 172.21.15.119:0/3341253021 wait complete. 2024-04-25T10:11:57.349 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:11:57.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:57.349 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:57.429 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:11:57.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:57.430 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:11:57.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:57.477 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-25T10:11:57.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.667+0000 7f0662a00640 1 Processor -- start 2024-04-25T10:11:57.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.667+0000 7f0662a00640 1 -- start start 2024-04-25T10:11:57.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.669+0000 7f0662a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f065c0ca5b0 0x7f065c0ca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.669+0000 7f0662a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065c0caed0 con 0x7f065c06b260 2024-04-25T10:11:57.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.669+0000 7f0662a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065c06c030 con 0x7f065c0ce4b0 2024-04-25T10:11:57.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.669+0000 7f0662a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065c0ce020 con 0x7f065c0ca5b0 2024-04-25T10:11:57.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.669+0000 7f065b400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f065c06b260 0x7f065c06b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38032/0 (socket says 172.21.15.119:38032) 2024-04-25T10:11:57.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.669+0000 7f065b400640 1 -- 172.21.15.119:0/3249577883 learned_addr learned my addr 172.21.15.119:0/3249577883 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:11:57.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065be00640 1 --2- 172.21.15.119:0/3249577883 >> v2:172.21.15.132:3300/0 conn(0x7f065c0ca5b0 0x7f065c0ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 -- 172.21.15.119:0/3249577883 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480437615 0 0) 0x7f065c0caed0 con 0x7f065c06b260 2024-04-25T10:11:57.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 -- 172.21.15.119:0/3249577883 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0640003520 con 0x7f065c06b260 2024-04-25T10:11:57.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 -- 172.21.15.119:0/3249577883 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 946109650 0 0) 0x7f0640003520 con 0x7f065c06b260 2024-04-25T10:11:57.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 -- 172.21.15.119:0/3249577883 >> v2:172.21.15.132:3300/0 conn(0x7f065c0ca5b0 msgr2=0x7f065c0ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 --2- 172.21.15.119:0/3249577883 >> v2:172.21.15.132:3300/0 conn(0x7f065c0ca5b0 0x7f065c0ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 -- 172.21.15.119:0/3249577883 >> v1:172.21.15.132:6789/0 conn(0x7f065c0ce4b0 legacy=0x7f065c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 -- 172.21.15.119:0/3249577883 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c06c310 con 0x7f065c06b260 2024-04-25T10:11:57.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 -- 172.21.15.119:0/3249577883 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f0648001670 con 0x7f065c06b260 2024-04-25T10:11:57.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 -- 172.21.15.119:0/3249577883 >> v1:172.21.15.119:6789/0 conn(0x7f065c06b260 legacy=0x7f065c06b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 --2- 172.21.15.119:0/3249577883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06400039b0 0x7f0640003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 --2- 172.21.15.119:0/3249577883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640004800 0x7f0640004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 --2- 172.21.15.119:0/3249577883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06400051c0 0x7f064000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 -- 172.21.15.119:0/3249577883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0ce020 con 0x7f06400051c0 2024-04-25T10:11:57.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 -- 172.21.15.119:0/3249577883 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064000c040 con 0x7f0640004800 2024-04-25T10:11:57.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.670+0000 7f065aa00640 1 -- 172.21.15.119:0/3249577883 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064000c240 con 0x7f06400039b0 2024-04-25T10:11:57.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.671+0000 7f0660a00640 1 --2- 172.21.15.119:0/3249577883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06400051c0 0x7f064000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.671+0000 7f065b400640 1 --2- 172.21.15.119:0/3249577883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640004800 0x7f0640004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.671+0000 7f065be00640 1 --2- 172.21.15.119:0/3249577883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06400039b0 0x7f0640003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.671+0000 7f0660a00640 1 -- 172.21.15.119:0/3249577883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06400039b0 msgr2=0x7f0640003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.671+0000 7f0660a00640 1 --2- 172.21.15.119:0/3249577883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06400039b0 0x7f0640003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.671+0000 7f0660a00640 1 -- 172.21.15.119:0/3249577883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640004800 msgr2=0x7f0640004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.671+0000 7f0660a00640 1 --2- 172.21.15.119:0/3249577883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0640004800 0x7f0640004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.671+0000 7f0660a00640 1 -- 172.21.15.119:0/3249577883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0640003520 con 0x7f06400051c0 2024-04-25T10:11:57.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.672+0000 7f0660a00640 1 -- 172.21.15.119:0/3249577883 reap_dead start 2024-04-25T10:11:57.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.672+0000 7f0660a00640 1 --2- 172.21.15.119:0/3249577883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06400051c0 0x7f064000b9c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f0654003db0 tx=0x7f0640005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bf107250a7cb14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.672+0000 7f065aa00640 1 -- 172.21.15.119:0/3249577883 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654037070 con 0x7f06400051c0 2024-04-25T10:11:57.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.672+0000 7f065aa00640 1 -- 172.21.15.119:0/3249577883 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065402aa00 con 0x7f06400051c0 2024-04-25T10:11:57.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.672+0000 7f065aa00640 1 -- 172.21.15.119:0/3249577883 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654003980 con 0x7f06400051c0 2024-04-25T10:11:57.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.673+0000 7f0662a00640 1 -- 172.21.15.119:0/3249577883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06400051c0 msgr2=0x7f064000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.673+0000 7f0662a00640 1 --2- 172.21.15.119:0/3249577883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06400051c0 0x7f064000b9c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f0654003db0 tx=0x7f0640005830 comp rx=0 tx=0).stop 2024-04-25T10:11:57.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.673+0000 7f0662a00640 1 -- 172.21.15.119:0/3249577883 shutdown_connections 2024-04-25T10:11:57.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.673+0000 7f0662a00640 1 --2- 172.21.15.119:0/3249577883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06400051c0 0x7f064000b9c0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.673+0000 7f0662a00640 1 -- 172.21.15.119:0/3249577883 >> 172.21.15.119:0/3249577883 conn(0x7f065c0cc9e0 msgr2=0x7f065c0ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.673+0000 7f0662a00640 1 -- 172.21.15.119:0/3249577883 shutdown_connections 2024-04-25T10:11:57.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.673+0000 7f0662a00640 1 -- 172.21.15.119:0/3249577883 wait complete. 2024-04-25T10:11:57.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.675+0000 7f0662a00640 1 Processor -- start 2024-04-25T10:11:57.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.675+0000 7f0662a00640 1 -- start start 2024-04-25T10:11:57.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.676+0000 7f0662a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06400051c0 0x7f065c1f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.676+0000 7f0662a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f065c0c9320 0x7f065c0c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.676+0000 7f0662a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f065c0c72d0 0x7f065c0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.676+0000 7f0662a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c1457c0 con 0x7f06400051c0 2024-04-25T10:11:57.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.676+0000 7f0662a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c145640 con 0x7f065c0c9320 2024-04-25T10:11:57.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.676+0000 7f0662a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0ca990 con 0x7f065c0c72d0 2024-04-25T10:11:57.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.676+0000 7f065be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06400051c0 0x7f065c1f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.676+0000 7f065be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06400051c0 0x7f065c1f34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35100/0 (socket says 172.21.15.119:35100) 2024-04-25T10:11:57.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.676+0000 7f065be00640 1 -- 172.21.15.119:0/1559106998 learned_addr learned my addr 172.21.15.119:0/1559106998 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:57.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.677+0000 7f065b400640 1 --2- 172.21.15.119:0/1559106998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f065c0c9320 0x7f065c0c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.677+0000 7f065be00640 1 -- 172.21.15.119:0/1559106998 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f065c0c72d0 msgr2=0x7f065c0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.677+0000 7f065be00640 1 --2- 172.21.15.119:0/1559106998 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f065c0c72d0 0x7f065c0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.677+0000 7f065be00640 1 -- 172.21.15.119:0/1559106998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f065c0c9320 msgr2=0x7f065c0c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.677+0000 7f065be00640 1 --2- 172.21.15.119:0/1559106998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f065c0c9320 0x7f065c0c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.677+0000 7f065be00640 1 -- 172.21.15.119:0/1559106998 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c0c8000 con 0x7f06400051c0 2024-04-25T10:11:57.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.677+0000 7f065b400640 1 --2- 172.21.15.119:0/1559106998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f065c0c9320 0x7f065c0c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:57.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.677+0000 7f065be00640 1 --2- 172.21.15.119:0/1559106998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06400051c0 0x7f065c1f34c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f064c00f380 tx=0x7f064c00f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.677+0000 7f0653e00640 1 -- 172.21.15.119:0/1559106998 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c01c070 con 0x7f06400051c0 2024-04-25T10:11:57.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.677+0000 7f0653e00640 1 -- 172.21.15.119:0/1559106998 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c017620 con 0x7f06400051c0 2024-04-25T10:11:57.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.677+0000 7f0653e00640 1 -- 172.21.15.119:0/1559106998 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c017a10 con 0x7f06400051c0 2024-04-25T10:11:57.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.677+0000 7f0662a00640 1 -- 172.21.15.119:0/1559106998 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065c0c82e0 con 0x7f06400051c0 2024-04-25T10:11:57.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.677+0000 7f0662a00640 1 -- 172.21.15.119:0/1559106998 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065c0c88a0 con 0x7f06400051c0 2024-04-25T10:11:57.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.680+0000 7f0653e00640 1 -- 172.21.15.119:0/1559106998 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f064c01f070 con 0x7f06400051c0 2024-04-25T10:11:57.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.681+0000 7f0653e00640 1 --2- 172.21.15.119:0/1559106998 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f063c06b500 0x7f063c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.681+0000 7f0653e00640 1 -- 172.21.15.119:0/1559106998 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f064c08a470 con 0x7f06400051c0 2024-04-25T10:11:57.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.681+0000 7f065b400640 1 --2- 172.21.15.119:0/1559106998 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f063c06b500 0x7f063c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.681+0000 7f06659bf740 1 Processor -- start 2024-04-25T10:11:57.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.682+0000 7f06659bf740 1 -- start start 2024-04-25T10:11:57.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.682+0000 7f065b400640 1 --2- 172.21.15.119:0/1559106998 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f063c06b500 0x7f063c06d9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0648001eb0 tx=0x7f0648000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.682+0000 7f06659bf740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67a9e360 0x558a67aa18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.682+0000 7f06659bf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67aa1e30 0x558a67a9e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.682+0000 7f06659bf740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67a9ec80 0x558a67abdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.682+0000 7f06659bf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a679daee0 con 0x558a67aa1e30 2024-04-25T10:11:57.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.682+0000 7f06659bf740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a67983cf0 con 0x558a67a9ec80 2024-04-25T10:11:57.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.682+0000 7f06659bf740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a679d6f80 con 0x558a67a9e360 2024-04-25T10:11:57.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.683+0000 7f065b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67aa1e30 0x558a67a9e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.683+0000 7f065b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67aa1e30 0x558a67a9e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35112/0 (socket says 172.21.15.119:35112) 2024-04-25T10:11:57.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.683+0000 7f065b400640 1 -- 172.21.15.119:0/1266784502 learned_addr learned my addr 172.21.15.119:0/1266784502 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:57.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.683+0000 7f0660a00640 1 --2- 172.21.15.119:0/1266784502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67a9ec80 0x558a67abdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.683+0000 7f065be00640 1 --2- 172.21.15.119:0/1266784502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67a9e360 0x558a67aa18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.683+0000 7f065b400640 1 -- 172.21.15.119:0/1266784502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67a9e360 msgr2=0x558a67aa18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.683+0000 7f065b400640 1 --2- 172.21.15.119:0/1266784502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67a9e360 0x558a67aa18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.683+0000 7f065b400640 1 -- 172.21.15.119:0/1266784502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67a9ec80 msgr2=0x558a67abdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.683+0000 7f065b400640 1 --2- 172.21.15.119:0/1266784502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67a9ec80 0x558a67abdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.683+0000 7f065b400640 1 -- 172.21.15.119:0/1266784502 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a677bba20 con 0x558a67aa1e30 2024-04-25T10:11:57.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.684+0000 7f065b400640 1 --2- 172.21.15.119:0/1266784502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67aa1e30 0x558a67a9e740 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f064800fae0 tx=0x7f0648012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab0356522431f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.684+0000 7f0647e00640 1 -- 172.21.15.119:0/1266784502 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0648010040 con 0x558a67aa1e30 2024-04-25T10:11:57.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.684+0000 7f0647e00640 1 -- 172.21.15.119:0/1266784502 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06480169d0 con 0x558a67aa1e30 2024-04-25T10:11:57.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.684+0000 7f0647e00640 1 -- 172.21.15.119:0/1266784502 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0648016d80 con 0x558a67aa1e30 2024-04-25T10:11:57.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.684+0000 7f06659bf740 1 -- 172.21.15.119:0/1266784502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67aa1e30 msgr2=0x558a67a9e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.684+0000 7f06659bf740 1 --2- 172.21.15.119:0/1266784502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67aa1e30 0x558a67a9e740 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f064800fae0 tx=0x7f0648012000 comp rx=0 tx=0).stop 2024-04-25T10:11:57.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.685+0000 7f06659bf740 1 -- 172.21.15.119:0/1266784502 shutdown_connections 2024-04-25T10:11:57.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.685+0000 7f06659bf740 1 --2- 172.21.15.119:0/1266784502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67a9ec80 0x558a67abdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.685+0000 7f06659bf740 1 --2- 172.21.15.119:0/1266784502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67aa1e30 0x558a67a9e740 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.685+0000 7f06659bf740 1 --2- 172.21.15.119:0/1266784502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67a9e360 0x558a67aa18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.685+0000 7f06659bf740 1 -- 172.21.15.119:0/1266784502 >> 172.21.15.119:0/1266784502 conn(0x558a67a74150 msgr2=0x558a67a75150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.685+0000 7f06659bf740 1 -- 172.21.15.119:0/1266784502 shutdown_connections 2024-04-25T10:11:57.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.685+0000 7f06659bf740 1 -- 172.21.15.119:0/1266784502 wait complete. 2024-04-25T10:11:57.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.685+0000 7f06659bf740 1 -- start start 2024-04-25T10:11:57.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.686+0000 7f06659bf740 1 Processor -- start 2024-04-25T10:11:57.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f06659bf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67a9ec80 0x558a67a77640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f06659bf740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67aa1e30 0x558a67a76830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f06659bf740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67a76d70 0x558a67abdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f06659bf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a67a72b90 con 0x558a67a9ec80 2024-04-25T10:11:57.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f06659bf740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a67979240 con 0x558a67a76d70 2024-04-25T10:11:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f06659bf740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a67a71f80 con 0x558a67aa1e30 2024-04-25T10:11:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f06659bf740 10 client.0 _ll_register_callbacks cb 0x558a676eee00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f065be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67a9ec80 0x558a67a77640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f065be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67a9ec80 0x558a67a77640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35128/0 (socket says 172.21.15.119:35128) 2024-04-25T10:11:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f065be00640 1 -- 172.21.15.119:0/3212952206 learned_addr learned my addr 172.21.15.119:0/3212952206 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f065be00640 1 -- 172.21.15.119:0/3212952206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67aa1e30 msgr2=0x558a67a76830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f065be00640 1 --2- 172.21.15.119:0/3212952206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67aa1e30 0x558a67a76830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f065be00640 1 -- 172.21.15.119:0/3212952206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67a76d70 msgr2=0x558a67abdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f065be00640 1 --2- 172.21.15.119:0/3212952206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67a76d70 0x558a67abdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.687+0000 7f065be00640 1 -- 172.21.15.119:0/3212952206 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a676af310 con 0x558a67a9ec80 2024-04-25T10:11:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.688+0000 7f065be00640 1 --2- 172.21.15.119:0/3212952206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67a9ec80 0x558a67a77640 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f064c058870 tx=0x7f064c05cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d896445280696600 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.688+0000 7f0644c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:11:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.688+0000 7f0644c00640 1 -- 172.21.15.119:0/3212952206 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c056e70 con 0x558a67a9ec80 2024-04-25T10:11:57.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.688+0000 7f0644c00640 1 -- 172.21.15.119:0/3212952206 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f064c08a0c0 con 0x558a67a9ec80 2024-04-25T10:11:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.688+0000 7f06659bf740 1 -- 172.21.15.119:0/3212952206 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06480169d0 con 0x558a67a9ec80 2024-04-25T10:11:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.688+0000 7f0644c00640 1 -- 172.21.15.119:0/3212952206 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f064c058dd0 con 0x558a67a9ec80 2024-04-25T10:11:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.688+0000 7f06659bf740 10 client.6804 fetch_fsmap learned FSMap version 50 2024-04-25T10:11:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.688+0000 7f06659bf740 1 -- 172.21.15.119:0/3212952206 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x558a6781c1b0 con 0x558a67a9ec80 2024-04-25T10:11:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.688+0000 7f0644c00640 1 -- 172.21.15.119:0/3212952206 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06480169d0 con 0x558a67a9ec80 2024-04-25T10:11:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.688+0000 7f0644c00640 1 -- 172.21.15.119:0/3212952206 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f064c060d40 con 0x558a67a9ec80 2024-04-25T10:11:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.689+0000 7f06659bf740 10 client.6804 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:11:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.689+0000 7f06659bf740 10 client.6804 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:11:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.689+0000 7f06659bf740 20 client.6804 populate_metadata read hostname 'smithi119' 2024-04-25T10:11:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.689+0000 7f06659bf740 1 --2- 172.21.15.119:0/3212952206 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x558a67ab7380 0x558a67ab77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.689+0000 7f06659bf740 4 client.6804 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.689+0000 7f06659bf740 1 -- 172.21.15.119:0/3212952206 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a67a390c0 con 0x558a67ab7380 2024-04-25T10:11:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.689+0000 7f065b400640 1 --2- 172.21.15.119:0/3212952206 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x558a67ab7380 0x558a67ab77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.690+0000 7f065b400640 1 --2- 172.21.15.119:0/3212952206 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x558a67ab7380 0x558a67ab77a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f70f3855b43e1a6b server_cookie=74fedfc33e2f7944 in_seq=0 out_seq=0 2024-04-25T10:11:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.690+0000 7f0644c00640 10 client.6804 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:11:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.690+0000 7f0644c00640 1 -- 172.21.15.119:0/3212952206 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558a67a390c0 con 0x558a67ab7380 2024-04-25T10:11:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.690+0000 7f0644c00640 10 client.6804 handle_command_reply: tid=0 2024-04-25T10:11:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.690+0000 7f06659bf740 1 client.6804 shutdown 2024-04-25T10:11:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.691+0000 7f06659bf740 1 -- 172.21.15.119:0/3212952206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67a9ec80 msgr2=0x558a67a77640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.691+0000 7f06659bf740 1 --2- 172.21.15.119:0/3212952206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67a9ec80 0x558a67a77640 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f064c058870 tx=0x7f064c05cd50 comp rx=0 tx=0).stop 2024-04-25T10:11:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.691+0000 7f06659bf740 1 -- 172.21.15.119:0/3212952206 shutdown_connections 2024-04-25T10:11:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.691+0000 7f06659bf740 1 --2- 172.21.15.119:0/3212952206 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x558a67ab7380 0x558a67ab77a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.691+0000 7f06659bf740 1 --2- 172.21.15.119:0/3212952206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67a76d70 0x558a67abdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.691+0000 7f06659bf740 1 --2- 172.21.15.119:0/3212952206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67aa1e30 0x558a67a76830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.691+0000 7f06659bf740 1 --2- 172.21.15.119:0/3212952206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67a9ec80 0x558a67a77640 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.691+0000 7f0644c00640 0 client.6804 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:11:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.691+0000 7f06659bf740 1 -- 172.21.15.119:0/3212952206 >> 172.21.15.119:0/3212952206 conn(0x558a67a9e360 msgr2=0x558a67a71720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.691+0000 7f06659bf740 1 -- 172.21.15.119:0/3212952206 shutdown_connections 2024-04-25T10:11:57.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.691+0000 7f06659bf740 1 -- 172.21.15.119:0/3212952206 wait complete. 2024-04-25T10:11:57.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.691+0000 7f06659bf740 20 client.6804 trim_cache size 0 max 16384 2024-04-25T10:11:57.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.708+0000 7f06659bf740 1 Processor -- start 2024-04-25T10:11:57.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.708+0000 7f06659bf740 1 -- start start 2024-04-25T10:11:57.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.709+0000 7f06659bf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67a76d70 0x558a67ab68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.709+0000 7f06659bf740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67a9ec80 0x558a67a78830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.709+0000 7f06659bf740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67aa1e30 0x558a67aa51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.709+0000 7f06659bf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c200400 con 0x558a67a76d70 2024-04-25T10:11:57.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.709+0000 7f06659bf740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c1ff5e0 con 0x558a67aa1e30 2024-04-25T10:11:57.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.709+0000 7f06659bf740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c06f2d0 con 0x558a67a9ec80 2024-04-25T10:11:57.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.709+0000 7f065be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67a76d70 0x558a67ab68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.709+0000 7f065be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67a76d70 0x558a67ab68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35132/0 (socket says 172.21.15.119:35132) 2024-04-25T10:11:57.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.709+0000 7f065be00640 1 -- 172.21.15.119:0/2745001295 learned_addr learned my addr 172.21.15.119:0/2745001295 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:57.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.709+0000 7f065be00640 1 -- 172.21.15.119:0/2745001295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67a9ec80 msgr2=0x558a67a78830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.709+0000 7f065be00640 1 --2- 172.21.15.119:0/2745001295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67a9ec80 0x558a67a78830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.709+0000 7f065be00640 1 -- 172.21.15.119:0/2745001295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67aa1e30 msgr2=0x558a67aa51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.709+0000 7f065be00640 1 --2- 172.21.15.119:0/2745001295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67aa1e30 0x558a67aa51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.709+0000 7f065be00640 1 -- 172.21.15.119:0/2745001295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a67abe0e0 con 0x558a67a76d70 2024-04-25T10:11:57.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.710+0000 7f065be00640 1 --2- 172.21.15.119:0/2745001295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67a76d70 0x558a67ab68b0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f064c054fd0 tx=0x7f064c08a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7115f866eeacd10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.710+0000 7f0647400640 1 -- 172.21.15.119:0/2745001295 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c060d40 con 0x558a67a76d70 2024-04-25T10:11:57.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.710+0000 7f0647400640 1 -- 172.21.15.119:0/2745001295 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c08a0c0 con 0x558a67a76d70 2024-04-25T10:11:57.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.710+0000 7f0647400640 1 -- 172.21.15.119:0/2745001295 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c08a270 con 0x558a67a76d70 2024-04-25T10:11:57.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.710+0000 7f06659bf740 1 -- 172.21.15.119:0/2745001295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67a76d70 msgr2=0x558a67ab68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.710+0000 7f06659bf740 1 --2- 172.21.15.119:0/2745001295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67a76d70 0x558a67ab68b0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f064c054fd0 tx=0x7f064c08a050 comp rx=0 tx=0).stop 2024-04-25T10:11:57.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.711+0000 7f06659bf740 1 -- 172.21.15.119:0/2745001295 shutdown_connections 2024-04-25T10:11:57.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.711+0000 7f06659bf740 1 --2- 172.21.15.119:0/2745001295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67aa1e30 0x558a67aa51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.711+0000 7f06659bf740 1 --2- 172.21.15.119:0/2745001295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67a9ec80 0x558a67a78830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.711+0000 7f06659bf740 1 --2- 172.21.15.119:0/2745001295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67a76d70 0x558a67ab68b0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.711+0000 7f06659bf740 1 -- 172.21.15.119:0/2745001295 >> 172.21.15.119:0/2745001295 conn(0x558a67a9e360 msgr2=0x558a67ab9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.711+0000 7f06659bf740 1 -- 172.21.15.119:0/2745001295 shutdown_connections 2024-04-25T10:11:57.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.711+0000 7f06659bf740 1 -- 172.21.15.119:0/2745001295 wait complete. 2024-04-25T10:11:57.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.711+0000 7f06659bf740 1 -- start start 2024-04-25T10:11:57.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.712+0000 7f06659bf740 1 Processor -- start 2024-04-25T10:11:57.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.712+0000 7f06659bf740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67a76d70 0x558a67aba8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.712+0000 7f06659bf740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67a9ec80 0x558a67b27580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f06659bf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67aa1e30 0x558a67a9d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f06659bf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a67aa34f0 con 0x558a67aa1e30 2024-04-25T10:11:57.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f06659bf740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c06f8d0 con 0x558a67a76d70 2024-04-25T10:11:57.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f06659bf740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a67abb100 con 0x558a67a9ec80 2024-04-25T10:11:57.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f06659bf740 10 client.0 _ll_register_callbacks cb 0x7f0648005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:57.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f0660a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67aa1e30 0x558a67a9d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f0660a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67aa1e30 0x558a67a9d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35142/0 (socket says 172.21.15.119:35142) 2024-04-25T10:11:57.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f0660a00640 1 -- 172.21.15.119:0/908488250 learned_addr learned my addr 172.21.15.119:0/908488250 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:57.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f065b400640 1 --2- 172.21.15.119:0/908488250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67a9ec80 0x558a67b27580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f065be00640 1 --2- 172.21.15.119:0/908488250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67a76d70 0x558a67aba8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f0660a00640 1 -- 172.21.15.119:0/908488250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67a9ec80 msgr2=0x558a67b27580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f0660a00640 1 --2- 172.21.15.119:0/908488250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67a9ec80 0x558a67b27580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f0660a00640 1 -- 172.21.15.119:0/908488250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67a76d70 msgr2=0x558a67aba8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f0660a00640 1 --2- 172.21.15.119:0/908488250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67a76d70 0x558a67aba8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.713+0000 7f0660a00640 1 -- 172.21.15.119:0/908488250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a67688490 con 0x558a67aa1e30 2024-04-25T10:11:57.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.714+0000 7f0660a00640 1 --2- 172.21.15.119:0/908488250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67aa1e30 0x558a67a9d230 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f065402ae80 tx=0x7f065402aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3010a4486cd2e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.714+0000 7f0644c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:11:57.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.714+0000 7f0644c00640 1 -- 172.21.15.119:0/908488250 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654040020 con 0x558a67aa1e30 2024-04-25T10:11:57.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.714+0000 7f0644c00640 1 -- 172.21.15.119:0/908488250 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f06540069d0 con 0x558a67aa1e30 2024-04-25T10:11:57.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.714+0000 7f06659bf740 1 -- 172.21.15.119:0/908488250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f064c08a0c0 con 0x558a67aa1e30 2024-04-25T10:11:57.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.714+0000 7f0644c00640 1 -- 172.21.15.119:0/908488250 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0654037040 con 0x558a67aa1e30 2024-04-25T10:11:57.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.714+0000 7f06659bf740 10 client.6810 fetch_fsmap learned FSMap version 50 2024-04-25T10:11:57.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.714+0000 7f06659bf740 1 -- 172.21.15.119:0/908488250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x558a67abae40 con 0x558a67aa1e30 2024-04-25T10:11:57.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.714+0000 7f0644c00640 1 -- 172.21.15.119:0/908488250 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c08a0c0 con 0x558a67aa1e30 2024-04-25T10:11:57.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.715+0000 7f0644c00640 1 -- 172.21.15.119:0/908488250 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0654006bd0 con 0x558a67aa1e30 2024-04-25T10:11:57.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.715+0000 7f06659bf740 10 client.6810 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:11:57.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.715+0000 7f06659bf740 10 client.6810 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:11:57.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.715+0000 7f06659bf740 20 client.6810 populate_metadata read hostname 'smithi119' 2024-04-25T10:11:57.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.715+0000 7f06659bf740 1 --2- 172.21.15.119:0/908488250 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x558a67a99920 0x558a67a9d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.715+0000 7f06659bf740 4 client.6810 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-25T10:11:57.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.715+0000 7f06659bf740 1 -- 172.21.15.119:0/908488250 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) v1 -- 0x558a679e8420 con 0x558a67a99920 2024-04-25T10:11:57.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.715+0000 7f065be00640 1 --2- 172.21.15.119:0/908488250 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x558a67a99920 0x558a67a9d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.715+0000 7f065be00640 1 --2- 172.21.15.119:0/908488250 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x558a67a99920 0x558a67a9d770 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d0ee690064ab515 server_cookie=b955c1067492eaad in_seq=0 out_seq=0 2024-04-25T10:11:57.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.715+0000 7f0644c00640 10 client.6810 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:11:57.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.716+0000 7f0644c00640 1 -- 172.21.15.119:0/908488250 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558a679e8420 con 0x558a67a99920 2024-04-25T10:11:57.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.716+0000 7f0644c00640 10 client.6810 handle_command_reply: tid=0 2024-04-25T10:11:57.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.716+0000 7f06659bf740 1 client.6810 shutdown 2024-04-25T10:11:57.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.716+0000 7f06659bf740 1 -- 172.21.15.119:0/908488250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67aa1e30 msgr2=0x558a67a9d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.716+0000 7f06659bf740 1 --2- 172.21.15.119:0/908488250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67aa1e30 0x558a67a9d230 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f065402ae80 tx=0x7f065402aeb0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.716+0000 7f06659bf740 1 -- 172.21.15.119:0/908488250 shutdown_connections 2024-04-25T10:11:57.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.716+0000 7f06659bf740 1 --2- 172.21.15.119:0/908488250 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x558a67a99920 0x558a67a9d770 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.716+0000 7f06659bf740 1 --2- 172.21.15.119:0/908488250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558a67aa1e30 0x558a67a9d230 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.716+0000 7f06659bf740 1 --2- 172.21.15.119:0/908488250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558a67a9ec80 0x558a67b27580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.716+0000 7f06659bf740 1 --2- 172.21.15.119:0/908488250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a67a76d70 0x558a67aba8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.716+0000 7f06659bf740 1 -- 172.21.15.119:0/908488250 >> 172.21.15.119:0/908488250 conn(0x558a67a9e360 msgr2=0x558a67aa1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.716+0000 7f0644c00640 0 client.6810 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:11:57.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.716+0000 7f06659bf740 1 -- 172.21.15.119:0/908488250 shutdown_connections 2024-04-25T10:11:57.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.716+0000 7f06659bf740 1 -- 172.21.15.119:0/908488250 wait complete. 2024-04-25T10:11:57.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.717+0000 7f06659bf740 20 client.6810 trim_cache size 0 max 16384 2024-04-25T10:11:57.730 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:11:57.730 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2024-04-25T10:11:57.730 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "b16c4690-d490-48fe-a0e2-d71376744360", 2024-04-25T10:11:57.730 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2024-04-25T10:11:57.731 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:11:57.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.717+0000 7f0662a00640 1 -- 172.21.15.119:0/1559106998 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f063c06b500 msgr2=0x7f063c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.717+0000 7f0662a00640 1 --2- 172.21.15.119:0/1559106998 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f063c06b500 0x7f063c06d9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0648001eb0 tx=0x7f0648000ed0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.717+0000 7f0662a00640 1 -- 172.21.15.119:0/1559106998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06400051c0 msgr2=0x7f065c1f34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.717+0000 7f0662a00640 1 --2- 172.21.15.119:0/1559106998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06400051c0 0x7f065c1f34c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f064c00f380 tx=0x7f064c00f840 comp rx=0 tx=0).stop 2024-04-25T10:11:57.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.717+0000 7f0662a00640 1 -- 172.21.15.119:0/1559106998 shutdown_connections 2024-04-25T10:11:57.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.717+0000 7f0662a00640 1 --2- 172.21.15.119:0/1559106998 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f063c06b500 0x7f063c06d9c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.717+0000 7f0662a00640 1 --2- 172.21.15.119:0/1559106998 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f065c0c72d0 0x7f065c0c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.717+0000 7f0662a00640 1 --2- 172.21.15.119:0/1559106998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f065c0c9320 0x7f065c0c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.717+0000 7f0662a00640 1 --2- 172.21.15.119:0/1559106998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06400051c0 0x7f065c1f34c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.717+0000 7f0662a00640 1 -- 172.21.15.119:0/1559106998 >> 172.21.15.119:0/1559106998 conn(0x7f065c0cc9e0 msgr2=0x7f065c0ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.718+0000 7f0662a00640 1 -- 172.21.15.119:0/1559106998 shutdown_connections 2024-04-25T10:11:57.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.718+0000 7f0662a00640 1 -- 172.21.15.119:0/1559106998 wait complete. 2024-04-25T10:11:57.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:57.739 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:11:57.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.928+0000 7f3e29000640 1 Processor -- start 2024-04-25T10:11:57.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.929+0000 7f3e29000640 1 -- start start 2024-04-25T10:11:57.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.930+0000 7f3e29000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3e24144e70 0x7f3e2406d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.930+0000 7f3e29000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3e2406d7a0 0x7f3e240ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.930+0000 7f3e29000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e240cef10 0x7f3e24071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.930+0000 7f3e29000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e2415bfe0 con 0x7f3e2406d7a0 2024-04-25T10:11:57.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.930+0000 7f3e29000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e2406dcb0 con 0x7f3e24144e70 2024-04-25T10:11:57.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.930+0000 7f3e29000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2406df90 con 0x7f3e240cef10 2024-04-25T10:11:57.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.930+0000 7f3e19400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3e2406d7a0 0x7f3e240ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.930+0000 7f3e19400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3e2406d7a0 0x7f3e240ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35154/0 (socket says 172.21.15.119:35154) 2024-04-25T10:11:57.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.930+0000 7f3e19400640 1 -- 172.21.15.119:0/2281488410 learned_addr learned my addr 172.21.15.119:0/2281488410 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:57.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.931+0000 7f3e21e00640 1 --2- 172.21.15.119:0/2281488410 >> v2:172.21.15.132:3300/0 conn(0x7f3e24144e70 0x7f3e2406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.931+0000 7f3e19400640 1 -- 172.21.15.119:0/2281488410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e240cef10 msgr2=0x7f3e24071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.931+0000 7f3e19400640 1 --2- 172.21.15.119:0/2281488410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e240cef10 0x7f3e24071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.931+0000 7f3e19400640 1 -- 172.21.15.119:0/2281488410 >> v2:172.21.15.132:3300/0 conn(0x7f3e24144e70 msgr2=0x7f3e2406d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.931+0000 7f3e19400640 1 --2- 172.21.15.119:0/2281488410 >> v2:172.21.15.132:3300/0 conn(0x7f3e24144e70 0x7f3e2406d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.931+0000 7f3e19400640 1 -- 172.21.15.119:0/2281488410 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2406e270 con 0x7f3e2406d7a0 2024-04-25T10:11:57.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.932+0000 7f3e19400640 1 --2- 172.21.15.119:0/2281488410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e2406d7a0 0x7f3e240ce9d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3e240b4a20 tx=0x7f3e0c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef5d48cbed4ffc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.932+0000 7f3e21400640 1 -- 172.21.15.119:0/2281488410 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c037070 con 0x7f3e2406d7a0 2024-04-25T10:11:57.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.932+0000 7f3e21400640 1 -- 172.21.15.119:0/2281488410 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c032440 con 0x7f3e2406d7a0 2024-04-25T10:11:57.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.932+0000 7f3e21400640 1 -- 172.21.15.119:0/2281488410 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c032810 con 0x7f3e2406d7a0 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.933+0000 7f3e29000640 1 -- 172.21.15.119:0/2281488410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e2406d7a0 msgr2=0x7f3e240ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.933+0000 7f3e29000640 1 --2- 172.21.15.119:0/2281488410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e2406d7a0 0x7f3e240ce9d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3e240b4a20 tx=0x7f3e0c02a140 comp rx=0 tx=0).stop 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.933+0000 7f3e29000640 1 -- 172.21.15.119:0/2281488410 shutdown_connections 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.933+0000 7f3e29000640 1 --2- 172.21.15.119:0/2281488410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e240cef10 0x7f3e24071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.933+0000 7f3e29000640 1 --2- 172.21.15.119:0/2281488410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e2406d7a0 0x7f3e240ce9d0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.933+0000 7f3e29000640 1 --2- 172.21.15.119:0/2281488410 >> v2:172.21.15.132:3300/0 conn(0x7f3e24144e70 0x7f3e2406d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.933+0000 7f3e29000640 1 -- 172.21.15.119:0/2281488410 >> 172.21.15.119:0/2281488410 conn(0x7f3e2406aba0 msgr2=0x7f3e2406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.933+0000 7f3e29000640 1 -- 172.21.15.119:0/2281488410 shutdown_connections 2024-04-25T10:11:57.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.933+0000 7f3e29000640 1 -- 172.21.15.119:0/2281488410 wait complete. 2024-04-25T10:11:57.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.935+0000 7f3e29000640 1 Processor -- start 2024-04-25T10:11:57.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.935+0000 7f3e29000640 1 -- start start 2024-04-25T10:11:57.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.936+0000 7f3e29000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e2406d7a0 0x7f3e2413f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.936+0000 7f3e29000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e240cef10 0x7f3e240b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.936+0000 7f3e29000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e24144e70 0x7f3e240b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.936+0000 7f3e29000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e241454b0 con 0x7f3e2406d7a0 2024-04-25T10:11:57.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.936+0000 7f3e29000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240cf370 con 0x7f3e24144e70 2024-04-25T10:11:57.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.936+0000 7f3e29000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24145630 con 0x7f3e240cef10 2024-04-25T10:11:57.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.936+0000 7f3e21e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e2406d7a0 0x7f3e2413f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.936+0000 7f3e21e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e2406d7a0 0x7f3e2413f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35168/0 (socket says 172.21.15.119:35168) 2024-04-25T10:11:57.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.936+0000 7f3e21e00640 1 -- 172.21.15.119:0/3936438056 learned_addr learned my addr 172.21.15.119:0/3936438056 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:57.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.936+0000 7f3e21e00640 1 -- 172.21.15.119:0/3936438056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e240cef10 msgr2=0x7f3e240b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.936+0000 7f3e21e00640 1 --2- 172.21.15.119:0/3936438056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e240cef10 0x7f3e240b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.936+0000 7f3e21e00640 1 -- 172.21.15.119:0/3936438056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e24144e70 msgr2=0x7f3e240b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.936+0000 7f3e21e00640 1 --2- 172.21.15.119:0/3936438056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e24144e70 0x7f3e240b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.936+0000 7f3e21e00640 1 -- 172.21.15.119:0/3936438056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2413dc10 con 0x7f3e2406d7a0 2024-04-25T10:11:57.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.937+0000 7f3e21e00640 1 --2- 172.21.15.119:0/3936438056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e2406d7a0 0x7f3e2413f2d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400c7f0 tx=0x7f3e1400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.937+0000 7f3e1aa00640 1 -- 172.21.15.119:0/3936438056 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14015070 con 0x7f3e2406d7a0 2024-04-25T10:11:57.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.937+0000 7f3e1aa00640 1 -- 172.21.15.119:0/3936438056 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e14004220 con 0x7f3e2406d7a0 2024-04-25T10:11:57.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.937+0000 7f3e1aa00640 1 -- 172.21.15.119:0/3936438056 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14007620 con 0x7f3e2406d7a0 2024-04-25T10:11:57.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.937+0000 7f3e29000640 1 -- 172.21.15.119:0/3936438056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2413def0 con 0x7f3e2406d7a0 2024-04-25T10:11:57.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.937+0000 7f3e29000640 1 -- 172.21.15.119:0/3936438056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2413e4b0 con 0x7f3e2406d7a0 2024-04-25T10:11:57.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.939+0000 7f3e1aa00640 1 -- 172.21.15.119:0/3936438056 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e140079d0 con 0x7f3e2406d7a0 2024-04-25T10:11:57.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.941+0000 7f3e1aa00640 1 --2- 172.21.15.119:0/3936438056 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e0006b4d0 0x7f3e0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.941+0000 7f3e1aa00640 1 -- 172.21.15.119:0/3936438056 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f3e1408a3b0 con 0x7f3e2406d7a0 2024-04-25T10:11:57.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.941+0000 7f3e19400640 1 --2- 172.21.15.119:0/3936438056 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e0006b4d0 0x7f3e0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.941+0000 7f3e2bd92740 1 Processor -- start 2024-04-25T10:11:57.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.942+0000 7f3e2bd92740 1 -- start start 2024-04-25T10:11:57.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.942+0000 7f3e19400640 1 --2- 172.21.15.119:0/3936438056 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e0006b4d0 0x7f3e0006d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c034000 tx=0x7f3e0c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.942+0000 7f3e2bd92740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558877282bb0 0x558877284d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.942+0000 7f3e2bd92740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5588772bf280 0x558877282f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.942+0000 7f3e2bd92740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588772ad670 0x5588772adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.943+0000 7f3e2bd92740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588771e6c70 con 0x5588772ad670 2024-04-25T10:11:57.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.943+0000 7f3e2bd92740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887718fcf0 con 0x558877282bb0 2024-04-25T10:11:57.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.943+0000 7f3e2bd92740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588771e2d10 con 0x5588772bf280 2024-04-25T10:11:57.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.943+0000 7f3e22800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588772ad670 0x5588772adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.943+0000 7f3e22800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588772ad670 0x5588772adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35184/0 (socket says 172.21.15.119:35184) 2024-04-25T10:11:57.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.943+0000 7f3e22800640 1 -- 172.21.15.119:0/955753056 learned_addr learned my addr 172.21.15.119:0/955753056 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:57.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.943+0000 7f3e19400640 1 --2- 172.21.15.119:0/955753056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5588772bf280 0x558877282f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.943+0000 7f3e21e00640 1 --2- 172.21.15.119:0/955753056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558877282bb0 0x558877284d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.943+0000 7f3e22800640 1 -- 172.21.15.119:0/955753056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5588772bf280 msgr2=0x558877282f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.943+0000 7f3e22800640 1 --2- 172.21.15.119:0/955753056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5588772bf280 0x558877282f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.943+0000 7f3e22800640 1 -- 172.21.15.119:0/955753056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558877282bb0 msgr2=0x558877284d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.943+0000 7f3e22800640 1 --2- 172.21.15.119:0/955753056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558877282bb0 0x558877284d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.943+0000 7f3e22800640 1 -- 172.21.15.119:0/955753056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558876fc57d0 con 0x5588772ad670 2024-04-25T10:11:57.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.943+0000 7f3e19400640 1 --2- 172.21.15.119:0/955753056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5588772bf280 0x558877282f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:57.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.944+0000 7f3e22800640 1 --2- 172.21.15.119:0/955753056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588772ad670 0x5588772adaf0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009d30 tx=0x7f3e1c00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ab82e8db92d559 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.944+0000 7f3e12a00640 1 -- 172.21.15.119:0/955753056 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c00bce0 con 0x5588772ad670 2024-04-25T10:11:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.944+0000 7f3e12a00640 1 -- 172.21.15.119:0/955753056 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c017070 con 0x5588772ad670 2024-04-25T10:11:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.944+0000 7f3e12a00640 1 -- 172.21.15.119:0/955753056 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c012670 con 0x5588772ad670 2024-04-25T10:11:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.945+0000 7f3e2bd92740 1 -- 172.21.15.119:0/955753056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588772ad670 msgr2=0x5588772adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.945+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/955753056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588772ad670 0x5588772adaf0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009d30 tx=0x7f3e1c00b420 comp rx=0 tx=0).stop 2024-04-25T10:11:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.945+0000 7f3e2bd92740 1 -- 172.21.15.119:0/955753056 shutdown_connections 2024-04-25T10:11:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.945+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/955753056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588772ad670 0x5588772adaf0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.945+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/955753056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5588772bf280 0x558877282f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.945+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/955753056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558877282bb0 0x558877284d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.945+0000 7f3e2bd92740 1 -- 172.21.15.119:0/955753056 >> 172.21.15.119:0/955753056 conn(0x55887724aa30 msgr2=0x5588772c8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.945+0000 7f3e2bd92740 1 -- 172.21.15.119:0/955753056 shutdown_connections 2024-04-25T10:11:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.945+0000 7f3e2bd92740 1 -- 172.21.15.119:0/955753056 wait complete. 2024-04-25T10:11:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.946+0000 7f3e2bd92740 1 -- start start 2024-04-25T10:11:57.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.946+0000 7f3e2bd92740 1 Processor -- start 2024-04-25T10:11:57.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.947+0000 7f3e2bd92740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558877282bb0 0x5588772aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.947+0000 7f3e2bd92740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5588772ad670 0x5588772c3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.947+0000 7f3e2bd92740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588772bf280 0x5588772b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.947+0000 7f3e2bd92740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588771853a0 con 0x558877282bb0 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.947+0000 7f3e2bd92740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588772ab1c0 con 0x5588772bf280 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.947+0000 7f3e2bd92740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588772d9040 con 0x5588772ad670 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.947+0000 7f3e2bd92740 10 client.0 _ll_register_callbacks cb 0x5588771a2b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.947+0000 7f3e21e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558877282bb0 0x5588772aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.947+0000 7f3e21e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558877282bb0 0x5588772aac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35200/0 (socket says 172.21.15.119:35200) 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.947+0000 7f3e21e00640 1 -- 172.21.15.119:0/929685736 learned_addr learned my addr 172.21.15.119:0/929685736 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.947+0000 7f3e21e00640 1 -- 172.21.15.119:0/929685736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5588772ad670 msgr2=0x5588772c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.947+0000 7f3e21e00640 1 --2- 172.21.15.119:0/929685736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5588772ad670 0x5588772c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.947+0000 7f3e21e00640 1 -- 172.21.15.119:0/929685736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588772bf280 msgr2=0x5588772b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.947+0000 7f3e21e00640 1 --2- 172.21.15.119:0/929685736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588772bf280 0x5588772b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.948+0000 7f3e21e00640 1 -- 172.21.15.119:0/929685736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558876efb3f0 con 0x558877282bb0 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.948+0000 7f3e21e00640 1 --2- 172.21.15.119:0/929685736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558877282bb0 0x5588772aac80 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3e14058700 tx=0x7f3e14059240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6571a53e7c0020 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.948+0000 7f3e07400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:11:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.948+0000 7f3e07400640 1 -- 172.21.15.119:0/929685736 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1401e040 con 0x558877282bb0 2024-04-25T10:11:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.948+0000 7f3e07400640 1 -- 172.21.15.119:0/929685736 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f3e14007cc0 con 0x558877282bb0 2024-04-25T10:11:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.948+0000 7f3e2bd92740 1 -- 172.21.15.119:0/929685736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e1c017070 con 0x558877282bb0 2024-04-25T10:11:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.948+0000 7f3e07400640 1 -- 172.21.15.119:0/929685736 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e14057e50 con 0x558877282bb0 2024-04-25T10:11:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.948+0000 7f3e2bd92740 10 client.6822 fetch_fsmap learned FSMap version 50 2024-04-25T10:11:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.948+0000 7f3e2bd92740 1 -- 172.21.15.119:0/929685736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x558876ebb0f0 con 0x558877282bb0 2024-04-25T10:11:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.948+0000 7f3e07400640 1 -- 172.21.15.119:0/929685736 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c017070 con 0x558877282bb0 2024-04-25T10:11:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.949+0000 7f3e07400640 1 -- 172.21.15.119:0/929685736 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3e14057ff0 con 0x558877282bb0 2024-04-25T10:11:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.949+0000 7f3e2bd92740 10 client.6822 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:11:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.949+0000 7f3e2bd92740 10 client.6822 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:11:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.949+0000 7f3e2bd92740 20 client.6822 populate_metadata read hostname 'smithi119' 2024-04-25T10:11:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.949+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/929685736 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5588772d8900 0x5588772cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.949+0000 7f3e2bd92740 4 client.6822 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.949+0000 7f3e2bd92740 1 -- 172.21.15.119:0/929685736 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588771f41b0 con 0x5588772d8900 2024-04-25T10:11:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.949+0000 7f3e19400640 1 --2- 172.21.15.119:0/929685736 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5588772d8900 0x5588772cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.950+0000 7f3e19400640 1 --2- 172.21.15.119:0/929685736 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5588772d8900 0x5588772cbdf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32ae453dd3cff89 server_cookie=31ef738d775985ec in_seq=0 out_seq=0 2024-04-25T10:11:57.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.950+0000 7f3e07400640 10 client.6822 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:11:57.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.950+0000 7f3e07400640 1 -- 172.21.15.119:0/929685736 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588771f41b0 con 0x5588772d8900 2024-04-25T10:11:57.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.950+0000 7f3e07400640 10 client.6822 handle_command_reply: tid=0 2024-04-25T10:11:57.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.950+0000 7f3e2bd92740 1 client.6822 shutdown 2024-04-25T10:11:57.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.950+0000 7f3e2bd92740 1 -- 172.21.15.119:0/929685736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558877282bb0 msgr2=0x5588772aac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.950+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/929685736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558877282bb0 0x5588772aac80 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3e14058700 tx=0x7f3e14059240 comp rx=0 tx=0).stop 2024-04-25T10:11:57.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.951+0000 7f3e2bd92740 1 -- 172.21.15.119:0/929685736 shutdown_connections 2024-04-25T10:11:57.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.951+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/929685736 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5588772d8900 0x5588772cbdf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.951+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/929685736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588772bf280 0x5588772b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.951+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/929685736 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5588772ad670 0x5588772c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.951+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/929685736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558877282bb0 0x5588772aac80 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.951+0000 7f3e2bd92740 1 -- 172.21.15.119:0/929685736 >> 172.21.15.119:0/929685736 conn(0x55887724aa30 msgr2=0x558877281450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.951+0000 7f3e07400640 0 client.6822 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:11:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.951+0000 7f3e2bd92740 1 -- 172.21.15.119:0/929685736 shutdown_connections 2024-04-25T10:11:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.951+0000 7f3e2bd92740 1 -- 172.21.15.119:0/929685736 wait complete. 2024-04-25T10:11:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.951+0000 7f3e2bd92740 20 client.6822 trim_cache size 0 max 16384 2024-04-25T10:11:57.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.967+0000 7f3e2bd92740 1 Processor -- start 2024-04-25T10:11:57.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.968+0000 7f3e2bd92740 1 -- start start 2024-04-25T10:11:57.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.968+0000 7f3e2bd92740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558877282bb0 0x5588772aa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.968+0000 7f3e2bd92740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588772ad670 0x5588772c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.968+0000 7f3e2bd92740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588772bf280 0x5588772b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.968+0000 7f3e2bd92740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24200210 con 0x5588772bf280 2024-04-25T10:11:57.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.968+0000 7f3e2bd92740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e241ff520 con 0x5588772ad670 2024-04-25T10:11:57.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.968+0000 7f3e2bd92740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e241f7f90 con 0x558877282bb0 2024-04-25T10:11:57.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.969+0000 7f3e22800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588772bf280 0x5588772b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.969+0000 7f3e22800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588772bf280 0x5588772b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35204/0 (socket says 172.21.15.119:35204) 2024-04-25T10:11:57.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.969+0000 7f3e22800640 1 -- 172.21.15.119:0/2770165220 learned_addr learned my addr 172.21.15.119:0/2770165220 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:57.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.969+0000 7f3e22800640 1 -- 172.21.15.119:0/2770165220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558877282bb0 msgr2=0x5588772aa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.969+0000 7f3e22800640 1 --2- 172.21.15.119:0/2770165220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558877282bb0 0x5588772aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.969+0000 7f3e22800640 1 -- 172.21.15.119:0/2770165220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588772ad670 msgr2=0x5588772c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.969+0000 7f3e22800640 1 --2- 172.21.15.119:0/2770165220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588772ad670 0x5588772c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.969+0000 7f3e22800640 1 -- 172.21.15.119:0/2770165220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558877280f10 con 0x5588772bf280 2024-04-25T10:11:57.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.970+0000 7f3e22800640 1 --2- 172.21.15.119:0/2770165220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588772bf280 0x5588772b0f40 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009d00 tx=0x7f3e1c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33dada50fafb46e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.970+0000 7f3e12000640 1 -- 172.21.15.119:0/2770165220 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c012630 con 0x5588772bf280 2024-04-25T10:11:57.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.970+0000 7f3e12000640 1 -- 172.21.15.119:0/2770165220 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c017040 con 0x5588772bf280 2024-04-25T10:11:57.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.970+0000 7f3e12000640 1 -- 172.21.15.119:0/2770165220 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c012a30 con 0x5588772bf280 2024-04-25T10:11:57.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.970+0000 7f3e2bd92740 1 -- 172.21.15.119:0/2770165220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588772bf280 msgr2=0x5588772b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.970+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/2770165220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588772bf280 0x5588772b0f40 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009d00 tx=0x7f3e1c00e000 comp rx=0 tx=0).stop 2024-04-25T10:11:57.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.971+0000 7f3e2bd92740 1 -- 172.21.15.119:0/2770165220 shutdown_connections 2024-04-25T10:11:57.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.971+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/2770165220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588772bf280 0x5588772b0f40 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.971+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/2770165220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588772ad670 0x5588772c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.971+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/2770165220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558877282bb0 0x5588772aa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.971+0000 7f3e2bd92740 1 -- 172.21.15.119:0/2770165220 >> 172.21.15.119:0/2770165220 conn(0x55887724aa30 msgr2=0x5588772aac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.971+0000 7f3e2bd92740 1 -- 172.21.15.119:0/2770165220 shutdown_connections 2024-04-25T10:11:57.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.971+0000 7f3e2bd92740 1 -- 172.21.15.119:0/2770165220 wait complete. 2024-04-25T10:11:57.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.971+0000 7f3e2bd92740 1 -- start start 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.972+0000 7f3e2bd92740 1 Processor -- start 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.972+0000 7f3e2bd92740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558877282bb0 0x558877332560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.972+0000 7f3e2bd92740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5588772ad670 0x5588772d6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e2bd92740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588772bf280 0x5588772d7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e2bd92740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240b3bc0 con 0x558877282bb0 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e2bd92740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558877332eb0 con 0x5588772bf280 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e2bd92740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55887727d170 con 0x5588772ad670 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e2bd92740 10 client.0 _ll_register_callbacks cb 0x7f3e1400a3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e21e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558877282bb0 0x558877332560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e21e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558877282bb0 0x558877332560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35214/0 (socket says 172.21.15.119:35214) 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e21e00640 1 -- 172.21.15.119:0/1796095782 learned_addr learned my addr 172.21.15.119:0/1796095782 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e21e00640 1 -- 172.21.15.119:0/1796095782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5588772ad670 msgr2=0x5588772d6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e21e00640 1 --2- 172.21.15.119:0/1796095782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5588772ad670 0x5588772d6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e21e00640 1 -- 172.21.15.119:0/1796095782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588772bf280 msgr2=0x5588772d7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:57.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e21e00640 1 --2- 172.21.15.119:0/1796095782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588772bf280 0x5588772d7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e21e00640 1 -- 172.21.15.119:0/1796095782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558877336cd0 con 0x558877282bb0 2024-04-25T10:11:57.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e21e00640 1 --2- 172.21.15.119:0/1796095782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558877282bb0 0x558877332560 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3e14060be0 tx=0x7f3e140045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b26cbf3339d62d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:57.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.973+0000 7f3e07400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:11:57.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e07400640 1 -- 172.21.15.119:0/1796095782 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1401e040 con 0x558877282bb0 2024-04-25T10:11:57.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e07400640 1 -- 172.21.15.119:0/1796095782 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f3e14077380 con 0x558877282bb0 2024-04-25T10:11:57.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e2bd92740 1 -- 172.21.15.119:0/1796095782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e1c017040 con 0x558877282bb0 2024-04-25T10:11:57.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e07400640 1 -- 172.21.15.119:0/1796095782 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e14058360 con 0x558877282bb0 2024-04-25T10:11:57.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e2bd92740 10 client.6828 fetch_fsmap learned FSMap version 50 2024-04-25T10:11:57.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e2bd92740 1 -- 172.21.15.119:0/1796095782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x558877332cb0 con 0x558877282bb0 2024-04-25T10:11:57.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e07400640 1 -- 172.21.15.119:0/1796095782 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c017040 con 0x558877282bb0 2024-04-25T10:11:57.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e07400640 1 -- 172.21.15.119:0/1796095782 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3e14054750 con 0x558877282bb0 2024-04-25T10:11:57.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e2bd92740 10 client.6828 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e2bd92740 10 client.6828 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e2bd92740 20 client.6828 populate_metadata read hostname 'smithi119' 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/1796095782 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5588772d7980 0x5588772a7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e2bd92740 4 client.6828 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e2bd92740 1 -- 172.21.15.119:0/1796095782 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5588772c4cf0 con 0x5588772d7980 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.974+0000 7f3e19400640 1 --2- 172.21.15.119:0/1796095782 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5588772d7980 0x5588772a7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.975+0000 7f3e19400640 1 --2- 172.21.15.119:0/1796095782 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5588772d7980 0x5588772a7550 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b34b2089457d83a server_cookie=2635703a63e0c941 in_seq=0 out_seq=0 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.975+0000 7f3e07400640 10 client.6828 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.975+0000 7f3e07400640 1 -- 172.21.15.119:0/1796095782 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5588772c4cf0 con 0x5588772d7980 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.975+0000 7f3e07400640 10 client.6828 handle_command_reply: tid=0 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.975+0000 7f3e2bd92740 1 client.6828 shutdown 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.975+0000 7f3e2bd92740 1 -- 172.21.15.119:0/1796095782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558877282bb0 msgr2=0x558877332560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.975+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/1796095782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558877282bb0 0x558877332560 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3e14060be0 tx=0x7f3e140045f0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.975+0000 7f3e2bd92740 1 -- 172.21.15.119:0/1796095782 shutdown_connections 2024-04-25T10:11:57.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.975+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/1796095782 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5588772d7980 0x5588772a7550 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.975+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/1796095782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5588772bf280 0x5588772d7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.975+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/1796095782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5588772ad670 0x5588772d6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.975+0000 7f3e07400640 0 client.6828 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:11:57.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.975+0000 7f3e2bd92740 1 --2- 172.21.15.119:0/1796095782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558877282bb0 0x558877332560 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.975+0000 7f3e2bd92740 1 -- 172.21.15.119:0/1796095782 >> 172.21.15.119:0/1796095782 conn(0x55887724aa30 msgr2=0x5588772aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.976+0000 7f3e2bd92740 1 -- 172.21.15.119:0/1796095782 shutdown_connections 2024-04-25T10:11:57.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.976+0000 7f3e2bd92740 1 -- 172.21.15.119:0/1796095782 wait complete. 2024-04-25T10:11:57.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.976+0000 7f3e2bd92740 20 client.6828 trim_cache size 0 max 16384 2024-04-25T10:11:57.985 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:11:57.985 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:11:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2024-04-25T10:11:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: "b16c4690-d490-48fe-a0e2-d71376744360": { 2024-04-25T10:11:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/subdir", 2024-04-25T10:11:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "b16c4690-d490-48fe-a0e2-d71376744360", 2024-04-25T10:11:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive" 2024-04-25T10:11:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:57.986 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:11:57.986 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:11:57.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.976+0000 7f3e29000640 1 -- 172.21.15.119:0/3936438056 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e0006b4d0 msgr2=0x7f3e0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.976+0000 7f3e29000640 1 --2- 172.21.15.119:0/3936438056 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e0006b4d0 0x7f3e0006d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c034000 tx=0x7f3e0c02c000 comp rx=0 tx=0).stop 2024-04-25T10:11:57.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.976+0000 7f3e29000640 1 -- 172.21.15.119:0/3936438056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e2406d7a0 msgr2=0x7f3e2413f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:57.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.976+0000 7f3e29000640 1 --2- 172.21.15.119:0/3936438056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e2406d7a0 0x7f3e2413f2d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400c7f0 tx=0x7f3e1400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.977+0000 7f3e29000640 1 -- 172.21.15.119:0/3936438056 shutdown_connections 2024-04-25T10:11:57.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.977+0000 7f3e29000640 1 --2- 172.21.15.119:0/3936438056 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3e0006b4d0 0x7f3e0006d990 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.977+0000 7f3e29000640 1 --2- 172.21.15.119:0/3936438056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e24144e70 0x7f3e240b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.977+0000 7f3e29000640 1 --2- 172.21.15.119:0/3936438056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e240cef10 0x7f3e240b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.977+0000 7f3e29000640 1 --2- 172.21.15.119:0/3936438056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e2406d7a0 0x7f3e2413f2d0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:57.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.977+0000 7f3e29000640 1 -- 172.21.15.119:0/3936438056 >> 172.21.15.119:0/3936438056 conn(0x7f3e2406aba0 msgr2=0x7f3e240ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:57.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.977+0000 7f3e29000640 1 -- 172.21.15.119:0/3936438056 shutdown_connections 2024-04-25T10:11:57.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:11:57.977+0000 7f3e29000640 1 -- 172.21.15.119:0/3936438056 wait complete. 2024-04-25T10:11:57.999 INFO:tasks.cephfs.filesystem:scrub status for tag:b16c4690-d490-48fe-a0e2-d71376744360 - {'path': '/subdir', 'tag': 'b16c4690-d490-48fe-a0e2-d71376744360', 'options': 'recursive'} 2024-04-25T10:11:58.460 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:11:58.455+0000 7f8c67800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:12:01.285 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:01.294 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:28.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:28.002 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:12:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.208+0000 7f2ee3200640 1 Processor -- start 2024-04-25T10:12:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.209+0000 7f2ee3200640 1 -- start start 2024-04-25T10:12:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.210+0000 7f2ee3200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2ee40cea40 0x7f2ee406e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.210+0000 7f2ee3200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee40712c0 con 0x7f2ee40c6150 2024-04-25T10:12:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.210+0000 7f2ee3200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee40715a0 con 0x7f2ee415bcb0 2024-04-25T10:12:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.210+0000 7f2ee3200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee406e790 con 0x7f2ee40cea40 2024-04-25T10:12:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.210+0000 7f2ee1e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2ee40c6150 0x7f2ee40ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51158/0 (socket says 172.21.15.119:51158) 2024-04-25T10:12:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.210+0000 7f2ee1e00640 1 -- 172.21.15.119:0/934018064 learned_addr learned my addr 172.21.15.119:0/934018064 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:12:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee2800640 1 --2- 172.21.15.119:0/934018064 >> v2:172.21.15.132:3300/0 conn(0x7f2ee40cea40 0x7f2ee406e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 -- 172.21.15.119:0/934018064 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190082479 0 0) 0x7f2ee40712c0 con 0x7f2ee40c6150 2024-04-25T10:12:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 -- 172.21.15.119:0/934018064 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ec8003520 con 0x7f2ee40c6150 2024-04-25T10:12:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 -- 172.21.15.119:0/934018064 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2365919486 0 0) 0x7f2ec8003520 con 0x7f2ee40c6150 2024-04-25T10:12:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 -- 172.21.15.119:0/934018064 >> v2:172.21.15.132:3300/0 conn(0x7f2ee40cea40 msgr2=0x7f2ee406e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 --2- 172.21.15.119:0/934018064 >> v2:172.21.15.132:3300/0 conn(0x7f2ee40cea40 0x7f2ee406e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 -- 172.21.15.119:0/934018064 >> v1:172.21.15.132:6789/0 conn(0x7f2ee415bcb0 legacy=0x7f2ee406da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 -- 172.21.15.119:0/934018064 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee4071880 con 0x7f2ee40c6150 2024-04-25T10:12:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 -- 172.21.15.119:0/934018064 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f2ed4001690 con 0x7f2ee40c6150 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 -- 172.21.15.119:0/934018064 >> v1:172.21.15.119:6789/0 conn(0x7f2ee40c6150 legacy=0x7f2ee40ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 --2- 172.21.15.119:0/934018064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ec80039b0 0x7f2ec8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 --2- 172.21.15.119:0/934018064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec8004800 0x7f2ec8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 --2- 172.21.15.119:0/934018064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ec80051c0 0x7f2ec800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 -- 172.21.15.119:0/934018064 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec800c030 con 0x7f2ec80051c0 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 -- 172.21.15.119:0/934018064 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec800c1e0 con 0x7f2ec8004800 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.211+0000 7f2ee1400640 1 -- 172.21.15.119:0/934018064 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec800c3e0 con 0x7f2ec80039b0 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.212+0000 7f2ee2800640 1 --2- 172.21.15.119:0/934018064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ec80051c0 0x7f2ec800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.212+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/934018064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ec80039b0 0x7f2ec8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.212+0000 7f2ee2800640 1 -- 172.21.15.119:0/934018064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ec80039b0 msgr2=0x7f2ec8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.212+0000 7f2ee2800640 1 --2- 172.21.15.119:0/934018064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ec80039b0 0x7f2ec8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.212+0000 7f2ee2800640 1 -- 172.21.15.119:0/934018064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec8004800 msgr2=0x7f2ec8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.212+0000 7f2ed9400640 1 --2- 172.21.15.119:0/934018064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec8004800 0x7f2ec8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.212+0000 7f2ee2800640 1 --2- 172.21.15.119:0/934018064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec8004800 0x7f2ec8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.212+0000 7f2ee2800640 1 -- 172.21.15.119:0/934018064 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec8003520 con 0x7f2ec80051c0 2024-04-25T10:12:28.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.212+0000 7f2ed9400640 1 --2- 172.21.15.119:0/934018064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec8004800 0x7f2ec8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:28.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.213+0000 7f2ee2800640 1 -- 172.21.15.119:0/934018064 reap_dead start 2024-04-25T10:12:28.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.213+0000 7f2ee2800640 1 --2- 172.21.15.119:0/934018064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ec80051c0 0x7f2ec800b9c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009fc0 tx=0x7f2ec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ede7f3b4791cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.213+0000 7f2ee1400640 1 -- 172.21.15.119:0/934018064 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc035020 con 0x7f2ec80051c0 2024-04-25T10:12:28.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.213+0000 7f2ee1400640 1 -- 172.21.15.119:0/934018064 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc003bd0 con 0x7f2ec80051c0 2024-04-25T10:12:28.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.213+0000 7f2ee1400640 1 -- 172.21.15.119:0/934018064 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc009c10 con 0x7f2ec80051c0 2024-04-25T10:12:28.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.214+0000 7f2ee3200640 1 -- 172.21.15.119:0/934018064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ec80051c0 msgr2=0x7f2ec800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.214+0000 7f2ee3200640 1 --2- 172.21.15.119:0/934018064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ec80051c0 0x7f2ec800b9c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009fc0 tx=0x7f2ec8005830 comp rx=0 tx=0).stop 2024-04-25T10:12:28.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.214+0000 7f2ee3200640 1 -- 172.21.15.119:0/934018064 shutdown_connections 2024-04-25T10:12:28.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.214+0000 7f2ee3200640 1 --2- 172.21.15.119:0/934018064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ec80051c0 0x7f2ec800b9c0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.214+0000 7f2ee3200640 1 -- 172.21.15.119:0/934018064 >> 172.21.15.119:0/934018064 conn(0x7f2ee406aba0 msgr2=0x7f2ee406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.214+0000 7f2ee3200640 1 -- 172.21.15.119:0/934018064 shutdown_connections 2024-04-25T10:12:28.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.214+0000 7f2ee3200640 1 -- 172.21.15.119:0/934018064 wait complete. 2024-04-25T10:12:28.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.216+0000 7f2ee3200640 1 Processor -- start 2024-04-25T10:12:28.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.216+0000 7f2ee3200640 1 -- start start 2024-04-25T10:12:28.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.217+0000 7f2ee3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec80051c0 0x7f2ee413f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.217+0000 7f2ee3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ee4142050 0x7f2ee41424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.217+0000 7f2ee3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ee4140070 0x7f2ee41404f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.217+0000 7f2ee3200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40ce330 con 0x7f2ee4142050 2024-04-25T10:12:28.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.217+0000 7f2ee3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40c6530 con 0x7f2ec80051c0 2024-04-25T10:12:28.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.217+0000 7f2ee3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40ce4b0 con 0x7f2ee4140070 2024-04-25T10:12:28.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.217+0000 7f2ed9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ee4142050 0x7f2ee41424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.217+0000 7f2ed9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ee4142050 0x7f2ee41424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51136/0 (socket says 172.21.15.119:51136) 2024-04-25T10:12:28.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.217+0000 7f2ed9400640 1 -- 172.21.15.119:0/692929180 learned_addr learned my addr 172.21.15.119:0/692929180 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:28.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.218+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/692929180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec80051c0 0x7f2ee413f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.218+0000 7f2ee2800640 1 --2- 172.21.15.119:0/692929180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ee4140070 0x7f2ee41404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.218+0000 7f2ed9400640 1 -- 172.21.15.119:0/692929180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ee4140070 msgr2=0x7f2ee41404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.218+0000 7f2ed9400640 1 --2- 172.21.15.119:0/692929180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ee4140070 0x7f2ee41404f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.218+0000 7f2ed9400640 1 -- 172.21.15.119:0/692929180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec80051c0 msgr2=0x7f2ee413f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.218+0000 7f2ed9400640 1 --2- 172.21.15.119:0/692929180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec80051c0 0x7f2ee413f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.218+0000 7f2ed9400640 1 -- 172.21.15.119:0/692929180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee4140e10 con 0x7f2ee4142050 2024-04-25T10:12:28.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.218+0000 7f2ed9400640 1 --2- 172.21.15.119:0/692929180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ee4142050 0x7f2ee41424b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00d560 tx=0x7f2ecc00da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.218+0000 7f2edaa00640 1 -- 172.21.15.119:0/692929180 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc005480 con 0x7f2ee4142050 2024-04-25T10:12:28.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.218+0000 7f2edaa00640 1 -- 172.21.15.119:0/692929180 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc017070 con 0x7f2ee4142050 2024-04-25T10:12:28.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.218+0000 7f2ee3200640 1 -- 172.21.15.119:0/692929180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee41410f0 con 0x7f2ee4142050 2024-04-25T10:12:28.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.218+0000 7f2edaa00640 1 -- 172.21.15.119:0/692929180 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc003810 con 0x7f2ee4142050 2024-04-25T10:12:28.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.218+0000 7f2ee3200640 1 -- 172.21.15.119:0/692929180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee41416b0 con 0x7f2ee4142050 2024-04-25T10:12:28.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.221+0000 7f2edaa00640 1 -- 172.21.15.119:0/692929180 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ecc0039b0 con 0x7f2ee4142050 2024-04-25T10:12:28.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.222+0000 7f2edaa00640 1 --2- 172.21.15.119:0/692929180 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ec006b4d0 0x7f2ec006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.222+0000 7f2edaa00640 1 -- 172.21.15.119:0/692929180 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2ecc08a100 con 0x7f2ee4142050 2024-04-25T10:12:28.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.222+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/692929180 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ec006b4d0 0x7f2ec006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.223+0000 7f2eebdba740 1 Processor -- start 2024-04-25T10:12:28.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.223+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/692929180 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ec006b4d0 0x7f2ec006d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4001800 tx=0x7f2ed4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.223+0000 7f2eebdba740 1 -- start start 2024-04-25T10:12:28.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.224+0000 7f2eebdba740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b95152d70 0x563b95154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.224+0000 7f2eebdba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9518f440 0x563b95153150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.224+0000 7f2eebdba740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b9517d830 0x563b9517dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.224+0000 7f2eebdba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b950b6c70 con 0x563b9518f440 2024-04-25T10:12:28.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.224+0000 7f2eebdba740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9505fcf0 con 0x563b95152d70 2024-04-25T10:12:28.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.224+0000 7f2eebdba740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b950b2d10 con 0x563b9517d830 2024-04-25T10:12:28.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.224+0000 7f2ed9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9518f440 0x563b95153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.224+0000 7f2ed9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9518f440 0x563b95153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51142/0 (socket says 172.21.15.119:51142) 2024-04-25T10:12:28.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.224+0000 7f2ed9400640 1 -- 172.21.15.119:0/2021865237 learned_addr learned my addr 172.21.15.119:0/2021865237 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:28.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.224+0000 7f2ed9400640 1 -- 172.21.15.119:0/2021865237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b9517d830 msgr2=0x563b9517dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.224+0000 7f2ed9400640 1 --2- 172.21.15.119:0/2021865237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b9517d830 0x563b9517dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.224+0000 7f2ed9400640 1 -- 172.21.15.119:0/2021865237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b95152d70 msgr2=0x563b95154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.224+0000 7f2ed9400640 1 --2- 172.21.15.119:0/2021865237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b95152d70 0x563b95154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.224+0000 7f2ed9400640 1 -- 172.21.15.119:0/2021865237 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b94e957d0 con 0x563b9518f440 2024-04-25T10:12:28.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.225+0000 7f2ed9400640 1 --2- 172.21.15.119:0/2021865237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9518f440 0x563b95153150 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc060930 tx=0x7f2ecc058e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618acaee37f7dd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.225+0000 7f2ed2a00640 1 -- 172.21.15.119:0/2021865237 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc081e60 con 0x563b9518f440 2024-04-25T10:12:28.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.225+0000 7f2ed2a00640 1 -- 172.21.15.119:0/2021865237 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc060e60 con 0x563b9518f440 2024-04-25T10:12:28.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.225+0000 7f2ed2a00640 1 -- 172.21.15.119:0/2021865237 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc088610 con 0x563b9518f440 2024-04-25T10:12:28.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.226+0000 7f2eebdba740 1 -- 172.21.15.119:0/2021865237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9518f440 msgr2=0x563b95153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.226+0000 7f2eebdba740 1 --2- 172.21.15.119:0/2021865237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9518f440 0x563b95153150 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc060930 tx=0x7f2ecc058e50 comp rx=0 tx=0).stop 2024-04-25T10:12:28.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.226+0000 7f2eebdba740 1 -- 172.21.15.119:0/2021865237 shutdown_connections 2024-04-25T10:12:28.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.226+0000 7f2eebdba740 1 --2- 172.21.15.119:0/2021865237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b9517d830 0x563b9517dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.226+0000 7f2eebdba740 1 --2- 172.21.15.119:0/2021865237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9518f440 0x563b95153150 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.226+0000 7f2eebdba740 1 --2- 172.21.15.119:0/2021865237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b95152d70 0x563b95154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.226+0000 7f2eebdba740 1 -- 172.21.15.119:0/2021865237 >> 172.21.15.119:0/2021865237 conn(0x563b9511aa30 msgr2=0x563b95198830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.226+0000 7f2eebdba740 1 -- 172.21.15.119:0/2021865237 shutdown_connections 2024-04-25T10:12:28.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.226+0000 7f2eebdba740 1 -- 172.21.15.119:0/2021865237 wait complete. 2024-04-25T10:12:28.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.227+0000 7f2eebdba740 1 -- start start 2024-04-25T10:12:28.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.227+0000 7f2eebdba740 1 Processor -- start 2024-04-25T10:12:28.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.228+0000 7f2eebdba740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b95152d70 0x563b95193d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.228+0000 7f2eebdba740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b9517d830 0x563b95198d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.228+0000 7f2eebdba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9518f440 0x563b951992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.228+0000 7f2eebdba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b950553a0 con 0x563b9518f440 2024-04-25T10:12:28.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.228+0000 7f2eebdba740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9517b380 con 0x563b95152d70 2024-04-25T10:12:28.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.228+0000 7f2eebdba740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b9517eee0 con 0x563b9517d830 2024-04-25T10:12:28.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.228+0000 7f2eebdba740 10 client.0 _ll_register_callbacks cb 0x563b95072b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:28.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.228+0000 7f2ee2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9518f440 0x563b951992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.228+0000 7f2ee2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9518f440 0x563b951992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51154/0 (socket says 172.21.15.119:51154) 2024-04-25T10:12:28.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.228+0000 7f2ee2800640 1 -- 172.21.15.119:0/1082852286 learned_addr learned my addr 172.21.15.119:0/1082852286 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:28.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2ee2800640 1 -- 172.21.15.119:0/1082852286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b9517d830 msgr2=0x563b95198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/1082852286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b95152d70 0x563b95193d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2ed9400640 1 --2- 172.21.15.119:0/1082852286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b9517d830 0x563b95198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2ee2800640 1 --2- 172.21.15.119:0/1082852286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b9517d830 0x563b95198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2ee2800640 1 -- 172.21.15.119:0/1082852286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b95152d70 msgr2=0x563b95193d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2ee2800640 1 --2- 172.21.15.119:0/1082852286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b95152d70 0x563b95193d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2ee2800640 1 -- 172.21.15.119:0/1082852286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b94dcb3f0 con 0x563b9518f440 2024-04-25T10:12:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2ee2800640 1 --2- 172.21.15.119:0/1082852286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9518f440 0x563b951992a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00e200 tx=0x7f2edc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426826d2fdfa7cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2ec7400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:12:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2ec7400640 1 -- 172.21.15.119:0/1082852286 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc040030 con 0x563b9518f440 2024-04-25T10:12:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2ec7400640 1 -- 172.21.15.119:0/1082852286 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2edc009c10 con 0x563b9518f440 2024-04-25T10:12:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2eebdba740 1 -- 172.21.15.119:0/1082852286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ecc060e60 con 0x563b9518f440 2024-04-25T10:12:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2ec7400640 1 -- 172.21.15.119:0/1082852286 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2edc04c050 con 0x563b9518f440 2024-04-25T10:12:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2eebdba740 10 client.6840 fetch_fsmap learned FSMap version 51 2024-04-25T10:12:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.229+0000 7f2eebdba740 1 -- 172.21.15.119:0/1082852286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x563b94d8b0f0 con 0x563b9518f440 2024-04-25T10:12:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.230+0000 7f2ec7400640 1 -- 172.21.15.119:0/1082852286 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc060e60 con 0x563b9518f440 2024-04-25T10:12:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.230+0000 7f2ec7400640 1 -- 172.21.15.119:0/1082852286 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2edc04f020 con 0x563b9518f440 2024-04-25T10:12:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.230+0000 7f2eebdba740 10 client.6840 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:12:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.230+0000 7f2eebdba740 10 client.6840 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:12:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.230+0000 7f2eebdba740 20 client.6840 populate_metadata read hostname 'smithi119' 2024-04-25T10:12:28.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.230+0000 7f2eebdba740 1 --2- 172.21.15.119:0/1082852286 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x563b9517dcb0 0x563b9517e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.230+0000 7f2eebdba740 4 client.6840 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:28.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.230+0000 7f2eebdba740 1 -- 172.21.15.119:0/1082852286 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b950c41b0 con 0x563b9517dcb0 2024-04-25T10:12:28.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.230+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/1082852286 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x563b9517dcb0 0x563b9517e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.231+0000 7f2ec7400640 10 client.6840 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:28.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.231+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/1082852286 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x563b9517dcb0 0x563b9517e0c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2050ba56a6e86ab server_cookie=81bfae2a178a26c6 in_seq=0 out_seq=0 2024-04-25T10:12:28.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.231+0000 7f2ec7400640 1 -- 172.21.15.119:0/1082852286 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b950c41b0 con 0x563b9517dcb0 2024-04-25T10:12:28.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.231+0000 7f2ec7400640 10 client.6840 handle_command_reply: tid=0 2024-04-25T10:12:28.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.231+0000 7f2eebdba740 1 client.6840 shutdown 2024-04-25T10:12:28.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.232+0000 7f2eebdba740 1 -- 172.21.15.119:0/1082852286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9518f440 msgr2=0x563b951992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.232+0000 7f2eebdba740 1 --2- 172.21.15.119:0/1082852286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9518f440 0x563b951992a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00e200 tx=0x7f2edc009750 comp rx=0 tx=0).stop 2024-04-25T10:12:28.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.232+0000 7f2eebdba740 1 -- 172.21.15.119:0/1082852286 shutdown_connections 2024-04-25T10:12:28.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.232+0000 7f2eebdba740 1 --2- 172.21.15.119:0/1082852286 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x563b9517dcb0 0x563b9517e0c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.232+0000 7f2eebdba740 1 --2- 172.21.15.119:0/1082852286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9518f440 0x563b951992a0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.232+0000 7f2eebdba740 1 --2- 172.21.15.119:0/1082852286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b9517d830 0x563b95198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.232+0000 7f2eebdba740 1 --2- 172.21.15.119:0/1082852286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b95152d70 0x563b95193d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.232+0000 7f2ec7400640 0 client.6840 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:28.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.232+0000 7f2eebdba740 1 -- 172.21.15.119:0/1082852286 >> 172.21.15.119:0/1082852286 conn(0x563b9511aa30 msgr2=0x563b9519b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.232+0000 7f2eebdba740 1 -- 172.21.15.119:0/1082852286 shutdown_connections 2024-04-25T10:12:28.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.232+0000 7f2eebdba740 1 -- 172.21.15.119:0/1082852286 wait complete. 2024-04-25T10:12:28.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.232+0000 7f2eebdba740 20 client.6840 trim_cache size 0 max 16384 2024-04-25T10:12:28.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.248+0000 7f2eebdba740 1 Processor -- start 2024-04-25T10:12:28.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.249+0000 7f2eebdba740 1 -- start start 2024-04-25T10:12:28.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.249+0000 7f2eebdba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b95152d70 0x563b951fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.249+0000 7f2eebdba740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b9517d830 0x563b9519bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.249+0000 7f2eebdba740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b9518f440 0x563b9519c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.249+0000 7f2eebdba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee4200530 con 0x563b95152d70 2024-04-25T10:12:28.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.249+0000 7f2eebdba740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee41ff6d0 con 0x563b9517d830 2024-04-25T10:12:28.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.249+0000 7f2eebdba740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee41e5b00 con 0x563b9518f440 2024-04-25T10:12:28.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.250+0000 7f2ee1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b95152d70 0x563b951fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.250+0000 7f2ee1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b95152d70 0x563b951fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51156/0 (socket says 172.21.15.119:51156) 2024-04-25T10:12:28.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.250+0000 7f2ee1e00640 1 -- 172.21.15.119:0/4024778977 learned_addr learned my addr 172.21.15.119:0/4024778977 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:28.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.250+0000 7f2ed9400640 1 --2- 172.21.15.119:0/4024778977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b9517d830 0x563b9519bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.250+0000 7f2ee2800640 1 --2- 172.21.15.119:0/4024778977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b9518f440 0x563b9519c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.250+0000 7f2ee1e00640 1 -- 172.21.15.119:0/4024778977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b9518f440 msgr2=0x563b9519c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.250+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/4024778977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b9518f440 0x563b9519c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.250+0000 7f2ee1e00640 1 -- 172.21.15.119:0/4024778977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b9517d830 msgr2=0x563b9519bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.250+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/4024778977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b9517d830 0x563b9519bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.250+0000 7f2ee1e00640 1 -- 172.21.15.119:0/4024778977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b95191f00 con 0x563b95152d70 2024-04-25T10:12:28.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.251+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/4024778977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b95152d70 0x563b951fd1c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4007300 tx=0x7f2ed4017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5c834a13efe277 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.251+0000 7f2ed2000640 1 -- 172.21.15.119:0/4024778977 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4005db0 con 0x563b95152d70 2024-04-25T10:12:28.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.251+0000 7f2ed2000640 1 -- 172.21.15.119:0/4024778977 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed4027020 con 0x563b95152d70 2024-04-25T10:12:28.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.251+0000 7f2ed2000640 1 -- 172.21.15.119:0/4024778977 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40225a0 con 0x563b95152d70 2024-04-25T10:12:28.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.251+0000 7f2eebdba740 1 -- 172.21.15.119:0/4024778977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b95152d70 msgr2=0x563b951fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.251+0000 7f2eebdba740 1 --2- 172.21.15.119:0/4024778977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b95152d70 0x563b951fd1c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4007300 tx=0x7f2ed4017040 comp rx=0 tx=0).stop 2024-04-25T10:12:28.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.252+0000 7f2eebdba740 1 -- 172.21.15.119:0/4024778977 shutdown_connections 2024-04-25T10:12:28.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.252+0000 7f2eebdba740 1 --2- 172.21.15.119:0/4024778977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b9518f440 0x563b9519c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.252+0000 7f2eebdba740 1 --2- 172.21.15.119:0/4024778977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b9517d830 0x563b9519bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.252+0000 7f2eebdba740 1 --2- 172.21.15.119:0/4024778977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b95152d70 0x563b951fd1c0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.252+0000 7f2eebdba740 1 -- 172.21.15.119:0/4024778977 >> 172.21.15.119:0/4024778977 conn(0x563b9511aa30 msgr2=0x563b9517e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.252+0000 7f2eebdba740 1 -- 172.21.15.119:0/4024778977 shutdown_connections 2024-04-25T10:12:28.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.252+0000 7f2eebdba740 1 -- 172.21.15.119:0/4024778977 wait complete. 2024-04-25T10:12:28.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.252+0000 7f2eebdba740 1 -- start start 2024-04-25T10:12:28.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.252+0000 7f2eebdba740 1 Processor -- start 2024-04-25T10:12:28.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.253+0000 7f2eebdba740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b95152d70 0x563b951e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.253+0000 7f2eebdba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9517d830 0x563b9517e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.253+0000 7f2eebdba740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b9518f440 0x563b9517e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.253+0000 7f2eebdba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b951fd700 con 0x563b9517d830 2024-04-25T10:12:28.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.253+0000 7f2eebdba740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee41ff0d0 con 0x563b9518f440 2024-04-25T10:12:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.253+0000 7f2eebdba740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b9516f1b0 con 0x563b95152d70 2024-04-25T10:12:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.254+0000 7f2eebdba740 10 client.0 _ll_register_callbacks cb 0x7f2ec8004700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.254+0000 7f2ed9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9517d830 0x563b9517e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.254+0000 7f2ed9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9517d830 0x563b9517e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51160/0 (socket says 172.21.15.119:51160) 2024-04-25T10:12:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.254+0000 7f2ed9400640 1 -- 172.21.15.119:0/838901550 learned_addr learned my addr 172.21.15.119:0/838901550 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.254+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/838901550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b95152d70 0x563b951e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.254+0000 7f2ee2800640 1 --2- 172.21.15.119:0/838901550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b9518f440 0x563b9517e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.254+0000 7f2ed9400640 1 -- 172.21.15.119:0/838901550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b95152d70 msgr2=0x563b951e93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.254+0000 7f2ed9400640 1 --2- 172.21.15.119:0/838901550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b95152d70 0x563b951e93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.254+0000 7f2ed9400640 1 -- 172.21.15.119:0/838901550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b9518f440 msgr2=0x563b9517e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.254+0000 7f2ed9400640 1 --2- 172.21.15.119:0/838901550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b9518f440 0x563b9517e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.254+0000 7f2ed9400640 1 -- 172.21.15.119:0/838901550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b951f99e0 con 0x563b9517d830 2024-04-25T10:12:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2ed9400640 1 --2- 172.21.15.119:0/838901550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9517d830 0x563b9517e090 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc058910 tx=0x7f2ecc001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c3a8103d8fc2c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2ec7400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2ec7400640 1 -- 172.21.15.119:0/838901550 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc076060 con 0x563b9517d830 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2ec7400640 1 -- 172.21.15.119:0/838901550 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2ecc003ca0 con 0x563b9517d830 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2eebdba740 1 -- 172.21.15.119:0/838901550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ed4027020 con 0x563b9517d830 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2ec7400640 1 -- 172.21.15.119:0/838901550 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ecc081e60 con 0x563b9517d830 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2eebdba740 10 client.6846 fetch_fsmap learned FSMap version 51 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2eebdba740 1 -- 172.21.15.119:0/838901550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x563b951fa050 con 0x563b9517d830 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2ec7400640 1 -- 172.21.15.119:0/838901550 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed4027020 con 0x563b9517d830 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2ec7400640 1 -- 172.21.15.119:0/838901550 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2ecc057ba0 con 0x563b9517d830 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2eebdba740 10 client.6846 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2eebdba740 10 client.6846 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2eebdba740 20 client.6846 populate_metadata read hostname 'smithi119' 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2eebdba740 1 --2- 172.21.15.119:0/838901550 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x563b951ab050 0x563b951b6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2eebdba740 4 client.6846 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.255+0000 7f2eebdba740 1 -- 172.21.15.119:0/838901550 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563b950be500 con 0x563b951ab050 2024-04-25T10:12:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.256+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/838901550 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x563b951ab050 0x563b951b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.256+0000 7f2ee1e00640 1 --2- 172.21.15.119:0/838901550 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x563b951ab050 0x563b951b6540 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abc0fdfee7c5a614 server_cookie=7911a871884e89bd in_seq=0 out_seq=0 2024-04-25T10:12:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.256+0000 7f2ec7400640 10 client.6846 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2ec7400640 1 -- 172.21.15.119:0/838901550 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563b950be500 con 0x563b951ab050 2024-04-25T10:12:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2ec7400640 10 client.6846 handle_command_reply: tid=0 2024-04-25T10:12:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2eebdba740 1 client.6846 shutdown 2024-04-25T10:12:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2eebdba740 1 -- 172.21.15.119:0/838901550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9517d830 msgr2=0x563b9517e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2eebdba740 1 --2- 172.21.15.119:0/838901550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9517d830 0x563b9517e090 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc058910 tx=0x7f2ecc001330 comp rx=0 tx=0).stop 2024-04-25T10:12:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2eebdba740 1 -- 172.21.15.119:0/838901550 shutdown_connections 2024-04-25T10:12:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2eebdba740 1 --2- 172.21.15.119:0/838901550 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x563b951ab050 0x563b951b6540 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2eebdba740 1 --2- 172.21.15.119:0/838901550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b9518f440 0x563b9517e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2eebdba740 1 --2- 172.21.15.119:0/838901550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b9517d830 0x563b9517e090 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2eebdba740 1 --2- 172.21.15.119:0/838901550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b95152d70 0x563b951e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2ec7400640 0 client.6846 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:28.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2eebdba740 1 -- 172.21.15.119:0/838901550 >> 172.21.15.119:0/838901550 conn(0x563b9511aa30 msgr2=0x563b951fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2eebdba740 1 -- 172.21.15.119:0/838901550 shutdown_connections 2024-04-25T10:12:28.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2eebdba740 1 -- 172.21.15.119:0/838901550 wait complete. 2024-04-25T10:12:28.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.257+0000 7f2eebdba740 20 client.6846 trim_cache size 0 max 16384 2024-04-25T10:12:28.266 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:12:28.266 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2024-04-25T10:12:28.266 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2024-04-25T10:12:28.266 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:12:28.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.258+0000 7f2ee3200640 1 -- 172.21.15.119:0/692929180 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ec006b4d0 msgr2=0x7f2ec006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.258+0000 7f2ee3200640 1 --2- 172.21.15.119:0/692929180 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ec006b4d0 0x7f2ec006d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4001800 tx=0x7f2ed4009040 comp rx=0 tx=0).stop 2024-04-25T10:12:28.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.258+0000 7f2ee3200640 1 -- 172.21.15.119:0/692929180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ee4142050 msgr2=0x7f2ee41424b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.258+0000 7f2ee3200640 1 --2- 172.21.15.119:0/692929180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ee4142050 0x7f2ee41424b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00d560 tx=0x7f2ecc00da20 comp rx=0 tx=0).stop 2024-04-25T10:12:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.258+0000 7f2ee3200640 1 -- 172.21.15.119:0/692929180 shutdown_connections 2024-04-25T10:12:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.258+0000 7f2ee3200640 1 --2- 172.21.15.119:0/692929180 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ec006b4d0 0x7f2ec006d990 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.258+0000 7f2ee3200640 1 --2- 172.21.15.119:0/692929180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ee4140070 0x7f2ee41404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.258+0000 7f2ee3200640 1 --2- 172.21.15.119:0/692929180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ee4142050 0x7f2ee41424b0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.258+0000 7f2ee3200640 1 --2- 172.21.15.119:0/692929180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ec80051c0 0x7f2ee413f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.258+0000 7f2ee3200640 1 -- 172.21.15.119:0/692929180 >> 172.21.15.119:0/692929180 conn(0x7f2ee406aba0 msgr2=0x7f2ee40b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.258+0000 7f2ee3200640 1 -- 172.21.15.119:0/692929180 shutdown_connections 2024-04-25T10:12:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.258+0000 7f2ee3200640 1 -- 172.21.15.119:0/692929180 wait complete. 2024-04-25T10:12:28.281 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:12:28.281 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:28.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.477+0000 7ff540e00640 1 Processor -- start 2024-04-25T10:12:28.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.477+0000 7ff540e00640 1 -- start start 2024-04-25T10:12:28.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.479+0000 7ff540e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff53c15dce0 0x7ff53c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.479+0000 7ff540e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff53c0b5b60 0x7ff53c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.479+0000 7ff540e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff53c15e0c0 con 0x7ff53c0d35b0 2024-04-25T10:12:28.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.479+0000 7ff540e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff53c0d2510 con 0x7ff53c0b5b60 2024-04-25T10:12:28.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.479+0000 7ff540e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c0d27f0 con 0x7ff53c15dce0 2024-04-25T10:12:28.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.479+0000 7ff53a800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff53c0b5b60 0x7ff53c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.479+0000 7ff539e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff53c15dce0 0x7ff53c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.479+0000 7ff53a800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff53c0b5b60 0x7ff53c0b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:38098/0 (socket says 172.21.15.119:38098) 2024-04-25T10:12:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.479+0000 7ff539e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff53c15dce0 0x7ff53c0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:50474/0 (socket says 172.21.15.119:50474) 2024-04-25T10:12:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.479+0000 7ff53a800640 1 -- 172.21.15.119:0/2364653540 learned_addr learned my addr 172.21.15.119:0/2364653540 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.480+0000 7ff53a800640 1 -- 172.21.15.119:0/2364653540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff53c15dce0 msgr2=0x7ff53c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.480+0000 7ff53a800640 1 --2- 172.21.15.119:0/2364653540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff53c15dce0 0x7ff53c0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.480+0000 7ff53a800640 1 -- 172.21.15.119:0/2364653540 >> v1:172.21.15.132:6789/0 conn(0x7ff53c0d35b0 legacy=0x7ff53c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.480+0000 7ff53a800640 1 -- 172.21.15.119:0/2364653540 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c0d2ad0 con 0x7ff53c0b5b60 2024-04-25T10:12:28.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.480+0000 7ff539e00640 1 --2- 172.21.15.119:0/2364653540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff53c15dce0 0x7ff53c0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:28.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.482+0000 7ff53a800640 1 --2- 172.21.15.119:0/2364653540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff53c0b5b60 0x7ff53c0b6b00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff534009870 tx=0x7ff53402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=710b157dd9356c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.482+0000 7ff538a00640 1 -- 172.21.15.119:0/2364653540 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534037070 con 0x7ff53c0b5b60 2024-04-25T10:12:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.482+0000 7ff538a00640 1 -- 172.21.15.119:0/2364653540 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff534032440 con 0x7ff53c0b5b60 2024-04-25T10:12:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.482+0000 7ff538a00640 1 -- 172.21.15.119:0/2364653540 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534032810 con 0x7ff53c0b5b60 2024-04-25T10:12:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.483+0000 7ff540e00640 1 -- 172.21.15.119:0/2364653540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff53c0b5b60 msgr2=0x7ff53c0b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.483+0000 7ff540e00640 1 --2- 172.21.15.119:0/2364653540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff53c0b5b60 0x7ff53c0b6b00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff534009870 tx=0x7ff53402a250 comp rx=0 tx=0).stop 2024-04-25T10:12:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.483+0000 7ff540e00640 1 -- 172.21.15.119:0/2364653540 shutdown_connections 2024-04-25T10:12:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.483+0000 7ff540e00640 1 --2- 172.21.15.119:0/2364653540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff53c0b5b60 0x7ff53c0b6b00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.483+0000 7ff540e00640 1 --2- 172.21.15.119:0/2364653540 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff53c15dce0 0x7ff53c0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.483+0000 7ff540e00640 1 -- 172.21.15.119:0/2364653540 >> 172.21.15.119:0/2364653540 conn(0x7ff53c072900 msgr2=0x7ff53c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.483+0000 7ff540e00640 1 -- 172.21.15.119:0/2364653540 shutdown_connections 2024-04-25T10:12:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.483+0000 7ff540e00640 1 -- 172.21.15.119:0/2364653540 wait complete. 2024-04-25T10:12:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.485+0000 7ff540e00640 1 Processor -- start 2024-04-25T10:12:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.485+0000 7ff540e00640 1 -- start start 2024-04-25T10:12:28.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff540e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff53c0b5b60 0x7ff53c15ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff540e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff53c0d35b0 0x7ff53c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff540e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff53c15dce0 0x7ff53c0c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff540e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c0b9820 con 0x7ff53c15dce0 2024-04-25T10:12:28.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff540e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c0d3090 con 0x7ff53c0b5b60 2024-04-25T10:12:28.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff540e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c0b99a0 con 0x7ff53c0d35b0 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff53a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff53c15dce0 0x7ff53c0c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff53a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff53c15dce0 0x7ff53c0c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51164/0 (socket says 172.21.15.119:51164) 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff53a800640 1 -- 172.21.15.119:0/939940630 learned_addr learned my addr 172.21.15.119:0/939940630 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff539e00640 1 --2- 172.21.15.119:0/939940630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff53c0b5b60 0x7ff53c15ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff539400640 1 --2- 172.21.15.119:0/939940630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff53c0d35b0 0x7ff53c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff53a800640 1 -- 172.21.15.119:0/939940630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff53c0d35b0 msgr2=0x7ff53c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff53a800640 1 --2- 172.21.15.119:0/939940630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff53c0d35b0 0x7ff53c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff53a800640 1 -- 172.21.15.119:0/939940630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff53c0b5b60 msgr2=0x7ff53c15ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff53a800640 1 --2- 172.21.15.119:0/939940630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff53c0b5b60 0x7ff53c15ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.486+0000 7ff53a800640 1 -- 172.21.15.119:0/939940630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c0cd8d0 con 0x7ff53c15dce0 2024-04-25T10:12:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.487+0000 7ff53a800640 1 --2- 172.21.15.119:0/939940630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff53c15dce0 0x7ff53c0c96b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff534009870 tx=0x7ff5340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.487+0000 7ff532000640 1 -- 172.21.15.119:0/939940630 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534041020 con 0x7ff53c15dce0 2024-04-25T10:12:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.487+0000 7ff532000640 1 -- 172.21.15.119:0/939940630 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53403d640 con 0x7ff53c15dce0 2024-04-25T10:12:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.487+0000 7ff540e00640 1 -- 172.21.15.119:0/939940630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53c0caaa0 con 0x7ff53c15dce0 2024-04-25T10:12:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.487+0000 7ff532000640 1 -- 172.21.15.119:0/939940630 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534037040 con 0x7ff53c15dce0 2024-04-25T10:12:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.487+0000 7ff540e00640 1 -- 172.21.15.119:0/939940630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53c160870 con 0x7ff53c15dce0 2024-04-25T10:12:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.489+0000 7ff540e00640 1 -- 172.21.15.119:0/939940630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff53c0749b0 con 0x7ff53c15dce0 2024-04-25T10:12:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.490+0000 7ff532000640 1 -- 172.21.15.119:0/939940630 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff534044020 con 0x7ff53c15dce0 2024-04-25T10:12:28.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.490+0000 7ff532000640 1 --2- 172.21.15.119:0/939940630 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff51806b4d0 0x7ff51806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.491+0000 7ff532000640 1 -- 172.21.15.119:0/939940630 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff5340aa950 con 0x7ff53c15dce0 2024-04-25T10:12:28.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.491+0000 7ff539e00640 1 --2- 172.21.15.119:0/939940630 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff51806b4d0 0x7ff51806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.492+0000 7ff539e00640 1 --2- 172.21.15.119:0/939940630 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff51806b4d0 0x7ff51806d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff52c007440 tx=0x7ff52c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.495+0000 7ff532000640 1 -- 172.21.15.119:0/939940630 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff53404b030 con 0x7ff53c15dce0 2024-04-25T10:12:28.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.731+0000 7ff540e00640 1 -- 172.21.15.119:0/939940630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff53c1f0a60 con 0x7ff53c15dce0 2024-04-25T10:12:28.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.732+0000 7ff532000640 1 -- 172.21.15.119:0/939940630 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4950 (secure 0 0 0) 0x7ff534079750 con 0x7ff53c15dce0 2024-04-25T10:12:28.737 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:12:28.737 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2798722683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2798722683},{"type":"v1","addr":"172.21.15.132:6833","nonce":2798722683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2834911408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2834911408},{"type":"v1","addr":"172.21.15.132:6835","nonce":2834911408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/643040335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":643040335},{"type":"v1","addr":"172.21.15.119:6835","nonce":643040335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:34.712290+0000","modified":"2024-04-25T10:11:58.403554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/2127292269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2127292269},{"type":"v1","addr":"172.21.15.119:6837","nonce":2127292269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:12:28.737 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 51 2024-04-25T10:12:28.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.735+0000 7ff540e00640 1 -- 172.21.15.119:0/939940630 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff51806b4d0 msgr2=0x7ff51806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.735+0000 7ff540e00640 1 --2- 172.21.15.119:0/939940630 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff51806b4d0 0x7ff51806d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff52c007440 tx=0x7ff52c00b040 comp rx=0 tx=0).stop 2024-04-25T10:12:28.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.735+0000 7ff540e00640 1 -- 172.21.15.119:0/939940630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff53c15dce0 msgr2=0x7ff53c0c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.735+0000 7ff540e00640 1 --2- 172.21.15.119:0/939940630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff53c15dce0 0x7ff53c0c96b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff534009870 tx=0x7ff5340046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.735+0000 7ff540e00640 1 -- 172.21.15.119:0/939940630 shutdown_connections 2024-04-25T10:12:28.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.735+0000 7ff540e00640 1 --2- 172.21.15.119:0/939940630 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff51806b4d0 0x7ff51806d990 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.735+0000 7ff540e00640 1 --2- 172.21.15.119:0/939940630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff53c15dce0 0x7ff53c0c96b0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.735+0000 7ff540e00640 1 --2- 172.21.15.119:0/939940630 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff53c0d35b0 0x7ff53c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.735+0000 7ff540e00640 1 --2- 172.21.15.119:0/939940630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff53c0b5b60 0x7ff53c15ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.735+0000 7ff540e00640 1 -- 172.21.15.119:0/939940630 >> 172.21.15.119:0/939940630 conn(0x7ff53c072900 msgr2=0x7ff53c0d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.735+0000 7ff540e00640 1 -- 172.21.15.119:0/939940630 shutdown_connections 2024-04-25T10:12:28.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.735+0000 7ff540e00640 1 -- 172.21.15.119:0/939940630 wait complete. 2024-04-25T10:12:28.760 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T10:12:28.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.957+0000 7f4ce4c00640 1 Processor -- start 2024-04-25T10:12:28.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.957+0000 7f4ce4c00640 1 -- start start 2024-04-25T10:12:28.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.958+0000 7f4ce4c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4ce00ce9d0 0x7f4ce00cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.958+0000 7f4ce4c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4ce00cc300 0x7f4ce00cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.958+0000 7f4ce4c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce006dcb0 con 0x7f4ce0144e70 2024-04-25T10:12:28.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.958+0000 7f4ce4c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce015bfe0 con 0x7f4ce00cc300 2024-04-25T10:12:28.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.958+0000 7f4ce4c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce006df90 con 0x7f4ce00ce9d0 2024-04-25T10:12:28.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.959+0000 7f4cdde00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4ce0144e70 0x7f4ce006d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51168/0 (socket says 172.21.15.119:51168) 2024-04-25T10:12:28.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.959+0000 7f4cdde00640 1 -- 172.21.15.119:0/635550779 learned_addr learned my addr 172.21.15.119:0/635550779 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:12:28.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.959+0000 7f4cde800640 1 --2- 172.21.15.119:0/635550779 >> v2:172.21.15.119:3300/0 conn(0x7f4ce00cc300 0x7f4ce00cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.959+0000 7f4cdd400640 1 --2- 172.21.15.119:0/635550779 >> v2:172.21.15.132:3300/0 conn(0x7f4ce00ce9d0 0x7f4ce00cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.959+0000 7f4cdca00640 1 -- 172.21.15.119:0/635550779 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673900934 0 0) 0x7f4ce006dcb0 con 0x7f4ce0144e70 2024-04-25T10:12:28.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.959+0000 7f4cdca00640 1 -- 172.21.15.119:0/635550779 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cc0003520 con 0x7f4ce0144e70 2024-04-25T10:12:28.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.959+0000 7f4cde800640 1 -- 172.21.15.119:0/635550779 >> v2:172.21.15.132:3300/0 conn(0x7f4ce00ce9d0 msgr2=0x7f4ce00cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.959+0000 7f4cde800640 1 --2- 172.21.15.119:0/635550779 >> v2:172.21.15.132:3300/0 conn(0x7f4ce00ce9d0 0x7f4ce00cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.959+0000 7f4cde800640 1 -- 172.21.15.119:0/635550779 >> v1:172.21.15.119:6789/0 conn(0x7f4ce0144e70 legacy=0x7f4ce006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.959+0000 7f4cde800640 1 -- 172.21.15.119:0/635550779 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce006e270 con 0x7f4ce00cc300 2024-04-25T10:12:28.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.960+0000 7f4cde800640 1 --2- 172.21.15.119:0/635550779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00cc300 0x7f4ce00cc740 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0072560 tx=0x7f4cd802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8eff84e21842a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.961+0000 7f4cdca00640 1 -- 172.21.15.119:0/635550779 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8037070 con 0x7f4ce00cc300 2024-04-25T10:12:28.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.961+0000 7f4cdca00640 1 -- 172.21.15.119:0/635550779 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd8032440 con 0x7f4ce00cc300 2024-04-25T10:12:28.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.961+0000 7f4cdca00640 1 -- 172.21.15.119:0/635550779 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd802ae60 con 0x7f4ce00cc300 2024-04-25T10:12:28.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.961+0000 7f4ce4c00640 1 -- 172.21.15.119:0/635550779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00cc300 msgr2=0x7f4ce00cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.961+0000 7f4ce4c00640 1 --2- 172.21.15.119:0/635550779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00cc300 0x7f4ce00cc740 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0072560 tx=0x7f4cd802a110 comp rx=0 tx=0).stop 2024-04-25T10:12:28.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.961+0000 7f4ce4c00640 1 -- 172.21.15.119:0/635550779 shutdown_connections 2024-04-25T10:12:28.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.961+0000 7f4ce4c00640 1 --2- 172.21.15.119:0/635550779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00cc300 0x7f4ce00cc740 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.961+0000 7f4ce4c00640 1 --2- 172.21.15.119:0/635550779 >> v2:172.21.15.132:3300/0 conn(0x7f4ce00ce9d0 0x7f4ce00cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.961+0000 7f4ce4c00640 1 -- 172.21.15.119:0/635550779 >> 172.21.15.119:0/635550779 conn(0x7f4ce006aba0 msgr2=0x7f4ce006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.962+0000 7f4ce4c00640 1 -- 172.21.15.119:0/635550779 shutdown_connections 2024-04-25T10:12:28.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.962+0000 7f4ce4c00640 1 -- 172.21.15.119:0/635550779 wait complete. 2024-04-25T10:12:28.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.963+0000 7f4ce4c00640 1 Processor -- start 2024-04-25T10:12:28.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.963+0000 7f4ce4c00640 1 -- start start 2024-04-25T10:12:28.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.964+0000 7f4ce4c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ce00cc300 0x7f4ce013c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.964+0000 7f4ce4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00ce9d0 0x7f4ce0142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.964+0000 7f4ce4c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ce0144e70 0x7f4ce01412d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.964+0000 7f4ce4c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce006d970 con 0x7f4ce00ce9d0 2024-04-25T10:12:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.964+0000 7f4ce4c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce00cf3a0 con 0x7f4ce00cc300 2024-04-25T10:12:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.964+0000 7f4ce4c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce01454b0 con 0x7f4ce0144e70 2024-04-25T10:12:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.965+0000 7f4cdd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00ce9d0 0x7f4ce0142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.965+0000 7f4cdd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00ce9d0 0x7f4ce0142bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51188/0 (socket says 172.21.15.119:51188) 2024-04-25T10:12:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.965+0000 7f4cdd400640 1 -- 172.21.15.119:0/934820503 learned_addr learned my addr 172.21.15.119:0/934820503 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.965+0000 7f4cde800640 1 --2- 172.21.15.119:0/934820503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ce0144e70 0x7f4ce01412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.965+0000 7f4cdde00640 1 --2- 172.21.15.119:0/934820503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ce00cc300 0x7f4ce013c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.965+0000 7f4cdd400640 1 -- 172.21.15.119:0/934820503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ce0144e70 msgr2=0x7f4ce01412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.965+0000 7f4cdd400640 1 --2- 172.21.15.119:0/934820503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ce0144e70 0x7f4ce01412d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.965+0000 7f4cdd400640 1 -- 172.21.15.119:0/934820503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ce00cc300 msgr2=0x7f4ce013c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.965+0000 7f4cdd400640 1 --2- 172.21.15.119:0/934820503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ce00cc300 0x7f4ce013c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.965+0000 7f4cdd400640 1 -- 172.21.15.119:0/934820503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce013c6e0 con 0x7f4ce00ce9d0 2024-04-25T10:12:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.965+0000 7f4cdd400640 1 --2- 172.21.15.119:0/934820503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00ce9d0 0x7f4ce0142bc0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8007ef0 tx=0x7f4cc800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.966+0000 7f4cd6000640 1 -- 172.21.15.119:0/934820503 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8014620 con 0x7f4ce00ce9d0 2024-04-25T10:12:28.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.966+0000 7f4cd6000640 1 -- 172.21.15.119:0/934820503 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc8019070 con 0x7f4ce00ce9d0 2024-04-25T10:12:28.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.966+0000 7f4cd6000640 1 -- 172.21.15.119:0/934820503 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8014a00 con 0x7f4ce00ce9d0 2024-04-25T10:12:28.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.966+0000 7f4ce4c00640 1 -- 172.21.15.119:0/934820503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce0141b20 con 0x7f4ce00ce9d0 2024-04-25T10:12:28.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.966+0000 7f4ce4c00640 1 -- 172.21.15.119:0/934820503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce01420e0 con 0x7f4ce00ce9d0 2024-04-25T10:12:28.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.968+0000 7f4cd6000640 1 -- 172.21.15.119:0/934820503 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cc8014ba0 con 0x7f4ce00ce9d0 2024-04-25T10:12:28.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.969+0000 7f4cd6000640 1 --2- 172.21.15.119:0/934820503 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4cbc06b4d0 0x7f4cbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.969+0000 7f4cd6000640 1 -- 172.21.15.119:0/934820503 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4cc808ac60 con 0x7f4ce00ce9d0 2024-04-25T10:12:28.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.969+0000 7f4cdde00640 1 --2- 172.21.15.119:0/934820503 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4cbc06b4d0 0x7f4cbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.970+0000 7f4ce79a1740 1 Processor -- start 2024-04-25T10:12:28.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.970+0000 7f4ce79a1740 1 -- start start 2024-04-25T10:12:28.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.970+0000 7f4cdde00640 1 --2- 172.21.15.119:0/934820503 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4cbc06b4d0 0x7f4cbc06d990 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0000f60 tx=0x7f4cd0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.971+0000 7f4ce79a1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa58370 0x55fd1fa58750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.971+0000 7f4ce79a1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa75c10 0x55fd1fa76160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.971+0000 7f4ce79a1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa2e050 0x55fd1fa742d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.971+0000 7f4ce79a1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1f990ae0 con 0x55fd1fa75c10 2024-04-25T10:12:28.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.971+0000 7f4ce79a1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1f9397c0 con 0x55fd1fa2e050 2024-04-25T10:12:28.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.971+0000 7f4ce79a1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd1f98cb80 con 0x55fd1fa58370 2024-04-25T10:12:28.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.971+0000 7f4cdd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa75c10 0x55fd1fa76160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.971+0000 7f4cdd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa75c10 0x55fd1fa76160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51198/0 (socket says 172.21.15.119:51198) 2024-04-25T10:12:28.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.971+0000 7f4cdd400640 1 -- 172.21.15.119:0/3967773743 learned_addr learned my addr 172.21.15.119:0/3967773743 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:28.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.971+0000 7f4cdd400640 1 -- 172.21.15.119:0/3967773743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa58370 msgr2=0x55fd1fa58750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.971+0000 7f4cdd400640 1 --2- 172.21.15.119:0/3967773743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa58370 0x55fd1fa58750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.971+0000 7f4cdd400640 1 -- 172.21.15.119:0/3967773743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa2e050 msgr2=0x55fd1fa742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.971+0000 7f4cdd400640 1 --2- 172.21.15.119:0/3967773743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa2e050 0x55fd1fa742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.971+0000 7f4cdd400640 1 -- 172.21.15.119:0/3967773743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd1f7717c0 con 0x55fd1fa75c10 2024-04-25T10:12:28.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.972+0000 7f4cdd400640 1 --2- 172.21.15.119:0/3967773743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa75c10 0x55fd1fa76160 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8061490 tx=0x7f4cc8059a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b1a0e53eb059fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.972+0000 7f4cce600640 1 -- 172.21.15.119:0/3967773743 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc80619c0 con 0x55fd1fa75c10 2024-04-25T10:12:28.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.972+0000 7f4cce600640 1 -- 172.21.15.119:0/3967773743 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc8057350 con 0x55fd1fa75c10 2024-04-25T10:12:28.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.972+0000 7f4cce600640 1 -- 172.21.15.119:0/3967773743 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8055920 con 0x55fd1fa75c10 2024-04-25T10:12:28.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.973+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3967773743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa75c10 msgr2=0x55fd1fa76160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.973+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/3967773743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa75c10 0x55fd1fa76160 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8061490 tx=0x7f4cc8059a70 comp rx=0 tx=0).stop 2024-04-25T10:12:28.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.973+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3967773743 shutdown_connections 2024-04-25T10:12:28.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.973+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/3967773743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa2e050 0x55fd1fa742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.973+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/3967773743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa75c10 0x55fd1fa76160 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.973+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/3967773743 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa58370 0x55fd1fa58750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.973+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3967773743 >> 172.21.15.119:0/3967773743 conn(0x55fd1fa2a980 msgr2=0x55fd1fa73180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.973+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3967773743 shutdown_connections 2024-04-25T10:12:28.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.973+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3967773743 wait complete. 2024-04-25T10:12:28.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.974+0000 7f4ce79a1740 1 -- start start 2024-04-25T10:12:28.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.974+0000 7f4ce79a1740 1 Processor -- start 2024-04-25T10:12:28.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.975+0000 7f4ce79a1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa58370 0x55fd1fa443a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.975+0000 7f4ce79a1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa75c10 0x55fd1fa6a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.975+0000 7f4ce79a1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa276e0 0x55fd1fa27b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.975+0000 7f4ce79a1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1fa2edf0 con 0x55fd1fa58370 2024-04-25T10:12:28.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.975+0000 7f4ce79a1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1f92ed40 con 0x55fd1fa75c10 2024-04-25T10:12:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.975+0000 7f4ce79a1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd1fa288b0 con 0x55fd1fa276e0 2024-04-25T10:12:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.975+0000 7f4ce79a1740 10 client.0 _ll_register_callbacks cb 0x55fd1f6a4ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.975+0000 7f4cdde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa58370 0x55fd1fa443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.975+0000 7f4cdde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa58370 0x55fd1fa443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51202/0 (socket says 172.21.15.119:51202) 2024-04-25T10:12:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.975+0000 7f4cdde00640 1 -- 172.21.15.119:0/3162200607 learned_addr learned my addr 172.21.15.119:0/3162200607 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.976+0000 7f4cdd400640 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa75c10 0x55fd1fa6a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.976+0000 7f4cde800640 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa276e0 0x55fd1fa27b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.976+0000 7f4cdde00640 1 -- 172.21.15.119:0/3162200607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa276e0 msgr2=0x55fd1fa27b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.976+0000 7f4cdde00640 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa276e0 0x55fd1fa27b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.976+0000 7f4cdde00640 1 -- 172.21.15.119:0/3162200607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa75c10 msgr2=0x55fd1fa6a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.976+0000 7f4cdde00640 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa75c10 0x55fd1fa6a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.976+0000 7f4cdde00640 1 -- 172.21.15.119:0/3162200607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd1f7d1f50 con 0x55fd1fa58370 2024-04-25T10:12:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.976+0000 7f4cdd400640 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa75c10 0x55fd1fa6a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.976+0000 7f4cde800640 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa276e0 0x55fd1fa27b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.976+0000 7f4cdde00640 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa58370 0x55fd1fa443a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0007480 tx=0x7f4cd0007880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bd364d277cc7cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.976+0000 7f4cc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.976+0000 7f4cc6a00640 1 -- 172.21.15.119:0/3162200607 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0014cb0 con 0x55fd1fa58370 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.976+0000 7f4cc6a00640 1 -- 172.21.15.119:0/3162200607 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4cd0022b60 con 0x55fd1fa58370 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.976+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3162200607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cc8057350 con 0x55fd1fa58370 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.977+0000 7f4cc6a00640 1 -- 172.21.15.119:0/3162200607 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cd0014e50 con 0x55fd1fa58370 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.977+0000 7f4ce79a1740 10 client.6864 fetch_fsmap learned FSMap version 51 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.977+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3162200607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55fd1f6650b0 con 0x55fd1fa58370 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.977+0000 7f4cc6a00640 1 -- 172.21.15.119:0/3162200607 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc8057350 con 0x55fd1fa58370 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.977+0000 7f4cc6a00640 1 -- 172.21.15.119:0/3162200607 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4cd0029050 con 0x55fd1fa58370 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.977+0000 7f4ce79a1740 10 client.6864 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.977+0000 7f4ce79a1740 10 client.6864 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.977+0000 7f4ce79a1740 20 client.6864 populate_metadata read hostname 'smithi119' 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.977+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fd1fa42950 0x55fd1fa42d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.977+0000 7f4ce79a1740 4 client.6864 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:28.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.977+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3162200607 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd1f9efb60 con 0x55fd1fa42950 2024-04-25T10:12:28.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.978+0000 7f4cdd400640 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fd1fa42950 0x55fd1fa42d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.978+0000 7f4cc6a00640 10 client.6864 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:28.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.978+0000 7f4cdd400640 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fd1fa42950 0x55fd1fa42d30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a48e47c25ffc8e server_cookie=838cb3630679d619 in_seq=0 out_seq=0 2024-04-25T10:12:28.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.979+0000 7f4cc6a00640 1 -- 172.21.15.119:0/3162200607 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd1f9efb60 con 0x55fd1fa42950 2024-04-25T10:12:28.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.979+0000 7f4cc6a00640 10 client.6864 handle_command_reply: tid=0 2024-04-25T10:12:28.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.979+0000 7f4ce79a1740 1 client.6864 shutdown 2024-04-25T10:12:28.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.979+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3162200607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa58370 msgr2=0x55fd1fa443a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.979+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa58370 0x55fd1fa443a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0007480 tx=0x7f4cd0007880 comp rx=0 tx=0).stop 2024-04-25T10:12:28.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.979+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3162200607 shutdown_connections 2024-04-25T10:12:28.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.979+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fd1fa42950 0x55fd1fa42d30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.979+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa276e0 0x55fd1fa27b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.979+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa75c10 0x55fd1fa6a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.979+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/3162200607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa58370 0x55fd1fa443a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.979+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3162200607 >> 172.21.15.119:0/3162200607 conn(0x55fd1fa2e050 msgr2=0x55fd1fa2c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.979+0000 7f4cc6a00640 0 client.6864 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:28.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.979+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3162200607 shutdown_connections 2024-04-25T10:12:28.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.979+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3162200607 wait complete. 2024-04-25T10:12:28.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.980+0000 7f4ce79a1740 20 client.6864 trim_cache size 0 max 16384 2024-04-25T10:12:28.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.992+0000 7f4ce79a1740 1 Processor -- start 2024-04-25T10:12:28.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.992+0000 7f4ce79a1740 1 -- start start 2024-04-25T10:12:28.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.993+0000 7f4ce79a1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa276e0 0x55fd1fa6b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.993+0000 7f4ce79a1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa58370 0x55fd1fa2c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.993+0000 7f4ce79a1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa75c10 0x55fd1fa2c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.993+0000 7f4ce79a1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce01ff1a0 con 0x55fd1fa276e0 2024-04-25T10:12:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.993+0000 7f4ce79a1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce0140240 con 0x55fd1fa58370 2024-04-25T10:12:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.993+0000 7f4ce79a1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce0140840 con 0x55fd1fa75c10 2024-04-25T10:12:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.994+0000 7f4cdde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa276e0 0x55fd1fa6b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.994+0000 7f4cdde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa276e0 0x55fd1fa6b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51212/0 (socket says 172.21.15.119:51212) 2024-04-25T10:12:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.994+0000 7f4cdde00640 1 -- 172.21.15.119:0/3876431236 learned_addr learned my addr 172.21.15.119:0/3876431236 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.994+0000 7f4cdde00640 1 -- 172.21.15.119:0/3876431236 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa75c10 msgr2=0x55fd1fa2c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.994+0000 7f4cdde00640 1 --2- 172.21.15.119:0/3876431236 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa75c10 0x55fd1fa2c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.994+0000 7f4cdde00640 1 -- 172.21.15.119:0/3876431236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa58370 msgr2=0x55fd1fa2c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.994+0000 7f4cdde00640 1 --2- 172.21.15.119:0/3876431236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa58370 0x55fd1fa2c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.994+0000 7f4cdde00640 1 -- 172.21.15.119:0/3876431236 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd1fa761a0 con 0x55fd1fa276e0 2024-04-25T10:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.994+0000 7f4cdde00640 1 --2- 172.21.15.119:0/3876431236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa276e0 0x55fd1fa6b600 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0014940 tx=0x7f4cd001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c5dc5132f1cd56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.995+0000 7f4ccdc00640 1 -- 172.21.15.119:0/3876431236 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0014ab0 con 0x55fd1fa276e0 2024-04-25T10:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.995+0000 7f4ccdc00640 1 -- 172.21.15.119:0/3876431236 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd002b470 con 0x55fd1fa276e0 2024-04-25T10:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.995+0000 7f4ccdc00640 1 -- 172.21.15.119:0/3876431236 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd002b820 con 0x55fd1fa276e0 2024-04-25T10:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.995+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3876431236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa276e0 msgr2=0x55fd1fa6b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.995+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/3876431236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa276e0 0x55fd1fa6b600 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0014940 tx=0x7f4cd001c000 comp rx=0 tx=0).stop 2024-04-25T10:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.995+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3876431236 shutdown_connections 2024-04-25T10:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.995+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/3876431236 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa75c10 0x55fd1fa2c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.995+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/3876431236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa58370 0x55fd1fa2c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.995+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/3876431236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa276e0 0x55fd1fa6b600 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.995+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3876431236 >> 172.21.15.119:0/3876431236 conn(0x55fd1fa2e050 msgr2=0x55fd1fa42d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.995+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3876431236 shutdown_connections 2024-04-25T10:12:29.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.995+0000 7f4ce79a1740 1 -- 172.21.15.119:0/3876431236 wait complete. 2024-04-25T10:12:29.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.996+0000 7f4ce79a1740 1 -- start start 2024-04-25T10:12:29.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.996+0000 7f4ce79a1740 1 Processor -- start 2024-04-25T10:12:29.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.997+0000 7f4ce79a1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa276e0 0x55fd1fac3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.997+0000 7f4ce79a1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa58370 0x55fd1fa72b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.997+0000 7f4ce79a1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa75c10 0x55fd1fa2e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.997+0000 7f4ce79a1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1fa6a770 con 0x55fd1fa58370 2024-04-25T10:12:29.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.997+0000 7f4ce79a1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd1fa6ada0 con 0x55fd1fa276e0 2024-04-25T10:12:29.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.997+0000 7f4ce79a1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd1fa76dc0 con 0x55fd1fa75c10 2024-04-25T10:12:29.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.997+0000 7f4ce79a1740 10 client.0 _ll_register_callbacks cb 0x55fd1fa28d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:29.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.998+0000 7f4cdd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa58370 0x55fd1fa72b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.998+0000 7f4cdd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa58370 0x55fd1fa72b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51220/0 (socket says 172.21.15.119:51220) 2024-04-25T10:12:29.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.998+0000 7f4cdd400640 1 -- 172.21.15.119:0/2147114945 learned_addr learned my addr 172.21.15.119:0/2147114945 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:29.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.998+0000 7f4cde800640 1 --2- 172.21.15.119:0/2147114945 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa75c10 0x55fd1fa2e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.998+0000 7f4cdd400640 1 -- 172.21.15.119:0/2147114945 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa75c10 msgr2=0x55fd1fa2e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.998+0000 7f4cdde00640 1 --2- 172.21.15.119:0/2147114945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa276e0 0x55fd1fac3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.998+0000 7f4cdd400640 1 --2- 172.21.15.119:0/2147114945 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa75c10 0x55fd1fa2e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.998+0000 7f4cdd400640 1 -- 172.21.15.119:0/2147114945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa276e0 msgr2=0x55fd1fac3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.998+0000 7f4cdd400640 1 --2- 172.21.15.119:0/2147114945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa276e0 0x55fd1fac3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.998+0000 7f4cdd400640 1 -- 172.21.15.119:0/2147114945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd1fa58c50 con 0x55fd1fa58370 2024-04-25T10:12:29.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.998+0000 7f4cdde00640 1 --2- 172.21.15.119:0/2147114945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa276e0 0x55fd1fac3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:29.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.998+0000 7f4cdd400640 1 --2- 172.21.15.119:0/2147114945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa58370 0x55fd1fa72b20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8058fb0 tx=0x7f4cc8016070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7034a94ce1dd1299 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.998+0000 7f4cc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:12:29.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.998+0000 7f4cc6a00640 1 -- 172.21.15.119:0/2147114945 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc80575b0 con 0x55fd1fa58370 2024-04-25T10:12:29.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.999+0000 7f4cc6a00640 1 -- 172.21.15.119:0/2147114945 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4cc8055920 con 0x55fd1fa58370 2024-04-25T10:12:29.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.999+0000 7f4ce79a1740 1 -- 172.21.15.119:0/2147114945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cd002b470 con 0x55fd1fa58370 2024-04-25T10:12:29.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.999+0000 7f4cc6a00640 1 -- 172.21.15.119:0/2147114945 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cc8059510 con 0x55fd1fa58370 2024-04-25T10:12:29.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.999+0000 7f4ce79a1740 10 client.6870 fetch_fsmap learned FSMap version 51 2024-04-25T10:12:29.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.999+0000 7f4ce79a1740 1 -- 172.21.15.119:0/2147114945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55fd1fa72600 con 0x55fd1fa58370 2024-04-25T10:12:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.999+0000 7f4cc6a00640 1 -- 172.21.15.119:0/2147114945 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd002b470 con 0x55fd1fa58370 2024-04-25T10:12:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.999+0000 7f4cc6a00640 1 -- 172.21.15.119:0/2147114945 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4cc8061530 con 0x55fd1fa58370 2024-04-25T10:12:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.999+0000 7f4ce79a1740 10 client.6870 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:12:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.999+0000 7f4ce79a1740 10 client.6870 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:12:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.999+0000 7f4ce79a1740 20 client.6870 populate_metadata read hostname 'smithi119' 2024-04-25T10:12:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.999+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/2147114945 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fd1fa2f390 0x55fd1fa2f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.999+0000 7f4ce79a1740 4 client.6870 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:12:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.999+0000 7f4ce79a1740 1 -- 172.21.15.119:0/2147114945 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55fd1f99e020 con 0x55fd1fa2f390 2024-04-25T10:12:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:28.999+0000 7f4cdde00640 1 --2- 172.21.15.119:0/2147114945 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fd1fa2f390 0x55fd1fa2f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.000+0000 7f4cdde00640 1 --2- 172.21.15.119:0/2147114945 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fd1fa2f390 0x55fd1fa2f770 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb4edce2c00dd19 server_cookie=dc78bb3602b6323f in_seq=0 out_seq=0 2024-04-25T10:12:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.000+0000 7f4cc6a00640 10 client.6870 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.000+0000 7f4cc6a00640 1 -- 172.21.15.119:0/2147114945 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+236 (crc 0 0 0) 0x55fd1f99e020 con 0x55fd1fa2f390 2024-04-25T10:12:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.000+0000 7f4cc6a00640 10 client.6870 handle_command_reply: tid=0 2024-04-25T10:12:29.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.000+0000 7f4ce79a1740 1 client.6870 shutdown 2024-04-25T10:12:29.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.000+0000 7f4ce79a1740 1 -- 172.21.15.119:0/2147114945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa58370 msgr2=0x55fd1fa72b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.000+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/2147114945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa58370 0x55fd1fa72b20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8058fb0 tx=0x7f4cc8016070 comp rx=0 tx=0).stop 2024-04-25T10:12:29.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.001+0000 7f4ce79a1740 1 -- 172.21.15.119:0/2147114945 shutdown_connections 2024-04-25T10:12:29.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.001+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/2147114945 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fd1fa2f390 0x55fd1fa2f770 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.001+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/2147114945 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fd1fa75c10 0x55fd1fa2e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.001+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/2147114945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd1fa58370 0x55fd1fa72b20 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.001+0000 7f4ce79a1740 1 --2- 172.21.15.119:0/2147114945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fd1fa276e0 0x55fd1fac3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.001+0000 7f4cc6a00640 0 client.6870 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:29.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.001+0000 7f4ce79a1740 1 -- 172.21.15.119:0/2147114945 >> 172.21.15.119:0/2147114945 conn(0x55fd1fa2e050 msgr2=0x55fd1fa29430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.001+0000 7f4ce79a1740 1 -- 172.21.15.119:0/2147114945 shutdown_connections 2024-04-25T10:12:29.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.001+0000 7f4ce79a1740 1 -- 172.21.15.119:0/2147114945 wait complete. 2024-04-25T10:12:29.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.001+0000 7f4ce79a1740 20 client.6870 trim_cache size 0 max 16384 2024-04-25T10:12:29.010 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:12:29.010 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2024-04-25T10:12:29.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:12:29.010 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "dentry", 2024-04-25T10:12:29.010 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 112189911, 2024-04-25T10:12:29.010 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 1099511627776, 2024-04-25T10:12:29.010 INFO:teuthology.orchestra.run.smithi119.stdout: "frag": "*", 2024-04-25T10:12:29.010 INFO:teuthology.orchestra.run.smithi119.stdout: "dname": "file_to_be_damaged", 2024-04-25T10:12:29.010 INFO:teuthology.orchestra.run.smithi119.stdout: "snap_id": "head", 2024-04-25T10:12:29.010 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-25T10:12:29.011 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:12:29.011 INFO:teuthology.orchestra.run.smithi119.stdout:] 2024-04-25T10:12:29.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.001+0000 7f4ce4c00640 1 -- 172.21.15.119:0/934820503 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4cbc06b4d0 msgr2=0x7f4cbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.001+0000 7f4ce4c00640 1 --2- 172.21.15.119:0/934820503 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4cbc06b4d0 0x7f4cbc06d990 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0000f60 tx=0x7f4cd0009040 comp rx=0 tx=0).stop 2024-04-25T10:12:29.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.002+0000 7f4ce4c00640 1 -- 172.21.15.119:0/934820503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00ce9d0 msgr2=0x7f4ce0142bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.002+0000 7f4ce4c00640 1 --2- 172.21.15.119:0/934820503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00ce9d0 0x7f4ce0142bc0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8007ef0 tx=0x7f4cc800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.002+0000 7f4ce4c00640 1 -- 172.21.15.119:0/934820503 shutdown_connections 2024-04-25T10:12:29.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.002+0000 7f4ce4c00640 1 --2- 172.21.15.119:0/934820503 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4cbc06b4d0 0x7f4cbc06d990 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.002+0000 7f4ce4c00640 1 --2- 172.21.15.119:0/934820503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ce0144e70 0x7f4ce01412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.002+0000 7f4ce4c00640 1 --2- 172.21.15.119:0/934820503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce00ce9d0 0x7f4ce0142bc0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.002+0000 7f4ce4c00640 1 --2- 172.21.15.119:0/934820503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ce00cc300 0x7f4ce013c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.002+0000 7f4ce4c00640 1 -- 172.21.15.119:0/934820503 >> 172.21.15.119:0/934820503 conn(0x7f4ce006aba0 msgr2=0x7f4ce00cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.002+0000 7f4ce4c00640 1 -- 172.21.15.119:0/934820503 shutdown_connections 2024-04-25T10:12:29.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.002+0000 7f4ce4c00640 1 -- 172.21.15.119:0/934820503 wait complete. 2024-04-25T10:12:29.024 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:12:29.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.207+0000 7f7407e00640 1 Processor -- start 2024-04-25T10:12:29.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.208+0000 7f7407e00640 1 -- start start 2024-04-25T10:12:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.209+0000 7f7407e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f74080d3f80 0x7f74080d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.209+0000 7f7407e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74080b9270 0x7f74080d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.209+0000 7f7407e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740815ef10 con 0x7f740815db60 2024-04-25T10:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.209+0000 7f7407e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740815e390 con 0x7f74080d3f80 2024-04-25T10:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.209+0000 7f7407e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740815e670 con 0x7f74080b9270 2024-04-25T10:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.209+0000 7f7406000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f74080d3f80 0x7f74080d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.209+0000 7f7406000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f74080d3f80 0x7f74080d1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51232/0 (socket says 172.21.15.119:51232) 2024-04-25T10:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.210+0000 7f7406000640 1 -- 172.21.15.119:0/912763102 learned_addr learned my addr 172.21.15.119:0/912763102 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.210+0000 7f7407400640 1 --2- 172.21.15.119:0/912763102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74080b9270 0x7f74080d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.210+0000 7f7406000640 1 -- 172.21.15.119:0/912763102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74080b9270 msgr2=0x7f74080d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.210+0000 7f7406000640 1 --2- 172.21.15.119:0/912763102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74080b9270 0x7f74080d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.210+0000 7f7406000640 1 -- 172.21.15.119:0/912763102 >> v1:172.21.15.132:6789/0 conn(0x7f740815db60 legacy=0x7f74080d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.210+0000 7f7406000640 1 -- 172.21.15.119:0/912763102 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740815e950 con 0x7f74080d3f80 2024-04-25T10:12:29.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.211+0000 7f7406000640 1 --2- 172.21.15.119:0/912763102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74080d3f80 0x7f74080d1f90 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f74080d1c80 tx=0x7f73f002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374d00c863a98f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.211+0000 7f7405600640 1 -- 172.21.15.119:0/912763102 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0004030 con 0x7f74080d3f80 2024-04-25T10:12:29.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.212+0000 7f7405600640 1 -- 172.21.15.119:0/912763102 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f00041d0 con 0x7f74080d3f80 2024-04-25T10:12:29.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.212+0000 7f7405600640 1 -- 172.21.15.119:0/912763102 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0003610 con 0x7f74080d3f80 2024-04-25T10:12:29.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.212+0000 7f7407e00640 1 -- 172.21.15.119:0/912763102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74080d3f80 msgr2=0x7f74080d1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.212+0000 7f7407e00640 1 --2- 172.21.15.119:0/912763102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74080d3f80 0x7f74080d1f90 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f74080d1c80 tx=0x7f73f002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.212+0000 7f7407e00640 1 -- 172.21.15.119:0/912763102 shutdown_connections 2024-04-25T10:12:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.212+0000 7f7407e00640 1 --2- 172.21.15.119:0/912763102 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74080b9270 0x7f74080d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.212+0000 7f7407e00640 1 --2- 172.21.15.119:0/912763102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74080d3f80 0x7f74080d1f90 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.212+0000 7f7407e00640 1 -- 172.21.15.119:0/912763102 >> 172.21.15.119:0/912763102 conn(0x7f74080726c0 msgr2=0x7f7408072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.212+0000 7f7407e00640 1 -- 172.21.15.119:0/912763102 shutdown_connections 2024-04-25T10:12:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.212+0000 7f7407e00640 1 -- 172.21.15.119:0/912763102 wait complete. 2024-04-25T10:12:29.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.214+0000 7f7407e00640 1 Processor -- start 2024-04-25T10:12:29.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.214+0000 7f7407e00640 1 -- start start 2024-04-25T10:12:29.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.215+0000 7f7407e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74080b9270 0x7f74081f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.215+0000 7f7407e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74080d3f80 0x7f74080b8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.215+0000 7f7407e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740815db60 0x7f74080b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.215+0000 7f7407e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74080b3120 con 0x7f740815db60 2024-04-25T10:12:29.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.215+0000 7f7407e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74080b2fa0 con 0x7f74080d3f80 2024-04-25T10:12:29.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.215+0000 7f7407e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740815df40 con 0x7f74080b9270 2024-04-25T10:12:29.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.215+0000 7f7407400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740815db60 0x7f74080b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.216+0000 7f7407400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740815db60 0x7f74080b8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51236/0 (socket says 172.21.15.119:51236) 2024-04-25T10:12:29.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.216+0000 7f7407400640 1 -- 172.21.15.119:0/3237615097 learned_addr learned my addr 172.21.15.119:0/3237615097 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:29.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.216+0000 7f7406000640 1 --2- 172.21.15.119:0/3237615097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74080d3f80 0x7f74080b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.216+0000 7f7406a00640 1 --2- 172.21.15.119:0/3237615097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74080b9270 0x7f74081f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.216+0000 7f7407400640 1 -- 172.21.15.119:0/3237615097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74080b9270 msgr2=0x7f74081f7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.216+0000 7f7407400640 1 --2- 172.21.15.119:0/3237615097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74080b9270 0x7f74081f7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.216+0000 7f7407400640 1 -- 172.21.15.119:0/3237615097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74080d3f80 msgr2=0x7f74080b8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.216+0000 7f7407400640 1 --2- 172.21.15.119:0/3237615097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74080d3f80 0x7f74080b8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.216+0000 7f7407400640 1 -- 172.21.15.119:0/3237615097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74081f8170 con 0x7f740815db60 2024-04-25T10:12:29.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.216+0000 7f7407400640 1 --2- 172.21.15.119:0/3237615097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740815db60 0x7f74080b8ba0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f73f800d5d0 tx=0x7f73f800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.217+0000 7f73fea00640 1 -- 172.21.15.119:0/3237615097 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f80125f0 con 0x7f740815db60 2024-04-25T10:12:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.217+0000 7f73fea00640 1 -- 172.21.15.119:0/3237615097 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f8019070 con 0x7f740815db60 2024-04-25T10:12:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.217+0000 7f73fea00640 1 -- 172.21.15.119:0/3237615097 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f80129f0 con 0x7f740815db60 2024-04-25T10:12:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.217+0000 7f7407e00640 1 -- 172.21.15.119:0/3237615097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74080ccf20 con 0x7f740815db60 2024-04-25T10:12:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.217+0000 7f7407e00640 1 -- 172.21.15.119:0/3237615097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74080aa390 con 0x7f740815db60 2024-04-25T10:12:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.218+0000 7f7407e00640 1 -- 172.21.15.119:0/3237615097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74080749b0 con 0x7f740815db60 2024-04-25T10:12:29.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.219+0000 7f73fea00640 1 -- 172.21.15.119:0/3237615097 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73f8012b90 con 0x7f740815db60 2024-04-25T10:12:29.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.220+0000 7f73fea00640 1 --2- 172.21.15.119:0/3237615097 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f73e406b500 0x7f73e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.220+0000 7f73fea00640 1 -- 172.21.15.119:0/3237615097 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f73f808a6e0 con 0x7f740815db60 2024-04-25T10:12:29.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.220+0000 7f7406a00640 1 --2- 172.21.15.119:0/3237615097 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f73e406b500 0x7f73e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.221+0000 7f7406a00640 1 --2- 172.21.15.119:0/3237615097 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f73e406b500 0x7f73e406d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7400009f20 tx=0x7f74000094d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.224+0000 7f73fea00640 1 -- 172.21.15.119:0/3237615097 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74081f8170 con 0x7f740815db60 2024-04-25T10:12:29.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.457+0000 7f7407e00640 1 -- 172.21.15.119:0/3237615097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f74080b2150 con 0x7f740815db60 2024-04-25T10:12:29.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.457+0000 7f73fea00640 1 -- 172.21.15.119:0/3237615097 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f73f8059440 con 0x7f740815db60 2024-04-25T10:12:29.462 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:12:29.462 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:12:29.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.460+0000 7f7407e00640 1 -- 172.21.15.119:0/3237615097 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f73e406b500 msgr2=0x7f73e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.460+0000 7f7407e00640 1 --2- 172.21.15.119:0/3237615097 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f73e406b500 0x7f73e406d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7400009f20 tx=0x7f74000094d0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.460+0000 7f7407e00640 1 -- 172.21.15.119:0/3237615097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740815db60 msgr2=0x7f74080b8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.460+0000 7f7407e00640 1 --2- 172.21.15.119:0/3237615097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740815db60 0x7f74080b8ba0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f73f800d5d0 tx=0x7f73f800da90 comp rx=0 tx=0).stop 2024-04-25T10:12:29.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.460+0000 7f7407e00640 1 -- 172.21.15.119:0/3237615097 shutdown_connections 2024-04-25T10:12:29.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.460+0000 7f7407e00640 1 --2- 172.21.15.119:0/3237615097 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f73e406b500 0x7f73e406d9c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.460+0000 7f7407e00640 1 --2- 172.21.15.119:0/3237615097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740815db60 0x7f74080b8ba0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.460+0000 7f7407e00640 1 --2- 172.21.15.119:0/3237615097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74080d3f80 0x7f74080b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.460+0000 7f7407e00640 1 --2- 172.21.15.119:0/3237615097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74080b9270 0x7f74081f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.460+0000 7f7407e00640 1 -- 172.21.15.119:0/3237615097 >> 172.21.15.119:0/3237615097 conn(0x7f74080726c0 msgr2=0x7f740815ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.460+0000 7f7407e00640 1 -- 172.21.15.119:0/3237615097 shutdown_connections 2024-04-25T10:12:29.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.460+0000 7f7407e00640 1 -- 172.21.15.119:0/3237615097 wait complete. 2024-04-25T10:12:29.484 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:12:29.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:29.484 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-25T10:12:29.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.689+0000 7fe884e00640 1 Processor -- start 2024-04-25T10:12:29.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.689+0000 7fe884e00640 1 -- start start 2024-04-25T10:12:29.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.691+0000 7fe884e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe88015dc50 0x7fe880070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.691+0000 7fe884e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8800cd940 0x7fe8800713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.691+0000 7fe884e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88006c220 con 0x7fe8800cf600 2024-04-25T10:12:29.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.691+0000 7fe884e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8800cfe00 con 0x7fe88015dc50 2024-04-25T10:12:29.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.691+0000 7fe884e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8800cec10 con 0x7fe8800cd940 2024-04-25T10:12:29.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.691+0000 7fe87e800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe8800cf600 0x7fe880161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51178/0 (socket says 172.21.15.119:51178) 2024-04-25T10:12:29.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.691+0000 7fe87e800640 1 -- 172.21.15.119:0/1355033631 learned_addr learned my addr 172.21.15.119:0/1355033631 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:12:29.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87de00640 1 --2- 172.21.15.119:0/1355033631 >> v2:172.21.15.132:3300/0 conn(0x7fe88015dc50 0x7fe880070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe875400640 1 --2- 172.21.15.119:0/1355033631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8800cd940 0x7fe8800713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 -- 172.21.15.119:0/1355033631 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304325134 0 0) 0x7fe88006c220 con 0x7fe8800cf600 2024-04-25T10:12:29.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 -- 172.21.15.119:0/1355033631 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe864003520 con 0x7fe8800cf600 2024-04-25T10:12:29.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 -- 172.21.15.119:0/1355033631 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2752145797 0 0) 0x7fe864003520 con 0x7fe8800cf600 2024-04-25T10:12:29.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 -- 172.21.15.119:0/1355033631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8800cd940 msgr2=0x7fe8800713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 --2- 172.21.15.119:0/1355033631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8800cd940 0x7fe8800713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 -- 172.21.15.119:0/1355033631 >> v2:172.21.15.132:3300/0 conn(0x7fe88015dc50 msgr2=0x7fe880070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 --2- 172.21.15.119:0/1355033631 >> v2:172.21.15.132:3300/0 conn(0x7fe88015dc50 0x7fe880070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 -- 172.21.15.119:0/1355033631 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8800d0050 con 0x7fe8800cf600 2024-04-25T10:12:29.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 -- 172.21.15.119:0/1355033631 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fe8780019f0 con 0x7fe8800cf600 2024-04-25T10:12:29.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 -- 172.21.15.119:0/1355033631 >> v1:172.21.15.119:6789/0 conn(0x7fe8800cf600 legacy=0x7fe880161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 --2- 172.21.15.119:0/1355033631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8640039b0 0x7fe864003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 --2- 172.21.15.119:0/1355033631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe864004800 0x7fe864004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 --2- 172.21.15.119:0/1355033631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8800cd940 0x7fe8640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 -- 172.21.15.119:0/1355033631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8800cfe00 con 0x7fe8640039b0 2024-04-25T10:12:29.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 -- 172.21.15.119:0/1355033631 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8800cec10 con 0x7fe864004800 2024-04-25T10:12:29.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.692+0000 7fe87d400640 1 -- 172.21.15.119:0/1355033631 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86400b9c0 con 0x7fe8800cd940 2024-04-25T10:12:29.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.693+0000 7fe87de00640 1 --2- 172.21.15.119:0/1355033631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8640039b0 0x7fe864003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.693+0000 7fe875400640 1 --2- 172.21.15.119:0/1355033631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe864004800 0x7fe864004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.693+0000 7fe87e800640 1 --2- 172.21.15.119:0/1355033631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8800cd940 0x7fe8640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.693+0000 7fe87de00640 1 -- 172.21.15.119:0/1355033631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8800cd940 msgr2=0x7fe8640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.693+0000 7fe87de00640 1 --2- 172.21.15.119:0/1355033631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8800cd940 0x7fe8640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.693+0000 7fe87de00640 1 -- 172.21.15.119:0/1355033631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe864004800 msgr2=0x7fe864004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.693+0000 7fe87de00640 1 --2- 172.21.15.119:0/1355033631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe864004800 0x7fe864004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.693+0000 7fe87de00640 1 -- 172.21.15.119:0/1355033631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe864003520 con 0x7fe8640039b0 2024-04-25T10:12:29.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.693+0000 7fe87e800640 1 --2- 172.21.15.119:0/1355033631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8800cd940 0x7fe8640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:29.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.693+0000 7fe875400640 1 --2- 172.21.15.119:0/1355033631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe864004800 0x7fe864004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:29.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.694+0000 7fe87de00640 1 --2- 172.21.15.119:0/1355033631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8640039b0 0x7fe864003dd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe870009ac0 tx=0x7fe864005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f435febe3a204a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.694+0000 7fe87d400640 1 -- 172.21.15.119:0/1355033631 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870036070 con 0x7fe8640039b0 2024-04-25T10:12:29.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.694+0000 7fe87d400640 1 -- 172.21.15.119:0/1355033631 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe870031400 con 0x7fe8640039b0 2024-04-25T10:12:29.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.694+0000 7fe87d400640 1 -- 172.21.15.119:0/1355033631 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870009690 con 0x7fe8640039b0 2024-04-25T10:12:29.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.695+0000 7fe884e00640 1 -- 172.21.15.119:0/1355033631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8640039b0 msgr2=0x7fe864003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.695+0000 7fe884e00640 1 --2- 172.21.15.119:0/1355033631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8640039b0 0x7fe864003dd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe870009ac0 tx=0x7fe864005830 comp rx=0 tx=0).stop 2024-04-25T10:12:29.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.695+0000 7fe87e800640 1 -- 172.21.15.119:0/1355033631 reap_dead start 2024-04-25T10:12:29.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.695+0000 7fe884e00640 1 -- 172.21.15.119:0/1355033631 shutdown_connections 2024-04-25T10:12:29.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.695+0000 7fe884e00640 1 -- 172.21.15.119:0/1355033631 >> 172.21.15.119:0/1355033631 conn(0x7fe88006f430 msgr2=0x7fe88006f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.695+0000 7fe884e00640 1 -- 172.21.15.119:0/1355033631 shutdown_connections 2024-04-25T10:12:29.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.695+0000 7fe884e00640 1 -- 172.21.15.119:0/1355033631 wait complete. 2024-04-25T10:12:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.697+0000 7fe884e00640 1 Processor -- start 2024-04-25T10:12:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.697+0000 7fe884e00640 1 -- start start 2024-04-25T10:12:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.698+0000 7fe884e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8640039b0 0x7fe88015e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.698+0000 7fe884e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe880160f00 0x7fe8800b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.698+0000 7fe884e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe880151690 0x7fe880151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.698+0000 7fe884e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8800ce8a0 con 0x7fe880151690 2024-04-25T10:12:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.698+0000 7fe884e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8800cc0c0 con 0x7fe880160f00 2024-04-25T10:12:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.698+0000 7fe884e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8800cd620 con 0x7fe8640039b0 2024-04-25T10:12:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.698+0000 7fe87e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe880151690 0x7fe880151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.698+0000 7fe87e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe880151690 0x7fe880151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51264/0 (socket says 172.21.15.119:51264) 2024-04-25T10:12:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.698+0000 7fe87e800640 1 -- 172.21.15.119:0/2201605009 learned_addr learned my addr 172.21.15.119:0/2201605009 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.699+0000 7fe875400640 1 --2- 172.21.15.119:0/2201605009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe880160f00 0x7fe8800b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.699+0000 7fe87e800640 1 -- 172.21.15.119:0/2201605009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8640039b0 msgr2=0x7fe88015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.699+0000 7fe87e800640 1 --2- 172.21.15.119:0/2201605009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8640039b0 0x7fe88015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.699+0000 7fe87e800640 1 -- 172.21.15.119:0/2201605009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe880160f00 msgr2=0x7fe8800b67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.699+0000 7fe87e800640 1 --2- 172.21.15.119:0/2201605009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe880160f00 0x7fe8800b67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.699+0000 7fe87e800640 1 -- 172.21.15.119:0/2201605009 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8800b55d0 con 0x7fe880151690 2024-04-25T10:12:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.699+0000 7fe875400640 1 --2- 172.21.15.119:0/2201605009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe880160f00 0x7fe8800b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.699+0000 7fe87e800640 1 --2- 172.21.15.119:0/2201605009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe880151690 0x7fe880151b40 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe8800b4430 tx=0x7fe87800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.699+0000 7fe876a00640 1 -- 172.21.15.119:0/2201605009 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87800b870 con 0x7fe880151690 2024-04-25T10:12:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.699+0000 7fe876a00640 1 -- 172.21.15.119:0/2201605009 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87800ba10 con 0x7fe880151690 2024-04-25T10:12:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.699+0000 7fe876a00640 1 -- 172.21.15.119:0/2201605009 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87800bdc0 con 0x7fe880151690 2024-04-25T10:12:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.699+0000 7fe884e00640 1 -- 172.21.15.119:0/2201605009 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8800b58b0 con 0x7fe880151690 2024-04-25T10:12:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.700+0000 7fe884e00640 1 -- 172.21.15.119:0/2201605009 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88015e570 con 0x7fe880151690 2024-04-25T10:12:29.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.702+0000 7fe876a00640 1 -- 172.21.15.119:0/2201605009 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe87801b020 con 0x7fe880151690 2024-04-25T10:12:29.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.702+0000 7fe876a00640 1 --2- 172.21.15.119:0/2201605009 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe85c06b500 0x7fe85c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.703+0000 7fe876a00640 1 -- 172.21.15.119:0/2201605009 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe87808a350 con 0x7fe880151690 2024-04-25T10:12:29.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.703+0000 7fe87de00640 1 --2- 172.21.15.119:0/2201605009 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe85c06b500 0x7fe85c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.704+0000 7fe887c4c740 1 Processor -- start 2024-04-25T10:12:29.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.704+0000 7fe87de00640 1 --2- 172.21.15.119:0/2201605009 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe85c06b500 0x7fe85c06d9c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe870033000 tx=0x7fe87002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.704+0000 7fe887c4c740 1 -- start start 2024-04-25T10:12:29.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.705+0000 7fe887c4c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f91f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.705+0000 7fe887c4c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f91f6f0 0x55df3f8eee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.705+0000 7fe887c4c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f8ef390 0x55df3f917150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.705+0000 7fe887c4c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3f857b50 con 0x55df3f91ecd0 2024-04-25T10:12:29.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.705+0000 7fe887c4c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3f800fb0 con 0x55df3f91f6f0 2024-04-25T10:12:29.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.705+0000 7fe887c4c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df3f853bf0 con 0x55df3f8ef390 2024-04-25T10:12:29.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.705+0000 7fe87de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f91f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.705+0000 7fe87de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f91f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51272/0 (socket says 172.21.15.119:51272) 2024-04-25T10:12:29.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.705+0000 7fe87de00640 1 -- 172.21.15.119:0/1522494925 learned_addr learned my addr 172.21.15.119:0/1522494925 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:29.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.705+0000 7fe87de00640 1 -- 172.21.15.119:0/1522494925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f8ef390 msgr2=0x55df3f917150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.705+0000 7fe87de00640 1 --2- 172.21.15.119:0/1522494925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f8ef390 0x55df3f917150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.705+0000 7fe87de00640 1 -- 172.21.15.119:0/1522494925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f91f6f0 msgr2=0x55df3f8eee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.705+0000 7fe875400640 1 --2- 172.21.15.119:0/1522494925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f91f6f0 0x55df3f8eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.706+0000 7fe87de00640 1 --2- 172.21.15.119:0/1522494925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f91f6f0 0x55df3f8eee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.706+0000 7fe87de00640 1 -- 172.21.15.119:0/1522494925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df3f634a50 con 0x55df3f91ecd0 2024-04-25T10:12:29.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.706+0000 7fe87de00640 1 --2- 172.21.15.119:0/1522494925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f91f0b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe870034040 tx=0x7fe87002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50283a55c1b2953 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.706+0000 7fe86ea00640 1 -- 172.21.15.119:0/1522494925 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87004d070 con 0x55df3f91ecd0 2024-04-25T10:12:29.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.706+0000 7fe86ea00640 1 -- 172.21.15.119:0/1522494925 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe870003b60 con 0x55df3f91ecd0 2024-04-25T10:12:29.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.706+0000 7fe86ea00640 1 -- 172.21.15.119:0/1522494925 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870036040 con 0x55df3f91ecd0 2024-04-25T10:12:29.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.707+0000 7fe887c4c740 1 -- 172.21.15.119:0/1522494925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 msgr2=0x55df3f91f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.707+0000 7fe887c4c740 1 --2- 172.21.15.119:0/1522494925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f91f0b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe870034040 tx=0x7fe87002f000 comp rx=0 tx=0).stop 2024-04-25T10:12:29.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.707+0000 7fe887c4c740 1 -- 172.21.15.119:0/1522494925 shutdown_connections 2024-04-25T10:12:29.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.707+0000 7fe887c4c740 1 --2- 172.21.15.119:0/1522494925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f8ef390 0x55df3f917150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.707+0000 7fe887c4c740 1 --2- 172.21.15.119:0/1522494925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f91f6f0 0x55df3f8eee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.707+0000 7fe887c4c740 1 --2- 172.21.15.119:0/1522494925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f91f0b0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.707+0000 7fe887c4c740 1 -- 172.21.15.119:0/1522494925 >> 172.21.15.119:0/1522494925 conn(0x55df3f8f10c0 msgr2=0x55df3f939ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.707+0000 7fe887c4c740 1 -- 172.21.15.119:0/1522494925 shutdown_connections 2024-04-25T10:12:29.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.707+0000 7fe887c4c740 1 -- 172.21.15.119:0/1522494925 wait complete. 2024-04-25T10:12:29.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.708+0000 7fe887c4c740 1 -- start start 2024-04-25T10:12:29.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.708+0000 7fe887c4c740 1 Processor -- start 2024-04-25T10:12:29.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.709+0000 7fe887c4c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f90e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.709+0000 7fe887c4c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f91f6f0 0x55df3f90eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.709+0000 7fe887c4c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f90f0c0 0x55df3f91fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.709+0000 7fe887c4c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3f857b50 con 0x55df3f91ecd0 2024-04-25T10:12:29.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.709+0000 7fe887c4c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3f7f6590 con 0x55df3f90f0c0 2024-04-25T10:12:29.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.709+0000 7fe887c4c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df3f91b420 con 0x55df3f91f6f0 2024-04-25T10:12:29.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.709+0000 7fe887c4c740 10 client.0 _ll_register_callbacks cb 0x55df3f56bed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:29.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.709+0000 7fe87de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f90e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.709+0000 7fe87de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f90e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51282/0 (socket says 172.21.15.119:51282) 2024-04-25T10:12:29.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.709+0000 7fe87de00640 1 -- 172.21.15.119:0/1790394290 learned_addr learned my addr 172.21.15.119:0/1790394290 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:29.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.710+0000 7fe87de00640 1 -- 172.21.15.119:0/1790394290 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f91f6f0 msgr2=0x55df3f90eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.710+0000 7fe87de00640 1 --2- 172.21.15.119:0/1790394290 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f91f6f0 0x55df3f90eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.710+0000 7fe87de00640 1 -- 172.21.15.119:0/1790394290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f90f0c0 msgr2=0x55df3f91fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.710+0000 7fe87de00640 1 --2- 172.21.15.119:0/1790394290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f90f0c0 0x55df3f91fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.710+0000 7fe87de00640 1 -- 172.21.15.119:0/1790394290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df3f52c3e0 con 0x55df3f91ecd0 2024-04-25T10:12:29.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.710+0000 7fe87de00640 1 --2- 172.21.15.119:0/1790394290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f90e380 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe870009be0 tx=0x7fe8700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b9032d921f04ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.710+0000 7fe863400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:12:29.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.710+0000 7fe863400640 1 -- 172.21.15.119:0/1790394290 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870058060 con 0x55df3f91ecd0 2024-04-25T10:12:29.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.710+0000 7fe863400640 1 -- 172.21.15.119:0/1790394290 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe87003c470 con 0x55df3f91ecd0 2024-04-25T10:12:29.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.710+0000 7fe887c4c740 1 -- 172.21.15.119:0/1790394290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe870003b60 con 0x55df3f91ecd0 2024-04-25T10:12:29.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.710+0000 7fe863400640 1 -- 172.21.15.119:0/1790394290 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe87004d040 con 0x55df3f91ecd0 2024-04-25T10:12:29.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.711+0000 7fe887c4c740 10 client.6888 fetch_fsmap learned FSMap version 51 2024-04-25T10:12:29.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.711+0000 7fe887c4c740 1 -- 172.21.15.119:0/1790394290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55df3f6951e0 con 0x55df3f91ecd0 2024-04-25T10:12:29.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.711+0000 7fe863400640 1 -- 172.21.15.119:0/1790394290 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe870003b60 con 0x55df3f91ecd0 2024-04-25T10:12:29.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.711+0000 7fe863400640 1 -- 172.21.15.119:0/1790394290 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe870050030 con 0x55df3f91ecd0 2024-04-25T10:12:29.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.711+0000 7fe887c4c740 10 client.6888 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:12:29.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.711+0000 7fe887c4c740 10 client.6888 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:12:29.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.711+0000 7fe887c4c740 20 client.6888 populate_metadata read hostname 'smithi119' 2024-04-25T10:12:29.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.711+0000 7fe887c4c740 1 --2- 172.21.15.119:0/1790394290 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55df3f913600 0x55df3f957350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.711+0000 7fe887c4c740 4 client.6888 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:29.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.711+0000 7fe887c4c740 1 -- 172.21.15.119:0/1790394290 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df3f865090 con 0x55df3f913600 2024-04-25T10:12:29.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.711+0000 7fe875400640 1 --2- 172.21.15.119:0/1790394290 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55df3f913600 0x55df3f957350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.712+0000 7fe875400640 1 --2- 172.21.15.119:0/1790394290 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55df3f913600 0x55df3f957350 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75c9813a29620b7a server_cookie=523eef4d7aa72fdd in_seq=0 out_seq=0 2024-04-25T10:12:29.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.712+0000 7fe863400640 10 client.6888 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:29.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.712+0000 7fe863400640 1 -- 172.21.15.119:0/1790394290 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df3f865090 con 0x55df3f913600 2024-04-25T10:12:29.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.713+0000 7fe863400640 10 client.6888 handle_command_reply: tid=0 2024-04-25T10:12:29.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.713+0000 7fe887c4c740 1 client.6888 shutdown 2024-04-25T10:12:29.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.713+0000 7fe887c4c740 1 -- 172.21.15.119:0/1790394290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 msgr2=0x55df3f90e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.713+0000 7fe887c4c740 1 --2- 172.21.15.119:0/1790394290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f90e380 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe870009be0 tx=0x7fe8700046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.713+0000 7fe887c4c740 1 -- 172.21.15.119:0/1790394290 shutdown_connections 2024-04-25T10:12:29.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.713+0000 7fe887c4c740 1 --2- 172.21.15.119:0/1790394290 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55df3f913600 0x55df3f957350 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.713+0000 7fe887c4c740 1 --2- 172.21.15.119:0/1790394290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f90f0c0 0x55df3f91fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.713+0000 7fe887c4c740 1 --2- 172.21.15.119:0/1790394290 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f91f6f0 0x55df3f90eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.713+0000 7fe887c4c740 1 --2- 172.21.15.119:0/1790394290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f90e380 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.713+0000 7fe863400640 0 client.6888 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:29.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.713+0000 7fe887c4c740 1 -- 172.21.15.119:0/1790394290 >> 172.21.15.119:0/1790394290 conn(0x55df3f8ef390 msgr2=0x55df3f914400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.713+0000 7fe887c4c740 1 -- 172.21.15.119:0/1790394290 shutdown_connections 2024-04-25T10:12:29.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.713+0000 7fe887c4c740 1 -- 172.21.15.119:0/1790394290 wait complete. 2024-04-25T10:12:29.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.713+0000 7fe887c4c740 20 client.6888 trim_cache size 0 max 16384 2024-04-25T10:12:29.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.730+0000 7fe887c4c740 1 Processor -- start 2024-04-25T10:12:29.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.730+0000 7fe887c4c740 1 -- start start 2024-04-25T10:12:29.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe887c4c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f90f0c0 0x55df3f94c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe887c4c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f91ecd0 0x55df3f8f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe887c4c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91f6f0 0x55df3f8f4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe887c4c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe880200820 con 0x55df3f91f6f0 2024-04-25T10:12:29.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe887c4c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8801ffa10 con 0x55df3f90f0c0 2024-04-25T10:12:29.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe887c4c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8800ca0d0 con 0x55df3f91ecd0 2024-04-25T10:12:29.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe87e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91f6f0 0x55df3f8f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe87e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91f6f0 0x55df3f8f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51292/0 (socket says 172.21.15.119:51292) 2024-04-25T10:12:29.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe87e800640 1 -- 172.21.15.119:0/1135435959 learned_addr learned my addr 172.21.15.119:0/1135435959 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:29.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe875400640 1 --2- 172.21.15.119:0/1135435959 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f91ecd0 0x55df3f8f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe87e800640 1 -- 172.21.15.119:0/1135435959 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f91ecd0 msgr2=0x55df3f8f3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe87e800640 1 --2- 172.21.15.119:0/1135435959 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f91ecd0 0x55df3f8f3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe87e800640 1 -- 172.21.15.119:0/1135435959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f90f0c0 msgr2=0x55df3f94c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe87e800640 1 --2- 172.21.15.119:0/1135435959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f90f0c0 0x55df3f94c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.731+0000 7fe87e800640 1 -- 172.21.15.119:0/1135435959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df3f5054a0 con 0x55df3f91f6f0 2024-04-25T10:12:29.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.732+0000 7fe87e800640 1 --2- 172.21.15.119:0/1135435959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91f6f0 0x55df3f8f4100 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe878054e00 tx=0x7fe878058430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb48e5ee10bea1ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.732+0000 7fe86e000640 1 -- 172.21.15.119:0/1135435959 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8780610b0 con 0x55df3f91f6f0 2024-04-25T10:12:29.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.732+0000 7fe86e000640 1 -- 172.21.15.119:0/1135435959 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe878056a40 con 0x55df3f91f6f0 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.732+0000 7fe86e000640 1 -- 172.21.15.119:0/1135435959 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe878055010 con 0x55df3f91f6f0 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.732+0000 7fe887c4c740 1 -- 172.21.15.119:0/1135435959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91f6f0 msgr2=0x55df3f8f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.732+0000 7fe887c4c740 1 --2- 172.21.15.119:0/1135435959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91f6f0 0x55df3f8f4100 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe878054e00 tx=0x7fe878058430 comp rx=0 tx=0).stop 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.733+0000 7fe887c4c740 1 -- 172.21.15.119:0/1135435959 shutdown_connections 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.733+0000 7fe887c4c740 1 --2- 172.21.15.119:0/1135435959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91f6f0 0x55df3f8f4100 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.733+0000 7fe887c4c740 1 --2- 172.21.15.119:0/1135435959 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f91ecd0 0x55df3f8f3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.733+0000 7fe887c4c740 1 --2- 172.21.15.119:0/1135435959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f90f0c0 0x55df3f94c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.733+0000 7fe887c4c740 1 -- 172.21.15.119:0/1135435959 >> 172.21.15.119:0/1135435959 conn(0x55df3f8ef390 msgr2=0x55df3f8f5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.733+0000 7fe887c4c740 1 -- 172.21.15.119:0/1135435959 shutdown_connections 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.733+0000 7fe887c4c740 1 -- 172.21.15.119:0/1135435959 wait complete. 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.733+0000 7fe887c4c740 1 -- start start 2024-04-25T10:12:29.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.734+0000 7fe887c4c740 1 Processor -- start 2024-04-25T10:12:29.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe887c4c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f90f0c0 0x55df3f9117a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe887c4c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f90c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe887c4c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f91f6f0 0x55df3f90ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe887c4c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8801ff4a0 con 0x55df3f91ecd0 2024-04-25T10:12:29.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe887c4c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df3f8eabf0 con 0x55df3f90f0c0 2024-04-25T10:12:29.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe887c4c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df3f919bb0 con 0x55df3f91f6f0 2024-04-25T10:12:29.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe887c4c740 10 client.0 _ll_register_callbacks cb 0x7fe870034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:29.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe875400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f90c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe875400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f90c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51296/0 (socket says 172.21.15.119:51296) 2024-04-25T10:12:29.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe875400640 1 -- 172.21.15.119:0/2358773727 learned_addr learned my addr 172.21.15.119:0/2358773727 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:29.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe875400640 1 -- 172.21.15.119:0/2358773727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f91f6f0 msgr2=0x55df3f90ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe875400640 1 --2- 172.21.15.119:0/2358773727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f91f6f0 0x55df3f90ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe875400640 1 -- 172.21.15.119:0/2358773727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f90f0c0 msgr2=0x55df3f9117a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe87de00640 1 --2- 172.21.15.119:0/2358773727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f90f0c0 0x55df3f9117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe875400640 1 --2- 172.21.15.119:0/2358773727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f90f0c0 0x55df3f9117a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.735+0000 7fe875400640 1 -- 172.21.15.119:0/2358773727 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df3f8f4d70 con 0x55df3f91ecd0 2024-04-25T10:12:29.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.736+0000 7fe875400640 1 --2- 172.21.15.119:0/2358773727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f90c8d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x55df3f9104f0 tx=0x7fe868006ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab34cb19813aee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.736+0000 7fe863400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:12:29.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.736+0000 7fe863400640 1 -- 172.21.15.119:0/2358773727 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868016cf0 con 0x55df3f91ecd0 2024-04-25T10:12:29.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.736+0000 7fe863400640 1 -- 172.21.15.119:0/2358773727 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe868004030 con 0x55df3f91ecd0 2024-04-25T10:12:29.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.736+0000 7fe887c4c740 1 -- 172.21.15.119:0/2358773727 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe878056a40 con 0x55df3f91ecd0 2024-04-25T10:12:29.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.736+0000 7fe863400640 1 -- 172.21.15.119:0/2358773727 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe86802a050 con 0x55df3f91ecd0 2024-04-25T10:12:29.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.736+0000 7fe887c4c740 10 client.6894 fetch_fsmap learned FSMap version 51 2024-04-25T10:12:29.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.736+0000 7fe887c4c740 1 -- 172.21.15.119:0/2358773727 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55df3f919950 con 0x55df3f91ecd0 2024-04-25T10:12:29.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.736+0000 7fe863400640 1 -- 172.21.15.119:0/2358773727 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe878056a40 con 0x55df3f91ecd0 2024-04-25T10:12:29.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.736+0000 7fe863400640 1 -- 172.21.15.119:0/2358773727 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe86802d020 con 0x55df3f91ecd0 2024-04-25T10:12:29.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.737+0000 7fe887c4c740 10 client.6894 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:12:29.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.737+0000 7fe887c4c740 10 client.6894 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:12:29.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.737+0000 7fe887c4c740 20 client.6894 populate_metadata read hostname 'smithi119' 2024-04-25T10:12:29.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.737+0000 7fe887c4c740 1 --2- 172.21.15.119:0/2358773727 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55df3f99b7e0 0x55df3f8ede90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.737+0000 7fe887c4c740 4 client.6894 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:12:29.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.737+0000 7fe887c4c740 1 -- 172.21.15.119:0/2358773727 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) v1 -- 0x55df3f914220 con 0x55df3f99b7e0 2024-04-25T10:12:29.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.737+0000 7fe87de00640 1 --2- 172.21.15.119:0/2358773727 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55df3f99b7e0 0x55df3f8ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.737+0000 7fe87de00640 1 --2- 172.21.15.119:0/2358773727 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55df3f99b7e0 0x55df3f8ede90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=485ea50159cc861e server_cookie=59103c000d66b9f4 in_seq=0 out_seq=0 2024-04-25T10:12:29.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.737+0000 7fe863400640 10 client.6894 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:29.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.738+0000 7fe863400640 1 -- 172.21.15.119:0/2358773727 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55df3f914220 con 0x55df3f99b7e0 2024-04-25T10:12:29.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.738+0000 7fe863400640 10 client.6894 handle_command_reply: tid=0 2024-04-25T10:12:29.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.738+0000 7fe887c4c740 1 client.6894 shutdown 2024-04-25T10:12:29.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.738+0000 7fe887c4c740 1 -- 172.21.15.119:0/2358773727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 msgr2=0x55df3f90c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.738+0000 7fe887c4c740 1 --2- 172.21.15.119:0/2358773727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f90c8d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x55df3f9104f0 tx=0x7fe868006ec0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.738+0000 7fe887c4c740 1 -- 172.21.15.119:0/2358773727 shutdown_connections 2024-04-25T10:12:29.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.738+0000 7fe887c4c740 1 --2- 172.21.15.119:0/2358773727 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55df3f99b7e0 0x55df3f8ede90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.738+0000 7fe887c4c740 1 --2- 172.21.15.119:0/2358773727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55df3f91f6f0 0x55df3f90ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.738+0000 7fe887c4c740 1 --2- 172.21.15.119:0/2358773727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3f91ecd0 0x55df3f90c8d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.738+0000 7fe887c4c740 1 --2- 172.21.15.119:0/2358773727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df3f90f0c0 0x55df3f9117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.738+0000 7fe863400640 0 client.6894 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:29.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.738+0000 7fe887c4c740 1 -- 172.21.15.119:0/2358773727 >> 172.21.15.119:0/2358773727 conn(0x55df3f8ef390 msgr2=0x55df3f9224b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.738+0000 7fe887c4c740 1 -- 172.21.15.119:0/2358773727 shutdown_connections 2024-04-25T10:12:29.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.739+0000 7fe887c4c740 1 -- 172.21.15.119:0/2358773727 wait complete. 2024-04-25T10:12:29.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.739+0000 7fe887c4c740 20 client.6894 trim_cache size 0 max 16384 2024-04-25T10:12:29.750 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:12:29.750 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2024-04-25T10:12:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "0bc2c628-04ea-423b-baf7-d32f078fcfe6", 2024-04-25T10:12:29.751 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2024-04-25T10:12:29.751 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:12:29.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.739+0000 7fe884e00640 1 -- 172.21.15.119:0/2201605009 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe85c06b500 msgr2=0x7fe85c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.739+0000 7fe884e00640 1 --2- 172.21.15.119:0/2201605009 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe85c06b500 0x7fe85c06d9c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe870033000 tx=0x7fe87002e000 comp rx=0 tx=0).stop 2024-04-25T10:12:29.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.739+0000 7fe884e00640 1 -- 172.21.15.119:0/2201605009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe880151690 msgr2=0x7fe880151b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.739+0000 7fe884e00640 1 --2- 172.21.15.119:0/2201605009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe880151690 0x7fe880151b40 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe8800b4430 tx=0x7fe87800b0d0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.739+0000 7fe884e00640 1 -- 172.21.15.119:0/2201605009 shutdown_connections 2024-04-25T10:12:29.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.739+0000 7fe884e00640 1 --2- 172.21.15.119:0/2201605009 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe85c06b500 0x7fe85c06d9c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.739+0000 7fe884e00640 1 --2- 172.21.15.119:0/2201605009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe880151690 0x7fe880151b40 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.739+0000 7fe884e00640 1 --2- 172.21.15.119:0/2201605009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe880160f00 0x7fe8800b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.739+0000 7fe884e00640 1 --2- 172.21.15.119:0/2201605009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe8640039b0 0x7fe88015e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.739+0000 7fe884e00640 1 -- 172.21.15.119:0/2201605009 >> 172.21.15.119:0/2201605009 conn(0x7fe88006f430 msgr2=0x7fe8800b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.740+0000 7fe884e00640 1 -- 172.21.15.119:0/2201605009 shutdown_connections 2024-04-25T10:12:29.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.740+0000 7fe884e00640 1 -- 172.21.15.119:0/2201605009 wait complete. 2024-04-25T10:12:29.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:29.761 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:12:29.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.954+0000 7f386be00640 1 Processor -- start 2024-04-25T10:12:29.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.954+0000 7f386be00640 1 -- start start 2024-04-25T10:12:29.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.955+0000 7f386be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f386c15d980 0x7f386c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.955+0000 7f386be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f386c0b2440 0x7f386c0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.955+0000 7f386be00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386c0b6ac0 con 0x7f386c0b2a00 2024-04-25T10:12:29.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.955+0000 7f386be00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386c0cefe0 con 0x7f386c15d980 2024-04-25T10:12:29.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.955+0000 7f386be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0b70d0 con 0x7f386c0b2440 2024-04-25T10:12:29.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.955+0000 7f386aa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f386c15d980 0x7f386c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.955+0000 7f386aa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f386c15d980 0x7f386c0ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51310/0 (socket says 172.21.15.119:51310) 2024-04-25T10:12:29.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.955+0000 7f386aa00640 1 -- 172.21.15.119:0/2173859185 learned_addr learned my addr 172.21.15.119:0/2173859185 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:29.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.956+0000 7f386b400640 1 --2- 172.21.15.119:0/2173859185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f386c0b2440 0x7f386c0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.956+0000 7f386aa00640 1 -- 172.21.15.119:0/2173859185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f386c0b2440 msgr2=0x7f386c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.956+0000 7f386aa00640 1 --2- 172.21.15.119:0/2173859185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f386c0b2440 0x7f386c0ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.956+0000 7f386aa00640 1 -- 172.21.15.119:0/2173859185 >> v1:172.21.15.132:6789/0 conn(0x7f386c0b2a00 legacy=0x7f386c0b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.956+0000 7f386aa00640 1 -- 172.21.15.119:0/2173859185 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c0b94f0 con 0x7f386c15d980 2024-04-25T10:12:29.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.957+0000 7f386aa00640 1 --2- 172.21.15.119:0/2173859185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f386c15d980 0x7f386c0ce560 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f385c009870 tx=0x7f385c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d8e2f7f6db287 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.957+0000 7f386a000640 1 -- 172.21.15.119:0/2173859185 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c037070 con 0x7f386c15d980 2024-04-25T10:12:29.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.958+0000 7f386a000640 1 -- 172.21.15.119:0/2173859185 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c032440 con 0x7f386c15d980 2024-04-25T10:12:29.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.958+0000 7f386a000640 1 -- 172.21.15.119:0/2173859185 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c032810 con 0x7f386c15d980 2024-04-25T10:12:29.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.958+0000 7f386be00640 1 -- 172.21.15.119:0/2173859185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f386c15d980 msgr2=0x7f386c0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.958+0000 7f386be00640 1 --2- 172.21.15.119:0/2173859185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f386c15d980 0x7f386c0ce560 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f385c009870 tx=0x7f385c02a180 comp rx=0 tx=0).stop 2024-04-25T10:12:29.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.958+0000 7f386be00640 1 -- 172.21.15.119:0/2173859185 shutdown_connections 2024-04-25T10:12:29.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.958+0000 7f386be00640 1 --2- 172.21.15.119:0/2173859185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f386c0b2440 0x7f386c0ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.958+0000 7f386be00640 1 --2- 172.21.15.119:0/2173859185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f386c15d980 0x7f386c0ce560 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.958+0000 7f386be00640 1 -- 172.21.15.119:0/2173859185 >> 172.21.15.119:0/2173859185 conn(0x7f386c0726c0 msgr2=0x7f386c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.958+0000 7f386be00640 1 -- 172.21.15.119:0/2173859185 shutdown_connections 2024-04-25T10:12:29.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.959+0000 7f386be00640 1 -- 172.21.15.119:0/2173859185 wait complete. 2024-04-25T10:12:29.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.960+0000 7f386be00640 1 Processor -- start 2024-04-25T10:12:29.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.960+0000 7f386be00640 1 -- start start 2024-04-25T10:12:29.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.961+0000 7f386be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f386c0b2440 0x7f386c152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.961+0000 7f386be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f386c0b2a00 0x7f386c156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.961+0000 7f386be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f386c15d980 0x7f386c156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.961+0000 7f386be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c15dd60 con 0x7f386c15d980 2024-04-25T10:12:29.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.961+0000 7f386be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0b97d0 con 0x7f386c0b2440 2024-04-25T10:12:29.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.961+0000 7f386be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0b9110 con 0x7f386c0b2a00 2024-04-25T10:12:29.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.962+0000 7f386b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f386c15d980 0x7f386c156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.962+0000 7f386b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f386c15d980 0x7f386c156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51316/0 (socket says 172.21.15.119:51316) 2024-04-25T10:12:29.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.962+0000 7f386b400640 1 -- 172.21.15.119:0/3482663098 learned_addr learned my addr 172.21.15.119:0/3482663098 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:29.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.962+0000 7f386aa00640 1 --2- 172.21.15.119:0/3482663098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f386c0b2440 0x7f386c152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.962+0000 7f3862000640 1 --2- 172.21.15.119:0/3482663098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f386c0b2a00 0x7f386c156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.962+0000 7f386b400640 1 -- 172.21.15.119:0/3482663098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f386c0b2a00 msgr2=0x7f386c156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.962+0000 7f386b400640 1 --2- 172.21.15.119:0/3482663098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f386c0b2a00 0x7f386c156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.962+0000 7f386b400640 1 -- 172.21.15.119:0/3482663098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f386c0b2440 msgr2=0x7f386c152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.962+0000 7f386b400640 1 --2- 172.21.15.119:0/3482663098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f386c0b2440 0x7f386c152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.962+0000 7f386b400640 1 -- 172.21.15.119:0/3482663098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c0cbc50 con 0x7f386c15d980 2024-04-25T10:12:29.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.963+0000 7f386b400640 1 --2- 172.21.15.119:0/3482663098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f386c15d980 0x7f386c156950 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f386400cd70 tx=0x7f386400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.963+0000 7f3863e00640 1 -- 172.21.15.119:0/3482663098 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864003b90 con 0x7f386c15d980 2024-04-25T10:12:29.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.963+0000 7f3863e00640 1 -- 172.21.15.119:0/3482663098 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3864003d30 con 0x7f386c15d980 2024-04-25T10:12:29.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.963+0000 7f386be00640 1 -- 172.21.15.119:0/3482663098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c15a090 con 0x7f386c15d980 2024-04-25T10:12:29.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.963+0000 7f3863e00640 1 -- 172.21.15.119:0/3482663098 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864004240 con 0x7f386c15d980 2024-04-25T10:12:29.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.963+0000 7f386be00640 1 -- 172.21.15.119:0/3482663098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c0cbf10 con 0x7f386c15d980 2024-04-25T10:12:29.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.965+0000 7f3863e00640 1 -- 172.21.15.119:0/3482663098 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38640043e0 con 0x7f386c15d980 2024-04-25T10:12:29.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.966+0000 7f3863e00640 1 --2- 172.21.15.119:0/3482663098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f384806b500 0x7f384806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.966+0000 7f3863e00640 1 -- 172.21.15.119:0/3482663098 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f386408adf0 con 0x7f386c15d980 2024-04-25T10:12:29.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.967+0000 7f386aa00640 1 --2- 172.21.15.119:0/3482663098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f384806b500 0x7f384806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.967+0000 7f3874c39740 1 Processor -- start 2024-04-25T10:12:29.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.967+0000 7f3874c39740 1 -- start start 2024-04-25T10:12:29.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.967+0000 7f386aa00640 1 --2- 172.21.15.119:0/3482663098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f384806b500 0x7f384806d9c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f385c034000 tx=0x7f385c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.968+0000 7f3874c39740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dbe2d70 0x56298dbe4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.968+0000 7f3874c39740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dc1f440 0x56298dbe3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.968+0000 7f3874c39740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dc0dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.968+0000 7f3874c39740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56298db46c70 con 0x56298dc0d830 2024-04-25T10:12:29.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.968+0000 7f3874c39740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56298daefcf0 con 0x56298dc1f440 2024-04-25T10:12:29.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.968+0000 7f3874c39740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56298db42d10 con 0x56298dbe2d70 2024-04-25T10:12:29.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.968+0000 7f386b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dc0dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.968+0000 7f386b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dc0dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51326/0 (socket says 172.21.15.119:51326) 2024-04-25T10:12:29.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.968+0000 7f386b400640 1 -- 172.21.15.119:0/2025274730 learned_addr learned my addr 172.21.15.119:0/2025274730 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:29.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.969+0000 7f386b400640 1 -- 172.21.15.119:0/2025274730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dbe2d70 msgr2=0x56298dbe4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.969+0000 7f386aa00640 1 --2- 172.21.15.119:0/2025274730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dbe2d70 0x56298dbe4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.969+0000 7f3862000640 1 --2- 172.21.15.119:0/2025274730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dc1f440 0x56298dbe3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.969+0000 7f386b400640 1 --2- 172.21.15.119:0/2025274730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dbe2d70 0x56298dbe4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.969+0000 7f386b400640 1 -- 172.21.15.119:0/2025274730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dc1f440 msgr2=0x56298dbe3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.969+0000 7f386b400640 1 --2- 172.21.15.119:0/2025274730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dc1f440 0x56298dbe3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.969+0000 7f386b400640 1 -- 172.21.15.119:0/2025274730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56298d9257d0 con 0x56298dc0d830 2024-04-25T10:12:29.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.969+0000 7f386b400640 1 --2- 172.21.15.119:0/2025274730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dc0dcb0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f38640558a0 tx=0x7f3864058e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a602fcd88442978f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.969+0000 7f385b400640 1 -- 172.21.15.119:0/2025274730 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386405d590 con 0x56298dc0d830 2024-04-25T10:12:29.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.969+0000 7f385b400640 1 -- 172.21.15.119:0/2025274730 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3864087070 con 0x56298dc0d830 2024-04-25T10:12:29.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.969+0000 7f385b400640 1 -- 172.21.15.119:0/2025274730 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864061b50 con 0x56298dc0d830 2024-04-25T10:12:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.970+0000 7f3874c39740 1 -- 172.21.15.119:0/2025274730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 msgr2=0x56298dc0dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.970+0000 7f3874c39740 1 --2- 172.21.15.119:0/2025274730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dc0dcb0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f38640558a0 tx=0x7f3864058e60 comp rx=0 tx=0).stop 2024-04-25T10:12:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.970+0000 7f3874c39740 1 -- 172.21.15.119:0/2025274730 shutdown_connections 2024-04-25T10:12:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.970+0000 7f3874c39740 1 --2- 172.21.15.119:0/2025274730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dc0dcb0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.970+0000 7f3874c39740 1 --2- 172.21.15.119:0/2025274730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dc1f440 0x56298dbe3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.970+0000 7f3874c39740 1 --2- 172.21.15.119:0/2025274730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dbe2d70 0x56298dbe4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.970+0000 7f3874c39740 1 -- 172.21.15.119:0/2025274730 >> 172.21.15.119:0/2025274730 conn(0x56298dbaaa30 msgr2=0x56298dc28830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.970+0000 7f3874c39740 1 -- 172.21.15.119:0/2025274730 shutdown_connections 2024-04-25T10:12:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.970+0000 7f3874c39740 1 -- 172.21.15.119:0/2025274730 wait complete. 2024-04-25T10:12:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.971+0000 7f3874c39740 1 -- start start 2024-04-25T10:12:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.971+0000 7f3874c39740 1 Processor -- start 2024-04-25T10:12:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.972+0000 7f3874c39740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dbe2d70 0x56298dbf91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.972+0000 7f3874c39740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dbe1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.972+0000 7f3874c39740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 0x56298dbe2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.972+0000 7f3874c39740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56298dae53a0 con 0x56298dc0d830 2024-04-25T10:12:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.972+0000 7f3874c39740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56298dc0b380 con 0x56298dbe2d70 2024-04-25T10:12:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.972+0000 7f3874c39740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56298dbf9cb0 con 0x56298dc1f440 2024-04-25T10:12:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.972+0000 7f3874c39740 10 client.0 _ll_register_callbacks cb 0x56298db02b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.972+0000 7f3862000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dbe1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.972+0000 7f3862000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dbe1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51328/0 (socket says 172.21.15.119:51328) 2024-04-25T10:12:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.972+0000 7f3862000640 1 -- 172.21.15.119:0/1349248871 learned_addr learned my addr 172.21.15.119:0/1349248871 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.973+0000 7f386b400640 1 --2- 172.21.15.119:0/1349248871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 0x56298dbe2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.973+0000 7f386aa00640 1 --2- 172.21.15.119:0/1349248871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dbe2d70 0x56298dbf91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.973+0000 7f3862000640 1 -- 172.21.15.119:0/1349248871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 msgr2=0x56298dbe2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.973+0000 7f3862000640 1 --2- 172.21.15.119:0/1349248871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 0x56298dbe2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.973+0000 7f3862000640 1 -- 172.21.15.119:0/1349248871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dbe2d70 msgr2=0x56298dbf91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.973+0000 7f3862000640 1 --2- 172.21.15.119:0/1349248871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dbe2d70 0x56298dbf91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.973+0000 7f3862000640 1 -- 172.21.15.119:0/1349248871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56298d85b3f0 con 0x56298dc0d830 2024-04-25T10:12:29.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.973+0000 7f3862000640 1 --2- 172.21.15.119:0/1349248871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dbe1d00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3854005460 tx=0x7f385400f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1b5d1b8c8f5f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.973+0000 7f384fe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:12:29.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.973+0000 7f384fe00640 1 -- 172.21.15.119:0/1349248871 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854010040 con 0x56298dc0d830 2024-04-25T10:12:29.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.973+0000 7f384fe00640 1 -- 172.21.15.119:0/1349248871 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f385400fdd0 con 0x56298dc0d830 2024-04-25T10:12:29.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.973+0000 7f3874c39740 1 -- 172.21.15.119:0/1349248871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3864087070 con 0x56298dc0d830 2024-04-25T10:12:29.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.974+0000 7f384fe00640 1 -- 172.21.15.119:0/1349248871 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3854020050 con 0x56298dc0d830 2024-04-25T10:12:29.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.974+0000 7f3874c39740 10 client.6906 fetch_fsmap learned FSMap version 51 2024-04-25T10:12:29.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.974+0000 7f3874c39740 1 -- 172.21.15.119:0/1349248871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x56298d81b0f0 con 0x56298dc0d830 2024-04-25T10:12:29.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.974+0000 7f384fe00640 1 -- 172.21.15.119:0/1349248871 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3864087070 con 0x56298dc0d830 2024-04-25T10:12:29.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.974+0000 7f384fe00640 1 -- 172.21.15.119:0/1349248871 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3854023020 con 0x56298dc0d830 2024-04-25T10:12:29.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.974+0000 7f3874c39740 10 client.6906 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:12:29.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.974+0000 7f3874c39740 10 client.6906 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:12:29.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.974+0000 7f3874c39740 20 client.6906 populate_metadata read hostname 'smithi119' 2024-04-25T10:12:29.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.974+0000 7f3874c39740 1 --2- 172.21.15.119:0/1349248871 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x56298dc09b60 0x56298dc09f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.974+0000 7f3874c39740 4 client.6906 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:29.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.974+0000 7f3874c39740 1 -- 172.21.15.119:0/1349248871 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56298db541b0 con 0x56298dc09b60 2024-04-25T10:12:29.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.974+0000 7f386aa00640 1 --2- 172.21.15.119:0/1349248871 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x56298dc09b60 0x56298dc09f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.975+0000 7f386aa00640 1 --2- 172.21.15.119:0/1349248871 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x56298dc09b60 0x56298dc09f80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab140951b49222bb server_cookie=8fce302a1441c931 in_seq=0 out_seq=0 2024-04-25T10:12:29.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.975+0000 7f384fe00640 10 client.6906 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:29.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.975+0000 7f384fe00640 1 -- 172.21.15.119:0/1349248871 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56298db541b0 con 0x56298dc09b60 2024-04-25T10:12:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.975+0000 7f384fe00640 10 client.6906 handle_command_reply: tid=0 2024-04-25T10:12:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.975+0000 7f3874c39740 1 client.6906 shutdown 2024-04-25T10:12:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.976+0000 7f3874c39740 1 -- 172.21.15.119:0/1349248871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 msgr2=0x56298dbe1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.976+0000 7f3874c39740 1 --2- 172.21.15.119:0/1349248871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dbe1d00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3854005460 tx=0x7f385400f540 comp rx=0 tx=0).stop 2024-04-25T10:12:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.976+0000 7f3874c39740 1 -- 172.21.15.119:0/1349248871 shutdown_connections 2024-04-25T10:12:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.976+0000 7f3874c39740 1 --2- 172.21.15.119:0/1349248871 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x56298dc09b60 0x56298dc09f80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.976+0000 7f3874c39740 1 --2- 172.21.15.119:0/1349248871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 0x56298dbe2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.976+0000 7f3874c39740 1 --2- 172.21.15.119:0/1349248871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dbe1d00 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.976+0000 7f3874c39740 1 --2- 172.21.15.119:0/1349248871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dbe2d70 0x56298dbf91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.976+0000 7f384fe00640 0 client.6906 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.976+0000 7f3874c39740 1 -- 172.21.15.119:0/1349248871 >> 172.21.15.119:0/1349248871 conn(0x56298dbaaa30 msgr2=0x56298dc0b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.976+0000 7f3874c39740 1 -- 172.21.15.119:0/1349248871 shutdown_connections 2024-04-25T10:12:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.976+0000 7f3874c39740 1 -- 172.21.15.119:0/1349248871 wait complete. 2024-04-25T10:12:29.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.976+0000 7f3874c39740 20 client.6906 trim_cache size 0 max 16384 2024-04-25T10:12:29.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.992+0000 7f3874c39740 1 Processor -- start 2024-04-25T10:12:29.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.992+0000 7f3874c39740 1 -- start start 2024-04-25T10:12:30.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.993+0000 7f3874c39740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dbe2d70 0x56298dc09f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.993+0000 7f3874c39740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dc2aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.993+0000 7f3874c39740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 0x56298dbe3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.993+0000 7f3874c39740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c200070 con 0x56298dc0d830 2024-04-25T10:12:30.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.993+0000 7f3874c39740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c1ff350 con 0x56298dbe2d70 2024-04-25T10:12:30.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.993+0000 7f3874c39740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c15fdc0 con 0x56298dc1f440 2024-04-25T10:12:30.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.994+0000 7f3862000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dc2aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.994+0000 7f3862000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dc2aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51344/0 (socket says 172.21.15.119:51344) 2024-04-25T10:12:30.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.994+0000 7f3862000640 1 -- 172.21.15.119:0/269330874 learned_addr learned my addr 172.21.15.119:0/269330874 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:30.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.994+0000 7f386b400640 1 --2- 172.21.15.119:0/269330874 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 0x56298dbe3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.994+0000 7f386aa00640 1 --2- 172.21.15.119:0/269330874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dbe2d70 0x56298dc09f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.994+0000 7f3862000640 1 -- 172.21.15.119:0/269330874 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 msgr2=0x56298dbe3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.994+0000 7f3862000640 1 --2- 172.21.15.119:0/269330874 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 0x56298dbe3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.994+0000 7f3862000640 1 -- 172.21.15.119:0/269330874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dbe2d70 msgr2=0x56298dc09f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.994+0000 7f3862000640 1 --2- 172.21.15.119:0/269330874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dbe2d70 0x56298dc09f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.994+0000 7f3862000640 1 -- 172.21.15.119:0/269330874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56298dbf78f0 con 0x56298dc0d830 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.994+0000 7f3862000640 1 --2- 172.21.15.119:0/269330874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dc2aec0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f385400fed0 tx=0x7f3854013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef17adaf3abc88f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.994+0000 7f385aa00640 1 -- 172.21.15.119:0/269330874 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854007040 con 0x56298dc0d830 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.995+0000 7f385aa00640 1 -- 172.21.15.119:0/269330874 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3854028020 con 0x56298dc0d830 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.995+0000 7f385aa00640 1 -- 172.21.15.119:0/269330874 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38540071e0 con 0x56298dc0d830 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.995+0000 7f3874c39740 1 -- 172.21.15.119:0/269330874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 msgr2=0x56298dc2aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.995+0000 7f3874c39740 1 --2- 172.21.15.119:0/269330874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dc2aec0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f385400fed0 tx=0x7f3854013000 comp rx=0 tx=0).stop 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.995+0000 7f3874c39740 1 -- 172.21.15.119:0/269330874 shutdown_connections 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.995+0000 7f3874c39740 1 --2- 172.21.15.119:0/269330874 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 0x56298dbe3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.995+0000 7f3874c39740 1 --2- 172.21.15.119:0/269330874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dc0d830 0x56298dc2aec0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.995+0000 7f3874c39740 1 --2- 172.21.15.119:0/269330874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dbe2d70 0x56298dc09f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.995+0000 7f3874c39740 1 -- 172.21.15.119:0/269330874 >> 172.21.15.119:0/269330874 conn(0x56298dbaaa30 msgr2=0x56298dc4aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.995+0000 7f3874c39740 1 -- 172.21.15.119:0/269330874 shutdown_connections 2024-04-25T10:12:30.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.995+0000 7f3874c39740 1 -- 172.21.15.119:0/269330874 wait complete. 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.996+0000 7f3874c39740 1 -- start start 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.996+0000 7f3874c39740 1 Processor -- start 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.997+0000 7f3874c39740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dbe2d70 0x56298dc92010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.997+0000 7f3874c39740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dc0d830 0x56298dc2a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.997+0000 7f3874c39740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 0x56298dc95ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.997+0000 7f3874c39740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56298db79ca0 con 0x56298dbe2d70 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.997+0000 7f3874c39740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c1603c0 con 0x56298dc0d830 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.997+0000 7f3874c39740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56298dc11240 con 0x56298dc1f440 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.997+0000 7f3874c39740 10 client.0 _ll_register_callbacks cb 0x7f3854007cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.997+0000 7f386aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dbe2d70 0x56298dc92010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.998+0000 7f386aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dbe2d70 0x56298dc92010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51348/0 (socket says 172.21.15.119:51348) 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.998+0000 7f386aa00640 1 -- 172.21.15.119:0/3208730584 learned_addr learned my addr 172.21.15.119:0/3208730584 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.998+0000 7f386b400640 1 --2- 172.21.15.119:0/3208730584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 0x56298dc95ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.998+0000 7f386aa00640 1 -- 172.21.15.119:0/3208730584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 msgr2=0x56298dc95ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.998+0000 7f3862000640 1 --2- 172.21.15.119:0/3208730584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dc0d830 0x56298dc2a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.998+0000 7f386aa00640 1 --2- 172.21.15.119:0/3208730584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 0x56298dc95ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.998+0000 7f386aa00640 1 -- 172.21.15.119:0/3208730584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dc0d830 msgr2=0x56298dc2a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.998+0000 7f386aa00640 1 --2- 172.21.15.119:0/3208730584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dc0d830 0x56298dc2a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.998+0000 7f386aa00640 1 -- 172.21.15.119:0/3208730584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56298dc4b700 con 0x56298dbe2d70 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.998+0000 7f386aa00640 1 --2- 172.21.15.119:0/3208730584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dbe2d70 0x56298dc92010 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f385c02a1c0 tx=0x7f385c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f10a52d9f57c1d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.998+0000 7f384fe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.998+0000 7f384fe00640 1 -- 172.21.15.119:0/3208730584 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c050060 con 0x56298dbe2d70 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.999+0000 7f384fe00640 1 -- 172.21.15.119:0/3208730584 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f385c04b5d0 con 0x56298dbe2d70 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.999+0000 7f3874c39740 1 -- 172.21.15.119:0/3208730584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3854028020 con 0x56298dbe2d70 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.999+0000 7f384fe00640 1 -- 172.21.15.119:0/3208730584 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f385c056050 con 0x56298dbe2d70 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.999+0000 7f3874c39740 10 client.6912 fetch_fsmap learned FSMap version 51 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.999+0000 7f3874c39740 1 -- 172.21.15.119:0/3208730584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x56298dbf6340 con 0x56298dbe2d70 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.999+0000 7f384fe00640 1 -- 172.21.15.119:0/3208730584 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3854028020 con 0x56298dbe2d70 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.999+0000 7f384fe00640 1 -- 172.21.15.119:0/3208730584 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f385c059020 con 0x56298dbe2d70 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.999+0000 7f3874c39740 10 client.6912 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.999+0000 7f3874c39740 10 client.6912 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.999+0000 7f3874c39740 20 client.6912 populate_metadata read hostname 'smithi119' 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.999+0000 7f3874c39740 1 --2- 172.21.15.119:0/3208730584 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x56298dc960e0 0x56298dc09f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.999+0000 7f3874c39740 4 client.6912 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.999+0000 7f3874c39740 1 -- 172.21.15.119:0/3208730584 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56298db4e500 con 0x56298dc960e0 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:29.999+0000 7f3862000640 1 --2- 172.21.15.119:0/3208730584 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x56298dc960e0 0x56298dc09f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.000+0000 7f3862000640 1 --2- 172.21.15.119:0/3208730584 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x56298dc960e0 0x56298dc09f40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6477dcfde0bd4913 server_cookie=1b54e7ad59191e57 in_seq=0 out_seq=0 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.000+0000 7f384fe00640 10 client.6912 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.000+0000 7f384fe00640 1 -- 172.21.15.119:0/3208730584 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x56298db4e500 con 0x56298dc960e0 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.000+0000 7f384fe00640 10 client.6912 handle_command_reply: tid=0 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.000+0000 7f3874c39740 1 client.6912 shutdown 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.001+0000 7f3874c39740 1 -- 172.21.15.119:0/3208730584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dbe2d70 msgr2=0x56298dc92010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.001+0000 7f3874c39740 1 --2- 172.21.15.119:0/3208730584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dbe2d70 0x56298dc92010 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f385c02a1c0 tx=0x7f385c02c070 comp rx=0 tx=0).stop 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.001+0000 7f3874c39740 1 -- 172.21.15.119:0/3208730584 shutdown_connections 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.001+0000 7f3874c39740 1 --2- 172.21.15.119:0/3208730584 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x56298dc960e0 0x56298dc09f40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.001+0000 7f3874c39740 1 --2- 172.21.15.119:0/3208730584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56298dc1f440 0x56298dc95ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.001+0000 7f3874c39740 1 --2- 172.21.15.119:0/3208730584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56298dc0d830 0x56298dc2a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.001+0000 7f3874c39740 1 --2- 172.21.15.119:0/3208730584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56298dbe2d70 0x56298dc92010 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.001+0000 7f384fe00640 0 client.6912 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.001+0000 7f3874c39740 1 -- 172.21.15.119:0/3208730584 >> 172.21.15.119:0/3208730584 conn(0x56298dbaaa30 msgr2=0x56298dc103a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.001+0000 7f3874c39740 1 -- 172.21.15.119:0/3208730584 shutdown_connections 2024-04-25T10:12:30.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.001+0000 7f3874c39740 1 -- 172.21.15.119:0/3208730584 wait complete. 2024-04-25T10:12:30.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.001+0000 7f3874c39740 20 client.6912 trim_cache size 0 max 16384 2024-04-25T10:12:30.010 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:12:30.010 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:12:30.010 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2024-04-25T10:12:30.010 INFO:teuthology.orchestra.run.smithi119.stdout: "0bc2c628-04ea-423b-baf7-d32f078fcfe6": { 2024-04-25T10:12:30.010 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/subdir", 2024-04-25T10:12:30.010 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "0bc2c628-04ea-423b-baf7-d32f078fcfe6", 2024-04-25T10:12:30.011 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,repair" 2024-04-25T10:12:30.011 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:12:30.011 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:12:30.011 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.002+0000 7f386be00640 1 -- 172.21.15.119:0/3482663098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f384806b500 msgr2=0x7f384806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.002+0000 7f386be00640 1 --2- 172.21.15.119:0/3482663098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f384806b500 0x7f384806d9c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f385c034000 tx=0x7f385c02e000 comp rx=0 tx=0).stop 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.002+0000 7f386be00640 1 -- 172.21.15.119:0/3482663098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f386c15d980 msgr2=0x7f386c156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.002+0000 7f386be00640 1 --2- 172.21.15.119:0/3482663098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f386c15d980 0x7f386c156950 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f386400cd70 tx=0x7f386400a660 comp rx=0 tx=0).stop 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.002+0000 7f386be00640 1 -- 172.21.15.119:0/3482663098 shutdown_connections 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.002+0000 7f386be00640 1 --2- 172.21.15.119:0/3482663098 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f384806b500 0x7f384806d9c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.002+0000 7f386be00640 1 --2- 172.21.15.119:0/3482663098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f386c15d980 0x7f386c156950 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.002+0000 7f386be00640 1 --2- 172.21.15.119:0/3482663098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f386c0b2a00 0x7f386c156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.002+0000 7f386be00640 1 --2- 172.21.15.119:0/3482663098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f386c0b2440 0x7f386c152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.002+0000 7f386be00640 1 -- 172.21.15.119:0/3482663098 >> 172.21.15.119:0/3482663098 conn(0x7f386c0726c0 msgr2=0x7f386c0d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.002+0000 7f386be00640 1 -- 172.21.15.119:0/3482663098 shutdown_connections 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:12:30.002+0000 7f386be00640 1 -- 172.21.15.119:0/3482663098 wait complete. 2024-04-25T10:12:30.023 INFO:tasks.cephfs.filesystem:scrub status for tag:0bc2c628-04ea-423b-baf7-d32f078fcfe6 - {'path': '/subdir', 'tag': '0bc2c628-04ea-423b-baf7-d32f078fcfe6', 'options': 'recursive,repair'} 2024-04-25T10:12:31.433 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:31.441 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:00.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:00.025 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:13:00.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.251+0000 7efd9d600640 1 Processor -- start 2024-04-25T10:13:00.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.251+0000 7efd9d600640 1 -- start start 2024-04-25T10:13:00.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.253+0000 7efd9d600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980aed40 0x7efd980cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.253+0000 7efd9d600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efd980cfa90 0x7efd980acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.253+0000 7efd9d600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9815e360 con 0x7efd9815b720 2024-04-25T10:13:00.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.253+0000 7efd9d600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9815e5d0 con 0x7efd980cfa90 2024-04-25T10:13:00.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.253+0000 7efd9d600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9815e8b0 con 0x7efd980aed40 2024-04-25T10:13:00.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.254+0000 7efd96000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980aed40 0x7efd980cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.254+0000 7efd97400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efd980cfa90 0x7efd980acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.254+0000 7efd97400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efd980cfa90 0x7efd980acfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:36486/0 (socket says 172.21.15.119:36486) 2024-04-25T10:13:00.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.254+0000 7efd97400640 1 -- 172.21.15.119:0/2484981180 learned_addr learned my addr 172.21.15.119:0/2484981180 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:00.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.254+0000 7efd96a00640 1 --1- 172.21.15.119:0/2484981180 >> v1:172.21.15.132:6789/0 conn(0x7efd9815b720 0x7efd980ae630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:48574/0 (socket says 172.21.15.119:48574) 2024-04-25T10:13:00.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.255+0000 7efd96000640 1 -- 172.21.15.119:0/2484981180 >> v2:172.21.15.132:3300/0 conn(0x7efd980cfa90 msgr2=0x7efd980acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.255+0000 7efd96000640 1 --2- 172.21.15.119:0/2484981180 >> v2:172.21.15.132:3300/0 conn(0x7efd980cfa90 0x7efd980acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.255+0000 7efd96000640 1 -- 172.21.15.119:0/2484981180 >> v1:172.21.15.132:6789/0 conn(0x7efd9815b720 legacy=0x7efd980ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.255+0000 7efd96000640 1 -- 172.21.15.119:0/2484981180 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9815eb90 con 0x7efd980aed40 2024-04-25T10:13:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.256+0000 7efd96000640 1 --2- 172.21.15.119:0/2484981180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980aed40 0x7efd980cf550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efd840097a0 tx=0x7efd8402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=964f5a658e347790 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.256+0000 7efd95600640 1 -- 172.21.15.119:0/2484981180 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84037070 con 0x7efd980aed40 2024-04-25T10:13:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.257+0000 7efd95600640 1 -- 172.21.15.119:0/2484981180 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd84032440 con 0x7efd980aed40 2024-04-25T10:13:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.257+0000 7efd95600640 1 -- 172.21.15.119:0/2484981180 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84032810 con 0x7efd980aed40 2024-04-25T10:13:00.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.257+0000 7efd9d600640 1 -- 172.21.15.119:0/2484981180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980aed40 msgr2=0x7efd980cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.257+0000 7efd9d600640 1 --2- 172.21.15.119:0/2484981180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980aed40 0x7efd980cf550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efd840097a0 tx=0x7efd8402a180 comp rx=0 tx=0).stop 2024-04-25T10:13:00.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.257+0000 7efd9d600640 1 -- 172.21.15.119:0/2484981180 shutdown_connections 2024-04-25T10:13:00.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.257+0000 7efd9d600640 1 --2- 172.21.15.119:0/2484981180 >> v2:172.21.15.132:3300/0 conn(0x7efd980cfa90 0x7efd980acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.257+0000 7efd9d600640 1 --2- 172.21.15.119:0/2484981180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd980aed40 0x7efd980cf550 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.257+0000 7efd9d600640 1 -- 172.21.15.119:0/2484981180 >> 172.21.15.119:0/2484981180 conn(0x7efd9806aba0 msgr2=0x7efd9806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.257+0000 7efd9d600640 1 -- 172.21.15.119:0/2484981180 shutdown_connections 2024-04-25T10:13:00.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.258+0000 7efd9d600640 1 -- 172.21.15.119:0/2484981180 wait complete. 2024-04-25T10:13:00.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.259+0000 7efd9d600640 1 Processor -- start 2024-04-25T10:13:00.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.259+0000 7efd9d600640 1 -- start start 2024-04-25T10:13:00.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.260+0000 7efd9d600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980aed40 0x7efd9814a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.260+0000 7efd9d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd980cfa90 0x7efd980cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.260+0000 7efd9d600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd9815b720 0x7efd98149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.260+0000 7efd9d600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd980cff00 con 0x7efd980cfa90 2024-04-25T10:13:00.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.260+0000 7efd9d600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9815bb00 con 0x7efd980aed40 2024-04-25T10:13:00.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.260+0000 7efd9d600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd980d0080 con 0x7efd9815b720 2024-04-25T10:13:00.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.261+0000 7efd96000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd980cfa90 0x7efd980cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.261+0000 7efd96000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd980cfa90 0x7efd980cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47106/0 (socket says 172.21.15.119:47106) 2024-04-25T10:13:00.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.261+0000 7efd96000640 1 -- 172.21.15.119:0/2351988510 learned_addr learned my addr 172.21.15.119:0/2351988510 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:00.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.261+0000 7efd96a00640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980aed40 0x7efd9814a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.261+0000 7efd97400640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd9815b720 0x7efd98149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.261+0000 7efd96000640 1 -- 172.21.15.119:0/2351988510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd9815b720 msgr2=0x7efd98149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.261+0000 7efd96000640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd9815b720 0x7efd98149b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.261+0000 7efd96000640 1 -- 172.21.15.119:0/2351988510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980aed40 msgr2=0x7efd9814a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.261+0000 7efd96000640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980aed40 0x7efd9814a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.261+0000 7efd96000640 1 -- 172.21.15.119:0/2351988510 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9814a0d0 con 0x7efd980cfa90 2024-04-25T10:13:00.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.261+0000 7efd96a00640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980aed40 0x7efd9814a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:00.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.262+0000 7efd96000640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd980cfa90 0x7efd980cdda0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7efd840098c0 tx=0x7efd840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.262+0000 7efd8ea00640 1 -- 172.21.15.119:0/2351988510 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84037040 con 0x7efd980cfa90 2024-04-25T10:13:00.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.262+0000 7efd8ea00640 1 -- 172.21.15.119:0/2351988510 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd84030500 con 0x7efd980cfa90 2024-04-25T10:13:00.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.262+0000 7efd8ea00640 1 -- 172.21.15.119:0/2351988510 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd840308d0 con 0x7efd980cfa90 2024-04-25T10:13:00.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.262+0000 7efd9d600640 1 -- 172.21.15.119:0/2351988510 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9814efe0 con 0x7efd980cfa90 2024-04-25T10:13:00.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.262+0000 7efd9d600640 1 -- 172.21.15.119:0/2351988510 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9814f5a0 con 0x7efd980cfa90 2024-04-25T10:13:00.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.264+0000 7efd8ea00640 1 -- 172.21.15.119:0/2351988510 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd84044020 con 0x7efd980cfa90 2024-04-25T10:13:00.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.265+0000 7efd8ea00640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd7806b4d0 0x7efd7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.265+0000 7efd8ea00640 1 -- 172.21.15.119:0/2351988510 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7efd840aa640 con 0x7efd980cfa90 2024-04-25T10:13:00.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.265+0000 7efd96a00640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd7806b4d0 0x7efd7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.266+0000 7efd96a00640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd7806b4d0 0x7efd7806d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efd88000f60 tx=0x7efd88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.266+0000 7efda047e740 1 Processor -- start 2024-04-25T10:13:00.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.266+0000 7efda047e740 1 -- start start 2024-04-25T10:13:00.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.267+0000 7efda047e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b0cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.267+0000 7efda047e740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b47280 0x5632d6b0af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.267+0000 7efda047e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b35670 0x5632d6b35af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.267+0000 7efda047e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d6a6ec70 con 0x5632d6b0abb0 2024-04-25T10:13:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.267+0000 7efda047e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d6a17cf0 con 0x5632d6b35670 2024-04-25T10:13:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.267+0000 7efda047e740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d6a6ad10 con 0x5632d6b47280 2024-04-25T10:13:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.267+0000 7efd96a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b0cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.268+0000 7efd96a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b0cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47116/0 (socket says 172.21.15.119:47116) 2024-04-25T10:13:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.268+0000 7efd96a00640 1 -- 172.21.15.119:0/2969129251 learned_addr learned my addr 172.21.15.119:0/2969129251 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.268+0000 7efd96a00640 1 -- 172.21.15.119:0/2969129251 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b47280 msgr2=0x5632d6b0af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.268+0000 7efd96a00640 1 --2- 172.21.15.119:0/2969129251 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b47280 0x5632d6b0af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.268+0000 7efd96a00640 1 -- 172.21.15.119:0/2969129251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b35670 msgr2=0x5632d6b35af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.268+0000 7efd96a00640 1 --2- 172.21.15.119:0/2969129251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b35670 0x5632d6b35af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.268+0000 7efd96a00640 1 -- 172.21.15.119:0/2969129251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632d684d7d0 con 0x5632d6b0abb0 2024-04-25T10:13:00.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.268+0000 7efd96a00640 1 --2- 172.21.15.119:0/2969129251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b0cd10 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7efd88007390 tx=0x7efd88010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9484abbcd23dbf85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.268+0000 7efd83000640 1 -- 172.21.15.119:0/2969129251 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8800d970 con 0x5632d6b0abb0 2024-04-25T10:13:00.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.269+0000 7efd83000640 1 -- 172.21.15.119:0/2969129251 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8800db10 con 0x5632d6b0abb0 2024-04-25T10:13:00.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.269+0000 7efd83000640 1 -- 172.21.15.119:0/2969129251 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88020450 con 0x5632d6b0abb0 2024-04-25T10:13:00.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.269+0000 7efda047e740 1 -- 172.21.15.119:0/2969129251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 msgr2=0x5632d6b0cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.269+0000 7efda047e740 1 --2- 172.21.15.119:0/2969129251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b0cd10 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7efd88007390 tx=0x7efd88010000 comp rx=0 tx=0).stop 2024-04-25T10:13:00.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.269+0000 7efda047e740 1 -- 172.21.15.119:0/2969129251 shutdown_connections 2024-04-25T10:13:00.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.269+0000 7efda047e740 1 --2- 172.21.15.119:0/2969129251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b35670 0x5632d6b35af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.269+0000 7efda047e740 1 --2- 172.21.15.119:0/2969129251 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b47280 0x5632d6b0af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.269+0000 7efda047e740 1 --2- 172.21.15.119:0/2969129251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b0cd10 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.269+0000 7efda047e740 1 -- 172.21.15.119:0/2969129251 >> 172.21.15.119:0/2969129251 conn(0x5632d6ad2a30 msgr2=0x5632d6b50670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.269+0000 7efda047e740 1 -- 172.21.15.119:0/2969129251 shutdown_connections 2024-04-25T10:13:00.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.269+0000 7efda047e740 1 -- 172.21.15.119:0/2969129251 wait complete. 2024-04-25T10:13:00.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.270+0000 7efda047e740 1 -- start start 2024-04-25T10:13:00.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.270+0000 7efda047e740 1 Processor -- start 2024-04-25T10:13:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.271+0000 7efda047e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b4ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.271+0000 7efda047e740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b35670 0x5632d6b230e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.271+0000 7efda047e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 0x5632d6b23620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.271+0000 7efda047e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d6a0d3a0 con 0x5632d6b0abb0 2024-04-25T10:13:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.271+0000 7efda047e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d6b331c0 con 0x5632d6b47280 2024-04-25T10:13:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.271+0000 7efda047e740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d6b481d0 con 0x5632d6b35670 2024-04-25T10:13:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.271+0000 7efda047e740 10 client.0 _ll_register_callbacks cb 0x5632d6a2ab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.272+0000 7efd96a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b4ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.272+0000 7efd96a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b4ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47122/0 (socket says 172.21.15.119:47122) 2024-04-25T10:13:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.272+0000 7efd96a00640 1 -- 172.21.15.119:0/62049689 learned_addr learned my addr 172.21.15.119:0/62049689 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.272+0000 7efd97400640 1 --2- 172.21.15.119:0/62049689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 0x5632d6b23620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.272+0000 7efd96000640 1 --2- 172.21.15.119:0/62049689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b35670 0x5632d6b230e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.272+0000 7efd96a00640 1 -- 172.21.15.119:0/62049689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b35670 msgr2=0x5632d6b230e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.272+0000 7efd96a00640 1 --2- 172.21.15.119:0/62049689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b35670 0x5632d6b230e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.272+0000 7efd96a00640 1 -- 172.21.15.119:0/62049689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 msgr2=0x5632d6b23620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.272+0000 7efd96a00640 1 --2- 172.21.15.119:0/62049689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 0x5632d6b23620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.272+0000 7efd96a00640 1 -- 172.21.15.119:0/62049689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632d67833f0 con 0x5632d6b0abb0 2024-04-25T10:13:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.272+0000 7efd97400640 1 --2- 172.21.15.119:0/62049689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 0x5632d6b23620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.272+0000 7efd96a00640 1 --2- 172.21.15.119:0/62049689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b4ab80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7efd88007e90 tx=0x7efd88000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f01504bc2964b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.272+0000 7efd77400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:13:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.273+0000 7efd77400640 1 -- 172.21.15.119:0/62049689 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88030060 con 0x5632d6b0abb0 2024-04-25T10:13:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.273+0000 7efd77400640 1 -- 172.21.15.119:0/62049689 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7efd880225f0 con 0x5632d6b0abb0 2024-04-25T10:13:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.273+0000 7efda047e740 1 -- 172.21.15.119:0/62049689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd8800db10 con 0x5632d6b0abb0 2024-04-25T10:13:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.273+0000 7efd77400640 1 -- 172.21.15.119:0/62049689 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd880227f0 con 0x5632d6b0abb0 2024-04-25T10:13:00.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.273+0000 7efda047e740 10 client.6921 fetch_fsmap learned FSMap version 52 2024-04-25T10:13:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.273+0000 7efda047e740 1 -- 172.21.15.119:0/62049689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x5632d67430f0 con 0x5632d6b0abb0 2024-04-25T10:13:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.273+0000 7efd77400640 1 -- 172.21.15.119:0/62049689 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8800db10 con 0x5632d6b0abb0 2024-04-25T10:13:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.273+0000 7efd77400640 1 -- 172.21.15.119:0/62049689 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efd88039080 con 0x5632d6b0abb0 2024-04-25T10:13:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.273+0000 7efda047e740 10 client.6921 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T10:13:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.273+0000 7efda047e740 10 client.6921 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:13:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.273+0000 7efda047e740 20 client.6921 populate_metadata read hostname 'smithi119' 2024-04-25T10:13:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.273+0000 7efda047e740 1 --2- 172.21.15.119:0/62049689 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5632d6b47a60 0x5632d6b22960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.273+0000 7efda047e740 4 client.6921 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.273+0000 7efda047e740 1 -- 172.21.15.119:0/62049689 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632d6a7c1b0 con 0x5632d6b47a60 2024-04-25T10:13:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.274+0000 7efd96000640 1 --2- 172.21.15.119:0/62049689 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5632d6b47a60 0x5632d6b22960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.274+0000 7efd96000640 1 --2- 172.21.15.119:0/62049689 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5632d6b47a60 0x5632d6b22960 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5b746dbf05b99a server_cookie=28aa308b5809d630 in_seq=0 out_seq=0 2024-04-25T10:13:00.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.274+0000 7efd77400640 10 client.6921 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:13:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efd77400640 1 -- 172.21.15.119:0/62049689 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632d6a7c1b0 con 0x5632d6b47a60 2024-04-25T10:13:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efd77400640 10 client.6921 handle_command_reply: tid=0 2024-04-25T10:13:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efda047e740 1 client.6921 shutdown 2024-04-25T10:13:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efda047e740 1 -- 172.21.15.119:0/62049689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 msgr2=0x5632d6b4ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efda047e740 1 --2- 172.21.15.119:0/62049689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b4ab80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7efd88007e90 tx=0x7efd88000ed0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efda047e740 1 -- 172.21.15.119:0/62049689 shutdown_connections 2024-04-25T10:13:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efda047e740 1 --2- 172.21.15.119:0/62049689 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5632d6b47a60 0x5632d6b22960 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efda047e740 1 --2- 172.21.15.119:0/62049689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 0x5632d6b23620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efda047e740 1 --2- 172.21.15.119:0/62049689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b35670 0x5632d6b230e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efda047e740 1 --2- 172.21.15.119:0/62049689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b4ab80 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efd77400640 0 client.6921 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:13:00.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efda047e740 1 -- 172.21.15.119:0/62049689 >> 172.21.15.119:0/62049689 conn(0x5632d6ad2a30 msgr2=0x5632d6b32890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efda047e740 1 -- 172.21.15.119:0/62049689 shutdown_connections 2024-04-25T10:13:00.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efda047e740 1 -- 172.21.15.119:0/62049689 wait complete. 2024-04-25T10:13:00.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.275+0000 7efda047e740 20 client.6921 trim_cache size 0 max 16384 2024-04-25T10:13:00.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.292+0000 7efda047e740 1 Processor -- start 2024-04-25T10:13:00.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.292+0000 7efda047e740 1 -- start start 2024-04-25T10:13:00.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.293+0000 7efda047e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b34a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.293+0000 7efda047e740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b35670 0x5632d6b22fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.293+0000 7efda047e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 0x5632d6b23500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.293+0000 7efda047e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd98200170 con 0x5632d6b0abb0 2024-04-25T10:13:00.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.293+0000 7efda047e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd980ccec0 con 0x5632d6b47280 2024-04-25T10:13:00.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.293+0000 7efda047e740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd980cc2c0 con 0x5632d6b35670 2024-04-25T10:13:00.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.293+0000 7efd96a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b34a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.293+0000 7efd96a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b34a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47124/0 (socket says 172.21.15.119:47124) 2024-04-25T10:13:00.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.293+0000 7efd96a00640 1 -- 172.21.15.119:0/3206925997 learned_addr learned my addr 172.21.15.119:0/3206925997 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:00.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.293+0000 7efd97400640 1 --2- 172.21.15.119:0/3206925997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 0x5632d6b23500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.294+0000 7efd96000640 1 --2- 172.21.15.119:0/3206925997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b35670 0x5632d6b22fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.294+0000 7efd96a00640 1 -- 172.21.15.119:0/3206925997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b35670 msgr2=0x5632d6b22fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.294+0000 7efd96a00640 1 --2- 172.21.15.119:0/3206925997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b35670 0x5632d6b22fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.294+0000 7efd96a00640 1 -- 172.21.15.119:0/3206925997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 msgr2=0x5632d6b23500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.294+0000 7efd96a00640 1 --2- 172.21.15.119:0/3206925997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 0x5632d6b23500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.294+0000 7efd96a00640 1 -- 172.21.15.119:0/3206925997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632d6b07550 con 0x5632d6b0abb0 2024-04-25T10:13:00.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.294+0000 7efd96a00640 1 --2- 172.21.15.119:0/3206925997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b34a40 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7efd88022300 tx=0x7efd88024050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9865c1dc1fa0471b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.294+0000 7efd82600640 1 -- 172.21.15.119:0/3206925997 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8803c070 con 0x5632d6b0abb0 2024-04-25T10:13:00.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.294+0000 7efd82600640 1 -- 172.21.15.119:0/3206925997 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd880071e0 con 0x5632d6b0abb0 2024-04-25T10:13:00.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.294+0000 7efd82600640 1 -- 172.21.15.119:0/3206925997 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88022470 con 0x5632d6b0abb0 2024-04-25T10:13:00.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.295+0000 7efda047e740 1 -- 172.21.15.119:0/3206925997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 msgr2=0x5632d6b34a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.295+0000 7efda047e740 1 --2- 172.21.15.119:0/3206925997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b34a40 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7efd88022300 tx=0x7efd88024050 comp rx=0 tx=0).stop 2024-04-25T10:13:00.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.295+0000 7efda047e740 1 -- 172.21.15.119:0/3206925997 shutdown_connections 2024-04-25T10:13:00.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.295+0000 7efda047e740 1 --2- 172.21.15.119:0/3206925997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 0x5632d6b23500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.295+0000 7efda047e740 1 --2- 172.21.15.119:0/3206925997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b35670 0x5632d6b22fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.295+0000 7efda047e740 1 --2- 172.21.15.119:0/3206925997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b0abb0 0x5632d6b34a40 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.295+0000 7efda047e740 1 -- 172.21.15.119:0/3206925997 >> 172.21.15.119:0/3206925997 conn(0x5632d6ad2a30 msgr2=0x5632d6b04d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.295+0000 7efda047e740 1 -- 172.21.15.119:0/3206925997 shutdown_connections 2024-04-25T10:13:00.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.295+0000 7efda047e740 1 -- 172.21.15.119:0/3206925997 wait complete. 2024-04-25T10:13:00.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.296+0000 7efda047e740 1 -- start start 2024-04-25T10:13:00.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.296+0000 7efda047e740 1 Processor -- start 2024-04-25T10:13:00.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.297+0000 7efda047e740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b0abb0 0x5632d6b22fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.297+0000 7efda047e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b35670 0x5632d6b23500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.297+0000 7efda047e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 0x5632d6b1f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.297+0000 7efda047e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d6a2c040 con 0x5632d6b35670 2024-04-25T10:13:00.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.297+0000 7efda047e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd980cc8c0 con 0x5632d6b47280 2024-04-25T10:13:00.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.297+0000 7efda047e740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d6bbbfc0 con 0x5632d6b0abb0 2024-04-25T10:13:00.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.297+0000 7efda047e740 10 client.0 _ll_register_callbacks cb 0x5632d6b4a870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:00.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.297+0000 7efd96000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b35670 0x5632d6b23500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.297+0000 7efd96000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b35670 0x5632d6b23500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47132/0 (socket says 172.21.15.119:47132) 2024-04-25T10:13:00.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.297+0000 7efd96000640 1 -- 172.21.15.119:0/421748338 learned_addr learned my addr 172.21.15.119:0/421748338 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:00.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.298+0000 7efd96000640 1 -- 172.21.15.119:0/421748338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b0abb0 msgr2=0x5632d6b22fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.298+0000 7efd96000640 1 --2- 172.21.15.119:0/421748338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b0abb0 0x5632d6b22fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.298+0000 7efd96000640 1 -- 172.21.15.119:0/421748338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 msgr2=0x5632d6b1f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.298+0000 7efd96000640 1 --2- 172.21.15.119:0/421748338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 0x5632d6b1f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.298+0000 7efd96000640 1 -- 172.21.15.119:0/421748338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632d6b064a0 con 0x5632d6b35670 2024-04-25T10:13:00.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.298+0000 7efd96000640 1 --2- 172.21.15.119:0/421748338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b35670 0x5632d6b23500 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7efd84078900 tx=0x7efd840aa040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bde2966bd8f474 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.298+0000 7efd77400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:13:00.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.298+0000 7efd77400640 1 -- 172.21.15.119:0/421748338 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd840813a0 con 0x5632d6b35670 2024-04-25T10:13:00.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.298+0000 7efd77400640 1 -- 172.21.15.119:0/421748338 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7efd84075270 con 0x5632d6b35670 2024-04-25T10:13:00.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.298+0000 7efda047e740 1 -- 172.21.15.119:0/421748338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd880071e0 con 0x5632d6b35670 2024-04-25T10:13:00.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.298+0000 7efd77400640 1 -- 172.21.15.119:0/421748338 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd84078e60 con 0x5632d6b35670 2024-04-25T10:13:00.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.298+0000 7efda047e740 10 client.6927 fetch_fsmap learned FSMap version 52 2024-04-25T10:13:00.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.298+0000 7efda047e740 1 -- 172.21.15.119:0/421748338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x5632d6b4a9b0 con 0x5632d6b35670 2024-04-25T10:13:00.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.299+0000 7efd77400640 1 -- 172.21.15.119:0/421748338 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd880071e0 con 0x5632d6b35670 2024-04-25T10:13:00.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.299+0000 7efd77400640 1 -- 172.21.15.119:0/421748338 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efd8403a070 con 0x5632d6b35670 2024-04-25T10:13:00.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.299+0000 7efda047e740 10 client.6927 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T10:13:00.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.299+0000 7efda047e740 10 client.6927 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:13:00.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.299+0000 7efda047e740 20 client.6927 populate_metadata read hostname 'smithi119' 2024-04-25T10:13:00.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.299+0000 7efda047e740 1 --2- 172.21.15.119:0/421748338 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5632d6b63290 0x5632d6b1fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.299+0000 7efda047e740 4 client.6927 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:13:00.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.299+0000 7efda047e740 1 -- 172.21.15.119:0/421748338 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5632d6a76500 con 0x5632d6b63290 2024-04-25T10:13:00.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.299+0000 7efd96a00640 1 --2- 172.21.15.119:0/421748338 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5632d6b63290 0x5632d6b1fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efd96a00640 1 --2- 172.21.15.119:0/421748338 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5632d6b63290 0x5632d6b1fa10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6cad78862f093c5 server_cookie=e3af3548f8ed7dc9 in_seq=0 out_seq=0 2024-04-25T10:13:00.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efd77400640 10 client.6927 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:13:00.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efd77400640 1 -- 172.21.15.119:0/421748338 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5632d6a76500 con 0x5632d6b63290 2024-04-25T10:13:00.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efd77400640 10 client.6927 handle_command_reply: tid=0 2024-04-25T10:13:00.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efda047e740 1 client.6927 shutdown 2024-04-25T10:13:00.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efda047e740 1 -- 172.21.15.119:0/421748338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b35670 msgr2=0x5632d6b23500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efda047e740 1 --2- 172.21.15.119:0/421748338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b35670 0x5632d6b23500 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7efd84078900 tx=0x7efd840aa040 comp rx=0 tx=0).stop 2024-04-25T10:13:00.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efda047e740 1 -- 172.21.15.119:0/421748338 shutdown_connections 2024-04-25T10:13:00.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efda047e740 1 --2- 172.21.15.119:0/421748338 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x5632d6b63290 0x5632d6b1fa10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efda047e740 1 --2- 172.21.15.119:0/421748338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5632d6b47280 0x5632d6b1f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efda047e740 1 --2- 172.21.15.119:0/421748338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5632d6b35670 0x5632d6b23500 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efda047e740 1 --2- 172.21.15.119:0/421748338 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5632d6b0abb0 0x5632d6b22fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efd77400640 0 client.6927 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:13:00.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efda047e740 1 -- 172.21.15.119:0/421748338 >> 172.21.15.119:0/421748338 conn(0x5632d6ad2a30 msgr2=0x5632d6b38de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efda047e740 1 -- 172.21.15.119:0/421748338 shutdown_connections 2024-04-25T10:13:00.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efda047e740 1 -- 172.21.15.119:0/421748338 wait complete. 2024-04-25T10:13:00.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.300+0000 7efda047e740 20 client.6927 trim_cache size 0 max 16384 2024-04-25T10:13:00.310 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:13:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2024-04-25T10:13:00.310 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2024-04-25T10:13:00.311 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:13:00.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.301+0000 7efd9d600640 1 -- 172.21.15.119:0/2351988510 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd7806b4d0 msgr2=0x7efd7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.301+0000 7efd9d600640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd7806b4d0 0x7efd7806d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efd88000f60 tx=0x7efd88009040 comp rx=0 tx=0).stop 2024-04-25T10:13:00.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.301+0000 7efd9d600640 1 -- 172.21.15.119:0/2351988510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd980cfa90 msgr2=0x7efd980cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.301+0000 7efd9d600640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd980cfa90 0x7efd980cdda0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7efd840098c0 tx=0x7efd840046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.301+0000 7efd9d600640 1 -- 172.21.15.119:0/2351988510 shutdown_connections 2024-04-25T10:13:00.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.301+0000 7efd9d600640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd7806b4d0 0x7efd7806d990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.301+0000 7efd9d600640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd9815b720 0x7efd98149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.301+0000 7efd9d600640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd980cfa90 0x7efd980cdda0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.301+0000 7efd9d600640 1 --2- 172.21.15.119:0/2351988510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd980aed40 0x7efd9814a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.301+0000 7efd9d600640 1 -- 172.21.15.119:0/2351988510 >> 172.21.15.119:0/2351988510 conn(0x7efd9806aba0 msgr2=0x7efd9815bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.301+0000 7efd9d600640 1 -- 172.21.15.119:0/2351988510 shutdown_connections 2024-04-25T10:13:00.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.301+0000 7efd9d600640 1 -- 172.21.15.119:0/2351988510 wait complete. 2024-04-25T10:13:00.324 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:13:00.324 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:00.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.516+0000 7f89bfe00640 1 Processor -- start 2024-04-25T10:13:00.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.516+0000 7f89bfe00640 1 -- start start 2024-04-25T10:13:00.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.518+0000 7f89bfe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89c0144040 0x7f89c00cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.518+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89c015acd0 0x7f89c006e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.518+0000 7f89bfe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f89c00cce40 0x7f89c006e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.518+0000 7f89bfe00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c0144420 con 0x7f89c0144040 2024-04-25T10:13:00.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.518+0000 7f89bfe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c00b3d80 con 0x7f89c00cce40 2024-04-25T10:13:00.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.518+0000 7f89bfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00b4060 con 0x7f89c015acd0 2024-04-25T10:13:00.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.518+0000 7f89bea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89c0144040 0x7f89c00cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.518+0000 7f89bea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89c0144040 0x7f89c00cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47144/0 (socket says 172.21.15.119:47144) 2024-04-25T10:13:00.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.518+0000 7f89bea00640 1 -- 172.21.15.119:0/1262785778 learned_addr learned my addr 172.21.15.119:0/1262785778 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:00.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.519+0000 7f89bea00640 1 -- 172.21.15.119:0/1262785778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89c015acd0 msgr2=0x7f89c006e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.519+0000 7f89b7e00640 1 --2- 172.21.15.119:0/1262785778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89c015acd0 0x7f89c006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.519+0000 7f89bf400640 1 --2- 172.21.15.119:0/1262785778 >> v2:172.21.15.132:3300/0 conn(0x7f89c00cce40 0x7f89c006e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.519+0000 7f89bea00640 1 --2- 172.21.15.119:0/1262785778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89c015acd0 0x7f89c006e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.519+0000 7f89bea00640 1 -- 172.21.15.119:0/1262785778 >> v2:172.21.15.132:3300/0 conn(0x7f89c00cce40 msgr2=0x7f89c006e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.519+0000 7f89bea00640 1 --2- 172.21.15.119:0/1262785778 >> v2:172.21.15.132:3300/0 conn(0x7f89c00cce40 0x7f89c006e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.519+0000 7f89bea00640 1 -- 172.21.15.119:0/1262785778 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c00b4340 con 0x7f89c0144040 2024-04-25T10:13:00.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.520+0000 7f89bea00640 1 --2- 172.21.15.119:0/1262785778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c0144040 0x7f89c00cc900 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f89c006ef90 tx=0x7f89b002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b50758df16bff7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.520+0000 7f89be000640 1 -- 172.21.15.119:0/1262785778 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b0037070 con 0x7f89c0144040 2024-04-25T10:13:00.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.520+0000 7f89be000640 1 -- 172.21.15.119:0/1262785778 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b0032440 con 0x7f89c0144040 2024-04-25T10:13:00.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.520+0000 7f89be000640 1 -- 172.21.15.119:0/1262785778 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b0032810 con 0x7f89c0144040 2024-04-25T10:13:00.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.521+0000 7f89bfe00640 1 -- 172.21.15.119:0/1262785778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c0144040 msgr2=0x7f89c00cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.521+0000 7f89bfe00640 1 --2- 172.21.15.119:0/1262785778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c0144040 0x7f89c00cc900 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f89c006ef90 tx=0x7f89b002a140 comp rx=0 tx=0).stop 2024-04-25T10:13:00.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.521+0000 7f89bfe00640 1 -- 172.21.15.119:0/1262785778 shutdown_connections 2024-04-25T10:13:00.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.521+0000 7f89bfe00640 1 --2- 172.21.15.119:0/1262785778 >> v2:172.21.15.132:3300/0 conn(0x7f89c00cce40 0x7f89c006e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.521+0000 7f89bfe00640 1 --2- 172.21.15.119:0/1262785778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89c015acd0 0x7f89c006e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.521+0000 7f89bfe00640 1 --2- 172.21.15.119:0/1262785778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c0144040 0x7f89c00cc900 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.521+0000 7f89bfe00640 1 -- 172.21.15.119:0/1262785778 >> 172.21.15.119:0/1262785778 conn(0x7f89c006b1d0 msgr2=0x7f89c00b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.521+0000 7f89bfe00640 1 -- 172.21.15.119:0/1262785778 shutdown_connections 2024-04-25T10:13:00.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.521+0000 7f89bfe00640 1 -- 172.21.15.119:0/1262785778 wait complete. 2024-04-25T10:13:00.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.523+0000 7f89bfe00640 1 Processor -- start 2024-04-25T10:13:00.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.523+0000 7f89bfe00640 1 -- start start 2024-04-25T10:13:00.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89c00cce40 0x7f89c0159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c0144040 0x7f89c015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89c015acd0 0x7f89c006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89bfe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00b5180 con 0x7f89c0144040 2024-04-25T10:13:00.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89bfe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00b4900 con 0x7f89c015acd0 2024-04-25T10:13:00.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89bfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00b5300 con 0x7f89c00cce40 2024-04-25T10:13:00.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89b7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c0144040 0x7f89c015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89b7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c0144040 0x7f89c015ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47148/0 (socket says 172.21.15.119:47148) 2024-04-25T10:13:00.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89b7e00640 1 -- 172.21.15.119:0/295891163 learned_addr learned my addr 172.21.15.119:0/295891163 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:00.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89bf400640 1 --2- 172.21.15.119:0/295891163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89c015acd0 0x7f89c006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89b7e00640 1 -- 172.21.15.119:0/295891163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89c00cce40 msgr2=0x7f89c0159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89b7e00640 1 --2- 172.21.15.119:0/295891163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89c00cce40 0x7f89c0159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89b7e00640 1 -- 172.21.15.119:0/295891163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89c015acd0 msgr2=0x7f89c006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89b7e00640 1 --2- 172.21.15.119:0/295891163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89c015acd0 0x7f89c006dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.524+0000 7f89b7e00640 1 -- 172.21.15.119:0/295891163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c015a420 con 0x7f89c0144040 2024-04-25T10:13:00.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.525+0000 7f89b7e00640 1 --2- 172.21.15.119:0/295891163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c0144040 0x7f89c015ba80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f89a800c7f0 tx=0x7f89a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.525+0000 7f89b6a00640 1 -- 172.21.15.119:0/295891163 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8015070 con 0x7f89c0144040 2024-04-25T10:13:00.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.525+0000 7f89b6a00640 1 -- 172.21.15.119:0/295891163 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a8004240 con 0x7f89c0144040 2024-04-25T10:13:00.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.525+0000 7f89b6a00640 1 -- 172.21.15.119:0/295891163 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8007620 con 0x7f89c0144040 2024-04-25T10:13:00.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.525+0000 7f89bfe00640 1 -- 172.21.15.119:0/295891163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c015cca0 con 0x7f89c0144040 2024-04-25T10:13:00.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.525+0000 7f89bfe00640 1 -- 172.21.15.119:0/295891163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c015d260 con 0x7f89c0144040 2024-04-25T10:13:00.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.526+0000 7f89bfe00640 1 -- 172.21.15.119:0/295891163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c00749b0 con 0x7f89c0144040 2024-04-25T10:13:00.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.527+0000 7f89b6a00640 1 -- 172.21.15.119:0/295891163 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89a80079d0 con 0x7f89c0144040 2024-04-25T10:13:00.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.529+0000 7f89b6a00640 1 --2- 172.21.15.119:0/295891163 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f899c06b4d0 0x7f899c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.529+0000 7f89b6a00640 1 -- 172.21.15.119:0/295891163 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f89a808a660 con 0x7f89c0144040 2024-04-25T10:13:00.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.529+0000 7f89bea00640 1 --2- 172.21.15.119:0/295891163 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f899c06b4d0 0x7f899c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.530+0000 7f89bea00640 1 --2- 172.21.15.119:0/295891163 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f899c06b4d0 0x7f899c06d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f89b0034000 tx=0x7f89b002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.534+0000 7f89b6a00640 1 -- 172.21.15.119:0/295891163 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89a8059440 con 0x7f89c0144040 2024-04-25T10:13:00.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.766+0000 7f89bfe00640 1 -- 172.21.15.119:0/295891163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89c00b3410 con 0x7f89c0144040 2024-04-25T10:13:00.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.767+0000 7f89b6a00640 1 -- 172.21.15.119:0/295891163 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f89a8058d40 con 0x7f89c0144040 2024-04-25T10:13:00.772 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:00.772 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2798722683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2798722683},{"type":"v1","addr":"172.21.15.132:6833","nonce":2798722683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2834911408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2834911408},{"type":"v1","addr":"172.21.15.132:6835","nonce":2834911408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/643040335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":643040335},{"type":"v1","addr":"172.21.15.119:6835","nonce":643040335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:34.712290+0000","modified":"2024-04-25T10:12:34.407591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6837/2127292269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2127292269},{"type":"v1","addr":"172.21.15.119:6837","nonce":2127292269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:13:00.773 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 52 2024-04-25T10:13:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.770+0000 7f89bfe00640 1 -- 172.21.15.119:0/295891163 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f899c06b4d0 msgr2=0x7f899c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.770+0000 7f89bfe00640 1 --2- 172.21.15.119:0/295891163 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f899c06b4d0 0x7f899c06d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f89b0034000 tx=0x7f89b002c000 comp rx=0 tx=0).stop 2024-04-25T10:13:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.770+0000 7f89bfe00640 1 -- 172.21.15.119:0/295891163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c0144040 msgr2=0x7f89c015ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.770+0000 7f89bfe00640 1 --2- 172.21.15.119:0/295891163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c0144040 0x7f89c015ba80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f89a800c7f0 tx=0x7f89a800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.770+0000 7f89bfe00640 1 -- 172.21.15.119:0/295891163 shutdown_connections 2024-04-25T10:13:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.770+0000 7f89bfe00640 1 --2- 172.21.15.119:0/295891163 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f899c06b4d0 0x7f899c06d990 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.770+0000 7f89bfe00640 1 --2- 172.21.15.119:0/295891163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89c015acd0 0x7f89c006dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.770+0000 7f89bfe00640 1 --2- 172.21.15.119:0/295891163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89c0144040 0x7f89c015ba80 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.770+0000 7f89bfe00640 1 --2- 172.21.15.119:0/295891163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89c00cce40 0x7f89c0159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.770+0000 7f89bfe00640 1 -- 172.21.15.119:0/295891163 >> 172.21.15.119:0/295891163 conn(0x7f89c006b1d0 msgr2=0x7f89c00af460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.770+0000 7f89bfe00640 1 -- 172.21.15.119:0/295891163 shutdown_connections 2024-04-25T10:13:00.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.770+0000 7f89bfe00640 1 -- 172.21.15.119:0/295891163 wait complete. 2024-04-25T10:13:00.793 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T10:13:00.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.992+0000 7fd18d200640 1 Processor -- start 2024-04-25T10:13:00.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.992+0000 7fd18d200640 1 -- start start 2024-04-25T10:13:01.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.994+0000 7fd18d200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd1880ce940 0x7fd1880b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.994+0000 7fd18d200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1880cfcf0 con 0x7fd1880b7030 2024-04-25T10:13:01.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.994+0000 7fd18d200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1880cffd0 con 0x7fd18815ba70 2024-04-25T10:13:01.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.994+0000 7fd18d200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1880d02b0 con 0x7fd1880ce940 2024-04-25T10:13:01.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.994+0000 7fd185400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd1880b7030 0x7fd1880ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41496/0 (socket says 172.21.15.119:41496) 2024-04-25T10:13:01.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.994+0000 7fd185400640 1 -- 172.21.15.119:0/476407054 learned_addr learned my addr 172.21.15.119:0/476407054 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:01.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.995+0000 7fd186800640 1 --2- 172.21.15.119:0/476407054 >> v2:172.21.15.132:3300/0 conn(0x7fd1880ce940 0x7fd1880b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.995+0000 7fd184a00640 1 -- 172.21.15.119:0/476407054 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736084811 0 0) 0x7fd1880cfcf0 con 0x7fd1880b7030 2024-04-25T10:13:01.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.995+0000 7fd184a00640 1 -- 172.21.15.119:0/476407054 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd16c003520 con 0x7fd1880b7030 2024-04-25T10:13:01.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.995+0000 7fd184a00640 1 -- 172.21.15.119:0/476407054 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 67225023 0 0) 0x7fd16c003520 con 0x7fd1880b7030 2024-04-25T10:13:01.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.995+0000 7fd184a00640 1 -- 172.21.15.119:0/476407054 >> v2:172.21.15.132:3300/0 conn(0x7fd1880ce940 msgr2=0x7fd1880b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.995+0000 7fd184a00640 1 --2- 172.21.15.119:0/476407054 >> v2:172.21.15.132:3300/0 conn(0x7fd1880ce940 0x7fd1880b74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.995+0000 7fd184a00640 1 -- 172.21.15.119:0/476407054 >> v1:172.21.15.132:6789/0 conn(0x7fd18815ba70 legacy=0x7fd1880d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.995+0000 7fd184a00640 1 -- 172.21.15.119:0/476407054 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1880d0590 con 0x7fd1880b7030 2024-04-25T10:13:01.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.995+0000 7fd184a00640 1 -- 172.21.15.119:0/476407054 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fd170001670 con 0x7fd1880b7030 2024-04-25T10:13:01.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.995+0000 7fd184a00640 1 -- 172.21.15.119:0/476407054 >> v1:172.21.15.119:6789/0 conn(0x7fd1880b7030 legacy=0x7fd1880ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.995+0000 7fd184a00640 1 --2- 172.21.15.119:0/476407054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd16c0039b0 0x7fd16c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.996+0000 7fd184a00640 1 --2- 172.21.15.119:0/476407054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd16c004800 0x7fd16c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.996+0000 7fd184a00640 1 --2- 172.21.15.119:0/476407054 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd16c0051c0 0x7fd16c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.996+0000 7fd184a00640 1 -- 172.21.15.119:0/476407054 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880d02b0 con 0x7fd16c004800 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.996+0000 7fd184a00640 1 -- 172.21.15.119:0/476407054 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c00c060 con 0x7fd16c0039b0 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.996+0000 7fd184a00640 1 -- 172.21.15.119:0/476407054 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c00c260 con 0x7fd16c0051c0 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.996+0000 7fd185400640 1 --2- 172.21.15.119:0/476407054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd16c004800 0x7fd16c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.996+0000 7fd185400640 1 -- 172.21.15.119:0/476407054 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd16c0051c0 msgr2=0x7fd16c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.996+0000 7fd185400640 1 --2- 172.21.15.119:0/476407054 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd16c0051c0 0x7fd16c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.996+0000 7fd185400640 1 -- 172.21.15.119:0/476407054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd16c0039b0 msgr2=0x7fd16c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.996+0000 7fd185400640 1 --2- 172.21.15.119:0/476407054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd16c0039b0 0x7fd16c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.996+0000 7fd185400640 1 -- 172.21.15.119:0/476407054 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c003520 con 0x7fd16c004800 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.996+0000 7fd186800640 1 -- 172.21.15.119:0/476407054 reap_dead start 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.997+0000 7fd185400640 1 --2- 172.21.15.119:0/476407054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd16c004800 0x7fd16c004c80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fd17000c240 tx=0x7fd16c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bf07e1c8440a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.997+0000 7fd184a00640 1 -- 172.21.15.119:0/476407054 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170007cf0 con 0x7fd16c004800 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.997+0000 7fd184a00640 1 -- 172.21.15.119:0/476407054 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd170005af0 con 0x7fd16c004800 2024-04-25T10:13:01.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.997+0000 7fd184a00640 1 -- 172.21.15.119:0/476407054 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170007070 con 0x7fd16c004800 2024-04-25T10:13:01.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.998+0000 7fd18d200640 1 -- 172.21.15.119:0/476407054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd16c004800 msgr2=0x7fd16c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.998+0000 7fd18d200640 1 --2- 172.21.15.119:0/476407054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd16c004800 0x7fd16c004c80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fd17000c240 tx=0x7fd16c005830 comp rx=0 tx=0).stop 2024-04-25T10:13:01.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.998+0000 7fd18d200640 1 -- 172.21.15.119:0/476407054 shutdown_connections 2024-04-25T10:13:01.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.998+0000 7fd18d200640 1 --2- 172.21.15.119:0/476407054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd16c004800 0x7fd16c004c80 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.998+0000 7fd18d200640 1 -- 172.21.15.119:0/476407054 >> 172.21.15.119:0/476407054 conn(0x7fd18806f4a0 msgr2=0x7fd18806f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.998+0000 7fd18d200640 1 -- 172.21.15.119:0/476407054 shutdown_connections 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:00.998+0000 7fd18d200640 1 -- 172.21.15.119:0/476407054 wait complete. 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.000+0000 7fd18d200640 1 Processor -- start 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.000+0000 7fd18d200640 1 -- start start 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.001+0000 7fd18d200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd16c004800 0x7fd18814b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.001+0000 7fd18d200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd18814ad90 0x7fd1880b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.001+0000 7fd18d200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1880b2150 0x7fd1880c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.001+0000 7fd18d200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18815be50 con 0x7fd1880b2150 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.001+0000 7fd18d200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880cf980 con 0x7fd18814ad90 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.001+0000 7fd18d200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880b2720 con 0x7fd16c004800 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.001+0000 7fd186800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1880b2150 0x7fd1880c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.001+0000 7fd186800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1880b2150 0x7fd1880c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34964/0 (socket says 172.21.15.119:34964) 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.001+0000 7fd186800640 1 -- 172.21.15.119:0/3722037328 learned_addr learned my addr 172.21.15.119:0/3722037328 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:01.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.002+0000 7fd185e00640 1 --2- 172.21.15.119:0/3722037328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd16c004800 0x7fd18814b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.002+0000 7fd185400640 1 --2- 172.21.15.119:0/3722037328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd18814ad90 0x7fd1880b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.002+0000 7fd186800640 1 -- 172.21.15.119:0/3722037328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd16c004800 msgr2=0x7fd18814b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.002+0000 7fd186800640 1 --2- 172.21.15.119:0/3722037328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd16c004800 0x7fd18814b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.002+0000 7fd186800640 1 -- 172.21.15.119:0/3722037328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd18814ad90 msgr2=0x7fd1880b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.002+0000 7fd186800640 1 --2- 172.21.15.119:0/3722037328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd18814ad90 0x7fd1880b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.002+0000 7fd186800640 1 -- 172.21.15.119:0/3722037328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18814b210 con 0x7fd1880b2150 2024-04-25T10:13:01.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.002+0000 7fd186800640 1 --2- 172.21.15.119:0/3722037328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1880b2150 0x7fd1880c9a10 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fd1880ccf10 tx=0x7fd18000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.002+0000 7fd17e000640 1 -- 172.21.15.119:0/3722037328 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180017070 con 0x7fd1880b2150 2024-04-25T10:13:01.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.002+0000 7fd17e000640 1 -- 172.21.15.119:0/3722037328 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18000c680 con 0x7fd1880b2150 2024-04-25T10:13:01.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.002+0000 7fd17e000640 1 -- 172.21.15.119:0/3722037328 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000cab0 con 0x7fd1880b2150 2024-04-25T10:13:01.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.003+0000 7fd18d200640 1 -- 172.21.15.119:0/3722037328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1880c6700 con 0x7fd1880b2150 2024-04-25T10:13:01.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.003+0000 7fd18d200640 1 -- 172.21.15.119:0/3722037328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1880c6cc0 con 0x7fd1880b2150 2024-04-25T10:13:01.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.006+0000 7fd17e000640 1 -- 172.21.15.119:0/3722037328 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd18001a070 con 0x7fd1880b2150 2024-04-25T10:13:01.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.007+0000 7fd17e000640 1 --2- 172.21.15.119:0/3722037328 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd16406b4d0 0x7fd16406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.007+0000 7fd17e000640 1 -- 172.21.15.119:0/3722037328 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd180089c10 con 0x7fd1880b2150 2024-04-25T10:13:01.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.007+0000 7fd185e00640 1 --2- 172.21.15.119:0/3722037328 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd16406b4d0 0x7fd16406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.007+0000 7fd18ff8d740 1 Processor -- start 2024-04-25T10:13:01.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.007+0000 7fd18ff8d740 1 -- start start 2024-04-25T10:13:01.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.008+0000 7fd185e00640 1 --2- 172.21.15.119:0/3722037328 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd16406b4d0 0x7fd16406d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd178007fd0 tx=0x7fd178009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.008+0000 7fd18ff8d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe86924370 0x55fe86924750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.008+0000 7fd18ff8d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 0x55fe86942160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.008+0000 7fd18ff8d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe868fa050 0x55fe869402d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.008+0000 7fd18ff8d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8685cae0 con 0x55fe868fa050 2024-04-25T10:13:01.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.008+0000 7fd18ff8d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe868057c0 con 0x55fe86941c10 2024-04-25T10:13:01.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.008+0000 7fd18ff8d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe86858b80 con 0x55fe86924370 2024-04-25T10:13:01.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.008+0000 7fd186800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe868fa050 0x55fe869402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.009+0000 7fd186800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe868fa050 0x55fe869402d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34976/0 (socket says 172.21.15.119:34976) 2024-04-25T10:13:01.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.009+0000 7fd186800640 1 -- 172.21.15.119:0/2291119168 learned_addr learned my addr 172.21.15.119:0/2291119168 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:01.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.009+0000 7fd185400640 1 --2- 172.21.15.119:0/2291119168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 0x55fe86942160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.009+0000 7fd185e00640 1 --2- 172.21.15.119:0/2291119168 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe86924370 0x55fe86924750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.009+0000 7fd186800640 1 -- 172.21.15.119:0/2291119168 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe86924370 msgr2=0x55fe86924750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.009+0000 7fd186800640 1 --2- 172.21.15.119:0/2291119168 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe86924370 0x55fe86924750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.009+0000 7fd186800640 1 -- 172.21.15.119:0/2291119168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 msgr2=0x55fe86942160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.009+0000 7fd186800640 1 --2- 172.21.15.119:0/2291119168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 0x55fe86942160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.009+0000 7fd186800640 1 -- 172.21.15.119:0/2291119168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe8663d7c0 con 0x55fe868fa050 2024-04-25T10:13:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.009+0000 7fd185400640 1 --2- 172.21.15.119:0/2291119168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 0x55fe86942160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.009+0000 7fd186800640 1 --2- 172.21.15.119:0/2291119168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe868fa050 0x55fe869402d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd180001eb0 tx=0x7fd180057cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46584c0784996f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.009+0000 7fd176a00640 1 -- 172.21.15.119:0/2291119168 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180021070 con 0x55fe868fa050 2024-04-25T10:13:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.009+0000 7fd176a00640 1 -- 172.21.15.119:0/2291119168 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18005c170 con 0x55fe868fa050 2024-04-25T10:13:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.009+0000 7fd176a00640 1 -- 172.21.15.119:0/2291119168 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180060970 con 0x55fe868fa050 2024-04-25T10:13:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.010+0000 7fd18ff8d740 1 -- 172.21.15.119:0/2291119168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe868fa050 msgr2=0x55fe869402d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.010+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/2291119168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe868fa050 0x55fe869402d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd180001eb0 tx=0x7fd180057cf0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.010+0000 7fd18ff8d740 1 -- 172.21.15.119:0/2291119168 shutdown_connections 2024-04-25T10:13:01.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.010+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/2291119168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe868fa050 0x55fe869402d0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.010+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/2291119168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 0x55fe86942160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.010+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/2291119168 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe86924370 0x55fe86924750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.010+0000 7fd18ff8d740 1 -- 172.21.15.119:0/2291119168 >> 172.21.15.119:0/2291119168 conn(0x55fe868f6980 msgr2=0x55fe8693f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.010+0000 7fd18ff8d740 1 -- 172.21.15.119:0/2291119168 shutdown_connections 2024-04-25T10:13:01.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.010+0000 7fd18ff8d740 1 -- 172.21.15.119:0/2291119168 wait complete. 2024-04-25T10:13:01.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.011+0000 7fd18ff8d740 1 -- start start 2024-04-25T10:13:01.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.011+0000 7fd18ff8d740 1 Processor -- start 2024-04-25T10:13:01.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.012+0000 7fd18ff8d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe86925320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.012+0000 7fd18ff8d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 0x55fe869139f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.012+0000 7fd18ff8d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe869276c0 0x55fe86927b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.012+0000 7fd18ff8d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe868fadf0 con 0x55fe86924370 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.012+0000 7fd18ff8d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe867fad40 con 0x55fe86941c10 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.012+0000 7fd18ff8d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe868f48b0 con 0x55fe869276c0 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.012+0000 7fd18ff8d740 10 client.0 _ll_register_callbacks cb 0x55fe86570ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.012+0000 7fd185e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe86925320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.012+0000 7fd185e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe86925320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34992/0 (socket says 172.21.15.119:34992) 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.012+0000 7fd185e00640 1 -- 172.21.15.119:0/3318682482 learned_addr learned my addr 172.21.15.119:0/3318682482 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.013+0000 7fd185400640 1 --2- 172.21.15.119:0/3318682482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 0x55fe869139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.013+0000 7fd186800640 1 --2- 172.21.15.119:0/3318682482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe869276c0 0x55fe86927b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.013+0000 7fd185e00640 1 -- 172.21.15.119:0/3318682482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe869276c0 msgr2=0x55fe86927b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.013+0000 7fd185e00640 1 --2- 172.21.15.119:0/3318682482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe869276c0 0x55fe86927b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.013+0000 7fd185e00640 1 -- 172.21.15.119:0/3318682482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 msgr2=0x55fe869139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.013+0000 7fd185e00640 1 --2- 172.21.15.119:0/3318682482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 0x55fe869139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.013+0000 7fd185e00640 1 -- 172.21.15.119:0/3318682482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe8669df50 con 0x55fe86924370 2024-04-25T10:13:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.013+0000 7fd185e00640 1 --2- 172.21.15.119:0/3318682482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe86925320 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fd17800eb00 tx=0x7fd1780014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df29d8c2ec07e6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.013+0000 7fd16b400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:13:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.013+0000 7fd16b400640 1 -- 172.21.15.119:0/3318682482 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1780057e0 con 0x55fe86924370 2024-04-25T10:13:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.014+0000 7fd16b400640 1 -- 172.21.15.119:0/3318682482 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd178021b10 con 0x55fe86924370 2024-04-25T10:13:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.014+0000 7fd18ff8d740 1 -- 172.21.15.119:0/3318682482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd18005c170 con 0x55fe86924370 2024-04-25T10:13:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.014+0000 7fd16b400640 1 -- 172.21.15.119:0/3318682482 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd178021d10 con 0x55fe86924370 2024-04-25T10:13:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.014+0000 7fd18ff8d740 10 client.6945 fetch_fsmap learned FSMap version 52 2024-04-25T10:13:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.014+0000 7fd18ff8d740 1 -- 172.21.15.119:0/3318682482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55fe865310b0 con 0x55fe86924370 2024-04-25T10:13:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.014+0000 7fd16b400640 1 -- 172.21.15.119:0/3318682482 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18005c170 con 0x55fe86924370 2024-04-25T10:13:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.014+0000 7fd16b400640 1 -- 172.21.15.119:0/3318682482 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd178021420 con 0x55fe86924370 2024-04-25T10:13:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.014+0000 7fd18ff8d740 10 client.6945 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T10:13:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.014+0000 7fd18ff8d740 10 client.6945 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.014+0000 7fd18ff8d740 20 client.6945 populate_metadata read hostname 'smithi119' 2024-04-25T10:13:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.014+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/3318682482 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fe8695a5e0 0x55fe8695c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.014+0000 7fd18ff8d740 4 client.6945 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.014+0000 7fd18ff8d740 1 -- 172.21.15.119:0/3318682482 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe868bbb60 con 0x55fe8695a5e0 2024-04-25T10:13:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.015+0000 7fd185400640 1 --2- 172.21.15.119:0/3318682482 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fe8695a5e0 0x55fe8695c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.015+0000 7fd185400640 1 --2- 172.21.15.119:0/3318682482 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fe8695a5e0 0x55fe8695c9d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b1667d5abc9b36 server_cookie=ae7c8d752c6682fd in_seq=0 out_seq=0 2024-04-25T10:13:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.015+0000 7fd16b400640 10 client.6945 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:13:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.016+0000 7fd16b400640 1 -- 172.21.15.119:0/3318682482 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe868bbb60 con 0x55fe8695a5e0 2024-04-25T10:13:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.016+0000 7fd16b400640 10 client.6945 handle_command_reply: tid=0 2024-04-25T10:13:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.016+0000 7fd18ff8d740 1 client.6945 shutdown 2024-04-25T10:13:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.016+0000 7fd18ff8d740 1 -- 172.21.15.119:0/3318682482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 msgr2=0x55fe86925320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.016+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/3318682482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe86925320 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fd17800eb00 tx=0x7fd1780014a0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.016+0000 7fd18ff8d740 1 -- 172.21.15.119:0/3318682482 shutdown_connections 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.016+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/3318682482 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fe8695a5e0 0x55fe8695c9d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.016+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/3318682482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe869276c0 0x55fe86927b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.016+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/3318682482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 0x55fe869139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.016+0000 7fd16b400640 0 client.6945 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.016+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/3318682482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe86925320 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.016+0000 7fd18ff8d740 1 -- 172.21.15.119:0/3318682482 >> 172.21.15.119:0/3318682482 conn(0x55fe868fa050 msgr2=0x55fe868f3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.016+0000 7fd18ff8d740 1 -- 172.21.15.119:0/3318682482 shutdown_connections 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.016+0000 7fd18ff8d740 1 -- 172.21.15.119:0/3318682482 wait complete. 2024-04-25T10:13:01.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.017+0000 7fd18ff8d740 20 client.6945 trim_cache size 0 max 16384 2024-04-25T10:13:01.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.029+0000 7fd18ff8d740 1 Processor -- start 2024-04-25T10:13:01.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.029+0000 7fd18ff8d740 1 -- start start 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.030+0000 7fd18ff8d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe868efe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.030+0000 7fd18ff8d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe869276c0 0x55fe868f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.030+0000 7fd18ff8d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe86941c10 0x55fe86927b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.030+0000 7fd18ff8d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18806d4c0 con 0x55fe86924370 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.030+0000 7fd18ff8d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880c5d30 con 0x55fe869276c0 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.030+0000 7fd18ff8d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18806cec0 con 0x55fe86941c10 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.030+0000 7fd185e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe868efe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.030+0000 7fd185e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe868efe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34996/0 (socket says 172.21.15.119:34996) 2024-04-25T10:13:01.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.030+0000 7fd185e00640 1 -- 172.21.15.119:0/3737787044 learned_addr learned my addr 172.21.15.119:0/3737787044 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:01.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.031+0000 7fd185e00640 1 -- 172.21.15.119:0/3737787044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe86941c10 msgr2=0x55fe86927b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.031+0000 7fd185e00640 1 --2- 172.21.15.119:0/3737787044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe86941c10 0x55fe86927b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.031+0000 7fd185e00640 1 -- 172.21.15.119:0/3737787044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe869276c0 msgr2=0x55fe868f96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.031+0000 7fd185e00640 1 --2- 172.21.15.119:0/3737787044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe869276c0 0x55fe868f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.031+0000 7fd185e00640 1 -- 172.21.15.119:0/3737787044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe86870630 con 0x55fe86924370 2024-04-25T10:13:01.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.031+0000 7fd185e00640 1 --2- 172.21.15.119:0/3737787044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe868efe90 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd17800ead0 tx=0x7fd17801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d8c70f6c87e2eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.031+0000 7fd176000640 1 -- 172.21.15.119:0/3737787044 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178030040 con 0x55fe86924370 2024-04-25T10:13:01.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.031+0000 7fd176000640 1 -- 172.21.15.119:0/3737787044 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1780057e0 con 0x55fe86924370 2024-04-25T10:13:01.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.031+0000 7fd176000640 1 -- 172.21.15.119:0/3737787044 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178021970 con 0x55fe86924370 2024-04-25T10:13:01.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.032+0000 7fd18ff8d740 1 -- 172.21.15.119:0/3737787044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 msgr2=0x55fe868efe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.032+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/3737787044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe868efe90 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd17800ead0 tx=0x7fd17801e000 comp rx=0 tx=0).stop 2024-04-25T10:13:01.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.032+0000 7fd18ff8d740 1 -- 172.21.15.119:0/3737787044 shutdown_connections 2024-04-25T10:13:01.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.032+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/3737787044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe86941c10 0x55fe86927b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.032+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/3737787044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe869276c0 0x55fe868f96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.032+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/3737787044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe868efe90 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.032+0000 7fd18ff8d740 1 -- 172.21.15.119:0/3737787044 >> 172.21.15.119:0/3737787044 conn(0x55fe868fa050 msgr2=0x55fe8695ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.032+0000 7fd18ff8d740 1 -- 172.21.15.119:0/3737787044 shutdown_connections 2024-04-25T10:13:01.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.032+0000 7fd18ff8d740 1 -- 172.21.15.119:0/3737787044 wait complete. 2024-04-25T10:13:01.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.033+0000 7fd18ff8d740 1 -- start start 2024-04-25T10:13:01.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.033+0000 7fd18ff8d740 1 Processor -- start 2024-04-25T10:13:01.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd18ff8d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe8699f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd18ff8d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe869276c0 0x55fe8699fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd18ff8d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 0x55fe86958210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd18ff8d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe86926920 con 0x55fe86924370 2024-04-25T10:13:01.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd18ff8d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe869265a0 con 0x55fe86941c10 2024-04-25T10:13:01.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd18ff8d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe86938220 con 0x55fe869276c0 2024-04-25T10:13:01.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd18ff8d740 10 client.0 _ll_register_callbacks cb 0x55fe868f4d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:01.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd185e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe8699f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd185e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe8699f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35012/0 (socket says 172.21.15.119:35012) 2024-04-25T10:13:01.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd185e00640 1 -- 172.21.15.119:0/1407932270 learned_addr learned my addr 172.21.15.119:0/1407932270 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:01.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd185e00640 1 -- 172.21.15.119:0/1407932270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe869276c0 msgr2=0x55fe8699fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd185e00640 1 --2- 172.21.15.119:0/1407932270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe869276c0 0x55fe8699fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd185e00640 1 -- 172.21.15.119:0/1407932270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 msgr2=0x55fe86958210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd185e00640 1 --2- 172.21.15.119:0/1407932270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 0x55fe86958210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.034+0000 7fd185e00640 1 -- 172.21.15.119:0/1407932270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe869a0370 con 0x55fe86924370 2024-04-25T10:13:01.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.035+0000 7fd185e00640 1 --2- 172.21.15.119:0/1407932270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe8699f5c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd17800ffd0 tx=0x7fd178005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7e07e65a7af3a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.035+0000 7fd16b400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:13:01.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.035+0000 7fd16b400640 1 -- 172.21.15.119:0/1407932270 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178030040 con 0x55fe86924370 2024-04-25T10:13:01.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.035+0000 7fd16b400640 1 -- 172.21.15.119:0/1407932270 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd17803f450 con 0x55fe86924370 2024-04-25T10:13:01.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.035+0000 7fd18ff8d740 1 -- 172.21.15.119:0/1407932270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1780057e0 con 0x55fe86924370 2024-04-25T10:13:01.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.035+0000 7fd16b400640 1 -- 172.21.15.119:0/1407932270 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd178047050 con 0x55fe86924370 2024-04-25T10:13:01.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.035+0000 7fd18ff8d740 10 client.6951 fetch_fsmap learned FSMap version 52 2024-04-25T10:13:01.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.035+0000 7fd18ff8d740 1 -- 172.21.15.119:0/1407932270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55fe86911670 con 0x55fe86924370 2024-04-25T10:13:01.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.036+0000 7fd16b400640 1 -- 172.21.15.119:0/1407932270 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1780057e0 con 0x55fe86924370 2024-04-25T10:13:01.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.036+0000 7fd16b400640 1 -- 172.21.15.119:0/1407932270 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd17804a020 con 0x55fe86924370 2024-04-25T10:13:01.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.036+0000 7fd18ff8d740 10 client.6951 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T10:13:01.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.036+0000 7fd18ff8d740 10 client.6951 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:01.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.036+0000 7fd18ff8d740 20 client.6951 populate_metadata read hostname 'smithi119' 2024-04-25T10:13:01.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.036+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/1407932270 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fe86910040 0x55fe86910420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.036+0000 7fd18ff8d740 4 client.6951 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:13:01.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.036+0000 7fd18ff8d740 1 -- 172.21.15.119:0/1407932270 --> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55fe86939c60 con 0x55fe86910040 2024-04-25T10:13:01.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.036+0000 7fd185400640 1 --2- 172.21.15.119:0/1407932270 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fe86910040 0x55fe86910420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.036+0000 7fd185400640 1 --2- 172.21.15.119:0/1407932270 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fe86910040 0x55fe86910420 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d353fedb91b56735 server_cookie=9bf803597f6a2fe6 in_seq=0 out_seq=0 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.036+0000 7fd16b400640 10 client.6951 ms_handle_connect on v2:172.21.15.119:6836/2127292269 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd16b400640 1 -- 172.21.15.119:0/1407932270 <== mds.0 v2:172.21.15.119:6836/2127292269 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fe86939c60 con 0x55fe86910040 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd16b400640 10 client.6951 handle_command_reply: tid=0 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd18ff8d740 1 client.6951 shutdown 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd18ff8d740 1 -- 172.21.15.119:0/1407932270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 msgr2=0x55fe8699f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/1407932270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe8699f5c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd17800ffd0 tx=0x7fd178005000 comp rx=0 tx=0).stop 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd18ff8d740 1 -- 172.21.15.119:0/1407932270 shutdown_connections 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/1407932270 >> [v2:172.21.15.119:6836/2127292269,v1:172.21.15.119:6837/2127292269] conn(0x55fe86910040 0x55fe86910420 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/1407932270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe86941c10 0x55fe86958210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/1407932270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55fe869276c0 0x55fe8699fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd18ff8d740 1 --2- 172.21.15.119:0/1407932270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fe86924370 0x55fe8699f5c0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd18ff8d740 1 -- 172.21.15.119:0/1407932270 >> 172.21.15.119:0/1407932270 conn(0x55fe868fa050 msgr2=0x55fe86927ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd16b400640 0 client.6951 ms_handle_reset on v2:172.21.15.119:6836/2127292269 2024-04-25T10:13:01.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd18ff8d740 1 -- 172.21.15.119:0/1407932270 shutdown_connections 2024-04-25T10:13:01.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd18ff8d740 1 -- 172.21.15.119:0/1407932270 wait complete. 2024-04-25T10:13:01.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.037+0000 7fd18ff8d740 20 client.6951 trim_cache size 0 max 16384 2024-04-25T10:13:01.046 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:01.046 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2024-04-25T10:13:01.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.038+0000 7fd18d200640 1 -- 172.21.15.119:0/3722037328 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd16406b4d0 msgr2=0x7fd16406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.038+0000 7fd18d200640 1 --2- 172.21.15.119:0/3722037328 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd16406b4d0 0x7fd16406d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd178007fd0 tx=0x7fd178009040 comp rx=0 tx=0).stop 2024-04-25T10:13:01.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.038+0000 7fd18d200640 1 -- 172.21.15.119:0/3722037328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1880b2150 msgr2=0x7fd1880c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.038+0000 7fd18d200640 1 --2- 172.21.15.119:0/3722037328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1880b2150 0x7fd1880c9a10 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fd1880ccf10 tx=0x7fd18000ec00 comp rx=0 tx=0).stop 2024-04-25T10:13:01.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.038+0000 7fd18d200640 1 -- 172.21.15.119:0/3722037328 shutdown_connections 2024-04-25T10:13:01.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.038+0000 7fd18d200640 1 --2- 172.21.15.119:0/3722037328 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd16406b4d0 0x7fd16406d990 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.038+0000 7fd18d200640 1 --2- 172.21.15.119:0/3722037328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1880b2150 0x7fd1880c9a10 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.038+0000 7fd18d200640 1 --2- 172.21.15.119:0/3722037328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd18814ad90 0x7fd1880b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.038+0000 7fd18d200640 1 --2- 172.21.15.119:0/3722037328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd16c004800 0x7fd18814b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.038+0000 7fd18d200640 1 -- 172.21.15.119:0/3722037328 >> 172.21.15.119:0/3722037328 conn(0x7fd18806f4a0 msgr2=0x7fd18806f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.038+0000 7fd18d200640 1 -- 172.21.15.119:0/3722037328 shutdown_connections 2024-04-25T10:13:01.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.038+0000 7fd18d200640 1 -- 172.21.15.119:0/3722037328 wait complete. 2024-04-25T10:13:01.063 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:13:01.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.253+0000 7f861f200640 1 Processor -- start 2024-04-25T10:13:01.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.253+0000 7f861f200640 1 -- start start 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.254+0000 7f861f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8620159ca0 0x7f862015d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.255+0000 7f861f200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f86200b5470 0x7f862006d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.255+0000 7f861f200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86200cec30 con 0x7f862015a080 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.255+0000 7f861f200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86200cef10 con 0x7f86200b5470 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.255+0000 7f861f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86200cf1f0 con 0x7f8620159ca0 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.255+0000 7f8615400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f862015a080 0x7f862015de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41506/0 (socket says 172.21.15.119:41506) 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.255+0000 7f8615400640 1 -- 172.21.15.119:0/3815135621 learned_addr learned my addr 172.21.15.119:0/3815135621 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.255+0000 7f861e800640 1 --2- 172.21.15.119:0/3815135621 >> v2:172.21.15.119:3300/0 conn(0x7f86200b5470 0x7f862006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.255+0000 7f861de00640 1 --2- 172.21.15.119:0/3815135621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8620159ca0 0x7f862015d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.255+0000 7f861d400640 1 -- 172.21.15.119:0/3815135621 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448155810 0 0) 0x7f86200cec30 con 0x7f862015a080 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.256+0000 7f861d400640 1 -- 172.21.15.119:0/3815135621 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8604005730 con 0x7f862015a080 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.256+0000 7f861e800640 1 -- 172.21.15.119:0/3815135621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8620159ca0 msgr2=0x7f862015d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.256+0000 7f861e800640 1 --2- 172.21.15.119:0/3815135621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8620159ca0 0x7f862015d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.256+0000 7f861e800640 1 -- 172.21.15.119:0/3815135621 >> v1:172.21.15.119:6789/0 conn(0x7f862015a080 legacy=0x7f862015de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.256+0000 7f861e800640 1 -- 172.21.15.119:0/3815135621 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862006d420 con 0x7f86200b5470 2024-04-25T10:13:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.257+0000 7f861e800640 1 --2- 172.21.15.119:0/3815135621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86200b5470 0x7f862006d940 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8618009d90 tx=0x7f861802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809c036e02dc9743 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.257+0000 7f861d400640 1 -- 172.21.15.119:0/3815135621 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618007740 con 0x7f86200b5470 2024-04-25T10:13:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.257+0000 7f861d400640 1 -- 172.21.15.119:0/3815135621 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86180078e0 con 0x7f86200b5470 2024-04-25T10:13:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.257+0000 7f861d400640 1 -- 172.21.15.119:0/3815135621 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618007cb0 con 0x7f86200b5470 2024-04-25T10:13:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.258+0000 7f861f200640 1 -- 172.21.15.119:0/3815135621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86200b5470 msgr2=0x7f862006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.258+0000 7f861f200640 1 --2- 172.21.15.119:0/3815135621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86200b5470 0x7f862006d940 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8618009d90 tx=0x7f861802a660 comp rx=0 tx=0).stop 2024-04-25T10:13:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.258+0000 7f861f200640 1 -- 172.21.15.119:0/3815135621 shutdown_connections 2024-04-25T10:13:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.258+0000 7f861f200640 1 --2- 172.21.15.119:0/3815135621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86200b5470 0x7f862006d940 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.258+0000 7f861f200640 1 --2- 172.21.15.119:0/3815135621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8620159ca0 0x7f862015d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.258+0000 7f861f200640 1 -- 172.21.15.119:0/3815135621 >> 172.21.15.119:0/3815135621 conn(0x7f862006b1d0 msgr2=0x7f86200b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.258+0000 7f861f200640 1 -- 172.21.15.119:0/3815135621 shutdown_connections 2024-04-25T10:13:01.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.258+0000 7f861f200640 1 -- 172.21.15.119:0/3815135621 wait complete. 2024-04-25T10:13:01.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.259+0000 7f861f200640 1 Processor -- start 2024-04-25T10:13:01.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.260+0000 7f861f200640 1 -- start start 2024-04-25T10:13:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.260+0000 7f861f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86200b5470 0x7f8620149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.260+0000 7f861f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8620159ca0 0x7f86201f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.260+0000 7f861f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86201f31e0 0x7f86201f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.260+0000 7f861f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862015e6e0 con 0x7f86200b5470 2024-04-25T10:13:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.260+0000 7f861f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862015e560 con 0x7f8620159ca0 2024-04-25T10:13:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.260+0000 7f861f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86200b1a30 con 0x7f86201f31e0 2024-04-25T10:13:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.261+0000 7f861de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86200b5470 0x7f8620149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.261+0000 7f861de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86200b5470 0x7f8620149530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35034/0 (socket says 172.21.15.119:35034) 2024-04-25T10:13:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.261+0000 7f861de00640 1 -- 172.21.15.119:0/1695166349 learned_addr learned my addr 172.21.15.119:0/1695166349 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.261+0000 7f861de00640 1 -- 172.21.15.119:0/1695166349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86201f31e0 msgr2=0x7f86201f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.261+0000 7f861de00640 1 --2- 172.21.15.119:0/1695166349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86201f31e0 0x7f86201f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.261+0000 7f861de00640 1 -- 172.21.15.119:0/1695166349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8620159ca0 msgr2=0x7f86201f2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.261+0000 7f8615400640 1 --2- 172.21.15.119:0/1695166349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8620159ca0 0x7f86201f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.261+0000 7f861de00640 1 --2- 172.21.15.119:0/1695166349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8620159ca0 0x7f86201f2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.261+0000 7f861de00640 1 -- 172.21.15.119:0/1695166349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8620155240 con 0x7f86200b5470 2024-04-25T10:13:01.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.261+0000 7f861de00640 1 --2- 172.21.15.119:0/1695166349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86200b5470 0x7f8620149530 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f861000d800 tx=0x7f861000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.262+0000 7f8616a00640 1 -- 172.21.15.119:0/1695166349 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861000b6c0 con 0x7f86200b5470 2024-04-25T10:13:01.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.262+0000 7f8616a00640 1 -- 172.21.15.119:0/1695166349 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8610019070 con 0x7f86200b5470 2024-04-25T10:13:01.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.262+0000 7f8616a00640 1 -- 172.21.15.119:0/1695166349 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861000ba80 con 0x7f86200b5470 2024-04-25T10:13:01.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.262+0000 7f861f200640 1 -- 172.21.15.119:0/1695166349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8620155520 con 0x7f86200b5470 2024-04-25T10:13:01.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.262+0000 7f861f200640 1 -- 172.21.15.119:0/1695166349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8620155ae0 con 0x7f86200b5470 2024-04-25T10:13:01.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.263+0000 7f861f200640 1 -- 172.21.15.119:0/1695166349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86200749b0 con 0x7f86200b5470 2024-04-25T10:13:01.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.264+0000 7f8616a00640 1 -- 172.21.15.119:0/1695166349 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f861000bc20 con 0x7f86200b5470 2024-04-25T10:13:01.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.265+0000 7f8616a00640 1 --2- 172.21.15.119:0/1695166349 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f85fc06b4d0 0x7f85fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.265+0000 7f8616a00640 1 -- 172.21.15.119:0/1695166349 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f861008a750 con 0x7f86200b5470 2024-04-25T10:13:01.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.265+0000 7f8615400640 1 --2- 172.21.15.119:0/1695166349 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f85fc06b4d0 0x7f85fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.266+0000 7f8615400640 1 --2- 172.21.15.119:0/1695166349 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f85fc06b4d0 0x7f85fc06d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f862014a020 tx=0x7f8608009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.269+0000 7f8616a00640 1 -- 172.21.15.119:0/1695166349 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8610059530 con 0x7f86200b5470 2024-04-25T10:13:01.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.496+0000 7f861f200640 1 -- 172.21.15.119:0/1695166349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f86200b2d20 con 0x7f86200b5470 2024-04-25T10:13:01.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.497+0000 7f8616a00640 1 -- 172.21.15.119:0/1695166349 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f8610058e30 con 0x7f86200b5470 2024-04-25T10:13:01.502 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:01.502 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:13:01.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.499+0000 7f861f200640 1 -- 172.21.15.119:0/1695166349 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f85fc06b4d0 msgr2=0x7f85fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.499+0000 7f861f200640 1 --2- 172.21.15.119:0/1695166349 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f85fc06b4d0 0x7f85fc06d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f862014a020 tx=0x7f8608009040 comp rx=0 tx=0).stop 2024-04-25T10:13:01.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.499+0000 7f861f200640 1 -- 172.21.15.119:0/1695166349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86200b5470 msgr2=0x7f8620149530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.499+0000 7f861f200640 1 --2- 172.21.15.119:0/1695166349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86200b5470 0x7f8620149530 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f861000d800 tx=0x7f861000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.499+0000 7f861f200640 1 -- 172.21.15.119:0/1695166349 shutdown_connections 2024-04-25T10:13:01.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.499+0000 7f861f200640 1 --2- 172.21.15.119:0/1695166349 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f85fc06b4d0 0x7f85fc06d990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.499+0000 7f861f200640 1 --2- 172.21.15.119:0/1695166349 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86201f31e0 0x7f86201f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.499+0000 7f861f200640 1 --2- 172.21.15.119:0/1695166349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8620159ca0 0x7f86201f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.499+0000 7f861f200640 1 --2- 172.21.15.119:0/1695166349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86200b5470 0x7f8620149530 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.499+0000 7f861f200640 1 -- 172.21.15.119:0/1695166349 >> 172.21.15.119:0/1695166349 conn(0x7f862006b1d0 msgr2=0x7f8620156960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.500+0000 7f861f200640 1 -- 172.21.15.119:0/1695166349 shutdown_connections 2024-04-25T10:13:01.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.500+0000 7f861f200640 1 -- 172.21.15.119:0/1695166349 wait complete. 2024-04-25T10:13:01.522 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:13:01.523 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:13:01.569 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:01.573 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:01.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.704+0000 7f99d0200640 1 Processor -- start 2024-04-25T10:13:01.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.704+0000 7f99d0200640 1 -- start start 2024-04-25T10:13:01.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.705+0000 7f99d0200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c815a870 0x7f99c80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.705+0000 7f99d0200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f99c806e2e0 0x7f99c815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.705+0000 7f99d0200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c80cdb10 con 0x7f99c80cd580 2024-04-25T10:13:01.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.705+0000 7f99d0200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c80ce720 con 0x7f99c806e2e0 2024-04-25T10:13:01.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.705+0000 7f99d0200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80cddf0 con 0x7f99c815a870 2024-04-25T10:13:01.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.705+0000 7f99cce00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f99c806e2e0 0x7f99c815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.705+0000 7f99cce00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f99c806e2e0 0x7f99c815ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35040/0 (socket says 172.21.15.119:35040) 2024-04-25T10:13:01.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.705+0000 7f99cce00640 1 -- 172.21.15.119:0/3779718958 learned_addr learned my addr 172.21.15.119:0/3779718958 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:01.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.706+0000 7f99cd800640 1 --2- 172.21.15.119:0/3779718958 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c815a870 0x7f99c80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.706+0000 7f99cce00640 1 -- 172.21.15.119:0/3779718958 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c815a870 msgr2=0x7f99c80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.706+0000 7f99cce00640 1 --2- 172.21.15.119:0/3779718958 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c815a870 0x7f99c80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.706+0000 7f99cce00640 1 -- 172.21.15.119:0/3779718958 >> v1:172.21.15.132:6789/0 conn(0x7f99c80cd580 legacy=0x7f99c80cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:01.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.706+0000 7f99cce00640 1 -- 172.21.15.119:0/3779718958 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c80ce0d0 con 0x7f99c806e2e0 2024-04-25T10:13:01.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.707+0000 7f99cce00640 1 --2- 172.21.15.119:0/3779718958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c806e2e0 0x7f99c815ac50 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009870 tx=0x7f99b802c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cd5676969d0c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.707+0000 7f99c7e00640 1 -- 172.21.15.119:0/3779718958 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8038070 con 0x7f99c806e2e0 2024-04-25T10:13:01.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.707+0000 7f99c7e00640 1 -- 172.21.15.119:0/3779718958 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b8031710 con 0x7f99c806e2e0 2024-04-25T10:13:01.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.707+0000 7f99c7e00640 1 -- 172.21.15.119:0/3779718958 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8031ae0 con 0x7f99c806e2e0 2024-04-25T10:13:01.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.708+0000 7f99d0200640 1 -- 172.21.15.119:0/3779718958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c806e2e0 msgr2=0x7f99c815ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.708+0000 7f99d0200640 1 --2- 172.21.15.119:0/3779718958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c806e2e0 0x7f99c815ac50 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009870 tx=0x7f99b802c420 comp rx=0 tx=0).stop 2024-04-25T10:13:01.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.708+0000 7f99d0200640 1 -- 172.21.15.119:0/3779718958 shutdown_connections 2024-04-25T10:13:01.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.708+0000 7f99d0200640 1 --2- 172.21.15.119:0/3779718958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c806e2e0 0x7f99c815ac50 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.708+0000 7f99d0200640 1 --2- 172.21.15.119:0/3779718958 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c815a870 0x7f99c80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.708+0000 7f99d0200640 1 -- 172.21.15.119:0/3779718958 >> 172.21.15.119:0/3779718958 conn(0x7f99c806aba0 msgr2=0x7f99c806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.708+0000 7f99d0200640 1 -- 172.21.15.119:0/3779718958 shutdown_connections 2024-04-25T10:13:01.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.708+0000 7f99d0200640 1 -- 172.21.15.119:0/3779718958 wait complete. 2024-04-25T10:13:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.710+0000 7f99d0200640 1 Processor -- start 2024-04-25T10:13:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.710+0000 7f99d0200640 1 -- start start 2024-04-25T10:13:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99d0200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c806e2e0 0x7f99c81f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99d0200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c80cd580 0x7f99c81ec4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99d0200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c815a870 0x7f99c81ec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99d0200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80cfdc0 con 0x7f99c80cd580 2024-04-25T10:13:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99d0200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80cfc40 con 0x7f99c815a870 2024-04-25T10:13:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99d0200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c815bf10 con 0x7f99c806e2e0 2024-04-25T10:13:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99cce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c80cd580 0x7f99c81ec4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99cce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c80cd580 0x7f99c81ec4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35046/0 (socket says 172.21.15.119:35046) 2024-04-25T10:13:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99cce00640 1 -- 172.21.15.119:0/3864287947 learned_addr learned my addr 172.21.15.119:0/3864287947 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99cd800640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c806e2e0 0x7f99c81f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99ce200640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c815a870 0x7f99c81ec9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99cce00640 1 -- 172.21.15.119:0/3864287947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c806e2e0 msgr2=0x7f99c81f7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99cce00640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c806e2e0 0x7f99c81f7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99cce00640 1 -- 172.21.15.119:0/3864287947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c815a870 msgr2=0x7f99c81ec9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.711+0000 7f99cce00640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c815a870 0x7f99c81ec9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.712+0000 7f99cce00640 1 -- 172.21.15.119:0/3864287947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c81ed010 con 0x7f99c80cd580 2024-04-25T10:13:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.712+0000 7f99cd800640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c806e2e0 0x7f99c81f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.712+0000 7f99cce00640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c80cd580 0x7f99c81ec4a0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f99b8000c00 tx=0x7f99b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.712+0000 7f99c5600640 1 -- 172.21.15.119:0/3864287947 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8038040 con 0x7f99c80cd580 2024-04-25T10:13:01.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.712+0000 7f99c5600640 1 -- 172.21.15.119:0/3864287947 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b8033830 con 0x7f99c80cd580 2024-04-25T10:13:01.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.712+0000 7f99c5600640 1 -- 172.21.15.119:0/3864287947 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8033c00 con 0x7f99c80cd580 2024-04-25T10:13:01.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.712+0000 7f99d0200640 1 -- 172.21.15.119:0/3864287947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c81ed2f0 con 0x7f99c80cd580 2024-04-25T10:13:01.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.712+0000 7f99d0200640 1 -- 172.21.15.119:0/3864287947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c81ed8b0 con 0x7f99c80cd580 2024-04-25T10:13:01.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.713+0000 7f99d0200640 1 -- 172.21.15.119:0/3864287947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c80749b0 con 0x7f99c80cd580 2024-04-25T10:13:01.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.714+0000 7f99c5600640 1 -- 172.21.15.119:0/3864287947 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99b803e430 con 0x7f99c80cd580 2024-04-25T10:13:01.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.715+0000 7f99c5600640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f99ac06b4d0 0x7f99ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:01.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.715+0000 7f99c5600640 1 -- 172.21.15.119:0/3864287947 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f99b80aadb0 con 0x7f99c80cd580 2024-04-25T10:13:01.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.715+0000 7f99cd800640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f99ac06b4d0 0x7f99ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:01.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.716+0000 7f99cd800640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f99ac06b4d0 0x7f99ac06d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f99bc006fd0 tx=0x7f99bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:01.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.720+0000 7f99c5600640 1 -- 172.21.15.119:0/3864287947 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99b80af070 con 0x7f99c80cd580 2024-04-25T10:13:01.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.957+0000 7f99d0200640 1 -- 172.21.15.119:0/3864287947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f99c80cf810 con 0x7f99c80cd580 2024-04-25T10:13:01.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.958+0000 7f99c5600640 1 -- 172.21.15.119:0/3864287947 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) v1 ==== 53+0+73 (secure 0 0 0) 0x7f99b8079a70 con 0x7f99c80cd580 2024-04-25T10:13:01.963 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:13:01.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.960+0000 7f99d0200640 1 -- 172.21.15.119:0/3864287947 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f99ac06b4d0 msgr2=0x7f99ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.960+0000 7f99d0200640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f99ac06b4d0 0x7f99ac06d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f99bc006fd0 tx=0x7f99bc008040 comp rx=0 tx=0).stop 2024-04-25T10:13:01.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.960+0000 7f99d0200640 1 -- 172.21.15.119:0/3864287947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c80cd580 msgr2=0x7f99c81ec4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.960+0000 7f99d0200640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c80cd580 0x7f99c81ec4a0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f99b8000c00 tx=0x7f99b80046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.961+0000 7f99d0200640 1 -- 172.21.15.119:0/3864287947 shutdown_connections 2024-04-25T10:13:01.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.961+0000 7f99d0200640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f99ac06b4d0 0x7f99ac06d990 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.961+0000 7f99d0200640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99c815a870 0x7f99c81ec9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.961+0000 7f99d0200640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c80cd580 0x7f99c81ec4a0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.961+0000 7f99d0200640 1 --2- 172.21.15.119:0/3864287947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99c806e2e0 0x7f99c81f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.961+0000 7f99d0200640 1 -- 172.21.15.119:0/3864287947 >> 172.21.15.119:0/3864287947 conn(0x7f99c806aba0 msgr2=0x7f99c80cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.961+0000 7f99d0200640 1 -- 172.21.15.119:0/3864287947 shutdown_connections 2024-04-25T10:13:01.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:01.961+0000 7f99d0200640 1 -- 172.21.15.119:0/3864287947 wait complete. 2024-04-25T10:13:01.983 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:13:01.983 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:13:01.983 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:13:01.983 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:13:01.983 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:01.983 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:13:01.984 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:13:01.984 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:13:01.984 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:01.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:01.984 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:13:02.033 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:13:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:13:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:13:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:13:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:02.033 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:02.033 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:13:02.034 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:13:02.034 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:02.034 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:13:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:13:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42307sec preferred_lft 42307sec 2024-04-25T10:13:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:13:02.034 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:02.034 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:13:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:13:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:13:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:02.035 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:13:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:02.035 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:13:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:02.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.036 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:13:02.108 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:13:02.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.112 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:13:02.160 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:13:02.160 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:13:02.160 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:13:02.160 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:13:02.160 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:13:02.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.161 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:13:02.161 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:13:02.161 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:13:02.161 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:13:02.161 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:13:02.161 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:13:02.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.303 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:13:02.349 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:13:02.349 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:13:02.350 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:02.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.351 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:13:02.428 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:02.428 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:02.482 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:02.482 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:02.482 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:02.536 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:13:02.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.536 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:02.666 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:02.666 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:02.666 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:13:02.689 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:02.689 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:02.689 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:13:02.689 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:13:02.689 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:02.714 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:02.715 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:02.715 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:13:02.715 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:13:02.715 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:02.715 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:13:02.715 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:13:02.715 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:13:02.716 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:13:02.717 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:13:02.717 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:13:02.717 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:13:02.717 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:13:02.721 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:13:02.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:02.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.794 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:13:02.846 INFO:teuthology.orchestra.run.smithi119.stdout:client5129 2024-04-25T10:13:02.847 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:13:02.847 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:02.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.899 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T10:13:03.002 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:13:03.003 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T10:13:03.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.107+0000 7f8d7fc1ef40 1 Processor -- start 2024-04-25T10:13:03.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.107+0000 7f8d7fc1ef40 1 -- start start 2024-04-25T10:13:03.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.108+0000 7f8d7fc1ef40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558398ff2220 0x558399158630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.108+0000 7f8d7fc1ef40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55839903d080 0x55839903f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.108+0000 7f8d7fc1ef40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5583990c9c70 0x55839903f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.108+0000 7f8d7fc1ef40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583990252f0 con 0x558398ff2220 2024-04-25T10:13:03.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.108+0000 7f8d7fc1ef40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558398f99dd0 con 0x55839903d080 2024-04-25T10:13:03.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.108+0000 7f8d7fc1ef40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558398ff2960 con 0x5583990c9c70 2024-04-25T10:13:03.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.108+0000 7f8d7c200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558398ff2220 0x558399158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.108+0000 7f8d7c200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558398ff2220 0x558399158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35060/0 (socket says 172.21.15.119:35060) 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.108+0000 7f8d7c200640 1 -- 172.21.15.119:0/3672989977 learned_addr learned my addr 172.21.15.119:0/3672989977 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.109+0000 7f8d7cc00640 1 --2- 172.21.15.119:0/3672989977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5583990c9c70 0x55839903f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.109+0000 7f8d7b800640 1 --2- 172.21.15.119:0/3672989977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55839903d080 0x55839903f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.109+0000 7f8d7c200640 1 -- 172.21.15.119:0/3672989977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5583990c9c70 msgr2=0x55839903f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.109+0000 7f8d7c200640 1 --2- 172.21.15.119:0/3672989977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5583990c9c70 0x55839903f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.109+0000 7f8d7c200640 1 -- 172.21.15.119:0/3672989977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55839903d080 msgr2=0x55839903f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.109+0000 7f8d7c200640 1 --2- 172.21.15.119:0/3672989977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55839903d080 0x55839903f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.109+0000 7f8d7c200640 1 -- 172.21.15.119:0/3672989977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583990404f0 con 0x558398ff2220 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.109+0000 7f8d7b800640 1 --2- 172.21.15.119:0/3672989977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55839903d080 0x55839903f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.109+0000 7f8d7c200640 1 --2- 172.21.15.119:0/3672989977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558398ff2220 0x558399158630 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00a470 tx=0x7f8d6c04e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2aa312dfc794b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.110+0000 7f8d7a400640 1 -- 172.21.15.119:0/3672989977 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c065070 con 0x558398ff2220 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.110+0000 7f8d7a400640 1 -- 172.21.15.119:0/3672989977 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c05a040 con 0x558398ff2220 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.110+0000 7f8d7a400640 1 -- 172.21.15.119:0/3672989977 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c059070 con 0x558398ff2220 2024-04-25T10:13:03.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.110+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/3672989977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558398ff2220 msgr2=0x558399158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.110+0000 7f8d7fc1ef40 1 --2- 172.21.15.119:0/3672989977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558398ff2220 0x558399158630 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00a470 tx=0x7f8d6c04e000 comp rx=0 tx=0).stop 2024-04-25T10:13:03.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.110+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/3672989977 shutdown_connections 2024-04-25T10:13:03.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.110+0000 7f8d7fc1ef40 1 --2- 172.21.15.119:0/3672989977 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5583990c9c70 0x55839903f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.110+0000 7f8d7fc1ef40 1 --2- 172.21.15.119:0/3672989977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55839903d080 0x55839903f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.110+0000 7f8d7fc1ef40 1 --2- 172.21.15.119:0/3672989977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558398ff2220 0x558399158630 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.110+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/3672989977 >> 172.21.15.119:0/3672989977 conn(0x558399040070 msgr2=0x5583990ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.110+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/3672989977 shutdown_connections 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.110+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/3672989977 wait complete. 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.111+0000 7f8d7fc1ef40 1 Processor -- start 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.111+0000 7f8d7fc1ef40 1 -- start start 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.112+0000 7f8d7fc1ef40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558398ff2220 0x5583990cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.112+0000 7f8d7fc1ef40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55839903d080 0x5583990cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.112+0000 7f8d7fc1ef40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583990c9c70 0x558399163b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.112+0000 7f8d7fc1ef40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583990c82d0 con 0x5583990c9c70 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.112+0000 7f8d7fc1ef40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583990c8920 con 0x55839903d080 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.112+0000 7f8d7fc1ef40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583990c85b0 con 0x558398ff2220 2024-04-25T10:13:03.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.112+0000 7f8d7cc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583990c9c70 0x558399163b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.112+0000 7f8d7cc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583990c9c70 0x558399163b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35074/0 (socket says 172.21.15.119:35074) 2024-04-25T10:13:03.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.112+0000 7f8d7cc00640 1 -- 172.21.15.119:0/4204360712 learned_addr learned my addr 172.21.15.119:0/4204360712 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:03.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.113+0000 7f8d7b800640 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55839903d080 0x5583990cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.113+0000 7f8d7c200640 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558398ff2220 0x5583990cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.113+0000 7f8d7cc00640 1 -- 172.21.15.119:0/4204360712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558398ff2220 msgr2=0x5583990cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.113+0000 7f8d7cc00640 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558398ff2220 0x5583990cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.113+0000 7f8d7cc00640 1 -- 172.21.15.119:0/4204360712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55839903d080 msgr2=0x5583990cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.113+0000 7f8d7cc00640 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55839903d080 0x5583990cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.113+0000 7f8d7cc00640 1 -- 172.21.15.119:0/4204360712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583990b5dc0 con 0x5583990c9c70 2024-04-25T10:13:03.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.113+0000 7f8d7c200640 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558398ff2220 0x5583990cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:03.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.113+0000 7f8d7cc00640 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583990c9c70 0x558399163b80 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8d74004250 tx=0x7f8d7400d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.113+0000 7f8d79000640 1 -- 172.21.15.119:0/4204360712 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74018070 con 0x5583990c9c70 2024-04-25T10:13:03.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.114+0000 7f8d79000640 1 -- 172.21.15.119:0/4204360712 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d740030d0 con 0x5583990c9c70 2024-04-25T10:13:03.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.114+0000 7f8d79000640 1 -- 172.21.15.119:0/4204360712 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74001540 con 0x5583990c9c70 2024-04-25T10:13:03.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.114+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/4204360712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5583990ba6e0 con 0x5583990c9c70 2024-04-25T10:13:03.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.114+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/4204360712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583990b9940 con 0x5583990c9c70 2024-04-25T10:13:03.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.116+0000 7f8d79000640 1 -- 172.21.15.119:0/4204360712 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d740016e0 con 0x5583990c9c70 2024-04-25T10:13:03.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.117+0000 7f8d79000640 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8d5c069eb0 0x7f8d5c06c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.117+0000 7f8d79000640 1 -- 172.21.15.119:0/4204360712 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f8d74088540 con 0x5583990c9c70 2024-04-25T10:13:03.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.117+0000 7f8d7c200640 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8d5c069eb0 0x7f8d5c06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.117+0000 7f8d7fc1ef40 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5583991645d0 0x5583990b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.117+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/4204360712 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70) v9 -- 0x5583990b1910 con 0x5583991645d0 2024-04-25T10:13:03.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.118+0000 7f8d7c200640 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8d5c069eb0 0x7f8d5c06c370 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c002000 tx=0x7f8d6c051000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.118+0000 7f8d7b800640 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5583991645d0 0x5583990b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.119+0000 7f8d7b800640 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5583991645d0 0x5583990b13d0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.124+0000 7f8d7b800640 1 -- 172.21.15.119:0/4204360712 <== osd.4 v2:172.21.15.132:6800/1740646323 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v70'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8d6800aa50 con 0x5583991645d0 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.124+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/4204360712 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5583991645d0 msgr2=0x5583990b13d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.124+0000 7f8d7fc1ef40 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5583991645d0 0x5583990b13d0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.124+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/4204360712 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8d5c069eb0 msgr2=0x7f8d5c06c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.124+0000 7f8d7fc1ef40 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8d5c069eb0 0x7f8d5c06c370 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c002000 tx=0x7f8d6c051000 comp rx=0 tx=0).stop 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.124+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/4204360712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583990c9c70 msgr2=0x558399163b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.124+0000 7f8d7fc1ef40 1 --2- 172.21.15.119:0/4204360712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583990c9c70 0x558399163b80 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8d74004250 tx=0x7f8d7400d5f0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.124+0000 7f8d7cc00640 1 -- 172.21.15.119:0/4204360712 reap_dead start 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.124+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/4204360712 shutdown_connections 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.124+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/4204360712 >> 172.21.15.119:0/4204360712 conn(0x558399040070 msgr2=0x5583990cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.124+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/4204360712 shutdown_connections 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.124+0000 7f8d7fc1ef40 1 -- 172.21.15.119:0/4204360712 wait complete. 2024-04-25T10:13:03.136 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:13:03.136 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:03.233 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:03.362 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:13:03.362 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:13:03.363 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:03.420 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:13:03.420 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:03.472 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:03.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.666+0000 7fd0bc000640 1 Processor -- start 2024-04-25T10:13:03.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.666+0000 7fd0bc000640 1 -- start start 2024-04-25T10:13:03.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.668+0000 7fd0bc000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd0b415a870 0x7fd0b40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.668+0000 7fd0bc000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b40cfc40 0x7fd0b415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.668+0000 7fd0bc000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b40cdb10 con 0x7fd0b406e2e0 2024-04-25T10:13:03.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.668+0000 7fd0bc000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b40ce720 con 0x7fd0b415a870 2024-04-25T10:13:03.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.668+0000 7fd0bc000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40cddf0 con 0x7fd0b40cfc40 2024-04-25T10:13:03.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.668+0000 7fd0b8c00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd0b406e2e0 0x7fd0b40cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41522/0 (socket says 172.21.15.119:41522) 2024-04-25T10:13:03.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.668+0000 7fd0b8c00640 1 -- 172.21.15.119:0/1029113210 learned_addr learned my addr 172.21.15.119:0/1029113210 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.668+0000 7fd0b9600640 1 --2- 172.21.15.119:0/1029113210 >> v2:172.21.15.132:3300/0 conn(0x7fd0b415a870 0x7fd0b40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0ba000640 1 --2- 172.21.15.119:0/1029113210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b40cfc40 0x7fd0b415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 -- 172.21.15.119:0/1029113210 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800220899 0 0) 0x7fd0b40cdb10 con 0x7fd0b406e2e0 2024-04-25T10:13:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 -- 172.21.15.119:0/1029113210 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd09c003520 con 0x7fd0b406e2e0 2024-04-25T10:13:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 -- 172.21.15.119:0/1029113210 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4169623137 0 0) 0x7fd09c003520 con 0x7fd0b406e2e0 2024-04-25T10:13:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 -- 172.21.15.119:0/1029113210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b40cfc40 msgr2=0x7fd0b415ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 --2- 172.21.15.119:0/1029113210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b40cfc40 0x7fd0b415ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 -- 172.21.15.119:0/1029113210 >> v2:172.21.15.132:3300/0 conn(0x7fd0b415a870 msgr2=0x7fd0b40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 --2- 172.21.15.119:0/1029113210 >> v2:172.21.15.132:3300/0 conn(0x7fd0b415a870 0x7fd0b40cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 -- 172.21.15.119:0/1029113210 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b40ce0d0 con 0x7fd0b406e2e0 2024-04-25T10:13:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 -- 172.21.15.119:0/1029113210 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fd0a0001670 con 0x7fd0b406e2e0 2024-04-25T10:13:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 -- 172.21.15.119:0/1029113210 >> v1:172.21.15.119:6789/0 conn(0x7fd0b406e2e0 legacy=0x7fd0b40cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 --2- 172.21.15.119:0/1029113210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b40cfc40 0x7fd09c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 --2- 172.21.15.119:0/1029113210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd09c003f30 0x7fd09c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 --2- 172.21.15.119:0/1029113210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd09c004d40 0x7fd09c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 -- 172.21.15.119:0/1029113210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40ce720 con 0x7fd09c003f30 2024-04-25T10:13:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 -- 172.21.15.119:0/1029113210 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40cddf0 con 0x7fd09c004d40 2024-04-25T10:13:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b3e00640 1 -- 172.21.15.119:0/1029113210 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c00dbd0 con 0x7fd0b40cfc40 2024-04-25T10:13:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.669+0000 7fd0b8c00640 1 --2- 172.21.15.119:0/1029113210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd09c003f30 0x7fd09c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.670+0000 7fd0ba000640 1 --2- 172.21.15.119:0/1029113210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd09c004d40 0x7fd09c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.670+0000 7fd0b9600640 1 --2- 172.21.15.119:0/1029113210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b40cfc40 0x7fd09c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.670+0000 7fd0b8c00640 1 -- 172.21.15.119:0/1029113210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b40cfc40 msgr2=0x7fd09c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.670+0000 7fd0b8c00640 1 --2- 172.21.15.119:0/1029113210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b40cfc40 0x7fd09c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.670+0000 7fd0b8c00640 1 -- 172.21.15.119:0/1029113210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd09c004d40 msgr2=0x7fd09c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.670+0000 7fd0b8c00640 1 --2- 172.21.15.119:0/1029113210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd09c004d40 0x7fd09c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.670+0000 7fd0b8c00640 1 -- 172.21.15.119:0/1029113210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c003520 con 0x7fd09c003f30 2024-04-25T10:13:03.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.670+0000 7fd0ba000640 1 --2- 172.21.15.119:0/1029113210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd09c004d40 0x7fd09c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:03.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.671+0000 7fd0b8c00640 1 --2- 172.21.15.119:0/1029113210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd09c003f30 0x7fd09c004800 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000c240 tx=0x7fd09c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c897788a7e7b15c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.671+0000 7fd0b3e00640 1 -- 172.21.15.119:0/1029113210 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0007d30 con 0x7fd09c003f30 2024-04-25T10:13:03.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.671+0000 7fd0b3e00640 1 -- 172.21.15.119:0/1029113210 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a0005ad0 con 0x7fd09c003f30 2024-04-25T10:13:03.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.671+0000 7fd0b3e00640 1 -- 172.21.15.119:0/1029113210 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0007070 con 0x7fd09c003f30 2024-04-25T10:13:03.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.672+0000 7fd0bc000640 1 -- 172.21.15.119:0/1029113210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd09c003f30 msgr2=0x7fd09c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.672+0000 7fd0bc000640 1 --2- 172.21.15.119:0/1029113210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd09c003f30 0x7fd09c004800 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000c240 tx=0x7fd09c005830 comp rx=0 tx=0).stop 2024-04-25T10:13:03.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.672+0000 7fd0ba000640 1 -- 172.21.15.119:0/1029113210 reap_dead start 2024-04-25T10:13:03.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.672+0000 7fd0bc000640 1 -- 172.21.15.119:0/1029113210 shutdown_connections 2024-04-25T10:13:03.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.672+0000 7fd0bc000640 1 -- 172.21.15.119:0/1029113210 >> 172.21.15.119:0/1029113210 conn(0x7fd0b406aba0 msgr2=0x7fd0b406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.672+0000 7fd0bc000640 1 -- 172.21.15.119:0/1029113210 shutdown_connections 2024-04-25T10:13:03.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.672+0000 7fd0bc000640 1 -- 172.21.15.119:0/1029113210 wait complete. 2024-04-25T10:13:03.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.674+0000 7fd0bc000640 1 Processor -- start 2024-04-25T10:13:03.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.674+0000 7fd0bc000640 1 -- start start 2024-04-25T10:13:03.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.675+0000 7fd0bc000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd09c003f30 0x7fd0b4145040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.675+0000 7fd0bc000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b415bff0 0x7fd0b415c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.675+0000 7fd0bc000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0b40ac8b0 0x7fd0b40acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.675+0000 7fd0bc000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40cd700 con 0x7fd09c003f30 2024-04-25T10:13:03.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.675+0000 7fd0bc000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40cd580 con 0x7fd0b40ac8b0 2024-04-25T10:13:03.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.675+0000 7fd0bc000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40cd880 con 0x7fd0b415bff0 2024-04-25T10:13:03.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.675+0000 7fd0b9600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd09c003f30 0x7fd0b4145040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.675+0000 7fd0b9600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd09c003f30 0x7fd0b4145040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35104/0 (socket says 172.21.15.119:35104) 2024-04-25T10:13:03.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.675+0000 7fd0b9600640 1 -- 172.21.15.119:0/227642399 learned_addr learned my addr 172.21.15.119:0/227642399 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:03.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.675+0000 7fd0b9600640 1 -- 172.21.15.119:0/227642399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b415bff0 msgr2=0x7fd0b415c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.675+0000 7fd0b9600640 1 --2- 172.21.15.119:0/227642399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b415bff0 0x7fd0b415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.675+0000 7fd0b9600640 1 -- 172.21.15.119:0/227642399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0b40ac8b0 msgr2=0x7fd0b40acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.675+0000 7fd0b9600640 1 --2- 172.21.15.119:0/227642399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0b40ac8b0 0x7fd0b40acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.675+0000 7fd0b9600640 1 -- 172.21.15.119:0/227642399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b4145580 con 0x7fd09c003f30 2024-04-25T10:13:03.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.676+0000 7fd0b9600640 1 --2- 172.21.15.119:0/227642399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd09c003f30 0x7fd0b4145040 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80108e0 tx=0x7fd0a8010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.676+0000 7fd0b1600640 1 -- 172.21.15.119:0/227642399 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a800f040 con 0x7fd09c003f30 2024-04-25T10:13:03.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.676+0000 7fd0b1600640 1 -- 172.21.15.119:0/227642399 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a800eae0 con 0x7fd09c003f30 2024-04-25T10:13:03.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.676+0000 7fd0b1600640 1 -- 172.21.15.119:0/227642399 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a801c400 con 0x7fd09c003f30 2024-04-25T10:13:03.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.676+0000 7fd0bc000640 1 -- 172.21.15.119:0/227642399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b415cb60 con 0x7fd09c003f30 2024-04-25T10:13:03.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.676+0000 7fd0bc000640 1 -- 172.21.15.119:0/227642399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b415d120 con 0x7fd09c003f30 2024-04-25T10:13:03.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.677+0000 7fd0bc000640 1 -- 172.21.15.119:0/227642399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b40749b0 con 0x7fd09c003f30 2024-04-25T10:13:03.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.678+0000 7fd0b1600640 1 -- 172.21.15.119:0/227642399 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0a801c5a0 con 0x7fd09c003f30 2024-04-25T10:13:03.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.679+0000 7fd0b1600640 1 --2- 172.21.15.119:0/227642399 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd09406b4d0 0x7fd09406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.679+0000 7fd0b1600640 1 -- 172.21.15.119:0/227642399 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd0a808a9d0 con 0x7fd09c003f30 2024-04-25T10:13:03.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.680+0000 7fd0b8c00640 1 --2- 172.21.15.119:0/227642399 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd09406b4d0 0x7fd09406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.680+0000 7fd0b8c00640 1 --2- 172.21.15.119:0/227642399 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd09406b4d0 0x7fd09406d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0033000 tx=0x7fd0a0007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.684+0000 7fd0b1600640 1 -- 172.21.15.119:0/227642399 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0b4145580 con 0x7fd09c003f30 2024-04-25T10:13:03.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.880+0000 7fd0bc000640 1 -- 172.21.15.119:0/227642399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0b4158740 con 0x7fd09c003f30 2024-04-25T10:13:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.882+0000 7fd0b1600640 1 -- 172.21.15.119:0/227642399 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16933 (secure 0 0 0) 0x7fd0a8018030 con 0x7fd09c003f30 2024-04-25T10:13:03.888 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:03.888 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":70,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:11:51.402126+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:32.500199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:33.708192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.22","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/2673349178":"2024-04-26T10:11:47.336991+0000","172.21.15.119:6836/2673349178":"2024-04-26T10:11:47.336991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:03.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.884+0000 7fd0bc000640 1 -- 172.21.15.119:0/227642399 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd09406b4d0 msgr2=0x7fd09406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.884+0000 7fd0bc000640 1 --2- 172.21.15.119:0/227642399 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd09406b4d0 0x7fd09406d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0033000 tx=0x7fd0a0007000 comp rx=0 tx=0).stop 2024-04-25T10:13:03.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.884+0000 7fd0bc000640 1 -- 172.21.15.119:0/227642399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd09c003f30 msgr2=0x7fd0b4145040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.884+0000 7fd0bc000640 1 --2- 172.21.15.119:0/227642399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd09c003f30 0x7fd0b4145040 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80108e0 tx=0x7fd0a8010da0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.885+0000 7fd0bc000640 1 -- 172.21.15.119:0/227642399 shutdown_connections 2024-04-25T10:13:03.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.885+0000 7fd0bc000640 1 --2- 172.21.15.119:0/227642399 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd09406b4d0 0x7fd09406d990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.885+0000 7fd0bc000640 1 --2- 172.21.15.119:0/227642399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0b40ac8b0 0x7fd0b40acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.885+0000 7fd0bc000640 1 --2- 172.21.15.119:0/227642399 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0b415bff0 0x7fd0b415c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.885+0000 7fd0bc000640 1 --2- 172.21.15.119:0/227642399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd09c003f30 0x7fd0b4145040 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.885+0000 7fd0bc000640 1 -- 172.21.15.119:0/227642399 >> 172.21.15.119:0/227642399 conn(0x7fd0b406aba0 msgr2=0x7fd0b4158960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.885+0000 7fd0bc000640 1 -- 172.21.15.119:0/227642399 shutdown_connections 2024-04-25T10:13:03.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:03.885+0000 7fd0bc000640 1 -- 172.21.15.119:0/227642399 wait complete. 2024-04-25T10:13:03.910 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:32.500199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:33.708192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:03.910 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:04.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.108+0000 7f86e2a00640 1 Processor -- start 2024-04-25T10:13:04.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.109+0000 7f86e2a00640 1 -- start start 2024-04-25T10:13:04.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.110+0000 7f86e2a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86dc0ada20 0x7f86dc0acb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.110+0000 7f86e2a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f86dc0b36a0 0x7f86dc06b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.110+0000 7f86e2a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86dc15e120 con 0x7f86dc06aba0 2024-04-25T10:13:04.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.110+0000 7f86e2a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86dc15dec0 con 0x7f86dc0b36a0 2024-04-25T10:13:04.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.110+0000 7f86e2a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc15e400 con 0x7f86dc0ada20 2024-04-25T10:13:04.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.111+0000 7f86e1600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86dc0ada20 0x7f86dc0acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.111+0000 7f86e2000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f86dc0b36a0 0x7f86dc06b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.111+0000 7f86e1600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86dc0ada20 0x7f86dc0acb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:44950/0 (socket says 172.21.15.119:44950) 2024-04-25T10:13:04.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.111+0000 7f86e1600640 1 -- 172.21.15.119:0/1967775972 learned_addr learned my addr 172.21.15.119:0/1967775972 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:04.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.112+0000 7f86e1600640 1 -- 172.21.15.119:0/1967775972 >> v2:172.21.15.132:3300/0 conn(0x7f86dc0b36a0 msgr2=0x7f86dc06b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.112+0000 7f86e1600640 1 --2- 172.21.15.119:0/1967775972 >> v2:172.21.15.132:3300/0 conn(0x7f86dc0b36a0 0x7f86dc06b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.112+0000 7f86e1600640 1 -- 172.21.15.119:0/1967775972 >> v1:172.21.15.132:6789/0 conn(0x7f86dc06aba0 legacy=0x7f86dc0b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.112+0000 7f86e1600640 1 -- 172.21.15.119:0/1967775972 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc15e6e0 con 0x7f86dc0ada20 2024-04-25T10:13:04.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.113+0000 7f86e1600640 1 --2- 172.21.15.119:0/1967775972 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86dc0ada20 0x7f86dc0acb20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009870 tx=0x7f86d002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abb0aa37082e1633 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.114+0000 7f86dbe00640 1 -- 172.21.15.119:0/1967775972 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0037070 con 0x7f86dc0ada20 2024-04-25T10:13:04.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.114+0000 7f86dbe00640 1 -- 172.21.15.119:0/1967775972 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d0032440 con 0x7f86dc0ada20 2024-04-25T10:13:04.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.114+0000 7f86dbe00640 1 -- 172.21.15.119:0/1967775972 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0032810 con 0x7f86dc0ada20 2024-04-25T10:13:04.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.114+0000 7f86e2a00640 1 -- 172.21.15.119:0/1967775972 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86dc0ada20 msgr2=0x7f86dc0acb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.114+0000 7f86e2a00640 1 --2- 172.21.15.119:0/1967775972 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86dc0ada20 0x7f86dc0acb20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009870 tx=0x7f86d002a270 comp rx=0 tx=0).stop 2024-04-25T10:13:04.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.115+0000 7f86e2a00640 1 -- 172.21.15.119:0/1967775972 shutdown_connections 2024-04-25T10:13:04.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.115+0000 7f86e2a00640 1 --2- 172.21.15.119:0/1967775972 >> v2:172.21.15.132:3300/0 conn(0x7f86dc0b36a0 0x7f86dc06b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.115+0000 7f86e2a00640 1 --2- 172.21.15.119:0/1967775972 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86dc0ada20 0x7f86dc0acb20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.115+0000 7f86e2a00640 1 -- 172.21.15.119:0/1967775972 >> 172.21.15.119:0/1967775972 conn(0x7f86dc066ff0 msgr2=0x7f86dc0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.115+0000 7f86e2a00640 1 -- 172.21.15.119:0/1967775972 shutdown_connections 2024-04-25T10:13:04.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.115+0000 7f86e2a00640 1 -- 172.21.15.119:0/1967775972 wait complete. 2024-04-25T10:13:04.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.117+0000 7f86e2a00640 1 Processor -- start 2024-04-25T10:13:04.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.117+0000 7f86e2a00640 1 -- start start 2024-04-25T10:13:04.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.118+0000 7f86e2a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86dc06aba0 0x7f86dc13cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.118+0000 7f86e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86dc0ada20 0x7f86dc154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.118+0000 7f86e2a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86dc0b36a0 0x7f86dc154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.118+0000 7f86e2a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc0ade00 con 0x7f86dc0ada20 2024-04-25T10:13:04.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.118+0000 7f86e2a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc15eca0 con 0x7f86dc06aba0 2024-04-25T10:13:04.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.118+0000 7f86e2a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc157700 con 0x7f86dc0b36a0 2024-04-25T10:13:04.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.119+0000 7f86e0c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86dc0ada20 0x7f86dc154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.119+0000 7f86e0c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86dc0ada20 0x7f86dc154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35108/0 (socket says 172.21.15.119:35108) 2024-04-25T10:13:04.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.119+0000 7f86e0c00640 1 -- 172.21.15.119:0/4287719228 learned_addr learned my addr 172.21.15.119:0/4287719228 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:04.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.119+0000 7f86e1600640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86dc06aba0 0x7f86dc13cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.119+0000 7f86e2000640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86dc0b36a0 0x7f86dc154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.119+0000 7f86e0c00640 1 -- 172.21.15.119:0/4287719228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86dc0b36a0 msgr2=0x7f86dc154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.119+0000 7f86e0c00640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86dc0b36a0 0x7f86dc154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.119+0000 7f86e0c00640 1 -- 172.21.15.119:0/4287719228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86dc06aba0 msgr2=0x7f86dc13cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.119+0000 7f86e0c00640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86dc06aba0 0x7f86dc13cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.119+0000 7f86e0c00640 1 -- 172.21.15.119:0/4287719228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc1f4ea0 con 0x7f86dc0ada20 2024-04-25T10:13:04.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.119+0000 7f86e1600640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86dc06aba0 0x7f86dc13cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:04.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.120+0000 7f86e0c00640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86dc0ada20 0x7f86dc154580 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f86c800c430 tx=0x7f86c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.120+0000 7f86d9600640 1 -- 172.21.15.119:0/4287719228 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c80052d0 con 0x7f86dc0ada20 2024-04-25T10:13:04.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.120+0000 7f86d9600640 1 -- 172.21.15.119:0/4287719228 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c8005490 con 0x7f86dc0ada20 2024-04-25T10:13:04.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.120+0000 7f86d9600640 1 -- 172.21.15.119:0/4287719228 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8007260 con 0x7f86dc0ada20 2024-04-25T10:13:04.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.120+0000 7f86e2a00640 1 -- 172.21.15.119:0/4287719228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc155140 con 0x7f86dc0ada20 2024-04-25T10:13:04.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.120+0000 7f86e2a00640 1 -- 172.21.15.119:0/4287719228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc13d340 con 0x7f86dc0ada20 2024-04-25T10:13:04.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.121+0000 7f86e2a00640 1 -- 172.21.15.119:0/4287719228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86dc067490 con 0x7f86dc0ada20 2024-04-25T10:13:04.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.122+0000 7f86d9600640 1 -- 172.21.15.119:0/4287719228 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f86c8007400 con 0x7f86dc0ada20 2024-04-25T10:13:04.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.123+0000 7f86d9600640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f86bc06b4d0 0x7f86bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.124+0000 7f86d9600640 1 -- 172.21.15.119:0/4287719228 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f86c8089820 con 0x7f86dc0ada20 2024-04-25T10:13:04.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.124+0000 7f86e1600640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f86bc06b4d0 0x7f86bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.124+0000 7f86e1600640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f86bc06b4d0 0x7f86bc06d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f86d0034000 tx=0x7f86d002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.128+0000 7f86d9600640 1 -- 172.21.15.119:0/4287719228 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f86dc067490 con 0x7f86dc0ada20 2024-04-25T10:13:04.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.323+0000 7f86e2a00640 1 -- 172.21.15.119:0/4287719228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86dc0749b0 con 0x7f86dc0ada20 2024-04-25T10:13:04.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.323+0000 7f86d9600640 1 -- 172.21.15.119:0/4287719228 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86c8058570 con 0x7f86dc0ada20 2024-04-25T10:13:04.328 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:13:04.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.326+0000 7f86e2a00640 1 -- 172.21.15.119:0/4287719228 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f86bc06b4d0 msgr2=0x7f86bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.326+0000 7f86e2a00640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f86bc06b4d0 0x7f86bc06d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f86d0034000 tx=0x7f86d002e000 comp rx=0 tx=0).stop 2024-04-25T10:13:04.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.326+0000 7f86e2a00640 1 -- 172.21.15.119:0/4287719228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86dc0ada20 msgr2=0x7f86dc154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.326+0000 7f86e2a00640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86dc0ada20 0x7f86dc154580 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f86c800c430 tx=0x7f86c800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.326+0000 7f86e2a00640 1 -- 172.21.15.119:0/4287719228 shutdown_connections 2024-04-25T10:13:04.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.326+0000 7f86e2a00640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f86bc06b4d0 0x7f86bc06d990 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.326+0000 7f86e2a00640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86dc0b36a0 0x7f86dc154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.326+0000 7f86e2a00640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86dc0ada20 0x7f86dc154580 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.326+0000 7f86e2a00640 1 --2- 172.21.15.119:0/4287719228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86dc06aba0 0x7f86dc13cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.326+0000 7f86e2a00640 1 -- 172.21.15.119:0/4287719228 >> 172.21.15.119:0/4287719228 conn(0x7f86dc066ff0 msgr2=0x7f86dc0acb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.326+0000 7f86e2a00640 1 -- 172.21.15.119:0/4287719228 shutdown_connections 2024-04-25T10:13:04.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.326+0000 7f86e2a00640 1 -- 172.21.15.119:0/4287719228 wait complete. 2024-04-25T10:13:04.349 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:04.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.546+0000 7f4a8a600640 1 Processor -- start 2024-04-25T10:13:04.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.547+0000 7f4a8a600640 1 -- start start 2024-04-25T10:13:04.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.548+0000 7f4a8a600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4a8415b7e0 0x7f4a8415e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.548+0000 7f4a8a600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a840b5710 con 0x7f4a840ce860 2024-04-25T10:13:04.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.548+0000 7f4a8a600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a840b59f0 con 0x7f4a840d07b0 2024-04-25T10:13:04.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.548+0000 7f4a8a600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a840b5cd0 con 0x7f4a8415b7e0 2024-04-25T10:13:04.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.548+0000 7f4a83e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4a840ce860 0x7f4a840d00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41536/0 (socket says 172.21.15.119:41536) 2024-04-25T10:13:04.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.548+0000 7f4a83e00640 1 -- 172.21.15.119:0/2693553013 learned_addr learned my addr 172.21.15.119:0/2693553013 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:04.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.549+0000 7f4a89200640 1 --2- 172.21.15.119:0/2693553013 >> v2:172.21.15.132:3300/0 conn(0x7f4a8415b7e0 0x7f4a8415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.549+0000 7f4a83400640 1 -- 172.21.15.119:0/2693553013 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126879036 0 0) 0x7f4a840b5710 con 0x7f4a840ce860 2024-04-25T10:13:04.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.549+0000 7f4a83400640 1 -- 172.21.15.119:0/2693553013 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a68003520 con 0x7f4a840ce860 2024-04-25T10:13:04.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.549+0000 7f4a83400640 1 -- 172.21.15.119:0/2693553013 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 363181395 0 0) 0x7f4a68003520 con 0x7f4a840ce860 2024-04-25T10:13:04.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.549+0000 7f4a83400640 1 -- 172.21.15.119:0/2693553013 >> v2:172.21.15.132:3300/0 conn(0x7f4a8415b7e0 msgr2=0x7f4a8415e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.549+0000 7f4a83400640 1 --2- 172.21.15.119:0/2693553013 >> v2:172.21.15.132:3300/0 conn(0x7f4a8415b7e0 0x7f4a8415e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.549+0000 7f4a83400640 1 -- 172.21.15.119:0/2693553013 >> v1:172.21.15.132:6789/0 conn(0x7f4a840d07b0 legacy=0x7f4a84161960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.549+0000 7f4a83400640 1 -- 172.21.15.119:0/2693553013 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a840b5fb0 con 0x7f4a840ce860 2024-04-25T10:13:04.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.549+0000 7f4a83400640 1 -- 172.21.15.119:0/2693553013 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f4a740019f0 con 0x7f4a840ce860 2024-04-25T10:13:04.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.549+0000 7f4a83400640 1 -- 172.21.15.119:0/2693553013 >> v1:172.21.15.119:6789/0 conn(0x7f4a840ce860 legacy=0x7f4a840d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.549+0000 7f4a83400640 1 --2- 172.21.15.119:0/2693553013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a68003a00 0x7f4a68003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.549+0000 7f4a83400640 1 --2- 172.21.15.119:0/2693553013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a68004800 0x7f4a68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.549+0000 7f4a83400640 1 --2- 172.21.15.119:0/2693553013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a680052e0 0x7f4a6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.550+0000 7f4a83400640 1 -- 172.21.15.119:0/2693553013 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840b5cd0 con 0x7f4a68004800 2024-04-25T10:13:04.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.550+0000 7f4a83400640 1 -- 172.21.15.119:0/2693553013 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6800c1e0 con 0x7f4a680052e0 2024-04-25T10:13:04.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.550+0000 7f4a83400640 1 -- 172.21.15.119:0/2693553013 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6800c3e0 con 0x7f4a68003a00 2024-04-25T10:13:04.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.550+0000 7f4a83e00640 1 --2- 172.21.15.119:0/2693553013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a68004800 0x7f4a68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.550+0000 7f4a89c00640 1 --2- 172.21.15.119:0/2693553013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a680052e0 0x7f4a6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.550+0000 7f4a89200640 1 --2- 172.21.15.119:0/2693553013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a68003a00 0x7f4a68003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.550+0000 7f4a83e00640 1 -- 172.21.15.119:0/2693553013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a68003a00 msgr2=0x7f4a68003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.550+0000 7f4a83e00640 1 --2- 172.21.15.119:0/2693553013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a68003a00 0x7f4a68003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.550+0000 7f4a83e00640 1 -- 172.21.15.119:0/2693553013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a680052e0 msgr2=0x7f4a6800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.550+0000 7f4a83e00640 1 --2- 172.21.15.119:0/2693553013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a680052e0 0x7f4a6800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.550+0000 7f4a83e00640 1 -- 172.21.15.119:0/2693553013 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a68003520 con 0x7f4a68004800 2024-04-25T10:13:04.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.550+0000 7f4a89c00640 1 -- 172.21.15.119:0/2693553013 reap_dead start 2024-04-25T10:13:04.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.551+0000 7f4a83e00640 1 --2- 172.21.15.119:0/2693553013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a68004800 0x7f4a68004c80 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400a7b0 tx=0x7f4a68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e89d1ad3a60828 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.552+0000 7f4a83400640 1 -- 172.21.15.119:0/2693553013 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7400a3d0 con 0x7f4a68004800 2024-04-25T10:13:04.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.552+0000 7f4a83400640 1 -- 172.21.15.119:0/2693553013 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7402f070 con 0x7f4a68004800 2024-04-25T10:13:04.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.552+0000 7f4a83400640 1 -- 172.21.15.119:0/2693553013 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7402bc80 con 0x7f4a68004800 2024-04-25T10:13:04.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.552+0000 7f4a8a600640 1 -- 172.21.15.119:0/2693553013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a68004800 msgr2=0x7f4a68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.552+0000 7f4a8a600640 1 --2- 172.21.15.119:0/2693553013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a68004800 0x7f4a68004c80 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400a7b0 tx=0x7f4a68005830 comp rx=0 tx=0).stop 2024-04-25T10:13:04.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.552+0000 7f4a8a600640 1 -- 172.21.15.119:0/2693553013 shutdown_connections 2024-04-25T10:13:04.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.552+0000 7f4a8a600640 1 --2- 172.21.15.119:0/2693553013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a68004800 0x7f4a68004c80 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.552+0000 7f4a8a600640 1 -- 172.21.15.119:0/2693553013 >> 172.21.15.119:0/2693553013 conn(0x7f4a840722d0 msgr2=0x7f4a840726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.552+0000 7f4a8a600640 1 -- 172.21.15.119:0/2693553013 shutdown_connections 2024-04-25T10:13:04.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.552+0000 7f4a8a600640 1 -- 172.21.15.119:0/2693553013 wait complete. 2024-04-25T10:13:04.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.554+0000 7f4a8a600640 1 Processor -- start 2024-04-25T10:13:04.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.554+0000 7f4a8a600640 1 -- start start 2024-04-25T10:13:04.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.555+0000 7f4a8a600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a68004800 0x7f4a84155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.555+0000 7f4a8a600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a840c7bf0 0x7f4a840b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.555+0000 7f4a8a600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a840b7040 0x7f4a84158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.555+0000 7f4a8a600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840cee70 con 0x7f4a68004800 2024-04-25T10:13:04.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.555+0000 7f4a8a600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840cecf0 con 0x7f4a840b7040 2024-04-25T10:13:04.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.555+0000 7f4a8a600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840ceff0 con 0x7f4a840c7bf0 2024-04-25T10:13:04.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.555+0000 7f4a89200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a68004800 0x7f4a84155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.556+0000 7f4a89200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a68004800 0x7f4a84155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35138/0 (socket says 172.21.15.119:35138) 2024-04-25T10:13:04.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.556+0000 7f4a89200640 1 -- 172.21.15.119:0/3706678931 learned_addr learned my addr 172.21.15.119:0/3706678931 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:04.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.556+0000 7f4a83e00640 1 --2- 172.21.15.119:0/3706678931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a840c7bf0 0x7f4a840b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.556+0000 7f4a89200640 1 -- 172.21.15.119:0/3706678931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a840c7bf0 msgr2=0x7f4a840b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.556+0000 7f4a89200640 1 --2- 172.21.15.119:0/3706678931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a840c7bf0 0x7f4a840b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.556+0000 7f4a89c00640 1 --2- 172.21.15.119:0/3706678931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a840b7040 0x7f4a84158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.556+0000 7f4a89200640 1 -- 172.21.15.119:0/3706678931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a840b7040 msgr2=0x7f4a84158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.556+0000 7f4a89200640 1 --2- 172.21.15.119:0/3706678931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a840b7040 0x7f4a84158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.556+0000 7f4a89200640 1 -- 172.21.15.119:0/3706678931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a840cb6f0 con 0x7f4a68004800 2024-04-25T10:13:04.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.556+0000 7f4a89200640 1 --2- 172.21.15.119:0/3706678931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a68004800 0x7f4a84155180 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0107c0 tx=0x7f4a6c010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.556+0000 7f4a80c00640 1 -- 172.21.15.119:0/3706678931 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00f040 con 0x7f4a68004800 2024-04-25T10:13:04.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.557+0000 7f4a80c00640 1 -- 172.21.15.119:0/3706678931 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6c00eb00 con 0x7f4a68004800 2024-04-25T10:13:04.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.557+0000 7f4a80c00640 1 -- 172.21.15.119:0/3706678931 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c01c400 con 0x7f4a68004800 2024-04-25T10:13:04.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.557+0000 7f4a8a600640 1 -- 172.21.15.119:0/3706678931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a840cd340 con 0x7f4a68004800 2024-04-25T10:13:04.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.557+0000 7f4a8a600640 1 -- 172.21.15.119:0/3706678931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a840cbd20 con 0x7f4a68004800 2024-04-25T10:13:04.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.558+0000 7f4a8a600640 1 -- 172.21.15.119:0/3706678931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a840cd860 con 0x7f4a68004800 2024-04-25T10:13:04.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.559+0000 7f4a80c00640 1 -- 172.21.15.119:0/3706678931 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a6c01c5a0 con 0x7f4a68004800 2024-04-25T10:13:04.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.560+0000 7f4a80c00640 1 --2- 172.21.15.119:0/3706678931 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4a6006b4d0 0x7f4a6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.560+0000 7f4a80c00640 1 -- 172.21.15.119:0/3706678931 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4a6c08a800 con 0x7f4a68004800 2024-04-25T10:13:04.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.560+0000 7f4a83e00640 1 --2- 172.21.15.119:0/3706678931 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4a6006b4d0 0x7f4a6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.561+0000 7f4a83e00640 1 --2- 172.21.15.119:0/3706678931 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4a6006b4d0 0x7f4a6006d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400a780 tx=0x7f4a74009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.564+0000 7f4a80c00640 1 -- 172.21.15.119:0/3706678931 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a840cd860 con 0x7f4a68004800 2024-04-25T10:13:04.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.764+0000 7f4a8a600640 1 -- 172.21.15.119:0/3706678931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a840749b0 con 0x7f4a68004800 2024-04-25T10:13:04.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.765+0000 7f4a80c00640 1 -- 172.21.15.119:0/3706678931 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a6c018030 con 0x7f4a68004800 2024-04-25T10:13:04.770 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:13:04.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.768+0000 7f4a8a600640 1 -- 172.21.15.119:0/3706678931 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4a6006b4d0 msgr2=0x7f4a6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.768+0000 7f4a8a600640 1 --2- 172.21.15.119:0/3706678931 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4a6006b4d0 0x7f4a6006d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400a780 tx=0x7f4a74009030 comp rx=0 tx=0).stop 2024-04-25T10:13:04.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.768+0000 7f4a8a600640 1 -- 172.21.15.119:0/3706678931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a68004800 msgr2=0x7f4a84155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.768+0000 7f4a8a600640 1 --2- 172.21.15.119:0/3706678931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a68004800 0x7f4a84155180 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0107c0 tx=0x7f4a6c010c80 comp rx=0 tx=0).stop 2024-04-25T10:13:04.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.768+0000 7f4a8a600640 1 -- 172.21.15.119:0/3706678931 shutdown_connections 2024-04-25T10:13:04.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.768+0000 7f4a8a600640 1 --2- 172.21.15.119:0/3706678931 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4a6006b4d0 0x7f4a6006d990 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.768+0000 7f4a8a600640 1 --2- 172.21.15.119:0/3706678931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a840b7040 0x7f4a84158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.768+0000 7f4a8a600640 1 --2- 172.21.15.119:0/3706678931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4a840c7bf0 0x7f4a840b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.768+0000 7f4a8a600640 1 --2- 172.21.15.119:0/3706678931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a68004800 0x7f4a84155180 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.768+0000 7f4a8a600640 1 -- 172.21.15.119:0/3706678931 >> 172.21.15.119:0/3706678931 conn(0x7f4a840722d0 msgr2=0x7f4a840d2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.768+0000 7f4a8a600640 1 -- 172.21.15.119:0/3706678931 shutdown_connections 2024-04-25T10:13:04.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.768+0000 7f4a8a600640 1 -- 172.21.15.119:0/3706678931 wait complete. 2024-04-25T10:13:04.791 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:04.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.988+0000 7f72b1800640 1 Processor -- start 2024-04-25T10:13:04.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.988+0000 7f72b1800640 1 -- start start 2024-04-25T10:13:04.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.989+0000 7f72b1800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f72ac0b42c0 0x7f72ac15ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.989+0000 7f72b1800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f72ac15aef0 0x7f72ac0cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.989+0000 7f72b1800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ac15b370 con 0x7f72ac0b46a0 2024-04-25T10:13:04.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.989+0000 7f72b1800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ac0d05d0 con 0x7f72ac15aef0 2024-04-25T10:13:04.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.989+0000 7f72b1800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ac15b650 con 0x7f72ac0b42c0 2024-04-25T10:13:04.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.990+0000 7f72aa000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f72ac15aef0 0x7f72ac0cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.990+0000 7f72ab400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f72ac0b46a0 0x7f72ac1549d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41546/0 (socket says 172.21.15.119:41546) 2024-04-25T10:13:04.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.990+0000 7f72ab400640 1 -- 172.21.15.119:0/4215340981 learned_addr learned my addr 172.21.15.119:0/4215340981 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:04.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.990+0000 7f72aaa00640 1 --2- 172.21.15.119:0/4215340981 >> v2:172.21.15.132:3300/0 conn(0x7f72ac0b42c0 0x7f72ac15ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.990+0000 7f72aa000640 1 -- 172.21.15.119:0/4215340981 >> v2:172.21.15.132:3300/0 conn(0x7f72ac0b42c0 msgr2=0x7f72ac15ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.990+0000 7f72aa000640 1 --2- 172.21.15.119:0/4215340981 >> v2:172.21.15.132:3300/0 conn(0x7f72ac0b42c0 0x7f72ac15ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.990+0000 7f72aa000640 1 -- 172.21.15.119:0/4215340981 >> v1:172.21.15.119:6789/0 conn(0x7f72ac0b46a0 legacy=0x7f72ac1549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.990+0000 7f72aa000640 1 -- 172.21.15.119:0/4215340981 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ac15b930 con 0x7f72ac15aef0 2024-04-25T10:13:04.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.990+0000 7f72aaa00640 1 --2- 172.21.15.119:0/4215340981 >> v2:172.21.15.132:3300/0 conn(0x7f72ac0b42c0 0x7f72ac15ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:04.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.991+0000 7f72aa000640 1 --2- 172.21.15.119:0/4215340981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72ac15aef0 0x7f72ac0cf440 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0d07c0 tx=0x7f729802c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc4fedd155fd253 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.991+0000 7f72a9600640 1 -- 172.21.15.119:0/4215340981 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298039070 con 0x7f72ac15aef0 2024-04-25T10:13:04.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.992+0000 7f72a9600640 1 -- 172.21.15.119:0/4215340981 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7298034640 con 0x7f72ac15aef0 2024-04-25T10:13:04.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.992+0000 7f72a9600640 1 -- 172.21.15.119:0/4215340981 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298034a10 con 0x7f72ac15aef0 2024-04-25T10:13:04.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.992+0000 7f72b1800640 1 -- 172.21.15.119:0/4215340981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72ac15aef0 msgr2=0x7f72ac0cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.992+0000 7f72b1800640 1 --2- 172.21.15.119:0/4215340981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72ac15aef0 0x7f72ac0cf440 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0d07c0 tx=0x7f729802c320 comp rx=0 tx=0).stop 2024-04-25T10:13:04.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.992+0000 7f72b1800640 1 -- 172.21.15.119:0/4215340981 shutdown_connections 2024-04-25T10:13:04.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.992+0000 7f72b1800640 1 --2- 172.21.15.119:0/4215340981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72ac15aef0 0x7f72ac0cf440 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.992+0000 7f72b1800640 1 --2- 172.21.15.119:0/4215340981 >> v2:172.21.15.132:3300/0 conn(0x7f72ac0b42c0 0x7f72ac15ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.992+0000 7f72b1800640 1 -- 172.21.15.119:0/4215340981 >> 172.21.15.119:0/4215340981 conn(0x7f72ac06aba0 msgr2=0x7f72ac06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.992+0000 7f72b1800640 1 -- 172.21.15.119:0/4215340981 shutdown_connections 2024-04-25T10:13:04.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.992+0000 7f72b1800640 1 -- 172.21.15.119:0/4215340981 wait complete. 2024-04-25T10:13:04.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.994+0000 7f72b1800640 1 Processor -- start 2024-04-25T10:13:04.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.994+0000 7f72b1800640 1 -- start start 2024-04-25T10:13:05.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.995+0000 7f72b1800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72ac0b42c0 0x7f72ac0c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.995+0000 7f72b1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72ac0b46a0 0x7f72ac151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.995+0000 7f72b1800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72ac15aef0 0x7f72ac142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.995+0000 7f72b1800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac06eae0 con 0x7f72ac0b46a0 2024-04-25T10:13:05.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.995+0000 7f72b1800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac0cf980 con 0x7f72ac15aef0 2024-04-25T10:13:05.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.995+0000 7f72b1800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac06ec60 con 0x7f72ac0b42c0 2024-04-25T10:13:05.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.995+0000 7f72aa000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72ac0b46a0 0x7f72ac151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.995+0000 7f72aa000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72ac0b46a0 0x7f72ac151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35156/0 (socket says 172.21.15.119:35156) 2024-04-25T10:13:05.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.995+0000 7f72aa000640 1 -- 172.21.15.119:0/143031775 learned_addr learned my addr 172.21.15.119:0/143031775 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:05.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.996+0000 7f72aa000640 1 -- 172.21.15.119:0/143031775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72ac0b42c0 msgr2=0x7f72ac0c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.996+0000 7f72aa000640 1 --2- 172.21.15.119:0/143031775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72ac0b42c0 0x7f72ac0c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.996+0000 7f72ab400640 1 --2- 172.21.15.119:0/143031775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72ac15aef0 0x7f72ac142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.996+0000 7f72aa000640 1 -- 172.21.15.119:0/143031775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72ac15aef0 msgr2=0x7f72ac142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.996+0000 7f72aa000640 1 --2- 172.21.15.119:0/143031775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72ac15aef0 0x7f72ac142200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.996+0000 7f72aa000640 1 -- 172.21.15.119:0/143031775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ac151fd0 con 0x7f72ac0b46a0 2024-04-25T10:13:05.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.996+0000 7f72aa000640 1 --2- 172.21.15.119:0/143031775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72ac0b46a0 0x7f72ac151940 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7298000c00 tx=0x7f72980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.996+0000 7f72a2a00640 1 -- 172.21.15.119:0/143031775 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298039070 con 0x7f72ac0b46a0 2024-04-25T10:13:05.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.996+0000 7f72a2a00640 1 -- 172.21.15.119:0/143031775 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7298032980 con 0x7f72ac0b46a0 2024-04-25T10:13:05.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.996+0000 7f72b1800640 1 -- 172.21.15.119:0/143031775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ac1522b0 con 0x7f72ac0b46a0 2024-04-25T10:13:05.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.996+0000 7f72a2a00640 1 -- 172.21.15.119:0/143031775 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298032d50 con 0x7f72ac0b46a0 2024-04-25T10:13:05.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.996+0000 7f72b1800640 1 -- 172.21.15.119:0/143031775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ac152870 con 0x7f72ac0b46a0 2024-04-25T10:13:05.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.998+0000 7f72b1800640 1 -- 172.21.15.119:0/143031775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ac0750b0 con 0x7f72ac0b46a0 2024-04-25T10:13:05.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.999+0000 7f72a2a00640 1 -- 172.21.15.119:0/143031775 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f729803f440 con 0x7f72ac0b46a0 2024-04-25T10:13:05.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:04.999+0000 7f72a2a00640 1 --2- 172.21.15.119:0/143031775 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f728c06b4d0 0x7f728c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.000+0000 7f72a2a00640 1 -- 172.21.15.119:0/143031775 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f72980aaf30 con 0x7f72ac0b46a0 2024-04-25T10:13:05.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.000+0000 7f72aaa00640 1 --2- 172.21.15.119:0/143031775 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f728c06b4d0 0x7f728c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.001+0000 7f72aaa00640 1 --2- 172.21.15.119:0/143031775 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f728c06b4d0 0x7f728c06d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f729c006fd0 tx=0x7f729c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.004+0000 7f72a2a00640 1 -- 172.21.15.119:0/143031775 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7298079c80 con 0x7f72ac0b46a0 2024-04-25T10:13:05.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.198+0000 7f72b1800640 1 -- 172.21.15.119:0/143031775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72ac06e700 con 0x7f72ac0b46a0 2024-04-25T10:13:05.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.199+0000 7f72a2a00640 1 -- 172.21.15.119:0/143031775 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7298079580 con 0x7f72ac0b46a0 2024-04-25T10:13:05.204 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:13:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.201+0000 7f72b1800640 1 -- 172.21.15.119:0/143031775 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f728c06b4d0 msgr2=0x7f728c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.201+0000 7f72b1800640 1 --2- 172.21.15.119:0/143031775 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f728c06b4d0 0x7f728c06d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f729c006fd0 tx=0x7f729c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.201+0000 7f72b1800640 1 -- 172.21.15.119:0/143031775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72ac0b46a0 msgr2=0x7f72ac151940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.201+0000 7f72b1800640 1 --2- 172.21.15.119:0/143031775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72ac0b46a0 0x7f72ac151940 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7298000c00 tx=0x7f72980046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.202+0000 7f72b1800640 1 -- 172.21.15.119:0/143031775 shutdown_connections 2024-04-25T10:13:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.202+0000 7f72b1800640 1 --2- 172.21.15.119:0/143031775 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f728c06b4d0 0x7f728c06d990 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.202+0000 7f72b1800640 1 --2- 172.21.15.119:0/143031775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72ac15aef0 0x7f72ac142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.202+0000 7f72b1800640 1 --2- 172.21.15.119:0/143031775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72ac0b46a0 0x7f72ac151940 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.202+0000 7f72b1800640 1 --2- 172.21.15.119:0/143031775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72ac0b42c0 0x7f72ac0c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.202+0000 7f72b1800640 1 -- 172.21.15.119:0/143031775 >> 172.21.15.119:0/143031775 conn(0x7f72ac06aba0 msgr2=0x7f72ac15c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.202+0000 7f72b1800640 1 -- 172.21.15.119:0/143031775 shutdown_connections 2024-04-25T10:13:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.202+0000 7f72b1800640 1 -- 172.21.15.119:0/143031775 wait complete. 2024-04-25T10:13:05.225 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:05.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.415+0000 7fd2eec00640 1 Processor -- start 2024-04-25T10:13:05.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.416+0000 7fd2eec00640 1 -- start start 2024-04-25T10:13:05.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.417+0000 7fd2eec00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd2e815b7e0 0x7fd2e815e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.417+0000 7fd2eec00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2e80ce860 0x7fd2e80cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.417+0000 7fd2eec00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e80b5710 con 0x7fd2e80d00a0 2024-04-25T10:13:05.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.417+0000 7fd2eec00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e80b59f0 con 0x7fd2e80ce860 2024-04-25T10:13:05.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.417+0000 7fd2eec00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e80b5cd0 con 0x7fd2e815b7e0 2024-04-25T10:13:05.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.417+0000 7fd2ece00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2e80ce860 0x7fd2e80cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.417+0000 7fd2ee200640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd2e80d00a0 0x7fd2e80d0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41550/0 (socket says 172.21.15.119:41550) 2024-04-25T10:13:05.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.417+0000 7fd2ee200640 1 -- 172.21.15.119:0/911083518 learned_addr learned my addr 172.21.15.119:0/911083518 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:05.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.418+0000 7fd2ed800640 1 --2- 172.21.15.119:0/911083518 >> v2:172.21.15.132:3300/0 conn(0x7fd2e815b7e0 0x7fd2e815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.418+0000 7fd2ece00640 1 -- 172.21.15.119:0/911083518 >> v2:172.21.15.132:3300/0 conn(0x7fd2e815b7e0 msgr2=0x7fd2e815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:05.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.418+0000 7fd2ece00640 1 --2- 172.21.15.119:0/911083518 >> v2:172.21.15.132:3300/0 conn(0x7fd2e815b7e0 0x7fd2e815e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.418+0000 7fd2ece00640 1 -- 172.21.15.119:0/911083518 >> v1:172.21.15.119:6789/0 conn(0x7fd2e80d00a0 legacy=0x7fd2e80d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.418+0000 7fd2ece00640 1 -- 172.21.15.119:0/911083518 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e80b5fb0 con 0x7fd2e80ce860 2024-04-25T10:13:05.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.418+0000 7fd2e7e00640 1 -- 172.21.15.119:0/911083518 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798034416 0 0) 0x7fd2e80b5710 con 0x7fd2e80d00a0 2024-04-25T10:13:05.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.419+0000 7fd2ece00640 1 --2- 172.21.15.119:0/911083518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2e80ce860 0x7fd2e80cecf0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd2e808bac0 tx=0x7fd2d402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483cdffc0838038a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.419+0000 7fd2e7e00640 1 -- 172.21.15.119:0/911083518 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4037070 con 0x7fd2e80ce860 2024-04-25T10:13:05.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.419+0000 7fd2e7e00640 1 -- 172.21.15.119:0/911083518 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d402add0 con 0x7fd2e80ce860 2024-04-25T10:13:05.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.419+0000 7fd2e7e00640 1 -- 172.21.15.119:0/911083518 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4032660 con 0x7fd2e80ce860 2024-04-25T10:13:05.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.420+0000 7fd2eec00640 1 -- 172.21.15.119:0/911083518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2e80ce860 msgr2=0x7fd2e80cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.420+0000 7fd2eec00640 1 --2- 172.21.15.119:0/911083518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2e80ce860 0x7fd2e80cecf0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd2e808bac0 tx=0x7fd2d402a080 comp rx=0 tx=0).stop 2024-04-25T10:13:05.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.420+0000 7fd2eec00640 1 -- 172.21.15.119:0/911083518 shutdown_connections 2024-04-25T10:13:05.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.420+0000 7fd2eec00640 1 --2- 172.21.15.119:0/911083518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2e80ce860 0x7fd2e80cecf0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.420+0000 7fd2eec00640 1 --2- 172.21.15.119:0/911083518 >> v2:172.21.15.132:3300/0 conn(0x7fd2e815b7e0 0x7fd2e815e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.420+0000 7fd2eec00640 1 -- 172.21.15.119:0/911083518 >> 172.21.15.119:0/911083518 conn(0x7fd2e80722d0 msgr2=0x7fd2e80726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.420+0000 7fd2eec00640 1 -- 172.21.15.119:0/911083518 shutdown_connections 2024-04-25T10:13:05.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.420+0000 7fd2eec00640 1 -- 172.21.15.119:0/911083518 wait complete. 2024-04-25T10:13:05.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.422+0000 7fd2eec00640 1 Processor -- start 2024-04-25T10:13:05.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.422+0000 7fd2eec00640 1 -- start start 2024-04-25T10:13:05.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2eec00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2e80ce860 0x7fd2e80cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2eec00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2e80d00a0 0x7fd2e81f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2eec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2e815b7e0 0x7fd2e81585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2eec00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e8161960 con 0x7fd2e80d00a0 2024-04-25T10:13:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2eec00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80b53c0 con 0x7fd2e80ce860 2024-04-25T10:13:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2eec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e8161ae0 con 0x7fd2e815b7e0 2024-04-25T10:13:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2ece00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2e80d00a0 0x7fd2e81f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2ece00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2e80d00a0 0x7fd2e81f7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35180/0 (socket says 172.21.15.119:35180) 2024-04-25T10:13:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2ece00640 1 -- 172.21.15.119:0/4076264114 learned_addr learned my addr 172.21.15.119:0/4076264114 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2ed800640 1 --2- 172.21.15.119:0/4076264114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2e80ce860 0x7fd2e80cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2ece00640 1 -- 172.21.15.119:0/4076264114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2e815b7e0 msgr2=0x7fd2e81585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2ece00640 1 --2- 172.21.15.119:0/4076264114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2e815b7e0 0x7fd2e81585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2ece00640 1 -- 172.21.15.119:0/4076264114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2e80ce860 msgr2=0x7fd2e80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2ece00640 1 --2- 172.21.15.119:0/4076264114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2e80ce860 0x7fd2e80cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.423+0000 7fd2ece00640 1 -- 172.21.15.119:0/4076264114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e815f760 con 0x7fd2e80d00a0 2024-04-25T10:13:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.424+0000 7fd2ece00640 1 --2- 172.21.15.119:0/4076264114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2e80d00a0 0x7fd2e81f7310 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4000c00 tx=0x7fd2d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.424+0000 7fd2e5600640 1 -- 172.21.15.119:0/4076264114 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4037040 con 0x7fd2e80d00a0 2024-04-25T10:13:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.424+0000 7fd2e5600640 1 -- 172.21.15.119:0/4076264114 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d4042070 con 0x7fd2e80d00a0 2024-04-25T10:13:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.424+0000 7fd2e5600640 1 -- 172.21.15.119:0/4076264114 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d403b890 con 0x7fd2e80d00a0 2024-04-25T10:13:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.424+0000 7fd2eec00640 1 -- 172.21.15.119:0/4076264114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e815fa40 con 0x7fd2e80d00a0 2024-04-25T10:13:05.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.424+0000 7fd2eec00640 1 -- 172.21.15.119:0/4076264114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e8160000 con 0x7fd2e80d00a0 2024-04-25T10:13:05.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.425+0000 7fd2eec00640 1 -- 172.21.15.119:0/4076264114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e80b51e0 con 0x7fd2e80d00a0 2024-04-25T10:13:05.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.426+0000 7fd2e5600640 1 -- 172.21.15.119:0/4076264114 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2d402ac50 con 0x7fd2e80d00a0 2024-04-25T10:13:05.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.427+0000 7fd2e5600640 1 --2- 172.21.15.119:0/4076264114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd2c806b4d0 0x7fd2c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.427+0000 7fd2e5600640 1 -- 172.21.15.119:0/4076264114 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd2d40aaa60 con 0x7fd2e80d00a0 2024-04-25T10:13:05.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.428+0000 7fd2ed800640 1 --2- 172.21.15.119:0/4076264114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd2c806b4d0 0x7fd2c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.428+0000 7fd2ed800640 1 --2- 172.21.15.119:0/4076264114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd2c806b4d0 0x7fd2c806d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8006fd0 tx=0x7fd2d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.432+0000 7fd2e5600640 1 -- 172.21.15.119:0/4076264114 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2e80b51e0 con 0x7fd2e80d00a0 2024-04-25T10:13:05.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.640+0000 7fd2eec00640 1 -- 172.21.15.119:0/4076264114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd2e80749b0 con 0x7fd2e80d00a0 2024-04-25T10:13:05.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.641+0000 7fd2e5600640 1 -- 172.21.15.119:0/4076264114 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v70) v1 ==== 80+0+124 (secure 0 0 0) 0x7fd2d4079960 con 0x7fd2e80d00a0 2024-04-25T10:13:05.646 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6837/2673349178 2024-04-26T10:11:47.336991+0000 2024-04-25T10:13:05.646 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6836/2673349178 2024-04-26T10:11:47.336991+0000 2024-04-25T10:13:05.646 INFO:teuthology.orchestra.run.smithi119.stderr:listed 2 entries 2024-04-25T10:13:05.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.643+0000 7fd2eec00640 1 -- 172.21.15.119:0/4076264114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd2c806b4d0 msgr2=0x7fd2c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.643+0000 7fd2eec00640 1 --2- 172.21.15.119:0/4076264114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd2c806b4d0 0x7fd2c806d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8006fd0 tx=0x7fd2d8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.643+0000 7fd2eec00640 1 -- 172.21.15.119:0/4076264114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2e80d00a0 msgr2=0x7fd2e81f7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.644+0000 7fd2eec00640 1 --2- 172.21.15.119:0/4076264114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2e80d00a0 0x7fd2e81f7310 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4000c00 tx=0x7fd2d40046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.644+0000 7fd2eec00640 1 -- 172.21.15.119:0/4076264114 shutdown_connections 2024-04-25T10:13:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.644+0000 7fd2eec00640 1 --2- 172.21.15.119:0/4076264114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd2c806b4d0 0x7fd2c806d990 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.644+0000 7fd2eec00640 1 --2- 172.21.15.119:0/4076264114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2e815b7e0 0x7fd2e81585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.644+0000 7fd2eec00640 1 --2- 172.21.15.119:0/4076264114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2e80d00a0 0x7fd2e81f7310 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.644+0000 7fd2eec00640 1 --2- 172.21.15.119:0/4076264114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2e80ce860 0x7fd2e80cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.644+0000 7fd2eec00640 1 -- 172.21.15.119:0/4076264114 >> 172.21.15.119:0/4076264114 conn(0x7fd2e80722d0 msgr2=0x7fd2e815acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.644+0000 7fd2eec00640 1 -- 172.21.15.119:0/4076264114 shutdown_connections 2024-04-25T10:13:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:05.644+0000 7fd2eec00640 1 -- 172.21.15.119:0/4076264114 wait complete. 2024-04-25T10:13:05.669 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:13:05.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:05.669 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:05.761 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:13:05.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:05.761 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:05.809 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:13:05.809 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:05.867 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:05.867 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:13:05.867 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:05.927 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:05.927 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:13:05.927 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:05.986 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:05.987 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:06.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.187+0000 7fd04be00640 1 Processor -- start 2024-04-25T10:13:06.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.187+0000 7fd04be00640 1 -- start start 2024-04-25T10:13:06.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.189+0000 7fd04be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd04c15db60 0x7fd04c0d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.189+0000 7fd04be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd04c0b9270 0x7fd04c0d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.189+0000 7fd04be00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04c15ef10 con 0x7fd04c0b5360 2024-04-25T10:13:06.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.189+0000 7fd04be00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c15e390 con 0x7fd04c15db60 2024-04-25T10:13:06.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.189+0000 7fd04be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c15e670 con 0x7fd04c0b9270 2024-04-25T10:13:06.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.190+0000 7fd04aa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd04c15db60 0x7fd04c0d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.190+0000 7fd04b400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd04c0b5360 0x7fd04c0b57d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41566/0 (socket says 172.21.15.119:41566) 2024-04-25T10:13:06.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.190+0000 7fd04b400640 1 -- 172.21.15.119:0/1992463805 learned_addr learned my addr 172.21.15.119:0/1992463805 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:06.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.190+0000 7fd042000640 1 --2- 172.21.15.119:0/1992463805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd04c0b9270 0x7fd04c0d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.190+0000 7fd04aa00640 1 -- 172.21.15.119:0/1992463805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd04c0b9270 msgr2=0x7fd04c0d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:06.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.190+0000 7fd04aa00640 1 --2- 172.21.15.119:0/1992463805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd04c0b9270 0x7fd04c0d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.190+0000 7fd04aa00640 1 -- 172.21.15.119:0/1992463805 >> v1:172.21.15.119:6789/0 conn(0x7fd04c0b5360 legacy=0x7fd04c0b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.190+0000 7fd04aa00640 1 -- 172.21.15.119:0/1992463805 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c15e950 con 0x7fd04c15db60 2024-04-25T10:13:06.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.190+0000 7fd04a000640 1 -- 172.21.15.119:0/1992463805 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309945391 0 0) 0x7fd04c15ef10 con 0x7fd04c0b5360 2024-04-25T10:13:06.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.191+0000 7fd04aa00640 1 --2- 172.21.15.119:0/1992463805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd04c15db60 0x7fd04c0d3870 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0097a0 tx=0x7fd03c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc413f6c4e7fc4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.191+0000 7fd04a000640 1 -- 172.21.15.119:0/1992463805 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c037070 con 0x7fd04c15db60 2024-04-25T10:13:06.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.191+0000 7fd04a000640 1 -- 172.21.15.119:0/1992463805 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c032440 con 0x7fd04c15db60 2024-04-25T10:13:06.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.192+0000 7fd04a000640 1 -- 172.21.15.119:0/1992463805 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c032810 con 0x7fd04c15db60 2024-04-25T10:13:06.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.192+0000 7fd04be00640 1 -- 172.21.15.119:0/1992463805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd04c15db60 msgr2=0x7fd04c0d3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.192+0000 7fd04be00640 1 --2- 172.21.15.119:0/1992463805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd04c15db60 0x7fd04c0d3870 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0097a0 tx=0x7fd03c02a180 comp rx=0 tx=0).stop 2024-04-25T10:13:06.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.192+0000 7fd04be00640 1 -- 172.21.15.119:0/1992463805 shutdown_connections 2024-04-25T10:13:06.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.192+0000 7fd04be00640 1 --2- 172.21.15.119:0/1992463805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd04c0b9270 0x7fd04c0d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.192+0000 7fd04be00640 1 --2- 172.21.15.119:0/1992463805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd04c15db60 0x7fd04c0d3870 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.192+0000 7fd04be00640 1 -- 172.21.15.119:0/1992463805 >> 172.21.15.119:0/1992463805 conn(0x7fd04c0726c0 msgr2=0x7fd04c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:06.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.192+0000 7fd04be00640 1 -- 172.21.15.119:0/1992463805 shutdown_connections 2024-04-25T10:13:06.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.192+0000 7fd04be00640 1 -- 172.21.15.119:0/1992463805 wait complete. 2024-04-25T10:13:06.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.194+0000 7fd04be00640 1 Processor -- start 2024-04-25T10:13:06.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.194+0000 7fd04be00640 1 -- start start 2024-04-25T10:13:06.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.195+0000 7fd04be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd04c0b5360 0x7fd04c15bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.195+0000 7fd04be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd04c0b9270 0x7fd04c15b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.195+0000 7fd04be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd04c15db60 0x7fd04c0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.195+0000 7fd04be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0b3120 con 0x7fd04c0b9270 2024-04-25T10:13:06.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.195+0000 7fd04be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0b2fa0 con 0x7fd04c15db60 2024-04-25T10:13:06.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.195+0000 7fd04be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c15df40 con 0x7fd04c0b5360 2024-04-25T10:13:06.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.195+0000 7fd042000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd04c0b9270 0x7fd04c15b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.196+0000 7fd042000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd04c0b9270 0x7fd04c15b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35194/0 (socket says 172.21.15.119:35194) 2024-04-25T10:13:06.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.196+0000 7fd042000640 1 -- 172.21.15.119:0/106793505 learned_addr learned my addr 172.21.15.119:0/106793505 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:06.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.196+0000 7fd04aa00640 1 --2- 172.21.15.119:0/106793505 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd04c0b5360 0x7fd04c15bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.196+0000 7fd042000640 1 -- 172.21.15.119:0/106793505 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd04c0b5360 msgr2=0x7fd04c15bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.196+0000 7fd042000640 1 --2- 172.21.15.119:0/106793505 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd04c0b5360 0x7fd04c15bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.196+0000 7fd04b400640 1 --2- 172.21.15.119:0/106793505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd04c15db60 0x7fd04c0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.196+0000 7fd042000640 1 -- 172.21.15.119:0/106793505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd04c15db60 msgr2=0x7fd04c0ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.196+0000 7fd042000640 1 --2- 172.21.15.119:0/106793505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd04c15db60 0x7fd04c0ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.196+0000 7fd042000640 1 -- 172.21.15.119:0/106793505 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c15b880 con 0x7fd04c0b9270 2024-04-25T10:13:06.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.196+0000 7fd042000640 1 --2- 172.21.15.119:0/106793505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd04c0b9270 0x7fd04c15b340 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd03400d830 tx=0x7fd03400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.197+0000 7fd043400640 1 -- 172.21.15.119:0/106793505 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03400b6c0 con 0x7fd04c0b9270 2024-04-25T10:13:06.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.197+0000 7fd043400640 1 -- 172.21.15.119:0/106793505 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd034019070 con 0x7fd04c0b9270 2024-04-25T10:13:06.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.197+0000 7fd043400640 1 -- 172.21.15.119:0/106793505 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03400baa0 con 0x7fd04c0b9270 2024-04-25T10:13:06.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.197+0000 7fd04be00640 1 -- 172.21.15.119:0/106793505 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c0ce450 con 0x7fd04c0b9270 2024-04-25T10:13:06.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.197+0000 7fd04be00640 1 -- 172.21.15.119:0/106793505 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c0cc540 con 0x7fd04c0b9270 2024-04-25T10:13:06.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.198+0000 7fd04be00640 1 -- 172.21.15.119:0/106793505 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04c0749b0 con 0x7fd04c0b9270 2024-04-25T10:13:06.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.199+0000 7fd043400640 1 -- 172.21.15.119:0/106793505 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd03401e460 con 0x7fd04c0b9270 2024-04-25T10:13:06.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.200+0000 7fd043400640 1 --2- 172.21.15.119:0/106793505 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd02806b4d0 0x7fd02806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.200+0000 7fd043400640 1 -- 172.21.15.119:0/106793505 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd03408aa00 con 0x7fd04c0b9270 2024-04-25T10:13:06.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.200+0000 7fd04aa00640 1 --2- 172.21.15.119:0/106793505 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd02806b4d0 0x7fd02806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.201+0000 7fd04aa00640 1 --2- 172.21.15.119:0/106793505 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd02806b4d0 0x7fd02806d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd03c034000 tx=0x7fd03c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.204+0000 7fd043400640 1 -- 172.21.15.119:0/106793505 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd034059750 con 0x7fd04c0b9270 2024-04-25T10:13:06.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.434+0000 7fd04be00640 1 -- 172.21.15.119:0/106793505 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd04c0d1a60 con 0x7fd04c0b9270 2024-04-25T10:13:06.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.435+0000 7fd043400640 1 -- 172.21.15.119:0/106793505 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4951 (secure 0 0 0) 0x7fd034059050 con 0x7fd04c0b9270 2024-04-25T10:13:06.440 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:06.440 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2798722683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2798722683},{"type":"v1","addr":"172.21.15.132:6833","nonce":2798722683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2834911408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2834911408},{"type":"v1","addr":"172.21.15.132:6835","nonce":2834911408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/643040335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":643040335},{"type":"v1","addr":"172.21.15.119:6835","nonce":643040335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:34.712290+0000","modified":"2024-04-25T10:12:34.407591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6837/2127292269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2127292269},{"type":"v1","addr":"172.21.15.119:6837","nonce":2127292269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:13:06.440 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 52 2024-04-25T10:13:06.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.438+0000 7fd04be00640 1 -- 172.21.15.119:0/106793505 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd02806b4d0 msgr2=0x7fd02806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.438+0000 7fd04be00640 1 --2- 172.21.15.119:0/106793505 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd02806b4d0 0x7fd02806d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd03c034000 tx=0x7fd03c02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:06.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.438+0000 7fd04be00640 1 -- 172.21.15.119:0/106793505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd04c0b9270 msgr2=0x7fd04c15b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.438+0000 7fd04be00640 1 --2- 172.21.15.119:0/106793505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd04c0b9270 0x7fd04c15b340 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd03400d830 tx=0x7fd03400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.438+0000 7fd04be00640 1 -- 172.21.15.119:0/106793505 shutdown_connections 2024-04-25T10:13:06.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.438+0000 7fd04be00640 1 --2- 172.21.15.119:0/106793505 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd02806b4d0 0x7fd02806d990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.438+0000 7fd04be00640 1 --2- 172.21.15.119:0/106793505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd04c15db60 0x7fd04c0ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.438+0000 7fd04be00640 1 --2- 172.21.15.119:0/106793505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd04c0b9270 0x7fd04c15b340 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.438+0000 7fd04be00640 1 --2- 172.21.15.119:0/106793505 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd04c0b5360 0x7fd04c15bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.438+0000 7fd04be00640 1 -- 172.21.15.119:0/106793505 >> 172.21.15.119:0/106793505 conn(0x7fd04c0726c0 msgr2=0x7fd04c0d1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:06.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.438+0000 7fd04be00640 1 -- 172.21.15.119:0/106793505 shutdown_connections 2024-04-25T10:13:06.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.438+0000 7fd04be00640 1 -- 172.21.15.119:0/106793505 wait complete. 2024-04-25T10:13:06.463 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:06.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.654+0000 7f30df200640 1 Processor -- start 2024-04-25T10:13:06.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.654+0000 7f30df200640 1 -- start start 2024-04-25T10:13:06.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.656+0000 7f30df200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30e0144040 0x7f30e00cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.656+0000 7f30df200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f30e006ea30 0x7f30e00b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.656+0000 7f30df200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e0144420 con 0x7f30e015acd0 2024-04-25T10:13:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.656+0000 7f30df200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e00b3d80 con 0x7f30e006ea30 2024-04-25T10:13:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.656+0000 7f30df200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00b4060 con 0x7f30e0144040 2024-04-25T10:13:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.656+0000 7f30de800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f30e006ea30 0x7f30e00b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.656+0000 7f30dde00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30e0144040 0x7f30e00cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.657+0000 7f30d5400640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f30e015acd0 0x7f30e006e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:58648/0 (socket says 172.21.15.119:58648) 2024-04-25T10:13:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.657+0000 7f30d5400640 1 -- 172.21.15.119:0/1677838727 learned_addr learned my addr 172.21.15.119:0/1677838727 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:06.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.657+0000 7f30de800640 1 -- 172.21.15.119:0/1677838727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30e0144040 msgr2=0x7f30e00cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:06.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.657+0000 7f30de800640 1 --2- 172.21.15.119:0/1677838727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30e0144040 0x7f30e00cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.657+0000 7f30de800640 1 -- 172.21.15.119:0/1677838727 >> v1:172.21.15.132:6789/0 conn(0x7f30e015acd0 legacy=0x7f30e006e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.657+0000 7f30de800640 1 -- 172.21.15.119:0/1677838727 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e00b4340 con 0x7f30e006ea30 2024-04-25T10:13:06.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.657+0000 7f30dde00640 1 --2- 172.21.15.119:0/1677838727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30e0144040 0x7f30e00cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:06.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.659+0000 7f30de800640 1 --2- 172.21.15.119:0/1677838727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30e006ea30 0x7f30e00b26c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f30d80097a0 tx=0x7f30d802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10d2afd7ace26db3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.659+0000 7f30dd400640 1 -- 172.21.15.119:0/1677838727 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8037070 con 0x7f30e006ea30 2024-04-25T10:13:06.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.659+0000 7f30dd400640 1 -- 172.21.15.119:0/1677838727 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d802ae50 con 0x7f30e006ea30 2024-04-25T10:13:06.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.659+0000 7f30dd400640 1 -- 172.21.15.119:0/1677838727 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8032660 con 0x7f30e006ea30 2024-04-25T10:13:06.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.660+0000 7f30df200640 1 -- 172.21.15.119:0/1677838727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30e006ea30 msgr2=0x7f30e00b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.660+0000 7f30df200640 1 --2- 172.21.15.119:0/1677838727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30e006ea30 0x7f30e00b26c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f30d80097a0 tx=0x7f30d802a100 comp rx=0 tx=0).stop 2024-04-25T10:13:06.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.660+0000 7f30df200640 1 -- 172.21.15.119:0/1677838727 shutdown_connections 2024-04-25T10:13:06.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.660+0000 7f30df200640 1 --2- 172.21.15.119:0/1677838727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30e006ea30 0x7f30e00b26c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.660+0000 7f30df200640 1 --2- 172.21.15.119:0/1677838727 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30e0144040 0x7f30e00cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.660+0000 7f30df200640 1 -- 172.21.15.119:0/1677838727 >> 172.21.15.119:0/1677838727 conn(0x7f30e006b1d0 msgr2=0x7f30e00b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:06.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.660+0000 7f30df200640 1 -- 172.21.15.119:0/1677838727 shutdown_connections 2024-04-25T10:13:06.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.660+0000 7f30df200640 1 -- 172.21.15.119:0/1677838727 wait complete. 2024-04-25T10:13:06.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.662+0000 7f30df200640 1 Processor -- start 2024-04-25T10:13:06.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.662+0000 7f30df200640 1 -- start start 2024-04-25T10:13:06.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.663+0000 7f30df200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30e006ea30 0x7f30e01f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.663+0000 7f30df200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30e0144040 0x7f30e01e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.663+0000 7f30df200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30e015acd0 0x7f30e015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.663+0000 7f30df200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00b5180 con 0x7f30e006ea30 2024-04-25T10:13:06.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.663+0000 7f30df200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00b4900 con 0x7f30e0144040 2024-04-25T10:13:06.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.663+0000 7f30df200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00b5300 con 0x7f30e015acd0 2024-04-25T10:13:06.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.663+0000 7f30dde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30e006ea30 0x7f30e01f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.663+0000 7f30dde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30e006ea30 0x7f30e01f6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35210/0 (socket says 172.21.15.119:35210) 2024-04-25T10:13:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.663+0000 7f30dde00640 1 -- 172.21.15.119:0/1736539179 learned_addr learned my addr 172.21.15.119:0/1736539179 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.663+0000 7f30d5400640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30e0144040 0x7f30e01e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.663+0000 7f30de800640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30e015acd0 0x7f30e015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.663+0000 7f30dde00640 1 -- 172.21.15.119:0/1736539179 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30e015acd0 msgr2=0x7f30e015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.664+0000 7f30dde00640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30e015acd0 0x7f30e015a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.664+0000 7f30dde00640 1 -- 172.21.15.119:0/1736539179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30e0144040 msgr2=0x7f30e01e4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.664+0000 7f30dde00640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30e0144040 0x7f30e01e4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.664+0000 7f30dde00640 1 -- 172.21.15.119:0/1736539179 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e01f7170 con 0x7f30e006ea30 2024-04-25T10:13:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.664+0000 7f30d5400640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30e0144040 0x7f30e01e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.664+0000 7f30dde00640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30e006ea30 0x7f30e01f6b40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f30d000ec10 tx=0x7f30d000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.664+0000 7f30d6a00640 1 -- 172.21.15.119:0/1736539179 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0017070 con 0x7f30e006ea30 2024-04-25T10:13:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.664+0000 7f30d6a00640 1 -- 172.21.15.119:0/1736539179 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d000cc30 con 0x7f30e006ea30 2024-04-25T10:13:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.664+0000 7f30df200640 1 -- 172.21.15.119:0/1736539179 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e01f7450 con 0x7f30e006ea30 2024-04-25T10:13:06.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.664+0000 7f30d6a00640 1 -- 172.21.15.119:0/1736539179 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0013470 con 0x7f30e006ea30 2024-04-25T10:13:06.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.664+0000 7f30df200640 1 -- 172.21.15.119:0/1736539179 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e01f7a10 con 0x7f30e006ea30 2024-04-25T10:13:06.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.665+0000 7f30df200640 1 -- 172.21.15.119:0/1736539179 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e006bb30 con 0x7f30e006ea30 2024-04-25T10:13:06.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.666+0000 7f30d6a00640 1 -- 172.21.15.119:0/1736539179 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30d001a070 con 0x7f30e006ea30 2024-04-25T10:13:06.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.668+0000 7f30d6a00640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f30bc06b4d0 0x7f30bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.668+0000 7f30d6a00640 1 -- 172.21.15.119:0/1736539179 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f30d008a570 con 0x7f30e006ea30 2024-04-25T10:13:06.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.668+0000 7f30d5400640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f30bc06b4d0 0x7f30bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.669+0000 7f30d5400640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f30bc06b4d0 0x7f30bc06d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f30e0143790 tx=0x7f30c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.672+0000 7f30d6a00640 1 -- 172.21.15.119:0/1736539179 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30d0059350 con 0x7f30e006ea30 2024-04-25T10:13:06.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.872+0000 7f30df200640 1 -- 172.21.15.119:0/1736539179 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30e00749b0 con 0x7f30e006ea30 2024-04-25T10:13:06.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.875+0000 7f30d6a00640 1 -- 172.21.15.119:0/1736539179 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f30d0058c50 con 0x7f30e006ea30 2024-04-25T10:13:06.880 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:06.881 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":70,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:11:51.402126+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:32.500199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:33.708192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.22","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/2673349178":"2024-04-26T10:11:47.336991+0000","172.21.15.119:6836/2673349178":"2024-04-26T10:11:47.336991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.878+0000 7f30df200640 1 -- 172.21.15.119:0/1736539179 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f30bc06b4d0 msgr2=0x7f30bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.878+0000 7f30df200640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f30bc06b4d0 0x7f30bc06d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f30e0143790 tx=0x7f30c8009040 comp rx=0 tx=0).stop 2024-04-25T10:13:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.878+0000 7f30df200640 1 -- 172.21.15.119:0/1736539179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30e006ea30 msgr2=0x7f30e01f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.878+0000 7f30df200640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30e006ea30 0x7f30e01f6b40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f30d000ec10 tx=0x7f30d000c520 comp rx=0 tx=0).stop 2024-04-25T10:13:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.878+0000 7f30df200640 1 -- 172.21.15.119:0/1736539179 shutdown_connections 2024-04-25T10:13:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.878+0000 7f30df200640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f30bc06b4d0 0x7f30bc06d990 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.878+0000 7f30df200640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30e015acd0 0x7f30e015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.878+0000 7f30df200640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30e0144040 0x7f30e01e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.878+0000 7f30df200640 1 --2- 172.21.15.119:0/1736539179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30e006ea30 0x7f30e01f6b40 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.878+0000 7f30df200640 1 -- 172.21.15.119:0/1736539179 >> 172.21.15.119:0/1736539179 conn(0x7f30e006b1d0 msgr2=0x7f30e00cbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:06.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.878+0000 7f30df200640 1 -- 172.21.15.119:0/1736539179 shutdown_connections 2024-04-25T10:13:06.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:06.878+0000 7f30df200640 1 -- 172.21.15.119:0/1736539179 wait complete. 2024-04-25T10:13:06.903 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:32.500199+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:33.708192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:06.903 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:07.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.094+0000 7f07e2400640 1 Processor -- start 2024-04-25T10:13:07.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.094+0000 7f07e2400640 1 -- start start 2024-04-25T10:13:07.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.096+0000 7f07e2400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f07dc144e70 0x7f07dc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.096+0000 7f07e2400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07dc06dcb0 con 0x7f07dc06d7a0 2024-04-25T10:13:07.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.096+0000 7f07e2400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07dc06df90 con 0x7f07dc0cf0e0 2024-04-25T10:13:07.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.096+0000 7f07e2400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07dc15bfe0 con 0x7f07dc144e70 2024-04-25T10:13:07.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.096+0000 7f07daa00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f07dc06d7a0 0x7f07dc0ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41574/0 (socket says 172.21.15.119:41574) 2024-04-25T10:13:07.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.096+0000 7f07daa00640 1 -- 172.21.15.119:0/2665986681 learned_addr learned my addr 172.21.15.119:0/2665986681 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:07.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.096+0000 7f07db400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f07dc144e70 0x7f07dc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.097+0000 7f07db400640 1 -- 172.21.15.119:0/2665986681 >> v1:172.21.15.132:6789/0 conn(0x7f07dc0cf0e0 legacy=0x7f07dc0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.097+0000 7f07da000640 1 -- 172.21.15.119:0/2665986681 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3272809939 0 0) 0x7f07dc06dcb0 con 0x7f07dc06d7a0 2024-04-25T10:13:07.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.097+0000 7f07db400640 1 -- 172.21.15.119:0/2665986681 >> v1:172.21.15.119:6789/0 conn(0x7f07dc06d7a0 legacy=0x7f07dc0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.097+0000 7f07db400640 1 -- 172.21.15.119:0/2665986681 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc06e270 con 0x7f07dc144e70 2024-04-25T10:13:07.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.098+0000 7f07db400640 1 --2- 172.21.15.119:0/2665986681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc144e70 0x7f07dc06d260 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f07dc15c160 tx=0x7f07cc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9f9d7693b9e603 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.098+0000 7f07da000640 1 -- 172.21.15.119:0/2665986681 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc037070 con 0x7f07dc144e70 2024-04-25T10:13:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.098+0000 7f07da000640 1 -- 172.21.15.119:0/2665986681 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc032440 con 0x7f07dc144e70 2024-04-25T10:13:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.098+0000 7f07da000640 1 -- 172.21.15.119:0/2665986681 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc032810 con 0x7f07dc144e70 2024-04-25T10:13:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.099+0000 7f07e2400640 1 -- 172.21.15.119:0/2665986681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc144e70 msgr2=0x7f07dc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.099+0000 7f07e2400640 1 --2- 172.21.15.119:0/2665986681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc144e70 0x7f07dc06d260 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f07dc15c160 tx=0x7f07cc02a180 comp rx=0 tx=0).stop 2024-04-25T10:13:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.099+0000 7f07e2400640 1 -- 172.21.15.119:0/2665986681 shutdown_connections 2024-04-25T10:13:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.099+0000 7f07e2400640 1 --2- 172.21.15.119:0/2665986681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc144e70 0x7f07dc06d260 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.099+0000 7f07e2400640 1 -- 172.21.15.119:0/2665986681 >> 172.21.15.119:0/2665986681 conn(0x7f07dc06aba0 msgr2=0x7f07dc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.099+0000 7f07e2400640 1 -- 172.21.15.119:0/2665986681 shutdown_connections 2024-04-25T10:13:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.099+0000 7f07e2400640 1 -- 172.21.15.119:0/2665986681 wait complete. 2024-04-25T10:13:07.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.101+0000 7f07e2400640 1 Processor -- start 2024-04-25T10:13:07.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.101+0000 7f07e2400640 1 -- start start 2024-04-25T10:13:07.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.102+0000 7f07e2400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07dc06d7a0 0x7f07dc0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.102+0000 7f07e2400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc0cf0e0 0x7f07dc1e5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.102+0000 7f07e2400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07dc144e70 0x7f07dc1437c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.102+0000 7f07e2400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc145630 con 0x7f07dc0cf0e0 2024-04-25T10:13:07.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.102+0000 7f07e2400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc1454b0 con 0x7f07dc144e70 2024-04-25T10:13:07.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.102+0000 7f07e2400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc0cca10 con 0x7f07dc06d7a0 2024-04-25T10:13:07.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.102+0000 7f07daa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc0cf0e0 0x7f07dc1e5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.102+0000 7f07daa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc0cf0e0 0x7f07dc1e5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35224/0 (socket says 172.21.15.119:35224) 2024-04-25T10:13:07.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.102+0000 7f07daa00640 1 -- 172.21.15.119:0/3860867010 learned_addr learned my addr 172.21.15.119:0/3860867010 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:07.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.103+0000 7f07daa00640 1 -- 172.21.15.119:0/3860867010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07dc06d7a0 msgr2=0x7f07dc0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.103+0000 7f07db400640 1 --2- 172.21.15.119:0/3860867010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07dc06d7a0 0x7f07dc0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.103+0000 7f07daa00640 1 --2- 172.21.15.119:0/3860867010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07dc06d7a0 0x7f07dc0b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.103+0000 7f07daa00640 1 -- 172.21.15.119:0/3860867010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07dc144e70 msgr2=0x7f07dc1437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.103+0000 7f07daa00640 1 --2- 172.21.15.119:0/3860867010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07dc144e70 0x7f07dc1437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.103+0000 7f07daa00640 1 -- 172.21.15.119:0/3860867010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc1e5e50 con 0x7f07dc0cf0e0 2024-04-25T10:13:07.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.103+0000 7f07daa00640 1 --2- 172.21.15.119:0/3860867010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc0cf0e0 0x7f07dc1e5390 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f07c800c4a0 tx=0x7f07c800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.103+0000 7f07d3400640 1 -- 172.21.15.119:0/3860867010 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8018070 con 0x7f07dc0cf0e0 2024-04-25T10:13:07.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.103+0000 7f07d3400640 1 -- 172.21.15.119:0/3860867010 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c80074e0 con 0x7f07dc0cf0e0 2024-04-25T10:13:07.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.103+0000 7f07d3400640 1 -- 172.21.15.119:0/3860867010 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8004a20 con 0x7f07dc0cf0e0 2024-04-25T10:13:07.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.103+0000 7f07e2400640 1 -- 172.21.15.119:0/3860867010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc1e6130 con 0x7f07dc0cf0e0 2024-04-25T10:13:07.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.103+0000 7f07e2400640 1 -- 172.21.15.119:0/3860867010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc1e66f0 con 0x7f07dc0cf0e0 2024-04-25T10:13:07.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.105+0000 7f07e2400640 1 -- 172.21.15.119:0/3860867010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07dc0750b0 con 0x7f07dc0cf0e0 2024-04-25T10:13:07.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.105+0000 7f07d3400640 1 -- 172.21.15.119:0/3860867010 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07c8004bc0 con 0x7f07dc0cf0e0 2024-04-25T10:13:07.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.106+0000 7f07d3400640 1 --2- 172.21.15.119:0/3860867010 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f07bc06b4d0 0x7f07bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.107+0000 7f07d3400640 1 -- 172.21.15.119:0/3860867010 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f07c8089680 con 0x7f07dc0cf0e0 2024-04-25T10:13:07.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.107+0000 7f07db400640 1 --2- 172.21.15.119:0/3860867010 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f07bc06b4d0 0x7f07bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.107+0000 7f07db400640 1 --2- 172.21.15.119:0/3860867010 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f07bc06b4d0 0x7f07bc06d990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f07cc034000 tx=0x7f07cc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.111+0000 7f07d3400640 1 -- 172.21.15.119:0/3860867010 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f07c80583e0 con 0x7f07dc0cf0e0 2024-04-25T10:13:07.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.304+0000 7f07e2400640 1 -- 172.21.15.119:0/3860867010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07dc0aebe0 con 0x7f07dc0cf0e0 2024-04-25T10:13:07.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.305+0000 7f07d3400640 1 -- 172.21.15.119:0/3860867010 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f07c8057ce0 con 0x7f07dc0cf0e0 2024-04-25T10:13:07.310 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:13:07.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.308+0000 7f07e2400640 1 -- 172.21.15.119:0/3860867010 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f07bc06b4d0 msgr2=0x7f07bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.308+0000 7f07e2400640 1 --2- 172.21.15.119:0/3860867010 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f07bc06b4d0 0x7f07bc06d990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f07cc034000 tx=0x7f07cc02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:07.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.308+0000 7f07e2400640 1 -- 172.21.15.119:0/3860867010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc0cf0e0 msgr2=0x7f07dc1e5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.308+0000 7f07e2400640 1 --2- 172.21.15.119:0/3860867010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc0cf0e0 0x7f07dc1e5390 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f07c800c4a0 tx=0x7f07c800c960 comp rx=0 tx=0).stop 2024-04-25T10:13:07.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.308+0000 7f07e2400640 1 -- 172.21.15.119:0/3860867010 shutdown_connections 2024-04-25T10:13:07.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.308+0000 7f07e2400640 1 --2- 172.21.15.119:0/3860867010 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f07bc06b4d0 0x7f07bc06d990 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.308+0000 7f07e2400640 1 --2- 172.21.15.119:0/3860867010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07dc144e70 0x7f07dc1437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.308+0000 7f07e2400640 1 --2- 172.21.15.119:0/3860867010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07dc0cf0e0 0x7f07dc1e5390 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.308+0000 7f07e2400640 1 --2- 172.21.15.119:0/3860867010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f07dc06d7a0 0x7f07dc0b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.308+0000 7f07e2400640 1 -- 172.21.15.119:0/3860867010 >> 172.21.15.119:0/3860867010 conn(0x7f07dc06aba0 msgr2=0x7f07dc141210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.308+0000 7f07e2400640 1 -- 172.21.15.119:0/3860867010 shutdown_connections 2024-04-25T10:13:07.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.308+0000 7f07e2400640 1 -- 172.21.15.119:0/3860867010 wait complete. 2024-04-25T10:13:07.333 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:07.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.529+0000 7f9f97200640 1 Processor -- start 2024-04-25T10:13:07.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.529+0000 7f9f97200640 1 -- start start 2024-04-25T10:13:07.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.530+0000 7f9f97200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9f9815adc0 0x7f9f9815b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.531+0000 7f9f97200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f9815ba00 0x7f9f98071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.531+0000 7f9f97200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f980b2850 con 0x7f9f98159ee0 2024-04-25T10:13:07.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.531+0000 7f9f97200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f980b2b30 con 0x7f9f9815adc0 2024-04-25T10:13:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.531+0000 7f9f97200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980b2e10 con 0x7f9f9815ba00 2024-04-25T10:13:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.531+0000 7f9f95e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9f98159ee0 0x7f9f980b38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41590/0 (socket says 172.21.15.119:41590) 2024-04-25T10:13:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.531+0000 7f9f95e00640 1 -- 172.21.15.119:0/2459806634 learned_addr learned my addr 172.21.15.119:0/2459806634 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.531+0000 7f9f95400640 1 -- 172.21.15.119:0/2459806634 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178756159 0 0) 0x7f9f980b2850 con 0x7f9f98159ee0 2024-04-25T10:13:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.531+0000 7f9f8fe00640 1 --2- 172.21.15.119:0/2459806634 >> v2:172.21.15.132:3300/0 conn(0x7f9f9815adc0 0x7f9f9815b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.531+0000 7f9f96800640 1 --2- 172.21.15.119:0/2459806634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f9815ba00 0x7f9f98071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.531+0000 7f9f95400640 1 -- 172.21.15.119:0/2459806634 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f7c003520 con 0x7f9f98159ee0 2024-04-25T10:13:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f95400640 1 -- 172.21.15.119:0/2459806634 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2252734581 0 0) 0x7f9f7c003520 con 0x7f9f98159ee0 2024-04-25T10:13:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f95400640 1 -- 172.21.15.119:0/2459806634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f9815ba00 msgr2=0x7f9f98071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f95400640 1 --2- 172.21.15.119:0/2459806634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f9815ba00 0x7f9f98071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f95400640 1 -- 172.21.15.119:0/2459806634 >> v2:172.21.15.132:3300/0 conn(0x7f9f9815adc0 msgr2=0x7f9f9815b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f95400640 1 --2- 172.21.15.119:0/2459806634 >> v2:172.21.15.132:3300/0 conn(0x7f9f9815adc0 0x7f9f9815b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f95400640 1 -- 172.21.15.119:0/2459806634 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f980b30f0 con 0x7f9f98159ee0 2024-04-25T10:13:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f95400640 1 -- 172.21.15.119:0/2459806634 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f9f88001690 con 0x7f9f98159ee0 2024-04-25T10:13:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f95400640 1 -- 172.21.15.119:0/2459806634 >> v1:172.21.15.119:6789/0 conn(0x7f9f98159ee0 legacy=0x7f9f980b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f95400640 1 --2- 172.21.15.119:0/2459806634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f9815ba00 0x7f9f7c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f95400640 1 --2- 172.21.15.119:0/2459806634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f7c003fe0 0x7f9f7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f95400640 1 --2- 172.21.15.119:0/2459806634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f7c004d40 0x7f9f7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f95400640 1 -- 172.21.15.119:0/2459806634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980b2b30 con 0x7f9f7c003fe0 2024-04-25T10:13:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f95400640 1 -- 172.21.15.119:0/2459806634 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980b2e10 con 0x7f9f7c004d40 2024-04-25T10:13:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f95400640 1 -- 172.21.15.119:0/2459806634 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c00bac0 con 0x7f9f9815ba00 2024-04-25T10:13:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f8fe00640 1 --2- 172.21.15.119:0/2459806634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f7c003fe0 0x7f9f7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f8fe00640 1 -- 172.21.15.119:0/2459806634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f9815ba00 msgr2=0x7f9f7c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f8fe00640 1 --2- 172.21.15.119:0/2459806634 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f9815ba00 0x7f9f7c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f8fe00640 1 -- 172.21.15.119:0/2459806634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f7c004d40 msgr2=0x7f9f7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f8fe00640 1 --2- 172.21.15.119:0/2459806634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f7c004d40 0x7f9f7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.532+0000 7f9f8fe00640 1 -- 172.21.15.119:0/2459806634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c003520 con 0x7f9f7c003fe0 2024-04-25T10:13:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.533+0000 7f9f8fe00640 1 --2- 172.21.15.119:0/2459806634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f7c003fe0 0x7f9f7c004800 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000ba30 tx=0x7f9f7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493045a8db9aef86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.534+0000 7f9f95400640 1 -- 172.21.15.119:0/2459806634 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f80030070 con 0x7f9f7c003fe0 2024-04-25T10:13:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.534+0000 7f9f95400640 1 -- 172.21.15.119:0/2459806634 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8002ae10 con 0x7f9f7c003fe0 2024-04-25T10:13:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.534+0000 7f9f95400640 1 -- 172.21.15.119:0/2459806634 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8000b600 con 0x7f9f7c003fe0 2024-04-25T10:13:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.534+0000 7f9f97200640 1 -- 172.21.15.119:0/2459806634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f7c003fe0 msgr2=0x7f9f7c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.534+0000 7f9f97200640 1 --2- 172.21.15.119:0/2459806634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f7c003fe0 0x7f9f7c004800 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000ba30 tx=0x7f9f7c005830 comp rx=0 tx=0).stop 2024-04-25T10:13:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.534+0000 7f9f96800640 1 -- 172.21.15.119:0/2459806634 reap_dead start 2024-04-25T10:13:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.534+0000 7f9f97200640 1 -- 172.21.15.119:0/2459806634 shutdown_connections 2024-04-25T10:13:07.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.534+0000 7f9f97200640 1 -- 172.21.15.119:0/2459806634 >> 172.21.15.119:0/2459806634 conn(0x7f9f98066ff0 msgr2=0x7f9f980b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.534+0000 7f9f97200640 1 -- 172.21.15.119:0/2459806634 shutdown_connections 2024-04-25T10:13:07.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.534+0000 7f9f97200640 1 -- 172.21.15.119:0/2459806634 wait complete. 2024-04-25T10:13:07.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.536+0000 7f9f97200640 1 Processor -- start 2024-04-25T10:13:07.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.536+0000 7f9f97200640 1 -- start start 2024-04-25T10:13:07.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.537+0000 7f9f97200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f7c003fe0 0x7f9f980b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.537+0000 7f9f97200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f981549c0 0x7f9f981566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.537+0000 7f9f97200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f98156bf0 0x7f9f98152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.537+0000 7f9f97200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980b4000 con 0x7f9f98156bf0 2024-04-25T10:13:07.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.537+0000 7f9f97200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9806efc0 con 0x7f9f7c003fe0 2024-04-25T10:13:07.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.537+0000 7f9f97200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980b4180 con 0x7f9f981549c0 2024-04-25T10:13:07.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.537+0000 7f9f96800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f98156bf0 0x7f9f98152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.537+0000 7f9f96800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f98156bf0 0x7f9f98152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35246/0 (socket says 172.21.15.119:35246) 2024-04-25T10:13:07.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.537+0000 7f9f96800640 1 -- 172.21.15.119:0/4148536667 learned_addr learned my addr 172.21.15.119:0/4148536667 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:07.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.538+0000 7f9f95e00640 1 --2- 172.21.15.119:0/4148536667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f7c003fe0 0x7f9f980b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.538+0000 7f9f8fe00640 1 --2- 172.21.15.119:0/4148536667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f981549c0 0x7f9f981566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.538+0000 7f9f96800640 1 -- 172.21.15.119:0/4148536667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f981549c0 msgr2=0x7f9f981566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.538+0000 7f9f96800640 1 --2- 172.21.15.119:0/4148536667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f981549c0 0x7f9f981566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.538+0000 7f9f96800640 1 -- 172.21.15.119:0/4148536667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f7c003fe0 msgr2=0x7f9f980b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.538+0000 7f9f96800640 1 --2- 172.21.15.119:0/4148536667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f7c003fe0 0x7f9f980b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.538+0000 7f9f96800640 1 -- 172.21.15.119:0/4148536667 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9814d470 con 0x7f9f98156bf0 2024-04-25T10:13:07.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.538+0000 7f9f96800640 1 --2- 172.21.15.119:0/4148536667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f98156bf0 0x7f9f98152c90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9f98071fb0 tx=0x7f9f9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.538+0000 7f9f8e000640 1 -- 172.21.15.119:0/4148536667 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9000edc0 con 0x7f9f98156bf0 2024-04-25T10:13:07.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.538+0000 7f9f8e000640 1 -- 172.21.15.119:0/4148536667 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f90019070 con 0x7f9f98156bf0 2024-04-25T10:13:07.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.538+0000 7f9f8e000640 1 -- 172.21.15.119:0/4148536667 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90014660 con 0x7f9f98156bf0 2024-04-25T10:13:07.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.538+0000 7f9f97200640 1 -- 172.21.15.119:0/4148536667 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9814ceb0 con 0x7f9f98156bf0 2024-04-25T10:13:07.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.539+0000 7f9f97200640 1 -- 172.21.15.119:0/4148536667 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f98152490 con 0x7f9f98156bf0 2024-04-25T10:13:07.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.540+0000 7f9f97200640 1 -- 172.21.15.119:0/4148536667 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f980749b0 con 0x7f9f98156bf0 2024-04-25T10:13:07.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.541+0000 7f9f8e000640 1 -- 172.21.15.119:0/4148536667 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f90014800 con 0x7f9f98156bf0 2024-04-25T10:13:07.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.541+0000 7f9f8e000640 1 --2- 172.21.15.119:0/4148536667 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9f7406b500 0x7f9f7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.542+0000 7f9f8e000640 1 -- 172.21.15.119:0/4148536667 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9f9008a5b0 con 0x7f9f98156bf0 2024-04-25T10:13:07.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.542+0000 7f9f95e00640 1 --2- 172.21.15.119:0/4148536667 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9f7406b500 0x7f9f7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.542+0000 7f9f95e00640 1 --2- 172.21.15.119:0/4148536667 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9f7406b500 0x7f9f7406d9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9f9806e600 tx=0x7f9f88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.546+0000 7f9f8e000640 1 -- 172.21.15.119:0/4148536667 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f90059390 con 0x7f9f98156bf0 2024-04-25T10:13:07.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.722+0000 7f9f97200640 1 -- 172.21.15.119:0/4148536667 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f980aeb50 con 0x7f9f98156bf0 2024-04-25T10:13:07.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.723+0000 7f9f8e000640 1 -- 172.21.15.119:0/4148536667 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f90058c90 con 0x7f9f98156bf0 2024-04-25T10:13:07.728 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:13:07.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.726+0000 7f9f97200640 1 -- 172.21.15.119:0/4148536667 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9f7406b500 msgr2=0x7f9f7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.726+0000 7f9f97200640 1 --2- 172.21.15.119:0/4148536667 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9f7406b500 0x7f9f7406d9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9f9806e600 tx=0x7f9f88009040 comp rx=0 tx=0).stop 2024-04-25T10:13:07.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.726+0000 7f9f97200640 1 -- 172.21.15.119:0/4148536667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f98156bf0 msgr2=0x7f9f98152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.726+0000 7f9f97200640 1 --2- 172.21.15.119:0/4148536667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f98156bf0 0x7f9f98152c90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9f98071fb0 tx=0x7f9f9000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.726+0000 7f9f97200640 1 -- 172.21.15.119:0/4148536667 shutdown_connections 2024-04-25T10:13:07.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.726+0000 7f9f97200640 1 --2- 172.21.15.119:0/4148536667 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9f7406b500 0x7f9f7406d9c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.726+0000 7f9f97200640 1 --2- 172.21.15.119:0/4148536667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f98156bf0 0x7f9f98152c90 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.726+0000 7f9f97200640 1 --2- 172.21.15.119:0/4148536667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9f981549c0 0x7f9f981566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.726+0000 7f9f97200640 1 --2- 172.21.15.119:0/4148536667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f7c003fe0 0x7f9f980b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.726+0000 7f9f97200640 1 -- 172.21.15.119:0/4148536667 >> 172.21.15.119:0/4148536667 conn(0x7f9f98066ff0 msgr2=0x7f9f980cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.726+0000 7f9f97200640 1 -- 172.21.15.119:0/4148536667 shutdown_connections 2024-04-25T10:13:07.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.726+0000 7f9f97200640 1 -- 172.21.15.119:0/4148536667 wait complete. 2024-04-25T10:13:07.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:07.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.928+0000 7f1659c00640 1 Processor -- start 2024-04-25T10:13:07.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.928+0000 7f1659c00640 1 -- start start 2024-04-25T10:13:07.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.930+0000 7f1659c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16540b3e90 0x7f16540cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.930+0000 7f1659c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f165415f400 con 0x7f165406b2b0 2024-04-25T10:13:07.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.930+0000 7f1659c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f165415f6e0 con 0x7f16541450e0 2024-04-25T10:13:07.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.930+0000 7f1659c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16541454c0 con 0x7f16540b3e90 2024-04-25T10:13:07.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.930+0000 7f164be00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f165406b2b0 0x7f16540cc2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41600/0 (socket says 172.21.15.119:41600) 2024-04-25T10:13:07.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.930+0000 7f164be00640 1 -- 172.21.15.119:0/1112462317 learned_addr learned my addr 172.21.15.119:0/1112462317 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:07.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.930+0000 7f1653400640 1 -- 172.21.15.119:0/1112462317 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133599789 0 0) 0x7f165415f400 con 0x7f165406b2b0 2024-04-25T10:13:07.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.930+0000 7f1653400640 1 -- 172.21.15.119:0/1112462317 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1638003520 con 0x7f165406b2b0 2024-04-25T10:13:07.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.930+0000 7f1659200640 1 --2- 172.21.15.119:0/1112462317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16540b3e90 0x7f16540cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.931+0000 7f1653400640 1 -- 172.21.15.119:0/1112462317 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2474718852 0 0) 0x7f1638003520 con 0x7f165406b2b0 2024-04-25T10:13:07.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.931+0000 7f1653400640 1 -- 172.21.15.119:0/1112462317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16540b3e90 msgr2=0x7f16540cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.931+0000 7f1653400640 1 --2- 172.21.15.119:0/1112462317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16540b3e90 0x7f16540cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.931+0000 7f1653400640 1 -- 172.21.15.119:0/1112462317 >> v1:172.21.15.132:6789/0 conn(0x7f16541450e0 legacy=0x7f165406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.931+0000 7f1653400640 1 -- 172.21.15.119:0/1112462317 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165415f9c0 con 0x7f165406b2b0 2024-04-25T10:13:07.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.931+0000 7f1653400640 1 -- 172.21.15.119:0/1112462317 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f163c001670 con 0x7f165406b2b0 2024-04-25T10:13:07.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.931+0000 7f1653400640 1 -- 172.21.15.119:0/1112462317 >> v1:172.21.15.119:6789/0 conn(0x7f165406b2b0 legacy=0x7f16540cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.931+0000 7f1653400640 1 --2- 172.21.15.119:0/1112462317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16380039e0 0x7f1638003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.931+0000 7f1653400640 1 --2- 172.21.15.119:0/1112462317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16540b3e90 0x7f1638004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.931+0000 7f1653400640 1 --2- 172.21.15.119:0/1112462317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1638004d90 0x7f1638005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.931+0000 7f1653400640 1 -- 172.21.15.119:0/1112462317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163800bc20 con 0x7f16380039e0 2024-04-25T10:13:07.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.931+0000 7f1653400640 1 -- 172.21.15.119:0/1112462317 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163800bdd0 con 0x7f1638004d90 2024-04-25T10:13:07.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.931+0000 7f1653400640 1 -- 172.21.15.119:0/1112462317 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163800bfd0 con 0x7f16540b3e90 2024-04-25T10:13:07.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.931+0000 7f1653e00640 1 --2- 172.21.15.119:0/1112462317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16380039e0 0x7f1638003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.932+0000 7f1659200640 1 --2- 172.21.15.119:0/1112462317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1638004d90 0x7f1638005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.932+0000 7f164be00640 1 --2- 172.21.15.119:0/1112462317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16540b3e90 0x7f1638004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.932+0000 7f1653e00640 1 -- 172.21.15.119:0/1112462317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16540b3e90 msgr2=0x7f1638004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.932+0000 7f1653e00640 1 --2- 172.21.15.119:0/1112462317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16540b3e90 0x7f1638004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.932+0000 7f1653e00640 1 -- 172.21.15.119:0/1112462317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1638004d90 msgr2=0x7f1638005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.932+0000 7f1653e00640 1 --2- 172.21.15.119:0/1112462317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1638004d90 0x7f1638005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.932+0000 7f1653e00640 1 -- 172.21.15.119:0/1112462317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1638003520 con 0x7f16380039e0 2024-04-25T10:13:07.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.932+0000 7f164be00640 1 --2- 172.21.15.119:0/1112462317 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16540b3e90 0x7f1638004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:07.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.933+0000 7f1653e00640 1 --2- 172.21.15.119:0/1112462317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16380039e0 0x7f1638003de0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1644003d80 tx=0x7f1638005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff43a919694c85b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.933+0000 7f1653400640 1 -- 172.21.15.119:0/1112462317 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1644037070 con 0x7f16380039e0 2024-04-25T10:13:07.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.933+0000 7f1653400640 1 -- 172.21.15.119:0/1112462317 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164402a9d0 con 0x7f16380039e0 2024-04-25T10:13:07.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.933+0000 7f1653400640 1 -- 172.21.15.119:0/1112462317 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1644003980 con 0x7f16380039e0 2024-04-25T10:13:07.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.934+0000 7f1659c00640 1 -- 172.21.15.119:0/1112462317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16380039e0 msgr2=0x7f1638003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.934+0000 7f1659c00640 1 --2- 172.21.15.119:0/1112462317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16380039e0 0x7f1638003de0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1644003d80 tx=0x7f1638005830 comp rx=0 tx=0).stop 2024-04-25T10:13:07.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.934+0000 7f1659200640 1 -- 172.21.15.119:0/1112462317 reap_dead start 2024-04-25T10:13:07.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.934+0000 7f1659c00640 1 -- 172.21.15.119:0/1112462317 shutdown_connections 2024-04-25T10:13:07.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.934+0000 7f1659c00640 1 -- 172.21.15.119:0/1112462317 >> 172.21.15.119:0/1112462317 conn(0x7f1654066ff0 msgr2=0x7f16540b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.934+0000 7f1659c00640 1 -- 172.21.15.119:0/1112462317 shutdown_connections 2024-04-25T10:13:07.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.934+0000 7f1659c00640 1 -- 172.21.15.119:0/1112462317 wait complete. 2024-04-25T10:13:07.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.936+0000 7f1659c00640 1 Processor -- start 2024-04-25T10:13:07.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.936+0000 7f1659c00640 1 -- start start 2024-04-25T10:13:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.937+0000 7f1659c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16380039e0 0x7f1654141270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.937+0000 7f1659c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f165415cda0 0x7f16541f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.937+0000 7f1659c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16541f38b0 0x7f165413f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.937+0000 7f1659c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16540ccb60 con 0x7f165415cda0 2024-04-25T10:13:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.937+0000 7f1659c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16540cc9e0 con 0x7f16380039e0 2024-04-25T10:13:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.937+0000 7f1659c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16540ccce0 con 0x7f16541f38b0 2024-04-25T10:13:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.937+0000 7f164be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f165415cda0 0x7f16541f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.937+0000 7f164be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f165415cda0 0x7f16541f3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35268/0 (socket says 172.21.15.119:35268) 2024-04-25T10:13:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.937+0000 7f164be00640 1 -- 172.21.15.119:0/2528406309 learned_addr learned my addr 172.21.15.119:0/2528406309 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.937+0000 7f1653e00640 1 --2- 172.21.15.119:0/2528406309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16380039e0 0x7f1654141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.938+0000 7f1659200640 1 --2- 172.21.15.119:0/2528406309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16541f38b0 0x7f165413f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.938+0000 7f164be00640 1 -- 172.21.15.119:0/2528406309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16541f38b0 msgr2=0x7f165413f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.938+0000 7f164be00640 1 --2- 172.21.15.119:0/2528406309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16541f38b0 0x7f165413f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.938+0000 7f164be00640 1 -- 172.21.15.119:0/2528406309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16380039e0 msgr2=0x7f1654141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.938+0000 7f164be00640 1 --2- 172.21.15.119:0/2528406309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16380039e0 0x7f1654141270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.938+0000 7f164be00640 1 -- 172.21.15.119:0/2528406309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16541f3f50 con 0x7f165415cda0 2024-04-25T10:13:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.938+0000 7f164be00640 1 --2- 172.21.15.119:0/2528406309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f165415cda0 0x7f16541f3370 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f163c00d7a0 tx=0x7f163c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.938+0000 7f1650c00640 1 -- 172.21.15.119:0/2528406309 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c019070 con 0x7f165415cda0 2024-04-25T10:13:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.938+0000 7f1650c00640 1 -- 172.21.15.119:0/2528406309 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c0074e0 con 0x7f165415cda0 2024-04-25T10:13:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.938+0000 7f1659c00640 1 -- 172.21.15.119:0/2528406309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16540af590 con 0x7f165415cda0 2024-04-25T10:13:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.938+0000 7f1650c00640 1 -- 172.21.15.119:0/2528406309 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c004a00 con 0x7f165415cda0 2024-04-25T10:13:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.938+0000 7f1659c00640 1 -- 172.21.15.119:0/2528406309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16540afb50 con 0x7f165415cda0 2024-04-25T10:13:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.940+0000 7f1659c00640 1 -- 172.21.15.119:0/2528406309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16540749b0 con 0x7f165415cda0 2024-04-25T10:13:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.940+0000 7f1650c00640 1 -- 172.21.15.119:0/2528406309 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f163c01c070 con 0x7f165415cda0 2024-04-25T10:13:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.941+0000 7f1650c00640 1 --2- 172.21.15.119:0/2528406309 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f163006b4d0 0x7f163006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.941+0000 7f1650c00640 1 -- 172.21.15.119:0/2528406309 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f163c089d30 con 0x7f165415cda0 2024-04-25T10:13:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.942+0000 7f1653e00640 1 --2- 172.21.15.119:0/2528406309 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f163006b4d0 0x7f163006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.942+0000 7f1653e00640 1 --2- 172.21.15.119:0/2528406309 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f163006b4d0 0x7f163006d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1644034000 tx=0x7f164402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:07.946+0000 7f1650c00640 1 -- 172.21.15.119:0/2528406309 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f163c058a80 con 0x7f165415cda0 2024-04-25T10:13:08.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.139+0000 7f1659c00640 1 -- 172.21.15.119:0/2528406309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16541f3c90 con 0x7f165415cda0 2024-04-25T10:13:08.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.140+0000 7f1650c00640 1 -- 172.21.15.119:0/2528406309 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f163c058380 con 0x7f165415cda0 2024-04-25T10:13:08.145 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:13:08.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.142+0000 7f1659c00640 1 -- 172.21.15.119:0/2528406309 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f163006b4d0 msgr2=0x7f163006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.142+0000 7f1659c00640 1 --2- 172.21.15.119:0/2528406309 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f163006b4d0 0x7f163006d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1644034000 tx=0x7f164402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:08.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.142+0000 7f1659c00640 1 -- 172.21.15.119:0/2528406309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f165415cda0 msgr2=0x7f16541f3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.142+0000 7f1659c00640 1 --2- 172.21.15.119:0/2528406309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f165415cda0 0x7f16541f3370 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f163c00d7a0 tx=0x7f163c00dc60 comp rx=0 tx=0).stop 2024-04-25T10:13:08.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.142+0000 7f1659c00640 1 -- 172.21.15.119:0/2528406309 shutdown_connections 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.142+0000 7f1659c00640 1 --2- 172.21.15.119:0/2528406309 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f163006b4d0 0x7f163006d990 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.142+0000 7f1659c00640 1 --2- 172.21.15.119:0/2528406309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f16541f38b0 0x7f165413f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.142+0000 7f1659c00640 1 --2- 172.21.15.119:0/2528406309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f165415cda0 0x7f16541f3370 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.142+0000 7f1659c00640 1 --2- 172.21.15.119:0/2528406309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16380039e0 0x7f1654141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.142+0000 7f1659c00640 1 -- 172.21.15.119:0/2528406309 >> 172.21.15.119:0/2528406309 conn(0x7f1654066ff0 msgr2=0x7f16540cc180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.143+0000 7f1659c00640 1 -- 172.21.15.119:0/2528406309 shutdown_connections 2024-04-25T10:13:08.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.143+0000 7f1659c00640 1 -- 172.21.15.119:0/2528406309 wait complete. 2024-04-25T10:13:08.165 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:08.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.341+0000 7fb41b200640 1 Processor -- start 2024-04-25T10:13:08.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.341+0000 7fb41b200640 1 -- start start 2024-04-25T10:13:08.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.342+0000 7fb41b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb414144040 0x7fb4140cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.342+0000 7fb41b200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb41406ea30 0x7fb4140b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.342+0000 7fb41b200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb414144420 con 0x7fb41415acd0 2024-04-25T10:13:08.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.342+0000 7fb41b200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4140b3d80 con 0x7fb41406ea30 2024-04-25T10:13:08.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.342+0000 7fb41b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140b4060 con 0x7fb414144040 2024-04-25T10:13:08.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.342+0000 7fb41a800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb41406ea30 0x7fb4140b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.343+0000 7fb41a800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb41406ea30 0x7fb4140b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35280/0 (socket says 172.21.15.119:35280) 2024-04-25T10:13:08.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.343+0000 7fb41a800640 1 -- 172.21.15.119:0/34517774 learned_addr learned my addr 172.21.15.119:0/34517774 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:08.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.343+0000 7fb41a800640 1 -- 172.21.15.119:0/34517774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb414144040 msgr2=0x7fb4140cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.343+0000 7fb41a800640 1 --2- 172.21.15.119:0/34517774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb414144040 0x7fb4140cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.343+0000 7fb41a800640 1 -- 172.21.15.119:0/34517774 >> v1:172.21.15.132:6789/0 conn(0x7fb41415acd0 legacy=0x7fb41406e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.343+0000 7fb41a800640 1 -- 172.21.15.119:0/34517774 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4140b4340 con 0x7fb41406ea30 2024-04-25T10:13:08.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.344+0000 7fb41a800640 1 --2- 172.21.15.119:0/34517774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41406ea30 0x7fb4140b26c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb410009e50 tx=0x7fb41002a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9696dcf2bae09ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.344+0000 7fb418a00640 1 -- 172.21.15.119:0/34517774 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410036070 con 0x7fb41406ea30 2024-04-25T10:13:08.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.344+0000 7fb418a00640 1 -- 172.21.15.119:0/34517774 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb410031600 con 0x7fb41406ea30 2024-04-25T10:13:08.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.344+0000 7fb418a00640 1 -- 172.21.15.119:0/34517774 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4100319d0 con 0x7fb41406ea30 2024-04-25T10:13:08.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.345+0000 7fb41b200640 1 -- 172.21.15.119:0/34517774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41406ea30 msgr2=0x7fb4140b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.345+0000 7fb41b200640 1 --2- 172.21.15.119:0/34517774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41406ea30 0x7fb4140b26c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb410009e50 tx=0x7fb41002a640 comp rx=0 tx=0).stop 2024-04-25T10:13:08.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.345+0000 7fb41b200640 1 -- 172.21.15.119:0/34517774 shutdown_connections 2024-04-25T10:13:08.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.345+0000 7fb41b200640 1 --2- 172.21.15.119:0/34517774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41406ea30 0x7fb4140b26c0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.345+0000 7fb41b200640 1 --2- 172.21.15.119:0/34517774 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb414144040 0x7fb4140cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.345+0000 7fb41b200640 1 -- 172.21.15.119:0/34517774 >> 172.21.15.119:0/34517774 conn(0x7fb41406b1d0 msgr2=0x7fb4140b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.345+0000 7fb41b200640 1 -- 172.21.15.119:0/34517774 shutdown_connections 2024-04-25T10:13:08.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.345+0000 7fb41b200640 1 -- 172.21.15.119:0/34517774 wait complete. 2024-04-25T10:13:08.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.347+0000 7fb41b200640 1 Processor -- start 2024-04-25T10:13:08.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.347+0000 7fb41b200640 1 -- start start 2024-04-25T10:13:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.348+0000 7fb41b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41406ea30 0x7fb414151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.348+0000 7fb41b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb414144040 0x7fb41413e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.348+0000 7fb41b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb41415acd0 0x7fb41413b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.348+0000 7fb41b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140b5180 con 0x7fb41406ea30 2024-04-25T10:13:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.348+0000 7fb41b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140b4900 con 0x7fb41415acd0 2024-04-25T10:13:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.348+0000 7fb41b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140b5300 con 0x7fb414144040 2024-04-25T10:13:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.348+0000 7fb419e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41406ea30 0x7fb414151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.348+0000 7fb419e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41406ea30 0x7fb414151810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35286/0 (socket says 172.21.15.119:35286) 2024-04-25T10:13:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.348+0000 7fb419e00640 1 -- 172.21.15.119:0/1367638450 learned_addr learned my addr 172.21.15.119:0/1367638450 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.348+0000 7fb41a800640 1 --2- 172.21.15.119:0/1367638450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb41415acd0 0x7fb41413b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.349+0000 7fb419e00640 1 -- 172.21.15.119:0/1367638450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb414144040 msgr2=0x7fb41413e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.349+0000 7fb419e00640 1 --2- 172.21.15.119:0/1367638450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb414144040 0x7fb41413e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.349+0000 7fb419e00640 1 -- 172.21.15.119:0/1367638450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb41415acd0 msgr2=0x7fb41413b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.349+0000 7fb419e00640 1 --2- 172.21.15.119:0/1367638450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb41415acd0 0x7fb41413b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.349+0000 7fb419e00640 1 -- 172.21.15.119:0/1367638450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41413bbd0 con 0x7fb41406ea30 2024-04-25T10:13:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.349+0000 7fb419e00640 1 --2- 172.21.15.119:0/1367638450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41406ea30 0x7fb414151810 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b4d0 tx=0x7fb40800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.349+0000 7fb40e000640 1 -- 172.21.15.119:0/1367638450 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408014070 con 0x7fb41406ea30 2024-04-25T10:13:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.349+0000 7fb40e000640 1 -- 172.21.15.119:0/1367638450 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb408010870 con 0x7fb41406ea30 2024-04-25T10:13:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.349+0000 7fb40e000640 1 -- 172.21.15.119:0/1367638450 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408010c60 con 0x7fb41406ea30 2024-04-25T10:13:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.349+0000 7fb41b200640 1 -- 172.21.15.119:0/1367638450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41413beb0 con 0x7fb41406ea30 2024-04-25T10:13:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.349+0000 7fb41b200640 1 -- 172.21.15.119:0/1367638450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41415ce40 con 0x7fb41406ea30 2024-04-25T10:13:08.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.350+0000 7fb41b200640 1 -- 172.21.15.119:0/1367638450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41406bb30 con 0x7fb41406ea30 2024-04-25T10:13:08.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.351+0000 7fb40e000640 1 -- 172.21.15.119:0/1367638450 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb408017070 con 0x7fb41406ea30 2024-04-25T10:13:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.352+0000 7fb40e000640 1 --2- 172.21.15.119:0/1367638450 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb3f406b4d0 0x7fb3f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.352+0000 7fb40e000640 1 -- 172.21.15.119:0/1367638450 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb40808a140 con 0x7fb41406ea30 2024-04-25T10:13:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.353+0000 7fb419400640 1 --2- 172.21.15.119:0/1367638450 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb3f406b4d0 0x7fb3f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.353+0000 7fb419400640 1 --2- 172.21.15.119:0/1367638450 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb3f406b4d0 0x7fb3f406d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb414154830 tx=0x7fb400006ba0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.356+0000 7fb40e000640 1 -- 172.21.15.119:0/1367638450 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb408058e90 con 0x7fb41406ea30 2024-04-25T10:13:08.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.571+0000 7fb41b200640 1 -- 172.21.15.119:0/1367638450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb41406e3c0 con 0x7fb41406ea30 2024-04-25T10:13:08.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.572+0000 7fb40e000640 1 -- 172.21.15.119:0/1367638450 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4951 (secure 0 0 0) 0x7fb408058790 con 0x7fb41406ea30 2024-04-25T10:13:08.577 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:08.577 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2798722683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2798722683},{"type":"v1","addr":"172.21.15.132:6833","nonce":2798722683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2834911408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2834911408},{"type":"v1","addr":"172.21.15.132:6835","nonce":2834911408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/643040335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":643040335},{"type":"v1","addr":"172.21.15.119:6835","nonce":643040335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:34.712290+0000","modified":"2024-04-25T10:12:34.407591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6837/2127292269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2127292269},{"type":"v1","addr":"172.21.15.119:6837","nonce":2127292269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:13:08.578 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 52 2024-04-25T10:13:08.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.575+0000 7fb41b200640 1 -- 172.21.15.119:0/1367638450 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb3f406b4d0 msgr2=0x7fb3f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.575+0000 7fb41b200640 1 --2- 172.21.15.119:0/1367638450 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb3f406b4d0 0x7fb3f406d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb414154830 tx=0x7fb400006ba0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.575+0000 7fb41b200640 1 -- 172.21.15.119:0/1367638450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41406ea30 msgr2=0x7fb414151810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.575+0000 7fb41b200640 1 --2- 172.21.15.119:0/1367638450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41406ea30 0x7fb414151810 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b4d0 tx=0x7fb40800b990 comp rx=0 tx=0).stop 2024-04-25T10:13:08.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.575+0000 7fb41b200640 1 -- 172.21.15.119:0/1367638450 shutdown_connections 2024-04-25T10:13:08.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.575+0000 7fb41b200640 1 --2- 172.21.15.119:0/1367638450 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb3f406b4d0 0x7fb3f406d990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.575+0000 7fb41b200640 1 --2- 172.21.15.119:0/1367638450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb41415acd0 0x7fb41413b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.575+0000 7fb41b200640 1 --2- 172.21.15.119:0/1367638450 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb414144040 0x7fb41413e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.575+0000 7fb41b200640 1 --2- 172.21.15.119:0/1367638450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41406ea30 0x7fb414151810 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.575+0000 7fb41b200640 1 -- 172.21.15.119:0/1367638450 >> 172.21.15.119:0/1367638450 conn(0x7fb41406b1d0 msgr2=0x7fb4140712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.575+0000 7fb41b200640 1 -- 172.21.15.119:0/1367638450 shutdown_connections 2024-04-25T10:13:08.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.575+0000 7fb41b200640 1 -- 172.21.15.119:0/1367638450 wait complete. 2024-04-25T10:13:08.597 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:08.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.790+0000 7f777c600640 1 Processor -- start 2024-04-25T10:13:08.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.790+0000 7f777c600640 1 -- start start 2024-04-25T10:13:08.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.791+0000 7f777c600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f777415a870 0x7f77740cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.791+0000 7f777c600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f777406e2e0 0x7f777415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.791+0000 7f777c600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77740cdb10 con 0x7f77740cd580 2024-04-25T10:13:08.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.791+0000 7f777c600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77740ce720 con 0x7f777406e2e0 2024-04-25T10:13:08.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.792+0000 7f777c600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77740cddf0 con 0x7f777415a870 2024-04-25T10:13:08.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.792+0000 7f777a600640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f77740cd580 0x7f77740cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41604/0 (socket says 172.21.15.119:41604) 2024-04-25T10:13:08.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.792+0000 7f777a600640 1 -- 172.21.15.119:0/1755007629 learned_addr learned my addr 172.21.15.119:0/1755007629 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:08.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.792+0000 7f7779c00640 1 --2- 172.21.15.119:0/1755007629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f777415a870 0x7f77740cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.792+0000 7f7779200640 1 --2- 172.21.15.119:0/1755007629 >> v2:172.21.15.132:3300/0 conn(0x7f777406e2e0 0x7f777415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.792+0000 7f7773e00640 1 -- 172.21.15.119:0/1755007629 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663236993 0 0) 0x7f77740cdb10 con 0x7f77740cd580 2024-04-25T10:13:08.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.792+0000 7f7773e00640 1 -- 172.21.15.119:0/1755007629 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f775c005730 con 0x7f77740cd580 2024-04-25T10:13:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7773e00640 1 -- 172.21.15.119:0/1755007629 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3656744398 0 0) 0x7f775c005730 con 0x7f77740cd580 2024-04-25T10:13:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7773e00640 1 -- 172.21.15.119:0/1755007629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f777415a870 msgr2=0x7f77740cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7773e00640 1 --2- 172.21.15.119:0/1755007629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f777415a870 0x7f77740cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7773e00640 1 -- 172.21.15.119:0/1755007629 >> v2:172.21.15.132:3300/0 conn(0x7f777406e2e0 msgr2=0x7f777415ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7773e00640 1 --2- 172.21.15.119:0/1755007629 >> v2:172.21.15.132:3300/0 conn(0x7f777406e2e0 0x7f777415ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7773e00640 1 -- 172.21.15.119:0/1755007629 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77740ce0d0 con 0x7f77740cd580 2024-04-25T10:13:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7773e00640 1 -- 172.21.15.119:0/1755007629 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f776c001670 con 0x7f77740cd580 2024-04-25T10:13:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7773e00640 1 -- 172.21.15.119:0/1755007629 >> v1:172.21.15.119:6789/0 conn(0x7f77740cd580 legacy=0x7f77740cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7773e00640 1 --2- 172.21.15.119:0/1755007629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c005bc0 0x7f775c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7773e00640 1 --2- 172.21.15.119:0/1755007629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f775c006a10 0x7f775c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7773e00640 1 --2- 172.21.15.119:0/1755007629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f777415a870 0x7f775c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7773e00640 1 -- 172.21.15.119:0/1755007629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77740ce720 con 0x7f775c005bc0 2024-04-25T10:13:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7773e00640 1 -- 172.21.15.119:0/1755007629 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77740cddf0 con 0x7f775c006a10 2024-04-25T10:13:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7773e00640 1 -- 172.21.15.119:0/1755007629 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775c00dbd0 con 0x7f777415a870 2024-04-25T10:13:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.793+0000 7f7779c00640 1 --2- 172.21.15.119:0/1755007629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c005bc0 0x7f775c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.794+0000 7f7779c00640 1 -- 172.21.15.119:0/1755007629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f777415a870 msgr2=0x7f775c0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.794+0000 7f7779c00640 1 --2- 172.21.15.119:0/1755007629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f777415a870 0x7f775c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.794+0000 7f7779c00640 1 -- 172.21.15.119:0/1755007629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f775c006a10 msgr2=0x7f775c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.794+0000 7f7779c00640 1 --2- 172.21.15.119:0/1755007629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f775c006a10 0x7f775c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.794+0000 7f7779c00640 1 -- 172.21.15.119:0/1755007629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c005730 con 0x7f775c005bc0 2024-04-25T10:13:08.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.794+0000 7f7779c00640 1 --2- 172.21.15.119:0/1755007629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c005bc0 0x7f775c005fe0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f7768009ac0 tx=0x7f775c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e7041e09e337ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.795+0000 7f7773e00640 1 -- 172.21.15.119:0/1755007629 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7768036070 con 0x7f775c005bc0 2024-04-25T10:13:08.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.795+0000 7f7773e00640 1 -- 172.21.15.119:0/1755007629 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7768031400 con 0x7f775c005bc0 2024-04-25T10:13:08.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.795+0000 7f7773e00640 1 -- 172.21.15.119:0/1755007629 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7768009690 con 0x7f775c005bc0 2024-04-25T10:13:08.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.795+0000 7f777c600640 1 -- 172.21.15.119:0/1755007629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c005bc0 msgr2=0x7f775c005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.795+0000 7f777c600640 1 --2- 172.21.15.119:0/1755007629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c005bc0 0x7f775c005fe0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f7768009ac0 tx=0x7f775c007a40 comp rx=0 tx=0).stop 2024-04-25T10:13:08.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.795+0000 7f777a600640 1 -- 172.21.15.119:0/1755007629 reap_dead start 2024-04-25T10:13:08.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.795+0000 7f777c600640 1 -- 172.21.15.119:0/1755007629 shutdown_connections 2024-04-25T10:13:08.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.795+0000 7f777c600640 1 -- 172.21.15.119:0/1755007629 >> 172.21.15.119:0/1755007629 conn(0x7f777406aba0 msgr2=0x7f777406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.795+0000 7f777c600640 1 -- 172.21.15.119:0/1755007629 shutdown_connections 2024-04-25T10:13:08.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.795+0000 7f777c600640 1 -- 172.21.15.119:0/1755007629 wait complete. 2024-04-25T10:13:08.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.797+0000 7f777c600640 1 Processor -- start 2024-04-25T10:13:08.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.797+0000 7f777c600640 1 -- start start 2024-04-25T10:13:08.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.798+0000 7f777c600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c005bc0 0x7f777415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.798+0000 7f777c600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7774153150 0x7f77740b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.798+0000 7f777c600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f777415b9b0 0x7f777415be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.798+0000 7f777c600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77740cfdc0 con 0x7f775c005bc0 2024-04-25T10:13:08.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.798+0000 7f777c600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77740cfc40 con 0x7f7774153150 2024-04-25T10:13:08.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.798+0000 7f777c600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77740cebb0 con 0x7f777415b9b0 2024-04-25T10:13:08.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.798+0000 7f7779c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c005bc0 0x7f777415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.798+0000 7f7779c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c005bc0 0x7f777415d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35300/0 (socket says 172.21.15.119:35300) 2024-04-25T10:13:08.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.798+0000 7f7779c00640 1 -- 172.21.15.119:0/3074454438 learned_addr learned my addr 172.21.15.119:0/3074454438 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:08.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.799+0000 7f7779c00640 1 -- 172.21.15.119:0/3074454438 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f777415b9b0 msgr2=0x7f777415be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.799+0000 7f7779c00640 1 --2- 172.21.15.119:0/3074454438 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f777415b9b0 0x7f777415be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.799+0000 7f7779c00640 1 -- 172.21.15.119:0/3074454438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7774153150 msgr2=0x7f77740b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.799+0000 7f7779c00640 1 --2- 172.21.15.119:0/3074454438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7774153150 0x7f77740b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.799+0000 7f7779c00640 1 -- 172.21.15.119:0/3074454438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77741ef5c0 con 0x7f775c005bc0 2024-04-25T10:13:08.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.799+0000 7f7779c00640 1 --2- 172.21.15.119:0/3074454438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c005bc0 0x7f777415d8f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f7768034040 tx=0x7f7768004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.799+0000 7f7771600640 1 -- 172.21.15.119:0/3074454438 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7768040020 con 0x7f775c005bc0 2024-04-25T10:13:08.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.799+0000 7f7771600640 1 -- 172.21.15.119:0/3074454438 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77680038d0 con 0x7f775c005bc0 2024-04-25T10:13:08.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.799+0000 7f7771600640 1 -- 172.21.15.119:0/3074454438 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7768036040 con 0x7f775c005bc0 2024-04-25T10:13:08.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.799+0000 7f777c600640 1 -- 172.21.15.119:0/3074454438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77741ef860 con 0x7f775c005bc0 2024-04-25T10:13:08.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.799+0000 7f777c600640 1 -- 172.21.15.119:0/3074454438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77741f0100 con 0x7f775c005bc0 2024-04-25T10:13:08.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.801+0000 7f777c600640 1 -- 172.21.15.119:0/3074454438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77741efc50 con 0x7f775c005bc0 2024-04-25T10:13:08.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.801+0000 7f7771600640 1 -- 172.21.15.119:0/3074454438 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7768043020 con 0x7f775c005bc0 2024-04-25T10:13:08.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.803+0000 7f7771600640 1 --2- 172.21.15.119:0/3074454438 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f775806b500 0x7f775806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.803+0000 7f7771600640 1 -- 172.21.15.119:0/3074454438 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f77680aa3b0 con 0x7f775c005bc0 2024-04-25T10:13:08.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.803+0000 7f7779200640 1 --2- 172.21.15.119:0/3074454438 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f775806b500 0x7f775806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.803+0000 7f7779200640 1 --2- 172.21.15.119:0/3074454438 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f775806b500 0x7f775806d9c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f77640070b0 tx=0x7f776400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:08.806+0000 7f7771600640 1 -- 172.21.15.119:0/3074454438 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f776804a070 con 0x7f775c005bc0 2024-04-25T10:13:09.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.001+0000 7f777c600640 1 -- 172.21.15.119:0/3074454438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77740749b0 con 0x7f775c005bc0 2024-04-25T10:13:09.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.004+0000 7f7771600640 1 -- 172.21.15.119:0/3074454438 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f7768079100 con 0x7f775c005bc0 2024-04-25T10:13:09.011 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:09.011 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":70,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:11:51.402126+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:32.500199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:33.708192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.22","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/2673349178":"2024-04-26T10:11:47.336991+0000","172.21.15.119:6836/2673349178":"2024-04-26T10:11:47.336991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.006+0000 7f777c600640 1 -- 172.21.15.119:0/3074454438 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f775806b500 msgr2=0x7f775806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.006+0000 7f777c600640 1 --2- 172.21.15.119:0/3074454438 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f775806b500 0x7f775806d9c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f77640070b0 tx=0x7f776400b040 comp rx=0 tx=0).stop 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.006+0000 7f777c600640 1 -- 172.21.15.119:0/3074454438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c005bc0 msgr2=0x7f777415d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.006+0000 7f777c600640 1 --2- 172.21.15.119:0/3074454438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c005bc0 0x7f777415d8f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f7768034040 tx=0x7f7768004730 comp rx=0 tx=0).stop 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.006+0000 7f777c600640 1 -- 172.21.15.119:0/3074454438 shutdown_connections 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.006+0000 7f777c600640 1 --2- 172.21.15.119:0/3074454438 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f775806b500 0x7f775806d9c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.006+0000 7f777c600640 1 --2- 172.21.15.119:0/3074454438 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f777415b9b0 0x7f777415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.006+0000 7f777c600640 1 --2- 172.21.15.119:0/3074454438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7774153150 0x7f77740b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.006+0000 7f777c600640 1 --2- 172.21.15.119:0/3074454438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f775c005bc0 0x7f777415d8f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.006+0000 7f777c600640 1 -- 172.21.15.119:0/3074454438 >> 172.21.15.119:0/3074454438 conn(0x7f777406aba0 msgr2=0x7f7774071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.007+0000 7f777c600640 1 -- 172.21.15.119:0/3074454438 shutdown_connections 2024-04-25T10:13:09.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.007+0000 7f777c600640 1 -- 172.21.15.119:0/3074454438 wait complete. 2024-04-25T10:13:09.031 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:13:09.032 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:09.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.228+0000 7f2208c00640 1 Processor -- start 2024-04-25T10:13:09.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.228+0000 7f2208c00640 1 -- start start 2024-04-25T10:13:09.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.229+0000 7f2208c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220415bf00 0x7f22040cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.229+0000 7f2208c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f22040cc310 0x7f220415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.229+0000 7f2208c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22040ce6c0 con 0x7f220415c820 2024-04-25T10:13:09.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.229+0000 7f2208c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22041455e0 con 0x7f22040cc310 2024-04-25T10:13:09.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.229+0000 7f2208c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220406d450 con 0x7f220415bf00 2024-04-25T10:13:09.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.230+0000 7f2202800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f220415c820 0x7f220406d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41612/0 (socket says 172.21.15.119:41612) 2024-04-25T10:13:09.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.230+0000 7f2202800640 1 -- 172.21.15.119:0/2135410725 learned_addr learned my addr 172.21.15.119:0/2135410725 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:09.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.230+0000 7f2201e00640 1 --2- 172.21.15.119:0/2135410725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220415bf00 0x7f22040cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.230+0000 7f21f9400640 1 --2- 172.21.15.119:0/2135410725 >> v2:172.21.15.132:3300/0 conn(0x7f22040cc310 0x7f220415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.230+0000 7f2201400640 1 -- 172.21.15.119:0/2135410725 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951939309 0 0) 0x7f21fc001490 con 0x7f220415c820 2024-04-25T10:13:09.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.230+0000 7f2201400640 1 -- 172.21.15.119:0/2135410725 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e8003520 con 0x7f220415c820 2024-04-25T10:13:09.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2201400640 1 -- 172.21.15.119:0/2135410725 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1795300572 0 0) 0x7f21e8003520 con 0x7f220415c820 2024-04-25T10:13:09.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2201400640 1 -- 172.21.15.119:0/2135410725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220415bf00 msgr2=0x7f22040cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2201400640 1 --2- 172.21.15.119:0/2135410725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220415bf00 0x7f22040cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2201400640 1 -- 172.21.15.119:0/2135410725 >> v2:172.21.15.132:3300/0 conn(0x7f22040cc310 msgr2=0x7f220415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2201400640 1 --2- 172.21.15.119:0/2135410725 >> v2:172.21.15.132:3300/0 conn(0x7f22040cc310 0x7f220415c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2201400640 1 -- 172.21.15.119:0/2135410725 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22040cc7b0 con 0x7f220415c820 2024-04-25T10:13:09.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2201400640 1 -- 172.21.15.119:0/2135410725 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f21fc001bc0 con 0x7f220415c820 2024-04-25T10:13:09.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2201400640 1 -- 172.21.15.119:0/2135410725 >> v1:172.21.15.119:6789/0 conn(0x7f220415c820 legacy=0x7f220406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2201400640 1 --2- 172.21.15.119:0/2135410725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e80039b0 0x7f21e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2201400640 1 --2- 172.21.15.119:0/2135410725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220415bf00 0x7f21e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2201400640 1 --2- 172.21.15.119:0/2135410725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21e8004d40 0x7f21e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2201400640 1 -- 172.21.15.119:0/2135410725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22041455e0 con 0x7f21e8004d40 2024-04-25T10:13:09.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2201400640 1 -- 172.21.15.119:0/2135410725 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220406d450 con 0x7f21e80039b0 2024-04-25T10:13:09.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2201400640 1 -- 172.21.15.119:0/2135410725 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e800b9c0 con 0x7f220415bf00 2024-04-25T10:13:09.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.231+0000 7f2202800640 1 --2- 172.21.15.119:0/2135410725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21e8004d40 0x7f21e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.232+0000 7f21f9400640 1 --2- 172.21.15.119:0/2135410725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220415bf00 0x7f21e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.232+0000 7f2201e00640 1 --2- 172.21.15.119:0/2135410725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e80039b0 0x7f21e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.232+0000 7f2202800640 1 -- 172.21.15.119:0/2135410725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220415bf00 msgr2=0x7f21e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.232+0000 7f2202800640 1 --2- 172.21.15.119:0/2135410725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f220415bf00 0x7f21e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.232+0000 7f2202800640 1 -- 172.21.15.119:0/2135410725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e80039b0 msgr2=0x7f21e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.232+0000 7f2202800640 1 --2- 172.21.15.119:0/2135410725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e80039b0 0x7f21e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.232+0000 7f2202800640 1 -- 172.21.15.119:0/2135410725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e8003520 con 0x7f21e8004d40 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.232+0000 7f2201e00640 1 --2- 172.21.15.119:0/2135410725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e80039b0 0x7f21e8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.233+0000 7f2202800640 1 --2- 172.21.15.119:0/2135410725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21e8004d40 0x7f21e80051c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00aa70 tx=0x7f21e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ca4972a8123a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.233+0000 7f2201400640 1 -- 172.21.15.119:0/2135410725 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc00a690 con 0x7f21e8004d40 2024-04-25T10:13:09.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.233+0000 7f2201400640 1 -- 172.21.15.119:0/2135410725 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc02f070 con 0x7f21e8004d40 2024-04-25T10:13:09.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.233+0000 7f2201400640 1 -- 172.21.15.119:0/2135410725 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc034470 con 0x7f21e8004d40 2024-04-25T10:13:09.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.234+0000 7f2208c00640 1 -- 172.21.15.119:0/2135410725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21e8004d40 msgr2=0x7f21e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.234+0000 7f2208c00640 1 --2- 172.21.15.119:0/2135410725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21e8004d40 0x7f21e80051c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00aa70 tx=0x7f21e8005830 comp rx=0 tx=0).stop 2024-04-25T10:13:09.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.234+0000 7f2202800640 1 -- 172.21.15.119:0/2135410725 reap_dead start 2024-04-25T10:13:09.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.234+0000 7f2208c00640 1 -- 172.21.15.119:0/2135410725 shutdown_connections 2024-04-25T10:13:09.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.234+0000 7f2208c00640 1 -- 172.21.15.119:0/2135410725 >> 172.21.15.119:0/2135410725 conn(0x7f220406aba0 msgr2=0x7f220406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.234+0000 7f2208c00640 1 -- 172.21.15.119:0/2135410725 shutdown_connections 2024-04-25T10:13:09.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.234+0000 7f2208c00640 1 -- 172.21.15.119:0/2135410725 wait complete. 2024-04-25T10:13:09.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.236+0000 7f2208c00640 1 Processor -- start 2024-04-25T10:13:09.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.236+0000 7f2208c00640 1 -- start start 2024-04-25T10:13:09.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.237+0000 7f2208c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e8004d40 0x7f22041394d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.237+0000 7f2208c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22040caf10 0x7f22040b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.237+0000 7f2208c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22040ca020 0x7f22040b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.237+0000 7f2208c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22040cd050 con 0x7f22040caf10 2024-04-25T10:13:09.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.237+0000 7f2208c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22040ccd70 con 0x7f22040ca020 2024-04-25T10:13:09.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.237+0000 7f2208c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220415cca0 con 0x7f21e8004d40 2024-04-25T10:13:09.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.237+0000 7f21f9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22040caf10 0x7f22040b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.237+0000 7f21f9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22040caf10 0x7f22040b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35308/0 (socket says 172.21.15.119:35308) 2024-04-25T10:13:09.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.237+0000 7f21f9400640 1 -- 172.21.15.119:0/1321858931 learned_addr learned my addr 172.21.15.119:0/1321858931 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:09.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.238+0000 7f2201e00640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e8004d40 0x7f22041394d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.238+0000 7f2202800640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22040ca020 0x7f22040b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.238+0000 7f21f9400640 1 -- 172.21.15.119:0/1321858931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e8004d40 msgr2=0x7f22041394d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.238+0000 7f21f9400640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e8004d40 0x7f22041394d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.238+0000 7f21f9400640 1 -- 172.21.15.119:0/1321858931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22040ca020 msgr2=0x7f22040b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.238+0000 7f21f9400640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22040ca020 0x7f22040b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.238+0000 7f21f9400640 1 -- 172.21.15.119:0/1321858931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22040cb390 con 0x7f22040caf10 2024-04-25T10:13:09.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.238+0000 7f2201e00640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e8004d40 0x7f22041394d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:09.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.238+0000 7f2202800640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22040ca020 0x7f22040b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:09.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.238+0000 7f21f9400640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22040caf10 0x7f22040b31f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f21ec011610 tx=0x7f21ec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.238+0000 7f21faa00640 1 -- 172.21.15.119:0/1321858931 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec004220 con 0x7f22040caf10 2024-04-25T10:13:09.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.238+0000 7f21faa00640 1 -- 172.21.15.119:0/1321858931 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ec0043c0 con 0x7f22040caf10 2024-04-25T10:13:09.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.238+0000 7f21faa00640 1 -- 172.21.15.119:0/1321858931 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec00f930 con 0x7f22040caf10 2024-04-25T10:13:09.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.238+0000 7f2208c00640 1 -- 172.21.15.119:0/1321858931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22040cb670 con 0x7f22040caf10 2024-04-25T10:13:09.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.239+0000 7f2208c00640 1 -- 172.21.15.119:0/1321858931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22040b29f0 con 0x7f22040caf10 2024-04-25T10:13:09.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.240+0000 7f2208c00640 1 -- 172.21.15.119:0/1321858931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22040749b0 con 0x7f22040caf10 2024-04-25T10:13:09.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.241+0000 7f21faa00640 1 -- 172.21.15.119:0/1321858931 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21ec00fad0 con 0x7f22040caf10 2024-04-25T10:13:09.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.241+0000 7f21faa00640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f21e006b4d0 0x7f21e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.242+0000 7f21faa00640 1 -- 172.21.15.119:0/1321858931 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f21ec08aa30 con 0x7f22040caf10 2024-04-25T10:13:09.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.242+0000 7f2201e00640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f21e006b4d0 0x7f21e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.242+0000 7f2201e00640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f21e006b4d0 0x7f21e006d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009aa0 tx=0x7f21f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.246+0000 7f21faa00640 1 -- 172.21.15.119:0/1321858931 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21ec019030 con 0x7f22040caf10 2024-04-25T10:13:09.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.474+0000 7f2208c00640 1 -- 172.21.15.119:0/1321858931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f220406c170 con 0x7f22040caf10 2024-04-25T10:13:09.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.475+0000 7f21faa00640 1 -- 172.21.15.119:0/1321858931 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f21ec059780 con 0x7f22040caf10 2024-04-25T10:13:09.480 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:09.481 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2798722683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2798722683},{"type":"v1","addr":"172.21.15.132:6833","nonce":2798722683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2834911408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2834911408},{"type":"v1","addr":"172.21.15.132:6835","nonce":2834911408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/643040335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":643040335},{"type":"v1","addr":"172.21.15.119:6835","nonce":643040335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:34.712290+0000","modified":"2024-04-25T10:12:34.407591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6837/2127292269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2127292269},{"type":"v1","addr":"172.21.15.119:6837","nonce":2127292269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:13:09.481 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 52 2024-04-25T10:13:09.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.478+0000 7f2208c00640 1 -- 172.21.15.119:0/1321858931 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f21e006b4d0 msgr2=0x7f21e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.478+0000 7f2208c00640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f21e006b4d0 0x7f21e006d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009aa0 tx=0x7f21f4008040 comp rx=0 tx=0).stop 2024-04-25T10:13:09.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.478+0000 7f2208c00640 1 -- 172.21.15.119:0/1321858931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22040caf10 msgr2=0x7f22040b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.478+0000 7f2208c00640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22040caf10 0x7f22040b31f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f21ec011610 tx=0x7f21ec011ad0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.478+0000 7f2208c00640 1 -- 172.21.15.119:0/1321858931 shutdown_connections 2024-04-25T10:13:09.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.478+0000 7f2208c00640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f21e006b4d0 0x7f21e006d990 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.478+0000 7f2208c00640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22040ca020 0x7f22040b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.478+0000 7f2208c00640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22040caf10 0x7f22040b31f0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.478+0000 7f2208c00640 1 --2- 172.21.15.119:0/1321858931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e8004d40 0x7f22041394d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.478+0000 7f2208c00640 1 -- 172.21.15.119:0/1321858931 >> 172.21.15.119:0/1321858931 conn(0x7f220406aba0 msgr2=0x7f220415d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.478+0000 7f2208c00640 1 -- 172.21.15.119:0/1321858931 shutdown_connections 2024-04-25T10:13:09.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.478+0000 7f2208c00640 1 -- 172.21.15.119:0/1321858931 wait complete. 2024-04-25T10:13:09.501 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:09.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.690+0000 7fea8bc00640 1 Processor -- start 2024-04-25T10:13:09.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.691+0000 7fea8bc00640 1 -- start start 2024-04-25T10:13:09.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.692+0000 7fea8bc00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fea8415c9f0 0x7fea841664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.692+0000 7fea8bc00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea840ce6c0 con 0x7fea8406e8f0 2024-04-25T10:13:09.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.692+0000 7fea8bc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea840cc4e0 con 0x7fea8415bf00 2024-04-25T10:13:09.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.692+0000 7fea8bc00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea841455e0 con 0x7fea8415c9f0 2024-04-25T10:13:09.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.692+0000 7fea83e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fea8406e8f0 0x7fea8415c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41614/0 (socket says 172.21.15.119:41614) 2024-04-25T10:13:09.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.692+0000 7fea83e00640 1 -- 172.21.15.119:0/2230960340 learned_addr learned my addr 172.21.15.119:0/2230960340 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:09.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.693+0000 7fea89c00640 1 --2- 172.21.15.119:0/2230960340 >> v2:172.21.15.119:3300/0 conn(0x7fea8415c9f0 0x7fea841664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.693+0000 7fea83400640 1 -- 172.21.15.119:0/2230960340 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745377317 0 0) 0x7fea70001110 con 0x7fea8406e8f0 2024-04-25T10:13:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.693+0000 7fea83400640 1 -- 172.21.15.119:0/2230960340 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea68003520 con 0x7fea8406e8f0 2024-04-25T10:13:09.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.693+0000 7fea89c00640 1 -- 172.21.15.119:0/2230960340 >> v1:172.21.15.132:6789/0 conn(0x7fea8415bf00 legacy=0x7fea840cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.693+0000 7fea89c00640 1 -- 172.21.15.119:0/2230960340 >> v1:172.21.15.119:6789/0 conn(0x7fea8406e8f0 legacy=0x7fea8415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.693+0000 7fea89c00640 1 -- 172.21.15.119:0/2230960340 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea840cc7b0 con 0x7fea8415c9f0 2024-04-25T10:13:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.694+0000 7fea89c00640 1 --2- 172.21.15.119:0/2230960340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea8415c9f0 0x7fea841664c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009d90 tx=0x7fea7c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c6d76b0bb971b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.695+0000 7fea83400640 1 -- 172.21.15.119:0/2230960340 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c007740 con 0x7fea8415c9f0 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.695+0000 7fea83400640 1 -- 172.21.15.119:0/2230960340 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7c0078e0 con 0x7fea8415c9f0 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.695+0000 7fea83400640 1 -- 172.21.15.119:0/2230960340 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c007cb0 con 0x7fea8415c9f0 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.695+0000 7fea8bc00640 1 -- 172.21.15.119:0/2230960340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea8415c9f0 msgr2=0x7fea841664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.695+0000 7fea8bc00640 1 --2- 172.21.15.119:0/2230960340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea8415c9f0 0x7fea841664c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009d90 tx=0x7fea7c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.696+0000 7fea8bc00640 1 -- 172.21.15.119:0/2230960340 shutdown_connections 2024-04-25T10:13:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.696+0000 7fea8bc00640 1 --2- 172.21.15.119:0/2230960340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea8415c9f0 0x7fea841664c0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.696+0000 7fea8bc00640 1 -- 172.21.15.119:0/2230960340 >> 172.21.15.119:0/2230960340 conn(0x7fea8406aba0 msgr2=0x7fea8406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.696+0000 7fea8bc00640 1 -- 172.21.15.119:0/2230960340 shutdown_connections 2024-04-25T10:13:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.696+0000 7fea8bc00640 1 -- 172.21.15.119:0/2230960340 wait complete. 2024-04-25T10:13:09.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.697+0000 7fea8bc00640 1 Processor -- start 2024-04-25T10:13:09.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.697+0000 7fea8bc00640 1 -- start start 2024-04-25T10:13:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.698+0000 7fea8bc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea8406e8f0 0x7fea840c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.698+0000 7fea8bc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea8415bf00 0x7fea840cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.698+0000 7fea8bc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea8415c9f0 0x7fea8413b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.698+0000 7fea8bc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea840ccd70 con 0x7fea8415bf00 2024-04-25T10:13:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.698+0000 7fea8bc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8406d450 con 0x7fea8415c9f0 2024-04-25T10:13:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.698+0000 7fea8bc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea840cd050 con 0x7fea8406e8f0 2024-04-25T10:13:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.699+0000 7fea83e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea8415bf00 0x7fea840cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.699+0000 7fea83e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea8415bf00 0x7fea840cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35332/0 (socket says 172.21.15.119:35332) 2024-04-25T10:13:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.699+0000 7fea83e00640 1 -- 172.21.15.119:0/2259967325 learned_addr learned my addr 172.21.15.119:0/2259967325 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.699+0000 7fea89200640 1 --2- 172.21.15.119:0/2259967325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea8406e8f0 0x7fea840c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.699+0000 7fea89c00640 1 --2- 172.21.15.119:0/2259967325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea8415c9f0 0x7fea8413b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.699+0000 7fea83e00640 1 -- 172.21.15.119:0/2259967325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea8406e8f0 msgr2=0x7fea840c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.699+0000 7fea83e00640 1 --2- 172.21.15.119:0/2259967325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea8406e8f0 0x7fea840c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.699+0000 7fea83e00640 1 -- 172.21.15.119:0/2259967325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea8415c9f0 msgr2=0x7fea8413b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.699+0000 7fea83e00640 1 --2- 172.21.15.119:0/2259967325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea8415c9f0 0x7fea8413b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.699+0000 7fea83e00640 1 -- 172.21.15.119:0/2259967325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea840b28c0 con 0x7fea8415bf00 2024-04-25T10:13:09.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.699+0000 7fea83e00640 1 --2- 172.21.15.119:0/2259967325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea8415bf00 0x7fea840cf730 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fea7000c730 tx=0x7fea7000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.700+0000 7fea80c00640 1 -- 172.21.15.119:0/2259967325 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70017070 con 0x7fea8415bf00 2024-04-25T10:13:09.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.700+0000 7fea80c00640 1 -- 172.21.15.119:0/2259967325 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea70005ab0 con 0x7fea8415bf00 2024-04-25T10:13:09.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.700+0000 7fea8bc00640 1 -- 172.21.15.119:0/2259967325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea840b2ba0 con 0x7fea8415bf00 2024-04-25T10:13:09.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.700+0000 7fea80c00640 1 -- 172.21.15.119:0/2259967325 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70007070 con 0x7fea8415bf00 2024-04-25T10:13:09.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.700+0000 7fea8bc00640 1 -- 172.21.15.119:0/2259967325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea840b3160 con 0x7fea8415bf00 2024-04-25T10:13:09.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.701+0000 7fea8bc00640 1 -- 172.21.15.119:0/2259967325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea8406bb30 con 0x7fea8415bf00 2024-04-25T10:13:09.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.702+0000 7fea80c00640 1 -- 172.21.15.119:0/2259967325 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea7001a070 con 0x7fea8415bf00 2024-04-25T10:13:09.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.703+0000 7fea80c00640 1 --2- 172.21.15.119:0/2259967325 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fea6406b4d0 0x7fea6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.703+0000 7fea80c00640 1 -- 172.21.15.119:0/2259967325 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fea70089760 con 0x7fea8415bf00 2024-04-25T10:13:09.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.703+0000 7fea89200640 1 --2- 172.21.15.119:0/2259967325 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fea6406b4d0 0x7fea6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.704+0000 7fea89200640 1 --2- 172.21.15.119:0/2259967325 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fea6406b4d0 0x7fea6406d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fea74000f60 tx=0x7fea74009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.707+0000 7fea80c00640 1 -- 172.21.15.119:0/2259967325 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea840ce6c0 con 0x7fea8415bf00 2024-04-25T10:13:09.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.938+0000 7fea8bc00640 1 -- 172.21.15.119:0/2259967325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea840749b0 con 0x7fea8415bf00 2024-04-25T10:13:09.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.939+0000 7fea80c00640 1 -- 172.21.15.119:0/2259967325 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4951 (secure 0 0 0) 0x7fea70058540 con 0x7fea8415bf00 2024-04-25T10:13:09.944 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:09.945 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2798722683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2798722683},{"type":"v1","addr":"172.21.15.132:6833","nonce":2798722683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2834911408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2834911408},{"type":"v1","addr":"172.21.15.132:6835","nonce":2834911408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/643040335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":643040335},{"type":"v1","addr":"172.21.15.119:6835","nonce":643040335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:34.712290+0000","modified":"2024-04-25T10:12:34.407591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6837/2127292269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2127292269},{"type":"v1","addr":"172.21.15.119:6837","nonce":2127292269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:13:09.945 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 52 2024-04-25T10:13:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.942+0000 7fea8bc00640 1 -- 172.21.15.119:0/2259967325 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fea6406b4d0 msgr2=0x7fea6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.942+0000 7fea8bc00640 1 --2- 172.21.15.119:0/2259967325 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fea6406b4d0 0x7fea6406d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fea74000f60 tx=0x7fea74009040 comp rx=0 tx=0).stop 2024-04-25T10:13:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.942+0000 7fea8bc00640 1 -- 172.21.15.119:0/2259967325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea8415bf00 msgr2=0x7fea840cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.942+0000 7fea8bc00640 1 --2- 172.21.15.119:0/2259967325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea8415bf00 0x7fea840cf730 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fea7000c730 tx=0x7fea7000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.942+0000 7fea8bc00640 1 -- 172.21.15.119:0/2259967325 shutdown_connections 2024-04-25T10:13:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.942+0000 7fea8bc00640 1 --2- 172.21.15.119:0/2259967325 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fea6406b4d0 0x7fea6406d990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.942+0000 7fea8bc00640 1 --2- 172.21.15.119:0/2259967325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fea8415c9f0 0x7fea8413b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.942+0000 7fea8bc00640 1 --2- 172.21.15.119:0/2259967325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea8415bf00 0x7fea840cf730 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.942+0000 7fea8bc00640 1 --2- 172.21.15.119:0/2259967325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fea8406e8f0 0x7fea840c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.942+0000 7fea8bc00640 1 -- 172.21.15.119:0/2259967325 >> 172.21.15.119:0/2259967325 conn(0x7fea8406aba0 msgr2=0x7fea84071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.942+0000 7fea8bc00640 1 -- 172.21.15.119:0/2259967325 shutdown_connections 2024-04-25T10:13:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:09.943+0000 7fea8bc00640 1 -- 172.21.15.119:0/2259967325 wait complete. 2024-04-25T10:13:09.967 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.164+0000 7f6b53600640 1 Processor -- start 2024-04-25T10:13:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.164+0000 7f6b53600640 1 -- start start 2024-04-25T10:13:10.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.166+0000 7f6b53600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6b4c15d980 0x7f6b4c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.166+0000 7f6b53600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6b4c0aeb20 0x7f6b4c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.166+0000 7f6b53600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4c0cfbd0 con 0x7f6b4c0cf7c0 2024-04-25T10:13:10.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.166+0000 7f6b53600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b4c071ed0 con 0x7f6b4c0aeb20 2024-04-25T10:13:10.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.166+0000 7f6b53600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b4c0712c0 con 0x7f6b4c15d980 2024-04-25T10:13:10.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.166+0000 7f6b51600640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6b4c0cf7c0 0x7f6b4c15dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41626/0 (socket says 172.21.15.119:41626) 2024-04-25T10:13:10.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.166+0000 7f6b43e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6b4c0aeb20 0x7f6b4c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.166+0000 7f6b51600640 1 -- 172.21.15.119:0/321042164 learned_addr learned my addr 172.21.15.119:0/321042164 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:10.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.167+0000 7f6b50c00640 1 --2- 172.21.15.119:0/321042164 >> v2:172.21.15.132:3300/0 conn(0x7f6b4c15d980 0x7f6b4c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.167+0000 7f6b43e00640 1 -- 172.21.15.119:0/321042164 >> v2:172.21.15.132:3300/0 conn(0x7f6b4c15d980 msgr2=0x7f6b4c0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:10.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.167+0000 7f6b43e00640 1 --2- 172.21.15.119:0/321042164 >> v2:172.21.15.132:3300/0 conn(0x7f6b4c15d980 0x7f6b4c0ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.167+0000 7f6b43e00640 1 -- 172.21.15.119:0/321042164 >> v1:172.21.15.119:6789/0 conn(0x7f6b4c0cf7c0 legacy=0x7f6b4c15dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.167+0000 7f6b43e00640 1 -- 172.21.15.119:0/321042164 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c0715a0 con 0x7f6b4c0aeb20 2024-04-25T10:13:10.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.167+0000 7f6b4be00640 1 -- 172.21.15.119:0/321042164 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571227576 0 0) 0x7f6b44001490 con 0x7f6b4c0cf7c0 2024-04-25T10:13:10.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.168+0000 7f6b43e00640 1 --2- 172.21.15.119:0/321042164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b4c0aeb20 0x7f6b4c0cf280 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c067460 tx=0x7f6b3402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22dd574057a19f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.168+0000 7f6b4be00640 1 -- 172.21.15.119:0/321042164 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34037070 con 0x7f6b4c0aeb20 2024-04-25T10:13:10.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.168+0000 7f6b4be00640 1 -- 172.21.15.119:0/321042164 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b34032440 con 0x7f6b4c0aeb20 2024-04-25T10:13:10.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.168+0000 7f6b4be00640 1 -- 172.21.15.119:0/321042164 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3402ae60 con 0x7f6b4c0aeb20 2024-04-25T10:13:10.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.169+0000 7f6b53600640 1 -- 172.21.15.119:0/321042164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b4c0aeb20 msgr2=0x7f6b4c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.169+0000 7f6b53600640 1 --2- 172.21.15.119:0/321042164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b4c0aeb20 0x7f6b4c0cf280 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c067460 tx=0x7f6b3402a110 comp rx=0 tx=0).stop 2024-04-25T10:13:10.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.169+0000 7f6b53600640 1 -- 172.21.15.119:0/321042164 shutdown_connections 2024-04-25T10:13:10.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.169+0000 7f6b53600640 1 --2- 172.21.15.119:0/321042164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b4c0aeb20 0x7f6b4c0cf280 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.169+0000 7f6b53600640 1 --2- 172.21.15.119:0/321042164 >> v2:172.21.15.132:3300/0 conn(0x7f6b4c15d980 0x7f6b4c0ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.169+0000 7f6b53600640 1 -- 172.21.15.119:0/321042164 >> 172.21.15.119:0/321042164 conn(0x7f6b4c06aba0 msgr2=0x7f6b4c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.169+0000 7f6b53600640 1 -- 172.21.15.119:0/321042164 shutdown_connections 2024-04-25T10:13:10.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.169+0000 7f6b53600640 1 -- 172.21.15.119:0/321042164 wait complete. 2024-04-25T10:13:10.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.171+0000 7f6b53600640 1 Processor -- start 2024-04-25T10:13:10.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.171+0000 7f6b53600640 1 -- start start 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.172+0000 7f6b53600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b4c0aeb20 0x7f6b4c150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.172+0000 7f6b53600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4c0cf7c0 0x7f6b4c1567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.172+0000 7f6b53600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b4c15d980 0x7f6b4c156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.172+0000 7f6b53600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c0adf80 con 0x7f6b4c15d980 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.172+0000 7f6b53600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c071b60 con 0x7f6b4c0aeb20 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.172+0000 7f6b53600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c0ae100 con 0x7f6b4c0cf7c0 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.172+0000 7f6b51600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b4c15d980 0x7f6b4c156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.172+0000 7f6b51600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b4c15d980 0x7f6b4c156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35350/0 (socket says 172.21.15.119:35350) 2024-04-25T10:13:10.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.172+0000 7f6b51600640 1 -- 172.21.15.119:0/3126203959 learned_addr learned my addr 172.21.15.119:0/3126203959 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.173+0000 7f6b51600640 1 -- 172.21.15.119:0/3126203959 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4c0cf7c0 msgr2=0x7f6b4c1567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.173+0000 7f6b43e00640 1 --2- 172.21.15.119:0/3126203959 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4c0cf7c0 0x7f6b4c1567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.173+0000 7f6b50c00640 1 --2- 172.21.15.119:0/3126203959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b4c0aeb20 0x7f6b4c150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.173+0000 7f6b51600640 1 --2- 172.21.15.119:0/3126203959 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4c0cf7c0 0x7f6b4c1567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.173+0000 7f6b51600640 1 -- 172.21.15.119:0/3126203959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b4c0aeb20 msgr2=0x7f6b4c150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.173+0000 7f6b51600640 1 --2- 172.21.15.119:0/3126203959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b4c0aeb20 0x7f6b4c150640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.173+0000 7f6b51600640 1 -- 172.21.15.119:0/3126203959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c15aa30 con 0x7f6b4c15d980 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.173+0000 7f6b51600640 1 --2- 172.21.15.119:0/3126203959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b4c15d980 0x7f6b4c156d10 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400cb60 tx=0x7f6b4400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.173+0000 7f6b49600640 1 -- 172.21.15.119:0/3126203959 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4400ace0 con 0x7f6b4c15d980 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.173+0000 7f6b49600640 1 -- 172.21.15.119:0/3126203959 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b44012400 con 0x7f6b4c15d980 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.173+0000 7f6b49600640 1 -- 172.21.15.119:0/3126203959 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b440127f0 con 0x7f6b4c15d980 2024-04-25T10:13:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.173+0000 7f6b53600640 1 -- 172.21.15.119:0/3126203959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4c15ad10 con 0x7f6b4c15d980 2024-04-25T10:13:10.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.174+0000 7f6b53600640 1 -- 172.21.15.119:0/3126203959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4c15b2d0 con 0x7f6b4c15d980 2024-04-25T10:13:10.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.175+0000 7f6b53600640 1 -- 172.21.15.119:0/3126203959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4c0749b0 con 0x7f6b4c15d980 2024-04-25T10:13:10.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.176+0000 7f6b49600640 1 -- 172.21.15.119:0/3126203959 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b44012990 con 0x7f6b4c15d980 2024-04-25T10:13:10.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.176+0000 7f6b49600640 1 --2- 172.21.15.119:0/3126203959 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b2806b4d0 0x7f6b2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.176+0000 7f6b49600640 1 -- 172.21.15.119:0/3126203959 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6b4408a9f0 con 0x7f6b4c15d980 2024-04-25T10:13:10.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.177+0000 7f6b50c00640 1 --2- 172.21.15.119:0/3126203959 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b2806b4d0 0x7f6b2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.177+0000 7f6b50c00640 1 --2- 172.21.15.119:0/3126203959 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b2806b4d0 0x7f6b2806d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c006fd0 tx=0x7f6b3c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.181+0000 7f6b49600640 1 -- 172.21.15.119:0/3126203959 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b4c0cfbd0 con 0x7f6b4c15d980 2024-04-25T10:13:10.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.380+0000 7f6b53600640 1 -- 172.21.15.119:0/3126203959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b4c06d4c0 con 0x7f6b4c15d980 2024-04-25T10:13:10.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.382+0000 7f6b49600640 1 -- 172.21.15.119:0/3126203959 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f6b44059740 con 0x7f6b4c15d980 2024-04-25T10:13:10.388 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:10.388 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":70,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:11:51.402126+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:32.500199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:33.708192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.22","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/2673349178":"2024-04-26T10:11:47.336991+0000","172.21.15.119:6836/2673349178":"2024-04-26T10:11:47.336991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.385+0000 7f6b53600640 1 -- 172.21.15.119:0/3126203959 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b2806b4d0 msgr2=0x7f6b2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.385+0000 7f6b53600640 1 --2- 172.21.15.119:0/3126203959 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b2806b4d0 0x7f6b2806d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c006fd0 tx=0x7f6b3c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.385+0000 7f6b53600640 1 -- 172.21.15.119:0/3126203959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b4c15d980 msgr2=0x7f6b4c156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.385+0000 7f6b53600640 1 --2- 172.21.15.119:0/3126203959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b4c15d980 0x7f6b4c156d10 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400cb60 tx=0x7f6b4400a430 comp rx=0 tx=0).stop 2024-04-25T10:13:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.385+0000 7f6b53600640 1 -- 172.21.15.119:0/3126203959 shutdown_connections 2024-04-25T10:13:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.385+0000 7f6b53600640 1 --2- 172.21.15.119:0/3126203959 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b2806b4d0 0x7f6b2806d990 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.385+0000 7f6b53600640 1 --2- 172.21.15.119:0/3126203959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b4c15d980 0x7f6b4c156d10 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.385+0000 7f6b53600640 1 --2- 172.21.15.119:0/3126203959 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b4c0cf7c0 0x7f6b4c1567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.386+0000 7f6b53600640 1 --2- 172.21.15.119:0/3126203959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b4c0aeb20 0x7f6b4c150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.386+0000 7f6b53600640 1 -- 172.21.15.119:0/3126203959 >> 172.21.15.119:0/3126203959 conn(0x7f6b4c06aba0 msgr2=0x7f6b4c0afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.386+0000 7f6b53600640 1 -- 172.21.15.119:0/3126203959 shutdown_connections 2024-04-25T10:13:10.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.386+0000 7f6b53600640 1 -- 172.21.15.119:0/3126203959 wait complete. 2024-04-25T10:13:10.409 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:13:10.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.598+0000 7fe12b400640 1 Processor -- start 2024-04-25T10:13:10.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.598+0000 7fe12b400640 1 -- start start 2024-04-25T10:13:10.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.600+0000 7fe12b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1241610b0 0x7fe1240b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.600+0000 7fe12b400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe1241618d0 0x7fe1240ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.600+0000 7fe12b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe1240ced70 0x7fe12406c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.600+0000 7fe12b400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1240d1ea0 con 0x7fe1240ced70 2024-04-25T10:13:10.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.600+0000 7fe12b400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe124162040 con 0x7fe1241618d0 2024-04-25T10:13:10.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.600+0000 7fe12b400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe124162320 con 0x7fe1241610b0 2024-04-25T10:13:10.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.600+0000 7fe129400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe1240ced70 0x7fe12406c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.600+0000 7fe129400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe1240ced70 0x7fe12406c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35362/0 (socket says 172.21.15.119:35362) 2024-04-25T10:13:10.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.600+0000 7fe129400640 1 -- 172.21.15.119:0/1818323261 learned_addr learned my addr 172.21.15.119:0/1818323261 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:10.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.601+0000 7fe123e00640 1 --2- 172.21.15.119:0/1818323261 >> v2:172.21.15.132:3300/0 conn(0x7fe1241618d0 0x7fe1240ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.601+0000 7fe128a00640 1 --2- 172.21.15.119:0/1818323261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1241610b0 0x7fe1240b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.601+0000 7fe129400640 1 -- 172.21.15.119:0/1818323261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1241610b0 msgr2=0x7fe1240b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:10.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.601+0000 7fe129400640 1 --2- 172.21.15.119:0/1818323261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1241610b0 0x7fe1240b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.601+0000 7fe129400640 1 -- 172.21.15.119:0/1818323261 >> v2:172.21.15.132:3300/0 conn(0x7fe1241618d0 msgr2=0x7fe1240ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:10.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.601+0000 7fe129400640 1 --2- 172.21.15.119:0/1818323261 >> v2:172.21.15.132:3300/0 conn(0x7fe1241618d0 0x7fe1240ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.601+0000 7fe129400640 1 -- 172.21.15.119:0/1818323261 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe124162600 con 0x7fe1240ced70 2024-04-25T10:13:10.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.602+0000 7fe129400640 1 --2- 172.21.15.119:0/1818323261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240ced70 0x7fe12406c250 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe1240d4b60 tx=0x7fe11c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e509aa5c28ce03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.603+0000 7fe123400640 1 -- 172.21.15.119:0/1818323261 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c036070 con 0x7fe1240ced70 2024-04-25T10:13:10.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.603+0000 7fe123400640 1 -- 172.21.15.119:0/1818323261 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11c031600 con 0x7fe1240ced70 2024-04-25T10:13:10.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.603+0000 7fe123400640 1 -- 172.21.15.119:0/1818323261 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c0319d0 con 0x7fe1240ced70 2024-04-25T10:13:10.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.603+0000 7fe12b400640 1 -- 172.21.15.119:0/1818323261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240ced70 msgr2=0x7fe12406c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.603+0000 7fe12b400640 1 --2- 172.21.15.119:0/1818323261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240ced70 0x7fe12406c250 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe1240d4b60 tx=0x7fe11c02a610 comp rx=0 tx=0).stop 2024-04-25T10:13:10.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.603+0000 7fe12b400640 1 -- 172.21.15.119:0/1818323261 shutdown_connections 2024-04-25T10:13:10.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.603+0000 7fe12b400640 1 --2- 172.21.15.119:0/1818323261 >> v2:172.21.15.132:3300/0 conn(0x7fe1241618d0 0x7fe1240ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.603+0000 7fe12b400640 1 --2- 172.21.15.119:0/1818323261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240ced70 0x7fe12406c250 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.603+0000 7fe12b400640 1 --2- 172.21.15.119:0/1818323261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1241610b0 0x7fe1240b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.603+0000 7fe12b400640 1 -- 172.21.15.119:0/1818323261 >> 172.21.15.119:0/1818323261 conn(0x7fe1240b5570 msgr2=0x7fe1240b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.604+0000 7fe12b400640 1 -- 172.21.15.119:0/1818323261 shutdown_connections 2024-04-25T10:13:10.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.604+0000 7fe12b400640 1 -- 172.21.15.119:0/1818323261 wait complete. 2024-04-25T10:13:10.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.605+0000 7fe12b400640 1 Processor -- start 2024-04-25T10:13:10.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.605+0000 7fe12b400640 1 -- start start 2024-04-25T10:13:10.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.606+0000 7fe12b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240ced70 0x7fe1241f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.606+0000 7fe12b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1241610b0 0x7fe124154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.606+0000 7fe12b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1241618d0 0x7fe1240ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.606+0000 7fe12b400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe124161d10 con 0x7fe1240ced70 2024-04-25T10:13:10.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.606+0000 7fe12b400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1240b6c00 con 0x7fe1241610b0 2024-04-25T10:13:10.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.606+0000 7fe12b400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe124161490 con 0x7fe1241618d0 2024-04-25T10:13:10.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.607+0000 7fe128a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240ced70 0x7fe1241f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.607+0000 7fe128a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240ced70 0x7fe1241f3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35368/0 (socket says 172.21.15.119:35368) 2024-04-25T10:13:10.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.607+0000 7fe128a00640 1 -- 172.21.15.119:0/4185438257 learned_addr learned my addr 172.21.15.119:0/4185438257 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:10.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.607+0000 7fe128a00640 1 -- 172.21.15.119:0/4185438257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1241618d0 msgr2=0x7fe1240ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.607+0000 7fe128a00640 1 --2- 172.21.15.119:0/4185438257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1241618d0 0x7fe1240ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.607+0000 7fe128a00640 1 -- 172.21.15.119:0/4185438257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1241610b0 msgr2=0x7fe124154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.607+0000 7fe128a00640 1 --2- 172.21.15.119:0/4185438257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1241610b0 0x7fe124154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.607+0000 7fe128a00640 1 -- 172.21.15.119:0/4185438257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12414cc20 con 0x7fe1240ced70 2024-04-25T10:13:10.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.607+0000 7fe128a00640 1 --2- 172.21.15.119:0/4185438257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240ced70 0x7fe1241f3980 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe11400d800 tx=0x7fe11400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.608+0000 7fe120c00640 1 -- 172.21.15.119:0/4185438257 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11400b6c0 con 0x7fe1240ced70 2024-04-25T10:13:10.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.608+0000 7fe120c00640 1 -- 172.21.15.119:0/4185438257 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe114019070 con 0x7fe1240ced70 2024-04-25T10:13:10.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.608+0000 7fe120c00640 1 -- 172.21.15.119:0/4185438257 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11400ba80 con 0x7fe1240ced70 2024-04-25T10:13:10.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.608+0000 7fe12b400640 1 -- 172.21.15.119:0/4185438257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe124153c90 con 0x7fe1240ced70 2024-04-25T10:13:10.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.608+0000 7fe12b400640 1 -- 172.21.15.119:0/4185438257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe124154250 con 0x7fe1240ced70 2024-04-25T10:13:10.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.609+0000 7fe12b400640 1 -- 172.21.15.119:0/4185438257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1240750b0 con 0x7fe1240ced70 2024-04-25T10:13:10.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.610+0000 7fe120c00640 1 -- 172.21.15.119:0/4185438257 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe11400bc20 con 0x7fe1240ced70 2024-04-25T10:13:10.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.611+0000 7fe120c00640 1 --2- 172.21.15.119:0/4185438257 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe10006b4d0 0x7fe10006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.611+0000 7fe120c00640 1 -- 172.21.15.119:0/4185438257 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe11408a6b0 con 0x7fe1240ced70 2024-04-25T10:13:10.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.611+0000 7fe123e00640 1 --2- 172.21.15.119:0/4185438257 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe10006b4d0 0x7fe10006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.612+0000 7fe123e00640 1 --2- 172.21.15.119:0/4185438257 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe10006b4d0 0x7fe10006d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009870 tx=0x7fe10c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.615+0000 7fe120c00640 1 -- 172.21.15.119:0/4185438257 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe114059530 con 0x7fe1240ced70 2024-04-25T10:13:10.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.851+0000 7fe12b400640 1 -- 172.21.15.119:0/4185438257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe1240ca3c0 con 0x7fe1240ced70 2024-04-25T10:13:10.857 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:13:10.853+0000 7f8c64600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:13:10.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.857+0000 7fe120c00640 1 -- 172.21.15.119:0/4185438257 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe114058e30 con 0x7fe1240ced70 2024-04-25T10:13:10.862 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:13:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.860+0000 7fe12b400640 1 -- 172.21.15.119:0/4185438257 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe10006b4d0 msgr2=0x7fe10006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.860+0000 7fe12b400640 1 --2- 172.21.15.119:0/4185438257 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe10006b4d0 0x7fe10006d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009870 tx=0x7fe10c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.860+0000 7fe12b400640 1 -- 172.21.15.119:0/4185438257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240ced70 msgr2=0x7fe1241f3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.860+0000 7fe12b400640 1 --2- 172.21.15.119:0/4185438257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240ced70 0x7fe1241f3980 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe11400d800 tx=0x7fe11400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.861+0000 7fe12b400640 1 -- 172.21.15.119:0/4185438257 shutdown_connections 2024-04-25T10:13:10.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.861+0000 7fe12b400640 1 --2- 172.21.15.119:0/4185438257 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe10006b4d0 0x7fe10006d990 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.861+0000 7fe12b400640 1 --2- 172.21.15.119:0/4185438257 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe1241618d0 0x7fe1240ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.861+0000 7fe12b400640 1 --2- 172.21.15.119:0/4185438257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1241610b0 0x7fe124154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.861+0000 7fe12b400640 1 --2- 172.21.15.119:0/4185438257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240ced70 0x7fe1241f3980 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.861+0000 7fe12b400640 1 -- 172.21.15.119:0/4185438257 >> 172.21.15.119:0/4185438257 conn(0x7fe1240b5570 msgr2=0x7fe1240b8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.861+0000 7fe12b400640 1 -- 172.21.15.119:0/4185438257 shutdown_connections 2024-04-25T10:13:10.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:10.862+0000 7fe12b400640 1 -- 172.21.15.119:0/4185438257 wait complete. 2024-04-25T10:13:10.886 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:13:10.893 INFO:tasks.ceph.mds.c.smithi119.stderr: -4182> 2024-04-25T10:11:53.391+0000 7f4b2fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:10.964 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2024-04-25T10:13:11.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.070+0000 7fd487400640 1 Processor -- start 2024-04-25T10:13:11.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.070+0000 7fd487400640 1 -- start start 2024-04-25T10:13:11.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.072+0000 7fd487400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800b59b0 0x7fd4800d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.072+0000 7fd487400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd480161500 con 0x7fd480161120 2024-04-25T10:13:11.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.072+0000 7fd487400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd480161eb0 con 0x7fd4800b5550 2024-04-25T10:13:11.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.072+0000 7fd487400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd480162740 con 0x7fd4800b59b0 2024-04-25T10:13:11.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.072+0000 7fd484a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd480161120 0x7fd4801617a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56240/0 (socket says 172.21.15.119:56240) 2024-04-25T10:13:11.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.072+0000 7fd484a00640 1 -- 172.21.15.119:0/683771887 learned_addr learned my addr 172.21.15.119:0/683771887 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:11.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd485400640 1 --2- 172.21.15.119:0/683771887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800b59b0 0x7fd4800d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 -- 172.21.15.119:0/683771887 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636440053 0 0) 0x7fd480161500 con 0x7fd480161120 2024-04-25T10:13:11.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 -- 172.21.15.119:0/683771887 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd460003520 con 0x7fd480161120 2024-04-25T10:13:11.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 -- 172.21.15.119:0/683771887 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 147068185 0 0) 0x7fd460003520 con 0x7fd480161120 2024-04-25T10:13:11.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 -- 172.21.15.119:0/683771887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800b59b0 msgr2=0x7fd4800d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 --2- 172.21.15.119:0/683771887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800b59b0 0x7fd4800d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 -- 172.21.15.119:0/683771887 >> v1:172.21.15.132:6789/0 conn(0x7fd4800b5550 legacy=0x7fd4800d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 -- 172.21.15.119:0/683771887 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd480162180 con 0x7fd480161120 2024-04-25T10:13:11.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 -- 172.21.15.119:0/683771887 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fd470001690 con 0x7fd480161120 2024-04-25T10:13:11.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 -- 172.21.15.119:0/683771887 >> v1:172.21.15.119:6789/0 conn(0x7fd480161120 legacy=0x7fd4801617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 --2- 172.21.15.119:0/683771887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4600039e0 0x7fd460003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 --2- 172.21.15.119:0/683771887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd460004800 0x7fd460004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 --2- 172.21.15.119:0/683771887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800b59b0 0x7fd460005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 -- 172.21.15.119:0/683771887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd480162740 con 0x7fd4600039e0 2024-04-25T10:13:11.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 -- 172.21.15.119:0/683771887 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46000baf0 con 0x7fd460004800 2024-04-25T10:13:11.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd47f400640 1 -- 172.21.15.119:0/683771887 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46000bcf0 con 0x7fd4800b59b0 2024-04-25T10:13:11.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.073+0000 7fd484a00640 1 --2- 172.21.15.119:0/683771887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4600039e0 0x7fd460003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.074+0000 7fd47fe00640 1 --2- 172.21.15.119:0/683771887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd460004800 0x7fd460004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.074+0000 7fd485400640 1 --2- 172.21.15.119:0/683771887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800b59b0 0x7fd460005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.074+0000 7fd484a00640 1 -- 172.21.15.119:0/683771887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800b59b0 msgr2=0x7fd460005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.074+0000 7fd484a00640 1 --2- 172.21.15.119:0/683771887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4800b59b0 0x7fd460005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.074+0000 7fd484a00640 1 -- 172.21.15.119:0/683771887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd460004800 msgr2=0x7fd460004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.074+0000 7fd484a00640 1 --2- 172.21.15.119:0/683771887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd460004800 0x7fd460004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.074+0000 7fd484a00640 1 -- 172.21.15.119:0/683771887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd460003520 con 0x7fd4600039e0 2024-04-25T10:13:11.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.074+0000 7fd47fe00640 1 --2- 172.21.15.119:0/683771887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd460004800 0x7fd460004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:11.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.075+0000 7fd484a00640 1 --2- 172.21.15.119:0/683771887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4600039e0 0x7fd460003e00 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd470007fd0 tx=0x7fd460005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e012729bb5fe6b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.075+0000 7fd47f400640 1 -- 172.21.15.119:0/683771887 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd470007d50 con 0x7fd4600039e0 2024-04-25T10:13:11.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.075+0000 7fd47f400640 1 -- 172.21.15.119:0/683771887 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4700058d0 con 0x7fd4600039e0 2024-04-25T10:13:11.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.075+0000 7fd47f400640 1 -- 172.21.15.119:0/683771887 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd470005ca0 con 0x7fd4600039e0 2024-04-25T10:13:11.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.076+0000 7fd487400640 1 -- 172.21.15.119:0/683771887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4600039e0 msgr2=0x7fd460003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.076+0000 7fd487400640 1 --2- 172.21.15.119:0/683771887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4600039e0 0x7fd460003e00 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd470007fd0 tx=0x7fd460005830 comp rx=0 tx=0).stop 2024-04-25T10:13:11.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.076+0000 7fd485400640 1 -- 172.21.15.119:0/683771887 reap_dead start 2024-04-25T10:13:11.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.076+0000 7fd487400640 1 -- 172.21.15.119:0/683771887 shutdown_connections 2024-04-25T10:13:11.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.076+0000 7fd487400640 1 -- 172.21.15.119:0/683771887 >> 172.21.15.119:0/683771887 conn(0x7fd4800d0550 msgr2=0x7fd4800d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.076+0000 7fd487400640 1 -- 172.21.15.119:0/683771887 shutdown_connections 2024-04-25T10:13:11.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.076+0000 7fd487400640 1 -- 172.21.15.119:0/683771887 wait complete. 2024-04-25T10:13:11.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.078+0000 7fd487400640 1 Processor -- start 2024-04-25T10:13:11.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.078+0000 7fd487400640 1 -- start start 2024-04-25T10:13:11.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.079+0000 7fd487400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4600039e0 0x7fd48015fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.079+0000 7fd487400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4801528e0 0x7fd4800cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.079+0000 7fd487400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd480152d40 0x7fd4800ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.079+0000 7fd487400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800d2040 con 0x7fd4600039e0 2024-04-25T10:13:11.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.079+0000 7fd487400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800d1d60 con 0x7fd4801528e0 2024-04-25T10:13:11.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.079+0000 7fd487400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800b5e20 con 0x7fd480152d40 2024-04-25T10:13:11.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.079+0000 7fd484a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4600039e0 0x7fd48015fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.079+0000 7fd484a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4600039e0 0x7fd48015fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35044/0 (socket says 172.21.15.119:35044) 2024-04-25T10:13:11.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.079+0000 7fd484a00640 1 -- 172.21.15.119:0/968319542 learned_addr learned my addr 172.21.15.119:0/968319542 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:11.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.080+0000 7fd47fe00640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4801528e0 0x7fd4800cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.080+0000 7fd484a00640 1 -- 172.21.15.119:0/968319542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd480152d40 msgr2=0x7fd4800ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.080+0000 7fd485400640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd480152d40 0x7fd4800ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.080+0000 7fd484a00640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd480152d40 0x7fd4800ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.080+0000 7fd484a00640 1 -- 172.21.15.119:0/968319542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4801528e0 msgr2=0x7fd4800cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.080+0000 7fd484a00640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4801528e0 0x7fd4800cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.080+0000 7fd484a00640 1 -- 172.21.15.119:0/968319542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4800cd7c0 con 0x7fd4600039e0 2024-04-25T10:13:11.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.080+0000 7fd47fe00640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4801528e0 0x7fd4800cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:11.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.080+0000 7fd484a00640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4600039e0 0x7fd48015fee0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd47000c350 tx=0x7fd470001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.080+0000 7fd47cc00640 1 -- 172.21.15.119:0/968319542 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd470007d50 con 0x7fd4600039e0 2024-04-25T10:13:11.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.080+0000 7fd47cc00640 1 -- 172.21.15.119:0/968319542 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd470041070 con 0x7fd4600039e0 2024-04-25T10:13:11.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.080+0000 7fd47cc00640 1 -- 172.21.15.119:0/968319542 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd470005e60 con 0x7fd4600039e0 2024-04-25T10:13:11.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.080+0000 7fd487400640 1 -- 172.21.15.119:0/968319542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48015e1d0 con 0x7fd4600039e0 2024-04-25T10:13:11.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.081+0000 7fd487400640 1 -- 172.21.15.119:0/968319542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48015e790 con 0x7fd4600039e0 2024-04-25T10:13:11.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.082+0000 7fd487400640 1 -- 172.21.15.119:0/968319542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48015dfe0 con 0x7fd4600039e0 2024-04-25T10:13:11.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.083+0000 7fd47cc00640 1 -- 172.21.15.119:0/968319542 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd47003b070 con 0x7fd4600039e0 2024-04-25T10:13:11.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.084+0000 7fd47cc00640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd45c06b4d0 0x7fd45c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.084+0000 7fd47cc00640 1 -- 172.21.15.119:0/968319542 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd4700a9e00 con 0x7fd4600039e0 2024-04-25T10:13:11.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.084+0000 7fd47fe00640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd45c06b4d0 0x7fd45c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.085+0000 7fd47fe00640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd45c06b4d0 0x7fd45c06d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd468007fd0 tx=0x7fd468009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.088+0000 7fd47cc00640 1 -- 172.21.15.119:0/968319542 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd470078af0 con 0x7fd4600039e0 2024-04-25T10:13:11.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.333+0000 7fd487400640 1 -- 172.21.15.119:0/968319542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd4800750b0 con 0x7fd4600039e0 2024-04-25T10:13:11.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.860+0000 7fd47cc00640 1 -- 172.21.15.119:0/968319542 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd4700783f0 con 0x7fd4600039e0 2024-04-25T10:13:11.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.864+0000 7fd487400640 1 -- 172.21.15.119:0/968319542 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd45c06b4d0 msgr2=0x7fd45c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.864+0000 7fd487400640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd45c06b4d0 0x7fd45c06d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd468007fd0 tx=0x7fd468009040 comp rx=0 tx=0).stop 2024-04-25T10:13:11.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.864+0000 7fd487400640 1 -- 172.21.15.119:0/968319542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4600039e0 msgr2=0x7fd48015fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.864+0000 7fd487400640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4600039e0 0x7fd48015fee0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd47000c350 tx=0x7fd470001a60 comp rx=0 tx=0).stop 2024-04-25T10:13:11.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.864+0000 7fd487400640 1 -- 172.21.15.119:0/968319542 shutdown_connections 2024-04-25T10:13:11.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.864+0000 7fd487400640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd45c06b4d0 0x7fd45c06d990 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.864+0000 7fd487400640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd480152d40 0x7fd4800ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.864+0000 7fd487400640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4801528e0 0x7fd4800cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.864+0000 7fd487400640 1 --2- 172.21.15.119:0/968319542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4600039e0 0x7fd48015fee0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.864+0000 7fd487400640 1 -- 172.21.15.119:0/968319542 >> 172.21.15.119:0/968319542 conn(0x7fd4800d0550 msgr2=0x7fd4800b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.865+0000 7fd487400640 1 -- 172.21.15.119:0/968319542 shutdown_connections 2024-04-25T10:13:11.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:11.865+0000 7fd487400640 1 -- 172.21.15.119:0/968319542 wait complete. 2024-04-25T10:13:11.891 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:13:12.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.135+0000 7fabb8600640 1 Processor -- start 2024-04-25T10:13:12.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.135+0000 7fabb8600640 1 -- start start 2024-04-25T10:13:12.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.136+0000 7fabb8600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabb015dbe0 0x7fabb0071c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.137+0000 7fabb8600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb00ace70 con 0x7fabb00aedc0 2024-04-25T10:13:12.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.137+0000 7fabb8600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb0072150 con 0x7fabb00cfad0 2024-04-25T10:13:12.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.137+0000 7fabb8600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb00ad110 con 0x7fabb015dbe0 2024-04-25T10:13:12.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.137+0000 7fabb5200640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fabb00aedc0 0x7fabb00cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56246/0 (socket says 172.21.15.119:56246) 2024-04-25T10:13:12.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.137+0000 7fabb5200640 1 -- 172.21.15.119:0/7173406 learned_addr learned my addr 172.21.15.119:0/7173406 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:12.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.137+0000 7fabafe00640 1 -- 172.21.15.119:0/7173406 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508610550 0 0) 0x7fabb00ace70 con 0x7fabb00aedc0 2024-04-25T10:13:12.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.137+0000 7fabafe00640 1 -- 172.21.15.119:0/7173406 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab98003520 con 0x7fabb00aedc0 2024-04-25T10:13:12.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.137+0000 7fabb5c00640 1 --2- 172.21.15.119:0/7173406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabb015dbe0 0x7fabb0071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.138+0000 7fabafe00640 1 -- 172.21.15.119:0/7173406 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 227893048 0 0) 0x7fab98003520 con 0x7fabb00aedc0 2024-04-25T10:13:12.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.138+0000 7fabafe00640 1 -- 172.21.15.119:0/7173406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabb015dbe0 msgr2=0x7fabb0071c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:12.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.138+0000 7fabafe00640 1 --2- 172.21.15.119:0/7173406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabb015dbe0 0x7fabb0071c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.138+0000 7fabafe00640 1 -- 172.21.15.119:0/7173406 >> v1:172.21.15.132:6789/0 conn(0x7fabb00cfad0 legacy=0x7fabb006d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.138+0000 7fabafe00640 1 -- 172.21.15.119:0/7173406 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb00ad3f0 con 0x7fabb00aedc0 2024-04-25T10:13:12.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.138+0000 7fabafe00640 1 -- 172.21.15.119:0/7173406 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fab9c001690 con 0x7fabb00aedc0 2024-04-25T10:13:12.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.138+0000 7fabafe00640 1 -- 172.21.15.119:0/7173406 >> v1:172.21.15.119:6789/0 conn(0x7fabb00aedc0 legacy=0x7fabb00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.138+0000 7fabafe00640 1 --2- 172.21.15.119:0/7173406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab980039e0 0x7fab98003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.138+0000 7fabafe00640 1 --2- 172.21.15.119:0/7173406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab98004800 0x7fab98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.138+0000 7fabafe00640 1 --2- 172.21.15.119:0/7173406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabb015dbe0 0x7fab98005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.138+0000 7fabafe00640 1 -- 172.21.15.119:0/7173406 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb00ad110 con 0x7fab98004800 2024-04-25T10:13:12.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.138+0000 7fabafe00640 1 -- 172.21.15.119:0/7173406 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9800bb50 con 0x7fab980039e0 2024-04-25T10:13:12.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.138+0000 7fabafe00640 1 -- 172.21.15.119:0/7173406 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9800bd50 con 0x7fabb015dbe0 2024-04-25T10:13:12.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.138+0000 7fabb5200640 1 --2- 172.21.15.119:0/7173406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab98004800 0x7fab98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.139+0000 7fabb5c00640 1 --2- 172.21.15.119:0/7173406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab980039e0 0x7fab98003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.139+0000 7fabb5200640 1 -- 172.21.15.119:0/7173406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabb015dbe0 msgr2=0x7fab98005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:12.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.139+0000 7fabb6600640 1 --2- 172.21.15.119:0/7173406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabb015dbe0 0x7fab98005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.139+0000 7fabb5200640 1 --2- 172.21.15.119:0/7173406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabb015dbe0 0x7fab98005250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.139+0000 7fabb5200640 1 -- 172.21.15.119:0/7173406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab980039e0 msgr2=0x7fab98003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:12.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.139+0000 7fabb5200640 1 --2- 172.21.15.119:0/7173406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab980039e0 0x7fab98003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.139+0000 7fabb5200640 1 -- 172.21.15.119:0/7173406 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab98003520 con 0x7fab98004800 2024-04-25T10:13:12.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.139+0000 7fabb5c00640 1 --2- 172.21.15.119:0/7173406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab980039e0 0x7fab98003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:12.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.140+0000 7fabb5200640 1 --2- 172.21.15.119:0/7173406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab98004800 0x7fab98004c80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fabb00af200 tx=0x7fab98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85da01693ff8a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.140+0000 7fabafe00640 1 -- 172.21.15.119:0/7173406 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c007c40 con 0x7fab98004800 2024-04-25T10:13:12.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.140+0000 7fabafe00640 1 -- 172.21.15.119:0/7173406 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c005ad0 con 0x7fab98004800 2024-04-25T10:13:12.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.140+0000 7fabafe00640 1 -- 172.21.15.119:0/7173406 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c007070 con 0x7fab98004800 2024-04-25T10:13:12.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.141+0000 7fabb8600640 1 -- 172.21.15.119:0/7173406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab98004800 msgr2=0x7fab98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.141+0000 7fabb8600640 1 --2- 172.21.15.119:0/7173406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab98004800 0x7fab98004c80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fabb00af200 tx=0x7fab98005830 comp rx=0 tx=0).stop 2024-04-25T10:13:12.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.141+0000 7fabb6600640 1 -- 172.21.15.119:0/7173406 reap_dead start 2024-04-25T10:13:12.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.141+0000 7fabb8600640 1 -- 172.21.15.119:0/7173406 shutdown_connections 2024-04-25T10:13:12.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.141+0000 7fabb8600640 1 -- 172.21.15.119:0/7173406 >> 172.21.15.119:0/7173406 conn(0x7fabb006aba0 msgr2=0x7fabb006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.141+0000 7fabb8600640 1 -- 172.21.15.119:0/7173406 shutdown_connections 2024-04-25T10:13:12.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.141+0000 7fabb8600640 1 -- 172.21.15.119:0/7173406 wait complete. 2024-04-25T10:13:12.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.143+0000 7fabb8600640 1 Processor -- start 2024-04-25T10:13:12.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.143+0000 7fabb8600640 1 -- start start 2024-04-25T10:13:12.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb8600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98004800 0x7fabb00af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb8600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabb014f170 0x7fabb0152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb8600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb00b08f0 0x7fabb00b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb8600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb00adc90 con 0x7fabb00b08f0 2024-04-25T10:13:12.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb8600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb00ad9b0 con 0x7fab98004800 2024-04-25T10:13:12.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb8600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb015dfc0 con 0x7fabb014f170 2024-04-25T10:13:12.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb6600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb00b08f0 0x7fabb00b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb6600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb00b08f0 0x7fabb00b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35058/0 (socket says 172.21.15.119:35058) 2024-04-25T10:13:12.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb6600640 1 -- 172.21.15.119:0/2567987937 learned_addr learned my addr 172.21.15.119:0/2567987937 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:12.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb5c00640 1 --2- 172.21.15.119:0/2567987937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98004800 0x7fabb00af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb6600640 1 -- 172.21.15.119:0/2567987937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabb014f170 msgr2=0x7fabb0152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb6600640 1 --2- 172.21.15.119:0/2567987937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabb014f170 0x7fabb0152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb6600640 1 -- 172.21.15.119:0/2567987937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98004800 msgr2=0x7fabb00af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb6600640 1 --2- 172.21.15.119:0/2567987937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98004800 0x7fabb00af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb6600640 1 -- 172.21.15.119:0/2567987937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb00af850 con 0x7fabb00b08f0 2024-04-25T10:13:12.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.144+0000 7fabb5c00640 1 --2- 172.21.15.119:0/2567987937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98004800 0x7fabb00af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:12.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.145+0000 7fabb6600640 1 --2- 172.21.15.119:0/2567987937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb00b08f0 0x7fabb00b0d70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fabb00b4c30 tx=0x7faba800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.145+0000 7fabad600640 1 -- 172.21.15.119:0/2567987937 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8018070 con 0x7fabb00b08f0 2024-04-25T10:13:12.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.145+0000 7fabad600640 1 -- 172.21.15.119:0/2567987937 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba8003b70 con 0x7fabb00b08f0 2024-04-25T10:13:12.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.145+0000 7fabad600640 1 -- 172.21.15.119:0/2567987937 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8005070 con 0x7fabb00b08f0 2024-04-25T10:13:12.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.145+0000 7fabb8600640 1 -- 172.21.15.119:0/2567987937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb00afb30 con 0x7fabb00b08f0 2024-04-25T10:13:12.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.145+0000 7fabb8600640 1 -- 172.21.15.119:0/2567987937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb00b00f0 con 0x7fabb00b08f0 2024-04-25T10:13:12.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.146+0000 7fabb8600640 1 -- 172.21.15.119:0/2567987937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb00750b0 con 0x7fabb00b08f0 2024-04-25T10:13:12.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.147+0000 7fabad600640 1 -- 172.21.15.119:0/2567987937 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faba801b070 con 0x7fabb00b08f0 2024-04-25T10:13:12.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.148+0000 7fabad600640 1 --2- 172.21.15.119:0/2567987937 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fab9006b4d0 0x7fab9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.148+0000 7fabad600640 1 -- 172.21.15.119:0/2567987937 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5939+0+0 (secure 0 0 0) 0x7faba8089a90 con 0x7fabb00b08f0 2024-04-25T10:13:12.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.148+0000 7fabb5c00640 1 --2- 172.21.15.119:0/2567987937 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fab9006b4d0 0x7fab9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.149+0000 7fabb5c00640 1 --2- 172.21.15.119:0/2567987937 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fab9006b4d0 0x7fab9006d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faba4006fd0 tx=0x7faba4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.152+0000 7fabad600640 1 -- 172.21.15.119:0/2567987937 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faba8058780 con 0x7fabb00b08f0 2024-04-25T10:13:12.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.345+0000 7fabb8600640 1 -- 172.21.15.119:0/2567987937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fabb01602d0 con 0x7fabb00b08f0 2024-04-25T10:13:12.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.864+0000 7fabad600640 1 -- 172.21.15.119:0/2567987937 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) v1 ==== 181+0+0 (secure 0 0 0) 0x7faba8058080 con 0x7fabb00b08f0 2024-04-25T10:13:12.869 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:13:12.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.868+0000 7fabb8600640 1 -- 172.21.15.119:0/2567987937 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fab9006b4d0 msgr2=0x7fab9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.868+0000 7fabb8600640 1 --2- 172.21.15.119:0/2567987937 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fab9006b4d0 0x7fab9006d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faba4006fd0 tx=0x7faba4008040 comp rx=0 tx=0).stop 2024-04-25T10:13:12.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.868+0000 7fabb8600640 1 -- 172.21.15.119:0/2567987937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb00b08f0 msgr2=0x7fabb00b0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.868+0000 7fabb8600640 1 --2- 172.21.15.119:0/2567987937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb00b08f0 0x7fabb00b0d70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fabb00b4c30 tx=0x7faba800dc00 comp rx=0 tx=0).stop 2024-04-25T10:13:12.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.869+0000 7fabb8600640 1 -- 172.21.15.119:0/2567987937 shutdown_connections 2024-04-25T10:13:12.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.870+0000 7fabb8600640 1 --2- 172.21.15.119:0/2567987937 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fab9006b4d0 0x7fab9006d990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.870+0000 7fabb8600640 1 --2- 172.21.15.119:0/2567987937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb00b08f0 0x7fabb00b0d70 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.870+0000 7fabb8600640 1 --2- 172.21.15.119:0/2567987937 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fabb014f170 0x7fabb0152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.870+0000 7fabb8600640 1 --2- 172.21.15.119:0/2567987937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab98004800 0x7fabb00af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.870+0000 7fabb8600640 1 -- 172.21.15.119:0/2567987937 >> 172.21.15.119:0/2567987937 conn(0x7fabb006aba0 msgr2=0x7fabb006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.873+0000 7fabb8600640 1 -- 172.21.15.119:0/2567987937 shutdown_connections 2024-04-25T10:13:12.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:12.873+0000 7fabb8600640 1 -- 172.21.15.119:0/2567987937 wait complete. 2024-04-25T10:13:12.901 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:13:13.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.089+0000 7f5626e00640 1 Processor -- start 2024-04-25T10:13:13.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.090+0000 7f5626e00640 1 -- start start 2024-04-25T10:13:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.091+0000 7f5626e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5620071200 0x7f5620071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.091+0000 7f5626e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56200cee90 0x7f56200cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.091+0000 7f5626e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56200cc9a0 con 0x7f5620145260 2024-04-25T10:13:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.091+0000 7f5626e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5620071ba0 con 0x7f5620071200 2024-04-25T10:13:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.091+0000 7f5626e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56200cea00 con 0x7f56200cee90 2024-04-25T10:13:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.091+0000 7f561f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5620071200 0x7f5620071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.092+0000 7f561f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5620071200 0x7f5620071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35068/0 (socket says 172.21.15.119:35068) 2024-04-25T10:13:13.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.092+0000 7f561f400640 1 -- 172.21.15.119:0/217569232 learned_addr learned my addr 172.21.15.119:0/217569232 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:13.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.092+0000 7f561f400640 1 -- 172.21.15.119:0/217569232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56200cee90 msgr2=0x7f56200cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.092+0000 7f561f400640 1 --2- 172.21.15.119:0/217569232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56200cee90 0x7f56200cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.092+0000 7f561f400640 1 -- 172.21.15.119:0/217569232 >> v1:172.21.15.132:6789/0 conn(0x7f5620145260 legacy=0x7f56200cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.092+0000 7f561f400640 1 -- 172.21.15.119:0/217569232 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56200ccc80 con 0x7f5620071200 2024-04-25T10:13:13.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.093+0000 7f561f400640 1 --2- 172.21.15.119:0/217569232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5620071200 0x7f5620071660 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5608009e50 tx=0x7f560802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf62abae109e1b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.093+0000 7f561ea00640 1 -- 172.21.15.119:0/217569232 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608036070 con 0x7f5620071200 2024-04-25T10:13:13.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.093+0000 7f561ea00640 1 -- 172.21.15.119:0/217569232 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5608031600 con 0x7f5620071200 2024-04-25T10:13:13.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.093+0000 7f561ea00640 1 -- 172.21.15.119:0/217569232 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56080319d0 con 0x7f5620071200 2024-04-25T10:13:13.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.094+0000 7f5626e00640 1 -- 172.21.15.119:0/217569232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5620071200 msgr2=0x7f5620071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.094+0000 7f5626e00640 1 --2- 172.21.15.119:0/217569232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5620071200 0x7f5620071660 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5608009e50 tx=0x7f560802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.094+0000 7f5626e00640 1 -- 172.21.15.119:0/217569232 shutdown_connections 2024-04-25T10:13:13.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.094+0000 7f5626e00640 1 --2- 172.21.15.119:0/217569232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56200cee90 0x7f56200cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.094+0000 7f5626e00640 1 --2- 172.21.15.119:0/217569232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5620071200 0x7f5620071660 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.094+0000 7f5626e00640 1 -- 172.21.15.119:0/217569232 >> 172.21.15.119:0/217569232 conn(0x7f562006aba0 msgr2=0x7f562006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.094+0000 7f5626e00640 1 -- 172.21.15.119:0/217569232 shutdown_connections 2024-04-25T10:13:13.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.094+0000 7f5626e00640 1 -- 172.21.15.119:0/217569232 wait complete. 2024-04-25T10:13:13.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.096+0000 7f5626e00640 1 Processor -- start 2024-04-25T10:13:13.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.096+0000 7f5626e00640 1 -- start start 2024-04-25T10:13:13.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.097+0000 7f5626e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5620071200 0x7f562013bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.097+0000 7f5626e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56200cee90 0x7f56201f7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.097+0000 7f5626e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5620145260 0x7f562013a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.097+0000 7f5626e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200cc660 con 0x7f56200cee90 2024-04-25T10:13:13.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.097+0000 7f5626e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200cd240 con 0x7f5620145260 2024-04-25T10:13:13.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.097+0000 7f5626e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5620145640 con 0x7f5620071200 2024-04-25T10:13:13.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.097+0000 7f561f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56200cee90 0x7f56201f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.097+0000 7f561f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56200cee90 0x7f56201f7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35070/0 (socket says 172.21.15.119:35070) 2024-04-25T10:13:13.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.097+0000 7f561f400640 1 -- 172.21.15.119:0/191196294 learned_addr learned my addr 172.21.15.119:0/191196294 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:13.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.098+0000 7f561fe00640 1 --2- 172.21.15.119:0/191196294 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5620071200 0x7f562013bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.098+0000 7f561f400640 1 -- 172.21.15.119:0/191196294 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5620071200 msgr2=0x7f562013bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.098+0000 7f561f400640 1 --2- 172.21.15.119:0/191196294 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5620071200 0x7f562013bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.098+0000 7f5624e00640 1 --2- 172.21.15.119:0/191196294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5620145260 0x7f562013a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.098+0000 7f561f400640 1 -- 172.21.15.119:0/191196294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5620145260 msgr2=0x7f562013a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.098+0000 7f561f400640 1 --2- 172.21.15.119:0/191196294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5620145260 0x7f562013a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.098+0000 7f561f400640 1 -- 172.21.15.119:0/191196294 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562013ad50 con 0x7f56200cee90 2024-04-25T10:13:13.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.098+0000 7f561f400640 1 --2- 172.21.15.119:0/191196294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56200cee90 0x7f56201f7770 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5608034040 tx=0x7f5608004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.098+0000 7f5617e00640 1 -- 172.21.15.119:0/191196294 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608041020 con 0x7f56200cee90 2024-04-25T10:13:13.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.098+0000 7f5617e00640 1 -- 172.21.15.119:0/191196294 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5608007800 con 0x7f56200cee90 2024-04-25T10:13:13.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.098+0000 7f5626e00640 1 -- 172.21.15.119:0/191196294 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562013b030 con 0x7f56200cee90 2024-04-25T10:13:13.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.099+0000 7f5617e00640 1 -- 172.21.15.119:0/191196294 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608036040 con 0x7f56200cee90 2024-04-25T10:13:13.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.099+0000 7f5626e00640 1 -- 172.21.15.119:0/191196294 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562013b5f0 con 0x7f56200cee90 2024-04-25T10:13:13.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.100+0000 7f5626e00640 1 -- 172.21.15.119:0/191196294 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56200750b0 con 0x7f56200cee90 2024-04-25T10:13:13.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.101+0000 7f5617e00640 1 -- 172.21.15.119:0/191196294 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5608044020 con 0x7f56200cee90 2024-04-25T10:13:13.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.102+0000 7f5617e00640 1 --2- 172.21.15.119:0/191196294 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55fc06b4d0 0x7f55fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.102+0000 7f5617e00640 1 -- 172.21.15.119:0/191196294 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f56080ab1b0 con 0x7f56200cee90 2024-04-25T10:13:13.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.102+0000 7f561fe00640 1 --2- 172.21.15.119:0/191196294 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55fc06b4d0 0x7f55fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.103+0000 7f561fe00640 1 --2- 172.21.15.119:0/191196294 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55fc06b4d0 0x7f55fc06d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5610003d90 tx=0x7f5610009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.107+0000 7f5617e00640 1 -- 172.21.15.119:0/191196294 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f560804b030 con 0x7f56200cee90 2024-04-25T10:13:13.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.291+0000 7f5626e00640 1 -- 172.21.15.119:0/191196294 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f56200b0e20 con 0x7f56200cee90 2024-04-25T10:13:13.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.879+0000 7f5617e00640 1 -- 172.21.15.119:0/191196294 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5608079fd0 con 0x7f56200cee90 2024-04-25T10:13:13.885 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.882+0000 7f5626e00640 1 -- 172.21.15.119:0/191196294 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55fc06b4d0 msgr2=0x7f55fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.882+0000 7f5626e00640 1 --2- 172.21.15.119:0/191196294 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55fc06b4d0 0x7f55fc06d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5610003d90 tx=0x7f5610009040 comp rx=0 tx=0).stop 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.882+0000 7f5626e00640 1 -- 172.21.15.119:0/191196294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56200cee90 msgr2=0x7f56201f7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.882+0000 7f5626e00640 1 --2- 172.21.15.119:0/191196294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56200cee90 0x7f56201f7770 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5608034040 tx=0x7f5608004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.883+0000 7f5626e00640 1 -- 172.21.15.119:0/191196294 shutdown_connections 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.883+0000 7f5626e00640 1 --2- 172.21.15.119:0/191196294 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55fc06b4d0 0x7f55fc06d990 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.883+0000 7f5626e00640 1 --2- 172.21.15.119:0/191196294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5620145260 0x7f562013a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.883+0000 7f5626e00640 1 --2- 172.21.15.119:0/191196294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56200cee90 0x7f56201f7770 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.883+0000 7f5626e00640 1 --2- 172.21.15.119:0/191196294 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5620071200 0x7f562013bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.883+0000 7f5626e00640 1 -- 172.21.15.119:0/191196294 >> 172.21.15.119:0/191196294 conn(0x7f562006aba0 msgr2=0x7f562015be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.883+0000 7f5626e00640 1 -- 172.21.15.119:0/191196294 shutdown_connections 2024-04-25T10:13:13.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:13.883+0000 7f5626e00640 1 -- 172.21.15.119:0/191196294 wait complete. 2024-04-25T10:13:13.912 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:13:14.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.096+0000 7f1b5c800640 1 Processor -- start 2024-04-25T10:13:14.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.096+0000 7f1b5c800640 1 -- start start 2024-04-25T10:13:14.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.097+0000 7f1b5c800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1b54144e70 0x7f1b5406d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.097+0000 7f1b5c800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b5406d7a0 0x7f1b540ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.097+0000 7f1b5c800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b540cef10 0x7f1b54071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.097+0000 7f1b5c800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5415bfe0 con 0x7f1b5406d7a0 2024-04-25T10:13:14.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.097+0000 7f1b5c800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5406dcb0 con 0x7f1b54144e70 2024-04-25T10:13:14.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.097+0000 7f1b5c800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5406df90 con 0x7f1b540cef10 2024-04-25T10:13:14.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.098+0000 7f1b59400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b5406d7a0 0x7f1b540ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.098+0000 7f1b59400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b5406d7a0 0x7f1b540ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35082/0 (socket says 172.21.15.119:35082) 2024-04-25T10:13:14.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.098+0000 7f1b59400640 1 -- 172.21.15.119:0/1929104613 learned_addr learned my addr 172.21.15.119:0/1929104613 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:14.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.098+0000 7f1b59400640 1 -- 172.21.15.119:0/1929104613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b540cef10 msgr2=0x7f1b54071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:14.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.098+0000 7f1b5a800640 1 --2- 172.21.15.119:0/1929104613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b540cef10 0x7f1b54071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.098+0000 7f1b59e00640 1 --2- 172.21.15.119:0/1929104613 >> v2:172.21.15.132:3300/0 conn(0x7f1b54144e70 0x7f1b5406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.098+0000 7f1b59400640 1 --2- 172.21.15.119:0/1929104613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b540cef10 0x7f1b54071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.098+0000 7f1b59400640 1 -- 172.21.15.119:0/1929104613 >> v2:172.21.15.132:3300/0 conn(0x7f1b54144e70 msgr2=0x7f1b5406d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:14.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.098+0000 7f1b59400640 1 --2- 172.21.15.119:0/1929104613 >> v2:172.21.15.132:3300/0 conn(0x7f1b54144e70 0x7f1b5406d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.098+0000 7f1b59400640 1 -- 172.21.15.119:0/1929104613 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5406e270 con 0x7f1b5406d7a0 2024-04-25T10:13:14.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.099+0000 7f1b59400640 1 --2- 172.21.15.119:0/1929104613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b5406d7a0 0x7f1b540ce9d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f1b540b4a20 tx=0x7f1b4402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8731ec77a44a0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.099+0000 7f1b58a00640 1 -- 172.21.15.119:0/1929104613 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44037070 con 0x7f1b5406d7a0 2024-04-25T10:13:14.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.099+0000 7f1b58a00640 1 -- 172.21.15.119:0/1929104613 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b44032440 con 0x7f1b5406d7a0 2024-04-25T10:13:14.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.100+0000 7f1b58a00640 1 -- 172.21.15.119:0/1929104613 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44032810 con 0x7f1b5406d7a0 2024-04-25T10:13:14.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.100+0000 7f1b5c800640 1 -- 172.21.15.119:0/1929104613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b5406d7a0 msgr2=0x7f1b540ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.100+0000 7f1b5c800640 1 --2- 172.21.15.119:0/1929104613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b5406d7a0 0x7f1b540ce9d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f1b540b4a20 tx=0x7f1b4402a140 comp rx=0 tx=0).stop 2024-04-25T10:13:14.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.100+0000 7f1b5c800640 1 -- 172.21.15.119:0/1929104613 shutdown_connections 2024-04-25T10:13:14.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.100+0000 7f1b5c800640 1 --2- 172.21.15.119:0/1929104613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b540cef10 0x7f1b54071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.100+0000 7f1b5c800640 1 --2- 172.21.15.119:0/1929104613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b5406d7a0 0x7f1b540ce9d0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.100+0000 7f1b5c800640 1 --2- 172.21.15.119:0/1929104613 >> v2:172.21.15.132:3300/0 conn(0x7f1b54144e70 0x7f1b5406d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.100+0000 7f1b5c800640 1 -- 172.21.15.119:0/1929104613 >> 172.21.15.119:0/1929104613 conn(0x7f1b5406aba0 msgr2=0x7f1b5406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:14.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.100+0000 7f1b5c800640 1 -- 172.21.15.119:0/1929104613 shutdown_connections 2024-04-25T10:13:14.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.100+0000 7f1b5c800640 1 -- 172.21.15.119:0/1929104613 wait complete. 2024-04-25T10:13:14.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.102+0000 7f1b5c800640 1 Processor -- start 2024-04-25T10:13:14.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.102+0000 7f1b5c800640 1 -- start start 2024-04-25T10:13:14.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.103+0000 7f1b5c800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b5406d7a0 0x7f1b54154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.103+0000 7f1b5c800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b540cef10 0x7f1b5413fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.103+0000 7f1b5c800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b54144e70 0x7f1b5415e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.103+0000 7f1b5c800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b541454b0 con 0x7f1b5406d7a0 2024-04-25T10:13:14.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.103+0000 7f1b5c800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b540cf370 con 0x7f1b54144e70 2024-04-25T10:13:14.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.103+0000 7f1b5c800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b54145630 con 0x7f1b540cef10 2024-04-25T10:13:14.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.103+0000 7f1b59e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b5406d7a0 0x7f1b54154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.103+0000 7f1b59e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b5406d7a0 0x7f1b54154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35096/0 (socket says 172.21.15.119:35096) 2024-04-25T10:13:14.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.103+0000 7f1b59e00640 1 -- 172.21.15.119:0/2930354463 learned_addr learned my addr 172.21.15.119:0/2930354463 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:14.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.104+0000 7f1b59e00640 1 -- 172.21.15.119:0/2930354463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b540cef10 msgr2=0x7f1b5413fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.104+0000 7f1b59e00640 1 --2- 172.21.15.119:0/2930354463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b540cef10 0x7f1b5413fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.104+0000 7f1b59e00640 1 -- 172.21.15.119:0/2930354463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b54144e70 msgr2=0x7f1b5415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.104+0000 7f1b59e00640 1 --2- 172.21.15.119:0/2930354463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b54144e70 0x7f1b5415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.104+0000 7f1b59e00640 1 -- 172.21.15.119:0/2930354463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5413cef0 con 0x7f1b5406d7a0 2024-04-25T10:13:14.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.104+0000 7f1b59e00640 1 --2- 172.21.15.119:0/2930354463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b5406d7a0 0x7f1b54154140 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800c7f0 tx=0x7f1b4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.104+0000 7f1b4e000640 1 -- 172.21.15.119:0/2930354463 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48015070 con 0x7f1b5406d7a0 2024-04-25T10:13:14.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.104+0000 7f1b4e000640 1 -- 172.21.15.119:0/2930354463 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b48004220 con 0x7f1b5406d7a0 2024-04-25T10:13:14.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.104+0000 7f1b4e000640 1 -- 172.21.15.119:0/2930354463 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48007620 con 0x7f1b5406d7a0 2024-04-25T10:13:14.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.104+0000 7f1b5c800640 1 -- 172.21.15.119:0/2930354463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5413d1d0 con 0x7f1b5406d7a0 2024-04-25T10:13:14.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.104+0000 7f1b5c800640 1 -- 172.21.15.119:0/2930354463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5413d790 con 0x7f1b5406d7a0 2024-04-25T10:13:14.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.106+0000 7f1b5c800640 1 -- 172.21.15.119:0/2930354463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b540750b0 con 0x7f1b5406d7a0 2024-04-25T10:13:14.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.106+0000 7f1b4e000640 1 -- 172.21.15.119:0/2930354463 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b480079d0 con 0x7f1b5406d7a0 2024-04-25T10:13:14.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.107+0000 7f1b4e000640 1 --2- 172.21.15.119:0/2930354463 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b3806b4d0 0x7f1b3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.107+0000 7f1b4e000640 1 -- 172.21.15.119:0/2930354463 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f1b4808a290 con 0x7f1b5406d7a0 2024-04-25T10:13:14.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.108+0000 7f1b59400640 1 --2- 172.21.15.119:0/2930354463 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b3806b4d0 0x7f1b3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.108+0000 7f1b59400640 1 --2- 172.21.15.119:0/2930354463 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b3806b4d0 0x7f1b3806d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1b44034000 tx=0x7f1b4402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.112+0000 7f1b4e000640 1 -- 172.21.15.119:0/2930354463 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b48059440 con 0x7f1b5406d7a0 2024-04-25T10:13:14.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.291+0000 7f1b5c800640 1 -- 172.21.15.119:0/2930354463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1b5406c220 con 0x7f1b5406d7a0 2024-04-25T10:13:14.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.297+0000 7f1b4e000640 1 -- 172.21.15.119:0/2930354463 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1b48058d40 con 0x7f1b5406d7a0 2024-04-25T10:13:14.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.300+0000 7f1b5c800640 1 -- 172.21.15.119:0/2930354463 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b3806b4d0 msgr2=0x7f1b3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.300+0000 7f1b5c800640 1 --2- 172.21.15.119:0/2930354463 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b3806b4d0 0x7f1b3806d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1b44034000 tx=0x7f1b4402c000 comp rx=0 tx=0).stop 2024-04-25T10:13:14.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.300+0000 7f1b5c800640 1 -- 172.21.15.119:0/2930354463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b5406d7a0 msgr2=0x7f1b54154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.300+0000 7f1b5c800640 1 --2- 172.21.15.119:0/2930354463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b5406d7a0 0x7f1b54154140 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800c7f0 tx=0x7f1b4800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.300+0000 7f1b5c800640 1 -- 172.21.15.119:0/2930354463 shutdown_connections 2024-04-25T10:13:14.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.300+0000 7f1b5c800640 1 --2- 172.21.15.119:0/2930354463 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b3806b4d0 0x7f1b3806d990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.300+0000 7f1b5c800640 1 --2- 172.21.15.119:0/2930354463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b54144e70 0x7f1b5415e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.300+0000 7f1b5c800640 1 --2- 172.21.15.119:0/2930354463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b540cef10 0x7f1b5413fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.300+0000 7f1b5c800640 1 --2- 172.21.15.119:0/2930354463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b5406d7a0 0x7f1b54154140 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.300+0000 7f1b5c800640 1 -- 172.21.15.119:0/2930354463 >> 172.21.15.119:0/2930354463 conn(0x7f1b5406aba0 msgr2=0x7f1b540ae1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:14.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.300+0000 7f1b5c800640 1 -- 172.21.15.119:0/2930354463 shutdown_connections 2024-04-25T10:13:14.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.300+0000 7f1b5c800640 1 -- 172.21.15.119:0/2930354463 wait complete. 2024-04-25T10:13:14.323 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T10:13:14.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.511+0000 7fdda7e00640 1 Processor -- start 2024-04-25T10:13:14.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.512+0000 7fdda7e00640 1 -- start start 2024-04-25T10:13:14.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.513+0000 7fdda7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdda00d35b0 0x7fdda00b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.513+0000 7fdda7e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdda00b5990 0x7fdda00b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.513+0000 7fdda7e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda015e0c0 con 0x7fdda015dce0 2024-04-25T10:13:14.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.513+0000 7fdda7e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda00b5de0 con 0x7fdda00b5990 2024-04-25T10:13:14.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.513+0000 7fdda7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda00d2510 con 0x7fdda00d35b0 2024-04-25T10:13:14.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.514+0000 7fdda5400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdda015dce0 0x7fdda00d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56262/0 (socket says 172.21.15.119:56262) 2024-04-25T10:13:14.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.514+0000 7fdda5400640 1 -- 172.21.15.119:0/3028876112 learned_addr learned my addr 172.21.15.119:0/3028876112 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:14.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.514+0000 7fdda5e00640 1 --2- 172.21.15.119:0/3028876112 >> v2:172.21.15.132:3300/0 conn(0x7fdda00b5990 0x7fdda00b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.514+0000 7fdda4a00640 1 --2- 172.21.15.119:0/3028876112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdda00d35b0 0x7fdda00b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.514+0000 7fdd9fe00640 1 -- 172.21.15.119:0/3028876112 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462088477 0 0) 0x7fdda015e0c0 con 0x7fdda015dce0 2024-04-25T10:13:14.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.514+0000 7fdd9fe00640 1 -- 172.21.15.119:0/3028876112 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd84003520 con 0x7fdda015dce0 2024-04-25T10:13:14.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdd9fe00640 1 -- 172.21.15.119:0/3028876112 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4208858636 0 0) 0x7fdd84003520 con 0x7fdda015dce0 2024-04-25T10:13:14.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdd9fe00640 1 -- 172.21.15.119:0/3028876112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdda00d35b0 msgr2=0x7fdda00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:14.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdd9fe00640 1 --2- 172.21.15.119:0/3028876112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdda00d35b0 0x7fdda00b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdd9fe00640 1 -- 172.21.15.119:0/3028876112 >> v2:172.21.15.132:3300/0 conn(0x7fdda00b5990 msgr2=0x7fdda00b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:14.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdd9fe00640 1 --2- 172.21.15.119:0/3028876112 >> v2:172.21.15.132:3300/0 conn(0x7fdda00b5990 0x7fdda00b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdd9fe00640 1 -- 172.21.15.119:0/3028876112 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda00d27f0 con 0x7fdda015dce0 2024-04-25T10:13:14.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdd9fe00640 1 -- 172.21.15.119:0/3028876112 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fdd900019f0 con 0x7fdda015dce0 2024-04-25T10:13:14.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdd9fe00640 1 -- 172.21.15.119:0/3028876112 >> v1:172.21.15.119:6789/0 conn(0x7fdda015dce0 legacy=0x7fdda00d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdd9fe00640 1 --2- 172.21.15.119:0/3028876112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdda00d35b0 0x7fdd840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdd9fe00640 1 --2- 172.21.15.119:0/3028876112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84003f30 0x7fdd84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdd9fe00640 1 --2- 172.21.15.119:0/3028876112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd84004d40 0x7fdd840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdd9fe00640 1 -- 172.21.15.119:0/3028876112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda00b5de0 con 0x7fdd84003f30 2024-04-25T10:13:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdd9fe00640 1 -- 172.21.15.119:0/3028876112 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda00d2510 con 0x7fdd84004d40 2024-04-25T10:13:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdd9fe00640 1 -- 172.21.15.119:0/3028876112 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8400b9c0 con 0x7fdda00d35b0 2024-04-25T10:13:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdda4a00640 1 --2- 172.21.15.119:0/3028876112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84003f30 0x7fdd84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdda5400640 1 --2- 172.21.15.119:0/3028876112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdda00d35b0 0x7fdd840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdda4a00640 1 -- 172.21.15.119:0/3028876112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdda00d35b0 msgr2=0x7fdd840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdda4a00640 1 --2- 172.21.15.119:0/3028876112 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdda00d35b0 0x7fdd840039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdda4a00640 1 -- 172.21.15.119:0/3028876112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd84004d40 msgr2=0x7fdd840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdda4a00640 1 --2- 172.21.15.119:0/3028876112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd84004d40 0x7fdd840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.515+0000 7fdda4a00640 1 -- 172.21.15.119:0/3028876112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84003520 con 0x7fdd84003f30 2024-04-25T10:13:14.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.516+0000 7fdda4a00640 1 --2- 172.21.15.119:0/3028876112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84003f30 0x7fdd84004800 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009ac0 tx=0x7fdd84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf245482d7e9c60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.517+0000 7fdd9fe00640 1 -- 172.21.15.119:0/3028876112 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c036070 con 0x7fdd84003f30 2024-04-25T10:13:14.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.517+0000 7fdd9fe00640 1 -- 172.21.15.119:0/3028876112 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8c031400 con 0x7fdd84003f30 2024-04-25T10:13:14.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.517+0000 7fdd9fe00640 1 -- 172.21.15.119:0/3028876112 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c009690 con 0x7fdd84003f30 2024-04-25T10:13:14.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.517+0000 7fdda7e00640 1 -- 172.21.15.119:0/3028876112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84003f30 msgr2=0x7fdd84004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.517+0000 7fdda7e00640 1 --2- 172.21.15.119:0/3028876112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd84003f30 0x7fdd84004800 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009ac0 tx=0x7fdd84005830 comp rx=0 tx=0).stop 2024-04-25T10:13:14.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.517+0000 7fdda5e00640 1 -- 172.21.15.119:0/3028876112 reap_dead start 2024-04-25T10:13:14.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.517+0000 7fdda7e00640 1 -- 172.21.15.119:0/3028876112 shutdown_connections 2024-04-25T10:13:14.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.517+0000 7fdda7e00640 1 -- 172.21.15.119:0/3028876112 >> 172.21.15.119:0/3028876112 conn(0x7fdda0072900 msgr2=0x7fdda00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:14.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.517+0000 7fdda7e00640 1 -- 172.21.15.119:0/3028876112 shutdown_connections 2024-04-25T10:13:14.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.517+0000 7fdda7e00640 1 -- 172.21.15.119:0/3028876112 wait complete. 2024-04-25T10:13:14.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.519+0000 7fdda7e00640 1 Processor -- start 2024-04-25T10:13:14.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.519+0000 7fdda7e00640 1 -- start start 2024-04-25T10:13:14.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.520+0000 7fdda7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd84003f30 0x7fdda0154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.520+0000 7fdda7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdda0160b00 0x7fdda00cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.520+0000 7fdda7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdda00cb050 0x7fdda015efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.520+0000 7fdda7e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda00d3090 con 0x7fdda0160b00 2024-04-25T10:13:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.520+0000 7fdda7e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda00d2db0 con 0x7fdd84003f30 2024-04-25T10:13:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.520+0000 7fdda7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda00d2160 con 0x7fdda00cb050 2024-04-25T10:13:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.520+0000 7fdda4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdda0160b00 0x7fdda00cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.520+0000 7fdda4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdda0160b00 0x7fdda00cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35110/0 (socket says 172.21.15.119:35110) 2024-04-25T10:13:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.520+0000 7fdda4a00640 1 -- 172.21.15.119:0/3721104084 learned_addr learned my addr 172.21.15.119:0/3721104084 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.521+0000 7fdda5400640 1 --2- 172.21.15.119:0/3721104084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd84003f30 0x7fdda0154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.521+0000 7fdda5e00640 1 --2- 172.21.15.119:0/3721104084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdda00cb050 0x7fdda015efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.521+0000 7fdda4a00640 1 -- 172.21.15.119:0/3721104084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdda00cb050 msgr2=0x7fdda015efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.521+0000 7fdda4a00640 1 --2- 172.21.15.119:0/3721104084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdda00cb050 0x7fdda015efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.521+0000 7fdda4a00640 1 -- 172.21.15.119:0/3721104084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd84003f30 msgr2=0x7fdda0154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.521+0000 7fdda4a00640 1 --2- 172.21.15.119:0/3721104084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd84003f30 0x7fdda0154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.521+0000 7fdda4a00640 1 -- 172.21.15.119:0/3721104084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda0161010 con 0x7fdda0160b00 2024-04-25T10:13:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.521+0000 7fdda4a00640 1 --2- 172.21.15.119:0/3721104084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdda0160b00 0x7fdda00cab10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c034040 tx=0x7fdd8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.522+0000 7fdd9d600640 1 -- 172.21.15.119:0/3721104084 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c040020 con 0x7fdda0160b00 2024-04-25T10:13:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.522+0000 7fdd9d600640 1 -- 172.21.15.119:0/3721104084 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8c003c60 con 0x7fdda0160b00 2024-04-25T10:13:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.522+0000 7fdda7e00640 1 -- 172.21.15.119:0/3721104084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda015e500 con 0x7fdda0160b00 2024-04-25T10:13:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.522+0000 7fdd9d600640 1 -- 172.21.15.119:0/3721104084 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c036040 con 0x7fdda0160b00 2024-04-25T10:13:14.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.522+0000 7fdda7e00640 1 -- 172.21.15.119:0/3721104084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda015eac0 con 0x7fdda0160b00 2024-04-25T10:13:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.523+0000 7fdda7e00640 1 -- 172.21.15.119:0/3721104084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda00749b0 con 0x7fdda0160b00 2024-04-25T10:13:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.524+0000 7fdd9d600640 1 -- 172.21.15.119:0/3721104084 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd8c043020 con 0x7fdda0160b00 2024-04-25T10:13:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.525+0000 7fdd9d600640 1 --2- 172.21.15.119:0/3721104084 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdd8006b4d0 0x7fdd8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.525+0000 7fdd9d600640 1 -- 172.21.15.119:0/3721104084 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fdd8c0a9ff0 con 0x7fdda0160b00 2024-04-25T10:13:14.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.525+0000 7fdda5400640 1 --2- 172.21.15.119:0/3721104084 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdd8006b4d0 0x7fdd8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.526+0000 7fdda5400640 1 --2- 172.21.15.119:0/3721104084 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdd8006b4d0 0x7fdd8006d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fdd900012e0 tx=0x7fdd90009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.529+0000 7fdd9d600640 1 -- 172.21.15.119:0/3721104084 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd8c04a070 con 0x7fdda0160b00 2024-04-25T10:13:14.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.806+0000 7fdda7e00640 1 -- 172.21.15.119:0/3721104084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7fdda00cdf90 con 0x7fdda0160b00 2024-04-25T10:13:14.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.882+0000 7fdd9d600640 1 -- 172.21.15.119:0/3721104084 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v306) v1 ==== 166+0+0 (secure 0 0 0) 0x7fdd8c0044e0 con 0x7fdda0160b00 2024-04-25T10:13:14.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.886+0000 7fdda7e00640 1 -- 172.21.15.119:0/3721104084 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdd8006b4d0 msgr2=0x7fdd8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.886+0000 7fdda7e00640 1 --2- 172.21.15.119:0/3721104084 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdd8006b4d0 0x7fdd8006d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fdd900012e0 tx=0x7fdd90009040 comp rx=0 tx=0).stop 2024-04-25T10:13:14.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.886+0000 7fdda7e00640 1 -- 172.21.15.119:0/3721104084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdda0160b00 msgr2=0x7fdda00cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.886+0000 7fdda7e00640 1 --2- 172.21.15.119:0/3721104084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdda0160b00 0x7fdda00cab10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c034040 tx=0x7fdd8c004730 comp rx=0 tx=0).stop 2024-04-25T10:13:14.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.886+0000 7fdda7e00640 1 -- 172.21.15.119:0/3721104084 shutdown_connections 2024-04-25T10:13:14.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.886+0000 7fdda7e00640 1 --2- 172.21.15.119:0/3721104084 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdd8006b4d0 0x7fdd8006d990 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.886+0000 7fdda7e00640 1 --2- 172.21.15.119:0/3721104084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdda00cb050 0x7fdda015efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.886+0000 7fdda7e00640 1 --2- 172.21.15.119:0/3721104084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdda0160b00 0x7fdda00cab10 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.886+0000 7fdda7e00640 1 --2- 172.21.15.119:0/3721104084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd84003f30 0x7fdda0154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.886+0000 7fdda7e00640 1 -- 172.21.15.119:0/3721104084 >> 172.21.15.119:0/3721104084 conn(0x7fdda0072900 msgr2=0x7fdda006bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:14.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.886+0000 7fdda7e00640 1 -- 172.21.15.119:0/3721104084 shutdown_connections 2024-04-25T10:13:14.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:14.886+0000 7fdda7e00640 1 -- 172.21.15.119:0/3721104084 wait complete. 2024-04-25T10:13:14.910 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:13:14.910 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:13:14.910 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:13:14.911 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.105+0000 7fa3c2800640 1 Processor -- start 2024-04-25T10:13:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.105+0000 7fa3c2800640 1 -- start start 2024-04-25T10:13:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.107+0000 7fa3c2800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3bc0cea00 0x7fa3bc0cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.107+0000 7fa3c2800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa3bc0cfbe0 0x7fa3bc0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.107+0000 7fa3c2800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3bc0d0430 con 0x7fa3bc15f120 2024-04-25T10:13:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.107+0000 7fa3c2800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3bc0d09f0 con 0x7fa3bc0cfbe0 2024-04-25T10:13:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.107+0000 7fa3c2800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0d0710 con 0x7fa3bc0cea00 2024-04-25T10:13:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.108+0000 7fa3bbe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa3bc0cfbe0 0x7fa3bc0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.108+0000 7fa3baa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3bc0cea00 0x7fa3bc0cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.108+0000 7fa3baa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3bc0cea00 0x7fa3bc0cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:36310/0 (socket says 172.21.15.119:36310) 2024-04-25T10:13:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.108+0000 7fa3baa00640 1 -- 172.21.15.119:0/801300189 learned_addr learned my addr 172.21.15.119:0/801300189 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.108+0000 7fa3baa00640 1 -- 172.21.15.119:0/801300189 >> v2:172.21.15.132:3300/0 conn(0x7fa3bc0cfbe0 msgr2=0x7fa3bc0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.108+0000 7fa3baa00640 1 --2- 172.21.15.119:0/801300189 >> v2:172.21.15.132:3300/0 conn(0x7fa3bc0cfbe0 0x7fa3bc0ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.108+0000 7fa3baa00640 1 -- 172.21.15.119:0/801300189 >> v1:172.21.15.132:6789/0 conn(0x7fa3bc15f120 legacy=0x7fa3bc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.108+0000 7fa3baa00640 1 -- 172.21.15.119:0/801300189 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc0cc7d0 con 0x7fa3bc0cea00 2024-04-25T10:13:15.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.110+0000 7fa3baa00640 1 --2- 172.21.15.119:0/801300189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3bc0cea00 0x7fa3bc0cf6a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009870 tx=0x7fa3a402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1f1f0265dd63469 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.110+0000 7fa3ba000640 1 -- 172.21.15.119:0/801300189 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4037070 con 0x7fa3bc0cea00 2024-04-25T10:13:15.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.110+0000 7fa3ba000640 1 -- 172.21.15.119:0/801300189 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a402ae50 con 0x7fa3bc0cea00 2024-04-25T10:13:15.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.110+0000 7fa3ba000640 1 -- 172.21.15.119:0/801300189 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4032860 con 0x7fa3bc0cea00 2024-04-25T10:13:15.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.111+0000 7fa3c2800640 1 -- 172.21.15.119:0/801300189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3bc0cea00 msgr2=0x7fa3bc0cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.111+0000 7fa3c2800640 1 --2- 172.21.15.119:0/801300189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3bc0cea00 0x7fa3bc0cf6a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009870 tx=0x7fa3a402a300 comp rx=0 tx=0).stop 2024-04-25T10:13:15.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.111+0000 7fa3c2800640 1 -- 172.21.15.119:0/801300189 shutdown_connections 2024-04-25T10:13:15.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.111+0000 7fa3c2800640 1 --2- 172.21.15.119:0/801300189 >> v2:172.21.15.132:3300/0 conn(0x7fa3bc0cfbe0 0x7fa3bc0ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.111+0000 7fa3c2800640 1 --2- 172.21.15.119:0/801300189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3bc0cea00 0x7fa3bc0cf6a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.111+0000 7fa3c2800640 1 -- 172.21.15.119:0/801300189 >> 172.21.15.119:0/801300189 conn(0x7fa3bc0d1900 msgr2=0x7fa3bc0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.111+0000 7fa3c2800640 1 -- 172.21.15.119:0/801300189 shutdown_connections 2024-04-25T10:13:15.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.111+0000 7fa3c2800640 1 -- 172.21.15.119:0/801300189 wait complete. 2024-04-25T10:13:15.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.114+0000 7fa3c2800640 1 Processor -- start 2024-04-25T10:13:15.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.114+0000 7fa3c2800640 1 -- start start 2024-04-25T10:13:15.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.115+0000 7fa3c2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc0cea00 0x7fa3bc1583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.115+0000 7fa3c2800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3bc0cfbe0 0x7fa3bc0c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.115+0000 7fa3c2800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3bc15f120 0x7fa3bc0ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.115+0000 7fa3c2800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0c5450 con 0x7fa3bc0cea00 2024-04-25T10:13:15.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.115+0000 7fa3c2800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0cc480 con 0x7fa3bc15f120 2024-04-25T10:13:15.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.115+0000 7fa3c2800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0d0090 con 0x7fa3bc0cfbe0 2024-04-25T10:13:15.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.115+0000 7fa3bb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc0cea00 0x7fa3bc1583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.116+0000 7fa3bb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc0cea00 0x7fa3bc1583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35122/0 (socket says 172.21.15.119:35122) 2024-04-25T10:13:15.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.116+0000 7fa3bb400640 1 -- 172.21.15.119:0/3093876800 learned_addr learned my addr 172.21.15.119:0/3093876800 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:15.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.116+0000 7fa3baa00640 1 --2- 172.21.15.119:0/3093876800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3bc0cfbe0 0x7fa3bc0c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.116+0000 7fa3bb400640 1 -- 172.21.15.119:0/3093876800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3bc0cfbe0 msgr2=0x7fa3bc0c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.116+0000 7fa3bbe00640 1 --2- 172.21.15.119:0/3093876800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3bc15f120 0x7fa3bc0ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.116+0000 7fa3bb400640 1 --2- 172.21.15.119:0/3093876800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3bc0cfbe0 0x7fa3bc0c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.116+0000 7fa3bb400640 1 -- 172.21.15.119:0/3093876800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3bc15f120 msgr2=0x7fa3bc0ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.116+0000 7fa3bb400640 1 --2- 172.21.15.119:0/3093876800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3bc15f120 0x7fa3bc0ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.116+0000 7fa3bb400640 1 -- 172.21.15.119:0/3093876800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc0c9220 con 0x7fa3bc0cea00 2024-04-25T10:13:15.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.116+0000 7fa3bb400640 1 --2- 172.21.15.119:0/3093876800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc0cea00 0x7fa3bc1583c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00c760 tx=0x7fa3ac00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.117+0000 7fa3b3400640 1 -- 172.21.15.119:0/3093876800 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac003b70 con 0x7fa3bc0cea00 2024-04-25T10:13:15.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.117+0000 7fa3b3400640 1 -- 172.21.15.119:0/3093876800 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac003d30 con 0x7fa3bc0cea00 2024-04-25T10:13:15.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.117+0000 7fa3b3400640 1 -- 172.21.15.119:0/3093876800 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac00aa90 con 0x7fa3bc0cea00 2024-04-25T10:13:15.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.117+0000 7fa3c2800640 1 -- 172.21.15.119:0/3093876800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3bc0c9500 con 0x7fa3bc0cea00 2024-04-25T10:13:15.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.117+0000 7fa3c2800640 1 -- 172.21.15.119:0/3093876800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3bc0b46b0 con 0x7fa3bc0cea00 2024-04-25T10:13:15.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.118+0000 7fa3c2800640 1 -- 172.21.15.119:0/3093876800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3bc067850 con 0x7fa3bc0cea00 2024-04-25T10:13:15.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.119+0000 7fa3b3400640 1 -- 172.21.15.119:0/3093876800 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3ac00ac30 con 0x7fa3bc0cea00 2024-04-25T10:13:15.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.120+0000 7fa3b3400640 1 --2- 172.21.15.119:0/3093876800 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa39806b4d0 0x7fa39806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.120+0000 7fa3b3400640 1 -- 172.21.15.119:0/3093876800 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fa3ac089f70 con 0x7fa3bc0cea00 2024-04-25T10:13:15.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.121+0000 7fa3baa00640 1 --2- 172.21.15.119:0/3093876800 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa39806b4d0 0x7fa39806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.121+0000 7fa3baa00640 1 --2- 172.21.15.119:0/3093876800 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa39806b4d0 0x7fa39806d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4034000 tx=0x7fa3a402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.124+0000 7fa3b3400640 1 -- 172.21.15.119:0/3093876800 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3ac058ff0 con 0x7fa3bc0cea00 2024-04-25T10:13:15.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.319+0000 7fa3c2800640 1 -- 172.21.15.119:0/3093876800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3bc0749b0 con 0x7fa3bc0cea00 2024-04-25T10:13:15.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.321+0000 7fa3b3400640 1 -- 172.21.15.119:0/3093876800 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+13191 (secure 0 0 0) 0x7fa3ac0588f0 con 0x7fa3bc0cea00 2024-04-25T10:13:15.326 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:15.326 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":73,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:13:13.876344+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/2127292269":"2024-04-26T10:13:10.853477+0000","172.21.15.119:6837/2673349178":"2024-04-26T10:11:47.336991+0000","172.21.15.119:6836/2127292269":"2024-04-26T10:13:10.853477+0000","172.21.15.119:6836/2673349178":"2024-04-26T10:11:47.336991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:15.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.323+0000 7fa3c2800640 1 -- 172.21.15.119:0/3093876800 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa39806b4d0 msgr2=0x7fa39806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.323+0000 7fa3c2800640 1 --2- 172.21.15.119:0/3093876800 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa39806b4d0 0x7fa39806d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4034000 tx=0x7fa3a402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.323+0000 7fa3c2800640 1 -- 172.21.15.119:0/3093876800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc0cea00 msgr2=0x7fa3bc1583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.323+0000 7fa3c2800640 1 --2- 172.21.15.119:0/3093876800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc0cea00 0x7fa3bc1583c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00c760 tx=0x7fa3ac00cc20 comp rx=0 tx=0).stop 2024-04-25T10:13:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.324+0000 7fa3c2800640 1 -- 172.21.15.119:0/3093876800 shutdown_connections 2024-04-25T10:13:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.324+0000 7fa3c2800640 1 --2- 172.21.15.119:0/3093876800 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa39806b4d0 0x7fa39806d990 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.324+0000 7fa3c2800640 1 --2- 172.21.15.119:0/3093876800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3bc15f120 0x7fa3bc0ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.324+0000 7fa3c2800640 1 --2- 172.21.15.119:0/3093876800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3bc0cfbe0 0x7fa3bc0c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.324+0000 7fa3c2800640 1 --2- 172.21.15.119:0/3093876800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3bc0cea00 0x7fa3bc1583c0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.324+0000 7fa3c2800640 1 -- 172.21.15.119:0/3093876800 >> 172.21.15.119:0/3093876800 conn(0x7fa3bc0d1900 msgr2=0x7fa3bc0cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.324+0000 7fa3c2800640 1 -- 172.21.15.119:0/3093876800 shutdown_connections 2024-04-25T10:13:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.324+0000 7fa3c2800640 1 -- 172.21.15.119:0/3093876800 wait complete. 2024-04-25T10:13:15.347 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:15.348 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:15.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.543+0000 7f5622a00640 1 Processor -- start 2024-04-25T10:13:15.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.543+0000 7f5622a00640 1 -- start start 2024-04-25T10:13:15.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.545+0000 7f5622a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561c0cd310 0x7f561c166670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.545+0000 7f5622a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561c06dcb0 con 0x7f561c0ce9d0 2024-04-25T10:13:15.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.545+0000 7f5622a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561c06df90 con 0x7f561c144e70 2024-04-25T10:13:15.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.545+0000 7f5622a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561c15bfe0 con 0x7f561c0cd310 2024-04-25T10:13:15.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.545+0000 7f561b400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f561c0ce9d0 0x7f561c0cee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56268/0 (socket says 172.21.15.119:56268) 2024-04-25T10:13:15.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.545+0000 7f561b400640 1 -- 172.21.15.119:0/1243599090 learned_addr learned my addr 172.21.15.119:0/1243599090 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:15.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.545+0000 7f5620a00640 1 --2- 172.21.15.119:0/1243599090 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561c0cd310 0x7f561c166670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 -- 172.21.15.119:0/1243599090 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695388525 0 0) 0x7f561c06dcb0 con 0x7f561c0ce9d0 2024-04-25T10:13:15.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 -- 172.21.15.119:0/1243599090 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5600003520 con 0x7f561c0ce9d0 2024-04-25T10:13:15.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 -- 172.21.15.119:0/1243599090 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 782634769 0 0) 0x7f5600003520 con 0x7f561c0ce9d0 2024-04-25T10:13:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 -- 172.21.15.119:0/1243599090 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561c0cd310 msgr2=0x7f561c166670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 --2- 172.21.15.119:0/1243599090 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561c0cd310 0x7f561c166670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 -- 172.21.15.119:0/1243599090 >> v1:172.21.15.132:6789/0 conn(0x7f561c144e70 legacy=0x7f561c06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 -- 172.21.15.119:0/1243599090 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c06e270 con 0x7f561c0ce9d0 2024-04-25T10:13:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 -- 172.21.15.119:0/1243599090 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f5608001690 con 0x7f561c0ce9d0 2024-04-25T10:13:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 -- 172.21.15.119:0/1243599090 >> v1:172.21.15.119:6789/0 conn(0x7f561c0ce9d0 legacy=0x7f561c0cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 --2- 172.21.15.119:0/1243599090 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561c0cd310 0x7f56000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 --2- 172.21.15.119:0/1243599090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5600003f60 0x7f5600004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 --2- 172.21.15.119:0/1243599090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5600004d40 0x7f56000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 -- 172.21.15.119:0/1243599090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c15bfe0 con 0x7f5600004d40 2024-04-25T10:13:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 -- 172.21.15.119:0/1243599090 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560000bc00 con 0x7f5600003f60 2024-04-25T10:13:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.546+0000 7f561aa00640 1 -- 172.21.15.119:0/1243599090 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560000be00 con 0x7f561c0cd310 2024-04-25T10:13:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.547+0000 7f5620a00640 1 --2- 172.21.15.119:0/1243599090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5600004d40 0x7f56000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.547+0000 7f561b400640 1 --2- 172.21.15.119:0/1243599090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5600003f60 0x7f5600004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.547+0000 7f561be00640 1 --2- 172.21.15.119:0/1243599090 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561c0cd310 0x7f56000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.547+0000 7f5620a00640 1 -- 172.21.15.119:0/1243599090 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561c0cd310 msgr2=0x7f56000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.547+0000 7f5620a00640 1 --2- 172.21.15.119:0/1243599090 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561c0cd310 0x7f56000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.547+0000 7f5620a00640 1 -- 172.21.15.119:0/1243599090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5600003f60 msgr2=0x7f5600004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.547+0000 7f5620a00640 1 --2- 172.21.15.119:0/1243599090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5600003f60 0x7f5600004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.547+0000 7f5620a00640 1 -- 172.21.15.119:0/1243599090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5600003520 con 0x7f5600004d40 2024-04-25T10:13:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.547+0000 7f561b400640 1 --2- 172.21.15.119:0/1243599090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5600003f60 0x7f5600004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.547+0000 7f561be00640 1 --2- 172.21.15.119:0/1243599090 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561c0cd310 0x7f56000039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.548+0000 7f5620a00640 1 --2- 172.21.15.119:0/1243599090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5600004d40 0x7f56000051c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5614009fc0 tx=0x7f5600005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1b2059b568ecbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.548+0000 7f561aa00640 1 -- 172.21.15.119:0/1243599090 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614009c10 con 0x7f5600004d40 2024-04-25T10:13:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.548+0000 7f561aa00640 1 -- 172.21.15.119:0/1243599090 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5614036070 con 0x7f5600004d40 2024-04-25T10:13:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.548+0000 7f561aa00640 1 -- 172.21.15.119:0/1243599090 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561402d070 con 0x7f5600004d40 2024-04-25T10:13:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.549+0000 7f5622a00640 1 -- 172.21.15.119:0/1243599090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5600004d40 msgr2=0x7f56000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.549+0000 7f5622a00640 1 --2- 172.21.15.119:0/1243599090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5600004d40 0x7f56000051c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5614009fc0 tx=0x7f5600005830 comp rx=0 tx=0).stop 2024-04-25T10:13:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.549+0000 7f5620a00640 1 -- 172.21.15.119:0/1243599090 reap_dead start 2024-04-25T10:13:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.549+0000 7f5622a00640 1 -- 172.21.15.119:0/1243599090 shutdown_connections 2024-04-25T10:13:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.549+0000 7f5622a00640 1 -- 172.21.15.119:0/1243599090 >> 172.21.15.119:0/1243599090 conn(0x7f561c06aba0 msgr2=0x7f561c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.549+0000 7f5622a00640 1 -- 172.21.15.119:0/1243599090 shutdown_connections 2024-04-25T10:13:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.549+0000 7f5622a00640 1 -- 172.21.15.119:0/1243599090 wait complete. 2024-04-25T10:13:15.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.551+0000 7f5622a00640 1 Processor -- start 2024-04-25T10:13:15.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.551+0000 7f5622a00640 1 -- start start 2024-04-25T10:13:15.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.552+0000 7f5622a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5600004d40 0x7f561c13d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.552+0000 7f5622a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f561c15e100 0x7f561c15c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.552+0000 7f5622a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561c15c890 0x7f561c13df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.552+0000 7f5622a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c1454b0 con 0x7f561c15e100 2024-04-25T10:13:15.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.552+0000 7f5622a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c06d970 con 0x7f5600004d40 2024-04-25T10:13:15.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.552+0000 7f5622a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561c145630 con 0x7f561c15c890 2024-04-25T10:13:15.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.552+0000 7f561b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f561c15e100 0x7f561c15c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.552+0000 7f561b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f561c15e100 0x7f561c15c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35154/0 (socket says 172.21.15.119:35154) 2024-04-25T10:13:15.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.552+0000 7f561b400640 1 -- 172.21.15.119:0/1078950794 learned_addr learned my addr 172.21.15.119:0/1078950794 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:15.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.553+0000 7f561b400640 1 -- 172.21.15.119:0/1078950794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561c15c890 msgr2=0x7f561c13df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.553+0000 7f5620a00640 1 --2- 172.21.15.119:0/1078950794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561c15c890 0x7f561c13df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.553+0000 7f561be00640 1 --2- 172.21.15.119:0/1078950794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5600004d40 0x7f561c13d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.553+0000 7f561b400640 1 --2- 172.21.15.119:0/1078950794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561c15c890 0x7f561c13df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.553+0000 7f561b400640 1 -- 172.21.15.119:0/1078950794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5600004d40 msgr2=0x7f561c13d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.553+0000 7f561b400640 1 --2- 172.21.15.119:0/1078950794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5600004d40 0x7f561c13d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.553+0000 7f561b400640 1 -- 172.21.15.119:0/1078950794 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c15d080 con 0x7f561c15e100 2024-04-25T10:13:15.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.553+0000 7f561b400640 1 --2- 172.21.15.119:0/1078950794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f561c15e100 0x7f561c15c350 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f560800e820 tx=0x7f560800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.553+0000 7f5613e00640 1 -- 172.21.15.119:0/1078950794 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560800c870 con 0x7f561c15e100 2024-04-25T10:13:15.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.553+0000 7f5613e00640 1 -- 172.21.15.119:0/1078950794 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5608018070 con 0x7f561c15e100 2024-04-25T10:13:15.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.553+0000 7f5622a00640 1 -- 172.21.15.119:0/1078950794 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561c15d360 con 0x7f561c15e100 2024-04-25T10:13:15.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.553+0000 7f5613e00640 1 -- 172.21.15.119:0/1078950794 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560800cc50 con 0x7f561c15e100 2024-04-25T10:13:15.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.554+0000 7f5622a00640 1 -- 172.21.15.119:0/1078950794 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561c15dc00 con 0x7f561c15e100 2024-04-25T10:13:15.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.555+0000 7f5622a00640 1 -- 172.21.15.119:0/1078950794 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561c0750b0 con 0x7f561c15e100 2024-04-25T10:13:15.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.556+0000 7f5613e00640 1 -- 172.21.15.119:0/1078950794 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f560801d460 con 0x7f561c15e100 2024-04-25T10:13:15.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.557+0000 7f5613e00640 1 --2- 172.21.15.119:0/1078950794 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55fc06b4d0 0x7f55fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.557+0000 7f5613e00640 1 -- 172.21.15.119:0/1078950794 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f5608089ec0 con 0x7f561c15e100 2024-04-25T10:13:15.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.557+0000 7f561be00640 1 --2- 172.21.15.119:0/1078950794 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55fc06b4d0 0x7f55fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.558+0000 7f561be00640 1 --2- 172.21.15.119:0/1078950794 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55fc06b4d0 0x7f55fc06d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f560c001630 tx=0x7f560c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.561+0000 7f5613e00640 1 -- 172.21.15.119:0/1078950794 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5608058f40 con 0x7f561c15e100 2024-04-25T10:13:15.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.755+0000 7f5622a00640 1 -- 172.21.15.119:0/1078950794 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f561c06f130 con 0x7f561c15e100 2024-04-25T10:13:15.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.756+0000 7f5613e00640 1 -- 172.21.15.119:0/1078950794 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5608058840 con 0x7f561c15e100 2024-04-25T10:13:15.761 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:13:15.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.758+0000 7f5622a00640 1 -- 172.21.15.119:0/1078950794 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55fc06b4d0 msgr2=0x7f55fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.758+0000 7f5622a00640 1 --2- 172.21.15.119:0/1078950794 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55fc06b4d0 0x7f55fc06d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f560c001630 tx=0x7f560c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:15.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.758+0000 7f5622a00640 1 -- 172.21.15.119:0/1078950794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f561c15e100 msgr2=0x7f561c15c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.758+0000 7f5622a00640 1 --2- 172.21.15.119:0/1078950794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f561c15e100 0x7f561c15c350 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f560800e820 tx=0x7f560800ece0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.758+0000 7f5622a00640 1 -- 172.21.15.119:0/1078950794 shutdown_connections 2024-04-25T10:13:15.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.758+0000 7f5622a00640 1 --2- 172.21.15.119:0/1078950794 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55fc06b4d0 0x7f55fc06d990 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.758+0000 7f5622a00640 1 --2- 172.21.15.119:0/1078950794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f561c15c890 0x7f561c13df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.758+0000 7f5622a00640 1 --2- 172.21.15.119:0/1078950794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f561c15e100 0x7f561c15c350 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.758+0000 7f5622a00640 1 --2- 172.21.15.119:0/1078950794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5600004d40 0x7f561c13d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.758+0000 7f5622a00640 1 -- 172.21.15.119:0/1078950794 >> 172.21.15.119:0/1078950794 conn(0x7f561c06aba0 msgr2=0x7f561c071ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.759+0000 7f5622a00640 1 -- 172.21.15.119:0/1078950794 shutdown_connections 2024-04-25T10:13:15.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.759+0000 7f5622a00640 1 -- 172.21.15.119:0/1078950794 wait complete. 2024-04-25T10:13:15.783 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T10:13:15.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.976+0000 7f410b200640 1 Processor -- start 2024-04-25T10:13:15.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.976+0000 7f410b200640 1 -- start start 2024-04-25T10:13:15.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.977+0000 7f410b200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f410c06bbb0 0x7f410c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.977+0000 7f410b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f410c15baa0 0x7f410c15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.977+0000 7f410b200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f410c070220 con 0x7f410c066be0 2024-04-25T10:13:15.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.977+0000 7f410b200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f410c06e170 con 0x7f410c06bbb0 2024-04-25T10:13:15.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.977+0000 7f410b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410c06e450 con 0x7f410c15baa0 2024-04-25T10:13:15.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.978+0000 7f4109e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f410c06bbb0 0x7f410c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.978+0000 7f410a800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f410c15baa0 0x7f410c15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.978+0000 7f4109e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f410c06bbb0 0x7f410c06fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:42582/0 (socket says 172.21.15.119:42582) 2024-04-25T10:13:15.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.978+0000 7f4109e00640 1 -- 172.21.15.119:0/2637351935 learned_addr learned my addr 172.21.15.119:0/2637351935 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:15.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.979+0000 7f4109e00640 1 -- 172.21.15.119:0/2637351935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f410c15baa0 msgr2=0x7f410c15bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.979+0000 7f4109e00640 1 --2- 172.21.15.119:0/2637351935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f410c15baa0 0x7f410c15bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.979+0000 7f4109e00640 1 -- 172.21.15.119:0/2637351935 >> v1:172.21.15.132:6789/0 conn(0x7f410c066be0 legacy=0x7f410c06bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.979+0000 7f4109e00640 1 -- 172.21.15.119:0/2637351935 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c0b36e0 con 0x7f410c06bbb0 2024-04-25T10:13:15.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.980+0000 7f4109e00640 1 --2- 172.21.15.119:0/2637351935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f410c06bbb0 0x7f410c06fce0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009870 tx=0x7f40fc02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dee41583a4f43947 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.981+0000 7f4108a00640 1 -- 172.21.15.119:0/2637351935 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc037070 con 0x7f410c06bbb0 2024-04-25T10:13:15.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.981+0000 7f4108a00640 1 -- 172.21.15.119:0/2637351935 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40fc032440 con 0x7f410c06bbb0 2024-04-25T10:13:15.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.981+0000 7f4108a00640 1 -- 172.21.15.119:0/2637351935 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc032810 con 0x7f410c06bbb0 2024-04-25T10:13:15.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.982+0000 7f410b200640 1 -- 172.21.15.119:0/2637351935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f410c06bbb0 msgr2=0x7f410c06fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.982+0000 7f410b200640 1 --2- 172.21.15.119:0/2637351935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f410c06bbb0 0x7f410c06fce0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009870 tx=0x7f40fc02a250 comp rx=0 tx=0).stop 2024-04-25T10:13:15.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.982+0000 7f410b200640 1 -- 172.21.15.119:0/2637351935 shutdown_connections 2024-04-25T10:13:15.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.982+0000 7f410b200640 1 --2- 172.21.15.119:0/2637351935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f410c15baa0 0x7f410c15bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.982+0000 7f410b200640 1 --2- 172.21.15.119:0/2637351935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f410c06bbb0 0x7f410c06fce0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.982+0000 7f410b200640 1 -- 172.21.15.119:0/2637351935 >> 172.21.15.119:0/2637351935 conn(0x7f410c071870 msgr2=0x7f410c071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.982+0000 7f410b200640 1 -- 172.21.15.119:0/2637351935 shutdown_connections 2024-04-25T10:13:15.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.982+0000 7f410b200640 1 -- 172.21.15.119:0/2637351935 wait complete. 2024-04-25T10:13:15.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.984+0000 7f410b200640 1 Processor -- start 2024-04-25T10:13:15.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.984+0000 7f410b200640 1 -- start start 2024-04-25T10:13:15.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.985+0000 7f410b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c066be0 0x7f410c14c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.985+0000 7f410b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f410c06bbb0 0x7f410c09b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.985+0000 7f410b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f410c15baa0 0x7f410c099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.985+0000 7f410b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c0b4060 con 0x7f410c066be0 2024-04-25T10:13:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.985+0000 7f410b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c0b3ee0 con 0x7f410c06bbb0 2024-04-25T10:13:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.985+0000 7f410b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410c0b41e0 con 0x7f410c15baa0 2024-04-25T10:13:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.986+0000 7f4109e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c066be0 0x7f410c14c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.986+0000 7f4109e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c066be0 0x7f410c14c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35160/0 (socket says 172.21.15.119:35160) 2024-04-25T10:13:15.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.986+0000 7f4109e00640 1 -- 172.21.15.119:0/3264401629 learned_addr learned my addr 172.21.15.119:0/3264401629 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:15.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.986+0000 7f4109400640 1 --2- 172.21.15.119:0/3264401629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f410c06bbb0 0x7f410c09b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.986+0000 7f410a800640 1 --2- 172.21.15.119:0/3264401629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f410c15baa0 0x7f410c099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.986+0000 7f4109e00640 1 -- 172.21.15.119:0/3264401629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f410c15baa0 msgr2=0x7f410c099d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.986+0000 7f4109e00640 1 --2- 172.21.15.119:0/3264401629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f410c15baa0 0x7f410c099d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.986+0000 7f4109e00640 1 -- 172.21.15.119:0/3264401629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f410c06bbb0 msgr2=0x7f410c09b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.986+0000 7f4109e00640 1 --2- 172.21.15.119:0/3264401629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f410c06bbb0 0x7f410c09b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.986+0000 7f4109e00640 1 -- 172.21.15.119:0/3264401629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c09a8a0 con 0x7f410c066be0 2024-04-25T10:13:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.987+0000 7f4109e00640 1 --2- 172.21.15.119:0/3264401629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c066be0 0x7f410c14c590 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f40fc02a290 tx=0x7f40fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.987+0000 7f4102000640 1 -- 172.21.15.119:0/3264401629 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc037040 con 0x7f410c066be0 2024-04-25T10:13:15.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.987+0000 7f4102000640 1 -- 172.21.15.119:0/3264401629 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40fc042070 con 0x7f410c066be0 2024-04-25T10:13:15.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.987+0000 7f410b200640 1 -- 172.21.15.119:0/3264401629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410c09ab80 con 0x7f410c066be0 2024-04-25T10:13:15.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.987+0000 7f4102000640 1 -- 172.21.15.119:0/3264401629 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc030880 con 0x7f410c066be0 2024-04-25T10:13:15.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.987+0000 7f410b200640 1 -- 172.21.15.119:0/3264401629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410c09b140 con 0x7f410c066be0 2024-04-25T10:13:15.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.988+0000 7f410b200640 1 -- 172.21.15.119:0/3264401629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410c0749b0 con 0x7f410c066be0 2024-04-25T10:13:16.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.989+0000 7f4102000640 1 -- 172.21.15.119:0/3264401629 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40fc030a20 con 0x7f410c066be0 2024-04-25T10:13:16.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.990+0000 7f4102000640 1 --2- 172.21.15.119:0/3264401629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f40e806b500 0x7f40e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.990+0000 7f4102000640 1 -- 172.21.15.119:0/3264401629 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f40fc0aa860 con 0x7f410c066be0 2024-04-25T10:13:16.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.990+0000 7f4109400640 1 --2- 172.21.15.119:0/3264401629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f40e806b500 0x7f40e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.991+0000 7f4109400640 1 --2- 172.21.15.119:0/3264401629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f40e806b500 0x7f40e806d9c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f410c0afaa0 tx=0x7f40f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:15.994+0000 7f4102000640 1 -- 172.21.15.119:0/3264401629 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f40fc0798e0 con 0x7f410c066be0 2024-04-25T10:13:16.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:16.265+0000 7f410b200640 1 -- 172.21.15.119:0/3264401629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7f410c159540 con 0x7f410c066be0 2024-04-25T10:13:16.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:16.898+0000 7f4102000640 1 -- 172.21.15.119:0/3264401629 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v308) v1 ==== 170+0+0 (secure 0 0 0) 0x7f40fc0791e0 con 0x7f410c066be0 2024-04-25T10:13:16.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:16.902+0000 7f410b200640 1 -- 172.21.15.119:0/3264401629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f40e806b500 msgr2=0x7f40e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:16.902+0000 7f410b200640 1 --2- 172.21.15.119:0/3264401629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f40e806b500 0x7f40e806d9c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f410c0afaa0 tx=0x7f40f4009040 comp rx=0 tx=0).stop 2024-04-25T10:13:16.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:16.902+0000 7f410b200640 1 -- 172.21.15.119:0/3264401629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c066be0 msgr2=0x7f410c14c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:16.902+0000 7f410b200640 1 --2- 172.21.15.119:0/3264401629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c066be0 0x7f410c14c590 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f40fc02a290 tx=0x7f40fc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:16.902+0000 7f410b200640 1 -- 172.21.15.119:0/3264401629 shutdown_connections 2024-04-25T10:13:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:16.902+0000 7f410b200640 1 --2- 172.21.15.119:0/3264401629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f40e806b500 0x7f40e806d9c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:16.902+0000 7f410b200640 1 --2- 172.21.15.119:0/3264401629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f410c15baa0 0x7f410c099d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:16.902+0000 7f410b200640 1 --2- 172.21.15.119:0/3264401629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f410c06bbb0 0x7f410c09b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:16.902+0000 7f410b200640 1 --2- 172.21.15.119:0/3264401629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f410c066be0 0x7f410c14c590 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:16.902+0000 7f410b200640 1 -- 172.21.15.119:0/3264401629 >> 172.21.15.119:0/3264401629 conn(0x7f410c071870 msgr2=0x7f410c0b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:16.902+0000 7f410b200640 1 -- 172.21.15.119:0/3264401629 shutdown_connections 2024-04-25T10:13:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:16.902+0000 7f410b200640 1 -- 172.21.15.119:0/3264401629 wait complete. 2024-04-25T10:13:16.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:13:17.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.129+0000 7f2302a00640 1 Processor -- start 2024-04-25T10:13:17.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.129+0000 7f2302a00640 1 -- start start 2024-04-25T10:13:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.131+0000 7f2302a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22fc15dc80 0x7f22fc161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.131+0000 7f2302a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22fc0b97c0 0x7f22fc0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.131+0000 7f2302a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f22fc0b5a30 0x7f22fc06bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.131+0000 7f2302a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22fc0b5ea0 con 0x7f22fc15dc80 2024-04-25T10:13:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.131+0000 7f2302a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22fc1627a0 con 0x7f22fc0b5a30 2024-04-25T10:13:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.131+0000 7f2302a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc1621e0 con 0x7f22fc0b97c0 2024-04-25T10:13:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.131+0000 7f22fbe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22fc15dc80 0x7f22fc161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.131+0000 7f22fbe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22fc15dc80 0x7f22fc161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35164/0 (socket says 172.21.15.119:35164) 2024-04-25T10:13:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.131+0000 7f22fbe00640 1 -- 172.21.15.119:0/3215561817 learned_addr learned my addr 172.21.15.119:0/3215561817 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.132+0000 7f22fbe00640 1 -- 172.21.15.119:0/3215561817 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22fc0b97c0 msgr2=0x7f22fc0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.132+0000 7f22fbe00640 1 --2- 172.21.15.119:0/3215561817 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22fc0b97c0 0x7f22fc0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.132+0000 7f22fbe00640 1 -- 172.21.15.119:0/3215561817 >> v2:172.21.15.132:3300/0 conn(0x7f22fc0b5a30 msgr2=0x7f22fc06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.132+0000 7f22fbe00640 1 --2- 172.21.15.119:0/3215561817 >> v2:172.21.15.132:3300/0 conn(0x7f22fc0b5a30 0x7f22fc06bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.132+0000 7f22fbe00640 1 -- 172.21.15.119:0/3215561817 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc1624c0 con 0x7f22fc15dc80 2024-04-25T10:13:17.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.133+0000 7f22fbe00640 1 --2- 172.21.15.119:0/3215561817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22fc15dc80 0x7f22fc161870 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0d16d0 tx=0x7f22ec02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d7344687f7f37d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.133+0000 7f22fb400640 1 -- 172.21.15.119:0/3215561817 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec036070 con 0x7f22fc15dc80 2024-04-25T10:13:17.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.133+0000 7f22fb400640 1 -- 172.21.15.119:0/3215561817 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec031600 con 0x7f22fc15dc80 2024-04-25T10:13:17.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.133+0000 7f22fb400640 1 -- 172.21.15.119:0/3215561817 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0319d0 con 0x7f22fc15dc80 2024-04-25T10:13:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.134+0000 7f2302a00640 1 -- 172.21.15.119:0/3215561817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22fc15dc80 msgr2=0x7f22fc161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.134+0000 7f2302a00640 1 --2- 172.21.15.119:0/3215561817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22fc15dc80 0x7f22fc161870 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0d16d0 tx=0x7f22ec02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.134+0000 7f2302a00640 1 -- 172.21.15.119:0/3215561817 shutdown_connections 2024-04-25T10:13:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.134+0000 7f2302a00640 1 --2- 172.21.15.119:0/3215561817 >> v2:172.21.15.132:3300/0 conn(0x7f22fc0b5a30 0x7f22fc06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.134+0000 7f2302a00640 1 --2- 172.21.15.119:0/3215561817 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22fc0b97c0 0x7f22fc0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.134+0000 7f2302a00640 1 --2- 172.21.15.119:0/3215561817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22fc15dc80 0x7f22fc161870 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.134+0000 7f2302a00640 1 -- 172.21.15.119:0/3215561817 >> 172.21.15.119:0/3215561817 conn(0x7f22fc0d04f0 msgr2=0x7f22fc0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.134+0000 7f2302a00640 1 -- 172.21.15.119:0/3215561817 shutdown_connections 2024-04-25T10:13:17.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.134+0000 7f2302a00640 1 -- 172.21.15.119:0/3215561817 wait complete. 2024-04-25T10:13:17.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.136+0000 7f2302a00640 1 Processor -- start 2024-04-25T10:13:17.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.136+0000 7f2302a00640 1 -- start start 2024-04-25T10:13:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.137+0000 7f2302a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22fc0b5a30 0x7f22fc0cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.137+0000 7f2302a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22fc0b97c0 0x7f22fc156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.137+0000 7f2302a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22fc15dc80 0x7f22fc0c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.137+0000 7f2302a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc161db0 con 0x7f22fc0b97c0 2024-04-25T10:13:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.137+0000 7f2302a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc15e060 con 0x7f22fc0b5a30 2024-04-25T10:13:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.137+0000 7f2302a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc161f30 con 0x7f22fc15dc80 2024-04-25T10:13:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.137+0000 7f22f3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22fc0b97c0 0x7f22fc156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.137+0000 7f22f3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22fc0b97c0 0x7f22fc156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35172/0 (socket says 172.21.15.119:35172) 2024-04-25T10:13:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.137+0000 7f22f3400640 1 -- 172.21.15.119:0/4092267953 learned_addr learned my addr 172.21.15.119:0/4092267953 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.137+0000 7f22f3400640 1 -- 172.21.15.119:0/4092267953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22fc15dc80 msgr2=0x7f22fc0c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.137+0000 7f22f3400640 1 --2- 172.21.15.119:0/4092267953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22fc15dc80 0x7f22fc0c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.137+0000 7f22f3400640 1 -- 172.21.15.119:0/4092267953 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22fc0b5a30 msgr2=0x7f22fc0cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.137+0000 7f22f3400640 1 --2- 172.21.15.119:0/4092267953 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22fc0b5a30 0x7f22fc0cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.137+0000 7f22f3400640 1 -- 172.21.15.119:0/4092267953 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc0cbb40 con 0x7f22fc0b97c0 2024-04-25T10:13:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.138+0000 7f22f3400640 1 --2- 172.21.15.119:0/4092267953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22fc0b97c0 0x7f22fc156ed0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f22e400b480 tx=0x7f22e400b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.138+0000 7f22f8c00640 1 -- 172.21.15.119:0/4092267953 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4014070 con 0x7f22fc0b97c0 2024-04-25T10:13:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.138+0000 7f22f8c00640 1 -- 172.21.15.119:0/4092267953 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e4010870 con 0x7f22fc0b97c0 2024-04-25T10:13:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.138+0000 7f2302a00640 1 -- 172.21.15.119:0/4092267953 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22fc1579d0 con 0x7f22fc0b97c0 2024-04-25T10:13:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.138+0000 7f22f8c00640 1 -- 172.21.15.119:0/4092267953 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4010c60 con 0x7f22fc0b97c0 2024-04-25T10:13:17.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.138+0000 7f2302a00640 1 -- 172.21.15.119:0/4092267953 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22fc0cb750 con 0x7f22fc0b97c0 2024-04-25T10:13:17.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.139+0000 7f2302a00640 1 -- 172.21.15.119:0/4092267953 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22fc0cff60 con 0x7f22fc0b97c0 2024-04-25T10:13:17.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.140+0000 7f22f8c00640 1 -- 172.21.15.119:0/4092267953 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22e4017070 con 0x7f22fc0b97c0 2024-04-25T10:13:17.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.141+0000 7f22f8c00640 1 --2- 172.21.15.119:0/4092267953 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f22d806b4d0 0x7f22d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.141+0000 7f22f8c00640 1 -- 172.21.15.119:0/4092267953 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f22e4089dd0 con 0x7f22fc0b97c0 2024-04-25T10:13:17.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.141+0000 7f22fbe00640 1 --2- 172.21.15.119:0/4092267953 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f22d806b4d0 0x7f22d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.142+0000 7f22fbe00640 1 --2- 172.21.15.119:0/4092267953 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f22d806b4d0 0x7f22d806d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f22ec033000 tx=0x7f22ec02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.147+0000 7f22f8c00640 1 -- 172.21.15.119:0/4092267953 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22fc0cff60 con 0x7f22fc0b97c0 2024-04-25T10:13:17.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.340+0000 7f2302a00640 1 -- 172.21.15.119:0/4092267953 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f22fc0b4550 con 0x7f22fc0b97c0 2024-04-25T10:13:17.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.348+0000 7f22f8c00640 1 -- 172.21.15.119:0/4092267953 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f22e4058ee0 con 0x7f22fc0b97c0 2024-04-25T10:13:17.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.351+0000 7f2302a00640 1 -- 172.21.15.119:0/4092267953 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f22d806b4d0 msgr2=0x7f22d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.351+0000 7f2302a00640 1 --2- 172.21.15.119:0/4092267953 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f22d806b4d0 0x7f22d806d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f22ec033000 tx=0x7f22ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.351+0000 7f2302a00640 1 -- 172.21.15.119:0/4092267953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22fc0b97c0 msgr2=0x7f22fc156ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.351+0000 7f2302a00640 1 --2- 172.21.15.119:0/4092267953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22fc0b97c0 0x7f22fc156ed0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f22e400b480 tx=0x7f22e400b940 comp rx=0 tx=0).stop 2024-04-25T10:13:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.351+0000 7f2302a00640 1 -- 172.21.15.119:0/4092267953 shutdown_connections 2024-04-25T10:13:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.351+0000 7f2302a00640 1 --2- 172.21.15.119:0/4092267953 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f22d806b4d0 0x7f22d806d990 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.351+0000 7f2302a00640 1 --2- 172.21.15.119:0/4092267953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f22fc15dc80 0x7f22fc0c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.351+0000 7f2302a00640 1 --2- 172.21.15.119:0/4092267953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22fc0b97c0 0x7f22fc156ed0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.351+0000 7f2302a00640 1 --2- 172.21.15.119:0/4092267953 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22fc0b5a30 0x7f22fc0cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.351+0000 7f2302a00640 1 -- 172.21.15.119:0/4092267953 >> 172.21.15.119:0/4092267953 conn(0x7f22fc0d04f0 msgr2=0x7f22fc0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.351+0000 7f2302a00640 1 -- 172.21.15.119:0/4092267953 shutdown_connections 2024-04-25T10:13:17.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:17.351+0000 7f2302a00640 1 -- 172.21.15.119:0/4092267953 wait complete. 2024-04-25T10:13:17.376 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:13:17.376 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:17.481 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:17.623 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:13:17.623 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:13:17.624 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:17.681 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:17.682 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:13:17.682 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:17.739 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:17.739 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:13:17.739 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:17.796 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:17.796 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:13:17.797 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:17.855 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:17.855 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:13:17.855 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.062+0000 7f1160a00640 1 Processor -- start 2024-04-25T10:13:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.062+0000 7f1160a00640 1 -- start start 2024-04-25T10:13:18.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.064+0000 7f1160a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f115c06e8f0 0x7f115c15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.064+0000 7f1160a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f115c15c820 0x7f115c1654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.064+0000 7f1160a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f115c0ce6c0 con 0x7f115c15bf00 2024-04-25T10:13:18.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.064+0000 7f1160a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115c1455e0 con 0x7f115c06e8f0 2024-04-25T10:13:18.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.064+0000 7f1160a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115c06d450 con 0x7f115c15c820 2024-04-25T10:13:18.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.064+0000 7f1159e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f115c15bf00 0x7f115c0cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56282/0 (socket says 172.21.15.119:56282) 2024-04-25T10:13:18.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.064+0000 7f1159400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f115c06e8f0 0x7f115c15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.064+0000 7f1159e00640 1 -- 172.21.15.119:0/872989923 learned_addr learned my addr 172.21.15.119:0/872989923 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.065+0000 7f115a800640 1 --2- 172.21.15.119:0/872989923 >> v2:172.21.15.132:3300/0 conn(0x7f115c15c820 0x7f115c1654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.065+0000 7f1159400640 1 -- 172.21.15.119:0/872989923 >> v2:172.21.15.132:3300/0 conn(0x7f115c15c820 msgr2=0x7f115c1654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.065+0000 7f1159400640 1 --2- 172.21.15.119:0/872989923 >> v2:172.21.15.132:3300/0 conn(0x7f115c15c820 0x7f115c1654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.065+0000 7f1159400640 1 -- 172.21.15.119:0/872989923 >> v1:172.21.15.119:6789/0 conn(0x7f115c15bf00 legacy=0x7f115c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.065+0000 7f1159400640 1 -- 172.21.15.119:0/872989923 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c0cc4e0 con 0x7f115c06e8f0 2024-04-25T10:13:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.065+0000 7f1158a00640 1 -- 172.21.15.119:0/872989923 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039919452 0 0) 0x7f114c001110 con 0x7f115c15bf00 2024-04-25T10:13:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.066+0000 7f1159400640 1 --2- 172.21.15.119:0/872989923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f115c06e8f0 0x7f115c15c2e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f115c0cdf50 tx=0x7f114802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753a19f69e19fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.066+0000 7f1158a00640 1 -- 172.21.15.119:0/872989923 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148037070 con 0x7f115c06e8f0 2024-04-25T10:13:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.066+0000 7f1158a00640 1 -- 172.21.15.119:0/872989923 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1148032440 con 0x7f115c06e8f0 2024-04-25T10:13:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.066+0000 7f1158a00640 1 -- 172.21.15.119:0/872989923 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148032810 con 0x7f115c06e8f0 2024-04-25T10:13:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.067+0000 7f1160a00640 1 -- 172.21.15.119:0/872989923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f115c06e8f0 msgr2=0x7f115c15c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.067+0000 7f1160a00640 1 --2- 172.21.15.119:0/872989923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f115c06e8f0 0x7f115c15c2e0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f115c0cdf50 tx=0x7f114802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.067+0000 7f1160a00640 1 -- 172.21.15.119:0/872989923 shutdown_connections 2024-04-25T10:13:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.067+0000 7f1160a00640 1 --2- 172.21.15.119:0/872989923 >> v2:172.21.15.132:3300/0 conn(0x7f115c15c820 0x7f115c1654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.067+0000 7f1160a00640 1 --2- 172.21.15.119:0/872989923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f115c06e8f0 0x7f115c15c2e0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.067+0000 7f1160a00640 1 -- 172.21.15.119:0/872989923 >> 172.21.15.119:0/872989923 conn(0x7f115c06aba0 msgr2=0x7f115c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.067+0000 7f1160a00640 1 -- 172.21.15.119:0/872989923 shutdown_connections 2024-04-25T10:13:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.067+0000 7f1160a00640 1 -- 172.21.15.119:0/872989923 wait complete. 2024-04-25T10:13:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.069+0000 7f1160a00640 1 Processor -- start 2024-04-25T10:13:18.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.069+0000 7f1160a00640 1 -- start start 2024-04-25T10:13:18.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.070+0000 7f1160a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f115c06e8f0 0x7f115c140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.070+0000 7f1160a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f115c15bf00 0x7f115c13e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.070+0000 7f1160a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115c15c820 0x7f115c157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.070+0000 7f1160a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c0ccd70 con 0x7f115c06e8f0 2024-04-25T10:13:18.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.070+0000 7f1160a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c0cca90 con 0x7f115c15bf00 2024-04-25T10:13:18.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.070+0000 7f1160a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115c0cd050 con 0x7f115c15c820 2024-04-25T10:13:18.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.070+0000 7f1159e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f115c06e8f0 0x7f115c140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.070+0000 7f1159e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f115c06e8f0 0x7f115c140910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35182/0 (socket says 172.21.15.119:35182) 2024-04-25T10:13:18.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.070+0000 7f1159e00640 1 -- 172.21.15.119:0/517509956 learned_addr learned my addr 172.21.15.119:0/517509956 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.071+0000 7f115a800640 1 --2- 172.21.15.119:0/517509956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115c15c820 0x7f115c157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.071+0000 7f1159e00640 1 -- 172.21.15.119:0/517509956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115c15c820 msgr2=0x7f115c157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.071+0000 7f1159e00640 1 --2- 172.21.15.119:0/517509956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115c15c820 0x7f115c157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.071+0000 7f1159e00640 1 -- 172.21.15.119:0/517509956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f115c15bf00 msgr2=0x7f115c13e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.071+0000 7f1159e00640 1 --2- 172.21.15.119:0/517509956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f115c15bf00 0x7f115c13e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.071+0000 7f1159e00640 1 -- 172.21.15.119:0/517509956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c1f3a70 con 0x7f115c06e8f0 2024-04-25T10:13:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.071+0000 7f115a800640 1 --2- 172.21.15.119:0/517509956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115c15c820 0x7f115c157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.071+0000 7f1159e00640 1 --2- 172.21.15.119:0/517509956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f115c06e8f0 0x7f115c140910 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f114c00c7b0 tx=0x7f114c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.071+0000 7f1152000640 1 -- 172.21.15.119:0/517509956 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c018070 con 0x7f115c06e8f0 2024-04-25T10:13:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.072+0000 7f1152000640 1 -- 172.21.15.119:0/517509956 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114c0072d0 con 0x7f115c06e8f0 2024-04-25T10:13:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.072+0000 7f1160a00640 1 -- 172.21.15.119:0/517509956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115c1f3d50 con 0x7f115c06e8f0 2024-04-25T10:13:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.072+0000 7f1152000640 1 -- 172.21.15.119:0/517509956 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c0047e0 con 0x7f115c06e8f0 2024-04-25T10:13:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.072+0000 7f1160a00640 1 -- 172.21.15.119:0/517509956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c1f3720 con 0x7f115c06e8f0 2024-04-25T10:13:18.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.073+0000 7f1160a00640 1 -- 172.21.15.119:0/517509956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115c06bb30 con 0x7f115c06e8f0 2024-04-25T10:13:18.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.074+0000 7f1152000640 1 -- 172.21.15.119:0/517509956 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f114c01b070 con 0x7f115c06e8f0 2024-04-25T10:13:18.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.075+0000 7f1152000640 1 --2- 172.21.15.119:0/517509956 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f113c06b4d0 0x7f113c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.075+0000 7f1152000640 1 -- 172.21.15.119:0/517509956 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f114c089320 con 0x7f115c06e8f0 2024-04-25T10:13:18.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.075+0000 7f1159400640 1 --2- 172.21.15.119:0/517509956 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f113c06b4d0 0x7f113c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.075+0000 7f1159400640 1 --2- 172.21.15.119:0/517509956 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f113c06b4d0 0x7f113c06d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1148034000 tx=0x7f114802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.079+0000 7f1152000640 1 -- 172.21.15.119:0/517509956 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f115c0ce6c0 con 0x7f115c06e8f0 2024-04-25T10:13:18.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.313+0000 7f1160a00640 1 -- 172.21.15.119:0/517509956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f115c1f3f20 con 0x7f115c06e8f0 2024-04-25T10:13:18.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.314+0000 7f1152000640 1 -- 172.21.15.119:0/517509956 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f114c0583a0 con 0x7f115c06e8f0 2024-04-25T10:13:18.319 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:18.319 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":54,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2798722683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2798722683},{"type":"v1","addr":"172.21.15.132:6833","nonce":2798722683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2834911408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":2834911408},{"type":"v1","addr":"172.21.15.132:6835","nonce":2834911408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/643040335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":643040335},{"type":"v1","addr":"172.21.15.119:6835","nonce":643040335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/932891497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":932891497},{"type":"v1","addr":"172.21.15.119:6837","nonce":932891497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[]} 2024-04-25T10:13:18.320 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 54 2024-04-25T10:13:18.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.317+0000 7f1160a00640 1 -- 172.21.15.119:0/517509956 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f113c06b4d0 msgr2=0x7f113c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.317+0000 7f1160a00640 1 --2- 172.21.15.119:0/517509956 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f113c06b4d0 0x7f113c06d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1148034000 tx=0x7f114802c000 comp rx=0 tx=0).stop 2024-04-25T10:13:18.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.317+0000 7f1160a00640 1 -- 172.21.15.119:0/517509956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f115c06e8f0 msgr2=0x7f115c140910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.317+0000 7f1160a00640 1 --2- 172.21.15.119:0/517509956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f115c06e8f0 0x7f115c140910 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f114c00c7b0 tx=0x7f114c00cc70 comp rx=0 tx=0).stop 2024-04-25T10:13:18.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.317+0000 7f1160a00640 1 -- 172.21.15.119:0/517509956 shutdown_connections 2024-04-25T10:13:18.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.317+0000 7f1160a00640 1 --2- 172.21.15.119:0/517509956 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f113c06b4d0 0x7f113c06d990 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.317+0000 7f1160a00640 1 --2- 172.21.15.119:0/517509956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f115c15c820 0x7f115c157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.317+0000 7f1160a00640 1 --2- 172.21.15.119:0/517509956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f115c15bf00 0x7f115c13e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.317+0000 7f1160a00640 1 --2- 172.21.15.119:0/517509956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f115c06e8f0 0x7f115c140910 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.317+0000 7f1160a00640 1 -- 172.21.15.119:0/517509956 >> 172.21.15.119:0/517509956 conn(0x7f115c06aba0 msgr2=0x7f115c143da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.317+0000 7f1160a00640 1 -- 172.21.15.119:0/517509956 shutdown_connections 2024-04-25T10:13:18.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.317+0000 7f1160a00640 1 -- 172.21.15.119:0/517509956 wait complete. 2024-04-25T10:13:18.340 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:13:18.340 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:13:18.341 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:13:18.341 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:18.341 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:13:18.341 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:13:18.341 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:13:18.341 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:18.341 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:13:18.342 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:13:18.342 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:13:18.342 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:18.342 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:13:18.342 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:13:18.342 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:13:18.342 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:18.359 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:13:18.359 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:13:18.360 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:13:18.360 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:13:18.363 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:13:18.363 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:13:18.370 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:13:18.370 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:13:18.408 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:13:18.408 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:18.414 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:13:18.414 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:18.417 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:13:18.417 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:18.424 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:13:18.424 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:18.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:13:18.557 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:13:18.557 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2024-04-25T10:13:18.573 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-04-25T10:13:18.576 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-04-25T10:13:18.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.599+0000 7f6d90c00640 1 Processor -- start 2024-04-25T10:13:18.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.599+0000 7f6d90c00640 1 -- start start 2024-04-25T10:13:18.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.600+0000 7f6d90c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6d8c0ce860 0x7f6d8c0cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.600+0000 7f6d90c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c0b5710 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.600+0000 7f6d90c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c0b59f0 con 0x7f6d8c15b7e0 2024-04-25T10:13:18.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.600+0000 7f6d90c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d8c0b5cd0 con 0x7f6d8c0ce860 2024-04-25T10:13:18.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.600+0000 7f6d8be00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6d8c0d00a0 0x7f6d8c0d0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56294/0 (socket says 172.21.15.119:56294) 2024-04-25T10:13:18.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.600+0000 7f6d83e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6d8c0ce860 0x7f6d8c0cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.601+0000 7f6d8be00640 1 -- 172.21.15.119:0/2000380902 learned_addr learned my addr 172.21.15.119:0/2000380902 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:18.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.601+0000 7f6d83e00640 1 -- 172.21.15.119:0/2000380902 >> v1:172.21.15.132:6789/0 conn(0x7f6d8c15b7e0 legacy=0x7f6d8c15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.601+0000 7f6d83e00640 1 -- 172.21.15.119:0/2000380902 >> v1:172.21.15.119:6789/0 conn(0x7f6d8c0d00a0 legacy=0x7f6d8c0d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.601+0000 7f6d83e00640 1 -- 172.21.15.119:0/2000380902 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c0b5fb0 con 0x7f6d8c0ce860 2024-04-25T10:13:18.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.601+0000 7f6d8aa00640 1 -- 172.21.15.119:0/2000380902 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035097192 0 0) 0x7f6d8c0b5710 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.602+0000 7f6d83e00640 1 --2- 172.21.15.119:0/2000380902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c0ce860 0x7f6d8c0cecf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6d740097a0 tx=0x7f6d7402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf596bb6b1fd3a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.602+0000 7f6d8aa00640 1 -- 172.21.15.119:0/2000380902 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74037070 con 0x7f6d8c0ce860 2024-04-25T10:13:18.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.602+0000 7f6d8aa00640 1 -- 172.21.15.119:0/2000380902 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d74032440 con 0x7f6d8c0ce860 2024-04-25T10:13:18.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.602+0000 7f6d8aa00640 1 -- 172.21.15.119:0/2000380902 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7402ae60 con 0x7f6d8c0ce860 2024-04-25T10:13:18.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.603+0000 7f6d90c00640 1 -- 172.21.15.119:0/2000380902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c0ce860 msgr2=0x7f6d8c0cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.603+0000 7f6d90c00640 1 --2- 172.21.15.119:0/2000380902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c0ce860 0x7f6d8c0cecf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6d740097a0 tx=0x7f6d7402a110 comp rx=0 tx=0).stop 2024-04-25T10:13:18.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.603+0000 7f6d90c00640 1 -- 172.21.15.119:0/2000380902 shutdown_connections 2024-04-25T10:13:18.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.603+0000 7f6d90c00640 1 --2- 172.21.15.119:0/2000380902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c0ce860 0x7f6d8c0cecf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.603+0000 7f6d90c00640 1 -- 172.21.15.119:0/2000380902 >> 172.21.15.119:0/2000380902 conn(0x7f6d8c0722d0 msgr2=0x7f6d8c0726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.603+0000 7f6d90c00640 1 -- 172.21.15.119:0/2000380902 shutdown_connections 2024-04-25T10:13:18.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.603+0000 7f6d90c00640 1 -- 172.21.15.119:0/2000380902 wait complete. 2024-04-25T10:13:18.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.605+0000 7f6d90c00640 1 Processor -- start 2024-04-25T10:13:18.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.605+0000 7f6d90c00640 1 -- start start 2024-04-25T10:13:18.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.606+0000 7f6d90c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d8c0ce860 0x7f6d8c1f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.606+0000 7f6d90c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d8c0d00a0 0x7f6d8c1608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.606+0000 7f6d90c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c15b7e0 0x7f6d8c15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.606+0000 7f6d90c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c161ae0 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.606+0000 7f6d90c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c161960 con 0x7f6d8c15b7e0 2024-04-25T10:13:18.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.606+0000 7f6d90c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c161c60 con 0x7f6d8c0ce860 2024-04-25T10:13:18.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.606+0000 7f6d83e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d8c0d00a0 0x7f6d8c1608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.606+0000 7f6d83e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d8c0d00a0 0x7f6d8c1608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35236/0 (socket says 172.21.15.119:35236) 2024-04-25T10:13:18.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.606+0000 7f6d83e00640 1 -- 172.21.15.119:0/3558042495 learned_addr learned my addr 172.21.15.119:0/3558042495 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:18.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.606+0000 7f6d8be00640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c15b7e0 0x7f6d8c15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.606+0000 7f6d8b400640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d8c0ce860 0x7f6d8c1f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.607+0000 7f6d83e00640 1 -- 172.21.15.119:0/3558042495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d8c0ce860 msgr2=0x7f6d8c1f31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.607+0000 7f6d83e00640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d8c0ce860 0x7f6d8c1f31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.607+0000 7f6d83e00640 1 -- 172.21.15.119:0/3558042495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c15b7e0 msgr2=0x7f6d8c15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.607+0000 7f6d83e00640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c15b7e0 0x7f6d8c15cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.607+0000 7f6d83e00640 1 -- 172.21.15.119:0/3558042495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c1f6ee0 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.607+0000 7f6d8be00640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c15b7e0 0x7f6d8c15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:18.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.607+0000 7f6d83e00640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d8c0d00a0 0x7f6d8c1608a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6d740097a0 tx=0x7f6d740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.607+0000 7f6d83400640 1 -- 172.21.15.119:0/3558042495 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74037050 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.607+0000 7f6d83400640 1 -- 172.21.15.119:0/3558042495 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d74038040 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.607+0000 7f6d83400640 1 -- 172.21.15.119:0/3558042495 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74004470 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.607+0000 7f6d90c00640 1 -- 172.21.15.119:0/3558042495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c1f71c0 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.608+0000 7f6d90c00640 1 -- 172.21.15.119:0/3558042495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c1f36e0 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.608+0000 7f6d90c00640 1 -- 172.21.15.119:0/3558042495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c0749b0 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.609+0000 7f6d83400640 1 -- 172.21.15.119:0/3558042495 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d74030900 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.610+0000 7f6d83400640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6d6806b4d0 0x7f6d6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.610+0000 7f6d83400640 1 -- 172.21.15.119:0/3558042495 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f6d740aad30 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.611+0000 7f6d8b400640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6d6806b4d0 0x7f6d6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.611+0000 7f6d8b400640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6d6806b4d0 0x7f6d6806d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c001620 tx=0x7f6d7c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.615+0000 7f6d83400640 1 -- 172.21.15.119:0/3558042495 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d74079f60 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.819+0000 7f6d90c00640 1 -- 172.21.15.119:0/3558042495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6d8c0d1da0 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.924+0000 7f6d83400640 1 -- 172.21.15.119:0/3558042495 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6d74079860 con 0x7f6d8c0d00a0 2024-04-25T10:13:18.929 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2024-04-25T10:13:18.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.928+0000 7f6d90c00640 1 -- 172.21.15.119:0/3558042495 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6d6806b4d0 msgr2=0x7f6d6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.928+0000 7f6d90c00640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6d6806b4d0 0x7f6d6806d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c001620 tx=0x7f6d7c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:18.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.928+0000 7f6d90c00640 1 -- 172.21.15.119:0/3558042495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d8c0d00a0 msgr2=0x7f6d8c1608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.928+0000 7f6d90c00640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d8c0d00a0 0x7f6d8c1608a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6d740097a0 tx=0x7f6d740046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.928+0000 7f6d90c00640 1 -- 172.21.15.119:0/3558042495 shutdown_connections 2024-04-25T10:13:18.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.928+0000 7f6d90c00640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6d6806b4d0 0x7f6d6806d990 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.928+0000 7f6d90c00640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d8c15b7e0 0x7f6d8c15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.928+0000 7f6d90c00640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d8c0d00a0 0x7f6d8c1608a0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.928+0000 7f6d90c00640 1 --2- 172.21.15.119:0/3558042495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6d8c0ce860 0x7f6d8c1f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.928+0000 7f6d90c00640 1 -- 172.21.15.119:0/3558042495 >> 172.21.15.119:0/3558042495 conn(0x7f6d8c0722d0 msgr2=0x7f6d8c0b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.928+0000 7f6d90c00640 1 -- 172.21.15.119:0/3558042495 shutdown_connections 2024-04-25T10:13:18.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:18.928+0000 7f6d90c00640 1 -- 172.21.15.119:0/3558042495 wait complete. 2024-04-25T10:13:18.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:13:19.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.141+0000 7f589aa00640 1 Processor -- start 2024-04-25T10:13:19.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.141+0000 7f589aa00640 1 -- start start 2024-04-25T10:13:19.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.142+0000 7f589aa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f58940ca1a0 0x7f58940ca930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.142+0000 7f589aa00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58940caf10 con 0x7f589415a270 2024-04-25T10:13:19.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.142+0000 7f589aa00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58940cb1f0 con 0x7f58940710b0 2024-04-25T10:13:19.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.142+0000 7f589aa00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58940cb4d0 con 0x7f58940ca1a0 2024-04-25T10:13:19.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.142+0000 7f5893e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f589415a270 0x7f589406dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56298/0 (socket says 172.21.15.119:56298) 2024-04-25T10:13:19.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.142+0000 7f5893e00640 1 -- 172.21.15.119:0/1919011897 learned_addr learned my addr 172.21.15.119:0/1919011897 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:19.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5893400640 1 --2- 172.21.15.119:0/1919011897 >> v2:172.21.15.132:3300/0 conn(0x7f58940ca1a0 0x7f58940ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 -- 172.21.15.119:0/1919011897 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271917197 0 0) 0x7f58940caf10 con 0x7f589415a270 2024-04-25T10:13:19.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 -- 172.21.15.119:0/1919011897 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5878003520 con 0x7f589415a270 2024-04-25T10:13:19.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 -- 172.21.15.119:0/1919011897 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 124408277 0 0) 0x7f5878003520 con 0x7f589415a270 2024-04-25T10:13:19.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 -- 172.21.15.119:0/1919011897 >> v2:172.21.15.132:3300/0 conn(0x7f58940ca1a0 msgr2=0x7f58940ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 --2- 172.21.15.119:0/1919011897 >> v2:172.21.15.132:3300/0 conn(0x7f58940ca1a0 0x7f58940ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 -- 172.21.15.119:0/1919011897 >> v1:172.21.15.132:6789/0 conn(0x7f58940710b0 legacy=0x7f58940cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 -- 172.21.15.119:0/1919011897 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58940cd180 con 0x7f589415a270 2024-04-25T10:13:19.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 -- 172.21.15.119:0/1919011897 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f5884001670 con 0x7f589415a270 2024-04-25T10:13:19.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 -- 172.21.15.119:0/1919011897 >> v1:172.21.15.119:6789/0 conn(0x7f589415a270 legacy=0x7f589406dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 --2- 172.21.15.119:0/1919011897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58780039b0 0x7f5878003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 --2- 172.21.15.119:0/1919011897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5878004800 0x7f5878004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 --2- 172.21.15.119:0/1919011897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5878005230 0x7f587800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 -- 172.21.15.119:0/1919011897 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58940cb4d0 con 0x7f5878005230 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 -- 172.21.15.119:0/1919011897 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587800c190 con 0x7f5878004800 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.143+0000 7f5892a00640 1 -- 172.21.15.119:0/1919011897 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587800e5a0 con 0x7f58780039b0 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.144+0000 7f5898a00640 1 --2- 172.21.15.119:0/1919011897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5878005230 0x7f587800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.144+0000 7f5893400640 1 --2- 172.21.15.119:0/1919011897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5878004800 0x7f5878004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.144+0000 7f5893e00640 1 --2- 172.21.15.119:0/1919011897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58780039b0 0x7f5878003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.144+0000 7f5898a00640 1 -- 172.21.15.119:0/1919011897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58780039b0 msgr2=0x7f5878003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.144+0000 7f5898a00640 1 --2- 172.21.15.119:0/1919011897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58780039b0 0x7f5878003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.144+0000 7f5898a00640 1 -- 172.21.15.119:0/1919011897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5878004800 msgr2=0x7f5878004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.144+0000 7f5898a00640 1 --2- 172.21.15.119:0/1919011897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5878004800 0x7f5878004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.144+0000 7f5898a00640 1 -- 172.21.15.119:0/1919011897 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5878003520 con 0x7f5878005230 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.144+0000 7f5893400640 1 --2- 172.21.15.119:0/1919011897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5878004800 0x7f5878004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:19.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.145+0000 7f5898a00640 1 -- 172.21.15.119:0/1919011897 reap_dead start 2024-04-25T10:13:19.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.145+0000 7f5898a00640 1 --2- 172.21.15.119:0/1919011897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5878005230 0x7f587800b9c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f588c0044b0 tx=0x7f5878005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fba2d16910ab79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.145+0000 7f5892a00640 1 -- 172.21.15.119:0/1919011897 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c004080 con 0x7f5878005230 2024-04-25T10:13:19.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.145+0000 7f5892a00640 1 -- 172.21.15.119:0/1919011897 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588c02ada0 con 0x7f5878005230 2024-04-25T10:13:19.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.145+0000 7f5892a00640 1 -- 172.21.15.119:0/1919011897 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c006d90 con 0x7f5878005230 2024-04-25T10:13:19.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.146+0000 7f589aa00640 1 -- 172.21.15.119:0/1919011897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5878005230 msgr2=0x7f587800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.146+0000 7f589aa00640 1 --2- 172.21.15.119:0/1919011897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5878005230 0x7f587800b9c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f588c0044b0 tx=0x7f5878005830 comp rx=0 tx=0).stop 2024-04-25T10:13:19.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.146+0000 7f589aa00640 1 -- 172.21.15.119:0/1919011897 shutdown_connections 2024-04-25T10:13:19.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.146+0000 7f589aa00640 1 --2- 172.21.15.119:0/1919011897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5878005230 0x7f587800b9c0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.146+0000 7f589aa00640 1 -- 172.21.15.119:0/1919011897 >> 172.21.15.119:0/1919011897 conn(0x7f58940cf5a0 msgr2=0x7f58940cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.146+0000 7f589aa00640 1 -- 172.21.15.119:0/1919011897 shutdown_connections 2024-04-25T10:13:19.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.146+0000 7f589aa00640 1 -- 172.21.15.119:0/1919011897 wait complete. 2024-04-25T10:13:19.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.148+0000 7f589aa00640 1 Processor -- start 2024-04-25T10:13:19.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.148+0000 7f589aa00640 1 -- start start 2024-04-25T10:13:19.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.149+0000 7f589aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5878005230 0x7f5894152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.149+0000 7f589aa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58940b0560 0x7f5894141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.149+0000 7f589aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58940b1b00 0x7f589415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.149+0000 7f589aa00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58940ca930 con 0x7f58940b1b00 2024-04-25T10:13:19.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.149+0000 7f589aa00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58940cf1e0 con 0x7f58940b0560 2024-04-25T10:13:19.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.149+0000 7f589aa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58940caab0 con 0x7f5878005230 2024-04-25T10:13:19.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.149+0000 7f5898a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58940b1b00 0x7f589415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.149+0000 7f5898a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58940b1b00 0x7f589415a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35250/0 (socket says 172.21.15.119:35250) 2024-04-25T10:13:19.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.149+0000 7f5898a00640 1 -- 172.21.15.119:0/419092142 learned_addr learned my addr 172.21.15.119:0/419092142 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:19.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.150+0000 7f5893e00640 1 --2- 172.21.15.119:0/419092142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5878005230 0x7f5894152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.150+0000 7f5893400640 1 --2- 172.21.15.119:0/419092142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58940b0560 0x7f5894141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.150+0000 7f5898a00640 1 -- 172.21.15.119:0/419092142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5878005230 msgr2=0x7f5894152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.150+0000 7f5898a00640 1 --2- 172.21.15.119:0/419092142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5878005230 0x7f5894152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.150+0000 7f5898a00640 1 -- 172.21.15.119:0/419092142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58940b0560 msgr2=0x7f5894141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.150+0000 7f5898a00640 1 --2- 172.21.15.119:0/419092142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58940b0560 0x7f5894141ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.150+0000 7f5898a00640 1 -- 172.21.15.119:0/419092142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58940b0a60 con 0x7f58940b1b00 2024-04-25T10:13:19.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.150+0000 7f5898a00640 1 --2- 172.21.15.119:0/419092142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58940b1b00 0x7f589415a7e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f588c0045d0 tx=0x7f588c003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.150+0000 7f588be00640 1 -- 172.21.15.119:0/419092142 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c007df0 con 0x7f58940b1b00 2024-04-25T10:13:19.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.150+0000 7f588be00640 1 -- 172.21.15.119:0/419092142 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588c007660 con 0x7f58940b1b00 2024-04-25T10:13:19.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.150+0000 7f588be00640 1 -- 172.21.15.119:0/419092142 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c007a10 con 0x7f58940b1b00 2024-04-25T10:13:19.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.150+0000 7f589aa00640 1 -- 172.21.15.119:0/419092142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58940b0d40 con 0x7f58940b1b00 2024-04-25T10:13:19.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.151+0000 7f589aa00640 1 -- 172.21.15.119:0/419092142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58940b1300 con 0x7f58940b1b00 2024-04-25T10:13:19.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.152+0000 7f589aa00640 1 -- 172.21.15.119:0/419092142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58940749b0 con 0x7f58940b1b00 2024-04-25T10:13:19.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.152+0000 7f588be00640 1 -- 172.21.15.119:0/419092142 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f588c045020 con 0x7f58940b1b00 2024-04-25T10:13:19.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.153+0000 7f588be00640 1 --2- 172.21.15.119:0/419092142 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f587406b500 0x7f587406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.153+0000 7f588be00640 1 -- 172.21.15.119:0/419092142 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f588c0a9d80 con 0x7f58940b1b00 2024-04-25T10:13:19.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.153+0000 7f5893e00640 1 --2- 172.21.15.119:0/419092142 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f587406b500 0x7f587406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.154+0000 7f5893e00640 1 --2- 172.21.15.119:0/419092142 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f587406b500 0x7f587406d9c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5884001200 tx=0x7f5884009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.157+0000 7f588be00640 1 -- 172.21.15.119:0/419092142 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f588c079000 con 0x7f58940b1b00 2024-04-25T10:13:19.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.421+0000 7f589aa00640 1 -- 172.21.15.119:0/419092142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f58940c98a0 con 0x7f58940b1b00 2024-04-25T10:13:19.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.423+0000 7f588be00640 1 -- 172.21.15.119:0/419092142 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f588c078900 con 0x7f58940b1b00 2024-04-25T10:13:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:19.428 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:13:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2024-04-25T10:13:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2024-04-25T10:13:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmFDz9FBAAdC/N+8UnkYkK5wrDlgpkgA==", 2024-04-25T10:13:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:13:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:13:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:13:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.428 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmmh7lLhAAkzsFzTL6Ko/J/0ZJOpvR4A==", 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2024-04-25T10:13:19.429 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmojotIhAAdSIhFOXoYZN+01nbCs3e+Q==", 2024-04-25T10:13:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:13:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:13:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:13:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.430 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypm1gp7ORAAvtG28IuhGtR1/Vj6JCPq5A==", 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAPKypmOI9YMRAAmMrg0xRUG+GG+waJ12llJQ==", 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:19.431 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQATKypmHV5GDxAAW0XzpAxF7oxgVBKy5yzavw==", 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2024-04-25T10:13:19.432 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAWKypmXTiRGhAAZxct/Ck3A3DMBjqNW3HXrg==", 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAZKypmEeGSMhAAwVNTny5OmP/KONYhIXzrrQ==", 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:19.433 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:13:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2024-04-25T10:13:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAgKypmh7gPBRAApx1VM3WpYW7hIQzu7+6J7w==", 2024-04-25T10:13:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:13:19.434 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAjKypmQ2EaEBAAoFYMlnh5/ar0pAlynwjYOA==", 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2024-04-25T10:13:19.435 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAmKypmDATkKRAAsIDViOBeDrd5N91wf8zGLA==", 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAqKypmNskbCBAAQIYhJ9lfHHr9EP/n5QJoRw==", 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.436 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA==", 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmm0DLGhAApkxEcc1jUU5T/9ErkTkT4A==", 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.437 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmMZ89ERAAICfWm/ZgOLQMVSv5xG0Crg==", 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:13:19.438 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmrokCIxAAwi8FtvYSVhMguvZ0OyYt+g==", 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.439 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2024-04-25T10:13:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAKKypmN9WFGRAAi+stjuG2ZDBjafeR9Ygrug==", 2024-04-25T10:13:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:13:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:13:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:13:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2024-04-25T10:13:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypm/KyKDBAA646fW6qrcXB31usxeIApaA==", 2024-04-25T10:13:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:13:19.440 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2024-04-25T10:13:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:13:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2024-04-25T10:13:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmr8SqBBAA8l4CjUEidXz1obWJFM4DHA==", 2024-04-25T10:13:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:13:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:13:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2024-04-25T10:13:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:13:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-25T10:13:19.442 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.425+0000 7f589aa00640 1 -- 172.21.15.119:0/419092142 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f587406b500 msgr2=0x7f587406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.425+0000 7f589aa00640 1 --2- 172.21.15.119:0/419092142 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f587406b500 0x7f587406d9c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5884001200 tx=0x7f5884009040 comp rx=0 tx=0).stop 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.425+0000 7f589aa00640 1 -- 172.21.15.119:0/419092142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58940b1b00 msgr2=0x7f589415a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.425+0000 7f589aa00640 1 --2- 172.21.15.119:0/419092142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58940b1b00 0x7f589415a7e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f588c0045d0 tx=0x7f588c003a70 comp rx=0 tx=0).stop 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.425+0000 7f589aa00640 1 -- 172.21.15.119:0/419092142 shutdown_connections 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.425+0000 7f589aa00640 1 --2- 172.21.15.119:0/419092142 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f587406b500 0x7f587406d9c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.425+0000 7f589aa00640 1 --2- 172.21.15.119:0/419092142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58940b1b00 0x7f589415a7e0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.425+0000 7f589aa00640 1 --2- 172.21.15.119:0/419092142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58940b0560 0x7f5894141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.425+0000 7f589aa00640 1 --2- 172.21.15.119:0/419092142 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5878005230 0x7f5894152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.425+0000 7f589aa00640 1 -- 172.21.15.119:0/419092142 >> 172.21.15.119:0/419092142 conn(0x7f58940cf5a0 msgr2=0x7f589415bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.426+0000 7f589aa00640 1 -- 172.21.15.119:0/419092142 shutdown_connections 2024-04-25T10:13:19.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.426+0000 7f589aa00640 1 -- 172.21.15.119:0/419092142 wait complete. 2024-04-25T10:13:19.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:19.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.644+0000 7f17df200640 1 Processor -- start 2024-04-25T10:13:19.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.644+0000 7f17df200640 1 -- start start 2024-04-25T10:13:19.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.646+0000 7f17df200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17e00b5050 0x7f17e015fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.646+0000 7f17df200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f17e006e430 0x7f17e015bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.646+0000 7f17df200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e00cd440 con 0x7f17e0144470 2024-04-25T10:13:19.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.646+0000 7f17df200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e0145340 con 0x7f17e00b5050 2024-04-25T10:13:19.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.646+0000 7f17df200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e00cd720 con 0x7f17e006e430 2024-04-25T10:13:19.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.646+0000 7f17dd400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17e00b5050 0x7f17e015fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.646+0000 7f17dde00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f17e0144470 0x7f17e00b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56304/0 (socket says 172.21.15.119:56304) 2024-04-25T10:13:19.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.646+0000 7f17dde00640 1 -- 172.21.15.119:0/3784982123 learned_addr learned my addr 172.21.15.119:0/3784982123 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:19.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.646+0000 7f17de800640 1 --2- 172.21.15.119:0/3784982123 >> v2:172.21.15.132:3300/0 conn(0x7f17e006e430 0x7f17e015bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.647+0000 7f17dd400640 1 -- 172.21.15.119:0/3784982123 >> v2:172.21.15.132:3300/0 conn(0x7f17e006e430 msgr2=0x7f17e015bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.647+0000 7f17dd400640 1 --2- 172.21.15.119:0/3784982123 >> v2:172.21.15.132:3300/0 conn(0x7f17e006e430 0x7f17e015bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.647+0000 7f17dd400640 1 -- 172.21.15.119:0/3784982123 >> v1:172.21.15.119:6789/0 conn(0x7f17e0144470 legacy=0x7f17e00b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.647+0000 7f17dd400640 1 -- 172.21.15.119:0/3784982123 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e00cda00 con 0x7f17e00b5050 2024-04-25T10:13:19.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.647+0000 7f17de800640 1 --2- 172.21.15.119:0/3784982123 >> v2:172.21.15.132:3300/0 conn(0x7f17e006e430 0x7f17e015bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:19.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.647+0000 7f17dca00640 1 -- 172.21.15.119:0/3784982123 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769597806 0 0) 0x7f17e00cd440 con 0x7f17e0144470 2024-04-25T10:13:19.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.648+0000 7f17dd400640 1 --2- 172.21.15.119:0/3784982123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e00b5050 0x7f17e015fc90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f17e006ebf0 tx=0x7f17cc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834755d0f38b139e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.648+0000 7f17dca00640 1 -- 172.21.15.119:0/3784982123 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc037070 con 0x7f17e00b5050 2024-04-25T10:13:19.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.648+0000 7f17dca00640 1 -- 172.21.15.119:0/3784982123 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17cc032440 con 0x7f17e00b5050 2024-04-25T10:13:19.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.648+0000 7f17dca00640 1 -- 172.21.15.119:0/3784982123 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc032810 con 0x7f17e00b5050 2024-04-25T10:13:19.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.649+0000 7f17df200640 1 -- 172.21.15.119:0/3784982123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e00b5050 msgr2=0x7f17e015fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.649+0000 7f17df200640 1 --2- 172.21.15.119:0/3784982123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e00b5050 0x7f17e015fc90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f17e006ebf0 tx=0x7f17cc02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.649+0000 7f17df200640 1 -- 172.21.15.119:0/3784982123 shutdown_connections 2024-04-25T10:13:19.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.649+0000 7f17df200640 1 --2- 172.21.15.119:0/3784982123 >> v2:172.21.15.132:3300/0 conn(0x7f17e006e430 0x7f17e015bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.649+0000 7f17df200640 1 --2- 172.21.15.119:0/3784982123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e00b5050 0x7f17e015fc90 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.649+0000 7f17df200640 1 -- 172.21.15.119:0/3784982123 >> 172.21.15.119:0/3784982123 conn(0x7f17e006b1d0 msgr2=0x7f17e00b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.649+0000 7f17df200640 1 -- 172.21.15.119:0/3784982123 shutdown_connections 2024-04-25T10:13:19.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.649+0000 7f17df200640 1 -- 172.21.15.119:0/3784982123 wait complete. 2024-04-25T10:13:19.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.651+0000 7f17df200640 1 Processor -- start 2024-04-25T10:13:19.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.651+0000 7f17df200640 1 -- start start 2024-04-25T10:13:19.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.652+0000 7f17df200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e006e430 0x7f17e015b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.652+0000 7f17df200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17e00b5050 0x7f17e00abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.652+0000 7f17df200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e0144470 0x7f17e00adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.652+0000 7f17df200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e00ce200 con 0x7f17e0144470 2024-04-25T10:13:19.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.652+0000 7f17df200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e0144850 con 0x7f17e00b5050 2024-04-25T10:13:19.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.652+0000 7f17df200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e015c230 con 0x7f17e006e430 2024-04-25T10:13:19.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.652+0000 7f17de800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e0144470 0x7f17e00adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.652+0000 7f17de800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e0144470 0x7f17e00adbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35274/0 (socket says 172.21.15.119:35274) 2024-04-25T10:13:19.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.652+0000 7f17de800640 1 -- 172.21.15.119:0/4249588805 learned_addr learned my addr 172.21.15.119:0/4249588805 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:19.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.652+0000 7f17de800640 1 -- 172.21.15.119:0/4249588805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e006e430 msgr2=0x7f17e015b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.652+0000 7f17de800640 1 --2- 172.21.15.119:0/4249588805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e006e430 0x7f17e015b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.652+0000 7f17de800640 1 -- 172.21.15.119:0/4249588805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17e00b5050 msgr2=0x7f17e00abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.652+0000 7f17de800640 1 --2- 172.21.15.119:0/4249588805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17e00b5050 0x7f17e00abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.652+0000 7f17de800640 1 -- 172.21.15.119:0/4249588805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e0130b80 con 0x7f17e0144470 2024-04-25T10:13:19.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.653+0000 7f17de800640 1 --2- 172.21.15.119:0/4249588805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e0144470 0x7f17e00adbb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f17d800c7f0 tx=0x7f17d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.653+0000 7f17d6000640 1 -- 172.21.15.119:0/4249588805 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d8015070 con 0x7f17e0144470 2024-04-25T10:13:19.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.653+0000 7f17d6000640 1 -- 172.21.15.119:0/4249588805 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d8007820 con 0x7f17e0144470 2024-04-25T10:13:19.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.653+0000 7f17d6000640 1 -- 172.21.15.119:0/4249588805 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d8007c50 con 0x7f17e0144470 2024-04-25T10:13:19.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.653+0000 7f17df200640 1 -- 172.21.15.119:0/4249588805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e0130e60 con 0x7f17e0144470 2024-04-25T10:13:19.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.653+0000 7f17df200640 1 -- 172.21.15.119:0/4249588805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e0131420 con 0x7f17e0144470 2024-04-25T10:13:19.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.654+0000 7f17df200640 1 -- 172.21.15.119:0/4249588805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e00749b0 con 0x7f17e0144470 2024-04-25T10:13:19.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.655+0000 7f17d6000640 1 -- 172.21.15.119:0/4249588805 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17d8018070 con 0x7f17e0144470 2024-04-25T10:13:19.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.657+0000 7f17d6000640 1 --2- 172.21.15.119:0/4249588805 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f17c006b4d0 0x7f17c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.657+0000 7f17d6000640 1 -- 172.21.15.119:0/4249588805 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f17d808a150 con 0x7f17e0144470 2024-04-25T10:13:19.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.657+0000 7f17dde00640 1 --2- 172.21.15.119:0/4249588805 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f17c006b4d0 0x7f17c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.657+0000 7f17dde00640 1 --2- 172.21.15.119:0/4249588805 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f17c006b4d0 0x7f17c006d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f17d0007fd0 tx=0x7f17d0007ce0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.661+0000 7f17d6000640 1 -- 172.21.15.119:0/4249588805 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17d8059280 con 0x7f17e0144470 2024-04-25T10:13:19.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.858+0000 7f17df200640 1 -- 172.21.15.119:0/4249588805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17e0144f90 con 0x7f17e0144470 2024-04-25T10:13:19.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.859+0000 7f17d6000640 1 -- 172.21.15.119:0/4249588805 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+12928 (secure 0 0 0) 0x7f17d8058b80 con 0x7f17e0144470 2024-04-25T10:13:19.864 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:19.865 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":77,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:13:18.937534+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:19.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.862+0000 7f17df200640 1 -- 172.21.15.119:0/4249588805 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f17c006b4d0 msgr2=0x7f17c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.862+0000 7f17df200640 1 --2- 172.21.15.119:0/4249588805 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f17c006b4d0 0x7f17c006d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f17d0007fd0 tx=0x7f17d0007ce0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.862+0000 7f17df200640 1 -- 172.21.15.119:0/4249588805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e0144470 msgr2=0x7f17e00adbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.862+0000 7f17df200640 1 --2- 172.21.15.119:0/4249588805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e0144470 0x7f17e00adbb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f17d800c7f0 tx=0x7f17d800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.863+0000 7f17df200640 1 -- 172.21.15.119:0/4249588805 shutdown_connections 2024-04-25T10:13:19.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.863+0000 7f17df200640 1 --2- 172.21.15.119:0/4249588805 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f17c006b4d0 0x7f17c006d990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.863+0000 7f17df200640 1 --2- 172.21.15.119:0/4249588805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17e0144470 0x7f17e00adbb0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.863+0000 7f17df200640 1 --2- 172.21.15.119:0/4249588805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17e00b5050 0x7f17e00abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.863+0000 7f17df200640 1 --2- 172.21.15.119:0/4249588805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f17e006e430 0x7f17e015b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.863+0000 7f17df200640 1 -- 172.21.15.119:0/4249588805 >> 172.21.15.119:0/4249588805 conn(0x7f17e006b1d0 msgr2=0x7f17e00af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.863+0000 7f17df200640 1 -- 172.21.15.119:0/4249588805 shutdown_connections 2024-04-25T10:13:19.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:19.863+0000 7f17df200640 1 -- 172.21.15.119:0/4249588805 wait complete. 2024-04-25T10:13:19.887 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:19.887 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:20.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.076+0000 7fdb38800640 1 Processor -- start 2024-04-25T10:13:20.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.076+0000 7fdb38800640 1 -- start start 2024-04-25T10:13:20.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.078+0000 7fdb38800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdb30144e70 0x7fdb3006d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.078+0000 7fdb38800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdb300cf0e0 0x7fdb300cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.078+0000 7fdb38800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb3006dcb0 con 0x7fdb3006d7a0 2024-04-25T10:13:20.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.078+0000 7fdb38800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3015bfe0 con 0x7fdb30144e70 2024-04-25T10:13:20.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.078+0000 7fdb38800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3006df90 con 0x7fdb300cf0e0 2024-04-25T10:13:20.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.078+0000 7fdb35e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdb30144e70 0x7fdb3006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.078+0000 7fdb35400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdb3006d7a0 0x7fdb300ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56306/0 (socket says 172.21.15.119:56306) 2024-04-25T10:13:20.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.078+0000 7fdb35400640 1 -- 172.21.15.119:0/3276799825 learned_addr learned my addr 172.21.15.119:0/3276799825 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:20.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.079+0000 7fdb36800640 1 --2- 172.21.15.119:0/3276799825 >> v2:172.21.15.132:3300/0 conn(0x7fdb300cf0e0 0x7fdb300cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.079+0000 7fdb34a00640 1 -- 172.21.15.119:0/3276799825 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610070597 0 0) 0x7fdb3006dcb0 con 0x7fdb3006d7a0 2024-04-25T10:13:20.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.079+0000 7fdb34a00640 1 -- 172.21.15.119:0/3276799825 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb18003520 con 0x7fdb3006d7a0 2024-04-25T10:13:20.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.079+0000 7fdb35e00640 1 -- 172.21.15.119:0/3276799825 >> v2:172.21.15.132:3300/0 conn(0x7fdb300cf0e0 msgr2=0x7fdb300cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.079+0000 7fdb35e00640 1 --2- 172.21.15.119:0/3276799825 >> v2:172.21.15.132:3300/0 conn(0x7fdb300cf0e0 0x7fdb300cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.079+0000 7fdb35e00640 1 -- 172.21.15.119:0/3276799825 >> v1:172.21.15.119:6789/0 conn(0x7fdb3006d7a0 legacy=0x7fdb300ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.079+0000 7fdb35e00640 1 -- 172.21.15.119:0/3276799825 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3006e270 con 0x7fdb30144e70 2024-04-25T10:13:20.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.080+0000 7fdb35e00640 1 --2- 172.21.15.119:0/3276799825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30144e70 0x7fdb3006d260 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdb301458f0 tx=0x7fdb2402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bc6db5be9c940e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.080+0000 7fdb34a00640 1 -- 172.21.15.119:0/3276799825 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24037070 con 0x7fdb30144e70 2024-04-25T10:13:20.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.080+0000 7fdb34a00640 1 -- 172.21.15.119:0/3276799825 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb24032440 con 0x7fdb30144e70 2024-04-25T10:13:20.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.080+0000 7fdb34a00640 1 -- 172.21.15.119:0/3276799825 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2402ae60 con 0x7fdb30144e70 2024-04-25T10:13:20.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.081+0000 7fdb38800640 1 -- 172.21.15.119:0/3276799825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30144e70 msgr2=0x7fdb3006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.081+0000 7fdb38800640 1 --2- 172.21.15.119:0/3276799825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30144e70 0x7fdb3006d260 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdb301458f0 tx=0x7fdb2402a110 comp rx=0 tx=0).stop 2024-04-25T10:13:20.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.081+0000 7fdb38800640 1 -- 172.21.15.119:0/3276799825 shutdown_connections 2024-04-25T10:13:20.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.081+0000 7fdb38800640 1 --2- 172.21.15.119:0/3276799825 >> v2:172.21.15.132:3300/0 conn(0x7fdb300cf0e0 0x7fdb300cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.081+0000 7fdb38800640 1 --2- 172.21.15.119:0/3276799825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30144e70 0x7fdb3006d260 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.081+0000 7fdb38800640 1 -- 172.21.15.119:0/3276799825 >> 172.21.15.119:0/3276799825 conn(0x7fdb3006aba0 msgr2=0x7fdb3006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.081+0000 7fdb38800640 1 -- 172.21.15.119:0/3276799825 shutdown_connections 2024-04-25T10:13:20.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.081+0000 7fdb38800640 1 -- 172.21.15.119:0/3276799825 wait complete. 2024-04-25T10:13:20.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.083+0000 7fdb38800640 1 Processor -- start 2024-04-25T10:13:20.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.083+0000 7fdb38800640 1 -- start start 2024-04-25T10:13:20.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.084+0000 7fdb38800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb3006d7a0 0x7fdb3013d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.084+0000 7fdb38800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb300cf0e0 0x7fdb301f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.084+0000 7fdb38800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb30144e70 0x7fdb301f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.084+0000 7fdb38800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb30145630 con 0x7fdb300cf0e0 2024-04-25T10:13:20.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.084+0000 7fdb38800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb301454b0 con 0x7fdb3006d7a0 2024-04-25T10:13:20.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.084+0000 7fdb38800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300cc840 con 0x7fdb30144e70 2024-04-25T10:13:20.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.084+0000 7fdb35400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb300cf0e0 0x7fdb301f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.084+0000 7fdb35400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb300cf0e0 0x7fdb301f4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35288/0 (socket says 172.21.15.119:35288) 2024-04-25T10:13:20.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.084+0000 7fdb35400640 1 -- 172.21.15.119:0/581833916 learned_addr learned my addr 172.21.15.119:0/581833916 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.085+0000 7fdb36800640 1 --2- 172.21.15.119:0/581833916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb30144e70 0x7fdb301f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.085+0000 7fdb35400640 1 -- 172.21.15.119:0/581833916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb30144e70 msgr2=0x7fdb301f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.085+0000 7fdb35400640 1 --2- 172.21.15.119:0/581833916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb30144e70 0x7fdb301f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.085+0000 7fdb35400640 1 -- 172.21.15.119:0/581833916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb3006d7a0 msgr2=0x7fdb3013d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.085+0000 7fdb35400640 1 --2- 172.21.15.119:0/581833916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb3006d7a0 0x7fdb3013d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.085+0000 7fdb35400640 1 -- 172.21.15.119:0/581833916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb300aa7b0 con 0x7fdb300cf0e0 2024-04-25T10:13:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.085+0000 7fdb35400640 1 --2- 172.21.15.119:0/581833916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb300cf0e0 0x7fdb301f4a40 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000c430 tx=0x7fdb2000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.085+0000 7fdb2a000640 1 -- 172.21.15.119:0/581833916 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20017070 con 0x7fdb300cf0e0 2024-04-25T10:13:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.085+0000 7fdb2a000640 1 -- 172.21.15.119:0/581833916 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb20005ce0 con 0x7fdb300cf0e0 2024-04-25T10:13:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.085+0000 7fdb2a000640 1 -- 172.21.15.119:0/581833916 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb200072f0 con 0x7fdb300cf0e0 2024-04-25T10:13:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.085+0000 7fdb38800640 1 -- 172.21.15.119:0/581833916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb300aaa90 con 0x7fdb300cf0e0 2024-04-25T10:13:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.086+0000 7fdb38800640 1 -- 172.21.15.119:0/581833916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb300ab050 con 0x7fdb300cf0e0 2024-04-25T10:13:20.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.087+0000 7fdb38800640 1 -- 172.21.15.119:0/581833916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb300750b0 con 0x7fdb300cf0e0 2024-04-25T10:13:20.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.088+0000 7fdb2a000640 1 -- 172.21.15.119:0/581833916 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb2001a070 con 0x7fdb300cf0e0 2024-04-25T10:13:20.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.089+0000 7fdb2a000640 1 --2- 172.21.15.119:0/581833916 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb1406b4d0 0x7fdb1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.089+0000 7fdb2a000640 1 -- 172.21.15.119:0/581833916 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdb200893a0 con 0x7fdb300cf0e0 2024-04-25T10:13:20.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.089+0000 7fdb35e00640 1 --2- 172.21.15.119:0/581833916 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb1406b4d0 0x7fdb1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.090+0000 7fdb35e00640 1 --2- 172.21.15.119:0/581833916 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb1406b4d0 0x7fdb1406d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdb24034000 tx=0x7fdb2402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.094+0000 7fdb2a000640 1 -- 172.21.15.119:0/581833916 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb20058560 con 0x7fdb300cf0e0 2024-04-25T10:13:20.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.289+0000 7fdb38800640 1 -- 172.21.15.119:0/581833916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb300ac650 con 0x7fdb300cf0e0 2024-04-25T10:13:20.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.291+0000 7fdb2a000640 1 -- 172.21.15.119:0/581833916 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb20057e60 con 0x7fdb300cf0e0 2024-04-25T10:13:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:13:20.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.294+0000 7fdb38800640 1 -- 172.21.15.119:0/581833916 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb1406b4d0 msgr2=0x7fdb1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.294+0000 7fdb38800640 1 --2- 172.21.15.119:0/581833916 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb1406b4d0 0x7fdb1406d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdb24034000 tx=0x7fdb2402c000 comp rx=0 tx=0).stop 2024-04-25T10:13:20.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.294+0000 7fdb38800640 1 -- 172.21.15.119:0/581833916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb300cf0e0 msgr2=0x7fdb301f4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.294+0000 7fdb38800640 1 --2- 172.21.15.119:0/581833916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb300cf0e0 0x7fdb301f4a40 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000c430 tx=0x7fdb2000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.294+0000 7fdb38800640 1 -- 172.21.15.119:0/581833916 shutdown_connections 2024-04-25T10:13:20.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.294+0000 7fdb38800640 1 --2- 172.21.15.119:0/581833916 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb1406b4d0 0x7fdb1406d990 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.294+0000 7fdb38800640 1 --2- 172.21.15.119:0/581833916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb30144e70 0x7fdb301f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.294+0000 7fdb38800640 1 --2- 172.21.15.119:0/581833916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb300cf0e0 0x7fdb301f4a40 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.294+0000 7fdb38800640 1 --2- 172.21.15.119:0/581833916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb3006d7a0 0x7fdb3013d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.294+0000 7fdb38800640 1 -- 172.21.15.119:0/581833916 >> 172.21.15.119:0/581833916 conn(0x7fdb3006aba0 msgr2=0x7fdb30144120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.294+0000 7fdb38800640 1 -- 172.21.15.119:0/581833916 shutdown_connections 2024-04-25T10:13:20.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.294+0000 7fdb38800640 1 -- 172.21.15.119:0/581833916 wait complete. 2024-04-25T10:13:20.317 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:13:20.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.515+0000 7f492e000640 1 Processor -- start 2024-04-25T10:13:20.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.515+0000 7f492e000640 1 -- start start 2024-04-25T10:13:20.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.517+0000 7f492e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f492815db10 0x7f49280cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.517+0000 7f492e000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f49280d0900 0x7f49280cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.517+0000 7f492e000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49280d1550 con 0x7f49280d25f0 2024-04-25T10:13:20.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.517+0000 7f492e000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49280d1830 con 0x7f49280d0900 2024-04-25T10:13:20.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.517+0000 7f492e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49280d1b10 con 0x7f492815db10 2024-04-25T10:13:20.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.517+0000 7f491fe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f49280d25f0 0x7f49280d01f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56318/0 (socket says 172.21.15.119:56318) 2024-04-25T10:13:20.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.517+0000 7f492d600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f49280d0900 0x7f49280cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.517+0000 7f491fe00640 1 -- 172.21.15.119:0/3877637496 learned_addr learned my addr 172.21.15.119:0/3877637496 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:20.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.517+0000 7f492cc00640 1 --2- 172.21.15.119:0/3877637496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f492815db10 0x7f49280cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.518+0000 7f4927e00640 1 -- 172.21.15.119:0/3877637496 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793190225 0 0) 0x7f49280d1550 con 0x7f49280d25f0 2024-04-25T10:13:20.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.518+0000 7f492d600640 1 -- 172.21.15.119:0/3877637496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f492815db10 msgr2=0x7f49280cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.518+0000 7f492d600640 1 --2- 172.21.15.119:0/3877637496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f492815db10 0x7f49280cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.518+0000 7f492d600640 1 -- 172.21.15.119:0/3877637496 >> v1:172.21.15.119:6789/0 conn(0x7f49280d25f0 legacy=0x7f49280d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.518+0000 7f492d600640 1 -- 172.21.15.119:0/3877637496 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49280d1df0 con 0x7f49280d0900 2024-04-25T10:13:20.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.519+0000 7f492d600640 1 --2- 172.21.15.119:0/3877637496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49280d0900 0x7f49280cf4e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f4920009d90 tx=0x7f492002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed8c3e0d6e629ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.519+0000 7f4927e00640 1 -- 172.21.15.119:0/3877637496 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920007740 con 0x7f49280d0900 2024-04-25T10:13:20.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.519+0000 7f4927e00640 1 -- 172.21.15.119:0/3877637496 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49200078e0 con 0x7f49280d0900 2024-04-25T10:13:20.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.519+0000 7f4927e00640 1 -- 172.21.15.119:0/3877637496 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920007cb0 con 0x7f49280d0900 2024-04-25T10:13:20.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.520+0000 7f492e000640 1 -- 172.21.15.119:0/3877637496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49280d0900 msgr2=0x7f49280cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.520+0000 7f492e000640 1 --2- 172.21.15.119:0/3877637496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49280d0900 0x7f49280cf4e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f4920009d90 tx=0x7f492002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.520+0000 7f492e000640 1 -- 172.21.15.119:0/3877637496 shutdown_connections 2024-04-25T10:13:20.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.520+0000 7f492e000640 1 --2- 172.21.15.119:0/3877637496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49280d0900 0x7f49280cf4e0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.520+0000 7f492e000640 1 --2- 172.21.15.119:0/3877637496 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f492815db10 0x7f49280cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.520+0000 7f492e000640 1 -- 172.21.15.119:0/3877637496 >> 172.21.15.119:0/3877637496 conn(0x7f492808afd0 msgr2=0x7f49280d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.520+0000 7f492e000640 1 -- 172.21.15.119:0/3877637496 shutdown_connections 2024-04-25T10:13:20.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.520+0000 7f492e000640 1 -- 172.21.15.119:0/3877637496 wait complete. 2024-04-25T10:13:20.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.522+0000 7f492e000640 1 Processor -- start 2024-04-25T10:13:20.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.522+0000 7f492e000640 1 -- start start 2024-04-25T10:13:20.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.522+0000 7f492e000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49280d0900 0x7f492815ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.523+0000 7f492e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49280d25f0 0x7f49280cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.523+0000 7f492e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f492815db10 0x7f49281f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.523+0000 7f492e000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492806bdf0 con 0x7f49280d25f0 2024-04-25T10:13:20.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.523+0000 7f492e000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492815def0 con 0x7f49280d0900 2024-04-25T10:13:20.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.523+0000 7f492e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492806bf70 con 0x7f492815db10 2024-04-25T10:13:20.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.523+0000 7f491fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49280d25f0 0x7f49280cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.523+0000 7f491fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49280d25f0 0x7f49280cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35304/0 (socket says 172.21.15.119:35304) 2024-04-25T10:13:20.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.523+0000 7f491fe00640 1 -- 172.21.15.119:0/2596228975 learned_addr learned my addr 172.21.15.119:0/2596228975 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:20.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.523+0000 7f491fe00640 1 -- 172.21.15.119:0/2596228975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f492815db10 msgr2=0x7f49281f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.523+0000 7f491fe00640 1 --2- 172.21.15.119:0/2596228975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f492815db10 0x7f49281f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.523+0000 7f491fe00640 1 -- 172.21.15.119:0/2596228975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49280d0900 msgr2=0x7f492815ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.523+0000 7f491fe00640 1 --2- 172.21.15.119:0/2596228975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49280d0900 0x7f492815ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.523+0000 7f491fe00640 1 -- 172.21.15.119:0/2596228975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49280cc9c0 con 0x7f49280d25f0 2024-04-25T10:13:20.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.523+0000 7f491fe00640 1 --2- 172.21.15.119:0/2596228975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49280d25f0 0x7f49280cb780 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f491000c4d0 tx=0x7f491000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.524+0000 7f4925600640 1 -- 172.21.15.119:0/2596228975 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910018070 con 0x7f49280d25f0 2024-04-25T10:13:20.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.524+0000 7f4925600640 1 -- 172.21.15.119:0/2596228975 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4910007500 con 0x7f49280d25f0 2024-04-25T10:13:20.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.524+0000 7f4925600640 1 -- 172.21.15.119:0/2596228975 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910004a20 con 0x7f49280d25f0 2024-04-25T10:13:20.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.524+0000 7f492e000640 1 -- 172.21.15.119:0/2596228975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49280ccca0 con 0x7f49280d25f0 2024-04-25T10:13:20.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.524+0000 7f492e000640 1 -- 172.21.15.119:0/2596228975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49280cd260 con 0x7f49280d25f0 2024-04-25T10:13:20.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.525+0000 7f492e000640 1 -- 172.21.15.119:0/2596228975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49280749b0 con 0x7f49280d25f0 2024-04-25T10:13:20.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.526+0000 7f4925600640 1 -- 172.21.15.119:0/2596228975 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f491001b070 con 0x7f49280d25f0 2024-04-25T10:13:20.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.527+0000 7f4925600640 1 --2- 172.21.15.119:0/2596228975 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f490406b4d0 0x7f490406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.527+0000 7f4925600640 1 -- 172.21.15.119:0/2596228975 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4910089290 con 0x7f49280d25f0 2024-04-25T10:13:20.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.527+0000 7f492cc00640 1 --2- 172.21.15.119:0/2596228975 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f490406b4d0 0x7f490406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.528+0000 7f492cc00640 1 --2- 172.21.15.119:0/2596228975 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f490406b4d0 0x7f490406d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4918006fd0 tx=0x7f4918008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.531+0000 7f4925600640 1 -- 172.21.15.119:0/2596228975 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49100583c0 con 0x7f49280d25f0 2024-04-25T10:13:20.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.724+0000 7f492e000640 1 -- 172.21.15.119:0/2596228975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f49280d1160 con 0x7f49280d25f0 2024-04-25T10:13:20.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.725+0000 7f4925600640 1 -- 172.21.15.119:0/2596228975 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7f491005e0c0 con 0x7f49280d25f0 2024-04-25T10:13:20.730 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:20.730 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2024-04-25T10:13:20.730 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:13:20.731 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2024-04-25T10:13:20.731 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2024-04-25T10:13:20.731 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:20.731 INFO:teuthology.orchestra.run.smithi119.stdout:] 2024-04-25T10:13:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.728+0000 7f492e000640 1 -- 172.21.15.119:0/2596228975 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f490406b4d0 msgr2=0x7f490406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.728+0000 7f492e000640 1 --2- 172.21.15.119:0/2596228975 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f490406b4d0 0x7f490406d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4918006fd0 tx=0x7f4918008040 comp rx=0 tx=0).stop 2024-04-25T10:13:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.728+0000 7f492e000640 1 -- 172.21.15.119:0/2596228975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49280d25f0 msgr2=0x7f49280cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.728+0000 7f492e000640 1 --2- 172.21.15.119:0/2596228975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49280d25f0 0x7f49280cb780 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f491000c4d0 tx=0x7f491000c990 comp rx=0 tx=0).stop 2024-04-25T10:13:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.728+0000 7f492e000640 1 -- 172.21.15.119:0/2596228975 shutdown_connections 2024-04-25T10:13:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.728+0000 7f492e000640 1 --2- 172.21.15.119:0/2596228975 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f490406b4d0 0x7f490406d990 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.728+0000 7f492e000640 1 --2- 172.21.15.119:0/2596228975 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f492815db10 0x7f49281f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.728+0000 7f492e000640 1 --2- 172.21.15.119:0/2596228975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49280d25f0 0x7f49280cb780 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.728+0000 7f492e000640 1 --2- 172.21.15.119:0/2596228975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49280d0900 0x7f492815ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.728+0000 7f492e000640 1 -- 172.21.15.119:0/2596228975 >> 172.21.15.119:0/2596228975 conn(0x7f492808afd0 msgr2=0x7f49280b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.728+0000 7f492e000640 1 -- 172.21.15.119:0/2596228975 shutdown_connections 2024-04-25T10:13:20.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.728+0000 7f492e000640 1 -- 172.21.15.119:0/2596228975 wait complete. 2024-04-25T10:13:20.751 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:13:20.751 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:13:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.944+0000 7fee1aa00640 1 Processor -- start 2024-04-25T10:13:20.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.944+0000 7fee1aa00640 1 -- start start 2024-04-25T10:13:20.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.946+0000 7fee1aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee140b9720 0x7fee140ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.946+0000 7fee1aa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fee140cece0 0x7fee140d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.946+0000 7fee1aa00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee14161f80 con 0x7fee14161090 2024-04-25T10:13:20.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.946+0000 7fee1aa00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee140cf130 con 0x7fee140cece0 2024-04-25T10:13:20.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.946+0000 7fee1aa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee140d21a0 con 0x7fee140b9720 2024-04-25T10:13:20.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.946+0000 7fee13e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fee14161090 0x7fee14161730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48426/0 (socket says 172.21.15.119:48426) 2024-04-25T10:13:20.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.946+0000 7fee13e00640 1 -- 172.21.15.119:0/663155088 learned_addr learned my addr 172.21.15.119:0/663155088 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:20.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee18a00640 1 --2- 172.21.15.119:0/663155088 >> v2:172.21.15.132:3300/0 conn(0x7fee140cece0 0x7fee140d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee0be00640 1 --2- 172.21.15.119:0/663155088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee140b9720 0x7fee140ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 -- 172.21.15.119:0/663155088 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238432662 0 0) 0x7fee14161f80 con 0x7fee14161090 2024-04-25T10:13:20.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 -- 172.21.15.119:0/663155088 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedf8003520 con 0x7fee14161090 2024-04-25T10:13:20.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 -- 172.21.15.119:0/663155088 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2969931941 0 0) 0x7fedf8003520 con 0x7fee14161090 2024-04-25T10:13:20.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 -- 172.21.15.119:0/663155088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee140b9720 msgr2=0x7fee140ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 --2- 172.21.15.119:0/663155088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee140b9720 0x7fee140ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 -- 172.21.15.119:0/663155088 >> v2:172.21.15.132:3300/0 conn(0x7fee140cece0 msgr2=0x7fee140d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 --2- 172.21.15.119:0/663155088 >> v2:172.21.15.132:3300/0 conn(0x7fee140cece0 0x7fee140d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 -- 172.21.15.119:0/663155088 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee14162260 con 0x7fee14161090 2024-04-25T10:13:20.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 -- 172.21.15.119:0/663155088 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fee04001670 con 0x7fee14161090 2024-04-25T10:13:20.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 -- 172.21.15.119:0/663155088 >> v1:172.21.15.119:6789/0 conn(0x7fee14161090 legacy=0x7fee14161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 --2- 172.21.15.119:0/663155088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee140b9720 0x7fedf80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 --2- 172.21.15.119:0/663155088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedf8003f80 0x7fedf8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 --2- 172.21.15.119:0/663155088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf8004d40 0x7fedf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 -- 172.21.15.119:0/663155088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee140cf130 con 0x7fedf8004d40 2024-04-25T10:13:20.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 -- 172.21.15.119:0/663155088 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee140d21a0 con 0x7fedf8003f80 2024-04-25T10:13:20.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.947+0000 7fee13400640 1 -- 172.21.15.119:0/663155088 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf800b9c0 con 0x7fee140b9720 2024-04-25T10:13:20.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.948+0000 7fee18a00640 1 --2- 172.21.15.119:0/663155088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf8004d40 0x7fedf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.948+0000 7fee13e00640 1 --2- 172.21.15.119:0/663155088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee140b9720 0x7fedf80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.948+0000 7fee0be00640 1 --2- 172.21.15.119:0/663155088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedf8003f80 0x7fedf8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.948+0000 7fee18a00640 1 -- 172.21.15.119:0/663155088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee140b9720 msgr2=0x7fedf80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.948+0000 7fee18a00640 1 --2- 172.21.15.119:0/663155088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fee140b9720 0x7fedf80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.948+0000 7fee18a00640 1 -- 172.21.15.119:0/663155088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedf8003f80 msgr2=0x7fedf8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.948+0000 7fee18a00640 1 --2- 172.21.15.119:0/663155088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedf8003f80 0x7fedf8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.948+0000 7fee18a00640 1 -- 172.21.15.119:0/663155088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf8003520 con 0x7fedf8004d40 2024-04-25T10:13:20.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.948+0000 7fee0be00640 1 --2- 172.21.15.119:0/663155088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedf8003f80 0x7fedf8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:20.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.949+0000 7fee18a00640 1 --2- 172.21.15.119:0/663155088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf8004d40 0x7fedf80051c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00e170 tx=0x7fedf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d8d844055aa0fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.949+0000 7fee13400640 1 -- 172.21.15.119:0/663155088 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c034020 con 0x7fedf8004d40 2024-04-25T10:13:20.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.949+0000 7fee13400640 1 -- 172.21.15.119:0/663155088 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c030430 con 0x7fedf8004d40 2024-04-25T10:13:20.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.949+0000 7fee13400640 1 -- 172.21.15.119:0/663155088 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c009c10 con 0x7fedf8004d40 2024-04-25T10:13:20.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.950+0000 7fee1aa00640 1 -- 172.21.15.119:0/663155088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf8004d40 msgr2=0x7fedf80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.950+0000 7fee1aa00640 1 --2- 172.21.15.119:0/663155088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf8004d40 0x7fedf80051c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00e170 tx=0x7fedf8005830 comp rx=0 tx=0).stop 2024-04-25T10:13:20.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.950+0000 7fee18a00640 1 -- 172.21.15.119:0/663155088 reap_dead start 2024-04-25T10:13:20.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.950+0000 7fee1aa00640 1 -- 172.21.15.119:0/663155088 shutdown_connections 2024-04-25T10:13:20.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.950+0000 7fee1aa00640 1 -- 172.21.15.119:0/663155088 >> 172.21.15.119:0/663155088 conn(0x7fee140b55e0 msgr2=0x7fee140b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.950+0000 7fee1aa00640 1 -- 172.21.15.119:0/663155088 shutdown_connections 2024-04-25T10:13:20.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.950+0000 7fee1aa00640 1 -- 172.21.15.119:0/663155088 wait complete. 2024-04-25T10:13:20.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.952+0000 7fee1aa00640 1 Processor -- start 2024-04-25T10:13:20.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.952+0000 7fee1aa00640 1 -- start start 2024-04-25T10:13:20.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.953+0000 7fee1aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fedf8004d40 0x7fee140b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.953+0000 7fee1aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee1415d6c0 0x7fee1415f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.953+0000 7fee1aa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee141f6540 0x7fee141f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.953+0000 7fee1aa00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee14161470 con 0x7fee1415d6c0 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.953+0000 7fee1aa00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee140d1e30 con 0x7fee141f6540 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.953+0000 7fee1aa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee140ce7a0 con 0x7fedf8004d40 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.953+0000 7fee0be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee1415d6c0 0x7fee1415f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.953+0000 7fee0be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee1415d6c0 0x7fee1415f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52548/0 (socket says 172.21.15.119:52548) 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.953+0000 7fee0be00640 1 -- 172.21.15.119:0/333910099 learned_addr learned my addr 172.21.15.119:0/333910099 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.954+0000 7fee13e00640 1 --2- 172.21.15.119:0/333910099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fedf8004d40 0x7fee140b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.954+0000 7fee18a00640 1 --2- 172.21.15.119:0/333910099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee141f6540 0x7fee141f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.954+0000 7fee0be00640 1 -- 172.21.15.119:0/333910099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fedf8004d40 msgr2=0x7fee140b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.954+0000 7fee0be00640 1 --2- 172.21.15.119:0/333910099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fedf8004d40 0x7fee140b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.954+0000 7fee0be00640 1 -- 172.21.15.119:0/333910099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee141f6540 msgr2=0x7fee141f69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.954+0000 7fee0be00640 1 --2- 172.21.15.119:0/333910099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee141f6540 0x7fee141f69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.954+0000 7fee0be00640 1 -- 172.21.15.119:0/333910099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1415dbc0 con 0x7fee1415d6c0 2024-04-25T10:13:20.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.954+0000 7fee0be00640 1 --2- 172.21.15.119:0/333910099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee1415d6c0 0x7fee1415f550 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fedfc011540 tx=0x7fedfc011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.954+0000 7fee10c00640 1 -- 172.21.15.119:0/333910099 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc015870 con 0x7fee1415d6c0 2024-04-25T10:13:20.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.954+0000 7fee10c00640 1 -- 172.21.15.119:0/333910099 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedfc015a10 con 0x7fee1415d6c0 2024-04-25T10:13:20.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.954+0000 7fee10c00640 1 -- 172.21.15.119:0/333910099 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc015dc0 con 0x7fee1415d6c0 2024-04-25T10:13:20.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.954+0000 7fee1aa00640 1 -- 172.21.15.119:0/333910099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1415dea0 con 0x7fee1415d6c0 2024-04-25T10:13:20.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.954+0000 7fee1aa00640 1 -- 172.21.15.119:0/333910099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1415e460 con 0x7fee1415d6c0 2024-04-25T10:13:20.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.956+0000 7fee1aa00640 1 -- 172.21.15.119:0/333910099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee140750b0 con 0x7fee1415d6c0 2024-04-25T10:13:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.957+0000 7fee10c00640 1 -- 172.21.15.119:0/333910099 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fedfc00f460 con 0x7fee1415d6c0 2024-04-25T10:13:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.957+0000 7fee10c00640 1 --2- 172.21.15.119:0/333910099 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fedf006b4d0 0x7fedf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.958+0000 7fee10c00640 1 -- 172.21.15.119:0/333910099 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fedfc08a880 con 0x7fee1415d6c0 2024-04-25T10:13:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.958+0000 7fee13e00640 1 --2- 172.21.15.119:0/333910099 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fedf006b4d0 0x7fedf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.958+0000 7fee13e00640 1 --2- 172.21.15.119:0/333910099 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fedf006b4d0 0x7fedf006d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fee04001eb0 tx=0x7fee04000f50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:20.962+0000 7fee10c00640 1 -- 172.21.15.119:0/333910099 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fedfc019030 con 0x7fee1415d6c0 2024-04-25T10:13:21.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:21.156+0000 7fee1aa00640 1 -- 172.21.15.119:0/333910099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fee1406cf90 con 0x7fee1415d6c0 2024-04-25T10:13:21.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:21.952+0000 7fee10c00640 1 -- 172.21.15.119:0/333910099 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7fedfc059920 con 0x7fee1415d6c0 2024-04-25T10:13:21.957 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2024-04-25T10:13:21.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:21.955+0000 7fee1aa00640 1 -- 172.21.15.119:0/333910099 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fedf006b4d0 msgr2=0x7fedf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:21.955+0000 7fee1aa00640 1 --2- 172.21.15.119:0/333910099 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fedf006b4d0 0x7fedf006d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fee04001eb0 tx=0x7fee04000f50 comp rx=0 tx=0).stop 2024-04-25T10:13:21.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:21.955+0000 7fee1aa00640 1 -- 172.21.15.119:0/333910099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee1415d6c0 msgr2=0x7fee1415f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:21.955+0000 7fee1aa00640 1 --2- 172.21.15.119:0/333910099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee1415d6c0 0x7fee1415f550 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fedfc011540 tx=0x7fedfc011a00 comp rx=0 tx=0).stop 2024-04-25T10:13:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:21.956+0000 7fee1aa00640 1 -- 172.21.15.119:0/333910099 shutdown_connections 2024-04-25T10:13:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:21.956+0000 7fee1aa00640 1 --2- 172.21.15.119:0/333910099 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fedf006b4d0 0x7fedf006d990 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:21.956+0000 7fee1aa00640 1 --2- 172.21.15.119:0/333910099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee141f6540 0x7fee141f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:21.956+0000 7fee1aa00640 1 --2- 172.21.15.119:0/333910099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee1415d6c0 0x7fee1415f550 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:21.956+0000 7fee1aa00640 1 --2- 172.21.15.119:0/333910099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fedf8004d40 0x7fee140b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:21.956+0000 7fee1aa00640 1 -- 172.21.15.119:0/333910099 >> 172.21.15.119:0/333910099 conn(0x7fee140b55e0 msgr2=0x7fee140b83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:21.956+0000 7fee1aa00640 1 -- 172.21.15.119:0/333910099 shutdown_connections 2024-04-25T10:13:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:21.956+0000 7fee1aa00640 1 -- 172.21.15.119:0/333910099 wait complete. 2024-04-25T10:13:21.988 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:13:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.175+0000 7f4d64c00640 1 Processor -- start 2024-04-25T10:13:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.176+0000 7f4d64c00640 1 -- start start 2024-04-25T10:13:22.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.177+0000 7f4d64c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4d6015daf0 0x7f4d600b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.177+0000 7f4d64c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4d600b2c90 0x7f4d600d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.177+0000 7f4d64c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d600b6ab0 0x7f4d600d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.177+0000 7f4d64c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d600d17b0 con 0x7f4d600b2c90 2024-04-25T10:13:22.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.177+0000 7f4d64c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d600ce760 con 0x7f4d6015daf0 2024-04-25T10:13:22.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.177+0000 7f4d64c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d600cea40 con 0x7f4d600b6ab0 2024-04-25T10:13:22.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.178+0000 7f4d57e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4d600b2c90 0x7f4d600d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.178+0000 7f4d57e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4d600b2c90 0x7f4d600d0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52562/0 (socket says 172.21.15.119:52562) 2024-04-25T10:13:22.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.178+0000 7f4d57e00640 1 -- 172.21.15.119:0/3663227672 learned_addr learned my addr 172.21.15.119:0/3663227672 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:22.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.178+0000 7f4d5f400640 1 --2- 172.21.15.119:0/3663227672 >> v2:172.21.15.132:3300/0 conn(0x7f4d6015daf0 0x7f4d600b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.178+0000 7f4d5fe00640 1 --2- 172.21.15.119:0/3663227672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d600b6ab0 0x7f4d600d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.178+0000 7f4d57e00640 1 -- 172.21.15.119:0/3663227672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d600b6ab0 msgr2=0x7f4d600d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.178+0000 7f4d57e00640 1 --2- 172.21.15.119:0/3663227672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d600b6ab0 0x7f4d600d09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.178+0000 7f4d57e00640 1 -- 172.21.15.119:0/3663227672 >> v2:172.21.15.132:3300/0 conn(0x7f4d6015daf0 msgr2=0x7f4d600b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.178+0000 7f4d57e00640 1 --2- 172.21.15.119:0/3663227672 >> v2:172.21.15.132:3300/0 conn(0x7f4d6015daf0 0x7f4d600b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.178+0000 7f4d57e00640 1 -- 172.21.15.119:0/3663227672 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d600ced20 con 0x7f4d600b2c90 2024-04-25T10:13:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.178+0000 7f4d5fe00640 1 --2- 172.21.15.119:0/3663227672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d600b6ab0 0x7f4d600d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.179+0000 7f4d57e00640 1 --2- 172.21.15.119:0/3663227672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d600b2c90 0x7f4d600d0470 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4d600b6580 tx=0x7f4d4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed91ab7e3703f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.180+0000 7f4d5ea00640 1 -- 172.21.15.119:0/3663227672 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48037070 con 0x7f4d600b2c90 2024-04-25T10:13:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.180+0000 7f4d5ea00640 1 -- 172.21.15.119:0/3663227672 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d48032440 con 0x7f4d600b2c90 2024-04-25T10:13:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.180+0000 7f4d5ea00640 1 -- 172.21.15.119:0/3663227672 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48032810 con 0x7f4d600b2c90 2024-04-25T10:13:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.180+0000 7f4d64c00640 1 -- 172.21.15.119:0/3663227672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d600b2c90 msgr2=0x7f4d600d0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.180+0000 7f4d64c00640 1 --2- 172.21.15.119:0/3663227672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d600b2c90 0x7f4d600d0470 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4d600b6580 tx=0x7f4d4802a270 comp rx=0 tx=0).stop 2024-04-25T10:13:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.180+0000 7f4d64c00640 1 -- 172.21.15.119:0/3663227672 shutdown_connections 2024-04-25T10:13:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.180+0000 7f4d64c00640 1 --2- 172.21.15.119:0/3663227672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d600b6ab0 0x7f4d600d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.180+0000 7f4d64c00640 1 --2- 172.21.15.119:0/3663227672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d600b2c90 0x7f4d600d0470 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.181+0000 7f4d64c00640 1 --2- 172.21.15.119:0/3663227672 >> v2:172.21.15.132:3300/0 conn(0x7f4d6015daf0 0x7f4d600b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.181+0000 7f4d64c00640 1 -- 172.21.15.119:0/3663227672 >> 172.21.15.119:0/3663227672 conn(0x7f4d600b55c0 msgr2=0x7f4d600b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.181+0000 7f4d64c00640 1 -- 172.21.15.119:0/3663227672 shutdown_connections 2024-04-25T10:13:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.181+0000 7f4d64c00640 1 -- 172.21.15.119:0/3663227672 wait complete. 2024-04-25T10:13:22.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.182+0000 7f4d64c00640 1 Processor -- start 2024-04-25T10:13:22.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.183+0000 7f4d64c00640 1 -- start start 2024-04-25T10:13:22.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.183+0000 7f4d64c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d600b6ab0 0x7f4d600b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.183+0000 7f4d64c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d6015daf0 0x7f4d60153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.183+0000 7f4d64c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d60154080 0x7f4d6015e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.183+0000 7f4d64c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d600ce310 con 0x7f4d6015daf0 2024-04-25T10:13:22.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.183+0000 7f4d64c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d600cf000 con 0x7f4d60154080 2024-04-25T10:13:22.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.183+0000 7f4d64c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d600ce490 con 0x7f4d600b6ab0 2024-04-25T10:13:22.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.184+0000 7f4d57e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d6015daf0 0x7f4d60153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.184+0000 7f4d57e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d6015daf0 0x7f4d60153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52572/0 (socket says 172.21.15.119:52572) 2024-04-25T10:13:22.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.184+0000 7f4d57e00640 1 -- 172.21.15.119:0/661548731 learned_addr learned my addr 172.21.15.119:0/661548731 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:22.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.184+0000 7f4d57e00640 1 -- 172.21.15.119:0/661548731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d600b6ab0 msgr2=0x7f4d600b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.184+0000 7f4d57e00640 1 --2- 172.21.15.119:0/661548731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d600b6ab0 0x7f4d600b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.184+0000 7f4d57e00640 1 -- 172.21.15.119:0/661548731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d60154080 msgr2=0x7f4d6015e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.184+0000 7f4d5fe00640 1 --2- 172.21.15.119:0/661548731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d60154080 0x7f4d6015e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.184+0000 7f4d57e00640 1 --2- 172.21.15.119:0/661548731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d60154080 0x7f4d6015e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.184+0000 7f4d57e00640 1 -- 172.21.15.119:0/661548731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6015edd0 con 0x7f4d6015daf0 2024-04-25T10:13:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.185+0000 7f4d57e00640 1 --2- 172.21.15.119:0/661548731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d6015daf0 0x7f4d60153b40 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4d4802a2b0 tx=0x7f4d480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.185+0000 7f4d57400640 1 -- 172.21.15.119:0/661548731 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48041020 con 0x7f4d6015daf0 2024-04-25T10:13:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.185+0000 7f4d57400640 1 -- 172.21.15.119:0/661548731 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d480306c0 con 0x7f4d6015daf0 2024-04-25T10:13:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.185+0000 7f4d57400640 1 -- 172.21.15.119:0/661548731 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48037040 con 0x7f4d6015daf0 2024-04-25T10:13:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.185+0000 7f4d64c00640 1 -- 172.21.15.119:0/661548731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6015f0b0 con 0x7f4d6015daf0 2024-04-25T10:13:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.185+0000 7f4d64c00640 1 -- 172.21.15.119:0/661548731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6015f670 con 0x7f4d6015daf0 2024-04-25T10:13:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.186+0000 7f4d64c00640 1 -- 172.21.15.119:0/661548731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d600749b0 con 0x7f4d6015daf0 2024-04-25T10:13:22.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.187+0000 7f4d57400640 1 -- 172.21.15.119:0/661548731 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d48044020 con 0x7f4d6015daf0 2024-04-25T10:13:22.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.188+0000 7f4d57400640 1 --2- 172.21.15.119:0/661548731 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d3c06b500 0x7f4d3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.188+0000 7f4d57400640 1 -- 172.21.15.119:0/661548731 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4d480aa950 con 0x7f4d6015daf0 2024-04-25T10:13:22.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.188+0000 7f4d5f400640 1 --2- 172.21.15.119:0/661548731 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d3c06b500 0x7f4d3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.189+0000 7f4d5f400640 1 --2- 172.21.15.119:0/661548731 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d3c06b500 0x7f4d3c06d9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000b7f0 tx=0x7f4d50008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.193+0000 7f4d57400640 1 -- 172.21.15.119:0/661548731 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d4804b030 con 0x7f4d6015daf0 2024-04-25T10:13:22.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.387+0000 7f4d64c00640 1 -- 172.21.15.119:0/661548731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4d6015b2f0 con 0x7f4d6015daf0 2024-04-25T10:13:22.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.966+0000 7f4d57400640 1 -- 172.21.15.119:0/661548731 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4d48079880 con 0x7f4d6015daf0 2024-04-25T10:13:22.972 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2024-04-25T10:13:22.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.970+0000 7f4d64c00640 1 -- 172.21.15.119:0/661548731 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d3c06b500 msgr2=0x7f4d3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.970+0000 7f4d64c00640 1 --2- 172.21.15.119:0/661548731 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d3c06b500 0x7f4d3c06d9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000b7f0 tx=0x7f4d50008040 comp rx=0 tx=0).stop 2024-04-25T10:13:22.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.970+0000 7f4d64c00640 1 -- 172.21.15.119:0/661548731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d6015daf0 msgr2=0x7f4d60153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.970+0000 7f4d64c00640 1 --2- 172.21.15.119:0/661548731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d6015daf0 0x7f4d60153b40 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4d4802a2b0 tx=0x7f4d480046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.971+0000 7f4d64c00640 1 -- 172.21.15.119:0/661548731 shutdown_connections 2024-04-25T10:13:22.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.971+0000 7f4d64c00640 1 --2- 172.21.15.119:0/661548731 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d3c06b500 0x7f4d3c06d9c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.971+0000 7f4d64c00640 1 --2- 172.21.15.119:0/661548731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d60154080 0x7f4d6015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.971+0000 7f4d64c00640 1 --2- 172.21.15.119:0/661548731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d6015daf0 0x7f4d60153b40 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.971+0000 7f4d64c00640 1 --2- 172.21.15.119:0/661548731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d600b6ab0 0x7f4d600b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.971+0000 7f4d64c00640 1 -- 172.21.15.119:0/661548731 >> 172.21.15.119:0/661548731 conn(0x7f4d600b55c0 msgr2=0x7f4d60159510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.971+0000 7f4d64c00640 1 -- 172.21.15.119:0/661548731 shutdown_connections 2024-04-25T10:13:22.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:22.971+0000 7f4d64c00640 1 -- 172.21.15.119:0/661548731 wait complete. 2024-04-25T10:13:23.008 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:13:23.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.179+0000 7f19bca00640 1 Processor -- start 2024-04-25T10:13:23.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.179+0000 7f19bca00640 1 -- start start 2024-04-25T10:13:23.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.181+0000 7f19bca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19b80ce9d0 0x7f19b80cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.181+0000 7f19bca00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b806dcb0 con 0x7f19b8144e70 2024-04-25T10:13:23.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.181+0000 7f19bca00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b806df90 con 0x7f19b80cc300 2024-04-25T10:13:23.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.181+0000 7f19bca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b815bfe0 con 0x7f19b80ce9d0 2024-04-25T10:13:23.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.181+0000 7f19b5e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f19b8144e70 0x7f19b806d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48442/0 (socket says 172.21.15.119:48442) 2024-04-25T10:13:23.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.181+0000 7f19b5e00640 1 -- 172.21.15.119:0/521233019 learned_addr learned my addr 172.21.15.119:0/521233019 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.182+0000 7f19b5400640 1 --2- 172.21.15.119:0/521233019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19b80ce9d0 0x7f19b80cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.182+0000 7f19b4a00640 1 -- 172.21.15.119:0/521233019 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1324124599 0 0) 0x7f19b806dcb0 con 0x7f19b8144e70 2024-04-25T10:13:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.182+0000 7f19b4a00640 1 -- 172.21.15.119:0/521233019 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f199c003520 con 0x7f19b8144e70 2024-04-25T10:13:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.182+0000 7f19b4a00640 1 -- 172.21.15.119:0/521233019 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 23956330 0 0) 0x7f199c003520 con 0x7f19b8144e70 2024-04-25T10:13:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.182+0000 7f19b4a00640 1 -- 172.21.15.119:0/521233019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19b80ce9d0 msgr2=0x7f19b80cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.182+0000 7f19b4a00640 1 --2- 172.21.15.119:0/521233019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19b80ce9d0 0x7f19b80cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.182+0000 7f19b4a00640 1 -- 172.21.15.119:0/521233019 >> v1:172.21.15.132:6789/0 conn(0x7f19b80cc300 legacy=0x7f19b80cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.182+0000 7f19b4a00640 1 -- 172.21.15.119:0/521233019 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b806e270 con 0x7f19b8144e70 2024-04-25T10:13:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.182+0000 7f19b4a00640 1 -- 172.21.15.119:0/521233019 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f19a8001690 con 0x7f19b8144e70 2024-04-25T10:13:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.182+0000 7f19b4a00640 1 -- 172.21.15.119:0/521233019 >> v1:172.21.15.119:6789/0 conn(0x7f19b8144e70 legacy=0x7f19b806d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.182+0000 7f19b4a00640 1 --2- 172.21.15.119:0/521233019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19b80ce9d0 0x7f199c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.182+0000 7f19b4a00640 1 --2- 172.21.15.119:0/521233019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f199c003f60 0x7f199c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.182+0000 7f19b4a00640 1 --2- 172.21.15.119:0/521233019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f199c004d40 0x7f199c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.183+0000 7f19b4a00640 1 -- 172.21.15.119:0/521233019 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b815bfe0 con 0x7f199c003f60 2024-04-25T10:13:23.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.183+0000 7f19b4a00640 1 -- 172.21.15.119:0/521233019 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c00bb50 con 0x7f199c004d40 2024-04-25T10:13:23.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.183+0000 7f19b4a00640 1 -- 172.21.15.119:0/521233019 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c00bd50 con 0x7f19b80ce9d0 2024-04-25T10:13:23.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.183+0000 7f19b5400640 1 --2- 172.21.15.119:0/521233019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f199c003f60 0x7f199c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.183+0000 7f19b5e00640 1 --2- 172.21.15.119:0/521233019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19b80ce9d0 0x7f199c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.183+0000 7f19b6800640 1 --2- 172.21.15.119:0/521233019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f199c004d40 0x7f199c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.183+0000 7f19b5400640 1 -- 172.21.15.119:0/521233019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19b80ce9d0 msgr2=0x7f199c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.183+0000 7f19b5400640 1 --2- 172.21.15.119:0/521233019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19b80ce9d0 0x7f199c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.183+0000 7f19b5400640 1 -- 172.21.15.119:0/521233019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f199c004d40 msgr2=0x7f199c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.183+0000 7f19b5400640 1 --2- 172.21.15.119:0/521233019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f199c004d40 0x7f199c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.183+0000 7f19b5400640 1 -- 172.21.15.119:0/521233019 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c003520 con 0x7f199c003f60 2024-04-25T10:13:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.183+0000 7f19b6800640 1 --2- 172.21.15.119:0/521233019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f199c004d40 0x7f199c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.183+0000 7f19b5e00640 1 --2- 172.21.15.119:0/521233019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f19b80ce9d0 0x7f199c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:23.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.184+0000 7f19b5400640 1 --2- 172.21.15.119:0/521233019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f199c003f60 0x7f199c004800 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009ac0 tx=0x7f199c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7891b9896c4c337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:23.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.185+0000 7f19b4a00640 1 -- 172.21.15.119:0/521233019 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4036070 con 0x7f199c003f60 2024-04-25T10:13:23.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.185+0000 7f19b4a00640 1 -- 172.21.15.119:0/521233019 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a4031400 con 0x7f199c003f60 2024-04-25T10:13:23.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.185+0000 7f19b4a00640 1 -- 172.21.15.119:0/521233019 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4009690 con 0x7f199c003f60 2024-04-25T10:13:23.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.185+0000 7f19bca00640 1 -- 172.21.15.119:0/521233019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f199c003f60 msgr2=0x7f199c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.185+0000 7f19bca00640 1 --2- 172.21.15.119:0/521233019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f199c003f60 0x7f199c004800 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009ac0 tx=0x7f199c005830 comp rx=0 tx=0).stop 2024-04-25T10:13:23.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.185+0000 7f19b6800640 1 -- 172.21.15.119:0/521233019 reap_dead start 2024-04-25T10:13:23.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.185+0000 7f19bca00640 1 -- 172.21.15.119:0/521233019 shutdown_connections 2024-04-25T10:13:23.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.185+0000 7f19bca00640 1 -- 172.21.15.119:0/521233019 >> 172.21.15.119:0/521233019 conn(0x7f19b806aba0 msgr2=0x7f19b806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:23.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.185+0000 7f19bca00640 1 -- 172.21.15.119:0/521233019 shutdown_connections 2024-04-25T10:13:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.186+0000 7f19bca00640 1 -- 172.21.15.119:0/521233019 wait complete. 2024-04-25T10:13:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.187+0000 7f19bca00640 1 Processor -- start 2024-04-25T10:13:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.187+0000 7f19bca00640 1 -- start start 2024-04-25T10:13:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.188+0000 7f19bca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f199c003f60 0x7f19b813c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.188+0000 7f19bca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19b80b28e0 0x7f19b80b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.188+0000 7f19bca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19b813c970 0x7f19b813cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.188+0000 7f19bca00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b81454b0 con 0x7f19b80b28e0 2024-04-25T10:13:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.188+0000 7f19bca00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b806d970 con 0x7f19b813c970 2024-04-25T10:13:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.188+0000 7f19bca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b8145630 con 0x7f199c003f60 2024-04-25T10:13:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.188+0000 7f19b5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19b80b28e0 0x7f19b80b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.189+0000 7f19b5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19b80b28e0 0x7f19b80b2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52596/0 (socket says 172.21.15.119:52596) 2024-04-25T10:13:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.189+0000 7f19b5400640 1 -- 172.21.15.119:0/3611445511 learned_addr learned my addr 172.21.15.119:0/3611445511 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.189+0000 7f19b5e00640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f199c003f60 0x7f19b813c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.189+0000 7f19b6800640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19b813c970 0x7f19b813cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.189+0000 7f19b5400640 1 -- 172.21.15.119:0/3611445511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f199c003f60 msgr2=0x7f19b813c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.189+0000 7f19b5400640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f199c003f60 0x7f19b813c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.189+0000 7f19b5400640 1 -- 172.21.15.119:0/3611445511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19b813c970 msgr2=0x7f19b813cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.189+0000 7f19b5400640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19b813c970 0x7f19b813cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.189+0000 7f19b5400640 1 -- 172.21.15.119:0/3611445511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b80b3640 con 0x7f19b80b28e0 2024-04-25T10:13:23.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.189+0000 7f19b6800640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19b813c970 0x7f19b813cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:23.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.189+0000 7f19b5400640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19b80b28e0 0x7f19b80b2d60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009be0 tx=0x7f19a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:23.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.190+0000 7f19ae000640 1 -- 172.21.15.119:0/3611445511 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4040020 con 0x7f19b80b28e0 2024-04-25T10:13:23.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.190+0000 7f19ae000640 1 -- 172.21.15.119:0/3611445511 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a4003bc0 con 0x7f19b80b28e0 2024-04-25T10:13:23.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.190+0000 7f19ae000640 1 -- 172.21.15.119:0/3611445511 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4036040 con 0x7f19b80b28e0 2024-04-25T10:13:23.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.190+0000 7f19bca00640 1 -- 172.21.15.119:0/3611445511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b80b3920 con 0x7f19b80b28e0 2024-04-25T10:13:23.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.190+0000 7f19bca00640 1 -- 172.21.15.119:0/3611445511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b80b3ee0 con 0x7f19b80b28e0 2024-04-25T10:13:23.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.191+0000 7f19bca00640 1 -- 172.21.15.119:0/3611445511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b80750b0 con 0x7f19b80b28e0 2024-04-25T10:13:23.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.192+0000 7f19ae000640 1 -- 172.21.15.119:0/3611445511 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19a4043020 con 0x7f19b80b28e0 2024-04-25T10:13:23.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.193+0000 7f19ae000640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f199806b4d0 0x7f199806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.193+0000 7f19ae000640 1 -- 172.21.15.119:0/3611445511 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f19a40aa9a0 con 0x7f19b80b28e0 2024-04-25T10:13:23.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.193+0000 7f19b5e00640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f199806b4d0 0x7f199806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.194+0000 7f19b5e00640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f199806b4d0 0x7f199806d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f19a80011e0 tx=0x7f19a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:23.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.198+0000 7f19ae000640 1 -- 172.21.15.119:0/3611445511 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19a404a020 con 0x7f19b80b28e0 2024-04-25T10:13:23.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.425+0000 7f19bca00640 1 -- 172.21.15.119:0/3611445511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f19b80aebe0 con 0x7f19b80b28e0 2024-04-25T10:13:23.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:23.993+0000 7f19ae000640 1 -- 172.21.15.119:0/3611445511 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v57) v1 ==== 174+0+0 (secure 0 0 0) 0x7f19a40044e0 con 0x7f19b80b28e0 2024-04-25T10:13:23.999 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 12 and data pool 13 2024-04-25T10:13:24.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.000+0000 7f19bca00640 1 -- 172.21.15.119:0/3611445511 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f199806b4d0 msgr2=0x7f199806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.000+0000 7f19bca00640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f199806b4d0 0x7f199806d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f19a80011e0 tx=0x7f19a8009040 comp rx=0 tx=0).stop 2024-04-25T10:13:24.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.000+0000 7f19bca00640 1 -- 172.21.15.119:0/3611445511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19b80b28e0 msgr2=0x7f19b80b2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.000+0000 7f19bca00640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19b80b28e0 0x7f19b80b2d60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009be0 tx=0x7f19a4004730 comp rx=0 tx=0).stop 2024-04-25T10:13:24.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.002+0000 7f19bca00640 1 -- 172.21.15.119:0/3611445511 shutdown_connections 2024-04-25T10:13:24.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.002+0000 7f19bca00640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f199806b4d0 0x7f199806d990 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.002+0000 7f19bca00640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19b813c970 0x7f19b813cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.002+0000 7f19bca00640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19b80b28e0 0x7f19b80b2d60 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.002+0000 7f19bca00640 1 --2- 172.21.15.119:0/3611445511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f199c003f60 0x7f19b813c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.002+0000 7f19bca00640 1 -- 172.21.15.119:0/3611445511 >> 172.21.15.119:0/3611445511 conn(0x7f19b806aba0 msgr2=0x7f19b8140210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.003+0000 7f19bca00640 1 -- 172.21.15.119:0/3611445511 shutdown_connections 2024-04-25T10:13:24.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.003+0000 7f19bca00640 1 -- 172.21.15.119:0/3611445511 wait complete. 2024-04-25T10:13:24.043 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:24.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.205+0000 7f5864c00640 1 Processor -- start 2024-04-25T10:13:24.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.205+0000 7f5864c00640 1 -- start start 2024-04-25T10:13:24.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.207+0000 7f5864c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f58600d3f80 0x7f58600d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.207+0000 7f5864c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586015ef10 con 0x7f586015db60 2024-04-25T10:13:24.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.207+0000 7f5864c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586015e390 con 0x7f58600b9270 2024-04-25T10:13:24.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.207+0000 7f5864c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f586015e670 con 0x7f58600d3f80 2024-04-25T10:13:24.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.207+0000 7f585f400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f586015db60 0x7f58600d3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48456/0 (socket says 172.21.15.119:48456) 2024-04-25T10:13:24.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.207+0000 7f585f400640 1 -- 172.21.15.119:0/2773395671 learned_addr learned my addr 172.21.15.119:0/2773395671 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:24.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.207+0000 7f585ea00640 1 --2- 172.21.15.119:0/2773395671 >> v2:172.21.15.119:3300/0 conn(0x7f58600d3f80 0x7f58600d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.207+0000 7f585e000640 1 -- 172.21.15.119:0/2773395671 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685207788 0 0) 0x7f586015ef10 con 0x7f586015db60 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.207+0000 7f585e000640 1 -- 172.21.15.119:0/2773395671 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5844003520 con 0x7f586015db60 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.208+0000 7f585ea00640 1 -- 172.21.15.119:0/2773395671 >> v1:172.21.15.132:6789/0 conn(0x7f58600b9270 legacy=0x7f58600d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.208+0000 7f585e000640 1 -- 172.21.15.119:0/2773395671 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 662224324 0 0) 0x7f5844003520 con 0x7f586015db60 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.208+0000 7f585ea00640 1 -- 172.21.15.119:0/2773395671 >> v1:172.21.15.119:6789/0 conn(0x7f586015db60 legacy=0x7f58600d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.208+0000 7f585ea00640 1 -- 172.21.15.119:0/2773395671 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586015e950 con 0x7f58600d3f80 2024-04-25T10:13:24.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.209+0000 7f585ea00640 1 --2- 172.21.15.119:0/2773395671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58600d3f80 0x7f58600d1f90 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f5848009de0 tx=0x7f584802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d232762a95114fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.209+0000 7f585e000640 1 -- 172.21.15.119:0/2773395671 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850001670 con 0x7f58600d3f80 2024-04-25T10:13:24.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.209+0000 7f585e000640 1 -- 172.21.15.119:0/2773395671 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5848007540 con 0x7f58600d3f80 2024-04-25T10:13:24.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.209+0000 7f585e000640 1 -- 172.21.15.119:0/2773395671 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58480078f0 con 0x7f58600d3f80 2024-04-25T10:13:24.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.209+0000 7f5864c00640 1 -- 172.21.15.119:0/2773395671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58600d3f80 msgr2=0x7f58600d1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.210+0000 7f5864c00640 1 --2- 172.21.15.119:0/2773395671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58600d3f80 0x7f58600d1f90 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f5848009de0 tx=0x7f584802a680 comp rx=0 tx=0).stop 2024-04-25T10:13:24.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.210+0000 7f5864c00640 1 -- 172.21.15.119:0/2773395671 shutdown_connections 2024-04-25T10:13:24.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.210+0000 7f5864c00640 1 --2- 172.21.15.119:0/2773395671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58600d3f80 0x7f58600d1f90 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.210+0000 7f5864c00640 1 -- 172.21.15.119:0/2773395671 >> 172.21.15.119:0/2773395671 conn(0x7f58600726c0 msgr2=0x7f5860072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.210+0000 7f5864c00640 1 -- 172.21.15.119:0/2773395671 shutdown_connections 2024-04-25T10:13:24.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.210+0000 7f5864c00640 1 -- 172.21.15.119:0/2773395671 wait complete. 2024-04-25T10:13:24.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.211+0000 7f5864c00640 1 Processor -- start 2024-04-25T10:13:24.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.212+0000 7f5864c00640 1 -- start start 2024-04-25T10:13:24.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.212+0000 7f5864c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58600b9270 0x7f5860158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.212+0000 7f5864c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58600d3f80 0x7f58601494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.212+0000 7f5864c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f586015db60 0x7f58600cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.212+0000 7f5864c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58600b3120 con 0x7f58600b9270 2024-04-25T10:13:24.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.212+0000 7f5864c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58600b2fa0 con 0x7f58600d3f80 2024-04-25T10:13:24.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.212+0000 7f5864c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586015df40 con 0x7f586015db60 2024-04-25T10:13:24.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.213+0000 7f585f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58600b9270 0x7f5860158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.213+0000 7f585f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58600b9270 0x7f5860158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52602/0 (socket says 172.21.15.119:52602) 2024-04-25T10:13:24.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.213+0000 7f585f400640 1 -- 172.21.15.119:0/9045525 learned_addr learned my addr 172.21.15.119:0/9045525 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:24.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.213+0000 7f585ea00640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58600d3f80 0x7f58601494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.213+0000 7f585fe00640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f586015db60 0x7f58600cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.213+0000 7f585f400640 1 -- 172.21.15.119:0/9045525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f586015db60 msgr2=0x7f58600cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.213+0000 7f585f400640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f586015db60 0x7f58600cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.213+0000 7f585f400640 1 -- 172.21.15.119:0/9045525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58600d3f80 msgr2=0x7f58601494f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.213+0000 7f585f400640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58600d3f80 0x7f58601494f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.213+0000 7f585f400640 1 -- 172.21.15.119:0/9045525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58600cba00 con 0x7f58600b9270 2024-04-25T10:13:24.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.213+0000 7f585ea00640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58600d3f80 0x7f58601494f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:24.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.214+0000 7f585f400640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58600b9270 0x7f5860158790 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f585000c780 tx=0x7f585000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.214+0000 7f5857400640 1 -- 172.21.15.119:0/9045525 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850017070 con 0x7f58600b9270 2024-04-25T10:13:24.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.214+0000 7f5857400640 1 -- 172.21.15.119:0/9045525 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5850005ad0 con 0x7f58600b9270 2024-04-25T10:13:24.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.214+0000 7f5857400640 1 -- 172.21.15.119:0/9045525 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850007070 con 0x7f58600b9270 2024-04-25T10:13:24.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.214+0000 7f5864c00640 1 -- 172.21.15.119:0/9045525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58600cbcc0 con 0x7f58600b9270 2024-04-25T10:13:24.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.214+0000 7f5864c00640 1 -- 172.21.15.119:0/9045525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58600cc280 con 0x7f58600b9270 2024-04-25T10:13:24.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.215+0000 7f5864c00640 1 -- 172.21.15.119:0/9045525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58600749b0 con 0x7f58600b9270 2024-04-25T10:13:24.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.216+0000 7f5857400640 1 -- 172.21.15.119:0/9045525 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f585001a070 con 0x7f58600b9270 2024-04-25T10:13:24.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.217+0000 7f5857400640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f583c06b4d0 0x7f583c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.217+0000 7f5857400640 1 -- 172.21.15.119:0/9045525 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58500897b0 con 0x7f58600b9270 2024-04-25T10:13:24.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.217+0000 7f585ea00640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f583c06b4d0 0x7f583c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.218+0000 7f585ea00640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f583c06b4d0 0x7f583c06d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5848034000 tx=0x7f584802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.222+0000 7f5857400640 1 -- 172.21.15.119:0/9045525 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5850058620 con 0x7f58600b9270 2024-04-25T10:13:24.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.412+0000 7f5864c00640 1 -- 172.21.15.119:0/9045525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58600d1a60 con 0x7f58600b9270 2024-04-25T10:13:24.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.414+0000 7f5857400640 1 -- 172.21.15.119:0/9045525 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16709 (secure 0 0 0) 0x7f5850057f20 con 0x7f58600b9270 2024-04-25T10:13:24.419 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:24.419 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":80,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:13:23.426821+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:21.210576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:22.438674+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.417+0000 7f5864c00640 1 -- 172.21.15.119:0/9045525 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f583c06b4d0 msgr2=0x7f583c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.417+0000 7f5864c00640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f583c06b4d0 0x7f583c06d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5848034000 tx=0x7f584802f000 comp rx=0 tx=0).stop 2024-04-25T10:13:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.417+0000 7f5864c00640 1 -- 172.21.15.119:0/9045525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58600b9270 msgr2=0x7f5860158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.417+0000 7f5864c00640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58600b9270 0x7f5860158790 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f585000c780 tx=0x7f585000cc40 comp rx=0 tx=0).stop 2024-04-25T10:13:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.417+0000 7f5864c00640 1 -- 172.21.15.119:0/9045525 shutdown_connections 2024-04-25T10:13:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.417+0000 7f5864c00640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f583c06b4d0 0x7f583c06d990 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.417+0000 7f5864c00640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f586015db60 0x7f58600cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.417+0000 7f5864c00640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58600d3f80 0x7f58601494f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.417+0000 7f5864c00640 1 --2- 172.21.15.119:0/9045525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58600b9270 0x7f5860158790 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.417+0000 7f5864c00640 1 -- 172.21.15.119:0/9045525 >> 172.21.15.119:0/9045525 conn(0x7f58600726c0 msgr2=0x7f58600c7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.417+0000 7f5864c00640 1 -- 172.21.15.119:0/9045525 shutdown_connections 2024-04-25T10:13:24.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.417+0000 7f5864c00640 1 -- 172.21.15.119:0/9045525 wait complete. 2024-04-25T10:13:24.439 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.628+0000 7f5821c00640 1 Processor -- start 2024-04-25T10:13:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.628+0000 7f5821c00640 1 -- start start 2024-04-25T10:13:24.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.630+0000 7f5821c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c147270 0x7f581c0b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.630+0000 7f5821c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f581c15dee0 0x7f581c15e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.630+0000 7f5821c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f581c15e860 0x7f581c1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.630+0000 7f5821c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c0af000 con 0x7f581c15e860 2024-04-25T10:13:24.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.630+0000 7f5821c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c0b4a80 con 0x7f581c15dee0 2024-04-25T10:13:24.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.630+0000 7f5821c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0b4d60 con 0x7f581c147270 2024-04-25T10:13:24.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.630+0000 7f5821200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f581c15e860 0x7f581c1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.630+0000 7f5821200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f581c15e860 0x7f581c1614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52608/0 (socket says 172.21.15.119:52608) 2024-04-25T10:13:24.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.630+0000 7f5821200640 1 -- 172.21.15.119:0/3281734646 learned_addr learned my addr 172.21.15.119:0/3281734646 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:24.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.631+0000 7f5821200640 1 -- 172.21.15.119:0/3281734646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c147270 msgr2=0x7f581c0b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.631+0000 7f581be00640 1 --2- 172.21.15.119:0/3281734646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c147270 0x7f581c0b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.631+0000 7f581b400640 1 --2- 172.21.15.119:0/3281734646 >> v2:172.21.15.132:3300/0 conn(0x7f581c15dee0 0x7f581c15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.631+0000 7f5821200640 1 --2- 172.21.15.119:0/3281734646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c147270 0x7f581c0b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.631+0000 7f5821200640 1 -- 172.21.15.119:0/3281734646 >> v2:172.21.15.132:3300/0 conn(0x7f581c15dee0 msgr2=0x7f581c15e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.631+0000 7f5821200640 1 --2- 172.21.15.119:0/3281734646 >> v2:172.21.15.132:3300/0 conn(0x7f581c15dee0 0x7f581c15e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.631+0000 7f5821200640 1 -- 172.21.15.119:0/3281734646 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c0b5040 con 0x7f581c15e860 2024-04-25T10:13:24.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.631+0000 7f581be00640 1 --2- 172.21.15.119:0/3281734646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c147270 0x7f581c0b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:24.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.632+0000 7f5821200640 1 --2- 172.21.15.119:0/3281734646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f581c15e860 0x7f581c1614a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f581c06f410 tx=0x7f581402a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c6589320aa831b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.632+0000 7f581aa00640 1 -- 172.21.15.119:0/3281734646 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814036070 con 0x7f581c15e860 2024-04-25T10:13:24.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.632+0000 7f581aa00640 1 -- 172.21.15.119:0/3281734646 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5814031600 con 0x7f581c15e860 2024-04-25T10:13:24.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.632+0000 7f581aa00640 1 -- 172.21.15.119:0/3281734646 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58140319d0 con 0x7f581c15e860 2024-04-25T10:13:24.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.633+0000 7f5821c00640 1 -- 172.21.15.119:0/3281734646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f581c15e860 msgr2=0x7f581c1614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.633+0000 7f5821c00640 1 --2- 172.21.15.119:0/3281734646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f581c15e860 0x7f581c1614a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f581c06f410 tx=0x7f581402a600 comp rx=0 tx=0).stop 2024-04-25T10:13:24.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.633+0000 7f5821c00640 1 -- 172.21.15.119:0/3281734646 shutdown_connections 2024-04-25T10:13:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.633+0000 7f5821c00640 1 --2- 172.21.15.119:0/3281734646 >> v2:172.21.15.132:3300/0 conn(0x7f581c15dee0 0x7f581c15e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.633+0000 7f5821c00640 1 --2- 172.21.15.119:0/3281734646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f581c15e860 0x7f581c1614a0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.633+0000 7f5821c00640 1 --2- 172.21.15.119:0/3281734646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c147270 0x7f581c0b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.633+0000 7f5821c00640 1 -- 172.21.15.119:0/3281734646 >> 172.21.15.119:0/3281734646 conn(0x7f581c06b1d0 msgr2=0x7f581c0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.633+0000 7f5821c00640 1 -- 172.21.15.119:0/3281734646 shutdown_connections 2024-04-25T10:13:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.633+0000 7f5821c00640 1 -- 172.21.15.119:0/3281734646 wait complete. 2024-04-25T10:13:24.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.635+0000 7f5821c00640 1 Processor -- start 2024-04-25T10:13:24.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.635+0000 7f5821c00640 1 -- start start 2024-04-25T10:13:24.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f5821c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c147270 0x7f581c1423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f5821c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581c15dee0 0x7f581c0c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f5821c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f581c15e860 0x7f581c13d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f5821c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c15ecf0 con 0x7f581c15e860 2024-04-25T10:13:24.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f5821c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0b5600 con 0x7f581c15dee0 2024-04-25T10:13:24.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f5821c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c1619e0 con 0x7f581c147270 2024-04-25T10:13:24.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f5821200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f581c15e860 0x7f581c13d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f5821200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f581c15e860 0x7f581c13d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52610/0 (socket says 172.21.15.119:52610) 2024-04-25T10:13:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f5821200640 1 -- 172.21.15.119:0/2609951735 learned_addr learned my addr 172.21.15.119:0/2609951735 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f581be00640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c147270 0x7f581c1423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f581b400640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581c15dee0 0x7f581c0c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f5821200640 1 -- 172.21.15.119:0/2609951735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c147270 msgr2=0x7f581c1423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f5821200640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c147270 0x7f581c1423b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f5821200640 1 -- 172.21.15.119:0/2609951735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581c15dee0 msgr2=0x7f581c0c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f5821200640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581c15dee0 0x7f581c0c4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f5821200640 1 -- 172.21.15.119:0/2609951735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c1f3a70 con 0x7f581c15e860 2024-04-25T10:13:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.636+0000 7f581b400640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581c15dee0 0x7f581c0c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.637+0000 7f5821200640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f581c15e860 0x7f581c13d560 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f581402abf0 tx=0x7f5814004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.637+0000 7f5813e00640 1 -- 172.21.15.119:0/2609951735 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581403f020 con 0x7f581c15e860 2024-04-25T10:13:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.637+0000 7f5813e00640 1 -- 172.21.15.119:0/2609951735 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5814037040 con 0x7f581c15e860 2024-04-25T10:13:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.637+0000 7f5813e00640 1 -- 172.21.15.119:0/2609951735 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814036070 con 0x7f581c15e860 2024-04-25T10:13:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.637+0000 7f5821c00640 1 -- 172.21.15.119:0/2609951735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581c1415f0 con 0x7f581c15e860 2024-04-25T10:13:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.637+0000 7f5821c00640 1 -- 172.21.15.119:0/2609951735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c141bb0 con 0x7f581c15e860 2024-04-25T10:13:24.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.638+0000 7f5821c00640 1 -- 172.21.15.119:0/2609951735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c0749b0 con 0x7f581c15e860 2024-04-25T10:13:24.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.639+0000 7f5813e00640 1 -- 172.21.15.119:0/2609951735 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5814042020 con 0x7f581c15e860 2024-04-25T10:13:24.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.640+0000 7f5813e00640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f57fc06b4d0 0x7f57fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.640+0000 7f5813e00640 1 -- 172.21.15.119:0/2609951735 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58140aa890 con 0x7f581c15e860 2024-04-25T10:13:24.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.640+0000 7f581be00640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f57fc06b4d0 0x7f57fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.641+0000 7f581be00640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f57fc06b4d0 0x7f57fc06d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f580c006fd0 tx=0x7f580c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.645+0000 7f5813e00640 1 -- 172.21.15.119:0/2609951735 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5814049030 con 0x7f581c15e860 2024-04-25T10:13:24.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.838+0000 7f5821c00640 1 -- 172.21.15.119:0/2609951735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f581c06e430 con 0x7f581c15e860 2024-04-25T10:13:24.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.840+0000 7f5813e00640 1 -- 172.21.15.119:0/2609951735 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16709 (secure 0 0 0) 0x7f581402ae20 con 0x7f581c15e860 2024-04-25T10:13:24.845 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:24.845 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":80,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:13:23.426821+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:21.210576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:22.438674+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:24.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.843+0000 7f5821c00640 1 -- 172.21.15.119:0/2609951735 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f57fc06b4d0 msgr2=0x7f57fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.843+0000 7f5821c00640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f57fc06b4d0 0x7f57fc06d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f580c006fd0 tx=0x7f580c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:24.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.843+0000 7f5821c00640 1 -- 172.21.15.119:0/2609951735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f581c15e860 msgr2=0x7f581c13d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.843+0000 7f5821c00640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f581c15e860 0x7f581c13d560 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f581402abf0 tx=0x7f5814004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:24.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.843+0000 7f5821c00640 1 -- 172.21.15.119:0/2609951735 shutdown_connections 2024-04-25T10:13:24.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.843+0000 7f5821c00640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f57fc06b4d0 0x7f57fc06d990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.843+0000 7f5821c00640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f581c15e860 0x7f581c13d560 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.843+0000 7f5821c00640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581c15dee0 0x7f581c0c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.843+0000 7f5821c00640 1 --2- 172.21.15.119:0/2609951735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581c147270 0x7f581c1423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.843+0000 7f5821c00640 1 -- 172.21.15.119:0/2609951735 >> 172.21.15.119:0/2609951735 conn(0x7f581c06b1d0 msgr2=0x7f581c142b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.843+0000 7f5821c00640 1 -- 172.21.15.119:0/2609951735 shutdown_connections 2024-04-25T10:13:24.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:24.843+0000 7f5821c00640 1 -- 172.21.15.119:0/2609951735 wait complete. 2024-04-25T10:13:24.866 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:13:25.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.069+0000 7fde49200640 1 Processor -- start 2024-04-25T10:13:25.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.069+0000 7fde49200640 1 -- start start 2024-04-25T10:13:25.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.071+0000 7fde49200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde440ce9d0 0x7fde440cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.071+0000 7fde49200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fde440cc300 0x7fde440cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.071+0000 7fde49200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4406dcb0 con 0x7fde44144e70 2024-04-25T10:13:25.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.071+0000 7fde49200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4415bfe0 con 0x7fde440ce9d0 2024-04-25T10:13:25.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.071+0000 7fde49200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4406df90 con 0x7fde440cc300 2024-04-25T10:13:25.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.071+0000 7fde41400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde440ce9d0 0x7fde440cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.071+0000 7fde41400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde440ce9d0 0x7fde440cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52618/0 (socket says 172.21.15.119:52618) 2024-04-25T10:13:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.071+0000 7fde41400640 1 -- 172.21.15.119:0/331317961 learned_addr learned my addr 172.21.15.119:0/331317961 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.071+0000 7fde42800640 1 --2- 172.21.15.119:0/331317961 >> v2:172.21.15.132:3300/0 conn(0x7fde440cc300 0x7fde440cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.071+0000 7fde41400640 1 -- 172.21.15.119:0/331317961 >> v2:172.21.15.132:3300/0 conn(0x7fde440cc300 msgr2=0x7fde440cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.071+0000 7fde41400640 1 --2- 172.21.15.119:0/331317961 >> v2:172.21.15.132:3300/0 conn(0x7fde440cc300 0x7fde440cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.071+0000 7fde41400640 1 -- 172.21.15.119:0/331317961 >> v1:172.21.15.132:6789/0 conn(0x7fde44144e70 legacy=0x7fde4406d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.072+0000 7fde41400640 1 -- 172.21.15.119:0/331317961 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4406e270 con 0x7fde440ce9d0 2024-04-25T10:13:25.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.073+0000 7fde41400640 1 --2- 172.21.15.119:0/331317961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde440ce9d0 0x7fde440cee60 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fde441458f0 tx=0x7fde3002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d89212a559d07cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.073+0000 7fde40a00640 1 -- 172.21.15.119:0/331317961 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30037070 con 0x7fde440ce9d0 2024-04-25T10:13:25.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.073+0000 7fde40a00640 1 -- 172.21.15.119:0/331317961 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde30032440 con 0x7fde440ce9d0 2024-04-25T10:13:25.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.073+0000 7fde40a00640 1 -- 172.21.15.119:0/331317961 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30032810 con 0x7fde440ce9d0 2024-04-25T10:13:25.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.074+0000 7fde49200640 1 -- 172.21.15.119:0/331317961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde440ce9d0 msgr2=0x7fde440cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.074+0000 7fde49200640 1 --2- 172.21.15.119:0/331317961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde440ce9d0 0x7fde440cee60 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fde441458f0 tx=0x7fde3002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.074+0000 7fde49200640 1 -- 172.21.15.119:0/331317961 shutdown_connections 2024-04-25T10:13:25.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.074+0000 7fde49200640 1 --2- 172.21.15.119:0/331317961 >> v2:172.21.15.132:3300/0 conn(0x7fde440cc300 0x7fde440cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.074+0000 7fde49200640 1 --2- 172.21.15.119:0/331317961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde440ce9d0 0x7fde440cee60 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.074+0000 7fde49200640 1 -- 172.21.15.119:0/331317961 >> 172.21.15.119:0/331317961 conn(0x7fde4406aba0 msgr2=0x7fde4406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.074+0000 7fde49200640 1 -- 172.21.15.119:0/331317961 shutdown_connections 2024-04-25T10:13:25.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.074+0000 7fde49200640 1 -- 172.21.15.119:0/331317961 wait complete. 2024-04-25T10:13:25.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.075+0000 7fde49200640 1 Processor -- start 2024-04-25T10:13:25.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.076+0000 7fde49200640 1 -- start start 2024-04-25T10:13:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.076+0000 7fde49200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde440cc300 0x7fde44156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde49200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde440ce9d0 0x7fde440af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde49200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde44144e70 0x7fde44157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde49200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4406d970 con 0x7fde44144e70 2024-04-25T10:13:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde49200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440cf3a0 con 0x7fde440cc300 2024-04-25T10:13:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde49200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde441454b0 con 0x7fde440ce9d0 2024-04-25T10:13:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde42800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde44144e70 0x7fde44157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde42800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde44144e70 0x7fde44157490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52622/0 (socket says 172.21.15.119:52622) 2024-04-25T10:13:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde42800640 1 -- 172.21.15.119:0/2393055344 learned_addr learned my addr 172.21.15.119:0/2393055344 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde41e00640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde440cc300 0x7fde44156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde41400640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde440ce9d0 0x7fde440af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde42800640 1 -- 172.21.15.119:0/2393055344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde440ce9d0 msgr2=0x7fde440af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde42800640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde440ce9d0 0x7fde440af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde42800640 1 -- 172.21.15.119:0/2393055344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde440cc300 msgr2=0x7fde44156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde42800640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde440cc300 0x7fde44156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde42800640 1 -- 172.21.15.119:0/2393055344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde44157e80 con 0x7fde44144e70 2024-04-25T10:13:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.077+0000 7fde41e00640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde440cc300 0x7fde44156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.078+0000 7fde42800640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde44144e70 0x7fde44157490 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00c820 tx=0x7fde3c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.078+0000 7fde3a000640 1 -- 172.21.15.119:0/2393055344 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c015070 con 0x7fde44144e70 2024-04-25T10:13:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.078+0000 7fde3a000640 1 -- 172.21.15.119:0/2393055344 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c004020 con 0x7fde44144e70 2024-04-25T10:13:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.078+0000 7fde3a000640 1 -- 172.21.15.119:0/2393055344 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c004410 con 0x7fde44144e70 2024-04-25T10:13:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.078+0000 7fde49200640 1 -- 172.21.15.119:0/2393055344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4413caa0 con 0x7fde44144e70 2024-04-25T10:13:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.078+0000 7fde49200640 1 -- 172.21.15.119:0/2393055344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4413d020 con 0x7fde44144e70 2024-04-25T10:13:25.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.079+0000 7fde49200640 1 -- 172.21.15.119:0/2393055344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde440750b0 con 0x7fde44144e70 2024-04-25T10:13:25.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.080+0000 7fde3a000640 1 -- 172.21.15.119:0/2393055344 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde3c007830 con 0x7fde44144e70 2024-04-25T10:13:25.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.081+0000 7fde3a000640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fde2406b4d0 0x7fde2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.081+0000 7fde3a000640 1 -- 172.21.15.119:0/2393055344 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde3c08a060 con 0x7fde44144e70 2024-04-25T10:13:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.081+0000 7fde41e00640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fde2406b4d0 0x7fde2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.082+0000 7fde41e00640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fde2406b4d0 0x7fde2406d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fde34007eb0 tx=0x7fde340073c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.085+0000 7fde3a000640 1 -- 172.21.15.119:0/2393055344 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde3c058ed0 con 0x7fde44144e70 2024-04-25T10:13:25.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:25.318+0000 7fde49200640 1 -- 172.21.15.119:0/2393055344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fde4406b300 con 0x7fde44144e70 2024-04-25T10:13:26.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.016+0000 7fde3a000640 1 -- 172.21.15.119:0/2393055344 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) v1 ==== 118+0+0 (secure 0 0 0) 0x7fde3c0587d0 con 0x7fde44144e70 2024-04-25T10:13:26.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.020+0000 7fde49200640 1 -- 172.21.15.119:0/2393055344 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fde2406b4d0 msgr2=0x7fde2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.020+0000 7fde49200640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fde2406b4d0 0x7fde2406d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fde34007eb0 tx=0x7fde340073c0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.020+0000 7fde49200640 1 -- 172.21.15.119:0/2393055344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde44144e70 msgr2=0x7fde44157490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.020+0000 7fde49200640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde44144e70 0x7fde44157490 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00c820 tx=0x7fde3c00cce0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.020+0000 7fde49200640 1 -- 172.21.15.119:0/2393055344 shutdown_connections 2024-04-25T10:13:26.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.020+0000 7fde49200640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fde2406b4d0 0x7fde2406d990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.020+0000 7fde49200640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde44144e70 0x7fde44157490 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.020+0000 7fde49200640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde440ce9d0 0x7fde440af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.020+0000 7fde49200640 1 --2- 172.21.15.119:0/2393055344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde440cc300 0x7fde44156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.020+0000 7fde49200640 1 -- 172.21.15.119:0/2393055344 >> 172.21.15.119:0/2393055344 conn(0x7fde4406aba0 msgr2=0x7fde4413fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:26.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.020+0000 7fde49200640 1 -- 172.21.15.119:0/2393055344 shutdown_connections 2024-04-25T10:13:26.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.020+0000 7fde49200640 1 -- 172.21.15.119:0/2393055344 wait complete. 2024-04-25T10:13:26.047 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:13:26.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:13:26.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.250+0000 7f9203200640 1 Processor -- start 2024-04-25T10:13:26.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.250+0000 7f9203200640 1 -- start start 2024-04-25T10:13:26.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.251+0000 7f9203200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f92041450e0 0x7f920406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.251+0000 7f9203200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f920406b0e0 0x7f92040cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.251+0000 7f9203200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920415f400 con 0x7f92040cc810 2024-04-25T10:13:26.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.251+0000 7f9203200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92041454c0 con 0x7f920406b0e0 2024-04-25T10:13:26.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.251+0000 7f9203200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920406b540 con 0x7f92041450e0 2024-04-25T10:13:26.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.252+0000 7f9201400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f920406b0e0 0x7f92040cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.252+0000 7f9202800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f92040cc810 0x7f9204071050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48472/0 (socket says 172.21.15.119:48472) 2024-04-25T10:13:26.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.252+0000 7f9202800640 1 -- 172.21.15.119:0/2398438945 learned_addr learned my addr 172.21.15.119:0/2398438945 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:26.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.252+0000 7f9201e00640 1 --2- 172.21.15.119:0/2398438945 >> v2:172.21.15.132:3300/0 conn(0x7f92041450e0 0x7f920406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.252+0000 7f9201400640 1 -- 172.21.15.119:0/2398438945 >> v2:172.21.15.132:3300/0 conn(0x7f92041450e0 msgr2=0x7f920406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:26.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.252+0000 7f9201400640 1 --2- 172.21.15.119:0/2398438945 >> v2:172.21.15.132:3300/0 conn(0x7f92041450e0 0x7f920406aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.252+0000 7f9201400640 1 -- 172.21.15.119:0/2398438945 >> v1:172.21.15.119:6789/0 conn(0x7f92040cc810 legacy=0x7f9204071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.252+0000 7f9201400640 1 -- 172.21.15.119:0/2398438945 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920415f6e0 con 0x7f920406b0e0 2024-04-25T10:13:26.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.252+0000 7f9200a00640 1 -- 172.21.15.119:0/2398438945 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998042286 0 0) 0x7f920415f400 con 0x7f92040cc810 2024-04-25T10:13:26.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.253+0000 7f9201400640 1 --2- 172.21.15.119:0/2398438945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f920406b0e0 0x7f92040cc2d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9204072170 tx=0x7f91f002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e2859efbdc4e52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.254+0000 7f9200a00640 1 -- 172.21.15.119:0/2398438945 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0037070 con 0x7f920406b0e0 2024-04-25T10:13:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.254+0000 7f9200a00640 1 -- 172.21.15.119:0/2398438945 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f002add0 con 0x7f920406b0e0 2024-04-25T10:13:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.254+0000 7f9200a00640 1 -- 172.21.15.119:0/2398438945 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0032660 con 0x7f920406b0e0 2024-04-25T10:13:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.254+0000 7f9203200640 1 -- 172.21.15.119:0/2398438945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f920406b0e0 msgr2=0x7f92040cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.254+0000 7f9203200640 1 --2- 172.21.15.119:0/2398438945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f920406b0e0 0x7f92040cc2d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9204072170 tx=0x7f91f002a080 comp rx=0 tx=0).stop 2024-04-25T10:13:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.254+0000 7f9203200640 1 -- 172.21.15.119:0/2398438945 shutdown_connections 2024-04-25T10:13:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.254+0000 7f9203200640 1 --2- 172.21.15.119:0/2398438945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f920406b0e0 0x7f92040cc2d0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.254+0000 7f9203200640 1 --2- 172.21.15.119:0/2398438945 >> v2:172.21.15.132:3300/0 conn(0x7f92041450e0 0x7f920406aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.254+0000 7f9203200640 1 -- 172.21.15.119:0/2398438945 >> 172.21.15.119:0/2398438945 conn(0x7f9204066ff0 msgr2=0x7f92040b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:26.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.254+0000 7f9203200640 1 -- 172.21.15.119:0/2398438945 shutdown_connections 2024-04-25T10:13:26.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.254+0000 7f9203200640 1 -- 172.21.15.119:0/2398438945 wait complete. 2024-04-25T10:13:26.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.256+0000 7f9203200640 1 Processor -- start 2024-04-25T10:13:26.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.256+0000 7f9203200640 1 -- start start 2024-04-25T10:13:26.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.257+0000 7f9203200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f920406b0e0 0x7f92040ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.257+0000 7f9203200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92040cc810 0x7f92040aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.257+0000 7f9203200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92041450e0 0x7f92040a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.257+0000 7f9203200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92040ccc30 con 0x7f920406b0e0 2024-04-25T10:13:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.257+0000 7f9203200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920415fca0 con 0x7f92041450e0 2024-04-25T10:13:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.257+0000 7f9203200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92040ccdb0 con 0x7f92040cc810 2024-04-25T10:13:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.258+0000 7f9201e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f920406b0e0 0x7f92040ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.258+0000 7f9201e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f920406b0e0 0x7f92040ad750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52652/0 (socket says 172.21.15.119:52652) 2024-04-25T10:13:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.258+0000 7f9201e00640 1 -- 172.21.15.119:0/2723457340 learned_addr learned my addr 172.21.15.119:0/2723457340 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.258+0000 7f9201e00640 1 -- 172.21.15.119:0/2723457340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92040cc810 msgr2=0x7f92040aaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.258+0000 7f9201e00640 1 --2- 172.21.15.119:0/2723457340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92040cc810 0x7f92040aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.258+0000 7f9201e00640 1 -- 172.21.15.119:0/2723457340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92041450e0 msgr2=0x7f92040a99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.258+0000 7f9201e00640 1 --2- 172.21.15.119:0/2723457340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92041450e0 0x7f92040a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.258+0000 7f9201e00640 1 -- 172.21.15.119:0/2723457340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92041313c0 con 0x7f920406b0e0 2024-04-25T10:13:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.258+0000 7f9201e00640 1 --2- 172.21.15.119:0/2723457340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f920406b0e0 0x7f92040ad750 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f91f400d5f0 tx=0x7f91f400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.258+0000 7f91fa000640 1 -- 172.21.15.119:0/2723457340 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f40125f0 con 0x7f920406b0e0 2024-04-25T10:13:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.259+0000 7f91fa000640 1 -- 172.21.15.119:0/2723457340 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f4019070 con 0x7f920406b0e0 2024-04-25T10:13:26.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.259+0000 7f91fa000640 1 -- 172.21.15.119:0/2723457340 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f40129d0 con 0x7f920406b0e0 2024-04-25T10:13:26.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.259+0000 7f9203200640 1 -- 172.21.15.119:0/2723457340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92041316a0 con 0x7f920406b0e0 2024-04-25T10:13:26.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.259+0000 7f9203200640 1 -- 172.21.15.119:0/2723457340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9204131c60 con 0x7f920406b0e0 2024-04-25T10:13:26.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.260+0000 7f9203200640 1 -- 172.21.15.119:0/2723457340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92040749b0 con 0x7f920406b0e0 2024-04-25T10:13:26.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.261+0000 7f91fa000640 1 -- 172.21.15.119:0/2723457340 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91f4012b70 con 0x7f920406b0e0 2024-04-25T10:13:26.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.262+0000 7f91fa000640 1 --2- 172.21.15.119:0/2723457340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f91e406b4d0 0x7f91e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.262+0000 7f91fa000640 1 -- 172.21.15.119:0/2723457340 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91f408a6f0 con 0x7f920406b0e0 2024-04-25T10:13:26.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.262+0000 7f9201400640 1 --2- 172.21.15.119:0/2723457340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f91e406b4d0 0x7f91e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.263+0000 7f9201400640 1 --2- 172.21.15.119:0/2723457340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f91e406b4d0 0x7f91e406d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f91f0034000 tx=0x7f91f002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.266+0000 7f91fa000640 1 -- 172.21.15.119:0/2723457340 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91f40594d0 con 0x7f920406b0e0 2024-04-25T10:13:26.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:26.498+0000 7f9203200640 1 -- 172.21.15.119:0/2723457340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9204142290 con 0x7f920406b0e0 2024-04-25T10:13:27.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.029+0000 7f91fa000640 1 -- 172.21.15.119:0/2723457340 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v61) v1 ==== 122+0+0 (secure 0 0 0) 0x7f91f4058dd0 con 0x7f920406b0e0 2024-04-25T10:13:27.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.032+0000 7f9203200640 1 -- 172.21.15.119:0/2723457340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f91e406b4d0 msgr2=0x7f91e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.032+0000 7f9203200640 1 --2- 172.21.15.119:0/2723457340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f91e406b4d0 0x7f91e406d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f91f0034000 tx=0x7f91f002c000 comp rx=0 tx=0).stop 2024-04-25T10:13:27.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.032+0000 7f9203200640 1 -- 172.21.15.119:0/2723457340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f920406b0e0 msgr2=0x7f92040ad750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.032+0000 7f9203200640 1 --2- 172.21.15.119:0/2723457340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f920406b0e0 0x7f92040ad750 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f91f400d5f0 tx=0x7f91f400dab0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.032+0000 7f9203200640 1 -- 172.21.15.119:0/2723457340 shutdown_connections 2024-04-25T10:13:27.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.032+0000 7f9203200640 1 --2- 172.21.15.119:0/2723457340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f91e406b4d0 0x7f91e406d990 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.033+0000 7f9203200640 1 --2- 172.21.15.119:0/2723457340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92041450e0 0x7f92040a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.033+0000 7f9203200640 1 --2- 172.21.15.119:0/2723457340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92040cc810 0x7f92040aaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.033+0000 7f9203200640 1 --2- 172.21.15.119:0/2723457340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f920406b0e0 0x7f92040ad750 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.033+0000 7f9203200640 1 -- 172.21.15.119:0/2723457340 >> 172.21.15.119:0/2723457340 conn(0x7f9204066ff0 msgr2=0x7f92040b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.033+0000 7f9203200640 1 -- 172.21.15.119:0/2723457340 shutdown_connections 2024-04-25T10:13:27.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.033+0000 7f9203200640 1 -- 172.21.15.119:0/2723457340 wait complete. 2024-04-25T10:13:27.058 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:13:27.058 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:27.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.247+0000 7f20a7e00640 1 Processor -- start 2024-04-25T10:13:27.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.247+0000 7f20a7e00640 1 -- start start 2024-04-25T10:13:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.249+0000 7f20a7e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f20a80b4410 0x7f20a815e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.249+0000 7f20a7e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a80af000 con 0x7f20a815eb30 2024-04-25T10:13:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.249+0000 7f20a7e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a80b4a80 con 0x7f20a8147270 2024-04-25T10:13:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.249+0000 7f20a7e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a80b4d60 con 0x7f20a80b4410 2024-04-25T10:13:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.249+0000 7f20a6000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f20a80b4410 0x7f20a815e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.249+0000 7f20a7400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f20a815eb30 0x7f20a80acf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48488/0 (socket says 172.21.15.119:48488) 2024-04-25T10:13:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.249+0000 7f20a7400640 1 -- 172.21.15.119:0/394364342 learned_addr learned my addr 172.21.15.119:0/394364342 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:27.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.250+0000 7f20a6000640 1 -- 172.21.15.119:0/394364342 >> v1:172.21.15.132:6789/0 conn(0x7f20a8147270 legacy=0x7f20a815dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.250+0000 7f20a6000640 1 -- 172.21.15.119:0/394364342 >> v1:172.21.15.119:6789/0 conn(0x7f20a815eb30 legacy=0x7f20a80acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.250+0000 7f20a6000640 1 -- 172.21.15.119:0/394364342 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a80b5040 con 0x7f20a80b4410 2024-04-25T10:13:27.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.250+0000 7f20a5600640 1 -- 172.21.15.119:0/394364342 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783939719 0 0) 0x7f20a80af000 con 0x7f20a815eb30 2024-04-25T10:13:27.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.251+0000 7f20a6000640 1 --2- 172.21.15.119:0/394364342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20a80b4410 0x7f20a815e5f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f20900097a0 tx=0x7f209002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9980c3f14642e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.251+0000 7f20a5600640 1 -- 172.21.15.119:0/394364342 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090037070 con 0x7f20a80b4410 2024-04-25T10:13:27.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.251+0000 7f20a5600640 1 -- 172.21.15.119:0/394364342 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2090032440 con 0x7f20a80b4410 2024-04-25T10:13:27.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.251+0000 7f20a5600640 1 -- 172.21.15.119:0/394364342 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090032810 con 0x7f20a80b4410 2024-04-25T10:13:27.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.252+0000 7f20a7e00640 1 -- 172.21.15.119:0/394364342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20a80b4410 msgr2=0x7f20a815e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.252+0000 7f20a7e00640 1 --2- 172.21.15.119:0/394364342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20a80b4410 0x7f20a815e5f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f20900097a0 tx=0x7f209002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.252+0000 7f20a7e00640 1 -- 172.21.15.119:0/394364342 shutdown_connections 2024-04-25T10:13:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.252+0000 7f20a7e00640 1 --2- 172.21.15.119:0/394364342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20a80b4410 0x7f20a815e5f0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.252+0000 7f20a7e00640 1 -- 172.21.15.119:0/394364342 >> 172.21.15.119:0/394364342 conn(0x7f20a806b1d0 msgr2=0x7f20a80b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.252+0000 7f20a7e00640 1 -- 172.21.15.119:0/394364342 shutdown_connections 2024-04-25T10:13:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.252+0000 7f20a7e00640 1 -- 172.21.15.119:0/394364342 wait complete. 2024-04-25T10:13:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.254+0000 7f20a7e00640 1 Processor -- start 2024-04-25T10:13:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.254+0000 7f20a7e00640 1 -- start start 2024-04-25T10:13:27.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.255+0000 7f20a7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20a80b4410 0x7f20a8134540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.255+0000 7f20a7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20a8147270 0x7f20a81e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.255+0000 7f20a7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20a815eb30 0x7f20a81f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.255+0000 7f20a7e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80ad690 con 0x7f20a8147270 2024-04-25T10:13:27.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.255+0000 7f20a7e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80b5600 con 0x7f20a815eb30 2024-04-25T10:13:27.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.255+0000 7f20a7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80ad810 con 0x7f20a80b4410 2024-04-25T10:13:27.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.255+0000 7f20a6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20a8147270 0x7f20a81e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.255+0000 7f20a6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20a8147270 0x7f20a81e6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52666/0 (socket says 172.21.15.119:52666) 2024-04-25T10:13:27.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.255+0000 7f20a6000640 1 -- 172.21.15.119:0/1929931487 learned_addr learned my addr 172.21.15.119:0/1929931487 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:27.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.255+0000 7f20a6000640 1 -- 172.21.15.119:0/1929931487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20a80b4410 msgr2=0x7f20a8134540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.255+0000 7f20a6000640 1 --2- 172.21.15.119:0/1929931487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20a80b4410 0x7f20a8134540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.255+0000 7f20a6000640 1 -- 172.21.15.119:0/1929931487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20a815eb30 msgr2=0x7f20a81f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.255+0000 7f20a6000640 1 --2- 172.21.15.119:0/1929931487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20a815eb30 0x7f20a81f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.255+0000 7f20a6000640 1 -- 172.21.15.119:0/1929931487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a81311f0 con 0x7f20a8147270 2024-04-25T10:13:27.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.256+0000 7f20a6000640 1 --2- 172.21.15.119:0/1929931487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20a8147270 0x7f20a81e6c90 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f20900097a0 tx=0x7f20900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.256+0000 7f209ea00640 1 -- 172.21.15.119:0/1929931487 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090041070 con 0x7f20a8147270 2024-04-25T10:13:27.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.256+0000 7f209ea00640 1 -- 172.21.15.119:0/1929931487 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209003c870 con 0x7f20a8147270 2024-04-25T10:13:27.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.256+0000 7f20a7e00640 1 -- 172.21.15.119:0/1929931487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a81314d0 con 0x7f20a8147270 2024-04-25T10:13:27.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.256+0000 7f209ea00640 1 -- 172.21.15.119:0/1929931487 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090037040 con 0x7f20a8147270 2024-04-25T10:13:27.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.256+0000 7f20a7e00640 1 -- 172.21.15.119:0/1929931487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a8131a90 con 0x7f20a8147270 2024-04-25T10:13:27.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.257+0000 7f20a7e00640 1 -- 172.21.15.119:0/1929931487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a80749b0 con 0x7f20a8147270 2024-04-25T10:13:27.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.258+0000 7f209ea00640 1 -- 172.21.15.119:0/1929931487 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2090044070 con 0x7f20a8147270 2024-04-25T10:13:27.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.259+0000 7f209ea00640 1 --2- 172.21.15.119:0/1929931487 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f208406b4d0 0x7f208406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.259+0000 7f209ea00640 1 -- 172.21.15.119:0/1929931487 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20900aad90 con 0x7f20a8147270 2024-04-25T10:13:27.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.259+0000 7f20a6a00640 1 --2- 172.21.15.119:0/1929931487 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f208406b4d0 0x7f208406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.260+0000 7f20a6a00640 1 --2- 172.21.15.119:0/1929931487 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f208406b4d0 0x7f208406d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2098003cc0 tx=0x7f20980079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.263+0000 7f209ea00640 1 -- 172.21.15.119:0/1929931487 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2090079c00 con 0x7f20a8147270 2024-04-25T10:13:27.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.493+0000 7f20a7e00640 1 -- 172.21.15.119:0/1929931487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20a80b2180 con 0x7f20a8147270 2024-04-25T10:13:27.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.494+0000 7f209ea00640 1 -- 172.21.15.119:0/1929931487 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f2090079500 con 0x7f20a8147270 2024-04-25T10:13:27.498 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:27.498 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:13:26.498824+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7158},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7158":{"gid":7158,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/3997964942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3997964942},{"type":"v1","addr":"172.21.15.119:6837","nonce":3997964942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:13:27.498 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 61 2024-04-25T10:13:27.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.497+0000 7f20a7e00640 1 -- 172.21.15.119:0/1929931487 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f208406b4d0 msgr2=0x7f208406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.497+0000 7f20a7e00640 1 --2- 172.21.15.119:0/1929931487 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f208406b4d0 0x7f208406d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2098003cc0 tx=0x7f20980079b0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.497+0000 7f20a7e00640 1 -- 172.21.15.119:0/1929931487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20a8147270 msgr2=0x7f20a81e6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.497+0000 7f20a7e00640 1 --2- 172.21.15.119:0/1929931487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20a8147270 0x7f20a81e6c90 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f20900097a0 tx=0x7f20900046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.497+0000 7f20a7e00640 1 -- 172.21.15.119:0/1929931487 shutdown_connections 2024-04-25T10:13:27.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.497+0000 7f20a7e00640 1 --2- 172.21.15.119:0/1929931487 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f208406b4d0 0x7f208406d990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.497+0000 7f20a7e00640 1 --2- 172.21.15.119:0/1929931487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20a815eb30 0x7f20a81f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.497+0000 7f20a7e00640 1 --2- 172.21.15.119:0/1929931487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20a8147270 0x7f20a81e6c90 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.497+0000 7f20a7e00640 1 --2- 172.21.15.119:0/1929931487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20a80b4410 0x7f20a8134540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.497+0000 7f20a7e00640 1 -- 172.21.15.119:0/1929931487 >> 172.21.15.119:0/1929931487 conn(0x7f20a806b1d0 msgr2=0x7f20a80ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.497+0000 7f20a7e00640 1 -- 172.21.15.119:0/1929931487 shutdown_connections 2024-04-25T10:13:27.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.497+0000 7f20a7e00640 1 -- 172.21.15.119:0/1929931487 wait complete. 2024-04-25T10:13:27.519 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:27.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.714+0000 7fd73be00640 1 Processor -- start 2024-04-25T10:13:27.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.714+0000 7fd73be00640 1 -- start start 2024-04-25T10:13:27.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.716+0000 7fd73be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd73c144470 0x7fd73c15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.716+0000 7fd73be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd73c0b5050 0x7fd73c15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.716+0000 7fd73be00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd73c0cd440 con 0x7fd73c06e430 2024-04-25T10:13:27.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.716+0000 7fd73be00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd73c145340 con 0x7fd73c144470 2024-04-25T10:13:27.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.716+0000 7fd73be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c0cd720 con 0x7fd73c0b5050 2024-04-25T10:13:27.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.717+0000 7fd73b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd73c0b5050 0x7fd73c15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.717+0000 7fd73aa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd73c144470 0x7fd73c15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.717+0000 7fd73aa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd73c144470 0x7fd73c15fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:34454/0 (socket says 172.21.15.119:34454) 2024-04-25T10:13:27.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.717+0000 7fd73aa00640 1 -- 172.21.15.119:0/4226606574 learned_addr learned my addr 172.21.15.119:0/4226606574 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:27.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.717+0000 7fd73b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd73c0b5050 0x7fd73c15bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:35258/0 (socket says 172.21.15.119:35258) 2024-04-25T10:13:27.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.717+0000 7fd73a000640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd73c06e430 0x7fd73c0b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:36712/0 (socket says 172.21.15.119:36712) 2024-04-25T10:13:27.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.718+0000 7fd73b400640 1 -- 172.21.15.119:0/4226606574 >> v2:172.21.15.132:3300/0 conn(0x7fd73c144470 msgr2=0x7fd73c15fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.718+0000 7fd73b400640 1 --2- 172.21.15.119:0/4226606574 >> v2:172.21.15.132:3300/0 conn(0x7fd73c144470 0x7fd73c15fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.718+0000 7fd73b400640 1 -- 172.21.15.119:0/4226606574 >> v1:172.21.15.132:6789/0 conn(0x7fd73c06e430 legacy=0x7fd73c0b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.718+0000 7fd73b400640 1 -- 172.21.15.119:0/4226606574 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c0cda00 con 0x7fd73c0b5050 2024-04-25T10:13:27.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.720+0000 7fd73b400640 1 --2- 172.21.15.119:0/4226606574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd73c0b5050 0x7fd73c15bcf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd7340097a0 tx=0x7fd73402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=323969b765b22c5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.720+0000 7fd739600640 1 -- 172.21.15.119:0/4226606574 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734037070 con 0x7fd73c0b5050 2024-04-25T10:13:27.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.720+0000 7fd739600640 1 -- 172.21.15.119:0/4226606574 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd734032440 con 0x7fd73c0b5050 2024-04-25T10:13:27.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.720+0000 7fd739600640 1 -- 172.21.15.119:0/4226606574 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734032810 con 0x7fd73c0b5050 2024-04-25T10:13:27.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.721+0000 7fd73be00640 1 -- 172.21.15.119:0/4226606574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd73c0b5050 msgr2=0x7fd73c15bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.721+0000 7fd73be00640 1 --2- 172.21.15.119:0/4226606574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd73c0b5050 0x7fd73c15bcf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd7340097a0 tx=0x7fd73402a180 comp rx=0 tx=0).stop 2024-04-25T10:13:27.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.721+0000 7fd73be00640 1 -- 172.21.15.119:0/4226606574 shutdown_connections 2024-04-25T10:13:27.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.721+0000 7fd73be00640 1 --2- 172.21.15.119:0/4226606574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd73c0b5050 0x7fd73c15bcf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.721+0000 7fd73be00640 1 --2- 172.21.15.119:0/4226606574 >> v2:172.21.15.132:3300/0 conn(0x7fd73c144470 0x7fd73c15fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.721+0000 7fd73be00640 1 -- 172.21.15.119:0/4226606574 >> 172.21.15.119:0/4226606574 conn(0x7fd73c06b1d0 msgr2=0x7fd73c0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.721+0000 7fd73be00640 1 -- 172.21.15.119:0/4226606574 shutdown_connections 2024-04-25T10:13:27.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.721+0000 7fd73be00640 1 -- 172.21.15.119:0/4226606574 wait complete. 2024-04-25T10:13:27.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.723+0000 7fd73be00640 1 Processor -- start 2024-04-25T10:13:27.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.724+0000 7fd73be00640 1 -- start start 2024-04-25T10:13:27.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.725+0000 7fd73be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd73c06e430 0x7fd73c1399c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.725+0000 7fd73be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd73c0b5050 0x7fd73c15d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.725+0000 7fd73be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd73c144470 0x7fd73c13d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.725+0000 7fd73be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c0ce200 con 0x7fd73c0b5050 2024-04-25T10:13:27.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.725+0000 7fd73be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c144850 con 0x7fd73c06e430 2024-04-25T10:13:27.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.725+0000 7fd73be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c15c230 con 0x7fd73c144470 2024-04-25T10:13:27.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.725+0000 7fd73a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd73c0b5050 0x7fd73c15d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.725+0000 7fd73a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd73c0b5050 0x7fd73c15d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52680/0 (socket says 172.21.15.119:52680) 2024-04-25T10:13:27.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.725+0000 7fd73a000640 1 -- 172.21.15.119:0/705926232 learned_addr learned my addr 172.21.15.119:0/705926232 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:27.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.725+0000 7fd73aa00640 1 --2- 172.21.15.119:0/705926232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd73c06e430 0x7fd73c1399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.725+0000 7fd73b400640 1 --2- 172.21.15.119:0/705926232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd73c144470 0x7fd73c13d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.726+0000 7fd73a000640 1 -- 172.21.15.119:0/705926232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd73c144470 msgr2=0x7fd73c13d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.726+0000 7fd73a000640 1 --2- 172.21.15.119:0/705926232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd73c144470 0x7fd73c13d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.726+0000 7fd73a000640 1 -- 172.21.15.119:0/705926232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd73c06e430 msgr2=0x7fd73c1399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.726+0000 7fd73a000640 1 --2- 172.21.15.119:0/705926232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd73c06e430 0x7fd73c1399c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.726+0000 7fd73a000640 1 -- 172.21.15.119:0/705926232 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c139f00 con 0x7fd73c0b5050 2024-04-25T10:13:27.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.726+0000 7fd73a000640 1 --2- 172.21.15.119:0/705926232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd73c0b5050 0x7fd73c15d5f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd72400c440 tx=0x7fd724001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.727+0000 7fd732a00640 1 -- 172.21.15.119:0/705926232 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7240074e0 con 0x7fd73c0b5050 2024-04-25T10:13:27.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.727+0000 7fd732a00640 1 -- 172.21.15.119:0/705926232 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd724007680 con 0x7fd73c0b5050 2024-04-25T10:13:27.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.727+0000 7fd73be00640 1 -- 172.21.15.119:0/705926232 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73c1e61a0 con 0x7fd73c0b5050 2024-04-25T10:13:27.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.727+0000 7fd732a00640 1 -- 172.21.15.119:0/705926232 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd724004a20 con 0x7fd73c0b5050 2024-04-25T10:13:27.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.727+0000 7fd73be00640 1 -- 172.21.15.119:0/705926232 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73c1e6760 con 0x7fd73c0b5050 2024-04-25T10:13:27.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.728+0000 7fd73be00640 1 -- 172.21.15.119:0/705926232 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73c0749b0 con 0x7fd73c0b5050 2024-04-25T10:13:27.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.729+0000 7fd732a00640 1 -- 172.21.15.119:0/705926232 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd72401a020 con 0x7fd73c0b5050 2024-04-25T10:13:27.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.730+0000 7fd732a00640 1 --2- 172.21.15.119:0/705926232 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd71806b4d0 0x7fd71806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.730+0000 7fd732a00640 1 -- 172.21.15.119:0/705926232 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd724089920 con 0x7fd73c0b5050 2024-04-25T10:13:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.730+0000 7fd73aa00640 1 --2- 172.21.15.119:0/705926232 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd71806b4d0 0x7fd71806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.731+0000 7fd73aa00640 1 --2- 172.21.15.119:0/705926232 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd71806b4d0 0x7fd71806d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd72c011340 tx=0x7fd72c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.734+0000 7fd732a00640 1 -- 172.21.15.119:0/705926232 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd73c139f00 con 0x7fd73c0b5050 2024-04-25T10:13:27.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.931+0000 7fd73be00640 1 -- 172.21.15.119:0/705926232 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd73c0b2090 con 0x7fd73c0b5050 2024-04-25T10:13:27.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.934+0000 7fd732a00640 1 -- 172.21.15.119:0/705926232 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16696 (secure 0 0 0) 0x7fd7240580b0 con 0x7fd73c0b5050 2024-04-25T10:13:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:27.939 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":81,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:13:24.972756+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:21.210576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:22.438674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:27.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.936+0000 7fd73be00640 1 -- 172.21.15.119:0/705926232 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd71806b4d0 msgr2=0x7fd71806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.936+0000 7fd73be00640 1 --2- 172.21.15.119:0/705926232 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd71806b4d0 0x7fd71806d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd72c011340 tx=0x7fd72c00b040 comp rx=0 tx=0).stop 2024-04-25T10:13:27.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.936+0000 7fd73be00640 1 -- 172.21.15.119:0/705926232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd73c0b5050 msgr2=0x7fd73c15d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.936+0000 7fd73be00640 1 --2- 172.21.15.119:0/705926232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd73c0b5050 0x7fd73c15d5f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd72400c440 tx=0x7fd724001e70 comp rx=0 tx=0).stop 2024-04-25T10:13:27.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.936+0000 7fd73be00640 1 -- 172.21.15.119:0/705926232 shutdown_connections 2024-04-25T10:13:27.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.936+0000 7fd73be00640 1 --2- 172.21.15.119:0/705926232 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd71806b4d0 0x7fd71806d990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.936+0000 7fd73be00640 1 --2- 172.21.15.119:0/705926232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd73c144470 0x7fd73c13d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.936+0000 7fd73be00640 1 --2- 172.21.15.119:0/705926232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd73c0b5050 0x7fd73c15d5f0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.936+0000 7fd73be00640 1 --2- 172.21.15.119:0/705926232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd73c06e430 0x7fd73c1399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.936+0000 7fd73be00640 1 -- 172.21.15.119:0/705926232 >> 172.21.15.119:0/705926232 conn(0x7fd73c06b1d0 msgr2=0x7fd73c141a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.936+0000 7fd73be00640 1 -- 172.21.15.119:0/705926232 shutdown_connections 2024-04-25T10:13:27.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:27.936+0000 7fd73be00640 1 -- 172.21.15.119:0/705926232 wait complete. 2024-04-25T10:13:27.960 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:13:27.961 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:28.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.143+0000 7f6893200640 1 Processor -- start 2024-04-25T10:13:28.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.143+0000 7f6893200640 1 -- start start 2024-04-25T10:13:28.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.145+0000 7f6893200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f688c15d980 0x7f688c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.145+0000 7f6893200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f688c0712c0 con 0x7f688c15ed70 2024-04-25T10:13:28.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.145+0000 7f6893200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f688c0715a0 con 0x7f688c0aeb20 2024-04-25T10:13:28.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.145+0000 7f6893200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f688c071ed0 con 0x7f688c15d980 2024-04-25T10:13:28.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.145+0000 7f6891200640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f688c15ed70 0x7f688c0acf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48496/0 (socket says 172.21.15.119:48496) 2024-04-25T10:13:28.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.145+0000 7f6891200640 1 -- 172.21.15.119:0/1602308515 learned_addr learned my addr 172.21.15.119:0/1602308515 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:28.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.145+0000 7f688b400640 1 -- 172.21.15.119:0/1602308515 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705560088 0 0) 0x7f688c0712c0 con 0x7f688c15ed70 2024-04-25T10:13:28.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.145+0000 7f688b400640 1 -- 172.21.15.119:0/1602308515 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6870003520 con 0x7f688c15ed70 2024-04-25T10:13:28.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.145+0000 7f688be00640 1 --2- 172.21.15.119:0/1602308515 >> v2:172.21.15.132:3300/0 conn(0x7f688c15d980 0x7f688c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f688b400640 1 -- 172.21.15.119:0/1602308515 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1527006118 0 0) 0x7f6870003520 con 0x7f688c15ed70 2024-04-25T10:13:28.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f688b400640 1 -- 172.21.15.119:0/1602308515 >> v2:172.21.15.132:3300/0 conn(0x7f688c15d980 msgr2=0x7f688c0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f688b400640 1 --2- 172.21.15.119:0/1602308515 >> v2:172.21.15.132:3300/0 conn(0x7f688c15d980 0x7f688c0ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f688b400640 1 -- 172.21.15.119:0/1602308515 >> v1:172.21.15.132:6789/0 conn(0x7f688c0aeb20 legacy=0x7f688c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f688b400640 1 -- 172.21.15.119:0/1602308515 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c071880 con 0x7f688c15ed70 2024-04-25T10:13:28.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f688b400640 1 -- 172.21.15.119:0/1602308515 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f68840019f0 con 0x7f688c15ed70 2024-04-25T10:13:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f688b400640 1 -- 172.21.15.119:0/1602308515 >> v1:172.21.15.119:6789/0 conn(0x7f688c15ed70 legacy=0x7f688c0acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f688b400640 1 --2- 172.21.15.119:0/1602308515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68700039b0 0x7f6870003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f688b400640 1 --2- 172.21.15.119:0/1602308515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6870004800 0x7f6870004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f688b400640 1 --2- 172.21.15.119:0/1602308515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68700052e0 0x7f687000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f688b400640 1 -- 172.21.15.119:0/1602308515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c071ed0 con 0x7f68700052e0 2024-04-25T10:13:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f688b400640 1 -- 172.21.15.119:0/1602308515 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687000c1b0 con 0x7f68700039b0 2024-04-25T10:13:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f688b400640 1 -- 172.21.15.119:0/1602308515 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687000c3b0 con 0x7f6870004800 2024-04-25T10:13:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f6891200640 1 --2- 172.21.15.119:0/1602308515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68700052e0 0x7f687000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.146+0000 7f688be00640 1 --2- 172.21.15.119:0/1602308515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68700039b0 0x7f6870003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.147+0000 7f6883400640 1 --2- 172.21.15.119:0/1602308515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6870004800 0x7f6870004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.147+0000 7f6891200640 1 -- 172.21.15.119:0/1602308515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6870004800 msgr2=0x7f6870004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.147+0000 7f6891200640 1 --2- 172.21.15.119:0/1602308515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6870004800 0x7f6870004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.147+0000 7f6891200640 1 -- 172.21.15.119:0/1602308515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68700039b0 msgr2=0x7f6870003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.147+0000 7f6891200640 1 --2- 172.21.15.119:0/1602308515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68700039b0 0x7f6870003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.147+0000 7f6891200640 1 -- 172.21.15.119:0/1602308515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6870003520 con 0x7f68700052e0 2024-04-25T10:13:28.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.147+0000 7f6891200640 1 -- 172.21.15.119:0/1602308515 reap_dead start 2024-04-25T10:13:28.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.148+0000 7f6891200640 1 --2- 172.21.15.119:0/1602308515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68700052e0 0x7f687000b9c0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f688400a830 tx=0x7f6870005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc28282b905e3be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.148+0000 7f688b400640 1 -- 172.21.15.119:0/1602308515 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688400a480 con 0x7f68700052e0 2024-04-25T10:13:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.148+0000 7f688b400640 1 -- 172.21.15.119:0/1602308515 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688402f070 con 0x7f68700052e0 2024-04-25T10:13:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.148+0000 7f688b400640 1 -- 172.21.15.119:0/1602308515 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688402bc70 con 0x7f68700052e0 2024-04-25T10:13:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.148+0000 7f6893200640 1 -- 172.21.15.119:0/1602308515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68700052e0 msgr2=0x7f687000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.148+0000 7f6893200640 1 --2- 172.21.15.119:0/1602308515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68700052e0 0x7f687000b9c0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f688400a830 tx=0x7f6870005830 comp rx=0 tx=0).stop 2024-04-25T10:13:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.149+0000 7f6893200640 1 -- 172.21.15.119:0/1602308515 shutdown_connections 2024-04-25T10:13:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.149+0000 7f6893200640 1 --2- 172.21.15.119:0/1602308515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68700052e0 0x7f687000b9c0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.149+0000 7f6893200640 1 -- 172.21.15.119:0/1602308515 >> 172.21.15.119:0/1602308515 conn(0x7f688c06aba0 msgr2=0x7f688c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.149+0000 7f6893200640 1 -- 172.21.15.119:0/1602308515 shutdown_connections 2024-04-25T10:13:28.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.149+0000 7f6893200640 1 -- 172.21.15.119:0/1602308515 wait complete. 2024-04-25T10:13:28.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.150+0000 7f6893200640 1 Processor -- start 2024-04-25T10:13:28.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.150+0000 7f6893200640 1 -- start start 2024-04-25T10:13:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.151+0000 7f6893200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68700052e0 0x7f688c1f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.151+0000 7f6893200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f688c151650 0x7f688c14e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.151+0000 7f6893200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f688c155b10 0x7f688c155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.151+0000 7f6893200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c0ae760 con 0x7f688c155b10 2024-04-25T10:13:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.151+0000 7f6893200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c0ae5e0 con 0x7f68700052e0 2024-04-25T10:13:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.151+0000 7f6893200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c0ae8e0 con 0x7f688c151650 2024-04-25T10:13:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.152+0000 7f6891200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f688c155b10 0x7f688c155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.152+0000 7f6891200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f688c155b10 0x7f688c155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52708/0 (socket says 172.21.15.119:52708) 2024-04-25T10:13:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.152+0000 7f6891200640 1 -- 172.21.15.119:0/951728206 learned_addr learned my addr 172.21.15.119:0/951728206 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.152+0000 7f6883400640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f688c151650 0x7f688c14e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.152+0000 7f6891200640 1 -- 172.21.15.119:0/951728206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f688c151650 msgr2=0x7f688c14e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.152+0000 7f688be00640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68700052e0 0x7f688c1f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.152+0000 7f6891200640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f688c151650 0x7f688c14e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.152+0000 7f6891200640 1 -- 172.21.15.119:0/951728206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68700052e0 msgr2=0x7f688c1f7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.152+0000 7f6891200640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68700052e0 0x7f688c1f7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.152+0000 7f6891200640 1 -- 172.21.15.119:0/951728206 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c151b10 con 0x7f688c155b10 2024-04-25T10:13:28.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.152+0000 7f6883400640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f688c151650 0x7f688c14e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:28.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.152+0000 7f688be00640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68700052e0 0x7f688c1f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:28.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.152+0000 7f6891200640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f688c155b10 0x7f688c155fc0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f688400a830 tx=0x7f688c071490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.153+0000 7f6888c00640 1 -- 172.21.15.119:0/951728206 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884044070 con 0x7f688c155b10 2024-04-25T10:13:28.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.153+0000 7f6888c00640 1 -- 172.21.15.119:0/951728206 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688402bd70 con 0x7f688c155b10 2024-04-25T10:13:28.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.153+0000 7f6888c00640 1 -- 172.21.15.119:0/951728206 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688400a480 con 0x7f688c155b10 2024-04-25T10:13:28.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.153+0000 7f6893200640 1 -- 172.21.15.119:0/951728206 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f688c151df0 con 0x7f688c155b10 2024-04-25T10:13:28.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.153+0000 7f6893200640 1 -- 172.21.15.119:0/951728206 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f688c1523b0 con 0x7f688c155b10 2024-04-25T10:13:28.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.154+0000 7f6893200640 1 -- 172.21.15.119:0/951728206 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f688c0749b0 con 0x7f688c155b10 2024-04-25T10:13:28.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.155+0000 7f6888c00640 1 -- 172.21.15.119:0/951728206 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6884047070 con 0x7f688c155b10 2024-04-25T10:13:28.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.156+0000 7f6888c00640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f686806b500 0x7f686806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.157+0000 7f6888c00640 1 -- 172.21.15.119:0/951728206 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68840a9d40 con 0x7f688c155b10 2024-04-25T10:13:28.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.157+0000 7f688be00640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f686806b500 0x7f686806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.157+0000 7f688be00640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f686806b500 0x7f686806d9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f687c006fd0 tx=0x7f687c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.161+0000 7f6888c00640 1 -- 172.21.15.119:0/951728206 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6884078a90 con 0x7f688c155b10 2024-04-25T10:13:28.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.349+0000 7f6893200640 1 -- 172.21.15.119:0/951728206 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f688c14f8b0 con 0x7f686806b500 2024-04-25T10:13:28.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.361+0000 7f6888c00640 1 -- 172.21.15.119:0/951728206 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252039 (secure 0 0 0) 0x7f687c00ee20 con 0x7f686806b500 2024-04-25T10:13:28.369 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:28.371 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:13:28.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.367+0000 7f6893200640 1 -- 172.21.15.119:0/951728206 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f686806b500 msgr2=0x7f686806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.367+0000 7f6893200640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f686806b500 0x7f686806d9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f687c006fd0 tx=0x7f687c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:28.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.367+0000 7f6893200640 1 -- 172.21.15.119:0/951728206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f688c155b10 msgr2=0x7f688c155fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.367+0000 7f6893200640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f688c155b10 0x7f688c155fc0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f688400a830 tx=0x7f688c071490 comp rx=0 tx=0).stop 2024-04-25T10:13:28.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.367+0000 7f6893200640 1 -- 172.21.15.119:0/951728206 shutdown_connections 2024-04-25T10:13:28.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.367+0000 7f6893200640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f686806b500 0x7f686806d9c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.367+0000 7f6893200640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f688c155b10 0x7f688c155fc0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.367+0000 7f6893200640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f688c151650 0x7f688c14e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.367+0000 7f6893200640 1 --2- 172.21.15.119:0/951728206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68700052e0 0x7f688c1f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.367+0000 7f6893200640 1 -- 172.21.15.119:0/951728206 >> 172.21.15.119:0/951728206 conn(0x7f688c06aba0 msgr2=0x7f688c0ad720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.367+0000 7f6893200640 1 -- 172.21.15.119:0/951728206 shutdown_connections 2024-04-25T10:13:28.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.367+0000 7f6893200640 1 -- 172.21.15.119:0/951728206 wait complete. 2024-04-25T10:13:28.390 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-04-25T10:13:28.051514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":102,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258416,"kb_used_data":8952,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716304,"statfs":{"total":773094113280,"available":772829495296,"internally_reserved":0,"allocated":9166848,"data_stored":4547572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001286"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.018713+0000","last_change":"2024-04-25T10:13:24.010507+0000","last_active":"2024-04-25T10:13:24.018713+0000","last_peered":"2024-04-25T10:13:24.018713+0000","last_clean":"2024-04-25T10:13:24.018713+0000","last_became_active":"2024-04-25T10:13:24.010086+0000","last_became_peered":"2024-04-25T10:13:24.010086+0000","last_unstale":"2024-04-25T10:13:24.018713+0000","last_undegraded":"2024-04-25T10:13:24.018713+0000","last_fullsized":"2024-04-25T10:13:24.018713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:04.603474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.991484+0000","last_change":"2024-04-25T10:13:24.018773+0000","last_active":"2024-04-25T10:13:24.991484+0000","last_peered":"2024-04-25T10:13:24.991484+0000","last_clean":"2024-04-25T10:13:24.991484+0000","last_became_active":"2024-04-25T10:13:24.018471+0000","last_became_peered":"2024-04-25T10:13:24.018471+0000","last_unstale":"2024-04-25T10:13:24.991484+0000","last_undegraded":"2024-04-25T10:13:24.991484+0000","last_fullsized":"2024-04-25T10:13:24.991484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:17.620390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.019562+0000","last_change":"2024-04-25T10:13:24.010249+0000","last_active":"2024-04-25T10:13:24.019562+0000","last_peered":"2024-04-25T10:13:24.019562+0000","last_clean":"2024-04-25T10:13:24.019562+0000","last_became_active":"2024-04-25T10:13:24.009914+0000","last_became_peered":"2024-04-25T10:13:24.009914+0000","last_unstale":"2024-04-25T10:13:24.019562+0000","last_undegraded":"2024-04-25T10:13:24.019562+0000","last_fullsized":"2024-04-25T10:13:24.019562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:47.884140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.991360+0000","last_change":"2024-04-25T10:13:23.999818+0000","last_active":"2024-04-25T10:13:24.991360+0000","last_peered":"2024-04-25T10:13:24.991360+0000","last_clean":"2024-04-25T10:13:24.991360+0000","last_became_active":"2024-04-25T10:13:23.999321+0000","last_became_peered":"2024-04-25T10:13:23.999321+0000","last_unstale":"2024-04-25T10:13:24.991360+0000","last_undegraded":"2024-04-25T10:13:24.991360+0000","last_fullsized":"2024-04-25T10:13:24.991360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:49.058295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.015514+0000","last_change":"2024-04-25T10:13:24.006603+0000","last_active":"2024-04-25T10:13:24.015514+0000","last_peered":"2024-04-25T10:13:24.015514+0000","last_clean":"2024-04-25T10:13:24.015514+0000","last_became_active":"2024-04-25T10:13:24.006316+0000","last_became_peered":"2024-04-25T10:13:24.006316+0000","last_unstale":"2024-04-25T10:13:24.015514+0000","last_undegraded":"2024-04-25T10:13:24.015514+0000","last_fullsized":"2024-04-25T10:13:24.015514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:00.850177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989012+0000","last_change":"2024-04-25T10:13:23.999873+0000","last_active":"2024-04-25T10:13:24.989012+0000","last_peered":"2024-04-25T10:13:24.989012+0000","last_clean":"2024-04-25T10:13:24.989012+0000","last_became_active":"2024-04-25T10:13:23.999404+0000","last_became_peered":"2024-04-25T10:13:23.999404+0000","last_unstale":"2024-04-25T10:13:24.989012+0000","last_undegraded":"2024-04-25T10:13:24.989012+0000","last_fullsized":"2024-04-25T10:13:24.989012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:38.250963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.988205+0000","last_change":"2024-04-25T10:13:24.005263+0000","last_active":"2024-04-25T10:13:24.988205+0000","last_peered":"2024-04-25T10:13:24.988205+0000","last_clean":"2024-04-25T10:13:24.988205+0000","last_became_active":"2024-04-25T10:13:24.003741+0000","last_became_peered":"2024-04-25T10:13:24.003741+0000","last_unstale":"2024-04-25T10:13:24.988205+0000","last_undegraded":"2024-04-25T10:13:24.988205+0000","last_fullsized":"2024-04-25T10:13:24.988205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.427088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.988932+0000","last_change":"2024-04-25T10:13:23.999788+0000","last_active":"2024-04-25T10:13:24.988932+0000","last_peered":"2024-04-25T10:13:24.988932+0000","last_clean":"2024-04-25T10:13:24.988932+0000","last_became_active":"2024-04-25T10:13:23.998715+0000","last_became_peered":"2024-04-25T10:13:23.998715+0000","last_unstale":"2024-04-25T10:13:24.988932+0000","last_undegraded":"2024-04-25T10:13:24.988932+0000","last_fullsized":"2024-04-25T10:13:24.988932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:05.505912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989551+0000","last_change":"2024-04-25T10:13:24.013606+0000","last_active":"2024-04-25T10:13:24.989551+0000","last_peered":"2024-04-25T10:13:24.989551+0000","last_clean":"2024-04-25T10:13:24.989551+0000","last_became_active":"2024-04-25T10:13:24.013081+0000","last_became_peered":"2024-04-25T10:13:24.013081+0000","last_unstale":"2024-04-25T10:13:24.989551+0000","last_undegraded":"2024-04-25T10:13:24.989551+0000","last_fullsized":"2024-04-25T10:13:24.989551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:18.208308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.015507+0000","last_change":"2024-04-25T10:13:24.009693+0000","last_active":"2024-04-25T10:13:24.015507+0000","last_peered":"2024-04-25T10:13:24.015507+0000","last_clean":"2024-04-25T10:13:24.015507+0000","last_became_active":"2024-04-25T10:13:24.009366+0000","last_became_peered":"2024-04-25T10:13:24.009366+0000","last_unstale":"2024-04-25T10:13:24.015507+0000","last_undegraded":"2024-04-25T10:13:24.015507+0000","last_fullsized":"2024-04-25T10:13:24.015507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:03.093656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.990932+0000","last_change":"2024-04-25T10:13:24.009294+0000","last_active":"2024-04-25T10:13:24.990932+0000","last_peered":"2024-04-25T10:13:24.990932+0000","last_clean":"2024-04-25T10:13:24.990932+0000","last_became_active":"2024-04-25T10:13:24.009003+0000","last_became_peered":"2024-04-25T10:13:24.009003+0000","last_unstale":"2024-04-25T10:13:24.990932+0000","last_undegraded":"2024-04-25T10:13:24.990932+0000","last_fullsized":"2024-04-25T10:13:24.990932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:16.064820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.015702+0000","last_change":"2024-04-25T10:13:24.009354+0000","last_active":"2024-04-25T10:13:24.015702+0000","last_peered":"2024-04-25T10:13:24.015702+0000","last_clean":"2024-04-25T10:13:24.015702+0000","last_became_active":"2024-04-25T10:13:24.008996+0000","last_became_peered":"2024-04-25T10:13:24.008996+0000","last_unstale":"2024-04-25T10:13:24.015702+0000","last_undegraded":"2024-04-25T10:13:24.015702+0000","last_fullsized":"2024-04-25T10:13:24.015702+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:13.327909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.021016+0000","last_change":"2024-04-25T10:13:24.012737+0000","last_active":"2024-04-25T10:13:24.021016+0000","last_peered":"2024-04-25T10:13:24.021016+0000","last_clean":"2024-04-25T10:13:24.021016+0000","last_became_active":"2024-04-25T10:13:24.012447+0000","last_became_peered":"2024-04-25T10:13:24.012447+0000","last_unstale":"2024-04-25T10:13:24.021016+0000","last_undegraded":"2024-04-25T10:13:24.021016+0000","last_fullsized":"2024-04-25T10:13:24.021016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:48.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989381+0000","last_change":"2024-04-25T10:13:23.999821+0000","last_active":"2024-04-25T10:13:24.989381+0000","last_peered":"2024-04-25T10:13:24.989381+0000","last_clean":"2024-04-25T10:13:24.989381+0000","last_became_active":"2024-04-25T10:13:23.999538+0000","last_became_peered":"2024-04-25T10:13:23.999538+0000","last_unstale":"2024-04-25T10:13:24.989381+0000","last_undegraded":"2024-04-25T10:13:24.989381+0000","last_fullsized":"2024-04-25T10:13:24.989381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:00.702260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.018725+0000","last_change":"2024-04-25T10:13:24.009945+0000","last_active":"2024-04-25T10:13:24.018725+0000","last_peered":"2024-04-25T10:13:24.018725+0000","last_clean":"2024-04-25T10:13:24.018725+0000","last_became_active":"2024-04-25T10:13:24.009338+0000","last_became_peered":"2024-04-25T10:13:24.009338+0000","last_unstale":"2024-04-25T10:13:24.018725+0000","last_undegraded":"2024-04-25T10:13:24.018725+0000","last_fullsized":"2024-04-25T10:13:24.018725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:24.024737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.022371+0000","last_change":"2024-04-25T10:13:24.016796+0000","last_active":"2024-04-25T10:13:24.022371+0000","last_peered":"2024-04-25T10:13:24.022371+0000","last_clean":"2024-04-25T10:13:24.022371+0000","last_became_active":"2024-04-25T10:13:24.016439+0000","last_became_peered":"2024-04-25T10:13:24.016439+0000","last_unstale":"2024-04-25T10:13:24.022371+0000","last_undegraded":"2024-04-25T10:13:24.022371+0000","last_fullsized":"2024-04-25T10:13:24.022371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:05.505912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.013390+0000","last_change":"2024-04-25T10:13:24.005384+0000","last_active":"2024-04-25T10:13:24.013390+0000","last_peered":"2024-04-25T10:13:24.013390+0000","last_clean":"2024-04-25T10:13:24.013390+0000","last_became_active":"2024-04-25T10:13:24.005084+0000","last_became_peered":"2024-04-25T10:13:24.005084+0000","last_unstale":"2024-04-25T10:13:24.013390+0000","last_undegraded":"2024-04-25T10:13:24.013390+0000","last_fullsized":"2024-04-25T10:13:24.013390+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:04.603474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.990797+0000","last_change":"2024-04-25T10:13:24.006534+0000","last_active":"2024-04-25T10:13:24.990797+0000","last_peered":"2024-04-25T10:13:24.990797+0000","last_clean":"2024-04-25T10:13:24.990797+0000","last_became_active":"2024-04-25T10:13:24.006272+0000","last_became_peered":"2024-04-25T10:13:24.006272+0000","last_unstale":"2024-04-25T10:13:24.990797+0000","last_undegraded":"2024-04-25T10:13:24.990797+0000","last_fullsized":"2024-04-25T10:13:24.990797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:17.329165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.998689+0000","last_change":"2024-04-25T10:13:24.019612+0000","last_active":"2024-04-25T10:13:24.998689+0000","last_peered":"2024-04-25T10:13:24.998689+0000","last_clean":"2024-04-25T10:13:24.998689+0000","last_became_active":"2024-04-25T10:13:24.019345+0000","last_became_peered":"2024-04-25T10:13:24.019345+0000","last_unstale":"2024-04-25T10:13:24.998689+0000","last_undegraded":"2024-04-25T10:13:24.998689+0000","last_fullsized":"2024-04-25T10:13:24.998689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:37.303075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.991542+0000","last_change":"2024-04-25T10:13:24.016722+0000","last_active":"2024-04-25T10:13:24.991542+0000","last_peered":"2024-04-25T10:13:24.991542+0000","last_clean":"2024-04-25T10:13:24.991542+0000","last_became_active":"2024-04-25T10:13:24.013861+0000","last_became_peered":"2024-04-25T10:13:24.013861+0000","last_unstale":"2024-04-25T10:13:24.991542+0000","last_undegraded":"2024-04-25T10:13:24.991542+0000","last_fullsized":"2024-04-25T10:13:24.991542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:29.480916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.991145+0000","last_change":"2024-04-25T10:13:24.014024+0000","last_active":"2024-04-25T10:13:24.991145+0000","last_peered":"2024-04-25T10:13:24.991145+0000","last_clean":"2024-04-25T10:13:24.991145+0000","last_became_active":"2024-04-25T10:13:24.013665+0000","last_became_peered":"2024-04-25T10:13:24.013665+0000","last_unstale":"2024-04-25T10:13:24.991145+0000","last_undegraded":"2024-04-25T10:13:24.991145+0000","last_fullsized":"2024-04-25T10:13:24.991145+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:24.038453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989626+0000","last_change":"2024-04-25T10:13:24.020989+0000","last_active":"2024-04-25T10:13:24.989626+0000","last_peered":"2024-04-25T10:13:24.989626+0000","last_clean":"2024-04-25T10:13:24.989626+0000","last_became_active":"2024-04-25T10:13:24.018973+0000","last_became_peered":"2024-04-25T10:13:24.018973+0000","last_unstale":"2024-04-25T10:13:24.989626+0000","last_undegraded":"2024-04-25T10:13:24.989626+0000","last_fullsized":"2024-04-25T10:13:24.989626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:39.078191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.022468+0000","last_change":"2024-04-25T10:13:24.019153+0000","last_active":"2024-04-25T10:13:24.022468+0000","last_peered":"2024-04-25T10:13:24.022468+0000","last_clean":"2024-04-25T10:13:24.022468+0000","last_became_active":"2024-04-25T10:13:24.018808+0000","last_became_peered":"2024-04-25T10:13:24.018808+0000","last_unstale":"2024-04-25T10:13:24.022468+0000","last_undegraded":"2024-04-25T10:13:24.022468+0000","last_fullsized":"2024-04-25T10:13:24.022468+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:05.505912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.988643+0000","last_change":"2024-04-25T10:13:24.007918+0000","last_active":"2024-04-25T10:13:24.988643+0000","last_peered":"2024-04-25T10:13:24.988643+0000","last_clean":"2024-04-25T10:13:24.988643+0000","last_became_active":"2024-04-25T10:13:24.007615+0000","last_became_peered":"2024-04-25T10:13:24.007615+0000","last_unstale":"2024-04-25T10:13:24.988643+0000","last_undegraded":"2024-04-25T10:13:24.988643+0000","last_fullsized":"2024-04-25T10:13:24.988643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:12.761807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.990299+0000","last_change":"2024-04-25T10:13:24.013766+0000","last_active":"2024-04-25T10:13:24.990299+0000","last_peered":"2024-04-25T10:13:24.990299+0000","last_clean":"2024-04-25T10:13:24.990299+0000","last_became_active":"2024-04-25T10:13:24.013458+0000","last_became_peered":"2024-04-25T10:13:24.013458+0000","last_unstale":"2024-04-25T10:13:24.990299+0000","last_undegraded":"2024-04-25T10:13:24.990299+0000","last_fullsized":"2024-04-25T10:13:24.990299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:26.190357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.015184+0000","last_change":"2024-04-25T10:13:24.011097+0000","last_active":"2024-04-25T10:13:24.015184+0000","last_peered":"2024-04-25T10:13:24.015184+0000","last_clean":"2024-04-25T10:13:24.015184+0000","last_became_active":"2024-04-25T10:13:24.010823+0000","last_became_peered":"2024-04-25T10:13:24.010823+0000","last_unstale":"2024-04-25T10:13:24.015184+0000","last_undegraded":"2024-04-25T10:13:24.015184+0000","last_fullsized":"2024-04-25T10:13:24.015184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:05.505912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.017003+0000","last_change":"2024-04-25T10:13:24.010357+0000","last_active":"2024-04-25T10:13:24.017003+0000","last_peered":"2024-04-25T10:13:24.017003+0000","last_clean":"2024-04-25T10:13:24.017003+0000","last_became_active":"2024-04-25T10:13:24.010072+0000","last_became_peered":"2024-04-25T10:13:24.010072+0000","last_unstale":"2024-04-25T10:13:24.017003+0000","last_undegraded":"2024-04-25T10:13:24.017003+0000","last_fullsized":"2024-04-25T10:13:24.017003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:26.954691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.018907+0000","last_change":"2024-04-25T10:13:24.010940+0000","last_active":"2024-04-25T10:13:24.018907+0000","last_peered":"2024-04-25T10:13:24.018907+0000","last_clean":"2024-04-25T10:13:24.018907+0000","last_became_active":"2024-04-25T10:13:24.010227+0000","last_became_peered":"2024-04-25T10:13:24.010227+0000","last_unstale":"2024-04-25T10:13:24.018907+0000","last_undegraded":"2024-04-25T10:13:24.018907+0000","last_fullsized":"2024-04-25T10:13:24.018907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:00.850177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989345+0000","last_change":"2024-04-25T10:13:23.999935+0000","last_active":"2024-04-25T10:13:24.989345+0000","last_peered":"2024-04-25T10:13:24.989345+0000","last_clean":"2024-04-25T10:13:24.989345+0000","last_became_active":"2024-04-25T10:13:23.999004+0000","last_became_peered":"2024-04-25T10:13:23.999004+0000","last_unstale":"2024-04-25T10:13:24.989345+0000","last_undegraded":"2024-04-25T10:13:24.989345+0000","last_fullsized":"2024-04-25T10:13:24.989345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:15.541886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989947+0000","last_change":"2024-04-25T10:13:24.007528+0000","last_active":"2024-04-25T10:13:24.989947+0000","last_peered":"2024-04-25T10:13:24.989947+0000","last_clean":"2024-04-25T10:13:24.989947+0000","last_became_active":"2024-04-25T10:13:24.007227+0000","last_became_peered":"2024-04-25T10:13:24.007227+0000","last_unstale":"2024-04-25T10:13:24.989947+0000","last_undegraded":"2024-04-25T10:13:24.989947+0000","last_fullsized":"2024-04-25T10:13:24.989947+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:41.394449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.025248+0000","last_change":"2024-04-25T10:13:24.021565+0000","last_active":"2024-04-25T10:13:24.025248+0000","last_peered":"2024-04-25T10:13:24.025248+0000","last_clean":"2024-04-25T10:13:24.025248+0000","last_became_active":"2024-04-25T10:13:24.019295+0000","last_became_peered":"2024-04-25T10:13:24.019295+0000","last_unstale":"2024-04-25T10:13:24.025248+0000","last_undegraded":"2024-04-25T10:13:24.025248+0000","last_fullsized":"2024-04-25T10:13:24.025248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:48.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.019512+0000","last_change":"2024-04-25T10:13:24.012372+0000","last_active":"2024-04-25T10:13:24.019512+0000","last_peered":"2024-04-25T10:13:24.019512+0000","last_clean":"2024-04-25T10:13:24.019512+0000","last_became_active":"2024-04-25T10:13:24.012022+0000","last_became_peered":"2024-04-25T10:13:24.012022+0000","last_unstale":"2024-04-25T10:13:24.019512+0000","last_undegraded":"2024-04-25T10:13:24.019512+0000","last_fullsized":"2024-04-25T10:13:24.019512+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:58.978817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.025379+0000","last_change":"2024-04-25T10:13:24.018840+0000","last_active":"2024-04-25T10:13:24.025379+0000","last_peered":"2024-04-25T10:13:24.025379+0000","last_clean":"2024-04-25T10:13:24.025379+0000","last_became_active":"2024-04-25T10:13:24.016393+0000","last_became_peered":"2024-04-25T10:13:24.016393+0000","last_unstale":"2024-04-25T10:13:24.025379+0000","last_undegraded":"2024-04-25T10:13:24.025379+0000","last_fullsized":"2024-04-25T10:13:24.025379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:38.250963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.030616+0000","last_change":"2024-04-25T10:13:22.998926+0000","last_active":"2024-04-25T10:13:24.030616+0000","last_peered":"2024-04-25T10:13:24.030616+0000","last_clean":"2024-04-25T10:13:24.030616+0000","last_became_active":"2024-04-25T10:13:22.998506+0000","last_became_peered":"2024-04-25T10:13:22.998506+0000","last_unstale":"2024-04-25T10:13:24.030616+0000","last_undegraded":"2024-04-25T10:13:24.030616+0000","last_fullsized":"2024-04-25T10:13:24.030616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:29.194713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.020589+0000","last_change":"2024-04-25T10:13:24.018569+0000","last_active":"2024-04-25T10:13:24.020589+0000","last_peered":"2024-04-25T10:13:24.020589+0000","last_clean":"2024-04-25T10:13:24.020589+0000","last_became_active":"2024-04-25T10:13:24.017194+0000","last_became_peered":"2024-04-25T10:13:24.017194+0000","last_unstale":"2024-04-25T10:13:24.020589+0000","last_undegraded":"2024-04-25T10:13:24.020589+0000","last_fullsized":"2024-04-25T10:13:24.020589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:26.954691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.031082+0000","last_change":"2024-04-25T10:13:23.009419+0000","last_active":"2024-04-25T10:13:24.031082+0000","last_peered":"2024-04-25T10:13:24.031082+0000","last_clean":"2024-04-25T10:13:24.031082+0000","last_became_active":"2024-04-25T10:13:23.008994+0000","last_became_peered":"2024-04-25T10:13:23.008994+0000","last_unstale":"2024-04-25T10:13:24.031082+0000","last_undegraded":"2024-04-25T10:13:24.031082+0000","last_fullsized":"2024-04-25T10:13:24.031082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:10.452376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.018931+0000","last_change":"2024-04-25T10:13:24.014086+0000","last_active":"2024-04-25T10:13:24.018931+0000","last_peered":"2024-04-25T10:13:24.018931+0000","last_clean":"2024-04-25T10:13:24.018931+0000","last_became_active":"2024-04-25T10:13:24.013738+0000","last_became_peered":"2024-04-25T10:13:24.013738+0000","last_unstale":"2024-04-25T10:13:24.018931+0000","last_undegraded":"2024-04-25T10:13:24.018931+0000","last_fullsized":"2024-04-25T10:13:24.018931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:41.926767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.998547+0000","last_change":"2024-04-25T10:13:22.994402+0000","last_active":"2024-04-25T10:13:24.998547+0000","last_peered":"2024-04-25T10:13:24.998547+0000","last_clean":"2024-04-25T10:13:24.998547+0000","last_became_active":"2024-04-25T10:13:22.994131+0000","last_became_peered":"2024-04-25T10:13:22.994131+0000","last_unstale":"2024-04-25T10:13:24.998547+0000","last_undegraded":"2024-04-25T10:13:24.998547+0000","last_fullsized":"2024-04-25T10:13:24.998547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:59.801932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.015071+0000","last_change":"2024-04-25T10:13:24.011323+0000","last_active":"2024-04-25T10:13:24.015071+0000","last_peered":"2024-04-25T10:13:24.015071+0000","last_clean":"2024-04-25T10:13:24.015071+0000","last_became_active":"2024-04-25T10:13:24.009597+0000","last_became_peered":"2024-04-25T10:13:24.009597+0000","last_unstale":"2024-04-25T10:13:24.015071+0000","last_undegraded":"2024-04-25T10:13:24.015071+0000","last_fullsized":"2024-04-25T10:13:24.015071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:38.250963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.998175+0000","last_change":"2024-04-25T10:13:22.993183+0000","last_active":"2024-04-25T10:13:24.998175+0000","last_peered":"2024-04-25T10:13:24.998175+0000","last_clean":"2024-04-25T10:13:24.998175+0000","last_became_active":"2024-04-25T10:13:22.992882+0000","last_became_peered":"2024-04-25T10:13:22.992882+0000","last_unstale":"2024-04-25T10:13:24.998175+0000","last_undegraded":"2024-04-25T10:13:24.998175+0000","last_fullsized":"2024-04-25T10:13:24.998175+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:17.191895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.990328+0000","last_change":"2024-04-25T10:13:24.005952+0000","last_active":"2024-04-25T10:13:24.990328+0000","last_peered":"2024-04-25T10:13:24.990328+0000","last_clean":"2024-04-25T10:13:24.990328+0000","last_became_active":"2024-04-25T10:13:24.005698+0000","last_became_peered":"2024-04-25T10:13:24.005698+0000","last_unstale":"2024-04-25T10:13:24.990328+0000","last_undegraded":"2024-04-25T10:13:24.990328+0000","last_fullsized":"2024-04-25T10:13:24.990328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:30.211125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.032971+0000","last_change":"2024-04-25T10:13:23.008592+0000","last_active":"2024-04-25T10:13:24.032971+0000","last_peered":"2024-04-25T10:13:24.032971+0000","last_clean":"2024-04-25T10:13:24.032971+0000","last_became_active":"2024-04-25T10:13:23.008229+0000","last_became_peered":"2024-04-25T10:13:23.008229+0000","last_unstale":"2024-04-25T10:13:24.032971+0000","last_undegraded":"2024-04-25T10:13:24.032971+0000","last_fullsized":"2024-04-25T10:13:24.032971+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:02.077244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.016325+0000","last_change":"2024-04-25T10:13:24.009987+0000","last_active":"2024-04-25T10:13:24.016325+0000","last_peered":"2024-04-25T10:13:24.016325+0000","last_clean":"2024-04-25T10:13:24.016325+0000","last_became_active":"2024-04-25T10:13:24.008146+0000","last_became_peered":"2024-04-25T10:13:24.008146+0000","last_unstale":"2024-04-25T10:13:24.016325+0000","last_undegraded":"2024-04-25T10:13:24.016325+0000","last_fullsized":"2024-04-25T10:13:24.016325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:41.926767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.028460+0000","last_change":"2024-04-25T10:13:23.009539+0000","last_active":"2024-04-25T10:13:24.028460+0000","last_peered":"2024-04-25T10:13:24.028460+0000","last_clean":"2024-04-25T10:13:24.028460+0000","last_became_active":"2024-04-25T10:13:23.009245+0000","last_became_peered":"2024-04-25T10:13:23.009245+0000","last_unstale":"2024-04-25T10:13:24.028460+0000","last_undegraded":"2024-04-25T10:13:24.028460+0000","last_fullsized":"2024-04-25T10:13:24.028460+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:16.603977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.022091+0000","last_change":"2024-04-25T10:13:24.019434+0000","last_active":"2024-04-25T10:13:24.022091+0000","last_peered":"2024-04-25T10:13:24.022091+0000","last_clean":"2024-04-25T10:13:24.022091+0000","last_became_active":"2024-04-25T10:13:24.019102+0000","last_became_peered":"2024-04-25T10:13:24.019102+0000","last_unstale":"2024-04-25T10:13:24.022091+0000","last_undegraded":"2024-04-25T10:13:24.022091+0000","last_fullsized":"2024-04-25T10:13:24.022091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:40.922603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.997329+0000","last_change":"2024-04-25T10:13:22.996359+0000","last_active":"2024-04-25T10:13:24.997329+0000","last_peered":"2024-04-25T10:13:24.997329+0000","last_clean":"2024-04-25T10:13:24.997329+0000","last_became_active":"2024-04-25T10:13:22.996022+0000","last_became_peered":"2024-04-25T10:13:22.996022+0000","last_unstale":"2024-04-25T10:13:24.997329+0000","last_undegraded":"2024-04-25T10:13:24.997329+0000","last_fullsized":"2024-04-25T10:13:24.997329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:04.489500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.023499+0000","last_change":"2024-04-25T10:13:24.010184+0000","last_active":"2024-04-25T10:13:24.023499+0000","last_peered":"2024-04-25T10:13:24.023499+0000","last_clean":"2024-04-25T10:13:24.023499+0000","last_became_active":"2024-04-25T10:13:24.009685+0000","last_became_peered":"2024-04-25T10:13:24.009685+0000","last_unstale":"2024-04-25T10:13:24.023499+0000","last_undegraded":"2024-04-25T10:13:24.023499+0000","last_fullsized":"2024-04-25T10:13:24.023499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:22.976314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.990249+0000","last_change":"2024-04-25T10:13:23.002238+0000","last_active":"2024-04-25T10:13:24.990249+0000","last_peered":"2024-04-25T10:13:24.990249+0000","last_clean":"2024-04-25T10:13:24.990249+0000","last_became_active":"2024-04-25T10:13:23.001958+0000","last_became_peered":"2024-04-25T10:13:23.001958+0000","last_unstale":"2024-04-25T10:13:24.990249+0000","last_undegraded":"2024-04-25T10:13:24.990249+0000","last_fullsized":"2024-04-25T10:13:24.990249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:16.312752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.018527+0000","last_change":"2024-04-25T10:13:24.008140+0000","last_active":"2024-04-25T10:13:24.018527+0000","last_peered":"2024-04-25T10:13:24.018527+0000","last_clean":"2024-04-25T10:13:24.018527+0000","last_became_active":"2024-04-25T10:13:24.007823+0000","last_became_peered":"2024-04-25T10:13:24.007823+0000","last_unstale":"2024-04-25T10:13:24.018527+0000","last_undegraded":"2024-04-25T10:13:24.018527+0000","last_fullsized":"2024-04-25T10:13:24.018527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:12.448900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.002302+0000","last_change":"2024-04-25T10:13:23.003490+0000","last_active":"2024-04-25T10:13:24.002302+0000","last_peered":"2024-04-25T10:13:24.002302+0000","last_clean":"2024-04-25T10:13:24.002302+0000","last_became_active":"2024-04-25T10:13:23.003184+0000","last_became_peered":"2024-04-25T10:13:23.003184+0000","last_unstale":"2024-04-25T10:13:24.002302+0000","last_undegraded":"2024-04-25T10:13:24.002302+0000","last_fullsized":"2024-04-25T10:13:24.002302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:15.048408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989679+0000","last_change":"2024-04-25T10:13:23.994415+0000","last_active":"2024-04-25T10:13:24.989679+0000","last_peered":"2024-04-25T10:13:24.989679+0000","last_clean":"2024-04-25T10:13:24.989679+0000","last_became_active":"2024-04-25T10:13:23.993798+0000","last_became_peered":"2024-04-25T10:13:23.993798+0000","last_unstale":"2024-04-25T10:13:24.989679+0000","last_undegraded":"2024-04-25T10:13:24.989679+0000","last_fullsized":"2024-04-25T10:13:24.989679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:40.922603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.993630+0000","last_change":"2024-04-25T10:13:23.007596+0000","last_active":"2024-04-25T10:13:23.993630+0000","last_peered":"2024-04-25T10:13:23.993630+0000","last_clean":"2024-04-25T10:13:23.993630+0000","last_became_active":"2024-04-25T10:13:23.007117+0000","last_became_peered":"2024-04-25T10:13:23.007117+0000","last_unstale":"2024-04-25T10:13:23.993630+0000","last_undegraded":"2024-04-25T10:13:23.993630+0000","last_fullsized":"2024-04-25T10:13:23.993630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:25.173945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.998093+0000","last_change":"2024-04-25T10:13:24.012051+0000","last_active":"2024-04-25T10:13:24.998093+0000","last_peered":"2024-04-25T10:13:24.998093+0000","last_clean":"2024-04-25T10:13:24.998093+0000","last_became_active":"2024-04-25T10:13:24.011710+0000","last_became_peered":"2024-04-25T10:13:24.011710+0000","last_unstale":"2024-04-25T10:13:24.998093+0000","last_undegraded":"2024-04-25T10:13:24.998093+0000","last_fullsized":"2024-04-25T10:13:24.998093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:30.728512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.030444+0000","last_change":"2024-04-25T10:13:22.999446+0000","last_active":"2024-04-25T10:13:24.030444+0000","last_peered":"2024-04-25T10:13:24.030444+0000","last_clean":"2024-04-25T10:13:24.030444+0000","last_became_active":"2024-04-25T10:13:22.999158+0000","last_became_peered":"2024-04-25T10:13:22.999158+0000","last_unstale":"2024-04-25T10:13:24.030444+0000","last_undegraded":"2024-04-25T10:13:24.030444+0000","last_fullsized":"2024-04-25T10:13:24.030444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:38.061778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.025303+0000","last_change":"2024-04-25T10:13:24.020074+0000","last_active":"2024-04-25T10:13:24.025303+0000","last_peered":"2024-04-25T10:13:24.025303+0000","last_clean":"2024-04-25T10:13:24.025303+0000","last_became_active":"2024-04-25T10:13:24.019782+0000","last_became_peered":"2024-04-25T10:13:24.019782+0000","last_unstale":"2024-04-25T10:13:24.025303+0000","last_undegraded":"2024-04-25T10:13:24.025303+0000","last_fullsized":"2024-04-25T10:13:24.025303+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:57.015200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"80'6","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989760+0000","last_change":"2024-04-25T10:13:22.993448+0000","last_active":"2024-04-25T10:13:24.989760+0000","last_peered":"2024-04-25T10:13:24.989760+0000","last_clean":"2024-04-25T10:13:24.989760+0000","last_became_active":"2024-04-25T10:13:22.992735+0000","last_became_peered":"2024-04-25T10:13:22.992735+0000","last_unstale":"2024-04-25T10:13:24.989760+0000","last_undegraded":"2024-04-25T10:13:24.989760+0000","last_fullsized":"2024-04-25T10:13:24.989760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:02.077244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.997429+0000","last_change":"2024-04-25T10:13:24.017887+0000","last_active":"2024-04-25T10:13:24.997429+0000","last_peered":"2024-04-25T10:13:24.997429+0000","last_clean":"2024-04-25T10:13:24.997429+0000","last_became_active":"2024-04-25T10:13:24.017417+0000","last_became_peered":"2024-04-25T10:13:24.017417+0000","last_unstale":"2024-04-25T10:13:24.997429+0000","last_undegraded":"2024-04-25T10:13:24.997429+0000","last_fullsized":"2024-04-25T10:13:24.997429+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:21.178705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989548+0000","last_change":"2024-04-25T10:13:22.991745+0000","last_active":"2024-04-25T10:13:24.989548+0000","last_peered":"2024-04-25T10:13:24.989548+0000","last_clean":"2024-04-25T10:13:24.989548+0000","last_became_active":"2024-04-25T10:13:22.991425+0000","last_became_peered":"2024-04-25T10:13:22.991425+0000","last_unstale":"2024-04-25T10:13:24.989548+0000","last_undegraded":"2024-04-25T10:13:24.989548+0000","last_fullsized":"2024-04-25T10:13:24.989548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:23.022040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.022025+0000","last_change":"2024-04-25T10:13:24.018544+0000","last_active":"2024-04-25T10:13:24.022025+0000","last_peered":"2024-04-25T10:13:24.022025+0000","last_clean":"2024-04-25T10:13:24.022025+0000","last_became_active":"2024-04-25T10:13:24.018216+0000","last_became_peered":"2024-04-25T10:13:24.018216+0000","last_unstale":"2024-04-25T10:13:24.022025+0000","last_undegraded":"2024-04-25T10:13:24.022025+0000","last_fullsized":"2024-04-25T10:13:24.022025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:18.208308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.993127+0000","last_change":"2024-04-25T10:13:22.988489+0000","last_active":"2024-04-25T10:13:23.993127+0000","last_peered":"2024-04-25T10:13:23.993127+0000","last_clean":"2024-04-25T10:13:23.993127+0000","last_became_active":"2024-04-25T10:13:22.988215+0000","last_became_peered":"2024-04-25T10:13:22.988215+0000","last_unstale":"2024-04-25T10:13:23.993127+0000","last_undegraded":"2024-04-25T10:13:23.993127+0000","last_fullsized":"2024-04-25T10:13:23.993127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:41.820046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989889+0000","last_change":"2024-04-25T10:13:24.002780+0000","last_active":"2024-04-25T10:13:24.989889+0000","last_peered":"2024-04-25T10:13:24.989889+0000","last_clean":"2024-04-25T10:13:24.989889+0000","last_became_active":"2024-04-25T10:13:24.002404+0000","last_became_peered":"2024-04-25T10:13:24.002404+0000","last_unstale":"2024-04-25T10:13:24.989889+0000","last_undegraded":"2024-04-25T10:13:24.989889+0000","last_fullsized":"2024-04-25T10:13:24.989889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:46.881163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989037+0000","last_change":"2024-04-25T10:13:23.002980+0000","last_active":"2024-04-25T10:13:24.989037+0000","last_peered":"2024-04-25T10:13:24.989037+0000","last_clean":"2024-04-25T10:13:24.989037+0000","last_became_active":"2024-04-25T10:13:23.001528+0000","last_became_peered":"2024-04-25T10:13:23.001528+0000","last_unstale":"2024-04-25T10:13:24.989037+0000","last_undegraded":"2024-04-25T10:13:24.989037+0000","last_fullsized":"2024-04-25T10:13:24.989037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.988954+0000","last_change":"2024-04-25T10:13:24.011392+0000","last_active":"2024-04-25T10:13:24.988954+0000","last_peered":"2024-04-25T10:13:24.988954+0000","last_clean":"2024-04-25T10:13:24.988954+0000","last_became_active":"2024-04-25T10:13:24.008777+0000","last_became_peered":"2024-04-25T10:13:24.008777+0000","last_unstale":"2024-04-25T10:13:24.988954+0000","last_undegraded":"2024-04-25T10:13:24.988954+0000","last_fullsized":"2024-04-25T10:13:24.988954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:39.120682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"80'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.997278+0000","last_change":"2024-04-25T10:13:23.004422+0000","last_active":"2024-04-25T10:13:24.997278+0000","last_peered":"2024-04-25T10:13:24.997278+0000","last_clean":"2024-04-25T10:13:24.997278+0000","last_became_active":"2024-04-25T10:13:23.004015+0000","last_became_peered":"2024-04-25T10:13:23.004015+0000","last_unstale":"2024-04-25T10:13:24.997278+0000","last_undegraded":"2024-04-25T10:13:24.997278+0000","last_fullsized":"2024-04-25T10:13:24.997278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:40.910354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.014491+0000","last_change":"2024-04-25T10:13:24.010254+0000","last_active":"2024-04-25T10:13:24.014491+0000","last_peered":"2024-04-25T10:13:24.014491+0000","last_clean":"2024-04-25T10:13:24.014491+0000","last_became_active":"2024-04-25T10:13:24.009889+0000","last_became_peered":"2024-04-25T10:13:24.009889+0000","last_unstale":"2024-04-25T10:13:24.014491+0000","last_undegraded":"2024-04-25T10:13:24.014491+0000","last_fullsized":"2024-04-25T10:13:24.014491+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:26.954691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.026025+0000","last_change":"2024-04-25T10:13:22.998671+0000","last_active":"2024-04-25T10:13:24.026025+0000","last_peered":"2024-04-25T10:13:24.026025+0000","last_clean":"2024-04-25T10:13:24.026025+0000","last_became_active":"2024-04-25T10:13:22.998346+0000","last_became_peered":"2024-04-25T10:13:22.998346+0000","last_unstale":"2024-04-25T10:13:24.026025+0000","last_undegraded":"2024-04-25T10:13:24.026025+0000","last_fullsized":"2024-04-25T10:13:24.026025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:16.603977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.016659+0000","last_change":"2024-04-25T10:13:24.009842+0000","last_active":"2024-04-25T10:13:24.016659+0000","last_peered":"2024-04-25T10:13:24.016659+0000","last_clean":"2024-04-25T10:13:24.016659+0000","last_became_active":"2024-04-25T10:13:24.009266+0000","last_became_peered":"2024-04-25T10:13:24.009266+0000","last_unstale":"2024-04-25T10:13:24.016659+0000","last_undegraded":"2024-04-25T10:13:24.016659+0000","last_fullsized":"2024-04-25T10:13:24.016659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:11.468788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.998406+0000","last_change":"2024-04-25T10:13:23.006615+0000","last_active":"2024-04-25T10:13:24.998406+0000","last_peered":"2024-04-25T10:13:24.998406+0000","last_clean":"2024-04-25T10:13:24.998406+0000","last_became_active":"2024-04-25T10:13:23.006325+0000","last_became_peered":"2024-04-25T10:13:23.006325+0000","last_unstale":"2024-04-25T10:13:24.998406+0000","last_undegraded":"2024-04-25T10:13:24.998406+0000","last_fullsized":"2024-04-25T10:13:24.998406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:04.548441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.014595+0000","last_change":"2024-04-25T10:13:24.005789+0000","last_active":"2024-04-25T10:13:24.014595+0000","last_peered":"2024-04-25T10:13:24.014595+0000","last_clean":"2024-04-25T10:13:24.014595+0000","last_became_active":"2024-04-25T10:13:24.004638+0000","last_became_peered":"2024-04-25T10:13:24.004638+0000","last_unstale":"2024-04-25T10:13:24.014595+0000","last_undegraded":"2024-04-25T10:13:24.014595+0000","last_fullsized":"2024-04-25T10:13:24.014595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:40.842115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.998344+0000","last_change":"2024-04-25T10:13:22.997925+0000","last_active":"2024-04-25T10:13:24.998344+0000","last_peered":"2024-04-25T10:13:24.998344+0000","last_clean":"2024-04-25T10:13:24.998344+0000","last_became_active":"2024-04-25T10:13:22.997509+0000","last_became_peered":"2024-04-25T10:13:22.997509+0000","last_unstale":"2024-04-25T10:13:24.998344+0000","last_undegraded":"2024-04-25T10:13:24.998344+0000","last_fullsized":"2024-04-25T10:13:24.998344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:39.906191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989736+0000","last_change":"2024-04-25T10:13:23.998240+0000","last_active":"2024-04-25T10:13:24.989736+0000","last_peered":"2024-04-25T10:13:24.989736+0000","last_clean":"2024-04-25T10:13:24.989736+0000","last_became_active":"2024-04-25T10:13:23.997851+0000","last_became_peered":"2024-04-25T10:13:23.997851+0000","last_unstale":"2024-04-25T10:13:24.989736+0000","last_undegraded":"2024-04-25T10:13:24.989736+0000","last_fullsized":"2024-04-25T10:13:24.989736+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:12.335210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.024603+0000","last_change":"2024-04-25T10:13:22.992024+0000","last_active":"2024-04-25T10:13:24.024603+0000","last_peered":"2024-04-25T10:13:24.024603+0000","last_clean":"2024-04-25T10:13:24.024603+0000","last_became_active":"2024-04-25T10:13:22.991694+0000","last_became_peered":"2024-04-25T10:13:22.991694+0000","last_unstale":"2024-04-25T10:13:24.024603+0000","last_undegraded":"2024-04-25T10:13:24.024603+0000","last_fullsized":"2024-04-25T10:13:24.024603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:32.634237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989277+0000","last_change":"2024-04-25T10:13:24.005569+0000","last_active":"2024-04-25T10:13:24.989277+0000","last_peered":"2024-04-25T10:13:24.989277+0000","last_clean":"2024-04-25T10:13:24.989277+0000","last_became_active":"2024-04-25T10:13:24.005216+0000","last_became_peered":"2024-04-25T10:13:24.005216+0000","last_unstale":"2024-04-25T10:13:24.989277+0000","last_undegraded":"2024-04-25T10:13:24.989277+0000","last_fullsized":"2024-04-25T10:13:24.989277+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:47.884140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989833+0000","last_change":"2024-04-25T10:13:22.996032+0000","last_active":"2024-04-25T10:13:24.989833+0000","last_peered":"2024-04-25T10:13:24.989833+0000","last_clean":"2024-04-25T10:13:24.989833+0000","last_became_active":"2024-04-25T10:13:22.994865+0000","last_became_peered":"2024-04-25T10:13:22.994865+0000","last_unstale":"2024-04-25T10:13:24.989833+0000","last_undegraded":"2024-04-25T10:13:24.989833+0000","last_fullsized":"2024-04-25T10:13:24.989833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:17.307630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989029+0000","last_change":"2024-04-25T10:13:23.991686+0000","last_active":"2024-04-25T10:13:24.989029+0000","last_peered":"2024-04-25T10:13:24.989029+0000","last_clean":"2024-04-25T10:13:24.989029+0000","last_became_active":"2024-04-25T10:13:23.991197+0000","last_became_peered":"2024-04-25T10:13:23.991197+0000","last_unstale":"2024-04-25T10:13:24.989029+0000","last_undegraded":"2024-04-25T10:13:24.989029+0000","last_fullsized":"2024-04-25T10:13:24.989029+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:14.229367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.988361+0000","last_change":"2024-04-25T10:13:23.003898+0000","last_active":"2024-04-25T10:13:24.988361+0000","last_peered":"2024-04-25T10:13:24.988361+0000","last_clean":"2024-04-25T10:13:24.988361+0000","last_became_active":"2024-04-25T10:13:23.003646+0000","last_became_peered":"2024-04-25T10:13:23.003646+0000","last_unstale":"2024-04-25T10:13:24.988361+0000","last_undegraded":"2024-04-25T10:13:24.988361+0000","last_fullsized":"2024-04-25T10:13:24.988361+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:22.447906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989288+0000","last_change":"2024-04-25T10:13:24.008647+0000","last_active":"2024-04-25T10:13:24.989288+0000","last_peered":"2024-04-25T10:13:24.989288+0000","last_clean":"2024-04-25T10:13:24.989288+0000","last_became_active":"2024-04-25T10:13:24.008195+0000","last_became_peered":"2024-04-25T10:13:24.008195+0000","last_unstale":"2024-04-25T10:13:24.989288+0000","last_undegraded":"2024-04-25T10:13:24.989288+0000","last_fullsized":"2024-04-25T10:13:24.989288+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:57.015200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.992208+0000","last_change":"2024-04-25T10:13:22.992050+0000","last_active":"2024-04-25T10:13:23.992208+0000","last_peered":"2024-04-25T10:13:23.992208+0000","last_clean":"2024-04-25T10:13:23.992208+0000","last_became_active":"2024-04-25T10:13:22.991057+0000","last_became_peered":"2024-04-25T10:13:22.991057+0000","last_unstale":"2024-04-25T10:13:23.992208+0000","last_undegraded":"2024-04-25T10:13:23.992208+0000","last_fullsized":"2024-04-25T10:13:23.992208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:12.725506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.015562+0000","last_change":"2024-04-25T10:13:24.008381+0000","last_active":"2024-04-25T10:13:24.015562+0000","last_peered":"2024-04-25T10:13:24.015562+0000","last_clean":"2024-04-25T10:13:24.015562+0000","last_became_active":"2024-04-25T10:13:24.008093+0000","last_became_peered":"2024-04-25T10:13:24.008093+0000","last_unstale":"2024-04-25T10:13:24.015562+0000","last_undegraded":"2024-04-25T10:13:24.015562+0000","last_fullsized":"2024-04-25T10:13:24.015562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:58.978817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.990222+0000","last_change":"2024-04-25T10:13:23.004889+0000","last_active":"2024-04-25T10:13:24.990222+0000","last_peered":"2024-04-25T10:13:24.990222+0000","last_clean":"2024-04-25T10:13:24.990222+0000","last_became_active":"2024-04-25T10:13:23.004533+0000","last_became_peered":"2024-04-25T10:13:23.004533+0000","last_unstale":"2024-04-25T10:13:24.990222+0000","last_undegraded":"2024-04-25T10:13:24.990222+0000","last_fullsized":"2024-04-25T10:13:24.990222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:48.041883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.019647+0000","last_change":"2024-04-25T10:13:24.013144+0000","last_active":"2024-04-25T10:13:24.019647+0000","last_peered":"2024-04-25T10:13:24.019647+0000","last_clean":"2024-04-25T10:13:24.019647+0000","last_became_active":"2024-04-25T10:13:24.012739+0000","last_became_peered":"2024-04-25T10:13:24.012739+0000","last_unstale":"2024-04-25T10:13:24.019647+0000","last_undegraded":"2024-04-25T10:13:24.019647+0000","last_fullsized":"2024-04-25T10:13:24.019647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:57.015200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:25.014420+0000","last_change":"2024-04-25T10:13:22.995435+0000","last_active":"2024-04-25T10:13:25.014420+0000","last_peered":"2024-04-25T10:13:25.014420+0000","last_clean":"2024-04-25T10:13:25.014420+0000","last_became_active":"2024-04-25T10:13:22.995080+0000","last_became_peered":"2024-04-25T10:13:22.995080+0000","last_unstale":"2024-04-25T10:13:25.014420+0000","last_undegraded":"2024-04-25T10:13:25.014420+0000","last_fullsized":"2024-04-25T10:13:25.014420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:28.464504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.017422+0000","last_change":"2024-04-25T10:13:24.010507+0000","last_active":"2024-04-25T10:13:24.017422+0000","last_peered":"2024-04-25T10:13:24.017422+0000","last_clean":"2024-04-25T10:13:24.017422+0000","last_became_active":"2024-04-25T10:13:24.009733+0000","last_became_peered":"2024-04-25T10:13:24.009733+0000","last_unstale":"2024-04-25T10:13:24.017422+0000","last_undegraded":"2024-04-25T10:13:24.017422+0000","last_fullsized":"2024-04-25T10:13:24.017422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:40.842115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.992254+0000","last_change":"2024-04-25T10:13:23.003882+0000","last_active":"2024-04-25T10:13:23.992254+0000","last_peered":"2024-04-25T10:13:23.992254+0000","last_clean":"2024-04-25T10:13:23.992254+0000","last_became_active":"2024-04-25T10:13:23.002275+0000","last_became_peered":"2024-04-25T10:13:23.002275+0000","last_unstale":"2024-04-25T10:13:23.992254+0000","last_undegraded":"2024-04-25T10:13:23.992254+0000","last_fullsized":"2024-04-25T10:13:23.992254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:45.864750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.025033+0000","last_change":"2024-04-25T10:13:24.018534+0000","last_active":"2024-04-25T10:13:24.025033+0000","last_peered":"2024-04-25T10:13:24.025033+0000","last_clean":"2024-04-25T10:13:24.025033+0000","last_became_active":"2024-04-25T10:13:24.017879+0000","last_became_peered":"2024-04-25T10:13:24.017879+0000","last_unstale":"2024-04-25T10:13:24.025033+0000","last_undegraded":"2024-04-25T10:13:24.025033+0000","last_fullsized":"2024-04-25T10:13:24.025033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:47.884140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.002188+0000","last_change":"2024-04-25T10:13:23.003835+0000","last_active":"2024-04-25T10:13:24.002188+0000","last_peered":"2024-04-25T10:13:24.002188+0000","last_clean":"2024-04-25T10:13:24.002188+0000","last_became_active":"2024-04-25T10:13:23.003533+0000","last_became_peered":"2024-04-25T10:13:23.003533+0000","last_unstale":"2024-04-25T10:13:24.002188+0000","last_undegraded":"2024-04-25T10:13:24.002188+0000","last_fullsized":"2024-04-25T10:13:24.002188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:46.510168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989413+0000","last_change":"2024-04-25T10:13:24.003742+0000","last_active":"2024-04-25T10:13:24.989413+0000","last_peered":"2024-04-25T10:13:24.989413+0000","last_clean":"2024-04-25T10:13:24.989413+0000","last_became_active":"2024-04-25T10:13:24.003410+0000","last_became_peered":"2024-04-25T10:13:24.003410+0000","last_unstale":"2024-04-25T10:13:24.989413+0000","last_undegraded":"2024-04-25T10:13:24.989413+0000","last_fullsized":"2024-04-25T10:13:24.989413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:48.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.998096+0000","last_change":"2024-04-25T10:13:23.001639+0000","last_active":"2024-04-25T10:13:24.998096+0000","last_peered":"2024-04-25T10:13:24.998096+0000","last_clean":"2024-04-25T10:13:24.998096+0000","last_became_active":"2024-04-25T10:13:23.001368+0000","last_became_peered":"2024-04-25T10:13:23.001368+0000","last_unstale":"2024-04-25T10:13:24.998096+0000","last_undegraded":"2024-04-25T10:13:24.998096+0000","last_fullsized":"2024-04-25T10:13:24.998096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:55.998787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.997838+0000","last_change":"2024-04-25T10:13:24.015722+0000","last_active":"2024-04-25T10:13:24.997838+0000","last_peered":"2024-04-25T10:13:24.997838+0000","last_clean":"2024-04-25T10:13:24.997838+0000","last_became_active":"2024-04-25T10:13:24.013495+0000","last_became_peered":"2024-04-25T10:13:24.013495+0000","last_unstale":"2024-04-25T10:13:24.997838+0000","last_undegraded":"2024-04-25T10:13:24.997838+0000","last_fullsized":"2024-04-25T10:13:24.997838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:16.985205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.993311+0000","last_change":"2024-04-25T10:13:23.008601+0000","last_active":"2024-04-25T10:13:23.993311+0000","last_peered":"2024-04-25T10:13:23.993311+0000","last_clean":"2024-04-25T10:13:23.993311+0000","last_became_active":"2024-04-25T10:13:23.008298+0000","last_became_peered":"2024-04-25T10:13:23.008298+0000","last_unstale":"2024-04-25T10:13:23.993311+0000","last_undegraded":"2024-04-25T10:13:23.993311+0000","last_fullsized":"2024-04-25T10:13:23.993311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.644390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.014602+0000","last_change":"2024-04-25T10:13:24.010223+0000","last_active":"2024-04-25T10:13:24.014602+0000","last_peered":"2024-04-25T10:13:24.014602+0000","last_clean":"2024-04-25T10:13:24.014602+0000","last_became_active":"2024-04-25T10:13:24.009951+0000","last_became_peered":"2024-04-25T10:13:24.009951+0000","last_unstale":"2024-04-25T10:13:24.014602+0000","last_undegraded":"2024-04-25T10:13:24.014602+0000","last_fullsized":"2024-04-25T10:13:24.014602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:41.926767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.993626+0000","last_change":"2024-04-25T10:13:22.995695+0000","last_active":"2024-04-25T10:13:23.993626+0000","last_peered":"2024-04-25T10:13:23.993626+0000","last_clean":"2024-04-25T10:13:23.993626+0000","last_became_active":"2024-04-25T10:13:22.994550+0000","last_became_peered":"2024-04-25T10:13:22.994550+0000","last_unstale":"2024-04-25T10:13:23.993626+0000","last_undegraded":"2024-04-25T10:13:23.993626+0000","last_fullsized":"2024-04-25T10:13:23.993626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:40.378036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.014490+0000","last_change":"2024-04-25T10:13:24.006698+0000","last_active":"2024-04-25T10:13:24.014490+0000","last_peered":"2024-04-25T10:13:24.014490+0000","last_clean":"2024-04-25T10:13:24.014490+0000","last_became_active":"2024-04-25T10:13:24.006147+0000","last_became_peered":"2024-04-25T10:13:24.006147+0000","last_unstale":"2024-04-25T10:13:24.014490+0000","last_undegraded":"2024-04-25T10:13:24.014490+0000","last_fullsized":"2024-04-25T10:13:24.014490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:21.660803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.994407+0000","last_change":"2024-04-25T10:13:23.008216+0000","last_active":"2024-04-25T10:13:23.994407+0000","last_peered":"2024-04-25T10:13:23.994407+0000","last_clean":"2024-04-25T10:13:23.994407+0000","last_became_active":"2024-04-25T10:13:23.007330+0000","last_became_peered":"2024-04-25T10:13:23.007330+0000","last_unstale":"2024-04-25T10:13:23.994407+0000","last_undegraded":"2024-04-25T10:13:23.994407+0000","last_fullsized":"2024-04-25T10:13:23.994407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.990362+0000","last_change":"2024-04-25T10:13:24.001340+0000","last_active":"2024-04-25T10:13:24.990362+0000","last_peered":"2024-04-25T10:13:24.990362+0000","last_clean":"2024-04-25T10:13:24.990362+0000","last_became_active":"2024-04-25T10:13:24.001067+0000","last_became_peered":"2024-04-25T10:13:24.001067+0000","last_unstale":"2024-04-25T10:13:24.990362+0000","last_undegraded":"2024-04-25T10:13:24.990362+0000","last_fullsized":"2024-04-25T10:13:24.990362+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:47.526581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.025989+0000","last_change":"2024-04-25T10:13:23.007164+0000","last_active":"2024-04-25T10:13:24.025989+0000","last_peered":"2024-04-25T10:13:24.025989+0000","last_clean":"2024-04-25T10:13:24.025989+0000","last_became_active":"2024-04-25T10:13:23.006828+0000","last_became_peered":"2024-04-25T10:13:23.006828+0000","last_unstale":"2024-04-25T10:13:24.025989+0000","last_undegraded":"2024-04-25T10:13:24.025989+0000","last_fullsized":"2024-04-25T10:13:24.025989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:57.962405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":283,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.995589+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:13:23.995589+0000","last_peered":"2024-04-25T10:13:23.995589+0000","last_clean":"2024-04-25T10:13:23.995589+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:13:23.995589+0000","last_undegraded":"2024-04-25T10:13:23.995589+0000","last_fullsized":"2024-04-25T10:13:23.995589+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1581056,"data_stored":1007540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":417796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1011712,"data_stored":425036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":983040,"data_stored":417102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":418714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":423984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":423596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1593344,"data_stored":1013804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:28.395 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:28.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.630+0000 7f1a08a00640 1 Processor -- start 2024-04-25T10:13:28.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.631+0000 7f1a08a00640 1 -- start start 2024-04-25T10:13:28.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.632+0000 7f1a08a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a040b5050 0x7f1a0415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.632+0000 7f1a08a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1a0406e430 0x7f1a0415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.632+0000 7f1a08a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a040cd440 con 0x7f1a04144470 2024-04-25T10:13:28.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.632+0000 7f1a08a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a04145340 con 0x7f1a040b5050 2024-04-25T10:13:28.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.632+0000 7f1a08a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a040cd720 con 0x7f1a0406e430 2024-04-25T10:13:28.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.633+0000 7f1a02800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1a04144470 0x7f1a040b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48500/0 (socket says 172.21.15.119:48500) 2024-04-25T10:13:28.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.633+0000 7f1a02800640 1 -- 172.21.15.119:0/3680203007 learned_addr learned my addr 172.21.15.119:0/3680203007 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:28.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.633+0000 7f19f9e00640 1 --2- 172.21.15.119:0/3680203007 >> v2:172.21.15.119:3300/0 conn(0x7f1a040b5050 0x7f1a0415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.633+0000 7f1a03200640 1 --2- 172.21.15.119:0/3680203007 >> v2:172.21.15.132:3300/0 conn(0x7f1a0406e430 0x7f1a0415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.633+0000 7f1a01e00640 1 -- 172.21.15.119:0/3680203007 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295699839 0 0) 0x7f1a040cd440 con 0x7f1a04144470 2024-04-25T10:13:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.633+0000 7f1a01e00640 1 -- 172.21.15.119:0/3680203007 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e8003520 con 0x7f1a04144470 2024-04-25T10:13:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.633+0000 7f19f9e00640 1 -- 172.21.15.119:0/3680203007 >> v2:172.21.15.132:3300/0 conn(0x7f1a0406e430 msgr2=0x7f1a0415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.633+0000 7f19f9e00640 1 --2- 172.21.15.119:0/3680203007 >> v2:172.21.15.132:3300/0 conn(0x7f1a0406e430 0x7f1a0415bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.633+0000 7f19f9e00640 1 -- 172.21.15.119:0/3680203007 >> v1:172.21.15.119:6789/0 conn(0x7f1a04144470 legacy=0x7f1a040b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.633+0000 7f19f9e00640 1 -- 172.21.15.119:0/3680203007 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a040cda00 con 0x7f1a040b5050 2024-04-25T10:13:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.634+0000 7f19f9e00640 1 --2- 172.21.15.119:0/3680203007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a040b5050 0x7f1a0415fc90 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1a0406ebf0 tx=0x7f19ec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265036639f054ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.635+0000 7f1a01e00640 1 -- 172.21.15.119:0/3680203007 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec037070 con 0x7f1a040b5050 2024-04-25T10:13:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.635+0000 7f1a01e00640 1 -- 172.21.15.119:0/3680203007 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec032440 con 0x7f1a040b5050 2024-04-25T10:13:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.635+0000 7f1a01e00640 1 -- 172.21.15.119:0/3680203007 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec032810 con 0x7f1a040b5050 2024-04-25T10:13:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.635+0000 7f1a08a00640 1 -- 172.21.15.119:0/3680203007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a040b5050 msgr2=0x7f1a0415fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.635+0000 7f1a08a00640 1 --2- 172.21.15.119:0/3680203007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a040b5050 0x7f1a0415fc90 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1a0406ebf0 tx=0x7f19ec02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.635+0000 7f1a08a00640 1 -- 172.21.15.119:0/3680203007 shutdown_connections 2024-04-25T10:13:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.635+0000 7f1a08a00640 1 --2- 172.21.15.119:0/3680203007 >> v2:172.21.15.132:3300/0 conn(0x7f1a0406e430 0x7f1a0415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.635+0000 7f1a08a00640 1 --2- 172.21.15.119:0/3680203007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a040b5050 0x7f1a0415fc90 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.635+0000 7f1a08a00640 1 -- 172.21.15.119:0/3680203007 >> 172.21.15.119:0/3680203007 conn(0x7f1a0406b1d0 msgr2=0x7f1a040b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.635+0000 7f1a08a00640 1 -- 172.21.15.119:0/3680203007 shutdown_connections 2024-04-25T10:13:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.635+0000 7f1a08a00640 1 -- 172.21.15.119:0/3680203007 wait complete. 2024-04-25T10:13:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.637+0000 7f1a08a00640 1 Processor -- start 2024-04-25T10:13:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.638+0000 7f1a08a00640 1 -- start start 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.638+0000 7f1a08a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a0406e430 0x7f1a041e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.638+0000 7f1a08a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a040b5050 0x7f1a0413f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.639+0000 7f1a08a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a04144470 0x7f1a0415ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.639+0000 7f1a08a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040ce200 con 0x7f1a04144470 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.639+0000 7f1a08a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a04144850 con 0x7f1a0406e430 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.639+0000 7f1a08a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0415c230 con 0x7f1a040b5050 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.639+0000 7f1a03200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a04144470 0x7f1a0415ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.639+0000 7f1a03200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a04144470 0x7f1a0415ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52736/0 (socket says 172.21.15.119:52736) 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.639+0000 7f1a03200640 1 -- 172.21.15.119:0/3870887693 learned_addr learned my addr 172.21.15.119:0/3870887693 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.639+0000 7f1a02800640 1 --2- 172.21.15.119:0/3870887693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a0406e430 0x7f1a041e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.639+0000 7f19f9e00640 1 --2- 172.21.15.119:0/3870887693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a040b5050 0x7f1a0413f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.639+0000 7f1a03200640 1 -- 172.21.15.119:0/3870887693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a040b5050 msgr2=0x7f1a0413f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.639+0000 7f1a03200640 1 --2- 172.21.15.119:0/3870887693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a040b5050 0x7f1a0413f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.639+0000 7f1a03200640 1 -- 172.21.15.119:0/3870887693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a0406e430 msgr2=0x7f1a041e4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.639+0000 7f1a03200640 1 --2- 172.21.15.119:0/3870887693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a0406e430 0x7f1a041e4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.639+0000 7f1a03200640 1 -- 172.21.15.119:0/3870887693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a041e54f0 con 0x7f1a04144470 2024-04-25T10:13:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.640+0000 7f1a03200640 1 --2- 172.21.15.119:0/3870887693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a04144470 0x7f1a0415ecd0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f19fc007e40 tx=0x7f19fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.640+0000 7f19fb400640 1 -- 172.21.15.119:0/3870887693 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc014620 con 0x7f1a04144470 2024-04-25T10:13:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.640+0000 7f19fb400640 1 -- 172.21.15.119:0/3870887693 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc019070 con 0x7f1a04144470 2024-04-25T10:13:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.640+0000 7f1a08a00640 1 -- 172.21.15.119:0/3870887693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a041e5790 con 0x7f1a04144470 2024-04-25T10:13:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.640+0000 7f19fb400640 1 -- 172.21.15.119:0/3870887693 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc014a20 con 0x7f1a04144470 2024-04-25T10:13:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.640+0000 7f1a08a00640 1 -- 172.21.15.119:0/3870887693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a041e5d50 con 0x7f1a04144470 2024-04-25T10:13:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.641+0000 7f1a08a00640 1 -- 172.21.15.119:0/3870887693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a040749b0 con 0x7f1a04144470 2024-04-25T10:13:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.642+0000 7f19fb400640 1 -- 172.21.15.119:0/3870887693 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19fc014bc0 con 0x7f1a04144470 2024-04-25T10:13:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.643+0000 7f19fb400640 1 --2- 172.21.15.119:0/3870887693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f19e006b500 0x7f19e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.644+0000 7f19fb400640 1 -- 172.21.15.119:0/3870887693 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19fc08aa40 con 0x7f1a04144470 2024-04-25T10:13:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.644+0000 7f1a02800640 1 --2- 172.21.15.119:0/3870887693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f19e006b500 0x7f19e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.644+0000 7f1a02800640 1 --2- 172.21.15.119:0/3870887693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f19e006b500 0x7f19e006d9c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f19f4005880 tx=0x7f19f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.648+0000 7f19fb400640 1 -- 172.21.15.119:0/3870887693 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19fc059950 con 0x7f1a04144470 2024-04-25T10:13:28.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.836+0000 7f1a08a00640 1 -- 172.21.15.119:0/3870887693 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a040b2070 con 0x7f19e006b500 2024-04-25T10:13:28.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.845+0000 7f19fb400640 1 -- 172.21.15.119:0/3870887693 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252039 (secure 0 0 0) 0x7f1a040b2070 con 0x7f19e006b500 2024-04-25T10:13:28.851 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:28.855 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.851+0000 7f1a08a00640 1 -- 172.21.15.119:0/3870887693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f19e006b500 msgr2=0x7f19e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.851+0000 7f1a08a00640 1 --2- 172.21.15.119:0/3870887693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f19e006b500 0x7f19e006d9c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f19f4005880 tx=0x7f19f4009040 comp rx=0 tx=0).stop 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.851+0000 7f1a08a00640 1 -- 172.21.15.119:0/3870887693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a04144470 msgr2=0x7f1a0415ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.851+0000 7f1a08a00640 1 --2- 172.21.15.119:0/3870887693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a04144470 0x7f1a0415ecd0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f19fc007e40 tx=0x7f19fc00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.852+0000 7f1a08a00640 1 -- 172.21.15.119:0/3870887693 shutdown_connections 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.852+0000 7f1a08a00640 1 --2- 172.21.15.119:0/3870887693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f19e006b500 0x7f19e006d9c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.852+0000 7f1a08a00640 1 --2- 172.21.15.119:0/3870887693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a04144470 0x7f1a0415ecd0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.852+0000 7f1a08a00640 1 --2- 172.21.15.119:0/3870887693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a040b5050 0x7f1a0413f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.852+0000 7f1a08a00640 1 --2- 172.21.15.119:0/3870887693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a0406e430 0x7f1a041e4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.852+0000 7f1a08a00640 1 -- 172.21.15.119:0/3870887693 >> 172.21.15.119:0/3870887693 conn(0x7f1a0406b1d0 msgr2=0x7f1a040b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.852+0000 7f1a08a00640 1 -- 172.21.15.119:0/3870887693 shutdown_connections 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:28.852+0000 7f1a08a00640 1 -- 172.21.15.119:0/3870887693 wait complete. 2024-04-25T10:13:28.876 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-04-25T10:13:28.051514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":102,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258416,"kb_used_data":8952,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716304,"statfs":{"total":773094113280,"available":772829495296,"internally_reserved":0,"allocated":9166848,"data_stored":4547572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001286"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.018713+0000","last_change":"2024-04-25T10:13:24.010507+0000","last_active":"2024-04-25T10:13:24.018713+0000","last_peered":"2024-04-25T10:13:24.018713+0000","last_clean":"2024-04-25T10:13:24.018713+0000","last_became_active":"2024-04-25T10:13:24.010086+0000","last_became_peered":"2024-04-25T10:13:24.010086+0000","last_unstale":"2024-04-25T10:13:24.018713+0000","last_undegraded":"2024-04-25T10:13:24.018713+0000","last_fullsized":"2024-04-25T10:13:24.018713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:04.603474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.991484+0000","last_change":"2024-04-25T10:13:24.018773+0000","last_active":"2024-04-25T10:13:24.991484+0000","last_peered":"2024-04-25T10:13:24.991484+0000","last_clean":"2024-04-25T10:13:24.991484+0000","last_became_active":"2024-04-25T10:13:24.018471+0000","last_became_peered":"2024-04-25T10:13:24.018471+0000","last_unstale":"2024-04-25T10:13:24.991484+0000","last_undegraded":"2024-04-25T10:13:24.991484+0000","last_fullsized":"2024-04-25T10:13:24.991484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:17.620390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.019562+0000","last_change":"2024-04-25T10:13:24.010249+0000","last_active":"2024-04-25T10:13:24.019562+0000","last_peered":"2024-04-25T10:13:24.019562+0000","last_clean":"2024-04-25T10:13:24.019562+0000","last_became_active":"2024-04-25T10:13:24.009914+0000","last_became_peered":"2024-04-25T10:13:24.009914+0000","last_unstale":"2024-04-25T10:13:24.019562+0000","last_undegraded":"2024-04-25T10:13:24.019562+0000","last_fullsized":"2024-04-25T10:13:24.019562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:47.884140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.991360+0000","last_change":"2024-04-25T10:13:23.999818+0000","last_active":"2024-04-25T10:13:24.991360+0000","last_peered":"2024-04-25T10:13:24.991360+0000","last_clean":"2024-04-25T10:13:24.991360+0000","last_became_active":"2024-04-25T10:13:23.999321+0000","last_became_peered":"2024-04-25T10:13:23.999321+0000","last_unstale":"2024-04-25T10:13:24.991360+0000","last_undegraded":"2024-04-25T10:13:24.991360+0000","last_fullsized":"2024-04-25T10:13:24.991360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:49.058295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.015514+0000","last_change":"2024-04-25T10:13:24.006603+0000","last_active":"2024-04-25T10:13:24.015514+0000","last_peered":"2024-04-25T10:13:24.015514+0000","last_clean":"2024-04-25T10:13:24.015514+0000","last_became_active":"2024-04-25T10:13:24.006316+0000","last_became_peered":"2024-04-25T10:13:24.006316+0000","last_unstale":"2024-04-25T10:13:24.015514+0000","last_undegraded":"2024-04-25T10:13:24.015514+0000","last_fullsized":"2024-04-25T10:13:24.015514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:00.850177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989012+0000","last_change":"2024-04-25T10:13:23.999873+0000","last_active":"2024-04-25T10:13:24.989012+0000","last_peered":"2024-04-25T10:13:24.989012+0000","last_clean":"2024-04-25T10:13:24.989012+0000","last_became_active":"2024-04-25T10:13:23.999404+0000","last_became_peered":"2024-04-25T10:13:23.999404+0000","last_unstale":"2024-04-25T10:13:24.989012+0000","last_undegraded":"2024-04-25T10:13:24.989012+0000","last_fullsized":"2024-04-25T10:13:24.989012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:38.250963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.988205+0000","last_change":"2024-04-25T10:13:24.005263+0000","last_active":"2024-04-25T10:13:24.988205+0000","last_peered":"2024-04-25T10:13:24.988205+0000","last_clean":"2024-04-25T10:13:24.988205+0000","last_became_active":"2024-04-25T10:13:24.003741+0000","last_became_peered":"2024-04-25T10:13:24.003741+0000","last_unstale":"2024-04-25T10:13:24.988205+0000","last_undegraded":"2024-04-25T10:13:24.988205+0000","last_fullsized":"2024-04-25T10:13:24.988205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.427088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.988932+0000","last_change":"2024-04-25T10:13:23.999788+0000","last_active":"2024-04-25T10:13:24.988932+0000","last_peered":"2024-04-25T10:13:24.988932+0000","last_clean":"2024-04-25T10:13:24.988932+0000","last_became_active":"2024-04-25T10:13:23.998715+0000","last_became_peered":"2024-04-25T10:13:23.998715+0000","last_unstale":"2024-04-25T10:13:24.988932+0000","last_undegraded":"2024-04-25T10:13:24.988932+0000","last_fullsized":"2024-04-25T10:13:24.988932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:05.505912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989551+0000","last_change":"2024-04-25T10:13:24.013606+0000","last_active":"2024-04-25T10:13:24.989551+0000","last_peered":"2024-04-25T10:13:24.989551+0000","last_clean":"2024-04-25T10:13:24.989551+0000","last_became_active":"2024-04-25T10:13:24.013081+0000","last_became_peered":"2024-04-25T10:13:24.013081+0000","last_unstale":"2024-04-25T10:13:24.989551+0000","last_undegraded":"2024-04-25T10:13:24.989551+0000","last_fullsized":"2024-04-25T10:13:24.989551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:18.208308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.015507+0000","last_change":"2024-04-25T10:13:24.009693+0000","last_active":"2024-04-25T10:13:24.015507+0000","last_peered":"2024-04-25T10:13:24.015507+0000","last_clean":"2024-04-25T10:13:24.015507+0000","last_became_active":"2024-04-25T10:13:24.009366+0000","last_became_peered":"2024-04-25T10:13:24.009366+0000","last_unstale":"2024-04-25T10:13:24.015507+0000","last_undegraded":"2024-04-25T10:13:24.015507+0000","last_fullsized":"2024-04-25T10:13:24.015507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:03.093656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.990932+0000","last_change":"2024-04-25T10:13:24.009294+0000","last_active":"2024-04-25T10:13:24.990932+0000","last_peered":"2024-04-25T10:13:24.990932+0000","last_clean":"2024-04-25T10:13:24.990932+0000","last_became_active":"2024-04-25T10:13:24.009003+0000","last_became_peered":"2024-04-25T10:13:24.009003+0000","last_unstale":"2024-04-25T10:13:24.990932+0000","last_undegraded":"2024-04-25T10:13:24.990932+0000","last_fullsized":"2024-04-25T10:13:24.990932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:16.064820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.015702+0000","last_change":"2024-04-25T10:13:24.009354+0000","last_active":"2024-04-25T10:13:24.015702+0000","last_peered":"2024-04-25T10:13:24.015702+0000","last_clean":"2024-04-25T10:13:24.015702+0000","last_became_active":"2024-04-25T10:13:24.008996+0000","last_became_peered":"2024-04-25T10:13:24.008996+0000","last_unstale":"2024-04-25T10:13:24.015702+0000","last_undegraded":"2024-04-25T10:13:24.015702+0000","last_fullsized":"2024-04-25T10:13:24.015702+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:13.327909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.021016+0000","last_change":"2024-04-25T10:13:24.012737+0000","last_active":"2024-04-25T10:13:24.021016+0000","last_peered":"2024-04-25T10:13:24.021016+0000","last_clean":"2024-04-25T10:13:24.021016+0000","last_became_active":"2024-04-25T10:13:24.012447+0000","last_became_peered":"2024-04-25T10:13:24.012447+0000","last_unstale":"2024-04-25T10:13:24.021016+0000","last_undegraded":"2024-04-25T10:13:24.021016+0000","last_fullsized":"2024-04-25T10:13:24.021016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:48.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989381+0000","last_change":"2024-04-25T10:13:23.999821+0000","last_active":"2024-04-25T10:13:24.989381+0000","last_peered":"2024-04-25T10:13:24.989381+0000","last_clean":"2024-04-25T10:13:24.989381+0000","last_became_active":"2024-04-25T10:13:23.999538+0000","last_became_peered":"2024-04-25T10:13:23.999538+0000","last_unstale":"2024-04-25T10:13:24.989381+0000","last_undegraded":"2024-04-25T10:13:24.989381+0000","last_fullsized":"2024-04-25T10:13:24.989381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:00.702260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.018725+0000","last_change":"2024-04-25T10:13:24.009945+0000","last_active":"2024-04-25T10:13:24.018725+0000","last_peered":"2024-04-25T10:13:24.018725+0000","last_clean":"2024-04-25T10:13:24.018725+0000","last_became_active":"2024-04-25T10:13:24.009338+0000","last_became_peered":"2024-04-25T10:13:24.009338+0000","last_unstale":"2024-04-25T10:13:24.018725+0000","last_undegraded":"2024-04-25T10:13:24.018725+0000","last_fullsized":"2024-04-25T10:13:24.018725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:24.024737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.022371+0000","last_change":"2024-04-25T10:13:24.016796+0000","last_active":"2024-04-25T10:13:24.022371+0000","last_peered":"2024-04-25T10:13:24.022371+0000","last_clean":"2024-04-25T10:13:24.022371+0000","last_became_active":"2024-04-25T10:13:24.016439+0000","last_became_peered":"2024-04-25T10:13:24.016439+0000","last_unstale":"2024-04-25T10:13:24.022371+0000","last_undegraded":"2024-04-25T10:13:24.022371+0000","last_fullsized":"2024-04-25T10:13:24.022371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:05.505912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.013390+0000","last_change":"2024-04-25T10:13:24.005384+0000","last_active":"2024-04-25T10:13:24.013390+0000","last_peered":"2024-04-25T10:13:24.013390+0000","last_clean":"2024-04-25T10:13:24.013390+0000","last_became_active":"2024-04-25T10:13:24.005084+0000","last_became_peered":"2024-04-25T10:13:24.005084+0000","last_unstale":"2024-04-25T10:13:24.013390+0000","last_undegraded":"2024-04-25T10:13:24.013390+0000","last_fullsized":"2024-04-25T10:13:24.013390+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:04.603474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.990797+0000","last_change":"2024-04-25T10:13:24.006534+0000","last_active":"2024-04-25T10:13:24.990797+0000","last_peered":"2024-04-25T10:13:24.990797+0000","last_clean":"2024-04-25T10:13:24.990797+0000","last_became_active":"2024-04-25T10:13:24.006272+0000","last_became_peered":"2024-04-25T10:13:24.006272+0000","last_unstale":"2024-04-25T10:13:24.990797+0000","last_undegraded":"2024-04-25T10:13:24.990797+0000","last_fullsized":"2024-04-25T10:13:24.990797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:17.329165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.998689+0000","last_change":"2024-04-25T10:13:24.019612+0000","last_active":"2024-04-25T10:13:24.998689+0000","last_peered":"2024-04-25T10:13:24.998689+0000","last_clean":"2024-04-25T10:13:24.998689+0000","last_became_active":"2024-04-25T10:13:24.019345+0000","last_became_peered":"2024-04-25T10:13:24.019345+0000","last_unstale":"2024-04-25T10:13:24.998689+0000","last_undegraded":"2024-04-25T10:13:24.998689+0000","last_fullsized":"2024-04-25T10:13:24.998689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:37.303075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.991542+0000","last_change":"2024-04-25T10:13:24.016722+0000","last_active":"2024-04-25T10:13:24.991542+0000","last_peered":"2024-04-25T10:13:24.991542+0000","last_clean":"2024-04-25T10:13:24.991542+0000","last_became_active":"2024-04-25T10:13:24.013861+0000","last_became_peered":"2024-04-25T10:13:24.013861+0000","last_unstale":"2024-04-25T10:13:24.991542+0000","last_undegraded":"2024-04-25T10:13:24.991542+0000","last_fullsized":"2024-04-25T10:13:24.991542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:29.480916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.991145+0000","last_change":"2024-04-25T10:13:24.014024+0000","last_active":"2024-04-25T10:13:24.991145+0000","last_peered":"2024-04-25T10:13:24.991145+0000","last_clean":"2024-04-25T10:13:24.991145+0000","last_became_active":"2024-04-25T10:13:24.013665+0000","last_became_peered":"2024-04-25T10:13:24.013665+0000","last_unstale":"2024-04-25T10:13:24.991145+0000","last_undegraded":"2024-04-25T10:13:24.991145+0000","last_fullsized":"2024-04-25T10:13:24.991145+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:24.038453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989626+0000","last_change":"2024-04-25T10:13:24.020989+0000","last_active":"2024-04-25T10:13:24.989626+0000","last_peered":"2024-04-25T10:13:24.989626+0000","last_clean":"2024-04-25T10:13:24.989626+0000","last_became_active":"2024-04-25T10:13:24.018973+0000","last_became_peered":"2024-04-25T10:13:24.018973+0000","last_unstale":"2024-04-25T10:13:24.989626+0000","last_undegraded":"2024-04-25T10:13:24.989626+0000","last_fullsized":"2024-04-25T10:13:24.989626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:39.078191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.022468+0000","last_change":"2024-04-25T10:13:24.019153+0000","last_active":"2024-04-25T10:13:24.022468+0000","last_peered":"2024-04-25T10:13:24.022468+0000","last_clean":"2024-04-25T10:13:24.022468+0000","last_became_active":"2024-04-25T10:13:24.018808+0000","last_became_peered":"2024-04-25T10:13:24.018808+0000","last_unstale":"2024-04-25T10:13:24.022468+0000","last_undegraded":"2024-04-25T10:13:24.022468+0000","last_fullsized":"2024-04-25T10:13:24.022468+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:05.505912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.988643+0000","last_change":"2024-04-25T10:13:24.007918+0000","last_active":"2024-04-25T10:13:24.988643+0000","last_peered":"2024-04-25T10:13:24.988643+0000","last_clean":"2024-04-25T10:13:24.988643+0000","last_became_active":"2024-04-25T10:13:24.007615+0000","last_became_peered":"2024-04-25T10:13:24.007615+0000","last_unstale":"2024-04-25T10:13:24.988643+0000","last_undegraded":"2024-04-25T10:13:24.988643+0000","last_fullsized":"2024-04-25T10:13:24.988643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:12.761807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.990299+0000","last_change":"2024-04-25T10:13:24.013766+0000","last_active":"2024-04-25T10:13:24.990299+0000","last_peered":"2024-04-25T10:13:24.990299+0000","last_clean":"2024-04-25T10:13:24.990299+0000","last_became_active":"2024-04-25T10:13:24.013458+0000","last_became_peered":"2024-04-25T10:13:24.013458+0000","last_unstale":"2024-04-25T10:13:24.990299+0000","last_undegraded":"2024-04-25T10:13:24.990299+0000","last_fullsized":"2024-04-25T10:13:24.990299+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:26.190357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.015184+0000","last_change":"2024-04-25T10:13:24.011097+0000","last_active":"2024-04-25T10:13:24.015184+0000","last_peered":"2024-04-25T10:13:24.015184+0000","last_clean":"2024-04-25T10:13:24.015184+0000","last_became_active":"2024-04-25T10:13:24.010823+0000","last_became_peered":"2024-04-25T10:13:24.010823+0000","last_unstale":"2024-04-25T10:13:24.015184+0000","last_undegraded":"2024-04-25T10:13:24.015184+0000","last_fullsized":"2024-04-25T10:13:24.015184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:05.505912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.017003+0000","last_change":"2024-04-25T10:13:24.010357+0000","last_active":"2024-04-25T10:13:24.017003+0000","last_peered":"2024-04-25T10:13:24.017003+0000","last_clean":"2024-04-25T10:13:24.017003+0000","last_became_active":"2024-04-25T10:13:24.010072+0000","last_became_peered":"2024-04-25T10:13:24.010072+0000","last_unstale":"2024-04-25T10:13:24.017003+0000","last_undegraded":"2024-04-25T10:13:24.017003+0000","last_fullsized":"2024-04-25T10:13:24.017003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:26.954691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.018907+0000","last_change":"2024-04-25T10:13:24.010940+0000","last_active":"2024-04-25T10:13:24.018907+0000","last_peered":"2024-04-25T10:13:24.018907+0000","last_clean":"2024-04-25T10:13:24.018907+0000","last_became_active":"2024-04-25T10:13:24.010227+0000","last_became_peered":"2024-04-25T10:13:24.010227+0000","last_unstale":"2024-04-25T10:13:24.018907+0000","last_undegraded":"2024-04-25T10:13:24.018907+0000","last_fullsized":"2024-04-25T10:13:24.018907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:00.850177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989345+0000","last_change":"2024-04-25T10:13:23.999935+0000","last_active":"2024-04-25T10:13:24.989345+0000","last_peered":"2024-04-25T10:13:24.989345+0000","last_clean":"2024-04-25T10:13:24.989345+0000","last_became_active":"2024-04-25T10:13:23.999004+0000","last_became_peered":"2024-04-25T10:13:23.999004+0000","last_unstale":"2024-04-25T10:13:24.989345+0000","last_undegraded":"2024-04-25T10:13:24.989345+0000","last_fullsized":"2024-04-25T10:13:24.989345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:15.541886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989947+0000","last_change":"2024-04-25T10:13:24.007528+0000","last_active":"2024-04-25T10:13:24.989947+0000","last_peered":"2024-04-25T10:13:24.989947+0000","last_clean":"2024-04-25T10:13:24.989947+0000","last_became_active":"2024-04-25T10:13:24.007227+0000","last_became_peered":"2024-04-25T10:13:24.007227+0000","last_unstale":"2024-04-25T10:13:24.989947+0000","last_undegraded":"2024-04-25T10:13:24.989947+0000","last_fullsized":"2024-04-25T10:13:24.989947+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:41.394449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.025248+0000","last_change":"2024-04-25T10:13:24.021565+0000","last_active":"2024-04-25T10:13:24.025248+0000","last_peered":"2024-04-25T10:13:24.025248+0000","last_clean":"2024-04-25T10:13:24.025248+0000","last_became_active":"2024-04-25T10:13:24.019295+0000","last_became_peered":"2024-04-25T10:13:24.019295+0000","last_unstale":"2024-04-25T10:13:24.025248+0000","last_undegraded":"2024-04-25T10:13:24.025248+0000","last_fullsized":"2024-04-25T10:13:24.025248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:48.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.019512+0000","last_change":"2024-04-25T10:13:24.012372+0000","last_active":"2024-04-25T10:13:24.019512+0000","last_peered":"2024-04-25T10:13:24.019512+0000","last_clean":"2024-04-25T10:13:24.019512+0000","last_became_active":"2024-04-25T10:13:24.012022+0000","last_became_peered":"2024-04-25T10:13:24.012022+0000","last_unstale":"2024-04-25T10:13:24.019512+0000","last_undegraded":"2024-04-25T10:13:24.019512+0000","last_fullsized":"2024-04-25T10:13:24.019512+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:58.978817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.025379+0000","last_change":"2024-04-25T10:13:24.018840+0000","last_active":"2024-04-25T10:13:24.025379+0000","last_peered":"2024-04-25T10:13:24.025379+0000","last_clean":"2024-04-25T10:13:24.025379+0000","last_became_active":"2024-04-25T10:13:24.016393+0000","last_became_peered":"2024-04-25T10:13:24.016393+0000","last_unstale":"2024-04-25T10:13:24.025379+0000","last_undegraded":"2024-04-25T10:13:24.025379+0000","last_fullsized":"2024-04-25T10:13:24.025379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:38.250963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.030616+0000","last_change":"2024-04-25T10:13:22.998926+0000","last_active":"2024-04-25T10:13:24.030616+0000","last_peered":"2024-04-25T10:13:24.030616+0000","last_clean":"2024-04-25T10:13:24.030616+0000","last_became_active":"2024-04-25T10:13:22.998506+0000","last_became_peered":"2024-04-25T10:13:22.998506+0000","last_unstale":"2024-04-25T10:13:24.030616+0000","last_undegraded":"2024-04-25T10:13:24.030616+0000","last_fullsized":"2024-04-25T10:13:24.030616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:29.194713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.020589+0000","last_change":"2024-04-25T10:13:24.018569+0000","last_active":"2024-04-25T10:13:24.020589+0000","last_peered":"2024-04-25T10:13:24.020589+0000","last_clean":"2024-04-25T10:13:24.020589+0000","last_became_active":"2024-04-25T10:13:24.017194+0000","last_became_peered":"2024-04-25T10:13:24.017194+0000","last_unstale":"2024-04-25T10:13:24.020589+0000","last_undegraded":"2024-04-25T10:13:24.020589+0000","last_fullsized":"2024-04-25T10:13:24.020589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:26.954691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.031082+0000","last_change":"2024-04-25T10:13:23.009419+0000","last_active":"2024-04-25T10:13:24.031082+0000","last_peered":"2024-04-25T10:13:24.031082+0000","last_clean":"2024-04-25T10:13:24.031082+0000","last_became_active":"2024-04-25T10:13:23.008994+0000","last_became_peered":"2024-04-25T10:13:23.008994+0000","last_unstale":"2024-04-25T10:13:24.031082+0000","last_undegraded":"2024-04-25T10:13:24.031082+0000","last_fullsized":"2024-04-25T10:13:24.031082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:10.452376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.018931+0000","last_change":"2024-04-25T10:13:24.014086+0000","last_active":"2024-04-25T10:13:24.018931+0000","last_peered":"2024-04-25T10:13:24.018931+0000","last_clean":"2024-04-25T10:13:24.018931+0000","last_became_active":"2024-04-25T10:13:24.013738+0000","last_became_peered":"2024-04-25T10:13:24.013738+0000","last_unstale":"2024-04-25T10:13:24.018931+0000","last_undegraded":"2024-04-25T10:13:24.018931+0000","last_fullsized":"2024-04-25T10:13:24.018931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:41.926767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.998547+0000","last_change":"2024-04-25T10:13:22.994402+0000","last_active":"2024-04-25T10:13:24.998547+0000","last_peered":"2024-04-25T10:13:24.998547+0000","last_clean":"2024-04-25T10:13:24.998547+0000","last_became_active":"2024-04-25T10:13:22.994131+0000","last_became_peered":"2024-04-25T10:13:22.994131+0000","last_unstale":"2024-04-25T10:13:24.998547+0000","last_undegraded":"2024-04-25T10:13:24.998547+0000","last_fullsized":"2024-04-25T10:13:24.998547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:59.801932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.015071+0000","last_change":"2024-04-25T10:13:24.011323+0000","last_active":"2024-04-25T10:13:24.015071+0000","last_peered":"2024-04-25T10:13:24.015071+0000","last_clean":"2024-04-25T10:13:24.015071+0000","last_became_active":"2024-04-25T10:13:24.009597+0000","last_became_peered":"2024-04-25T10:13:24.009597+0000","last_unstale":"2024-04-25T10:13:24.015071+0000","last_undegraded":"2024-04-25T10:13:24.015071+0000","last_fullsized":"2024-04-25T10:13:24.015071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:38.250963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.998175+0000","last_change":"2024-04-25T10:13:22.993183+0000","last_active":"2024-04-25T10:13:24.998175+0000","last_peered":"2024-04-25T10:13:24.998175+0000","last_clean":"2024-04-25T10:13:24.998175+0000","last_became_active":"2024-04-25T10:13:22.992882+0000","last_became_peered":"2024-04-25T10:13:22.992882+0000","last_unstale":"2024-04-25T10:13:24.998175+0000","last_undegraded":"2024-04-25T10:13:24.998175+0000","last_fullsized":"2024-04-25T10:13:24.998175+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:17.191895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.990328+0000","last_change":"2024-04-25T10:13:24.005952+0000","last_active":"2024-04-25T10:13:24.990328+0000","last_peered":"2024-04-25T10:13:24.990328+0000","last_clean":"2024-04-25T10:13:24.990328+0000","last_became_active":"2024-04-25T10:13:24.005698+0000","last_became_peered":"2024-04-25T10:13:24.005698+0000","last_unstale":"2024-04-25T10:13:24.990328+0000","last_undegraded":"2024-04-25T10:13:24.990328+0000","last_fullsized":"2024-04-25T10:13:24.990328+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:30.211125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.032971+0000","last_change":"2024-04-25T10:13:23.008592+0000","last_active":"2024-04-25T10:13:24.032971+0000","last_peered":"2024-04-25T10:13:24.032971+0000","last_clean":"2024-04-25T10:13:24.032971+0000","last_became_active":"2024-04-25T10:13:23.008229+0000","last_became_peered":"2024-04-25T10:13:23.008229+0000","last_unstale":"2024-04-25T10:13:24.032971+0000","last_undegraded":"2024-04-25T10:13:24.032971+0000","last_fullsized":"2024-04-25T10:13:24.032971+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:02.077244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.016325+0000","last_change":"2024-04-25T10:13:24.009987+0000","last_active":"2024-04-25T10:13:24.016325+0000","last_peered":"2024-04-25T10:13:24.016325+0000","last_clean":"2024-04-25T10:13:24.016325+0000","last_became_active":"2024-04-25T10:13:24.008146+0000","last_became_peered":"2024-04-25T10:13:24.008146+0000","last_unstale":"2024-04-25T10:13:24.016325+0000","last_undegraded":"2024-04-25T10:13:24.016325+0000","last_fullsized":"2024-04-25T10:13:24.016325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:41.926767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.028460+0000","last_change":"2024-04-25T10:13:23.009539+0000","last_active":"2024-04-25T10:13:24.028460+0000","last_peered":"2024-04-25T10:13:24.028460+0000","last_clean":"2024-04-25T10:13:24.028460+0000","last_became_active":"2024-04-25T10:13:23.009245+0000","last_became_peered":"2024-04-25T10:13:23.009245+0000","last_unstale":"2024-04-25T10:13:24.028460+0000","last_undegraded":"2024-04-25T10:13:24.028460+0000","last_fullsized":"2024-04-25T10:13:24.028460+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:16.603977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.022091+0000","last_change":"2024-04-25T10:13:24.019434+0000","last_active":"2024-04-25T10:13:24.022091+0000","last_peered":"2024-04-25T10:13:24.022091+0000","last_clean":"2024-04-25T10:13:24.022091+0000","last_became_active":"2024-04-25T10:13:24.019102+0000","last_became_peered":"2024-04-25T10:13:24.019102+0000","last_unstale":"2024-04-25T10:13:24.022091+0000","last_undegraded":"2024-04-25T10:13:24.022091+0000","last_fullsized":"2024-04-25T10:13:24.022091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:40.922603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.997329+0000","last_change":"2024-04-25T10:13:22.996359+0000","last_active":"2024-04-25T10:13:24.997329+0000","last_peered":"2024-04-25T10:13:24.997329+0000","last_clean":"2024-04-25T10:13:24.997329+0000","last_became_active":"2024-04-25T10:13:22.996022+0000","last_became_peered":"2024-04-25T10:13:22.996022+0000","last_unstale":"2024-04-25T10:13:24.997329+0000","last_undegraded":"2024-04-25T10:13:24.997329+0000","last_fullsized":"2024-04-25T10:13:24.997329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:04.489500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.023499+0000","last_change":"2024-04-25T10:13:24.010184+0000","last_active":"2024-04-25T10:13:24.023499+0000","last_peered":"2024-04-25T10:13:24.023499+0000","last_clean":"2024-04-25T10:13:24.023499+0000","last_became_active":"2024-04-25T10:13:24.009685+0000","last_became_peered":"2024-04-25T10:13:24.009685+0000","last_unstale":"2024-04-25T10:13:24.023499+0000","last_undegraded":"2024-04-25T10:13:24.023499+0000","last_fullsized":"2024-04-25T10:13:24.023499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:22.976314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.990249+0000","last_change":"2024-04-25T10:13:23.002238+0000","last_active":"2024-04-25T10:13:24.990249+0000","last_peered":"2024-04-25T10:13:24.990249+0000","last_clean":"2024-04-25T10:13:24.990249+0000","last_became_active":"2024-04-25T10:13:23.001958+0000","last_became_peered":"2024-04-25T10:13:23.001958+0000","last_unstale":"2024-04-25T10:13:24.990249+0000","last_undegraded":"2024-04-25T10:13:24.990249+0000","last_fullsized":"2024-04-25T10:13:24.990249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:16.312752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.018527+0000","last_change":"2024-04-25T10:13:24.008140+0000","last_active":"2024-04-25T10:13:24.018527+0000","last_peered":"2024-04-25T10:13:24.018527+0000","last_clean":"2024-04-25T10:13:24.018527+0000","last_became_active":"2024-04-25T10:13:24.007823+0000","last_became_peered":"2024-04-25T10:13:24.007823+0000","last_unstale":"2024-04-25T10:13:24.018527+0000","last_undegraded":"2024-04-25T10:13:24.018527+0000","last_fullsized":"2024-04-25T10:13:24.018527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:12.448900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.002302+0000","last_change":"2024-04-25T10:13:23.003490+0000","last_active":"2024-04-25T10:13:24.002302+0000","last_peered":"2024-04-25T10:13:24.002302+0000","last_clean":"2024-04-25T10:13:24.002302+0000","last_became_active":"2024-04-25T10:13:23.003184+0000","last_became_peered":"2024-04-25T10:13:23.003184+0000","last_unstale":"2024-04-25T10:13:24.002302+0000","last_undegraded":"2024-04-25T10:13:24.002302+0000","last_fullsized":"2024-04-25T10:13:24.002302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:15.048408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989679+0000","last_change":"2024-04-25T10:13:23.994415+0000","last_active":"2024-04-25T10:13:24.989679+0000","last_peered":"2024-04-25T10:13:24.989679+0000","last_clean":"2024-04-25T10:13:24.989679+0000","last_became_active":"2024-04-25T10:13:23.993798+0000","last_became_peered":"2024-04-25T10:13:23.993798+0000","last_unstale":"2024-04-25T10:13:24.989679+0000","last_undegraded":"2024-04-25T10:13:24.989679+0000","last_fullsized":"2024-04-25T10:13:24.989679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:40.922603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.993630+0000","last_change":"2024-04-25T10:13:23.007596+0000","last_active":"2024-04-25T10:13:23.993630+0000","last_peered":"2024-04-25T10:13:23.993630+0000","last_clean":"2024-04-25T10:13:23.993630+0000","last_became_active":"2024-04-25T10:13:23.007117+0000","last_became_peered":"2024-04-25T10:13:23.007117+0000","last_unstale":"2024-04-25T10:13:23.993630+0000","last_undegraded":"2024-04-25T10:13:23.993630+0000","last_fullsized":"2024-04-25T10:13:23.993630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:25.173945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.998093+0000","last_change":"2024-04-25T10:13:24.012051+0000","last_active":"2024-04-25T10:13:24.998093+0000","last_peered":"2024-04-25T10:13:24.998093+0000","last_clean":"2024-04-25T10:13:24.998093+0000","last_became_active":"2024-04-25T10:13:24.011710+0000","last_became_peered":"2024-04-25T10:13:24.011710+0000","last_unstale":"2024-04-25T10:13:24.998093+0000","last_undegraded":"2024-04-25T10:13:24.998093+0000","last_fullsized":"2024-04-25T10:13:24.998093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:30.728512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.030444+0000","last_change":"2024-04-25T10:13:22.999446+0000","last_active":"2024-04-25T10:13:24.030444+0000","last_peered":"2024-04-25T10:13:24.030444+0000","last_clean":"2024-04-25T10:13:24.030444+0000","last_became_active":"2024-04-25T10:13:22.999158+0000","last_became_peered":"2024-04-25T10:13:22.999158+0000","last_unstale":"2024-04-25T10:13:24.030444+0000","last_undegraded":"2024-04-25T10:13:24.030444+0000","last_fullsized":"2024-04-25T10:13:24.030444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:38.061778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.025303+0000","last_change":"2024-04-25T10:13:24.020074+0000","last_active":"2024-04-25T10:13:24.025303+0000","last_peered":"2024-04-25T10:13:24.025303+0000","last_clean":"2024-04-25T10:13:24.025303+0000","last_became_active":"2024-04-25T10:13:24.019782+0000","last_became_peered":"2024-04-25T10:13:24.019782+0000","last_unstale":"2024-04-25T10:13:24.025303+0000","last_undegraded":"2024-04-25T10:13:24.025303+0000","last_fullsized":"2024-04-25T10:13:24.025303+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:57.015200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"80'6","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989760+0000","last_change":"2024-04-25T10:13:22.993448+0000","last_active":"2024-04-25T10:13:24.989760+0000","last_peered":"2024-04-25T10:13:24.989760+0000","last_clean":"2024-04-25T10:13:24.989760+0000","last_became_active":"2024-04-25T10:13:22.992735+0000","last_became_peered":"2024-04-25T10:13:22.992735+0000","last_unstale":"2024-04-25T10:13:24.989760+0000","last_undegraded":"2024-04-25T10:13:24.989760+0000","last_fullsized":"2024-04-25T10:13:24.989760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:02.077244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.997429+0000","last_change":"2024-04-25T10:13:24.017887+0000","last_active":"2024-04-25T10:13:24.997429+0000","last_peered":"2024-04-25T10:13:24.997429+0000","last_clean":"2024-04-25T10:13:24.997429+0000","last_became_active":"2024-04-25T10:13:24.017417+0000","last_became_peered":"2024-04-25T10:13:24.017417+0000","last_unstale":"2024-04-25T10:13:24.997429+0000","last_undegraded":"2024-04-25T10:13:24.997429+0000","last_fullsized":"2024-04-25T10:13:24.997429+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:21.178705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989548+0000","last_change":"2024-04-25T10:13:22.991745+0000","last_active":"2024-04-25T10:13:24.989548+0000","last_peered":"2024-04-25T10:13:24.989548+0000","last_clean":"2024-04-25T10:13:24.989548+0000","last_became_active":"2024-04-25T10:13:22.991425+0000","last_became_peered":"2024-04-25T10:13:22.991425+0000","last_unstale":"2024-04-25T10:13:24.989548+0000","last_undegraded":"2024-04-25T10:13:24.989548+0000","last_fullsized":"2024-04-25T10:13:24.989548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:23.022040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.022025+0000","last_change":"2024-04-25T10:13:24.018544+0000","last_active":"2024-04-25T10:13:24.022025+0000","last_peered":"2024-04-25T10:13:24.022025+0000","last_clean":"2024-04-25T10:13:24.022025+0000","last_became_active":"2024-04-25T10:13:24.018216+0000","last_became_peered":"2024-04-25T10:13:24.018216+0000","last_unstale":"2024-04-25T10:13:24.022025+0000","last_undegraded":"2024-04-25T10:13:24.022025+0000","last_fullsized":"2024-04-25T10:13:24.022025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:18.208308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.993127+0000","last_change":"2024-04-25T10:13:22.988489+0000","last_active":"2024-04-25T10:13:23.993127+0000","last_peered":"2024-04-25T10:13:23.993127+0000","last_clean":"2024-04-25T10:13:23.993127+0000","last_became_active":"2024-04-25T10:13:22.988215+0000","last_became_peered":"2024-04-25T10:13:22.988215+0000","last_unstale":"2024-04-25T10:13:23.993127+0000","last_undegraded":"2024-04-25T10:13:23.993127+0000","last_fullsized":"2024-04-25T10:13:23.993127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:41.820046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989889+0000","last_change":"2024-04-25T10:13:24.002780+0000","last_active":"2024-04-25T10:13:24.989889+0000","last_peered":"2024-04-25T10:13:24.989889+0000","last_clean":"2024-04-25T10:13:24.989889+0000","last_became_active":"2024-04-25T10:13:24.002404+0000","last_became_peered":"2024-04-25T10:13:24.002404+0000","last_unstale":"2024-04-25T10:13:24.989889+0000","last_undegraded":"2024-04-25T10:13:24.989889+0000","last_fullsized":"2024-04-25T10:13:24.989889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:46.881163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989037+0000","last_change":"2024-04-25T10:13:23.002980+0000","last_active":"2024-04-25T10:13:24.989037+0000","last_peered":"2024-04-25T10:13:24.989037+0000","last_clean":"2024-04-25T10:13:24.989037+0000","last_became_active":"2024-04-25T10:13:23.001528+0000","last_became_peered":"2024-04-25T10:13:23.001528+0000","last_unstale":"2024-04-25T10:13:24.989037+0000","last_undegraded":"2024-04-25T10:13:24.989037+0000","last_fullsized":"2024-04-25T10:13:24.989037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.988954+0000","last_change":"2024-04-25T10:13:24.011392+0000","last_active":"2024-04-25T10:13:24.988954+0000","last_peered":"2024-04-25T10:13:24.988954+0000","last_clean":"2024-04-25T10:13:24.988954+0000","last_became_active":"2024-04-25T10:13:24.008777+0000","last_became_peered":"2024-04-25T10:13:24.008777+0000","last_unstale":"2024-04-25T10:13:24.988954+0000","last_undegraded":"2024-04-25T10:13:24.988954+0000","last_fullsized":"2024-04-25T10:13:24.988954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:39.120682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"80'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.997278+0000","last_change":"2024-04-25T10:13:23.004422+0000","last_active":"2024-04-25T10:13:24.997278+0000","last_peered":"2024-04-25T10:13:24.997278+0000","last_clean":"2024-04-25T10:13:24.997278+0000","last_became_active":"2024-04-25T10:13:23.004015+0000","last_became_peered":"2024-04-25T10:13:23.004015+0000","last_unstale":"2024-04-25T10:13:24.997278+0000","last_undegraded":"2024-04-25T10:13:24.997278+0000","last_fullsized":"2024-04-25T10:13:24.997278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:40.910354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.014491+0000","last_change":"2024-04-25T10:13:24.010254+0000","last_active":"2024-04-25T10:13:24.014491+0000","last_peered":"2024-04-25T10:13:24.014491+0000","last_clean":"2024-04-25T10:13:24.014491+0000","last_became_active":"2024-04-25T10:13:24.009889+0000","last_became_peered":"2024-04-25T10:13:24.009889+0000","last_unstale":"2024-04-25T10:13:24.014491+0000","last_undegraded":"2024-04-25T10:13:24.014491+0000","last_fullsized":"2024-04-25T10:13:24.014491+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:26.954691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.026025+0000","last_change":"2024-04-25T10:13:22.998671+0000","last_active":"2024-04-25T10:13:24.026025+0000","last_peered":"2024-04-25T10:13:24.026025+0000","last_clean":"2024-04-25T10:13:24.026025+0000","last_became_active":"2024-04-25T10:13:22.998346+0000","last_became_peered":"2024-04-25T10:13:22.998346+0000","last_unstale":"2024-04-25T10:13:24.026025+0000","last_undegraded":"2024-04-25T10:13:24.026025+0000","last_fullsized":"2024-04-25T10:13:24.026025+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:16.603977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.016659+0000","last_change":"2024-04-25T10:13:24.009842+0000","last_active":"2024-04-25T10:13:24.016659+0000","last_peered":"2024-04-25T10:13:24.016659+0000","last_clean":"2024-04-25T10:13:24.016659+0000","last_became_active":"2024-04-25T10:13:24.009266+0000","last_became_peered":"2024-04-25T10:13:24.009266+0000","last_unstale":"2024-04-25T10:13:24.016659+0000","last_undegraded":"2024-04-25T10:13:24.016659+0000","last_fullsized":"2024-04-25T10:13:24.016659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:11.468788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.998406+0000","last_change":"2024-04-25T10:13:23.006615+0000","last_active":"2024-04-25T10:13:24.998406+0000","last_peered":"2024-04-25T10:13:24.998406+0000","last_clean":"2024-04-25T10:13:24.998406+0000","last_became_active":"2024-04-25T10:13:23.006325+0000","last_became_peered":"2024-04-25T10:13:23.006325+0000","last_unstale":"2024-04-25T10:13:24.998406+0000","last_undegraded":"2024-04-25T10:13:24.998406+0000","last_fullsized":"2024-04-25T10:13:24.998406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:04.548441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.014595+0000","last_change":"2024-04-25T10:13:24.005789+0000","last_active":"2024-04-25T10:13:24.014595+0000","last_peered":"2024-04-25T10:13:24.014595+0000","last_clean":"2024-04-25T10:13:24.014595+0000","last_became_active":"2024-04-25T10:13:24.004638+0000","last_became_peered":"2024-04-25T10:13:24.004638+0000","last_unstale":"2024-04-25T10:13:24.014595+0000","last_undegraded":"2024-04-25T10:13:24.014595+0000","last_fullsized":"2024-04-25T10:13:24.014595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:40.842115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.998344+0000","last_change":"2024-04-25T10:13:22.997925+0000","last_active":"2024-04-25T10:13:24.998344+0000","last_peered":"2024-04-25T10:13:24.998344+0000","last_clean":"2024-04-25T10:13:24.998344+0000","last_became_active":"2024-04-25T10:13:22.997509+0000","last_became_peered":"2024-04-25T10:13:22.997509+0000","last_unstale":"2024-04-25T10:13:24.998344+0000","last_undegraded":"2024-04-25T10:13:24.998344+0000","last_fullsized":"2024-04-25T10:13:24.998344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:39.906191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989736+0000","last_change":"2024-04-25T10:13:23.998240+0000","last_active":"2024-04-25T10:13:24.989736+0000","last_peered":"2024-04-25T10:13:24.989736+0000","last_clean":"2024-04-25T10:13:24.989736+0000","last_became_active":"2024-04-25T10:13:23.997851+0000","last_became_peered":"2024-04-25T10:13:23.997851+0000","last_unstale":"2024-04-25T10:13:24.989736+0000","last_undegraded":"2024-04-25T10:13:24.989736+0000","last_fullsized":"2024-04-25T10:13:24.989736+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:12.335210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.024603+0000","last_change":"2024-04-25T10:13:22.992024+0000","last_active":"2024-04-25T10:13:24.024603+0000","last_peered":"2024-04-25T10:13:24.024603+0000","last_clean":"2024-04-25T10:13:24.024603+0000","last_became_active":"2024-04-25T10:13:22.991694+0000","last_became_peered":"2024-04-25T10:13:22.991694+0000","last_unstale":"2024-04-25T10:13:24.024603+0000","last_undegraded":"2024-04-25T10:13:24.024603+0000","last_fullsized":"2024-04-25T10:13:24.024603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:32.634237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989277+0000","last_change":"2024-04-25T10:13:24.005569+0000","last_active":"2024-04-25T10:13:24.989277+0000","last_peered":"2024-04-25T10:13:24.989277+0000","last_clean":"2024-04-25T10:13:24.989277+0000","last_became_active":"2024-04-25T10:13:24.005216+0000","last_became_peered":"2024-04-25T10:13:24.005216+0000","last_unstale":"2024-04-25T10:13:24.989277+0000","last_undegraded":"2024-04-25T10:13:24.989277+0000","last_fullsized":"2024-04-25T10:13:24.989277+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:47.884140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989833+0000","last_change":"2024-04-25T10:13:22.996032+0000","last_active":"2024-04-25T10:13:24.989833+0000","last_peered":"2024-04-25T10:13:24.989833+0000","last_clean":"2024-04-25T10:13:24.989833+0000","last_became_active":"2024-04-25T10:13:22.994865+0000","last_became_peered":"2024-04-25T10:13:22.994865+0000","last_unstale":"2024-04-25T10:13:24.989833+0000","last_undegraded":"2024-04-25T10:13:24.989833+0000","last_fullsized":"2024-04-25T10:13:24.989833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:17.307630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989029+0000","last_change":"2024-04-25T10:13:23.991686+0000","last_active":"2024-04-25T10:13:24.989029+0000","last_peered":"2024-04-25T10:13:24.989029+0000","last_clean":"2024-04-25T10:13:24.989029+0000","last_became_active":"2024-04-25T10:13:23.991197+0000","last_became_peered":"2024-04-25T10:13:23.991197+0000","last_unstale":"2024-04-25T10:13:24.989029+0000","last_undegraded":"2024-04-25T10:13:24.989029+0000","last_fullsized":"2024-04-25T10:13:24.989029+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:14.229367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.988361+0000","last_change":"2024-04-25T10:13:23.003898+0000","last_active":"2024-04-25T10:13:24.988361+0000","last_peered":"2024-04-25T10:13:24.988361+0000","last_clean":"2024-04-25T10:13:24.988361+0000","last_became_active":"2024-04-25T10:13:23.003646+0000","last_became_peered":"2024-04-25T10:13:23.003646+0000","last_unstale":"2024-04-25T10:13:24.988361+0000","last_undegraded":"2024-04-25T10:13:24.988361+0000","last_fullsized":"2024-04-25T10:13:24.988361+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:22.447906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989288+0000","last_change":"2024-04-25T10:13:24.008647+0000","last_active":"2024-04-25T10:13:24.989288+0000","last_peered":"2024-04-25T10:13:24.989288+0000","last_clean":"2024-04-25T10:13:24.989288+0000","last_became_active":"2024-04-25T10:13:24.008195+0000","last_became_peered":"2024-04-25T10:13:24.008195+0000","last_unstale":"2024-04-25T10:13:24.989288+0000","last_undegraded":"2024-04-25T10:13:24.989288+0000","last_fullsized":"2024-04-25T10:13:24.989288+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:57.015200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.992208+0000","last_change":"2024-04-25T10:13:22.992050+0000","last_active":"2024-04-25T10:13:23.992208+0000","last_peered":"2024-04-25T10:13:23.992208+0000","last_clean":"2024-04-25T10:13:23.992208+0000","last_became_active":"2024-04-25T10:13:22.991057+0000","last_became_peered":"2024-04-25T10:13:22.991057+0000","last_unstale":"2024-04-25T10:13:23.992208+0000","last_undegraded":"2024-04-25T10:13:23.992208+0000","last_fullsized":"2024-04-25T10:13:23.992208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:12.725506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.015562+0000","last_change":"2024-04-25T10:13:24.008381+0000","last_active":"2024-04-25T10:13:24.015562+0000","last_peered":"2024-04-25T10:13:24.015562+0000","last_clean":"2024-04-25T10:13:24.015562+0000","last_became_active":"2024-04-25T10:13:24.008093+0000","last_became_peered":"2024-04-25T10:13:24.008093+0000","last_unstale":"2024-04-25T10:13:24.015562+0000","last_undegraded":"2024-04-25T10:13:24.015562+0000","last_fullsized":"2024-04-25T10:13:24.015562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:58.978817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.990222+0000","last_change":"2024-04-25T10:13:23.004889+0000","last_active":"2024-04-25T10:13:24.990222+0000","last_peered":"2024-04-25T10:13:24.990222+0000","last_clean":"2024-04-25T10:13:24.990222+0000","last_became_active":"2024-04-25T10:13:23.004533+0000","last_became_peered":"2024-04-25T10:13:23.004533+0000","last_unstale":"2024-04-25T10:13:24.990222+0000","last_undegraded":"2024-04-25T10:13:24.990222+0000","last_fullsized":"2024-04-25T10:13:24.990222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:48.041883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.019647+0000","last_change":"2024-04-25T10:13:24.013144+0000","last_active":"2024-04-25T10:13:24.019647+0000","last_peered":"2024-04-25T10:13:24.019647+0000","last_clean":"2024-04-25T10:13:24.019647+0000","last_became_active":"2024-04-25T10:13:24.012739+0000","last_became_peered":"2024-04-25T10:13:24.012739+0000","last_unstale":"2024-04-25T10:13:24.019647+0000","last_undegraded":"2024-04-25T10:13:24.019647+0000","last_fullsized":"2024-04-25T10:13:24.019647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:57.015200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:25.014420+0000","last_change":"2024-04-25T10:13:22.995435+0000","last_active":"2024-04-25T10:13:25.014420+0000","last_peered":"2024-04-25T10:13:25.014420+0000","last_clean":"2024-04-25T10:13:25.014420+0000","last_became_active":"2024-04-25T10:13:22.995080+0000","last_became_peered":"2024-04-25T10:13:22.995080+0000","last_unstale":"2024-04-25T10:13:25.014420+0000","last_undegraded":"2024-04-25T10:13:25.014420+0000","last_fullsized":"2024-04-25T10:13:25.014420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:28.464504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.017422+0000","last_change":"2024-04-25T10:13:24.010507+0000","last_active":"2024-04-25T10:13:24.017422+0000","last_peered":"2024-04-25T10:13:24.017422+0000","last_clean":"2024-04-25T10:13:24.017422+0000","last_became_active":"2024-04-25T10:13:24.009733+0000","last_became_peered":"2024-04-25T10:13:24.009733+0000","last_unstale":"2024-04-25T10:13:24.017422+0000","last_undegraded":"2024-04-25T10:13:24.017422+0000","last_fullsized":"2024-04-25T10:13:24.017422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:40.842115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.992254+0000","last_change":"2024-04-25T10:13:23.003882+0000","last_active":"2024-04-25T10:13:23.992254+0000","last_peered":"2024-04-25T10:13:23.992254+0000","last_clean":"2024-04-25T10:13:23.992254+0000","last_became_active":"2024-04-25T10:13:23.002275+0000","last_became_peered":"2024-04-25T10:13:23.002275+0000","last_unstale":"2024-04-25T10:13:23.992254+0000","last_undegraded":"2024-04-25T10:13:23.992254+0000","last_fullsized":"2024-04-25T10:13:23.992254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:45.864750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.025033+0000","last_change":"2024-04-25T10:13:24.018534+0000","last_active":"2024-04-25T10:13:24.025033+0000","last_peered":"2024-04-25T10:13:24.025033+0000","last_clean":"2024-04-25T10:13:24.025033+0000","last_became_active":"2024-04-25T10:13:24.017879+0000","last_became_peered":"2024-04-25T10:13:24.017879+0000","last_unstale":"2024-04-25T10:13:24.025033+0000","last_undegraded":"2024-04-25T10:13:24.025033+0000","last_fullsized":"2024-04-25T10:13:24.025033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:47.884140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.002188+0000","last_change":"2024-04-25T10:13:23.003835+0000","last_active":"2024-04-25T10:13:24.002188+0000","last_peered":"2024-04-25T10:13:24.002188+0000","last_clean":"2024-04-25T10:13:24.002188+0000","last_became_active":"2024-04-25T10:13:23.003533+0000","last_became_peered":"2024-04-25T10:13:23.003533+0000","last_unstale":"2024-04-25T10:13:24.002188+0000","last_undegraded":"2024-04-25T10:13:24.002188+0000","last_fullsized":"2024-04-25T10:13:24.002188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:46.510168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.989413+0000","last_change":"2024-04-25T10:13:24.003742+0000","last_active":"2024-04-25T10:13:24.989413+0000","last_peered":"2024-04-25T10:13:24.989413+0000","last_clean":"2024-04-25T10:13:24.989413+0000","last_became_active":"2024-04-25T10:13:24.003410+0000","last_became_peered":"2024-04-25T10:13:24.003410+0000","last_unstale":"2024-04-25T10:13:24.989413+0000","last_undegraded":"2024-04-25T10:13:24.989413+0000","last_fullsized":"2024-04-25T10:13:24.989413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:48.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.998096+0000","last_change":"2024-04-25T10:13:23.001639+0000","last_active":"2024-04-25T10:13:24.998096+0000","last_peered":"2024-04-25T10:13:24.998096+0000","last_clean":"2024-04-25T10:13:24.998096+0000","last_became_active":"2024-04-25T10:13:23.001368+0000","last_became_peered":"2024-04-25T10:13:23.001368+0000","last_unstale":"2024-04-25T10:13:24.998096+0000","last_undegraded":"2024-04-25T10:13:24.998096+0000","last_fullsized":"2024-04-25T10:13:24.998096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:55.998787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.997838+0000","last_change":"2024-04-25T10:13:24.015722+0000","last_active":"2024-04-25T10:13:24.997838+0000","last_peered":"2024-04-25T10:13:24.997838+0000","last_clean":"2024-04-25T10:13:24.997838+0000","last_became_active":"2024-04-25T10:13:24.013495+0000","last_became_peered":"2024-04-25T10:13:24.013495+0000","last_unstale":"2024-04-25T10:13:24.997838+0000","last_undegraded":"2024-04-25T10:13:24.997838+0000","last_fullsized":"2024-04-25T10:13:24.997838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:16.985205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.993311+0000","last_change":"2024-04-25T10:13:23.008601+0000","last_active":"2024-04-25T10:13:23.993311+0000","last_peered":"2024-04-25T10:13:23.993311+0000","last_clean":"2024-04-25T10:13:23.993311+0000","last_became_active":"2024-04-25T10:13:23.008298+0000","last_became_peered":"2024-04-25T10:13:23.008298+0000","last_unstale":"2024-04-25T10:13:23.993311+0000","last_undegraded":"2024-04-25T10:13:23.993311+0000","last_fullsized":"2024-04-25T10:13:23.993311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.644390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.014602+0000","last_change":"2024-04-25T10:13:24.010223+0000","last_active":"2024-04-25T10:13:24.014602+0000","last_peered":"2024-04-25T10:13:24.014602+0000","last_clean":"2024-04-25T10:13:24.014602+0000","last_became_active":"2024-04-25T10:13:24.009951+0000","last_became_peered":"2024-04-25T10:13:24.009951+0000","last_unstale":"2024-04-25T10:13:24.014602+0000","last_undegraded":"2024-04-25T10:13:24.014602+0000","last_fullsized":"2024-04-25T10:13:24.014602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:41.926767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.993626+0000","last_change":"2024-04-25T10:13:22.995695+0000","last_active":"2024-04-25T10:13:23.993626+0000","last_peered":"2024-04-25T10:13:23.993626+0000","last_clean":"2024-04-25T10:13:23.993626+0000","last_became_active":"2024-04-25T10:13:22.994550+0000","last_became_peered":"2024-04-25T10:13:22.994550+0000","last_unstale":"2024-04-25T10:13:23.993626+0000","last_undegraded":"2024-04-25T10:13:23.993626+0000","last_fullsized":"2024-04-25T10:13:23.993626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:40.378036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.014490+0000","last_change":"2024-04-25T10:13:24.006698+0000","last_active":"2024-04-25T10:13:24.014490+0000","last_peered":"2024-04-25T10:13:24.014490+0000","last_clean":"2024-04-25T10:13:24.014490+0000","last_became_active":"2024-04-25T10:13:24.006147+0000","last_became_peered":"2024-04-25T10:13:24.006147+0000","last_unstale":"2024-04-25T10:13:24.014490+0000","last_undegraded":"2024-04-25T10:13:24.014490+0000","last_fullsized":"2024-04-25T10:13:24.014490+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:21.660803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.994407+0000","last_change":"2024-04-25T10:13:23.008216+0000","last_active":"2024-04-25T10:13:23.994407+0000","last_peered":"2024-04-25T10:13:23.994407+0000","last_clean":"2024-04-25T10:13:23.994407+0000","last_became_active":"2024-04-25T10:13:23.007330+0000","last_became_peered":"2024-04-25T10:13:23.007330+0000","last_unstale":"2024-04-25T10:13:23.994407+0000","last_undegraded":"2024-04-25T10:13:23.994407+0000","last_fullsized":"2024-04-25T10:13:23.994407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.990362+0000","last_change":"2024-04-25T10:13:24.001340+0000","last_active":"2024-04-25T10:13:24.990362+0000","last_peered":"2024-04-25T10:13:24.990362+0000","last_clean":"2024-04-25T10:13:24.990362+0000","last_became_active":"2024-04-25T10:13:24.001067+0000","last_became_peered":"2024-04-25T10:13:24.001067+0000","last_unstale":"2024-04-25T10:13:24.990362+0000","last_undegraded":"2024-04-25T10:13:24.990362+0000","last_fullsized":"2024-04-25T10:13:24.990362+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:22.953801+0000","last_clean_scrub_stamp":"2024-04-25T10:13:22.953801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:47.526581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.025989+0000","last_change":"2024-04-25T10:13:23.007164+0000","last_active":"2024-04-25T10:13:24.025989+0000","last_peered":"2024-04-25T10:13:24.025989+0000","last_clean":"2024-04-25T10:13:24.025989+0000","last_became_active":"2024-04-25T10:13:23.006828+0000","last_became_peered":"2024-04-25T10:13:23.006828+0000","last_unstale":"2024-04-25T10:13:24.025989+0000","last_undegraded":"2024-04-25T10:13:24.025989+0000","last_fullsized":"2024-04-25T10:13:24.025989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.937389+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.937389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:57.962405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":283,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.995589+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:13:23.995589+0000","last_peered":"2024-04-25T10:13:23.995589+0000","last_clean":"2024-04-25T10:13:23.995589+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:13:23.995589+0000","last_undegraded":"2024-04-25T10:13:23.995589+0000","last_fullsized":"2024-04-25T10:13:23.995589+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1581056,"data_stored":1007540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":417796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1011712,"data_stored":425036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":983040,"data_stored":417102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":418714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":423984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":423596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1593344,"data_stored":1013804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:28.880 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:13:28.880 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:13:29.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.131+0000 7fced7200640 1 Processor -- start 2024-04-25T10:13:29.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.131+0000 7fced7200640 1 -- start start 2024-04-25T10:13:29.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.133+0000 7fced7200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fced80b2c90 0x7fced80ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.133+0000 7fced7200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced815f4f0 con 0x7fced8144fc0 2024-04-25T10:13:29.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.133+0000 7fced7200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced815f7d0 con 0x7fced806b2b0 2024-04-25T10:13:29.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.133+0000 7fced7200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced8160100 con 0x7fced80b2c90 2024-04-25T10:13:29.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.133+0000 7fced5e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fced8144fc0 0x7fced806aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48516/0 (socket says 172.21.15.119:48516) 2024-04-25T10:13:29.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.133+0000 7fced5e00640 1 -- 172.21.15.119:0/2993108832 learned_addr learned my addr 172.21.15.119:0/2993108832 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:29.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.133+0000 7fced4a00640 1 -- 172.21.15.119:0/2993108832 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556746002 0 0) 0x7fced815f4f0 con 0x7fced8144fc0 2024-04-25T10:13:29.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.133+0000 7fced6800640 1 --2- 172.21.15.119:0/2993108832 >> v2:172.21.15.132:3300/0 conn(0x7fced80b2c90 0x7fced80ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.133+0000 7fced4a00640 1 -- 172.21.15.119:0/2993108832 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcebc003520 con 0x7fced8144fc0 2024-04-25T10:13:29.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.134+0000 7fced4a00640 1 -- 172.21.15.119:0/2993108832 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2063194495 0 0) 0x7fcebc003520 con 0x7fced8144fc0 2024-04-25T10:13:29.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.134+0000 7fced4a00640 1 -- 172.21.15.119:0/2993108832 >> v2:172.21.15.132:3300/0 conn(0x7fced80b2c90 msgr2=0x7fced80ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:29.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.134+0000 7fced4a00640 1 --2- 172.21.15.119:0/2993108832 >> v2:172.21.15.132:3300/0 conn(0x7fced80b2c90 0x7fced80ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.134+0000 7fced4a00640 1 -- 172.21.15.119:0/2993108832 >> v1:172.21.15.132:6789/0 conn(0x7fced806b2b0 legacy=0x7fced80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:29.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.134+0000 7fced4a00640 1 -- 172.21.15.119:0/2993108832 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced815fab0 con 0x7fced8144fc0 2024-04-25T10:13:29.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.134+0000 7fced4a00640 1 -- 172.21.15.119:0/2993108832 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fcec8001670 con 0x7fced8144fc0 2024-04-25T10:13:29.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.134+0000 7fced4a00640 1 -- 172.21.15.119:0/2993108832 >> v1:172.21.15.119:6789/0 conn(0x7fced8144fc0 legacy=0x7fced806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.134+0000 7fced4a00640 1 --2- 172.21.15.119:0/2993108832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcebc0039b0 0x7fcebc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.134+0000 7fced4a00640 1 --2- 172.21.15.119:0/2993108832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcebc004800 0x7fcebc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.134+0000 7fced4a00640 1 --2- 172.21.15.119:0/2993108832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc0052e0 0x7fcebc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.134+0000 7fced4a00640 1 -- 172.21.15.119:0/2993108832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced8160100 con 0x7fcebc0052e0 2024-04-25T10:13:29.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.134+0000 7fced4a00640 1 -- 172.21.15.119:0/2993108832 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc00c1b0 con 0x7fcebc0039b0 2024-04-25T10:13:29.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.134+0000 7fced4a00640 1 -- 172.21.15.119:0/2993108832 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc00c3b0 con 0x7fcebc004800 2024-04-25T10:13:29.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.134+0000 7fced6800640 1 --2- 172.21.15.119:0/2993108832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc0052e0 0x7fcebc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.135+0000 7fced5e00640 1 --2- 172.21.15.119:0/2993108832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcebc0039b0 0x7fcebc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.135+0000 7fced6800640 1 -- 172.21.15.119:0/2993108832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcebc004800 msgr2=0x7fcebc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:29.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.135+0000 7fced5400640 1 --2- 172.21.15.119:0/2993108832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcebc004800 0x7fcebc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.135+0000 7fced6800640 1 --2- 172.21.15.119:0/2993108832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcebc004800 0x7fcebc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.135+0000 7fced6800640 1 -- 172.21.15.119:0/2993108832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcebc0039b0 msgr2=0x7fcebc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:29.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.135+0000 7fced6800640 1 --2- 172.21.15.119:0/2993108832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcebc0039b0 0x7fcebc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.135+0000 7fced6800640 1 -- 172.21.15.119:0/2993108832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc003520 con 0x7fcebc0052e0 2024-04-25T10:13:29.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.135+0000 7fced5400640 1 --2- 172.21.15.119:0/2993108832 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcebc004800 0x7fcebc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:29.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.136+0000 7fced6800640 1 -- 172.21.15.119:0/2993108832 reap_dead start 2024-04-25T10:13:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.136+0000 7fced6800640 1 --2- 172.21.15.119:0/2993108832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc0052e0 0x7fcebc00b9c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fced0007330 tx=0x7fcebc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae4e88fa92da956 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.136+0000 7fced4a00640 1 -- 172.21.15.119:0/2993108832 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0009be0 con 0x7fcebc0052e0 2024-04-25T10:13:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.136+0000 7fced4a00640 1 -- 172.21.15.119:0/2993108832 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced0003b90 con 0x7fcebc0052e0 2024-04-25T10:13:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.136+0000 7fced4a00640 1 -- 172.21.15.119:0/2993108832 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0004020 con 0x7fcebc0052e0 2024-04-25T10:13:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.137+0000 7fced7200640 1 -- 172.21.15.119:0/2993108832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc0052e0 msgr2=0x7fcebc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.137+0000 7fced7200640 1 --2- 172.21.15.119:0/2993108832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc0052e0 0x7fcebc00b9c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fced0007330 tx=0x7fcebc005830 comp rx=0 tx=0).stop 2024-04-25T10:13:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.137+0000 7fced7200640 1 -- 172.21.15.119:0/2993108832 shutdown_connections 2024-04-25T10:13:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.137+0000 7fced7200640 1 --2- 172.21.15.119:0/2993108832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc0052e0 0x7fcebc00b9c0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.137+0000 7fced7200640 1 -- 172.21.15.119:0/2993108832 >> 172.21.15.119:0/2993108832 conn(0x7fced8067040 msgr2=0x7fced80b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.137+0000 7fced7200640 1 -- 172.21.15.119:0/2993108832 shutdown_connections 2024-04-25T10:13:29.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.137+0000 7fced7200640 1 -- 172.21.15.119:0/2993108832 wait complete. 2024-04-25T10:13:29.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.139+0000 7fced7200640 1 Processor -- start 2024-04-25T10:13:29.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.139+0000 7fced7200640 1 -- start start 2024-04-25T10:13:29.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced7200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc0052e0 0x7fced81431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced7200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fced815d1e0 0x7fced815d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced7200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fced80c79e0 0x7fced80c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced7200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced806aba0 con 0x7fcebc0052e0 2024-04-25T10:13:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced7200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced81453a0 con 0x7fced815d1e0 2024-04-25T10:13:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced7200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced806ad20 con 0x7fced80c79e0 2024-04-25T10:13:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc0052e0 0x7fced81431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc0052e0 0x7fced81431c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52744/0 (socket says 172.21.15.119:52744) 2024-04-25T10:13:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced5e00640 1 -- 172.21.15.119:0/2316968052 learned_addr learned my addr 172.21.15.119:0/2316968052 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced5400640 1 --2- 172.21.15.119:0/2316968052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fced815d1e0 0x7fced815d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced5e00640 1 -- 172.21.15.119:0/2316968052 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fced80c79e0 msgr2=0x7fced80c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced5e00640 1 --2- 172.21.15.119:0/2316968052 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fced80c79e0 0x7fced80c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced5e00640 1 -- 172.21.15.119:0/2316968052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fced815d1e0 msgr2=0x7fced815d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced5e00640 1 --2- 172.21.15.119:0/2316968052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fced815d1e0 0x7fced815d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.140+0000 7fced5e00640 1 -- 172.21.15.119:0/2316968052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced815dca0 con 0x7fcebc0052e0 2024-04-25T10:13:29.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.141+0000 7fced5e00640 1 --2- 172.21.15.119:0/2316968052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc0052e0 0x7fced81431c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fcec800e730 tx=0x7fcec800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:29.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.141+0000 7fcece000640 1 -- 172.21.15.119:0/2316968052 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec800c6c0 con 0x7fcebc0052e0 2024-04-25T10:13:29.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.141+0000 7fcece000640 1 -- 172.21.15.119:0/2316968052 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec8018070 con 0x7fcebc0052e0 2024-04-25T10:13:29.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.141+0000 7fcece000640 1 -- 172.21.15.119:0/2316968052 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec800caa0 con 0x7fcebc0052e0 2024-04-25T10:13:29.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.141+0000 7fced7200640 1 -- 172.21.15.119:0/2316968052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced815df80 con 0x7fcebc0052e0 2024-04-25T10:13:29.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.141+0000 7fced7200640 1 -- 172.21.15.119:0/2316968052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced815e540 con 0x7fcebc0052e0 2024-04-25T10:13:29.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.142+0000 7fced7200640 1 -- 172.21.15.119:0/2316968052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced80b4480 con 0x7fcebc0052e0 2024-04-25T10:13:29.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.143+0000 7fcece000640 1 -- 172.21.15.119:0/2316968052 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcec800cc40 con 0x7fcebc0052e0 2024-04-25T10:13:29.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.144+0000 7fcece000640 1 --2- 172.21.15.119:0/2316968052 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fceb806b520 0x7fceb806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.144+0000 7fcece000640 1 -- 172.21.15.119:0/2316968052 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcec8089ee0 con 0x7fcebc0052e0 2024-04-25T10:13:29.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.145+0000 7fced5400640 1 --2- 172.21.15.119:0/2316968052 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fceb806b520 0x7fceb806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.145+0000 7fced5400640 1 --2- 172.21.15.119:0/2316968052 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fceb806b520 0x7fceb806d9e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fced8144030 tx=0x7fcec4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:29.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.149+0000 7fcece000640 1 -- 172.21.15.119:0/2316968052 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fced80b4480 con 0x7fcebc0052e0 2024-04-25T10:13:29.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.415+0000 7fced7200640 1 -- 172.21.15.119:0/2316968052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fced80749b0 con 0x7fcebc0052e0 2024-04-25T10:13:29.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.421+0000 7fcece000640 1 -- 172.21.15.119:0/2316968052 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7fcec8058dd0 con 0x7fcebc0052e0 2024-04-25T10:13:29.426 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-04-25T10:13:29.426 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA== 2024-04-25T10:13:29.426 INFO:teuthology.orchestra.run.smithi119.stdout: caps mds = "allow" 2024-04-25T10:13:29.426 INFO:teuthology.orchestra.run.smithi119.stdout: caps mon = "allow r" 2024-04-25T10:13:29.426 INFO:teuthology.orchestra.run.smithi119.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:13:29.426 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2024-04-25T10:13:29.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.423+0000 7fced7200640 1 -- 172.21.15.119:0/2316968052 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fceb806b520 msgr2=0x7fceb806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.423+0000 7fced7200640 1 --2- 172.21.15.119:0/2316968052 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fceb806b520 0x7fceb806d9e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fced8144030 tx=0x7fcec4009040 comp rx=0 tx=0).stop 2024-04-25T10:13:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.423+0000 7fced7200640 1 -- 172.21.15.119:0/2316968052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc0052e0 msgr2=0x7fced81431c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.423+0000 7fced7200640 1 --2- 172.21.15.119:0/2316968052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc0052e0 0x7fced81431c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fcec800e730 tx=0x7fcec800ebf0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.424+0000 7fced7200640 1 -- 172.21.15.119:0/2316968052 shutdown_connections 2024-04-25T10:13:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.424+0000 7fced7200640 1 --2- 172.21.15.119:0/2316968052 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fceb806b520 0x7fceb806d9e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.424+0000 7fced7200640 1 --2- 172.21.15.119:0/2316968052 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fced80c79e0 0x7fced80c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.424+0000 7fced7200640 1 --2- 172.21.15.119:0/2316968052 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fced815d1e0 0x7fced815d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.424+0000 7fced7200640 1 --2- 172.21.15.119:0/2316968052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcebc0052e0 0x7fced81431c0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.424+0000 7fced7200640 1 -- 172.21.15.119:0/2316968052 >> 172.21.15.119:0/2316968052 conn(0x7fced8067040 msgr2=0x7fced80b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.424+0000 7fced7200640 1 -- 172.21.15.119:0/2316968052 shutdown_connections 2024-04-25T10:13:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.424+0000 7fced7200640 1 -- 172.21.15.119:0/2316968052 wait complete. 2024-04-25T10:13:29.447 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:29.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.640+0000 7f075f200640 1 Processor -- start 2024-04-25T10:13:29.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.640+0000 7f075f200640 1 -- start start 2024-04-25T10:13:29.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.642+0000 7f075f200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0760144470 0x7f076015fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.642+0000 7f075f200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f076006e430 0x7f07600b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.642+0000 7f075f200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07600cd440 con 0x7f07600b4e80 2024-04-25T10:13:29.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.642+0000 7f075f200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0760145340 con 0x7f076006e430 2024-04-25T10:13:29.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.642+0000 7f075f200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07600cd720 con 0x7f0760144470 2024-04-25T10:13:29.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.642+0000 7f0757e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f076006e430 0x7f07600b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.642+0000 7f0757e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f076006e430 0x7f07600b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52748/0 (socket says 172.21.15.119:52748) 2024-04-25T10:13:29.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.642+0000 7f0757e00640 1 -- 172.21.15.119:0/2074257004 learned_addr learned my addr 172.21.15.119:0/2074257004 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:29.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.643+0000 7f075de00640 1 --2- 172.21.15.119:0/2074257004 >> v2:172.21.15.132:3300/0 conn(0x7f0760144470 0x7f076015fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.643+0000 7f0757e00640 1 -- 172.21.15.119:0/2074257004 >> v2:172.21.15.132:3300/0 conn(0x7f0760144470 msgr2=0x7f076015fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:29.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.643+0000 7f0757e00640 1 --2- 172.21.15.119:0/2074257004 >> v2:172.21.15.132:3300/0 conn(0x7f0760144470 0x7f076015fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.643+0000 7f0757e00640 1 -- 172.21.15.119:0/2074257004 >> v1:172.21.15.132:6789/0 conn(0x7f07600b4e80 legacy=0x7f076006d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:29.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.643+0000 7f0757e00640 1 -- 172.21.15.119:0/2074257004 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07600cda00 con 0x7f076006e430 2024-04-25T10:13:29.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.644+0000 7f0757e00640 1 --2- 172.21.15.119:0/2074257004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f076006e430 0x7f07600b4940 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f07600b5490 tx=0x7f074802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72852de8c0ff3e11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:29.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.644+0000 7f075d400640 1 -- 172.21.15.119:0/2074257004 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748037070 con 0x7f076006e430 2024-04-25T10:13:29.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.644+0000 7f075d400640 1 -- 172.21.15.119:0/2074257004 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0748032440 con 0x7f076006e430 2024-04-25T10:13:29.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.644+0000 7f075d400640 1 -- 172.21.15.119:0/2074257004 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748032810 con 0x7f076006e430 2024-04-25T10:13:29.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.645+0000 7f075f200640 1 -- 172.21.15.119:0/2074257004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f076006e430 msgr2=0x7f07600b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.645+0000 7f075f200640 1 --2- 172.21.15.119:0/2074257004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f076006e430 0x7f07600b4940 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f07600b5490 tx=0x7f074802a230 comp rx=0 tx=0).stop 2024-04-25T10:13:29.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.645+0000 7f075f200640 1 -- 172.21.15.119:0/2074257004 shutdown_connections 2024-04-25T10:13:29.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.645+0000 7f075f200640 1 --2- 172.21.15.119:0/2074257004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f076006e430 0x7f07600b4940 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.645+0000 7f075f200640 1 --2- 172.21.15.119:0/2074257004 >> v2:172.21.15.132:3300/0 conn(0x7f0760144470 0x7f076015fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.645+0000 7f075f200640 1 -- 172.21.15.119:0/2074257004 >> 172.21.15.119:0/2074257004 conn(0x7f076006b1d0 msgr2=0x7f07600b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:29.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.645+0000 7f075f200640 1 -- 172.21.15.119:0/2074257004 shutdown_connections 2024-04-25T10:13:29.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.645+0000 7f075f200640 1 -- 172.21.15.119:0/2074257004 wait complete. 2024-04-25T10:13:29.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.647+0000 7f075f200640 1 Processor -- start 2024-04-25T10:13:29.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.647+0000 7f075f200640 1 -- start start 2024-04-25T10:13:29.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.648+0000 7f075f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f076006e430 0x7f07601f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.648+0000 7f075f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07600b4e80 0x7f07600adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.648+0000 7f075f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0760144470 0x7f07601f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.648+0000 7f075f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07600ce200 con 0x7f0760144470 2024-04-25T10:13:29.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.648+0000 7f075f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0760144850 con 0x7f07600b4e80 2024-04-25T10:13:29.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.648+0000 7f075f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076015cd00 con 0x7f076006e430 2024-04-25T10:13:29.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.648+0000 7f075e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0760144470 0x7f07601f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.648+0000 7f075e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0760144470 0x7f07601f4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52754/0 (socket says 172.21.15.119:52754) 2024-04-25T10:13:29.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.648+0000 7f075e800640 1 -- 172.21.15.119:0/2816454195 learned_addr learned my addr 172.21.15.119:0/2816454195 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:29.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.649+0000 7f0757e00640 1 --2- 172.21.15.119:0/2816454195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07600b4e80 0x7f07600adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.649+0000 7f075de00640 1 --2- 172.21.15.119:0/2816454195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f076006e430 0x7f07601f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.649+0000 7f075e800640 1 -- 172.21.15.119:0/2816454195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f076006e430 msgr2=0x7f07601f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.649+0000 7f075e800640 1 --2- 172.21.15.119:0/2816454195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f076006e430 0x7f07601f4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.649+0000 7f075e800640 1 -- 172.21.15.119:0/2816454195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07600b4e80 msgr2=0x7f07600adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.649+0000 7f075e800640 1 --2- 172.21.15.119:0/2816454195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07600b4e80 0x7f07600adff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.649+0000 7f075e800640 1 -- 172.21.15.119:0/2816454195 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07601f48c0 con 0x7f0760144470 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.649+0000 7f075e800640 1 --2- 172.21.15.119:0/2816454195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0760144470 0x7f07601f4b10 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f075800c780 tx=0x7f075800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.649+0000 7f0756000640 1 -- 172.21.15.119:0/2816454195 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758017070 con 0x7f0760144470 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.649+0000 7f0756000640 1 -- 172.21.15.119:0/2816454195 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0758003da0 con 0x7f0760144470 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.649+0000 7f0756000640 1 -- 172.21.15.119:0/2816454195 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07580052b0 con 0x7f0760144470 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.650+0000 7f075f200640 1 -- 172.21.15.119:0/2816454195 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07600acf50 con 0x7f0760144470 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.650+0000 7f075f200640 1 -- 172.21.15.119:0/2816454195 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07600ad510 con 0x7f0760144470 2024-04-25T10:13:29.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.651+0000 7f075f200640 1 -- 172.21.15.119:0/2816454195 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07600749b0 con 0x7f0760144470 2024-04-25T10:13:29.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.652+0000 7f0756000640 1 -- 172.21.15.119:0/2816454195 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0758005450 con 0x7f0760144470 2024-04-25T10:13:29.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.653+0000 7f0756000640 1 --2- 172.21.15.119:0/2816454195 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f073c06b4d0 0x7f073c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.653+0000 7f0756000640 1 -- 172.21.15.119:0/2816454195 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0758089860 con 0x7f0760144470 2024-04-25T10:13:29.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.653+0000 7f075de00640 1 --2- 172.21.15.119:0/2816454195 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f073c06b4d0 0x7f073c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.653+0000 7f075de00640 1 --2- 172.21.15.119:0/2816454195 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f073c06b4d0 0x7f073c06d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0750006fd0 tx=0x7f0750008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:29.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.657+0000 7f0756000640 1 -- 172.21.15.119:0/2816454195 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0758058640 con 0x7f0760144470 2024-04-25T10:13:29.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.870+0000 7f075f200640 1 -- 172.21.15.119:0/2816454195 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07600b2090 con 0x7f0760144470 2024-04-25T10:13:29.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.871+0000 7f0756000640 1 -- 172.21.15.119:0/2816454195 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f0758057f40 con 0x7f0760144470 2024-04-25T10:13:29.876 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:29.876 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:13:26.498824+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7158},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7158":{"gid":7158,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/3997964942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3997964942},{"type":"v1","addr":"172.21.15.119:6837","nonce":3997964942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:13:29.876 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 61 2024-04-25T10:13:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.873+0000 7f075f200640 1 -- 172.21.15.119:0/2816454195 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f073c06b4d0 msgr2=0x7f073c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.873+0000 7f075f200640 1 --2- 172.21.15.119:0/2816454195 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f073c06b4d0 0x7f073c06d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0750006fd0 tx=0x7f0750008040 comp rx=0 tx=0).stop 2024-04-25T10:13:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.873+0000 7f075f200640 1 -- 172.21.15.119:0/2816454195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0760144470 msgr2=0x7f07601f4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.873+0000 7f075f200640 1 --2- 172.21.15.119:0/2816454195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0760144470 0x7f07601f4b10 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f075800c780 tx=0x7f075800cc40 comp rx=0 tx=0).stop 2024-04-25T10:13:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.873+0000 7f075f200640 1 -- 172.21.15.119:0/2816454195 shutdown_connections 2024-04-25T10:13:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.873+0000 7f075f200640 1 --2- 172.21.15.119:0/2816454195 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f073c06b4d0 0x7f073c06d990 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.873+0000 7f075f200640 1 --2- 172.21.15.119:0/2816454195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0760144470 0x7f07601f4b10 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.874+0000 7f075f200640 1 --2- 172.21.15.119:0/2816454195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07600b4e80 0x7f07600adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.874+0000 7f075f200640 1 --2- 172.21.15.119:0/2816454195 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f076006e430 0x7f07601f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.874+0000 7f075f200640 1 -- 172.21.15.119:0/2816454195 >> 172.21.15.119:0/2816454195 conn(0x7f076006b1d0 msgr2=0x7f07600afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.874+0000 7f075f200640 1 -- 172.21.15.119:0/2816454195 shutdown_connections 2024-04-25T10:13:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:29.874+0000 7f075f200640 1 -- 172.21.15.119:0/2816454195 wait complete. 2024-04-25T10:13:29.938 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:13:29.933+0000 7f8284c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:30.300 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:23.427654+0000', 'modified': '2024-04-25T10:13:26.498824+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7158}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7158': {'gid': 7158, 'name': 'a', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6837/3997964942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 3997964942}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 3997964942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:30.300 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:13:30.300 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:13:30.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.505+0000 7f1aba600640 1 Processor -- start 2024-04-25T10:13:30.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.505+0000 7f1aba600640 1 -- start start 2024-04-25T10:13:30.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.507+0000 7f1aba600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4145260 0x7f1ab40cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.507+0000 7f1aba600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1ab4071910 0x7f1ab40cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.507+0000 7f1aba600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab40cc9a0 con 0x7f1ab40cc490 2024-04-25T10:13:30.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.507+0000 7f1aba600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab40cea00 con 0x7f1ab4071910 2024-04-25T10:13:30.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.507+0000 7f1aba600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab40ccc80 con 0x7f1ab4145260 2024-04-25T10:13:30.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.508+0000 7f1ab3e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1ab4071910 0x7f1ab40cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.508+0000 7f1ab3e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1ab4071910 0x7f1ab40cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52766/0 (socket says 172.21.15.119:52766) 2024-04-25T10:13:30.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.508+0000 7f1ab3e00640 1 -- 172.21.15.119:0/808166159 learned_addr learned my addr 172.21.15.119:0/808166159 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:30.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.508+0000 7f1ab2a00640 1 --1- 172.21.15.119:0/808166159 >> v1:172.21.15.132:6789/0 conn(0x7f1ab40cc490 0x7f1ab4071200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:36760/0 (socket says 172.21.15.119:36760) 2024-04-25T10:13:30.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.508+0000 7f1ab3400640 1 --2- 172.21.15.119:0/808166159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4145260 0x7f1ab40cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.508+0000 7f1ab3e00640 1 -- 172.21.15.119:0/808166159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4145260 msgr2=0x7f1ab40cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:30.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.508+0000 7f1ab3e00640 1 --2- 172.21.15.119:0/808166159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4145260 0x7f1ab40cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.508+0000 7f1ab3e00640 1 -- 172.21.15.119:0/808166159 >> v1:172.21.15.132:6789/0 conn(0x7f1ab40cc490 legacy=0x7f1ab4071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.508+0000 7f1ab3e00640 1 -- 172.21.15.119:0/808166159 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab40ccf60 con 0x7f1ab4071910 2024-04-25T10:13:30.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.509+0000 7f1ab3e00640 1 --2- 172.21.15.119:0/808166159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab4071910 0x7f1ab40cee90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009e50 tx=0x7f1aac02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4183a35d4cdf011 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:30.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.509+0000 7f1ab2000640 1 -- 172.21.15.119:0/808166159 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac036070 con 0x7f1ab4071910 2024-04-25T10:13:30.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.510+0000 7f1ab2000640 1 -- 172.21.15.119:0/808166159 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac031600 con 0x7f1ab4071910 2024-04-25T10:13:30.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.510+0000 7f1ab2000640 1 -- 172.21.15.119:0/808166159 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0319d0 con 0x7f1ab4071910 2024-04-25T10:13:30.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.510+0000 7f1aba600640 1 -- 172.21.15.119:0/808166159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab4071910 msgr2=0x7f1ab40cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.510+0000 7f1aba600640 1 --2- 172.21.15.119:0/808166159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab4071910 0x7f1ab40cee90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009e50 tx=0x7f1aac02a710 comp rx=0 tx=0).stop 2024-04-25T10:13:30.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.510+0000 7f1aba600640 1 -- 172.21.15.119:0/808166159 shutdown_connections 2024-04-25T10:13:30.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.510+0000 7f1aba600640 1 --2- 172.21.15.119:0/808166159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab4071910 0x7f1ab40cee90 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.510+0000 7f1aba600640 1 --2- 172.21.15.119:0/808166159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4145260 0x7f1ab40cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.510+0000 7f1aba600640 1 -- 172.21.15.119:0/808166159 >> 172.21.15.119:0/808166159 conn(0x7f1ab406aba0 msgr2=0x7f1ab406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:30.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.510+0000 7f1aba600640 1 -- 172.21.15.119:0/808166159 shutdown_connections 2024-04-25T10:13:30.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.510+0000 7f1aba600640 1 -- 172.21.15.119:0/808166159 wait complete. 2024-04-25T10:13:30.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.512+0000 7f1aba600640 1 Processor -- start 2024-04-25T10:13:30.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.512+0000 7f1aba600640 1 -- start start 2024-04-25T10:13:30.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.513+0000 7f1aba600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4071910 0x7f1ab40b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.513+0000 7f1aba600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ab40cc490 0x7f1ab415b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.513+0000 7f1aba600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab4145260 0x7f1ab41f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.513+0000 7f1aba600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab415c270 con 0x7f1ab4145260 2024-04-25T10:13:30.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.513+0000 7f1aba600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab4145640 con 0x7f1ab40cc490 2024-04-25T10:13:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.513+0000 7f1aba600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab415c3f0 con 0x7f1ab4071910 2024-04-25T10:13:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.513+0000 7f1ab3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab4145260 0x7f1ab41f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.514+0000 7f1ab3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab4145260 0x7f1ab41f3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52778/0 (socket says 172.21.15.119:52778) 2024-04-25T10:13:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.514+0000 7f1ab3e00640 1 -- 172.21.15.119:0/3011339510 learned_addr learned my addr 172.21.15.119:0/3011339510 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.514+0000 7f1ab3400640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4071910 0x7f1ab40b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.514+0000 7f1ab2a00640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ab40cc490 0x7f1ab415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.514+0000 7f1ab3e00640 1 -- 172.21.15.119:0/3011339510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4071910 msgr2=0x7f1ab40b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.514+0000 7f1ab3e00640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4071910 0x7f1ab40b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.514+0000 7f1ab3e00640 1 -- 172.21.15.119:0/3011339510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ab40cc490 msgr2=0x7f1ab415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.514+0000 7f1ab3e00640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ab40cc490 0x7f1ab415b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.514+0000 7f1ab3e00640 1 -- 172.21.15.119:0/3011339510 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab40ac950 con 0x7f1ab4145260 2024-04-25T10:13:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.514+0000 7f1ab2a00640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ab40cc490 0x7f1ab415b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.514+0000 7f1ab3e00640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab4145260 0x7f1ab41f3c40 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009f70 tx=0x7f1aac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:30.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.515+0000 7f1aab400640 1 -- 172.21.15.119:0/3011339510 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac03f020 con 0x7f1ab4145260 2024-04-25T10:13:30.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.515+0000 7f1aab400640 1 -- 172.21.15.119:0/3011339510 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac037040 con 0x7f1ab4145260 2024-04-25T10:13:30.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.515+0000 7f1aab400640 1 -- 172.21.15.119:0/3011339510 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac036070 con 0x7f1ab4145260 2024-04-25T10:13:30.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.515+0000 7f1aba600640 1 -- 172.21.15.119:0/3011339510 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab40ae7d0 con 0x7f1ab4145260 2024-04-25T10:13:30.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.515+0000 7f1aba600640 1 -- 172.21.15.119:0/3011339510 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab40aced0 con 0x7f1ab4145260 2024-04-25T10:13:30.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.517+0000 7f1aab400640 1 -- 172.21.15.119:0/3011339510 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1aac042020 con 0x7f1ab4145260 2024-04-25T10:13:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.518+0000 7f1aab400640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a9406b500 0x7f1a9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.518+0000 7f1aab400640 1 -- 172.21.15.119:0/3011339510 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1aac0aa700 con 0x7f1ab4145260 2024-04-25T10:13:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.519+0000 7f1abd413740 1 Processor -- start 2024-04-25T10:13:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.519+0000 7f1ab3400640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a9406b500 0x7f1a9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.519+0000 7f1abd413740 1 -- start start 2024-04-25T10:13:30.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.519+0000 7f1ab3400640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a9406b500 0x7f1a9406d9c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009a30 tx=0x7f1aa4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:30.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.520+0000 7f1abd413740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fd5b950 0x55a63fd83790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.520+0000 7f1abd413740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd83cd0 0x55a63fd5bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.520+0000 7f1abd413740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fda45c0 0x55a63fd5a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.520+0000 7f1abd413740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a63fcbf620 con 0x55a63fd83cd0 2024-04-25T10:13:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.520+0000 7f1abd413740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a63fc68ca0 con 0x55a63fda45c0 2024-04-25T10:13:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.520+0000 7f1abd413740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a63fcbb6c0 con 0x55a63fd5b950 2024-04-25T10:13:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.520+0000 7f1ab2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd83cd0 0x55a63fd5bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.520+0000 7f1ab2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd83cd0 0x55a63fd5bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52788/0 (socket says 172.21.15.119:52788) 2024-04-25T10:13:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.520+0000 7f1ab2a00640 1 -- 172.21.15.119:0/4063035218 learned_addr learned my addr 172.21.15.119:0/4063035218 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.520+0000 7f1ab2a00640 1 -- 172.21.15.119:0/4063035218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fd5b950 msgr2=0x55a63fd83790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.520+0000 7f1ab2a00640 1 --2- 172.21.15.119:0/4063035218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fd5b950 0x55a63fd83790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.520+0000 7f1ab2a00640 1 -- 172.21.15.119:0/4063035218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fda45c0 msgr2=0x55a63fd5a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.520+0000 7f1ab2a00640 1 --2- 172.21.15.119:0/4063035218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fda45c0 0x55a63fd5a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.520+0000 7f1ab2a00640 1 -- 172.21.15.119:0/4063035218 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a63fc6ff40 con 0x55a63fd83cd0 2024-04-25T10:13:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.521+0000 7f1ab2a00640 1 --2- 172.21.15.119:0/4063035218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd83cd0 0x55a63fd5bd30 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0000f60 tx=0x7f1aa0001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7635a801a07d166e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.521+0000 7f1a9fe00640 1 -- 172.21.15.119:0/4063035218 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa00074e0 con 0x55a63fd83cd0 2024-04-25T10:13:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.521+0000 7f1a9fe00640 1 -- 172.21.15.119:0/4063035218 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa0007680 con 0x55a63fd83cd0 2024-04-25T10:13:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.521+0000 7f1a9fe00640 1 -- 172.21.15.119:0/4063035218 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0004a00 con 0x55a63fd83cd0 2024-04-25T10:13:30.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.522+0000 7f1abd413740 1 -- 172.21.15.119:0/4063035218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd83cd0 msgr2=0x55a63fd5bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.522+0000 7f1abd413740 1 --2- 172.21.15.119:0/4063035218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd83cd0 0x55a63fd5bd30 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0000f60 tx=0x7f1aa0001b50 comp rx=0 tx=0).stop 2024-04-25T10:13:30.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.522+0000 7f1abd413740 1 -- 172.21.15.119:0/4063035218 shutdown_connections 2024-04-25T10:13:30.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.522+0000 7f1abd413740 1 --2- 172.21.15.119:0/4063035218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fda45c0 0x55a63fd5a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.522+0000 7f1abd413740 1 --2- 172.21.15.119:0/4063035218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd83cd0 0x55a63fd5bd30 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.522+0000 7f1abd413740 1 --2- 172.21.15.119:0/4063035218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fd5b950 0x55a63fd83790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.522+0000 7f1abd413740 1 -- 172.21.15.119:0/4063035218 >> 172.21.15.119:0/4063035218 conn(0x55a63fd58b70 msgr2=0x55a63fda1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:30.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.522+0000 7f1abd413740 1 -- 172.21.15.119:0/4063035218 shutdown_connections 2024-04-25T10:13:30.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.522+0000 7f1abd413740 1 -- 172.21.15.119:0/4063035218 wait complete. 2024-04-25T10:13:30.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.522+0000 7f1abd413740 1 -- start start 2024-04-25T10:13:30.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.523+0000 7f1abd413740 1 Processor -- start 2024-04-25T10:13:30.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1abd413740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd83cd0 0x55a63fd89d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1abd413740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fda45c0 0x55a63fda3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1abd413740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fd7d440 0x55a63fd7c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1abd413740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a63fd83300 con 0x55a63fd83cd0 2024-04-25T10:13:30.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1abd413740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a63fc5e1f0 con 0x55a63fd7d440 2024-04-25T10:13:30.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1abd413740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a63fd82cf0 con 0x55a63fda45c0 2024-04-25T10:13:30.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1abd413740 10 client.0 _ll_register_callbacks cb 0x55a63fc7bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:30.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1ab3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd83cd0 0x55a63fd89d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1ab3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd83cd0 0x55a63fd89d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52794/0 (socket says 172.21.15.119:52794) 2024-04-25T10:13:30.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1ab3400640 1 -- 172.21.15.119:0/4170014872 learned_addr learned my addr 172.21.15.119:0/4170014872 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:30.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1ab3400640 1 -- 172.21.15.119:0/4170014872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fda45c0 msgr2=0x55a63fda3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:30.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1ab3400640 1 --2- 172.21.15.119:0/4170014872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fda45c0 0x55a63fda3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1ab3400640 1 -- 172.21.15.119:0/4170014872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fd7d440 msgr2=0x55a63fd7c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:30.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1ab3400640 1 --2- 172.21.15.119:0/4170014872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fd7d440 0x55a63fd7c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.524+0000 7f1ab3400640 1 -- 172.21.15.119:0/4170014872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a63fc7dc50 con 0x55a63fd83cd0 2024-04-25T10:13:30.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.525+0000 7f1ab3400640 1 --2- 172.21.15.119:0/4170014872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd83cd0 0x55a63fd89d50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40156c0 tx=0x7f1aa4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70eee3e3e3ad47fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:30.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.525+0000 7f1a9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:13:30.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.525+0000 7f1a9cc00640 1 -- 172.21.15.119:0/4170014872 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa401c050 con 0x55a63fd83cd0 2024-04-25T10:13:30.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.525+0000 7f1a9cc00640 1 -- 172.21.15.119:0/4170014872 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1aa40145f0 con 0x55a63fd83cd0 2024-04-25T10:13:30.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.525+0000 7f1abd413740 1 -- 172.21.15.119:0/4170014872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1aa0007680 con 0x55a63fd83cd0 2024-04-25T10:13:30.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.525+0000 7f1a9cc00640 1 -- 172.21.15.119:0/4170014872 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aa402e050 con 0x55a63fd83cd0 2024-04-25T10:13:30.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.525+0000 7f1abd413740 10 client.7305 fetch_fsmap learned FSMap version 61 2024-04-25T10:13:30.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.525+0000 7f1abd413740 1 -- 172.21.15.119:0/4170014872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55a63f9d42d0 con 0x55a63fd83cd0 2024-04-25T10:13:30.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.525+0000 7f1a9cc00640 1 -- 172.21.15.119:0/4170014872 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa0007680 con 0x55a63fd83cd0 2024-04-25T10:13:30.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.525+0000 7f1a9cc00640 1 -- 172.21.15.119:0/4170014872 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1aa4031020 con 0x55a63fd83cd0 2024-04-25T10:13:30.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.525+0000 7f1abd413740 10 client.7305 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T10:13:30.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.526+0000 7f1abd413740 10 client.7305 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:13:30.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.526+0000 7f1abd413740 20 client.7305 populate_metadata read hostname 'smithi119' 2024-04-25T10:13:30.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.526+0000 7f1abd413740 1 --2- 172.21.15.119:0/4170014872 >> [v2:172.21.15.119:6835/3997964942,v1:172.21.15.119:6837/3997964942] conn(0x55a63fd88210 0x55a63fd7f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.526+0000 7f1abd413740 4 client.7305 mds_command: new command op to 7158 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:30.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.526+0000 7f1abd413740 1 -- 172.21.15.119:0/4170014872 --> [v2:172.21.15.119:6835/3997964942,v1:172.21.15.119:6837/3997964942] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a63fcccb60 con 0x55a63fd88210 2024-04-25T10:13:30.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.526+0000 7f1ab2a00640 1 --2- 172.21.15.119:0/4170014872 >> [v2:172.21.15.119:6835/3997964942,v1:172.21.15.119:6837/3997964942] conn(0x55a63fd88210 0x55a63fd7f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.526+0000 7f1ab2a00640 1 --2- 172.21.15.119:0/4170014872 >> [v2:172.21.15.119:6835/3997964942,v1:172.21.15.119:6837/3997964942] conn(0x55a63fd88210 0x55a63fd7f920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a18994aa18f13b2 server_cookie=ffef299b5d70721c in_seq=0 out_seq=0 2024-04-25T10:13:30.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.526+0000 7f1a9cc00640 10 client.7305 ms_handle_connect on v2:172.21.15.119:6835/3997964942 2024-04-25T10:13:30.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.527+0000 7f1a9cc00640 1 -- 172.21.15.119:0/4170014872 <== mds.0 v2:172.21.15.119:6835/3997964942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a63fcccb60 con 0x55a63fd88210 2024-04-25T10:13:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.527+0000 7f1a9cc00640 10 client.7305 handle_command_reply: tid=0 2024-04-25T10:13:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.527+0000 7f1abd413740 1 client.7305 shutdown 2024-04-25T10:13:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.528+0000 7f1abd413740 1 -- 172.21.15.119:0/4170014872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd83cd0 msgr2=0x55a63fd89d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.528+0000 7f1abd413740 1 --2- 172.21.15.119:0/4170014872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd83cd0 0x55a63fd89d50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40156c0 tx=0x7f1aa4009190 comp rx=0 tx=0).stop 2024-04-25T10:13:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.528+0000 7f1abd413740 1 -- 172.21.15.119:0/4170014872 shutdown_connections 2024-04-25T10:13:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.528+0000 7f1abd413740 1 --2- 172.21.15.119:0/4170014872 >> [v2:172.21.15.119:6835/3997964942,v1:172.21.15.119:6837/3997964942] conn(0x55a63fd88210 0x55a63fd7f920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.528+0000 7f1abd413740 1 --2- 172.21.15.119:0/4170014872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fd7d440 0x55a63fd7c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.528+0000 7f1abd413740 1 --2- 172.21.15.119:0/4170014872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fda45c0 0x55a63fda3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.528+0000 7f1abd413740 1 --2- 172.21.15.119:0/4170014872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd83cd0 0x55a63fd89d50 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.528+0000 7f1abd413740 1 -- 172.21.15.119:0/4170014872 >> 172.21.15.119:0/4170014872 conn(0x55a63fd5b950 msgr2=0x55a63fd558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.528+0000 7f1a9cc00640 0 client.7305 ms_handle_reset on v2:172.21.15.119:6835/3997964942 2024-04-25T10:13:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.528+0000 7f1abd413740 1 -- 172.21.15.119:0/4170014872 shutdown_connections 2024-04-25T10:13:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.528+0000 7f1abd413740 1 -- 172.21.15.119:0/4170014872 wait complete. 2024-04-25T10:13:30.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.528+0000 7f1abd413740 20 client.7305 trim_cache size 0 max 16384 2024-04-25T10:13:30.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.545+0000 7f1abd413740 1 Processor -- start 2024-04-25T10:13:30.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.545+0000 7f1abd413740 1 -- start start 2024-04-25T10:13:30.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.546+0000 7f1abd413740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd7d440 0x55a63fd871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.546+0000 7f1abd413740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fd83cd0 0x55a63fd5bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.546+0000 7f1abd413740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fda45c0 0x55a63fd5c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.546+0000 7f1abd413740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab40b0000 con 0x55a63fd7d440 2024-04-25T10:13:30.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.546+0000 7f1abd413740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab40af400 con 0x55a63fda45c0 2024-04-25T10:13:30.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.546+0000 7f1abd413740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab40afa00 con 0x55a63fd83cd0 2024-04-25T10:13:30.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.546+0000 7f1ab3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd7d440 0x55a63fd871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.546+0000 7f1ab3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd7d440 0x55a63fd871e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52804/0 (socket says 172.21.15.119:52804) 2024-04-25T10:13:30.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.546+0000 7f1ab3400640 1 -- 172.21.15.119:0/1297040821 learned_addr learned my addr 172.21.15.119:0/1297040821 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:30.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.547+0000 7f1ab3400640 1 -- 172.21.15.119:0/1297040821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fd83cd0 msgr2=0x55a63fd5bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:30.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.547+0000 7f1ab3400640 1 --2- 172.21.15.119:0/1297040821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fd83cd0 0x55a63fd5bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.547+0000 7f1ab3400640 1 -- 172.21.15.119:0/1297040821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fda45c0 msgr2=0x55a63fd5c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:30.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.547+0000 7f1ab3400640 1 --2- 172.21.15.119:0/1297040821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fda45c0 0x55a63fd5c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.547+0000 7f1ab3400640 1 -- 172.21.15.119:0/1297040821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a63fa9e6b0 con 0x55a63fd7d440 2024-04-25T10:13:30.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.547+0000 7f1ab3400640 1 --2- 172.21.15.119:0/1297040821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd7d440 0x55a63fd871e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4008010 tx=0x7f1aa4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe44e80c42dd446 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:30.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.547+0000 7f1a9f400640 1 -- 172.21.15.119:0/1297040821 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa401c040 con 0x55a63fd7d440 2024-04-25T10:13:30.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.547+0000 7f1a9f400640 1 -- 172.21.15.119:0/1297040821 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa4026070 con 0x55a63fd7d440 2024-04-25T10:13:30.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.547+0000 7f1a9f400640 1 -- 172.21.15.119:0/1297040821 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4014d70 con 0x55a63fd7d440 2024-04-25T10:13:30.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.548+0000 7f1abd413740 1 -- 172.21.15.119:0/1297040821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd7d440 msgr2=0x55a63fd871e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.548+0000 7f1abd413740 1 --2- 172.21.15.119:0/1297040821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd7d440 0x55a63fd871e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4008010 tx=0x7f1aa4020000 comp rx=0 tx=0).stop 2024-04-25T10:13:30.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.548+0000 7f1abd413740 1 -- 172.21.15.119:0/1297040821 shutdown_connections 2024-04-25T10:13:30.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.548+0000 7f1abd413740 1 --2- 172.21.15.119:0/1297040821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fda45c0 0x55a63fd5c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.548+0000 7f1abd413740 1 --2- 172.21.15.119:0/1297040821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fd83cd0 0x55a63fd5bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.548+0000 7f1abd413740 1 --2- 172.21.15.119:0/1297040821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fd7d440 0x55a63fd871e0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.548+0000 7f1abd413740 1 -- 172.21.15.119:0/1297040821 >> 172.21.15.119:0/1297040821 conn(0x55a63fd5b950 msgr2=0x55a63fd7c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:30.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.548+0000 7f1abd413740 1 -- 172.21.15.119:0/1297040821 shutdown_connections 2024-04-25T10:13:30.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.548+0000 7f1abd413740 1 -- 172.21.15.119:0/1297040821 wait complete. 2024-04-25T10:13:30.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.549+0000 7f1abd413740 1 -- start start 2024-04-25T10:13:30.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.549+0000 7f1abd413740 1 Processor -- start 2024-04-25T10:13:30.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.550+0000 7f1abd413740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fd7d440 0x55a63fda1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.550+0000 7f1abd413740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fd83cd0 0x55a63fda1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.550+0000 7f1abd413740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fda45c0 0x55a63fd7d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.550+0000 7f1abd413740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a63fd88090 con 0x55a63fda45c0 2024-04-25T10:13:30.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.550+0000 7f1abd413740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a63fd82330 con 0x55a63fd83cd0 2024-04-25T10:13:30.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.550+0000 7f1abd413740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a63fd88690 con 0x55a63fd7d440 2024-04-25T10:13:30.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.550+0000 7f1abd413740 10 client.0 _ll_register_callbacks cb 0x7f1aa4009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:30.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.550+0000 7f1ab3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fda45c0 0x55a63fd7d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.550+0000 7f1ab3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fda45c0 0x55a63fd7d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52806/0 (socket says 172.21.15.119:52806) 2024-04-25T10:13:30.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.550+0000 7f1ab3e00640 1 -- 172.21.15.119:0/2019933375 learned_addr learned my addr 172.21.15.119:0/2019933375 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:30.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.551+0000 7f1ab3400640 1 --2- 172.21.15.119:0/2019933375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fd7d440 0x55a63fda1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.551+0000 7f1ab3e00640 1 -- 172.21.15.119:0/2019933375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fd7d440 msgr2=0x55a63fda1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:30.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.551+0000 7f1ab3e00640 1 --2- 172.21.15.119:0/2019933375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fd7d440 0x55a63fda1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.551+0000 7f1ab3e00640 1 -- 172.21.15.119:0/2019933375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fd83cd0 msgr2=0x55a63fda1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.551+0000 7f1ab2a00640 1 --2- 172.21.15.119:0/2019933375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fd83cd0 0x55a63fda1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.551+0000 7f1ab3e00640 1 --2- 172.21.15.119:0/2019933375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fd83cd0 0x55a63fda1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.551+0000 7f1ab3e00640 1 -- 172.21.15.119:0/2019933375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a63fdc37d0 con 0x55a63fda45c0 2024-04-25T10:13:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.551+0000 7f1ab3400640 1 --2- 172.21.15.119:0/2019933375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fd7d440 0x55a63fda1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.551+0000 7f1ab3e00640 1 --2- 172.21.15.119:0/2019933375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fda45c0 0x55a63fd7d900 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009f40 tx=0x7f1aac078a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450e92526b199c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.551+0000 7f1a9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:13:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.551+0000 7f1a9cc00640 1 -- 172.21.15.119:0/2019933375 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac081260 con 0x55a63fda45c0 2024-04-25T10:13:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.551+0000 7f1a9cc00640 1 -- 172.21.15.119:0/2019933375 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1aac078250 con 0x55a63fda45c0 2024-04-25T10:13:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.551+0000 7f1abd413740 1 -- 172.21.15.119:0/2019933375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1aa4026070 con 0x55a63fda45c0 2024-04-25T10:13:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.552+0000 7f1a9cc00640 1 -- 172.21.15.119:0/2019933375 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aac078e00 con 0x55a63fda45c0 2024-04-25T10:13:30.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.552+0000 7f1abd413740 10 client.7311 fetch_fsmap learned FSMap version 61 2024-04-25T10:13:30.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.552+0000 7f1abd413740 1 -- 172.21.15.119:0/2019933375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55a63fd7e5d0 con 0x55a63fda45c0 2024-04-25T10:13:30.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.552+0000 7f1a9cc00640 1 -- 172.21.15.119:0/2019933375 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa4026070 con 0x55a63fda45c0 2024-04-25T10:13:30.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.552+0000 7f1a9cc00640 1 -- 172.21.15.119:0/2019933375 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1aac090b70 con 0x55a63fda45c0 2024-04-25T10:13:30.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.552+0000 7f1abd413740 10 client.7311 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T10:13:30.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.552+0000 7f1abd413740 10 client.7311 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:13:30.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.552+0000 7f1abd413740 20 client.7311 populate_metadata read hostname 'smithi119' 2024-04-25T10:13:30.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.552+0000 7f1abd413740 1 --2- 172.21.15.119:0/2019933375 >> [v2:172.21.15.119:6835/3997964942,v1:172.21.15.119:6837/3997964942] conn(0x55a63fd7de40 0x55a63fd558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.552+0000 7f1abd413740 4 client.7311 mds_command: new command op to 7158 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:13:30.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.552+0000 7f1abd413740 1 -- 172.21.15.119:0/2019933375 --> [v2:172.21.15.119:6835/3997964942,v1:172.21.15.119:6837/3997964942] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a63fd592d0 con 0x55a63fd7de40 2024-04-25T10:13:30.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.552+0000 7f1ab3400640 1 --2- 172.21.15.119:0/2019933375 >> [v2:172.21.15.119:6835/3997964942,v1:172.21.15.119:6837/3997964942] conn(0x55a63fd7de40 0x55a63fd558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1ab3400640 1 --2- 172.21.15.119:0/2019933375 >> [v2:172.21.15.119:6835/3997964942,v1:172.21.15.119:6837/3997964942] conn(0x55a63fd7de40 0x55a63fd558b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c68fcafcfab83b server_cookie=852c16378e1f26b0 in_seq=0 out_seq=0 2024-04-25T10:13:30.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1a9cc00640 10 client.7311 ms_handle_connect on v2:172.21.15.119:6835/3997964942 2024-04-25T10:13:30.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1a9cc00640 1 -- 172.21.15.119:0/2019933375 <== mds.0 v2:172.21.15.119:6835/3997964942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55a63fd592d0 con 0x55a63fd7de40 2024-04-25T10:13:30.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1a9cc00640 10 client.7311 handle_command_reply: tid=0 2024-04-25T10:13:30.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1abd413740 1 client.7311 shutdown 2024-04-25T10:13:30.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1abd413740 1 -- 172.21.15.119:0/2019933375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fda45c0 msgr2=0x55a63fd7d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1abd413740 1 --2- 172.21.15.119:0/2019933375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fda45c0 0x55a63fd7d900 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009f40 tx=0x7f1aac078a70 comp rx=0 tx=0).stop 2024-04-25T10:13:30.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1abd413740 1 -- 172.21.15.119:0/2019933375 shutdown_connections 2024-04-25T10:13:30.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1abd413740 1 --2- 172.21.15.119:0/2019933375 >> [v2:172.21.15.119:6835/3997964942,v1:172.21.15.119:6837/3997964942] conn(0x55a63fd7de40 0x55a63fd558b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1abd413740 1 --2- 172.21.15.119:0/2019933375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a63fda45c0 0x55a63fd7d900 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1abd413740 1 --2- 172.21.15.119:0/2019933375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a63fd83cd0 0x55a63fda1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1abd413740 1 --2- 172.21.15.119:0/2019933375 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a63fd7d440 0x55a63fda1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1a9cc00640 0 client.7311 ms_handle_reset on v2:172.21.15.119:6835/3997964942 2024-04-25T10:13:30.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1abd413740 1 -- 172.21.15.119:0/2019933375 >> 172.21.15.119:0/2019933375 conn(0x55a63fd5b950 msgr2=0x55a63fd86490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:30.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1abd413740 1 -- 172.21.15.119:0/2019933375 shutdown_connections 2024-04-25T10:13:30.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.553+0000 7f1abd413740 1 -- 172.21.15.119:0/2019933375 wait complete. 2024-04-25T10:13:30.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.554+0000 7f1abd413740 20 client.7311 trim_cache size 0 max 16384 2024-04-25T10:13:30.565 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:13:30.565 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "e0b0f428-b441-4237-a4d1-ed53321f87a5", 2024-04-25T10:13:30.565 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-04-25T10:13:30.565 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 7158, 2024-04-25T10:13:30.565 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-04-25T10:13:30.566 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-04-25T10:13:30.566 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-04-25T10:13:30.566 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.5589662820000001, 2024-04-25T10:13:30.566 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 61, 2024-04-25T10:13:30.566 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 81, 2024-04-25T10:13:30.566 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 81, 2024-04-25T10:13:30.566 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 11.994631792 2024-04-25T10:13:30.566 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:13:30.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.554+0000 7f1aba600640 1 -- 172.21.15.119:0/3011339510 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a9406b500 msgr2=0x7f1a9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.554+0000 7f1aba600640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a9406b500 0x7f1a9406d9c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009a30 tx=0x7f1aa4008040 comp rx=0 tx=0).stop 2024-04-25T10:13:30.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.554+0000 7f1aba600640 1 -- 172.21.15.119:0/3011339510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab4145260 msgr2=0x7f1ab41f3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.554+0000 7f1aba600640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab4145260 0x7f1ab41f3c40 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009f70 tx=0x7f1aac004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:30.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.554+0000 7f1aba600640 1 -- 172.21.15.119:0/3011339510 shutdown_connections 2024-04-25T10:13:30.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.554+0000 7f1aba600640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a9406b500 0x7f1a9406d9c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.554+0000 7f1aba600640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab4145260 0x7f1ab41f3c40 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.554+0000 7f1aba600640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ab40cc490 0x7f1ab415b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.554+0000 7f1aba600640 1 --2- 172.21.15.119:0/3011339510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4071910 0x7f1ab40b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.554+0000 7f1aba600640 1 -- 172.21.15.119:0/3011339510 >> 172.21.15.119:0/3011339510 conn(0x7f1ab406aba0 msgr2=0x7f1ab40cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:30.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.554+0000 7f1aba600640 1 -- 172.21.15.119:0/3011339510 shutdown_connections 2024-04-25T10:13:30.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.554+0000 7f1aba600640 1 -- 172.21.15.119:0/3011339510 wait complete. 2024-04-25T10:13:30.575 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:13:30.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.761+0000 7fa480e00640 1 Processor -- start 2024-04-25T10:13:30.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.761+0000 7fa480e00640 1 -- start start 2024-04-25T10:13:30.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.763+0000 7fa480e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa47c15ed70 0x7fa47c168a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.763+0000 7fa480e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c0712c0 con 0x7fa47c15d980 2024-04-25T10:13:30.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.763+0000 7fa480e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c0715a0 con 0x7fa47c0aecf0 2024-04-25T10:13:30.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.763+0000 7fa480e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47c071ed0 con 0x7fa47c15ed70 2024-04-25T10:13:30.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.763+0000 7fa479e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa47c15d980 0x7fa47c0ae5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48524/0 (socket says 172.21.15.119:48524) 2024-04-25T10:13:30.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.763+0000 7fa479e00640 1 -- 172.21.15.119:0/4251761654 learned_addr learned my addr 172.21.15.119:0/4251761654 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:30.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.763+0000 7fa47a800640 1 --2- 172.21.15.119:0/4251761654 >> v2:172.21.15.119:3300/0 conn(0x7fa47c15ed70 0x7fa47c168a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.764+0000 7fa47a800640 1 -- 172.21.15.119:0/4251761654 >> v1:172.21.15.132:6789/0 conn(0x7fa47c0aecf0 legacy=0x7fa47c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:30.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.764+0000 7fa47a800640 1 -- 172.21.15.119:0/4251761654 >> v1:172.21.15.119:6789/0 conn(0x7fa47c15d980 legacy=0x7fa47c0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.764+0000 7fa47a800640 1 -- 172.21.15.119:0/4251761654 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c071880 con 0x7fa47c15ed70 2024-04-25T10:13:30.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.764+0000 7fa479400640 1 -- 172.21.15.119:0/4251761654 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 133939894 0 0) 0x7fa47c0712c0 con 0x7fa47c15d980 2024-04-25T10:13:30.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.765+0000 7fa47a800640 1 --2- 172.21.15.119:0/4251761654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa47c15ed70 0x7fa47c168a40 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fa474009d90 tx=0x7fa47402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf30db15e478eff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:30.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.765+0000 7fa479400640 1 -- 172.21.15.119:0/4251761654 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474031600 con 0x7fa47c15ed70 2024-04-25T10:13:30.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.765+0000 7fa479400640 1 -- 172.21.15.119:0/4251761654 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4740317a0 con 0x7fa47c15ed70 2024-04-25T10:13:30.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.765+0000 7fa479400640 1 -- 172.21.15.119:0/4251761654 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474031b70 con 0x7fa47c15ed70 2024-04-25T10:13:30.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.766+0000 7fa480e00640 1 -- 172.21.15.119:0/4251761654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa47c15ed70 msgr2=0x7fa47c168a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.766+0000 7fa480e00640 1 --2- 172.21.15.119:0/4251761654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa47c15ed70 0x7fa47c168a40 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fa474009d90 tx=0x7fa47402a610 comp rx=0 tx=0).stop 2024-04-25T10:13:30.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.766+0000 7fa480e00640 1 -- 172.21.15.119:0/4251761654 shutdown_connections 2024-04-25T10:13:30.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.766+0000 7fa480e00640 1 --2- 172.21.15.119:0/4251761654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa47c15ed70 0x7fa47c168a40 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.766+0000 7fa480e00640 1 -- 172.21.15.119:0/4251761654 >> 172.21.15.119:0/4251761654 conn(0x7fa47c06aba0 msgr2=0x7fa47c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:30.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.766+0000 7fa480e00640 1 -- 172.21.15.119:0/4251761654 shutdown_connections 2024-04-25T10:13:30.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.766+0000 7fa480e00640 1 -- 172.21.15.119:0/4251761654 wait complete. 2024-04-25T10:13:30.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.768+0000 7fa480e00640 1 Processor -- start 2024-04-25T10:13:30.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.768+0000 7fa480e00640 1 -- start start 2024-04-25T10:13:30.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.769+0000 7fa480e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa47c15d980 0x7fa47c0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.769+0000 7fa480e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa47c145360 0x7fa47c0b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.769+0000 7fa480e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa47c0b1250 0x7fa47c0b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.769+0000 7fa480e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0cfb10 con 0x7fa47c0b1250 2024-04-25T10:13:30.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.769+0000 7fa480e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0cf990 con 0x7fa47c145360 2024-04-25T10:13:30.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.769+0000 7fa480e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0adf80 con 0x7fa47c15d980 2024-04-25T10:13:30.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.769+0000 7fa47a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa47c0b1250 0x7fa47c0b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.769+0000 7fa47a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa47c0b1250 0x7fa47c0b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52834/0 (socket says 172.21.15.119:52834) 2024-04-25T10:13:30.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.769+0000 7fa47a800640 1 -- 172.21.15.119:0/333931231 learned_addr learned my addr 172.21.15.119:0/333931231 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:30.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.770+0000 7fa473e00640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa47c145360 0x7fa47c0b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.770+0000 7fa479e00640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa47c15d980 0x7fa47c0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.770+0000 7fa47a800640 1 -- 172.21.15.119:0/333931231 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa47c15d980 msgr2=0x7fa47c0cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.770+0000 7fa47a800640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa47c15d980 0x7fa47c0cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.770+0000 7fa47a800640 1 -- 172.21.15.119:0/333931231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa47c145360 msgr2=0x7fa47c0b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.770+0000 7fa47a800640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa47c145360 0x7fa47c0b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.770+0000 7fa47a800640 1 -- 172.21.15.119:0/333931231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c145a90 con 0x7fa47c0b1250 2024-04-25T10:13:30.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.770+0000 7fa473e00640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa47c145360 0x7fa47c0b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:30.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.770+0000 7fa479e00640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa47c15d980 0x7fa47c0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:30.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.770+0000 7fa47a800640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa47c0b1250 0x7fa47c0b16d0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa474009eb0 tx=0x7fa474004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:30.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.770+0000 7fa472000640 1 -- 172.21.15.119:0/333931231 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474040070 con 0x7fa47c0b1250 2024-04-25T10:13:30.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.770+0000 7fa472000640 1 -- 172.21.15.119:0/333931231 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa474030b60 con 0x7fa47c0b1250 2024-04-25T10:13:30.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.770+0000 7fa480e00640 1 -- 172.21.15.119:0/333931231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47c145d70 con 0x7fa47c0b1250 2024-04-25T10:13:30.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.770+0000 7fa472000640 1 -- 172.21.15.119:0/333931231 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474045460 con 0x7fa47c0b1250 2024-04-25T10:13:30.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.771+0000 7fa480e00640 1 -- 172.21.15.119:0/333931231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47c146330 con 0x7fa47c0b1250 2024-04-25T10:13:30.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.772+0000 7fa480e00640 1 -- 172.21.15.119:0/333931231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47c0b1c10 con 0x7fa47c0b1250 2024-04-25T10:13:30.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.772+0000 7fa472000640 1 -- 172.21.15.119:0/333931231 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa474039040 con 0x7fa47c0b1250 2024-04-25T10:13:30.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.773+0000 7fa472000640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa45806b4d0 0x7fa45806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:30.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.773+0000 7fa472000640 1 -- 172.21.15.119:0/333931231 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4740abaa0 con 0x7fa47c0b1250 2024-04-25T10:13:30.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.774+0000 7fa479e00640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa45806b4d0 0x7fa45806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:30.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.774+0000 7fa479e00640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa45806b4d0 0x7fa45806d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa46c000f60 tx=0x7fa46c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:30.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:30.778+0000 7fa472000640 1 -- 172.21.15.119:0/333931231 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa47407a940 con 0x7fa47c0b1250 2024-04-25T10:13:31.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:31.013+0000 7fa480e00640 1 -- 172.21.15.119:0/333931231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa47c0749b0 con 0x7fa47c0b1250 2024-04-25T10:13:31.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:31.014+0000 7fa472000640 1 -- 172.21.15.119:0/333931231 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa47407a240 con 0x7fa47c0b1250 2024-04-25T10:13:31.018 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:13:31.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:31.016+0000 7fa480e00640 1 -- 172.21.15.119:0/333931231 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa45806b4d0 msgr2=0x7fa45806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:31.016+0000 7fa480e00640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa45806b4d0 0x7fa45806d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa46c000f60 tx=0x7fa46c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:31.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:31.016+0000 7fa480e00640 1 -- 172.21.15.119:0/333931231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa47c0b1250 msgr2=0x7fa47c0b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:31.016+0000 7fa480e00640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa47c0b1250 0x7fa47c0b16d0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa474009eb0 tx=0x7fa474004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:31.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:31.016+0000 7fa480e00640 1 -- 172.21.15.119:0/333931231 shutdown_connections 2024-04-25T10:13:31.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:31.016+0000 7fa480e00640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa45806b4d0 0x7fa45806d990 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:31.016+0000 7fa480e00640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa47c0b1250 0x7fa47c0b16d0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:31.016+0000 7fa480e00640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa47c145360 0x7fa47c0b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:31.016+0000 7fa480e00640 1 --2- 172.21.15.119:0/333931231 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa47c15d980 0x7fa47c0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:31.016+0000 7fa480e00640 1 -- 172.21.15.119:0/333931231 >> 172.21.15.119:0/333931231 conn(0x7fa47c06aba0 msgr2=0x7fa47c15caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:31.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:31.016+0000 7fa480e00640 1 -- 172.21.15.119:0/333931231 shutdown_connections 2024-04-25T10:13:31.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:31.016+0000 7fa480e00640 1 -- 172.21.15.119:0/333931231 wait complete. 2024-04-25T10:13:31.039 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:13:31.039 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:13:31.039 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:13:31.039 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:13:31.039 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:31.039 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:13:31.040 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:13:31.040 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:13:31.040 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:31.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:31.040 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:13:31.087 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:13:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:13:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:13:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:13:31.087 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:31.087 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:13:31.088 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:13:31.088 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:31.088 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:13:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:13:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42278sec preferred_lft 42278sec 2024-04-25T10:13:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:13:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:31.088 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:13:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:13:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:13:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:31.089 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:13:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:31.089 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:13:31.089 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:31.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:31.090 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:13:31.156 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:13:31.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:31.160 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:13:31.208 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:13:31.209 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:13:31.209 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:13:31.209 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:13:31.209 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:13:31.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:31.209 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:13:31.209 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:13:31.210 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:13:31.210 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:13:31.210 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:13:31.210 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:13:31.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:31.388 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:13:31.437 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:13:31.437 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:13:31.438 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:31.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:31.438 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:13:31.522 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:31.523 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:31.577 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:31.578 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:31.578 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:31.628 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:13:31.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:31.628 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:31.716 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:31.723 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:31.755 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:31.755 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:31.755 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:13:31.774 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:31.774 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:31.774 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:13:31.774 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:13:31.774 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:31.798 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:31.798 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:31.798 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:13:31.798 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:13:31.798 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:31.798 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:13:31.798 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:13:31.798 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:13:31.800 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:13:31.800 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:13:31.800 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:13:31.800 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:13:31.800 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:13:31.803 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:13:31.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:31.804 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:31.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:31.944 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:13:32.009 INFO:teuthology.orchestra.run.smithi119.stdout:client5243 2024-04-25T10:13:32.010 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:13:32.010 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:32.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:32.060 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-25T10:13:32.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:32.115 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-25T10:13:32.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:32.165 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-25T10:13:32.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:32.214 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-25T10:13:32.272 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:13:32.273 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:13:32.273 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:13:32.273 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:13:32.273 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-25T10:13:32.273 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:13:32.397 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627776 2024-04-25T10:13:32.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:32.410 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:13:32.410 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:32.505 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:13:32.505 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:32.555 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:32.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.800+0000 7fbb04e00640 1 Processor -- start 2024-04-25T10:13:32.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.800+0000 7fbb04e00640 1 -- start start 2024-04-25T10:13:32.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.802+0000 7fbb04e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb0015acd0 0x7fbb0006e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.802+0000 7fbb04e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbb0006e860 0x7fbb000b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.802+0000 7fbb04e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb00144420 con 0x7fbb00144040 2024-04-25T10:13:32.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.802+0000 7fbb04e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb0006ec90 con 0x7fbb0006e860 2024-04-25T10:13:32.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.802+0000 7fbb04e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb000b3d80 con 0x7fbb0015acd0 2024-04-25T10:13:32.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.802+0000 7fbaff400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbb00144040 0x7fbb000cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36958/0 (socket says 172.21.15.119:36958) 2024-04-25T10:13:32.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.802+0000 7fbaff400640 1 -- 172.21.15.119:0/160335662 learned_addr learned my addr 172.21.15.119:0/160335662 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:32.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.802+0000 7fbaffe00640 1 --2- 172.21.15.119:0/160335662 >> v2:172.21.15.119:3300/0 conn(0x7fbb0006e860 0x7fbb000b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.803+0000 7fbafea00640 1 --2- 172.21.15.119:0/160335662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb0015acd0 0x7fbb0006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.803+0000 7fbafe000640 1 -- 172.21.15.119:0/160335662 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963983673 0 0) 0x7fbb00144420 con 0x7fbb00144040 2024-04-25T10:13:32.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.803+0000 7fbafe000640 1 -- 172.21.15.119:0/160335662 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbae4003520 con 0x7fbb00144040 2024-04-25T10:13:32.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.803+0000 7fbaffe00640 1 -- 172.21.15.119:0/160335662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb0015acd0 msgr2=0x7fbb0006e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.803+0000 7fbaffe00640 1 --2- 172.21.15.119:0/160335662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb0015acd0 0x7fbb0006e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.803+0000 7fbaffe00640 1 -- 172.21.15.119:0/160335662 >> v1:172.21.15.119:6789/0 conn(0x7fbb00144040 legacy=0x7fbb000cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.803+0000 7fbaffe00640 1 -- 172.21.15.119:0/160335662 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb000b4060 con 0x7fbb0006e860 2024-04-25T10:13:32.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.804+0000 7fbaffe00640 1 --2- 172.21.15.119:0/160335662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb0006e860 0x7fbb000b26c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009d90 tx=0x7fbaf802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3636ecdbc753d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.804+0000 7fbafe000640 1 -- 172.21.15.119:0/160335662 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8031600 con 0x7fbb0006e860 2024-04-25T10:13:32.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.804+0000 7fbafe000640 1 -- 172.21.15.119:0/160335662 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf80317a0 con 0x7fbb0006e860 2024-04-25T10:13:32.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.804+0000 7fbafe000640 1 -- 172.21.15.119:0/160335662 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8031b70 con 0x7fbb0006e860 2024-04-25T10:13:32.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.805+0000 7fbb04e00640 1 -- 172.21.15.119:0/160335662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb0006e860 msgr2=0x7fbb000b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.805+0000 7fbb04e00640 1 --2- 172.21.15.119:0/160335662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb0006e860 0x7fbb000b26c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009d90 tx=0x7fbaf802a5d0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.805+0000 7fbb04e00640 1 -- 172.21.15.119:0/160335662 shutdown_connections 2024-04-25T10:13:32.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.805+0000 7fbb04e00640 1 --2- 172.21.15.119:0/160335662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb0006e860 0x7fbb000b26c0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.805+0000 7fbb04e00640 1 --2- 172.21.15.119:0/160335662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb0015acd0 0x7fbb0006e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.805+0000 7fbb04e00640 1 -- 172.21.15.119:0/160335662 >> 172.21.15.119:0/160335662 conn(0x7fbb0006b1d0 msgr2=0x7fbb000b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.805+0000 7fbb04e00640 1 -- 172.21.15.119:0/160335662 shutdown_connections 2024-04-25T10:13:32.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.805+0000 7fbb04e00640 1 -- 172.21.15.119:0/160335662 wait complete. 2024-04-25T10:13:32.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.807+0000 7fbb04e00640 1 Processor -- start 2024-04-25T10:13:32.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.807+0000 7fbb04e00640 1 -- start start 2024-04-25T10:13:32.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.808+0000 7fbb04e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb0006e860 0x7fbb0015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.808+0000 7fbb04e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb00144040 0x7fbb000b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.808+0000 7fbb04e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb0015acd0 0x7fbb000b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.808+0000 7fbb04e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb000b4900 con 0x7fbb0006e860 2024-04-25T10:13:32.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.808+0000 7fbb04e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb000b4620 con 0x7fbb0015acd0 2024-04-25T10:13:32.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.808+0000 7fbb04e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb000b5180 con 0x7fbb00144040 2024-04-25T10:13:32.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.808+0000 7fbaff400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb0006e860 0x7fbb0015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.808+0000 7fbaff400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb0006e860 0x7fbb0015a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43104/0 (socket says 172.21.15.119:43104) 2024-04-25T10:13:32.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.808+0000 7fbaff400640 1 -- 172.21.15.119:0/1598621291 learned_addr learned my addr 172.21.15.119:0/1598621291 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:32.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.808+0000 7fbafea00640 1 --2- 172.21.15.119:0/1598621291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb00144040 0x7fbb000b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.809+0000 7fbaff400640 1 -- 172.21.15.119:0/1598621291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb00144040 msgr2=0x7fbb000b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.809+0000 7fbaff400640 1 --2- 172.21.15.119:0/1598621291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb00144040 0x7fbb000b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.809+0000 7fbaff400640 1 -- 172.21.15.119:0/1598621291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb0015acd0 msgr2=0x7fbb000b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.809+0000 7fbaffe00640 1 --2- 172.21.15.119:0/1598621291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb0015acd0 0x7fbb000b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.809+0000 7fbaff400640 1 --2- 172.21.15.119:0/1598621291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb0015acd0 0x7fbb000b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.809+0000 7fbaff400640 1 -- 172.21.15.119:0/1598621291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0013e3e0 con 0x7fbb0006e860 2024-04-25T10:13:32.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.809+0000 7fbaff400640 1 --2- 172.21.15.119:0/1598621291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb0006e860 0x7fbb0015a1f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000c430 tx=0x7fbaf000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.809+0000 7fbaf7400640 1 -- 172.21.15.119:0/1598621291 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0017070 con 0x7fbb0006e860 2024-04-25T10:13:32.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.809+0000 7fbaf7400640 1 -- 172.21.15.119:0/1598621291 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf0005ce0 con 0x7fbb0006e860 2024-04-25T10:13:32.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.809+0000 7fbaf7400640 1 -- 172.21.15.119:0/1598621291 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf00072f0 con 0x7fbb0006e860 2024-04-25T10:13:32.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.809+0000 7fbb04e00640 1 -- 172.21.15.119:0/1598621291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0013e6c0 con 0x7fbb0006e860 2024-04-25T10:13:32.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.809+0000 7fbb04e00640 1 -- 172.21.15.119:0/1598621291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0013ec80 con 0x7fbb0006e860 2024-04-25T10:13:32.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.810+0000 7fbb04e00640 1 -- 172.21.15.119:0/1598621291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb0006bb30 con 0x7fbb0006e860 2024-04-25T10:13:32.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.811+0000 7fbaf7400640 1 -- 172.21.15.119:0/1598621291 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbaf001a070 con 0x7fbb0006e860 2024-04-25T10:13:32.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.812+0000 7fbaf7400640 1 --2- 172.21.15.119:0/1598621291 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbae006b4d0 0x7fbae006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.812+0000 7fbaf7400640 1 -- 172.21.15.119:0/1598621291 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbaf00896f0 con 0x7fbb0006e860 2024-04-25T10:13:32.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.813+0000 7fbafea00640 1 --2- 172.21.15.119:0/1598621291 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbae006b4d0 0x7fbae006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.813+0000 7fbafea00640 1 --2- 172.21.15.119:0/1598621291 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbae006b4d0 0x7fbae006d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbb000ae4d0 tx=0x7fbaec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:32.817+0000 7fbaf7400640 1 -- 172.21.15.119:0/1598621291 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbaf0058560 con 0x7fbb0006e860 2024-04-25T10:13:33.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.012+0000 7fbb04e00640 1 -- 172.21.15.119:0/1598621291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb00067c00 con 0x7fbb0006e860 2024-04-25T10:13:33.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.016+0000 7fbaf7400640 1 -- 172.21.15.119:0/1598621291 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16696 (secure 0 0 0) 0x7fbaf0057e60 con 0x7fbb0006e860 2024-04-25T10:13:33.021 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:33.021 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":81,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:13:24.972756+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:21.210576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:22.438674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:33.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.018+0000 7fbb04e00640 1 -- 172.21.15.119:0/1598621291 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbae006b4d0 msgr2=0x7fbae006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.018+0000 7fbb04e00640 1 --2- 172.21.15.119:0/1598621291 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbae006b4d0 0x7fbae006d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbb000ae4d0 tx=0x7fbaec008040 comp rx=0 tx=0).stop 2024-04-25T10:13:33.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.018+0000 7fbb04e00640 1 -- 172.21.15.119:0/1598621291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb0006e860 msgr2=0x7fbb0015a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.018+0000 7fbb04e00640 1 --2- 172.21.15.119:0/1598621291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb0006e860 0x7fbb0015a1f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000c430 tx=0x7fbaf000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.018+0000 7fbb04e00640 1 -- 172.21.15.119:0/1598621291 shutdown_connections 2024-04-25T10:13:33.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.018+0000 7fbb04e00640 1 --2- 172.21.15.119:0/1598621291 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbae006b4d0 0x7fbae006d990 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.018+0000 7fbb04e00640 1 --2- 172.21.15.119:0/1598621291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb0015acd0 0x7fbb000b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.018+0000 7fbb04e00640 1 --2- 172.21.15.119:0/1598621291 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb00144040 0x7fbb000b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.018+0000 7fbb04e00640 1 --2- 172.21.15.119:0/1598621291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb0006e860 0x7fbb0015a1f0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.018+0000 7fbb04e00640 1 -- 172.21.15.119:0/1598621291 >> 172.21.15.119:0/1598621291 conn(0x7fbb0006b1d0 msgr2=0x7fbb000ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.018+0000 7fbb04e00640 1 -- 172.21.15.119:0/1598621291 shutdown_connections 2024-04-25T10:13:33.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.019+0000 7fbb04e00640 1 -- 172.21.15.119:0/1598621291 wait complete. 2024-04-25T10:13:33.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:21.210576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:22.438674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:33.042 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:33.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.233+0000 7f060b800640 1 Processor -- start 2024-04-25T10:13:33.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.233+0000 7f060b800640 1 -- start start 2024-04-25T10:13:33.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.235+0000 7f060b800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f06040d3b90 0x7f06040d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.235+0000 7f060b800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f06040d0f60 0x7f06040b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.235+0000 7f060b800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06040d4090 con 0x7f060415dca0 2024-04-25T10:13:33.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.235+0000 7f060b800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06040b6b20 con 0x7f06040d0f60 2024-04-25T10:13:33.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.235+0000 7f060b800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06040d1390 con 0x7f06040d3b90 2024-04-25T10:13:33.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.235+0000 7f0609800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f06040d0f60 0x7f06040b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.235+0000 7f0609800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f06040d0f60 0x7f06040b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43106/0 (socket says 172.21.15.119:43106) 2024-04-25T10:13:33.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.235+0000 7f0609800640 1 -- 172.21.15.119:0/1109727101 learned_addr learned my addr 172.21.15.119:0/1109727101 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:33.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.235+0000 7f0603e00640 1 --2- 172.21.15.119:0/1109727101 >> v2:172.21.15.132:3300/0 conn(0x7f06040d3b90 0x7f06040d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.236+0000 7f0609800640 1 -- 172.21.15.119:0/1109727101 >> v2:172.21.15.132:3300/0 conn(0x7f06040d3b90 msgr2=0x7f06040d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:33.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.236+0000 7f0609800640 1 --2- 172.21.15.119:0/1109727101 >> v2:172.21.15.132:3300/0 conn(0x7f06040d3b90 0x7f06040d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.236+0000 7f0609800640 1 -- 172.21.15.119:0/1109727101 >> v1:172.21.15.132:6789/0 conn(0x7f060415dca0 legacy=0x7f06040ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.236+0000 7f0609800640 1 -- 172.21.15.119:0/1109727101 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06040d4370 con 0x7f06040d0f60 2024-04-25T10:13:33.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.237+0000 7f0609800640 1 --2- 172.21.15.119:0/1109727101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06040d0f60 0x7f06040b2500 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f06040b6560 tx=0x7f05fc02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63ffd25ed0d6db6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.238+0000 7f0603400640 1 -- 172.21.15.119:0/1109727101 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc036070 con 0x7f06040d0f60 2024-04-25T10:13:33.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.238+0000 7f0603400640 1 -- 172.21.15.119:0/1109727101 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc031600 con 0x7f06040d0f60 2024-04-25T10:13:33.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.238+0000 7f0603400640 1 -- 172.21.15.119:0/1109727101 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc0319d0 con 0x7f06040d0f60 2024-04-25T10:13:33.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.238+0000 7f060b800640 1 -- 172.21.15.119:0/1109727101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06040d0f60 msgr2=0x7f06040b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.238+0000 7f060b800640 1 --2- 172.21.15.119:0/1109727101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06040d0f60 0x7f06040b2500 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f06040b6560 tx=0x7f05fc02a760 comp rx=0 tx=0).stop 2024-04-25T10:13:33.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.238+0000 7f060b800640 1 -- 172.21.15.119:0/1109727101 shutdown_connections 2024-04-25T10:13:33.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.238+0000 7f060b800640 1 --2- 172.21.15.119:0/1109727101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06040d0f60 0x7f06040b2500 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.238+0000 7f060b800640 1 --2- 172.21.15.119:0/1109727101 >> v2:172.21.15.132:3300/0 conn(0x7f06040d3b90 0x7f06040d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.238+0000 7f060b800640 1 -- 172.21.15.119:0/1109727101 >> 172.21.15.119:0/1109727101 conn(0x7f06040726c0 msgr2=0x7f0604072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.239+0000 7f060b800640 1 -- 172.21.15.119:0/1109727101 shutdown_connections 2024-04-25T10:13:33.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.239+0000 7f060b800640 1 -- 172.21.15.119:0/1109727101 wait complete. 2024-04-25T10:13:33.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.240+0000 7f060b800640 1 Processor -- start 2024-04-25T10:13:33.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.240+0000 7f060b800640 1 -- start start 2024-04-25T10:13:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.241+0000 7f060b800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06040d0f60 0x7f0604153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.241+0000 7f060b800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06040d3b90 0x7f060415f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.241+0000 7f060b800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f060415dca0 0x7f0604150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.241+0000 7f060b800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06040cf050 con 0x7f060415dca0 2024-04-25T10:13:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.241+0000 7f060b800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06040b6d50 con 0x7f06040d0f60 2024-04-25T10:13:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.241+0000 7f060b800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060415e080 con 0x7f06040d3b90 2024-04-25T10:13:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.242+0000 7f0609800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f060415dca0 0x7f0604150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.242+0000 7f0609800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f060415dca0 0x7f0604150710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43112/0 (socket says 172.21.15.119:43112) 2024-04-25T10:13:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.242+0000 7f0609800640 1 -- 172.21.15.119:0/2407738637 learned_addr learned my addr 172.21.15.119:0/2407738637 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.242+0000 7f0608e00640 1 --2- 172.21.15.119:0/2407738637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06040d0f60 0x7f0604153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.242+0000 7f0603e00640 1 --2- 172.21.15.119:0/2407738637 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06040d3b90 0x7f060415f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.242+0000 7f0609800640 1 -- 172.21.15.119:0/2407738637 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06040d3b90 msgr2=0x7f060415f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.242+0000 7f0609800640 1 --2- 172.21.15.119:0/2407738637 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06040d3b90 0x7f060415f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.242+0000 7f0609800640 1 -- 172.21.15.119:0/2407738637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06040d0f60 msgr2=0x7f0604153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.242+0000 7f0609800640 1 --2- 172.21.15.119:0/2407738637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06040d0f60 0x7f0604153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.242+0000 7f0609800640 1 -- 172.21.15.119:0/2407738637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0604160300 con 0x7f060415dca0 2024-04-25T10:13:33.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.243+0000 7f0609800640 1 --2- 172.21.15.119:0/2407738637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f060415dca0 0x7f0604150710 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f05fc034040 tx=0x7f05fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.243+0000 7f0600c00640 1 -- 172.21.15.119:0/2407738637 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc036050 con 0x7f060415dca0 2024-04-25T10:13:33.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.243+0000 7f0600c00640 1 -- 172.21.15.119:0/2407738637 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc042070 con 0x7f060415dca0 2024-04-25T10:13:33.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.243+0000 7f060b800640 1 -- 172.21.15.119:0/2407738637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06041605e0 con 0x7f060415dca0 2024-04-25T10:13:33.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.243+0000 7f0600c00640 1 -- 172.21.15.119:0/2407738637 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc009510 con 0x7f060415dca0 2024-04-25T10:13:33.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.243+0000 7f060b800640 1 -- 172.21.15.119:0/2407738637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0604160ba0 con 0x7f060415dca0 2024-04-25T10:13:33.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.244+0000 7f060b800640 1 -- 172.21.15.119:0/2407738637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06040750b0 con 0x7f060415dca0 2024-04-25T10:13:33.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.245+0000 7f0600c00640 1 -- 172.21.15.119:0/2407738637 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05fc007bb0 con 0x7f060415dca0 2024-04-25T10:13:33.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.246+0000 7f0600c00640 1 --2- 172.21.15.119:0/2407738637 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f05e006b500 0x7f05e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.246+0000 7f0600c00640 1 -- 172.21.15.119:0/2407738637 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05fc0aafe0 con 0x7f060415dca0 2024-04-25T10:13:33.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.246+0000 7f0608e00640 1 --2- 172.21.15.119:0/2407738637 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f05e006b500 0x7f05e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.247+0000 7f0608e00640 1 --2- 172.21.15.119:0/2407738637 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f05e006b500 0x7f05e006d9c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f05f4007fd0 tx=0x7f05f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.251+0000 7f0600c00640 1 -- 172.21.15.119:0/2407738637 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05fc079d90 con 0x7f060415dca0 2024-04-25T10:13:33.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.448+0000 7f060b800640 1 -- 172.21.15.119:0/2407738637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06040c70b0 con 0x7f060415dca0 2024-04-25T10:13:33.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.449+0000 7f0600c00640 1 -- 172.21.15.119:0/2407738637 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05fc079690 con 0x7f060415dca0 2024-04-25T10:13:33.454 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:13:33.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.452+0000 7f060b800640 1 -- 172.21.15.119:0/2407738637 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f05e006b500 msgr2=0x7f05e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.452+0000 7f060b800640 1 --2- 172.21.15.119:0/2407738637 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f05e006b500 0x7f05e006d9c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f05f4007fd0 tx=0x7f05f4009040 comp rx=0 tx=0).stop 2024-04-25T10:13:33.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.452+0000 7f060b800640 1 -- 172.21.15.119:0/2407738637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f060415dca0 msgr2=0x7f0604150710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.452+0000 7f060b800640 1 --2- 172.21.15.119:0/2407738637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f060415dca0 0x7f0604150710 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f05fc034040 tx=0x7f05fc004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:33.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.453+0000 7f060b800640 1 -- 172.21.15.119:0/2407738637 shutdown_connections 2024-04-25T10:13:33.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.453+0000 7f060b800640 1 --2- 172.21.15.119:0/2407738637 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f05e006b500 0x7f05e006d9c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.453+0000 7f060b800640 1 --2- 172.21.15.119:0/2407738637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f060415dca0 0x7f0604150710 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.453+0000 7f060b800640 1 --2- 172.21.15.119:0/2407738637 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06040d3b90 0x7f060415f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.453+0000 7f060b800640 1 --2- 172.21.15.119:0/2407738637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06040d0f60 0x7f0604153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.453+0000 7f060b800640 1 -- 172.21.15.119:0/2407738637 >> 172.21.15.119:0/2407738637 conn(0x7f06040726c0 msgr2=0x7f06040b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.453+0000 7f060b800640 1 -- 172.21.15.119:0/2407738637 shutdown_connections 2024-04-25T10:13:33.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.453+0000 7f060b800640 1 -- 172.21.15.119:0/2407738637 wait complete. 2024-04-25T10:13:33.477 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:33.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.677+0000 7f94dd800640 1 Processor -- start 2024-04-25T10:13:33.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.677+0000 7f94dd800640 1 -- start start 2024-04-25T10:13:33.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.678+0000 7f94dd800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f94d815dca0 0x7f94d80ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.678+0000 7f94dd800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f94d80d3b90 0x7f94d80d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.679+0000 7f94dd800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d80d4090 con 0x7f94d80cee80 2024-04-25T10:13:33.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.679+0000 7f94dd800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d80b6b20 con 0x7f94d815dca0 2024-04-25T10:13:33.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.679+0000 7f94dd800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d80d4370 con 0x7f94d80d3b90 2024-04-25T10:13:33.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.679+0000 7f94d7400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f94d80cee80 0x7f94d80b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36962/0 (socket says 172.21.15.119:36962) 2024-04-25T10:13:33.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.679+0000 7f94d7400640 1 -- 172.21.15.119:0/602084067 learned_addr learned my addr 172.21.15.119:0/602084067 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:33.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.679+0000 7f94d6a00640 1 --2- 172.21.15.119:0/602084067 >> v2:172.21.15.119:3300/0 conn(0x7f94d815dca0 0x7f94d80ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.679+0000 7f94d6000640 1 --2- 172.21.15.119:0/602084067 >> v2:172.21.15.132:3300/0 conn(0x7f94d80d3b90 0x7f94d80d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.679+0000 7f94d5600640 1 -- 172.21.15.119:0/602084067 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792260845 0 0) 0x7f94d80d4090 con 0x7f94d80cee80 2024-04-25T10:13:33.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.679+0000 7f94d5600640 1 -- 172.21.15.119:0/602084067 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94bc003520 con 0x7f94d80cee80 2024-04-25T10:13:33.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.680+0000 7f94d6a00640 1 -- 172.21.15.119:0/602084067 >> v2:172.21.15.132:3300/0 conn(0x7f94d80d3b90 msgr2=0x7f94d80d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:33.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.680+0000 7f94d6a00640 1 --2- 172.21.15.119:0/602084067 >> v2:172.21.15.132:3300/0 conn(0x7f94d80d3b90 0x7f94d80d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.680+0000 7f94d6a00640 1 -- 172.21.15.119:0/602084067 >> v1:172.21.15.119:6789/0 conn(0x7f94d80cee80 legacy=0x7f94d80b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.680+0000 7f94d6a00640 1 -- 172.21.15.119:0/602084067 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d80d4650 con 0x7f94d815dca0 2024-04-25T10:13:33.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.681+0000 7f94d6a00640 1 --2- 172.21.15.119:0/602084067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d815dca0 0x7f94d80ce940 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f94d80b6560 tx=0x7f94c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c024169502001ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.681+0000 7f94d5600640 1 -- 172.21.15.119:0/602084067 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8037070 con 0x7f94d815dca0 2024-04-25T10:13:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.681+0000 7f94d5600640 1 -- 172.21.15.119:0/602084067 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c8032440 con 0x7f94d815dca0 2024-04-25T10:13:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.681+0000 7f94d5600640 1 -- 172.21.15.119:0/602084067 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8032810 con 0x7f94d815dca0 2024-04-25T10:13:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.681+0000 7f94dd800640 1 -- 172.21.15.119:0/602084067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d815dca0 msgr2=0x7f94d80ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.681+0000 7f94dd800640 1 --2- 172.21.15.119:0/602084067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d815dca0 0x7f94d80ce940 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f94d80b6560 tx=0x7f94c802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.682+0000 7f94dd800640 1 -- 172.21.15.119:0/602084067 shutdown_connections 2024-04-25T10:13:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.682+0000 7f94dd800640 1 --2- 172.21.15.119:0/602084067 >> v2:172.21.15.132:3300/0 conn(0x7f94d80d3b90 0x7f94d80d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.682+0000 7f94dd800640 1 --2- 172.21.15.119:0/602084067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d815dca0 0x7f94d80ce940 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.682+0000 7f94dd800640 1 -- 172.21.15.119:0/602084067 >> 172.21.15.119:0/602084067 conn(0x7f94d80726c0 msgr2=0x7f94d8072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.682+0000 7f94dd800640 1 -- 172.21.15.119:0/602084067 shutdown_connections 2024-04-25T10:13:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.682+0000 7f94dd800640 1 -- 172.21.15.119:0/602084067 wait complete. 2024-04-25T10:13:33.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.683+0000 7f94dd800640 1 Processor -- start 2024-04-25T10:13:33.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.684+0000 7f94dd800640 1 -- start start 2024-04-25T10:13:33.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.684+0000 7f94dd800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d80cee80 0x7f94d8146780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.684+0000 7f94dd800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94d80d3b90 0x7f94d8154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.684+0000 7f94dd800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94d815dca0 0x7f94d81607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.684+0000 7f94dd800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d815e200 con 0x7f94d80cee80 2024-04-25T10:13:33.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.684+0000 7f94dd800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d815e080 con 0x7f94d815dca0 2024-04-25T10:13:33.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.684+0000 7f94dd800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d80d0f60 con 0x7f94d80d3b90 2024-04-25T10:13:33.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.685+0000 7f94d6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d80cee80 0x7f94d8146780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.685+0000 7f94d6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d80cee80 0x7f94d8146780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43134/0 (socket says 172.21.15.119:43134) 2024-04-25T10:13:33.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.685+0000 7f94d6a00640 1 -- 172.21.15.119:0/3291461434 learned_addr learned my addr 172.21.15.119:0/3291461434 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:33.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.685+0000 7f94d6a00640 1 -- 172.21.15.119:0/3291461434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94d80d3b90 msgr2=0x7f94d8154890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:13:33.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.685+0000 7f94d6a00640 1 --2- 172.21.15.119:0/3291461434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94d80d3b90 0x7f94d8154890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.685+0000 7f94d6a00640 1 -- 172.21.15.119:0/3291461434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94d815dca0 msgr2=0x7f94d81607c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:13:33.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.685+0000 7f94d6a00640 1 --2- 172.21.15.119:0/3291461434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94d815dca0 0x7f94d81607c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.685+0000 7f94d6a00640 1 -- 172.21.15.119:0/3291461434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d8155000 con 0x7f94d80cee80 2024-04-25T10:13:33.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.685+0000 7f94d6a00640 1 --2- 172.21.15.119:0/3291461434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d80cee80 0x7f94d8146780 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f94c8000c00 tx=0x7f94c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.686+0000 7f94cea00640 1 -- 172.21.15.119:0/3291461434 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8037040 con 0x7f94d80cee80 2024-04-25T10:13:33.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.686+0000 7f94cea00640 1 -- 172.21.15.119:0/3291461434 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c8042070 con 0x7f94d80cee80 2024-04-25T10:13:33.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.686+0000 7f94dd800640 1 -- 172.21.15.119:0/3291461434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d81552e0 con 0x7f94d80cee80 2024-04-25T10:13:33.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.686+0000 7f94cea00640 1 -- 172.21.15.119:0/3291461434 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8030960 con 0x7f94d80cee80 2024-04-25T10:13:33.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.686+0000 7f94dd800640 1 -- 172.21.15.119:0/3291461434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d80cc640 con 0x7f94d80cee80 2024-04-25T10:13:33.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.687+0000 7f94dd800640 1 -- 172.21.15.119:0/3291461434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d80750b0 con 0x7f94d80cee80 2024-04-25T10:13:33.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.688+0000 7f94cea00640 1 -- 172.21.15.119:0/3291461434 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94c8030b00 con 0x7f94d80cee80 2024-04-25T10:13:33.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.689+0000 7f94cea00640 1 --2- 172.21.15.119:0/3291461434 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f94b406b500 0x7f94b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.689+0000 7f94cea00640 1 -- 172.21.15.119:0/3291461434 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94c80aab70 con 0x7f94d80cee80 2024-04-25T10:13:33.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.689+0000 7f94d6000640 1 --2- 172.21.15.119:0/3291461434 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f94b406b500 0x7f94b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.690+0000 7f94d6000640 1 --2- 172.21.15.119:0/3291461434 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f94b406b500 0x7f94b406d9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f94d80b54a0 tx=0x7f94c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.693+0000 7f94cea00640 1 -- 172.21.15.119:0/3291461434 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94c8079960 con 0x7f94d80cee80 2024-04-25T10:13:33.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.890+0000 7f94dd800640 1 -- 172.21.15.119:0/3291461434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94d815c470 con 0x7f94d80cee80 2024-04-25T10:13:33.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.891+0000 7f94cea00640 1 -- 172.21.15.119:0/3291461434 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7f94c8079260 con 0x7f94d80cee80 2024-04-25T10:13:33.896 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:13:33.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.894+0000 7f94dd800640 1 -- 172.21.15.119:0/3291461434 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f94b406b500 msgr2=0x7f94b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.894+0000 7f94dd800640 1 --2- 172.21.15.119:0/3291461434 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f94b406b500 0x7f94b406d9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f94d80b54a0 tx=0x7f94c0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:33.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.894+0000 7f94dd800640 1 -- 172.21.15.119:0/3291461434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d80cee80 msgr2=0x7f94d8146780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.894+0000 7f94dd800640 1 --2- 172.21.15.119:0/3291461434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d80cee80 0x7f94d8146780 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f94c8000c00 tx=0x7f94c80046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.894+0000 7f94dd800640 1 -- 172.21.15.119:0/3291461434 shutdown_connections 2024-04-25T10:13:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.894+0000 7f94dd800640 1 --2- 172.21.15.119:0/3291461434 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f94b406b500 0x7f94b406d9c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.894+0000 7f94dd800640 1 --2- 172.21.15.119:0/3291461434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94d815dca0 0x7f94d81607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.894+0000 7f94dd800640 1 --2- 172.21.15.119:0/3291461434 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f94d80d3b90 0x7f94d8154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.894+0000 7f94dd800640 1 --2- 172.21.15.119:0/3291461434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94d80cee80 0x7f94d8146780 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.894+0000 7f94dd800640 1 -- 172.21.15.119:0/3291461434 >> 172.21.15.119:0/3291461434 conn(0x7f94d80726c0 msgr2=0x7f94d80b4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.894+0000 7f94dd800640 1 -- 172.21.15.119:0/3291461434 shutdown_connections 2024-04-25T10:13:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:33.894+0000 7f94dd800640 1 -- 172.21.15.119:0/3291461434 wait complete. 2024-04-25T10:13:33.919 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:34.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.120+0000 7fde3dc00640 1 Processor -- start 2024-04-25T10:13:34.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.120+0000 7fde3dc00640 1 -- start start 2024-04-25T10:13:34.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.122+0000 7fde3dc00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde3815adc0 0x7fde3815b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.122+0000 7fde3dc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fde3815ba00 0x7fde38071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.122+0000 7fde3dc00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde380b2850 con 0x7fde38159ee0 2024-04-25T10:13:34.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.122+0000 7fde3dc00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde380b2b30 con 0x7fde3815adc0 2024-04-25T10:13:34.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.122+0000 7fde3dc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde380b2e10 con 0x7fde3815ba00 2024-04-25T10:13:34.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.122+0000 7fde37e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fde38159ee0 0x7fde380b38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36976/0 (socket says 172.21.15.119:36976) 2024-04-25T10:13:34.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.122+0000 7fde37e00640 1 -- 172.21.15.119:0/524601170 learned_addr learned my addr 172.21.15.119:0/524601170 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:34.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.122+0000 7fde37400640 1 --2- 172.21.15.119:0/524601170 >> v2:172.21.15.119:3300/0 conn(0x7fde3815adc0 0x7fde3815b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.122+0000 7fde36a00640 1 -- 172.21.15.119:0/524601170 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1355604091 0 0) 0x7fde380b2850 con 0x7fde38159ee0 2024-04-25T10:13:34.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.122+0000 7fde3d200640 1 --2- 172.21.15.119:0/524601170 >> v2:172.21.15.132:3300/0 conn(0x7fde3815ba00 0x7fde38071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.122+0000 7fde36a00640 1 -- 172.21.15.119:0/524601170 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde1c003520 con 0x7fde38159ee0 2024-04-25T10:13:34.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.123+0000 7fde37400640 1 -- 172.21.15.119:0/524601170 >> v2:172.21.15.132:3300/0 conn(0x7fde3815ba00 msgr2=0x7fde38071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.123+0000 7fde37400640 1 --2- 172.21.15.119:0/524601170 >> v2:172.21.15.132:3300/0 conn(0x7fde3815ba00 0x7fde38071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.123+0000 7fde37400640 1 -- 172.21.15.119:0/524601170 >> v1:172.21.15.119:6789/0 conn(0x7fde38159ee0 legacy=0x7fde380b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.123+0000 7fde37400640 1 -- 172.21.15.119:0/524601170 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde380b30f0 con 0x7fde3815adc0 2024-04-25T10:13:34.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.123+0000 7fde3d200640 1 --2- 172.21.15.119:0/524601170 >> v2:172.21.15.132:3300/0 conn(0x7fde3815ba00 0x7fde38071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:34.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.124+0000 7fde37400640 1 --2- 172.21.15.119:0/524601170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde3815adc0 0x7fde3815b4c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fde3806e5d0 tx=0x7fde2002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7e835397c7bd4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.124+0000 7fde36a00640 1 -- 172.21.15.119:0/524601170 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20037070 con 0x7fde3815adc0 2024-04-25T10:13:34.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.124+0000 7fde36a00640 1 -- 172.21.15.119:0/524601170 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde20032440 con 0x7fde3815adc0 2024-04-25T10:13:34.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.124+0000 7fde36a00640 1 -- 172.21.15.119:0/524601170 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20032810 con 0x7fde3815adc0 2024-04-25T10:13:34.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.124+0000 7fde3dc00640 1 -- 172.21.15.119:0/524601170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde3815adc0 msgr2=0x7fde3815b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.124+0000 7fde3dc00640 1 --2- 172.21.15.119:0/524601170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde3815adc0 0x7fde3815b4c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fde3806e5d0 tx=0x7fde2002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.125+0000 7fde3dc00640 1 -- 172.21.15.119:0/524601170 shutdown_connections 2024-04-25T10:13:34.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.125+0000 7fde3dc00640 1 --2- 172.21.15.119:0/524601170 >> v2:172.21.15.132:3300/0 conn(0x7fde3815ba00 0x7fde38071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.125+0000 7fde3dc00640 1 --2- 172.21.15.119:0/524601170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde3815adc0 0x7fde3815b4c0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.125+0000 7fde3dc00640 1 -- 172.21.15.119:0/524601170 >> 172.21.15.119:0/524601170 conn(0x7fde38066ff0 msgr2=0x7fde380b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.125+0000 7fde3dc00640 1 -- 172.21.15.119:0/524601170 shutdown_connections 2024-04-25T10:13:34.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.125+0000 7fde3dc00640 1 -- 172.21.15.119:0/524601170 wait complete. 2024-04-25T10:13:34.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.126+0000 7fde3dc00640 1 Processor -- start 2024-04-25T10:13:34.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.127+0000 7fde3dc00640 1 -- start start 2024-04-25T10:13:34.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.127+0000 7fde3dc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde38159ee0 0x7fde3814e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.127+0000 7fde3dc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde3815adc0 0x7fde3814e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.127+0000 7fde3dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde3815ba00 0x7fde38151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.127+0000 7fde3dc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde380b4000 con 0x7fde3815adc0 2024-04-25T10:13:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.127+0000 7fde3dc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3806efc0 con 0x7fde38159ee0 2024-04-25T10:13:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.127+0000 7fde3dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde380b4180 con 0x7fde3815ba00 2024-04-25T10:13:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.128+0000 7fde37400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde3815adc0 0x7fde3814e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.128+0000 7fde37400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde3815adc0 0x7fde3814e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43142/0 (socket says 172.21.15.119:43142) 2024-04-25T10:13:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.128+0000 7fde37400640 1 -- 172.21.15.119:0/1177056116 learned_addr learned my addr 172.21.15.119:0/1177056116 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.128+0000 7fde37e00640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde38159ee0 0x7fde3814e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.128+0000 7fde3d200640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde3815ba00 0x7fde38151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.128+0000 7fde37400640 1 -- 172.21.15.119:0/1177056116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde3815ba00 msgr2=0x7fde38151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.128+0000 7fde37400640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde3815ba00 0x7fde38151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.128+0000 7fde37400640 1 -- 172.21.15.119:0/1177056116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde38159ee0 msgr2=0x7fde3814e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.128+0000 7fde37400640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde38159ee0 0x7fde3814e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.128+0000 7fde37400640 1 -- 172.21.15.119:0/1177056116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde381517f0 con 0x7fde3815adc0 2024-04-25T10:13:34.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.128+0000 7fde37e00640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde38159ee0 0x7fde3814e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:34.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.129+0000 7fde37400640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde3815adc0 0x7fde3814e7e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fde20000c00 tx=0x7fde200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.129+0000 7fde2fe00640 1 -- 172.21.15.119:0/1177056116 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20037040 con 0x7fde3815adc0 2024-04-25T10:13:34.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.129+0000 7fde2fe00640 1 -- 172.21.15.119:0/1177056116 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde20042070 con 0x7fde3815adc0 2024-04-25T10:13:34.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.129+0000 7fde3dc00640 1 -- 172.21.15.119:0/1177056116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde38151ad0 con 0x7fde3815adc0 2024-04-25T10:13:34.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.129+0000 7fde2fe00640 1 -- 172.21.15.119:0/1177056116 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20030960 con 0x7fde3815adc0 2024-04-25T10:13:34.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.129+0000 7fde3dc00640 1 -- 172.21.15.119:0/1177056116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde381588f0 con 0x7fde3815adc0 2024-04-25T10:13:34.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.130+0000 7fde3dc00640 1 -- 172.21.15.119:0/1177056116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde380749b0 con 0x7fde3815adc0 2024-04-25T10:13:34.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.131+0000 7fde2fe00640 1 -- 172.21.15.119:0/1177056116 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde20030b00 con 0x7fde3815adc0 2024-04-25T10:13:34.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.132+0000 7fde2fe00640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fde1406b4d0 0x7fde1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.132+0000 7fde2fe00640 1 -- 172.21.15.119:0/1177056116 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde200aac00 con 0x7fde3815adc0 2024-04-25T10:13:34.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.132+0000 7fde37e00640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fde1406b4d0 0x7fde1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.133+0000 7fde37e00640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fde1406b4d0 0x7fde1406d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fde28005880 tx=0x7fde28009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.136+0000 7fde2fe00640 1 -- 172.21.15.119:0/1177056116 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde20045360 con 0x7fde3815adc0 2024-04-25T10:13:34.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.330+0000 7fde3dc00640 1 -- 172.21.15.119:0/1177056116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde380cf0a0 con 0x7fde3815adc0 2024-04-25T10:13:34.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.331+0000 7fde2fe00640 1 -- 172.21.15.119:0/1177056116 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7fde20079b00 con 0x7fde3815adc0 2024-04-25T10:13:34.336 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:13:34.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.334+0000 7fde3dc00640 1 -- 172.21.15.119:0/1177056116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fde1406b4d0 msgr2=0x7fde1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.334+0000 7fde3dc00640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fde1406b4d0 0x7fde1406d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fde28005880 tx=0x7fde28009040 comp rx=0 tx=0).stop 2024-04-25T10:13:34.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.334+0000 7fde3dc00640 1 -- 172.21.15.119:0/1177056116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde3815adc0 msgr2=0x7fde3814e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.334+0000 7fde3dc00640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde3815adc0 0x7fde3814e7e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fde20000c00 tx=0x7fde200046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.334+0000 7fde3dc00640 1 -- 172.21.15.119:0/1177056116 shutdown_connections 2024-04-25T10:13:34.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.334+0000 7fde3dc00640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fde1406b4d0 0x7fde1406d990 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.334+0000 7fde3dc00640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde3815ba00 0x7fde38151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.334+0000 7fde3dc00640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde3815adc0 0x7fde3814e7e0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.334+0000 7fde3dc00640 1 --2- 172.21.15.119:0/1177056116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde38159ee0 0x7fde3814e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.334+0000 7fde3dc00640 1 -- 172.21.15.119:0/1177056116 >> 172.21.15.119:0/1177056116 conn(0x7fde38066ff0 msgr2=0x7fde380b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.334+0000 7fde3dc00640 1 -- 172.21.15.119:0/1177056116 shutdown_connections 2024-04-25T10:13:34.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.334+0000 7fde3dc00640 1 -- 172.21.15.119:0/1177056116 wait complete. 2024-04-25T10:13:34.358 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:34.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.548+0000 7f4202600640 1 Processor -- start 2024-04-25T10:13:34.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.548+0000 7f4202600640 1 -- start start 2024-04-25T10:13:34.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.549+0000 7f4202600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41fc1450e0 0x7f41fc06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.549+0000 7f4202600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f41fc06b0e0 0x7f41fc0cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.549+0000 7f4202600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41fc15f400 con 0x7f41fc0cc810 2024-04-25T10:13:34.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.549+0000 7f4202600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41fc1454c0 con 0x7f41fc06b0e0 2024-04-25T10:13:34.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.549+0000 7f4202600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc06b540 con 0x7f41fc1450e0 2024-04-25T10:13:34.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.550+0000 7f4201c00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f41fc0cc810 0x7f41fc071050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36988/0 (socket says 172.21.15.119:36988) 2024-04-25T10:13:34.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.550+0000 7f4201c00640 1 -- 172.21.15.119:0/1823877739 learned_addr learned my addr 172.21.15.119:0/1823877739 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:34.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.550+0000 7f4201200640 1 --2- 172.21.15.119:0/1823877739 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41fc1450e0 0x7f41fc06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.550+0000 7f41fbe00640 1 --2- 172.21.15.119:0/1823877739 >> v2:172.21.15.132:3300/0 conn(0x7f41fc06b0e0 0x7f41fc0cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.550+0000 7f41fb400640 1 -- 172.21.15.119:0/1823877739 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017392950 0 0) 0x7f41fc15f400 con 0x7f41fc0cc810 2024-04-25T10:13:34.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.550+0000 7f41fb400640 1 -- 172.21.15.119:0/1823877739 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41e4003520 con 0x7f41fc0cc810 2024-04-25T10:13:34.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fb400640 1 -- 172.21.15.119:0/1823877739 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1903218288 0 0) 0x7f41e4003520 con 0x7f41fc0cc810 2024-04-25T10:13:34.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fb400640 1 -- 172.21.15.119:0/1823877739 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41fc1450e0 msgr2=0x7f41fc06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fb400640 1 --2- 172.21.15.119:0/1823877739 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41fc1450e0 0x7f41fc06aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fb400640 1 -- 172.21.15.119:0/1823877739 >> v2:172.21.15.132:3300/0 conn(0x7f41fc06b0e0 msgr2=0x7f41fc0cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fb400640 1 --2- 172.21.15.119:0/1823877739 >> v2:172.21.15.132:3300/0 conn(0x7f41fc06b0e0 0x7f41fc0cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fb400640 1 -- 172.21.15.119:0/1823877739 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc15f6e0 con 0x7f41fc0cc810 2024-04-25T10:13:34.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fb400640 1 -- 172.21.15.119:0/1823877739 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f41f40019f0 con 0x7f41fc0cc810 2024-04-25T10:13:34.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fb400640 1 -- 172.21.15.119:0/1823877739 >> v1:172.21.15.119:6789/0 conn(0x7f41fc0cc810 legacy=0x7f41fc071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fb400640 1 --2- 172.21.15.119:0/1823877739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41e40039b0 0x7f41e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fb400640 1 --2- 172.21.15.119:0/1823877739 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41fc1450e0 0x7f41e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fb400640 1 --2- 172.21.15.119:0/1823877739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e4004d40 0x7f41e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fb400640 1 -- 172.21.15.119:0/1823877739 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e400b9c0 con 0x7f41e4004d40 2024-04-25T10:13:34.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fb400640 1 -- 172.21.15.119:0/1823877739 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e400bb40 con 0x7f41e40039b0 2024-04-25T10:13:34.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fb400640 1 -- 172.21.15.119:0/1823877739 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e400bd40 con 0x7f41fc1450e0 2024-04-25T10:13:34.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f4201c00640 1 --2- 172.21.15.119:0/1823877739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e4004d40 0x7f41e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f41fbe00640 1 --2- 172.21.15.119:0/1823877739 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41fc1450e0 0x7f41e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f4201c00640 1 -- 172.21.15.119:0/1823877739 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41fc1450e0 msgr2=0x7f41e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.551+0000 7f4201c00640 1 --2- 172.21.15.119:0/1823877739 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41fc1450e0 0x7f41e4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.552+0000 7f4201200640 1 --2- 172.21.15.119:0/1823877739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41e40039b0 0x7f41e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.552+0000 7f4201c00640 1 -- 172.21.15.119:0/1823877739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41e40039b0 msgr2=0x7f41e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.552+0000 7f4201c00640 1 --2- 172.21.15.119:0/1823877739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41e40039b0 0x7f41e4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.552+0000 7f4201c00640 1 -- 172.21.15.119:0/1823877739 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e4003520 con 0x7f41e4004d40 2024-04-25T10:13:34.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.553+0000 7f4201c00640 1 --2- 172.21.15.119:0/1823877739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e4004d40 0x7f41e40051c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f41f400a920 tx=0x7f41e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bbff48b1fa7d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.553+0000 7f41fb400640 1 -- 172.21.15.119:0/1823877739 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f400a4c0 con 0x7f41e4004d40 2024-04-25T10:13:34.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.553+0000 7f41fb400640 1 -- 172.21.15.119:0/1823877739 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f402f070 con 0x7f41e4004d40 2024-04-25T10:13:34.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.553+0000 7f41fb400640 1 -- 172.21.15.119:0/1823877739 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f402bd00 con 0x7f41e4004d40 2024-04-25T10:13:34.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.553+0000 7f4202600640 1 -- 172.21.15.119:0/1823877739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e4004d40 msgr2=0x7f41e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.553+0000 7f4202600640 1 --2- 172.21.15.119:0/1823877739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41e4004d40 0x7f41e40051c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f41f400a920 tx=0x7f41e4005830 comp rx=0 tx=0).stop 2024-04-25T10:13:34.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.553+0000 7f4201c00640 1 -- 172.21.15.119:0/1823877739 reap_dead start 2024-04-25T10:13:34.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.554+0000 7f4202600640 1 -- 172.21.15.119:0/1823877739 shutdown_connections 2024-04-25T10:13:34.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.554+0000 7f4202600640 1 -- 172.21.15.119:0/1823877739 >> 172.21.15.119:0/1823877739 conn(0x7f41fc066ff0 msgr2=0x7f41fc0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.554+0000 7f4202600640 1 -- 172.21.15.119:0/1823877739 shutdown_connections 2024-04-25T10:13:34.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.554+0000 7f4202600640 1 -- 172.21.15.119:0/1823877739 wait complete. 2024-04-25T10:13:34.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.555+0000 7f4202600640 1 Processor -- start 2024-04-25T10:13:34.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.556+0000 7f4202600640 1 -- start start 2024-04-25T10:13:34.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.556+0000 7f4202600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41e4004d40 0x7f41fc15b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.556+0000 7f4202600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41fc0ad0a0 0x7f41fc0ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.556+0000 7f4202600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc0aa970 0x7f41fc0aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.556+0000 7f4202600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc0ccca0 con 0x7f41fc0aa970 2024-04-25T10:13:34.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.556+0000 7f4202600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc15fca0 con 0x7f41e4004d40 2024-04-25T10:13:34.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.556+0000 7f4202600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc0ab6a0 con 0x7f41fc0ad0a0 2024-04-25T10:13:34.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.557+0000 7f4201c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc0aa970 0x7f41fc0aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.557+0000 7f4201c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc0aa970 0x7f41fc0aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43166/0 (socket says 172.21.15.119:43166) 2024-04-25T10:13:34.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.557+0000 7f4201c00640 1 -- 172.21.15.119:0/2812742256 learned_addr learned my addr 172.21.15.119:0/2812742256 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:34.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.557+0000 7f4201200640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41e4004d40 0x7f41fc15b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.557+0000 7f41fbe00640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41fc0ad0a0 0x7f41fc0ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.557+0000 7f4201c00640 1 -- 172.21.15.119:0/2812742256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41fc0ad0a0 msgr2=0x7f41fc0ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.557+0000 7f4201c00640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41fc0ad0a0 0x7f41fc0ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.557+0000 7f4201c00640 1 -- 172.21.15.119:0/2812742256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41e4004d40 msgr2=0x7f41fc15b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.557+0000 7f4201c00640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41e4004d40 0x7f41fc15b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.557+0000 7f4201c00640 1 -- 172.21.15.119:0/2812742256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc0ab980 con 0x7f41fc0aa970 2024-04-25T10:13:34.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.557+0000 7f4201200640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41e4004d40 0x7f41fc15b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:34.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.557+0000 7f4201c00640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc0aa970 0x7f41fc0aae20 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f41f400aa10 tx=0x7f41fc15f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.558+0000 7f41f8c00640 1 -- 172.21.15.119:0/2812742256 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f400a4c0 con 0x7f41fc0aa970 2024-04-25T10:13:34.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.558+0000 7f41f8c00640 1 -- 172.21.15.119:0/2812742256 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f402b8a0 con 0x7f41fc0aa970 2024-04-25T10:13:34.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.558+0000 7f41f8c00640 1 -- 172.21.15.119:0/2812742256 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4002000 con 0x7f41fc0aa970 2024-04-25T10:13:34.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.558+0000 7f4202600640 1 -- 172.21.15.119:0/2812742256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41fc0abc60 con 0x7f41fc0aa970 2024-04-25T10:13:34.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.558+0000 7f4202600640 1 -- 172.21.15.119:0/2812742256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41fc0ac220 con 0x7f41fc0aa970 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.559+0000 7f4202600640 1 -- 172.21.15.119:0/2812742256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41fc0749b0 con 0x7f41fc0aa970 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.560+0000 7f41f8c00640 1 -- 172.21.15.119:0/2812742256 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41f402bc50 con 0x7f41fc0aa970 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.561+0000 7f41f8c00640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41dc06b500 0x7f41dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.561+0000 7f41f8c00640 1 -- 172.21.15.119:0/2812742256 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41f40aa230 con 0x7f41fc0aa970 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.561+0000 7f4201200640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41dc06b500 0x7f41dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.562+0000 7f4201200640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41dc06b500 0x7f41dc06d9c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f41f00099e0 tx=0x7f41f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.566+0000 7f41f8c00640 1 -- 172.21.15.119:0/2812742256 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41f4078f80 con 0x7f41fc0aa970 2024-04-25T10:13:34.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.774+0000 7f4202600640 1 -- 172.21.15.119:0/2812742256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f41fc0b3ba0 con 0x7f41fc0aa970 2024-04-25T10:13:34.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.775+0000 7f41f8c00640 1 -- 172.21.15.119:0/2812742256 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v81) v1 ==== 80+0+0 (secure 0 0 0) 0x7f41f4078880 con 0x7f41fc0aa970 2024-04-25T10:13:34.780 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2024-04-25T10:13:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.777+0000 7f4202600640 1 -- 172.21.15.119:0/2812742256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41dc06b500 msgr2=0x7f41dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.777+0000 7f4202600640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41dc06b500 0x7f41dc06d9c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f41f00099e0 tx=0x7f41f0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.777+0000 7f4202600640 1 -- 172.21.15.119:0/2812742256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc0aa970 msgr2=0x7f41fc0aae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.777+0000 7f4202600640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc0aa970 0x7f41fc0aae20 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f41f400aa10 tx=0x7f41fc15f5d0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.778+0000 7f4202600640 1 -- 172.21.15.119:0/2812742256 shutdown_connections 2024-04-25T10:13:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.778+0000 7f4202600640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41dc06b500 0x7f41dc06d9c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.778+0000 7f4202600640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc0aa970 0x7f41fc0aae20 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.778+0000 7f4202600640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41fc0ad0a0 0x7f41fc0ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.778+0000 7f4202600640 1 --2- 172.21.15.119:0/2812742256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41e4004d40 0x7f41fc15b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.778+0000 7f4202600640 1 -- 172.21.15.119:0/2812742256 >> 172.21.15.119:0/2812742256 conn(0x7f41fc066ff0 msgr2=0x7f41fc0af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.778+0000 7f4202600640 1 -- 172.21.15.119:0/2812742256 shutdown_connections 2024-04-25T10:13:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:34.778+0000 7f4202600640 1 -- 172.21.15.119:0/2812742256 wait complete. 2024-04-25T10:13:34.802 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:13:34.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:34.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:34.942 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:13:34.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:34.942 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:34.987 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:35.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.199+0000 7f8242a00640 1 Processor -- start 2024-04-25T10:13:35.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.199+0000 7f8242a00640 1 -- start start 2024-04-25T10:13:35.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.201+0000 7f8242a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f823c15d980 0x7f823c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.201+0000 7f8242a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f823c0aeb20 0x7f823c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.201+0000 7f8242a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823c0cfbd0 con 0x7f823c0cf7c0 2024-04-25T10:13:35.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.201+0000 7f8242a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c071ed0 con 0x7f823c0aeb20 2024-04-25T10:13:35.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.201+0000 7f8242a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c0712c0 con 0x7f823c15d980 2024-04-25T10:13:35.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.201+0000 7f8240a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f823c0cf7c0 0x7f823c15dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36994/0 (socket says 172.21.15.119:36994) 2024-04-25T10:13:35.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.201+0000 7f8240a00640 1 -- 172.21.15.119:0/3451921723 learned_addr learned my addr 172.21.15.119:0/3451921723 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:35.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.201+0000 7f823b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f823c0aeb20 0x7f823c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.202+0000 7f823be00640 1 --2- 172.21.15.119:0/3451921723 >> v2:172.21.15.132:3300/0 conn(0x7f823c15d980 0x7f823c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.202+0000 7f823b400640 1 -- 172.21.15.119:0/3451921723 >> v2:172.21.15.132:3300/0 conn(0x7f823c15d980 msgr2=0x7f823c0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:35.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.202+0000 7f823b400640 1 --2- 172.21.15.119:0/3451921723 >> v2:172.21.15.132:3300/0 conn(0x7f823c15d980 0x7f823c0ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.202+0000 7f823b400640 1 -- 172.21.15.119:0/3451921723 >> v1:172.21.15.119:6789/0 conn(0x7f823c0cf7c0 legacy=0x7f823c15dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.202+0000 7f823b400640 1 -- 172.21.15.119:0/3451921723 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c0715a0 con 0x7f823c0aeb20 2024-04-25T10:13:35.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.202+0000 7f823aa00640 1 -- 172.21.15.119:0/3451921723 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649713896 0 0) 0x7f8234001490 con 0x7f823c0cf7c0 2024-04-25T10:13:35.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.203+0000 7f823b400640 1 --2- 172.21.15.119:0/3451921723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f823c0aeb20 0x7f823c0cf280 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f823c067460 tx=0x7f822402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8914107164392d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.203+0000 7f823aa00640 1 -- 172.21.15.119:0/3451921723 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224037070 con 0x7f823c0aeb20 2024-04-25T10:13:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.203+0000 7f823aa00640 1 -- 172.21.15.119:0/3451921723 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8224032440 con 0x7f823c0aeb20 2024-04-25T10:13:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.203+0000 7f823aa00640 1 -- 172.21.15.119:0/3451921723 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822402ae60 con 0x7f823c0aeb20 2024-04-25T10:13:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.204+0000 7f8242a00640 1 -- 172.21.15.119:0/3451921723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f823c0aeb20 msgr2=0x7f823c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.204+0000 7f8242a00640 1 --2- 172.21.15.119:0/3451921723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f823c0aeb20 0x7f823c0cf280 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f823c067460 tx=0x7f822402a110 comp rx=0 tx=0).stop 2024-04-25T10:13:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.204+0000 7f8242a00640 1 -- 172.21.15.119:0/3451921723 shutdown_connections 2024-04-25T10:13:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.204+0000 7f8242a00640 1 --2- 172.21.15.119:0/3451921723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f823c0aeb20 0x7f823c0cf280 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.204+0000 7f8242a00640 1 --2- 172.21.15.119:0/3451921723 >> v2:172.21.15.132:3300/0 conn(0x7f823c15d980 0x7f823c0ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.204+0000 7f8242a00640 1 -- 172.21.15.119:0/3451921723 >> 172.21.15.119:0/3451921723 conn(0x7f823c06aba0 msgr2=0x7f823c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.204+0000 7f8242a00640 1 -- 172.21.15.119:0/3451921723 shutdown_connections 2024-04-25T10:13:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.204+0000 7f8242a00640 1 -- 172.21.15.119:0/3451921723 wait complete. 2024-04-25T10:13:35.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.206+0000 7f8242a00640 1 Processor -- start 2024-04-25T10:13:35.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.206+0000 7f8242a00640 1 -- start start 2024-04-25T10:13:35.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.207+0000 7f8242a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f823c0aeb20 0x7f823c150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.207+0000 7f8242a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823c0cf7c0 0x7f823c1567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.207+0000 7f8242a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f823c15d980 0x7f823c156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.207+0000 7f8242a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c0adf80 con 0x7f823c0aeb20 2024-04-25T10:13:35.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.207+0000 7f8242a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c071b60 con 0x7f823c15d980 2024-04-25T10:13:35.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.207+0000 7f8242a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c0ae100 con 0x7f823c0cf7c0 2024-04-25T10:13:35.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.207+0000 7f823be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f823c0aeb20 0x7f823c150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.207+0000 7f823be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f823c0aeb20 0x7f823c150640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43178/0 (socket says 172.21.15.119:43178) 2024-04-25T10:13:35.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.207+0000 7f823be00640 1 -- 172.21.15.119:0/3076556541 learned_addr learned my addr 172.21.15.119:0/3076556541 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:35.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.207+0000 7f823be00640 1 -- 172.21.15.119:0/3076556541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823c0cf7c0 msgr2=0x7f823c1567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.207+0000 7f823be00640 1 --2- 172.21.15.119:0/3076556541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823c0cf7c0 0x7f823c1567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.207+0000 7f823be00640 1 -- 172.21.15.119:0/3076556541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f823c15d980 msgr2=0x7f823c156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.207+0000 7f823be00640 1 --2- 172.21.15.119:0/3076556541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f823c15d980 0x7f823c156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.207+0000 7f823be00640 1 -- 172.21.15.119:0/3076556541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c15aa30 con 0x7f823c0aeb20 2024-04-25T10:13:35.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.208+0000 7f823be00640 1 --2- 172.21.15.119:0/3076556541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f823c0aeb20 0x7f823c150640 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f822c00d800 tx=0x7f822c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.208+0000 7f8233e00640 1 -- 172.21.15.119:0/3076556541 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00b6c0 con 0x7f823c0aeb20 2024-04-25T10:13:35.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.208+0000 7f8233e00640 1 -- 172.21.15.119:0/3076556541 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c019070 con 0x7f823c0aeb20 2024-04-25T10:13:35.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.208+0000 7f8233e00640 1 -- 172.21.15.119:0/3076556541 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00ba80 con 0x7f823c0aeb20 2024-04-25T10:13:35.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.208+0000 7f8242a00640 1 -- 172.21.15.119:0/3076556541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823c15ad10 con 0x7f823c0aeb20 2024-04-25T10:13:35.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.208+0000 7f8242a00640 1 -- 172.21.15.119:0/3076556541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823c15b2d0 con 0x7f823c0aeb20 2024-04-25T10:13:35.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.209+0000 7f8242a00640 1 -- 172.21.15.119:0/3076556541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823c0749b0 con 0x7f823c0aeb20 2024-04-25T10:13:35.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.210+0000 7f8233e00640 1 -- 172.21.15.119:0/3076556541 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f822c00bc20 con 0x7f823c0aeb20 2024-04-25T10:13:35.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.211+0000 7f8233e00640 1 --2- 172.21.15.119:0/3076556541 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f821806b4d0 0x7f821806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.211+0000 7f8233e00640 1 -- 172.21.15.119:0/3076556541 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f822c08a780 con 0x7f823c0aeb20 2024-04-25T10:13:35.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.211+0000 7f823b400640 1 --2- 172.21.15.119:0/3076556541 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f821806b4d0 0x7f821806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.212+0000 7f823b400640 1 --2- 172.21.15.119:0/3076556541 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f821806b4d0 0x7f821806d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8224034000 tx=0x7f822402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.216+0000 7f8233e00640 1 -- 172.21.15.119:0/3076556541 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f822c059560 con 0x7f823c0aeb20 2024-04-25T10:13:35.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.453+0000 7f8242a00640 1 -- 172.21.15.119:0/3076556541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f823c06d4c0 con 0x7f823c0aeb20 2024-04-25T10:13:35.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.454+0000 7f8233e00640 1 -- 172.21.15.119:0/3076556541 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f822c058e60 con 0x7f823c0aeb20 2024-04-25T10:13:35.459 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:35.459 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:13:26.498824+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7158},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7158":{"gid":7158,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/3997964942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3997964942},{"type":"v1","addr":"172.21.15.119:6837","nonce":3997964942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:13:35.459 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 61 2024-04-25T10:13:35.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.457+0000 7f8242a00640 1 -- 172.21.15.119:0/3076556541 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f821806b4d0 msgr2=0x7f821806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.457+0000 7f8242a00640 1 --2- 172.21.15.119:0/3076556541 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f821806b4d0 0x7f821806d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8224034000 tx=0x7f822402c000 comp rx=0 tx=0).stop 2024-04-25T10:13:35.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.457+0000 7f8242a00640 1 -- 172.21.15.119:0/3076556541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f823c0aeb20 msgr2=0x7f823c150640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.457+0000 7f8242a00640 1 --2- 172.21.15.119:0/3076556541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f823c0aeb20 0x7f823c150640 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f822c00d800 tx=0x7f822c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.457+0000 7f8242a00640 1 -- 172.21.15.119:0/3076556541 shutdown_connections 2024-04-25T10:13:35.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.457+0000 7f8242a00640 1 --2- 172.21.15.119:0/3076556541 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f821806b4d0 0x7f821806d990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.457+0000 7f8242a00640 1 --2- 172.21.15.119:0/3076556541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f823c15d980 0x7f823c156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.457+0000 7f8242a00640 1 --2- 172.21.15.119:0/3076556541 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f823c0cf7c0 0x7f823c1567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.457+0000 7f8242a00640 1 --2- 172.21.15.119:0/3076556541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f823c0aeb20 0x7f823c150640 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.457+0000 7f8242a00640 1 -- 172.21.15.119:0/3076556541 >> 172.21.15.119:0/3076556541 conn(0x7f823c06aba0 msgr2=0x7f823c0afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:35.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.457+0000 7f8242a00640 1 -- 172.21.15.119:0/3076556541 shutdown_connections 2024-04-25T10:13:35.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.457+0000 7f8242a00640 1 -- 172.21.15.119:0/3076556541 wait complete. 2024-04-25T10:13:35.480 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:13:35.707 INFO:teuthology.orchestra.run.smithi119.stdout:{"message":"","return_code":0} 2024-04-25T10:13:35.707 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:13:35.708 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:13:35.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.901+0000 7f2551200640 1 Processor -- start 2024-04-25T10:13:35.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.901+0000 7f2551200640 1 -- start start 2024-04-25T10:13:35.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.903+0000 7f2551200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c15dbe0 0x7f254c071c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.903+0000 7f2551200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f254c0cfad0 0x7f254c06d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.903+0000 7f2551200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254c0ace70 con 0x7f254c0aedc0 2024-04-25T10:13:35.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.903+0000 7f2551200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254c072150 con 0x7f254c0cfad0 2024-04-25T10:13:35.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.903+0000 7f2551200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0ad110 con 0x7f254c15dbe0 2024-04-25T10:13:35.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.904+0000 7f2549e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c15dbe0 0x7f254c071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.904+0000 7f254a800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f254c0cfad0 0x7f254c06d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.904+0000 7f2549e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c15dbe0 0x7f254c071c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:60158/0 (socket says 172.21.15.119:60158) 2024-04-25T10:13:35.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.904+0000 7f2549e00640 1 -- 172.21.15.119:0/3592929924 learned_addr learned my addr 172.21.15.119:0/3592929924 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:35.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.904+0000 7f2549e00640 1 -- 172.21.15.119:0/3592929924 >> v2:172.21.15.132:3300/0 conn(0x7f254c0cfad0 msgr2=0x7f254c06d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:35.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.904+0000 7f2549e00640 1 --2- 172.21.15.119:0/3592929924 >> v2:172.21.15.132:3300/0 conn(0x7f254c0cfad0 0x7f254c06d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.904+0000 7f2549e00640 1 -- 172.21.15.119:0/3592929924 >> v1:172.21.15.132:6789/0 conn(0x7f254c0aedc0 legacy=0x7f254c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.905+0000 7f2549e00640 1 -- 172.21.15.119:0/3592929924 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c0ad3f0 con 0x7f254c15dbe0 2024-04-25T10:13:35.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.906+0000 7f2549e00640 1 --2- 172.21.15.119:0/3592929924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c15dbe0 0x7f254c071c10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f253c0098a0 tx=0x7f253c02a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b33ef9e88a18fbc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.906+0000 7f2548a00640 1 -- 172.21.15.119:0/3592929924 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c037070 con 0x7f254c15dbe0 2024-04-25T10:13:35.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.906+0000 7f2548a00640 1 -- 172.21.15.119:0/3592929924 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253c032440 con 0x7f254c15dbe0 2024-04-25T10:13:35.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.906+0000 7f2548a00640 1 -- 172.21.15.119:0/3592929924 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c032810 con 0x7f254c15dbe0 2024-04-25T10:13:35.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.907+0000 7f2551200640 1 -- 172.21.15.119:0/3592929924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c15dbe0 msgr2=0x7f254c071c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.907+0000 7f2551200640 1 --2- 172.21.15.119:0/3592929924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c15dbe0 0x7f254c071c10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f253c0098a0 tx=0x7f253c02a290 comp rx=0 tx=0).stop 2024-04-25T10:13:35.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.907+0000 7f2551200640 1 -- 172.21.15.119:0/3592929924 shutdown_connections 2024-04-25T10:13:35.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.907+0000 7f2551200640 1 --2- 172.21.15.119:0/3592929924 >> v2:172.21.15.132:3300/0 conn(0x7f254c0cfad0 0x7f254c06d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.907+0000 7f2551200640 1 --2- 172.21.15.119:0/3592929924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c15dbe0 0x7f254c071c10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.907+0000 7f2551200640 1 -- 172.21.15.119:0/3592929924 >> 172.21.15.119:0/3592929924 conn(0x7f254c06aba0 msgr2=0x7f254c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:35.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.907+0000 7f2551200640 1 -- 172.21.15.119:0/3592929924 shutdown_connections 2024-04-25T10:13:35.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.907+0000 7f2551200640 1 -- 172.21.15.119:0/3592929924 wait complete. 2024-04-25T10:13:35.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.909+0000 7f2551200640 1 Processor -- start 2024-04-25T10:13:35.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.909+0000 7f2551200640 1 -- start start 2024-04-25T10:13:35.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.910+0000 7f2551200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c0aedc0 0x7f254c153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.910+0000 7f2551200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c0cfad0 0x7f254c0ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.910+0000 7f2551200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f254c15dbe0 0x7f254c0aba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.910+0000 7f2551200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0adc90 con 0x7f254c0cfad0 2024-04-25T10:13:35.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.910+0000 7f2551200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0ad9b0 con 0x7f254c15dbe0 2024-04-25T10:13:35.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.910+0000 7f2551200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c15dfc0 con 0x7f254c0aedc0 2024-04-25T10:13:35.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.910+0000 7f2549400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c0cfad0 0x7f254c0ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.910+0000 7f2549400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c0cfad0 0x7f254c0ab4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43192/0 (socket says 172.21.15.119:43192) 2024-04-25T10:13:35.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.910+0000 7f2549400640 1 -- 172.21.15.119:0/1067992466 learned_addr learned my addr 172.21.15.119:0/1067992466 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:35.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.911+0000 7f2549e00640 1 --2- 172.21.15.119:0/1067992466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c0aedc0 0x7f254c153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.911+0000 7f254a800640 1 --2- 172.21.15.119:0/1067992466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f254c15dbe0 0x7f254c0aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.911+0000 7f2549400640 1 -- 172.21.15.119:0/1067992466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c0aedc0 msgr2=0x7f254c153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.911+0000 7f2549400640 1 --2- 172.21.15.119:0/1067992466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c0aedc0 0x7f254c153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.911+0000 7f2549400640 1 -- 172.21.15.119:0/1067992466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f254c15dbe0 msgr2=0x7f254c0aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.911+0000 7f2549400640 1 --2- 172.21.15.119:0/1067992466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f254c15dbe0 0x7f254c0aba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.911+0000 7f2549400640 1 -- 172.21.15.119:0/1067992466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c0b31b0 con 0x7f254c0cfad0 2024-04-25T10:13:35.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.911+0000 7f2549400640 1 --2- 172.21.15.119:0/1067992466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c0cfad0 0x7f254c0ab4c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f253800c7a0 tx=0x7f253800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.911+0000 7f2542000640 1 -- 172.21.15.119:0/1067992466 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538003b70 con 0x7f254c0cfad0 2024-04-25T10:13:35.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.911+0000 7f2542000640 1 -- 172.21.15.119:0/1067992466 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2538003d30 con 0x7f254c0cfad0 2024-04-25T10:13:35.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.911+0000 7f2542000640 1 -- 172.21.15.119:0/1067992466 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253800aa30 con 0x7f254c0cfad0 2024-04-25T10:13:35.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.911+0000 7f2551200640 1 -- 172.21.15.119:0/1067992466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c0ac030 con 0x7f254c0cfad0 2024-04-25T10:13:35.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.911+0000 7f2551200640 1 -- 172.21.15.119:0/1067992466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c0a8620 con 0x7f254c0cfad0 2024-04-25T10:13:35.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.912+0000 7f2551200640 1 -- 172.21.15.119:0/1067992466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254c0750b0 con 0x7f254c0cfad0 2024-04-25T10:13:35.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.914+0000 7f2542000640 1 -- 172.21.15.119:0/1067992466 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f253800abd0 con 0x7f254c0cfad0 2024-04-25T10:13:35.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.915+0000 7f2542000640 1 --2- 172.21.15.119:0/1067992466 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f252c06b500 0x7f252c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.915+0000 7f2542000640 1 -- 172.21.15.119:0/1067992466 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f253808a040 con 0x7f254c0cfad0 2024-04-25T10:13:35.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.915+0000 7f2549e00640 1 --2- 172.21.15.119:0/1067992466 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f252c06b500 0x7f252c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.916+0000 7f2549e00640 1 --2- 172.21.15.119:0/1067992466 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f252c06b500 0x7f252c06d9c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f253c034000 tx=0x7f253c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:35.920+0000 7f2542000640 1 -- 172.21.15.119:0/1067992466 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2538058f50 con 0x7f254c0cfad0 2024-04-25T10:13:36.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.158+0000 7f2551200640 1 -- 172.21.15.119:0/1067992466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f254c06edc0 con 0x7f254c0cfad0 2024-04-25T10:13:36.164 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:13:36.159+0000 7f8c64600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:13:36.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.163+0000 7f2542000640 1 -- 172.21.15.119:0/1067992466 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2538058850 con 0x7f254c0cfad0 2024-04-25T10:13:36.168 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:13:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.167+0000 7f2551200640 1 -- 172.21.15.119:0/1067992466 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f252c06b500 msgr2=0x7f252c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.167+0000 7f2551200640 1 --2- 172.21.15.119:0/1067992466 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f252c06b500 0x7f252c06d9c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f253c034000 tx=0x7f253c02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.167+0000 7f2551200640 1 -- 172.21.15.119:0/1067992466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c0cfad0 msgr2=0x7f254c0ab4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.167+0000 7f2551200640 1 --2- 172.21.15.119:0/1067992466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c0cfad0 0x7f254c0ab4c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f253800c7a0 tx=0x7f253800cc60 comp rx=0 tx=0).stop 2024-04-25T10:13:36.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.167+0000 7f2551200640 1 -- 172.21.15.119:0/1067992466 shutdown_connections 2024-04-25T10:13:36.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.167+0000 7f2551200640 1 --2- 172.21.15.119:0/1067992466 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f252c06b500 0x7f252c06d9c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.167+0000 7f2551200640 1 --2- 172.21.15.119:0/1067992466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f254c15dbe0 0x7f254c0aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.167+0000 7f2551200640 1 --2- 172.21.15.119:0/1067992466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c0cfad0 0x7f254c0ab4c0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.167+0000 7f2551200640 1 --2- 172.21.15.119:0/1067992466 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c0aedc0 0x7f254c153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.167+0000 7f2551200640 1 -- 172.21.15.119:0/1067992466 >> 172.21.15.119:0/1067992466 conn(0x7f254c06aba0 msgr2=0x7f254c158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.167+0000 7f2551200640 1 -- 172.21.15.119:0/1067992466 shutdown_connections 2024-04-25T10:13:36.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.167+0000 7f2551200640 1 -- 172.21.15.119:0/1067992466 wait complete. 2024-04-25T10:13:36.186 INFO:tasks.ceph.mds.a.smithi119.stderr: -4805> 2024-04-25T10:13:29.933+0000 7f8284c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:36.190 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-25T10:13:36.255 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:13:36.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.281+0000 7f11d2178f40 1 Processor -- start 2024-04-25T10:13:36.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.281+0000 7f11d2178f40 1 -- start start 2024-04-25T10:13:36.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.282+0000 7f11d2178f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562eade6c220 0x562eadf408c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.282+0000 7f11d2178f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562eadeb7080 0x562eadf371a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.282+0000 7f11d2178f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eadf43c70 0x562eadeb9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.282+0000 7f11d2178f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eade6c6c0 con 0x562eadf43c70 2024-04-25T10:13:36.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.282+0000 7f11d2178f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eade6c960 con 0x562eadeb7080 2024-04-25T10:13:36.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.282+0000 7f11d2178f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x562eade9e1c0 con 0x562eade6c220 2024-04-25T10:13:36.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.282+0000 7f11cfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eadf43c70 0x562eadeb9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.283+0000 7f11cfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eadf43c70 0x562eadeb9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43218/0 (socket says 172.21.15.119:43218) 2024-04-25T10:13:36.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.283+0000 7f11cfe00640 1 -- 172.21.15.119:0/1357564246 learned_addr learned my addr 172.21.15.119:0/1357564246 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:36.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.283+0000 7f11cfe00640 1 -- 172.21.15.119:0/1357564246 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562eade6c220 msgr2=0x562eadf408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.283+0000 7f11cfe00640 1 --2- 172.21.15.119:0/1357564246 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562eade6c220 0x562eadf408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.283+0000 7f11cfe00640 1 -- 172.21.15.119:0/1357564246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562eadeb7080 msgr2=0x562eadf371a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.283+0000 7f11cfe00640 1 --2- 172.21.15.119:0/1357564246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562eadeb7080 0x562eadf371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.283+0000 7f11cfe00640 1 -- 172.21.15.119:0/1357564246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eaddd93b0 con 0x562eadf43c70 2024-04-25T10:13:36.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.283+0000 7f11cfe00640 1 --2- 172.21.15.119:0/1357564246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eadf43c70 0x562eadeb9310 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f11c800b5b0 tx=0x7f11c800ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5ff92c66412047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.283+0000 7f11cd600640 1 -- 172.21.15.119:0/1357564246 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8014030 con 0x562eadf43c70 2024-04-25T10:13:36.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.283+0000 7f11cd600640 1 -- 172.21.15.119:0/1357564246 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c8010600 con 0x562eadf43c70 2024-04-25T10:13:36.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.283+0000 7f11cd600640 1 -- 172.21.15.119:0/1357564246 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c80109f0 con 0x562eadf43c70 2024-04-25T10:13:36.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.284+0000 7f11d2178f40 1 -- 172.21.15.119:0/1357564246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eadf43c70 msgr2=0x562eadeb9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.284+0000 7f11d2178f40 1 --2- 172.21.15.119:0/1357564246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eadf43c70 0x562eadeb9310 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f11c800b5b0 tx=0x7f11c800ba70 comp rx=0 tx=0).stop 2024-04-25T10:13:36.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.284+0000 7f11d2178f40 1 -- 172.21.15.119:0/1357564246 shutdown_connections 2024-04-25T10:13:36.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.284+0000 7f11d2178f40 1 --2- 172.21.15.119:0/1357564246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eadf43c70 0x562eadeb9310 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.284+0000 7f11d2178f40 1 --2- 172.21.15.119:0/1357564246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562eadeb7080 0x562eadf371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.284+0000 7f11d2178f40 1 --2- 172.21.15.119:0/1357564246 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562eade6c220 0x562eadf408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.284+0000 7f11d2178f40 1 -- 172.21.15.119:0/1357564246 >> 172.21.15.119:0/1357564246 conn(0x562eadeba070 msgr2=0x562eadeba450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.284+0000 7f11d2178f40 1 -- 172.21.15.119:0/1357564246 shutdown_connections 2024-04-25T10:13:36.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.284+0000 7f11d2178f40 1 -- 172.21.15.119:0/1357564246 wait complete. 2024-04-25T10:13:36.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.285+0000 7f11d2178f40 1 Processor -- start 2024-04-25T10:13:36.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.285+0000 7f11d2178f40 1 -- start start 2024-04-25T10:13:36.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.286+0000 7f11d2178f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562eade6c220 0x562eadf32660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.286+0000 7f11d2178f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562eadeb7080 0x562eadfd20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.286+0000 7f11d2178f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eadf43c70 0x562eadeb64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.286+0000 7f11d2178f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eadf40e00 con 0x562eadf43c70 2024-04-25T10:13:36.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.286+0000 7f11d2178f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eadf46780 con 0x562eade6c220 2024-04-25T10:13:36.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.286+0000 7f11d2178f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x562eade6b8f0 con 0x562eadeb7080 2024-04-25T10:13:36.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.286+0000 7f11cfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eadf43c70 0x562eadeb64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.286+0000 7f11cfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eadf43c70 0x562eadeb64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43230/0 (socket says 172.21.15.119:43230) 2024-04-25T10:13:36.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.286+0000 7f11cfe00640 1 -- 172.21.15.119:0/402929801 learned_addr learned my addr 172.21.15.119:0/402929801 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:36.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.287+0000 7f11cfe00640 1 -- 172.21.15.119:0/402929801 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562eadeb7080 msgr2=0x562eadfd20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.287+0000 7f11cfe00640 1 --2- 172.21.15.119:0/402929801 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562eadeb7080 0x562eadfd20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.287+0000 7f11cfe00640 1 -- 172.21.15.119:0/402929801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562eade6c220 msgr2=0x562eadf32660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.287+0000 7f11cfe00640 1 --2- 172.21.15.119:0/402929801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562eade6c220 0x562eadf32660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.287+0000 7f11cfe00640 1 -- 172.21.15.119:0/402929801 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eadfcf4a0 con 0x562eadf43c70 2024-04-25T10:13:36.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.287+0000 7f11cfe00640 1 --2- 172.21.15.119:0/402929801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eadf43c70 0x562eadeb64c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f11c8007010 tx=0x7f11c80033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.287+0000 7f11c7e00640 1 -- 172.21.15.119:0/402929801 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8021070 con 0x562eadf43c70 2024-04-25T10:13:36.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.287+0000 7f11c7e00640 1 -- 172.21.15.119:0/402929801 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c801abd0 con 0x562eadf43c70 2024-04-25T10:13:36.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.287+0000 7f11d2178f40 1 -- 172.21.15.119:0/402929801 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562eadf32ee0 con 0x562eadf43c70 2024-04-25T10:13:36.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.287+0000 7f11c7e00640 1 -- 172.21.15.119:0/402929801 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8005870 con 0x562eadf43c70 2024-04-25T10:13:36.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.287+0000 7f11d2178f40 1 -- 172.21.15.119:0/402929801 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562eadf43460 con 0x562eadf43c70 2024-04-25T10:13:36.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.289+0000 7f11c7e00640 1 -- 172.21.15.119:0/402929801 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11c8024070 con 0x562eadf43c70 2024-04-25T10:13:36.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.290+0000 7f11c7e00640 1 --2- 172.21.15.119:0/402929801 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f11b0069ee0 0x7f11b006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.290+0000 7f11c7e00640 1 -- 172.21.15.119:0/402929801 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f11c808ab30 con 0x562eadf43c70 2024-04-25T10:13:36.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.290+0000 7f11cf400640 1 --2- 172.21.15.119:0/402929801 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f11b0069ee0 0x7f11b006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.291+0000 7f11d2178f40 1 --2- 172.21.15.119:0/402929801 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x562eadeb6a00 0x562eadf35900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.291+0000 7f11d2178f40 1 -- 172.21.15.119:0/402929801 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e82) v9 -- 0x562eadeb9680 con 0x562eadeb6a00 2024-04-25T10:13:36.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.291+0000 7f11cea00640 1 --2- 172.21.15.119:0/402929801 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x562eadeb6a00 0x562eadf35900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.291+0000 7f11cf400640 1 --2- 172.21.15.119:0/402929801 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f11b0069ee0 0x7f11b006c3a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x562eadf40050 tx=0x7f11c0005580 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.291+0000 7f11cea00640 1 --2- 172.21.15.119:0/402929801 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x562eadeb6a00 0x562eadf35900 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.295+0000 7f11cea00640 1 -- 172.21.15.119:0/402929801 <== osd.0 v2:172.21.15.119:6808/2890625333 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v82'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f11bc0534f0 con 0x562eadeb6a00 2024-04-25T10:13:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.295+0000 7f11d2178f40 1 -- 172.21.15.119:0/402929801 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x562eadeb6a00 msgr2=0x562eadf35900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.295+0000 7f11d2178f40 1 --2- 172.21.15.119:0/402929801 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x562eadeb6a00 0x562eadf35900 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.295+0000 7f11d2178f40 1 -- 172.21.15.119:0/402929801 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f11b0069ee0 msgr2=0x7f11b006c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.295+0000 7f11d2178f40 1 --2- 172.21.15.119:0/402929801 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f11b0069ee0 0x7f11b006c3a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x562eadf40050 tx=0x7f11c0005580 comp rx=0 tx=0).stop 2024-04-25T10:13:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.295+0000 7f11d2178f40 1 -- 172.21.15.119:0/402929801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eadf43c70 msgr2=0x562eadeb64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.295+0000 7f11d2178f40 1 --2- 172.21.15.119:0/402929801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562eadf43c70 0x562eadeb64c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f11c8007010 tx=0x7f11c80033a0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.295+0000 7f11cfe00640 1 -- 172.21.15.119:0/402929801 reap_dead start 2024-04-25T10:13:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.295+0000 7f11d2178f40 1 -- 172.21.15.119:0/402929801 shutdown_connections 2024-04-25T10:13:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.295+0000 7f11d2178f40 1 -- 172.21.15.119:0/402929801 >> 172.21.15.119:0/402929801 conn(0x562eadeba070 msgr2=0x562eadeba450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.296+0000 7f11d2178f40 1 -- 172.21.15.119:0/402929801 shutdown_connections 2024-04-25T10:13:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.296+0000 7f11d2178f40 1 -- 172.21.15.119:0/402929801 wait complete. 2024-04-25T10:13:36.305 DEBUG:teuthology.orchestra.run.smithi119:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-25T10:13:36.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.383+0000 7f63f5c0f280 1 -- start start 2024-04-25T10:13:36.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.383+0000 7f63f5c0f280 1 Processor -- start 2024-04-25T10:13:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f5c0f280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856b6a000 0x564855e11080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f5c0f280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564855da5c00 0x564855e11b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f5c0f280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856b6a800 0x564855e11600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f5c0f280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564856b7c480 con 0x564855da5c00 2024-04-25T10:13:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f5c0f280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564856b7c600 con 0x564856b6a000 2024-04-25T10:13:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f5c0f280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564856b7c780 con 0x564856b6a800 2024-04-25T10:13:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f2600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564855da5c00 0x564855e11b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f2600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564855da5c00 0x564855e11b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43246/0 (socket says 172.21.15.119:43246) 2024-04-25T10:13:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f2600640 1 -- 172.21.15.119:0/934387211 learned_addr learned my addr 172.21.15.119:0/934387211 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f3a00640 1 --2- 172.21.15.119:0/934387211 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856b6a800 0x564855e11600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f2600640 1 -- 172.21.15.119:0/934387211 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856b6a800 msgr2=0x564855e11600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f2600640 1 --2- 172.21.15.119:0/934387211 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856b6a800 0x564855e11600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f3000640 1 --2- 172.21.15.119:0/934387211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856b6a000 0x564855e11080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f2600640 1 -- 172.21.15.119:0/934387211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856b6a000 msgr2=0x564855e11080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f2600640 1 --2- 172.21.15.119:0/934387211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856b6a000 0x564855e11080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.384+0000 7f63f2600640 1 -- 172.21.15.119:0/934387211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564855dd8780 con 0x564855da5c00 2024-04-25T10:13:36.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.385+0000 7f63f2600640 1 --2- 172.21.15.119:0/934387211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564855da5c00 0x564855e11b80 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x564856bc54a0 tx=0x564856b27f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337aa1f8a255b36f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.385+0000 7f63efe00640 1 -- 172.21.15.119:0/934387211 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564855dcad00 con 0x564855da5c00 2024-04-25T10:13:36.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.385+0000 7f63efe00640 1 -- 172.21.15.119:0/934387211 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564855dd0fc0 con 0x564855da5c00 2024-04-25T10:13:36.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.385+0000 7f63efe00640 1 -- 172.21.15.119:0/934387211 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564855dca680 con 0x564855da5c00 2024-04-25T10:13:36.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.385+0000 7f63f5c0f280 1 -- 172.21.15.119:0/934387211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564855dd8000 con 0x564855da5c00 2024-04-25T10:13:36.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.385+0000 7f63f5c0f280 1 -- 172.21.15.119:0/934387211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x564855dd9680 con 0x564855da5c00 2024-04-25T10:13:36.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.385+0000 7f63f5c0f280 4 waiting for MDS map... 2024-04-25T10:13:36.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.385+0000 7f63efe00640 1 -- 172.21.15.119:0/934387211 <== mon.0 v2:172.21.15.119:3300/0 4 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x564856c76600 con 0x564855da5c00 2024-04-25T10:13:36.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.385+0000 7f63efe00640 1 -- 172.21.15.119:0/934387211 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 62) v1 ==== 2723+0+0 (secure 0 0 0) 0x564855dff180 con 0x564855da5c00 2024-04-25T10:13:36.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.386+0000 7f63f5c0f280 4 Got MDS map 62 2024-04-25T10:13:36.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.386+0000 7f63f5c0f280 10 main: JournalTool::main 2024-04-25T10:13:36.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.386+0000 7f63f5c0f280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:13:36.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.386+0000 7f63f5c0f280 1 Processor -- start 2024-04-25T10:13:36.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.387+0000 7f63f5c0f280 1 -- start start 2024-04-25T10:13:36.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.387+0000 7f63f5c0f280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856c74c00 0x564855e10b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.387+0000 7f63f5c0f280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856c75000 0x564855e12100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.387+0000 7f63f5c0f280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856c75400 0x564855e12680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.387+0000 7f63f5c0f280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564855e06180 con 0x564856c75000 2024-04-25T10:13:36.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.387+0000 7f63f5c0f280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564856b7cf00 con 0x564856c74c00 2024-04-25T10:13:36.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.387+0000 7f63f5c0f280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564856b7d080 con 0x564856c75400 2024-04-25T10:13:36.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.387+0000 7f63f3000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856c75000 0x564855e12100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.388+0000 7f63f3000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856c75000 0x564855e12100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43256/0 (socket says 172.21.15.119:43256) 2024-04-25T10:13:36.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.388+0000 7f63f3000640 1 -- 172.21.15.119:0/2767019076 learned_addr learned my addr 172.21.15.119:0/2767019076 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.388+0000 7f63f3a00640 1 --2- 172.21.15.119:0/2767019076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856c74c00 0x564855e10b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.388+0000 7f63f3000640 1 -- 172.21.15.119:0/2767019076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856c75400 msgr2=0x564855e12680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.388+0000 7f63f3000640 1 --2- 172.21.15.119:0/2767019076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856c75400 0x564855e12680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.388+0000 7f63f3000640 1 -- 172.21.15.119:0/2767019076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856c74c00 msgr2=0x564855e10b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.388+0000 7f63f3000640 1 --2- 172.21.15.119:0/2767019076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856c74c00 0x564855e10b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.388+0000 7f63f3000640 1 -- 172.21.15.119:0/2767019076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564855dd9860 con 0x564856c75000 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.388+0000 7f63f3000640 1 --2- 172.21.15.119:0/2767019076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856c75000 0x564855e12100 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x564856bc4d80 tx=0x564856b359e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc30de5e07f781c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.388+0000 7f63ed600640 1 -- 172.21.15.119:0/2767019076 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564855dcad00 con 0x564856c75000 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.388+0000 7f63ed600640 1 -- 172.21.15.119:0/2767019076 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564855dd0700 con 0x564856c75000 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.388+0000 7f63ed600640 1 -- 172.21.15.119:0/2767019076 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564855dca680 con 0x564856c75000 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.389+0000 7f63f5c0f280 1 -- 172.21.15.119:0/2767019076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856c75000 msgr2=0x564855e12100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.389+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/2767019076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856c75000 0x564855e12100 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x564856bc4d80 tx=0x564856b359e0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.389+0000 7f63f5c0f280 1 -- 172.21.15.119:0/2767019076 shutdown_connections 2024-04-25T10:13:36.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.389+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/2767019076 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856c75400 0x564855e12680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.389+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/2767019076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856c75000 0x564855e12100 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.389+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/2767019076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856c74c00 0x564855e10b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.389+0000 7f63f5c0f280 1 -- 172.21.15.119:0/2767019076 >> 172.21.15.119:0/2767019076 conn(0x564856b6a400 msgr2=0x564855eab800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.389+0000 7f63f5c0f280 1 -- 172.21.15.119:0/2767019076 shutdown_connections 2024-04-25T10:13:36.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.389+0000 7f63f5c0f280 1 -- 172.21.15.119:0/2767019076 wait complete. 2024-04-25T10:13:36.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.390+0000 7f63f5c0f280 1 Processor -- start 2024-04-25T10:13:36.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.390+0000 7f63f5c0f280 1 -- start start 2024-04-25T10:13:36.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f5c0f280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856c74c00 0x564855e10b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f5c0f280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856c75000 0x564855e12100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f5c0f280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856c75400 0x564855e12680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f5c0f280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564856b7d380 con 0x564856c75400 2024-04-25T10:13:36.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f5c0f280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564856b7d200 con 0x564856c75000 2024-04-25T10:13:36.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f5c0f280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564856b7d500 con 0x564856c74c00 2024-04-25T10:13:36.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f2600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856c75400 0x564855e12680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f2600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856c75400 0x564855e12680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43272/0 (socket says 172.21.15.119:43272) 2024-04-25T10:13:36.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f2600640 1 -- 172.21.15.119:0/2472925946 learned_addr learned my addr 172.21.15.119:0/2472925946 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:36.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f2600640 1 -- 172.21.15.119:0/2472925946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856c74c00 msgr2=0x564855e10b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f2600640 1 --2- 172.21.15.119:0/2472925946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856c74c00 0x564855e10b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f2600640 1 -- 172.21.15.119:0/2472925946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856c75000 msgr2=0x564855e12100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f3000640 1 --2- 172.21.15.119:0/2472925946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856c75000 0x564855e12100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f2600640 1 --2- 172.21.15.119:0/2472925946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856c75000 0x564855e12100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.391+0000 7f63f2600640 1 -- 172.21.15.119:0/2472925946 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564855dd94a0 con 0x564856c75400 2024-04-25T10:13:36.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.392+0000 7f63f2600640 1 --2- 172.21.15.119:0/2472925946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856c75400 0x564855e12680 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x564856c876b0 tx=0x564856ba0360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.392+0000 7f63eb800640 1 -- 172.21.15.119:0/2472925946 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564855dcab60 con 0x564856c75400 2024-04-25T10:13:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.392+0000 7f63eb800640 1 -- 172.21.15.119:0/2472925946 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564855dd0700 con 0x564856c75400 2024-04-25T10:13:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.392+0000 7f63f5c0f280 1 -- 172.21.15.119:0/2472925946 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564855dd8960 con 0x564856c75400 2024-04-25T10:13:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.392+0000 7f63eb800640 1 -- 172.21.15.119:0/2472925946 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564855dcad00 con 0x564856c75400 2024-04-25T10:13:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.392+0000 7f63f5c0f280 1 -- 172.21.15.119:0/2472925946 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564855dd83c0 con 0x564856c75400 2024-04-25T10:13:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.392+0000 7f63f5c0f280 4 main: JournalTool: resolving pool 12 2024-04-25T10:13:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.394+0000 7f63eb800640 1 -- 172.21.15.119:0/2472925946 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x564856afec00 con 0x564856c75400 2024-04-25T10:13:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.394+0000 7f63eb800640 1 --2- 172.21.15.119:0/2472925946 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x564856b02c00 0x564855e13700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.394+0000 7f63eb800640 1 -- 172.21.15.119:0/2472925946 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x564856cdec00 con 0x564856c75400 2024-04-25T10:13:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.394+0000 7f63f5c0f280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:13:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.394+0000 7f63f5c0f280 4 main: Executing for rank 0 2024-04-25T10:13:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.395+0000 7f63f3a00640 1 --2- 172.21.15.119:0/2472925946 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x564856b02c00 0x564855e13700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.395+0000 7f63f3a00640 1 --2- 172.21.15.119:0/2472925946 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x564856b02c00 0x564855e13700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x564855e9a030 tx=0x56485610cc00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.395+0000 7f63f5c0f280 1 -- start start 2024-04-25T10:13:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.395+0000 7f63f5c0f280 1 Processor -- start 2024-04-25T10:13:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.396+0000 7f63f5c0f280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856c75c00 0x564855e12c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.396+0000 7f63f5c0f280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856b03000 0x564855e13180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.396+0000 7f63f5c0f280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856b03400 0x564856164000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.396+0000 7f63f5c0f280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564856b7d800 con 0x564856b03000 2024-04-25T10:13:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.396+0000 7f63f5c0f280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564856b7d980 con 0x564856c75c00 2024-04-25T10:13:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.396+0000 7f63f5c0f280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x564856b7db00 con 0x564856b03400 2024-04-25T10:13:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.397+0000 7f63f3000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856b03000 0x564855e13180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.397+0000 7f63f3000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856b03000 0x564855e13180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43280/0 (socket says 172.21.15.119:43280) 2024-04-25T10:13:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.397+0000 7f63f3000640 1 -- 172.21.15.119:0/1938211554 learned_addr learned my addr 172.21.15.119:0/1938211554 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.397+0000 7f63f3a00640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856c75c00 0x564855e12c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.397+0000 7f63f2600640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856b03400 0x564856164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.397+0000 7f63f3000640 1 -- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856b03400 msgr2=0x564856164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.397+0000 7f63f3000640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856b03400 0x564856164000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.397+0000 7f63f3000640 1 -- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856c75c00 msgr2=0x564855e12c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.397+0000 7f63f3000640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856c75c00 0x564855e12c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.397+0000 7f63f3000640 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564855dd9860 con 0x564856b03000 2024-04-25T10:13:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.397+0000 7f63f3000640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856b03000 0x564855e13180 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x564856c86960 tx=0x564856b27b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e7b137f8ac5024 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.397+0000 7f63e7c00640 1 -- 172.21.15.119:0/1938211554 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564855dca820 con 0x564856b03000 2024-04-25T10:13:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.397+0000 7f63e7c00640 1 -- 172.21.15.119:0/1938211554 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564855dd0c40 con 0x564856b03000 2024-04-25T10:13:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.398+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564855dd9680 con 0x564856b03000 2024-04-25T10:13:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.398+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x564855dd94a0 con 0x564856b03000 2024-04-25T10:13:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.398+0000 7f63f5c0f280 4 waiting for MDS map... 2024-04-25T10:13:36.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.398+0000 7f63e7c00640 1 -- 172.21.15.119:0/1938211554 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5648560909c0 con 0x564856b03000 2024-04-25T10:13:36.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.398+0000 7f63e7c00640 1 -- 172.21.15.119:0/1938211554 <== mon.0 v2:172.21.15.119:3300/0 4 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x56485616c000 con 0x564856b03000 2024-04-25T10:13:36.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.398+0000 7f63e7c00640 1 -- 172.21.15.119:0/1938211554 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 62) v1 ==== 2723+0+0 (secure 0 0 0) 0x56485617a000 con 0x564856b03000 2024-04-25T10:13:36.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.398+0000 7f63f5c0f280 4 Got MDS map 62 2024-04-25T10:13:36.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.398+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x564856b03800 0x564856164580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.398+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v9 -- 0x564856b03c00 con 0x564856b03800 2024-04-25T10:13:36.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.399+0000 7f63f3a00640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x564856b03800 0x564856164580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.400+0000 7f63f3a00640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x564856b03800 0x564856164580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=f053cc0cd9a76b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.401+0000 7f63f3a00640 1 -- 172.21.15.119:0/1938211554 <== osd.6 v2:172.21.15.132:6806/353653608 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x564856b22000 con 0x564856b03800 2024-04-25T10:13:36.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.401+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x564856174400 0x564856164b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.401+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82) v9 -- 0x564856174800 con 0x564856174400 2024-04-25T10:13:36.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.402+0000 7f63f2600640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x564856174400 0x564856164b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.403+0000 7f63f2600640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x564856174400 0x564856164b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=c6db971159caba61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.404+0000 7f63f2600640 1 -- 172.21.15.119:0/1938211554 <== osd.5 v2:172.21.15.132:6815/2978152964 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x564856b22500 con 0x564856174400 2024-04-25T10:13:36.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.404+0000 7f63e5e00640 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82) v9 -- 0x564856192000 con 0x564856174400 2024-04-25T10:13:36.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.404+0000 7f63e5e00640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x564856192400 0x564856165b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.404+0000 7f63e5e00640 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82) v9 -- 0x564856192800 con 0x564856192400 2024-04-25T10:13:36.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.405+0000 7f63f2600640 1 -- 172.21.15.119:0/1938211554 <== osd.5 v2:172.21.15.132:6815/2978152964 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x564856b22500 con 0x564856174400 2024-04-25T10:13:36.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.405+0000 7f63f3a00640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x564856192400 0x564856165b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.406+0000 7f63f3a00640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x564856192400 0x564856165b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=8af59f0edb67016b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.406+0000 7f63f3a00640 1 -- 172.21.15.119:0/1938211554 <== osd.4 v2:172.21.15.132:6800/1740646323 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x564856b22000 con 0x564856192400 2024-04-25T10:13:36.411 INFO:teuthology.orchestra.run.smithi119.stdout:old journal was 4212750~2030 2024-04-25T10:13:36.411 INFO:teuthology.orchestra.run.smithi119.stdout:new journal start will be 8388608 (4173828 bytes past old end) 2024-04-25T10:13:36.411 INFO:teuthology.orchestra.run.smithi119.stdout:writing journal head 2024-04-25T10:13:36.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.407+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:5 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v9 -- 0x564856174c00 con 0x564856174400 2024-04-25T10:13:36.415 INFO:teuthology.orchestra.run.smithi119.stdout:writing EResetJournal entry 2024-04-25T10:13:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.410+0000 7f63f2600640 1 -- 172.21.15.119:0/1938211554 <== osd.5 v2:172.21.15.132:6815/2978152964 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v82'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x564856b22500 con 0x564856174400 2024-04-25T10:13:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.410+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:6 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v9 -- 0x564856175000 con 0x564856192400 2024-04-25T10:13:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.410+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x564856175400 0x564856165080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.410+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:7 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v9 -- 0x564856175800 con 0x564856175400 2024-04-25T10:13:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.410+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:8 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v9 -- 0x564856175c00 con 0x564856b03800 2024-04-25T10:13:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.410+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:9 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v9 -- 0x564856192c00 con 0x564856192400 2024-04-25T10:13:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.410+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:10 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v9 -- 0x564856193000 con 0x564856192400 2024-04-25T10:13:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.410+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x564856193400 0x564856165600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.410+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:11 12.1d 12:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v9 -- 0x564856193800 con 0x564856193400 2024-04-25T10:13:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.410+0000 7f63f3000640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x564856175400 0x564856165080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.410+0000 7f63f2600640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x564856193400 0x564856165600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.411+0000 7f63f2600640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x564856193400 0x564856165600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=61ab589569dd867f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.411+0000 7f63f3000640 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x564856175400 0x564856165080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=6438ce6d8a0f6333 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.413+0000 7f63f2600640 1 -- 172.21.15.119:0/1938211554 <== osd.0 v2:172.21.15.119:6808/2890625333 1 ==== osd_op_reply(11 200.00000007 [delete] v82'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x564856b22500 con 0x564856193400 2024-04-25T10:13:36.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.413+0000 7f63f3a00640 1 -- 172.21.15.119:0/1938211554 <== osd.4 v2:172.21.15.132:6800/1740646323 2 ==== osd_op_reply(9 200.00000005 [delete] v82'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x564856b22000 con 0x564856192400 2024-04-25T10:13:36.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.413+0000 7f63f3000640 1 -- 172.21.15.119:0/1938211554 <== osd.3 v2:172.21.15.119:6816/1142971786 1 ==== osd_op_reply(7 200.00000003 [delete] v82'4 uv3 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x564856b22780 con 0x564856175400 2024-04-25T10:13:36.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.413+0000 7f63f3a00640 1 -- 172.21.15.119:0/1938211554 <== osd.4 v2:172.21.15.132:6800/1740646323 3 ==== osd_op_reply(6 200.00000002 [delete] v82'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x564856b22000 con 0x564856192400 2024-04-25T10:13:36.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.413+0000 7f63e5e00640 1 -- 172.21.15.119:0/1938211554 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:12 12.1b 12:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82) v9 -- 0x564856175000 con 0x564856192400 2024-04-25T10:13:36.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.413+0000 7f63f3a00640 1 -- 172.21.15.119:0/1938211554 <== osd.6 v2:172.21.15.132:6806/353653608 2 ==== osd_op_reply(8 200.00000004 [delete] v82'7 uv6 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x564856b22000 con 0x564856b03800 2024-04-25T10:13:36.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.414+0000 7f63f3a00640 1 -- 172.21.15.119:0/1938211554 <== osd.4 v2:172.21.15.132:6800/1740646323 4 ==== osd_op_reply(10 200.00000006 [delete] v82'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x564856b22000 con 0x564856192400 2024-04-25T10:13:36.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f3a00640 1 -- 172.21.15.119:0/1938211554 <== osd.4 v2:172.21.15.132:6800/1740646323 5 ==== osd_op_reply(12 200.00000002 [write 0~48] v82'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x564856b22000 con 0x564856192400 2024-04-25T10:13:36.421 INFO:teuthology.orchestra.run.smithi119.stdout:done 2024-04-25T10:13:36.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x564856193400 msgr2=0x564856165600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x564856193400 0x564856165600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x564856175400 msgr2=0x564856165080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x564856175400 0x564856165080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x564856192400 msgr2=0x564856165b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x564856192400 0x564856165b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x564856174400 msgr2=0x564856164b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x564856174400 0x564856164b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x564856b03800 msgr2=0x564856164580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x564856b03800 0x564856164580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856b03000 msgr2=0x564855e13180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/1938211554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856b03000 0x564855e13180 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x564856c86960 tx=0x564856b27b60 comp rx=0 tx=0).stop 2024-04-25T10:13:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f3000640 1 -- 172.21.15.119:0/1938211554 reap_dead start 2024-04-25T10:13:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f3000640 1 -- 172.21.15.119:0/1938211554 reap_dead start 2024-04-25T10:13:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f3000640 1 -- 172.21.15.119:0/1938211554 reap_dead start 2024-04-25T10:13:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 shutdown_connections 2024-04-25T10:13:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.416+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 >> 172.21.15.119:0/1938211554 conn(0x564856c75800 msgr2=0x56485615a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.417+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 shutdown_connections 2024-04-25T10:13:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.417+0000 7f63f5c0f280 1 -- 172.21.15.119:0/1938211554 wait complete. 2024-04-25T10:13:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.417+0000 7f63f5c0f280 1 -- 172.21.15.119:0/2472925946 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x564856b02c00 msgr2=0x564855e13700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.417+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/2472925946 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x564856b02c00 0x564855e13700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x564855e9a030 tx=0x56485610cc00 comp rx=0 tx=0).stop 2024-04-25T10:13:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.417+0000 7f63f5c0f280 1 -- 172.21.15.119:0/2472925946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856c75400 msgr2=0x564855e12680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.417+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/2472925946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856c75400 0x564855e12680 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x564856c876b0 tx=0x564856ba0360 comp rx=0 tx=0).stop 2024-04-25T10:13:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.417+0000 7f63f5c0f280 1 -- 172.21.15.119:0/2472925946 shutdown_connections 2024-04-25T10:13:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.417+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/2472925946 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x564856b02c00 0x564855e13700 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.417+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/2472925946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564856c75400 0x564855e12680 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.417+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/2472925946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856c75000 0x564855e12100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.417+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/2472925946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856c74c00 0x564855e10b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.417+0000 7f63f5c0f280 1 -- 172.21.15.119:0/2472925946 >> 172.21.15.119:0/2472925946 conn(0x564856b6a400 msgr2=0x564855eab800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.417+0000 7f63f5c0f280 1 -- 172.21.15.119:0/2472925946 shutdown_connections 2024-04-25T10:13:36.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.417+0000 7f63f5c0f280 1 -- 172.21.15.119:0/2472925946 wait complete. 2024-04-25T10:13:36.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.418+0000 7f63f5c0f280 1 -- 172.21.15.119:0/934387211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564855da5c00 msgr2=0x564855e11b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.418+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/934387211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564855da5c00 0x564855e11b80 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x564856bc54a0 tx=0x564856b27f20 comp rx=0 tx=0).stop 2024-04-25T10:13:36.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.418+0000 7f63f5c0f280 1 -- 172.21.15.119:0/934387211 shutdown_connections 2024-04-25T10:13:36.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.418+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/934387211 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x564856b6a800 0x564855e11600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.418+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/934387211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564855da5c00 0x564855e11b80 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.418+0000 7f63f5c0f280 1 --2- 172.21.15.119:0/934387211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564856b6a000 0x564855e11080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.418+0000 7f63f5c0f280 1 -- 172.21.15.119:0/934387211 >> 172.21.15.119:0/934387211 conn(0x564855da5000 msgr2=0x564855eab000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.418+0000 7f63f5c0f280 1 -- 172.21.15.119:0/934387211 shutdown_connections 2024-04-25T10:13:36.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.418+0000 7f63f5c0f280 1 -- 172.21.15.119:0/934387211 wait complete. 2024-04-25T10:13:36.431 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.126181, result: old journal was 4212750~2030 new journal start will be 8388608 (4173828 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-25T10:13:36.431 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:13:36.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.614+0000 7f49c1800640 1 Processor -- start 2024-04-25T10:13:36.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.614+0000 7f49c1800640 1 -- start start 2024-04-25T10:13:36.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.615+0000 7f49c1800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49bc0b3e90 0x7f49bc0cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.615+0000 7f49c1800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49bc15f400 con 0x7f49bc1450e0 2024-04-25T10:13:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.615+0000 7f49c1800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49bc15f6e0 con 0x7f49bc06b2b0 2024-04-25T10:13:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.615+0000 7f49c1800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc1454c0 con 0x7f49bc0b3e90 2024-04-25T10:13:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.616+0000 7f49bbe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f49bc1450e0 0x7f49bc06aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37010/0 (socket says 172.21.15.119:37010) 2024-04-25T10:13:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.616+0000 7f49bbe00640 1 -- 172.21.15.119:0/3676654099 learned_addr learned my addr 172.21.15.119:0/3676654099 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.616+0000 7f49baa00640 1 -- 172.21.15.119:0/3676654099 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591261745 0 0) 0x7f49bc15f400 con 0x7f49bc1450e0 2024-04-25T10:13:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.616+0000 7f49c0e00640 1 --2- 172.21.15.119:0/3676654099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49bc0b3e90 0x7f49bc0cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.616+0000 7f49baa00640 1 -- 172.21.15.119:0/3676654099 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a0003520 con 0x7f49bc1450e0 2024-04-25T10:13:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.616+0000 7f49baa00640 1 -- 172.21.15.119:0/3676654099 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1401709659 0 0) 0x7f49a0003520 con 0x7f49bc1450e0 2024-04-25T10:13:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.616+0000 7f49baa00640 1 -- 172.21.15.119:0/3676654099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49bc0b3e90 msgr2=0x7f49bc0cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.616+0000 7f49baa00640 1 --2- 172.21.15.119:0/3676654099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49bc0b3e90 0x7f49bc0cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.616+0000 7f49baa00640 1 -- 172.21.15.119:0/3676654099 >> v1:172.21.15.132:6789/0 conn(0x7f49bc06b2b0 legacy=0x7f49bc0cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.616+0000 7f49baa00640 1 -- 172.21.15.119:0/3676654099 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc15f9c0 con 0x7f49bc1450e0 2024-04-25T10:13:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.616+0000 7f49baa00640 1 -- 172.21.15.119:0/3676654099 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f49ac001670 con 0x7f49bc1450e0 2024-04-25T10:13:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49baa00640 1 -- 172.21.15.119:0/3676654099 >> v1:172.21.15.119:6789/0 conn(0x7f49bc1450e0 legacy=0x7f49bc06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49baa00640 1 --2- 172.21.15.119:0/3676654099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49bc0b3e90 0x7f49a00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49baa00640 1 --2- 172.21.15.119:0/3676654099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49a0004010 0x7f49a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49baa00640 1 --2- 172.21.15.119:0/3676654099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a0004d90 0x7f49a0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49baa00640 1 -- 172.21.15.119:0/3676654099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a000bc70 con 0x7f49a0004d90 2024-04-25T10:13:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49baa00640 1 -- 172.21.15.119:0/3676654099 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a000be20 con 0x7f49a0004010 2024-04-25T10:13:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49baa00640 1 -- 172.21.15.119:0/3676654099 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a000c020 con 0x7f49bc0b3e90 2024-04-25T10:13:36.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49c0e00640 1 --2- 172.21.15.119:0/3676654099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a0004d90 0x7f49a0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49bbe00640 1 --2- 172.21.15.119:0/3676654099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49bc0b3e90 0x7f49a00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49bb400640 1 --2- 172.21.15.119:0/3676654099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49a0004010 0x7f49a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49c0e00640 1 -- 172.21.15.119:0/3676654099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49bc0b3e90 msgr2=0x7f49a00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49c0e00640 1 --2- 172.21.15.119:0/3676654099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49bc0b3e90 0x7f49a00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49c0e00640 1 -- 172.21.15.119:0/3676654099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49a0004010 msgr2=0x7f49a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49c0e00640 1 --2- 172.21.15.119:0/3676654099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49a0004010 0x7f49a0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49c0e00640 1 -- 172.21.15.119:0/3676654099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a0003520 con 0x7f49a0004d90 2024-04-25T10:13:36.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.617+0000 7f49bbe00640 1 --2- 172.21.15.119:0/3676654099 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49bc0b3e90 0x7f49a00039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:36.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.618+0000 7f49c0e00640 1 --2- 172.21.15.119:0/3676654099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a0004d90 0x7f49a0005210 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f49b4007330 tx=0x7f49a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93270763824a572 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.619+0000 7f49baa00640 1 -- 172.21.15.119:0/3676654099 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4033070 con 0x7f49a0004d90 2024-04-25T10:13:36.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.619+0000 7f49baa00640 1 -- 172.21.15.119:0/3676654099 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b402f650 con 0x7f49a0004d90 2024-04-25T10:13:36.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.619+0000 7f49baa00640 1 -- 172.21.15.119:0/3676654099 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4009be0 con 0x7f49a0004d90 2024-04-25T10:13:36.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.619+0000 7f49c1800640 1 -- 172.21.15.119:0/3676654099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a0004d90 msgr2=0x7f49a0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.619+0000 7f49c1800640 1 --2- 172.21.15.119:0/3676654099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a0004d90 0x7f49a0005210 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f49b4007330 tx=0x7f49a0005830 comp rx=0 tx=0).stop 2024-04-25T10:13:36.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.619+0000 7f49c0e00640 1 -- 172.21.15.119:0/3676654099 reap_dead start 2024-04-25T10:13:36.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.619+0000 7f49c1800640 1 -- 172.21.15.119:0/3676654099 shutdown_connections 2024-04-25T10:13:36.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.619+0000 7f49c1800640 1 -- 172.21.15.119:0/3676654099 >> 172.21.15.119:0/3676654099 conn(0x7f49bc066ff0 msgr2=0x7f49bc0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.619+0000 7f49c1800640 1 -- 172.21.15.119:0/3676654099 shutdown_connections 2024-04-25T10:13:36.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.619+0000 7f49c1800640 1 -- 172.21.15.119:0/3676654099 wait complete. 2024-04-25T10:13:36.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.621+0000 7f49c1800640 1 Processor -- start 2024-04-25T10:13:36.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.621+0000 7f49c1800640 1 -- start start 2024-04-25T10:13:36.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.622+0000 7f49c1800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a0004d90 0x7f49bc15b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.622+0000 7f49c1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49bc0a9950 0x7f49bc13e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.622+0000 7f49c1800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49bc13ed10 0x7f49bc1f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.622+0000 7f49c1800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0ccb60 con 0x7f49bc0a9950 2024-04-25T10:13:36.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.622+0000 7f49c1800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0cc9e0 con 0x7f49bc13ed10 2024-04-25T10:13:36.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.622+0000 7f49c1800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0ccce0 con 0x7f49a0004d90 2024-04-25T10:13:36.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.622+0000 7f49bb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49bc0a9950 0x7f49bc13e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.622+0000 7f49bb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49bc0a9950 0x7f49bc13e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43302/0 (socket says 172.21.15.119:43302) 2024-04-25T10:13:36.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.622+0000 7f49bb400640 1 -- 172.21.15.119:0/3785854863 learned_addr learned my addr 172.21.15.119:0/3785854863 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:36.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.623+0000 7f49bbe00640 1 --2- 172.21.15.119:0/3785854863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a0004d90 0x7f49bc15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.623+0000 7f49c0e00640 1 --2- 172.21.15.119:0/3785854863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49bc13ed10 0x7f49bc1f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.623+0000 7f49bb400640 1 -- 172.21.15.119:0/3785854863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a0004d90 msgr2=0x7f49bc15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.623+0000 7f49bb400640 1 --2- 172.21.15.119:0/3785854863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a0004d90 0x7f49bc15b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.623+0000 7f49bb400640 1 -- 172.21.15.119:0/3785854863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49bc13ed10 msgr2=0x7f49bc1f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.623+0000 7f49bb400640 1 --2- 172.21.15.119:0/3785854863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49bc13ed10 0x7f49bc1f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.623+0000 7f49bb400640 1 -- 172.21.15.119:0/3785854863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc0a8b90 con 0x7f49bc0a9950 2024-04-25T10:13:36.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.623+0000 7f49bb400640 1 --2- 172.21.15.119:0/3785854863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49bc0a9950 0x7f49bc13e7d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f49a800d5a0 tx=0x7f49a800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.623+0000 7f49b3e00640 1 -- 172.21.15.119:0/3785854863 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8003610 con 0x7f49bc0a9950 2024-04-25T10:13:36.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.624+0000 7f49b3e00640 1 -- 172.21.15.119:0/3785854863 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a8017070 con 0x7f49bc0a9950 2024-04-25T10:13:36.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.624+0000 7f49b3e00640 1 -- 172.21.15.119:0/3785854863 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a80039d0 con 0x7f49bc0a9950 2024-04-25T10:13:36.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.624+0000 7f49c1800640 1 -- 172.21.15.119:0/3785854863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49bc0a8e70 con 0x7f49bc0a9950 2024-04-25T10:13:36.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.624+0000 7f49c1800640 1 -- 172.21.15.119:0/3785854863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc15b690 con 0x7f49bc0a9950 2024-04-25T10:13:36.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.625+0000 7f49c1800640 1 -- 172.21.15.119:0/3785854863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49bc0749b0 con 0x7f49bc0a9950 2024-04-25T10:13:36.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.626+0000 7f49b3e00640 1 -- 172.21.15.119:0/3785854863 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49a8003b70 con 0x7f49bc0a9950 2024-04-25T10:13:36.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.627+0000 7f49b3e00640 1 --2- 172.21.15.119:0/3785854863 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f499c06b500 0x7f499c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.627+0000 7f49b3e00640 1 -- 172.21.15.119:0/3785854863 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f49a808a0b0 con 0x7f49bc0a9950 2024-04-25T10:13:36.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.627+0000 7f49bbe00640 1 --2- 172.21.15.119:0/3785854863 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f499c06b500 0x7f499c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.628+0000 7f49bbe00640 1 --2- 172.21.15.119:0/3785854863 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f499c06b500 0x7f499c06d9c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f49ac005b70 tx=0x7f49ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.631+0000 7f49b3e00640 1 -- 172.21.15.119:0/3785854863 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49a8058f80 con 0x7f49bc0a9950 2024-04-25T10:13:36.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:36.865+0000 7f49c1800640 1 -- 172.21.15.119:0/3785854863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f49bc13f120 con 0x7f49bc0a9950 2024-04-25T10:13:37.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.167+0000 7f49b3e00640 1 -- 172.21.15.119:0/3785854863 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v63) v1 ==== 162+0+0 (secure 0 0 0) 0x7f49a8058880 con 0x7f49bc0a9950 2024-04-25T10:13:37.172 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:13:37.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.171+0000 7f49c1800640 1 -- 172.21.15.119:0/3785854863 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f499c06b500 msgr2=0x7f499c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.171+0000 7f49c1800640 1 --2- 172.21.15.119:0/3785854863 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f499c06b500 0x7f499c06d9c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f49ac005b70 tx=0x7f49ac009040 comp rx=0 tx=0).stop 2024-04-25T10:13:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.171+0000 7f49c1800640 1 -- 172.21.15.119:0/3785854863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49bc0a9950 msgr2=0x7f49bc13e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.171+0000 7f49c1800640 1 --2- 172.21.15.119:0/3785854863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49bc0a9950 0x7f49bc13e7d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f49a800d5a0 tx=0x7f49a800da60 comp rx=0 tx=0).stop 2024-04-25T10:13:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.171+0000 7f49c1800640 1 -- 172.21.15.119:0/3785854863 shutdown_connections 2024-04-25T10:13:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.171+0000 7f49c1800640 1 --2- 172.21.15.119:0/3785854863 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f499c06b500 0x7f499c06d9c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.171+0000 7f49c1800640 1 --2- 172.21.15.119:0/3785854863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49bc13ed10 0x7f49bc1f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.171+0000 7f49c1800640 1 --2- 172.21.15.119:0/3785854863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49bc0a9950 0x7f49bc13e7d0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.171+0000 7f49c1800640 1 --2- 172.21.15.119:0/3785854863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49a0004d90 0x7f49bc15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.171+0000 7f49c1800640 1 -- 172.21.15.119:0/3785854863 >> 172.21.15.119:0/3785854863 conn(0x7f49bc066ff0 msgr2=0x7f49bc0b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.171+0000 7f49c1800640 1 -- 172.21.15.119:0/3785854863 shutdown_connections 2024-04-25T10:13:37.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.171+0000 7f49c1800640 1 -- 172.21.15.119:0/3785854863 wait complete. 2024-04-25T10:13:37.198 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.373+0000 7fcf15200640 1 Processor -- start 2024-04-25T10:13:37.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.373+0000 7fcf15200640 1 -- start start 2024-04-25T10:13:37.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.374+0000 7fcf15200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10144040 0x7fcf100cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.374+0000 7fcf15200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcf1015acd0 0x7fcf1006e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.374+0000 7fcf15200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf10144420 con 0x7fcf100cce40 2024-04-25T10:13:37.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.374+0000 7fcf15200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf100b36d0 con 0x7fcf1015acd0 2024-04-25T10:13:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.374+0000 7fcf15200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100b3d80 con 0x7fcf10144040 2024-04-25T10:13:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.375+0000 7fcf0fe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcf100cce40 0x7fcf1015e2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37014/0 (socket says 172.21.15.119:37014) 2024-04-25T10:13:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.375+0000 7fcf06a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcf1015acd0 0x7fcf1006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.375+0000 7fcf0fe00640 1 -- 172.21.15.119:0/128023992 learned_addr learned my addr 172.21.15.119:0/128023992 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.375+0000 7fcf0f400640 1 --2- 172.21.15.119:0/128023992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10144040 0x7fcf100cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.375+0000 7fcf0ea00640 1 -- 172.21.15.119:0/128023992 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356716293 0 0) 0x7fcf10144420 con 0x7fcf100cce40 2024-04-25T10:13:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.375+0000 7fcf06a00640 1 -- 172.21.15.119:0/128023992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10144040 msgr2=0x7fcf100cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.375+0000 7fcf06a00640 1 --2- 172.21.15.119:0/128023992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10144040 0x7fcf100cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.375+0000 7fcf06a00640 1 -- 172.21.15.119:0/128023992 >> v1:172.21.15.119:6789/0 conn(0x7fcf100cce40 legacy=0x7fcf1015e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.375+0000 7fcf06a00640 1 -- 172.21.15.119:0/128023992 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf100b4060 con 0x7fcf1015acd0 2024-04-25T10:13:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.376+0000 7fcf06a00640 1 --2- 172.21.15.119:0/128023992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf1015acd0 0x7fcf1006e320 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcef80097a0 tx=0x7fcef802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e09317042f5f104 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.377+0000 7fcf0ea00640 1 -- 172.21.15.119:0/128023992 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8037070 con 0x7fcf1015acd0 2024-04-25T10:13:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.377+0000 7fcf0ea00640 1 -- 172.21.15.119:0/128023992 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef802ae50 con 0x7fcf1015acd0 2024-04-25T10:13:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.377+0000 7fcf0ea00640 1 -- 172.21.15.119:0/128023992 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8032660 con 0x7fcf1015acd0 2024-04-25T10:13:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.377+0000 7fcf15200640 1 -- 172.21.15.119:0/128023992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf1015acd0 msgr2=0x7fcf1006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.377+0000 7fcf15200640 1 --2- 172.21.15.119:0/128023992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf1015acd0 0x7fcf1006e320 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcef80097a0 tx=0x7fcef802a100 comp rx=0 tx=0).stop 2024-04-25T10:13:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.377+0000 7fcf15200640 1 -- 172.21.15.119:0/128023992 shutdown_connections 2024-04-25T10:13:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.377+0000 7fcf15200640 1 --2- 172.21.15.119:0/128023992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf1015acd0 0x7fcf1006e320 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.377+0000 7fcf15200640 1 --2- 172.21.15.119:0/128023992 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10144040 0x7fcf100cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.377+0000 7fcf15200640 1 -- 172.21.15.119:0/128023992 >> 172.21.15.119:0/128023992 conn(0x7fcf1006b1d0 msgr2=0x7fcf100b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.377+0000 7fcf15200640 1 -- 172.21.15.119:0/128023992 shutdown_connections 2024-04-25T10:13:37.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.377+0000 7fcf15200640 1 -- 172.21.15.119:0/128023992 wait complete. 2024-04-25T10:13:37.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.379+0000 7fcf15200640 1 Processor -- start 2024-04-25T10:13:37.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.379+0000 7fcf15200640 1 -- start start 2024-04-25T10:13:37.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.380+0000 7fcf15200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf100cce40 0x7fcf101e4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.380+0000 7fcf15200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10144040 0x7fcf100b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.380+0000 7fcf15200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf1015acd0 0x7fcf100b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.380+0000 7fcf15200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100b4900 con 0x7fcf100cce40 2024-04-25T10:13:37.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.380+0000 7fcf15200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100b4620 con 0x7fcf1015acd0 2024-04-25T10:13:37.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.380+0000 7fcf15200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100b5180 con 0x7fcf10144040 2024-04-25T10:13:37.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.380+0000 7fcf0f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf100cce40 0x7fcf101e4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.380+0000 7fcf0f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf100cce40 0x7fcf101e4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43312/0 (socket says 172.21.15.119:43312) 2024-04-25T10:13:37.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.380+0000 7fcf0f400640 1 -- 172.21.15.119:0/1600761200 learned_addr learned my addr 172.21.15.119:0/1600761200 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.381+0000 7fcf0f400640 1 -- 172.21.15.119:0/1600761200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10144040 msgr2=0x7fcf100b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.381+0000 7fcf06a00640 1 --2- 172.21.15.119:0/1600761200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10144040 0x7fcf100b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.381+0000 7fcf0f400640 1 --2- 172.21.15.119:0/1600761200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10144040 0x7fcf100b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.381+0000 7fcf0f400640 1 -- 172.21.15.119:0/1600761200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf1015acd0 msgr2=0x7fcf100b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.381+0000 7fcf0f400640 1 --2- 172.21.15.119:0/1600761200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf1015acd0 0x7fcf100b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.381+0000 7fcf0f400640 1 -- 172.21.15.119:0/1600761200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf101e4b50 con 0x7fcf100cce40 2024-04-25T10:13:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.381+0000 7fcf0f400640 1 --2- 172.21.15.119:0/1600761200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf100cce40 0x7fcf101e4610 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000d800 tx=0x7fcf0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.381+0000 7fcf07e00640 1 -- 172.21.15.119:0/1600761200 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000b6c0 con 0x7fcf100cce40 2024-04-25T10:13:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.381+0000 7fcf07e00640 1 -- 172.21.15.119:0/1600761200 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf00019070 con 0x7fcf100cce40 2024-04-25T10:13:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.381+0000 7fcf07e00640 1 -- 172.21.15.119:0/1600761200 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000ba80 con 0x7fcf100cce40 2024-04-25T10:13:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.381+0000 7fcf15200640 1 -- 172.21.15.119:0/1600761200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf101406b0 con 0x7fcf100cce40 2024-04-25T10:13:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.382+0000 7fcf15200640 1 -- 172.21.15.119:0/1600761200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1012df80 con 0x7fcf100cce40 2024-04-25T10:13:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.383+0000 7fcf15200640 1 -- 172.21.15.119:0/1600761200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1006bb30 con 0x7fcf100cce40 2024-04-25T10:13:37.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.384+0000 7fcf07e00640 1 -- 172.21.15.119:0/1600761200 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf0000bc20 con 0x7fcf100cce40 2024-04-25T10:13:37.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.385+0000 7fcf07e00640 1 --2- 172.21.15.119:0/1600761200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fceec06b4d0 0x7fceec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.385+0000 7fcf07e00640 1 -- 172.21.15.119:0/1600761200 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcf0008a680 con 0x7fcf100cce40 2024-04-25T10:13:37.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.385+0000 7fcf06a00640 1 --2- 172.21.15.119:0/1600761200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fceec06b4d0 0x7fceec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.386+0000 7fcf06a00640 1 --2- 172.21.15.119:0/1600761200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fceec06b4d0 0x7fceec06d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcef8034000 tx=0x7fcef802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.389+0000 7fcf07e00640 1 -- 172.21.15.119:0/1600761200 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf00059530 con 0x7fcf100cce40 2024-04-25T10:13:37.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.618+0000 7fcf15200640 1 -- 172.21.15.119:0/1600761200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf100b5690 con 0x7fcf100cce40 2024-04-25T10:13:37.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.619+0000 7fcf07e00640 1 -- 172.21.15.119:0/1600761200 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4953 (secure 0 0 0) 0x7fcf100b5690 con 0x7fcf100cce40 2024-04-25T10:13:37.624 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:37.624 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:13:37.173204+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"a","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.119:6837/1734507541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1734507541},{"type":"v1","addr":"172.21.15.119:6837","nonce":1734507541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:13:37.624 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 64 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.622+0000 7fcf15200640 1 -- 172.21.15.119:0/1600761200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fceec06b4d0 msgr2=0x7fceec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.622+0000 7fcf15200640 1 --2- 172.21.15.119:0/1600761200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fceec06b4d0 0x7fceec06d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcef8034000 tx=0x7fcef802e000 comp rx=0 tx=0).stop 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.622+0000 7fcf15200640 1 -- 172.21.15.119:0/1600761200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf100cce40 msgr2=0x7fcf101e4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.622+0000 7fcf15200640 1 --2- 172.21.15.119:0/1600761200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf100cce40 0x7fcf101e4610 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000d800 tx=0x7fcf0000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.622+0000 7fcf15200640 1 -- 172.21.15.119:0/1600761200 shutdown_connections 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.622+0000 7fcf15200640 1 --2- 172.21.15.119:0/1600761200 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fceec06b4d0 0x7fceec06d990 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.622+0000 7fcf15200640 1 --2- 172.21.15.119:0/1600761200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf1015acd0 0x7fcf100b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.622+0000 7fcf15200640 1 --2- 172.21.15.119:0/1600761200 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf10144040 0x7fcf100b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.622+0000 7fcf15200640 1 --2- 172.21.15.119:0/1600761200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf100cce40 0x7fcf101e4610 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.622+0000 7fcf15200640 1 -- 172.21.15.119:0/1600761200 >> 172.21.15.119:0/1600761200 conn(0x7fcf1006b1d0 msgr2=0x7fcf100b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.622+0000 7fcf15200640 1 -- 172.21.15.119:0/1600761200 shutdown_connections 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:37.622+0000 7fcf15200640 1 -- 172.21.15.119:0/1600761200 wait complete. 2024-04-25T10:13:38.047 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:23.427654+0000', 'modified': '2024-04-25T10:13:37.173204+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 82, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7374}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7374': {'gid': 7374, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.119:6837/1734507541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 1734507541}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 1734507541}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:38.047 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7374:up:replay 2024-04-25T10:13:39.048 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:39.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.256+0000 7f404a800640 1 Processor -- start 2024-04-25T10:13:39.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.256+0000 7f404a800640 1 -- start start 2024-04-25T10:13:39.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.258+0000 7f404a800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4044147270 0x7f40440b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.258+0000 7f404a800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f404415dee0 0x7f404415e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.258+0000 7f404a800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f404415e890 0x7f40441614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.258+0000 7f404a800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f404415ed20 con 0x7f404415dee0 2024-04-25T10:13:39.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.258+0000 7f404a800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40440af000 con 0x7f4044147270 2024-04-25T10:13:39.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.258+0000 7f404a800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40440b4a80 con 0x7f404415e890 2024-04-25T10:13:39.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.258+0000 7f4048a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f404415dee0 0x7f404415e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.258+0000 7f4048a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f404415dee0 0x7f404415e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43318/0 (socket says 172.21.15.119:43318) 2024-04-25T10:13:39.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.258+0000 7f4048a00640 1 -- 172.21.15.119:0/2569118230 learned_addr learned my addr 172.21.15.119:0/2569118230 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:39.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.258+0000 7f4049e00640 1 --2- 172.21.15.119:0/2569118230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f404415e890 0x7f40441614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.259+0000 7f4049400640 1 --2- 172.21.15.119:0/2569118230 >> v2:172.21.15.132:3300/0 conn(0x7f4044147270 0x7f40440b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.259+0000 7f4048a00640 1 -- 172.21.15.119:0/2569118230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f404415e890 msgr2=0x7f40441614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.259+0000 7f4048a00640 1 --2- 172.21.15.119:0/2569118230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f404415e890 0x7f40441614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.259+0000 7f4048a00640 1 -- 172.21.15.119:0/2569118230 >> v2:172.21.15.132:3300/0 conn(0x7f4044147270 msgr2=0x7f40440b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.259+0000 7f4048a00640 1 --2- 172.21.15.119:0/2569118230 >> v2:172.21.15.132:3300/0 conn(0x7f4044147270 0x7f40440b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.259+0000 7f4048a00640 1 -- 172.21.15.119:0/2569118230 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40440b4d60 con 0x7f404415dee0 2024-04-25T10:13:39.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.259+0000 7f4049400640 1 --2- 172.21.15.119:0/2569118230 >> v2:172.21.15.132:3300/0 conn(0x7f4044147270 0x7f40440b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:39.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.260+0000 7f4048a00640 1 --2- 172.21.15.119:0/2569118230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404415dee0 0x7f404415e350 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f404406f5a0 tx=0x7f403002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195ca9efbfbf4b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.260+0000 7f4043e00640 1 -- 172.21.15.119:0/2569118230 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4030037070 con 0x7f404415dee0 2024-04-25T10:13:39.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.260+0000 7f4043e00640 1 -- 172.21.15.119:0/2569118230 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4030032440 con 0x7f404415dee0 2024-04-25T10:13:39.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.260+0000 7f4043e00640 1 -- 172.21.15.119:0/2569118230 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4030032810 con 0x7f404415dee0 2024-04-25T10:13:39.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.261+0000 7f404a800640 1 -- 172.21.15.119:0/2569118230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404415dee0 msgr2=0x7f404415e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.261+0000 7f404a800640 1 --2- 172.21.15.119:0/2569118230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404415dee0 0x7f404415e350 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f404406f5a0 tx=0x7f403002a140 comp rx=0 tx=0).stop 2024-04-25T10:13:39.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.261+0000 7f404a800640 1 -- 172.21.15.119:0/2569118230 shutdown_connections 2024-04-25T10:13:39.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.261+0000 7f404a800640 1 --2- 172.21.15.119:0/2569118230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f404415e890 0x7f40441614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.261+0000 7f404a800640 1 --2- 172.21.15.119:0/2569118230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404415dee0 0x7f404415e350 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.261+0000 7f404a800640 1 --2- 172.21.15.119:0/2569118230 >> v2:172.21.15.132:3300/0 conn(0x7f4044147270 0x7f40440b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.261+0000 7f404a800640 1 -- 172.21.15.119:0/2569118230 >> 172.21.15.119:0/2569118230 conn(0x7f404406b1d0 msgr2=0x7f40440b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.261+0000 7f404a800640 1 -- 172.21.15.119:0/2569118230 shutdown_connections 2024-04-25T10:13:39.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.261+0000 7f404a800640 1 -- 172.21.15.119:0/2569118230 wait complete. 2024-04-25T10:13:39.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.262+0000 7f404a800640 1 Processor -- start 2024-04-25T10:13:39.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.263+0000 7f404a800640 1 -- start start 2024-04-25T10:13:39.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.263+0000 7f404a800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4044147270 0x7f404415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.263+0000 7f404a800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f404415dee0 0x7f40440ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.263+0000 7f404a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404415e890 0x7f40440ad350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.264+0000 7f404a800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40440b5600 con 0x7f404415e890 2024-04-25T10:13:39.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.264+0000 7f404a800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40440b5320 con 0x7f4044147270 2024-04-25T10:13:39.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.264+0000 7f404a800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40441619e0 con 0x7f404415dee0 2024-04-25T10:13:39.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.264+0000 7f4049e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404415e890 0x7f40440ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.264+0000 7f4049e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404415e890 0x7f40440ad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43328/0 (socket says 172.21.15.119:43328) 2024-04-25T10:13:39.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.264+0000 7f4049e00640 1 -- 172.21.15.119:0/1341371845 learned_addr learned my addr 172.21.15.119:0/1341371845 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:39.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.264+0000 7f4048a00640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f404415dee0 0x7f40440ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.264+0000 7f4049400640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4044147270 0x7f404415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.264+0000 7f4049e00640 1 -- 172.21.15.119:0/1341371845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f404415dee0 msgr2=0x7f40440ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.264+0000 7f4049e00640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f404415dee0 0x7f40440ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.264+0000 7f4049e00640 1 -- 172.21.15.119:0/1341371845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4044147270 msgr2=0x7f404415a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.264+0000 7f4049e00640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4044147270 0x7f404415a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.264+0000 7f4049e00640 1 -- 172.21.15.119:0/1341371845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404415f5f0 con 0x7f404415e890 2024-04-25T10:13:39.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.264+0000 7f4049400640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4044147270 0x7f404415a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:39.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.265+0000 7f4049e00640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404415e890 0x7f40440ad350 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f403c007e90 tx=0x7f403c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.265+0000 7f4041600640 1 -- 172.21.15.119:0/1341371845 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c014420 con 0x7f404415e890 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.265+0000 7f4041600640 1 -- 172.21.15.119:0/1341371845 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c019070 con 0x7f404415e890 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.265+0000 7f4041600640 1 -- 172.21.15.119:0/1341371845 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c014800 con 0x7f404415e890 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.265+0000 7f404a800640 1 -- 172.21.15.119:0/1341371845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404415f8d0 con 0x7f404415e890 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.265+0000 7f404a800640 1 -- 172.21.15.119:0/1341371845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40441f39b0 con 0x7f404415e890 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.266+0000 7f404a800640 1 -- 172.21.15.119:0/1341371845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4044067530 con 0x7f404415e890 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.267+0000 7f4041600640 1 -- 172.21.15.119:0/1341371845 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f403c0149a0 con 0x7f404415e890 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.268+0000 7f4041600640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f402406b4d0 0x7f402406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.268+0000 7f4041600640 1 -- 172.21.15.119:0/1341371845 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f403c08aaa0 con 0x7f404415e890 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.268+0000 7f4049400640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f402406b4d0 0x7f402406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.269+0000 7f4049400640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f402406b4d0 0x7f402406d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4044160ad0 tx=0x7f4038008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.273+0000 7f4041600640 1 -- 172.21.15.119:0/1341371845 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4044067530 con 0x7f404415e890 2024-04-25T10:13:39.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.512+0000 7f404a800640 1 -- 172.21.15.119:0/1341371845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40440749b0 con 0x7f404415e890 2024-04-25T10:13:39.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.513+0000 7f4041600640 1 -- 172.21.15.119:0/1341371845 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f403c0598b0 con 0x7f404415e890 2024-04-25T10:13:39.517 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:39.517 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:13:38.192095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"a","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.119:6837/1734507541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1734507541},{"type":"v1","addr":"172.21.15.119:6837","nonce":1734507541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:13:39.518 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 66 2024-04-25T10:13:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.515+0000 7f404a800640 1 -- 172.21.15.119:0/1341371845 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f402406b4d0 msgr2=0x7f402406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.515+0000 7f404a800640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f402406b4d0 0x7f402406d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4044160ad0 tx=0x7f4038008040 comp rx=0 tx=0).stop 2024-04-25T10:13:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.515+0000 7f404a800640 1 -- 172.21.15.119:0/1341371845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404415e890 msgr2=0x7f40440ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.515+0000 7f404a800640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404415e890 0x7f40440ad350 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f403c007e90 tx=0x7f403c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.516+0000 7f404a800640 1 -- 172.21.15.119:0/1341371845 shutdown_connections 2024-04-25T10:13:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.516+0000 7f404a800640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f402406b4d0 0x7f402406d990 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.516+0000 7f404a800640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404415e890 0x7f40440ad350 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.516+0000 7f404a800640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f404415dee0 0x7f40440ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.516+0000 7f404a800640 1 --2- 172.21.15.119:0/1341371845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4044147270 0x7f404415a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.516+0000 7f404a800640 1 -- 172.21.15.119:0/1341371845 >> 172.21.15.119:0/1341371845 conn(0x7f404406b1d0 msgr2=0x7f4044071900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.516+0000 7f404a800640 1 -- 172.21.15.119:0/1341371845 shutdown_connections 2024-04-25T10:13:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:39.516+0000 7f404a800640 1 -- 172.21.15.119:0/1341371845 wait complete. 2024-04-25T10:13:39.942 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:23.427654+0000', 'modified': '2024-04-25T10:13:38.192095+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 82, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7374}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7374': {'gid': 7374, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.119:6837/1734507541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 1734507541}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 1734507541}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:39.942 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7374:up:rejoin 2024-04-25T10:13:40.943 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:41.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.149+0000 7f8fda800640 1 Processor -- start 2024-04-25T10:13:41.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.149+0000 7f8fda800640 1 -- start start 2024-04-25T10:13:41.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.150+0000 7f8fda800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fd415d980 0x7f8fd40ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.150+0000 7f8fda800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8fd40aeb20 0x7f8fd40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.150+0000 7f8fda800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd40712c0 con 0x7f8fd40cf7c0 2024-04-25T10:13:41.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.150+0000 7f8fda800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd4071ed0 con 0x7f8fd40aeb20 2024-04-25T10:13:41.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.151+0000 7f8fda800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd40cfbf0 con 0x7f8fd415d980 2024-04-25T10:13:41.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.151+0000 7f8fd3e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8fd40cf7c0 0x7f8fd415dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36614/0 (socket says 172.21.15.119:36614) 2024-04-25T10:13:41.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.151+0000 7f8fd3e00640 1 -- 172.21.15.119:0/3718982406 learned_addr learned my addr 172.21.15.119:0/3718982406 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:41.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.151+0000 7f8fd3400640 1 --2- 172.21.15.119:0/3718982406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fd415d980 0x7f8fd40ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.151+0000 7f8fcaa00640 1 --2- 172.21.15.119:0/3718982406 >> v2:172.21.15.132:3300/0 conn(0x7f8fd40aeb20 0x7f8fd40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.151+0000 7f8fd2a00640 1 -- 172.21.15.119:0/3718982406 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946533606 0 0) 0x7f8fd40712c0 con 0x7f8fd40cf7c0 2024-04-25T10:13:41.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.151+0000 7f8fd2a00640 1 -- 172.21.15.119:0/3718982406 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fb8003520 con 0x7f8fd40cf7c0 2024-04-25T10:13:41.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fd2a00640 1 -- 172.21.15.119:0/3718982406 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1092927564 0 0) 0x7f8fb8003520 con 0x7f8fd40cf7c0 2024-04-25T10:13:41.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fd2a00640 1 -- 172.21.15.119:0/3718982406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fd415d980 msgr2=0x7f8fd40ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:41.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fd2a00640 1 --2- 172.21.15.119:0/3718982406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fd415d980 0x7f8fd40ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fd2a00640 1 -- 172.21.15.119:0/3718982406 >> v2:172.21.15.132:3300/0 conn(0x7f8fd40aeb20 msgr2=0x7f8fd40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:41.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fd2a00640 1 --2- 172.21.15.119:0/3718982406 >> v2:172.21.15.132:3300/0 conn(0x7f8fd40aeb20 0x7f8fd40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fd2a00640 1 -- 172.21.15.119:0/3718982406 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd40715a0 con 0x7f8fd40cf7c0 2024-04-25T10:13:41.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fd2a00640 1 -- 172.21.15.119:0/3718982406 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f8fcc0019f0 con 0x7f8fd40cf7c0 2024-04-25T10:13:41.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fd2a00640 1 -- 172.21.15.119:0/3718982406 >> v1:172.21.15.119:6789/0 conn(0x7f8fd40cf7c0 legacy=0x7f8fd415dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fd2a00640 1 --2- 172.21.15.119:0/3718982406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fd415d980 0x7f8fb80039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fd2a00640 1 --2- 172.21.15.119:0/3718982406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fb8004000 0x7f8fb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fd2a00640 1 --2- 172.21.15.119:0/3718982406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb8004d40 0x7f8fb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fd2a00640 1 -- 172.21.15.119:0/3718982406 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd4071ed0 con 0x7f8fb8004000 2024-04-25T10:13:41.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fd2a00640 1 -- 172.21.15.119:0/3718982406 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd40cfbf0 con 0x7f8fb8004d40 2024-04-25T10:13:41.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fd2a00640 1 -- 172.21.15.119:0/3718982406 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb800bb80 con 0x7f8fd415d980 2024-04-25T10:13:41.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.152+0000 7f8fcaa00640 1 --2- 172.21.15.119:0/3718982406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fb8004000 0x7f8fb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.153+0000 7f8fd3e00640 1 --2- 172.21.15.119:0/3718982406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb8004d40 0x7f8fb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.153+0000 7f8fd3400640 1 --2- 172.21.15.119:0/3718982406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fd415d980 0x7f8fb80039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.153+0000 7f8fcaa00640 1 -- 172.21.15.119:0/3718982406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fd415d980 msgr2=0x7f8fb80039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:41.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.153+0000 7f8fcaa00640 1 --2- 172.21.15.119:0/3718982406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fd415d980 0x7f8fb80039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.153+0000 7f8fcaa00640 1 -- 172.21.15.119:0/3718982406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb8004d40 msgr2=0x7f8fb80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:41.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.153+0000 7f8fcaa00640 1 --2- 172.21.15.119:0/3718982406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb8004d40 0x7f8fb80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.153+0000 7f8fcaa00640 1 -- 172.21.15.119:0/3718982406 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb8003520 con 0x7f8fb8004000 2024-04-25T10:13:41.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.153+0000 7f8fd3e00640 1 --2- 172.21.15.119:0/3718982406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb8004d40 0x7f8fb80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:41.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.154+0000 7f8fcaa00640 1 --2- 172.21.15.119:0/3718982406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fb8004000 0x7f8fb8004800 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00ba30 tx=0x7f8fb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6653baeeecde074b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:41.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.154+0000 7f8fd2a00640 1 -- 172.21.15.119:0/3718982406 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc030070 con 0x7f8fb8004000 2024-04-25T10:13:41.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.154+0000 7f8fd2a00640 1 -- 172.21.15.119:0/3718982406 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc02ae10 con 0x7f8fb8004000 2024-04-25T10:13:41.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.154+0000 7f8fd2a00640 1 -- 172.21.15.119:0/3718982406 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00b600 con 0x7f8fb8004000 2024-04-25T10:13:41.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.154+0000 7f8fda800640 1 -- 172.21.15.119:0/3718982406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fb8004000 msgr2=0x7f8fb8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.154+0000 7f8fda800640 1 --2- 172.21.15.119:0/3718982406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fb8004000 0x7f8fb8004800 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00ba30 tx=0x7f8fb8005830 comp rx=0 tx=0).stop 2024-04-25T10:13:41.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.155+0000 7f8fd3e00640 1 -- 172.21.15.119:0/3718982406 reap_dead start 2024-04-25T10:13:41.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.155+0000 7f8fda800640 1 -- 172.21.15.119:0/3718982406 shutdown_connections 2024-04-25T10:13:41.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.155+0000 7f8fda800640 1 -- 172.21.15.119:0/3718982406 >> 172.21.15.119:0/3718982406 conn(0x7f8fd406aba0 msgr2=0x7f8fd406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:41.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.155+0000 7f8fda800640 1 -- 172.21.15.119:0/3718982406 shutdown_connections 2024-04-25T10:13:41.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.155+0000 7f8fda800640 1 -- 172.21.15.119:0/3718982406 wait complete. 2024-04-25T10:13:41.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.156+0000 7f8fda800640 1 Processor -- start 2024-04-25T10:13:41.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.157+0000 7f8fda800640 1 -- start start 2024-04-25T10:13:41.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.157+0000 7f8fda800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb8004000 0x7f8fd414f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fda800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd4152da0 0x7f8fd4154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fda800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fd41552d0 0x7f8fd41f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fda800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd40adf80 con 0x7f8fd4152da0 2024-04-25T10:13:41.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fda800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd4071b60 con 0x7f8fb8004000 2024-04-25T10:13:41.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fda800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd40ae100 con 0x7f8fd41552d0 2024-04-25T10:13:41.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fcaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd4152da0 0x7f8fd4154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fcaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd4152da0 0x7f8fd4154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49360/0 (socket says 172.21.15.119:49360) 2024-04-25T10:13:41.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fcaa00640 1 -- 172.21.15.119:0/629270657 learned_addr learned my addr 172.21.15.119:0/629270657 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:41.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fd3e00640 1 --2- 172.21.15.119:0/629270657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fd41552d0 0x7f8fd41f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fd3400640 1 --2- 172.21.15.119:0/629270657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb8004000 0x7f8fd414f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fcaa00640 1 -- 172.21.15.119:0/629270657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fd41552d0 msgr2=0x7f8fd41f6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fcaa00640 1 --2- 172.21.15.119:0/629270657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fd41552d0 0x7f8fd41f6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fcaa00640 1 -- 172.21.15.119:0/629270657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb8004000 msgr2=0x7f8fd414f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fcaa00640 1 --2- 172.21.15.119:0/629270657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb8004000 0x7f8fd414f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.158+0000 7f8fcaa00640 1 -- 172.21.15.119:0/629270657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd4154af0 con 0x7f8fd4152da0 2024-04-25T10:13:41.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.159+0000 7f8fcaa00640 1 --2- 172.21.15.119:0/629270657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd4152da0 0x7f8fd4154d90 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc02a900 tx=0x7f8fbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:41.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.159+0000 7f8fcbe00640 1 -- 172.21.15.119:0/629270657 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc043030 con 0x7f8fd4152da0 2024-04-25T10:13:41.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.159+0000 7f8fcbe00640 1 -- 172.21.15.119:0/629270657 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc034e50 con 0x7f8fd4152da0 2024-04-25T10:13:41.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.159+0000 7f8fda800640 1 -- 172.21.15.119:0/629270657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd4153a50 con 0x7f8fd4152da0 2024-04-25T10:13:41.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.159+0000 7f8fcbe00640 1 -- 172.21.15.119:0/629270657 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc004020 con 0x7f8fd4152da0 2024-04-25T10:13:41.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.159+0000 7f8fda800640 1 -- 172.21.15.119:0/629270657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd4154010 con 0x7f8fd4152da0 2024-04-25T10:13:41.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.160+0000 7f8fda800640 1 -- 172.21.15.119:0/629270657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd40749b0 con 0x7f8fd4152da0 2024-04-25T10:13:41.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.161+0000 7f8fcbe00640 1 -- 172.21.15.119:0/629270657 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fbc046070 con 0x7f8fd4152da0 2024-04-25T10:13:41.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.162+0000 7f8fcbe00640 1 --2- 172.21.15.119:0/629270657 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fb006b4d0 0x7f8fb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.162+0000 7f8fcbe00640 1 -- 172.21.15.119:0/629270657 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8fbc0aacf0 con 0x7f8fd4152da0 2024-04-25T10:13:41.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.162+0000 7f8fd3400640 1 --2- 172.21.15.119:0/629270657 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fb006b4d0 0x7f8fb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.163+0000 7f8fd3400640 1 --2- 172.21.15.119:0/629270657 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fb006b4d0 0x7f8fb006d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4006fd0 tx=0x7f8fc4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:41.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.166+0000 7f8fcbe00640 1 -- 172.21.15.119:0/629270657 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8fbc04d070 con 0x7f8fd4152da0 2024-04-25T10:13:41.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.401+0000 7f8fda800640 1 -- 172.21.15.119:0/629270657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fd41556b0 con 0x7f8fd4152da0 2024-04-25T10:13:41.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.402+0000 7f8fcbe00640 1 -- 172.21.15.119:0/629270657 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f8fbc0333b0 con 0x7f8fd4152da0 2024-04-25T10:13:41.407 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:13:41.407 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:13:39.203189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/1734507541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1734507541},{"type":"v1","addr":"172.21.15.119:6837","nonce":1734507541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:13:41.407 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 67 2024-04-25T10:13:41.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.404+0000 7f8fda800640 1 -- 172.21.15.119:0/629270657 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fb006b4d0 msgr2=0x7f8fb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.404+0000 7f8fda800640 1 --2- 172.21.15.119:0/629270657 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fb006b4d0 0x7f8fb006d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4006fd0 tx=0x7f8fc4008040 comp rx=0 tx=0).stop 2024-04-25T10:13:41.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.404+0000 7f8fda800640 1 -- 172.21.15.119:0/629270657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd4152da0 msgr2=0x7f8fd4154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.404+0000 7f8fda800640 1 --2- 172.21.15.119:0/629270657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd4152da0 0x7f8fd4154d90 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc02a900 tx=0x7f8fbc004730 comp rx=0 tx=0).stop 2024-04-25T10:13:41.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.404+0000 7f8fda800640 1 -- 172.21.15.119:0/629270657 shutdown_connections 2024-04-25T10:13:41.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.404+0000 7f8fda800640 1 --2- 172.21.15.119:0/629270657 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8fb006b4d0 0x7f8fb006d990 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.404+0000 7f8fda800640 1 --2- 172.21.15.119:0/629270657 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8fd41552d0 0x7f8fd41f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.404+0000 7f8fda800640 1 --2- 172.21.15.119:0/629270657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd4152da0 0x7f8fd4154d90 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.404+0000 7f8fda800640 1 --2- 172.21.15.119:0/629270657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fb8004000 0x7f8fd414f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.404+0000 7f8fda800640 1 -- 172.21.15.119:0/629270657 >> 172.21.15.119:0/629270657 conn(0x7f8fd406aba0 msgr2=0x7f8fd40af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:41.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.404+0000 7f8fda800640 1 -- 172.21.15.119:0/629270657 shutdown_connections 2024-04-25T10:13:41.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:41.404+0000 7f8fda800640 1 -- 172.21.15.119:0/629270657 wait complete. 2024-04-25T10:13:41.830 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:23.427654+0000', 'modified': '2024-04-25T10:13:39.203189+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 82, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7374}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7374': {'gid': 7374, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.119:6837/1734507541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 1734507541}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 1734507541}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:41.830 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:13:41.831 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:13:42.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.037+0000 7f8c99e00640 1 Processor -- start 2024-04-25T10:13:42.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.037+0000 7f8c99e00640 1 -- start start 2024-04-25T10:13:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.039+0000 7f8c99e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c94071660 0x7f8c940cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.039+0000 7f8c99e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c940cc9a0 con 0x7f8c94145260 2024-04-25T10:13:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.039+0000 7f8c99e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c940ccc80 con 0x7f8c94071200 2024-04-25T10:13:42.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.039+0000 7f8c99e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c940cea00 con 0x7f8c94071660 2024-04-25T10:13:42.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.040+0000 7f8c93400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c94071660 0x7f8c940cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.039+0000 7f8c92a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8c94145260 0x7f8c940cbf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36630/0 (socket says 172.21.15.119:36630) 2024-04-25T10:13:42.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.040+0000 7f8c92a00640 1 -- 172.21.15.119:0/2519435783 learned_addr learned my addr 172.21.15.119:0/2519435783 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:42.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.040+0000 7f8c93400640 1 -- 172.21.15.119:0/2519435783 >> v1:172.21.15.132:6789/0 conn(0x7f8c94071200 legacy=0x7f8c940cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.040+0000 7f8c93400640 1 -- 172.21.15.119:0/2519435783 >> v1:172.21.15.119:6789/0 conn(0x7f8c94145260 legacy=0x7f8c940cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.040+0000 7f8c93400640 1 -- 172.21.15.119:0/2519435783 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c940ccf60 con 0x7f8c94071660 2024-04-25T10:13:42.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.041+0000 7f8c93400640 1 --2- 172.21.15.119:0/2519435783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94071660 0x7f8c940cf5a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009d90 tx=0x7f8c8c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f013312d37ba0666 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.041+0000 7f8c92000640 1 -- 172.21.15.119:0/2519435783 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c007740 con 0x7f8c94071660 2024-04-25T10:13:42.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.042+0000 7f8c92000640 1 -- 172.21.15.119:0/2519435783 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c0078e0 con 0x7f8c94071660 2024-04-25T10:13:42.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.042+0000 7f8c92000640 1 -- 172.21.15.119:0/2519435783 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c007cb0 con 0x7f8c94071660 2024-04-25T10:13:42.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.042+0000 7f8c99e00640 1 -- 172.21.15.119:0/2519435783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94071660 msgr2=0x7f8c940cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.042+0000 7f8c99e00640 1 --2- 172.21.15.119:0/2519435783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94071660 0x7f8c940cf5a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009d90 tx=0x7f8c8c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.042+0000 7f8c99e00640 1 -- 172.21.15.119:0/2519435783 shutdown_connections 2024-04-25T10:13:42.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.042+0000 7f8c99e00640 1 --2- 172.21.15.119:0/2519435783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94071660 0x7f8c940cf5a0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.042+0000 7f8c99e00640 1 -- 172.21.15.119:0/2519435783 >> 172.21.15.119:0/2519435783 conn(0x7f8c9406aba0 msgr2=0x7f8c9406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.042+0000 7f8c99e00640 1 -- 172.21.15.119:0/2519435783 shutdown_connections 2024-04-25T10:13:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.042+0000 7f8c99e00640 1 -- 172.21.15.119:0/2519435783 wait complete. 2024-04-25T10:13:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.044+0000 7f8c99e00640 1 Processor -- start 2024-04-25T10:13:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.044+0000 7f8c99e00640 1 -- start start 2024-04-25T10:13:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.045+0000 7f8c99e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c94071200 0x7f8c9415b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.045+0000 7f8c99e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94071660 0x7f8c940b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.045+0000 7f8c99e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c94145260 0x7f8c940ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.045+0000 7f8c99e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c94071a90 con 0x7f8c94071660 2024-04-25T10:13:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.045+0000 7f8c99e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940cc660 con 0x7f8c94145260 2024-04-25T10:13:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.045+0000 7f8c99e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c94145640 con 0x7f8c94071200 2024-04-25T10:13:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.045+0000 7f8c8be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94071660 0x7f8c940b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.045+0000 7f8c8be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94071660 0x7f8c940b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49382/0 (socket says 172.21.15.119:49382) 2024-04-25T10:13:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.045+0000 7f8c8be00640 1 -- 172.21.15.119:0/2892970795 learned_addr learned my addr 172.21.15.119:0/2892970795 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.046+0000 7f8c8be00640 1 -- 172.21.15.119:0/2892970795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c94071200 msgr2=0x7f8c9415b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.046+0000 7f8c92a00640 1 --2- 172.21.15.119:0/2892970795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c94071200 0x7f8c9415b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.046+0000 7f8c8be00640 1 --2- 172.21.15.119:0/2892970795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c94071200 0x7f8c9415b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.046+0000 7f8c8be00640 1 -- 172.21.15.119:0/2892970795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c94145260 msgr2=0x7f8c940ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.046+0000 7f8c8be00640 1 --2- 172.21.15.119:0/2892970795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c94145260 0x7f8c940ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.046+0000 7f8c8be00640 1 -- 172.21.15.119:0/2892970795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c940b3f10 con 0x7f8c94071660 2024-04-25T10:13:42.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.046+0000 7f8c8be00640 1 --2- 172.21.15.119:0/2892970795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94071660 0x7f8c940b3690 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00b3f0 tx=0x7f8c7c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.046+0000 7f8c8b400640 1 -- 172.21.15.119:0/2892970795 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c015020 con 0x7f8c94071660 2024-04-25T10:13:42.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.046+0000 7f8c8b400640 1 -- 172.21.15.119:0/2892970795 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c011830 con 0x7f8c94071660 2024-04-25T10:13:42.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.047+0000 7f8c8b400640 1 -- 172.21.15.119:0/2892970795 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c011c20 con 0x7f8c94071660 2024-04-25T10:13:42.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.047+0000 7f8c99e00640 1 -- 172.21.15.119:0/2892970795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c941e5630 con 0x7f8c94071660 2024-04-25T10:13:42.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.047+0000 7f8c99e00640 1 -- 172.21.15.119:0/2892970795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c941e5bf0 con 0x7f8c94071660 2024-04-25T10:13:42.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.049+0000 7f8c8b400640 1 -- 172.21.15.119:0/2892970795 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c7c018020 con 0x7f8c94071660 2024-04-25T10:13:42.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.050+0000 7f8c8b400640 1 --2- 172.21.15.119:0/2892970795 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.050+0000 7f8c8b400640 1 -- 172.21.15.119:0/2892970795 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8c7c089f00 con 0x7f8c94071660 2024-04-25T10:13:42.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.050+0000 7f8c92a00640 1 --2- 172.21.15.119:0/2892970795 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.050+0000 7f8c9cbf6740 1 Processor -- start 2024-04-25T10:13:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.051+0000 7f8c9cbf6740 1 -- start start 2024-04-25T10:13:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.051+0000 7f8c92a00640 1 --2- 172.21.15.119:0/2892970795 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8c7006b4d0 0x7f8c7006d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8c84000f60 tx=0x7f8c84009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.051+0000 7f8c9cbf6740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e951ff4950 0x55e95201c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c9cbf6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95201ccd0 0x55e951ff4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c9cbf6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95203d5c0 0x55e951ff3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c9cbf6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e951f58620 con 0x55e95203d5c0 2024-04-25T10:13:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c9cbf6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e951f01ca0 con 0x55e95201ccd0 2024-04-25T10:13:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c9cbf6740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e951f546c0 con 0x55e951ff4950 2024-04-25T10:13:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c93400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95203d5c0 0x55e951ff3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c93400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95203d5c0 0x55e951ff3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49396/0 (socket says 172.21.15.119:49396) 2024-04-25T10:13:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c93400640 1 -- 172.21.15.119:0/2090056521 learned_addr learned my addr 172.21.15.119:0/2090056521 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c92a00640 1 --2- 172.21.15.119:0/2090056521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e951ff4950 0x55e95201c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c8be00640 1 --2- 172.21.15.119:0/2090056521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95201ccd0 0x55e951ff4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c93400640 1 -- 172.21.15.119:0/2090056521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e951ff4950 msgr2=0x55e95201c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c93400640 1 --2- 172.21.15.119:0/2090056521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e951ff4950 0x55e95201c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c93400640 1 -- 172.21.15.119:0/2090056521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95201ccd0 msgr2=0x55e951ff4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c93400640 1 --2- 172.21.15.119:0/2090056521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95201ccd0 0x55e951ff4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c93400640 1 -- 172.21.15.119:0/2090056521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e951f08f40 con 0x55e95203d5c0 2024-04-25T10:13:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.052+0000 7f8c8be00640 1 --2- 172.21.15.119:0/2090056521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95201ccd0 0x55e951ff4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.053+0000 7f8c93400640 1 --2- 172.21.15.119:0/2090056521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95203d5c0 0x55e951ff3110 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009eb0 tx=0x7f8c8c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbec90b860730cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.053+0000 7f8c83400640 1 -- 172.21.15.119:0/2090056521 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c040070 con 0x55e95203d5c0 2024-04-25T10:13:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.053+0000 7f8c83400640 1 -- 172.21.15.119:0/2090056521 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c007840 con 0x55e95203d5c0 2024-04-25T10:13:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.053+0000 7f8c83400640 1 -- 172.21.15.119:0/2090056521 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c007bf0 con 0x55e95203d5c0 2024-04-25T10:13:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.054+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/2090056521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95203d5c0 msgr2=0x55e951ff3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.054+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/2090056521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95203d5c0 0x55e951ff3110 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009eb0 tx=0x7f8c8c02c000 comp rx=0 tx=0).stop 2024-04-25T10:13:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.054+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/2090056521 shutdown_connections 2024-04-25T10:13:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.054+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/2090056521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95203d5c0 0x55e951ff3110 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.054+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/2090056521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95201ccd0 0x55e951ff4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.054+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/2090056521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e951ff4950 0x55e95201c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.054+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/2090056521 >> 172.21.15.119:0/2090056521 conn(0x55e951ff1b70 msgr2=0x55e95203a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.054+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/2090056521 shutdown_connections 2024-04-25T10:13:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.054+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/2090056521 wait complete. 2024-04-25T10:13:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.055+0000 7f8c9cbf6740 1 -- start start 2024-04-25T10:13:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.055+0000 7f8c9cbf6740 1 Processor -- start 2024-04-25T10:13:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.056+0000 7f8c9cbf6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95201ccd0 0x55e95204aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.056+0000 7f8c9cbf6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95203d5c0 0x55e95201aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.056+0000 7f8c9cbf6740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e9520481e0 0x55e95201afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.056+0000 7f8c9cbf6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95201c300 con 0x55e95201ccd0 2024-04-25T10:13:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.056+0000 7f8c9cbf6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e951ef71f0 con 0x55e95203d5c0 2024-04-25T10:13:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.056+0000 7f8c9cbf6740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e95201bcf0 con 0x55e9520481e0 2024-04-25T10:13:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.056+0000 7f8c9cbf6740 10 client.0 _ll_register_callbacks cb 0x55e951f14ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.056+0000 7f8c92a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95201ccd0 0x55e95204aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.056+0000 7f8c92a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95201ccd0 0x55e95204aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49412/0 (socket says 172.21.15.119:49412) 2024-04-25T10:13:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.056+0000 7f8c92a00640 1 -- 172.21.15.119:0/3054587822 learned_addr learned my addr 172.21.15.119:0/3054587822 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.057+0000 7f8c93400640 1 --2- 172.21.15.119:0/3054587822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e9520481e0 0x55e95201afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.057+0000 7f8c92a00640 1 -- 172.21.15.119:0/3054587822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e9520481e0 msgr2=0x55e95201afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.057+0000 7f8c92a00640 1 --2- 172.21.15.119:0/3054587822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e9520481e0 0x55e95201afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.057+0000 7f8c8be00640 1 --2- 172.21.15.119:0/3054587822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95203d5c0 0x55e95201aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.057+0000 7f8c92a00640 1 -- 172.21.15.119:0/3054587822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95203d5c0 msgr2=0x55e95201aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.057+0000 7f8c92a00640 1 --2- 172.21.15.119:0/3054587822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95203d5c0 0x55e95201aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.057+0000 7f8c92a00640 1 -- 172.21.15.119:0/3054587822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e951f16c50 con 0x55e95201ccd0 2024-04-25T10:13:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.057+0000 7f8c92a00640 1 --2- 172.21.15.119:0/3054587822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95201ccd0 0x55e95204aff0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f8c84007680 tx=0x7f8c84000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8d98727ff10890 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.057+0000 7f8c77e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:13:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.057+0000 7f8c77e00640 1 -- 172.21.15.119:0/3054587822 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84026060 con 0x55e95201ccd0 2024-04-25T10:13:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.057+0000 7f8c77e00640 1 -- 172.21.15.119:0/3054587822 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8c84021af0 con 0x55e95201ccd0 2024-04-25T10:13:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.057+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/3054587822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c8c007840 con 0x55e95201ccd0 2024-04-25T10:13:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.058+0000 7f8c77e00640 1 -- 172.21.15.119:0/3054587822 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c84005e20 con 0x55e95201ccd0 2024-04-25T10:13:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.058+0000 7f8c9cbf6740 10 client.7440 fetch_fsmap learned FSMap version 67 2024-04-25T10:13:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.058+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/3054587822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55e951c6d2d0 con 0x55e95201ccd0 2024-04-25T10:13:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.058+0000 7f8c77e00640 1 -- 172.21.15.119:0/3054587822 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c007840 con 0x55e95201ccd0 2024-04-25T10:13:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.058+0000 7f8c77e00640 1 -- 172.21.15.119:0/3054587822 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8c84021430 con 0x55e95201ccd0 2024-04-25T10:13:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.058+0000 7f8c9cbf6740 10 client.7440 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:13:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.058+0000 7f8c9cbf6740 10 client.7440 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:13:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.058+0000 7f8c9cbf6740 20 client.7440 populate_metadata read hostname 'smithi119' 2024-04-25T10:13:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.058+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/3054587822 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55e95201d5c0 0x55e95201d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.058+0000 7f8c9cbf6740 4 client.7440 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:42.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.058+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/3054587822 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e951f65b60 con 0x55e95201d5c0 2024-04-25T10:13:42.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.059+0000 7f8c8be00640 1 --2- 172.21.15.119:0/3054587822 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55e95201d5c0 0x55e95201d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.059+0000 7f8c8be00640 1 --2- 172.21.15.119:0/3054587822 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55e95201d5c0 0x55e95201d9a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66e6dc85e195c0e2 server_cookie=65e51840afa3b4e8 in_seq=0 out_seq=0 2024-04-25T10:13:42.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.059+0000 7f8c77e00640 10 client.7440 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:13:42.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c77e00640 1 -- 172.21.15.119:0/3054587822 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e951f65b60 con 0x55e95201d5c0 2024-04-25T10:13:42.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c77e00640 10 client.7440 handle_command_reply: tid=0 2024-04-25T10:13:42.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c9cbf6740 1 client.7440 shutdown 2024-04-25T10:13:42.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/3054587822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95201ccd0 msgr2=0x55e95204aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/3054587822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95201ccd0 0x55e95204aff0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f8c84007680 tx=0x7f8c84000bc0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/3054587822 shutdown_connections 2024-04-25T10:13:42.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/3054587822 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55e95201d5c0 0x55e95201d9a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/3054587822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e9520481e0 0x55e95201afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/3054587822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95203d5c0 0x55e95201aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/3054587822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95201ccd0 0x55e95204aff0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c77e00640 0 client.7440 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:13:42.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/3054587822 >> 172.21.15.119:0/3054587822 conn(0x55e951ff4950 msgr2=0x55e951ff3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/3054587822 shutdown_connections 2024-04-25T10:13:42.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/3054587822 wait complete. 2024-04-25T10:13:42.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.060+0000 7f8c9cbf6740 20 client.7440 trim_cache size 0 max 16384 2024-04-25T10:13:42.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.077+0000 7f8c9cbf6740 1 Processor -- start 2024-04-25T10:13:42.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.078+0000 7f8c9cbf6740 1 -- start start 2024-04-25T10:13:42.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.078+0000 7f8c9cbf6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95201ccd0 0x55e952039ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.078+0000 7f8c9cbf6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95203d5c0 0x55e95201aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.078+0000 7f8c9cbf6740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e9520481e0 0x55e95201afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.078+0000 7f8c9cbf6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c941ff150 con 0x55e95203d5c0 2024-04-25T10:13:42.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.078+0000 7f8c9cbf6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c941f4680 con 0x55e95201ccd0 2024-04-25T10:13:42.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.078+0000 7f8c9cbf6740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c941e6e50 con 0x55e9520481e0 2024-04-25T10:13:42.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.079+0000 7f8c8be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95203d5c0 0x55e95201aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.079+0000 7f8c8be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95203d5c0 0x55e95201aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49418/0 (socket says 172.21.15.119:49418) 2024-04-25T10:13:42.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.079+0000 7f8c8be00640 1 -- 172.21.15.119:0/1941591852 learned_addr learned my addr 172.21.15.119:0/1941591852 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:42.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.079+0000 7f8c8be00640 1 -- 172.21.15.119:0/1941591852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e9520481e0 msgr2=0x55e95201afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.079+0000 7f8c8be00640 1 --2- 172.21.15.119:0/1941591852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e9520481e0 0x55e95201afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.079+0000 7f8c8be00640 1 -- 172.21.15.119:0/1941591852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95201ccd0 msgr2=0x55e952039ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.079+0000 7f8c8be00640 1 --2- 172.21.15.119:0/1941591852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95201ccd0 0x55e952039ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.079+0000 7f8c8be00640 1 -- 172.21.15.119:0/1941591852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e951d376b0 con 0x55e95203d5c0 2024-04-25T10:13:42.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.079+0000 7f8c8be00640 1 --2- 172.21.15.119:0/1941591852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95203d5c0 0x55e95201aa70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c058310 tx=0x7f8c7c0548b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de01d2f51b9251f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.080+0000 7f8c82a00640 1 -- 172.21.15.119:0/1941591852 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c070070 con 0x55e95203d5c0 2024-04-25T10:13:42.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.080+0000 7f8c82a00640 1 -- 172.21.15.119:0/1941591852 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c011dc0 con 0x55e95203d5c0 2024-04-25T10:13:42.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.080+0000 7f8c82a00640 1 -- 172.21.15.119:0/1941591852 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c088440 con 0x55e95203d5c0 2024-04-25T10:13:42.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.080+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/1941591852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95203d5c0 msgr2=0x55e95201aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.080+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/1941591852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95203d5c0 0x55e95201aa70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c058310 tx=0x7f8c7c0548b0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.080+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/1941591852 shutdown_connections 2024-04-25T10:13:42.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.080+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/1941591852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e9520481e0 0x55e95201afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.080+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/1941591852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e95203d5c0 0x55e95201aa70 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.080+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/1941591852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95201ccd0 0x55e952039ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.080+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/1941591852 >> 172.21.15.119:0/1941591852 conn(0x55e951ff4950 msgr2=0x55e9520393a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.080+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/1941591852 shutdown_connections 2024-04-25T10:13:42.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.080+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/1941591852 wait complete. 2024-04-25T10:13:42.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.081+0000 7f8c9cbf6740 1 -- start start 2024-04-25T10:13:42.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.081+0000 7f8c9cbf6740 1 Processor -- start 2024-04-25T10:13:42.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.082+0000 7f8c9cbf6740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e95201ccd0 0x55e95201ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.082+0000 7f8c9cbf6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95203d5c0 0x55e95209b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.082+0000 7f8c9cbf6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9520481e0 0x55e95209bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.082+0000 7f8c9cbf6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95204c4c0 con 0x55e9520481e0 2024-04-25T10:13:42.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.082+0000 7f8c9cbf6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e95204bfc0 con 0x55e95203d5c0 2024-04-25T10:13:42.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.082+0000 7f8c9cbf6740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e95204cff0 con 0x55e95201ccd0 2024-04-25T10:13:42.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.082+0000 7f8c9cbf6740 10 client.0 _ll_register_callbacks cb 0x55e951ff3f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:42.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.083+0000 7f8c93400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9520481e0 0x55e95209bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.083+0000 7f8c93400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9520481e0 0x55e95209bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49422/0 (socket says 172.21.15.119:49422) 2024-04-25T10:13:42.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.083+0000 7f8c93400640 1 -- 172.21.15.119:0/1363427281 learned_addr learned my addr 172.21.15.119:0/1363427281 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:42.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.083+0000 7f8c92a00640 1 --2- 172.21.15.119:0/1363427281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e95201ccd0 0x55e95201ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.083+0000 7f8c8be00640 1 --2- 172.21.15.119:0/1363427281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95203d5c0 0x55e95209b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.083+0000 7f8c93400640 1 -- 172.21.15.119:0/1363427281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e95201ccd0 msgr2=0x55e95201ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.083+0000 7f8c93400640 1 --2- 172.21.15.119:0/1363427281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e95201ccd0 0x55e95201ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.083+0000 7f8c93400640 1 -- 172.21.15.119:0/1363427281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95203d5c0 msgr2=0x55e95209b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.083+0000 7f8c93400640 1 --2- 172.21.15.119:0/1363427281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95203d5c0 0x55e95209b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.083+0000 7f8c93400640 1 -- 172.21.15.119:0/1363427281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e95203d9a0 con 0x55e9520481e0 2024-04-25T10:13:42.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.083+0000 7f8c93400640 1 --2- 172.21.15.119:0/1363427281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9520481e0 0x55e95209bb30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c02a6e0 tx=0x7f8c8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fff0aa62242a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.083+0000 7f8c77e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:13:42.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.084+0000 7f8c77e00640 1 -- 172.21.15.119:0/1363427281 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c053060 con 0x55e9520481e0 2024-04-25T10:13:42.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.084+0000 7f8c77e00640 1 -- 172.21.15.119:0/1363427281 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8c8c004780 con 0x55e9520481e0 2024-04-25T10:13:42.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.084+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/1363427281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c7c011dc0 con 0x55e9520481e0 2024-04-25T10:13:42.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.084+0000 7f8c77e00640 1 -- 172.21.15.119:0/1363427281 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c8c059050 con 0x55e9520481e0 2024-04-25T10:13:42.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.084+0000 7f8c9cbf6740 10 client.7446 fetch_fsmap learned FSMap version 67 2024-04-25T10:13:42.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.084+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/1363427281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55e952054320 con 0x55e9520481e0 2024-04-25T10:13:42.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.084+0000 7f8c77e00640 1 -- 172.21.15.119:0/1363427281 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c011dc0 con 0x55e9520481e0 2024-04-25T10:13:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.084+0000 7f8c77e00640 1 -- 172.21.15.119:0/1363427281 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8c8c05c020 con 0x55e9520481e0 2024-04-25T10:13:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.084+0000 7f8c9cbf6740 10 client.7446 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:13:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.084+0000 7f8c9cbf6740 10 client.7446 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:13:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.084+0000 7f8c9cbf6740 20 client.7446 populate_metadata read hostname 'smithi119' 2024-04-25T10:13:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.084+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/1363427281 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55e95203caf0 0x55e952017420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.084+0000 7f8c9cbf6740 4 client.7446 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:13:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.084+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/1363427281 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e951ff22d0 con 0x55e95203caf0 2024-04-25T10:13:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.085+0000 7f8c92a00640 1 --2- 172.21.15.119:0/1363427281 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55e95203caf0 0x55e952017420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.085+0000 7f8c92a00640 1 --2- 172.21.15.119:0/1363427281 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55e95203caf0 0x55e952017420 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ffd1828c2865951 server_cookie=50619e1cb021a2c9 in_seq=0 out_seq=0 2024-04-25T10:13:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.085+0000 7f8c77e00640 10 client.7446 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:13:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.085+0000 7f8c77e00640 1 -- 172.21.15.119:0/1363427281 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55e951ff22d0 con 0x55e95203caf0 2024-04-25T10:13:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.085+0000 7f8c77e00640 10 client.7446 handle_command_reply: tid=0 2024-04-25T10:13:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.085+0000 7f8c9cbf6740 1 client.7446 shutdown 2024-04-25T10:13:42.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.086+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/1363427281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9520481e0 msgr2=0x55e95209bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.086+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/1363427281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9520481e0 0x55e95209bb30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c02a6e0 tx=0x7f8c8c004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.086+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/1363427281 shutdown_connections 2024-04-25T10:13:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.086+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/1363427281 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55e95203caf0 0x55e952017420 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.086+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/1363427281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e9520481e0 0x55e95209bb30 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.086+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/1363427281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e95203d5c0 0x55e95209b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.086+0000 7f8c9cbf6740 1 --2- 172.21.15.119:0/1363427281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e95201ccd0 0x55e95201ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.086+0000 7f8c77e00640 0 client.7446 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:13:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.086+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/1363427281 >> 172.21.15.119:0/1363427281 conn(0x55e951ff4950 msgr2=0x55e952039630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.086+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/1363427281 shutdown_connections 2024-04-25T10:13:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.086+0000 7f8c9cbf6740 1 -- 172.21.15.119:0/1363427281 wait complete. 2024-04-25T10:13:42.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.086+0000 7f8c9cbf6740 20 client.7446 trim_cache size 0 max 16384 2024-04-25T10:13:42.095 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:13:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "e0b0f428-b441-4237-a4d1-ed53321f87a5", 2024-04-25T10:13:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-04-25T10:13:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 7374, 2024-04-25T10:13:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-04-25T10:13:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-04-25T10:13:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-04-25T10:13:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 4.9055097319999996, 2024-04-25T10:13:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 67, 2024-04-25T10:13:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 82, 2024-04-25T10:13:42.095 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 82, 2024-04-25T10:13:42.096 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 5.8289817389999996 2024-04-25T10:13:42.096 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:13:42.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.087+0000 7f8c99e00640 1 -- 172.21.15.119:0/2892970795 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8c7006b4d0 msgr2=0x7f8c7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.087+0000 7f8c99e00640 1 --2- 172.21.15.119:0/2892970795 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8c7006b4d0 0x7f8c7006d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8c84000f60 tx=0x7f8c84009040 comp rx=0 tx=0).stop 2024-04-25T10:13:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.087+0000 7f8c99e00640 1 -- 172.21.15.119:0/2892970795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94071660 msgr2=0x7f8c940b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.087+0000 7f8c99e00640 1 --2- 172.21.15.119:0/2892970795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94071660 0x7f8c940b3690 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00b3f0 tx=0x7f8c7c00b8b0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.087+0000 7f8c99e00640 1 -- 172.21.15.119:0/2892970795 shutdown_connections 2024-04-25T10:13:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.087+0000 7f8c99e00640 1 --2- 172.21.15.119:0/2892970795 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.087+0000 7f8c99e00640 1 --2- 172.21.15.119:0/2892970795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c94145260 0x7f8c940ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.087+0000 7f8c99e00640 1 --2- 172.21.15.119:0/2892970795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94071660 0x7f8c940b3690 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.087+0000 7f8c99e00640 1 --2- 172.21.15.119:0/2892970795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8c94071200 0x7f8c9415b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.087+0000 7f8c99e00640 1 -- 172.21.15.119:0/2892970795 >> 172.21.15.119:0/2892970795 conn(0x7f8c9406aba0 msgr2=0x7f8c940afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.087+0000 7f8c99e00640 1 -- 172.21.15.119:0/2892970795 shutdown_connections 2024-04-25T10:13:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.087+0000 7f8c99e00640 1 -- 172.21.15.119:0/2892970795 wait complete. 2024-04-25T10:13:42.109 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:13:42.188 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:13:42.184+0000 7f5368c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:42.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.295+0000 7f7302a00640 1 Processor -- start 2024-04-25T10:13:42.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.295+0000 7f7302a00640 1 -- start start 2024-04-25T10:13:42.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.297+0000 7f7302a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f72fc15a730 0x7f72fc0ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.297+0000 7f7302a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72fc0b4020 0x7f72fc0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.297+0000 7f7302a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72fc0cc020 con 0x7f72fc06d1b0 2024-04-25T10:13:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.297+0000 7f7302a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72fc0b49a0 con 0x7f72fc15a730 2024-04-25T10:13:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.297+0000 7f7302a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc0b32b0 con 0x7f72fc0b4020 2024-04-25T10:13:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.297+0000 7f72fbe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f72fc15a730 0x7f72fc0ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.297+0000 7f72fbe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f72fc15a730 0x7f72fc0ae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49424/0 (socket says 172.21.15.119:49424) 2024-04-25T10:13:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.297+0000 7f72fbe00640 1 -- 172.21.15.119:0/1953096081 learned_addr learned my addr 172.21.15.119:0/1953096081 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.298+0000 7f72fb400640 1 --2- 172.21.15.119:0/1953096081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72fc0b4020 0x7f72fc0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.298+0000 7f72fbe00640 1 -- 172.21.15.119:0/1953096081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72fc0b4020 msgr2=0x7f72fc0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.298+0000 7f72fbe00640 1 --2- 172.21.15.119:0/1953096081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72fc0b4020 0x7f72fc0b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.298+0000 7f72fbe00640 1 -- 172.21.15.119:0/1953096081 >> v1:172.21.15.132:6789/0 conn(0x7f72fc06d1b0 legacy=0x7f72fc06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.298+0000 7f72fbe00640 1 -- 172.21.15.119:0/1953096081 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc0b3590 con 0x7f72fc15a730 2024-04-25T10:13:42.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.299+0000 7f72fbe00640 1 --2- 172.21.15.119:0/1953096081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72fc15a730 0x7f72fc0ae9b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0b4b20 tx=0x7f72ec02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11131b17b6a7e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.299+0000 7f72faa00640 1 -- 172.21.15.119:0/1953096081 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec037070 con 0x7f72fc15a730 2024-04-25T10:13:42.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.300+0000 7f72faa00640 1 -- 172.21.15.119:0/1953096081 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec032440 con 0x7f72fc15a730 2024-04-25T10:13:42.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.300+0000 7f72faa00640 1 -- 172.21.15.119:0/1953096081 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec032810 con 0x7f72fc15a730 2024-04-25T10:13:42.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.300+0000 7f7302a00640 1 -- 172.21.15.119:0/1953096081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72fc15a730 msgr2=0x7f72fc0ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.300+0000 7f7302a00640 1 --2- 172.21.15.119:0/1953096081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72fc15a730 0x7f72fc0ae9b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0b4b20 tx=0x7f72ec02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.300+0000 7f7302a00640 1 -- 172.21.15.119:0/1953096081 shutdown_connections 2024-04-25T10:13:42.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.300+0000 7f7302a00640 1 --2- 172.21.15.119:0/1953096081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72fc0b4020 0x7f72fc0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.300+0000 7f7302a00640 1 --2- 172.21.15.119:0/1953096081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72fc15a730 0x7f72fc0ae9b0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.300+0000 7f7302a00640 1 -- 172.21.15.119:0/1953096081 >> 172.21.15.119:0/1953096081 conn(0x7f72fc06aba0 msgr2=0x7f72fc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.300+0000 7f7302a00640 1 -- 172.21.15.119:0/1953096081 shutdown_connections 2024-04-25T10:13:42.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.300+0000 7f7302a00640 1 -- 172.21.15.119:0/1953096081 wait complete. 2024-04-25T10:13:42.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.302+0000 7f7302a00640 1 Processor -- start 2024-04-25T10:13:42.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.302+0000 7f7302a00640 1 -- start start 2024-04-25T10:13:42.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.303+0000 7f7302a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72fc06d1b0 0x7f72fc13bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.303+0000 7f7302a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72fc0b4020 0x7f72fc0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.303+0000 7f7302a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72fc15a730 0x7f72fc1e71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.303+0000 7f7302a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc0cd620 con 0x7f72fc06d1b0 2024-04-25T10:13:42.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.303+0000 7f7302a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc0b3b50 con 0x7f72fc0b4020 2024-04-25T10:13:42.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.303+0000 7f7302a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc0aeef0 con 0x7f72fc15a730 2024-04-25T10:13:42.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.303+0000 7f72fbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72fc06d1b0 0x7f72fc13bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.303+0000 7f72fbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72fc06d1b0 0x7f72fc13bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49438/0 (socket says 172.21.15.119:49438) 2024-04-25T10:13:42.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.304+0000 7f72fbe00640 1 -- 172.21.15.119:0/2159957303 learned_addr learned my addr 172.21.15.119:0/2159957303 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:42.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.304+0000 7f72fb400640 1 --2- 172.21.15.119:0/2159957303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72fc0b4020 0x7f72fc0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.304+0000 7f72fbe00640 1 -- 172.21.15.119:0/2159957303 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72fc15a730 msgr2=0x7f72fc1e71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.304+0000 7f72fbe00640 1 --2- 172.21.15.119:0/2159957303 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72fc15a730 0x7f72fc1e71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.304+0000 7f72fbe00640 1 -- 172.21.15.119:0/2159957303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72fc0b4020 msgr2=0x7f72fc0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.304+0000 7f72fbe00640 1 --2- 172.21.15.119:0/2159957303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72fc0b4020 0x7f72fc0cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.304+0000 7f72fbe00640 1 -- 172.21.15.119:0/2159957303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc13c330 con 0x7f72fc06d1b0 2024-04-25T10:13:42.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.304+0000 7f72fbe00640 1 --2- 172.21.15.119:0/2159957303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72fc06d1b0 0x7f72fc13bdf0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f72ec000c00 tx=0x7f72ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.304+0000 7f72f3e00640 1 -- 172.21.15.119:0/2159957303 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec041020 con 0x7f72fc06d1b0 2024-04-25T10:13:42.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.305+0000 7f72f3e00640 1 -- 172.21.15.119:0/2159957303 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec030970 con 0x7f72fc06d1b0 2024-04-25T10:13:42.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.305+0000 7f7302a00640 1 -- 172.21.15.119:0/2159957303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72fc0ceb50 con 0x7f72fc06d1b0 2024-04-25T10:13:42.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.305+0000 7f72f3e00640 1 -- 172.21.15.119:0/2159957303 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec037040 con 0x7f72fc06d1b0 2024-04-25T10:13:42.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.305+0000 7f7302a00640 1 -- 172.21.15.119:0/2159957303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72fc0ce870 con 0x7f72fc06d1b0 2024-04-25T10:13:42.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.306+0000 7f7302a00640 1 -- 172.21.15.119:0/2159957303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72fc0749b0 con 0x7f72fc06d1b0 2024-04-25T10:13:42.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.307+0000 7f72f3e00640 1 -- 172.21.15.119:0/2159957303 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72ec044020 con 0x7f72fc06d1b0 2024-04-25T10:13:42.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.308+0000 7f72f3e00640 1 --2- 172.21.15.119:0/2159957303 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f72dc06b4d0 0x7f72dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.308+0000 7f72f3e00640 1 -- 172.21.15.119:0/2159957303 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f72ec0aac50 con 0x7f72fc06d1b0 2024-04-25T10:13:42.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.308+0000 7f72fb400640 1 --2- 172.21.15.119:0/2159957303 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f72dc06b4d0 0x7f72dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.309+0000 7f72fb400640 1 --2- 172.21.15.119:0/2159957303 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f72dc06b4d0 0x7f72dc06d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f72e8006fd0 tx=0x7f72e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.313+0000 7f72f3e00640 1 -- 172.21.15.119:0/2159957303 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f72ec04b030 con 0x7f72fc06d1b0 2024-04-25T10:13:42.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.547+0000 7f7302a00640 1 -- 172.21.15.119:0/2159957303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f72fc0d0010 con 0x7f72fc06d1b0 2024-04-25T10:13:42.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.547+0000 7f72f3e00640 1 -- 172.21.15.119:0/2159957303 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) v1 ==== 53+0+73 (secure 0 0 0) 0x7f72ec079a60 con 0x7f72fc06d1b0 2024-04-25T10:13:42.552 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:13:42.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.550+0000 7f7302a00640 1 -- 172.21.15.119:0/2159957303 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f72dc06b4d0 msgr2=0x7f72dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.550+0000 7f7302a00640 1 --2- 172.21.15.119:0/2159957303 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f72dc06b4d0 0x7f72dc06d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f72e8006fd0 tx=0x7f72e8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:42.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.550+0000 7f7302a00640 1 -- 172.21.15.119:0/2159957303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72fc06d1b0 msgr2=0x7f72fc13bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.550+0000 7f7302a00640 1 --2- 172.21.15.119:0/2159957303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72fc06d1b0 0x7f72fc13bdf0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f72ec000c00 tx=0x7f72ec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.550+0000 7f7302a00640 1 -- 172.21.15.119:0/2159957303 shutdown_connections 2024-04-25T10:13:42.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.550+0000 7f7302a00640 1 --2- 172.21.15.119:0/2159957303 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f72dc06b4d0 0x7f72dc06d990 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.550+0000 7f7302a00640 1 --2- 172.21.15.119:0/2159957303 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72fc15a730 0x7f72fc1e71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.550+0000 7f7302a00640 1 --2- 172.21.15.119:0/2159957303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72fc0b4020 0x7f72fc0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.550+0000 7f7302a00640 1 --2- 172.21.15.119:0/2159957303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72fc06d1b0 0x7f72fc13bdf0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.550+0000 7f7302a00640 1 -- 172.21.15.119:0/2159957303 >> 172.21.15.119:0/2159957303 conn(0x7f72fc06aba0 msgr2=0x7f72fc157800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.550+0000 7f7302a00640 1 -- 172.21.15.119:0/2159957303 shutdown_connections 2024-04-25T10:13:42.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:42.550+0000 7f7302a00640 1 -- 172.21.15.119:0/2159957303 wait complete. 2024-04-25T10:13:42.573 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:13:42.573 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:13:42.574 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:13:42.574 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:13:42.574 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:42.574 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:13:42.574 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:13:42.574 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:13:42.574 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:42.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:42.574 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:13:42.620 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:13:42.620 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:13:42.620 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:13:42.621 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:42.621 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:13:42.621 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:42.621 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:42.621 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:42.621 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:13:42.621 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:42.621 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:42.621 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:13:42.621 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:42.621 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:42.621 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:13:42.621 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:42.621 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:13:42.622 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42267sec preferred_lft 42267sec 2024-04-25T10:13:42.622 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:13:42.622 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:42.622 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:13:42.622 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:42.622 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:13:42.622 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:42.622 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:13:42.622 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:42.622 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:42.622 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:42.622 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:13:42.622 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:42.622 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:42.623 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:42.623 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:13:42.623 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:42.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:42.624 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:13:42.696 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:13:42.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:42.700 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:13:42.750 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:13:42.750 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:13:42.750 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:13:42.750 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:13:42.750 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:13:42.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:42.751 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:13:42.751 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:13:42.751 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:13:42.752 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:13:42.752 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:13:42.752 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:13:42.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:42.896 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:13:42.945 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:13:42.945 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:13:42.946 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:42.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:42.947 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:13:43.030 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:43.030 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:43.085 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:43.085 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:43.085 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:43.134 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:13:43.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:43.135 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:43.262 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:43.262 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:43.262 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:13:43.283 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:43.283 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:43.283 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:13:43.283 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:13:43.283 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:43.307 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:43.307 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:43.307 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:13:43.308 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:13:43.308 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:43.308 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:13:43.308 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:13:43.308 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:13:43.309 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:13:43.309 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:13:43.309 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:13:43.309 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:13:43.310 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:13:43.314 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:13:43.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:43.314 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:43.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:43.384 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:13:43.432 INFO:teuthology.orchestra.run.smithi119.stdout:client7455 2024-04-25T10:13:43.432 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:13:43.432 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:43.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:43.525 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-25T10:13:43.579 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:13:43.574+0000 7f5363c00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T10:13:43.581 INFO:teuthology.orchestra.run.smithi119.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-25T10:13:43.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:13:43.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:43.582 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-25T10:13:43.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.780+0000 7fa9e1400640 1 Processor -- start 2024-04-25T10:13:43.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.781+0000 7fa9e1400640 1 -- start start 2024-04-25T10:13:43.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.782+0000 7fa9e1400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9dc146410 0x7fa9dc0cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.782+0000 7fa9e1400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa9dc0cb960 0x7fa9dc1467f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.782+0000 7fa9e1400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9dc0c9c40 con 0x7fa9dc146d30 2024-04-25T10:13:43.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.782+0000 7fa9e1400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9dc0cab30 con 0x7fa9dc0cb960 2024-04-25T10:13:43.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.783+0000 7fa9e1400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0c9f20 con 0x7fa9dc146410 2024-04-25T10:13:43.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.783+0000 7fa9e0a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa9dc146d30 0x7fa9dc161510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36634/0 (socket says 172.21.15.119:36634) 2024-04-25T10:13:43.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.783+0000 7fa9e0a00640 1 -- 172.21.15.119:0/27035472 learned_addr learned my addr 172.21.15.119:0/27035472 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:43.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.783+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/27035472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9dc146410 0x7fa9dc0cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.783+0000 7fa9d3400640 1 --2- 172.21.15.119:0/27035472 >> v2:172.21.15.132:3300/0 conn(0x7fa9dc0cb960 0x7fa9dc1467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.783+0000 7fa9db400640 1 -- 172.21.15.119:0/27035472 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714221097 0 0) 0x7fa9dc0c9c40 con 0x7fa9dc146d30 2024-04-25T10:13:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.783+0000 7fa9db400640 1 -- 172.21.15.119:0/27035472 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c0003520 con 0x7fa9dc146d30 2024-04-25T10:13:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9db400640 1 -- 172.21.15.119:0/27035472 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3479372341 0 0) 0x7fa9c0003520 con 0x7fa9dc146d30 2024-04-25T10:13:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9db400640 1 -- 172.21.15.119:0/27035472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9dc146410 msgr2=0x7fa9dc0cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9db400640 1 --2- 172.21.15.119:0/27035472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9dc146410 0x7fa9dc0cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9db400640 1 -- 172.21.15.119:0/27035472 >> v2:172.21.15.132:3300/0 conn(0x7fa9dc0cb960 msgr2=0x7fa9dc1467f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9db400640 1 --2- 172.21.15.119:0/27035472 >> v2:172.21.15.132:3300/0 conn(0x7fa9dc0cb960 0x7fa9dc1467f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9db400640 1 -- 172.21.15.119:0/27035472 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc0ca200 con 0x7fa9dc146d30 2024-04-25T10:13:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9db400640 1 -- 172.21.15.119:0/27035472 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fa9d40019f0 con 0x7fa9dc146d30 2024-04-25T10:13:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9db400640 1 -- 172.21.15.119:0/27035472 >> v1:172.21.15.119:6789/0 conn(0x7fa9dc146d30 legacy=0x7fa9dc161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9db400640 1 --2- 172.21.15.119:0/27035472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c00039b0 0x7fa9c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9db400640 1 --2- 172.21.15.119:0/27035472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9dc146410 0x7fa9c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9db400640 1 --2- 172.21.15.119:0/27035472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c0004d40 0x7fa9c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9db400640 1 -- 172.21.15.119:0/27035472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0cab30 con 0x7fa9c00039b0 2024-04-25T10:13:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9db400640 1 -- 172.21.15.119:0/27035472 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0c9f20 con 0x7fa9c0004d40 2024-04-25T10:13:43.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9db400640 1 -- 172.21.15.119:0/27035472 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c000b9c0 con 0x7fa9dc146410 2024-04-25T10:13:43.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.784+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/27035472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c00039b0 0x7fa9c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.785+0000 7fa9d3400640 1 --2- 172.21.15.119:0/27035472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9dc146410 0x7fa9c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.785+0000 7fa9e0a00640 1 --2- 172.21.15.119:0/27035472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c0004d40 0x7fa9c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.785+0000 7fa9dbe00640 1 -- 172.21.15.119:0/27035472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9dc146410 msgr2=0x7fa9c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.785+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/27035472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9dc146410 0x7fa9c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.785+0000 7fa9dbe00640 1 -- 172.21.15.119:0/27035472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c0004d40 msgr2=0x7fa9c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.785+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/27035472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c0004d40 0x7fa9c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.785+0000 7fa9dbe00640 1 -- 172.21.15.119:0/27035472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c0003520 con 0x7fa9c00039b0 2024-04-25T10:13:43.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.785+0000 7fa9e0a00640 1 --2- 172.21.15.119:0/27035472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c0004d40 0x7fa9c00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:43.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.786+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/27035472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c00039b0 0x7fa9c0003dd0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009ac0 tx=0x7fa9c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8645db6090e767 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.786+0000 7fa9db400640 1 -- 172.21.15.119:0/27035472 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc036070 con 0x7fa9c00039b0 2024-04-25T10:13:43.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.786+0000 7fa9db400640 1 -- 172.21.15.119:0/27035472 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc031400 con 0x7fa9c00039b0 2024-04-25T10:13:43.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.786+0000 7fa9db400640 1 -- 172.21.15.119:0/27035472 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc009690 con 0x7fa9c00039b0 2024-04-25T10:13:43.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.787+0000 7fa9e1400640 1 -- 172.21.15.119:0/27035472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c00039b0 msgr2=0x7fa9c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.787+0000 7fa9e1400640 1 --2- 172.21.15.119:0/27035472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9c00039b0 0x7fa9c0003dd0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009ac0 tx=0x7fa9c0005830 comp rx=0 tx=0).stop 2024-04-25T10:13:43.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.787+0000 7fa9e0a00640 1 -- 172.21.15.119:0/27035472 reap_dead start 2024-04-25T10:13:43.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.787+0000 7fa9e1400640 1 -- 172.21.15.119:0/27035472 shutdown_connections 2024-04-25T10:13:43.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.787+0000 7fa9e1400640 1 -- 172.21.15.119:0/27035472 >> 172.21.15.119:0/27035472 conn(0x7fa9dc06aba0 msgr2=0x7fa9dc06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:43.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.787+0000 7fa9e1400640 1 -- 172.21.15.119:0/27035472 shutdown_connections 2024-04-25T10:13:43.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.787+0000 7fa9e1400640 1 -- 172.21.15.119:0/27035472 wait complete. 2024-04-25T10:13:43.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.789+0000 7fa9e1400640 1 Processor -- start 2024-04-25T10:13:43.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.789+0000 7fa9e1400640 1 -- start start 2024-04-25T10:13:43.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.790+0000 7fa9e1400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c00039b0 0x7fa9dc15df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.790+0000 7fa9e1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9dc15ceb0 0x7fa9dc139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.790+0000 7fa9e1400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9dc160240 0x7fa9dc160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.790+0000 7fa9e1400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0cafc0 con 0x7fa9dc15ceb0 2024-04-25T10:13:43.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.790+0000 7fa9e1400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc0ca7c0 con 0x7fa9c00039b0 2024-04-25T10:13:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.790+0000 7fa9e1400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc06e1a0 con 0x7fa9dc160240 2024-04-25T10:13:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.790+0000 7fa9d3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9dc15ceb0 0x7fa9dc139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.790+0000 7fa9d3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9dc15ceb0 0x7fa9dc139920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49458/0 (socket says 172.21.15.119:49458) 2024-04-25T10:13:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.790+0000 7fa9d3400640 1 -- 172.21.15.119:0/3716879521 learned_addr learned my addr 172.21.15.119:0/3716879521 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.791+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c00039b0 0x7fa9dc15df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.791+0000 7fa9e0a00640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9dc160240 0x7fa9dc160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.791+0000 7fa9d3400640 1 -- 172.21.15.119:0/3716879521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9dc160240 msgr2=0x7fa9dc160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.791+0000 7fa9d3400640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9dc160240 0x7fa9dc160810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.791+0000 7fa9d3400640 1 -- 172.21.15.119:0/3716879521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c00039b0 msgr2=0x7fa9dc15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.791+0000 7fa9d3400640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c00039b0 0x7fa9dc15df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.791+0000 7fa9d3400640 1 -- 172.21.15.119:0/3716879521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc1e7790 con 0x7fa9dc15ceb0 2024-04-25T10:13:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.791+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c00039b0 0x7fa9dc15df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.791+0000 7fa9e0a00640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9dc160240 0x7fa9dc160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.791+0000 7fa9d3400640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9dc15ceb0 0x7fa9dc139920 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4011700 tx=0x7fa9c4011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.791+0000 7fa9d8c00640 1 -- 172.21.15.119:0/3716879521 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400f9c0 con 0x7fa9dc15ceb0 2024-04-25T10:13:43.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.792+0000 7fa9d8c00640 1 -- 172.21.15.119:0/3716879521 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c400fb60 con 0x7fa9dc15ceb0 2024-04-25T10:13:43.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.792+0000 7fa9e1400640 1 -- 172.21.15.119:0/3716879521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9dc1e7a70 con 0x7fa9dc15ceb0 2024-04-25T10:13:43.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.792+0000 7fa9d8c00640 1 -- 172.21.15.119:0/3716879521 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4015460 con 0x7fa9dc15ceb0 2024-04-25T10:13:43.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.792+0000 7fa9e1400640 1 -- 172.21.15.119:0/3716879521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9dc15e490 con 0x7fa9dc15ceb0 2024-04-25T10:13:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.794+0000 7fa9d8c00640 1 -- 172.21.15.119:0/3716879521 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9c4015600 con 0x7fa9dc15ceb0 2024-04-25T10:13:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.795+0000 7fa9d8c00640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa9b806b4d0 0x7fa9b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.795+0000 7fa9d8c00640 1 -- 172.21.15.119:0/3716879521 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa9c408ac40 con 0x7fa9dc15ceb0 2024-04-25T10:13:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.795+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa9b806b4d0 0x7fa9b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.796+0000 7fa9e57c1740 1 Processor -- start 2024-04-25T10:13:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.796+0000 7fa9e57c1740 1 -- start start 2024-04-25T10:13:43.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.796+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa9b806b4d0 0x7fa9b806d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc033000 tx=0x7fa9cc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.797+0000 7fa9e57c1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe39c20 0x5565bfdec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.797+0000 7fa9e57c1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 0x5565bfe1c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.797+0000 7fa9e57c1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe1c600 0x5565bfdf1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.797+0000 7fa9e57c1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bfd55920 con 0x5565bfe1c600 2024-04-25T10:13:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.797+0000 7fa9e57c1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bfcfeda0 con 0x5565bfe1cd40 2024-04-25T10:13:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.797+0000 7fa9e57c1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565bfd519c0 con 0x5565bfe39c20 2024-04-25T10:13:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.797+0000 7fa9e0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe1c600 0x5565bfdf1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.797+0000 7fa9e0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe1c600 0x5565bfdf1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49462/0 (socket says 172.21.15.119:49462) 2024-04-25T10:13:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.797+0000 7fa9e0a00640 1 -- 172.21.15.119:0/980662940 learned_addr learned my addr 172.21.15.119:0/980662940 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.798+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/980662940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe39c20 0x5565bfdec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.798+0000 7fa9d3400640 1 --2- 172.21.15.119:0/980662940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 0x5565bfe1c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.798+0000 7fa9e0a00640 1 -- 172.21.15.119:0/980662940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe39c20 msgr2=0x5565bfdec5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.798+0000 7fa9e0a00640 1 --2- 172.21.15.119:0/980662940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe39c20 0x5565bfdec5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.798+0000 7fa9e0a00640 1 -- 172.21.15.119:0/980662940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 msgr2=0x5565bfe1c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.798+0000 7fa9e0a00640 1 --2- 172.21.15.119:0/980662940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 0x5565bfe1c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.798+0000 7fa9e0a00640 1 -- 172.21.15.119:0/980662940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565bfd0b290 con 0x5565bfe1c600 2024-04-25T10:13:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.798+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/980662940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe39c20 0x5565bfdec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.798+0000 7fa9e0a00640 1 --2- 172.21.15.119:0/980662940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe1c600 0x5565bfdf1410 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc072390 tx=0x7fa9d4001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56a33d6a93c765e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.798+0000 7fa9cbe00640 1 -- 172.21.15.119:0/980662940 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4018070 con 0x5565bfe1c600 2024-04-25T10:13:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.798+0000 7fa9cbe00640 1 -- 172.21.15.119:0/980662940 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4004d60 con 0x5565bfe1c600 2024-04-25T10:13:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.798+0000 7fa9cbe00640 1 -- 172.21.15.119:0/980662940 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4007270 con 0x5565bfe1c600 2024-04-25T10:13:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.799+0000 7fa9e57c1740 1 -- 172.21.15.119:0/980662940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe1c600 msgr2=0x5565bfdf1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.799+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/980662940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe1c600 0x5565bfdf1410 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc072390 tx=0x7fa9d4001b20 comp rx=0 tx=0).stop 2024-04-25T10:13:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.799+0000 7fa9e57c1740 1 -- 172.21.15.119:0/980662940 shutdown_connections 2024-04-25T10:13:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.799+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/980662940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe1c600 0x5565bfdf1410 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.799+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/980662940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 0x5565bfe1c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.799+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/980662940 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe39c20 0x5565bfdec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.799+0000 7fa9e57c1740 1 -- 172.21.15.119:0/980662940 >> 172.21.15.119:0/980662940 conn(0x5565bfdee820 msgr2=0x5565bfe370c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.799+0000 7fa9e57c1740 1 -- 172.21.15.119:0/980662940 shutdown_connections 2024-04-25T10:13:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.799+0000 7fa9e57c1740 1 -- 172.21.15.119:0/980662940 wait complete. 2024-04-25T10:13:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.800+0000 7fa9e57c1740 1 -- start start 2024-04-25T10:13:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.800+0000 7fa9e57c1740 1 Processor -- start 2024-04-25T10:13:43.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.801+0000 7fa9e57c1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 0x5565bfe35f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.801+0000 7fa9e57c1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe1e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.801+0000 7fa9e57c1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe36460 0x5565bfe1e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.801+0000 7fa9e57c1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bfdf09d0 con 0x5565bfe39c20 2024-04-25T10:13:43.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.801+0000 7fa9e57c1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bfcf4260 con 0x5565bfe1cd40 2024-04-25T10:13:43.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.801+0000 7fa9e57c1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565bfdefdc0 con 0x5565bfe36460 2024-04-25T10:13:43.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.801+0000 7fa9e57c1740 10 client.0 _ll_register_callbacks cb 0x5565bfd11bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:43.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.801+0000 7fa9d3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe1e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.802+0000 7fa9d3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe1e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49468/0 (socket says 172.21.15.119:49468) 2024-04-25T10:13:43.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.802+0000 7fa9d3400640 1 -- 172.21.15.119:0/2501719738 learned_addr learned my addr 172.21.15.119:0/2501719738 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:43.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.802+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/2501719738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 0x5565bfe35f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.802+0000 7fa9e0a00640 1 --2- 172.21.15.119:0/2501719738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe36460 0x5565bfe1e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.802+0000 7fa9d3400640 1 -- 172.21.15.119:0/2501719738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe36460 msgr2=0x5565bfe1e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.802+0000 7fa9d3400640 1 --2- 172.21.15.119:0/2501719738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe36460 0x5565bfe1e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.802+0000 7fa9d3400640 1 -- 172.21.15.119:0/2501719738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 msgr2=0x5565bfe35f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.802+0000 7fa9d3400640 1 --2- 172.21.15.119:0/2501719738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 0x5565bfe35f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.802+0000 7fa9d3400640 1 -- 172.21.15.119:0/2501719738 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565bfb367b0 con 0x5565bfe39c20 2024-04-25T10:13:43.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.802+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/2501719738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 0x5565bfe35f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:43.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.802+0000 7fa9d3400640 1 --2- 172.21.15.119:0/2501719738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe1e190 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4089010 tx=0x7fa9c4059a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628c1eef3056eb6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.802+0000 7fa9c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:13:43.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.802+0000 7fa9c8c00640 1 -- 172.21.15.119:0/2501719738 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c40619d0 con 0x5565bfe39c20 2024-04-25T10:13:43.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.803+0000 7fa9c8c00640 1 -- 172.21.15.119:0/2501719738 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa9c4061540 con 0x5565bfe39c20 2024-04-25T10:13:43.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.803+0000 7fa9e57c1740 1 -- 172.21.15.119:0/2501719738 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9d4004d60 con 0x5565bfe39c20 2024-04-25T10:13:43.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.803+0000 7fa9c8c00640 1 -- 172.21.15.119:0/2501719738 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9c4061740 con 0x5565bfe39c20 2024-04-25T10:13:43.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.803+0000 7fa9e57c1740 10 client.7467 fetch_fsmap learned FSMap version 67 2024-04-25T10:13:43.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.803+0000 7fa9e57c1740 1 -- 172.21.15.119:0/2501719738 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x5565bfb96f40 con 0x5565bfe39c20 2024-04-25T10:13:43.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.803+0000 7fa9c8c00640 1 -- 172.21.15.119:0/2501719738 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4004d60 con 0x5565bfe39c20 2024-04-25T10:13:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.803+0000 7fa9c8c00640 1 -- 172.21.15.119:0/2501719738 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa9c4058710 con 0x5565bfe39c20 2024-04-25T10:13:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.803+0000 7fa9e57c1740 10 client.7467 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:13:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.803+0000 7fa9e57c1740 10 client.7467 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:13:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.803+0000 7fa9e57c1740 20 client.7467 populate_metadata read hostname 'smithi119' 2024-04-25T10:13:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.803+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/2501719738 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5565bfe1ee00 0x5565bfe1f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.803+0000 7fa9e57c1740 4 client.7467 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.803+0000 7fa9e57c1740 1 -- 172.21.15.119:0/2501719738 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565bfd62e60 con 0x5565bfe1ee00 2024-04-25T10:13:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.804+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/2501719738 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5565bfe1ee00 0x5565bfe1f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.804+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/2501719738 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5565bfe1ee00 0x5565bfe1f270 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f87f255664dfdf89 server_cookie=3148e3730b764e18 in_seq=0 out_seq=0 2024-04-25T10:13:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.804+0000 7fa9c8c00640 10 client.7467 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:13:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.805+0000 7fa9c8c00640 1 -- 172.21.15.119:0/2501719738 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565bfd62e60 con 0x5565bfe1ee00 2024-04-25T10:13:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.805+0000 7fa9c8c00640 10 client.7467 handle_command_reply: tid=0 2024-04-25T10:13:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.805+0000 7fa9e57c1740 1 client.7467 shutdown 2024-04-25T10:13:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.805+0000 7fa9e57c1740 1 -- 172.21.15.119:0/2501719738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 msgr2=0x5565bfe1e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.805+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/2501719738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe1e190 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4089010 tx=0x7fa9c4059a80 comp rx=0 tx=0).stop 2024-04-25T10:13:43.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.805+0000 7fa9e57c1740 1 -- 172.21.15.119:0/2501719738 shutdown_connections 2024-04-25T10:13:43.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.805+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/2501719738 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5565bfe1ee00 0x5565bfe1f270 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.805+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/2501719738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe36460 0x5565bfe1e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.805+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/2501719738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe1e190 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.805+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/2501719738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 0x5565bfe35f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.805+0000 7fa9e57c1740 1 -- 172.21.15.119:0/2501719738 >> 172.21.15.119:0/2501719738 conn(0x5565bfe1c600 msgr2=0x5565bfe19310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:43.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.805+0000 7fa9c8c00640 0 client.7467 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:13:43.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.805+0000 7fa9e57c1740 1 -- 172.21.15.119:0/2501719738 shutdown_connections 2024-04-25T10:13:43.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.805+0000 7fa9e57c1740 1 -- 172.21.15.119:0/2501719738 wait complete. 2024-04-25T10:13:43.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.806+0000 7fa9e57c1740 20 client.7467 trim_cache size 0 max 16384 2024-04-25T10:13:43.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.822+0000 7fa9e57c1740 1 Processor -- start 2024-04-25T10:13:43.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.822+0000 7fa9e57c1740 1 -- start start 2024-04-25T10:13:43.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.823+0000 7fa9e57c1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 0x5565bfe1e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.823+0000 7fa9e57c1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe36460 0x5565bfe34c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.823+0000 7fa9e57c1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe35190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.823+0000 7fa9e57c1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc200690 con 0x5565bfe39c20 2024-04-25T10:13:43.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.823+0000 7fa9e57c1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc1ff850 con 0x5565bfe1cd40 2024-04-25T10:13:43.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.823+0000 7fa9e57c1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc1fec80 con 0x5565bfe36460 2024-04-25T10:13:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.823+0000 7fa9e0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe35190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.823+0000 7fa9e0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe35190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49484/0 (socket says 172.21.15.119:49484) 2024-04-25T10:13:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.823+0000 7fa9e0a00640 1 -- 172.21.15.119:0/728177949 learned_addr learned my addr 172.21.15.119:0/728177949 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.824+0000 7fa9e0a00640 1 -- 172.21.15.119:0/728177949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe36460 msgr2=0x5565bfe34c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.824+0000 7fa9e0a00640 1 --2- 172.21.15.119:0/728177949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe36460 0x5565bfe34c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.824+0000 7fa9e0a00640 1 -- 172.21.15.119:0/728177949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 msgr2=0x5565bfe1e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.824+0000 7fa9e0a00640 1 --2- 172.21.15.119:0/728177949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 0x5565bfe1e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.824+0000 7fa9e0a00640 1 -- 172.21.15.119:0/728177949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565bfa2a0a0 con 0x5565bfe39c20 2024-04-25T10:13:43.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.824+0000 7fa9e0a00640 1 --2- 172.21.15.119:0/728177949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe35190 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4001370 tx=0x7fa9d4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f1f9086063ba48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.824+0000 7fa9cb400640 1 -- 172.21.15.119:0/728177949 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4018070 con 0x5565bfe39c20 2024-04-25T10:13:43.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.824+0000 7fa9cb400640 1 -- 172.21.15.119:0/728177949 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4023070 con 0x5565bfe39c20 2024-04-25T10:13:43.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.824+0000 7fa9cb400640 1 -- 172.21.15.119:0/728177949 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40072b0 con 0x5565bfe39c20 2024-04-25T10:13:43.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.825+0000 7fa9e57c1740 1 -- 172.21.15.119:0/728177949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 msgr2=0x5565bfe35190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.825+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/728177949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe35190 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4001370 tx=0x7fa9d4011000 comp rx=0 tx=0).stop 2024-04-25T10:13:43.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.825+0000 7fa9e57c1740 1 -- 172.21.15.119:0/728177949 shutdown_connections 2024-04-25T10:13:43.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.825+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/728177949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe35190 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.825+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/728177949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe36460 0x5565bfe34c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.825+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/728177949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe1cd40 0x5565bfe1e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.825+0000 7fa9e57c1740 1 -- 172.21.15.119:0/728177949 >> 172.21.15.119:0/728177949 conn(0x5565bfe1c600 msgr2=0x5565bfdec410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:43.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.825+0000 7fa9e57c1740 1 -- 172.21.15.119:0/728177949 shutdown_connections 2024-04-25T10:13:43.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.825+0000 7fa9e57c1740 1 -- 172.21.15.119:0/728177949 wait complete. 2024-04-25T10:13:43.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.826+0000 7fa9e57c1740 1 -- start start 2024-04-25T10:13:43.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.826+0000 7fa9e57c1740 1 Processor -- start 2024-04-25T10:13:43.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.827+0000 7fa9e57c1740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe1cd40 0x5565bfeaa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.827+0000 7fa9e57c1740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe36460 0x5565bfeae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.827+0000 7fa9e57c1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe1e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.827+0000 7fa9e57c1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc1ff250 con 0x5565bfe39c20 2024-04-25T10:13:43.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.827+0000 7fa9e57c1740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565bfeaedf0 con 0x5565bfe36460 2024-04-25T10:13:43.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.827+0000 7fa9e57c1740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565bfe16db0 con 0x5565bfe1cd40 2024-04-25T10:13:43.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.827+0000 7fa9e57c1740 10 client.0 _ll_register_callbacks cb 0x5565bfe201b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:43.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.827+0000 7fa9e0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe1e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.827+0000 7fa9e0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe1e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49494/0 (socket says 172.21.15.119:49494) 2024-04-25T10:13:43.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.827+0000 7fa9e0a00640 1 -- 172.21.15.119:0/1552963777 learned_addr learned my addr 172.21.15.119:0/1552963777 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:43.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.828+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/1552963777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe1cd40 0x5565bfeaa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.828+0000 7fa9d3400640 1 --2- 172.21.15.119:0/1552963777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe36460 0x5565bfeae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.828+0000 7fa9e0a00640 1 -- 172.21.15.119:0/1552963777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe1cd40 msgr2=0x5565bfeaa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.828+0000 7fa9e0a00640 1 --2- 172.21.15.119:0/1552963777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe1cd40 0x5565bfeaa750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.828+0000 7fa9e0a00640 1 -- 172.21.15.119:0/1552963777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe36460 msgr2=0x5565bfeae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.828+0000 7fa9e0a00640 1 --2- 172.21.15.119:0/1552963777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe36460 0x5565bfeae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.828+0000 7fa9e0a00640 1 -- 172.21.15.119:0/1552963777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565bfdebb90 con 0x5565bfe39c20 2024-04-25T10:13:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.828+0000 7fa9e0a00640 1 --2- 172.21.15.119:0/1552963777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe1e460 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4007c40 tx=0x7fa9dc0c9e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3203b1c5d90a6bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.828+0000 7fa9c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:13:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.828+0000 7fa9c8c00640 1 -- 172.21.15.119:0/1552963777 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4018040 con 0x5565bfe39c20 2024-04-25T10:13:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.828+0000 7fa9c8c00640 1 -- 172.21.15.119:0/1552963777 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa9d4013920 con 0x5565bfe39c20 2024-04-25T10:13:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.828+0000 7fa9e57c1740 1 -- 172.21.15.119:0/1552963777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9d4023070 con 0x5565bfe39c20 2024-04-25T10:13:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.829+0000 7fa9c8c00640 1 -- 172.21.15.119:0/1552963777 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9d403a050 con 0x5565bfe39c20 2024-04-25T10:13:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.829+0000 7fa9e57c1740 10 client.7473 fetch_fsmap learned FSMap version 67 2024-04-25T10:13:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.829+0000 7fa9e57c1740 1 -- 172.21.15.119:0/1552963777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x5565bfeae8a0 con 0x5565bfe39c20 2024-04-25T10:13:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.829+0000 7fa9c8c00640 1 -- 172.21.15.119:0/1552963777 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4023070 con 0x5565bfe39c20 2024-04-25T10:13:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.829+0000 7fa9c8c00640 1 -- 172.21.15.119:0/1552963777 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa9d403d020 con 0x5565bfe39c20 2024-04-25T10:13:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.829+0000 7fa9e57c1740 10 client.7473 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:13:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.829+0000 7fa9e57c1740 10 client.7473 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:13:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.829+0000 7fa9e57c1740 20 client.7473 populate_metadata read hostname 'smithi119' 2024-04-25T10:13:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.829+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/1552963777 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5565bfe35f20 0x5565bfe38030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.829+0000 7fa9e57c1740 4 client.7473 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-25T10:13:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.829+0000 7fa9e57c1740 1 -- 172.21.15.119:0/1552963777 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) v1 -- 0x5565bfd5d1b0 con 0x5565bfe35f20 2024-04-25T10:13:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.829+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/1552963777 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5565bfe35f20 0x5565bfe38030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.830+0000 7fa9dbe00640 1 --2- 172.21.15.119:0/1552963777 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5565bfe35f20 0x5565bfe38030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81686c55d7a1617f server_cookie=7cc420aa6d6a7902 in_seq=0 out_seq=0 2024-04-25T10:13:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.830+0000 7fa9c8c00640 10 client.7473 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:13:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.830+0000 7fa9c8c00640 1 -- 172.21.15.119:0/1552963777 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5565bfd5d1b0 con 0x5565bfe35f20 2024-04-25T10:13:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.830+0000 7fa9c8c00640 10 client.7473 handle_command_reply: tid=0 2024-04-25T10:13:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.830+0000 7fa9e57c1740 1 client.7473 shutdown 2024-04-25T10:13:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.830+0000 7fa9e57c1740 1 -- 172.21.15.119:0/1552963777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 msgr2=0x5565bfe1e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.830+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/1552963777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe1e460 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4007c40 tx=0x7fa9dc0c9e10 comp rx=0 tx=0).stop 2024-04-25T10:13:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.831+0000 7fa9e57c1740 1 -- 172.21.15.119:0/1552963777 shutdown_connections 2024-04-25T10:13:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.831+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/1552963777 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5565bfe35f20 0x5565bfe38030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.831+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/1552963777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565bfe39c20 0x5565bfe1e460 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.831+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/1552963777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5565bfe36460 0x5565bfeae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.831+0000 7fa9e57c1740 1 --2- 172.21.15.119:0/1552963777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5565bfe1cd40 0x5565bfeaa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.831+0000 7fa9e57c1740 1 -- 172.21.15.119:0/1552963777 >> 172.21.15.119:0/1552963777 conn(0x5565bfe1c600 msgr2=0x5565bfe36ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:43.839 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:13:43.831+0000 7f5363c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-25T10:13:32.156889+0000 1=1+0) n(v0 rc2024-04-25T10:13:32.264886+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x561f3d610100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:13:32.156889+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:13:32.264886+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T10:13:43.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.831+0000 7fa9c8c00640 0 client.7473 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:13:43.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.831+0000 7fa9e57c1740 1 -- 172.21.15.119:0/1552963777 shutdown_connections 2024-04-25T10:13:43.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.831+0000 7fa9e57c1740 1 -- 172.21.15.119:0/1552963777 wait complete. 2024-04-25T10:13:43.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.831+0000 7fa9e57c1740 20 client.7473 trim_cache size 0 max 16384 2024-04-25T10:13:43.840 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:13:43.840 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2024-04-25T10:13:43.840 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "c6494c6d-9eb1-4876-b0f2-bcf84b895017", 2024-04-25T10:13:43.840 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2024-04-25T10:13:43.840 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:13:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.832+0000 7fa9e1400640 1 -- 172.21.15.119:0/3716879521 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa9b806b4d0 msgr2=0x7fa9b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.832+0000 7fa9e1400640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa9b806b4d0 0x7fa9b806d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc033000 tx=0x7fa9cc008030 comp rx=0 tx=0).stop 2024-04-25T10:13:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.832+0000 7fa9e1400640 1 -- 172.21.15.119:0/3716879521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9dc15ceb0 msgr2=0x7fa9dc139920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.832+0000 7fa9e1400640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9dc15ceb0 0x7fa9dc139920 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4011700 tx=0x7fa9c4011bc0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.832+0000 7fa9e1400640 1 -- 172.21.15.119:0/3716879521 shutdown_connections 2024-04-25T10:13:43.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.832+0000 7fa9e1400640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa9b806b4d0 0x7fa9b806d990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.832+0000 7fa9e1400640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa9dc160240 0x7fa9dc160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.832+0000 7fa9e1400640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9dc15ceb0 0x7fa9dc139920 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.832+0000 7fa9e1400640 1 --2- 172.21.15.119:0/3716879521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9c00039b0 0x7fa9dc15df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.832+0000 7fa9e1400640 1 -- 172.21.15.119:0/3716879521 >> 172.21.15.119:0/3716879521 conn(0x7fa9dc06aba0 msgr2=0x7fa9dc0cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:43.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.832+0000 7fa9e1400640 1 -- 172.21.15.119:0/3716879521 shutdown_connections 2024-04-25T10:13:43.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:43.832+0000 7fa9e1400640 1 -- 172.21.15.119:0/3716879521 wait complete. 2024-04-25T10:13:43.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:43.856 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:13:44.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.044+0000 7f7d7ae00640 1 Processor -- start 2024-04-25T10:13:44.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.044+0000 7f7d7ae00640 1 -- start start 2024-04-25T10:13:44.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.046+0000 7f7d7ae00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7d7415ef60 0x7f7d740b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.046+0000 7f7d7ae00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d7406aba0 0x7f7d7415d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.046+0000 7f7d7ae00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7406b3d0 con 0x7f7d740cfad0 2024-04-25T10:13:44.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.046+0000 7f7d7ae00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7406b6b0 con 0x7f7d7415ef60 2024-04-25T10:13:44.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.046+0000 7f7d7ae00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7415ead0 con 0x7f7d7406aba0 2024-04-25T10:13:44.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.046+0000 7f7d73e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7d740cfad0 0x7f7d740cfeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36650/0 (socket says 172.21.15.119:36650) 2024-04-25T10:13:44.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.046+0000 7f7d73e00640 1 -- 172.21.15.119:0/898355083 learned_addr learned my addr 172.21.15.119:0/898355083 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:13:44.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.047+0000 7f7d73400640 1 --2- 172.21.15.119:0/898355083 >> v2:172.21.15.132:3300/0 conn(0x7f7d7415ef60 0x7f7d740b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.047+0000 7f7d78e00640 1 --2- 172.21.15.119:0/898355083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d7406aba0 0x7f7d7415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.047+0000 7f7d72a00640 1 -- 172.21.15.119:0/898355083 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853879334 0 0) 0x7f7d7406b3d0 con 0x7f7d740cfad0 2024-04-25T10:13:44.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.047+0000 7f7d72a00640 1 -- 172.21.15.119:0/898355083 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d58003520 con 0x7f7d740cfad0 2024-04-25T10:13:44.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.047+0000 7f7d72a00640 1 -- 172.21.15.119:0/898355083 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1134210640 0 0) 0x7f7d58003520 con 0x7f7d740cfad0 2024-04-25T10:13:44.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.047+0000 7f7d72a00640 1 -- 172.21.15.119:0/898355083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d7406aba0 msgr2=0x7f7d7415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.047+0000 7f7d72a00640 1 --2- 172.21.15.119:0/898355083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d7406aba0 0x7f7d7415d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.047+0000 7f7d72a00640 1 -- 172.21.15.119:0/898355083 >> v2:172.21.15.132:3300/0 conn(0x7f7d7415ef60 msgr2=0x7f7d740b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.047+0000 7f7d72a00640 1 --2- 172.21.15.119:0/898355083 >> v2:172.21.15.132:3300/0 conn(0x7f7d7415ef60 0x7f7d740b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.047+0000 7f7d72a00640 1 -- 172.21.15.119:0/898355083 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7415e480 con 0x7f7d740cfad0 2024-04-25T10:13:44.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.047+0000 7f7d72a00640 1 -- 172.21.15.119:0/898355083 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f7d64001670 con 0x7f7d740cfad0 2024-04-25T10:13:44.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.047+0000 7f7d72a00640 1 -- 172.21.15.119:0/898355083 >> v1:172.21.15.119:6789/0 conn(0x7f7d740cfad0 legacy=0x7f7d740cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d72a00640 1 --2- 172.21.15.119:0/898355083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d7406aba0 0x7f7d580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d72a00640 1 --2- 172.21.15.119:0/898355083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58003f30 0x7f7d58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d72a00640 1 --2- 172.21.15.119:0/898355083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d58004d40 0x7f7d580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d72a00640 1 -- 172.21.15.119:0/898355083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7406b6b0 con 0x7f7d58003f30 2024-04-25T10:13:44.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d72a00640 1 -- 172.21.15.119:0/898355083 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7415ead0 con 0x7f7d58004d40 2024-04-25T10:13:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d72a00640 1 -- 172.21.15.119:0/898355083 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5800ba10 con 0x7f7d7406aba0 2024-04-25T10:13:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d73400640 1 --2- 172.21.15.119:0/898355083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58003f30 0x7f7d58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d73e00640 1 --2- 172.21.15.119:0/898355083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d7406aba0 0x7f7d580039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d78e00640 1 --2- 172.21.15.119:0/898355083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d58004d40 0x7f7d580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d73400640 1 -- 172.21.15.119:0/898355083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d7406aba0 msgr2=0x7f7d580039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d73400640 1 --2- 172.21.15.119:0/898355083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d7406aba0 0x7f7d580039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d73400640 1 -- 172.21.15.119:0/898355083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d58004d40 msgr2=0x7f7d580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d73400640 1 --2- 172.21.15.119:0/898355083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d58004d40 0x7f7d580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d73400640 1 -- 172.21.15.119:0/898355083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d58003520 con 0x7f7d58003f30 2024-04-25T10:13:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.048+0000 7f7d78e00640 1 --2- 172.21.15.119:0/898355083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d58004d40 0x7f7d580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.049+0000 7f7d73400640 1 --2- 172.21.15.119:0/898355083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58003f30 0x7f7d58004800 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009fc0 tx=0x7f7d58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fa2293c402319a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.050+0000 7f7d72a00640 1 -- 172.21.15.119:0/898355083 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c009c10 con 0x7f7d58003f30 2024-04-25T10:13:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.050+0000 7f7d72a00640 1 -- 172.21.15.119:0/898355083 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5c036070 con 0x7f7d58003f30 2024-04-25T10:13:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.050+0000 7f7d72a00640 1 -- 172.21.15.119:0/898355083 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c02d070 con 0x7f7d58003f30 2024-04-25T10:13:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.050+0000 7f7d7ae00640 1 -- 172.21.15.119:0/898355083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58003f30 msgr2=0x7f7d58004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.050+0000 7f7d7ae00640 1 --2- 172.21.15.119:0/898355083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58003f30 0x7f7d58004800 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009fc0 tx=0x7f7d58005830 comp rx=0 tx=0).stop 2024-04-25T10:13:44.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.050+0000 7f7d78e00640 1 -- 172.21.15.119:0/898355083 reap_dead start 2024-04-25T10:13:44.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.050+0000 7f7d7ae00640 1 -- 172.21.15.119:0/898355083 shutdown_connections 2024-04-25T10:13:44.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.050+0000 7f7d7ae00640 1 -- 172.21.15.119:0/898355083 >> 172.21.15.119:0/898355083 conn(0x7f7d74099a40 msgr2=0x7f7d740b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:44.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.050+0000 7f7d7ae00640 1 -- 172.21.15.119:0/898355083 shutdown_connections 2024-04-25T10:13:44.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.050+0000 7f7d7ae00640 1 -- 172.21.15.119:0/898355083 wait complete. 2024-04-25T10:13:44.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.052+0000 7f7d7ae00640 1 Processor -- start 2024-04-25T10:13:44.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.052+0000 7f7d7ae00640 1 -- start start 2024-04-25T10:13:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.053+0000 7f7d7ae00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d58003f30 0x7f7d74148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.053+0000 7f7d7ae00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d7415cb40 0x7f7d7415cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.053+0000 7f7d7ae00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d741f75a0 0x7f7d741f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.053+0000 7f7d7ae00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7406b1b0 con 0x7f7d7415cb40 2024-04-25T10:13:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.053+0000 7f7d7ae00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7406b030 con 0x7f7d58003f30 2024-04-25T10:13:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.053+0000 7f7d7ae00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d740afd50 con 0x7f7d741f75a0 2024-04-25T10:13:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.054+0000 7f7d73400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d7415cb40 0x7f7d7415cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.054+0000 7f7d73400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d7415cb40 0x7f7d7415cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49518/0 (socket says 172.21.15.119:49518) 2024-04-25T10:13:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.054+0000 7f7d73400640 1 -- 172.21.15.119:0/3011048853 learned_addr learned my addr 172.21.15.119:0/3011048853 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.054+0000 7f7d73e00640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d58003f30 0x7f7d74148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.054+0000 7f7d78e00640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d741f75a0 0x7f7d741f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.054+0000 7f7d73400640 1 -- 172.21.15.119:0/3011048853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d741f75a0 msgr2=0x7f7d741f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.054+0000 7f7d73400640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d741f75a0 0x7f7d741f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.054+0000 7f7d73400640 1 -- 172.21.15.119:0/3011048853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d58003f30 msgr2=0x7f7d74148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.054+0000 7f7d73400640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d58003f30 0x7f7d74148520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.054+0000 7f7d73400640 1 -- 172.21.15.119:0/3011048853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d74148e10 con 0x7f7d7415cb40 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.054+0000 7f7d73e00640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d58003f30 0x7f7d74148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.055+0000 7f7d73400640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d7415cb40 0x7f7d7415cfc0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00e230 tx=0x7f7d5c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.055+0000 7f7d6be00640 1 -- 172.21.15.119:0/3011048853 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c009c10 con 0x7f7d7415cb40 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.055+0000 7f7d6be00640 1 -- 172.21.15.119:0/3011048853 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5c003b90 con 0x7f7d7415cb40 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.055+0000 7f7d7ae00640 1 -- 172.21.15.119:0/3011048853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7414b0f0 con 0x7f7d7415cb40 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.055+0000 7f7d6be00640 1 -- 172.21.15.119:0/3011048853 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c004020 con 0x7f7d7415cb40 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.055+0000 7f7d7ae00640 1 -- 172.21.15.119:0/3011048853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7414b6b0 con 0x7f7d7415cb40 2024-04-25T10:13:44.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.057+0000 7f7d6be00640 1 -- 172.21.15.119:0/3011048853 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d5c005d10 con 0x7f7d7415cb40 2024-04-25T10:13:44.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.058+0000 7f7d6be00640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d5006b4d0 0x7f7d5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.058+0000 7f7d6be00640 1 -- 172.21.15.119:0/3011048853 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7d5c0ab790 con 0x7f7d7415cb40 2024-04-25T10:13:44.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.059+0000 7f7d73e00640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d5006b4d0 0x7f7d5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.059+0000 7f7d7db64740 1 Processor -- start 2024-04-25T10:13:44.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.059+0000 7f7d7db64740 1 -- start start 2024-04-25T10:13:44.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.059+0000 7f7d73e00640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d5006b4d0 0x7f7d5006d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7d740b1d40 tx=0x7f7d64009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.060+0000 7f7d7db64740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111923bb0 0x563111925d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.060+0000 7f7d7db64740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563111960280 0x563111923f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.060+0000 7f7d7db64740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56311194e670 0x56311194eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.060+0000 7f7d7db64740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563111887c70 con 0x563111923bb0 2024-04-25T10:13:44.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.060+0000 7f7d7db64740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563111830cf0 con 0x56311194e670 2024-04-25T10:13:44.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.060+0000 7f7d7db64740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563111883d10 con 0x563111960280 2024-04-25T10:13:44.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.060+0000 7f7d73e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111923bb0 0x563111925d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.061+0000 7f7d73e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111923bb0 0x563111925d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49526/0 (socket says 172.21.15.119:49526) 2024-04-25T10:13:44.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.061+0000 7f7d73e00640 1 -- 172.21.15.119:0/1827711658 learned_addr learned my addr 172.21.15.119:0/1827711658 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:44.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.061+0000 7f7d73e00640 1 -- 172.21.15.119:0/1827711658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563111960280 msgr2=0x563111923f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.061+0000 7f7d73e00640 1 --2- 172.21.15.119:0/1827711658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563111960280 0x563111923f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.061+0000 7f7d73e00640 1 -- 172.21.15.119:0/1827711658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56311194e670 msgr2=0x56311194eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.061+0000 7f7d78e00640 1 --2- 172.21.15.119:0/1827711658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56311194e670 0x56311194eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.061+0000 7f7d73e00640 1 --2- 172.21.15.119:0/1827711658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56311194e670 0x56311194eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.061+0000 7f7d73e00640 1 -- 172.21.15.119:0/1827711658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631116667d0 con 0x563111923bb0 2024-04-25T10:13:44.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.061+0000 7f7d73e00640 1 --2- 172.21.15.119:0/1827711658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111923bb0 0x563111925d10 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400fec0 tx=0x7f7d64012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5189ca31144f6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.061+0000 7f7d68c00640 1 -- 172.21.15.119:0/1827711658 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64026070 con 0x563111923bb0 2024-04-25T10:13:44.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.062+0000 7f7d68c00640 1 -- 172.21.15.119:0/1827711658 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d640053a0 con 0x563111923bb0 2024-04-25T10:13:44.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.062+0000 7f7d68c00640 1 -- 172.21.15.119:0/1827711658 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64005790 con 0x563111923bb0 2024-04-25T10:13:44.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.062+0000 7f7d7db64740 1 -- 172.21.15.119:0/1827711658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111923bb0 msgr2=0x563111925d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.062+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1827711658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111923bb0 0x563111925d10 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400fec0 tx=0x7f7d64012000 comp rx=0 tx=0).stop 2024-04-25T10:13:44.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.062+0000 7f7d7db64740 1 -- 172.21.15.119:0/1827711658 shutdown_connections 2024-04-25T10:13:44.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.062+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1827711658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56311194e670 0x56311194eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.062+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1827711658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563111960280 0x563111923f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.062+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1827711658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111923bb0 0x563111925d10 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.062+0000 7f7d7db64740 1 -- 172.21.15.119:0/1827711658 >> 172.21.15.119:0/1827711658 conn(0x5631118eba30 msgr2=0x563111969670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:44.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.062+0000 7f7d7db64740 1 -- 172.21.15.119:0/1827711658 shutdown_connections 2024-04-25T10:13:44.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.063+0000 7f7d7db64740 1 -- 172.21.15.119:0/1827711658 wait complete. 2024-04-25T10:13:44.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.063+0000 7f7d7db64740 1 -- start start 2024-04-25T10:13:44.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.063+0000 7f7d7db64740 1 Processor -- start 2024-04-25T10:13:44.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.064+0000 7f7d7db64740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563111923bb0 0x5631119610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.064+0000 7f7d7db64740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56311194e670 0x563111961a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.064+0000 7f7d7db64740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111960280 0x563111961f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d7db64740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631118263a0 con 0x563111960280 2024-04-25T10:13:44.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d7db64740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56311194c1c0 con 0x563111923bb0 2024-04-25T10:13:44.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d7db64740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56311194fd20 con 0x56311194e670 2024-04-25T10:13:44.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d7db64740 10 client.0 _ll_register_callbacks cb 0x563111843b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:44.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d78e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111960280 0x563111961f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d78e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111960280 0x563111961f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49538/0 (socket says 172.21.15.119:49538) 2024-04-25T10:13:44.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d78e00640 1 -- 172.21.15.119:0/1383481698 learned_addr learned my addr 172.21.15.119:0/1383481698 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d73400640 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56311194e670 0x563111961a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d73e00640 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563111923bb0 0x5631119610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d78e00640 1 -- 172.21.15.119:0/1383481698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56311194e670 msgr2=0x563111961a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d78e00640 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56311194e670 0x563111961a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d78e00640 1 -- 172.21.15.119:0/1383481698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563111923bb0 msgr2=0x5631119610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d78e00640 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563111923bb0 0x5631119610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d78e00640 1 -- 172.21.15.119:0/1383481698 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56311159c3f0 con 0x563111960280 2024-04-25T10:13:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d73e00640 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563111923bb0 0x5631119610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.065+0000 7f7d73400640 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56311194e670 0x563111961a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.066+0000 7f7d78e00640 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111960280 0x563111961f90 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c006fd0 tx=0x7f7d6c0115d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4cc132a10eb770 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.066+0000 7f7d61600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:13:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.066+0000 7f7d61600640 1 -- 172.21.15.119:0/1383481698 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c019070 con 0x563111960280 2024-04-25T10:13:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.066+0000 7f7d61600640 1 -- 172.21.15.119:0/1383481698 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7d6c015400 con 0x563111960280 2024-04-25T10:13:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.066+0000 7f7d7db64740 1 -- 172.21.15.119:0/1383481698 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d640053a0 con 0x563111960280 2024-04-25T10:13:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.066+0000 7f7d61600640 1 -- 172.21.15.119:0/1383481698 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d6c015600 con 0x563111960280 2024-04-25T10:13:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.066+0000 7f7d7db64740 10 client.7485 fetch_fsmap learned FSMap version 67 2024-04-25T10:13:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.066+0000 7f7d7db64740 1 -- 172.21.15.119:0/1383481698 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x56311155c0f0 con 0x563111960280 2024-04-25T10:13:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.066+0000 7f7d61600640 1 -- 172.21.15.119:0/1383481698 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d640053a0 con 0x563111960280 2024-04-25T10:13:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.066+0000 7f7d61600640 1 -- 172.21.15.119:0/1383481698 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7d6c027080 con 0x563111960280 2024-04-25T10:13:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.067+0000 7f7d7db64740 10 client.7485 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:13:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.067+0000 7f7d7db64740 10 client.7485 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:13:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.067+0000 7f7d7db64740 20 client.7485 populate_metadata read hostname 'smithi119' 2024-04-25T10:13:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.067+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x563111961600 0x56311194efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.067+0000 7f7d7db64740 4 client.7485 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.067+0000 7f7d7db64740 1 -- 172.21.15.119:0/1383481698 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631118951b0 con 0x563111961600 2024-04-25T10:13:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.067+0000 7f7d73e00640 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x563111961600 0x56311194efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.067+0000 7f7d73e00640 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x563111961600 0x56311194efc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27d4e622754ed9a1 server_cookie=6225100655eb60a6 in_seq=0 out_seq=0 2024-04-25T10:13:44.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.067+0000 7f7d61600640 10 client.7485 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:13:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.068+0000 7f7d61600640 1 -- 172.21.15.119:0/1383481698 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5631118951b0 con 0x563111961600 2024-04-25T10:13:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.068+0000 7f7d61600640 10 client.7485 handle_command_reply: tid=0 2024-04-25T10:13:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.068+0000 7f7d7db64740 1 client.7485 shutdown 2024-04-25T10:13:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.068+0000 7f7d7db64740 1 -- 172.21.15.119:0/1383481698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111960280 msgr2=0x563111961f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.068+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111960280 0x563111961f90 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c006fd0 tx=0x7f7d6c0115d0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.068+0000 7f7d7db64740 1 -- 172.21.15.119:0/1383481698 shutdown_connections 2024-04-25T10:13:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.068+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x563111961600 0x56311194efc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.068+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111960280 0x563111961f90 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.068+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56311194e670 0x563111961a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.068+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1383481698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563111923bb0 0x5631119610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.068+0000 7f7d61600640 0 client.7485 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:13:44.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.068+0000 7f7d7db64740 1 -- 172.21.15.119:0/1383481698 >> 172.21.15.119:0/1383481698 conn(0x5631118eba30 msgr2=0x56311194c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:44.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.069+0000 7f7d7db64740 1 -- 172.21.15.119:0/1383481698 shutdown_connections 2024-04-25T10:13:44.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.069+0000 7f7d7db64740 1 -- 172.21.15.119:0/1383481698 wait complete. 2024-04-25T10:13:44.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.069+0000 7f7d7db64740 20 client.7485 trim_cache size 0 max 16384 2024-04-25T10:13:44.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.085+0000 7f7d7db64740 1 Processor -- start 2024-04-25T10:13:44.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.085+0000 7f7d7db64740 1 -- start start 2024-04-25T10:13:44.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.086+0000 7f7d7db64740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563111923bb0 0x56311194efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.086+0000 7f7d7db64740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56311194e670 0x563111960a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.086+0000 7f7d7db64740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111960280 0x563111961bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.086+0000 7f7d7db64740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d74200800 con 0x563111960280 2024-04-25T10:13:44.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.086+0000 7f7d7db64740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d741ffa30 con 0x56311194e670 2024-04-25T10:13:44.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.086+0000 7f7d7db64740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7414aaa0 con 0x563111923bb0 2024-04-25T10:13:44.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.086+0000 7f7d78e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111960280 0x563111961bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.086+0000 7f7d78e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111960280 0x563111961bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49554/0 (socket says 172.21.15.119:49554) 2024-04-25T10:13:44.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.086+0000 7f7d78e00640 1 -- 172.21.15.119:0/3033570480 learned_addr learned my addr 172.21.15.119:0/3033570480 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:44.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.087+0000 7f7d78e00640 1 -- 172.21.15.119:0/3033570480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563111923bb0 msgr2=0x56311194efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.087+0000 7f7d78e00640 1 --2- 172.21.15.119:0/3033570480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563111923bb0 0x56311194efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.087+0000 7f7d78e00640 1 -- 172.21.15.119:0/3033570480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56311194e670 msgr2=0x563111960a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.087+0000 7f7d78e00640 1 --2- 172.21.15.119:0/3033570480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56311194e670 0x563111960a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.087+0000 7f7d78e00640 1 -- 172.21.15.119:0/3033570480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56311194c5c0 con 0x563111960280 2024-04-25T10:13:44.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.087+0000 7f7d78e00640 1 --2- 172.21.15.119:0/3033570480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111960280 0x563111961bd0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c008010 tx=0x7f7d6c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e486d7ab8f5ad69f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.087+0000 7f7d63e00640 1 -- 172.21.15.119:0/3033570480 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c019040 con 0x563111960280 2024-04-25T10:13:44.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.087+0000 7f7d63e00640 1 -- 172.21.15.119:0/3033570480 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6c010040 con 0x563111960280 2024-04-25T10:13:44.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.087+0000 7f7d63e00640 1 -- 172.21.15.119:0/3033570480 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c015b90 con 0x563111960280 2024-04-25T10:13:44.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.088+0000 7f7d7db64740 1 -- 172.21.15.119:0/3033570480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111960280 msgr2=0x563111961bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.088+0000 7f7d7db64740 1 --2- 172.21.15.119:0/3033570480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111960280 0x563111961bd0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c008010 tx=0x7f7d6c016000 comp rx=0 tx=0).stop 2024-04-25T10:13:44.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.088+0000 7f7d7db64740 1 -- 172.21.15.119:0/3033570480 shutdown_connections 2024-04-25T10:13:44.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.088+0000 7f7d7db64740 1 --2- 172.21.15.119:0/3033570480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111960280 0x563111961bd0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.088+0000 7f7d7db64740 1 --2- 172.21.15.119:0/3033570480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56311194e670 0x563111960a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.088+0000 7f7d7db64740 1 --2- 172.21.15.119:0/3033570480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563111923bb0 0x56311194efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.088+0000 7f7d7db64740 1 -- 172.21.15.119:0/3033570480 >> 172.21.15.119:0/3033570480 conn(0x5631118eba30 msgr2=0x56311194feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:44.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.088+0000 7f7d7db64740 1 -- 172.21.15.119:0/3033570480 shutdown_connections 2024-04-25T10:13:44.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.088+0000 7f7d7db64740 1 -- 172.21.15.119:0/3033570480 wait complete. 2024-04-25T10:13:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.089+0000 7f7d7db64740 1 -- start start 2024-04-25T10:13:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.089+0000 7f7d7db64740 1 Processor -- start 2024-04-25T10:13:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.090+0000 7f7d7db64740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111923bb0 0x563111965b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.090+0000 7f7d7db64740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56311194e670 0x56311194dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.090+0000 7f7d7db64740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563111960280 0x56311194e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.090+0000 7f7d7db64740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56311194c9a0 con 0x563111923bb0 2024-04-25T10:13:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.090+0000 7f7d7db64740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d741ff480 con 0x563111960280 2024-04-25T10:13:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.090+0000 7f7d7db64740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56311196d460 con 0x56311194e670 2024-04-25T10:13:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.090+0000 7f7d7db64740 10 client.0 _ll_register_callbacks cb 0x5631119245e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.090+0000 7f7d73e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111923bb0 0x563111965b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.090+0000 7f7d73e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111923bb0 0x563111965b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49556/0 (socket says 172.21.15.119:49556) 2024-04-25T10:13:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.090+0000 7f7d73e00640 1 -- 172.21.15.119:0/1449391772 learned_addr learned my addr 172.21.15.119:0/1449391772 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:13:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.091+0000 7f7d73e00640 1 -- 172.21.15.119:0/1449391772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56311194e670 msgr2=0x56311194dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.091+0000 7f7d73e00640 1 --2- 172.21.15.119:0/1449391772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56311194e670 0x56311194dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.091+0000 7f7d73e00640 1 -- 172.21.15.119:0/1449391772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563111960280 msgr2=0x56311194e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.091+0000 7f7d73e00640 1 --2- 172.21.15.119:0/1449391772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563111960280 0x56311194e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.091+0000 7f7d73e00640 1 -- 172.21.15.119:0/1449391772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563111961c70 con 0x563111923bb0 2024-04-25T10:13:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.091+0000 7f7d73e00640 1 --2- 172.21.15.119:0/1449391772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111923bb0 0x563111965b10 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400fe90 tx=0x7f7d64001110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271f6757e0f65138 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.091+0000 7f7d61600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:13:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.091+0000 7f7d61600640 1 -- 172.21.15.119:0/1449391772 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64026040 con 0x563111923bb0 2024-04-25T10:13:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.091+0000 7f7d61600640 1 -- 172.21.15.119:0/1449391772 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7d6402e440 con 0x563111923bb0 2024-04-25T10:13:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.091+0000 7f7d7db64740 1 -- 172.21.15.119:0/1449391772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d6c010040 con 0x563111923bb0 2024-04-25T10:13:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.091+0000 7f7d61600640 1 -- 172.21.15.119:0/1449391772 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d6403e050 con 0x563111923bb0 2024-04-25T10:13:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.092+0000 7f7d7db64740 10 client.7491 fetch_fsmap learned FSMap version 67 2024-04-25T10:13:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.092+0000 7f7d7db64740 1 -- 172.21.15.119:0/1449391772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x563111922370 con 0x563111923bb0 2024-04-25T10:13:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.092+0000 7f7d61600640 1 -- 172.21.15.119:0/1449391772 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6c010040 con 0x563111923bb0 2024-04-25T10:13:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.092+0000 7f7d61600640 1 -- 172.21.15.119:0/1449391772 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7d64041020 con 0x563111923bb0 2024-04-25T10:13:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.092+0000 7f7d7db64740 10 client.7491 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:13:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.092+0000 7f7d7db64740 10 client.7491 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:13:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.092+0000 7f7d7db64740 20 client.7491 populate_metadata read hostname 'smithi119' 2024-04-25T10:13:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.092+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1449391772 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5631119507d0 0x56311194a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.092+0000 7f7d7db64740 4 client.7491 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:13:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.092+0000 7f7d7db64740 1 -- 172.21.15.119:0/1449391772 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56311188f500 con 0x5631119507d0 2024-04-25T10:13:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.092+0000 7f7d73400640 1 --2- 172.21.15.119:0/1449391772 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5631119507d0 0x56311194a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d73400640 1 --2- 172.21.15.119:0/1449391772 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5631119507d0 0x56311194a8f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7afcb20a5c289cef server_cookie=86c6ebfec048dcb in_seq=0 out_seq=0 2024-04-25T10:13:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d61600640 10 client.7491 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:13:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d61600640 1 -- 172.21.15.119:0/1449391772 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+256 (crc 0 0 0) 0x56311188f500 con 0x5631119507d0 2024-04-25T10:13:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d61600640 10 client.7491 handle_command_reply: tid=0 2024-04-25T10:13:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d7db64740 1 client.7491 shutdown 2024-04-25T10:13:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d7db64740 1 -- 172.21.15.119:0/1449391772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111923bb0 msgr2=0x563111965b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1449391772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111923bb0 0x563111965b10 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400fe90 tx=0x7f7d64001110 comp rx=0 tx=0).stop 2024-04-25T10:13:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d7db64740 1 -- 172.21.15.119:0/1449391772 shutdown_connections 2024-04-25T10:13:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1449391772 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5631119507d0 0x56311194a8f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1449391772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563111960280 0x56311194e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1449391772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56311194e670 0x56311194dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d7db64740 1 --2- 172.21.15.119:0/1449391772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563111923bb0 0x563111965b10 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d61600640 0 client.7491 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:13:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d7db64740 1 -- 172.21.15.119:0/1449391772 >> 172.21.15.119:0/1449391772 conn(0x5631118eba30 msgr2=0x5631119609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:44.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d7db64740 1 -- 172.21.15.119:0/1449391772 shutdown_connections 2024-04-25T10:13:44.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.093+0000 7f7d7db64740 1 -- 172.21.15.119:0/1449391772 wait complete. 2024-04-25T10:13:44.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.094+0000 7f7d7db64740 20 client.7491 trim_cache size 0 max 16384 2024-04-25T10:13:44.102 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:13:44.102 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:13:44.102 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2024-04-25T10:13:44.102 INFO:teuthology.orchestra.run.smithi119.stdout: "c6494c6d-9eb1-4876-b0f2-bcf84b895017": { 2024-04-25T10:13:44.102 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/dir", 2024-04-25T10:13:44.102 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "c6494c6d-9eb1-4876-b0f2-bcf84b895017", 2024-04-25T10:13:44.103 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive" 2024-04-25T10:13:44.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:44.103 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:13:44.103 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:13:44.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.094+0000 7f7d7ae00640 1 -- 172.21.15.119:0/3011048853 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d5006b4d0 msgr2=0x7f7d5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.094+0000 7f7d7ae00640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d5006b4d0 0x7f7d5006d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7d740b1d40 tx=0x7f7d64009040 comp rx=0 tx=0).stop 2024-04-25T10:13:44.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.094+0000 7f7d7ae00640 1 -- 172.21.15.119:0/3011048853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d7415cb40 msgr2=0x7f7d7415cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.094+0000 7f7d7ae00640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d7415cb40 0x7f7d7415cfc0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00e230 tx=0x7f7d5c009750 comp rx=0 tx=0).stop 2024-04-25T10:13:44.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.094+0000 7f7d7ae00640 1 -- 172.21.15.119:0/3011048853 shutdown_connections 2024-04-25T10:13:44.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.094+0000 7f7d7ae00640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d5006b4d0 0x7f7d5006d990 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.094+0000 7f7d7ae00640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d741f75a0 0x7f7d741f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.094+0000 7f7d7ae00640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d7415cb40 0x7f7d7415cfc0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.094+0000 7f7d7ae00640 1 --2- 172.21.15.119:0/3011048853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d58003f30 0x7f7d74148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.094+0000 7f7d7ae00640 1 -- 172.21.15.119:0/3011048853 >> 172.21.15.119:0/3011048853 conn(0x7f7d74099a40 msgr2=0x7f7d74159dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:44.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.094+0000 7f7d7ae00640 1 -- 172.21.15.119:0/3011048853 shutdown_connections 2024-04-25T10:13:44.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:13:44.094+0000 7f7d7ae00640 1 -- 172.21.15.119:0/3011048853 wait complete. 2024-04-25T10:13:44.118 INFO:tasks.cephfs.filesystem:scrub status for tag:c6494c6d-9eb1-4876-b0f2-bcf84b895017 - {'path': '/dir', 'tag': 'c6494c6d-9eb1-4876-b0f2-bcf84b895017', 'options': 'recursive'} 2024-04-25T10:13:47.233 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:13:47.228+0000 7f8c67800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:14:01.846 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:01.855 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:14.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:14.120 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:14:14.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.328+0000 7f5571e00640 1 Processor -- start 2024-04-25T10:14:14.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.328+0000 7f5571e00640 1 -- start start 2024-04-25T10:14:14.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.329+0000 7f5571e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f556c13d5c0 0x7f556c1407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.329+0000 7f5571e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f556c071830 0x7f556c15b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.330+0000 7f5571e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f556c140ce0 con 0x7f556c13dce0 2024-04-25T10:14:14.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.330+0000 7f5571e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f556c15abf0 con 0x7f556c071830 2024-04-25T10:14:14.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.330+0000 7f5571e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f556c13d9a0 con 0x7f556c13d5c0 2024-04-25T10:14:14.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.330+0000 7f5571400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f556c071830 0x7f556c15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.330+0000 7f5571400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f556c071830 0x7f556c15b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54718/0 (socket says 172.21.15.119:54718) 2024-04-25T10:14:14.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.330+0000 7f5571400640 1 -- 172.21.15.119:0/2575500665 learned_addr learned my addr 172.21.15.119:0/2575500665 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:14.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.330+0000 7f5570a00640 1 --2- 172.21.15.119:0/2575500665 >> v2:172.21.15.132:3300/0 conn(0x7f556c13d5c0 0x7f556c1407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.330+0000 7f5571400640 1 -- 172.21.15.119:0/2575500665 >> v2:172.21.15.132:3300/0 conn(0x7f556c13d5c0 msgr2=0x7f556c1407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.330+0000 7f5571400640 1 --2- 172.21.15.119:0/2575500665 >> v2:172.21.15.132:3300/0 conn(0x7f556c13d5c0 0x7f556c1407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.330+0000 7f5571400640 1 -- 172.21.15.119:0/2575500665 >> v1:172.21.15.132:6789/0 conn(0x7f556c13dce0 legacy=0x7f556c071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.330+0000 7f5571400640 1 -- 172.21.15.119:0/2575500665 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c15e650 con 0x7f556c071830 2024-04-25T10:14:14.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.331+0000 7f5571400640 1 --2- 172.21.15.119:0/2575500665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f556c071830 0x7f556c15b090 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f556c06f210 tx=0x7f556402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799e7aeec0b99b15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.332+0000 7f556b400640 1 -- 172.21.15.119:0/2575500665 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564036070 con 0x7f556c071830 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.332+0000 7f556b400640 1 -- 172.21.15.119:0/2575500665 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5564031600 con 0x7f556c071830 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.332+0000 7f556b400640 1 -- 172.21.15.119:0/2575500665 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55640319d0 con 0x7f556c071830 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.332+0000 7f5571e00640 1 -- 172.21.15.119:0/2575500665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f556c071830 msgr2=0x7f556c15b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.332+0000 7f5571e00640 1 --2- 172.21.15.119:0/2575500665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f556c071830 0x7f556c15b090 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f556c06f210 tx=0x7f556402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.332+0000 7f5571e00640 1 -- 172.21.15.119:0/2575500665 shutdown_connections 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.332+0000 7f5571e00640 1 --2- 172.21.15.119:0/2575500665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f556c071830 0x7f556c15b090 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.332+0000 7f5571e00640 1 --2- 172.21.15.119:0/2575500665 >> v2:172.21.15.132:3300/0 conn(0x7f556c13d5c0 0x7f556c1407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.332+0000 7f5571e00640 1 -- 172.21.15.119:0/2575500665 >> 172.21.15.119:0/2575500665 conn(0x7f556c06aba0 msgr2=0x7f556c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.333+0000 7f5571e00640 1 -- 172.21.15.119:0/2575500665 shutdown_connections 2024-04-25T10:14:14.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.333+0000 7f5571e00640 1 -- 172.21.15.119:0/2575500665 wait complete. 2024-04-25T10:14:14.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.334+0000 7f5571e00640 1 Processor -- start 2024-04-25T10:14:14.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.335+0000 7f5571e00640 1 -- start start 2024-04-25T10:14:14.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.335+0000 7f5571e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f556c071830 0x7f556c136a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.335+0000 7f5571e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f556c13d5c0 0x7f556c139990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.335+0000 7f5571e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f556c13dce0 0x7f556c139ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.336+0000 7f5571e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c15eef0 con 0x7f556c13dce0 2024-04-25T10:14:14.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.336+0000 7f5571e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c15ec10 con 0x7f556c071830 2024-04-25T10:14:14.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.336+0000 7f5571e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0b1980 con 0x7f556c13d5c0 2024-04-25T10:14:14.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.336+0000 7f5571400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f556c13dce0 0x7f556c139ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.336+0000 7f5571400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f556c13dce0 0x7f556c139ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54724/0 (socket says 172.21.15.119:54724) 2024-04-25T10:14:14.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.336+0000 7f5571400640 1 -- 172.21.15.119:0/1029050105 learned_addr learned my addr 172.21.15.119:0/1029050105 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:14.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.336+0000 7f5570a00640 1 --2- 172.21.15.119:0/1029050105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f556c071830 0x7f556c136a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.336+0000 7f5571400640 1 -- 172.21.15.119:0/1029050105 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f556c13d5c0 msgr2=0x7f556c139990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.336+0000 7f5571400640 1 --2- 172.21.15.119:0/1029050105 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f556c13d5c0 0x7f556c139990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.336+0000 7f5571400640 1 -- 172.21.15.119:0/1029050105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f556c071830 msgr2=0x7f556c136a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.336+0000 7f5571400640 1 --2- 172.21.15.119:0/1029050105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f556c071830 0x7f556c136a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.336+0000 7f5571400640 1 -- 172.21.15.119:0/1029050105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c141220 con 0x7f556c13dce0 2024-04-25T10:14:14.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.337+0000 7f5571400640 1 --2- 172.21.15.119:0/1029050105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f556c13dce0 0x7f556c139ed0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5564009f40 tx=0x7f5564004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.337+0000 7f5568c00640 1 -- 172.21.15.119:0/1029050105 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564036050 con 0x7f556c13dce0 2024-04-25T10:14:14.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.337+0000 7f5568c00640 1 -- 172.21.15.119:0/1029050105 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5564042070 con 0x7f556c13dce0 2024-04-25T10:14:14.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.337+0000 7f5568c00640 1 -- 172.21.15.119:0/1029050105 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564009510 con 0x7f556c13dce0 2024-04-25T10:14:14.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.337+0000 7f5571e00640 1 -- 172.21.15.119:0/1029050105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556c141500 con 0x7f556c13dce0 2024-04-25T10:14:14.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.337+0000 7f5571e00640 1 -- 172.21.15.119:0/1029050105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c141ac0 con 0x7f556c13dce0 2024-04-25T10:14:14.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.339+0000 7f5568c00640 1 -- 172.21.15.119:0/1029050105 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5564007bd0 con 0x7f556c13dce0 2024-04-25T10:14:14.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.340+0000 7f5568c00640 1 --2- 172.21.15.119:0/1029050105 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f554c06b4d0 0x7f554c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.340+0000 7f5568c00640 1 -- 172.21.15.119:0/1029050105 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f55640ab1a0 con 0x7f556c13dce0 2024-04-25T10:14:14.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.341+0000 7f5570a00640 1 --2- 172.21.15.119:0/1029050105 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f554c06b4d0 0x7f554c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.341+0000 7f55761de740 1 Processor -- start 2024-04-25T10:14:14.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.341+0000 7f5570a00640 1 --2- 172.21.15.119:0/1029050105 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f554c06b4d0 0x7f554c06d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5560006fd0 tx=0x7f5560008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.341+0000 7f55761de740 1 -- start start 2024-04-25T10:14:14.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.342+0000 7f55761de740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf88ad70 0x560ebf88ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.342+0000 7f55761de740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf8c7440 0x560ebf88b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.342+0000 7f55761de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8b5830 0x560ebf8b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.342+0000 7f55761de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebf7eec70 con 0x560ebf8b5830 2024-04-25T10:14:14.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.342+0000 7f55761de740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebf797cf0 con 0x560ebf8c7440 2024-04-25T10:14:14.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.342+0000 7f55761de740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ebf7ead10 con 0x560ebf88ad70 2024-04-25T10:14:14.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.343+0000 7f5571400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8b5830 0x560ebf8b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.343+0000 7f5571400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8b5830 0x560ebf8b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54728/0 (socket says 172.21.15.119:54728) 2024-04-25T10:14:14.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.343+0000 7f5571400640 1 -- 172.21.15.119:0/1771854250 learned_addr learned my addr 172.21.15.119:0/1771854250 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:14.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.343+0000 7f5570a00640 1 --2- 172.21.15.119:0/1771854250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf88ad70 0x560ebf88ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.343+0000 7f5571400640 1 -- 172.21.15.119:0/1771854250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf88ad70 msgr2=0x560ebf88ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.343+0000 7f5571400640 1 --2- 172.21.15.119:0/1771854250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf88ad70 0x560ebf88ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.343+0000 7f5571400640 1 -- 172.21.15.119:0/1771854250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf8c7440 msgr2=0x560ebf88b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.343+0000 7f5571400640 1 --2- 172.21.15.119:0/1771854250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf8c7440 0x560ebf88b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.343+0000 7f5571400640 1 -- 172.21.15.119:0/1771854250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ebf5cd7d0 con 0x560ebf8b5830 2024-04-25T10:14:14.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.343+0000 7f5570a00640 1 --2- 172.21.15.119:0/1771854250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf88ad70 0x560ebf88ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:14.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.343+0000 7f5571400640 1 --2- 172.21.15.119:0/1771854250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8b5830 0x560ebf8b5cb0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5564034040 tx=0x7f55640796c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd86ae0b344225 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.344+0000 7f555d200640 1 -- 172.21.15.119:0/1771854250 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564031600 con 0x560ebf8b5830 2024-04-25T10:14:14.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.344+0000 7f555d200640 1 -- 172.21.15.119:0/1771854250 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5564037040 con 0x560ebf8b5830 2024-04-25T10:14:14.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.344+0000 7f555d200640 1 -- 172.21.15.119:0/1771854250 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564079f30 con 0x560ebf8b5830 2024-04-25T10:14:14.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.344+0000 7f55761de740 1 -- 172.21.15.119:0/1771854250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8b5830 msgr2=0x560ebf8b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.344+0000 7f55761de740 1 --2- 172.21.15.119:0/1771854250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8b5830 0x560ebf8b5cb0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f5564034040 tx=0x7f55640796c0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.344+0000 7f55761de740 1 -- 172.21.15.119:0/1771854250 shutdown_connections 2024-04-25T10:14:14.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.344+0000 7f55761de740 1 --2- 172.21.15.119:0/1771854250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8b5830 0x560ebf8b5cb0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.344+0000 7f55761de740 1 --2- 172.21.15.119:0/1771854250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf8c7440 0x560ebf88b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.344+0000 7f55761de740 1 --2- 172.21.15.119:0/1771854250 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf88ad70 0x560ebf88ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.344+0000 7f55761de740 1 -- 172.21.15.119:0/1771854250 >> 172.21.15.119:0/1771854250 conn(0x560ebf852a30 msgr2=0x560ebf8d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.345+0000 7f55761de740 1 -- 172.21.15.119:0/1771854250 shutdown_connections 2024-04-25T10:14:14.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.345+0000 7f55761de740 1 -- 172.21.15.119:0/1771854250 wait complete. 2024-04-25T10:14:14.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.345+0000 7f55761de740 1 -- start start 2024-04-25T10:14:14.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.346+0000 7f55761de740 1 Processor -- start 2024-04-25T10:14:14.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f55761de740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf88ad70 0x560ebf8b8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f55761de740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf8b5830 0x560ebf8cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f55761de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8c7440 0x560ebf8a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f55761de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebf7eec70 con 0x560ebf8c7440 2024-04-25T10:14:14.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f55761de740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebf78d3a0 con 0x560ebf8b5830 2024-04-25T10:14:14.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f55761de740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ebf8a4af0 con 0x560ebf88ad70 2024-04-25T10:14:14.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f55761de740 10 client.0 _ll_register_callbacks cb 0x560ebf7aab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:14.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f5571400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8c7440 0x560ebf8a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f5571400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8c7440 0x560ebf8a6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54738/0 (socket says 172.21.15.119:54738) 2024-04-25T10:14:14.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f5571400640 1 -- 172.21.15.119:0/2330378233 learned_addr learned my addr 172.21.15.119:0/2330378233 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:14.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f5570a00640 1 --2- 172.21.15.119:0/2330378233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf88ad70 0x560ebf8b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f5571400640 1 -- 172.21.15.119:0/2330378233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf88ad70 msgr2=0x560ebf8b8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f5571400640 1 --2- 172.21.15.119:0/2330378233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf88ad70 0x560ebf8b8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f556be00640 1 --2- 172.21.15.119:0/2330378233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf8b5830 0x560ebf8cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f5571400640 1 -- 172.21.15.119:0/2330378233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf8b5830 msgr2=0x560ebf8cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f5571400640 1 --2- 172.21.15.119:0/2330378233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf8b5830 0x560ebf8cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.347+0000 7f5571400640 1 -- 172.21.15.119:0/2330378233 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ebf5033f0 con 0x560ebf8c7440 2024-04-25T10:14:14.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.348+0000 7f5571400640 1 --2- 172.21.15.119:0/2330378233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8c7440 0x560ebf8a6f60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5564034040 tx=0x7f5564009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1e2cc75da56163 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.348+0000 7f5551600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.348+0000 7f5551600640 1 -- 172.21.15.119:0/2330378233 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564096060 con 0x560ebf8c7440 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.348+0000 7f5551600640 1 -- 172.21.15.119:0/2330378233 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f556407ab60 con 0x560ebf8c7440 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.348+0000 7f55761de740 1 -- 172.21.15.119:0/2330378233 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5564037040 con 0x560ebf8c7440 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.348+0000 7f5551600640 1 -- 172.21.15.119:0/2330378233 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f556409d050 con 0x560ebf8c7440 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.348+0000 7f55761de740 10 client.7503 fetch_fsmap learned FSMap version 68 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.348+0000 7f55761de740 1 -- 172.21.15.119:0/2330378233 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x560ebf4c30f0 con 0x560ebf8c7440 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.348+0000 7f5551600640 1 -- 172.21.15.119:0/2330378233 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5564037040 con 0x560ebf8c7440 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.349+0000 7f5551600640 1 -- 172.21.15.119:0/2330378233 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f55640982b0 con 0x560ebf8c7440 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.349+0000 7f55761de740 10 client.7503 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.349+0000 7f55761de740 10 client.7503 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.349+0000 7f55761de740 20 client.7503 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.349+0000 7f55761de740 1 --2- 172.21.15.119:0/2330378233 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x560ebf8cb750 0x560ebf8e3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.349+0000 7f55761de740 4 client.7503 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.349+0000 7f55761de740 1 -- 172.21.15.119:0/2330378233 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ebf7fc1b0 con 0x560ebf8cb750 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.349+0000 7f5570a00640 1 --2- 172.21.15.119:0/2330378233 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x560ebf8cb750 0x560ebf8e3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.350+0000 7f5570a00640 1 --2- 172.21.15.119:0/2330378233 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x560ebf8cb750 0x560ebf8e3d00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2cd7de496d5ba45 server_cookie=a04c209e4ab4f949 in_seq=0 out_seq=0 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.350+0000 7f5551600640 10 client.7503 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.350+0000 7f5551600640 1 -- 172.21.15.119:0/2330378233 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560ebf7fc1b0 con 0x560ebf8cb750 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.350+0000 7f5551600640 10 client.7503 handle_command_reply: tid=0 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.350+0000 7f55761de740 1 client.7503 shutdown 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.351+0000 7f55761de740 1 -- 172.21.15.119:0/2330378233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8c7440 msgr2=0x560ebf8a6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.351+0000 7f55761de740 1 --2- 172.21.15.119:0/2330378233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8c7440 0x560ebf8a6f60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5564034040 tx=0x7f5564009810 comp rx=0 tx=0).stop 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.351+0000 7f55761de740 1 -- 172.21.15.119:0/2330378233 shutdown_connections 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.351+0000 7f55761de740 1 --2- 172.21.15.119:0/2330378233 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x560ebf8cb750 0x560ebf8e3d00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.351+0000 7f55761de740 1 --2- 172.21.15.119:0/2330378233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8c7440 0x560ebf8a6f60 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.351+0000 7f55761de740 1 --2- 172.21.15.119:0/2330378233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf8b5830 0x560ebf8cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.351+0000 7f55761de740 1 --2- 172.21.15.119:0/2330378233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf88ad70 0x560ebf8b8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.351+0000 7f5551600640 0 client.7503 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:14.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.351+0000 7f55761de740 1 -- 172.21.15.119:0/2330378233 >> 172.21.15.119:0/2330378233 conn(0x560ebf852a30 msgr2=0x560ebf8b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.351+0000 7f55761de740 1 -- 172.21.15.119:0/2330378233 shutdown_connections 2024-04-25T10:14:14.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.351+0000 7f55761de740 1 -- 172.21.15.119:0/2330378233 wait complete. 2024-04-25T10:14:14.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.351+0000 7f55761de740 20 client.7503 trim_cache size 0 max 16384 2024-04-25T10:14:14.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.367+0000 7f55761de740 1 Processor -- start 2024-04-25T10:14:14.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.367+0000 7f55761de740 1 -- start start 2024-04-25T10:14:14.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.368+0000 7f55761de740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf88ad70 0x560ebf8f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.368+0000 7f55761de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8b5830 0x560ebf8c9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.368+0000 7f55761de740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf8c7440 0x560ebf8e3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.368+0000 7f55761de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c200000 con 0x560ebf8b5830 2024-04-25T10:14:14.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.368+0000 7f55761de740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c1ff200 con 0x560ebf88ad70 2024-04-25T10:14:14.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.368+0000 7f55761de740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c142d00 con 0x560ebf8c7440 2024-04-25T10:14:14.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.369+0000 7f556be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8b5830 0x560ebf8c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.369+0000 7f556be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8b5830 0x560ebf8c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54742/0 (socket says 172.21.15.119:54742) 2024-04-25T10:14:14.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.369+0000 7f556be00640 1 -- 172.21.15.119:0/532956872 learned_addr learned my addr 172.21.15.119:0/532956872 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:14.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.369+0000 7f5570a00640 1 --2- 172.21.15.119:0/532956872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf88ad70 0x560ebf8f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.369+0000 7f556be00640 1 -- 172.21.15.119:0/532956872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf8c7440 msgr2=0x560ebf8e3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.369+0000 7f556be00640 1 --2- 172.21.15.119:0/532956872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf8c7440 0x560ebf8e3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.369+0000 7f556be00640 1 -- 172.21.15.119:0/532956872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf88ad70 msgr2=0x560ebf8f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.369+0000 7f556be00640 1 --2- 172.21.15.119:0/532956872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf88ad70 0x560ebf8f34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.369+0000 7f556be00640 1 -- 172.21.15.119:0/532956872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ebf4c30f0 con 0x560ebf8b5830 2024-04-25T10:14:14.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.369+0000 7f556be00640 1 --2- 172.21.15.119:0/532956872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8b5830 0x560ebf8c9930 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f555800c680 tx=0x7f555800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a57901e6c11ae2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.369+0000 7f5553e00640 1 -- 172.21.15.119:0/532956872 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5558015020 con 0x560ebf8b5830 2024-04-25T10:14:14.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.370+0000 7f5553e00640 1 -- 172.21.15.119:0/532956872 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555800a8d0 con 0x560ebf8b5830 2024-04-25T10:14:14.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.370+0000 7f5553e00640 1 -- 172.21.15.119:0/532956872 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555800acc0 con 0x560ebf8b5830 2024-04-25T10:14:14.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.370+0000 7f55761de740 1 -- 172.21.15.119:0/532956872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8b5830 msgr2=0x560ebf8c9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.370+0000 7f55761de740 1 --2- 172.21.15.119:0/532956872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8b5830 0x560ebf8c9930 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f555800c680 tx=0x7f555800cb40 comp rx=0 tx=0).stop 2024-04-25T10:14:14.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.370+0000 7f55761de740 1 -- 172.21.15.119:0/532956872 shutdown_connections 2024-04-25T10:14:14.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.370+0000 7f55761de740 1 --2- 172.21.15.119:0/532956872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf8c7440 0x560ebf8e3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.370+0000 7f55761de740 1 --2- 172.21.15.119:0/532956872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8b5830 0x560ebf8c9930 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.370+0000 7f55761de740 1 --2- 172.21.15.119:0/532956872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf88ad70 0x560ebf8f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.370+0000 7f55761de740 1 -- 172.21.15.119:0/532956872 >> 172.21.15.119:0/532956872 conn(0x560ebf852a30 msgr2=0x560ebf888b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.370+0000 7f55761de740 1 -- 172.21.15.119:0/532956872 shutdown_connections 2024-04-25T10:14:14.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.371+0000 7f55761de740 1 -- 172.21.15.119:0/532956872 wait complete. 2024-04-25T10:14:14.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.371+0000 7f55761de740 1 -- start start 2024-04-25T10:14:14.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.371+0000 7f55761de740 1 Processor -- start 2024-04-25T10:14:14.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.372+0000 7f55761de740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf88ad70 0x560ebf8b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.372+0000 7f55761de740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf8b5830 0x560ebf8b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.372+0000 7f55761de740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8c7440 0x560ebf8e9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.372+0000 7f55761de740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ebf8f3a70 con 0x560ebf8c7440 2024-04-25T10:14:14.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.372+0000 7f55761de740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c143300 con 0x560ebf88ad70 2024-04-25T10:14:14.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.372+0000 7f55761de740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ebf8b8550 con 0x560ebf8b5830 2024-04-25T10:14:14.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.372+0000 7f55761de740 10 client.0 _ll_register_callbacks cb 0x7f5564075ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:14.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.373+0000 7f5571400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8c7440 0x560ebf8e9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.373+0000 7f5571400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8c7440 0x560ebf8e9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54746/0 (socket says 172.21.15.119:54746) 2024-04-25T10:14:14.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.373+0000 7f5571400640 1 -- 172.21.15.119:0/966335760 learned_addr learned my addr 172.21.15.119:0/966335760 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:14.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.373+0000 7f5571400640 1 -- 172.21.15.119:0/966335760 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf8b5830 msgr2=0x560ebf8b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.373+0000 7f5571400640 1 --2- 172.21.15.119:0/966335760 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf8b5830 0x560ebf8b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.373+0000 7f5571400640 1 -- 172.21.15.119:0/966335760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf88ad70 msgr2=0x560ebf8b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.373+0000 7f5571400640 1 --2- 172.21.15.119:0/966335760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf88ad70 0x560ebf8b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.373+0000 7f5571400640 1 -- 172.21.15.119:0/966335760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ebf8b98f0 con 0x560ebf8c7440 2024-04-25T10:14:14.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.373+0000 7f5571400640 1 --2- 172.21.15.119:0/966335760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8c7440 0x560ebf8e9cc0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5564079530 tx=0x7f55640a94b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faddff3f9dcd5bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.373+0000 7f5551600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:14.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.373+0000 7f5551600640 1 -- 172.21.15.119:0/966335760 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556407ad60 con 0x560ebf8c7440 2024-04-25T10:14:14.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.373+0000 7f5551600640 1 -- 172.21.15.119:0/966335760 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f5564078c80 con 0x560ebf8c7440 2024-04-25T10:14:14.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.373+0000 7f55761de740 1 -- 172.21.15.119:0/966335760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f555800a8d0 con 0x560ebf8c7440 2024-04-25T10:14:14.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.374+0000 7f5551600640 1 -- 172.21.15.119:0/966335760 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f556408c050 con 0x560ebf8c7440 2024-04-25T10:14:14.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.374+0000 7f55761de740 10 client.7509 fetch_fsmap learned FSMap version 68 2024-04-25T10:14:14.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.374+0000 7f55761de740 1 -- 172.21.15.119:0/966335760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x560ebf8b98f0 con 0x560ebf8c7440 2024-04-25T10:14:14.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.374+0000 7f5551600640 1 -- 172.21.15.119:0/966335760 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555800a8d0 con 0x560ebf8c7440 2024-04-25T10:14:14.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.374+0000 7f5551600640 1 -- 172.21.15.119:0/966335760 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f556408d240 con 0x560ebf8c7440 2024-04-25T10:14:14.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.374+0000 7f55761de740 10 client.7509 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:14:14.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.374+0000 7f55761de740 10 client.7509 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:14:14.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.374+0000 7f55761de740 20 client.7509 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:14.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.374+0000 7f55761de740 1 --2- 172.21.15.119:0/966335760 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x560ebf88bc50 0x560ebf8ea200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.374+0000 7f55761de740 4 client.7509 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:14:14.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.374+0000 7f55761de740 1 -- 172.21.15.119:0/966335760 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560ebf7f6500 con 0x560ebf88bc50 2024-04-25T10:14:14.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.374+0000 7f5570a00640 1 --2- 172.21.15.119:0/966335760 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x560ebf88bc50 0x560ebf8ea200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.375+0000 7f5570a00640 1 --2- 172.21.15.119:0/966335760 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x560ebf88bc50 0x560ebf8ea200 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c66d9ad574a6b server_cookie=93619f148ea1dc25 in_seq=0 out_seq=0 2024-04-25T10:14:14.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.375+0000 7f5551600640 10 client.7509 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:14.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.375+0000 7f5551600640 1 -- 172.21.15.119:0/966335760 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560ebf7f6500 con 0x560ebf88bc50 2024-04-25T10:14:14.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.375+0000 7f5551600640 10 client.7509 handle_command_reply: tid=0 2024-04-25T10:14:14.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.375+0000 7f55761de740 1 client.7509 shutdown 2024-04-25T10:14:14.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.375+0000 7f55761de740 1 -- 172.21.15.119:0/966335760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8c7440 msgr2=0x560ebf8e9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.375+0000 7f55761de740 1 --2- 172.21.15.119:0/966335760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8c7440 0x560ebf8e9cc0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5564079530 tx=0x7f55640a94b0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.375+0000 7f55761de740 1 -- 172.21.15.119:0/966335760 shutdown_connections 2024-04-25T10:14:14.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.375+0000 7f55761de740 1 --2- 172.21.15.119:0/966335760 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x560ebf88bc50 0x560ebf8ea200 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.375+0000 7f55761de740 1 --2- 172.21.15.119:0/966335760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ebf8c7440 0x560ebf8e9cc0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.375+0000 7f5551600640 0 client.7509 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:14.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.375+0000 7f55761de740 1 --2- 172.21.15.119:0/966335760 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560ebf8b5830 0x560ebf8b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.375+0000 7f55761de740 1 --2- 172.21.15.119:0/966335760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ebf88ad70 0x560ebf8b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.375+0000 7f55761de740 1 -- 172.21.15.119:0/966335760 >> 172.21.15.119:0/966335760 conn(0x560ebf852a30 msgr2=0x560ebf884bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.376+0000 7f55761de740 1 -- 172.21.15.119:0/966335760 shutdown_connections 2024-04-25T10:14:14.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.376+0000 7f55761de740 1 -- 172.21.15.119:0/966335760 wait complete. 2024-04-25T10:14:14.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.376+0000 7f55761de740 20 client.7509 trim_cache size 0 max 16384 2024-04-25T10:14:14.384 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:14:14.384 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2024-04-25T10:14:14.384 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2024-04-25T10:14:14.384 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:14:14.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.376+0000 7f5571e00640 1 -- 172.21.15.119:0/1029050105 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f554c06b4d0 msgr2=0x7f554c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.376+0000 7f5571e00640 1 --2- 172.21.15.119:0/1029050105 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f554c06b4d0 0x7f554c06d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5560006fd0 tx=0x7f5560008040 comp rx=0 tx=0).stop 2024-04-25T10:14:14.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.376+0000 7f5571e00640 1 -- 172.21.15.119:0/1029050105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f556c13dce0 msgr2=0x7f556c139ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.376+0000 7f5571e00640 1 --2- 172.21.15.119:0/1029050105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f556c13dce0 0x7f556c139ed0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5564009f40 tx=0x7f5564004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:14.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.376+0000 7f5571e00640 1 -- 172.21.15.119:0/1029050105 shutdown_connections 2024-04-25T10:14:14.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.376+0000 7f5571e00640 1 --2- 172.21.15.119:0/1029050105 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f554c06b4d0 0x7f554c06d990 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.376+0000 7f5571e00640 1 --2- 172.21.15.119:0/1029050105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f556c13dce0 0x7f556c139ed0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.376+0000 7f5571e00640 1 --2- 172.21.15.119:0/1029050105 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f556c13d5c0 0x7f556c139990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.376+0000 7f5571e00640 1 --2- 172.21.15.119:0/1029050105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f556c071830 0x7f556c136a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.376+0000 7f5571e00640 1 -- 172.21.15.119:0/1029050105 >> 172.21.15.119:0/1029050105 conn(0x7f556c06aba0 msgr2=0x7f556c159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.377+0000 7f5571e00640 1 -- 172.21.15.119:0/1029050105 shutdown_connections 2024-04-25T10:14:14.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.377+0000 7f5571e00640 1 -- 172.21.15.119:0/1029050105 wait complete. 2024-04-25T10:14:14.400 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:14:14.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:14.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.594+0000 7f2551600640 1 Processor -- start 2024-04-25T10:14:14.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.594+0000 7f2551600640 1 -- start start 2024-04-25T10:14:14.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.596+0000 7f2551600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c071190 0x7f254c0ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.596+0000 7f2551600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f254c071570 0x7f254c15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.596+0000 7f2551600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254c06d550 con 0x7f254c15e030 2024-04-25T10:14:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.596+0000 7f2551600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254c0aeee0 con 0x7f254c071570 2024-04-25T10:14:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.596+0000 7f2551600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c06d830 con 0x7f254c071190 2024-04-25T10:14:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.596+0000 7f2542000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f254c071570 0x7f254c15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.596+0000 7f2542000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f254c071570 0x7f254c15daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54762/0 (socket says 172.21.15.119:54762) 2024-04-25T10:14:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.596+0000 7f2542000640 1 -- 172.21.15.119:0/2133030973 learned_addr learned my addr 172.21.15.119:0/2133030973 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.596+0000 7f2542000640 1 -- 172.21.15.119:0/2133030973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c071190 msgr2=0x7f254c0ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.596+0000 7f2542000640 1 --2- 172.21.15.119:0/2133030973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c071190 0x7f254c0ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.596+0000 7f2542000640 1 -- 172.21.15.119:0/2133030973 >> v1:172.21.15.132:6789/0 conn(0x7f254c15e030 legacy=0x7f254c0b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:14.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.596+0000 7f2542000640 1 -- 172.21.15.119:0/2133030973 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c06db10 con 0x7f254c071570 2024-04-25T10:14:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.597+0000 7f2542000640 1 --2- 172.21.15.119:0/2133030973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c071570 0x7f254c15daf0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f2534009870 tx=0x7f253402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d924a9c28a60ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.598+0000 7f254a000640 1 -- 172.21.15.119:0/2133030973 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534037070 con 0x7f254c071570 2024-04-25T10:14:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.598+0000 7f254a000640 1 -- 172.21.15.119:0/2133030973 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2534032440 con 0x7f254c071570 2024-04-25T10:14:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.598+0000 7f254a000640 1 -- 172.21.15.119:0/2133030973 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534032810 con 0x7f254c071570 2024-04-25T10:14:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.598+0000 7f2551600640 1 -- 172.21.15.119:0/2133030973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c071570 msgr2=0x7f254c15daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.598+0000 7f2551600640 1 --2- 172.21.15.119:0/2133030973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c071570 0x7f254c15daf0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f2534009870 tx=0x7f253402a210 comp rx=0 tx=0).stop 2024-04-25T10:14:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.599+0000 7f2551600640 1 -- 172.21.15.119:0/2133030973 shutdown_connections 2024-04-25T10:14:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.599+0000 7f2551600640 1 --2- 172.21.15.119:0/2133030973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c071570 0x7f254c15daf0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.599+0000 7f2551600640 1 --2- 172.21.15.119:0/2133030973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c071190 0x7f254c0ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.599+0000 7f2551600640 1 -- 172.21.15.119:0/2133030973 >> 172.21.15.119:0/2133030973 conn(0x7f254c06aba0 msgr2=0x7f254c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.599+0000 7f2551600640 1 -- 172.21.15.119:0/2133030973 shutdown_connections 2024-04-25T10:14:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.599+0000 7f2551600640 1 -- 172.21.15.119:0/2133030973 wait complete. 2024-04-25T10:14:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.600+0000 7f2551600640 1 Processor -- start 2024-04-25T10:14:14.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.600+0000 7f2551600640 1 -- start start 2024-04-25T10:14:14.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.601+0000 7f2551600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c071190 0x7f254c0af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.601+0000 7f2551600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c071570 0x7f254c0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.601+0000 7f2551600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f254c15e030 0x7f254c153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.601+0000 7f2551600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c06e8b0 con 0x7f254c071570 2024-04-25T10:14:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.601+0000 7f2551600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c06e0d0 con 0x7f254c15e030 2024-04-25T10:14:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.601+0000 7f2551600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0719d0 con 0x7f254c071190 2024-04-25T10:14:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.602+0000 7f2542000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c071570 0x7f254c0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.602+0000 7f2542000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c071570 0x7f254c0b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54766/0 (socket says 172.21.15.119:54766) 2024-04-25T10:14:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.602+0000 7f2542000640 1 -- 172.21.15.119:0/2730676615 learned_addr learned my addr 172.21.15.119:0/2730676615 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.602+0000 7f254aa00640 1 --2- 172.21.15.119:0/2730676615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c071190 0x7f254c0af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.602+0000 7f2542000640 1 -- 172.21.15.119:0/2730676615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c071190 msgr2=0x7f254c0af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.602+0000 7f2542000640 1 --2- 172.21.15.119:0/2730676615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c071190 0x7f254c0af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.602+0000 7f2542000640 1 -- 172.21.15.119:0/2730676615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f254c15e030 msgr2=0x7f254c153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.602+0000 7f2542000640 1 --2- 172.21.15.119:0/2730676615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f254c15e030 0x7f254c153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.602+0000 7f2542000640 1 -- 172.21.15.119:0/2730676615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c0af640 con 0x7f254c071570 2024-04-25T10:14:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.602+0000 7f254aa00640 1 --2- 172.21.15.119:0/2730676615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c071190 0x7f254c0af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:14.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.603+0000 7f2542000640 1 --2- 172.21.15.119:0/2730676615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c071570 0x7f254c0b11c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f2534009870 tx=0x7f25340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.603+0000 7f2543400640 1 -- 172.21.15.119:0/2730676615 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534041070 con 0x7f254c071570 2024-04-25T10:14:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.603+0000 7f2543400640 1 -- 172.21.15.119:0/2730676615 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253403c870 con 0x7f254c071570 2024-04-25T10:14:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.603+0000 7f2543400640 1 -- 172.21.15.119:0/2730676615 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534037040 con 0x7f254c071570 2024-04-25T10:14:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.603+0000 7f2551600640 1 -- 172.21.15.119:0/2730676615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c0af920 con 0x7f254c071570 2024-04-25T10:14:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.603+0000 7f2551600640 1 -- 172.21.15.119:0/2730676615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c0afee0 con 0x7f254c071570 2024-04-25T10:14:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.604+0000 7f2551600640 1 -- 172.21.15.119:0/2730676615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254c0749b0 con 0x7f254c071570 2024-04-25T10:14:14.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.605+0000 7f2543400640 1 -- 172.21.15.119:0/2730676615 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2534044070 con 0x7f254c071570 2024-04-25T10:14:14.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.606+0000 7f2543400640 1 --2- 172.21.15.119:0/2730676615 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f252806b4d0 0x7f252806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:14.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.607+0000 7f2543400640 1 -- 172.21.15.119:0/2730676615 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f25340ab120 con 0x7f254c071570 2024-04-25T10:14:14.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.607+0000 7f254aa00640 1 --2- 172.21.15.119:0/2730676615 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f252806b4d0 0x7f252806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:14.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.608+0000 7f254aa00640 1 --2- 172.21.15.119:0/2730676615 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f252806b4d0 0x7f252806d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f253c006fd0 tx=0x7f253c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:14.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.612+0000 7f2543400640 1 -- 172.21.15.119:0/2730676615 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2534079fd0 con 0x7f254c071570 2024-04-25T10:14:14.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.841+0000 7f2551600640 1 -- 172.21.15.119:0/2730676615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f254c0720f0 con 0x7f254c071570 2024-04-25T10:14:14.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.842+0000 7f2543400640 1 -- 172.21.15.119:0/2730676615 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f25340798d0 con 0x7f254c071570 2024-04-25T10:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:13:47.200726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/1734507541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1734507541},{"type":"v1","addr":"172.21.15.119:6837","nonce":1734507541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:14:14.847 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 68 2024-04-25T10:14:14.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.845+0000 7f2551600640 1 -- 172.21.15.119:0/2730676615 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f252806b4d0 msgr2=0x7f252806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.845+0000 7f2551600640 1 --2- 172.21.15.119:0/2730676615 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f252806b4d0 0x7f252806d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f253c006fd0 tx=0x7f253c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:14.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.845+0000 7f2551600640 1 -- 172.21.15.119:0/2730676615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c071570 msgr2=0x7f254c0b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:14.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.845+0000 7f2551600640 1 --2- 172.21.15.119:0/2730676615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c071570 0x7f254c0b11c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f2534009870 tx=0x7f25340046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.845+0000 7f2551600640 1 -- 172.21.15.119:0/2730676615 shutdown_connections 2024-04-25T10:14:14.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.845+0000 7f2551600640 1 --2- 172.21.15.119:0/2730676615 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f252806b4d0 0x7f252806d990 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.845+0000 7f2551600640 1 --2- 172.21.15.119:0/2730676615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f254c15e030 0x7f254c153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.845+0000 7f2551600640 1 --2- 172.21.15.119:0/2730676615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f254c071570 0x7f254c0b11c0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.845+0000 7f2551600640 1 --2- 172.21.15.119:0/2730676615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f254c071190 0x7f254c0af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:14.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.845+0000 7f2551600640 1 -- 172.21.15.119:0/2730676615 >> 172.21.15.119:0/2730676615 conn(0x7f254c06aba0 msgr2=0x7f254c0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:14.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.845+0000 7f2551600640 1 -- 172.21.15.119:0/2730676615 shutdown_connections 2024-04-25T10:14:14.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:14.845+0000 7f2551600640 1 -- 172.21.15.119:0/2730676615 wait complete. 2024-04-25T10:14:14.867 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:14:15.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.061+0000 7f40edc00640 1 Processor -- start 2024-04-25T10:14:15.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.062+0000 7f40edc00640 1 -- start start 2024-04-25T10:14:15.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.063+0000 7f40edc00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f40e80cd310 0x7f40e8166670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.063+0000 7f40edc00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e806dcb0 con 0x7f40e80ce9d0 2024-04-25T10:14:15.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.063+0000 7f40edc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e806df90 con 0x7f40e8144e70 2024-04-25T10:14:15.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.063+0000 7f40edc00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e815bfe0 con 0x7f40e80cd310 2024-04-25T10:14:15.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.063+0000 7f40de000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f40e80ce9d0 0x7f40e80cee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51962/0 (socket says 172.21.15.119:51962) 2024-04-25T10:14:15.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.063+0000 7f40e7400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f40e80cd310 0x7f40e8166670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.063+0000 7f40de000640 1 -- 172.21.15.119:0/723860135 learned_addr learned my addr 172.21.15.119:0/723860135 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:14:15.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.064+0000 7f40e6000640 1 -- 172.21.15.119:0/723860135 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567235153 0 0) 0x7f40e806dcb0 con 0x7f40e80ce9d0 2024-04-25T10:14:15.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.064+0000 7f40e6000640 1 -- 172.21.15.119:0/723860135 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40cc003520 con 0x7f40e80ce9d0 2024-04-25T10:14:15.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.064+0000 7f40e7400640 1 -- 172.21.15.119:0/723860135 >> v1:172.21.15.132:6789/0 conn(0x7f40e8144e70 legacy=0x7f40e806d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.064+0000 7f40e7400640 1 -- 172.21.15.119:0/723860135 >> v1:172.21.15.119:6789/0 conn(0x7f40e80ce9d0 legacy=0x7f40e80cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.064+0000 7f40e7400640 1 -- 172.21.15.119:0/723860135 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e806e270 con 0x7f40e80cd310 2024-04-25T10:14:15.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.065+0000 7f40e7400640 1 --2- 172.21.15.119:0/723860135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40e80cd310 0x7f40e8166670 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f40e815c160 tx=0x7f40e002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f8cdd3333fca77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.065+0000 7f40e6000640 1 -- 172.21.15.119:0/723860135 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0007740 con 0x7f40e80cd310 2024-04-25T10:14:15.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.065+0000 7f40e6000640 1 -- 172.21.15.119:0/723860135 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e00078e0 con 0x7f40e80cd310 2024-04-25T10:14:15.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.066+0000 7f40e6000640 1 -- 172.21.15.119:0/723860135 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0007c90 con 0x7f40e80cd310 2024-04-25T10:14:15.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.066+0000 7f40edc00640 1 -- 172.21.15.119:0/723860135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40e80cd310 msgr2=0x7f40e8166670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.066+0000 7f40edc00640 1 --2- 172.21.15.119:0/723860135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40e80cd310 0x7f40e8166670 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f40e815c160 tx=0x7f40e002a570 comp rx=0 tx=0).stop 2024-04-25T10:14:15.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.066+0000 7f40edc00640 1 -- 172.21.15.119:0/723860135 shutdown_connections 2024-04-25T10:14:15.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.066+0000 7f40edc00640 1 --2- 172.21.15.119:0/723860135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40e80cd310 0x7f40e8166670 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.066+0000 7f40edc00640 1 -- 172.21.15.119:0/723860135 >> 172.21.15.119:0/723860135 conn(0x7f40e806aba0 msgr2=0x7f40e806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.066+0000 7f40edc00640 1 -- 172.21.15.119:0/723860135 shutdown_connections 2024-04-25T10:14:15.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.066+0000 7f40edc00640 1 -- 172.21.15.119:0/723860135 wait complete. 2024-04-25T10:14:15.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.068+0000 7f40edc00640 1 Processor -- start 2024-04-25T10:14:15.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.068+0000 7f40edc00640 1 -- start start 2024-04-25T10:14:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.069+0000 7f40edc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40e80cd310 0x7f40e813c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.069+0000 7f40edc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40e80ce9d0 0x7f40e8157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.069+0000 7f40edc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40e8144e70 0x7f40e813e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.069+0000 7f40edc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e81454b0 con 0x7f40e80cd310 2024-04-25T10:14:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.069+0000 7f40edc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e806d970 con 0x7f40e8144e70 2024-04-25T10:14:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.069+0000 7f40edc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e8145630 con 0x7f40e80ce9d0 2024-04-25T10:14:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.069+0000 7f40e6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40e80cd310 0x7f40e813c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.069+0000 7f40e6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40e80cd310 0x7f40e813c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54790/0 (socket says 172.21.15.119:54790) 2024-04-25T10:14:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.069+0000 7f40e6a00640 1 -- 172.21.15.119:0/2189584216 learned_addr learned my addr 172.21.15.119:0/2189584216 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.070+0000 7f40e6a00640 1 -- 172.21.15.119:0/2189584216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40e80ce9d0 msgr2=0x7f40e8157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.070+0000 7f40e6a00640 1 --2- 172.21.15.119:0/2189584216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40e80ce9d0 0x7f40e8157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.070+0000 7f40e6a00640 1 -- 172.21.15.119:0/2189584216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40e8144e70 msgr2=0x7f40e813e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.070+0000 7f40e7400640 1 --2- 172.21.15.119:0/2189584216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40e8144e70 0x7f40e813e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.070+0000 7f40e6a00640 1 --2- 172.21.15.119:0/2189584216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40e8144e70 0x7f40e813e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.070+0000 7f40e6a00640 1 -- 172.21.15.119:0/2189584216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e8158c80 con 0x7f40e80cd310 2024-04-25T10:14:15.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.070+0000 7f40e6a00640 1 --2- 172.21.15.119:0/2189584216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40e80cd310 0x7f40e813c130 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f40d800c390 tx=0x7f40d800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.070+0000 7f40df400640 1 -- 172.21.15.119:0/2189584216 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d8011790 con 0x7f40e80cd310 2024-04-25T10:14:15.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.071+0000 7f40df400640 1 -- 172.21.15.119:0/2189584216 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d8017070 con 0x7f40e80cd310 2024-04-25T10:14:15.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.071+0000 7f40df400640 1 -- 172.21.15.119:0/2189584216 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d8011bb0 con 0x7f40e80cd310 2024-04-25T10:14:15.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.071+0000 7f40edc00640 1 -- 172.21.15.119:0/2189584216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e8157be0 con 0x7f40e80cd310 2024-04-25T10:14:15.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.071+0000 7f40edc00640 1 -- 172.21.15.119:0/2189584216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e81581a0 con 0x7f40e80cd310 2024-04-25T10:14:15.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.073+0000 7f40df400640 1 -- 172.21.15.119:0/2189584216 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40d801c460 con 0x7f40e80cd310 2024-04-25T10:14:15.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.074+0000 7f40df400640 1 --2- 172.21.15.119:0/2189584216 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f40c406b4d0 0x7f40c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.074+0000 7f40df400640 1 -- 172.21.15.119:0/2189584216 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f40d8089f60 con 0x7f40e80cd310 2024-04-25T10:14:15.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.074+0000 7f40de000640 1 --2- 172.21.15.119:0/2189584216 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f40c406b4d0 0x7f40c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.075+0000 7f40f0ba4740 1 Processor -- start 2024-04-25T10:14:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.075+0000 7f40f0ba4740 1 -- start start 2024-04-25T10:14:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.075+0000 7f40de000640 1 --2- 172.21.15.119:0/2189584216 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f40c406b4d0 0x7f40c406d990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f40e80b23e0 tx=0x7f40d0007930 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.076+0000 7f40f0ba4740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a3e920 0x55a5f7a16f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.076+0000 7f40f0ba4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a3ed00 0x55a5f7a41180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.076+0000 7f40f0ba4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a416c0 0x55a5f7a14f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.076+0000 7f40f0ba4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f7979ae0 con 0x55a5f7a3ed00 2024-04-25T10:14:15.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.076+0000 7f40f0ba4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f79227c0 con 0x55a5f7a416c0 2024-04-25T10:14:15.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.076+0000 7f40f0ba4740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5f7975b80 con 0x55a5f7a3e920 2024-04-25T10:14:15.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.076+0000 7f40de000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a3ed00 0x55a5f7a41180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.076+0000 7f40de000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a3ed00 0x55a5f7a41180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54794/0 (socket says 172.21.15.119:54794) 2024-04-25T10:14:15.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.076+0000 7f40de000640 1 -- 172.21.15.119:0/3922276421 learned_addr learned my addr 172.21.15.119:0/3922276421 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:15.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.076+0000 7f40de000640 1 -- 172.21.15.119:0/3922276421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a3e920 msgr2=0x55a5f7a16f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.076+0000 7f40de000640 1 --2- 172.21.15.119:0/3922276421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a3e920 0x55a5f7a16f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.076+0000 7f40de000640 1 -- 172.21.15.119:0/3922276421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a416c0 msgr2=0x55a5f7a14f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.076+0000 7f40de000640 1 --2- 172.21.15.119:0/3922276421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a416c0 0x55a5f7a14f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.076+0000 7f40de000640 1 -- 172.21.15.119:0/3922276421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5f775a7c0 con 0x55a5f7a3ed00 2024-04-25T10:14:15.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.077+0000 7f40de000640 1 --2- 172.21.15.119:0/3922276421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a3ed00 0x55a5f7a41180 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f40d0005d00 tx=0x7f40d000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a23326ff110f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.077+0000 7f40d7400640 1 -- 172.21.15.119:0/3922276421 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0016110 con 0x55a5f7a3ed00 2024-04-25T10:14:15.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.077+0000 7f40d7400640 1 -- 172.21.15.119:0/3922276421 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d0007560 con 0x55a5f7a3ed00 2024-04-25T10:14:15.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.077+0000 7f40d7400640 1 -- 172.21.15.119:0/3922276421 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d00162b0 con 0x55a5f7a3ed00 2024-04-25T10:14:15.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.077+0000 7f40f0ba4740 1 -- 172.21.15.119:0/3922276421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a3ed00 msgr2=0x55a5f7a41180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.077+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/3922276421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a3ed00 0x55a5f7a41180 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f40d0005d00 tx=0x7f40d000f000 comp rx=0 tx=0).stop 2024-04-25T10:14:15.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.078+0000 7f40f0ba4740 1 -- 172.21.15.119:0/3922276421 shutdown_connections 2024-04-25T10:14:15.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.078+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/3922276421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a416c0 0x55a5f7a14f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.078+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/3922276421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a3ed00 0x55a5f7a41180 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.078+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/3922276421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a3e920 0x55a5f7a16f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.078+0000 7f40f0ba4740 1 -- 172.21.15.119:0/3922276421 >> 172.21.15.119:0/3922276421 conn(0x55a5f7a13980 msgr2=0x55a5f7a5c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.078+0000 7f40f0ba4740 1 -- 172.21.15.119:0/3922276421 shutdown_connections 2024-04-25T10:14:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.078+0000 7f40f0ba4740 1 -- 172.21.15.119:0/3922276421 wait complete. 2024-04-25T10:14:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.078+0000 7f40f0ba4740 1 -- start start 2024-04-25T10:14:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.078+0000 7f40f0ba4740 1 Processor -- start 2024-04-25T10:14:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.079+0000 7f40f0ba4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a3ed00 0x55a5f7a2b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.079+0000 7f40f0ba4740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a416c0 0x55a5f7a5b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.079+0000 7f40f0ba4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a5b8a0 0x55a5f7a44b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.079+0000 7f40f0ba4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f7917d40 con 0x55a5f7a5b8a0 2024-04-25T10:14:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.079+0000 7f40f0ba4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f7a59e80 con 0x55a5f7a3ed00 2024-04-25T10:14:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.079+0000 7f40f0ba4740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5f7a2a270 con 0x55a5f7a416c0 2024-04-25T10:14:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.080+0000 7f40f0ba4740 10 client.0 _ll_register_callbacks cb 0x55a5f768dba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.080+0000 7f40e7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a5b8a0 0x55a5f7a44b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.080+0000 7f40e7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a5b8a0 0x55a5f7a44b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54806/0 (socket says 172.21.15.119:54806) 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.080+0000 7f40e7400640 1 -- 172.21.15.119:0/1060012880 learned_addr learned my addr 172.21.15.119:0/1060012880 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.080+0000 7f40e6a00640 1 --2- 172.21.15.119:0/1060012880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a3ed00 0x55a5f7a2b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.080+0000 7f40de000640 1 --2- 172.21.15.119:0/1060012880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a416c0 0x55a5f7a5b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.080+0000 7f40e7400640 1 -- 172.21.15.119:0/1060012880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a416c0 msgr2=0x55a5f7a5b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.080+0000 7f40e7400640 1 --2- 172.21.15.119:0/1060012880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a416c0 0x55a5f7a5b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.080+0000 7f40e7400640 1 -- 172.21.15.119:0/1060012880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a3ed00 msgr2=0x55a5f7a2b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.080+0000 7f40e7400640 1 --2- 172.21.15.119:0/1060012880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a3ed00 0x55a5f7a2b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.080+0000 7f40e7400640 1 -- 172.21.15.119:0/1060012880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5f77baf50 con 0x55a5f7a5b8a0 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.081+0000 7f40e7400640 1 --2- 172.21.15.119:0/1060012880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a5b8a0 0x55a5f7a44b40 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f40e002ab60 tx=0x7f40e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d8938702191529 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.081+0000 7f40cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.081+0000 7f40cbe00640 1 -- 172.21.15.119:0/1060012880 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0009450 con 0x55a5f7a5b8a0 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.081+0000 7f40cbe00640 1 -- 172.21.15.119:0/1060012880 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f40e0004780 con 0x55a5f7a5b8a0 2024-04-25T10:14:15.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.081+0000 7f40f0ba4740 1 -- 172.21.15.119:0/1060012880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40d0007560 con 0x55a5f7a5b8a0 2024-04-25T10:14:15.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.081+0000 7f40cbe00640 1 -- 172.21.15.119:0/1060012880 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40e004c050 con 0x55a5f7a5b8a0 2024-04-25T10:14:15.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.081+0000 7f40f0ba4740 10 client.7530 fetch_fsmap learned FSMap version 68 2024-04-25T10:14:15.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.081+0000 7f40f0ba4740 1 -- 172.21.15.119:0/1060012880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55a5f764e0b0 con 0x55a5f7a5b8a0 2024-04-25T10:14:15.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.081+0000 7f40cbe00640 1 -- 172.21.15.119:0/1060012880 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d0007560 con 0x55a5f7a5b8a0 2024-04-25T10:14:15.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.081+0000 7f40cbe00640 1 -- 172.21.15.119:0/1060012880 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f40e004f020 con 0x55a5f7a5b8a0 2024-04-25T10:14:15.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.081+0000 7f40f0ba4740 10 client.7530 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:14:15.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.082+0000 7f40f0ba4740 10 client.7530 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:14:15.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.082+0000 7f40f0ba4740 20 client.7530 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:15.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.082+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/1060012880 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55a5f7a281c0 0x55a5f7a3d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.082+0000 7f40f0ba4740 4 client.7530 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:15.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.082+0000 7f40f0ba4740 1 -- 172.21.15.119:0/1060012880 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5f79d8b60 con 0x55a5f7a281c0 2024-04-25T10:14:15.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.082+0000 7f40e6a00640 1 --2- 172.21.15.119:0/1060012880 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55a5f7a281c0 0x55a5f7a3d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.082+0000 7f40e6a00640 1 --2- 172.21.15.119:0/1060012880 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55a5f7a281c0 0x55a5f7a3d510 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34920e44ef86282b server_cookie=ee28af3a29711b31 in_seq=0 out_seq=0 2024-04-25T10:14:15.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.082+0000 7f40cbe00640 10 client.7530 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:15.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.083+0000 7f40cbe00640 1 -- 172.21.15.119:0/1060012880 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a5f79d8b60 con 0x55a5f7a281c0 2024-04-25T10:14:15.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.083+0000 7f40cbe00640 10 client.7530 handle_command_reply: tid=0 2024-04-25T10:14:15.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.083+0000 7f40f0ba4740 1 client.7530 shutdown 2024-04-25T10:14:15.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.083+0000 7f40f0ba4740 1 -- 172.21.15.119:0/1060012880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a5b8a0 msgr2=0x55a5f7a44b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.083+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/1060012880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a5b8a0 0x55a5f7a44b40 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f40e002ab60 tx=0x7f40e0004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:15.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.083+0000 7f40f0ba4740 1 -- 172.21.15.119:0/1060012880 shutdown_connections 2024-04-25T10:14:15.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.083+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/1060012880 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55a5f7a281c0 0x55a5f7a3d510 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.083+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/1060012880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a5b8a0 0x55a5f7a44b40 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.083+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/1060012880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a416c0 0x55a5f7a5b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.083+0000 7f40cbe00640 0 client.7530 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.083+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/1060012880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a3ed00 0x55a5f7a2b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.083+0000 7f40f0ba4740 1 -- 172.21.15.119:0/1060012880 >> 172.21.15.119:0/1060012880 conn(0x55a5f7a3e920 msgr2=0x55a5f7a106e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.083+0000 7f40f0ba4740 1 -- 172.21.15.119:0/1060012880 shutdown_connections 2024-04-25T10:14:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.084+0000 7f40f0ba4740 1 -- 172.21.15.119:0/1060012880 wait complete. 2024-04-25T10:14:15.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.084+0000 7f40f0ba4740 20 client.7530 trim_cache size 0 max 16384 2024-04-25T10:14:15.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.096+0000 7f40f0ba4740 1 Processor -- start 2024-04-25T10:14:15.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.096+0000 7f40f0ba4740 1 -- start start 2024-04-25T10:14:15.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.097+0000 7f40f0ba4740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a3ed00 0x55a5f7a0cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.097+0000 7f40f0ba4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a416c0 0x55a5f7a545a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.097+0000 7f40f0ba4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a5b8a0 0x55a5f7a3d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.097+0000 7f40f0ba4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e80b3470 con 0x55a5f7a416c0 2024-04-25T10:14:15.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.097+0000 7f40f0ba4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e80b2870 con 0x55a5f7a5b8a0 2024-04-25T10:14:15.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.097+0000 7f40f0ba4740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e80b2e70 con 0x55a5f7a3ed00 2024-04-25T10:14:15.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.098+0000 7f40de000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a416c0 0x55a5f7a545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.098+0000 7f40de000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a416c0 0x55a5f7a545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54820/0 (socket says 172.21.15.119:54820) 2024-04-25T10:14:15.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.098+0000 7f40de000640 1 -- 172.21.15.119:0/3031444932 learned_addr learned my addr 172.21.15.119:0/3031444932 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.098+0000 7f40e7400640 1 --2- 172.21.15.119:0/3031444932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a5b8a0 0x55a5f7a3d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.098+0000 7f40de000640 1 -- 172.21.15.119:0/3031444932 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a3ed00 msgr2=0x55a5f7a0cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.098+0000 7f40de000640 1 --2- 172.21.15.119:0/3031444932 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a3ed00 0x55a5f7a0cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.098+0000 7f40de000640 1 -- 172.21.15.119:0/3031444932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a5b8a0 msgr2=0x55a5f7a3d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.098+0000 7f40de000640 1 --2- 172.21.15.119:0/3031444932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a5b8a0 0x55a5f7a3d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.098+0000 7f40de000640 1 -- 172.21.15.119:0/3031444932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5f7a54ae0 con 0x55a5f7a416c0 2024-04-25T10:14:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.098+0000 7f40de000640 1 --2- 172.21.15.119:0/3031444932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a416c0 0x55a5f7a545a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f40d0001200 tx=0x7f40d0007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7abde5242727b033 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.099+0000 7f40d6a00640 1 -- 172.21.15.119:0/3031444932 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0007660 con 0x55a5f7a416c0 2024-04-25T10:14:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.099+0000 7f40d6a00640 1 -- 172.21.15.119:0/3031444932 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d0016090 con 0x55a5f7a416c0 2024-04-25T10:14:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.099+0000 7f40d6a00640 1 -- 172.21.15.119:0/3031444932 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0016260 con 0x55a5f7a416c0 2024-04-25T10:14:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.099+0000 7f40f0ba4740 1 -- 172.21.15.119:0/3031444932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a416c0 msgr2=0x55a5f7a545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.099+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/3031444932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a416c0 0x55a5f7a545a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f40d0001200 tx=0x7f40d0007000 comp rx=0 tx=0).stop 2024-04-25T10:14:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.099+0000 7f40f0ba4740 1 -- 172.21.15.119:0/3031444932 shutdown_connections 2024-04-25T10:14:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.099+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/3031444932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a5b8a0 0x55a5f7a3d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.099+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/3031444932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a416c0 0x55a5f7a545a0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.099+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/3031444932 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a3ed00 0x55a5f7a0cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.099+0000 7f40f0ba4740 1 -- 172.21.15.119:0/3031444932 >> 172.21.15.119:0/3031444932 conn(0x55a5f7a3e920 msgr2=0x55a5f7a28f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.099+0000 7f40f0ba4740 1 -- 172.21.15.119:0/3031444932 shutdown_connections 2024-04-25T10:14:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.099+0000 7f40f0ba4740 1 -- 172.21.15.119:0/3031444932 wait complete. 2024-04-25T10:14:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.100+0000 7f40f0ba4740 1 -- start start 2024-04-25T10:14:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.100+0000 7f40f0ba4740 1 Processor -- start 2024-04-25T10:14:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.101+0000 7f40f0ba4740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a3ed00 0x55a5f7a72540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.101+0000 7f40f0ba4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a416c0 0x55a5f7a72a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.101+0000 7f40f0ba4740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a5b8a0 0x55a5f7a3b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.101+0000 7f40f0ba4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f7a28040 con 0x55a5f7a416c0 2024-04-25T10:14:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.101+0000 7f40f0ba4740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5f7a29fb0 con 0x55a5f7a3ed00 2024-04-25T10:14:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.101+0000 7f40f0ba4740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5f7a28610 con 0x55a5f7a5b8a0 2024-04-25T10:14:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.101+0000 7f40f0ba4740 10 client.0 _ll_register_callbacks cb 0x7f40d0007c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.101+0000 7f40de000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a416c0 0x55a5f7a72a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.101+0000 7f40de000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a416c0 0x55a5f7a72a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54826/0 (socket says 172.21.15.119:54826) 2024-04-25T10:14:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.101+0000 7f40de000640 1 -- 172.21.15.119:0/2777865936 learned_addr learned my addr 172.21.15.119:0/2777865936 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.102+0000 7f40de000640 1 -- 172.21.15.119:0/2777865936 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a5b8a0 msgr2=0x55a5f7a3b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.102+0000 7f40de000640 1 --2- 172.21.15.119:0/2777865936 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a5b8a0 0x55a5f7a3b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.102+0000 7f40de000640 1 -- 172.21.15.119:0/2777865936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a3ed00 msgr2=0x55a5f7a72540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.102+0000 7f40de000640 1 --2- 172.21.15.119:0/2777865936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a3ed00 0x55a5f7a72540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.102+0000 7f40de000640 1 -- 172.21.15.119:0/2777865936 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5f7a5f4a0 con 0x55a5f7a416c0 2024-04-25T10:14:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.102+0000 7f40de000640 1 --2- 172.21.15.119:0/2777865936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a416c0 0x55a5f7a72a80 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f40d0005cd0 tx=0x7f40d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57485947c09b06f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.102+0000 7f40cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.102+0000 7f40cbe00640 1 -- 172.21.15.119:0/2777865936 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0016be0 con 0x55a5f7a416c0 2024-04-25T10:14:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.102+0000 7f40cbe00640 1 -- 172.21.15.119:0/2777865936 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f40d001c040 con 0x55a5f7a416c0 2024-04-25T10:14:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.102+0000 7f40f0ba4740 1 -- 172.21.15.119:0/2777865936 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40d0016090 con 0x55a5f7a416c0 2024-04-25T10:14:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.103+0000 7f40cbe00640 1 -- 172.21.15.119:0/2777865936 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40d0037050 con 0x55a5f7a416c0 2024-04-25T10:14:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.103+0000 7f40f0ba4740 10 client.7536 fetch_fsmap learned FSMap version 68 2024-04-25T10:14:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.103+0000 7f40f0ba4740 1 -- 172.21.15.119:0/2777865936 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55a5f7acec00 con 0x55a5f7a416c0 2024-04-25T10:14:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.103+0000 7f40cbe00640 1 -- 172.21.15.119:0/2777865936 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d0016090 con 0x55a5f7a416c0 2024-04-25T10:14:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.103+0000 7f40cbe00640 1 -- 172.21.15.119:0/2777865936 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f40d003a020 con 0x55a5f7a416c0 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.103+0000 7f40f0ba4740 10 client.7536 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.103+0000 7f40f0ba4740 10 client.7536 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.103+0000 7f40f0ba4740 20 client.7536 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.103+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/2777865936 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55a5f7a3b5e0 0x55a5f7a3b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.103+0000 7f40f0ba4740 4 client.7536 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.103+0000 7f40f0ba4740 1 -- 172.21.15.119:0/2777865936 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55a5f7987020 con 0x55a5f7a3b5e0 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.103+0000 7f40e6a00640 1 --2- 172.21.15.119:0/2777865936 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55a5f7a3b5e0 0x55a5f7a3b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40cbe00640 10 client.7536 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40e6a00640 1 --2- 172.21.15.119:0/2777865936 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55a5f7a3b5e0 0x55a5f7a3b9c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd281bd1d3d7e3a2 server_cookie=93fa0963b923d8 in_seq=0 out_seq=0 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40cbe00640 1 -- 172.21.15.119:0/2777865936 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+418 (crc 0 0 0) 0x55a5f7987020 con 0x55a5f7a3b5e0 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40cbe00640 10 client.7536 handle_command_reply: tid=0 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40f0ba4740 1 client.7536 shutdown 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40f0ba4740 1 -- 172.21.15.119:0/2777865936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a416c0 msgr2=0x55a5f7a72a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/2777865936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a416c0 0x55a5f7a72a80 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f40d0005cd0 tx=0x7f40d00057f0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40f0ba4740 1 -- 172.21.15.119:0/2777865936 shutdown_connections 2024-04-25T10:14:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/2777865936 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55a5f7a3b5e0 0x55a5f7a3b9c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/2777865936 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a5f7a5b8a0 0x55a5f7a3b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/2777865936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5f7a416c0 0x55a5f7a72a80 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40cbe00640 0 client.7536 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:15.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40f0ba4740 1 --2- 172.21.15.119:0/2777865936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a5f7a3ed00 0x55a5f7a72540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40f0ba4740 1 -- 172.21.15.119:0/2777865936 >> 172.21.15.119:0/2777865936 conn(0x55a5f7a3e920 msgr2=0x55a5f7a28e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40f0ba4740 1 -- 172.21.15.119:0/2777865936 shutdown_connections 2024-04-25T10:14:15.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.104+0000 7f40f0ba4740 1 -- 172.21.15.119:0/2777865936 wait complete. 2024-04-25T10:14:15.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.105+0000 7f40f0ba4740 20 client.7536 trim_cache size 0 max 16384 2024-04-25T10:14:15.114 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:14:15.114 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2024-04-25T10:14:15.114 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:14:15.114 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2024-04-25T10:14:15.114 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 468801377, 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 1099511627777, 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/testdir/hardlink" 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 731408009, 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 1099511627776, 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/dir" 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "dir_frag", 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 1777586847, 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 1099511627776, 2024-04-25T10:14:15.115 INFO:teuthology.orchestra.run.smithi119.stdout: "frag": "*", 2024-04-25T10:14:15.116 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/dir" 2024-04-25T10:14:15.116 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:14:15.116 INFO:teuthology.orchestra.run.smithi119.stdout:] 2024-04-25T10:14:15.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.105+0000 7f40edc00640 1 -- 172.21.15.119:0/2189584216 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f40c406b4d0 msgr2=0x7f40c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.105+0000 7f40edc00640 1 --2- 172.21.15.119:0/2189584216 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f40c406b4d0 0x7f40c406d990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f40e80b23e0 tx=0x7f40d0007930 comp rx=0 tx=0).stop 2024-04-25T10:14:15.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.105+0000 7f40edc00640 1 -- 172.21.15.119:0/2189584216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40e80cd310 msgr2=0x7f40e813c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.105+0000 7f40edc00640 1 --2- 172.21.15.119:0/2189584216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40e80cd310 0x7f40e813c130 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f40d800c390 tx=0x7f40d800c850 comp rx=0 tx=0).stop 2024-04-25T10:14:15.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.105+0000 7f40edc00640 1 -- 172.21.15.119:0/2189584216 shutdown_connections 2024-04-25T10:14:15.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.105+0000 7f40edc00640 1 --2- 172.21.15.119:0/2189584216 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f40c406b4d0 0x7f40c406d990 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.105+0000 7f40edc00640 1 --2- 172.21.15.119:0/2189584216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40e8144e70 0x7f40e813e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.105+0000 7f40edc00640 1 --2- 172.21.15.119:0/2189584216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40e80ce9d0 0x7f40e8157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.105+0000 7f40edc00640 1 --2- 172.21.15.119:0/2189584216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40e80cd310 0x7f40e813c130 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.105+0000 7f40edc00640 1 -- 172.21.15.119:0/2189584216 >> 172.21.15.119:0/2189584216 conn(0x7f40e806aba0 msgr2=0x7f40e806d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.105+0000 7f40edc00640 1 -- 172.21.15.119:0/2189584216 shutdown_connections 2024-04-25T10:14:15.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.105+0000 7f40edc00640 1 -- 172.21.15.119:0/2189584216 wait complete. 2024-04-25T10:14:15.125 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:14:15.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.301+0000 7f7d7b200640 1 Processor -- start 2024-04-25T10:14:15.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.301+0000 7f7d7b200640 1 -- start start 2024-04-25T10:14:15.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.303+0000 7f7d7b200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7d7c15fc90 0x7f7d7c06e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.303+0000 7f7d7b200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7c0cd440 con 0x7f7d7c144470 2024-04-25T10:14:15.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.303+0000 7f7d7b200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7c0cd720 con 0x7f7d7c0b5050 2024-04-25T10:14:15.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.303+0000 7f7d7b200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7c145340 con 0x7f7d7c15fc90 2024-04-25T10:14:15.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.303+0000 7f7d79e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7d7c144470 0x7f7d7c0b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51966/0 (socket says 172.21.15.119:51966) 2024-04-25T10:14:15.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.303+0000 7f7d79e00640 1 -- 172.21.15.119:0/3013555115 learned_addr learned my addr 172.21.15.119:0/3013555115 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:14:15.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d7a800640 1 --2- 172.21.15.119:0/3013555115 >> v2:172.21.15.132:3300/0 conn(0x7f7d7c15fc90 0x7f7d7c06e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 -- 172.21.15.119:0/3013555115 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091702273 0 0) 0x7f7d7c0cd440 con 0x7f7d7c144470 2024-04-25T10:14:15.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 -- 172.21.15.119:0/3013555115 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d60003520 con 0x7f7d7c144470 2024-04-25T10:14:15.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 -- 172.21.15.119:0/3013555115 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1644160857 0 0) 0x7f7d60003520 con 0x7f7d7c144470 2024-04-25T10:14:15.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 -- 172.21.15.119:0/3013555115 >> v2:172.21.15.132:3300/0 conn(0x7f7d7c15fc90 msgr2=0x7f7d7c06e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 --2- 172.21.15.119:0/3013555115 >> v2:172.21.15.132:3300/0 conn(0x7f7d7c15fc90 0x7f7d7c06e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 -- 172.21.15.119:0/3013555115 >> v1:172.21.15.132:6789/0 conn(0x7f7d7c0b5050 legacy=0x7f7d7c15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 -- 172.21.15.119:0/3013555115 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c0cda00 con 0x7f7d7c144470 2024-04-25T10:14:15.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 -- 172.21.15.119:0/3013555115 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f7d6c001670 con 0x7f7d7c144470 2024-04-25T10:14:15.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 -- 172.21.15.119:0/3013555115 >> v1:172.21.15.119:6789/0 conn(0x7f7d7c144470 legacy=0x7f7d7c0b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 --2- 172.21.15.119:0/3013555115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d60003a20 0x7f7d60003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 --2- 172.21.15.119:0/3013555115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d60004800 0x7f7d60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 --2- 172.21.15.119:0/3013555115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d600051c0 0x7f7d6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 -- 172.21.15.119:0/3013555115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6000c0c0 con 0x7f7d60003a20 2024-04-25T10:14:15.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 -- 172.21.15.119:0/3013555115 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6000c270 con 0x7f7d600051c0 2024-04-25T10:14:15.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.304+0000 7f7d78a00640 1 -- 172.21.15.119:0/3013555115 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6000c470 con 0x7f7d60004800 2024-04-25T10:14:15.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.305+0000 7f7d79e00640 1 --2- 172.21.15.119:0/3013555115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d60003a20 0x7f7d60003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.305+0000 7f7d79e00640 1 -- 172.21.15.119:0/3013555115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d60004800 msgr2=0x7f7d60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.305+0000 7f7d79e00640 1 --2- 172.21.15.119:0/3013555115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d60004800 0x7f7d60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.305+0000 7f7d79e00640 1 -- 172.21.15.119:0/3013555115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d600051c0 msgr2=0x7f7d6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.305+0000 7f7d79e00640 1 --2- 172.21.15.119:0/3013555115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d600051c0 0x7f7d6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.305+0000 7f7d79e00640 1 -- 172.21.15.119:0/3013555115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d60003520 con 0x7f7d60003a20 2024-04-25T10:14:15.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.305+0000 7f7d7a800640 1 -- 172.21.15.119:0/3013555115 reap_dead start 2024-04-25T10:14:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.306+0000 7f7d79e00640 1 --2- 172.21.15.119:0/3013555115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d60003a20 0x7f7d60003e40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00c240 tx=0x7f7d60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656b62b5c48f193d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.306+0000 7f7d78a00640 1 -- 172.21.15.119:0/3013555115 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c007cf0 con 0x7f7d60003a20 2024-04-25T10:14:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.306+0000 7f7d78a00640 1 -- 172.21.15.119:0/3013555115 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6c0058d0 con 0x7f7d60003a20 2024-04-25T10:14:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.306+0000 7f7d78a00640 1 -- 172.21.15.119:0/3013555115 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c005d20 con 0x7f7d60003a20 2024-04-25T10:14:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.307+0000 7f7d7b200640 1 -- 172.21.15.119:0/3013555115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d60003a20 msgr2=0x7f7d60003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.307+0000 7f7d7b200640 1 --2- 172.21.15.119:0/3013555115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d60003a20 0x7f7d60003e40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00c240 tx=0x7f7d60005830 comp rx=0 tx=0).stop 2024-04-25T10:14:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.307+0000 7f7d7b200640 1 -- 172.21.15.119:0/3013555115 shutdown_connections 2024-04-25T10:14:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.307+0000 7f7d7b200640 1 --2- 172.21.15.119:0/3013555115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d60003a20 0x7f7d60003e40 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.307+0000 7f7d7b200640 1 -- 172.21.15.119:0/3013555115 >> 172.21.15.119:0/3013555115 conn(0x7f7d7c06b1d0 msgr2=0x7f7d7c0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.307+0000 7f7d7b200640 1 -- 172.21.15.119:0/3013555115 shutdown_connections 2024-04-25T10:14:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.307+0000 7f7d7b200640 1 -- 172.21.15.119:0/3013555115 wait complete. 2024-04-25T10:14:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.309+0000 7f7d7b200640 1 Processor -- start 2024-04-25T10:14:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.309+0000 7f7d7b200640 1 -- start start 2024-04-25T10:14:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d7b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d60003a20 0x7f7d7c1f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d7b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d7c15a8c0 0x7f7d7c1f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d7b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d7c15bbf0 0x7f7d7c15c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d7b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0ce200 con 0x7f7d7c15bbf0 2024-04-25T10:14:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d7b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c144850 con 0x7f7d7c15a8c0 2024-04-25T10:14:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d7b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c06e430 con 0x7f7d60003a20 2024-04-25T10:14:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d7a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d7c15bbf0 0x7f7d7c15c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d7a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d7c15bbf0 0x7f7d7c15c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54830/0 (socket says 172.21.15.119:54830) 2024-04-25T10:14:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d7a800640 1 -- 172.21.15.119:0/618444318 learned_addr learned my addr 172.21.15.119:0/618444318 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d79e00640 1 --2- 172.21.15.119:0/618444318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d60003a20 0x7f7d7c1f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d7a800640 1 -- 172.21.15.119:0/618444318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d60003a20 msgr2=0x7f7d7c1f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d7a800640 1 --2- 172.21.15.119:0/618444318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d60003a20 0x7f7d7c1f3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d7a800640 1 -- 172.21.15.119:0/618444318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d7c15a8c0 msgr2=0x7f7d7c1f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d7a800640 1 --2- 172.21.15.119:0/618444318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d7c15a8c0 0x7f7d7c1f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.310+0000 7f7d7a800640 1 -- 172.21.15.119:0/618444318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c15e670 con 0x7f7d7c15bbf0 2024-04-25T10:14:15.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.311+0000 7f7d7a800640 1 --2- 172.21.15.119:0/618444318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d7c15bbf0 0x7f7d7c15c0a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c06e9a0 tx=0x7f7d7400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.311+0000 7f7d72000640 1 -- 172.21.15.119:0/618444318 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74018020 con 0x7f7d7c15bbf0 2024-04-25T10:14:15.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.311+0000 7f7d72000640 1 -- 172.21.15.119:0/618444318 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7400bb30 con 0x7f7d7c15bbf0 2024-04-25T10:14:15.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.311+0000 7f7d72000640 1 -- 172.21.15.119:0/618444318 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74014400 con 0x7f7d7c15bbf0 2024-04-25T10:14:15.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.311+0000 7f7d7b200640 1 -- 172.21.15.119:0/618444318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c13a710 con 0x7f7d7c15bbf0 2024-04-25T10:14:15.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.311+0000 7f7d7b200640 1 -- 172.21.15.119:0/618444318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c13acd0 con 0x7f7d7c15bbf0 2024-04-25T10:14:15.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.312+0000 7f7d7b200640 1 -- 172.21.15.119:0/618444318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7c0749b0 con 0x7f7d7c15bbf0 2024-04-25T10:14:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.313+0000 7f7d72000640 1 -- 172.21.15.119:0/618444318 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d7401b020 con 0x7f7d7c15bbf0 2024-04-25T10:14:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.315+0000 7f7d72000640 1 --2- 172.21.15.119:0/618444318 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d5c06b500 0x7f7d5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.315+0000 7f7d72000640 1 -- 172.21.15.119:0/618444318 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f7d7408a8f0 con 0x7f7d7c15bbf0 2024-04-25T10:14:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.315+0000 7f7d79e00640 1 --2- 172.21.15.119:0/618444318 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d5c06b500 0x7f7d5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.316+0000 7f7d79e00640 1 --2- 172.21.15.119:0/618444318 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d5c06b500 0x7f7d5c06d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c033000 tx=0x7f7d6c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.319+0000 7f7d72000640 1 -- 172.21.15.119:0/618444318 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d74022030 con 0x7f7d7c15bbf0 2024-04-25T10:14:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.552+0000 7f7d7b200640 1 -- 172.21.15.119:0/618444318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7d7c0b2090 con 0x7f7d7c15bbf0 2024-04-25T10:14:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.553+0000 7f7d72000640 1 -- 172.21.15.119:0/618444318 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f7d74059790 con 0x7f7d7c15bbf0 2024-04-25T10:14:15.559 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:14:15.559 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:14:15.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.555+0000 7f7d7b200640 1 -- 172.21.15.119:0/618444318 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d5c06b500 msgr2=0x7f7d5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.555+0000 7f7d7b200640 1 --2- 172.21.15.119:0/618444318 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d5c06b500 0x7f7d5c06d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c033000 tx=0x7f7d6c009030 comp rx=0 tx=0).stop 2024-04-25T10:14:15.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.556+0000 7f7d7b200640 1 -- 172.21.15.119:0/618444318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d7c15bbf0 msgr2=0x7f7d7c15c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.556+0000 7f7d7b200640 1 --2- 172.21.15.119:0/618444318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d7c15bbf0 0x7f7d7c15c0a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c06e9a0 tx=0x7f7d7400ef50 comp rx=0 tx=0).stop 2024-04-25T10:14:15.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.556+0000 7f7d7b200640 1 -- 172.21.15.119:0/618444318 shutdown_connections 2024-04-25T10:14:15.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.556+0000 7f7d7b200640 1 --2- 172.21.15.119:0/618444318 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d5c06b500 0x7f7d5c06d9c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.556+0000 7f7d7b200640 1 --2- 172.21.15.119:0/618444318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d7c15bbf0 0x7f7d7c15c0a0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.556+0000 7f7d7b200640 1 --2- 172.21.15.119:0/618444318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d7c15a8c0 0x7f7d7c1f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.556+0000 7f7d7b200640 1 --2- 172.21.15.119:0/618444318 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d60003a20 0x7f7d7c1f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.556+0000 7f7d7b200640 1 -- 172.21.15.119:0/618444318 >> 172.21.15.119:0/618444318 conn(0x7f7d7c06b1d0 msgr2=0x7f7d7c06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.556+0000 7f7d7b200640 1 -- 172.21.15.119:0/618444318 shutdown_connections 2024-04-25T10:14:15.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.556+0000 7f7d7b200640 1 -- 172.21.15.119:0/618444318 wait complete. 2024-04-25T10:14:15.578 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:14:15.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:15.579 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T10:14:15.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.786+0000 7fb9dde00640 1 Processor -- start 2024-04-25T10:14:15.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.786+0000 7fb9dde00640 1 -- start start 2024-04-25T10:14:15.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.788+0000 7fb9dde00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb9d8071200 0x7fb9d8071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.788+0000 7fb9dde00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d80cee90 0x7fb9d80cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.788+0000 7fb9dde00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d80cc9a0 con 0x7fb9d8145260 2024-04-25T10:14:15.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.788+0000 7fb9dde00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d8071ba0 con 0x7fb9d8071200 2024-04-25T10:14:15.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.788+0000 7fb9dde00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80cea00 con 0x7fb9d80cee90 2024-04-25T10:14:15.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.788+0000 7fb9d6a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb9d8145260 0x7fb9d80cbf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51980/0 (socket says 172.21.15.119:51980) 2024-04-25T10:14:15.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.788+0000 7fb9d6a00640 1 -- 172.21.15.119:0/4054393850 learned_addr learned my addr 172.21.15.119:0/4054393850 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:14:15.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 -- 172.21.15.119:0/4054393850 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918563949 0 0) 0x7fb9d80cc9a0 con 0x7fb9d8145260 2024-04-25T10:14:15.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 -- 172.21.15.119:0/4054393850 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9bc003520 con 0x7fb9d8145260 2024-04-25T10:14:15.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9cfe00640 1 --2- 172.21.15.119:0/4054393850 >> v2:172.21.15.132:3300/0 conn(0x7fb9d8071200 0x7fb9d8071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d7400640 1 --2- 172.21.15.119:0/4054393850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d80cee90 0x7fb9d80cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 -- 172.21.15.119:0/4054393850 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1632250755 0 0) 0x7fb9bc003520 con 0x7fb9d8145260 2024-04-25T10:14:15.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 -- 172.21.15.119:0/4054393850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d80cee90 msgr2=0x7fb9d80cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 --2- 172.21.15.119:0/4054393850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d80cee90 0x7fb9d80cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 -- 172.21.15.119:0/4054393850 >> v2:172.21.15.132:3300/0 conn(0x7fb9d8071200 msgr2=0x7fb9d8071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 --2- 172.21.15.119:0/4054393850 >> v2:172.21.15.132:3300/0 conn(0x7fb9d8071200 0x7fb9d8071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 -- 172.21.15.119:0/4054393850 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d80ccc80 con 0x7fb9d8145260 2024-04-25T10:14:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 -- 172.21.15.119:0/4054393850 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fb9c8001670 con 0x7fb9d8145260 2024-04-25T10:14:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 -- 172.21.15.119:0/4054393850 >> v1:172.21.15.119:6789/0 conn(0x7fb9d8145260 legacy=0x7fb9d80cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 --2- 172.21.15.119:0/4054393850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9bc0039b0 0x7fb9bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 --2- 172.21.15.119:0/4054393850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9bc004800 0x7fb9bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 --2- 172.21.15.119:0/4054393850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9bc0051c0 0x7fb9bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 -- 172.21.15.119:0/4054393850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80cea00 con 0x7fb9bc004800 2024-04-25T10:14:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 -- 172.21.15.119:0/4054393850 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc00bfb0 con 0x7fb9bc0039b0 2024-04-25T10:14:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d6000640 1 -- 172.21.15.119:0/4054393850 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc00c1b0 con 0x7fb9bc0051c0 2024-04-25T10:14:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9cfe00640 1 --2- 172.21.15.119:0/4054393850 >> v2:172.21.15.132:3300/0 conn(0x7fb9d8071200 0x7fb9d8071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.789+0000 7fb9d7400640 1 --2- 172.21.15.119:0/4054393850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d80cee90 0x7fb9d80cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.790+0000 7fb9cfe00640 1 --2- 172.21.15.119:0/4054393850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9bc004800 0x7fb9bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.790+0000 7fb9d6a00640 1 --2- 172.21.15.119:0/4054393850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9bc0039b0 0x7fb9bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.790+0000 7fb9cfe00640 1 -- 172.21.15.119:0/4054393850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9bc0051c0 msgr2=0x7fb9bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.790+0000 7fb9cfe00640 1 --2- 172.21.15.119:0/4054393850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9bc0051c0 0x7fb9bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.790+0000 7fb9cfe00640 1 -- 172.21.15.119:0/4054393850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9bc0039b0 msgr2=0x7fb9bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.790+0000 7fb9cfe00640 1 --2- 172.21.15.119:0/4054393850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9bc0039b0 0x7fb9bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.790+0000 7fb9cfe00640 1 -- 172.21.15.119:0/4054393850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc003520 con 0x7fb9bc004800 2024-04-25T10:14:15.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.791+0000 7fb9cfe00640 1 --2- 172.21.15.119:0/4054393850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9bc004800 0x7fb9bc004c80 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00098a0 tx=0x7fb9bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9efba674bb15246 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.791+0000 7fb9d6000640 1 -- 172.21.15.119:0/4054393850 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0037070 con 0x7fb9bc004800 2024-04-25T10:14:15.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.791+0000 7fb9d6000640 1 -- 172.21.15.119:0/4054393850 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c002ad20 con 0x7fb9bc004800 2024-04-25T10:14:15.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.791+0000 7fb9d6000640 1 -- 172.21.15.119:0/4054393850 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c000f040 con 0x7fb9bc004800 2024-04-25T10:14:15.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.792+0000 7fb9dde00640 1 -- 172.21.15.119:0/4054393850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9bc004800 msgr2=0x7fb9bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.792+0000 7fb9dde00640 1 --2- 172.21.15.119:0/4054393850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9bc004800 0x7fb9bc004c80 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00098a0 tx=0x7fb9bc005830 comp rx=0 tx=0).stop 2024-04-25T10:14:15.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.792+0000 7fb9d7400640 1 -- 172.21.15.119:0/4054393850 reap_dead start 2024-04-25T10:14:15.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.792+0000 7fb9dde00640 1 -- 172.21.15.119:0/4054393850 shutdown_connections 2024-04-25T10:14:15.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.792+0000 7fb9dde00640 1 -- 172.21.15.119:0/4054393850 >> 172.21.15.119:0/4054393850 conn(0x7fb9d806aba0 msgr2=0x7fb9d806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.792+0000 7fb9dde00640 1 -- 172.21.15.119:0/4054393850 shutdown_connections 2024-04-25T10:14:15.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.792+0000 7fb9dde00640 1 -- 172.21.15.119:0/4054393850 wait complete. 2024-04-25T10:14:15.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.794+0000 7fb9dde00640 1 Processor -- start 2024-04-25T10:14:15.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.794+0000 7fb9dde00640 1 -- start start 2024-04-25T10:14:15.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.795+0000 7fb9dde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9bc004800 0x7fb9d81f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.795+0000 7fb9dde00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d80b3240 0x7fb9d815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.795+0000 7fb9dde00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9d80b36c0 0x7fb9d815fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.795+0000 7fb9dde00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80cc660 con 0x7fb9bc004800 2024-04-25T10:14:15.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.795+0000 7fb9dde00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80cd240 con 0x7fb9d80b36c0 2024-04-25T10:14:15.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.795+0000 7fb9dde00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d8145640 con 0x7fb9d80b3240 2024-04-25T10:14:15.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.796+0000 7fb9d6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9bc004800 0x7fb9d81f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.796+0000 7fb9d6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9bc004800 0x7fb9d81f6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54848/0 (socket says 172.21.15.119:54848) 2024-04-25T10:14:15.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.796+0000 7fb9d6a00640 1 -- 172.21.15.119:0/2818335916 learned_addr learned my addr 172.21.15.119:0/2818335916 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:15.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.796+0000 7fb9d6a00640 1 -- 172.21.15.119:0/2818335916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d80b3240 msgr2=0x7fb9d815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.796+0000 7fb9d6a00640 1 --2- 172.21.15.119:0/2818335916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d80b3240 0x7fb9d815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.796+0000 7fb9d6a00640 1 -- 172.21.15.119:0/2818335916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9d80b36c0 msgr2=0x7fb9d815fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.796+0000 7fb9d6a00640 1 --2- 172.21.15.119:0/2818335916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9d80b36c0 0x7fb9d815fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.796+0000 7fb9d6a00640 1 -- 172.21.15.119:0/2818335916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d80b3c00 con 0x7fb9bc004800 2024-04-25T10:14:15.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.796+0000 7fb9d6a00640 1 --2- 172.21.15.119:0/2818335916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9bc004800 0x7fb9d81f6930 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800e780 tx=0x7fb9c800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.797+0000 7fb9cf400640 1 -- 172.21.15.119:0/2818335916 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c800ee90 con 0x7fb9bc004800 2024-04-25T10:14:15.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.797+0000 7fb9cf400640 1 -- 172.21.15.119:0/2818335916 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c8018070 con 0x7fb9bc004800 2024-04-25T10:14:15.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.797+0000 7fb9cf400640 1 -- 172.21.15.119:0/2818335916 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c800ca90 con 0x7fb9bc004800 2024-04-25T10:14:15.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.797+0000 7fb9dde00640 1 -- 172.21.15.119:0/2818335916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d80b3ee0 con 0x7fb9bc004800 2024-04-25T10:14:15.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.797+0000 7fb9dde00640 1 -- 172.21.15.119:0/2818335916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d80b44a0 con 0x7fb9bc004800 2024-04-25T10:14:15.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.799+0000 7fb9cf400640 1 -- 172.21.15.119:0/2818335916 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9c800cc30 con 0x7fb9bc004800 2024-04-25T10:14:15.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.800+0000 7fb9cf400640 1 --2- 172.21.15.119:0/2818335916 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb9b406b670 0x7fb9b406db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.801+0000 7fb9cf400640 1 -- 172.21.15.119:0/2818335916 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fb9c808a1e0 con 0x7fb9bc004800 2024-04-25T10:14:15.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.801+0000 7fb9cfe00640 1 --2- 172.21.15.119:0/2818335916 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb9b406b670 0x7fb9b406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.801+0000 7fb9e0bda740 1 Processor -- start 2024-04-25T10:14:15.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.801+0000 7fb9e0bda740 1 -- start start 2024-04-25T10:14:15.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.801+0000 7fb9cfe00640 1 --2- 172.21.15.119:0/2818335916 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb9b406b670 0x7fb9b406db30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00098a0 tx=0x7fb9c002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.802+0000 7fb9e0bda740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 0x55ee9415e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.802+0000 7fb9e0bda740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee9417bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.802+0000 7fb9e0bda740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee94133e80 0x55ee9417a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.802+0000 7fb9e0bda740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee94096980 con 0x55ee9417ba40 2024-04-25T10:14:15.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.802+0000 7fb9e0bda740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee9403fde0 con 0x55ee94133e80 2024-04-25T10:14:15.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.802+0000 7fb9e0bda740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee94092a20 con 0x55ee9415e1a0 2024-04-25T10:14:15.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.802+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee9417bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.802+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee9417bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54850/0 (socket says 172.21.15.119:54850) 2024-04-25T10:14:15.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.802+0000 7fb9cfe00640 1 -- 172.21.15.119:0/399231303 learned_addr learned my addr 172.21.15.119:0/399231303 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:15.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.803+0000 7fb9d7400640 1 --2- 172.21.15.119:0/399231303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee94133e80 0x55ee9417a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.803+0000 7fb9cfe00640 1 -- 172.21.15.119:0/399231303 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 msgr2=0x55ee9415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.803+0000 7fb9d6a00640 1 --2- 172.21.15.119:0/399231303 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 0x55ee9415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.803+0000 7fb9cfe00640 1 --2- 172.21.15.119:0/399231303 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 0x55ee9415e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.803+0000 7fb9cfe00640 1 -- 172.21.15.119:0/399231303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee94133e80 msgr2=0x55ee9417a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.803+0000 7fb9cfe00640 1 --2- 172.21.15.119:0/399231303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee94133e80 0x55ee9417a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.803+0000 7fb9cfe00640 1 -- 172.21.15.119:0/399231303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee9405de30 con 0x55ee9417ba40 2024-04-25T10:14:15.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.803+0000 7fb9d7400640 1 --2- 172.21.15.119:0/399231303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee94133e80 0x55ee9417a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:15.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.803+0000 7fb9cfe00640 1 --2- 172.21.15.119:0/399231303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee9417bf90 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0035040 tx=0x7fb9c002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc04c0d148353d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.804+0000 7fb9c7400640 1 -- 172.21.15.119:0/399231303 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c004f070 con 0x55ee9417ba40 2024-04-25T10:14:15.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.804+0000 7fb9c7400640 1 -- 172.21.15.119:0/399231303 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c0038040 con 0x55ee9417ba40 2024-04-25T10:14:15.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.804+0000 7fb9c7400640 1 -- 172.21.15.119:0/399231303 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c003fb30 con 0x55ee9417ba40 2024-04-25T10:14:15.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.804+0000 7fb9e0bda740 1 -- 172.21.15.119:0/399231303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 msgr2=0x55ee9417bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.804+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/399231303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee9417bf90 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0035040 tx=0x7fb9c002f000 comp rx=0 tx=0).stop 2024-04-25T10:14:15.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.804+0000 7fb9e0bda740 1 -- 172.21.15.119:0/399231303 shutdown_connections 2024-04-25T10:14:15.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.804+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/399231303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee94133e80 0x55ee9417a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.804+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/399231303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee9417bf90 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.804+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/399231303 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 0x55ee9415e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.804+0000 7fb9e0bda740 1 -- 172.21.15.119:0/399231303 >> 172.21.15.119:0/399231303 conn(0x55ee941307b0 msgr2=0x55ee94178fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.804+0000 7fb9e0bda740 1 -- 172.21.15.119:0/399231303 shutdown_connections 2024-04-25T10:14:15.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.804+0000 7fb9e0bda740 1 -- 172.21.15.119:0/399231303 wait complete. 2024-04-25T10:14:15.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.805+0000 7fb9e0bda740 1 -- start start 2024-04-25T10:14:15.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.805+0000 7fb9e0bda740 1 Processor -- start 2024-04-25T10:14:15.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.806+0000 7fb9e0bda740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 0x55ee94189800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.806+0000 7fb9e0bda740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9417ba40 0x55ee9415a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.806+0000 7fb9e0bda740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9415a8d0 0x55ee941727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.806+0000 7fb9e0bda740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee94134c20 con 0x55ee9415a8d0 2024-04-25T10:14:15.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.806+0000 7fb9e0bda740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee940353c0 con 0x55ee9417ba40 2024-04-25T10:14:15.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.806+0000 7fb9e0bda740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee9412e6e0 con 0x55ee9415e1a0 2024-04-25T10:14:15.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.806+0000 7fb9e0bda740 10 client.0 _ll_register_callbacks cb 0x55ee93daac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:15.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.807+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9415a8d0 0x55ee941727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.807+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9415a8d0 0x55ee941727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54866/0 (socket says 172.21.15.119:54866) 2024-04-25T10:14:15.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.807+0000 7fb9d7400640 1 -- 172.21.15.119:0/1718042758 learned_addr learned my addr 172.21.15.119:0/1718042758 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:15.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.807+0000 7fb9d6a00640 1 --2- 172.21.15.119:0/1718042758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 0x55ee94189800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.807+0000 7fb9cfe00640 1 --2- 172.21.15.119:0/1718042758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9417ba40 0x55ee9415a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.807+0000 7fb9d7400640 1 -- 172.21.15.119:0/1718042758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 msgr2=0x55ee94189800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.807+0000 7fb9d7400640 1 --2- 172.21.15.119:0/1718042758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 0x55ee94189800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.807+0000 7fb9d7400640 1 -- 172.21.15.119:0/1718042758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9417ba40 msgr2=0x55ee9415a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.807+0000 7fb9d7400640 1 --2- 172.21.15.119:0/1718042758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9417ba40 0x55ee9415a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.807+0000 7fb9d7400640 1 -- 172.21.15.119:0/1718042758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee93e737e0 con 0x55ee9415a8d0 2024-04-25T10:14:15.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.807+0000 7fb9d7400640 1 --2- 172.21.15.119:0/1718042758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9415a8d0 0x55ee941727b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8072490 tx=0x7fb9d0008c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6d3f1adee1dd46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.807+0000 7fb9bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:15.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.808+0000 7fb9bbe00640 1 -- 172.21.15.119:0/1718042758 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0019060 con 0x55ee9415a8d0 2024-04-25T10:14:15.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.808+0000 7fb9bbe00640 1 -- 172.21.15.119:0/1718042758 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fb9d0007ad0 con 0x55ee9415a8d0 2024-04-25T10:14:15.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.808+0000 7fb9e0bda740 1 -- 172.21.15.119:0/1718042758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9c0038040 con 0x55ee9415a8d0 2024-04-25T10:14:15.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.808+0000 7fb9bbe00640 1 -- 172.21.15.119:0/1718042758 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9d001f050 con 0x55ee9415a8d0 2024-04-25T10:14:15.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.808+0000 7fb9e0bda740 10 client.7554 fetch_fsmap learned FSMap version 68 2024-04-25T10:14:15.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.808+0000 7fb9e0bda740 1 -- 172.21.15.119:0/1718042758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55ee93ed3f70 con 0x55ee9415a8d0 2024-04-25T10:14:15.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.808+0000 7fb9bbe00640 1 -- 172.21.15.119:0/1718042758 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c0038040 con 0x55ee9415a8d0 2024-04-25T10:14:15.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.808+0000 7fb9bbe00640 1 -- 172.21.15.119:0/1718042758 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb9d0022020 con 0x55ee9415a8d0 2024-04-25T10:14:15.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.808+0000 7fb9e0bda740 10 client.7554 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:14:15.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.808+0000 7fb9e0bda740 10 client.7554 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:14:15.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.808+0000 7fb9e0bda740 20 client.7554 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:15.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.808+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/1718042758 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55ee9412da20 0x55ee9415feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.808+0000 7fb9e0bda740 4 client.7554 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:15.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.808+0000 7fb9e0bda740 1 -- 172.21.15.119:0/1718042758 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee940a3ec0 con 0x55ee9412da20 2024-04-25T10:14:15.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.809+0000 7fb9d6a00640 1 --2- 172.21.15.119:0/1718042758 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55ee9412da20 0x55ee9415feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.809+0000 7fb9d6a00640 1 --2- 172.21.15.119:0/1718042758 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55ee9412da20 0x55ee9415feb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=764c591943ab8f76 server_cookie=f6a7a3e495571cb1 in_seq=0 out_seq=0 2024-04-25T10:14:15.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.809+0000 7fb9bbe00640 10 client.7554 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:15.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.810+0000 7fb9bbe00640 1 -- 172.21.15.119:0/1718042758 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee940a3ec0 con 0x55ee9412da20 2024-04-25T10:14:15.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.810+0000 7fb9bbe00640 10 client.7554 handle_command_reply: tid=0 2024-04-25T10:14:15.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.810+0000 7fb9e0bda740 1 client.7554 shutdown 2024-04-25T10:14:15.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.810+0000 7fb9e0bda740 1 -- 172.21.15.119:0/1718042758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9415a8d0 msgr2=0x55ee941727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.810+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/1718042758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9415a8d0 0x55ee941727b0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8072490 tx=0x7fb9d0008c90 comp rx=0 tx=0).stop 2024-04-25T10:14:15.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.810+0000 7fb9e0bda740 1 -- 172.21.15.119:0/1718042758 shutdown_connections 2024-04-25T10:14:15.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.810+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/1718042758 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55ee9412da20 0x55ee9415feb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.810+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/1718042758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9415a8d0 0x55ee941727b0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.810+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/1718042758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9417ba40 0x55ee9415a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.810+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/1718042758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 0x55ee94189800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.810+0000 7fb9bbe00640 0 client.7554 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:15.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.810+0000 7fb9e0bda740 1 -- 172.21.15.119:0/1718042758 >> 172.21.15.119:0/1718042758 conn(0x55ee94133e80 msgr2=0x55ee9415d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.810+0000 7fb9e0bda740 1 -- 172.21.15.119:0/1718042758 shutdown_connections 2024-04-25T10:14:15.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.811+0000 7fb9e0bda740 1 -- 172.21.15.119:0/1718042758 wait complete. 2024-04-25T10:14:15.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.811+0000 7fb9e0bda740 20 client.7554 trim_cache size 0 max 16384 2024-04-25T10:14:15.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.827+0000 7fb9e0bda740 1 Processor -- start 2024-04-25T10:14:15.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.827+0000 7fb9e0bda740 1 -- start start 2024-04-25T10:14:15.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.828+0000 7fb9e0bda740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415a8d0 0x55ee941590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.828+0000 7fb9e0bda740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9415e1a0 0x55ee94162080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.828+0000 7fb9e0bda740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee94133320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.828+0000 7fb9e0bda740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d8200350 con 0x55ee9417ba40 2024-04-25T10:14:15.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.828+0000 7fb9e0bda740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d81ff580 con 0x55ee9415e1a0 2024-04-25T10:14:15.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.828+0000 7fb9e0bda740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d81f4990 con 0x55ee9415a8d0 2024-04-25T10:14:15.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.828+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee94133320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.828+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee94133320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54876/0 (socket says 172.21.15.119:54876) 2024-04-25T10:14:15.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.828+0000 7fb9d7400640 1 -- 172.21.15.119:0/4182929161 learned_addr learned my addr 172.21.15.119:0/4182929161 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:15.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.829+0000 7fb9d7400640 1 -- 172.21.15.119:0/4182929161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415a8d0 msgr2=0x55ee941590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.829+0000 7fb9d7400640 1 --2- 172.21.15.119:0/4182929161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415a8d0 0x55ee941590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.829+0000 7fb9d7400640 1 -- 172.21.15.119:0/4182929161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9415e1a0 msgr2=0x55ee94162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.829+0000 7fb9d7400640 1 --2- 172.21.15.119:0/4182929161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9415e1a0 0x55ee94162080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.829+0000 7fb9d7400640 1 -- 172.21.15.119:0/4182929161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee9415a4f0 con 0x55ee9417ba40 2024-04-25T10:14:15.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.829+0000 7fb9d7400640 1 --2- 172.21.15.119:0/4182929161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee94133320 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009f60 tx=0x7fb9d0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c424851362c11b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.829+0000 7fb9c6a00640 1 -- 172.21.15.119:0/4182929161 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00078d0 con 0x55ee9417ba40 2024-04-25T10:14:15.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.829+0000 7fb9c6a00640 1 -- 172.21.15.119:0/4182929161 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d0027020 con 0x55ee9417ba40 2024-04-25T10:14:15.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.829+0000 7fb9c6a00640 1 -- 172.21.15.119:0/4182929161 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0007c70 con 0x55ee9417ba40 2024-04-25T10:14:15.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.830+0000 7fb9e0bda740 1 -- 172.21.15.119:0/4182929161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 msgr2=0x55ee94133320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.830+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/4182929161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee94133320 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009f60 tx=0x7fb9d0012000 comp rx=0 tx=0).stop 2024-04-25T10:14:15.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.830+0000 7fb9e0bda740 1 -- 172.21.15.119:0/4182929161 shutdown_connections 2024-04-25T10:14:15.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.830+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/4182929161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee94133320 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.830+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/4182929161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9415e1a0 0x55ee94162080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.830+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/4182929161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415a8d0 0x55ee941590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.830+0000 7fb9e0bda740 1 -- 172.21.15.119:0/4182929161 >> 172.21.15.119:0/4182929161 conn(0x55ee94133e80 msgr2=0x55ee941297d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.830+0000 7fb9e0bda740 1 -- 172.21.15.119:0/4182929161 shutdown_connections 2024-04-25T10:14:15.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.830+0000 7fb9e0bda740 1 -- 172.21.15.119:0/4182929161 wait complete. 2024-04-25T10:14:15.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.831+0000 7fb9e0bda740 1 -- start start 2024-04-25T10:14:15.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.831+0000 7fb9e0bda740 1 Processor -- start 2024-04-25T10:14:15.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.832+0000 7fb9e0bda740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9415a8d0 0x55ee94191b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.832+0000 7fb9e0bda740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 0x55ee9418fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.832+0000 7fb9e0bda740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee94190060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.832+0000 7fb9e0bda740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee9412f7b0 con 0x55ee9417ba40 2024-04-25T10:14:15.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.832+0000 7fb9e0bda740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d81f4f90 con 0x55ee9415a8d0 2024-04-25T10:14:15.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.832+0000 7fb9e0bda740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee9403bf50 con 0x55ee9415e1a0 2024-04-25T10:14:15.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.832+0000 7fb9e0bda740 10 client.0 _ll_register_callbacks cb 0x55ee9412ebc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:15.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.832+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee94190060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.832+0000 7fb9d7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee94190060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54882/0 (socket says 172.21.15.119:54882) 2024-04-25T10:14:15.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.832+0000 7fb9d7400640 1 -- 172.21.15.119:0/3862187281 learned_addr learned my addr 172.21.15.119:0/3862187281 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:15.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.832+0000 7fb9cfe00640 1 --2- 172.21.15.119:0/3862187281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 0x55ee9418fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.833+0000 7fb9d6a00640 1 --2- 172.21.15.119:0/3862187281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9415a8d0 0x55ee94191b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.833+0000 7fb9d7400640 1 -- 172.21.15.119:0/3862187281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 msgr2=0x55ee9418fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.833+0000 7fb9d7400640 1 --2- 172.21.15.119:0/3862187281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 0x55ee9418fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.833+0000 7fb9d7400640 1 -- 172.21.15.119:0/3862187281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9415a8d0 msgr2=0x55ee94191b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:15.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.833+0000 7fb9d7400640 1 --2- 172.21.15.119:0/3862187281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9415a8d0 0x55ee94191b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.833+0000 7fb9d7400640 1 -- 172.21.15.119:0/3862187281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee93d6b170 con 0x55ee9417ba40 2024-04-25T10:14:15.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.833+0000 7fb9d6a00640 1 --2- 172.21.15.119:0/3862187281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9415a8d0 0x55ee94191b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:15.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.833+0000 7fb9d7400640 1 --2- 172.21.15.119:0/3862187281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee94190060 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0016040 tx=0x7fb9d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b108e82ebb65abb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:15.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.833+0000 7fb9bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:15.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.833+0000 7fb9bbe00640 1 -- 172.21.15.119:0/3862187281 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0031030 con 0x55ee9417ba40 2024-04-25T10:14:15.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.833+0000 7fb9bbe00640 1 -- 172.21.15.119:0/3862187281 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fb9d0028040 con 0x55ee9417ba40 2024-04-25T10:14:15.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.833+0000 7fb9e0bda740 1 -- 172.21.15.119:0/3862187281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9d0027020 con 0x55ee9417ba40 2024-04-25T10:14:15.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.834+0000 7fb9bbe00640 1 -- 172.21.15.119:0/3862187281 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9d003d050 con 0x55ee9417ba40 2024-04-25T10:14:15.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.834+0000 7fb9e0bda740 10 client.7560 fetch_fsmap learned FSMap version 68 2024-04-25T10:14:15.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.834+0000 7fb9e0bda740 1 -- 172.21.15.119:0/3862187281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55ee94188b10 con 0x55ee9417ba40 2024-04-25T10:14:15.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.834+0000 7fb9bbe00640 1 -- 172.21.15.119:0/3862187281 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d0027020 con 0x55ee9417ba40 2024-04-25T10:14:15.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.834+0000 7fb9bbe00640 1 -- 172.21.15.119:0/3862187281 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb9d0040020 con 0x55ee9417ba40 2024-04-25T10:14:15.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.834+0000 7fb9e0bda740 10 client.7560 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:14:15.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.834+0000 7fb9e0bda740 10 client.7560 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:14:15.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.834+0000 7fb9e0bda740 20 client.7560 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:15.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.834+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/3862187281 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55ee941ddb30 0x55ee9416e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:15.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.834+0000 7fb9e0bda740 4 client.7560 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:14:15.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.834+0000 7fb9e0bda740 1 -- 172.21.15.119:0/3862187281 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x55ee9409e210 con 0x55ee941ddb30 2024-04-25T10:14:15.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.834+0000 7fb9d6a00640 1 --2- 172.21.15.119:0/3862187281 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55ee941ddb30 0x55ee9416e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:15.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.835+0000 7fb9d6a00640 1 --2- 172.21.15.119:0/3862187281 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55ee941ddb30 0x55ee9416e940 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23dac661d0c1e4c2 server_cookie=e2805fad2a14e97a in_seq=0 out_seq=0 2024-04-25T10:14:15.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.835+0000 7fb9bbe00640 10 client.7560 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:15.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.835+0000 7fb9bbe00640 1 -- 172.21.15.119:0/3862187281 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ee9409e210 con 0x55ee941ddb30 2024-04-25T10:14:15.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.835+0000 7fb9bbe00640 10 client.7560 handle_command_reply: tid=0 2024-04-25T10:14:15.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.835+0000 7fb9e0bda740 1 client.7560 shutdown 2024-04-25T10:14:15.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.836+0000 7fb9e0bda740 1 -- 172.21.15.119:0/3862187281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 msgr2=0x55ee94190060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.836+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/3862187281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee94190060 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0016040 tx=0x7fb9d0004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:15.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.836+0000 7fb9e0bda740 1 -- 172.21.15.119:0/3862187281 shutdown_connections 2024-04-25T10:14:15.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.836+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/3862187281 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55ee941ddb30 0x55ee9416e940 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.836+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/3862187281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee9417ba40 0x55ee94190060 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.836+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/3862187281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ee9415e1a0 0x55ee9418fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.836+0000 7fb9e0bda740 1 --2- 172.21.15.119:0/3862187281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee9415a8d0 0x55ee94191b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.836+0000 7fb9e0bda740 1 -- 172.21.15.119:0/3862187281 >> 172.21.15.119:0/3862187281 conn(0x55ee94133e80 msgr2=0x55ee9417c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.836+0000 7fb9bbe00640 0 client.7560 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:15.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.836+0000 7fb9e0bda740 1 -- 172.21.15.119:0/3862187281 shutdown_connections 2024-04-25T10:14:15.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.836+0000 7fb9e0bda740 1 -- 172.21.15.119:0/3862187281 wait complete. 2024-04-25T10:14:15.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.836+0000 7fb9e0bda740 20 client.7560 trim_cache size 0 max 16384 2024-04-25T10:14:15.845 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:14:15.836+0000 7f5363c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv18 ap=3 DIRTYPARENT f(v0 m2024-04-25T10:13:32.156889+0000 1=1+0)->f(v1 m2024-04-25T10:13:32.156889+0000) n(v0 rc2024-04-25T10:13:32.264886+0000 2=1+1)->n(v0 rc2024-04-25T10:13:32.264886+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x561f3d610100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:13:32.156889+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:13:32.264886+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T10:14:15.845 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:14:15.845 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2024-04-25T10:14:15.845 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "977ce22f-8a0b-49dc-a917-68e7f9d7f2f5", 2024-04-25T10:14:15.846 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2024-04-25T10:14:15.846 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:14:15.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.837+0000 7fb9dde00640 1 -- 172.21.15.119:0/2818335916 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb9b406b670 msgr2=0x7fb9b406db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.837+0000 7fb9dde00640 1 --2- 172.21.15.119:0/2818335916 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb9b406b670 0x7fb9b406db30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00098a0 tx=0x7fb9c002c000 comp rx=0 tx=0).stop 2024-04-25T10:14:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.837+0000 7fb9dde00640 1 -- 172.21.15.119:0/2818335916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9bc004800 msgr2=0x7fb9d81f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.837+0000 7fb9dde00640 1 --2- 172.21.15.119:0/2818335916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9bc004800 0x7fb9d81f6930 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800e780 tx=0x7fb9c800ec40 comp rx=0 tx=0).stop 2024-04-25T10:14:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.837+0000 7fb9dde00640 1 -- 172.21.15.119:0/2818335916 shutdown_connections 2024-04-25T10:14:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.837+0000 7fb9dde00640 1 --2- 172.21.15.119:0/2818335916 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb9b406b670 0x7fb9b406db30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.837+0000 7fb9dde00640 1 --2- 172.21.15.119:0/2818335916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9d80b36c0 0x7fb9d815fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.837+0000 7fb9dde00640 1 --2- 172.21.15.119:0/2818335916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d80b3240 0x7fb9d815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.837+0000 7fb9dde00640 1 --2- 172.21.15.119:0/2818335916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9bc004800 0x7fb9d81f6930 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.837+0000 7fb9dde00640 1 -- 172.21.15.119:0/2818335916 >> 172.21.15.119:0/2818335916 conn(0x7fb9d806aba0 msgr2=0x7fb9d806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.837+0000 7fb9dde00640 1 -- 172.21.15.119:0/2818335916 shutdown_connections 2024-04-25T10:14:15.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:15.837+0000 7fb9dde00640 1 -- 172.21.15.119:0/2818335916 wait complete. 2024-04-25T10:14:15.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:15.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:14:16.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.059+0000 7f88a3c00640 1 Processor -- start 2024-04-25T10:14:16.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.060+0000 7f88a3c00640 1 -- start start 2024-04-25T10:14:16.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.061+0000 7f88a3c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f889c071930 0x7f889c15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.061+0000 7f88a3c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f889c159e00 0x7f889c15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.061+0000 7f88a3c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889c0b3670 con 0x7f889c159a20 2024-04-25T10:14:16.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.061+0000 7f88a3c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889c0b41f0 con 0x7f889c159e00 2024-04-25T10:14:16.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.061+0000 7f88a3c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0b3950 con 0x7f889c071930 2024-04-25T10:14:16.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.062+0000 7f889be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f889c071930 0x7f889c15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.062+0000 7f88a1c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f889c159e00 0x7f889c15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.062+0000 7f889be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f889c071930 0x7f889c15a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:53094/0 (socket says 172.21.15.119:53094) 2024-04-25T10:14:16.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.062+0000 7f88a1c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f889c159e00 0x7f889c15ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:55140/0 (socket says 172.21.15.119:55140) 2024-04-25T10:14:16.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.062+0000 7f889be00640 1 -- 172.21.15.119:0/2032858515 learned_addr learned my addr 172.21.15.119:0/2032858515 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:16.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.063+0000 7f889be00640 1 -- 172.21.15.119:0/2032858515 >> v2:172.21.15.132:3300/0 conn(0x7f889c159e00 msgr2=0x7f889c15ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.063+0000 7f889be00640 1 --2- 172.21.15.119:0/2032858515 >> v2:172.21.15.132:3300/0 conn(0x7f889c159e00 0x7f889c15ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.063+0000 7f889be00640 1 -- 172.21.15.119:0/2032858515 >> v1:172.21.15.132:6789/0 conn(0x7f889c159a20 legacy=0x7f889c071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.063+0000 7f889be00640 1 -- 172.21.15.119:0/2032858515 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c0b3c30 con 0x7f889c071930 2024-04-25T10:14:16.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.064+0000 7f889be00640 1 --2- 172.21.15.119:0/2032858515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f889c071930 0x7f889c15a5c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f888c009870 tx=0x7f888c02c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b6748e8262b1f01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.065+0000 7f889b400640 1 -- 172.21.15.119:0/2032858515 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c038070 con 0x7f889c071930 2024-04-25T10:14:16.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.065+0000 7f889b400640 1 -- 172.21.15.119:0/2032858515 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c031710 con 0x7f889c071930 2024-04-25T10:14:16.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.065+0000 7f889b400640 1 -- 172.21.15.119:0/2032858515 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c031ae0 con 0x7f889c071930 2024-04-25T10:14:16.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.065+0000 7f88a3c00640 1 -- 172.21.15.119:0/2032858515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f889c071930 msgr2=0x7f889c15a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.065+0000 7f88a3c00640 1 --2- 172.21.15.119:0/2032858515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f889c071930 0x7f889c15a5c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f888c009870 tx=0x7f888c02c3d0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.065+0000 7f88a3c00640 1 -- 172.21.15.119:0/2032858515 shutdown_connections 2024-04-25T10:14:16.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.065+0000 7f88a3c00640 1 --2- 172.21.15.119:0/2032858515 >> v2:172.21.15.132:3300/0 conn(0x7f889c159e00 0x7f889c15ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.065+0000 7f88a3c00640 1 --2- 172.21.15.119:0/2032858515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f889c071930 0x7f889c15a5c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.065+0000 7f88a3c00640 1 -- 172.21.15.119:0/2032858515 >> 172.21.15.119:0/2032858515 conn(0x7f889c06aba0 msgr2=0x7f889c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:16.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.066+0000 7f88a3c00640 1 -- 172.21.15.119:0/2032858515 shutdown_connections 2024-04-25T10:14:16.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.066+0000 7f88a3c00640 1 -- 172.21.15.119:0/2032858515 wait complete. 2024-04-25T10:14:16.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.067+0000 7f88a3c00640 1 Processor -- start 2024-04-25T10:14:16.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.068+0000 7f88a3c00640 1 -- start start 2024-04-25T10:14:16.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.068+0000 7f88a3c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f889c071930 0x7f889c15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.068+0000 7f88a3c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f889c159a20 0x7f889c0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.068+0000 7f88a3c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c159e00 0x7f889c0addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.068+0000 7f88a3c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c06f310 con 0x7f889c159e00 2024-04-25T10:14:16.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.068+0000 7f88a3c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c071fc0 con 0x7f889c159a20 2024-04-25T10:14:16.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.068+0000 7f88a3c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c06f490 con 0x7f889c071930 2024-04-25T10:14:16.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.069+0000 7f88a1c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c159e00 0x7f889c0addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.069+0000 7f88a1c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c159e00 0x7f889c0addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54888/0 (socket says 172.21.15.119:54888) 2024-04-25T10:14:16.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.069+0000 7f88a1c00640 1 -- 172.21.15.119:0/4158979664 learned_addr learned my addr 172.21.15.119:0/4158979664 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:16.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.069+0000 7f88a1200640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f889c071930 0x7f889c15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.069+0000 7f889be00640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f889c159a20 0x7f889c0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.069+0000 7f88a1c00640 1 -- 172.21.15.119:0/4158979664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f889c071930 msgr2=0x7f889c15a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.069+0000 7f88a1c00640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f889c071930 0x7f889c15a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.069+0000 7f88a1c00640 1 -- 172.21.15.119:0/4158979664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f889c159a20 msgr2=0x7f889c0cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.069+0000 7f88a1c00640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f889c159a20 0x7f889c0cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.069+0000 7f88a1c00640 1 -- 172.21.15.119:0/4158979664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c1f4930 con 0x7f889c159e00 2024-04-25T10:14:16.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.069+0000 7f889be00640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f889c159a20 0x7f889c0cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:16.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.070+0000 7f88a1c00640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c159e00 0x7f889c0addb0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f8894009d30 tx=0x7f889400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.070+0000 7f8898c00640 1 -- 172.21.15.119:0/4158979664 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894018020 con 0x7f889c159e00 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.070+0000 7f8898c00640 1 -- 172.21.15.119:0/4158979664 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8894014480 con 0x7f889c159e00 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.070+0000 7f8898c00640 1 -- 172.21.15.119:0/4158979664 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894014870 con 0x7f889c159e00 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.070+0000 7f88a3c00640 1 -- 172.21.15.119:0/4158979664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889c1f3db0 con 0x7f889c159e00 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.070+0000 7f88a3c00640 1 -- 172.21.15.119:0/4158979664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889c1f4370 con 0x7f889c159e00 2024-04-25T10:14:16.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.072+0000 7f8898c00640 1 -- 172.21.15.119:0/4158979664 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8894005d10 con 0x7f889c159e00 2024-04-25T10:14:16.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.074+0000 7f8898c00640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f888006b4d0 0x7f888006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.074+0000 7f8898c00640 1 -- 172.21.15.119:0/4158979664 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f889408ad00 con 0x7f889c159e00 2024-04-25T10:14:16.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.074+0000 7f88a1200640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f888006b4d0 0x7f888006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.075+0000 7f88a6a36740 1 Processor -- start 2024-04-25T10:14:16.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.075+0000 7f88a6a36740 1 -- start start 2024-04-25T10:14:16.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.075+0000 7f88a1200640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f888006b4d0 0x7f888006d990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8890000f60 tx=0x7f8890009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.076+0000 7f88a6a36740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98644bb0 0x559f98646d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.076+0000 7f88a6a36740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f98681280 0x559f98644f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.076+0000 7f88a6a36740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f9866f670 0x559f9866faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.076+0000 7f88a6a36740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f985a8c70 con 0x559f98644bb0 2024-04-25T10:14:16.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.076+0000 7f88a6a36740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f98551cf0 con 0x559f9866f670 2024-04-25T10:14:16.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.076+0000 7f88a6a36740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f985a4d10 con 0x559f98681280 2024-04-25T10:14:16.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.076+0000 7f88a1200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98644bb0 0x559f98646d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.076+0000 7f88a1200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98644bb0 0x559f98646d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54890/0 (socket says 172.21.15.119:54890) 2024-04-25T10:14:16.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.076+0000 7f88a1200640 1 -- 172.21.15.119:0/3499056484 learned_addr learned my addr 172.21.15.119:0/3499056484 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:16.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.076+0000 7f88a1200640 1 -- 172.21.15.119:0/3499056484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f98681280 msgr2=0x559f98644f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.076+0000 7f88a1200640 1 --2- 172.21.15.119:0/3499056484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f98681280 0x559f98644f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.076+0000 7f88a1200640 1 -- 172.21.15.119:0/3499056484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f9866f670 msgr2=0x559f9866faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.076+0000 7f88a1200640 1 --2- 172.21.15.119:0/3499056484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f9866f670 0x559f9866faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.076+0000 7f88a1200640 1 -- 172.21.15.119:0/3499056484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f983877d0 con 0x559f98644bb0 2024-04-25T10:14:16.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.077+0000 7f88a1200640 1 --2- 172.21.15.119:0/3499056484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98644bb0 0x559f98646d10 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f8890001e70 tx=0x7f8890011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6f290546237f1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.077+0000 7f8889600640 1 -- 172.21.15.119:0/3499056484 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890003800 con 0x559f98644bb0 2024-04-25T10:14:16.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.077+0000 7f8889600640 1 -- 172.21.15.119:0/3499056484 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88900039a0 con 0x559f98644bb0 2024-04-25T10:14:16.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.077+0000 7f8889600640 1 -- 172.21.15.119:0/3499056484 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890003d70 con 0x559f98644bb0 2024-04-25T10:14:16.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.078+0000 7f88a6a36740 1 -- 172.21.15.119:0/3499056484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98644bb0 msgr2=0x559f98646d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.078+0000 7f88a6a36740 1 --2- 172.21.15.119:0/3499056484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98644bb0 0x559f98646d10 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f8890001e70 tx=0x7f8890011000 comp rx=0 tx=0).stop 2024-04-25T10:14:16.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.078+0000 7f88a6a36740 1 -- 172.21.15.119:0/3499056484 shutdown_connections 2024-04-25T10:14:16.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.078+0000 7f88a6a36740 1 --2- 172.21.15.119:0/3499056484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f9866f670 0x559f9866faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.078+0000 7f88a6a36740 1 --2- 172.21.15.119:0/3499056484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f98681280 0x559f98644f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.078+0000 7f88a6a36740 1 --2- 172.21.15.119:0/3499056484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98644bb0 0x559f98646d10 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.078+0000 7f88a6a36740 1 -- 172.21.15.119:0/3499056484 >> 172.21.15.119:0/3499056484 conn(0x559f9860ca30 msgr2=0x559f9868a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:16.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.078+0000 7f88a6a36740 1 -- 172.21.15.119:0/3499056484 shutdown_connections 2024-04-25T10:14:16.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.078+0000 7f88a6a36740 1 -- 172.21.15.119:0/3499056484 wait complete. 2024-04-25T10:14:16.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.078+0000 7f88a6a36740 1 -- start start 2024-04-25T10:14:16.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.079+0000 7f88a6a36740 1 Processor -- start 2024-04-25T10:14:16.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a6a36740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98644bb0 0x559f98671a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a6a36740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f9866f670 0x559f98671fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a6a36740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f98681280 0x559f98681d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a6a36740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f985473a0 con 0x559f98644bb0 2024-04-25T10:14:16.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a6a36740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f9866d1c0 con 0x559f9866f670 2024-04-25T10:14:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a6a36740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f9868cf70 con 0x559f98681280 2024-04-25T10:14:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a6a36740 10 client.0 _ll_register_callbacks cb 0x559f98564b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a1200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98644bb0 0x559f98671a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a1200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98644bb0 0x559f98671a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54898/0 (socket says 172.21.15.119:54898) 2024-04-25T10:14:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a1200640 1 -- 172.21.15.119:0/1989853093 learned_addr learned my addr 172.21.15.119:0/1989853093 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a1200640 1 -- 172.21.15.119:0/1989853093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f98681280 msgr2=0x559f98681d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a1200640 1 --2- 172.21.15.119:0/1989853093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f98681280 0x559f98681d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a1200640 1 -- 172.21.15.119:0/1989853093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f9866f670 msgr2=0x559f98671fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a1200640 1 --2- 172.21.15.119:0/1989853093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f9866f670 0x559f98671fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.080+0000 7f88a1200640 1 -- 172.21.15.119:0/1989853093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f982bd3f0 con 0x559f98644bb0 2024-04-25T10:14:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.081+0000 7f88a1200640 1 --2- 172.21.15.119:0/1989853093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98644bb0 0x559f98671a70 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f8890003d80 tx=0x7f8890000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e328cddf8efec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.081+0000 7f887e000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.081+0000 7f887e000640 1 -- 172.21.15.119:0/1989853093 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890021720 con 0x559f98644bb0 2024-04-25T10:14:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.081+0000 7f887e000640 1 -- 172.21.15.119:0/1989853093 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f88900218c0 con 0x559f98644bb0 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.081+0000 7f88a6a36740 1 -- 172.21.15.119:0/1989853093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88900039a0 con 0x559f98644bb0 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.081+0000 7f887e000640 1 -- 172.21.15.119:0/1989853093 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8890032050 con 0x559f98644bb0 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.081+0000 7f88a6a36740 10 client.7569 fetch_fsmap learned FSMap version 68 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.081+0000 7f88a6a36740 1 -- 172.21.15.119:0/1989853093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x559f9827d0f0 con 0x559f98644bb0 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.081+0000 7f887e000640 1 -- 172.21.15.119:0/1989853093 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88900039a0 con 0x559f98644bb0 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.082+0000 7f887e000640 1 -- 172.21.15.119:0/1989853093 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8890035020 con 0x559f98644bb0 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.082+0000 7f88a6a36740 10 client.7569 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.082+0000 7f88a6a36740 10 client.7569 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.082+0000 7f88a6a36740 20 client.7569 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.082+0000 7f88a6a36740 1 --2- 172.21.15.119:0/1989853093 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x559f9869dd00 0x559f9869e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.082+0000 7f88a6a36740 4 client.7569 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.082+0000 7f88a6a36740 1 -- 172.21.15.119:0/1989853093 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f985b61b0 con 0x559f9869dd00 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.082+0000 7f889be00640 1 --2- 172.21.15.119:0/1989853093 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x559f9869dd00 0x559f9869e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.083+0000 7f889be00640 1 --2- 172.21.15.119:0/1989853093 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x559f9869dd00 0x559f9869e140 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c9b085b5a20333 server_cookie=42dbd978d3d6d182 in_seq=0 out_seq=0 2024-04-25T10:14:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.083+0000 7f887e000640 10 client.7569 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:16.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.083+0000 7f887e000640 1 -- 172.21.15.119:0/1989853093 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559f985b61b0 con 0x559f9869dd00 2024-04-25T10:14:16.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.083+0000 7f887e000640 10 client.7569 handle_command_reply: tid=0 2024-04-25T10:14:16.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.083+0000 7f88a6a36740 1 client.7569 shutdown 2024-04-25T10:14:16.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.083+0000 7f88a6a36740 1 -- 172.21.15.119:0/1989853093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98644bb0 msgr2=0x559f98671a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.083+0000 7f88a6a36740 1 --2- 172.21.15.119:0/1989853093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98644bb0 0x559f98671a70 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f8890003d80 tx=0x7f8890000ed0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.084+0000 7f88a6a36740 1 -- 172.21.15.119:0/1989853093 shutdown_connections 2024-04-25T10:14:16.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.084+0000 7f88a6a36740 1 --2- 172.21.15.119:0/1989853093 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x559f9869dd00 0x559f9869e140 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.084+0000 7f88a6a36740 1 --2- 172.21.15.119:0/1989853093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f98681280 0x559f98681d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.084+0000 7f88a6a36740 1 --2- 172.21.15.119:0/1989853093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f9866f670 0x559f98671fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.084+0000 7f88a6a36740 1 --2- 172.21.15.119:0/1989853093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98644bb0 0x559f98671a70 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.084+0000 7f887e000640 0 client.7569 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:16.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.084+0000 7f88a6a36740 1 -- 172.21.15.119:0/1989853093 >> 172.21.15.119:0/1989853093 conn(0x559f9860ca30 msgr2=0x559f9866a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:16.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.084+0000 7f88a6a36740 1 -- 172.21.15.119:0/1989853093 shutdown_connections 2024-04-25T10:14:16.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.084+0000 7f88a6a36740 1 -- 172.21.15.119:0/1989853093 wait complete. 2024-04-25T10:14:16.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.084+0000 7f88a6a36740 20 client.7569 trim_cache size 0 max 16384 2024-04-25T10:14:16.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.100+0000 7f88a6a36740 1 Processor -- start 2024-04-25T10:14:16.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.100+0000 7f88a6a36740 1 -- start start 2024-04-25T10:14:16.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.101+0000 7f88a6a36740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f98644bb0 0x559f9866cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.101+0000 7f88a6a36740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f9866f670 0x559f98642a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.101+0000 7f88a6a36740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98681280 0x559f9869e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.101+0000 7f88a6a36740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c200110 con 0x559f98681280 2024-04-25T10:14:16.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.101+0000 7f88a6a36740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c1ff330 con 0x559f98644bb0 2024-04-25T10:14:16.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.101+0000 7f88a6a36740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0b12c0 con 0x559f9866f670 2024-04-25T10:14:16.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.101+0000 7f88a1c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98681280 0x559f9869e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.102+0000 7f88a1c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98681280 0x559f9869e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54912/0 (socket says 172.21.15.119:54912) 2024-04-25T10:14:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.102+0000 7f88a1c00640 1 -- 172.21.15.119:0/1934872448 learned_addr learned my addr 172.21.15.119:0/1934872448 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.102+0000 7f88a1200640 1 --2- 172.21.15.119:0/1934872448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f98644bb0 0x559f9866cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.102+0000 7f889be00640 1 --2- 172.21.15.119:0/1934872448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f9866f670 0x559f98642a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.102+0000 7f88a1c00640 1 -- 172.21.15.119:0/1934872448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f9866f670 msgr2=0x559f98642a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.102+0000 7f88a1c00640 1 --2- 172.21.15.119:0/1934872448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f9866f670 0x559f98642a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.102+0000 7f88a1c00640 1 -- 172.21.15.119:0/1934872448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f98644bb0 msgr2=0x559f9866cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.102+0000 7f88a1c00640 1 --2- 172.21.15.119:0/1934872448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f98644bb0 0x559f9866cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.102+0000 7f88a1c00640 1 -- 172.21.15.119:0/1934872448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f986454b0 con 0x559f98681280 2024-04-25T10:14:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.102+0000 7f88a1c00640 1 --2- 172.21.15.119:0/1934872448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98681280 0x559f9869e0e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f8894059120 tx=0x7f8894059be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc9e0106003f7ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.102+0000 7f8888c00640 1 -- 172.21.15.119:0/1934872448 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894061b30 con 0x559f98681280 2024-04-25T10:14:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.103+0000 7f8888c00640 1 -- 172.21.15.119:0/1934872448 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8894057720 con 0x559f98681280 2024-04-25T10:14:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.103+0000 7f8888c00640 1 -- 172.21.15.119:0/1934872448 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889405d170 con 0x559f98681280 2024-04-25T10:14:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.103+0000 7f88a6a36740 1 -- 172.21.15.119:0/1934872448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98681280 msgr2=0x559f9869e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.103+0000 7f88a6a36740 1 --2- 172.21.15.119:0/1934872448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98681280 0x559f9869e0e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f8894059120 tx=0x7f8894059be0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.103+0000 7f88a6a36740 1 -- 172.21.15.119:0/1934872448 shutdown_connections 2024-04-25T10:14:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.103+0000 7f88a6a36740 1 --2- 172.21.15.119:0/1934872448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f98681280 0x559f9869e0e0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.103+0000 7f88a6a36740 1 --2- 172.21.15.119:0/1934872448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f9866f670 0x559f98642a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.103+0000 7f88a6a36740 1 --2- 172.21.15.119:0/1934872448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f98644bb0 0x559f9866cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.103+0000 7f88a6a36740 1 -- 172.21.15.119:0/1934872448 >> 172.21.15.119:0/1934872448 conn(0x559f9860ca30 msgr2=0x559f98681b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.103+0000 7f88a6a36740 1 -- 172.21.15.119:0/1934872448 shutdown_connections 2024-04-25T10:14:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.103+0000 7f88a6a36740 1 -- 172.21.15.119:0/1934872448 wait complete. 2024-04-25T10:14:16.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.104+0000 7f88a6a36740 1 -- start start 2024-04-25T10:14:16.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.104+0000 7f88a6a36740 1 Processor -- start 2024-04-25T10:14:16.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.105+0000 7f88a6a36740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f98644bb0 0x559f986f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.105+0000 7f88a6a36740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9866f670 0x559f98680000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.105+0000 7f88a6a36740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f98681280 0x559f9866fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.105+0000 7f88a6a36740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f98645ea0 con 0x559f9866f670 2024-04-25T10:14:16.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.105+0000 7f88a6a36740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0b18c0 con 0x559f98644bb0 2024-04-25T10:14:16.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.105+0000 7f88a6a36740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f98565fb0 con 0x559f98681280 2024-04-25T10:14:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.105+0000 7f88a6a36740 10 client.0 _ll_register_callbacks cb 0x7f8890001d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.105+0000 7f889be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9866f670 0x559f98680000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.105+0000 7f889be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9866f670 0x559f98680000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54920/0 (socket says 172.21.15.119:54920) 2024-04-25T10:14:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.105+0000 7f889be00640 1 -- 172.21.15.119:0/2834881306 learned_addr learned my addr 172.21.15.119:0/2834881306 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.106+0000 7f889be00640 1 -- 172.21.15.119:0/2834881306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f98681280 msgr2=0x559f9866fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.106+0000 7f889be00640 1 --2- 172.21.15.119:0/2834881306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f98681280 0x559f9866fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.106+0000 7f889be00640 1 -- 172.21.15.119:0/2834881306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f98644bb0 msgr2=0x559f986f7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.106+0000 7f889be00640 1 --2- 172.21.15.119:0/2834881306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f98644bb0 0x559f986f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.106+0000 7f889be00640 1 -- 172.21.15.119:0/2834881306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f9866d300 con 0x559f9866f670 2024-04-25T10:14:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.106+0000 7f889be00640 1 --2- 172.21.15.119:0/2834881306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9866f670 0x559f98680000 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f888c000c00 tx=0x7f888c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a5147c0734abd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.106+0000 7f887e000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.106+0000 7f887e000640 1 -- 172.21.15.119:0/2834881306 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c038040 con 0x559f9866f670 2024-04-25T10:14:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.106+0000 7f88a6a36740 1 -- 172.21.15.119:0/2834881306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8894057720 con 0x559f9866f670 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.106+0000 7f887e000640 1 -- 172.21.15.119:0/2834881306 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f888c004030 con 0x559f9866f670 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.107+0000 7f887e000640 1 -- 172.21.15.119:0/2834881306 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f888c04f050 con 0x559f9866f670 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.107+0000 7f88a6a36740 10 client.7575 fetch_fsmap learned FSMap version 68 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.107+0000 7f88a6a36740 1 -- 172.21.15.119:0/2834881306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x559f9866c260 con 0x559f9866f670 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.107+0000 7f887e000640 1 -- 172.21.15.119:0/2834881306 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8894057720 con 0x559f9866f670 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.107+0000 7f887e000640 1 -- 172.21.15.119:0/2834881306 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f888c052020 con 0x559f9866f670 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.107+0000 7f88a6a36740 10 client.7575 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.107+0000 7f88a6a36740 10 client.7575 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.107+0000 7f88a6a36740 20 client.7575 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.107+0000 7f88a6a36740 1 --2- 172.21.15.119:0/2834881306 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x559f9866ffd0 0x559f9865e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.107+0000 7f88a6a36740 4 client.7575 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.107+0000 7f88a6a36740 1 -- 172.21.15.119:0/2834881306 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559f985b0500 con 0x559f9866ffd0 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.107+0000 7f88a1200640 1 --2- 172.21.15.119:0/2834881306 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x559f9866ffd0 0x559f9865e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.108+0000 7f88a1200640 1 --2- 172.21.15.119:0/2834881306 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x559f9866ffd0 0x559f9865e590 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b45ba9e20a708785 server_cookie=ea2c453a583cb4c0 in_seq=0 out_seq=0 2024-04-25T10:14:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.108+0000 7f887e000640 10 client.7575 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.108+0000 7f887e000640 1 -- 172.21.15.119:0/2834881306 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x559f985b0500 con 0x559f9866ffd0 2024-04-25T10:14:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.108+0000 7f887e000640 10 client.7575 handle_command_reply: tid=0 2024-04-25T10:14:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.108+0000 7f88a6a36740 1 client.7575 shutdown 2024-04-25T10:14:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.108+0000 7f88a6a36740 1 -- 172.21.15.119:0/2834881306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9866f670 msgr2=0x559f98680000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.108+0000 7f88a6a36740 1 --2- 172.21.15.119:0/2834881306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9866f670 0x559f98680000 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f888c000c00 tx=0x7f888c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.108+0000 7f88a6a36740 1 -- 172.21.15.119:0/2834881306 shutdown_connections 2024-04-25T10:14:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.108+0000 7f88a6a36740 1 --2- 172.21.15.119:0/2834881306 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x559f9866ffd0 0x559f9865e590 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.108+0000 7f88a6a36740 1 --2- 172.21.15.119:0/2834881306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x559f98681280 0x559f9866fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.108+0000 7f88a6a36740 1 --2- 172.21.15.119:0/2834881306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f9866f670 0x559f98680000 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.108+0000 7f887e000640 0 client.7575 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.108+0000 7f88a6a36740 1 --2- 172.21.15.119:0/2834881306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559f98644bb0 0x559f986f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.108+0000 7f88a6a36740 1 -- 172.21.15.119:0/2834881306 >> 172.21.15.119:0/2834881306 conn(0x559f9860ca30 msgr2=0x559f9868d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.109+0000 7f88a6a36740 1 -- 172.21.15.119:0/2834881306 shutdown_connections 2024-04-25T10:14:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.109+0000 7f88a6a36740 1 -- 172.21.15.119:0/2834881306 wait complete. 2024-04-25T10:14:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.109+0000 7f88a6a36740 20 client.7575 trim_cache size 0 max 16384 2024-04-25T10:14:16.117 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:14:16.117 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:14:16.117 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2024-04-25T10:14:16.117 INFO:teuthology.orchestra.run.smithi119.stdout: "977ce22f-8a0b-49dc-a917-68e7f9d7f2f5": { 2024-04-25T10:14:16.117 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/dir", 2024-04-25T10:14:16.117 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "977ce22f-8a0b-49dc-a917-68e7f9d7f2f5", 2024-04-25T10:14:16.117 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,repair" 2024-04-25T10:14:16.117 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:14:16.118 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:14:16.118 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:14:16.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.109+0000 7f88a3c00640 1 -- 172.21.15.119:0/4158979664 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f888006b4d0 msgr2=0x7f888006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.109+0000 7f88a3c00640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f888006b4d0 0x7f888006d990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8890000f60 tx=0x7f8890009040 comp rx=0 tx=0).stop 2024-04-25T10:14:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.109+0000 7f88a3c00640 1 -- 172.21.15.119:0/4158979664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c159e00 msgr2=0x7f889c0addb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.109+0000 7f88a3c00640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c159e00 0x7f889c0addb0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f8894009d30 tx=0x7f889400c490 comp rx=0 tx=0).stop 2024-04-25T10:14:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.109+0000 7f88a3c00640 1 -- 172.21.15.119:0/4158979664 shutdown_connections 2024-04-25T10:14:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.110+0000 7f88a3c00640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f888006b4d0 0x7f888006d990 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.110+0000 7f88a3c00640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f889c159e00 0x7f889c0addb0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.110+0000 7f88a3c00640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f889c159a20 0x7f889c0cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.110+0000 7f88a3c00640 1 --2- 172.21.15.119:0/4158979664 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f889c071930 0x7f889c15a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.110+0000 7f88a3c00640 1 -- 172.21.15.119:0/4158979664 >> 172.21.15.119:0/4158979664 conn(0x7f889c06aba0 msgr2=0x7f889c159060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.110+0000 7f88a3c00640 1 -- 172.21.15.119:0/4158979664 shutdown_connections 2024-04-25T10:14:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:16.110+0000 7f88a3c00640 1 -- 172.21.15.119:0/4158979664 wait complete. 2024-04-25T10:14:16.131 INFO:tasks.cephfs.filesystem:scrub status for tag:977ce22f-8a0b-49dc-a917-68e7f9d7f2f5 - {'path': '/dir', 'tag': '977ce22f-8a0b-49dc-a917-68e7f9d7f2f5', 'options': 'recursive,repair'} 2024-04-25T10:14:31.975 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:31.984 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:46.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:46.133 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:14:46.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.343+0000 7ff63a000640 1 Processor -- start 2024-04-25T10:14:46.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.343+0000 7ff63a000640 1 -- start start 2024-04-25T10:14:46.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.345+0000 7ff63a000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff634071830 0x7ff63415b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.345+0000 7ff63a000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff63413d9a0 con 0x7ff63413dce0 2024-04-25T10:14:46.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.345+0000 7ff63a000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff63415e650 con 0x7ff63413d5c0 2024-04-25T10:14:46.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.345+0000 7ff63a000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63415abf0 con 0x7ff634071830 2024-04-25T10:14:46.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.345+0000 7ff62be00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff63413dce0 0x7ff634071120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59442/0 (socket says 172.21.15.119:59442) 2024-04-25T10:14:46.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.345+0000 7ff62be00640 1 -- 172.21.15.119:0/1398070425 learned_addr learned my addr 172.21.15.119:0/1398070425 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:14:46.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff639600640 1 --2- 172.21.15.119:0/1398070425 >> v2:172.21.15.132:3300/0 conn(0x7ff634071830 0x7ff63415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 -- 172.21.15.119:0/1398070425 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801792817 0 0) 0x7ff63413d9a0 con 0x7ff63413dce0 2024-04-25T10:14:46.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 -- 172.21.15.119:0/1398070425 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff618003520 con 0x7ff63413dce0 2024-04-25T10:14:46.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 -- 172.21.15.119:0/1398070425 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 579804970 0 0) 0x7ff618003520 con 0x7ff63413dce0 2024-04-25T10:14:46.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 -- 172.21.15.119:0/1398070425 >> v2:172.21.15.132:3300/0 conn(0x7ff634071830 msgr2=0x7ff63415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:46.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 --2- 172.21.15.119:0/1398070425 >> v2:172.21.15.132:3300/0 conn(0x7ff634071830 0x7ff63415b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 -- 172.21.15.119:0/1398070425 >> v1:172.21.15.132:6789/0 conn(0x7ff63413d5c0 legacy=0x7ff6341407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:46.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 -- 172.21.15.119:0/1398070425 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63415e930 con 0x7ff63413dce0 2024-04-25T10:14:46.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 -- 172.21.15.119:0/1398070425 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7ff61c001670 con 0x7ff63413dce0 2024-04-25T10:14:46.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 -- 172.21.15.119:0/1398070425 >> v1:172.21.15.119:6789/0 conn(0x7ff63413dce0 legacy=0x7ff634071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 --2- 172.21.15.119:0/1398070425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6180039b0 0x7ff618003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 --2- 172.21.15.119:0/1398070425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff618004800 0x7ff618004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 --2- 172.21.15.119:0/1398070425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6180051c0 0x7ff61800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 -- 172.21.15.119:0/1398070425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63415abf0 con 0x7ff6180039b0 2024-04-25T10:14:46.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 -- 172.21.15.119:0/1398070425 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61800e300 con 0x7ff618004800 2024-04-25T10:14:46.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff633e00640 1 -- 172.21.15.119:0/1398070425 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61800e500 con 0x7ff6180051c0 2024-04-25T10:14:46.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.346+0000 7ff638c00640 1 --2- 172.21.15.119:0/1398070425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6180039b0 0x7ff618003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.347+0000 7ff62be00640 1 --2- 172.21.15.119:0/1398070425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff618004800 0x7ff618004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.347+0000 7ff638c00640 1 -- 172.21.15.119:0/1398070425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6180051c0 msgr2=0x7ff61800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:46.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.347+0000 7ff638c00640 1 --2- 172.21.15.119:0/1398070425 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff6180051c0 0x7ff61800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.347+0000 7ff638c00640 1 -- 172.21.15.119:0/1398070425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff618004800 msgr2=0x7ff618004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:46.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.347+0000 7ff638c00640 1 --2- 172.21.15.119:0/1398070425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff618004800 0x7ff618004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.347+0000 7ff638c00640 1 -- 172.21.15.119:0/1398070425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff618003520 con 0x7ff6180039b0 2024-04-25T10:14:46.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.347+0000 7ff62be00640 1 --2- 172.21.15.119:0/1398070425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff618004800 0x7ff618004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:46.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.347+0000 7ff639600640 1 -- 172.21.15.119:0/1398070425 reap_dead start 2024-04-25T10:14:46.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.348+0000 7ff638c00640 1 --2- 172.21.15.119:0/1398070425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6180039b0 0x7ff618003db0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff624003d80 tx=0x7ff618005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcfda99eef233a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:46.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.348+0000 7ff633e00640 1 -- 172.21.15.119:0/1398070425 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624037070 con 0x7ff6180039b0 2024-04-25T10:14:46.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.348+0000 7ff633e00640 1 -- 172.21.15.119:0/1398070425 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62402a990 con 0x7ff6180039b0 2024-04-25T10:14:46.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.348+0000 7ff633e00640 1 -- 172.21.15.119:0/1398070425 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624003980 con 0x7ff6180039b0 2024-04-25T10:14:46.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.349+0000 7ff63a000640 1 -- 172.21.15.119:0/1398070425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6180039b0 msgr2=0x7ff618003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.349+0000 7ff63a000640 1 --2- 172.21.15.119:0/1398070425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6180039b0 0x7ff618003db0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff624003d80 tx=0x7ff618005830 comp rx=0 tx=0).stop 2024-04-25T10:14:46.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.349+0000 7ff63a000640 1 -- 172.21.15.119:0/1398070425 shutdown_connections 2024-04-25T10:14:46.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.349+0000 7ff63a000640 1 --2- 172.21.15.119:0/1398070425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6180039b0 0x7ff618003db0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.349+0000 7ff63a000640 1 -- 172.21.15.119:0/1398070425 >> 172.21.15.119:0/1398070425 conn(0x7ff63406aba0 msgr2=0x7ff63406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:46.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.349+0000 7ff63a000640 1 -- 172.21.15.119:0/1398070425 shutdown_connections 2024-04-25T10:14:46.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.349+0000 7ff63a000640 1 -- 172.21.15.119:0/1398070425 wait complete. 2024-04-25T10:14:46.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.351+0000 7ff63a000640 1 Processor -- start 2024-04-25T10:14:46.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.351+0000 7ff63a000640 1 -- start start 2024-04-25T10:14:46.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.352+0000 7ff63a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6180039b0 0x7ff634136450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.352+0000 7ff63a000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff634152030 0x7ff634154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.352+0000 7ff63a000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6341524b0 0x7ff6341545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.352+0000 7ff63a000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6340b1980 con 0x7ff6180039b0 2024-04-25T10:14:46.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.352+0000 7ff63a000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63415eef0 con 0x7ff6341524b0 2024-04-25T10:14:46.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.352+0000 7ff63a000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63413e150 con 0x7ff634152030 2024-04-25T10:14:46.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.352+0000 7ff638c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6180039b0 0x7ff634136450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.352+0000 7ff638c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6180039b0 0x7ff634136450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43006/0 (socket says 172.21.15.119:43006) 2024-04-25T10:14:46.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.352+0000 7ff638c00640 1 -- 172.21.15.119:0/3434668609 learned_addr learned my addr 172.21.15.119:0/3434668609 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:46.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.353+0000 7ff62be00640 1 --2- 172.21.15.119:0/3434668609 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff634152030 0x7ff634154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.353+0000 7ff638c00640 1 -- 172.21.15.119:0/3434668609 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff634152030 msgr2=0x7ff634154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.353+0000 7ff639600640 1 --2- 172.21.15.119:0/3434668609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6341524b0 0x7ff6341545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.353+0000 7ff638c00640 1 --2- 172.21.15.119:0/3434668609 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff634152030 0x7ff634154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.353+0000 7ff638c00640 1 -- 172.21.15.119:0/3434668609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6341524b0 msgr2=0x7ff6341545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.353+0000 7ff638c00640 1 --2- 172.21.15.119:0/3434668609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6341524b0 0x7ff6341545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.353+0000 7ff638c00640 1 -- 172.21.15.119:0/3434668609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff634152a80 con 0x7ff6180039b0 2024-04-25T10:14:46.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.353+0000 7ff638c00640 1 --2- 172.21.15.119:0/3434668609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6180039b0 0x7ff634136450 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff624003ea0 tx=0x7ff6240036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:46.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.353+0000 7ff631600640 1 -- 172.21.15.119:0/3434668609 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624037050 con 0x7ff6180039b0 2024-04-25T10:14:46.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.353+0000 7ff631600640 1 -- 172.21.15.119:0/3434668609 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff624038040 con 0x7ff6180039b0 2024-04-25T10:14:46.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.353+0000 7ff63a000640 1 -- 172.21.15.119:0/3434668609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6340b2dc0 con 0x7ff6180039b0 2024-04-25T10:14:46.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.354+0000 7ff631600640 1 -- 172.21.15.119:0/3434668609 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624006e50 con 0x7ff6180039b0 2024-04-25T10:14:46.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.354+0000 7ff63a000640 1 -- 172.21.15.119:0/3434668609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6340b3380 con 0x7ff6180039b0 2024-04-25T10:14:46.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.356+0000 7ff631600640 1 -- 172.21.15.119:0/3434668609 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff624045020 con 0x7ff6180039b0 2024-04-25T10:14:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.357+0000 7ff631600640 1 --2- 172.21.15.119:0/3434668609 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff61006b670 0x7ff61006db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.357+0000 7ff631600640 1 -- 172.21.15.119:0/3434668609 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7ff6240a9fd0 con 0x7ff6180039b0 2024-04-25T10:14:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.357+0000 7ff62be00640 1 --2- 172.21.15.119:0/3434668609 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff61006b670 0x7ff61006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.357+0000 7ff63e5aa740 1 Processor -- start 2024-04-25T10:14:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.358+0000 7ff62be00640 1 --2- 172.21.15.119:0/3434668609 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff61006b670 0x7ff61006db30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff61c000fd0 tx=0x7ff61c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:46.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.358+0000 7ff63e5aa740 1 -- start start 2024-04-25T10:14:46.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.358+0000 7ff63e5aa740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558933950d70 0x558933952ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.358+0000 7ff63e5aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55893398d440 0x558933951150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.358+0000 7ff63e5aa740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893397b830 0x55893397bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.359+0000 7ff63e5aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589338b4c70 con 0x55893398d440 2024-04-25T10:14:46.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.359+0000 7ff63e5aa740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893385dcf0 con 0x55893397b830 2024-04-25T10:14:46.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.359+0000 7ff63e5aa740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589338b0d10 con 0x558933950d70 2024-04-25T10:14:46.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.359+0000 7ff62be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55893398d440 0x558933951150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.359+0000 7ff62be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55893398d440 0x558933951150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43012/0 (socket says 172.21.15.119:43012) 2024-04-25T10:14:46.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.359+0000 7ff62be00640 1 -- 172.21.15.119:0/2729370071 learned_addr learned my addr 172.21.15.119:0/2729370071 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:46.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.359+0000 7ff62be00640 1 -- 172.21.15.119:0/2729370071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558933950d70 msgr2=0x558933952ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:46.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.359+0000 7ff62be00640 1 --2- 172.21.15.119:0/2729370071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558933950d70 0x558933952ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.359+0000 7ff62be00640 1 -- 172.21.15.119:0/2729370071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893397b830 msgr2=0x55893397bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:46.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.359+0000 7ff62be00640 1 --2- 172.21.15.119:0/2729370071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893397b830 0x55893397bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.359+0000 7ff62be00640 1 -- 172.21.15.119:0/2729370071 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589336937d0 con 0x55893398d440 2024-04-25T10:14:46.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.359+0000 7ff62be00640 1 --2- 172.21.15.119:0/2729370071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55893398d440 0x558933951150 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ff61c007440 tx=0x7ff61c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d9a09a4e19f1d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:46.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.360+0000 7ff629200640 1 -- 172.21.15.119:0/2729370071 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c007470 con 0x55893398d440 2024-04-25T10:14:46.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.360+0000 7ff629200640 1 -- 172.21.15.119:0/2729370071 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61c007610 con 0x55893398d440 2024-04-25T10:14:46.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.360+0000 7ff629200640 1 -- 172.21.15.119:0/2729370071 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c013680 con 0x55893398d440 2024-04-25T10:14:46.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.360+0000 7ff63e5aa740 1 -- 172.21.15.119:0/2729370071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55893398d440 msgr2=0x558933951150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.360+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/2729370071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55893398d440 0x558933951150 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7ff61c007440 tx=0x7ff61c011000 comp rx=0 tx=0).stop 2024-04-25T10:14:46.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.360+0000 7ff63e5aa740 1 -- 172.21.15.119:0/2729370071 shutdown_connections 2024-04-25T10:14:46.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.360+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/2729370071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893397b830 0x55893397bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.360+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/2729370071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55893398d440 0x558933951150 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.360+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/2729370071 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558933950d70 0x558933952ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.360+0000 7ff63e5aa740 1 -- 172.21.15.119:0/2729370071 >> 172.21.15.119:0/2729370071 conn(0x558933918a30 msgr2=0x558933996830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.361+0000 7ff63e5aa740 1 -- 172.21.15.119:0/2729370071 shutdown_connections 2024-04-25T10:14:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.361+0000 7ff63e5aa740 1 -- 172.21.15.119:0/2729370071 wait complete. 2024-04-25T10:14:46.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.361+0000 7ff63e5aa740 1 -- start start 2024-04-25T10:14:46.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.362+0000 7ff63e5aa740 1 Processor -- start 2024-04-25T10:14:46.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.362+0000 7ff63e5aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558933950d70 0x5589339904f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.362+0000 7ff63e5aa740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55893397b830 0x55893397db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.362+0000 7ff63e5aa740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893398d440 0x55893397e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.362+0000 7ff63e5aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589338533a0 con 0x558933950d70 2024-04-25T10:14:46.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.363+0000 7ff63e5aa740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558933979380 con 0x55893398d440 2024-04-25T10:14:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.363+0000 7ff63e5aa740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589339a6790 con 0x55893397b830 2024-04-25T10:14:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.363+0000 7ff63e5aa740 10 client.0 _ll_register_callbacks cb 0x558933870b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.363+0000 7ff638c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558933950d70 0x5589339904f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.363+0000 7ff638c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558933950d70 0x5589339904f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43022/0 (socket says 172.21.15.119:43022) 2024-04-25T10:14:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.363+0000 7ff638c00640 1 -- 172.21.15.119:0/807937188 learned_addr learned my addr 172.21.15.119:0/807937188 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.363+0000 7ff638c00640 1 -- 172.21.15.119:0/807937188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55893397b830 msgr2=0x55893397db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.363+0000 7ff62be00640 1 --2- 172.21.15.119:0/807937188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55893397b830 0x55893397db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.363+0000 7ff639600640 1 --2- 172.21.15.119:0/807937188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893398d440 0x55893397e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.363+0000 7ff638c00640 1 --2- 172.21.15.119:0/807937188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55893397b830 0x55893397db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.363+0000 7ff638c00640 1 -- 172.21.15.119:0/807937188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893398d440 msgr2=0x55893397e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.363+0000 7ff638c00640 1 --2- 172.21.15.119:0/807937188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893398d440 0x55893397e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.363+0000 7ff638c00640 1 -- 172.21.15.119:0/807937188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589335c93f0 con 0x558933950d70 2024-04-25T10:14:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff638c00640 1 --2- 172.21.15.119:0/807937188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558933950d70 0x5589339904f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff624080840 tx=0x7ff624078720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ff29cc0427ccef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:46.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff621600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff621600640 1 -- 172.21.15.119:0/807937188 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624090020 con 0x558933950d70 2024-04-25T10:14:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff621600640 1 -- 172.21.15.119:0/807937188 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7ff624080ae0 con 0x558933950d70 2024-04-25T10:14:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff63e5aa740 1 -- 172.21.15.119:0/807937188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff61c007610 con 0x558933950d70 2024-04-25T10:14:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff621600640 1 -- 172.21.15.119:0/807937188 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6240a8050 con 0x558933950d70 2024-04-25T10:14:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff63e5aa740 10 client.7587 fetch_fsmap learned FSMap version 68 2024-04-25T10:14:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff63e5aa740 1 -- 172.21.15.119:0/807937188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x5589335890f0 con 0x558933950d70 2024-04-25T10:14:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff621600640 1 -- 172.21.15.119:0/807937188 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61c007610 con 0x558933950d70 2024-04-25T10:14:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff621600640 1 -- 172.21.15.119:0/807937188 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff624077ab0 con 0x558933950d70 2024-04-25T10:14:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff63e5aa740 10 client.7587 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:14:46.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff63e5aa740 10 client.7587 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:14:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff63e5aa740 20 client.7587 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/807937188 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55893396c370 0x5589339a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.364+0000 7ff63e5aa740 4 client.7587 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.365+0000 7ff63e5aa740 1 -- 172.21.15.119:0/807937188 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589338c21b0 con 0x55893396c370 2024-04-25T10:14:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.365+0000 7ff62be00640 1 --2- 172.21.15.119:0/807937188 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55893396c370 0x5589339a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.365+0000 7ff62be00640 1 --2- 172.21.15.119:0/807937188 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55893396c370 0x5589339a9fe0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dba68377b36bda7c server_cookie=8dfe63f238b01725 in_seq=0 out_seq=0 2024-04-25T10:14:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.365+0000 7ff621600640 10 client.7587 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.366+0000 7ff621600640 1 -- 172.21.15.119:0/807937188 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5589338c21b0 con 0x55893396c370 2024-04-25T10:14:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.366+0000 7ff621600640 10 client.7587 handle_command_reply: tid=0 2024-04-25T10:14:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.366+0000 7ff63e5aa740 1 client.7587 shutdown 2024-04-25T10:14:46.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.366+0000 7ff63e5aa740 1 -- 172.21.15.119:0/807937188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558933950d70 msgr2=0x5589339904f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.366+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/807937188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558933950d70 0x5589339904f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff624080840 tx=0x7ff624078720 comp rx=0 tx=0).stop 2024-04-25T10:14:46.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.366+0000 7ff63e5aa740 1 -- 172.21.15.119:0/807937188 shutdown_connections 2024-04-25T10:14:46.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.366+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/807937188 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55893396c370 0x5589339a9fe0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.366+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/807937188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893398d440 0x55893397e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.366+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/807937188 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55893397b830 0x55893397db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.366+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/807937188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558933950d70 0x5589339904f0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.366+0000 7ff621600640 0 client.7587 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:46.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.366+0000 7ff63e5aa740 1 -- 172.21.15.119:0/807937188 >> 172.21.15.119:0/807937188 conn(0x558933918a30 msgr2=0x55893394f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:46.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.367+0000 7ff63e5aa740 1 -- 172.21.15.119:0/807937188 shutdown_connections 2024-04-25T10:14:46.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.367+0000 7ff63e5aa740 1 -- 172.21.15.119:0/807937188 wait complete. 2024-04-25T10:14:46.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.367+0000 7ff63e5aa740 20 client.7587 trim_cache size 0 max 16384 2024-04-25T10:14:46.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.383+0000 7ff63e5aa740 1 Processor -- start 2024-04-25T10:14:46.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.383+0000 7ff63e5aa740 1 -- start start 2024-04-25T10:14:46.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.384+0000 7ff63e5aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558933950d70 0x55893397da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.384+0000 7ff63e5aa740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893397b830 0x55893397df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.384+0000 7ff63e5aa740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55893398d440 0x558933998cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.384+0000 7ff63e5aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6342003b0 con 0x558933950d70 2024-04-25T10:14:46.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.384+0000 7ff63e5aa740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6341ff570 con 0x55893397b830 2024-04-25T10:14:46.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.384+0000 7ff63e5aa740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6340b4dd0 con 0x55893398d440 2024-04-25T10:14:46.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.384+0000 7ff638c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558933950d70 0x55893397da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.384+0000 7ff638c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558933950d70 0x55893397da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43036/0 (socket says 172.21.15.119:43036) 2024-04-25T10:14:46.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.384+0000 7ff638c00640 1 -- 172.21.15.119:0/1066190878 learned_addr learned my addr 172.21.15.119:0/1066190878 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:46.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.385+0000 7ff638c00640 1 -- 172.21.15.119:0/1066190878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55893398d440 msgr2=0x558933998cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:46.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.385+0000 7ff638c00640 1 --2- 172.21.15.119:0/1066190878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55893398d440 0x558933998cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.385+0000 7ff638c00640 1 -- 172.21.15.119:0/1066190878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893397b830 msgr2=0x55893397df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:46.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.385+0000 7ff638c00640 1 --2- 172.21.15.119:0/1066190878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893397b830 0x55893397df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.385+0000 7ff638c00640 1 -- 172.21.15.119:0/1066190878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589339902a0 con 0x558933950d70 2024-04-25T10:14:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.385+0000 7ff638c00640 1 --2- 172.21.15.119:0/1066190878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558933950d70 0x55893397da20 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff624080f10 tx=0x7ff624049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aef4d867dda02b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.385+0000 7ff623e00640 1 -- 172.21.15.119:0/1066190878 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624090040 con 0x558933950d70 2024-04-25T10:14:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.385+0000 7ff623e00640 1 -- 172.21.15.119:0/1066190878 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6240901e0 con 0x558933950d70 2024-04-25T10:14:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.385+0000 7ff623e00640 1 -- 172.21.15.119:0/1066190878 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6240808e0 con 0x558933950d70 2024-04-25T10:14:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.386+0000 7ff63e5aa740 1 -- 172.21.15.119:0/1066190878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558933950d70 msgr2=0x55893397da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.386+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/1066190878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558933950d70 0x55893397da20 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff624080f10 tx=0x7ff624049000 comp rx=0 tx=0).stop 2024-04-25T10:14:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.386+0000 7ff63e5aa740 1 -- 172.21.15.119:0/1066190878 shutdown_connections 2024-04-25T10:14:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.386+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/1066190878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55893398d440 0x558933998cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.386+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/1066190878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893397b830 0x55893397df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.386+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/1066190878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558933950d70 0x55893397da20 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.386+0000 7ff63e5aa740 1 -- 172.21.15.119:0/1066190878 >> 172.21.15.119:0/1066190878 conn(0x558933918a30 msgr2=0x558933990f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.386+0000 7ff63e5aa740 1 -- 172.21.15.119:0/1066190878 shutdown_connections 2024-04-25T10:14:46.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.386+0000 7ff63e5aa740 1 -- 172.21.15.119:0/1066190878 wait complete. 2024-04-25T10:14:46.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.386+0000 7ff63e5aa740 1 -- start start 2024-04-25T10:14:46.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.387+0000 7ff63e5aa740 1 Processor -- start 2024-04-25T10:14:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff63e5aa740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558933950d70 0x5589339b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff63e5aa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55893397b830 0x5589339fa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff63e5aa740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893398d440 0x55893398c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff63e5aa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6341fef70 con 0x55893397b830 2024-04-25T10:14:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff63e5aa740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55893394c660 con 0x55893398d440 2024-04-25T10:14:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff63e5aa740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558933a01bb0 con 0x558933950d70 2024-04-25T10:14:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff63e5aa740 10 client.0 _ll_register_callbacks cb 0x7ff624003ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:46.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff62be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55893397b830 0x5589339fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff62be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55893397b830 0x5589339fa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43046/0 (socket says 172.21.15.119:43046) 2024-04-25T10:14:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff62be00640 1 -- 172.21.15.119:0/2266273829 learned_addr learned my addr 172.21.15.119:0/2266273829 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff62be00640 1 -- 172.21.15.119:0/2266273829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558933950d70 msgr2=0x5589339b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff62be00640 1 --2- 172.21.15.119:0/2266273829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558933950d70 0x5589339b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff62be00640 1 -- 172.21.15.119:0/2266273829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893398d440 msgr2=0x55893398c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff62be00640 1 --2- 172.21.15.119:0/2266273829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893398d440 0x55893398c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.388+0000 7ff62be00640 1 -- 172.21.15.119:0/2266273829 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55893398de20 con 0x55893397b830 2024-04-25T10:14:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.389+0000 7ff62be00640 1 --2- 172.21.15.119:0/2266273829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55893397b830 0x5589339fa680 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00f010 tx=0x7ff63415e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7155ba21a13cff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.389+0000 7ff621600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.389+0000 7ff621600640 1 -- 172.21.15.119:0/2266273829 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c02e380 con 0x55893397b830 2024-04-25T10:14:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.389+0000 7ff621600640 1 -- 172.21.15.119:0/2266273829 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7ff61c02e520 con 0x55893397b830 2024-04-25T10:14:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.389+0000 7ff63e5aa740 1 -- 172.21.15.119:0/2266273829 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6240901e0 con 0x55893397b830 2024-04-25T10:14:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.389+0000 7ff621600640 1 -- 172.21.15.119:0/2266273829 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff61c03c050 con 0x55893397b830 2024-04-25T10:14:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.389+0000 7ff63e5aa740 10 client.7593 fetch_fsmap learned FSMap version 68 2024-04-25T10:14:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.389+0000 7ff63e5aa740 1 -- 172.21.15.119:0/2266273829 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x558933a01930 con 0x55893397b830 2024-04-25T10:14:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.389+0000 7ff621600640 1 -- 172.21.15.119:0/2266273829 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6240901e0 con 0x55893397b830 2024-04-25T10:14:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.389+0000 7ff621600640 1 -- 172.21.15.119:0/2266273829 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff61c03f020 con 0x55893397b830 2024-04-25T10:14:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.390+0000 7ff63e5aa740 10 client.7593 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:14:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.390+0000 7ff63e5aa740 10 client.7593 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:14:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.390+0000 7ff63e5aa740 20 client.7593 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.390+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/2266273829 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55893398cdf0 0x558933a00850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.390+0000 7ff63e5aa740 4 client.7593 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:14:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.390+0000 7ff63e5aa740 1 -- 172.21.15.119:0/2266273829 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5589338bc500 con 0x55893398cdf0 2024-04-25T10:14:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.390+0000 7ff638c00640 1 --2- 172.21.15.119:0/2266273829 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55893398cdf0 0x558933a00850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.390+0000 7ff638c00640 1 --2- 172.21.15.119:0/2266273829 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55893398cdf0 0x558933a00850 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c3a766533a4dbb server_cookie=3794b9822bec2389 in_seq=0 out_seq=0 2024-04-25T10:14:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.390+0000 7ff621600640 10 client.7593 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff621600640 1 -- 172.21.15.119:0/2266273829 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5589338bc500 con 0x55893398cdf0 2024-04-25T10:14:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff621600640 10 client.7593 handle_command_reply: tid=0 2024-04-25T10:14:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff63e5aa740 1 client.7593 shutdown 2024-04-25T10:14:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff63e5aa740 1 -- 172.21.15.119:0/2266273829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55893397b830 msgr2=0x5589339fa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/2266273829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55893397b830 0x5589339fa680 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00f010 tx=0x7ff63415e820 comp rx=0 tx=0).stop 2024-04-25T10:14:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff63e5aa740 1 -- 172.21.15.119:0/2266273829 shutdown_connections 2024-04-25T10:14:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/2266273829 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x55893398cdf0 0x558933a00850 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/2266273829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55893398d440 0x55893398c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/2266273829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55893397b830 0x5589339fa680 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff63e5aa740 1 --2- 172.21.15.119:0/2266273829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558933950d70 0x5589339b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff621600640 0 client.7593 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff63e5aa740 1 -- 172.21.15.119:0/2266273829 >> 172.21.15.119:0/2266273829 conn(0x558933918a30 msgr2=0x55893397da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff63e5aa740 1 -- 172.21.15.119:0/2266273829 shutdown_connections 2024-04-25T10:14:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff63e5aa740 1 -- 172.21.15.119:0/2266273829 wait complete. 2024-04-25T10:14:46.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.391+0000 7ff63e5aa740 20 client.7593 trim_cache size 0 max 16384 2024-04-25T10:14:46.399 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:14:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2024-04-25T10:14:46.399 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2024-04-25T10:14:46.399 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:14:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.392+0000 7ff63a000640 1 -- 172.21.15.119:0/3434668609 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff61006b670 msgr2=0x7ff61006db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.392+0000 7ff63a000640 1 --2- 172.21.15.119:0/3434668609 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff61006b670 0x7ff61006db30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff61c000fd0 tx=0x7ff61c009040 comp rx=0 tx=0).stop 2024-04-25T10:14:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.392+0000 7ff63a000640 1 -- 172.21.15.119:0/3434668609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6180039b0 msgr2=0x7ff634136450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.392+0000 7ff63a000640 1 --2- 172.21.15.119:0/3434668609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6180039b0 0x7ff634136450 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff624003ea0 tx=0x7ff6240036f0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.392+0000 7ff63a000640 1 -- 172.21.15.119:0/3434668609 shutdown_connections 2024-04-25T10:14:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.392+0000 7ff63a000640 1 --2- 172.21.15.119:0/3434668609 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff61006b670 0x7ff61006db30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.392+0000 7ff63a000640 1 --2- 172.21.15.119:0/3434668609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6341524b0 0x7ff6341545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.392+0000 7ff63a000640 1 --2- 172.21.15.119:0/3434668609 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff634152030 0x7ff634154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.392+0000 7ff63a000640 1 --2- 172.21.15.119:0/3434668609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6180039b0 0x7ff634136450 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.392+0000 7ff63a000640 1 -- 172.21.15.119:0/3434668609 >> 172.21.15.119:0/3434668609 conn(0x7ff63406aba0 msgr2=0x7ff634159710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.392+0000 7ff63a000640 1 -- 172.21.15.119:0/3434668609 shutdown_connections 2024-04-25T10:14:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.392+0000 7ff63a000640 1 -- 172.21.15.119:0/3434668609 wait complete. 2024-04-25T10:14:46.414 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:14:46.414 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:46.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.607+0000 7f0abf200640 1 Processor -- start 2024-04-25T10:14:46.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.607+0000 7f0abf200640 1 -- start start 2024-04-25T10:14:46.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.608+0000 7f0abf200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0ab8144470 0x7f0ab815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.608+0000 7f0abf200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ab80b5050 0x7f0ab815bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.608+0000 7f0abf200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab80cd440 con 0x7f0ab806e430 2024-04-25T10:14:46.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.608+0000 7f0abf200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab8145340 con 0x7f0ab80b5050 2024-04-25T10:14:46.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.608+0000 7f0abf200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab80cd720 con 0x7f0ab8144470 2024-04-25T10:14:46.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.609+0000 7f0abe800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ab80b5050 0x7f0ab815bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.609+0000 7f0abe800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ab80b5050 0x7f0ab815bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43050/0 (socket says 172.21.15.119:43050) 2024-04-25T10:14:46.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.609+0000 7f0abe800640 1 -- 172.21.15.119:0/4166585007 learned_addr learned my addr 172.21.15.119:0/4166585007 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:46.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.609+0000 7f0abde00640 1 --2- 172.21.15.119:0/4166585007 >> v2:172.21.15.132:3300/0 conn(0x7f0ab8144470 0x7f0ab815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.609+0000 7f0abe800640 1 -- 172.21.15.119:0/4166585007 >> v2:172.21.15.132:3300/0 conn(0x7f0ab8144470 msgr2=0x7f0ab815fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:46.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.609+0000 7f0abe800640 1 --2- 172.21.15.119:0/4166585007 >> v2:172.21.15.132:3300/0 conn(0x7f0ab8144470 0x7f0ab815fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.609+0000 7f0abe800640 1 -- 172.21.15.119:0/4166585007 >> v1:172.21.15.132:6789/0 conn(0x7f0ab806e430 legacy=0x7f0ab80b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:46.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.609+0000 7f0abe800640 1 -- 172.21.15.119:0/4166585007 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80cda00 con 0x7f0ab80b5050 2024-04-25T10:14:46.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.610+0000 7f0abe800640 1 --2- 172.21.15.119:0/4166585007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab80b5050 0x7f0ab815bcf0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80b5490 tx=0x7f0ab402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215ad2341dd2048b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:46.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.610+0000 7f0abca00640 1 -- 172.21.15.119:0/4166585007 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4036070 con 0x7f0ab80b5050 2024-04-25T10:14:46.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.611+0000 7f0abca00640 1 -- 172.21.15.119:0/4166585007 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab4031600 con 0x7f0ab80b5050 2024-04-25T10:14:46.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.611+0000 7f0abca00640 1 -- 172.21.15.119:0/4166585007 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40319d0 con 0x7f0ab80b5050 2024-04-25T10:14:46.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.611+0000 7f0abf200640 1 -- 172.21.15.119:0/4166585007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab80b5050 msgr2=0x7f0ab815bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.611+0000 7f0abf200640 1 --2- 172.21.15.119:0/4166585007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab80b5050 0x7f0ab815bcf0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80b5490 tx=0x7f0ab402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.611+0000 7f0abf200640 1 -- 172.21.15.119:0/4166585007 shutdown_connections 2024-04-25T10:14:46.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.611+0000 7f0abf200640 1 --2- 172.21.15.119:0/4166585007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab80b5050 0x7f0ab815bcf0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.611+0000 7f0abf200640 1 --2- 172.21.15.119:0/4166585007 >> v2:172.21.15.132:3300/0 conn(0x7f0ab8144470 0x7f0ab815fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.611+0000 7f0abf200640 1 -- 172.21.15.119:0/4166585007 >> 172.21.15.119:0/4166585007 conn(0x7f0ab806b1d0 msgr2=0x7f0ab80b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:46.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.611+0000 7f0abf200640 1 -- 172.21.15.119:0/4166585007 shutdown_connections 2024-04-25T10:14:46.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.611+0000 7f0abf200640 1 -- 172.21.15.119:0/4166585007 wait complete. 2024-04-25T10:14:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.613+0000 7f0abf200640 1 Processor -- start 2024-04-25T10:14:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.613+0000 7f0abf200640 1 -- start start 2024-04-25T10:14:46.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.614+0000 7f0abf200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab806e430 0x7f0ab815aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.614+0000 7f0abf200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab80b5050 0x7f0ab81e5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.614+0000 7f0abf200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab8144470 0x7f0ab81e5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.614+0000 7f0abf200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80ce200 con 0x7f0ab806e430 2024-04-25T10:14:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.614+0000 7f0abf200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8144850 con 0x7f0ab80b5050 2024-04-25T10:14:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.614+0000 7f0abf200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab815c230 con 0x7f0ab8144470 2024-04-25T10:14:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.614+0000 7f0abde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab806e430 0x7f0ab815aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.614+0000 7f0abde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab806e430 0x7f0ab815aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43058/0 (socket says 172.21.15.119:43058) 2024-04-25T10:14:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.614+0000 7f0abde00640 1 -- 172.21.15.119:0/1384715693 learned_addr learned my addr 172.21.15.119:0/1384715693 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.615+0000 7f0abd400640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab80b5050 0x7f0ab81e5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.615+0000 7f0abe800640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab8144470 0x7f0ab81e5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.615+0000 7f0abde00640 1 -- 172.21.15.119:0/1384715693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab8144470 msgr2=0x7f0ab81e5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.615+0000 7f0abde00640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab8144470 0x7f0ab81e5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.615+0000 7f0abde00640 1 -- 172.21.15.119:0/1384715693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab80b5050 msgr2=0x7f0ab81e5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.615+0000 7f0abde00640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab80b5050 0x7f0ab81e5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.615+0000 7f0abde00640 1 -- 172.21.15.119:0/1384715693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab815b530 con 0x7f0ab806e430 2024-04-25T10:14:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.615+0000 7f0abd400640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab80b5050 0x7f0ab81e5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.615+0000 7f0abe800640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab8144470 0x7f0ab81e5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.615+0000 7f0abde00640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab806e430 0x7f0ab815aff0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00d5f0 tx=0x7f0aac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.616+0000 7f0ab2000640 1 -- 172.21.15.119:0/1384715693 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0125f0 con 0x7f0ab806e430 2024-04-25T10:14:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.616+0000 7f0ab2000640 1 -- 172.21.15.119:0/1384715693 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac019070 con 0x7f0ab806e430 2024-04-25T10:14:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.616+0000 7f0ab2000640 1 -- 172.21.15.119:0/1384715693 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0129d0 con 0x7f0ab806e430 2024-04-25T10:14:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.616+0000 7f0abf200640 1 -- 172.21.15.119:0/1384715693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab81e5d90 con 0x7f0ab806e430 2024-04-25T10:14:46.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.616+0000 7f0abf200640 1 -- 172.21.15.119:0/1384715693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab815c940 con 0x7f0ab806e430 2024-04-25T10:14:46.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.617+0000 7f0abf200640 1 -- 172.21.15.119:0/1384715693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab80749b0 con 0x7f0ab806e430 2024-04-25T10:14:46.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.618+0000 7f0ab2000640 1 -- 172.21.15.119:0/1384715693 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0aac012b70 con 0x7f0ab806e430 2024-04-25T10:14:46.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.619+0000 7f0ab2000640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0a9c06b4d0 0x7f0a9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:46.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.619+0000 7f0ab2000640 1 -- 172.21.15.119:0/1384715693 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f0aac08a650 con 0x7f0ab806e430 2024-04-25T10:14:46.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.619+0000 7f0abd400640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0a9c06b4d0 0x7f0a9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:46.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.620+0000 7f0abd400640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0a9c06b4d0 0x7f0a9c06d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8071bc0 tx=0x7f0aa8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:46.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.624+0000 7f0ab2000640 1 -- 172.21.15.119:0/1384715693 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0aac059470 con 0x7f0ab806e430 2024-04-25T10:14:46.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.855+0000 7f0abf200640 1 -- 172.21.15.119:0/1384715693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ab80b2090 con 0x7f0ab806e430 2024-04-25T10:14:46.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.856+0000 7f0ab2000640 1 -- 172.21.15.119:0/1384715693 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f0aac058d70 con 0x7f0ab806e430 2024-04-25T10:14:46.860 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:14:46.860 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:13:47.200726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/1734507541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1734507541},{"type":"v1","addr":"172.21.15.119:6837","nonce":1734507541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:14:46.861 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 68 2024-04-25T10:14:46.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.858+0000 7f0abf200640 1 -- 172.21.15.119:0/1384715693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0a9c06b4d0 msgr2=0x7f0a9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.858+0000 7f0abf200640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0a9c06b4d0 0x7f0a9c06d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8071bc0 tx=0x7f0aa8009040 comp rx=0 tx=0).stop 2024-04-25T10:14:46.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.858+0000 7f0abf200640 1 -- 172.21.15.119:0/1384715693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab806e430 msgr2=0x7f0ab815aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.858+0000 7f0abf200640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab806e430 0x7f0ab815aff0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00d5f0 tx=0x7f0aac00dab0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.859+0000 7f0abf200640 1 -- 172.21.15.119:0/1384715693 shutdown_connections 2024-04-25T10:14:46.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.859+0000 7f0abf200640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0a9c06b4d0 0x7f0a9c06d990 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.859+0000 7f0abf200640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ab8144470 0x7f0ab81e5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.859+0000 7f0abf200640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ab80b5050 0x7f0ab81e5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.859+0000 7f0abf200640 1 --2- 172.21.15.119:0/1384715693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab806e430 0x7f0ab815aff0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.859+0000 7f0abf200640 1 -- 172.21.15.119:0/1384715693 >> 172.21.15.119:0/1384715693 conn(0x7f0ab806b1d0 msgr2=0x7f0ab81435d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:46.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.859+0000 7f0abf200640 1 -- 172.21.15.119:0/1384715693 shutdown_connections 2024-04-25T10:14:46.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:46.859+0000 7f0abf200640 1 -- 172.21.15.119:0/1384715693 wait complete. 2024-04-25T10:14:46.882 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:14:47.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.081+0000 7fb35e800640 1 Processor -- start 2024-04-25T10:14:47.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.082+0000 7fb35e800640 1 -- start start 2024-04-25T10:14:47.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.083+0000 7fb35e800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3580adfd0 0x7fb3580cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.083+0000 7fb35e800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35815e360 con 0x7fb35815b720 2024-04-25T10:14:47.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.083+0000 7fb35e800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35815e5d0 con 0x7fb3580aed40 2024-04-25T10:14:47.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.083+0000 7fb35e800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35815e8b0 con 0x7fb3580adfd0 2024-04-25T10:14:47.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.084+0000 7fb357400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb35815b720 0x7fb3580ae630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59458/0 (socket says 172.21.15.119:59458) 2024-04-25T10:14:47.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.084+0000 7fb357400640 1 -- 172.21.15.119:0/1090684863 learned_addr learned my addr 172.21.15.119:0/1090684863 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:14:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.084+0000 7fb357e00640 1 --2- 172.21.15.119:0/1090684863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3580adfd0 0x7fb3580cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.084+0000 7fb356000640 1 -- 172.21.15.119:0/1090684863 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747037996 0 0) 0x7fb35815e360 con 0x7fb35815b720 2024-04-25T10:14:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.084+0000 7fb356000640 1 -- 172.21.15.119:0/1090684863 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb33c003520 con 0x7fb35815b720 2024-04-25T10:14:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb356000640 1 -- 172.21.15.119:0/1090684863 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3542020535 0 0) 0x7fb33c003520 con 0x7fb35815b720 2024-04-25T10:14:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb356000640 1 -- 172.21.15.119:0/1090684863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3580adfd0 msgr2=0x7fb3580cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb356000640 1 --2- 172.21.15.119:0/1090684863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3580adfd0 0x7fb3580cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb356000640 1 -- 172.21.15.119:0/1090684863 >> v1:172.21.15.132:6789/0 conn(0x7fb3580aed40 legacy=0x7fb3580cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb356000640 1 -- 172.21.15.119:0/1090684863 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35815eb90 con 0x7fb35815b720 2024-04-25T10:14:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb356000640 1 -- 172.21.15.119:0/1090684863 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fb348001670 con 0x7fb35815b720 2024-04-25T10:14:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb356000640 1 -- 172.21.15.119:0/1090684863 >> v1:172.21.15.119:6789/0 conn(0x7fb35815b720 legacy=0x7fb3580ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb356000640 1 --2- 172.21.15.119:0/1090684863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb33c0039e0 0x7fb33c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb356000640 1 --2- 172.21.15.119:0/1090684863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3580adfd0 0x7fb33c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb356000640 1 --2- 172.21.15.119:0/1090684863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb33c004d40 0x7fb33c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb356000640 1 -- 172.21.15.119:0/1090684863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35815e8b0 con 0x7fb33c004d40 2024-04-25T10:14:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb356000640 1 -- 172.21.15.119:0/1090684863 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c00baa0 con 0x7fb33c0039e0 2024-04-25T10:14:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb356000640 1 -- 172.21.15.119:0/1090684863 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c00bca0 con 0x7fb3580adfd0 2024-04-25T10:14:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb357e00640 1 --2- 172.21.15.119:0/1090684863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb33c004d40 0x7fb33c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb356a00640 1 --2- 172.21.15.119:0/1090684863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3580adfd0 0x7fb33c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.085+0000 7fb357e00640 1 -- 172.21.15.119:0/1090684863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3580adfd0 msgr2=0x7fb33c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.086+0000 7fb357400640 1 --2- 172.21.15.119:0/1090684863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb33c0039e0 0x7fb33c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.086+0000 7fb357e00640 1 --2- 172.21.15.119:0/1090684863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3580adfd0 0x7fb33c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.086+0000 7fb357e00640 1 -- 172.21.15.119:0/1090684863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb33c0039e0 msgr2=0x7fb33c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.086+0000 7fb357e00640 1 --2- 172.21.15.119:0/1090684863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb33c0039e0 0x7fb33c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.086+0000 7fb357e00640 1 -- 172.21.15.119:0/1090684863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c003520 con 0x7fb33c004d40 2024-04-25T10:14:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.086+0000 7fb356a00640 1 --2- 172.21.15.119:0/1090684863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb3580adfd0 0x7fb33c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.086+0000 7fb357400640 1 --2- 172.21.15.119:0/1090684863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb33c0039e0 0x7fb33c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.087+0000 7fb357e00640 1 --2- 172.21.15.119:0/1090684863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb33c004d40 0x7fb33c0051c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fb3580ae3f0 tx=0x7fb33c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcbaae0e954817 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.087+0000 7fb356000640 1 -- 172.21.15.119:0/1090684863 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350037070 con 0x7fb33c004d40 2024-04-25T10:14:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.087+0000 7fb356000640 1 -- 172.21.15.119:0/1090684863 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35002e050 con 0x7fb33c004d40 2024-04-25T10:14:47.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.087+0000 7fb356000640 1 -- 172.21.15.119:0/1090684863 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350009c10 con 0x7fb33c004d40 2024-04-25T10:14:47.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.087+0000 7fb35e800640 1 -- 172.21.15.119:0/1090684863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb33c004d40 msgr2=0x7fb33c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.087+0000 7fb35e800640 1 --2- 172.21.15.119:0/1090684863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb33c004d40 0x7fb33c0051c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fb3580ae3f0 tx=0x7fb33c005830 comp rx=0 tx=0).stop 2024-04-25T10:14:47.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.087+0000 7fb357e00640 1 -- 172.21.15.119:0/1090684863 reap_dead start 2024-04-25T10:14:47.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.088+0000 7fb35e800640 1 -- 172.21.15.119:0/1090684863 shutdown_connections 2024-04-25T10:14:47.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.088+0000 7fb35e800640 1 -- 172.21.15.119:0/1090684863 >> 172.21.15.119:0/1090684863 conn(0x7fb35806aba0 msgr2=0x7fb35806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.088+0000 7fb35e800640 1 -- 172.21.15.119:0/1090684863 shutdown_connections 2024-04-25T10:14:47.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.088+0000 7fb35e800640 1 -- 172.21.15.119:0/1090684863 wait complete. 2024-04-25T10:14:47.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.089+0000 7fb35e800640 1 Processor -- start 2024-04-25T10:14:47.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.090+0000 7fb35e800640 1 -- start start 2024-04-25T10:14:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.090+0000 7fb35e800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb33c004d40 0x7fb3581f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.090+0000 7fb35e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3580b1bd0 0x7fb3580ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.090+0000 7fb35e800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb35814ec90 0x7fb35814f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.090+0000 7fb35e800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3580cfc60 con 0x7fb3580b1bd0 2024-04-25T10:14:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.090+0000 7fb35e800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35815bb00 con 0x7fb35814ec90 2024-04-25T10:14:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.090+0000 7fb35e800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3580cfde0 con 0x7fb33c004d40 2024-04-25T10:14:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.091+0000 7fb356a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3580b1bd0 0x7fb3580ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.091+0000 7fb356a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3580b1bd0 0x7fb3580ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43074/0 (socket says 172.21.15.119:43074) 2024-04-25T10:14:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.091+0000 7fb356a00640 1 -- 172.21.15.119:0/1641337765 learned_addr learned my addr 172.21.15.119:0/1641337765 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.091+0000 7fb357400640 1 --2- 172.21.15.119:0/1641337765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb33c004d40 0x7fb3581f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.091+0000 7fb357e00640 1 --2- 172.21.15.119:0/1641337765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb35814ec90 0x7fb35814f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.091+0000 7fb356a00640 1 -- 172.21.15.119:0/1641337765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb33c004d40 msgr2=0x7fb3581f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.091+0000 7fb356a00640 1 --2- 172.21.15.119:0/1641337765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb33c004d40 0x7fb3581f7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.091+0000 7fb356a00640 1 -- 172.21.15.119:0/1641337765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb35814ec90 msgr2=0x7fb35814f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.091+0000 7fb356a00640 1 --2- 172.21.15.119:0/1641337765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb35814ec90 0x7fb35814f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.091+0000 7fb356a00640 1 -- 172.21.15.119:0/1641337765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3580ce730 con 0x7fb3580b1bd0 2024-04-25T10:14:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.091+0000 7fb356a00640 1 --2- 172.21.15.119:0/1641337765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3580b1bd0 0x7fb3580ce1f0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fb34000d800 tx=0x7fb34000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.092+0000 7fb34f400640 1 -- 172.21.15.119:0/1641337765 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340005070 con 0x7fb3580b1bd0 2024-04-25T10:14:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.092+0000 7fb34f400640 1 -- 172.21.15.119:0/1641337765 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb340005210 con 0x7fb3580b1bd0 2024-04-25T10:14:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.092+0000 7fb34f400640 1 -- 172.21.15.119:0/1641337765 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340007030 con 0x7fb3580b1bd0 2024-04-25T10:14:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.092+0000 7fb35e800640 1 -- 172.21.15.119:0/1641337765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35814f9f0 con 0x7fb3580b1bd0 2024-04-25T10:14:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.092+0000 7fb35e800640 1 -- 172.21.15.119:0/1641337765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb358151310 con 0x7fb3580b1bd0 2024-04-25T10:14:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.094+0000 7fb34f400640 1 -- 172.21.15.119:0/1641337765 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb34001a020 con 0x7fb3580b1bd0 2024-04-25T10:14:47.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.095+0000 7fb34f400640 1 --2- 172.21.15.119:0/1641337765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb33406b4d0 0x7fb33406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.095+0000 7fb34f400640 1 -- 172.21.15.119:0/1641337765 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fb340089aa0 con 0x7fb3580b1bd0 2024-04-25T10:14:47.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.095+0000 7fb357400640 1 --2- 172.21.15.119:0/1641337765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb33406b4d0 0x7fb33406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.096+0000 7fb3615e3740 1 Processor -- start 2024-04-25T10:14:47.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.096+0000 7fb3615e3740 1 -- start start 2024-04-25T10:14:47.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.096+0000 7fb357400640 1 --2- 172.21.15.119:0/1641337765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb33406b4d0 0x7fb33406d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb348001200 tx=0x7fb348009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.097+0000 7fb3615e3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592b1370 0x5579592b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.097+0000 7fb3615e3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.097+0000 7fb3615e3740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557959287050 0x5579592cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.097+0000 7fb3615e3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579591e9ae0 con 0x5579592cec10 2024-04-25T10:14:47.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.097+0000 7fb3615e3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579591927c0 con 0x5579592b1370 2024-04-25T10:14:47.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.097+0000 7fb3615e3740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579591e5b80 con 0x557959287050 2024-04-25T10:14:47.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.097+0000 7fb356a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.097+0000 7fb356a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43086/0 (socket says 172.21.15.119:43086) 2024-04-25T10:14:47.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.097+0000 7fb356a00640 1 -- 172.21.15.119:0/547857521 learned_addr learned my addr 172.21.15.119:0/547857521 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.097+0000 7fb356a00640 1 -- 172.21.15.119:0/547857521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557959287050 msgr2=0x5579592cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.097+0000 7fb356a00640 1 --2- 172.21.15.119:0/547857521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557959287050 0x5579592cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.097+0000 7fb356a00640 1 -- 172.21.15.119:0/547857521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592b1370 msgr2=0x5579592b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.097+0000 7fb356a00640 1 --2- 172.21.15.119:0/547857521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592b1370 0x5579592b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.097+0000 7fb356a00640 1 -- 172.21.15.119:0/547857521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557958fca7c0 con 0x5579592cec10 2024-04-25T10:14:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.098+0000 7fb356a00640 1 --2- 172.21.15.119:0/547857521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592cf160 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fb340007fd0 tx=0x7fb340058a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9117b3419196b5ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.098+0000 7fb347e00640 1 -- 172.21.15.119:0/547857521 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340085070 con 0x5579592cec10 2024-04-25T10:14:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.098+0000 7fb347e00640 1 -- 172.21.15.119:0/547857521 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34005c340 con 0x5579592cec10 2024-04-25T10:14:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.098+0000 7fb347e00640 1 -- 172.21.15.119:0/547857521 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340060900 con 0x5579592cec10 2024-04-25T10:14:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.098+0000 7fb3615e3740 1 -- 172.21.15.119:0/547857521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 msgr2=0x5579592cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.098+0000 7fb3615e3740 1 --2- 172.21.15.119:0/547857521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592cf160 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fb340007fd0 tx=0x7fb340058a30 comp rx=0 tx=0).stop 2024-04-25T10:14:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.099+0000 7fb3615e3740 1 -- 172.21.15.119:0/547857521 shutdown_connections 2024-04-25T10:14:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.099+0000 7fb3615e3740 1 --2- 172.21.15.119:0/547857521 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557959287050 0x5579592cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.099+0000 7fb3615e3740 1 --2- 172.21.15.119:0/547857521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592cf160 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.099+0000 7fb3615e3740 1 --2- 172.21.15.119:0/547857521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592b1370 0x5579592b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.099+0000 7fb3615e3740 1 -- 172.21.15.119:0/547857521 >> 172.21.15.119:0/547857521 conn(0x557959283980 msgr2=0x5579592cc180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.099+0000 7fb3615e3740 1 -- 172.21.15.119:0/547857521 shutdown_connections 2024-04-25T10:14:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.099+0000 7fb3615e3740 1 -- 172.21.15.119:0/547857521 wait complete. 2024-04-25T10:14:47.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.099+0000 7fb3615e3740 1 -- start start 2024-04-25T10:14:47.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.100+0000 7fb3615e3740 1 Processor -- start 2024-04-25T10:14:47.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.100+0000 7fb3615e3740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592b1370 0x5579592b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.100+0000 7fb3615e3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592cec10 0x5579592cd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.100+0000 7fb3615e3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592adaa0 0x5579592cd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.100+0000 7fb3615e3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557959287df0 con 0x5579592adaa0 2024-04-25T10:14:47.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.101+0000 7fb3615e3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557959187d40 con 0x5579592cec10 2024-04-25T10:14:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.101+0000 7fb3615e3740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579592818b0 con 0x5579592b1370 2024-04-25T10:14:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.101+0000 7fb3615e3740 10 client.0 _ll_register_callbacks cb 0x557958efdba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.101+0000 7fb357e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592adaa0 0x5579592cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.101+0000 7fb357e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592adaa0 0x5579592cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43092/0 (socket says 172.21.15.119:43092) 2024-04-25T10:14:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.101+0000 7fb357e00640 1 -- 172.21.15.119:0/1145461504 learned_addr learned my addr 172.21.15.119:0/1145461504 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.101+0000 7fb357400640 1 --2- 172.21.15.119:0/1145461504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592b1370 0x5579592b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.101+0000 7fb356a00640 1 --2- 172.21.15.119:0/1145461504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592cec10 0x5579592cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.101+0000 7fb357e00640 1 -- 172.21.15.119:0/1145461504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592b1370 msgr2=0x5579592b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.101+0000 7fb357e00640 1 --2- 172.21.15.119:0/1145461504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592b1370 0x5579592b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.101+0000 7fb357e00640 1 -- 172.21.15.119:0/1145461504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592cec10 msgr2=0x5579592cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.101+0000 7fb357e00640 1 --2- 172.21.15.119:0/1145461504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592cec10 0x5579592cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.101+0000 7fb357e00640 1 -- 172.21.15.119:0/1145461504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55795902af50 con 0x5579592adaa0 2024-04-25T10:14:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.101+0000 7fb357400640 1 --2- 172.21.15.119:0/1145461504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592b1370 0x5579592b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.102+0000 7fb357e00640 1 --2- 172.21.15.119:0/1145461504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592adaa0 0x5579592cd640 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb35000e200 tx=0x7fb350009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd10c3e03b1a21b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.102+0000 7fb344c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.102+0000 7fb344c00640 1 -- 172.21.15.119:0/1145461504 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350037070 con 0x5579592adaa0 2024-04-25T10:14:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.102+0000 7fb344c00640 1 -- 172.21.15.119:0/1145461504 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fb3500041c0 con 0x5579592adaa0 2024-04-25T10:14:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.102+0000 7fb3615e3740 1 -- 172.21.15.119:0/1145461504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb34005c340 con 0x5579592adaa0 2024-04-25T10:14:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.102+0000 7fb344c00640 1 -- 172.21.15.119:0/1145461504 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb350049050 con 0x5579592adaa0 2024-04-25T10:14:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.102+0000 7fb3615e3740 10 client.7611 fetch_fsmap learned FSMap version 68 2024-04-25T10:14:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.102+0000 7fb3615e3740 1 -- 172.21.15.119:0/1145461504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x557958ebe0b0 con 0x5579592adaa0 2024-04-25T10:14:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.103+0000 7fb344c00640 1 -- 172.21.15.119:0/1145461504 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34005c340 con 0x5579592adaa0 2024-04-25T10:14:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.103+0000 7fb344c00640 1 -- 172.21.15.119:0/1145461504 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb35004c020 con 0x5579592adaa0 2024-04-25T10:14:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.103+0000 7fb3615e3740 10 client.7611 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:14:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.103+0000 7fb3615e3740 10 client.7611 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:14:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.103+0000 7fb3615e3740 20 client.7611 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.103+0000 7fb3615e3740 1 --2- 172.21.15.119:0/1145461504 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5579592dc010 0x5579592dc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.103+0000 7fb3615e3740 4 client.7611 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:47.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.103+0000 7fb3615e3740 1 -- 172.21.15.119:0/1145461504 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557959248b60 con 0x5579592dc010 2024-04-25T10:14:47.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.103+0000 7fb357400640 1 --2- 172.21.15.119:0/1145461504 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5579592dc010 0x5579592dc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.104+0000 7fb357400640 1 --2- 172.21.15.119:0/1145461504 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5579592dc010 0x5579592dc3f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=206bd54a0d5129f2 server_cookie=60b56746c84a9952 in_seq=0 out_seq=0 2024-04-25T10:14:47.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.104+0000 7fb344c00640 10 client.7611 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:47.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.104+0000 7fb344c00640 1 -- 172.21.15.119:0/1145461504 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557959248b60 con 0x5579592dc010 2024-04-25T10:14:47.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.104+0000 7fb344c00640 10 client.7611 handle_command_reply: tid=0 2024-04-25T10:14:47.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.104+0000 7fb3615e3740 1 client.7611 shutdown 2024-04-25T10:14:47.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.104+0000 7fb3615e3740 1 -- 172.21.15.119:0/1145461504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592adaa0 msgr2=0x5579592cd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.104+0000 7fb3615e3740 1 --2- 172.21.15.119:0/1145461504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592adaa0 0x5579592cd640 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb35000e200 tx=0x7fb350009750 comp rx=0 tx=0).stop 2024-04-25T10:14:47.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.105+0000 7fb3615e3740 1 -- 172.21.15.119:0/1145461504 shutdown_connections 2024-04-25T10:14:47.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.105+0000 7fb3615e3740 1 --2- 172.21.15.119:0/1145461504 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5579592dc010 0x5579592dc3f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.105+0000 7fb3615e3740 1 --2- 172.21.15.119:0/1145461504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592adaa0 0x5579592cd640 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.105+0000 7fb3615e3740 1 --2- 172.21.15.119:0/1145461504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592cec10 0x5579592cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.105+0000 7fb3615e3740 1 --2- 172.21.15.119:0/1145461504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592b1370 0x5579592b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.105+0000 7fb3615e3740 1 -- 172.21.15.119:0/1145461504 >> 172.21.15.119:0/1145461504 conn(0x557959287050 msgr2=0x5579592806e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.105+0000 7fb344c00640 0 client.7611 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:47.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.105+0000 7fb3615e3740 1 -- 172.21.15.119:0/1145461504 shutdown_connections 2024-04-25T10:14:47.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.105+0000 7fb3615e3740 1 -- 172.21.15.119:0/1145461504 wait complete. 2024-04-25T10:14:47.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.105+0000 7fb3615e3740 20 client.7611 trim_cache size 0 max 16384 2024-04-25T10:14:47.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.118+0000 7fb3615e3740 1 Processor -- start 2024-04-25T10:14:47.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.118+0000 7fb3615e3740 1 -- start start 2024-04-25T10:14:47.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.119+0000 7fb3615e3740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592adaa0 0x557959287430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.119+0000 7fb3615e3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592b1370 0x557959287970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.119+0000 7fb3615e3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.119+0000 7fb3615e3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3581ffb10 con 0x5579592cec10 2024-04-25T10:14:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.119+0000 7fb3615e3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3581f4850 con 0x5579592b1370 2024-04-25T10:14:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.119+0000 7fb3615e3740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3581ff510 con 0x5579592adaa0 2024-04-25T10:14:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.120+0000 7fb357e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.120+0000 7fb357e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43108/0 (socket says 172.21.15.119:43108) 2024-04-25T10:14:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.120+0000 7fb357e00640 1 -- 172.21.15.119:0/554685176 learned_addr learned my addr 172.21.15.119:0/554685176 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:47.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.120+0000 7fb357400640 1 --2- 172.21.15.119:0/554685176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592adaa0 0x557959287430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.120+0000 7fb356a00640 1 --2- 172.21.15.119:0/554685176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592b1370 0x557959287970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.120+0000 7fb357e00640 1 -- 172.21.15.119:0/554685176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592adaa0 msgr2=0x557959287430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.120+0000 7fb357e00640 1 --2- 172.21.15.119:0/554685176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592adaa0 0x557959287430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.120+0000 7fb357e00640 1 -- 172.21.15.119:0/554685176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592b1370 msgr2=0x557959287970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.120+0000 7fb357e00640 1 --2- 172.21.15.119:0/554685176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592b1370 0x557959287970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.120+0000 7fb357e00640 1 -- 172.21.15.119:0/554685176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579592c40e0 con 0x5579592cec10 2024-04-25T10:14:47.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.121+0000 7fb357e00640 1 --2- 172.21.15.119:0/554685176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592ccf30 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb3500094d0 tx=0x7fb350037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19ba6f0dd6e05fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.121+0000 7fb347400640 1 -- 172.21.15.119:0/554685176 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3500047a0 con 0x5579592cec10 2024-04-25T10:14:47.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.121+0000 7fb347400640 1 -- 172.21.15.119:0/554685176 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb350041070 con 0x5579592cec10 2024-04-25T10:14:47.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.121+0000 7fb347400640 1 -- 172.21.15.119:0/554685176 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3500040f0 con 0x5579592cec10 2024-04-25T10:14:47.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.122+0000 7fb3615e3740 1 -- 172.21.15.119:0/554685176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 msgr2=0x5579592ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.122+0000 7fb3615e3740 1 --2- 172.21.15.119:0/554685176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592ccf30 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb3500094d0 tx=0x7fb350037000 comp rx=0 tx=0).stop 2024-04-25T10:14:47.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.122+0000 7fb3615e3740 1 -- 172.21.15.119:0/554685176 shutdown_connections 2024-04-25T10:14:47.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.122+0000 7fb3615e3740 1 --2- 172.21.15.119:0/554685176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592ccf30 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.122+0000 7fb3615e3740 1 --2- 172.21.15.119:0/554685176 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592b1370 0x557959287970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.122+0000 7fb3615e3740 1 --2- 172.21.15.119:0/554685176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592adaa0 0x557959287430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.122+0000 7fb3615e3740 1 -- 172.21.15.119:0/554685176 >> 172.21.15.119:0/554685176 conn(0x557959287050 msgr2=0x5579592b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.122+0000 7fb3615e3740 1 -- 172.21.15.119:0/554685176 shutdown_connections 2024-04-25T10:14:47.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.122+0000 7fb3615e3740 1 -- 172.21.15.119:0/554685176 wait complete. 2024-04-25T10:14:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.123+0000 7fb3615e3740 1 -- start start 2024-04-25T10:14:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.123+0000 7fb3615e3740 1 Processor -- start 2024-04-25T10:14:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb3615e3740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592adaa0 0x557959287940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb3615e3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592b1370 0x5579592dd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb3615e3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb3615e3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579592b4c90 con 0x5579592cec10 2024-04-25T10:14:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb3615e3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579592c42b0 con 0x5579592b1370 2024-04-25T10:14:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb3615e3740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579592cfe60 con 0x5579592adaa0 2024-04-25T10:14:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb3615e3740 10 client.0 _ll_register_callbacks cb 0x557959281d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb357e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb357e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43112/0 (socket says 172.21.15.119:43112) 2024-04-25T10:14:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb357e00640 1 -- 172.21.15.119:0/3510458447 learned_addr learned my addr 172.21.15.119:0/3510458447 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb357400640 1 --2- 172.21.15.119:0/3510458447 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592adaa0 0x557959287940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb357e00640 1 -- 172.21.15.119:0/3510458447 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592adaa0 msgr2=0x557959287940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb357e00640 1 --2- 172.21.15.119:0/3510458447 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592adaa0 0x557959287940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb357e00640 1 -- 172.21.15.119:0/3510458447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592b1370 msgr2=0x5579592dd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb357e00640 1 --2- 172.21.15.119:0/3510458447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592b1370 0x5579592dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.124+0000 7fb357e00640 1 -- 172.21.15.119:0/3510458447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557959281010 con 0x5579592cec10 2024-04-25T10:14:47.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.125+0000 7fb357e00640 1 --2- 172.21.15.119:0/3510458447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592cc040 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb350009590 tx=0x7fb35002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e4580c8b113ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.125+0000 7fb344c00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:47.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.125+0000 7fb344c00640 1 -- 172.21.15.119:0/3510458447 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35005a060 con 0x5579592cec10 2024-04-25T10:14:47.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.125+0000 7fb344c00640 1 -- 172.21.15.119:0/3510458447 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fb350032640 con 0x5579592cec10 2024-04-25T10:14:47.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.125+0000 7fb3615e3740 1 -- 172.21.15.119:0/3510458447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb350041070 con 0x5579592cec10 2024-04-25T10:14:47.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.125+0000 7fb344c00640 1 -- 172.21.15.119:0/3510458447 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb350060050 con 0x5579592cec10 2024-04-25T10:14:47.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.125+0000 7fb3615e3740 10 client.7617 fetch_fsmap learned FSMap version 68 2024-04-25T10:14:47.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.125+0000 7fb3615e3740 1 -- 172.21.15.119:0/3510458447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x557959280be0 con 0x5579592cec10 2024-04-25T10:14:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.126+0000 7fb344c00640 1 -- 172.21.15.119:0/3510458447 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb350041070 con 0x5579592cec10 2024-04-25T10:14:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.126+0000 7fb344c00640 1 -- 172.21.15.119:0/3510458447 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb350063020 con 0x5579592cec10 2024-04-25T10:14:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.126+0000 7fb3615e3740 10 client.7617 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:14:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.126+0000 7fb3615e3740 10 client.7617 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:14:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.126+0000 7fb3615e3740 20 client.7617 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.126+0000 7fb3615e3740 1 --2- 172.21.15.119:0/3510458447 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5579592cc580 0x5579592c2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.126+0000 7fb3615e3740 4 client.7617 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:14:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.126+0000 7fb3615e3740 1 -- 172.21.15.119:0/3510458447 --> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5579591f7020 con 0x5579592cc580 2024-04-25T10:14:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.126+0000 7fb357400640 1 --2- 172.21.15.119:0/3510458447 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5579592cc580 0x5579592c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb357400640 1 --2- 172.21.15.119:0/3510458447 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5579592cc580 0x5579592c2280 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=994907a72e0955b2 server_cookie=596b6c46d17703d8 in_seq=0 out_seq=0 2024-04-25T10:14:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb344c00640 10 client.7617 ms_handle_connect on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb344c00640 1 -- 172.21.15.119:0/3510458447 <== mds.0 v2:172.21.15.119:6835/1734507541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+143 (crc 0 0 0) 0x5579591f7020 con 0x5579592cc580 2024-04-25T10:14:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb344c00640 10 client.7617 handle_command_reply: tid=0 2024-04-25T10:14:47.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb3615e3740 1 client.7617 shutdown 2024-04-25T10:14:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb3615e3740 1 -- 172.21.15.119:0/3510458447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 msgr2=0x5579592cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb3615e3740 1 --2- 172.21.15.119:0/3510458447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592cc040 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb350009590 tx=0x7fb35002f000 comp rx=0 tx=0).stop 2024-04-25T10:14:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb3615e3740 1 -- 172.21.15.119:0/3510458447 shutdown_connections 2024-04-25T10:14:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb3615e3740 1 --2- 172.21.15.119:0/3510458447 >> [v2:172.21.15.119:6835/1734507541,v1:172.21.15.119:6837/1734507541] conn(0x5579592cc580 0x5579592c2280 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb3615e3740 1 --2- 172.21.15.119:0/3510458447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579592cec10 0x5579592cc040 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb3615e3740 1 --2- 172.21.15.119:0/3510458447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5579592b1370 0x5579592dd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb344c00640 0 client.7617 ms_handle_reset on v2:172.21.15.119:6835/1734507541 2024-04-25T10:14:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb3615e3740 1 --2- 172.21.15.119:0/3510458447 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5579592adaa0 0x557959287940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb3615e3740 1 -- 172.21.15.119:0/3510458447 >> 172.21.15.119:0/3510458447 conn(0x557959287050 msgr2=0x557959286750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb3615e3740 1 -- 172.21.15.119:0/3510458447 shutdown_connections 2024-04-25T10:14:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb3615e3740 1 -- 172.21.15.119:0/3510458447 wait complete. 2024-04-25T10:14:47.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.127+0000 7fb3615e3740 20 client.7617 trim_cache size 0 max 16384 2024-04-25T10:14:47.136 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:14:47.136 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2024-04-25T10:14:47.136 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:14:47.136 INFO:teuthology.orchestra.run.smithi119.stdout: "damage_type": "backtrace", 2024-04-25T10:14:47.136 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 468801377, 2024-04-25T10:14:47.136 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 1099511627777, 2024-04-25T10:14:47.136 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/testdir/hardlink" 2024-04-25T10:14:47.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:14:47.136 INFO:teuthology.orchestra.run.smithi119.stdout:] 2024-04-25T10:14:47.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.128+0000 7fb35e800640 1 -- 172.21.15.119:0/1641337765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb33406b4d0 msgr2=0x7fb33406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.128+0000 7fb35e800640 1 --2- 172.21.15.119:0/1641337765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb33406b4d0 0x7fb33406d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb348001200 tx=0x7fb348009040 comp rx=0 tx=0).stop 2024-04-25T10:14:47.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.128+0000 7fb35e800640 1 -- 172.21.15.119:0/1641337765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3580b1bd0 msgr2=0x7fb3580ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.128+0000 7fb35e800640 1 --2- 172.21.15.119:0/1641337765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3580b1bd0 0x7fb3580ce1f0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fb34000d800 tx=0x7fb34000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.128+0000 7fb35e800640 1 -- 172.21.15.119:0/1641337765 shutdown_connections 2024-04-25T10:14:47.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.128+0000 7fb35e800640 1 --2- 172.21.15.119:0/1641337765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb33406b4d0 0x7fb33406d990 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.128+0000 7fb35e800640 1 --2- 172.21.15.119:0/1641337765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb35814ec90 0x7fb35814f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.128+0000 7fb35e800640 1 --2- 172.21.15.119:0/1641337765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3580b1bd0 0x7fb3580ce1f0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.128+0000 7fb35e800640 1 --2- 172.21.15.119:0/1641337765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb33c004d40 0x7fb3581f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.128+0000 7fb35e800640 1 -- 172.21.15.119:0/1641337765 >> 172.21.15.119:0/1641337765 conn(0x7fb35806aba0 msgr2=0x7fb3581469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.128+0000 7fb35e800640 1 -- 172.21.15.119:0/1641337765 shutdown_connections 2024-04-25T10:14:47.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.128+0000 7fb35e800640 1 -- 172.21.15.119:0/1641337765 wait complete. 2024-04-25T10:14:47.150 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:14:47.150 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:47.196 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:14:47.196 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:47.290 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.483+0000 7f400e600640 1 Processor -- start 2024-04-25T10:14:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.484+0000 7f400e600640 1 -- start start 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.485+0000 7f400e600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f400815bf00 0x7f40080cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.485+0000 7f400e600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40080ce6c0 con 0x7f40080cc310 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.485+0000 7f400e600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40080cc7b0 con 0x7f400806e8f0 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.485+0000 7f400e600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40081455e0 con 0x7f400815bf00 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.485+0000 7f4007400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f400815bf00 0x7f40080cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.485+0000 7f4007400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f400815bf00 0x7f40080cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43124/0 (socket says 172.21.15.119:43124) 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.485+0000 7f4007400640 1 -- 172.21.15.119:0/540352623 learned_addr learned my addr 172.21.15.119:0/540352623 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.485+0000 7f3fffe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f40080cc310 0x7f400815c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59460/0 (socket says 172.21.15.119:59460) 2024-04-25T10:14:47.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.486+0000 7f4007400640 1 -- 172.21.15.119:0/540352623 >> v1:172.21.15.132:6789/0 conn(0x7f400806e8f0 legacy=0x7f40081654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.486+0000 7f4007400640 1 -- 172.21.15.119:0/540352623 >> v1:172.21.15.119:6789/0 conn(0x7f40080cc310 legacy=0x7f400815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.486+0000 7f4007400640 1 -- 172.21.15.119:0/540352623 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40080cca90 con 0x7f400815bf00 2024-04-25T10:14:47.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.487+0000 7f4007400640 1 --2- 172.21.15.119:0/540352623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400815bf00 0x7f40080cbdd0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009870 tx=0x7f3ff802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b0d4e836119332 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.487+0000 7f4006a00640 1 -- 172.21.15.119:0/540352623 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8037070 con 0x7f400815bf00 2024-04-25T10:14:47.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.487+0000 7f4006a00640 1 -- 172.21.15.119:0/540352623 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff8032440 con 0x7f400815bf00 2024-04-25T10:14:47.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.487+0000 7f4006a00640 1 -- 172.21.15.119:0/540352623 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8032810 con 0x7f400815bf00 2024-04-25T10:14:47.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.488+0000 7f400e600640 1 -- 172.21.15.119:0/540352623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400815bf00 msgr2=0x7f40080cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.488+0000 7f400e600640 1 --2- 172.21.15.119:0/540352623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400815bf00 0x7f40080cbdd0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009870 tx=0x7f3ff802a2a0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.488+0000 7f400e600640 1 -- 172.21.15.119:0/540352623 shutdown_connections 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.488+0000 7f400e600640 1 --2- 172.21.15.119:0/540352623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400815bf00 0x7f40080cbdd0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.488+0000 7f400e600640 1 -- 172.21.15.119:0/540352623 >> 172.21.15.119:0/540352623 conn(0x7f400806aba0 msgr2=0x7f400806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.488+0000 7f400e600640 1 -- 172.21.15.119:0/540352623 shutdown_connections 2024-04-25T10:14:47.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.488+0000 7f400e600640 1 -- 172.21.15.119:0/540352623 wait complete. 2024-04-25T10:14:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.489+0000 7f400e600640 1 Processor -- start 2024-04-25T10:14:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.490+0000 7f400e600640 1 -- start start 2024-04-25T10:14:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.490+0000 7f400e600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f400806e8f0 0x7f40080cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.490+0000 7f400e600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40080cc310 0x7f40080b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.491+0000 7f400e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400815bf00 0x7f400815a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.491+0000 7f400e600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40080cd050 con 0x7f400815bf00 2024-04-25T10:14:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.491+0000 7f400e600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400806d450 con 0x7f400806e8f0 2024-04-25T10:14:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.491+0000 7f400e600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400806ee60 con 0x7f40080cc310 2024-04-25T10:14:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.491+0000 7f4007e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400815bf00 0x7f400815a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.491+0000 7f4007e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400815bf00 0x7f400815a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43126/0 (socket says 172.21.15.119:43126) 2024-04-25T10:14:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.491+0000 7f4007e00640 1 -- 172.21.15.119:0/2983336330 learned_addr learned my addr 172.21.15.119:0/2983336330 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.491+0000 7f4007e00640 1 -- 172.21.15.119:0/2983336330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40080cc310 msgr2=0x7f40080b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.491+0000 7f4007e00640 1 --2- 172.21.15.119:0/2983336330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40080cc310 0x7f40080b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.491+0000 7f4007e00640 1 -- 172.21.15.119:0/2983336330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f400806e8f0 msgr2=0x7f40080cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.491+0000 7f4007e00640 1 --2- 172.21.15.119:0/2983336330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f400806e8f0 0x7f40080cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.491+0000 7f4007e00640 1 -- 172.21.15.119:0/2983336330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40081f3b80 con 0x7f400815bf00 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.491+0000 7f4007e00640 1 --2- 172.21.15.119:0/2983336330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400815bf00 0x7f400815a8b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f400000b970 tx=0x7f400000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.492+0000 7f3fff400640 1 -- 172.21.15.119:0/2983336330 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000019070 con 0x7f400815bf00 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.492+0000 7f3fff400640 1 -- 172.21.15.119:0/2983336330 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400000cb40 con 0x7f400815bf00 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.492+0000 7f3fff400640 1 -- 172.21.15.119:0/2983336330 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000014440 con 0x7f400815bf00 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.492+0000 7f400e600640 1 -- 172.21.15.119:0/2983336330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40081f3e60 con 0x7f400815bf00 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.492+0000 7f400e600640 1 -- 172.21.15.119:0/2983336330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40080cb090 con 0x7f400815bf00 2024-04-25T10:14:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.493+0000 7f400e600640 1 -- 172.21.15.119:0/2983336330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400806bb30 con 0x7f400815bf00 2024-04-25T10:14:47.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.494+0000 7f3fff400640 1 -- 172.21.15.119:0/2983336330 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f400001c070 con 0x7f400815bf00 2024-04-25T10:14:47.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.495+0000 7f3fff400640 1 --2- 172.21.15.119:0/2983336330 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3fe406b4d0 0x7f3fe406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.496+0000 7f3fff400640 1 -- 172.21.15.119:0/2983336330 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f4000089e10 con 0x7f400815bf00 2024-04-25T10:14:47.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.496+0000 7f4007400640 1 --2- 172.21.15.119:0/2983336330 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3fe406b4d0 0x7f3fe406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.496+0000 7f4007400640 1 --2- 172.21.15.119:0/2983336330 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3fe406b4d0 0x7f3fe406d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8034000 tx=0x7f3ff802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.500+0000 7f3fff400640 1 -- 172.21.15.119:0/2983336330 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4000058cb0 con 0x7f400815bf00 2024-04-25T10:14:47.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.694+0000 7f400e600640 1 -- 172.21.15.119:0/2983336330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40080749b0 con 0x7f400815bf00 2024-04-25T10:14:47.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.697+0000 7f3fff400640 1 -- 172.21.15.119:0/2983336330 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f40000585b0 con 0x7f400815bf00 2024-04-25T10:14:47.702 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:14:47.702 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":84,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:13:52.265021+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:21.210576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:22.438674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/3997964942":"2024-04-26T10:13:36.160373+0000","172.21.15.119:6835/3997964942":"2024-04-26T10:13:36.160373+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:47.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.700+0000 7f400e600640 1 -- 172.21.15.119:0/2983336330 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3fe406b4d0 msgr2=0x7f3fe406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.700+0000 7f400e600640 1 --2- 172.21.15.119:0/2983336330 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3fe406b4d0 0x7f3fe406d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8034000 tx=0x7f3ff802c000 comp rx=0 tx=0).stop 2024-04-25T10:14:47.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.700+0000 7f400e600640 1 -- 172.21.15.119:0/2983336330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400815bf00 msgr2=0x7f400815a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.700+0000 7f400e600640 1 --2- 172.21.15.119:0/2983336330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400815bf00 0x7f400815a8b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f400000b970 tx=0x7f400000be30 comp rx=0 tx=0).stop 2024-04-25T10:14:47.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.700+0000 7f400e600640 1 -- 172.21.15.119:0/2983336330 shutdown_connections 2024-04-25T10:14:47.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.700+0000 7f400e600640 1 --2- 172.21.15.119:0/2983336330 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3fe406b4d0 0x7f3fe406d990 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.700+0000 7f400e600640 1 --2- 172.21.15.119:0/2983336330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400815bf00 0x7f400815a8b0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.700+0000 7f400e600640 1 --2- 172.21.15.119:0/2983336330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40080cc310 0x7f40080b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.700+0000 7f400e600640 1 --2- 172.21.15.119:0/2983336330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f400806e8f0 0x7f40080cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.700+0000 7f400e600640 1 -- 172.21.15.119:0/2983336330 >> 172.21.15.119:0/2983336330 conn(0x7f400806aba0 msgr2=0x7f40080cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.700+0000 7f400e600640 1 -- 172.21.15.119:0/2983336330 shutdown_connections 2024-04-25T10:14:47.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.700+0000 7f400e600640 1 -- 172.21.15.119:0/2983336330 wait complete. 2024-04-25T10:14:47.723 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:21.210576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:22.438674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:47.724 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:47.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.924+0000 7f9a59800640 1 Processor -- start 2024-04-25T10:14:47.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.924+0000 7f9a59800640 1 -- start start 2024-04-25T10:14:47.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a59800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a541450e0 0x7f9a5406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a59800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9a5406b0e0 0x7f9a540cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a59800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9a540cc810 0x7f9a54071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a59800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a541454c0 con 0x7f9a540cc810 2024-04-25T10:14:47.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a59800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a5406b540 con 0x7f9a5406b0e0 2024-04-25T10:14:47.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a59800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5415f400 con 0x7f9a541450e0 2024-04-25T10:14:47.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a58e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9a540cc810 0x7f9a54071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a58e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9a540cc810 0x7f9a54071050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43142/0 (socket says 172.21.15.119:43142) 2024-04-25T10:14:47.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a58e00640 1 -- 172.21.15.119:0/4210559070 learned_addr learned my addr 172.21.15.119:0/4210559070 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:47.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a53e00640 1 --2- 172.21.15.119:0/4210559070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a541450e0 0x7f9a5406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a4b400640 1 --2- 172.21.15.119:0/4210559070 >> v2:172.21.15.132:3300/0 conn(0x7f9a5406b0e0 0x7f9a540cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a58e00640 1 -- 172.21.15.119:0/4210559070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a541450e0 msgr2=0x7f9a5406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a58e00640 1 --2- 172.21.15.119:0/4210559070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a541450e0 0x7f9a5406aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a58e00640 1 -- 172.21.15.119:0/4210559070 >> v2:172.21.15.132:3300/0 conn(0x7f9a5406b0e0 msgr2=0x7f9a540cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a58e00640 1 --2- 172.21.15.119:0/4210559070 >> v2:172.21.15.132:3300/0 conn(0x7f9a5406b0e0 0x7f9a540cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.926+0000 7f9a58e00640 1 -- 172.21.15.119:0/4210559070 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5415f6e0 con 0x7f9a540cc810 2024-04-25T10:14:47.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.928+0000 7f9a58e00640 1 --2- 172.21.15.119:0/4210559070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a540cc810 0x7f9a54071050 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f9a5406f4c0 tx=0x7f9a4c02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef12cda1baacdc8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.929+0000 7f9a53400640 1 -- 172.21.15.119:0/4210559070 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c036070 con 0x7f9a540cc810 2024-04-25T10:14:47.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.929+0000 7f9a53400640 1 -- 172.21.15.119:0/4210559070 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c031600 con 0x7f9a540cc810 2024-04-25T10:14:47.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.929+0000 7f9a53400640 1 -- 172.21.15.119:0/4210559070 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0319d0 con 0x7f9a540cc810 2024-04-25T10:14:47.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.929+0000 7f9a59800640 1 -- 172.21.15.119:0/4210559070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a540cc810 msgr2=0x7f9a54071050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.929+0000 7f9a59800640 1 --2- 172.21.15.119:0/4210559070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a540cc810 0x7f9a54071050 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f9a5406f4c0 tx=0x7f9a4c02a570 comp rx=0 tx=0).stop 2024-04-25T10:14:47.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.929+0000 7f9a59800640 1 -- 172.21.15.119:0/4210559070 shutdown_connections 2024-04-25T10:14:47.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.929+0000 7f9a59800640 1 --2- 172.21.15.119:0/4210559070 >> v2:172.21.15.132:3300/0 conn(0x7f9a5406b0e0 0x7f9a540cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.929+0000 7f9a59800640 1 --2- 172.21.15.119:0/4210559070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a540cc810 0x7f9a54071050 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.929+0000 7f9a59800640 1 --2- 172.21.15.119:0/4210559070 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a541450e0 0x7f9a5406aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.929+0000 7f9a59800640 1 -- 172.21.15.119:0/4210559070 >> 172.21.15.119:0/4210559070 conn(0x7f9a54066ff0 msgr2=0x7f9a540b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.929+0000 7f9a59800640 1 -- 172.21.15.119:0/4210559070 shutdown_connections 2024-04-25T10:14:47.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.930+0000 7f9a59800640 1 -- 172.21.15.119:0/4210559070 wait complete. 2024-04-25T10:14:47.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.930+0000 7f9a59800640 1 Processor -- start 2024-04-25T10:14:47.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.930+0000 7f9a59800640 1 -- start start 2024-04-25T10:14:47.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.931+0000 7f9a59800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a5406b0e0 0x7f9a540c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.931+0000 7f9a59800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a540cc810 0x7f9a541f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.931+0000 7f9a59800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a541450e0 0x7f9a541f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.931+0000 7f9a59800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a540ccca0 con 0x7f9a5406b0e0 2024-04-25T10:14:47.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.931+0000 7f9a59800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5415fca0 con 0x7f9a541450e0 2024-04-25T10:14:47.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.931+0000 7f9a59800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5406e030 con 0x7f9a540cc810 2024-04-25T10:14:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.933+0000 7f9a53e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a5406b0e0 0x7f9a540c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.933+0000 7f9a53e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a5406b0e0 0x7f9a540c4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43154/0 (socket says 172.21.15.119:43154) 2024-04-25T10:14:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.933+0000 7f9a53e00640 1 -- 172.21.15.119:0/3096142061 learned_addr learned my addr 172.21.15.119:0/3096142061 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.933+0000 7f9a58e00640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a541450e0 0x7f9a541f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.933+0000 7f9a4b400640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a540cc810 0x7f9a541f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.933+0000 7f9a53e00640 1 -- 172.21.15.119:0/3096142061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a540cc810 msgr2=0x7f9a541f7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.933+0000 7f9a53e00640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a540cc810 0x7f9a541f7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.933+0000 7f9a53e00640 1 -- 172.21.15.119:0/3096142061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a541450e0 msgr2=0x7f9a541f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.933+0000 7f9a53e00640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a541450e0 0x7f9a541f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.933+0000 7f9a53e00640 1 -- 172.21.15.119:0/3096142061 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5406e310 con 0x7f9a5406b0e0 2024-04-25T10:14:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.933+0000 7f9a58e00640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a541450e0 0x7f9a541f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.934+0000 7f9a53e00640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a5406b0e0 0x7f9a540c4fb0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400d800 tx=0x7f9a4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.934+0000 7f9a50c00640 1 -- 172.21.15.119:0/3096142061 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400b6c0 con 0x7f9a5406b0e0 2024-04-25T10:14:47.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.934+0000 7f9a50c00640 1 -- 172.21.15.119:0/3096142061 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a44019070 con 0x7f9a5406b0e0 2024-04-25T10:14:47.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.934+0000 7f9a50c00640 1 -- 172.21.15.119:0/3096142061 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400ba80 con 0x7f9a5406b0e0 2024-04-25T10:14:47.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.934+0000 7f9a59800640 1 -- 172.21.15.119:0/3096142061 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5406e5f0 con 0x7f9a5406b0e0 2024-04-25T10:14:47.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.934+0000 7f9a59800640 1 -- 172.21.15.119:0/3096142061 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a540acc30 con 0x7f9a5406b0e0 2024-04-25T10:14:47.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.934+0000 7f9a59800640 1 -- 172.21.15.119:0/3096142061 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a54067490 con 0x7f9a5406b0e0 2024-04-25T10:14:47.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.936+0000 7f9a50c00640 1 -- 172.21.15.119:0/3096142061 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a4400bc20 con 0x7f9a5406b0e0 2024-04-25T10:14:47.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.936+0000 7f9a50c00640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a3006b4d0 0x7f9a3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.936+0000 7f9a50c00640 1 -- 172.21.15.119:0/3096142061 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f9a4408a7a0 con 0x7f9a5406b0e0 2024-04-25T10:14:47.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.936+0000 7f9a4b400640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a3006b4d0 0x7f9a3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.937+0000 7f9a4b400640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a3006b4d0 0x7f9a3006d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c006fd0 tx=0x7f9a3c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:47.941+0000 7f9a50c00640 1 -- 172.21.15.119:0/3096142061 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a54067490 con 0x7f9a5406b0e0 2024-04-25T10:14:48.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.133+0000 7f9a59800640 1 -- 172.21.15.119:0/3096142061 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a540749b0 con 0x7f9a5406b0e0 2024-04-25T10:14:48.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.134+0000 7f9a50c00640 1 -- 172.21.15.119:0/3096142061 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a44059530 con 0x7f9a5406b0e0 2024-04-25T10:14:48.138 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:14:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.136+0000 7f9a59800640 1 -- 172.21.15.119:0/3096142061 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a3006b4d0 msgr2=0x7f9a3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.136+0000 7f9a59800640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a3006b4d0 0x7f9a3006d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c006fd0 tx=0x7f9a3c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.136+0000 7f9a59800640 1 -- 172.21.15.119:0/3096142061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a5406b0e0 msgr2=0x7f9a540c4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.136+0000 7f9a59800640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a5406b0e0 0x7f9a540c4fb0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400d800 tx=0x7f9a4400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.136+0000 7f9a59800640 1 -- 172.21.15.119:0/3096142061 shutdown_connections 2024-04-25T10:14:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.136+0000 7f9a59800640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a3006b4d0 0x7f9a3006d990 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.136+0000 7f9a59800640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a541450e0 0x7f9a541f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.136+0000 7f9a59800640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a540cc810 0x7f9a541f7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.136+0000 7f9a59800640 1 --2- 172.21.15.119:0/3096142061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a5406b0e0 0x7f9a540c4fb0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.137+0000 7f9a59800640 1 -- 172.21.15.119:0/3096142061 >> 172.21.15.119:0/3096142061 conn(0x7f9a54066ff0 msgr2=0x7f9a5415e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.137+0000 7f9a59800640 1 -- 172.21.15.119:0/3096142061 shutdown_connections 2024-04-25T10:14:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.137+0000 7f9a59800640 1 -- 172.21.15.119:0/3096142061 wait complete. 2024-04-25T10:14:48.159 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:48.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.345+0000 7f87c7200640 1 Processor -- start 2024-04-25T10:14:48.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.345+0000 7f87c7200640 1 -- start start 2024-04-25T10:14:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.347+0000 7f87c7200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f87c806aba0 0x7f87c806af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.347+0000 7f87c7200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c806f040 0x7f87c815e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.347+0000 7f87c7200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c806b4c0 con 0x7f87c80cec60 2024-04-25T10:14:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.347+0000 7f87c7200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c806d080 con 0x7f87c806aba0 2024-04-25T10:14:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.347+0000 7f87c7200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c806d360 con 0x7f87c806f040 2024-04-25T10:14:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.347+0000 7f87c5e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f87c806aba0 0x7f87c806af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.348+0000 7f87c5e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f87c806aba0 0x7f87c806af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:42506/0 (socket says 172.21.15.119:42506) 2024-04-25T10:14:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.348+0000 7f87c5e00640 1 -- 172.21.15.119:0/351811571 learned_addr learned my addr 172.21.15.119:0/351811571 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.347+0000 7f87c5400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c806f040 0x7f87c815e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.348+0000 7f87c5e00640 1 -- 172.21.15.119:0/351811571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c806f040 msgr2=0x7f87c815e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.348+0000 7f87c5e00640 1 --2- 172.21.15.119:0/351811571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c806f040 0x7f87c815e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.348+0000 7f87c5e00640 1 -- 172.21.15.119:0/351811571 >> v1:172.21.15.132:6789/0 conn(0x7f87c80cec60 legacy=0x7f87c815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.348+0000 7f87c5e00640 1 -- 172.21.15.119:0/351811571 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c806d640 con 0x7f87c806aba0 2024-04-25T10:14:48.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.348+0000 7f87c5400640 1 --2- 172.21.15.119:0/351811571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c806f040 0x7f87c815e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:48.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.350+0000 7f87c5e00640 1 --2- 172.21.15.119:0/351811571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c806aba0 0x7f87c806af80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009870 tx=0x7f87b802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eb1698a684b2524 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.350+0000 7f87c4a00640 1 -- 172.21.15.119:0/351811571 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8037070 con 0x7f87c806aba0 2024-04-25T10:14:48.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.350+0000 7f87c4a00640 1 -- 172.21.15.119:0/351811571 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b8032440 con 0x7f87c806aba0 2024-04-25T10:14:48.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.350+0000 7f87c4a00640 1 -- 172.21.15.119:0/351811571 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8032810 con 0x7f87c806aba0 2024-04-25T10:14:48.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.351+0000 7f87c7200640 1 -- 172.21.15.119:0/351811571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c806aba0 msgr2=0x7f87c806af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.351+0000 7f87c7200640 1 --2- 172.21.15.119:0/351811571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c806aba0 0x7f87c806af80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009870 tx=0x7f87b802a250 comp rx=0 tx=0).stop 2024-04-25T10:14:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.351+0000 7f87c7200640 1 -- 172.21.15.119:0/351811571 shutdown_connections 2024-04-25T10:14:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.351+0000 7f87c7200640 1 --2- 172.21.15.119:0/351811571 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c806f040 0x7f87c815e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.351+0000 7f87c7200640 1 --2- 172.21.15.119:0/351811571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c806aba0 0x7f87c806af80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.351+0000 7f87c7200640 1 -- 172.21.15.119:0/351811571 >> 172.21.15.119:0/351811571 conn(0x7f87c8066ff0 msgr2=0x7f87c80b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.351+0000 7f87c7200640 1 -- 172.21.15.119:0/351811571 shutdown_connections 2024-04-25T10:14:48.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.351+0000 7f87c7200640 1 -- 172.21.15.119:0/351811571 wait complete. 2024-04-25T10:14:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.353+0000 7f87c7200640 1 Processor -- start 2024-04-25T10:14:48.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.354+0000 7f87c7200640 1 -- start start 2024-04-25T10:14:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.355+0000 7f87c7200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87c806aba0 0x7f87c81f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.355+0000 7f87c7200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c806f040 0x7f87c81f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.355+0000 7f87c7200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c80cec60 0x7f87c8152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.355+0000 7f87c7200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c815a980 con 0x7f87c806aba0 2024-04-25T10:14:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.355+0000 7f87c7200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c806dc00 con 0x7f87c80cec60 2024-04-25T10:14:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.355+0000 7f87c7200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c80aed40 con 0x7f87c806f040 2024-04-25T10:14:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.355+0000 7f87c5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87c806aba0 0x7f87c81f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.355+0000 7f87c5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87c806aba0 0x7f87c81f13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43158/0 (socket says 172.21.15.119:43158) 2024-04-25T10:14:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.355+0000 7f87c5e00640 1 -- 172.21.15.119:0/4179400817 learned_addr learned my addr 172.21.15.119:0/4179400817 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.355+0000 7f87c5e00640 1 -- 172.21.15.119:0/4179400817 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c806f040 msgr2=0x7f87c81f7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.355+0000 7f87c5e00640 1 --2- 172.21.15.119:0/4179400817 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c806f040 0x7f87c81f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.355+0000 7f87c5e00640 1 -- 172.21.15.119:0/4179400817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c80cec60 msgr2=0x7f87c8152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.355+0000 7f87c6800640 1 --2- 172.21.15.119:0/4179400817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c80cec60 0x7f87c8152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.356+0000 7f87c5e00640 1 --2- 172.21.15.119:0/4179400817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c80cec60 0x7f87c8152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.356+0000 7f87c5e00640 1 -- 172.21.15.119:0/4179400817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c81545c0 con 0x7f87c806aba0 2024-04-25T10:14:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.356+0000 7f87c6800640 1 --2- 172.21.15.119:0/4179400817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c80cec60 0x7f87c8152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.356+0000 7f87c5e00640 1 --2- 172.21.15.119:0/4179400817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87c806aba0 0x7f87c81f13b0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009870 tx=0x7f87b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.356+0000 7f87be000640 1 -- 172.21.15.119:0/4179400817 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8041070 con 0x7f87c806aba0 2024-04-25T10:14:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.356+0000 7f87be000640 1 -- 172.21.15.119:0/4179400817 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b803c890 con 0x7f87c806aba0 2024-04-25T10:14:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.356+0000 7f87be000640 1 -- 172.21.15.119:0/4179400817 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8037040 con 0x7f87c806aba0 2024-04-25T10:14:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.356+0000 7f87c7200640 1 -- 172.21.15.119:0/4179400817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c81548a0 con 0x7f87c806aba0 2024-04-25T10:14:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.356+0000 7f87c7200640 1 -- 172.21.15.119:0/4179400817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c81f18f0 con 0x7f87c806aba0 2024-04-25T10:14:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.358+0000 7f87c7200640 1 -- 172.21.15.119:0/4179400817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c80749b0 con 0x7f87c806aba0 2024-04-25T10:14:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.359+0000 7f87be000640 1 -- 172.21.15.119:0/4179400817 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87b8044070 con 0x7f87c806aba0 2024-04-25T10:14:48.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.359+0000 7f87be000640 1 --2- 172.21.15.119:0/4179400817 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87a806b500 0x7f87a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.359+0000 7f87be000640 1 -- 172.21.15.119:0/4179400817 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f87b80aabd0 con 0x7f87c806aba0 2024-04-25T10:14:48.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.360+0000 7f87c5400640 1 --2- 172.21.15.119:0/4179400817 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87a806b500 0x7f87a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.360+0000 7f87c5400640 1 --2- 172.21.15.119:0/4179400817 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87a806b500 0x7f87a806d9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f87b4006e80 tx=0x7f87b4006dd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.365+0000 7f87be000640 1 -- 172.21.15.119:0/4179400817 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87b80799f0 con 0x7f87c806aba0 2024-04-25T10:14:48.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.546+0000 7f87c7200640 1 -- 172.21.15.119:0/4179400817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87c80cf1b0 con 0x7f87c806aba0 2024-04-25T10:14:48.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.547+0000 7f87be000640 1 -- 172.21.15.119:0/4179400817 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f87b80792f0 con 0x7f87c806aba0 2024-04-25T10:14:48.550 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:14:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.549+0000 7f87c7200640 1 -- 172.21.15.119:0/4179400817 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87a806b500 msgr2=0x7f87a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.549+0000 7f87c7200640 1 --2- 172.21.15.119:0/4179400817 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87a806b500 0x7f87a806d9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f87b4006e80 tx=0x7f87b4006dd0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.549+0000 7f87c7200640 1 -- 172.21.15.119:0/4179400817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87c806aba0 msgr2=0x7f87c81f13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.549+0000 7f87c7200640 1 --2- 172.21.15.119:0/4179400817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87c806aba0 0x7f87c81f13b0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009870 tx=0x7f87b80046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.549+0000 7f87c7200640 1 -- 172.21.15.119:0/4179400817 shutdown_connections 2024-04-25T10:14:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.549+0000 7f87c7200640 1 --2- 172.21.15.119:0/4179400817 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f87a806b500 0x7f87a806d9c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.549+0000 7f87c7200640 1 --2- 172.21.15.119:0/4179400817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87c80cec60 0x7f87c8152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.549+0000 7f87c7200640 1 --2- 172.21.15.119:0/4179400817 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87c806f040 0x7f87c81f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.549+0000 7f87c7200640 1 --2- 172.21.15.119:0/4179400817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87c806aba0 0x7f87c81f13b0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.549+0000 7f87c7200640 1 -- 172.21.15.119:0/4179400817 >> 172.21.15.119:0/4179400817 conn(0x7f87c8066ff0 msgr2=0x7f87c80cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.549+0000 7f87c7200640 1 -- 172.21.15.119:0/4179400817 shutdown_connections 2024-04-25T10:14:48.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.550+0000 7f87c7200640 1 -- 172.21.15.119:0/4179400817 wait complete. 2024-04-25T10:14:48.571 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:48.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.762+0000 7ff8d6a00640 1 Processor -- start 2024-04-25T10:14:48.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.762+0000 7ff8d6a00640 1 -- start start 2024-04-25T10:14:48.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.764+0000 7ff8d6a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff8d0144e70 0x7ff8d006d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.764+0000 7ff8d6a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff8d006d7a0 0x7ff8d00ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.765+0000 7ff8d6a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d006dcb0 con 0x7ff8d00cef10 2024-04-25T10:14:48.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.765+0000 7ff8d6a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d015bfe0 con 0x7ff8d0144e70 2024-04-25T10:14:48.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.765+0000 7ff8d6a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d006df90 con 0x7ff8d006d7a0 2024-04-25T10:14:48.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.765+0000 7ff8cfe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff8d0144e70 0x7ff8d006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.765+0000 7ff8d4a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff8d00cef10 0x7ff8d0071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59476/0 (socket says 172.21.15.119:59476) 2024-04-25T10:14:48.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.765+0000 7ff8d4a00640 1 -- 172.21.15.119:0/2712816247 learned_addr learned my addr 172.21.15.119:0/2712816247 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:14:48.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.765+0000 7ff8cf400640 1 --2- 172.21.15.119:0/2712816247 >> v2:172.21.15.132:3300/0 conn(0x7ff8d006d7a0 0x7ff8d00ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.765+0000 7ff8cfe00640 1 -- 172.21.15.119:0/2712816247 >> v2:172.21.15.132:3300/0 conn(0x7ff8d006d7a0 msgr2=0x7ff8d00ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.765+0000 7ff8cfe00640 1 --2- 172.21.15.119:0/2712816247 >> v2:172.21.15.132:3300/0 conn(0x7ff8d006d7a0 0x7ff8d00ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.765+0000 7ff8cfe00640 1 -- 172.21.15.119:0/2712816247 >> v1:172.21.15.119:6789/0 conn(0x7ff8d00cef10 legacy=0x7ff8d0071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.765+0000 7ff8cea00640 1 -- 172.21.15.119:0/2712816247 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3387100191 0 0) 0x7ff8d006dcb0 con 0x7ff8d00cef10 2024-04-25T10:14:48.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.765+0000 7ff8cfe00640 1 -- 172.21.15.119:0/2712816247 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d006e270 con 0x7ff8d0144e70 2024-04-25T10:14:48.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.766+0000 7ff8cfe00640 1 --2- 172.21.15.119:0/2712816247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8d0144e70 0x7ff8d006d260 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0072560 tx=0x7ff8c002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f158457ff07772 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.767+0000 7ff8cea00640 1 -- 172.21.15.119:0/2712816247 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0037070 con 0x7ff8d0144e70 2024-04-25T10:14:48.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.767+0000 7ff8cea00640 1 -- 172.21.15.119:0/2712816247 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c0032440 con 0x7ff8d0144e70 2024-04-25T10:14:48.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.767+0000 7ff8cea00640 1 -- 172.21.15.119:0/2712816247 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c002ae60 con 0x7ff8d0144e70 2024-04-25T10:14:48.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.767+0000 7ff8d6a00640 1 -- 172.21.15.119:0/2712816247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8d0144e70 msgr2=0x7ff8d006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.767+0000 7ff8d6a00640 1 --2- 172.21.15.119:0/2712816247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8d0144e70 0x7ff8d006d260 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0072560 tx=0x7ff8c002a110 comp rx=0 tx=0).stop 2024-04-25T10:14:48.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.767+0000 7ff8d6a00640 1 -- 172.21.15.119:0/2712816247 shutdown_connections 2024-04-25T10:14:48.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.767+0000 7ff8d6a00640 1 --2- 172.21.15.119:0/2712816247 >> v2:172.21.15.132:3300/0 conn(0x7ff8d006d7a0 0x7ff8d00ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.767+0000 7ff8d6a00640 1 --2- 172.21.15.119:0/2712816247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8d0144e70 0x7ff8d006d260 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.767+0000 7ff8d6a00640 1 -- 172.21.15.119:0/2712816247 >> 172.21.15.119:0/2712816247 conn(0x7ff8d006aba0 msgr2=0x7ff8d006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.768+0000 7ff8d6a00640 1 -- 172.21.15.119:0/2712816247 shutdown_connections 2024-04-25T10:14:48.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.768+0000 7ff8d6a00640 1 -- 172.21.15.119:0/2712816247 wait complete. 2024-04-25T10:14:48.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.769+0000 7ff8d6a00640 1 Processor -- start 2024-04-25T10:14:48.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.769+0000 7ff8d6a00640 1 -- start start 2024-04-25T10:14:48.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.770+0000 7ff8d6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8d006d7a0 0x7ff8d01f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.770+0000 7ff8d6a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8d00cef10 0x7ff8d00b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.770+0000 7ff8d6a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d0144e70 0x7ff8d01f0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.770+0000 7ff8d6a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d01454b0 con 0x7ff8d006d7a0 2024-04-25T10:14:48.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.770+0000 7ff8d6a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d00cf370 con 0x7ff8d0144e70 2024-04-25T10:14:48.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.770+0000 7ff8d6a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0145630 con 0x7ff8d00cef10 2024-04-25T10:14:48.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.771+0000 7ff8cfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8d006d7a0 0x7ff8d01f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.771+0000 7ff8cfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8d006d7a0 0x7ff8d01f4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43184/0 (socket says 172.21.15.119:43184) 2024-04-25T10:14:48.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.771+0000 7ff8cfe00640 1 -- 172.21.15.119:0/4288016109 learned_addr learned my addr 172.21.15.119:0/4288016109 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:48.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.771+0000 7ff8cf400640 1 --2- 172.21.15.119:0/4288016109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8d00cef10 0x7ff8d00b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.771+0000 7ff8d4a00640 1 --2- 172.21.15.119:0/4288016109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d0144e70 0x7ff8d01f0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.771+0000 7ff8cfe00640 1 -- 172.21.15.119:0/4288016109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8d00cef10 msgr2=0x7ff8d00b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.771+0000 7ff8cfe00640 1 --2- 172.21.15.119:0/4288016109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8d00cef10 0x7ff8d00b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.771+0000 7ff8cfe00640 1 -- 172.21.15.119:0/4288016109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d0144e70 msgr2=0x7ff8d01f0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.771+0000 7ff8cfe00640 1 --2- 172.21.15.119:0/4288016109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d0144e70 0x7ff8d01f0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.771+0000 7ff8cfe00640 1 -- 172.21.15.119:0/4288016109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d00b3800 con 0x7ff8d006d7a0 2024-04-25T10:14:48.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.772+0000 7ff8cfe00640 1 --2- 172.21.15.119:0/4288016109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8d006d7a0 0x7ff8d01f4040 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0000c00 tx=0x7ff8c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.772+0000 7ff8c7e00640 1 -- 172.21.15.119:0/4288016109 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0037040 con 0x7ff8d006d7a0 2024-04-25T10:14:48.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.772+0000 7ff8c7e00640 1 -- 172.21.15.119:0/4288016109 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c0042070 con 0x7ff8d006d7a0 2024-04-25T10:14:48.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.772+0000 7ff8d6a00640 1 -- 172.21.15.119:0/4288016109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d00b2c90 con 0x7ff8d006d7a0 2024-04-25T10:14:48.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.772+0000 7ff8c7e00640 1 -- 172.21.15.119:0/4288016109 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0030a50 con 0x7ff8d006d7a0 2024-04-25T10:14:48.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.772+0000 7ff8d6a00640 1 -- 172.21.15.119:0/4288016109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d00b3240 con 0x7ff8d006d7a0 2024-04-25T10:14:48.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.773+0000 7ff8d6a00640 1 -- 172.21.15.119:0/4288016109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d00750b0 con 0x7ff8d006d7a0 2024-04-25T10:14:48.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.774+0000 7ff8c7e00640 1 -- 172.21.15.119:0/4288016109 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8c0030bf0 con 0x7ff8d006d7a0 2024-04-25T10:14:48.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.775+0000 7ff8c7e00640 1 --2- 172.21.15.119:0/4288016109 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff8b006b500 0x7ff8b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.775+0000 7ff8c7e00640 1 -- 172.21.15.119:0/4288016109 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7ff8c00aace0 con 0x7ff8d006d7a0 2024-04-25T10:14:48.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.775+0000 7ff8cf400640 1 --2- 172.21.15.119:0/4288016109 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff8b006b500 0x7ff8b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.776+0000 7ff8cf400640 1 --2- 172.21.15.119:0/4288016109 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff8b006b500 0x7ff8b006d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc006fd0 tx=0x7ff8bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.779+0000 7ff8c7e00640 1 -- 172.21.15.119:0/4288016109 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff8c0045360 con 0x7ff8d006d7a0 2024-04-25T10:14:48.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.973+0000 7ff8d6a00640 1 -- 172.21.15.119:0/4288016109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff8d00cff80 con 0x7ff8d006d7a0 2024-04-25T10:14:48.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.974+0000 7ff8c7e00640 1 -- 172.21.15.119:0/4288016109 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff8c0079af0 con 0x7ff8d006d7a0 2024-04-25T10:14:48.978 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:14:48.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.977+0000 7ff8d6a00640 1 -- 172.21.15.119:0/4288016109 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff8b006b500 msgr2=0x7ff8b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.977+0000 7ff8d6a00640 1 --2- 172.21.15.119:0/4288016109 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff8b006b500 0x7ff8b006d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc006fd0 tx=0x7ff8bc008040 comp rx=0 tx=0).stop 2024-04-25T10:14:48.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.977+0000 7ff8d6a00640 1 -- 172.21.15.119:0/4288016109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8d006d7a0 msgr2=0x7ff8d01f4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.977+0000 7ff8d6a00640 1 --2- 172.21.15.119:0/4288016109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8d006d7a0 0x7ff8d01f4040 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0000c00 tx=0x7ff8c00046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.977+0000 7ff8d6a00640 1 -- 172.21.15.119:0/4288016109 shutdown_connections 2024-04-25T10:14:48.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.977+0000 7ff8d6a00640 1 --2- 172.21.15.119:0/4288016109 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff8b006b500 0x7ff8b006d9c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.977+0000 7ff8d6a00640 1 --2- 172.21.15.119:0/4288016109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d0144e70 0x7ff8d01f0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.977+0000 7ff8d6a00640 1 --2- 172.21.15.119:0/4288016109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff8d00cef10 0x7ff8d00b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.977+0000 7ff8d6a00640 1 --2- 172.21.15.119:0/4288016109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8d006d7a0 0x7ff8d01f4040 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.977+0000 7ff8d6a00640 1 -- 172.21.15.119:0/4288016109 >> 172.21.15.119:0/4288016109 conn(0x7ff8d006aba0 msgr2=0x7ff8d006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.977+0000 7ff8d6a00640 1 -- 172.21.15.119:0/4288016109 shutdown_connections 2024-04-25T10:14:48.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:48.978+0000 7ff8d6a00640 1 -- 172.21.15.119:0/4288016109 wait complete. 2024-04-25T10:14:49.000 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:14:49.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.197+0000 7f6b42a00640 1 Processor -- start 2024-04-25T10:14:49.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.198+0000 7f6b42a00640 1 -- start start 2024-04-25T10:14:49.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.199+0000 7f6b42a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6b3c15b7e0 0x7f6b3c15e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.199+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0ce860 0x7f6b3c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.199+0000 7f6b42a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6b3c0d00a0 0x7f6b3c0d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.199+0000 7f6b42a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3c0d0a70 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.199+0000 7f6b42a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3c0b5710 con 0x7f6b3c0d00a0 2024-04-25T10:14:49.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.199+0000 7f6b42a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c0b59f0 con 0x7f6b3c0ce860 2024-04-25T10:14:49.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.199+0000 7f6b41600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6b3c15b7e0 0x7f6b3c15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.199+0000 7f6b41600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6b3c15b7e0 0x7f6b3c15e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43202/0 (socket says 172.21.15.119:43202) 2024-04-25T10:14:49.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.199+0000 7f6b41600640 1 -- 172.21.15.119:0/1502857385 learned_addr learned my addr 172.21.15.119:0/1502857385 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:49.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.200+0000 7f6b42000640 1 --2- 172.21.15.119:0/1502857385 >> v2:172.21.15.132:3300/0 conn(0x7f6b3c0d00a0 0x7f6b3c0d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.200+0000 7f6b40c00640 1 --2- 172.21.15.119:0/1502857385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0ce860 0x7f6b3c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.200+0000 7f6b41600640 1 -- 172.21.15.119:0/1502857385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0ce860 msgr2=0x7f6b3c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.200+0000 7f6b41600640 1 --2- 172.21.15.119:0/1502857385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0ce860 0x7f6b3c0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.200+0000 7f6b41600640 1 -- 172.21.15.119:0/1502857385 >> v2:172.21.15.132:3300/0 conn(0x7f6b3c0d00a0 msgr2=0x7f6b3c0d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.200+0000 7f6b41600640 1 --2- 172.21.15.119:0/1502857385 >> v2:172.21.15.132:3300/0 conn(0x7f6b3c0d00a0 0x7f6b3c0d0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.200+0000 7f6b41600640 1 -- 172.21.15.119:0/1502857385 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c0b5cd0 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.201+0000 7f6b41600640 1 --2- 172.21.15.119:0/1502857385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b3c15b7e0 0x7f6b3c15e000 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c08b9c0 tx=0x7f6b2c02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17792374462936e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.201+0000 7f6b3be00640 1 -- 172.21.15.119:0/1502857385 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c036070 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.201+0000 7f6b3be00640 1 -- 172.21.15.119:0/1502857385 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2c031600 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.201+0000 7f6b3be00640 1 -- 172.21.15.119:0/1502857385 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0319d0 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.202+0000 7f6b42a00640 1 -- 172.21.15.119:0/1502857385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b3c15b7e0 msgr2=0x7f6b3c15e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.202+0000 7f6b42a00640 1 --2- 172.21.15.119:0/1502857385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b3c15b7e0 0x7f6b3c15e000 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c08b9c0 tx=0x7f6b2c02a570 comp rx=0 tx=0).stop 2024-04-25T10:14:49.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.202+0000 7f6b42a00640 1 -- 172.21.15.119:0/1502857385 shutdown_connections 2024-04-25T10:14:49.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.202+0000 7f6b42a00640 1 --2- 172.21.15.119:0/1502857385 >> v2:172.21.15.132:3300/0 conn(0x7f6b3c0d00a0 0x7f6b3c0d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.202+0000 7f6b42a00640 1 --2- 172.21.15.119:0/1502857385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0ce860 0x7f6b3c0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.202+0000 7f6b42a00640 1 --2- 172.21.15.119:0/1502857385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b3c15b7e0 0x7f6b3c15e000 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.202+0000 7f6b42a00640 1 -- 172.21.15.119:0/1502857385 >> 172.21.15.119:0/1502857385 conn(0x7f6b3c0722d0 msgr2=0x7f6b3c0726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.202+0000 7f6b42a00640 1 -- 172.21.15.119:0/1502857385 shutdown_connections 2024-04-25T10:14:49.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.202+0000 7f6b42a00640 1 -- 172.21.15.119:0/1502857385 wait complete. 2024-04-25T10:14:49.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.204+0000 7f6b42a00640 1 Processor -- start 2024-04-25T10:14:49.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.204+0000 7f6b42a00640 1 -- start start 2024-04-25T10:14:49.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.205+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b3c0ce860 0x7f6b3c0b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.205+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0d00a0 0x7f6b3c0b86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.205+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b3c15b7e0 0x7f6b3c1501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.205+0000 7f6b42a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c0b53c0 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.205+0000 7f6b42a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c15e540 con 0x7f6b3c0ce860 2024-04-25T10:14:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.205+0000 7f6b42a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c161960 con 0x7f6b3c0d00a0 2024-04-25T10:14:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.205+0000 7f6b42000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b3c15b7e0 0x7f6b3c1501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.205+0000 7f6b42000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b3c15b7e0 0x7f6b3c1501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43206/0 (socket says 172.21.15.119:43206) 2024-04-25T10:14:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.205+0000 7f6b42000640 1 -- 172.21.15.119:0/2206830621 learned_addr learned my addr 172.21.15.119:0/2206830621 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.206+0000 7f6b42000640 1 -- 172.21.15.119:0/2206830621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0d00a0 msgr2=0x7f6b3c0b86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.206+0000 7f6b42000640 1 --2- 172.21.15.119:0/2206830621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0d00a0 0x7f6b3c0b86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.206+0000 7f6b42000640 1 -- 172.21.15.119:0/2206830621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b3c0ce860 msgr2=0x7f6b3c0b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.206+0000 7f6b41600640 1 --2- 172.21.15.119:0/2206830621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b3c0ce860 0x7f6b3c0b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.206+0000 7f6b42000640 1 --2- 172.21.15.119:0/2206830621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b3c0ce860 0x7f6b3c0b6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.206+0000 7f6b42000640 1 -- 172.21.15.119:0/2206830621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c1506e0 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.206+0000 7f6b42000640 1 --2- 172.21.15.119:0/2206830621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b3c15b7e0 0x7f6b3c1501a0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400d830 tx=0x7f6b3400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.206+0000 7f6b39600640 1 -- 172.21.15.119:0/2206830621 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3400b6c0 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.206+0000 7f6b39600640 1 -- 172.21.15.119:0/2206830621 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b34019070 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.206+0000 7f6b42a00640 1 -- 172.21.15.119:0/2206830621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3c0b71c0 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.206+0000 7f6b39600640 1 -- 172.21.15.119:0/2206830621 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3400bac0 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.206+0000 7f6b42a00640 1 -- 172.21.15.119:0/2206830621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3c0b7780 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.208+0000 7f6b42a00640 1 -- 172.21.15.119:0/2206830621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3c0749b0 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.208+0000 7f6b39600640 1 -- 172.21.15.119:0/2206830621 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b3401e460 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.209+0000 7f6b39600640 1 --2- 172.21.15.119:0/2206830621 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b1c06b500 0x7f6b1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.209+0000 7f6b39600640 1 -- 172.21.15.119:0/2206830621 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f6b3408a940 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.210+0000 7f6b41600640 1 --2- 172.21.15.119:0/2206830621 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b1c06b500 0x7f6b1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.210+0000 7f6b41600640 1 --2- 172.21.15.119:0/2206830621 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b1c06b500 0x7f6b1c06d9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c033000 tx=0x7f6b2c009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.214+0000 7f6b39600640 1 -- 172.21.15.119:0/2206830621 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b340596d0 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.420+0000 7f6b42a00640 1 -- 172.21.15.119:0/2206830621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6b3c0d47e0 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.421+0000 7f6b39600640 1 -- 172.21.15.119:0/2206830621 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v84) v1 ==== 80+0+124 (secure 0 0 0) 0x7f6b34058fd0 con 0x7f6b3c15b7e0 2024-04-25T10:14:49.425 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6837/3997964942 2024-04-26T10:13:36.160373+0000 2024-04-25T10:14:49.425 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6835/3997964942 2024-04-26T10:13:36.160373+0000 2024-04-25T10:14:49.426 INFO:teuthology.orchestra.run.smithi119.stderr:listed 2 entries 2024-04-25T10:14:49.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.424+0000 7f6b42a00640 1 -- 172.21.15.119:0/2206830621 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b1c06b500 msgr2=0x7f6b1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.424+0000 7f6b42a00640 1 --2- 172.21.15.119:0/2206830621 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b1c06b500 0x7f6b1c06d9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c033000 tx=0x7f6b2c009000 comp rx=0 tx=0).stop 2024-04-25T10:14:49.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.424+0000 7f6b42a00640 1 -- 172.21.15.119:0/2206830621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b3c15b7e0 msgr2=0x7f6b3c1501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.424+0000 7f6b42a00640 1 --2- 172.21.15.119:0/2206830621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b3c15b7e0 0x7f6b3c1501a0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400d830 tx=0x7f6b3400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.424+0000 7f6b42a00640 1 -- 172.21.15.119:0/2206830621 shutdown_connections 2024-04-25T10:14:49.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.424+0000 7f6b42a00640 1 --2- 172.21.15.119:0/2206830621 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b1c06b500 0x7f6b1c06d9c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.424+0000 7f6b42a00640 1 --2- 172.21.15.119:0/2206830621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b3c15b7e0 0x7f6b3c1501a0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.424+0000 7f6b42a00640 1 --2- 172.21.15.119:0/2206830621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b3c0d00a0 0x7f6b3c0b86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.424+0000 7f6b42a00640 1 --2- 172.21.15.119:0/2206830621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b3c0ce860 0x7f6b3c0b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.424+0000 7f6b42a00640 1 -- 172.21.15.119:0/2206830621 >> 172.21.15.119:0/2206830621 conn(0x7f6b3c0722d0 msgr2=0x7f6b3c0d2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.424+0000 7f6b42a00640 1 -- 172.21.15.119:0/2206830621 shutdown_connections 2024-04-25T10:14:49.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.424+0000 7f6b42a00640 1 -- 172.21.15.119:0/2206830621 wait complete. 2024-04-25T10:14:49.447 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:14:49.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:49.448 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:49.536 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:14:49.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:49.537 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:49.587 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:49.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.773+0000 7f7a69a00640 1 Processor -- start 2024-04-25T10:14:49.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.773+0000 7f7a69a00640 1 -- start start 2024-04-25T10:14:49.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.774+0000 7f7a69a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7a64147270 0x7f7a640b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.774+0000 7f7a69a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7a6415dee0 0x7f7a6415e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.774+0000 7f7a69a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a640af000 con 0x7f7a6415e890 2024-04-25T10:14:49.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.774+0000 7f7a69a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a640b4a80 con 0x7f7a64147270 2024-04-25T10:14:49.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.774+0000 7f7a69a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a640b4d60 con 0x7f7a6415dee0 2024-04-25T10:14:49.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.775+0000 7f7a63e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7a64147270 0x7f7a640b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.775+0000 7f7a63e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7a64147270 0x7f7a640b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43208/0 (socket says 172.21.15.119:43208) 2024-04-25T10:14:49.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.775+0000 7f7a63e00640 1 -- 172.21.15.119:0/1978706224 learned_addr learned my addr 172.21.15.119:0/1978706224 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:49.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.775+0000 7f7a5b400640 1 --2- 172.21.15.119:0/1978706224 >> v2:172.21.15.132:3300/0 conn(0x7f7a6415dee0 0x7f7a6415e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.775+0000 7f7a63e00640 1 -- 172.21.15.119:0/1978706224 >> v2:172.21.15.132:3300/0 conn(0x7f7a6415dee0 msgr2=0x7f7a6415e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.775+0000 7f7a63e00640 1 --2- 172.21.15.119:0/1978706224 >> v2:172.21.15.132:3300/0 conn(0x7f7a6415dee0 0x7f7a6415e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.775+0000 7f7a63e00640 1 -- 172.21.15.119:0/1978706224 >> v1:172.21.15.132:6789/0 conn(0x7f7a6415e890 legacy=0x7f7a641614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.775+0000 7f7a63e00640 1 -- 172.21.15.119:0/1978706224 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a640b5040 con 0x7f7a64147270 2024-04-25T10:14:49.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.776+0000 7f7a63e00640 1 --2- 172.21.15.119:0/1978706224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a64147270 0x7f7a640b4410 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f7a6406f500 tx=0x7f7a5402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd362c7eaaaabe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.777+0000 7f7a63400640 1 -- 172.21.15.119:0/1978706224 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54037070 con 0x7f7a64147270 2024-04-25T10:14:49.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.777+0000 7f7a63400640 1 -- 172.21.15.119:0/1978706224 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a54032440 con 0x7f7a64147270 2024-04-25T10:14:49.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.777+0000 7f7a63400640 1 -- 172.21.15.119:0/1978706224 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54032810 con 0x7f7a64147270 2024-04-25T10:14:49.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.777+0000 7f7a69a00640 1 -- 172.21.15.119:0/1978706224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a64147270 msgr2=0x7f7a640b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.777+0000 7f7a69a00640 1 --2- 172.21.15.119:0/1978706224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a64147270 0x7f7a640b4410 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f7a6406f500 tx=0x7f7a5402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.777+0000 7f7a69a00640 1 -- 172.21.15.119:0/1978706224 shutdown_connections 2024-04-25T10:14:49.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.777+0000 7f7a69a00640 1 --2- 172.21.15.119:0/1978706224 >> v2:172.21.15.132:3300/0 conn(0x7f7a6415dee0 0x7f7a6415e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.777+0000 7f7a69a00640 1 --2- 172.21.15.119:0/1978706224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a64147270 0x7f7a640b4410 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.778+0000 7f7a69a00640 1 -- 172.21.15.119:0/1978706224 >> 172.21.15.119:0/1978706224 conn(0x7f7a6406b1d0 msgr2=0x7f7a640b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.778+0000 7f7a69a00640 1 -- 172.21.15.119:0/1978706224 shutdown_connections 2024-04-25T10:14:49.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.778+0000 7f7a69a00640 1 -- 172.21.15.119:0/1978706224 wait complete. 2024-04-25T10:14:49.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.779+0000 7f7a69a00640 1 Processor -- start 2024-04-25T10:14:49.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.780+0000 7f7a69a00640 1 -- start start 2024-04-25T10:14:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.780+0000 7f7a69a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a64147270 0x7f7a641e4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.780+0000 7f7a69a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a6415dee0 0x7f7a641e4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.780+0000 7f7a69a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a6415e890 0x7f7a6413ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.780+0000 7f7a69a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6415ece0 con 0x7f7a6415e890 2024-04-25T10:14:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.780+0000 7f7a69a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a640b5600 con 0x7f7a6415dee0 2024-04-25T10:14:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.780+0000 7f7a69a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a640adf90 con 0x7f7a64147270 2024-04-25T10:14:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.781+0000 7f7a69000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a6415e890 0x7f7a6413ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.781+0000 7f7a69000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a6415e890 0x7f7a6413ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43212/0 (socket says 172.21.15.119:43212) 2024-04-25T10:14:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.781+0000 7f7a69000640 1 -- 172.21.15.119:0/2917967211 learned_addr learned my addr 172.21.15.119:0/2917967211 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.781+0000 7f7a5b400640 1 --2- 172.21.15.119:0/2917967211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a6415dee0 0x7f7a641e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.781+0000 7f7a63e00640 1 --2- 172.21.15.119:0/2917967211 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a64147270 0x7f7a641e4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.781+0000 7f7a69000640 1 -- 172.21.15.119:0/2917967211 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a64147270 msgr2=0x7f7a641e4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.781+0000 7f7a69000640 1 --2- 172.21.15.119:0/2917967211 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a64147270 0x7f7a641e4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.781+0000 7f7a69000640 1 -- 172.21.15.119:0/2917967211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a6415dee0 msgr2=0x7f7a641e4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.781+0000 7f7a69000640 1 --2- 172.21.15.119:0/2917967211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a6415dee0 0x7f7a641e4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.781+0000 7f7a69000640 1 -- 172.21.15.119:0/2917967211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a641e5130 con 0x7f7a6415e890 2024-04-25T10:14:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.782+0000 7f7a69000640 1 --2- 172.21.15.119:0/2917967211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a6415e890 0x7f7a6413ad90 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00b970 tx=0x7f7a5c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.782+0000 7f7a60c00640 1 -- 172.21.15.119:0/2917967211 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c019070 con 0x7f7a6415e890 2024-04-25T10:14:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.782+0000 7f7a60c00640 1 -- 172.21.15.119:0/2917967211 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c00cb40 con 0x7f7a6415e890 2024-04-25T10:14:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.782+0000 7f7a60c00640 1 -- 172.21.15.119:0/2917967211 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c014440 con 0x7f7a6415e890 2024-04-25T10:14:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.782+0000 7f7a69a00640 1 -- 172.21.15.119:0/2917967211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a64145350 con 0x7f7a6415e890 2024-04-25T10:14:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.782+0000 7f7a69a00640 1 -- 172.21.15.119:0/2917967211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a64145910 con 0x7f7a6415e890 2024-04-25T10:14:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.783+0000 7f7a69a00640 1 -- 172.21.15.119:0/2917967211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a640b1ec0 con 0x7f7a6415e890 2024-04-25T10:14:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.784+0000 7f7a60c00640 1 -- 172.21.15.119:0/2917967211 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a5c01c070 con 0x7f7a6415e890 2024-04-25T10:14:49.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.785+0000 7f7a60c00640 1 --2- 172.21.15.119:0/2917967211 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a4006b4d0 0x7f7a4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.785+0000 7f7a60c00640 1 -- 172.21.15.119:0/2917967211 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f7a5c089f10 con 0x7f7a6415e890 2024-04-25T10:14:49.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.785+0000 7f7a63e00640 1 --2- 172.21.15.119:0/2917967211 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a4006b4d0 0x7f7a4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.786+0000 7f7a63e00640 1 --2- 172.21.15.119:0/2917967211 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a4006b4d0 0x7f7a4006d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7a54034000 tx=0x7f7a5402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:49.789+0000 7f7a60c00640 1 -- 172.21.15.119:0/2917967211 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a5c058ca0 con 0x7f7a6415e890 2024-04-25T10:14:50.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.021+0000 7f7a69a00640 1 -- 172.21.15.119:0/2917967211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a640749b0 con 0x7f7a6415e890 2024-04-25T10:14:50.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.022+0000 7f7a60c00640 1 -- 172.21.15.119:0/2917967211 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f7a5c0585a0 con 0x7f7a6415e890 2024-04-25T10:14:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:14:50.027 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:13:47.200726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6837/1734507541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1734507541},{"type":"v1","addr":"172.21.15.119:6837","nonce":1734507541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:14:50.027 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 68 2024-04-25T10:14:50.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.025+0000 7f7a69a00640 1 -- 172.21.15.119:0/2917967211 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a4006b4d0 msgr2=0x7f7a4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.025+0000 7f7a69a00640 1 --2- 172.21.15.119:0/2917967211 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a4006b4d0 0x7f7a4006d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7a54034000 tx=0x7f7a5402e000 comp rx=0 tx=0).stop 2024-04-25T10:14:50.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.025+0000 7f7a69a00640 1 -- 172.21.15.119:0/2917967211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a6415e890 msgr2=0x7f7a6413ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.025+0000 7f7a69a00640 1 --2- 172.21.15.119:0/2917967211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a6415e890 0x7f7a6413ad90 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00b970 tx=0x7f7a5c00be30 comp rx=0 tx=0).stop 2024-04-25T10:14:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.025+0000 7f7a69a00640 1 -- 172.21.15.119:0/2917967211 shutdown_connections 2024-04-25T10:14:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.025+0000 7f7a69a00640 1 --2- 172.21.15.119:0/2917967211 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7a4006b4d0 0x7f7a4006d990 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.025+0000 7f7a69a00640 1 --2- 172.21.15.119:0/2917967211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a6415e890 0x7f7a6413ad90 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.025+0000 7f7a69a00640 1 --2- 172.21.15.119:0/2917967211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a6415dee0 0x7f7a641e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.025+0000 7f7a69a00640 1 --2- 172.21.15.119:0/2917967211 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a64147270 0x7f7a641e4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.025+0000 7f7a69a00640 1 -- 172.21.15.119:0/2917967211 >> 172.21.15.119:0/2917967211 conn(0x7f7a6406b1d0 msgr2=0x7f7a640ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.025+0000 7f7a69a00640 1 -- 172.21.15.119:0/2917967211 shutdown_connections 2024-04-25T10:14:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.025+0000 7f7a69a00640 1 -- 172.21.15.119:0/2917967211 wait complete. 2024-04-25T10:14:50.049 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:14:50.280 INFO:teuthology.orchestra.run.smithi119.stdout:{"message":"","return_code":0} 2024-04-25T10:14:50.280 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:14:50.280 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:14:50.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.470+0000 7f5068e00640 1 Processor -- start 2024-04-25T10:14:50.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.470+0000 7f5068e00640 1 -- start start 2024-04-25T10:14:50.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.471+0000 7f5068e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f506415ae20 0x7f50640b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.471+0000 7f5068e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f50640b4730 0x7f506415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.472+0000 7f5068e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f506406da40 con 0x7f50640aedc0 2024-04-25T10:14:50.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.472+0000 7f5068e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f506415c050 con 0x7f50640b4730 2024-04-25T10:14:50.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.472+0000 7f5068e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50640b4b70 con 0x7f506415ae20 2024-04-25T10:14:50.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.472+0000 7f5061e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f50640aedc0 0x7f506406d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59492/0 (socket says 172.21.15.119:59492) 2024-04-25T10:14:50.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.472+0000 7f5061e00640 1 -- 172.21.15.119:0/2781983383 learned_addr learned my addr 172.21.15.119:0/2781983383 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:14:50.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.472+0000 7f5062800640 1 --2- 172.21.15.119:0/2781983383 >> v2:172.21.15.119:3300/0 conn(0x7f50640b4730 0x7f506415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.472+0000 7f5059400640 1 --2- 172.21.15.119:0/2781983383 >> v2:172.21.15.132:3300/0 conn(0x7f506415ae20 0x7f50640b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.473+0000 7f5062800640 1 -- 172.21.15.119:0/2781983383 >> v2:172.21.15.132:3300/0 conn(0x7f506415ae20 msgr2=0x7f50640b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.473+0000 7f5062800640 1 --2- 172.21.15.119:0/2781983383 >> v2:172.21.15.132:3300/0 conn(0x7f506415ae20 0x7f50640b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.473+0000 7f5062800640 1 -- 172.21.15.119:0/2781983383 >> v1:172.21.15.119:6789/0 conn(0x7f50640aedc0 legacy=0x7f506406d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.473+0000 7f5062800640 1 -- 172.21.15.119:0/2781983383 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506406dd20 con 0x7f50640b4730 2024-04-25T10:14:50.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.474+0000 7f5062800640 1 --2- 172.21.15.119:0/2781983383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50640b4730 0x7f506415c7c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f50640d0500 tx=0x7f505c02c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777c3cb91cfdfb48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.475+0000 7f5061400640 1 -- 172.21.15.119:0/2781983383 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c007740 con 0x7f50640b4730 2024-04-25T10:14:50.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.475+0000 7f5061400640 1 -- 172.21.15.119:0/2781983383 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c0078e0 con 0x7f50640b4730 2024-04-25T10:14:50.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.475+0000 7f5061400640 1 -- 172.21.15.119:0/2781983383 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c007cb0 con 0x7f50640b4730 2024-04-25T10:14:50.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.476+0000 7f5068e00640 1 -- 172.21.15.119:0/2781983383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50640b4730 msgr2=0x7f506415c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.476+0000 7f5068e00640 1 --2- 172.21.15.119:0/2781983383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50640b4730 0x7f506415c7c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f50640d0500 tx=0x7f505c02c800 comp rx=0 tx=0).stop 2024-04-25T10:14:50.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.476+0000 7f5068e00640 1 -- 172.21.15.119:0/2781983383 shutdown_connections 2024-04-25T10:14:50.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.476+0000 7f5068e00640 1 --2- 172.21.15.119:0/2781983383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50640b4730 0x7f506415c7c0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.476+0000 7f5068e00640 1 --2- 172.21.15.119:0/2781983383 >> v2:172.21.15.132:3300/0 conn(0x7f506415ae20 0x7f50640b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.476+0000 7f5068e00640 1 -- 172.21.15.119:0/2781983383 >> 172.21.15.119:0/2781983383 conn(0x7f506406aba0 msgr2=0x7f506406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.476+0000 7f5068e00640 1 -- 172.21.15.119:0/2781983383 shutdown_connections 2024-04-25T10:14:50.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.476+0000 7f5068e00640 1 -- 172.21.15.119:0/2781983383 wait complete. 2024-04-25T10:14:50.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.478+0000 7f5068e00640 1 Processor -- start 2024-04-25T10:14:50.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.478+0000 7f5068e00640 1 -- start start 2024-04-25T10:14:50.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.479+0000 7f5068e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50640aedc0 0x7f50641f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.479+0000 7f5068e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50640b4730 0x7f50640ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.479+0000 7f5068e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f506415ae20 0x7f5064147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.479+0000 7f5068e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506415c2a0 con 0x7f50640aedc0 2024-04-25T10:14:50.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.479+0000 7f5068e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50640af1a0 con 0x7f50640b4730 2024-04-25T10:14:50.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.479+0000 7f5068e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506415cd00 con 0x7f506415ae20 2024-04-25T10:14:50.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.479+0000 7f5061e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50640aedc0 0x7f50641f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.480+0000 7f5061e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50640aedc0 0x7f50641f48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43232/0 (socket says 172.21.15.119:43232) 2024-04-25T10:14:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.480+0000 7f5061e00640 1 -- 172.21.15.119:0/399705889 learned_addr learned my addr 172.21.15.119:0/399705889 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.480+0000 7f5059400640 1 --2- 172.21.15.119:0/399705889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50640b4730 0x7f50640ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.480+0000 7f5062800640 1 --2- 172.21.15.119:0/399705889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f506415ae20 0x7f5064147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.480+0000 7f5061e00640 1 -- 172.21.15.119:0/399705889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f506415ae20 msgr2=0x7f5064147df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.480+0000 7f5061e00640 1 --2- 172.21.15.119:0/399705889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f506415ae20 0x7f5064147df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.480+0000 7f5061e00640 1 -- 172.21.15.119:0/399705889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50640b4730 msgr2=0x7f50640ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.480+0000 7f5061e00640 1 --2- 172.21.15.119:0/399705889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50640b4730 0x7f50640ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.480+0000 7f5061e00640 1 -- 172.21.15.119:0/399705889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50641e3100 con 0x7f50640aedc0 2024-04-25T10:14:50.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.480+0000 7f5061e00640 1 --2- 172.21.15.119:0/399705889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50640aedc0 0x7f50641f48b0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f505400c400 tx=0x7f505400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.482+0000 7f505aa00640 1 -- 172.21.15.119:0/399705889 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054017070 con 0x7f50640aedc0 2024-04-25T10:14:50.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.482+0000 7f505aa00640 1 -- 172.21.15.119:0/399705889 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5054005ce0 con 0x7f50640aedc0 2024-04-25T10:14:50.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.482+0000 7f505aa00640 1 -- 172.21.15.119:0/399705889 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50540072f0 con 0x7f50640aedc0 2024-04-25T10:14:50.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.482+0000 7f5068e00640 1 -- 172.21.15.119:0/399705889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50641f4df0 con 0x7f50640aedc0 2024-04-25T10:14:50.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.482+0000 7f5068e00640 1 -- 172.21.15.119:0/399705889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50641e3680 con 0x7f50640aedc0 2024-04-25T10:14:50.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.483+0000 7f5068e00640 1 -- 172.21.15.119:0/399705889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5064072150 con 0x7f50640aedc0 2024-04-25T10:14:50.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.484+0000 7f505aa00640 1 -- 172.21.15.119:0/399705889 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f505401a070 con 0x7f50640aedc0 2024-04-25T10:14:50.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.485+0000 7f505aa00640 1 --2- 172.21.15.119:0/399705889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f504006b4d0 0x7f504006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.485+0000 7f505aa00640 1 -- 172.21.15.119:0/399705889 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f5054089700 con 0x7f50640aedc0 2024-04-25T10:14:50.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.486+0000 7f5059400640 1 --2- 172.21.15.119:0/399705889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f504006b4d0 0x7f504006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.486+0000 7f5059400640 1 --2- 172.21.15.119:0/399705889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f504006b4d0 0x7f504006d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f504c006fd0 tx=0x7f504c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.490+0000 7f505aa00640 1 -- 172.21.15.119:0/399705889 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5064072150 con 0x7f50640aedc0 2024-04-25T10:14:50.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.733+0000 7f5068e00640 1 -- 172.21.15.119:0/399705889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5064067c00 con 0x7f50640aedc0 2024-04-25T10:14:50.738 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:14:50.735+0000 7f8c64600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:14:50.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.738+0000 7f505aa00640 1 -- 172.21.15.119:0/399705889 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5064067c00 con 0x7f50640aedc0 2024-04-25T10:14:50.743 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.741+0000 7f5068e00640 1 -- 172.21.15.119:0/399705889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f504006b4d0 msgr2=0x7f504006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.741+0000 7f5068e00640 1 --2- 172.21.15.119:0/399705889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f504006b4d0 0x7f504006d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f504c006fd0 tx=0x7f504c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.741+0000 7f5068e00640 1 -- 172.21.15.119:0/399705889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50640aedc0 msgr2=0x7f50641f48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.741+0000 7f5068e00640 1 --2- 172.21.15.119:0/399705889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50640aedc0 0x7f50641f48b0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f505400c400 tx=0x7f505400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.742+0000 7f5068e00640 1 -- 172.21.15.119:0/399705889 shutdown_connections 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.742+0000 7f5068e00640 1 --2- 172.21.15.119:0/399705889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f504006b4d0 0x7f504006d990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.742+0000 7f5068e00640 1 --2- 172.21.15.119:0/399705889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f506415ae20 0x7f5064147df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.742+0000 7f5068e00640 1 --2- 172.21.15.119:0/399705889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50640b4730 0x7f50640ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.742+0000 7f5068e00640 1 --2- 172.21.15.119:0/399705889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50640aedc0 0x7f50641f48b0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.742+0000 7f5068e00640 1 -- 172.21.15.119:0/399705889 >> 172.21.15.119:0/399705889 conn(0x7f506406aba0 msgr2=0x7f506406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.742+0000 7f5068e00640 1 -- 172.21.15.119:0/399705889 shutdown_connections 2024-04-25T10:14:50.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.743+0000 7f5068e00640 1 -- 172.21.15.119:0/399705889 wait complete. 2024-04-25T10:14:50.757 INFO:tasks.ceph.mds.a.smithi119.stderr: -4585> 2024-04-25T10:13:42.184+0000 7f5368c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:50.758 INFO:tasks.ceph.mds.a.smithi119.stderr: -4215> 2024-04-25T10:13:43.574+0000 7f5363c00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T10:14:50.758 INFO:tasks.ceph.mds.a.smithi119.stderr: -4057> 2024-04-25T10:13:43.831+0000 7f5363c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-25T10:13:32.156889+0000 1=1+0) n(v0 rc2024-04-25T10:13:32.264886+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x561f3d610100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:13:32.156889+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:13:32.264886+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T10:14:50.762 INFO:tasks.ceph.mds.a.smithi119.stderr: -2303> 2024-04-25T10:14:15.836+0000 7f5363c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv18 ap=3 DIRTYPARENT f(v0 m2024-04-25T10:13:32.156889+0000 1=1+0)->f(v1 m2024-04-25T10:13:32.156889+0000) n(v0 rc2024-04-25T10:13:32.264886+0000 2=1+1)->n(v0 rc2024-04-25T10:13:32.264886+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x561f3d610100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:13:32.156889+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:13:32.264886+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T10:14:50.767 DEBUG:teuthology.orchestra.run.smithi119:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T10:14:50.809 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:14:50.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.847+0000 7f7cfc212740 1 -- start start 2024-04-25T10:14:50.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.847+0000 7f7cfc212740 1 Processor -- start 2024-04-25T10:14:50.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.848+0000 7f7cfc212740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56001fdaec00 0x56001fe4e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.848+0000 7f7cfc212740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56001fdaf800 0x56001fe4eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.848+0000 7f7cfc212740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56001fdafc00 0x56001fe4f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.848+0000 7f7cfc212740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560020b88480 con 0x56001fdaec00 2024-04-25T10:14:50.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.848+0000 7f7cfc212740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560020b88300 con 0x56001fdafc00 2024-04-25T10:14:50.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.848+0000 7f7cfc212740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x560020b88180 con 0x56001fdaf800 2024-04-25T10:14:50.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.848+0000 7f7cf9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56001fdaec00 0x56001fe4e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.848+0000 7f7cf9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56001fdaec00 0x56001fe4e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60470/0 (socket says 172.21.15.119:60470) 2024-04-25T10:14:50.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.848+0000 7f7cf9800640 1 -- 172.21.15.119:0/2474746403 learned_addr learned my addr 172.21.15.119:0/2474746403 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:50.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.849+0000 7f7cfa200640 1 --2- 172.21.15.119:0/2474746403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56001fdafc00 0x56001fe4f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.849+0000 7f7cf9800640 1 -- 172.21.15.119:0/2474746403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56001fdaf800 msgr2=0x56001fe4eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.849+0000 7f7cf9800640 1 --2- 172.21.15.119:0/2474746403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56001fdaf800 0x56001fe4eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.849+0000 7f7cf9800640 1 -- 172.21.15.119:0/2474746403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56001fdafc00 msgr2=0x56001fe4f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.849+0000 7f7cf9800640 1 --2- 172.21.15.119:0/2474746403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56001fdafc00 0x56001fe4f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.849+0000 7f7cf9800640 1 -- 172.21.15.119:0/2474746403 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56001fddb680 con 0x56001fdaec00 2024-04-25T10:14:50.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.849+0000 7f7cfa200640 1 --2- 172.21.15.119:0/2474746403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56001fdafc00 0x56001fe4f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:50.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.849+0000 7f7cf9800640 1 --2- 172.21.15.119:0/2474746403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56001fdaec00 0x56001fe4e580 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x560020bb9c80 tx=0x560020c66300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953a50f0fa491d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.849+0000 7f7cf6600640 1 -- 172.21.15.119:0/2474746403 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56001fdd4680 con 0x56001fdaec00 2024-04-25T10:14:50.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.849+0000 7f7cf6600640 1 -- 172.21.15.119:0/2474746403 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56001fdf8e00 con 0x56001fdaec00 2024-04-25T10:14:50.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.849+0000 7f7cf6600640 1 -- 172.21.15.119:0/2474746403 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56001fdd4b60 con 0x56001fdaec00 2024-04-25T10:14:50.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.849+0000 7f7cfc212740 1 -- 172.21.15.119:0/2474746403 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56001fddb860 con 0x56001fdaec00 2024-04-25T10:14:50.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.849+0000 7f7cfc212740 1 -- 172.21.15.119:0/2474746403 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56001fddb2c0 con 0x56001fdaec00 2024-04-25T10:14:50.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.849+0000 7f7cfc212740 4 waiting for MDS map... 2024-04-25T10:14:50.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.850+0000 7f7cf6600640 1 -- 172.21.15.119:0/2474746403 <== mon.0 v2:172.21.15.119:3300/0 4 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x560020c68600 con 0x56001fdaec00 2024-04-25T10:14:50.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.850+0000 7f7cf6600640 1 -- 172.21.15.119:0/2474746403 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 69) v1 ==== 2723+0+0 (secure 0 0 0) 0x56001fe07180 con 0x56001fdaec00 2024-04-25T10:14:50.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.850+0000 7f7cfc212740 4 Got MDS map 69 2024-04-25T10:14:50.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.850+0000 7f7cfc212740 4 datascan.main: Using metadata pool output 2024-04-25T10:14:50.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.850+0000 7f7cfc212740 4 datascan.main: connecting to RADOS... 2024-04-25T10:14:50.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.851+0000 7f7cfc212740 1 Processor -- start 2024-04-25T10:14:50.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.851+0000 7f7cfc212740 1 -- start start 2024-04-25T10:14:50.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cfc212740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560020c6b000 0x56001fe4f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cfc212740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560020c6b400 0x56001fe4fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cfc212740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560020c6b800 0x56001fe50100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cfc212740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560020b88780 con 0x560020c6b400 2024-04-25T10:14:50.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cfc212740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001fe0e180 con 0x560020c6b000 2024-04-25T10:14:50.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cfc212740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56001fe0e900 con 0x560020c6b800 2024-04-25T10:14:50.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cf9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560020c6b400 0x56001fe4fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cf9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560020c6b400 0x56001fe4fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60474/0 (socket says 172.21.15.119:60474) 2024-04-25T10:14:50.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cf9800640 1 -- 172.21.15.119:0/3071984478 learned_addr learned my addr 172.21.15.119:0/3071984478 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:50.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cf9800640 1 -- 172.21.15.119:0/3071984478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560020c6b800 msgr2=0x56001fe50100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cf8e00640 1 --2- 172.21.15.119:0/3071984478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560020c6b800 0x56001fe50100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cfa200640 1 --2- 172.21.15.119:0/3071984478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560020c6b000 0x56001fe4f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cf9800640 1 --2- 172.21.15.119:0/3071984478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560020c6b800 0x56001fe50100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cf9800640 1 -- 172.21.15.119:0/3071984478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560020c6b000 msgr2=0x56001fe4f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cf9800640 1 --2- 172.21.15.119:0/3071984478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560020c6b000 0x56001fe4f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cf9800640 1 -- 172.21.15.119:0/3071984478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56001fddb4a0 con 0x560020c6b400 2024-04-25T10:14:50.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cf8e00640 1 --2- 172.21.15.119:0/3071984478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560020c6b800 0x56001fe50100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:50.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.852+0000 7f7cfa200640 1 --2- 172.21.15.119:0/3071984478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560020c6b000 0x56001fe4f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:50.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.853+0000 7f7cf9800640 1 --2- 172.21.15.119:0/3071984478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560020c6b400 0x56001fe4fb80 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x560020c76b10 tx=0x560020b3f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb99f7488420d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.853+0000 7f7cf3e00640 1 -- 172.21.15.119:0/3071984478 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56001fdd4820 con 0x560020c6b400 2024-04-25T10:14:50.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.853+0000 7f7cf3e00640 1 -- 172.21.15.119:0/3071984478 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56001fdf8c40 con 0x560020c6b400 2024-04-25T10:14:50.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.853+0000 7f7cf3e00640 1 -- 172.21.15.119:0/3071984478 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56001fdd4680 con 0x560020c6b400 2024-04-25T10:14:50.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.853+0000 7f7cfc212740 1 -- 172.21.15.119:0/3071984478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560020c6b400 msgr2=0x56001fe4fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.853+0000 7f7cfc212740 1 --2- 172.21.15.119:0/3071984478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560020c6b400 0x56001fe4fb80 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x560020c76b10 tx=0x560020b3f5c0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.853+0000 7f7cfc212740 1 -- 172.21.15.119:0/3071984478 shutdown_connections 2024-04-25T10:14:50.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.853+0000 7f7cfc212740 1 --2- 172.21.15.119:0/3071984478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560020c6b800 0x56001fe50100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.853+0000 7f7cfc212740 1 --2- 172.21.15.119:0/3071984478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560020c6b400 0x56001fe4fb80 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.853+0000 7f7cfc212740 1 --2- 172.21.15.119:0/3071984478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560020c6b000 0x56001fe4f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.853+0000 7f7cfc212740 1 -- 172.21.15.119:0/3071984478 >> 172.21.15.119:0/3071984478 conn(0x560020c6ac00 msgr2=0x560020b2b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.854+0000 7f7cfc212740 1 -- 172.21.15.119:0/3071984478 shutdown_connections 2024-04-25T10:14:50.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.854+0000 7f7cfc212740 1 -- 172.21.15.119:0/3071984478 wait complete. 2024-04-25T10:14:50.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.854+0000 7f7cfc212740 1 Processor -- start 2024-04-25T10:14:50.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.854+0000 7f7cfc212740 1 -- start start 2024-04-25T10:14:50.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.855+0000 7f7cfc212740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560020c6b000 0x56001fe51180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.855+0000 7f7cfc212740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560020c6b400 0x56001fe50c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.855+0000 7f7cfc212740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560020c6b800 0x56001fe50680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.855+0000 7f7cfc212740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560020b89500 con 0x560020c6b000 2024-04-25T10:14:50.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.855+0000 7f7cfc212740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560020b89680 con 0x560020c6b400 2024-04-25T10:14:50.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.855+0000 7f7cfc212740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x560020b89380 con 0x560020c6b800 2024-04-25T10:14:50.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.855+0000 7f7cfa200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560020c6b000 0x56001fe51180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.855+0000 7f7cfa200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560020c6b000 0x56001fe51180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60476/0 (socket says 172.21.15.119:60476) 2024-04-25T10:14:50.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.855+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 learned_addr learned my addr 172.21.15.119:0/2402654154 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:50.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.856+0000 7f7cf9800640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560020c6b400 0x56001fe50c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.856+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560020c6b800 msgr2=0x56001fe50680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.856+0000 7f7cf8e00640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560020c6b800 0x56001fe50680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.856+0000 7f7cfa200640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560020c6b800 0x56001fe50680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.856+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560020c6b400 msgr2=0x56001fe50c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.856+0000 7f7cfa200640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560020c6b400 0x56001fe50c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.856+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56001fddb680 con 0x560020c6b000 2024-04-25T10:14:50.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.856+0000 7f7cf9800640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560020c6b400 0x56001fe50c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:50.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.856+0000 7f7cfa200640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560020c6b000 0x56001fe51180 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x560020bb9200 tx=0x560020c66120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.856+0000 7f7cf2000640 1 -- 172.21.15.119:0/2402654154 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56001fdd49c0 con 0x560020c6b000 2024-04-25T10:14:50.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.856+0000 7f7cf2000640 1 -- 172.21.15.119:0/2402654154 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56001fdf9340 con 0x560020c6b000 2024-04-25T10:14:50.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.856+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56001fddb860 con 0x560020c6b000 2024-04-25T10:14:50.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.856+0000 7f7cf2000640 1 -- 172.21.15.119:0/2402654154 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56001fdd4820 con 0x560020c6b000 2024-04-25T10:14:50.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.856+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56001fddb0e0 con 0x560020c6b000 2024-04-25T10:14:50.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.857+0000 7f7cfc212740 4 datascan.init: resolving metadata pool 12 2024-04-25T10:14:50.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.858+0000 7f7cf2000640 1 -- 172.21.15.119:0/2402654154 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x560020cdac00 con 0x560020c6b000 2024-04-25T10:14:50.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.859+0000 7f7cf2000640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x560020c6bc00 0x56001fe51700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.859+0000 7f7cf2000640 1 -- 172.21.15.119:0/2402654154 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x560020cde600 con 0x560020c6b000 2024-04-25T10:14:50.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.859+0000 7f7cfc212740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:14:50.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.859+0000 7f7cfc212740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-25T10:14:50.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.859+0000 7f7cfc212740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:14:50.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.859+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56002010c000 0x56001fe50100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.859+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002010c400 con 0x56002010c000 2024-04-25T10:14:50.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.859+0000 7f7cf9800640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x560020c6bc00 0x56001fe51700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.860+0000 7f7cf9800640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x560020c6bc00 0x56001fe51700 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x560020c76b10 tx=0x560020cd8060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.860+0000 7f7cf8e00640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56002010c000 0x56001fe50100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.861+0000 7f7cf8e00640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56002010c000 0x56001fe50100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.866+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.7 v2:172.21.15.132:6821/2727385616 1 ==== osd_op_reply(1 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010c000 2024-04-25T10:14:50.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.867+0000 7f7cf3400640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x56002010d400 0x56002011c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.867+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002010d800 con 0x56002010d400 2024-04-25T10:14:50.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.867+0000 7f7cfa200640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x56002010d400 0x56002011c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.868+0000 7f7cfa200640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x56002010d400 0x56002011c000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.874+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.6 v2:172.21.15.132:6806/353653608 1 ==== osd_op_reply(2 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x56002010d400 2024-04-25T10:14:50.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.874+0000 7f7cf3e00640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x560020128000 0x56002011c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.875+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020128400 con 0x560020128000 2024-04-25T10:14:50.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.875+0000 7f7cf9800640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x560020128000 0x56002011c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.875+0000 7f7cf9800640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x560020128000 0x56002011c580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.876+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== osd_op_reply(3 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020128000 2024-04-25T10:14:50.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.876+0000 7f7cf3400640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56002010dc00 0x56002011cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.876+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020128c00 con 0x56002010dc00 2024-04-25T10:14:50.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.877+0000 7f7cf8e00640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56002010dc00 0x56002011cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.878+0000 7f7cf8e00640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56002010dc00 0x56002011cb00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.884+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.4 v2:172.21.15.132:6800/1740646323 1 ==== osd_op_reply(4 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010dc00 2024-04-25T10:14:50.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.884+0000 7f7cf3e00640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x560020128800 0x56002011d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.884+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020128c00 con 0x560020128800 2024-04-25T10:14:50.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.885+0000 7f7cfa200640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x560020128800 0x56002011d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.885+0000 7f7cfa200640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x560020128800 0x56002011d600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.886+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.2 v2:172.21.15.119:6824/1001762963 1 ==== osd_op_reply(5 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x560020128800 2024-04-25T10:14:50.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.886+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020129000 con 0x560020128800 2024-04-25T10:14:50.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.886+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.2 v2:172.21.15.119:6824/1001762963 2 ==== osd_op_reply(6 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x560020128800 2024-04-25T10:14:50.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.886+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020129800 con 0x56002010c000 2024-04-25T10:14:50.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.887+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.7 v2:172.21.15.132:6821/2727385616 2 ==== osd_op_reply(7 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010c000 2024-04-25T10:14:50.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.887+0000 7f7cf3400640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x560020129400 0x56002011d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.887+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020129c00 con 0x560020129400 2024-04-25T10:14:50.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.888+0000 7f7cf9800640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x560020129400 0x56002011d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.889+0000 7f7cf9800640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x560020129400 0x56002011d080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.890+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.5 v2:172.21.15.132:6815/2978152964 1 ==== osd_op_reply(8 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020129400 2024-04-25T10:14:50.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.890+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002010c400 con 0x56002010d400 2024-04-25T10:14:50.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.891+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.6 v2:172.21.15.132:6806/353653608 2 ==== osd_op_reply(9 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x56002010d400 2024-04-25T10:14:50.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.891+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002010d800 con 0x56002010d400 2024-04-25T10:14:50.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.892+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.6 v2:172.21.15.132:6806/353653608 3 ==== osd_op_reply(10 [pgnls start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x56002010d400 2024-04-25T10:14:50.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.892+0000 7f7cf3e00640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x56002013e000 0x56002011db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.892+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013e400 con 0x56002013e000 2024-04-25T10:14:50.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.892+0000 7f7cf8e00640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x56002013e000 0x56002011db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.893+0000 7f7cf8e00640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x56002013e000 0x56002011db80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.897+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.0 v2:172.21.15.119:6808/2890625333 1 ==== osd_op_reply(11 [pgnls start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002013e000 2024-04-25T10:14:50.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.897+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002010c400 con 0x56002010d400 2024-04-25T10:14:50.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.898+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.6 v2:172.21.15.132:6806/353653608 4 ==== osd_op_reply(12 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x56002010d400 2024-04-25T10:14:50.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.898+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013e800 con 0x560020128000 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.898+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.1 v2:172.21.15.119:6800/298545098 2 ==== osd_op_reply(13 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020128000 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.898+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020128c00 con 0x560020128800 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.899+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.2 v2:172.21.15.119:6824/1001762963 3 ==== osd_op_reply(14 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x560020128800 2024-04-25T10:14:50.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.899+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013ec00 con 0x56002010d400 2024-04-25T10:14:50.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.900+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.6 v2:172.21.15.132:6806/353653608 5 ==== osd_op_reply(15 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x56002010d400 2024-04-25T10:14:50.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.900+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020129000 con 0x560020128800 2024-04-25T10:14:50.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.900+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.2 v2:172.21.15.119:6824/1001762963 4 ==== osd_op_reply(16 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x560020128800 2024-04-25T10:14:50.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.900+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013f000 con 0x560020129400 2024-04-25T10:14:50.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.901+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.5 v2:172.21.15.132:6815/2978152964 2 ==== osd_op_reply(17 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020129400 2024-04-25T10:14:50.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.901+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020b85400 con 0x560020128000 2024-04-25T10:14:50.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.902+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.1 v2:172.21.15.119:6800/298545098 3 ==== osd_op_reply(18 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020128000 2024-04-25T10:14:50.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.902+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013f400 con 0x56002013e000 2024-04-25T10:14:50.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.903+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.0 v2:172.21.15.119:6808/2890625333 2 ==== osd_op_reply(19 [pgnls start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002013e000 2024-04-25T10:14:50.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.903+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013e400 con 0x560020128800 2024-04-25T10:14:50.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.903+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.2 v2:172.21.15.119:6824/1001762963 5 ==== osd_op_reply(20 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x560020128800 2024-04-25T10:14:50.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.903+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013f800 con 0x560020128800 2024-04-25T10:14:50.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.904+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.2 v2:172.21.15.119:6824/1001762963 6 ==== osd_op_reply(21 [pgnls start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x560020128800 2024-04-25T10:14:50.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.904+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013f400 con 0x560020128000 2024-04-25T10:14:50.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.905+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.1 v2:172.21.15.119:6800/298545098 4 ==== osd_op_reply(22 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020128000 2024-04-25T10:14:50.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.905+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013fc00 con 0x560020128000 2024-04-25T10:14:50.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.905+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.1 v2:172.21.15.119:6800/298545098 5 ==== osd_op_reply(23 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020128000 2024-04-25T10:14:50.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.905+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013e800 con 0x560020129400 2024-04-25T10:14:50.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.906+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.5 v2:172.21.15.132:6815/2978152964 3 ==== osd_op_reply(24 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020129400 2024-04-25T10:14:50.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.907+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013ec00 con 0x56002010dc00 2024-04-25T10:14:50.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.908+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.4 v2:172.21.15.132:6800/1740646323 2 ==== osd_op_reply(25 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010dc00 2024-04-25T10:14:50.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.908+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020129800 con 0x56002010c000 2024-04-25T10:14:50.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.909+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.7 v2:172.21.15.132:6821/2727385616 3 ==== osd_op_reply(26 [pgnls start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010c000 2024-04-25T10:14:50.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.909+0000 7f7cf3e00640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x56002013e400 0x56002011e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.909+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020128c00 con 0x56002013e400 2024-04-25T10:14:50.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.910+0000 7f7cfa200640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x56002013e400 0x56002011e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.910+0000 7f7cfa200640 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x56002013e400 0x56002011e100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.915+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.3 v2:172.21.15.119:6816/1142971786 1 ==== osd_op_reply(27 [pgnls start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x56002013e400 2024-04-25T10:14:50.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.915+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020128400 con 0x56002010d400 2024-04-25T10:14:50.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.916+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.6 v2:172.21.15.132:6806/353653608 6 ==== osd_op_reply(28 [pgnls start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x56002010d400 2024-04-25T10:14:50.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.916+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020129000 con 0x56002010c000 2024-04-25T10:14:50.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.917+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.7 v2:172.21.15.132:6821/2727385616 4 ==== osd_op_reply(29 [pgnls start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010c000 2024-04-25T10:14:50.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.917+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020129c00 con 0x560020128800 2024-04-25T10:14:50.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.918+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.2 v2:172.21.15.119:6824/1001762963 7 ==== osd_op_reply(30 [pgnls start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x560020128800 2024-04-25T10:14:50.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.918+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002010d800 con 0x56002010dc00 2024-04-25T10:14:50.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.919+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.4 v2:172.21.15.132:6800/1740646323 3 ==== osd_op_reply(31 [pgnls start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010dc00 2024-04-25T10:14:50.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.919+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020b85400 con 0x560020128000 2024-04-25T10:14:50.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.920+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.1 v2:172.21.15.119:6800/298545098 6 ==== osd_op_reply(32 [pgnls start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020128000 2024-04-25T10:14:50.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.920+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002010c400 con 0x56002010d400 2024-04-25T10:14:50.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.921+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.6 v2:172.21.15.132:6806/353653608 7 ==== osd_op_reply(33 [pgnls start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x56002010d400 2024-04-25T10:14:50.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.921+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013ec00 con 0x56002010dc00 2024-04-25T10:14:50.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.922+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.4 v2:172.21.15.132:6800/1740646323 4 ==== osd_op_reply(34 [pgnls start_epoch 85 out=49b] v81'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010dc00 2024-04-25T10:14:50.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.922+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002014c000 con 0x560020128000 2024-04-25T10:14:50.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.923+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.1 v2:172.21.15.119:6800/298545098 7 ==== osd_op_reply(35 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020128000 2024-04-25T10:14:50.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.923+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013f000 con 0x56002013e400 2024-04-25T10:14:50.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.924+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.3 v2:172.21.15.119:6816/1142971786 2 ==== osd_op_reply(36 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x56002013e400 2024-04-25T10:14:50.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.924+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002014c400 con 0x560020129400 2024-04-25T10:14:50.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.925+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.5 v2:172.21.15.132:6815/2978152964 4 ==== osd_op_reply(37 [pgnls start_epoch 85 out=49b] v81'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020129400 2024-04-25T10:14:50.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.925+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002010d800 con 0x56002010d400 2024-04-25T10:14:50.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.926+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.6 v2:172.21.15.132:6806/353653608 8 ==== osd_op_reply(38 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x56002010d400 2024-04-25T10:14:50.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.926+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002014c800 con 0x56002013e400 2024-04-25T10:14:50.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.927+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.3 v2:172.21.15.119:6816/1142971786 3 ==== osd_op_reply(39 [pgnls start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x56002013e400 2024-04-25T10:14:50.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.927+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020129800 con 0x560020128800 2024-04-25T10:14:50.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.928+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.2 v2:172.21.15.119:6824/1001762963 8 ==== osd_op_reply(40 [pgnls start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x560020128800 2024-04-25T10:14:50.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.928+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002014cc00 con 0x56002010dc00 2024-04-25T10:14:50.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.929+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.4 v2:172.21.15.132:6800/1740646323 5 ==== osd_op_reply(41 [pgnls start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010dc00 2024-04-25T10:14:50.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.929+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020129000 con 0x56002010dc00 2024-04-25T10:14:50.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.930+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.4 v2:172.21.15.132:6800/1740646323 6 ==== osd_op_reply(42 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010dc00 2024-04-25T10:14:50.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.930+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002014d000 con 0x56002010dc00 2024-04-25T10:14:50.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.931+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.4 v2:172.21.15.132:6800/1740646323 7 ==== osd_op_reply(43 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010dc00 2024-04-25T10:14:50.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.931+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013f800 con 0x560020128000 2024-04-25T10:14:50.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.932+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.1 v2:172.21.15.119:6800/298545098 8 ==== osd_op_reply(44 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020128000 2024-04-25T10:14:50.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.932+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002014d400 con 0x56002010dc00 2024-04-25T10:14:50.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.934+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.4 v2:172.21.15.132:6800/1740646323 8 ==== osd_op_reply(45 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010dc00 2024-04-25T10:14:50.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.934+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013f000 con 0x56002013e000 2024-04-25T10:14:50.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.934+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.0 v2:172.21.15.119:6808/2890625333 3 ==== osd_op_reply(46 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002013e000 2024-04-25T10:14:50.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.934+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002014d800 con 0x56002010c000 2024-04-25T10:14:50.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.936+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.7 v2:172.21.15.132:6821/2727385616 5 ==== osd_op_reply(47 [pgnls start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010c000 2024-04-25T10:14:50.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.936+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013f400 con 0x56002010c000 2024-04-25T10:14:50.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.937+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.7 v2:172.21.15.132:6821/2727385616 6 ==== osd_op_reply(48 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010c000 2024-04-25T10:14:50.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.937+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002014cc00 con 0x56002013e000 2024-04-25T10:14:50.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.938+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.0 v2:172.21.15.119:6808/2890625333 4 ==== osd_op_reply(49 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002013e000 2024-04-25T10:14:50.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.938+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002010c400 con 0x560020128000 2024-04-25T10:14:50.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.938+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.1 v2:172.21.15.119:6800/298545098 9 ==== osd_op_reply(50 [pgnls start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020128000 2024-04-25T10:14:50.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.938+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002014d000 con 0x56002010c000 2024-04-25T10:14:50.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.940+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.7 v2:172.21.15.132:6821/2727385616 7 ==== osd_op_reply(51 [pgnls start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010c000 2024-04-25T10:14:50.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.940+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002010d800 con 0x56002010c000 2024-04-25T10:14:50.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.941+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.7 v2:172.21.15.132:6821/2727385616 8 ==== osd_op_reply(52 [pgnls start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010c000 2024-04-25T10:14:50.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.941+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002014d400 con 0x560020129400 2024-04-25T10:14:50.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.942+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.5 v2:172.21.15.132:6815/2978152964 5 ==== osd_op_reply(53 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020129400 2024-04-25T10:14:50.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.942+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020128c00 con 0x560020128800 2024-04-25T10:14:50.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.943+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.2 v2:172.21.15.119:6824/1001762963 9 ==== osd_op_reply(54 [pgnls start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x560020128800 2024-04-25T10:14:50.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.943+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020b85400 con 0x56002010dc00 2024-04-25T10:14:50.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.945+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.4 v2:172.21.15.132:6800/1740646323 9 ==== osd_op_reply(55 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010dc00 2024-04-25T10:14:50.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.945+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020128400 con 0x56002010dc00 2024-04-25T10:14:50.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.946+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.4 v2:172.21.15.132:6800/1740646323 10 ==== osd_op_reply(56 [pgnls start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010dc00 2024-04-25T10:14:50.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.946+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013ec00 con 0x56002010c000 2024-04-25T10:14:50.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.947+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.7 v2:172.21.15.132:6821/2727385616 9 ==== osd_op_reply(57 [pgnls start_epoch 85 out=81b] v81'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x560020b2c000 con 0x56002010c000 2024-04-25T10:14:50.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.947+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002010c800 con 0x56002010c000 2024-04-25T10:14:50.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.948+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.7 v2:172.21.15.132:6821/2727385616 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x560020b2c000 con 0x56002010c000 2024-04-25T10:14:50.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.948+0000 7f7cfc212740 10 datascan.operator(): handling object 1099511627777.0 2024-04-25T10:14:50.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.948+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002010cc00 con 0x56002010c000 2024-04-25T10:14:50.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.952+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.7 v2:172.21.15.132:6821/2727385616 11 ==== osd_op_reply(59 10000000001.00000000 [call] v85'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x560020b2c000 con 0x56002010c000 2024-04-25T10:14:50.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.952+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002010d000 con 0x560020128000 2024-04-25T10:14:50.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.953+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.1 v2:172.21.15.119:6800/298545098 10 ==== osd_op_reply(60 [pgnls start_epoch 85 out=49b] v81'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020128000 2024-04-25T10:14:50.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.953+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013fc00 con 0x560020129400 2024-04-25T10:14:50.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.954+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.5 v2:172.21.15.132:6815/2978152964 6 ==== osd_op_reply(61 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020129400 2024-04-25T10:14:50.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.954+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020129c00 con 0x560020129400 2024-04-25T10:14:50.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.956+0000 7f7cf9800640 1 -- 172.21.15.119:0/2402654154 <== osd.5 v2:172.21.15.132:6815/2978152964 7 ==== osd_op_reply(62 [pgnls start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c780 con 0x560020129400 2024-04-25T10:14:50.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.956+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002013e800 con 0x56002010d400 2024-04-25T10:14:50.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.957+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.6 v2:172.21.15.132:6806/353653608 9 ==== osd_op_reply(63 [pgnls start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x56002010d400 2024-04-25T10:14:50.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.957+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002014d800 con 0x56002010dc00 2024-04-25T10:14:50.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.958+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 <== osd.4 v2:172.21.15.132:6800/1740646323 11 ==== osd_op_reply(64 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c000 con 0x56002010dc00 2024-04-25T10:14:50.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.958+0000 7f7cf3e00640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x560020129000 con 0x56002010d400 2024-04-25T10:14:50.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.960+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.6 v2:172.21.15.132:6806/353653608 10 ==== osd_op_reply(65 [pgnls start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x56002010d400 2024-04-25T10:14:50.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.960+0000 7f7cf3400640 1 -- 172.21.15.119:0/2402654154 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x56002014cc00 con 0x56002010d400 2024-04-25T10:14:50.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfa200640 1 -- 172.21.15.119:0/2402654154 <== osd.6 v2:172.21.15.132:6806/353653608 11 ==== osd_op_reply(66 [pgnls start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560020b2c500 con 0x56002010d400 2024-04-25T10:14:50.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x56002013e000 msgr2=0x56002011db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x56002013e000 0x56002011db80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x560020128000 msgr2=0x56002011c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x560020128000 0x56002011c580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x560020128800 msgr2=0x56002011d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x560020128800 0x56002011d600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x56002013e400 msgr2=0x56002011e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x56002013e400 0x56002011e100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56002010dc00 msgr2=0x56002011cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56002010dc00 0x56002011cb00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x560020129400 msgr2=0x56002011d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x560020129400 0x56002011d080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x56002010d400 msgr2=0x56002011c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x56002010d400 0x56002011c000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56002010c000 msgr2=0x56001fe50100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56002010c000 0x56001fe50100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 reap_dead start 2024-04-25T10:14:50.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.961+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 reap_dead start 2024-04-25T10:14:50.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.962+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 reap_dead start 2024-04-25T10:14:50.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.962+0000 7f7cf8e00640 1 -- 172.21.15.119:0/2402654154 reap_dead start 2024-04-25T10:14:50.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.962+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x560020c6bc00 msgr2=0x56001fe51700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.962+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x560020c6bc00 0x56001fe51700 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x560020c76b10 tx=0x560020cd8060 comp rx=0 tx=0).stop 2024-04-25T10:14:50.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.962+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560020c6b000 msgr2=0x56001fe51180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.962+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560020c6b000 0x56001fe51180 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x560020bb9200 tx=0x560020c66120 comp rx=0 tx=0).stop 2024-04-25T10:14:50.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.962+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 shutdown_connections 2024-04-25T10:14:50.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.962+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x560020c6bc00 0x56001fe51700 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.962+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2402654154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560020c6b000 0x56001fe51180 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.962+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 >> 172.21.15.119:0/2402654154 conn(0x560020c6ac00 msgr2=0x560020b2b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.963+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 shutdown_connections 2024-04-25T10:14:50.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.963+0000 7f7cfc212740 1 -- 172.21.15.119:0/2402654154 wait complete. 2024-04-25T10:14:50.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.964+0000 7f7cfc212740 1 -- 172.21.15.119:0/2474746403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56001fdaec00 msgr2=0x56001fe4e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.964+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2474746403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56001fdaec00 0x56001fe4e580 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x560020bb9c80 tx=0x560020c66300 comp rx=0 tx=0).stop 2024-04-25T10:14:50.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.964+0000 7f7cfc212740 1 -- 172.21.15.119:0/2474746403 shutdown_connections 2024-04-25T10:14:50.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.964+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2474746403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56001fdafc00 0x56001fe4f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.964+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2474746403 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56001fdaf800 0x56001fe4eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.964+0000 7f7cfc212740 1 --2- 172.21.15.119:0/2474746403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56001fdaec00 0x56001fe4e580 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.964+0000 7f7cfc212740 1 -- 172.21.15.119:0/2474746403 >> 172.21.15.119:0/2474746403 conn(0x56001fdaf000 msgr2=0x560020b2a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.964+0000 7f7cfc212740 1 -- 172.21.15.119:0/2474746403 shutdown_connections 2024-04-25T10:14:50.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:50.965+0000 7f7cfc212740 1 -- 172.21.15.119:0/2474746403 wait complete. 2024-04-25T10:14:50.979 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.212051, result: 2024-04-25T10:14:50.980 DEBUG:teuthology.orchestra.run.smithi119:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-25T10:14:51.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.122+0000 7fa9f491e740 1 -- start start 2024-04-25T10:14:51.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.122+0000 7fa9f491e740 1 Processor -- start 2024-04-25T10:14:51.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.123+0000 7fa9f491e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e879c000 0x5648e7a71080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.123+0000 7fa9f491e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e79d1c00 0x5648e7a71b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.123+0000 7fa9f491e740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e879c800 0x5648e7a71600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.123+0000 7fa9f491e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648e87b2480 con 0x5648e879c000 2024-04-25T10:14:51.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.123+0000 7fa9f491e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648e87b2600 con 0x5648e79d1c00 2024-04-25T10:14:51.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.123+0000 7fa9f491e740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648e87b2780 con 0x5648e879c800 2024-04-25T10:14:51.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.123+0000 7fa9f2200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e879c000 0x5648e7a71080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.123+0000 7fa9f2200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e879c000 0x5648e7a71080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60480/0 (socket says 172.21.15.119:60480) 2024-04-25T10:14:51.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.123+0000 7fa9f2200640 1 -- 172.21.15.119:0/4278333754 learned_addr learned my addr 172.21.15.119:0/4278333754 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.124+0000 7fa9f2c00640 1 --2- 172.21.15.119:0/4278333754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e879c800 0x5648e7a71600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.124+0000 7fa9f2200640 1 -- 172.21.15.119:0/4278333754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e879c800 msgr2=0x5648e7a71600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.124+0000 7fa9f1800640 1 --2- 172.21.15.119:0/4278333754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e79d1c00 0x5648e7a71b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.124+0000 7fa9f2200640 1 --2- 172.21.15.119:0/4278333754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e879c800 0x5648e7a71600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.124+0000 7fa9f2200640 1 -- 172.21.15.119:0/4278333754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e79d1c00 msgr2=0x5648e7a71b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.124+0000 7fa9f2200640 1 --2- 172.21.15.119:0/4278333754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e79d1c00 0x5648e7a71b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.124+0000 7fa9f2200640 1 -- 172.21.15.119:0/4278333754 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648e79fd680 con 0x5648e879c000 2024-04-25T10:14:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.124+0000 7fa9f2c00640 1 --2- 172.21.15.119:0/4278333754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e879c800 0x5648e7a71600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.124+0000 7fa9f2200640 1 --2- 172.21.15.119:0/4278333754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e879c000 0x5648e7a71080 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x5648e87f17d0 tx=0x5648e873a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd962a4fe52ab9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.124+0000 7fa9ef000640 1 -- 172.21.15.119:0/4278333754 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5648e79f6d00 con 0x5648e879c000 2024-04-25T10:14:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.124+0000 7fa9ef000640 1 -- 172.21.15.119:0/4278333754 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5648e7a1a1c0 con 0x5648e879c000 2024-04-25T10:14:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.124+0000 7fa9ef000640 1 -- 172.21.15.119:0/4278333754 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5648e79f6680 con 0x5648e879c000 2024-04-25T10:14:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.124+0000 7fa9f491e740 1 -- 172.21.15.119:0/4278333754 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648e79fd860 con 0x5648e879c000 2024-04-25T10:14:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.124+0000 7fa9f491e740 1 -- 172.21.15.119:0/4278333754 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5648e79fd0e0 con 0x5648e879c000 2024-04-25T10:14:51.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.125+0000 7fa9f491e740 4 waiting for MDS map... 2024-04-25T10:14:51.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.125+0000 7fa9ef000640 1 -- 172.21.15.119:0/4278333754 <== mon.0 v2:172.21.15.119:3300/0 4 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x5648e873c400 con 0x5648e879c000 2024-04-25T10:14:51.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.125+0000 7fa9ef000640 1 -- 172.21.15.119:0/4278333754 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 69) v1 ==== 2723+0+0 (secure 0 0 0) 0x5648e7a29180 con 0x5648e879c000 2024-04-25T10:14:51.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.125+0000 7fa9f491e740 4 Got MDS map 69 2024-04-25T10:14:51.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.125+0000 7fa9f491e740 4 datascan.main: Using metadata pool output 2024-04-25T10:14:51.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.125+0000 7fa9f491e740 4 datascan.main: connecting to RADOS... 2024-04-25T10:14:51.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.126+0000 7fa9f491e740 1 Processor -- start 2024-04-25T10:14:51.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.126+0000 7fa9f491e740 1 -- start start 2024-04-25T10:14:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.127+0000 7fa9f491e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e88f0000 0x5648e7a70b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.127+0000 7fa9f491e740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e88f0400 0x5648e7a72100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.127+0000 7fa9f491e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e88f0800 0x5648e7a72680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.127+0000 7fa9f491e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648e7a30180 con 0x5648e88f0800 2024-04-25T10:14:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.127+0000 7fa9f491e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648e87b2f00 con 0x5648e88f0000 2024-04-25T10:14:51.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.127+0000 7fa9f491e740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648e87b3080 con 0x5648e88f0400 2024-04-25T10:14:51.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.127+0000 7fa9f1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e88f0800 0x5648e7a72680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.127+0000 7fa9f1800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e88f0800 0x5648e7a72680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60494/0 (socket says 172.21.15.119:60494) 2024-04-25T10:14:51.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.127+0000 7fa9f1800640 1 -- 172.21.15.119:0/1345687858 learned_addr learned my addr 172.21.15.119:0/1345687858 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:51.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.128+0000 7fa9f2200640 1 --2- 172.21.15.119:0/1345687858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e88f0400 0x5648e7a72100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.128+0000 7fa9f2c00640 1 --2- 172.21.15.119:0/1345687858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e88f0000 0x5648e7a70b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.128+0000 7fa9f1800640 1 -- 172.21.15.119:0/1345687858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e88f0400 msgr2=0x5648e7a72100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:51.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.128+0000 7fa9f1800640 1 --2- 172.21.15.119:0/1345687858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e88f0400 0x5648e7a72100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.128+0000 7fa9f1800640 1 -- 172.21.15.119:0/1345687858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e88f0000 msgr2=0x5648e7a70b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:51.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.128+0000 7fa9f1800640 1 --2- 172.21.15.119:0/1345687858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e88f0000 0x5648e7a70b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.128+0000 7fa9f1800640 1 -- 172.21.15.119:0/1345687858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648e79fd4a0 con 0x5648e88f0800 2024-04-25T10:14:51.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.128+0000 7fa9f2c00640 1 --2- 172.21.15.119:0/1345687858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e88f0000 0x5648e7a70b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:51.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.128+0000 7fa9f1800640 1 --2- 172.21.15.119:0/1345687858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e88f0800 0x5648e7a72680 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x5648e88a0de0 tx=0x5648e88fe000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3f1307fafb679f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.128+0000 7fa9ec800640 1 -- 172.21.15.119:0/1345687858 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5648e79f6d00 con 0x5648e88f0800 2024-04-25T10:14:51.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.128+0000 7fa9ec800640 1 -- 172.21.15.119:0/1345687858 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5648e7a1ae00 con 0x5648e88f0800 2024-04-25T10:14:51.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.128+0000 7fa9ec800640 1 -- 172.21.15.119:0/1345687858 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5648e79f6680 con 0x5648e88f0800 2024-04-25T10:14:51.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.129+0000 7fa9f491e740 1 -- 172.21.15.119:0/1345687858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e88f0800 msgr2=0x5648e7a72680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.129+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1345687858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e88f0800 0x5648e7a72680 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x5648e88a0de0 tx=0x5648e88fe000 comp rx=0 tx=0).stop 2024-04-25T10:14:51.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.129+0000 7fa9f491e740 1 -- 172.21.15.119:0/1345687858 shutdown_connections 2024-04-25T10:14:51.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.129+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1345687858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e88f0800 0x5648e7a72680 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.129+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1345687858 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e88f0400 0x5648e7a72100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.129+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1345687858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e88f0000 0x5648e7a70b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.129+0000 7fa9f491e740 1 -- 172.21.15.119:0/1345687858 >> 172.21.15.119:0/1345687858 conn(0x5648e879c400 msgr2=0x5648e874d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:51.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.129+0000 7fa9f491e740 1 -- 172.21.15.119:0/1345687858 shutdown_connections 2024-04-25T10:14:51.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.129+0000 7fa9f491e740 1 -- 172.21.15.119:0/1345687858 wait complete. 2024-04-25T10:14:51.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.130+0000 7fa9f491e740 1 Processor -- start 2024-04-25T10:14:51.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.130+0000 7fa9f491e740 1 -- start start 2024-04-25T10:14:51.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f491e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e88f0000 0x5648e7a70b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f491e740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e88f0400 0x5648e7a72100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f491e740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e88f0800 0x5648e7a72680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f491e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648e87b3380 con 0x5648e88f0000 2024-04-25T10:14:51.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f491e740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648e87b3200 con 0x5648e88f0400 2024-04-25T10:14:51.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f491e740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648e87b3500 con 0x5648e88f0800 2024-04-25T10:14:51.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f2c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e88f0000 0x5648e7a70b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f2c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e88f0000 0x5648e7a70b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60496/0 (socket says 172.21.15.119:60496) 2024-04-25T10:14:51.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 learned_addr learned my addr 172.21.15.119:0/1589154042 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:51.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f1800640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e88f0800 0x5648e7a72680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f2200640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e88f0400 0x5648e7a72100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e88f0800 msgr2=0x5648e7a72680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f2c00640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e88f0800 0x5648e7a72680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e88f0400 msgr2=0x5648e7a72100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f2c00640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e88f0400 0x5648e7a72100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.131+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648e79fd680 con 0x5648e88f0000 2024-04-25T10:14:51.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.132+0000 7fa9f2c00640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e88f0000 0x5648e7a70b00 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x5648e88a1ad0 tx=0x5648e88fe120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.132+0000 7fa9eaa00640 1 -- 172.21.15.119:0/1589154042 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5648e79f6d00 con 0x5648e88f0000 2024-04-25T10:14:51.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.132+0000 7fa9eaa00640 1 -- 172.21.15.119:0/1589154042 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5648e7a1ac40 con 0x5648e88f0000 2024-04-25T10:14:51.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.132+0000 7fa9eaa00640 1 -- 172.21.15.119:0/1589154042 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5648e79f6680 con 0x5648e88f0000 2024-04-25T10:14:51.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.132+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5648e79fd2c0 con 0x5648e88f0000 2024-04-25T10:14:51.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.132+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648e79fda40 con 0x5648e88f0000 2024-04-25T10:14:51.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.132+0000 7fa9f491e740 4 datascan.init: resolving metadata pool 12 2024-04-25T10:14:51.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.134+0000 7fa9eaa00640 1 -- 172.21.15.119:0/1589154042 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x5648e88fcc00 con 0x5648e88f0000 2024-04-25T10:14:51.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.134+0000 7fa9eaa00640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x5648e890a800 0x5648e7a73700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.134+0000 7fa9eaa00640 1 -- 172.21.15.119:0/1589154042 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x5648e88fb000 con 0x5648e88f0000 2024-04-25T10:14:51.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.135+0000 7fa9f491e740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:14:51.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.135+0000 7fa9f491e740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-25T10:14:51.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.135+0000 7fa9f491e740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:14:51.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.135+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5648e88f0c00 0x5648e7a72c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.135+0000 7fa9f2200640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x5648e890a800 0x5648e7a73700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.135+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1000 con 0x5648e88f0c00 2024-04-25T10:14:51.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.135+0000 7fa9f2200640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x5648e890a800 0x5648e7a73700 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x5648e87f1860 tx=0x5648e873a1e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.135+0000 7fa9f1800640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5648e88f0c00 0x5648e7a72c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.136+0000 7fa9f1800640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5648e88f0c00 0x5648e7a72c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.137+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.5 v2:172.21.15.132:6815/2978152964 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5648e874e000 con 0x5648e88f0c00 2024-04-25T10:14:51.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.137+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5648e890bc00 0x5648e7a73180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.137+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1c00 con 0x5648e890bc00 2024-04-25T10:14:51.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.138+0000 7fa9f2c00640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5648e890bc00 0x5648e7a73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.139+0000 7fa9f2c00640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5648e890bc00 0x5648e7a73180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.140+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.140+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5648e88f1400 0x5648e7d4c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.140+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1800 con 0x5648e88f1400 2024-04-25T10:14:51.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.141+0000 7fa9f2200640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5648e88f1400 0x5648e7d4c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.142+0000 7fa9f2200640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5648e88f1400 0x5648e7d4c000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.143+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.143+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d58000 con 0x5648e890bc00 2024-04-25T10:14:51.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.144+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.144+0000 7fa9ec800640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5648e7d58400 0x5648e7d4db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.144+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d58800 con 0x5648e7d58400 2024-04-25T10:14:51.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.145+0000 7fa9f1800640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5648e7d58400 0x5648e7d4db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.145+0000 7fa9f1800640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5648e7d58400 0x5648e7d4db80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.146+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.146+0000 7fa9ebe00640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5648e7d59000 0x5648e7d4e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.146+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59400 con 0x5648e7d59000 2024-04-25T10:14:51.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.147+0000 7fa9f2c00640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5648e7d59000 0x5648e7d4e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.148+0000 7fa9f2c00640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5648e7d59000 0x5648e7d4e100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.148+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.148+0000 7fa9ec800640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x5648e7d58c00 0x5648e7d4e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.148+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d58000 con 0x5648e7d58c00 2024-04-25T10:14:51.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.149+0000 7fa9f2200640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x5648e7d58c00 0x5648e7d4e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.149+0000 7fa9f2200640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x5648e7d58c00 0x5648e7d4e680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.150+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.150+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59400 con 0x5648e7d58c00 2024-04-25T10:14:51.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.150+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.150+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1c00 con 0x5648e88f1400 2024-04-25T10:14:51.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.151+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.151+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59800 con 0x5648e88f0c00 2024-04-25T10:14:51.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.152+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.5 v2:172.21.15.132:6815/2978152964 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e88f0c00 2024-04-25T10:14:51.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.152+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890a400 con 0x5648e890bc00 2024-04-25T10:14:51.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.153+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.153+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59c00 con 0x5648e890bc00 2024-04-25T10:14:51.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.154+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.154+0000 7fa9ec800640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x5648e7d58000 0x5648e7d4ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.154+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59400 con 0x5648e7d58000 2024-04-25T10:14:51.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.155+0000 7fa9f1800640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x5648e7d58000 0x5648e7d4ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.156+0000 7fa9f1800640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x5648e7d58000 0x5648e7d4ec00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.156+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.0 v2:172.21.15.119:6808/2890625333 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58000 2024-04-25T10:14:51.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.156+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1800 con 0x5648e890bc00 2024-04-25T10:14:51.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.157+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.157+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890b800 con 0x5648e7d58400 2024-04-25T10:14:51.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.158+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 2 ==== osd_op_reply(15 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.158+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1c00 con 0x5648e7d58c00 2024-04-25T10:14:51.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.159+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.159+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890b400 con 0x5648e890bc00 2024-04-25T10:14:51.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.159+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.160+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6c000 con 0x5648e7d58c00 2024-04-25T10:14:51.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.160+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.160+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6d400 con 0x5648e88f0c00 2024-04-25T10:14:51.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.161+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.5 v2:172.21.15.132:6815/2978152964 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e88f0c00 2024-04-25T10:14:51.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.162+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6c400 con 0x5648e7d58400 2024-04-25T10:14:51.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.162+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.162+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6d800 con 0x5648e7d58000 2024-04-25T10:14:51.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.163+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.0 v2:172.21.15.119:6808/2890625333 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58000 2024-04-25T10:14:51.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.163+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6c800 con 0x5648e7d58c00 2024-04-25T10:14:51.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.163+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.163+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6dc00 con 0x5648e7d58c00 2024-04-25T10:14:51.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.164+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.164+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6cc00 con 0x5648e7d58400 2024-04-25T10:14:51.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.164+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.164+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6d400 con 0x5648e7d58400 2024-04-25T10:14:51.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.165+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.165+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6d000 con 0x5648e88f0c00 2024-04-25T10:14:51.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.166+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.5 v2:172.21.15.132:6815/2978152964 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e88f0c00 2024-04-25T10:14:51.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.166+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890b800 con 0x5648e7d59000 2024-04-25T10:14:51.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.167+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.167+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890a400 con 0x5648e88f1400 2024-04-25T10:14:51.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.168+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.168+0000 7fa9ec800640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x5648e7d58800 0x5648e7d4f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.168+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59800 con 0x5648e7d58800 2024-04-25T10:14:51.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.168+0000 7fa9f2c00640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x5648e7d58800 0x5648e7d4f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.169+0000 7fa9f2c00640 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x5648e7d58800 0x5648e7d4f180 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.170+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.3 v2:172.21.15.119:6816/1142971786 1 ==== osd_op_reply(29 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d58800 2024-04-25T10:14:51.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.170+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890b400 con 0x5648e890bc00 2024-04-25T10:14:51.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.171+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.171+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59400 con 0x5648e88f1400 2024-04-25T10:14:51.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.172+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.172+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890b800 con 0x5648e7d58c00 2024-04-25T10:14:51.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.173+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 7 ==== osd_op_reply(32 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.173+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890a400 con 0x5648e7d59000 2024-04-25T10:14:51.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.174+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 3 ==== osd_op_reply(33 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.174+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1800 con 0x5648e7d58400 2024-04-25T10:14:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.174+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.174+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890b800 con 0x5648e890bc00 2024-04-25T10:14:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.175+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 8 ==== osd_op_reply(35 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.175+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59c00 con 0x5648e7d59000 2024-04-25T10:14:51.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.177+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 85 out=49b] v81'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.177+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59400 con 0x5648e7d58400 2024-04-25T10:14:51.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.177+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.177+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1800 con 0x5648e7d58800 2024-04-25T10:14:51.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.178+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.3 v2:172.21.15.119:6816/1142971786 2 ==== osd_op_reply(38 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d58800 2024-04-25T10:14:51.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.178+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6c000 con 0x5648e88f0c00 2024-04-25T10:14:51.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.179+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.5 v2:172.21.15.132:6815/2978152964 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 85 out=49b] v81'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e88f0c00 2024-04-25T10:14:51.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.179+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6c400 con 0x5648e890bc00 2024-04-25T10:14:51.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.180+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 9 ==== osd_op_reply(40 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.180+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6c800 con 0x5648e7d58800 2024-04-25T10:14:51.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.181+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.3 v2:172.21.15.119:6816/1142971786 3 ==== osd_op_reply(41 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d58800 2024-04-25T10:14:51.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.181+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6d800 con 0x5648e7d58c00 2024-04-25T10:14:51.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.182+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.182+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6dc00 con 0x5648e7d59000 2024-04-25T10:14:51.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.183+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.183+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6cc00 con 0x5648e7d59000 2024-04-25T10:14:51.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.184+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.184+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1c00 con 0x5648e7d59000 2024-04-25T10:14:51.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.185+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 7 ==== osd_op_reply(45 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.185+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6d400 con 0x5648e7d58400 2024-04-25T10:14:51.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.186+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.186+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1800 con 0x5648e7d59000 2024-04-25T10:14:51.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.187+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 8 ==== osd_op_reply(47 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.187+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6d000 con 0x5648e7d58000 2024-04-25T10:14:51.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.188+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.0 v2:172.21.15.119:6808/2890625333 3 ==== osd_op_reply(48 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58000 2024-04-25T10:14:51.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.188+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59800 con 0x5648e88f1400 2024-04-25T10:14:51.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.189+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.189+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7a000 con 0x5648e88f1400 2024-04-25T10:14:51.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.190+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 6 ==== osd_op_reply(50 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.191+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59c00 con 0x5648e7d58000 2024-04-25T10:14:51.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.191+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.0 v2:172.21.15.119:6808/2890625333 4 ==== osd_op_reply(51 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58000 2024-04-25T10:14:51.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.192+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7a400 con 0x5648e7d58400 2024-04-25T10:14:51.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.192+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 9 ==== osd_op_reply(52 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.192+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890b400 con 0x5648e88f1400 2024-04-25T10:14:51.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.193+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.193+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7a800 con 0x5648e88f1400 2024-04-25T10:14:51.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.195+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.195+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890a400 con 0x5648e88f0c00 2024-04-25T10:14:51.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.196+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.5 v2:172.21.15.132:6815/2978152964 6 ==== osd_op_reply(55 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e88f0c00 2024-04-25T10:14:51.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.196+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7ac00 con 0x5648e7d58c00 2024-04-25T10:14:51.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.197+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 9 ==== osd_op_reply(56 [pgnls-filter start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.197+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890b800 con 0x5648e7d59000 2024-04-25T10:14:51.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.198+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.198+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7b000 con 0x5648e7d59000 2024-04-25T10:14:51.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.199+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 10 ==== osd_op_reply(58 [pgnls-filter start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.199+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 85 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6c400 con 0x5648e88f1400 2024-04-25T10:14:51.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.200+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 9 ==== osd_op_reply(59 [pgnls-filter start_epoch 85 out=81b] v85'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.200+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1000 con 0x5648e88f1400 2024-04-25T10:14:51.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.201+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.202+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6c800 con 0x5648e890bc00 2024-04-25T10:14:51.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.203+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 10 ==== osd_op_reply(61 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.203+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7b400 con 0x5648e7d58400 2024-04-25T10:14:51.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.204+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 10 ==== osd_op_reply(62 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.204+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6dc00 con 0x5648e7d59000 2024-04-25T10:14:51.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.205+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 11 ==== osd_op_reply(63 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.205+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7b800 con 0x5648e7d58c00 2024-04-25T10:14:51.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.206+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 10 ==== osd_op_reply(64 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.206+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6cc00 con 0x5648e7d58c00 2024-04-25T10:14:51.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.207+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 11 ==== osd_op_reply(65 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.207+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7bc00 con 0x5648e88f1400 2024-04-25T10:14:51.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.208+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 11 ==== osd_op_reply(66 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.208+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6c000 con 0x5648e88f0c00 2024-04-25T10:14:51.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.209+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.5 v2:172.21.15.132:6815/2978152964 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e88f0c00 2024-04-25T10:14:51.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.209+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7a000 con 0x5648e890bc00 2024-04-25T10:14:51.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.210+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.210+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6d400 con 0x5648e890bc00 2024-04-25T10:14:51.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.211+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.211+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7a800 con 0x5648e7d58000 2024-04-25T10:14:51.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.212+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.0 v2:172.21.15.119:6808/2890625333 5 ==== osd_op_reply(70 [pgnls-filter start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58000 2024-04-25T10:14:51.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.213+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6d000 con 0x5648e890bc00 2024-04-25T10:14:51.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.214+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 13 ==== osd_op_reply(71 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.214+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7ac00 con 0x5648e7d58400 2024-04-25T10:14:51.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.214+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.215+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1c00 con 0x5648e7d58c00 2024-04-25T10:14:51.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.215+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.215+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59800 con 0x5648e890bc00 2024-04-25T10:14:51.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.216+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 14 ==== osd_op_reply(74 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.217+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1800 con 0x5648e7d58c00 2024-04-25T10:14:51.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.217+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 13 ==== osd_op_reply(75 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.218+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59400 con 0x5648e88f0c00 2024-04-25T10:14:51.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.219+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.5 v2:172.21.15.132:6815/2978152964 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e88f0c00 2024-04-25T10:14:51.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.219+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890b400 con 0x5648e7d58400 2024-04-25T10:14:51.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.219+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 12 ==== osd_op_reply(77 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.219+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59c00 con 0x5648e7d58000 2024-04-25T10:14:51.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.220+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.0 v2:172.21.15.119:6808/2890625333 6 ==== osd_op_reply(78 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58000 2024-04-25T10:14:51.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.220+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890b800 con 0x5648e7d58c00 2024-04-25T10:14:51.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.221+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.221+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1000 con 0x5648e7d58c00 2024-04-25T10:14:51.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.222+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 15 ==== osd_op_reply(80 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.222+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890a400 con 0x5648e7d58400 2024-04-25T10:14:51.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.223+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 13 ==== osd_op_reply(81 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.223+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6d800 con 0x5648e7d58400 2024-04-25T10:14:51.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.224+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 14 ==== osd_op_reply(82 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.225+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7a400 con 0x5648e88f0c00 2024-04-25T10:14:51.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.226+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.5 v2:172.21.15.132:6815/2978152964 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e88f0c00 2024-04-25T10:14:51.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.226+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6c400 con 0x5648e7d59000 2024-04-25T10:14:51.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.227+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.227+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7b000 con 0x5648e88f1400 2024-04-25T10:14:51.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.228+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.228+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890b400 con 0x5648e7d58800 2024-04-25T10:14:51.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.229+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.3 v2:172.21.15.119:6816/1142971786 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d58800 2024-04-25T10:14:51.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.229+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7a000 con 0x5648e890bc00 2024-04-25T10:14:51.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.231+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 15 ==== osd_op_reply(87 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.231+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890a400 con 0x5648e88f1400 2024-04-25T10:14:51.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.232+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.232+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7b400 con 0x5648e7d58c00 2024-04-25T10:14:51.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.233+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.233+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59800 con 0x5648e7d59000 2024-04-25T10:14:51.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.234+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.234+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7a800 con 0x5648e7d58400 2024-04-25T10:14:51.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.235+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 15 ==== osd_op_reply(91 [pgnls-filter start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.235+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59400 con 0x5648e890bc00 2024-04-25T10:14:51.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.236+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 16 ==== osd_op_reply(92 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.236+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7ac00 con 0x5648e7d59000 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.238+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 14 ==== osd_op_reply(93 [pgnls-filter start_epoch 85 out=49b] v81'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.238+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59c00 con 0x5648e7d58400 2024-04-25T10:14:51.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.239+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 16 ==== osd_op_reply(94 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.239+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6c800 con 0x5648e7d58800 2024-04-25T10:14:51.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.240+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.3 v2:172.21.15.119:6816/1142971786 5 ==== osd_op_reply(95 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d58800 2024-04-25T10:14:51.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.240+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890b800 con 0x5648e88f0c00 2024-04-25T10:14:51.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.241+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.5 v2:172.21.15.132:6815/2978152964 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 85 out=49b] v81'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e88f0c00 2024-04-25T10:14:51.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.241+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6dc00 con 0x5648e890bc00 2024-04-25T10:14:51.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.242+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 17 ==== osd_op_reply(97 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.242+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890a400 con 0x5648e7d58800 2024-04-25T10:14:51.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.243+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.3 v2:172.21.15.119:6816/1142971786 6 ==== osd_op_reply(98 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d58800 2024-04-25T10:14:51.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.243+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6d400 con 0x5648e7d58c00 2024-04-25T10:14:51.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.244+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 17 ==== osd_op_reply(99 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.244+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7b800 con 0x5648e7d59000 2024-04-25T10:14:51.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.245+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 15 ==== osd_op_reply(100 [pgnls-filter start_epoch 85 out=49b] v81'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.245+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6d000 con 0x5648e7d59000 2024-04-25T10:14:51.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.246+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 16 ==== osd_op_reply(101 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.246+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7bc00 con 0x5648e7d59000 2024-04-25T10:14:51.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.247+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 17 ==== osd_op_reply(102 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.247+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6c400 con 0x5648e7d58400 2024-04-25T10:14:51.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.248+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 17 ==== osd_op_reply(103 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.249+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7b000 con 0x5648e7d59000 2024-04-25T10:14:51.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.250+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 18 ==== osd_op_reply(104 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.250+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6c000 con 0x5648e7d58000 2024-04-25T10:14:51.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.251+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.0 v2:172.21.15.119:6808/2890625333 7 ==== osd_op_reply(105 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58000 2024-04-25T10:14:51.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.251+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1c00 con 0x5648e88f1400 2024-04-25T10:14:51.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.252+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 14 ==== osd_op_reply(106 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.252+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7a000 con 0x5648e88f1400 2024-04-25T10:14:51.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.253+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.253+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1800 con 0x5648e7d58000 2024-04-25T10:14:51.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.254+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.0 v2:172.21.15.119:6808/2890625333 8 ==== osd_op_reply(108 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58000 2024-04-25T10:14:51.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.254+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7ac00 con 0x5648e7d58400 2024-04-25T10:14:51.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.255+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 18 ==== osd_op_reply(109 [pgnls-filter start_epoch 85 out=49b] v81'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.255+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1000 con 0x5648e88f1400 2024-04-25T10:14:51.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.256+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 16 ==== osd_op_reply(110 [pgnls-filter start_epoch 85 out=49b] v81'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.257+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7b800 con 0x5648e88f1400 2024-04-25T10:14:51.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.258+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 17 ==== osd_op_reply(111 [pgnls-filter start_epoch 85 out=49b] v81'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.258+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6cc00 con 0x5648e88f0c00 2024-04-25T10:14:51.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.259+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.5 v2:172.21.15.132:6815/2978152964 11 ==== osd_op_reply(112 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e88f0c00 2024-04-25T10:14:51.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.259+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890b400 con 0x5648e7d58c00 2024-04-25T10:14:51.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.260+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.2 v2:172.21.15.119:6824/1001762963 18 ==== osd_op_reply(113 [pgnls-filter start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e780 con 0x5648e7d58c00 2024-04-25T10:14:51.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.260+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7a400 con 0x5648e7d59000 2024-04-25T10:14:51.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.261+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 19 ==== osd_op_reply(114 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.261+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e890a400 con 0x5648e7d59000 2024-04-25T10:14:51.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.262+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 20 ==== osd_op_reply(115 [pgnls-filter start_epoch 85 out=49b] v81'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.262+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7a800 con 0x5648e88f1400 2024-04-25T10:14:51.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.264+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 18 ==== osd_op_reply(116 [pgnls-filter start_epoch 85 out=81b] v85'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.264+0000 7fa9f491e740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-25T10:14:51.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.264+0000 7fa9f491e740 10 datascan.operator(): handling object 10000000001.0 2024-04-25T10:14:51.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.264+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d90400 con 0x5648e88f1400 2024-04-25T10:14:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.266+0000 7fa9f2200640 1 -- 172.21.15.119:0/1589154042 <== osd.7 v2:172.21.15.132:6821/2727385616 19 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5648e874e780 con 0x5648e88f1400 2024-04-25T10:14:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.266+0000 7fa9f491e740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-25T10:14:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.266+0000 7fa9f491e740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-25T10:14:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.266+0000 7fa9f491e740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-25T10:14:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.266+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:118 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d90800 con 0x5648e7d58000 2024-04-25T10:14:51.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.267+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.0 v2:172.21.15.119:6808/2890625333 9 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+59 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58000 2024-04-25T10:14:51.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.267+0000 7fa9f491e740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-25T10:14:51.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.267+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d90c00 con 0x5648e890bc00 2024-04-25T10:14:51.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.268+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 18 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=581b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+581 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.268+0000 7fa9f491e740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-25T10:14:51.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.268+0000 7fa9f491e740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T10:14:51.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.268+0000 7fa9f491e740 20 datascan.get_frag_of: frag is * 2024-04-25T10:14:51.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.268+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d91000 con 0x5648e7d58000 2024-04-25T10:14:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.269+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.0 v2:172.21.15.119:6808/2890625333 10 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58000 2024-04-25T10:14:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.269+0000 7fa9f491e740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-25T10:14:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.269+0000 7fa9f491e740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-25T10:14:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.269+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d91400 con 0x5648e7d58000 2024-04-25T10:14:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.270+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.0 v2:172.21.15.119:6808/2890625333 11 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58000 2024-04-25T10:14:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.270+0000 7fa9f491e740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:14:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.270+0000 7fa9f491e740 20 datascan.read_dentry: file_head not found in result 2024-04-25T10:14:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.270+0000 7fa9f491e740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-25T10:14:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.270+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d91800 con 0x5648e7d58000 2024-04-25T10:14:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.274+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.0 v2:172.21.15.119:6808/2890625333 12 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v85'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58000 2024-04-25T10:14:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.275+0000 7fa9f491e740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:14:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.275+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:123 13.27 13:e4000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7b400 con 0x5648e7d58400 2024-04-25T10:14:51.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.275+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.1 v2:172.21.15.119:6800/298545098 19 ==== osd_op_reply(123 [pgnls-filter start_epoch 85 out=49b] v81'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e7d58400 2024-04-25T10:14:51.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.276+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:124 13.17 13:e8000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d7ac00 con 0x5648e88f0c00 2024-04-25T10:14:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.277+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.5 v2:172.21.15.132:6815/2978152964 12 ==== osd_op_reply(124 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e88f0c00 2024-04-25T10:14:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.277+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:125 13.37 13:ec000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6c800 con 0x5648e88f0c00 2024-04-25T10:14:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.278+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 <== osd.5 v2:172.21.15.132:6815/2978152964 13 ==== osd_op_reply(125 [pgnls-filter start_epoch 85 out=49b] v81'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e000 con 0x5648e88f0c00 2024-04-25T10:14:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.278+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:126 13.f 13:f0000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e88f1800 con 0x5648e890bc00 2024-04-25T10:14:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.279+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 19 ==== osd_op_reply(126 [pgnls-filter start_epoch 85 out=49b] v81'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.279+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:127 13.2f 13:f4000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6dc00 con 0x5648e7d59000 2024-04-25T10:14:51.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.280+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.4 v2:172.21.15.132:6800/1740646323 21 ==== osd_op_reply(127 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e7d59000 2024-04-25T10:14:51.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.280+0000 7fa9ebe00640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:128 13.1f 13:f8000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d6d800 con 0x5648e890bc00 2024-04-25T10:14:51.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.281+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 20 ==== osd_op_reply(128 [pgnls-filter start_epoch 85 out=49b] v81'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.281+0000 7fa9ec800640 1 -- 172.21.15.119:0/1589154042 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:129 13.3f 13:fc000000::::head [pgnls-filter start_epoch 85 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x5648e7d59800 con 0x5648e890bc00 2024-04-25T10:14:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.282+0000 7fa9f2c00640 1 -- 172.21.15.119:0/1589154042 <== osd.6 v2:172.21.15.132:6806/353653608 21 ==== osd_op_reply(129 [pgnls-filter start_epoch 85 out=49b] v81'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5648e874e500 con 0x5648e890bc00 2024-04-25T10:14:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.282+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x5648e7d58000 msgr2=0x5648e7d4ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.282+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x5648e7d58000 0x5648e7d4ec00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5648e7d58400 msgr2=0x5648e7d4db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5648e7d58400 0x5648e7d4db80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x5648e7d58c00 msgr2=0x5648e7d4e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x5648e7d58c00 0x5648e7d4e680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x5648e7d58800 msgr2=0x5648e7d4f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x5648e7d58800 0x5648e7d4f180 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5648e7d59000 msgr2=0x5648e7d4e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5648e7d59000 0x5648e7d4e100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5648e88f0c00 msgr2=0x5648e7a72c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5648e88f0c00 0x5648e7a72c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5648e890bc00 msgr2=0x5648e7a73180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5648e890bc00 0x5648e7a73180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5648e88f1400 msgr2=0x5648e7d4c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5648e88f1400 0x5648e7d4c000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 reap_dead start 2024-04-25T10:14:51.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 reap_dead start 2024-04-25T10:14:51.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f1800640 1 -- 172.21.15.119:0/1589154042 reap_dead start 2024-04-25T10:14:51.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x5648e890a800 msgr2=0x5648e7a73700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x5648e890a800 0x5648e7a73700 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x5648e87f1860 tx=0x5648e873a1e0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e88f0000 msgr2=0x5648e7a70b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.283+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e88f0000 0x5648e7a70b00 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x5648e88a1ad0 tx=0x5648e88fe120 comp rx=0 tx=0).stop 2024-04-25T10:14:51.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.284+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 shutdown_connections 2024-04-25T10:14:51.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.284+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x5648e890a800 0x5648e7a73700 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.284+0000 7fa9f491e740 1 --2- 172.21.15.119:0/1589154042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e88f0000 0x5648e7a70b00 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.284+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 >> 172.21.15.119:0/1589154042 conn(0x5648e879c400 msgr2=0x5648e874d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:51.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.284+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 shutdown_connections 2024-04-25T10:14:51.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.285+0000 7fa9f491e740 1 -- 172.21.15.119:0/1589154042 wait complete. 2024-04-25T10:14:51.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.285+0000 7fa9f491e740 1 -- 172.21.15.119:0/4278333754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e879c000 msgr2=0x5648e7a71080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.286+0000 7fa9f491e740 1 --2- 172.21.15.119:0/4278333754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e879c000 0x5648e7a71080 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x5648e87f17d0 tx=0x5648e873a120 comp rx=0 tx=0).stop 2024-04-25T10:14:51.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.286+0000 7fa9f491e740 1 -- 172.21.15.119:0/4278333754 shutdown_connections 2024-04-25T10:14:51.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.286+0000 7fa9f491e740 1 --2- 172.21.15.119:0/4278333754 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5648e879c800 0x5648e7a71600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.286+0000 7fa9f491e740 1 --2- 172.21.15.119:0/4278333754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5648e79d1c00 0x5648e7a71b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.286+0000 7fa9f491e740 1 --2- 172.21.15.119:0/4278333754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5648e879c000 0x5648e7a71080 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.286+0000 7fa9f491e740 1 -- 172.21.15.119:0/4278333754 >> 172.21.15.119:0/4278333754 conn(0x5648e79d1000 msgr2=0x5648e874d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:51.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.286+0000 7fa9f491e740 1 -- 172.21.15.119:0/4278333754 shutdown_connections 2024-04-25T10:14:51.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.286+0000 7fa9f491e740 1 -- 172.21.15.119:0/4278333754 wait complete. 2024-04-25T10:14:51.302 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.322589, result: 2024-04-25T10:14:51.302 DEBUG:teuthology.orchestra.run.smithi119:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T10:14:51.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.401+0000 7f0fc8a0f740 1 -- start start 2024-04-25T10:14:51.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.401+0000 7f0fc8a0f740 1 Processor -- start 2024-04-25T10:14:51.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.402+0000 7f0fc8a0f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169728c00 0x55c1697c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.402+0000 7f0fc8a0f740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c169729800 0x55c1697c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.402+0000 7f0fc8a0f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c169729c00 0x55c1697c9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.402+0000 7f0fc8a0f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16a5da180 con 0x55c169728c00 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.402+0000 7f0fc8a0f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16a5da300 con 0x55c169729c00 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.402+0000 7f0fc8a0f740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c16a5da480 con 0x55c169729800 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.403+0000 7f0fc5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169728c00 0x55c1697c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.403+0000 7f0fc5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169728c00 0x55c1697c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60518/0 (socket says 172.21.15.119:60518) 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.403+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2498050750 learned_addr learned my addr 172.21.15.119:0/2498050750 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.403+0000 7f0fc5400640 1 --2- 172.21.15.119:0/2498050750 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c169729800 0x55c1697c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.403+0000 7f0fc6800640 1 --2- 172.21.15.119:0/2498050750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c169729c00 0x55c1697c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.403+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2498050750 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c169729800 msgr2=0x55c1697c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.403+0000 7f0fc5e00640 1 --2- 172.21.15.119:0/2498050750 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c169729800 0x55c1697c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.403+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2498050750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c169729c00 msgr2=0x55c1697c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.403+0000 7f0fc5e00640 1 --2- 172.21.15.119:0/2498050750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c169729c00 0x55c1697c9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.403+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2498050750 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c169755680 con 0x55c169728c00 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.403+0000 7f0fc5e00640 1 --2- 172.21.15.119:0/2498050750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169728c00 0x55c1697c8580 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x55c16a537200 tx=0x55c16a5e4120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4d93128ab25ded server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.404+0000 7f0fc2c00640 1 -- 172.21.15.119:0/2498050750 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c16974e9c0 con 0x55c169728c00 2024-04-25T10:14:51.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.404+0000 7f0fc2c00640 1 -- 172.21.15.119:0/2498050750 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c169772e00 con 0x55c169728c00 2024-04-25T10:14:51.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.404+0000 7f0fc2c00640 1 -- 172.21.15.119:0/2498050750 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c16974e680 con 0x55c169728c00 2024-04-25T10:14:51.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.404+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2498050750 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c169755860 con 0x55c169728c00 2024-04-25T10:14:51.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.404+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2498050750 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55c1697552c0 con 0x55c169728c00 2024-04-25T10:14:51.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.404+0000 7f0fc8a0f740 4 waiting for MDS map... 2024-04-25T10:14:51.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.404+0000 7f0fc2c00640 1 -- 172.21.15.119:0/2498050750 <== mon.0 v2:172.21.15.119:3300/0 4 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x55c16a5f4200 con 0x55c169728c00 2024-04-25T10:14:51.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.404+0000 7f0fc2c00640 1 -- 172.21.15.119:0/2498050750 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 69) v1 ==== 2723+0+0 (secure 0 0 0) 0x55c169781180 con 0x55c169728c00 2024-04-25T10:14:51.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.404+0000 7f0fc8a0f740 4 Got MDS map 69 2024-04-25T10:14:51.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.404+0000 7f0fc8a0f740 4 datascan.main: Using metadata pool output 2024-04-25T10:14:51.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.404+0000 7f0fc8a0f740 4 datascan.main: connecting to RADOS... 2024-04-25T10:14:51.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.405+0000 7f0fc8a0f740 1 Processor -- start 2024-04-25T10:14:51.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.405+0000 7f0fc8a0f740 1 -- start start 2024-04-25T10:14:51.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.406+0000 7f0fc8a0f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c16a505c00 0x55c1697c9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.406+0000 7f0fc8a0f740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c16a640000 0x55c1697c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.406+0000 7f0fc8a0f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c16a640400 0x55c1697ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.406+0000 7f0fc8a0f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16a5da600 con 0x55c16a640400 2024-04-25T10:14:51.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.406+0000 7f0fc8a0f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16a5da780 con 0x55c16a505c00 2024-04-25T10:14:51.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.406+0000 7f0fc8a0f740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c16a5da900 con 0x55c16a640000 2024-04-25T10:14:51.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.406+0000 7f0fc5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c16a640400 0x55c1697ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.406+0000 7f0fc5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c16a640400 0x55c1697ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60530/0 (socket says 172.21.15.119:60530) 2024-04-25T10:14:51.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.406+0000 7f0fc5400640 1 -- 172.21.15.119:0/3176589095 learned_addr learned my addr 172.21.15.119:0/3176589095 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:51.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.407+0000 7f0fc5e00640 1 --2- 172.21.15.119:0/3176589095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c16a640000 0x55c1697c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.407+0000 7f0fc5400640 1 -- 172.21.15.119:0/3176589095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c16a640000 msgr2=0x55c1697c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:51.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.407+0000 7f0fc5400640 1 --2- 172.21.15.119:0/3176589095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c16a640000 0x55c1697c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.407+0000 7f0fc6800640 1 --2- 172.21.15.119:0/3176589095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c16a505c00 0x55c1697c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.407+0000 7f0fc5400640 1 -- 172.21.15.119:0/3176589095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c16a505c00 msgr2=0x55c1697c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:51.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.407+0000 7f0fc5400640 1 --2- 172.21.15.119:0/3176589095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c16a505c00 0x55c1697c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.407+0000 7f0fc5400640 1 -- 172.21.15.119:0/3176589095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1697554a0 con 0x55c16a640400 2024-04-25T10:14:51.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.407+0000 7f0fc5400640 1 --2- 172.21.15.119:0/3176589095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c16a640400 0x55c1697ca100 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x55c16a51c810 tx=0x55c16a650000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf29f6e2bb7a57e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.407+0000 7f0fc0400640 1 -- 172.21.15.119:0/3176589095 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c16974e9c0 con 0x55c16a640400 2024-04-25T10:14:51.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.407+0000 7f0fc0400640 1 -- 172.21.15.119:0/3176589095 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c169772540 con 0x55c16a640400 2024-04-25T10:14:51.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.407+0000 7f0fc0400640 1 -- 172.21.15.119:0/3176589095 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c16974e680 con 0x55c16a640400 2024-04-25T10:14:51.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.408+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/3176589095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c16a640400 msgr2=0x55c1697ca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.408+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/3176589095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c16a640400 0x55c1697ca100 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x55c16a51c810 tx=0x55c16a650000 comp rx=0 tx=0).stop 2024-04-25T10:14:51.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.408+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/3176589095 shutdown_connections 2024-04-25T10:14:51.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.408+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/3176589095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c16a640400 0x55c1697ca100 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.408+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/3176589095 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c16a640000 0x55c1697c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.408+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/3176589095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c16a505c00 0x55c1697c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.408+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/3176589095 >> 172.21.15.119:0/3176589095 conn(0x55c16a505800 msgr2=0x55c16a4a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:51.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.408+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/3176589095 shutdown_connections 2024-04-25T10:14:51.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.408+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/3176589095 wait complete. 2024-04-25T10:14:51.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.409+0000 7f0fc8a0f740 1 Processor -- start 2024-04-25T10:14:51.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.409+0000 7f0fc8a0f740 1 -- start start 2024-04-25T10:14:51.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.409+0000 7f0fc8a0f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c16a505800 0x55c1697cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.409+0000 7f0fc8a0f740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c16a641000 0x55c1697cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc8a0f740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c16a640c00 0x55c1697ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc8a0f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16a5dbc80 con 0x55c16a505800 2024-04-25T10:14:51.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc8a0f740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16a5dbb00 con 0x55c16a640c00 2024-04-25T10:14:51.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc8a0f740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c16a5db980 con 0x55c16a641000 2024-04-25T10:14:51.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc6800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c16a505800 0x55c1697cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc6800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c16a505800 0x55c1697cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60540/0 (socket says 172.21.15.119:60540) 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 learned_addr learned my addr 172.21.15.119:0/2881682649 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc5e00640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c16a641000 0x55c1697cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc5400640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c16a640c00 0x55c1697ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c16a641000 msgr2=0x55c1697cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc6800640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c16a641000 0x55c1697cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c16a640c00 msgr2=0x55c1697ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc6800640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c16a640c00 0x55c1697ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1697550e0 con 0x55c16a505800 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.410+0000 7f0fc5400640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c16a640c00 0x55c1697ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.411+0000 7f0fc6800640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c16a505800 0x55c1697cb180 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x55c16a51dad0 tx=0x55c16a650d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.411+0000 7f0fbe600640 1 -- 172.21.15.119:0/2881682649 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c16974e9c0 con 0x55c16a505800 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.411+0000 7f0fbe600640 1 -- 172.21.15.119:0/2881682649 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c169772a80 con 0x55c16a505800 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.411+0000 7f0fbe600640 1 -- 172.21.15.119:0/2881682649 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c16974e680 con 0x55c16a505800 2024-04-25T10:14:51.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.411+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c169755a40 con 0x55c16a505800 2024-04-25T10:14:51.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.411+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1697543c0 con 0x55c16a505800 2024-04-25T10:14:51.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.411+0000 7f0fc8a0f740 4 datascan.init: resolving metadata pool 12 2024-04-25T10:14:51.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.413+0000 7f0fbe600640 1 -- 172.21.15.119:0/2881682649 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55c16a656c00 con 0x55c16a505800 2024-04-25T10:14:51.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.413+0000 7f0fbe600640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x55c16a65ec00 0x55c1697cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.413+0000 7f0fbe600640 1 -- 172.21.15.119:0/2881682649 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x55c16a647a00 con 0x55c16a505800 2024-04-25T10:14:51.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.414+0000 7f0fc8a0f740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:14:51.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.414+0000 7f0fc8a0f740 4 datascan.main: resolving metadata pool 12 2024-04-25T10:14:51.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.414+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x55c16a640800 0x55c1697ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.414+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a640400 con 0x55c16a640800 2024-04-25T10:14:51.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.414+0000 7f0fc5e00640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x55c16a65ec00 0x55c1697cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.414+0000 7f0fc5e00640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x55c16a65ec00 0x55c1697cb700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55c16a537dd0 tx=0x55c16a5e41e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.414+0000 7f0fc5400640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x55c16a640800 0x55c1697ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.415+0000 7f0fc5400640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x55c16a640800 0x55c1697ca100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.417+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.4 v2:172.21.15.132:6800/1740646323 1 ==== osd_op_reply(1 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6000 con 0x55c16a640800 2024-04-25T10:14:51.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.417+0000 7f0fc5400640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x55c16a641c00 0x55c169a86000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.417+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a641400 con 0x55c16a641c00 2024-04-25T10:14:51.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.417+0000 7f0fc6800640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x55c16a641c00 0x55c169a86000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.417+0000 7f0fc6800640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x55c16a641c00 0x55c169a86000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.418+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 1 ==== osd_op_reply(2 [pgnls start_epoch 85 out=177b] v85'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.418+0000 7f0fc6800640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55c16a641400 0x55c169a86580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.418+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e000 con 0x55c16a641400 2024-04-25T10:14:51.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.418+0000 7f0fc5e00640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55c16a641400 0x55c169a86580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.419+0000 7f0fc5e00640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55c16a641400 0x55c169a86580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.419+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 <== osd.3 v2:172.21.15.119:6816/1142971786 1 ==== osd_op_reply(3 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6780 con 0x55c16a641400 2024-04-25T10:14:51.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.419+0000 7f0fc5e00640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55c169a8e000 0x55c169a86b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.419+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a65f400 con 0x55c169a8e000 2024-04-25T10:14:51.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.419+0000 7f0fc5400640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55c169a8e000 0x55c169a86b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.420+0000 7f0fc5400640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55c169a8e000 0x55c169a86b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.420+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== osd_op_reply(4 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e000 2024-04-25T10:14:51.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.420+0000 7f0fc5400640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55c16a65f400 0x55c169a87080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.420+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a641800 con 0x55c16a65f400 2024-04-25T10:14:51.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.421+0000 7f0fc6800640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55c16a65f400 0x55c169a87080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.422+0000 7f0fc6800640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55c16a65f400 0x55c169a87080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.423+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 1 ==== osd_op_reply(5 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.423+0000 7f0fc6800640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55c16a641800 0x55c169a87b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.423+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e400 con 0x55c16a641800 2024-04-25T10:14:51.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.423+0000 7f0fc5e00640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55c16a641800 0x55c169a87b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.424+0000 7f0fc5e00640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55c16a641800 0x55c169a87b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.425+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 <== osd.5 v2:172.21.15.132:6815/2978152964 1 ==== osd_op_reply(6 [pgnls start_epoch 85 out=149b] v84'28 uv28 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55c16a4a6780 con 0x55c16a641800 2024-04-25T10:14:51.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.425+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e400 con 0x55c16a641c00 2024-04-25T10:14:51.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.426+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 2 ==== osd_op_reply(7 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.426+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e400 con 0x55c16a641c00 2024-04-25T10:14:51.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.426+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 3 ==== osd_op_reply(8 [pgnls start_epoch 85 out=73b] v84'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.426+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e400 con 0x55c16a640800 2024-04-25T10:14:51.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.427+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.4 v2:172.21.15.132:6800/1740646323 2 ==== osd_op_reply(9 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6000 con 0x55c16a640800 2024-04-25T10:14:51.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.427+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e400 con 0x55c16a641400 2024-04-25T10:14:51.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.427+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 <== osd.3 v2:172.21.15.119:6816/1142971786 2 ==== osd_op_reply(10 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6780 con 0x55c16a641400 2024-04-25T10:14:51.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.427+0000 7f0fc5e00640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55c169a8e400 0x55c169a88680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.427+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a65f800 con 0x55c169a8e400 2024-04-25T10:14:51.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.428+0000 7f0fc5400640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55c169a8e400 0x55c169a88680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.428+0000 7f0fc5400640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55c169a8e400 0x55c169a88680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.429+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.2 v2:172.21.15.119:6824/1001762963 1 ==== osd_op_reply(11 [pgnls start_epoch 85 out=81b] v84'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e400 2024-04-25T10:14:51.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.429+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a65f800 con 0x55c16a640800 2024-04-25T10:14:51.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.429+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.4 v2:172.21.15.132:6800/1740646323 3 ==== osd_op_reply(12 [pgnls start_epoch 85 out=49b] v82'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6000 con 0x55c16a640800 2024-04-25T10:14:51.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.429+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a65f800 con 0x55c169a8e000 2024-04-25T10:14:51.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.430+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.1 v2:172.21.15.119:6800/298545098 2 ==== osd_op_reply(13 [pgnls start_epoch 85 out=77b] v84'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e000 2024-04-25T10:14:51.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.430+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a65f800 con 0x55c16a65f400 2024-04-25T10:14:51.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.431+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 2 ==== osd_op_reply(14 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.431+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a65f800 con 0x55c16a641c00 2024-04-25T10:14:51.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.431+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 4 ==== osd_op_reply(15 [pgnls start_epoch 85 out=73b] v84'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.431+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a65f800 con 0x55c16a640800 2024-04-25T10:14:51.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.432+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.4 v2:172.21.15.132:6800/1740646323 4 ==== osd_op_reply(16 [pgnls start_epoch 85 out=49b] v82'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6000 con 0x55c16a640800 2024-04-25T10:14:51.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.432+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a65f800 con 0x55c16a65f400 2024-04-25T10:14:51.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.433+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 3 ==== osd_op_reply(17 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.434+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a65f800 con 0x55c16a641800 2024-04-25T10:14:51.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.435+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 <== osd.5 v2:172.21.15.132:6815/2978152964 2 ==== osd_op_reply(18 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6780 con 0x55c16a641800 2024-04-25T10:14:51.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.435+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a65f800 con 0x55c169a8e400 2024-04-25T10:14:51.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.435+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.2 v2:172.21.15.119:6824/1001762963 2 ==== osd_op_reply(19 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e400 2024-04-25T10:14:51.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.435+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a65f800 con 0x55c16a641c00 2024-04-25T10:14:51.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.435+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 5 ==== osd_op_reply(20 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.435+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a65f800 con 0x55c16a65f400 2024-04-25T10:14:51.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.436+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 4 ==== osd_op_reply(21 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.436+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a65f800 con 0x55c16a65f400 2024-04-25T10:14:51.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.437+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 5 ==== osd_op_reply(22 [pgnls start_epoch 85 out=97b] v84'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.437+0000 7f0fc6800640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55c16a65f800 0x55c169a88100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.437+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e800 con 0x55c16a65f800 2024-04-25T10:14:51.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.438+0000 7f0fc6800640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55c16a65f800 0x55c169a88100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.439+0000 7f0fc6800640 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55c16a65f800 0x55c169a88100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.440+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.7 v2:172.21.15.132:6821/2727385616 1 ==== osd_op_reply(23 [pgnls start_epoch 85 out=74b] v80'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f800 2024-04-25T10:14:51.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.440+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e800 con 0x55c16a641c00 2024-04-25T10:14:51.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.440+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 6 ==== osd_op_reply(24 [pgnls start_epoch 85 out=73b] v84'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.440+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e800 con 0x55c16a641c00 2024-04-25T10:14:51.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.441+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 7 ==== osd_op_reply(25 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.441+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e800 con 0x55c169a8e000 2024-04-25T10:14:51.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.441+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.1 v2:172.21.15.119:6800/298545098 3 ==== osd_op_reply(26 [pgnls start_epoch 85 out=98b] v84'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e000 2024-04-25T10:14:51.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.441+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e800 con 0x55c16a641400 2024-04-25T10:14:51.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.442+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 <== osd.3 v2:172.21.15.119:6816/1142971786 3 ==== osd_op_reply(27 [pgnls start_epoch 85 out=76b] v84'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55c16a4a6780 con 0x55c16a641400 2024-04-25T10:14:51.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.442+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e800 con 0x55c16a640800 2024-04-25T10:14:51.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.443+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.4 v2:172.21.15.132:6800/1740646323 5 ==== osd_op_reply(28 [pgnls start_epoch 85 out=73b] v84'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c16a4a6000 con 0x55c16a640800 2024-04-25T10:14:51.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.443+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e800 con 0x55c169a8e000 2024-04-25T10:14:51.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.444+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.1 v2:172.21.15.119:6800/298545098 4 ==== osd_op_reply(29 [pgnls start_epoch 85 out=73b] v84'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e000 2024-04-25T10:14:51.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.444+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e800 con 0x55c16a65f400 2024-04-25T10:14:51.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.445+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 6 ==== osd_op_reply(30 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.445+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e800 con 0x55c16a65f400 2024-04-25T10:14:51.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.446+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 7 ==== osd_op_reply(31 [pgnls start_epoch 85 out=73b] v80'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.446+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e800 con 0x55c16a65f400 2024-04-25T10:14:51.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.447+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 8 ==== osd_op_reply(32 [pgnls start_epoch 85 out=101b] v84'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.447+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T10:14:51.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.447+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a640000 con 0x55c16a641c00 2024-04-25T10:14:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.447+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.448+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T10:14:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.448+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8ec00 con 0x55c16a641c00 2024-04-25T10:14:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.448+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.448+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T10:14:51.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.448+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8f000 con 0x55c16a641c00 2024-04-25T10:14:51.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.449+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.449+0000 7f0fc8a0f740 20 datascan.scan_links: injected ino detected 2024-04-25T10:14:51.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.449+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T10:14:51.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.449+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8f400 con 0x55c16a641c00 2024-04-25T10:14:51.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.449+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.449+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T10:14:51.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.449+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8f800 con 0x55c16a641c00 2024-04-25T10:14:51.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.450+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.450+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T10:14:51.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.450+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:14:51.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.450+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T10:14:51.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.450+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:14:51.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.450+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T10:14:51.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.450+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:14:51.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.450+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T10:14:51.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.450+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:38 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8fc00 con 0x55c16a641800 2024-04-25T10:14:51.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.451+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 <== osd.5 v2:172.21.15.132:6815/2978152964 3 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6780 con 0x55c16a641800 2024-04-25T10:14:51.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.451+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T10:14:51.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.451+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:39 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e800 con 0x55c16a641c00 2024-04-25T10:14:51.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.452+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.452+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-25T10:14:51.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.452+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:40 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8ec00 con 0x55c169a8e400 2024-04-25T10:14:51.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.452+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.2 v2:172.21.15.119:6824/1001762963 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e400 2024-04-25T10:14:51.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.452+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T10:14:51.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.452+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:14:51.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.452+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T10:14:51.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.452+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:41 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8f000 con 0x55c16a641c00 2024-04-25T10:14:51.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.453+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.453+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T10:14:51.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.453+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:42 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8f400 con 0x55c16a65f400 2024-04-25T10:14:51.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.454+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.454+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T10:14:51.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.454+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:14:51.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.454+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T10:14:51.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.454+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:14:51.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.454+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T10:14:51.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.454+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:43 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a640000 con 0x55c16a641c00 2024-04-25T10:14:51.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.455+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.455+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T10:14:51.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.455+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:14:51.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.455+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T10:14:51.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.455+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:44 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abe000 con 0x55c169a8e000 2024-04-25T10:14:51.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.456+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.1 v2:172.21.15.119:6800/298545098 5 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e000 2024-04-25T10:14:51.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.456+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T10:14:51.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.456+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:14:51.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.456+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-25T10:14:51.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.456+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T10:14:51.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.456+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T10:14:51.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.456+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:45 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abe400 con 0x55c169a8e000 2024-04-25T10:14:51.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.457+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.1 v2:172.21.15.119:6800/298545098 6 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e000 2024-04-25T10:14:51.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.457+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T10:14:51.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.457+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:14:51.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.457+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T10:14:51.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.457+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:14:51.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.457+0000 7f0fc8a0f740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T10:14:51.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.457+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:46 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abe800 con 0x55c16a65f400 2024-04-25T10:14:51.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.458+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 10 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1163 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.458+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:47 12.0 12:00000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a640800 2024-04-25T10:14:51.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.459+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.4 v2:172.21.15.132:6800/1740646323 6 ==== osd_op_reply(47 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6000 con 0x55c16a640800 2024-04-25T10:14:51.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.459+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:48 12.10 12:08000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a641c00 2024-04-25T10:14:51.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.461+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 16 ==== osd_op_reply(48 [pgnls start_epoch 85 out=177b] v85'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.461+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:49 12.8 12:10000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a641400 2024-04-25T10:14:51.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.461+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 <== osd.3 v2:172.21.15.119:6816/1142971786 4 ==== osd_op_reply(49 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6780 con 0x55c16a641400 2024-04-25T10:14:51.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.461+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:50 12.18 12:18000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c169a8e000 2024-04-25T10:14:51.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.462+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.1 v2:172.21.15.119:6800/298545098 7 ==== osd_op_reply(50 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e000 2024-04-25T10:14:51.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.462+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:51 12.4 12:20000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a65f400 2024-04-25T10:14:51.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.463+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 11 ==== osd_op_reply(51 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.463+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:52 12.14 12:28000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a641800 2024-04-25T10:14:51.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.464+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 <== osd.5 v2:172.21.15.132:6815/2978152964 4 ==== osd_op_reply(52 [pgnls start_epoch 85 out=149b] v84'28 uv28 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55c16a4a6780 con 0x55c16a641800 2024-04-25T10:14:51.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.465+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:53 12.c 12:30000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a641c00 2024-04-25T10:14:51.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.465+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 17 ==== osd_op_reply(53 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.465+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:54 12.1c 12:38000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a641c00 2024-04-25T10:14:51.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.466+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 18 ==== osd_op_reply(54 [pgnls start_epoch 85 out=73b] v84'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.466+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:55 12.2 12:40000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a640800 2024-04-25T10:14:51.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.467+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.4 v2:172.21.15.132:6800/1740646323 7 ==== osd_op_reply(55 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6000 con 0x55c16a640800 2024-04-25T10:14:51.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.468+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:56 12.12 12:48000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a641400 2024-04-25T10:14:51.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.468+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 <== osd.3 v2:172.21.15.119:6816/1142971786 5 ==== osd_op_reply(56 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6780 con 0x55c16a641400 2024-04-25T10:14:51.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.468+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:57 12.a 12:50000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c169a8e400 2024-04-25T10:14:51.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.469+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.2 v2:172.21.15.119:6824/1001762963 4 ==== osd_op_reply(57 [pgnls start_epoch 85 out=81b] v84'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e400 2024-04-25T10:14:51.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.469+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:58 12.1a 12:58000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a640800 2024-04-25T10:14:51.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.470+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.4 v2:172.21.15.132:6800/1740646323 8 ==== osd_op_reply(58 [pgnls start_epoch 85 out=49b] v82'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6000 con 0x55c16a640800 2024-04-25T10:14:51.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.470+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:59 12.6 12:60000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c169a8e000 2024-04-25T10:14:51.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.471+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.1 v2:172.21.15.119:6800/298545098 8 ==== osd_op_reply(59 [pgnls start_epoch 85 out=77b] v84'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e000 2024-04-25T10:14:51.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.471+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:60 12.16 12:68000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a65f400 2024-04-25T10:14:51.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.472+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 12 ==== osd_op_reply(60 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.472+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:61 12.e 12:70000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a641c00 2024-04-25T10:14:51.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.473+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 19 ==== osd_op_reply(61 [pgnls start_epoch 85 out=73b] v84'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.473+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:62 12.1e 12:78000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a640800 2024-04-25T10:14:51.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.474+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.4 v2:172.21.15.132:6800/1740646323 9 ==== osd_op_reply(62 [pgnls start_epoch 85 out=49b] v82'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6000 con 0x55c16a640800 2024-04-25T10:14:51.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.474+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:63 12.1 12:80000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a65f400 2024-04-25T10:14:51.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.475+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 13 ==== osd_op_reply(63 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.475+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:64 12.11 12:88000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a641800 2024-04-25T10:14:51.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.476+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 <== osd.5 v2:172.21.15.132:6815/2978152964 5 ==== osd_op_reply(64 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6780 con 0x55c16a641800 2024-04-25T10:14:51.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.476+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:65 12.9 12:90000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c169a8e400 2024-04-25T10:14:51.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.477+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.2 v2:172.21.15.119:6824/1001762963 5 ==== osd_op_reply(65 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e400 2024-04-25T10:14:51.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.477+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a641c00 2024-04-25T10:14:51.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.478+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 20 ==== osd_op_reply(66 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.478+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:67 12.5 12:a0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a65f400 2024-04-25T10:14:51.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.479+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 14 ==== osd_op_reply(67 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.479+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:68 12.15 12:a8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a65f400 2024-04-25T10:14:51.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.480+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 15 ==== osd_op_reply(68 [pgnls start_epoch 85 out=97b] v84'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.480+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:69 12.d 12:b0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a65f800 2024-04-25T10:14:51.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.481+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.7 v2:172.21.15.132:6821/2727385616 2 ==== osd_op_reply(69 [pgnls start_epoch 85 out=74b] v80'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f800 2024-04-25T10:14:51.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.481+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:70 12.1d 12:b8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a641c00 2024-04-25T10:14:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.482+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 21 ==== osd_op_reply(70 [pgnls start_epoch 85 out=73b] v84'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.482+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:71 12.3 12:c0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a641c00 2024-04-25T10:14:51.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.483+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 22 ==== osd_op_reply(71 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.483+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:72 12.13 12:c8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c169a8e000 2024-04-25T10:14:51.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.484+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.1 v2:172.21.15.119:6800/298545098 9 ==== osd_op_reply(72 [pgnls start_epoch 85 out=98b] v84'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e000 2024-04-25T10:14:51.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.484+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:73 12.b 12:d0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a641400 2024-04-25T10:14:51.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.485+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 <== osd.3 v2:172.21.15.119:6816/1142971786 6 ==== osd_op_reply(73 [pgnls start_epoch 85 out=76b] v84'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55c16a4a6780 con 0x55c16a641400 2024-04-25T10:14:51.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.485+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:74 12.1b 12:d8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a640800 2024-04-25T10:14:51.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.486+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.4 v2:172.21.15.132:6800/1740646323 10 ==== osd_op_reply(74 [pgnls start_epoch 85 out=73b] v84'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c16a4a6000 con 0x55c16a640800 2024-04-25T10:14:51.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.486+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:75 12.7 12:e0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c169a8e000 2024-04-25T10:14:51.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.487+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.1 v2:172.21.15.119:6800/298545098 10 ==== osd_op_reply(75 [pgnls start_epoch 85 out=73b] v84'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e000 2024-04-25T10:14:51.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.487+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:76 12.17 12:e8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a65f400 2024-04-25T10:14:51.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.488+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 16 ==== osd_op_reply(76 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.488+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:77 12.f 12:f0000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a65f400 2024-04-25T10:14:51.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.489+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 17 ==== osd_op_reply(77 [pgnls start_epoch 85 out=73b] v80'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.489+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:78 12.1f 12:f8000000::::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c16a65f400 2024-04-25T10:14:51.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.490+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 18 ==== osd_op_reply(78 [pgnls start_epoch 85 out=101b] v84'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.490+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T10:14:51.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.490+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:79 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abf000 con 0x55c16a641c00 2024-04-25T10:14:51.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.491+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 23 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.491+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T10:14:51.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.491+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:80 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abf400 con 0x55c16a641c00 2024-04-25T10:14:51.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.492+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 24 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.492+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T10:14:51.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.492+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:81 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abf800 con 0x55c16a641c00 2024-04-25T10:14:51.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.493+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 25 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.493+0000 7f0fc8a0f740 20 datascan.scan_links: injected ino detected 2024-04-25T10:14:51.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.493+0000 7f0fc8a0f740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-25T10:14:51.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.493+0000 7f0fc8a0f740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-25T10:14:51.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.493+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T10:14:51.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.493+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:82 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abfc00 con 0x55c16a641c00 2024-04-25T10:14:51.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.494+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 26 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.494+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T10:14:51.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.494+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:83 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abe800 con 0x55c16a641c00 2024-04-25T10:14:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.495+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 27 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.495+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T10:14:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.495+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:14:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.495+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T10:14:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.495+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:14:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.495+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T10:14:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.495+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:14:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.495+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T10:14:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.495+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:84 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c16a640000 con 0x55c16a641800 2024-04-25T10:14:51.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.496+0000 7f0fc5e00640 1 -- 172.21.15.119:0/2881682649 <== osd.5 v2:172.21.15.132:6815/2978152964 6 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6780 con 0x55c16a641800 2024-04-25T10:14:51.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.496+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T10:14:51.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.496+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:85 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8f800 con 0x55c16a641c00 2024-04-25T10:14:51.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.497+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 28 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.497+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-25T10:14:51.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.497+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:86 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8e800 con 0x55c169a8e400 2024-04-25T10:14:51.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.498+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.2 v2:172.21.15.119:6824/1001762963 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e400 2024-04-25T10:14:51.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.498+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T10:14:51.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.498+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:14:51.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.498+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T10:14:51.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.498+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:87 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8f000 con 0x55c16a641c00 2024-04-25T10:14:51.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.499+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 29 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.499+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T10:14:51.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.499+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:88 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8f400 con 0x55c16a65f400 2024-04-25T10:14:51.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.501+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 19 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.501+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T10:14:51.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.501+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:14:51.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.501+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T10:14:51.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.501+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:14:51.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.501+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T10:14:51.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.501+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8f800 con 0x55c16a641c00 2024-04-25T10:14:51.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.501+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 30 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.501+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T10:14:51.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.501+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:14:51.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.501+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T10:14:51.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.501+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:90 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c169a8e000 2024-04-25T10:14:51.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.503+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.1 v2:172.21.15.119:6800/298545098 11 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e000 2024-04-25T10:14:51.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.503+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T10:14:51.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.503+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:14:51.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.503+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-25T10:14:51.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.503+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T10:14:51.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.503+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T10:14:51.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.503+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:91 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abf000 con 0x55c169a8e000 2024-04-25T10:14:51.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.504+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.1 v2:172.21.15.119:6800/298545098 12 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e000 2024-04-25T10:14:51.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.504+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T10:14:51.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.504+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:14:51.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.504+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T10:14:51.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.504+0000 7f0fc8a0f740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:14:51.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.504+0000 7f0fc8a0f740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T10:14:51.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.504+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:92 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abf400 con 0x55c16a65f400 2024-04-25T10:14:51.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.505+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.6 v2:172.21.15.132:6806/353653608 20 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1163 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a65f400 2024-04-25T10:14:51.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.505+0000 7f0fc8a0f740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-25T10:14:51.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.506+0000 7f0fc8a0f740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T10:14:51.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.506+0000 7f0fc8a0f740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-25T10:14:51.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.506+0000 7f0fc8a0f740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-25T10:14:51.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.506+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:93 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abf800 con 0x55c16a641c00 2024-04-25T10:14:51.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.506+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 31 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.506+0000 7f0fc8a0f740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:14:51.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.507+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:94 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abfc00 con 0x55c16a641c00 2024-04-25T10:14:51.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.511+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 32 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v85'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.511+0000 7f0fc8a0f740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:14:51.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.511+0000 7f0fc8a0f740 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T10:14:51.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.511+0000 7f0fc8a0f740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-25T10:14:51.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.511+0000 7f0fc8a0f740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-25T10:14:51.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.511+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:95 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abe800 con 0x55c16a641c00 2024-04-25T10:14:51.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.512+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 33 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.512+0000 7f0fc8a0f740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:14:51.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.512+0000 7f0fc8a0f740 20 datascan.scan_links: first is now 2 2024-04-25T10:14:51.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.512+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:96 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abe000 con 0x55c16a641c00 2024-04-25T10:14:51.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.515+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.0 v2:172.21.15.119:6808/2890625333 34 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v85'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c16a4a6500 con 0x55c16a641c00 2024-04-25T10:14:51.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.515+0000 7f0fc8a0f740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:14:51.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.515+0000 7f0fc8a0f740 10 datascan.scan_links: updating inotable 2024-04-25T10:14:51.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.515+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abe400 con 0x55c169a8e000 2024-04-25T10:14:51.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.516+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.1 v2:172.21.15.119:6800/298545098 13 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e000 2024-04-25T10:14:51.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.516+0000 7f0fc8a0f740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-25T10:14:51.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.516+0000 7f0fc8a0f740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-25T10:14:51.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.516+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169abec00 con 0x55c169a8e000 2024-04-25T10:14:51.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.521+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 <== osd.1 v2:172.21.15.119:6800/298545098 14 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v85'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55c16a4a6000 con 0x55c169a8e000 2024-04-25T10:14:51.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.521+0000 7f0fc8a0f740 10 datascan.scan_links: updating snaptable 2024-04-25T10:14:51.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.521+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:99 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85) v9 -- 0x55c169a8ec00 con 0x55c16a65f800 2024-04-25T10:14:51.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.522+0000 7f0fc6800640 1 -- 172.21.15.119:0/2881682649 <== osd.7 v2:172.21.15.132:6821/2727385616 3 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55c16a4a6280 con 0x55c16a65f800 2024-04-25T10:14:51.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.522+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x55c16a641c00 msgr2=0x55c169a86000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.522+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x55c16a641c00 0x55c169a86000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.522+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55c169a8e000 msgr2=0x55c169a86b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.522+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55c169a8e000 0x55c169a86b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55c169a8e400 msgr2=0x55c169a88680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55c169a8e400 0x55c169a88680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55c16a641400 msgr2=0x55c169a86580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55c16a641400 0x55c169a86580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x55c16a640800 msgr2=0x55c1697ca100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x55c16a640800 0x55c1697ca100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55c16a641800 msgr2=0x55c169a87b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55c16a641800 0x55c169a87b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55c16a65f400 msgr2=0x55c169a87080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55c16a65f400 0x55c169a87080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55c16a65f800 msgr2=0x55c169a88100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 reap_dead start 2024-04-25T10:14:51.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55c16a65f800 0x55c169a88100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 reap_dead start 2024-04-25T10:14:51.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 reap_dead start 2024-04-25T10:14:51.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc5400640 1 -- 172.21.15.119:0/2881682649 reap_dead start 2024-04-25T10:14:51.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x55c16a65ec00 msgr2=0x55c1697cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x55c16a65ec00 0x55c1697cb700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55c16a537dd0 tx=0x55c16a5e41e0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c16a505800 msgr2=0x55c1697cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.523+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c16a505800 0x55c1697cb180 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x55c16a51dad0 tx=0x55c16a650d20 comp rx=0 tx=0).stop 2024-04-25T10:14:51.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.524+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 shutdown_connections 2024-04-25T10:14:51.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.524+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x55c16a65ec00 0x55c1697cb700 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.524+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2881682649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c16a505800 0x55c1697cb180 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.524+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 >> 172.21.15.119:0/2881682649 conn(0x55c16a505c00 msgr2=0x55c16a4a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:51.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.524+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 shutdown_connections 2024-04-25T10:14:51.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.524+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2881682649 wait complete. 2024-04-25T10:14:51.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.525+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2498050750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169728c00 msgr2=0x55c1697c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.525+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2498050750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169728c00 0x55c1697c8580 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x55c16a537200 tx=0x55c16a5e4120 comp rx=0 tx=0).stop 2024-04-25T10:14:51.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.525+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2498050750 shutdown_connections 2024-04-25T10:14:51.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.525+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2498050750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c169729c00 0x55c1697c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.525+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2498050750 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c169729800 0x55c1697c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.525+0000 7f0fc8a0f740 1 --2- 172.21.15.119:0/2498050750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c169728c00 0x55c1697c8580 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.525+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2498050750 >> 172.21.15.119:0/2498050750 conn(0x55c169729000 msgr2=0x55c16a4a4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:51.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.526+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2498050750 shutdown_connections 2024-04-25T10:14:51.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.526+0000 7f0fc8a0f740 1 -- 172.21.15.119:0/2498050750 wait complete. 2024-04-25T10:14:51.539 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.237116, result: 2024-04-25T10:14:51.540 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:14:51.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.742+0000 7fc56be00640 1 Processor -- start 2024-04-25T10:14:51.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.742+0000 7fc56be00640 1 -- start start 2024-04-25T10:14:51.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.744+0000 7fc56be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc56c15b7e0 0x7fc56c15e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.744+0000 7fc56be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc56c0ce860 0x7fc56c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.745+0000 7fc56be00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56c0b5710 con 0x7fc56c0d00a0 2024-04-25T10:14:51.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.745+0000 7fc56be00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c0b59f0 con 0x7fc56c15b7e0 2024-04-25T10:14:51.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.745+0000 7fc56be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c0b5cd0 con 0x7fc56c0ce860 2024-04-25T10:14:51.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.745+0000 7fc56b400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc56c0d00a0 0x7fc56c0d0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36150/0 (socket says 172.21.15.119:36150) 2024-04-25T10:14:51.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.745+0000 7fc56b400640 1 -- 172.21.15.119:0/3701641819 learned_addr learned my addr 172.21.15.119:0/3701641819 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:14:51.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.745+0000 7fc56aa00640 1 --2- 172.21.15.119:0/3701641819 >> v2:172.21.15.119:3300/0 conn(0x7fc56c15b7e0 0x7fc56c15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.745+0000 7fc56a000640 1 --2- 172.21.15.119:0/3701641819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc56c0ce860 0x7fc56c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.745+0000 7fc569600640 1 -- 172.21.15.119:0/3701641819 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480721864 0 0) 0x7fc56c0b5710 con 0x7fc56c0d00a0 2024-04-25T10:14:51.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.746+0000 7fc569600640 1 -- 172.21.15.119:0/3701641819 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc550003520 con 0x7fc56c0d00a0 2024-04-25T10:14:51.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.746+0000 7fc56aa00640 1 -- 172.21.15.119:0/3701641819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc56c0ce860 msgr2=0x7fc56c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:51.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.746+0000 7fc56aa00640 1 --2- 172.21.15.119:0/3701641819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc56c0ce860 0x7fc56c0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.746+0000 7fc56aa00640 1 -- 172.21.15.119:0/3701641819 >> v1:172.21.15.119:6789/0 conn(0x7fc56c0d00a0 legacy=0x7fc56c0d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.746+0000 7fc56aa00640 1 -- 172.21.15.119:0/3701641819 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c0b5fb0 con 0x7fc56c15b7e0 2024-04-25T10:14:51.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.747+0000 7fc56aa00640 1 --2- 172.21.15.119:0/3701641819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc56c15b7e0 0x7fc56c15e000 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0097a0 tx=0x7fc55c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a792339c40b49fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.747+0000 7fc569600640 1 -- 172.21.15.119:0/3701641819 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c037070 con 0x7fc56c15b7e0 2024-04-25T10:14:51.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.747+0000 7fc569600640 1 -- 172.21.15.119:0/3701641819 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55c02adb0 con 0x7fc56c15b7e0 2024-04-25T10:14:51.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.747+0000 7fc569600640 1 -- 172.21.15.119:0/3701641819 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c032660 con 0x7fc56c15b7e0 2024-04-25T10:14:51.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.748+0000 7fc56be00640 1 -- 172.21.15.119:0/3701641819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc56c15b7e0 msgr2=0x7fc56c15e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.748+0000 7fc56be00640 1 --2- 172.21.15.119:0/3701641819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc56c15b7e0 0x7fc56c15e000 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0097a0 tx=0x7fc55c02a060 comp rx=0 tx=0).stop 2024-04-25T10:14:51.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.748+0000 7fc56be00640 1 -- 172.21.15.119:0/3701641819 shutdown_connections 2024-04-25T10:14:51.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.748+0000 7fc56be00640 1 --2- 172.21.15.119:0/3701641819 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc56c0ce860 0x7fc56c0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.748+0000 7fc56be00640 1 --2- 172.21.15.119:0/3701641819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc56c15b7e0 0x7fc56c15e000 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.748+0000 7fc56be00640 1 -- 172.21.15.119:0/3701641819 >> 172.21.15.119:0/3701641819 conn(0x7fc56c0722d0 msgr2=0x7fc56c0726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:51.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.748+0000 7fc56be00640 1 -- 172.21.15.119:0/3701641819 shutdown_connections 2024-04-25T10:14:51.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.748+0000 7fc56be00640 1 -- 172.21.15.119:0/3701641819 wait complete. 2024-04-25T10:14:51.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.750+0000 7fc56be00640 1 Processor -- start 2024-04-25T10:14:51.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.750+0000 7fc56be00640 1 -- start start 2024-04-25T10:14:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.751+0000 7fc56be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc56c0ce860 0x7fc56c151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.751+0000 7fc56be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc56c0d00a0 0x7fc56c1596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.751+0000 7fc56be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc56c15b7e0 0x7fc56c157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.751+0000 7fc56be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c161960 con 0x7fc56c15b7e0 2024-04-25T10:14:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.751+0000 7fc56be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c0b53c0 con 0x7fc56c0ce860 2024-04-25T10:14:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.751+0000 7fc56be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c161ae0 con 0x7fc56c0d00a0 2024-04-25T10:14:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.752+0000 7fc56b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc56c15b7e0 0x7fc56c157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.752+0000 7fc56b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc56c15b7e0 0x7fc56c157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60572/0 (socket says 172.21.15.119:60572) 2024-04-25T10:14:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.752+0000 7fc56b400640 1 -- 172.21.15.119:0/587728876 learned_addr learned my addr 172.21.15.119:0/587728876 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.752+0000 7fc56a000640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc56c0d00a0 0x7fc56c1596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.752+0000 7fc56aa00640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc56c0ce860 0x7fc56c151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.752+0000 7fc56b400640 1 -- 172.21.15.119:0/587728876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc56c0d00a0 msgr2=0x7fc56c1596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.752+0000 7fc56b400640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc56c0d00a0 0x7fc56c1596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.752+0000 7fc56b400640 1 -- 172.21.15.119:0/587728876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc56c0ce860 msgr2=0x7fc56c151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.752+0000 7fc56b400640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc56c0ce860 0x7fc56c151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.752+0000 7fc56b400640 1 -- 172.21.15.119:0/587728876 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c158630 con 0x7fc56c15b7e0 2024-04-25T10:14:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.752+0000 7fc56a000640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc56c0d00a0 0x7fc56c1596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.752+0000 7fc56aa00640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc56c0ce860 0x7fc56c151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.752+0000 7fc56b400640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc56c15b7e0 0x7fc56c157d30 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc56400cbb0 tx=0x7fc56400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.753+0000 7fc563400640 1 -- 172.21.15.119:0/587728876 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc564017070 con 0x7fc56c15b7e0 2024-04-25T10:14:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.753+0000 7fc563400640 1 -- 172.21.15.119:0/587728876 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56400ad40 con 0x7fc56c15b7e0 2024-04-25T10:14:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.753+0000 7fc563400640 1 -- 172.21.15.119:0/587728876 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc564012600 con 0x7fc56c15b7e0 2024-04-25T10:14:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.753+0000 7fc56be00640 1 -- 172.21.15.119:0/587728876 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c158910 con 0x7fc56c15b7e0 2024-04-25T10:14:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.753+0000 7fc56be00640 1 -- 172.21.15.119:0/587728876 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c158ed0 con 0x7fc56c15b7e0 2024-04-25T10:14:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.754+0000 7fc56be00640 1 -- 172.21.15.119:0/587728876 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56c0749b0 con 0x7fc56c15b7e0 2024-04-25T10:14:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.755+0000 7fc563400640 1 -- 172.21.15.119:0/587728876 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc56401a070 con 0x7fc56c15b7e0 2024-04-25T10:14:51.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.756+0000 7fc563400640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc54c06b500 0x7fc54c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.757+0000 7fc563400640 1 -- 172.21.15.119:0/587728876 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fc56408a0c0 con 0x7fc56c15b7e0 2024-04-25T10:14:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.757+0000 7fc56aa00640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc54c06b500 0x7fc54c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.757+0000 7fc56aa00640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc54c06b500 0x7fc54c06d9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc55c034000 tx=0x7fc55c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.761+0000 7fc563400640 1 -- 172.21.15.119:0/587728876 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5640586f0 con 0x7fc56c15b7e0 2024-04-25T10:14:52.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:51.998+0000 7fc56be00640 1 -- 172.21.15.119:0/587728876 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fc56c0d27d0 con 0x7fc56c15b7e0 2024-04-25T10:14:52.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.738+0000 7fc563400640 1 -- 172.21.15.119:0/587728876 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v71) v1 ==== 162+0+0 (secure 0 0 0) 0x7fc56405c790 con 0x7fc56c15b7e0 2024-04-25T10:14:52.743 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:14:52.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.743+0000 7fc56be00640 1 -- 172.21.15.119:0/587728876 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc54c06b500 msgr2=0x7fc54c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.743+0000 7fc56be00640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc54c06b500 0x7fc54c06d9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc55c034000 tx=0x7fc55c02e000 comp rx=0 tx=0).stop 2024-04-25T10:14:52.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.743+0000 7fc56be00640 1 -- 172.21.15.119:0/587728876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc56c15b7e0 msgr2=0x7fc56c157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.743+0000 7fc56be00640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc56c15b7e0 0x7fc56c157d30 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc56400cbb0 tx=0x7fc56400a430 comp rx=0 tx=0).stop 2024-04-25T10:14:52.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.743+0000 7fc56be00640 1 -- 172.21.15.119:0/587728876 shutdown_connections 2024-04-25T10:14:52.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.743+0000 7fc56be00640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc54c06b500 0x7fc54c06d9c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.743+0000 7fc56be00640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc56c15b7e0 0x7fc56c157d30 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.743+0000 7fc56be00640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc56c0d00a0 0x7fc56c1596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.743+0000 7fc56be00640 1 --2- 172.21.15.119:0/587728876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc56c0ce860 0x7fc56c151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.743+0000 7fc56be00640 1 -- 172.21.15.119:0/587728876 >> 172.21.15.119:0/587728876 conn(0x7fc56c0722d0 msgr2=0x7fc56c15bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.744+0000 7fc56be00640 1 -- 172.21.15.119:0/587728876 shutdown_connections 2024-04-25T10:14:52.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.744+0000 7fc56be00640 1 -- 172.21.15.119:0/587728876 wait complete. 2024-04-25T10:14:52.770 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:52.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.943+0000 7f7940c00640 1 Processor -- start 2024-04-25T10:14:52.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.943+0000 7f7940c00640 1 -- start start 2024-04-25T10:14:52.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.944+0000 7f7940c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f793c0ae9a0 0x7f793c0cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.944+0000 7f7940c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f793c071620 0x7f793c15e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.944+0000 7f7940c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793c071ab0 con 0x7f793c071190 2024-04-25T10:14:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.944+0000 7f7940c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793c06d550 con 0x7f793c0ae9a0 2024-04-25T10:14:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.944+0000 7f7940c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c06d830 con 0x7f793c071620 2024-04-25T10:14:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.945+0000 7f7939e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f793c0ae9a0 0x7f793c0cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.945+0000 7f7939e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f793c0ae9a0 0x7f793c0cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60580/0 (socket says 172.21.15.119:60580) 2024-04-25T10:14:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.945+0000 7f7939e00640 1 -- 172.21.15.119:0/3870104353 learned_addr learned my addr 172.21.15.119:0/3870104353 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:52.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.945+0000 7f7939e00640 1 -- 172.21.15.119:0/3870104353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f793c071620 msgr2=0x7f793c15e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.945+0000 7f793a800640 1 --2- 172.21.15.119:0/3870104353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f793c071620 0x7f793c15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.945+0000 7f7939e00640 1 --2- 172.21.15.119:0/3870104353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f793c071620 0x7f793c15e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.945+0000 7f7939e00640 1 -- 172.21.15.119:0/3870104353 >> v1:172.21.15.132:6789/0 conn(0x7f793c071190 legacy=0x7f793c15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.945+0000 7f7939e00640 1 -- 172.21.15.119:0/3870104353 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c06db10 con 0x7f793c0ae9a0 2024-04-25T10:14:52.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.945+0000 7f793a800640 1 --2- 172.21.15.119:0/3870104353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f793c071620 0x7f793c15e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:52.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.946+0000 7f7939e00640 1 --2- 172.21.15.119:0/3870104353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f793c0ae9a0 0x7f793c0cf580 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f792c0098a0 tx=0x7f792c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebaf6024df02e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.946+0000 7f7939400640 1 -- 172.21.15.119:0/3870104353 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c037070 con 0x7f793c0ae9a0 2024-04-25T10:14:52.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.947+0000 7f7939400640 1 -- 172.21.15.119:0/3870104353 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c032440 con 0x7f793c0ae9a0 2024-04-25T10:14:52.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.947+0000 7f7939400640 1 -- 172.21.15.119:0/3870104353 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c032810 con 0x7f793c0ae9a0 2024-04-25T10:14:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.947+0000 7f7940c00640 1 -- 172.21.15.119:0/3870104353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f793c0ae9a0 msgr2=0x7f793c0cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.947+0000 7f7940c00640 1 --2- 172.21.15.119:0/3870104353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f793c0ae9a0 0x7f793c0cf580 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f792c0098a0 tx=0x7f792c02a250 comp rx=0 tx=0).stop 2024-04-25T10:14:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.947+0000 7f7940c00640 1 -- 172.21.15.119:0/3870104353 shutdown_connections 2024-04-25T10:14:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.947+0000 7f7940c00640 1 --2- 172.21.15.119:0/3870104353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f793c071620 0x7f793c15e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.947+0000 7f7940c00640 1 --2- 172.21.15.119:0/3870104353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f793c0ae9a0 0x7f793c0cf580 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.947+0000 7f7940c00640 1 -- 172.21.15.119:0/3870104353 >> 172.21.15.119:0/3870104353 conn(0x7f793c06aba0 msgr2=0x7f793c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.947+0000 7f7940c00640 1 -- 172.21.15.119:0/3870104353 shutdown_connections 2024-04-25T10:14:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.947+0000 7f7940c00640 1 -- 172.21.15.119:0/3870104353 wait complete. 2024-04-25T10:14:52.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.949+0000 7f7940c00640 1 Processor -- start 2024-04-25T10:14:52.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.949+0000 7f7940c00640 1 -- start start 2024-04-25T10:14:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.950+0000 7f7940c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f793c071190 0x7f793c151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.950+0000 7f7940c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f793c071620 0x7f793c0cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.950+0000 7f7940c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f793c0ae9a0 0x7f793c153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.950+0000 7f7940c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c06e8b0 con 0x7f793c071190 2024-04-25T10:14:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.950+0000 7f7940c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c06e0d0 con 0x7f793c0ae9a0 2024-04-25T10:14:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.950+0000 7f7940c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c06d1b0 con 0x7f793c071620 2024-04-25T10:14:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.950+0000 7f7939e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f793c071190 0x7f793c151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.950+0000 7f7939e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f793c071190 0x7f793c151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60582/0 (socket says 172.21.15.119:60582) 2024-04-25T10:14:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.950+0000 7f7939e00640 1 -- 172.21.15.119:0/2192392668 learned_addr learned my addr 172.21.15.119:0/2192392668 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.951+0000 7f7939e00640 1 -- 172.21.15.119:0/2192392668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f793c071620 msgr2=0x7f793c0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.951+0000 7f793a800640 1 --2- 172.21.15.119:0/2192392668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f793c0ae9a0 0x7f793c153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.951+0000 7f7931400640 1 --2- 172.21.15.119:0/2192392668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f793c071620 0x7f793c0cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.951+0000 7f7939e00640 1 --2- 172.21.15.119:0/2192392668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f793c071620 0x7f793c0cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.951+0000 7f7939e00640 1 -- 172.21.15.119:0/2192392668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f793c0ae9a0 msgr2=0x7f793c153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.951+0000 7f7939e00640 1 --2- 172.21.15.119:0/2192392668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f793c0ae9a0 0x7f793c153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.951+0000 7f7939e00640 1 -- 172.21.15.119:0/2192392668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c151b10 con 0x7f793c071190 2024-04-25T10:14:52.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.951+0000 7f7939e00640 1 --2- 172.21.15.119:0/2192392668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f793c071190 0x7f793c151480 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f792c000c00 tx=0x7f792c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.951+0000 7f7932a00640 1 -- 172.21.15.119:0/2192392668 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c037040 con 0x7f793c071190 2024-04-25T10:14:52.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.951+0000 7f7932a00640 1 -- 172.21.15.119:0/2192392668 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c004250 con 0x7f793c071190 2024-04-25T10:14:52.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.951+0000 7f7932a00640 1 -- 172.21.15.119:0/2192392668 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c030770 con 0x7f793c071190 2024-04-25T10:14:52.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.951+0000 7f7940c00640 1 -- 172.21.15.119:0/2192392668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c151df0 con 0x7f793c071190 2024-04-25T10:14:52.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.952+0000 7f7940c00640 1 -- 172.21.15.119:0/2192392668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c1523b0 con 0x7f793c071190 2024-04-25T10:14:52.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.953+0000 7f7940c00640 1 -- 172.21.15.119:0/2192392668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c0749b0 con 0x7f793c071190 2024-04-25T10:14:52.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.954+0000 7f7932a00640 1 -- 172.21.15.119:0/2192392668 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f792c030910 con 0x7f793c071190 2024-04-25T10:14:52.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.955+0000 7f7932a00640 1 --2- 172.21.15.119:0/2192392668 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f791806b4d0 0x7f791806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.955+0000 7f7932a00640 1 -- 172.21.15.119:0/2192392668 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f792c0ab2b0 con 0x7f793c071190 2024-04-25T10:14:52.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.955+0000 7f7931400640 1 --2- 172.21.15.119:0/2192392668 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f791806b4d0 0x7f791806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.956+0000 7f7931400640 1 --2- 172.21.15.119:0/2192392668 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f791806b4d0 0x7f791806d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f793c0ce2f0 tx=0x7f7924009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:52.959+0000 7f7932a00640 1 -- 172.21.15.119:0/2192392668 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f792c07a0e0 con 0x7f793c071190 2024-04-25T10:14:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:53.192+0000 7f7940c00640 1 -- 172.21.15.119:0/2192392668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f793c0720f0 con 0x7f793c071190 2024-04-25T10:14:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:53.193+0000 7f7932a00640 1 -- 172.21.15.119:0/2192392668 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f792c0799e0 con 0x7f793c071190 2024-04-25T10:14:53.197 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:14:53.197 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:14:52.733476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7668":{"gid":7668,"name":"a","rank":0,"incarnation":71,"state":"up:replay","state_seq":1,"addr":"172.21.15.119:6837/2043821095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2043821095},{"type":"v1","addr":"172.21.15.119:6837","nonce":2043821095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:14:53.198 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 71 2024-04-25T10:14:53.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:53.196+0000 7f7940c00640 1 -- 172.21.15.119:0/2192392668 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f791806b4d0 msgr2=0x7f791806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:53.196+0000 7f7940c00640 1 --2- 172.21.15.119:0/2192392668 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f791806b4d0 0x7f791806d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f793c0ce2f0 tx=0x7f7924009040 comp rx=0 tx=0).stop 2024-04-25T10:14:53.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:53.196+0000 7f7940c00640 1 -- 172.21.15.119:0/2192392668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f793c071190 msgr2=0x7f793c151480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:53.196+0000 7f7940c00640 1 --2- 172.21.15.119:0/2192392668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f793c071190 0x7f793c151480 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f792c000c00 tx=0x7f792c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:53.196+0000 7f7940c00640 1 -- 172.21.15.119:0/2192392668 shutdown_connections 2024-04-25T10:14:53.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:53.196+0000 7f7940c00640 1 --2- 172.21.15.119:0/2192392668 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f791806b4d0 0x7f791806d990 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:53.196+0000 7f7940c00640 1 --2- 172.21.15.119:0/2192392668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f793c0ae9a0 0x7f793c153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:53.196+0000 7f7940c00640 1 --2- 172.21.15.119:0/2192392668 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f793c071620 0x7f793c0cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:53.196+0000 7f7940c00640 1 --2- 172.21.15.119:0/2192392668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f793c071190 0x7f793c151480 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:53.196+0000 7f7940c00640 1 -- 172.21.15.119:0/2192392668 >> 172.21.15.119:0/2192392668 conn(0x7f793c06aba0 msgr2=0x7f793c15c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:53.196+0000 7f7940c00640 1 -- 172.21.15.119:0/2192392668 shutdown_connections 2024-04-25T10:14:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:53.196+0000 7f7940c00640 1 -- 172.21.15.119:0/2192392668 wait complete. 2024-04-25T10:14:53.622 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:23.427654+0000', 'modified': '2024-04-25T10:14:52.733476+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 85, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7668}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7668': {'gid': 7668, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.119:6837/2043821095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 2043821095}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2043821095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:53.622 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7668:up:replay 2024-04-25T10:14:54.623 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:54.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.830+0000 7f72bfe00640 1 Processor -- start 2024-04-25T10:14:54.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.830+0000 7f72bfe00640 1 -- start start 2024-04-25T10:14:54.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.832+0000 7f72bfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72b80b3670 0x7f72b81622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.832+0000 7f72bfe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f72b81664d0 0x7f72b8162800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.832+0000 7f72bfe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b8071330 con 0x7f72b8161ee0 2024-04-25T10:14:54.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.832+0000 7f72bfe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b80720f0 con 0x7f72b81664d0 2024-04-25T10:14:54.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.832+0000 7f72bfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b8071610 con 0x7f72b80b3670 2024-04-25T10:14:54.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.833+0000 7f72bde00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f72b81664d0 0x7f72b8162800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.833+0000 7f72bca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72b80b3670 0x7f72b81622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.833+0000 7f72bde00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f72b81664d0 0x7f72b8162800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:44622/0 (socket says 172.21.15.119:44622) 2024-04-25T10:14:54.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.833+0000 7f72bde00640 1 -- 172.21.15.119:0/795976795 learned_addr learned my addr 172.21.15.119:0/795976795 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:54.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.833+0000 7f72bde00640 1 -- 172.21.15.119:0/795976795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72b80b3670 msgr2=0x7f72b81622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.833+0000 7f72bde00640 1 --2- 172.21.15.119:0/795976795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72b80b3670 0x7f72b81622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.834+0000 7f72bde00640 1 -- 172.21.15.119:0/795976795 >> v1:172.21.15.132:6789/0 conn(0x7f72b8161ee0 legacy=0x7f72b80b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.834+0000 7f72bde00640 1 -- 172.21.15.119:0/795976795 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b80718f0 con 0x7f72b81664d0 2024-04-25T10:14:54.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.835+0000 7f72bde00640 1 --2- 172.21.15.119:0/795976795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b81664d0 0x7f72b8162800 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009e50 tx=0x7f72b002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=978a98ea3a529a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.835+0000 7f72b7e00640 1 -- 172.21.15.119:0/795976795 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0036070 con 0x7f72b81664d0 2024-04-25T10:14:54.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.836+0000 7f72b7e00640 1 -- 172.21.15.119:0/795976795 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b0031600 con 0x7f72b81664d0 2024-04-25T10:14:54.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.836+0000 7f72b7e00640 1 -- 172.21.15.119:0/795976795 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b00319d0 con 0x7f72b81664d0 2024-04-25T10:14:54.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.836+0000 7f72bfe00640 1 -- 172.21.15.119:0/795976795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b81664d0 msgr2=0x7f72b8162800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.836+0000 7f72bfe00640 1 --2- 172.21.15.119:0/795976795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b81664d0 0x7f72b8162800 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009e50 tx=0x7f72b002a710 comp rx=0 tx=0).stop 2024-04-25T10:14:54.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.836+0000 7f72bfe00640 1 -- 172.21.15.119:0/795976795 shutdown_connections 2024-04-25T10:14:54.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.836+0000 7f72bfe00640 1 --2- 172.21.15.119:0/795976795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b81664d0 0x7f72b8162800 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.836+0000 7f72bfe00640 1 --2- 172.21.15.119:0/795976795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72b80b3670 0x7f72b81622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.836+0000 7f72bfe00640 1 -- 172.21.15.119:0/795976795 >> 172.21.15.119:0/795976795 conn(0x7f72b806aba0 msgr2=0x7f72b806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.836+0000 7f72bfe00640 1 -- 172.21.15.119:0/795976795 shutdown_connections 2024-04-25T10:14:54.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.836+0000 7f72bfe00640 1 -- 172.21.15.119:0/795976795 wait complete. 2024-04-25T10:14:54.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.838+0000 7f72bfe00640 1 Processor -- start 2024-04-25T10:14:54.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.838+0000 7f72bfe00640 1 -- start start 2024-04-25T10:14:54.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.839+0000 7f72bfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72b80b3670 0x7f72b81f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.839+0000 7f72bfe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b8161ee0 0x7f72b80cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.839+0000 7f72bfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b81664d0 0x7f72b81f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.839+0000 7f72bfe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b80b3d30 con 0x7f72b81664d0 2024-04-25T10:14:54.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.839+0000 7f72bfe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b80b3bb0 con 0x7f72b8161ee0 2024-04-25T10:14:54.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.839+0000 7f72bfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b80b2570 con 0x7f72b80b3670 2024-04-25T10:14:54.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.840+0000 7f72bde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b81664d0 0x7f72b81f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.840+0000 7f72bde00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b81664d0 0x7f72b81f8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60596/0 (socket says 172.21.15.119:60596) 2024-04-25T10:14:54.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.840+0000 7f72bde00640 1 -- 172.21.15.119:0/2388244448 learned_addr learned my addr 172.21.15.119:0/2388244448 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:54.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.840+0000 7f72bca00640 1 --2- 172.21.15.119:0/2388244448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b8161ee0 0x7f72b80cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.840+0000 7f72bd400640 1 --2- 172.21.15.119:0/2388244448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72b80b3670 0x7f72b81f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.840+0000 7f72bde00640 1 -- 172.21.15.119:0/2388244448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72b80b3670 msgr2=0x7f72b81f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.840+0000 7f72bde00640 1 --2- 172.21.15.119:0/2388244448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72b80b3670 0x7f72b81f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.840+0000 7f72bde00640 1 -- 172.21.15.119:0/2388244448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b8161ee0 msgr2=0x7f72b80cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.840+0000 7f72bde00640 1 --2- 172.21.15.119:0/2388244448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b8161ee0 0x7f72b80cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.840+0000 7f72bde00640 1 -- 172.21.15.119:0/2388244448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b8162360 con 0x7f72b81664d0 2024-04-25T10:14:54.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.840+0000 7f72bde00640 1 --2- 172.21.15.119:0/2388244448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b81664d0 0x7f72b81f8ad0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009f70 tx=0x7f72b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.841+0000 7f72b5600640 1 -- 172.21.15.119:0/2388244448 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b003f020 con 0x7f72b81664d0 2024-04-25T10:14:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.841+0000 7f72b5600640 1 -- 172.21.15.119:0/2388244448 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b0037040 con 0x7f72b81664d0 2024-04-25T10:14:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.841+0000 7f72b5600640 1 -- 172.21.15.119:0/2388244448 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0036070 con 0x7f72b81664d0 2024-04-25T10:14:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.841+0000 7f72bfe00640 1 -- 172.21.15.119:0/2388244448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b80ceb30 con 0x7f72b81664d0 2024-04-25T10:14:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.841+0000 7f72bfe00640 1 -- 172.21.15.119:0/2388244448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b81f8520 con 0x7f72b81664d0 2024-04-25T10:14:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.842+0000 7f72bfe00640 1 -- 172.21.15.119:0/2388244448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b80749b0 con 0x7f72b81664d0 2024-04-25T10:14:54.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.843+0000 7f72b5600640 1 -- 172.21.15.119:0/2388244448 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72b0042020 con 0x7f72b81664d0 2024-04-25T10:14:54.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.844+0000 7f72b5600640 1 --2- 172.21.15.119:0/2388244448 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f729806b4d0 0x7f729806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.844+0000 7f72b5600640 1 -- 172.21.15.119:0/2388244448 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f72b00aa6a0 con 0x7f72b81664d0 2024-04-25T10:14:54.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.844+0000 7f72bd400640 1 --2- 172.21.15.119:0/2388244448 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f729806b4d0 0x7f729806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.845+0000 7f72bd400640 1 --2- 172.21.15.119:0/2388244448 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f729806b4d0 0x7f729806d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f72ac007fd0 tx=0x7f72ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:54.848+0000 7f72b5600640 1 -- 172.21.15.119:0/2388244448 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f72b0049030 con 0x7f72b81664d0 2024-04-25T10:14:55.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:55.080+0000 7f72bfe00640 1 -- 172.21.15.119:0/2388244448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72b806d4c0 con 0x7f72b81664d0 2024-04-25T10:14:55.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:55.081+0000 7f72b5600640 1 -- 172.21.15.119:0/2388244448 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f72b00794e0 con 0x7f72b81664d0 2024-04-25T10:14:55.085 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:14:55.085 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:14:53.766320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7668":{"gid":7668,"name":"a","rank":0,"incarnation":71,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.119:6837/2043821095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2043821095},{"type":"v1","addr":"172.21.15.119:6837","nonce":2043821095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:14:55.086 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 73 2024-04-25T10:14:55.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:55.083+0000 7f72bfe00640 1 -- 172.21.15.119:0/2388244448 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f729806b4d0 msgr2=0x7f729806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:55.083+0000 7f72bfe00640 1 --2- 172.21.15.119:0/2388244448 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f729806b4d0 0x7f729806d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f72ac007fd0 tx=0x7f72ac009040 comp rx=0 tx=0).stop 2024-04-25T10:14:55.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:55.083+0000 7f72bfe00640 1 -- 172.21.15.119:0/2388244448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b81664d0 msgr2=0x7f72b81f8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:55.083+0000 7f72bfe00640 1 --2- 172.21.15.119:0/2388244448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b81664d0 0x7f72b81f8ad0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009f70 tx=0x7f72b0004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:55.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:55.084+0000 7f72bfe00640 1 -- 172.21.15.119:0/2388244448 shutdown_connections 2024-04-25T10:14:55.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:55.084+0000 7f72bfe00640 1 --2- 172.21.15.119:0/2388244448 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f729806b4d0 0x7f729806d990 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:55.084+0000 7f72bfe00640 1 --2- 172.21.15.119:0/2388244448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b81664d0 0x7f72b81f8ad0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:55.084+0000 7f72bfe00640 1 --2- 172.21.15.119:0/2388244448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b8161ee0 0x7f72b80cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:55.084+0000 7f72bfe00640 1 --2- 172.21.15.119:0/2388244448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f72b80b3670 0x7f72b81f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:55.084+0000 7f72bfe00640 1 -- 172.21.15.119:0/2388244448 >> 172.21.15.119:0/2388244448 conn(0x7f72b806aba0 msgr2=0x7f72b80b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:55.084+0000 7f72bfe00640 1 -- 172.21.15.119:0/2388244448 shutdown_connections 2024-04-25T10:14:55.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:55.084+0000 7f72bfe00640 1 -- 172.21.15.119:0/2388244448 wait complete. 2024-04-25T10:14:55.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:23.427654+0000', 'modified': '2024-04-25T10:14:53.766320+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 85, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7668}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7668': {'gid': 7668, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.119:6837/2043821095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 2043821095}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2043821095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:55.511 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7668:up:rejoin 2024-04-25T10:14:56.512 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:56.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.712+0000 7f3bf3200640 1 Processor -- start 2024-04-25T10:14:56.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.712+0000 7f3bf3200640 1 -- start start 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.714+0000 7f3bf3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3bf415fc90 0x7f3bf406e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.714+0000 7f3bf3200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf40cd440 con 0x7f3bf40b5050 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.714+0000 7f3bf3200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf40cd720 con 0x7f3bf4144470 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.714+0000 7f3bf3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4145340 con 0x7f3bf415fc90 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.714+0000 7f3bebe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3bf40b5050 0x7f3bf415bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36160/0 (socket says 172.21.15.119:36160) 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.714+0000 7f3bebe00640 1 -- 172.21.15.119:0/1042499274 learned_addr learned my addr 172.21.15.119:0/1042499274 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.714+0000 7f3bf2800640 1 --2- 172.21.15.119:0/1042499274 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3bf415fc90 0x7f3bf406e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.714+0000 7f3bf1400640 1 -- 172.21.15.119:0/1042499274 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092464675 0 0) 0x7f3bf40cd440 con 0x7f3bf40b5050 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bf1400640 1 -- 172.21.15.119:0/1042499274 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bd8003520 con 0x7f3bf40b5050 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bf1400640 1 -- 172.21.15.119:0/1042499274 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 292651204 0 0) 0x7f3bd8003520 con 0x7f3bf40b5050 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bf1400640 1 -- 172.21.15.119:0/1042499274 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3bf415fc90 msgr2=0x7f3bf406e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bf1400640 1 --2- 172.21.15.119:0/1042499274 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3bf415fc90 0x7f3bf406e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bf1400640 1 -- 172.21.15.119:0/1042499274 >> v1:172.21.15.132:6789/0 conn(0x7f3bf4144470 legacy=0x7f3bf40b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bf1400640 1 -- 172.21.15.119:0/1042499274 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf40cda00 con 0x7f3bf40b5050 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bf1400640 1 -- 172.21.15.119:0/1042499274 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f3bdc001670 con 0x7f3bf40b5050 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bf1400640 1 -- 172.21.15.119:0/1042499274 >> v1:172.21.15.119:6789/0 conn(0x7f3bf40b5050 legacy=0x7f3bf415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bf1400640 1 --2- 172.21.15.119:0/1042499274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bd8003a50 0x7f3bd8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bf1400640 1 --2- 172.21.15.119:0/1042499274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bd8004800 0x7f3bd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bf1400640 1 --2- 172.21.15.119:0/1042499274 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3bf415fc90 0x7f3bd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bf1400640 1 -- 172.21.15.119:0/1042499274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd800ba90 con 0x7f3bd8004800 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bf1400640 1 -- 172.21.15.119:0/1042499274 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd800bc40 con 0x7f3bd8003a50 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bf1400640 1 -- 172.21.15.119:0/1042499274 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd800be40 con 0x7f3bf415fc90 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.715+0000 7f3bebe00640 1 --2- 172.21.15.119:0/1042499274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bd8004800 0x7f3bd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.716+0000 7f3bf1e00640 1 --2- 172.21.15.119:0/1042499274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bd8003a50 0x7f3bd8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.716+0000 7f3bebe00640 1 -- 172.21.15.119:0/1042499274 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3bf415fc90 msgr2=0x7f3bd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.716+0000 7f3bf2800640 1 --2- 172.21.15.119:0/1042499274 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3bf415fc90 0x7f3bd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.716+0000 7f3bebe00640 1 --2- 172.21.15.119:0/1042499274 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3bf415fc90 0x7f3bd80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.716+0000 7f3bebe00640 1 -- 172.21.15.119:0/1042499274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bd8003a50 msgr2=0x7f3bd8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.716+0000 7f3bebe00640 1 --2- 172.21.15.119:0/1042499274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bd8003a50 0x7f3bd8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.716+0000 7f3bebe00640 1 -- 172.21.15.119:0/1042499274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd8003520 con 0x7f3bd8004800 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.716+0000 7f3bf1e00640 1 --2- 172.21.15.119:0/1042499274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bd8003a50 0x7f3bd8003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:56.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.717+0000 7f3bebe00640 1 --2- 172.21.15.119:0/1042499274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bd8004800 0x7f3bd8004c80 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00c240 tx=0x7f3bd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9010a26a27ed8ff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.717+0000 7f3bf1400640 1 -- 172.21.15.119:0/1042499274 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc007cf0 con 0x7f3bd8004800 2024-04-25T10:14:56.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.717+0000 7f3bf1400640 1 -- 172.21.15.119:0/1042499274 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc0058d0 con 0x7f3bd8004800 2024-04-25T10:14:56.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.717+0000 7f3bf1400640 1 -- 172.21.15.119:0/1042499274 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc005ce0 con 0x7f3bd8004800 2024-04-25T10:14:56.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.718+0000 7f3bf3200640 1 -- 172.21.15.119:0/1042499274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bd8004800 msgr2=0x7f3bd8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.718+0000 7f3bf3200640 1 --2- 172.21.15.119:0/1042499274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bd8004800 0x7f3bd8004c80 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00c240 tx=0x7f3bd8005830 comp rx=0 tx=0).stop 2024-04-25T10:14:56.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.718+0000 7f3bf2800640 1 -- 172.21.15.119:0/1042499274 reap_dead start 2024-04-25T10:14:56.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.718+0000 7f3bf3200640 1 -- 172.21.15.119:0/1042499274 shutdown_connections 2024-04-25T10:14:56.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.718+0000 7f3bf3200640 1 -- 172.21.15.119:0/1042499274 >> 172.21.15.119:0/1042499274 conn(0x7f3bf406b1d0 msgr2=0x7f3bf40b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.718+0000 7f3bf3200640 1 -- 172.21.15.119:0/1042499274 shutdown_connections 2024-04-25T10:14:56.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.718+0000 7f3bf3200640 1 -- 172.21.15.119:0/1042499274 wait complete. 2024-04-25T10:14:56.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.720+0000 7f3bf3200640 1 Processor -- start 2024-04-25T10:14:56.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.720+0000 7f3bf3200640 1 -- start start 2024-04-25T10:14:56.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.721+0000 7f3bf3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3bd8004800 0x7f3bf413bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.721+0000 7f3bf3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bf415ad70 0x7f3bf40b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.721+0000 7f3bf3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bf40b0d20 0x7f3bf415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.721+0000 7f3bf3200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf40ce200 con 0x7f3bf40b0d20 2024-04-25T10:14:56.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.721+0000 7f3bf3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4144850 con 0x7f3bf415ad70 2024-04-25T10:14:56.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.721+0000 7f3bf3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf406e430 con 0x7f3bd8004800 2024-04-25T10:14:56.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.721+0000 7f3bf2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bf40b0d20 0x7f3bf415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.721+0000 7f3bf2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bf40b0d20 0x7f3bf415bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60610/0 (socket says 172.21.15.119:60610) 2024-04-25T10:14:56.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.721+0000 7f3bf2800640 1 -- 172.21.15.119:0/660668159 learned_addr learned my addr 172.21.15.119:0/660668159 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:56.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.721+0000 7f3bf1e00640 1 --2- 172.21.15.119:0/660668159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3bd8004800 0x7f3bf413bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.721+0000 7f3bebe00640 1 --2- 172.21.15.119:0/660668159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bf415ad70 0x7f3bf40b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.721+0000 7f3bf2800640 1 -- 172.21.15.119:0/660668159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3bd8004800 msgr2=0x7f3bf413bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.721+0000 7f3bf2800640 1 --2- 172.21.15.119:0/660668159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3bd8004800 0x7f3bf413bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.721+0000 7f3bf2800640 1 -- 172.21.15.119:0/660668159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bf415ad70 msgr2=0x7f3bf40b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.722+0000 7f3bf2800640 1 --2- 172.21.15.119:0/660668159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bf415ad70 0x7f3bf40b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.722+0000 7f3bf2800640 1 -- 172.21.15.119:0/660668159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf413ce30 con 0x7f3bf40b0d20 2024-04-25T10:14:56.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.722+0000 7f3bf2800640 1 --2- 172.21.15.119:0/660668159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bf40b0d20 0x7f3bf415bbf0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40b23d0 tx=0x7f3bec00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.722+0000 7f3bea000640 1 -- 172.21.15.119:0/660668159 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec01c070 con 0x7f3bf40b0d20 2024-04-25T10:14:56.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.722+0000 7f3bea000640 1 -- 172.21.15.119:0/660668159 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bec017400 con 0x7f3bf40b0d20 2024-04-25T10:14:56.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.722+0000 7f3bea000640 1 -- 172.21.15.119:0/660668159 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0177f0 con 0x7f3bf40b0d20 2024-04-25T10:14:56.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.722+0000 7f3bf3200640 1 -- 172.21.15.119:0/660668159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf414c550 con 0x7f3bf40b0d20 2024-04-25T10:14:56.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.722+0000 7f3bf3200640 1 -- 172.21.15.119:0/660668159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf414cb10 con 0x7f3bf40b0d20 2024-04-25T10:14:56.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.724+0000 7f3bf3200640 1 -- 172.21.15.119:0/660668159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf40749b0 con 0x7f3bf40b0d20 2024-04-25T10:14:56.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.724+0000 7f3bea000640 1 -- 172.21.15.119:0/660668159 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bec005d10 con 0x7f3bf40b0d20 2024-04-25T10:14:56.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.725+0000 7f3bea000640 1 --2- 172.21.15.119:0/660668159 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bd006b500 0x7f3bd006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.725+0000 7f3bea000640 1 -- 172.21.15.119:0/660668159 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f3bec08a6b0 con 0x7f3bf40b0d20 2024-04-25T10:14:56.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.726+0000 7f3bf1e00640 1 --2- 172.21.15.119:0/660668159 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bd006b500 0x7f3bd006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.726+0000 7f3bf1e00640 1 --2- 172.21.15.119:0/660668159 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bd006b500 0x7f3bd006d9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3be4000f60 tx=0x7f3be4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.729+0000 7f3bea000640 1 -- 172.21.15.119:0/660668159 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3bec012070 con 0x7f3bf40b0d20 2024-04-25T10:14:56.754 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:14:56.750+0000 7fb532000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:56.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.963+0000 7f3bf3200640 1 -- 172.21.15.119:0/660668159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bf406dff0 con 0x7f3bf40b0d20 2024-04-25T10:14:56.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.965+0000 7f3bea000640 1 -- 172.21.15.119:0/660668159 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f3bec059460 con 0x7f3bf40b0d20 2024-04-25T10:14:56.969 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:14:56.969 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:14:54.775138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7668":{"gid":7668,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/2043821095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2043821095},{"type":"v1","addr":"172.21.15.119:6837","nonce":2043821095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:14:56.969 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 74 2024-04-25T10:14:56.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.967+0000 7f3bf3200640 1 -- 172.21.15.119:0/660668159 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bd006b500 msgr2=0x7f3bd006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.967+0000 7f3bf3200640 1 --2- 172.21.15.119:0/660668159 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bd006b500 0x7f3bd006d9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3be4000f60 tx=0x7f3be4009040 comp rx=0 tx=0).stop 2024-04-25T10:14:56.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.967+0000 7f3bf3200640 1 -- 172.21.15.119:0/660668159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bf40b0d20 msgr2=0x7f3bf415bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.967+0000 7f3bf3200640 1 --2- 172.21.15.119:0/660668159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bf40b0d20 0x7f3bf415bbf0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40b23d0 tx=0x7f3bec00f4a0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.967+0000 7f3bf3200640 1 -- 172.21.15.119:0/660668159 shutdown_connections 2024-04-25T10:14:56.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.967+0000 7f3bf3200640 1 --2- 172.21.15.119:0/660668159 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bd006b500 0x7f3bd006d9c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.967+0000 7f3bf3200640 1 --2- 172.21.15.119:0/660668159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3bf40b0d20 0x7f3bf415bbf0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.967+0000 7f3bf3200640 1 --2- 172.21.15.119:0/660668159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3bf415ad70 0x7f3bf40b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.967+0000 7f3bf3200640 1 --2- 172.21.15.119:0/660668159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3bd8004800 0x7f3bf413bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.967+0000 7f3bf3200640 1 -- 172.21.15.119:0/660668159 >> 172.21.15.119:0/660668159 conn(0x7f3bf406b1d0 msgr2=0x7f3bf40ae380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.968+0000 7f3bf3200640 1 -- 172.21.15.119:0/660668159 shutdown_connections 2024-04-25T10:14:56.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:56.968+0000 7f3bf3200640 1 -- 172.21.15.119:0/660668159 wait complete. 2024-04-25T10:14:57.393 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:23.427654+0000', 'modified': '2024-04-25T10:14:54.775138+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 85, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7668}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7668': {'gid': 7668, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.119:6837/2043821095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 2043821095}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2043821095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:57.393 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:14:57.393 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:14:57.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.601+0000 7f1c18000640 1 Processor -- start 2024-04-25T10:14:57.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.602+0000 7f1c18000640 1 -- start start 2024-04-25T10:14:57.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.603+0000 7f1c18000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c10145260 0x7f1c100cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.603+0000 7f1c18000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1c100cc490 0x7f1c10071200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.603+0000 7f1c18000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c10071740 0x7f1c1015f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.603+0000 7f1c18000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c10071bb0 con 0x7f1c10145260 2024-04-25T10:14:57.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.603+0000 7f1c18000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c100cea00 con 0x7f1c100cc490 2024-04-25T10:14:57.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.603+0000 7f1c18000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c100cc9a0 con 0x7f1c10071740 2024-04-25T10:14:57.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.603+0000 7f1c15600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c10145260 0x7f1c100cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.603+0000 7f1c15600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c10145260 0x7f1c100cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60614/0 (socket says 172.21.15.119:60614) 2024-04-25T10:14:57.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.603+0000 7f1c15600640 1 -- 172.21.15.119:0/1675481018 learned_addr learned my addr 172.21.15.119:0/1675481018 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:57.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.604+0000 7f1c16000640 1 --2- 172.21.15.119:0/1675481018 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c10071740 0x7f1c1015f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.604+0000 7f1c14c00640 1 --2- 172.21.15.119:0/1675481018 >> v2:172.21.15.132:3300/0 conn(0x7f1c100cc490 0x7f1c10071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.604+0000 7f1c15600640 1 -- 172.21.15.119:0/1675481018 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c10071740 msgr2=0x7f1c1015f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.604+0000 7f1c15600640 1 --2- 172.21.15.119:0/1675481018 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c10071740 0x7f1c1015f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.604+0000 7f1c15600640 1 -- 172.21.15.119:0/1675481018 >> v2:172.21.15.132:3300/0 conn(0x7f1c100cc490 msgr2=0x7f1c10071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.604+0000 7f1c15600640 1 --2- 172.21.15.119:0/1675481018 >> v2:172.21.15.132:3300/0 conn(0x7f1c100cc490 0x7f1c10071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.604+0000 7f1c15600640 1 -- 172.21.15.119:0/1675481018 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c100ccc80 con 0x7f1c10145260 2024-04-25T10:14:57.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.604+0000 7f1c14c00640 1 --2- 172.21.15.119:0/1675481018 >> v2:172.21.15.132:3300/0 conn(0x7f1c100cc490 0x7f1c10071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:57.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.605+0000 7f1c15600640 1 --2- 172.21.15.119:0/1675481018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c10145260 0x7f1c100cbf50 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1c100cb960 tx=0x7f1c0002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc7cd8b2ff24999 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.605+0000 7f1c0fe00640 1 -- 172.21.15.119:0/1675481018 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00037070 con 0x7f1c10145260 2024-04-25T10:14:57.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.605+0000 7f1c0fe00640 1 -- 172.21.15.119:0/1675481018 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c00032440 con 0x7f1c10145260 2024-04-25T10:14:57.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.605+0000 7f1c0fe00640 1 -- 172.21.15.119:0/1675481018 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00032810 con 0x7f1c10145260 2024-04-25T10:14:57.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.606+0000 7f1c18000640 1 -- 172.21.15.119:0/1675481018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c10145260 msgr2=0x7f1c100cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.606+0000 7f1c18000640 1 --2- 172.21.15.119:0/1675481018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c10145260 0x7f1c100cbf50 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1c100cb960 tx=0x7f1c0002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.606+0000 7f1c18000640 1 -- 172.21.15.119:0/1675481018 shutdown_connections 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.606+0000 7f1c18000640 1 --2- 172.21.15.119:0/1675481018 >> v2:172.21.15.132:3300/0 conn(0x7f1c100cc490 0x7f1c10071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.606+0000 7f1c18000640 1 --2- 172.21.15.119:0/1675481018 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c10071740 0x7f1c1015f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.606+0000 7f1c18000640 1 --2- 172.21.15.119:0/1675481018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c10145260 0x7f1c100cbf50 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.606+0000 7f1c18000640 1 -- 172.21.15.119:0/1675481018 >> 172.21.15.119:0/1675481018 conn(0x7f1c1006aba0 msgr2=0x7f1c1006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.606+0000 7f1c18000640 1 -- 172.21.15.119:0/1675481018 shutdown_connections 2024-04-25T10:14:57.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.606+0000 7f1c18000640 1 -- 172.21.15.119:0/1675481018 wait complete. 2024-04-25T10:14:57.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.608+0000 7f1c18000640 1 Processor -- start 2024-04-25T10:14:57.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.608+0000 7f1c18000640 1 -- start start 2024-04-25T10:14:57.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c18000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c10071740 0x7f1c1015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c18000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c100cc490 0x7f1c1015dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c18000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c10145260 0x7f1c1015e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c18000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c10145640 con 0x7f1c100cc490 2024-04-25T10:14:57.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c18000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c100cd240 con 0x7f1c10071740 2024-04-25T10:14:57.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c18000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1015c270 con 0x7f1c10145260 2024-04-25T10:14:57.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c14c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c100cc490 0x7f1c1015dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c14c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c100cc490 0x7f1c1015dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60628/0 (socket says 172.21.15.119:60628) 2024-04-25T10:14:57.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c14c00640 1 -- 172.21.15.119:0/4088637784 learned_addr learned my addr 172.21.15.119:0/4088637784 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:57.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c16000640 1 --2- 172.21.15.119:0/4088637784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c10145260 0x7f1c1015e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c15600640 1 --2- 172.21.15.119:0/4088637784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c10071740 0x7f1c1015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c14c00640 1 -- 172.21.15.119:0/4088637784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c10145260 msgr2=0x7f1c1015e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c14c00640 1 --2- 172.21.15.119:0/4088637784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c10145260 0x7f1c1015e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c14c00640 1 -- 172.21.15.119:0/4088637784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c10071740 msgr2=0x7f1c1015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c14c00640 1 --2- 172.21.15.119:0/4088637784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c10071740 0x7f1c1015b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.609+0000 7f1c14c00640 1 -- 172.21.15.119:0/4088637784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c100b3a00 con 0x7f1c100cc490 2024-04-25T10:14:57.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.610+0000 7f1c14c00640 1 --2- 172.21.15.119:0/4088637784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c100cc490 0x7f1c1015dfa0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00cf30 tx=0x7f1bfc00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.610+0000 7f1c0d600640 1 -- 172.21.15.119:0/4088637784 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc01a070 con 0x7f1c100cc490 2024-04-25T10:14:57.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.610+0000 7f1c0d600640 1 -- 172.21.15.119:0/4088637784 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bfc00d840 con 0x7f1c100cc490 2024-04-25T10:14:57.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.610+0000 7f1c0d600640 1 -- 172.21.15.119:0/4088637784 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00dc10 con 0x7f1c100cc490 2024-04-25T10:14:57.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.610+0000 7f1c18000640 1 -- 172.21.15.119:0/4088637784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c100b3ce0 con 0x7f1c100cc490 2024-04-25T10:14:57.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.610+0000 7f1c18000640 1 -- 172.21.15.119:0/4088637784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c100b42a0 con 0x7f1c100cc490 2024-04-25T10:14:57.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.612+0000 7f1c0d600640 1 -- 172.21.15.119:0/4088637784 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1bfc01d070 con 0x7f1c100cc490 2024-04-25T10:14:57.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.613+0000 7f1c0d600640 1 --2- 172.21.15.119:0/4088637784 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1bf006b4d0 0x7f1bf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.613+0000 7f1c0d600640 1 -- 172.21.15.119:0/4088637784 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f1bfc08ab40 con 0x7f1c100cc490 2024-04-25T10:14:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.614+0000 7f1c15600640 1 --2- 172.21.15.119:0/4088637784 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1bf006b4d0 0x7f1bf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.614+0000 7f1c1ad96740 1 Processor -- start 2024-04-25T10:14:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.614+0000 7f1c15600640 1 --2- 172.21.15.119:0/4088637784 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1bf006b4d0 0x7f1bf006d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1c00034000 tx=0x7f1c0002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.614+0000 7f1c1ad96740 1 -- start start 2024-04-25T10:14:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.615+0000 7f1c1ad96740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dfc8950 0x55e62dff0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.615+0000 7f1c1ad96740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62dff0cd0 0x55e62dfc8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.615+0000 7f1c1ad96740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62e0115c0 0x55e62dfc7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.615+0000 7f1c1ad96740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e62df2c620 con 0x55e62dfc8950 2024-04-25T10:14:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.615+0000 7f1c1ad96740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e62ded5ca0 con 0x55e62dff0cd0 2024-04-25T10:14:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.615+0000 7f1c1ad96740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e62df286c0 con 0x55e62e0115c0 2024-04-25T10:14:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.615+0000 7f1c15600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dfc8950 0x55e62dff0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.615+0000 7f1c15600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dfc8950 0x55e62dff0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60642/0 (socket says 172.21.15.119:60642) 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.615+0000 7f1c15600640 1 -- 172.21.15.119:0/3515330335 learned_addr learned my addr 172.21.15.119:0/3515330335 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.616+0000 7f1c16000640 1 --2- 172.21.15.119:0/3515330335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62e0115c0 0x55e62dfc7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.616+0000 7f1c15600640 1 -- 172.21.15.119:0/3515330335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62e0115c0 msgr2=0x55e62dfc7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.616+0000 7f1c15600640 1 --2- 172.21.15.119:0/3515330335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62e0115c0 0x55e62dfc7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.616+0000 7f1c15600640 1 -- 172.21.15.119:0/3515330335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62dff0cd0 msgr2=0x55e62dfc8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.616+0000 7f1c15600640 1 --2- 172.21.15.119:0/3515330335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62dff0cd0 0x55e62dfc8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.616+0000 7f1c15600640 1 -- 172.21.15.119:0/3515330335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e62dedcf40 con 0x55e62dfc8950 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.616+0000 7f1c15600640 1 --2- 172.21.15.119:0/3515330335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dfc8950 0x55e62dff0790 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f1c00035040 tx=0x7f1c0002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c53535d93bff45a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.616+0000 7f1c05c00640 1 -- 172.21.15.119:0/3515330335 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0004d070 con 0x55e62dfc8950 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.616+0000 7f1c05c00640 1 -- 172.21.15.119:0/3515330335 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c00032cb0 con 0x55e62dfc8950 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.616+0000 7f1c05c00640 1 -- 172.21.15.119:0/3515330335 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00037040 con 0x55e62dfc8950 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.617+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3515330335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dfc8950 msgr2=0x55e62dff0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.617+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/3515330335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dfc8950 0x55e62dff0790 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f1c00035040 tx=0x7f1c0002c000 comp rx=0 tx=0).stop 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.617+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3515330335 shutdown_connections 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.617+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/3515330335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62e0115c0 0x55e62dfc7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.617+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/3515330335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62dff0cd0 0x55e62dfc8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.617+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/3515330335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dfc8950 0x55e62dff0790 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.617+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3515330335 >> 172.21.15.119:0/3515330335 conn(0x55e62dfc5b70 msgr2=0x55e62e00e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.617+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3515330335 shutdown_connections 2024-04-25T10:14:57.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.617+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3515330335 wait complete. 2024-04-25T10:14:57.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.618+0000 7f1c1ad96740 1 -- start start 2024-04-25T10:14:57.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.618+0000 7f1c1ad96740 1 Processor -- start 2024-04-25T10:14:57.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.619+0000 7f1c1ad96740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dff0cd0 0x55e62dfdd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.619+0000 7f1c1ad96740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62e0115c0 0x55e62dfeea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.619+0000 7f1c1ad96740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62dff3630 0x55e62dfeefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.619+0000 7f1c1ad96740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e62dff0300 con 0x55e62dff0cd0 2024-04-25T10:14:57.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.619+0000 7f1c1ad96740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e62decb1f0 con 0x55e62e0115c0 2024-04-25T10:14:57.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.619+0000 7f1c1ad96740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e62dfefcf0 con 0x55e62dff3630 2024-04-25T10:14:57.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.619+0000 7f1c1ad96740 10 client.0 _ll_register_callbacks cb 0x55e62dee8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:57.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.619+0000 7f1c15600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dff0cd0 0x55e62dfdd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.619+0000 7f1c15600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dff0cd0 0x55e62dfdd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60644/0 (socket says 172.21.15.119:60644) 2024-04-25T10:14:57.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.619+0000 7f1c15600640 1 -- 172.21.15.119:0/3910942638 learned_addr learned my addr 172.21.15.119:0/3910942638 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:57.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.620+0000 7f1c15600640 1 -- 172.21.15.119:0/3910942638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62dff3630 msgr2=0x55e62dfeefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.620+0000 7f1c15600640 1 --2- 172.21.15.119:0/3910942638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62dff3630 0x55e62dfeefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.620+0000 7f1c15600640 1 -- 172.21.15.119:0/3910942638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62e0115c0 msgr2=0x55e62dfeea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.620+0000 7f1c15600640 1 --2- 172.21.15.119:0/3910942638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62e0115c0 0x55e62dfeea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.620+0000 7f1c15600640 1 -- 172.21.15.119:0/3910942638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e62deeac50 con 0x55e62dff0cd0 2024-04-25T10:14:57.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.620+0000 7f1c15600640 1 --2- 172.21.15.119:0/3910942638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dff0cd0 0x55e62dfdd800 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f1c0002a7d0 tx=0x7f1c00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b877d778a5f2b67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.620+0000 7f1bfa000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.620+0000 7f1bfa000640 1 -- 172.21.15.119:0/3910942638 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0004d070 con 0x55e62dff0cd0 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.620+0000 7f1bfa000640 1 -- 172.21.15.119:0/3910942638 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f1c00052b90 con 0x55e62dff0cd0 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.620+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3910942638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c00032cb0 con 0x55e62dff0cd0 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.620+0000 7f1bfa000640 1 -- 172.21.15.119:0/3910942638 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c00059050 con 0x55e62dff0cd0 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.620+0000 7f1c1ad96740 10 client.7740 fetch_fsmap learned FSMap version 74 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.620+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3910942638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55e62dc412d0 con 0x55e62dff0cd0 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.621+0000 7f1bfa000640 1 -- 172.21.15.119:0/3910942638 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c00032cb0 con 0x55e62dff0cd0 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.621+0000 7f1bfa000640 1 -- 172.21.15.119:0/3910942638 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1c00003610 con 0x55e62dff0cd0 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.621+0000 7f1c1ad96740 10 client.7740 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.621+0000 7f1c1ad96740 10 client.7740 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.621+0000 7f1c1ad96740 20 client.7740 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.621+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/3910942638 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55e62dff14f0 0x55e62dff18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.621+0000 7f1c1ad96740 4 client.7740 mds_command: new command op to 7668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.621+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3910942638 --> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e62df39b60 con 0x55e62dff14f0 2024-04-25T10:14:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.621+0000 7f1c14c00640 1 --2- 172.21.15.119:0/3910942638 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55e62dff14f0 0x55e62dff18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.622+0000 7f1c14c00640 1 --2- 172.21.15.119:0/3910942638 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55e62dff14f0 0x55e62dff18d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42aa12dcccd19437 server_cookie=6337f0324e16b278 in_seq=0 out_seq=0 2024-04-25T10:14:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.622+0000 7f1bfa000640 10 client.7740 ms_handle_connect on v2:172.21.15.119:6835/2043821095 2024-04-25T10:14:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.622+0000 7f1bfa000640 1 -- 172.21.15.119:0/3910942638 <== mds.0 v2:172.21.15.119:6835/2043821095 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e62df39b60 con 0x55e62dff14f0 2024-04-25T10:14:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.622+0000 7f1bfa000640 10 client.7740 handle_command_reply: tid=0 2024-04-25T10:14:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.622+0000 7f1c1ad96740 1 client.7740 shutdown 2024-04-25T10:14:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.623+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3910942638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dff0cd0 msgr2=0x55e62dfdd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.623+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/3910942638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dff0cd0 0x55e62dfdd800 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f1c0002a7d0 tx=0x7f1c00004730 comp rx=0 tx=0).stop 2024-04-25T10:14:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.623+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3910942638 shutdown_connections 2024-04-25T10:14:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.623+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/3910942638 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55e62dff14f0 0x55e62dff18d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.623+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/3910942638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62dff3630 0x55e62dfeefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.623+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/3910942638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62e0115c0 0x55e62dfeea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.623+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/3910942638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dff0cd0 0x55e62dfdd800 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.623+0000 7f1bfa000640 0 client.7740 ms_handle_reset on v2:172.21.15.119:6835/2043821095 2024-04-25T10:14:57.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.623+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3910942638 >> 172.21.15.119:0/3910942638 conn(0x55e62dfc8950 msgr2=0x55e62dfc7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.623+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3910942638 shutdown_connections 2024-04-25T10:14:57.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.623+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3910942638 wait complete. 2024-04-25T10:14:57.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.623+0000 7f1c1ad96740 20 client.7740 trim_cache size 0 max 16384 2024-04-25T10:14:57.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.640+0000 7f1c1ad96740 1 Processor -- start 2024-04-25T10:14:57.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.640+0000 7f1c1ad96740 1 -- start start 2024-04-25T10:14:57.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.641+0000 7f1c1ad96740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62dff0cd0 0x55e62dff2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.641+0000 7f1c1ad96740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dff3630 0x55e62dff3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.641+0000 7f1c1ad96740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62e0115c0 0x55e62dfc71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.641+0000 7f1c1ad96740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101ff0e0 con 0x55e62dff3630 2024-04-25T10:14:57.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.641+0000 7f1c1ad96740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c100ad9e0 con 0x55e62e0115c0 2024-04-25T10:14:57.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.641+0000 7f1c1ad96740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101feae0 con 0x55e62dff0cd0 2024-04-25T10:14:57.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.641+0000 7f1c14c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dff3630 0x55e62dff3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.641+0000 7f1c14c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dff3630 0x55e62dff3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60646/0 (socket says 172.21.15.119:60646) 2024-04-25T10:14:57.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.641+0000 7f1c14c00640 1 -- 172.21.15.119:0/3780912672 learned_addr learned my addr 172.21.15.119:0/3780912672 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:57.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.642+0000 7f1c16000640 1 --2- 172.21.15.119:0/3780912672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62e0115c0 0x55e62dfc71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.642+0000 7f1c15600640 1 --2- 172.21.15.119:0/3780912672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62dff0cd0 0x55e62dff2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.642+0000 7f1c14c00640 1 -- 172.21.15.119:0/3780912672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62dff0cd0 msgr2=0x55e62dff2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.642+0000 7f1c14c00640 1 --2- 172.21.15.119:0/3780912672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62dff0cd0 0x55e62dff2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.642+0000 7f1c14c00640 1 -- 172.21.15.119:0/3780912672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62e0115c0 msgr2=0x55e62dfc71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.642+0000 7f1c14c00640 1 --2- 172.21.15.119:0/3780912672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62e0115c0 0x55e62dfc71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.642+0000 7f1c14c00640 1 -- 172.21.15.119:0/3780912672 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e62dd0b6b0 con 0x55e62dff3630 2024-04-25T10:14:57.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.642+0000 7f1c14c00640 1 --2- 172.21.15.119:0/3780912672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dff3630 0x55e62dff3040 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00cf00 tx=0x7f1bfc00f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebffcdde1e575d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.642+0000 7f1c05200640 1 -- 172.21.15.119:0/3780912672 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc086070 con 0x55e62dff3630 2024-04-25T10:14:57.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.642+0000 7f1c05200640 1 -- 172.21.15.119:0/3780912672 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bfc089010 con 0x55e62dff3630 2024-04-25T10:14:57.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.642+0000 7f1c05200640 1 -- 172.21.15.119:0/3780912672 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc055870 con 0x55e62dff3630 2024-04-25T10:14:57.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.643+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3780912672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dff3630 msgr2=0x55e62dff3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.643+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/3780912672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dff3630 0x55e62dff3040 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00cf00 tx=0x7f1bfc00f040 comp rx=0 tx=0).stop 2024-04-25T10:14:57.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.643+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3780912672 shutdown_connections 2024-04-25T10:14:57.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.643+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/3780912672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62e0115c0 0x55e62dfc71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.643+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/3780912672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62dff3630 0x55e62dff3040 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.643+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/3780912672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62dff0cd0 0x55e62dff2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.643+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3780912672 >> 172.21.15.119:0/3780912672 conn(0x55e62dfc8950 msgr2=0x55e62dff61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.643+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3780912672 shutdown_connections 2024-04-25T10:14:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.643+0000 7f1c1ad96740 1 -- 172.21.15.119:0/3780912672 wait complete. 2024-04-25T10:14:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.643+0000 7f1c1ad96740 1 -- start start 2024-04-25T10:14:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.644+0000 7f1c1ad96740 1 Processor -- start 2024-04-25T10:14:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.644+0000 7f1c1ad96740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62dff0cd0 0x55e62dfc2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.644+0000 7f1c1ad96740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62dff3630 0x55e62e0780d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c1ad96740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62e0115c0 0x55e62dff61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c1ad96740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e62dff3ae0 con 0x55e62e0115c0 2024-04-25T10:14:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c1ad96740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e62dff0300 con 0x55e62dff0cd0 2024-04-25T10:14:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c1ad96740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e62dfddd40 con 0x55e62dff3630 2024-04-25T10:14:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c1ad96740 10 client.0 _ll_register_callbacks cb 0x7f1c0004b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c16000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62e0115c0 0x55e62dff61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c16000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62e0115c0 0x55e62dff61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60654/0 (socket says 172.21.15.119:60654) 2024-04-25T10:14:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c16000640 1 -- 172.21.15.119:0/2971610893 learned_addr learned my addr 172.21.15.119:0/2971610893 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c15600640 1 --2- 172.21.15.119:0/2971610893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62dff0cd0 0x55e62dfc2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c14c00640 1 --2- 172.21.15.119:0/2971610893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62dff3630 0x55e62e0780d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c16000640 1 -- 172.21.15.119:0/2971610893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62dff3630 msgr2=0x55e62e0780d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c16000640 1 --2- 172.21.15.119:0/2971610893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62dff3630 0x55e62e0780d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c16000640 1 -- 172.21.15.119:0/2971610893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62dff0cd0 msgr2=0x55e62dfc2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c16000640 1 --2- 172.21.15.119:0/2971610893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62dff0cd0 0x55e62dfc2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c16000640 1 -- 172.21.15.119:0/2971610893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e62dfef6a0 con 0x55e62e0115c0 2024-04-25T10:14:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1c16000640 1 --2- 172.21.15.119:0/2971610893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62e0115c0 0x55e62dff61f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f1c08006fd0 tx=0x7f1c0800f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc9ae7ec499ccbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.645+0000 7f1bfa000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:57.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.646+0000 7f1bfa000640 1 -- 172.21.15.119:0/2971610893 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08019070 con 0x55e62e0115c0 2024-04-25T10:14:57.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.646+0000 7f1bfa000640 1 -- 172.21.15.119:0/2971610893 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f1c08015480 con 0x55e62e0115c0 2024-04-25T10:14:57.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.646+0000 7f1c1ad96740 1 -- 172.21.15.119:0/2971610893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bfc089010 con 0x55e62e0115c0 2024-04-25T10:14:57.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.646+0000 7f1bfa000640 1 -- 172.21.15.119:0/2971610893 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c08024050 con 0x55e62e0115c0 2024-04-25T10:14:57.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.646+0000 7f1c1ad96740 10 client.7746 fetch_fsmap learned FSMap version 74 2024-04-25T10:14:57.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.646+0000 7f1c1ad96740 1 -- 172.21.15.119:0/2971610893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55e62dfda030 con 0x55e62e0115c0 2024-04-25T10:14:57.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.646+0000 7f1bfa000640 1 -- 172.21.15.119:0/2971610893 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bfc089010 con 0x55e62e0115c0 2024-04-25T10:14:57.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.646+0000 7f1bfa000640 1 -- 172.21.15.119:0/2971610893 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1c08027020 con 0x55e62e0115c0 2024-04-25T10:14:57.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.646+0000 7f1c1ad96740 10 client.7746 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:14:57.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.646+0000 7f1c1ad96740 10 client.7746 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:14:57.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.646+0000 7f1c1ad96740 20 client.7746 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:57.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.646+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/2971610893 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55e62dfc6f80 0x55e62dfc7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.646+0000 7f1c1ad96740 4 client.7746 mds_command: new command op to 7668 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:14:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.646+0000 7f1c1ad96740 1 -- 172.21.15.119:0/2971610893 --> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e62dfc62d0 con 0x55e62dfc6f80 2024-04-25T10:14:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.647+0000 7f1c15600640 1 --2- 172.21.15.119:0/2971610893 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55e62dfc6f80 0x55e62dfc7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.647+0000 7f1bfa000640 10 client.7746 ms_handle_connect on v2:172.21.15.119:6835/2043821095 2024-04-25T10:14:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.647+0000 7f1c15600640 1 --2- 172.21.15.119:0/2971610893 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55e62dfc6f80 0x55e62dfc7360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc44db5148253a6e server_cookie=4006436bb650c7d8 in_seq=0 out_seq=0 2024-04-25T10:14:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.647+0000 7f1bfa000640 1 -- 172.21.15.119:0/2971610893 <== mds.0 v2:172.21.15.119:6835/2043821095 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55e62dfc62d0 con 0x55e62dfc6f80 2024-04-25T10:14:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.647+0000 7f1bfa000640 10 client.7746 handle_command_reply: tid=0 2024-04-25T10:14:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.647+0000 7f1c1ad96740 1 client.7746 shutdown 2024-04-25T10:14:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.648+0000 7f1c1ad96740 1 -- 172.21.15.119:0/2971610893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62e0115c0 msgr2=0x55e62dff61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.648+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/2971610893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62e0115c0 0x55e62dff61f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f1c08006fd0 tx=0x7f1c0800f5b0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.648+0000 7f1c1ad96740 1 -- 172.21.15.119:0/2971610893 shutdown_connections 2024-04-25T10:14:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.648+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/2971610893 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55e62dfc6f80 0x55e62dfc7360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.648+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/2971610893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e62e0115c0 0x55e62dff61f0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.648+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/2971610893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e62dff3630 0x55e62e0780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.648+0000 7f1c1ad96740 1 --2- 172.21.15.119:0/2971610893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e62dff0cd0 0x55e62dfc2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.648+0000 7f1bfa000640 0 client.7746 ms_handle_reset on v2:172.21.15.119:6835/2043821095 2024-04-25T10:14:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.648+0000 7f1c1ad96740 1 -- 172.21.15.119:0/2971610893 >> 172.21.15.119:0/2971610893 conn(0x55e62dfc8950 msgr2=0x55e62dff18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.648+0000 7f1c1ad96740 1 -- 172.21.15.119:0/2971610893 shutdown_connections 2024-04-25T10:14:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.648+0000 7f1c1ad96740 1 -- 172.21.15.119:0/2971610893 wait complete. 2024-04-25T10:14:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.648+0000 7f1c1ad96740 20 client.7746 trim_cache size 0 max 16384 2024-04-25T10:14:57.658 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:14:57.658 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "e0b0f428-b441-4237-a4d1-ed53321f87a5", 2024-04-25T10:14:57.658 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-04-25T10:14:57.659 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 7668, 2024-04-25T10:14:57.659 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-04-25T10:14:57.659 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-04-25T10:14:57.659 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-04-25T10:14:57.659 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 4.9077026799999999, 2024-04-25T10:14:57.659 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 74, 2024-04-25T10:14:57.659 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 85, 2024-04-25T10:14:57.659 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T10:14:57.659 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 6.8389676220000002 2024-04-25T10:14:57.659 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:14:57.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.649+0000 7f1c18000640 1 -- 172.21.15.119:0/4088637784 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1bf006b4d0 msgr2=0x7f1bf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.649+0000 7f1c18000640 1 --2- 172.21.15.119:0/4088637784 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1bf006b4d0 0x7f1bf006d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1c00034000 tx=0x7f1c0002e000 comp rx=0 tx=0).stop 2024-04-25T10:14:57.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.649+0000 7f1c18000640 1 -- 172.21.15.119:0/4088637784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c100cc490 msgr2=0x7f1c1015dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.649+0000 7f1c18000640 1 --2- 172.21.15.119:0/4088637784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c100cc490 0x7f1c1015dfa0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00cf30 tx=0x7f1bfc00a7d0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.649+0000 7f1c18000640 1 -- 172.21.15.119:0/4088637784 shutdown_connections 2024-04-25T10:14:57.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.649+0000 7f1c18000640 1 --2- 172.21.15.119:0/4088637784 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1bf006b4d0 0x7f1bf006d990 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.649+0000 7f1c18000640 1 --2- 172.21.15.119:0/4088637784 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c10145260 0x7f1c1015e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.649+0000 7f1c18000640 1 --2- 172.21.15.119:0/4088637784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c100cc490 0x7f1c1015dfa0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.649+0000 7f1c18000640 1 --2- 172.21.15.119:0/4088637784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c10071740 0x7f1c1015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.649+0000 7f1c18000640 1 -- 172.21.15.119:0/4088637784 >> 172.21.15.119:0/4088637784 conn(0x7f1c1006aba0 msgr2=0x7f1c1006da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.649+0000 7f1c18000640 1 -- 172.21.15.119:0/4088637784 shutdown_connections 2024-04-25T10:14:57.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.649+0000 7f1c18000640 1 -- 172.21.15.119:0/4088637784 wait complete. 2024-04-25T10:14:57.669 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:14:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.852+0000 7efd4dc00640 1 Processor -- start 2024-04-25T10:14:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.852+0000 7efd4dc00640 1 -- start start 2024-04-25T10:14:57.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.853+0000 7efd4dc00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efd4815a870 0x7efd480cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.854+0000 7efd4dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd4806e2e0 0x7efd4815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.854+0000 7efd4dc00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd480cdb10 con 0x7efd480cd580 2024-04-25T10:14:57.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.854+0000 7efd4dc00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd480ce720 con 0x7efd4815a870 2024-04-25T10:14:57.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.854+0000 7efd4dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd480cddf0 con 0x7efd4806e2e0 2024-04-25T10:14:57.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.854+0000 7efd47400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7efd480cd580 0x7efd480cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36176/0 (socket says 172.21.15.119:36176) 2024-04-25T10:14:57.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.854+0000 7efd47400640 1 -- 172.21.15.119:0/1441078340 learned_addr learned my addr 172.21.15.119:0/1441078340 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:14:57.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.854+0000 7efd46a00640 1 --2- 172.21.15.119:0/1441078340 >> v2:172.21.15.119:3300/0 conn(0x7efd4815a870 0x7efd480cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.854+0000 7efd46000640 1 --2- 172.21.15.119:0/1441078340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd4806e2e0 0x7efd4815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.854+0000 7efd45600640 1 -- 172.21.15.119:0/1441078340 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453715700 0 0) 0x7efd480cdb10 con 0x7efd480cd580 2024-04-25T10:14:57.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.854+0000 7efd46a00640 1 -- 172.21.15.119:0/1441078340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd4806e2e0 msgr2=0x7efd4815ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.854+0000 7efd46a00640 1 --2- 172.21.15.119:0/1441078340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd4806e2e0 0x7efd4815ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.854+0000 7efd46a00640 1 -- 172.21.15.119:0/1441078340 >> v1:172.21.15.119:6789/0 conn(0x7efd480cd580 legacy=0x7efd480cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.855+0000 7efd46a00640 1 -- 172.21.15.119:0/1441078340 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd480ce0d0 con 0x7efd4815a870 2024-04-25T10:14:57.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.855+0000 7efd46000640 1 --2- 172.21.15.119:0/1441078340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd4806e2e0 0x7efd4815ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:57.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.856+0000 7efd46a00640 1 --2- 172.21.15.119:0/1441078340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd4815a870 0x7efd480cebb0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7efd380097a0 tx=0x7efd3802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b41e66479f35a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.856+0000 7efd45600640 1 -- 172.21.15.119:0/1441078340 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38037070 con 0x7efd4815a870 2024-04-25T10:14:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.856+0000 7efd45600640 1 -- 172.21.15.119:0/1441078340 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd38032440 con 0x7efd4815a870 2024-04-25T10:14:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.856+0000 7efd45600640 1 -- 172.21.15.119:0/1441078340 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38032810 con 0x7efd4815a870 2024-04-25T10:14:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.856+0000 7efd4dc00640 1 -- 172.21.15.119:0/1441078340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd4815a870 msgr2=0x7efd480cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.856+0000 7efd4dc00640 1 --2- 172.21.15.119:0/1441078340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd4815a870 0x7efd480cebb0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7efd380097a0 tx=0x7efd3802a180 comp rx=0 tx=0).stop 2024-04-25T10:14:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.857+0000 7efd4dc00640 1 -- 172.21.15.119:0/1441078340 shutdown_connections 2024-04-25T10:14:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.857+0000 7efd4dc00640 1 --2- 172.21.15.119:0/1441078340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd4806e2e0 0x7efd4815ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.857+0000 7efd4dc00640 1 --2- 172.21.15.119:0/1441078340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd4815a870 0x7efd480cebb0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.857+0000 7efd4dc00640 1 -- 172.21.15.119:0/1441078340 >> 172.21.15.119:0/1441078340 conn(0x7efd4806aba0 msgr2=0x7efd4806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.857+0000 7efd4dc00640 1 -- 172.21.15.119:0/1441078340 shutdown_connections 2024-04-25T10:14:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.857+0000 7efd4dc00640 1 -- 172.21.15.119:0/1441078340 wait complete. 2024-04-25T10:14:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.858+0000 7efd4dc00640 1 Processor -- start 2024-04-25T10:14:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.859+0000 7efd4dc00640 1 -- start start 2024-04-25T10:14:57.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.859+0000 7efd4dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd4806e2e0 0x7efd48151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.859+0000 7efd4dc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd480cd580 0x7efd480c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.859+0000 7efd4dc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd4815a870 0x7efd480c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.859+0000 7efd4dc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd480cfdc0 con 0x7efd4815a870 2024-04-25T10:14:57.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.859+0000 7efd4dc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd480cfc40 con 0x7efd480cd580 2024-04-25T10:14:57.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.859+0000 7efd4dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4815bf10 con 0x7efd4806e2e0 2024-04-25T10:14:57.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.860+0000 7efd47400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd4815a870 0x7efd480c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.860+0000 7efd47400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd4815a870 0x7efd480c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60674/0 (socket says 172.21.15.119:60674) 2024-04-25T10:14:57.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.860+0000 7efd47400640 1 -- 172.21.15.119:0/1296083138 learned_addr learned my addr 172.21.15.119:0/1296083138 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:57.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.860+0000 7efd46a00640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd4806e2e0 0x7efd48151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.860+0000 7efd46000640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd480cd580 0x7efd480c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.860+0000 7efd47400640 1 -- 172.21.15.119:0/1296083138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd4806e2e0 msgr2=0x7efd48151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.860+0000 7efd47400640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd4806e2e0 0x7efd48151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.860+0000 7efd47400640 1 -- 172.21.15.119:0/1296083138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd480cd580 msgr2=0x7efd480c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.860+0000 7efd47400640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd480cd580 0x7efd480c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.860+0000 7efd47400640 1 -- 172.21.15.119:0/1296083138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd480c8b20 con 0x7efd4815a870 2024-04-25T10:14:57.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.860+0000 7efd46000640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd480cd580 0x7efd480c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:57.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.861+0000 7efd47400640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd4815a870 0x7efd480c6f90 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7efd4000c7a0 tx=0x7efd4000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.861+0000 7efd3ea00640 1 -- 172.21.15.119:0/1296083138 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40018070 con 0x7efd4815a870 2024-04-25T10:14:57.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.861+0000 7efd3ea00640 1 -- 172.21.15.119:0/1296083138 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4000a640 con 0x7efd4815a870 2024-04-25T10:14:57.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.861+0000 7efd4dc00640 1 -- 172.21.15.119:0/1296083138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd480b3c30 con 0x7efd4815a870 2024-04-25T10:14:57.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.861+0000 7efd3ea00640 1 -- 172.21.15.119:0/1296083138 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4000aab0 con 0x7efd4815a870 2024-04-25T10:14:57.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.861+0000 7efd4dc00640 1 -- 172.21.15.119:0/1296083138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd480c4580 con 0x7efd4815a870 2024-04-25T10:14:57.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.862+0000 7efd4dc00640 1 -- 172.21.15.119:0/1296083138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd480749b0 con 0x7efd4815a870 2024-04-25T10:14:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.863+0000 7efd3ea00640 1 -- 172.21.15.119:0/1296083138 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd4001b070 con 0x7efd4815a870 2024-04-25T10:14:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.864+0000 7efd3ea00640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd2806b500 0x7efd2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.865+0000 7efd3ea00640 1 -- 172.21.15.119:0/1296083138 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7efd400897f0 con 0x7efd4815a870 2024-04-25T10:14:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.865+0000 7efd46a00640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd2806b500 0x7efd2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.865+0000 7efd46a00640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd2806b500 0x7efd2806d9c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efd38034000 tx=0x7efd3802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:57.869+0000 7efd3ea00640 1 -- 172.21.15.119:0/1296083138 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd480c8b20 con 0x7efd4815a870 2024-04-25T10:14:58.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:58.107+0000 7efd4dc00640 1 -- 172.21.15.119:0/1296083138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efd480cee00 con 0x7efd4815a870 2024-04-25T10:14:58.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:58.107+0000 7efd3ea00640 1 -- 172.21.15.119:0/1296083138 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) v1 ==== 53+0+73 (secure 0 0 0) 0x7efd40058630 con 0x7efd4815a870 2024-04-25T10:14:58.112 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:14:58.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:58.110+0000 7efd4dc00640 1 -- 172.21.15.119:0/1296083138 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd2806b500 msgr2=0x7efd2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:58.110+0000 7efd4dc00640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd2806b500 0x7efd2806d9c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efd38034000 tx=0x7efd3802e000 comp rx=0 tx=0).stop 2024-04-25T10:14:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:58.110+0000 7efd4dc00640 1 -- 172.21.15.119:0/1296083138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd4815a870 msgr2=0x7efd480c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:58.110+0000 7efd4dc00640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd4815a870 0x7efd480c6f90 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7efd4000c7a0 tx=0x7efd4000cc60 comp rx=0 tx=0).stop 2024-04-25T10:14:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:58.110+0000 7efd4dc00640 1 -- 172.21.15.119:0/1296083138 shutdown_connections 2024-04-25T10:14:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:58.110+0000 7efd4dc00640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efd2806b500 0x7efd2806d9c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:58.110+0000 7efd4dc00640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd4815a870 0x7efd480c6f90 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:58.110+0000 7efd4dc00640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd480cd580 0x7efd480c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:58.110+0000 7efd4dc00640 1 --2- 172.21.15.119:0/1296083138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efd4806e2e0 0x7efd48151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:58.110+0000 7efd4dc00640 1 -- 172.21.15.119:0/1296083138 >> 172.21.15.119:0/1296083138 conn(0x7efd4806aba0 msgr2=0x7efd4815cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:58.110+0000 7efd4dc00640 1 -- 172.21.15.119:0/1296083138 shutdown_connections 2024-04-25T10:14:58.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:58.111+0000 7efd4dc00640 1 -- 172.21.15.119:0/1296083138 wait complete. 2024-04-25T10:14:58.133 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:14:58.133 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:14:58.133 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:14:58.133 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:14:58.133 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:58.133 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:14:58.133 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:14:58.133 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:14:58.133 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:58.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:58.134 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:14:58.184 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:14:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:14:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:14:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:14:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:58.184 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:58.184 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:14:58.185 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:14:58.185 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:58.185 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:14:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:14:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42191sec preferred_lft 42191sec 2024-04-25T10:14:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:14:58.185 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:58.185 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:14:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:14:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:14:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:58.186 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:58.186 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:14:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:58.187 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:14:58.187 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:58.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:58.187 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:14:58.266 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:14:58.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:58.270 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:14:58.325 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:14:58.325 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:14:58.325 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:14:58.326 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:14:58.326 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:14:58.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:58.326 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:14:58.327 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:14:58.327 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:14:58.327 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:14:58.327 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:14:58.327 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:14:58.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:58.466 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:14:58.515 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:14:58.515 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:14:58.516 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:58.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:58.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:14:58.603 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:58.603 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:58.657 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:58.657 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:58.657 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:58.710 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:14:58.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:58.711 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:58.839 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:58.840 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:58.840 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:14:58.859 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:58.860 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:58.860 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:14:58.860 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:14:58.860 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:14:58.885 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:58.885 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:14:58.886 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:14:58.887 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:14:58.888 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:14:58.888 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:14:58.888 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:14:58.888 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:14:58.893 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:14:58.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:58.893 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:58.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:58.963 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:14:59.012 INFO:teuthology.orchestra.run.smithi119.stdout:client7758 2024-04-25T10:14:59.012 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:14:59.013 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:59.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:59.060 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T10:14:59.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.308+0000 7f4c01600640 1 Processor -- start 2024-04-25T10:14:59.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.309+0000 7f4c01600640 1 -- start start 2024-04-25T10:14:59.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.310+0000 7f4c01600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4bfc15dbe0 0x7f4bfc071c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.310+0000 7f4c01600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bfc0cfad0 0x7f4bfc06d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.310+0000 7f4c01600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bfc0ace70 con 0x7f4bfc0aedc0 2024-04-25T10:14:59.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.310+0000 7f4c01600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bfc072150 con 0x7f4bfc15dbe0 2024-04-25T10:14:59.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.310+0000 7f4c01600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc0ad110 con 0x7f4bfc0cfad0 2024-04-25T10:14:59.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.311+0000 7f4bfb400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bfc0cfad0 0x7f4bfc06d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.311+0000 7f4bfaa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4bfc15dbe0 0x7f4bfc071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.311+0000 7f4bfb400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bfc0cfad0 0x7f4bfc06d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:51666/0 (socket says 172.21.15.119:51666) 2024-04-25T10:14:59.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.311+0000 7f4bfb400640 1 -- 172.21.15.119:0/3825951762 learned_addr learned my addr 172.21.15.119:0/3825951762 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:59.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.312+0000 7f4bfb400640 1 -- 172.21.15.119:0/3825951762 >> v2:172.21.15.132:3300/0 conn(0x7f4bfc15dbe0 msgr2=0x7f4bfc071c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.312+0000 7f4bfb400640 1 --2- 172.21.15.119:0/3825951762 >> v2:172.21.15.132:3300/0 conn(0x7f4bfc15dbe0 0x7f4bfc071c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.312+0000 7f4bfb400640 1 -- 172.21.15.119:0/3825951762 >> v1:172.21.15.132:6789/0 conn(0x7f4bfc0aedc0 legacy=0x7f4bfc0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.312+0000 7f4bfb400640 1 -- 172.21.15.119:0/3825951762 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc0ad3f0 con 0x7f4bfc0cfad0 2024-04-25T10:14:59.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.313+0000 7f4bfb400640 1 --2- 172.21.15.119:0/3825951762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bfc0cfad0 0x7f4bfc06d9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009e50 tx=0x7f4bf402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e3ddd00a5fa286f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.314+0000 7f4bfa000640 1 -- 172.21.15.119:0/3825951762 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4036070 con 0x7f4bfc0cfad0 2024-04-25T10:14:59.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.314+0000 7f4bfa000640 1 -- 172.21.15.119:0/3825951762 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf4031600 con 0x7f4bfc0cfad0 2024-04-25T10:14:59.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.314+0000 7f4bfa000640 1 -- 172.21.15.119:0/3825951762 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf40319d0 con 0x7f4bfc0cfad0 2024-04-25T10:14:59.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.314+0000 7f4c01600640 1 -- 172.21.15.119:0/3825951762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bfc0cfad0 msgr2=0x7f4bfc06d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.314+0000 7f4c01600640 1 --2- 172.21.15.119:0/3825951762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bfc0cfad0 0x7f4bfc06d9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009e50 tx=0x7f4bf402a710 comp rx=0 tx=0).stop 2024-04-25T10:14:59.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.315+0000 7f4c01600640 1 -- 172.21.15.119:0/3825951762 shutdown_connections 2024-04-25T10:14:59.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.315+0000 7f4c01600640 1 --2- 172.21.15.119:0/3825951762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bfc0cfad0 0x7f4bfc06d9a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.315+0000 7f4c01600640 1 --2- 172.21.15.119:0/3825951762 >> v2:172.21.15.132:3300/0 conn(0x7f4bfc15dbe0 0x7f4bfc071c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.315+0000 7f4c01600640 1 -- 172.21.15.119:0/3825951762 >> 172.21.15.119:0/3825951762 conn(0x7f4bfc06aba0 msgr2=0x7f4bfc06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.315+0000 7f4c01600640 1 -- 172.21.15.119:0/3825951762 shutdown_connections 2024-04-25T10:14:59.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.315+0000 7f4c01600640 1 -- 172.21.15.119:0/3825951762 wait complete. 2024-04-25T10:14:59.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.316+0000 7f4c01600640 1 Processor -- start 2024-04-25T10:14:59.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.316+0000 7f4c01600640 1 -- start start 2024-04-25T10:14:59.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.317+0000 7f4c01600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bfc0aedc0 0x7f4bfc147dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.317+0000 7f4c01600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bfc0cfad0 0x7f4bfc0cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.317+0000 7f4c01600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bfc15dbe0 0x7f4bfc0c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.317+0000 7f4c01600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc0adc90 con 0x7f4bfc0cfad0 2024-04-25T10:14:59.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.317+0000 7f4c01600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc0ad9b0 con 0x7f4bfc0aedc0 2024-04-25T10:14:59.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.317+0000 7f4c01600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc15dfc0 con 0x7f4bfc15dbe0 2024-04-25T10:14:59.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.318+0000 7f4bf3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bfc0cfad0 0x7f4bfc0cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.318+0000 7f4bf3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bfc0cfad0 0x7f4bfc0cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60684/0 (socket says 172.21.15.119:60684) 2024-04-25T10:14:59.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.318+0000 7f4bf3e00640 1 -- 172.21.15.119:0/3734430973 learned_addr learned my addr 172.21.15.119:0/3734430973 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:59.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.318+0000 7f4bfb400640 1 --2- 172.21.15.119:0/3734430973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bfc15dbe0 0x7f4bfc0c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.318+0000 7f4bfaa00640 1 --2- 172.21.15.119:0/3734430973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bfc0aedc0 0x7f4bfc147dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.318+0000 7f4bf3e00640 1 -- 172.21.15.119:0/3734430973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bfc15dbe0 msgr2=0x7f4bfc0c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.318+0000 7f4bf3e00640 1 --2- 172.21.15.119:0/3734430973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bfc15dbe0 0x7f4bfc0c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.318+0000 7f4bf3e00640 1 -- 172.21.15.119:0/3734430973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bfc0aedc0 msgr2=0x7f4bfc147dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.318+0000 7f4bf3e00640 1 --2- 172.21.15.119:0/3734430973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bfc0aedc0 0x7f4bfc147dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.318+0000 7f4bf3e00640 1 -- 172.21.15.119:0/3734430973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc0ce5f0 con 0x7f4bfc0cfad0 2024-04-25T10:14:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.319+0000 7f4bf3e00640 1 --2- 172.21.15.119:0/3734430973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bfc0cfad0 0x7f4bfc0cdb70 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4be400c440 tx=0x7f4be400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.319+0000 7f4bf2a00640 1 -- 172.21.15.119:0/3734430973 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40052d0 con 0x7f4bfc0cfad0 2024-04-25T10:14:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.319+0000 7f4bf2a00640 1 -- 172.21.15.119:0/3734430973 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be4005470 con 0x7f4bfc0cfad0 2024-04-25T10:14:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.319+0000 7f4bf2a00640 1 -- 172.21.15.119:0/3734430973 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4007260 con 0x7f4bfc0cfad0 2024-04-25T10:14:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.319+0000 7f4c01600640 1 -- 172.21.15.119:0/3734430973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bfc0ce8d0 con 0x7f4bfc0cfad0 2024-04-25T10:14:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.319+0000 7f4c01600640 1 -- 172.21.15.119:0/3734430973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bfc148ac0 con 0x7f4bfc0cfad0 2024-04-25T10:14:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.321+0000 7f4bf2a00640 1 -- 172.21.15.119:0/3734430973 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4be40074c0 con 0x7f4bfc0cfad0 2024-04-25T10:14:59.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.322+0000 7f4bf2a00640 1 --2- 172.21.15.119:0/3734430973 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bd806b4d0 0x7f4bd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.322+0000 7f4bf2a00640 1 -- 172.21.15.119:0/3734430973 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f4be4089730 con 0x7f4bfc0cfad0 2024-04-25T10:14:59.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.322+0000 7f4bfaa00640 1 --2- 172.21.15.119:0/3734430973 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bd806b4d0 0x7f4bd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.323+0000 7f4c04370740 1 Processor -- start 2024-04-25T10:14:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.323+0000 7f4c04370740 1 -- start start 2024-04-25T10:14:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.323+0000 7f4bfaa00640 1 --2- 172.21.15.119:0/3734430973 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bd806b4d0 0x7f4bd806d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009ad0 tx=0x7f4bec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.324+0000 7f4c04370740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec615311a0 0x55ec61531580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.324+0000 7f4c04370740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec6154ea40 0x55ec6154ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.324+0000 7f4c04370740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec61506e80 0x55ec6154d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.324+0000 7f4c04370740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec61469980 con 0x55ec615311a0 2024-04-25T10:14:59.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.324+0000 7f4c04370740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec61412de0 con 0x55ec61506e80 2024-04-25T10:14:59.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.324+0000 7f4c04370740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec61465a20 con 0x55ec6154ea40 2024-04-25T10:14:59.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.324+0000 7f4bfaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec615311a0 0x55ec61531580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.324+0000 7f4bfaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec615311a0 0x55ec61531580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60692/0 (socket says 172.21.15.119:60692) 2024-04-25T10:14:59.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.324+0000 7f4bfaa00640 1 -- 172.21.15.119:0/2047542997 learned_addr learned my addr 172.21.15.119:0/2047542997 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:59.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.325+0000 7f4bf3e00640 1 --2- 172.21.15.119:0/2047542997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec6154ea40 0x55ec6154ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.325+0000 7f4bfaa00640 1 -- 172.21.15.119:0/2047542997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec6154ea40 msgr2=0x55ec6154ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.325+0000 7f4bfaa00640 1 --2- 172.21.15.119:0/2047542997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec6154ea40 0x55ec6154ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.325+0000 7f4bfaa00640 1 -- 172.21.15.119:0/2047542997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec61506e80 msgr2=0x55ec6154d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.325+0000 7f4bfaa00640 1 --2- 172.21.15.119:0/2047542997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec61506e80 0x55ec6154d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.325+0000 7f4bfaa00640 1 -- 172.21.15.119:0/2047542997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec61430e30 con 0x55ec615311a0 2024-04-25T10:14:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.325+0000 7f4bfaa00640 1 --2- 172.21.15.119:0/2047542997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec615311a0 0x55ec61531580 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f4bec011db0 tx=0x7f4bec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f38d2465a3e3f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.325+0000 7f4beb000640 1 -- 172.21.15.119:0/2047542997 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec011de0 con 0x55ec615311a0 2024-04-25T10:14:59.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.325+0000 7f4beb000640 1 -- 172.21.15.119:0/2047542997 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec026070 con 0x55ec615311a0 2024-04-25T10:14:59.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.325+0000 7f4beb000640 1 -- 172.21.15.119:0/2047542997 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec021660 con 0x55ec615311a0 2024-04-25T10:14:59.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.326+0000 7f4c04370740 1 -- 172.21.15.119:0/2047542997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec615311a0 msgr2=0x55ec61531580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.326+0000 7f4c04370740 1 --2- 172.21.15.119:0/2047542997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec615311a0 0x55ec61531580 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f4bec011db0 tx=0x7f4bec014000 comp rx=0 tx=0).stop 2024-04-25T10:14:59.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.326+0000 7f4c04370740 1 -- 172.21.15.119:0/2047542997 shutdown_connections 2024-04-25T10:14:59.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.326+0000 7f4c04370740 1 --2- 172.21.15.119:0/2047542997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec61506e80 0x55ec6154d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.326+0000 7f4c04370740 1 --2- 172.21.15.119:0/2047542997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec6154ea40 0x55ec6154ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.326+0000 7f4c04370740 1 --2- 172.21.15.119:0/2047542997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec615311a0 0x55ec61531580 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.326+0000 7f4c04370740 1 -- 172.21.15.119:0/2047542997 >> 172.21.15.119:0/2047542997 conn(0x55ec615037b0 msgr2=0x55ec6154bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.326+0000 7f4c04370740 1 -- 172.21.15.119:0/2047542997 shutdown_connections 2024-04-25T10:14:59.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.326+0000 7f4c04370740 1 -- 172.21.15.119:0/2047542997 wait complete. 2024-04-25T10:14:59.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.327+0000 7f4c04370740 1 -- start start 2024-04-25T10:14:59.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.327+0000 7f4c04370740 1 Processor -- start 2024-04-25T10:14:59.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.328+0000 7f4c04370740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec615311a0 0x55ec6152d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.328+0000 7f4c04370740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec6154ea40 0x55ec6152ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.328+0000 7f4c04370740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec61532a00 0x55ec61534830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.328+0000 7f4c04370740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec61507c20 con 0x55ec61532a00 2024-04-25T10:14:59.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.328+0000 7f4c04370740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec614083c0 con 0x55ec615311a0 2024-04-25T10:14:59.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.328+0000 7f4c04370740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec615016e0 con 0x55ec6154ea40 2024-04-25T10:14:59.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.328+0000 7f4c04370740 10 client.0 _ll_register_callbacks cb 0x55ec6117dc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:59.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.329+0000 7f4bfb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec61532a00 0x55ec61534830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.329+0000 7f4bfb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec61532a00 0x55ec61534830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60708/0 (socket says 172.21.15.119:60708) 2024-04-25T10:14:59.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.329+0000 7f4bfb400640 1 -- 172.21.15.119:0/3628287415 learned_addr learned my addr 172.21.15.119:0/3628287415 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:59.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.329+0000 7f4bf3e00640 1 --2- 172.21.15.119:0/3628287415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec6154ea40 0x55ec6152ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.329+0000 7f4bfaa00640 1 --2- 172.21.15.119:0/3628287415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec615311a0 0x55ec6152d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.329+0000 7f4bfb400640 1 -- 172.21.15.119:0/3628287415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec6154ea40 msgr2=0x55ec6152ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.329+0000 7f4bfb400640 1 --2- 172.21.15.119:0/3628287415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec6154ea40 0x55ec6152ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.329+0000 7f4bfb400640 1 -- 172.21.15.119:0/3628287415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec615311a0 msgr2=0x55ec6152d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.329+0000 7f4bfb400640 1 --2- 172.21.15.119:0/3628287415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec615311a0 0x55ec6152d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.329+0000 7f4bfb400640 1 -- 172.21.15.119:0/3628287415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec612467e0 con 0x55ec61532a00 2024-04-25T10:14:59.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.330+0000 7f4bfb400640 1 --2- 172.21.15.119:0/3628287415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec61532a00 0x55ec61534830 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009f70 tx=0x7f4bf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2353b3160ad7314 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.330+0000 7f4bdf400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:59.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.330+0000 7f4bdf400640 1 -- 172.21.15.119:0/3628287415 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4041070 con 0x55ec61532a00 2024-04-25T10:14:59.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.330+0000 7f4c04370740 1 -- 172.21.15.119:0/3628287415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4bec026070 con 0x55ec61532a00 2024-04-25T10:14:59.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.330+0000 7f4bdf400640 1 -- 172.21.15.119:0/3628287415 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f4bf4031d10 con 0x55ec61532a00 2024-04-25T10:14:59.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.330+0000 7f4bdf400640 1 -- 172.21.15.119:0/3628287415 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bf404c050 con 0x55ec61532a00 2024-04-25T10:14:59.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.330+0000 7f4c04370740 10 client.7767 fetch_fsmap learned FSMap version 74 2024-04-25T10:14:59.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.330+0000 7f4c04370740 1 -- 172.21.15.119:0/3628287415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55ec612a6f70 con 0x55ec61532a00 2024-04-25T10:14:59.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.331+0000 7f4bdf400640 1 -- 172.21.15.119:0/3628287415 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec026070 con 0x55ec61532a00 2024-04-25T10:14:59.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.331+0000 7f4bdf400640 1 -- 172.21.15.119:0/3628287415 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4bf403c440 con 0x55ec61532a00 2024-04-25T10:14:59.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.331+0000 7f4c04370740 10 client.7767 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:14:59.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.331+0000 7f4c04370740 10 client.7767 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:14:59.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.331+0000 7f4c04370740 20 client.7767 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:59.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.331+0000 7f4c04370740 1 --2- 172.21.15.119:0/3628287415 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55ec6152db00 0x55ec6152df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.331+0000 7f4c04370740 4 client.7767 mds_command: new command op to 7668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:59.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.331+0000 7f4c04370740 1 -- 172.21.15.119:0/3628287415 --> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec61476ec0 con 0x55ec6152db00 2024-04-25T10:14:59.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.331+0000 7f4bfaa00640 1 --2- 172.21.15.119:0/3628287415 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55ec6152db00 0x55ec6152df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.332+0000 7f4bfaa00640 1 --2- 172.21.15.119:0/3628287415 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55ec6152db00 0x55ec6152df10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e51bd870f9d220cf server_cookie=ecc25ca1e59971d2 in_seq=0 out_seq=0 2024-04-25T10:14:59.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.332+0000 7f4bdf400640 10 client.7767 ms_handle_connect on v2:172.21.15.119:6835/2043821095 2024-04-25T10:14:59.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.332+0000 7f4bdf400640 1 -- 172.21.15.119:0/3628287415 <== mds.0 v2:172.21.15.119:6835/2043821095 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ec61476ec0 con 0x55ec6152db00 2024-04-25T10:14:59.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.332+0000 7f4bdf400640 10 client.7767 handle_command_reply: tid=0 2024-04-25T10:14:59.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.332+0000 7f4c04370740 1 client.7767 shutdown 2024-04-25T10:14:59.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.333+0000 7f4c04370740 1 -- 172.21.15.119:0/3628287415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec61532a00 msgr2=0x55ec61534830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.333+0000 7f4c04370740 1 --2- 172.21.15.119:0/3628287415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec61532a00 0x55ec61534830 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009f70 tx=0x7f4bf4004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:59.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.333+0000 7f4c04370740 1 -- 172.21.15.119:0/3628287415 shutdown_connections 2024-04-25T10:14:59.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.333+0000 7f4c04370740 1 --2- 172.21.15.119:0/3628287415 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55ec6152db00 0x55ec6152df10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.333+0000 7f4c04370740 1 --2- 172.21.15.119:0/3628287415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec61532a00 0x55ec61534830 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.333+0000 7f4c04370740 1 --2- 172.21.15.119:0/3628287415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec6154ea40 0x55ec6152ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.333+0000 7f4c04370740 1 --2- 172.21.15.119:0/3628287415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec615311a0 0x55ec6152d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.333+0000 7f4c04370740 1 -- 172.21.15.119:0/3628287415 >> 172.21.15.119:0/3628287415 conn(0x55ec61506e80 msgr2=0x55ec6154f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.333+0000 7f4bdf400640 0 client.7767 ms_handle_reset on v2:172.21.15.119:6835/2043821095 2024-04-25T10:14:59.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.333+0000 7f4c04370740 1 -- 172.21.15.119:0/3628287415 shutdown_connections 2024-04-25T10:14:59.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.333+0000 7f4c04370740 1 -- 172.21.15.119:0/3628287415 wait complete. 2024-04-25T10:14:59.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.333+0000 7f4c04370740 20 client.7767 trim_cache size 0 max 16384 2024-04-25T10:14:59.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.349+0000 7f4c04370740 1 Processor -- start 2024-04-25T10:14:59.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.349+0000 7f4c04370740 1 -- start start 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.350+0000 7f4c04370740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec615311a0 0x55ec615062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.350+0000 7f4c04370740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec61532a00 0x55ec6155f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.350+0000 7f4c04370740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6154ea40 0x55ec6154f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.350+0000 7f4c04370740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc2000c0 con 0x55ec615311a0 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.350+0000 7f4c04370740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc1ff240 con 0x55ec6154ea40 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.350+0000 7f4c04370740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc149df0 con 0x55ec61532a00 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.350+0000 7f4bfaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec615311a0 0x55ec615062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.351+0000 7f4bfaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec615311a0 0x55ec615062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60720/0 (socket says 172.21.15.119:60720) 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.351+0000 7f4bfaa00640 1 -- 172.21.15.119:0/2014386870 learned_addr learned my addr 172.21.15.119:0/2014386870 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:59.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.351+0000 7f4bfaa00640 1 -- 172.21.15.119:0/2014386870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec61532a00 msgr2=0x55ec6155f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.351+0000 7f4bfaa00640 1 --2- 172.21.15.119:0/2014386870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec61532a00 0x55ec6155f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.351+0000 7f4bfaa00640 1 -- 172.21.15.119:0/2014386870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6154ea40 msgr2=0x55ec6154f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.351+0000 7f4bfaa00640 1 --2- 172.21.15.119:0/2014386870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6154ea40 0x55ec6154f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.351+0000 7f4bfaa00640 1 -- 172.21.15.119:0/2014386870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec6113e170 con 0x55ec615311a0 2024-04-25T10:14:59.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.351+0000 7f4bfaa00640 1 --2- 172.21.15.119:0/2014386870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec615311a0 0x55ec615062d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f4bec023040 tx=0x7f4bec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c61b3509315dfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.351+0000 7f4bea600640 1 -- 172.21.15.119:0/2014386870 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec033b70 con 0x55ec615311a0 2024-04-25T10:14:59.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.351+0000 7f4bea600640 1 -- 172.21.15.119:0/2014386870 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec026040 con 0x55ec615311a0 2024-04-25T10:14:59.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.351+0000 7f4bea600640 1 -- 172.21.15.119:0/2014386870 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec033390 con 0x55ec615311a0 2024-04-25T10:14:59.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.352+0000 7f4c04370740 1 -- 172.21.15.119:0/2014386870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec615311a0 msgr2=0x55ec615062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.352+0000 7f4c04370740 1 --2- 172.21.15.119:0/2014386870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec615311a0 0x55ec615062d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f4bec023040 tx=0x7f4bec016000 comp rx=0 tx=0).stop 2024-04-25T10:14:59.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.352+0000 7f4c04370740 1 -- 172.21.15.119:0/2014386870 shutdown_connections 2024-04-25T10:14:59.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.352+0000 7f4c04370740 1 --2- 172.21.15.119:0/2014386870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6154ea40 0x55ec6154f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.352+0000 7f4c04370740 1 --2- 172.21.15.119:0/2014386870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec61532a00 0x55ec6155f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.352+0000 7f4c04370740 1 --2- 172.21.15.119:0/2014386870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec615311a0 0x55ec615062d0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.352+0000 7f4c04370740 1 -- 172.21.15.119:0/2014386870 >> 172.21.15.119:0/2014386870 conn(0x55ec61506e80 msgr2=0x55ec6152ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.352+0000 7f4c04370740 1 -- 172.21.15.119:0/2014386870 shutdown_connections 2024-04-25T10:14:59.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.352+0000 7f4c04370740 1 -- 172.21.15.119:0/2014386870 wait complete. 2024-04-25T10:14:59.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.353+0000 7f4c04370740 1 -- start start 2024-04-25T10:14:59.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.353+0000 7f4c04370740 1 Processor -- start 2024-04-25T10:14:59.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.354+0000 7f4c04370740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec615311a0 0x55ec615b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.354+0000 7f4c04370740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec61532a00 0x55ec61507260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.354+0000 7f4c04370740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec6154ea40 0x55ec615077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.354+0000 7f4c04370740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc1fed10 con 0x55ec6154ea40 2024-04-25T10:14:59.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.354+0000 7f4c04370740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec61544080 con 0x55ec61532a00 2024-04-25T10:14:59.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.354+0000 7f4c04370740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec6152b410 con 0x55ec615311a0 2024-04-25T10:14:59.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.354+0000 7f4c04370740 10 client.0 _ll_register_callbacks cb 0x55ec61501bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:59.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.354+0000 7f4bfb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec6154ea40 0x55ec615077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.354+0000 7f4bfb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec6154ea40 0x55ec615077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60732/0 (socket says 172.21.15.119:60732) 2024-04-25T10:14:59.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.354+0000 7f4bfb400640 1 -- 172.21.15.119:0/1331731861 learned_addr learned my addr 172.21.15.119:0/1331731861 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:59.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.355+0000 7f4bfb400640 1 -- 172.21.15.119:0/1331731861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec615311a0 msgr2=0x55ec615b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.355+0000 7f4bfb400640 1 --2- 172.21.15.119:0/1331731861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec615311a0 0x55ec615b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.355+0000 7f4bfb400640 1 -- 172.21.15.119:0/1331731861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec61532a00 msgr2=0x55ec61507260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.355+0000 7f4bfb400640 1 --2- 172.21.15.119:0/1331731861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec61532a00 0x55ec61507260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.355+0000 7f4bfb400640 1 -- 172.21.15.119:0/1331731861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec61541940 con 0x55ec6154ea40 2024-04-25T10:14:59.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.355+0000 7f4bfb400640 1 --2- 172.21.15.119:0/1331731861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec6154ea40 0x55ec615077a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f4bf402a750 tx=0x7f4bf4009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef35e52b39dd06ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.355+0000 7f4bdf400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:59.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.355+0000 7f4bdf400640 1 -- 172.21.15.119:0/1331731861 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4055060 con 0x55ec6154ea40 2024-04-25T10:14:59.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.355+0000 7f4bdf400640 1 -- 172.21.15.119:0/1331731861 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f4bf403cdf0 con 0x55ec6154ea40 2024-04-25T10:14:59.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.355+0000 7f4c04370740 1 -- 172.21.15.119:0/1331731861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4bec026040 con 0x55ec6154ea40 2024-04-25T10:14:59.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.355+0000 7f4bdf400640 1 -- 172.21.15.119:0/1331731861 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bf405b050 con 0x55ec6154ea40 2024-04-25T10:14:59.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.355+0000 7f4c04370740 10 client.7773 fetch_fsmap learned FSMap version 74 2024-04-25T10:14:59.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.356+0000 7f4c04370740 1 -- 172.21.15.119:0/1331731861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55ec6154db50 con 0x55ec6154ea40 2024-04-25T10:14:59.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.356+0000 7f4bdf400640 1 -- 172.21.15.119:0/1331731861 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec026040 con 0x55ec6154ea40 2024-04-25T10:14:59.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.356+0000 7f4bdf400640 1 -- 172.21.15.119:0/1331731861 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4bf405e020 con 0x55ec6154ea40 2024-04-25T10:14:59.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.356+0000 7f4c04370740 10 client.7773 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:14:59.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.356+0000 7f4c04370740 10 client.7773 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:14:59.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.356+0000 7f4c04370740 20 client.7773 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:59.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.356+0000 7f4c04370740 1 --2- 172.21.15.119:0/1331731861 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55ec6154e050 0x55ec6155aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.356+0000 7f4c04370740 4 client.7773 mds_command: new command op to 7668 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:14:59.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.356+0000 7f4c04370740 1 -- 172.21.15.119:0/1331731861 --> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x55ec61471210 con 0x55ec6154e050 2024-04-25T10:14:59.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.356+0000 7f4bfaa00640 1 --2- 172.21.15.119:0/1331731861 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55ec6154e050 0x55ec6155aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.357+0000 7f4bdf400640 10 client.7773 ms_handle_connect on v2:172.21.15.119:6835/2043821095 2024-04-25T10:14:59.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.357+0000 7f4bfaa00640 1 --2- 172.21.15.119:0/1331731861 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55ec6154e050 0x55ec6155aa40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f19afd1d2656cb58 server_cookie=6321809bcc806d60 in_seq=0 out_seq=0 2024-04-25T10:14:59.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.357+0000 7f4bdf400640 1 -- 172.21.15.119:0/1331731861 <== mds.0 v2:172.21.15.119:6835/2043821095 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ec61471210 con 0x55ec6154e050 2024-04-25T10:14:59.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.357+0000 7f4bdf400640 10 client.7773 handle_command_reply: tid=0 2024-04-25T10:14:59.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.357+0000 7f4c04370740 1 client.7773 shutdown 2024-04-25T10:14:59.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.357+0000 7f4c04370740 1 -- 172.21.15.119:0/1331731861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec6154ea40 msgr2=0x55ec615077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.357+0000 7f4c04370740 1 --2- 172.21.15.119:0/1331731861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec6154ea40 0x55ec615077a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f4bf402a750 tx=0x7f4bf4009810 comp rx=0 tx=0).stop 2024-04-25T10:14:59.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.357+0000 7f4c04370740 1 -- 172.21.15.119:0/1331731861 shutdown_connections 2024-04-25T10:14:59.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.357+0000 7f4c04370740 1 --2- 172.21.15.119:0/1331731861 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x55ec6154e050 0x55ec6155aa40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.357+0000 7f4c04370740 1 --2- 172.21.15.119:0/1331731861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec6154ea40 0x55ec615077a0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.357+0000 7f4c04370740 1 --2- 172.21.15.119:0/1331731861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec61532a00 0x55ec61507260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.357+0000 7f4c04370740 1 --2- 172.21.15.119:0/1331731861 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55ec615311a0 0x55ec615b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.358+0000 7f4c04370740 1 -- 172.21.15.119:0/1331731861 >> 172.21.15.119:0/1331731861 conn(0x55ec61506e80 msgr2=0x55ec6155f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.358+0000 7f4bdf400640 0 client.7773 ms_handle_reset on v2:172.21.15.119:6835/2043821095 2024-04-25T10:14:59.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.358+0000 7f4c04370740 1 -- 172.21.15.119:0/1331731861 shutdown_connections 2024-04-25T10:14:59.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.358+0000 7f4c04370740 1 -- 172.21.15.119:0/1331731861 wait complete. 2024-04-25T10:14:59.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.358+0000 7f4c04370740 20 client.7773 trim_cache size 0 max 16384 2024-04-25T10:14:59.365 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:14:59.365 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2024-04-25T10:14:59.365 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "c2ac8ee9-b67f-4fda-a3c7-2f444331a7f1", 2024-04-25T10:14:59.365 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2024-04-25T10:14:59.365 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:14:59.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.359+0000 7f4c01600640 1 -- 172.21.15.119:0/3734430973 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bd806b4d0 msgr2=0x7f4bd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.359+0000 7f4c01600640 1 --2- 172.21.15.119:0/3734430973 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bd806b4d0 0x7f4bd806d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009ad0 tx=0x7f4bec008040 comp rx=0 tx=0).stop 2024-04-25T10:14:59.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.359+0000 7f4c01600640 1 -- 172.21.15.119:0/3734430973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bfc0cfad0 msgr2=0x7f4bfc0cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.359+0000 7f4c01600640 1 --2- 172.21.15.119:0/3734430973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bfc0cfad0 0x7f4bfc0cdb70 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4be400c440 tx=0x7f4be400c900 comp rx=0 tx=0).stop 2024-04-25T10:14:59.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.359+0000 7f4c01600640 1 -- 172.21.15.119:0/3734430973 shutdown_connections 2024-04-25T10:14:59.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.359+0000 7f4c01600640 1 --2- 172.21.15.119:0/3734430973 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4bd806b4d0 0x7f4bd806d990 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.359+0000 7f4c01600640 1 --2- 172.21.15.119:0/3734430973 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4bfc15dbe0 0x7f4bfc0c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.359+0000 7f4c01600640 1 --2- 172.21.15.119:0/3734430973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bfc0cfad0 0x7f4bfc0cdb70 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.359+0000 7f4c01600640 1 --2- 172.21.15.119:0/3734430973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bfc0aedc0 0x7f4bfc147dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.359+0000 7f4c01600640 1 -- 172.21.15.119:0/3734430973 >> 172.21.15.119:0/3734430973 conn(0x7f4bfc06aba0 msgr2=0x7f4bfc1591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.359+0000 7f4c01600640 1 -- 172.21.15.119:0/3734430973 shutdown_connections 2024-04-25T10:14:59.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.359+0000 7f4c01600640 1 -- 172.21.15.119:0/3734430973 wait complete. 2024-04-25T10:14:59.367 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:14:59.360+0000 7fb52d000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d54ff0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:14:59.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:59.380 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:14:59.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.563+0000 7f79cf600640 1 Processor -- start 2024-04-25T10:14:59.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.563+0000 7f79cf600640 1 -- start start 2024-04-25T10:14:59.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.564+0000 7f79cf600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79c8144e70 0x7f79c806d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.564+0000 7f79cf600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f79c80cf0e0 0x7f79c80cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.564+0000 7f79cf600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c806dcb0 con 0x7f79c806d7a0 2024-04-25T10:14:59.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.564+0000 7f79cf600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c815bfe0 con 0x7f79c80cf0e0 2024-04-25T10:14:59.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.564+0000 7f79cf600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c806df90 con 0x7f79c8144e70 2024-04-25T10:14:59.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.565+0000 7f79ccc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79c8144e70 0x7f79c806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.565+0000 7f79cd600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f79c80cf0e0 0x7f79c80cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.565+0000 7f79ccc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79c8144e70 0x7f79c806d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:51722/0 (socket says 172.21.15.119:51722) 2024-04-25T10:14:59.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.565+0000 7f79ccc00640 1 -- 172.21.15.119:0/2491202254 learned_addr learned my addr 172.21.15.119:0/2491202254 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:59.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.566+0000 7f79ccc00640 1 -- 172.21.15.119:0/2491202254 >> v2:172.21.15.132:3300/0 conn(0x7f79c80cf0e0 msgr2=0x7f79c80cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.566+0000 7f79ccc00640 1 --2- 172.21.15.119:0/2491202254 >> v2:172.21.15.132:3300/0 conn(0x7f79c80cf0e0 0x7f79c80cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.566+0000 7f79ccc00640 1 -- 172.21.15.119:0/2491202254 >> v1:172.21.15.132:6789/0 conn(0x7f79c806d7a0 legacy=0x7f79c80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.566+0000 7f79ccc00640 1 -- 172.21.15.119:0/2491202254 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c806e270 con 0x7f79c8144e70 2024-04-25T10:14:59.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.567+0000 7f79ccc00640 1 --2- 172.21.15.119:0/2491202254 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79c8144e70 0x7f79c806d260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009870 tx=0x7f79ac02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bca1ef37644f1a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.568+0000 7f79c7e00640 1 -- 172.21.15.119:0/2491202254 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac037070 con 0x7f79c8144e70 2024-04-25T10:14:59.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.568+0000 7f79c7e00640 1 -- 172.21.15.119:0/2491202254 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ac032440 con 0x7f79c8144e70 2024-04-25T10:14:59.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.568+0000 7f79c7e00640 1 -- 172.21.15.119:0/2491202254 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac032810 con 0x7f79c8144e70 2024-04-25T10:14:59.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.569+0000 7f79cf600640 1 -- 172.21.15.119:0/2491202254 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79c8144e70 msgr2=0x7f79c806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.569+0000 7f79cf600640 1 --2- 172.21.15.119:0/2491202254 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79c8144e70 0x7f79c806d260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009870 tx=0x7f79ac02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.569+0000 7f79cf600640 1 -- 172.21.15.119:0/2491202254 shutdown_connections 2024-04-25T10:14:59.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.569+0000 7f79cf600640 1 --2- 172.21.15.119:0/2491202254 >> v2:172.21.15.132:3300/0 conn(0x7f79c80cf0e0 0x7f79c80cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.569+0000 7f79cf600640 1 --2- 172.21.15.119:0/2491202254 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79c8144e70 0x7f79c806d260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.569+0000 7f79cf600640 1 -- 172.21.15.119:0/2491202254 >> 172.21.15.119:0/2491202254 conn(0x7f79c806aba0 msgr2=0x7f79c806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.569+0000 7f79cf600640 1 -- 172.21.15.119:0/2491202254 shutdown_connections 2024-04-25T10:14:59.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.569+0000 7f79cf600640 1 -- 172.21.15.119:0/2491202254 wait complete. 2024-04-25T10:14:59.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.571+0000 7f79cf600640 1 Processor -- start 2024-04-25T10:14:59.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.571+0000 7f79cf600640 1 -- start start 2024-04-25T10:14:59.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.572+0000 7f79cf600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79c806d7a0 0x7f79c815a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.572+0000 7f79cf600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c80cf0e0 0x7f79c815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.572+0000 7f79cf600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79c8144e70 0x7f79c81e4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.572+0000 7f79cf600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c8145630 con 0x7f79c80cf0e0 2024-04-25T10:14:59.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.572+0000 7f79cf600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c81454b0 con 0x7f79c806d7a0 2024-04-25T10:14:59.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.572+0000 7f79cf600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c80cc840 con 0x7f79c8144e70 2024-04-25T10:14:59.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.572+0000 7f79bfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c80cf0e0 0x7f79c815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.572+0000 7f79bfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c80cf0e0 0x7f79c815aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60748/0 (socket says 172.21.15.119:60748) 2024-04-25T10:14:59.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.572+0000 7f79bfe00640 1 -- 172.21.15.119:0/3119398620 learned_addr learned my addr 172.21.15.119:0/3119398620 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:59.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.573+0000 7f79ccc00640 1 --2- 172.21.15.119:0/3119398620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79c806d7a0 0x7f79c815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.573+0000 7f79cd600640 1 --2- 172.21.15.119:0/3119398620 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79c8144e70 0x7f79c81e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.573+0000 7f79bfe00640 1 -- 172.21.15.119:0/3119398620 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79c8144e70 msgr2=0x7f79c81e4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.573+0000 7f79bfe00640 1 --2- 172.21.15.119:0/3119398620 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79c8144e70 0x7f79c81e4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.573+0000 7f79bfe00640 1 -- 172.21.15.119:0/3119398620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79c806d7a0 msgr2=0x7f79c815a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.573+0000 7f79bfe00640 1 --2- 172.21.15.119:0/3119398620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79c806d7a0 0x7f79c815a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.573+0000 7f79bfe00640 1 -- 172.21.15.119:0/3119398620 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c813c970 con 0x7f79c80cf0e0 2024-04-25T10:14:59.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.573+0000 7f79bfe00640 1 --2- 172.21.15.119:0/3119398620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c80cf0e0 0x7f79c815aaa0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f79b400c3c0 tx=0x7f79b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.574+0000 7f79c5600640 1 -- 172.21.15.119:0/3119398620 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4017070 con 0x7f79c80cf0e0 2024-04-25T10:14:59.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.574+0000 7f79c5600640 1 -- 172.21.15.119:0/3119398620 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b40052f0 con 0x7f79c80cf0e0 2024-04-25T10:14:59.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.574+0000 7f79cf600640 1 -- 172.21.15.119:0/3119398620 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c81418c0 con 0x7f79c80cf0e0 2024-04-25T10:14:59.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.574+0000 7f79c5600640 1 -- 172.21.15.119:0/3119398620 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4003850 con 0x7f79c80cf0e0 2024-04-25T10:14:59.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.574+0000 7f79cf600640 1 -- 172.21.15.119:0/3119398620 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c813cf10 con 0x7f79c80cf0e0 2024-04-25T10:14:59.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.576+0000 7f79c5600640 1 -- 172.21.15.119:0/3119398620 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79b401a070 con 0x7f79c80cf0e0 2024-04-25T10:14:59.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.577+0000 7f79c5600640 1 --2- 172.21.15.119:0/3119398620 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f799c06b500 0x7f799c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.578+0000 7f79c5600640 1 -- 172.21.15.119:0/3119398620 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f79b40896f0 con 0x7f79c80cf0e0 2024-04-25T10:14:59.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.578+0000 7f79ccc00640 1 --2- 172.21.15.119:0/3119398620 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f799c06b500 0x7f799c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.578+0000 7f79d2473740 1 Processor -- start 2024-04-25T10:14:59.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.578+0000 7f79ccc00640 1 --2- 172.21.15.119:0/3119398620 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f799c06b500 0x7f799c06d9c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f79ac034000 tx=0x7f79ac02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.579+0000 7f79d2473740 1 -- start start 2024-04-25T10:14:59.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.579+0000 7f79d2473740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a83a49a0 0x5642a83a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.579+0000 7f79d2473740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a83c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.579+0000 7f79d2473740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83c3d00 0x5642a837bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.579+0000 7f79d2473740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642a82e0c70 con 0x5642a837ed50 2024-04-25T10:14:59.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.579+0000 7f79d2473740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642a8289cf0 con 0x5642a83c3d00 2024-04-25T10:14:59.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.579+0000 7f79d2473740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642a82dcd10 con 0x5642a83a49a0 2024-04-25T10:14:59.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.580+0000 7f79bfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a83c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.580+0000 7f79bfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a83c37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60762/0 (socket says 172.21.15.119:60762) 2024-04-25T10:14:59.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.580+0000 7f79bfe00640 1 -- 172.21.15.119:0/737261747 learned_addr learned my addr 172.21.15.119:0/737261747 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:59.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.580+0000 7f79ccc00640 1 --2- 172.21.15.119:0/737261747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a83a49a0 0x5642a83a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.580+0000 7f79bfe00640 1 -- 172.21.15.119:0/737261747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a83a49a0 msgr2=0x5642a83a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.580+0000 7f79bfe00640 1 --2- 172.21.15.119:0/737261747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a83a49a0 0x5642a83a4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.580+0000 7f79cd600640 1 --2- 172.21.15.119:0/737261747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83c3d00 0x5642a837bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.580+0000 7f79bfe00640 1 -- 172.21.15.119:0/737261747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83c3d00 msgr2=0x5642a837bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.580+0000 7f79bfe00640 1 --2- 172.21.15.119:0/737261747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83c3d00 0x5642a837bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.580+0000 7f79bfe00640 1 -- 172.21.15.119:0/737261747 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642a80bf7d0 con 0x5642a837ed50 2024-04-25T10:14:59.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.580+0000 7f79bfe00640 1 --2- 172.21.15.119:0/737261747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a83c37c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f79b40540e0 tx=0x7f79b4057d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860dea385ee2b13c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.581+0000 7f79bd200640 1 -- 172.21.15.119:0/737261747 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b405e0c0 con 0x5642a837ed50 2024-04-25T10:14:59.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.581+0000 7f79bd200640 1 -- 172.21.15.119:0/737261747 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b4055f80 con 0x5642a837ed50 2024-04-25T10:14:59.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.581+0000 7f79bd200640 1 -- 172.21.15.119:0/737261747 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b40570d0 con 0x5642a837ed50 2024-04-25T10:14:59.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.581+0000 7f79d2473740 1 -- 172.21.15.119:0/737261747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 msgr2=0x5642a83c37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.581+0000 7f79d2473740 1 --2- 172.21.15.119:0/737261747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a83c37c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f79b40540e0 tx=0x7f79b4057d40 comp rx=0 tx=0).stop 2024-04-25T10:14:59.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.581+0000 7f79d2473740 1 -- 172.21.15.119:0/737261747 shutdown_connections 2024-04-25T10:14:59.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.581+0000 7f79d2473740 1 --2- 172.21.15.119:0/737261747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83c3d00 0x5642a837bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.581+0000 7f79d2473740 1 --2- 172.21.15.119:0/737261747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a83c37c0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.581+0000 7f79d2473740 1 --2- 172.21.15.119:0/737261747 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a83a49a0 0x5642a83a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.581+0000 7f79d2473740 1 -- 172.21.15.119:0/737261747 >> 172.21.15.119:0/737261747 conn(0x5642a8344a30 msgr2=0x5642a83c2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.581+0000 7f79d2473740 1 -- 172.21.15.119:0/737261747 shutdown_connections 2024-04-25T10:14:59.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.581+0000 7f79d2473740 1 -- 172.21.15.119:0/737261747 wait complete. 2024-04-25T10:14:59.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.582+0000 7f79d2473740 1 -- start start 2024-04-25T10:14:59.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.582+0000 7f79d2473740 1 Processor -- start 2024-04-25T10:14:59.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.583+0000 7f79d2473740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a837ed50 0x5642a83d70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.583+0000 7f79d2473740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a83a49a0 0x5642a83d7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.583+0000 7f79d2473740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83a53d0 0x5642a83a5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.583+0000 7f79d2473740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642a83791b0 con 0x5642a83a49a0 2024-04-25T10:14:59.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.583+0000 7f79d2473740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642a827f3a0 con 0x5642a83a53d0 2024-04-25T10:14:59.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.583+0000 7f79d2473740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642a837dce0 con 0x5642a837ed50 2024-04-25T10:14:59.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.583+0000 7f79d2473740 10 client.0 _ll_register_callbacks cb 0x5642a829cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:59.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.584+0000 7f79bfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a83a49a0 0x5642a83d7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.584+0000 7f79bfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a83a49a0 0x5642a83d7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60768/0 (socket says 172.21.15.119:60768) 2024-04-25T10:14:59.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.584+0000 7f79bfe00640 1 -- 172.21.15.119:0/275742815 learned_addr learned my addr 172.21.15.119:0/275742815 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:59.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.584+0000 7f79bfe00640 1 -- 172.21.15.119:0/275742815 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a837ed50 msgr2=0x5642a83d70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.584+0000 7f79bfe00640 1 --2- 172.21.15.119:0/275742815 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a837ed50 0x5642a83d70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.584+0000 7f79bfe00640 1 -- 172.21.15.119:0/275742815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83a53d0 msgr2=0x5642a83a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.584+0000 7f79bfe00640 1 --2- 172.21.15.119:0/275742815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83a53d0 0x5642a83a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.584+0000 7f79bfe00640 1 -- 172.21.15.119:0/275742815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642a7ff53f0 con 0x5642a83a49a0 2024-04-25T10:14:59.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.584+0000 7f79bfe00640 1 --2- 172.21.15.119:0/275742815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a83a49a0 0x5642a83d7600 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f79b400cd80 tx=0x7f79b4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f114e279dff1aeb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.584+0000 7f79ba000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:59.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.584+0000 7f79ba000640 1 -- 172.21.15.119:0/275742815 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b405e0c0 con 0x5642a83a49a0 2024-04-25T10:14:59.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.584+0000 7f79ba000640 1 -- 172.21.15.119:0/275742815 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f79b40570d0 con 0x5642a83a49a0 2024-04-25T10:14:59.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.584+0000 7f79d2473740 1 -- 172.21.15.119:0/275742815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79b4055f80 con 0x5642a83a49a0 2024-04-25T10:14:59.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.585+0000 7f79ba000640 1 -- 172.21.15.119:0/275742815 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79b4087050 con 0x5642a83a49a0 2024-04-25T10:14:59.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.585+0000 7f79d2473740 10 client.7782 fetch_fsmap learned FSMap version 74 2024-04-25T10:14:59.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.585+0000 7f79d2473740 1 -- 172.21.15.119:0/275742815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x5642a7fb50f0 con 0x5642a83a49a0 2024-04-25T10:14:59.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.585+0000 7f79ba000640 1 -- 172.21.15.119:0/275742815 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b4055f80 con 0x5642a83a49a0 2024-04-25T10:14:59.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.585+0000 7f79ba000640 1 -- 172.21.15.119:0/275742815 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f79b40539d0 con 0x5642a83a49a0 2024-04-25T10:14:59.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.585+0000 7f79d2473740 10 client.7782 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:14:59.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.585+0000 7f79d2473740 10 client.7782 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:14:59.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.585+0000 7f79d2473740 20 client.7782 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:59.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.585+0000 7f79d2473740 1 --2- 172.21.15.119:0/275742815 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x5642a83e17e0 0x5642a83e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.585+0000 7f79d2473740 4 client.7782 mds_command: new command op to 7668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:59.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.585+0000 7f79d2473740 1 -- 172.21.15.119:0/275742815 --> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642a82ee1b0 con 0x5642a83e17e0 2024-04-25T10:14:59.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.586+0000 7f79ccc00640 1 --2- 172.21.15.119:0/275742815 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x5642a83e17e0 0x5642a83e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.586+0000 7f79ba000640 10 client.7782 ms_handle_connect on v2:172.21.15.119:6835/2043821095 2024-04-25T10:14:59.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.586+0000 7f79ccc00640 1 --2- 172.21.15.119:0/275742815 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x5642a83e17e0 0x5642a83e3bd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf7dd252bd1e4bf server_cookie=85a98e23e6c19568 in_seq=0 out_seq=0 2024-04-25T10:14:59.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.587+0000 7f79ba000640 1 -- 172.21.15.119:0/275742815 <== mds.0 v2:172.21.15.119:6835/2043821095 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5642a82ee1b0 con 0x5642a83e17e0 2024-04-25T10:14:59.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.587+0000 7f79ba000640 10 client.7782 handle_command_reply: tid=0 2024-04-25T10:14:59.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.587+0000 7f79d2473740 1 client.7782 shutdown 2024-04-25T10:14:59.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.587+0000 7f79d2473740 1 -- 172.21.15.119:0/275742815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a83a49a0 msgr2=0x5642a83d7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.587+0000 7f79d2473740 1 --2- 172.21.15.119:0/275742815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a83a49a0 0x5642a83d7600 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f79b400cd80 tx=0x7f79b4000bc0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.587+0000 7f79d2473740 1 -- 172.21.15.119:0/275742815 shutdown_connections 2024-04-25T10:14:59.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.587+0000 7f79d2473740 1 --2- 172.21.15.119:0/275742815 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x5642a83e17e0 0x5642a83e3bd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.587+0000 7f79d2473740 1 --2- 172.21.15.119:0/275742815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83a53d0 0x5642a83a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.587+0000 7f79d2473740 1 --2- 172.21.15.119:0/275742815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a83a49a0 0x5642a83d7600 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.587+0000 7f79d2473740 1 --2- 172.21.15.119:0/275742815 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a837ed50 0x5642a83d70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.587+0000 7f79ba000640 0 client.7782 ms_handle_reset on v2:172.21.15.119:6835/2043821095 2024-04-25T10:14:59.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.587+0000 7f79d2473740 1 -- 172.21.15.119:0/275742815 >> 172.21.15.119:0/275742815 conn(0x5642a8344a30 msgr2=0x5642a83a3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.588+0000 7f79d2473740 1 -- 172.21.15.119:0/275742815 shutdown_connections 2024-04-25T10:14:59.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.588+0000 7f79d2473740 1 -- 172.21.15.119:0/275742815 wait complete. 2024-04-25T10:14:59.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.588+0000 7f79d2473740 20 client.7782 trim_cache size 0 max 16384 2024-04-25T10:14:59.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.604+0000 7f79d2473740 1 Processor -- start 2024-04-25T10:14:59.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.604+0000 7f79d2473740 1 -- start start 2024-04-25T10:14:59.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.605+0000 7f79d2473740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a837b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.605+0000 7f79d2473740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83a49a0 0x5642a837a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.605+0000 7f79d2473740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a83a53d0 0x5642a83a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.605+0000 7f79d2473740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c8143ff0 con 0x5642a837ed50 2024-04-25T10:14:59.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.605+0000 7f79d2473740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c80b47a0 con 0x5642a83a49a0 2024-04-25T10:14:59.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.605+0000 7f79d2473740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c80b3ba0 con 0x5642a83a53d0 2024-04-25T10:14:59.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.605+0000 7f79ccc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a837b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.605+0000 7f79ccc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a837b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60774/0 (socket says 172.21.15.119:60774) 2024-04-25T10:14:59.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.605+0000 7f79ccc00640 1 -- 172.21.15.119:0/32184818 learned_addr learned my addr 172.21.15.119:0/32184818 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:59.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.606+0000 7f79cd600640 1 --2- 172.21.15.119:0/32184818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a83a53d0 0x5642a83a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.606+0000 7f79bfe00640 1 --2- 172.21.15.119:0/32184818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83a49a0 0x5642a837a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.606+0000 7f79ccc00640 1 -- 172.21.15.119:0/32184818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a83a53d0 msgr2=0x5642a83a3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.606+0000 7f79ccc00640 1 --2- 172.21.15.119:0/32184818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a83a53d0 0x5642a83a3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.606+0000 7f79ccc00640 1 -- 172.21.15.119:0/32184818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83a49a0 msgr2=0x5642a837a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.606+0000 7f79ccc00640 1 --2- 172.21.15.119:0/32184818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83a49a0 0x5642a837a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.606+0000 7f79ccc00640 1 -- 172.21.15.119:0/32184818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642a83a5ba0 con 0x5642a837ed50 2024-04-25T10:14:59.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.606+0000 7f79ccc00640 1 --2- 172.21.15.119:0/32184818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a837b580 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f79ac000c00 tx=0x7f79ac02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960db31e62a0c7c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.606+0000 7f79bc800640 1 -- 172.21.15.119:0/32184818 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac050030 con 0x5642a837ed50 2024-04-25T10:14:59.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.606+0000 7f79bc800640 1 -- 172.21.15.119:0/32184818 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ac032d60 con 0x5642a837ed50 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.607+0000 7f79bc800640 1 -- 172.21.15.119:0/32184818 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac037040 con 0x5642a837ed50 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.607+0000 7f79d2473740 1 -- 172.21.15.119:0/32184818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 msgr2=0x5642a837b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.607+0000 7f79d2473740 1 --2- 172.21.15.119:0/32184818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a837b580 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f79ac000c00 tx=0x7f79ac02c000 comp rx=0 tx=0).stop 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.607+0000 7f79d2473740 1 -- 172.21.15.119:0/32184818 shutdown_connections 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.607+0000 7f79d2473740 1 --2- 172.21.15.119:0/32184818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a83a53d0 0x5642a83a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.607+0000 7f79d2473740 1 --2- 172.21.15.119:0/32184818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83a49a0 0x5642a837a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.607+0000 7f79d2473740 1 --2- 172.21.15.119:0/32184818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a837b580 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.607+0000 7f79d2473740 1 -- 172.21.15.119:0/32184818 >> 172.21.15.119:0/32184818 conn(0x5642a8344a30 msgr2=0x5642a83ab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.607+0000 7f79d2473740 1 -- 172.21.15.119:0/32184818 shutdown_connections 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.607+0000 7f79d2473740 1 -- 172.21.15.119:0/32184818 wait complete. 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.608+0000 7f79d2473740 1 -- start start 2024-04-25T10:14:59.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.608+0000 7f79d2473740 1 Processor -- start 2024-04-25T10:14:59.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.609+0000 7f79d2473740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a8430640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.609+0000 7f79d2473740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83a49a0 0x5642a83a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.609+0000 7f79d2473740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a83a53d0 0x5642a83a3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.609+0000 7f79d2473740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642a83e0460 con 0x5642a837ed50 2024-04-25T10:14:59.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.609+0000 7f79d2473740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c80b41a0 con 0x5642a83a49a0 2024-04-25T10:14:59.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.609+0000 7f79d2473740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642a83cdf50 con 0x5642a83a53d0 2024-04-25T10:14:59.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.609+0000 7f79d2473740 10 client.0 _ll_register_callbacks cb 0x7f79b4058380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:59.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.609+0000 7f79ccc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a8430640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.609+0000 7f79ccc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a8430640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60790/0 (socket says 172.21.15.119:60790) 2024-04-25T10:14:59.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.609+0000 7f79ccc00640 1 -- 172.21.15.119:0/469494225 learned_addr learned my addr 172.21.15.119:0/469494225 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:14:59.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.610+0000 7f79ccc00640 1 -- 172.21.15.119:0/469494225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a83a53d0 msgr2=0x5642a83a3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.610+0000 7f79ccc00640 1 --2- 172.21.15.119:0/469494225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a83a53d0 0x5642a83a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.610+0000 7f79ccc00640 1 -- 172.21.15.119:0/469494225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83a49a0 msgr2=0x5642a83a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.610+0000 7f79ccc00640 1 --2- 172.21.15.119:0/469494225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83a49a0 0x5642a83a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.610+0000 7f79ccc00640 1 -- 172.21.15.119:0/469494225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642a83e7f20 con 0x5642a837ed50 2024-04-25T10:14:59.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.610+0000 7f79ccc00640 1 --2- 172.21.15.119:0/469494225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a8430640 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f79ac02a7b0 tx=0x7f79ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e5eed84bb296ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.610+0000 7f79ba000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:14:59.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.610+0000 7f79ba000640 1 -- 172.21.15.119:0/469494225 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac037070 con 0x5642a837ed50 2024-04-25T10:14:59.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.610+0000 7f79ba000640 1 -- 172.21.15.119:0/469494225 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f79ac04c380 con 0x5642a837ed50 2024-04-25T10:14:59.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.610+0000 7f79d2473740 1 -- 172.21.15.119:0/469494225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79ac032d60 con 0x5642a837ed50 2024-04-25T10:14:59.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.611+0000 7f79ba000640 1 -- 172.21.15.119:0/469494225 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79ac062050 con 0x5642a837ed50 2024-04-25T10:14:59.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.611+0000 7f79d2473740 10 client.7788 fetch_fsmap learned FSMap version 74 2024-04-25T10:14:59.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.611+0000 7f79d2473740 1 -- 172.21.15.119:0/469494225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x5642a839d9e0 con 0x5642a837ed50 2024-04-25T10:14:59.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.611+0000 7f79ba000640 1 -- 172.21.15.119:0/469494225 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ac032d60 con 0x5642a837ed50 2024-04-25T10:14:59.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.611+0000 7f79ba000640 1 -- 172.21.15.119:0/469494225 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f79ac065020 con 0x5642a837ed50 2024-04-25T10:14:59.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.611+0000 7f79d2473740 10 client.7788 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:14:59.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.611+0000 7f79d2473740 10 client.7788 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:14:59.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.611+0000 7f79d2473740 20 client.7788 populate_metadata read hostname 'smithi119' 2024-04-25T10:14:59.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.611+0000 7f79d2473740 1 --2- 172.21.15.119:0/469494225 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x5642a839d580 0x5642a8376f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.611+0000 7f79d2473740 4 client.7788 mds_command: new command op to 7668 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:14:59.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.611+0000 7f79d2473740 1 -- 172.21.15.119:0/469494225 --> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5642a82e8500 con 0x5642a839d580 2024-04-25T10:14:59.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.611+0000 7f79bfe00640 1 --2- 172.21.15.119:0/469494225 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x5642a839d580 0x5642a8376f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.612+0000 7f79bfe00640 1 --2- 172.21.15.119:0/469494225 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x5642a839d580 0x5642a8376f50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=922fc358b2c72293 server_cookie=5b1baa91fe496841 in_seq=0 out_seq=0 2024-04-25T10:14:59.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.612+0000 7f79ba000640 10 client.7788 ms_handle_connect on v2:172.21.15.119:6835/2043821095 2024-04-25T10:14:59.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.612+0000 7f79ba000640 1 -- 172.21.15.119:0/469494225 <== mds.0 v2:172.21.15.119:6835/2043821095 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x5642a82e8500 con 0x5642a839d580 2024-04-25T10:14:59.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.612+0000 7f79ba000640 10 client.7788 handle_command_reply: tid=0 2024-04-25T10:14:59.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.612+0000 7f79d2473740 1 client.7788 shutdown 2024-04-25T10:14:59.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.612+0000 7f79d2473740 1 -- 172.21.15.119:0/469494225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 msgr2=0x5642a8430640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.612+0000 7f79d2473740 1 --2- 172.21.15.119:0/469494225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a8430640 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f79ac02a7b0 tx=0x7f79ac004730 comp rx=0 tx=0).stop 2024-04-25T10:14:59.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.612+0000 7f79d2473740 1 -- 172.21.15.119:0/469494225 shutdown_connections 2024-04-25T10:14:59.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.612+0000 7f79d2473740 1 --2- 172.21.15.119:0/469494225 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x5642a839d580 0x5642a8376f50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.612+0000 7f79d2473740 1 --2- 172.21.15.119:0/469494225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5642a83a53d0 0x5642a83a3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.612+0000 7f79d2473740 1 --2- 172.21.15.119:0/469494225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5642a83a49a0 0x5642a83a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.612+0000 7f79d2473740 1 --2- 172.21.15.119:0/469494225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642a837ed50 0x5642a8430640 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.612+0000 7f79d2473740 1 -- 172.21.15.119:0/469494225 >> 172.21.15.119:0/469494225 conn(0x5642a8344a30 msgr2=0x5642a837a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.612+0000 7f79ba000640 0 client.7788 ms_handle_reset on v2:172.21.15.119:6835/2043821095 2024-04-25T10:14:59.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.613+0000 7f79d2473740 1 -- 172.21.15.119:0/469494225 shutdown_connections 2024-04-25T10:14:59.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.613+0000 7f79d2473740 1 -- 172.21.15.119:0/469494225 wait complete. 2024-04-25T10:14:59.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.613+0000 7f79d2473740 20 client.7788 trim_cache size 0 max 16384 2024-04-25T10:14:59.621 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:14:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:14:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2024-04-25T10:14:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "c2ac8ee9-b67f-4fda-a3c7-2f444331a7f1": { 2024-04-25T10:14:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/dir", 2024-04-25T10:14:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "c2ac8ee9-b67f-4fda-a3c7-2f444331a7f1", 2024-04-25T10:14:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,repair" 2024-04-25T10:14:59.621 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:14:59.622 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:14:59.622 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:14:59.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.613+0000 7f79cf600640 1 -- 172.21.15.119:0/3119398620 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f799c06b500 msgr2=0x7f799c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.613+0000 7f79cf600640 1 --2- 172.21.15.119:0/3119398620 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f799c06b500 0x7f799c06d9c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f79ac034000 tx=0x7f79ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:14:59.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.613+0000 7f79cf600640 1 -- 172.21.15.119:0/3119398620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c80cf0e0 msgr2=0x7f79c815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.613+0000 7f79cf600640 1 --2- 172.21.15.119:0/3119398620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c80cf0e0 0x7f79c815aaa0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f79b400c3c0 tx=0x7f79b400c880 comp rx=0 tx=0).stop 2024-04-25T10:14:59.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.613+0000 7f79cf600640 1 -- 172.21.15.119:0/3119398620 shutdown_connections 2024-04-25T10:14:59.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.613+0000 7f79cf600640 1 --2- 172.21.15.119:0/3119398620 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f799c06b500 0x7f799c06d9c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.613+0000 7f79cf600640 1 --2- 172.21.15.119:0/3119398620 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f79c8144e70 0x7f79c81e4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.613+0000 7f79cf600640 1 --2- 172.21.15.119:0/3119398620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c80cf0e0 0x7f79c815aaa0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.613+0000 7f79cf600640 1 --2- 172.21.15.119:0/3119398620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79c806d7a0 0x7f79c815a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.613+0000 7f79cf600640 1 -- 172.21.15.119:0/3119398620 >> 172.21.15.119:0/3119398620 conn(0x7f79c806aba0 msgr2=0x7f79c80cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.614+0000 7f79cf600640 1 -- 172.21.15.119:0/3119398620 shutdown_connections 2024-04-25T10:14:59.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:14:59.614+0000 7f79cf600640 1 -- 172.21.15.119:0/3119398620 wait complete. 2024-04-25T10:14:59.634 INFO:tasks.cephfs.filesystem:scrub status for tag:c2ac8ee9-b67f-4fda-a3c7-2f444331a7f1 - {'path': '/dir', 'tag': 'c2ac8ee9-b67f-4fda-a3c7-2f444331a7f1', 'options': 'recursive,repair'} 2024-04-25T10:15:02.115 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:02.123 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:29.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:29.635 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:15:29.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.848+0000 7f8bdec00640 1 Processor -- start 2024-04-25T10:15:29.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.848+0000 7f8bdec00640 1 -- start start 2024-04-25T10:15:29.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.850+0000 7f8bdec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd815d980 0x7f8bd80ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.850+0000 7f8bdec00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8bd80b2440 0x7f8bd80ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.850+0000 7f8bdec00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd80b6ac0 con 0x7f8bd80b2a00 2024-04-25T10:15:29.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.850+0000 7f8bdec00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd80cefe0 con 0x7f8bd80b2440 2024-04-25T10:15:29.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.850+0000 7f8bdec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80b70d0 con 0x7f8bd815d980 2024-04-25T10:15:29.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.850+0000 7f8bde200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8bd80b2440 0x7f8bd80ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.850+0000 7f8bde200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8bd80b2440 0x7f8bd80ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53724/0 (socket says 172.21.15.119:53724) 2024-04-25T10:15:29.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.850+0000 7f8bde200640 1 -- 172.21.15.119:0/2151137001 learned_addr learned my addr 172.21.15.119:0/2151137001 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:29.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.850+0000 7f8bdd800640 1 --2- 172.21.15.119:0/2151137001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd815d980 0x7f8bd80ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.851+0000 7f8bde200640 1 -- 172.21.15.119:0/2151137001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd815d980 msgr2=0x7f8bd80ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:29.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.851+0000 7f8bde200640 1 --2- 172.21.15.119:0/2151137001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd815d980 0x7f8bd80ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.851+0000 7f8bde200640 1 -- 172.21.15.119:0/2151137001 >> v1:172.21.15.132:6789/0 conn(0x7f8bd80b2a00 legacy=0x7f8bd80b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:29.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.851+0000 7f8bde200640 1 -- 172.21.15.119:0/2151137001 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd80b94f0 con 0x7f8bd80b2440 2024-04-25T10:15:29.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.852+0000 7f8bde200640 1 --2- 172.21.15.119:0/2151137001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd80b2440 0x7f8bd80ceaa0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009e50 tx=0x7f8bd002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299a14b6df244fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:29.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.852+0000 7f8bd7e00640 1 -- 172.21.15.119:0/2151137001 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0036070 con 0x7f8bd80b2440 2024-04-25T10:15:29.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.852+0000 7f8bd7e00640 1 -- 172.21.15.119:0/2151137001 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd0031600 con 0x7f8bd80b2440 2024-04-25T10:15:29.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.852+0000 7f8bd7e00640 1 -- 172.21.15.119:0/2151137001 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00319d0 con 0x7f8bd80b2440 2024-04-25T10:15:29.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.853+0000 7f8bdec00640 1 -- 172.21.15.119:0/2151137001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd80b2440 msgr2=0x7f8bd80ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:29.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.853+0000 7f8bdec00640 1 --2- 172.21.15.119:0/2151137001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd80b2440 0x7f8bd80ceaa0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009e50 tx=0x7f8bd002a6c0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.853+0000 7f8bdec00640 1 -- 172.21.15.119:0/2151137001 shutdown_connections 2024-04-25T10:15:29.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.853+0000 7f8bdec00640 1 --2- 172.21.15.119:0/2151137001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd80b2440 0x7f8bd80ceaa0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.853+0000 7f8bdec00640 1 --2- 172.21.15.119:0/2151137001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd815d980 0x7f8bd80ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.853+0000 7f8bdec00640 1 -- 172.21.15.119:0/2151137001 >> 172.21.15.119:0/2151137001 conn(0x7f8bd80726c0 msgr2=0x7f8bd8072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:29.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.853+0000 7f8bdec00640 1 -- 172.21.15.119:0/2151137001 shutdown_connections 2024-04-25T10:15:29.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.853+0000 7f8bdec00640 1 -- 172.21.15.119:0/2151137001 wait complete. 2024-04-25T10:15:29.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.855+0000 7f8bdec00640 1 Processor -- start 2024-04-25T10:15:29.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.855+0000 7f8bdec00640 1 -- start start 2024-04-25T10:15:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.856+0000 7f8bdec00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd80b2440 0x7f8bd8154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.856+0000 7f8bdec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd80b2a00 0x7f8bd8152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.856+0000 7f8bdec00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd815d980 0x7f8bd815edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.856+0000 7f8bdec00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd815dd60 con 0x7f8bd815d980 2024-04-25T10:15:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.856+0000 7f8bdec00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80b97d0 con 0x7f8bd80b2440 2024-04-25T10:15:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.856+0000 7f8bdec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80b9110 con 0x7f8bd80b2a00 2024-04-25T10:15:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.856+0000 7f8bde200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd815d980 0x7f8bd815edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.856+0000 7f8bde200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd815d980 0x7f8bd815edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53734/0 (socket says 172.21.15.119:53734) 2024-04-25T10:15:29.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.856+0000 7f8bde200640 1 -- 172.21.15.119:0/2989689547 learned_addr learned my addr 172.21.15.119:0/2989689547 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.856+0000 7f8bdd800640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd80b2440 0x7f8bd8154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.856+0000 7f8bde200640 1 -- 172.21.15.119:0/2989689547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd80b2a00 msgr2=0x7f8bd8152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.856+0000 7f8bdce00640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd80b2a00 0x7f8bd8152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.857+0000 7f8bde200640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd80b2a00 0x7f8bd8152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.857+0000 7f8bde200640 1 -- 172.21.15.119:0/2989689547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd80b2440 msgr2=0x7f8bd8154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.857+0000 7f8bde200640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd80b2440 0x7f8bd8154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.857+0000 7f8bde200640 1 -- 172.21.15.119:0/2989689547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd81532f0 con 0x7f8bd815d980 2024-04-25T10:15:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.857+0000 7f8bdce00640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd80b2a00 0x7f8bd8152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.857+0000 7f8bde200640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd815d980 0x7f8bd815edc0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009e50 tx=0x7f8bd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:29.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.857+0000 7f8bd5600640 1 -- 172.21.15.119:0/2989689547 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0036050 con 0x7f8bd815d980 2024-04-25T10:15:29.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.857+0000 7f8bd5600640 1 -- 172.21.15.119:0/2989689547 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd0042070 con 0x7f8bd815d980 2024-04-25T10:15:29.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.857+0000 7f8bd5600640 1 -- 172.21.15.119:0/2989689547 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0009510 con 0x7f8bd815d980 2024-04-25T10:15:29.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.857+0000 7f8bdec00640 1 -- 172.21.15.119:0/2989689547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd81540c0 con 0x7f8bd815d980 2024-04-25T10:15:29.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.857+0000 7f8bdec00640 1 -- 172.21.15.119:0/2989689547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd80cc4f0 con 0x7f8bd815d980 2024-04-25T10:15:29.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.859+0000 7f8bd5600640 1 -- 172.21.15.119:0/2989689547 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8bd0007bd0 con 0x7f8bd815d980 2024-04-25T10:15:29.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.860+0000 7f8bd5600640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8bb806b4d0 0x7f8bb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.860+0000 7f8bd5600640 1 -- 172.21.15.119:0/2989689547 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f8bd00ab1d0 con 0x7f8bd815d980 2024-04-25T10:15:29.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.861+0000 7f8bdd800640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8bb806b4d0 0x7f8bb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.861+0000 7f8be3185740 1 Processor -- start 2024-04-25T10:15:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.861+0000 7f8be3185740 1 -- start start 2024-04-25T10:15:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.861+0000 7f8bdd800640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8bb806b4d0 0x7f8bb806d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc006fd0 tx=0x7f8bcc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:29.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.862+0000 7f8be3185740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06baed70 0x560b06bb0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.862+0000 7f8be3185740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06beb440 0x560b06baf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.862+0000 7f8be3185740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06bd9830 0x560b06bd9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.862+0000 7f8be3185740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b06b12c70 con 0x560b06baed70 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.862+0000 7f8be3185740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b06abbcf0 con 0x560b06beb440 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.862+0000 7f8be3185740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b06b0ed10 con 0x560b06bd9830 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.863+0000 7f8bdd800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06baed70 0x560b06bb0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.863+0000 7f8bdd800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06baed70 0x560b06bb0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53748/0 (socket says 172.21.15.119:53748) 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.863+0000 7f8bdd800640 1 -- 172.21.15.119:0/2917761362 learned_addr learned my addr 172.21.15.119:0/2917761362 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.863+0000 7f8bde200640 1 --2- 172.21.15.119:0/2917761362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06bd9830 0x560b06bd9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.863+0000 7f8bdce00640 1 --2- 172.21.15.119:0/2917761362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06beb440 0x560b06baf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.863+0000 7f8bdd800640 1 -- 172.21.15.119:0/2917761362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06bd9830 msgr2=0x560b06bd9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.863+0000 7f8bdd800640 1 --2- 172.21.15.119:0/2917761362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06bd9830 0x560b06bd9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.863+0000 7f8bdd800640 1 -- 172.21.15.119:0/2917761362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06beb440 msgr2=0x560b06baf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.863+0000 7f8bdd800640 1 --2- 172.21.15.119:0/2917761362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06beb440 0x560b06baf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.863+0000 7f8bdd800640 1 -- 172.21.15.119:0/2917761362 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b068f17d0 con 0x560b06baed70 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.863+0000 7f8bde200640 1 --2- 172.21.15.119:0/2917761362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06bd9830 0x560b06bd9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:29.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.863+0000 7f8bdd800640 1 --2- 172.21.15.119:0/2917761362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06baed70 0x560b06bb0ed0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00e920 tx=0x7f8bcc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc790ecc6dba741 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:29.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.864+0000 7f8bca000640 1 -- 172.21.15.119:0/2917761362 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc025070 con 0x560b06baed70 2024-04-25T10:15:29.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.864+0000 7f8bca000640 1 -- 172.21.15.119:0/2917761362 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc015700 con 0x560b06baed70 2024-04-25T10:15:29.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.864+0000 7f8bca000640 1 -- 172.21.15.119:0/2917761362 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00f040 con 0x560b06baed70 2024-04-25T10:15:29.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.864+0000 7f8be3185740 1 -- 172.21.15.119:0/2917761362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06baed70 msgr2=0x560b06bb0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:29.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.864+0000 7f8be3185740 1 --2- 172.21.15.119:0/2917761362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06baed70 0x560b06bb0ed0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00e920 tx=0x7f8bcc011000 comp rx=0 tx=0).stop 2024-04-25T10:15:29.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.864+0000 7f8be3185740 1 -- 172.21.15.119:0/2917761362 shutdown_connections 2024-04-25T10:15:29.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.864+0000 7f8be3185740 1 --2- 172.21.15.119:0/2917761362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06bd9830 0x560b06bd9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.864+0000 7f8be3185740 1 --2- 172.21.15.119:0/2917761362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06beb440 0x560b06baf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.864+0000 7f8be3185740 1 --2- 172.21.15.119:0/2917761362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06baed70 0x560b06bb0ed0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.864+0000 7f8be3185740 1 -- 172.21.15.119:0/2917761362 >> 172.21.15.119:0/2917761362 conn(0x560b06b76a30 msgr2=0x560b06bf4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:29.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.865+0000 7f8be3185740 1 -- 172.21.15.119:0/2917761362 shutdown_connections 2024-04-25T10:15:29.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.865+0000 7f8be3185740 1 -- 172.21.15.119:0/2917761362 wait complete. 2024-04-25T10:15:29.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.865+0000 7f8be3185740 1 -- start start 2024-04-25T10:15:29.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.866+0000 7f8be3185740 1 Processor -- start 2024-04-25T10:15:29.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.866+0000 7f8be3185740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06baed70 0x560b06c08880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.866+0000 7f8be3185740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06bd9830 0x560b06bf8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.866+0000 7f8be3185740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06beb440 0x560b06bd6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.867+0000 7f8be3185740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b06ab13a0 con 0x560b06bd9830 2024-04-25T10:15:29.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.867+0000 7f8be3185740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b06bd7380 con 0x560b06beb440 2024-04-25T10:15:29.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.867+0000 7f8be3185740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b06c04df0 con 0x560b06baed70 2024-04-25T10:15:29.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.867+0000 7f8be3185740 10 client.0 _ll_register_callbacks cb 0x560b06aceb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:29.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.867+0000 7f8bdce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06bd9830 0x560b06bf8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.867+0000 7f8bdce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06bd9830 0x560b06bf8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53758/0 (socket says 172.21.15.119:53758) 2024-04-25T10:15:29.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.867+0000 7f8bdce00640 1 -- 172.21.15.119:0/1097403526 learned_addr learned my addr 172.21.15.119:0/1097403526 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:29.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.867+0000 7f8bdd800640 1 --2- 172.21.15.119:0/1097403526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06baed70 0x560b06c08880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.867+0000 7f8bdce00640 1 -- 172.21.15.119:0/1097403526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06baed70 msgr2=0x560b06c08880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:29.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.867+0000 7f8bdce00640 1 --2- 172.21.15.119:0/1097403526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06baed70 0x560b06c08880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.867+0000 7f8bdce00640 1 -- 172.21.15.119:0/1097403526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06beb440 msgr2=0x560b06bd6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:29.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.867+0000 7f8bde200640 1 --2- 172.21.15.119:0/1097403526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06beb440 0x560b06bd6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.867+0000 7f8bdce00640 1 --2- 172.21.15.119:0/1097403526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06beb440 0x560b06bd6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.867+0000 7f8bdce00640 1 -- 172.21.15.119:0/1097403526 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b068273f0 con 0x560b06bd9830 2024-04-25T10:15:29.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.868+0000 7f8bdce00640 1 --2- 172.21.15.119:0/1097403526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06bd9830 0x560b06bf8190 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f8bd808b390 tx=0x7f8bc400e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a2603ea03288c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:29.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.868+0000 7f8bbea00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:15:29.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.868+0000 7f8bbea00640 1 -- 172.21.15.119:0/1097403526 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4003a60 con 0x560b06bd9830 2024-04-25T10:15:29.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.868+0000 7f8bbea00640 1 -- 172.21.15.119:0/1097403526 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f8bc4003c00 con 0x560b06bd9830 2024-04-25T10:15:29.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.868+0000 7f8be3185740 1 -- 172.21.15.119:0/1097403526 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bcc015700 con 0x560b06bd9830 2024-04-25T10:15:29.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.868+0000 7f8bbea00640 1 -- 172.21.15.119:0/1097403526 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bc4020050 con 0x560b06bd9830 2024-04-25T10:15:29.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.868+0000 7f8be3185740 10 client.7800 fetch_fsmap learned FSMap version 74 2024-04-25T10:15:29.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.868+0000 7f8be3185740 1 -- 172.21.15.119:0/1097403526 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x560b067e70f0 con 0x560b06bd9830 2024-04-25T10:15:29.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.868+0000 7f8bbea00640 1 -- 172.21.15.119:0/1097403526 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc015700 con 0x560b06bd9830 2024-04-25T10:15:29.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.868+0000 7f8bbea00640 1 -- 172.21.15.119:0/1097403526 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8bc4023020 con 0x560b06bd9830 2024-04-25T10:15:29.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.868+0000 7f8be3185740 10 client.7800 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:15:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.869+0000 7f8be3185740 10 client.7800 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:15:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.869+0000 7f8be3185740 20 client.7800 populate_metadata read hostname 'smithi119' 2024-04-25T10:15:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.869+0000 7f8be3185740 1 --2- 172.21.15.119:0/1097403526 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x560b06befa80 0x560b06bdc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.869+0000 7f8be3185740 4 client.7800 mds_command: new command op to 7668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.869+0000 7f8be3185740 1 -- 172.21.15.119:0/1097403526 --> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b06b201b0 con 0x560b06befa80 2024-04-25T10:15:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.869+0000 7f8bdd800640 1 --2- 172.21.15.119:0/1097403526 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x560b06befa80 0x560b06bdc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.869+0000 7f8bbea00640 10 client.7800 ms_handle_connect on v2:172.21.15.119:6835/2043821095 2024-04-25T10:15:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.869+0000 7f8bdd800640 1 --2- 172.21.15.119:0/1097403526 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x560b06befa80 0x560b06bdc7a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f80c349ff28b9f3 server_cookie=c5b283f27f2296e0 in_seq=0 out_seq=0 2024-04-25T10:15:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.870+0000 7f8bbea00640 1 -- 172.21.15.119:0/1097403526 <== mds.0 v2:172.21.15.119:6835/2043821095 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560b06b201b0 con 0x560b06befa80 2024-04-25T10:15:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.870+0000 7f8bbea00640 10 client.7800 handle_command_reply: tid=0 2024-04-25T10:15:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.870+0000 7f8be3185740 1 client.7800 shutdown 2024-04-25T10:15:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.870+0000 7f8be3185740 1 -- 172.21.15.119:0/1097403526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06bd9830 msgr2=0x560b06bf8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:29.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.870+0000 7f8be3185740 1 --2- 172.21.15.119:0/1097403526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06bd9830 0x560b06bf8190 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f8bd808b390 tx=0x7f8bc400e590 comp rx=0 tx=0).stop 2024-04-25T10:15:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.870+0000 7f8be3185740 1 -- 172.21.15.119:0/1097403526 shutdown_connections 2024-04-25T10:15:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.870+0000 7f8be3185740 1 --2- 172.21.15.119:0/1097403526 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x560b06befa80 0x560b06bdc7a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.870+0000 7f8be3185740 1 --2- 172.21.15.119:0/1097403526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06beb440 0x560b06bd6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.870+0000 7f8be3185740 1 --2- 172.21.15.119:0/1097403526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06bd9830 0x560b06bf8190 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.870+0000 7f8be3185740 1 --2- 172.21.15.119:0/1097403526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06baed70 0x560b06c08880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.870+0000 7f8be3185740 1 -- 172.21.15.119:0/1097403526 >> 172.21.15.119:0/1097403526 conn(0x560b06b76a30 msgr2=0x560b06bad610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.870+0000 7f8bbea00640 0 client.7800 ms_handle_reset on v2:172.21.15.119:6835/2043821095 2024-04-25T10:15:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.871+0000 7f8be3185740 1 -- 172.21.15.119:0/1097403526 shutdown_connections 2024-04-25T10:15:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.871+0000 7f8be3185740 1 -- 172.21.15.119:0/1097403526 wait complete. 2024-04-25T10:15:29.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.871+0000 7f8be3185740 20 client.7800 trim_cache size 0 max 16384 2024-04-25T10:15:29.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.887+0000 7f8be3185740 1 Processor -- start 2024-04-25T10:15:29.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.887+0000 7f8be3185740 1 -- start start 2024-04-25T10:15:29.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.888+0000 7f8be3185740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06baed70 0x560b06beaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.888+0000 7f8be3185740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06bd9830 0x560b06bea1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.888+0000 7f8be3185740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06beb440 0x560b06bad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.888+0000 7f8be3185740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b06bad120 con 0x560b06bd9830 2024-04-25T10:15:29.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.888+0000 7f8be3185740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd81ffe10 con 0x560b06baed70 2024-04-25T10:15:29.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.888+0000 7f8be3185740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd81ff010 con 0x560b06beb440 2024-04-25T10:15:29.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.888+0000 7f8bdce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06bd9830 0x560b06bea1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.888+0000 7f8bdce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06bd9830 0x560b06bea1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53768/0 (socket says 172.21.15.119:53768) 2024-04-25T10:15:29.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.888+0000 7f8bdce00640 1 -- 172.21.15.119:0/2759822083 learned_addr learned my addr 172.21.15.119:0/2759822083 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:29.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.889+0000 7f8bde200640 1 --2- 172.21.15.119:0/2759822083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06beb440 0x560b06bad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.889+0000 7f8bdce00640 1 -- 172.21.15.119:0/2759822083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06beb440 msgr2=0x560b06bad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:29.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.889+0000 7f8bdd800640 1 --2- 172.21.15.119:0/2759822083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06baed70 0x560b06beaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.889+0000 7f8bdce00640 1 --2- 172.21.15.119:0/2759822083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06beb440 0x560b06bad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.889+0000 7f8bdce00640 1 -- 172.21.15.119:0/2759822083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06baed70 msgr2=0x560b06beaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.889+0000 7f8bdce00640 1 --2- 172.21.15.119:0/2759822083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06baed70 0x560b06beaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.889+0000 7f8bdce00640 1 -- 172.21.15.119:0/2759822083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b06c04590 con 0x560b06bd9830 2024-04-25T10:15:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.889+0000 7f8bdce00640 1 --2- 172.21.15.119:0/2759822083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06bd9830 0x560b06bea1c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400e5d0 tx=0x7f8bc4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9876f896f3d6c56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.889+0000 7f8bc9600640 1 -- 172.21.15.119:0/2759822083 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4003860 con 0x560b06bd9830 2024-04-25T10:15:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.889+0000 7f8bc9600640 1 -- 172.21.15.119:0/2759822083 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc4018070 con 0x560b06bd9830 2024-04-25T10:15:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.889+0000 7f8bc9600640 1 -- 172.21.15.119:0/2759822083 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4003c20 con 0x560b06bd9830 2024-04-25T10:15:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.890+0000 7f8be3185740 1 -- 172.21.15.119:0/2759822083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06bd9830 msgr2=0x560b06bea1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.890+0000 7f8be3185740 1 --2- 172.21.15.119:0/2759822083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06bd9830 0x560b06bea1c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400e5d0 tx=0x7f8bc4012000 comp rx=0 tx=0).stop 2024-04-25T10:15:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.890+0000 7f8be3185740 1 -- 172.21.15.119:0/2759822083 shutdown_connections 2024-04-25T10:15:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.890+0000 7f8be3185740 1 --2- 172.21.15.119:0/2759822083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06beb440 0x560b06bad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.890+0000 7f8be3185740 1 --2- 172.21.15.119:0/2759822083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06bd9830 0x560b06bea1c0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.890+0000 7f8be3185740 1 --2- 172.21.15.119:0/2759822083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06baed70 0x560b06beaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.890+0000 7f8be3185740 1 -- 172.21.15.119:0/2759822083 >> 172.21.15.119:0/2759822083 conn(0x560b06b76a30 msgr2=0x560b06bef240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:29.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.890+0000 7f8be3185740 1 -- 172.21.15.119:0/2759822083 shutdown_connections 2024-04-25T10:15:29.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.890+0000 7f8be3185740 1 -- 172.21.15.119:0/2759822083 wait complete. 2024-04-25T10:15:29.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.891+0000 7f8be3185740 1 -- start start 2024-04-25T10:15:29.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.891+0000 7f8be3185740 1 Processor -- start 2024-04-25T10:15:29.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.892+0000 7f8be3185740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06baed70 0x560b06bdc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.892+0000 7f8be3185740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06bd9830 0x560b06bdcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.892+0000 7f8be3185740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06beb440 0x560b06bd9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.892+0000 7f8be3185740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd81fea10 con 0x560b06baed70 2024-04-25T10:15:29.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.892+0000 7f8be3185740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd81fe410 con 0x560b06beb440 2024-04-25T10:15:29.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.892+0000 7f8be3185740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b06c5fec0 con 0x560b06bd9830 2024-04-25T10:15:29.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.892+0000 7f8be3185740 10 client.0 _ll_register_callbacks cb 0x7f8bc4007ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:29.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.892+0000 7f8bdd800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06baed70 0x560b06bdc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.892+0000 7f8bdd800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06baed70 0x560b06bdc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53782/0 (socket says 172.21.15.119:53782) 2024-04-25T10:15:29.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.892+0000 7f8bdd800640 1 -- 172.21.15.119:0/828255163 learned_addr learned my addr 172.21.15.119:0/828255163 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:29.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.893+0000 7f8bdd800640 1 -- 172.21.15.119:0/828255163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06bd9830 msgr2=0x560b06bdcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:29.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.893+0000 7f8bdd800640 1 --2- 172.21.15.119:0/828255163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06bd9830 0x560b06bdcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.893+0000 7f8bdd800640 1 -- 172.21.15.119:0/828255163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06beb440 msgr2=0x560b06bd9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:29.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.893+0000 7f8bdd800640 1 --2- 172.21.15.119:0/828255163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06beb440 0x560b06bd9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.893+0000 7f8bdd800640 1 -- 172.21.15.119:0/828255163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b06ba5510 con 0x560b06baed70 2024-04-25T10:15:29.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.893+0000 7f8bdd800640 1 --2- 172.21.15.119:0/828255163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06baed70 0x560b06bdc680 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc008010 tx=0x7f8bcc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b98935acd83f054 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:29.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.893+0000 7f8bbea00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:15:29.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.893+0000 7f8bbea00640 1 -- 172.21.15.119:0/828255163 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc02e070 con 0x560b06baed70 2024-04-25T10:15:29.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.893+0000 7f8bbea00640 1 -- 172.21.15.119:0/828255163 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f8bcc015b90 con 0x560b06baed70 2024-04-25T10:15:29.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.893+0000 7f8be3185740 1 -- 172.21.15.119:0/828255163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bc4018070 con 0x560b06baed70 2024-04-25T10:15:29.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.894+0000 7f8bbea00640 1 -- 172.21.15.119:0/828255163 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bcc039050 con 0x560b06baed70 2024-04-25T10:15:29.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.894+0000 7f8be3185740 10 client.7806 fetch_fsmap learned FSMap version 74 2024-04-25T10:15:29.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.894+0000 7f8be3185740 1 -- 172.21.15.119:0/828255163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x560b06c03d80 con 0x560b06baed70 2024-04-25T10:15:29.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.894+0000 7f8bbea00640 1 -- 172.21.15.119:0/828255163 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc4018070 con 0x560b06baed70 2024-04-25T10:15:29.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.894+0000 7f8bbea00640 1 -- 172.21.15.119:0/828255163 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8bcc03c020 con 0x560b06baed70 2024-04-25T10:15:29.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.894+0000 7f8be3185740 10 client.7806 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:15:29.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.894+0000 7f8be3185740 10 client.7806 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:15:29.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.894+0000 7f8be3185740 20 client.7806 populate_metadata read hostname 'smithi119' 2024-04-25T10:15:29.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.894+0000 7f8be3185740 1 --2- 172.21.15.119:0/828255163 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x560b06c03710 0x560b06bda150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:29.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.894+0000 7f8be3185740 4 client.7806 mds_command: new command op to 7668 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:15:29.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.894+0000 7f8be3185740 1 -- 172.21.15.119:0/828255163 --> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560b06b1a500 con 0x560b06c03710 2024-04-25T10:15:29.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.894+0000 7f8bdce00640 1 --2- 172.21.15.119:0/828255163 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x560b06c03710 0x560b06bda150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:29.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8bdce00640 1 --2- 172.21.15.119:0/828255163 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x560b06c03710 0x560b06bda150 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de86487309e780a2 server_cookie=c04a39ef911da376 in_seq=0 out_seq=0 2024-04-25T10:15:29.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8bbea00640 10 client.7806 ms_handle_connect on v2:172.21.15.119:6835/2043821095 2024-04-25T10:15:29.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8bbea00640 1 -- 172.21.15.119:0/828255163 <== mds.0 v2:172.21.15.119:6835/2043821095 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560b06b1a500 con 0x560b06c03710 2024-04-25T10:15:29.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8bbea00640 10 client.7806 handle_command_reply: tid=0 2024-04-25T10:15:29.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8be3185740 1 client.7806 shutdown 2024-04-25T10:15:29.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8be3185740 1 -- 172.21.15.119:0/828255163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06baed70 msgr2=0x560b06bdc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:29.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8be3185740 1 --2- 172.21.15.119:0/828255163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06baed70 0x560b06bdc680 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc008010 tx=0x7f8bcc015000 comp rx=0 tx=0).stop 2024-04-25T10:15:29.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8be3185740 1 -- 172.21.15.119:0/828255163 shutdown_connections 2024-04-25T10:15:29.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8be3185740 1 --2- 172.21.15.119:0/828255163 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x560b06c03710 0x560b06bda150 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8be3185740 1 --2- 172.21.15.119:0/828255163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560b06beb440 0x560b06bd9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8be3185740 1 --2- 172.21.15.119:0/828255163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x560b06bd9830 0x560b06bdcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8be3185740 1 --2- 172.21.15.119:0/828255163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560b06baed70 0x560b06bdc680 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8bbea00640 0 client.7806 ms_handle_reset on v2:172.21.15.119:6835/2043821095 2024-04-25T10:15:29.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8be3185740 1 -- 172.21.15.119:0/828255163 >> 172.21.15.119:0/828255163 conn(0x560b06b76a30 msgr2=0x560b06bef240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:29.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.895+0000 7f8be3185740 1 -- 172.21.15.119:0/828255163 shutdown_connections 2024-04-25T10:15:29.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.896+0000 7f8be3185740 1 -- 172.21.15.119:0/828255163 wait complete. 2024-04-25T10:15:29.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.896+0000 7f8be3185740 20 client.7806 trim_cache size 0 max 16384 2024-04-25T10:15:29.902 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:15:29.902 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2024-04-25T10:15:29.902 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2024-04-25T10:15:29.902 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:15:29.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.896+0000 7f8bdec00640 1 -- 172.21.15.119:0/2989689547 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8bb806b4d0 msgr2=0x7f8bb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:29.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.896+0000 7f8bdec00640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8bb806b4d0 0x7f8bb806d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc006fd0 tx=0x7f8bcc008040 comp rx=0 tx=0).stop 2024-04-25T10:15:29.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.896+0000 7f8bdec00640 1 -- 172.21.15.119:0/2989689547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd815d980 msgr2=0x7f8bd815edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:29.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.896+0000 7f8bdec00640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd815d980 0x7f8bd815edc0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009e50 tx=0x7f8bd0004a40 comp rx=0 tx=0).stop 2024-04-25T10:15:29.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.896+0000 7f8bdec00640 1 -- 172.21.15.119:0/2989689547 shutdown_connections 2024-04-25T10:15:29.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.896+0000 7f8bdec00640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8bb806b4d0 0x7f8bb806d990 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.896+0000 7f8bdec00640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bd815d980 0x7f8bd815edc0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.896+0000 7f8bdec00640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bd80b2a00 0x7f8bd8152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.896+0000 7f8bdec00640 1 --2- 172.21.15.119:0/2989689547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd80b2440 0x7f8bd8154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:29.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.896+0000 7f8bdec00640 1 -- 172.21.15.119:0/2989689547 >> 172.21.15.119:0/2989689547 conn(0x7f8bd80726c0 msgr2=0x7f8bd80b7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:29.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.897+0000 7f8bdec00640 1 -- 172.21.15.119:0/2989689547 shutdown_connections 2024-04-25T10:15:29.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:29.897+0000 7f8bdec00640 1 -- 172.21.15.119:0/2989689547 wait complete. 2024-04-25T10:15:29.916 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:15:29.917 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:30.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.101+0000 7f70db200640 1 Processor -- start 2024-04-25T10:15:30.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.102+0000 7f70db200640 1 -- start start 2024-04-25T10:15:30.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.103+0000 7f70db200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70dc0d3f80 0x7f70dc0d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.103+0000 7f70db200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70dc15ef10 con 0x7f70dc0b9270 2024-04-25T10:15:30.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.103+0000 7f70db200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70dc15e390 con 0x7f70dc15db60 2024-04-25T10:15:30.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.103+0000 7f70db200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70dc15e670 con 0x7f70dc0d3f80 2024-04-25T10:15:30.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.104+0000 7f70da800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f70dc0b9270 0x7f70dc0d24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54664/0 (socket says 172.21.15.119:54664) 2024-04-25T10:15:30.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.104+0000 7f70da800640 1 -- 172.21.15.119:0/1867414911 learned_addr learned my addr 172.21.15.119:0/1867414911 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:30.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.104+0000 7f70d9400640 1 --2- 172.21.15.119:0/1867414911 >> v2:172.21.15.119:3300/0 conn(0x7f70dc0d3f80 0x7f70dc0d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.104+0000 7f70d9400640 1 -- 172.21.15.119:0/1867414911 >> v1:172.21.15.132:6789/0 conn(0x7f70dc15db60 legacy=0x7f70dc0d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:30.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.104+0000 7f70d9400640 1 -- 172.21.15.119:0/1867414911 >> v1:172.21.15.119:6789/0 conn(0x7f70dc0b9270 legacy=0x7f70dc0d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.104+0000 7f70d8a00640 1 -- 172.21.15.119:0/1867414911 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680121567 0 0) 0x7f70dc15ef10 con 0x7f70dc0b9270 2024-04-25T10:15:30.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.104+0000 7f70d9400640 1 -- 172.21.15.119:0/1867414911 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc15e950 con 0x7f70dc0d3f80 2024-04-25T10:15:30.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.105+0000 7f70d9400640 1 --2- 172.21.15.119:0/1867414911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc0d3f80 0x7f70dc0d1f90 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009de0 tx=0x7f70cc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785d0133e281b8bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:30.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.106+0000 7f70d8a00640 1 -- 172.21.15.119:0/1867414911 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc007740 con 0x7f70dc0d3f80 2024-04-25T10:15:30.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.106+0000 7f70d8a00640 1 -- 172.21.15.119:0/1867414911 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70cc0078e0 con 0x7f70dc0d3f80 2024-04-25T10:15:30.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.106+0000 7f70d8a00640 1 -- 172.21.15.119:0/1867414911 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc007cb0 con 0x7f70dc0d3f80 2024-04-25T10:15:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.106+0000 7f70db200640 1 -- 172.21.15.119:0/1867414911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc0d3f80 msgr2=0x7f70dc0d1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.106+0000 7f70db200640 1 --2- 172.21.15.119:0/1867414911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc0d3f80 0x7f70dc0d1f90 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009de0 tx=0x7f70cc02a710 comp rx=0 tx=0).stop 2024-04-25T10:15:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.107+0000 7f70db200640 1 -- 172.21.15.119:0/1867414911 shutdown_connections 2024-04-25T10:15:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.107+0000 7f70db200640 1 --2- 172.21.15.119:0/1867414911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc0d3f80 0x7f70dc0d1f90 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.107+0000 7f70db200640 1 -- 172.21.15.119:0/1867414911 >> 172.21.15.119:0/1867414911 conn(0x7f70dc0726c0 msgr2=0x7f70dc072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.107+0000 7f70db200640 1 -- 172.21.15.119:0/1867414911 shutdown_connections 2024-04-25T10:15:30.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.107+0000 7f70db200640 1 -- 172.21.15.119:0/1867414911 wait complete. 2024-04-25T10:15:30.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.108+0000 7f70db200640 1 Processor -- start 2024-04-25T10:15:30.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.109+0000 7f70db200640 1 -- start start 2024-04-25T10:15:30.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.109+0000 7f70db200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70dc0b9270 0x7f70dc0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.109+0000 7f70db200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0d3f80 0x7f70dc157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.109+0000 7f70db200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15db60 0x7f70dc1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.109+0000 7f70db200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0b3120 con 0x7f70dc15db60 2024-04-25T10:15:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.109+0000 7f70db200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0b2fa0 con 0x7f70dc0b9270 2024-04-25T10:15:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.109+0000 7f70db200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc15df40 con 0x7f70dc0d3f80 2024-04-25T10:15:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.110+0000 7f70da800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15db60 0x7f70dc1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.110+0000 7f70da800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15db60 0x7f70dc1593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53792/0 (socket says 172.21.15.119:53792) 2024-04-25T10:15:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.110+0000 7f70da800640 1 -- 172.21.15.119:0/2595721574 learned_addr learned my addr 172.21.15.119:0/2595721574 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.110+0000 7f70d9400640 1 --2- 172.21.15.119:0/2595721574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0d3f80 0x7f70dc157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.110+0000 7f70da800640 1 -- 172.21.15.119:0/2595721574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0d3f80 msgr2=0x7f70dc157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.110+0000 7f70d9e00640 1 --2- 172.21.15.119:0/2595721574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70dc0b9270 0x7f70dc0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.110+0000 7f70da800640 1 --2- 172.21.15.119:0/2595721574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0d3f80 0x7f70dc157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.110+0000 7f70da800640 1 -- 172.21.15.119:0/2595721574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70dc0b9270 msgr2=0x7f70dc0b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.110+0000 7f70da800640 1 --2- 172.21.15.119:0/2595721574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70dc0b9270 0x7f70dc0b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.110+0000 7f70da800640 1 -- 172.21.15.119:0/2595721574 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0b4290 con 0x7f70dc15db60 2024-04-25T10:15:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.111+0000 7f70da800640 1 --2- 172.21.15.119:0/2595721574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15db60 0x7f70dc1593f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f70d400c440 tx=0x7f70d400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.111+0000 7f70d2000640 1 -- 172.21.15.119:0/2595721574 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d40074e0 con 0x7f70dc15db60 2024-04-25T10:15:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.111+0000 7f70d2000640 1 -- 172.21.15.119:0/2595721574 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d40047e0 con 0x7f70dc15db60 2024-04-25T10:15:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.111+0000 7f70d2000640 1 -- 172.21.15.119:0/2595721574 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4004bd0 con 0x7f70dc15db60 2024-04-25T10:15:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.111+0000 7f70db200640 1 -- 172.21.15.119:0/2595721574 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70dc0cac00 con 0x7f70dc15db60 2024-04-25T10:15:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.111+0000 7f70db200640 1 -- 172.21.15.119:0/2595721574 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70dc0cb1c0 con 0x7f70dc15db60 2024-04-25T10:15:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.112+0000 7f70db200640 1 -- 172.21.15.119:0/2595721574 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70dc0749b0 con 0x7f70dc15db60 2024-04-25T10:15:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.113+0000 7f70d2000640 1 -- 172.21.15.119:0/2595721574 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70d4011070 con 0x7f70dc15db60 2024-04-25T10:15:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.114+0000 7f70d2000640 1 --2- 172.21.15.119:0/2595721574 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70b806b4d0 0x7f70b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.114+0000 7f70d2000640 1 -- 172.21.15.119:0/2595721574 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f70d4089e50 con 0x7f70dc15db60 2024-04-25T10:15:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.114+0000 7f70d9e00640 1 --2- 172.21.15.119:0/2595721574 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70b806b4d0 0x7f70b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.115+0000 7f70d9e00640 1 --2- 172.21.15.119:0/2595721574 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70b806b4d0 0x7f70b806d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f70c4003d90 tx=0x7f70c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:30.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.118+0000 7f70d2000640 1 -- 172.21.15.119:0/2595721574 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70d4058b60 con 0x7f70dc15db60 2024-04-25T10:15:30.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.347+0000 7f70db200640 1 -- 172.21.15.119:0/2595721574 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70dc0cd5e0 con 0x7f70dc15db60 2024-04-25T10:15:30.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.348+0000 7f70d2000640 1 -- 172.21.15.119:0/2595721574 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f70d4058460 con 0x7f70dc15db60 2024-04-25T10:15:30.352 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:30.352 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:14:54.775138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7668":{"gid":7668,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/2043821095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2043821095},{"type":"v1","addr":"172.21.15.119:6837","nonce":2043821095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:15:30.353 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 74 2024-04-25T10:15:30.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.351+0000 7f70db200640 1 -- 172.21.15.119:0/2595721574 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70b806b4d0 msgr2=0x7f70b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.351+0000 7f70db200640 1 --2- 172.21.15.119:0/2595721574 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70b806b4d0 0x7f70b806d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f70c4003d90 tx=0x7f70c4009040 comp rx=0 tx=0).stop 2024-04-25T10:15:30.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.351+0000 7f70db200640 1 -- 172.21.15.119:0/2595721574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15db60 msgr2=0x7f70dc1593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.351+0000 7f70db200640 1 --2- 172.21.15.119:0/2595721574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15db60 0x7f70dc1593f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f70d400c440 tx=0x7f70d400c900 comp rx=0 tx=0).stop 2024-04-25T10:15:30.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.351+0000 7f70db200640 1 -- 172.21.15.119:0/2595721574 shutdown_connections 2024-04-25T10:15:30.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.351+0000 7f70db200640 1 --2- 172.21.15.119:0/2595721574 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f70b806b4d0 0x7f70b806d990 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.351+0000 7f70db200640 1 --2- 172.21.15.119:0/2595721574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15db60 0x7f70dc1593f0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.351+0000 7f70db200640 1 --2- 172.21.15.119:0/2595721574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70dc0d3f80 0x7f70dc157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.351+0000 7f70db200640 1 --2- 172.21.15.119:0/2595721574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70dc0b9270 0x7f70dc0b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.351+0000 7f70db200640 1 -- 172.21.15.119:0/2595721574 >> 172.21.15.119:0/2595721574 conn(0x7f70dc0726c0 msgr2=0x7f70dc0d3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:30.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.351+0000 7f70db200640 1 -- 172.21.15.119:0/2595721574 shutdown_connections 2024-04-25T10:15:30.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.352+0000 7f70db200640 1 -- 172.21.15.119:0/2595721574 wait complete. 2024-04-25T10:15:30.374 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:15:30.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.571+0000 7fdfd8000640 1 Processor -- start 2024-04-25T10:15:30.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.571+0000 7fdfd8000640 1 -- start start 2024-04-25T10:15:30.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.573+0000 7fdfd8000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfd00b42c0 0x7fdfd015ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.573+0000 7fdfd8000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdfd015aef0 0x7fdfd00cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.573+0000 7fdfd8000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd015b370 con 0x7fdfd00b46a0 2024-04-25T10:15:30.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.573+0000 7fdfd8000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd00d05d0 con 0x7fdfd015aef0 2024-04-25T10:15:30.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.573+0000 7fdfd8000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd015b650 con 0x7fdfd00b42c0 2024-04-25T10:15:30.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.573+0000 7fdfd6000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdfd00b46a0 0x7fdfd01549d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54668/0 (socket says 172.21.15.119:54668) 2024-04-25T10:15:30.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.573+0000 7fdfd6000640 1 -- 172.21.15.119:0/613766344 learned_addr learned my addr 172.21.15.119:0/613766344 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:30.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/613766344 >> v2:172.21.15.132:3300/0 conn(0x7fdfd015aef0 0x7fdfd00cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfd5600640 1 --2- 172.21.15.119:0/613766344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfd00b42c0 0x7fdfd015ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 -- 172.21.15.119:0/613766344 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634937179 0 0) 0x7fdfd015b370 con 0x7fdfd00b46a0 2024-04-25T10:15:30.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 -- 172.21.15.119:0/613766344 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb8003520 con 0x7fdfd00b46a0 2024-04-25T10:15:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 -- 172.21.15.119:0/613766344 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3475274649 0 0) 0x7fdfb8003520 con 0x7fdfd00b46a0 2024-04-25T10:15:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 -- 172.21.15.119:0/613766344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfd00b42c0 msgr2=0x7fdfd015ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 --2- 172.21.15.119:0/613766344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfd00b42c0 0x7fdfd015ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 -- 172.21.15.119:0/613766344 >> v2:172.21.15.132:3300/0 conn(0x7fdfd015aef0 msgr2=0x7fdfd00cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 --2- 172.21.15.119:0/613766344 >> v2:172.21.15.132:3300/0 conn(0x7fdfd015aef0 0x7fdfd00cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 -- 172.21.15.119:0/613766344 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd015b930 con 0x7fdfd00b46a0 2024-04-25T10:15:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 -- 172.21.15.119:0/613766344 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fdfc8001670 con 0x7fdfd00b46a0 2024-04-25T10:15:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 -- 172.21.15.119:0/613766344 >> v1:172.21.15.119:6789/0 conn(0x7fdfd00b46a0 legacy=0x7fdfd01549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 --2- 172.21.15.119:0/613766344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb80039b0 0x7fdfb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 --2- 172.21.15.119:0/613766344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfb8004800 0x7fdfb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 --2- 172.21.15.119:0/613766344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfb80051c0 0x7fdfb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 -- 172.21.15.119:0/613766344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd00d05d0 con 0x7fdfb80051c0 2024-04-25T10:15:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 -- 172.21.15.119:0/613766344 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd015b650 con 0x7fdfb8004800 2024-04-25T10:15:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.574+0000 7fdfcfe00640 1 -- 172.21.15.119:0/613766344 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb800c030 con 0x7fdfb80039b0 2024-04-25T10:15:30.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.575+0000 7fdfd6000640 1 --2- 172.21.15.119:0/613766344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfb80051c0 0x7fdfb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.575+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/613766344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfb8004800 0x7fdfb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.575+0000 7fdfd5600640 1 --2- 172.21.15.119:0/613766344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb80039b0 0x7fdfb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.575+0000 7fdfd6000640 1 -- 172.21.15.119:0/613766344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb80039b0 msgr2=0x7fdfb8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:30.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.575+0000 7fdfd6000640 1 --2- 172.21.15.119:0/613766344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb80039b0 0x7fdfb8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.575+0000 7fdfd6000640 1 -- 172.21.15.119:0/613766344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfb8004800 msgr2=0x7fdfb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:30.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.575+0000 7fdfd6000640 1 --2- 172.21.15.119:0/613766344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfb8004800 0x7fdfb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.575+0000 7fdfd6000640 1 -- 172.21.15.119:0/613766344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb8003520 con 0x7fdfb80051c0 2024-04-25T10:15:30.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.575+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/613766344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfb8004800 0x7fdfb8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:30.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.575+0000 7fdfd5600640 1 --2- 172.21.15.119:0/613766344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfb80039b0 0x7fdfb8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:30.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.577+0000 7fdfd6000640 1 --2- 172.21.15.119:0/613766344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfb80051c0 0x7fdfb800b9c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800c160 tx=0x7fdfb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f528c00ebbc270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:30.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.577+0000 7fdfcfe00640 1 -- 172.21.15.119:0/613766344 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8007c80 con 0x7fdfb80051c0 2024-04-25T10:15:30.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.577+0000 7fdfcfe00640 1 -- 172.21.15.119:0/613766344 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc8005ad0 con 0x7fdfb80051c0 2024-04-25T10:15:30.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.577+0000 7fdfcfe00640 1 -- 172.21.15.119:0/613766344 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8007070 con 0x7fdfb80051c0 2024-04-25T10:15:30.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.577+0000 7fdfd8000640 1 -- 172.21.15.119:0/613766344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfb80051c0 msgr2=0x7fdfb800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.577+0000 7fdfd8000640 1 --2- 172.21.15.119:0/613766344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfb80051c0 0x7fdfb800b9c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800c160 tx=0x7fdfb8005830 comp rx=0 tx=0).stop 2024-04-25T10:15:30.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.577+0000 7fdfd6000640 1 -- 172.21.15.119:0/613766344 reap_dead start 2024-04-25T10:15:30.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.578+0000 7fdfd8000640 1 -- 172.21.15.119:0/613766344 shutdown_connections 2024-04-25T10:15:30.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.578+0000 7fdfd8000640 1 -- 172.21.15.119:0/613766344 >> 172.21.15.119:0/613766344 conn(0x7fdfd006aba0 msgr2=0x7fdfd006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:30.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.578+0000 7fdfd8000640 1 -- 172.21.15.119:0/613766344 shutdown_connections 2024-04-25T10:15:30.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.578+0000 7fdfd8000640 1 -- 172.21.15.119:0/613766344 wait complete. 2024-04-25T10:15:30.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.579+0000 7fdfd8000640 1 Processor -- start 2024-04-25T10:15:30.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.580+0000 7fdfd8000640 1 -- start start 2024-04-25T10:15:30.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.580+0000 7fdfd8000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfb80051c0 0x7fdfd014a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.580+0000 7fdfd8000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfd0148510 0x7fdfd0148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.581+0000 7fdfd8000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfd0148970 0x7fdfd01f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.581+0000 7fdfd8000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd006eae0 con 0x7fdfb80051c0 2024-04-25T10:15:30.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.581+0000 7fdfd8000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd00cf980 con 0x7fdfd0148970 2024-04-25T10:15:30.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.581+0000 7fdfd8000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd006ec60 con 0x7fdfd0148510 2024-04-25T10:15:30.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.581+0000 7fdfd5600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfb80051c0 0x7fdfd014a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.581+0000 7fdfd5600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfb80051c0 0x7fdfd014a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53818/0 (socket says 172.21.15.119:53818) 2024-04-25T10:15:30.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.581+0000 7fdfd5600640 1 -- 172.21.15.119:0/1142704161 learned_addr learned my addr 172.21.15.119:0/1142704161 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:30.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.581+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/1142704161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfd0148510 0x7fdfd0148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.581+0000 7fdfd6000640 1 --2- 172.21.15.119:0/1142704161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfd0148970 0x7fdfd01f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.581+0000 7fdfd5600640 1 -- 172.21.15.119:0/1142704161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfd0148510 msgr2=0x7fdfd0148f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.581+0000 7fdfd5600640 1 --2- 172.21.15.119:0/1142704161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfd0148510 0x7fdfd0148f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.581+0000 7fdfd5600640 1 -- 172.21.15.119:0/1142704161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfd0148970 msgr2=0x7fdfd01f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.581+0000 7fdfd5600640 1 --2- 172.21.15.119:0/1142704161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfd0148970 0x7fdfd01f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.581+0000 7fdfd5600640 1 -- 172.21.15.119:0/1142704161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd01f34e0 con 0x7fdfb80051c0 2024-04-25T10:15:30.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.582+0000 7fdfd5600640 1 --2- 172.21.15.119:0/1142704161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfb80051c0 0x7fdfd014a330 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4007f00 tx=0x7fdfc4010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:30.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.582+0000 7fdfcd600640 1 -- 172.21.15.119:0/1142704161 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc401a030 con 0x7fdfb80051c0 2024-04-25T10:15:30.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.582+0000 7fdfcd600640 1 -- 172.21.15.119:0/1142704161 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc4003990 con 0x7fdfb80051c0 2024-04-25T10:15:30.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.582+0000 7fdfcd600640 1 -- 172.21.15.119:0/1142704161 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4003d40 con 0x7fdfb80051c0 2024-04-25T10:15:30.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.582+0000 7fdfd8000640 1 -- 172.21.15.119:0/1142704161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd01f37c0 con 0x7fdfb80051c0 2024-04-25T10:15:30.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.582+0000 7fdfd8000640 1 -- 172.21.15.119:0/1142704161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd01f3d80 con 0x7fdfb80051c0 2024-04-25T10:15:30.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.584+0000 7fdfcd600640 1 -- 172.21.15.119:0/1142704161 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdfc401d070 con 0x7fdfb80051c0 2024-04-25T10:15:30.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.585+0000 7fdfcd600640 1 --2- 172.21.15.119:0/1142704161 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdfb006b4d0 0x7fdfb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.585+0000 7fdfcd600640 1 -- 172.21.15.119:0/1142704161 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fdfc408b290 con 0x7fdfb80051c0 2024-04-25T10:15:30.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.586+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/1142704161 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdfb006b4d0 0x7fdfb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.586+0000 7fdfdade6740 1 Processor -- start 2024-04-25T10:15:30.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.586+0000 7fdfdade6740 1 -- start start 2024-04-25T10:15:30.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.586+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/1142704161 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdfb006b4d0 0x7fdfb006d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00b26c0 tx=0x7fdfbc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:30.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.587+0000 7fdfdade6740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 0x561a968c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.587+0000 7fdfdade6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a968de160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.587+0000 7fdfdade6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a96896050 0x561a968dc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.587+0000 7fdfdade6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a967f8ae0 con 0x561a96896050 2024-04-25T10:15:30.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.587+0000 7fdfdade6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a967a17c0 con 0x561a968ddc10 2024-04-25T10:15:30.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.587+0000 7fdfdade6740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a967f4b80 con 0x561a968c0370 2024-04-25T10:15:30.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.587+0000 7fdfd6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a96896050 0x561a968dc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.587+0000 7fdfd6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a96896050 0x561a968dc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53828/0 (socket says 172.21.15.119:53828) 2024-04-25T10:15:30.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.587+0000 7fdfd6000640 1 -- 172.21.15.119:0/344408965 learned_addr learned my addr 172.21.15.119:0/344408965 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:30.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.588+0000 7fdfd5600640 1 --2- 172.21.15.119:0/344408965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 0x561a968c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.588+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/344408965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a968de160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.588+0000 7fdfd6000640 1 -- 172.21.15.119:0/344408965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 msgr2=0x561a968c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:30.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.588+0000 7fdfd6000640 1 --2- 172.21.15.119:0/344408965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 0x561a968c0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.588+0000 7fdfd6000640 1 -- 172.21.15.119:0/344408965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 msgr2=0x561a968de160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:30.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.588+0000 7fdfd6000640 1 --2- 172.21.15.119:0/344408965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a968de160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.588+0000 7fdfd6000640 1 -- 172.21.15.119:0/344408965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a965d97c0 con 0x561a96896050 2024-04-25T10:15:30.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.588+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/344408965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a968de160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:30.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.588+0000 7fdfd6000640 1 --2- 172.21.15.119:0/344408965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a96896050 0x561a968dc2d0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800c280 tx=0x7fdfc802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5989170a886c6af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:30.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.589+0000 7fdfc1c00640 1 -- 172.21.15.119:0/344408965 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8043070 con 0x561a96896050 2024-04-25T10:15:30.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.589+0000 7fdfc1c00640 1 -- 172.21.15.119:0/344408965 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc8038040 con 0x561a96896050 2024-04-25T10:15:30.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.589+0000 7fdfc1c00640 1 -- 172.21.15.119:0/344408965 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc80047e0 con 0x561a96896050 2024-04-25T10:15:30.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.589+0000 7fdfdade6740 1 -- 172.21.15.119:0/344408965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a96896050 msgr2=0x561a968dc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.589+0000 7fdfdade6740 1 --2- 172.21.15.119:0/344408965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a96896050 0x561a968dc2d0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800c280 tx=0x7fdfc802c000 comp rx=0 tx=0).stop 2024-04-25T10:15:30.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.589+0000 7fdfdade6740 1 -- 172.21.15.119:0/344408965 shutdown_connections 2024-04-25T10:15:30.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.589+0000 7fdfdade6740 1 --2- 172.21.15.119:0/344408965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a96896050 0x561a968dc2d0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.589+0000 7fdfdade6740 1 --2- 172.21.15.119:0/344408965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a968de160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.589+0000 7fdfdade6740 1 --2- 172.21.15.119:0/344408965 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 0x561a968c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.589+0000 7fdfdade6740 1 -- 172.21.15.119:0/344408965 >> 172.21.15.119:0/344408965 conn(0x561a96892980 msgr2=0x561a968db180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:30.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.589+0000 7fdfdade6740 1 -- 172.21.15.119:0/344408965 shutdown_connections 2024-04-25T10:15:30.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.590+0000 7fdfdade6740 1 -- 172.21.15.119:0/344408965 wait complete. 2024-04-25T10:15:30.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.590+0000 7fdfdade6740 1 -- start start 2024-04-25T10:15:30.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.590+0000 7fdfdade6740 1 Processor -- start 2024-04-25T10:15:30.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.591+0000 7fdfdade6740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 0x561a968c1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.591+0000 7fdfdade6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a968c1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.591+0000 7fdfdade6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968b2e50 0x561a968e1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfdade6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a96896df0 con 0x561a968b2e50 2024-04-25T10:15:30.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfdade6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a96796d40 con 0x561a968ddc10 2024-04-25T10:15:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfdade6740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a968908b0 con 0x561a968c0370 2024-04-25T10:15:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfdade6740 10 client.0 _ll_register_callbacks cb 0x561a9650cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfd6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968b2e50 0x561a968e1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfd6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968b2e50 0x561a968e1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53834/0 (socket says 172.21.15.119:53834) 2024-04-25T10:15:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfd6000640 1 -- 172.21.15.119:0/354839517 learned_addr learned my addr 172.21.15.119:0/354839517 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfd5600640 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 0x561a968c1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a968c1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfd6000640 1 -- 172.21.15.119:0/354839517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 msgr2=0x561a968c1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfd6000640 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 0x561a968c1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfd6000640 1 -- 172.21.15.119:0/354839517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 msgr2=0x561a968c1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfd6000640 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a968c1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfd6000640 1 -- 172.21.15.119:0/354839517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a96639f50 con 0x561a968b2e50 2024-04-25T10:15:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfd5600640 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 0x561a968c1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.592+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a968c1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:30.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.593+0000 7fdfd6000640 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968b2e50 0x561a968e1580 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800c160 tx=0x7fdfc8001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca06f3fff50a6c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:30.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.593+0000 7fdfb6000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:15:30.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.593+0000 7fdfb6000640 1 -- 172.21.15.119:0/354839517 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8043070 con 0x561a968b2e50 2024-04-25T10:15:30.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.593+0000 7fdfb6000640 1 -- 172.21.15.119:0/354839517 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fdfc80049e0 con 0x561a968b2e50 2024-04-25T10:15:30.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.593+0000 7fdfdade6740 1 -- 172.21.15.119:0/354839517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdfc8038040 con 0x561a968b2e50 2024-04-25T10:15:30.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.593+0000 7fdfb6000640 1 -- 172.21.15.119:0/354839517 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfc8053050 con 0x561a968b2e50 2024-04-25T10:15:30.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.593+0000 7fdfdade6740 10 client.7827 fetch_fsmap learned FSMap version 74 2024-04-25T10:15:30.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.593+0000 7fdfdade6740 1 -- 172.21.15.119:0/354839517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x561a964cd0b0 con 0x561a968b2e50 2024-04-25T10:15:30.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.593+0000 7fdfb6000640 1 -- 172.21.15.119:0/354839517 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc8038040 con 0x561a968b2e50 2024-04-25T10:15:30.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.594+0000 7fdfb6000640 1 -- 172.21.15.119:0/354839517 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdfc8056020 con 0x561a968b2e50 2024-04-25T10:15:30.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.594+0000 7fdfdade6740 10 client.7827 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:15:30.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.594+0000 7fdfdade6740 10 client.7827 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:15:30.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.594+0000 7fdfdade6740 20 client.7827 populate_metadata read hostname 'smithi119' 2024-04-25T10:15:30.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.594+0000 7fdfdade6740 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x561a968dff80 0x561a968e0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.594+0000 7fdfdade6740 4 client.7827 mds_command: new command op to 7668 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:30.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.594+0000 7fdfdade6740 1 -- 172.21.15.119:0/354839517 --> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a96857b60 con 0x561a968dff80 2024-04-25T10:15:30.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.594+0000 7fdfd5600640 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x561a968dff80 0x561a968e0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.594+0000 7fdfd5600640 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x561a968dff80 0x561a968e0360 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e1ed6865cb333b4 server_cookie=ccee7f7b48302d40 in_seq=0 out_seq=0 2024-04-25T10:15:30.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.594+0000 7fdfb6000640 10 client.7827 ms_handle_connect on v2:172.21.15.119:6835/2043821095 2024-04-25T10:15:30.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.595+0000 7fdfb6000640 1 -- 172.21.15.119:0/354839517 <== mds.0 v2:172.21.15.119:6835/2043821095 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561a96857b60 con 0x561a968dff80 2024-04-25T10:15:30.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.595+0000 7fdfb6000640 10 client.7827 handle_command_reply: tid=0 2024-04-25T10:15:30.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.595+0000 7fdfdade6740 1 client.7827 shutdown 2024-04-25T10:15:30.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.595+0000 7fdfdade6740 1 -- 172.21.15.119:0/354839517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968b2e50 msgr2=0x561a968e1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.595+0000 7fdfdade6740 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968b2e50 0x561a968e1580 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800c160 tx=0x7fdfc8001970 comp rx=0 tx=0).stop 2024-04-25T10:15:30.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.595+0000 7fdfdade6740 1 -- 172.21.15.119:0/354839517 shutdown_connections 2024-04-25T10:15:30.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.595+0000 7fdfdade6740 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x561a968dff80 0x561a968e0360 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.595+0000 7fdfdade6740 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968b2e50 0x561a968e1580 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.595+0000 7fdfdade6740 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a968c1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.595+0000 7fdfdade6740 1 --2- 172.21.15.119:0/354839517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 0x561a968c1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.595+0000 7fdfb6000640 0 client.7827 ms_handle_reset on v2:172.21.15.119:6835/2043821095 2024-04-25T10:15:30.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.595+0000 7fdfdade6740 1 -- 172.21.15.119:0/354839517 >> 172.21.15.119:0/354839517 conn(0x561a96896050 msgr2=0x561a9688ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:30.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.596+0000 7fdfdade6740 1 -- 172.21.15.119:0/354839517 shutdown_connections 2024-04-25T10:15:30.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.596+0000 7fdfdade6740 1 -- 172.21.15.119:0/354839517 wait complete. 2024-04-25T10:15:30.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.596+0000 7fdfdade6740 20 client.7827 trim_cache size 0 max 16384 2024-04-25T10:15:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.609+0000 7fdfdade6740 1 Processor -- start 2024-04-25T10:15:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.609+0000 7fdfdade6740 1 -- start start 2024-04-25T10:15:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.609+0000 7fdfdade6740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968b2e50 0x561a968df190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.609+0000 7fdfdade6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968c0370 0x561a968c1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.609+0000 7fdfdade6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a96895230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.609+0000 7fdfdade6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd0168e30 con 0x561a968c0370 2024-04-25T10:15:30.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.610+0000 7fdfdade6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd01f5040 con 0x561a968ddc10 2024-04-25T10:15:30.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.610+0000 7fdfdade6740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd0147b40 con 0x561a968b2e50 2024-04-25T10:15:30.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.610+0000 7fdfd4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968c0370 0x561a968c1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.610+0000 7fdfd4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968c0370 0x561a968c1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53836/0 (socket says 172.21.15.119:53836) 2024-04-25T10:15:30.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.610+0000 7fdfd4c00640 1 -- 172.21.15.119:0/862973433 learned_addr learned my addr 172.21.15.119:0/862973433 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:30.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.610+0000 7fdfd6000640 1 --2- 172.21.15.119:0/862973433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a96895230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.610+0000 7fdfd5600640 1 --2- 172.21.15.119:0/862973433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968b2e50 0x561a968df190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.610+0000 7fdfd4c00640 1 -- 172.21.15.119:0/862973433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968b2e50 msgr2=0x561a968df190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:30.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.610+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/862973433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968b2e50 0x561a968df190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.610+0000 7fdfd4c00640 1 -- 172.21.15.119:0/862973433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 msgr2=0x561a96895230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:30.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.610+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/862973433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a96895230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.610+0000 7fdfd4c00640 1 -- 172.21.15.119:0/862973433 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a968c1a90 con 0x561a968c0370 2024-04-25T10:15:30.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.610+0000 7fdfd6000640 1 --2- 172.21.15.119:0/862973433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a96895230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:30.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.611+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/862973433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968c0370 0x561a968c1550 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc012cd0 tx=0x7fdfbc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9525781d494e358f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:30.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.611+0000 7fdfc1200640 1 -- 172.21.15.119:0/862973433 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc026070 con 0x561a968c0370 2024-04-25T10:15:30.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.611+0000 7fdfc1200640 1 -- 172.21.15.119:0/862973433 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc01b040 con 0x561a968c0370 2024-04-25T10:15:30.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.611+0000 7fdfc1200640 1 -- 172.21.15.119:0/862973433 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc022810 con 0x561a968c0370 2024-04-25T10:15:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.612+0000 7fdfdade6740 1 -- 172.21.15.119:0/862973433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968c0370 msgr2=0x561a968c1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.612+0000 7fdfdade6740 1 --2- 172.21.15.119:0/862973433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968c0370 0x561a968c1550 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc012cd0 tx=0x7fdfbc015000 comp rx=0 tx=0).stop 2024-04-25T10:15:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.612+0000 7fdfdade6740 1 -- 172.21.15.119:0/862973433 shutdown_connections 2024-04-25T10:15:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.612+0000 7fdfdade6740 1 --2- 172.21.15.119:0/862973433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a96895230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.612+0000 7fdfdade6740 1 --2- 172.21.15.119:0/862973433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968c0370 0x561a968c1550 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.612+0000 7fdfdade6740 1 --2- 172.21.15.119:0/862973433 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968b2e50 0x561a968df190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.612+0000 7fdfdade6740 1 -- 172.21.15.119:0/862973433 >> 172.21.15.119:0/862973433 conn(0x561a96896050 msgr2=0x561a9688be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.612+0000 7fdfdade6740 1 -- 172.21.15.119:0/862973433 shutdown_connections 2024-04-25T10:15:30.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.612+0000 7fdfdade6740 1 -- 172.21.15.119:0/862973433 wait complete. 2024-04-25T10:15:30.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.612+0000 7fdfdade6740 1 -- start start 2024-04-25T10:15:30.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.613+0000 7fdfdade6740 1 Processor -- start 2024-04-25T10:15:30.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfdade6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968b2e50 0x561a96895630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfdade6740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 0x561a96948ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfdade6740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a96949000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfdade6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a968e1000 con 0x561a968b2e50 2024-04-25T10:15:30.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfdade6740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a96896df0 con 0x561a968ddc10 2024-04-25T10:15:30.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfdade6740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a968f2a20 con 0x561a968c0370 2024-04-25T10:15:30.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfdade6740 10 client.0 _ll_register_callbacks cb 0x561a96890d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:30.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfd5600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968b2e50 0x561a96895630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfd5600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968b2e50 0x561a96895630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53850/0 (socket says 172.21.15.119:53850) 2024-04-25T10:15:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfd5600640 1 -- 172.21.15.119:0/2849149181 learned_addr learned my addr 172.21.15.119:0/2849149181 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfd5600640 1 -- 172.21.15.119:0/2849149181 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 msgr2=0x561a96948ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfd5600640 1 --2- 172.21.15.119:0/2849149181 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 0x561a96948ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfd5600640 1 -- 172.21.15.119:0/2849149181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 msgr2=0x561a96949000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfd5600640 1 --2- 172.21.15.119:0/2849149181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a96949000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfd5600640 1 -- 172.21.15.119:0/2849149181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a96895b70 con 0x561a968b2e50 2024-04-25T10:15:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.614+0000 7fdfd5600640 1 --2- 172.21.15.119:0/2849149181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968b2e50 0x561a96895630 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4010a90 tx=0x7fdfc4009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332f6b4d0f9e230f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfb6000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:15:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfb6000640 1 -- 172.21.15.119:0/2849149181 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc405d9f0 con 0x561a968b2e50 2024-04-25T10:15:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfb6000640 1 -- 172.21.15.119:0/2849149181 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fdfc4055f10 con 0x561a968b2e50 2024-04-25T10:15:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfdade6740 1 -- 172.21.15.119:0/2849149181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdfbc01b040 con 0x561a968b2e50 2024-04-25T10:15:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfb6000640 1 -- 172.21.15.119:0/2849149181 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfc4061fb0 con 0x561a968b2e50 2024-04-25T10:15:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfdade6740 10 client.7833 fetch_fsmap learned FSMap version 74 2024-04-25T10:15:30.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfdade6740 1 -- 172.21.15.119:0/2849149181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x561a9694a9b0 con 0x561a968b2e50 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfb6000640 1 -- 172.21.15.119:0/2849149181 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc01b040 con 0x561a968b2e50 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfb6000640 1 -- 172.21.15.119:0/2849149181 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdfc4027060 con 0x561a968b2e50 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfdade6740 10 client.7833 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfdade6740 10 client.7833 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfdade6740 20 client.7833 populate_metadata read hostname 'smithi119' 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfdade6740 1 --2- 172.21.15.119:0/2849149181 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x561a96890160 0x561a96949930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfdade6740 4 client.7833 mds_command: new command op to 7668 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.615+0000 7fdfdade6740 1 -- 172.21.15.119:0/2849149181 --> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x561a96806020 con 0x561a96890160 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.616+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/2849149181 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x561a96890160 0x561a96949930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.616+0000 7fdfb6000640 10 client.7833 ms_handle_connect on v2:172.21.15.119:6835/2043821095 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.616+0000 7fdfd4c00640 1 --2- 172.21.15.119:0/2849149181 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x561a96890160 0x561a96949930 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7ad6db8974a8e8d server_cookie=b32d04d1d47d9faf in_seq=0 out_seq=0 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.616+0000 7fdfb6000640 1 -- 172.21.15.119:0/2849149181 <== mds.0 v2:172.21.15.119:6835/2043821095 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561a96806020 con 0x561a96890160 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.616+0000 7fdfb6000640 10 client.7833 handle_command_reply: tid=0 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.616+0000 7fdfdade6740 1 client.7833 shutdown 2024-04-25T10:15:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.617+0000 7fdfdade6740 1 -- 172.21.15.119:0/2849149181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968b2e50 msgr2=0x561a96895630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.617+0000 7fdfdade6740 1 --2- 172.21.15.119:0/2849149181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968b2e50 0x561a96895630 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4010a90 tx=0x7fdfc4009d20 comp rx=0 tx=0).stop 2024-04-25T10:15:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.617+0000 7fdfdade6740 1 -- 172.21.15.119:0/2849149181 shutdown_connections 2024-04-25T10:15:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.617+0000 7fdfdade6740 1 --2- 172.21.15.119:0/2849149181 >> [v2:172.21.15.119:6835/2043821095,v1:172.21.15.119:6837/2043821095] conn(0x561a96890160 0x561a96949930 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.617+0000 7fdfdade6740 1 --2- 172.21.15.119:0/2849149181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a968ddc10 0x561a96949000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.617+0000 7fdfdade6740 1 --2- 172.21.15.119:0/2849149181 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561a968c0370 0x561a96948ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.617+0000 7fdfdade6740 1 --2- 172.21.15.119:0/2849149181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a968b2e50 0x561a96895630 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.617+0000 7fdfb6000640 0 client.7833 ms_handle_reset on v2:172.21.15.119:6835/2043821095 2024-04-25T10:15:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.617+0000 7fdfdade6740 1 -- 172.21.15.119:0/2849149181 >> 172.21.15.119:0/2849149181 conn(0x561a96896050 msgr2=0x561a9688be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.617+0000 7fdfdade6740 1 -- 172.21.15.119:0/2849149181 shutdown_connections 2024-04-25T10:15:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.617+0000 7fdfdade6740 1 -- 172.21.15.119:0/2849149181 wait complete. 2024-04-25T10:15:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.617+0000 7fdfdade6740 20 client.7833 trim_cache size 0 max 16384 2024-04-25T10:15:30.625 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:30.625 INFO:teuthology.orchestra.run.smithi119.stdout:[] 2024-04-25T10:15:30.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.618+0000 7fdfd8000640 1 -- 172.21.15.119:0/1142704161 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdfb006b4d0 msgr2=0x7fdfb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.618+0000 7fdfd8000640 1 --2- 172.21.15.119:0/1142704161 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdfb006b4d0 0x7fdfb006d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00b26c0 tx=0x7fdfbc008040 comp rx=0 tx=0).stop 2024-04-25T10:15:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.618+0000 7fdfd8000640 1 -- 172.21.15.119:0/1142704161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfb80051c0 msgr2=0x7fdfd014a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.618+0000 7fdfd8000640 1 --2- 172.21.15.119:0/1142704161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfb80051c0 0x7fdfd014a330 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4007f00 tx=0x7fdfc4010590 comp rx=0 tx=0).stop 2024-04-25T10:15:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.618+0000 7fdfd8000640 1 -- 172.21.15.119:0/1142704161 shutdown_connections 2024-04-25T10:15:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.618+0000 7fdfd8000640 1 --2- 172.21.15.119:0/1142704161 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdfb006b4d0 0x7fdfb006d990 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.618+0000 7fdfd8000640 1 --2- 172.21.15.119:0/1142704161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdfd0148970 0x7fdfd01f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.618+0000 7fdfd8000640 1 --2- 172.21.15.119:0/1142704161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdfd0148510 0x7fdfd0148f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.618+0000 7fdfd8000640 1 --2- 172.21.15.119:0/1142704161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfb80051c0 0x7fdfd014a330 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.618+0000 7fdfd8000640 1 -- 172.21.15.119:0/1142704161 >> 172.21.15.119:0/1142704161 conn(0x7fdfd006aba0 msgr2=0x7fdfd015d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.618+0000 7fdfd8000640 1 -- 172.21.15.119:0/1142704161 shutdown_connections 2024-04-25T10:15:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.618+0000 7fdfd8000640 1 -- 172.21.15.119:0/1142704161 wait complete. 2024-04-25T10:15:30.638 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:15:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.828+0000 7fa23d600640 1 Processor -- start 2024-04-25T10:15:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.828+0000 7fa23d600640 1 -- start start 2024-04-25T10:15:30.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.830+0000 7fa23d600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa23815bf00 0x7fa2380cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.830+0000 7fa23d600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2380ce6c0 con 0x7fa23806e8f0 2024-04-25T10:15:30.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.830+0000 7fa23d600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2380cc7b0 con 0x7fa2380cc310 2024-04-25T10:15:30.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.830+0000 7fa23d600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2381455e0 con 0x7fa23815bf00 2024-04-25T10:15:30.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.830+0000 7fa236a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa23815bf00 0x7fa2380cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.830+0000 7fa236a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa23815bf00 0x7fa2380cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43494/0 (socket says 172.21.15.119:43494) 2024-04-25T10:15:30.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.830+0000 7fa237400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa23806e8f0 0x7fa2381654b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50002/0 (socket says 172.21.15.119:50002) 2024-04-25T10:15:30.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.830+0000 7fa236a00640 1 -- 172.21.15.119:0/2527599811 learned_addr learned my addr 172.21.15.119:0/2527599811 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:30.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.831+0000 7fa236a00640 1 -- 172.21.15.119:0/2527599811 >> v1:172.21.15.132:6789/0 conn(0x7fa2380cc310 legacy=0x7fa23815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:30.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.831+0000 7fa236a00640 1 -- 172.21.15.119:0/2527599811 >> v1:172.21.15.119:6789/0 conn(0x7fa23806e8f0 legacy=0x7fa2381654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.831+0000 7fa236a00640 1 -- 172.21.15.119:0/2527599811 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2380cca90 con 0x7fa23815bf00 2024-04-25T10:15:30.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.832+0000 7fa236a00640 1 --2- 172.21.15.119:0/2527599811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23815bf00 0x7fa2380cbdd0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fa228009870 tx=0x7fa22802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d3a7e7de1d08c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:30.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.833+0000 7fa235600640 1 -- 172.21.15.119:0/2527599811 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228037070 con 0x7fa23815bf00 2024-04-25T10:15:30.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.833+0000 7fa235600640 1 -- 172.21.15.119:0/2527599811 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa228032440 con 0x7fa23815bf00 2024-04-25T10:15:30.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.833+0000 7fa235600640 1 -- 172.21.15.119:0/2527599811 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228032810 con 0x7fa23815bf00 2024-04-25T10:15:30.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.833+0000 7fa23d600640 1 -- 172.21.15.119:0/2527599811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23815bf00 msgr2=0x7fa2380cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.834+0000 7fa23d600640 1 --2- 172.21.15.119:0/2527599811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23815bf00 0x7fa2380cbdd0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fa228009870 tx=0x7fa22802a2a0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.834+0000 7fa23d600640 1 -- 172.21.15.119:0/2527599811 shutdown_connections 2024-04-25T10:15:30.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.834+0000 7fa23d600640 1 --2- 172.21.15.119:0/2527599811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23815bf00 0x7fa2380cbdd0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.834+0000 7fa23d600640 1 -- 172.21.15.119:0/2527599811 >> 172.21.15.119:0/2527599811 conn(0x7fa23806aba0 msgr2=0x7fa23806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:30.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.834+0000 7fa23d600640 1 -- 172.21.15.119:0/2527599811 shutdown_connections 2024-04-25T10:15:30.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.834+0000 7fa23d600640 1 -- 172.21.15.119:0/2527599811 wait complete. 2024-04-25T10:15:30.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.835+0000 7fa23d600640 1 Processor -- start 2024-04-25T10:15:30.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.836+0000 7fa23d600640 1 -- start start 2024-04-25T10:15:30.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.836+0000 7fa23d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23806e8f0 0x7fa238156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.837+0000 7fa23d600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2380cc310 0x7fa238159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.837+0000 7fa23d600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa23815bf00 0x7fa2381598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.837+0000 7fa23d600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380cd050 con 0x7fa23806e8f0 2024-04-25T10:15:30.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.837+0000 7fa23d600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23806d450 con 0x7fa23815bf00 2024-04-25T10:15:30.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.837+0000 7fa23d600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23806ee60 con 0x7fa2380cc310 2024-04-25T10:15:30.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.837+0000 7fa236a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23806e8f0 0x7fa238156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.837+0000 7fa236a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23806e8f0 0x7fa238156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43496/0 (socket says 172.21.15.119:43496) 2024-04-25T10:15:30.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.837+0000 7fa236a00640 1 -- 172.21.15.119:0/2800198684 learned_addr learned my addr 172.21.15.119:0/2800198684 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:30.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.837+0000 7fa236a00640 1 -- 172.21.15.119:0/2800198684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2380cc310 msgr2=0x7fa238159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.837+0000 7fa236a00640 1 --2- 172.21.15.119:0/2800198684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2380cc310 0x7fa238159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.837+0000 7fa236a00640 1 -- 172.21.15.119:0/2800198684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa23815bf00 msgr2=0x7fa2381598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:30.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.837+0000 7fa236a00640 1 --2- 172.21.15.119:0/2800198684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa23815bf00 0x7fa2381598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:30.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.837+0000 7fa236a00640 1 -- 172.21.15.119:0/2800198684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa238156790 con 0x7fa23806e8f0 2024-04-25T10:15:30.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.838+0000 7fa236a00640 1 --2- 172.21.15.119:0/2800198684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23806e8f0 0x7fa238156250 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fa228009870 tx=0x7fa2280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:30.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.838+0000 7fa22ea00640 1 -- 172.21.15.119:0/2800198684 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228041070 con 0x7fa23806e8f0 2024-04-25T10:15:30.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.838+0000 7fa22ea00640 1 -- 172.21.15.119:0/2800198684 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22803c870 con 0x7fa23806e8f0 2024-04-25T10:15:30.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.838+0000 7fa22ea00640 1 -- 172.21.15.119:0/2800198684 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228037040 con 0x7fa23806e8f0 2024-04-25T10:15:30.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.838+0000 7fa23d600640 1 -- 172.21.15.119:0/2800198684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2381328e0 con 0x7fa23806e8f0 2024-04-25T10:15:30.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.838+0000 7fa23d600640 1 -- 172.21.15.119:0/2800198684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa238132ea0 con 0x7fa23806e8f0 2024-04-25T10:15:30.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.839+0000 7fa23d600640 1 -- 172.21.15.119:0/2800198684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2380749b0 con 0x7fa23806e8f0 2024-04-25T10:15:30.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.840+0000 7fa22ea00640 1 -- 172.21.15.119:0/2800198684 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa228044070 con 0x7fa23806e8f0 2024-04-25T10:15:30.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.841+0000 7fa22ea00640 1 --2- 172.21.15.119:0/2800198684 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa21406b4d0 0x7fa21406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:30.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.841+0000 7fa22ea00640 1 -- 172.21.15.119:0/2800198684 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fa2280aae70 con 0x7fa23806e8f0 2024-04-25T10:15:30.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.841+0000 7fa236000640 1 --2- 172.21.15.119:0/2800198684 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa21406b4d0 0x7fa21406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:30.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.842+0000 7fa236000640 1 --2- 172.21.15.119:0/2800198684 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa21406b4d0 0x7fa21406d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa2380cfae0 tx=0x7fa2200079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:30.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:30.846+0000 7fa22ea00640 1 -- 172.21.15.119:0/2800198684 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa238156790 con 0x7fa23806e8f0 2024-04-25T10:15:31.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.083+0000 7fa23d600640 1 -- 172.21.15.119:0/2800198684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa238133980 con 0x7fa23806e8f0 2024-04-25T10:15:31.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.084+0000 7fa22ea00640 1 -- 172.21.15.119:0/2800198684 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa228079c10 con 0x7fa23806e8f0 2024-04-25T10:15:31.088 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:31.088 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:15:31.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.086+0000 7fa23d600640 1 -- 172.21.15.119:0/2800198684 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa21406b4d0 msgr2=0x7fa21406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:31.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.086+0000 7fa23d600640 1 --2- 172.21.15.119:0/2800198684 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa21406b4d0 0x7fa21406d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa2380cfae0 tx=0x7fa2200079b0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.086+0000 7fa23d600640 1 -- 172.21.15.119:0/2800198684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23806e8f0 msgr2=0x7fa238156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:31.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.086+0000 7fa23d600640 1 --2- 172.21.15.119:0/2800198684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23806e8f0 0x7fa238156250 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fa228009870 tx=0x7fa2280046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.086+0000 7fa23d600640 1 -- 172.21.15.119:0/2800198684 shutdown_connections 2024-04-25T10:15:31.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.086+0000 7fa23d600640 1 --2- 172.21.15.119:0/2800198684 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa21406b4d0 0x7fa21406d990 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.086+0000 7fa23d600640 1 --2- 172.21.15.119:0/2800198684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa23815bf00 0x7fa2381598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.086+0000 7fa23d600640 1 --2- 172.21.15.119:0/2800198684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2380cc310 0x7fa238159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.086+0000 7fa23d600640 1 --2- 172.21.15.119:0/2800198684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23806e8f0 0x7fa238156250 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.086+0000 7fa23d600640 1 -- 172.21.15.119:0/2800198684 >> 172.21.15.119:0/2800198684 conn(0x7fa23806aba0 msgr2=0x7fa238143bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:31.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.086+0000 7fa23d600640 1 -- 172.21.15.119:0/2800198684 shutdown_connections 2024-04-25T10:15:31.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.086+0000 7fa23d600640 1 -- 172.21.15.119:0/2800198684 wait complete. 2024-04-25T10:15:31.108 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:15:31.109 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:15:31.109 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:31.197 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:31.313 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:15:31.313 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:15:31.313 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:31.369 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:15:31.369 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:31.421 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:31.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.621+0000 7fcd75e00640 1 Processor -- start 2024-04-25T10:15:31.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.621+0000 7fcd75e00640 1 -- start start 2024-04-25T10:15:31.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.623+0000 7fcd75e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7015d980 0x7fcd700ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:31.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.623+0000 7fcd75e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcd700aeb20 0x7fcd700cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:31.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.623+0000 7fcd75e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd700712c0 con 0x7fcd700cf7c0 2024-04-25T10:15:31.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.623+0000 7fcd75e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd70071ed0 con 0x7fcd700aeb20 2024-04-25T10:15:31.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.623+0000 7fcd75e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700cfbf0 con 0x7fcd7015d980 2024-04-25T10:15:31.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.623+0000 7fcd6f400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcd700cf7c0 0x7fcd7015dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50008/0 (socket says 172.21.15.119:50008) 2024-04-25T10:15:31.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.623+0000 7fcd6f400640 1 -- 172.21.15.119:0/1920865589 learned_addr learned my addr 172.21.15.119:0/1920865589 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:31.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.623+0000 7fcd6e000640 1 --2- 172.21.15.119:0/1920865589 >> v2:172.21.15.132:3300/0 conn(0x7fcd700aeb20 0x7fcd700cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:31.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6ea00640 1 --2- 172.21.15.119:0/1920865589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7015d980 0x7fcd700ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:31.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 -- 172.21.15.119:0/1920865589 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645730282 0 0) 0x7fcd700712c0 con 0x7fcd700cf7c0 2024-04-25T10:15:31.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 -- 172.21.15.119:0/1920865589 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd54003520 con 0x7fcd700cf7c0 2024-04-25T10:15:31.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 -- 172.21.15.119:0/1920865589 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 488717091 0 0) 0x7fcd54003520 con 0x7fcd700cf7c0 2024-04-25T10:15:31.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 -- 172.21.15.119:0/1920865589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7015d980 msgr2=0x7fcd700ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:31.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 --2- 172.21.15.119:0/1920865589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7015d980 0x7fcd700ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 -- 172.21.15.119:0/1920865589 >> v2:172.21.15.132:3300/0 conn(0x7fcd700aeb20 msgr2=0x7fcd700cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:31.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 --2- 172.21.15.119:0/1920865589 >> v2:172.21.15.132:3300/0 conn(0x7fcd700aeb20 0x7fcd700cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 -- 172.21.15.119:0/1920865589 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd700715a0 con 0x7fcd700cf7c0 2024-04-25T10:15:31.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 -- 172.21.15.119:0/1920865589 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fcd680019f0 con 0x7fcd700cf7c0 2024-04-25T10:15:31.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 -- 172.21.15.119:0/1920865589 >> v1:172.21.15.119:6789/0 conn(0x7fcd700cf7c0 legacy=0x7fcd7015dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:31.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 --2- 172.21.15.119:0/1920865589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7015d980 0x7fcd540039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:31.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 --2- 172.21.15.119:0/1920865589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd54003f50 0x7fcd54004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:31.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 --2- 172.21.15.119:0/1920865589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54004d40 0x7fcd540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:31.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 -- 172.21.15.119:0/1920865589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70071ed0 con 0x7fcd54004d40 2024-04-25T10:15:31.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 -- 172.21.15.119:0/1920865589 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700cfbf0 con 0x7fcd54003f50 2024-04-25T10:15:31.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.624+0000 7fcd6d600640 1 -- 172.21.15.119:0/1920865589 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5400ba40 con 0x7fcd7015d980 2024-04-25T10:15:31.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.625+0000 7fcd6f400640 1 --2- 172.21.15.119:0/1920865589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54004d40 0x7fcd540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:31.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.625+0000 7fcd6ea00640 1 --2- 172.21.15.119:0/1920865589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7015d980 0x7fcd540039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:31.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.625+0000 7fcd6e000640 1 --2- 172.21.15.119:0/1920865589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd54003f50 0x7fcd54004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:31.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.625+0000 7fcd6f400640 1 -- 172.21.15.119:0/1920865589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7015d980 msgr2=0x7fcd540039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:31.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.625+0000 7fcd6f400640 1 --2- 172.21.15.119:0/1920865589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7015d980 0x7fcd540039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.625+0000 7fcd6f400640 1 -- 172.21.15.119:0/1920865589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd54003f50 msgr2=0x7fcd54004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:31.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.625+0000 7fcd6f400640 1 --2- 172.21.15.119:0/1920865589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd54003f50 0x7fcd54004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.625+0000 7fcd6f400640 1 -- 172.21.15.119:0/1920865589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd54003520 con 0x7fcd54004d40 2024-04-25T10:15:31.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.625+0000 7fcd6ea00640 1 --2- 172.21.15.119:0/1920865589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7015d980 0x7fcd540039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:31.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.626+0000 7fcd6f400640 1 --2- 172.21.15.119:0/1920865589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54004d40 0x7fcd540051c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800a920 tx=0x7fcd54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed63b206a30105c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.626+0000 7fcd6d600640 1 -- 172.21.15.119:0/1920865589 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800a4c0 con 0x7fcd54004d40 2024-04-25T10:15:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.626+0000 7fcd6d600640 1 -- 172.21.15.119:0/1920865589 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6802f070 con 0x7fcd54004d40 2024-04-25T10:15:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.626+0000 7fcd6d600640 1 -- 172.21.15.119:0/1920865589 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6802bd40 con 0x7fcd54004d40 2024-04-25T10:15:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.627+0000 7fcd75e00640 1 -- 172.21.15.119:0/1920865589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54004d40 msgr2=0x7fcd540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.627+0000 7fcd75e00640 1 --2- 172.21.15.119:0/1920865589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54004d40 0x7fcd540051c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800a920 tx=0x7fcd54005830 comp rx=0 tx=0).stop 2024-04-25T10:15:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.627+0000 7fcd6f400640 1 -- 172.21.15.119:0/1920865589 reap_dead start 2024-04-25T10:15:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.627+0000 7fcd75e00640 1 -- 172.21.15.119:0/1920865589 shutdown_connections 2024-04-25T10:15:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.627+0000 7fcd75e00640 1 -- 172.21.15.119:0/1920865589 >> 172.21.15.119:0/1920865589 conn(0x7fcd7006aba0 msgr2=0x7fcd7006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.627+0000 7fcd75e00640 1 -- 172.21.15.119:0/1920865589 shutdown_connections 2024-04-25T10:15:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.627+0000 7fcd75e00640 1 -- 172.21.15.119:0/1920865589 wait complete. 2024-04-25T10:15:31.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.629+0000 7fcd75e00640 1 Processor -- start 2024-04-25T10:15:31.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.629+0000 7fcd75e00640 1 -- start start 2024-04-25T10:15:31.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.630+0000 7fcd75e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54004d40 0x7fcd70150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:31.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.630+0000 7fcd75e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd7015bc80 0x7fcd70142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:31.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.630+0000 7fcd75e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7015c0e0 0x7fcd701430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:31.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.630+0000 7fcd75e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700adf80 con 0x7fcd54004d40 2024-04-25T10:15:31.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.630+0000 7fcd75e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70071b60 con 0x7fcd7015bc80 2024-04-25T10:15:31.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.630+0000 7fcd75e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700ae100 con 0x7fcd7015c0e0 2024-04-25T10:15:31.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.630+0000 7fcd6ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54004d40 0x7fcd70150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:31.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.630+0000 7fcd6ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54004d40 0x7fcd70150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43522/0 (socket says 172.21.15.119:43522) 2024-04-25T10:15:31.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.630+0000 7fcd6ea00640 1 -- 172.21.15.119:0/1532538944 learned_addr learned my addr 172.21.15.119:0/1532538944 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:31.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.630+0000 7fcd6ea00640 1 -- 172.21.15.119:0/1532538944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7015c0e0 msgr2=0x7fcd701430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:31.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.630+0000 7fcd6ea00640 1 --2- 172.21.15.119:0/1532538944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7015c0e0 0x7fcd701430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.630+0000 7fcd6ea00640 1 -- 172.21.15.119:0/1532538944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd7015bc80 msgr2=0x7fcd70142b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:31.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.630+0000 7fcd6ea00640 1 --2- 172.21.15.119:0/1532538944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd7015bc80 0x7fcd70142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.630+0000 7fcd6ea00640 1 -- 172.21.15.119:0/1532538944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7015df00 con 0x7fcd54004d40 2024-04-25T10:15:31.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.631+0000 7fcd6ea00640 1 --2- 172.21.15.119:0/1532538944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54004d40 0x7fcd70150280 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fcd600116d0 tx=0x7fcd60011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:31.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.631+0000 7fcd67400640 1 -- 172.21.15.119:0/1532538944 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000f940 con 0x7fcd54004d40 2024-04-25T10:15:31.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.631+0000 7fcd67400640 1 -- 172.21.15.119:0/1532538944 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6000fae0 con 0x7fcd54004d40 2024-04-25T10:15:31.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.631+0000 7fcd67400640 1 -- 172.21.15.119:0/1532538944 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60015460 con 0x7fcd54004d40 2024-04-25T10:15:31.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.631+0000 7fcd75e00640 1 -- 172.21.15.119:0/1532538944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7015e1e0 con 0x7fcd54004d40 2024-04-25T10:15:31.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.631+0000 7fcd75e00640 1 -- 172.21.15.119:0/1532538944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7015e7a0 con 0x7fcd54004d40 2024-04-25T10:15:31.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.632+0000 7fcd75e00640 1 -- 172.21.15.119:0/1532538944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd700749b0 con 0x7fcd54004d40 2024-04-25T10:15:31.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.633+0000 7fcd67400640 1 -- 172.21.15.119:0/1532538944 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd60015600 con 0x7fcd54004d40 2024-04-25T10:15:31.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.634+0000 7fcd67400640 1 --2- 172.21.15.119:0/1532538944 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd5006b4d0 0x7fcd5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:31.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.634+0000 7fcd67400640 1 -- 172.21.15.119:0/1532538944 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fcd6008ad60 con 0x7fcd54004d40 2024-04-25T10:15:31.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.634+0000 7fcd6e000640 1 --2- 172.21.15.119:0/1532538944 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd5006b4d0 0x7fcd5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:31.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.635+0000 7fcd6e000640 1 --2- 172.21.15.119:0/1532538944 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd5006b4d0 0x7fcd5006d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcd7015cb40 tx=0x7fcd5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:31.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.638+0000 7fcd67400640 1 -- 172.21.15.119:0/1532538944 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd60019030 con 0x7fcd54004d40 2024-04-25T10:15:31.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.834+0000 7fcd75e00640 1 -- 172.21.15.119:0/1532538944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd7006d4c0 con 0x7fcd54004d40 2024-04-25T10:15:31.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.836+0000 7fcd67400640 1 -- 172.21.15.119:0/1532538944 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17009 (secure 0 0 0) 0x7fcd60059a70 con 0x7fcd54004d40 2024-04-25T10:15:31.840 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:31.840 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":85,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:14:50.735243+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:21.210576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:22.438674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/1734507541":"2024-04-26T10:14:50.735180+0000","172.21.15.119:6835/1734507541":"2024-04-26T10:14:50.735180+0000","172.21.15.119:6837/3997964942":"2024-04-26T10:13:36.160373+0000","172.21.15.119:6835/3997964942":"2024-04-26T10:13:36.160373+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:31.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.838+0000 7fcd75e00640 1 -- 172.21.15.119:0/1532538944 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd5006b4d0 msgr2=0x7fcd5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:31.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.838+0000 7fcd75e00640 1 --2- 172.21.15.119:0/1532538944 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd5006b4d0 0x7fcd5006d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcd7015cb40 tx=0x7fcd5c008040 comp rx=0 tx=0).stop 2024-04-25T10:15:31.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.838+0000 7fcd75e00640 1 -- 172.21.15.119:0/1532538944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54004d40 msgr2=0x7fcd70150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:31.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.838+0000 7fcd75e00640 1 --2- 172.21.15.119:0/1532538944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54004d40 0x7fcd70150280 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fcd600116d0 tx=0x7fcd60011b90 comp rx=0 tx=0).stop 2024-04-25T10:15:31.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.838+0000 7fcd75e00640 1 -- 172.21.15.119:0/1532538944 shutdown_connections 2024-04-25T10:15:31.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.838+0000 7fcd75e00640 1 --2- 172.21.15.119:0/1532538944 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd5006b4d0 0x7fcd5006d990 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.838+0000 7fcd75e00640 1 --2- 172.21.15.119:0/1532538944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd7015c0e0 0x7fcd701430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.838+0000 7fcd75e00640 1 --2- 172.21.15.119:0/1532538944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd7015bc80 0x7fcd70142b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.838+0000 7fcd75e00640 1 --2- 172.21.15.119:0/1532538944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54004d40 0x7fcd70150280 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:31.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.838+0000 7fcd75e00640 1 -- 172.21.15.119:0/1532538944 >> 172.21.15.119:0/1532538944 conn(0x7fcd7006aba0 msgr2=0x7fcd700aef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:31.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.839+0000 7fcd75e00640 1 -- 172.21.15.119:0/1532538944 shutdown_connections 2024-04-25T10:15:31.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:31.839+0000 7fcd75e00640 1 -- 172.21.15.119:0/1532538944 wait complete. 2024-04-25T10:15:31.862 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:21.210576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:22.438674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:31.862 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:32.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.063+0000 7fae7c400640 1 Processor -- start 2024-04-25T10:15:32.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.063+0000 7fae7c400640 1 -- start start 2024-04-25T10:15:32.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.064+0000 7fae7c400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fae74144e70 0x7fae7406d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.064+0000 7fae7c400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fae7406d7a0 0x7fae740ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.064+0000 7fae7c400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7406dcb0 con 0x7fae740cef10 2024-04-25T10:15:32.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.065+0000 7fae7c400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7415bfe0 con 0x7fae7406d7a0 2024-04-25T10:15:32.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.065+0000 7fae7c400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7406df90 con 0x7fae74144e70 2024-04-25T10:15:32.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.065+0000 7fae79000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fae7406d7a0 0x7fae740ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.065+0000 7fae79000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fae7406d7a0 0x7fae740ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43524/0 (socket says 172.21.15.119:43524) 2024-04-25T10:15:32.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.065+0000 7fae79000640 1 -- 172.21.15.119:0/288766564 learned_addr learned my addr 172.21.15.119:0/288766564 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:32.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.065+0000 7fae79a00640 1 --2- 172.21.15.119:0/288766564 >> v2:172.21.15.132:3300/0 conn(0x7fae74144e70 0x7fae7406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.065+0000 7fae79000640 1 -- 172.21.15.119:0/288766564 >> v2:172.21.15.132:3300/0 conn(0x7fae74144e70 msgr2=0x7fae7406d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:32.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.065+0000 7fae79000640 1 --2- 172.21.15.119:0/288766564 >> v2:172.21.15.132:3300/0 conn(0x7fae74144e70 0x7fae7406d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.065+0000 7fae79000640 1 -- 172.21.15.119:0/288766564 >> v1:172.21.15.132:6789/0 conn(0x7fae740cef10 legacy=0x7fae74071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:32.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.065+0000 7fae79000640 1 -- 172.21.15.119:0/288766564 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7406e270 con 0x7fae7406d7a0 2024-04-25T10:15:32.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.066+0000 7fae79000640 1 --2- 172.21.15.119:0/288766564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae7406d7a0 0x7fae740ce9d0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fae740b4a20 tx=0x7fae6002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a551d890accf81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:32.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.067+0000 7fae73e00640 1 -- 172.21.15.119:0/288766564 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60037070 con 0x7fae7406d7a0 2024-04-25T10:15:32.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.067+0000 7fae73e00640 1 -- 172.21.15.119:0/288766564 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae60032440 con 0x7fae7406d7a0 2024-04-25T10:15:32.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.067+0000 7fae73e00640 1 -- 172.21.15.119:0/288766564 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60032810 con 0x7fae7406d7a0 2024-04-25T10:15:32.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.067+0000 7fae7c400640 1 -- 172.21.15.119:0/288766564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae7406d7a0 msgr2=0x7fae740ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.067+0000 7fae7c400640 1 --2- 172.21.15.119:0/288766564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae7406d7a0 0x7fae740ce9d0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fae740b4a20 tx=0x7fae6002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.068+0000 7fae7c400640 1 -- 172.21.15.119:0/288766564 shutdown_connections 2024-04-25T10:15:32.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.068+0000 7fae7c400640 1 --2- 172.21.15.119:0/288766564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae7406d7a0 0x7fae740ce9d0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.068+0000 7fae7c400640 1 --2- 172.21.15.119:0/288766564 >> v2:172.21.15.132:3300/0 conn(0x7fae74144e70 0x7fae7406d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.068+0000 7fae7c400640 1 -- 172.21.15.119:0/288766564 >> 172.21.15.119:0/288766564 conn(0x7fae7406aba0 msgr2=0x7fae7406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:32.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.068+0000 7fae7c400640 1 -- 172.21.15.119:0/288766564 shutdown_connections 2024-04-25T10:15:32.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.068+0000 7fae7c400640 1 -- 172.21.15.119:0/288766564 wait complete. 2024-04-25T10:15:32.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.070+0000 7fae7c400640 1 Processor -- start 2024-04-25T10:15:32.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.070+0000 7fae7c400640 1 -- start start 2024-04-25T10:15:32.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.071+0000 7fae7c400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae7406d7a0 0x7fae7413c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.071+0000 7fae7c400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae740cef10 0x7fae7413dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.071+0000 7fae7c400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae74144e70 0x7fae74152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.071+0000 7fae7c400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae741454b0 con 0x7fae740cef10 2024-04-25T10:15:32.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.071+0000 7fae7c400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae740cf370 con 0x7fae7406d7a0 2024-04-25T10:15:32.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.071+0000 7fae7c400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae74145630 con 0x7fae74144e70 2024-04-25T10:15:32.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.071+0000 7fae79000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae740cef10 0x7fae7413dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.071+0000 7fae79000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae740cef10 0x7fae7413dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43534/0 (socket says 172.21.15.119:43534) 2024-04-25T10:15:32.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.071+0000 7fae79000640 1 -- 172.21.15.119:0/3788975531 learned_addr learned my addr 172.21.15.119:0/3788975531 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:32.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.072+0000 7fae79a00640 1 --2- 172.21.15.119:0/3788975531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae7406d7a0 0x7fae7413c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.072+0000 7fae79000640 1 -- 172.21.15.119:0/3788975531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae74144e70 msgr2=0x7fae74152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.072+0000 7fae79000640 1 --2- 172.21.15.119:0/3788975531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae74144e70 0x7fae74152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.072+0000 7fae79000640 1 -- 172.21.15.119:0/3788975531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae7406d7a0 msgr2=0x7fae7413c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.072+0000 7fae79000640 1 --2- 172.21.15.119:0/3788975531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae7406d7a0 0x7fae7413c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.072+0000 7fae79000640 1 -- 172.21.15.119:0/3788975531 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae74138d60 con 0x7fae740cef10 2024-04-25T10:15:32.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.072+0000 7fae79000640 1 --2- 172.21.15.119:0/3788975531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae740cef10 0x7fae7413dd30 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fae60000c00 tx=0x7fae600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:32.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.073+0000 7fae71600640 1 -- 172.21.15.119:0/3788975531 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60041020 con 0x7fae740cef10 2024-04-25T10:15:32.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.073+0000 7fae71600640 1 -- 172.21.15.119:0/3788975531 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae600306c0 con 0x7fae740cef10 2024-04-25T10:15:32.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.073+0000 7fae7c400640 1 -- 172.21.15.119:0/3788975531 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7413bb10 con 0x7fae740cef10 2024-04-25T10:15:32.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.073+0000 7fae71600640 1 -- 172.21.15.119:0/3788975531 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60037040 con 0x7fae740cef10 2024-04-25T10:15:32.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.073+0000 7fae7c400640 1 -- 172.21.15.119:0/3788975531 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7413c0d0 con 0x7fae740cef10 2024-04-25T10:15:32.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.074+0000 7fae7c400640 1 -- 172.21.15.119:0/3788975531 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae740750b0 con 0x7fae740cef10 2024-04-25T10:15:32.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.075+0000 7fae71600640 1 -- 172.21.15.119:0/3788975531 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae60044020 con 0x7fae740cef10 2024-04-25T10:15:32.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.076+0000 7fae71600640 1 --2- 172.21.15.119:0/3788975531 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fae5406b4d0 0x7fae5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.076+0000 7fae71600640 1 -- 172.21.15.119:0/3788975531 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fae600aaba0 con 0x7fae740cef10 2024-04-25T10:15:32.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.076+0000 7fae79a00640 1 --2- 172.21.15.119:0/3788975531 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fae5406b4d0 0x7fae5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.077+0000 7fae79a00640 1 --2- 172.21.15.119:0/3788975531 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fae5406b4d0 0x7fae5406d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fae68006fd0 tx=0x7fae68008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:32.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.081+0000 7fae71600640 1 -- 172.21.15.119:0/3788975531 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae6004b030 con 0x7fae740cef10 2024-04-25T10:15:32.242 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:32.250 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:32.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.279+0000 7fae7c400640 1 -- 172.21.15.119:0/3788975531 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae7406c220 con 0x7fae740cef10 2024-04-25T10:15:32.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.280+0000 7fae71600640 1 -- 172.21.15.119:0/3788975531 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fae600798b0 con 0x7fae740cef10 2024-04-25T10:15:32.284 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:15:32.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.282+0000 7fae7c400640 1 -- 172.21.15.119:0/3788975531 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fae5406b4d0 msgr2=0x7fae5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.282+0000 7fae7c400640 1 --2- 172.21.15.119:0/3788975531 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fae5406b4d0 0x7fae5406d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fae68006fd0 tx=0x7fae68008040 comp rx=0 tx=0).stop 2024-04-25T10:15:32.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.282+0000 7fae7c400640 1 -- 172.21.15.119:0/3788975531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae740cef10 msgr2=0x7fae7413dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.282+0000 7fae7c400640 1 --2- 172.21.15.119:0/3788975531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae740cef10 0x7fae7413dd30 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fae60000c00 tx=0x7fae600046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.282+0000 7fae7c400640 1 -- 172.21.15.119:0/3788975531 shutdown_connections 2024-04-25T10:15:32.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.282+0000 7fae7c400640 1 --2- 172.21.15.119:0/3788975531 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fae5406b4d0 0x7fae5406d990 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.282+0000 7fae7c400640 1 --2- 172.21.15.119:0/3788975531 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae74144e70 0x7fae74152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.282+0000 7fae7c400640 1 --2- 172.21.15.119:0/3788975531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae740cef10 0x7fae7413dd30 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.282+0000 7fae7c400640 1 --2- 172.21.15.119:0/3788975531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae7406d7a0 0x7fae7413c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.282+0000 7fae7c400640 1 -- 172.21.15.119:0/3788975531 >> 172.21.15.119:0/3788975531 conn(0x7fae7406aba0 msgr2=0x7fae7406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:32.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.282+0000 7fae7c400640 1 -- 172.21.15.119:0/3788975531 shutdown_connections 2024-04-25T10:15:32.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.283+0000 7fae7c400640 1 -- 172.21.15.119:0/3788975531 wait complete. 2024-04-25T10:15:32.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:15:32.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.532+0000 7fa6c3200640 1 Processor -- start 2024-04-25T10:15:32.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.532+0000 7fa6c3200640 1 -- start start 2024-04-25T10:15:32.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.534+0000 7fa6c3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c40cec60 0x7fa6c40cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.534+0000 7fa6c3200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c406d080 con 0x7fa6c406f040 2024-04-25T10:15:32.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.534+0000 7fa6c3200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c406d360 con 0x7fa6c406aba0 2024-04-25T10:15:32.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.534+0000 7fa6c3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c406d640 con 0x7fa6c40cec60 2024-04-25T10:15:32.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.534+0000 7fa6c1400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa6c406f040 0x7fa6c40cd2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50014/0 (socket says 172.21.15.119:50014) 2024-04-25T10:15:32.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.534+0000 7fa6c1400640 1 -- 172.21.15.119:0/3028044897 learned_addr learned my addr 172.21.15.119:0/3028044897 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:32.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.534+0000 7fa6c0a00640 1 -- 172.21.15.119:0/3028044897 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349120328 0 0) 0x7fa6c406d080 con 0x7fa6c406f040 2024-04-25T10:15:32.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c0a00640 1 -- 172.21.15.119:0/3028044897 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6a8003520 con 0x7fa6c406f040 2024-04-25T10:15:32.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c2800640 1 --2- 172.21.15.119:0/3028044897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c40cec60 0x7fa6c40cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c0a00640 1 -- 172.21.15.119:0/3028044897 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2627852346 0 0) 0x7fa6a8003520 con 0x7fa6c406f040 2024-04-25T10:15:32.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c0a00640 1 -- 172.21.15.119:0/3028044897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c40cec60 msgr2=0x7fa6c40cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:32.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c0a00640 1 --2- 172.21.15.119:0/3028044897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c40cec60 0x7fa6c40cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c0a00640 1 -- 172.21.15.119:0/3028044897 >> v1:172.21.15.132:6789/0 conn(0x7fa6c406aba0 legacy=0x7fa6c406af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c0a00640 1 -- 172.21.15.119:0/3028044897 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c406d920 con 0x7fa6c406f040 2024-04-25T10:15:32.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c0a00640 1 -- 172.21.15.119:0/3028044897 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fa6b0001670 con 0x7fa6c406f040 2024-04-25T10:15:32.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c0a00640 1 -- 172.21.15.119:0/3028044897 >> v1:172.21.15.119:6789/0 conn(0x7fa6c406f040 legacy=0x7fa6c40cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c0a00640 1 --2- 172.21.15.119:0/3028044897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a80039e0 0x7fa6a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c0a00640 1 --2- 172.21.15.119:0/3028044897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c40cec60 0x7fa6a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c0a00640 1 --2- 172.21.15.119:0/3028044897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a8004d40 0x7fa6a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c0a00640 1 -- 172.21.15.119:0/3028044897 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c406d640 con 0x7fa6a80039e0 2024-04-25T10:15:32.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c0a00640 1 -- 172.21.15.119:0/3028044897 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a800bbd0 con 0x7fa6a8004d40 2024-04-25T10:15:32.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c0a00640 1 -- 172.21.15.119:0/3028044897 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a800bdd0 con 0x7fa6c40cec60 2024-04-25T10:15:32.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.535+0000 7fa6c1e00640 1 --2- 172.21.15.119:0/3028044897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a80039e0 0x7fa6a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.536+0000 7fa6c1400640 1 --2- 172.21.15.119:0/3028044897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c40cec60 0x7fa6a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.536+0000 7fa6c1e00640 1 -- 172.21.15.119:0/3028044897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c40cec60 msgr2=0x7fa6a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:32.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.536+0000 7fa6c1e00640 1 --2- 172.21.15.119:0/3028044897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c40cec60 0x7fa6a8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.536+0000 7fa6c1e00640 1 -- 172.21.15.119:0/3028044897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a8004d40 msgr2=0x7fa6a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:32.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.536+0000 7fa6c1e00640 1 --2- 172.21.15.119:0/3028044897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a8004d40 0x7fa6a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.536+0000 7fa6c1e00640 1 -- 172.21.15.119:0/3028044897 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a8003520 con 0x7fa6a80039e0 2024-04-25T10:15:32.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.537+0000 7fa6c1e00640 1 --2- 172.21.15.119:0/3028044897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a80039e0 0x7fa6a8003e00 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4001770 tx=0x7fa6a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd36908c3ddbfe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:32.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.537+0000 7fa6c0a00640 1 -- 172.21.15.119:0/3028044897 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4001390 con 0x7fa6a80039e0 2024-04-25T10:15:32.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.537+0000 7fa6c0a00640 1 -- 172.21.15.119:0/3028044897 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b40052d0 con 0x7fa6a80039e0 2024-04-25T10:15:32.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.537+0000 7fa6c0a00640 1 -- 172.21.15.119:0/3028044897 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b402ab40 con 0x7fa6a80039e0 2024-04-25T10:15:32.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.537+0000 7fa6c3200640 1 -- 172.21.15.119:0/3028044897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a80039e0 msgr2=0x7fa6a8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.537+0000 7fa6c3200640 1 --2- 172.21.15.119:0/3028044897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a80039e0 0x7fa6a8003e00 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4001770 tx=0x7fa6a8005830 comp rx=0 tx=0).stop 2024-04-25T10:15:32.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.538+0000 7fa6c2800640 1 -- 172.21.15.119:0/3028044897 reap_dead start 2024-04-25T10:15:32.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.538+0000 7fa6c3200640 1 -- 172.21.15.119:0/3028044897 shutdown_connections 2024-04-25T10:15:32.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.538+0000 7fa6c3200640 1 -- 172.21.15.119:0/3028044897 >> 172.21.15.119:0/3028044897 conn(0x7fa6c4066ff0 msgr2=0x7fa6c40b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:32.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.538+0000 7fa6c3200640 1 -- 172.21.15.119:0/3028044897 shutdown_connections 2024-04-25T10:15:32.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.538+0000 7fa6c3200640 1 -- 172.21.15.119:0/3028044897 wait complete. 2024-04-25T10:15:32.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.539+0000 7fa6c3200640 1 Processor -- start 2024-04-25T10:15:32.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.539+0000 7fa6c3200640 1 -- start start 2024-04-25T10:15:32.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.540+0000 7fa6c3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a80039e0 0x7fa6c4152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.540+0000 7fa6c3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6c414b1b0 0x7fa6c41533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.540+0000 7fa6c3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c415bbb0 0x7fa6c415c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.540+0000 7fa6c3200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40aed40 con 0x7fa6c415bbb0 2024-04-25T10:15:32.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.540+0000 7fa6c3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c415a980 con 0x7fa6c414b1b0 2024-04-25T10:15:32.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.540+0000 7fa6c3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40aeec0 con 0x7fa6a80039e0 2024-04-25T10:15:32.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.541+0000 7fa6c2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c415bbb0 0x7fa6c415c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.541+0000 7fa6c2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c415bbb0 0x7fa6c415c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43554/0 (socket says 172.21.15.119:43554) 2024-04-25T10:15:32.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.541+0000 7fa6c2800640 1 -- 172.21.15.119:0/1470234685 learned_addr learned my addr 172.21.15.119:0/1470234685 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:32.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.541+0000 7fa6c2800640 1 -- 172.21.15.119:0/1470234685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a80039e0 msgr2=0x7fa6c4152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.541+0000 7fa6c1400640 1 --2- 172.21.15.119:0/1470234685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6c414b1b0 0x7fa6c41533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.541+0000 7fa6c1e00640 1 --2- 172.21.15.119:0/1470234685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a80039e0 0x7fa6c4152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.541+0000 7fa6c2800640 1 --2- 172.21.15.119:0/1470234685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a80039e0 0x7fa6c4152eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.541+0000 7fa6c2800640 1 -- 172.21.15.119:0/1470234685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6c414b1b0 msgr2=0x7fa6c41533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.541+0000 7fa6c2800640 1 --2- 172.21.15.119:0/1470234685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6c414b1b0 0x7fa6c41533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.541+0000 7fa6c2800640 1 -- 172.21.15.119:0/1470234685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c41f7af0 con 0x7fa6c415bbb0 2024-04-25T10:15:32.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.541+0000 7fa6c2800640 1 --2- 172.21.15.119:0/1470234685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c415bbb0 0x7fa6c415c040 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40cbb30 tx=0x7fa6bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:32.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.541+0000 7fa6ba000640 1 -- 172.21.15.119:0/1470234685 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00ebc0 con 0x7fa6c415bbb0 2024-04-25T10:15:32.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.542+0000 7fa6ba000640 1 -- 172.21.15.119:0/1470234685 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc00ed80 con 0x7fa6c415bbb0 2024-04-25T10:15:32.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.542+0000 7fa6ba000640 1 -- 172.21.15.119:0/1470234685 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc003bd0 con 0x7fa6c415bbb0 2024-04-25T10:15:32.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.542+0000 7fa6c3200640 1 -- 172.21.15.119:0/1470234685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c41f7d90 con 0x7fa6c415bbb0 2024-04-25T10:15:32.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.542+0000 7fa6c3200640 1 -- 172.21.15.119:0/1470234685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c41f8350 con 0x7fa6c415bbb0 2024-04-25T10:15:32.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.543+0000 7fa6c3200640 1 -- 172.21.15.119:0/1470234685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c40749b0 con 0x7fa6c415bbb0 2024-04-25T10:15:32.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.544+0000 7fa6ba000640 1 -- 172.21.15.119:0/1470234685 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6bc01b020 con 0x7fa6c415bbb0 2024-04-25T10:15:32.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.544+0000 7fa6ba000640 1 --2- 172.21.15.119:0/1470234685 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa6a406b440 0x7fa6a406d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.545+0000 7fa6ba000640 1 -- 172.21.15.119:0/1470234685 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fa6bc089ff0 con 0x7fa6c415bbb0 2024-04-25T10:15:32.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.545+0000 7fa6c1e00640 1 --2- 172.21.15.119:0/1470234685 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa6a406b440 0x7fa6a406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.545+0000 7fa6c1e00640 1 --2- 172.21.15.119:0/1470234685 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa6a406b440 0x7fa6a406d900 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4032000 tx=0x7fa6b402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:32.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.549+0000 7fa6ba000640 1 -- 172.21.15.119:0/1470234685 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6bc058dd0 con 0x7fa6c415bbb0 2024-04-25T10:15:32.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.743+0000 7fa6c3200640 1 -- 172.21.15.119:0/1470234685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6c40ce090 con 0x7fa6c415bbb0 2024-04-25T10:15:32.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.744+0000 7fa6ba000640 1 -- 172.21.15.119:0/1470234685 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa6bc0586d0 con 0x7fa6c415bbb0 2024-04-25T10:15:32.748 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:15:32.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.747+0000 7fa6c3200640 1 -- 172.21.15.119:0/1470234685 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa6a406b440 msgr2=0x7fa6a406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.747+0000 7fa6c3200640 1 --2- 172.21.15.119:0/1470234685 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa6a406b440 0x7fa6a406d900 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4032000 tx=0x7fa6b402e000 comp rx=0 tx=0).stop 2024-04-25T10:15:32.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.747+0000 7fa6c3200640 1 -- 172.21.15.119:0/1470234685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c415bbb0 msgr2=0x7fa6c415c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.747+0000 7fa6c3200640 1 --2- 172.21.15.119:0/1470234685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c415bbb0 0x7fa6c415c040 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40cbb30 tx=0x7fa6bc00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.747+0000 7fa6c3200640 1 -- 172.21.15.119:0/1470234685 shutdown_connections 2024-04-25T10:15:32.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.747+0000 7fa6c3200640 1 --2- 172.21.15.119:0/1470234685 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa6a406b440 0x7fa6a406d900 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.747+0000 7fa6c3200640 1 --2- 172.21.15.119:0/1470234685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c415bbb0 0x7fa6c415c040 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.747+0000 7fa6c3200640 1 --2- 172.21.15.119:0/1470234685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6c414b1b0 0x7fa6c41533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.747+0000 7fa6c3200640 1 --2- 172.21.15.119:0/1470234685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a80039e0 0x7fa6c4152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.747+0000 7fa6c3200640 1 -- 172.21.15.119:0/1470234685 >> 172.21.15.119:0/1470234685 conn(0x7fa6c4066ff0 msgr2=0x7fa6c40b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:32.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.747+0000 7fa6c3200640 1 -- 172.21.15.119:0/1470234685 shutdown_connections 2024-04-25T10:15:32.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.747+0000 7fa6c3200640 1 -- 172.21.15.119:0/1470234685 wait complete. 2024-04-25T10:15:32.770 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:15:32.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.958+0000 7fb77cc00640 1 Processor -- start 2024-04-25T10:15:32.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.958+0000 7fb77cc00640 1 -- start start 2024-04-25T10:15:32.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.959+0000 7fb77cc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb778144e70 0x7fb77806d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.959+0000 7fb77cc00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb77806d7a0 0x7fb7780ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.959+0000 7fb77cc00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77806dcb0 con 0x7fb7780cef10 2024-04-25T10:15:32.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.959+0000 7fb77cc00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb77815bfe0 con 0x7fb77806d7a0 2024-04-25T10:15:32.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.959+0000 7fb77cc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77806df90 con 0x7fb778144e70 2024-04-25T10:15:32.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.960+0000 7fb775400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb77806d7a0 0x7fb7780ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.960+0000 7fb775400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb77806d7a0 0x7fb7780ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43570/0 (socket says 172.21.15.119:43570) 2024-04-25T10:15:32.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.960+0000 7fb775400640 1 -- 172.21.15.119:0/2459244950 learned_addr learned my addr 172.21.15.119:0/2459244950 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:32.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.960+0000 7fb776800640 1 --1- 172.21.15.119:0/2459244950 >> v1:172.21.15.119:6789/0 conn(0x7fb7780cef10 0x7fb778071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50028/0 (socket says 172.21.15.119:50028) 2024-04-25T10:15:32.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.960+0000 7fb775e00640 1 --2- 172.21.15.119:0/2459244950 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb778144e70 0x7fb77806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.960+0000 7fb775400640 1 -- 172.21.15.119:0/2459244950 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb778144e70 msgr2=0x7fb77806d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:32.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.960+0000 7fb775400640 1 --2- 172.21.15.119:0/2459244950 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb778144e70 0x7fb77806d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.960+0000 7fb775400640 1 -- 172.21.15.119:0/2459244950 >> v1:172.21.15.119:6789/0 conn(0x7fb7780cef10 legacy=0x7fb778071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.960+0000 7fb775400640 1 -- 172.21.15.119:0/2459244950 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77806e270 con 0x7fb77806d7a0 2024-04-25T10:15:32.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.960+0000 7fb774a00640 1 -- 172.21.15.119:0/2459244950 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052344517 0 0) 0x7fb77806dcb0 con 0x7fb7780cef10 2024-04-25T10:15:32.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.961+0000 7fb775400640 1 --2- 172.21.15.119:0/2459244950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb77806d7a0 0x7fb7780ce9d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fb760009e50 tx=0x7fb76002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbf0a3dcf790eea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:32.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.962+0000 7fb774a00640 1 -- 172.21.15.119:0/2459244950 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760036070 con 0x7fb77806d7a0 2024-04-25T10:15:32.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.962+0000 7fb774a00640 1 -- 172.21.15.119:0/2459244950 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb760031600 con 0x7fb77806d7a0 2024-04-25T10:15:32.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.962+0000 7fb774a00640 1 -- 172.21.15.119:0/2459244950 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7600319d0 con 0x7fb77806d7a0 2024-04-25T10:15:32.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.962+0000 7fb77cc00640 1 -- 172.21.15.119:0/2459244950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb77806d7a0 msgr2=0x7fb7780ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.962+0000 7fb77cc00640 1 --2- 172.21.15.119:0/2459244950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb77806d7a0 0x7fb7780ce9d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fb760009e50 tx=0x7fb76002a680 comp rx=0 tx=0).stop 2024-04-25T10:15:32.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.962+0000 7fb77cc00640 1 -- 172.21.15.119:0/2459244950 shutdown_connections 2024-04-25T10:15:32.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.962+0000 7fb77cc00640 1 --2- 172.21.15.119:0/2459244950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb77806d7a0 0x7fb7780ce9d0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.962+0000 7fb77cc00640 1 --2- 172.21.15.119:0/2459244950 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb778144e70 0x7fb77806d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.962+0000 7fb77cc00640 1 -- 172.21.15.119:0/2459244950 >> 172.21.15.119:0/2459244950 conn(0x7fb77806aba0 msgr2=0x7fb77806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:32.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.963+0000 7fb77cc00640 1 -- 172.21.15.119:0/2459244950 shutdown_connections 2024-04-25T10:15:32.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.963+0000 7fb77cc00640 1 -- 172.21.15.119:0/2459244950 wait complete. 2024-04-25T10:15:32.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.964+0000 7fb77cc00640 1 Processor -- start 2024-04-25T10:15:32.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.964+0000 7fb77cc00640 1 -- start start 2024-04-25T10:15:32.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.965+0000 7fb77cc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb77806d7a0 0x7fb7781426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.965+0000 7fb77cc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7780cef10 0x7fb7780b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.965+0000 7fb77cc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb778144e70 0x7fb7780b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.965+0000 7fb77cc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7781454b0 con 0x7fb77806d7a0 2024-04-25T10:15:32.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.965+0000 7fb77cc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780cf380 con 0x7fb7780cef10 2024-04-25T10:15:32.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.965+0000 7fb77cc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb778145630 con 0x7fb778144e70 2024-04-25T10:15:32.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.965+0000 7fb775e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb77806d7a0 0x7fb7781426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.966+0000 7fb775e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb77806d7a0 0x7fb7781426e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43572/0 (socket says 172.21.15.119:43572) 2024-04-25T10:15:32.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.966+0000 7fb775e00640 1 -- 172.21.15.119:0/1147706347 learned_addr learned my addr 172.21.15.119:0/1147706347 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:32.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.966+0000 7fb775e00640 1 -- 172.21.15.119:0/1147706347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb778144e70 msgr2=0x7fb7780b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.966+0000 7fb775e00640 1 --2- 172.21.15.119:0/1147706347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb778144e70 0x7fb7780b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.966+0000 7fb775e00640 1 -- 172.21.15.119:0/1147706347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7780cef10 msgr2=0x7fb7780b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:32.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.966+0000 7fb775e00640 1 --2- 172.21.15.119:0/1147706347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7780cef10 0x7fb7780b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:32.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.966+0000 7fb775e00640 1 -- 172.21.15.119:0/1147706347 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7780b0650 con 0x7fb77806d7a0 2024-04-25T10:15:32.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.966+0000 7fb775e00640 1 --2- 172.21.15.119:0/1147706347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb77806d7a0 0x7fb7781426e0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb76800d5f0 tx=0x7fb76800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:32.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.966+0000 7fb76e000640 1 -- 172.21.15.119:0/1147706347 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7680125f0 con 0x7fb77806d7a0 2024-04-25T10:15:32.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.966+0000 7fb76e000640 1 -- 172.21.15.119:0/1147706347 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb768019070 con 0x7fb77806d7a0 2024-04-25T10:15:32.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.966+0000 7fb76e000640 1 -- 172.21.15.119:0/1147706347 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7680129b0 con 0x7fb77806d7a0 2024-04-25T10:15:32.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.967+0000 7fb77cc00640 1 -- 172.21.15.119:0/1147706347 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb778140a80 con 0x7fb77806d7a0 2024-04-25T10:15:32.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.967+0000 7fb77cc00640 1 -- 172.21.15.119:0/1147706347 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb778141040 con 0x7fb77806d7a0 2024-04-25T10:15:32.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.968+0000 7fb77cc00640 1 -- 172.21.15.119:0/1147706347 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7780750b0 con 0x7fb77806d7a0 2024-04-25T10:15:32.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.969+0000 7fb76e000640 1 -- 172.21.15.119:0/1147706347 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb768012b50 con 0x7fb77806d7a0 2024-04-25T10:15:32.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.970+0000 7fb76e000640 1 --2- 172.21.15.119:0/1147706347 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb75406b4d0 0x7fb75406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:32.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.970+0000 7fb76e000640 1 -- 172.21.15.119:0/1147706347 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fb76808a790 con 0x7fb77806d7a0 2024-04-25T10:15:32.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.970+0000 7fb775400640 1 --2- 172.21.15.119:0/1147706347 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb75406b4d0 0x7fb75406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:32.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.971+0000 7fb775400640 1 --2- 172.21.15.119:0/1147706347 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb75406b4d0 0x7fb75406d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb760033000 tx=0x7fb76002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:32.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:32.975+0000 7fb76e000640 1 -- 172.21.15.119:0/1147706347 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb7680594a0 con 0x7fb77806d7a0 2024-04-25T10:15:33.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.156+0000 7fb77cc00640 1 -- 172.21.15.119:0/1147706347 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb7780aebe0 con 0x7fb77806d7a0 2024-04-25T10:15:33.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.157+0000 7fb76e000640 1 -- 172.21.15.119:0/1147706347 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb768058da0 con 0x7fb77806d7a0 2024-04-25T10:15:33.161 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:15:33.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.159+0000 7fb77cc00640 1 -- 172.21.15.119:0/1147706347 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb75406b4d0 msgr2=0x7fb75406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.159+0000 7fb77cc00640 1 --2- 172.21.15.119:0/1147706347 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb75406b4d0 0x7fb75406d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb760033000 tx=0x7fb76002e000 comp rx=0 tx=0).stop 2024-04-25T10:15:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.159+0000 7fb77cc00640 1 -- 172.21.15.119:0/1147706347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb77806d7a0 msgr2=0x7fb7781426e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.159+0000 7fb77cc00640 1 --2- 172.21.15.119:0/1147706347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb77806d7a0 0x7fb7781426e0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb76800d5f0 tx=0x7fb76800dab0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.159+0000 7fb77cc00640 1 -- 172.21.15.119:0/1147706347 shutdown_connections 2024-04-25T10:15:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.159+0000 7fb77cc00640 1 --2- 172.21.15.119:0/1147706347 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb75406b4d0 0x7fb75406d990 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.159+0000 7fb77cc00640 1 --2- 172.21.15.119:0/1147706347 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb778144e70 0x7fb7780b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.159+0000 7fb77cc00640 1 --2- 172.21.15.119:0/1147706347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7780cef10 0x7fb7780b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.159+0000 7fb77cc00640 1 --2- 172.21.15.119:0/1147706347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb77806d7a0 0x7fb7781426e0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.159+0000 7fb77cc00640 1 -- 172.21.15.119:0/1147706347 >> 172.21.15.119:0/1147706347 conn(0x7fb77806aba0 msgr2=0x7fb7780ccac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.160+0000 7fb77cc00640 1 -- 172.21.15.119:0/1147706347 shutdown_connections 2024-04-25T10:15:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.160+0000 7fb77cc00640 1 -- 172.21.15.119:0/1147706347 wait complete. 2024-04-25T10:15:33.181 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:15:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.376+0000 7f6b32200640 1 Processor -- start 2024-04-25T10:15:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.377+0000 7f6b32200640 1 -- start start 2024-04-25T10:15:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.378+0000 7f6b32200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2c071fc0 0x7f6b2c06d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.378+0000 7f6b32200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6b2c0b3950 0x7f6b2c162e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.378+0000 7f6b32200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c06e3f0 con 0x7f6b2c06dee0 2024-04-25T10:15:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.378+0000 7f6b32200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c06e6d0 con 0x7f6b2c0b3950 2024-04-25T10:15:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.378+0000 7f6b32200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c06e9b0 con 0x7f6b2c071fc0 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.379+0000 7f6b22a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6b2c06dee0 0x7f6b2c0b3240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50042/0 (socket says 172.21.15.119:50042) 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.379+0000 7f6b22a00640 1 -- 172.21.15.119:0/1973992331 learned_addr learned my addr 172.21.15.119:0/1973992331 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.379+0000 7f6b2b400640 1 --2- 172.21.15.119:0/1973992331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2c071fc0 0x7f6b2c06d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.379+0000 7f6b2be00640 1 --2- 172.21.15.119:0/1973992331 >> v2:172.21.15.132:3300/0 conn(0x7f6b2c0b3950 0x7f6b2c162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.379+0000 7f6b2aa00640 1 -- 172.21.15.119:0/1973992331 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707847583 0 0) 0x7f6b2c06e3f0 con 0x7f6b2c06dee0 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.379+0000 7f6b2aa00640 1 -- 172.21.15.119:0/1973992331 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b10003520 con 0x7f6b2c06dee0 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2aa00640 1 -- 172.21.15.119:0/1973992331 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1324770153 0 0) 0x7f6b10003520 con 0x7f6b2c06dee0 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2aa00640 1 -- 172.21.15.119:0/1973992331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2c071fc0 msgr2=0x7f6b2c06d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2aa00640 1 --2- 172.21.15.119:0/1973992331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2c071fc0 0x7f6b2c06d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2aa00640 1 -- 172.21.15.119:0/1973992331 >> v2:172.21.15.132:3300/0 conn(0x7f6b2c0b3950 msgr2=0x7f6b2c162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2aa00640 1 --2- 172.21.15.119:0/1973992331 >> v2:172.21.15.132:3300/0 conn(0x7f6b2c0b3950 0x7f6b2c162e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2aa00640 1 -- 172.21.15.119:0/1973992331 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c162910 con 0x7f6b2c06dee0 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2aa00640 1 -- 172.21.15.119:0/1973992331 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f6b14001670 con 0x7f6b2c06dee0 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2aa00640 1 -- 172.21.15.119:0/1973992331 >> v1:172.21.15.119:6789/0 conn(0x7f6b2c06dee0 legacy=0x7f6b2c0b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2aa00640 1 --2- 172.21.15.119:0/1973992331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2c071fc0 0x7f6b100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2aa00640 1 --2- 172.21.15.119:0/1973992331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b10003f30 0x7f6b10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2aa00640 1 --2- 172.21.15.119:0/1973992331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b10004d40 0x7f6b100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2aa00640 1 -- 172.21.15.119:0/1973992331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c06e6d0 con 0x7f6b10004d40 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2aa00640 1 -- 172.21.15.119:0/1973992331 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c06e9b0 con 0x7f6b10003f30 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2aa00640 1 -- 172.21.15.119:0/1973992331 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1000b9c0 con 0x7f6b2c071fc0 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.380+0000 7f6b2be00640 1 --2- 172.21.15.119:0/1973992331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b10004d40 0x7f6b100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.381+0000 7f6b2b400640 1 --2- 172.21.15.119:0/1973992331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2c071fc0 0x7f6b100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.381+0000 7f6b22a00640 1 --2- 172.21.15.119:0/1973992331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b10003f30 0x7f6b10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.381+0000 7f6b2be00640 1 -- 172.21.15.119:0/1973992331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2c071fc0 msgr2=0x7f6b100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.381+0000 7f6b2be00640 1 --2- 172.21.15.119:0/1973992331 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b2c071fc0 0x7f6b100039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.381+0000 7f6b2be00640 1 -- 172.21.15.119:0/1973992331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b10003f30 msgr2=0x7f6b10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.381+0000 7f6b2be00640 1 --2- 172.21.15.119:0/1973992331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b10003f30 0x7f6b10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.381+0000 7f6b2be00640 1 -- 172.21.15.119:0/1973992331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b10003520 con 0x7f6b10004d40 2024-04-25T10:15:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.381+0000 7f6b22a00640 1 --2- 172.21.15.119:0/1973992331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b10003f30 0x7f6b10004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.382+0000 7f6b2be00640 1 --2- 172.21.15.119:0/1973992331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b10004d40 0x7f6b100051c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0b2cf0 tx=0x7f6b10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b0a36d68b4144e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.382+0000 7f6b2aa00640 1 -- 172.21.15.119:0/1973992331 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24009c10 con 0x7f6b10004d40 2024-04-25T10:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.382+0000 7f6b2aa00640 1 -- 172.21.15.119:0/1973992331 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b24036070 con 0x7f6b10004d40 2024-04-25T10:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.382+0000 7f6b2aa00640 1 -- 172.21.15.119:0/1973992331 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24031670 con 0x7f6b10004d40 2024-04-25T10:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.383+0000 7f6b32200640 1 -- 172.21.15.119:0/1973992331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b10004d40 msgr2=0x7f6b100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.383+0000 7f6b32200640 1 --2- 172.21.15.119:0/1973992331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b10004d40 0x7f6b100051c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0b2cf0 tx=0x7f6b10005830 comp rx=0 tx=0).stop 2024-04-25T10:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.383+0000 7f6b2be00640 1 -- 172.21.15.119:0/1973992331 reap_dead start 2024-04-25T10:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.383+0000 7f6b32200640 1 -- 172.21.15.119:0/1973992331 shutdown_connections 2024-04-25T10:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.383+0000 7f6b32200640 1 -- 172.21.15.119:0/1973992331 >> 172.21.15.119:0/1973992331 conn(0x7f6b2c06aba0 msgr2=0x7f6b2c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.383+0000 7f6b32200640 1 -- 172.21.15.119:0/1973992331 shutdown_connections 2024-04-25T10:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.383+0000 7f6b32200640 1 -- 172.21.15.119:0/1973992331 wait complete. 2024-04-25T10:15:33.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.385+0000 7f6b32200640 1 Processor -- start 2024-04-25T10:15:33.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.385+0000 7f6b32200640 1 -- start start 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.386+0000 7f6b32200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b10004d40 0x7f6b2c14c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.386+0000 7f6b32200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b2c15d170 0x7f6b2c0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.386+0000 7f6b32200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b2c0b0b40 0x7f6b2c15b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.386+0000 7f6b32200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0713c0 con 0x7f6b2c0b0b40 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.386+0000 7f6b32200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c162590 con 0x7f6b2c15d170 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.386+0000 7f6b32200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c071540 con 0x7f6b10004d40 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.386+0000 7f6b2be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b2c0b0b40 0x7f6b2c15b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.386+0000 7f6b2be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b2c0b0b40 0x7f6b2c15b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43584/0 (socket says 172.21.15.119:43584) 2024-04-25T10:15:33.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.386+0000 7f6b2be00640 1 -- 172.21.15.119:0/2118583732 learned_addr learned my addr 172.21.15.119:0/2118583732 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.387+0000 7f6b2b400640 1 --2- 172.21.15.119:0/2118583732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b10004d40 0x7f6b2c14c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.387+0000 7f6b2be00640 1 -- 172.21.15.119:0/2118583732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b10004d40 msgr2=0x7f6b2c14c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.387+0000 7f6b2be00640 1 --2- 172.21.15.119:0/2118583732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b10004d40 0x7f6b2c14c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.387+0000 7f6b2be00640 1 -- 172.21.15.119:0/2118583732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b2c15d170 msgr2=0x7f6b2c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.387+0000 7f6b2be00640 1 --2- 172.21.15.119:0/2118583732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b2c15d170 0x7f6b2c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.387+0000 7f6b2be00640 1 -- 172.21.15.119:0/2118583732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c14cd70 con 0x7f6b2c0b0b40 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.387+0000 7f6b2be00640 1 --2- 172.21.15.119:0/2118583732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b2c0b0b40 0x7f6b2c15b980 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400e230 tx=0x7f6b24009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.387+0000 7f6b23e00640 1 -- 172.21.15.119:0/2118583732 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24009c10 con 0x7f6b2c0b0b40 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.387+0000 7f6b23e00640 1 -- 172.21.15.119:0/2118583732 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b24031db0 con 0x7f6b2c0b0b40 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.387+0000 7f6b32200640 1 -- 172.21.15.119:0/2118583732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c157720 con 0x7f6b2c0b0b40 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.387+0000 7f6b23e00640 1 -- 172.21.15.119:0/2118583732 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24003bb0 con 0x7f6b2c0b0b40 2024-04-25T10:15:33.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.387+0000 7f6b32200640 1 -- 172.21.15.119:0/2118583732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c157ce0 con 0x7f6b2c0b0b40 2024-04-25T10:15:33.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.389+0000 7f6b32200640 1 -- 172.21.15.119:0/2118583732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c0b0fc0 con 0x7f6b2c0b0b40 2024-04-25T10:15:33.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.389+0000 7f6b23e00640 1 -- 172.21.15.119:0/2118583732 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b24005d10 con 0x7f6b2c0b0b40 2024-04-25T10:15:33.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.390+0000 7f6b23e00640 1 --2- 172.21.15.119:0/2118583732 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b0806b4d0 0x7f6b0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:33.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.390+0000 7f6b23e00640 1 -- 172.21.15.119:0/2118583732 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f6b240ab090 con 0x7f6b2c0b0b40 2024-04-25T10:15:33.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.391+0000 7f6b2b400640 1 --2- 172.21.15.119:0/2118583732 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b0806b4d0 0x7f6b0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:33.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.392+0000 7f6b2b400640 1 --2- 172.21.15.119:0/2118583732 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b0806b4d0 0x7f6b0806d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009950 tx=0x7f6b1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:33.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.395+0000 7f6b23e00640 1 -- 172.21.15.119:0/2118583732 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b24049060 con 0x7f6b2c0b0b40 2024-04-25T10:15:33.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.601+0000 7f6b32200640 1 -- 172.21.15.119:0/2118583732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6b2c0cd180 con 0x7f6b2c0b0b40 2024-04-25T10:15:33.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.602+0000 7f6b23e00640 1 -- 172.21.15.119:0/2118583732 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v85) v1 ==== 80+0+248 (secure 0 0 0) 0x7f6b2c0cd180 con 0x7f6b2c0b0b40 2024-04-25T10:15:33.606 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6837/1734507541 2024-04-26T10:14:50.735180+0000 2024-04-25T10:15:33.606 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6835/1734507541 2024-04-26T10:14:50.735180+0000 2024-04-25T10:15:33.606 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6837/3997964942 2024-04-26T10:13:36.160373+0000 2024-04-25T10:15:33.606 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6835/3997964942 2024-04-26T10:13:36.160373+0000 2024-04-25T10:15:33.606 INFO:teuthology.orchestra.run.smithi119.stderr:listed 4 entries 2024-04-25T10:15:33.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.605+0000 7f6b32200640 1 -- 172.21.15.119:0/2118583732 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b0806b4d0 msgr2=0x7f6b0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.605+0000 7f6b32200640 1 --2- 172.21.15.119:0/2118583732 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b0806b4d0 0x7f6b0806d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009950 tx=0x7f6b1c008040 comp rx=0 tx=0).stop 2024-04-25T10:15:33.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.605+0000 7f6b32200640 1 -- 172.21.15.119:0/2118583732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b2c0b0b40 msgr2=0x7f6b2c15b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:33.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.605+0000 7f6b32200640 1 --2- 172.21.15.119:0/2118583732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b2c0b0b40 0x7f6b2c15b980 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400e230 tx=0x7f6b24009750 comp rx=0 tx=0).stop 2024-04-25T10:15:33.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.605+0000 7f6b32200640 1 -- 172.21.15.119:0/2118583732 shutdown_connections 2024-04-25T10:15:33.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.605+0000 7f6b32200640 1 --2- 172.21.15.119:0/2118583732 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6b0806b4d0 0x7f6b0806d990 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.605+0000 7f6b32200640 1 --2- 172.21.15.119:0/2118583732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b2c0b0b40 0x7f6b2c15b980 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.605+0000 7f6b32200640 1 --2- 172.21.15.119:0/2118583732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b2c15d170 0x7f6b2c0b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.605+0000 7f6b32200640 1 --2- 172.21.15.119:0/2118583732 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6b10004d40 0x7f6b2c14c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:33.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.605+0000 7f6b32200640 1 -- 172.21.15.119:0/2118583732 >> 172.21.15.119:0/2118583732 conn(0x7f6b2c06aba0 msgr2=0x7f6b2c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:33.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.605+0000 7f6b32200640 1 -- 172.21.15.119:0/2118583732 shutdown_connections 2024-04-25T10:15:33.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:33.605+0000 7f6b32200640 1 -- 172.21.15.119:0/2118583732 wait complete. 2024-04-25T10:15:33.627 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:15:33.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:33.627 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:33.716 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:15:33.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:33.716 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:15:33.764 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:15:33.764 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:33.822 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:33.822 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:15:33.822 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:33.880 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:33.880 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:15:33.880 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:33.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:33.934 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:34.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.145+0000 7f729ee00640 1 Processor -- start 2024-04-25T10:15:34.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.145+0000 7f729ee00640 1 -- start start 2024-04-25T10:15:34.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.147+0000 7f729ee00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f729815d980 0x7f72980ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.147+0000 7f729ee00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f72980aeb20 0x7f72980cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.147+0000 7f729ee00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72980cfbd0 con 0x7f72980cf7c0 2024-04-25T10:15:34.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.147+0000 7f729ee00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7298071ed0 con 0x7f72980aeb20 2024-04-25T10:15:34.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.147+0000 7f729ee00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72980712c0 con 0x7f729815d980 2024-04-25T10:15:34.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.148+0000 7f7297400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f72980aeb20 0x7f72980cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.148+0000 7f729ce00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f72980cf7c0 0x7f729815dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50050/0 (socket says 172.21.15.119:50050) 2024-04-25T10:15:34.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.148+0000 7f729ce00640 1 -- 172.21.15.119:0/662395998 learned_addr learned my addr 172.21.15.119:0/662395998 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:34.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.148+0000 7f7297e00640 1 --2- 172.21.15.119:0/662395998 >> v2:172.21.15.132:3300/0 conn(0x7f729815d980 0x7f72980ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.148+0000 7f7297400640 1 -- 172.21.15.119:0/662395998 >> v2:172.21.15.132:3300/0 conn(0x7f729815d980 msgr2=0x7f72980ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:34.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.148+0000 7f7297400640 1 --2- 172.21.15.119:0/662395998 >> v2:172.21.15.132:3300/0 conn(0x7f729815d980 0x7f72980ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.148+0000 7f7297400640 1 -- 172.21.15.119:0/662395998 >> v1:172.21.15.119:6789/0 conn(0x7f72980cf7c0 legacy=0x7f729815dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.148+0000 7f7297400640 1 -- 172.21.15.119:0/662395998 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72980715a0 con 0x7f72980aeb20 2024-04-25T10:15:34.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.149+0000 7f7297400640 1 --2- 172.21.15.119:0/662395998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980aeb20 0x7f72980cf280 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7298067460 tx=0x7f728402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b996f821e69feb63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:34.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.150+0000 7f7296a00640 1 -- 172.21.15.119:0/662395998 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284037070 con 0x7f72980aeb20 2024-04-25T10:15:34.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.150+0000 7f7296a00640 1 -- 172.21.15.119:0/662395998 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7284032440 con 0x7f72980aeb20 2024-04-25T10:15:34.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.150+0000 7f7296a00640 1 -- 172.21.15.119:0/662395998 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728402ae60 con 0x7f72980aeb20 2024-04-25T10:15:34.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.150+0000 7f729ee00640 1 -- 172.21.15.119:0/662395998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980aeb20 msgr2=0x7f72980cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.150+0000 7f729ee00640 1 --2- 172.21.15.119:0/662395998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980aeb20 0x7f72980cf280 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7298067460 tx=0x7f728402a110 comp rx=0 tx=0).stop 2024-04-25T10:15:34.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.150+0000 7f729ee00640 1 -- 172.21.15.119:0/662395998 shutdown_connections 2024-04-25T10:15:34.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.150+0000 7f729ee00640 1 --2- 172.21.15.119:0/662395998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980aeb20 0x7f72980cf280 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.150+0000 7f729ee00640 1 --2- 172.21.15.119:0/662395998 >> v2:172.21.15.132:3300/0 conn(0x7f729815d980 0x7f72980ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.150+0000 7f729ee00640 1 -- 172.21.15.119:0/662395998 >> 172.21.15.119:0/662395998 conn(0x7f729806aba0 msgr2=0x7f729806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:34.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.150+0000 7f729ee00640 1 -- 172.21.15.119:0/662395998 shutdown_connections 2024-04-25T10:15:34.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.150+0000 7f729ee00640 1 -- 172.21.15.119:0/662395998 wait complete. 2024-04-25T10:15:34.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.152+0000 7f729ee00640 1 Processor -- start 2024-04-25T10:15:34.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.152+0000 7f729ee00640 1 -- start start 2024-04-25T10:15:34.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.153+0000 7f729ee00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72980aeb20 0x7f7298144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.153+0000 7f729ee00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980cf7c0 0x7f7298157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.153+0000 7f729ee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f729815d980 0x7f729815df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.153+0000 7f729ee00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72980adf80 con 0x7f72980cf7c0 2024-04-25T10:15:34.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.153+0000 7f729ee00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7298071b60 con 0x7f72980aeb20 2024-04-25T10:15:34.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.153+0000 7f729ee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72980ae100 con 0x7f729815d980 2024-04-25T10:15:34.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.154+0000 7f7297400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980cf7c0 0x7f7298157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.154+0000 7f7297400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980cf7c0 0x7f7298157810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43596/0 (socket says 172.21.15.119:43596) 2024-04-25T10:15:34.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.154+0000 7f7297400640 1 -- 172.21.15.119:0/3137209271 learned_addr learned my addr 172.21.15.119:0/3137209271 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:34.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.154+0000 7f7297e00640 1 --2- 172.21.15.119:0/3137209271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72980aeb20 0x7f7298144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.154+0000 7f729ce00640 1 --2- 172.21.15.119:0/3137209271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f729815d980 0x7f729815df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.154+0000 7f7297400640 1 -- 172.21.15.119:0/3137209271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f729815d980 msgr2=0x7f729815df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.154+0000 7f7297400640 1 --2- 172.21.15.119:0/3137209271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f729815d980 0x7f729815df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.154+0000 7f7297400640 1 -- 172.21.15.119:0/3137209271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72980aeb20 msgr2=0x7f7298144fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.154+0000 7f7297400640 1 --2- 172.21.15.119:0/3137209271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72980aeb20 0x7f7298144fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.154+0000 7f7297400640 1 -- 172.21.15.119:0/3137209271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7298145510 con 0x7f72980cf7c0 2024-04-25T10:15:34.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.155+0000 7f7297400640 1 --2- 172.21.15.119:0/3137209271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980cf7c0 0x7f7298157810 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f7284009890 tx=0x7f72840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:34.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.155+0000 7f728fe00640 1 -- 172.21.15.119:0/3137209271 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284037040 con 0x7f72980cf7c0 2024-04-25T10:15:34.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.155+0000 7f728fe00640 1 -- 172.21.15.119:0/3137209271 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7284042070 con 0x7f72980cf7c0 2024-04-25T10:15:34.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.155+0000 7f728fe00640 1 -- 172.21.15.119:0/3137209271 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284030bd0 con 0x7f72980cf7c0 2024-04-25T10:15:34.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.155+0000 7f729ee00640 1 -- 172.21.15.119:0/3137209271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72981581b0 con 0x7f72980cf7c0 2024-04-25T10:15:34.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.155+0000 7f729ee00640 1 -- 172.21.15.119:0/3137209271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72980cc860 con 0x7f72980cf7c0 2024-04-25T10:15:34.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.156+0000 7f729ee00640 1 -- 172.21.15.119:0/3137209271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72980749b0 con 0x7f72980cf7c0 2024-04-25T10:15:34.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.157+0000 7f728fe00640 1 -- 172.21.15.119:0/3137209271 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7284047460 con 0x7f72980cf7c0 2024-04-25T10:15:34.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.158+0000 7f728fe00640 1 --2- 172.21.15.119:0/3137209271 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f727806b500 0x7f727806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.159+0000 7f728fe00640 1 -- 172.21.15.119:0/3137209271 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f72840aae90 con 0x7f72980cf7c0 2024-04-25T10:15:34.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.159+0000 7f7297e00640 1 --2- 172.21.15.119:0/3137209271 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f727806b500 0x7f727806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.159+0000 7f7297e00640 1 --2- 172.21.15.119:0/3137209271 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f727806b500 0x7f727806d9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7288006fd0 tx=0x7f7288008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:34.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.163+0000 7f728fe00640 1 -- 172.21.15.119:0/3137209271 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7284032e20 con 0x7f72980cf7c0 2024-04-25T10:15:34.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.395+0000 7f729ee00640 1 -- 172.21.15.119:0/3137209271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f729806e8a0 con 0x7f72980cf7c0 2024-04-25T10:15:34.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.395+0000 7f728fe00640 1 -- 172.21.15.119:0/3137209271 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f72840ac020 con 0x7f72980cf7c0 2024-04-25T10:15:34.400 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:34.400 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:14:54.775138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7668":{"gid":7668,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/2043821095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2043821095},{"type":"v1","addr":"172.21.15.119:6837","nonce":2043821095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:15:34.400 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 74 2024-04-25T10:15:34.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.398+0000 7f729ee00640 1 -- 172.21.15.119:0/3137209271 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f727806b500 msgr2=0x7f727806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.398+0000 7f729ee00640 1 --2- 172.21.15.119:0/3137209271 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f727806b500 0x7f727806d9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7288006fd0 tx=0x7f7288008040 comp rx=0 tx=0).stop 2024-04-25T10:15:34.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.398+0000 7f729ee00640 1 -- 172.21.15.119:0/3137209271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980cf7c0 msgr2=0x7f7298157810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.398+0000 7f729ee00640 1 --2- 172.21.15.119:0/3137209271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980cf7c0 0x7f7298157810 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f7284009890 tx=0x7f72840046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.398+0000 7f729ee00640 1 -- 172.21.15.119:0/3137209271 shutdown_connections 2024-04-25T10:15:34.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.398+0000 7f729ee00640 1 --2- 172.21.15.119:0/3137209271 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f727806b500 0x7f727806d9c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.398+0000 7f729ee00640 1 --2- 172.21.15.119:0/3137209271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f729815d980 0x7f729815df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.398+0000 7f729ee00640 1 --2- 172.21.15.119:0/3137209271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72980cf7c0 0x7f7298157810 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.398+0000 7f729ee00640 1 --2- 172.21.15.119:0/3137209271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72980aeb20 0x7f7298144fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.398+0000 7f729ee00640 1 -- 172.21.15.119:0/3137209271 >> 172.21.15.119:0/3137209271 conn(0x7f729806aba0 msgr2=0x7f72980aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:34.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.399+0000 7f729ee00640 1 -- 172.21.15.119:0/3137209271 shutdown_connections 2024-04-25T10:15:34.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.399+0000 7f729ee00640 1 -- 172.21.15.119:0/3137209271 wait complete. 2024-04-25T10:15:34.422 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:34.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.620+0000 7f7a0fe00640 1 Processor -- start 2024-04-25T10:15:34.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.620+0000 7f7a0fe00640 1 -- start start 2024-04-25T10:15:34.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.621+0000 7f7a0fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7a100b4410 0x7f7a1015e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.621+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a1015eb30 0x7f7a100acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.622+0000 7f7a0fe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a100af000 con 0x7f7a10147270 2024-04-25T10:15:34.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.622+0000 7f7a0fe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a100b4a80 con 0x7f7a100b4410 2024-04-25T10:15:34.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.622+0000 7f7a0fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a100b4d60 con 0x7f7a1015eb30 2024-04-25T10:15:34.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.622+0000 7f7a0e000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7a100b4410 0x7f7a1015e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.622+0000 7f7a0f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a1015eb30 0x7f7a100acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.622+0000 7f7a0e000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7a100b4410 0x7f7a1015e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:36770/0 (socket says 172.21.15.119:36770) 2024-04-25T10:15:34.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.622+0000 7f7a0e000640 1 -- 172.21.15.119:0/1252399968 learned_addr learned my addr 172.21.15.119:0/1252399968 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:34.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.623+0000 7f7a0e000640 1 -- 172.21.15.119:0/1252399968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a1015eb30 msgr2=0x7f7a100acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:34.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.623+0000 7f7a0e000640 1 --2- 172.21.15.119:0/1252399968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a1015eb30 0x7f7a100acf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.623+0000 7f7a0e000640 1 -- 172.21.15.119:0/1252399968 >> v1:172.21.15.132:6789/0 conn(0x7f7a10147270 legacy=0x7f7a1015dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.623+0000 7f7a0e000640 1 -- 172.21.15.119:0/1252399968 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a100b5040 con 0x7f7a100b4410 2024-04-25T10:15:34.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.624+0000 7f7a0e000640 1 --2- 172.21.15.119:0/1252399968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a100b4410 0x7f7a1015e5f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009870 tx=0x7f79f802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c820f759381d687f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:34.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.625+0000 7f7a0d600640 1 -- 172.21.15.119:0/1252399968 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8037070 con 0x7f7a100b4410 2024-04-25T10:15:34.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.625+0000 7f7a0d600640 1 -- 172.21.15.119:0/1252399968 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8032440 con 0x7f7a100b4410 2024-04-25T10:15:34.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.625+0000 7f7a0d600640 1 -- 172.21.15.119:0/1252399968 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8032810 con 0x7f7a100b4410 2024-04-25T10:15:34.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.626+0000 7f7a0fe00640 1 -- 172.21.15.119:0/1252399968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a100b4410 msgr2=0x7f7a1015e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.626+0000 7f7a0fe00640 1 --2- 172.21.15.119:0/1252399968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a100b4410 0x7f7a1015e5f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009870 tx=0x7f79f802a250 comp rx=0 tx=0).stop 2024-04-25T10:15:34.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.626+0000 7f7a0fe00640 1 -- 172.21.15.119:0/1252399968 shutdown_connections 2024-04-25T10:15:34.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.626+0000 7f7a0fe00640 1 --2- 172.21.15.119:0/1252399968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a1015eb30 0x7f7a100acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.626+0000 7f7a0fe00640 1 --2- 172.21.15.119:0/1252399968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a100b4410 0x7f7a1015e5f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.626+0000 7f7a0fe00640 1 -- 172.21.15.119:0/1252399968 >> 172.21.15.119:0/1252399968 conn(0x7f7a1006b1d0 msgr2=0x7f7a100b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:34.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.626+0000 7f7a0fe00640 1 -- 172.21.15.119:0/1252399968 shutdown_connections 2024-04-25T10:15:34.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.626+0000 7f7a0fe00640 1 -- 172.21.15.119:0/1252399968 wait complete. 2024-04-25T10:15:34.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.628+0000 7f7a0fe00640 1 Processor -- start 2024-04-25T10:15:34.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.628+0000 7f7a0fe00640 1 -- start start 2024-04-25T10:15:34.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.629+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a100b4410 0x7f7a100ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.629+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a10147270 0x7f7a1013e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.629+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a1015eb30 0x7f7a1013ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.629+0000 7f7a0fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a100ad4c0 con 0x7f7a1015eb30 2024-04-25T10:15:34.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.629+0000 7f7a0fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a100b5600 con 0x7f7a10147270 2024-04-25T10:15:34.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.629+0000 7f7a0fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a100ad640 con 0x7f7a100b4410 2024-04-25T10:15:34.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.630+0000 7f7a0f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a1015eb30 0x7f7a1013ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.630+0000 7f7a0f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a1015eb30 0x7f7a1013ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43610/0 (socket says 172.21.15.119:43610) 2024-04-25T10:15:34.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.630+0000 7f7a0f400640 1 -- 172.21.15.119:0/3301200782 learned_addr learned my addr 172.21.15.119:0/3301200782 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:34.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.630+0000 7f7a0e000640 1 --2- 172.21.15.119:0/3301200782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a10147270 0x7f7a1013e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.630+0000 7f7a0ea00640 1 --2- 172.21.15.119:0/3301200782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a100b4410 0x7f7a100ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.630+0000 7f7a0f400640 1 -- 172.21.15.119:0/3301200782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a100b4410 msgr2=0x7f7a100ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.630+0000 7f7a0f400640 1 --2- 172.21.15.119:0/3301200782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a100b4410 0x7f7a100ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.630+0000 7f7a0f400640 1 -- 172.21.15.119:0/3301200782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a10147270 msgr2=0x7f7a1013e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.630+0000 7f7a0f400640 1 --2- 172.21.15.119:0/3301200782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a10147270 0x7f7a1013e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.630+0000 7f7a0f400640 1 -- 172.21.15.119:0/3301200782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a101e5ff0 con 0x7f7a1015eb30 2024-04-25T10:15:34.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.631+0000 7f7a0f400640 1 --2- 172.21.15.119:0/3301200782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a1015eb30 0x7f7a1013ee90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800efc0 tx=0x7f7a0800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:34.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.631+0000 7f7a06a00640 1 -- 172.21.15.119:0/3301200782 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08018020 con 0x7f7a1015eb30 2024-04-25T10:15:34.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.631+0000 7f7a06a00640 1 -- 172.21.15.119:0/3301200782 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a08014480 con 0x7f7a1015eb30 2024-04-25T10:15:34.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.631+0000 7f7a06a00640 1 -- 172.21.15.119:0/3301200782 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08009e60 con 0x7f7a1015eb30 2024-04-25T10:15:34.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.631+0000 7f7a0fe00640 1 -- 172.21.15.119:0/3301200782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a101e62d0 con 0x7f7a1015eb30 2024-04-25T10:15:34.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.631+0000 7f7a0fe00640 1 -- 172.21.15.119:0/3301200782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a101e6890 con 0x7f7a1015eb30 2024-04-25T10:15:34.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.633+0000 7f7a0fe00640 1 -- 172.21.15.119:0/3301200782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1012fc50 con 0x7f7a1015eb30 2024-04-25T10:15:34.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.634+0000 7f7a06a00640 1 -- 172.21.15.119:0/3301200782 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a08005d10 con 0x7f7a1015eb30 2024-04-25T10:15:34.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.635+0000 7f7a06a00640 1 --2- 172.21.15.119:0/3301200782 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f79ec06b4d0 0x7f79ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:34.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.636+0000 7f7a06a00640 1 -- 172.21.15.119:0/3301200782 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f7a0808b170 con 0x7f7a1015eb30 2024-04-25T10:15:34.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.636+0000 7f7a0ea00640 1 --2- 172.21.15.119:0/3301200782 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f79ec06b4d0 0x7f79ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:34.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.636+0000 7f7a0ea00640 1 --2- 172.21.15.119:0/3301200782 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f79ec06b4d0 0x7f79ec06d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7a00000f60 tx=0x7f7a00009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:34.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.640+0000 7f7a06a00640 1 -- 172.21.15.119:0/3301200782 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a1012fc50 con 0x7f7a1015eb30 2024-04-25T10:15:34.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.834+0000 7f7a0fe00640 1 -- 172.21.15.119:0/3301200782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a100749b0 con 0x7f7a1015eb30 2024-04-25T10:15:34.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.836+0000 7f7a06a00640 1 -- 172.21.15.119:0/3301200782 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17009 (secure 0 0 0) 0x7f7a08022030 con 0x7f7a1015eb30 2024-04-25T10:15:34.841 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:34.841 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":85,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:14:50.735243+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:21.210576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:22.438674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/1734507541":"2024-04-26T10:14:50.735180+0000","172.21.15.119:6835/1734507541":"2024-04-26T10:14:50.735180+0000","172.21.15.119:6837/3997964942":"2024-04-26T10:13:36.160373+0000","172.21.15.119:6835/3997964942":"2024-04-26T10:13:36.160373+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:34.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.839+0000 7f7a0fe00640 1 -- 172.21.15.119:0/3301200782 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f79ec06b4d0 msgr2=0x7f79ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.839+0000 7f7a0fe00640 1 --2- 172.21.15.119:0/3301200782 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f79ec06b4d0 0x7f79ec06d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7a00000f60 tx=0x7f7a00009040 comp rx=0 tx=0).stop 2024-04-25T10:15:34.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.839+0000 7f7a0fe00640 1 -- 172.21.15.119:0/3301200782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a1015eb30 msgr2=0x7f7a1013ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:34.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.839+0000 7f7a0fe00640 1 --2- 172.21.15.119:0/3301200782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a1015eb30 0x7f7a1013ee90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800efc0 tx=0x7f7a0800c490 comp rx=0 tx=0).stop 2024-04-25T10:15:34.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.839+0000 7f7a0fe00640 1 -- 172.21.15.119:0/3301200782 shutdown_connections 2024-04-25T10:15:34.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.839+0000 7f7a0fe00640 1 --2- 172.21.15.119:0/3301200782 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f79ec06b4d0 0x7f79ec06d990 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.839+0000 7f7a0fe00640 1 --2- 172.21.15.119:0/3301200782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a1015eb30 0x7f7a1013ee90 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.839+0000 7f7a0fe00640 1 --2- 172.21.15.119:0/3301200782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a10147270 0x7f7a1013e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.839+0000 7f7a0fe00640 1 --2- 172.21.15.119:0/3301200782 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a100b4410 0x7f7a100ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:34.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.839+0000 7f7a0fe00640 1 -- 172.21.15.119:0/3301200782 >> 172.21.15.119:0/3301200782 conn(0x7f7a1006b1d0 msgr2=0x7f7a10070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:34.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.839+0000 7f7a0fe00640 1 -- 172.21.15.119:0/3301200782 shutdown_connections 2024-04-25T10:15:34.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:34.839+0000 7f7a0fe00640 1 -- 172.21.15.119:0/3301200782 wait complete. 2024-04-25T10:15:34.862 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:21.210576+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:22.438674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:34.862 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:35.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.058+0000 7fa8a2200640 1 Processor -- start 2024-04-25T10:15:35.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.059+0000 7fa8a2200640 1 -- start start 2024-04-25T10:15:35.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.060+0000 7fa8a2200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa89c0b5580 0x7fa89c0b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.060+0000 7fa8a2200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89c06c4b0 con 0x7fa89c0b24c0 2024-04-25T10:15:35.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.060+0000 7fa8a2200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89c06c790 con 0x7fa89c0b98b0 2024-04-25T10:15:35.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.060+0000 7fa8a2200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c15bb80 con 0x7fa89c0b5580 2024-04-25T10:15:35.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.060+0000 7fa89be00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa89c0b24c0 0x7fa89c0b2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50060/0 (socket says 172.21.15.119:50060) 2024-04-25T10:15:35.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.060+0000 7fa89be00640 1 -- 172.21.15.119:0/575951649 learned_addr learned my addr 172.21.15.119:0/575951649 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:35.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.061+0000 7fa89aa00640 1 --2- 172.21.15.119:0/575951649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa89c0b5580 0x7fa89c0b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.061+0000 7fa89a000640 1 -- 172.21.15.119:0/575951649 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250491760 0 0) 0x7fa89c06c4b0 con 0x7fa89c0b24c0 2024-04-25T10:15:35.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.061+0000 7fa89a000640 1 -- 172.21.15.119:0/575951649 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa87c003520 con 0x7fa89c0b24c0 2024-04-25T10:15:35.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.061+0000 7fa89a000640 1 -- 172.21.15.119:0/575951649 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2213106553 0 0) 0x7fa87c003520 con 0x7fa89c0b24c0 2024-04-25T10:15:35.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.061+0000 7fa89a000640 1 -- 172.21.15.119:0/575951649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa89c0b5580 msgr2=0x7fa89c0b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.061+0000 7fa89a000640 1 --2- 172.21.15.119:0/575951649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa89c0b5580 0x7fa89c0b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.061+0000 7fa89a000640 1 -- 172.21.15.119:0/575951649 >> v1:172.21.15.132:6789/0 conn(0x7fa89c0b98b0 legacy=0x7fa89c15eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.061+0000 7fa89a000640 1 -- 172.21.15.119:0/575951649 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c06ca70 con 0x7fa89c0b24c0 2024-04-25T10:15:35.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.061+0000 7fa89a000640 1 -- 172.21.15.119:0/575951649 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fa894001670 con 0x7fa89c0b24c0 2024-04-25T10:15:35.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.061+0000 7fa89a000640 1 -- 172.21.15.119:0/575951649 >> v1:172.21.15.119:6789/0 conn(0x7fa89c0b24c0 legacy=0x7fa89c0b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.061+0000 7fa89a000640 1 --2- 172.21.15.119:0/575951649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c0039e0 0x7fa87c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.061+0000 7fa89a000640 1 --2- 172.21.15.119:0/575951649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa89c0b5580 0x7fa87c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.062+0000 7fa89a000640 1 --2- 172.21.15.119:0/575951649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa87c004d40 0x7fa87c0051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.062+0000 7fa89a000640 1 -- 172.21.15.119:0/575951649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c15bb80 con 0x7fa87c0039e0 2024-04-25T10:15:35.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.062+0000 7fa89a000640 1 -- 172.21.15.119:0/575951649 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c00bc00 con 0x7fa87c004d40 2024-04-25T10:15:35.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.062+0000 7fa89a000640 1 -- 172.21.15.119:0/575951649 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c00be00 con 0x7fa89c0b5580 2024-04-25T10:15:35.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.062+0000 7fa89b400640 1 --2- 172.21.15.119:0/575951649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c0039e0 0x7fa87c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.062+0000 7fa89aa00640 1 --2- 172.21.15.119:0/575951649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa89c0b5580 0x7fa87c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.062+0000 7fa89be00640 1 --2- 172.21.15.119:0/575951649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa87c004d40 0x7fa87c0051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.062+0000 7fa89b400640 1 -- 172.21.15.119:0/575951649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa89c0b5580 msgr2=0x7fa87c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.062+0000 7fa89b400640 1 --2- 172.21.15.119:0/575951649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa89c0b5580 0x7fa87c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.062+0000 7fa89b400640 1 -- 172.21.15.119:0/575951649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa87c004d40 msgr2=0x7fa87c0051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.062+0000 7fa89b400640 1 --2- 172.21.15.119:0/575951649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa87c004d40 0x7fa87c0051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.062+0000 7fa89b400640 1 -- 172.21.15.119:0/575951649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c003520 con 0x7fa87c0039e0 2024-04-25T10:15:35.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.063+0000 7fa89b400640 1 --2- 172.21.15.119:0/575951649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c0039e0 0x7fa87c003e00 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fa88c001750 tx=0x7fa87c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2800ee760a82dd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.064+0000 7fa89a000640 1 -- 172.21.15.119:0/575951649 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c001370 con 0x7fa87c0039e0 2024-04-25T10:15:35.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.064+0000 7fa89a000640 1 -- 172.21.15.119:0/575951649 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88c0052d0 con 0x7fa87c0039e0 2024-04-25T10:15:35.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.064+0000 7fa89a000640 1 -- 172.21.15.119:0/575951649 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c02abe0 con 0x7fa87c0039e0 2024-04-25T10:15:35.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.064+0000 7fa8a2200640 1 -- 172.21.15.119:0/575951649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c0039e0 msgr2=0x7fa87c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.064+0000 7fa8a2200640 1 --2- 172.21.15.119:0/575951649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa87c0039e0 0x7fa87c003e00 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fa88c001750 tx=0x7fa87c005830 comp rx=0 tx=0).stop 2024-04-25T10:15:35.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.064+0000 7fa89be00640 1 -- 172.21.15.119:0/575951649 reap_dead start 2024-04-25T10:15:35.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.064+0000 7fa8a2200640 1 -- 172.21.15.119:0/575951649 shutdown_connections 2024-04-25T10:15:35.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.064+0000 7fa8a2200640 1 -- 172.21.15.119:0/575951649 >> 172.21.15.119:0/575951649 conn(0x7fa89c0d0580 msgr2=0x7fa89c0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.065+0000 7fa8a2200640 1 -- 172.21.15.119:0/575951649 shutdown_connections 2024-04-25T10:15:35.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.065+0000 7fa8a2200640 1 -- 172.21.15.119:0/575951649 wait complete. 2024-04-25T10:15:35.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.066+0000 7fa8a2200640 1 Processor -- start 2024-04-25T10:15:35.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.066+0000 7fa8a2200640 1 -- start start 2024-04-25T10:15:35.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.067+0000 7fa8a2200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa87c0039e0 0x7fa89c1f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.067+0000 7fa8a2200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa89c15bf30 0x7fa89c15e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.067+0000 7fa8a2200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa89c1f6510 0x7fa89c160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.067+0000 7fa8a2200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c0cec40 con 0x7fa89c1f6510 2024-04-25T10:15:35.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.067+0000 7fa8a2200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c06c140 con 0x7fa87c0039e0 2024-04-25T10:15:35.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.067+0000 7fa8a2200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c0b3060 con 0x7fa89c15bf30 2024-04-25T10:15:35.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.068+0000 7fa89be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa89c1f6510 0x7fa89c160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.068+0000 7fa89be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa89c1f6510 0x7fa89c160400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43636/0 (socket says 172.21.15.119:43636) 2024-04-25T10:15:35.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.068+0000 7fa89be00640 1 -- 172.21.15.119:0/1097273503 learned_addr learned my addr 172.21.15.119:0/1097273503 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:35.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.068+0000 7fa89aa00640 1 --2- 172.21.15.119:0/1097273503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa89c15bf30 0x7fa89c15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.068+0000 7fa89b400640 1 --2- 172.21.15.119:0/1097273503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa87c0039e0 0x7fa89c1f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.068+0000 7fa89be00640 1 -- 172.21.15.119:0/1097273503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa89c15bf30 msgr2=0x7fa89c15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.068+0000 7fa89be00640 1 --2- 172.21.15.119:0/1097273503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa89c15bf30 0x7fa89c15e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.068+0000 7fa89be00640 1 -- 172.21.15.119:0/1097273503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa87c0039e0 msgr2=0x7fa89c1f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.068+0000 7fa89be00640 1 --2- 172.21.15.119:0/1097273503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa87c0039e0 0x7fa89c1f4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.068+0000 7fa89be00640 1 -- 172.21.15.119:0/1097273503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c15c3b0 con 0x7fa89c1f6510 2024-04-25T10:15:35.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.068+0000 7fa89be00640 1 --2- 172.21.15.119:0/1097273503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa89c1f6510 0x7fa89c160400 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fa89c162790 tx=0x7fa89400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.068+0000 7fa893400640 1 -- 172.21.15.119:0/1097273503 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894007440 con 0x7fa89c1f6510 2024-04-25T10:15:35.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.069+0000 7fa893400640 1 -- 172.21.15.119:0/1097273503 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8940075e0 con 0x7fa89c1f6510 2024-04-25T10:15:35.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.069+0000 7fa893400640 1 -- 172.21.15.119:0/1097273503 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894004a20 con 0x7fa89c1f6510 2024-04-25T10:15:35.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.069+0000 7fa8a2200640 1 -- 172.21.15.119:0/1097273503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89c15ead0 con 0x7fa89c1f6510 2024-04-25T10:15:35.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.069+0000 7fa8a2200640 1 -- 172.21.15.119:0/1097273503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89c0c7d30 con 0x7fa89c1f6510 2024-04-25T10:15:35.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.070+0000 7fa8a2200640 1 -- 172.21.15.119:0/1097273503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89c0d1bf0 con 0x7fa89c1f6510 2024-04-25T10:15:35.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.071+0000 7fa893400640 1 -- 172.21.15.119:0/1097273503 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa89401b020 con 0x7fa89c1f6510 2024-04-25T10:15:35.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.072+0000 7fa893400640 1 --2- 172.21.15.119:0/1097273503 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa87806b500 0x7fa87806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.072+0000 7fa893400640 1 -- 172.21.15.119:0/1097273503 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fa894089ca0 con 0x7fa89c1f6510 2024-04-25T10:15:35.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.072+0000 7fa89b400640 1 --2- 172.21.15.119:0/1097273503 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa87806b500 0x7fa87806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.073+0000 7fa89b400640 1 --2- 172.21.15.119:0/1097273503 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa87806b500 0x7fa87806d9c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa88c032000 tx=0x7fa88c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.076+0000 7fa893400640 1 -- 172.21.15.119:0/1097273503 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa89c0d1bf0 con 0x7fa89c1f6510 2024-04-25T10:15:35.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.273+0000 7fa8a2200640 1 -- 172.21.15.119:0/1097273503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa89c0ce1d0 con 0x7fa89c1f6510 2024-04-25T10:15:35.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.274+0000 7fa893400640 1 -- 172.21.15.119:0/1097273503 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa894058a40 con 0x7fa89c1f6510 2024-04-25T10:15:35.278 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:15:35.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.277+0000 7fa8a2200640 1 -- 172.21.15.119:0/1097273503 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa87806b500 msgr2=0x7fa87806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.277+0000 7fa8a2200640 1 --2- 172.21.15.119:0/1097273503 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa87806b500 0x7fa87806d9c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa88c032000 tx=0x7fa88c02e000 comp rx=0 tx=0).stop 2024-04-25T10:15:35.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.277+0000 7fa8a2200640 1 -- 172.21.15.119:0/1097273503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa89c1f6510 msgr2=0x7fa89c160400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.277+0000 7fa8a2200640 1 --2- 172.21.15.119:0/1097273503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa89c1f6510 0x7fa89c160400 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fa89c162790 tx=0x7fa89400d9d0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.277+0000 7fa8a2200640 1 -- 172.21.15.119:0/1097273503 shutdown_connections 2024-04-25T10:15:35.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.277+0000 7fa8a2200640 1 --2- 172.21.15.119:0/1097273503 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa87806b500 0x7fa87806d9c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.277+0000 7fa8a2200640 1 --2- 172.21.15.119:0/1097273503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa89c1f6510 0x7fa89c160400 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.277+0000 7fa8a2200640 1 --2- 172.21.15.119:0/1097273503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa89c15bf30 0x7fa89c15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.277+0000 7fa8a2200640 1 --2- 172.21.15.119:0/1097273503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa87c0039e0 0x7fa89c1f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.277+0000 7fa8a2200640 1 -- 172.21.15.119:0/1097273503 >> 172.21.15.119:0/1097273503 conn(0x7fa89c0d0580 msgr2=0x7fa89c0d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.277+0000 7fa8a2200640 1 -- 172.21.15.119:0/1097273503 shutdown_connections 2024-04-25T10:15:35.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.277+0000 7fa8a2200640 1 -- 172.21.15.119:0/1097273503 wait complete. 2024-04-25T10:15:35.301 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:15:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.496+0000 7fc06f200640 1 Processor -- start 2024-04-25T10:15:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.496+0000 7fc06f200640 1 -- start start 2024-04-25T10:15:35.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.497+0000 7fc06f200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc070161d60 0x7fc070166380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.497+0000 7fc06f200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc070162140 con 0x7fc0700cea30 2024-04-25T10:15:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.498+0000 7fc06f200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0700cf210 con 0x7fc0700b3500 2024-04-25T10:15:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.498+0000 7fc06f200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0700b2470 con 0x7fc070161d60 2024-04-25T10:15:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.498+0000 7fc06d400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc0700cea30 0x7fc0700b2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50068/0 (socket says 172.21.15.119:50068) 2024-04-25T10:15:35.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.498+0000 7fc06d400640 1 -- 172.21.15.119:0/3353024667 learned_addr learned my addr 172.21.15.119:0/3353024667 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:35.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.498+0000 7fc06de00640 1 --2- 172.21.15.119:0/3353024667 >> v2:172.21.15.132:3300/0 conn(0x7fc070161d60 0x7fc070166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.498+0000 7fc06ca00640 1 -- 172.21.15.119:0/3353024667 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965271002 0 0) 0x7fc05c001490 con 0x7fc0700cea30 2024-04-25T10:15:35.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.498+0000 7fc06ca00640 1 -- 172.21.15.119:0/3353024667 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc054003520 con 0x7fc0700cea30 2024-04-25T10:15:35.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.499+0000 7fc06ca00640 1 -- 172.21.15.119:0/3353024667 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3930703669 0 0) 0x7fc054003520 con 0x7fc0700cea30 2024-04-25T10:15:35.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.499+0000 7fc06ca00640 1 -- 172.21.15.119:0/3353024667 >> v2:172.21.15.132:3300/0 conn(0x7fc070161d60 msgr2=0x7fc070166380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.499+0000 7fc06ca00640 1 --2- 172.21.15.119:0/3353024667 >> v2:172.21.15.132:3300/0 conn(0x7fc070161d60 0x7fc070166380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.499+0000 7fc06ca00640 1 -- 172.21.15.119:0/3353024667 >> v1:172.21.15.132:6789/0 conn(0x7fc0700b3500 legacy=0x7fc07006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.499+0000 7fc06ca00640 1 -- 172.21.15.119:0/3353024667 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0700b26c0 con 0x7fc0700cea30 2024-04-25T10:15:35.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.499+0000 7fc06ca00640 1 -- 172.21.15.119:0/3353024667 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fc05c001bc0 con 0x7fc0700cea30 2024-04-25T10:15:35.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.499+0000 7fc06ca00640 1 -- 172.21.15.119:0/3353024667 >> v1:172.21.15.119:6789/0 conn(0x7fc0700cea30 legacy=0x7fc0700b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.499+0000 7fc06ca00640 1 --2- 172.21.15.119:0/3353024667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc054003a00 0x7fc054003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.499+0000 7fc06ca00640 1 --2- 172.21.15.119:0/3353024667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc054004800 0x7fc054004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.499+0000 7fc06ca00640 1 --2- 172.21.15.119:0/3353024667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054005230 0x7fc05400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.499+0000 7fc06ca00640 1 -- 172.21.15.119:0/3353024667 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0700b2470 con 0x7fc054003a00 2024-04-25T10:15:35.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.499+0000 7fc06ca00640 1 -- 172.21.15.119:0/3353024667 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05400c190 con 0x7fc054005230 2024-04-25T10:15:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.499+0000 7fc06ca00640 1 -- 172.21.15.119:0/3353024667 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05400c390 con 0x7fc054004800 2024-04-25T10:15:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.499+0000 7fc06de00640 1 --2- 172.21.15.119:0/3353024667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc054003a00 0x7fc054003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.500+0000 7fc06e800640 1 --2- 172.21.15.119:0/3353024667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054005230 0x7fc05400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.500+0000 7fc06d400640 1 --2- 172.21.15.119:0/3353024667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc054004800 0x7fc054004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.500+0000 7fc06de00640 1 -- 172.21.15.119:0/3353024667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc054004800 msgr2=0x7fc054004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.500+0000 7fc06de00640 1 --2- 172.21.15.119:0/3353024667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc054004800 0x7fc054004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.500+0000 7fc06de00640 1 -- 172.21.15.119:0/3353024667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054005230 msgr2=0x7fc05400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.500+0000 7fc06de00640 1 --2- 172.21.15.119:0/3353024667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054005230 0x7fc05400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.500+0000 7fc06de00640 1 -- 172.21.15.119:0/3353024667 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc054003520 con 0x7fc054003a00 2024-04-25T10:15:35.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.500+0000 7fc06e800640 1 --2- 172.21.15.119:0/3353024667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054005230 0x7fc05400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:35.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.500+0000 7fc06e800640 1 -- 172.21.15.119:0/3353024667 reap_dead start 2024-04-25T10:15:35.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.501+0000 7fc06de00640 1 --2- 172.21.15.119:0/3353024667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc054003a00 0x7fc054003e40 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc060009ac0 tx=0x7fc054005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb050ea6262d939 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.501+0000 7fc06ca00640 1 -- 172.21.15.119:0/3353024667 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc060036070 con 0x7fc054003a00 2024-04-25T10:15:35.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.501+0000 7fc06ca00640 1 -- 172.21.15.119:0/3353024667 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc060031400 con 0x7fc054003a00 2024-04-25T10:15:35.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.501+0000 7fc06ca00640 1 -- 172.21.15.119:0/3353024667 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc060009690 con 0x7fc054003a00 2024-04-25T10:15:35.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.502+0000 7fc06f200640 1 -- 172.21.15.119:0/3353024667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc054003a00 msgr2=0x7fc054003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.502+0000 7fc06f200640 1 --2- 172.21.15.119:0/3353024667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc054003a00 0x7fc054003e40 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc060009ac0 tx=0x7fc054005830 comp rx=0 tx=0).stop 2024-04-25T10:15:35.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.502+0000 7fc06f200640 1 -- 172.21.15.119:0/3353024667 shutdown_connections 2024-04-25T10:15:35.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.502+0000 7fc06f200640 1 --2- 172.21.15.119:0/3353024667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc054003a00 0x7fc054003e40 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.502+0000 7fc06f200640 1 -- 172.21.15.119:0/3353024667 >> 172.21.15.119:0/3353024667 conn(0x7fc070066ff0 msgr2=0x7fc0700b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.502+0000 7fc06f200640 1 -- 172.21.15.119:0/3353024667 shutdown_connections 2024-04-25T10:15:35.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.502+0000 7fc06f200640 1 -- 172.21.15.119:0/3353024667 wait complete. 2024-04-25T10:15:35.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.504+0000 7fc06f200640 1 Processor -- start 2024-04-25T10:15:35.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.504+0000 7fc06f200640 1 -- start start 2024-04-25T10:15:35.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.505+0000 7fc06f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054003a00 0x7fc07015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.505+0000 7fc06f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0701530b0 0x7fc0701606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.505+0000 7fc06f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc070156d50 0x7fc070160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.505+0000 7fc06f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0701658a0 con 0x7fc070156d50 2024-04-25T10:15:35.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.505+0000 7fc06f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc070165ef0 con 0x7fc054003a00 2024-04-25T10:15:35.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.505+0000 7fc06f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc070165b80 con 0x7fc0701530b0 2024-04-25T10:15:35.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.505+0000 7fc06e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc070156d50 0x7fc070160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.505+0000 7fc06e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc070156d50 0x7fc070160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43660/0 (socket says 172.21.15.119:43660) 2024-04-25T10:15:35.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.505+0000 7fc06e800640 1 -- 172.21.15.119:0/1175085328 learned_addr learned my addr 172.21.15.119:0/1175085328 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:35.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.506+0000 7fc06e800640 1 -- 172.21.15.119:0/1175085328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0701530b0 msgr2=0x7fc0701606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.506+0000 7fc06e800640 1 --2- 172.21.15.119:0/1175085328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0701530b0 0x7fc0701606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.506+0000 7fc06e800640 1 -- 172.21.15.119:0/1175085328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054003a00 msgr2=0x7fc07015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.506+0000 7fc06de00640 1 --2- 172.21.15.119:0/1175085328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054003a00 0x7fc07015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.506+0000 7fc06e800640 1 --2- 172.21.15.119:0/1175085328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054003a00 0x7fc07015c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.506+0000 7fc06e800640 1 -- 172.21.15.119:0/1175085328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07015cd10 con 0x7fc070156d50 2024-04-25T10:15:35.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.506+0000 7fc06e800640 1 --2- 172.21.15.119:0/1175085328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc070156d50 0x7fc070160c10 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc06800d1d0 tx=0x7fc068007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.506+0000 7fc066000640 1 -- 172.21.15.119:0/1175085328 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068018070 con 0x7fc070156d50 2024-04-25T10:15:35.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.506+0000 7fc066000640 1 -- 172.21.15.119:0/1175085328 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0680052f0 con 0x7fc070156d50 2024-04-25T10:15:35.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.506+0000 7fc066000640 1 -- 172.21.15.119:0/1175085328 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068003830 con 0x7fc070156d50 2024-04-25T10:15:35.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.506+0000 7fc06f200640 1 -- 172.21.15.119:0/1175085328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0701f8110 con 0x7fc070156d50 2024-04-25T10:15:35.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.506+0000 7fc06f200640 1 -- 172.21.15.119:0/1175085328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0701f86d0 con 0x7fc070156d50 2024-04-25T10:15:35.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.508+0000 7fc06f200640 1 -- 172.21.15.119:0/1175085328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0700749b0 con 0x7fc070156d50 2024-04-25T10:15:35.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.508+0000 7fc066000640 1 -- 172.21.15.119:0/1175085328 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc06801b070 con 0x7fc070156d50 2024-04-25T10:15:35.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.509+0000 7fc066000640 1 --2- 172.21.15.119:0/1175085328 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc05006b500 0x7fc05006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.509+0000 7fc066000640 1 -- 172.21.15.119:0/1175085328 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fc0680896a0 con 0x7fc070156d50 2024-04-25T10:15:35.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.510+0000 7fc06de00640 1 --2- 172.21.15.119:0/1175085328 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc05006b500 0x7fc05006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.510+0000 7fc06de00640 1 --2- 172.21.15.119:0/1175085328 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc05006b500 0x7fc05006d9c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc060033000 tx=0x7fc060008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.514+0000 7fc066000640 1 -- 172.21.15.119:0/1175085328 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc0700cf210 con 0x7fc070156d50 2024-04-25T10:15:35.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.705+0000 7fc06f200640 1 -- 172.21.15.119:0/1175085328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc0700b0580 con 0x7fc070156d50 2024-04-25T10:15:35.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.706+0000 7fc066000640 1 -- 172.21.15.119:0/1175085328 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc068058440 con 0x7fc070156d50 2024-04-25T10:15:35.710 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:15:35.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.708+0000 7fc06f200640 1 -- 172.21.15.119:0/1175085328 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc05006b500 msgr2=0x7fc05006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.708+0000 7fc06f200640 1 --2- 172.21.15.119:0/1175085328 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc05006b500 0x7fc05006d9c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc060033000 tx=0x7fc060008030 comp rx=0 tx=0).stop 2024-04-25T10:15:35.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.709+0000 7fc06f200640 1 -- 172.21.15.119:0/1175085328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc070156d50 msgr2=0x7fc070160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.709+0000 7fc06f200640 1 --2- 172.21.15.119:0/1175085328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc070156d50 0x7fc070160c10 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc06800d1d0 tx=0x7fc068007f90 comp rx=0 tx=0).stop 2024-04-25T10:15:35.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.709+0000 7fc06f200640 1 -- 172.21.15.119:0/1175085328 shutdown_connections 2024-04-25T10:15:35.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.709+0000 7fc06f200640 1 --2- 172.21.15.119:0/1175085328 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc05006b500 0x7fc05006d9c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.709+0000 7fc06f200640 1 --2- 172.21.15.119:0/1175085328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc070156d50 0x7fc070160c10 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.709+0000 7fc06f200640 1 --2- 172.21.15.119:0/1175085328 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc0701530b0 0x7fc0701606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.709+0000 7fc06f200640 1 --2- 172.21.15.119:0/1175085328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc054003a00 0x7fc07015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.709+0000 7fc06f200640 1 -- 172.21.15.119:0/1175085328 >> 172.21.15.119:0/1175085328 conn(0x7fc070066ff0 msgr2=0x7fc0700afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.709+0000 7fc06f200640 1 -- 172.21.15.119:0/1175085328 shutdown_connections 2024-04-25T10:15:35.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.709+0000 7fc06f200640 1 -- 172.21.15.119:0/1175085328 wait complete. 2024-04-25T10:15:35.732 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:15:35.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.924+0000 7f12b5e00640 1 Processor -- start 2024-04-25T10:15:35.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.924+0000 7f12b5e00640 1 -- start start 2024-04-25T10:15:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.925+0000 7f12b5e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f12b00cec60 0x7f12b00cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.925+0000 7f12b5e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b006d080 con 0x7f12b006f040 2024-04-25T10:15:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.925+0000 7f12b5e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b006d360 con 0x7f12b006aba0 2024-04-25T10:15:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.925+0000 7f12b5e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b006d640 con 0x7f12b00cec60 2024-04-25T10:15:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.926+0000 7f12a7e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f12b006f040 0x7f12b00cd2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50084/0 (socket says 172.21.15.119:50084) 2024-04-25T10:15:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.926+0000 7f12a7e00640 1 -- 172.21.15.119:0/1511446010 learned_addr learned my addr 172.21.15.119:0/1511446010 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.926+0000 7f12b5400640 1 --2- 172.21.15.119:0/1511446010 >> v2:172.21.15.132:3300/0 conn(0x7f12b00cec60 0x7f12b00cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.926+0000 7f12afe00640 1 -- 172.21.15.119:0/1511446010 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660416191 0 0) 0x7f12b006d080 con 0x7f12b006f040 2024-04-25T10:15:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.926+0000 7f12afe00640 1 -- 172.21.15.119:0/1511446010 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1294005730 con 0x7f12b006f040 2024-04-25T10:15:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.926+0000 7f12afe00640 1 -- 172.21.15.119:0/1511446010 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2799141785 0 0) 0x7f1294005730 con 0x7f12b006f040 2024-04-25T10:15:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.926+0000 7f12afe00640 1 -- 172.21.15.119:0/1511446010 >> v2:172.21.15.132:3300/0 conn(0x7f12b00cec60 msgr2=0x7f12b00cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.926+0000 7f12afe00640 1 --2- 172.21.15.119:0/1511446010 >> v2:172.21.15.132:3300/0 conn(0x7f12b00cec60 0x7f12b00cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.926+0000 7f12afe00640 1 -- 172.21.15.119:0/1511446010 >> v1:172.21.15.132:6789/0 conn(0x7f12b006aba0 legacy=0x7f12b006af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.926+0000 7f12afe00640 1 -- 172.21.15.119:0/1511446010 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b006d920 con 0x7f12b006f040 2024-04-25T10:15:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12afe00640 1 -- 172.21.15.119:0/1511446010 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f1298001670 con 0x7f12b006f040 2024-04-25T10:15:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12afe00640 1 -- 172.21.15.119:0/1511446010 >> v1:172.21.15.119:6789/0 conn(0x7f12b006f040 legacy=0x7f12b00cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12afe00640 1 --2- 172.21.15.119:0/1511446010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1294005bc0 0x7f1294005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12afe00640 1 --2- 172.21.15.119:0/1511446010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1294006a10 0x7f1294006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12afe00640 1 --2- 172.21.15.119:0/1511446010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12940073d0 0x7f129400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12afe00640 1 -- 172.21.15.119:0/1511446010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b006d640 con 0x7f12940073d0 2024-04-25T10:15:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12afe00640 1 -- 172.21.15.119:0/1511446010 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129400e270 con 0x7f1294006a10 2024-04-25T10:15:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12afe00640 1 -- 172.21.15.119:0/1511446010 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129400e470 con 0x7f1294005bc0 2024-04-25T10:15:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12b5400640 1 --2- 172.21.15.119:0/1511446010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12940073d0 0x7f129400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12a7e00640 1 --2- 172.21.15.119:0/1511446010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1294006a10 0x7f1294006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12b4a00640 1 --2- 172.21.15.119:0/1511446010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1294005bc0 0x7f1294005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12b5400640 1 -- 172.21.15.119:0/1511446010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1294005bc0 msgr2=0x7f1294005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12b5400640 1 --2- 172.21.15.119:0/1511446010 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1294005bc0 0x7f1294005fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12b5400640 1 -- 172.21.15.119:0/1511446010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1294006a10 msgr2=0x7f1294006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12b5400640 1 --2- 172.21.15.119:0/1511446010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1294006a10 0x7f1294006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.927+0000 7f12b5400640 1 -- 172.21.15.119:0/1511446010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1294005730 con 0x7f12940073d0 2024-04-25T10:15:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.928+0000 7f12b5400640 1 -- 172.21.15.119:0/1511446010 reap_dead start 2024-04-25T10:15:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.928+0000 7f12b5400640 1 --2- 172.21.15.119:0/1511446010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12940073d0 0x7f129400dbd0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009ac0 tx=0x7f1294007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a565a08bd08a81ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.929+0000 7f12afe00640 1 -- 172.21.15.119:0/1511446010 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8009690 con 0x7f12940073d0 2024-04-25T10:15:35.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.929+0000 7f12afe00640 1 -- 172.21.15.119:0/1511446010 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a8004020 con 0x7f12940073d0 2024-04-25T10:15:35.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.929+0000 7f12afe00640 1 -- 172.21.15.119:0/1511446010 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80043d0 con 0x7f12940073d0 2024-04-25T10:15:35.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.929+0000 7f12b5e00640 1 -- 172.21.15.119:0/1511446010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12940073d0 msgr2=0x7f129400dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.929+0000 7f12b5e00640 1 --2- 172.21.15.119:0/1511446010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12940073d0 0x7f129400dbd0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009ac0 tx=0x7f1294007a40 comp rx=0 tx=0).stop 2024-04-25T10:15:35.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.929+0000 7f12b5e00640 1 -- 172.21.15.119:0/1511446010 shutdown_connections 2024-04-25T10:15:35.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.929+0000 7f12b5e00640 1 --2- 172.21.15.119:0/1511446010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12940073d0 0x7f129400dbd0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.929+0000 7f12b5e00640 1 -- 172.21.15.119:0/1511446010 >> 172.21.15.119:0/1511446010 conn(0x7f12b0066ff0 msgr2=0x7f12b00b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:35.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.929+0000 7f12b5e00640 1 -- 172.21.15.119:0/1511446010 shutdown_connections 2024-04-25T10:15:35.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.929+0000 7f12b5e00640 1 -- 172.21.15.119:0/1511446010 wait complete. 2024-04-25T10:15:35.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.931+0000 7f12b5e00640 1 Processor -- start 2024-04-25T10:15:35.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.931+0000 7f12b5e00640 1 -- start start 2024-04-25T10:15:35.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.932+0000 7f12b5e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12940073d0 0x7f12b0159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.932+0000 7f12b5e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12b01500b0 0x7f12b0155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.932+0000 7f12b5e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b0155e70 0x7f12b015be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.932+0000 7f12b5e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b00aed40 con 0x7f12b01500b0 2024-04-25T10:15:35.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.932+0000 7f12b5e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b015a980 con 0x7f12b0155e70 2024-04-25T10:15:35.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.932+0000 7f12b5e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b00aeec0 con 0x7f12940073d0 2024-04-25T10:15:35.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.932+0000 7f12a7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12b01500b0 0x7f12b0155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.932+0000 7f12a7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12b01500b0 0x7f12b0155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43670/0 (socket says 172.21.15.119:43670) 2024-04-25T10:15:35.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.932+0000 7f12a7e00640 1 -- 172.21.15.119:0/3984090626 learned_addr learned my addr 172.21.15.119:0/3984090626 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:35.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.933+0000 7f12b4a00640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12940073d0 0x7f12b0159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.933+0000 7f12a7e00640 1 -- 172.21.15.119:0/3984090626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12940073d0 msgr2=0x7f12b0159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.933+0000 7f12b5400640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b0155e70 0x7f12b015be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.933+0000 7f12a7e00640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12940073d0 0x7f12b0159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.933+0000 7f12a7e00640 1 -- 172.21.15.119:0/3984090626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b0155e70 msgr2=0x7f12b015be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:35.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.933+0000 7f12a7e00640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b0155e70 0x7f12b015be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:35.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.933+0000 7f12a7e00640 1 -- 172.21.15.119:0/3984090626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b0151c50 con 0x7f12b01500b0 2024-04-25T10:15:35.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.933+0000 7f12b4a00640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12940073d0 0x7f12b0159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:35.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.933+0000 7f12a7e00640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12b01500b0 0x7f12b0155930 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f129800e730 tx=0x7f129800eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.934+0000 7f12ad600640 1 -- 172.21.15.119:0/3984090626 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129800c6c0 con 0x7f12b01500b0 2024-04-25T10:15:35.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.934+0000 7f12ad600640 1 -- 172.21.15.119:0/3984090626 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1298018070 con 0x7f12b01500b0 2024-04-25T10:15:35.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.934+0000 7f12ad600640 1 -- 172.21.15.119:0/3984090626 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129800caa0 con 0x7f12b01500b0 2024-04-25T10:15:35.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.934+0000 7f12b5e00640 1 -- 172.21.15.119:0/3984090626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b0151ef0 con 0x7f12b01500b0 2024-04-25T10:15:35.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.934+0000 7f12b5e00640 1 -- 172.21.15.119:0/3984090626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b015c350 con 0x7f12b01500b0 2024-04-25T10:15:35.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.935+0000 7f12b5e00640 1 -- 172.21.15.119:0/3984090626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b00749b0 con 0x7f12b01500b0 2024-04-25T10:15:35.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.936+0000 7f12ad600640 1 -- 172.21.15.119:0/3984090626 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f129800cc40 con 0x7f12b01500b0 2024-04-25T10:15:35.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.937+0000 7f12ad600640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f128c06b500 0x7f128c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:35.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.937+0000 7f12ad600640 1 -- 172.21.15.119:0/3984090626 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f129808a070 con 0x7f12b01500b0 2024-04-25T10:15:35.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.938+0000 7f12b4a00640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f128c06b500 0x7f128c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:35.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.938+0000 7f12b4a00640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f128c06b500 0x7f128c06d9c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f12a0001ce0 tx=0x7f12a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:35.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:35.942+0000 7f12ad600640 1 -- 172.21.15.119:0/3984090626 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1298058e10 con 0x7f12b01500b0 2024-04-25T10:15:36.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.125+0000 7f12b5e00640 1 -- 172.21.15.119:0/3984090626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12b00cf1b0 con 0x7f12b01500b0 2024-04-25T10:15:36.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.126+0000 7f12ad600640 1 -- 172.21.15.119:0/3984090626 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1298058710 con 0x7f12b01500b0 2024-04-25T10:15:36.130 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:15:36.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.128+0000 7f12b5e00640 1 -- 172.21.15.119:0/3984090626 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f128c06b500 msgr2=0x7f128c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.128+0000 7f12b5e00640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f128c06b500 0x7f128c06d9c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f12a0001ce0 tx=0x7f12a0009040 comp rx=0 tx=0).stop 2024-04-25T10:15:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.128+0000 7f12b5e00640 1 -- 172.21.15.119:0/3984090626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12b01500b0 msgr2=0x7f12b0155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.128+0000 7f12b5e00640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12b01500b0 0x7f12b0155930 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f129800e730 tx=0x7f129800eb30 comp rx=0 tx=0).stop 2024-04-25T10:15:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.128+0000 7f12b5e00640 1 -- 172.21.15.119:0/3984090626 shutdown_connections 2024-04-25T10:15:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.128+0000 7f12b5e00640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f128c06b500 0x7f128c06d9c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.128+0000 7f12b5e00640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12b0155e70 0x7f12b015be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.128+0000 7f12b5e00640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12b01500b0 0x7f12b0155930 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.128+0000 7f12b5e00640 1 --2- 172.21.15.119:0/3984090626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f12940073d0 0x7f12b0159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.128+0000 7f12b5e00640 1 -- 172.21.15.119:0/3984090626 >> 172.21.15.119:0/3984090626 conn(0x7f12b0066ff0 msgr2=0x7f12b00b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.129+0000 7f12b5e00640 1 -- 172.21.15.119:0/3984090626 shutdown_connections 2024-04-25T10:15:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.129+0000 7f12b5e00640 1 -- 172.21.15.119:0/3984090626 wait complete. 2024-04-25T10:15:36.151 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:36.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.341+0000 7f4b5f200640 1 Processor -- start 2024-04-25T10:15:36.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.341+0000 7f4b5f200640 1 -- start start 2024-04-25T10:15:36.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.343+0000 7f4b5f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b600b9270 0x7f4b600d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.343+0000 7f4b5f200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b6015ef10 con 0x7f4b600d3f80 2024-04-25T10:15:36.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.343+0000 7f4b5f200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b6015e390 con 0x7f4b6015db60 2024-04-25T10:15:36.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.343+0000 7f4b5f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6015e670 con 0x7f4b600b9270 2024-04-25T10:15:36.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.343+0000 7f4b57e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4b600d3f80 0x7f4b600d1f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50090/0 (socket says 172.21.15.119:50090) 2024-04-25T10:15:36.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.343+0000 7f4b57e00640 1 -- 172.21.15.119:0/960398912 learned_addr learned my addr 172.21.15.119:0/960398912 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.344+0000 7f4b5e800640 1 --2- 172.21.15.119:0/960398912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b600b9270 0x7f4b600d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.344+0000 7f4b5d400640 1 -- 172.21.15.119:0/960398912 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561069881 0 0) 0x7f4b6015ef10 con 0x7f4b600d3f80 2024-04-25T10:15:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.344+0000 7f4b5d400640 1 -- 172.21.15.119:0/960398912 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b44003520 con 0x7f4b600d3f80 2024-04-25T10:15:36.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.344+0000 7f4b5d400640 1 -- 172.21.15.119:0/960398912 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1472293346 0 0) 0x7f4b44003520 con 0x7f4b600d3f80 2024-04-25T10:15:36.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.344+0000 7f4b5d400640 1 -- 172.21.15.119:0/960398912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b600b9270 msgr2=0x7f4b600d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:36.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.344+0000 7f4b5d400640 1 --2- 172.21.15.119:0/960398912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b600b9270 0x7f4b600d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.344+0000 7f4b5d400640 1 -- 172.21.15.119:0/960398912 >> v1:172.21.15.132:6789/0 conn(0x7f4b6015db60 legacy=0x7f4b600d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.344+0000 7f4b5d400640 1 -- 172.21.15.119:0/960398912 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6015e950 con 0x7f4b600d3f80 2024-04-25T10:15:36.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.344+0000 7f4b5d400640 1 -- 172.21.15.119:0/960398912 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f4b48001810 con 0x7f4b600d3f80 2024-04-25T10:15:36.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.344+0000 7f4b5d400640 1 -- 172.21.15.119:0/960398912 >> v1:172.21.15.119:6789/0 conn(0x7f4b600d3f80 legacy=0x7f4b600d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.344+0000 7f4b5d400640 1 --2- 172.21.15.119:0/960398912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b44003a50 0x7f4b44003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.344+0000 7f4b5d400640 1 --2- 172.21.15.119:0/960398912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b44004800 0x7f4b44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.344+0000 7f4b5d400640 1 --2- 172.21.15.119:0/960398912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b600b9270 0x7f4b440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.345+0000 7f4b5d400640 1 -- 172.21.15.119:0/960398912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6015e670 con 0x7f4b44003a50 2024-04-25T10:15:36.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.345+0000 7f4b5d400640 1 -- 172.21.15.119:0/960398912 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4400baf0 con 0x7f4b44004800 2024-04-25T10:15:36.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.345+0000 7f4b5d400640 1 -- 172.21.15.119:0/960398912 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4400bcf0 con 0x7f4b600b9270 2024-04-25T10:15:36.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.345+0000 7f4b5de00640 1 --2- 172.21.15.119:0/960398912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b44003a50 0x7f4b44003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.345+0000 7f4b5de00640 1 -- 172.21.15.119:0/960398912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b600b9270 msgr2=0x7f4b440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:36.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.345+0000 7f4b5de00640 1 --2- 172.21.15.119:0/960398912 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b600b9270 0x7f4b440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.345+0000 7f4b5de00640 1 -- 172.21.15.119:0/960398912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b44004800 msgr2=0x7f4b44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:36.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.345+0000 7f4b5de00640 1 --2- 172.21.15.119:0/960398912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b44004800 0x7f4b44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.345+0000 7f4b5de00640 1 -- 172.21.15.119:0/960398912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b44003520 con 0x7f4b44003a50 2024-04-25T10:15:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.346+0000 7f4b5de00640 1 --2- 172.21.15.119:0/960398912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b44003a50 0x7f4b44003e70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4b50001750 tx=0x7f4b44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f196dac00818fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.346+0000 7f4b5d400640 1 -- 172.21.15.119:0/960398912 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50001370 con 0x7f4b44003a50 2024-04-25T10:15:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.346+0000 7f4b5d400640 1 -- 172.21.15.119:0/960398912 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b500052d0 con 0x7f4b44003a50 2024-04-25T10:15:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.346+0000 7f4b5d400640 1 -- 172.21.15.119:0/960398912 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5002abe0 con 0x7f4b44003a50 2024-04-25T10:15:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.347+0000 7f4b5f200640 1 -- 172.21.15.119:0/960398912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b44003a50 msgr2=0x7f4b44003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.347+0000 7f4b5f200640 1 --2- 172.21.15.119:0/960398912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b44003a50 0x7f4b44003e70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4b50001750 tx=0x7f4b44005830 comp rx=0 tx=0).stop 2024-04-25T10:15:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.347+0000 7f4b5e800640 1 -- 172.21.15.119:0/960398912 reap_dead start 2024-04-25T10:15:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.347+0000 7f4b5f200640 1 -- 172.21.15.119:0/960398912 shutdown_connections 2024-04-25T10:15:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.347+0000 7f4b5f200640 1 -- 172.21.15.119:0/960398912 >> 172.21.15.119:0/960398912 conn(0x7f4b600726c0 msgr2=0x7f4b60072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.347+0000 7f4b5f200640 1 -- 172.21.15.119:0/960398912 shutdown_connections 2024-04-25T10:15:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.347+0000 7f4b5f200640 1 -- 172.21.15.119:0/960398912 wait complete. 2024-04-25T10:15:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.349+0000 7f4b5f200640 1 Processor -- start 2024-04-25T10:15:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.349+0000 7f4b5f200640 1 -- start start 2024-04-25T10:15:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.350+0000 7f4b5f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b44003a50 0x7f4b60158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.350+0000 7f4b5f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b600c8370 0x7f4b600c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.350+0000 7f4b5f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b600c9450 0x7f4b60160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.350+0000 7f4b5f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b600b3120 con 0x7f4b600c9450 2024-04-25T10:15:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.350+0000 7f4b5f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b600b2fa0 con 0x7f4b44003a50 2024-04-25T10:15:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.350+0000 7f4b5f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6015df40 con 0x7f4b600c8370 2024-04-25T10:15:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.350+0000 7f4b5e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b600c9450 0x7f4b60160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.350+0000 7f4b5e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b600c9450 0x7f4b60160150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43688/0 (socket says 172.21.15.119:43688) 2024-04-25T10:15:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.350+0000 7f4b5e800640 1 -- 172.21.15.119:0/1726335116 learned_addr learned my addr 172.21.15.119:0/1726335116 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.351+0000 7f4b5de00640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b44003a50 0x7f4b60158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.351+0000 7f4b57e00640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b600c8370 0x7f4b600c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.351+0000 7f4b5e800640 1 -- 172.21.15.119:0/1726335116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b600c8370 msgr2=0x7f4b600c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.351+0000 7f4b5e800640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b600c8370 0x7f4b600c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.351+0000 7f4b5e800640 1 -- 172.21.15.119:0/1726335116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b44003a50 msgr2=0x7f4b60158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.351+0000 7f4b5e800640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b44003a50 0x7f4b60158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.351+0000 7f4b5e800640 1 -- 172.21.15.119:0/1726335116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b600c75b0 con 0x7f4b600c9450 2024-04-25T10:15:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.351+0000 7f4b57e00640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b600c8370 0x7f4b600c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.351+0000 7f4b5de00640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b44003a50 0x7f4b60158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.351+0000 7f4b5e800640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b600c9450 0x7f4b60160150 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f4b600d13b0 tx=0x7f4b5800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.351+0000 7f4b56000640 1 -- 172.21.15.119:0/1726335116 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800bb10 con 0x7f4b600c9450 2024-04-25T10:15:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.351+0000 7f4b56000640 1 -- 172.21.15.119:0/1726335116 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5801a070 con 0x7f4b600c9450 2024-04-25T10:15:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.351+0000 7f4b5f200640 1 -- 172.21.15.119:0/1726335116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b600c7890 con 0x7f4b600c9450 2024-04-25T10:15:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.351+0000 7f4b56000640 1 -- 172.21.15.119:0/1726335116 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58014400 con 0x7f4b600c9450 2024-04-25T10:15:36.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.352+0000 7f4b5f200640 1 -- 172.21.15.119:0/1726335116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b600c7e50 con 0x7f4b600c9450 2024-04-25T10:15:36.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.353+0000 7f4b5f200640 1 -- 172.21.15.119:0/1726335116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b600749b0 con 0x7f4b600c9450 2024-04-25T10:15:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.354+0000 7f4b56000640 1 -- 172.21.15.119:0/1726335116 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b58005d10 con 0x7f4b600c9450 2024-04-25T10:15:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.355+0000 7f4b56000640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b3c06b4d0 0x7f4b3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.355+0000 7f4b56000640 1 -- 172.21.15.119:0/1726335116 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f4b5808a4f0 con 0x7f4b600c9450 2024-04-25T10:15:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.355+0000 7f4b5de00640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b3c06b4d0 0x7f4b3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.356+0000 7f4b5de00640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b3c06b4d0 0x7f4b3c06d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4b50032000 tx=0x7f4b5002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:36.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.360+0000 7f4b56000640 1 -- 172.21.15.119:0/1726335116 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b58024020 con 0x7f4b600c9450 2024-04-25T10:15:36.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.592+0000 7f4b5f200640 1 -- 172.21.15.119:0/1726335116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b600c8020 con 0x7f4b600c9450 2024-04-25T10:15:36.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.592+0000 7f4b56000640 1 -- 172.21.15.119:0/1726335116 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f4b58059200 con 0x7f4b600c9450 2024-04-25T10:15:36.596 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:36.597 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:14:54.775138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7668":{"gid":7668,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/2043821095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2043821095},{"type":"v1","addr":"172.21.15.119:6837","nonce":2043821095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:15:36.597 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 74 2024-04-25T10:15:36.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.595+0000 7f4b5f200640 1 -- 172.21.15.119:0/1726335116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b3c06b4d0 msgr2=0x7f4b3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.595+0000 7f4b5f200640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b3c06b4d0 0x7f4b3c06d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4b50032000 tx=0x7f4b5002e000 comp rx=0 tx=0).stop 2024-04-25T10:15:36.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.595+0000 7f4b5f200640 1 -- 172.21.15.119:0/1726335116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b600c9450 msgr2=0x7f4b60160150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.595+0000 7f4b5f200640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b600c9450 0x7f4b60160150 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f4b600d13b0 tx=0x7f4b5800b480 comp rx=0 tx=0).stop 2024-04-25T10:15:36.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.595+0000 7f4b5f200640 1 -- 172.21.15.119:0/1726335116 shutdown_connections 2024-04-25T10:15:36.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.595+0000 7f4b5f200640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b3c06b4d0 0x7f4b3c06d990 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.595+0000 7f4b5f200640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b600c9450 0x7f4b60160150 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.595+0000 7f4b5f200640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b600c8370 0x7f4b600c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.595+0000 7f4b5f200640 1 --2- 172.21.15.119:0/1726335116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b44003a50 0x7f4b60158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.595+0000 7f4b5f200640 1 -- 172.21.15.119:0/1726335116 >> 172.21.15.119:0/1726335116 conn(0x7f4b600726c0 msgr2=0x7f4b600b6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:36.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.595+0000 7f4b5f200640 1 -- 172.21.15.119:0/1726335116 shutdown_connections 2024-04-25T10:15:36.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.595+0000 7f4b5f200640 1 -- 172.21.15.119:0/1726335116 wait complete. 2024-04-25T10:15:36.618 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:36.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.816+0000 7ff4ef800640 1 Processor -- start 2024-04-25T10:15:36.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.817+0000 7ff4ef800640 1 -- start start 2024-04-25T10:15:36.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.818+0000 7ff4ef800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff4e815bf00 0x7ff4e80cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.818+0000 7ff4ef800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff4e806e8f0 0x7ff4e81654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.818+0000 7ff4ef800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e80ce6c0 con 0x7ff4e80cc310 2024-04-25T10:15:36.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.818+0000 7ff4ef800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e81455e0 con 0x7ff4e815bf00 2024-04-25T10:15:36.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.818+0000 7ff4ef800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e806d450 con 0x7ff4e806e8f0 2024-04-25T10:15:36.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.819+0000 7ff4ece00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff4e815bf00 0x7ff4e80cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.819+0000 7ff4ece00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff4e815bf00 0x7ff4e80cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43696/0 (socket says 172.21.15.119:43696) 2024-04-25T10:15:36.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.819+0000 7ff4ece00640 1 -- 172.21.15.119:0/1987145383 learned_addr learned my addr 172.21.15.119:0/1987145383 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:36.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.819+0000 7ff4ed800640 1 --2- 172.21.15.119:0/1987145383 >> v2:172.21.15.132:3300/0 conn(0x7ff4e806e8f0 0x7ff4e81654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.819+0000 7ff4ece00640 1 -- 172.21.15.119:0/1987145383 >> v2:172.21.15.132:3300/0 conn(0x7ff4e806e8f0 msgr2=0x7ff4e81654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:36.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.819+0000 7ff4ece00640 1 --2- 172.21.15.119:0/1987145383 >> v2:172.21.15.132:3300/0 conn(0x7ff4e806e8f0 0x7ff4e81654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.819+0000 7ff4ece00640 1 -- 172.21.15.119:0/1987145383 >> v1:172.21.15.132:6789/0 conn(0x7ff4e80cc310 legacy=0x7ff4e815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:36.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.819+0000 7ff4ece00640 1 -- 172.21.15.119:0/1987145383 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e80cc7b0 con 0x7ff4e815bf00 2024-04-25T10:15:36.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.821+0000 7ff4ece00640 1 --2- 172.21.15.119:0/1987145383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4e815bf00 0x7ff4e80cbdd0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80cdf50 tx=0x7ff4d802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23252e3aaf67f3c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:36.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.821+0000 7ff4e7400640 1 -- 172.21.15.119:0/1987145383 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8036070 con 0x7ff4e815bf00 2024-04-25T10:15:36.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.821+0000 7ff4e7400640 1 -- 172.21.15.119:0/1987145383 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d8031600 con 0x7ff4e815bf00 2024-04-25T10:15:36.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.821+0000 7ff4e7400640 1 -- 172.21.15.119:0/1987145383 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d80319d0 con 0x7ff4e815bf00 2024-04-25T10:15:36.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.821+0000 7ff4ef800640 1 -- 172.21.15.119:0/1987145383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4e815bf00 msgr2=0x7ff4e80cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.821+0000 7ff4ef800640 1 --2- 172.21.15.119:0/1987145383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4e815bf00 0x7ff4e80cbdd0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80cdf50 tx=0x7ff4d802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.822+0000 7ff4ef800640 1 -- 172.21.15.119:0/1987145383 shutdown_connections 2024-04-25T10:15:36.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.822+0000 7ff4ef800640 1 --2- 172.21.15.119:0/1987145383 >> v2:172.21.15.132:3300/0 conn(0x7ff4e806e8f0 0x7ff4e81654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.822+0000 7ff4ef800640 1 --2- 172.21.15.119:0/1987145383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4e815bf00 0x7ff4e80cbdd0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.822+0000 7ff4ef800640 1 -- 172.21.15.119:0/1987145383 >> 172.21.15.119:0/1987145383 conn(0x7ff4e806aba0 msgr2=0x7ff4e806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:36.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.822+0000 7ff4ef800640 1 -- 172.21.15.119:0/1987145383 shutdown_connections 2024-04-25T10:15:36.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.822+0000 7ff4ef800640 1 -- 172.21.15.119:0/1987145383 wait complete. 2024-04-25T10:15:36.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.824+0000 7ff4ef800640 1 Processor -- start 2024-04-25T10:15:36.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.824+0000 7ff4ef800640 1 -- start start 2024-04-25T10:15:36.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.825+0000 7ff4ef800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4e806e8f0 0x7ff4e81e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.825+0000 7ff4ef800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4e80cc310 0x7ff4e81f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.825+0000 7ff4ef800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4e815bf00 0x7ff4e81f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.825+0000 7ff4ef800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80cd050 con 0x7ff4e815bf00 2024-04-25T10:15:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.825+0000 7ff4ef800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80ccd70 con 0x7ff4e806e8f0 2024-04-25T10:15:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.825+0000 7ff4ef800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e806ee60 con 0x7ff4e80cc310 2024-04-25T10:15:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.825+0000 7ff4ed800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4e815bf00 0x7ff4e81f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.825+0000 7ff4ed800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4e815bf00 0x7ff4e81f3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43702/0 (socket says 172.21.15.119:43702) 2024-04-25T10:15:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.825+0000 7ff4ed800640 1 -- 172.21.15.119:0/3847460048 learned_addr learned my addr 172.21.15.119:0/3847460048 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.825+0000 7ff4ece00640 1 --2- 172.21.15.119:0/3847460048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4e806e8f0 0x7ff4e81e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.825+0000 7ff4e7e00640 1 --2- 172.21.15.119:0/3847460048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4e80cc310 0x7ff4e81f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.825+0000 7ff4ed800640 1 -- 172.21.15.119:0/3847460048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4e80cc310 msgr2=0x7ff4e81f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.826+0000 7ff4ed800640 1 --2- 172.21.15.119:0/3847460048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4e80cc310 0x7ff4e81f3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.826+0000 7ff4ed800640 1 -- 172.21.15.119:0/3847460048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4e806e8f0 msgr2=0x7ff4e81e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.826+0000 7ff4ed800640 1 --2- 172.21.15.119:0/3847460048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4e806e8f0 0x7ff4e81e4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.826+0000 7ff4ed800640 1 -- 172.21.15.119:0/3847460048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e81e53e0 con 0x7ff4e815bf00 2024-04-25T10:15:36.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.826+0000 7ff4ed800640 1 --2- 172.21.15.119:0/3847460048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4e815bf00 0x7ff4e81f3c90 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000d800 tx=0x7ff4e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.826+0000 7ff4e4c00640 1 -- 172.21.15.119:0/3847460048 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e000b6c0 con 0x7ff4e815bf00 2024-04-25T10:15:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.826+0000 7ff4e4c00640 1 -- 172.21.15.119:0/3847460048 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e0019070 con 0x7ff4e815bf00 2024-04-25T10:15:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.826+0000 7ff4e4c00640 1 -- 172.21.15.119:0/3847460048 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e000bac0 con 0x7ff4e815bf00 2024-04-25T10:15:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.826+0000 7ff4ef800640 1 -- 172.21.15.119:0/3847460048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e81e56c0 con 0x7ff4e815bf00 2024-04-25T10:15:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.826+0000 7ff4ef800640 1 -- 172.21.15.119:0/3847460048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e81e5c80 con 0x7ff4e815bf00 2024-04-25T10:15:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.827+0000 7ff4ef800640 1 -- 172.21.15.119:0/3847460048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e806bb30 con 0x7ff4e815bf00 2024-04-25T10:15:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.828+0000 7ff4e4c00640 1 -- 172.21.15.119:0/3847460048 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4e000bc60 con 0x7ff4e815bf00 2024-04-25T10:15:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.829+0000 7ff4e4c00640 1 --2- 172.21.15.119:0/3847460048 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff4c406b4d0 0x7ff4c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.830+0000 7ff4e4c00640 1 -- 172.21.15.119:0/3847460048 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7ff4e008a750 con 0x7ff4e815bf00 2024-04-25T10:15:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.830+0000 7ff4ece00640 1 --2- 172.21.15.119:0/3847460048 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff4c406b4d0 0x7ff4c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.830+0000 7ff4ece00640 1 --2- 172.21.15.119:0/3847460048 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff4c406b4d0 0x7ff4c406d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8033000 tx=0x7ff4d8009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:36.834+0000 7ff4e4c00640 1 -- 172.21.15.119:0/3847460048 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4e00594f0 con 0x7ff4e815bf00 2024-04-25T10:15:37.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.031+0000 7ff4ef800640 1 -- 172.21.15.119:0/3847460048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4e80b3630 con 0x7ff4e815bf00 2024-04-25T10:15:37.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.034+0000 7ff4e4c00640 1 -- 172.21.15.119:0/3847460048 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17009 (secure 0 0 0) 0x7ff4e0058df0 con 0x7ff4e815bf00 2024-04-25T10:15:37.038 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:37.038 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":85,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:14:50.735243+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:21.210576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:22.438674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/1734507541":"2024-04-26T10:14:50.735180+0000","172.21.15.119:6835/1734507541":"2024-04-26T10:14:50.735180+0000","172.21.15.119:6837/3997964942":"2024-04-26T10:13:36.160373+0000","172.21.15.119:6835/3997964942":"2024-04-26T10:13:36.160373+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:37.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.037+0000 7ff4ef800640 1 -- 172.21.15.119:0/3847460048 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff4c406b4d0 msgr2=0x7ff4c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.037+0000 7ff4ef800640 1 --2- 172.21.15.119:0/3847460048 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff4c406b4d0 0x7ff4c406d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8033000 tx=0x7ff4d8009000 comp rx=0 tx=0).stop 2024-04-25T10:15:37.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.037+0000 7ff4ef800640 1 -- 172.21.15.119:0/3847460048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4e815bf00 msgr2=0x7ff4e81f3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.037+0000 7ff4ef800640 1 --2- 172.21.15.119:0/3847460048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4e815bf00 0x7ff4e81f3c90 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000d800 tx=0x7ff4e000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.037+0000 7ff4ef800640 1 -- 172.21.15.119:0/3847460048 shutdown_connections 2024-04-25T10:15:37.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.037+0000 7ff4ef800640 1 --2- 172.21.15.119:0/3847460048 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff4c406b4d0 0x7ff4c406d990 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.037+0000 7ff4ef800640 1 --2- 172.21.15.119:0/3847460048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4e815bf00 0x7ff4e81f3c90 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.037+0000 7ff4ef800640 1 --2- 172.21.15.119:0/3847460048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff4e80cc310 0x7ff4e81f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.037+0000 7ff4ef800640 1 --2- 172.21.15.119:0/3847460048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4e806e8f0 0x7ff4e81e4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.037+0000 7ff4ef800640 1 -- 172.21.15.119:0/3847460048 >> 172.21.15.119:0/3847460048 conn(0x7ff4e806aba0 msgr2=0x7ff4e80b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.037+0000 7ff4ef800640 1 -- 172.21.15.119:0/3847460048 shutdown_connections 2024-04-25T10:15:37.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.037+0000 7ff4ef800640 1 -- 172.21.15.119:0/3847460048 wait complete. 2024-04-25T10:15:37.061 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:15:37.062 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.257+0000 7fdc07200640 1 Processor -- start 2024-04-25T10:15:37.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.257+0000 7fdc07200640 1 -- start start 2024-04-25T10:15:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.259+0000 7fdc07200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdc0015c9f0 0x7fdc001664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.259+0000 7fdc07200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc000ce6c0 con 0x7fdc0006e8f0 2024-04-25T10:15:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.259+0000 7fdc07200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc000cc4e0 con 0x7fdc0015bf00 2024-04-25T10:15:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.259+0000 7fdc07200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc001455e0 con 0x7fdc0015c9f0 2024-04-25T10:15:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.259+0000 7fdbff400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdc0006e8f0 0x7fdc0015c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50100/0 (socket says 172.21.15.119:50100) 2024-04-25T10:15:37.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.259+0000 7fdbff400640 1 -- 172.21.15.119:0/3933487329 learned_addr learned my addr 172.21.15.119:0/3933487329 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.259+0000 7fdc05200640 1 --2- 172.21.15.119:0/3933487329 >> v2:172.21.15.132:3300/0 conn(0x7fdc0015c9f0 0x7fdc001664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.259+0000 7fdbfea00640 1 -- 172.21.15.119:0/3933487329 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295185664 0 0) 0x7fdbec001110 con 0x7fdc0006e8f0 2024-04-25T10:15:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.260+0000 7fdbfea00640 1 -- 172.21.15.119:0/3933487329 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbe4003520 con 0x7fdc0006e8f0 2024-04-25T10:15:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.260+0000 7fdbfea00640 1 -- 172.21.15.119:0/3933487329 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 769677382 0 0) 0x7fdbe4003520 con 0x7fdc0006e8f0 2024-04-25T10:15:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.260+0000 7fdbfea00640 1 -- 172.21.15.119:0/3933487329 >> v2:172.21.15.132:3300/0 conn(0x7fdc0015c9f0 msgr2=0x7fdc001664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.260+0000 7fdbfea00640 1 --2- 172.21.15.119:0/3933487329 >> v2:172.21.15.132:3300/0 conn(0x7fdc0015c9f0 0x7fdc001664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.260+0000 7fdbfea00640 1 -- 172.21.15.119:0/3933487329 >> v1:172.21.15.132:6789/0 conn(0x7fdc0015bf00 legacy=0x7fdc000cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.260+0000 7fdbfea00640 1 -- 172.21.15.119:0/3933487329 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc000cc7b0 con 0x7fdc0006e8f0 2024-04-25T10:15:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.260+0000 7fdbfea00640 1 -- 172.21.15.119:0/3933487329 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fdbec001840 con 0x7fdc0006e8f0 2024-04-25T10:15:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.260+0000 7fdbfea00640 1 -- 172.21.15.119:0/3933487329 >> v1:172.21.15.119:6789/0 conn(0x7fdc0006e8f0 legacy=0x7fdc0015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.260+0000 7fdbfea00640 1 --2- 172.21.15.119:0/3933487329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe40039b0 0x7fdbe4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.260+0000 7fdbfea00640 1 --2- 172.21.15.119:0/3933487329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdbe4004800 0x7fdbe4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.260+0000 7fdbfea00640 1 --2- 172.21.15.119:0/3933487329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe40051c0 0x7fdbe400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.260+0000 7fdbfea00640 1 -- 172.21.15.119:0/3933487329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc001455e0 con 0x7fdbe40051c0 2024-04-25T10:15:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.260+0000 7fdbfea00640 1 -- 172.21.15.119:0/3933487329 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe400c060 con 0x7fdbe40039b0 2024-04-25T10:15:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.260+0000 7fdbfea00640 1 -- 172.21.15.119:0/3933487329 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe400c260 con 0x7fdbe4004800 2024-04-25T10:15:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.261+0000 7fdc05200640 1 --2- 172.21.15.119:0/3933487329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe40051c0 0x7fdbe400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.261+0000 7fdbffe00640 1 --2- 172.21.15.119:0/3933487329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe40039b0 0x7fdbe4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.261+0000 7fdbff400640 1 --2- 172.21.15.119:0/3933487329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdbe4004800 0x7fdbe4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.261+0000 7fdc05200640 1 -- 172.21.15.119:0/3933487329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdbe4004800 msgr2=0x7fdbe4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.261+0000 7fdc05200640 1 --2- 172.21.15.119:0/3933487329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdbe4004800 0x7fdbe4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.261+0000 7fdc05200640 1 -- 172.21.15.119:0/3933487329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe40039b0 msgr2=0x7fdbe4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.261+0000 7fdc05200640 1 --2- 172.21.15.119:0/3933487329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe40039b0 0x7fdbe4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.261+0000 7fdc05200640 1 -- 172.21.15.119:0/3933487329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe4003520 con 0x7fdbe40051c0 2024-04-25T10:15:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.262+0000 7fdc05200640 1 -- 172.21.15.119:0/3933487329 reap_dead start 2024-04-25T10:15:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.262+0000 7fdc05200640 1 --2- 172.21.15.119:0/3933487329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe40051c0 0x7fdbe400b9c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009fc0 tx=0x7fdbe4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17491e98a5cb7cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.263+0000 7fdbfea00640 1 -- 172.21.15.119:0/3933487329 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8035020 con 0x7fdbe40051c0 2024-04-25T10:15:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.263+0000 7fdbfea00640 1 -- 172.21.15.119:0/3933487329 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf8003b90 con 0x7fdbe40051c0 2024-04-25T10:15:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.263+0000 7fdbfea00640 1 -- 172.21.15.119:0/3933487329 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8009c10 con 0x7fdbe40051c0 2024-04-25T10:15:37.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.263+0000 7fdc07200640 1 -- 172.21.15.119:0/3933487329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe40051c0 msgr2=0x7fdbe400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.263+0000 7fdc07200640 1 --2- 172.21.15.119:0/3933487329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe40051c0 0x7fdbe400b9c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009fc0 tx=0x7fdbe4005830 comp rx=0 tx=0).stop 2024-04-25T10:15:37.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.263+0000 7fdc07200640 1 -- 172.21.15.119:0/3933487329 shutdown_connections 2024-04-25T10:15:37.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.263+0000 7fdc07200640 1 --2- 172.21.15.119:0/3933487329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe40051c0 0x7fdbe400b9c0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.263+0000 7fdc07200640 1 -- 172.21.15.119:0/3933487329 >> 172.21.15.119:0/3933487329 conn(0x7fdc0006aba0 msgr2=0x7fdc0006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.264+0000 7fdc07200640 1 -- 172.21.15.119:0/3933487329 shutdown_connections 2024-04-25T10:15:37.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.264+0000 7fdc07200640 1 -- 172.21.15.119:0/3933487329 wait complete. 2024-04-25T10:15:37.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.265+0000 7fdc07200640 1 Processor -- start 2024-04-25T10:15:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.265+0000 7fdc07200640 1 -- start start 2024-04-25T10:15:37.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.266+0000 7fdc07200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe40051c0 0x7fdc00143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.266+0000 7fdc07200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc0015b430 0x7fdc00131b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.266+0000 7fdc07200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc000cf3f0 0x7fdc000cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.266+0000 7fdc07200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000ccd70 con 0x7fdc000cf3f0 2024-04-25T10:15:37.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.266+0000 7fdc07200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0006d450 con 0x7fdbe40051c0 2024-04-25T10:15:37.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.266+0000 7fdc07200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000cd050 con 0x7fdc0015b430 2024-04-25T10:15:37.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.267+0000 7fdc05200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc000cf3f0 0x7fdc000cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.267+0000 7fdc05200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc000cf3f0 0x7fdc000cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43710/0 (socket says 172.21.15.119:43710) 2024-04-25T10:15:37.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.267+0000 7fdc05200640 1 -- 172.21.15.119:0/1099214846 learned_addr learned my addr 172.21.15.119:0/1099214846 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:37.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.267+0000 7fdbff400640 1 --2- 172.21.15.119:0/1099214846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc0015b430 0x7fdc00131b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.267+0000 7fdc05200640 1 -- 172.21.15.119:0/1099214846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc0015b430 msgr2=0x7fdc00131b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.267+0000 7fdc05200640 1 --2- 172.21.15.119:0/1099214846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc0015b430 0x7fdc00131b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.267+0000 7fdc05200640 1 -- 172.21.15.119:0/1099214846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe40051c0 msgr2=0x7fdc00143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.267+0000 7fdc05200640 1 --2- 172.21.15.119:0/1099214846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe40051c0 0x7fdc00143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.267+0000 7fdc05200640 1 -- 172.21.15.119:0/1099214846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc001320b0 con 0x7fdc000cf3f0 2024-04-25T10:15:37.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.267+0000 7fdc05200640 1 --2- 172.21.15.119:0/1099214846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc000cf3f0 0x7fdc000cf8a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fdbf802af10 tx=0x7fdbf8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.268+0000 7fdbf7e00640 1 -- 172.21.15.119:0/1099214846 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf803f030 con 0x7fdc000cf3f0 2024-04-25T10:15:37.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.268+0000 7fdbf7e00640 1 -- 172.21.15.119:0/1099214846 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf8003990 con 0x7fdc000cf3f0 2024-04-25T10:15:37.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.268+0000 7fdbf7e00640 1 -- 172.21.15.119:0/1099214846 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8035070 con 0x7fdc000cf3f0 2024-04-25T10:15:37.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.268+0000 7fdc07200640 1 -- 172.21.15.119:0/1099214846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc00132380 con 0x7fdc000cf3f0 2024-04-25T10:15:37.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.268+0000 7fdc07200640 1 -- 172.21.15.119:0/1099214846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc00132940 con 0x7fdc000cf3f0 2024-04-25T10:15:37.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.269+0000 7fdc07200640 1 -- 172.21.15.119:0/1099214846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0006bb30 con 0x7fdc000cf3f0 2024-04-25T10:15:37.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.270+0000 7fdbf7e00640 1 -- 172.21.15.119:0/1099214846 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdbf8042030 con 0x7fdc000cf3f0 2024-04-25T10:15:37.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.271+0000 7fdbf7e00640 1 --2- 172.21.15.119:0/1099214846 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdbe006b500 0x7fdbe006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.271+0000 7fdbf7e00640 1 -- 172.21.15.119:0/1099214846 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fdbf80aabc0 con 0x7fdc000cf3f0 2024-04-25T10:15:37.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.271+0000 7fdbffe00640 1 --2- 172.21.15.119:0/1099214846 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdbe006b500 0x7fdbe006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.272+0000 7fdbffe00640 1 --2- 172.21.15.119:0/1099214846 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdbe006b500 0x7fdbe006d9c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00014f0 tx=0x7fdbf0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.277+0000 7fdbf7e00640 1 -- 172.21.15.119:0/1099214846 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdbf8079280 con 0x7fdc000cf3f0 2024-04-25T10:15:37.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.515+0000 7fdc07200640 1 -- 172.21.15.119:0/1099214846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc0006e710 con 0x7fdc000cf3f0 2024-04-25T10:15:37.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.516+0000 7fdbf7e00640 1 -- 172.21.15.119:0/1099214846 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4953 (secure 0 0 0) 0x7fdc0006e710 con 0x7fdc000cf3f0 2024-04-25T10:15:37.521 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:37.521 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:14:54.775138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7668":{"gid":7668,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/2043821095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2043821095},{"type":"v1","addr":"172.21.15.119:6837","nonce":2043821095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:15:37.521 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 74 2024-04-25T10:15:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.519+0000 7fdc07200640 1 -- 172.21.15.119:0/1099214846 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdbe006b500 msgr2=0x7fdbe006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.519+0000 7fdc07200640 1 --2- 172.21.15.119:0/1099214846 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdbe006b500 0x7fdbe006d9c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00014f0 tx=0x7fdbf0009040 comp rx=0 tx=0).stop 2024-04-25T10:15:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.519+0000 7fdc07200640 1 -- 172.21.15.119:0/1099214846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc000cf3f0 msgr2=0x7fdc000cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.519+0000 7fdc07200640 1 --2- 172.21.15.119:0/1099214846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc000cf3f0 0x7fdc000cf8a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fdbf802af10 tx=0x7fdbf8009750 comp rx=0 tx=0).stop 2024-04-25T10:15:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.519+0000 7fdc07200640 1 -- 172.21.15.119:0/1099214846 shutdown_connections 2024-04-25T10:15:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.519+0000 7fdc07200640 1 --2- 172.21.15.119:0/1099214846 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdbe006b500 0x7fdbe006d9c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.519+0000 7fdc07200640 1 --2- 172.21.15.119:0/1099214846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc000cf3f0 0x7fdc000cf8a0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.519+0000 7fdc07200640 1 --2- 172.21.15.119:0/1099214846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc0015b430 0x7fdc00131b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.519+0000 7fdc07200640 1 --2- 172.21.15.119:0/1099214846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe40051c0 0x7fdc00143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.519+0000 7fdc07200640 1 -- 172.21.15.119:0/1099214846 >> 172.21.15.119:0/1099214846 conn(0x7fdc0006aba0 msgr2=0x7fdc0015e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.519+0000 7fdc07200640 1 -- 172.21.15.119:0/1099214846 shutdown_connections 2024-04-25T10:15:37.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.519+0000 7fdc07200640 1 -- 172.21.15.119:0/1099214846 wait complete. 2024-04-25T10:15:37.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:37.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.725+0000 7f0557a00640 1 Processor -- start 2024-04-25T10:15:37.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.726+0000 7f0557a00640 1 -- start start 2024-04-25T10:15:37.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.727+0000 7f0557a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0550162360 0x7f05500cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.727+0000 7f0557a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0550161970 con 0x7f05500cdf60 2024-04-25T10:15:37.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.727+0000 7f0557a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055006b780 con 0x7f055006b2b0 2024-04-25T10:15:37.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.727+0000 7f0557a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05500ce3c0 con 0x7f0550162360 2024-04-25T10:15:37.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.728+0000 7f0547e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f05500cdf60 0x7f055006aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50106/0 (socket says 172.21.15.119:50106) 2024-04-25T10:15:37.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.728+0000 7f0547e00640 1 -- 172.21.15.119:0/966487427 learned_addr learned my addr 172.21.15.119:0/966487427 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:37.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.728+0000 7f0555000640 1 --2- 172.21.15.119:0/966487427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0550162360 0x7f05500cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.728+0000 7f054fe00640 1 -- 172.21.15.119:0/966487427 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877253241 0 0) 0x7f0538001110 con 0x7f05500cdf60 2024-04-25T10:15:37.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.728+0000 7f054fe00640 1 -- 172.21.15.119:0/966487427 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0534003520 con 0x7f05500cdf60 2024-04-25T10:15:37.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.728+0000 7f054fe00640 1 -- 172.21.15.119:0/966487427 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 951954138 0 0) 0x7f0534003520 con 0x7f05500cdf60 2024-04-25T10:15:37.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.728+0000 7f054fe00640 1 -- 172.21.15.119:0/966487427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0550162360 msgr2=0x7f05500cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.728+0000 7f054fe00640 1 --2- 172.21.15.119:0/966487427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0550162360 0x7f05500cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.728+0000 7f054fe00640 1 -- 172.21.15.119:0/966487427 >> v1:172.21.15.132:6789/0 conn(0x7f055006b2b0 legacy=0x7f055006dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.728+0000 7f054fe00640 1 -- 172.21.15.119:0/966487427 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055006ba60 con 0x7f05500cdf60 2024-04-25T10:15:37.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.728+0000 7f054fe00640 1 -- 172.21.15.119:0/966487427 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f0538001840 con 0x7f05500cdf60 2024-04-25T10:15:37.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.728+0000 7f054fe00640 1 -- 172.21.15.119:0/966487427 >> v1:172.21.15.119:6789/0 conn(0x7f05500cdf60 legacy=0x7f055006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.729+0000 7f054fe00640 1 --2- 172.21.15.119:0/966487427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05340039e0 0x7f0534003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.729+0000 7f054fe00640 1 --2- 172.21.15.119:0/966487427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004800 0x7f0534004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.729+0000 7f054fe00640 1 --2- 172.21.15.119:0/966487427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05340051c0 0x7f053400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.729+0000 7f054fe00640 1 -- 172.21.15.119:0/966487427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053400bf00 con 0x7f0534004800 2024-04-25T10:15:37.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.729+0000 7f054fe00640 1 -- 172.21.15.119:0/966487427 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053400c080 con 0x7f05340039e0 2024-04-25T10:15:37.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.729+0000 7f054fe00640 1 -- 172.21.15.119:0/966487427 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053400c200 con 0x7f05340051c0 2024-04-25T10:15:37.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.729+0000 7f0547e00640 1 --2- 172.21.15.119:0/966487427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004800 0x7f0534004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.729+0000 7f0555000640 1 --2- 172.21.15.119:0/966487427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05340039e0 0x7f0534003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.729+0000 7f0547e00640 1 -- 172.21.15.119:0/966487427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05340051c0 msgr2=0x7f053400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.729+0000 7f0547e00640 1 --2- 172.21.15.119:0/966487427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05340051c0 0x7f053400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.729+0000 7f0547e00640 1 -- 172.21.15.119:0/966487427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05340039e0 msgr2=0x7f0534003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:37.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.729+0000 7f0547e00640 1 --2- 172.21.15.119:0/966487427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05340039e0 0x7f0534003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.729+0000 7f0547e00640 1 -- 172.21.15.119:0/966487427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534003520 con 0x7f0534004800 2024-04-25T10:15:37.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.730+0000 7f0547e00640 1 --2- 172.21.15.119:0/966487427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004800 0x7f0534004c80 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f0538007fd0 tx=0x7f0534005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddec095bef0de10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.731+0000 7f054fe00640 1 -- 172.21.15.119:0/966487427 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538007e30 con 0x7f0534004800 2024-04-25T10:15:37.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.731+0000 7f054fe00640 1 -- 172.21.15.119:0/966487427 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0538005af0 con 0x7f0534004800 2024-04-25T10:15:37.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.731+0000 7f054fe00640 1 -- 172.21.15.119:0/966487427 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538007070 con 0x7f0534004800 2024-04-25T10:15:37.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.731+0000 7f0557a00640 1 -- 172.21.15.119:0/966487427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004800 msgr2=0x7f0534004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.731+0000 7f0557a00640 1 --2- 172.21.15.119:0/966487427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004800 0x7f0534004c80 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f0538007fd0 tx=0x7f0534005830 comp rx=0 tx=0).stop 2024-04-25T10:15:37.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.731+0000 7f0555a00640 1 -- 172.21.15.119:0/966487427 reap_dead start 2024-04-25T10:15:37.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.731+0000 7f0557a00640 1 -- 172.21.15.119:0/966487427 shutdown_connections 2024-04-25T10:15:37.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.731+0000 7f0557a00640 1 -- 172.21.15.119:0/966487427 >> 172.21.15.119:0/966487427 conn(0x7f05500cc8d0 msgr2=0x7f05500ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.731+0000 7f0557a00640 1 -- 172.21.15.119:0/966487427 shutdown_connections 2024-04-25T10:15:37.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.731+0000 7f0557a00640 1 -- 172.21.15.119:0/966487427 wait complete. 2024-04-25T10:15:37.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.733+0000 7f0557a00640 1 Processor -- start 2024-04-25T10:15:37.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.733+0000 7f0557a00640 1 -- start start 2024-04-25T10:15:37.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.734+0000 7f0557a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004800 0x7f05500c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.734+0000 7f0557a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05501f39a0 0x7f05501f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.734+0000 7f0557a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f055015c8b0 0x7f055015cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.734+0000 7f0557a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055006c300 con 0x7f0534004800 2024-04-25T10:15:37.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.734+0000 7f0557a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055006c020 con 0x7f05501f39a0 2024-04-25T10:15:37.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.734+0000 7f0557a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05500cda20 con 0x7f055015c8b0 2024-04-25T10:15:37.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.734+0000 7f0555000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004800 0x7f05500c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.735+0000 7f0555000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004800 0x7f05500c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43722/0 (socket says 172.21.15.119:43722) 2024-04-25T10:15:37.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.735+0000 7f0555000640 1 -- 172.21.15.119:0/882296006 learned_addr learned my addr 172.21.15.119:0/882296006 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:37.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.735+0000 7f0555a00640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f055015c8b0 0x7f055015cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.735+0000 7f0547e00640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05501f39a0 0x7f05501f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.735+0000 7f0555000640 1 -- 172.21.15.119:0/882296006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f055015c8b0 msgr2=0x7f055015cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.735+0000 7f0555000640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f055015c8b0 0x7f055015cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.735+0000 7f0555000640 1 -- 172.21.15.119:0/882296006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05501f39a0 msgr2=0x7f05501f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.735+0000 7f0555000640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05501f39a0 0x7f05501f3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.735+0000 7f0555000640 1 -- 172.21.15.119:0/882296006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05501f4500 con 0x7f0534004800 2024-04-25T10:15:37.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.735+0000 7f0555a00640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f055015c8b0 0x7f055015cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:37.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.735+0000 7f0555000640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004800 0x7f05500c60d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0540010890 tx=0x7f0540010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.736+0000 7f054d600640 1 -- 172.21.15.119:0/882296006 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054000f040 con 0x7f0534004800 2024-04-25T10:15:37.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.736+0000 7f054d600640 1 -- 172.21.15.119:0/882296006 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054000eb60 con 0x7f0534004800 2024-04-25T10:15:37.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.736+0000 7f054d600640 1 -- 172.21.15.119:0/882296006 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054001c400 con 0x7f0534004800 2024-04-25T10:15:37.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.736+0000 7f0557a00640 1 -- 172.21.15.119:0/882296006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05501f47e0 con 0x7f0534004800 2024-04-25T10:15:37.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.736+0000 7f0557a00640 1 -- 172.21.15.119:0/882296006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05501f4da0 con 0x7f0534004800 2024-04-25T10:15:37.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.737+0000 7f0557a00640 1 -- 172.21.15.119:0/882296006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05500749b0 con 0x7f0534004800 2024-04-25T10:15:37.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.738+0000 7f054d600640 1 -- 172.21.15.119:0/882296006 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f054001c5a0 con 0x7f0534004800 2024-04-25T10:15:37.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.739+0000 7f054d600640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f052c06b4d0 0x7f052c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:37.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.739+0000 7f054d600640 1 -- 172.21.15.119:0/882296006 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f054008a900 con 0x7f0534004800 2024-04-25T10:15:37.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.739+0000 7f0547e00640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f052c06b4d0 0x7f052c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:37.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.740+0000 7f0547e00640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f052c06b4d0 0x7f052c06d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0538033000 tx=0x7f053802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:37.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.743+0000 7f054d600640 1 -- 172.21.15.119:0/882296006 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0540018030 con 0x7f0534004800 2024-04-25T10:15:37.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.971+0000 7f0557a00640 1 -- 172.21.15.119:0/882296006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0550066a30 con 0x7f0534004800 2024-04-25T10:15:37.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.975+0000 7f054d600640 1 -- 172.21.15.119:0/882296006 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4953 (secure 0 0 0) 0x7f0540059650 con 0x7f0534004800 2024-04-25T10:15:37.980 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:37.980 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:23.427654+0000","modified":"2024-04-25T10:14:54.775138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7668},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7668":{"gid":7668,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/2043821095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2043821095},{"type":"v1","addr":"172.21.15.119:6837","nonce":2043821095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:15:37.980 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 74 2024-04-25T10:15:37.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.978+0000 7f0557a00640 1 -- 172.21.15.119:0/882296006 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f052c06b4d0 msgr2=0x7f052c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.978+0000 7f0557a00640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f052c06b4d0 0x7f052c06d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0538033000 tx=0x7f053802e000 comp rx=0 tx=0).stop 2024-04-25T10:15:37.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.978+0000 7f0557a00640 1 -- 172.21.15.119:0/882296006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004800 msgr2=0x7f05500c60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:37.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.978+0000 7f0557a00640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004800 0x7f05500c60d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0540010890 tx=0x7f0540010d50 comp rx=0 tx=0).stop 2024-04-25T10:15:37.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.978+0000 7f0557a00640 1 -- 172.21.15.119:0/882296006 shutdown_connections 2024-04-25T10:15:37.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.978+0000 7f0557a00640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f052c06b4d0 0x7f052c06d990 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.978+0000 7f0557a00640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f055015c8b0 0x7f055015cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.978+0000 7f0557a00640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05501f39a0 0x7f05501f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.978+0000 7f0557a00640 1 --2- 172.21.15.119:0/882296006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004800 0x7f05500c60d0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:37.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.978+0000 7f0557a00640 1 -- 172.21.15.119:0/882296006 >> 172.21.15.119:0/882296006 conn(0x7f05500cc8d0 msgr2=0x7f055006ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:37.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.978+0000 7f0557a00640 1 -- 172.21.15.119:0/882296006 shutdown_connections 2024-04-25T10:15:37.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:37.978+0000 7f0557a00640 1 -- 172.21.15.119:0/882296006 wait complete. 2024-04-25T10:15:37.999 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:38.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.189+0000 7fca4be00640 1 Processor -- start 2024-04-25T10:15:38.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.189+0000 7fca4be00640 1 -- start start 2024-04-25T10:15:38.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.190+0000 7fca4be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca4406e8f0 0x7fca4415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:38.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.191+0000 7fca4be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca4415c820 0x7fca441654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:38.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.191+0000 7fca4be00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca440ce6c0 con 0x7fca4415bf00 2024-04-25T10:15:38.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.191+0000 7fca4be00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca441455e0 con 0x7fca4406e8f0 2024-04-25T10:15:38.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.191+0000 7fca4be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4406d450 con 0x7fca4415c820 2024-04-25T10:15:38.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.191+0000 7fca48a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca4406e8f0 0x7fca4415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:38.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.191+0000 7fca49e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca4415c820 0x7fca441654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:38.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.191+0000 7fca48a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca4406e8f0 0x7fca4415c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:36966/0 (socket says 172.21.15.119:36966) 2024-04-25T10:15:38.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.191+0000 7fca48a00640 1 -- 172.21.15.119:0/3911892428 learned_addr learned my addr 172.21.15.119:0/3911892428 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:38.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.192+0000 7fca49e00640 1 -- 172.21.15.119:0/3911892428 >> v2:172.21.15.132:3300/0 conn(0x7fca4406e8f0 msgr2=0x7fca4415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:38.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.192+0000 7fca49e00640 1 --2- 172.21.15.119:0/3911892428 >> v2:172.21.15.132:3300/0 conn(0x7fca4406e8f0 0x7fca4415c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.192+0000 7fca49e00640 1 -- 172.21.15.119:0/3911892428 >> v1:172.21.15.132:6789/0 conn(0x7fca4415bf00 legacy=0x7fca440cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:38.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.192+0000 7fca49e00640 1 -- 172.21.15.119:0/3911892428 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca440cc4e0 con 0x7fca4415c820 2024-04-25T10:15:38.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.194+0000 7fca49e00640 1 --2- 172.21.15.119:0/3911892428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca4415c820 0x7fca441654b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009d90 tx=0x7fca3c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd23ea148d63542 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:38.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.194+0000 7fca43e00640 1 -- 172.21.15.119:0/3911892428 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c007740 con 0x7fca4415c820 2024-04-25T10:15:38.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.194+0000 7fca43e00640 1 -- 172.21.15.119:0/3911892428 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c0078e0 con 0x7fca4415c820 2024-04-25T10:15:38.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.194+0000 7fca43e00640 1 -- 172.21.15.119:0/3911892428 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c007cb0 con 0x7fca4415c820 2024-04-25T10:15:38.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.195+0000 7fca4be00640 1 -- 172.21.15.119:0/3911892428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca4415c820 msgr2=0x7fca441654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:38.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.195+0000 7fca4be00640 1 --2- 172.21.15.119:0/3911892428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca4415c820 0x7fca441654b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009d90 tx=0x7fca3c02a660 comp rx=0 tx=0).stop 2024-04-25T10:15:38.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.195+0000 7fca4be00640 1 -- 172.21.15.119:0/3911892428 shutdown_connections 2024-04-25T10:15:38.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.195+0000 7fca4be00640 1 --2- 172.21.15.119:0/3911892428 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca4415c820 0x7fca441654b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.195+0000 7fca4be00640 1 --2- 172.21.15.119:0/3911892428 >> v2:172.21.15.132:3300/0 conn(0x7fca4406e8f0 0x7fca4415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.195+0000 7fca4be00640 1 -- 172.21.15.119:0/3911892428 >> 172.21.15.119:0/3911892428 conn(0x7fca4406aba0 msgr2=0x7fca4406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:38.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.195+0000 7fca4be00640 1 -- 172.21.15.119:0/3911892428 shutdown_connections 2024-04-25T10:15:38.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.195+0000 7fca4be00640 1 -- 172.21.15.119:0/3911892428 wait complete. 2024-04-25T10:15:38.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.197+0000 7fca4be00640 1 Processor -- start 2024-04-25T10:15:38.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.197+0000 7fca4be00640 1 -- start start 2024-04-25T10:15:38.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.198+0000 7fca4be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca4406e8f0 0x7fca44158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:38.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.198+0000 7fca4be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca4415bf00 0x7fca44142410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:38.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.198+0000 7fca4be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca4415c820 0x7fca4413b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:38.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.198+0000 7fca4be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca440ccd70 con 0x7fca4406e8f0 2024-04-25T10:15:38.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.198+0000 7fca4be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca440cca90 con 0x7fca4415c820 2024-04-25T10:15:38.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.198+0000 7fca4be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca440cd050 con 0x7fca4415bf00 2024-04-25T10:15:38.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.198+0000 7fca49400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca4406e8f0 0x7fca44158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:38.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.198+0000 7fca49400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca4406e8f0 0x7fca44158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43728/0 (socket says 172.21.15.119:43728) 2024-04-25T10:15:38.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.198+0000 7fca49400640 1 -- 172.21.15.119:0/2495405132 learned_addr learned my addr 172.21.15.119:0/2495405132 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:38.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.199+0000 7fca48a00640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca4415bf00 0x7fca44142410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:38.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.199+0000 7fca49e00640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca4415c820 0x7fca4413b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:38.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.199+0000 7fca49400640 1 -- 172.21.15.119:0/2495405132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca4415bf00 msgr2=0x7fca44142410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:38.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.199+0000 7fca49400640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca4415bf00 0x7fca44142410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.199+0000 7fca49400640 1 -- 172.21.15.119:0/2495405132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca4415c820 msgr2=0x7fca4413b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:38.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.199+0000 7fca49400640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca4415c820 0x7fca4413b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.199+0000 7fca49400640 1 -- 172.21.15.119:0/2495405132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca44159430 con 0x7fca4406e8f0 2024-04-25T10:15:38.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.199+0000 7fca49e00640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca4415c820 0x7fca4413b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:38.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.199+0000 7fca49400640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca4406e8f0 0x7fca44158d50 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c400 tx=0x7fca3800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.200+0000 7fca41600640 1 -- 172.21.15.119:0/2495405132 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380052d0 con 0x7fca4406e8f0 2024-04-25T10:15:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.200+0000 7fca41600640 1 -- 172.21.15.119:0/2495405132 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca38005490 con 0x7fca4406e8f0 2024-04-25T10:15:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.200+0000 7fca41600640 1 -- 172.21.15.119:0/2495405132 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38007260 con 0x7fca4406e8f0 2024-04-25T10:15:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.200+0000 7fca4be00640 1 -- 172.21.15.119:0/2495405132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca44159710 con 0x7fca4406e8f0 2024-04-25T10:15:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.200+0000 7fca4be00640 1 -- 172.21.15.119:0/2495405132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca44159cd0 con 0x7fca4406e8f0 2024-04-25T10:15:38.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.201+0000 7fca4be00640 1 -- 172.21.15.119:0/2495405132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4406bb30 con 0x7fca4406e8f0 2024-04-25T10:15:38.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.202+0000 7fca41600640 1 -- 172.21.15.119:0/2495405132 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca38007400 con 0x7fca4406e8f0 2024-04-25T10:15:38.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.203+0000 7fca41600640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca2406b4d0 0x7fca2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:38.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.203+0000 7fca41600640 1 -- 172.21.15.119:0/2495405132 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fca38089840 con 0x7fca4406e8f0 2024-04-25T10:15:38.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.203+0000 7fca48a00640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca2406b4d0 0x7fca2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:38.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.204+0000 7fca48a00640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca2406b4d0 0x7fca2406d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fca4415ab80 tx=0x7fca30008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:38.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.210+0000 7fca41600640 1 -- 172.21.15.119:0/2495405132 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca440ce6c0 con 0x7fca4406e8f0 2024-04-25T10:15:38.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.437+0000 7fca4be00640 1 -- 172.21.15.119:0/2495405132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca4406b290 con 0x7fca4406e8f0 2024-04-25T10:15:38.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.439+0000 7fca41600640 1 -- 172.21.15.119:0/2495405132 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17009 (secure 0 0 0) 0x7fca380585e0 con 0x7fca4406e8f0 2024-04-25T10:15:38.444 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:38.444 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":85,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:14:50.735243+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:21.210576+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:22.438674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/1734507541":"2024-04-26T10:14:50.735180+0000","172.21.15.119:6835/1734507541":"2024-04-26T10:14:50.735180+0000","172.21.15.119:6837/3997964942":"2024-04-26T10:13:36.160373+0000","172.21.15.119:6835/3997964942":"2024-04-26T10:13:36.160373+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:38.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.442+0000 7fca4be00640 1 -- 172.21.15.119:0/2495405132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca2406b4d0 msgr2=0x7fca2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:38.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.442+0000 7fca4be00640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca2406b4d0 0x7fca2406d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fca4415ab80 tx=0x7fca30008040 comp rx=0 tx=0).stop 2024-04-25T10:15:38.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.442+0000 7fca4be00640 1 -- 172.21.15.119:0/2495405132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca4406e8f0 msgr2=0x7fca44158d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:38.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.442+0000 7fca4be00640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca4406e8f0 0x7fca44158d50 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c400 tx=0x7fca3800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.442+0000 7fca4be00640 1 -- 172.21.15.119:0/2495405132 shutdown_connections 2024-04-25T10:15:38.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.442+0000 7fca4be00640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fca2406b4d0 0x7fca2406d990 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.442+0000 7fca4be00640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca4415c820 0x7fca4413b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.442+0000 7fca4be00640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca4415bf00 0x7fca44142410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.442+0000 7fca4be00640 1 --2- 172.21.15.119:0/2495405132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca4406e8f0 0x7fca44158d50 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.442+0000 7fca4be00640 1 -- 172.21.15.119:0/2495405132 >> 172.21.15.119:0/2495405132 conn(0x7fca4406aba0 msgr2=0x7fca440ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:38.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.443+0000 7fca4be00640 1 -- 172.21.15.119:0/2495405132 shutdown_connections 2024-04-25T10:15:38.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.443+0000 7fca4be00640 1 -- 172.21.15.119:0/2495405132 wait complete. 2024-04-25T10:15:38.464 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:15:38.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.643+0000 7f4267600640 1 Processor -- start 2024-04-25T10:15:38.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.644+0000 7f4267600640 1 -- start start 2024-04-25T10:15:38.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.645+0000 7f4267600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4260071fc0 0x7f426006d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:38.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.645+0000 7f4267600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426006e3f0 con 0x7f426006dee0 2024-04-25T10:15:38.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.645+0000 7f4267600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426006e6d0 con 0x7f42600b3950 2024-04-25T10:15:38.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.645+0000 7f4267600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426006e9b0 con 0x7f4260071fc0 2024-04-25T10:15:38.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.646+0000 7f4264c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4260071fc0 0x7f426006d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:38.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.646+0000 7f4257e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f426006dee0 0x7f42600b3240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50114/0 (socket says 172.21.15.119:50114) 2024-04-25T10:15:38.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.646+0000 7f4257e00640 1 -- 172.21.15.119:0/3397271403 learned_addr learned my addr 172.21.15.119:0/3397271403 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:38.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.646+0000 7f4264c00640 1 -- 172.21.15.119:0/3397271403 >> v1:172.21.15.132:6789/0 conn(0x7f42600b3950 legacy=0x7f4260162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:38.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.646+0000 7f4264c00640 1 -- 172.21.15.119:0/3397271403 >> v1:172.21.15.119:6789/0 conn(0x7f426006dee0 legacy=0x7f42600b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:38.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.646+0000 7f4264c00640 1 -- 172.21.15.119:0/3397271403 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4260162910 con 0x7f4260071fc0 2024-04-25T10:15:38.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.646+0000 7f425fe00640 1 -- 172.21.15.119:0/3397271403 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927356629 0 0) 0x7f426006e3f0 con 0x7f426006dee0 2024-04-25T10:15:38.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.647+0000 7f4264c00640 1 --2- 172.21.15.119:0/3397271403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4260071fc0 0x7f426006d9a0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f42480097a0 tx=0x7f424802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc1ba5a685efb71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:38.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.647+0000 7f425fe00640 1 -- 172.21.15.119:0/3397271403 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248037070 con 0x7f4260071fc0 2024-04-25T10:15:38.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.647+0000 7f425fe00640 1 -- 172.21.15.119:0/3397271403 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4248032440 con 0x7f4260071fc0 2024-04-25T10:15:38.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.647+0000 7f425fe00640 1 -- 172.21.15.119:0/3397271403 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248032810 con 0x7f4260071fc0 2024-04-25T10:15:38.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.648+0000 7f4267600640 1 -- 172.21.15.119:0/3397271403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4260071fc0 msgr2=0x7f426006d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:38.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.648+0000 7f4267600640 1 --2- 172.21.15.119:0/3397271403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4260071fc0 0x7f426006d9a0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f42480097a0 tx=0x7f424802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.648+0000 7f4267600640 1 -- 172.21.15.119:0/3397271403 shutdown_connections 2024-04-25T10:15:38.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.648+0000 7f4267600640 1 --2- 172.21.15.119:0/3397271403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4260071fc0 0x7f426006d9a0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.648+0000 7f4267600640 1 -- 172.21.15.119:0/3397271403 >> 172.21.15.119:0/3397271403 conn(0x7f426006aba0 msgr2=0x7f426006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:38.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.648+0000 7f4267600640 1 -- 172.21.15.119:0/3397271403 shutdown_connections 2024-04-25T10:15:38.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.648+0000 7f4267600640 1 -- 172.21.15.119:0/3397271403 wait complete. 2024-04-25T10:15:38.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.650+0000 7f4267600640 1 Processor -- start 2024-04-25T10:15:38.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.650+0000 7f4267600640 1 -- start start 2024-04-25T10:15:38.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4267600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f426006dee0 0x7f4260152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:38.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4267600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4260071fc0 0x7f42601640f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:38.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4267600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42600b3950 0x7f42600adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:38.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4267600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42600713c0 con 0x7f42600b3950 2024-04-25T10:15:38.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4267600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4260162590 con 0x7f426006dee0 2024-04-25T10:15:38.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4267600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4260071540 con 0x7f4260071fc0 2024-04-25T10:15:38.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4265600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42600b3950 0x7f42600adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:38.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4265600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42600b3950 0x7f42600adbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43744/0 (socket says 172.21.15.119:43744) 2024-04-25T10:15:38.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4265600640 1 -- 172.21.15.119:0/3299773827 learned_addr learned my addr 172.21.15.119:0/3299773827 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:38.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4257e00640 1 --2- 172.21.15.119:0/3299773827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4260071fc0 0x7f42601640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:38.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4264c00640 1 --2- 172.21.15.119:0/3299773827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f426006dee0 0x7f4260152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:38.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4265600640 1 -- 172.21.15.119:0/3299773827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4260071fc0 msgr2=0x7f42601640f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:38.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4265600640 1 --2- 172.21.15.119:0/3299773827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4260071fc0 0x7f42601640f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4265600640 1 -- 172.21.15.119:0/3299773827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f426006dee0 msgr2=0x7f4260152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:38.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4265600640 1 --2- 172.21.15.119:0/3299773827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f426006dee0 0x7f4260152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.651+0000 7f4265600640 1 -- 172.21.15.119:0/3299773827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426014f0d0 con 0x7f42600b3950 2024-04-25T10:15:38.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.652+0000 7f4265600640 1 --2- 172.21.15.119:0/3299773827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42600b3950 0x7f42600adbd0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f425800b970 tx=0x7f425800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:38.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.652+0000 7f425d600640 1 -- 172.21.15.119:0/3299773827 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258019070 con 0x7f42600b3950 2024-04-25T10:15:38.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.652+0000 7f425d600640 1 -- 172.21.15.119:0/3299773827 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425800cb40 con 0x7f42600b3950 2024-04-25T10:15:38.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.652+0000 7f4267600640 1 -- 172.21.15.119:0/3299773827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426014f3b0 con 0x7f42600b3950 2024-04-25T10:15:38.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.652+0000 7f425d600640 1 -- 172.21.15.119:0/3299773827 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258014440 con 0x7f42600b3950 2024-04-25T10:15:38.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.652+0000 7f4267600640 1 -- 172.21.15.119:0/3299773827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426014f970 con 0x7f42600b3950 2024-04-25T10:15:38.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.653+0000 7f4267600640 1 -- 172.21.15.119:0/3299773827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42600750b0 con 0x7f42600b3950 2024-04-25T10:15:38.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.654+0000 7f425d600640 1 -- 172.21.15.119:0/3299773827 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f425801c070 con 0x7f42600b3950 2024-04-25T10:15:38.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.656+0000 7f425d600640 1 --2- 172.21.15.119:0/3299773827 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f423c06b500 0x7f423c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:38.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.656+0000 7f425d600640 1 -- 172.21.15.119:0/3299773827 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f4258089ed0 con 0x7f42600b3950 2024-04-25T10:15:38.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.656+0000 7f4264c00640 1 --2- 172.21.15.119:0/3299773827 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f423c06b500 0x7f423c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:38.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.657+0000 7f4264c00640 1 --2- 172.21.15.119:0/3299773827 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f423c06b500 0x7f423c06d9c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4248034000 tx=0x7f424802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:38.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.660+0000 7f425d600640 1 -- 172.21.15.119:0/3299773827 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4258058c70 con 0x7f42600b3950 2024-04-25T10:15:38.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.892+0000 7f4267600640 1 -- 172.21.15.119:0/3299773827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f426006af80 con 0x7f42600b3950 2024-04-25T10:15:38.897 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:15:38.893+0000 7f8c64600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:15:38.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.897+0000 7f425d600640 1 -- 172.21.15.119:0/3299773827 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4258058570 con 0x7f42600b3950 2024-04-25T10:15:38.901 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:15:38.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.901+0000 7f4267600640 1 -- 172.21.15.119:0/3299773827 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f423c06b500 msgr2=0x7f423c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:38.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.901+0000 7f4267600640 1 --2- 172.21.15.119:0/3299773827 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f423c06b500 0x7f423c06d9c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4248034000 tx=0x7f424802e000 comp rx=0 tx=0).stop 2024-04-25T10:15:38.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.901+0000 7f4267600640 1 -- 172.21.15.119:0/3299773827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42600b3950 msgr2=0x7f42600adbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:38.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.901+0000 7f4267600640 1 --2- 172.21.15.119:0/3299773827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42600b3950 0x7f42600adbd0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f425800b970 tx=0x7f425800be30 comp rx=0 tx=0).stop 2024-04-25T10:15:38.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.901+0000 7f4267600640 1 -- 172.21.15.119:0/3299773827 shutdown_connections 2024-04-25T10:15:38.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.901+0000 7f4267600640 1 --2- 172.21.15.119:0/3299773827 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f423c06b500 0x7f423c06d9c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.901+0000 7f4267600640 1 --2- 172.21.15.119:0/3299773827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42600b3950 0x7f42600adbd0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.901+0000 7f4267600640 1 --2- 172.21.15.119:0/3299773827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4260071fc0 0x7f42601640f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.901+0000 7f4267600640 1 --2- 172.21.15.119:0/3299773827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f426006dee0 0x7f4260152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:38.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.901+0000 7f4267600640 1 -- 172.21.15.119:0/3299773827 >> 172.21.15.119:0/3299773827 conn(0x7f426006aba0 msgr2=0x7f42600b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:38.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.902+0000 7f4267600640 1 -- 172.21.15.119:0/3299773827 shutdown_connections 2024-04-25T10:15:38.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:38.902+0000 7f4267600640 1 -- 172.21.15.119:0/3299773827 wait complete. 2024-04-25T10:15:38.915 INFO:tasks.ceph.mds.a.smithi119.stderr: -2482> 2024-04-25T10:14:56.750+0000 7fb532000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:15:38.916 INFO:tasks.ceph.mds.a.smithi119.stderr: -2022> 2024-04-25T10:14:59.360+0000 7fb52d000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x564d54ff0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:15:38.930 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:15:38.969 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:15:39.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.101+0000 7f44c1000640 1 Processor -- start 2024-04-25T10:15:39.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.101+0000 7f44c1000640 1 -- start start 2024-04-25T10:15:39.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.103+0000 7f44c1000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f44bc0b2dd0 0x7f44bc15d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:39.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.103+0000 7f44c1000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44bc06b3d0 con 0x7f44bc15ef60 2024-04-25T10:15:39.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.103+0000 7f44c1000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44bc15e480 con 0x7f44bc0cfad0 2024-04-25T10:15:39.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.103+0000 7f44c1000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44bc06b6b0 con 0x7f44bc0b2dd0 2024-04-25T10:15:39.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.103+0000 7f44b1400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f44bc15ef60 0x7f44bc06aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50120/0 (socket says 172.21.15.119:50120) 2024-04-25T10:15:39.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.103+0000 7f44b1400640 1 -- 172.21.15.119:0/1154203000 learned_addr learned my addr 172.21.15.119:0/1154203000 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:39.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44ba800640 1 --2- 172.21.15.119:0/1154203000 >> v2:172.21.15.132:3300/0 conn(0x7f44bc0b2dd0 0x7f44bc15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:39.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 -- 172.21.15.119:0/1154203000 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614873776 0 0) 0x7f44bc06b3d0 con 0x7f44bc15ef60 2024-04-25T10:15:39.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 -- 172.21.15.119:0/1154203000 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44a0003520 con 0x7f44bc15ef60 2024-04-25T10:15:39.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 -- 172.21.15.119:0/1154203000 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2307430401 0 0) 0x7f44a0003520 con 0x7f44bc15ef60 2024-04-25T10:15:39.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 -- 172.21.15.119:0/1154203000 >> v2:172.21.15.132:3300/0 conn(0x7f44bc0b2dd0 msgr2=0x7f44bc15d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:39.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 --2- 172.21.15.119:0/1154203000 >> v2:172.21.15.132:3300/0 conn(0x7f44bc0b2dd0 0x7f44bc15d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:39.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 -- 172.21.15.119:0/1154203000 >> v1:172.21.15.132:6789/0 conn(0x7f44bc0cfad0 legacy=0x7f44bc0cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:39.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 -- 172.21.15.119:0/1154203000 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc15e760 con 0x7f44bc15ef60 2024-04-25T10:15:39.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 -- 172.21.15.119:0/1154203000 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f44a4001670 con 0x7f44bc15ef60 2024-04-25T10:15:39.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 -- 172.21.15.119:0/1154203000 >> v1:172.21.15.119:6789/0 conn(0x7f44bc15ef60 legacy=0x7f44bc06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:39.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 --2- 172.21.15.119:0/1154203000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a0003a00 0x7f44a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:39.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 --2- 172.21.15.119:0/1154203000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a0004800 0x7f44a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:39.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 --2- 172.21.15.119:0/1154203000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a00051c0 0x7f44a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:39.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 -- 172.21.15.119:0/1154203000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc06b6b0 con 0x7f44a0003a00 2024-04-25T10:15:39.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 -- 172.21.15.119:0/1154203000 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a000c0c0 con 0x7f44a00051c0 2024-04-25T10:15:39.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.104+0000 7f44b9400640 1 -- 172.21.15.119:0/1154203000 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a000c2c0 con 0x7f44a0004800 2024-04-25T10:15:39.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.105+0000 7f44b9e00640 1 --2- 172.21.15.119:0/1154203000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a0003a00 0x7f44a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:39.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.105+0000 7f44ba800640 1 --2- 172.21.15.119:0/1154203000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a00051c0 0x7f44a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:39.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.105+0000 7f44b1400640 1 --2- 172.21.15.119:0/1154203000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a0004800 0x7f44a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:39.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.105+0000 7f44b9e00640 1 -- 172.21.15.119:0/1154203000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a0004800 msgr2=0x7f44a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:39.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.105+0000 7f44b9e00640 1 --2- 172.21.15.119:0/1154203000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a0004800 0x7f44a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:39.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.105+0000 7f44b9e00640 1 -- 172.21.15.119:0/1154203000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a00051c0 msgr2=0x7f44a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:39.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.105+0000 7f44b9e00640 1 --2- 172.21.15.119:0/1154203000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a00051c0 0x7f44a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:39.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.105+0000 7f44b9e00640 1 -- 172.21.15.119:0/1154203000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a0003520 con 0x7f44a0003a00 2024-04-25T10:15:39.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.105+0000 7f44ba800640 1 --2- 172.21.15.119:0/1154203000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a00051c0 0x7f44a000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:39.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.105+0000 7f44ba800640 1 -- 172.21.15.119:0/1154203000 reap_dead start 2024-04-25T10:15:39.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.105+0000 7f44b1400640 1 --2- 172.21.15.119:0/1154203000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a0004800 0x7f44a0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:39.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.106+0000 7f44b9e00640 1 --2- 172.21.15.119:0/1154203000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a0003a00 0x7f44a0003e20 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f44ac001750 tx=0x7f44a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba87fbd3813434d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:39.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.106+0000 7f44b9400640 1 -- 172.21.15.119:0/1154203000 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac001370 con 0x7f44a0003a00 2024-04-25T10:15:39.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.106+0000 7f44b9400640 1 -- 172.21.15.119:0/1154203000 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac0052d0 con 0x7f44a0003a00 2024-04-25T10:15:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.106+0000 7f44b9400640 1 -- 172.21.15.119:0/1154203000 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac02ab40 con 0x7f44a0003a00 2024-04-25T10:15:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.107+0000 7f44c1000640 1 -- 172.21.15.119:0/1154203000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a0003a00 msgr2=0x7f44a0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.107+0000 7f44c1000640 1 --2- 172.21.15.119:0/1154203000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a0003a00 0x7f44a0003e20 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f44ac001750 tx=0x7f44a0005830 comp rx=0 tx=0).stop 2024-04-25T10:15:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.107+0000 7f44c1000640 1 -- 172.21.15.119:0/1154203000 shutdown_connections 2024-04-25T10:15:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.107+0000 7f44c1000640 1 --2- 172.21.15.119:0/1154203000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a0003a00 0x7f44a0003e20 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.107+0000 7f44c1000640 1 -- 172.21.15.119:0/1154203000 >> 172.21.15.119:0/1154203000 conn(0x7f44bc099a40 msgr2=0x7f44bc0b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.107+0000 7f44c1000640 1 -- 172.21.15.119:0/1154203000 shutdown_connections 2024-04-25T10:15:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.107+0000 7f44c1000640 1 -- 172.21.15.119:0/1154203000 wait complete. 2024-04-25T10:15:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.109+0000 7f44c1000640 1 Processor -- start 2024-04-25T10:15:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.109+0000 7f44c1000640 1 -- start start 2024-04-25T10:15:39.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.110+0000 7f44c1000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a0003a00 0x7f44bc150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:39.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.110+0000 7f44c1000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44bc152090 0x7f44bc0c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:39.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.110+0000 7f44c1000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44bc0c6350 0x7f44bc15b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:39.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.110+0000 7f44c1000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc0cfeb0 con 0x7f44bc152090 2024-04-25T10:15:39.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.110+0000 7f44c1000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc15ead0 con 0x7f44bc0c6350 2024-04-25T10:15:39.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.110+0000 7f44c1000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc0d0030 con 0x7f44a0003a00 2024-04-25T10:15:39.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.111+0000 7f44b1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44bc152090 0x7f44bc0c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:39.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.111+0000 7f44b1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44bc152090 0x7f44bc0c5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43786/0 (socket says 172.21.15.119:43786) 2024-04-25T10:15:39.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.111+0000 7f44b1400640 1 -- 172.21.15.119:0/295108738 learned_addr learned my addr 172.21.15.119:0/295108738 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:39.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.111+0000 7f44b9e00640 1 --2- 172.21.15.119:0/295108738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a0003a00 0x7f44bc150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:39.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.111+0000 7f44b1400640 1 -- 172.21.15.119:0/295108738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a0003a00 msgr2=0x7f44bc150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:39.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.111+0000 7f44b1400640 1 --2- 172.21.15.119:0/295108738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a0003a00 0x7f44bc150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:39.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.111+0000 7f44b1400640 1 -- 172.21.15.119:0/295108738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44bc0c6350 msgr2=0x7f44bc15b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:39.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.111+0000 7f44b1400640 1 --2- 172.21.15.119:0/295108738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44bc0c6350 0x7f44bc15b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:39.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.111+0000 7f44b1400640 1 -- 172.21.15.119:0/295108738 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc152610 con 0x7f44bc152090 2024-04-25T10:15:39.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.112+0000 7f44b1400640 1 --2- 172.21.15.119:0/295108738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44bc152090 0x7f44bc0c5e10 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f44a400d800 tx=0x7f44a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:39.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.112+0000 7f44b2a00640 1 -- 172.21.15.119:0/295108738 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4019070 con 0x7f44bc152090 2024-04-25T10:15:39.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.112+0000 7f44b2a00640 1 -- 172.21.15.119:0/295108738 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a40072d0 con 0x7f44bc152090 2024-04-25T10:15:39.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.112+0000 7f44b2a00640 1 -- 172.21.15.119:0/295108738 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a40047e0 con 0x7f44bc152090 2024-04-25T10:15:39.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.112+0000 7f44c1000640 1 -- 172.21.15.119:0/295108738 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44bc1528f0 con 0x7f44bc152090 2024-04-25T10:15:39.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.112+0000 7f44c1000640 1 -- 172.21.15.119:0/295108738 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44bc152eb0 con 0x7f44bc152090 2024-04-25T10:15:39.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.114+0000 7f44c1000640 1 -- 172.21.15.119:0/295108738 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44bc0750b0 con 0x7f44bc152090 2024-04-25T10:15:39.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.115+0000 7f44b2a00640 1 -- 172.21.15.119:0/295108738 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44a401c070 con 0x7f44bc152090 2024-04-25T10:15:39.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.115+0000 7f44b2a00640 1 --2- 172.21.15.119:0/295108738 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f449806b520 0x7f449806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:39.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.116+0000 7f44b2a00640 1 -- 172.21.15.119:0/295108738 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5996+0+0 (secure 0 0 0) 0x7f44a4089c80 con 0x7f44bc152090 2024-04-25T10:15:39.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.116+0000 7f44b9e00640 1 --2- 172.21.15.119:0/295108738 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f449806b520 0x7f449806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:39.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.117+0000 7f44b9e00640 1 --2- 172.21.15.119:0/295108738 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f449806b520 0x7f449806d9e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f44ac032000 tx=0x7f44ac02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:39.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.120+0000 7f44b2a00640 1 -- 172.21.15.119:0/295108738 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44a40589c0 con 0x7f44bc152090 2024-04-25T10:15:39.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.352+0000 7f44c1000640 1 -- 172.21.15.119:0/295108738 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f44bc099e50 con 0x7f44bc152090 2024-04-25T10:15:39.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.975+0000 7f44b2a00640 1 -- 172.21.15.119:0/295108738 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 104+0+0 (secure 0 0 0) 0x7f44a40582c0 con 0x7f44bc152090 2024-04-25T10:15:39.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.979+0000 7f44c1000640 1 -- 172.21.15.119:0/295108738 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f449806b520 msgr2=0x7f449806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:39.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.979+0000 7f44c1000640 1 --2- 172.21.15.119:0/295108738 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f449806b520 0x7f449806d9e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f44ac032000 tx=0x7f44ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:15:39.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.979+0000 7f44c1000640 1 -- 172.21.15.119:0/295108738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44bc152090 msgr2=0x7f44bc0c5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:39.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.979+0000 7f44c1000640 1 --2- 172.21.15.119:0/295108738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44bc152090 0x7f44bc0c5e10 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f44a400d800 tx=0x7f44a400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:15:39.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.980+0000 7f44c1000640 1 -- 172.21.15.119:0/295108738 shutdown_connections 2024-04-25T10:15:39.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.980+0000 7f44c1000640 1 --2- 172.21.15.119:0/295108738 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f449806b520 0x7f449806d9e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:39.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.980+0000 7f44c1000640 1 --2- 172.21.15.119:0/295108738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44bc0c6350 0x7f44bc15b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:39.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.980+0000 7f44c1000640 1 --2- 172.21.15.119:0/295108738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44bc152090 0x7f44bc0c5e10 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:39.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.980+0000 7f44c1000640 1 --2- 172.21.15.119:0/295108738 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a0003a00 0x7f44bc150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:39.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.980+0000 7f44c1000640 1 -- 172.21.15.119:0/295108738 >> 172.21.15.119:0/295108738 conn(0x7f44bc099a40 msgr2=0x7f44bc0cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:39.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.980+0000 7f44c1000640 1 -- 172.21.15.119:0/295108738 shutdown_connections 2024-04-25T10:15:39.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:39.980+0000 7f44c1000640 1 -- 172.21.15.119:0/295108738 wait complete. 2024-04-25T10:15:40.007 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:15:40.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.243+0000 7fbdcb000640 1 Processor -- start 2024-04-25T10:15:40.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.244+0000 7fbdcb000640 1 -- start start 2024-04-25T10:15:40.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.245+0000 7fbdcb000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdc40ceeb0 0x7fbdc4168a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:40.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.245+0000 7fbdcb000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc4161f80 con 0x7fbdc40b9720 2024-04-25T10:15:40.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.245+0000 7fbdcb000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc4162260 con 0x7fbdc4161090 2024-04-25T10:15:40.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.245+0000 7fbdcb000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40d21a0 con 0x7fbdc40ceeb0 2024-04-25T10:15:40.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.245+0000 7fbdc3400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbdc40b9720 0x7fbdc40ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50134/0 (socket says 172.21.15.119:50134) 2024-04-25T10:15:40.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.245+0000 7fbdc3400640 1 -- 172.21.15.119:0/296880898 learned_addr learned my addr 172.21.15.119:0/296880898 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:40.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc9000640 1 --2- 172.21.15.119:0/296880898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdc40ceeb0 0x7fbdc4168a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:40.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 -- 172.21.15.119:0/296880898 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981146820 0 0) 0x7fbdc4161f80 con 0x7fbdc40b9720 2024-04-25T10:15:40.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 -- 172.21.15.119:0/296880898 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbda4003520 con 0x7fbdc40b9720 2024-04-25T10:15:40.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 -- 172.21.15.119:0/296880898 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3535602500 0 0) 0x7fbda4003520 con 0x7fbdc40b9720 2024-04-25T10:15:40.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 -- 172.21.15.119:0/296880898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdc40ceeb0 msgr2=0x7fbdc4168a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:40.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 --2- 172.21.15.119:0/296880898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdc40ceeb0 0x7fbdc4168a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 -- 172.21.15.119:0/296880898 >> v1:172.21.15.132:6789/0 conn(0x7fbdc4161090 legacy=0x7fbdc4161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 -- 172.21.15.119:0/296880898 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc4162540 con 0x7fbdc40b9720 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 -- 172.21.15.119:0/296880898 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fbdac001a10 con 0x7fbdc40b9720 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 -- 172.21.15.119:0/296880898 >> v1:172.21.15.119:6789/0 conn(0x7fbdc40b9720 legacy=0x7fbdc40ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 --2- 172.21.15.119:0/296880898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdc40ceeb0 0x7fbda40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 --2- 172.21.15.119:0/296880898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbda4003f60 0x7fbda4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 --2- 172.21.15.119:0/296880898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda4004d40 0x7fbda40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 -- 172.21.15.119:0/296880898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40d21a0 con 0x7fbda4004d40 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 -- 172.21.15.119:0/296880898 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda400bb50 con 0x7fbda4003f60 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc2a00640 1 -- 172.21.15.119:0/296880898 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda400bd50 con 0x7fbdc40ceeb0 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.246+0000 7fbdc9000640 1 --2- 172.21.15.119:0/296880898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda4004d40 0x7fbda40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.247+0000 7fbdc3400640 1 --2- 172.21.15.119:0/296880898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbda4003f60 0x7fbda4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.247+0000 7fbdc9000640 1 -- 172.21.15.119:0/296880898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdc40ceeb0 msgr2=0x7fbda40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.247+0000 7fbdc3e00640 1 --2- 172.21.15.119:0/296880898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdc40ceeb0 0x7fbda40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.247+0000 7fbdc9000640 1 --2- 172.21.15.119:0/296880898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdc40ceeb0 0x7fbda40039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:40.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.247+0000 7fbdc9000640 1 -- 172.21.15.119:0/296880898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbda4003f60 msgr2=0x7fbda4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:40.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.247+0000 7fbdc9000640 1 --2- 172.21.15.119:0/296880898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbda4003f60 0x7fbda4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:40.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.247+0000 7fbdc9000640 1 -- 172.21.15.119:0/296880898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4003520 con 0x7fbda4004d40 2024-04-25T10:15:40.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.248+0000 7fbdc9000640 1 --2- 172.21.15.119:0/296880898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda4004d40 0x7fbda40051c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009a90 tx=0x7fbda4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88baa9f30b16909 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:40.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.248+0000 7fbdc2a00640 1 -- 172.21.15.119:0/296880898 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc036070 con 0x7fbda4004d40 2024-04-25T10:15:40.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.248+0000 7fbdc2a00640 1 -- 172.21.15.119:0/296880898 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdbc031400 con 0x7fbda4004d40 2024-04-25T10:15:40.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.248+0000 7fbdc2a00640 1 -- 172.21.15.119:0/296880898 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc009690 con 0x7fbda4004d40 2024-04-25T10:15:40.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.249+0000 7fbdcb000640 1 -- 172.21.15.119:0/296880898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda4004d40 msgr2=0x7fbda40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:40.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.249+0000 7fbdcb000640 1 --2- 172.21.15.119:0/296880898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda4004d40 0x7fbda40051c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009a90 tx=0x7fbda4005830 comp rx=0 tx=0).stop 2024-04-25T10:15:40.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.249+0000 7fbdc9000640 1 -- 172.21.15.119:0/296880898 reap_dead start 2024-04-25T10:15:40.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.249+0000 7fbdcb000640 1 -- 172.21.15.119:0/296880898 shutdown_connections 2024-04-25T10:15:40.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.249+0000 7fbdcb000640 1 -- 172.21.15.119:0/296880898 >> 172.21.15.119:0/296880898 conn(0x7fbdc40b55e0 msgr2=0x7fbdc40b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:40.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.249+0000 7fbdcb000640 1 -- 172.21.15.119:0/296880898 shutdown_connections 2024-04-25T10:15:40.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.249+0000 7fbdcb000640 1 -- 172.21.15.119:0/296880898 wait complete. 2024-04-25T10:15:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.251+0000 7fbdcb000640 1 Processor -- start 2024-04-25T10:15:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.251+0000 7fbdcb000640 1 -- start start 2024-04-25T10:15:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdcb000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda4004d40 0x7fbdc4151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdcb000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdc4155570 0x7fbdc4153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdcb000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdc4153950 0x7fbdc40cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdcb000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40d3640 con 0x7fbda4004d40 2024-04-25T10:15:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdcb000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc4161470 con 0x7fbdc4153950 2024-04-25T10:15:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdcb000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40d37c0 con 0x7fbdc4155570 2024-04-25T10:15:40.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdc3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda4004d40 0x7fbdc4151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:40.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdc3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda4004d40 0x7fbdc4151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43798/0 (socket says 172.21.15.119:43798) 2024-04-25T10:15:40.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdc3e00640 1 -- 172.21.15.119:0/162414515 learned_addr learned my addr 172.21.15.119:0/162414515 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:40.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdc3e00640 1 -- 172.21.15.119:0/162414515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdc4155570 msgr2=0x7fbdc4153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:40.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdc3e00640 1 --2- 172.21.15.119:0/162414515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdc4155570 0x7fbdc4153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:40.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdc3e00640 1 -- 172.21.15.119:0/162414515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdc4153950 msgr2=0x7fbdc40cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:40.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdc3e00640 1 --2- 172.21.15.119:0/162414515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdc4153950 0x7fbdc40cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:40.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdc3e00640 1 -- 172.21.15.119:0/162414515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc4152500 con 0x7fbda4004d40 2024-04-25T10:15:40.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.252+0000 7fbdc3e00640 1 --2- 172.21.15.119:0/162414515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda4004d40 0x7fbdc4151fc0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400d570 tx=0x7fbdb400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:40.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.253+0000 7fbdbbe00640 1 -- 172.21.15.119:0/162414515 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4005480 con 0x7fbda4004d40 2024-04-25T10:15:40.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.253+0000 7fbdbbe00640 1 -- 172.21.15.119:0/162414515 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb4017070 con 0x7fbda4004d40 2024-04-25T10:15:40.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.253+0000 7fbdcb000640 1 -- 172.21.15.119:0/162414515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc40c76f0 con 0x7fbda4004d40 2024-04-25T10:15:40.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.253+0000 7fbdbbe00640 1 -- 172.21.15.119:0/162414515 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4003810 con 0x7fbda4004d40 2024-04-25T10:15:40.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.253+0000 7fbdcb000640 1 -- 172.21.15.119:0/162414515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc40c7cb0 con 0x7fbda4004d40 2024-04-25T10:15:40.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.254+0000 7fbdcb000640 1 -- 172.21.15.119:0/162414515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc40750b0 con 0x7fbda4004d40 2024-04-25T10:15:40.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.255+0000 7fbdbbe00640 1 -- 172.21.15.119:0/162414515 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbdb40039b0 con 0x7fbda4004d40 2024-04-25T10:15:40.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.256+0000 7fbdbbe00640 1 --2- 172.21.15.119:0/162414515 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbda006b500 0x7fbda006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:40.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.256+0000 7fbdbbe00640 1 -- 172.21.15.119:0/162414515 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5996+0+0 (secure 0 0 0) 0x7fbdb408a120 con 0x7fbda4004d40 2024-04-25T10:15:40.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.256+0000 7fbdc3400640 1 --2- 172.21.15.119:0/162414515 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbda006b500 0x7fbda006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:40.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.257+0000 7fbdc3400640 1 --2- 172.21.15.119:0/162414515 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbda006b500 0x7fbda006d9c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbdac001b80 tx=0x7fbdac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:40.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.260+0000 7fbdbbe00640 1 -- 172.21.15.119:0/162414515 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbdc4152500 con 0x7fbda4004d40 2024-04-25T10:15:40.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:40.453+0000 7fbdcb000640 1 -- 172.21.15.119:0/162414515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbdc406ccc0 con 0x7fbda4004d40 2024-04-25T10:15:41.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.001+0000 7fbdbbe00640 1 -- 172.21.15.119:0/162414515 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbdb4058e70 con 0x7fbda4004d40 2024-04-25T10:15:41.005 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:15:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.005+0000 7fbdcb000640 1 -- 172.21.15.119:0/162414515 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbda006b500 msgr2=0x7fbda006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.005+0000 7fbdcb000640 1 --2- 172.21.15.119:0/162414515 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbda006b500 0x7fbda006d9c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbdac001b80 tx=0x7fbdac009040 comp rx=0 tx=0).stop 2024-04-25T10:15:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.005+0000 7fbdcb000640 1 -- 172.21.15.119:0/162414515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda4004d40 msgr2=0x7fbdc4151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:41.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.005+0000 7fbdcb000640 1 --2- 172.21.15.119:0/162414515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda4004d40 0x7fbdc4151fc0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400d570 tx=0x7fbdb400da30 comp rx=0 tx=0).stop 2024-04-25T10:15:41.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.006+0000 7fbdcb000640 1 -- 172.21.15.119:0/162414515 shutdown_connections 2024-04-25T10:15:41.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.006+0000 7fbdcb000640 1 --2- 172.21.15.119:0/162414515 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbda006b500 0x7fbda006d9c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:41.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.006+0000 7fbdcb000640 1 --2- 172.21.15.119:0/162414515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdc4153950 0x7fbdc40cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:41.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.006+0000 7fbdcb000640 1 --2- 172.21.15.119:0/162414515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdc4155570 0x7fbdc4153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:41.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.006+0000 7fbdcb000640 1 --2- 172.21.15.119:0/162414515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbda4004d40 0x7fbdc4151fc0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:41.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.007+0000 7fbdcb000640 1 -- 172.21.15.119:0/162414515 >> 172.21.15.119:0/162414515 conn(0x7fbdc40b55e0 msgr2=0x7fbdc40b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:41.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.007+0000 7fbdcb000640 1 -- 172.21.15.119:0/162414515 shutdown_connections 2024-04-25T10:15:41.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.007+0000 7fbdcb000640 1 -- 172.21.15.119:0/162414515 wait complete. 2024-04-25T10:15:41.034 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:15:41.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.236+0000 7fbe8d200640 1 Processor -- start 2024-04-25T10:15:41.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.236+0000 7fbe8d200640 1 -- start start 2024-04-25T10:15:41.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.238+0000 7fbe8d200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbe880b9720 0x7fbe880ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:41.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.238+0000 7fbe8d200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbe880cece0 0x7fbe880d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:41.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.238+0000 7fbe8d200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe880cf0f0 con 0x7fbe88161090 2024-04-25T10:15:41.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.238+0000 7fbe8d200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe880d21a0 con 0x7fbe880cece0 2024-04-25T10:15:41.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.238+0000 7fbe8d200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe88161f80 con 0x7fbe880b9720 2024-04-25T10:15:41.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.238+0000 7fbe85e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbe88161090 0x7fbe88161730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47854/0 (socket says 172.21.15.119:47854) 2024-04-25T10:15:41.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.238+0000 7fbe85e00640 1 -- 172.21.15.119:0/1510715839 learned_addr learned my addr 172.21.15.119:0/1510715839 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:41.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.238+0000 7fbe86800640 1 --2- 172.21.15.119:0/1510715839 >> v2:172.21.15.119:3300/0 conn(0x7fbe880cece0 0x7fbe880d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.238+0000 7fbe85400640 1 --2- 172.21.15.119:0/1510715839 >> v2:172.21.15.132:3300/0 conn(0x7fbe880b9720 0x7fbe880ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.239+0000 7fbe86800640 1 -- 172.21.15.119:0/1510715839 >> v2:172.21.15.132:3300/0 conn(0x7fbe880b9720 msgr2=0x7fbe880ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.239+0000 7fbe86800640 1 --2- 172.21.15.119:0/1510715839 >> v2:172.21.15.132:3300/0 conn(0x7fbe880b9720 0x7fbe880ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.239+0000 7fbe86800640 1 -- 172.21.15.119:0/1510715839 >> v1:172.21.15.119:6789/0 conn(0x7fbe88161090 legacy=0x7fbe88161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.239+0000 7fbe86800640 1 -- 172.21.15.119:0/1510715839 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe88162260 con 0x7fbe880cece0 2024-04-25T10:15:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.239+0000 7fbe84a00640 1 -- 172.21.15.119:0/1510715839 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670184704 0 0) 0x7fbe78001490 con 0x7fbe88161090 2024-04-25T10:15:41.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.240+0000 7fbe86800640 1 --2- 172.21.15.119:0/1510715839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe880cece0 0x7fbe880d2630 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fbe8808cf10 tx=0x7fbe8002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe97a5d831b7915e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:41.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.240+0000 7fbe84a00640 1 -- 172.21.15.119:0/1510715839 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe80037070 con 0x7fbe880cece0 2024-04-25T10:15:41.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.240+0000 7fbe84a00640 1 -- 172.21.15.119:0/1510715839 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe80032440 con 0x7fbe880cece0 2024-04-25T10:15:41.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.240+0000 7fbe84a00640 1 -- 172.21.15.119:0/1510715839 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8002ae60 con 0x7fbe880cece0 2024-04-25T10:15:41.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.241+0000 7fbe8d200640 1 -- 172.21.15.119:0/1510715839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe880cece0 msgr2=0x7fbe880d2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:41.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.241+0000 7fbe8d200640 1 --2- 172.21.15.119:0/1510715839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe880cece0 0x7fbe880d2630 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fbe8808cf10 tx=0x7fbe8002a110 comp rx=0 tx=0).stop 2024-04-25T10:15:41.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.241+0000 7fbe8d200640 1 -- 172.21.15.119:0/1510715839 shutdown_connections 2024-04-25T10:15:41.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.241+0000 7fbe8d200640 1 --2- 172.21.15.119:0/1510715839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe880cece0 0x7fbe880d2630 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:41.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.241+0000 7fbe8d200640 1 --2- 172.21.15.119:0/1510715839 >> v2:172.21.15.132:3300/0 conn(0x7fbe880b9720 0x7fbe880ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:41.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.241+0000 7fbe8d200640 1 -- 172.21.15.119:0/1510715839 >> 172.21.15.119:0/1510715839 conn(0x7fbe880b55e0 msgr2=0x7fbe880b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:41.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.241+0000 7fbe8d200640 1 -- 172.21.15.119:0/1510715839 shutdown_connections 2024-04-25T10:15:41.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.241+0000 7fbe8d200640 1 -- 172.21.15.119:0/1510715839 wait complete. 2024-04-25T10:15:41.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.242+0000 7fbe8d200640 1 Processor -- start 2024-04-25T10:15:41.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.243+0000 7fbe8d200640 1 -- start start 2024-04-25T10:15:41.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.243+0000 7fbe8d200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe880b9720 0x7fbe8814de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:41.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe8d200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe880cece0 0x7fbe8815ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:41.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe8d200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe88161090 0x7fbe880cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:41.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe8d200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe88161470 con 0x7fbe88161090 2024-04-25T10:15:41.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe8d200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe880d1e30 con 0x7fbe880b9720 2024-04-25T10:15:41.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe8d200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe880d2b70 con 0x7fbe880cece0 2024-04-25T10:15:41.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe86800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe88161090 0x7fbe880cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:41.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe86800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe88161090 0x7fbe880cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59316/0 (socket says 172.21.15.119:59316) 2024-04-25T10:15:41.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe86800640 1 -- 172.21.15.119:0/4163230162 learned_addr learned my addr 172.21.15.119:0/4163230162 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:41.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe85e00640 1 --2- 172.21.15.119:0/4163230162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe880b9720 0x7fbe8814de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:41.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe86800640 1 -- 172.21.15.119:0/4163230162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe880cece0 msgr2=0x7fbe8815ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:41.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe86800640 1 --2- 172.21.15.119:0/4163230162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe880cece0 0x7fbe8815ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:41.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe86800640 1 -- 172.21.15.119:0/4163230162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe880b9720 msgr2=0x7fbe8814de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:41.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe86800640 1 --2- 172.21.15.119:0/4163230162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe880b9720 0x7fbe8814de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:41.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe86800640 1 -- 172.21.15.119:0/4163230162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8815fc20 con 0x7fbe88161090 2024-04-25T10:15:41.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.244+0000 7fbe85e00640 1 --2- 172.21.15.119:0/4163230162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe880b9720 0x7fbe8814de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:41.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.245+0000 7fbe86800640 1 --2- 172.21.15.119:0/4163230162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe88161090 0x7fbe880cc1e0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fbe80035040 tx=0x7fbe800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:41.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.245+0000 7fbe7e000640 1 -- 172.21.15.119:0/4163230162 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe80037040 con 0x7fbe88161090 2024-04-25T10:15:41.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.245+0000 7fbe7e000640 1 -- 172.21.15.119:0/4163230162 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe80042070 con 0x7fbe88161090 2024-04-25T10:15:41.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.245+0000 7fbe7e000640 1 -- 172.21.15.119:0/4163230162 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe80030b50 con 0x7fbe88161090 2024-04-25T10:15:41.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.245+0000 7fbe8d200640 1 -- 172.21.15.119:0/4163230162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8815ff00 con 0x7fbe88161090 2024-04-25T10:15:41.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.245+0000 7fbe8d200640 1 -- 172.21.15.119:0/4163230162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe881604c0 con 0x7fbe88161090 2024-04-25T10:15:41.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.246+0000 7fbe8d200640 1 -- 172.21.15.119:0/4163230162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe880750b0 con 0x7fbe88161090 2024-04-25T10:15:41.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.247+0000 7fbe7e000640 1 -- 172.21.15.119:0/4163230162 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe80030cf0 con 0x7fbe88161090 2024-04-25T10:15:41.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.249+0000 7fbe7e000640 1 --2- 172.21.15.119:0/4163230162 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe6806b4d0 0x7fbe6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:41.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.249+0000 7fbe7e000640 1 -- 172.21.15.119:0/4163230162 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5558+0+0 (secure 0 0 0) 0x7fbe800aaeb0 con 0x7fbe88161090 2024-04-25T10:15:41.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.249+0000 7fbe85e00640 1 --2- 172.21.15.119:0/4163230162 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe6806b4d0 0x7fbe6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:41.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.250+0000 7fbe85e00640 1 --2- 172.21.15.119:0/4163230162 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe6806b4d0 0x7fbe6806d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbe78007fd0 tx=0x7fbe78009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:41.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.253+0000 7fbe7e000640 1 -- 172.21.15.119:0/4163230162 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe80045360 con 0x7fbe88161090 2024-04-25T10:15:41.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:41.446+0000 7fbe8d200640 1 -- 172.21.15.119:0/4163230162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbe880b5350 con 0x7fbe88161090 2024-04-25T10:15:42.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.012+0000 7fbe7e000640 1 -- 172.21.15.119:0/4163230162 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbe80079d20 con 0x7fbe88161090 2024-04-25T10:15:42.017 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2024-04-25T10:15:42.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.017+0000 7fbe8d200640 1 -- 172.21.15.119:0/4163230162 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe6806b4d0 msgr2=0x7fbe6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:42.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.017+0000 7fbe8d200640 1 --2- 172.21.15.119:0/4163230162 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe6806b4d0 0x7fbe6806d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbe78007fd0 tx=0x7fbe78009040 comp rx=0 tx=0).stop 2024-04-25T10:15:42.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.017+0000 7fbe8d200640 1 -- 172.21.15.119:0/4163230162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe88161090 msgr2=0x7fbe880cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:42.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.017+0000 7fbe8d200640 1 --2- 172.21.15.119:0/4163230162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe88161090 0x7fbe880cc1e0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fbe80035040 tx=0x7fbe800046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.017+0000 7fbe8d200640 1 -- 172.21.15.119:0/4163230162 shutdown_connections 2024-04-25T10:15:42.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.017+0000 7fbe8d200640 1 --2- 172.21.15.119:0/4163230162 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbe6806b4d0 0x7fbe6806d990 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.017+0000 7fbe8d200640 1 --2- 172.21.15.119:0/4163230162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe88161090 0x7fbe880cc1e0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.017+0000 7fbe8d200640 1 --2- 172.21.15.119:0/4163230162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbe880cece0 0x7fbe8815ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.017+0000 7fbe8d200640 1 --2- 172.21.15.119:0/4163230162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe880b9720 0x7fbe8814de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.017+0000 7fbe8d200640 1 -- 172.21.15.119:0/4163230162 >> 172.21.15.119:0/4163230162 conn(0x7fbe880b55e0 msgr2=0x7fbe880d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:42.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.020+0000 7fbe8d200640 1 -- 172.21.15.119:0/4163230162 shutdown_connections 2024-04-25T10:15:42.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.020+0000 7fbe8d200640 1 -- 172.21.15.119:0/4163230162 wait complete. 2024-04-25T10:15:42.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:15:42.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.232+0000 7fc8a2800640 1 Processor -- start 2024-04-25T10:15:42.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.233+0000 7fc8a2800640 1 -- start start 2024-04-25T10:15:42.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.234+0000 7fc8a2800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc89c15d980 0x7fc89c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.234+0000 7fc8a2800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc89c0b2a00 0x7fc89c0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.234+0000 7fc8a2800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c0b6ac0 con 0x7fc89c0b2440 2024-04-25T10:15:42.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.234+0000 7fc8a2800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c0cefe0 con 0x7fc89c15d980 2024-04-25T10:15:42.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.234+0000 7fc8a2800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0b70d0 con 0x7fc89c0b2a00 2024-04-25T10:15:42.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.234+0000 7fc8a1e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc89c0b2440 0x7fc89c0b7560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47866/0 (socket says 172.21.15.119:47866) 2024-04-25T10:15:42.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.234+0000 7fc8a1e00640 1 -- 172.21.15.119:0/483935941 learned_addr learned my addr 172.21.15.119:0/483935941 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:42.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.235+0000 7fc8a1400640 1 --2- 172.21.15.119:0/483935941 >> v2:172.21.15.132:3300/0 conn(0x7fc89c15d980 0x7fc89c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.235+0000 7fc8a0a00640 1 --2- 172.21.15.119:0/483935941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc89c0b2a00 0x7fc89c0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.235+0000 7fc89be00640 1 -- 172.21.15.119:0/483935941 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569868020 0 0) 0x7fc89c0b6ac0 con 0x7fc89c0b2440 2024-04-25T10:15:42.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.235+0000 7fc89be00640 1 -- 172.21.15.119:0/483935941 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc880003520 con 0x7fc89c0b2440 2024-04-25T10:15:42.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.235+0000 7fc89be00640 1 -- 172.21.15.119:0/483935941 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3843795734 0 0) 0x7fc880003520 con 0x7fc89c0b2440 2024-04-25T10:15:42.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.235+0000 7fc89be00640 1 -- 172.21.15.119:0/483935941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc89c0b2a00 msgr2=0x7fc89c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:42.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.235+0000 7fc89be00640 1 --2- 172.21.15.119:0/483935941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc89c0b2a00 0x7fc89c0ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.235+0000 7fc89be00640 1 -- 172.21.15.119:0/483935941 >> v2:172.21.15.132:3300/0 conn(0x7fc89c15d980 msgr2=0x7fc89c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:42.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.235+0000 7fc89be00640 1 --2- 172.21.15.119:0/483935941 >> v2:172.21.15.132:3300/0 conn(0x7fc89c15d980 0x7fc89c0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.235+0000 7fc89be00640 1 -- 172.21.15.119:0/483935941 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c0b94f0 con 0x7fc89c0b2440 2024-04-25T10:15:42.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.235+0000 7fc89be00640 1 -- 172.21.15.119:0/483935941 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fc894001670 con 0x7fc89c0b2440 2024-04-25T10:15:42.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc89be00640 1 -- 172.21.15.119:0/483935941 >> v1:172.21.15.119:6789/0 conn(0x7fc89c0b2440 legacy=0x7fc89c0b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:42.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc89be00640 1 --2- 172.21.15.119:0/483935941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8800039b0 0x7fc880003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc89be00640 1 --2- 172.21.15.119:0/483935941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc880004800 0x7fc880004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc89be00640 1 --2- 172.21.15.119:0/483935941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc89c0b2a00 0x7fc8800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc89be00640 1 -- 172.21.15.119:0/483935941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0cefe0 con 0x7fc8800039b0 2024-04-25T10:15:42.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc89be00640 1 -- 172.21.15.119:0/483935941 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88000b9c0 con 0x7fc880004800 2024-04-25T10:15:42.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc89be00640 1 -- 172.21.15.119:0/483935941 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88000bb40 con 0x7fc89c0b2a00 2024-04-25T10:15:42.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc8a1400640 1 --2- 172.21.15.119:0/483935941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8800039b0 0x7fc880003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc8a0a00640 1 --2- 172.21.15.119:0/483935941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc880004800 0x7fc880004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc8a1e00640 1 --2- 172.21.15.119:0/483935941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc89c0b2a00 0x7fc8800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc8a1400640 1 -- 172.21.15.119:0/483935941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc89c0b2a00 msgr2=0x7fc8800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:42.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc8a1400640 1 --2- 172.21.15.119:0/483935941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc89c0b2a00 0x7fc8800051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc8a1400640 1 -- 172.21.15.119:0/483935941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc880004800 msgr2=0x7fc880004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:42.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc8a1400640 1 --2- 172.21.15.119:0/483935941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc880004800 0x7fc880004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.236+0000 7fc8a1400640 1 -- 172.21.15.119:0/483935941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc880003520 con 0x7fc8800039b0 2024-04-25T10:15:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.237+0000 7fc8a1400640 1 --2- 172.21.15.119:0/483935941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8800039b0 0x7fc880003dd0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009fc0 tx=0x7fc880005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4458a2d61cf3e090 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.238+0000 7fc89be00640 1 -- 172.21.15.119:0/483935941 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c009c10 con 0x7fc8800039b0 2024-04-25T10:15:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.238+0000 7fc89be00640 1 -- 172.21.15.119:0/483935941 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c036070 con 0x7fc8800039b0 2024-04-25T10:15:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.238+0000 7fc89be00640 1 -- 172.21.15.119:0/483935941 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c02d070 con 0x7fc8800039b0 2024-04-25T10:15:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.238+0000 7fc8a2800640 1 -- 172.21.15.119:0/483935941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8800039b0 msgr2=0x7fc880003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.238+0000 7fc8a2800640 1 --2- 172.21.15.119:0/483935941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8800039b0 0x7fc880003dd0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009fc0 tx=0x7fc880005830 comp rx=0 tx=0).stop 2024-04-25T10:15:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.238+0000 7fc8a1e00640 1 -- 172.21.15.119:0/483935941 reap_dead start 2024-04-25T10:15:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.239+0000 7fc8a2800640 1 -- 172.21.15.119:0/483935941 shutdown_connections 2024-04-25T10:15:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.239+0000 7fc8a2800640 1 -- 172.21.15.119:0/483935941 >> 172.21.15.119:0/483935941 conn(0x7fc89c0726c0 msgr2=0x7fc89c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.239+0000 7fc8a2800640 1 -- 172.21.15.119:0/483935941 shutdown_connections 2024-04-25T10:15:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.239+0000 7fc8a2800640 1 -- 172.21.15.119:0/483935941 wait complete. 2024-04-25T10:15:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.240+0000 7fc8a2800640 1 Processor -- start 2024-04-25T10:15:42.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.241+0000 7fc8a2800640 1 -- start start 2024-04-25T10:15:42.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.241+0000 7fc8a2800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8800039b0 0x7fc89c156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.241+0000 7fc8a2800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc89c153110 0x7fc89c1605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.241+0000 7fc8a2800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc89c160b30 0x7fc89c159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.241+0000 7fc8a2800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c15dd60 con 0x7fc8800039b0 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.241+0000 7fc8a2800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0b97d0 con 0x7fc89c153110 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.241+0000 7fc8a2800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0b9110 con 0x7fc89c160b30 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.242+0000 7fc8a1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8800039b0 0x7fc89c156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.242+0000 7fc8a1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8800039b0 0x7fc89c156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59336/0 (socket says 172.21.15.119:59336) 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.242+0000 7fc8a1400640 1 -- 172.21.15.119:0/91455059 learned_addr learned my addr 172.21.15.119:0/91455059 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.242+0000 7fc8a1e00640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc89c160b30 0x7fc89c159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.242+0000 7fc8a0a00640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc89c153110 0x7fc89c1605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.242+0000 7fc8a1400640 1 -- 172.21.15.119:0/91455059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc89c160b30 msgr2=0x7fc89c159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.242+0000 7fc8a1400640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc89c160b30 0x7fc89c159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.242+0000 7fc8a1400640 1 -- 172.21.15.119:0/91455059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc89c153110 msgr2=0x7fc89c1605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.242+0000 7fc8a1400640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc89c153110 0x7fc89c1605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.242+0000 7fc8a1400640 1 -- 172.21.15.119:0/91455059 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c154910 con 0x7fc8800039b0 2024-04-25T10:15:42.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.242+0000 7fc8a1e00640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc89c160b30 0x7fc89c159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:42.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.242+0000 7fc8a0a00640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc89c153110 0x7fc89c1605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:42.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.243+0000 7fc8a1400640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8800039b0 0x7fc89c156e80 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00e200 tx=0x7fc88c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:42.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.243+0000 7fc899600640 1 -- 172.21.15.119:0/91455059 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c009c10 con 0x7fc8800039b0 2024-04-25T10:15:42.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.243+0000 7fc899600640 1 -- 172.21.15.119:0/91455059 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c003b90 con 0x7fc8800039b0 2024-04-25T10:15:42.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.243+0000 7fc899600640 1 -- 172.21.15.119:0/91455059 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c004020 con 0x7fc8800039b0 2024-04-25T10:15:42.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.243+0000 7fc8a2800640 1 -- 172.21.15.119:0/91455059 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89c157a40 con 0x7fc8800039b0 2024-04-25T10:15:42.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.243+0000 7fc8a2800640 1 -- 172.21.15.119:0/91455059 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c154bd0 con 0x7fc8800039b0 2024-04-25T10:15:42.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.244+0000 7fc8a2800640 1 -- 172.21.15.119:0/91455059 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c0749b0 con 0x7fc8800039b0 2024-04-25T10:15:42.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.245+0000 7fc899600640 1 -- 172.21.15.119:0/91455059 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc88c005d10 con 0x7fc8800039b0 2024-04-25T10:15:42.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.246+0000 7fc899600640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc87c06b550 0x7fc87c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.246+0000 7fc899600640 1 -- 172.21.15.119:0/91455059 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fc88c0ab4f0 con 0x7fc8800039b0 2024-04-25T10:15:42.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.247+0000 7fc8a0a00640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc87c06b550 0x7fc87c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.247+0000 7fc8a0a00640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc87c06b550 0x7fc87c06da10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0ca6d0 tx=0x7fc888008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:42.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.251+0000 7fc899600640 1 -- 172.21.15.119:0/91455059 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc88c07a510 con 0x7fc8800039b0 2024-04-25T10:15:42.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.461+0000 7fc8a2800640 1 -- 172.21.15.119:0/91455059 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc89c0b81f0 con 0x7fc8800039b0 2024-04-25T10:15:42.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.469+0000 7fc899600640 1 -- 172.21.15.119:0/91455059 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc88c079e10 con 0x7fc8800039b0 2024-04-25T10:15:42.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.471+0000 7fc8a2800640 1 -- 172.21.15.119:0/91455059 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc87c06b550 msgr2=0x7fc87c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:42.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.471+0000 7fc8a2800640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc87c06b550 0x7fc87c06da10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0ca6d0 tx=0x7fc888008040 comp rx=0 tx=0).stop 2024-04-25T10:15:42.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.471+0000 7fc8a2800640 1 -- 172.21.15.119:0/91455059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8800039b0 msgr2=0x7fc89c156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:42.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.471+0000 7fc8a2800640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8800039b0 0x7fc89c156e80 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00e200 tx=0x7fc88c009750 comp rx=0 tx=0).stop 2024-04-25T10:15:42.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.471+0000 7fc8a2800640 1 -- 172.21.15.119:0/91455059 shutdown_connections 2024-04-25T10:15:42.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.471+0000 7fc8a2800640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc87c06b550 0x7fc87c06da10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.471+0000 7fc8a2800640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc89c160b30 0x7fc89c159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.471+0000 7fc8a2800640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc89c153110 0x7fc89c1605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.471+0000 7fc8a2800640 1 --2- 172.21.15.119:0/91455059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8800039b0 0x7fc89c156e80 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.471+0000 7fc8a2800640 1 -- 172.21.15.119:0/91455059 >> 172.21.15.119:0/91455059 conn(0x7fc89c0726c0 msgr2=0x7fc89c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:42.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.472+0000 7fc8a2800640 1 -- 172.21.15.119:0/91455059 shutdown_connections 2024-04-25T10:15:42.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.472+0000 7fc8a2800640 1 -- 172.21.15.119:0/91455059 wait complete. 2024-04-25T10:15:42.495 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T10:15:42.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.678+0000 7f007c200640 1 Processor -- start 2024-04-25T10:15:42.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.678+0000 7f007c200640 1 -- start start 2024-04-25T10:15:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.680+0000 7f007c200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f007415bf00 0x7f00740cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.680+0000 7f007c200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00740cc310 0x7f007415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.680+0000 7f007c200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f007415c820 0x7f007406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.680+0000 7f007c200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00741455e0 con 0x7f00740cc310 2024-04-25T10:15:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.680+0000 7f007c200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007406d450 con 0x7f007415bf00 2024-04-25T10:15:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.680+0000 7f007c200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00740ce6c0 con 0x7f007415c820 2024-04-25T10:15:42.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.680+0000 7f0078e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00740cc310 0x7f007415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.681+0000 7f0078e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00740cc310 0x7f007415c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59342/0 (socket says 172.21.15.119:59342) 2024-04-25T10:15:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.681+0000 7f0078e00640 1 -- 172.21.15.119:0/1023649216 learned_addr learned my addr 172.21.15.119:0/1023649216 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.681+0000 7f0079800640 1 --2- 172.21.15.119:0/1023649216 >> v2:172.21.15.132:3300/0 conn(0x7f007415bf00 0x7f00740cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.681+0000 7f007a200640 1 --2- 172.21.15.119:0/1023649216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f007415c820 0x7f007406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.681+0000 7f0078e00640 1 -- 172.21.15.119:0/1023649216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f007415c820 msgr2=0x7f007406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.681+0000 7f0078e00640 1 --2- 172.21.15.119:0/1023649216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f007415c820 0x7f007406d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.681+0000 7f0078e00640 1 -- 172.21.15.119:0/1023649216 >> v2:172.21.15.132:3300/0 conn(0x7f007415bf00 msgr2=0x7f00740cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.681+0000 7f0078e00640 1 --2- 172.21.15.119:0/1023649216 >> v2:172.21.15.132:3300/0 conn(0x7f007415bf00 0x7f00740cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.681+0000 7f0078e00640 1 -- 172.21.15.119:0/1023649216 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00740cc7b0 con 0x7f00740cc310 2024-04-25T10:15:42.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.682+0000 7f0078e00640 1 --2- 172.21.15.119:0/1023649216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00740cc310 0x7f007415c2e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f007406ba10 tx=0x7f006002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d26cdf508ca0da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:42.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.683+0000 7f0073e00640 1 -- 172.21.15.119:0/1023649216 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060037070 con 0x7f00740cc310 2024-04-25T10:15:42.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.683+0000 7f0073e00640 1 -- 172.21.15.119:0/1023649216 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0060032440 con 0x7f00740cc310 2024-04-25T10:15:42.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.683+0000 7f0073e00640 1 -- 172.21.15.119:0/1023649216 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060032810 con 0x7f00740cc310 2024-04-25T10:15:42.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.683+0000 7f007c200640 1 -- 172.21.15.119:0/1023649216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00740cc310 msgr2=0x7f007415c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:42.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.683+0000 7f007c200640 1 --2- 172.21.15.119:0/1023649216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00740cc310 0x7f007415c2e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f007406ba10 tx=0x7f006002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.684+0000 7f007c200640 1 -- 172.21.15.119:0/1023649216 shutdown_connections 2024-04-25T10:15:42.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.684+0000 7f007c200640 1 --2- 172.21.15.119:0/1023649216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f007415c820 0x7f007406d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.684+0000 7f007c200640 1 --2- 172.21.15.119:0/1023649216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00740cc310 0x7f007415c2e0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.684+0000 7f007c200640 1 --2- 172.21.15.119:0/1023649216 >> v2:172.21.15.132:3300/0 conn(0x7f007415bf00 0x7f00740cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.684+0000 7f007c200640 1 -- 172.21.15.119:0/1023649216 >> 172.21.15.119:0/1023649216 conn(0x7f007406aba0 msgr2=0x7f007406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:42.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.684+0000 7f007c200640 1 -- 172.21.15.119:0/1023649216 shutdown_connections 2024-04-25T10:15:42.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.684+0000 7f007c200640 1 -- 172.21.15.119:0/1023649216 wait complete. 2024-04-25T10:15:42.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.686+0000 7f007c200640 1 Processor -- start 2024-04-25T10:15:42.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.686+0000 7f007c200640 1 -- start start 2024-04-25T10:15:42.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f007c200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740cc310 0x7f007406e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f007c200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f007415bf00 0x7f007406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f007c200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f007415c820 0x7f00741538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f007c200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00740cd050 con 0x7f007415bf00 2024-04-25T10:15:42.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f007c200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00740ccd70 con 0x7f007415c820 2024-04-25T10:15:42.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f007c200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007415ccd0 con 0x7f00740cc310 2024-04-25T10:15:42.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f0078e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f007415bf00 0x7f007406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f0078e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f007415bf00 0x7f007406e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59352/0 (socket says 172.21.15.119:59352) 2024-04-25T10:15:42.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f0078e00640 1 -- 172.21.15.119:0/866632480 learned_addr learned my addr 172.21.15.119:0/866632480 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:42.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f0079800640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740cc310 0x7f007406e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f007a200640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f007415c820 0x7f00741538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f0078e00640 1 -- 172.21.15.119:0/866632480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740cc310 msgr2=0x7f007406e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:42.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f0078e00640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740cc310 0x7f007406e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f0078e00640 1 -- 172.21.15.119:0/866632480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f007415c820 msgr2=0x7f00741538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:42.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.687+0000 7f0078e00640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f007415c820 0x7f00741538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:42.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.688+0000 7f0078e00640 1 -- 172.21.15.119:0/866632480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00741f7fa0 con 0x7f007415bf00 2024-04-25T10:15:42.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.688+0000 7f007a200640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f007415c820 0x7f00741538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:42.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.688+0000 7f0078e00640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f007415bf00 0x7f007406e7c0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0060000c00 tx=0x7f00600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:42.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.688+0000 7f0071600640 1 -- 172.21.15.119:0/866632480 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060041020 con 0x7f007415bf00 2024-04-25T10:15:42.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.688+0000 7f0071600640 1 -- 172.21.15.119:0/866632480 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00600306c0 con 0x7f007415bf00 2024-04-25T10:15:42.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.688+0000 7f0071600640 1 -- 172.21.15.119:0/866632480 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060037040 con 0x7f007415bf00 2024-04-25T10:15:42.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.688+0000 7f007c200640 1 -- 172.21.15.119:0/866632480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00741f71e0 con 0x7f007415bf00 2024-04-25T10:15:42.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.688+0000 7f007c200640 1 -- 172.21.15.119:0/866632480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00741f77a0 con 0x7f007415bf00 2024-04-25T10:15:42.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.689+0000 7f007c200640 1 -- 172.21.15.119:0/866632480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007406bb30 con 0x7f007415bf00 2024-04-25T10:15:42.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.690+0000 7f0071600640 1 -- 172.21.15.119:0/866632480 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0060044020 con 0x7f007415bf00 2024-04-25T10:15:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.691+0000 7f0071600640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f005406b4d0 0x7f005406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.691+0000 7f0071600640 1 -- 172.21.15.119:0/866632480 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f00600aa850 con 0x7f007415bf00 2024-04-25T10:15:42.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.692+0000 7f0079800640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f005406b4d0 0x7f005406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:42.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.692+0000 7f0079800640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f005406b4d0 0x7f005406d990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0064006fd0 tx=0x7f0064008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:42.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.696+0000 7f0071600640 1 -- 172.21.15.119:0/866632480 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f006004b030 con 0x7f007415bf00 2024-04-25T10:15:42.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:42.962+0000 7f007c200640 1 -- 172.21.15.119:0/866632480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7f0074067610 con 0x7f007415bf00 2024-04-25T10:15:43.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.025+0000 7f0071600640 1 -- 172.21.15.119:0/866632480 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v410) v1 ==== 167+0+0 (secure 0 0 0) 0x7f0074067610 con 0x7f007415bf00 2024-04-25T10:15:43.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.028+0000 7f007c200640 1 -- 172.21.15.119:0/866632480 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f005406b4d0 msgr2=0x7f005406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.028+0000 7f007c200640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f005406b4d0 0x7f005406d990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0064006fd0 tx=0x7f0064008040 comp rx=0 tx=0).stop 2024-04-25T10:15:43.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.028+0000 7f007c200640 1 -- 172.21.15.119:0/866632480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f007415bf00 msgr2=0x7f007406e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.028+0000 7f007c200640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f007415bf00 0x7f007406e7c0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0060000c00 tx=0x7f00600046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.028+0000 7f007c200640 1 -- 172.21.15.119:0/866632480 shutdown_connections 2024-04-25T10:15:43.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.028+0000 7f007c200640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f005406b4d0 0x7f005406d990 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.028+0000 7f007c200640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f007415c820 0x7f00741538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.028+0000 7f007c200640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f007415bf00 0x7f007406e7c0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.028+0000 7f007c200640 1 --2- 172.21.15.119:0/866632480 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00740cc310 0x7f007406e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.028+0000 7f007c200640 1 -- 172.21.15.119:0/866632480 >> 172.21.15.119:0/866632480 conn(0x7f007406aba0 msgr2=0x7f0074141a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.028+0000 7f007c200640 1 -- 172.21.15.119:0/866632480 shutdown_connections 2024-04-25T10:15:43.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.028+0000 7f007c200640 1 -- 172.21.15.119:0/866632480 wait complete. 2024-04-25T10:15:43.051 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:15:43.051 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:15:43.052 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:43.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.244+0000 7f5263a00640 1 Processor -- start 2024-04-25T10:15:43.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.244+0000 7f5263a00640 1 -- start start 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.246+0000 7f5263a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f525c15bf00 0x7f525c0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.246+0000 7f5263a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525c0ce6c0 con 0x7f525c0cc310 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.246+0000 7f5263a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525c0cc7b0 con 0x7f525c06e8f0 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.246+0000 7f5263a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525c1455e0 con 0x7f525c15bf00 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.246+0000 7f5261000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f525c15bf00 0x7f525c0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.246+0000 7f525be00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f525c0cc310 0x7f525c15c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47882/0 (socket says 172.21.15.119:47882) 2024-04-25T10:15:43.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.246+0000 7f525be00640 1 -- 172.21.15.119:0/1927773661 learned_addr learned my addr 172.21.15.119:0/1927773661 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:43.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.247+0000 7f5261000640 1 -- 172.21.15.119:0/1927773661 >> v1:172.21.15.132:6789/0 conn(0x7f525c06e8f0 legacy=0x7f525c1654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.247+0000 7f5261000640 1 -- 172.21.15.119:0/1927773661 >> v1:172.21.15.119:6789/0 conn(0x7f525c0cc310 legacy=0x7f525c15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.247+0000 7f5261000640 1 -- 172.21.15.119:0/1927773661 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c0cca90 con 0x7f525c15bf00 2024-04-25T10:15:43.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.247+0000 7f525b400640 1 -- 172.21.15.119:0/1927773661 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349059360 0 0) 0x7f5248001110 con 0x7f525c0cc310 2024-04-25T10:15:43.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.248+0000 7f5261000640 1 --2- 172.21.15.119:0/1927773661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525c15bf00 0x7f525c0cbdd0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f52500097a0 tx=0x7f525002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f1aba02ab12759 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.248+0000 7f525b400640 1 -- 172.21.15.119:0/1927773661 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250037070 con 0x7f525c15bf00 2024-04-25T10:15:43.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.248+0000 7f525b400640 1 -- 172.21.15.119:0/1927773661 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5250032440 con 0x7f525c15bf00 2024-04-25T10:15:43.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.248+0000 7f525b400640 1 -- 172.21.15.119:0/1927773661 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250032810 con 0x7f525c15bf00 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.249+0000 7f5263a00640 1 -- 172.21.15.119:0/1927773661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525c15bf00 msgr2=0x7f525c0cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.249+0000 7f5263a00640 1 --2- 172.21.15.119:0/1927773661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525c15bf00 0x7f525c0cbdd0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f52500097a0 tx=0x7f525002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.249+0000 7f5263a00640 1 -- 172.21.15.119:0/1927773661 shutdown_connections 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.249+0000 7f5263a00640 1 --2- 172.21.15.119:0/1927773661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525c15bf00 0x7f525c0cbdd0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.249+0000 7f5263a00640 1 -- 172.21.15.119:0/1927773661 >> 172.21.15.119:0/1927773661 conn(0x7f525c06aba0 msgr2=0x7f525c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.249+0000 7f5263a00640 1 -- 172.21.15.119:0/1927773661 shutdown_connections 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.249+0000 7f5263a00640 1 -- 172.21.15.119:0/1927773661 wait complete. 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.251+0000 7f5263a00640 1 Processor -- start 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.251+0000 7f5263a00640 1 -- start start 2024-04-25T10:15:43.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f5263a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f525c06e8f0 0x7f525c1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f5263a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525c0cc310 0x7f525c13a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f5263a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f525c15bf00 0x7f525c157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f5263a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0cd050 con 0x7f525c0cc310 2024-04-25T10:15:43.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f5263a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c06d450 con 0x7f525c06e8f0 2024-04-25T10:15:43.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f5263a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525c06ee60 con 0x7f525c15bf00 2024-04-25T10:15:43.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f525be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525c0cc310 0x7f525c13a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f525be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525c0cc310 0x7f525c13a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59364/0 (socket says 172.21.15.119:59364) 2024-04-25T10:15:43.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f525be00640 1 -- 172.21.15.119:0/2039608043 learned_addr learned my addr 172.21.15.119:0/2039608043 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:43.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f5261000640 1 --2- 172.21.15.119:0/2039608043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f525c06e8f0 0x7f525c1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f525be00640 1 -- 172.21.15.119:0/2039608043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f525c15bf00 msgr2=0x7f525c157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f525be00640 1 --2- 172.21.15.119:0/2039608043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f525c15bf00 0x7f525c157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f525be00640 1 -- 172.21.15.119:0/2039608043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f525c06e8f0 msgr2=0x7f525c1597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f525be00640 1 --2- 172.21.15.119:0/2039608043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f525c06e8f0 0x7f525c1597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.252+0000 7f525be00640 1 -- 172.21.15.119:0/2039608043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c156b30 con 0x7f525c0cc310 2024-04-25T10:15:43.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.253+0000 7f525be00640 1 --2- 172.21.15.119:0/2039608043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525c0cc310 0x7f525c13a6b0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f524800c750 tx=0x7f524800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.253+0000 7f5258c00640 1 -- 172.21.15.119:0/2039608043 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248018070 con 0x7f525c0cc310 2024-04-25T10:15:43.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.253+0000 7f5258c00640 1 -- 172.21.15.119:0/2039608043 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52480072d0 con 0x7f525c0cc310 2024-04-25T10:15:43.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.253+0000 7f5258c00640 1 -- 172.21.15.119:0/2039608043 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52480047e0 con 0x7f525c0cc310 2024-04-25T10:15:43.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.253+0000 7f5263a00640 1 -- 172.21.15.119:0/2039608043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525c156e10 con 0x7f525c0cc310 2024-04-25T10:15:43.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.253+0000 7f5263a00640 1 -- 172.21.15.119:0/2039608043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525c1573d0 con 0x7f525c0cc310 2024-04-25T10:15:43.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.254+0000 7f5263a00640 1 -- 172.21.15.119:0/2039608043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525c0749b0 con 0x7f525c0cc310 2024-04-25T10:15:43.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.255+0000 7f5258c00640 1 -- 172.21.15.119:0/2039608043 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5248004980 con 0x7f525c0cc310 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.256+0000 7f5258c00640 1 --2- 172.21.15.119:0/2039608043 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f523c06b4d0 0x7f523c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.256+0000 7f5258c00640 1 -- 172.21.15.119:0/2039608043 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f52480893a0 con 0x7f525c0cc310 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.256+0000 7f5261000640 1 --2- 172.21.15.119:0/2039608043 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f523c06b4d0 0x7f523c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.257+0000 7f5261000640 1 --2- 172.21.15.119:0/2039608043 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f523c06b4d0 0x7f523c06d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5250034000 tx=0x7f525002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.261+0000 7f5258c00640 1 -- 172.21.15.119:0/2039608043 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f525c0ce6c0 con 0x7f525c0cc310 2024-04-25T10:15:43.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.458+0000 7f5263a00640 1 -- 172.21.15.119:0/2039608043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f525c157770 con 0x7f525c0cc310 2024-04-25T10:15:43.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.459+0000 7f5258c00640 1 -- 172.21.15.119:0/2039608043 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+13323 (secure 0 0 0) 0x7f5248004c70 con 0x7f525c0cc310 2024-04-25T10:15:43.463 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:43.463 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":88,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:15:42.002534+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/1734507541":"2024-04-26T10:14:50.735180+0000","172.21.15.119:6837/2043821095":"2024-04-26T10:15:38.894291+0000","172.21.15.119:6835/2043821095":"2024-04-26T10:15:38.894291+0000","172.21.15.119:6835/1734507541":"2024-04-26T10:14:50.735180+0000","172.21.15.119:6837/3997964942":"2024-04-26T10:13:36.160373+0000","172.21.15.119:6835/3997964942":"2024-04-26T10:13:36.160373+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:43.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.461+0000 7f5263a00640 1 -- 172.21.15.119:0/2039608043 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f523c06b4d0 msgr2=0x7f523c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.461+0000 7f5263a00640 1 --2- 172.21.15.119:0/2039608043 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f523c06b4d0 0x7f523c06d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5250034000 tx=0x7f525002e000 comp rx=0 tx=0).stop 2024-04-25T10:15:43.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.461+0000 7f5263a00640 1 -- 172.21.15.119:0/2039608043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525c0cc310 msgr2=0x7f525c13a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.461+0000 7f5263a00640 1 --2- 172.21.15.119:0/2039608043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525c0cc310 0x7f525c13a6b0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f524800c750 tx=0x7f524800cc10 comp rx=0 tx=0).stop 2024-04-25T10:15:43.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.462+0000 7f5263a00640 1 -- 172.21.15.119:0/2039608043 shutdown_connections 2024-04-25T10:15:43.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.462+0000 7f5263a00640 1 --2- 172.21.15.119:0/2039608043 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f523c06b4d0 0x7f523c06d990 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.462+0000 7f5263a00640 1 --2- 172.21.15.119:0/2039608043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f525c15bf00 0x7f525c157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.462+0000 7f5263a00640 1 --2- 172.21.15.119:0/2039608043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525c0cc310 0x7f525c13a6b0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.462+0000 7f5263a00640 1 --2- 172.21.15.119:0/2039608043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f525c06e8f0 0x7f525c1597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.462+0000 7f5263a00640 1 -- 172.21.15.119:0/2039608043 >> 172.21.15.119:0/2039608043 conn(0x7f525c06aba0 msgr2=0x7f525c0cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.462+0000 7f5263a00640 1 -- 172.21.15.119:0/2039608043 shutdown_connections 2024-04-25T10:15:43.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.462+0000 7f5263a00640 1 -- 172.21.15.119:0/2039608043 wait complete. 2024-04-25T10:15:43.483 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:43.483 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:43.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.678+0000 7efc5e200640 1 Processor -- start 2024-04-25T10:15:43.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.678+0000 7efc5e200640 1 -- start start 2024-04-25T10:15:43.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.680+0000 7efc5e200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efc580b3500 0x7efc58166380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.680+0000 7efc5e200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc580cea30 0x7efc5806dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.680+0000 7efc5e200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc58162140 con 0x7efc58161d60 2024-04-25T10:15:43.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.680+0000 7efc5e200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc580b2470 con 0x7efc580cea30 2024-04-25T10:15:43.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.680+0000 7efc5e200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc58165ef0 con 0x7efc580b3500 2024-04-25T10:15:43.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.680+0000 7efc5d800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc580cea30 0x7efc5806dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.681+0000 7efc5d800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc580cea30 0x7efc5806dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59368/0 (socket says 172.21.15.119:59368) 2024-04-25T10:15:43.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.681+0000 7efc5d800640 1 -- 172.21.15.119:0/936256158 learned_addr learned my addr 172.21.15.119:0/936256158 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:43.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.681+0000 7efc4fe00640 1 --2- 172.21.15.119:0/936256158 >> v2:172.21.15.132:3300/0 conn(0x7efc580b3500 0x7efc58166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.681+0000 7efc5d800640 1 -- 172.21.15.119:0/936256158 >> v2:172.21.15.132:3300/0 conn(0x7efc580b3500 msgr2=0x7efc58166380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.681+0000 7efc5d800640 1 --2- 172.21.15.119:0/936256158 >> v2:172.21.15.132:3300/0 conn(0x7efc580b3500 0x7efc58166380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.681+0000 7efc5d800640 1 -- 172.21.15.119:0/936256158 >> v1:172.21.15.132:6789/0 conn(0x7efc58161d60 legacy=0x7efc580b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:43.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.681+0000 7efc5d800640 1 -- 172.21.15.119:0/936256158 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc580cf210 con 0x7efc580cea30 2024-04-25T10:15:43.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.682+0000 7efc5d800640 1 --2- 172.21.15.119:0/936256158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cea30 0x7efc5806dcf0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efc580b2060 tx=0x7efc5002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efef46a4cc57752 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.683+0000 7efc57e00640 1 -- 172.21.15.119:0/936256158 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50036070 con 0x7efc580cea30 2024-04-25T10:15:43.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.683+0000 7efc57e00640 1 -- 172.21.15.119:0/936256158 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc50031600 con 0x7efc580cea30 2024-04-25T10:15:43.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.683+0000 7efc57e00640 1 -- 172.21.15.119:0/936256158 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc500319d0 con 0x7efc580cea30 2024-04-25T10:15:43.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.683+0000 7efc5e200640 1 -- 172.21.15.119:0/936256158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cea30 msgr2=0x7efc5806dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.683+0000 7efc5e200640 1 --2- 172.21.15.119:0/936256158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cea30 0x7efc5806dcf0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efc580b2060 tx=0x7efc5002a660 comp rx=0 tx=0).stop 2024-04-25T10:15:43.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.683+0000 7efc5e200640 1 -- 172.21.15.119:0/936256158 shutdown_connections 2024-04-25T10:15:43.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.683+0000 7efc5e200640 1 --2- 172.21.15.119:0/936256158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cea30 0x7efc5806dcf0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.683+0000 7efc5e200640 1 --2- 172.21.15.119:0/936256158 >> v2:172.21.15.132:3300/0 conn(0x7efc580b3500 0x7efc58166380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.683+0000 7efc5e200640 1 -- 172.21.15.119:0/936256158 >> 172.21.15.119:0/936256158 conn(0x7efc58066ff0 msgr2=0x7efc580b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.683+0000 7efc5e200640 1 -- 172.21.15.119:0/936256158 shutdown_connections 2024-04-25T10:15:43.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.683+0000 7efc5e200640 1 -- 172.21.15.119:0/936256158 wait complete. 2024-04-25T10:15:43.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.685+0000 7efc5e200640 1 Processor -- start 2024-04-25T10:15:43.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.685+0000 7efc5e200640 1 -- start start 2024-04-25T10:15:43.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.686+0000 7efc5e200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc580b3500 0x7efc581608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.686+0000 7efc5e200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cea30 0x7efc581eb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.686+0000 7efc5e200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc58161d60 0x7efc5815b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.686+0000 7efc5e200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc581658a0 con 0x7efc580cea30 2024-04-25T10:15:43.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.686+0000 7efc5e200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc580b29a0 con 0x7efc58161d60 2024-04-25T10:15:43.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.686+0000 7efc5e200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc58165b80 con 0x7efc580b3500 2024-04-25T10:15:43.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.686+0000 7efc4fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cea30 0x7efc581eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.686+0000 7efc4fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cea30 0x7efc581eb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59384/0 (socket says 172.21.15.119:59384) 2024-04-25T10:15:43.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.686+0000 7efc4fe00640 1 -- 172.21.15.119:0/787311989 learned_addr learned my addr 172.21.15.119:0/787311989 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:43.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.687+0000 7efc5d800640 1 --2- 172.21.15.119:0/787311989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc58161d60 0x7efc5815b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.687+0000 7efc5ce00640 1 --2- 172.21.15.119:0/787311989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc580b3500 0x7efc581608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.687+0000 7efc4fe00640 1 -- 172.21.15.119:0/787311989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc580b3500 msgr2=0x7efc581608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.687+0000 7efc4fe00640 1 --2- 172.21.15.119:0/787311989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc580b3500 0x7efc581608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.687+0000 7efc4fe00640 1 -- 172.21.15.119:0/787311989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc58161d60 msgr2=0x7efc5815b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.687+0000 7efc4fe00640 1 --2- 172.21.15.119:0/787311989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc58161d60 0x7efc5815b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.687+0000 7efc4fe00640 1 -- 172.21.15.119:0/787311989 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc581ecf40 con 0x7efc580cea30 2024-04-25T10:15:43.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.687+0000 7efc4fe00640 1 --2- 172.21.15.119:0/787311989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cea30 0x7efc581eb890 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efc4000d800 tx=0x7efc4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.687+0000 7efc55600640 1 -- 172.21.15.119:0/787311989 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4000b6c0 con 0x7efc580cea30 2024-04-25T10:15:43.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.688+0000 7efc55600640 1 -- 172.21.15.119:0/787311989 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc40019070 con 0x7efc580cea30 2024-04-25T10:15:43.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.688+0000 7efc55600640 1 -- 172.21.15.119:0/787311989 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4000ba80 con 0x7efc580cea30 2024-04-25T10:15:43.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.688+0000 7efc5e200640 1 -- 172.21.15.119:0/787311989 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc581611d0 con 0x7efc580cea30 2024-04-25T10:15:43.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.688+0000 7efc5e200640 1 -- 172.21.15.119:0/787311989 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc581eb090 con 0x7efc580cea30 2024-04-25T10:15:43.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.689+0000 7efc5e200640 1 -- 172.21.15.119:0/787311989 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc580cef40 con 0x7efc580cea30 2024-04-25T10:15:43.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.690+0000 7efc55600640 1 -- 172.21.15.119:0/787311989 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc4000bc20 con 0x7efc580cea30 2024-04-25T10:15:43.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.691+0000 7efc55600640 1 --2- 172.21.15.119:0/787311989 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc3406b4d0 0x7efc3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:43.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.691+0000 7efc55600640 1 -- 172.21.15.119:0/787311989 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5115+0+0 (secure 0 0 0) 0x7efc4008a500 con 0x7efc580cea30 2024-04-25T10:15:43.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.691+0000 7efc5ce00640 1 --2- 172.21.15.119:0/787311989 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc3406b4d0 0x7efc3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:43.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.692+0000 7efc5ce00640 1 --2- 172.21.15.119:0/787311989 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc3406b4d0 0x7efc3406d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7efc4800a750 tx=0x7efc48008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:43.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.695+0000 7efc55600640 1 -- 172.21.15.119:0/787311989 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc580cef40 con 0x7efc580cea30 2024-04-25T10:15:43.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.876+0000 7efc5e200640 1 -- 172.21.15.119:0/787311989 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc580749b0 con 0x7efc580cea30 2024-04-25T10:15:43.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.877+0000 7efc55600640 1 -- 172.21.15.119:0/787311989 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc581ecf40 con 0x7efc580cea30 2024-04-25T10:15:43.881 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:15:43.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.880+0000 7efc5e200640 1 -- 172.21.15.119:0/787311989 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc3406b4d0 msgr2=0x7efc3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.880+0000 7efc5e200640 1 --2- 172.21.15.119:0/787311989 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc3406b4d0 0x7efc3406d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7efc4800a750 tx=0x7efc48008040 comp rx=0 tx=0).stop 2024-04-25T10:15:43.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.880+0000 7efc5e200640 1 -- 172.21.15.119:0/787311989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cea30 msgr2=0x7efc581eb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:43.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.880+0000 7efc5e200640 1 --2- 172.21.15.119:0/787311989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cea30 0x7efc581eb890 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efc4000d800 tx=0x7efc4000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.880+0000 7efc5e200640 1 -- 172.21.15.119:0/787311989 shutdown_connections 2024-04-25T10:15:43.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.880+0000 7efc5e200640 1 --2- 172.21.15.119:0/787311989 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc3406b4d0 0x7efc3406d990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.880+0000 7efc5e200640 1 --2- 172.21.15.119:0/787311989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc58161d60 0x7efc5815b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.880+0000 7efc5e200640 1 --2- 172.21.15.119:0/787311989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cea30 0x7efc581eb890 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.880+0000 7efc5e200640 1 --2- 172.21.15.119:0/787311989 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc580b3500 0x7efc581608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:43.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.880+0000 7efc5e200640 1 -- 172.21.15.119:0/787311989 >> 172.21.15.119:0/787311989 conn(0x7efc58066ff0 msgr2=0x7efc581536b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:43.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.880+0000 7efc5e200640 1 -- 172.21.15.119:0/787311989 shutdown_connections 2024-04-25T10:15:43.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:43.880+0000 7efc5e200640 1 -- 172.21.15.119:0/787311989 wait complete. 2024-04-25T10:15:43.902 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T10:15:44.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.090+0000 7fb8adc00640 1 Processor -- start 2024-04-25T10:15:44.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.091+0000 7fb8adc00640 1 -- start start 2024-04-25T10:15:44.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.092+0000 7fb8adc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb8a80aecf0 0x7fb8a80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.092+0000 7fb8adc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8a815ed70 0x7fb8a80cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.092+0000 7fb8adc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a80712c0 con 0x7fb8a815d980 2024-04-25T10:15:44.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.092+0000 7fb8adc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a8071ed0 con 0x7fb8a80aecf0 2024-04-25T10:15:44.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.092+0000 7fb8adc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80715a0 con 0x7fb8a815ed70 2024-04-25T10:15:44.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.093+0000 7fb8a7400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8a815ed70 0x7fb8a80cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.093+0000 7fb8a6a00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fb8a815d980 0x7fb8a80ae5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:54460/0 (socket says 172.21.15.119:54460) 2024-04-25T10:15:44.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.093+0000 7fb8a6a00640 1 -- 172.21.15.119:0/1848224456 learned_addr learned my addr 172.21.15.119:0/1848224456 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:44.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.093+0000 7fb89fe00640 1 --2- 172.21.15.119:0/1848224456 >> v2:172.21.15.132:3300/0 conn(0x7fb8a80aecf0 0x7fb8a80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.094+0000 7fb8a6000640 1 -- 172.21.15.119:0/1848224456 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419243719 0 0) 0x7fb8a80712c0 con 0x7fb8a815d980 2024-04-25T10:15:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.094+0000 7fb8a7400640 1 -- 172.21.15.119:0/1848224456 >> v2:172.21.15.132:3300/0 conn(0x7fb8a80aecf0 msgr2=0x7fb8a80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.094+0000 7fb8a7400640 1 --2- 172.21.15.119:0/1848224456 >> v2:172.21.15.132:3300/0 conn(0x7fb8a80aecf0 0x7fb8a80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.094+0000 7fb8a7400640 1 -- 172.21.15.119:0/1848224456 >> v1:172.21.15.132:6789/0 conn(0x7fb8a815d980 legacy=0x7fb8a80ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.094+0000 7fb8a7400640 1 -- 172.21.15.119:0/1848224456 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a8071880 con 0x7fb8a815ed70 2024-04-25T10:15:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.094+0000 7fb89fe00640 1 --2- 172.21.15.119:0/1848224456 >> v2:172.21.15.132:3300/0 conn(0x7fb8a80aecf0 0x7fb8a80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.095+0000 7fb8a7400640 1 --2- 172.21.15.119:0/1848224456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8a815ed70 0x7fb8a80cf7c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009d90 tx=0x7fb8a002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4a27a36fb84d316 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.096+0000 7fb8a6000640 1 -- 172.21.15.119:0/1848224456 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0031600 con 0x7fb8a815ed70 2024-04-25T10:15:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.096+0000 7fb8a6000640 1 -- 172.21.15.119:0/1848224456 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a00317a0 con 0x7fb8a815ed70 2024-04-25T10:15:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.096+0000 7fb8a6000640 1 -- 172.21.15.119:0/1848224456 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0031b70 con 0x7fb8a815ed70 2024-04-25T10:15:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.096+0000 7fb8adc00640 1 -- 172.21.15.119:0/1848224456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8a815ed70 msgr2=0x7fb8a80cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.096+0000 7fb8adc00640 1 --2- 172.21.15.119:0/1848224456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8a815ed70 0x7fb8a80cf7c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009d90 tx=0x7fb8a002a5d0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.096+0000 7fb8adc00640 1 -- 172.21.15.119:0/1848224456 shutdown_connections 2024-04-25T10:15:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.096+0000 7fb8adc00640 1 --2- 172.21.15.119:0/1848224456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8a815ed70 0x7fb8a80cf7c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.096+0000 7fb8adc00640 1 --2- 172.21.15.119:0/1848224456 >> v2:172.21.15.132:3300/0 conn(0x7fb8a80aecf0 0x7fb8a80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.096+0000 7fb8adc00640 1 -- 172.21.15.119:0/1848224456 >> 172.21.15.119:0/1848224456 conn(0x7fb8a806aba0 msgr2=0x7fb8a806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.097+0000 7fb8adc00640 1 -- 172.21.15.119:0/1848224456 shutdown_connections 2024-04-25T10:15:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.097+0000 7fb8adc00640 1 -- 172.21.15.119:0/1848224456 wait complete. 2024-04-25T10:15:44.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.098+0000 7fb8adc00640 1 Processor -- start 2024-04-25T10:15:44.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.098+0000 7fb8adc00640 1 -- start start 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.099+0000 7fb8adc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8a815d980 0x7fb8a80cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.099+0000 7fb8adc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a814ebd0 0x7fb8a81552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.099+0000 7fb8adc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8a814f120 0x7fb8a814dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.099+0000 7fb8adc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80ad0f0 con 0x7fb8a814ebd0 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.099+0000 7fb8adc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80acf70 con 0x7fb8a814f120 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.099+0000 7fb8adc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80ad270 con 0x7fb8a815d980 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.100+0000 7fb89fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a814ebd0 0x7fb8a81552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.100+0000 7fb89fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a814ebd0 0x7fb8a81552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59396/0 (socket says 172.21.15.119:59396) 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.100+0000 7fb89fe00640 1 -- 172.21.15.119:0/2787012968 learned_addr learned my addr 172.21.15.119:0/2787012968 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.100+0000 7fb8a6a00640 1 --2- 172.21.15.119:0/2787012968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8a815d980 0x7fb8a80cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.100+0000 7fb89fe00640 1 -- 172.21.15.119:0/2787012968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8a815d980 msgr2=0x7fb8a80cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.100+0000 7fb89fe00640 1 --2- 172.21.15.119:0/2787012968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8a815d980 0x7fb8a80cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.100+0000 7fb8a7400640 1 --2- 172.21.15.119:0/2787012968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8a814f120 0x7fb8a814dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.100+0000 7fb89fe00640 1 -- 172.21.15.119:0/2787012968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8a814f120 msgr2=0x7fb8a814dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:44.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.100+0000 7fb89fe00640 1 --2- 172.21.15.119:0/2787012968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8a814f120 0x7fb8a814dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:44.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.100+0000 7fb89fe00640 1 -- 172.21.15.119:0/2787012968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a8154d10 con 0x7fb8a814ebd0 2024-04-25T10:15:44.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.100+0000 7fb89fe00640 1 --2- 172.21.15.119:0/2787012968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a814ebd0 0x7fb8a81552c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fb89000e9b0 tx=0x7fb89000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:44.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.101+0000 7fb89ea00640 1 -- 172.21.15.119:0/2787012968 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890017070 con 0x7fb8a814ebd0 2024-04-25T10:15:44.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.101+0000 7fb89ea00640 1 -- 172.21.15.119:0/2787012968 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89000ca00 con 0x7fb8a814ebd0 2024-04-25T10:15:44.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.101+0000 7fb8adc00640 1 -- 172.21.15.119:0/2787012968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a8154a30 con 0x7fb8a814ebd0 2024-04-25T10:15:44.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.101+0000 7fb89ea00640 1 -- 172.21.15.119:0/2787012968 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89000cdf0 con 0x7fb8a814ebd0 2024-04-25T10:15:44.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.101+0000 7fb8adc00640 1 -- 172.21.15.119:0/2787012968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a8150520 con 0x7fb8a814ebd0 2024-04-25T10:15:44.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.102+0000 7fb8adc00640 1 -- 172.21.15.119:0/2787012968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a80749b0 con 0x7fb8a814ebd0 2024-04-25T10:15:44.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.103+0000 7fb89ea00640 1 -- 172.21.15.119:0/2787012968 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb89001a070 con 0x7fb8a814ebd0 2024-04-25T10:15:44.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.104+0000 7fb89ea00640 1 --2- 172.21.15.119:0/2787012968 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb88406b4d0 0x7fb88406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:44.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.105+0000 7fb89ea00640 1 -- 172.21.15.119:0/2787012968 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fb890089fe0 con 0x7fb8a814ebd0 2024-04-25T10:15:44.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.105+0000 7fb8a6a00640 1 --2- 172.21.15.119:0/2787012968 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb88406b4d0 0x7fb88406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:44.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.105+0000 7fb8a6a00640 1 --2- 172.21.15.119:0/2787012968 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb88406b4d0 0x7fb88406d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb898007fd0 tx=0x7fb898009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:44.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.109+0000 7fb89ea00640 1 -- 172.21.15.119:0/2787012968 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb890059010 con 0x7fb8a814ebd0 2024-04-25T10:15:44.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:44.375+0000 7fb8adc00640 1 -- 172.21.15.119:0/2787012968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7fb8a806d4c0 con 0x7fb8a814ebd0 2024-04-25T10:15:45.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.040+0000 7fb89ea00640 1 -- 172.21.15.119:0/2787012968 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v412) v1 ==== 149+0+0 (secure 0 0 0) 0x7fb890058910 con 0x7fb8a814ebd0 2024-04-25T10:15:45.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.045+0000 7fb8adc00640 1 -- 172.21.15.119:0/2787012968 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb88406b4d0 msgr2=0x7fb88406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.045+0000 7fb8adc00640 1 --2- 172.21.15.119:0/2787012968 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb88406b4d0 0x7fb88406d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb898007fd0 tx=0x7fb898009040 comp rx=0 tx=0).stop 2024-04-25T10:15:45.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.045+0000 7fb8adc00640 1 -- 172.21.15.119:0/2787012968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a814ebd0 msgr2=0x7fb8a81552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.045+0000 7fb8adc00640 1 --2- 172.21.15.119:0/2787012968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a814ebd0 0x7fb8a81552c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fb89000e9b0 tx=0x7fb89000ee70 comp rx=0 tx=0).stop 2024-04-25T10:15:45.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.045+0000 7fb8adc00640 1 -- 172.21.15.119:0/2787012968 shutdown_connections 2024-04-25T10:15:45.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.045+0000 7fb8adc00640 1 --2- 172.21.15.119:0/2787012968 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb88406b4d0 0x7fb88406d990 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.045+0000 7fb8adc00640 1 --2- 172.21.15.119:0/2787012968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8a814f120 0x7fb8a814dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.045+0000 7fb8adc00640 1 --2- 172.21.15.119:0/2787012968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a814ebd0 0x7fb8a81552c0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.045+0000 7fb8adc00640 1 --2- 172.21.15.119:0/2787012968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb8a815d980 0x7fb8a80cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.045+0000 7fb8adc00640 1 -- 172.21.15.119:0/2787012968 >> 172.21.15.119:0/2787012968 conn(0x7fb8a806aba0 msgr2=0x7fb8a80af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:45.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.045+0000 7fb8adc00640 1 -- 172.21.15.119:0/2787012968 shutdown_connections 2024-04-25T10:15:45.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.045+0000 7fb8adc00640 1 -- 172.21.15.119:0/2787012968 wait complete. 2024-04-25T10:15:45.071 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:15:45.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.259+0000 7fcd9ba00640 1 Processor -- start 2024-04-25T10:15:45.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.259+0000 7fcd9ba00640 1 -- start start 2024-04-25T10:15:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.261+0000 7fcd9ba00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd9415dca0 0x7fcd940ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.261+0000 7fcd9ba00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd940d3b90 0x7fcd940d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.261+0000 7fcd9ba00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcd940cee80 0x7fcd940d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.261+0000 7fcd9ba00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd940b6b20 con 0x7fcd9415dca0 2024-04-25T10:15:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.261+0000 7fcd9ba00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd940d4090 con 0x7fcd940cee80 2024-04-25T10:15:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.261+0000 7fcd9ba00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940d4370 con 0x7fcd940d3b90 2024-04-25T10:15:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.261+0000 7fcd99000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd9415dca0 0x7fcd940ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.261+0000 7fcd99000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd9415dca0 0x7fcd940ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59412/0 (socket says 172.21.15.119:59412) 2024-04-25T10:15:45.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.261+0000 7fcd99000640 1 -- 172.21.15.119:0/2026857777 learned_addr learned my addr 172.21.15.119:0/2026857777 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:45.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.262+0000 7fcd99000640 1 -- 172.21.15.119:0/2026857777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd940d3b90 msgr2=0x7fcd940d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:45.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.262+0000 7fcd99000640 1 --2- 172.21.15.119:0/2026857777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd940d3b90 0x7fcd940d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.262+0000 7fcd99000640 1 -- 172.21.15.119:0/2026857777 >> v2:172.21.15.132:3300/0 conn(0x7fcd940cee80 msgr2=0x7fcd940d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:45.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.262+0000 7fcd99000640 1 --2- 172.21.15.119:0/2026857777 >> v2:172.21.15.132:3300/0 conn(0x7fcd940cee80 0x7fcd940d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.262+0000 7fcd99000640 1 -- 172.21.15.119:0/2026857777 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd940d4650 con 0x7fcd9415dca0 2024-04-25T10:15:45.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.263+0000 7fcd99000640 1 --2- 172.21.15.119:0/2026857777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9415dca0 0x7fcd940ce940 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fcd940d04c0 tx=0x7fcd8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe3f27dd4fe5686 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:45.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.263+0000 7fcd93e00640 1 -- 172.21.15.119:0/2026857777 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84037070 con 0x7fcd9415dca0 2024-04-25T10:15:45.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.263+0000 7fcd93e00640 1 -- 172.21.15.119:0/2026857777 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd84032440 con 0x7fcd9415dca0 2024-04-25T10:15:45.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.263+0000 7fcd93e00640 1 -- 172.21.15.119:0/2026857777 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84032810 con 0x7fcd9415dca0 2024-04-25T10:15:45.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.264+0000 7fcd9ba00640 1 -- 172.21.15.119:0/2026857777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9415dca0 msgr2=0x7fcd940ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.264+0000 7fcd9ba00640 1 --2- 172.21.15.119:0/2026857777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9415dca0 0x7fcd940ce940 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fcd940d04c0 tx=0x7fcd8402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.264+0000 7fcd9ba00640 1 -- 172.21.15.119:0/2026857777 shutdown_connections 2024-04-25T10:15:45.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.264+0000 7fcd9ba00640 1 --2- 172.21.15.119:0/2026857777 >> v2:172.21.15.132:3300/0 conn(0x7fcd940cee80 0x7fcd940d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.264+0000 7fcd9ba00640 1 --2- 172.21.15.119:0/2026857777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd940d3b90 0x7fcd940d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.264+0000 7fcd9ba00640 1 --2- 172.21.15.119:0/2026857777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9415dca0 0x7fcd940ce940 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.264+0000 7fcd9ba00640 1 -- 172.21.15.119:0/2026857777 >> 172.21.15.119:0/2026857777 conn(0x7fcd940726c0 msgr2=0x7fcd94072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:45.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.264+0000 7fcd9ba00640 1 -- 172.21.15.119:0/2026857777 shutdown_connections 2024-04-25T10:15:45.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.264+0000 7fcd9ba00640 1 -- 172.21.15.119:0/2026857777 wait complete. 2024-04-25T10:15:45.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.265+0000 7fcd9ba00640 1 Processor -- start 2024-04-25T10:15:45.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.266+0000 7fcd9ba00640 1 -- start start 2024-04-25T10:15:45.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.266+0000 7fcd9ba00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd940cee80 0x7fcd941550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd9ba00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd940d3b90 0x7fcd9415aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd9ba00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd9415dca0 0x7fcd940cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd9ba00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9415e200 con 0x7fcd940cee80 2024-04-25T10:15:45.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd9ba00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9415e080 con 0x7fcd940d3b90 2024-04-25T10:15:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd9ba00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940b2500 con 0x7fcd9415dca0 2024-04-25T10:15:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd99000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd940cee80 0x7fcd941550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd99000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd940cee80 0x7fcd941550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59418/0 (socket says 172.21.15.119:59418) 2024-04-25T10:15:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd99000640 1 -- 172.21.15.119:0/262815854 learned_addr learned my addr 172.21.15.119:0/262815854 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd99a00640 1 --2- 172.21.15.119:0/262815854 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd9415dca0 0x7fcd940cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd8be00640 1 --2- 172.21.15.119:0/262815854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd940d3b90 0x7fcd9415aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd99000640 1 -- 172.21.15.119:0/262815854 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd9415dca0 msgr2=0x7fcd940cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd99000640 1 --2- 172.21.15.119:0/262815854 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd9415dca0 0x7fcd940cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd99000640 1 -- 172.21.15.119:0/262815854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd940d3b90 msgr2=0x7fcd9415aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd99000640 1 --2- 172.21.15.119:0/262815854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd940d3b90 0x7fcd9415aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.267+0000 7fcd99000640 1 -- 172.21.15.119:0/262815854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd940cc320 con 0x7fcd940cee80 2024-04-25T10:15:45.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.268+0000 7fcd99000640 1 --2- 172.21.15.119:0/262815854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd940cee80 0x7fcd941550f0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fcd84000c00 tx=0x7fcd840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:45.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.268+0000 7fcd91600640 1 -- 172.21.15.119:0/262815854 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84041070 con 0x7fcd940cee80 2024-04-25T10:15:45.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.268+0000 7fcd91600640 1 -- 172.21.15.119:0/262815854 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8403c870 con 0x7fcd940cee80 2024-04-25T10:15:45.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.268+0000 7fcd91600640 1 -- 172.21.15.119:0/262815854 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84037040 con 0x7fcd940cee80 2024-04-25T10:15:45.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.268+0000 7fcd9ba00640 1 -- 172.21.15.119:0/262815854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd940cc8e0 con 0x7fcd940cee80 2024-04-25T10:15:45.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.268+0000 7fcd9ba00640 1 -- 172.21.15.119:0/262815854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd940ccea0 con 0x7fcd940cee80 2024-04-25T10:15:45.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.269+0000 7fcd9ba00640 1 -- 172.21.15.119:0/262815854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd940750b0 con 0x7fcd940cee80 2024-04-25T10:15:45.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.270+0000 7fcd91600640 1 -- 172.21.15.119:0/262815854 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd84044070 con 0x7fcd940cee80 2024-04-25T10:15:45.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.271+0000 7fcd91600640 1 --2- 172.21.15.119:0/262815854 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd7006b4d0 0x7fcd7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:45.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.271+0000 7fcd91600640 1 -- 172.21.15.119:0/262815854 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fcd840aab70 con 0x7fcd940cee80 2024-04-25T10:15:45.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.272+0000 7fcd8be00640 1 --2- 172.21.15.119:0/262815854 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd7006b4d0 0x7fcd7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:45.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.272+0000 7fcd8be00640 1 --2- 172.21.15.119:0/262815854 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd7006b4d0 0x7fcd7006d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcd9415c4c0 tx=0x7fcd7c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:45.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.276+0000 7fcd91600640 1 -- 172.21.15.119:0/262815854 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd84079c70 con 0x7fcd940cee80 2024-04-25T10:15:45.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.474+0000 7fcd9ba00640 1 -- 172.21.15.119:0/262815854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcd940d33f0 con 0x7fcd940cee80 2024-04-25T10:15:45.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.480+0000 7fcd91600640 1 -- 172.21.15.119:0/262815854 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcd84079570 con 0x7fcd940cee80 2024-04-25T10:15:45.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.483+0000 7fcd9ba00640 1 -- 172.21.15.119:0/262815854 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd7006b4d0 msgr2=0x7fcd7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.483+0000 7fcd9ba00640 1 --2- 172.21.15.119:0/262815854 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd7006b4d0 0x7fcd7006d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcd9415c4c0 tx=0x7fcd7c008040 comp rx=0 tx=0).stop 2024-04-25T10:15:45.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.483+0000 7fcd9ba00640 1 -- 172.21.15.119:0/262815854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd940cee80 msgr2=0x7fcd941550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:45.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.483+0000 7fcd9ba00640 1 --2- 172.21.15.119:0/262815854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd940cee80 0x7fcd941550f0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fcd84000c00 tx=0x7fcd840046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.483+0000 7fcd9ba00640 1 -- 172.21.15.119:0/262815854 shutdown_connections 2024-04-25T10:15:45.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.483+0000 7fcd9ba00640 1 --2- 172.21.15.119:0/262815854 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd7006b4d0 0x7fcd7006d990 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.483+0000 7fcd9ba00640 1 --2- 172.21.15.119:0/262815854 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd9415dca0 0x7fcd940cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.483+0000 7fcd9ba00640 1 --2- 172.21.15.119:0/262815854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd940d3b90 0x7fcd9415aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.483+0000 7fcd9ba00640 1 --2- 172.21.15.119:0/262815854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd940cee80 0x7fcd941550f0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:45.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.483+0000 7fcd9ba00640 1 -- 172.21.15.119:0/262815854 >> 172.21.15.119:0/262815854 conn(0x7fcd940726c0 msgr2=0x7fcd940d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:45.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.483+0000 7fcd9ba00640 1 -- 172.21.15.119:0/262815854 shutdown_connections 2024-04-25T10:15:45.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:45.483+0000 7fcd9ba00640 1 -- 172.21.15.119:0/262815854 wait complete. 2024-04-25T10:15:45.505 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:15:45.505 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:45.603 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:45.706 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:15:45.707 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:15:45.707 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:45.765 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:45.765 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:15:45.765 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:45.822 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:45.822 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:15:45.822 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:45.878 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:45.878 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:15:45.878 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:45.935 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:45.935 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:15:45.936 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:46.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.143+0000 7fbd78800640 1 Processor -- start 2024-04-25T10:15:46.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.143+0000 7fbd78800640 1 -- start start 2024-04-25T10:15:46.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.145+0000 7fbd78800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbd7015a870 0x7fbd700cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.145+0000 7fbd78800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbd7006e2e0 0x7fbd7015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.145+0000 7fbd78800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd700cd580 0x7fbd700cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.145+0000 7fbd78800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd700ce720 con 0x7fbd7006e2e0 2024-04-25T10:15:46.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.145+0000 7fbd78800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd700cdb10 con 0x7fbd7015a870 2024-04-25T10:15:46.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.145+0000 7fbd78800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700cddf0 con 0x7fbd700cd580 2024-04-25T10:15:46.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.145+0000 7fbd75400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbd7006e2e0 0x7fbd7015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.145+0000 7fbd75400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbd7006e2e0 0x7fbd7015ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59422/0 (socket says 172.21.15.119:59422) 2024-04-25T10:15:46.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.145+0000 7fbd75400640 1 -- 172.21.15.119:0/988024232 learned_addr learned my addr 172.21.15.119:0/988024232 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:46.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.145+0000 7fbd75400640 1 -- 172.21.15.119:0/988024232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd700cd580 msgr2=0x7fbd700cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:46.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.145+0000 7fbd75400640 1 --2- 172.21.15.119:0/988024232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd700cd580 0x7fbd700cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.145+0000 7fbd75400640 1 -- 172.21.15.119:0/988024232 >> v2:172.21.15.132:3300/0 conn(0x7fbd7015a870 msgr2=0x7fbd700cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:46.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.145+0000 7fbd75e00640 1 --2- 172.21.15.119:0/988024232 >> v2:172.21.15.132:3300/0 conn(0x7fbd7015a870 0x7fbd700cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.146+0000 7fbd75400640 1 --2- 172.21.15.119:0/988024232 >> v2:172.21.15.132:3300/0 conn(0x7fbd7015a870 0x7fbd700cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.146+0000 7fbd75400640 1 -- 172.21.15.119:0/988024232 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd700ce0d0 con 0x7fbd7006e2e0 2024-04-25T10:15:46.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.146+0000 7fbd75e00640 1 --2- 172.21.15.119:0/988024232 >> v2:172.21.15.132:3300/0 conn(0x7fbd7015a870 0x7fbd700cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:46.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.147+0000 7fbd75400640 1 --2- 172.21.15.119:0/988024232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7006e2e0 0x7fbd7015ac50 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fbd7006ba10 tx=0x7fbd6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c78d383cf37859 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:46.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.147+0000 7fbd74a00640 1 -- 172.21.15.119:0/988024232 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60037070 con 0x7fbd7006e2e0 2024-04-25T10:15:46.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.147+0000 7fbd74a00640 1 -- 172.21.15.119:0/988024232 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd60032440 con 0x7fbd7006e2e0 2024-04-25T10:15:46.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.147+0000 7fbd74a00640 1 -- 172.21.15.119:0/988024232 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60032810 con 0x7fbd7006e2e0 2024-04-25T10:15:46.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.148+0000 7fbd78800640 1 -- 172.21.15.119:0/988024232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7006e2e0 msgr2=0x7fbd7015ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.148+0000 7fbd78800640 1 --2- 172.21.15.119:0/988024232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7006e2e0 0x7fbd7015ac50 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fbd7006ba10 tx=0x7fbd6002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.148+0000 7fbd78800640 1 -- 172.21.15.119:0/988024232 shutdown_connections 2024-04-25T10:15:46.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.148+0000 7fbd78800640 1 --2- 172.21.15.119:0/988024232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd700cd580 0x7fbd700cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.148+0000 7fbd78800640 1 --2- 172.21.15.119:0/988024232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7006e2e0 0x7fbd7015ac50 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.148+0000 7fbd78800640 1 --2- 172.21.15.119:0/988024232 >> v2:172.21.15.132:3300/0 conn(0x7fbd7015a870 0x7fbd700cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.148+0000 7fbd78800640 1 -- 172.21.15.119:0/988024232 >> 172.21.15.119:0/988024232 conn(0x7fbd7006aba0 msgr2=0x7fbd7006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:46.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.148+0000 7fbd78800640 1 -- 172.21.15.119:0/988024232 shutdown_connections 2024-04-25T10:15:46.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.148+0000 7fbd78800640 1 -- 172.21.15.119:0/988024232 wait complete. 2024-04-25T10:15:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.150+0000 7fbd78800640 1 Processor -- start 2024-04-25T10:15:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.150+0000 7fbd78800640 1 -- start start 2024-04-25T10:15:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.151+0000 7fbd78800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd7006e2e0 0x7fbd700a8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.151+0000 7fbd78800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd700cd580 0x7fbd701480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.151+0000 7fbd78800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd7015a870 0x7fbd701e39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.151+0000 7fbd78800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700cfbf0 con 0x7fbd700cd580 2024-04-25T10:15:46.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.151+0000 7fbd78800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700cfa70 con 0x7fbd7006e2e0 2024-04-25T10:15:46.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.151+0000 7fbd78800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700cfd70 con 0x7fbd7015a870 2024-04-25T10:15:46.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.151+0000 7fbd75400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd700cd580 0x7fbd701480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.151+0000 7fbd75400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd700cd580 0x7fbd701480e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59430/0 (socket says 172.21.15.119:59430) 2024-04-25T10:15:46.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.151+0000 7fbd75400640 1 -- 172.21.15.119:0/1338372647 learned_addr learned my addr 172.21.15.119:0/1338372647 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:46.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.151+0000 7fbd75400640 1 -- 172.21.15.119:0/1338372647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd7015a870 msgr2=0x7fbd701e39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.151+0000 7fbd75400640 1 --2- 172.21.15.119:0/1338372647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd7015a870 0x7fbd701e39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.151+0000 7fbd75400640 1 -- 172.21.15.119:0/1338372647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd7006e2e0 msgr2=0x7fbd700a8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.151+0000 7fbd75400640 1 --2- 172.21.15.119:0/1338372647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd7006e2e0 0x7fbd700a8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.151+0000 7fbd75400640 1 -- 172.21.15.119:0/1338372647 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd70148620 con 0x7fbd700cd580 2024-04-25T10:15:46.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.152+0000 7fbd75400640 1 --2- 172.21.15.119:0/1338372647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd700cd580 0x7fbd701480e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fbd6002a220 tx=0x7fbd600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:46.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.152+0000 7fbd6a000640 1 -- 172.21.15.119:0/1338372647 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60037040 con 0x7fbd700cd580 2024-04-25T10:15:46.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.152+0000 7fbd6a000640 1 -- 172.21.15.119:0/1338372647 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd60042070 con 0x7fbd700cd580 2024-04-25T10:15:46.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.152+0000 7fbd6a000640 1 -- 172.21.15.119:0/1338372647 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60030bb0 con 0x7fbd700cd580 2024-04-25T10:15:46.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.152+0000 7fbd78800640 1 -- 172.21.15.119:0/1338372647 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd701e4220 con 0x7fbd700cd580 2024-04-25T10:15:46.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.152+0000 7fbd78800640 1 -- 172.21.15.119:0/1338372647 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd701e47e0 con 0x7fbd700cd580 2024-04-25T10:15:46.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.153+0000 7fbd78800640 1 -- 172.21.15.119:0/1338372647 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd700749b0 con 0x7fbd700cd580 2024-04-25T10:15:46.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.154+0000 7fbd6a000640 1 -- 172.21.15.119:0/1338372647 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd60047460 con 0x7fbd700cd580 2024-04-25T10:15:46.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.155+0000 7fbd6a000640 1 --2- 172.21.15.119:0/1338372647 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd5406b4d0 0x7fbd5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.155+0000 7fbd6a000640 1 -- 172.21.15.119:0/1338372647 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fbd600aad00 con 0x7fbd700cd580 2024-04-25T10:15:46.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.155+0000 7fbd75e00640 1 --2- 172.21.15.119:0/1338372647 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd5406b4d0 0x7fbd5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.156+0000 7fbd75e00640 1 --2- 172.21.15.119:0/1338372647 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd5406b4d0 0x7fbd5406d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbd64006fd0 tx=0x7fbd64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:46.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.160+0000 7fbd6a000640 1 -- 172.21.15.119:0/1338372647 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd60045360 con 0x7fbd700cd580 2024-04-25T10:15:46.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.391+0000 7fbd78800640 1 -- 172.21.15.119:0/1338372647 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd701e3f30 con 0x7fbd700cd580 2024-04-25T10:15:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.393+0000 7fbd6a000640 1 -- 172.21.15.119:0/1338372647 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+3711 (secure 0 0 0) 0x7fbd60079d30 con 0x7fbd700cd580 2024-04-25T10:15:46.397 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:46.397 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/1052781019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1052781019},{"type":"v1","addr":"172.21.15.132:6833","nonce":1052781019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3820828584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3820828584},{"type":"v1","addr":"172.21.15.132:6835","nonce":3820828584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7155,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1637958253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1637958253},{"type":"v1","addr":"172.21.15.119:6836","nonce":1637958253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7950,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2286604643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2286604643},{"type":"v1","addr":"172.21.15.119:6837","nonce":2286604643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[]} 2024-04-25T10:15:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 76 2024-04-25T10:15:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.395+0000 7fbd78800640 1 -- 172.21.15.119:0/1338372647 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd5406b4d0 msgr2=0x7fbd5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.395+0000 7fbd78800640 1 --2- 172.21.15.119:0/1338372647 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd5406b4d0 0x7fbd5406d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbd64006fd0 tx=0x7fbd64008040 comp rx=0 tx=0).stop 2024-04-25T10:15:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.395+0000 7fbd78800640 1 -- 172.21.15.119:0/1338372647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd700cd580 msgr2=0x7fbd701480e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.395+0000 7fbd78800640 1 --2- 172.21.15.119:0/1338372647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd700cd580 0x7fbd701480e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fbd6002a220 tx=0x7fbd600046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.396+0000 7fbd78800640 1 -- 172.21.15.119:0/1338372647 shutdown_connections 2024-04-25T10:15:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.396+0000 7fbd78800640 1 --2- 172.21.15.119:0/1338372647 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbd5406b4d0 0x7fbd5406d990 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.396+0000 7fbd78800640 1 --2- 172.21.15.119:0/1338372647 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd7015a870 0x7fbd701e39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.396+0000 7fbd78800640 1 --2- 172.21.15.119:0/1338372647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd700cd580 0x7fbd701480e0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.396+0000 7fbd78800640 1 --2- 172.21.15.119:0/1338372647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd7006e2e0 0x7fbd700a8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.396+0000 7fbd78800640 1 -- 172.21.15.119:0/1338372647 >> 172.21.15.119:0/1338372647 conn(0x7fbd7006aba0 msgr2=0x7fbd700af150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.396+0000 7fbd78800640 1 -- 172.21.15.119:0/1338372647 shutdown_connections 2024-04-25T10:15:46.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.396+0000 7fbd78800640 1 -- 172.21.15.119:0/1338372647 wait complete. 2024-04-25T10:15:46.420 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:15:46.420 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:15:46.420 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:15:46.420 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:46.420 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:15:46.421 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:15:46.421 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:15:46.421 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:46.421 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:15:46.421 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:15:46.421 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:15:46.421 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:46.422 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:15:46.422 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:15:46.422 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:15:46.422 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:46.441 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:15:46.441 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:15:46.443 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:15:46.443 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:15:46.444 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:15:46.444 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:15:46.448 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:15:46.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:15:46.491 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:15:46.492 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:46.493 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:15:46.494 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:46.498 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:15:46.498 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:46.499 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:15:46.499 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:46.499 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:15:46.644 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2024-04-25T10:15:46.645 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-04-25T10:15:46.645 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:15:46.645 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-04-25T10:15:46.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.672+0000 7f44ad000640 1 Processor -- start 2024-04-25T10:15:46.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.673+0000 7f44ad000640 1 -- start start 2024-04-25T10:15:46.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.674+0000 7f44ad000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44a81611e0 0x7f44a80acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.674+0000 7f44ad000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f44a806ba90 0x7f44a806bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.674+0000 7f44ad000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a806c580 con 0x7f44a80cdb30 2024-04-25T10:15:46.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.674+0000 7f44ad000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a806dde0 con 0x7f44a81611e0 2024-04-25T10:15:46.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.674+0000 7f44ad000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a806e0c0 con 0x7f44a806ba90 2024-04-25T10:15:46.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.674+0000 7f44a5e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44a81611e0 0x7f44a80acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.674+0000 7f44a6800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f44a80cdb30 0x7f44a806eba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47922/0 (socket says 172.21.15.119:47922) 2024-04-25T10:15:46.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.674+0000 7f44a6800640 1 -- 172.21.15.119:0/3116886494 learned_addr learned my addr 172.21.15.119:0/3116886494 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:46.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.674+0000 7f449fe00640 1 --2- 172.21.15.119:0/3116886494 >> v2:172.21.15.132:3300/0 conn(0x7f44a806ba90 0x7f44a806bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.675+0000 7f44a5e00640 1 -- 172.21.15.119:0/3116886494 >> v2:172.21.15.132:3300/0 conn(0x7f44a806ba90 msgr2=0x7f44a806bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:46.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.675+0000 7f44a5e00640 1 --2- 172.21.15.119:0/3116886494 >> v2:172.21.15.132:3300/0 conn(0x7f44a806ba90 0x7f44a806bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.675+0000 7f44a5e00640 1 -- 172.21.15.119:0/3116886494 >> v1:172.21.15.119:6789/0 conn(0x7f44a80cdb30 legacy=0x7f44a806eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.675+0000 7f44a5400640 1 -- 172.21.15.119:0/3116886494 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628774656 0 0) 0x7f44a806c580 con 0x7f44a80cdb30 2024-04-25T10:15:46.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.675+0000 7f44a5e00640 1 -- 172.21.15.119:0/3116886494 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a806e3a0 con 0x7f44a81611e0 2024-04-25T10:15:46.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.676+0000 7f44a5e00640 1 --2- 172.21.15.119:0/3116886494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a81611e0 0x7f44a80acdf0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f44a8071a90 tx=0x7f449802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d071f1688f8e606 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:46.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.676+0000 7f44a5400640 1 -- 172.21.15.119:0/3116886494 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498037070 con 0x7f44a81611e0 2024-04-25T10:15:46.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.676+0000 7f44a5400640 1 -- 172.21.15.119:0/3116886494 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449802add0 con 0x7f44a81611e0 2024-04-25T10:15:46.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.676+0000 7f44a5400640 1 -- 172.21.15.119:0/3116886494 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498032660 con 0x7f44a81611e0 2024-04-25T10:15:46.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.677+0000 7f44ad000640 1 -- 172.21.15.119:0/3116886494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a81611e0 msgr2=0x7f44a80acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.677+0000 7f44ad000640 1 --2- 172.21.15.119:0/3116886494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a81611e0 0x7f44a80acdf0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f44a8071a90 tx=0x7f449802a080 comp rx=0 tx=0).stop 2024-04-25T10:15:46.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.677+0000 7f44ad000640 1 -- 172.21.15.119:0/3116886494 shutdown_connections 2024-04-25T10:15:46.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.677+0000 7f44ad000640 1 --2- 172.21.15.119:0/3116886494 >> v2:172.21.15.132:3300/0 conn(0x7f44a806ba90 0x7f44a806bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.677+0000 7f44ad000640 1 --2- 172.21.15.119:0/3116886494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a81611e0 0x7f44a80acdf0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.677+0000 7f44ad000640 1 -- 172.21.15.119:0/3116886494 >> 172.21.15.119:0/3116886494 conn(0x7f44a80cc9e0 msgr2=0x7f44a80ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:46.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.677+0000 7f44ad000640 1 -- 172.21.15.119:0/3116886494 shutdown_connections 2024-04-25T10:15:46.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.677+0000 7f44ad000640 1 -- 172.21.15.119:0/3116886494 wait complete. 2024-04-25T10:15:46.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.679+0000 7f44ad000640 1 Processor -- start 2024-04-25T10:15:46.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.679+0000 7f44ad000640 1 -- start start 2024-04-25T10:15:46.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.680+0000 7f44ad000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a806ba90 0x7f44a80c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.680+0000 7f44ad000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a80cdb30 0x7f44a81618c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.680+0000 7f44ad000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a81611e0 0x7f44a81f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.680+0000 7f44ad000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80cdf60 con 0x7f44a80cdb30 2024-04-25T10:15:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.680+0000 7f44ad000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a806ae30 con 0x7f44a81611e0 2024-04-25T10:15:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.680+0000 7f44ad000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80ce0e0 con 0x7f44a806ba90 2024-04-25T10:15:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.680+0000 7f449fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a80cdb30 0x7f44a81618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.680+0000 7f449fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a80cdb30 0x7f44a81618c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59460/0 (socket says 172.21.15.119:59460) 2024-04-25T10:15:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.680+0000 7f449fe00640 1 -- 172.21.15.119:0/2882942922 learned_addr learned my addr 172.21.15.119:0/2882942922 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.680+0000 7f44a6800640 1 --2- 172.21.15.119:0/2882942922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a81611e0 0x7f44a81f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.680+0000 7f44a5e00640 1 --2- 172.21.15.119:0/2882942922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a806ba90 0x7f44a80c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.681+0000 7f449fe00640 1 -- 172.21.15.119:0/2882942922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a806ba90 msgr2=0x7f44a80c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.681+0000 7f449fe00640 1 --2- 172.21.15.119:0/2882942922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a806ba90 0x7f44a80c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.681+0000 7f449fe00640 1 -- 172.21.15.119:0/2882942922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a81611e0 msgr2=0x7f44a81f28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.681+0000 7f449fe00640 1 --2- 172.21.15.119:0/2882942922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a81611e0 0x7f44a81f28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.681+0000 7f449fe00640 1 -- 172.21.15.119:0/2882942922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a80c4a50 con 0x7f44a80cdb30 2024-04-25T10:15:46.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.681+0000 7f449fe00640 1 --2- 172.21.15.119:0/2882942922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a80cdb30 0x7f44a81618c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f449000e9b0 tx=0x7f449000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:46.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.681+0000 7f449e000640 1 -- 172.21.15.119:0/2882942922 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4490017070 con 0x7f44a80cdb30 2024-04-25T10:15:46.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.681+0000 7f449e000640 1 -- 172.21.15.119:0/2882942922 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449000c9a0 con 0x7f44a80cdb30 2024-04-25T10:15:46.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.681+0000 7f449e000640 1 -- 172.21.15.119:0/2882942922 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449000cd90 con 0x7f44a80cdb30 2024-04-25T10:15:46.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.681+0000 7f44ad000640 1 -- 172.21.15.119:0/2882942922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a80c4d30 con 0x7f44a80cdb30 2024-04-25T10:15:46.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.681+0000 7f44ad000640 1 -- 172.21.15.119:0/2882942922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a80c52f0 con 0x7f44a80cdb30 2024-04-25T10:15:46.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.682+0000 7f44ad000640 1 -- 172.21.15.119:0/2882942922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a80750b0 con 0x7f44a80cdb30 2024-04-25T10:15:46.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.683+0000 7f449e000640 1 -- 172.21.15.119:0/2882942922 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f449001a070 con 0x7f44a80cdb30 2024-04-25T10:15:46.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.684+0000 7f449e000640 1 --2- 172.21.15.119:0/2882942922 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f448406b4d0 0x7f448406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:46.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.684+0000 7f449e000640 1 -- 172.21.15.119:0/2882942922 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f4490089f30 con 0x7f44a80cdb30 2024-04-25T10:15:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.685+0000 7f44a5e00640 1 --2- 172.21.15.119:0/2882942922 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f448406b4d0 0x7f448406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.685+0000 7f44a5e00640 1 --2- 172.21.15.119:0/2882942922 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f448406b4d0 0x7f448406d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4498034000 tx=0x7f449802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:46.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.689+0000 7f449e000640 1 -- 172.21.15.119:0/2882942922 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4490058910 con 0x7f44a80cdb30 2024-04-25T10:15:46.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:46.883+0000 7f44ad000640 1 -- 172.21.15.119:0/2882942922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f44a80cc3a0 con 0x7f44a80cdb30 2024-04-25T10:15:47.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.065+0000 7f449e000640 1 -- 172.21.15.119:0/2882942922 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v90) v1 ==== 97+0+0 (secure 0 0 0) 0x7f449005c9b0 con 0x7f44a80cdb30 2024-04-25T10:15:47.069 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2024-04-25T10:15:47.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.068+0000 7f44ad000640 1 -- 172.21.15.119:0/2882942922 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f448406b4d0 msgr2=0x7f448406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.068+0000 7f44ad000640 1 --2- 172.21.15.119:0/2882942922 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f448406b4d0 0x7f448406d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4498034000 tx=0x7f449802e000 comp rx=0 tx=0).stop 2024-04-25T10:15:47.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.068+0000 7f44ad000640 1 -- 172.21.15.119:0/2882942922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a80cdb30 msgr2=0x7f44a81618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.068+0000 7f44ad000640 1 --2- 172.21.15.119:0/2882942922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a80cdb30 0x7f44a81618c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f449000e9b0 tx=0x7f449000ee70 comp rx=0 tx=0).stop 2024-04-25T10:15:47.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.068+0000 7f44ad000640 1 -- 172.21.15.119:0/2882942922 shutdown_connections 2024-04-25T10:15:47.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.068+0000 7f44ad000640 1 --2- 172.21.15.119:0/2882942922 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f448406b4d0 0x7f448406d990 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.068+0000 7f44ad000640 1 --2- 172.21.15.119:0/2882942922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a81611e0 0x7f44a81f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.068+0000 7f44ad000640 1 --2- 172.21.15.119:0/2882942922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44a80cdb30 0x7f44a81618c0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.068+0000 7f44ad000640 1 --2- 172.21.15.119:0/2882942922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44a806ba90 0x7f44a80c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.068+0000 7f44ad000640 1 -- 172.21.15.119:0/2882942922 >> 172.21.15.119:0/2882942922 conn(0x7f44a80cc9e0 msgr2=0x7f44a815f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:47.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.068+0000 7f44ad000640 1 -- 172.21.15.119:0/2882942922 shutdown_connections 2024-04-25T10:15:47.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.069+0000 7f44ad000640 1 -- 172.21.15.119:0/2882942922 wait complete. 2024-04-25T10:15:47.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:15:47.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.272+0000 7f3db1c00640 1 Processor -- start 2024-04-25T10:15:47.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.272+0000 7f3db1c00640 1 -- start start 2024-04-25T10:15:47.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.273+0000 7f3db1c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3dac0b53e0 0x7f3dac0cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.274+0000 7f3db1c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dac0cb150 con 0x7f3dac0cd6a0 2024-04-25T10:15:47.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.274+0000 7f3db1c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dac0cb420 con 0x7f3dac15a850 2024-04-25T10:15:47.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.274+0000 7f3db1c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dac0cb700 con 0x7f3dac0b53e0 2024-04-25T10:15:47.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.274+0000 7f3dab400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3dac0b53e0 0x7f3dac0cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.274+0000 7f3daa000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3dac0cd6a0 0x7f3dac0b4cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47926/0 (socket says 172.21.15.119:47926) 2024-04-25T10:15:47.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.274+0000 7f3daa000640 1 -- 172.21.15.119:0/323307714 learned_addr learned my addr 172.21.15.119:0/323307714 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:47.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.274+0000 7f3dab400640 1 -- 172.21.15.119:0/323307714 >> v1:172.21.15.132:6789/0 conn(0x7f3dac15a850 legacy=0x7f3dac0ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:47.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.274+0000 7f3dab400640 1 -- 172.21.15.119:0/323307714 >> v1:172.21.15.119:6789/0 conn(0x7f3dac0cd6a0 legacy=0x7f3dac0b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.274+0000 7f3dab400640 1 -- 172.21.15.119:0/323307714 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac0cb9e0 con 0x7f3dac0b53e0 2024-04-25T10:15:47.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.275+0000 7f3dab400640 1 --2- 172.21.15.119:0/323307714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dac0b53e0 0x7f3dac0cdb10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009d90 tx=0x7f3da402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6c07dc3f41e31c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:47.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.276+0000 7f3da9600640 1 -- 172.21.15.119:0/323307714 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4007740 con 0x7f3dac0b53e0 2024-04-25T10:15:47.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.276+0000 7f3da9600640 1 -- 172.21.15.119:0/323307714 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da40078e0 con 0x7f3dac0b53e0 2024-04-25T10:15:47.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.276+0000 7f3da9600640 1 -- 172.21.15.119:0/323307714 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4007cb0 con 0x7f3dac0b53e0 2024-04-25T10:15:47.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.276+0000 7f3db1c00640 1 -- 172.21.15.119:0/323307714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dac0b53e0 msgr2=0x7f3dac0cdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.276+0000 7f3db1c00640 1 --2- 172.21.15.119:0/323307714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dac0b53e0 0x7f3dac0cdb10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009d90 tx=0x7f3da402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.276+0000 7f3db1c00640 1 -- 172.21.15.119:0/323307714 shutdown_connections 2024-04-25T10:15:47.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.276+0000 7f3db1c00640 1 --2- 172.21.15.119:0/323307714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dac0b53e0 0x7f3dac0cdb10 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.276+0000 7f3db1c00640 1 -- 172.21.15.119:0/323307714 >> 172.21.15.119:0/323307714 conn(0x7f3dac0cf5a0 msgr2=0x7f3dac0cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:47.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.276+0000 7f3db1c00640 1 -- 172.21.15.119:0/323307714 shutdown_connections 2024-04-25T10:15:47.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.277+0000 7f3db1c00640 1 -- 172.21.15.119:0/323307714 wait complete. 2024-04-25T10:15:47.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.278+0000 7f3db1c00640 1 Processor -- start 2024-04-25T10:15:47.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.278+0000 7f3db1c00640 1 -- start start 2024-04-25T10:15:47.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.279+0000 7f3db1c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dac0b53e0 0x7f3dac13f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.279+0000 7f3db1c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dac0cd6a0 0x7f3dac0b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.279+0000 7f3db1c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dac15a850 0x7f3dac0b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.279+0000 7f3db1c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac0cf240 con 0x7f3dac15a850 2024-04-25T10:15:47.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.279+0000 7f3db1c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac0cbcc0 con 0x7f3dac0b53e0 2024-04-25T10:15:47.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.279+0000 7f3db1c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac0cf3c0 con 0x7f3dac0cd6a0 2024-04-25T10:15:47.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.279+0000 7f3dab400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dac15a850 0x7f3dac0b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.280+0000 7f3dab400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dac15a850 0x7f3dac0b3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59474/0 (socket says 172.21.15.119:59474) 2024-04-25T10:15:47.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.280+0000 7f3dab400640 1 -- 172.21.15.119:0/1298501244 learned_addr learned my addr 172.21.15.119:0/1298501244 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:47.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.280+0000 7f3daaa00640 1 --2- 172.21.15.119:0/1298501244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dac0b53e0 0x7f3dac13f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.280+0000 7f3dab400640 1 -- 172.21.15.119:0/1298501244 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dac0cd6a0 msgr2=0x7f3dac0b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.280+0000 7f3dab400640 1 --2- 172.21.15.119:0/1298501244 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dac0cd6a0 0x7f3dac0b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.280+0000 7f3dab400640 1 -- 172.21.15.119:0/1298501244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dac0b53e0 msgr2=0x7f3dac13f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.280+0000 7f3dab400640 1 --2- 172.21.15.119:0/1298501244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dac0b53e0 0x7f3dac13f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.280+0000 7f3dab400640 1 -- 172.21.15.119:0/1298501244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac15a040 con 0x7f3dac15a850 2024-04-25T10:15:47.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.280+0000 7f3daaa00640 1 --2- 172.21.15.119:0/1298501244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dac0b53e0 0x7f3dac13f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:47.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.280+0000 7f3dab400640 1 --2- 172.21.15.119:0/1298501244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dac15a850 0x7f3dac0b3200 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009d90 tx=0x7f3da4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:47.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.280+0000 7f3da2a00640 1 -- 172.21.15.119:0/1298501244 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4040070 con 0x7f3dac15a850 2024-04-25T10:15:47.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.281+0000 7f3da2a00640 1 -- 172.21.15.119:0/1298501244 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da4031c90 con 0x7f3dac15a850 2024-04-25T10:15:47.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.281+0000 7f3db1c00640 1 -- 172.21.15.119:0/1298501244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dac13fc20 con 0x7f3dac15a850 2024-04-25T10:15:47.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.281+0000 7f3da2a00640 1 -- 172.21.15.119:0/1298501244 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da403c640 con 0x7f3dac15a850 2024-04-25T10:15:47.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.281+0000 7f3db1c00640 1 -- 172.21.15.119:0/1298501244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dac159280 con 0x7f3dac15a850 2024-04-25T10:15:47.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.282+0000 7f3db1c00640 1 -- 172.21.15.119:0/1298501244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dac0749b0 con 0x7f3dac15a850 2024-04-25T10:15:47.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.283+0000 7f3da2a00640 1 -- 172.21.15.119:0/1298501244 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3da4039040 con 0x7f3dac15a850 2024-04-25T10:15:47.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.284+0000 7f3da2a00640 1 --2- 172.21.15.119:0/1298501244 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d8c06b4d0 0x7f3d8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.284+0000 7f3da2a00640 1 -- 172.21.15.119:0/1298501244 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3da40ab570 con 0x7f3dac15a850 2024-04-25T10:15:47.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.284+0000 7f3daaa00640 1 --2- 172.21.15.119:0/1298501244 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d8c06b4d0 0x7f3d8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.284+0000 7f3daaa00640 1 --2- 172.21.15.119:0/1298501244 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d8c06b4d0 0x7f3d8c06d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00a830 tx=0x7f3d9c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:47.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.288+0000 7f3da2a00640 1 -- 172.21.15.119:0/1298501244 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3da407a760 con 0x7f3dac15a850 2024-04-25T10:15:47.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.554+0000 7f3db1c00640 1 -- 172.21.15.119:0/1298501244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3dac0ce400 con 0x7f3dac15a850 2024-04-25T10:15:47.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.555+0000 7f3da2a00640 1 -- 172.21.15.119:0/1298501244 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3da407a060 con 0x7f3dac15a850 2024-04-25T10:15:47.559 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:47.559 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:15:47.559 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2024-04-25T10:15:47.559 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.559 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2024-04-25T10:15:47.559 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmFDz9FBAAdC/N+8UnkYkK5wrDlgpkgA==", 2024-04-25T10:15:47.559 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.559 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:15:47.560 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:15:47.560 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:15:47.560 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.560 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.560 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.560 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.560 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2024-04-25T10:15:47.560 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmmh7lLhAAkzsFzTL6Ko/J/0ZJOpvR4A==", 2024-04-25T10:15:47.560 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.560 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:15:47.560 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:15:47.560 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:15:47.560 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.561 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.561 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2024-04-25T10:15:47.561 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmojotIhAAdSIhFOXoYZN+01nbCs3e+Q==", 2024-04-25T10:15:47.561 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.561 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:15:47.561 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:15:47.561 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:15:47.561 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.561 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.561 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.561 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.561 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2024-04-25T10:15:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypm1gp7ORAAvtG28IuhGtR1/Vj6JCPq5A==", 2024-04-25T10:15:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:15:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:15:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:15:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2024-04-25T10:15:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAPKypmOI9YMRAAmMrg0xRUG+GG+waJ12llJQ==", 2024-04-25T10:15:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:47.562 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:15:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2024-04-25T10:15:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQATKypmHV5GDxAAW0XzpAxF7oxgVBKy5yzavw==", 2024-04-25T10:15:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:15:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.563 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2024-04-25T10:15:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAWKypmXTiRGhAAZxct/Ck3A3DMBjqNW3HXrg==", 2024-04-25T10:15:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:15:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2024-04-25T10:15:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAZKypmEeGSMhAAwVNTny5OmP/KONYhIXzrrQ==", 2024-04-25T10:15:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.564 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAgKypmh7gPBRAApx1VM3WpYW7hIQzu7+6J7w==", 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.565 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2024-04-25T10:15:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAjKypmQ2EaEBAAoFYMlnh5/ar0pAlynwjYOA==", 2024-04-25T10:15:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:15:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2024-04-25T10:15:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAmKypmDATkKRAAsIDViOBeDrd5N91wf8zGLA==", 2024-04-25T10:15:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:15:47.566 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2024-04-25T10:15:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAqKypmNskbCBAAQIYhJ9lfHHr9EP/n5QJoRw==", 2024-04-25T10:15:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:15:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2024-04-25T10:15:47.567 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA==", 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmm0DLGhAApkxEcc1jUU5T/9ErkTkT4A==", 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:15:47.568 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2024-04-25T10:15:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmMZ89ERAAICfWm/ZgOLQMVSv5xG0Crg==", 2024-04-25T10:15:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:15:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:15:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:15:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:15:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2024-04-25T10:15:47.569 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmrokCIxAAwi8FtvYSVhMguvZ0OyYt+g==", 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAKKypmN9WFGRAAi+stjuG2ZDBjafeR9Ygrug==", 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:15:47.570 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2024-04-25T10:15:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypm/KyKDBAA646fW6qrcXB31usxeIApaA==", 2024-04-25T10:15:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:15:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2024-04-25T10:15:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:15:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2024-04-25T10:15:47.571 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmr8SqBBAA8l4CjUEidXz1obWJFM4DHA==", 2024-04-25T10:15:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:15:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:15:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2024-04-25T10:15:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:15:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:47.572 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-25T10:15:47.572 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:15:47.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.557+0000 7f3db1c00640 1 -- 172.21.15.119:0/1298501244 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d8c06b4d0 msgr2=0x7f3d8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.557+0000 7f3db1c00640 1 --2- 172.21.15.119:0/1298501244 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d8c06b4d0 0x7f3d8c06d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00a830 tx=0x7f3d9c008040 comp rx=0 tx=0).stop 2024-04-25T10:15:47.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.557+0000 7f3db1c00640 1 -- 172.21.15.119:0/1298501244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dac15a850 msgr2=0x7f3dac0b3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.557+0000 7f3db1c00640 1 --2- 172.21.15.119:0/1298501244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dac15a850 0x7f3dac0b3200 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009d90 tx=0x7f3da4004a40 comp rx=0 tx=0).stop 2024-04-25T10:15:47.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.558+0000 7f3db1c00640 1 -- 172.21.15.119:0/1298501244 shutdown_connections 2024-04-25T10:15:47.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.558+0000 7f3db1c00640 1 --2- 172.21.15.119:0/1298501244 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3d8c06b4d0 0x7f3d8c06d990 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.558+0000 7f3db1c00640 1 --2- 172.21.15.119:0/1298501244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dac15a850 0x7f3dac0b3200 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.558+0000 7f3db1c00640 1 --2- 172.21.15.119:0/1298501244 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3dac0cd6a0 0x7f3dac0b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.558+0000 7f3db1c00640 1 --2- 172.21.15.119:0/1298501244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dac0b53e0 0x7f3dac13f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.558+0000 7f3db1c00640 1 -- 172.21.15.119:0/1298501244 >> 172.21.15.119:0/1298501244 conn(0x7f3dac0cf5a0 msgr2=0x7f3dac15b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:47.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.558+0000 7f3db1c00640 1 -- 172.21.15.119:0/1298501244 shutdown_connections 2024-04-25T10:15:47.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.558+0000 7f3db1c00640 1 -- 172.21.15.119:0/1298501244 wait complete. 2024-04-25T10:15:47.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:47.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.774+0000 7fc1ccc00640 1 Processor -- start 2024-04-25T10:15:47.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.774+0000 7fc1ccc00640 1 -- start start 2024-04-25T10:15:47.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.776+0000 7fc1ccc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c80b5050 0x7fc1c815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.776+0000 7fc1ccc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc1c806e430 0x7fc1c815bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.776+0000 7fc1ccc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c80cd440 con 0x7fc1c8144470 2024-04-25T10:15:47.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.776+0000 7fc1ccc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c8145340 con 0x7fc1c806e430 2024-04-25T10:15:47.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.776+0000 7fc1ccc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80cd720 con 0x7fc1c80b5050 2024-04-25T10:15:47.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.776+0000 7fc1c7400640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fc1c8144470 0x7fc1c80b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:54482/0 (socket says 172.21.15.119:54482) 2024-04-25T10:15:47.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.777+0000 7fc1c7400640 1 -- 172.21.15.119:0/3025280045 learned_addr learned my addr 172.21.15.119:0/3025280045 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:47.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.777+0000 7fc1c6a00640 1 --2- 172.21.15.119:0/3025280045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c80b5050 0x7fc1c815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.777+0000 7fc1c7e00640 1 --2- 172.21.15.119:0/3025280045 >> v2:172.21.15.132:3300/0 conn(0x7fc1c806e430 0x7fc1c815bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.777+0000 7fc1c6000640 1 -- 172.21.15.119:0/3025280045 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794945411 0 0) 0x7fc1c80cd440 con 0x7fc1c8144470 2024-04-25T10:15:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.777+0000 7fc1c6000640 1 -- 172.21.15.119:0/3025280045 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1ac003520 con 0x7fc1c8144470 2024-04-25T10:15:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.777+0000 7fc1c7e00640 1 -- 172.21.15.119:0/3025280045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c80b5050 msgr2=0x7fc1c815fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.777+0000 7fc1c7e00640 1 --2- 172.21.15.119:0/3025280045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c80b5050 0x7fc1c815fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.777+0000 7fc1c7e00640 1 -- 172.21.15.119:0/3025280045 >> v1:172.21.15.132:6789/0 conn(0x7fc1c8144470 legacy=0x7fc1c80b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.777+0000 7fc1c7e00640 1 -- 172.21.15.119:0/3025280045 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c80cda00 con 0x7fc1c806e430 2024-04-25T10:15:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.779+0000 7fc1c7e00640 1 --2- 172.21.15.119:0/3025280045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c806e430 0x7fc1c815bcf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00097a0 tx=0x7fc1c002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb2de768b8b2be9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.779+0000 7fc1c6000640 1 -- 172.21.15.119:0/3025280045 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0037070 con 0x7fc1c806e430 2024-04-25T10:15:47.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.779+0000 7fc1c6000640 1 -- 172.21.15.119:0/3025280045 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c0032440 con 0x7fc1c806e430 2024-04-25T10:15:47.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.779+0000 7fc1c6000640 1 -- 172.21.15.119:0/3025280045 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0032810 con 0x7fc1c806e430 2024-04-25T10:15:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.780+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3025280045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c806e430 msgr2=0x7fc1c815bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.780+0000 7fc1ccc00640 1 --2- 172.21.15.119:0/3025280045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c806e430 0x7fc1c815bcf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00097a0 tx=0x7fc1c002a180 comp rx=0 tx=0).stop 2024-04-25T10:15:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.780+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3025280045 shutdown_connections 2024-04-25T10:15:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.780+0000 7fc1ccc00640 1 --2- 172.21.15.119:0/3025280045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c806e430 0x7fc1c815bcf0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.780+0000 7fc1ccc00640 1 --2- 172.21.15.119:0/3025280045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c80b5050 0x7fc1c815fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.780+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3025280045 >> 172.21.15.119:0/3025280045 conn(0x7fc1c806b1d0 msgr2=0x7fc1c80b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:47.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.780+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3025280045 shutdown_connections 2024-04-25T10:15:47.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.780+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3025280045 wait complete. 2024-04-25T10:15:47.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.782+0000 7fc1ccc00640 1 Processor -- start 2024-04-25T10:15:47.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.782+0000 7fc1ccc00640 1 -- start start 2024-04-25T10:15:47.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.783+0000 7fc1ccc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c806e430 0x7fc1c80ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.783+0000 7fc1ccc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c80b5050 0x7fc1c815af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.783+0000 7fc1ccc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1c8144470 0x7fc1c80b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.783+0000 7fc1ccc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80ce200 con 0x7fc1c8144470 2024-04-25T10:15:47.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.783+0000 7fc1ccc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c8144850 con 0x7fc1c80b5050 2024-04-25T10:15:47.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.783+0000 7fc1ccc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c815c230 con 0x7fc1c806e430 2024-04-25T10:15:47.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.783+0000 7fc1c7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1c8144470 0x7fc1c80b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.783+0000 7fc1c7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1c8144470 0x7fc1c80b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59486/0 (socket says 172.21.15.119:59486) 2024-04-25T10:15:47.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.783+0000 7fc1c7e00640 1 -- 172.21.15.119:0/3910076119 learned_addr learned my addr 172.21.15.119:0/3910076119 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:47.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.783+0000 7fc1c6a00640 1 --2- 172.21.15.119:0/3910076119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c80b5050 0x7fc1c815af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.784+0000 7fc1c7400640 1 --2- 172.21.15.119:0/3910076119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c806e430 0x7fc1c80ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.784+0000 7fc1c7e00640 1 -- 172.21.15.119:0/3910076119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c806e430 msgr2=0x7fc1c80ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.784+0000 7fc1c7e00640 1 --2- 172.21.15.119:0/3910076119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c806e430 0x7fc1c80ae430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.784+0000 7fc1c7e00640 1 -- 172.21.15.119:0/3910076119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c80b5050 msgr2=0x7fc1c815af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.784+0000 7fc1c7e00640 1 --2- 172.21.15.119:0/3910076119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c80b5050 0x7fc1c815af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.784+0000 7fc1c7e00640 1 -- 172.21.15.119:0/3910076119 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c80b0160 con 0x7fc1c8144470 2024-04-25T10:15:47.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.784+0000 7fc1c7e00640 1 --2- 172.21.15.119:0/3910076119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1c8144470 0x7fc1c80b4100 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0035040 tx=0x7fc1c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:47.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.784+0000 7fc1bf400640 1 -- 172.21.15.119:0/3910076119 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0041020 con 0x7fc1c8144470 2024-04-25T10:15:47.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.784+0000 7fc1bf400640 1 -- 172.21.15.119:0/3910076119 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c003d640 con 0x7fc1c8144470 2024-04-25T10:15:47.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.784+0000 7fc1bf400640 1 -- 172.21.15.119:0/3910076119 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0037040 con 0x7fc1c8144470 2024-04-25T10:15:47.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.784+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3910076119 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c80b0440 con 0x7fc1c8144470 2024-04-25T10:15:47.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.785+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3910076119 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c80b0a00 con 0x7fc1c8144470 2024-04-25T10:15:47.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.785+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3910076119 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c80749b0 con 0x7fc1c8144470 2024-04-25T10:15:47.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.787+0000 7fc1bf400640 1 -- 172.21.15.119:0/3910076119 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1c0044020 con 0x7fc1c8144470 2024-04-25T10:15:47.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.787+0000 7fc1bf400640 1 --2- 172.21.15.119:0/3910076119 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc1a406b4d0 0x7fc1a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:47.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.788+0000 7fc1bf400640 1 -- 172.21.15.119:0/3910076119 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc1c00aa670 con 0x7fc1c8144470 2024-04-25T10:15:47.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.788+0000 7fc1c7400640 1 --2- 172.21.15.119:0/3910076119 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc1a406b4d0 0x7fc1a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:47.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.788+0000 7fc1c7400640 1 --2- 172.21.15.119:0/3910076119 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc1a406b4d0 0x7fc1a406d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00011e0 tx=0x7fc1b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:47.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.792+0000 7fc1bf400640 1 -- 172.21.15.119:0/3910076119 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1c004b030 con 0x7fc1c8144470 2024-04-25T10:15:47.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.990+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3910076119 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1c80b2090 con 0x7fc1c8144470 2024-04-25T10:15:47.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.992+0000 7fc1bf400640 1 -- 172.21.15.119:0/3910076119 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+12928 (secure 0 0 0) 0x7fc1c00798d0 con 0x7fc1c8144470 2024-04-25T10:15:47.996 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:47.996 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":92,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:15:47.080202+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.994+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3910076119 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc1a406b4d0 msgr2=0x7fc1a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.994+0000 7fc1ccc00640 1 --2- 172.21.15.119:0/3910076119 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc1a406b4d0 0x7fc1a406d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00011e0 tx=0x7fc1b0009040 comp rx=0 tx=0).stop 2024-04-25T10:15:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.994+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3910076119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1c8144470 msgr2=0x7fc1c80b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.994+0000 7fc1ccc00640 1 --2- 172.21.15.119:0/3910076119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1c8144470 0x7fc1c80b4100 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0035040 tx=0x7fc1c00046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.995+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3910076119 shutdown_connections 2024-04-25T10:15:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.995+0000 7fc1ccc00640 1 --2- 172.21.15.119:0/3910076119 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc1a406b4d0 0x7fc1a406d990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.995+0000 7fc1ccc00640 1 --2- 172.21.15.119:0/3910076119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1c8144470 0x7fc1c80b4100 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.995+0000 7fc1ccc00640 1 --2- 172.21.15.119:0/3910076119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1c80b5050 0x7fc1c815af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.995+0000 7fc1ccc00640 1 --2- 172.21.15.119:0/3910076119 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1c806e430 0x7fc1c80ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.995+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3910076119 >> 172.21.15.119:0/3910076119 conn(0x7fc1c806b1d0 msgr2=0x7fc1c8143480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:48.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.995+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3910076119 shutdown_connections 2024-04-25T10:15:48.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:47.995+0000 7fc1ccc00640 1 -- 172.21.15.119:0/3910076119 wait complete. 2024-04-25T10:15:48.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:48.018 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:48.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.216+0000 7f4dbda00640 1 Processor -- start 2024-04-25T10:15:48.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.216+0000 7f4dbda00640 1 -- start start 2024-04-25T10:15:48.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.217+0000 7f4dbda00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4db80cf900 0x7f4db80b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.217+0000 7f4dbda00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db80b2b90 0x7f4db8162f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.218+0000 7f4dbda00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db8071350 con 0x7f4db80cfce0 2024-04-25T10:15:48.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.218+0000 7f4dbda00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db8071630 con 0x7f4db80cf900 2024-04-25T10:15:48.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.218+0000 7f4dbda00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db8071910 con 0x7f4db80b2b90 2024-04-25T10:15:48.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.218+0000 7f4db6a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4db80cf900 0x7f4db80b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.218+0000 7f4db7400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db80b2b90 0x7f4db8162f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.218+0000 7f4dafe00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f4db80cfce0 0x7f4db8162880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:54490/0 (socket says 172.21.15.119:54490) 2024-04-25T10:15:48.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.218+0000 7f4dafe00640 1 -- 172.21.15.119:0/1695285837 learned_addr learned my addr 172.21.15.119:0/1695285837 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:48.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.219+0000 7f4db6a00640 1 -- 172.21.15.119:0/1695285837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db80b2b90 msgr2=0x7f4db8162f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:48.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.219+0000 7f4db6a00640 1 --2- 172.21.15.119:0/1695285837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db80b2b90 0x7f4db8162f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.219+0000 7f4db6a00640 1 -- 172.21.15.119:0/1695285837 >> v1:172.21.15.132:6789/0 conn(0x7f4db80cfce0 legacy=0x7f4db8162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.219+0000 7f4db6a00640 1 -- 172.21.15.119:0/1695285837 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db8071bf0 con 0x7f4db80cf900 2024-04-25T10:15:48.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.221+0000 7f4db6a00640 1 --2- 172.21.15.119:0/1695285837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4db80cf900 0x7f4db80b2650 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4da80097a0 tx=0x7f4da802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62591d9587e7bad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.221+0000 7f4db6000640 1 -- 172.21.15.119:0/1695285837 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8037070 con 0x7f4db80cf900 2024-04-25T10:15:48.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.221+0000 7f4db6000640 1 -- 172.21.15.119:0/1695285837 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da802ae50 con 0x7f4db80cf900 2024-04-25T10:15:48.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.221+0000 7f4db6000640 1 -- 172.21.15.119:0/1695285837 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8032660 con 0x7f4db80cf900 2024-04-25T10:15:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.222+0000 7f4dbda00640 1 -- 172.21.15.119:0/1695285837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4db80cf900 msgr2=0x7f4db80b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.222+0000 7f4dbda00640 1 --2- 172.21.15.119:0/1695285837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4db80cf900 0x7f4db80b2650 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4da80097a0 tx=0x7f4da802a100 comp rx=0 tx=0).stop 2024-04-25T10:15:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.222+0000 7f4dbda00640 1 -- 172.21.15.119:0/1695285837 shutdown_connections 2024-04-25T10:15:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.222+0000 7f4dbda00640 1 --2- 172.21.15.119:0/1695285837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db80b2b90 0x7f4db8162f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.222+0000 7f4dbda00640 1 --2- 172.21.15.119:0/1695285837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4db80cf900 0x7f4db80b2650 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.222+0000 7f4dbda00640 1 -- 172.21.15.119:0/1695285837 >> 172.21.15.119:0/1695285837 conn(0x7f4db806aba0 msgr2=0x7f4db806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.222+0000 7f4dbda00640 1 -- 172.21.15.119:0/1695285837 shutdown_connections 2024-04-25T10:15:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.222+0000 7f4dbda00640 1 -- 172.21.15.119:0/1695285837 wait complete. 2024-04-25T10:15:48.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.224+0000 7f4dbda00640 1 Processor -- start 2024-04-25T10:15:48.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.224+0000 7f4dbda00640 1 -- start start 2024-04-25T10:15:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4dbda00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4db80b2b90 0x7f4db8157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4dbda00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db80cf900 0x7f4db815c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4dbda00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4db80cfce0 0x7f4db815de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4dbda00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db8165c30 con 0x7f4db80b2b90 2024-04-25T10:15:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4dbda00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db80b3e00 con 0x7f4db80cfce0 2024-04-25T10:15:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4dbda00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db8165db0 con 0x7f4db80cf900 2024-04-25T10:15:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4db6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4db80b2b90 0x7f4db8157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4db6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4db80b2b90 0x7f4db8157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59502/0 (socket says 172.21.15.119:59502) 2024-04-25T10:15:48.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4db6a00640 1 -- 172.21.15.119:0/1075002551 learned_addr learned my addr 172.21.15.119:0/1075002551 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:48.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4db7400640 1 --2- 172.21.15.119:0/1075002551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4db80cfce0 0x7f4db815de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4db6a00640 1 -- 172.21.15.119:0/1075002551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db80cf900 msgr2=0x7f4db815c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4db6a00640 1 --2- 172.21.15.119:0/1075002551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db80cf900 0x7f4db815c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4db6a00640 1 -- 172.21.15.119:0/1075002551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4db80cfce0 msgr2=0x7f4db815de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4db6a00640 1 --2- 172.21.15.119:0/1075002551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4db80cfce0 0x7f4db815de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.225+0000 7f4db6a00640 1 -- 172.21.15.119:0/1075002551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db8158f50 con 0x7f4db80b2b90 2024-04-25T10:15:48.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.226+0000 7f4db6a00640 1 --2- 172.21.15.119:0/1075002551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4db80b2b90 0x7f4db8157c00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f4da8035040 tx=0x7f4da80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.226+0000 7f4daea00640 1 -- 172.21.15.119:0/1075002551 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8041070 con 0x7f4db80b2b90 2024-04-25T10:15:48.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.226+0000 7f4daea00640 1 -- 172.21.15.119:0/1075002551 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da803c460 con 0x7f4db80b2b90 2024-04-25T10:15:48.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.226+0000 7f4dbda00640 1 -- 172.21.15.119:0/1075002551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db8158140 con 0x7f4db80b2b90 2024-04-25T10:15:48.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.226+0000 7f4daea00640 1 -- 172.21.15.119:0/1075002551 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8037040 con 0x7f4db80b2b90 2024-04-25T10:15:48.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.226+0000 7f4dbda00640 1 -- 172.21.15.119:0/1075002551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db81594f0 con 0x7f4db80b2b90 2024-04-25T10:15:48.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.227+0000 7f4dbda00640 1 -- 172.21.15.119:0/1075002551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db80750b0 con 0x7f4db80b2b90 2024-04-25T10:15:48.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.228+0000 7f4daea00640 1 -- 172.21.15.119:0/1075002551 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4da803a070 con 0x7f4db80b2b90 2024-04-25T10:15:48.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.229+0000 7f4daea00640 1 --2- 172.21.15.119:0/1075002551 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d9406b4d0 0x7f4d9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.229+0000 7f4daea00640 1 -- 172.21.15.119:0/1075002551 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4da80aa390 con 0x7f4db80b2b90 2024-04-25T10:15:48.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.229+0000 7f4dafe00640 1 --2- 172.21.15.119:0/1075002551 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d9406b4d0 0x7f4d9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.230+0000 7f4dafe00640 1 --2- 172.21.15.119:0/1075002551 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d9406b4d0 0x7f4d9406d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4da0005880 tx=0x7f4da0007a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.233+0000 7f4daea00640 1 -- 172.21.15.119:0/1075002551 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4da8079550 con 0x7f4db80b2b90 2024-04-25T10:15:48.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.426+0000 7f4dbda00640 1 -- 172.21.15.119:0/1075002551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4db80b1fa0 con 0x7f4db80b2b90 2024-04-25T10:15:48.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.426+0000 7f4daea00640 1 -- 172.21.15.119:0/1075002551 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4da8078e50 con 0x7f4db80b2b90 2024-04-25T10:15:48.430 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:15:48.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.429+0000 7f4dbda00640 1 -- 172.21.15.119:0/1075002551 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d9406b4d0 msgr2=0x7f4d9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.429+0000 7f4dbda00640 1 --2- 172.21.15.119:0/1075002551 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d9406b4d0 0x7f4d9406d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4da0005880 tx=0x7f4da0007a20 comp rx=0 tx=0).stop 2024-04-25T10:15:48.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.429+0000 7f4dbda00640 1 -- 172.21.15.119:0/1075002551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4db80b2b90 msgr2=0x7f4db8157c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.429+0000 7f4dbda00640 1 --2- 172.21.15.119:0/1075002551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4db80b2b90 0x7f4db8157c00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f4da8035040 tx=0x7f4da80046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.429+0000 7f4dbda00640 1 -- 172.21.15.119:0/1075002551 shutdown_connections 2024-04-25T10:15:48.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.429+0000 7f4dbda00640 1 --2- 172.21.15.119:0/1075002551 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d9406b4d0 0x7f4d9406d990 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.429+0000 7f4dbda00640 1 --2- 172.21.15.119:0/1075002551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4db80cfce0 0x7f4db815de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.429+0000 7f4dbda00640 1 --2- 172.21.15.119:0/1075002551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db80cf900 0x7f4db815c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.429+0000 7f4dbda00640 1 --2- 172.21.15.119:0/1075002551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4db80b2b90 0x7f4db8157c00 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.429+0000 7f4dbda00640 1 -- 172.21.15.119:0/1075002551 >> 172.21.15.119:0/1075002551 conn(0x7f4db806aba0 msgr2=0x7f4db80aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:48.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.429+0000 7f4dbda00640 1 -- 172.21.15.119:0/1075002551 shutdown_connections 2024-04-25T10:15:48.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.429+0000 7f4dbda00640 1 -- 172.21.15.119:0/1075002551 wait complete. 2024-04-25T10:15:48.452 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:15:48.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.651+0000 7f001c400640 1 Processor -- start 2024-04-25T10:15:48.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.651+0000 7f001c400640 1 -- start start 2024-04-25T10:15:48.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.652+0000 7f001c400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f00140b15c0 0x7f001406d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.652+0000 7f001c400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f001406d9d0 0x7f00140cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.652+0000 7f001c400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00140ca4a0 con 0x7f00140b11e0 2024-04-25T10:15:48.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.653+0000 7f001c400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00140ca250 con 0x7f00140b15c0 2024-04-25T10:15:48.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.653+0000 7f001c400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00140ca780 con 0x7f001406d9d0 2024-04-25T10:15:48.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.653+0000 7f0019a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f00140b11e0 0x7f001415d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47934/0 (socket says 172.21.15.119:47934) 2024-04-25T10:15:48.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.653+0000 7f0019a00640 1 -- 172.21.15.119:0/3303981798 learned_addr learned my addr 172.21.15.119:0/3303981798 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.653+0000 7f0019000640 1 --2- 172.21.15.119:0/3303981798 >> v2:172.21.15.132:3300/0 conn(0x7f00140b15c0 0x7f001406d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.653+0000 7f001a400640 1 --2- 172.21.15.119:0/3303981798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f001406d9d0 0x7f00140cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.653+0000 7f0013e00640 1 -- 172.21.15.119:0/3303981798 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656599707 0 0) 0x7f00140ca4a0 con 0x7f00140b11e0 2024-04-25T10:15:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.653+0000 7f0013e00640 1 -- 172.21.15.119:0/3303981798 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efffc003520 con 0x7f00140b11e0 2024-04-25T10:15:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0013e00640 1 -- 172.21.15.119:0/3303981798 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1166734021 0 0) 0x7efffc003520 con 0x7f00140b11e0 2024-04-25T10:15:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0013e00640 1 -- 172.21.15.119:0/3303981798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f001406d9d0 msgr2=0x7f00140cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0013e00640 1 --2- 172.21.15.119:0/3303981798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f001406d9d0 0x7f00140cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0013e00640 1 -- 172.21.15.119:0/3303981798 >> v2:172.21.15.132:3300/0 conn(0x7f00140b15c0 msgr2=0x7f001406d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0013e00640 1 --2- 172.21.15.119:0/3303981798 >> v2:172.21.15.132:3300/0 conn(0x7f00140b15c0 0x7f001406d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0013e00640 1 -- 172.21.15.119:0/3303981798 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00140caa60 con 0x7f00140b11e0 2024-04-25T10:15:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0013e00640 1 -- 172.21.15.119:0/3303981798 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f0008001670 con 0x7f00140b11e0 2024-04-25T10:15:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0013e00640 1 -- 172.21.15.119:0/3303981798 >> v1:172.21.15.119:6789/0 conn(0x7f00140b11e0 legacy=0x7f001415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0013e00640 1 --2- 172.21.15.119:0/3303981798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efffc0039b0 0x7efffc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0013e00640 1 --2- 172.21.15.119:0/3303981798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efffc004800 0x7efffc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0013e00640 1 --2- 172.21.15.119:0/3303981798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f001406d9d0 0x7efffc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0013e00640 1 -- 172.21.15.119:0/3303981798 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00140ca250 con 0x7efffc0039b0 2024-04-25T10:15:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0013e00640 1 -- 172.21.15.119:0/3303981798 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00140ca780 con 0x7efffc004800 2024-04-25T10:15:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0013e00640 1 -- 172.21.15.119:0/3303981798 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efffc00b9c0 con 0x7f001406d9d0 2024-04-25T10:15:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.654+0000 7f0019a00640 1 --2- 172.21.15.119:0/3303981798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efffc0039b0 0x7efffc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.655+0000 7f0019000640 1 --2- 172.21.15.119:0/3303981798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efffc004800 0x7efffc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.655+0000 7f001a400640 1 --2- 172.21.15.119:0/3303981798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f001406d9d0 0x7efffc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.655+0000 7f0019a00640 1 -- 172.21.15.119:0/3303981798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f001406d9d0 msgr2=0x7efffc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.655+0000 7f0019a00640 1 --2- 172.21.15.119:0/3303981798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f001406d9d0 0x7efffc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.655+0000 7f0019a00640 1 -- 172.21.15.119:0/3303981798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efffc004800 msgr2=0x7efffc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.655+0000 7f0019a00640 1 --2- 172.21.15.119:0/3303981798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efffc004800 0x7efffc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.655+0000 7f0019a00640 1 -- 172.21.15.119:0/3303981798 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00140caa60 con 0x7efffc0039b0 2024-04-25T10:15:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.655+0000 7f0019000640 1 --2- 172.21.15.119:0/3303981798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efffc004800 0x7efffc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:48.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.656+0000 7f0019a00640 1 --2- 172.21.15.119:0/3303981798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efffc0039b0 0x7efffc003dd0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f000800c260 tx=0x7efffc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07eac9416f10522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.657+0000 7f0013e00640 1 -- 172.21.15.119:0/3303981798 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008036070 con 0x7efffc0039b0 2024-04-25T10:15:48.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.657+0000 7f0013e00640 1 -- 172.21.15.119:0/3303981798 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0008005ad0 con 0x7efffc0039b0 2024-04-25T10:15:48.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.657+0000 7f0013e00640 1 -- 172.21.15.119:0/3303981798 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008007d70 con 0x7efffc0039b0 2024-04-25T10:15:48.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.657+0000 7f001c400640 1 -- 172.21.15.119:0/3303981798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efffc0039b0 msgr2=0x7efffc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.657+0000 7f001c400640 1 --2- 172.21.15.119:0/3303981798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efffc0039b0 0x7efffc003dd0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f000800c260 tx=0x7efffc005830 comp rx=0 tx=0).stop 2024-04-25T10:15:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.657+0000 7f001a400640 1 -- 172.21.15.119:0/3303981798 reap_dead start 2024-04-25T10:15:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.658+0000 7f001c400640 1 -- 172.21.15.119:0/3303981798 shutdown_connections 2024-04-25T10:15:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.658+0000 7f001c400640 1 -- 172.21.15.119:0/3303981798 >> 172.21.15.119:0/3303981798 conn(0x7f00140cf5a0 msgr2=0x7f00140cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.658+0000 7f001c400640 1 -- 172.21.15.119:0/3303981798 shutdown_connections 2024-04-25T10:15:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.658+0000 7f001c400640 1 -- 172.21.15.119:0/3303981798 wait complete. 2024-04-25T10:15:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.660+0000 7f001c400640 1 Processor -- start 2024-04-25T10:15:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.660+0000 7f001c400640 1 -- start start 2024-04-25T10:15:48.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.661+0000 7f001c400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efffc0039b0 0x7f0014146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.661+0000 7f001c400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00141513d0 0x7f0014151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.661+0000 7f001c400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f001414f270 0x7f001414f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.661+0000 7f001c400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00141597b0 con 0x7efffc0039b0 2024-04-25T10:15:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.661+0000 7f001c400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00140b1a30 con 0x7f001414f270 2024-04-25T10:15:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.661+0000 7f001c400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0014159930 con 0x7f00141513d0 2024-04-25T10:15:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.662+0000 7f0019a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efffc0039b0 0x7f0014146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.662+0000 7f0019a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efffc0039b0 0x7f0014146030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59516/0 (socket says 172.21.15.119:59516) 2024-04-25T10:15:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.662+0000 7f0019a00640 1 -- 172.21.15.119:0/1535618170 learned_addr learned my addr 172.21.15.119:0/1535618170 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.662+0000 7f001a400640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f001414f270 0x7f001414f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.662+0000 7f0019000640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00141513d0 0x7f0014151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.662+0000 7f0019a00640 1 -- 172.21.15.119:0/1535618170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00141513d0 msgr2=0x7f0014151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.662+0000 7f0019a00640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00141513d0 0x7f0014151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.662+0000 7f0019a00640 1 -- 172.21.15.119:0/1535618170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f001414f270 msgr2=0x7f001414f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.662+0000 7f0019a00640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f001414f270 0x7f001414f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.662+0000 7f0019a00640 1 -- 172.21.15.119:0/1535618170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0014146570 con 0x7efffc0039b0 2024-04-25T10:15:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.662+0000 7f0019000640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00141513d0 0x7f0014151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.662+0000 7f0019a00640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efffc0039b0 0x7f0014146030 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f0008034040 tx=0x7f0008000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.663+0000 7f0011600640 1 -- 172.21.15.119:0/1535618170 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008036030 con 0x7efffc0039b0 2024-04-25T10:15:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.663+0000 7f0011600640 1 -- 172.21.15.119:0/1535618170 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0008042070 con 0x7efffc0039b0 2024-04-25T10:15:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.663+0000 7f0011600640 1 -- 172.21.15.119:0/1535618170 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00080074c0 con 0x7efffc0039b0 2024-04-25T10:15:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.663+0000 7f001c400640 1 -- 172.21.15.119:0/1535618170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0014151d70 con 0x7efffc0039b0 2024-04-25T10:15:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.663+0000 7f001c400640 1 -- 172.21.15.119:0/1535618170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00141ef5a0 con 0x7efffc0039b0 2024-04-25T10:15:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.664+0000 7f001c400640 1 -- 172.21.15.119:0/1535618170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00140749b0 con 0x7efffc0039b0 2024-04-25T10:15:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.665+0000 7f0011600640 1 -- 172.21.15.119:0/1535618170 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0008007660 con 0x7efffc0039b0 2024-04-25T10:15:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.666+0000 7f0011600640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efff406b4d0 0x7efff406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.666+0000 7f0011600640 1 -- 172.21.15.119:0/1535618170 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f00080a9e60 con 0x7efffc0039b0 2024-04-25T10:15:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.666+0000 7f0019000640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efff406b4d0 0x7efff406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.667+0000 7f0019000640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efff406b4d0 0x7efff406d990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f001415bd40 tx=0x7f0000008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:48.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.670+0000 7f0011600640 1 -- 172.21.15.119:0/1535618170 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0014146570 con 0x7efffc0039b0 2024-04-25T10:15:48.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.865+0000 7f001c400640 1 -- 172.21.15.119:0/1535618170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0014158c60 con 0x7efffc0039b0 2024-04-25T10:15:48.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.866+0000 7f0011600640 1 -- 172.21.15.119:0/1535618170 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v92) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0008005aa0 con 0x7efffc0039b0 2024-04-25T10:15:48.870 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:48.870 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2024-04-25T10:15:48.871 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:15:48.871 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2024-04-25T10:15:48.871 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2024-04-25T10:15:48.871 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:15:48.871 INFO:teuthology.orchestra.run.smithi119.stdout:] 2024-04-25T10:15:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.869+0000 7f001c400640 1 -- 172.21.15.119:0/1535618170 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efff406b4d0 msgr2=0x7efff406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.869+0000 7f001c400640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efff406b4d0 0x7efff406d990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f001415bd40 tx=0x7f0000008040 comp rx=0 tx=0).stop 2024-04-25T10:15:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.869+0000 7f001c400640 1 -- 172.21.15.119:0/1535618170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efffc0039b0 msgr2=0x7f0014146030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.869+0000 7f001c400640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efffc0039b0 0x7f0014146030 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f0008034040 tx=0x7f0008000fe0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.870+0000 7f001c400640 1 -- 172.21.15.119:0/1535618170 shutdown_connections 2024-04-25T10:15:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.870+0000 7f001c400640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efff406b4d0 0x7efff406d990 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.870+0000 7f001c400640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f001414f270 0x7f001414f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.870+0000 7f001c400640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00141513d0 0x7f0014151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.870+0000 7f001c400640 1 --2- 172.21.15.119:0/1535618170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efffc0039b0 0x7f0014146030 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:48.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.870+0000 7f001c400640 1 -- 172.21.15.119:0/1535618170 >> 172.21.15.119:0/1535618170 conn(0x7f00140cf5a0 msgr2=0x7f00140b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:48.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.870+0000 7f001c400640 1 -- 172.21.15.119:0/1535618170 shutdown_connections 2024-04-25T10:15:48.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:48.870+0000 7f001c400640 1 -- 172.21.15.119:0/1535618170 wait complete. 2024-04-25T10:15:48.893 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:15:48.894 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:15:49.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.085+0000 7f417f200640 1 Processor -- start 2024-04-25T10:15:49.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.085+0000 7f417f200640 1 -- start start 2024-04-25T10:15:49.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.087+0000 7f417f200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f418015d9c0 0x7f41800ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.087+0000 7f417f200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41800b6b00 con 0x7f41800b2a40 2024-04-25T10:15:49.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.087+0000 7f417f200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41800b9530 con 0x7f41800b2480 2024-04-25T10:15:49.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.087+0000 7f417f200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41800b7110 con 0x7f418015d9c0 2024-04-25T10:15:49.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.087+0000 7f417d400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f41800b2a40 0x7f41800b75a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47938/0 (socket says 172.21.15.119:47938) 2024-04-25T10:15:49.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.087+0000 7f417d400640 1 -- 172.21.15.119:0/4215908270 learned_addr learned my addr 172.21.15.119:0/4215908270 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417de00640 1 --2- 172.21.15.119:0/4215908270 >> v2:172.21.15.132:3300/0 conn(0x7f418015d9c0 0x7f41800ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 -- 172.21.15.119:0/4215908270 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522320240 0 0) 0x7f41800b6b00 con 0x7f41800b2a40 2024-04-25T10:15:49.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 -- 172.21.15.119:0/4215908270 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4164003520 con 0x7f41800b2a40 2024-04-25T10:15:49.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 -- 172.21.15.119:0/4215908270 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3854637723 0 0) 0x7f4164003520 con 0x7f41800b2a40 2024-04-25T10:15:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 -- 172.21.15.119:0/4215908270 >> v2:172.21.15.132:3300/0 conn(0x7f418015d9c0 msgr2=0x7f41800ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 --2- 172.21.15.119:0/4215908270 >> v2:172.21.15.132:3300/0 conn(0x7f418015d9c0 0x7f41800ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 -- 172.21.15.119:0/4215908270 >> v1:172.21.15.132:6789/0 conn(0x7f41800b2480 legacy=0x7f41800b7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 -- 172.21.15.119:0/4215908270 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41800b6da0 con 0x7f41800b2a40 2024-04-25T10:15:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 -- 172.21.15.119:0/4215908270 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f416c001690 con 0x7f41800b2a40 2024-04-25T10:15:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 -- 172.21.15.119:0/4215908270 >> v1:172.21.15.119:6789/0 conn(0x7f41800b2a40 legacy=0x7f41800b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 --2- 172.21.15.119:0/4215908270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4164003a00 0x7f4164003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 --2- 172.21.15.119:0/4215908270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4164004800 0x7f4164004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 --2- 172.21.15.119:0/4215908270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41640051c0 0x7f416400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 -- 172.21.15.119:0/4215908270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416400c090 con 0x7f4164003a00 2024-04-25T10:15:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 -- 172.21.15.119:0/4215908270 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416400c240 con 0x7f4164004800 2024-04-25T10:15:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.088+0000 7f417ca00640 1 -- 172.21.15.119:0/4215908270 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416400c440 con 0x7f41640051c0 2024-04-25T10:15:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.089+0000 7f417de00640 1 --2- 172.21.15.119:0/4215908270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4164003a00 0x7f4164003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.089+0000 7f417d400640 1 --2- 172.21.15.119:0/4215908270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4164004800 0x7f4164004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.089+0000 7f417e800640 1 --2- 172.21.15.119:0/4215908270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41640051c0 0x7f416400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.089+0000 7f417de00640 1 -- 172.21.15.119:0/4215908270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41640051c0 msgr2=0x7f416400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.089+0000 7f417de00640 1 --2- 172.21.15.119:0/4215908270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41640051c0 0x7f416400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.089+0000 7f417de00640 1 -- 172.21.15.119:0/4215908270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4164004800 msgr2=0x7f4164004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.089+0000 7f417de00640 1 --2- 172.21.15.119:0/4215908270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4164004800 0x7f4164004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.089+0000 7f417de00640 1 -- 172.21.15.119:0/4215908270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4164003520 con 0x7f4164003a00 2024-04-25T10:15:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.089+0000 7f417e800640 1 --2- 172.21.15.119:0/4215908270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41640051c0 0x7f416400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.089+0000 7f417e800640 1 -- 172.21.15.119:0/4215908270 reap_dead start 2024-04-25T10:15:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.090+0000 7f417de00640 1 --2- 172.21.15.119:0/4215908270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4164003a00 0x7f4164003e20 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f4170009ac0 tx=0x7f4164005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b731f1e7860a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.090+0000 7f417ca00640 1 -- 172.21.15.119:0/4215908270 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170036070 con 0x7f4164003a00 2024-04-25T10:15:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.090+0000 7f417ca00640 1 -- 172.21.15.119:0/4215908270 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4170031400 con 0x7f4164003a00 2024-04-25T10:15:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.090+0000 7f417ca00640 1 -- 172.21.15.119:0/4215908270 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170009690 con 0x7f4164003a00 2024-04-25T10:15:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.091+0000 7f417f200640 1 -- 172.21.15.119:0/4215908270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4164003a00 msgr2=0x7f4164003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.091+0000 7f417f200640 1 --2- 172.21.15.119:0/4215908270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4164003a00 0x7f4164003e20 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f4170009ac0 tx=0x7f4164005830 comp rx=0 tx=0).stop 2024-04-25T10:15:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.091+0000 7f417f200640 1 -- 172.21.15.119:0/4215908270 shutdown_connections 2024-04-25T10:15:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.091+0000 7f417f200640 1 --2- 172.21.15.119:0/4215908270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4164003a00 0x7f4164003e20 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.091+0000 7f417f200640 1 -- 172.21.15.119:0/4215908270 >> 172.21.15.119:0/4215908270 conn(0x7f41800726c0 msgr2=0x7f4180072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.091+0000 7f417f200640 1 -- 172.21.15.119:0/4215908270 shutdown_connections 2024-04-25T10:15:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.091+0000 7f417f200640 1 -- 172.21.15.119:0/4215908270 wait complete. 2024-04-25T10:15:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.093+0000 7f417f200640 1 Processor -- start 2024-04-25T10:15:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.093+0000 7f417f200640 1 -- start start 2024-04-25T10:15:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.094+0000 7f417f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4164003a00 0x7f4180155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.094+0000 7f417f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4180153bd0 0x7f4180154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.094+0000 7f417f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4180146150 0x7f41801465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.094+0000 7f417f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800b9150 con 0x7f4164003a00 2024-04-25T10:15:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.094+0000 7f417f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418015dda0 con 0x7f4180146150 2024-04-25T10:15:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.094+0000 7f417f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41800b92d0 con 0x7f4180153bd0 2024-04-25T10:15:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.094+0000 7f417de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4164003a00 0x7f4180155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.094+0000 7f417de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4164003a00 0x7f4180155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59542/0 (socket says 172.21.15.119:59542) 2024-04-25T10:15:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.094+0000 7f417de00640 1 -- 172.21.15.119:0/2929114669 learned_addr learned my addr 172.21.15.119:0/2929114669 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.095+0000 7f417d400640 1 --2- 172.21.15.119:0/2929114669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4180153bd0 0x7f4180154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.095+0000 7f417de00640 1 -- 172.21.15.119:0/2929114669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4180153bd0 msgr2=0x7f4180154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.095+0000 7f417e800640 1 --2- 172.21.15.119:0/2929114669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4180146150 0x7f41801465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.095+0000 7f417de00640 1 --2- 172.21.15.119:0/2929114669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4180153bd0 0x7f4180154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.095+0000 7f417de00640 1 -- 172.21.15.119:0/2929114669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4180146150 msgr2=0x7f41801465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:49.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.095+0000 7f417de00640 1 --2- 172.21.15.119:0/2929114669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4180146150 0x7f41801465e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:49.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.095+0000 7f417de00640 1 -- 172.21.15.119:0/2929114669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41801563d0 con 0x7f4164003a00 2024-04-25T10:15:49.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.095+0000 7f417de00640 1 --2- 172.21.15.119:0/2929114669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4164003a00 0x7f4180155aa0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f4170034040 tx=0x7f4170004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:49.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.095+0000 7f4176000640 1 -- 172.21.15.119:0/2929114669 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417003f020 con 0x7f4164003a00 2024-04-25T10:15:49.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.095+0000 7f4176000640 1 -- 172.21.15.119:0/2929114669 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4170037040 con 0x7f4164003a00 2024-04-25T10:15:49.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.095+0000 7f417f200640 1 -- 172.21.15.119:0/2929114669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41801566b0 con 0x7f4164003a00 2024-04-25T10:15:49.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.095+0000 7f4176000640 1 -- 172.21.15.119:0/2929114669 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170036070 con 0x7f4164003a00 2024-04-25T10:15:49.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.096+0000 7f417f200640 1 -- 172.21.15.119:0/2929114669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4180156c70 con 0x7f4164003a00 2024-04-25T10:15:49.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.097+0000 7f417f200640 1 -- 172.21.15.119:0/2929114669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41800749b0 con 0x7f4164003a00 2024-04-25T10:15:49.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.098+0000 7f4176000640 1 -- 172.21.15.119:0/2929114669 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4170042020 con 0x7f4164003a00 2024-04-25T10:15:49.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.099+0000 7f4176000640 1 --2- 172.21.15.119:0/2929114669 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f416006b500 0x7f416006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:49.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.099+0000 7f4176000640 1 -- 172.21.15.119:0/2929114669 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f41700a9e50 con 0x7f4164003a00 2024-04-25T10:15:49.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.099+0000 7f417d400640 1 --2- 172.21.15.119:0/2929114669 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f416006b500 0x7f416006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:49.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.100+0000 7f417d400640 1 --2- 172.21.15.119:0/2929114669 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f416006b500 0x7f416006d9c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f41800b2e80 tx=0x7f416c000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:49.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.104+0000 7f4176000640 1 -- 172.21.15.119:0/2929114669 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4170049030 con 0x7f4164003a00 2024-04-25T10:15:49.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:49.297+0000 7f417f200640 1 -- 172.21.15.119:0/2929114669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f41800c9fd0 con 0x7f4164003a00 2024-04-25T10:15:50.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.097+0000 7f4176000640 1 -- 172.21.15.119:0/2929114669 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v93) v1 ==== 138+0+0 (secure 0 0 0) 0x7f41700790d0 con 0x7f4164003a00 2024-04-25T10:15:50.101 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2024-04-25T10:15:50.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.102+0000 7f417f200640 1 -- 172.21.15.119:0/2929114669 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f416006b500 msgr2=0x7f416006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.102+0000 7f417f200640 1 --2- 172.21.15.119:0/2929114669 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f416006b500 0x7f416006d9c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f41800b2e80 tx=0x7f416c000ed0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.102+0000 7f417f200640 1 -- 172.21.15.119:0/2929114669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4164003a00 msgr2=0x7f4180155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.102+0000 7f417f200640 1 --2- 172.21.15.119:0/2929114669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4164003a00 0x7f4180155aa0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f4170034040 tx=0x7f4170004730 comp rx=0 tx=0).stop 2024-04-25T10:15:50.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.103+0000 7f417f200640 1 -- 172.21.15.119:0/2929114669 shutdown_connections 2024-04-25T10:15:50.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.103+0000 7f417f200640 1 --2- 172.21.15.119:0/2929114669 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f416006b500 0x7f416006d9c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.103+0000 7f417f200640 1 --2- 172.21.15.119:0/2929114669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4180146150 0x7f41801465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.103+0000 7f417f200640 1 --2- 172.21.15.119:0/2929114669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4180153bd0 0x7f4180154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.103+0000 7f417f200640 1 --2- 172.21.15.119:0/2929114669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4164003a00 0x7f4180155aa0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.103+0000 7f417f200640 1 -- 172.21.15.119:0/2929114669 >> 172.21.15.119:0/2929114669 conn(0x7f41800726c0 msgr2=0x7f41800d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.104+0000 7f417f200640 1 -- 172.21.15.119:0/2929114669 shutdown_connections 2024-04-25T10:15:50.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.104+0000 7f417f200640 1 -- 172.21.15.119:0/2929114669 wait complete. 2024-04-25T10:15:50.133 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:15:50.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.320+0000 7f4219000640 1 Processor -- start 2024-04-25T10:15:50.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.321+0000 7f4219000640 1 -- start start 2024-04-25T10:15:50.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.322+0000 7f4219000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42141636d0 0x7f42140b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.322+0000 7f4219000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f42140b3610 0x7f4214162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.322+0000 7f4219000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42140b2d60 con 0x7f4214165480 2024-04-25T10:15:50.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.322+0000 7f4219000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42140b3a50 con 0x7f42140b3610 2024-04-25T10:15:50.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.322+0000 7f4219000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4214165ce0 con 0x7f42141636d0 2024-04-25T10:15:50.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.322+0000 7f4211e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4214165480 0x7f42140b2650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47944/0 (socket says 172.21.15.119:47944) 2024-04-25T10:15:50.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.323+0000 7f4211e00640 1 -- 172.21.15.119:0/599248024 learned_addr learned my addr 172.21.15.119:0/599248024 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:50.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.323+0000 7f4212800640 1 --2- 172.21.15.119:0/599248024 >> v2:172.21.15.119:3300/0 conn(0x7f42140b3610 0x7f4214162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.323+0000 7f420be00640 1 --2- 172.21.15.119:0/599248024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42141636d0 0x7f42140b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.323+0000 7f4212800640 1 -- 172.21.15.119:0/599248024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42141636d0 msgr2=0x7f42140b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.323+0000 7f4211400640 1 -- 172.21.15.119:0/599248024 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697198067 0 0) 0x7f42140b2d60 con 0x7f4214165480 2024-04-25T10:15:50.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.323+0000 7f4212800640 1 --2- 172.21.15.119:0/599248024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42141636d0 0x7f42140b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.323+0000 7f4212800640 1 -- 172.21.15.119:0/599248024 >> v1:172.21.15.119:6789/0 conn(0x7f4214165480 legacy=0x7f42140b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.323+0000 7f4212800640 1 -- 172.21.15.119:0/599248024 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4214165fc0 con 0x7f42140b3610 2024-04-25T10:15:50.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.324+0000 7f4212800640 1 --2- 172.21.15.119:0/599248024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42140b3610 0x7f4214162570 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f420c009d90 tx=0x7f420c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60e18345678a49c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.324+0000 7f4211400640 1 -- 172.21.15.119:0/599248024 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c031600 con 0x7f42140b3610 2024-04-25T10:15:50.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.324+0000 7f4211400640 1 -- 172.21.15.119:0/599248024 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c0317a0 con 0x7f42140b3610 2024-04-25T10:15:50.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.324+0000 7f4211400640 1 -- 172.21.15.119:0/599248024 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c031b70 con 0x7f42140b3610 2024-04-25T10:15:50.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.325+0000 7f4219000640 1 -- 172.21.15.119:0/599248024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42140b3610 msgr2=0x7f4214162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.325+0000 7f4219000640 1 --2- 172.21.15.119:0/599248024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42140b3610 0x7f4214162570 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f420c009d90 tx=0x7f420c02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.325+0000 7f4219000640 1 -- 172.21.15.119:0/599248024 shutdown_connections 2024-04-25T10:15:50.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.325+0000 7f4219000640 1 --2- 172.21.15.119:0/599248024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42140b3610 0x7f4214162570 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.325+0000 7f4219000640 1 --2- 172.21.15.119:0/599248024 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42141636d0 0x7f42140b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.325+0000 7f4219000640 1 -- 172.21.15.119:0/599248024 >> 172.21.15.119:0/599248024 conn(0x7f421406aba0 msgr2=0x7f421406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.325+0000 7f4219000640 1 -- 172.21.15.119:0/599248024 shutdown_connections 2024-04-25T10:15:50.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.325+0000 7f4219000640 1 -- 172.21.15.119:0/599248024 wait complete. 2024-04-25T10:15:50.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.327+0000 7f4219000640 1 Processor -- start 2024-04-25T10:15:50.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.327+0000 7f4219000640 1 -- start start 2024-04-25T10:15:50.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.328+0000 7f4219000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42140b3610 0x7f42141f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.328+0000 7f4219000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42141636d0 0x7f421414c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.328+0000 7f4219000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4214165480 0x7f4214159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.328+0000 7f4219000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4214166860 con 0x7f42140b3610 2024-04-25T10:15:50.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.328+0000 7f4219000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4214166580 con 0x7f4214165480 2024-04-25T10:15:50.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.328+0000 7f4219000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4214165860 con 0x7f42141636d0 2024-04-25T10:15:50.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.328+0000 7f4211e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42140b3610 0x7f42141f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.328+0000 7f4211e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42140b3610 0x7f42141f8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59556/0 (socket says 172.21.15.119:59556) 2024-04-25T10:15:50.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.328+0000 7f4211e00640 1 -- 172.21.15.119:0/1011959572 learned_addr learned my addr 172.21.15.119:0/1011959572 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:50.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.329+0000 7f420be00640 1 --2- 172.21.15.119:0/1011959572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42141636d0 0x7f421414c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.329+0000 7f4212800640 1 --2- 172.21.15.119:0/1011959572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4214165480 0x7f4214159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.329+0000 7f4211e00640 1 -- 172.21.15.119:0/1011959572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42141636d0 msgr2=0x7f421414c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.329+0000 7f4211e00640 1 --2- 172.21.15.119:0/1011959572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42141636d0 0x7f421414c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.329+0000 7f4211e00640 1 -- 172.21.15.119:0/1011959572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4214165480 msgr2=0x7f4214159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.329+0000 7f4211e00640 1 --2- 172.21.15.119:0/1011959572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4214165480 0x7f4214159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.329+0000 7f4211e00640 1 -- 172.21.15.119:0/1011959572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421414cd80 con 0x7f42140b3610 2024-04-25T10:15:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.329+0000 7f4211e00640 1 --2- 172.21.15.119:0/1011959572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42140b3610 0x7f42141f8110 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f420400c430 tx=0x7f420400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.329+0000 7f420a000640 1 -- 172.21.15.119:0/1011959572 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42040074e0 con 0x7f42140b3610 2024-04-25T10:15:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.330+0000 7f420a000640 1 -- 172.21.15.119:0/1011959572 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42040047e0 con 0x7f42140b3610 2024-04-25T10:15:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.330+0000 7f420a000640 1 -- 172.21.15.119:0/1011959572 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204004bd0 con 0x7f42140b3610 2024-04-25T10:15:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.330+0000 7f4219000640 1 -- 172.21.15.119:0/1011959572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42140c5410 con 0x7f42140b3610 2024-04-25T10:15:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.330+0000 7f4219000640 1 -- 172.21.15.119:0/1011959572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42140c59d0 con 0x7f42140b3610 2024-04-25T10:15:50.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.331+0000 7f4219000640 1 -- 172.21.15.119:0/1011959572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42140749b0 con 0x7f42140b3610 2024-04-25T10:15:50.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.332+0000 7f420a000640 1 -- 172.21.15.119:0/1011959572 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4204011070 con 0x7f42140b3610 2024-04-25T10:15:50.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.333+0000 7f420a000640 1 --2- 172.21.15.119:0/1011959572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41f006b4d0 0x7f41f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.333+0000 7f420a000640 1 -- 172.21.15.119:0/1011959572 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4204089d30 con 0x7f42140b3610 2024-04-25T10:15:50.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.333+0000 7f420be00640 1 --2- 172.21.15.119:0/1011959572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41f006b4d0 0x7f41f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.334+0000 7f420be00640 1 --2- 172.21.15.119:0/1011959572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41f006b4d0 0x7f41f006d990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4214161790 tx=0x7f41fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.338+0000 7f420a000640 1 -- 172.21.15.119:0/1011959572 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4204058cf0 con 0x7f42140b3610 2024-04-25T10:15:50.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:50.531+0000 7f4219000640 1 -- 172.21.15.119:0/1011959572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f42140b3d90 con 0x7f42140b3610 2024-04-25T10:15:51.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.112+0000 7f420a000640 1 -- 172.21.15.119:0/1011959572 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v94) v1 ==== 170+0+0 (secure 0 0 0) 0x7f42040585f0 con 0x7f42140b3610 2024-04-25T10:15:51.116 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2024-04-25T10:15:51.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.115+0000 7f4219000640 1 -- 172.21.15.119:0/1011959572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41f006b4d0 msgr2=0x7f41f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.115+0000 7f4219000640 1 --2- 172.21.15.119:0/1011959572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41f006b4d0 0x7f41f006d990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4214161790 tx=0x7f41fc008040 comp rx=0 tx=0).stop 2024-04-25T10:15:51.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.115+0000 7f4219000640 1 -- 172.21.15.119:0/1011959572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42140b3610 msgr2=0x7f42141f8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.115+0000 7f4219000640 1 --2- 172.21.15.119:0/1011959572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42140b3610 0x7f42141f8110 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f420400c430 tx=0x7f420400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.116+0000 7f4219000640 1 -- 172.21.15.119:0/1011959572 shutdown_connections 2024-04-25T10:15:51.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.116+0000 7f4219000640 1 --2- 172.21.15.119:0/1011959572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41f006b4d0 0x7f41f006d990 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.116+0000 7f4219000640 1 --2- 172.21.15.119:0/1011959572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4214165480 0x7f4214159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.116+0000 7f4219000640 1 --2- 172.21.15.119:0/1011959572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42141636d0 0x7f421414c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.116+0000 7f4219000640 1 --2- 172.21.15.119:0/1011959572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42140b3610 0x7f42141f8110 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.116+0000 7f4219000640 1 -- 172.21.15.119:0/1011959572 >> 172.21.15.119:0/1011959572 conn(0x7f421406aba0 msgr2=0x7f4214163b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.116+0000 7f4219000640 1 -- 172.21.15.119:0/1011959572 shutdown_connections 2024-04-25T10:15:51.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.116+0000 7f4219000640 1 -- 172.21.15.119:0/1011959572 wait complete. 2024-04-25T10:15:51.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:15:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.317+0000 7faa43800640 1 Processor -- start 2024-04-25T10:15:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.317+0000 7faa43800640 1 -- start start 2024-04-25T10:15:51.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.319+0000 7faa43800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa3c15dc80 0x7faa3c161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.319+0000 7faa43800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faa3c0b97c0 0x7faa3c0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.319+0000 7faa43800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa3c1627a0 con 0x7faa3c0b5a30 2024-04-25T10:15:51.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.319+0000 7faa43800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa3c0b5ea0 con 0x7faa3c0b97c0 2024-04-25T10:15:51.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.319+0000 7faa43800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c1621e0 con 0x7faa3c15dc80 2024-04-25T10:15:51.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.319+0000 7faa3be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faa3c0b97c0 0x7faa3c0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.319+0000 7faa3be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faa3c0b97c0 0x7faa3c0b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46186/0 (socket says 172.21.15.119:46186) 2024-04-25T10:15:51.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.319+0000 7faa3be00640 1 -- 172.21.15.119:0/1363272844 learned_addr learned my addr 172.21.15.119:0/1363272844 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:51.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.319+0000 7faa41800640 1 --1- 172.21.15.119:0/1363272844 >> v1:172.21.15.119:6789/0 conn(0x7faa3c0b5a30 0x7faa3c06bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57366/0 (socket says 172.21.15.119:57366) 2024-04-25T10:15:51.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.319+0000 7faa40e00640 1 --2- 172.21.15.119:0/1363272844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa3c15dc80 0x7faa3c161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.320+0000 7faa3be00640 1 -- 172.21.15.119:0/1363272844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa3c15dc80 msgr2=0x7faa3c161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:51.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.320+0000 7faa3be00640 1 --2- 172.21.15.119:0/1363272844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa3c15dc80 0x7faa3c161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.320+0000 7faa3be00640 1 -- 172.21.15.119:0/1363272844 >> v1:172.21.15.119:6789/0 conn(0x7faa3c0b5a30 legacy=0x7faa3c06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.320+0000 7faa3be00640 1 -- 172.21.15.119:0/1363272844 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c1624c0 con 0x7faa3c0b97c0 2024-04-25T10:15:51.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.320+0000 7faa3b400640 1 -- 172.21.15.119:0/1363272844 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502922833 0 0) 0x7faa3c1627a0 con 0x7faa3c0b5a30 2024-04-25T10:15:51.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.321+0000 7faa3be00640 1 --2- 172.21.15.119:0/1363272844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c0b97c0 0x7faa3c0b54f0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7faa28009870 tx=0x7faa2802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c7db813f191b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.321+0000 7faa3b400640 1 -- 172.21.15.119:0/1363272844 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28037070 con 0x7faa3c0b97c0 2024-04-25T10:15:51.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.321+0000 7faa3b400640 1 -- 172.21.15.119:0/1363272844 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa28032440 con 0x7faa3c0b97c0 2024-04-25T10:15:51.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.321+0000 7faa3b400640 1 -- 172.21.15.119:0/1363272844 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28032810 con 0x7faa3c0b97c0 2024-04-25T10:15:51.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.321+0000 7faa43800640 1 -- 172.21.15.119:0/1363272844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c0b97c0 msgr2=0x7faa3c0b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.321+0000 7faa43800640 1 --2- 172.21.15.119:0/1363272844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c0b97c0 0x7faa3c0b54f0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7faa28009870 tx=0x7faa2802a250 comp rx=0 tx=0).stop 2024-04-25T10:15:51.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.322+0000 7faa43800640 1 -- 172.21.15.119:0/1363272844 shutdown_connections 2024-04-25T10:15:51.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.322+0000 7faa43800640 1 --2- 172.21.15.119:0/1363272844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c0b97c0 0x7faa3c0b54f0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.322+0000 7faa43800640 1 --2- 172.21.15.119:0/1363272844 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa3c15dc80 0x7faa3c161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.322+0000 7faa43800640 1 -- 172.21.15.119:0/1363272844 >> 172.21.15.119:0/1363272844 conn(0x7faa3c0d04f0 msgr2=0x7faa3c0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.322+0000 7faa43800640 1 -- 172.21.15.119:0/1363272844 shutdown_connections 2024-04-25T10:15:51.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.322+0000 7faa43800640 1 -- 172.21.15.119:0/1363272844 wait complete. 2024-04-25T10:15:51.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.323+0000 7faa43800640 1 Processor -- start 2024-04-25T10:15:51.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.323+0000 7faa43800640 1 -- start start 2024-04-25T10:15:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.324+0000 7faa43800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa3c0b5a30 0x7faa3c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.324+0000 7faa43800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c0b97c0 0x7faa3c1f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.324+0000 7faa43800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa3c15dc80 0x7faa3c1f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.324+0000 7faa43800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c161db0 con 0x7faa3c0b97c0 2024-04-25T10:15:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.324+0000 7faa43800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c15e060 con 0x7faa3c15dc80 2024-04-25T10:15:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.324+0000 7faa43800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c161f30 con 0x7faa3c0b5a30 2024-04-25T10:15:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.325+0000 7faa3be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c0b97c0 0x7faa3c1f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.325+0000 7faa3be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c0b97c0 0x7faa3c1f0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46194/0 (socket says 172.21.15.119:46194) 2024-04-25T10:15:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.325+0000 7faa3be00640 1 -- 172.21.15.119:0/2092306316 learned_addr learned my addr 172.21.15.119:0/2092306316 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.325+0000 7faa3be00640 1 -- 172.21.15.119:0/2092306316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa3c0b5a30 msgr2=0x7faa3c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.325+0000 7faa3be00640 1 --2- 172.21.15.119:0/2092306316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa3c0b5a30 0x7faa3c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.325+0000 7faa3be00640 1 -- 172.21.15.119:0/2092306316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa3c15dc80 msgr2=0x7faa3c1f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.325+0000 7faa3be00640 1 --2- 172.21.15.119:0/2092306316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa3c15dc80 0x7faa3c1f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.325+0000 7faa3be00640 1 -- 172.21.15.119:0/2092306316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c1f4900 con 0x7faa3c0b97c0 2024-04-25T10:15:51.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.325+0000 7faa3be00640 1 --2- 172.21.15.119:0/2092306316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c0b97c0 0x7faa3c1f0ee0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7faa28000c00 tx=0x7faa280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.326+0000 7faa38c00640 1 -- 172.21.15.119:0/2092306316 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28041020 con 0x7faa3c0b97c0 2024-04-25T10:15:51.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.326+0000 7faa38c00640 1 -- 172.21.15.119:0/2092306316 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa280306c0 con 0x7faa3c0b97c0 2024-04-25T10:15:51.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.326+0000 7faa38c00640 1 -- 172.21.15.119:0/2092306316 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28037040 con 0x7faa3c0b97c0 2024-04-25T10:15:51.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.326+0000 7faa43800640 1 -- 172.21.15.119:0/2092306316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa3c1f4060 con 0x7faa3c0b97c0 2024-04-25T10:15:51.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.326+0000 7faa43800640 1 -- 172.21.15.119:0/2092306316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3c1f4620 con 0x7faa3c0b97c0 2024-04-25T10:15:51.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.327+0000 7faa43800640 1 -- 172.21.15.119:0/2092306316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa3c0750b0 con 0x7faa3c0b97c0 2024-04-25T10:15:51.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.328+0000 7faa38c00640 1 -- 172.21.15.119:0/2092306316 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa28044020 con 0x7faa3c0b97c0 2024-04-25T10:15:51.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.329+0000 7faa38c00640 1 --2- 172.21.15.119:0/2092306316 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa1c06b4d0 0x7faa1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.329+0000 7faa38c00640 1 -- 172.21.15.119:0/2092306316 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5609+0+0 (secure 0 0 0) 0x7faa280aa9d0 con 0x7faa3c0b97c0 2024-04-25T10:15:51.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.329+0000 7faa40e00640 1 --2- 172.21.15.119:0/2092306316 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa1c06b4d0 0x7faa1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.330+0000 7faa40e00640 1 --2- 172.21.15.119:0/2092306316 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa1c06b4d0 0x7faa1c06d990 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faa30006fd0 tx=0x7faa30008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.334+0000 7faa38c00640 1 -- 172.21.15.119:0/2092306316 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa2804b030 con 0x7faa3c0b97c0 2024-04-25T10:15:51.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:51.566+0000 7faa43800640 1 -- 172.21.15.119:0/2092306316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7faa3c0d1780 con 0x7faa3c0b97c0 2024-04-25T10:15:52.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.115+0000 7faa38c00640 1 -- 172.21.15.119:0/2092306316 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v79) v1 ==== 174+0+0 (secure 0 0 0) 0x7faa280798a0 con 0x7faa3c0b97c0 2024-04-25T10:15:52.120 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 14 and data pool 15 2024-04-25T10:15:52.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.121+0000 7faa43800640 1 -- 172.21.15.119:0/2092306316 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa1c06b4d0 msgr2=0x7faa1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.121+0000 7faa43800640 1 --2- 172.21.15.119:0/2092306316 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa1c06b4d0 0x7faa1c06d990 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faa30006fd0 tx=0x7faa30008040 comp rx=0 tx=0).stop 2024-04-25T10:15:52.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.121+0000 7faa43800640 1 -- 172.21.15.119:0/2092306316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c0b97c0 msgr2=0x7faa3c1f0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.121+0000 7faa43800640 1 --2- 172.21.15.119:0/2092306316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c0b97c0 0x7faa3c1f0ee0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7faa28000c00 tx=0x7faa280046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.121+0000 7faa43800640 1 -- 172.21.15.119:0/2092306316 shutdown_connections 2024-04-25T10:15:52.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.121+0000 7faa43800640 1 --2- 172.21.15.119:0/2092306316 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa1c06b4d0 0x7faa1c06d990 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.121+0000 7faa43800640 1 --2- 172.21.15.119:0/2092306316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa3c15dc80 0x7faa3c1f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.121+0000 7faa43800640 1 --2- 172.21.15.119:0/2092306316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c0b97c0 0x7faa3c1f0ee0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.121+0000 7faa43800640 1 --2- 172.21.15.119:0/2092306316 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa3c0b5a30 0x7faa3c0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.121+0000 7faa43800640 1 -- 172.21.15.119:0/2092306316 >> 172.21.15.119:0/2092306316 conn(0x7faa3c0d04f0 msgr2=0x7faa3c0cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.123+0000 7faa43800640 1 -- 172.21.15.119:0/2092306316 shutdown_connections 2024-04-25T10:15:52.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.124+0000 7faa43800640 1 -- 172.21.15.119:0/2092306316 wait complete. 2024-04-25T10:15:52.168 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:52.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.336+0000 7f3a58400640 1 Processor -- start 2024-04-25T10:15:52.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.336+0000 7f3a58400640 1 -- start start 2024-04-25T10:15:52.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.337+0000 7f3a58400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a5015bf00 0x7f3a500cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.337+0000 7f3a58400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3a500cc310 0x7f3a5015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.337+0000 7f3a58400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a500ce6c0 con 0x7f3a5015c820 2024-04-25T10:15:52.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.337+0000 7f3a58400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a501455e0 con 0x7f3a5015bf00 2024-04-25T10:15:52.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.337+0000 7f3a58400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a5006d450 con 0x7f3a500cc310 2024-04-25T10:15:52.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.338+0000 7f3a55a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a5015bf00 0x7f3a500cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.338+0000 7f3a55a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a5015bf00 0x7f3a500cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46202/0 (socket says 172.21.15.119:46202) 2024-04-25T10:15:52.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.338+0000 7f3a55a00640 1 -- 172.21.15.119:0/4175171556 learned_addr learned my addr 172.21.15.119:0/4175171556 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:52.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.338+0000 7f3a55a00640 1 -- 172.21.15.119:0/4175171556 >> v2:172.21.15.132:3300/0 conn(0x7f3a500cc310 msgr2=0x7f3a5015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.338+0000 7f3a55a00640 1 --2- 172.21.15.119:0/4175171556 >> v2:172.21.15.132:3300/0 conn(0x7f3a500cc310 0x7f3a5015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.338+0000 7f3a55a00640 1 -- 172.21.15.119:0/4175171556 >> v1:172.21.15.132:6789/0 conn(0x7f3a5015c820 legacy=0x7f3a5006d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.338+0000 7f3a55a00640 1 -- 172.21.15.119:0/4175171556 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a500cc7b0 con 0x7f3a5015bf00 2024-04-25T10:15:52.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.339+0000 7f3a55a00640 1 --2- 172.21.15.119:0/4175171556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5015bf00 0x7f3a500cbdd0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f3a5006b900 tx=0x7f3a4402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fd5f1796e9f155 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.339+0000 7f3a4fe00640 1 -- 172.21.15.119:0/4175171556 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a44037070 con 0x7f3a5015bf00 2024-04-25T10:15:52.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.339+0000 7f3a4fe00640 1 -- 172.21.15.119:0/4175171556 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a44032440 con 0x7f3a5015bf00 2024-04-25T10:15:52.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.339+0000 7f3a4fe00640 1 -- 172.21.15.119:0/4175171556 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a44032810 con 0x7f3a5015bf00 2024-04-25T10:15:52.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.340+0000 7f3a58400640 1 -- 172.21.15.119:0/4175171556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5015bf00 msgr2=0x7f3a500cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.340+0000 7f3a58400640 1 --2- 172.21.15.119:0/4175171556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5015bf00 0x7f3a500cbdd0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f3a5006b900 tx=0x7f3a4402a230 comp rx=0 tx=0).stop 2024-04-25T10:15:52.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.340+0000 7f3a58400640 1 -- 172.21.15.119:0/4175171556 shutdown_connections 2024-04-25T10:15:52.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.340+0000 7f3a58400640 1 --2- 172.21.15.119:0/4175171556 >> v2:172.21.15.132:3300/0 conn(0x7f3a500cc310 0x7f3a5015c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.340+0000 7f3a58400640 1 --2- 172.21.15.119:0/4175171556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5015bf00 0x7f3a500cbdd0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.340+0000 7f3a58400640 1 -- 172.21.15.119:0/4175171556 >> 172.21.15.119:0/4175171556 conn(0x7f3a5006aba0 msgr2=0x7f3a5006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.340+0000 7f3a58400640 1 -- 172.21.15.119:0/4175171556 shutdown_connections 2024-04-25T10:15:52.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.340+0000 7f3a58400640 1 -- 172.21.15.119:0/4175171556 wait complete. 2024-04-25T10:15:52.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.342+0000 7f3a58400640 1 Processor -- start 2024-04-25T10:15:52.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.342+0000 7f3a58400640 1 -- start start 2024-04-25T10:15:52.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.343+0000 7f3a58400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a500cc310 0x7f3a5013a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.343+0000 7f3a58400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5015bf00 0x7f3a5015cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.344+0000 7f3a58400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5015c820 0x7f3a5015d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.344+0000 7f3a58400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500cd050 con 0x7f3a5015bf00 2024-04-25T10:15:52.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.344+0000 7f3a58400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500ccd70 con 0x7f3a5015c820 2024-04-25T10:15:52.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.344+0000 7f3a58400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5015ccd0 con 0x7f3a500cc310 2024-04-25T10:15:52.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.344+0000 7f3a55000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5015bf00 0x7f3a5015cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.344+0000 7f3a55000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5015bf00 0x7f3a5015cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46214/0 (socket says 172.21.15.119:46214) 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.344+0000 7f3a55000640 1 -- 172.21.15.119:0/3128536615 learned_addr learned my addr 172.21.15.119:0/3128536615 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.344+0000 7f3a56400640 1 --2- 172.21.15.119:0/3128536615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5015c820 0x7f3a5015d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.344+0000 7f3a55000640 1 -- 172.21.15.119:0/3128536615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a500cc310 msgr2=0x7f3a5013a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.344+0000 7f3a55000640 1 --2- 172.21.15.119:0/3128536615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a500cc310 0x7f3a5013a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.344+0000 7f3a55000640 1 -- 172.21.15.119:0/3128536615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5015c820 msgr2=0x7f3a5015d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.344+0000 7f3a55000640 1 --2- 172.21.15.119:0/3128536615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5015c820 0x7f3a5015d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.344+0000 7f3a55000640 1 -- 172.21.15.119:0/3128536615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a50141e60 con 0x7f3a5015bf00 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.344+0000 7f3a56400640 1 --2- 172.21.15.119:0/3128536615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5015c820 0x7f3a5015d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:52.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.345+0000 7f3a55000640 1 --2- 172.21.15.119:0/3128536615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5015bf00 0x7f3a5015cef0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00b450 tx=0x7f3a3c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.345+0000 7f3a4d600640 1 -- 172.21.15.119:0/3128536615 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c015020 con 0x7f3a5015bf00 2024-04-25T10:15:52.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.345+0000 7f3a4d600640 1 -- 172.21.15.119:0/3128536615 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c011890 con 0x7f3a5015bf00 2024-04-25T10:15:52.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.345+0000 7f3a4d600640 1 -- 172.21.15.119:0/3128536615 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c011c80 con 0x7f3a5015bf00 2024-04-25T10:15:52.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.345+0000 7f3a58400640 1 -- 172.21.15.119:0/3128536615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a50142140 con 0x7f3a5015bf00 2024-04-25T10:15:52.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.345+0000 7f3a58400640 1 -- 172.21.15.119:0/3128536615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a50142700 con 0x7f3a5015bf00 2024-04-25T10:15:52.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.346+0000 7f3a58400640 1 -- 172.21.15.119:0/3128536615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a500749b0 con 0x7f3a5015bf00 2024-04-25T10:15:52.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.347+0000 7f3a4d600640 1 -- 172.21.15.119:0/3128536615 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a3c018020 con 0x7f3a5015bf00 2024-04-25T10:15:52.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.348+0000 7f3a4d600640 1 --2- 172.21.15.119:0/3128536615 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3a3006b4d0 0x7f3a3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.348+0000 7f3a4d600640 1 -- 172.21.15.119:0/3128536615 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a3c08a400 con 0x7f3a5015bf00 2024-04-25T10:15:52.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.349+0000 7f3a55a00640 1 --2- 172.21.15.119:0/3128536615 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3a3006b4d0 0x7f3a3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.350+0000 7f3a55a00640 1 --2- 172.21.15.119:0/3128536615 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3a3006b4d0 0x7f3a3006d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3a44034000 tx=0x7f3a4402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.353+0000 7f3a4d600640 1 -- 172.21.15.119:0/3128536615 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a3c01f020 con 0x7f3a5015bf00 2024-04-25T10:15:52.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.543+0000 7f3a58400640 1 -- 172.21.15.119:0/3128536615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a50142940 con 0x7f3a5015bf00 2024-04-25T10:15:52.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.546+0000 7f3a4d600640 1 -- 172.21.15.119:0/3128536615 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16657 (secure 0 0 0) 0x7f3a3c00d040 con 0x7f3a5015bf00 2024-04-25T10:15:52.551 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:52.551 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":95,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:15:51.567818+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:49.348469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:50.584869+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:52.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.549+0000 7f3a58400640 1 -- 172.21.15.119:0/3128536615 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3a3006b4d0 msgr2=0x7f3a3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.549+0000 7f3a58400640 1 --2- 172.21.15.119:0/3128536615 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3a3006b4d0 0x7f3a3006d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3a44034000 tx=0x7f3a4402c000 comp rx=0 tx=0).stop 2024-04-25T10:15:52.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.549+0000 7f3a58400640 1 -- 172.21.15.119:0/3128536615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5015bf00 msgr2=0x7f3a5015cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.549+0000 7f3a58400640 1 --2- 172.21.15.119:0/3128536615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5015bf00 0x7f3a5015cef0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00b450 tx=0x7f3a3c00b910 comp rx=0 tx=0).stop 2024-04-25T10:15:52.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.549+0000 7f3a58400640 1 -- 172.21.15.119:0/3128536615 shutdown_connections 2024-04-25T10:15:52.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.549+0000 7f3a58400640 1 --2- 172.21.15.119:0/3128536615 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3a3006b4d0 0x7f3a3006d990 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.549+0000 7f3a58400640 1 --2- 172.21.15.119:0/3128536615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5015c820 0x7f3a5015d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.549+0000 7f3a58400640 1 --2- 172.21.15.119:0/3128536615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a5015bf00 0x7f3a5015cef0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.549+0000 7f3a58400640 1 --2- 172.21.15.119:0/3128536615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a500cc310 0x7f3a5013a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.549+0000 7f3a58400640 1 -- 172.21.15.119:0/3128536615 >> 172.21.15.119:0/3128536615 conn(0x7f3a5006aba0 msgr2=0x7f3a500ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.549+0000 7f3a58400640 1 -- 172.21.15.119:0/3128536615 shutdown_connections 2024-04-25T10:15:52.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.549+0000 7f3a58400640 1 -- 172.21.15.119:0/3128536615 wait complete. 2024-04-25T10:15:52.571 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:52.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.754+0000 7f441a400640 1 Processor -- start 2024-04-25T10:15:52.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.754+0000 7f441a400640 1 -- start start 2024-04-25T10:15:52.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.756+0000 7f441a400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f441415acd0 0x7f441406e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.756+0000 7f441a400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f441406e860 0x7f44140b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.756+0000 7f441a400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441406ec70 con 0x7f4414144040 2024-04-25T10:15:52.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.756+0000 7f441a400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4414144420 con 0x7f441415acd0 2024-04-25T10:15:52.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.756+0000 7f441a400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44140b3d80 con 0x7f441406e860 2024-04-25T10:15:52.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.756+0000 7f4413e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f441415acd0 0x7f441406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.756+0000 7f4419000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4414144040 0x7f44140cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57376/0 (socket says 172.21.15.119:57376) 2024-04-25T10:15:52.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.756+0000 7f4419000640 1 -- 172.21.15.119:0/3468178902 learned_addr learned my addr 172.21.15.119:0/3468178902 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:52.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.757+0000 7f4413400640 1 -- 172.21.15.119:0/3468178902 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177653265 0 0) 0x7f4408001110 con 0x7f4414144040 2024-04-25T10:15:52.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.757+0000 7f4413400640 1 -- 172.21.15.119:0/3468178902 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43fc005730 con 0x7f4414144040 2024-04-25T10:15:52.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.757+0000 7f4419a00640 1 --2- 172.21.15.119:0/3468178902 >> v2:172.21.15.132:3300/0 conn(0x7f441406e860 0x7f44140b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.757+0000 7f4413e00640 1 -- 172.21.15.119:0/3468178902 >> v2:172.21.15.132:3300/0 conn(0x7f441406e860 msgr2=0x7f44140b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:52.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.757+0000 7f4413e00640 1 --2- 172.21.15.119:0/3468178902 >> v2:172.21.15.132:3300/0 conn(0x7f441406e860 0x7f44140b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.757+0000 7f4413e00640 1 -- 172.21.15.119:0/3468178902 >> v1:172.21.15.119:6789/0 conn(0x7f4414144040 legacy=0x7f44140cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.757+0000 7f4413e00640 1 -- 172.21.15.119:0/3468178902 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44140b4060 con 0x7f441415acd0 2024-04-25T10:15:52.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.758+0000 7f4413e00640 1 --2- 172.21.15.119:0/3468178902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f441415acd0 0x7f441406e320 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f44140b58d0 tx=0x7f440002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f26a6eb48982ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.760+0000 7f4413400640 1 -- 172.21.15.119:0/3468178902 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400037070 con 0x7f441415acd0 2024-04-25T10:15:52.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.760+0000 7f4413400640 1 -- 172.21.15.119:0/3468178902 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4400032440 con 0x7f441415acd0 2024-04-25T10:15:52.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.760+0000 7f4413400640 1 -- 172.21.15.119:0/3468178902 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440002ae60 con 0x7f441415acd0 2024-04-25T10:15:52.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.760+0000 7f441a400640 1 -- 172.21.15.119:0/3468178902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f441415acd0 msgr2=0x7f441406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.760+0000 7f441a400640 1 --2- 172.21.15.119:0/3468178902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f441415acd0 0x7f441406e320 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f44140b58d0 tx=0x7f440002a110 comp rx=0 tx=0).stop 2024-04-25T10:15:52.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.761+0000 7f441a400640 1 -- 172.21.15.119:0/3468178902 shutdown_connections 2024-04-25T10:15:52.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.761+0000 7f441a400640 1 --2- 172.21.15.119:0/3468178902 >> v2:172.21.15.132:3300/0 conn(0x7f441406e860 0x7f44140b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.761+0000 7f441a400640 1 --2- 172.21.15.119:0/3468178902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f441415acd0 0x7f441406e320 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.761+0000 7f441a400640 1 -- 172.21.15.119:0/3468178902 >> 172.21.15.119:0/3468178902 conn(0x7f441406b1d0 msgr2=0x7f44140b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.761+0000 7f441a400640 1 -- 172.21.15.119:0/3468178902 shutdown_connections 2024-04-25T10:15:52.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.761+0000 7f441a400640 1 -- 172.21.15.119:0/3468178902 wait complete. 2024-04-25T10:15:52.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.762+0000 7f441a400640 1 Processor -- start 2024-04-25T10:15:52.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.762+0000 7f441a400640 1 -- start start 2024-04-25T10:15:52.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.763+0000 7f441a400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f441406e860 0x7f4414149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.763+0000 7f441a400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4414144040 0x7f441413a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.763+0000 7f441a400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f441415acd0 0x7f441413a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.763+0000 7f441a400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44140b4900 con 0x7f4414144040 2024-04-25T10:15:52.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.763+0000 7f441a400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44140b4620 con 0x7f441406e860 2024-04-25T10:15:52.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.763+0000 7f441a400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44140b5180 con 0x7f441415acd0 2024-04-25T10:15:52.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.764+0000 7f4413e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4414144040 0x7f441413a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.764+0000 7f4413e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4414144040 0x7f441413a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46236/0 (socket says 172.21.15.119:46236) 2024-04-25T10:15:52.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.764+0000 7f4413e00640 1 -- 172.21.15.119:0/2977463137 learned_addr learned my addr 172.21.15.119:0/2977463137 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:52.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.764+0000 7f4419000640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f441406e860 0x7f4414149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.764+0000 7f4419a00640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f441415acd0 0x7f441413a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.764+0000 7f4413e00640 1 -- 172.21.15.119:0/2977463137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f441415acd0 msgr2=0x7f441413a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.764+0000 7f4413e00640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f441415acd0 0x7f441413a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.764+0000 7f4413e00640 1 -- 172.21.15.119:0/2977463137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f441406e860 msgr2=0x7f4414149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.764+0000 7f4413e00640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f441406e860 0x7f4414149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.764+0000 7f4413e00640 1 -- 172.21.15.119:0/2977463137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441413c5a0 con 0x7f4414144040 2024-04-25T10:15:52.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.764+0000 7f4419a00640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f441415acd0 0x7f441413a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:52.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.764+0000 7f4419000640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f441406e860 0x7f4414149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:52.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.764+0000 7f4413e00640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4414144040 0x7f441413a000 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f4400000c00 tx=0x7f44000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.765+0000 7f4410c00640 1 -- 172.21.15.119:0/2977463137 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400037040 con 0x7f4414144040 2024-04-25T10:15:52.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.765+0000 7f4410c00640 1 -- 172.21.15.119:0/2977463137 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4400042070 con 0x7f4414144040 2024-04-25T10:15:52.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.765+0000 7f4410c00640 1 -- 172.21.15.119:0/2977463137 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400032810 con 0x7f4414144040 2024-04-25T10:15:52.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.765+0000 7f441a400640 1 -- 172.21.15.119:0/2977463137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441413ef60 con 0x7f4414144040 2024-04-25T10:15:52.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.765+0000 7f441a400640 1 -- 172.21.15.119:0/2977463137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441413f520 con 0x7f4414144040 2024-04-25T10:15:52.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.766+0000 7f441a400640 1 -- 172.21.15.119:0/2977463137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44140749b0 con 0x7f4414144040 2024-04-25T10:15:52.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.767+0000 7f4410c00640 1 -- 172.21.15.119:0/2977463137 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44000329b0 con 0x7f4414144040 2024-04-25T10:15:52.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.768+0000 7f4410c00640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43f406b4d0 0x7f43f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:52.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.768+0000 7f4410c00640 1 -- 172.21.15.119:0/2977463137 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44000aabc0 con 0x7f4414144040 2024-04-25T10:15:52.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.768+0000 7f4419000640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43f406b4d0 0x7f43f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:52.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.769+0000 7f4419000640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43f406b4d0 0x7f43f406d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f44080012b0 tx=0x7f4408009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:52.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.772+0000 7f4410c00640 1 -- 172.21.15.119:0/2977463137 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4400079ac0 con 0x7f4414144040 2024-04-25T10:15:52.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.968+0000 7f441a400640 1 -- 172.21.15.119:0/2977463137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f441406ddc0 con 0x7f4414144040 2024-04-25T10:15:52.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.970+0000 7f4410c00640 1 -- 172.21.15.119:0/2977463137 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16657 (secure 0 0 0) 0x7f44000793c0 con 0x7f4414144040 2024-04-25T10:15:52.974 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:52.975 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":95,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:15:51.567818+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:49.348469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:50.584869+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.973+0000 7f441a400640 1 -- 172.21.15.119:0/2977463137 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43f406b4d0 msgr2=0x7f43f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.973+0000 7f441a400640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43f406b4d0 0x7f43f406d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f44080012b0 tx=0x7f4408009040 comp rx=0 tx=0).stop 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.973+0000 7f441a400640 1 -- 172.21.15.119:0/2977463137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4414144040 msgr2=0x7f441413a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.973+0000 7f441a400640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4414144040 0x7f441413a000 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f4400000c00 tx=0x7f44000046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.973+0000 7f441a400640 1 -- 172.21.15.119:0/2977463137 shutdown_connections 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.973+0000 7f441a400640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f43f406b4d0 0x7f43f406d990 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.973+0000 7f441a400640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f441415acd0 0x7f441413a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.973+0000 7f441a400640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4414144040 0x7f441413a000 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.973+0000 7f441a400640 1 --2- 172.21.15.119:0/2977463137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f441406e860 0x7f4414149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.973+0000 7f441a400640 1 -- 172.21.15.119:0/2977463137 >> 172.21.15.119:0/2977463137 conn(0x7f441406b1d0 msgr2=0x7f44140af310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.973+0000 7f441a400640 1 -- 172.21.15.119:0/2977463137 shutdown_connections 2024-04-25T10:15:52.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:52.973+0000 7f441a400640 1 -- 172.21.15.119:0/2977463137 wait complete. 2024-04-25T10:15:52.995 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:15:53.081 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:15:53.077+0000 7f1bcf400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:15:53.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.183+0000 7fd4f5600640 1 Processor -- start 2024-04-25T10:15:53.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.183+0000 7fd4f5600640 1 -- start start 2024-04-25T10:15:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.184+0000 7fd4f5600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4f00d01d0 0x7fd4f00cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.184+0000 7fd4f5600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd4f00d1b90 0x7fd4f00d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.184+0000 7fd4f5600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f01625f0 con 0x7fd4f00d05b0 2024-04-25T10:15:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.184+0000 7fd4f5600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f0162160 con 0x7fd4f00d01d0 2024-04-25T10:15:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.184+0000 7fd4f5600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f00cef60 con 0x7fd4f00d1b90 2024-04-25T10:15:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.185+0000 7fd4efe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4f00d01d0 0x7fd4f00cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.185+0000 7fd4efe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4f00d01d0 0x7fd4f00cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46240/0 (socket says 172.21.15.119:46240) 2024-04-25T10:15:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.185+0000 7fd4efe00640 1 -- 172.21.15.119:0/3266977221 learned_addr learned my addr 172.21.15.119:0/3266977221 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:53.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.185+0000 7fd4f4c00640 1 --2- 172.21.15.119:0/3266977221 >> v2:172.21.15.132:3300/0 conn(0x7fd4f00d1b90 0x7fd4f00d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.185+0000 7fd4efe00640 1 -- 172.21.15.119:0/3266977221 >> v2:172.21.15.132:3300/0 conn(0x7fd4f00d1b90 msgr2=0x7fd4f00d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.185+0000 7fd4efe00640 1 --2- 172.21.15.119:0/3266977221 >> v2:172.21.15.132:3300/0 conn(0x7fd4f00d1b90 0x7fd4f00d2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.185+0000 7fd4efe00640 1 -- 172.21.15.119:0/3266977221 >> v1:172.21.15.132:6789/0 conn(0x7fd4f00d05b0 legacy=0x7fd4f015ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.185+0000 7fd4efe00640 1 -- 172.21.15.119:0/3266977221 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f00cf200 con 0x7fd4f00d01d0 2024-04-25T10:15:53.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.186+0000 7fd4efe00640 1 --2- 172.21.15.119:0/3266977221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f00d01d0 0x7fd4f00cea20 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00d16e0 tx=0x7fd4e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e27338c26b7e7d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.187+0000 7fd4eea00640 1 -- 172.21.15.119:0/3266977221 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0037070 con 0x7fd4f00d01d0 2024-04-25T10:15:53.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.187+0000 7fd4eea00640 1 -- 172.21.15.119:0/3266977221 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e0032440 con 0x7fd4f00d01d0 2024-04-25T10:15:53.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.187+0000 7fd4eea00640 1 -- 172.21.15.119:0/3266977221 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0032810 con 0x7fd4f00d01d0 2024-04-25T10:15:53.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.187+0000 7fd4f5600640 1 -- 172.21.15.119:0/3266977221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f00d01d0 msgr2=0x7fd4f00cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.187+0000 7fd4f5600640 1 --2- 172.21.15.119:0/3266977221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f00d01d0 0x7fd4f00cea20 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00d16e0 tx=0x7fd4e002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.187+0000 7fd4f5600640 1 -- 172.21.15.119:0/3266977221 shutdown_connections 2024-04-25T10:15:53.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.187+0000 7fd4f5600640 1 --2- 172.21.15.119:0/3266977221 >> v2:172.21.15.132:3300/0 conn(0x7fd4f00d1b90 0x7fd4f00d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.187+0000 7fd4f5600640 1 --2- 172.21.15.119:0/3266977221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f00d01d0 0x7fd4f00cea20 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.187+0000 7fd4f5600640 1 -- 172.21.15.119:0/3266977221 >> 172.21.15.119:0/3266977221 conn(0x7fd4f006bed0 msgr2=0x7fd4f006c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.187+0000 7fd4f5600640 1 -- 172.21.15.119:0/3266977221 shutdown_connections 2024-04-25T10:15:53.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.187+0000 7fd4f5600640 1 -- 172.21.15.119:0/3266977221 wait complete. 2024-04-25T10:15:53.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.189+0000 7fd4f5600640 1 Processor -- start 2024-04-25T10:15:53.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.189+0000 7fd4f5600640 1 -- start start 2024-04-25T10:15:53.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.190+0000 7fd4f5600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f00d01d0 0x7fd4f0153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.190+0000 7fd4f5600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f00d05b0 0x7fd4f00c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.190+0000 7fd4f5600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f00d1b90 0x7fd4f0155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.190+0000 7fd4f5600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0161df0 con 0x7fd4f00d1b90 2024-04-25T10:15:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.190+0000 7fd4f5600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0161b10 con 0x7fd4f00d05b0 2024-04-25T10:15:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.190+0000 7fd4f5600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00d0a10 con 0x7fd4f00d01d0 2024-04-25T10:15:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.190+0000 7fd4f4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f00d1b90 0x7fd4f0155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.190+0000 7fd4f4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f00d1b90 0x7fd4f0155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46244/0 (socket says 172.21.15.119:46244) 2024-04-25T10:15:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.190+0000 7fd4f4c00640 1 -- 172.21.15.119:0/1900069473 learned_addr learned my addr 172.21.15.119:0/1900069473 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.191+0000 7fd4f4c00640 1 -- 172.21.15.119:0/1900069473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f00d01d0 msgr2=0x7fd4f0153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.191+0000 7fd4ef400640 1 --2- 172.21.15.119:0/1900069473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f00d05b0 0x7fd4f00c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.191+0000 7fd4efe00640 1 --2- 172.21.15.119:0/1900069473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f00d01d0 0x7fd4f0153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.191+0000 7fd4f4c00640 1 --2- 172.21.15.119:0/1900069473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f00d01d0 0x7fd4f0153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.191+0000 7fd4f4c00640 1 -- 172.21.15.119:0/1900069473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f00d05b0 msgr2=0x7fd4f00c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.191+0000 7fd4f4c00640 1 --2- 172.21.15.119:0/1900069473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f00d05b0 0x7fd4f00c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.191+0000 7fd4f4c00640 1 -- 172.21.15.119:0/1900069473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f00c9380 con 0x7fd4f00d1b90 2024-04-25T10:15:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.191+0000 7fd4f4c00640 1 --2- 172.21.15.119:0/1900069473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f00d1b90 0x7fd4f0155780 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800cc30 tx=0x7fd4e800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.191+0000 7fd4e7e00640 1 -- 172.21.15.119:0/1900069473 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800ce40 con 0x7fd4f00d1b90 2024-04-25T10:15:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.191+0000 7fd4e7e00640 1 -- 172.21.15.119:0/1900069473 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e8003990 con 0x7fd4f00d1b90 2024-04-25T10:15:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.191+0000 7fd4e7e00640 1 -- 172.21.15.119:0/1900069473 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8003d40 con 0x7fd4f00d1b90 2024-04-25T10:15:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.191+0000 7fd4f5600640 1 -- 172.21.15.119:0/1900069473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f00c9660 con 0x7fd4f00d1b90 2024-04-25T10:15:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.191+0000 7fd4f5600640 1 -- 172.21.15.119:0/1900069473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f00c9c20 con 0x7fd4f00d1b90 2024-04-25T10:15:53.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.192+0000 7fd4f5600640 1 -- 172.21.15.119:0/1900069473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f00749b0 con 0x7fd4f00d1b90 2024-04-25T10:15:53.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.193+0000 7fd4e7e00640 1 -- 172.21.15.119:0/1900069473 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4e8003ee0 con 0x7fd4f00d1b90 2024-04-25T10:15:53.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.194+0000 7fd4e7e00640 1 --2- 172.21.15.119:0/1900069473 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd4cc06b4d0 0x7fd4cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.194+0000 7fd4e7e00640 1 -- 172.21.15.119:0/1900069473 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4e808a750 con 0x7fd4f00d1b90 2024-04-25T10:15:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.195+0000 7fd4efe00640 1 --2- 172.21.15.119:0/1900069473 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd4cc06b4d0 0x7fd4cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.195+0000 7fd4efe00640 1 --2- 172.21.15.119:0/1900069473 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd4cc06b4d0 0x7fd4cc06d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0034000 tx=0x7fd4e002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.198+0000 7fd4e7e00640 1 -- 172.21.15.119:0/1900069473 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4e8014030 con 0x7fd4f00d1b90 2024-04-25T10:15:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:53.416+0000 7fd4f5600640 1 -- 172.21.15.119:0/1900069473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd4f00ce800 con 0x7fd4f00d1b90 2024-04-25T10:15:54.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.157+0000 7fd4e7e00640 1 -- 172.21.15.119:0/1900069473 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd4e8059530 con 0x7fd4f00d1b90 2024-04-25T10:15:54.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.161+0000 7fd4f5600640 1 -- 172.21.15.119:0/1900069473 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd4cc06b4d0 msgr2=0x7fd4cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.161+0000 7fd4f5600640 1 --2- 172.21.15.119:0/1900069473 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd4cc06b4d0 0x7fd4cc06d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0034000 tx=0x7fd4e002e000 comp rx=0 tx=0).stop 2024-04-25T10:15:54.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.161+0000 7fd4f5600640 1 -- 172.21.15.119:0/1900069473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f00d1b90 msgr2=0x7fd4f0155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.161+0000 7fd4f5600640 1 --2- 172.21.15.119:0/1900069473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f00d1b90 0x7fd4f0155780 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800cc30 tx=0x7fd4e800a5c0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.161+0000 7fd4f5600640 1 -- 172.21.15.119:0/1900069473 shutdown_connections 2024-04-25T10:15:54.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.161+0000 7fd4f5600640 1 --2- 172.21.15.119:0/1900069473 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd4cc06b4d0 0x7fd4cc06d990 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.161+0000 7fd4f5600640 1 --2- 172.21.15.119:0/1900069473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f00d1b90 0x7fd4f0155780 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.161+0000 7fd4f5600640 1 --2- 172.21.15.119:0/1900069473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f00d05b0 0x7fd4f00c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.161+0000 7fd4f5600640 1 --2- 172.21.15.119:0/1900069473 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd4f00d01d0 0x7fd4f0153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.161+0000 7fd4f5600640 1 -- 172.21.15.119:0/1900069473 >> 172.21.15.119:0/1900069473 conn(0x7fd4f006bed0 msgr2=0x7fd4f015bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.161+0000 7fd4f5600640 1 -- 172.21.15.119:0/1900069473 shutdown_connections 2024-04-25T10:15:54.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.161+0000 7fd4f5600640 1 -- 172.21.15.119:0/1900069473 wait complete. 2024-04-25T10:15:54.184 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:15:54.185 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:15:54.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.384+0000 7f6feaa00640 1 Processor -- start 2024-04-25T10:15:54.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.384+0000 7f6feaa00640 1 -- start start 2024-04-25T10:15:54.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.385+0000 7f6feaa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6fe4144e70 0x7f6fe406d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.386+0000 7f6feaa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6fe40cf0e0 0x7f6fe40cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.386+0000 7f6feaa00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe406dcb0 con 0x7f6fe406d7a0 2024-04-25T10:15:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.386+0000 7f6feaa00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe415bfe0 con 0x7f6fe40cf0e0 2024-04-25T10:15:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.386+0000 7f6feaa00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe406df90 con 0x7f6fe4144e70 2024-04-25T10:15:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.386+0000 7f6fe8a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6fe40cf0e0 0x7f6fe40cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.386+0000 7f6fe8a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6fe40cf0e0 0x7f6fe40cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46250/0 (socket says 172.21.15.119:46250) 2024-04-25T10:15:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.386+0000 7f6fe8a00640 1 -- 172.21.15.119:0/3054817448 learned_addr learned my addr 172.21.15.119:0/3054817448 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.386+0000 7f6fe3e00640 1 --2- 172.21.15.119:0/3054817448 >> v2:172.21.15.132:3300/0 conn(0x7f6fe4144e70 0x7f6fe406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.386+0000 7f6fe8a00640 1 -- 172.21.15.119:0/3054817448 >> v2:172.21.15.132:3300/0 conn(0x7f6fe4144e70 msgr2=0x7f6fe406d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:54.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.386+0000 7f6fe8a00640 1 --2- 172.21.15.119:0/3054817448 >> v2:172.21.15.132:3300/0 conn(0x7f6fe4144e70 0x7f6fe406d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.386+0000 7f6fe8a00640 1 -- 172.21.15.119:0/3054817448 >> v1:172.21.15.132:6789/0 conn(0x7f6fe406d7a0 legacy=0x7f6fe40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:54.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.386+0000 7f6fe8a00640 1 -- 172.21.15.119:0/3054817448 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe406e270 con 0x7f6fe40cf0e0 2024-04-25T10:15:54.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.386+0000 7f6fe3e00640 1 --2- 172.21.15.119:0/3054817448 >> v2:172.21.15.132:3300/0 conn(0x7f6fe4144e70 0x7f6fe406d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:54.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.387+0000 7f6fe8a00640 1 --2- 172.21.15.119:0/3054817448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe40cf0e0 0x7f6fe40cc300 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40b4a20 tx=0x7f6fdc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c564ffd5c2d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.388+0000 7f6fe2a00640 1 -- 172.21.15.119:0/3054817448 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc036070 con 0x7f6fe40cf0e0 2024-04-25T10:15:54.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.388+0000 7f6fe2a00640 1 -- 172.21.15.119:0/3054817448 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc031600 con 0x7f6fe40cf0e0 2024-04-25T10:15:54.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.388+0000 7f6fe2a00640 1 -- 172.21.15.119:0/3054817448 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc0319d0 con 0x7f6fe40cf0e0 2024-04-25T10:15:54.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.388+0000 7f6feaa00640 1 -- 172.21.15.119:0/3054817448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe40cf0e0 msgr2=0x7f6fe40cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.388+0000 7f6feaa00640 1 --2- 172.21.15.119:0/3054817448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe40cf0e0 0x7f6fe40cc300 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40b4a20 tx=0x7f6fdc02a660 comp rx=0 tx=0).stop 2024-04-25T10:15:54.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.388+0000 7f6feaa00640 1 -- 172.21.15.119:0/3054817448 shutdown_connections 2024-04-25T10:15:54.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.388+0000 7f6feaa00640 1 --2- 172.21.15.119:0/3054817448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe40cf0e0 0x7f6fe40cc300 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.388+0000 7f6feaa00640 1 --2- 172.21.15.119:0/3054817448 >> v2:172.21.15.132:3300/0 conn(0x7f6fe4144e70 0x7f6fe406d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.388+0000 7f6feaa00640 1 -- 172.21.15.119:0/3054817448 >> 172.21.15.119:0/3054817448 conn(0x7f6fe406aba0 msgr2=0x7f6fe406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:54.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.388+0000 7f6feaa00640 1 -- 172.21.15.119:0/3054817448 shutdown_connections 2024-04-25T10:15:54.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.388+0000 7f6feaa00640 1 -- 172.21.15.119:0/3054817448 wait complete. 2024-04-25T10:15:54.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.390+0000 7f6feaa00640 1 Processor -- start 2024-04-25T10:15:54.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.390+0000 7f6feaa00640 1 -- start start 2024-04-25T10:15:54.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.391+0000 7f6feaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe406d7a0 0x7f6fe415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.391+0000 7f6feaa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6fe40cf0e0 0x7f6fe413e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.391+0000 7f6feaa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fe4144e70 0x7f6fe415e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.391+0000 7f6feaa00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe4145630 con 0x7f6fe406d7a0 2024-04-25T10:15:54.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.391+0000 7f6feaa00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe41454b0 con 0x7f6fe4144e70 2024-04-25T10:15:54.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.391+0000 7f6feaa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40cc840 con 0x7f6fe40cf0e0 2024-04-25T10:15:54.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.391+0000 7f6fe3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe406d7a0 0x7f6fe415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.391+0000 7f6fe3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe406d7a0 0x7f6fe415d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46264/0 (socket says 172.21.15.119:46264) 2024-04-25T10:15:54.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.391+0000 7f6fe3e00640 1 -- 172.21.15.119:0/1618798746 learned_addr learned my addr 172.21.15.119:0/1618798746 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:54.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.392+0000 7f6fe3e00640 1 -- 172.21.15.119:0/1618798746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6fe40cf0e0 msgr2=0x7f6fe413e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.392+0000 7f6fe3e00640 1 --2- 172.21.15.119:0/1618798746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6fe40cf0e0 0x7f6fe413e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.392+0000 7f6fe3e00640 1 -- 172.21.15.119:0/1618798746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fe4144e70 msgr2=0x7f6fe415e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:54.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.392+0000 7f6fe3e00640 1 --2- 172.21.15.119:0/1618798746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fe4144e70 0x7f6fe415e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:54.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.392+0000 7f6fe3e00640 1 -- 172.21.15.119:0/1618798746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe415c620 con 0x7f6fe406d7a0 2024-04-25T10:15:54.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.392+0000 7f6fe3e00640 1 --2- 172.21.15.119:0/1618798746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe406d7a0 0x7f6fe415d6c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000c820 tx=0x7f6fd000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.392+0000 7f6fdbe00640 1 -- 172.21.15.119:0/1618798746 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0015070 con 0x7f6fe406d7a0 2024-04-25T10:15:54.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.392+0000 7f6fdbe00640 1 -- 172.21.15.119:0/1618798746 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd0007a30 con 0x7f6fe406d7a0 2024-04-25T10:15:54.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.392+0000 7f6fdbe00640 1 -- 172.21.15.119:0/1618798746 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0007de0 con 0x7f6fe406d7a0 2024-04-25T10:15:54.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.392+0000 7f6feaa00640 1 -- 172.21.15.119:0/1618798746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe415c900 con 0x7f6fe406d7a0 2024-04-25T10:15:54.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.393+0000 7f6feaa00640 1 -- 172.21.15.119:0/1618798746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe415cec0 con 0x7f6fe406d7a0 2024-04-25T10:15:54.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.394+0000 7f6feaa00640 1 -- 172.21.15.119:0/1618798746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe40750b0 con 0x7f6fe406d7a0 2024-04-25T10:15:54.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.395+0000 7f6fdbe00640 1 -- 172.21.15.119:0/1618798746 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fd0018070 con 0x7f6fe406d7a0 2024-04-25T10:15:54.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.396+0000 7f6fdbe00640 1 --2- 172.21.15.119:0/1618798746 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6fc406b4d0 0x7f6fc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:54.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.396+0000 7f6fdbe00640 1 -- 172.21.15.119:0/1618798746 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fd008a120 con 0x7f6fe406d7a0 2024-04-25T10:15:54.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.396+0000 7f6fe3400640 1 --2- 172.21.15.119:0/1618798746 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6fc406b4d0 0x7f6fc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:54.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.397+0000 7f6fe3400640 1 --2- 172.21.15.119:0/1618798746 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6fc406b4d0 0x7f6fc406d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40ae3a0 tx=0x7f6fd4007990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:54.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.400+0000 7f6fdbe00640 1 -- 172.21.15.119:0/1618798746 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6fd0058f90 con 0x7f6fe406d7a0 2024-04-25T10:15:54.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:54.631+0000 7f6feaa00640 1 -- 172.21.15.119:0/1618798746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6fe406c220 con 0x7f6fe406d7a0 2024-04-25T10:15:55.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.160+0000 7f6fdbe00640 1 -- 172.21.15.119:0/1618798746 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6fd0058890 con 0x7f6fe406d7a0 2024-04-25T10:15:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.164+0000 7f6feaa00640 1 -- 172.21.15.119:0/1618798746 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6fc406b4d0 msgr2=0x7f6fc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.164+0000 7f6feaa00640 1 --2- 172.21.15.119:0/1618798746 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6fc406b4d0 0x7f6fc406d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40ae3a0 tx=0x7f6fd4007990 comp rx=0 tx=0).stop 2024-04-25T10:15:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.164+0000 7f6feaa00640 1 -- 172.21.15.119:0/1618798746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe406d7a0 msgr2=0x7f6fe415d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.164+0000 7f6feaa00640 1 --2- 172.21.15.119:0/1618798746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe406d7a0 0x7f6fe415d6c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000c820 tx=0x7f6fd000cce0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.164+0000 7f6feaa00640 1 -- 172.21.15.119:0/1618798746 shutdown_connections 2024-04-25T10:15:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.164+0000 7f6feaa00640 1 --2- 172.21.15.119:0/1618798746 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6fc406b4d0 0x7f6fc406d990 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.164+0000 7f6feaa00640 1 --2- 172.21.15.119:0/1618798746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fe4144e70 0x7f6fe415e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.164+0000 7f6feaa00640 1 --2- 172.21.15.119:0/1618798746 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6fe40cf0e0 0x7f6fe413e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.164+0000 7f6feaa00640 1 --2- 172.21.15.119:0/1618798746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fe406d7a0 0x7f6fe415d6c0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.164+0000 7f6feaa00640 1 -- 172.21.15.119:0/1618798746 >> 172.21.15.119:0/1618798746 conn(0x7f6fe406aba0 msgr2=0x7f6fe4143470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.164+0000 7f6feaa00640 1 -- 172.21.15.119:0/1618798746 shutdown_connections 2024-04-25T10:15:55.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.164+0000 7f6feaa00640 1 -- 172.21.15.119:0/1618798746 wait complete. 2024-04-25T10:15:55.188 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:15:55.188 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:55.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.384+0000 7f49f3e00640 1 Processor -- start 2024-04-25T10:15:55.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.384+0000 7f49f3e00640 1 -- start start 2024-04-25T10:15:55.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.386+0000 7f49f3e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f49f4144470 0x7f49f415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.386+0000 7f49f3e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f49f40b5050 0x7f49f415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.386+0000 7f49f3e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f40cd440 con 0x7f49f406e430 2024-04-25T10:15:55.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.386+0000 7f49f3e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f4145340 con 0x7f49f40b5050 2024-04-25T10:15:55.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.386+0000 7f49f3e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f40cd720 con 0x7f49f4144470 2024-04-25T10:15:55.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.386+0000 7f49f3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f49f40b5050 0x7f49f415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.386+0000 7f49f2000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f49f406e430 0x7f49f40b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57386/0 (socket says 172.21.15.119:57386) 2024-04-25T10:15:55.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.386+0000 7f49f2000640 1 -- 172.21.15.119:0/3869473046 learned_addr learned my addr 172.21.15.119:0/3869473046 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.387+0000 7f49f2a00640 1 --2- 172.21.15.119:0/3869473046 >> v2:172.21.15.132:3300/0 conn(0x7f49f4144470 0x7f49f415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.387+0000 7f49f3400640 1 -- 172.21.15.119:0/3869473046 >> v2:172.21.15.132:3300/0 conn(0x7f49f4144470 msgr2=0x7f49f415fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.387+0000 7f49f3400640 1 --2- 172.21.15.119:0/3869473046 >> v2:172.21.15.132:3300/0 conn(0x7f49f4144470 0x7f49f415fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.387+0000 7f49f3400640 1 -- 172.21.15.119:0/3869473046 >> v1:172.21.15.119:6789/0 conn(0x7f49f406e430 legacy=0x7f49f40b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.387+0000 7f49f3400640 1 -- 172.21.15.119:0/3869473046 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f40cda00 con 0x7f49f40b5050 2024-04-25T10:15:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.387+0000 7f49f1600640 1 -- 172.21.15.119:0/3869473046 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795238450 0 0) 0x7f49f40cd440 con 0x7f49f406e430 2024-04-25T10:15:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.388+0000 7f49f3400640 1 --2- 172.21.15.119:0/3869473046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f40b5050 0x7f49f415bcf0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f49f40b5490 tx=0x7f49ec02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468e6569c30f9bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.388+0000 7f49f1600640 1 -- 172.21.15.119:0/3869473046 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec007740 con 0x7f49f40b5050 2024-04-25T10:15:55.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.388+0000 7f49f1600640 1 -- 172.21.15.119:0/3869473046 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec0078e0 con 0x7f49f40b5050 2024-04-25T10:15:55.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.388+0000 7f49f1600640 1 -- 172.21.15.119:0/3869473046 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec007cb0 con 0x7f49f40b5050 2024-04-25T10:15:55.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.389+0000 7f49f3e00640 1 -- 172.21.15.119:0/3869473046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f40b5050 msgr2=0x7f49f415bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.389+0000 7f49f3e00640 1 --2- 172.21.15.119:0/3869473046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f40b5050 0x7f49f415bcf0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f49f40b5490 tx=0x7f49ec02a680 comp rx=0 tx=0).stop 2024-04-25T10:15:55.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.389+0000 7f49f3e00640 1 -- 172.21.15.119:0/3869473046 shutdown_connections 2024-04-25T10:15:55.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.389+0000 7f49f3e00640 1 --2- 172.21.15.119:0/3869473046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f40b5050 0x7f49f415bcf0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.389+0000 7f49f3e00640 1 --2- 172.21.15.119:0/3869473046 >> v2:172.21.15.132:3300/0 conn(0x7f49f4144470 0x7f49f415fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.389+0000 7f49f3e00640 1 -- 172.21.15.119:0/3869473046 >> 172.21.15.119:0/3869473046 conn(0x7f49f406b1d0 msgr2=0x7f49f40b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.389+0000 7f49f3e00640 1 -- 172.21.15.119:0/3869473046 shutdown_connections 2024-04-25T10:15:55.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.389+0000 7f49f3e00640 1 -- 172.21.15.119:0/3869473046 wait complete. 2024-04-25T10:15:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.391+0000 7f49f3e00640 1 Processor -- start 2024-04-25T10:15:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.391+0000 7f49f3e00640 1 -- start start 2024-04-25T10:15:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.392+0000 7f49f3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f406e430 0x7f49f413a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.392+0000 7f49f3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f40b5050 0x7f49f415e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.392+0000 7f49f3e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49f4144470 0x7f49f415e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.392+0000 7f49f3e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f40ce200 con 0x7f49f406e430 2024-04-25T10:15:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.392+0000 7f49f3e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f4144850 con 0x7f49f4144470 2024-04-25T10:15:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.392+0000 7f49f3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f415c230 con 0x7f49f40b5050 2024-04-25T10:15:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.392+0000 7f49f2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f406e430 0x7f49f413a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.393+0000 7f49f2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f406e430 0x7f49f413a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46286/0 (socket says 172.21.15.119:46286) 2024-04-25T10:15:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.393+0000 7f49f2a00640 1 -- 172.21.15.119:0/2440864253 learned_addr learned my addr 172.21.15.119:0/2440864253 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.393+0000 7f49f3400640 1 --2- 172.21.15.119:0/2440864253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49f4144470 0x7f49f415e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.393+0000 7f49f2000640 1 --2- 172.21.15.119:0/2440864253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f40b5050 0x7f49f415e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.393+0000 7f49f2a00640 1 -- 172.21.15.119:0/2440864253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f40b5050 msgr2=0x7f49f415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.393+0000 7f49f2a00640 1 --2- 172.21.15.119:0/2440864253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f40b5050 0x7f49f415e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.393+0000 7f49f2a00640 1 -- 172.21.15.119:0/2440864253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49f4144470 msgr2=0x7f49f415e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.393+0000 7f49f2a00640 1 --2- 172.21.15.119:0/2440864253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49f4144470 0x7f49f415e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.393+0000 7f49f2a00640 1 -- 172.21.15.119:0/2440864253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f40ad730 con 0x7f49f406e430 2024-04-25T10:15:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.393+0000 7f49f2a00640 1 --2- 172.21.15.119:0/2440864253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f406e430 0x7f49f413a2d0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f49e400c7f0 tx=0x7f49e400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.394+0000 7f49eaa00640 1 -- 172.21.15.119:0/2440864253 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e4015070 con 0x7f49f406e430 2024-04-25T10:15:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.394+0000 7f49eaa00640 1 -- 172.21.15.119:0/2440864253 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e4004220 con 0x7f49f406e430 2024-04-25T10:15:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.394+0000 7f49eaa00640 1 -- 172.21.15.119:0/2440864253 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e4007620 con 0x7f49f406e430 2024-04-25T10:15:55.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.394+0000 7f49f3e00640 1 -- 172.21.15.119:0/2440864253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f40ada10 con 0x7f49f406e430 2024-04-25T10:15:55.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.394+0000 7f49f3e00640 1 -- 172.21.15.119:0/2440864253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f40adfd0 con 0x7f49f406e430 2024-04-25T10:15:55.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.395+0000 7f49f3e00640 1 -- 172.21.15.119:0/2440864253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f40749b0 con 0x7f49f406e430 2024-04-25T10:15:55.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.396+0000 7f49eaa00640 1 -- 172.21.15.119:0/2440864253 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49e40079d0 con 0x7f49f406e430 2024-04-25T10:15:55.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.397+0000 7f49eaa00640 1 --2- 172.21.15.119:0/2440864253 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f49d406b4d0 0x7f49d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.397+0000 7f49eaa00640 1 -- 172.21.15.119:0/2440864253 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49e408a5c0 con 0x7f49f406e430 2024-04-25T10:15:55.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.397+0000 7f49f2000640 1 --2- 172.21.15.119:0/2440864253 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f49d406b4d0 0x7f49d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.398+0000 7f49f2000640 1 --2- 172.21.15.119:0/2440864253 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f49d406b4d0 0x7f49d406d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f49f40b2a20 tx=0x7f49e0007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.402+0000 7f49eaa00640 1 -- 172.21.15.119:0/2440864253 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49e4059440 con 0x7f49f406e430 2024-04-25T10:15:55.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.627+0000 7f49f3e00640 1 -- 172.21.15.119:0/2440864253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49f40b2090 con 0x7f49f406e430 2024-04-25T10:15:55.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.629+0000 7f49eaa00640 1 -- 172.21.15.119:0/2440864253 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f49e4058d40 con 0x7f49f406e430 2024-04-25T10:15:55.633 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:55.633 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5459,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3768348114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3768348114},{"type":"v1","addr":"172.21.15.132:6834","nonce":3768348114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":5465,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/543190780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":543190780},{"type":"v1","addr":"172.21.15.132:6835","nonce":543190780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1145229172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1145229172},{"type":"v1","addr":"172.21.15.119:6836","nonce":1145229172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:51.568312+0000","modified":"2024-04-25T10:15:54.633087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1945631445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1945631445},{"type":"v1","addr":"172.21.15.119:6837","nonce":1945631445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:15:55.633 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 82 2024-04-25T10:15:55.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.631+0000 7f49f3e00640 1 -- 172.21.15.119:0/2440864253 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f49d406b4d0 msgr2=0x7f49d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.631+0000 7f49f3e00640 1 --2- 172.21.15.119:0/2440864253 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f49d406b4d0 0x7f49d406d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f49f40b2a20 tx=0x7f49e0007b50 comp rx=0 tx=0).stop 2024-04-25T10:15:55.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.631+0000 7f49f3e00640 1 -- 172.21.15.119:0/2440864253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f406e430 msgr2=0x7f49f413a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.631+0000 7f49f3e00640 1 --2- 172.21.15.119:0/2440864253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f406e430 0x7f49f413a2d0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f49e400c7f0 tx=0x7f49e400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.631+0000 7f49f3e00640 1 -- 172.21.15.119:0/2440864253 shutdown_connections 2024-04-25T10:15:55.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.631+0000 7f49f3e00640 1 --2- 172.21.15.119:0/2440864253 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f49d406b4d0 0x7f49d406d990 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.631+0000 7f49f3e00640 1 --2- 172.21.15.119:0/2440864253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49f4144470 0x7f49f415e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.631+0000 7f49f3e00640 1 --2- 172.21.15.119:0/2440864253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f40b5050 0x7f49f415e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.631+0000 7f49f3e00640 1 --2- 172.21.15.119:0/2440864253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49f406e430 0x7f49f413a2d0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.631+0000 7f49f3e00640 1 -- 172.21.15.119:0/2440864253 >> 172.21.15.119:0/2440864253 conn(0x7f49f406b1d0 msgr2=0x7f49f40ae8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.632+0000 7f49f3e00640 1 -- 172.21.15.119:0/2440864253 shutdown_connections 2024-04-25T10:15:55.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.632+0000 7f49f3e00640 1 -- 172.21.15.119:0/2440864253 wait complete. 2024-04-25T10:15:55.654 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:55.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.844+0000 7f9223e00640 1 Processor -- start 2024-04-25T10:15:55.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.845+0000 7f9223e00640 1 -- start start 2024-04-25T10:15:55.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.846+0000 7f9223e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f92240ce560 0x7f92240ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.846+0000 7f9223e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f922415dc80 0x7f92240b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.846+0000 7f9223e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f92240d3340 0x7f92240b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.846+0000 7f9223e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92240d0e30 con 0x7f92240ce560 2024-04-25T10:15:55.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.846+0000 7f9223e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92240d2b50 con 0x7f92240d3340 2024-04-25T10:15:55.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.846+0000 7f9223e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92240cee80 con 0x7f922415dc80 2024-04-25T10:15:55.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.846+0000 7f9222a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f92240ce560 0x7f92240ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.846+0000 7f9222a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f92240ce560 0x7f92240ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46292/0 (socket says 172.21.15.119:46292) 2024-04-25T10:15:55.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.846+0000 7f9222a00640 1 -- 172.21.15.119:0/1977783006 learned_addr learned my addr 172.21.15.119:0/1977783006 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:55.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.847+0000 7f9222a00640 1 -- 172.21.15.119:0/1977783006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f922415dc80 msgr2=0x7f92240b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.847+0000 7f9222a00640 1 --2- 172.21.15.119:0/1977783006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f922415dc80 0x7f92240b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.847+0000 7f9222a00640 1 -- 172.21.15.119:0/1977783006 >> v2:172.21.15.132:3300/0 conn(0x7f92240d3340 msgr2=0x7f92240b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.847+0000 7f9222a00640 1 --2- 172.21.15.119:0/1977783006 >> v2:172.21.15.132:3300/0 conn(0x7f92240d3340 0x7f92240b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.847+0000 7f9222a00640 1 -- 172.21.15.119:0/1977783006 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92240d1c80 con 0x7f92240ce560 2024-04-25T10:15:55.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.848+0000 7f9222a00640 1 --2- 172.21.15.119:0/1977783006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92240ce560 0x7f92240ce940 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f92240b6420 tx=0x7f921402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74df66ac1a47316 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.848+0000 7f9221600640 1 -- 172.21.15.119:0/1977783006 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214037070 con 0x7f92240ce560 2024-04-25T10:15:55.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.848+0000 7f9221600640 1 -- 172.21.15.119:0/1977783006 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9214032440 con 0x7f92240ce560 2024-04-25T10:15:55.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.848+0000 7f9221600640 1 -- 172.21.15.119:0/1977783006 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214032810 con 0x7f92240ce560 2024-04-25T10:15:55.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.848+0000 7f9223e00640 1 -- 172.21.15.119:0/1977783006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92240ce560 msgr2=0x7f92240ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.848+0000 7f9223e00640 1 --2- 172.21.15.119:0/1977783006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92240ce560 0x7f92240ce940 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f92240b6420 tx=0x7f921402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.849+0000 7f9223e00640 1 -- 172.21.15.119:0/1977783006 shutdown_connections 2024-04-25T10:15:55.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.849+0000 7f9223e00640 1 --2- 172.21.15.119:0/1977783006 >> v2:172.21.15.132:3300/0 conn(0x7f92240d3340 0x7f92240b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.849+0000 7f9223e00640 1 --2- 172.21.15.119:0/1977783006 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f922415dc80 0x7f92240b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.849+0000 7f9223e00640 1 --2- 172.21.15.119:0/1977783006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92240ce560 0x7f92240ce940 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.849+0000 7f9223e00640 1 -- 172.21.15.119:0/1977783006 >> 172.21.15.119:0/1977783006 conn(0x7f922406bde0 msgr2=0x7f922406c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.849+0000 7f9223e00640 1 -- 172.21.15.119:0/1977783006 shutdown_connections 2024-04-25T10:15:55.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.849+0000 7f9223e00640 1 -- 172.21.15.119:0/1977783006 wait complete. 2024-04-25T10:15:55.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.850+0000 7f9223e00640 1 Processor -- start 2024-04-25T10:15:55.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.851+0000 7f9223e00640 1 -- start start 2024-04-25T10:15:55.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.851+0000 7f9223e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240ce560 0x7f92240b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.851+0000 7f9223e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92240d3340 0x7f92240b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.851+0000 7f9223e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922415dc80 0x7f92240ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.851+0000 7f9223e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92240d2500 con 0x7f922415dc80 2024-04-25T10:15:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.851+0000 7f9223e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92240d2220 con 0x7f92240d3340 2024-04-25T10:15:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.851+0000 7f9223e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92240d27e0 con 0x7f92240ce560 2024-04-25T10:15:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.852+0000 7f9223400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922415dc80 0x7f92240ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.852+0000 7f9223400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922415dc80 0x7f92240ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46308/0 (socket says 172.21.15.119:46308) 2024-04-25T10:15:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.852+0000 7f9223400640 1 -- 172.21.15.119:0/2510049309 learned_addr learned my addr 172.21.15.119:0/2510049309 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.852+0000 7f9222000640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92240d3340 0x7f92240b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.852+0000 7f9222a00640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240ce560 0x7f92240b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.852+0000 7f9223400640 1 -- 172.21.15.119:0/2510049309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240ce560 msgr2=0x7f92240b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.852+0000 7f9223400640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240ce560 0x7f92240b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.852+0000 7f9223400640 1 -- 172.21.15.119:0/2510049309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92240d3340 msgr2=0x7f92240b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.852+0000 7f9223400640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92240d3340 0x7f92240b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.852+0000 7f9223400640 1 -- 172.21.15.119:0/2510049309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92240c8500 con 0x7f922415dc80 2024-04-25T10:15:55.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.852+0000 7f9222a00640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240ce560 0x7f92240b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:55.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.852+0000 7f9223400640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922415dc80 0x7f92240ca9f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f921c00b9e0 tx=0x7f921c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.853+0000 7f921aa00640 1 -- 172.21.15.119:0/2510049309 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c019070 con 0x7f922415dc80 2024-04-25T10:15:55.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.853+0000 7f921aa00640 1 -- 172.21.15.119:0/2510049309 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c00cb20 con 0x7f922415dc80 2024-04-25T10:15:55.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.853+0000 7f921aa00640 1 -- 172.21.15.119:0/2510049309 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c014440 con 0x7f922415dc80 2024-04-25T10:15:55.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.853+0000 7f9223e00640 1 -- 172.21.15.119:0/2510049309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92240c8770 con 0x7f922415dc80 2024-04-25T10:15:55.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.853+0000 7f9223e00640 1 -- 172.21.15.119:0/2510049309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92240c8d30 con 0x7f922415dc80 2024-04-25T10:15:55.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.854+0000 7f9223e00640 1 -- 172.21.15.119:0/2510049309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92240749b0 con 0x7f922415dc80 2024-04-25T10:15:55.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.855+0000 7f921aa00640 1 -- 172.21.15.119:0/2510049309 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f921c01c070 con 0x7f922415dc80 2024-04-25T10:15:55.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.856+0000 7f921aa00640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f920406b4d0 0x7f920406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.856+0000 7f921aa00640 1 -- 172.21.15.119:0/2510049309 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f921c089f40 con 0x7f922415dc80 2024-04-25T10:15:55.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.856+0000 7f9222a00640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f920406b4d0 0x7f920406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.857+0000 7f9222a00640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f920406b4d0 0x7f920406d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9214034000 tx=0x7f921402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:55.860+0000 7f921aa00640 1 -- 172.21.15.119:0/2510049309 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f921c058db0 con 0x7f922415dc80 2024-04-25T10:15:56.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.057+0000 7f9223e00640 1 -- 172.21.15.119:0/2510049309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f922406ae10 con 0x7f922415dc80 2024-04-25T10:15:56.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.059+0000 7f921aa00640 1 -- 172.21.15.119:0/2510049309 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16644 (secure 0 0 0) 0x7f921c0586b0 con 0x7f922415dc80 2024-04-25T10:15:56.064 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:56.064 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":96,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:15:53.131988+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:49.348469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:50.584869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:56.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.062+0000 7f9223e00640 1 -- 172.21.15.119:0/2510049309 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f920406b4d0 msgr2=0x7f920406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.062+0000 7f9223e00640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f920406b4d0 0x7f920406d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9214034000 tx=0x7f921402e000 comp rx=0 tx=0).stop 2024-04-25T10:15:56.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.062+0000 7f9223e00640 1 -- 172.21.15.119:0/2510049309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922415dc80 msgr2=0x7f92240ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.062+0000 7f9223e00640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922415dc80 0x7f92240ca9f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f921c00b9e0 tx=0x7f921c00bea0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.062+0000 7f9223e00640 1 -- 172.21.15.119:0/2510049309 shutdown_connections 2024-04-25T10:15:56.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.062+0000 7f9223e00640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f920406b4d0 0x7f920406d990 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.062+0000 7f9223e00640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f922415dc80 0x7f92240ca9f0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.062+0000 7f9223e00640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92240d3340 0x7f92240b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.062+0000 7f9223e00640 1 --2- 172.21.15.119:0/2510049309 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92240ce560 0x7f92240b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.062+0000 7f9223e00640 1 -- 172.21.15.119:0/2510049309 >> 172.21.15.119:0/2510049309 conn(0x7f922406bde0 msgr2=0x7f922415ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:56.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.062+0000 7f9223e00640 1 -- 172.21.15.119:0/2510049309 shutdown_connections 2024-04-25T10:15:56.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.062+0000 7f9223e00640 1 -- 172.21.15.119:0/2510049309 wait complete. 2024-04-25T10:15:56.087 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:15:56.087 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:56.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.284+0000 7fe0c8a00640 1 Processor -- start 2024-04-25T10:15:56.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.284+0000 7fe0c8a00640 1 -- start start 2024-04-25T10:15:56.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.285+0000 7fe0c8a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0c415acd0 0x7fe0c406e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.285+0000 7fe0c8a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c4144420 con 0x7fe0c4144040 2024-04-25T10:15:56.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.285+0000 7fe0c8a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c40b3d80 con 0x7fe0c406e860 2024-04-25T10:15:56.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.285+0000 7fe0c8a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c406ec90 con 0x7fe0c415acd0 2024-04-25T10:15:56.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.285+0000 7fe0c3400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe0c4144040 0x7fe0c40cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57400/0 (socket says 172.21.15.119:57400) 2024-04-25T10:15:56.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.286+0000 7fe0c3400640 1 -- 172.21.15.119:0/2542324125 learned_addr learned my addr 172.21.15.119:0/2542324125 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.286+0000 7fe0c2000640 1 -- 172.21.15.119:0/2542324125 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440773548 0 0) 0x7fe0c4144420 con 0x7fe0c4144040 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.286+0000 7fe0c2000640 1 -- 172.21.15.119:0/2542324125 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a8003520 con 0x7fe0c4144040 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.286+0000 7fe0c2a00640 1 --2- 172.21.15.119:0/2542324125 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0c415acd0 0x7fe0c406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.286+0000 7fe0c2000640 1 -- 172.21.15.119:0/2542324125 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 96057498 0 0) 0x7fe0a8003520 con 0x7fe0c4144040 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.286+0000 7fe0c2000640 1 -- 172.21.15.119:0/2542324125 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0c415acd0 msgr2=0x7fe0c406e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.286+0000 7fe0c2000640 1 --2- 172.21.15.119:0/2542324125 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0c415acd0 0x7fe0c406e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.286+0000 7fe0c2000640 1 -- 172.21.15.119:0/2542324125 >> v1:172.21.15.132:6789/0 conn(0x7fe0c406e860 legacy=0x7fe0c40b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.286+0000 7fe0c2000640 1 -- 172.21.15.119:0/2542324125 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c40b4060 con 0x7fe0c4144040 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.286+0000 7fe0c2000640 1 -- 172.21.15.119:0/2542324125 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fe0b4001670 con 0x7fe0c4144040 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.286+0000 7fe0c2000640 1 -- 172.21.15.119:0/2542324125 >> v1:172.21.15.119:6789/0 conn(0x7fe0c4144040 legacy=0x7fe0c40cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.286+0000 7fe0c2000640 1 --2- 172.21.15.119:0/2542324125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0a8003a50 0x7fe0a8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.286+0000 7fe0c2000640 1 --2- 172.21.15.119:0/2542324125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a8004800 0x7fe0a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.286+0000 7fe0c2000640 1 --2- 172.21.15.119:0/2542324125 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0c415acd0 0x7fe0a800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.287+0000 7fe0c2000640 1 -- 172.21.15.119:0/2542324125 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c406ec90 con 0x7fe0a8004800 2024-04-25T10:15:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.287+0000 7fe0c2000640 1 -- 172.21.15.119:0/2542324125 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a80055a0 con 0x7fe0a8003a50 2024-04-25T10:15:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.287+0000 7fe0c2000640 1 -- 172.21.15.119:0/2542324125 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a800e110 con 0x7fe0c415acd0 2024-04-25T10:15:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.287+0000 7fe0c2a00640 1 --2- 172.21.15.119:0/2542324125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a8004800 0x7fe0a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.287+0000 7fe0c3e00640 1 --2- 172.21.15.119:0/2542324125 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0c415acd0 0x7fe0a800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.287+0000 7fe0c2a00640 1 -- 172.21.15.119:0/2542324125 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0c415acd0 msgr2=0x7fe0a800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.287+0000 7fe0c3400640 1 --2- 172.21.15.119:0/2542324125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0a8003a50 0x7fe0a8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.287+0000 7fe0c2a00640 1 --2- 172.21.15.119:0/2542324125 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0c415acd0 0x7fe0a800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.287+0000 7fe0c2a00640 1 -- 172.21.15.119:0/2542324125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0a8003a50 msgr2=0x7fe0a8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.287+0000 7fe0c2a00640 1 --2- 172.21.15.119:0/2542324125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0a8003a50 0x7fe0a8003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.287+0000 7fe0c2a00640 1 -- 172.21.15.119:0/2542324125 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a8003520 con 0x7fe0a8004800 2024-04-25T10:15:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.287+0000 7fe0c3e00640 1 --2- 172.21.15.119:0/2542324125 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0c415acd0 0x7fe0a800dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.288+0000 7fe0c2a00640 1 --2- 172.21.15.119:0/2542324125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a8004800 0x7fe0a8004c80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00ba30 tx=0x7fe0a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135ac1628931a37a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.288+0000 7fe0c2000640 1 -- 172.21.15.119:0/2542324125 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac030070 con 0x7fe0a8004800 2024-04-25T10:15:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.289+0000 7fe0c2000640 1 -- 172.21.15.119:0/2542324125 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac02add0 con 0x7fe0a8004800 2024-04-25T10:15:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.289+0000 7fe0c2000640 1 -- 172.21.15.119:0/2542324125 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00b600 con 0x7fe0a8004800 2024-04-25T10:15:56.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.289+0000 7fe0c8a00640 1 -- 172.21.15.119:0/2542324125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a8004800 msgr2=0x7fe0a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.289+0000 7fe0c8a00640 1 --2- 172.21.15.119:0/2542324125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0a8004800 0x7fe0a8004c80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00ba30 tx=0x7fe0a8005830 comp rx=0 tx=0).stop 2024-04-25T10:15:56.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.289+0000 7fe0c3e00640 1 -- 172.21.15.119:0/2542324125 reap_dead start 2024-04-25T10:15:56.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.289+0000 7fe0c8a00640 1 -- 172.21.15.119:0/2542324125 shutdown_connections 2024-04-25T10:15:56.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.289+0000 7fe0c8a00640 1 -- 172.21.15.119:0/2542324125 >> 172.21.15.119:0/2542324125 conn(0x7fe0c406b1d0 msgr2=0x7fe0c40b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:56.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.289+0000 7fe0c8a00640 1 -- 172.21.15.119:0/2542324125 shutdown_connections 2024-04-25T10:15:56.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.289+0000 7fe0c8a00640 1 -- 172.21.15.119:0/2542324125 wait complete. 2024-04-25T10:15:56.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.291+0000 7fe0c8a00640 1 Processor -- start 2024-04-25T10:15:56.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.291+0000 7fe0c8a00640 1 -- start start 2024-04-25T10:15:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.292+0000 7fe0c8a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0a8004800 0x7fe0c415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.292+0000 7fe0c8a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41ef0e0 0x7fe0c40b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.292+0000 7fe0c8a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0c41ef560 0x7fe0c40b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.292+0000 7fe0c8a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40b4900 con 0x7fe0c41ef0e0 2024-04-25T10:15:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.292+0000 7fe0c8a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40b4620 con 0x7fe0c41ef560 2024-04-25T10:15:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.292+0000 7fe0c8a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40b5180 con 0x7fe0a8004800 2024-04-25T10:15:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.292+0000 7fe0c2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41ef0e0 0x7fe0c40b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.292+0000 7fe0c2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41ef0e0 0x7fe0c40b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46314/0 (socket says 172.21.15.119:46314) 2024-04-25T10:15:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.292+0000 7fe0c2a00640 1 -- 172.21.15.119:0/704406777 learned_addr learned my addr 172.21.15.119:0/704406777 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.293+0000 7fe0c3400640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0a8004800 0x7fe0c415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.293+0000 7fe0c3e00640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0c41ef560 0x7fe0c40b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.293+0000 7fe0c2a00640 1 -- 172.21.15.119:0/704406777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0a8004800 msgr2=0x7fe0c415a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.293+0000 7fe0c2a00640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0a8004800 0x7fe0c415a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.293+0000 7fe0c2a00640 1 -- 172.21.15.119:0/704406777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0c41ef560 msgr2=0x7fe0c40b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.293+0000 7fe0c2a00640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0c41ef560 0x7fe0c40b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.293+0000 7fe0c2a00640 1 -- 172.21.15.119:0/704406777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c41efb30 con 0x7fe0c41ef0e0 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.293+0000 7fe0c3e00640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0c41ef560 0x7fe0c40b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.293+0000 7fe0c2a00640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41ef0e0 0x7fe0c40b0320 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00bb20 tx=0x7fe0ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.293+0000 7fe0bb400640 1 -- 172.21.15.119:0/704406777 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac043030 con 0x7fe0c41ef0e0 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.293+0000 7fe0bb400640 1 -- 172.21.15.119:0/704406777 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac02abd0 con 0x7fe0c41ef0e0 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.293+0000 7fe0bb400640 1 -- 172.21.15.119:0/704406777 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00b600 con 0x7fe0c41ef0e0 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.294+0000 7fe0c8a00640 1 -- 172.21.15.119:0/704406777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c41efe10 con 0x7fe0c41ef0e0 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.294+0000 7fe0c8a00640 1 -- 172.21.15.119:0/704406777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c41395f0 con 0x7fe0c41ef0e0 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.295+0000 7fe0c8a00640 1 -- 172.21.15.119:0/704406777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c40749b0 con 0x7fe0c41ef0e0 2024-04-25T10:15:56.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.296+0000 7fe0bb400640 1 -- 172.21.15.119:0/704406777 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0ac046070 con 0x7fe0c41ef0e0 2024-04-25T10:15:56.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.297+0000 7fe0bb400640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe0a006b4d0 0x7fe0a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.297+0000 7fe0bb400640 1 -- 172.21.15.119:0/704406777 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0ac0aada0 con 0x7fe0c41ef0e0 2024-04-25T10:15:56.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.297+0000 7fe0c3400640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe0a006b4d0 0x7fe0a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.298+0000 7fe0c3400640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe0a006b4d0 0x7fe0a006d990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe0c413d1a0 tx=0x7fe0b4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.301+0000 7fe0bb400640 1 -- 172.21.15.119:0/704406777 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0ac03c3a0 con 0x7fe0c41ef0e0 2024-04-25T10:15:56.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.478+0000 7fe0c8a00640 1 -- 172.21.15.119:0/704406777 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe0c415a5a0 con 0x7fe0a006b4d0 2024-04-25T10:15:56.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.498+0000 7fe0bb400640 1 -- 172.21.15.119:0/704406777 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252086 (secure 0 0 0) 0x7fe0c415a5a0 con 0x7fe0a006b4d0 2024-04-25T10:15:56.505 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:56.507 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:15:56.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.503+0000 7fe0c8a00640 1 -- 172.21.15.119:0/704406777 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe0a006b4d0 msgr2=0x7fe0a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.503+0000 7fe0c8a00640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe0a006b4d0 0x7fe0a006d990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe0c413d1a0 tx=0x7fe0b4009040 comp rx=0 tx=0).stop 2024-04-25T10:15:56.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.503+0000 7fe0c8a00640 1 -- 172.21.15.119:0/704406777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41ef0e0 msgr2=0x7fe0c40b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.503+0000 7fe0c8a00640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41ef0e0 0x7fe0c40b0320 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00bb20 tx=0x7fe0ac004730 comp rx=0 tx=0).stop 2024-04-25T10:15:56.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.503+0000 7fe0c8a00640 1 -- 172.21.15.119:0/704406777 shutdown_connections 2024-04-25T10:15:56.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.503+0000 7fe0c8a00640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe0a006b4d0 0x7fe0a006d990 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.503+0000 7fe0c8a00640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe0c41ef560 0x7fe0c40b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.503+0000 7fe0c8a00640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c41ef0e0 0x7fe0c40b0320 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.503+0000 7fe0c8a00640 1 --2- 172.21.15.119:0/704406777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe0a8004800 0x7fe0c415a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.503+0000 7fe0c8a00640 1 -- 172.21.15.119:0/704406777 >> 172.21.15.119:0/704406777 conn(0x7fe0c406b1d0 msgr2=0x7fe0c40ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:56.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.504+0000 7fe0c8a00640 1 -- 172.21.15.119:0/704406777 shutdown_connections 2024-04-25T10:15:56.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.504+0000 7fe0c8a00640 1 -- 172.21.15.119:0/704406777 wait complete. 2024-04-25T10:15:56.526 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-04-25T10:15:56.102914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259984,"kb_used_data":10504,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714736,"statfs":{"total":773094113280,"available":772827889664,"internally_reserved":0,"allocated":10756096,"data_stored":5280492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002186"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269210+0000","last_change":"2024-04-25T10:15:52.154315+0000","last_active":"2024-04-25T10:15:53.269210+0000","last_peered":"2024-04-25T10:15:53.269210+0000","last_clean":"2024-04-25T10:15:53.269210+0000","last_became_active":"2024-04-25T10:15:52.154027+0000","last_became_peered":"2024-04-25T10:15:52.154027+0000","last_unstale":"2024-04-25T10:15:53.269210+0000","last_undegraded":"2024-04-25T10:15:53.269210+0000","last_fullsized":"2024-04-25T10:15:53.269210+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.802917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141267+0000","last_change":"2024-04-25T10:15:52.157426+0000","last_active":"2024-04-25T10:15:53.141267+0000","last_peered":"2024-04-25T10:15:53.141267+0000","last_clean":"2024-04-25T10:15:53.141267+0000","last_became_active":"2024-04-25T10:15:52.154661+0000","last_became_peered":"2024-04-25T10:15:52.154661+0000","last_unstale":"2024-04-25T10:15:53.141267+0000","last_undegraded":"2024-04-25T10:15:53.141267+0000","last_fullsized":"2024-04-25T10:15:53.141267+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:23.312018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269529+0000","last_change":"2024-04-25T10:15:52.155191+0000","last_active":"2024-04-25T10:15:53.269529+0000","last_peered":"2024-04-25T10:15:53.269529+0000","last_clean":"2024-04-25T10:15:53.269529+0000","last_became_active":"2024-04-25T10:15:52.154079+0000","last_became_peered":"2024-04-25T10:15:52.154079+0000","last_unstale":"2024-04-25T10:15:53.269529+0000","last_undegraded":"2024-04-25T10:15:53.269529+0000","last_fullsized":"2024-04-25T10:15:53.269529+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:27.710289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.271481+0000","last_change":"2024-04-25T10:15:52.135915+0000","last_active":"2024-04-25T10:15:53.271481+0000","last_peered":"2024-04-25T10:15:53.271481+0000","last_clean":"2024-04-25T10:15:53.271481+0000","last_became_active":"2024-04-25T10:15:52.135516+0000","last_became_peered":"2024-04-25T10:15:52.135516+0000","last_unstale":"2024-04-25T10:15:53.271481+0000","last_undegraded":"2024-04-25T10:15:53.271481+0000","last_fullsized":"2024-04-25T10:15:53.271481+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:25.250794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268424+0000","last_change":"2024-04-25T10:15:52.158208+0000","last_active":"2024-04-25T10:15:53.268424+0000","last_peered":"2024-04-25T10:15:53.268424+0000","last_clean":"2024-04-25T10:15:53.268424+0000","last_became_active":"2024-04-25T10:15:52.157763+0000","last_became_peered":"2024-04-25T10:15:52.157763+0000","last_unstale":"2024-04-25T10:15:53.268424+0000","last_undegraded":"2024-04-25T10:15:53.268424+0000","last_fullsized":"2024-04-25T10:15:53.268424+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.748468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267808+0000","last_change":"2024-04-25T10:15:52.156086+0000","last_active":"2024-04-25T10:15:53.267808+0000","last_peered":"2024-04-25T10:15:53.267808+0000","last_clean":"2024-04-25T10:15:53.267808+0000","last_became_active":"2024-04-25T10:15:52.155766+0000","last_became_peered":"2024-04-25T10:15:52.155766+0000","last_unstale":"2024-04-25T10:15:53.267808+0000","last_undegraded":"2024-04-25T10:15:53.267808+0000","last_fullsized":"2024-04-25T10:15:53.267808+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.740446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266705+0000","last_change":"2024-04-25T10:15:52.153556+0000","last_active":"2024-04-25T10:15:53.266705+0000","last_peered":"2024-04-25T10:15:53.266705+0000","last_clean":"2024-04-25T10:15:53.266705+0000","last_became_active":"2024-04-25T10:15:52.153231+0000","last_became_peered":"2024-04-25T10:15:52.153231+0000","last_unstale":"2024-04-25T10:15:53.266705+0000","last_undegraded":"2024-04-25T10:15:53.266705+0000","last_fullsized":"2024-04-25T10:15:53.266705+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:47.694850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140749+0000","last_change":"2024-04-25T10:15:52.151655+0000","last_active":"2024-04-25T10:15:53.140749+0000","last_peered":"2024-04-25T10:15:53.140749+0000","last_clean":"2024-04-25T10:15:53.140749+0000","last_became_active":"2024-04-25T10:15:52.151398+0000","last_became_peered":"2024-04-25T10:15:52.151398+0000","last_unstale":"2024-04-25T10:15:53.140749+0000","last_undegraded":"2024-04-25T10:15:53.140749+0000","last_fullsized":"2024-04-25T10:15:53.140749+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:05.831266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.144462+0000","last_change":"2024-04-25T10:15:52.138831+0000","last_active":"2024-04-25T10:15:53.144462+0000","last_peered":"2024-04-25T10:15:53.144462+0000","last_clean":"2024-04-25T10:15:53.144462+0000","last_became_active":"2024-04-25T10:15:52.138369+0000","last_became_peered":"2024-04-25T10:15:52.138369+0000","last_unstale":"2024-04-25T10:15:53.144462+0000","last_undegraded":"2024-04-25T10:15:53.144462+0000","last_fullsized":"2024-04-25T10:15:53.144462+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.740446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268347+0000","last_change":"2024-04-25T10:15:52.164913+0000","last_active":"2024-04-25T10:15:53.268347+0000","last_peered":"2024-04-25T10:15:53.268347+0000","last_clean":"2024-04-25T10:15:53.268347+0000","last_became_active":"2024-04-25T10:15:52.163399+0000","last_became_peered":"2024-04-25T10:15:52.163399+0000","last_unstale":"2024-04-25T10:15:53.268347+0000","last_undegraded":"2024-04-25T10:15:53.268347+0000","last_fullsized":"2024-04-25T10:15:53.268347+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:44.960622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268116+0000","last_change":"2024-04-25T10:15:52.159664+0000","last_active":"2024-04-25T10:15:53.268116+0000","last_peered":"2024-04-25T10:15:53.268116+0000","last_clean":"2024-04-25T10:15:53.268116+0000","last_became_active":"2024-04-25T10:15:52.159132+0000","last_became_peered":"2024-04-25T10:15:52.159132+0000","last_unstale":"2024-04-25T10:15:53.268116+0000","last_undegraded":"2024-04-25T10:15:53.268116+0000","last_fullsized":"2024-04-25T10:15:53.268116+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:13.450500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.144460+0000","last_change":"2024-04-25T10:15:52.161488+0000","last_active":"2024-04-25T10:15:53.144460+0000","last_peered":"2024-04-25T10:15:53.144460+0000","last_clean":"2024-04-25T10:15:53.144460+0000","last_became_active":"2024-04-25T10:15:52.161171+0000","last_became_peered":"2024-04-25T10:15:52.161171+0000","last_unstale":"2024-04-25T10:15:53.144460+0000","last_undegraded":"2024-04-25T10:15:53.144460+0000","last_fullsized":"2024-04-25T10:15:53.144460+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.748468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268687+0000","last_change":"2024-04-25T10:15:52.148522+0000","last_active":"2024-04-25T10:15:53.268687+0000","last_peered":"2024-04-25T10:15:53.268687+0000","last_clean":"2024-04-25T10:15:53.268687+0000","last_became_active":"2024-04-25T10:15:52.148251+0000","last_became_peered":"2024-04-25T10:15:52.148251+0000","last_unstale":"2024-04-25T10:15:53.268687+0000","last_undegraded":"2024-04-25T10:15:53.268687+0000","last_fullsized":"2024-04-25T10:15:53.268687+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:13.450500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265417+0000","last_change":"2024-04-25T10:15:52.158446+0000","last_active":"2024-04-25T10:15:53.265417+0000","last_peered":"2024-04-25T10:15:53.265417+0000","last_clean":"2024-04-25T10:15:53.265417+0000","last_became_active":"2024-04-25T10:15:52.158152+0000","last_became_peered":"2024-04-25T10:15:52.158152+0000","last_unstale":"2024-04-25T10:15:53.265417+0000","last_undegraded":"2024-04-25T10:15:53.265417+0000","last_fullsized":"2024-04-25T10:15:53.265417+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:44.426438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.271964+0000","last_change":"2024-04-25T10:15:52.153543+0000","last_active":"2024-04-25T10:15:53.271964+0000","last_peered":"2024-04-25T10:15:53.271964+0000","last_clean":"2024-04-25T10:15:53.271964+0000","last_became_active":"2024-04-25T10:15:52.152969+0000","last_became_peered":"2024-04-25T10:15:52.152969+0000","last_unstale":"2024-04-25T10:15:53.271964+0000","last_undegraded":"2024-04-25T10:15:53.271964+0000","last_fullsized":"2024-04-25T10:15:53.271964+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:21.257325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266474+0000","last_change":"2024-04-25T10:15:52.159324+0000","last_active":"2024-04-25T10:15:53.266474+0000","last_peered":"2024-04-25T10:15:53.266474+0000","last_clean":"2024-04-25T10:15:53.266474+0000","last_became_active":"2024-04-25T10:15:52.159032+0000","last_became_peered":"2024-04-25T10:15:52.159032+0000","last_unstale":"2024-04-25T10:15:53.266474+0000","last_undegraded":"2024-04-25T10:15:53.266474+0000","last_fullsized":"2024-04-25T10:15:53.266474+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:50.901390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141939+0000","last_change":"2024-04-25T10:15:52.149380+0000","last_active":"2024-04-25T10:15:53.141939+0000","last_peered":"2024-04-25T10:15:53.141939+0000","last_clean":"2024-04-25T10:15:53.141939+0000","last_became_active":"2024-04-25T10:15:52.148988+0000","last_became_peered":"2024-04-25T10:15:52.148988+0000","last_unstale":"2024-04-25T10:15:53.141939+0000","last_undegraded":"2024-04-25T10:15:53.141939+0000","last_fullsized":"2024-04-25T10:15:53.141939+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.853674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269343+0000","last_change":"2024-04-25T10:15:52.151357+0000","last_active":"2024-04-25T10:15:53.269343+0000","last_peered":"2024-04-25T10:15:53.269343+0000","last_clean":"2024-04-25T10:15:53.269343+0000","last_became_active":"2024-04-25T10:15:52.151080+0000","last_became_peered":"2024-04-25T10:15:52.151080+0000","last_unstale":"2024-04-25T10:15:53.269343+0000","last_undegraded":"2024-04-25T10:15:53.269343+0000","last_fullsized":"2024-04-25T10:15:53.269343+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:37.981100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266594+0000","last_change":"2024-04-25T10:15:52.147829+0000","last_active":"2024-04-25T10:15:53.266594+0000","last_peered":"2024-04-25T10:15:53.266594+0000","last_clean":"2024-04-25T10:15:53.266594+0000","last_became_active":"2024-04-25T10:15:52.147509+0000","last_became_peered":"2024-04-25T10:15:52.147509+0000","last_unstale":"2024-04-25T10:15:53.266594+0000","last_undegraded":"2024-04-25T10:15:53.266594+0000","last_fullsized":"2024-04-25T10:15:53.266594+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:03.554962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.144194+0000","last_change":"2024-04-25T10:15:52.141031+0000","last_active":"2024-04-25T10:15:53.144194+0000","last_peered":"2024-04-25T10:15:53.144194+0000","last_clean":"2024-04-25T10:15:53.144194+0000","last_became_active":"2024-04-25T10:15:52.140645+0000","last_became_peered":"2024-04-25T10:15:52.140645+0000","last_unstale":"2024-04-25T10:15:53.144194+0000","last_undegraded":"2024-04-25T10:15:53.144194+0000","last_fullsized":"2024-04-25T10:15:53.144194+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:31.640487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265726+0000","last_change":"2024-04-25T10:15:52.152135+0000","last_active":"2024-04-25T10:15:53.265726+0000","last_peered":"2024-04-25T10:15:53.265726+0000","last_clean":"2024-04-25T10:15:53.265726+0000","last_became_active":"2024-04-25T10:15:52.151680+0000","last_became_peered":"2024-04-25T10:15:52.151680+0000","last_unstale":"2024-04-25T10:15:53.265726+0000","last_undegraded":"2024-04-25T10:15:53.265726+0000","last_fullsized":"2024-04-25T10:15:53.265726+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:50.980785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140287+0000","last_change":"2024-04-25T10:15:52.159380+0000","last_active":"2024-04-25T10:15:53.140287+0000","last_peered":"2024-04-25T10:15:53.140287+0000","last_clean":"2024-04-25T10:15:53.140287+0000","last_became_active":"2024-04-25T10:15:52.159077+0000","last_became_peered":"2024-04-25T10:15:52.159077+0000","last_unstale":"2024-04-25T10:15:53.140287+0000","last_undegraded":"2024-04-25T10:15:53.140287+0000","last_fullsized":"2024-04-25T10:15:53.140287+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:00.650074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140579+0000","last_change":"2024-04-25T10:15:52.156440+0000","last_active":"2024-04-25T10:15:53.140579+0000","last_peered":"2024-04-25T10:15:53.140579+0000","last_clean":"2024-04-25T10:15:53.140579+0000","last_became_active":"2024-04-25T10:15:52.156176+0000","last_became_peered":"2024-04-25T10:15:52.156176+0000","last_unstale":"2024-04-25T10:15:53.140579+0000","last_undegraded":"2024-04-25T10:15:53.140579+0000","last_fullsized":"2024-04-25T10:15:53.140579+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:01.862561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266786+0000","last_change":"2024-04-25T10:15:52.160632+0000","last_active":"2024-04-25T10:15:53.266786+0000","last_peered":"2024-04-25T10:15:53.266786+0000","last_clean":"2024-04-25T10:15:53.266786+0000","last_became_active":"2024-04-25T10:15:52.160288+0000","last_became_peered":"2024-04-25T10:15:52.160288+0000","last_unstale":"2024-04-25T10:15:53.266786+0000","last_undegraded":"2024-04-25T10:15:53.266786+0000","last_fullsized":"2024-04-25T10:15:53.266786+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:44.960622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268419+0000","last_change":"2024-04-25T10:15:52.165387+0000","last_active":"2024-04-25T10:15:53.268419+0000","last_peered":"2024-04-25T10:15:53.268419+0000","last_clean":"2024-04-25T10:15:53.268419+0000","last_became_active":"2024-04-25T10:15:52.163584+0000","last_became_peered":"2024-04-25T10:15:52.163584+0000","last_unstale":"2024-04-25T10:15:53.268419+0000","last_undegraded":"2024-04-25T10:15:53.268419+0000","last_fullsized":"2024-04-25T10:15:53.268419+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.740446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141623+0000","last_change":"2024-04-25T10:15:52.146784+0000","last_active":"2024-04-25T10:15:53.141623+0000","last_peered":"2024-04-25T10:15:53.141623+0000","last_clean":"2024-04-25T10:15:53.141623+0000","last_became_active":"2024-04-25T10:15:52.146477+0000","last_became_peered":"2024-04-25T10:15:52.146477+0000","last_unstale":"2024-04-25T10:15:53.141623+0000","last_undegraded":"2024-04-25T10:15:53.141623+0000","last_fullsized":"2024-04-25T10:15:53.141623+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:25.764320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268049+0000","last_change":"2024-04-25T10:15:52.162183+0000","last_active":"2024-04-25T10:15:53.268049+0000","last_peered":"2024-04-25T10:15:53.268049+0000","last_clean":"2024-04-25T10:15:53.268049+0000","last_became_active":"2024-04-25T10:15:52.160494+0000","last_became_peered":"2024-04-25T10:15:52.160494+0000","last_unstale":"2024-04-25T10:15:53.268049+0000","last_undegraded":"2024-04-25T10:15:53.268049+0000","last_fullsized":"2024-04-25T10:15:53.268049+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.748468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267851+0000","last_change":"2024-04-25T10:15:52.157137+0000","last_active":"2024-04-25T10:15:53.267851+0000","last_peered":"2024-04-25T10:15:53.267851+0000","last_clean":"2024-04-25T10:15:53.267851+0000","last_became_active":"2024-04-25T10:15:52.156759+0000","last_became_peered":"2024-04-25T10:15:52.156759+0000","last_unstale":"2024-04-25T10:15:53.267851+0000","last_undegraded":"2024-04-25T10:15:53.267851+0000","last_fullsized":"2024-04-25T10:15:53.267851+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:47.694850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140771+0000","last_change":"2024-04-25T10:15:52.144492+0000","last_active":"2024-04-25T10:15:53.140771+0000","last_peered":"2024-04-25T10:15:53.140771+0000","last_clean":"2024-04-25T10:15:53.140771+0000","last_became_active":"2024-04-25T10:15:52.144181+0000","last_became_peered":"2024-04-25T10:15:52.144181+0000","last_unstale":"2024-04-25T10:15:53.140771+0000","last_undegraded":"2024-04-25T10:15:53.140771+0000","last_fullsized":"2024-04-25T10:15:53.140771+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:02.693633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265802+0000","last_change":"2024-04-25T10:15:52.149932+0000","last_active":"2024-04-25T10:15:53.265802+0000","last_peered":"2024-04-25T10:15:53.265802+0000","last_clean":"2024-04-25T10:15:53.265802+0000","last_became_active":"2024-04-25T10:15:52.149671+0000","last_became_peered":"2024-04-25T10:15:52.149671+0000","last_unstale":"2024-04-25T10:15:53.265802+0000","last_undegraded":"2024-04-25T10:15:53.265802+0000","last_fullsized":"2024-04-25T10:15:53.265802+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:59.613064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.270704+0000","last_change":"2024-04-25T10:15:52.163880+0000","last_active":"2024-04-25T10:15:53.270704+0000","last_peered":"2024-04-25T10:15:53.270704+0000","last_clean":"2024-04-25T10:15:53.270704+0000","last_became_active":"2024-04-25T10:15:52.161666+0000","last_became_peered":"2024-04-25T10:15:52.161666+0000","last_unstale":"2024-04-25T10:15:53.270704+0000","last_undegraded":"2024-04-25T10:15:53.270704+0000","last_fullsized":"2024-04-25T10:15:53.270704+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:50.787448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267225+0000","last_change":"2024-04-25T10:15:52.158023+0000","last_active":"2024-04-25T10:15:53.267225+0000","last_peered":"2024-04-25T10:15:53.267225+0000","last_clean":"2024-04-25T10:15:53.267225+0000","last_became_active":"2024-04-25T10:15:52.156527+0000","last_became_peered":"2024-04-25T10:15:52.156527+0000","last_unstale":"2024-04-25T10:15:53.267225+0000","last_undegraded":"2024-04-25T10:15:53.267225+0000","last_fullsized":"2024-04-25T10:15:53.267225+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:15.629123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267112+0000","last_change":"2024-04-25T10:15:52.153383+0000","last_active":"2024-04-25T10:15:53.267112+0000","last_peered":"2024-04-25T10:15:53.267112+0000","last_clean":"2024-04-25T10:15:53.267112+0000","last_became_active":"2024-04-25T10:15:52.152402+0000","last_became_peered":"2024-04-25T10:15:52.152402+0000","last_unstale":"2024-04-25T10:15:53.267112+0000","last_undegraded":"2024-04-25T10:15:53.267112+0000","last_fullsized":"2024-04-25T10:15:53.267112+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:21.045392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"95'4","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141688+0000","last_change":"2024-04-25T10:15:51.144455+0000","last_active":"2024-04-25T10:15:53.141688+0000","last_peered":"2024-04-25T10:15:53.141688+0000","last_clean":"2024-04-25T10:15:53.141688+0000","last_became_active":"2024-04-25T10:15:51.144147+0000","last_became_peered":"2024-04-25T10:15:51.144147+0000","last_unstale":"2024-04-25T10:15:53.141688+0000","last_undegraded":"2024-04-25T10:15:53.141688+0000","last_fullsized":"2024-04-25T10:15:53.141688+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:30.623594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.271999+0000","last_change":"2024-04-25T10:15:52.162603+0000","last_active":"2024-04-25T10:15:53.271999+0000","last_peered":"2024-04-25T10:15:53.271999+0000","last_clean":"2024-04-25T10:15:53.271999+0000","last_became_active":"2024-04-25T10:15:52.162307+0000","last_became_peered":"2024-04-25T10:15:52.162307+0000","last_unstale":"2024-04-25T10:15:53.271999+0000","last_undegraded":"2024-04-25T10:15:53.271999+0000","last_fullsized":"2024-04-25T10:15:53.271999+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:00.650074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268010+0000","last_change":"2024-04-25T10:15:51.149099+0000","last_active":"2024-04-25T10:15:53.268010+0000","last_peered":"2024-04-25T10:15:53.268010+0000","last_clean":"2024-04-25T10:15:53.268010+0000","last_became_active":"2024-04-25T10:15:51.148755+0000","last_became_peered":"2024-04-25T10:15:51.148755+0000","last_unstale":"2024-04-25T10:15:53.268010+0000","last_undegraded":"2024-04-25T10:15:53.268010+0000","last_fullsized":"2024-04-25T10:15:53.268010+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:10.014703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269116+0000","last_change":"2024-04-25T10:15:52.162825+0000","last_active":"2024-04-25T10:15:53.269116+0000","last_peered":"2024-04-25T10:15:53.269116+0000","last_clean":"2024-04-25T10:15:53.269116+0000","last_became_active":"2024-04-25T10:15:52.161569+0000","last_became_peered":"2024-04-25T10:15:52.161569+0000","last_unstale":"2024-04-25T10:15:53.269116+0000","last_undegraded":"2024-04-25T10:15:53.269116+0000","last_fullsized":"2024-04-25T10:15:53.269116+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:24.077417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141279+0000","last_change":"2024-04-25T10:15:51.139409+0000","last_active":"2024-04-25T10:15:53.141279+0000","last_peered":"2024-04-25T10:15:53.141279+0000","last_clean":"2024-04-25T10:15:53.141279+0000","last_became_active":"2024-04-25T10:15:51.139125+0000","last_became_peered":"2024-04-25T10:15:51.139125+0000","last_unstale":"2024-04-25T10:15:53.141279+0000","last_undegraded":"2024-04-25T10:15:53.141279+0000","last_fullsized":"2024-04-25T10:15:53.141279+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:49.884497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.143436+0000","last_change":"2024-04-25T10:15:52.153689+0000","last_active":"2024-04-25T10:15:53.143436+0000","last_peered":"2024-04-25T10:15:53.143436+0000","last_clean":"2024-04-25T10:15:53.143436+0000","last_became_active":"2024-04-25T10:15:52.153383+0000","last_became_peered":"2024-04-25T10:15:52.153383+0000","last_unstale":"2024-04-25T10:15:53.143436+0000","last_undegraded":"2024-04-25T10:15:53.143436+0000","last_fullsized":"2024-04-25T10:15:53.143436+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:47.694850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.142160+0000","last_change":"2024-04-25T10:15:51.140816+0000","last_active":"2024-04-25T10:15:53.142160+0000","last_peered":"2024-04-25T10:15:53.142160+0000","last_clean":"2024-04-25T10:15:53.142160+0000","last_became_active":"2024-04-25T10:15:51.140542+0000","last_became_peered":"2024-04-25T10:15:51.140542+0000","last_unstale":"2024-04-25T10:15:53.142160+0000","last_undegraded":"2024-04-25T10:15:53.142160+0000","last_fullsized":"2024-04-25T10:15:53.142160+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:07.385828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.142880+0000","last_change":"2024-04-25T10:15:52.151628+0000","last_active":"2024-04-25T10:15:53.142880+0000","last_peered":"2024-04-25T10:15:53.142880+0000","last_clean":"2024-04-25T10:15:53.142880+0000","last_became_active":"2024-04-25T10:15:52.150337+0000","last_became_peered":"2024-04-25T10:15:52.150337+0000","last_unstale":"2024-04-25T10:15:53.142880+0000","last_undegraded":"2024-04-25T10:15:53.142880+0000","last_fullsized":"2024-04-25T10:15:53.142880+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:53.142082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269025+0000","last_change":"2024-04-25T10:15:51.146840+0000","last_active":"2024-04-25T10:15:53.269025+0000","last_peered":"2024-04-25T10:15:53.269025+0000","last_clean":"2024-04-25T10:15:53.269025+0000","last_became_active":"2024-04-25T10:15:51.146522+0000","last_became_peered":"2024-04-25T10:15:51.146522+0000","last_unstale":"2024-04-25T10:15:53.269025+0000","last_undegraded":"2024-04-25T10:15:53.269025+0000","last_fullsized":"2024-04-25T10:15:53.269025+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:32.586239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267755+0000","last_change":"2024-04-25T10:15:52.166119+0000","last_active":"2024-04-25T10:15:53.267755+0000","last_peered":"2024-04-25T10:15:53.267755+0000","last_clean":"2024-04-25T10:15:53.267755+0000","last_became_active":"2024-04-25T10:15:52.165692+0000","last_became_peered":"2024-04-25T10:15:52.165692+0000","last_unstale":"2024-04-25T10:15:53.267755+0000","last_undegraded":"2024-04-25T10:15:53.267755+0000","last_fullsized":"2024-04-25T10:15:53.267755+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:31.640487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140351+0000","last_change":"2024-04-25T10:15:51.147606+0000","last_active":"2024-04-25T10:15:53.140351+0000","last_peered":"2024-04-25T10:15:53.140351+0000","last_clean":"2024-04-25T10:15:53.140351+0000","last_became_active":"2024-04-25T10:15:51.147244+0000","last_became_peered":"2024-04-25T10:15:51.147244+0000","last_unstale":"2024-04-25T10:15:53.140351+0000","last_undegraded":"2024-04-25T10:15:53.140351+0000","last_fullsized":"2024-04-25T10:15:53.140351+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:10.651021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268762+0000","last_change":"2024-04-25T10:15:52.145998+0000","last_active":"2024-04-25T10:15:53.268762+0000","last_peered":"2024-04-25T10:15:53.268762+0000","last_clean":"2024-04-25T10:15:53.268762+0000","last_became_active":"2024-04-25T10:15:52.145687+0000","last_became_peered":"2024-04-25T10:15:52.145687+0000","last_unstale":"2024-04-25T10:15:53.268762+0000","last_undegraded":"2024-04-25T10:15:53.268762+0000","last_fullsized":"2024-04-25T10:15:53.268762+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:17.808534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266846+0000","last_change":"2024-04-25T10:15:51.146376+0000","last_active":"2024-04-25T10:15:53.266846+0000","last_peered":"2024-04-25T10:15:53.266846+0000","last_clean":"2024-04-25T10:15:53.266846+0000","last_became_active":"2024-04-25T10:15:51.144259+0000","last_became_peered":"2024-04-25T10:15:51.144259+0000","last_unstale":"2024-04-25T10:15:53.266846+0000","last_undegraded":"2024-04-25T10:15:53.266846+0000","last_fullsized":"2024-04-25T10:15:53.266846+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:09.396399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.142668+0000","last_change":"2024-04-25T10:15:52.154542+0000","last_active":"2024-04-25T10:15:53.142668+0000","last_peered":"2024-04-25T10:15:53.142668+0000","last_clean":"2024-04-25T10:15:53.142668+0000","last_became_active":"2024-04-25T10:15:52.154244+0000","last_became_peered":"2024-04-25T10:15:52.154244+0000","last_unstale":"2024-04-25T10:15:53.142668+0000","last_undegraded":"2024-04-25T10:15:53.142668+0000","last_fullsized":"2024-04-25T10:15:53.142668+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.667915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267092+0000","last_change":"2024-04-25T10:15:51.144186+0000","last_active":"2024-04-25T10:15:53.267092+0000","last_peered":"2024-04-25T10:15:53.267092+0000","last_clean":"2024-04-25T10:15:53.267092+0000","last_became_active":"2024-04-25T10:15:51.142446+0000","last_became_peered":"2024-04-25T10:15:51.142446+0000","last_unstale":"2024-04-25T10:15:53.267092+0000","last_undegraded":"2024-04-25T10:15:53.267092+0000","last_fullsized":"2024-04-25T10:15:53.267092+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:02.538069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141811+0000","last_change":"2024-04-25T10:15:52.142486+0000","last_active":"2024-04-25T10:15:53.141811+0000","last_peered":"2024-04-25T10:15:53.141811+0000","last_clean":"2024-04-25T10:15:53.141811+0000","last_became_active":"2024-04-25T10:15:52.142214+0000","last_became_peered":"2024-04-25T10:15:52.142214+0000","last_unstale":"2024-04-25T10:15:53.141811+0000","last_undegraded":"2024-04-25T10:15:53.141811+0000","last_fullsized":"2024-04-25T10:15:53.141811+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:22.008403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269721+0000","last_change":"2024-04-25T10:15:51.146470+0000","last_active":"2024-04-25T10:15:53.269721+0000","last_peered":"2024-04-25T10:15:53.269721+0000","last_clean":"2024-04-25T10:15:53.269721+0000","last_became_active":"2024-04-25T10:15:51.146177+0000","last_became_peered":"2024-04-25T10:15:51.146177+0000","last_unstale":"2024-04-25T10:15:53.269721+0000","last_undegraded":"2024-04-25T10:15:53.269721+0000","last_fullsized":"2024-04-25T10:15:53.269721+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:20.123878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.144507+0000","last_change":"2024-04-25T10:15:52.156326+0000","last_active":"2024-04-25T10:15:53.144507+0000","last_peered":"2024-04-25T10:15:53.144507+0000","last_clean":"2024-04-25T10:15:53.144507+0000","last_became_active":"2024-04-25T10:15:52.155179+0000","last_became_peered":"2024-04-25T10:15:52.155179+0000","last_unstale":"2024-04-25T10:15:53.144507+0000","last_undegraded":"2024-04-25T10:15:53.144507+0000","last_fullsized":"2024-04-25T10:15:53.144507+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:13.450500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269417+0000","last_change":"2024-04-25T10:15:51.141167+0000","last_active":"2024-04-25T10:15:53.269417+0000","last_peered":"2024-04-25T10:15:53.269417+0000","last_clean":"2024-04-25T10:15:53.269417+0000","last_became_active":"2024-04-25T10:15:51.140839+0000","last_became_peered":"2024-04-25T10:15:51.140839+0000","last_unstale":"2024-04-25T10:15:53.269417+0000","last_undegraded":"2024-04-25T10:15:53.269417+0000","last_fullsized":"2024-04-25T10:15:53.269417+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:19.842248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268397+0000","last_change":"2024-04-25T10:15:52.163077+0000","last_active":"2024-04-25T10:15:53.268397+0000","last_peered":"2024-04-25T10:15:53.268397+0000","last_clean":"2024-04-25T10:15:53.268397+0000","last_became_active":"2024-04-25T10:15:52.162793+0000","last_became_peered":"2024-04-25T10:15:52.162793+0000","last_unstale":"2024-04-25T10:15:53.268397+0000","last_undegraded":"2024-04-25T10:15:53.268397+0000","last_fullsized":"2024-04-25T10:15:53.268397+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:31.640487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"95'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268577+0000","last_change":"2024-04-25T10:15:51.150229+0000","last_active":"2024-04-25T10:15:53.268577+0000","last_peered":"2024-04-25T10:15:53.268577+0000","last_clean":"2024-04-25T10:15:53.268577+0000","last_became_active":"2024-04-25T10:15:51.149922+0000","last_became_peered":"2024-04-25T10:15:51.149922+0000","last_unstale":"2024-04-25T10:15:53.268577+0000","last_undegraded":"2024-04-25T10:15:53.268577+0000","last_fullsized":"2024-04-25T10:15:53.268577+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:09.396399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268446+0000","last_change":"2024-04-25T10:15:52.163673+0000","last_active":"2024-04-25T10:15:53.268446+0000","last_peered":"2024-04-25T10:15:53.268446+0000","last_clean":"2024-04-25T10:15:53.268446+0000","last_became_active":"2024-04-25T10:15:52.163368+0000","last_became_peered":"2024-04-25T10:15:52.163368+0000","last_unstale":"2024-04-25T10:15:53.268446+0000","last_undegraded":"2024-04-25T10:15:53.268446+0000","last_fullsized":"2024-04-25T10:15:53.268446+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:17.808534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"95'6","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.144635+0000","last_change":"2024-04-25T10:15:51.134615+0000","last_active":"2024-04-25T10:15:53.144635+0000","last_peered":"2024-04-25T10:15:53.144635+0000","last_clean":"2024-04-25T10:15:53.144635+0000","last_became_active":"2024-04-25T10:15:51.133723+0000","last_became_peered":"2024-04-25T10:15:51.133723+0000","last_unstale":"2024-04-25T10:15:53.144635+0000","last_undegraded":"2024-04-25T10:15:53.144635+0000","last_fullsized":"2024-04-25T10:15:53.144635+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:09.396399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269009+0000","last_change":"2024-04-25T10:15:52.149991+0000","last_active":"2024-04-25T10:15:53.269009+0000","last_peered":"2024-04-25T10:15:53.269009+0000","last_clean":"2024-04-25T10:15:53.269009+0000","last_became_active":"2024-04-25T10:15:52.149492+0000","last_became_peered":"2024-04-25T10:15:52.149492+0000","last_unstale":"2024-04-25T10:15:53.269009+0000","last_undegraded":"2024-04-25T10:15:53.269009+0000","last_fullsized":"2024-04-25T10:15:53.269009+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:08.546400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265929+0000","last_change":"2024-04-25T10:15:51.143315+0000","last_active":"2024-04-25T10:15:53.265929+0000","last_peered":"2024-04-25T10:15:53.265929+0000","last_clean":"2024-04-25T10:15:53.265929+0000","last_became_active":"2024-04-25T10:15:51.142814+0000","last_became_peered":"2024-04-25T10:15:51.142814+0000","last_unstale":"2024-04-25T10:15:53.265929+0000","last_undegraded":"2024-04-25T10:15:53.265929+0000","last_fullsized":"2024-04-25T10:15:53.265929+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:51.152565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269208+0000","last_change":"2024-04-25T10:15:52.161587+0000","last_active":"2024-04-25T10:15:53.269208+0000","last_peered":"2024-04-25T10:15:53.269208+0000","last_clean":"2024-04-25T10:15:53.269208+0000","last_became_active":"2024-04-25T10:15:52.156903+0000","last_became_peered":"2024-04-25T10:15:52.156903+0000","last_unstale":"2024-04-25T10:15:53.269208+0000","last_undegraded":"2024-04-25T10:15:53.269208+0000","last_fullsized":"2024-04-25T10:15:53.269208+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:34.374748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266004+0000","last_change":"2024-04-25T10:15:51.146962+0000","last_active":"2024-04-25T10:15:53.266004+0000","last_peered":"2024-04-25T10:15:53.266004+0000","last_clean":"2024-04-25T10:15:53.266004+0000","last_became_active":"2024-04-25T10:15:51.146656+0000","last_became_peered":"2024-04-25T10:15:51.146656+0000","last_unstale":"2024-04-25T10:15:53.266004+0000","last_undegraded":"2024-04-25T10:15:53.266004+0000","last_fullsized":"2024-04-25T10:15:53.266004+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:39.576728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.143283+0000","last_change":"2024-04-25T10:15:52.153788+0000","last_active":"2024-04-25T10:15:53.143283+0000","last_peered":"2024-04-25T10:15:53.143283+0000","last_clean":"2024-04-25T10:15:53.143283+0000","last_became_active":"2024-04-25T10:15:52.153487+0000","last_became_peered":"2024-04-25T10:15:52.153487+0000","last_unstale":"2024-04-25T10:15:53.143283+0000","last_undegraded":"2024-04-25T10:15:53.143283+0000","last_fullsized":"2024-04-25T10:15:53.143283+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:50.901390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.142850+0000","last_change":"2024-04-25T10:15:51.143407+0000","last_active":"2024-04-25T10:15:53.142850+0000","last_peered":"2024-04-25T10:15:53.142850+0000","last_clean":"2024-04-25T10:15:53.142850+0000","last_became_active":"2024-04-25T10:15:51.143090+0000","last_became_peered":"2024-04-25T10:15:51.143090+0000","last_unstale":"2024-04-25T10:15:53.142850+0000","last_undegraded":"2024-04-25T10:15:53.142850+0000","last_fullsized":"2024-04-25T10:15:53.142850+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:10.014703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266102+0000","last_change":"2024-04-25T10:15:52.137618+0000","last_active":"2024-04-25T10:15:53.266102+0000","last_peered":"2024-04-25T10:15:53.266102+0000","last_clean":"2024-04-25T10:15:53.266102+0000","last_became_active":"2024-04-25T10:15:52.136972+0000","last_became_peered":"2024-04-25T10:15:52.136972+0000","last_unstale":"2024-04-25T10:15:53.266102+0000","last_undegraded":"2024-04-25T10:15:53.266102+0000","last_fullsized":"2024-04-25T10:15:53.266102+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:07.387774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"95'8","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267366+0000","last_change":"2024-04-25T10:15:51.146771+0000","last_active":"2024-04-25T10:15:53.267366+0000","last_peered":"2024-04-25T10:15:53.267366+0000","last_clean":"2024-04-25T10:15:53.267366+0000","last_became_active":"2024-04-25T10:15:51.146504+0000","last_became_peered":"2024-04-25T10:15:51.146504+0000","last_unstale":"2024-04-25T10:15:53.267366+0000","last_undegraded":"2024-04-25T10:15:53.267366+0000","last_fullsized":"2024-04-25T10:15:53.267366+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.028499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268538+0000","last_change":"2024-04-25T10:15:52.164645+0000","last_active":"2024-04-25T10:15:53.268538+0000","last_peered":"2024-04-25T10:15:53.268538+0000","last_clean":"2024-04-25T10:15:53.268538+0000","last_became_active":"2024-04-25T10:15:52.164334+0000","last_became_peered":"2024-04-25T10:15:52.164334+0000","last_unstale":"2024-04-25T10:15:53.268538+0000","last_undegraded":"2024-04-25T10:15:53.268538+0000","last_fullsized":"2024-04-25T10:15:53.268538+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:17.808534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.142960+0000","last_change":"2024-04-25T10:15:51.144474+0000","last_active":"2024-04-25T10:15:53.142960+0000","last_peered":"2024-04-25T10:15:53.142960+0000","last_clean":"2024-04-25T10:15:53.142960+0000","last_became_active":"2024-04-25T10:15:51.144143+0000","last_became_peered":"2024-04-25T10:15:51.144143+0000","last_unstale":"2024-04-25T10:15:53.142960+0000","last_undegraded":"2024-04-25T10:15:53.142960+0000","last_fullsized":"2024-04-25T10:15:53.142960+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:51.623284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.143744+0000","last_change":"2024-04-25T10:15:52.159942+0000","last_active":"2024-04-25T10:15:53.143744+0000","last_peered":"2024-04-25T10:15:53.143744+0000","last_clean":"2024-04-25T10:15:53.143744+0000","last_became_active":"2024-04-25T10:15:52.158275+0000","last_became_peered":"2024-04-25T10:15:52.158275+0000","last_unstale":"2024-04-25T10:15:53.143744+0000","last_undegraded":"2024-04-25T10:15:53.143744+0000","last_fullsized":"2024-04-25T10:15:53.143744+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:12.547980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267470+0000","last_change":"2024-04-25T10:15:51.144968+0000","last_active":"2024-04-25T10:15:53.267470+0000","last_peered":"2024-04-25T10:15:53.267470+0000","last_clean":"2024-04-25T10:15:53.267470+0000","last_became_active":"2024-04-25T10:15:51.144599+0000","last_became_peered":"2024-04-25T10:15:51.144599+0000","last_unstale":"2024-04-25T10:15:53.267470+0000","last_undegraded":"2024-04-25T10:15:53.267470+0000","last_fullsized":"2024-04-25T10:15:53.267470+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:10.014703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268466+0000","last_change":"2024-04-25T10:15:52.159908+0000","last_active":"2024-04-25T10:15:53.268466+0000","last_peered":"2024-04-25T10:15:53.268466+0000","last_clean":"2024-04-25T10:15:53.268466+0000","last_became_active":"2024-04-25T10:15:52.159444+0000","last_became_peered":"2024-04-25T10:15:52.159444+0000","last_unstale":"2024-04-25T10:15:53.268466+0000","last_undegraded":"2024-04-25T10:15:53.268466+0000","last_fullsized":"2024-04-25T10:15:53.268466+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:08.546400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.142152+0000","last_change":"2024-04-25T10:15:51.133376+0000","last_active":"2024-04-25T10:15:53.142152+0000","last_peered":"2024-04-25T10:15:53.142152+0000","last_clean":"2024-04-25T10:15:53.142152+0000","last_became_active":"2024-04-25T10:15:51.131804+0000","last_became_peered":"2024-04-25T10:15:51.131804+0000","last_unstale":"2024-04-25T10:15:53.142152+0000","last_undegraded":"2024-04-25T10:15:53.142152+0000","last_fullsized":"2024-04-25T10:15:53.142152+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:51.515019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265880+0000","last_change":"2024-04-25T10:15:52.147913+0000","last_active":"2024-04-25T10:15:53.265880+0000","last_peered":"2024-04-25T10:15:53.265880+0000","last_clean":"2024-04-25T10:15:53.265880+0000","last_became_active":"2024-04-25T10:15:52.147620+0000","last_became_peered":"2024-04-25T10:15:52.147620+0000","last_unstale":"2024-04-25T10:15:53.265880+0000","last_undegraded":"2024-04-25T10:15:53.265880+0000","last_fullsized":"2024-04-25T10:15:53.265880+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:49.205689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140903+0000","last_change":"2024-04-25T10:15:51.142018+0000","last_active":"2024-04-25T10:15:53.140903+0000","last_peered":"2024-04-25T10:15:53.140903+0000","last_clean":"2024-04-25T10:15:53.140903+0000","last_became_active":"2024-04-25T10:15:51.141754+0000","last_became_peered":"2024-04-25T10:15:51.141754+0000","last_unstale":"2024-04-25T10:15:53.140903+0000","last_undegraded":"2024-04-25T10:15:53.140903+0000","last_fullsized":"2024-04-25T10:15:53.140903+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:28.146412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265799+0000","last_change":"2024-04-25T10:15:52.133324+0000","last_active":"2024-04-25T10:15:53.265799+0000","last_peered":"2024-04-25T10:15:53.265799+0000","last_clean":"2024-04-25T10:15:53.265799+0000","last_became_active":"2024-04-25T10:15:52.132878+0000","last_became_peered":"2024-04-25T10:15:52.132878+0000","last_unstale":"2024-04-25T10:15:53.265799+0000","last_undegraded":"2024-04-25T10:15:53.265799+0000","last_fullsized":"2024-04-25T10:15:53.265799+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:09.055243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.143016+0000","last_change":"2024-04-25T10:15:51.147556+0000","last_active":"2024-04-25T10:15:53.143016+0000","last_peered":"2024-04-25T10:15:53.143016+0000","last_clean":"2024-04-25T10:15:53.143016+0000","last_became_active":"2024-04-25T10:15:51.147193+0000","last_became_peered":"2024-04-25T10:15:51.147193+0000","last_unstale":"2024-04-25T10:15:53.143016+0000","last_undegraded":"2024-04-25T10:15:53.143016+0000","last_fullsized":"2024-04-25T10:15:53.143016+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:59.944275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265029+0000","last_change":"2024-04-25T10:15:52.138728+0000","last_active":"2024-04-25T10:15:53.265029+0000","last_peered":"2024-04-25T10:15:53.265029+0000","last_clean":"2024-04-25T10:15:53.265029+0000","last_became_active":"2024-04-25T10:15:52.138460+0000","last_became_peered":"2024-04-25T10:15:52.138460+0000","last_unstale":"2024-04-25T10:15:53.265029+0000","last_undegraded":"2024-04-25T10:15:53.265029+0000","last_fullsized":"2024-04-25T10:15:53.265029+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:08.369733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141738+0000","last_change":"2024-04-25T10:15:51.138085+0000","last_active":"2024-04-25T10:15:53.141738+0000","last_peered":"2024-04-25T10:15:53.141738+0000","last_clean":"2024-04-25T10:15:53.141738+0000","last_became_active":"2024-04-25T10:15:51.137148+0000","last_became_peered":"2024-04-25T10:15:51.137148+0000","last_unstale":"2024-04-25T10:15:53.141738+0000","last_undegraded":"2024-04-25T10:15:53.141738+0000","last_fullsized":"2024-04-25T10:15:53.141738+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:06.786023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266655+0000","last_change":"2024-04-25T10:15:52.167093+0000","last_active":"2024-04-25T10:15:53.266655+0000","last_peered":"2024-04-25T10:15:53.266655+0000","last_clean":"2024-04-25T10:15:53.266655+0000","last_became_active":"2024-04-25T10:15:52.166750+0000","last_became_peered":"2024-04-25T10:15:52.166750+0000","last_unstale":"2024-04-25T10:15:53.266655+0000","last_undegraded":"2024-04-25T10:15:53.266655+0000","last_fullsized":"2024-04-25T10:15:53.266655+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:50.901390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140396+0000","last_change":"2024-04-25T10:15:51.149803+0000","last_active":"2024-04-25T10:15:53.140396+0000","last_peered":"2024-04-25T10:15:53.140396+0000","last_clean":"2024-04-25T10:15:53.140396+0000","last_became_active":"2024-04-25T10:15:51.149517+0000","last_became_peered":"2024-04-25T10:15:51.149517+0000","last_unstale":"2024-04-25T10:15:53.140396+0000","last_undegraded":"2024-04-25T10:15:53.140396+0000","last_fullsized":"2024-04-25T10:15:53.140396+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.568129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141986+0000","last_change":"2024-04-25T10:15:52.154948+0000","last_active":"2024-04-25T10:15:53.141986+0000","last_peered":"2024-04-25T10:15:53.141986+0000","last_clean":"2024-04-25T10:15:53.141986+0000","last_became_active":"2024-04-25T10:15:52.152707+0000","last_became_peered":"2024-04-25T10:15:52.152707+0000","last_unstale":"2024-04-25T10:15:53.141986+0000","last_undegraded":"2024-04-25T10:15:53.141986+0000","last_fullsized":"2024-04-25T10:15:53.141986+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:39.025309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.145174+0000","last_change":"2024-04-25T10:15:51.141768+0000","last_active":"2024-04-25T10:15:53.145174+0000","last_peered":"2024-04-25T10:15:53.145174+0000","last_clean":"2024-04-25T10:15:53.145174+0000","last_became_active":"2024-04-25T10:15:51.140502+0000","last_became_peered":"2024-04-25T10:15:51.140502+0000","last_unstale":"2024-04-25T10:15:53.145174+0000","last_undegraded":"2024-04-25T10:15:53.145174+0000","last_fullsized":"2024-04-25T10:15:53.145174+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:33.357854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269597+0000","last_change":"2024-04-25T10:15:52.158587+0000","last_active":"2024-04-25T10:15:53.269597+0000","last_peered":"2024-04-25T10:15:53.269597+0000","last_clean":"2024-04-25T10:15:53.269597+0000","last_became_active":"2024-04-25T10:15:52.158300+0000","last_became_peered":"2024-04-25T10:15:52.158300+0000","last_unstale":"2024-04-25T10:15:53.269597+0000","last_undegraded":"2024-04-25T10:15:53.269597+0000","last_fullsized":"2024-04-25T10:15:53.269597+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:35.672483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141933+0000","last_change":"2024-04-25T10:15:51.138821+0000","last_active":"2024-04-25T10:15:53.141933+0000","last_peered":"2024-04-25T10:15:53.141933+0000","last_clean":"2024-04-25T10:15:53.141933+0000","last_became_active":"2024-04-25T10:15:51.138377+0000","last_became_peered":"2024-04-25T10:15:51.138377+0000","last_unstale":"2024-04-25T10:15:53.141933+0000","last_undegraded":"2024-04-25T10:15:53.141933+0000","last_fullsized":"2024-04-25T10:15:53.141933+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:43.943728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269276+0000","last_change":"2024-04-25T10:15:52.163237+0000","last_active":"2024-04-25T10:15:53.269276+0000","last_peered":"2024-04-25T10:15:53.269276+0000","last_clean":"2024-04-25T10:15:53.269276+0000","last_became_active":"2024-04-25T10:15:52.162927+0000","last_became_peered":"2024-04-25T10:15:52.162927+0000","last_unstale":"2024-04-25T10:15:53.269276+0000","last_undegraded":"2024-04-25T10:15:53.269276+0000","last_fullsized":"2024-04-25T10:15:53.269276+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.961168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140984+0000","last_change":"2024-04-25T10:15:51.140544+0000","last_active":"2024-04-25T10:15:53.140984+0000","last_peered":"2024-04-25T10:15:53.140984+0000","last_clean":"2024-04-25T10:15:53.140984+0000","last_became_active":"2024-04-25T10:15:51.139959+0000","last_became_peered":"2024-04-25T10:15:51.139959+0000","last_unstale":"2024-04-25T10:15:53.140984+0000","last_undegraded":"2024-04-25T10:15:53.140984+0000","last_fullsized":"2024-04-25T10:15:53.140984+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:46.677957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268506+0000","last_change":"2024-04-25T10:15:52.164610+0000","last_active":"2024-04-25T10:15:53.268506+0000","last_peered":"2024-04-25T10:15:53.268506+0000","last_clean":"2024-04-25T10:15:53.268506+0000","last_became_active":"2024-04-25T10:15:52.164270+0000","last_became_peered":"2024-04-25T10:15:52.164270+0000","last_unstale":"2024-04-25T10:15:53.268506+0000","last_undegraded":"2024-04-25T10:15:53.268506+0000","last_fullsized":"2024-04-25T10:15:53.268506+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:13.450500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267977+0000","last_change":"2024-04-25T10:15:51.149153+0000","last_active":"2024-04-25T10:15:53.267977+0000","last_peered":"2024-04-25T10:15:53.267977+0000","last_clean":"2024-04-25T10:15:53.267977+0000","last_became_active":"2024-04-25T10:15:51.147849+0000","last_became_peered":"2024-04-25T10:15:51.147849+0000","last_unstale":"2024-04-25T10:15:53.267977+0000","last_undegraded":"2024-04-25T10:15:53.267977+0000","last_fullsized":"2024-04-25T10:15:53.267977+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:51.623284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265963+0000","last_change":"2024-04-25T10:15:52.143240+0000","last_active":"2024-04-25T10:15:53.265963+0000","last_peered":"2024-04-25T10:15:53.265963+0000","last_clean":"2024-04-25T10:15:53.265963+0000","last_became_active":"2024-04-25T10:15:52.142969+0000","last_became_peered":"2024-04-25T10:15:52.142969+0000","last_unstale":"2024-04-25T10:15:53.265963+0000","last_undegraded":"2024-04-25T10:15:53.265963+0000","last_fullsized":"2024-04-25T10:15:53.265963+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:03.086647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141191+0000","last_change":"2024-04-25T10:15:51.133337+0000","last_active":"2024-04-25T10:15:53.141191+0000","last_peered":"2024-04-25T10:15:53.141191+0000","last_clean":"2024-04-25T10:15:53.141191+0000","last_became_active":"2024-04-25T10:15:51.133007+0000","last_became_peered":"2024-04-25T10:15:51.133007+0000","last_unstale":"2024-04-25T10:15:53.141191+0000","last_undegraded":"2024-04-25T10:15:53.141191+0000","last_fullsized":"2024-04-25T10:15:53.141191+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:19.842248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.143685+0000","last_change":"2024-04-25T10:15:52.159727+0000","last_active":"2024-04-25T10:15:53.143685+0000","last_peered":"2024-04-25T10:15:53.143685+0000","last_clean":"2024-04-25T10:15:53.143685+0000","last_became_active":"2024-04-25T10:15:52.159436+0000","last_became_peered":"2024-04-25T10:15:52.159436+0000","last_unstale":"2024-04-25T10:15:53.143685+0000","last_undegraded":"2024-04-25T10:15:53.143685+0000","last_fullsized":"2024-04-25T10:15:53.143685+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:44.960622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266254+0000","last_change":"2024-04-25T10:15:51.149876+0000","last_active":"2024-04-25T10:15:53.266254+0000","last_peered":"2024-04-25T10:15:53.266254+0000","last_clean":"2024-04-25T10:15:53.266254+0000","last_became_active":"2024-04-25T10:15:51.149413+0000","last_became_peered":"2024-04-25T10:15:51.149413+0000","last_unstale":"2024-04-25T10:15:53.266254+0000","last_undegraded":"2024-04-25T10:15:53.266254+0000","last_fullsized":"2024-04-25T10:15:53.266254+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:11.531087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265276+0000","last_change":"2024-04-25T10:15:52.137308+0000","last_active":"2024-04-25T10:15:53.265276+0000","last_peered":"2024-04-25T10:15:53.265276+0000","last_clean":"2024-04-25T10:15:53.265276+0000","last_became_active":"2024-04-25T10:15:52.136959+0000","last_became_peered":"2024-04-25T10:15:52.136959+0000","last_unstale":"2024-04-25T10:15:53.265276+0000","last_undegraded":"2024-04-25T10:15:53.265276+0000","last_fullsized":"2024-04-25T10:15:53.265276+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:31.190720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.270058+0000","last_change":"2024-04-25T10:15:51.147246+0000","last_active":"2024-04-25T10:15:53.270058+0000","last_peered":"2024-04-25T10:15:53.270058+0000","last_clean":"2024-04-25T10:15:53.270058+0000","last_became_active":"2024-04-25T10:15:51.146971+0000","last_became_peered":"2024-04-25T10:15:51.146971+0000","last_unstale":"2024-04-25T10:15:53.270058+0000","last_undegraded":"2024-04-25T10:15:53.270058+0000","last_fullsized":"2024-04-25T10:15:53.270058+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:28.146412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266608+0000","last_change":"2024-04-25T10:15:52.163066+0000","last_active":"2024-04-25T10:15:53.266608+0000","last_peered":"2024-04-25T10:15:53.266608+0000","last_clean":"2024-04-25T10:15:53.266608+0000","last_became_active":"2024-04-25T10:15:52.162215+0000","last_became_peered":"2024-04-25T10:15:52.162215+0000","last_unstale":"2024-04-25T10:15:53.266608+0000","last_undegraded":"2024-04-25T10:15:53.266608+0000","last_fullsized":"2024-04-25T10:15:53.266608+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.667915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265099+0000","last_change":"2024-04-25T10:15:51.138274+0000","last_active":"2024-04-25T10:15:53.265099+0000","last_peered":"2024-04-25T10:15:53.265099+0000","last_clean":"2024-04-25T10:15:53.265099+0000","last_became_active":"2024-04-25T10:15:51.137961+0000","last_became_peered":"2024-04-25T10:15:51.137961+0000","last_unstale":"2024-04-25T10:15:53.265099+0000","last_undegraded":"2024-04-25T10:15:53.265099+0000","last_fullsized":"2024-04-25T10:15:53.265099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:09.964287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267498+0000","last_change":"2024-04-25T10:15:52.141968+0000","last_active":"2024-04-25T10:15:53.267498+0000","last_peered":"2024-04-25T10:15:53.267498+0000","last_clean":"2024-04-25T10:15:53.267498+0000","last_became_active":"2024-04-25T10:15:52.141646+0000","last_became_peered":"2024-04-25T10:15:52.141646+0000","last_unstale":"2024-04-25T10:15:53.267498+0000","last_undegraded":"2024-04-25T10:15:53.267498+0000","last_fullsized":"2024-04-25T10:15:53.267498+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:33.603132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267881+0000","last_change":"2024-04-25T10:15:51.145805+0000","last_active":"2024-04-25T10:15:53.267881+0000","last_peered":"2024-04-25T10:15:53.267881+0000","last_clean":"2024-04-25T10:15:53.267881+0000","last_became_active":"2024-04-25T10:15:51.145479+0000","last_became_peered":"2024-04-25T10:15:51.145479+0000","last_unstale":"2024-04-25T10:15:53.267881+0000","last_undegraded":"2024-04-25T10:15:53.267881+0000","last_fullsized":"2024-04-25T10:15:53.267881+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:11.531087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":314,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266874+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:15:53.266874+0000","last_peered":"2024-04-25T10:15:53.266874+0000","last_clean":"2024-04-25T10:15:53.266874+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:15:53.266874+0000","last_undegraded":"2024-04-25T10:15:53.266874+0000","last_fullsized":"2024-04-25T10:15:53.266874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574959,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1789952,"data_stored":1102347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574960,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":512519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574959,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":511937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574960,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1208320,"data_stored":513603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574959,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":513501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574960,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":511901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1187840,"data_stored":511867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574959,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1785856,"data_stored":1102817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:56.534 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:56.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.759+0000 7f2d2a800640 1 Processor -- start 2024-04-25T10:15:56.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.760+0000 7f2d2a800640 1 -- start start 2024-04-25T10:15:56.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.761+0000 7f2d2a800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2406ea30 0x7f2d241653e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.761+0000 7f2d2a800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d24144420 con 0x7f2d2415acd0 2024-04-25T10:15:56.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.761+0000 7f2d2a800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d240b3d80 con 0x7f2d24144040 2024-04-25T10:15:56.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.761+0000 7f2d2a800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240b36d0 con 0x7f2d2406ea30 2024-04-25T10:15:56.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.762+0000 7f2d28a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2d2415acd0 0x7f2d2406e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57406/0 (socket says 172.21.15.119:57406) 2024-04-25T10:15:56.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.762+0000 7f2d28a00640 1 -- 172.21.15.119:0/2695500786 learned_addr learned my addr 172.21.15.119:0/2695500786 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:56.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.762+0000 7f2d29e00640 1 --2- 172.21.15.119:0/2695500786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2406ea30 0x7f2d241653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.762+0000 7f2d23e00640 1 -- 172.21.15.119:0/2695500786 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676776139 0 0) 0x7f2d24144420 con 0x7f2d2415acd0 2024-04-25T10:15:56.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.762+0000 7f2d23e00640 1 -- 172.21.15.119:0/2695500786 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d0c005730 con 0x7f2d2415acd0 2024-04-25T10:15:56.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.762+0000 7f2d23e00640 1 -- 172.21.15.119:0/2695500786 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2776215566 0 0) 0x7f2d0c005730 con 0x7f2d2415acd0 2024-04-25T10:15:56.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.762+0000 7f2d23e00640 1 -- 172.21.15.119:0/2695500786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2406ea30 msgr2=0x7f2d241653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.762+0000 7f2d23e00640 1 --2- 172.21.15.119:0/2695500786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2406ea30 0x7f2d241653e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d23e00640 1 -- 172.21.15.119:0/2695500786 >> v1:172.21.15.132:6789/0 conn(0x7f2d24144040 legacy=0x7f2d240cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d23e00640 1 -- 172.21.15.119:0/2695500786 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d240b4060 con 0x7f2d2415acd0 2024-04-25T10:15:56.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d23e00640 1 -- 172.21.15.119:0/2695500786 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f2d10001670 con 0x7f2d2415acd0 2024-04-25T10:15:56.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d23e00640 1 -- 172.21.15.119:0/2695500786 >> v1:172.21.15.119:6789/0 conn(0x7f2d2415acd0 legacy=0x7f2d2406e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d23e00640 1 --2- 172.21.15.119:0/2695500786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2406ea30 0x7f2d0c005c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d23e00640 1 --2- 172.21.15.119:0/2695500786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c006250 0x7f2d0c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d23e00640 1 --2- 172.21.15.119:0/2695500786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d0c006f50 0x7f2d0c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d23e00640 1 -- 172.21.15.119:0/2695500786 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240b36d0 con 0x7f2d0c006250 2024-04-25T10:15:56.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d23e00640 1 -- 172.21.15.119:0/2695500786 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c00de30 con 0x7f2d0c006f50 2024-04-25T10:15:56.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d23e00640 1 -- 172.21.15.119:0/2695500786 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c00e030 con 0x7f2d2406ea30 2024-04-25T10:15:56.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d28a00640 1 --2- 172.21.15.119:0/2695500786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c006250 0x7f2d0c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d28a00640 1 -- 172.21.15.119:0/2695500786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2406ea30 msgr2=0x7f2d0c005c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d28a00640 1 --2- 172.21.15.119:0/2695500786 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2406ea30 0x7f2d0c005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d28a00640 1 -- 172.21.15.119:0/2695500786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d0c006f50 msgr2=0x7f2d0c0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:56.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d28a00640 1 --2- 172.21.15.119:0/2695500786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d0c006f50 0x7f2d0c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.763+0000 7f2d28a00640 1 -- 172.21.15.119:0/2695500786 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c005730 con 0x7f2d0c006250 2024-04-25T10:15:56.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.765+0000 7f2d28a00640 1 --2- 172.21.15.119:0/2695500786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c006250 0x7f2d0c006a10 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000c0e0 tx=0x7f2d0c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646ff657d36dab0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.765+0000 7f2d23e00640 1 -- 172.21.15.119:0/2695500786 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10007c20 con 0x7f2d0c006250 2024-04-25T10:15:56.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.765+0000 7f2d23e00640 1 -- 172.21.15.119:0/2695500786 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d10005af0 con 0x7f2d0c006250 2024-04-25T10:15:56.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.765+0000 7f2d23e00640 1 -- 172.21.15.119:0/2695500786 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10007070 con 0x7f2d0c006250 2024-04-25T10:15:56.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.765+0000 7f2d2a800640 1 -- 172.21.15.119:0/2695500786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c006250 msgr2=0x7f2d0c006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.766+0000 7f2d2a800640 1 --2- 172.21.15.119:0/2695500786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c006250 0x7f2d0c006a10 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000c0e0 tx=0x7f2d0c007a40 comp rx=0 tx=0).stop 2024-04-25T10:15:56.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.766+0000 7f2d29e00640 1 -- 172.21.15.119:0/2695500786 reap_dead start 2024-04-25T10:15:56.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.766+0000 7f2d2a800640 1 -- 172.21.15.119:0/2695500786 shutdown_connections 2024-04-25T10:15:56.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.766+0000 7f2d2a800640 1 -- 172.21.15.119:0/2695500786 >> 172.21.15.119:0/2695500786 conn(0x7f2d2406b1d0 msgr2=0x7f2d240b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:56.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.766+0000 7f2d2a800640 1 -- 172.21.15.119:0/2695500786 shutdown_connections 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.766+0000 7f2d2a800640 1 -- 172.21.15.119:0/2695500786 wait complete. 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.767+0000 7f2d2a800640 1 Processor -- start 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.768+0000 7f2d2a800640 1 -- start start 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d2a800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c006250 0x7f2d2415c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d2a800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d2413eda0 0x7f2d2413a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d2a800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2413f200 0x7f2d2413ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d2a800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240b4900 con 0x7f2d0c006250 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d2a800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240b4620 con 0x7f2d2413eda0 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d2a800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240b5180 con 0x7f2d2413f200 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d29400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c006250 0x7f2d2415c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d29400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c006250 0x7f2d2415c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46336/0 (socket says 172.21.15.119:46336) 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d29400640 1 -- 172.21.15.119:0/1274011983 learned_addr learned my addr 172.21.15.119:0/1274011983 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d28a00640 1 --2- 172.21.15.119:0/1274011983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d2413eda0 0x7f2d2413a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d29400640 1 -- 172.21.15.119:0/1274011983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2413f200 msgr2=0x7f2d2413ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d29400640 1 --2- 172.21.15.119:0/1274011983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2413f200 0x7f2d2413ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d29400640 1 -- 172.21.15.119:0/1274011983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d2413eda0 msgr2=0x7f2d2413a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d29400640 1 --2- 172.21.15.119:0/1274011983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d2413eda0 0x7f2d2413a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.769+0000 7f2d29400640 1 -- 172.21.15.119:0/1274011983 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d240ad360 con 0x7f2d0c006250 2024-04-25T10:15:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.770+0000 7f2d29400640 1 --2- 172.21.15.119:0/1274011983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c006250 0x7f2d2415c7b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800c500 tx=0x7f2d1800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.770+0000 7f2d21600640 1 -- 172.21.15.119:0/1274011983 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d180052d0 con 0x7f2d0c006250 2024-04-25T10:15:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.770+0000 7f2d21600640 1 -- 172.21.15.119:0/1274011983 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d18005470 con 0x7f2d0c006250 2024-04-25T10:15:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.770+0000 7f2d21600640 1 -- 172.21.15.119:0/1274011983 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18012870 con 0x7f2d0c006250 2024-04-25T10:15:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.770+0000 7f2d2a800640 1 -- 172.21.15.119:0/1274011983 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d240ad640 con 0x7f2d0c006250 2024-04-25T10:15:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.770+0000 7f2d2a800640 1 -- 172.21.15.119:0/1274011983 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d240adc00 con 0x7f2d0c006250 2024-04-25T10:15:56.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.771+0000 7f2d2a800640 1 -- 172.21.15.119:0/1274011983 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d240749b0 con 0x7f2d0c006250 2024-04-25T10:15:56.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.772+0000 7f2d21600640 1 -- 172.21.15.119:0/1274011983 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d18012a10 con 0x7f2d0c006250 2024-04-25T10:15:56.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.773+0000 7f2d21600640 1 --2- 172.21.15.119:0/1274011983 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d0406b4d0 0x7f2d0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:56.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.773+0000 7f2d21600640 1 -- 172.21.15.119:0/1274011983 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d18089ed0 con 0x7f2d0c006250 2024-04-25T10:15:56.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.773+0000 7f2d28a00640 1 --2- 172.21.15.119:0/1274011983 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d0406b4d0 0x7f2d0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:56.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.774+0000 7f2d28a00640 1 --2- 172.21.15.119:0/1274011983 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d0406b4d0 0x7f2d0406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2d10032000 tx=0x7f2d10009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.777+0000 7f2d21600640 1 -- 172.21.15.119:0/1274011983 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d18058d40 con 0x7f2d0c006250 2024-04-25T10:15:56.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.969+0000 7f2d2a800640 1 -- 172.21.15.119:0/1274011983 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d240b2740 con 0x7f2d0406b4d0 2024-04-25T10:15:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.987+0000 7f2d21600640 1 -- 172.21.15.119:0/1274011983 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252086 (secure 0 0 0) 0x7f2d240b2740 con 0x7f2d0406b4d0 2024-04-25T10:15:56.992 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:56.995 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:15:56.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.993+0000 7f2d2a800640 1 -- 172.21.15.119:0/1274011983 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d0406b4d0 msgr2=0x7f2d0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.993+0000 7f2d2a800640 1 --2- 172.21.15.119:0/1274011983 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d0406b4d0 0x7f2d0406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2d10032000 tx=0x7f2d10009030 comp rx=0 tx=0).stop 2024-04-25T10:15:56.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.993+0000 7f2d2a800640 1 -- 172.21.15.119:0/1274011983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c006250 msgr2=0x7f2d2415c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:56.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.993+0000 7f2d2a800640 1 --2- 172.21.15.119:0/1274011983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c006250 0x7f2d2415c7b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800c500 tx=0x7f2d1800c9c0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.993+0000 7f2d2a800640 1 -- 172.21.15.119:0/1274011983 shutdown_connections 2024-04-25T10:15:56.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.993+0000 7f2d2a800640 1 --2- 172.21.15.119:0/1274011983 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d0406b4d0 0x7f2d0406d990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.993+0000 7f2d2a800640 1 --2- 172.21.15.119:0/1274011983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d2413f200 0x7f2d2413ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.993+0000 7f2d2a800640 1 --2- 172.21.15.119:0/1274011983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d2413eda0 0x7f2d2413a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.993+0000 7f2d2a800640 1 --2- 172.21.15.119:0/1274011983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c006250 0x7f2d2415c7b0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:56.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.993+0000 7f2d2a800640 1 -- 172.21.15.119:0/1274011983 >> 172.21.15.119:0/1274011983 conn(0x7f2d2406b1d0 msgr2=0x7f2d240b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:56.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.993+0000 7f2d2a800640 1 -- 172.21.15.119:0/1274011983 shutdown_connections 2024-04-25T10:15:56.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:56.994+0000 7f2d2a800640 1 -- 172.21.15.119:0/1274011983 wait complete. 2024-04-25T10:15:57.018 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-04-25T10:15:56.102914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259984,"kb_used_data":10504,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714736,"statfs":{"total":773094113280,"available":772827889664,"internally_reserved":0,"allocated":10756096,"data_stored":5280492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002186"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269210+0000","last_change":"2024-04-25T10:15:52.154315+0000","last_active":"2024-04-25T10:15:53.269210+0000","last_peered":"2024-04-25T10:15:53.269210+0000","last_clean":"2024-04-25T10:15:53.269210+0000","last_became_active":"2024-04-25T10:15:52.154027+0000","last_became_peered":"2024-04-25T10:15:52.154027+0000","last_unstale":"2024-04-25T10:15:53.269210+0000","last_undegraded":"2024-04-25T10:15:53.269210+0000","last_fullsized":"2024-04-25T10:15:53.269210+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:07.802917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141267+0000","last_change":"2024-04-25T10:15:52.157426+0000","last_active":"2024-04-25T10:15:53.141267+0000","last_peered":"2024-04-25T10:15:53.141267+0000","last_clean":"2024-04-25T10:15:53.141267+0000","last_became_active":"2024-04-25T10:15:52.154661+0000","last_became_peered":"2024-04-25T10:15:52.154661+0000","last_unstale":"2024-04-25T10:15:53.141267+0000","last_undegraded":"2024-04-25T10:15:53.141267+0000","last_fullsized":"2024-04-25T10:15:53.141267+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:23.312018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269529+0000","last_change":"2024-04-25T10:15:52.155191+0000","last_active":"2024-04-25T10:15:53.269529+0000","last_peered":"2024-04-25T10:15:53.269529+0000","last_clean":"2024-04-25T10:15:53.269529+0000","last_became_active":"2024-04-25T10:15:52.154079+0000","last_became_peered":"2024-04-25T10:15:52.154079+0000","last_unstale":"2024-04-25T10:15:53.269529+0000","last_undegraded":"2024-04-25T10:15:53.269529+0000","last_fullsized":"2024-04-25T10:15:53.269529+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:27.710289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.271481+0000","last_change":"2024-04-25T10:15:52.135915+0000","last_active":"2024-04-25T10:15:53.271481+0000","last_peered":"2024-04-25T10:15:53.271481+0000","last_clean":"2024-04-25T10:15:53.271481+0000","last_became_active":"2024-04-25T10:15:52.135516+0000","last_became_peered":"2024-04-25T10:15:52.135516+0000","last_unstale":"2024-04-25T10:15:53.271481+0000","last_undegraded":"2024-04-25T10:15:53.271481+0000","last_fullsized":"2024-04-25T10:15:53.271481+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:25.250794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268424+0000","last_change":"2024-04-25T10:15:52.158208+0000","last_active":"2024-04-25T10:15:53.268424+0000","last_peered":"2024-04-25T10:15:53.268424+0000","last_clean":"2024-04-25T10:15:53.268424+0000","last_became_active":"2024-04-25T10:15:52.157763+0000","last_became_peered":"2024-04-25T10:15:52.157763+0000","last_unstale":"2024-04-25T10:15:53.268424+0000","last_undegraded":"2024-04-25T10:15:53.268424+0000","last_fullsized":"2024-04-25T10:15:53.268424+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.748468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267808+0000","last_change":"2024-04-25T10:15:52.156086+0000","last_active":"2024-04-25T10:15:53.267808+0000","last_peered":"2024-04-25T10:15:53.267808+0000","last_clean":"2024-04-25T10:15:53.267808+0000","last_became_active":"2024-04-25T10:15:52.155766+0000","last_became_peered":"2024-04-25T10:15:52.155766+0000","last_unstale":"2024-04-25T10:15:53.267808+0000","last_undegraded":"2024-04-25T10:15:53.267808+0000","last_fullsized":"2024-04-25T10:15:53.267808+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.740446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266705+0000","last_change":"2024-04-25T10:15:52.153556+0000","last_active":"2024-04-25T10:15:53.266705+0000","last_peered":"2024-04-25T10:15:53.266705+0000","last_clean":"2024-04-25T10:15:53.266705+0000","last_became_active":"2024-04-25T10:15:52.153231+0000","last_became_peered":"2024-04-25T10:15:52.153231+0000","last_unstale":"2024-04-25T10:15:53.266705+0000","last_undegraded":"2024-04-25T10:15:53.266705+0000","last_fullsized":"2024-04-25T10:15:53.266705+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:47.694850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140749+0000","last_change":"2024-04-25T10:15:52.151655+0000","last_active":"2024-04-25T10:15:53.140749+0000","last_peered":"2024-04-25T10:15:53.140749+0000","last_clean":"2024-04-25T10:15:53.140749+0000","last_became_active":"2024-04-25T10:15:52.151398+0000","last_became_peered":"2024-04-25T10:15:52.151398+0000","last_unstale":"2024-04-25T10:15:53.140749+0000","last_undegraded":"2024-04-25T10:15:53.140749+0000","last_fullsized":"2024-04-25T10:15:53.140749+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:05.831266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.144462+0000","last_change":"2024-04-25T10:15:52.138831+0000","last_active":"2024-04-25T10:15:53.144462+0000","last_peered":"2024-04-25T10:15:53.144462+0000","last_clean":"2024-04-25T10:15:53.144462+0000","last_became_active":"2024-04-25T10:15:52.138369+0000","last_became_peered":"2024-04-25T10:15:52.138369+0000","last_unstale":"2024-04-25T10:15:53.144462+0000","last_undegraded":"2024-04-25T10:15:53.144462+0000","last_fullsized":"2024-04-25T10:15:53.144462+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.740446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268347+0000","last_change":"2024-04-25T10:15:52.164913+0000","last_active":"2024-04-25T10:15:53.268347+0000","last_peered":"2024-04-25T10:15:53.268347+0000","last_clean":"2024-04-25T10:15:53.268347+0000","last_became_active":"2024-04-25T10:15:52.163399+0000","last_became_peered":"2024-04-25T10:15:52.163399+0000","last_unstale":"2024-04-25T10:15:53.268347+0000","last_undegraded":"2024-04-25T10:15:53.268347+0000","last_fullsized":"2024-04-25T10:15:53.268347+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:44.960622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268116+0000","last_change":"2024-04-25T10:15:52.159664+0000","last_active":"2024-04-25T10:15:53.268116+0000","last_peered":"2024-04-25T10:15:53.268116+0000","last_clean":"2024-04-25T10:15:53.268116+0000","last_became_active":"2024-04-25T10:15:52.159132+0000","last_became_peered":"2024-04-25T10:15:52.159132+0000","last_unstale":"2024-04-25T10:15:53.268116+0000","last_undegraded":"2024-04-25T10:15:53.268116+0000","last_fullsized":"2024-04-25T10:15:53.268116+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:13.450500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.144460+0000","last_change":"2024-04-25T10:15:52.161488+0000","last_active":"2024-04-25T10:15:53.144460+0000","last_peered":"2024-04-25T10:15:53.144460+0000","last_clean":"2024-04-25T10:15:53.144460+0000","last_became_active":"2024-04-25T10:15:52.161171+0000","last_became_peered":"2024-04-25T10:15:52.161171+0000","last_unstale":"2024-04-25T10:15:53.144460+0000","last_undegraded":"2024-04-25T10:15:53.144460+0000","last_fullsized":"2024-04-25T10:15:53.144460+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.748468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268687+0000","last_change":"2024-04-25T10:15:52.148522+0000","last_active":"2024-04-25T10:15:53.268687+0000","last_peered":"2024-04-25T10:15:53.268687+0000","last_clean":"2024-04-25T10:15:53.268687+0000","last_became_active":"2024-04-25T10:15:52.148251+0000","last_became_peered":"2024-04-25T10:15:52.148251+0000","last_unstale":"2024-04-25T10:15:53.268687+0000","last_undegraded":"2024-04-25T10:15:53.268687+0000","last_fullsized":"2024-04-25T10:15:53.268687+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:13.450500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265417+0000","last_change":"2024-04-25T10:15:52.158446+0000","last_active":"2024-04-25T10:15:53.265417+0000","last_peered":"2024-04-25T10:15:53.265417+0000","last_clean":"2024-04-25T10:15:53.265417+0000","last_became_active":"2024-04-25T10:15:52.158152+0000","last_became_peered":"2024-04-25T10:15:52.158152+0000","last_unstale":"2024-04-25T10:15:53.265417+0000","last_undegraded":"2024-04-25T10:15:53.265417+0000","last_fullsized":"2024-04-25T10:15:53.265417+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:44.426438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.271964+0000","last_change":"2024-04-25T10:15:52.153543+0000","last_active":"2024-04-25T10:15:53.271964+0000","last_peered":"2024-04-25T10:15:53.271964+0000","last_clean":"2024-04-25T10:15:53.271964+0000","last_became_active":"2024-04-25T10:15:52.152969+0000","last_became_peered":"2024-04-25T10:15:52.152969+0000","last_unstale":"2024-04-25T10:15:53.271964+0000","last_undegraded":"2024-04-25T10:15:53.271964+0000","last_fullsized":"2024-04-25T10:15:53.271964+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:21.257325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266474+0000","last_change":"2024-04-25T10:15:52.159324+0000","last_active":"2024-04-25T10:15:53.266474+0000","last_peered":"2024-04-25T10:15:53.266474+0000","last_clean":"2024-04-25T10:15:53.266474+0000","last_became_active":"2024-04-25T10:15:52.159032+0000","last_became_peered":"2024-04-25T10:15:52.159032+0000","last_unstale":"2024-04-25T10:15:53.266474+0000","last_undegraded":"2024-04-25T10:15:53.266474+0000","last_fullsized":"2024-04-25T10:15:53.266474+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:50.901390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141939+0000","last_change":"2024-04-25T10:15:52.149380+0000","last_active":"2024-04-25T10:15:53.141939+0000","last_peered":"2024-04-25T10:15:53.141939+0000","last_clean":"2024-04-25T10:15:53.141939+0000","last_became_active":"2024-04-25T10:15:52.148988+0000","last_became_peered":"2024-04-25T10:15:52.148988+0000","last_unstale":"2024-04-25T10:15:53.141939+0000","last_undegraded":"2024-04-25T10:15:53.141939+0000","last_fullsized":"2024-04-25T10:15:53.141939+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.853674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269343+0000","last_change":"2024-04-25T10:15:52.151357+0000","last_active":"2024-04-25T10:15:53.269343+0000","last_peered":"2024-04-25T10:15:53.269343+0000","last_clean":"2024-04-25T10:15:53.269343+0000","last_became_active":"2024-04-25T10:15:52.151080+0000","last_became_peered":"2024-04-25T10:15:52.151080+0000","last_unstale":"2024-04-25T10:15:53.269343+0000","last_undegraded":"2024-04-25T10:15:53.269343+0000","last_fullsized":"2024-04-25T10:15:53.269343+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:37.981100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266594+0000","last_change":"2024-04-25T10:15:52.147829+0000","last_active":"2024-04-25T10:15:53.266594+0000","last_peered":"2024-04-25T10:15:53.266594+0000","last_clean":"2024-04-25T10:15:53.266594+0000","last_became_active":"2024-04-25T10:15:52.147509+0000","last_became_peered":"2024-04-25T10:15:52.147509+0000","last_unstale":"2024-04-25T10:15:53.266594+0000","last_undegraded":"2024-04-25T10:15:53.266594+0000","last_fullsized":"2024-04-25T10:15:53.266594+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:03.554962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.144194+0000","last_change":"2024-04-25T10:15:52.141031+0000","last_active":"2024-04-25T10:15:53.144194+0000","last_peered":"2024-04-25T10:15:53.144194+0000","last_clean":"2024-04-25T10:15:53.144194+0000","last_became_active":"2024-04-25T10:15:52.140645+0000","last_became_peered":"2024-04-25T10:15:52.140645+0000","last_unstale":"2024-04-25T10:15:53.144194+0000","last_undegraded":"2024-04-25T10:15:53.144194+0000","last_fullsized":"2024-04-25T10:15:53.144194+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:31.640487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265726+0000","last_change":"2024-04-25T10:15:52.152135+0000","last_active":"2024-04-25T10:15:53.265726+0000","last_peered":"2024-04-25T10:15:53.265726+0000","last_clean":"2024-04-25T10:15:53.265726+0000","last_became_active":"2024-04-25T10:15:52.151680+0000","last_became_peered":"2024-04-25T10:15:52.151680+0000","last_unstale":"2024-04-25T10:15:53.265726+0000","last_undegraded":"2024-04-25T10:15:53.265726+0000","last_fullsized":"2024-04-25T10:15:53.265726+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:50.980785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140287+0000","last_change":"2024-04-25T10:15:52.159380+0000","last_active":"2024-04-25T10:15:53.140287+0000","last_peered":"2024-04-25T10:15:53.140287+0000","last_clean":"2024-04-25T10:15:53.140287+0000","last_became_active":"2024-04-25T10:15:52.159077+0000","last_became_peered":"2024-04-25T10:15:52.159077+0000","last_unstale":"2024-04-25T10:15:53.140287+0000","last_undegraded":"2024-04-25T10:15:53.140287+0000","last_fullsized":"2024-04-25T10:15:53.140287+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:00.650074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140579+0000","last_change":"2024-04-25T10:15:52.156440+0000","last_active":"2024-04-25T10:15:53.140579+0000","last_peered":"2024-04-25T10:15:53.140579+0000","last_clean":"2024-04-25T10:15:53.140579+0000","last_became_active":"2024-04-25T10:15:52.156176+0000","last_became_peered":"2024-04-25T10:15:52.156176+0000","last_unstale":"2024-04-25T10:15:53.140579+0000","last_undegraded":"2024-04-25T10:15:53.140579+0000","last_fullsized":"2024-04-25T10:15:53.140579+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:01.862561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266786+0000","last_change":"2024-04-25T10:15:52.160632+0000","last_active":"2024-04-25T10:15:53.266786+0000","last_peered":"2024-04-25T10:15:53.266786+0000","last_clean":"2024-04-25T10:15:53.266786+0000","last_became_active":"2024-04-25T10:15:52.160288+0000","last_became_peered":"2024-04-25T10:15:52.160288+0000","last_unstale":"2024-04-25T10:15:53.266786+0000","last_undegraded":"2024-04-25T10:15:53.266786+0000","last_fullsized":"2024-04-25T10:15:53.266786+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:44.960622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268419+0000","last_change":"2024-04-25T10:15:52.165387+0000","last_active":"2024-04-25T10:15:53.268419+0000","last_peered":"2024-04-25T10:15:53.268419+0000","last_clean":"2024-04-25T10:15:53.268419+0000","last_became_active":"2024-04-25T10:15:52.163584+0000","last_became_peered":"2024-04-25T10:15:52.163584+0000","last_unstale":"2024-04-25T10:15:53.268419+0000","last_undegraded":"2024-04-25T10:15:53.268419+0000","last_fullsized":"2024-04-25T10:15:53.268419+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.740446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141623+0000","last_change":"2024-04-25T10:15:52.146784+0000","last_active":"2024-04-25T10:15:53.141623+0000","last_peered":"2024-04-25T10:15:53.141623+0000","last_clean":"2024-04-25T10:15:53.141623+0000","last_became_active":"2024-04-25T10:15:52.146477+0000","last_became_peered":"2024-04-25T10:15:52.146477+0000","last_unstale":"2024-04-25T10:15:53.141623+0000","last_undegraded":"2024-04-25T10:15:53.141623+0000","last_fullsized":"2024-04-25T10:15:53.141623+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:25.764320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268049+0000","last_change":"2024-04-25T10:15:52.162183+0000","last_active":"2024-04-25T10:15:53.268049+0000","last_peered":"2024-04-25T10:15:53.268049+0000","last_clean":"2024-04-25T10:15:53.268049+0000","last_became_active":"2024-04-25T10:15:52.160494+0000","last_became_peered":"2024-04-25T10:15:52.160494+0000","last_unstale":"2024-04-25T10:15:53.268049+0000","last_undegraded":"2024-04-25T10:15:53.268049+0000","last_fullsized":"2024-04-25T10:15:53.268049+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.748468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267851+0000","last_change":"2024-04-25T10:15:52.157137+0000","last_active":"2024-04-25T10:15:53.267851+0000","last_peered":"2024-04-25T10:15:53.267851+0000","last_clean":"2024-04-25T10:15:53.267851+0000","last_became_active":"2024-04-25T10:15:52.156759+0000","last_became_peered":"2024-04-25T10:15:52.156759+0000","last_unstale":"2024-04-25T10:15:53.267851+0000","last_undegraded":"2024-04-25T10:15:53.267851+0000","last_fullsized":"2024-04-25T10:15:53.267851+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:47.694850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140771+0000","last_change":"2024-04-25T10:15:52.144492+0000","last_active":"2024-04-25T10:15:53.140771+0000","last_peered":"2024-04-25T10:15:53.140771+0000","last_clean":"2024-04-25T10:15:53.140771+0000","last_became_active":"2024-04-25T10:15:52.144181+0000","last_became_peered":"2024-04-25T10:15:52.144181+0000","last_unstale":"2024-04-25T10:15:53.140771+0000","last_undegraded":"2024-04-25T10:15:53.140771+0000","last_fullsized":"2024-04-25T10:15:53.140771+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:02.693633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265802+0000","last_change":"2024-04-25T10:15:52.149932+0000","last_active":"2024-04-25T10:15:53.265802+0000","last_peered":"2024-04-25T10:15:53.265802+0000","last_clean":"2024-04-25T10:15:53.265802+0000","last_became_active":"2024-04-25T10:15:52.149671+0000","last_became_peered":"2024-04-25T10:15:52.149671+0000","last_unstale":"2024-04-25T10:15:53.265802+0000","last_undegraded":"2024-04-25T10:15:53.265802+0000","last_fullsized":"2024-04-25T10:15:53.265802+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:59.613064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.270704+0000","last_change":"2024-04-25T10:15:52.163880+0000","last_active":"2024-04-25T10:15:53.270704+0000","last_peered":"2024-04-25T10:15:53.270704+0000","last_clean":"2024-04-25T10:15:53.270704+0000","last_became_active":"2024-04-25T10:15:52.161666+0000","last_became_peered":"2024-04-25T10:15:52.161666+0000","last_unstale":"2024-04-25T10:15:53.270704+0000","last_undegraded":"2024-04-25T10:15:53.270704+0000","last_fullsized":"2024-04-25T10:15:53.270704+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:50.787448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267225+0000","last_change":"2024-04-25T10:15:52.158023+0000","last_active":"2024-04-25T10:15:53.267225+0000","last_peered":"2024-04-25T10:15:53.267225+0000","last_clean":"2024-04-25T10:15:53.267225+0000","last_became_active":"2024-04-25T10:15:52.156527+0000","last_became_peered":"2024-04-25T10:15:52.156527+0000","last_unstale":"2024-04-25T10:15:53.267225+0000","last_undegraded":"2024-04-25T10:15:53.267225+0000","last_fullsized":"2024-04-25T10:15:53.267225+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:15.629123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267112+0000","last_change":"2024-04-25T10:15:52.153383+0000","last_active":"2024-04-25T10:15:53.267112+0000","last_peered":"2024-04-25T10:15:53.267112+0000","last_clean":"2024-04-25T10:15:53.267112+0000","last_became_active":"2024-04-25T10:15:52.152402+0000","last_became_peered":"2024-04-25T10:15:52.152402+0000","last_unstale":"2024-04-25T10:15:53.267112+0000","last_undegraded":"2024-04-25T10:15:53.267112+0000","last_fullsized":"2024-04-25T10:15:53.267112+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:21.045392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"95'4","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141688+0000","last_change":"2024-04-25T10:15:51.144455+0000","last_active":"2024-04-25T10:15:53.141688+0000","last_peered":"2024-04-25T10:15:53.141688+0000","last_clean":"2024-04-25T10:15:53.141688+0000","last_became_active":"2024-04-25T10:15:51.144147+0000","last_became_peered":"2024-04-25T10:15:51.144147+0000","last_unstale":"2024-04-25T10:15:53.141688+0000","last_undegraded":"2024-04-25T10:15:53.141688+0000","last_fullsized":"2024-04-25T10:15:53.141688+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:30.623594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.271999+0000","last_change":"2024-04-25T10:15:52.162603+0000","last_active":"2024-04-25T10:15:53.271999+0000","last_peered":"2024-04-25T10:15:53.271999+0000","last_clean":"2024-04-25T10:15:53.271999+0000","last_became_active":"2024-04-25T10:15:52.162307+0000","last_became_peered":"2024-04-25T10:15:52.162307+0000","last_unstale":"2024-04-25T10:15:53.271999+0000","last_undegraded":"2024-04-25T10:15:53.271999+0000","last_fullsized":"2024-04-25T10:15:53.271999+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:00.650074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268010+0000","last_change":"2024-04-25T10:15:51.149099+0000","last_active":"2024-04-25T10:15:53.268010+0000","last_peered":"2024-04-25T10:15:53.268010+0000","last_clean":"2024-04-25T10:15:53.268010+0000","last_became_active":"2024-04-25T10:15:51.148755+0000","last_became_peered":"2024-04-25T10:15:51.148755+0000","last_unstale":"2024-04-25T10:15:53.268010+0000","last_undegraded":"2024-04-25T10:15:53.268010+0000","last_fullsized":"2024-04-25T10:15:53.268010+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:10.014703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269116+0000","last_change":"2024-04-25T10:15:52.162825+0000","last_active":"2024-04-25T10:15:53.269116+0000","last_peered":"2024-04-25T10:15:53.269116+0000","last_clean":"2024-04-25T10:15:53.269116+0000","last_became_active":"2024-04-25T10:15:52.161569+0000","last_became_peered":"2024-04-25T10:15:52.161569+0000","last_unstale":"2024-04-25T10:15:53.269116+0000","last_undegraded":"2024-04-25T10:15:53.269116+0000","last_fullsized":"2024-04-25T10:15:53.269116+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:24.077417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141279+0000","last_change":"2024-04-25T10:15:51.139409+0000","last_active":"2024-04-25T10:15:53.141279+0000","last_peered":"2024-04-25T10:15:53.141279+0000","last_clean":"2024-04-25T10:15:53.141279+0000","last_became_active":"2024-04-25T10:15:51.139125+0000","last_became_peered":"2024-04-25T10:15:51.139125+0000","last_unstale":"2024-04-25T10:15:53.141279+0000","last_undegraded":"2024-04-25T10:15:53.141279+0000","last_fullsized":"2024-04-25T10:15:53.141279+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:49.884497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.143436+0000","last_change":"2024-04-25T10:15:52.153689+0000","last_active":"2024-04-25T10:15:53.143436+0000","last_peered":"2024-04-25T10:15:53.143436+0000","last_clean":"2024-04-25T10:15:53.143436+0000","last_became_active":"2024-04-25T10:15:52.153383+0000","last_became_peered":"2024-04-25T10:15:52.153383+0000","last_unstale":"2024-04-25T10:15:53.143436+0000","last_undegraded":"2024-04-25T10:15:53.143436+0000","last_fullsized":"2024-04-25T10:15:53.143436+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:47.694850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.142160+0000","last_change":"2024-04-25T10:15:51.140816+0000","last_active":"2024-04-25T10:15:53.142160+0000","last_peered":"2024-04-25T10:15:53.142160+0000","last_clean":"2024-04-25T10:15:53.142160+0000","last_became_active":"2024-04-25T10:15:51.140542+0000","last_became_peered":"2024-04-25T10:15:51.140542+0000","last_unstale":"2024-04-25T10:15:53.142160+0000","last_undegraded":"2024-04-25T10:15:53.142160+0000","last_fullsized":"2024-04-25T10:15:53.142160+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:07.385828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.142880+0000","last_change":"2024-04-25T10:15:52.151628+0000","last_active":"2024-04-25T10:15:53.142880+0000","last_peered":"2024-04-25T10:15:53.142880+0000","last_clean":"2024-04-25T10:15:53.142880+0000","last_became_active":"2024-04-25T10:15:52.150337+0000","last_became_peered":"2024-04-25T10:15:52.150337+0000","last_unstale":"2024-04-25T10:15:53.142880+0000","last_undegraded":"2024-04-25T10:15:53.142880+0000","last_fullsized":"2024-04-25T10:15:53.142880+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:53.142082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269025+0000","last_change":"2024-04-25T10:15:51.146840+0000","last_active":"2024-04-25T10:15:53.269025+0000","last_peered":"2024-04-25T10:15:53.269025+0000","last_clean":"2024-04-25T10:15:53.269025+0000","last_became_active":"2024-04-25T10:15:51.146522+0000","last_became_peered":"2024-04-25T10:15:51.146522+0000","last_unstale":"2024-04-25T10:15:53.269025+0000","last_undegraded":"2024-04-25T10:15:53.269025+0000","last_fullsized":"2024-04-25T10:15:53.269025+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:32.586239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267755+0000","last_change":"2024-04-25T10:15:52.166119+0000","last_active":"2024-04-25T10:15:53.267755+0000","last_peered":"2024-04-25T10:15:53.267755+0000","last_clean":"2024-04-25T10:15:53.267755+0000","last_became_active":"2024-04-25T10:15:52.165692+0000","last_became_peered":"2024-04-25T10:15:52.165692+0000","last_unstale":"2024-04-25T10:15:53.267755+0000","last_undegraded":"2024-04-25T10:15:53.267755+0000","last_fullsized":"2024-04-25T10:15:53.267755+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:31.640487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140351+0000","last_change":"2024-04-25T10:15:51.147606+0000","last_active":"2024-04-25T10:15:53.140351+0000","last_peered":"2024-04-25T10:15:53.140351+0000","last_clean":"2024-04-25T10:15:53.140351+0000","last_became_active":"2024-04-25T10:15:51.147244+0000","last_became_peered":"2024-04-25T10:15:51.147244+0000","last_unstale":"2024-04-25T10:15:53.140351+0000","last_undegraded":"2024-04-25T10:15:53.140351+0000","last_fullsized":"2024-04-25T10:15:53.140351+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:10.651021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268762+0000","last_change":"2024-04-25T10:15:52.145998+0000","last_active":"2024-04-25T10:15:53.268762+0000","last_peered":"2024-04-25T10:15:53.268762+0000","last_clean":"2024-04-25T10:15:53.268762+0000","last_became_active":"2024-04-25T10:15:52.145687+0000","last_became_peered":"2024-04-25T10:15:52.145687+0000","last_unstale":"2024-04-25T10:15:53.268762+0000","last_undegraded":"2024-04-25T10:15:53.268762+0000","last_fullsized":"2024-04-25T10:15:53.268762+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:17.808534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266846+0000","last_change":"2024-04-25T10:15:51.146376+0000","last_active":"2024-04-25T10:15:53.266846+0000","last_peered":"2024-04-25T10:15:53.266846+0000","last_clean":"2024-04-25T10:15:53.266846+0000","last_became_active":"2024-04-25T10:15:51.144259+0000","last_became_peered":"2024-04-25T10:15:51.144259+0000","last_unstale":"2024-04-25T10:15:53.266846+0000","last_undegraded":"2024-04-25T10:15:53.266846+0000","last_fullsized":"2024-04-25T10:15:53.266846+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:09.396399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.142668+0000","last_change":"2024-04-25T10:15:52.154542+0000","last_active":"2024-04-25T10:15:53.142668+0000","last_peered":"2024-04-25T10:15:53.142668+0000","last_clean":"2024-04-25T10:15:53.142668+0000","last_became_active":"2024-04-25T10:15:52.154244+0000","last_became_peered":"2024-04-25T10:15:52.154244+0000","last_unstale":"2024-04-25T10:15:53.142668+0000","last_undegraded":"2024-04-25T10:15:53.142668+0000","last_fullsized":"2024-04-25T10:15:53.142668+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.667915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267092+0000","last_change":"2024-04-25T10:15:51.144186+0000","last_active":"2024-04-25T10:15:53.267092+0000","last_peered":"2024-04-25T10:15:53.267092+0000","last_clean":"2024-04-25T10:15:53.267092+0000","last_became_active":"2024-04-25T10:15:51.142446+0000","last_became_peered":"2024-04-25T10:15:51.142446+0000","last_unstale":"2024-04-25T10:15:53.267092+0000","last_undegraded":"2024-04-25T10:15:53.267092+0000","last_fullsized":"2024-04-25T10:15:53.267092+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:02.538069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141811+0000","last_change":"2024-04-25T10:15:52.142486+0000","last_active":"2024-04-25T10:15:53.141811+0000","last_peered":"2024-04-25T10:15:53.141811+0000","last_clean":"2024-04-25T10:15:53.141811+0000","last_became_active":"2024-04-25T10:15:52.142214+0000","last_became_peered":"2024-04-25T10:15:52.142214+0000","last_unstale":"2024-04-25T10:15:53.141811+0000","last_undegraded":"2024-04-25T10:15:53.141811+0000","last_fullsized":"2024-04-25T10:15:53.141811+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:22.008403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269721+0000","last_change":"2024-04-25T10:15:51.146470+0000","last_active":"2024-04-25T10:15:53.269721+0000","last_peered":"2024-04-25T10:15:53.269721+0000","last_clean":"2024-04-25T10:15:53.269721+0000","last_became_active":"2024-04-25T10:15:51.146177+0000","last_became_peered":"2024-04-25T10:15:51.146177+0000","last_unstale":"2024-04-25T10:15:53.269721+0000","last_undegraded":"2024-04-25T10:15:53.269721+0000","last_fullsized":"2024-04-25T10:15:53.269721+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:20.123878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.144507+0000","last_change":"2024-04-25T10:15:52.156326+0000","last_active":"2024-04-25T10:15:53.144507+0000","last_peered":"2024-04-25T10:15:53.144507+0000","last_clean":"2024-04-25T10:15:53.144507+0000","last_became_active":"2024-04-25T10:15:52.155179+0000","last_became_peered":"2024-04-25T10:15:52.155179+0000","last_unstale":"2024-04-25T10:15:53.144507+0000","last_undegraded":"2024-04-25T10:15:53.144507+0000","last_fullsized":"2024-04-25T10:15:53.144507+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:13.450500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269417+0000","last_change":"2024-04-25T10:15:51.141167+0000","last_active":"2024-04-25T10:15:53.269417+0000","last_peered":"2024-04-25T10:15:53.269417+0000","last_clean":"2024-04-25T10:15:53.269417+0000","last_became_active":"2024-04-25T10:15:51.140839+0000","last_became_peered":"2024-04-25T10:15:51.140839+0000","last_unstale":"2024-04-25T10:15:53.269417+0000","last_undegraded":"2024-04-25T10:15:53.269417+0000","last_fullsized":"2024-04-25T10:15:53.269417+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:19.842248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268397+0000","last_change":"2024-04-25T10:15:52.163077+0000","last_active":"2024-04-25T10:15:53.268397+0000","last_peered":"2024-04-25T10:15:53.268397+0000","last_clean":"2024-04-25T10:15:53.268397+0000","last_became_active":"2024-04-25T10:15:52.162793+0000","last_became_peered":"2024-04-25T10:15:52.162793+0000","last_unstale":"2024-04-25T10:15:53.268397+0000","last_undegraded":"2024-04-25T10:15:53.268397+0000","last_fullsized":"2024-04-25T10:15:53.268397+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:31.640487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"95'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268577+0000","last_change":"2024-04-25T10:15:51.150229+0000","last_active":"2024-04-25T10:15:53.268577+0000","last_peered":"2024-04-25T10:15:53.268577+0000","last_clean":"2024-04-25T10:15:53.268577+0000","last_became_active":"2024-04-25T10:15:51.149922+0000","last_became_peered":"2024-04-25T10:15:51.149922+0000","last_unstale":"2024-04-25T10:15:53.268577+0000","last_undegraded":"2024-04-25T10:15:53.268577+0000","last_fullsized":"2024-04-25T10:15:53.268577+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:09.396399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268446+0000","last_change":"2024-04-25T10:15:52.163673+0000","last_active":"2024-04-25T10:15:53.268446+0000","last_peered":"2024-04-25T10:15:53.268446+0000","last_clean":"2024-04-25T10:15:53.268446+0000","last_became_active":"2024-04-25T10:15:52.163368+0000","last_became_peered":"2024-04-25T10:15:52.163368+0000","last_unstale":"2024-04-25T10:15:53.268446+0000","last_undegraded":"2024-04-25T10:15:53.268446+0000","last_fullsized":"2024-04-25T10:15:53.268446+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:17.808534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"95'6","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.144635+0000","last_change":"2024-04-25T10:15:51.134615+0000","last_active":"2024-04-25T10:15:53.144635+0000","last_peered":"2024-04-25T10:15:53.144635+0000","last_clean":"2024-04-25T10:15:53.144635+0000","last_became_active":"2024-04-25T10:15:51.133723+0000","last_became_peered":"2024-04-25T10:15:51.133723+0000","last_unstale":"2024-04-25T10:15:53.144635+0000","last_undegraded":"2024-04-25T10:15:53.144635+0000","last_fullsized":"2024-04-25T10:15:53.144635+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:09.396399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269009+0000","last_change":"2024-04-25T10:15:52.149991+0000","last_active":"2024-04-25T10:15:53.269009+0000","last_peered":"2024-04-25T10:15:53.269009+0000","last_clean":"2024-04-25T10:15:53.269009+0000","last_became_active":"2024-04-25T10:15:52.149492+0000","last_became_peered":"2024-04-25T10:15:52.149492+0000","last_unstale":"2024-04-25T10:15:53.269009+0000","last_undegraded":"2024-04-25T10:15:53.269009+0000","last_fullsized":"2024-04-25T10:15:53.269009+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:08.546400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265929+0000","last_change":"2024-04-25T10:15:51.143315+0000","last_active":"2024-04-25T10:15:53.265929+0000","last_peered":"2024-04-25T10:15:53.265929+0000","last_clean":"2024-04-25T10:15:53.265929+0000","last_became_active":"2024-04-25T10:15:51.142814+0000","last_became_peered":"2024-04-25T10:15:51.142814+0000","last_unstale":"2024-04-25T10:15:53.265929+0000","last_undegraded":"2024-04-25T10:15:53.265929+0000","last_fullsized":"2024-04-25T10:15:53.265929+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:51.152565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269208+0000","last_change":"2024-04-25T10:15:52.161587+0000","last_active":"2024-04-25T10:15:53.269208+0000","last_peered":"2024-04-25T10:15:53.269208+0000","last_clean":"2024-04-25T10:15:53.269208+0000","last_became_active":"2024-04-25T10:15:52.156903+0000","last_became_peered":"2024-04-25T10:15:52.156903+0000","last_unstale":"2024-04-25T10:15:53.269208+0000","last_undegraded":"2024-04-25T10:15:53.269208+0000","last_fullsized":"2024-04-25T10:15:53.269208+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:34.374748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266004+0000","last_change":"2024-04-25T10:15:51.146962+0000","last_active":"2024-04-25T10:15:53.266004+0000","last_peered":"2024-04-25T10:15:53.266004+0000","last_clean":"2024-04-25T10:15:53.266004+0000","last_became_active":"2024-04-25T10:15:51.146656+0000","last_became_peered":"2024-04-25T10:15:51.146656+0000","last_unstale":"2024-04-25T10:15:53.266004+0000","last_undegraded":"2024-04-25T10:15:53.266004+0000","last_fullsized":"2024-04-25T10:15:53.266004+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:39.576728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.143283+0000","last_change":"2024-04-25T10:15:52.153788+0000","last_active":"2024-04-25T10:15:53.143283+0000","last_peered":"2024-04-25T10:15:53.143283+0000","last_clean":"2024-04-25T10:15:53.143283+0000","last_became_active":"2024-04-25T10:15:52.153487+0000","last_became_peered":"2024-04-25T10:15:52.153487+0000","last_unstale":"2024-04-25T10:15:53.143283+0000","last_undegraded":"2024-04-25T10:15:53.143283+0000","last_fullsized":"2024-04-25T10:15:53.143283+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:50.901390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.142850+0000","last_change":"2024-04-25T10:15:51.143407+0000","last_active":"2024-04-25T10:15:53.142850+0000","last_peered":"2024-04-25T10:15:53.142850+0000","last_clean":"2024-04-25T10:15:53.142850+0000","last_became_active":"2024-04-25T10:15:51.143090+0000","last_became_peered":"2024-04-25T10:15:51.143090+0000","last_unstale":"2024-04-25T10:15:53.142850+0000","last_undegraded":"2024-04-25T10:15:53.142850+0000","last_fullsized":"2024-04-25T10:15:53.142850+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:10.014703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266102+0000","last_change":"2024-04-25T10:15:52.137618+0000","last_active":"2024-04-25T10:15:53.266102+0000","last_peered":"2024-04-25T10:15:53.266102+0000","last_clean":"2024-04-25T10:15:53.266102+0000","last_became_active":"2024-04-25T10:15:52.136972+0000","last_became_peered":"2024-04-25T10:15:52.136972+0000","last_unstale":"2024-04-25T10:15:53.266102+0000","last_undegraded":"2024-04-25T10:15:53.266102+0000","last_fullsized":"2024-04-25T10:15:53.266102+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:07.387774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"95'8","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267366+0000","last_change":"2024-04-25T10:15:51.146771+0000","last_active":"2024-04-25T10:15:53.267366+0000","last_peered":"2024-04-25T10:15:53.267366+0000","last_clean":"2024-04-25T10:15:53.267366+0000","last_became_active":"2024-04-25T10:15:51.146504+0000","last_became_peered":"2024-04-25T10:15:51.146504+0000","last_unstale":"2024-04-25T10:15:53.267366+0000","last_undegraded":"2024-04-25T10:15:53.267366+0000","last_fullsized":"2024-04-25T10:15:53.267366+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.028499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268538+0000","last_change":"2024-04-25T10:15:52.164645+0000","last_active":"2024-04-25T10:15:53.268538+0000","last_peered":"2024-04-25T10:15:53.268538+0000","last_clean":"2024-04-25T10:15:53.268538+0000","last_became_active":"2024-04-25T10:15:52.164334+0000","last_became_peered":"2024-04-25T10:15:52.164334+0000","last_unstale":"2024-04-25T10:15:53.268538+0000","last_undegraded":"2024-04-25T10:15:53.268538+0000","last_fullsized":"2024-04-25T10:15:53.268538+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:17.808534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.142960+0000","last_change":"2024-04-25T10:15:51.144474+0000","last_active":"2024-04-25T10:15:53.142960+0000","last_peered":"2024-04-25T10:15:53.142960+0000","last_clean":"2024-04-25T10:15:53.142960+0000","last_became_active":"2024-04-25T10:15:51.144143+0000","last_became_peered":"2024-04-25T10:15:51.144143+0000","last_unstale":"2024-04-25T10:15:53.142960+0000","last_undegraded":"2024-04-25T10:15:53.142960+0000","last_fullsized":"2024-04-25T10:15:53.142960+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:51.623284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.143744+0000","last_change":"2024-04-25T10:15:52.159942+0000","last_active":"2024-04-25T10:15:53.143744+0000","last_peered":"2024-04-25T10:15:53.143744+0000","last_clean":"2024-04-25T10:15:53.143744+0000","last_became_active":"2024-04-25T10:15:52.158275+0000","last_became_peered":"2024-04-25T10:15:52.158275+0000","last_unstale":"2024-04-25T10:15:53.143744+0000","last_undegraded":"2024-04-25T10:15:53.143744+0000","last_fullsized":"2024-04-25T10:15:53.143744+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:12.547980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267470+0000","last_change":"2024-04-25T10:15:51.144968+0000","last_active":"2024-04-25T10:15:53.267470+0000","last_peered":"2024-04-25T10:15:53.267470+0000","last_clean":"2024-04-25T10:15:53.267470+0000","last_became_active":"2024-04-25T10:15:51.144599+0000","last_became_peered":"2024-04-25T10:15:51.144599+0000","last_unstale":"2024-04-25T10:15:53.267470+0000","last_undegraded":"2024-04-25T10:15:53.267470+0000","last_fullsized":"2024-04-25T10:15:53.267470+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:10.014703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268466+0000","last_change":"2024-04-25T10:15:52.159908+0000","last_active":"2024-04-25T10:15:53.268466+0000","last_peered":"2024-04-25T10:15:53.268466+0000","last_clean":"2024-04-25T10:15:53.268466+0000","last_became_active":"2024-04-25T10:15:52.159444+0000","last_became_peered":"2024-04-25T10:15:52.159444+0000","last_unstale":"2024-04-25T10:15:53.268466+0000","last_undegraded":"2024-04-25T10:15:53.268466+0000","last_fullsized":"2024-04-25T10:15:53.268466+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:08.546400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.142152+0000","last_change":"2024-04-25T10:15:51.133376+0000","last_active":"2024-04-25T10:15:53.142152+0000","last_peered":"2024-04-25T10:15:53.142152+0000","last_clean":"2024-04-25T10:15:53.142152+0000","last_became_active":"2024-04-25T10:15:51.131804+0000","last_became_peered":"2024-04-25T10:15:51.131804+0000","last_unstale":"2024-04-25T10:15:53.142152+0000","last_undegraded":"2024-04-25T10:15:53.142152+0000","last_fullsized":"2024-04-25T10:15:53.142152+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:51.515019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265880+0000","last_change":"2024-04-25T10:15:52.147913+0000","last_active":"2024-04-25T10:15:53.265880+0000","last_peered":"2024-04-25T10:15:53.265880+0000","last_clean":"2024-04-25T10:15:53.265880+0000","last_became_active":"2024-04-25T10:15:52.147620+0000","last_became_peered":"2024-04-25T10:15:52.147620+0000","last_unstale":"2024-04-25T10:15:53.265880+0000","last_undegraded":"2024-04-25T10:15:53.265880+0000","last_fullsized":"2024-04-25T10:15:53.265880+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:49.205689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140903+0000","last_change":"2024-04-25T10:15:51.142018+0000","last_active":"2024-04-25T10:15:53.140903+0000","last_peered":"2024-04-25T10:15:53.140903+0000","last_clean":"2024-04-25T10:15:53.140903+0000","last_became_active":"2024-04-25T10:15:51.141754+0000","last_became_peered":"2024-04-25T10:15:51.141754+0000","last_unstale":"2024-04-25T10:15:53.140903+0000","last_undegraded":"2024-04-25T10:15:53.140903+0000","last_fullsized":"2024-04-25T10:15:53.140903+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:28.146412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265799+0000","last_change":"2024-04-25T10:15:52.133324+0000","last_active":"2024-04-25T10:15:53.265799+0000","last_peered":"2024-04-25T10:15:53.265799+0000","last_clean":"2024-04-25T10:15:53.265799+0000","last_became_active":"2024-04-25T10:15:52.132878+0000","last_became_peered":"2024-04-25T10:15:52.132878+0000","last_unstale":"2024-04-25T10:15:53.265799+0000","last_undegraded":"2024-04-25T10:15:53.265799+0000","last_fullsized":"2024-04-25T10:15:53.265799+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:09.055243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.143016+0000","last_change":"2024-04-25T10:15:51.147556+0000","last_active":"2024-04-25T10:15:53.143016+0000","last_peered":"2024-04-25T10:15:53.143016+0000","last_clean":"2024-04-25T10:15:53.143016+0000","last_became_active":"2024-04-25T10:15:51.147193+0000","last_became_peered":"2024-04-25T10:15:51.147193+0000","last_unstale":"2024-04-25T10:15:53.143016+0000","last_undegraded":"2024-04-25T10:15:53.143016+0000","last_fullsized":"2024-04-25T10:15:53.143016+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:59.944275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265029+0000","last_change":"2024-04-25T10:15:52.138728+0000","last_active":"2024-04-25T10:15:53.265029+0000","last_peered":"2024-04-25T10:15:53.265029+0000","last_clean":"2024-04-25T10:15:53.265029+0000","last_became_active":"2024-04-25T10:15:52.138460+0000","last_became_peered":"2024-04-25T10:15:52.138460+0000","last_unstale":"2024-04-25T10:15:53.265029+0000","last_undegraded":"2024-04-25T10:15:53.265029+0000","last_fullsized":"2024-04-25T10:15:53.265029+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:08.369733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141738+0000","last_change":"2024-04-25T10:15:51.138085+0000","last_active":"2024-04-25T10:15:53.141738+0000","last_peered":"2024-04-25T10:15:53.141738+0000","last_clean":"2024-04-25T10:15:53.141738+0000","last_became_active":"2024-04-25T10:15:51.137148+0000","last_became_peered":"2024-04-25T10:15:51.137148+0000","last_unstale":"2024-04-25T10:15:53.141738+0000","last_undegraded":"2024-04-25T10:15:53.141738+0000","last_fullsized":"2024-04-25T10:15:53.141738+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:06.786023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266655+0000","last_change":"2024-04-25T10:15:52.167093+0000","last_active":"2024-04-25T10:15:53.266655+0000","last_peered":"2024-04-25T10:15:53.266655+0000","last_clean":"2024-04-25T10:15:53.266655+0000","last_became_active":"2024-04-25T10:15:52.166750+0000","last_became_peered":"2024-04-25T10:15:52.166750+0000","last_unstale":"2024-04-25T10:15:53.266655+0000","last_undegraded":"2024-04-25T10:15:53.266655+0000","last_fullsized":"2024-04-25T10:15:53.266655+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:50.901390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140396+0000","last_change":"2024-04-25T10:15:51.149803+0000","last_active":"2024-04-25T10:15:53.140396+0000","last_peered":"2024-04-25T10:15:53.140396+0000","last_clean":"2024-04-25T10:15:53.140396+0000","last_became_active":"2024-04-25T10:15:51.149517+0000","last_became_peered":"2024-04-25T10:15:51.149517+0000","last_unstale":"2024-04-25T10:15:53.140396+0000","last_undegraded":"2024-04-25T10:15:53.140396+0000","last_fullsized":"2024-04-25T10:15:53.140396+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:21.568129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141986+0000","last_change":"2024-04-25T10:15:52.154948+0000","last_active":"2024-04-25T10:15:53.141986+0000","last_peered":"2024-04-25T10:15:53.141986+0000","last_clean":"2024-04-25T10:15:53.141986+0000","last_became_active":"2024-04-25T10:15:52.152707+0000","last_became_peered":"2024-04-25T10:15:52.152707+0000","last_unstale":"2024-04-25T10:15:53.141986+0000","last_undegraded":"2024-04-25T10:15:53.141986+0000","last_fullsized":"2024-04-25T10:15:53.141986+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:39.025309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.145174+0000","last_change":"2024-04-25T10:15:51.141768+0000","last_active":"2024-04-25T10:15:53.145174+0000","last_peered":"2024-04-25T10:15:53.145174+0000","last_clean":"2024-04-25T10:15:53.145174+0000","last_became_active":"2024-04-25T10:15:51.140502+0000","last_became_peered":"2024-04-25T10:15:51.140502+0000","last_unstale":"2024-04-25T10:15:53.145174+0000","last_undegraded":"2024-04-25T10:15:53.145174+0000","last_fullsized":"2024-04-25T10:15:53.145174+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:33.357854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269597+0000","last_change":"2024-04-25T10:15:52.158587+0000","last_active":"2024-04-25T10:15:53.269597+0000","last_peered":"2024-04-25T10:15:53.269597+0000","last_clean":"2024-04-25T10:15:53.269597+0000","last_became_active":"2024-04-25T10:15:52.158300+0000","last_became_peered":"2024-04-25T10:15:52.158300+0000","last_unstale":"2024-04-25T10:15:53.269597+0000","last_undegraded":"2024-04-25T10:15:53.269597+0000","last_fullsized":"2024-04-25T10:15:53.269597+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:35.672483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141933+0000","last_change":"2024-04-25T10:15:51.138821+0000","last_active":"2024-04-25T10:15:53.141933+0000","last_peered":"2024-04-25T10:15:53.141933+0000","last_clean":"2024-04-25T10:15:53.141933+0000","last_became_active":"2024-04-25T10:15:51.138377+0000","last_became_peered":"2024-04-25T10:15:51.138377+0000","last_unstale":"2024-04-25T10:15:53.141933+0000","last_undegraded":"2024-04-25T10:15:53.141933+0000","last_fullsized":"2024-04-25T10:15:53.141933+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:43.943728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.269276+0000","last_change":"2024-04-25T10:15:52.163237+0000","last_active":"2024-04-25T10:15:53.269276+0000","last_peered":"2024-04-25T10:15:53.269276+0000","last_clean":"2024-04-25T10:15:53.269276+0000","last_became_active":"2024-04-25T10:15:52.162927+0000","last_became_peered":"2024-04-25T10:15:52.162927+0000","last_unstale":"2024-04-25T10:15:53.269276+0000","last_undegraded":"2024-04-25T10:15:53.269276+0000","last_fullsized":"2024-04-25T10:15:53.269276+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.961168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.140984+0000","last_change":"2024-04-25T10:15:51.140544+0000","last_active":"2024-04-25T10:15:53.140984+0000","last_peered":"2024-04-25T10:15:53.140984+0000","last_clean":"2024-04-25T10:15:53.140984+0000","last_became_active":"2024-04-25T10:15:51.139959+0000","last_became_peered":"2024-04-25T10:15:51.139959+0000","last_unstale":"2024-04-25T10:15:53.140984+0000","last_undegraded":"2024-04-25T10:15:53.140984+0000","last_fullsized":"2024-04-25T10:15:53.140984+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:46.677957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.268506+0000","last_change":"2024-04-25T10:15:52.164610+0000","last_active":"2024-04-25T10:15:53.268506+0000","last_peered":"2024-04-25T10:15:53.268506+0000","last_clean":"2024-04-25T10:15:53.268506+0000","last_became_active":"2024-04-25T10:15:52.164270+0000","last_became_peered":"2024-04-25T10:15:52.164270+0000","last_unstale":"2024-04-25T10:15:53.268506+0000","last_undegraded":"2024-04-25T10:15:53.268506+0000","last_fullsized":"2024-04-25T10:15:53.268506+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:13.450500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267977+0000","last_change":"2024-04-25T10:15:51.149153+0000","last_active":"2024-04-25T10:15:53.267977+0000","last_peered":"2024-04-25T10:15:53.267977+0000","last_clean":"2024-04-25T10:15:53.267977+0000","last_became_active":"2024-04-25T10:15:51.147849+0000","last_became_peered":"2024-04-25T10:15:51.147849+0000","last_unstale":"2024-04-25T10:15:53.267977+0000","last_undegraded":"2024-04-25T10:15:53.267977+0000","last_fullsized":"2024-04-25T10:15:53.267977+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:51.623284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265963+0000","last_change":"2024-04-25T10:15:52.143240+0000","last_active":"2024-04-25T10:15:53.265963+0000","last_peered":"2024-04-25T10:15:53.265963+0000","last_clean":"2024-04-25T10:15:53.265963+0000","last_became_active":"2024-04-25T10:15:52.142969+0000","last_became_peered":"2024-04-25T10:15:52.142969+0000","last_unstale":"2024-04-25T10:15:53.265963+0000","last_undegraded":"2024-04-25T10:15:53.265963+0000","last_fullsized":"2024-04-25T10:15:53.265963+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:03.086647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.141191+0000","last_change":"2024-04-25T10:15:51.133337+0000","last_active":"2024-04-25T10:15:53.141191+0000","last_peered":"2024-04-25T10:15:53.141191+0000","last_clean":"2024-04-25T10:15:53.141191+0000","last_became_active":"2024-04-25T10:15:51.133007+0000","last_became_peered":"2024-04-25T10:15:51.133007+0000","last_unstale":"2024-04-25T10:15:53.141191+0000","last_undegraded":"2024-04-25T10:15:53.141191+0000","last_fullsized":"2024-04-25T10:15:53.141191+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:19.842248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.143685+0000","last_change":"2024-04-25T10:15:52.159727+0000","last_active":"2024-04-25T10:15:53.143685+0000","last_peered":"2024-04-25T10:15:53.143685+0000","last_clean":"2024-04-25T10:15:53.143685+0000","last_became_active":"2024-04-25T10:15:52.159436+0000","last_became_peered":"2024-04-25T10:15:52.159436+0000","last_unstale":"2024-04-25T10:15:53.143685+0000","last_undegraded":"2024-04-25T10:15:53.143685+0000","last_fullsized":"2024-04-25T10:15:53.143685+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:44.960622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266254+0000","last_change":"2024-04-25T10:15:51.149876+0000","last_active":"2024-04-25T10:15:53.266254+0000","last_peered":"2024-04-25T10:15:53.266254+0000","last_clean":"2024-04-25T10:15:53.266254+0000","last_became_active":"2024-04-25T10:15:51.149413+0000","last_became_peered":"2024-04-25T10:15:51.149413+0000","last_unstale":"2024-04-25T10:15:53.266254+0000","last_undegraded":"2024-04-25T10:15:53.266254+0000","last_fullsized":"2024-04-25T10:15:53.266254+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:11.531087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265276+0000","last_change":"2024-04-25T10:15:52.137308+0000","last_active":"2024-04-25T10:15:53.265276+0000","last_peered":"2024-04-25T10:15:53.265276+0000","last_clean":"2024-04-25T10:15:53.265276+0000","last_became_active":"2024-04-25T10:15:52.136959+0000","last_became_peered":"2024-04-25T10:15:52.136959+0000","last_unstale":"2024-04-25T10:15:53.265276+0000","last_undegraded":"2024-04-25T10:15:53.265276+0000","last_fullsized":"2024-04-25T10:15:53.265276+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:31.190720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.270058+0000","last_change":"2024-04-25T10:15:51.147246+0000","last_active":"2024-04-25T10:15:53.270058+0000","last_peered":"2024-04-25T10:15:53.270058+0000","last_clean":"2024-04-25T10:15:53.270058+0000","last_became_active":"2024-04-25T10:15:51.146971+0000","last_became_peered":"2024-04-25T10:15:51.146971+0000","last_unstale":"2024-04-25T10:15:53.270058+0000","last_undegraded":"2024-04-25T10:15:53.270058+0000","last_fullsized":"2024-04-25T10:15:53.270058+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:28.146412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266608+0000","last_change":"2024-04-25T10:15:52.163066+0000","last_active":"2024-04-25T10:15:53.266608+0000","last_peered":"2024-04-25T10:15:53.266608+0000","last_clean":"2024-04-25T10:15:53.266608+0000","last_became_active":"2024-04-25T10:15:52.162215+0000","last_became_peered":"2024-04-25T10:15:52.162215+0000","last_unstale":"2024-04-25T10:15:53.266608+0000","last_undegraded":"2024-04-25T10:15:53.266608+0000","last_fullsized":"2024-04-25T10:15:53.266608+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.667915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.265099+0000","last_change":"2024-04-25T10:15:51.138274+0000","last_active":"2024-04-25T10:15:53.265099+0000","last_peered":"2024-04-25T10:15:53.265099+0000","last_clean":"2024-04-25T10:15:53.265099+0000","last_became_active":"2024-04-25T10:15:51.137961+0000","last_became_peered":"2024-04-25T10:15:51.137961+0000","last_unstale":"2024-04-25T10:15:53.265099+0000","last_undegraded":"2024-04-25T10:15:53.265099+0000","last_fullsized":"2024-04-25T10:15:53.265099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:09.964287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267498+0000","last_change":"2024-04-25T10:15:52.141968+0000","last_active":"2024-04-25T10:15:53.267498+0000","last_peered":"2024-04-25T10:15:53.267498+0000","last_clean":"2024-04-25T10:15:53.267498+0000","last_became_active":"2024-04-25T10:15:52.141646+0000","last_became_peered":"2024-04-25T10:15:52.141646+0000","last_unstale":"2024-04-25T10:15:53.267498+0000","last_undegraded":"2024-04-25T10:15:53.267498+0000","last_fullsized":"2024-04-25T10:15:53.267498+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:51.098522+0000","last_clean_scrub_stamp":"2024-04-25T10:15:51.098522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:33.603132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.267881+0000","last_change":"2024-04-25T10:15:51.145805+0000","last_active":"2024-04-25T10:15:53.267881+0000","last_peered":"2024-04-25T10:15:53.267881+0000","last_clean":"2024-04-25T10:15:53.267881+0000","last_became_active":"2024-04-25T10:15:51.145479+0000","last_became_peered":"2024-04-25T10:15:51.145479+0000","last_unstale":"2024-04-25T10:15:53.267881+0000","last_undegraded":"2024-04-25T10:15:53.267881+0000","last_fullsized":"2024-04-25T10:15:53.267881+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:50.081629+0000","last_clean_scrub_stamp":"2024-04-25T10:15:50.081629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:11.531087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":314,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-04-25T10:15:53.266874+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:15:53.266874+0000","last_peered":"2024-04-25T10:15:53.266874+0000","last_clean":"2024-04-25T10:15:53.266874+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:15:53.266874+0000","last_undegraded":"2024-04-25T10:15:53.266874+0000","last_fullsized":"2024-04-25T10:15:53.266874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574959,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1789952,"data_stored":1102347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574960,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":512519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574959,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":511937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574960,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1208320,"data_stored":513603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574959,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":513501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574960,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":511901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1187840,"data_stored":511867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574959,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1785856,"data_stored":1102817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:57.024 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:15:57.024 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:15:57.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.274+0000 7f4d2e000640 1 Processor -- start 2024-04-25T10:15:57.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.275+0000 7f4d2e000640 1 -- start start 2024-04-25T10:15:57.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.276+0000 7f4d2e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d28144d00 0x7f4d2806d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.276+0000 7f4d2e000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d2815fdd0 con 0x7f4d280cf620 2024-04-25T10:15:57.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.276+0000 7f4d2e000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d281600b0 con 0x7f4d28144550 2024-04-25T10:15:57.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.276+0000 7f4d2e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d28160390 con 0x7f4d28144d00 2024-04-25T10:15:57.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.277+0000 7f4d27e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4d280cf620 0x7f4d280cc6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57412/0 (socket says 172.21.15.119:57412) 2024-04-25T10:15:57.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.277+0000 7f4d27e00640 1 -- 172.21.15.119:0/2577879591 learned_addr learned my addr 172.21.15.119:0/2577879591 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:57.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.277+0000 7f4d26a00640 1 --2- 172.21.15.119:0/2577879591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d28144d00 0x7f4d2806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.277+0000 7f4d26000640 1 -- 172.21.15.119:0/2577879591 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877239402 0 0) 0x7f4d2815fdd0 con 0x7f4d280cf620 2024-04-25T10:15:57.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.277+0000 7f4d26000640 1 -- 172.21.15.119:0/2577879591 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d0c003520 con 0x7f4d280cf620 2024-04-25T10:15:57.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d26000640 1 -- 172.21.15.119:0/2577879591 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2911496981 0 0) 0x7f4d0c003520 con 0x7f4d280cf620 2024-04-25T10:15:57.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d26000640 1 -- 172.21.15.119:0/2577879591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d28144d00 msgr2=0x7f4d2806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d26000640 1 --2- 172.21.15.119:0/2577879591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d28144d00 0x7f4d2806d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d26000640 1 -- 172.21.15.119:0/2577879591 >> v1:172.21.15.132:6789/0 conn(0x7f4d28144550 legacy=0x7f4d2815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d26000640 1 -- 172.21.15.119:0/2577879591 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d280cbeb0 con 0x7f4d280cf620 2024-04-25T10:15:57.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d26000640 1 -- 172.21.15.119:0/2577879591 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f4d200019f0 con 0x7f4d280cf620 2024-04-25T10:15:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d26000640 1 -- 172.21.15.119:0/2577879591 >> v1:172.21.15.119:6789/0 conn(0x7f4d280cf620 legacy=0x7f4d280cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d26000640 1 --2- 172.21.15.119:0/2577879591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d28144d00 0x7f4d0c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d26000640 1 --2- 172.21.15.119:0/2577879591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d0c003f60 0x7f4d0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d26000640 1 --2- 172.21.15.119:0/2577879591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d0c004d40 0x7f4d0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d26000640 1 -- 172.21.15.119:0/2577879591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d28160390 con 0x7f4d0c004d40 2024-04-25T10:15:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d26000640 1 -- 172.21.15.119:0/2577879591 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c00baa0 con 0x7f4d0c003f60 2024-04-25T10:15:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d26000640 1 -- 172.21.15.119:0/2577879591 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c00bca0 con 0x7f4d28144d00 2024-04-25T10:15:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d27e00640 1 --2- 172.21.15.119:0/2577879591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d0c004d40 0x7f4d0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.278+0000 7f4d26a00640 1 --2- 172.21.15.119:0/2577879591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d0c003f60 0x7f4d0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.279+0000 7f4d27400640 1 --2- 172.21.15.119:0/2577879591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d28144d00 0x7f4d0c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.279+0000 7f4d27e00640 1 -- 172.21.15.119:0/2577879591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d28144d00 msgr2=0x7f4d0c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.279+0000 7f4d27e00640 1 --2- 172.21.15.119:0/2577879591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d28144d00 0x7f4d0c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.279+0000 7f4d27e00640 1 -- 172.21.15.119:0/2577879591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d0c003f60 msgr2=0x7f4d0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.279+0000 7f4d27e00640 1 --2- 172.21.15.119:0/2577879591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d0c003f60 0x7f4d0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.279+0000 7f4d27e00640 1 -- 172.21.15.119:0/2577879591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c003520 con 0x7f4d0c004d40 2024-04-25T10:15:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.280+0000 7f4d27e00640 1 --2- 172.21.15.119:0/2577879591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d0c004d40 0x7f4d0c0051c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000a840 tx=0x7f4d0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1336139139a571eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.280+0000 7f4d26000640 1 -- 172.21.15.119:0/2577879591 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2000a460 con 0x7f4d0c004d40 2024-04-25T10:15:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.280+0000 7f4d26000640 1 -- 172.21.15.119:0/2577879591 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2002f070 con 0x7f4d0c004d40 2024-04-25T10:15:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.280+0000 7f4d26000640 1 -- 172.21.15.119:0/2577879591 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2002bcf0 con 0x7f4d0c004d40 2024-04-25T10:15:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.281+0000 7f4d2e000640 1 -- 172.21.15.119:0/2577879591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d0c004d40 msgr2=0x7f4d0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.281+0000 7f4d2e000640 1 --2- 172.21.15.119:0/2577879591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d0c004d40 0x7f4d0c0051c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000a840 tx=0x7f4d0c005830 comp rx=0 tx=0).stop 2024-04-25T10:15:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.281+0000 7f4d27e00640 1 -- 172.21.15.119:0/2577879591 reap_dead start 2024-04-25T10:15:57.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.281+0000 7f4d2e000640 1 -- 172.21.15.119:0/2577879591 shutdown_connections 2024-04-25T10:15:57.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.281+0000 7f4d2e000640 1 -- 172.21.15.119:0/2577879591 >> 172.21.15.119:0/2577879591 conn(0x7f4d2806aba0 msgr2=0x7f4d2806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.281+0000 7f4d2e000640 1 -- 172.21.15.119:0/2577879591 shutdown_connections 2024-04-25T10:15:57.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.281+0000 7f4d2e000640 1 -- 172.21.15.119:0/2577879591 wait complete. 2024-04-25T10:15:57.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.283+0000 7f4d2e000640 1 Processor -- start 2024-04-25T10:15:57.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.283+0000 7f4d2e000640 1 -- start start 2024-04-25T10:15:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d2e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d0c004d40 0x7f4d28139f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d2e000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d2815eac0 0x7f4d28138820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d2e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d28138d60 0x7f4d281391e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d2e000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d28144ab0 con 0x7f4d0c004d40 2024-04-25T10:15:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d2e000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d28144930 con 0x7f4d2815eac0 2024-04-25T10:15:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d2e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2806d590 con 0x7f4d28138d60 2024-04-25T10:15:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d27400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d0c004d40 0x7f4d28139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d27400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d0c004d40 0x7f4d28139f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46354/0 (socket says 172.21.15.119:46354) 2024-04-25T10:15:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d27400640 1 -- 172.21.15.119:0/819269870 learned_addr learned my addr 172.21.15.119:0/819269870 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d26a00640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d2815eac0 0x7f4d28138820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d27e00640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d28138d60 0x7f4d281391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d27400640 1 -- 172.21.15.119:0/819269870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d28138d60 msgr2=0x7f4d281391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d27400640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d28138d60 0x7f4d281391e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d27400640 1 -- 172.21.15.119:0/819269870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d2815eac0 msgr2=0x7f4d28138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d27400640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d2815eac0 0x7f4d28138820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d27400640 1 -- 172.21.15.119:0/819269870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2813a490 con 0x7f4d0c004d40 2024-04-25T10:15:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.285+0000 7f4d26a00640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d2815eac0 0x7f4d28138820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.286+0000 7f4d27400640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d0c004d40 0x7f4d28139f50 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800e550 tx=0x7f4d1800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.286+0000 7f4d1f400640 1 -- 172.21.15.119:0/819269870 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d180052e0 con 0x7f4d0c004d40 2024-04-25T10:15:57.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.286+0000 7f4d1f400640 1 -- 172.21.15.119:0/819269870 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d18018070 con 0x7f4d0c004d40 2024-04-25T10:15:57.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.286+0000 7f4d2e000640 1 -- 172.21.15.119:0/819269870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2813a740 con 0x7f4d0c004d40 2024-04-25T10:15:57.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.286+0000 7f4d1f400640 1 -- 172.21.15.119:0/819269870 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d18003810 con 0x7f4d0c004d40 2024-04-25T10:15:57.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.286+0000 7f4d2e000640 1 -- 172.21.15.119:0/819269870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2813ad00 con 0x7f4d0c004d40 2024-04-25T10:15:57.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.287+0000 7f4d2e000640 1 -- 172.21.15.119:0/819269870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d280749b0 con 0x7f4d0c004d40 2024-04-25T10:15:57.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.288+0000 7f4d1f400640 1 -- 172.21.15.119:0/819269870 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d180039b0 con 0x7f4d0c004d40 2024-04-25T10:15:57.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.289+0000 7f4d1f400640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d0806b500 0x7f4d0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.289+0000 7f4d1f400640 1 -- 172.21.15.119:0/819269870 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d18089e40 con 0x7f4d0c004d40 2024-04-25T10:15:57.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.289+0000 7f4d26a00640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d0806b500 0x7f4d0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.290+0000 7f4d26a00640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d0806b500 0x7f4d0806d9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4d2813ee20 tx=0x7f4d14008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.294+0000 7f4d1f400640 1 -- 172.21.15.119:0/819269870 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d18058cb0 con 0x7f4d0c004d40 2024-04-25T10:15:57.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.554+0000 7f4d2e000640 1 -- 172.21.15.119:0/819269870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4d2815f310 con 0x7f4d0c004d40 2024-04-25T10:15:57.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.559+0000 7f4d1f400640 1 -- 172.21.15.119:0/819269870 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4d180585b0 con 0x7f4d0c004d40 2024-04-25T10:15:57.564 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-04-25T10:15:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA== 2024-04-25T10:15:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: caps mds = "allow" 2024-04-25T10:15:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: caps mon = "allow r" 2024-04-25T10:15:57.564 INFO:teuthology.orchestra.run.smithi119.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:15:57.564 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2024-04-25T10:15:57.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.562+0000 7f4d2e000640 1 -- 172.21.15.119:0/819269870 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d0806b500 msgr2=0x7f4d0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.562+0000 7f4d2e000640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d0806b500 0x7f4d0806d9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4d2813ee20 tx=0x7f4d14008040 comp rx=0 tx=0).stop 2024-04-25T10:15:57.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.562+0000 7f4d2e000640 1 -- 172.21.15.119:0/819269870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d0c004d40 msgr2=0x7f4d28139f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.562+0000 7f4d2e000640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d0c004d40 0x7f4d28139f50 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800e550 tx=0x7f4d1800ea10 comp rx=0 tx=0).stop 2024-04-25T10:15:57.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.562+0000 7f4d2e000640 1 -- 172.21.15.119:0/819269870 shutdown_connections 2024-04-25T10:15:57.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.562+0000 7f4d2e000640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4d0806b500 0x7f4d0806d9c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.562+0000 7f4d2e000640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d28138d60 0x7f4d281391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.562+0000 7f4d2e000640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d2815eac0 0x7f4d28138820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.562+0000 7f4d2e000640 1 --2- 172.21.15.119:0/819269870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d0c004d40 0x7f4d28139f50 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.562+0000 7f4d2e000640 1 -- 172.21.15.119:0/819269870 >> 172.21.15.119:0/819269870 conn(0x7f4d2806aba0 msgr2=0x7f4d2815b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.563+0000 7f4d2e000640 1 -- 172.21.15.119:0/819269870 shutdown_connections 2024-04-25T10:15:57.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.563+0000 7f4d2e000640 1 -- 172.21.15.119:0/819269870 wait complete. 2024-04-25T10:15:57.585 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:57.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.775+0000 7f32e1e00640 1 Processor -- start 2024-04-25T10:15:57.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.776+0000 7f32e1e00640 1 -- start start 2024-04-25T10:15:57.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.777+0000 7f32e1e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc0d0050 0x7f32dc168960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.777+0000 7f32e1e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32dc0cd930 con 0x7f32dc0cd320 2024-04-25T10:15:57.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.777+0000 7f32e1e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32dc0cdc10 con 0x7f32dc15df40 2024-04-25T10:15:57.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.777+0000 7f32e1e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc0cc5b0 con 0x7f32dc0d0050 2024-04-25T10:15:57.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.777+0000 7f32da000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f32dc0cd320 0x7f32dc0cf940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57416/0 (socket says 172.21.15.119:57416) 2024-04-25T10:15:57.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.777+0000 7f32da000640 1 -- 172.21.15.119:0/1554301563 learned_addr learned my addr 172.21.15.119:0/1554301563 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:57.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.777+0000 7f32d9600640 1 -- 172.21.15.119:0/1554301563 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034277741 0 0) 0x7f32dc0cd930 con 0x7f32dc0cd320 2024-04-25T10:15:57.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32db400640 1 --2- 172.21.15.119:0/1554301563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc0d0050 0x7f32dc168960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32d9600640 1 -- 172.21.15.119:0/1554301563 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32c0003520 con 0x7f32dc0cd320 2024-04-25T10:15:57.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32d9600640 1 -- 172.21.15.119:0/1554301563 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3000408405 0 0) 0x7f32c0003520 con 0x7f32dc0cd320 2024-04-25T10:15:57.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32d9600640 1 -- 172.21.15.119:0/1554301563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc0d0050 msgr2=0x7f32dc168960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32d9600640 1 --2- 172.21.15.119:0/1554301563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc0d0050 0x7f32dc168960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32d9600640 1 -- 172.21.15.119:0/1554301563 >> v1:172.21.15.132:6789/0 conn(0x7f32dc15df40 legacy=0x7f32dc0b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32d9600640 1 -- 172.21.15.119:0/1554301563 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc0cdef0 con 0x7f32dc0cd320 2024-04-25T10:15:57.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32d9600640 1 -- 172.21.15.119:0/1554301563 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f32c40019f0 con 0x7f32dc0cd320 2024-04-25T10:15:57.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32d9600640 1 -- 172.21.15.119:0/1554301563 >> v1:172.21.15.119:6789/0 conn(0x7f32dc0cd320 legacy=0x7f32dc0cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32d9600640 1 --2- 172.21.15.119:0/1554301563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc0d0050 0x7f32c00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32d9600640 1 --2- 172.21.15.119:0/1554301563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c0004060 0x7f32c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32d9600640 1 --2- 172.21.15.119:0/1554301563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c0004d40 0x7f32c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32d9600640 1 -- 172.21.15.119:0/1554301563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc0cc5b0 con 0x7f32c0004060 2024-04-25T10:15:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32d9600640 1 -- 172.21.15.119:0/1554301563 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c000bca0 con 0x7f32c0004d40 2024-04-25T10:15:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32d9600640 1 -- 172.21.15.119:0/1554301563 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c000bea0 con 0x7f32dc0d0050 2024-04-25T10:15:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.778+0000 7f32da000640 1 --2- 172.21.15.119:0/1554301563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c0004060 0x7f32c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.779+0000 7f32db400640 1 --2- 172.21.15.119:0/1554301563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c0004d40 0x7f32c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.779+0000 7f32daa00640 1 --2- 172.21.15.119:0/1554301563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc0d0050 0x7f32c00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.779+0000 7f32da000640 1 -- 172.21.15.119:0/1554301563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc0d0050 msgr2=0x7f32c00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.779+0000 7f32da000640 1 --2- 172.21.15.119:0/1554301563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc0d0050 0x7f32c00039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.779+0000 7f32da000640 1 -- 172.21.15.119:0/1554301563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c0004d40 msgr2=0x7f32c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.779+0000 7f32da000640 1 --2- 172.21.15.119:0/1554301563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c0004d40 0x7f32c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.779+0000 7f32da000640 1 -- 172.21.15.119:0/1554301563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c0003520 con 0x7f32c0004060 2024-04-25T10:15:57.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.780+0000 7f32da000640 1 --2- 172.21.15.119:0/1554301563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c0004060 0x7f32c0004800 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f32c400a7b0 tx=0x7f32c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b5d538b0ac287a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.780+0000 7f32d9600640 1 -- 172.21.15.119:0/1554301563 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c400a3d0 con 0x7f32c0004060 2024-04-25T10:15:57.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.780+0000 7f32d9600640 1 -- 172.21.15.119:0/1554301563 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c402f070 con 0x7f32c0004060 2024-04-25T10:15:57.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.780+0000 7f32d9600640 1 -- 172.21.15.119:0/1554301563 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c402bc80 con 0x7f32c0004060 2024-04-25T10:15:57.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.781+0000 7f32e1e00640 1 -- 172.21.15.119:0/1554301563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c0004060 msgr2=0x7f32c0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.781+0000 7f32e1e00640 1 --2- 172.21.15.119:0/1554301563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32c0004060 0x7f32c0004800 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f32c400a7b0 tx=0x7f32c0005830 comp rx=0 tx=0).stop 2024-04-25T10:15:57.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.781+0000 7f32db400640 1 -- 172.21.15.119:0/1554301563 reap_dead start 2024-04-25T10:15:57.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.781+0000 7f32e1e00640 1 -- 172.21.15.119:0/1554301563 shutdown_connections 2024-04-25T10:15:57.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.781+0000 7f32e1e00640 1 -- 172.21.15.119:0/1554301563 >> 172.21.15.119:0/1554301563 conn(0x7f32dc0d06b0 msgr2=0x7f32dc0d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.781+0000 7f32e1e00640 1 -- 172.21.15.119:0/1554301563 shutdown_connections 2024-04-25T10:15:57.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.781+0000 7f32e1e00640 1 -- 172.21.15.119:0/1554301563 wait complete. 2024-04-25T10:15:57.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.783+0000 7f32e1e00640 1 Processor -- start 2024-04-25T10:15:57.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.783+0000 7f32e1e00640 1 -- start start 2024-04-25T10:15:57.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.784+0000 7f32e1e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c0004060 0x7f32dc1f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.784+0000 7f32e1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32dc1454e0 0x7f32dc152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.784+0000 7f32e1e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc152a60 0x7f32dc0b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.784+0000 7f32e1e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc0c4200 con 0x7f32dc1454e0 2024-04-25T10:15:57.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.784+0000 7f32e1e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc0cb080 con 0x7f32c0004060 2024-04-25T10:15:57.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.784+0000 7f32e1e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc0c4380 con 0x7f32dc152a60 2024-04-25T10:15:57.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.784+0000 7f32da000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32dc1454e0 0x7f32dc152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.784+0000 7f32da000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32dc1454e0 0x7f32dc152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46378/0 (socket says 172.21.15.119:46378) 2024-04-25T10:15:57.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.784+0000 7f32da000640 1 -- 172.21.15.119:0/1547013344 learned_addr learned my addr 172.21.15.119:0/1547013344 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:57.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.784+0000 7f32daa00640 1 --2- 172.21.15.119:0/1547013344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c0004060 0x7f32dc1f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.784+0000 7f32da000640 1 -- 172.21.15.119:0/1547013344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc152a60 msgr2=0x7f32dc0b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.784+0000 7f32da000640 1 --2- 172.21.15.119:0/1547013344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc152a60 0x7f32dc0b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.785+0000 7f32da000640 1 -- 172.21.15.119:0/1547013344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c0004060 msgr2=0x7f32dc1f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.785+0000 7f32da000640 1 --2- 172.21.15.119:0/1547013344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c0004060 0x7f32dc1f3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.785+0000 7f32da000640 1 -- 172.21.15.119:0/1547013344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc152fa0 con 0x7f32dc1454e0 2024-04-25T10:15:57.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.785+0000 7f32da000640 1 --2- 172.21.15.119:0/1547013344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32dc1454e0 0x7f32dc152520 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f32c400a7b0 tx=0x7f32c4001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.785+0000 7f32d3400640 1 -- 172.21.15.119:0/1547013344 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c4044070 con 0x7f32dc1454e0 2024-04-25T10:15:57.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.785+0000 7f32d3400640 1 -- 172.21.15.119:0/1547013344 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c4034670 con 0x7f32dc1454e0 2024-04-25T10:15:57.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.785+0000 7f32d3400640 1 -- 172.21.15.119:0/1547013344 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c400a3d0 con 0x7f32dc1454e0 2024-04-25T10:15:57.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.785+0000 7f32e1e00640 1 -- 172.21.15.119:0/1547013344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32dc0b49f0 con 0x7f32dc1454e0 2024-04-25T10:15:57.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.785+0000 7f32e1e00640 1 -- 172.21.15.119:0/1547013344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32dc0b4fb0 con 0x7f32dc1454e0 2024-04-25T10:15:57.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.787+0000 7f32e1e00640 1 -- 172.21.15.119:0/1547013344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32dc06edf0 con 0x7f32dc1454e0 2024-04-25T10:15:57.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.788+0000 7f32d3400640 1 -- 172.21.15.119:0/1547013344 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32c4047070 con 0x7f32dc1454e0 2024-04-25T10:15:57.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.788+0000 7f32d3400640 1 --2- 172.21.15.119:0/1547013344 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f32b806b4d0 0x7f32b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.789+0000 7f32d3400640 1 -- 172.21.15.119:0/1547013344 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32c40aa380 con 0x7f32dc1454e0 2024-04-25T10:15:57.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.789+0000 7f32daa00640 1 --2- 172.21.15.119:0/1547013344 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f32b806b4d0 0x7f32b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.789+0000 7f32daa00640 1 --2- 172.21.15.119:0/1547013344 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f32b806b4d0 0x7f32b806d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f32cc001590 tx=0x7f32cc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:57.793+0000 7f32d3400640 1 -- 172.21.15.119:0/1547013344 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32dc06edf0 con 0x7f32dc1454e0 2024-04-25T10:15:58.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.012+0000 7f32e1e00640 1 -- 172.21.15.119:0/1547013344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32dc067610 con 0x7f32dc1454e0 2024-04-25T10:15:58.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.013+0000 7f32d3400640 1 -- 172.21.15.119:0/1547013344 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f32dc067610 con 0x7f32dc1454e0 2024-04-25T10:15:58.017 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:15:58.017 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5459,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3768348114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3768348114},{"type":"v1","addr":"172.21.15.132:6834","nonce":3768348114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":5465,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/543190780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":543190780},{"type":"v1","addr":"172.21.15.132:6835","nonce":543190780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1145229172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1145229172},{"type":"v1","addr":"172.21.15.119:6836","nonce":1145229172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:51.568312+0000","modified":"2024-04-25T10:15:54.633087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1945631445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1945631445},{"type":"v1","addr":"172.21.15.119:6837","nonce":1945631445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:15:58.017 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 82 2024-04-25T10:15:58.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.015+0000 7f32e1e00640 1 -- 172.21.15.119:0/1547013344 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f32b806b4d0 msgr2=0x7f32b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.015+0000 7f32e1e00640 1 --2- 172.21.15.119:0/1547013344 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f32b806b4d0 0x7f32b806d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f32cc001590 tx=0x7f32cc009040 comp rx=0 tx=0).stop 2024-04-25T10:15:58.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.015+0000 7f32e1e00640 1 -- 172.21.15.119:0/1547013344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32dc1454e0 msgr2=0x7f32dc152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.015+0000 7f32e1e00640 1 --2- 172.21.15.119:0/1547013344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32dc1454e0 0x7f32dc152520 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f32c400a7b0 tx=0x7f32c4001cf0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.015+0000 7f32e1e00640 1 -- 172.21.15.119:0/1547013344 shutdown_connections 2024-04-25T10:15:58.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.015+0000 7f32e1e00640 1 --2- 172.21.15.119:0/1547013344 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f32b806b4d0 0x7f32b806d990 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.015+0000 7f32e1e00640 1 --2- 172.21.15.119:0/1547013344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc152a60 0x7f32dc0b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.015+0000 7f32e1e00640 1 --2- 172.21.15.119:0/1547013344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32dc1454e0 0x7f32dc152520 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.015+0000 7f32e1e00640 1 --2- 172.21.15.119:0/1547013344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c0004060 0x7f32dc1f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.015+0000 7f32e1e00640 1 -- 172.21.15.119:0/1547013344 >> 172.21.15.119:0/1547013344 conn(0x7f32dc0d06b0 msgr2=0x7f32dc06eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.016+0000 7f32e1e00640 1 -- 172.21.15.119:0/1547013344 shutdown_connections 2024-04-25T10:15:58.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.016+0000 7f32e1e00640 1 -- 172.21.15.119:0/1547013344 wait complete. 2024-04-25T10:15:58.439 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:15:51.568312+0000', 'modified': '2024-04-25T10:15:54.633087+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8040}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8040': {'gid': 8040, 'name': 'a', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6837/1945631445', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 1945631445}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 1945631445}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:15:58.439 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:15:58.440 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:15:58.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.647+0000 7f52dea00640 1 Processor -- start 2024-04-25T10:15:58.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.647+0000 7f52dea00640 1 -- start start 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.649+0000 7f52dea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52d81450e0 0x7f52d806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.649+0000 7f52dea00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d815f400 con 0x7f52d806b0e0 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.649+0000 7f52dea00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d815f6e0 con 0x7f52d80cc9e0 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.649+0000 7f52dea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d81454c0 con 0x7f52d81450e0 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.649+0000 7f52dcc00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f52d806b0e0 0x7f52d80cc2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57420/0 (socket says 172.21.15.119:57420) 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.649+0000 7f52dcc00640 1 -- 172.21.15.119:0/705199911 learned_addr learned my addr 172.21.15.119:0/705199911 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:15:58.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 -- 172.21.15.119:0/705199911 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983831426 0 0) 0x7f52d815f400 con 0x7f52d806b0e0 2024-04-25T10:15:58.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52dd600640 1 --2- 172.21.15.119:0/705199911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52d81450e0 0x7f52d806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 -- 172.21.15.119:0/705199911 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52c0003520 con 0x7f52d806b0e0 2024-04-25T10:15:58.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 -- 172.21.15.119:0/705199911 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 555063154 0 0) 0x7f52c0003520 con 0x7f52d806b0e0 2024-04-25T10:15:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 -- 172.21.15.119:0/705199911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52d81450e0 msgr2=0x7f52d806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 --2- 172.21.15.119:0/705199911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52d81450e0 0x7f52d806aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 -- 172.21.15.119:0/705199911 >> v1:172.21.15.132:6789/0 conn(0x7f52d80cc9e0 legacy=0x7f52d80b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 -- 172.21.15.119:0/705199911 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d815f9c0 con 0x7f52d806b0e0 2024-04-25T10:15:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 -- 172.21.15.119:0/705199911 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f52c4001670 con 0x7f52d806b0e0 2024-04-25T10:15:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 -- 172.21.15.119:0/705199911 >> v1:172.21.15.119:6789/0 conn(0x7f52d806b0e0 legacy=0x7f52d80cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 --2- 172.21.15.119:0/705199911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c00039e0 0x7f52c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 --2- 172.21.15.119:0/705199911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52c0004800 0x7f52c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 --2- 172.21.15.119:0/705199911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52d81450e0 0x7f52c0005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 -- 172.21.15.119:0/705199911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c000bb50 con 0x7f52c0004800 2024-04-25T10:15:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 -- 172.21.15.119:0/705199911 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c000bd00 con 0x7f52c00039e0 2024-04-25T10:15:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.650+0000 7f52d7e00640 1 -- 172.21.15.119:0/705199911 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c000bf00 con 0x7f52d81450e0 2024-04-25T10:15:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.651+0000 7f52dcc00640 1 --2- 172.21.15.119:0/705199911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52c0004800 0x7f52c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.651+0000 7f52dd600640 1 --2- 172.21.15.119:0/705199911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c00039e0 0x7f52c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.651+0000 7f52dcc00640 1 -- 172.21.15.119:0/705199911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52d81450e0 msgr2=0x7f52c0005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.651+0000 7f52dcc00640 1 --2- 172.21.15.119:0/705199911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52d81450e0 0x7f52c0005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.651+0000 7f52dcc00640 1 -- 172.21.15.119:0/705199911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c00039e0 msgr2=0x7f52c0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.651+0000 7f52dcc00640 1 --2- 172.21.15.119:0/705199911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52c00039e0 0x7f52c0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.651+0000 7f52dcc00640 1 -- 172.21.15.119:0/705199911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c0003520 con 0x7f52c0004800 2024-04-25T10:15:58.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.652+0000 7f52dcc00640 1 --2- 172.21.15.119:0/705199911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52c0004800 0x7f52c0004c80 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f52c400c240 tx=0x7f52c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0551d2c86a666d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.652+0000 7f52d7e00640 1 -- 172.21.15.119:0/705199911 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4007d10 con 0x7f52c0004800 2024-04-25T10:15:58.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.652+0000 7f52d7e00640 1 -- 172.21.15.119:0/705199911 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c4005ad0 con 0x7f52c0004800 2024-04-25T10:15:58.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.652+0000 7f52d7e00640 1 -- 172.21.15.119:0/705199911 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4007070 con 0x7f52c0004800 2024-04-25T10:15:58.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.653+0000 7f52dea00640 1 -- 172.21.15.119:0/705199911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52c0004800 msgr2=0x7f52c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.653+0000 7f52dea00640 1 --2- 172.21.15.119:0/705199911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52c0004800 0x7f52c0004c80 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f52c400c240 tx=0x7f52c0005830 comp rx=0 tx=0).stop 2024-04-25T10:15:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.653+0000 7f52de000640 1 -- 172.21.15.119:0/705199911 reap_dead start 2024-04-25T10:15:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.653+0000 7f52dea00640 1 -- 172.21.15.119:0/705199911 shutdown_connections 2024-04-25T10:15:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.653+0000 7f52dea00640 1 -- 172.21.15.119:0/705199911 >> 172.21.15.119:0/705199911 conn(0x7f52d8066ff0 msgr2=0x7f52d80b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.653+0000 7f52dea00640 1 -- 172.21.15.119:0/705199911 shutdown_connections 2024-04-25T10:15:58.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.653+0000 7f52dea00640 1 -- 172.21.15.119:0/705199911 wait complete. 2024-04-25T10:15:58.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.655+0000 7f52dea00640 1 Processor -- start 2024-04-25T10:15:58.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.655+0000 7f52dea00640 1 -- start start 2024-04-25T10:15:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.656+0000 7f52dea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c0004800 0x7f52d815b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.656+0000 7f52dea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d80c6860 0x7f52d80ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.656+0000 7f52dea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d80adf70 0x7f52d80c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.656+0000 7f52dea00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d806aba0 con 0x7f52d80adf70 2024-04-25T10:15:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.656+0000 7f52dea00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d806ad20 con 0x7f52d80c6860 2024-04-25T10:15:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.656+0000 7f52dea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d806aea0 con 0x7f52c0004800 2024-04-25T10:15:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.656+0000 7f52de000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d80adf70 0x7f52d80c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.656+0000 7f52de000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d80adf70 0x7f52d80c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46396/0 (socket says 172.21.15.119:46396) 2024-04-25T10:15:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.656+0000 7f52de000640 1 -- 172.21.15.119:0/3546365695 learned_addr learned my addr 172.21.15.119:0/3546365695 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.657+0000 7f52dd600640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c0004800 0x7f52d815b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.657+0000 7f52dcc00640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d80c6860 0x7f52d80ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.657+0000 7f52de000640 1 -- 172.21.15.119:0/3546365695 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c0004800 msgr2=0x7f52d815b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.657+0000 7f52de000640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c0004800 0x7f52d815b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.657+0000 7f52de000640 1 -- 172.21.15.119:0/3546365695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d80c6860 msgr2=0x7f52d80ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.657+0000 7f52de000640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d80c6860 0x7f52d80ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.657+0000 7f52de000640 1 -- 172.21.15.119:0/3546365695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d80c57c0 con 0x7f52d80adf70 2024-04-25T10:15:58.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.657+0000 7f52dd600640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c0004800 0x7f52d815b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:58.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.657+0000 7f52de000640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d80adf70 0x7f52d80c4ad0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f52d80cbe60 tx=0x7f52d000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.657+0000 7f52d5600640 1 -- 172.21.15.119:0/3546365695 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0017070 con 0x7f52d80adf70 2024-04-25T10:15:58.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.657+0000 7f52d5600640 1 -- 172.21.15.119:0/3546365695 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d0005d60 con 0x7f52d80adf70 2024-04-25T10:15:58.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.657+0000 7f52d5600640 1 -- 172.21.15.119:0/3546365695 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d000a850 con 0x7f52d80adf70 2024-04-25T10:15:58.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.657+0000 7f52dea00640 1 -- 172.21.15.119:0/3546365695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d80c5aa0 con 0x7f52d80adf70 2024-04-25T10:15:58.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.657+0000 7f52dea00640 1 -- 172.21.15.119:0/3546365695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d80c6060 con 0x7f52d80adf70 2024-04-25T10:15:58.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.659+0000 7f52d5600640 1 -- 172.21.15.119:0/3546365695 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f52d000a9f0 con 0x7f52d80adf70 2024-04-25T10:15:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.660+0000 7f52d5600640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f52b806b4d0 0x7f52b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.660+0000 7f52d5600640 1 -- 172.21.15.119:0/3546365695 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52d00892b0 con 0x7f52d80adf70 2024-04-25T10:15:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.661+0000 7f52dd600640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f52b806b4d0 0x7f52b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.661+0000 7f52e2da2740 1 Processor -- start 2024-04-25T10:15:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.661+0000 7f52e2da2740 1 -- start start 2024-04-25T10:15:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.661+0000 7f52dd600640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f52b806b4d0 0x7f52b806d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009a20 tx=0x7f52cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.662+0000 7f52e2da2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556381975b30 0x55638199d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.662+0000 7f52e2da2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 0x556381975f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.662+0000 7f52e2da2740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563819be7a0 0x5563819742f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.662+0000 7f52e2da2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563818d9620 con 0x556381975b30 2024-04-25T10:15:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.662+0000 7f52e2da2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556381882ca0 con 0x55638199deb0 2024-04-25T10:15:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.662+0000 7f52e2da2740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563818d56c0 con 0x5563819be7a0 2024-04-25T10:15:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.662+0000 7f52dd600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556381975b30 0x55638199d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.662+0000 7f52dd600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556381975b30 0x55638199d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46408/0 (socket says 172.21.15.119:46408) 2024-04-25T10:15:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.662+0000 7f52dd600640 1 -- 172.21.15.119:0/275169897 learned_addr learned my addr 172.21.15.119:0/275169897 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.663+0000 7f52dd600640 1 -- 172.21.15.119:0/275169897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563819be7a0 msgr2=0x5563819742f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.663+0000 7f52de000640 1 --2- 172.21.15.119:0/275169897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563819be7a0 0x5563819742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.663+0000 7f52dcc00640 1 --2- 172.21.15.119:0/275169897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 0x556381975f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.663+0000 7f52dd600640 1 --2- 172.21.15.119:0/275169897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563819be7a0 0x5563819742f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.663+0000 7f52dd600640 1 -- 172.21.15.119:0/275169897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 msgr2=0x556381975f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.663+0000 7f52dd600640 1 --2- 172.21.15.119:0/275169897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 0x556381975f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.663+0000 7f52dd600640 1 -- 172.21.15.119:0/275169897 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556381889f40 con 0x556381975b30 2024-04-25T10:15:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.663+0000 7f52dcc00640 1 --2- 172.21.15.119:0/275169897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 0x556381975f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.663+0000 7f52dd600640 1 --2- 172.21.15.119:0/275169897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556381975b30 0x55638199d970 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f52cc006a70 tx=0x7f52cc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beba2d85e71f8b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.663+0000 7f52c9c00640 1 -- 172.21.15.119:0/275169897 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc022440 con 0x556381975b30 2024-04-25T10:15:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.664+0000 7f52c9c00640 1 -- 172.21.15.119:0/275169897 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52cc027070 con 0x556381975b30 2024-04-25T10:15:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.664+0000 7f52c9c00640 1 -- 172.21.15.119:0/275169897 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0227e0 con 0x556381975b30 2024-04-25T10:15:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.664+0000 7f52e2da2740 1 -- 172.21.15.119:0/275169897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556381975b30 msgr2=0x55638199d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.664+0000 7f52e2da2740 1 --2- 172.21.15.119:0/275169897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556381975b30 0x55638199d970 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f52cc006a70 tx=0x7f52cc015000 comp rx=0 tx=0).stop 2024-04-25T10:15:58.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.664+0000 7f52e2da2740 1 -- 172.21.15.119:0/275169897 shutdown_connections 2024-04-25T10:15:58.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.664+0000 7f52e2da2740 1 --2- 172.21.15.119:0/275169897 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563819be7a0 0x5563819742f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.664+0000 7f52e2da2740 1 --2- 172.21.15.119:0/275169897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 0x556381975f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.664+0000 7f52e2da2740 1 --2- 172.21.15.119:0/275169897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556381975b30 0x55638199d970 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.664+0000 7f52e2da2740 1 -- 172.21.15.119:0/275169897 >> 172.21.15.119:0/275169897 conn(0x556381972d50 msgr2=0x5563819bb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.664+0000 7f52e2da2740 1 -- 172.21.15.119:0/275169897 shutdown_connections 2024-04-25T10:15:58.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.664+0000 7f52e2da2740 1 -- 172.21.15.119:0/275169897 wait complete. 2024-04-25T10:15:58.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.665+0000 7f52e2da2740 1 -- start start 2024-04-25T10:15:58.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.665+0000 7f52e2da2740 1 Processor -- start 2024-04-25T10:15:58.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.666+0000 7f52e2da2740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55638199deb0 0x556381987b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.666+0000 7f52e2da2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x5563819cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.666+0000 7f52e2da2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563819cf1a0 0x55638199bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.666+0000 7f52e2da2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55638199d4e0 con 0x5563819be7a0 2024-04-25T10:15:58.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.666+0000 7f52e2da2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563818781f0 con 0x5563819cf1a0 2024-04-25T10:15:58.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.666+0000 7f52e2da2740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55638199ced0 con 0x55638199deb0 2024-04-25T10:15:58.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.666+0000 7f52e2da2740 10 client.0 _ll_register_callbacks cb 0x556381895ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:58.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.666+0000 7f52dcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x5563819cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52dcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x5563819cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46414/0 (socket says 172.21.15.119:46414) 2024-04-25T10:15:58.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52dcc00640 1 -- 172.21.15.119:0/364746805 learned_addr learned my addr 172.21.15.119:0/364746805 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:58.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52dd600640 1 --2- 172.21.15.119:0/364746805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55638199deb0 0x556381987b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52de000640 1 --2- 172.21.15.119:0/364746805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563819cf1a0 0x55638199bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52dcc00640 1 -- 172.21.15.119:0/364746805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55638199deb0 msgr2=0x556381987b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52dcc00640 1 --2- 172.21.15.119:0/364746805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55638199deb0 0x556381987b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52dcc00640 1 -- 172.21.15.119:0/364746805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563819cf1a0 msgr2=0x55638199bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52dcc00640 1 --2- 172.21.15.119:0/364746805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563819cf1a0 0x55638199bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52dcc00640 1 -- 172.21.15.119:0/364746805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556381897c50 con 0x5563819be7a0 2024-04-25T10:15:58.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52dd600640 1 --2- 172.21.15.119:0/364746805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55638199deb0 0x556381987b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:58.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52dcc00640 1 --2- 172.21.15.119:0/364746805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x5563819cec60 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f52c402ae20 tx=0x7f52c402d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c87e31b08b281f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52be000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:15:58.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52be000640 1 -- 172.21.15.119:0/364746805 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4004bf0 con 0x5563819be7a0 2024-04-25T10:15:58.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52be000640 1 -- 172.21.15.119:0/364746805 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52c4004d90 con 0x5563819be7a0 2024-04-25T10:15:58.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.667+0000 7f52e2da2740 1 -- 172.21.15.119:0/364746805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52cc027070 con 0x5563819be7a0 2024-04-25T10:15:58.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.668+0000 7f52be000640 1 -- 172.21.15.119:0/364746805 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52c4007070 con 0x5563819be7a0 2024-04-25T10:15:58.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.668+0000 7f52e2da2740 10 client.8175 fetch_fsmap learned FSMap version 82 2024-04-25T10:15:58.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.668+0000 7f52e2da2740 1 -- 172.21.15.119:0/364746805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x5563815ee2d0 con 0x5563819be7a0 2024-04-25T10:15:58.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.668+0000 7f52be000640 1 -- 172.21.15.119:0/364746805 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52cc027070 con 0x5563819be7a0 2024-04-25T10:15:58.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.668+0000 7f52be000640 1 -- 172.21.15.119:0/364746805 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f52c4007210 con 0x5563819be7a0 2024-04-25T10:15:58.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.668+0000 7f52e2da2740 10 client.8175 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:15:58.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.668+0000 7f52e2da2740 10 client.8175 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:15:58.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.668+0000 7f52e2da2740 20 client.8175 populate_metadata read hostname 'smithi119' 2024-04-25T10:15:58.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.668+0000 7f52e2da2740 1 --2- 172.21.15.119:0/364746805 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55638199e6d0 0x55638199eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.668+0000 7f52e2da2740 4 client.8175 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:58.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.668+0000 7f52e2da2740 1 -- 172.21.15.119:0/364746805 --> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563818e6b60 con 0x55638199e6d0 2024-04-25T10:15:58.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.669+0000 7f52dd600640 1 --2- 172.21.15.119:0/364746805 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55638199e6d0 0x55638199eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.669+0000 7f52dd600640 1 --2- 172.21.15.119:0/364746805 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55638199e6d0 0x55638199eab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=202ca3dc24c195e5 server_cookie=4d513893e96f6cb0 in_seq=0 out_seq=0 2024-04-25T10:15:58.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.669+0000 7f52be000640 10 client.8175 ms_handle_connect on v2:172.21.15.119:6835/1945631445 2024-04-25T10:15:58.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.670+0000 7f52be000640 1 -- 172.21.15.119:0/364746805 <== mds.0 v2:172.21.15.119:6835/1945631445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5563818e6b60 con 0x55638199e6d0 2024-04-25T10:15:58.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.670+0000 7f52be000640 10 client.8175 handle_command_reply: tid=0 2024-04-25T10:15:58.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.670+0000 7f52e2da2740 1 client.8175 shutdown 2024-04-25T10:15:58.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.670+0000 7f52e2da2740 1 -- 172.21.15.119:0/364746805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 msgr2=0x5563819cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.670+0000 7f52e2da2740 1 --2- 172.21.15.119:0/364746805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x5563819cec60 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f52c402ae20 tx=0x7f52c402d000 comp rx=0 tx=0).stop 2024-04-25T10:15:58.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.670+0000 7f52e2da2740 1 -- 172.21.15.119:0/364746805 shutdown_connections 2024-04-25T10:15:58.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.670+0000 7f52e2da2740 1 --2- 172.21.15.119:0/364746805 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55638199e6d0 0x55638199eab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.670+0000 7f52e2da2740 1 --2- 172.21.15.119:0/364746805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5563819cf1a0 0x55638199bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.670+0000 7f52e2da2740 1 --2- 172.21.15.119:0/364746805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x5563819cec60 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.670+0000 7f52e2da2740 1 --2- 172.21.15.119:0/364746805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55638199deb0 0x556381987b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.670+0000 7f52be000640 0 client.8175 ms_handle_reset on v2:172.21.15.119:6835/1945631445 2024-04-25T10:15:58.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.670+0000 7f52e2da2740 1 -- 172.21.15.119:0/364746805 >> 172.21.15.119:0/364746805 conn(0x556381975b30 msgr2=0x5563819747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.670+0000 7f52e2da2740 1 -- 172.21.15.119:0/364746805 shutdown_connections 2024-04-25T10:15:58.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.670+0000 7f52e2da2740 1 -- 172.21.15.119:0/364746805 wait complete. 2024-04-25T10:15:58.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.671+0000 7f52e2da2740 20 client.8175 trim_cache size 0 max 16384 2024-04-25T10:15:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.688+0000 7f52e2da2740 1 Processor -- start 2024-04-25T10:15:58.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.688+0000 7f52e2da2740 1 -- start start 2024-04-25T10:15:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52e2da2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 0x5563819ba900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52e2da2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x5563819bee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52e2da2740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563819cf1a0 0x5563819bf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52e2da2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d81ff4b0 con 0x5563819be7a0 2024-04-25T10:15:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52e2da2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d815d830 con 0x55638199deb0 2024-04-25T10:15:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52e2da2740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d81ff000 con 0x5563819cf1a0 2024-04-25T10:15:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52dcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x5563819bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52dcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x5563819bee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46420/0 (socket says 172.21.15.119:46420) 2024-04-25T10:15:58.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52dcc00640 1 -- 172.21.15.119:0/3994019064 learned_addr learned my addr 172.21.15.119:0/3994019064 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:58.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52dd600640 1 --2- 172.21.15.119:0/3994019064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 0x5563819ba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52dcc00640 1 -- 172.21.15.119:0/3994019064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563819cf1a0 msgr2=0x5563819bf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52dcc00640 1 --2- 172.21.15.119:0/3994019064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563819cf1a0 0x5563819bf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52dcc00640 1 -- 172.21.15.119:0/3994019064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 msgr2=0x5563819ba900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52dcc00640 1 --2- 172.21.15.119:0/3994019064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 0x5563819ba900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.689+0000 7f52dcc00640 1 -- 172.21.15.119:0/3994019064 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563816b86b0 con 0x5563819be7a0 2024-04-25T10:15:58.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.690+0000 7f52dcc00640 1 --2- 172.21.15.119:0/3994019064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x5563819bee20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f52c4001b90 tx=0x7f52c403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb0152b05bee459 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.690+0000 7f52c9200640 1 -- 172.21.15.119:0/3994019064 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c40049f0 con 0x5563819be7a0 2024-04-25T10:15:58.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.690+0000 7f52c9200640 1 -- 172.21.15.119:0/3994019064 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c4004b90 con 0x5563819be7a0 2024-04-25T10:15:58.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.690+0000 7f52c9200640 1 -- 172.21.15.119:0/3994019064 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4052420 con 0x5563819be7a0 2024-04-25T10:15:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.691+0000 7f52e2da2740 1 -- 172.21.15.119:0/3994019064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 msgr2=0x5563819bee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.691+0000 7f52e2da2740 1 --2- 172.21.15.119:0/3994019064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x5563819bee20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f52c4001b90 tx=0x7f52c403f000 comp rx=0 tx=0).stop 2024-04-25T10:15:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.691+0000 7f52e2da2740 1 -- 172.21.15.119:0/3994019064 shutdown_connections 2024-04-25T10:15:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.691+0000 7f52e2da2740 1 --2- 172.21.15.119:0/3994019064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563819cf1a0 0x5563819bf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.691+0000 7f52e2da2740 1 --2- 172.21.15.119:0/3994019064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x5563819bee20 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.691+0000 7f52e2da2740 1 --2- 172.21.15.119:0/3994019064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 0x5563819ba900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.691+0000 7f52e2da2740 1 -- 172.21.15.119:0/3994019064 >> 172.21.15.119:0/3994019064 conn(0x556381975b30 msgr2=0x556381987980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.691+0000 7f52e2da2740 1 -- 172.21.15.119:0/3994019064 shutdown_connections 2024-04-25T10:15:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.691+0000 7f52e2da2740 1 -- 172.21.15.119:0/3994019064 wait complete. 2024-04-25T10:15:58.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.691+0000 7f52e2da2740 1 -- start start 2024-04-25T10:15:58.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.692+0000 7f52e2da2740 1 Processor -- start 2024-04-25T10:15:58.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52e2da2740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 0x5563819bd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52e2da2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x556381a26100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52e2da2740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563819cf1a0 0x5563819b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52e2da2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55638198b040 con 0x5563819be7a0 2024-04-25T10:15:58.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52e2da2740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55638198a750 con 0x55638199deb0 2024-04-25T10:15:58.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52e2da2740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556381988480 con 0x5563819cf1a0 2024-04-25T10:15:58.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52e2da2740 10 client.0 _ll_register_callbacks cb 0x7f52c00036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:58.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52dcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x556381a26100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52dcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x556381a26100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46430/0 (socket says 172.21.15.119:46430) 2024-04-25T10:15:58.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52dcc00640 1 -- 172.21.15.119:0/2996097235 learned_addr learned my addr 172.21.15.119:0/2996097235 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:58.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52de000640 1 --2- 172.21.15.119:0/2996097235 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563819cf1a0 0x5563819b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52dcc00640 1 -- 172.21.15.119:0/2996097235 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563819cf1a0 msgr2=0x5563819b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52dd600640 1 --2- 172.21.15.119:0/2996097235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 0x5563819bd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52dcc00640 1 --2- 172.21.15.119:0/2996097235 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563819cf1a0 0x5563819b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52dcc00640 1 -- 172.21.15.119:0/2996097235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 msgr2=0x5563819bd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52dcc00640 1 --2- 172.21.15.119:0/2996097235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 0x5563819bd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.693+0000 7f52dcc00640 1 -- 172.21.15.119:0/2996097235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556381a26640 con 0x5563819be7a0 2024-04-25T10:15:58.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.694+0000 7f52dcc00640 1 --2- 172.21.15.119:0/2996097235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x556381a26100 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f52c40078c0 tx=0x7f52c40078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d181ed6e0c2a74c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.694+0000 7f52be000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.694+0000 7f52be000640 1 -- 172.21.15.119:0/2996097235 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c405f060 con 0x5563819be7a0 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.694+0000 7f52be000640 1 -- 172.21.15.119:0/2996097235 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52c4052420 con 0x5563819be7a0 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.694+0000 7f52e2da2740 1 -- 172.21.15.119:0/2996097235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52c4004b90 con 0x5563819be7a0 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.694+0000 7f52be000640 1 -- 172.21.15.119:0/2996097235 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52c4065050 con 0x5563819be7a0 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.694+0000 7f52e2da2740 10 client.8181 fetch_fsmap learned FSMap version 82 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.694+0000 7f52e2da2740 1 -- 172.21.15.119:0/2996097235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x5563819ca9e0 con 0x5563819be7a0 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.694+0000 7f52be000640 1 -- 172.21.15.119:0/2996097235 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c4004b90 con 0x5563819be7a0 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.694+0000 7f52be000640 1 -- 172.21.15.119:0/2996097235 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f52c4068020 con 0x5563819be7a0 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.695+0000 7f52e2da2740 10 client.8181 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.695+0000 7f52e2da2740 10 client.8181 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.695+0000 7f52e2da2740 20 client.8181 populate_metadata read hostname 'smithi119' 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.695+0000 7f52e2da2740 1 --2- 172.21.15.119:0/2996097235 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x556381980540 0x556381980920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.695+0000 7f52e2da2740 4 client.8181 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.695+0000 7f52e2da2740 1 -- 172.21.15.119:0/2996097235 --> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5563818e0eb0 con 0x556381980540 2024-04-25T10:15:58.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.695+0000 7f52dd600640 1 --2- 172.21.15.119:0/2996097235 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x556381980540 0x556381980920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.695+0000 7f52dd600640 1 --2- 172.21.15.119:0/2996097235 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x556381980540 0x556381980920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a23f6779d849c93c server_cookie=c9839c1798ee847c in_seq=0 out_seq=0 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.695+0000 7f52be000640 10 client.8181 ms_handle_connect on v2:172.21.15.119:6835/1945631445 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52be000640 1 -- 172.21.15.119:0/2996097235 <== mds.0 v2:172.21.15.119:6835/1945631445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5563818e0eb0 con 0x556381980540 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52be000640 10 client.8181 handle_command_reply: tid=0 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52e2da2740 1 client.8181 shutdown 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52e2da2740 1 -- 172.21.15.119:0/2996097235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 msgr2=0x556381a26100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52e2da2740 1 --2- 172.21.15.119:0/2996097235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x556381a26100 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f52c40078c0 tx=0x7f52c40078f0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52e2da2740 1 -- 172.21.15.119:0/2996097235 shutdown_connections 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52e2da2740 1 --2- 172.21.15.119:0/2996097235 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x556381980540 0x556381980920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52e2da2740 1 --2- 172.21.15.119:0/2996097235 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5563819cf1a0 0x5563819b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52e2da2740 1 --2- 172.21.15.119:0/2996097235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563819be7a0 0x556381a26100 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52e2da2740 1 --2- 172.21.15.119:0/2996097235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55638199deb0 0x5563819bd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52e2da2740 1 -- 172.21.15.119:0/2996097235 >> 172.21.15.119:0/2996097235 conn(0x556381975b30 msgr2=0x55638196bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52be000640 0 client.8181 ms_handle_reset on v2:172.21.15.119:6835/1945631445 2024-04-25T10:15:58.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52e2da2740 1 -- 172.21.15.119:0/2996097235 shutdown_connections 2024-04-25T10:15:58.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52e2da2740 1 -- 172.21.15.119:0/2996097235 wait complete. 2024-04-25T10:15:58.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.696+0000 7f52e2da2740 20 client.8181 trim_cache size 0 max 16384 2024-04-25T10:15:58.704 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:15:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "e0b0f428-b441-4237-a4d1-ed53321f87a5", 2024-04-25T10:15:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-04-25T10:15:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 8040, 2024-04-25T10:15:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-04-25T10:15:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-04-25T10:15:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-04-25T10:15:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.5786049599999998, 2024-04-25T10:15:58.704 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 82, 2024-04-25T10:15:58.705 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 96, 2024-04-25T10:15:58.705 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 96, 2024-04-25T10:15:58.705 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 12.050434853000001 2024-04-25T10:15:58.705 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:15:58.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.697+0000 7f52dea00640 1 -- 172.21.15.119:0/3546365695 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f52b806b4d0 msgr2=0x7f52b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.697+0000 7f52dea00640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f52b806b4d0 0x7f52b806d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009a20 tx=0x7f52cc008040 comp rx=0 tx=0).stop 2024-04-25T10:15:58.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.697+0000 7f52dea00640 1 -- 172.21.15.119:0/3546365695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d80adf70 msgr2=0x7f52d80c4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.697+0000 7f52dea00640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d80adf70 0x7f52d80c4ad0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f52d80cbe60 tx=0x7f52d000cb90 comp rx=0 tx=0).stop 2024-04-25T10:15:58.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.697+0000 7f52dea00640 1 -- 172.21.15.119:0/3546365695 shutdown_connections 2024-04-25T10:15:58.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.697+0000 7f52dea00640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f52b806b4d0 0x7f52b806d990 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.697+0000 7f52dea00640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d80adf70 0x7f52d80c4ad0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.697+0000 7f52dea00640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d80c6860 0x7f52d80ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.697+0000 7f52dea00640 1 --2- 172.21.15.119:0/3546365695 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f52c0004800 0x7f52d815b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.697+0000 7f52dea00640 1 -- 172.21.15.119:0/3546365695 >> 172.21.15.119:0/3546365695 conn(0x7f52d8066ff0 msgr2=0x7f52d80cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.697+0000 7f52dea00640 1 -- 172.21.15.119:0/3546365695 shutdown_connections 2024-04-25T10:15:58.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.697+0000 7f52dea00640 1 -- 172.21.15.119:0/3546365695 wait complete. 2024-04-25T10:15:58.717 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:15:58.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.899+0000 7f5e62c00640 1 Processor -- start 2024-04-25T10:15:58.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.899+0000 7f5e62c00640 1 -- start start 2024-04-25T10:15:58.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.901+0000 7f5e62c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5e5c15bf00 0x7f5e5c0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.901+0000 7f5e62c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e5c0cc310 0x7f5e5c15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.901+0000 7f5e62c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e5c15c820 0x7f5e5c06d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.901+0000 7f5e62c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5c1455e0 con 0x7f5e5c15c820 2024-04-25T10:15:58.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.901+0000 7f5e62c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5c06d450 con 0x7f5e5c15bf00 2024-04-25T10:15:58.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.901+0000 7f5e62c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c0ce6c0 con 0x7f5e5c0cc310 2024-04-25T10:15:58.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.901+0000 7f5e60c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e5c15c820 0x7f5e5c06d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.901+0000 7f5e60c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e5c15c820 0x7f5e5c06d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46432/0 (socket says 172.21.15.119:46432) 2024-04-25T10:15:58.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.901+0000 7f5e60c00640 1 -- 172.21.15.119:0/3668989182 learned_addr learned my addr 172.21.15.119:0/3668989182 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:58.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.901+0000 7f5e5be00640 1 --2- 172.21.15.119:0/3668989182 >> v2:172.21.15.132:3300/0 conn(0x7f5e5c15bf00 0x7f5e5c0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.902+0000 7f5e5b400640 1 --2- 172.21.15.119:0/3668989182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e5c0cc310 0x7f5e5c15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.902+0000 7f5e60c00640 1 -- 172.21.15.119:0/3668989182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e5c0cc310 msgr2=0x7f5e5c15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.902+0000 7f5e60c00640 1 --2- 172.21.15.119:0/3668989182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e5c0cc310 0x7f5e5c15c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.902+0000 7f5e60c00640 1 -- 172.21.15.119:0/3668989182 >> v2:172.21.15.132:3300/0 conn(0x7f5e5c15bf00 msgr2=0x7f5e5c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.902+0000 7f5e60c00640 1 --2- 172.21.15.119:0/3668989182 >> v2:172.21.15.132:3300/0 conn(0x7f5e5c15bf00 0x7f5e5c0cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.902+0000 7f5e60c00640 1 -- 172.21.15.119:0/3668989182 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c0cc7b0 con 0x7f5e5c15c820 2024-04-25T10:15:58.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.902+0000 7f5e5be00640 1 --2- 172.21.15.119:0/3668989182 >> v2:172.21.15.132:3300/0 conn(0x7f5e5c15bf00 0x7f5e5c0cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:58.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.903+0000 7f5e60c00640 1 --2- 172.21.15.119:0/3668989182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c15c820 0x7f5e5c06d8e0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c06ba10 tx=0x7f5e5402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f4fd25e25d5e2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.903+0000 7f5e5aa00640 1 -- 172.21.15.119:0/3668989182 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54036070 con 0x7f5e5c15c820 2024-04-25T10:15:58.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.903+0000 7f5e5aa00640 1 -- 172.21.15.119:0/3668989182 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e54031600 con 0x7f5e5c15c820 2024-04-25T10:15:58.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.903+0000 7f5e5aa00640 1 -- 172.21.15.119:0/3668989182 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e540319d0 con 0x7f5e5c15c820 2024-04-25T10:15:58.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.904+0000 7f5e62c00640 1 -- 172.21.15.119:0/3668989182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c15c820 msgr2=0x7f5e5c06d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.904+0000 7f5e62c00640 1 --2- 172.21.15.119:0/3668989182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c15c820 0x7f5e5c06d8e0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c06ba10 tx=0x7f5e5402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.904+0000 7f5e62c00640 1 -- 172.21.15.119:0/3668989182 shutdown_connections 2024-04-25T10:15:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.904+0000 7f5e62c00640 1 --2- 172.21.15.119:0/3668989182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c15c820 0x7f5e5c06d8e0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.904+0000 7f5e62c00640 1 --2- 172.21.15.119:0/3668989182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e5c0cc310 0x7f5e5c15c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.904+0000 7f5e62c00640 1 --2- 172.21.15.119:0/3668989182 >> v2:172.21.15.132:3300/0 conn(0x7f5e5c15bf00 0x7f5e5c0cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.904+0000 7f5e62c00640 1 -- 172.21.15.119:0/3668989182 >> 172.21.15.119:0/3668989182 conn(0x7f5e5c06aba0 msgr2=0x7f5e5c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.904+0000 7f5e62c00640 1 -- 172.21.15.119:0/3668989182 shutdown_connections 2024-04-25T10:15:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.904+0000 7f5e62c00640 1 -- 172.21.15.119:0/3668989182 wait complete. 2024-04-25T10:15:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.905+0000 7f5e62c00640 1 Processor -- start 2024-04-25T10:15:58.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.906+0000 7f5e62c00640 1 -- start start 2024-04-25T10:15:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.906+0000 7f5e62c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c0cc310 0x7f5e5c1e4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.906+0000 7f5e62c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e5c15bf00 0x7f5e5c14bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.906+0000 7f5e62c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e5c15c820 0x7f5e5c14c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.906+0000 7f5e62c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c0cd050 con 0x7f5e5c0cc310 2024-04-25T10:15:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.906+0000 7f5e62c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c0ccd70 con 0x7f5e5c15bf00 2024-04-25T10:15:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.906+0000 7f5e62c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c15cca0 con 0x7f5e5c15c820 2024-04-25T10:15:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.907+0000 7f5e5be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c0cc310 0x7f5e5c1e4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.907+0000 7f5e5be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c0cc310 0x7f5e5c1e4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46446/0 (socket says 172.21.15.119:46446) 2024-04-25T10:15:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.907+0000 7f5e5be00640 1 -- 172.21.15.119:0/3186845896 learned_addr learned my addr 172.21.15.119:0/3186845896 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:15:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.907+0000 7f5e5b400640 1 --2- 172.21.15.119:0/3186845896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e5c15bf00 0x7f5e5c14bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.907+0000 7f5e5be00640 1 -- 172.21.15.119:0/3186845896 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e5c15c820 msgr2=0x7f5e5c14c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.907+0000 7f5e5be00640 1 --2- 172.21.15.119:0/3186845896 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e5c15c820 0x7f5e5c14c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.907+0000 7f5e5be00640 1 -- 172.21.15.119:0/3186845896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e5c15bf00 msgr2=0x7f5e5c14bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.907+0000 7f5e5be00640 1 --2- 172.21.15.119:0/3186845896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e5c15bf00 0x7f5e5c14bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.907+0000 7f5e5be00640 1 -- 172.21.15.119:0/3186845896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c1e6110 con 0x7f5e5c0cc310 2024-04-25T10:15:58.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.907+0000 7f5e5be00640 1 --2- 172.21.15.119:0/3186845896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c0cc310 0x7f5e5c1e4390 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00c7f0 tx=0x7f5e4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.908+0000 7f5e53e00640 1 -- 172.21.15.119:0/3186845896 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c015070 con 0x7f5e5c0cc310 2024-04-25T10:15:58.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.908+0000 7f5e53e00640 1 -- 172.21.15.119:0/3186845896 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4c007a30 con 0x7f5e5c0cc310 2024-04-25T10:15:58.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.908+0000 7f5e53e00640 1 -- 172.21.15.119:0/3186845896 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c007e00 con 0x7f5e5c0cc310 2024-04-25T10:15:58.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.908+0000 7f5e62c00640 1 -- 172.21.15.119:0/3186845896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5c13e820 con 0x7f5e5c0cc310 2024-04-25T10:15:58.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.908+0000 7f5e62c00640 1 -- 172.21.15.119:0/3186845896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5c1e6670 con 0x7f5e5c0cc310 2024-04-25T10:15:58.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.909+0000 7f5e62c00640 1 -- 172.21.15.119:0/3186845896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5c06bb30 con 0x7f5e5c0cc310 2024-04-25T10:15:58.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.910+0000 7f5e53e00640 1 -- 172.21.15.119:0/3186845896 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e4c018070 con 0x7f5e5c0cc310 2024-04-25T10:15:58.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.911+0000 7f5e53e00640 1 --2- 172.21.15.119:0/3186845896 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e3c06b4d0 0x7f5e3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.911+0000 7f5e53e00640 1 -- 172.21.15.119:0/3186845896 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e4c08a1a0 con 0x7f5e5c0cc310 2024-04-25T10:15:58.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.911+0000 7f5e5b400640 1 --2- 172.21.15.119:0/3186845896 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e3c06b4d0 0x7f5e3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.912+0000 7f5e5b400640 1 --2- 172.21.15.119:0/3186845896 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e3c06b4d0 0x7f5e3c06d990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5e48006fd0 tx=0x7f5e48008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:58.915+0000 7f5e53e00640 1 -- 172.21.15.119:0/3186845896 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e5c1e6110 con 0x7f5e5c0cc310 2024-04-25T10:15:59.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:59.146+0000 7f5e62c00640 1 -- 172.21.15.119:0/3186845896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5e5c072080 con 0x7f5e5c0cc310 2024-04-25T10:15:59.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:59.147+0000 7f5e53e00640 1 -- 172.21.15.119:0/3186845896 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5e5c072080 con 0x7f5e5c0cc310 2024-04-25T10:15:59.151 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:15:59.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:59.150+0000 7f5e62c00640 1 -- 172.21.15.119:0/3186845896 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e3c06b4d0 msgr2=0x7f5e3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:59.150+0000 7f5e62c00640 1 --2- 172.21.15.119:0/3186845896 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e3c06b4d0 0x7f5e3c06d990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5e48006fd0 tx=0x7f5e48008040 comp rx=0 tx=0).stop 2024-04-25T10:15:59.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:59.150+0000 7f5e62c00640 1 -- 172.21.15.119:0/3186845896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c0cc310 msgr2=0x7f5e5c1e4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:59.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:59.150+0000 7f5e62c00640 1 --2- 172.21.15.119:0/3186845896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c0cc310 0x7f5e5c1e4390 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00c7f0 tx=0x7f5e4c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:59.150+0000 7f5e62c00640 1 -- 172.21.15.119:0/3186845896 shutdown_connections 2024-04-25T10:15:59.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:59.150+0000 7f5e62c00640 1 --2- 172.21.15.119:0/3186845896 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e3c06b4d0 0x7f5e3c06d990 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:59.150+0000 7f5e62c00640 1 --2- 172.21.15.119:0/3186845896 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e5c15c820 0x7f5e5c14c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:59.150+0000 7f5e62c00640 1 --2- 172.21.15.119:0/3186845896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e5c15bf00 0x7f5e5c14bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:59.150+0000 7f5e62c00640 1 --2- 172.21.15.119:0/3186845896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e5c0cc310 0x7f5e5c1e4390 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:59.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:59.150+0000 7f5e62c00640 1 -- 172.21.15.119:0/3186845896 >> 172.21.15.119:0/3186845896 conn(0x7f5e5c06aba0 msgr2=0x7f5e5c0ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:59.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:59.150+0000 7f5e62c00640 1 -- 172.21.15.119:0/3186845896 shutdown_connections 2024-04-25T10:15:59.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:15:59.150+0000 7f5e62c00640 1 -- 172.21.15.119:0/3186845896 wait complete. 2024-04-25T10:15:59.171 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:15:59.171 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:15:59.171 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:15:59.171 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:15:59.171 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:59.171 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:15:59.171 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:15:59.171 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:15:59.171 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:15:59.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:59.172 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:15:59.222 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:15:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:15:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:15:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:15:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:59.222 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:15:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:59.222 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42130sec preferred_lft 42130sec 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:15:59.223 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:15:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:59.224 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:15:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:15:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:15:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:59.224 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:15:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:15:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:15:59.224 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:59.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:59.225 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:15:59.297 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:15:59.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:59.301 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:15:59.352 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:15:59.352 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:15:59.352 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:15:59.352 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:15:59.353 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:15:59.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:59.353 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:15:59.353 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:15:59.353 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:15:59.353 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:15:59.353 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:15:59.353 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:15:59.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:59.497 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:15:59.549 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:15:59.549 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:15:59.549 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:15:59.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:59.550 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:15:59.622 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:15:59.622 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:59.671 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:15:59.671 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:15:59.671 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:59.718 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:15:59.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:59.718 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:15:59.844 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:15:59.844 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:15:59.844 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:15:59.865 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:15:59.866 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:15:59.866 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:15:59.866 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:15:59.866 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:15:59.891 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:15:59.891 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:15:59.891 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:15:59.891 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:15:59.892 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:15:59.892 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:15:59.892 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:15:59.892 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:15:59.893 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:15:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:15:59.893 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:15:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:15:59.894 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:15:59.897 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:15:59.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:59.897 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:59.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:59.976 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:16:00.045 INFO:teuthology.orchestra.run.smithi119.stdout:client5510 2024-04-25T10:16:00.045 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:16:00.045 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:00.094 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:16:00.094 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:00.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:00.193 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:16:00.396 INFO:teuthology.orchestra.run.smithi119.stderr:6+0 records in 2024-04-25T10:16:00.396 INFO:teuthology.orchestra.run.smithi119.stderr:6+0 records out 2024-04-25T10:16:00.396 INFO:teuthology.orchestra.run.smithi119.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.145005 s, 43.4 MB/s 2024-04-25T10:16:00.398 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:16:00.398 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:00.449 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:16:00.449 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:00.500 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:00.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.744+0000 7fb92e000640 1 Processor -- start 2024-04-25T10:16:00.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.745+0000 7fb92e000640 1 -- start start 2024-04-25T10:16:00.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.746+0000 7fb92e000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb92815ec10 0x7fb92806dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.746+0000 7fb92e000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb92806e1b0 0x7fb9280b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.746+0000 7fb92e000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb92806e6c0 con 0x7fb9280b3580 2024-04-25T10:16:00.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.746+0000 7fb92e000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9280b39f0 con 0x7fb92806e1b0 2024-04-25T10:16:00.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.746+0000 7fb92e000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb92806f2d0 con 0x7fb92815ec10 2024-04-25T10:16:00.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.747+0000 7fb926a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb92806e1b0 0x7fb9280b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.747+0000 7fb926a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb92806e1b0 0x7fb9280b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46458/0 (socket says 172.21.15.119:46458) 2024-04-25T10:16:00.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.747+0000 7fb926a00640 1 -- 172.21.15.119:0/3933825141 learned_addr learned my addr 172.21.15.119:0/3933825141 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:00.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.747+0000 7fb927400640 1 --2- 172.21.15.119:0/3933825141 >> v2:172.21.15.132:3300/0 conn(0x7fb92815ec10 0x7fb92806dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.747+0000 7fb926a00640 1 -- 172.21.15.119:0/3933825141 >> v2:172.21.15.132:3300/0 conn(0x7fb92815ec10 msgr2=0x7fb92806dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:00.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.747+0000 7fb926a00640 1 --2- 172.21.15.119:0/3933825141 >> v2:172.21.15.132:3300/0 conn(0x7fb92815ec10 0x7fb92806dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.747+0000 7fb926a00640 1 -- 172.21.15.119:0/3933825141 >> v1:172.21.15.132:6789/0 conn(0x7fb9280b3580 legacy=0x7fb9281616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:00.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.747+0000 7fb926a00640 1 -- 172.21.15.119:0/3933825141 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92806e9a0 con 0x7fb92806e1b0 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.748+0000 7fb926a00640 1 --2- 172.21.15.119:0/3933825141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb92806e1b0 0x7fb9280b3040 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fb910009e50 tx=0x7fb91002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1451451f76046ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.748+0000 7fb926000640 1 -- 172.21.15.119:0/3933825141 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910036070 con 0x7fb92806e1b0 2024-04-25T10:16:00.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.749+0000 7fb926000640 1 -- 172.21.15.119:0/3933825141 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb910031600 con 0x7fb92806e1b0 2024-04-25T10:16:00.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.749+0000 7fb926000640 1 -- 172.21.15.119:0/3933825141 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9100319d0 con 0x7fb92806e1b0 2024-04-25T10:16:00.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.749+0000 7fb92e000640 1 -- 172.21.15.119:0/3933825141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb92806e1b0 msgr2=0x7fb9280b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.749+0000 7fb92e000640 1 --2- 172.21.15.119:0/3933825141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb92806e1b0 0x7fb9280b3040 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fb910009e50 tx=0x7fb91002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.749+0000 7fb92e000640 1 -- 172.21.15.119:0/3933825141 shutdown_connections 2024-04-25T10:16:00.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.749+0000 7fb92e000640 1 --2- 172.21.15.119:0/3933825141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb92806e1b0 0x7fb9280b3040 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.749+0000 7fb92e000640 1 --2- 172.21.15.119:0/3933825141 >> v2:172.21.15.132:3300/0 conn(0x7fb92815ec10 0x7fb92806dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.749+0000 7fb92e000640 1 -- 172.21.15.119:0/3933825141 >> 172.21.15.119:0/3933825141 conn(0x7fb9280b2220 msgr2=0x7fb9280cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:00.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.749+0000 7fb92e000640 1 -- 172.21.15.119:0/3933825141 shutdown_connections 2024-04-25T10:16:00.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.749+0000 7fb92e000640 1 -- 172.21.15.119:0/3933825141 wait complete. 2024-04-25T10:16:00.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.751+0000 7fb92e000640 1 Processor -- start 2024-04-25T10:16:00.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.751+0000 7fb92e000640 1 -- start start 2024-04-25T10:16:00.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.752+0000 7fb92e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb92806e1b0 0x7fb9280ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.752+0000 7fb92e000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9280b3580 0x7fb928140870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.752+0000 7fb92e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb92815ec10 0x7fb9281e6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.752+0000 7fb92e000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92815eff0 con 0x7fb92815ec10 2024-04-25T10:16:00.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.752+0000 7fb92e000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92806ef60 con 0x7fb9280b3580 2024-04-25T10:16:00.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.752+0000 7fb92e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92815e2e0 con 0x7fb92806e1b0 2024-04-25T10:16:00.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.752+0000 7fb927e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb92815ec10 0x7fb9281e6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.752+0000 7fb927e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb92815ec10 0x7fb9281e6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46468/0 (socket says 172.21.15.119:46468) 2024-04-25T10:16:00.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.752+0000 7fb927e00640 1 -- 172.21.15.119:0/3892461160 learned_addr learned my addr 172.21.15.119:0/3892461160 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:00.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.753+0000 7fb926a00640 1 --2- 172.21.15.119:0/3892461160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9280b3580 0x7fb928140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.753+0000 7fb927400640 1 --2- 172.21.15.119:0/3892461160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb92806e1b0 0x7fb9280ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.753+0000 7fb927e00640 1 -- 172.21.15.119:0/3892461160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb92806e1b0 msgr2=0x7fb9280ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.753+0000 7fb927e00640 1 --2- 172.21.15.119:0/3892461160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb92806e1b0 0x7fb9280ad7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.753+0000 7fb927e00640 1 -- 172.21.15.119:0/3892461160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9280b3580 msgr2=0x7fb928140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.753+0000 7fb927e00640 1 --2- 172.21.15.119:0/3892461160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9280b3580 0x7fb928140870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.753+0000 7fb927e00640 1 -- 172.21.15.119:0/3892461160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9280ae070 con 0x7fb92815ec10 2024-04-25T10:16:00.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.753+0000 7fb927e00640 1 --2- 172.21.15.119:0/3892461160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb92815ec10 0x7fb9281e6e30 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fb92000c410 tx=0x7fb92000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.753+0000 7fb91f400640 1 -- 172.21.15.119:0/3892461160 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920011790 con 0x7fb92815ec10 2024-04-25T10:16:00.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.753+0000 7fb91f400640 1 -- 172.21.15.119:0/3892461160 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb920017070 con 0x7fb92815ec10 2024-04-25T10:16:00.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.754+0000 7fb91f400640 1 -- 172.21.15.119:0/3892461160 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920011bb0 con 0x7fb92815ec10 2024-04-25T10:16:00.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.754+0000 7fb92e000640 1 -- 172.21.15.119:0/3892461160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9280ae350 con 0x7fb92815ec10 2024-04-25T10:16:00.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.754+0000 7fb92e000640 1 -- 172.21.15.119:0/3892461160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9280add30 con 0x7fb92815ec10 2024-04-25T10:16:00.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.755+0000 7fb92e000640 1 -- 172.21.15.119:0/3892461160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9280af0b0 con 0x7fb92815ec10 2024-04-25T10:16:00.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.756+0000 7fb91f400640 1 -- 172.21.15.119:0/3892461160 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb92001c460 con 0x7fb92815ec10 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.757+0000 7fb91f400640 1 --2- 172.21.15.119:0/3892461160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb90406b4d0 0x7fb90406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.757+0000 7fb91f400640 1 -- 172.21.15.119:0/3892461160 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb920089e90 con 0x7fb92815ec10 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.757+0000 7fb927400640 1 --2- 172.21.15.119:0/3892461160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb90406b4d0 0x7fb90406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:00.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.758+0000 7fb927400640 1 --2- 172.21.15.119:0/3892461160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb90406b4d0 0x7fb90406d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb918006fd0 tx=0x7fb918008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:00.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.761+0000 7fb91f400640 1 -- 172.21.15.119:0/3892461160 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb920021020 con 0x7fb92815ec10 2024-04-25T10:16:00.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.959+0000 7fb92e000640 1 -- 172.21.15.119:0/3892461160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9280749b0 con 0x7fb92815ec10 2024-04-25T10:16:00.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.961+0000 7fb91f400640 1 -- 172.21.15.119:0/3892461160 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16644 (secure 0 0 0) 0x7fb920058c70 con 0x7fb92815ec10 2024-04-25T10:16:00.965 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:00.965 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":96,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:15:53.131988+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:49.348469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:50.584869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:00.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.963+0000 7fb92e000640 1 -- 172.21.15.119:0/3892461160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb90406b4d0 msgr2=0x7fb90406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.963+0000 7fb92e000640 1 --2- 172.21.15.119:0/3892461160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb90406b4d0 0x7fb90406d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb918006fd0 tx=0x7fb918008040 comp rx=0 tx=0).stop 2024-04-25T10:16:00.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.964+0000 7fb92e000640 1 -- 172.21.15.119:0/3892461160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb92815ec10 msgr2=0x7fb9281e6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:00.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.964+0000 7fb92e000640 1 --2- 172.21.15.119:0/3892461160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb92815ec10 0x7fb9281e6e30 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fb92000c410 tx=0x7fb92000c8d0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.964+0000 7fb92e000640 1 -- 172.21.15.119:0/3892461160 shutdown_connections 2024-04-25T10:16:00.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.964+0000 7fb92e000640 1 --2- 172.21.15.119:0/3892461160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb90406b4d0 0x7fb90406d990 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.964+0000 7fb92e000640 1 --2- 172.21.15.119:0/3892461160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb92815ec10 0x7fb9281e6e30 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.964+0000 7fb92e000640 1 --2- 172.21.15.119:0/3892461160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9280b3580 0x7fb928140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.964+0000 7fb92e000640 1 --2- 172.21.15.119:0/3892461160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb92806e1b0 0x7fb9280ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:00.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.964+0000 7fb92e000640 1 -- 172.21.15.119:0/3892461160 >> 172.21.15.119:0/3892461160 conn(0x7fb9280b2220 msgr2=0x7fb9280ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:00.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.964+0000 7fb92e000640 1 -- 172.21.15.119:0/3892461160 shutdown_connections 2024-04-25T10:16:00.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:00.964+0000 7fb92e000640 1 -- 172.21.15.119:0/3892461160 wait complete. 2024-04-25T10:16:00.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:15:49.348469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:15:50.584869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:00.988 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:01.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.189+0000 7f6737800640 1 Processor -- start 2024-04-25T10:16:01.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.189+0000 7f6737800640 1 -- start start 2024-04-25T10:16:01.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.191+0000 7f6737800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6730144e70 0x7f673006d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.191+0000 7f6737800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f673006d7a0 0x7f67300ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.191+0000 7f6737800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f673006dcb0 con 0x7f67300cef10 2024-04-25T10:16:01.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.191+0000 7f6737800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673015bfe0 con 0x7f6730144e70 2024-04-25T10:16:01.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.191+0000 7f6737800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673006df90 con 0x7f673006d7a0 2024-04-25T10:16:01.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.192+0000 7f6734e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6730144e70 0x7f673006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.192+0000 7f6727e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f673006d7a0 0x7f67300ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.192+0000 7f6735800640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f67300cef10 0x7f6730071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:37038/0 (socket says 172.21.15.119:37038) 2024-04-25T10:16:01.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.192+0000 7f6735800640 1 -- 172.21.15.119:0/996419659 learned_addr learned my addr 172.21.15.119:0/996419659 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:16:01.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.193+0000 7f6727e00640 1 -- 172.21.15.119:0/996419659 >> v2:172.21.15.132:3300/0 conn(0x7f6730144e70 msgr2=0x7f673006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:01.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.193+0000 7f6727e00640 1 --2- 172.21.15.119:0/996419659 >> v2:172.21.15.132:3300/0 conn(0x7f6730144e70 0x7f673006d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.193+0000 7f6727e00640 1 -- 172.21.15.119:0/996419659 >> v1:172.21.15.132:6789/0 conn(0x7f67300cef10 legacy=0x7f6730071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.193+0000 7f6727e00640 1 -- 172.21.15.119:0/996419659 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673006e270 con 0x7f673006d7a0 2024-04-25T10:16:01.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.193+0000 7f6734e00640 1 --2- 172.21.15.119:0/996419659 >> v2:172.21.15.132:3300/0 conn(0x7f6730144e70 0x7f673006d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:01.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.194+0000 7f6727e00640 1 --2- 172.21.15.119:0/996419659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f673006d7a0 0x7f67300ce9d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f67180097a0 tx=0x7f671802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a928ccf8c6b2efa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.195+0000 7f672fe00640 1 -- 172.21.15.119:0/996419659 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718037070 con 0x7f673006d7a0 2024-04-25T10:16:01.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.195+0000 7f672fe00640 1 -- 172.21.15.119:0/996419659 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671802ae40 con 0x7f673006d7a0 2024-04-25T10:16:01.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.195+0000 7f672fe00640 1 -- 172.21.15.119:0/996419659 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718032660 con 0x7f673006d7a0 2024-04-25T10:16:01.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.195+0000 7f6737800640 1 -- 172.21.15.119:0/996419659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f673006d7a0 msgr2=0x7f67300ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.195+0000 7f6737800640 1 --2- 172.21.15.119:0/996419659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f673006d7a0 0x7f67300ce9d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f67180097a0 tx=0x7f671802a0f0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.196+0000 7f6737800640 1 -- 172.21.15.119:0/996419659 shutdown_connections 2024-04-25T10:16:01.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.196+0000 7f6737800640 1 --2- 172.21.15.119:0/996419659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f673006d7a0 0x7f67300ce9d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.196+0000 7f6737800640 1 --2- 172.21.15.119:0/996419659 >> v2:172.21.15.132:3300/0 conn(0x7f6730144e70 0x7f673006d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.196+0000 7f6737800640 1 -- 172.21.15.119:0/996419659 >> 172.21.15.119:0/996419659 conn(0x7f673006aba0 msgr2=0x7f673006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:01.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.196+0000 7f6737800640 1 -- 172.21.15.119:0/996419659 shutdown_connections 2024-04-25T10:16:01.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.196+0000 7f6737800640 1 -- 172.21.15.119:0/996419659 wait complete. 2024-04-25T10:16:01.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.197+0000 7f6737800640 1 Processor -- start 2024-04-25T10:16:01.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.198+0000 7f6737800640 1 -- start start 2024-04-25T10:16:01.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.198+0000 7f6737800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f673006d7a0 0x7f673013eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.198+0000 7f6737800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67300cef10 0x7f67300ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6737800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6730144e70 0x7f673015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6737800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67301454b0 con 0x7f67300cef10 2024-04-25T10:16:01.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6737800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67300cf380 con 0x7f6730144e70 2024-04-25T10:16:01.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6737800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6730145630 con 0x7f673006d7a0 2024-04-25T10:16:01.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6727e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67300cef10 0x7f67300ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6727e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67300cef10 0x7f67300ae270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52894/0 (socket says 172.21.15.119:52894) 2024-04-25T10:16:01.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6727e00640 1 -- 172.21.15.119:0/1411521584 learned_addr learned my addr 172.21.15.119:0/1411521584 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:01.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6734e00640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f673006d7a0 0x7f673013eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6735800640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6730144e70 0x7f673015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6727e00640 1 -- 172.21.15.119:0/1411521584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f673006d7a0 msgr2=0x7f673013eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6727e00640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f673006d7a0 0x7f673013eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6727e00640 1 -- 172.21.15.119:0/1411521584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6730144e70 msgr2=0x7f673015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6727e00640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6730144e70 0x7f673015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6727e00640 1 -- 172.21.15.119:0/1411521584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673013dfe0 con 0x7f67300cef10 2024-04-25T10:16:01.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6734e00640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f673006d7a0 0x7f673013eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:01.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.199+0000 7f6735800640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6730144e70 0x7f673015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:01.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.200+0000 7f6727e00640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67300cef10 0x7f67300ae270 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f67180098c0 tx=0x7f67180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.200+0000 7f672d600640 1 -- 172.21.15.119:0/1411521584 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718041030 con 0x7f67300cef10 2024-04-25T10:16:01.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.200+0000 7f672d600640 1 -- 172.21.15.119:0/1411521584 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671802ac40 con 0x7f67300cef10 2024-04-25T10:16:01.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.200+0000 7f672d600640 1 -- 172.21.15.119:0/1411521584 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718037040 con 0x7f67300cef10 2024-04-25T10:16:01.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.200+0000 7f6737800640 1 -- 172.21.15.119:0/1411521584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673013e2c0 con 0x7f67300cef10 2024-04-25T10:16:01.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.200+0000 7f6737800640 1 -- 172.21.15.119:0/1411521584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673013e880 con 0x7f67300cef10 2024-04-25T10:16:01.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.201+0000 7f6737800640 1 -- 172.21.15.119:0/1411521584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67300750b0 con 0x7f67300cef10 2024-04-25T10:16:01.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.203+0000 7f672d600640 1 -- 172.21.15.119:0/1411521584 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6718044070 con 0x7f67300cef10 2024-04-25T10:16:01.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.203+0000 7f672d600640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f670c06b4d0 0x7f670c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.204+0000 7f672d600640 1 -- 172.21.15.119:0/1411521584 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67180aa690 con 0x7f67300cef10 2024-04-25T10:16:01.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.204+0000 7f6734e00640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f670c06b4d0 0x7f670c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.205+0000 7f6734e00640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f670c06b4d0 0x7f670c06d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f67300adfb0 tx=0x7f6720008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.208+0000 7f672d600640 1 -- 172.21.15.119:0/1411521584 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f671804b070 con 0x7f67300cef10 2024-04-25T10:16:01.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.401+0000 7f6737800640 1 -- 172.21.15.119:0/1411521584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67300aebe0 con 0x7f67300cef10 2024-04-25T10:16:01.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.401+0000 7f672d600640 1 -- 172.21.15.119:0/1411521584 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6718044e40 con 0x7f67300cef10 2024-04-25T10:16:01.405 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:16:01.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.404+0000 7f6737800640 1 -- 172.21.15.119:0/1411521584 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f670c06b4d0 msgr2=0x7f670c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.404+0000 7f6737800640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f670c06b4d0 0x7f670c06d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f67300adfb0 tx=0x7f6720008040 comp rx=0 tx=0).stop 2024-04-25T10:16:01.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.404+0000 7f6737800640 1 -- 172.21.15.119:0/1411521584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67300cef10 msgr2=0x7f67300ae270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.404+0000 7f6737800640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67300cef10 0x7f67300ae270 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f67180098c0 tx=0x7f67180046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.404+0000 7f6737800640 1 -- 172.21.15.119:0/1411521584 shutdown_connections 2024-04-25T10:16:01.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.404+0000 7f6737800640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f670c06b4d0 0x7f670c06d990 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.404+0000 7f6737800640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6730144e70 0x7f673015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.404+0000 7f6737800640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67300cef10 0x7f67300ae270 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.404+0000 7f6737800640 1 --2- 172.21.15.119:0/1411521584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f673006d7a0 0x7f673013eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.404+0000 7f6737800640 1 -- 172.21.15.119:0/1411521584 >> 172.21.15.119:0/1411521584 conn(0x7f673006aba0 msgr2=0x7f6730140220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.404+0000 7f6737800640 1 -- 172.21.15.119:0/1411521584 shutdown_connections 2024-04-25T10:16:01.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.404+0000 7f6737800640 1 -- 172.21.15.119:0/1411521584 wait complete. 2024-04-25T10:16:01.427 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:01.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.627+0000 7fab34800640 1 Processor -- start 2024-04-25T10:16:01.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.627+0000 7fab34800640 1 -- start start 2024-04-25T10:16:01.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.629+0000 7fab34800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab2c144e70 0x7fab2c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.629+0000 7fab34800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab2c0cf0e0 0x7fab2c0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.629+0000 7fab34800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2c06dcb0 con 0x7fab2c06d7a0 2024-04-25T10:16:01.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.629+0000 7fab34800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab2c15bfe0 con 0x7fab2c0cf0e0 2024-04-25T10:16:01.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.629+0000 7fab34800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c06df90 con 0x7fab2c144e70 2024-04-25T10:16:01.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.629+0000 7fab32800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab2c0cf0e0 0x7fab2c0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.630+0000 7fab32800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab2c0cf0e0 0x7fab2c0cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52902/0 (socket says 172.21.15.119:52902) 2024-04-25T10:16:01.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.630+0000 7fab32800640 1 -- 172.21.15.119:0/4224264498 learned_addr learned my addr 172.21.15.119:0/4224264498 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:01.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.630+0000 7fab31e00640 1 --2- 172.21.15.119:0/4224264498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab2c144e70 0x7fab2c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.630+0000 7fab32800640 1 -- 172.21.15.119:0/4224264498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab2c144e70 msgr2=0x7fab2c06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:01.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.630+0000 7fab32800640 1 --2- 172.21.15.119:0/4224264498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab2c144e70 0x7fab2c06d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.630+0000 7fab32800640 1 -- 172.21.15.119:0/4224264498 >> v1:172.21.15.132:6789/0 conn(0x7fab2c06d7a0 legacy=0x7fab2c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:01.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.630+0000 7fab32800640 1 -- 172.21.15.119:0/4224264498 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c06e270 con 0x7fab2c0cf0e0 2024-04-25T10:16:01.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.631+0000 7fab32800640 1 --2- 172.21.15.119:0/4224264498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c0cf0e0 0x7fab2c0cc300 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0711b0 tx=0x7fab2802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbd37544c039244 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.631+0000 7fab30a00640 1 -- 172.21.15.119:0/4224264498 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28036070 con 0x7fab2c0cf0e0 2024-04-25T10:16:01.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.632+0000 7fab30a00640 1 -- 172.21.15.119:0/4224264498 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab28031600 con 0x7fab2c0cf0e0 2024-04-25T10:16:01.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.632+0000 7fab30a00640 1 -- 172.21.15.119:0/4224264498 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab280319d0 con 0x7fab2c0cf0e0 2024-04-25T10:16:01.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.632+0000 7fab34800640 1 -- 172.21.15.119:0/4224264498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c0cf0e0 msgr2=0x7fab2c0cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.632+0000 7fab34800640 1 --2- 172.21.15.119:0/4224264498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c0cf0e0 0x7fab2c0cc300 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0711b0 tx=0x7fab2802a630 comp rx=0 tx=0).stop 2024-04-25T10:16:01.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.632+0000 7fab34800640 1 -- 172.21.15.119:0/4224264498 shutdown_connections 2024-04-25T10:16:01.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.632+0000 7fab34800640 1 --2- 172.21.15.119:0/4224264498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c0cf0e0 0x7fab2c0cc300 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.632+0000 7fab34800640 1 --2- 172.21.15.119:0/4224264498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab2c144e70 0x7fab2c06d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.632+0000 7fab34800640 1 -- 172.21.15.119:0/4224264498 >> 172.21.15.119:0/4224264498 conn(0x7fab2c06aba0 msgr2=0x7fab2c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:01.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.632+0000 7fab34800640 1 -- 172.21.15.119:0/4224264498 shutdown_connections 2024-04-25T10:16:01.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.632+0000 7fab34800640 1 -- 172.21.15.119:0/4224264498 wait complete. 2024-04-25T10:16:01.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.634+0000 7fab34800640 1 Processor -- start 2024-04-25T10:16:01.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.634+0000 7fab34800640 1 -- start start 2024-04-25T10:16:01.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.635+0000 7fab34800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab2c06d7a0 0x7fab2c13e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.635+0000 7fab34800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab2c0cf0e0 0x7fab2c1e7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.635+0000 7fab34800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c144e70 0x7fab2c1e78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.635+0000 7fab34800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c145630 con 0x7fab2c144e70 2024-04-25T10:16:01.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.635+0000 7fab34800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c1454b0 con 0x7fab2c06d7a0 2024-04-25T10:16:01.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.635+0000 7fab34800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c0cc840 con 0x7fab2c0cf0e0 2024-04-25T10:16:01.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.635+0000 7fab32800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c144e70 0x7fab2c1e78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.635+0000 7fab32800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c144e70 0x7fab2c1e78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52914/0 (socket says 172.21.15.119:52914) 2024-04-25T10:16:01.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.635+0000 7fab32800640 1 -- 172.21.15.119:0/3825772887 learned_addr learned my addr 172.21.15.119:0/3825772887 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:01.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.636+0000 7fab31e00640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab2c06d7a0 0x7fab2c13e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.636+0000 7fab31400640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab2c0cf0e0 0x7fab2c1e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.636+0000 7fab32800640 1 -- 172.21.15.119:0/3825772887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab2c0cf0e0 msgr2=0x7fab2c1e7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.636+0000 7fab32800640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab2c0cf0e0 0x7fab2c1e7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.636+0000 7fab32800640 1 -- 172.21.15.119:0/3825772887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab2c06d7a0 msgr2=0x7fab2c13e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.636+0000 7fab32800640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab2c06d7a0 0x7fab2c13e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.636+0000 7fab32800640 1 -- 172.21.15.119:0/3825772887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c0b1a90 con 0x7fab2c144e70 2024-04-25T10:16:01.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.636+0000 7fab31400640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab2c0cf0e0 0x7fab2c1e7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:01.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.636+0000 7fab32800640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c144e70 0x7fab2c1e78c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fab28009f40 tx=0x7fab28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.636+0000 7fab26000640 1 -- 172.21.15.119:0/3825772887 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28036050 con 0x7fab2c144e70 2024-04-25T10:16:01.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.636+0000 7fab26000640 1 -- 172.21.15.119:0/3825772887 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab28042070 con 0x7fab2c144e70 2024-04-25T10:16:01.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.637+0000 7fab26000640 1 -- 172.21.15.119:0/3825772887 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28007b90 con 0x7fab2c144e70 2024-04-25T10:16:01.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.637+0000 7fab34800640 1 -- 172.21.15.119:0/3825772887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c0b17b0 con 0x7fab2c144e70 2024-04-25T10:16:01.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.637+0000 7fab34800640 1 -- 172.21.15.119:0/3825772887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c1e4f30 con 0x7fab2c144e70 2024-04-25T10:16:01.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.638+0000 7fab34800640 1 -- 172.21.15.119:0/3825772887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2c0750b0 con 0x7fab2c144e70 2024-04-25T10:16:01.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.639+0000 7fab26000640 1 -- 172.21.15.119:0/3825772887 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab28047460 con 0x7fab2c144e70 2024-04-25T10:16:01.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.640+0000 7fab26000640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fab1006b4d0 0x7fab1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:01.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.640+0000 7fab26000640 1 -- 172.21.15.119:0/3825772887 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab280ab170 con 0x7fab2c144e70 2024-04-25T10:16:01.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.640+0000 7fab31e00640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fab1006b4d0 0x7fab1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:01.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.641+0000 7fab31e00640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fab1006b4d0 0x7fab1006d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fab20006fd0 tx=0x7fab20008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:01.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.644+0000 7fab26000640 1 -- 172.21.15.119:0/3825772887 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab28045360 con 0x7fab2c144e70 2024-04-25T10:16:01.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.841+0000 7fab34800640 1 -- 172.21.15.119:0/3825772887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab2c0aebe0 con 0x7fab2c144e70 2024-04-25T10:16:01.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.842+0000 7fab26000640 1 -- 172.21.15.119:0/3825772887 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab28079f50 con 0x7fab2c144e70 2024-04-25T10:16:01.846 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:16:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.844+0000 7fab34800640 1 -- 172.21.15.119:0/3825772887 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fab1006b4d0 msgr2=0x7fab1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.844+0000 7fab34800640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fab1006b4d0 0x7fab1006d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fab20006fd0 tx=0x7fab20008040 comp rx=0 tx=0).stop 2024-04-25T10:16:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.844+0000 7fab34800640 1 -- 172.21.15.119:0/3825772887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c144e70 msgr2=0x7fab2c1e78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.845+0000 7fab34800640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c144e70 0x7fab2c1e78c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fab28009f40 tx=0x7fab28004a40 comp rx=0 tx=0).stop 2024-04-25T10:16:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.845+0000 7fab34800640 1 -- 172.21.15.119:0/3825772887 shutdown_connections 2024-04-25T10:16:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.845+0000 7fab34800640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fab1006b4d0 0x7fab1006d990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.845+0000 7fab34800640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab2c144e70 0x7fab2c1e78c0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.845+0000 7fab34800640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab2c0cf0e0 0x7fab2c1e7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.845+0000 7fab34800640 1 --2- 172.21.15.119:0/3825772887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab2c06d7a0 0x7fab2c13e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.845+0000 7fab34800640 1 -- 172.21.15.119:0/3825772887 >> 172.21.15.119:0/3825772887 conn(0x7fab2c06aba0 msgr2=0x7fab2c0ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.845+0000 7fab34800640 1 -- 172.21.15.119:0/3825772887 shutdown_connections 2024-04-25T10:16:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:01.845+0000 7fab34800640 1 -- 172.21.15.119:0/3825772887 wait complete. 2024-04-25T10:16:01.867 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:02.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.065+0000 7fc36ba00640 1 Processor -- start 2024-04-25T10:16:02.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.066+0000 7fc36ba00640 1 -- start start 2024-04-25T10:16:02.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.067+0000 7fc36ba00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc36415dca0 0x7fc3640ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.067+0000 7fc36ba00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3640d4090 con 0x7fc3640d1130 2024-04-25T10:16:02.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.067+0000 7fc36ba00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3640d4370 con 0x7fc3640d3b90 2024-04-25T10:16:02.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.067+0000 7fc36ba00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3640b6b20 con 0x7fc36415dca0 2024-04-25T10:16:02.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.067+0000 7fc369a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc3640d1130 0x7fc3640b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48780/0 (socket says 172.21.15.119:48780) 2024-04-25T10:16:02.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.067+0000 7fc369a00640 1 -- 172.21.15.119:0/3138961883 learned_addr learned my addr 172.21.15.119:0/3138961883 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:16:02.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.068+0000 7fc369000640 1 --2- 172.21.15.119:0/3138961883 >> v2:172.21.15.132:3300/0 conn(0x7fc36415dca0 0x7fc3640ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.068+0000 7fc363400640 1 -- 172.21.15.119:0/3138961883 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396747480 0 0) 0x7fc3640d4090 con 0x7fc3640d1130 2024-04-25T10:16:02.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.068+0000 7fc363400640 1 -- 172.21.15.119:0/3138961883 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc348003520 con 0x7fc3640d1130 2024-04-25T10:16:02.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.068+0000 7fc363400640 1 -- 172.21.15.119:0/3138961883 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 715814799 0 0) 0x7fc348003520 con 0x7fc3640d1130 2024-04-25T10:16:02.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.068+0000 7fc363400640 1 -- 172.21.15.119:0/3138961883 >> v2:172.21.15.132:3300/0 conn(0x7fc36415dca0 msgr2=0x7fc3640ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:02.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.068+0000 7fc363400640 1 --2- 172.21.15.119:0/3138961883 >> v2:172.21.15.132:3300/0 conn(0x7fc36415dca0 0x7fc3640ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.068+0000 7fc363400640 1 -- 172.21.15.119:0/3138961883 >> v1:172.21.15.132:6789/0 conn(0x7fc3640d3b90 legacy=0x7fc3640d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.068+0000 7fc363400640 1 -- 172.21.15.119:0/3138961883 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3640d4650 con 0x7fc3640d1130 2024-04-25T10:16:02.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.068+0000 7fc363400640 1 -- 172.21.15.119:0/3138961883 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fc35c001670 con 0x7fc3640d1130 2024-04-25T10:16:02.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.068+0000 7fc363400640 1 -- 172.21.15.119:0/3138961883 >> v1:172.21.15.119:6789/0 conn(0x7fc3640d1130 legacy=0x7fc3640b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc363400640 1 --2- 172.21.15.119:0/3138961883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3480039b0 0x7fc348003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc363400640 1 --2- 172.21.15.119:0/3138961883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc348004800 0x7fc348004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc363400640 1 --2- 172.21.15.119:0/3138961883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3480051c0 0x7fc34800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc363400640 1 -- 172.21.15.119:0/3138961883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34800c030 con 0x7fc3480039b0 2024-04-25T10:16:02.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc363400640 1 -- 172.21.15.119:0/3138961883 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34800c1e0 con 0x7fc3480051c0 2024-04-25T10:16:02.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc363400640 1 -- 172.21.15.119:0/3138961883 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34800c3e0 con 0x7fc348004800 2024-04-25T10:16:02.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc369000640 1 --2- 172.21.15.119:0/3138961883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3480039b0 0x7fc348003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc363e00640 1 --2- 172.21.15.119:0/3138961883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc348004800 0x7fc348004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc369a00640 1 --2- 172.21.15.119:0/3138961883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3480051c0 0x7fc34800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc369000640 1 -- 172.21.15.119:0/3138961883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc348004800 msgr2=0x7fc348004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:02.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc369000640 1 --2- 172.21.15.119:0/3138961883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc348004800 0x7fc348004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc369000640 1 -- 172.21.15.119:0/3138961883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3480051c0 msgr2=0x7fc34800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:02.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc369000640 1 --2- 172.21.15.119:0/3138961883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3480051c0 0x7fc34800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc369000640 1 -- 172.21.15.119:0/3138961883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc348003520 con 0x7fc3480039b0 2024-04-25T10:16:02.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc369a00640 1 -- 172.21.15.119:0/3138961883 reap_dead start 2024-04-25T10:16:02.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.069+0000 7fc363e00640 1 --2- 172.21.15.119:0/3138961883 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc348004800 0x7fc348004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:02.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.070+0000 7fc369000640 1 --2- 172.21.15.119:0/3138961883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3480039b0 0x7fc348003dd0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc354009ac0 tx=0x7fc348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84dea06a54b0bdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.071+0000 7fc363400640 1 -- 172.21.15.119:0/3138961883 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354036070 con 0x7fc3480039b0 2024-04-25T10:16:02.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.071+0000 7fc363400640 1 -- 172.21.15.119:0/3138961883 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc354031400 con 0x7fc3480039b0 2024-04-25T10:16:02.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.071+0000 7fc363400640 1 -- 172.21.15.119:0/3138961883 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354009690 con 0x7fc3480039b0 2024-04-25T10:16:02.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.071+0000 7fc36ba00640 1 -- 172.21.15.119:0/3138961883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3480039b0 msgr2=0x7fc348003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.071+0000 7fc36ba00640 1 --2- 172.21.15.119:0/3138961883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3480039b0 0x7fc348003dd0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc354009ac0 tx=0x7fc348005830 comp rx=0 tx=0).stop 2024-04-25T10:16:02.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.071+0000 7fc36ba00640 1 -- 172.21.15.119:0/3138961883 shutdown_connections 2024-04-25T10:16:02.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.071+0000 7fc36ba00640 1 --2- 172.21.15.119:0/3138961883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3480039b0 0x7fc348003dd0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.071+0000 7fc36ba00640 1 -- 172.21.15.119:0/3138961883 >> 172.21.15.119:0/3138961883 conn(0x7fc3640726c0 msgr2=0x7fc364072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.071+0000 7fc36ba00640 1 -- 172.21.15.119:0/3138961883 shutdown_connections 2024-04-25T10:16:02.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.071+0000 7fc36ba00640 1 -- 172.21.15.119:0/3138961883 wait complete. 2024-04-25T10:16:02.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.073+0000 7fc36ba00640 1 Processor -- start 2024-04-25T10:16:02.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.073+0000 7fc36ba00640 1 -- start start 2024-04-25T10:16:02.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.074+0000 7fc36ba00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3480039b0 0x7fc3640ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.074+0000 7fc36ba00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3640cab60 0x7fc3640b86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.074+0000 7fc36ba00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3640b8bf0 0x7fc364155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.074+0000 7fc36ba00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36415e200 con 0x7fc3640cab60 2024-04-25T10:16:02.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.074+0000 7fc36ba00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36415e080 con 0x7fc3480039b0 2024-04-25T10:16:02.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.074+0000 7fc36ba00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640d0a20 con 0x7fc3640b8bf0 2024-04-25T10:16:02.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.074+0000 7fc363e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3640cab60 0x7fc3640b86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.074+0000 7fc363e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3640cab60 0x7fc3640b86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52934/0 (socket says 172.21.15.119:52934) 2024-04-25T10:16:02.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.074+0000 7fc363e00640 1 -- 172.21.15.119:0/823922221 learned_addr learned my addr 172.21.15.119:0/823922221 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:02.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.075+0000 7fc369a00640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3640b8bf0 0x7fc364155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.075+0000 7fc369000640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3480039b0 0x7fc3640ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.075+0000 7fc363e00640 1 -- 172.21.15.119:0/823922221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3640b8bf0 msgr2=0x7fc364155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.075+0000 7fc363e00640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3640b8bf0 0x7fc364155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.075+0000 7fc363e00640 1 -- 172.21.15.119:0/823922221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3480039b0 msgr2=0x7fc3640ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.075+0000 7fc363e00640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3480039b0 0x7fc3640ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.075+0000 7fc363e00640 1 -- 172.21.15.119:0/823922221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3641f8260 con 0x7fc3640cab60 2024-04-25T10:16:02.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.075+0000 7fc369a00640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3640b8bf0 0x7fc364155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:02.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.075+0000 7fc369000640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3480039b0 0x7fc3640ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:02.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.075+0000 7fc363e00640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3640cab60 0x7fc3640b86b0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fc350007fd0 tx=0x7fc35000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.075+0000 7fc360c00640 1 -- 172.21.15.119:0/823922221 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc350003ba0 con 0x7fc3640cab60 2024-04-25T10:16:02.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.075+0000 7fc360c00640 1 -- 172.21.15.119:0/823922221 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc350003d60 con 0x7fc3640cab60 2024-04-25T10:16:02.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.075+0000 7fc360c00640 1 -- 172.21.15.119:0/823922221 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc350006d90 con 0x7fc3640cab60 2024-04-25T10:16:02.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.076+0000 7fc36ba00640 1 -- 172.21.15.119:0/823922221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3641f76e0 con 0x7fc3640cab60 2024-04-25T10:16:02.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.076+0000 7fc36ba00640 1 -- 172.21.15.119:0/823922221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3641f7ca0 con 0x7fc3640cab60 2024-04-25T10:16:02.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.077+0000 7fc36ba00640 1 -- 172.21.15.119:0/823922221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3640750b0 con 0x7fc3640cab60 2024-04-25T10:16:02.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.078+0000 7fc360c00640 1 -- 172.21.15.119:0/823922221 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc35001a020 con 0x7fc3640cab60 2024-04-25T10:16:02.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.079+0000 7fc360c00640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc34406b4d0 0x7fc34406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.079+0000 7fc360c00640 1 -- 172.21.15.119:0/823922221 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc350089b40 con 0x7fc3640cab60 2024-04-25T10:16:02.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.079+0000 7fc369000640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc34406b4d0 0x7fc34406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.080+0000 7fc369000640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc34406b4d0 0x7fc34406d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc354033000 tx=0x7fc354008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.083+0000 7fc360c00640 1 -- 172.21.15.119:0/823922221 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc350058a50 con 0x7fc3640cab60 2024-04-25T10:16:02.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.281+0000 7fc36ba00640 1 -- 172.21.15.119:0/823922221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc3640b8fd0 con 0x7fc3640cab60 2024-04-25T10:16:02.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.282+0000 7fc360c00640 1 -- 172.21.15.119:0/823922221 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc350058350 con 0x7fc3640cab60 2024-04-25T10:16:02.286 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:16:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.284+0000 7fc36ba00640 1 -- 172.21.15.119:0/823922221 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc34406b4d0 msgr2=0x7fc34406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.284+0000 7fc36ba00640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc34406b4d0 0x7fc34406d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc354033000 tx=0x7fc354008030 comp rx=0 tx=0).stop 2024-04-25T10:16:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.284+0000 7fc36ba00640 1 -- 172.21.15.119:0/823922221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3640cab60 msgr2=0x7fc3640b86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.284+0000 7fc36ba00640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3640cab60 0x7fc3640b86b0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fc350007fd0 tx=0x7fc35000dc90 comp rx=0 tx=0).stop 2024-04-25T10:16:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.285+0000 7fc36ba00640 1 -- 172.21.15.119:0/823922221 shutdown_connections 2024-04-25T10:16:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.285+0000 7fc36ba00640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc34406b4d0 0x7fc34406d990 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.285+0000 7fc36ba00640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc3640b8bf0 0x7fc364155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.285+0000 7fc36ba00640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3640cab60 0x7fc3640b86b0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.285+0000 7fc36ba00640 1 --2- 172.21.15.119:0/823922221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3480039b0 0x7fc3640ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.285+0000 7fc36ba00640 1 -- 172.21.15.119:0/823922221 >> 172.21.15.119:0/823922221 conn(0x7fc3640726c0 msgr2=0x7fc3640b7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.285+0000 7fc36ba00640 1 -- 172.21.15.119:0/823922221 shutdown_connections 2024-04-25T10:16:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.285+0000 7fc36ba00640 1 -- 172.21.15.119:0/823922221 wait complete. 2024-04-25T10:16:02.308 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:16:02.370 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:02.376 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:02.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.494+0000 7f52ffe00640 1 Processor -- start 2024-04-25T10:16:02.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.494+0000 7f52ffe00640 1 -- start start 2024-04-25T10:16:02.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.495+0000 7f52ffe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f53001450e0 0x7f530006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.495+0000 7f52ffe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f530006b0e0 0x7f53000cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.495+0000 7f52ffe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f53000cc810 0x7f5300071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.495+0000 7f52ffe00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53001454c0 con 0x7f53001450e0 2024-04-25T10:16:02.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.496+0000 7f52ffe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530006b540 con 0x7f53000cc810 2024-04-25T10:16:02.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.496+0000 7f52ffe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530015f400 con 0x7f530006b0e0 2024-04-25T10:16:02.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.496+0000 7f52fea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f53001450e0 0x7f530006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.496+0000 7f52fea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f53001450e0 0x7f530006aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52936/0 (socket says 172.21.15.119:52936) 2024-04-25T10:16:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.496+0000 7f52fea00640 1 -- 172.21.15.119:0/514111552 learned_addr learned my addr 172.21.15.119:0/514111552 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.496+0000 7f52f7e00640 1 --2- 172.21.15.119:0/514111552 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f530006b0e0 0x7f53000cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.496+0000 7f52fea00640 1 -- 172.21.15.119:0/514111552 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f530006b0e0 msgr2=0x7f53000cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.496+0000 7f52ff400640 1 --2- 172.21.15.119:0/514111552 >> v2:172.21.15.132:3300/0 conn(0x7f53000cc810 0x7f5300071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.496+0000 7f52fea00640 1 --2- 172.21.15.119:0/514111552 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f530006b0e0 0x7f53000cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.496+0000 7f52fea00640 1 -- 172.21.15.119:0/514111552 >> v2:172.21.15.132:3300/0 conn(0x7f53000cc810 msgr2=0x7f5300071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.496+0000 7f52fea00640 1 --2- 172.21.15.119:0/514111552 >> v2:172.21.15.132:3300/0 conn(0x7f53000cc810 0x7f5300071050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.496+0000 7f52fea00640 1 -- 172.21.15.119:0/514111552 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530015f6e0 con 0x7f53001450e0 2024-04-25T10:16:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.496+0000 7f52f7e00640 1 --2- 172.21.15.119:0/514111552 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f530006b0e0 0x7f53000cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:02.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.498+0000 7f52fea00640 1 --2- 172.21.15.119:0/514111552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53001450e0 0x7f530006aba0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f53000cbe90 tx=0x7f52f002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39718a21b4bc06a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.498+0000 7f52fe000640 1 -- 172.21.15.119:0/514111552 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0037070 con 0x7f53001450e0 2024-04-25T10:16:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.498+0000 7f52fe000640 1 -- 172.21.15.119:0/514111552 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f002ae00 con 0x7f53001450e0 2024-04-25T10:16:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.498+0000 7f52fe000640 1 -- 172.21.15.119:0/514111552 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0032660 con 0x7f53001450e0 2024-04-25T10:16:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.498+0000 7f52ffe00640 1 -- 172.21.15.119:0/514111552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53001450e0 msgr2=0x7f530006aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.498+0000 7f52ffe00640 1 --2- 172.21.15.119:0/514111552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53001450e0 0x7f530006aba0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f53000cbe90 tx=0x7f52f002a0b0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.498+0000 7f52ffe00640 1 -- 172.21.15.119:0/514111552 shutdown_connections 2024-04-25T10:16:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.498+0000 7f52ffe00640 1 --2- 172.21.15.119:0/514111552 >> v2:172.21.15.132:3300/0 conn(0x7f53000cc810 0x7f5300071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.498+0000 7f52ffe00640 1 --2- 172.21.15.119:0/514111552 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f530006b0e0 0x7f53000cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.498+0000 7f52ffe00640 1 --2- 172.21.15.119:0/514111552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53001450e0 0x7f530006aba0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.498+0000 7f52ffe00640 1 -- 172.21.15.119:0/514111552 >> 172.21.15.119:0/514111552 conn(0x7f5300066ff0 msgr2=0x7f53000b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.499+0000 7f52ffe00640 1 -- 172.21.15.119:0/514111552 shutdown_connections 2024-04-25T10:16:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.499+0000 7f52ffe00640 1 -- 172.21.15.119:0/514111552 wait complete. 2024-04-25T10:16:02.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.500+0000 7f52ffe00640 1 Processor -- start 2024-04-25T10:16:02.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.500+0000 7f52ffe00640 1 -- start start 2024-04-25T10:16:02.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.501+0000 7f52ffe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f530006b0e0 0x7f530013bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.501+0000 7f52ffe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53000cc810 0x7f53000acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.501+0000 7f52ffe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53001450e0 0x7f53000ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.501+0000 7f52ffe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53000ccca0 con 0x7f53000cc810 2024-04-25T10:16:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.501+0000 7f52ffe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530015fca0 con 0x7f530006b0e0 2024-04-25T10:16:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.501+0000 7f52ffe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53000add10 con 0x7f53001450e0 2024-04-25T10:16:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.502+0000 7f52f7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53000cc810 0x7f53000acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.502+0000 7f52f7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53000cc810 0x7f53000acf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52950/0 (socket says 172.21.15.119:52950) 2024-04-25T10:16:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.502+0000 7f52f7e00640 1 -- 172.21.15.119:0/247222546 learned_addr learned my addr 172.21.15.119:0/247222546 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.502+0000 7f52ff400640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53001450e0 0x7f53000ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.502+0000 7f52fea00640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f530006b0e0 0x7f530013bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.502+0000 7f52f7e00640 1 -- 172.21.15.119:0/247222546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53001450e0 msgr2=0x7f53000ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.502+0000 7f52f7e00640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53001450e0 0x7f53000ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.502+0000 7f52f7e00640 1 -- 172.21.15.119:0/247222546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f530006b0e0 msgr2=0x7f530013bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.502+0000 7f52f7e00640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f530006b0e0 0x7f530013bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.502+0000 7f52f7e00640 1 -- 172.21.15.119:0/247222546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53000adff0 con 0x7f53000cc810 2024-04-25T10:16:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.502+0000 7f52fea00640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f530006b0e0 0x7f530013bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.502+0000 7f52f7e00640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53000cc810 0x7f53000acf40 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f52e800c7f0 tx=0x7f52e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.502+0000 7f52f7400640 1 -- 172.21.15.119:0/247222546 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8015070 con 0x7f53000cc810 2024-04-25T10:16:02.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.503+0000 7f52f7400640 1 -- 172.21.15.119:0/247222546 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e8007820 con 0x7f53000cc810 2024-04-25T10:16:02.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.503+0000 7f52f7400640 1 -- 172.21.15.119:0/247222546 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8007c10 con 0x7f53000cc810 2024-04-25T10:16:02.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.503+0000 7f52ffe00640 1 -- 172.21.15.119:0/247222546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53000ae2d0 con 0x7f53000cc810 2024-04-25T10:16:02.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.503+0000 7f52ffe00640 1 -- 172.21.15.119:0/247222546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53000ae890 con 0x7f53000cc810 2024-04-25T10:16:02.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.504+0000 7f52ffe00640 1 -- 172.21.15.119:0/247222546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53000749b0 con 0x7f53000cc810 2024-04-25T10:16:02.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.505+0000 7f52f7400640 1 -- 172.21.15.119:0/247222546 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f52e8018070 con 0x7f53000cc810 2024-04-25T10:16:02.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.506+0000 7f52f7400640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f52dc06b4d0 0x7f52dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:02.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.506+0000 7f52f7400640 1 -- 172.21.15.119:0/247222546 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52e808a3a0 con 0x7f53000cc810 2024-04-25T10:16:02.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.506+0000 7f52fea00640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f52dc06b4d0 0x7f52dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:02.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.507+0000 7f52fea00640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f52dc06b4d0 0x7f52dc06d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f52f0034000 tx=0x7f52f002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:02.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.510+0000 7f52f7400640 1 -- 172.21.15.119:0/247222546 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f52e8059190 con 0x7f53000cc810 2024-04-25T10:16:02.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.722+0000 7f52ffe00640 1 -- 172.21.15.119:0/247222546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f53000cd3c0 con 0x7f53000cc810 2024-04-25T10:16:02.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.723+0000 7f52f7400640 1 -- 172.21.15.119:0/247222546 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v96) v1 ==== 80+0+0 (secure 0 0 0) 0x7f52e8058a90 con 0x7f53000cc810 2024-04-25T10:16:02.727 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2024-04-25T10:16:02.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.726+0000 7f52ffe00640 1 -- 172.21.15.119:0/247222546 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f52dc06b4d0 msgr2=0x7f52dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.726+0000 7f52ffe00640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f52dc06b4d0 0x7f52dc06d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f52f0034000 tx=0x7f52f002e000 comp rx=0 tx=0).stop 2024-04-25T10:16:02.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.726+0000 7f52ffe00640 1 -- 172.21.15.119:0/247222546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53000cc810 msgr2=0x7f53000acf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:02.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.726+0000 7f52ffe00640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53000cc810 0x7f53000acf40 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f52e800c7f0 tx=0x7f52e800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.726+0000 7f52ffe00640 1 -- 172.21.15.119:0/247222546 shutdown_connections 2024-04-25T10:16:02.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.726+0000 7f52ffe00640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f52dc06b4d0 0x7f52dc06d990 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.726+0000 7f52ffe00640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53001450e0 0x7f53000ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.726+0000 7f52ffe00640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53000cc810 0x7f53000acf40 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.726+0000 7f52ffe00640 1 --2- 172.21.15.119:0/247222546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f530006b0e0 0x7f530013bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:02.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.726+0000 7f52ffe00640 1 -- 172.21.15.119:0/247222546 >> 172.21.15.119:0/247222546 conn(0x7f5300066ff0 msgr2=0x7f53000cf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:02.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.726+0000 7f52ffe00640 1 -- 172.21.15.119:0/247222546 shutdown_connections 2024-04-25T10:16:02.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:02.726+0000 7f52ffe00640 1 -- 172.21.15.119:0/247222546 wait complete. 2024-04-25T10:16:02.747 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:16:02.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:02.748 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:02.891 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:16:02.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:02.892 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:16:02.942 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:03.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.135+0000 7f1f11c00640 1 Processor -- start 2024-04-25T10:16:03.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.135+0000 7f1f11c00640 1 -- start start 2024-04-25T10:16:03.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.137+0000 7f1f11c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1f0c15d980 0x7f1f0c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.137+0000 7f1f11c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f0c0aeb20 0x7f1f0c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.137+0000 7f1f11c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c0712c0 con 0x7f1f0c0cf7c0 2024-04-25T10:16:03.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.137+0000 7f1f11c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c071ed0 con 0x7f1f0c15d980 2024-04-25T10:16:03.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.137+0000 7f1f11c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0cfbf0 con 0x7f1f0c0aeb20 2024-04-25T10:16:03.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.137+0000 7f1f0aa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1f0c15d980 0x7f1f0c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.137+0000 7f1f0aa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1f0c15d980 0x7f1f0c0ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52958/0 (socket says 172.21.15.119:52958) 2024-04-25T10:16:03.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.137+0000 7f1f0aa00640 1 -- 172.21.15.119:0/1711444114 learned_addr learned my addr 172.21.15.119:0/1711444114 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:03.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.138+0000 7f1f0a000640 1 --2- 172.21.15.119:0/1711444114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f0c0aeb20 0x7f1f0c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.138+0000 7f1f0aa00640 1 -- 172.21.15.119:0/1711444114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f0c0aeb20 msgr2=0x7f1f0c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:03.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.138+0000 7f1f0aa00640 1 --2- 172.21.15.119:0/1711444114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f0c0aeb20 0x7f1f0c0cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.138+0000 7f1f0aa00640 1 -- 172.21.15.119:0/1711444114 >> v1:172.21.15.132:6789/0 conn(0x7f1f0c0cf7c0 legacy=0x7f1f0c15dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.138+0000 7f1f0aa00640 1 -- 172.21.15.119:0/1711444114 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0715a0 con 0x7f1f0c15d980 2024-04-25T10:16:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.139+0000 7f1f0aa00640 1 --2- 172.21.15.119:0/1711444114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f0c15d980 0x7f1f0c0ae5e0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c072050 tx=0x7f1efc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686b0e9af76d3d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.140+0000 7f1f09600640 1 -- 172.21.15.119:0/1711444114 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc037070 con 0x7f1f0c15d980 2024-04-25T10:16:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.140+0000 7f1f09600640 1 -- 172.21.15.119:0/1711444114 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc032440 con 0x7f1f0c15d980 2024-04-25T10:16:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.140+0000 7f1f09600640 1 -- 172.21.15.119:0/1711444114 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc032810 con 0x7f1f0c15d980 2024-04-25T10:16:03.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.140+0000 7f1f11c00640 1 -- 172.21.15.119:0/1711444114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f0c15d980 msgr2=0x7f1f0c0ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.140+0000 7f1f11c00640 1 --2- 172.21.15.119:0/1711444114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f0c15d980 0x7f1f0c0ae5e0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c072050 tx=0x7f1efc02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.141+0000 7f1f11c00640 1 -- 172.21.15.119:0/1711444114 shutdown_connections 2024-04-25T10:16:03.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.141+0000 7f1f11c00640 1 --2- 172.21.15.119:0/1711444114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f0c0aeb20 0x7f1f0c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.141+0000 7f1f11c00640 1 --2- 172.21.15.119:0/1711444114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f0c15d980 0x7f1f0c0ae5e0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.141+0000 7f1f11c00640 1 -- 172.21.15.119:0/1711444114 >> 172.21.15.119:0/1711444114 conn(0x7f1f0c06aba0 msgr2=0x7f1f0c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.141+0000 7f1f11c00640 1 -- 172.21.15.119:0/1711444114 shutdown_connections 2024-04-25T10:16:03.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.141+0000 7f1f11c00640 1 -- 172.21.15.119:0/1711444114 wait complete. 2024-04-25T10:16:03.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.143+0000 7f1f11c00640 1 Processor -- start 2024-04-25T10:16:03.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.143+0000 7f1f11c00640 1 -- start start 2024-04-25T10:16:03.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.144+0000 7f1f11c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f0c0aeb20 0x7f1f0c146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.144+0000 7f1f11c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f0c0cf7c0 0x7f1f0c15bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.144+0000 7f1f11c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f0c15d980 0x7f1f0c15c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.144+0000 7f1f11c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0adf80 con 0x7f1f0c0cf7c0 2024-04-25T10:16:03.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.144+0000 7f1f11c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c071b60 con 0x7f1f0c0aeb20 2024-04-25T10:16:03.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.144+0000 7f1f11c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0ae100 con 0x7f1f0c15d980 2024-04-25T10:16:03.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.145+0000 7f1f0a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f0c0cf7c0 0x7f1f0c15bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.145+0000 7f1f0a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f0c0cf7c0 0x7f1f0c15bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52970/0 (socket says 172.21.15.119:52970) 2024-04-25T10:16:03.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.145+0000 7f1f0a000640 1 -- 172.21.15.119:0/1335111967 learned_addr learned my addr 172.21.15.119:0/1335111967 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:03.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.145+0000 7f1f0aa00640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f0c0aeb20 0x7f1f0c146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.145+0000 7f1f0b400640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f0c15d980 0x7f1f0c15c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.145+0000 7f1f0a000640 1 -- 172.21.15.119:0/1335111967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f0c15d980 msgr2=0x7f1f0c15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.145+0000 7f1f0a000640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f0c15d980 0x7f1f0c15c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.145+0000 7f1f0a000640 1 -- 172.21.15.119:0/1335111967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f0c0aeb20 msgr2=0x7f1f0c146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.145+0000 7f1f0a000640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f0c0aeb20 0x7f1f0c146c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.145+0000 7f1f0a000640 1 -- 172.21.15.119:0/1335111967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c1474f0 con 0x7f1f0c0cf7c0 2024-04-25T10:16:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.145+0000 7f1f0b400640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f0c15d980 0x7f1f0c15c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.145+0000 7f1f0aa00640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f0c0aeb20 0x7f1f0c146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.146+0000 7f1f0a000640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f0c0cf7c0 0x7f1f0c15bf00 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800e9b0 tx=0x7f1ef800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.146+0000 7f1f02a00640 1 -- 172.21.15.119:0/1335111967 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8017070 con 0x7f1f0c0cf7c0 2024-04-25T10:16:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.146+0000 7f1f02a00640 1 -- 172.21.15.119:0/1335111967 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef800c9a0 con 0x7f1f0c0cf7c0 2024-04-25T10:16:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.146+0000 7f1f02a00640 1 -- 172.21.15.119:0/1335111967 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef800cd90 con 0x7f1f0c0cf7c0 2024-04-25T10:16:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.146+0000 7f1f11c00640 1 -- 172.21.15.119:0/1335111967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c14a3b0 con 0x7f1f0c0cf7c0 2024-04-25T10:16:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.146+0000 7f1f11c00640 1 -- 172.21.15.119:0/1335111967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c147a90 con 0x7f1f0c0cf7c0 2024-04-25T10:16:03.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.147+0000 7f1f11c00640 1 -- 172.21.15.119:0/1335111967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c0749b0 con 0x7f1f0c0cf7c0 2024-04-25T10:16:03.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.148+0000 7f1f02a00640 1 -- 172.21.15.119:0/1335111967 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ef801a070 con 0x7f1f0c0cf7c0 2024-04-25T10:16:03.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.149+0000 7f1f02a00640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1eec06b4d0 0x7f1eec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.149+0000 7f1f02a00640 1 -- 172.21.15.119:0/1335111967 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ef808a250 con 0x7f1f0c0cf7c0 2024-04-25T10:16:03.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.149+0000 7f1f0aa00640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1eec06b4d0 0x7f1eec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.150+0000 7f1f0aa00640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1eec06b4d0 0x7f1eec06d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1efc034000 tx=0x7f1efc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.155+0000 7f1f02a00640 1 -- 172.21.15.119:0/1335111967 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ef80590c0 con 0x7f1f0c0cf7c0 2024-04-25T10:16:03.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.383+0000 7f1f11c00640 1 -- 172.21.15.119:0/1335111967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f0c06d4c0 con 0x7f1f0c0cf7c0 2024-04-25T10:16:03.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.384+0000 7f1f02a00640 1 -- 172.21.15.119:0/1335111967 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f1ef80589c0 con 0x7f1f0c0cf7c0 2024-04-25T10:16:03.388 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:03.388 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5459,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3768348114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3768348114},{"type":"v1","addr":"172.21.15.132:6834","nonce":3768348114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":5465,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/543190780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":543190780},{"type":"v1","addr":"172.21.15.132:6835","nonce":543190780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1145229172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1145229172},{"type":"v1","addr":"172.21.15.119:6836","nonce":1145229172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:51.568312+0000","modified":"2024-04-25T10:15:54.633087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1945631445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1945631445},{"type":"v1","addr":"172.21.15.119:6837","nonce":1945631445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:16:03.389 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 82 2024-04-25T10:16:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.387+0000 7f1f11c00640 1 -- 172.21.15.119:0/1335111967 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1eec06b4d0 msgr2=0x7f1eec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.387+0000 7f1f11c00640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1eec06b4d0 0x7f1eec06d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1efc034000 tx=0x7f1efc02e000 comp rx=0 tx=0).stop 2024-04-25T10:16:03.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.387+0000 7f1f11c00640 1 -- 172.21.15.119:0/1335111967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f0c0cf7c0 msgr2=0x7f1f0c15bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.387+0000 7f1f11c00640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f0c0cf7c0 0x7f1f0c15bf00 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800e9b0 tx=0x7f1ef800ee70 comp rx=0 tx=0).stop 2024-04-25T10:16:03.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.387+0000 7f1f11c00640 1 -- 172.21.15.119:0/1335111967 shutdown_connections 2024-04-25T10:16:03.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.387+0000 7f1f11c00640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1eec06b4d0 0x7f1eec06d990 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.387+0000 7f1f11c00640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f0c15d980 0x7f1f0c15c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.387+0000 7f1f11c00640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f0c0cf7c0 0x7f1f0c15bf00 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.387+0000 7f1f11c00640 1 --2- 172.21.15.119:0/1335111967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f0c0aeb20 0x7f1f0c146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.387+0000 7f1f11c00640 1 -- 172.21.15.119:0/1335111967 >> 172.21.15.119:0/1335111967 conn(0x7f1f0c06aba0 msgr2=0x7f1f0c15fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.387+0000 7f1f11c00640 1 -- 172.21.15.119:0/1335111967 shutdown_connections 2024-04-25T10:16:03.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.387+0000 7f1f11c00640 1 -- 172.21.15.119:0/1335111967 wait complete. 2024-04-25T10:16:03.408 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:16:03.635 INFO:teuthology.orchestra.run.smithi119.stdout:{"message":"","return_code":0} 2024-04-25T10:16:03.635 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:16:03.635 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:03.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.830+0000 7f2077200640 1 Processor -- start 2024-04-25T10:16:03.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.830+0000 7f2077200640 1 -- start start 2024-04-25T10:16:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.832+0000 7f2077200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2078144470 0x7f207815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.832+0000 7f2077200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20780cd440 con 0x7f207806e430 2024-04-25T10:16:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.832+0000 7f2077200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20780cd720 con 0x7f20780b5050 2024-04-25T10:16:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.832+0000 7f2077200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2078145340 con 0x7f2078144470 2024-04-25T10:16:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.832+0000 7f2075e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2078144470 0x7f207815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.832+0000 7f2075e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2078144470 0x7f207815fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52976/0 (socket says 172.21.15.119:52976) 2024-04-25T10:16:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.832+0000 7f2075e00640 1 -- 172.21.15.119:0/1229168947 learned_addr learned my addr 172.21.15.119:0/1229168947 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.832+0000 7f2075400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f207806e430 0x7f20780b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48796/0 (socket says 172.21.15.119:48796) 2024-04-25T10:16:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.833+0000 7f2075e00640 1 -- 172.21.15.119:0/1229168947 >> v1:172.21.15.132:6789/0 conn(0x7f20780b5050 legacy=0x7f207815bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.833+0000 7f2075e00640 1 -- 172.21.15.119:0/1229168947 >> v1:172.21.15.119:6789/0 conn(0x7f207806e430 legacy=0x7f20780b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.833+0000 7f2075e00640 1 -- 172.21.15.119:0/1229168947 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20780cda00 con 0x7f2078144470 2024-04-25T10:16:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.833+0000 7f2074a00640 1 -- 172.21.15.119:0/1229168947 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409977131 0 0) 0x7f20780cd440 con 0x7f207806e430 2024-04-25T10:16:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.834+0000 7f2075e00640 1 --2- 172.21.15.119:0/1229168947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078144470 0x7f207815fc90 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f2060009870 tx=0x7f206002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc405eb5fb40764 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.834+0000 7f2074a00640 1 -- 172.21.15.119:0/1229168947 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060037070 con 0x7f2078144470 2024-04-25T10:16:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.834+0000 7f2074a00640 1 -- 172.21.15.119:0/1229168947 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2060032440 con 0x7f2078144470 2024-04-25T10:16:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.834+0000 7f2074a00640 1 -- 172.21.15.119:0/1229168947 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060032810 con 0x7f2078144470 2024-04-25T10:16:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.834+0000 7f2077200640 1 -- 172.21.15.119:0/1229168947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078144470 msgr2=0x7f207815fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.834+0000 7f2077200640 1 --2- 172.21.15.119:0/1229168947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078144470 0x7f207815fc90 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f2060009870 tx=0x7f206002a2a0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.835+0000 7f2077200640 1 -- 172.21.15.119:0/1229168947 shutdown_connections 2024-04-25T10:16:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.835+0000 7f2077200640 1 --2- 172.21.15.119:0/1229168947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078144470 0x7f207815fc90 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.835+0000 7f2077200640 1 -- 172.21.15.119:0/1229168947 >> 172.21.15.119:0/1229168947 conn(0x7f207806b1d0 msgr2=0x7f20780b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.835+0000 7f2077200640 1 -- 172.21.15.119:0/1229168947 shutdown_connections 2024-04-25T10:16:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.835+0000 7f2077200640 1 -- 172.21.15.119:0/1229168947 wait complete. 2024-04-25T10:16:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.836+0000 7f2077200640 1 Processor -- start 2024-04-25T10:16:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.836+0000 7f2077200640 1 -- start start 2024-04-25T10:16:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.837+0000 7f2077200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f207806e430 0x7f20780c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.837+0000 7f2077200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20780b5050 0x7f20780aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.837+0000 7f2077200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078144470 0x7f20780adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.837+0000 7f2077200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20780ce200 con 0x7f2078144470 2024-04-25T10:16:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.837+0000 7f2077200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2078144850 con 0x7f207806e430 2024-04-25T10:16:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.837+0000 7f2077200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207815c400 con 0x7f20780b5050 2024-04-25T10:16:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.838+0000 7f2076800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078144470 0x7f20780adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.838+0000 7f2076800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078144470 0x7f20780adc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52978/0 (socket says 172.21.15.119:52978) 2024-04-25T10:16:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.838+0000 7f2076800640 1 -- 172.21.15.119:0/1018893814 learned_addr learned my addr 172.21.15.119:0/1018893814 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.838+0000 7f2075e00640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f207806e430 0x7f20780c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.838+0000 7f2075400640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20780b5050 0x7f20780aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.838+0000 7f2076800640 1 -- 172.21.15.119:0/1018893814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20780b5050 msgr2=0x7f20780aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.838+0000 7f2076800640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20780b5050 0x7f20780aadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.838+0000 7f2076800640 1 -- 172.21.15.119:0/1018893814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f207806e430 msgr2=0x7f20780c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.838+0000 7f2076800640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f207806e430 0x7f20780c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.838+0000 7f2076800640 1 -- 172.21.15.119:0/1018893814 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20780acba0 con 0x7f2078144470 2024-04-25T10:16:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.838+0000 7f2075e00640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f207806e430 0x7f20780c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.838+0000 7f2076800640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078144470 0x7f20780adc40 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f207000c440 tx=0x7f207000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.839+0000 7f206e000640 1 -- 172.21.15.119:0/1018893814 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2070017070 con 0x7f2078144470 2024-04-25T10:16:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.839+0000 7f206e000640 1 -- 172.21.15.119:0/1018893814 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20700035f0 con 0x7f2078144470 2024-04-25T10:16:03.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.839+0000 7f206e000640 1 -- 172.21.15.119:0/1018893814 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20700039e0 con 0x7f2078144470 2024-04-25T10:16:03.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.839+0000 7f2077200640 1 -- 172.21.15.119:0/1018893814 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20780ace80 con 0x7f2078144470 2024-04-25T10:16:03.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.839+0000 7f2077200640 1 -- 172.21.15.119:0/1018893814 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20780ad440 con 0x7f2078144470 2024-04-25T10:16:03.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.840+0000 7f2077200640 1 -- 172.21.15.119:0/1018893814 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20780b24a0 con 0x7f2078144470 2024-04-25T10:16:03.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.841+0000 7f206e000640 1 -- 172.21.15.119:0/1018893814 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f207001a070 con 0x7f2078144470 2024-04-25T10:16:03.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.842+0000 7f206e000640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f205406b500 0x7f205406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.842+0000 7f206e000640 1 -- 172.21.15.119:0/1018893814 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2070089530 con 0x7f2078144470 2024-04-25T10:16:03.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.842+0000 7f2075e00640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f205406b500 0x7f205406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.843+0000 7f2075e00640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f205406b500 0x7f205406d9c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2060034000 tx=0x7f206002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:03.846+0000 7f206e000640 1 -- 172.21.15.119:0/1018893814 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20780b24a0 con 0x7f2078144470 2024-04-25T10:16:04.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.063+0000 7f2077200640 1 -- 172.21.15.119:0/1018893814 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20780749b0 con 0x7f2078144470 2024-04-25T10:16:04.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.064+0000 7f206e000640 1 -- 172.21.15.119:0/1018893814 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f2070058420 con 0x7f2078144470 2024-04-25T10:16:04.068 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5459,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3768348114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3768348114},{"type":"v1","addr":"172.21.15.132:6834","nonce":3768348114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":5465,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/543190780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":543190780},{"type":"v1","addr":"172.21.15.132:6835","nonce":543190780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1145229172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1145229172},{"type":"v1","addr":"172.21.15.119:6836","nonce":1145229172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:51.568312+0000","modified":"2024-04-25T10:15:54.633087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1945631445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1945631445},{"type":"v1","addr":"172.21.15.119:6837","nonce":1945631445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:16:04.069 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 82 2024-04-25T10:16:04.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.067+0000 7f2077200640 1 -- 172.21.15.119:0/1018893814 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f205406b500 msgr2=0x7f205406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.067+0000 7f2077200640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f205406b500 0x7f205406d9c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2060034000 tx=0x7f206002e000 comp rx=0 tx=0).stop 2024-04-25T10:16:04.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.067+0000 7f2077200640 1 -- 172.21.15.119:0/1018893814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078144470 msgr2=0x7f20780adc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.067+0000 7f2077200640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078144470 0x7f20780adc40 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f207000c440 tx=0x7f207000c900 comp rx=0 tx=0).stop 2024-04-25T10:16:04.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.067+0000 7f2077200640 1 -- 172.21.15.119:0/1018893814 shutdown_connections 2024-04-25T10:16:04.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.067+0000 7f2077200640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f205406b500 0x7f205406d9c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.067+0000 7f2077200640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2078144470 0x7f20780adc40 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.067+0000 7f2077200640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f20780b5050 0x7f20780aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.067+0000 7f2077200640 1 --2- 172.21.15.119:0/1018893814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f207806e430 0x7f20780c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.067+0000 7f2077200640 1 -- 172.21.15.119:0/1018893814 >> 172.21.15.119:0/1018893814 conn(0x7f207806b1d0 msgr2=0x7f207815cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:04.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.067+0000 7f2077200640 1 -- 172.21.15.119:0/1018893814 shutdown_connections 2024-04-25T10:16:04.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.067+0000 7f2077200640 1 -- 172.21.15.119:0/1018893814 wait complete. 2024-04-25T10:16:04.089 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-25T10:16:04.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.193+0000 7f0b26920f40 1 Processor -- start 2024-04-25T10:16:04.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.193+0000 7f0b26920f40 1 -- start start 2024-04-25T10:16:04.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.194+0000 7f0b26920f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b41262ff0 0x563b41302130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.194+0000 7f0b26920f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b41265020 0x563b41267450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.194+0000 7f0b26920f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b41305ff0 0x563b4130b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.194+0000 7f0b26920f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b411dddd0 con 0x563b41305ff0 2024-04-25T10:16:04.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.194+0000 7f0b26920f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b411e1d30 con 0x563b41262ff0 2024-04-25T10:16:04.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.194+0000 7f0b26920f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b41266de0 con 0x563b41265020 2024-04-25T10:16:04.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.194+0000 7f0b23c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b41305ff0 0x563b4130b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.194+0000 7f0b23c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b41305ff0 0x563b4130b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52996/0 (socket says 172.21.15.119:52996) 2024-04-25T10:16:04.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.194+0000 7f0b23c00640 1 -- 172.21.15.119:0/3487407822 learned_addr learned my addr 172.21.15.119:0/3487407822 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:04.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.195+0000 7f0b22800640 1 --2- 172.21.15.119:0/3487407822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b41265020 0x563b41267450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.195+0000 7f0b23c00640 1 -- 172.21.15.119:0/3487407822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b41265020 msgr2=0x563b41267450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:04.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.195+0000 7f0b23c00640 1 --2- 172.21.15.119:0/3487407822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b41265020 0x563b41267450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.195+0000 7f0b23c00640 1 -- 172.21.15.119:0/3487407822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b41262ff0 msgr2=0x563b41302130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.195+0000 7f0b23200640 1 --2- 172.21.15.119:0/3487407822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b41262ff0 0x563b41302130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.195+0000 7f0b23c00640 1 --2- 172.21.15.119:0/3487407822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b41262ff0 0x563b41302130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.195+0000 7f0b23c00640 1 -- 172.21.15.119:0/3487407822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b41302670 con 0x563b41305ff0 2024-04-25T10:16:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.195+0000 7f0b23c00640 1 --2- 172.21.15.119:0/3487407822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b41305ff0 0x563b4130b0e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c002d60 tx=0x7f0b1c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582cd8ac6fd84868 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.195+0000 7f0b21400640 1 -- 172.21.15.119:0/3487407822 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c013020 con 0x563b41305ff0 2024-04-25T10:16:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.195+0000 7f0b21400640 1 -- 172.21.15.119:0/3487407822 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c00f400 con 0x563b41305ff0 2024-04-25T10:16:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.195+0000 7f0b21400640 1 -- 172.21.15.119:0/3487407822 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c00f7f0 con 0x563b41305ff0 2024-04-25T10:16:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.196+0000 7f0b26920f40 1 -- 172.21.15.119:0/3487407822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b41305ff0 msgr2=0x563b4130b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.196+0000 7f0b26920f40 1 --2- 172.21.15.119:0/3487407822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b41305ff0 0x563b4130b0e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c002d60 tx=0x7f0b1c00b5b0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.196+0000 7f0b26920f40 1 -- 172.21.15.119:0/3487407822 shutdown_connections 2024-04-25T10:16:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.196+0000 7f0b26920f40 1 --2- 172.21.15.119:0/3487407822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b41305ff0 0x563b4130b0e0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.196+0000 7f0b26920f40 1 --2- 172.21.15.119:0/3487407822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b41265020 0x563b41267450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.196+0000 7f0b26920f40 1 --2- 172.21.15.119:0/3487407822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b41262ff0 0x563b41302130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.196+0000 7f0b26920f40 1 -- 172.21.15.119:0/3487407822 >> 172.21.15.119:0/3487407822 conn(0x563b41239980 msgr2=0x563b4130a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:04.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.196+0000 7f0b26920f40 1 -- 172.21.15.119:0/3487407822 shutdown_connections 2024-04-25T10:16:04.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.196+0000 7f0b26920f40 1 -- 172.21.15.119:0/3487407822 wait complete. 2024-04-25T10:16:04.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.197+0000 7f0b26920f40 1 Processor -- start 2024-04-25T10:16:04.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.197+0000 7f0b26920f40 1 -- start start 2024-04-25T10:16:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.198+0000 7f0b26920f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b41262ff0 0x563b41260070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.198+0000 7f0b26920f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b41265020 0x563b4130a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.198+0000 7f0b26920f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b41305ff0 0x563b41261220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.198+0000 7f0b26920f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b412607d0 con 0x563b41262ff0 2024-04-25T10:16:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.198+0000 7f0b26920f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b4130ea70 con 0x563b41305ff0 2024-04-25T10:16:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.198+0000 7f0b26920f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b4123a190 con 0x563b41265020 2024-04-25T10:16:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.198+0000 7f0b23200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b41262ff0 0x563b41260070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.198+0000 7f0b23200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b41262ff0 0x563b41260070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53000/0 (socket says 172.21.15.119:53000) 2024-04-25T10:16:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.198+0000 7f0b23200640 1 -- 172.21.15.119:0/3652811970 learned_addr learned my addr 172.21.15.119:0/3652811970 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.198+0000 7f0b23200640 1 -- 172.21.15.119:0/3652811970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b41265020 msgr2=0x563b4130a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.198+0000 7f0b23200640 1 --2- 172.21.15.119:0/3652811970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x563b41265020 0x563b4130a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.198+0000 7f0b23200640 1 -- 172.21.15.119:0/3652811970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b41305ff0 msgr2=0x563b41261220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.198+0000 7f0b23200640 1 --2- 172.21.15.119:0/3652811970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b41305ff0 0x563b41261220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.198+0000 7f0b23200640 1 -- 172.21.15.119:0/3652811970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b4125f5d0 con 0x563b41262ff0 2024-04-25T10:16:04.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.199+0000 7f0b23200640 1 --2- 172.21.15.119:0/3652811970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b41262ff0 0x563b41260070 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f0b14055040 tx=0x7f0b14003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.199+0000 7f0b1be00640 1 -- 172.21.15.119:0/3652811970 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14061070 con 0x563b41262ff0 2024-04-25T10:16:04.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.199+0000 7f0b1be00640 1 -- 172.21.15.119:0/3652811970 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1405b420 con 0x563b41262ff0 2024-04-25T10:16:04.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.199+0000 7f0b26920f40 1 -- 172.21.15.119:0/3652811970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563b4130a9c0 con 0x563b41262ff0 2024-04-25T10:16:04.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.199+0000 7f0b1be00640 1 -- 172.21.15.119:0/3652811970 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14056040 con 0x563b41262ff0 2024-04-25T10:16:04.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.199+0000 7f0b26920f40 1 -- 172.21.15.119:0/3652811970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b4130aca0 con 0x563b41262ff0 2024-04-25T10:16:04.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.201+0000 7f0b1be00640 1 -- 172.21.15.119:0/3652811970 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b14059020 con 0x563b41262ff0 2024-04-25T10:16:04.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.202+0000 7f0b1be00640 1 --2- 172.21.15.119:0/3652811970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b00069ee0 0x7f0b0006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.202+0000 7f0b1be00640 1 -- 172.21.15.119:0/3652811970 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b140cc650 con 0x563b41262ff0 2024-04-25T10:16:04.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.202+0000 7f0b22800640 1 --2- 172.21.15.119:0/3652811970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b00069ee0 0x7f0b0006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.202+0000 7f0b26920f40 1 --2- 172.21.15.119:0/3652811970 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x563b41261760 0x563b4125e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.202+0000 7f0b26920f40 1 -- 172.21.15.119:0/3652811970 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) v9 -- 0x563b413004d0 con 0x563b41261760 2024-04-25T10:16:04.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.203+0000 7f0b23c00640 1 --2- 172.21.15.119:0/3652811970 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x563b41261760 0x563b4125e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.203+0000 7f0b22800640 1 --2- 172.21.15.119:0/3652811970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b00069ee0 0x7f0b0006c3a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c004170 tx=0x7f0b0c001310 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.203+0000 7f0b23c00640 1 --2- 172.21.15.119:0/3652811970 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x563b41261760 0x563b4125e310 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.204+0000 7f0b23c00640 1 -- 172.21.15.119:0/3652811970 <== osd.2 v2:172.21.15.119:6824/1001762963 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7f0b1c00bd20 con 0x563b41261760 2024-04-25T10:16:04.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.204+0000 7f0b26920f40 1 -- 172.21.15.119:0/3652811970 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x563b41261760 msgr2=0x563b4125e310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.204+0000 7f0b26920f40 1 --2- 172.21.15.119:0/3652811970 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x563b41261760 0x563b4125e310 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.204+0000 7f0b26920f40 1 -- 172.21.15.119:0/3652811970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b00069ee0 msgr2=0x7f0b0006c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.204+0000 7f0b26920f40 1 --2- 172.21.15.119:0/3652811970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b00069ee0 0x7f0b0006c3a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c004170 tx=0x7f0b0c001310 comp rx=0 tx=0).stop 2024-04-25T10:16:04.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.204+0000 7f0b26920f40 1 -- 172.21.15.119:0/3652811970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b41262ff0 msgr2=0x563b41260070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.204+0000 7f0b26920f40 1 --2- 172.21.15.119:0/3652811970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b41262ff0 0x563b41260070 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f0b14055040 tx=0x7f0b14003330 comp rx=0 tx=0).stop 2024-04-25T10:16:04.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.204+0000 7f0b23c00640 1 -- 172.21.15.119:0/3652811970 reap_dead start 2024-04-25T10:16:04.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.204+0000 7f0b26920f40 1 -- 172.21.15.119:0/3652811970 shutdown_connections 2024-04-25T10:16:04.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.204+0000 7f0b26920f40 1 -- 172.21.15.119:0/3652811970 >> 172.21.15.119:0/3652811970 conn(0x563b41239980 msgr2=0x563b4130cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:04.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.204+0000 7f0b26920f40 1 -- 172.21.15.119:0/3652811970 shutdown_connections 2024-04-25T10:16:04.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.204+0000 7f0b26920f40 1 -- 172.21.15.119:0/3652811970 wait complete. 2024-04-25T10:16:04.212 INFO:teuthology.orchestra.run.smithi119.stdout:����� 2024-04-25T10:16:04.212 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:16:04.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.387+0000 7f969d200640 1 Processor -- start 2024-04-25T10:16:04.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.388+0000 7f969d200640 1 -- start start 2024-04-25T10:16:04.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.389+0000 7f969d200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f969815bf00 0x7f96980cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.389+0000 7f969d200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96980cc310 0x7f969815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.389+0000 7f969d200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96980ce6c0 con 0x7f969815c820 2024-04-25T10:16:04.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.389+0000 7f969d200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96981455e0 con 0x7f969815bf00 2024-04-25T10:16:04.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.389+0000 7f969d200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969806d450 con 0x7f96980cc310 2024-04-25T10:16:04.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.389+0000 7f9696800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f969815c820 0x7f969806d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48814/0 (socket says 172.21.15.119:48814) 2024-04-25T10:16:04.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.389+0000 7f9696800640 1 -- 172.21.15.119:0/2607527319 learned_addr learned my addr 172.21.15.119:0/2607527319 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:16:04.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f968fe00640 1 --2- 172.21.15.119:0/2607527319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96980cc310 0x7f969815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695e00640 1 --2- 172.21.15.119:0/2607527319 >> v2:172.21.15.132:3300/0 conn(0x7f969815bf00 0x7f96980cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 -- 172.21.15.119:0/2607527319 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205310294 0 0) 0x7f9690001490 con 0x7f969815c820 2024-04-25T10:16:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 -- 172.21.15.119:0/2607527319 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f967c003520 con 0x7f969815c820 2024-04-25T10:16:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 -- 172.21.15.119:0/2607527319 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3726173818 0 0) 0x7f967c003520 con 0x7f969815c820 2024-04-25T10:16:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 -- 172.21.15.119:0/2607527319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96980cc310 msgr2=0x7f969815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 --2- 172.21.15.119:0/2607527319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96980cc310 0x7f969815c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 -- 172.21.15.119:0/2607527319 >> v2:172.21.15.132:3300/0 conn(0x7f969815bf00 msgr2=0x7f96980cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 --2- 172.21.15.119:0/2607527319 >> v2:172.21.15.132:3300/0 conn(0x7f969815bf00 0x7f96980cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 -- 172.21.15.119:0/2607527319 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96980cc7b0 con 0x7f969815c820 2024-04-25T10:16:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 -- 172.21.15.119:0/2607527319 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f9690001bc0 con 0x7f969815c820 2024-04-25T10:16:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695e00640 1 --2- 172.21.15.119:0/2607527319 >> v2:172.21.15.132:3300/0 conn(0x7f969815bf00 0x7f96980cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:04.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 -- 172.21.15.119:0/2607527319 >> v1:172.21.15.119:6789/0 conn(0x7f969815c820 legacy=0x7f969806d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 --2- 172.21.15.119:0/2607527319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f967c0039b0 0x7f967c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 --2- 172.21.15.119:0/2607527319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f967c004800 0x7f967c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 --2- 172.21.15.119:0/2607527319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96980cc310 0x7f967c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 -- 172.21.15.119:0/2607527319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96981455e0 con 0x7f967c0039b0 2024-04-25T10:16:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 -- 172.21.15.119:0/2607527319 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969806d450 con 0x7f967c004800 2024-04-25T10:16:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.390+0000 7f9695400640 1 -- 172.21.15.119:0/2607527319 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967c00ba10 con 0x7f96980cc310 2024-04-25T10:16:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.391+0000 7f9695e00640 1 --2- 172.21.15.119:0/2607527319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f967c0039b0 0x7f967c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.391+0000 7f9695e00640 1 -- 172.21.15.119:0/2607527319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96980cc310 msgr2=0x7f967c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.391+0000 7f9695e00640 1 --2- 172.21.15.119:0/2607527319 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96980cc310 0x7f967c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.391+0000 7f9695e00640 1 -- 172.21.15.119:0/2607527319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f967c004800 msgr2=0x7f967c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.391+0000 7f9695e00640 1 --2- 172.21.15.119:0/2607527319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f967c004800 0x7f967c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.391+0000 7f9695e00640 1 -- 172.21.15.119:0/2607527319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c003520 con 0x7f967c0039b0 2024-04-25T10:16:04.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.392+0000 7f9695e00640 1 --2- 172.21.15.119:0/2607527319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f967c0039b0 0x7f967c003dd0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f96880098a0 tx=0x7f967c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c867ddd5948dbe45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.392+0000 7f9695400640 1 -- 172.21.15.119:0/2607527319 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968800f040 con 0x7f967c0039b0 2024-04-25T10:16:04.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.392+0000 7f9695400640 1 -- 172.21.15.119:0/2607527319 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968802ad20 con 0x7f967c0039b0 2024-04-25T10:16:04.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.392+0000 7f9695400640 1 -- 172.21.15.119:0/2607527319 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688004250 con 0x7f967c0039b0 2024-04-25T10:16:04.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.393+0000 7f969d200640 1 -- 172.21.15.119:0/2607527319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f967c0039b0 msgr2=0x7f967c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.393+0000 7f969d200640 1 --2- 172.21.15.119:0/2607527319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f967c0039b0 0x7f967c003dd0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f96880098a0 tx=0x7f967c005830 comp rx=0 tx=0).stop 2024-04-25T10:16:04.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.393+0000 7f9696800640 1 -- 172.21.15.119:0/2607527319 reap_dead start 2024-04-25T10:16:04.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.393+0000 7f969d200640 1 -- 172.21.15.119:0/2607527319 shutdown_connections 2024-04-25T10:16:04.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.393+0000 7f969d200640 1 -- 172.21.15.119:0/2607527319 >> 172.21.15.119:0/2607527319 conn(0x7f969806aba0 msgr2=0x7f969806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:04.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.393+0000 7f969d200640 1 -- 172.21.15.119:0/2607527319 shutdown_connections 2024-04-25T10:16:04.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.393+0000 7f969d200640 1 -- 172.21.15.119:0/2607527319 wait complete. 2024-04-25T10:16:04.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.395+0000 7f969d200640 1 Processor -- start 2024-04-25T10:16:04.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.395+0000 7f969d200640 1 -- start start 2024-04-25T10:16:04.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.396+0000 7f969d200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f967c0039b0 0x7f96981f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.396+0000 7f969d200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9698141d60 0x7f96981e9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.396+0000 7f969d200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96981421e0 0x7f96981f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.396+0000 7f969d200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96980cd050 con 0x7f967c0039b0 2024-04-25T10:16:04.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.396+0000 7f969d200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96980ccd70 con 0x7f96981421e0 2024-04-25T10:16:04.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.396+0000 7f969d200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969815cca0 con 0x7f9698141d60 2024-04-25T10:16:04.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.396+0000 7f9695e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f967c0039b0 0x7f96981f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.396+0000 7f9695e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f967c0039b0 0x7f96981f40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53028/0 (socket says 172.21.15.119:53028) 2024-04-25T10:16:04.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.396+0000 7f9695e00640 1 -- 172.21.15.119:0/3637166528 learned_addr learned my addr 172.21.15.119:0/3637166528 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:04.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.397+0000 7f9696800640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96981421e0 0x7f96981f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.397+0000 7f968fe00640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9698141d60 0x7f96981e9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.397+0000 7f9695e00640 1 -- 172.21.15.119:0/3637166528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9698141d60 msgr2=0x7f96981e9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.397+0000 7f9695e00640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9698141d60 0x7f96981e9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.397+0000 7f9695e00640 1 -- 172.21.15.119:0/3637166528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96981421e0 msgr2=0x7f96981f68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.397+0000 7f9695e00640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96981421e0 0x7f96981f68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.397+0000 7f9695e00640 1 -- 172.21.15.119:0/3637166528 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9698142740 con 0x7f967c0039b0 2024-04-25T10:16:04.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.397+0000 7f9696800640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96981421e0 0x7f96981f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:04.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.397+0000 7f968fe00640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9698141d60 0x7f96981e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:04.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.397+0000 7f9695e00640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f967c0039b0 0x7f96981f40d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f9688009990 tx=0x7f9688004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.397+0000 7f968e000640 1 -- 172.21.15.119:0/3637166528 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688041070 con 0x7f967c0039b0 2024-04-25T10:16:04.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.397+0000 7f968e000640 1 -- 172.21.15.119:0/3637166528 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968802ab20 con 0x7f967c0039b0 2024-04-25T10:16:04.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.397+0000 7f968e000640 1 -- 172.21.15.119:0/3637166528 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968803d430 con 0x7f967c0039b0 2024-04-25T10:16:04.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.397+0000 7f969d200640 1 -- 172.21.15.119:0/3637166528 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9698142a20 con 0x7f967c0039b0 2024-04-25T10:16:04.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.398+0000 7f969d200640 1 -- 172.21.15.119:0/3637166528 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9698142fe0 con 0x7f967c0039b0 2024-04-25T10:16:04.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.399+0000 7f969d200640 1 -- 172.21.15.119:0/3637166528 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969806bb30 con 0x7f967c0039b0 2024-04-25T10:16:04.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.399+0000 7f968e000640 1 -- 172.21.15.119:0/3637166528 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9688044070 con 0x7f967c0039b0 2024-04-25T10:16:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.401+0000 7f968e000640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f967406b4d0 0x7f967406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.401+0000 7f968e000640 1 -- 172.21.15.119:0/3637166528 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96880aad20 con 0x7f967c0039b0 2024-04-25T10:16:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.401+0000 7f968fe00640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f967406b4d0 0x7f967406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:04.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.402+0000 7f968fe00640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f967406b4d0 0x7f967406d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f969815b180 tx=0x7f9680008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:04.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.405+0000 7f968e000640 1 -- 172.21.15.119:0/3637166528 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9688079a70 con 0x7f967c0039b0 2024-04-25T10:16:04.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.641+0000 7f969d200640 1 -- 172.21.15.119:0/3637166528 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96980749b0 con 0x7f967c0039b0 2024-04-25T10:16:04.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.642+0000 7f968e000640 1 -- 172.21.15.119:0/3637166528 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7f968802acf0 con 0x7f967c0039b0 2024-04-25T10:16:04.645 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:16:04.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.644+0000 7f969d200640 1 -- 172.21.15.119:0/3637166528 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f967406b4d0 msgr2=0x7f967406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.644+0000 7f969d200640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f967406b4d0 0x7f967406d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f969815b180 tx=0x7f9680008040 comp rx=0 tx=0).stop 2024-04-25T10:16:04.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.644+0000 7f969d200640 1 -- 172.21.15.119:0/3637166528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f967c0039b0 msgr2=0x7f96981f40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.644+0000 7f969d200640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f967c0039b0 0x7f96981f40d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f9688009990 tx=0x7f9688004730 comp rx=0 tx=0).stop 2024-04-25T10:16:04.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.644+0000 7f969d200640 1 -- 172.21.15.119:0/3637166528 shutdown_connections 2024-04-25T10:16:04.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.644+0000 7f969d200640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f967406b4d0 0x7f967406d990 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.644+0000 7f969d200640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96981421e0 0x7f96981f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.644+0000 7f969d200640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9698141d60 0x7f96981e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.644+0000 7f969d200640 1 --2- 172.21.15.119:0/3637166528 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f967c0039b0 0x7f96981f40d0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.644+0000 7f969d200640 1 -- 172.21.15.119:0/3637166528 >> 172.21.15.119:0/3637166528 conn(0x7f969806aba0 msgr2=0x7f969806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:04.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.645+0000 7f969d200640 1 -- 172.21.15.119:0/3637166528 shutdown_connections 2024-04-25T10:16:04.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:04.645+0000 7f969d200640 1 -- 172.21.15.119:0/3637166528 wait complete. 2024-04-25T10:16:04.667 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:16:04.667 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:16:04.667 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:16:04.667 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:16:04.667 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:04.667 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:16:04.667 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:16:04.667 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:16:04.667 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:04.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:04.668 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:16:04.720 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:16:04.720 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:16:04.720 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:16:04.720 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:04.720 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:16:04.720 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:04.720 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:16:04.720 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:04.720 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42125sec preferred_lft 42125sec 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:16:04.721 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:04.722 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:16:04.722 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:04.722 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:04.722 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:04.722 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:16:04.722 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:04.722 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:04.722 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:04.722 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:16:04.722 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:04.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:04.723 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:16:04.799 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:16:04.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:04.804 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:16:04.854 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:16:04.854 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:16:04.854 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:16:04.854 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:16:04.854 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:16:04.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:04.854 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:16:04.854 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:16:04.855 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:16:04.855 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:16:04.855 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:16:04.855 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:16:04.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:04.998 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:16:05.047 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:16:05.047 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:16:05.048 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:05.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:05.049 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:16:05.130 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:05.130 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:05.183 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:05.184 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:05.184 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:05.236 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:16:05.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:05.236 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:05.365 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:05.365 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:05.365 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:16:05.384 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:05.384 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:05.384 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:16:05.385 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:16:05.385 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:16:05.409 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:05.409 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:05.409 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:16:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:16:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:16:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:16:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:16:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:16:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:16:05.411 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:16:05.412 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:16:05.412 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:16:05.412 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:16:05.415 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:16:05.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:05.416 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:05.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:05.487 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:16:05.541 INFO:teuthology.orchestra.run.smithi119.stdout:client5528 2024-04-25T10:16:05.541 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:16:05.541 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:05.588 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:16:05.588 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:05.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:05.686 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:16:05.889 INFO:teuthology.orchestra.run.smithi119.stderr:6+0 records in 2024-04-25T10:16:05.890 INFO:teuthology.orchestra.run.smithi119.stderr:6+0 records out 2024-04-25T10:16:05.890 INFO:teuthology.orchestra.run.smithi119.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.143558 s, 43.8 MB/s 2024-04-25T10:16:05.891 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:16:05.891 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:05.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:05.944 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:16:06.168 INFO:teuthology.orchestra.run.smithi119.stderr:6+0 records in 2024-04-25T10:16:06.168 INFO:teuthology.orchestra.run.smithi119.stderr:6+0 records out 2024-04-25T10:16:06.168 INFO:teuthology.orchestra.run.smithi119.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.172856 s, 36.4 MB/s 2024-04-25T10:16:06.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:06.170 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:16:06.222 INFO:teuthology.orchestra.run.smithi119.stdout:./ 2024-04-25T10:16:06.222 INFO:teuthology.orchestra.run.smithi119.stdout:./file2_sixmegs 2024-04-25T10:16:06.222 INFO:teuthology.orchestra.run.smithi119.stdout:./file3_sixmegs 2024-04-25T10:16:06.222 INFO:teuthology.orchestra.run.smithi119.stdout:./file1_sixmegs 2024-04-25T10:16:06.223 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:16:06.223 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:16:06.223 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:16:06.224 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:16:06.224 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:16:06.224 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:16:06.360 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:16:06.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:06.373 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:16:06.373 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:16:06.373 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:16:06.373 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:16:06.373 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-25T10:16:06.373 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:16:06.557 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627777 2024-04-25T10:16:06.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:06.570 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:16:06.570 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:16:06.570 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:16:06.571 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:16:06.571 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-25T10:16:06.571 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:16:06.754 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627778 2024-04-25T10:16:06.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:06.767 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:16:06.767 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:16:06.768 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:16:06.768 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:16:06.768 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-25T10:16:06.768 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:16:06.951 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627776 2024-04-25T10:16:06.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:16:06.965 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:16:06.965 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:07.061 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:16:07.061 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:07.116 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:07.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.315+0000 7f4fcfe00640 1 Processor -- start 2024-04-25T10:16:07.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.315+0000 7f4fcfe00640 1 -- start start 2024-04-25T10:16:07.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.316+0000 7f4fcfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc8071190 0x7f4fc80ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.316+0000 7f4fcfe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4fc8071570 0x7f4fc815daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.316+0000 7f4fcfe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc806d550 con 0x7f4fc815e030 2024-04-25T10:16:07.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.316+0000 7f4fcfe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc80aeee0 con 0x7f4fc8071570 2024-04-25T10:16:07.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.316+0000 7f4fcfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc806d830 con 0x7f4fc8071190 2024-04-25T10:16:07.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.317+0000 7f4fcd400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc8071190 0x7f4fc80ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.317+0000 7f4fcde00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f4fc815e030 0x7f4fc80b2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:37084/0 (socket says 172.21.15.119:37084) 2024-04-25T10:16:07.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.317+0000 7f4fcde00640 1 -- 172.21.15.119:0/3939726261 learned_addr learned my addr 172.21.15.119:0/3939726261 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:16:07.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.317+0000 7f4fcca00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4fc8071570 0x7f4fc815daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.318+0000 7f4fc7e00640 1 -- 172.21.15.119:0/3939726261 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680942960 0 0) 0x7f4fc806d550 con 0x7f4fc815e030 2024-04-25T10:16:07.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.318+0000 7f4fc7e00640 1 -- 172.21.15.119:0/3939726261 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb0005730 con 0x7f4fc815e030 2024-04-25T10:16:07.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.318+0000 7f4fcca00640 1 -- 172.21.15.119:0/3939726261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc8071190 msgr2=0x7f4fc80ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:07.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.318+0000 7f4fcca00640 1 --2- 172.21.15.119:0/3939726261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc8071190 0x7f4fc80ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.318+0000 7f4fcca00640 1 -- 172.21.15.119:0/3939726261 >> v1:172.21.15.132:6789/0 conn(0x7f4fc815e030 legacy=0x7f4fc80b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.318+0000 7f4fcca00640 1 -- 172.21.15.119:0/3939726261 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc806db10 con 0x7f4fc8071570 2024-04-25T10:16:07.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.318+0000 7f4fcd400640 1 --2- 172.21.15.119:0/3939726261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc8071190 0x7f4fc80ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:16:07.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.319+0000 7f4fcca00640 1 --2- 172.21.15.119:0/3939726261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc8071570 0x7f4fc815daf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80097a0 tx=0x7f4fb802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6462799a68b2a3ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.320+0000 7f4fc7e00640 1 -- 172.21.15.119:0/3939726261 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8037070 con 0x7f4fc8071570 2024-04-25T10:16:07.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.320+0000 7f4fc7e00640 1 -- 172.21.15.119:0/3939726261 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb8032440 con 0x7f4fc8071570 2024-04-25T10:16:07.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.320+0000 7f4fc7e00640 1 -- 172.21.15.119:0/3939726261 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8032810 con 0x7f4fc8071570 2024-04-25T10:16:07.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.320+0000 7f4fcfe00640 1 -- 172.21.15.119:0/3939726261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc8071570 msgr2=0x7f4fc815daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.320+0000 7f4fcfe00640 1 --2- 172.21.15.119:0/3939726261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc8071570 0x7f4fc815daf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80097a0 tx=0x7f4fb802a180 comp rx=0 tx=0).stop 2024-04-25T10:16:07.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.320+0000 7f4fcfe00640 1 -- 172.21.15.119:0/3939726261 shutdown_connections 2024-04-25T10:16:07.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.320+0000 7f4fcfe00640 1 --2- 172.21.15.119:0/3939726261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc8071570 0x7f4fc815daf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.320+0000 7f4fcfe00640 1 --2- 172.21.15.119:0/3939726261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc8071190 0x7f4fc80ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.320+0000 7f4fcfe00640 1 -- 172.21.15.119:0/3939726261 >> 172.21.15.119:0/3939726261 conn(0x7f4fc806aba0 msgr2=0x7f4fc806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:07.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.321+0000 7f4fcfe00640 1 -- 172.21.15.119:0/3939726261 shutdown_connections 2024-04-25T10:16:07.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.321+0000 7f4fcfe00640 1 -- 172.21.15.119:0/3939726261 wait complete. 2024-04-25T10:16:07.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.322+0000 7f4fcfe00640 1 Processor -- start 2024-04-25T10:16:07.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.322+0000 7f4fcfe00640 1 -- start start 2024-04-25T10:16:07.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.323+0000 7f4fcfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc8071190 0x7f4fc8141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.323+0000 7f4fcfe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc8071570 0x7f4fc80cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.323+0000 7f4fcfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc815e030 0x7f4fc80cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.323+0000 7f4fcfe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc806e8b0 con 0x7f4fc8071190 2024-04-25T10:16:07.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.323+0000 7f4fcfe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc806e0d0 con 0x7f4fc8071570 2024-04-25T10:16:07.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.323+0000 7f4fcfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc80719d0 con 0x7f4fc815e030 2024-04-25T10:16:07.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.324+0000 7f4fcd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc8071190 0x7f4fc8141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.324+0000 7f4fcd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc8071190 0x7f4fc8141550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53032/0 (socket says 172.21.15.119:53032) 2024-04-25T10:16:07.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.324+0000 7f4fcd400640 1 -- 172.21.15.119:0/1007295967 learned_addr learned my addr 172.21.15.119:0/1007295967 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:07.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.324+0000 7f4fcde00640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc815e030 0x7f4fc80cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.324+0000 7f4fcca00640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc8071570 0x7f4fc80cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.324+0000 7f4fcd400640 1 -- 172.21.15.119:0/1007295967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc815e030 msgr2=0x7f4fc80cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.324+0000 7f4fcd400640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc815e030 0x7f4fc80cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.324+0000 7f4fcd400640 1 -- 172.21.15.119:0/1007295967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc8071570 msgr2=0x7f4fc80cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.324+0000 7f4fcd400640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc8071570 0x7f4fc80cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.324+0000 7f4fcd400640 1 -- 172.21.15.119:0/1007295967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc8141e30 con 0x7f4fc8071190 2024-04-25T10:16:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.324+0000 7f4fcca00640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc8071570 0x7f4fc80cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.324+0000 7f4fcde00640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc815e030 0x7f4fc80cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.325+0000 7f4fcd400640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc8071190 0x7f4fc8141550 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00ebe0 tx=0x7f4fbc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.325+0000 7f4fc5600640 1 -- 172.21.15.119:0/1007295967 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc017070 con 0x7f4fc8071190 2024-04-25T10:16:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.325+0000 7f4fc5600640 1 -- 172.21.15.119:0/1007295967 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fbc00cc30 con 0x7f4fc8071190 2024-04-25T10:16:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.325+0000 7f4fc5600640 1 -- 172.21.15.119:0/1007295967 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc013470 con 0x7f4fc8071190 2024-04-25T10:16:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.325+0000 7f4fcfe00640 1 -- 172.21.15.119:0/1007295967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc8142110 con 0x7f4fc8071190 2024-04-25T10:16:07.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.325+0000 7f4fcfe00640 1 -- 172.21.15.119:0/1007295967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc80af060 con 0x7f4fc8071190 2024-04-25T10:16:07.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.326+0000 7f4fcfe00640 1 -- 172.21.15.119:0/1007295967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc80b4720 con 0x7f4fc8071190 2024-04-25T10:16:07.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.327+0000 7f4fc5600640 1 -- 172.21.15.119:0/1007295967 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fbc01a070 con 0x7f4fc8071190 2024-04-25T10:16:07.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.329+0000 7f4fc5600640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fac06b500 0x7f4fac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.329+0000 7f4fc5600640 1 -- 172.21.15.119:0/1007295967 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fbc08a500 con 0x7f4fc8071190 2024-04-25T10:16:07.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.329+0000 7f4fcca00640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fac06b500 0x7f4fac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.330+0000 7f4fcca00640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fac06b500 0x7f4fac06d9c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8034000 tx=0x7f4fb802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.333+0000 7f4fc5600640 1 -- 172.21.15.119:0/1007295967 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fc80b4720 con 0x7f4fc8071190 2024-04-25T10:16:07.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.532+0000 7f4fcfe00640 1 -- 172.21.15.119:0/1007295967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fc8153cb0 con 0x7f4fc8071190 2024-04-25T10:16:07.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.534+0000 7f4fc5600640 1 -- 172.21.15.119:0/1007295967 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16644 (secure 0 0 0) 0x7f4fc8153cb0 con 0x7f4fc8071190 2024-04-25T10:16:07.538 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:07.539 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":96,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:15:53.131988+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:49.348469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:50.584869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.537+0000 7f4fcfe00640 1 -- 172.21.15.119:0/1007295967 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fac06b500 msgr2=0x7f4fac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.537+0000 7f4fcfe00640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fac06b500 0x7f4fac06d9c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8034000 tx=0x7f4fb802e000 comp rx=0 tx=0).stop 2024-04-25T10:16:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.537+0000 7f4fcfe00640 1 -- 172.21.15.119:0/1007295967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc8071190 msgr2=0x7f4fc8141550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.537+0000 7f4fcfe00640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc8071190 0x7f4fc8141550 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00ebe0 tx=0x7f4fbc00c520 comp rx=0 tx=0).stop 2024-04-25T10:16:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.537+0000 7f4fcfe00640 1 -- 172.21.15.119:0/1007295967 shutdown_connections 2024-04-25T10:16:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.537+0000 7f4fcfe00640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4fac06b500 0x7f4fac06d9c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.537+0000 7f4fcfe00640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc815e030 0x7f4fc80cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.537+0000 7f4fcfe00640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc8071570 0x7f4fc80cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.537+0000 7f4fcfe00640 1 --2- 172.21.15.119:0/1007295967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc8071190 0x7f4fc8141550 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.537+0000 7f4fcfe00640 1 -- 172.21.15.119:0/1007295967 >> 172.21.15.119:0/1007295967 conn(0x7f4fc806aba0 msgr2=0x7f4fc80b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:07.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.537+0000 7f4fcfe00640 1 -- 172.21.15.119:0/1007295967 shutdown_connections 2024-04-25T10:16:07.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.537+0000 7f4fcfe00640 1 -- 172.21.15.119:0/1007295967 wait complete. 2024-04-25T10:16:07.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:15:49.348469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:15:50.584869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:07.561 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:07.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.764+0000 7f7070600640 1 Processor -- start 2024-04-25T10:16:07.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.764+0000 7f7070600640 1 -- start start 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.766+0000 7f7070600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70681550e0 0x7f706815ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.766+0000 7f7070600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706815b370 con 0x7f706815aef0 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.766+0000 7f7070600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706815b650 con 0x7f70680b42c0 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.766+0000 7f7070600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70680d05d0 con 0x7f70681550e0 2024-04-25T10:16:07.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.766+0000 7f706e600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70681550e0 0x7f706815ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.766+0000 7f706d200640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f706815aef0 0x7f70681549d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48824/0 (socket says 172.21.15.119:48824) 2024-04-25T10:16:07.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.766+0000 7f706d200640 1 -- 172.21.15.119:0/828357326 learned_addr learned my addr 172.21.15.119:0/828357326 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:16:07.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.767+0000 7f706e600640 1 -- 172.21.15.119:0/828357326 >> v1:172.21.15.132:6789/0 conn(0x7f70680b42c0 legacy=0x7f70680cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:07.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.767+0000 7f706e600640 1 -- 172.21.15.119:0/828357326 >> v1:172.21.15.119:6789/0 conn(0x7f706815aef0 legacy=0x7f70681549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.767+0000 7f706e600640 1 -- 172.21.15.119:0/828357326 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706815b930 con 0x7f70681550e0 2024-04-25T10:16:07.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.767+0000 7f7067e00640 1 -- 172.21.15.119:0/828357326 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724137994 0 0) 0x7f706815b370 con 0x7f706815aef0 2024-04-25T10:16:07.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.768+0000 7f706e600640 1 --2- 172.21.15.119:0/828357326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70681550e0 0x7f706815ef40 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f7060009d90 tx=0x7f706002a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28119d2698b96c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.768+0000 7f7067e00640 1 -- 172.21.15.119:0/828357326 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060007740 con 0x7f70681550e0 2024-04-25T10:16:07.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.768+0000 7f7067e00640 1 -- 172.21.15.119:0/828357326 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70600078e0 con 0x7f70681550e0 2024-04-25T10:16:07.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.768+0000 7f7067e00640 1 -- 172.21.15.119:0/828357326 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060007cb0 con 0x7f70681550e0 2024-04-25T10:16:07.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.769+0000 7f7070600640 1 -- 172.21.15.119:0/828357326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70681550e0 msgr2=0x7f706815ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.769+0000 7f7070600640 1 --2- 172.21.15.119:0/828357326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70681550e0 0x7f706815ef40 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f7060009d90 tx=0x7f706002a580 comp rx=0 tx=0).stop 2024-04-25T10:16:07.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.769+0000 7f7070600640 1 -- 172.21.15.119:0/828357326 shutdown_connections 2024-04-25T10:16:07.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.769+0000 7f7070600640 1 --2- 172.21.15.119:0/828357326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70681550e0 0x7f706815ef40 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.769+0000 7f7070600640 1 -- 172.21.15.119:0/828357326 >> 172.21.15.119:0/828357326 conn(0x7f706806aba0 msgr2=0x7f706806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:07.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.769+0000 7f7070600640 1 -- 172.21.15.119:0/828357326 shutdown_connections 2024-04-25T10:16:07.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.769+0000 7f7070600640 1 -- 172.21.15.119:0/828357326 wait complete. 2024-04-25T10:16:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.771+0000 7f7070600640 1 Processor -- start 2024-04-25T10:16:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.771+0000 7f7070600640 1 -- start start 2024-04-25T10:16:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f7070600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70680b42c0 0x7f70681e3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f7070600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70681550e0 0x7f70681e4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f7070600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706815aef0 0x7f70681432a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f7070600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706806ec60 con 0x7f706815aef0 2024-04-25T10:16:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f7070600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706806eae0 con 0x7f70681550e0 2024-04-25T10:16:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f7070600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70680b46a0 con 0x7f70680b42c0 2024-04-25T10:16:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f706e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706815aef0 0x7f70681432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f706e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706815aef0 0x7f70681432a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53052/0 (socket says 172.21.15.119:53052) 2024-04-25T10:16:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f706e600640 1 -- 172.21.15.119:0/1918105836 learned_addr learned my addr 172.21.15.119:0/1918105836 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f706d200640 1 --2- 172.21.15.119:0/1918105836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70681550e0 0x7f70681e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f706dc00640 1 --2- 172.21.15.119:0/1918105836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70680b42c0 0x7f70681e3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f706e600640 1 -- 172.21.15.119:0/1918105836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70680b42c0 msgr2=0x7f70681e3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f706e600640 1 --2- 172.21.15.119:0/1918105836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70680b42c0 0x7f70681e3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f706e600640 1 -- 172.21.15.119:0/1918105836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70681550e0 msgr2=0x7f70681e4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f706e600640 1 --2- 172.21.15.119:0/1918105836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70681550e0 0x7f70681e4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.772+0000 7f706e600640 1 -- 172.21.15.119:0/1918105836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70681e4e80 con 0x7f706815aef0 2024-04-25T10:16:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.773+0000 7f706e600640 1 --2- 172.21.15.119:0/1918105836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706815aef0 0x7f70681432a0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f7060009d90 tx=0x7f7060004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.773+0000 7f7065600640 1 -- 172.21.15.119:0/1918105836 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060040070 con 0x7f706815aef0 2024-04-25T10:16:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.773+0000 7f7065600640 1 -- 172.21.15.119:0/1918105836 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7060031c90 con 0x7f706815aef0 2024-04-25T10:16:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.773+0000 7f7065600640 1 -- 172.21.15.119:0/1918105836 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706003c640 con 0x7f706815aef0 2024-04-25T10:16:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.773+0000 7f7070600640 1 -- 172.21.15.119:0/1918105836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70681e4440 con 0x7f706815aef0 2024-04-25T10:16:07.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.773+0000 7f7070600640 1 -- 172.21.15.119:0/1918105836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7068156260 con 0x7f706815aef0 2024-04-25T10:16:07.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.774+0000 7f7070600640 1 -- 172.21.15.119:0/1918105836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70680750b0 con 0x7f706815aef0 2024-04-25T10:16:07.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.775+0000 7f7065600640 1 -- 172.21.15.119:0/1918105836 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7060039040 con 0x7f706815aef0 2024-04-25T10:16:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.776+0000 7f7065600640 1 --2- 172.21.15.119:0/1918105836 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f704806b4d0 0x7f704806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.776+0000 7f7065600640 1 -- 172.21.15.119:0/1918105836 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70600ab7f0 con 0x7f706815aef0 2024-04-25T10:16:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.776+0000 7f706dc00640 1 --2- 172.21.15.119:0/1918105836 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f704806b4d0 0x7f704806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.777+0000 7f706dc00640 1 --2- 172.21.15.119:0/1918105836 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f704806b4d0 0x7f704806d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f705c003d90 tx=0x7f705c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:07.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.781+0000 7f7065600640 1 -- 172.21.15.119:0/1918105836 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f706007a720 con 0x7f706815aef0 2024-04-25T10:16:07.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.973+0000 7f7070600640 1 -- 172.21.15.119:0/1918105836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f70681548b0 con 0x7f706815aef0 2024-04-25T10:16:07.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.974+0000 7f7065600640 1 -- 172.21.15.119:0/1918105836 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f706007a020 con 0x7f706815aef0 2024-04-25T10:16:07.978 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:16:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.976+0000 7f7070600640 1 -- 172.21.15.119:0/1918105836 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f704806b4d0 msgr2=0x7f704806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.976+0000 7f7070600640 1 --2- 172.21.15.119:0/1918105836 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f704806b4d0 0x7f704806d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f705c003d90 tx=0x7f705c009040 comp rx=0 tx=0).stop 2024-04-25T10:16:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.976+0000 7f7070600640 1 -- 172.21.15.119:0/1918105836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706815aef0 msgr2=0x7f70681432a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.976+0000 7f7070600640 1 --2- 172.21.15.119:0/1918105836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706815aef0 0x7f70681432a0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f7060009d90 tx=0x7f7060004a40 comp rx=0 tx=0).stop 2024-04-25T10:16:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.977+0000 7f7070600640 1 -- 172.21.15.119:0/1918105836 shutdown_connections 2024-04-25T10:16:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.977+0000 7f7070600640 1 --2- 172.21.15.119:0/1918105836 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f704806b4d0 0x7f704806d990 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.977+0000 7f7070600640 1 --2- 172.21.15.119:0/1918105836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706815aef0 0x7f70681432a0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.977+0000 7f7070600640 1 --2- 172.21.15.119:0/1918105836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70681550e0 0x7f70681e4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.977+0000 7f7070600640 1 --2- 172.21.15.119:0/1918105836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70680b42c0 0x7f70681e3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.977+0000 7f7070600640 1 -- 172.21.15.119:0/1918105836 >> 172.21.15.119:0/1918105836 conn(0x7f706806aba0 msgr2=0x7f706815a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.977+0000 7f7070600640 1 -- 172.21.15.119:0/1918105836 shutdown_connections 2024-04-25T10:16:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:07.977+0000 7f7070600640 1 -- 172.21.15.119:0/1918105836 wait complete. 2024-04-25T10:16:08.000 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:08.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.202+0000 7f5ee4000640 1 Processor -- start 2024-04-25T10:16:08.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.203+0000 7f5ee4000640 1 -- start start 2024-04-25T10:16:08.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.204+0000 7f5ee4000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5edc071930 0x7f5edc15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.204+0000 7f5ee4000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5edc0b3670 con 0x7f5edc159a20 2024-04-25T10:16:08.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.204+0000 7f5ee4000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5edc0b3950 con 0x7f5edc159e00 2024-04-25T10:16:08.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.204+0000 7f5ee4000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc0b41f0 con 0x7f5edc071930 2024-04-25T10:16:08.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.204+0000 7f5ee1600640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5edc159a20 0x7f5edc071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48838/0 (socket says 172.21.15.119:48838) 2024-04-25T10:16:08.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.205+0000 7f5ee1600640 1 -- 172.21.15.119:0/1146914550 learned_addr learned my addr 172.21.15.119:0/1146914550 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:16:08.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.205+0000 7f5ee0c00640 1 --2- 172.21.15.119:0/1146914550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5edc071930 0x7f5edc15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.205+0000 7f5edbe00640 1 -- 172.21.15.119:0/1146914550 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061543869 0 0) 0x7f5edc0b3670 con 0x7f5edc159a20 2024-04-25T10:16:08.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.205+0000 7f5edbe00640 1 -- 172.21.15.119:0/1146914550 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ec4003520 con 0x7f5edc159a20 2024-04-25T10:16:08.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.205+0000 7f5edbe00640 1 -- 172.21.15.119:0/1146914550 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3051477420 0 0) 0x7f5ec4003520 con 0x7f5edc159a20 2024-04-25T10:16:08.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.205+0000 7f5edbe00640 1 -- 172.21.15.119:0/1146914550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5edc071930 msgr2=0x7f5edc15a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:08.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.205+0000 7f5edbe00640 1 --2- 172.21.15.119:0/1146914550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5edc071930 0x7f5edc15a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.205+0000 7f5edbe00640 1 -- 172.21.15.119:0/1146914550 >> v1:172.21.15.132:6789/0 conn(0x7f5edc159e00 legacy=0x7f5edc15ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.205+0000 7f5edbe00640 1 -- 172.21.15.119:0/1146914550 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5edc0b3c30 con 0x7f5edc159a20 2024-04-25T10:16:08.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.205+0000 7f5edbe00640 1 -- 172.21.15.119:0/1146914550 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f5ed0001670 con 0x7f5edc159a20 2024-04-25T10:16:08.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5edbe00640 1 -- 172.21.15.119:0/1146914550 >> v1:172.21.15.119:6789/0 conn(0x7f5edc159a20 legacy=0x7f5edc071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5edbe00640 1 --2- 172.21.15.119:0/1146914550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec40039e0 0x7f5ec4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5edbe00640 1 --2- 172.21.15.119:0/1146914550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5edc071930 0x7f5ec4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5edbe00640 1 --2- 172.21.15.119:0/1146914550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec4004d40 0x7f5ec40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5edbe00640 1 -- 172.21.15.119:0/1146914550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc0b41f0 con 0x7f5ec4004d40 2024-04-25T10:16:08.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5edbe00640 1 -- 172.21.15.119:0/1146914550 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec400bb80 con 0x7f5ec40039e0 2024-04-25T10:16:08.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5edbe00640 1 -- 172.21.15.119:0/1146914550 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec400df90 con 0x7f5edc071930 2024-04-25T10:16:08.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5ee2000640 1 --2- 172.21.15.119:0/1146914550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec4004d40 0x7f5ec40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5ee1600640 1 --2- 172.21.15.119:0/1146914550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec40039e0 0x7f5ec4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5ee0c00640 1 --2- 172.21.15.119:0/1146914550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5edc071930 0x7f5ec4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5ee2000640 1 -- 172.21.15.119:0/1146914550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5edc071930 msgr2=0x7f5ec4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:08.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5ee2000640 1 --2- 172.21.15.119:0/1146914550 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5edc071930 0x7f5ec4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5ee2000640 1 -- 172.21.15.119:0/1146914550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec40039e0 msgr2=0x7f5ec4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:08.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5ee2000640 1 --2- 172.21.15.119:0/1146914550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec40039e0 0x7f5ec4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5ee2000640 1 -- 172.21.15.119:0/1146914550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec4003520 con 0x7f5ec4004d40 2024-04-25T10:16:08.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.206+0000 7f5ee1600640 1 --2- 172.21.15.119:0/1146914550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ec40039e0 0x7f5ec4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:08.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.207+0000 7f5ee2000640 1 --2- 172.21.15.119:0/1146914550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec4004d40 0x7f5ec40051c0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4001e50 tx=0x7f5ec4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee9e704a0d5be67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.208+0000 7f5edbe00640 1 -- 172.21.15.119:0/1146914550 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4001a70 con 0x7f5ec4004d40 2024-04-25T10:16:08.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.208+0000 7f5edbe00640 1 -- 172.21.15.119:0/1146914550 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed4003970 con 0x7f5ec4004d40 2024-04-25T10:16:08.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.208+0000 7f5edbe00640 1 -- 172.21.15.119:0/1146914550 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4003d60 con 0x7f5ec4004d40 2024-04-25T10:16:08.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.208+0000 7f5ee4000640 1 -- 172.21.15.119:0/1146914550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec4004d40 msgr2=0x7f5ec40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.208+0000 7f5ee4000640 1 --2- 172.21.15.119:0/1146914550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec4004d40 0x7f5ec40051c0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4001e50 tx=0x7f5ec4005830 comp rx=0 tx=0).stop 2024-04-25T10:16:08.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.208+0000 7f5ee2000640 1 -- 172.21.15.119:0/1146914550 reap_dead start 2024-04-25T10:16:08.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.208+0000 7f5ee4000640 1 -- 172.21.15.119:0/1146914550 shutdown_connections 2024-04-25T10:16:08.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.208+0000 7f5ee4000640 1 -- 172.21.15.119:0/1146914550 >> 172.21.15.119:0/1146914550 conn(0x7f5edc06aba0 msgr2=0x7f5edc06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:08.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.208+0000 7f5ee4000640 1 -- 172.21.15.119:0/1146914550 shutdown_connections 2024-04-25T10:16:08.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.208+0000 7f5ee4000640 1 -- 172.21.15.119:0/1146914550 wait complete. 2024-04-25T10:16:08.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.210+0000 7f5ee4000640 1 Processor -- start 2024-04-25T10:16:08.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.210+0000 7f5ee4000640 1 -- start start 2024-04-25T10:16:08.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.211+0000 7f5ee4000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ec4004d40 0x7f5edc14ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.211+0000 7f5ee4000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5edc154780 0x7f5edc154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.211+0000 7f5ee4000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5edc1ef9b0 0x7f5edc1efe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.211+0000 7f5ee4000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc06f310 con 0x7f5edc1ef9b0 2024-04-25T10:16:08.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.211+0000 7f5ee4000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc071fc0 con 0x7f5edc154780 2024-04-25T10:16:08.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.211+0000 7f5ee4000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc06f490 con 0x7f5ec4004d40 2024-04-25T10:16:08.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.211+0000 7f5ee2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5edc1ef9b0 0x7f5edc1efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.212+0000 7f5ee2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5edc1ef9b0 0x7f5edc1efe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53064/0 (socket says 172.21.15.119:53064) 2024-04-25T10:16:08.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.212+0000 7f5ee2000640 1 -- 172.21.15.119:0/2198269180 learned_addr learned my addr 172.21.15.119:0/2198269180 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:08.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.212+0000 7f5ee0c00640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5edc154780 0x7f5edc154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.212+0000 7f5ee1600640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ec4004d40 0x7f5edc14ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.212+0000 7f5ee2000640 1 -- 172.21.15.119:0/2198269180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ec4004d40 msgr2=0x7f5edc14ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.212+0000 7f5ee2000640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ec4004d40 0x7f5edc14ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.212+0000 7f5ee2000640 1 -- 172.21.15.119:0/2198269180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5edc154780 msgr2=0x7f5edc154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.212+0000 7f5ee2000640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5edc154780 0x7f5edc154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.212+0000 7f5ee2000640 1 -- 172.21.15.119:0/2198269180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5edc140770 con 0x7f5edc1ef9b0 2024-04-25T10:16:08.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.212+0000 7f5ee1600640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ec4004d40 0x7f5edc14ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:08.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.212+0000 7f5ee2000640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5edc1ef9b0 0x7f5edc1efe60 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f5ed402a920 tx=0x7f5ed40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.213+0000 7f5ed9600640 1 -- 172.21.15.119:0/2198269180 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4041070 con 0x7f5edc1ef9b0 2024-04-25T10:16:08.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.213+0000 7f5ed9600640 1 -- 172.21.15.119:0/2198269180 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed4006d60 con 0x7f5edc1ef9b0 2024-04-25T10:16:08.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.213+0000 7f5ed9600640 1 -- 172.21.15.119:0/2198269180 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4001a70 con 0x7f5edc1ef9b0 2024-04-25T10:16:08.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.213+0000 7f5ee4000640 1 -- 172.21.15.119:0/2198269180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5edc155140 con 0x7f5edc1ef9b0 2024-04-25T10:16:08.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.213+0000 7f5ee4000640 1 -- 172.21.15.119:0/2198269180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5edc14fba0 con 0x7f5edc1ef9b0 2024-04-25T10:16:08.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.214+0000 7f5ee4000640 1 -- 172.21.15.119:0/2198269180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5edc0750b0 con 0x7f5edc1ef9b0 2024-04-25T10:16:08.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.215+0000 7f5ed9600640 1 -- 172.21.15.119:0/2198269180 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ed4044070 con 0x7f5edc1ef9b0 2024-04-25T10:16:08.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.216+0000 7f5ed9600640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5ec006b500 0x7f5ec006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.216+0000 7f5ed9600640 1 -- 172.21.15.119:0/2198269180 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ed40a9350 con 0x7f5edc1ef9b0 2024-04-25T10:16:08.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.216+0000 7f5ee1600640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5ec006b500 0x7f5ec006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.217+0000 7f5ee1600640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5ec006b500 0x7f5ec006d9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00079d0 tx=0x7f5ed0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.220+0000 7f5ed9600640 1 -- 172.21.15.119:0/2198269180 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5edc155140 con 0x7f5edc1ef9b0 2024-04-25T10:16:08.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.413+0000 7f5ee4000640 1 -- 172.21.15.119:0/2198269180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5edc06da00 con 0x7f5edc1ef9b0 2024-04-25T10:16:08.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.414+0000 7f5ed9600640 1 -- 172.21.15.119:0/2198269180 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5ed40781b0 con 0x7f5edc1ef9b0 2024-04-25T10:16:08.418 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:16:08.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.416+0000 7f5ee4000640 1 -- 172.21.15.119:0/2198269180 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5ec006b500 msgr2=0x7f5ec006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.416+0000 7f5ee4000640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5ec006b500 0x7f5ec006d9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00079d0 tx=0x7f5ed0009040 comp rx=0 tx=0).stop 2024-04-25T10:16:08.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.416+0000 7f5ee4000640 1 -- 172.21.15.119:0/2198269180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5edc1ef9b0 msgr2=0x7f5edc1efe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.416+0000 7f5ee4000640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5edc1ef9b0 0x7f5edc1efe60 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f5ed402a920 tx=0x7f5ed40014a0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.416+0000 7f5ee4000640 1 -- 172.21.15.119:0/2198269180 shutdown_connections 2024-04-25T10:16:08.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.416+0000 7f5ee4000640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5ec006b500 0x7f5ec006d9c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.416+0000 7f5ee4000640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5edc1ef9b0 0x7f5edc1efe60 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.416+0000 7f5ee4000640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5edc154780 0x7f5edc154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.417+0000 7f5ee4000640 1 --2- 172.21.15.119:0/2198269180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ec4004d40 0x7f5edc14ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.417+0000 7f5ee4000640 1 -- 172.21.15.119:0/2198269180 >> 172.21.15.119:0/2198269180 conn(0x7f5edc06aba0 msgr2=0x7f5edc0cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:08.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.417+0000 7f5ee4000640 1 -- 172.21.15.119:0/2198269180 shutdown_connections 2024-04-25T10:16:08.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.417+0000 7f5ee4000640 1 -- 172.21.15.119:0/2198269180 wait complete. 2024-04-25T10:16:08.439 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:08.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.635+0000 7f4213000640 1 Processor -- start 2024-04-25T10:16:08.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.636+0000 7f4213000640 1 -- start start 2024-04-25T10:16:08.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.637+0000 7f4213000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f420c144e70 0x7f420c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.637+0000 7f4213000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f420c06dcb0 con 0x7f420c0cf0e0 2024-04-25T10:16:08.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.637+0000 7f4213000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f420c06df90 con 0x7f420c06d7a0 2024-04-25T10:16:08.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.637+0000 7f4213000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c15bfe0 con 0x7f420c144e70 2024-04-25T10:16:08.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.637+0000 7f4211000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f420c0cf0e0 0x7f420c0cc300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48850/0 (socket says 172.21.15.119:48850) 2024-04-25T10:16:08.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.637+0000 7f4211000640 1 -- 172.21.15.119:0/1722522650 learned_addr learned my addr 172.21.15.119:0/1722522650 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:16:08.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.638+0000 7f420be00640 1 --2- 172.21.15.119:0/1722522650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f420c144e70 0x7f420c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.638+0000 7f420b400640 1 -- 172.21.15.119:0/1722522650 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499274575 0 0) 0x7f420c06dcb0 con 0x7f420c0cf0e0 2024-04-25T10:16:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.638+0000 7f420b400640 1 -- 172.21.15.119:0/1722522650 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f0003520 con 0x7f420c0cf0e0 2024-04-25T10:16:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.638+0000 7f420b400640 1 -- 172.21.15.119:0/1722522650 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1644230733 0 0) 0x7f41f0003520 con 0x7f420c0cf0e0 2024-04-25T10:16:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.638+0000 7f420b400640 1 -- 172.21.15.119:0/1722522650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f420c144e70 msgr2=0x7f420c06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.638+0000 7f420b400640 1 --2- 172.21.15.119:0/1722522650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f420c144e70 0x7f420c06d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.638+0000 7f420b400640 1 -- 172.21.15.119:0/1722522650 >> v1:172.21.15.132:6789/0 conn(0x7f420c06d7a0 legacy=0x7f420c0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.638+0000 7f420b400640 1 -- 172.21.15.119:0/1722522650 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c06e270 con 0x7f420c0cf0e0 2024-04-25T10:16:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.638+0000 7f420b400640 1 -- 172.21.15.119:0/1722522650 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f4204001a10 con 0x7f420c0cf0e0 2024-04-25T10:16:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.639+0000 7f420b400640 1 -- 172.21.15.119:0/1722522650 >> v1:172.21.15.119:6789/0 conn(0x7f420c0cf0e0 legacy=0x7f420c0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.639+0000 7f420b400640 1 --2- 172.21.15.119:0/1722522650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41f00039e0 0x7f41f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.639+0000 7f420b400640 1 --2- 172.21.15.119:0/1722522650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41f0004800 0x7f41f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.639+0000 7f420b400640 1 --2- 172.21.15.119:0/1722522650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f420c144e70 0x7f41f0005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.639+0000 7f420b400640 1 -- 172.21.15.119:0/1722522650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c15bfe0 con 0x7f41f00039e0 2024-04-25T10:16:08.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.639+0000 7f420b400640 1 -- 172.21.15.119:0/1722522650 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f000bb50 con 0x7f41f0004800 2024-04-25T10:16:08.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.639+0000 7f420b400640 1 -- 172.21.15.119:0/1722522650 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f000bd50 con 0x7f420c144e70 2024-04-25T10:16:08.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.639+0000 7f420be00640 1 --2- 172.21.15.119:0/1722522650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41f00039e0 0x7f41f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.639+0000 7f4211000640 1 --2- 172.21.15.119:0/1722522650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f420c144e70 0x7f41f0005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.639+0000 7f420be00640 1 -- 172.21.15.119:0/1722522650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f420c144e70 msgr2=0x7f41f0005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:08.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.639+0000 7f420be00640 1 --2- 172.21.15.119:0/1722522650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f420c144e70 0x7f41f0005270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.639+0000 7f420be00640 1 -- 172.21.15.119:0/1722522650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41f0004800 msgr2=0x7f41f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:08.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.639+0000 7f420be00640 1 --2- 172.21.15.119:0/1722522650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41f0004800 0x7f41f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.639+0000 7f420be00640 1 -- 172.21.15.119:0/1722522650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f0003520 con 0x7f41f00039e0 2024-04-25T10:16:08.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.640+0000 7f420be00640 1 --2- 172.21.15.119:0/1722522650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41f00039e0 0x7f41f0003e00 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009ac0 tx=0x7f41f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613856d76ebacb1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.641+0000 7f420b400640 1 -- 172.21.15.119:0/1722522650 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc036070 con 0x7f41f00039e0 2024-04-25T10:16:08.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.641+0000 7f420b400640 1 -- 172.21.15.119:0/1722522650 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41fc031400 con 0x7f41f00039e0 2024-04-25T10:16:08.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.641+0000 7f420b400640 1 -- 172.21.15.119:0/1722522650 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc009690 con 0x7f41f00039e0 2024-04-25T10:16:08.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.641+0000 7f4213000640 1 -- 172.21.15.119:0/1722522650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41f00039e0 msgr2=0x7f41f0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.641+0000 7f4213000640 1 --2- 172.21.15.119:0/1722522650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41f00039e0 0x7f41f0003e00 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009ac0 tx=0x7f41f0005830 comp rx=0 tx=0).stop 2024-04-25T10:16:08.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.641+0000 7f4211000640 1 -- 172.21.15.119:0/1722522650 reap_dead start 2024-04-25T10:16:08.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.641+0000 7f4213000640 1 -- 172.21.15.119:0/1722522650 shutdown_connections 2024-04-25T10:16:08.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.641+0000 7f4213000640 1 -- 172.21.15.119:0/1722522650 >> 172.21.15.119:0/1722522650 conn(0x7f420c06aba0 msgr2=0x7f420c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:08.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.641+0000 7f4213000640 1 -- 172.21.15.119:0/1722522650 shutdown_connections 2024-04-25T10:16:08.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.642+0000 7f4213000640 1 -- 172.21.15.119:0/1722522650 wait complete. 2024-04-25T10:16:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.643+0000 7f4213000640 1 Processor -- start 2024-04-25T10:16:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.643+0000 7f4213000640 1 -- start start 2024-04-25T10:16:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.644+0000 7f4213000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41f00039e0 0x7f420c0b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.644+0000 7f4213000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f420c1f4a10 0x7f420c1f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.644+0000 7f4213000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f420c15acd0 0x7f420c15b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.644+0000 7f4213000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c145630 con 0x7f420c1f4a10 2024-04-25T10:16:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.644+0000 7f4213000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c1454b0 con 0x7f41f00039e0 2024-04-25T10:16:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.644+0000 7f4213000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c06d260 con 0x7f420c15acd0 2024-04-25T10:16:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.645+0000 7f4203400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f420c1f4a10 0x7f420c1f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.645+0000 7f4203400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f420c1f4a10 0x7f420c1f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53096/0 (socket says 172.21.15.119:53096) 2024-04-25T10:16:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.645+0000 7f4203400640 1 -- 172.21.15.119:0/1400707370 learned_addr learned my addr 172.21.15.119:0/1400707370 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.645+0000 7f420be00640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41f00039e0 0x7f420c0b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.645+0000 7f4211000640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f420c15acd0 0x7f420c15b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.645+0000 7f4203400640 1 -- 172.21.15.119:0/1400707370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f420c15acd0 msgr2=0x7f420c15b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.645+0000 7f4203400640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f420c15acd0 0x7f420c15b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.645+0000 7f4203400640 1 -- 172.21.15.119:0/1400707370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41f00039e0 msgr2=0x7f420c0b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.645+0000 7f4203400640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41f00039e0 0x7f420c0b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.645+0000 7f4203400640 1 -- 172.21.15.119:0/1400707370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c0b0f30 con 0x7f420c1f4a10 2024-04-25T10:16:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.645+0000 7f420be00640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41f00039e0 0x7f420c0b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.645+0000 7f4203400640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f420c1f4a10 0x7f420c1f4070 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f41f400c500 tx=0x7f41f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.646+0000 7f4208c00640 1 -- 172.21.15.119:0/1400707370 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f40052d0 con 0x7f420c1f4a10 2024-04-25T10:16:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.646+0000 7f4208c00640 1 -- 172.21.15.119:0/1400707370 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f4005470 con 0x7f420c1f4a10 2024-04-25T10:16:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.646+0000 7f4208c00640 1 -- 172.21.15.119:0/1400707370 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4012850 con 0x7f420c1f4a10 2024-04-25T10:16:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.646+0000 7f4213000640 1 -- 172.21.15.119:0/1400707370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420c0b1210 con 0x7f420c1f4a10 2024-04-25T10:16:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.646+0000 7f4213000640 1 -- 172.21.15.119:0/1400707370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420c0b17d0 con 0x7f420c1f4a10 2024-04-25T10:16:08.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.647+0000 7f4213000640 1 -- 172.21.15.119:0/1400707370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420c0750b0 con 0x7f420c1f4a10 2024-04-25T10:16:08.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.648+0000 7f4208c00640 1 -- 172.21.15.119:0/1400707370 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41f40129f0 con 0x7f420c1f4a10 2024-04-25T10:16:08.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.649+0000 7f4208c00640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41e806b4d0 0x7f41e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:08.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.649+0000 7f4208c00640 1 -- 172.21.15.119:0/1400707370 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41f4089ea0 con 0x7f420c1f4a10 2024-04-25T10:16:08.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.649+0000 7f420be00640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41e806b4d0 0x7f41e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:08.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.650+0000 7f420be00640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41e806b4d0 0x7f41e806d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009ac0 tx=0x7f41fc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:08.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.653+0000 7f4208c00640 1 -- 172.21.15.119:0/1400707370 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41f4058d10 con 0x7f420c1f4a10 2024-04-25T10:16:08.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.834+0000 7f4213000640 1 -- 172.21.15.119:0/1400707370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f420c15efa0 con 0x7f420c1f4a10 2024-04-25T10:16:08.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.834+0000 7f4208c00640 1 -- 172.21.15.119:0/1400707370 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f41f4058610 con 0x7f420c1f4a10 2024-04-25T10:16:08.838 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:16:08.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.836+0000 7f4213000640 1 -- 172.21.15.119:0/1400707370 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41e806b4d0 msgr2=0x7f41e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.836+0000 7f4213000640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41e806b4d0 0x7f41e806d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009ac0 tx=0x7f41fc008030 comp rx=0 tx=0).stop 2024-04-25T10:16:08.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.836+0000 7f4213000640 1 -- 172.21.15.119:0/1400707370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f420c1f4a10 msgr2=0x7f420c1f4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:08.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.836+0000 7f4213000640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f420c1f4a10 0x7f420c1f4070 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f41f400c500 tx=0x7f41f400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.837+0000 7f4213000640 1 -- 172.21.15.119:0/1400707370 shutdown_connections 2024-04-25T10:16:08.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.837+0000 7f4213000640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f41e806b4d0 0x7f41e806d990 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.837+0000 7f4213000640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f420c15acd0 0x7f420c15b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.837+0000 7f4213000640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f420c1f4a10 0x7f420c1f4070 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.837+0000 7f4213000640 1 --2- 172.21.15.119:0/1400707370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41f00039e0 0x7f420c0b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:08.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.837+0000 7f4213000640 1 -- 172.21.15.119:0/1400707370 >> 172.21.15.119:0/1400707370 conn(0x7f420c06aba0 msgr2=0x7f420c0ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:08.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.837+0000 7f4213000640 1 -- 172.21.15.119:0/1400707370 shutdown_connections 2024-04-25T10:16:08.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:08.837+0000 7f4213000640 1 -- 172.21.15.119:0/1400707370 wait complete. 2024-04-25T10:16:08.859 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:16:09.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.052+0000 7f234f200640 1 Processor -- start 2024-04-25T10:16:09.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.052+0000 7f234f200640 1 -- start start 2024-04-25T10:16:09.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.054+0000 7f234f200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f23501450e0 0x7f235006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.054+0000 7f234f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f235006b0e0 0x7f23500cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.054+0000 7f234f200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f23500cc810 0x7f2350071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.054+0000 7f234f200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23501454c0 con 0x7f23501450e0 2024-04-25T10:16:09.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.054+0000 7f234f200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235006b540 con 0x7f23500cc810 2024-04-25T10:16:09.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.054+0000 7f234f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235015f400 con 0x7f235006b0e0 2024-04-25T10:16:09.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.054+0000 7f234de00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f23501450e0 0x7f235006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.054+0000 7f234de00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f23501450e0 0x7f235006aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53106/0 (socket says 172.21.15.119:53106) 2024-04-25T10:16:09.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.054+0000 7f234de00640 1 -- 172.21.15.119:0/3799049678 learned_addr learned my addr 172.21.15.119:0/3799049678 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:09.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.054+0000 7f234de00640 1 -- 172.21.15.119:0/3799049678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f235006b0e0 msgr2=0x7f23500cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.054+0000 7f234de00640 1 --2- 172.21.15.119:0/3799049678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f235006b0e0 0x7f23500cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.054+0000 7f234de00640 1 -- 172.21.15.119:0/3799049678 >> v2:172.21.15.132:3300/0 conn(0x7f23500cc810 msgr2=0x7f2350071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.054+0000 7f234de00640 1 --2- 172.21.15.119:0/3799049678 >> v2:172.21.15.132:3300/0 conn(0x7f23500cc810 0x7f2350071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.054+0000 7f234de00640 1 -- 172.21.15.119:0/3799049678 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235015f6e0 con 0x7f23501450e0 2024-04-25T10:16:09.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.055+0000 7f234de00640 1 --2- 172.21.15.119:0/3799049678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23501450e0 0x7f235006aba0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f23500cbe90 tx=0x7f234002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25425a69b99d9bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.055+0000 7f234ca00640 1 -- 172.21.15.119:0/3799049678 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2340037070 con 0x7f23501450e0 2024-04-25T10:16:09.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.055+0000 7f234ca00640 1 -- 172.21.15.119:0/3799049678 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234002ae00 con 0x7f23501450e0 2024-04-25T10:16:09.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.055+0000 7f234ca00640 1 -- 172.21.15.119:0/3799049678 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2340032660 con 0x7f23501450e0 2024-04-25T10:16:09.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.056+0000 7f234f200640 1 -- 172.21.15.119:0/3799049678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23501450e0 msgr2=0x7f235006aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.056+0000 7f234f200640 1 --2- 172.21.15.119:0/3799049678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23501450e0 0x7f235006aba0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f23500cbe90 tx=0x7f234002a0b0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.056+0000 7f234f200640 1 -- 172.21.15.119:0/3799049678 shutdown_connections 2024-04-25T10:16:09.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.056+0000 7f234f200640 1 --2- 172.21.15.119:0/3799049678 >> v2:172.21.15.132:3300/0 conn(0x7f23500cc810 0x7f2350071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.056+0000 7f234f200640 1 --2- 172.21.15.119:0/3799049678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f235006b0e0 0x7f23500cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.056+0000 7f234f200640 1 --2- 172.21.15.119:0/3799049678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23501450e0 0x7f235006aba0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.056+0000 7f234f200640 1 -- 172.21.15.119:0/3799049678 >> 172.21.15.119:0/3799049678 conn(0x7f2350066ff0 msgr2=0x7f23500b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.056+0000 7f234f200640 1 -- 172.21.15.119:0/3799049678 shutdown_connections 2024-04-25T10:16:09.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.056+0000 7f234f200640 1 -- 172.21.15.119:0/3799049678 wait complete. 2024-04-25T10:16:09.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.058+0000 7f234f200640 1 Processor -- start 2024-04-25T10:16:09.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.059+0000 7f234f200640 1 -- start start 2024-04-25T10:16:09.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f235006b0e0 0x7f23501e51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23500cc810 0x7f235015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23501450e0 0x7f23501e6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23500ccca0 con 0x7f235006b0e0 2024-04-25T10:16:09.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235015fca0 con 0x7f23500cc810 2024-04-25T10:16:09.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23500aca50 con 0x7f23501450e0 2024-04-25T10:16:09.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f235006b0e0 0x7f23501e51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f235006b0e0 0x7f23501e51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53122/0 (socket says 172.21.15.119:53122) 2024-04-25T10:16:09.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234de00640 1 -- 172.21.15.119:0/2056657868 learned_addr learned my addr 172.21.15.119:0/2056657868 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:09.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234d400640 1 --2- 172.21.15.119:0/2056657868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23500cc810 0x7f235015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234de00640 1 -- 172.21.15.119:0/2056657868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23501450e0 msgr2=0x7f23501e6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234de00640 1 --2- 172.21.15.119:0/2056657868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23501450e0 0x7f23501e6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234de00640 1 -- 172.21.15.119:0/2056657868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23500cc810 msgr2=0x7f235015b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234de00640 1 --2- 172.21.15.119:0/2056657868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23500cc810 0x7f235015b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234de00640 1 -- 172.21.15.119:0/2056657868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235006f0c0 con 0x7f235006b0e0 2024-04-25T10:16:09.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f234de00640 1 --2- 172.21.15.119:0/2056657868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f235006b0e0 0x7f23501e51b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f2340000c00 tx=0x7f23400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.061+0000 7f2346000640 1 -- 172.21.15.119:0/2056657868 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2340042070 con 0x7f235006b0e0 2024-04-25T10:16:09.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.062+0000 7f2346000640 1 -- 172.21.15.119:0/2056657868 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234003e670 con 0x7f235006b0e0 2024-04-25T10:16:09.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.062+0000 7f234f200640 1 -- 172.21.15.119:0/2056657868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235015b820 con 0x7f235006b0e0 2024-04-25T10:16:09.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.062+0000 7f2346000640 1 -- 172.21.15.119:0/2056657868 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2340037040 con 0x7f235006b0e0 2024-04-25T10:16:09.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.062+0000 7f234f200640 1 -- 172.21.15.119:0/2056657868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235006eb00 con 0x7f235006b0e0 2024-04-25T10:16:09.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.063+0000 7f234f200640 1 -- 172.21.15.119:0/2056657868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23500749b0 con 0x7f235006b0e0 2024-04-25T10:16:09.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.064+0000 7f2346000640 1 -- 172.21.15.119:0/2056657868 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f234003a040 con 0x7f235006b0e0 2024-04-25T10:16:09.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.065+0000 7f2346000640 1 --2- 172.21.15.119:0/2056657868 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f233006b4d0 0x7f233006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.065+0000 7f2346000640 1 -- 172.21.15.119:0/2056657868 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23400aa7c0 con 0x7f235006b0e0 2024-04-25T10:16:09.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.065+0000 7f234d400640 1 --2- 172.21.15.119:0/2056657868 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f233006b4d0 0x7f233006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.066+0000 7f234d400640 1 --2- 172.21.15.119:0/2056657868 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f233006b4d0 0x7f233006d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f23501e56f0 tx=0x7f233c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.070+0000 7f2346000640 1 -- 172.21.15.119:0/2056657868 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f235015b820 con 0x7f235006b0e0 2024-04-25T10:16:09.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.277+0000 7f234f200640 1 -- 172.21.15.119:0/2056657868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f23500ae730 con 0x7f235006b0e0 2024-04-25T10:16:09.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.278+0000 7f2346000640 1 -- 172.21.15.119:0/2056657868 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v96) v1 ==== 80+0+0 (secure 0 0 0) 0x7f23400795a0 con 0x7f235006b0e0 2024-04-25T10:16:09.282 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2024-04-25T10:16:09.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.281+0000 7f234f200640 1 -- 172.21.15.119:0/2056657868 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f233006b4d0 msgr2=0x7f233006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.281+0000 7f234f200640 1 --2- 172.21.15.119:0/2056657868 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f233006b4d0 0x7f233006d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f23501e56f0 tx=0x7f233c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:09.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.281+0000 7f234f200640 1 -- 172.21.15.119:0/2056657868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f235006b0e0 msgr2=0x7f23501e51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.281+0000 7f234f200640 1 --2- 172.21.15.119:0/2056657868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f235006b0e0 0x7f23501e51b0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f2340000c00 tx=0x7f23400046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.281+0000 7f234f200640 1 -- 172.21.15.119:0/2056657868 shutdown_connections 2024-04-25T10:16:09.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.281+0000 7f234f200640 1 --2- 172.21.15.119:0/2056657868 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f233006b4d0 0x7f233006d990 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.281+0000 7f234f200640 1 --2- 172.21.15.119:0/2056657868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23501450e0 0x7f23501e6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.281+0000 7f234f200640 1 --2- 172.21.15.119:0/2056657868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23500cc810 0x7f235015b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.281+0000 7f234f200640 1 --2- 172.21.15.119:0/2056657868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f235006b0e0 0x7f23501e51b0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.281+0000 7f234f200640 1 -- 172.21.15.119:0/2056657868 >> 172.21.15.119:0/2056657868 conn(0x7f2350066ff0 msgr2=0x7f23500b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.281+0000 7f234f200640 1 -- 172.21.15.119:0/2056657868 shutdown_connections 2024-04-25T10:16:09.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.281+0000 7f234f200640 1 -- 172.21.15.119:0/2056657868 wait complete. 2024-04-25T10:16:09.303 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:16:09.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:09.303 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:09.427 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:16:09.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:09.428 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:16:09.483 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.696+0000 7fb5da800640 1 Processor -- start 2024-04-25T10:16:09.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.696+0000 7fb5da800640 1 -- start start 2024-04-25T10:16:09.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.696+0000 7fb5da800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5d415d980 0x7fb5d40ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.697+0000 7fb5da800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d40712c0 con 0x7fb5d415ed70 2024-04-25T10:16:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.697+0000 7fb5da800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d40715a0 con 0x7fb5d40aeb20 2024-04-25T10:16:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.697+0000 7fb5da800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d4071ed0 con 0x7fb5d415d980 2024-04-25T10:16:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.697+0000 7fb5d3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5d415d980 0x7fb5d40ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.697+0000 7fb5d3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5d415d980 0x7fb5d40ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53126/0 (socket says 172.21.15.119:53126) 2024-04-25T10:16:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.697+0000 7fb5d3400640 1 -- 172.21.15.119:0/713989916 learned_addr learned my addr 172.21.15.119:0/713989916 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.697+0000 7fb5d3e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb5d415ed70 0x7fb5d40acf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48864/0 (socket says 172.21.15.119:48864) 2024-04-25T10:16:09.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.697+0000 7fb5d3400640 1 -- 172.21.15.119:0/713989916 >> v1:172.21.15.132:6789/0 conn(0x7fb5d40aeb20 legacy=0x7fb5d40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.697+0000 7fb5d3400640 1 -- 172.21.15.119:0/713989916 >> v1:172.21.15.119:6789/0 conn(0x7fb5d415ed70 legacy=0x7fb5d40acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.697+0000 7fb5d3400640 1 -- 172.21.15.119:0/713989916 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d4071880 con 0x7fb5d415d980 2024-04-25T10:16:09.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.697+0000 7fb5d2a00640 1 -- 172.21.15.119:0/713989916 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059720561 0 0) 0x7fb5d40712c0 con 0x7fb5d415ed70 2024-04-25T10:16:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.699+0000 7fb5d3400640 1 --2- 172.21.15.119:0/713989916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d415d980 0x7fb5d40ae5e0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4072050 tx=0x7fb5c402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9993c17300b62d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.699+0000 7fb5d2a00640 1 -- 172.21.15.119:0/713989916 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4037070 con 0x7fb5d415d980 2024-04-25T10:16:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.699+0000 7fb5d2a00640 1 -- 172.21.15.119:0/713989916 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c4032440 con 0x7fb5d415d980 2024-04-25T10:16:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.699+0000 7fb5d2a00640 1 -- 172.21.15.119:0/713989916 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4032810 con 0x7fb5d415d980 2024-04-25T10:16:09.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.699+0000 7fb5da800640 1 -- 172.21.15.119:0/713989916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d415d980 msgr2=0x7fb5d40ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.699+0000 7fb5da800640 1 --2- 172.21.15.119:0/713989916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d415d980 0x7fb5d40ae5e0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4072050 tx=0x7fb5c402a210 comp rx=0 tx=0).stop 2024-04-25T10:16:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.700+0000 7fb5da800640 1 -- 172.21.15.119:0/713989916 shutdown_connections 2024-04-25T10:16:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.700+0000 7fb5da800640 1 --2- 172.21.15.119:0/713989916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d415d980 0x7fb5d40ae5e0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.700+0000 7fb5da800640 1 -- 172.21.15.119:0/713989916 >> 172.21.15.119:0/713989916 conn(0x7fb5d406aba0 msgr2=0x7fb5d406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.700+0000 7fb5da800640 1 -- 172.21.15.119:0/713989916 shutdown_connections 2024-04-25T10:16:09.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.700+0000 7fb5da800640 1 -- 172.21.15.119:0/713989916 wait complete. 2024-04-25T10:16:09.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.701+0000 7fb5da800640 1 Processor -- start 2024-04-25T10:16:09.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.701+0000 7fb5da800640 1 -- start start 2024-04-25T10:16:09.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.702+0000 7fb5da800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5d40aeb20 0x7fb5d40cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.702+0000 7fb5da800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5d415d980 0x7fb5d415b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.702+0000 7fb5da800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d415b6f0 0x7fb5d4159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.702+0000 7fb5da800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40cfb10 con 0x7fb5d415b6f0 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.702+0000 7fb5da800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40cf990 con 0x7fb5d415d980 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.702+0000 7fb5da800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40ad680 con 0x7fb5d40aeb20 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.703+0000 7fb5d3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d415b6f0 0x7fb5d4159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.703+0000 7fb5d3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d415b6f0 0x7fb5d4159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53142/0 (socket says 172.21.15.119:53142) 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.703+0000 7fb5d3e00640 1 -- 172.21.15.119:0/2076899068 learned_addr learned my addr 172.21.15.119:0/2076899068 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.703+0000 7fb5d3400640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5d40aeb20 0x7fb5d40cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.703+0000 7fb5d3e00640 1 -- 172.21.15.119:0/2076899068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5d40aeb20 msgr2=0x7fb5d40cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.703+0000 7fb5caa00640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5d415d980 0x7fb5d415b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.703+0000 7fb5d3e00640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5d40aeb20 0x7fb5d40cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.703+0000 7fb5d3e00640 1 -- 172.21.15.119:0/2076899068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5d415d980 msgr2=0x7fb5d415b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.703+0000 7fb5d3e00640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5d415d980 0x7fb5d415b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.703+0000 7fb5d3e00640 1 -- 172.21.15.119:0/2076899068 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d41f3770 con 0x7fb5d415b6f0 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.703+0000 7fb5d3400640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5d40aeb20 0x7fb5d40cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.703+0000 7fb5d3e00640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d415b6f0 0x7fb5d4159860 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00c730 tx=0x7fb5cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.704+0000 7fb5cbe00640 1 -- 172.21.15.119:0/2076899068 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc018070 con 0x7fb5d415b6f0 2024-04-25T10:16:09.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.704+0000 7fb5cbe00640 1 -- 172.21.15.119:0/2076899068 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc00ce40 con 0x7fb5d415b6f0 2024-04-25T10:16:09.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.704+0000 7fb5cbe00640 1 -- 172.21.15.119:0/2076899068 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc00a860 con 0x7fb5d415b6f0 2024-04-25T10:16:09.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.704+0000 7fb5da800640 1 -- 172.21.15.119:0/2076899068 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d415bf40 con 0x7fb5d415b6f0 2024-04-25T10:16:09.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.704+0000 7fb5da800640 1 -- 172.21.15.119:0/2076899068 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d415c780 con 0x7fb5d415b6f0 2024-04-25T10:16:09.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.705+0000 7fb5da800640 1 -- 172.21.15.119:0/2076899068 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d40749b0 con 0x7fb5d415b6f0 2024-04-25T10:16:09.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.706+0000 7fb5cbe00640 1 -- 172.21.15.119:0/2076899068 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5cc01b070 con 0x7fb5d415b6f0 2024-04-25T10:16:09.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.707+0000 7fb5cbe00640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb5b006b4d0 0x7fb5b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.707+0000 7fb5cbe00640 1 -- 172.21.15.119:0/2076899068 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5cc089800 con 0x7fb5d415b6f0 2024-04-25T10:16:09.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.707+0000 7fb5d3400640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb5b006b4d0 0x7fb5b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.708+0000 7fb5d3400640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb5b006b4d0 0x7fb5b006d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4034000 tx=0x7fb5c402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.712+0000 7fb5cbe00640 1 -- 172.21.15.119:0/2076899068 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5d415bf40 con 0x7fb5d415b6f0 2024-04-25T10:16:09.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.946+0000 7fb5da800640 1 -- 172.21.15.119:0/2076899068 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5d406d4c0 con 0x7fb5d415b6f0 2024-04-25T10:16:09.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.947+0000 7fb5cbe00640 1 -- 172.21.15.119:0/2076899068 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fb5d41f3770 con 0x7fb5d415b6f0 2024-04-25T10:16:09.950 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:09.950 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5459,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3768348114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3768348114},{"type":"v1","addr":"172.21.15.132:6834","nonce":3768348114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":5465,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/543190780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":543190780},{"type":"v1","addr":"172.21.15.132:6835","nonce":543190780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1145229172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1145229172},{"type":"v1","addr":"172.21.15.119:6836","nonce":1145229172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:51.568312+0000","modified":"2024-04-25T10:15:54.633087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1945631445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1945631445},{"type":"v1","addr":"172.21.15.119:6837","nonce":1945631445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:16:09.950 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 82 2024-04-25T10:16:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.950+0000 7fb5da800640 1 -- 172.21.15.119:0/2076899068 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb5b006b4d0 msgr2=0x7fb5b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.950+0000 7fb5da800640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb5b006b4d0 0x7fb5b006d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4034000 tx=0x7fb5c402e000 comp rx=0 tx=0).stop 2024-04-25T10:16:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.950+0000 7fb5da800640 1 -- 172.21.15.119:0/2076899068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d415b6f0 msgr2=0x7fb5d4159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.950+0000 7fb5da800640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d415b6f0 0x7fb5d4159860 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00c730 tx=0x7fb5cc00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.950+0000 7fb5da800640 1 -- 172.21.15.119:0/2076899068 shutdown_connections 2024-04-25T10:16:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.950+0000 7fb5da800640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb5b006b4d0 0x7fb5b006d990 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.950+0000 7fb5da800640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d415b6f0 0x7fb5d4159860 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.950+0000 7fb5da800640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5d415d980 0x7fb5d415b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.950+0000 7fb5da800640 1 --2- 172.21.15.119:0/2076899068 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5d40aeb20 0x7fb5d40cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.950+0000 7fb5da800640 1 -- 172.21.15.119:0/2076899068 >> 172.21.15.119:0/2076899068 conn(0x7fb5d406aba0 msgr2=0x7fb5d406dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.950+0000 7fb5da800640 1 -- 172.21.15.119:0/2076899068 shutdown_connections 2024-04-25T10:16:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:09.950+0000 7fb5da800640 1 -- 172.21.15.119:0/2076899068 wait complete. 2024-04-25T10:16:09.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:16:10.196 INFO:teuthology.orchestra.run.smithi119.stdout:{"message":"","return_code":0} 2024-04-25T10:16:10.197 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:16:10.197 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:16:10.197 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:10.245 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:16:10.246 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:16:10.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:10.246 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:16:10.337 INFO:teuthology.orchestra.run.smithi119.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:16:10.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:16:10.338 DEBUG:teuthology.orchestra.run.smithi119:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:16:10.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:10.343 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T10:16:10.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.523+0000 7f5b27e00640 1 Processor -- start 2024-04-25T10:16:10.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.523+0000 7f5b27e00640 1 -- start start 2024-04-25T10:16:10.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.525+0000 7f5b27e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b2815d9c0 0x7f5b280ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.525+0000 7f5b27e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5b280b2480 0x7f5b280ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.525+0000 7f5b27e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b280b6b00 con 0x7f5b280b2a40 2024-04-25T10:16:10.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.525+0000 7f5b27e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b280cf020 con 0x7f5b280b2480 2024-04-25T10:16:10.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.525+0000 7f5b27e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b280b7110 con 0x7f5b2815d9c0 2024-04-25T10:16:10.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.526+0000 7f5b26a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b2815d9c0 0x7f5b280ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.526+0000 7f5b27400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5b280b2480 0x7f5b280ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.526+0000 7f5b26a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b2815d9c0 0x7f5b280ce5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:33284/0 (socket says 172.21.15.119:33284) 2024-04-25T10:16:10.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.526+0000 7f5b26a00640 1 -- 172.21.15.119:0/4203831394 learned_addr learned my addr 172.21.15.119:0/4203831394 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:10.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.526+0000 7f5b26a00640 1 -- 172.21.15.119:0/4203831394 >> v2:172.21.15.132:3300/0 conn(0x7f5b280b2480 msgr2=0x7f5b280ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.526+0000 7f5b26a00640 1 --2- 172.21.15.119:0/4203831394 >> v2:172.21.15.132:3300/0 conn(0x7f5b280b2480 0x7f5b280ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.526+0000 7f5b26a00640 1 -- 172.21.15.119:0/4203831394 >> v1:172.21.15.132:6789/0 conn(0x7f5b280b2a40 legacy=0x7f5b280b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.526+0000 7f5b26a00640 1 -- 172.21.15.119:0/4203831394 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b280b9530 con 0x7f5b2815d9c0 2024-04-25T10:16:10.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.528+0000 7f5b26a00640 1 --2- 172.21.15.119:0/4203831394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b2815d9c0 0x7f5b280ce5a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009870 tx=0x7f5b1802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f4a13dc53b6c4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.528+0000 7f5b26000640 1 -- 172.21.15.119:0/4203831394 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18037070 con 0x7f5b2815d9c0 2024-04-25T10:16:10.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.528+0000 7f5b26000640 1 -- 172.21.15.119:0/4203831394 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b18032440 con 0x7f5b2815d9c0 2024-04-25T10:16:10.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.528+0000 7f5b26000640 1 -- 172.21.15.119:0/4203831394 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18032810 con 0x7f5b2815d9c0 2024-04-25T10:16:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.529+0000 7f5b27e00640 1 -- 172.21.15.119:0/4203831394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b2815d9c0 msgr2=0x7f5b280ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.529+0000 7f5b27e00640 1 --2- 172.21.15.119:0/4203831394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b2815d9c0 0x7f5b280ce5a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009870 tx=0x7f5b1802a250 comp rx=0 tx=0).stop 2024-04-25T10:16:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.529+0000 7f5b27e00640 1 -- 172.21.15.119:0/4203831394 shutdown_connections 2024-04-25T10:16:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.529+0000 7f5b27e00640 1 --2- 172.21.15.119:0/4203831394 >> v2:172.21.15.132:3300/0 conn(0x7f5b280b2480 0x7f5b280ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.529+0000 7f5b27e00640 1 --2- 172.21.15.119:0/4203831394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b2815d9c0 0x7f5b280ce5a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.529+0000 7f5b27e00640 1 -- 172.21.15.119:0/4203831394 >> 172.21.15.119:0/4203831394 conn(0x7f5b280726c0 msgr2=0x7f5b28072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.529+0000 7f5b27e00640 1 -- 172.21.15.119:0/4203831394 shutdown_connections 2024-04-25T10:16:10.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.529+0000 7f5b27e00640 1 -- 172.21.15.119:0/4203831394 wait complete. 2024-04-25T10:16:10.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.531+0000 7f5b27e00640 1 Processor -- start 2024-04-25T10:16:10.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.532+0000 7f5b27e00640 1 -- start start 2024-04-25T10:16:10.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.533+0000 7f5b27e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b280b2480 0x7f5b281551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.533+0000 7f5b27e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b280b2a40 0x7f5b280cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.533+0000 7f5b27e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b2815d9c0 0x7f5b28152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.533+0000 7f5b27e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2815dda0 con 0x7f5b280b2a40 2024-04-25T10:16:10.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.533+0000 7f5b27e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b280b9810 con 0x7f5b280b2480 2024-04-25T10:16:10.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.533+0000 7f5b27e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b280b9150 con 0x7f5b2815d9c0 2024-04-25T10:16:10.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.533+0000 7f5b1e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b280b2a40 0x7f5b280cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.533+0000 7f5b1e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b280b2a40 0x7f5b280cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53150/0 (socket says 172.21.15.119:53150) 2024-04-25T10:16:10.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.533+0000 7f5b1e000640 1 -- 172.21.15.119:0/4068503084 learned_addr learned my addr 172.21.15.119:0/4068503084 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:10.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.533+0000 7f5b27400640 1 --2- 172.21.15.119:0/4068503084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b2815d9c0 0x7f5b28152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.537 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.533+0000 7f5b26a00640 1 --2- 172.21.15.119:0/4068503084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b280b2480 0x7f5b281551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.534+0000 7f5b1e000640 1 -- 172.21.15.119:0/4068503084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b2815d9c0 msgr2=0x7f5b28152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.534+0000 7f5b1e000640 1 --2- 172.21.15.119:0/4068503084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b2815d9c0 0x7f5b28152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.534+0000 7f5b1e000640 1 -- 172.21.15.119:0/4068503084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b280b2480 msgr2=0x7f5b281551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.534+0000 7f5b1e000640 1 --2- 172.21.15.119:0/4068503084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b280b2480 0x7f5b281551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.534+0000 7f5b1e000640 1 -- 172.21.15.119:0/4068503084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b280cc1a0 con 0x7f5b280b2a40 2024-04-25T10:16:10.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.534+0000 7f5b1e000640 1 --2- 172.21.15.119:0/4068503084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b280b2a40 0x7f5b280cd520 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000c440 tx=0x7f5b10001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.535+0000 7f5b1f400640 1 -- 172.21.15.119:0/4068503084 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b100052d0 con 0x7f5b280b2a40 2024-04-25T10:16:10.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.535+0000 7f5b1f400640 1 -- 172.21.15.119:0/4068503084 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b10005470 con 0x7f5b280b2a40 2024-04-25T10:16:10.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.535+0000 7f5b1f400640 1 -- 172.21.15.119:0/4068503084 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10007240 con 0x7f5b280b2a40 2024-04-25T10:16:10.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.535+0000 7f5b27e00640 1 -- 172.21.15.119:0/4068503084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b280cc480 con 0x7f5b280b2a40 2024-04-25T10:16:10.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.535+0000 7f5b27e00640 1 -- 172.21.15.119:0/4068503084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b280cca40 con 0x7f5b280b2a40 2024-04-25T10:16:10.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.538+0000 7f5b1f400640 1 -- 172.21.15.119:0/4068503084 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b100073e0 con 0x7f5b280b2a40 2024-04-25T10:16:10.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.539+0000 7f5b1f400640 1 --2- 172.21.15.119:0/4068503084 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5b0406b500 0x7f5b0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.539+0000 7f5b1f400640 1 -- 172.21.15.119:0/4068503084 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b10089590 con 0x7f5b280b2a40 2024-04-25T10:16:10.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.539+0000 7f5b26a00640 1 --2- 172.21.15.119:0/4068503084 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5b0406b500 0x7f5b0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.539+0000 7f5b30842740 1 Processor -- start 2024-04-25T10:16:10.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.539+0000 7f5b30842740 1 -- start start 2024-04-25T10:16:10.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.540+0000 7f5b26a00640 1 --2- 172.21.15.119:0/4068503084 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5b0406b500 0x7f5b0406d9c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5b18034000 tx=0x7f5b1802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.540+0000 7f5b30842740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9ebb370 0x55c2c9ebb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.540+0000 7f5b30842740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ed8c10 0x55c2c9ed9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.540+0000 7f5b30842740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9e91050 0x55c2c9ed72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.540+0000 7f5b30842740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c9df3980 con 0x55c2c9ed8c10 2024-04-25T10:16:10.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.540+0000 7f5b30842740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c9d9cde0 con 0x55c2c9ebb370 2024-04-25T10:16:10.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.540+0000 7f5b30842740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2c9defa20 con 0x55c2c9e91050 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.541+0000 7f5b1e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ed8c10 0x55c2c9ed9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.541+0000 7f5b1e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ed8c10 0x55c2c9ed9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53152/0 (socket says 172.21.15.119:53152) 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.541+0000 7f5b1e000640 1 -- 172.21.15.119:0/92213621 learned_addr learned my addr 172.21.15.119:0/92213621 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.541+0000 7f5b27400640 1 --2- 172.21.15.119:0/92213621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9e91050 0x55c2c9ed72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.541+0000 7f5b26a00640 1 --2- 172.21.15.119:0/92213621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9ebb370 0x55c2c9ebb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.541+0000 7f5b1e000640 1 -- 172.21.15.119:0/92213621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9e91050 msgr2=0x55c2c9ed72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.541+0000 7f5b1e000640 1 --2- 172.21.15.119:0/92213621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9e91050 0x55c2c9ed72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.541+0000 7f5b1e000640 1 -- 172.21.15.119:0/92213621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9ebb370 msgr2=0x55c2c9ebb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.541+0000 7f5b1e000640 1 --2- 172.21.15.119:0/92213621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9ebb370 0x55c2c9ebb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.541+0000 7f5b1e000640 1 -- 172.21.15.119:0/92213621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2c9dbae30 con 0x55c2c9ed8c10 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.541+0000 7f5b27400640 1 --2- 172.21.15.119:0/92213621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9e91050 0x55c2c9ed72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.541+0000 7f5b1e000640 1 --2- 172.21.15.119:0/92213621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ed8c10 0x55c2c9ed9160 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f5b10087b70 tx=0x7f5b10011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0587ffcc3173ece server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.542+0000 7f5b17400640 1 -- 172.21.15.119:0/92213621 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1005b070 con 0x55c2c9ed8c10 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.542+0000 7f5b17400640 1 -- 172.21.15.119:0/92213621 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b100604b0 con 0x55c2c9ed8c10 2024-04-25T10:16:10.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.542+0000 7f5b17400640 1 -- 172.21.15.119:0/92213621 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1005e0c0 con 0x55c2c9ed8c10 2024-04-25T10:16:10.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.542+0000 7f5b30842740 1 -- 172.21.15.119:0/92213621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ed8c10 msgr2=0x55c2c9ed9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.542+0000 7f5b30842740 1 --2- 172.21.15.119:0/92213621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ed8c10 0x55c2c9ed9160 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f5b10087b70 tx=0x7f5b10011000 comp rx=0 tx=0).stop 2024-04-25T10:16:10.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.542+0000 7f5b30842740 1 -- 172.21.15.119:0/92213621 shutdown_connections 2024-04-25T10:16:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.542+0000 7f5b30842740 1 --2- 172.21.15.119:0/92213621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9e91050 0x55c2c9ed72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.542+0000 7f5b30842740 1 --2- 172.21.15.119:0/92213621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ed8c10 0x55c2c9ed9160 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.542+0000 7f5b30842740 1 --2- 172.21.15.119:0/92213621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9ebb370 0x55c2c9ebb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.542+0000 7f5b30842740 1 -- 172.21.15.119:0/92213621 >> 172.21.15.119:0/92213621 conn(0x55c2c9e8d980 msgr2=0x55c2c9ed6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.542+0000 7f5b30842740 1 -- 172.21.15.119:0/92213621 shutdown_connections 2024-04-25T10:16:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.543+0000 7f5b30842740 1 -- 172.21.15.119:0/92213621 wait complete. 2024-04-25T10:16:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.543+0000 7f5b30842740 1 -- start start 2024-04-25T10:16:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.543+0000 7f5b30842740 1 Processor -- start 2024-04-25T10:16:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.544+0000 7f5b30842740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9ebb370 0x55c2c9ecd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.544+0000 7f5b30842740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ed8c10 0x55c2c9ee7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.544+0000 7f5b30842740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ebd300 0x55c2c9eb79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.544+0000 7f5b30842740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c9e91df0 con 0x55c2c9ebd300 2024-04-25T10:16:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.544+0000 7f5b30842740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c9d923c0 con 0x55c2c9ebb370 2024-04-25T10:16:10.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.544+0000 7f5b30842740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2c9e8b8b0 con 0x55c2c9ed8c10 2024-04-25T10:16:10.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.544+0000 7f5b30842740 10 client.0 _ll_register_callbacks cb 0x55c2c9b07c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:10.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.545+0000 7f5b27400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ebd300 0x55c2c9eb79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.545+0000 7f5b27400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ebd300 0x55c2c9eb79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53164/0 (socket says 172.21.15.119:53164) 2024-04-25T10:16:10.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.545+0000 7f5b27400640 1 -- 172.21.15.119:0/1243770049 learned_addr learned my addr 172.21.15.119:0/1243770049 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:10.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.545+0000 7fc9ffc00640 1 Processor -- start 2024-04-25T10:16:10.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.545+0000 7f5b26a00640 1 --2- 172.21.15.119:0/1243770049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9ebb370 0x55c2c9ecd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.545+0000 7f5b1e000640 1 --2- 172.21.15.119:0/1243770049 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ed8c10 0x55c2c9ee7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.545+0000 7fc9ffc00640 1 -- start start 2024-04-25T10:16:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.545+0000 7f5b27400640 1 -- 172.21.15.119:0/1243770049 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ed8c10 msgr2=0x55c2c9ee7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.545+0000 7f5b27400640 1 --2- 172.21.15.119:0/1243770049 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ed8c10 0x55c2c9ee7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.545+0000 7f5b27400640 1 -- 172.21.15.119:0/1243770049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9ebb370 msgr2=0x55c2c9ecd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.545+0000 7f5b27400640 1 --2- 172.21.15.119:0/1243770049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9ebb370 0x55c2c9ecd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.545+0000 7f5b27400640 1 -- 172.21.15.119:0/1243770049 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2c9bd07e0 con 0x55c2c9ebd300 2024-04-25T10:16:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.545+0000 7f5b27400640 1 --2- 172.21.15.119:0/1243770049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ebd300 0x55c2c9eb79b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f5b20007dd0 tx=0x7f5b20012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0166ac4ff20bfd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.545+0000 7f5b0be00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:16:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.546+0000 7f5b0be00640 1 -- 172.21.15.119:0/1243770049 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2001a070 con 0x55c2c9ebd300 2024-04-25T10:16:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.546+0000 7f5b0be00640 1 -- 172.21.15.119:0/1243770049 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b20016440 con 0x55c2c9ebd300 2024-04-25T10:16:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.546+0000 7f5b30842740 1 -- 172.21.15.119:0/1243770049 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b100604b0 con 0x55c2c9ebd300 2024-04-25T10:16:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.546+0000 7f5b0be00640 1 -- 172.21.15.119:0/1243770049 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b20024050 con 0x55c2c9ebd300 2024-04-25T10:16:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.546+0000 7f5b30842740 10 client.8295 fetch_fsmap learned FSMap version 82 2024-04-25T10:16:10.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.546+0000 7f5b30842740 1 -- 172.21.15.119:0/1243770049 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55c2c9c30f70 con 0x55c2c9ebd300 2024-04-25T10:16:10.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.546+0000 7fc9ffc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9f80cea80 0x7fc9f8162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.546+0000 7fc9ffc00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f815f090 con 0x7fc9f815dca0 2024-04-25T10:16:10.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.546+0000 7fc9ffc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f80cf060 con 0x7fc9f80b5bb0 2024-04-25T10:16:10.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.546+0000 7fc9ffc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80cf340 con 0x7fc9f80cea80 2024-04-25T10:16:10.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.546+0000 7f5b0be00640 1 -- 172.21.15.119:0/1243770049 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b100604b0 con 0x55c2c9ebd300 2024-04-25T10:16:10.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9fd200640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc9f815dca0 0x7fc9f80b54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48880/0 (socket says 172.21.15.119:48880) 2024-04-25T10:16:10.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9fd200640 1 -- 172.21.15.119:0/1708489472 learned_addr learned my addr 172.21.15.119:0/1708489472 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:16:10.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.546+0000 7f5b0be00640 1 -- 172.21.15.119:0/1243770049 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5b20016640 con 0x55c2c9ebd300 2024-04-25T10:16:10.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7f5b30842740 10 client.8295 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:16:10.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7f5b30842740 10 client.8295 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:16:10.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7f5b30842740 20 client.8295 populate_metadata read hostname 'smithi119' 2024-04-25T10:16:10.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7f5b30842740 1 --2- 172.21.15.119:0/1243770049 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55c2c9ecf260 0x55c2c9ecf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7f5b30842740 4 client.8295 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:10.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7f5b30842740 1 -- 172.21.15.119:0/1243770049 --> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2c9e00ec0 con 0x55c2c9ecf260 2024-04-25T10:16:10.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 -- 172.21.15.119:0/1708489472 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194760325 0 0) 0x7fc9f815f090 con 0x7fc9f815dca0 2024-04-25T10:16:10.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 -- 172.21.15.119:0/1708489472 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9e0003520 con 0x7fc9f815dca0 2024-04-25T10:16:10.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9fdc00640 1 --2- 172.21.15.119:0/1708489472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9f80cea80 0x7fc9f8162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7f5b26a00640 1 --2- 172.21.15.119:0/1243770049 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55c2c9ecf260 0x55c2c9ecf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 -- 172.21.15.119:0/1708489472 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1084214327 0 0) 0x7fc9e0003520 con 0x7fc9f815dca0 2024-04-25T10:16:10.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 -- 172.21.15.119:0/1708489472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9f80cea80 msgr2=0x7fc9f8162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 --2- 172.21.15.119:0/1708489472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9f80cea80 0x7fc9f8162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 -- 172.21.15.119:0/1708489472 >> v1:172.21.15.132:6789/0 conn(0x7fc9f80b5bb0 legacy=0x7fc9f80d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 -- 172.21.15.119:0/1708489472 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f806c1a0 con 0x7fc9f815dca0 2024-04-25T10:16:10.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 -- 172.21.15.119:0/1708489472 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fc9ec001690 con 0x7fc9f815dca0 2024-04-25T10:16:10.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 -- 172.21.15.119:0/1708489472 >> v1:172.21.15.119:6789/0 conn(0x7fc9f815dca0 legacy=0x7fc9f80b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 --2- 172.21.15.119:0/1708489472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9e00039e0 0x7fc9e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 --2- 172.21.15.119:0/1708489472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9e0004800 0x7fc9e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 --2- 172.21.15.119:0/1708489472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9f80cea80 0x7fc9e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 -- 172.21.15.119:0/1708489472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80cf340 con 0x7fc9e00039e0 2024-04-25T10:16:10.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 -- 172.21.15.119:0/1708489472 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e0005550 con 0x7fc9e0004800 2024-04-25T10:16:10.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.547+0000 7fc9f7400640 1 -- 172.21.15.119:0/1708489472 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e000bf00 con 0x7fc9f80cea80 2024-04-25T10:16:10.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.548+0000 7f5b26a00640 1 --2- 172.21.15.119:0/1243770049 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55c2c9ecf260 0x55c2c9ecf660 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a75bb9d450b271 server_cookie=de15d97fe0a0d34b in_seq=0 out_seq=0 2024-04-25T10:16:10.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.548+0000 7f5b0be00640 10 client.8295 ms_handle_connect on v2:172.21.15.119:6835/1945631445 2024-04-25T10:16:10.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.548+0000 7fc9fd200640 1 --2- 172.21.15.119:0/1708489472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9e00039e0 0x7fc9e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.548+0000 7fc9fdc00640 1 --2- 172.21.15.119:0/1708489472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9f80cea80 0x7fc9e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.548+0000 7fc9f7e00640 1 --2- 172.21.15.119:0/1708489472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9e0004800 0x7fc9e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.548+0000 7fc9fd200640 1 -- 172.21.15.119:0/1708489472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9f80cea80 msgr2=0x7fc9e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.548+0000 7fc9fd200640 1 --2- 172.21.15.119:0/1708489472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9f80cea80 0x7fc9e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.548+0000 7fc9fd200640 1 -- 172.21.15.119:0/1708489472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9e0004800 msgr2=0x7fc9e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.548+0000 7fc9fd200640 1 --2- 172.21.15.119:0/1708489472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9e0004800 0x7fc9e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.548+0000 7fc9fd200640 1 -- 172.21.15.119:0/1708489472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f806c1a0 con 0x7fc9e00039e0 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.548+0000 7f5b0be00640 1 -- 172.21.15.119:0/1243770049 <== mds.0 v2:172.21.15.119:6835/1945631445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2c9e00ec0 con 0x55c2c9ecf260 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.548+0000 7f5b0be00640 10 client.8295 handle_command_reply: tid=0 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.549+0000 7f5b30842740 1 client.8295 shutdown 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.549+0000 7f5b30842740 1 -- 172.21.15.119:0/1243770049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ebd300 msgr2=0x55c2c9eb79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.549+0000 7f5b30842740 1 --2- 172.21.15.119:0/1243770049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ebd300 0x55c2c9eb79b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f5b20007dd0 tx=0x7f5b20012610 comp rx=0 tx=0).stop 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.549+0000 7f5b30842740 1 -- 172.21.15.119:0/1243770049 shutdown_connections 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.549+0000 7f5b30842740 1 --2- 172.21.15.119:0/1243770049 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55c2c9ecf260 0x55c2c9ecf660 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.549+0000 7f5b30842740 1 --2- 172.21.15.119:0/1243770049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ebd300 0x55c2c9eb79b0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.549+0000 7f5b30842740 1 --2- 172.21.15.119:0/1243770049 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ed8c10 0x55c2c9ee7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.549+0000 7f5b30842740 1 --2- 172.21.15.119:0/1243770049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9ebb370 0x55c2c9ecd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.549+0000 7f5b0be00640 0 client.8295 ms_handle_reset on v2:172.21.15.119:6835/1945631445 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.549+0000 7f5b30842740 1 -- 172.21.15.119:0/1243770049 >> 172.21.15.119:0/1243770049 conn(0x55c2c9e91050 msgr2=0x55c2c9ed9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.549+0000 7f5b30842740 1 -- 172.21.15.119:0/1243770049 shutdown_connections 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.549+0000 7f5b30842740 1 -- 172.21.15.119:0/1243770049 wait complete. 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.549+0000 7fc9fd200640 1 --2- 172.21.15.119:0/1708489472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9e00039e0 0x7fc9e0003e00 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80caaa0 tx=0x7fc9e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972fb8a99f4c841e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.549+0000 7f5b30842740 20 client.8295 trim_cache size 0 max 16384 2024-04-25T10:16:10.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.550+0000 7fc9f7400640 1 -- 172.21.15.119:0/1708489472 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec007d30 con 0x7fc9e00039e0 2024-04-25T10:16:10.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.550+0000 7fc9f7400640 1 -- 172.21.15.119:0/1708489472 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec02f070 con 0x7fc9e00039e0 2024-04-25T10:16:10.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.550+0000 7fc9f7400640 1 -- 172.21.15.119:0/1708489472 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec005d10 con 0x7fc9e00039e0 2024-04-25T10:16:10.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.550+0000 7fc9ffc00640 1 -- 172.21.15.119:0/1708489472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9e00039e0 msgr2=0x7fc9e0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.550+0000 7fc9ffc00640 1 --2- 172.21.15.119:0/1708489472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9e00039e0 0x7fc9e0003e00 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80caaa0 tx=0x7fc9e0005830 comp rx=0 tx=0).stop 2024-04-25T10:16:10.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.550+0000 7fc9fdc00640 1 -- 172.21.15.119:0/1708489472 reap_dead start 2024-04-25T10:16:10.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.551+0000 7fc9ffc00640 1 -- 172.21.15.119:0/1708489472 shutdown_connections 2024-04-25T10:16:10.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.551+0000 7fc9ffc00640 1 -- 172.21.15.119:0/1708489472 >> 172.21.15.119:0/1708489472 conn(0x7fc9f80d04a0 msgr2=0x7fc9f80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.551+0000 7fc9ffc00640 1 -- 172.21.15.119:0/1708489472 shutdown_connections 2024-04-25T10:16:10.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.551+0000 7fc9ffc00640 1 -- 172.21.15.119:0/1708489472 wait complete. 2024-04-25T10:16:10.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.552+0000 7fc9ffc00640 1 Processor -- start 2024-04-25T10:16:10.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.553+0000 7fc9ffc00640 1 -- start start 2024-04-25T10:16:10.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.554+0000 7fc9ffc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9e00039e0 0x7fc9f814ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.554+0000 7fc9ffc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9f81592e0 0x7fc9f8157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.554+0000 7fc9ffc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9f8157d70 0x7fc9f814a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.554+0000 7fc9ffc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f806c760 con 0x7fc9e00039e0 2024-04-25T10:16:10.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.554+0000 7fc9ffc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f806c480 con 0x7fc9f8157d70 2024-04-25T10:16:10.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.554+0000 7fc9ffc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f8162200 con 0x7fc9f81592e0 2024-04-25T10:16:10.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.554+0000 7fc9fd200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9e00039e0 0x7fc9f814ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.554+0000 7fc9fd200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9e00039e0 0x7fc9f814ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53182/0 (socket says 172.21.15.119:53182) 2024-04-25T10:16:10.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.554+0000 7fc9fd200640 1 -- 172.21.15.119:0/337013848 learned_addr learned my addr 172.21.15.119:0/337013848 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:10.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.554+0000 7fc9fd200640 1 -- 172.21.15.119:0/337013848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9f81592e0 msgr2=0x7fc9f8157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.554+0000 7fc9fd200640 1 --2- 172.21.15.119:0/337013848 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9f81592e0 0x7fc9f8157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.554+0000 7fc9fd200640 1 -- 172.21.15.119:0/337013848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9f8157d70 msgr2=0x7fc9f814a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.554+0000 7fc9fd200640 1 --2- 172.21.15.119:0/337013848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9f8157d70 0x7fc9f814a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.554+0000 7fc9fd200640 1 -- 172.21.15.119:0/337013848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f8158240 con 0x7fc9e00039e0 2024-04-25T10:16:10.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.555+0000 7fc9fd200640 1 --2- 172.21.15.119:0/337013848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9e00039e0 0x7fc9f814ca50 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec02ae20 tx=0x7fc9ec02d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.555+0000 7fc9f4c00640 1 -- 172.21.15.119:0/337013848 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec02c070 con 0x7fc9e00039e0 2024-04-25T10:16:10.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.555+0000 7fc9f4c00640 1 -- 172.21.15.119:0/337013848 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec007480 con 0x7fc9e00039e0 2024-04-25T10:16:10.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.555+0000 7fc9f4c00640 1 -- 172.21.15.119:0/337013848 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec007d10 con 0x7fc9e00039e0 2024-04-25T10:16:10.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.555+0000 7fc9ffc00640 1 -- 172.21.15.119:0/337013848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f8158520 con 0x7fc9e00039e0 2024-04-25T10:16:10.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.555+0000 7fc9ffc00640 1 -- 172.21.15.119:0/337013848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f8158ae0 con 0x7fc9e00039e0 2024-04-25T10:16:10.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.556+0000 7fc9ffc00640 1 -- 172.21.15.119:0/337013848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fc9f80749b0 con 0x7fc9e00039e0 2024-04-25T10:16:10.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.558+0000 7fc9f4c00640 1 -- 172.21.15.119:0/337013848 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc9ec03e070 con 0x7fc9e00039e0 2024-04-25T10:16:10.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.558+0000 7fc9f4c00640 1 --2- 172.21.15.119:0/337013848 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc9d806b500 0x7fc9d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.559+0000 7fc9f4c00640 1 -- 172.21.15.119:0/337013848 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9ec0aa800 con 0x7fc9e00039e0 2024-04-25T10:16:10.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.559+0000 7fc9f7e00640 1 --2- 172.21.15.119:0/337013848 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc9d806b500 0x7fc9d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.559+0000 7fc9f4c00640 1 -- 172.21.15.119:0/337013848 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+511 (secure 0 0 0) 0x7fc9ec0aabd0 con 0x7fc9e00039e0 2024-04-25T10:16:10.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.559+0000 7fc9ffc00640 1 -- 172.21.15.119:0/337013848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fc9f80d1810 con 0x7fc9e00039e0 2024-04-25T10:16:10.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.559+0000 7fc9f7e00640 1 --2- 172.21.15.119:0/337013848 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc9d806b500 0x7fc9d806d9c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4007fd0 tx=0x7fc9e4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.567+0000 7f5b30842740 1 Processor -- start 2024-04-25T10:16:10.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.567+0000 7f5b30842740 1 -- start start 2024-04-25T10:16:10.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.568+0000 7f5b30842740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9e91050 0x55c2c9ecf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.568+0000 7f5b30842740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ebb370 0x55c2c9ee3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.568+0000 7f5b30842740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9ebd300 0x55c2c9ed5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.568+0000 7f5b30842740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b281ffd00 con 0x55c2c9e91050 2024-04-25T10:16:10.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.568+0000 7f5b30842740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b281feec0 con 0x55c2c9ebd300 2024-04-25T10:16:10.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.568+0000 7f5b30842740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b280cade0 con 0x55c2c9ebb370 2024-04-25T10:16:10.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.568+0000 7f5b26a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9e91050 0x55c2c9ecf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.568+0000 7f5b26a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9e91050 0x55c2c9ecf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53198/0 (socket says 172.21.15.119:53198) 2024-04-25T10:16:10.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.568+0000 7f5b26a00640 1 -- 172.21.15.119:0/70228631 learned_addr learned my addr 172.21.15.119:0/70228631 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:10.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.568+0000 7f5b26a00640 1 -- 172.21.15.119:0/70228631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ebb370 msgr2=0x55c2c9ee3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.568+0000 7f5b26a00640 1 --2- 172.21.15.119:0/70228631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ebb370 0x55c2c9ee3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.568+0000 7f5b26a00640 1 -- 172.21.15.119:0/70228631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9ebd300 msgr2=0x55c2c9ed5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.568+0000 7f5b26a00640 1 --2- 172.21.15.119:0/70228631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9ebd300 0x55c2c9ed5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.568+0000 7f5b26a00640 1 -- 172.21.15.119:0/70228631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2c9eb7590 con 0x55c2c9e91050 2024-04-25T10:16:10.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.569+0000 7f5b26a00640 1 --2- 172.21.15.119:0/70228631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9e91050 0x55c2c9ecf640 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009870 tx=0x7f5b1802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55cb2507c0e17b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.569+0000 7f5b16a00640 1 -- 172.21.15.119:0/70228631 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18037040 con 0x55c2c9e91050 2024-04-25T10:16:10.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.569+0000 7f5b16a00640 1 -- 172.21.15.119:0/70228631 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1804d2d0 con 0x55c2c9e91050 2024-04-25T10:16:10.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.569+0000 7f5b16a00640 1 -- 172.21.15.119:0/70228631 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1804d680 con 0x55c2c9e91050 2024-04-25T10:16:10.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.570+0000 7f5b30842740 1 -- 172.21.15.119:0/70228631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9e91050 msgr2=0x55c2c9ecf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.570+0000 7f5b30842740 1 --2- 172.21.15.119:0/70228631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9e91050 0x55c2c9ecf640 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009870 tx=0x7f5b1802c000 comp rx=0 tx=0).stop 2024-04-25T10:16:10.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.570+0000 7f5b30842740 1 -- 172.21.15.119:0/70228631 shutdown_connections 2024-04-25T10:16:10.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.570+0000 7f5b30842740 1 --2- 172.21.15.119:0/70228631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9ebd300 0x55c2c9ed5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.570+0000 7f5b30842740 1 --2- 172.21.15.119:0/70228631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ebb370 0x55c2c9ee3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.570+0000 7f5b30842740 1 --2- 172.21.15.119:0/70228631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9e91050 0x55c2c9ecf640 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.570+0000 7f5b30842740 1 -- 172.21.15.119:0/70228631 >> 172.21.15.119:0/70228631 conn(0x55c2c9ed5790 msgr2=0x55c2c9ed0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.570+0000 7f5b30842740 1 -- 172.21.15.119:0/70228631 shutdown_connections 2024-04-25T10:16:10.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.570+0000 7f5b30842740 1 -- 172.21.15.119:0/70228631 wait complete. 2024-04-25T10:16:10.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.570+0000 7f5b30842740 1 -- start start 2024-04-25T10:16:10.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.571+0000 7f5b30842740 1 Processor -- start 2024-04-25T10:16:10.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b30842740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9e91050 0x55c2c9e8ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b30842740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ebb370 0x55c2c9f41370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b30842740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ebd300 0x55c2c9eb1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b30842740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b280cb3e0 con 0x55c2c9ebd300 2024-04-25T10:16:10.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b30842740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c9ecfec0 con 0x55c2c9e91050 2024-04-25T10:16:10.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b30842740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2c9e8f560 con 0x55c2c9ebb370 2024-04-25T10:16:10.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b30842740 10 client.0 _ll_register_callbacks cb 0x55c2c9e8bd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:10.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b27400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ebd300 0x55c2c9eb1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b27400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ebd300 0x55c2c9eb1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53206/0 (socket says 172.21.15.119:53206) 2024-04-25T10:16:10.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b27400640 1 -- 172.21.15.119:0/1416424715 learned_addr learned my addr 172.21.15.119:0/1416424715 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:10.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b1e000640 1 --2- 172.21.15.119:0/1416424715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ebb370 0x55c2c9f41370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b26a00640 1 --2- 172.21.15.119:0/1416424715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9e91050 0x55c2c9e8ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b27400640 1 -- 172.21.15.119:0/1416424715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ebb370 msgr2=0x55c2c9f41370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b27400640 1 --2- 172.21.15.119:0/1416424715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ebb370 0x55c2c9f41370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b27400640 1 -- 172.21.15.119:0/1416424715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9e91050 msgr2=0x55c2c9e8ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b27400640 1 --2- 172.21.15.119:0/1416424715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9e91050 0x55c2c9e8ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b27400640 1 -- 172.21.15.119:0/1416424715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2c9ac8170 con 0x55c2c9ebd300 2024-04-25T10:16:10.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.572+0000 7f5b1e000640 1 --2- 172.21.15.119:0/1416424715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ebb370 0x55c2c9f41370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.573+0000 7f5b27400640 1 --2- 172.21.15.119:0/1416424715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ebd300 0x55c2c9eb1230 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f5b20007220 tx=0x7f5b20004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3b3520bf50b1f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.573+0000 7f5b0be00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.573+0000 7f5b0be00640 1 -- 172.21.15.119:0/1416424715 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2001a030 con 0x55c2c9ebd300 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.573+0000 7f5b0be00640 1 -- 172.21.15.119:0/1416424715 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b20016640 con 0x55c2c9ebd300 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.573+0000 7f5b30842740 1 -- 172.21.15.119:0/1416424715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b1804d2d0 con 0x55c2c9ebd300 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.573+0000 7f5b0be00640 1 -- 172.21.15.119:0/1416424715 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b200098f0 con 0x55c2c9ebd300 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.573+0000 7f5b30842740 10 client.8307 fetch_fsmap learned FSMap version 82 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.573+0000 7f5b30842740 1 -- 172.21.15.119:0/1416424715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55c2c9ebabd0 con 0x55c2c9ebd300 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.574+0000 7f5b0be00640 1 -- 172.21.15.119:0/1416424715 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1804d2d0 con 0x55c2c9ebd300 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.574+0000 7f5b0be00640 1 -- 172.21.15.119:0/1416424715 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5b20039020 con 0x55c2c9ebd300 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.574+0000 7f5b30842740 10 client.8307 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.574+0000 7f5b30842740 10 client.8307 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.574+0000 7f5b30842740 20 client.8307 populate_metadata read hostname 'smithi119' 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.574+0000 7f5b30842740 1 --2- 172.21.15.119:0/1416424715 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55c2c9ed8c10 0x55c2c9eb3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.574+0000 7f5b30842740 4 client.8307 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:16:10.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.574+0000 7f5b30842740 1 -- 172.21.15.119:0/1416424715 --> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55c2c9dfb210 con 0x55c2c9ed8c10 2024-04-25T10:16:10.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.574+0000 7f5b26a00640 1 --2- 172.21.15.119:0/1416424715 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55c2c9ed8c10 0x55c2c9eb3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.575+0000 7f5b0be00640 10 client.8307 ms_handle_connect on v2:172.21.15.119:6835/1945631445 2024-04-25T10:16:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.575+0000 7f5b26a00640 1 --2- 172.21.15.119:0/1416424715 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55c2c9ed8c10 0x55c2c9eb3ca0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73a3d74144847c4c server_cookie=348cd0033f26a4b7 in_seq=0 out_seq=0 2024-04-25T10:16:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.576+0000 7f5b0be00640 1 -- 172.21.15.119:0/1416424715 <== mds.0 v2:172.21.15.119:6835/1945631445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c2c9dfb210 con 0x55c2c9ed8c10 2024-04-25T10:16:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.576+0000 7f5b0be00640 10 client.8307 handle_command_reply: tid=0 2024-04-25T10:16:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.576+0000 7f5b30842740 1 client.8307 shutdown 2024-04-25T10:16:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.576+0000 7f5b30842740 1 -- 172.21.15.119:0/1416424715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ebd300 msgr2=0x55c2c9eb1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.576+0000 7f5b30842740 1 --2- 172.21.15.119:0/1416424715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ebd300 0x55c2c9eb1230 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f5b20007220 tx=0x7f5b20004ab0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.576+0000 7f5b30842740 1 -- 172.21.15.119:0/1416424715 shutdown_connections 2024-04-25T10:16:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.576+0000 7f5b30842740 1 --2- 172.21.15.119:0/1416424715 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55c2c9ed8c10 0x55c2c9eb3ca0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.576+0000 7f5b30842740 1 --2- 172.21.15.119:0/1416424715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c2c9ebd300 0x55c2c9eb1230 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.576+0000 7f5b30842740 1 --2- 172.21.15.119:0/1416424715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c2c9ebb370 0x55c2c9f41370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.576+0000 7f5b30842740 1 --2- 172.21.15.119:0/1416424715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c2c9e91050 0x55c2c9e8ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.576+0000 7f5b30842740 1 -- 172.21.15.119:0/1416424715 >> 172.21.15.119:0/1416424715 conn(0x55c2c9ed5790 msgr2=0x55c2c9e903d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.576+0000 7f5b0be00640 0 client.8307 ms_handle_reset on v2:172.21.15.119:6835/1945631445 2024-04-25T10:16:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.576+0000 7f5b30842740 1 -- 172.21.15.119:0/1416424715 shutdown_connections 2024-04-25T10:16:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.576+0000 7f5b30842740 1 -- 172.21.15.119:0/1416424715 wait complete. 2024-04-25T10:16:10.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.577+0000 7f5b30842740 20 client.8307 trim_cache size 0 max 16384 2024-04-25T10:16:10.582 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:16:10.582 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2024-04-25T10:16:10.582 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "6c170817-658c-45d3-92c0-b06c09121ba5", 2024-04-25T10:16:10.583 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2024-04-25T10:16:10.583 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:16:10.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.577+0000 7f5b27e00640 1 -- 172.21.15.119:0/4068503084 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5b0406b500 msgr2=0x7f5b0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.577+0000 7f5b27e00640 1 --2- 172.21.15.119:0/4068503084 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5b0406b500 0x7f5b0406d9c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5b18034000 tx=0x7f5b1802e000 comp rx=0 tx=0).stop 2024-04-25T10:16:10.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.577+0000 7f5b27e00640 1 -- 172.21.15.119:0/4068503084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b280b2a40 msgr2=0x7f5b280cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.577+0000 7f5b27e00640 1 --2- 172.21.15.119:0/4068503084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b280b2a40 0x7f5b280cd520 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000c440 tx=0x7f5b10001e70 comp rx=0 tx=0).stop 2024-04-25T10:16:10.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.577+0000 7f5b27e00640 1 -- 172.21.15.119:0/4068503084 shutdown_connections 2024-04-25T10:16:10.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.577+0000 7f5b27e00640 1 --2- 172.21.15.119:0/4068503084 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5b0406b500 0x7f5b0406d9c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.577+0000 7f5b27e00640 1 --2- 172.21.15.119:0/4068503084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b2815d9c0 0x7f5b28152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.577+0000 7f5b27e00640 1 --2- 172.21.15.119:0/4068503084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b280b2a40 0x7f5b280cd520 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.577+0000 7f5b27e00640 1 --2- 172.21.15.119:0/4068503084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b280b2480 0x7f5b281551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.577+0000 7f5b27e00640 1 -- 172.21.15.119:0/4068503084 >> 172.21.15.119:0/4068503084 conn(0x7f5b280726c0 msgr2=0x7f5b280b75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.577+0000 7f5b27e00640 1 -- 172.21.15.119:0/4068503084 shutdown_connections 2024-04-25T10:16:10.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.577+0000 7f5b27e00640 1 -- 172.21.15.119:0/4068503084 wait complete. 2024-04-25T10:16:10.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:10.596 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:16:10.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.766+0000 7facfa600640 1 Processor -- start 2024-04-25T10:16:10.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.766+0000 7facfa600640 1 -- start start 2024-04-25T10:16:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.768+0000 7facfa600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7facf413dce0 0x7facf4071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.768+0000 7facfa600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7facf4071660 0x7facf415b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.768+0000 7facfa600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf413d9a0 con 0x7facf413d5c0 2024-04-25T10:16:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.768+0000 7facfa600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf415abf0 con 0x7facf4071660 2024-04-25T10:16:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.768+0000 7facfa600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf4071a90 con 0x7facf413dce0 2024-04-25T10:16:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.768+0000 7facf9200640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7facf413d5c0 0x7facf41407a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48884/0 (socket says 172.21.15.119:48884) 2024-04-25T10:16:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.768+0000 7facf9200640 1 -- 172.21.15.119:0/1242396827 learned_addr learned my addr 172.21.15.119:0/1242396827 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:16:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.769+0000 7facf9c00640 1 --2- 172.21.15.119:0/1242396827 >> v2:172.21.15.119:3300/0 conn(0x7facf4071660 0x7facf415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.769+0000 7facebe00640 1 --2- 172.21.15.119:0/1242396827 >> v2:172.21.15.132:3300/0 conn(0x7facf413dce0 0x7facf4071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.769+0000 7facf9c00640 1 -- 172.21.15.119:0/1242396827 >> v2:172.21.15.132:3300/0 conn(0x7facf413dce0 msgr2=0x7facf4071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.769+0000 7facf9c00640 1 --2- 172.21.15.119:0/1242396827 >> v2:172.21.15.132:3300/0 conn(0x7facf413dce0 0x7facf4071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.769+0000 7facf9c00640 1 -- 172.21.15.119:0/1242396827 >> v1:172.21.15.119:6789/0 conn(0x7facf413d5c0 legacy=0x7facf41407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.769+0000 7facf9c00640 1 -- 172.21.15.119:0/1242396827 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf415e650 con 0x7facf4071660 2024-04-25T10:16:10.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.770+0000 7facf9c00640 1 --2- 172.21.15.119:0/1242396827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facf4071660 0x7facf415b090 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7facf40721c0 tx=0x7facec02c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937b8ebcdbc45c75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.770+0000 7facf3e00640 1 -- 172.21.15.119:0/1242396827 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec032770 con 0x7facf4071660 2024-04-25T10:16:10.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.770+0000 7facf3e00640 1 -- 172.21.15.119:0/1242396827 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec038070 con 0x7facf4071660 2024-04-25T10:16:10.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.770+0000 7facf3e00640 1 -- 172.21.15.119:0/1242396827 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec032b30 con 0x7facf4071660 2024-04-25T10:16:10.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.771+0000 7facfa600640 1 -- 172.21.15.119:0/1242396827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facf4071660 msgr2=0x7facf415b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.771+0000 7facfa600640 1 --2- 172.21.15.119:0/1242396827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facf4071660 0x7facf415b090 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7facf40721c0 tx=0x7facec02c8b0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.771+0000 7facfa600640 1 -- 172.21.15.119:0/1242396827 shutdown_connections 2024-04-25T10:16:10.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.771+0000 7facfa600640 1 --2- 172.21.15.119:0/1242396827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facf4071660 0x7facf415b090 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.771+0000 7facfa600640 1 --2- 172.21.15.119:0/1242396827 >> v2:172.21.15.132:3300/0 conn(0x7facf413dce0 0x7facf4071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.771+0000 7facfa600640 1 -- 172.21.15.119:0/1242396827 >> 172.21.15.119:0/1242396827 conn(0x7facf406aba0 msgr2=0x7facf406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.771+0000 7facfa600640 1 -- 172.21.15.119:0/1242396827 shutdown_connections 2024-04-25T10:16:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.771+0000 7facfa600640 1 -- 172.21.15.119:0/1242396827 wait complete. 2024-04-25T10:16:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.773+0000 7facfa600640 1 Processor -- start 2024-04-25T10:16:10.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.773+0000 7facfa600640 1 -- start start 2024-04-25T10:16:10.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.774+0000 7facfa600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facf4071660 0x7facf413c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.774+0000 7facfa600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facf413d5c0 0x7facf413c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.774+0000 7facfa600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facf413dce0 0x7facf40b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.774+0000 7facfa600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf415eef0 con 0x7facf4071660 2024-04-25T10:16:10.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.774+0000 7facfa600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf415ec10 con 0x7facf413dce0 2024-04-25T10:16:10.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.774+0000 7facfa600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf40b1980 con 0x7facf413d5c0 2024-04-25T10:16:10.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.774+0000 7facf9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facf4071660 0x7facf413c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.774+0000 7facf9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facf4071660 0x7facf413c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53222/0 (socket says 172.21.15.119:53222) 2024-04-25T10:16:10.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.774+0000 7facf9200640 1 -- 172.21.15.119:0/2059381108 learned_addr learned my addr 172.21.15.119:0/2059381108 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:10.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.775+0000 7facebe00640 1 --2- 172.21.15.119:0/2059381108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facf413d5c0 0x7facf413c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.775+0000 7facf9200640 1 -- 172.21.15.119:0/2059381108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facf413d5c0 msgr2=0x7facf413c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.775+0000 7facf9c00640 1 --2- 172.21.15.119:0/2059381108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facf413dce0 0x7facf40b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.775+0000 7facf9200640 1 --2- 172.21.15.119:0/2059381108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facf413d5c0 0x7facf413c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.775+0000 7facf9200640 1 -- 172.21.15.119:0/2059381108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facf413dce0 msgr2=0x7facf40b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.775+0000 7facf9200640 1 --2- 172.21.15.119:0/2059381108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facf413dce0 0x7facf40b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.775+0000 7facf9200640 1 -- 172.21.15.119:0/2059381108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf40b5390 con 0x7facf4071660 2024-04-25T10:16:10.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.775+0000 7facf9200640 1 --2- 172.21.15.119:0/2059381108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facf4071660 0x7facf413c070 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7face400c400 tx=0x7face400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.775+0000 7facf1600640 1 -- 172.21.15.119:0/2059381108 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4017070 con 0x7facf4071660 2024-04-25T10:16:10.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.775+0000 7facf1600640 1 -- 172.21.15.119:0/2059381108 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face4005ce0 con 0x7facf4071660 2024-04-25T10:16:10.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.775+0000 7facf1600640 1 -- 172.21.15.119:0/2059381108 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40072f0 con 0x7facf4071660 2024-04-25T10:16:10.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.776+0000 7facfa600640 1 -- 172.21.15.119:0/2059381108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf40b42f0 con 0x7facf4071660 2024-04-25T10:16:10.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.776+0000 7facfa600640 1 -- 172.21.15.119:0/2059381108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf40b48b0 con 0x7facf4071660 2024-04-25T10:16:10.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.778+0000 7facf1600640 1 -- 172.21.15.119:0/2059381108 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7face401a070 con 0x7facf4071660 2024-04-25T10:16:10.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.779+0000 7facf1600640 1 --2- 172.21.15.119:0/2059381108 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7facd006b4d0 0x7facd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.779+0000 7facf1600640 1 -- 172.21.15.119:0/2059381108 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7face4089730 con 0x7facf4071660 2024-04-25T10:16:10.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.779+0000 7facebe00640 1 --2- 172.21.15.119:0/2059381108 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7facd006b4d0 0x7facd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.780+0000 7facfe978740 1 Processor -- start 2024-04-25T10:16:10.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.780+0000 7facfe978740 1 -- start start 2024-04-25T10:16:10.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.780+0000 7facebe00640 1 --2- 172.21.15.119:0/2059381108 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7facd006b4d0 0x7facd006d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7facdc006fd0 tx=0x7facdc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facfe978740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934703d70 0x557934705ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facfe978740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934740440 0x557934704150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facfe978740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55793472e830 0x55793472ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facfe978740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557934667c70 con 0x557934740440 2024-04-25T10:16:10.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facfe978740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557934610cf0 con 0x557934703d70 2024-04-25T10:16:10.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facfe978740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557934663d10 con 0x55793472e830 2024-04-25T10:16:10.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facebe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934740440 0x557934704150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facebe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934740440 0x557934704150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53236/0 (socket says 172.21.15.119:53236) 2024-04-25T10:16:10.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facebe00640 1 -- 172.21.15.119:0/1859053690 learned_addr learned my addr 172.21.15.119:0/1859053690 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facf9c00640 1 --2- 172.21.15.119:0/1859053690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55793472e830 0x55793472ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facf9200640 1 --2- 172.21.15.119:0/1859053690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934703d70 0x557934705ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facebe00640 1 -- 172.21.15.119:0/1859053690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55793472e830 msgr2=0x55793472ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facebe00640 1 --2- 172.21.15.119:0/1859053690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55793472e830 0x55793472ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facebe00640 1 -- 172.21.15.119:0/1859053690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934703d70 msgr2=0x557934705ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facebe00640 1 --2- 172.21.15.119:0/1859053690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934703d70 0x557934705ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facebe00640 1 -- 172.21.15.119:0/1859053690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579344467d0 con 0x557934740440 2024-04-25T10:16:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.781+0000 7facf9200640 1 --2- 172.21.15.119:0/1859053690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934703d70 0x557934705ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.782+0000 7facebe00640 1 --2- 172.21.15.119:0/1859053690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934740440 0x557934704150 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7facf40cecd0 tx=0x7facdc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1785e7aa5276643 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.782+0000 7face9600640 1 -- 172.21.15.119:0/1859053690 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc025070 con 0x557934740440 2024-04-25T10:16:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.782+0000 7face9600640 1 -- 172.21.15.119:0/1859053690 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc015640 con 0x557934740440 2024-04-25T10:16:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.782+0000 7face9600640 1 -- 172.21.15.119:0/1859053690 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc00f040 con 0x557934740440 2024-04-25T10:16:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.783+0000 7facfe978740 1 -- 172.21.15.119:0/1859053690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934740440 msgr2=0x557934704150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.783+0000 7facfe978740 1 --2- 172.21.15.119:0/1859053690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934740440 0x557934704150 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7facf40cecd0 tx=0x7facdc011000 comp rx=0 tx=0).stop 2024-04-25T10:16:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.783+0000 7facfe978740 1 -- 172.21.15.119:0/1859053690 shutdown_connections 2024-04-25T10:16:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.783+0000 7facfe978740 1 --2- 172.21.15.119:0/1859053690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55793472e830 0x55793472ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.783+0000 7facfe978740 1 --2- 172.21.15.119:0/1859053690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934740440 0x557934704150 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.783+0000 7facfe978740 1 --2- 172.21.15.119:0/1859053690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934703d70 0x557934705ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.783+0000 7facfe978740 1 -- 172.21.15.119:0/1859053690 >> 172.21.15.119:0/1859053690 conn(0x5579346cba30 msgr2=0x557934749830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.783+0000 7facfe978740 1 -- 172.21.15.119:0/1859053690 shutdown_connections 2024-04-25T10:16:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.783+0000 7facfe978740 1 -- 172.21.15.119:0/1859053690 wait complete. 2024-04-25T10:16:10.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.784+0000 7facfe978740 1 -- start start 2024-04-25T10:16:10.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.784+0000 7facfe978740 1 Processor -- start 2024-04-25T10:16:10.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.785+0000 7facfe978740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934703d70 0x557934742060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.785+0000 7facfe978740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55793472e830 0x55793471e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.785+0000 7facfe978740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934740440 0x55793474b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.785+0000 7facfe978740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579346063a0 con 0x557934740440 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.785+0000 7facfe978740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793472c380 con 0x557934703d70 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.785+0000 7facfe978740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557934729490 con 0x55793472e830 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.785+0000 7facfe978740 10 client.0 _ll_register_callbacks cb 0x557934623b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.785+0000 7facf9c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934740440 0x55793474b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.785+0000 7facf9c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934740440 0x55793474b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53244/0 (socket says 172.21.15.119:53244) 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.785+0000 7facf9c00640 1 -- 172.21.15.119:0/1834514116 learned_addr learned my addr 172.21.15.119:0/1834514116 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.785+0000 7facf9200640 1 --2- 172.21.15.119:0/1834514116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934703d70 0x557934742060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.785+0000 7facf9c00640 1 -- 172.21.15.119:0/1834514116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55793472e830 msgr2=0x55793471e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.785+0000 7facf9c00640 1 --2- 172.21.15.119:0/1834514116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55793472e830 0x55793471e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.786+0000 7facf9c00640 1 -- 172.21.15.119:0/1834514116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934703d70 msgr2=0x557934742060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.786+0000 7facf9c00640 1 --2- 172.21.15.119:0/1834514116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934703d70 0x557934742060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.786+0000 7facf9c00640 1 -- 172.21.15.119:0/1834514116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55793437c3f0 con 0x557934740440 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.786+0000 7facf9c00640 1 --2- 172.21.15.119:0/1834514116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934740440 0x55793474b9b0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7facec009eb0 tx=0x7facec02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dedd1f679318dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.786+0000 7face2000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.786+0000 7face2000640 1 -- 172.21.15.119:0/1834514116 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec009450 con 0x557934740440 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.786+0000 7face2000640 1 -- 172.21.15.119:0/1834514116 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facec047610 con 0x557934740440 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.786+0000 7facfe978740 1 -- 172.21.15.119:0/1834514116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facdc015640 con 0x557934740440 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.786+0000 7face2000640 1 -- 172.21.15.119:0/1834514116 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7facec00e120 con 0x557934740440 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.786+0000 7facfe978740 10 client.8322 fetch_fsmap learned FSMap version 82 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.786+0000 7facfe978740 1 -- 172.21.15.119:0/1834514116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55793433c0f0 con 0x557934740440 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.787+0000 7face2000640 1 -- 172.21.15.119:0/1834514116 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc015640 con 0x557934740440 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.787+0000 7face2000640 1 -- 172.21.15.119:0/1834514116 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7facec052020 con 0x557934740440 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.787+0000 7facfe978740 10 client.8322 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.787+0000 7facfe978740 10 client.8322 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.787+0000 7facfe978740 20 client.8322 populate_metadata read hostname 'smithi119' 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.787+0000 7facfe978740 1 --2- 172.21.15.119:0/1834514116 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55793472ecb0 0x55793472f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.787+0000 7facfe978740 4 client.8322 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.787+0000 7facfe978740 1 -- 172.21.15.119:0/1834514116 --> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579346751b0 con 0x55793472ecb0 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.787+0000 7facf9200640 1 --2- 172.21.15.119:0/1834514116 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55793472ecb0 0x55793472f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.788+0000 7facf9200640 1 --2- 172.21.15.119:0/1834514116 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55793472ecb0 0x55793472f120 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17406fa431c1cc8e server_cookie=720bde188a1679cf in_seq=0 out_seq=0 2024-04-25T10:16:10.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.788+0000 7face2000640 10 client.8322 ms_handle_connect on v2:172.21.15.119:6835/1945631445 2024-04-25T10:16:10.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.788+0000 7face2000640 1 -- 172.21.15.119:0/1834514116 <== mds.0 v2:172.21.15.119:6835/1945631445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5579346751b0 con 0x55793472ecb0 2024-04-25T10:16:10.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.788+0000 7face2000640 10 client.8322 handle_command_reply: tid=0 2024-04-25T10:16:10.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.788+0000 7facfe978740 1 client.8322 shutdown 2024-04-25T10:16:10.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.789+0000 7facfe978740 1 -- 172.21.15.119:0/1834514116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934740440 msgr2=0x55793474b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.789+0000 7facfe978740 1 --2- 172.21.15.119:0/1834514116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934740440 0x55793474b9b0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7facec009eb0 tx=0x7facec02f000 comp rx=0 tx=0).stop 2024-04-25T10:16:10.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.789+0000 7facfe978740 1 -- 172.21.15.119:0/1834514116 shutdown_connections 2024-04-25T10:16:10.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.789+0000 7facfe978740 1 --2- 172.21.15.119:0/1834514116 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55793472ecb0 0x55793472f120 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.789+0000 7facfe978740 1 --2- 172.21.15.119:0/1834514116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934740440 0x55793474b9b0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.789+0000 7facfe978740 1 --2- 172.21.15.119:0/1834514116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55793472e830 0x55793471e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.789+0000 7facfe978740 1 --2- 172.21.15.119:0/1834514116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934703d70 0x557934742060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.789+0000 7facfe978740 1 -- 172.21.15.119:0/1834514116 >> 172.21.15.119:0/1834514116 conn(0x5579346cba30 msgr2=0x55793474cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.789+0000 7face2000640 0 client.8322 ms_handle_reset on v2:172.21.15.119:6835/1945631445 2024-04-25T10:16:10.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.789+0000 7facfe978740 1 -- 172.21.15.119:0/1834514116 shutdown_connections 2024-04-25T10:16:10.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.789+0000 7facfe978740 1 -- 172.21.15.119:0/1834514116 wait complete. 2024-04-25T10:16:10.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.789+0000 7facfe978740 20 client.8322 trim_cache size 0 max 16384 2024-04-25T10:16:10.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.805+0000 7facfe978740 1 Processor -- start 2024-04-25T10:16:10.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.805+0000 7facfe978740 1 -- start start 2024-04-25T10:16:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.806+0000 7facfe978740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934703d70 0x55793472f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.806+0000 7facfe978740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55793472e830 0x55793472f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.806+0000 7facfe978740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557934740440 0x55793474ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.806+0000 7facfe978740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf41ffcb0 con 0x557934703d70 2024-04-25T10:16:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.806+0000 7facfe978740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf41feec0 con 0x55793472e830 2024-04-25T10:16:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.806+0000 7facfe978740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf41fe2c0 con 0x557934740440 2024-04-25T10:16:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.806+0000 7facf9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934703d70 0x55793472f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.806+0000 7facf9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934703d70 0x55793472f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53248/0 (socket says 172.21.15.119:53248) 2024-04-25T10:16:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.806+0000 7facf9200640 1 -- 172.21.15.119:0/2390239610 learned_addr learned my addr 172.21.15.119:0/2390239610 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.807+0000 7facebe00640 1 --2- 172.21.15.119:0/2390239610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55793472e830 0x55793472f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.807+0000 7facf9200640 1 -- 172.21.15.119:0/2390239610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557934740440 msgr2=0x55793474ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.807+0000 7facf9200640 1 --2- 172.21.15.119:0/2390239610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557934740440 0x55793474ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.807+0000 7facf9200640 1 -- 172.21.15.119:0/2390239610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55793472e830 msgr2=0x55793472f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.807+0000 7facf9200640 1 --2- 172.21.15.119:0/2390239610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55793472e830 0x55793472f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.807+0000 7facf9200640 1 -- 172.21.15.119:0/2390239610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557934663e90 con 0x557934703d70 2024-04-25T10:16:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.807+0000 7facebe00640 1 --2- 172.21.15.119:0/2390239610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55793472e830 0x55793472f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.807+0000 7facf9200640 1 --2- 172.21.15.119:0/2390239610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934703d70 0x55793472f090 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7face40541e0 tx=0x7face406d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85f1211f1bd3983 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.807+0000 7face8c00640 1 -- 172.21.15.119:0/2390239610 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4074070 con 0x557934703d70 2024-04-25T10:16:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.807+0000 7face8c00640 1 -- 172.21.15.119:0/2390239610 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face4087070 con 0x557934703d70 2024-04-25T10:16:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.807+0000 7face8c00640 1 -- 172.21.15.119:0/2390239610 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4056080 con 0x557934703d70 2024-04-25T10:16:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.808+0000 7facfe978740 1 -- 172.21.15.119:0/2390239610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934703d70 msgr2=0x55793472f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.808+0000 7facfe978740 1 --2- 172.21.15.119:0/2390239610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934703d70 0x55793472f090 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7face40541e0 tx=0x7face406d040 comp rx=0 tx=0).stop 2024-04-25T10:16:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.808+0000 7facfe978740 1 -- 172.21.15.119:0/2390239610 shutdown_connections 2024-04-25T10:16:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.808+0000 7facfe978740 1 --2- 172.21.15.119:0/2390239610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557934740440 0x55793474ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.808+0000 7facfe978740 1 --2- 172.21.15.119:0/2390239610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55793472e830 0x55793472f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.808+0000 7facfe978740 1 --2- 172.21.15.119:0/2390239610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934703d70 0x55793472f090 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.808+0000 7facfe978740 1 -- 172.21.15.119:0/2390239610 >> 172.21.15.119:0/2390239610 conn(0x5579346cba30 msgr2=0x557934705d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.808+0000 7facfe978740 1 -- 172.21.15.119:0/2390239610 shutdown_connections 2024-04-25T10:16:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.808+0000 7facfe978740 1 -- 172.21.15.119:0/2390239610 wait complete. 2024-04-25T10:16:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.809+0000 7facfe978740 1 -- start start 2024-04-25T10:16:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.809+0000 7facfe978740 1 Processor -- start 2024-04-25T10:16:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facfe978740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934703d70 0x5579347b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facfe978740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55793472e830 0x55793471c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facfe978740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934740440 0x55793471ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facfe978740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf41fe8c0 con 0x557934703d70 2024-04-25T10:16:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facfe978740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557934741e60 con 0x557934740440 2024-04-25T10:16:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facfe978740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55793473fdf0 con 0x55793472e830 2024-04-25T10:16:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facfe978740 10 client.0 _ll_register_callbacks cb 0x7facec0329e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facf9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934703d70 0x5579347b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facf9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934703d70 0x5579347b3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53262/0 (socket says 172.21.15.119:53262) 2024-04-25T10:16:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facf9200640 1 -- 172.21.15.119:0/174649152 learned_addr learned my addr 172.21.15.119:0/174649152 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facf9200640 1 -- 172.21.15.119:0/174649152 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55793472e830 msgr2=0x55793471c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facf9200640 1 --2- 172.21.15.119:0/174649152 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55793472e830 0x55793471c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facf9200640 1 -- 172.21.15.119:0/174649152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934740440 msgr2=0x55793471ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facf9200640 1 --2- 172.21.15.119:0/174649152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934740440 0x55793471ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.810+0000 7facf9200640 1 -- 172.21.15.119:0/174649152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55793472b1f0 con 0x557934703d70 2024-04-25T10:16:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7facf9200640 1 --2- 172.21.15.119:0/174649152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934703d70 0x5579347b3910 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7face4011000 tx=0x7face406f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8f88ba7475b6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7face2000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:16:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7face2000640 1 -- 172.21.15.119:0/174649152 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4074050 con 0x557934703d70 2024-04-25T10:16:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7face2000640 1 -- 172.21.15.119:0/174649152 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7face40543f0 con 0x557934703d70 2024-04-25T10:16:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7facfe978740 1 -- 172.21.15.119:0/174649152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7face4087070 con 0x557934703d70 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7face2000640 1 -- 172.21.15.119:0/174649152 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7face406fba0 con 0x557934703d70 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7facfe978740 10 client.8328 fetch_fsmap learned FSMap version 82 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7facfe978740 1 -- 172.21.15.119:0/174649152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x557934741c00 con 0x557934703d70 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7face2000640 1 -- 172.21.15.119:0/174649152 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face4087070 con 0x557934703d70 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7face2000640 1 -- 172.21.15.119:0/174649152 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7face4076ce0 con 0x557934703d70 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7facfe978740 10 client.8328 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7facfe978740 10 client.8328 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7facfe978740 20 client.8328 populate_metadata read hostname 'smithi119' 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7facfe978740 1 --2- 172.21.15.119:0/174649152 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55793471d480 0x5579347b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7facfe978740 4 client.8328 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.811+0000 7facfe978740 1 -- 172.21.15.119:0/174649152 --> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55793466f500 con 0x55793471d480 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.812+0000 7facebe00640 1 --2- 172.21.15.119:0/174649152 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55793471d480 0x5579347b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.812+0000 7facebe00640 1 --2- 172.21.15.119:0/174649152 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55793471d480 0x5579347b78a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29f11fe703071c18 server_cookie=91b9b0bc251fab5d in_seq=0 out_seq=0 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.812+0000 7face2000640 10 client.8328 ms_handle_connect on v2:172.21.15.119:6835/1945631445 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.812+0000 7face2000640 1 -- 172.21.15.119:0/174649152 <== mds.0 v2:172.21.15.119:6835/1945631445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55793466f500 con 0x55793471d480 2024-04-25T10:16:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.812+0000 7face2000640 10 client.8328 handle_command_reply: tid=0 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.812+0000 7facfe978740 1 client.8328 shutdown 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.813+0000 7facfe978740 1 -- 172.21.15.119:0/174649152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934703d70 msgr2=0x5579347b3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.813+0000 7facfe978740 1 --2- 172.21.15.119:0/174649152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934703d70 0x5579347b3910 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7face4011000 tx=0x7face406f040 comp rx=0 tx=0).stop 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.813+0000 7facfe978740 1 -- 172.21.15.119:0/174649152 shutdown_connections 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.813+0000 7facfe978740 1 --2- 172.21.15.119:0/174649152 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55793471d480 0x5579347b78a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.813+0000 7facfe978740 1 --2- 172.21.15.119:0/174649152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557934740440 0x55793471ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.813+0000 7face2000640 0 client.8328 ms_handle_reset on v2:172.21.15.119:6835/1945631445 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.813+0000 7facfe978740 1 --2- 172.21.15.119:0/174649152 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55793472e830 0x55793471c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.813+0000 7facfe978740 1 --2- 172.21.15.119:0/174649152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557934703d70 0x5579347b3910 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.813+0000 7facfe978740 1 -- 172.21.15.119:0/174649152 >> 172.21.15.119:0/174649152 conn(0x5579346cba30 msgr2=0x55793472f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.813+0000 7facfe978740 1 -- 172.21.15.119:0/174649152 shutdown_connections 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.813+0000 7facfe978740 1 -- 172.21.15.119:0/174649152 wait complete. 2024-04-25T10:16:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.813+0000 7facfe978740 20 client.8328 trim_cache size 0 max 16384 2024-04-25T10:16:10.822 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:16:10.822 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:16:10.822 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2024-04-25T10:16:10.822 INFO:teuthology.orchestra.run.smithi119.stdout: "6c170817-658c-45d3-92c0-b06c09121ba5": { 2024-04-25T10:16:10.822 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/", 2024-04-25T10:16:10.822 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "6c170817-658c-45d3-92c0-b06c09121ba5", 2024-04-25T10:16:10.822 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,repair" 2024-04-25T10:16:10.822 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:16:10.822 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:16:10.822 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:16:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.814+0000 7facfa600640 1 -- 172.21.15.119:0/2059381108 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7facd006b4d0 msgr2=0x7facd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.814+0000 7facfa600640 1 --2- 172.21.15.119:0/2059381108 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7facd006b4d0 0x7facd006d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7facdc006fd0 tx=0x7facdc008040 comp rx=0 tx=0).stop 2024-04-25T10:16:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.814+0000 7facfa600640 1 -- 172.21.15.119:0/2059381108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facf4071660 msgr2=0x7facf413c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.814+0000 7facfa600640 1 --2- 172.21.15.119:0/2059381108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facf4071660 0x7facf413c070 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7face400c400 tx=0x7face400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.814+0000 7facfa600640 1 -- 172.21.15.119:0/2059381108 shutdown_connections 2024-04-25T10:16:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.814+0000 7facfa600640 1 --2- 172.21.15.119:0/2059381108 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7facd006b4d0 0x7facd006d990 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.814+0000 7facfa600640 1 --2- 172.21.15.119:0/2059381108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7facf413dce0 0x7facf40b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.814+0000 7facfa600640 1 --2- 172.21.15.119:0/2059381108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7facf413d5c0 0x7facf413c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.814+0000 7facfa600640 1 --2- 172.21.15.119:0/2059381108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facf4071660 0x7facf413c070 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.814+0000 7facfa600640 1 -- 172.21.15.119:0/2059381108 >> 172.21.15.119:0/2059381108 conn(0x7facf406aba0 msgr2=0x7facf4139370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.814+0000 7facfa600640 1 -- 172.21.15.119:0/2059381108 shutdown_connections 2024-04-25T10:16:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:10.814+0000 7facfa600640 1 -- 172.21.15.119:0/2059381108 wait complete. 2024-04-25T10:16:10.833 INFO:tasks.cephfs.filesystem:scrub status for tag:6c170817-658c-45d3-92c0-b06c09121ba5 - {'path': '/', 'tag': '6c170817-658c-45d3-92c0-b06c09121ba5', 'options': 'recursive,repair'} 2024-04-25T10:16:11.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:11.630+0000 7fc9f4c00640 1 -- 172.21.15.119:0/337013848 <== mon.0 v2:172.21.15.119:3300/0 7 ==== log(4 entries from seq 1 at 2024-04-25T10:16:10.575710+0000) v1 ==== 695+0+0 (secure 0 0 0) 0x7fc9ec076f80 con 0x7fc9e00039e0 2024-04-25T10:16:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: cluster: 2024-04-25T10:16:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: id: e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:16:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: health: HEALTH_OK 2024-04-25T10:16:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: services: 2024-04-25T10:16:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-04-25T10:16:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: mgr: y(active, since 9m), standbys: x 2024-04-25T10:16:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:16:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-04-25T10:16:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:11.635 INFO:teuthology.orchestra.run.smithi119.stdout: data: 2024-04-25T10:16:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: volumes: 1/1 healthy 2024-04-25T10:16:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:16:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: objects: 28 objects, 13 MiB 2024-04-25T10:16:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: usage: 268 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:16:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: pgs: 97 active+clean 2024-04-25T10:16:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: io: 2024-04-25T10:16:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: client: 85 B/s rd, 1.0 MiB/s wr, 0 op/s rd, 2 op/s wr 2024-04-25T10:16:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:11.636 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:11.636 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:16:10.575710+0000 mds.a [INF] scrub queued for path: / 2024-04-25T10:16:11.636 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:16:10.575714+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:16:11.636 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:16:10.575718+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-25T10:16:11.636 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:16:10.577517+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:16:13.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:13.637+0000 7fc9f4c00640 1 -- 172.21.15.119:0/337013848 <== mon.0 v2:172.21.15.119:3300/0 8 ==== log(1 entries from seq 5 at 2024-04-25T10:16:11.651100+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7fc9ec0771e0 con 0x7fc9e00039e0 2024-04-25T10:16:13.642 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:16:11.651100+0000 mds.a [INF] scrub summary: idle 2024-04-25T10:16:32.515 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:32.523 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:40.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:40.835 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:16:41.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.049+0000 7f2d91e00640 1 Processor -- start 2024-04-25T10:16:41.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.050+0000 7f2d91e00640 1 -- start start 2024-04-25T10:16:41.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.051+0000 7f2d91e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d8c15b6b0 0x7f2d8c0b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.051+0000 7f2d91e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d8c0b4500 0x7f2d8c0ad070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.051+0000 7f2d91e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8c15ceb0 con 0x7f2d8c15c9b0 2024-04-25T10:16:41.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.051+0000 7f2d91e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8c0b4070 con 0x7f2d8c15b6b0 2024-04-25T10:16:41.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.051+0000 7f2d91e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0b3740 con 0x7f2d8c0b4500 2024-04-25T10:16:41.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.052+0000 7f2d8aa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d8c15b6b0 0x7f2d8c0b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.052+0000 7f2d8aa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d8c15b6b0 0x7f2d8c0b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42022/0 (socket says 172.21.15.119:42022) 2024-04-25T10:16:41.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.052+0000 7f2d8aa00640 1 -- 172.21.15.119:0/4171768856 learned_addr learned my addr 172.21.15.119:0/4171768856 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:41.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.052+0000 7f2d8a000640 1 --2- 172.21.15.119:0/4171768856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d8c0b4500 0x7f2d8c0ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.052+0000 7f2d8aa00640 1 -- 172.21.15.119:0/4171768856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d8c0b4500 msgr2=0x7f2d8c0ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.052+0000 7f2d8aa00640 1 --2- 172.21.15.119:0/4171768856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d8c0b4500 0x7f2d8c0ad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.052+0000 7f2d8aa00640 1 -- 172.21.15.119:0/4171768856 >> v1:172.21.15.132:6789/0 conn(0x7f2d8c15c9b0 legacy=0x7f2d8c15f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.052+0000 7f2d8aa00640 1 -- 172.21.15.119:0/4171768856 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c0b3a20 con 0x7f2d8c15b6b0 2024-04-25T10:16:41.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.053+0000 7f2d8aa00640 1 --2- 172.21.15.119:0/4171768856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c15b6b0 0x7f2d8c0b2e80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009870 tx=0x7f2d7c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2679ee60043ab284 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.054+0000 7f2d89600640 1 -- 172.21.15.119:0/4171768856 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c037070 con 0x7f2d8c15b6b0 2024-04-25T10:16:41.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.054+0000 7f2d89600640 1 -- 172.21.15.119:0/4171768856 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c032440 con 0x7f2d8c15b6b0 2024-04-25T10:16:41.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.054+0000 7f2d89600640 1 -- 172.21.15.119:0/4171768856 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c032810 con 0x7f2d8c15b6b0 2024-04-25T10:16:41.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.054+0000 7f2d91e00640 1 -- 172.21.15.119:0/4171768856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c15b6b0 msgr2=0x7f2d8c0b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.054+0000 7f2d91e00640 1 --2- 172.21.15.119:0/4171768856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c15b6b0 0x7f2d8c0b2e80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009870 tx=0x7f2d7c02a210 comp rx=0 tx=0).stop 2024-04-25T10:16:41.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.054+0000 7f2d91e00640 1 -- 172.21.15.119:0/4171768856 shutdown_connections 2024-04-25T10:16:41.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.054+0000 7f2d91e00640 1 --2- 172.21.15.119:0/4171768856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d8c0b4500 0x7f2d8c0ad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.054+0000 7f2d91e00640 1 --2- 172.21.15.119:0/4171768856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c15b6b0 0x7f2d8c0b2e80 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.054+0000 7f2d91e00640 1 -- 172.21.15.119:0/4171768856 >> 172.21.15.119:0/4171768856 conn(0x7f2d8c0cff30 msgr2=0x7f2d8c0d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.054+0000 7f2d91e00640 1 -- 172.21.15.119:0/4171768856 shutdown_connections 2024-04-25T10:16:41.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.054+0000 7f2d91e00640 1 -- 172.21.15.119:0/4171768856 wait complete. 2024-04-25T10:16:41.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.056+0000 7f2d91e00640 1 Processor -- start 2024-04-25T10:16:41.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.056+0000 7f2d91e00640 1 -- start start 2024-04-25T10:16:41.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.057+0000 7f2d91e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0b4500 0x7f2d8c099a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.057+0000 7f2d91e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d8c15b6b0 0x7f2d8c1454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.057+0000 7f2d91e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d8c15c9b0 0x7f2d8c15f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.057+0000 7f2d91e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c15ba90 con 0x7f2d8c0b4500 2024-04-25T10:16:41.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.057+0000 7f2d91e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0af680 con 0x7f2d8c15c9b0 2024-04-25T10:16:41.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.057+0000 7f2d91e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c15bc10 con 0x7f2d8c15b6b0 2024-04-25T10:16:41.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.057+0000 7f2d8aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0b4500 0x7f2d8c099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.057+0000 7f2d8aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0b4500 0x7f2d8c099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42028/0 (socket says 172.21.15.119:42028) 2024-04-25T10:16:41.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.057+0000 7f2d8aa00640 1 -- 172.21.15.119:0/2683123264 learned_addr learned my addr 172.21.15.119:0/2683123264 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:41.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.058+0000 7f2d8b400640 1 --2- 172.21.15.119:0/2683123264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d8c15c9b0 0x7f2d8c15f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.058+0000 7f2d8aa00640 1 -- 172.21.15.119:0/2683123264 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d8c15b6b0 msgr2=0x7f2d8c1454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.058+0000 7f2d8aa00640 1 --2- 172.21.15.119:0/2683123264 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d8c15b6b0 0x7f2d8c1454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.058+0000 7f2d8aa00640 1 -- 172.21.15.119:0/2683123264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d8c15c9b0 msgr2=0x7f2d8c15f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.058+0000 7f2d8aa00640 1 --2- 172.21.15.119:0/2683123264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d8c15c9b0 0x7f2d8c15f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.058+0000 7f2d8aa00640 1 -- 172.21.15.119:0/2683123264 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c09d870 con 0x7f2d8c0b4500 2024-04-25T10:16:41.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.058+0000 7f2d8aa00640 1 --2- 172.21.15.119:0/2683123264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0b4500 0x7f2d8c099a40 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009870 tx=0x7f2d7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.058+0000 7f2d83400640 1 -- 172.21.15.119:0/2683123264 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c041020 con 0x7f2d8c0b4500 2024-04-25T10:16:41.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.058+0000 7f2d83400640 1 -- 172.21.15.119:0/2683123264 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c0306c0 con 0x7f2d8c0b4500 2024-04-25T10:16:41.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.058+0000 7f2d83400640 1 -- 172.21.15.119:0/2683123264 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c037040 con 0x7f2d8c0b4500 2024-04-25T10:16:41.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.059+0000 7f2d91e00640 1 -- 172.21.15.119:0/2683123264 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c09db50 con 0x7f2d8c0b4500 2024-04-25T10:16:41.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.059+0000 7f2d91e00640 1 -- 172.21.15.119:0/2683123264 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c1e8050 con 0x7f2d8c0b4500 2024-04-25T10:16:41.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.061+0000 7f2d83400640 1 -- 172.21.15.119:0/2683123264 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d7c044020 con 0x7f2d8c0b4500 2024-04-25T10:16:41.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.062+0000 7f2d83400640 1 --2- 172.21.15.119:0/2683123264 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d6c06b4d0 0x7f2d6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.062+0000 7f2d83400640 1 -- 172.21.15.119:0/2683123264 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d7c0aaac0 con 0x7f2d8c0b4500 2024-04-25T10:16:41.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.062+0000 7f2d8a000640 1 --2- 172.21.15.119:0/2683123264 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d6c06b4d0 0x7f2d6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.062+0000 7f2d94c57740 1 Processor -- start 2024-04-25T10:16:41.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.062+0000 7f2d94c57740 1 -- start start 2024-04-25T10:16:41.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.063+0000 7f2d8a000640 1 --2- 172.21.15.119:0/2683123264 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d6c06b4d0 0x7f2d6c06d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2d78006fd0 tx=0x7f2d78008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.063+0000 7f2d94c57740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35cec0fc0 0x55e35cec3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.063+0000 7f2d94c57740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35cefd690 0x55e35cec13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.063+0000 7f2d94c57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35ceeba80 0x55e35ceebf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.063+0000 7f2d94c57740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35ce20c70 con 0x55e35ceeba80 2024-04-25T10:16:41.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.063+0000 7f2d94c57740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35cdc9cf0 con 0x55e35cec0fc0 2024-04-25T10:16:41.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.063+0000 7f2d94c57740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e35ce1cd10 con 0x55e35cefd690 2024-04-25T10:16:41.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.064+0000 7f2d8b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35ceeba80 0x55e35ceebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.064+0000 7f2d8b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35ceeba80 0x55e35ceebf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42030/0 (socket says 172.21.15.119:42030) 2024-04-25T10:16:41.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.064+0000 7f2d8b400640 1 -- 172.21.15.119:0/838703720 learned_addr learned my addr 172.21.15.119:0/838703720 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:41.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.064+0000 7f2d8aa00640 1 --2- 172.21.15.119:0/838703720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35cec0fc0 0x55e35cec3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.064+0000 7f2d8b400640 1 -- 172.21.15.119:0/838703720 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35cefd690 msgr2=0x55e35cec13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.064+0000 7f2d8b400640 1 --2- 172.21.15.119:0/838703720 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35cefd690 0x55e35cec13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.064+0000 7f2d8b400640 1 -- 172.21.15.119:0/838703720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35cec0fc0 msgr2=0x55e35cec3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.064+0000 7f2d8b400640 1 --2- 172.21.15.119:0/838703720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35cec0fc0 0x55e35cec3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.064+0000 7f2d8b400640 1 -- 172.21.15.119:0/838703720 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e35cbff7d0 con 0x55e35ceeba80 2024-04-25T10:16:41.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.064+0000 7f2d8aa00640 1 --2- 172.21.15.119:0/838703720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35cec0fc0 0x55e35cec3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:41.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.064+0000 7f2d8b400640 1 --2- 172.21.15.119:0/838703720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35ceeba80 0x55e35ceebf00 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f2d84004490 tx=0x7f2d8400d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ac6e75ee5f5766 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.065+0000 7f2d77400640 1 -- 172.21.15.119:0/838703720 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84016020 con 0x55e35ceeba80 2024-04-25T10:16:41.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.065+0000 7f2d77400640 1 -- 172.21.15.119:0/838703720 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8400dd50 con 0x55e35ceeba80 2024-04-25T10:16:41.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.065+0000 7f2d77400640 1 -- 172.21.15.119:0/838703720 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84012630 con 0x55e35ceeba80 2024-04-25T10:16:41.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.065+0000 7f2d94c57740 1 -- 172.21.15.119:0/838703720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35ceeba80 msgr2=0x55e35ceebf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.065+0000 7f2d94c57740 1 --2- 172.21.15.119:0/838703720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35ceeba80 0x55e35ceebf00 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f2d84004490 tx=0x7f2d8400d150 comp rx=0 tx=0).stop 2024-04-25T10:16:41.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.065+0000 7f2d94c57740 1 -- 172.21.15.119:0/838703720 shutdown_connections 2024-04-25T10:16:41.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.065+0000 7f2d94c57740 1 --2- 172.21.15.119:0/838703720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35ceeba80 0x55e35ceebf00 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.065+0000 7f2d94c57740 1 --2- 172.21.15.119:0/838703720 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35cefd690 0x55e35cec13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.065+0000 7f2d94c57740 1 --2- 172.21.15.119:0/838703720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35cec0fc0 0x55e35cec3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.065+0000 7f2d94c57740 1 -- 172.21.15.119:0/838703720 >> 172.21.15.119:0/838703720 conn(0x55e35ce84a30 msgr2=0x55e35cf06a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.066+0000 7f2d94c57740 1 -- 172.21.15.119:0/838703720 shutdown_connections 2024-04-25T10:16:41.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.066+0000 7f2d94c57740 1 -- 172.21.15.119:0/838703720 wait complete. 2024-04-25T10:16:41.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.066+0000 7f2d94c57740 1 -- start start 2024-04-25T10:16:41.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.066+0000 7f2d94c57740 1 Processor -- start 2024-04-25T10:16:41.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.067+0000 7f2d94c57740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35cec0fc0 0x55e35cee2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.067+0000 7f2d94c57740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35ceeba80 0x55e35ceebe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.067+0000 7f2d94c57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cefd690 0x55e35cee60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.067+0000 7f2d94c57740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35cdbf3a0 con 0x55e35cefd690 2024-04-25T10:16:41.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.068+0000 7f2d94c57740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35cee95d0 con 0x55e35cec0fc0 2024-04-25T10:16:41.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.068+0000 7f2d94c57740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e35cee4d60 con 0x55e35ceeba80 2024-04-25T10:16:41.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.068+0000 7f2d94c57740 10 client.0 _ll_register_callbacks cb 0x55e35cddcb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:41.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.068+0000 7f2d8b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cefd690 0x55e35cee60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.068+0000 7f2d8b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cefd690 0x55e35cee60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42036/0 (socket says 172.21.15.119:42036) 2024-04-25T10:16:41.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.068+0000 7f2d8b400640 1 -- 172.21.15.119:0/2143829552 learned_addr learned my addr 172.21.15.119:0/2143829552 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:41.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.068+0000 7f2d8aa00640 1 --2- 172.21.15.119:0/2143829552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35cec0fc0 0x55e35cee2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.068+0000 7f2d8a000640 1 --2- 172.21.15.119:0/2143829552 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35ceeba80 0x55e35ceebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.068+0000 7f2d8b400640 1 -- 172.21.15.119:0/2143829552 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35ceeba80 msgr2=0x55e35ceebe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.068+0000 7f2d8b400640 1 --2- 172.21.15.119:0/2143829552 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35ceeba80 0x55e35ceebe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.068+0000 7f2d8b400640 1 -- 172.21.15.119:0/2143829552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35cec0fc0 msgr2=0x55e35cee2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.068+0000 7f2d8b400640 1 --2- 172.21.15.119:0/2143829552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35cec0fc0 0x55e35cee2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.068+0000 7f2d8b400640 1 -- 172.21.15.119:0/2143829552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e35cb353f0 con 0x55e35cefd690 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.069+0000 7f2d8b400640 1 --2- 172.21.15.119:0/2143829552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cefd690 0x55e35cee60e0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009010 tx=0x7f2d84003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93be3f218f4ec746 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.069+0000 7f2d6be00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.069+0000 7f2d6be00640 1 -- 172.21.15.119:0/2143829552 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84004030 con 0x55e35cefd690 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.069+0000 7f2d6be00640 1 -- 172.21.15.119:0/2143829552 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d8401cc90 con 0x55e35cefd690 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.069+0000 7f2d94c57740 1 -- 172.21.15.119:0/2143829552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d8400dd50 con 0x55e35cefd690 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.069+0000 7f2d6be00640 1 -- 172.21.15.119:0/2143829552 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d84004950 con 0x55e35cefd690 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.069+0000 7f2d94c57740 10 client.8340 fetch_fsmap learned FSMap version 82 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.069+0000 7f2d94c57740 1 -- 172.21.15.119:0/2143829552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55e35caf50f0 con 0x55e35cefd690 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.070+0000 7f2d6be00640 1 -- 172.21.15.119:0/2143829552 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8400dd50 con 0x55e35cefd690 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.070+0000 7f2d6be00640 1 -- 172.21.15.119:0/2143829552 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d84030470 con 0x55e35cefd690 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.070+0000 7f2d94c57740 10 client.8340 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.070+0000 7f2d94c57740 10 client.8340 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.070+0000 7f2d94c57740 20 client.8340 populate_metadata read hostname 'smithi119' 2024-04-25T10:16:41.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.070+0000 7f2d94c57740 1 --2- 172.21.15.119:0/2143829552 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55e35cee2e20 0x55e35cf19bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.070+0000 7f2d94c57740 4 client.8340 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:41.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.070+0000 7f2d94c57740 1 -- 172.21.15.119:0/2143829552 --> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e35ce2e1b0 con 0x55e35cee2e20 2024-04-25T10:16:41.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.070+0000 7f2d8aa00640 1 --2- 172.21.15.119:0/2143829552 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55e35cee2e20 0x55e35cf19bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.070+0000 7f2d8aa00640 1 --2- 172.21.15.119:0/2143829552 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55e35cee2e20 0x55e35cf19bf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def3916115d05fde server_cookie=60285d2473560348 in_seq=0 out_seq=0 2024-04-25T10:16:41.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.070+0000 7f2d6be00640 10 client.8340 ms_handle_connect on v2:172.21.15.119:6835/1945631445 2024-04-25T10:16:41.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.071+0000 7f2d6be00640 1 -- 172.21.15.119:0/2143829552 <== mds.0 v2:172.21.15.119:6835/1945631445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e35ce2e1b0 con 0x55e35cee2e20 2024-04-25T10:16:41.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.071+0000 7f2d6be00640 10 client.8340 handle_command_reply: tid=0 2024-04-25T10:16:41.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.071+0000 7f2d94c57740 1 client.8340 shutdown 2024-04-25T10:16:41.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.071+0000 7f2d94c57740 1 -- 172.21.15.119:0/2143829552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cefd690 msgr2=0x55e35cee60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.071+0000 7f2d94c57740 1 --2- 172.21.15.119:0/2143829552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cefd690 0x55e35cee60e0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009010 tx=0x7f2d84003a70 comp rx=0 tx=0).stop 2024-04-25T10:16:41.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.071+0000 7f2d94c57740 1 -- 172.21.15.119:0/2143829552 shutdown_connections 2024-04-25T10:16:41.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.071+0000 7f2d94c57740 1 --2- 172.21.15.119:0/2143829552 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55e35cee2e20 0x55e35cf19bf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.071+0000 7f2d94c57740 1 --2- 172.21.15.119:0/2143829552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cefd690 0x55e35cee60e0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.071+0000 7f2d94c57740 1 --2- 172.21.15.119:0/2143829552 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35ceeba80 0x55e35ceebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.071+0000 7f2d94c57740 1 --2- 172.21.15.119:0/2143829552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35cec0fc0 0x55e35cee2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.071+0000 7f2d6be00640 0 client.8340 ms_handle_reset on v2:172.21.15.119:6835/1945631445 2024-04-25T10:16:41.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.072+0000 7f2d94c57740 1 -- 172.21.15.119:0/2143829552 >> 172.21.15.119:0/2143829552 conn(0x55e35ce84a30 msgr2=0x55e35cee99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.072+0000 7f2d94c57740 1 -- 172.21.15.119:0/2143829552 shutdown_connections 2024-04-25T10:16:41.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.072+0000 7f2d94c57740 1 -- 172.21.15.119:0/2143829552 wait complete. 2024-04-25T10:16:41.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.072+0000 7f2d94c57740 20 client.8340 trim_cache size 0 max 16384 2024-04-25T10:16:41.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.089+0000 7f2d94c57740 1 Processor -- start 2024-04-25T10:16:41.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.089+0000 7f2d94c57740 1 -- start start 2024-04-25T10:16:41.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.090+0000 7f2d94c57740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35cec0fc0 0x55e35ceedd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.090+0000 7f2d94c57740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35ceeba80 0x55e35ceee260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.090+0000 7f2d94c57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cefd690 0x55e35cf18b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.090+0000 7f2d94c57740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35cee5260 con 0x55e35cefd690 2024-04-25T10:16:41.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.090+0000 7f2d94c57740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c1e6d30 con 0x55e35ceeba80 2024-04-25T10:16:41.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.090+0000 7f2d94c57740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c06f7d0 con 0x55e35cec0fc0 2024-04-25T10:16:41.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.090+0000 7f2d8b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cefd690 0x55e35cf18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.090+0000 7f2d8b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cefd690 0x55e35cf18b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42044/0 (socket says 172.21.15.119:42044) 2024-04-25T10:16:41.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.090+0000 7f2d8b400640 1 -- 172.21.15.119:0/77554040 learned_addr learned my addr 172.21.15.119:0/77554040 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:41.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.091+0000 7f2d8a000640 1 --2- 172.21.15.119:0/77554040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35ceeba80 0x55e35ceee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.091+0000 7f2d8b400640 1 -- 172.21.15.119:0/77554040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35cec0fc0 msgr2=0x55e35ceedd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.091+0000 7f2d8b400640 1 --2- 172.21.15.119:0/77554040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35cec0fc0 0x55e35ceedd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.091+0000 7f2d8b400640 1 -- 172.21.15.119:0/77554040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35ceeba80 msgr2=0x55e35ceee260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.091+0000 7f2d8b400640 1 --2- 172.21.15.119:0/77554040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35ceeba80 0x55e35ceee260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.091+0000 7f2d8b400640 1 -- 172.21.15.119:0/77554040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e35cf0ac80 con 0x55e35cefd690 2024-04-25T10:16:41.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.091+0000 7f2d8b400640 1 --2- 172.21.15.119:0/77554040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cefd690 0x55e35cf18b00 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f2d84007c50 tx=0x7f2d8401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55602406603b1b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.091+0000 7f2d76a00640 1 -- 172.21.15.119:0/77554040 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84016030 con 0x55e35cefd690 2024-04-25T10:16:41.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.091+0000 7f2d76a00640 1 -- 172.21.15.119:0/77554040 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d84030d10 con 0x55e35cefd690 2024-04-25T10:16:41.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.091+0000 7f2d76a00640 1 -- 172.21.15.119:0/77554040 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84004030 con 0x55e35cefd690 2024-04-25T10:16:41.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.092+0000 7f2d94c57740 1 -- 172.21.15.119:0/77554040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cefd690 msgr2=0x55e35cf18b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.092+0000 7f2d94c57740 1 --2- 172.21.15.119:0/77554040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cefd690 0x55e35cf18b00 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f2d84007c50 tx=0x7f2d8401e040 comp rx=0 tx=0).stop 2024-04-25T10:16:41.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.092+0000 7f2d94c57740 1 -- 172.21.15.119:0/77554040 shutdown_connections 2024-04-25T10:16:41.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.092+0000 7f2d94c57740 1 --2- 172.21.15.119:0/77554040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cefd690 0x55e35cf18b00 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.092+0000 7f2d94c57740 1 --2- 172.21.15.119:0/77554040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35ceeba80 0x55e35ceee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.092+0000 7f2d94c57740 1 --2- 172.21.15.119:0/77554040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35cec0fc0 0x55e35ceedd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.092+0000 7f2d94c57740 1 -- 172.21.15.119:0/77554040 >> 172.21.15.119:0/77554040 conn(0x55e35ce84a30 msgr2=0x55e35cee2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.092+0000 7f2d94c57740 1 -- 172.21.15.119:0/77554040 shutdown_connections 2024-04-25T10:16:41.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.092+0000 7f2d94c57740 1 -- 172.21.15.119:0/77554040 wait complete. 2024-04-25T10:16:41.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.093+0000 7f2d94c57740 1 -- start start 2024-04-25T10:16:41.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.093+0000 7f2d94c57740 1 Processor -- start 2024-04-25T10:16:41.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.094+0000 7f2d94c57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cec0fc0 0x55e35cf706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.094+0000 7f2d94c57740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35ceeba80 0x55e35cf18b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.094+0000 7f2d94c57740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35cefd690 0x55e35cf19040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.094+0000 7f2d94c57740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c06f1d0 con 0x55e35cec0fc0 2024-04-25T10:16:41.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.094+0000 7f2d94c57740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c06ebd0 con 0x55e35ceeba80 2024-04-25T10:16:41.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.094+0000 7f2d94c57740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e35cf70db0 con 0x55e35cefd690 2024-04-25T10:16:41.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.094+0000 7f2d94c57740 10 client.0 _ll_register_callbacks cb 0x7f2d840042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:41.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.094+0000 7f2d8aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cec0fc0 0x55e35cf706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.094+0000 7f2d8aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cec0fc0 0x55e35cf706f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42046/0 (socket says 172.21.15.119:42046) 2024-04-25T10:16:41.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.094+0000 7f2d8aa00640 1 -- 172.21.15.119:0/734468756 learned_addr learned my addr 172.21.15.119:0/734468756 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:41.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.095+0000 7f2d8aa00640 1 -- 172.21.15.119:0/734468756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35cefd690 msgr2=0x55e35cf19040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.095+0000 7f2d8aa00640 1 --2- 172.21.15.119:0/734468756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35cefd690 0x55e35cf19040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.095+0000 7f2d8aa00640 1 -- 172.21.15.119:0/734468756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35ceeba80 msgr2=0x55e35cf18b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.095+0000 7f2d8aa00640 1 --2- 172.21.15.119:0/734468756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35ceeba80 0x55e35cf18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.095+0000 7f2d8aa00640 1 -- 172.21.15.119:0/734468756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e35cee6c40 con 0x55e35cec0fc0 2024-04-25T10:16:41.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.095+0000 7f2d8aa00640 1 --2- 172.21.15.119:0/734468756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cec0fc0 0x55e35cf706f0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009080 tx=0x7f2d7c07d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a753195b6401dfc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.095+0000 7f2d6be00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:16:41.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.095+0000 7f2d6be00640 1 -- 172.21.15.119:0/734468756 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c075810 con 0x55e35cec0fc0 2024-04-25T10:16:41.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.095+0000 7f2d6be00640 1 -- 172.21.15.119:0/734468756 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d7c081420 con 0x55e35cec0fc0 2024-04-25T10:16:41.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.095+0000 7f2d94c57740 1 -- 172.21.15.119:0/734468756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d84030d10 con 0x55e35cec0fc0 2024-04-25T10:16:41.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.095+0000 7f2d6be00640 1 -- 172.21.15.119:0/734468756 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d7c0759b0 con 0x55e35cec0fc0 2024-04-25T10:16:41.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.095+0000 7f2d94c57740 10 client.8346 fetch_fsmap learned FSMap version 82 2024-04-25T10:16:41.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.095+0000 7f2d94c57740 1 -- 172.21.15.119:0/734468756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55e35cf27af0 con 0x55e35cec0fc0 2024-04-25T10:16:41.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.096+0000 7f2d6be00640 1 -- 172.21.15.119:0/734468756 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d84030d10 con 0x55e35cec0fc0 2024-04-25T10:16:41.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.096+0000 7f2d6be00640 1 -- 172.21.15.119:0/734468756 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d7c03a030 con 0x55e35cec0fc0 2024-04-25T10:16:41.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.096+0000 7f2d94c57740 10 client.8346 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:16:41.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.096+0000 7f2d94c57740 10 client.8346 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:16:41.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.096+0000 7f2d94c57740 20 client.8346 populate_metadata read hostname 'smithi119' 2024-04-25T10:16:41.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.096+0000 7f2d94c57740 1 --2- 172.21.15.119:0/734468756 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55e35cee54e0 0x55e35cee5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.096+0000 7f2d94c57740 4 client.8346 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:16:41.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.096+0000 7f2d94c57740 1 -- 172.21.15.119:0/734468756 --> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e35ce28500 con 0x55e35cee54e0 2024-04-25T10:16:41.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.096+0000 7f2d8a000640 1 --2- 172.21.15.119:0/734468756 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55e35cee54e0 0x55e35cee5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d8a000640 1 --2- 172.21.15.119:0/734468756 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55e35cee54e0 0x55e35cee5910 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab415038ecf11855 server_cookie=f0af1dd1aee49c13 in_seq=0 out_seq=0 2024-04-25T10:16:41.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d6be00640 10 client.8346 ms_handle_connect on v2:172.21.15.119:6835/1945631445 2024-04-25T10:16:41.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d6be00640 1 -- 172.21.15.119:0/734468756 <== mds.0 v2:172.21.15.119:6835/1945631445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e35ce28500 con 0x55e35cee54e0 2024-04-25T10:16:41.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d6be00640 10 client.8346 handle_command_reply: tid=0 2024-04-25T10:16:41.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d94c57740 1 client.8346 shutdown 2024-04-25T10:16:41.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d94c57740 1 -- 172.21.15.119:0/734468756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cec0fc0 msgr2=0x55e35cf706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d94c57740 1 --2- 172.21.15.119:0/734468756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cec0fc0 0x55e35cf706f0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009080 tx=0x7f2d7c07d3b0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d94c57740 1 -- 172.21.15.119:0/734468756 shutdown_connections 2024-04-25T10:16:41.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d94c57740 1 --2- 172.21.15.119:0/734468756 >> [v2:172.21.15.119:6835/1945631445,v1:172.21.15.119:6837/1945631445] conn(0x55e35cee54e0 0x55e35cee5910 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d94c57740 1 --2- 172.21.15.119:0/734468756 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55e35cefd690 0x55e35cf19040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d94c57740 1 --2- 172.21.15.119:0/734468756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e35ceeba80 0x55e35cf18b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d94c57740 1 --2- 172.21.15.119:0/734468756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e35cec0fc0 0x55e35cf706f0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d6be00640 0 client.8346 ms_handle_reset on v2:172.21.15.119:6835/1945631445 2024-04-25T10:16:41.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d94c57740 1 -- 172.21.15.119:0/734468756 >> 172.21.15.119:0/734468756 conn(0x55e35ce84a30 msgr2=0x55e35cf09f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d94c57740 1 -- 172.21.15.119:0/734468756 shutdown_connections 2024-04-25T10:16:41.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.097+0000 7f2d94c57740 1 -- 172.21.15.119:0/734468756 wait complete. 2024-04-25T10:16:41.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.098+0000 7f2d94c57740 20 client.8346 trim_cache size 0 max 16384 2024-04-25T10:16:41.105 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:16:41.105 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2024-04-25T10:16:41.105 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2024-04-25T10:16:41.105 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:16:41.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.098+0000 7f2d91e00640 1 -- 172.21.15.119:0/2683123264 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d6c06b4d0 msgr2=0x7f2d6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.098+0000 7f2d91e00640 1 --2- 172.21.15.119:0/2683123264 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d6c06b4d0 0x7f2d6c06d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2d78006fd0 tx=0x7f2d78008040 comp rx=0 tx=0).stop 2024-04-25T10:16:41.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.098+0000 7f2d91e00640 1 -- 172.21.15.119:0/2683123264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0b4500 msgr2=0x7f2d8c099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.098+0000 7f2d91e00640 1 --2- 172.21.15.119:0/2683123264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0b4500 0x7f2d8c099a40 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009870 tx=0x7f2d7c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.098+0000 7f2d91e00640 1 -- 172.21.15.119:0/2683123264 shutdown_connections 2024-04-25T10:16:41.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.098+0000 7f2d91e00640 1 --2- 172.21.15.119:0/2683123264 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2d6c06b4d0 0x7f2d6c06d990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.098+0000 7f2d91e00640 1 --2- 172.21.15.119:0/2683123264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d8c15c9b0 0x7f2d8c15f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.098+0000 7f2d91e00640 1 --2- 172.21.15.119:0/2683123264 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2d8c15b6b0 0x7f2d8c1454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.098+0000 7f2d91e00640 1 --2- 172.21.15.119:0/2683123264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0b4500 0x7f2d8c099a40 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.098+0000 7f2d91e00640 1 -- 172.21.15.119:0/2683123264 >> 172.21.15.119:0/2683123264 conn(0x7f2d8c0cff30 msgr2=0x7f2d8c0cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.098+0000 7f2d91e00640 1 -- 172.21.15.119:0/2683123264 shutdown_connections 2024-04-25T10:16:41.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.099+0000 7f2d91e00640 1 -- 172.21.15.119:0/2683123264 wait complete. 2024-04-25T10:16:41.120 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:16:41.221 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:16:41.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.412+0000 7feff3800640 1 Processor -- start 2024-04-25T10:16:41.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.412+0000 7feff3800640 1 -- start start 2024-04-25T10:16:41.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.414+0000 7feff3800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fefec071200 0x7fefec071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.414+0000 7feff3800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefec0cc9a0 con 0x7fefec0cee90 2024-04-25T10:16:41.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.414+0000 7feff3800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefec0ccc80 con 0x7fefec145260 2024-04-25T10:16:41.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.414+0000 7feff3800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefec071ba0 con 0x7fefec071200 2024-04-25T10:16:41.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.414+0000 7feff1800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fefec0cee90 0x7fefec0cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56488/0 (socket says 172.21.15.119:56488) 2024-04-25T10:16:41.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.414+0000 7feff1800640 1 -- 172.21.15.119:0/867386579 learned_addr learned my addr 172.21.15.119:0/867386579 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:16:41.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.414+0000 7fefe3e00640 1 --2- 172.21.15.119:0/867386579 >> v2:172.21.15.132:3300/0 conn(0x7fefec071200 0x7fefec071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 -- 172.21.15.119:0/867386579 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217862558 0 0) 0x7fefec0cc9a0 con 0x7fefec0cee90 2024-04-25T10:16:41.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 -- 172.21.15.119:0/867386579 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefd0003520 con 0x7fefec0cee90 2024-04-25T10:16:41.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 -- 172.21.15.119:0/867386579 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2594039505 0 0) 0x7fefd0003520 con 0x7fefec0cee90 2024-04-25T10:16:41.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 -- 172.21.15.119:0/867386579 >> v2:172.21.15.132:3300/0 conn(0x7fefec071200 msgr2=0x7fefec071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 --2- 172.21.15.119:0/867386579 >> v2:172.21.15.132:3300/0 conn(0x7fefec071200 0x7fefec071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 -- 172.21.15.119:0/867386579 >> v1:172.21.15.132:6789/0 conn(0x7fefec145260 legacy=0x7fefec0cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 -- 172.21.15.119:0/867386579 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec0ccf60 con 0x7fefec0cee90 2024-04-25T10:16:41.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 -- 172.21.15.119:0/867386579 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fefe40019f0 con 0x7fefec0cee90 2024-04-25T10:16:41.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 -- 172.21.15.119:0/867386579 >> v1:172.21.15.119:6789/0 conn(0x7fefec0cee90 legacy=0x7fefec0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 --2- 172.21.15.119:0/867386579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd00039b0 0x7fefd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 --2- 172.21.15.119:0/867386579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefd0004800 0x7fefd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 --2- 172.21.15.119:0/867386579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefd00051c0 0x7fefd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 -- 172.21.15.119:0/867386579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd000c030 con 0x7fefd00039b0 2024-04-25T10:16:41.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 -- 172.21.15.119:0/867386579 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd000c1e0 con 0x7fefd0004800 2024-04-25T10:16:41.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.415+0000 7fefebe00640 1 -- 172.21.15.119:0/867386579 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd000c3e0 con 0x7fefd00051c0 2024-04-25T10:16:41.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.416+0000 7feff0e00640 1 --2- 172.21.15.119:0/867386579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd00039b0 0x7fefd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.416+0000 7fefe3e00640 1 --2- 172.21.15.119:0/867386579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefd0004800 0x7fefd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.416+0000 7feff1800640 1 --2- 172.21.15.119:0/867386579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefd00051c0 0x7fefd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.416+0000 7feff0e00640 1 -- 172.21.15.119:0/867386579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefd00051c0 msgr2=0x7fefd000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.416+0000 7feff0e00640 1 --2- 172.21.15.119:0/867386579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefd00051c0 0x7fefd000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.416+0000 7feff0e00640 1 -- 172.21.15.119:0/867386579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefd0004800 msgr2=0x7fefd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.416+0000 7feff0e00640 1 --2- 172.21.15.119:0/867386579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefd0004800 0x7fefd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.416+0000 7feff0e00640 1 -- 172.21.15.119:0/867386579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd0003520 con 0x7fefd00039b0 2024-04-25T10:16:41.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.416+0000 7fefe3e00640 1 --2- 172.21.15.119:0/867386579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefd0004800 0x7fefd0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:41.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.416+0000 7feff1800640 1 --2- 172.21.15.119:0/867386579 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefd00051c0 0x7fefd000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:41.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.416+0000 7feff1800640 1 -- 172.21.15.119:0/867386579 reap_dead start 2024-04-25T10:16:41.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.417+0000 7feff0e00640 1 --2- 172.21.15.119:0/867386579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd00039b0 0x7fefd0003dd0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fefdc001750 tx=0x7fefd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96c20b8ce0299ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.418+0000 7fefebe00640 1 -- 172.21.15.119:0/867386579 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc001370 con 0x7fefd00039b0 2024-04-25T10:16:41.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.418+0000 7fefebe00640 1 -- 172.21.15.119:0/867386579 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefdc0052d0 con 0x7fefd00039b0 2024-04-25T10:16:41.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.418+0000 7fefebe00640 1 -- 172.21.15.119:0/867386579 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc02abc0 con 0x7fefd00039b0 2024-04-25T10:16:41.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.418+0000 7feff3800640 1 -- 172.21.15.119:0/867386579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd00039b0 msgr2=0x7fefd0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.418+0000 7feff3800640 1 --2- 172.21.15.119:0/867386579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd00039b0 0x7fefd0003dd0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fefdc001750 tx=0x7fefd0005830 comp rx=0 tx=0).stop 2024-04-25T10:16:41.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.418+0000 7feff3800640 1 -- 172.21.15.119:0/867386579 shutdown_connections 2024-04-25T10:16:41.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.418+0000 7feff3800640 1 --2- 172.21.15.119:0/867386579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd00039b0 0x7fefd0003dd0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.418+0000 7feff3800640 1 -- 172.21.15.119:0/867386579 >> 172.21.15.119:0/867386579 conn(0x7fefec06aba0 msgr2=0x7fefec06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.418+0000 7feff3800640 1 -- 172.21.15.119:0/867386579 shutdown_connections 2024-04-25T10:16:41.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.418+0000 7feff3800640 1 -- 172.21.15.119:0/867386579 wait complete. 2024-04-25T10:16:41.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.420+0000 7feff3800640 1 Processor -- start 2024-04-25T10:16:41.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.420+0000 7feff3800640 1 -- start start 2024-04-25T10:16:41.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.421+0000 7feff3800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd00039b0 0x7fefec136f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.421+0000 7feff3800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefec15b0e0 0x7fefec0ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.421+0000 7feff3800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefec15ab90 0x7fefec0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.421+0000 7feff3800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec145640 con 0x7fefd00039b0 2024-04-25T10:16:41.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.421+0000 7feff3800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec0cea00 con 0x7fefec15b0e0 2024-04-25T10:16:41.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.421+0000 7feff3800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefec15c270 con 0x7fefec15ab90 2024-04-25T10:16:41.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.422+0000 7feff0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd00039b0 0x7fefec136f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.422+0000 7feff0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd00039b0 0x7fefec136f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42056/0 (socket says 172.21.15.119:42056) 2024-04-25T10:16:41.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.422+0000 7feff0e00640 1 -- 172.21.15.119:0/437453987 learned_addr learned my addr 172.21.15.119:0/437453987 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.422+0000 7feff0e00640 1 -- 172.21.15.119:0/437453987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefec15ab90 msgr2=0x7fefec0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.422+0000 7feff0e00640 1 --2- 172.21.15.119:0/437453987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefec15ab90 0x7fefec0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.422+0000 7feff0e00640 1 -- 172.21.15.119:0/437453987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefec15b0e0 msgr2=0x7fefec0ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.422+0000 7fefe3e00640 1 --2- 172.21.15.119:0/437453987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefec15b0e0 0x7fefec0ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.422+0000 7feff0e00640 1 --2- 172.21.15.119:0/437453987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefec15b0e0 0x7fefec0ad1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.422+0000 7feff0e00640 1 -- 172.21.15.119:0/437453987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec143730 con 0x7fefd00039b0 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.422+0000 7feff0e00640 1 --2- 172.21.15.119:0/437453987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd00039b0 0x7fefec136f50 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fefdc001870 tx=0x7fefdc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.423+0000 7fefe9600640 1 -- 172.21.15.119:0/437453987 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc037040 con 0x7fefd00039b0 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.423+0000 7fefe9600640 1 -- 172.21.15.119:0/437453987 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefdc040070 con 0x7fefd00039b0 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.423+0000 7fefe9600640 1 -- 172.21.15.119:0/437453987 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc001370 con 0x7fefd00039b0 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.423+0000 7feff3800640 1 -- 172.21.15.119:0/437453987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefec143a10 con 0x7fefd00039b0 2024-04-25T10:16:41.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.423+0000 7feff3800640 1 -- 172.21.15.119:0/437453987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefec143fd0 con 0x7fefd00039b0 2024-04-25T10:16:41.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.424+0000 7feff3800640 1 -- 172.21.15.119:0/437453987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefec0750b0 con 0x7fefd00039b0 2024-04-25T10:16:41.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.425+0000 7fefe9600640 1 -- 172.21.15.119:0/437453987 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fefdc0052d0 con 0x7fefd00039b0 2024-04-25T10:16:41.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.426+0000 7fefe9600640 1 --2- 172.21.15.119:0/437453987 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fefc806b4d0 0x7fefc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.426+0000 7fefe9600640 1 -- 172.21.15.119:0/437453987 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefdc0a9ee0 con 0x7fefd00039b0 2024-04-25T10:16:41.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.426+0000 7fefe3e00640 1 --2- 172.21.15.119:0/437453987 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fefc806b4d0 0x7fefc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.427+0000 7fefe3e00640 1 --2- 172.21.15.119:0/437453987 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fefc806b4d0 0x7fefc806d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fefec144c70 tx=0x7fefd4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.430+0000 7fefe9600640 1 -- 172.21.15.119:0/437453987 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fefdc078c30 con 0x7fefd00039b0 2024-04-25T10:16:41.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.665+0000 7feff3800640 1 -- 172.21.15.119:0/437453987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fefec0b1160 con 0x7fefd00039b0 2024-04-25T10:16:41.669 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:16:41.666+0000 7f8c64600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:16:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.669+0000 7fefe9600640 1 -- 172.21.15.119:0/437453987 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) v1 ==== 161+0+0 (secure 0 0 0) 0x7fefdc078530 con 0x7fefd00039b0 2024-04-25T10:16:41.673 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:16:41.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.672+0000 7feff3800640 1 -- 172.21.15.119:0/437453987 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fefc806b4d0 msgr2=0x7fefc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.672+0000 7feff3800640 1 --2- 172.21.15.119:0/437453987 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fefc806b4d0 0x7fefc806d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fefec144c70 tx=0x7fefd4008040 comp rx=0 tx=0).stop 2024-04-25T10:16:41.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.672+0000 7feff3800640 1 -- 172.21.15.119:0/437453987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd00039b0 msgr2=0x7fefec136f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.672+0000 7feff3800640 1 --2- 172.21.15.119:0/437453987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd00039b0 0x7fefec136f50 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fefdc001870 tx=0x7fefdc001120 comp rx=0 tx=0).stop 2024-04-25T10:16:41.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.673+0000 7feff3800640 1 -- 172.21.15.119:0/437453987 shutdown_connections 2024-04-25T10:16:41.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.673+0000 7feff3800640 1 --2- 172.21.15.119:0/437453987 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fefc806b4d0 0x7fefc806d990 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.673+0000 7feff3800640 1 --2- 172.21.15.119:0/437453987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fefec15ab90 0x7fefec0ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.673+0000 7feff3800640 1 --2- 172.21.15.119:0/437453987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefec15b0e0 0x7fefec0ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.673+0000 7feff3800640 1 --2- 172.21.15.119:0/437453987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefd00039b0 0x7fefec136f50 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.673+0000 7feff3800640 1 -- 172.21.15.119:0/437453987 >> 172.21.15.119:0/437453987 conn(0x7fefec06aba0 msgr2=0x7fefec142090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.673+0000 7feff3800640 1 -- 172.21.15.119:0/437453987 shutdown_connections 2024-04-25T10:16:41.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.673+0000 7feff3800640 1 -- 172.21.15.119:0/437453987 wait complete. 2024-04-25T10:16:41.696 DEBUG:teuthology.orchestra.run.smithi119:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-25T10:16:41.711 INFO:tasks.ceph.mds.a.smithi119.stderr: -7179> 2024-04-25T10:15:53.077+0000 7f1bcf400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:41.780 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:16:41.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.780+0000 7f067b60c280 1 -- start start 2024-04-25T10:16:41.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.780+0000 7f067b60c280 1 Processor -- start 2024-04-25T10:16:41.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.780+0000 7f067b60c280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56260d8bcc00 0x56260d928580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.781+0000 7f067b60c280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56260d8bd800 0x56260d928b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.781+0000 7f067b60c280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260d8bdc00 0x56260d929080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.781+0000 7f067b60c280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260e68a180 con 0x56260d8bdc00 2024-04-25T10:16:41.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.781+0000 7f067b60c280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260e68a300 con 0x56260d8bd800 2024-04-25T10:16:41.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.781+0000 7f067b60c280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56260e68a480 con 0x56260d8bcc00 2024-04-25T10:16:41.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.781+0000 7f0679400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260d8bdc00 0x56260d929080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.781+0000 7f0679400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260d8bdc00 0x56260d929080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42086/0 (socket says 172.21.15.119:42086) 2024-04-25T10:16:41.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.781+0000 7f0679400640 1 -- 172.21.15.119:0/2650624472 learned_addr learned my addr 172.21.15.119:0/2650624472 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:41.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.781+0000 7f0678000640 1 --2- 172.21.15.119:0/2650624472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56260d8bd800 0x56260d928b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.781+0000 7f0679400640 1 -- 172.21.15.119:0/2650624472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56260d8bcc00 msgr2=0x56260d928580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.781+0000 7f0679400640 1 --2- 172.21.15.119:0/2650624472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56260d8bcc00 0x56260d928580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.781+0000 7f0679400640 1 -- 172.21.15.119:0/2650624472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56260d8bd800 msgr2=0x56260d928b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.781+0000 7f0679400640 1 --2- 172.21.15.119:0/2650624472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56260d8bd800 0x56260d928b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.781+0000 7f0679400640 1 -- 172.21.15.119:0/2650624472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56260d8f0780 con 0x56260d8bdc00 2024-04-25T10:16:41.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.782+0000 7f0679400640 1 --2- 172.21.15.119:0/2650624472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260d8bdc00 0x56260d929080 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x56260e6d53b0 tx=0x56260e7843c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5506d1f19168a21f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.782+0000 7f0675800640 1 -- 172.21.15.119:0/2650624472 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56260d8e2680 con 0x56260d8bdc00 2024-04-25T10:16:41.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.782+0000 7f0675800640 1 -- 172.21.15.119:0/2650624472 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56260d8e8700 con 0x56260d8bdc00 2024-04-25T10:16:41.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.782+0000 7f0675800640 1 -- 172.21.15.119:0/2650624472 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56260d8e2b60 con 0x56260d8bdc00 2024-04-25T10:16:41.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.782+0000 7f067b60c280 1 -- 172.21.15.119:0/2650624472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56260d8f14a0 con 0x56260d8bdc00 2024-04-25T10:16:41.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.782+0000 7f067b60c280 1 -- 172.21.15.119:0/2650624472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56260d8f0000 con 0x56260d8bdc00 2024-04-25T10:16:41.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.782+0000 7f067b60c280 4 waiting for MDS map... 2024-04-25T10:16:41.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.782+0000 7f0675800640 1 -- 172.21.15.119:0/2650624472 <== mon.0 v2:172.21.15.119:3300/0 4 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x56260e78e600 con 0x56260d8bdc00 2024-04-25T10:16:41.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.782+0000 7f0675800640 1 -- 172.21.15.119:0/2650624472 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 83) v1 ==== 2723+0+0 (secure 0 0 0) 0x56260d917180 con 0x56260d8bdc00 2024-04-25T10:16:41.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.783+0000 7f067b60c280 4 Got MDS map 83 2024-04-25T10:16:41.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.783+0000 7f067b60c280 10 main: JournalTool::main 2024-04-25T10:16:41.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.783+0000 7f067b60c280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:16:41.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.784+0000 7f067b60c280 1 Processor -- start 2024-04-25T10:16:41.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.784+0000 7f067b60c280 1 -- start start 2024-04-25T10:16:41.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.784+0000 7f067b60c280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260e791000 0x56260d929600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.784+0000 7f067b60c280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56260e791400 0x56260d929b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.784+0000 7f067b60c280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56260e791800 0x56260d92a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.784+0000 7f067b60c280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260e68a600 con 0x56260e791000 2024-04-25T10:16:41.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.784+0000 7f067b60c280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260e68a780 con 0x56260e791800 2024-04-25T10:16:41.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.784+0000 7f067b60c280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56260e68a900 con 0x56260e791400 2024-04-25T10:16:41.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.785+0000 7f0678000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260e791000 0x56260d929600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.785+0000 7f0678000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260e791000 0x56260d929600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42106/0 (socket says 172.21.15.119:42106) 2024-04-25T10:16:41.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.785+0000 7f0678000640 1 -- 172.21.15.119:0/4116262002 learned_addr learned my addr 172.21.15.119:0/4116262002 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:41.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.785+0000 7f0678000640 1 -- 172.21.15.119:0/4116262002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56260e791400 msgr2=0x56260d929b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.785+0000 7f0678000640 1 --2- 172.21.15.119:0/4116262002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56260e791400 0x56260d929b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.785+0000 7f0678000640 1 -- 172.21.15.119:0/4116262002 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56260e791800 msgr2=0x56260d92a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.785+0000 7f0678000640 1 --2- 172.21.15.119:0/4116262002 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56260e791800 0x56260d92a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.785+0000 7f0678000640 1 -- 172.21.15.119:0/4116262002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56260d8f03c0 con 0x56260e791000 2024-04-25T10:16:41.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.785+0000 7f0678000640 1 --2- 172.21.15.119:0/4116262002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260e791000 0x56260d929600 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x56260e7a4600 tx=0x56260e7841e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4568eef1429bb275 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.785+0000 7f0673000640 1 -- 172.21.15.119:0/4116262002 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56260d8e29c0 con 0x56260e791000 2024-04-25T10:16:41.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.786+0000 7f0673000640 1 -- 172.21.15.119:0/4116262002 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56260d8e8fc0 con 0x56260e791000 2024-04-25T10:16:41.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.786+0000 7f0673000640 1 -- 172.21.15.119:0/4116262002 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56260d8e2680 con 0x56260e791000 2024-04-25T10:16:41.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.786+0000 7f067b60c280 1 -- 172.21.15.119:0/4116262002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260e791000 msgr2=0x56260d929600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.786+0000 7f067b60c280 1 --2- 172.21.15.119:0/4116262002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260e791000 0x56260d929600 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x56260e7a4600 tx=0x56260e7841e0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.786+0000 7f067b60c280 1 -- 172.21.15.119:0/4116262002 shutdown_connections 2024-04-25T10:16:41.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.786+0000 7f067b60c280 1 --2- 172.21.15.119:0/4116262002 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56260e791800 0x56260d92a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.786+0000 7f067b60c280 1 --2- 172.21.15.119:0/4116262002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56260e791400 0x56260d929b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.786+0000 7f067b60c280 1 --2- 172.21.15.119:0/4116262002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260e791000 0x56260d929600 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.786+0000 7f067b60c280 1 -- 172.21.15.119:0/4116262002 >> 172.21.15.119:0/4116262002 conn(0x56260e790c00 msgr2=0x56260d9c3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.786+0000 7f067b60c280 1 -- 172.21.15.119:0/4116262002 shutdown_connections 2024-04-25T10:16:41.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.786+0000 7f067b60c280 1 -- 172.21.15.119:0/4116262002 wait complete. 2024-04-25T10:16:41.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.787+0000 7f067b60c280 1 Processor -- start 2024-04-25T10:16:41.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.787+0000 7f067b60c280 1 -- start start 2024-04-25T10:16:41.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f067b60c280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56260e791000 0x56260d92b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f067b60c280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56260e791400 0x56260d92ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f067b60c280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260e791800 0x56260d92a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f067b60c280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260e68bb00 con 0x56260e791800 2024-04-25T10:16:41.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f067b60c280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260e68b980 con 0x56260e791000 2024-04-25T10:16:41.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f067b60c280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56260e68b800 con 0x56260e791400 2024-04-25T10:16:41.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f0678000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260e791800 0x56260d92a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f0678000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260e791800 0x56260d92a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42118/0 (socket says 172.21.15.119:42118) 2024-04-25T10:16:41.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f0678000640 1 -- 172.21.15.119:0/4281151558 learned_addr learned my addr 172.21.15.119:0/4281151558 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:41.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f0678000640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56260e791000 0x56260d92b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f0678a00640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56260e791400 0x56260d92ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f0678000640 1 -- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56260e791400 msgr2=0x56260d92ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f0678000640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56260e791400 0x56260d92ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f0678000640 1 -- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56260e791000 msgr2=0x56260d92b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f0678000640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56260e791000 0x56260d92b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f0678000640 1 -- 172.21.15.119:0/4281151558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56260d8f0780 con 0x56260e791800 2024-04-25T10:16:41.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.788+0000 7f0678a00640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56260e791400 0x56260d92ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:41.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.789+0000 7f0678000640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260e791800 0x56260d92a680 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x56260e7a4930 tx=0x56260e784120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.789+0000 7f0671200640 1 -- 172.21.15.119:0/4281151558 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56260d8e2b60 con 0x56260e791800 2024-04-25T10:16:41.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.789+0000 7f0671200640 1 -- 172.21.15.119:0/4281151558 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56260d8e8fc0 con 0x56260e791800 2024-04-25T10:16:41.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.789+0000 7f0671200640 1 -- 172.21.15.119:0/4281151558 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56260d8e24e0 con 0x56260e791800 2024-04-25T10:16:41.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.789+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56260d8f14a0 con 0x56260e791800 2024-04-25T10:16:41.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.789+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56260d8f0960 con 0x56260e791800 2024-04-25T10:16:41.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.789+0000 7f067b60c280 4 main: JournalTool: resolving pool 14 2024-04-25T10:16:41.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.791+0000 7f0671200640 1 -- 172.21.15.119:0/4281151558 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x56260e6a4c00 con 0x56260e791800 2024-04-25T10:16:41.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.791+0000 7f0671200640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x56260e791c00 0x56260d92b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.791+0000 7f0671200640 1 -- 172.21.15.119:0/4281151558 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x56260e804600 con 0x56260e791800 2024-04-25T10:16:41.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.791+0000 7f067b60c280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:16:41.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.791+0000 7f067b60c280 4 main: Executing for rank 0 2024-04-25T10:16:41.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.791+0000 7f067b60c280 4 Filtering by inode '1099511627777' 2024-04-25T10:16:41.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.791+0000 7f067b60c280 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56260dc30000 0x56260d92a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.792+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) v9 -- 0x56260dc30400 con 0x56260dc30000 2024-04-25T10:16:41.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.792+0000 7f0678a00640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x56260e791c00 0x56260d92b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.792+0000 7f0678000640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56260dc30000 0x56260d92a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.792+0000 7f0678a00640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x56260e791c00 0x56260d92b700 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x56260e6d5290 tx=0x56260e6ad020 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.793+0000 7f0678000640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56260dc30000 0x56260d92a100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.794+0000 7f0678000640 1 -- 172.21.15.119:0/4281151558 <== osd.7 v2:172.21.15.132:6821/2727385616 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x56260e63a000 con 0x56260dc30000 2024-04-25T10:16:41.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.794+0000 7f067b60c280 4 Pointer 400.00000000 is readable 2024-04-25T10:16:41.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.794+0000 7f067b60c280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T10:16:41.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.794+0000 7f067b60c280 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56260dc30800 0x56260d929b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.794+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) v9 -- 0x56260dc30c00 con 0x56260dc30800 2024-04-25T10:16:41.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.795+0000 7f0679400640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56260dc30800 0x56260d929b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.796+0000 7f0679400640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56260dc30800 0x56260d929b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.797+0000 7f0679400640 1 -- 172.21.15.119:0/4281151558 <== osd.4 v2:172.21.15.132:6800/1740646323 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x56260e63a500 con 0x56260dc30800 2024-04-25T10:16:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.797+0000 7f067b60c280 10 Header 0x400000 0x407d9d 0x408119 2024-04-25T10:16:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.797+0000 7f067b60c280 10 Starting journal scan from offset 0x407d9d 2024-04-25T10:16:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.797+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 32157~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) v9 -- 0x56260dc31000 con 0x56260dc30800 2024-04-25T10:16:41.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.798+0000 7f0679400640 1 -- 172.21.15.119:0/4281151558 <== osd.4 v2:172.21.15.132:6800/1740646323 2 ==== osd_op_reply(3 200.00000001 [read 32157~892 out=892b] v0'0 uv28 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x56260e63a500 con 0x56260dc30800 2024-04-25T10:16:41.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.798+0000 7f067b60c280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T10:16:41.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.798+0000 7f067b60c280 10 Parsing data, 0x37c bytes available 2024-04-25T10:16:41.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.798+0000 7f067b60c280 10 Attempting decode at 0x407d9d 2024-04-25T10:16:41.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.798+0000 7f067b60c280 10 Consumed 0x37c bytes 2024-04-25T10:16:41.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.798+0000 7f067b60c280 10 Valid entry at 0x407d9d 2024-04-25T10:16:41.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.798+0000 7f067b60c280 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x56260dc40000 0x56260d929600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.798+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) v9 -- 0x56260dc40400 con 0x56260dc40000 2024-04-25T10:16:41.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.799+0000 7f0678a00640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x56260dc40000 0x56260d929600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.799+0000 7f0678a00640 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x56260dc40000 0x56260d929600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.799+0000 7f0678a00640 1 -- 172.21.15.119:0/4281151558 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56260e63a780 con 0x56260dc40000 2024-04-25T10:16:41.803 INFO:teuthology.orchestra.run.smithi119.stdout:Events by type: 2024-04-25T10:16:41.804 INFO:teuthology.orchestra.run.smithi119.stdout:Errors: 0 2024-04-25T10:16:41.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.799+0000 7f067b60c280 4 Reached end of journal objects 2024-04-25T10:16:41.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.799+0000 7f067b60c280 4 Scanned objects, 0 missing, 1 valid 2024-04-25T10:16:41.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.799+0000 7f067b60c280 4 Events scanned, 0 gaps 2024-04-25T10:16:41.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.799+0000 7f067b60c280 4 Found 1 valid events 2024-04-25T10:16:41.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.799+0000 7f067b60c280 4 Selected 0 events events for processing 2024-04-25T10:16:41.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.799+0000 7f067b60c280 4 0 events 2024-04-25T10:16:41.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.799+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x56260dc40000 msgr2=0x56260d929600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.799+0000 7f067b60c280 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x56260dc40000 0x56260d929600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.799+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56260dc30800 msgr2=0x56260d929b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.799+0000 7f067b60c280 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56260dc30800 0x56260d929b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56260dc30000 msgr2=0x56260d92a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56260dc30000 0x56260d92a100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f0678a00640 1 -- 172.21.15.119:0/4281151558 reap_dead start 2024-04-25T10:16:41.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x56260e791c00 msgr2=0x56260d92b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x56260e791c00 0x56260d92b700 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x56260e6d5290 tx=0x56260e6ad020 comp rx=0 tx=0).stop 2024-04-25T10:16:41.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260e791800 msgr2=0x56260d92a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260e791800 0x56260d92a680 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x56260e7a4930 tx=0x56260e784120 comp rx=0 tx=0).stop 2024-04-25T10:16:41.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 shutdown_connections 2024-04-25T10:16:41.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x56260e791c00 0x56260d92b700 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 --2- 172.21.15.119:0/4281151558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260e791800 0x56260d92a680 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 >> 172.21.15.119:0/4281151558 conn(0x56260e790c00 msgr2=0x56260d9c3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 shutdown_connections 2024-04-25T10:16:41.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 -- 172.21.15.119:0/4281151558 wait complete. 2024-04-25T10:16:41.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 -- 172.21.15.119:0/2650624472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260d8bdc00 msgr2=0x56260d929080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 --2- 172.21.15.119:0/2650624472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260d8bdc00 0x56260d929080 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x56260e6d53b0 tx=0x56260e7843c0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 -- 172.21.15.119:0/2650624472 shutdown_connections 2024-04-25T10:16:41.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 --2- 172.21.15.119:0/2650624472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56260d8bdc00 0x56260d929080 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 --2- 172.21.15.119:0/2650624472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56260d8bd800 0x56260d928b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 --2- 172.21.15.119:0/2650624472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56260d8bcc00 0x56260d928580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.800+0000 7f067b60c280 1 -- 172.21.15.119:0/2650624472 >> 172.21.15.119:0/2650624472 conn(0x56260d8bd000 msgr2=0x56260d9c2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.801+0000 7f067b60c280 1 -- 172.21.15.119:0/2650624472 shutdown_connections 2024-04-25T10:16:41.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.801+0000 7f067b60c280 1 -- 172.21.15.119:0/2650624472 wait complete. 2024-04-25T10:16:41.809 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.112962, result: Events by type: Errors: 0 2024-04-25T10:16:41.809 DEBUG:teuthology.orchestra.run.smithi119:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-25T10:16:41.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.943+0000 7f0857dee280 1 -- start start 2024-04-25T10:16:41.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.943+0000 7f0857dee280 1 Processor -- start 2024-04-25T10:16:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.944+0000 7f0857dee280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561809b96c00 0x561809c02580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.944+0000 7f0857dee280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809b97800 0x561809c02b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.944+0000 7f0857dee280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561809b97c00 0x561809c03080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.944+0000 7f0857dee280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180a966600 con 0x561809b97800 2024-04-25T10:16:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.944+0000 7f0857dee280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180a966480 con 0x561809b97c00 2024-04-25T10:16:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.944+0000 7f0857dee280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56180a966300 con 0x561809b96c00 2024-04-25T10:16:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0855400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809b97800 0x561809c02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0855400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809b97800 0x561809c02b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42124/0 (socket says 172.21.15.119:42124) 2024-04-25T10:16:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0855400640 1 -- 172.21.15.119:0/2926463001 learned_addr learned my addr 172.21.15.119:0/2926463001 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0855e00640 1 --2- 172.21.15.119:0/2926463001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561809b96c00 0x561809c02580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0856800640 1 --2- 172.21.15.119:0/2926463001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561809b97c00 0x561809c03080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0855400640 1 -- 172.21.15.119:0/2926463001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561809b96c00 msgr2=0x561809c02580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0855400640 1 --2- 172.21.15.119:0/2926463001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561809b96c00 0x561809c02580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0855400640 1 -- 172.21.15.119:0/2926463001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561809b97c00 msgr2=0x561809c03080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0855400640 1 --2- 172.21.15.119:0/2926463001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561809b97c00 0x561809c03080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0855400640 1 -- 172.21.15.119:0/2926463001 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561809bca780 con 0x561809b97800 2024-04-25T10:16:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0855e00640 1 --2- 172.21.15.119:0/2926463001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561809b96c00 0x561809c02580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0855400640 1 --2- 172.21.15.119:0/2926463001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809b97800 0x561809c02b00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x56180a9a9380 tx=0x56180a927740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fced8eccacc203d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0852c00640 1 -- 172.21.15.119:0/2926463001 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561809bbc9c0 con 0x561809b97800 2024-04-25T10:16:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0852c00640 1 -- 172.21.15.119:0/2926463001 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561809bc2e00 con 0x561809b97800 2024-04-25T10:16:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.945+0000 7f0852c00640 1 -- 172.21.15.119:0/2926463001 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561809bbc680 con 0x561809b97800 2024-04-25T10:16:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.946+0000 7f0857dee280 1 -- 172.21.15.119:0/2926463001 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561809bca3c0 con 0x561809b97800 2024-04-25T10:16:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.946+0000 7f0857dee280 1 -- 172.21.15.119:0/2926463001 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x561809bcb680 con 0x561809b97800 2024-04-25T10:16:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.946+0000 7f0857dee280 4 waiting for MDS map... 2024-04-25T10:16:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.946+0000 7f0852c00640 1 -- 172.21.15.119:0/2926463001 <== mon.0 v2:172.21.15.119:3300/0 4 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x56180aa66600 con 0x561809b97800 2024-04-25T10:16:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.946+0000 7f0852c00640 1 -- 172.21.15.119:0/2926463001 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 83) v1 ==== 2723+0+0 (secure 0 0 0) 0x561809bf1180 con 0x561809b97800 2024-04-25T10:16:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.946+0000 7f0857dee280 4 Got MDS map 83 2024-04-25T10:16:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.946+0000 7f0857dee280 10 main: JournalTool::main 2024-04-25T10:16:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.946+0000 7f0857dee280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:16:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.947+0000 7f0857dee280 1 Processor -- start 2024-04-25T10:16:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.947+0000 7f0857dee280 1 -- start start 2024-04-25T10:16:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.948+0000 7f0857dee280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56180a969c00 0x561809c03600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.948+0000 7f0857dee280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56180aabe000 0x561809c03b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.948+0000 7f0857dee280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56180aabe400 0x561809c04100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.948+0000 7f0857dee280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180a966180 con 0x56180aabe000 2024-04-25T10:16:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.948+0000 7f0857dee280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561809bf8480 con 0x56180aabe400 2024-04-25T10:16:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.948+0000 7f0857dee280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561809bf8900 con 0x56180a969c00 2024-04-25T10:16:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.948+0000 7f0855e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56180aabe000 0x561809c03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.948+0000 7f0855e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56180aabe000 0x561809c03b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42134/0 (socket says 172.21.15.119:42134) 2024-04-25T10:16:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.948+0000 7f0855e00640 1 -- 172.21.15.119:0/1016214158 learned_addr learned my addr 172.21.15.119:0/1016214158 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.948+0000 7f0855e00640 1 -- 172.21.15.119:0/1016214158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56180a969c00 msgr2=0x561809c03600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.948+0000 7f0855e00640 1 --2- 172.21.15.119:0/1016214158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56180a969c00 0x561809c03600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.948+0000 7f0855e00640 1 -- 172.21.15.119:0/1016214158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56180aabe400 msgr2=0x561809c04100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.948+0000 7f0855e00640 1 --2- 172.21.15.119:0/1016214158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56180aabe400 0x561809c04100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.948+0000 7f0855e00640 1 -- 172.21.15.119:0/1016214158 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561809bcb4a0 con 0x56180aabe000 2024-04-25T10:16:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.949+0000 7f0855e00640 1 --2- 172.21.15.119:0/1016214158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56180aabe000 0x561809c03b80 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x56180aa6eae0 tx=0x56180aace0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d836446d44b73e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.949+0000 7f0850400640 1 -- 172.21.15.119:0/1016214158 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561809bbc9c0 con 0x56180aabe000 2024-04-25T10:16:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.949+0000 7f0850400640 1 -- 172.21.15.119:0/1016214158 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561809bc21c0 con 0x56180aabe000 2024-04-25T10:16:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.949+0000 7f0850400640 1 -- 172.21.15.119:0/1016214158 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561809bbc680 con 0x56180aabe000 2024-04-25T10:16:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.949+0000 7f0857dee280 1 -- 172.21.15.119:0/1016214158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56180aabe000 msgr2=0x561809c03b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.949+0000 7f0857dee280 1 --2- 172.21.15.119:0/1016214158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56180aabe000 0x561809c03b80 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x56180aa6eae0 tx=0x56180aace0c0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.949+0000 7f0857dee280 1 -- 172.21.15.119:0/1016214158 shutdown_connections 2024-04-25T10:16:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.949+0000 7f0857dee280 1 --2- 172.21.15.119:0/1016214158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56180aabe400 0x561809c04100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.949+0000 7f0857dee280 1 --2- 172.21.15.119:0/1016214158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56180aabe000 0x561809c03b80 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.949+0000 7f0857dee280 1 --2- 172.21.15.119:0/1016214158 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56180a969c00 0x561809c03600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.949+0000 7f0857dee280 1 -- 172.21.15.119:0/1016214158 >> 172.21.15.119:0/1016214158 conn(0x56180a969800 msgr2=0x561809c9a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.949+0000 7f0857dee280 1 -- 172.21.15.119:0/1016214158 shutdown_connections 2024-04-25T10:16:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.949+0000 7f0857dee280 1 -- 172.21.15.119:0/1016214158 wait complete. 2024-04-25T10:16:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.950+0000 7f0857dee280 1 Processor -- start 2024-04-25T10:16:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.950+0000 7f0857dee280 1 -- start start 2024-04-25T10:16:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.951+0000 7f0857dee280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56180a969800 0x561809c05180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.951+0000 7f0857dee280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56180aabf000 0x561809c04c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.951+0000 7f0857dee280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56180aabec00 0x561809c04680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.951+0000 7f0857dee280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180a967380 con 0x56180aabec00 2024-04-25T10:16:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.951+0000 7f0857dee280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56180a966180 con 0x56180a969800 2024-04-25T10:16:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.951+0000 7f0857dee280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x56180a967500 con 0x56180aabf000 2024-04-25T10:16:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.951+0000 7f0855400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56180aabec00 0x561809c04680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.951+0000 7f0855400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56180aabec00 0x561809c04680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42150/0 (socket says 172.21.15.119:42150) 2024-04-25T10:16:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.951+0000 7f0855400640 1 -- 172.21.15.119:0/4263844137 learned_addr learned my addr 172.21.15.119:0/4263844137 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.951+0000 7f0855400640 1 -- 172.21.15.119:0/4263844137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56180aabf000 msgr2=0x561809c04c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.951+0000 7f0855400640 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x56180aabf000 0x561809c04c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.952+0000 7f0855400640 1 -- 172.21.15.119:0/4263844137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56180a969800 msgr2=0x561809c05180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.952+0000 7f0855400640 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56180a969800 0x561809c05180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.952+0000 7f0855400640 1 -- 172.21.15.119:0/4263844137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561809bca000 con 0x56180aabec00 2024-04-25T10:16:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.952+0000 7f0855400640 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56180aabec00 0x561809c04680 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x56180a9a9980 tx=0x56180a927860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.952+0000 7f084e600640 1 -- 172.21.15.119:0/4263844137 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561809bbcb60 con 0x56180aabec00 2024-04-25T10:16:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.952+0000 7f084e600640 1 -- 172.21.15.119:0/4263844137 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561809bc2c40 con 0x56180aabec00 2024-04-25T10:16:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.952+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561809bca960 con 0x56180aabec00 2024-04-25T10:16:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.952+0000 7f084e600640 1 -- 172.21.15.119:0/4263844137 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561809bbc820 con 0x56180aabec00 2024-04-25T10:16:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.952+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561809bcb860 con 0x56180aabec00 2024-04-25T10:16:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.952+0000 7f0857dee280 4 main: JournalTool: resolving pool 14 2024-04-25T10:16:41.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.954+0000 7f084e600640 1 -- 172.21.15.119:0/4263844137 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x56180aad4c00 con 0x56180aabec00 2024-04-25T10:16:41.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.954+0000 7f084e600640 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x56180aadb000 0x561809c05700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.954+0000 7f084e600640 1 -- 172.21.15.119:0/4263844137 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x56180aac5200 con 0x56180aabec00 2024-04-25T10:16:41.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.955+0000 7f0857dee280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:16:41.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.955+0000 7f0857dee280 4 main: Executing for rank 0 2024-04-25T10:16:41.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.955+0000 7f0857dee280 4 Filtering by inode '1099511627778' 2024-04-25T10:16:41.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.955+0000 7f0857dee280 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56180aabe800 0x561809c04100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.955+0000 7f0856800640 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x56180aadb000 0x561809c05700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.955+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) v9 -- 0x56180aabe400 con 0x56180aabe800 2024-04-25T10:16:41.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.955+0000 7f0856800640 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x56180aadb000 0x561809c05700 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x561809c8c7e0 tx=0x56180aacf140 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.955+0000 7f0855e00640 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56180aabe800 0x561809c04100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.956+0000 7f0855e00640 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56180aabe800 0x561809c04100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.957+0000 7f0855e00640 1 -- 172.21.15.119:0/4263844137 <== osd.7 v2:172.21.15.132:6821/2727385616 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x56180a914000 con 0x56180aabe800 2024-04-25T10:16:41.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.957+0000 7f0857dee280 4 Pointer 400.00000000 is readable 2024-04-25T10:16:41.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.957+0000 7f0857dee280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T10:16:41.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.957+0000 7f0857dee280 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56180aabe000 0x561809c03b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.957+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) v9 -- 0x56180aabfc00 con 0x56180aabe000 2024-04-25T10:16:41.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.958+0000 7f0856800640 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56180aabe000 0x561809c03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.959+0000 7f0856800640 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56180aabe000 0x561809c03b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.960+0000 7f0856800640 1 -- 172.21.15.119:0/4263844137 <== osd.4 v2:172.21.15.132:6800/1740646323 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x56180a914500 con 0x56180aabe000 2024-04-25T10:16:41.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.960+0000 7f0857dee280 10 Header 0x400000 0x407d9d 0x408119 2024-04-25T10:16:41.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.960+0000 7f0857dee280 10 Starting journal scan from offset 0x407d9d 2024-04-25T10:16:41.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.960+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 32157~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) v9 -- 0x56180aabf400 con 0x56180aabe000 2024-04-25T10:16:41.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.961+0000 7f0856800640 1 -- 172.21.15.119:0/4263844137 <== osd.4 v2:172.21.15.132:6800/1740646323 2 ==== osd_op_reply(3 200.00000001 [read 32157~892 out=892b] v0'0 uv28 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x56180a914500 con 0x56180aabe000 2024-04-25T10:16:41.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.961+0000 7f0857dee280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T10:16:41.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.961+0000 7f0857dee280 10 Parsing data, 0x37c bytes available 2024-04-25T10:16:41.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.961+0000 7f0857dee280 10 Attempting decode at 0x407d9d 2024-04-25T10:16:41.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.961+0000 7f0857dee280 10 Consumed 0x37c bytes 2024-04-25T10:16:41.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.961+0000 7f0857dee280 10 Valid entry at 0x407d9d 2024-04-25T10:16:41.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.961+0000 7f0857dee280 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x56180aabf800 0x561809c03600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.961+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) v9 -- 0x56180aabe400 con 0x56180aabf800 2024-04-25T10:16:41.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.961+0000 7f0855e00640 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x56180aabf800 0x561809c03600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0855e00640 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x56180aabf800 0x561809c03600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0855e00640 1 -- 172.21.15.119:0/4263844137 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56180a914000 con 0x56180aabf800 2024-04-25T10:16:41.966 INFO:teuthology.orchestra.run.smithi119.stdout:Events by type: 2024-04-25T10:16:41.966 INFO:teuthology.orchestra.run.smithi119.stdout:Errors: 0 2024-04-25T10:16:41.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 4 Reached end of journal objects 2024-04-25T10:16:41.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 4 Scanned objects, 0 missing, 1 valid 2024-04-25T10:16:41.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 4 Events scanned, 0 gaps 2024-04-25T10:16:41.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 4 Found 1 valid events 2024-04-25T10:16:41.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 4 Selected 0 events events for processing 2024-04-25T10:16:41.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 4 0 events 2024-04-25T10:16:41.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x56180aabf800 msgr2=0x561809c03600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x56180aabf800 0x561809c03600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56180aabe000 msgr2=0x561809c03b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x56180aabe000 0x561809c03b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56180aabe800 msgr2=0x561809c04100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x56180aabe800 0x561809c04100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0855e00640 1 -- 172.21.15.119:0/4263844137 reap_dead start 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x56180aadb000 msgr2=0x561809c05700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x56180aadb000 0x561809c05700 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x561809c8c7e0 tx=0x56180aacf140 comp rx=0 tx=0).stop 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56180aabec00 msgr2=0x561809c04680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56180aabec00 0x561809c04680 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x56180a9a9980 tx=0x56180a927860 comp rx=0 tx=0).stop 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 shutdown_connections 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x56180aadb000 0x561809c05700 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 --2- 172.21.15.119:0/4263844137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56180aabec00 0x561809c04680 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 >> 172.21.15.119:0/4263844137 conn(0x56180a969c00 msgr2=0x561809c9a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.962+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 shutdown_connections 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.963+0000 7f0857dee280 1 -- 172.21.15.119:0/4263844137 wait complete. 2024-04-25T10:16:41.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.963+0000 7f0857dee280 1 -- 172.21.15.119:0/2926463001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809b97800 msgr2=0x561809c02b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.963+0000 7f0857dee280 1 --2- 172.21.15.119:0/2926463001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809b97800 0x561809c02b00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x56180a9a9380 tx=0x56180a927740 comp rx=0 tx=0).stop 2024-04-25T10:16:41.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.963+0000 7f0857dee280 1 -- 172.21.15.119:0/2926463001 shutdown_connections 2024-04-25T10:16:41.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.963+0000 7f0857dee280 1 --2- 172.21.15.119:0/2926463001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561809b97c00 0x561809c03080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.963+0000 7f0857dee280 1 --2- 172.21.15.119:0/2926463001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561809b97800 0x561809c02b00 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.963+0000 7f0857dee280 1 --2- 172.21.15.119:0/2926463001 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561809b96c00 0x561809c02580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.963+0000 7f0857dee280 1 -- 172.21.15.119:0/2926463001 >> 172.21.15.119:0/2926463001 conn(0x561809b97000 msgr2=0x561809c9b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.963+0000 7f0857dee280 1 -- 172.21.15.119:0/2926463001 shutdown_connections 2024-04-25T10:16:41.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:41.963+0000 7f0857dee280 1 -- 172.21.15.119:0/2926463001 wait complete. 2024-04-25T10:16:41.972 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.162480, result: Events by type: Errors: 0 2024-04-25T10:16:41.972 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-25T10:16:42.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.049+0000 7fafa7354f40 1 Processor -- start 2024-04-25T10:16:42.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.049+0000 7fafa7354f40 1 -- start start 2024-04-25T10:16:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.050+0000 7fafa7354f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55feeb09b220 0x55feeb09a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.050+0000 7fafa7354f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55feeb0e6080 0x55feeb1663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.050+0000 7fafa7354f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55feeb172c70 0x55feeb0e5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.050+0000 7fafa7354f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feeb0ce2f0 con 0x55feeb172c70 2024-04-25T10:16:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.050+0000 7fafa7354f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feeb042dd0 con 0x55feeb0e6080 2024-04-25T10:16:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.050+0000 7fafa7354f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55feeb09b960 con 0x55feeb09b220 2024-04-25T10:16:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.050+0000 7fafa5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55feeb172c70 0x55feeb0e5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.050+0000 7fafa5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55feeb172c70 0x55feeb0e5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42166/0 (socket says 172.21.15.119:42166) 2024-04-25T10:16:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.050+0000 7fafa5000640 1 -- 172.21.15.119:0/47131606 learned_addr learned my addr 172.21.15.119:0/47131606 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.050+0000 7fafa5000640 1 -- 172.21.15.119:0/47131606 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55feeb09b220 msgr2=0x55feeb09a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.050+0000 7fafa5000640 1 --2- 172.21.15.119:0/47131606 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55feeb09b220 0x55feeb09a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.050+0000 7fafa5000640 1 -- 172.21.15.119:0/47131606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55feeb0e6080 msgr2=0x55feeb1663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.050+0000 7fafa5000640 1 --2- 172.21.15.119:0/47131606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55feeb0e6080 0x55feeb1663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.050+0000 7fafa5000640 1 -- 172.21.15.119:0/47131606 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55feeb09a200 con 0x55feeb172c70 2024-04-25T10:16:42.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.051+0000 7fafa5000640 1 --2- 172.21.15.119:0/47131606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55feeb172c70 0x55feeb0e5340 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00b240 tx=0x7faf9c00b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2e4c60ca790dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.051+0000 7fafa2800640 1 -- 172.21.15.119:0/47131606 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c008070 con 0x55feeb172c70 2024-04-25T10:16:42.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.051+0000 7fafa2800640 1 -- 172.21.15.119:0/47131606 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c00d500 con 0x55feeb172c70 2024-04-25T10:16:42.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.051+0000 7fafa2800640 1 -- 172.21.15.119:0/47131606 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c00d930 con 0x55feeb172c70 2024-04-25T10:16:42.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.051+0000 7fafa7354f40 1 -- 172.21.15.119:0/47131606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55feeb172c70 msgr2=0x55feeb0e5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.051+0000 7fafa7354f40 1 --2- 172.21.15.119:0/47131606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55feeb172c70 0x55feeb0e5340 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00b240 tx=0x7faf9c00b700 comp rx=0 tx=0).stop 2024-04-25T10:16:42.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.051+0000 7fafa7354f40 1 -- 172.21.15.119:0/47131606 shutdown_connections 2024-04-25T10:16:42.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.051+0000 7fafa7354f40 1 --2- 172.21.15.119:0/47131606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55feeb172c70 0x55feeb0e5340 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.051+0000 7fafa7354f40 1 --2- 172.21.15.119:0/47131606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55feeb0e6080 0x55feeb1663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.051+0000 7fafa7354f40 1 --2- 172.21.15.119:0/47131606 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55feeb09b220 0x55feeb09a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.051+0000 7fafa7354f40 1 -- 172.21.15.119:0/47131606 >> 172.21.15.119:0/47131606 conn(0x55feeb0e9070 msgr2=0x55feeb0e9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:42.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.052+0000 7fafa7354f40 1 -- 172.21.15.119:0/47131606 shutdown_connections 2024-04-25T10:16:42.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.052+0000 7fafa7354f40 1 -- 172.21.15.119:0/47131606 wait complete. 2024-04-25T10:16:42.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.052+0000 7fafa7354f40 1 Processor -- start 2024-04-25T10:16:42.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.052+0000 7fafa7354f40 1 -- start start 2024-04-25T10:16:42.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.053+0000 7fafa7354f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55feeb09b220 0x55feeb170bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.053+0000 7fafa7354f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55feeb0e6080 0x55feeb1630e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.053+0000 7fafa7354f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55feeb172c70 0x55feeb0e5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.053+0000 7fafa7354f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feeb166ec0 con 0x55feeb0e6080 2024-04-25T10:16:42.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.053+0000 7fafa7354f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feeb0c8510 con 0x55feeb172c70 2024-04-25T10:16:42.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.053+0000 7fafa7354f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55feeb0c87f0 con 0x55feeb09b220 2024-04-25T10:16:42.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.053+0000 7fafa3c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55feeb0e6080 0x55feeb1630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.054+0000 7fafa3c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55feeb0e6080 0x55feeb1630e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42176/0 (socket says 172.21.15.119:42176) 2024-04-25T10:16:42.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.054+0000 7fafa3c00640 1 -- 172.21.15.119:0/1677498562 learned_addr learned my addr 172.21.15.119:0/1677498562 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:42.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.054+0000 7fafa3c00640 1 -- 172.21.15.119:0/1677498562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55feeb09b220 msgr2=0x55feeb170bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.054+0000 7fafa3c00640 1 --2- 172.21.15.119:0/1677498562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55feeb09b220 0x55feeb170bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.054+0000 7fafa3c00640 1 -- 172.21.15.119:0/1677498562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55feeb172c70 msgr2=0x55feeb0e5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.054+0000 7fafa3c00640 1 --2- 172.21.15.119:0/1677498562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55feeb172c70 0x55feeb0e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.054+0000 7fafa3c00640 1 -- 172.21.15.119:0/1677498562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55feeb163620 con 0x55feeb0e6080 2024-04-25T10:16:42.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.054+0000 7fafa3c00640 1 --2- 172.21.15.119:0/1677498562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55feeb0e6080 0x55feeb1630e0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7faf8c055040 tx=0x7faf8c053520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.054+0000 7fafa1400640 1 -- 172.21.15.119:0/1677498562 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c0539d0 con 0x55feeb0e6080 2024-04-25T10:16:42.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.054+0000 7fafa1400640 1 -- 172.21.15.119:0/1677498562 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c053b70 con 0x55feeb0e6080 2024-04-25T10:16:42.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.054+0000 7fafa1400640 1 -- 172.21.15.119:0/1677498562 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c0673f0 con 0x55feeb0e6080 2024-04-25T10:16:42.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.054+0000 7fafa7354f40 1 -- 172.21.15.119:0/1677498562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55feeb0ca1f0 con 0x55feeb0e6080 2024-04-25T10:16:42.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.055+0000 7fafa7354f40 1 -- 172.21.15.119:0/1677498562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55feeb20d400 con 0x55feeb0e6080 2024-04-25T10:16:42.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.056+0000 7fafa1400640 1 -- 172.21.15.119:0/1677498562 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf8c070430 con 0x55feeb0e6080 2024-04-25T10:16:42.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.057+0000 7fafa1400640 1 --2- 172.21.15.119:0/1677498562 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf80069eb0 0x7faf8006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.057+0000 7fafa1400640 1 -- 172.21.15.119:0/1677498562 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7faf8c050030 con 0x55feeb0e6080 2024-04-25T10:16:42.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.058+0000 7fafa7354f40 1 --2- 172.21.15.119:0/1677498562 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55feeb2090a0 0x55feeb2094a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.058+0000 7fafa4600640 1 --2- 172.21.15.119:0/1677498562 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf80069eb0 0x7faf8006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.058+0000 7fafa7354f40 1 -- 172.21.15.119:0/1677498562 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) v9 -- 0x55feeb2099e0 con 0x55feeb2090a0 2024-04-25T10:16:42.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.058+0000 7fafa5000640 1 --2- 172.21.15.119:0/1677498562 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55feeb2090a0 0x55feeb2094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.058+0000 7fafa4600640 1 --2- 172.21.15.119:0/1677498562 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf80069eb0 0x7faf8006c370 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7faf9400a470 tx=0x7faf940056e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.058+0000 7fafa5000640 1 --2- 172.21.15.119:0/1677498562 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55feeb2090a0 0x55feeb2094a0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.061+0000 7fafa5000640 1 -- 172.21.15.119:0/1677498562 <== osd.2 v2:172.21.15.119:6824/1001762963 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v97'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7faf9c00d6b0 con 0x55feeb2090a0 2024-04-25T10:16:42.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.061+0000 7fafa7354f40 1 -- 172.21.15.119:0/1677498562 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55feeb2090a0 msgr2=0x55feeb2094a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.061+0000 7fafa7354f40 1 --2- 172.21.15.119:0/1677498562 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55feeb2090a0 0x55feeb2094a0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.062+0000 7fafa7354f40 1 -- 172.21.15.119:0/1677498562 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf80069eb0 msgr2=0x7faf8006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.062+0000 7fafa7354f40 1 --2- 172.21.15.119:0/1677498562 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faf80069eb0 0x7faf8006c370 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7faf9400a470 tx=0x7faf940056e0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.062+0000 7fafa7354f40 1 -- 172.21.15.119:0/1677498562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55feeb0e6080 msgr2=0x55feeb1630e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.062+0000 7fafa7354f40 1 --2- 172.21.15.119:0/1677498562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55feeb0e6080 0x55feeb1630e0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7faf8c055040 tx=0x7faf8c053520 comp rx=0 tx=0).stop 2024-04-25T10:16:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.062+0000 7fafa5000640 1 -- 172.21.15.119:0/1677498562 reap_dead start 2024-04-25T10:16:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.062+0000 7fafa7354f40 1 -- 172.21.15.119:0/1677498562 shutdown_connections 2024-04-25T10:16:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.062+0000 7fafa7354f40 1 -- 172.21.15.119:0/1677498562 >> 172.21.15.119:0/1677498562 conn(0x55feeb0e9070 msgr2=0x55feeb0e9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.062+0000 7fafa7354f40 1 -- 172.21.15.119:0/1677498562 shutdown_connections 2024-04-25T10:16:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.062+0000 7fafa7354f40 1 -- 172.21.15.119:0/1677498562 wait complete. 2024-04-25T10:16:42.070 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:16:42.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.309+0000 7f895be00640 1 Processor -- start 2024-04-25T10:16:42.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.309+0000 7f895be00640 1 -- start start 2024-04-25T10:16:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.311+0000 7f895be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f895c06b2b0 0x7f895c0cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.311+0000 7f895be00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895c15f400 con 0x7f895c0cc810 2024-04-25T10:16:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.311+0000 7f895be00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895c15f6e0 con 0x7f895c1450e0 2024-04-25T10:16:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.311+0000 7f895be00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895c1454c0 con 0x7f895c06b2b0 2024-04-25T10:16:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.311+0000 7f895a000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f895c06b2b0 0x7f895c0cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.311+0000 7f895a000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f895c06b2b0 0x7f895c0cc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42180/0 (socket says 172.21.15.119:42180) 2024-04-25T10:16:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.311+0000 7f895a000640 1 -- 172.21.15.119:0/2966438230 learned_addr learned my addr 172.21.15.119:0/2966438230 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.311+0000 7f895b400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f895c0cc810 0x7f895c0b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56528/0 (socket says 172.21.15.119:56528) 2024-04-25T10:16:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.312+0000 7f895a000640 1 -- 172.21.15.119:0/2966438230 >> v1:172.21.15.132:6789/0 conn(0x7f895c1450e0 legacy=0x7f895c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.312+0000 7f895a000640 1 -- 172.21.15.119:0/2966438230 >> v1:172.21.15.119:6789/0 conn(0x7f895c0cc810 legacy=0x7f895c0b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.312+0000 7f895a000640 1 -- 172.21.15.119:0/2966438230 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c15f9c0 con 0x7f895c06b2b0 2024-04-25T10:16:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.313+0000 7f895a000640 1 --2- 172.21.15.119:0/2966438230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f895c06b2b0 0x7f895c0cc2d0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f8948009870 tx=0x7f894802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724bafacc4c80c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.313+0000 7f8959600640 1 -- 172.21.15.119:0/2966438230 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948037070 con 0x7f895c06b2b0 2024-04-25T10:16:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.313+0000 7f8959600640 1 -- 172.21.15.119:0/2966438230 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8948032440 con 0x7f895c06b2b0 2024-04-25T10:16:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.313+0000 7f8959600640 1 -- 172.21.15.119:0/2966438230 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948032810 con 0x7f895c06b2b0 2024-04-25T10:16:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.314+0000 7f895be00640 1 -- 172.21.15.119:0/2966438230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f895c06b2b0 msgr2=0x7f895c0cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.314+0000 7f895be00640 1 --2- 172.21.15.119:0/2966438230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f895c06b2b0 0x7f895c0cc2d0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f8948009870 tx=0x7f894802a180 comp rx=0 tx=0).stop 2024-04-25T10:16:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.314+0000 7f895be00640 1 -- 172.21.15.119:0/2966438230 shutdown_connections 2024-04-25T10:16:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.314+0000 7f895be00640 1 --2- 172.21.15.119:0/2966438230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f895c06b2b0 0x7f895c0cc2d0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.314+0000 7f895be00640 1 -- 172.21.15.119:0/2966438230 >> 172.21.15.119:0/2966438230 conn(0x7f895c066ff0 msgr2=0x7f895c0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.314+0000 7f895be00640 1 -- 172.21.15.119:0/2966438230 shutdown_connections 2024-04-25T10:16:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.314+0000 7f895be00640 1 -- 172.21.15.119:0/2966438230 wait complete. 2024-04-25T10:16:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.316+0000 7f895be00640 1 Processor -- start 2024-04-25T10:16:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.316+0000 7f895be00640 1 -- start start 2024-04-25T10:16:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.318+0000 7f895be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f895c06b2b0 0x7f895c15e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.318+0000 7f895be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f895c0cc810 0x7f895c1e7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.318+0000 7f895be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895c1450e0 0x7f895c1e7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.318+0000 7f895be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0ccde0 con 0x7f895c0cc810 2024-04-25T10:16:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.318+0000 7f895be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0ccc60 con 0x7f895c06b2b0 2024-04-25T10:16:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.318+0000 7f895be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0b3590 con 0x7f895c1450e0 2024-04-25T10:16:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.318+0000 7f895a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f895c0cc810 0x7f895c1e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.318+0000 7f895a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f895c0cc810 0x7f895c1e7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42196/0 (socket says 172.21.15.119:42196) 2024-04-25T10:16:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.318+0000 7f895a000640 1 -- 172.21.15.119:0/4283716340 learned_addr learned my addr 172.21.15.119:0/4283716340 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.318+0000 7f895a000640 1 -- 172.21.15.119:0/4283716340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895c1450e0 msgr2=0x7f895c1e7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.318+0000 7f895a000640 1 --2- 172.21.15.119:0/4283716340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895c1450e0 0x7f895c1e7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.318+0000 7f895a000640 1 -- 172.21.15.119:0/4283716340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f895c06b2b0 msgr2=0x7f895c15e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.318+0000 7f895a000640 1 --2- 172.21.15.119:0/4283716340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f895c06b2b0 0x7f895c15e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.318+0000 7f895a000640 1 -- 172.21.15.119:0/4283716340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c15cb30 con 0x7f895c0cc810 2024-04-25T10:16:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.319+0000 7f895a000640 1 --2- 172.21.15.119:0/4283716340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f895c0cc810 0x7f895c1e7230 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f8948009870 tx=0x7f89480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.319+0000 7f8952a00640 1 -- 172.21.15.119:0/4283716340 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948037040 con 0x7f895c0cc810 2024-04-25T10:16:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.319+0000 7f8952a00640 1 -- 172.21.15.119:0/4283716340 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8948042070 con 0x7f895c0cc810 2024-04-25T10:16:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.319+0000 7f8952a00640 1 -- 172.21.15.119:0/4283716340 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948030bd0 con 0x7f895c0cc810 2024-04-25T10:16:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.319+0000 7f895be00640 1 -- 172.21.15.119:0/4283716340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895c15ce10 con 0x7f895c0cc810 2024-04-25T10:16:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.319+0000 7f895be00640 1 -- 172.21.15.119:0/4283716340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c15d3d0 con 0x7f895c0cc810 2024-04-25T10:16:42.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.321+0000 7f895be00640 1 -- 172.21.15.119:0/4283716340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895c15d700 con 0x7f895c0cc810 2024-04-25T10:16:42.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.321+0000 7f8952a00640 1 -- 172.21.15.119:0/4283716340 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8948047460 con 0x7f895c0cc810 2024-04-25T10:16:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.322+0000 7f8952a00640 1 --2- 172.21.15.119:0/4283716340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f893c06b4d0 0x7f893c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.322+0000 7f8952a00640 1 -- 172.21.15.119:0/4283716340 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f89480aae90 con 0x7f895c0cc810 2024-04-25T10:16:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.322+0000 7f895aa00640 1 --2- 172.21.15.119:0/4283716340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f893c06b4d0 0x7f893c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.323+0000 7f895aa00640 1 --2- 172.21.15.119:0/4283716340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f893c06b4d0 0x7f893c06d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f894c006fd0 tx=0x7f894c006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.327+0000 7f8952a00640 1 -- 172.21.15.119:0/4283716340 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8948045360 con 0x7f895c0cc810 2024-04-25T10:16:42.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.564+0000 7f895be00640 1 -- 172.21.15.119:0/4283716340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f895c0aed00 con 0x7f895c0cc810 2024-04-25T10:16:42.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.724+0000 7f8952a00640 1 -- 172.21.15.119:0/4283716340 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v84) v1 ==== 162+0+0 (secure 0 0 0) 0x7f89480ac0b0 con 0x7f895c0cc810 2024-04-25T10:16:42.729 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:16:42.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.728+0000 7f895be00640 1 -- 172.21.15.119:0/4283716340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f893c06b4d0 msgr2=0x7f893c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.728+0000 7f895be00640 1 --2- 172.21.15.119:0/4283716340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f893c06b4d0 0x7f893c06d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f894c006fd0 tx=0x7f894c006bc0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.728+0000 7f895be00640 1 -- 172.21.15.119:0/4283716340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f895c0cc810 msgr2=0x7f895c1e7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.728+0000 7f895be00640 1 --2- 172.21.15.119:0/4283716340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f895c0cc810 0x7f895c1e7230 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f8948009870 tx=0x7f89480046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.728+0000 7f895be00640 1 -- 172.21.15.119:0/4283716340 shutdown_connections 2024-04-25T10:16:42.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.728+0000 7f895be00640 1 --2- 172.21.15.119:0/4283716340 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f893c06b4d0 0x7f893c06d990 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.728+0000 7f895be00640 1 --2- 172.21.15.119:0/4283716340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f895c1450e0 0x7f895c1e7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.728+0000 7f895be00640 1 --2- 172.21.15.119:0/4283716340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f895c0cc810 0x7f895c1e7230 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.728+0000 7f895be00640 1 --2- 172.21.15.119:0/4283716340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f895c06b2b0 0x7f895c15e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.728+0000 7f895be00640 1 -- 172.21.15.119:0/4283716340 >> 172.21.15.119:0/4283716340 conn(0x7f895c066ff0 msgr2=0x7f895c0b1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.728+0000 7f895be00640 1 -- 172.21.15.119:0/4283716340 shutdown_connections 2024-04-25T10:16:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.728+0000 7f895be00640 1 -- 172.21.15.119:0/4283716340 wait complete. 2024-04-25T10:16:42.754 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:42.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.916+0000 7f1444a00640 1 Processor -- start 2024-04-25T10:16:42.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.916+0000 7f1444a00640 1 -- start start 2024-04-25T10:16:42.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.918+0000 7f1444a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f144006ea30 0x7f14400b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.918+0000 7f1444a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144015e2a0 con 0x7f1440144040 2024-04-25T10:16:42.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.918+0000 7f1444a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14400cf020 con 0x7f144015ed40 2024-04-25T10:16:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.918+0000 7f1444a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1440144420 con 0x7f144006ea30 2024-04-25T10:16:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.918+0000 7f143f400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1440144040 0x7f14400ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56534/0 (socket says 172.21.15.119:56534) 2024-04-25T10:16:42.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.918+0000 7f143f400640 1 -- 172.21.15.119:0/852235590 learned_addr learned my addr 172.21.15.119:0/852235590 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:16:42.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.918+0000 7f143ea00640 1 -- 172.21.15.119:0/852235590 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491777704 0 0) 0x7f144015e2a0 con 0x7f1440144040 2024-04-25T10:16:42.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.918+0000 7f143ea00640 1 -- 172.21.15.119:0/852235590 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1424003520 con 0x7f1440144040 2024-04-25T10:16:42.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.918+0000 7f143fe00640 1 --2- 172.21.15.119:0/852235590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f144006ea30 0x7f14400b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.919+0000 7f143ea00640 1 -- 172.21.15.119:0/852235590 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3176816002 0 0) 0x7f1424003520 con 0x7f1440144040 2024-04-25T10:16:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.919+0000 7f143ea00640 1 -- 172.21.15.119:0/852235590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f144006ea30 msgr2=0x7f14400b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.919+0000 7f143ea00640 1 --2- 172.21.15.119:0/852235590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f144006ea30 0x7f14400b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.919+0000 7f143ea00640 1 -- 172.21.15.119:0/852235590 >> v1:172.21.15.132:6789/0 conn(0x7f144015ed40 legacy=0x7f144006e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.919+0000 7f143ea00640 1 -- 172.21.15.119:0/852235590 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144015e540 con 0x7f1440144040 2024-04-25T10:16:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.919+0000 7f143ea00640 1 -- 172.21.15.119:0/852235590 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f1430001670 con 0x7f1440144040 2024-04-25T10:16:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.919+0000 7f143ea00640 1 -- 172.21.15.119:0/852235590 >> v1:172.21.15.119:6789/0 conn(0x7f1440144040 legacy=0x7f14400ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.919+0000 7f143ea00640 1 --2- 172.21.15.119:0/852235590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1424003a50 0x7f1424003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.919+0000 7f143ea00640 1 --2- 172.21.15.119:0/852235590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f144006ea30 0x7f1424004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.919+0000 7f143ea00640 1 --2- 172.21.15.119:0/852235590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1424004d90 0x7f1424005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.919+0000 7f143ea00640 1 -- 172.21.15.119:0/852235590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1440144420 con 0x7f1424003a50 2024-04-25T10:16:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.919+0000 7f143ea00640 1 -- 172.21.15.119:0/852235590 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142400bc50 con 0x7f1424004d90 2024-04-25T10:16:42.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.919+0000 7f143ea00640 1 -- 172.21.15.119:0/852235590 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142400be50 con 0x7f144006ea30 2024-04-25T10:16:42.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.919+0000 7f143f400640 1 --2- 172.21.15.119:0/852235590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1424003a50 0x7f1424003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.920+0000 7f1437e00640 1 --2- 172.21.15.119:0/852235590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f144006ea30 0x7f1424004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.920+0000 7f143fe00640 1 --2- 172.21.15.119:0/852235590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1424004d90 0x7f1424005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.920+0000 7f143f400640 1 -- 172.21.15.119:0/852235590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f144006ea30 msgr2=0x7f1424004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:42.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.920+0000 7f143f400640 1 --2- 172.21.15.119:0/852235590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f144006ea30 0x7f1424004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.920+0000 7f143f400640 1 -- 172.21.15.119:0/852235590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1424004d90 msgr2=0x7f1424005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:42.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.920+0000 7f143f400640 1 --2- 172.21.15.119:0/852235590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1424004d90 0x7f1424005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.920+0000 7f143f400640 1 -- 172.21.15.119:0/852235590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1424003520 con 0x7f1424003a50 2024-04-25T10:16:42.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.920+0000 7f143fe00640 1 --2- 172.21.15.119:0/852235590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1424004d90 0x7f1424005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:42.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.921+0000 7f143f400640 1 --2- 172.21.15.119:0/852235590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1424003a50 0x7f1424003e70 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f143000c210 tx=0x7f1424005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67a140e379106a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.921+0000 7f143ea00640 1 -- 172.21.15.119:0/852235590 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430007d30 con 0x7f1424003a50 2024-04-25T10:16:42.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.921+0000 7f143ea00640 1 -- 172.21.15.119:0/852235590 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1430005ad0 con 0x7f1424003a50 2024-04-25T10:16:42.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.921+0000 7f143ea00640 1 -- 172.21.15.119:0/852235590 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430007070 con 0x7f1424003a50 2024-04-25T10:16:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.921+0000 7f1444a00640 1 -- 172.21.15.119:0/852235590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1424003a50 msgr2=0x7f1424003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.921+0000 7f1444a00640 1 --2- 172.21.15.119:0/852235590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1424003a50 0x7f1424003e70 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f143000c210 tx=0x7f1424005830 comp rx=0 tx=0).stop 2024-04-25T10:16:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.921+0000 7f143fe00640 1 -- 172.21.15.119:0/852235590 reap_dead start 2024-04-25T10:16:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.922+0000 7f1444a00640 1 -- 172.21.15.119:0/852235590 shutdown_connections 2024-04-25T10:16:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.922+0000 7f1444a00640 1 -- 172.21.15.119:0/852235590 >> 172.21.15.119:0/852235590 conn(0x7f144006b1d0 msgr2=0x7f14400b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.922+0000 7f1444a00640 1 -- 172.21.15.119:0/852235590 shutdown_connections 2024-04-25T10:16:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.922+0000 7f1444a00640 1 -- 172.21.15.119:0/852235590 wait complete. 2024-04-25T10:16:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.923+0000 7f1444a00640 1 Processor -- start 2024-04-25T10:16:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.924+0000 7f1444a00640 1 -- start start 2024-04-25T10:16:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.924+0000 7f1444a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1424003a50 0x7f144015a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.924+0000 7f1444a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14401e4770 0x7f14401e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.924+0000 7f1444a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14401f38a0 0x7f14401f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.924+0000 7f1444a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14400b51d0 con 0x7f14401e4770 2024-04-25T10:16:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.924+0000 7f1444a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14400cecc0 con 0x7f14401f38a0 2024-04-25T10:16:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.924+0000 7f1444a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14400b5350 con 0x7f1424003a50 2024-04-25T10:16:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.925+0000 7f1437e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14401e4770 0x7f14401e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.925+0000 7f1437e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14401e4770 0x7f14401e4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42208/0 (socket says 172.21.15.119:42208) 2024-04-25T10:16:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.925+0000 7f1437e00640 1 -- 172.21.15.119:0/3747691941 learned_addr learned my addr 172.21.15.119:0/3747691941 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.925+0000 7f143f400640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1424003a50 0x7f144015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.925+0000 7f143fe00640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14401f38a0 0x7f14401f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.925+0000 7f1437e00640 1 -- 172.21.15.119:0/3747691941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1424003a50 msgr2=0x7f144015a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.925+0000 7f1437e00640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1424003a50 0x7f144015a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.925+0000 7f1437e00640 1 -- 172.21.15.119:0/3747691941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14401f38a0 msgr2=0x7f14401f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.925+0000 7f1437e00640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14401f38a0 0x7f14401f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.925+0000 7f1437e00640 1 -- 172.21.15.119:0/3747691941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14401f4640 con 0x7f14401e4770 2024-04-25T10:16:42.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.925+0000 7f143fe00640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14401f38a0 0x7f14401f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:42.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.925+0000 7f1437e00640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14401e4770 0x7f14401e4bf0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f142800e490 tx=0x7f142800e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.926+0000 7f1437400640 1 -- 172.21.15.119:0/3747691941 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14280052e0 con 0x7f14401e4770 2024-04-25T10:16:42.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.926+0000 7f1437400640 1 -- 172.21.15.119:0/3747691941 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1428018070 con 0x7f14401e4770 2024-04-25T10:16:42.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.926+0000 7f1437400640 1 -- 172.21.15.119:0/3747691941 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428003830 con 0x7f14401e4770 2024-04-25T10:16:42.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.926+0000 7f1444a00640 1 -- 172.21.15.119:0/3747691941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1440140bf0 con 0x7f14401e4770 2024-04-25T10:16:42.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.926+0000 7f1444a00640 1 -- 172.21.15.119:0/3747691941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14401411b0 con 0x7f14401e4770 2024-04-25T10:16:42.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.927+0000 7f1444a00640 1 -- 172.21.15.119:0/3747691941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14400749b0 con 0x7f14401e4770 2024-04-25T10:16:42.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.928+0000 7f1437400640 1 -- 172.21.15.119:0/3747691941 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14280039d0 con 0x7f14401e4770 2024-04-25T10:16:42.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.929+0000 7f1437400640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f141c06b500 0x7f141c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.929+0000 7f1437400640 1 -- 172.21.15.119:0/3747691941 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1428089f50 con 0x7f14401e4770 2024-04-25T10:16:42.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.929+0000 7f143f400640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f141c06b500 0x7f141c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.930+0000 7f143f400640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f141c06b500 0x7f141c06d9c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1430033000 tx=0x7f143002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:42.934+0000 7f1437400640 1 -- 172.21.15.119:0/3747691941 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1428058cd0 con 0x7f14401e4770 2024-04-25T10:16:43.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:43.162+0000 7f1444a00640 1 -- 172.21.15.119:0/3747691941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14400ce610 con 0x7f14401e4770 2024-04-25T10:16:43.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:43.163+0000 7f1437400640 1 -- 172.21.15.119:0/3747691941 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f14280585d0 con 0x7f14401e4770 2024-04-25T10:16:43.167 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:43.167 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5459,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3768348114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3768348114},{"type":"v1","addr":"172.21.15.132:6834","nonce":3768348114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":5465,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/543190780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":543190780},{"type":"v1","addr":"172.21.15.132:6835","nonce":543190780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1145229172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1145229172},{"type":"v1","addr":"172.21.15.119:6836","nonce":1145229172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:51.568312+0000","modified":"2024-04-25T10:16:42.731379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8367},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8367":{"gid":8367,"name":"a","rank":0,"incarnation":85,"state":"up:replay","state_seq":1,"addr":"172.21.15.119:6837/464677621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":464677621},{"type":"v1","addr":"172.21.15.119:6837","nonce":464677621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:16:43.167 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 85 2024-04-25T10:16:43.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:43.165+0000 7f1444a00640 1 -- 172.21.15.119:0/3747691941 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f141c06b500 msgr2=0x7f141c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:43.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:43.165+0000 7f1444a00640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f141c06b500 0x7f141c06d9c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1430033000 tx=0x7f143002e000 comp rx=0 tx=0).stop 2024-04-25T10:16:43.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:43.165+0000 7f1444a00640 1 -- 172.21.15.119:0/3747691941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14401e4770 msgr2=0x7f14401e4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:43.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:43.165+0000 7f1444a00640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14401e4770 0x7f14401e4bf0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f142800e490 tx=0x7f142800e950 comp rx=0 tx=0).stop 2024-04-25T10:16:43.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:43.165+0000 7f1444a00640 1 -- 172.21.15.119:0/3747691941 shutdown_connections 2024-04-25T10:16:43.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:43.165+0000 7f1444a00640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f141c06b500 0x7f141c06d9c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:43.165+0000 7f1444a00640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14401f38a0 0x7f14401f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:43.165+0000 7f1444a00640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14401e4770 0x7f14401e4bf0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:43.165+0000 7f1444a00640 1 --2- 172.21.15.119:0/3747691941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1424003a50 0x7f144015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:43.166+0000 7f1444a00640 1 -- 172.21.15.119:0/3747691941 >> 172.21.15.119:0/3747691941 conn(0x7f144006b1d0 msgr2=0x7f144006df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:43.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:43.166+0000 7f1444a00640 1 -- 172.21.15.119:0/3747691941 shutdown_connections 2024-04-25T10:16:43.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:43.166+0000 7f1444a00640 1 -- 172.21.15.119:0/3747691941 wait complete. 2024-04-25T10:16:43.589 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:15:51.568312+0000', 'modified': '2024-04-25T10:16:42.731379+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8367}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8367': {'gid': 8367, 'name': 'a', 'rank': 0, 'incarnation': 85, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.119:6837/464677621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 464677621}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 464677621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:43.589 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8367:up:replay 2024-04-25T10:16:44.590 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:44.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.795+0000 7efc45e00640 1 Processor -- start 2024-04-25T10:16:44.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.795+0000 7efc45e00640 1 -- start start 2024-04-25T10:16:44.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.797+0000 7efc45e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efc400b9270 0x7efc400d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.797+0000 7efc45e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4015ef10 con 0x7efc400d3f80 2024-04-25T10:16:44.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.797+0000 7efc45e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4015e390 con 0x7efc4015db60 2024-04-25T10:16:44.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.797+0000 7efc45e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4015e670 con 0x7efc400b9270 2024-04-25T10:16:44.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.797+0000 7efc37e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7efc400d3f80 0x7efc400d1f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56536/0 (socket says 172.21.15.119:56536) 2024-04-25T10:16:44.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.797+0000 7efc37e00640 1 -- 172.21.15.119:0/3454006924 learned_addr learned my addr 172.21.15.119:0/3454006924 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:16:44.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc45400640 1 --2- 172.21.15.119:0/3454006924 >> v2:172.21.15.132:3300/0 conn(0x7efc400b9270 0x7efc400d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 -- 172.21.15.119:0/3454006924 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227095004 0 0) 0x7efc4015ef10 con 0x7efc400d3f80 2024-04-25T10:16:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 -- 172.21.15.119:0/3454006924 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc24003520 con 0x7efc400d3f80 2024-04-25T10:16:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 -- 172.21.15.119:0/3454006924 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 832763706 0 0) 0x7efc24003520 con 0x7efc400d3f80 2024-04-25T10:16:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 -- 172.21.15.119:0/3454006924 >> v2:172.21.15.132:3300/0 conn(0x7efc400b9270 msgr2=0x7efc400d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 --2- 172.21.15.119:0/3454006924 >> v2:172.21.15.132:3300/0 conn(0x7efc400b9270 0x7efc400d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 -- 172.21.15.119:0/3454006924 >> v1:172.21.15.132:6789/0 conn(0x7efc4015db60 legacy=0x7efc400d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 -- 172.21.15.119:0/3454006924 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4015e950 con 0x7efc400d3f80 2024-04-25T10:16:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 -- 172.21.15.119:0/3454006924 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7efc280017f0 con 0x7efc400d3f80 2024-04-25T10:16:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 -- 172.21.15.119:0/3454006924 >> v1:172.21.15.119:6789/0 conn(0x7efc400d3f80 legacy=0x7efc400d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 --2- 172.21.15.119:0/3454006924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc24003a70 0x7efc24003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 --2- 172.21.15.119:0/3454006924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc24004800 0x7efc24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 --2- 172.21.15.119:0/3454006924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc240052e0 0x7efc2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 -- 172.21.15.119:0/3454006924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4015e670 con 0x7efc240052e0 2024-04-25T10:16:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 -- 172.21.15.119:0/3454006924 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2400c1b0 con 0x7efc24004800 2024-04-25T10:16:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc3fe00640 1 -- 172.21.15.119:0/3454006924 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2400c3b0 con 0x7efc24003a70 2024-04-25T10:16:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.798+0000 7efc45400640 1 --2- 172.21.15.119:0/3454006924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc240052e0 0x7efc2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.799+0000 7efc37e00640 1 --2- 172.21.15.119:0/3454006924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc24004800 0x7efc24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.799+0000 7efc45400640 1 -- 172.21.15.119:0/3454006924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc24003a70 msgr2=0x7efc24003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.799+0000 7efc44a00640 1 --2- 172.21.15.119:0/3454006924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc24003a70 0x7efc24003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.799+0000 7efc45400640 1 --2- 172.21.15.119:0/3454006924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc24003a70 0x7efc24003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.799+0000 7efc45400640 1 -- 172.21.15.119:0/3454006924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc24004800 msgr2=0x7efc24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.799+0000 7efc45400640 1 --2- 172.21.15.119:0/3454006924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc24004800 0x7efc24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.799+0000 7efc45400640 1 -- 172.21.15.119:0/3454006924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc24003520 con 0x7efc240052e0 2024-04-25T10:16:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.799+0000 7efc37e00640 1 --2- 172.21.15.119:0/3454006924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc24004800 0x7efc24004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.799+0000 7efc44a00640 1 --2- 172.21.15.119:0/3454006924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc24003a70 0x7efc24003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.800+0000 7efc45400640 1 -- 172.21.15.119:0/3454006924 reap_dead start 2024-04-25T10:16:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.800+0000 7efc45400640 1 --2- 172.21.15.119:0/3454006924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc240052e0 0x7efc2400b9c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7efc38007330 tx=0x7efc24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d0e00467046c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.800+0000 7efc3fe00640 1 -- 172.21.15.119:0/3454006924 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38009be0 con 0x7efc240052e0 2024-04-25T10:16:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.800+0000 7efc3fe00640 1 -- 172.21.15.119:0/3454006924 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc38003b90 con 0x7efc240052e0 2024-04-25T10:16:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.800+0000 7efc3fe00640 1 -- 172.21.15.119:0/3454006924 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38004020 con 0x7efc240052e0 2024-04-25T10:16:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.801+0000 7efc45e00640 1 -- 172.21.15.119:0/3454006924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc240052e0 msgr2=0x7efc2400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.801+0000 7efc45e00640 1 --2- 172.21.15.119:0/3454006924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc240052e0 0x7efc2400b9c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7efc38007330 tx=0x7efc24005830 comp rx=0 tx=0).stop 2024-04-25T10:16:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.801+0000 7efc45e00640 1 -- 172.21.15.119:0/3454006924 shutdown_connections 2024-04-25T10:16:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.801+0000 7efc45e00640 1 --2- 172.21.15.119:0/3454006924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc240052e0 0x7efc2400b9c0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.801+0000 7efc45e00640 1 -- 172.21.15.119:0/3454006924 >> 172.21.15.119:0/3454006924 conn(0x7efc400726c0 msgr2=0x7efc40072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:44.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.801+0000 7efc45e00640 1 -- 172.21.15.119:0/3454006924 shutdown_connections 2024-04-25T10:16:44.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.801+0000 7efc45e00640 1 -- 172.21.15.119:0/3454006924 wait complete. 2024-04-25T10:16:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.803+0000 7efc45e00640 1 Processor -- start 2024-04-25T10:16:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.803+0000 7efc45e00640 1 -- start start 2024-04-25T10:16:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.804+0000 7efc45e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc240052e0 0x7efc400cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.804+0000 7efc45e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc400cbfa0 0x7efc401f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.804+0000 7efc45e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc401f7530 0x7efc401f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.804+0000 7efc45e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc400b3120 con 0x7efc240052e0 2024-04-25T10:16:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.804+0000 7efc45e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc400b2fa0 con 0x7efc400cbfa0 2024-04-25T10:16:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.804+0000 7efc45e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4015df40 con 0x7efc401f7530 2024-04-25T10:16:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.804+0000 7efc44a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc240052e0 0x7efc400cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.804+0000 7efc44a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc240052e0 0x7efc400cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42222/0 (socket says 172.21.15.119:42222) 2024-04-25T10:16:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.804+0000 7efc44a00640 1 -- 172.21.15.119:0/2350251492 learned_addr learned my addr 172.21.15.119:0/2350251492 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.804+0000 7efc44a00640 1 -- 172.21.15.119:0/2350251492 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc401f7530 msgr2=0x7efc401f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.804+0000 7efc44a00640 1 --2- 172.21.15.119:0/2350251492 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc401f7530 0x7efc401f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.804+0000 7efc44a00640 1 -- 172.21.15.119:0/2350251492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc400cbfa0 msgr2=0x7efc401f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.804+0000 7efc44a00640 1 --2- 172.21.15.119:0/2350251492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc400cbfa0 0x7efc401f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.804+0000 7efc44a00640 1 -- 172.21.15.119:0/2350251492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc400b3d50 con 0x7efc240052e0 2024-04-25T10:16:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.805+0000 7efc44a00640 1 --2- 172.21.15.119:0/2350251492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc240052e0 0x7efc400cd510 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7efc3000d590 tx=0x7efc3000da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.805+0000 7efc3d600640 1 -- 172.21.15.119:0/2350251492 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30016070 con 0x7efc240052e0 2024-04-25T10:16:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.805+0000 7efc3d600640 1 -- 172.21.15.119:0/2350251492 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc30007240 con 0x7efc240052e0 2024-04-25T10:16:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.805+0000 7efc3d600640 1 -- 172.21.15.119:0/2350251492 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30007630 con 0x7efc240052e0 2024-04-25T10:16:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.805+0000 7efc45e00640 1 -- 172.21.15.119:0/2350251492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc400b4030 con 0x7efc240052e0 2024-04-25T10:16:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.805+0000 7efc45e00640 1 -- 172.21.15.119:0/2350251492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc400b45f0 con 0x7efc240052e0 2024-04-25T10:16:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.806+0000 7efc45e00640 1 -- 172.21.15.119:0/2350251492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc400b47f0 con 0x7efc240052e0 2024-04-25T10:16:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.807+0000 7efc3d600640 1 -- 172.21.15.119:0/2350251492 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc300067d0 con 0x7efc240052e0 2024-04-25T10:16:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.808+0000 7efc3d600640 1 --2- 172.21.15.119:0/2350251492 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc1c06b4d0 0x7efc1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.808+0000 7efc3d600640 1 -- 172.21.15.119:0/2350251492 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efc3008a0f0 con 0x7efc240052e0 2024-04-25T10:16:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.808+0000 7efc37e00640 1 --2- 172.21.15.119:0/2350251492 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc1c06b4d0 0x7efc1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.809+0000 7efc37e00640 1 --2- 172.21.15.119:0/2350251492 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc1c06b4d0 0x7efc1c06d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efc28000c00 tx=0x7efc28009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:44.812+0000 7efc3d600640 1 -- 172.21.15.119:0/2350251492 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc30058f00 con 0x7efc240052e0 2024-04-25T10:16:45.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:45.043+0000 7efc45e00640 1 -- 172.21.15.119:0/2350251492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc400cc3e0 con 0x7efc240052e0 2024-04-25T10:16:45.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:45.044+0000 7efc3d600640 1 -- 172.21.15.119:0/2350251492 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4947 (secure 0 0 0) 0x7efc30058800 con 0x7efc240052e0 2024-04-25T10:16:45.048 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:45.048 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5459,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3768348114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3768348114},{"type":"v1","addr":"172.21.15.132:6834","nonce":3768348114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":5465,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/543190780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":543190780},{"type":"v1","addr":"172.21.15.132:6835","nonce":543190780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1145229172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1145229172},{"type":"v1","addr":"172.21.15.119:6836","nonce":1145229172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:51.568312+0000","modified":"2024-04-25T10:16:43.751425+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8367},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8367":{"gid":8367,"name":"a","rank":0,"incarnation":85,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.119:6837/464677621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":464677621},{"type":"v1","addr":"172.21.15.119:6837","nonce":464677621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:16:45.048 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 87 2024-04-25T10:16:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:45.047+0000 7efc45e00640 1 -- 172.21.15.119:0/2350251492 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc1c06b4d0 msgr2=0x7efc1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:45.047+0000 7efc45e00640 1 --2- 172.21.15.119:0/2350251492 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc1c06b4d0 0x7efc1c06d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efc28000c00 tx=0x7efc28009040 comp rx=0 tx=0).stop 2024-04-25T10:16:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:45.047+0000 7efc45e00640 1 -- 172.21.15.119:0/2350251492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc240052e0 msgr2=0x7efc400cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:45.047+0000 7efc45e00640 1 --2- 172.21.15.119:0/2350251492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc240052e0 0x7efc400cd510 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7efc3000d590 tx=0x7efc3000da50 comp rx=0 tx=0).stop 2024-04-25T10:16:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:45.047+0000 7efc45e00640 1 -- 172.21.15.119:0/2350251492 shutdown_connections 2024-04-25T10:16:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:45.047+0000 7efc45e00640 1 --2- 172.21.15.119:0/2350251492 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc1c06b4d0 0x7efc1c06d990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:45.047+0000 7efc45e00640 1 --2- 172.21.15.119:0/2350251492 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc401f7530 0x7efc401f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:45.047+0000 7efc45e00640 1 --2- 172.21.15.119:0/2350251492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc400cbfa0 0x7efc401f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:45.047+0000 7efc45e00640 1 --2- 172.21.15.119:0/2350251492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc240052e0 0x7efc400cd510 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:45.047+0000 7efc45e00640 1 -- 172.21.15.119:0/2350251492 >> 172.21.15.119:0/2350251492 conn(0x7efc400726c0 msgr2=0x7efc400c9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:45.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:45.047+0000 7efc45e00640 1 -- 172.21.15.119:0/2350251492 shutdown_connections 2024-04-25T10:16:45.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:45.047+0000 7efc45e00640 1 -- 172.21.15.119:0/2350251492 wait complete. 2024-04-25T10:16:45.472 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:15:51.568312+0000', 'modified': '2024-04-25T10:16:43.751425+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8367}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8367': {'gid': 8367, 'name': 'a', 'rank': 0, 'incarnation': 85, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.119:6837/464677621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 464677621}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 464677621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:45.472 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8367:up:rejoin 2024-04-25T10:16:46.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:46.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.680+0000 7fe2f5a00640 1 Processor -- start 2024-04-25T10:16:46.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.680+0000 7fe2f5a00640 1 -- start start 2024-04-25T10:16:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.682+0000 7fe2f5a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe2f015bf00 0x7fe2f00cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.682+0000 7fe2f5a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2f00cc310 0x7fe2f015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.682+0000 7fe2f5a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f00ce6c0 con 0x7fe2f015c820 2024-04-25T10:16:46.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.682+0000 7fe2f5a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f01455e0 con 0x7fe2f015bf00 2024-04-25T10:16:46.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.682+0000 7fe2f5a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f006d450 con 0x7fe2f00cc310 2024-04-25T10:16:46.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.682+0000 7fe2ef400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe2f015c820 0x7fe2f006d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56544/0 (socket says 172.21.15.119:56544) 2024-04-25T10:16:46.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.682+0000 7fe2ef400640 1 -- 172.21.15.119:0/975337242 learned_addr learned my addr 172.21.15.119:0/975337242 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:16:46.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ee000640 1 --2- 172.21.15.119:0/975337242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2f00cc310 0x7fe2f015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2eea00640 1 --2- 172.21.15.119:0/975337242 >> v2:172.21.15.132:3300/0 conn(0x7fe2f015bf00 0x7fe2f00cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 -- 172.21.15.119:0/975337242 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821882659 0 0) 0x7fe2e8001490 con 0x7fe2f015c820 2024-04-25T10:16:46.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 -- 172.21.15.119:0/975337242 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2d4003520 con 0x7fe2f015c820 2024-04-25T10:16:46.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 -- 172.21.15.119:0/975337242 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2224374827 0 0) 0x7fe2d4003520 con 0x7fe2f015c820 2024-04-25T10:16:46.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 -- 172.21.15.119:0/975337242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2f00cc310 msgr2=0x7fe2f015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 --2- 172.21.15.119:0/975337242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2f00cc310 0x7fe2f015c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 -- 172.21.15.119:0/975337242 >> v2:172.21.15.132:3300/0 conn(0x7fe2f015bf00 msgr2=0x7fe2f00cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 --2- 172.21.15.119:0/975337242 >> v2:172.21.15.132:3300/0 conn(0x7fe2f015bf00 0x7fe2f00cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 -- 172.21.15.119:0/975337242 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f00cc7b0 con 0x7fe2f015c820 2024-04-25T10:16:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 -- 172.21.15.119:0/975337242 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fe2e8001bc0 con 0x7fe2f015c820 2024-04-25T10:16:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 -- 172.21.15.119:0/975337242 >> v1:172.21.15.119:6789/0 conn(0x7fe2f015c820 legacy=0x7fe2f006d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 --2- 172.21.15.119:0/975337242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2d40039b0 0x7fe2d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 --2- 172.21.15.119:0/975337242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d4004800 0x7fe2d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 --2- 172.21.15.119:0/975337242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2f00cc310 0x7fe2d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 -- 172.21.15.119:0/975337242 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f01455e0 con 0x7fe2d4004800 2024-04-25T10:16:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 -- 172.21.15.119:0/975337242 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f006d450 con 0x7fe2d40039b0 2024-04-25T10:16:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.683+0000 7fe2ed600640 1 -- 172.21.15.119:0/975337242 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d400b9c0 con 0x7fe2f00cc310 2024-04-25T10:16:46.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.684+0000 7fe2ee000640 1 --2- 172.21.15.119:0/975337242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d4004800 0x7fe2d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.684+0000 7fe2eea00640 1 --2- 172.21.15.119:0/975337242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2d40039b0 0x7fe2d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.684+0000 7fe2ee000640 1 -- 172.21.15.119:0/975337242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2f00cc310 msgr2=0x7fe2d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:46.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.684+0000 7fe2ee000640 1 --2- 172.21.15.119:0/975337242 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2f00cc310 0x7fe2d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.684+0000 7fe2ee000640 1 -- 172.21.15.119:0/975337242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2d40039b0 msgr2=0x7fe2d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:46.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.684+0000 7fe2ee000640 1 --2- 172.21.15.119:0/975337242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2d40039b0 0x7fe2d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.684+0000 7fe2ee000640 1 -- 172.21.15.119:0/975337242 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d4003520 con 0x7fe2d4004800 2024-04-25T10:16:46.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.684+0000 7fe2eea00640 1 --2- 172.21.15.119:0/975337242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2d40039b0 0x7fe2d4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:46.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.685+0000 7fe2ee000640 1 --2- 172.21.15.119:0/975337242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d4004800 0x7fe2d4004c80 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009a90 tx=0x7fe2d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec44de7589d6339 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.685+0000 7fe2ed600640 1 -- 172.21.15.119:0/975337242 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc036070 con 0x7fe2d4004800 2024-04-25T10:16:46.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.685+0000 7fe2ed600640 1 -- 172.21.15.119:0/975337242 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc031400 con 0x7fe2d4004800 2024-04-25T10:16:46.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.685+0000 7fe2ed600640 1 -- 172.21.15.119:0/975337242 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc009690 con 0x7fe2d4004800 2024-04-25T10:16:46.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.686+0000 7fe2f5a00640 1 -- 172.21.15.119:0/975337242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d4004800 msgr2=0x7fe2d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.686+0000 7fe2f5a00640 1 --2- 172.21.15.119:0/975337242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d4004800 0x7fe2d4004c80 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009a90 tx=0x7fe2d4005830 comp rx=0 tx=0).stop 2024-04-25T10:16:46.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.686+0000 7fe2ef400640 1 -- 172.21.15.119:0/975337242 reap_dead start 2024-04-25T10:16:46.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.686+0000 7fe2f5a00640 1 -- 172.21.15.119:0/975337242 shutdown_connections 2024-04-25T10:16:46.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.686+0000 7fe2f5a00640 1 -- 172.21.15.119:0/975337242 >> 172.21.15.119:0/975337242 conn(0x7fe2f006aba0 msgr2=0x7fe2f006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:46.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.686+0000 7fe2f5a00640 1 -- 172.21.15.119:0/975337242 shutdown_connections 2024-04-25T10:16:46.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.686+0000 7fe2f5a00640 1 -- 172.21.15.119:0/975337242 wait complete. 2024-04-25T10:16:46.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.688+0000 7fe2f5a00640 1 Processor -- start 2024-04-25T10:16:46.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.688+0000 7fe2f5a00640 1 -- start start 2024-04-25T10:16:46.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.689+0000 7fe2f5a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2d4004800 0x7fe2f00b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.689+0000 7fe2f5a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2f0144440 0x7fe2f00cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.689+0000 7fe2f5a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2f01448a0 0x7fe2f00b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.689+0000 7fe2f5a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f00cd050 con 0x7fe2f01448a0 2024-04-25T10:16:46.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.689+0000 7fe2f5a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f00ccd70 con 0x7fe2f0144440 2024-04-25T10:16:46.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.689+0000 7fe2f5a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f015cca0 con 0x7fe2d4004800 2024-04-25T10:16:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.689+0000 7fe2ef400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2f01448a0 0x7fe2f00b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.689+0000 7fe2ef400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2f01448a0 0x7fe2f00b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42238/0 (socket says 172.21.15.119:42238) 2024-04-25T10:16:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.689+0000 7fe2ef400640 1 -- 172.21.15.119:0/3700992926 learned_addr learned my addr 172.21.15.119:0/3700992926 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.690+0000 7fe2ee000640 1 --2- 172.21.15.119:0/3700992926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2f0144440 0x7fe2f00cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.690+0000 7fe2eea00640 1 --2- 172.21.15.119:0/3700992926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2d4004800 0x7fe2f00b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.690+0000 7fe2ef400640 1 -- 172.21.15.119:0/3700992926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2d4004800 msgr2=0x7fe2f00b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.690+0000 7fe2ef400640 1 --2- 172.21.15.119:0/3700992926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2d4004800 0x7fe2f00b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.690+0000 7fe2ef400640 1 -- 172.21.15.119:0/3700992926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2f0144440 msgr2=0x7fe2f00cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.690+0000 7fe2ef400640 1 --2- 172.21.15.119:0/3700992926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2f0144440 0x7fe2f00cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.690+0000 7fe2ef400640 1 -- 172.21.15.119:0/3700992926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f01f70c0 con 0x7fe2f01448a0 2024-04-25T10:16:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.690+0000 7fe2ef400640 1 --2- 172.21.15.119:0/3700992926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2f01448a0 0x7fe2f00b37a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00d0190 tx=0x7fe2e800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.690+0000 7fe2e6a00640 1 -- 172.21.15.119:0/3700992926 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e800a610 con 0x7fe2f01448a0 2024-04-25T10:16:46.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.691+0000 7fe2e6a00640 1 -- 172.21.15.119:0/3700992926 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e800a7d0 con 0x7fe2f01448a0 2024-04-25T10:16:46.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.691+0000 7fe2e6a00640 1 -- 172.21.15.119:0/3700992926 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e800abc0 con 0x7fe2f01448a0 2024-04-25T10:16:46.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.691+0000 7fe2f5a00640 1 -- 172.21.15.119:0/3700992926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f01f73a0 con 0x7fe2f01448a0 2024-04-25T10:16:46.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.691+0000 7fe2f5a00640 1 -- 172.21.15.119:0/3700992926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f01f7960 con 0x7fe2f01448a0 2024-04-25T10:16:46.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.692+0000 7fe2f5a00640 1 -- 172.21.15.119:0/3700992926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f00749b0 con 0x7fe2f01448a0 2024-04-25T10:16:46.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.693+0000 7fe2e6a00640 1 -- 172.21.15.119:0/3700992926 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2e8011070 con 0x7fe2f01448a0 2024-04-25T10:16:46.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.694+0000 7fe2e6a00640 1 --2- 172.21.15.119:0/3700992926 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2d006b4d0 0x7fe2d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.694+0000 7fe2e6a00640 1 -- 172.21.15.119:0/3700992926 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe2e80898f0 con 0x7fe2f01448a0 2024-04-25T10:16:46.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.695+0000 7fe2eea00640 1 --2- 172.21.15.119:0/3700992926 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2d006b4d0 0x7fe2d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.695+0000 7fe2eea00640 1 --2- 172.21.15.119:0/3700992926 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2d006b4d0 0x7fe2d006d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009b20 tx=0x7fe2e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.699+0000 7fe2e6a00640 1 -- 172.21.15.119:0/3700992926 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2f00ce6c0 con 0x7fe2f01448a0 2024-04-25T10:16:46.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.932+0000 7fe2f5a00640 1 -- 172.21.15.119:0/3700992926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2f0144e00 con 0x7fe2f01448a0 2024-04-25T10:16:46.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.933+0000 7fe2e6a00640 1 -- 172.21.15.119:0/3700992926 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fe2e8058670 con 0x7fe2f01448a0 2024-04-25T10:16:46.937 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:46.937 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5459,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3768348114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3768348114},{"type":"v1","addr":"172.21.15.132:6834","nonce":3768348114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":5465,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/543190780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":543190780},{"type":"v1","addr":"172.21.15.132:6835","nonce":543190780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1145229172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1145229172},{"type":"v1","addr":"172.21.15.119:6836","nonce":1145229172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:51.568312+0000","modified":"2024-04-25T10:16:44.763797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8367},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8367":{"gid":8367,"name":"a","rank":0,"incarnation":85,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/464677621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":464677621},{"type":"v1","addr":"172.21.15.119:6837","nonce":464677621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:16:46.938 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 88 2024-04-25T10:16:46.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.936+0000 7fe2f5a00640 1 -- 172.21.15.119:0/3700992926 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2d006b4d0 msgr2=0x7fe2d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.936+0000 7fe2f5a00640 1 --2- 172.21.15.119:0/3700992926 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2d006b4d0 0x7fe2d006d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009b20 tx=0x7fe2e0008040 comp rx=0 tx=0).stop 2024-04-25T10:16:46.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.936+0000 7fe2f5a00640 1 -- 172.21.15.119:0/3700992926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2f01448a0 msgr2=0x7fe2f00b37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.936+0000 7fe2f5a00640 1 --2- 172.21.15.119:0/3700992926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2f01448a0 0x7fe2f00b37a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00d0190 tx=0x7fe2e800cc60 comp rx=0 tx=0).stop 2024-04-25T10:16:46.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.936+0000 7fe2f5a00640 1 -- 172.21.15.119:0/3700992926 shutdown_connections 2024-04-25T10:16:46.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.936+0000 7fe2f5a00640 1 --2- 172.21.15.119:0/3700992926 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2d006b4d0 0x7fe2d006d990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.936+0000 7fe2f5a00640 1 --2- 172.21.15.119:0/3700992926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2f01448a0 0x7fe2f00b37a0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.936+0000 7fe2f5a00640 1 --2- 172.21.15.119:0/3700992926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2f0144440 0x7fe2f00cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.936+0000 7fe2f5a00640 1 --2- 172.21.15.119:0/3700992926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2d4004800 0x7fe2f00b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.936+0000 7fe2f5a00640 1 -- 172.21.15.119:0/3700992926 >> 172.21.15.119:0/3700992926 conn(0x7fe2f006aba0 msgr2=0x7fe2f015cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:46.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.936+0000 7fe2f5a00640 1 -- 172.21.15.119:0/3700992926 shutdown_connections 2024-04-25T10:16:46.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:46.936+0000 7fe2f5a00640 1 -- 172.21.15.119:0/3700992926 wait complete. 2024-04-25T10:16:47.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:15:51.568312+0000', 'modified': '2024-04-25T10:16:44.763797+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8367}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8367': {'gid': 8367, 'name': 'a', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.119:6837/464677621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 464677621}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 464677621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:47.361 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:16:47.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:16:47.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.572+0000 7f7dbfe00640 1 Processor -- start 2024-04-25T10:16:47.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.572+0000 7f7dbfe00640 1 -- start start 2024-04-25T10:16:47.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.573+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dc015cc10 0x7f7dc00cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.573+0000 7f7dbfe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7dc015d2d0 0x7f7dc0099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.573+0000 7f7dbfe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc00cf250 con 0x7f7dc00cc780 2024-04-25T10:16:47.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.573+0000 7f7dbfe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc00ccca0 con 0x7f7dc015d2d0 2024-04-25T10:16:47.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.573+0000 7f7dbfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00ccf80 con 0x7f7dc015cc10 2024-04-25T10:16:47.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.574+0000 7f7dbea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dc015cc10 0x7f7dc00cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.574+0000 7f7dbe000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7dc015d2d0 0x7f7dc0099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.574+0000 7f7dbea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dc015cc10 0x7f7dc00cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:51014/0 (socket says 172.21.15.119:51014) 2024-04-25T10:16:47.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.574+0000 7f7dbea00640 1 -- 172.21.15.119:0/3314471905 learned_addr learned my addr 172.21.15.119:0/3314471905 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:47.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.575+0000 7f7dbea00640 1 -- 172.21.15.119:0/3314471905 >> v2:172.21.15.132:3300/0 conn(0x7f7dc015d2d0 msgr2=0x7f7dc0099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.575+0000 7f7dbea00640 1 --2- 172.21.15.119:0/3314471905 >> v2:172.21.15.132:3300/0 conn(0x7f7dc015d2d0 0x7f7dc0099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.575+0000 7f7dbea00640 1 -- 172.21.15.119:0/3314471905 >> v1:172.21.15.132:6789/0 conn(0x7f7dc00cc780 legacy=0x7f7dc015f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.575+0000 7f7dbea00640 1 -- 172.21.15.119:0/3314471905 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc00cd260 con 0x7f7dc015cc10 2024-04-25T10:16:47.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.576+0000 7f7dbea00640 1 --2- 172.21.15.119:0/3314471905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dc015cc10 0x7f7dc00cc240 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009870 tx=0x7f7db002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3f5b750e1d88843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.577+0000 7f7dbd600640 1 -- 172.21.15.119:0/3314471905 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db0037070 con 0x7f7dc015cc10 2024-04-25T10:16:47.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.577+0000 7f7dbd600640 1 -- 172.21.15.119:0/3314471905 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db0032440 con 0x7f7dc015cc10 2024-04-25T10:16:47.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.577+0000 7f7dbd600640 1 -- 172.21.15.119:0/3314471905 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db0032810 con 0x7f7dc015cc10 2024-04-25T10:16:47.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.577+0000 7f7dbfe00640 1 -- 172.21.15.119:0/3314471905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dc015cc10 msgr2=0x7f7dc00cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.577+0000 7f7dbfe00640 1 --2- 172.21.15.119:0/3314471905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dc015cc10 0x7f7dc00cc240 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009870 tx=0x7f7db002a270 comp rx=0 tx=0).stop 2024-04-25T10:16:47.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.578+0000 7f7dbfe00640 1 -- 172.21.15.119:0/3314471905 shutdown_connections 2024-04-25T10:16:47.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.578+0000 7f7dbfe00640 1 --2- 172.21.15.119:0/3314471905 >> v2:172.21.15.132:3300/0 conn(0x7f7dc015d2d0 0x7f7dc0099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.578+0000 7f7dbfe00640 1 --2- 172.21.15.119:0/3314471905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dc015cc10 0x7f7dc00cc240 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.578+0000 7f7dbfe00640 1 -- 172.21.15.119:0/3314471905 >> 172.21.15.119:0/3314471905 conn(0x7f7dc00b30d0 msgr2=0x7f7dc00b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.578+0000 7f7dbfe00640 1 -- 172.21.15.119:0/3314471905 shutdown_connections 2024-04-25T10:16:47.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.578+0000 7f7dbfe00640 1 -- 172.21.15.119:0/3314471905 wait complete. 2024-04-25T10:16:47.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.580+0000 7f7dbfe00640 1 Processor -- start 2024-04-25T10:16:47.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.580+0000 7f7dbfe00640 1 -- start start 2024-04-25T10:16:47.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dc00cc780 0x7f7dc015b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc015cc10 0x7f7dc015bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dc015d2d0 0x7f7dc00c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbfe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc0099f80 con 0x7f7dc015cc10 2024-04-25T10:16:47.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbfe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc009a9c0 con 0x7f7dc015d2d0 2024-04-25T10:16:47.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc009a100 con 0x7f7dc00cc780 2024-04-25T10:16:47.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbe000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc015cc10 0x7f7dc015bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbe000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc015cc10 0x7f7dc015bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42246/0 (socket says 172.21.15.119:42246) 2024-04-25T10:16:47.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbe000640 1 -- 172.21.15.119:0/2422055198 learned_addr learned my addr 172.21.15.119:0/2422055198 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:47.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbea00640 1 --2- 172.21.15.119:0/2422055198 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dc00cc780 0x7f7dc015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbe000640 1 -- 172.21.15.119:0/2422055198 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dc00cc780 msgr2=0x7f7dc015b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbe000640 1 --2- 172.21.15.119:0/2422055198 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dc00cc780 0x7f7dc015b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbf400640 1 --2- 172.21.15.119:0/2422055198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dc015d2d0 0x7f7dc00c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbe000640 1 -- 172.21.15.119:0/2422055198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dc015d2d0 msgr2=0x7f7dc00c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.581+0000 7f7dbe000640 1 --2- 172.21.15.119:0/2422055198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dc015d2d0 0x7f7dc00c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.582+0000 7f7dbe000640 1 -- 172.21.15.119:0/2422055198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc00c50f0 con 0x7f7dc015cc10 2024-04-25T10:16:47.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.582+0000 7f7dbe000640 1 --2- 172.21.15.119:0/2422055198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc015cc10 0x7f7dc015bd50 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f7da800e840 tx=0x7f7da800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.582+0000 7f7db6a00640 1 -- 172.21.15.119:0/2422055198 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8017070 con 0x7f7dc015cc10 2024-04-25T10:16:47.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.582+0000 7f7db6a00640 1 -- 172.21.15.119:0/2422055198 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da800c940 con 0x7f7dc015cc10 2024-04-25T10:16:47.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.582+0000 7f7db6a00640 1 -- 172.21.15.119:0/2422055198 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da800cd10 con 0x7f7dc015cc10 2024-04-25T10:16:47.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.582+0000 7f7dbfe00640 1 -- 172.21.15.119:0/2422055198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc00c53d0 con 0x7f7dc015cc10 2024-04-25T10:16:47.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.582+0000 7f7dbfe00640 1 -- 172.21.15.119:0/2422055198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc006ced0 con 0x7f7dc015cc10 2024-04-25T10:16:47.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.585+0000 7f7db6a00640 1 -- 172.21.15.119:0/2422055198 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7da801a070 con 0x7f7dc015cc10 2024-04-25T10:16:47.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.586+0000 7f7db6a00640 1 --2- 172.21.15.119:0/2422055198 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d9c06b4d0 0x7f7d9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.586+0000 7f7db6a00640 1 -- 172.21.15.119:0/2422055198 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7da808a540 con 0x7f7dc015cc10 2024-04-25T10:16:47.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.586+0000 7f7dbea00640 1 --2- 172.21.15.119:0/2422055198 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d9c06b4d0 0x7f7d9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.586+0000 7f7dc8a62740 1 Processor -- start 2024-04-25T10:16:47.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.586+0000 7f7dc8a62740 1 -- start start 2024-04-25T10:16:47.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.587+0000 7f7dbea00640 1 --2- 172.21.15.119:0/2422055198 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d9c06b4d0 0x7f7d9c06d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7db0034000 tx=0x7f7db002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.587+0000 7f7dc8a62740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edf74b30 0x55c5edf9c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.587+0000 7f7dc8a62740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edf9ceb0 0x55c5edf74f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.587+0000 7f7dc8a62740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edfbd7a0 0x55c5edf732f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.587+0000 7f7dc8a62740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5eded8620 con 0x55c5edf9ceb0 2024-04-25T10:16:47.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.587+0000 7f7dc8a62740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ede81ca0 con 0x55c5edfbd7a0 2024-04-25T10:16:47.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.587+0000 7f7dc8a62740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5eded46c0 con 0x55c5edf74b30 2024-04-25T10:16:47.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.588+0000 7f7dbe000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edf9ceb0 0x55c5edf74f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.588+0000 7f7dbe000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edf9ceb0 0x55c5edf74f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42260/0 (socket says 172.21.15.119:42260) 2024-04-25T10:16:47.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.588+0000 7f7dbe000640 1 -- 172.21.15.119:0/3448710270 learned_addr learned my addr 172.21.15.119:0/3448710270 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:47.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.588+0000 7f7dbe000640 1 -- 172.21.15.119:0/3448710270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edf74b30 msgr2=0x55c5edf9c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.588+0000 7f7dbe000640 1 --2- 172.21.15.119:0/3448710270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edf74b30 0x55c5edf9c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.588+0000 7f7dbe000640 1 -- 172.21.15.119:0/3448710270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edfbd7a0 msgr2=0x55c5edf732f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.588+0000 7f7dbf400640 1 --2- 172.21.15.119:0/3448710270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edfbd7a0 0x55c5edf732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.588+0000 7f7dbe000640 1 --2- 172.21.15.119:0/3448710270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edfbd7a0 0x55c5edf732f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.588+0000 7f7dbe000640 1 -- 172.21.15.119:0/3448710270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5ede88f40 con 0x55c5edf9ceb0 2024-04-25T10:16:47.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.588+0000 7f7dbe000640 1 --2- 172.21.15.119:0/3448710270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edf9ceb0 0x55c5edf74f10 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f7da80588c0 tx=0x7f7da8058520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee80062e7449009 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.589+0000 7f7daf400640 1 -- 172.21.15.119:0/3448710270 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8021050 con 0x55c5edf9ceb0 2024-04-25T10:16:47.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.589+0000 7f7daf400640 1 -- 172.21.15.119:0/3448710270 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da805cd10 con 0x55c5edf9ceb0 2024-04-25T10:16:47.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.589+0000 7f7daf400640 1 -- 172.21.15.119:0/3448710270 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da80612d0 con 0x55c5edf9ceb0 2024-04-25T10:16:47.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.589+0000 7f7dc8a62740 1 -- 172.21.15.119:0/3448710270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edf9ceb0 msgr2=0x55c5edf74f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.589+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/3448710270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edf9ceb0 0x55c5edf74f10 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f7da80588c0 tx=0x7f7da8058520 comp rx=0 tx=0).stop 2024-04-25T10:16:47.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.589+0000 7f7dc8a62740 1 -- 172.21.15.119:0/3448710270 shutdown_connections 2024-04-25T10:16:47.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.589+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/3448710270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edfbd7a0 0x55c5edf732f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.589+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/3448710270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edf9ceb0 0x55c5edf74f10 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.589+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/3448710270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edf74b30 0x55c5edf9c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.589+0000 7f7dc8a62740 1 -- 172.21.15.119:0/3448710270 >> 172.21.15.119:0/3448710270 conn(0x55c5edf71d50 msgr2=0x55c5edfba730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.589+0000 7f7dc8a62740 1 -- 172.21.15.119:0/3448710270 shutdown_connections 2024-04-25T10:16:47.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.589+0000 7f7dc8a62740 1 -- 172.21.15.119:0/3448710270 wait complete. 2024-04-25T10:16:47.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.590+0000 7f7dc8a62740 1 -- start start 2024-04-25T10:16:47.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.590+0000 7f7dc8a62740 1 Processor -- start 2024-04-25T10:16:47.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.591+0000 7f7dc8a62740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edf9ceb0 0x55c5edf99360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.591+0000 7f7dc8a62740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edfbd7a0 0x55c5edf97320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.591+0000 7f7dc8a62740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edfcdcd0 0x55c5edf97860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.591+0000 7f7dc8a62740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5edf9c4e0 con 0x55c5edfbd7a0 2024-04-25T10:16:47.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.591+0000 7f7dc8a62740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ede771f0 con 0x55c5edf9ceb0 2024-04-25T10:16:47.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.591+0000 7f7dc8a62740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5edf9bed0 con 0x55c5edfcdcd0 2024-04-25T10:16:47.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.591+0000 7f7dc8a62740 10 client.0 _ll_register_callbacks cb 0x55c5ede94ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:47.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7dbe000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edfbd7a0 0x55c5edf97320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7dbe000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edfbd7a0 0x55c5edf97320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42272/0 (socket says 172.21.15.119:42272) 2024-04-25T10:16:47.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7dbe000640 1 -- 172.21.15.119:0/3984019924 learned_addr learned my addr 172.21.15.119:0/3984019924 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7dbea00640 1 --2- 172.21.15.119:0/3984019924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edf9ceb0 0x55c5edf99360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7dbf400640 1 --2- 172.21.15.119:0/3984019924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edfcdcd0 0x55c5edf97860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7dbe000640 1 -- 172.21.15.119:0/3984019924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edfcdcd0 msgr2=0x55c5edf97860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7dbe000640 1 --2- 172.21.15.119:0/3984019924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edfcdcd0 0x55c5edf97860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7dbe000640 1 -- 172.21.15.119:0/3984019924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edf9ceb0 msgr2=0x55c5edf99360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7dbe000640 1 --2- 172.21.15.119:0/3984019924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edf9ceb0 0x55c5edf99360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7dbe000640 1 -- 172.21.15.119:0/3984019924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5ede96c50 con 0x55c5edfbd7a0 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7dbea00640 1 --2- 172.21.15.119:0/3984019924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edf9ceb0 0x55c5edf99360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7dbe000640 1 --2- 172.21.15.119:0/3984019924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edfbd7a0 0x55c5edf97320 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f7da80588c0 tx=0x7f7da8009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc14a28a2d3e208 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7da7e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7da7e00640 1 -- 172.21.15.119:0/3984019924 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da808a0b0 con 0x55c5edfbd7a0 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7da7e00640 1 -- 172.21.15.119:0/3984019924 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7da808a250 con 0x55c5edfbd7a0 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.592+0000 7f7dc8a62740 1 -- 172.21.15.119:0/3984019924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7da805cd10 con 0x55c5edfbd7a0 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.593+0000 7f7da7e00640 1 -- 172.21.15.119:0/3984019924 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7da806cb80 con 0x55c5edfbd7a0 2024-04-25T10:16:47.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.593+0000 7f7dc8a62740 10 client.8433 fetch_fsmap learned FSMap version 88 2024-04-25T10:16:47.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.593+0000 7f7dc8a62740 1 -- 172.21.15.119:0/3984019924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55c5edbed2d0 con 0x55c5edfbd7a0 2024-04-25T10:16:47.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.593+0000 7f7da7e00640 1 -- 172.21.15.119:0/3984019924 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da805cd10 con 0x55c5edfbd7a0 2024-04-25T10:16:47.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.593+0000 7f7da7e00640 1 -- 172.21.15.119:0/3984019924 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 88) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7da8058010 con 0x55c5edfbd7a0 2024-04-25T10:16:47.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.593+0000 7f7dc8a62740 10 client.8433 fetch_fsmap finished waiting for FSMap version 88 2024-04-25T10:16:47.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.593+0000 7f7dc8a62740 10 client.8433 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:16:47.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.593+0000 7f7dc8a62740 20 client.8433 populate_metadata read hostname 'smithi119' 2024-04-25T10:16:47.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.593+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/3984019924 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55c5edfce1b0 0x55c5edfce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.593+0000 7f7dc8a62740 4 client.8433 mds_command: new command op to 8367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:47.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.593+0000 7f7dc8a62740 1 -- 172.21.15.119:0/3984019924 --> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5edee5b60 con 0x55c5edfce1b0 2024-04-25T10:16:47.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.594+0000 7f7dbea00640 1 --2- 172.21.15.119:0/3984019924 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55c5edfce1b0 0x55c5edfce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.594+0000 7f7dbea00640 1 --2- 172.21.15.119:0/3984019924 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55c5edfce1b0 0x55c5edfce590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64a5d4e1f3e0ded0 server_cookie=6f92ad674f23d11c in_seq=0 out_seq=0 2024-04-25T10:16:47.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.594+0000 7f7da7e00640 10 client.8433 ms_handle_connect on v2:172.21.15.119:6835/464677621 2024-04-25T10:16:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.595+0000 7f7da7e00640 1 -- 172.21.15.119:0/3984019924 <== mds.0 v2:172.21.15.119:6835/464677621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5edee5b60 con 0x55c5edfce1b0 2024-04-25T10:16:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.595+0000 7f7da7e00640 10 client.8433 handle_command_reply: tid=0 2024-04-25T10:16:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.595+0000 7f7dc8a62740 1 client.8433 shutdown 2024-04-25T10:16:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.595+0000 7f7dc8a62740 1 -- 172.21.15.119:0/3984019924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edfbd7a0 msgr2=0x55c5edf97320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.595+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/3984019924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edfbd7a0 0x55c5edf97320 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f7da80588c0 tx=0x7f7da8009f20 comp rx=0 tx=0).stop 2024-04-25T10:16:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.595+0000 7f7dc8a62740 1 -- 172.21.15.119:0/3984019924 shutdown_connections 2024-04-25T10:16:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.595+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/3984019924 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55c5edfce1b0 0x55c5edfce590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.595+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/3984019924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edfcdcd0 0x55c5edf97860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.595+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/3984019924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edfbd7a0 0x55c5edf97320 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.595+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/3984019924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edf9ceb0 0x55c5edf99360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.595+0000 7f7dc8a62740 1 -- 172.21.15.119:0/3984019924 >> 172.21.15.119:0/3984019924 conn(0x55c5edf74b30 msgr2=0x55c5edfbdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.595+0000 7f7da7e00640 0 client.8433 ms_handle_reset on v2:172.21.15.119:6835/464677621 2024-04-25T10:16:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.596+0000 7f7dc8a62740 1 -- 172.21.15.119:0/3984019924 shutdown_connections 2024-04-25T10:16:47.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.596+0000 7f7dc8a62740 1 -- 172.21.15.119:0/3984019924 wait complete. 2024-04-25T10:16:47.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.596+0000 7f7dc8a62740 20 client.8433 trim_cache size 0 max 16384 2024-04-25T10:16:47.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.613+0000 7f7dc8a62740 1 Processor -- start 2024-04-25T10:16:47.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.613+0000 7f7dc8a62740 1 -- start start 2024-04-25T10:16:47.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dc8a62740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edf9ceb0 0x55c5edf9ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dc8a62740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edfbd7a0 0x55c5edf96e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dc8a62740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edfcdcd0 0x55c5edf973c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dc8a62740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc01ff230 con 0x55c5edfcdcd0 2024-04-25T10:16:47.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dc8a62740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc006cbb0 con 0x55c5edf9ceb0 2024-04-25T10:16:47.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dc8a62740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc01fec90 con 0x55c5edfbd7a0 2024-04-25T10:16:47.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edfcdcd0 0x55c5edf973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edfcdcd0 0x55c5edf973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42288/0 (socket says 172.21.15.119:42288) 2024-04-25T10:16:47.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dbf400640 1 -- 172.21.15.119:0/1208458470 learned_addr learned my addr 172.21.15.119:0/1208458470 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:47.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dbea00640 1 --2- 172.21.15.119:0/1208458470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edf9ceb0 0x55c5edf9ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dbe000640 1 --2- 172.21.15.119:0/1208458470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edfbd7a0 0x55c5edf96e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dbf400640 1 -- 172.21.15.119:0/1208458470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edfbd7a0 msgr2=0x55c5edf96e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dbf400640 1 --2- 172.21.15.119:0/1208458470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edfbd7a0 0x55c5edf96e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dbf400640 1 -- 172.21.15.119:0/1208458470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edf9ceb0 msgr2=0x55c5edf9ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.614+0000 7f7dbf400640 1 --2- 172.21.15.119:0/1208458470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edf9ceb0 0x55c5edf9ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.615+0000 7f7dbf400640 1 -- 172.21.15.119:0/1208458470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5edf6ff80 con 0x55c5edfcdcd0 2024-04-25T10:16:47.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.615+0000 7f7dbea00640 1 --2- 172.21.15.119:0/1208458470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edf9ceb0 0x55c5edf9ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.615+0000 7f7dbf400640 1 --2- 172.21.15.119:0/1208458470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edfcdcd0 0x55c5edf973c0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f7db8001b10 tx=0x7f7db8001fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97bc2b8526d2c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.615+0000 7f7daea00640 1 -- 172.21.15.119:0/1208458470 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db8017070 con 0x55c5edfcdcd0 2024-04-25T10:16:47.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.615+0000 7f7daea00640 1 -- 172.21.15.119:0/1208458470 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db800fc20 con 0x55c5edfcdcd0 2024-04-25T10:16:47.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.615+0000 7f7daea00640 1 -- 172.21.15.119:0/1208458470 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db8013430 con 0x55c5edfcdcd0 2024-04-25T10:16:47.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.616+0000 7f7dc8a62740 1 -- 172.21.15.119:0/1208458470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edfcdcd0 msgr2=0x55c5edf973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.616+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/1208458470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edfcdcd0 0x55c5edf973c0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f7db8001b10 tx=0x7f7db8001fd0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.616+0000 7f7dc8a62740 1 -- 172.21.15.119:0/1208458470 shutdown_connections 2024-04-25T10:16:47.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.616+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/1208458470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edfcdcd0 0x55c5edf973c0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.616+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/1208458470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edfbd7a0 0x55c5edf96e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.616+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/1208458470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edf9ceb0 0x55c5edf9ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.616+0000 7f7dc8a62740 1 -- 172.21.15.119:0/1208458470 >> 172.21.15.119:0/1208458470 conn(0x55c5edf74b30 msgr2=0x55c5edfcd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.616+0000 7f7dc8a62740 1 -- 172.21.15.119:0/1208458470 shutdown_connections 2024-04-25T10:16:47.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.616+0000 7f7dc8a62740 1 -- 172.21.15.119:0/1208458470 wait complete. 2024-04-25T10:16:47.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.617+0000 7f7dc8a62740 1 -- start start 2024-04-25T10:16:47.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.617+0000 7f7dc8a62740 1 Processor -- start 2024-04-25T10:16:47.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dc8a62740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edf9ceb0 0x55c5edf934b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dc8a62740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edfbd7a0 0x55c5edf939f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dc8a62740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edfcdcd0 0x55c5edfd4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dc8a62740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5edf9c4e0 con 0x55c5edf9ceb0 2024-04-25T10:16:47.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dc8a62740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ede771f0 con 0x55c5edfcdcd0 2024-04-25T10:16:47.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dc8a62740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5edf92c50 con 0x55c5edfbd7a0 2024-04-25T10:16:47.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dc8a62740 10 client.0 _ll_register_callbacks cb 0x55c5edf9ce00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:47.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dbea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edf9ceb0 0x55c5edf934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dbea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edf9ceb0 0x55c5edf934b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42300/0 (socket says 172.21.15.119:42300) 2024-04-25T10:16:47.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dbea00640 1 -- 172.21.15.119:0/632282839 learned_addr learned my addr 172.21.15.119:0/632282839 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:47.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dbea00640 1 -- 172.21.15.119:0/632282839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edfbd7a0 msgr2=0x55c5edf939f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dbea00640 1 --2- 172.21.15.119:0/632282839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edfbd7a0 0x55c5edf939f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dbea00640 1 -- 172.21.15.119:0/632282839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edfcdcd0 msgr2=0x55c5edfd4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dbea00640 1 --2- 172.21.15.119:0/632282839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edfcdcd0 0x55c5edfd4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.618+0000 7f7dbea00640 1 -- 172.21.15.119:0/632282839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5edcb76b0 con 0x55c5edf9ceb0 2024-04-25T10:16:47.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.619+0000 7f7dbea00640 1 --2- 172.21.15.119:0/632282839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edf9ceb0 0x55c5edf934b0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f7db0035040 tx=0x7f7db0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3d91e0b773c19d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.619+0000 7f7da7e00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:16:47.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.619+0000 7f7da7e00640 1 -- 172.21.15.119:0/632282839 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db0037040 con 0x55c5edf9ceb0 2024-04-25T10:16:47.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.619+0000 7f7da7e00640 1 -- 172.21.15.119:0/632282839 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7db004a550 con 0x55c5edf9ceb0 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.619+0000 7f7dc8a62740 1 -- 172.21.15.119:0/632282839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7db800fc20 con 0x55c5edf9ceb0 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.619+0000 7f7da7e00640 1 -- 172.21.15.119:0/632282839 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7db004a7a0 con 0x55c5edf9ceb0 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.619+0000 7f7dc8a62740 10 client.8439 fetch_fsmap learned FSMap version 88 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.619+0000 7f7dc8a62740 1 -- 172.21.15.119:0/632282839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55c5ee025120 con 0x55c5edf9ceb0 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.620+0000 7f7da7e00640 1 -- 172.21.15.119:0/632282839 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db800fc20 con 0x55c5edf9ceb0 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.620+0000 7f7da7e00640 1 -- 172.21.15.119:0/632282839 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 88) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7db005a020 con 0x55c5edf9ceb0 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.620+0000 7f7dc8a62740 10 client.8439 fetch_fsmap finished waiting for FSMap version 88 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.620+0000 7f7dc8a62740 10 client.8439 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.620+0000 7f7dc8a62740 20 client.8439 populate_metadata read hostname 'smithi119' 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.620+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/632282839 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55c5edfd4770 0x55c5edf6e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.620+0000 7f7dc8a62740 4 client.8439 mds_command: new command op to 8367 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.620+0000 7f7dc8a62740 1 -- 172.21.15.119:0/632282839 --> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c5edf724b0 con 0x55c5edfd4770 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.620+0000 7f7dbe000640 1 --2- 172.21.15.119:0/632282839 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55c5edfd4770 0x55c5edf6e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7dbe000640 1 --2- 172.21.15.119:0/632282839 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55c5edfd4770 0x55c5edf6e8b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e58389cea7b2259 server_cookie=eaec5340d6ee16e6 in_seq=0 out_seq=0 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7da7e00640 10 client.8439 ms_handle_connect on v2:172.21.15.119:6835/464677621 2024-04-25T10:16:47.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7da7e00640 1 -- 172.21.15.119:0/632282839 <== mds.0 v2:172.21.15.119:6835/464677621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55c5edf724b0 con 0x55c5edfd4770 2024-04-25T10:16:47.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7da7e00640 10 client.8439 handle_command_reply: tid=0 2024-04-25T10:16:47.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7dc8a62740 1 client.8439 shutdown 2024-04-25T10:16:47.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7dc8a62740 1 -- 172.21.15.119:0/632282839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edf9ceb0 msgr2=0x55c5edf934b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/632282839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edf9ceb0 0x55c5edf934b0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f7db0035040 tx=0x7f7db0004730 comp rx=0 tx=0).stop 2024-04-25T10:16:47.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7dc8a62740 1 -- 172.21.15.119:0/632282839 shutdown_connections 2024-04-25T10:16:47.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/632282839 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55c5edfd4770 0x55c5edf6e8b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/632282839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c5edfcdcd0 0x55c5edfd4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/632282839 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55c5edfbd7a0 0x55c5edf939f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7da7e00640 0 client.8439 ms_handle_reset on v2:172.21.15.119:6835/464677621 2024-04-25T10:16:47.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7dc8a62740 1 --2- 172.21.15.119:0/632282839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5edf9ceb0 0x55c5edf934b0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7dc8a62740 1 -- 172.21.15.119:0/632282839 >> 172.21.15.119:0/632282839 conn(0x55c5edf74b30 msgr2=0x55c5edf6b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.621+0000 7f7dc8a62740 1 -- 172.21.15.119:0/632282839 shutdown_connections 2024-04-25T10:16:47.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.622+0000 7f7dc8a62740 1 -- 172.21.15.119:0/632282839 wait complete. 2024-04-25T10:16:47.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.622+0000 7f7dc8a62740 20 client.8439 trim_cache size 0 max 16384 2024-04-25T10:16:47.629 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:16:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "e0b0f428-b441-4237-a4d1-ed53321f87a5", 2024-04-25T10:16:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-04-25T10:16:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 8367, 2024-04-25T10:16:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-04-25T10:16:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-04-25T10:16:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-04-25T10:16:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 4.8824275569999998, 2024-04-25T10:16:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 88, 2024-04-25T10:16:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 97, 2024-04-25T10:16:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 97, 2024-04-25T10:16:47.629 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 5.840660261 2024-04-25T10:16:47.629 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:16:47.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.622+0000 7f7dbfe00640 1 -- 172.21.15.119:0/2422055198 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d9c06b4d0 msgr2=0x7f7d9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.622+0000 7f7dbfe00640 1 --2- 172.21.15.119:0/2422055198 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d9c06b4d0 0x7f7d9c06d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7db0034000 tx=0x7f7db002e000 comp rx=0 tx=0).stop 2024-04-25T10:16:47.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.622+0000 7f7dbfe00640 1 -- 172.21.15.119:0/2422055198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc015cc10 msgr2=0x7f7dc015bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.622+0000 7f7dbfe00640 1 --2- 172.21.15.119:0/2422055198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc015cc10 0x7f7dc015bd50 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f7da800e840 tx=0x7f7da800ed00 comp rx=0 tx=0).stop 2024-04-25T10:16:47.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.622+0000 7f7dbfe00640 1 -- 172.21.15.119:0/2422055198 shutdown_connections 2024-04-25T10:16:47.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.622+0000 7f7dbfe00640 1 --2- 172.21.15.119:0/2422055198 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7d9c06b4d0 0x7f7d9c06d990 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.622+0000 7f7dbfe00640 1 --2- 172.21.15.119:0/2422055198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dc015d2d0 0x7f7dc00c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.622+0000 7f7dbfe00640 1 --2- 172.21.15.119:0/2422055198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc015cc10 0x7f7dc015bd50 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.622+0000 7f7dbfe00640 1 --2- 172.21.15.119:0/2422055198 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dc00cc780 0x7f7dc015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.622+0000 7f7dbfe00640 1 -- 172.21.15.119:0/2422055198 >> 172.21.15.119:0/2422055198 conn(0x7f7dc00b30d0 msgr2=0x7f7dc015f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.622+0000 7f7dbfe00640 1 -- 172.21.15.119:0/2422055198 shutdown_connections 2024-04-25T10:16:47.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.622+0000 7f7dbfe00640 1 -- 172.21.15.119:0/2422055198 wait complete. 2024-04-25T10:16:47.643 DEBUG:teuthology.orchestra.run.smithi119:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:16:47.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:47.648 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T10:16:47.745 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:16:47.741+0000 7f7a1dc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:47.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.827+0000 7fa3a2000640 1 Processor -- start 2024-04-25T10:16:47.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.827+0000 7fa3a2000640 1 -- start start 2024-04-25T10:16:47.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.829+0000 7fa3a2000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa39c161090 0x7fa39c161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.829+0000 7fa3a2000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa39c0b9720 0x7fa39c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.829+0000 7fa3a2000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39c161f80 con 0x7fa39c0cece0 2024-04-25T10:16:47.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.829+0000 7fa3a2000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39c0cf130 con 0x7fa39c161090 2024-04-25T10:16:47.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.829+0000 7fa3a2000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0d21a0 con 0x7fa39c0b9720 2024-04-25T10:16:47.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.829+0000 7fa39b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa39c161090 0x7fa39c161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.829+0000 7fa39b400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa39c161090 0x7fa39c161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42304/0 (socket says 172.21.15.119:42304) 2024-04-25T10:16:47.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.829+0000 7fa39b400640 1 -- 172.21.15.119:0/3094565532 learned_addr learned my addr 172.21.15.119:0/3094565532 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:47.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.829+0000 7fa39be00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa39c0cece0 0x7fa39c06c0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56560/0 (socket says 172.21.15.119:56560) 2024-04-25T10:16:47.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.830+0000 7fa392a00640 1 --2- 172.21.15.119:0/3094565532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa39c0b9720 0x7fa39c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.830+0000 7fa39b400640 1 -- 172.21.15.119:0/3094565532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa39c0b9720 msgr2=0x7fa39c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.830+0000 7fa39b400640 1 --2- 172.21.15.119:0/3094565532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa39c0b9720 0x7fa39c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.830+0000 7fa39b400640 1 -- 172.21.15.119:0/3094565532 >> v1:172.21.15.119:6789/0 conn(0x7fa39c0cece0 legacy=0x7fa39c06c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.830+0000 7fa39b400640 1 -- 172.21.15.119:0/3094565532 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c162260 con 0x7fa39c161090 2024-04-25T10:16:47.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.830+0000 7fa392a00640 1 --2- 172.21.15.119:0/3094565532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa39c0b9720 0x7fa39c0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.831+0000 7fa39b400640 1 --2- 172.21.15.119:0/3094565532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c161090 0x7fa39c161730 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009870 tx=0x7fa38c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dfd1d52f14ab6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.831+0000 7fa39aa00640 1 -- 172.21.15.119:0/3094565532 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c037070 con 0x7fa39c161090 2024-04-25T10:16:47.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.831+0000 7fa39aa00640 1 -- 172.21.15.119:0/3094565532 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c032440 con 0x7fa39c161090 2024-04-25T10:16:47.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.831+0000 7fa39aa00640 1 -- 172.21.15.119:0/3094565532 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c032810 con 0x7fa39c161090 2024-04-25T10:16:47.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.832+0000 7fa3a2000640 1 -- 172.21.15.119:0/3094565532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c161090 msgr2=0x7fa39c161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.832+0000 7fa3a2000640 1 --2- 172.21.15.119:0/3094565532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c161090 0x7fa39c161730 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009870 tx=0x7fa38c02a250 comp rx=0 tx=0).stop 2024-04-25T10:16:47.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.832+0000 7fa3a2000640 1 -- 172.21.15.119:0/3094565532 shutdown_connections 2024-04-25T10:16:47.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.832+0000 7fa3a2000640 1 --2- 172.21.15.119:0/3094565532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa39c0b9720 0x7fa39c0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.832+0000 7fa3a2000640 1 --2- 172.21.15.119:0/3094565532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c161090 0x7fa39c161730 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.832+0000 7fa3a2000640 1 -- 172.21.15.119:0/3094565532 >> 172.21.15.119:0/3094565532 conn(0x7fa39c0b55e0 msgr2=0x7fa39c0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.832+0000 7fa3a2000640 1 -- 172.21.15.119:0/3094565532 shutdown_connections 2024-04-25T10:16:47.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.832+0000 7fa3a2000640 1 -- 172.21.15.119:0/3094565532 wait complete. 2024-04-25T10:16:47.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.834+0000 7fa3a2000640 1 Processor -- start 2024-04-25T10:16:47.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.834+0000 7fa3a2000640 1 -- start start 2024-04-25T10:16:47.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.835+0000 7fa3a2000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa39c0b9720 0x7fa39c15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.835+0000 7fa3a2000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa39c0cece0 0x7fa39c1e9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.835+0000 7fa3a2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c161090 0x7fa39c1f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.835+0000 7fa3a2000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c161470 con 0x7fa39c161090 2024-04-25T10:16:47.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.835+0000 7fa3a2000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0d1e30 con 0x7fa39c0b9720 2024-04-25T10:16:47.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.835+0000 7fa3a2000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c161c70 con 0x7fa39c0cece0 2024-04-25T10:16:47.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.835+0000 7fa39be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c161090 0x7fa39c1f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.835+0000 7fa39be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c161090 0x7fa39c1f7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42316/0 (socket says 172.21.15.119:42316) 2024-04-25T10:16:47.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.835+0000 7fa39be00640 1 -- 172.21.15.119:0/3374270515 learned_addr learned my addr 172.21.15.119:0/3374270515 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:47.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.836+0000 7fa392a00640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa39c0cece0 0x7fa39c1e9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.836+0000 7fa39b400640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa39c0b9720 0x7fa39c15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.836+0000 7fa39be00640 1 -- 172.21.15.119:0/3374270515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa39c0cece0 msgr2=0x7fa39c1e9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.836+0000 7fa39be00640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa39c0cece0 0x7fa39c1e9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.836+0000 7fa39be00640 1 -- 172.21.15.119:0/3374270515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa39c0b9720 msgr2=0x7fa39c15cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.836+0000 7fa39be00640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa39c0b9720 0x7fa39c15cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.836+0000 7fa39be00640 1 -- 172.21.15.119:0/3374270515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c15f670 con 0x7fa39c161090 2024-04-25T10:16:47.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.836+0000 7fa39b400640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa39c0b9720 0x7fa39c15cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.836+0000 7fa39be00640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c161090 0x7fa39c1f7310 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fa39400c400 tx=0x7fa39400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.836+0000 7fa393e00640 1 -- 172.21.15.119:0/3374270515 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3940074e0 con 0x7fa39c161090 2024-04-25T10:16:47.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.836+0000 7fa393e00640 1 -- 172.21.15.119:0/3374270515 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa394007680 con 0x7fa39c161090 2024-04-25T10:16:47.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.836+0000 7fa3a2000640 1 -- 172.21.15.119:0/3374270515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39c15f910 con 0x7fa39c161090 2024-04-25T10:16:47.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.836+0000 7fa393e00640 1 -- 172.21.15.119:0/3374270515 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3940049e0 con 0x7fa39c161090 2024-04-25T10:16:47.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.837+0000 7fa3a2000640 1 -- 172.21.15.119:0/3374270515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39c15fed0 con 0x7fa39c161090 2024-04-25T10:16:47.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.839+0000 7fa393e00640 1 -- 172.21.15.119:0/3374270515 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa39401a020 con 0x7fa39c161090 2024-04-25T10:16:47.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.840+0000 7fa393e00640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa37806b4d0 0x7fa37806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.840+0000 7fa393e00640 1 -- 172.21.15.119:0/3374270515 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa394089a50 con 0x7fa39c161090 2024-04-25T10:16:47.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.840+0000 7fa39b400640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa37806b4d0 0x7fa37806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.841+0000 7fa39b400640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa37806b4d0 0x7fa37806d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa38c034000 tx=0x7fa38c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.841+0000 7fa3a4f94740 1 Processor -- start 2024-04-25T10:16:47.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.841+0000 7fa3a4f94740 1 -- start start 2024-04-25T10:16:47.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.842+0000 7fa3a4f94740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 0x5590cc024580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.842+0000 7fa3a4f94740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 0x5590cc041f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.842+0000 7fa3a4f94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cbff9e80 0x5590cc040100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.842+0000 7fa3a4f94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590cbf5c980 con 0x5590cbff9e80 2024-04-25T10:16:47.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.842+0000 7fa3a4f94740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590cbf05de0 con 0x5590cc0241a0 2024-04-25T10:16:47.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.842+0000 7fa3a4f94740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590cbf58a20 con 0x5590cc041a40 2024-04-25T10:16:47.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.842+0000 7fa39be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cbff9e80 0x5590cc040100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.842+0000 7fa39be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cbff9e80 0x5590cc040100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42322/0 (socket says 172.21.15.119:42322) 2024-04-25T10:16:47.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.842+0000 7fa39be00640 1 -- 172.21.15.119:0/2433262460 learned_addr learned my addr 172.21.15.119:0/2433262460 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:47.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.843+0000 7fa392a00640 1 --2- 172.21.15.119:0/2433262460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 0x5590cc041f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.843+0000 7fa39b400640 1 --2- 172.21.15.119:0/2433262460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 0x5590cc024580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.843+0000 7fa39be00640 1 -- 172.21.15.119:0/2433262460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 msgr2=0x5590cc041f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.843+0000 7fa39be00640 1 --2- 172.21.15.119:0/2433262460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 0x5590cc041f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.843+0000 7fa39be00640 1 -- 172.21.15.119:0/2433262460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 msgr2=0x5590cc024580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.843+0000 7fa39be00640 1 --2- 172.21.15.119:0/2433262460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 0x5590cc024580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.843+0000 7fa39be00640 1 -- 172.21.15.119:0/2433262460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590cbf23e30 con 0x5590cbff9e80 2024-04-25T10:16:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.843+0000 7fa39b400640 1 --2- 172.21.15.119:0/2433262460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 0x5590cc024580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.843+0000 7fa39be00640 1 --2- 172.21.15.119:0/2433262460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cbff9e80 0x5590cc040100 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fa3940070a0 tx=0x7fa3940582b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be61c4950ff2e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.843+0000 7fa38b400640 1 -- 172.21.15.119:0/2433262460 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa394056470 con 0x5590cbff9e80 2024-04-25T10:16:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.843+0000 7fa38b400640 1 -- 172.21.15.119:0/2433262460 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3940547e0 con 0x5590cbff9e80 2024-04-25T10:16:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.843+0000 7fa38b400640 1 -- 172.21.15.119:0/2433262460 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3940585d0 con 0x5590cbff9e80 2024-04-25T10:16:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.844+0000 7fa3a4f94740 1 -- 172.21.15.119:0/2433262460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cbff9e80 msgr2=0x5590cc040100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.844+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/2433262460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cbff9e80 0x5590cc040100 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fa3940070a0 tx=0x7fa3940582b0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.844+0000 7fa3a4f94740 1 -- 172.21.15.119:0/2433262460 shutdown_connections 2024-04-25T10:16:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.844+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/2433262460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cbff9e80 0x5590cc040100 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.844+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/2433262460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 0x5590cc041f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.844+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/2433262460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 0x5590cc024580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.844+0000 7fa3a4f94740 1 -- 172.21.15.119:0/2433262460 >> 172.21.15.119:0/2433262460 conn(0x5590cbff67b0 msgr2=0x5590cc03efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.844+0000 7fa3a4f94740 1 -- 172.21.15.119:0/2433262460 shutdown_connections 2024-04-25T10:16:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.844+0000 7fa3a4f94740 1 -- 172.21.15.119:0/2433262460 wait complete. 2024-04-25T10:16:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.845+0000 7fa3a4f94740 1 -- start start 2024-04-25T10:16:47.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.845+0000 7fa3a4f94740 1 Processor -- start 2024-04-25T10:16:47.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.846+0000 7fa3a4f94740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 0x5590cc0380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.846+0000 7fa3a4f94740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 0x5590cbff32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.846+0000 7fa3a4f94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cbff37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.846+0000 7fa3a4f94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590cbffac20 con 0x5590cc038620 2024-04-25T10:16:47.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.846+0000 7fa3a4f94740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590cbefb3c0 con 0x5590cc0241a0 2024-04-25T10:16:47.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.846+0000 7fa3a4f94740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590cbff46e0 con 0x5590cc041a40 2024-04-25T10:16:47.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.846+0000 7fa3a4f94740 10 client.0 _ll_register_callbacks cb 0x5590cbc70c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:47.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.847+0000 7fa39be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cbff37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.847+0000 7fa39be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cbff37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42332/0 (socket says 172.21.15.119:42332) 2024-04-25T10:16:47.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.847+0000 7fa39be00640 1 -- 172.21.15.119:0/3258432625 learned_addr learned my addr 172.21.15.119:0/3258432625 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:47.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.847+0000 7fa392a00640 1 --2- 172.21.15.119:0/3258432625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 0x5590cbff32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.847+0000 7fa39b400640 1 --2- 172.21.15.119:0/3258432625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 0x5590cc0380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.847+0000 7fa39be00640 1 -- 172.21.15.119:0/3258432625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 msgr2=0x5590cbff32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.847+0000 7fa39be00640 1 --2- 172.21.15.119:0/3258432625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 0x5590cbff32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.847+0000 7fa39be00640 1 -- 172.21.15.119:0/3258432625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 msgr2=0x5590cc0380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.847+0000 7fa39be00640 1 --2- 172.21.15.119:0/3258432625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 0x5590cc0380e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.847+0000 7fa39be00640 1 -- 172.21.15.119:0/3258432625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590cbd397e0 con 0x5590cc038620 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.847+0000 7fa392a00640 1 --2- 172.21.15.119:0/3258432625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 0x5590cbff32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.847+0000 7fa39be00640 1 --2- 172.21.15.119:0/3258432625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cbff37e0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fa394057e70 tx=0x7fa39401e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc2a8058a416f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa37fe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa37fe00640 1 -- 172.21.15.119:0/3258432625 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa394074060 con 0x5590cc038620 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3258432625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3940547e0 con 0x5590cc038620 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa37fe00640 1 -- 172.21.15.119:0/3258432625 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa3940605f0 con 0x5590cc038620 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa37fe00640 1 -- 172.21.15.119:0/3258432625 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa394004b80 con 0x5590cc038620 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa3a4f94740 10 client.8454 fetch_fsmap learned FSMap version 88 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3258432625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x5590cbd99f70 con 0x5590cc038620 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa37fe00640 1 -- 172.21.15.119:0/3258432625 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3940547e0 con 0x5590cc038620 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa37fe00640 1 -- 172.21.15.119:0/3258432625 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 88) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa39405e280 con 0x5590cc038620 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa3a4f94740 10 client.8454 fetch_fsmap finished waiting for FSMap version 88 2024-04-25T10:16:47.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa3a4f94740 10 client.8454 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:16:47.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa3a4f94740 20 client.8454 populate_metadata read hostname 'smithi119' 2024-04-25T10:16:47.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/3258432625 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x5590cc026840 0x5590cc026c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa3a4f94740 4 client.8454 mds_command: new command op to 8367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:47.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.848+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3258432625 --> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590cbf69ec0 con 0x5590cc026840 2024-04-25T10:16:47.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.849+0000 7fa39b400640 1 --2- 172.21.15.119:0/3258432625 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x5590cc026840 0x5590cc026c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.850+0000 7fa37fe00640 10 client.8454 ms_handle_connect on v2:172.21.15.119:6835/464677621 2024-04-25T10:16:47.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.850+0000 7fa39b400640 1 --2- 172.21.15.119:0/3258432625 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x5590cc026840 0x5590cc026c20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4dd4202e0c0191d server_cookie=532dce50fbd27255 in_seq=0 out_seq=0 2024-04-25T10:16:47.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.850+0000 7fa403200640 1 Processor -- start 2024-04-25T10:16:47.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.850+0000 7fa403200640 1 -- start start 2024-04-25T10:16:47.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.850+0000 7fa37fe00640 1 -- 172.21.15.119:0/3258432625 <== mds.0 v2:172.21.15.119:6835/464677621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5590cbf69ec0 con 0x5590cc026840 2024-04-25T10:16:47.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.850+0000 7fa37fe00640 10 client.8454 handle_command_reply: tid=0 2024-04-25T10:16:47.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.850+0000 7fa3a4f94740 1 client.8454 shutdown 2024-04-25T10:16:47.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3258432625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 msgr2=0x5590cbff37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/3258432625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cbff37e0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fa394057e70 tx=0x7fa39401e070 comp rx=0 tx=0).stop 2024-04-25T10:16:47.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3258432625 shutdown_connections 2024-04-25T10:16:47.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/3258432625 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x5590cc026840 0x5590cc026c20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/3258432625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cbff37e0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/3258432625 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 0x5590cbff32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/3258432625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 0x5590cc0380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa37fe00640 0 client.8454 ms_handle_reset on v2:172.21.15.119:6835/464677621 2024-04-25T10:16:47.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3258432625 >> 172.21.15.119:0/3258432625 conn(0x5590cbff9e80 msgr2=0x5590cc042530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3258432625 shutdown_connections 2024-04-25T10:16:47.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3258432625 wait complete. 2024-04-25T10:16:47.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa3a4f94740 20 client.8454 trim_cache size 0 max 16384 2024-04-25T10:16:47.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa403200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa4040d2d10 0x7fa4040b9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa403200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4040b6a10 0x7fa4040b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa403200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4040d1990 con 0x7fa4040b5360 2024-04-25T10:16:47.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.851+0000 7fa403200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4040d1c70 con 0x7fa4040d2d10 2024-04-25T10:16:47.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.852+0000 7fa403200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040d1f50 con 0x7fa4040b6a10 2024-04-25T10:16:47.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.852+0000 7fa401e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa4040b5360 0x7fa4040b5740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56570/0 (socket says 172.21.15.119:56570) 2024-04-25T10:16:47.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.852+0000 7fa401e00640 1 -- 172.21.15.119:0/3864434157 learned_addr learned my addr 172.21.15.119:0/3864434157 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:16:47.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.852+0000 7fa401400640 1 --2- 172.21.15.119:0/3864434157 >> v2:172.21.15.119:3300/0 conn(0x7fa4040d2d10 0x7fa4040b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.852+0000 7fa402800640 1 --2- 172.21.15.119:0/3864434157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4040b6a10 0x7fa4040b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.852+0000 7fa400a00640 1 -- 172.21.15.119:0/3864434157 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874341793 0 0) 0x7fa4040d1990 con 0x7fa4040b5360 2024-04-25T10:16:47.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.852+0000 7fa400a00640 1 -- 172.21.15.119:0/3864434157 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3e8003520 con 0x7fa4040b5360 2024-04-25T10:16:47.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.852+0000 7fa401400640 1 -- 172.21.15.119:0/3864434157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4040b6a10 msgr2=0x7fa4040b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.852+0000 7fa401400640 1 --2- 172.21.15.119:0/3864434157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4040b6a10 0x7fa4040b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.852+0000 7fa401400640 1 -- 172.21.15.119:0/3864434157 >> v1:172.21.15.119:6789/0 conn(0x7fa4040b5360 legacy=0x7fa4040b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.852+0000 7fa401400640 1 -- 172.21.15.119:0/3864434157 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4040d2230 con 0x7fa4040d2d10 2024-04-25T10:16:47.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.853+0000 7fa401400640 1 --2- 172.21.15.119:0/3864434157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4040d2d10 0x7fa4040b9200 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00097a0 tx=0x7fa3f002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f01a83e0fd0283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.854+0000 7fa400a00640 1 -- 172.21.15.119:0/3864434157 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0037070 con 0x7fa4040d2d10 2024-04-25T10:16:47.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.854+0000 7fa400a00640 1 -- 172.21.15.119:0/3864434157 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f0032440 con 0x7fa4040d2d10 2024-04-25T10:16:47.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.854+0000 7fa400a00640 1 -- 172.21.15.119:0/3864434157 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0032810 con 0x7fa4040d2d10 2024-04-25T10:16:47.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.854+0000 7fa403200640 1 -- 172.21.15.119:0/3864434157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4040d2d10 msgr2=0x7fa4040b9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.854+0000 7fa403200640 1 --2- 172.21.15.119:0/3864434157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4040d2d10 0x7fa4040b9200 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00097a0 tx=0x7fa3f002a180 comp rx=0 tx=0).stop 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.854+0000 7fa403200640 1 -- 172.21.15.119:0/3864434157 shutdown_connections 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.855+0000 7fa403200640 1 --2- 172.21.15.119:0/3864434157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4040b6a10 0x7fa4040b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.855+0000 7fa403200640 1 --2- 172.21.15.119:0/3864434157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4040d2d10 0x7fa4040b9200 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.855+0000 7fa403200640 1 -- 172.21.15.119:0/3864434157 >> 172.21.15.119:0/3864434157 conn(0x7fa4040726c0 msgr2=0x7fa404072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.855+0000 7fa403200640 1 -- 172.21.15.119:0/3864434157 shutdown_connections 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.855+0000 7fa403200640 1 -- 172.21.15.119:0/3864434157 wait complete. 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.856+0000 7fa403200640 1 Processor -- start 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.856+0000 7fa403200640 1 -- start start 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.857+0000 7fa403200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4040b5360 0x7fa40415b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.857+0000 7fa403200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4040b6a10 0x7fa40414b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.857+0000 7fa403200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4040d2d10 0x7fa4041568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.857+0000 7fa403200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040d27f0 con 0x7fa4040d2d10 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.857+0000 7fa403200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040d2510 con 0x7fa4040b6a10 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.857+0000 7fa403200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040b6130 con 0x7fa4040b5360 2024-04-25T10:16:47.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.858+0000 7fa402800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4040d2d10 0x7fa4041568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.858+0000 7fa402800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4040d2d10 0x7fa4041568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42358/0 (socket says 172.21.15.119:42358) 2024-04-25T10:16:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.858+0000 7fa402800640 1 -- 172.21.15.119:0/2602302363 learned_addr learned my addr 172.21.15.119:0/2602302363 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.858+0000 7fa401400640 1 --2- 172.21.15.119:0/2602302363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4040b6a10 0x7fa40414b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.858+0000 7fa401e00640 1 --2- 172.21.15.119:0/2602302363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4040b5360 0x7fa40415b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.858+0000 7fa402800640 1 -- 172.21.15.119:0/2602302363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4040b5360 msgr2=0x7fa40415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.858+0000 7fa402800640 1 --2- 172.21.15.119:0/2602302363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4040b5360 0x7fa40415b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.858+0000 7fa402800640 1 -- 172.21.15.119:0/2602302363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4040b6a10 msgr2=0x7fa40414b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.858+0000 7fa402800640 1 --2- 172.21.15.119:0/2602302363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4040b6a10 0x7fa40414b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.858+0000 7fa402800640 1 -- 172.21.15.119:0/2602302363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40415a1d0 con 0x7fa4040d2d10 2024-04-25T10:16:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.858+0000 7fa401e00640 1 --2- 172.21.15.119:0/2602302363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4040b5360 0x7fa40415b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.858+0000 7fa401400640 1 --2- 172.21.15.119:0/2602302363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4040b6a10 0x7fa40414b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:47.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.858+0000 7fa402800640 1 --2- 172.21.15.119:0/2602302363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4040d2d10 0x7fa4041568e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc007e60 tx=0x7fa3fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.859+0000 7fa3fa000640 1 -- 172.21.15.119:0/2602302363 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc014420 con 0x7fa4040d2d10 2024-04-25T10:16:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.859+0000 7fa3fa000640 1 -- 172.21.15.119:0/2602302363 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc019070 con 0x7fa4040d2d10 2024-04-25T10:16:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.859+0000 7fa403200640 1 -- 172.21.15.119:0/2602302363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40415a4b0 con 0x7fa4040d2d10 2024-04-25T10:16:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.859+0000 7fa3fa000640 1 -- 172.21.15.119:0/2602302363 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc014820 con 0x7fa4040d2d10 2024-04-25T10:16:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.859+0000 7fa403200640 1 -- 172.21.15.119:0/2602302363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40415aa70 con 0x7fa4040d2d10 2024-04-25T10:16:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.860+0000 7fa403200640 1 -- 172.21.15.119:0/2602302363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fa4040749b0 con 0x7fa4040d2d10 2024-04-25T10:16:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.861+0000 7fa3fa000640 1 -- 172.21.15.119:0/2602302363 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3fc0149c0 con 0x7fa4040d2d10 2024-04-25T10:16:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.862+0000 7fa3fa000640 1 --2- 172.21.15.119:0/2602302363 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3e406b4d0 0x7fa3e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.862+0000 7fa3fa000640 1 -- 172.21.15.119:0/2602302363 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa3fc08aad0 con 0x7fa4040d2d10 2024-04-25T10:16:47.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.862+0000 7fa3fa000640 1 -- 172.21.15.119:0/2602302363 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+514 (secure 0 0 0) 0x7fa3fc08f220 con 0x7fa4040d2d10 2024-04-25T10:16:47.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.862+0000 7fa401e00640 1 --2- 172.21.15.119:0/2602302363 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3e406b4d0 0x7fa3e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.862+0000 7fa403200640 1 -- 172.21.15.119:0/2602302363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fa4040b86b0 con 0x7fa4040d2d10 2024-04-25T10:16:47.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.863+0000 7fa401e00640 1 --2- 172.21.15.119:0/2602302363 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3e406b4d0 0x7fa3e406d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4005880 tx=0x7fa3f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.870+0000 7fa3a4f94740 1 Processor -- start 2024-04-25T10:16:47.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.870+0000 7fa3a4f94740 1 -- start start 2024-04-25T10:16:47.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa3a4f94740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 0x5590cc04e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa3a4f94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cc04edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa3a4f94740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 0x5590cc027a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa3a4f94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c1fff20 con 0x5590cc038620 2024-04-25T10:16:47.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa3a4f94740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c1ff0c0 con 0x5590cc0241a0 2024-04-25T10:16:47.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa3a4f94740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c1fe4c0 con 0x5590cc041a40 2024-04-25T10:16:47.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa392a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cc04edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa392a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cc04edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42374/0 (socket says 172.21.15.119:42374) 2024-04-25T10:16:47.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa392a00640 1 -- 172.21.15.119:0/375156012 learned_addr learned my addr 172.21.15.119:0/375156012 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:47.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa39b400640 1 --2- 172.21.15.119:0/375156012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 0x5590cc04e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa39be00640 1 --2- 172.21.15.119:0/375156012 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 0x5590cc027a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa392a00640 1 -- 172.21.15.119:0/375156012 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 msgr2=0x5590cc027a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa392a00640 1 --2- 172.21.15.119:0/375156012 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 0x5590cc027a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa392a00640 1 -- 172.21.15.119:0/375156012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 msgr2=0x5590cc04e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa392a00640 1 --2- 172.21.15.119:0/375156012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 0x5590cc04e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.871+0000 7fa392a00640 1 -- 172.21.15.119:0/375156012 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590cc038450 con 0x5590cc038620 2024-04-25T10:16:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.872+0000 7fa392a00640 1 --2- 172.21.15.119:0/375156012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cc04edf0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa39c159c80 tx=0x7fa384007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2166d8dcc8b42acc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.872+0000 7fa38aa00640 1 -- 172.21.15.119:0/375156012 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38401d070 con 0x5590cc038620 2024-04-25T10:16:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.872+0000 7fa38aa00640 1 -- 172.21.15.119:0/375156012 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa384003da0 con 0x5590cc038620 2024-04-25T10:16:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.872+0000 7fa38aa00640 1 -- 172.21.15.119:0/375156012 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa384004230 con 0x5590cc038620 2024-04-25T10:16:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.873+0000 7fa3a4f94740 1 -- 172.21.15.119:0/375156012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 msgr2=0x5590cc04edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.873+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/375156012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cc04edf0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa39c159c80 tx=0x7fa384007f90 comp rx=0 tx=0).stop 2024-04-25T10:16:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.873+0000 7fa3a4f94740 1 -- 172.21.15.119:0/375156012 shutdown_connections 2024-04-25T10:16:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.873+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/375156012 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc041a40 0x5590cc027a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.873+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/375156012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cc04edf0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.873+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/375156012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc0241a0 0x5590cc04e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.873+0000 7fa3a4f94740 1 -- 172.21.15.119:0/375156012 >> 172.21.15.119:0/375156012 conn(0x5590cbff9e80 msgr2=0x5590cbfef7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.873+0000 7fa3a4f94740 1 -- 172.21.15.119:0/375156012 shutdown_connections 2024-04-25T10:16:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.873+0000 7fa3a4f94740 1 -- 172.21.15.119:0/375156012 wait complete. 2024-04-25T10:16:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.874+0000 7fa3a4f94740 1 -- start start 2024-04-25T10:16:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.874+0000 7fa3a4f94740 1 Processor -- start 2024-04-25T10:16:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa3a4f94740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc0241a0 0x5590cc035d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa3a4f94740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cc0362a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa3a4f94740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc041a40 0x5590cc04d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa3a4f94740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c1feac0 con 0x5590cc038620 2024-04-25T10:16:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa3a4f94740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590cbf69ec0 con 0x5590cc041a40 2024-04-25T10:16:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa3a4f94740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590cc04e3e0 con 0x5590cc0241a0 2024-04-25T10:16:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa3a4f94740 10 client.0 _ll_register_callbacks cb 0x5590cbff4bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa392a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cc0362a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa392a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cc0362a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42388/0 (socket says 172.21.15.119:42388) 2024-04-25T10:16:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa392a00640 1 -- 172.21.15.119:0/3741322920 learned_addr learned my addr 172.21.15.119:0/3741322920 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa39be00640 1 --2- 172.21.15.119:0/3741322920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc041a40 0x5590cc04d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa392a00640 1 -- 172.21.15.119:0/3741322920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc0241a0 msgr2=0x5590cc035d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa392a00640 1 --2- 172.21.15.119:0/3741322920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc0241a0 0x5590cc035d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa392a00640 1 -- 172.21.15.119:0/3741322920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc041a40 msgr2=0x5590cc04d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa392a00640 1 --2- 172.21.15.119:0/3741322920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc041a40 0x5590cc04d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa392a00640 1 -- 172.21.15.119:0/3741322920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590cc026f90 con 0x5590cc038620 2024-04-25T10:16:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.875+0000 7fa39be00640 1 --2- 172.21.15.119:0/3741322920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc041a40 0x5590cc04d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.876+0000 7fa392a00640 1 --2- 172.21.15.119:0/3741322920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cc0362a0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fa38400efd0 tx=0x7fa384004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83875561c5d1e524 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.876+0000 7fa37fe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:16:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.876+0000 7fa37fe00640 1 -- 172.21.15.119:0/3741322920 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38401d070 con 0x5590cc038620 2024-04-25T10:16:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.876+0000 7fa37fe00640 1 -- 172.21.15.119:0/3741322920 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa384024480 con 0x5590cc038620 2024-04-25T10:16:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.876+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3741322920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa384003da0 con 0x5590cc038620 2024-04-25T10:16:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.876+0000 7fa37fe00640 1 -- 172.21.15.119:0/3741322920 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa384031050 con 0x5590cc038620 2024-04-25T10:16:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.876+0000 7fa3a4f94740 10 client.8469 fetch_fsmap learned FSMap version 88 2024-04-25T10:16:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.876+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3741322920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x5590cbc31170 con 0x5590cc038620 2024-04-25T10:16:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.877+0000 7fa37fe00640 1 -- 172.21.15.119:0/3741322920 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa384003da0 con 0x5590cc038620 2024-04-25T10:16:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.877+0000 7fa37fe00640 1 -- 172.21.15.119:0/3741322920 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 88) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa384034020 con 0x5590cc038620 2024-04-25T10:16:47.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.877+0000 7fa3a4f94740 10 client.8469 fetch_fsmap finished waiting for FSMap version 88 2024-04-25T10:16:47.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.877+0000 7fa3a4f94740 10 client.8469 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:16:47.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.877+0000 7fa3a4f94740 20 client.8469 populate_metadata read hostname 'smithi119' 2024-04-25T10:16:47.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.877+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/3741322920 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x5590cc035630 0x5590cc04da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.877+0000 7fa3a4f94740 4 client.8469 mds_command: new command op to 8367 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:16:47.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.877+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3741322920 --> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x5590cc039b00 con 0x5590cc035630 2024-04-25T10:16:47.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.877+0000 7fa39b400640 1 --2- 172.21.15.119:0/3741322920 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x5590cc035630 0x5590cc04da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.877+0000 7fa39b400640 1 --2- 172.21.15.119:0/3741322920 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x5590cc035630 0x5590cc04da90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8890b3ac85c78cfd server_cookie=a4a25ee42d80cff8 in_seq=0 out_seq=0 2024-04-25T10:16:47.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.877+0000 7fa37fe00640 10 client.8469 ms_handle_connect on v2:172.21.15.119:6835/464677621 2024-04-25T10:16:47.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.878+0000 7fa37fe00640 1 -- 172.21.15.119:0/3741322920 <== mds.0 v2:172.21.15.119:6835/464677621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5590cc039b00 con 0x5590cc035630 2024-04-25T10:16:47.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.878+0000 7fa37fe00640 10 client.8469 handle_command_reply: tid=0 2024-04-25T10:16:47.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.878+0000 7fa3a4f94740 1 client.8469 shutdown 2024-04-25T10:16:47.884 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:16:47.879+0000 7f7a18c00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:16:47.884 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:16:47.879+0000 7f7a18c00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T10:16:47.884 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:16:47.879+0000 7f7a18c00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:16:47.884 INFO:tasks.ceph.mds.a.smithi119.stderr:2024-04-25T10:16:47.879+0000 7f7a18c00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T10:16:47.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.879+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3741322920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 msgr2=0x5590cc0362a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.879+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/3741322920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cc0362a0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fa38400efd0 tx=0x7fa384004ab0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.879+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3741322920 shutdown_connections 2024-04-25T10:16:47.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.879+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/3741322920 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x5590cc035630 0x5590cc04da90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.879+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/3741322920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590cc041a40 0x5590cc04d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.879+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/3741322920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590cc038620 0x5590cc0362a0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.879+0000 7fa3a4f94740 1 --2- 172.21.15.119:0/3741322920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5590cc0241a0 0x5590cc035d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.879+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3741322920 >> 172.21.15.119:0/3741322920 conn(0x5590cbff9e80 msgr2=0x5590cc0279f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.879+0000 7fa37fe00640 0 client.8469 ms_handle_reset on v2:172.21.15.119:6835/464677621 2024-04-25T10:16:47.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.879+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3741322920 shutdown_connections 2024-04-25T10:16:47.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.879+0000 7fa3a4f94740 1 -- 172.21.15.119:0/3741322920 wait complete. 2024-04-25T10:16:47.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.880+0000 7fa3a4f94740 20 client.8469 trim_cache size 0 max 16384 2024-04-25T10:16:47.886 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:16:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0, 2024-04-25T10:16:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_tag": "bbb77ef1-e741-4ea2-b4b5-5d82f89ac394", 2024-04-25T10:16:47.887 INFO:teuthology.orchestra.run.smithi119.stdout: "mode": "asynchronous" 2024-04-25T10:16:47.887 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:16:47.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.880+0000 7fa3a2000640 1 -- 172.21.15.119:0/3374270515 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa37806b4d0 msgr2=0x7fa37806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.880+0000 7fa3a2000640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa37806b4d0 0x7fa37806d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa38c034000 tx=0x7fa38c02e000 comp rx=0 tx=0).stop 2024-04-25T10:16:47.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.880+0000 7fa3a2000640 1 -- 172.21.15.119:0/3374270515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c161090 msgr2=0x7fa39c1f7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.880+0000 7fa3a2000640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c161090 0x7fa39c1f7310 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fa39400c400 tx=0x7fa39400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.881+0000 7fa3a2000640 1 -- 172.21.15.119:0/3374270515 shutdown_connections 2024-04-25T10:16:47.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.881+0000 7fa3a2000640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa37806b4d0 0x7fa37806d990 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.881+0000 7fa3a2000640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa39c161090 0x7fa39c1f7310 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.881+0000 7fa3a2000640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa39c0cece0 0x7fa39c1e9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.881+0000 7fa3a2000640 1 --2- 172.21.15.119:0/3374270515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa39c0b9720 0x7fa39c15cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.881+0000 7fa3a2000640 1 -- 172.21.15.119:0/3374270515 >> 172.21.15.119:0/3374270515 conn(0x7fa39c0b55e0 msgr2=0x7fa39c0d3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.881+0000 7fa3a2000640 1 -- 172.21.15.119:0/3374270515 shutdown_connections 2024-04-25T10:16:47.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:47.881+0000 7fa3a2000640 1 -- 172.21.15.119:0/3374270515 wait complete. 2024-04-25T10:16:47.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:47.901 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:16:48.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.080+0000 7fc8c7400640 1 Processor -- start 2024-04-25T10:16:48.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.080+0000 7fc8c7400640 1 -- start start 2024-04-25T10:16:48.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.082+0000 7fc8c7400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc8c00aed40 0x7fc8c00cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.082+0000 7fc8c7400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00cfa90 0x7fc8c00acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.082+0000 7fc8c7400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c015e360 con 0x7fc8c015b720 2024-04-25T10:16:48.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.082+0000 7fc8c7400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c015e5d0 con 0x7fc8c00aed40 2024-04-25T10:16:48.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.082+0000 7fc8c7400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c015e8b0 con 0x7fc8c00cfa90 2024-04-25T10:16:48.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.082+0000 7fc8b7e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc8c00aed40 0x7fc8c00cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.082+0000 7fc8b7e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc8c00aed40 0x7fc8c00cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42392/0 (socket says 172.21.15.119:42392) 2024-04-25T10:16:48.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.082+0000 7fc8b7e00640 1 -- 172.21.15.119:0/4189587546 learned_addr learned my addr 172.21.15.119:0/4189587546 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:48.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.083+0000 7fc8c5400640 1 --2- 172.21.15.119:0/4189587546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00cfa90 0x7fc8c00acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.083+0000 7fc8b7e00640 1 -- 172.21.15.119:0/4189587546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00cfa90 msgr2=0x7fc8c00acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.083+0000 7fc8b7e00640 1 --2- 172.21.15.119:0/4189587546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00cfa90 0x7fc8c00acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.083+0000 7fc8b7e00640 1 -- 172.21.15.119:0/4189587546 >> v1:172.21.15.132:6789/0 conn(0x7fc8c015b720 legacy=0x7fc8c00ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.083+0000 7fc8b7e00640 1 -- 172.21.15.119:0/4189587546 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c015eb90 con 0x7fc8c00aed40 2024-04-25T10:16:48.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.083+0000 7fc8c5400640 1 --2- 172.21.15.119:0/4189587546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00cfa90 0x7fc8c00acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.084+0000 7fc8b7e00640 1 --2- 172.21.15.119:0/4189587546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c00aed40 0x7fc8c00cf550 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009870 tx=0x7fc8a802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ab27cc2cd9f034 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.084+0000 7fc8bfe00640 1 -- 172.21.15.119:0/4189587546 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8037070 con 0x7fc8c00aed40 2024-04-25T10:16:48.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.084+0000 7fc8bfe00640 1 -- 172.21.15.119:0/4189587546 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a8032440 con 0x7fc8c00aed40 2024-04-25T10:16:48.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.084+0000 7fc8bfe00640 1 -- 172.21.15.119:0/4189587546 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8032810 con 0x7fc8c00aed40 2024-04-25T10:16:48.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.085+0000 7fc8c7400640 1 -- 172.21.15.119:0/4189587546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c00aed40 msgr2=0x7fc8c00cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.085+0000 7fc8c7400640 1 --2- 172.21.15.119:0/4189587546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c00aed40 0x7fc8c00cf550 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009870 tx=0x7fc8a802a210 comp rx=0 tx=0).stop 2024-04-25T10:16:48.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.085+0000 7fc8c7400640 1 -- 172.21.15.119:0/4189587546 shutdown_connections 2024-04-25T10:16:48.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.085+0000 7fc8c7400640 1 --2- 172.21.15.119:0/4189587546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00cfa90 0x7fc8c00acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.085+0000 7fc8c7400640 1 --2- 172.21.15.119:0/4189587546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c00aed40 0x7fc8c00cf550 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.085+0000 7fc8c7400640 1 -- 172.21.15.119:0/4189587546 >> 172.21.15.119:0/4189587546 conn(0x7fc8c006aba0 msgr2=0x7fc8c006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.085+0000 7fc8c7400640 1 -- 172.21.15.119:0/4189587546 shutdown_connections 2024-04-25T10:16:48.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.085+0000 7fc8c7400640 1 -- 172.21.15.119:0/4189587546 wait complete. 2024-04-25T10:16:48.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.087+0000 7fc8c7400640 1 Processor -- start 2024-04-25T10:16:48.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.087+0000 7fc8c7400640 1 -- start start 2024-04-25T10:16:48.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.088+0000 7fc8c7400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c00aed40 0x7fc8c014de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.088+0000 7fc8c7400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00cfa90 0x7fc8c01549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.088+0000 7fc8c7400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c015b720 0x7fc8c00cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.088+0000 7fc8c7400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00cfed0 con 0x7fc8c00aed40 2024-04-25T10:16:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.088+0000 7fc8c7400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c015bb00 con 0x7fc8c015b720 2024-04-25T10:16:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.088+0000 7fc8c7400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c00d0050 con 0x7fc8c00cfa90 2024-04-25T10:16:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.089+0000 7fc8c4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c00aed40 0x7fc8c014de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.089+0000 7fc8c4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c00aed40 0x7fc8c014de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42404/0 (socket says 172.21.15.119:42404) 2024-04-25T10:16:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.089+0000 7fc8c4a00640 1 -- 172.21.15.119:0/2776293974 learned_addr learned my addr 172.21.15.119:0/2776293974 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.089+0000 7fc8b7e00640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00cfa90 0x7fc8c01549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.089+0000 7fc8c5400640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c015b720 0x7fc8c00cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.089+0000 7fc8c4a00640 1 -- 172.21.15.119:0/2776293974 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00cfa90 msgr2=0x7fc8c01549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.089+0000 7fc8c4a00640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00cfa90 0x7fc8c01549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.089+0000 7fc8c4a00640 1 -- 172.21.15.119:0/2776293974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c015b720 msgr2=0x7fc8c00cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.089+0000 7fc8c4a00640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c015b720 0x7fc8c00cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.089+0000 7fc8c4a00640 1 -- 172.21.15.119:0/2776293974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c00b04a0 con 0x7fc8c00aed40 2024-04-25T10:16:48.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.089+0000 7fc8c5400640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c015b720 0x7fc8c00cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.090+0000 7fc8c4a00640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c00aed40 0x7fc8c014de00 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000b420 tx=0x7fc8b000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.090+0000 7fc8bd600640 1 -- 172.21.15.119:0/2776293974 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0011830 con 0x7fc8c00aed40 2024-04-25T10:16:48.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.090+0000 7fc8bd600640 1 -- 172.21.15.119:0/2776293974 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b0016070 con 0x7fc8c00aed40 2024-04-25T10:16:48.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.090+0000 7fc8c7400640 1 -- 172.21.15.119:0/2776293974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c00b0780 con 0x7fc8c00aed40 2024-04-25T10:16:48.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.090+0000 7fc8bd600640 1 -- 172.21.15.119:0/2776293974 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0011c10 con 0x7fc8c00aed40 2024-04-25T10:16:48.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.090+0000 7fc8c7400640 1 -- 172.21.15.119:0/2776293974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c00b0d40 con 0x7fc8c00aed40 2024-04-25T10:16:48.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.093+0000 7fc8bd600640 1 -- 172.21.15.119:0/2776293974 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8b001b460 con 0x7fc8c00aed40 2024-04-25T10:16:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.093+0000 7fc8bd600640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc89c06b500 0x7fc89c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.094+0000 7fc8bd600640 1 -- 172.21.15.119:0/2776293974 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc8b0089e00 con 0x7fc8c00aed40 2024-04-25T10:16:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.094+0000 7fc8b7e00640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc89c06b500 0x7fc89c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.094+0000 7fc8ca3c0740 1 Processor -- start 2024-04-25T10:16:48.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.094+0000 7fc8ca3c0740 1 -- start start 2024-04-25T10:16:48.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.095+0000 7fc8b7e00640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc89c06b500 0x7fc89c06d9c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8034000 tx=0x7fc8a802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.095+0000 7fc8ca3c0740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b562bb0 0x55942b564d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.095+0000 7fc8ca3c0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b59f280 0x55942b562f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.095+0000 7fc8ca3c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b58daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.095+0000 7fc8ca3c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942b4c6c70 con 0x55942b58d670 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.095+0000 7fc8ca3c0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942b46fcf0 con 0x55942b59f280 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.095+0000 7fc8ca3c0740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55942b4c2d10 con 0x55942b562bb0 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.095+0000 7fc8c5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b58daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.095+0000 7fc8c5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b58daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42408/0 (socket says 172.21.15.119:42408) 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.095+0000 7fc8c5400640 1 -- 172.21.15.119:0/3966788707 learned_addr learned my addr 172.21.15.119:0/3966788707 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.096+0000 7fc8c4a00640 1 --2- 172.21.15.119:0/3966788707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b562bb0 0x55942b564d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.096+0000 7fc8c5400640 1 -- 172.21.15.119:0/3966788707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b562bb0 msgr2=0x55942b564d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.096+0000 7fc8c5400640 1 --2- 172.21.15.119:0/3966788707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b562bb0 0x55942b564d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.096+0000 7fc8c5400640 1 -- 172.21.15.119:0/3966788707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b59f280 msgr2=0x55942b562f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.096+0000 7fc8c5400640 1 --2- 172.21.15.119:0/3966788707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b59f280 0x55942b562f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.096+0000 7fc8c5400640 1 -- 172.21.15.119:0/3966788707 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55942b2a57d0 con 0x55942b58d670 2024-04-25T10:16:48.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.096+0000 7fc8c5400640 1 --2- 172.21.15.119:0/3966788707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b58daf0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800d4b0 tx=0x7fc8b800d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb178561be47f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.096+0000 7fc8b5200640 1 -- 172.21.15.119:0/3966788707 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b801a070 con 0x55942b58d670 2024-04-25T10:16:48.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.096+0000 7fc8b5200640 1 -- 172.21.15.119:0/3966788707 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b800b700 con 0x55942b58d670 2024-04-25T10:16:48.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.096+0000 7fc8b5200640 1 -- 172.21.15.119:0/3966788707 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b800bad0 con 0x55942b58d670 2024-04-25T10:16:48.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.097+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/3966788707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 msgr2=0x55942b58daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.097+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/3966788707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b58daf0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800d4b0 tx=0x7fc8b800d970 comp rx=0 tx=0).stop 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.097+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/3966788707 shutdown_connections 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.097+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/3966788707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b58daf0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.097+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/3966788707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b59f280 0x55942b562f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.097+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/3966788707 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b562bb0 0x55942b564d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.097+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/3966788707 >> 172.21.15.119:0/3966788707 conn(0x55942b52aa30 msgr2=0x55942b5a8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.097+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/3966788707 shutdown_connections 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.097+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/3966788707 wait complete. 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.098+0000 7fc8ca3c0740 1 -- start start 2024-04-25T10:16:48.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.098+0000 7fc8ca3c0740 1 Processor -- start 2024-04-25T10:16:48.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.099+0000 7fc8ca3c0740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b562bb0 0x55942b574f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.099+0000 7fc8ca3c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b58e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.099+0000 7fc8ca3c0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b59f280 0x55942b590070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.099+0000 7fc8ca3c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942b4653a0 con 0x55942b58d670 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.099+0000 7fc8ca3c0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942b58b1c0 con 0x55942b59f280 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.099+0000 7fc8ca3c0740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55942b5864f0 con 0x55942b562bb0 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.099+0000 7fc8ca3c0740 10 client.0 _ll_register_callbacks cb 0x55942b482b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.099+0000 7fc8b7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b58e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.100+0000 7fc8b7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b58e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42412/0 (socket says 172.21.15.119:42412) 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.100+0000 7fc8b7e00640 1 -- 172.21.15.119:0/1499532922 learned_addr learned my addr 172.21.15.119:0/1499532922 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.100+0000 7fc8b7e00640 1 -- 172.21.15.119:0/1499532922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b562bb0 msgr2=0x55942b574f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.100+0000 7fc8b7e00640 1 --2- 172.21.15.119:0/1499532922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b562bb0 0x55942b574f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.100+0000 7fc8b7e00640 1 -- 172.21.15.119:0/1499532922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b59f280 msgr2=0x55942b590070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.100+0000 7fc8b7e00640 1 --2- 172.21.15.119:0/1499532922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b59f280 0x55942b590070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.100+0000 7fc8b7e00640 1 -- 172.21.15.119:0/1499532922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55942b1db3f0 con 0x55942b58d670 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.100+0000 7fc8b7e00640 1 --2- 172.21.15.119:0/1499532922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b58e430 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009990 tx=0x7fc8a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b8ee8d2b87c2ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.100+0000 7fc8ad600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.101+0000 7fc8ad600640 1 -- 172.21.15.119:0/1499532922 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8037040 con 0x55942b58d670 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.101+0000 7fc8ad600640 1 -- 172.21.15.119:0/1499532922 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc8a803fb40 con 0x55942b58d670 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.101+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/1499532922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8b800b700 con 0x55942b58d670 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.101+0000 7fc8ad600640 1 -- 172.21.15.119:0/1499532922 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8a8053050 con 0x55942b58d670 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.101+0000 7fc8ca3c0740 10 client.8481 fetch_fsmap learned FSMap version 88 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.101+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/1499532922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55942b19b0f0 con 0x55942b58d670 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.101+0000 7fc8ad600640 1 -- 172.21.15.119:0/1499532922 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b800b700 con 0x55942b58d670 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.101+0000 7fc8ad600640 1 -- 172.21.15.119:0/1499532922 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 88) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc8a8056020 con 0x55942b58d670 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.101+0000 7fc8ca3c0740 10 client.8481 fetch_fsmap finished waiting for FSMap version 88 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.101+0000 7fc8ca3c0740 10 client.8481 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:16:48.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.101+0000 7fc8ca3c0740 20 client.8481 populate_metadata read hostname 'smithi119' 2024-04-25T10:16:48.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.101+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/1499532922 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55942b5851b0 0x55942b585600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.101+0000 7fc8ca3c0740 4 client.8481 mds_command: new command op to 8367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:48.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.101+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/1499532922 --> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55942b4d41b0 con 0x55942b5851b0 2024-04-25T10:16:48.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.102+0000 7fc8c4a00640 1 --2- 172.21.15.119:0/1499532922 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55942b5851b0 0x55942b585600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.102+0000 7fc8c4a00640 1 --2- 172.21.15.119:0/1499532922 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55942b5851b0 0x55942b585600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a7991665c98ed server_cookie=f9a4584442b06382 in_seq=0 out_seq=0 2024-04-25T10:16:48.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.102+0000 7fc8ad600640 10 client.8481 ms_handle_connect on v2:172.21.15.119:6835/464677621 2024-04-25T10:16:48.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.103+0000 7fc8ad600640 1 -- 172.21.15.119:0/1499532922 <== mds.0 v2:172.21.15.119:6835/464677621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55942b4d41b0 con 0x55942b5851b0 2024-04-25T10:16:48.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.103+0000 7fc8ad600640 10 client.8481 handle_command_reply: tid=0 2024-04-25T10:16:48.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.103+0000 7fc8ca3c0740 1 client.8481 shutdown 2024-04-25T10:16:48.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.103+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/1499532922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 msgr2=0x55942b58e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.103+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/1499532922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b58e430 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009990 tx=0x7fc8a80046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.103+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/1499532922 shutdown_connections 2024-04-25T10:16:48.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.103+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/1499532922 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55942b5851b0 0x55942b585600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.103+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/1499532922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b59f280 0x55942b590070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.103+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/1499532922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b58e430 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.103+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/1499532922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b562bb0 0x55942b574f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.103+0000 7fc8ad600640 0 client.8481 ms_handle_reset on v2:172.21.15.119:6835/464677621 2024-04-25T10:16:48.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.103+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/1499532922 >> 172.21.15.119:0/1499532922 conn(0x55942b52aa30 msgr2=0x55942b58a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.103+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/1499532922 shutdown_connections 2024-04-25T10:16:48.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.103+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/1499532922 wait complete. 2024-04-25T10:16:48.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.104+0000 7fc8ca3c0740 20 client.8481 trim_cache size 0 max 16384 2024-04-25T10:16:48.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.120+0000 7fc8ca3c0740 1 Processor -- start 2024-04-25T10:16:48.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.120+0000 7fc8ca3c0740 1 -- start start 2024-04-25T10:16:48.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8ca3c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b562bb0 0x55942b58fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8ca3c0740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b58d670 0x55942b583c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8ca3c0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b59f280 0x55942b5b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8ca3c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55942b58a840 con 0x55942b562bb0 2024-04-25T10:16:48.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8ca3c0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01fff90 con 0x55942b59f280 2024-04-25T10:16:48.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8ca3c0740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c0152e40 con 0x55942b58d670 2024-04-25T10:16:48.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8c4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b562bb0 0x55942b58fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8c4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b562bb0 0x55942b58fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42424/0 (socket says 172.21.15.119:42424) 2024-04-25T10:16:48.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8c4a00640 1 -- 172.21.15.119:0/2868586757 learned_addr learned my addr 172.21.15.119:0/2868586757 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:48.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8b7e00640 1 --2- 172.21.15.119:0/2868586757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b58d670 0x55942b583c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8c4a00640 1 -- 172.21.15.119:0/2868586757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b58d670 msgr2=0x55942b583c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8c4a00640 1 --2- 172.21.15.119:0/2868586757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b58d670 0x55942b583c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8c4a00640 1 -- 172.21.15.119:0/2868586757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b59f280 msgr2=0x55942b5b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8c4a00640 1 --2- 172.21.15.119:0/2868586757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b59f280 0x55942b5b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.121+0000 7fc8c4a00640 1 -- 172.21.15.119:0/2868586757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55942b55cce0 con 0x55942b562bb0 2024-04-25T10:16:48.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.122+0000 7fc8c4a00640 1 --2- 172.21.15.119:0/2868586757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b562bb0 0x55942b58fec0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00548e0 tx=0x7fc8b0054850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b1b01b327b525b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.122+0000 7fc8afe00640 1 -- 172.21.15.119:0/2868586757 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0086070 con 0x55942b562bb0 2024-04-25T10:16:48.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.122+0000 7fc8afe00640 1 -- 172.21.15.119:0/2868586757 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b0088400 con 0x55942b562bb0 2024-04-25T10:16:48.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.122+0000 7fc8afe00640 1 -- 172.21.15.119:0/2868586757 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0054af0 con 0x55942b562bb0 2024-04-25T10:16:48.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.123+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/2868586757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b562bb0 msgr2=0x55942b58fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.123+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/2868586757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b562bb0 0x55942b58fec0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00548e0 tx=0x7fc8b0054850 comp rx=0 tx=0).stop 2024-04-25T10:16:48.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.123+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/2868586757 shutdown_connections 2024-04-25T10:16:48.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.123+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/2868586757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b59f280 0x55942b5b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.123+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/2868586757 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b58d670 0x55942b583c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.123+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/2868586757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b562bb0 0x55942b58fec0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.123+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/2868586757 >> 172.21.15.119:0/2868586757 conn(0x55942b52aa30 msgr2=0x55942b582da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.123+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/2868586757 shutdown_connections 2024-04-25T10:16:48.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.123+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/2868586757 wait complete. 2024-04-25T10:16:48.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.123+0000 7fc8ca3c0740 1 -- start start 2024-04-25T10:16:48.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.124+0000 7fc8ca3c0740 1 Processor -- start 2024-04-25T10:16:48.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8ca3c0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b562bb0 0x55942b574350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8ca3c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b613290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8ca3c0740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b59f280 0x55942b5ba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8ca3c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c0152840 con 0x55942b58d670 2024-04-25T10:16:48.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8ca3c0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c0152240 con 0x55942b562bb0 2024-04-25T10:16:48.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8ca3c0740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55942b573770 con 0x55942b59f280 2024-04-25T10:16:48.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8ca3c0740 10 client.0 _ll_register_callbacks cb 0x7fc8a8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:48.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8b7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b613290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8b7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b613290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42426/0 (socket says 172.21.15.119:42426) 2024-04-25T10:16:48.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8b7e00640 1 -- 172.21.15.119:0/405693978 learned_addr learned my addr 172.21.15.119:0/405693978 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:16:48.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8b7e00640 1 -- 172.21.15.119:0/405693978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b59f280 msgr2=0x55942b5ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8b7e00640 1 --2- 172.21.15.119:0/405693978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b59f280 0x55942b5ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8b7e00640 1 -- 172.21.15.119:0/405693978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b562bb0 msgr2=0x55942b574350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8b7e00640 1 --2- 172.21.15.119:0/405693978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b562bb0 0x55942b574350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8b7e00640 1 -- 172.21.15.119:0/405693978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55942b6137d0 con 0x55942b58d670 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8b7e00640 1 --2- 172.21.15.119:0/405693978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b613290 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009960 tx=0x7fc8a8008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d9b2333520563 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.125+0000 7fc8ad600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.126+0000 7fc8ad600640 1 -- 172.21.15.119:0/405693978 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a805c060 con 0x55942b58d670 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.126+0000 7fc8ad600640 1 -- 172.21.15.119:0/405693978 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc8a8003610 con 0x55942b58d670 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.126+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/405693978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8b0088400 con 0x55942b58d670 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.126+0000 7fc8ad600640 1 -- 172.21.15.119:0/405693978 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8a8037070 con 0x55942b58d670 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.126+0000 7fc8ca3c0740 10 client.8487 fetch_fsmap learned FSMap version 88 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.126+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/405693978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55942b58ffe0 con 0x55942b58d670 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.126+0000 7fc8ad600640 1 -- 172.21.15.119:0/405693978 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b0088400 con 0x55942b58d670 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.126+0000 7fc8ad600640 1 -- 172.21.15.119:0/405693978 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 88) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc8a8065020 con 0x55942b58d670 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.126+0000 7fc8ca3c0740 10 client.8487 fetch_fsmap finished waiting for FSMap version 88 2024-04-25T10:16:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.126+0000 7fc8ca3c0740 10 client.8487 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:16:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.126+0000 7fc8ca3c0740 20 client.8487 populate_metadata read hostname 'smithi119' 2024-04-25T10:16:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.126+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/405693978 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55942b59e000 0x55942b59e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.126+0000 7fc8ca3c0740 4 client.8487 mds_command: new command op to 8367 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:16:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.126+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/405693978 --> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55942b58fbe0 con 0x55942b59e000 2024-04-25T10:16:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.127+0000 7fc8c4a00640 1 --2- 172.21.15.119:0/405693978 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55942b59e000 0x55942b59e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.127+0000 7fc8c4a00640 1 --2- 172.21.15.119:0/405693978 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55942b59e000 0x55942b59e470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa597d1ea8e8dae5 server_cookie=61280b021336e07b in_seq=0 out_seq=0 2024-04-25T10:16:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.127+0000 7fc8ad600640 10 client.8487 ms_handle_connect on v2:172.21.15.119:6835/464677621 2024-04-25T10:16:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.127+0000 7fc8ad600640 1 -- 172.21.15.119:0/405693978 <== mds.0 v2:172.21.15.119:6835/464677621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55942b58fbe0 con 0x55942b59e000 2024-04-25T10:16:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.127+0000 7fc8ad600640 10 client.8487 handle_command_reply: tid=0 2024-04-25T10:16:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.128+0000 7fc8ca3c0740 1 client.8487 shutdown 2024-04-25T10:16:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.128+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/405693978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 msgr2=0x55942b613290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.128+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/405693978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b613290 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009960 tx=0x7fc8a8008030 comp rx=0 tx=0).stop 2024-04-25T10:16:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.128+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/405693978 shutdown_connections 2024-04-25T10:16:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.128+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/405693978 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55942b59e000 0x55942b59e470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.128+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/405693978 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55942b59f280 0x55942b5ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.128+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/405693978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55942b58d670 0x55942b613290 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.128+0000 7fc8ca3c0740 1 --2- 172.21.15.119:0/405693978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55942b562bb0 0x55942b574350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.128+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/405693978 >> 172.21.15.119:0/405693978 conn(0x55942b52aa30 msgr2=0x55942b583970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.128+0000 7fc8ad600640 0 client.8487 ms_handle_reset on v2:172.21.15.119:6835/464677621 2024-04-25T10:16:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.128+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/405693978 shutdown_connections 2024-04-25T10:16:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.128+0000 7fc8ca3c0740 1 -- 172.21.15.119:0/405693978 wait complete. 2024-04-25T10:16:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.128+0000 7fc8ca3c0740 20 client.8487 trim_cache size 0 max 16384 2024-04-25T10:16:48.136 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:16:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:16:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": { 2024-04-25T10:16:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: "bbb77ef1-e741-4ea2-b4b5-5d82f89ac394": { 2024-04-25T10:16:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: "path": "/", 2024-04-25T10:16:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: "tag": "bbb77ef1-e741-4ea2-b4b5-5d82f89ac394", 2024-04-25T10:16:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: "options": "recursive,repair" 2024-04-25T10:16:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:16:48.136 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:16:48.136 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:16:48.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.129+0000 7fc8c7400640 1 -- 172.21.15.119:0/2776293974 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc89c06b500 msgr2=0x7fc89c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.129+0000 7fc8c7400640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc89c06b500 0x7fc89c06d9c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8034000 tx=0x7fc8a802e000 comp rx=0 tx=0).stop 2024-04-25T10:16:48.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.129+0000 7fc8c7400640 1 -- 172.21.15.119:0/2776293974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c00aed40 msgr2=0x7fc8c014de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.129+0000 7fc8c7400640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c00aed40 0x7fc8c014de00 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000b420 tx=0x7fc8b000b8e0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.129+0000 7fc8c7400640 1 -- 172.21.15.119:0/2776293974 shutdown_connections 2024-04-25T10:16:48.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.129+0000 7fc8c7400640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc89c06b500 0x7fc89c06d9c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.129+0000 7fc8c7400640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8c015b720 0x7fc8c00cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.129+0000 7fc8c7400640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc8c00cfa90 0x7fc8c01549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.129+0000 7fc8c7400640 1 --2- 172.21.15.119:0/2776293974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8c00aed40 0x7fc8c014de00 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.129+0000 7fc8c7400640 1 -- 172.21.15.119:0/2776293974 >> 172.21.15.119:0/2776293974 conn(0x7fc8c006aba0 msgr2=0x7fc8c00ae130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.129+0000 7fc8c7400640 1 -- 172.21.15.119:0/2776293974 shutdown_connections 2024-04-25T10:16:48.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:48.129+0000 7fc8c7400640 1 -- 172.21.15.119:0/2776293974 wait complete. 2024-04-25T10:16:48.150 INFO:tasks.cephfs.filesystem:scrub status for tag:bbb77ef1-e741-4ea2-b4b5-5d82f89ac394 - {'path': '/', 'tag': 'bbb77ef1-e741-4ea2-b4b5-5d82f89ac394', 'options': 'recursive,repair'} 2024-04-25T10:16:49.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:49.789+0000 7fa3fa000640 1 -- 172.21.15.119:0/2602302363 <== mon.0 v2:172.21.15.119:3300/0 7 ==== log(8 entries from seq 1 at 2024-04-25T10:16:47.879267+0000) v1 ==== 1407+0+0 (secure 0 0 0) 0x7fa3fc091070 con 0x7fa4040d2d10 2024-04-25T10:16:49.793 INFO:teuthology.orchestra.run.smithi119.stdout: cluster: 2024-04-25T10:16:49.794 INFO:teuthology.orchestra.run.smithi119.stdout: id: e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:16:49.794 INFO:teuthology.orchestra.run.smithi119.stdout: health: HEALTH_OK 2024-04-25T10:16:49.794 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:49.794 INFO:teuthology.orchestra.run.smithi119.stdout: services: 2024-04-25T10:16:49.794 INFO:teuthology.orchestra.run.smithi119.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-04-25T10:16:49.794 INFO:teuthology.orchestra.run.smithi119.stdout: mgr: y(active, since 9m), standbys: x 2024-04-25T10:16:49.794 INFO:teuthology.orchestra.run.smithi119.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:16:49.794 INFO:teuthology.orchestra.run.smithi119.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 10m) 2024-04-25T10:16:49.794 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:49.794 INFO:teuthology.orchestra.run.smithi119.stdout: data: 2024-04-25T10:16:49.794 INFO:teuthology.orchestra.run.smithi119.stdout: volumes: 1/1 healthy 2024-04-25T10:16:49.795 INFO:teuthology.orchestra.run.smithi119.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:16:49.795 INFO:teuthology.orchestra.run.smithi119.stdout: objects: 30 objects, 19 MiB 2024-04-25T10:16:49.795 INFO:teuthology.orchestra.run.smithi119.stdout: usage: 290 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:16:49.795 INFO:teuthology.orchestra.run.smithi119.stdout: pgs: 97 active+clean 2024-04-25T10:16:49.795 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:49.795 INFO:teuthology.orchestra.run.smithi119.stdout: io: 2024-04-25T10:16:49.795 INFO:teuthology.orchestra.run.smithi119.stdout: client: 8.9 KiB/s rd, 102 B/s wr, 8 op/s rd, 0 op/s wr 2024-04-25T10:16:49.795 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:49.795 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:16:49.795 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:16:47.879267+0000 mds.a [INF] scrub queued for path: / 2024-04-25T10:16:49.795 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:16:47.879272+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:16:49.795 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:16:47.879279+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-25T10:16:49.795 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:16:47.880150+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:16:49.795 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:16:47.880157+0000 mds.a [ERR] inode table repaired for inode: 0x10000000001 2024-04-25T10:16:49.796 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:16:47.880224+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:16:49.796 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:16:47.880230+0000 mds.a [ERR] inode table repaired for inode: 0x10000000002 2024-04-25T10:16:49.796 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:16:47.881014+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:16:52.813 INFO:teuthology.orchestra.run.smithi119.stdout:2024-04-25T10:16:51.785018+0000 mds.a [INF] scrub summary: idle 2024-04-25T10:16:52.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:16:52.809+0000 7fa3fa000640 1 -- 172.21.15.119:0/2602302363 <== mon.0 v2:172.21.15.119:3300/0 8 ==== log(1 entries from seq 9 at 2024-04-25T10:16:51.785018+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7fa3fc057280 con 0x7fa4040d2d10 2024-04-25T10:17:02.644 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:02.653 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:18.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:18.152 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:17:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.362+0000 7f4ccf000640 1 Processor -- start 2024-04-25T10:17:18.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.362+0000 7f4ccf000640 1 -- start start 2024-04-25T10:17:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.363+0000 7f4ccf000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4cc8144e70 0x7f4cc806d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.363+0000 7f4ccf000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4cc806d7a0 0x7f4cc80ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.363+0000 7f4ccf000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc806dcb0 con 0x7f4cc80cef10 2024-04-25T10:17:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.363+0000 7f4ccf000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc815bfe0 con 0x7f4cc8144e70 2024-04-25T10:17:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.364+0000 7f4ccf000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc806df90 con 0x7f4cc806d7a0 2024-04-25T10:17:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.364+0000 7f4cc7e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4cc8144e70 0x7f4cc806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.364+0000 7f4cc7e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4cc8144e70 0x7f4cc806d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46910/0 (socket says 172.21.15.119:46910) 2024-04-25T10:17:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.364+0000 7f4cc7e00640 1 -- 172.21.15.119:0/3159651777 learned_addr learned my addr 172.21.15.119:0/3159651777 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.364+0000 7f4cc7e00640 1 -- 172.21.15.119:0/3159651777 >> v2:172.21.15.132:3300/0 conn(0x7f4cc806d7a0 msgr2=0x7f4cc80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.364+0000 7f4cc7e00640 1 --2- 172.21.15.119:0/3159651777 >> v2:172.21.15.132:3300/0 conn(0x7f4cc806d7a0 0x7f4cc80ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.364+0000 7f4cc7e00640 1 -- 172.21.15.119:0/3159651777 >> v1:172.21.15.119:6789/0 conn(0x7f4cc80cef10 legacy=0x7f4cc8071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.364+0000 7f4cc7e00640 1 -- 172.21.15.119:0/3159651777 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc806e270 con 0x7f4cc8144e70 2024-04-25T10:17:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.365+0000 7f4cc7e00640 1 --2- 172.21.15.119:0/3159651777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cc8144e70 0x7f4cc806d260 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8072560 tx=0x7f4cb802a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cbc1c56f107bdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.365+0000 7f4cc7400640 1 -- 172.21.15.119:0/3159651777 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8037070 con 0x7f4cc8144e70 2024-04-25T10:17:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.366+0000 7f4cc7400640 1 -- 172.21.15.119:0/3159651777 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb8032440 con 0x7f4cc8144e70 2024-04-25T10:17:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.366+0000 7f4cc7400640 1 -- 172.21.15.119:0/3159651777 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8032810 con 0x7f4cc8144e70 2024-04-25T10:17:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.366+0000 7f4ccf000640 1 -- 172.21.15.119:0/3159651777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cc8144e70 msgr2=0x7f4cc806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.366+0000 7f4ccf000640 1 --2- 172.21.15.119:0/3159651777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cc8144e70 0x7f4cc806d260 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8072560 tx=0x7f4cb802a1f0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.366+0000 7f4ccf000640 1 -- 172.21.15.119:0/3159651777 shutdown_connections 2024-04-25T10:17:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.366+0000 7f4ccf000640 1 --2- 172.21.15.119:0/3159651777 >> v2:172.21.15.132:3300/0 conn(0x7f4cc806d7a0 0x7f4cc80ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.366+0000 7f4ccf000640 1 --2- 172.21.15.119:0/3159651777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cc8144e70 0x7f4cc806d260 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.366+0000 7f4ccf000640 1 -- 172.21.15.119:0/3159651777 >> 172.21.15.119:0/3159651777 conn(0x7f4cc806aba0 msgr2=0x7f4cc806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.366+0000 7f4ccf000640 1 -- 172.21.15.119:0/3159651777 shutdown_connections 2024-04-25T10:17:18.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.367+0000 7f4ccf000640 1 -- 172.21.15.119:0/3159651777 wait complete. 2024-04-25T10:17:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.368+0000 7f4ccf000640 1 Processor -- start 2024-04-25T10:17:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.368+0000 7f4ccf000640 1 -- start start 2024-04-25T10:17:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.369+0000 7f4ccf000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc806d7a0 0x7f4cc81e42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.369+0000 7f4ccf000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cc80cef10 0x7f4cc80b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.369+0000 7f4ccf000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cc8144e70 0x7f4cc813bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.369+0000 7f4ccf000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc81454b0 con 0x7f4cc80cef10 2024-04-25T10:17:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.369+0000 7f4ccf000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc80cf370 con 0x7f4cc806d7a0 2024-04-25T10:17:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.369+0000 7f4ccf000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc8145630 con 0x7f4cc8144e70 2024-04-25T10:17:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.369+0000 7f4cbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cc80cef10 0x7f4cc80b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.370+0000 7f4cbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cc80cef10 0x7f4cc80b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46922/0 (socket says 172.21.15.119:46922) 2024-04-25T10:17:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.370+0000 7f4cbf400640 1 -- 172.21.15.119:0/1492408849 learned_addr learned my addr 172.21.15.119:0/1492408849 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.370+0000 7f4cc7e00640 1 --2- 172.21.15.119:0/1492408849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc806d7a0 0x7f4cc81e42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.370+0000 7f4cbf400640 1 -- 172.21.15.119:0/1492408849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cc8144e70 msgr2=0x7f4cc813bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.370+0000 7f4cbf400640 1 --2- 172.21.15.119:0/1492408849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cc8144e70 0x7f4cc813bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.370+0000 7f4cbf400640 1 -- 172.21.15.119:0/1492408849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc806d7a0 msgr2=0x7f4cc81e42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.370+0000 7f4cbf400640 1 --2- 172.21.15.119:0/1492408849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc806d7a0 0x7f4cc81e42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.370+0000 7f4cbf400640 1 -- 172.21.15.119:0/1492408849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc80b1e40 con 0x7f4cc80cef10 2024-04-25T10:17:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.370+0000 7f4cc7e00640 1 --2- 172.21.15.119:0/1492408849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc806d7a0 0x7f4cc81e42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.370+0000 7f4cbf400640 1 --2- 172.21.15.119:0/1492408849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cc80cef10 0x7f4cc80b1900 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000b4a0 tx=0x7f4cb000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.371+0000 7f4cc4c00640 1 -- 172.21.15.119:0/1492408849 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0014070 con 0x7f4cc80cef10 2024-04-25T10:17:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.371+0000 7f4cc4c00640 1 -- 172.21.15.119:0/1492408849 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb0010610 con 0x7f4cc80cef10 2024-04-25T10:17:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.371+0000 7f4cc4c00640 1 -- 172.21.15.119:0/1492408849 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0010a00 con 0x7f4cc80cef10 2024-04-25T10:17:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.371+0000 7f4ccf000640 1 -- 172.21.15.119:0/1492408849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc81e48b0 con 0x7f4cc80cef10 2024-04-25T10:17:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.371+0000 7f4ccf000640 1 -- 172.21.15.119:0/1492408849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc81e4e70 con 0x7f4cc80cef10 2024-04-25T10:17:18.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.373+0000 7f4cc4c00640 1 -- 172.21.15.119:0/1492408849 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cb0017070 con 0x7f4cc80cef10 2024-04-25T10:17:18.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.375+0000 7f4cc4c00640 1 --2- 172.21.15.119:0/1492408849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ca406b4d0 0x7f4ca406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.375+0000 7f4cc4c00640 1 -- 172.21.15.119:0/1492408849 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4cb0089c90 con 0x7f4cc80cef10 2024-04-25T10:17:18.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.375+0000 7f4cc7e00640 1 --2- 172.21.15.119:0/1492408849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ca406b4d0 0x7f4ca406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.375+0000 7f4cd1e82740 1 Processor -- start 2024-04-25T10:17:18.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.376+0000 7f4cd1e82740 1 -- start start 2024-04-25T10:17:18.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.376+0000 7f4cc7e00640 1 --2- 172.21.15.119:0/1492408849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ca406b4d0 0x7f4ca406d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8034000 tx=0x7f4cb802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.376+0000 7f4cd1e82740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c8d9bb0 0x55a84c8dbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.376+0000 7f4cd1e82740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c916280 0x55a84c8d9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.376+0000 7f4cd1e82740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c904af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.376+0000 7f4cd1e82740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c83dc70 con 0x55a84c904670 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.376+0000 7f4cd1e82740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c7e6cf0 con 0x55a84c8d9bb0 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.376+0000 7f4cd1e82740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a84c839d10 con 0x55a84c916280 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.377+0000 7f4ccd000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c904af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.377+0000 7f4ccd000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c904af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46928/0 (socket says 172.21.15.119:46928) 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.377+0000 7f4ccd000640 1 -- 172.21.15.119:0/2191884632 learned_addr learned my addr 172.21.15.119:0/2191884632 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.377+0000 7f4cc7e00640 1 --2- 172.21.15.119:0/2191884632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c8d9bb0 0x55a84c8dbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.377+0000 7f4cbf400640 1 --2- 172.21.15.119:0/2191884632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c916280 0x55a84c8d9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.377+0000 7f4ccd000640 1 -- 172.21.15.119:0/2191884632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c916280 msgr2=0x55a84c8d9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.377+0000 7f4ccd000640 1 --2- 172.21.15.119:0/2191884632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c916280 0x55a84c8d9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.377+0000 7f4ccd000640 1 -- 172.21.15.119:0/2191884632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c8d9bb0 msgr2=0x55a84c8dbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.377+0000 7f4ccd000640 1 --2- 172.21.15.119:0/2191884632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c8d9bb0 0x55a84c8dbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.377+0000 7f4ccd000640 1 -- 172.21.15.119:0/2191884632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a84c61c7d0 con 0x55a84c904670 2024-04-25T10:17:18.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.378+0000 7f4ccd000640 1 --2- 172.21.15.119:0/2191884632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c904af0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0001d20 tx=0x7f4cc00021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba950ad233d8c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.378+0000 7f4cbcc00640 1 -- 172.21.15.119:0/2191884632 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0017070 con 0x55a84c904670 2024-04-25T10:17:18.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.378+0000 7f4cbcc00640 1 -- 172.21.15.119:0/2191884632 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc000cc70 con 0x55a84c904670 2024-04-25T10:17:18.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.378+0000 7f4cbcc00640 1 -- 172.21.15.119:0/2191884632 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0003b90 con 0x55a84c904670 2024-04-25T10:17:18.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.378+0000 7f4cd1e82740 1 -- 172.21.15.119:0/2191884632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 msgr2=0x55a84c904af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.379+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/2191884632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c904af0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0001d20 tx=0x7f4cc00021e0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.379+0000 7f4cd1e82740 1 -- 172.21.15.119:0/2191884632 shutdown_connections 2024-04-25T10:17:18.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.379+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/2191884632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c904af0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.379+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/2191884632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c916280 0x55a84c8d9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.379+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/2191884632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c8d9bb0 0x55a84c8dbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.379+0000 7f4cd1e82740 1 -- 172.21.15.119:0/2191884632 >> 172.21.15.119:0/2191884632 conn(0x55a84c8a1a30 msgr2=0x55a84c91f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.379+0000 7f4cd1e82740 1 -- 172.21.15.119:0/2191884632 shutdown_connections 2024-04-25T10:17:18.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.379+0000 7f4cd1e82740 1 -- 172.21.15.119:0/2191884632 wait complete. 2024-04-25T10:17:18.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.379+0000 7f4cd1e82740 1 -- start start 2024-04-25T10:17:18.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.380+0000 7f4cd1e82740 1 Processor -- start 2024-04-25T10:17:18.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cd1e82740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c8d9bb0 0x55a84c904a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cd1e82740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c921cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cd1e82740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c916280 0x55a84c922230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cd1e82740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c7dc3a0 con 0x55a84c904670 2024-04-25T10:17:18.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cd1e82740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c9021c0 con 0x55a84c8d9bb0 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cd1e82740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a84c91c760 con 0x55a84c916280 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cd1e82740 10 client.0 _ll_register_callbacks cb 0x55a84c7f9b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c921cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c921cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46930/0 (socket says 172.21.15.119:46930) 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cbf400640 1 -- 172.21.15.119:0/2191826221 learned_addr learned my addr 172.21.15.119:0/2191826221 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cbf400640 1 -- 172.21.15.119:0/2191826221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c916280 msgr2=0x55a84c922230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cbf400640 1 --2- 172.21.15.119:0/2191826221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c916280 0x55a84c922230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cbf400640 1 -- 172.21.15.119:0/2191826221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c8d9bb0 msgr2=0x55a84c904a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cbf400640 1 --2- 172.21.15.119:0/2191826221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c8d9bb0 0x55a84c904a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.381+0000 7f4cbf400640 1 -- 172.21.15.119:0/2191826221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a84c5523f0 con 0x55a84c904670 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.382+0000 7f4cbf400640 1 --2- 172.21.15.119:0/2191826221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c921cf0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0054800 tx=0x7f4cb005c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14fa2cf81590824 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.382+0000 7f4cb5600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.382+0000 7f4cb5600640 1 -- 172.21.15.119:0/2191826221 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0086060 con 0x55a84c904670 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.382+0000 7f4cb5600640 1 -- 172.21.15.119:0/2191826221 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4cb0010ba0 con 0x55a84c904670 2024-04-25T10:17:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.382+0000 7f4cd1e82740 1 -- 172.21.15.119:0/2191826221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cc000cc70 con 0x55a84c904670 2024-04-25T10:17:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.382+0000 7f4cb5600640 1 -- 172.21.15.119:0/2191826221 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cb001e040 con 0x55a84c904670 2024-04-25T10:17:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.382+0000 7f4cd1e82740 10 client.8499 fetch_fsmap learned FSMap version 88 2024-04-25T10:17:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.382+0000 7f4cd1e82740 1 -- 172.21.15.119:0/2191826221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55a84c5120f0 con 0x55a84c904670 2024-04-25T10:17:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.382+0000 7f4cb5600640 1 -- 172.21.15.119:0/2191826221 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc000cc70 con 0x55a84c904670 2024-04-25T10:17:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.382+0000 7f4cb5600640 1 -- 172.21.15.119:0/2191826221 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 88) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4cb00577f0 con 0x55a84c904670 2024-04-25T10:17:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.382+0000 7f4cd1e82740 10 client.8499 fetch_fsmap finished waiting for FSMap version 88 2024-04-25T10:17:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.382+0000 7f4cd1e82740 10 client.8499 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:17:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.383+0000 7f4cd1e82740 20 client.8499 populate_metadata read hostname 'smithi119' 2024-04-25T10:17:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.383+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/2191826221 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55a84c904fe0 0x55a84c906fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.383+0000 7f4cd1e82740 4 client.8499 mds_command: new command op to 8367 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.383+0000 7f4cd1e82740 1 -- 172.21.15.119:0/2191826221 --> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a84c84b1b0 con 0x55a84c904fe0 2024-04-25T10:17:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.383+0000 7f4cc7e00640 1 --2- 172.21.15.119:0/2191826221 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55a84c904fe0 0x55a84c906fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.383+0000 7f4cc7e00640 1 --2- 172.21.15.119:0/2191826221 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55a84c904fe0 0x55a84c906fe0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850d62584f33c975 server_cookie=5bf94f83a9c2ed19 in_seq=0 out_seq=0 2024-04-25T10:17:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.383+0000 7f4cb5600640 10 client.8499 ms_handle_connect on v2:172.21.15.119:6835/464677621 2024-04-25T10:17:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.384+0000 7f4cb5600640 1 -- 172.21.15.119:0/2191826221 <== mds.0 v2:172.21.15.119:6835/464677621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a84c84b1b0 con 0x55a84c904fe0 2024-04-25T10:17:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.384+0000 7f4cb5600640 10 client.8499 handle_command_reply: tid=0 2024-04-25T10:17:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.384+0000 7f4cd1e82740 1 client.8499 shutdown 2024-04-25T10:17:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.384+0000 7f4cd1e82740 1 -- 172.21.15.119:0/2191826221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 msgr2=0x55a84c921cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.384+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/2191826221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c921cf0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0054800 tx=0x7f4cb005c630 comp rx=0 tx=0).stop 2024-04-25T10:17:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.384+0000 7f4cd1e82740 1 -- 172.21.15.119:0/2191826221 shutdown_connections 2024-04-25T10:17:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.384+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/2191826221 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55a84c904fe0 0x55a84c906fe0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.384+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/2191826221 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c916280 0x55a84c922230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.384+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/2191826221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c921cf0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.384+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/2191826221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c8d9bb0 0x55a84c904a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.384+0000 7f4cd1e82740 1 -- 172.21.15.119:0/2191826221 >> 172.21.15.119:0/2191826221 conn(0x55a84c8a1a30 msgr2=0x55a84c9186f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.384+0000 7f4cb5600640 0 client.8499 ms_handle_reset on v2:172.21.15.119:6835/464677621 2024-04-25T10:17:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.385+0000 7f4cd1e82740 1 -- 172.21.15.119:0/2191826221 shutdown_connections 2024-04-25T10:17:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.385+0000 7f4cd1e82740 1 -- 172.21.15.119:0/2191826221 wait complete. 2024-04-25T10:17:18.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.385+0000 7f4cd1e82740 20 client.8499 trim_cache size 0 max 16384 2024-04-25T10:17:18.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.401+0000 7f4cd1e82740 1 Processor -- start 2024-04-25T10:17:18.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.402+0000 7f4cd1e82740 1 -- start start 2024-04-25T10:17:18.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.402+0000 7f4cd1e82740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c8d9bb0 0x55a84c907bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.402+0000 7f4cd1e82740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c918670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.402+0000 7f4cd1e82740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c916280 0x55a84c918bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.402+0000 7f4cd1e82740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc81fff60 con 0x55a84c904670 2024-04-25T10:17:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.402+0000 7f4cd1e82740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc81ff1a0 con 0x55a84c916280 2024-04-25T10:17:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.402+0000 7f4cd1e82740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc80af650 con 0x55a84c8d9bb0 2024-04-25T10:17:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.403+0000 7f4cbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c918670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.403+0000 7f4cbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c918670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46942/0 (socket says 172.21.15.119:46942) 2024-04-25T10:17:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.403+0000 7f4cbf400640 1 -- 172.21.15.119:0/769172554 learned_addr learned my addr 172.21.15.119:0/769172554 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.403+0000 7f4cbf400640 1 -- 172.21.15.119:0/769172554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c8d9bb0 msgr2=0x55a84c907bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.403+0000 7f4cbf400640 1 --2- 172.21.15.119:0/769172554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c8d9bb0 0x55a84c907bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.403+0000 7f4cbf400640 1 -- 172.21.15.119:0/769172554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c916280 msgr2=0x55a84c918bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.403+0000 7f4cbf400640 1 --2- 172.21.15.119:0/769172554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c916280 0x55a84c918bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.403+0000 7f4cbf400640 1 -- 172.21.15.119:0/769172554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a84c8d7f10 con 0x55a84c904670 2024-04-25T10:17:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.403+0000 7f4cbf400640 1 --2- 172.21.15.119:0/769172554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c918670 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00568f0 tx=0x7f4cb005a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8808d98bd9e41f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.404+0000 7f4cb7e00640 1 -- 172.21.15.119:0/769172554 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0080070 con 0x55a84c904670 2024-04-25T10:17:18.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.404+0000 7f4cb7e00640 1 -- 172.21.15.119:0/769172554 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb0072030 con 0x55a84c904670 2024-04-25T10:17:18.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.404+0000 7f4cb7e00640 1 -- 172.21.15.119:0/769172554 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0010ba0 con 0x55a84c904670 2024-04-25T10:17:18.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.404+0000 7f4cd1e82740 1 -- 172.21.15.119:0/769172554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 msgr2=0x55a84c918670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.404+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/769172554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c918670 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00568f0 tx=0x7f4cb005a000 comp rx=0 tx=0).stop 2024-04-25T10:17:18.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.404+0000 7f4cd1e82740 1 -- 172.21.15.119:0/769172554 shutdown_connections 2024-04-25T10:17:18.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.404+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/769172554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c916280 0x55a84c918bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.404+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/769172554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c918670 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.404+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/769172554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c8d9bb0 0x55a84c907bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.404+0000 7f4cd1e82740 1 -- 172.21.15.119:0/769172554 >> 172.21.15.119:0/769172554 conn(0x55a84c8a1a30 msgr2=0x55a84c906ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.405+0000 7f4cd1e82740 1 -- 172.21.15.119:0/769172554 shutdown_connections 2024-04-25T10:17:18.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.405+0000 7f4cd1e82740 1 -- 172.21.15.119:0/769172554 wait complete. 2024-04-25T10:17:18.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.405+0000 7f4cd1e82740 1 -- start start 2024-04-25T10:17:18.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.405+0000 7f4cd1e82740 1 Processor -- start 2024-04-25T10:17:18.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.406+0000 7f4cd1e82740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c8d9bb0 0x55a84c939600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.406+0000 7f4cd1e82740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c940060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.406+0000 7f4cd1e82740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c916280 0x55a84c9405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.406+0000 7f4cd1e82740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c91fb30 con 0x55a84c904670 2024-04-25T10:17:18.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.406+0000 7f4cd1e82740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc80afc50 con 0x55a84c916280 2024-04-25T10:17:18.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.406+0000 7f4cd1e82740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a84c906200 con 0x55a84c8d9bb0 2024-04-25T10:17:18.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.406+0000 7f4cd1e82740 10 client.0 _ll_register_callbacks cb 0x7f4cb0058470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:18.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.407+0000 7f4cbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c940060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.407+0000 7f4cbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c940060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46954/0 (socket says 172.21.15.119:46954) 2024-04-25T10:17:18.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.407+0000 7f4cbf400640 1 -- 172.21.15.119:0/333021703 learned_addr learned my addr 172.21.15.119:0/333021703 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:18.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.407+0000 7f4cbf400640 1 -- 172.21.15.119:0/333021703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c8d9bb0 msgr2=0x55a84c939600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.407+0000 7f4cbf400640 1 --2- 172.21.15.119:0/333021703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c8d9bb0 0x55a84c939600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.407+0000 7f4cbf400640 1 -- 172.21.15.119:0/333021703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c916280 msgr2=0x55a84c9405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.407+0000 7f4ccd000640 1 --2- 172.21.15.119:0/333021703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c916280 0x55a84c9405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.407+0000 7f4cbf400640 1 --2- 172.21.15.119:0/333021703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c916280 0x55a84c9405a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.407+0000 7f4cbf400640 1 -- 172.21.15.119:0/333021703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a84c8da470 con 0x55a84c904670 2024-04-25T10:17:18.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.407+0000 7f4cbf400640 1 --2- 172.21.15.119:0/333021703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c940060 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0056650 tx=0x7f4cb0088020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a297495a6a58662 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.407+0000 7f4cb5600640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:17:18.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.407+0000 7f4cb5600640 1 -- 172.21.15.119:0/333021703 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0080070 con 0x55a84c904670 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.408+0000 7f4cb5600640 1 -- 172.21.15.119:0/333021703 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4cb0010da0 con 0x55a84c904670 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.408+0000 7f4cd1e82740 1 -- 172.21.15.119:0/333021703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cb0072030 con 0x55a84c904670 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.408+0000 7f4cb5600640 1 -- 172.21.15.119:0/333021703 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cb00540f0 con 0x55a84c904670 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.408+0000 7f4cd1e82740 10 client.8505 fetch_fsmap learned FSMap version 88 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.408+0000 7f4cd1e82740 1 -- 172.21.15.119:0/333021703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55a84c9390e0 con 0x55a84c904670 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.408+0000 7f4cb5600640 1 -- 172.21.15.119:0/333021703 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb0072030 con 0x55a84c904670 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.408+0000 7f4cb5600640 1 -- 172.21.15.119:0/333021703 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 88) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4cb0054290 con 0x55a84c904670 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.408+0000 7f4cd1e82740 10 client.8505 fetch_fsmap finished waiting for FSMap version 88 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.408+0000 7f4cd1e82740 10 client.8505 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.408+0000 7f4cd1e82740 20 client.8505 populate_metadata read hostname 'smithi119' 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.408+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/333021703 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55a84c8d4790 0x55a84c918670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.408+0000 7f4cd1e82740 4 client.8505 mds_command: new command op to 8367 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.408+0000 7f4cd1e82740 1 -- 172.21.15.119:0/333021703 --> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a84c845500 con 0x55a84c8d4790 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.408+0000 7f4cc7e00640 1 --2- 172.21.15.119:0/333021703 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55a84c8d4790 0x55a84c918670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.409+0000 7f4cc7e00640 1 --2- 172.21.15.119:0/333021703 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55a84c8d4790 0x55a84c918670 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5f5022c58f1dc3 server_cookie=2e1957964b260797 in_seq=0 out_seq=0 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.409+0000 7f4cb5600640 10 client.8505 ms_handle_connect on v2:172.21.15.119:6835/464677621 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.409+0000 7f4cb5600640 1 -- 172.21.15.119:0/333021703 <== mds.0 v2:172.21.15.119:6835/464677621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a84c845500 con 0x55a84c8d4790 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.409+0000 7f4cb5600640 10 client.8505 handle_command_reply: tid=0 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.409+0000 7f4cd1e82740 1 client.8505 shutdown 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.409+0000 7f4cd1e82740 1 -- 172.21.15.119:0/333021703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 msgr2=0x55a84c940060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.409+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/333021703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c940060 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0056650 tx=0x7f4cb0088020 comp rx=0 tx=0).stop 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.409+0000 7f4cd1e82740 1 -- 172.21.15.119:0/333021703 shutdown_connections 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.409+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/333021703 >> [v2:172.21.15.119:6835/464677621,v1:172.21.15.119:6837/464677621] conn(0x55a84c8d4790 0x55a84c918670 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.409+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/333021703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a84c916280 0x55a84c9405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.409+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/333021703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a84c904670 0x55a84c940060 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.409+0000 7f4cd1e82740 1 --2- 172.21.15.119:0/333021703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a84c8d9bb0 0x55a84c939600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.409+0000 7f4cb5600640 0 client.8505 ms_handle_reset on v2:172.21.15.119:6835/464677621 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.409+0000 7f4cd1e82740 1 -- 172.21.15.119:0/333021703 >> 172.21.15.119:0/333021703 conn(0x55a84c8a1a30 msgr2=0x55a84c906ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.410+0000 7f4cd1e82740 1 -- 172.21.15.119:0/333021703 shutdown_connections 2024-04-25T10:17:18.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.410+0000 7f4cd1e82740 1 -- 172.21.15.119:0/333021703 wait complete. 2024-04-25T10:17:18.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.410+0000 7f4cd1e82740 20 client.8505 trim_cache size 0 max 16384 2024-04-25T10:17:18.417 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:17:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: "status": "no active scrubs running", 2024-04-25T10:17:18.417 INFO:teuthology.orchestra.run.smithi119.stdout: "scrubs": {} 2024-04-25T10:17:18.417 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:17:18.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.410+0000 7f4ccf000640 1 -- 172.21.15.119:0/1492408849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ca406b4d0 msgr2=0x7f4ca406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.410+0000 7f4ccf000640 1 --2- 172.21.15.119:0/1492408849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ca406b4d0 0x7f4ca406d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8034000 tx=0x7f4cb802e000 comp rx=0 tx=0).stop 2024-04-25T10:17:18.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.410+0000 7f4ccf000640 1 -- 172.21.15.119:0/1492408849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cc80cef10 msgr2=0x7f4cc80b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.410+0000 7f4ccf000640 1 --2- 172.21.15.119:0/1492408849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cc80cef10 0x7f4cc80b1900 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000b4a0 tx=0x7f4cb000b960 comp rx=0 tx=0).stop 2024-04-25T10:17:18.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.411+0000 7f4ccf000640 1 -- 172.21.15.119:0/1492408849 shutdown_connections 2024-04-25T10:17:18.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.411+0000 7f4ccf000640 1 --2- 172.21.15.119:0/1492408849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ca406b4d0 0x7f4ca406d990 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.411+0000 7f4ccf000640 1 --2- 172.21.15.119:0/1492408849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4cc8144e70 0x7f4cc813bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.411+0000 7f4ccf000640 1 --2- 172.21.15.119:0/1492408849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4cc80cef10 0x7f4cc80b1900 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.411+0000 7f4ccf000640 1 --2- 172.21.15.119:0/1492408849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc806d7a0 0x7f4cc81e42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.411+0000 7f4ccf000640 1 -- 172.21.15.119:0/1492408849 >> 172.21.15.119:0/1492408849 conn(0x7f4cc806aba0 msgr2=0x7f4cc806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.411+0000 7f4ccf000640 1 -- 172.21.15.119:0/1492408849 shutdown_connections 2024-04-25T10:17:18.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.411+0000 7f4ccf000640 1 -- 172.21.15.119:0/1492408849 wait complete. 2024-04-25T10:17:18.432 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:17:18.533 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:17:18.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.739+0000 7f3a62e00640 1 Processor -- start 2024-04-25T10:17:18.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.739+0000 7f3a62e00640 1 -- start start 2024-04-25T10:17:18.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.741+0000 7f3a62e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3a5c0b36a0 0x7f3a5c0acb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.741+0000 7f3a62e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5c15e120 con 0x7f3a5c06b2b0 2024-04-25T10:17:18.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.741+0000 7f3a62e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5c15e400 con 0x7f3a5c0ada20 2024-04-25T10:17:18.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.741+0000 7f3a62e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a5c15dec0 con 0x7f3a5c0b36a0 2024-04-25T10:17:18.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.741+0000 7f3a61000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3a5c06b2b0 0x7f3a5c0b2f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57666/0 (socket says 172.21.15.119:57666) 2024-04-25T10:17:18.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.741+0000 7f3a61000640 1 -- 172.21.15.119:0/1088266544 learned_addr learned my addr 172.21.15.119:0/1088266544 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:18.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a62400640 1 --2- 172.21.15.119:0/1088266544 >> v2:172.21.15.132:3300/0 conn(0x7f3a5c0b36a0 0x7f3a5c0acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 -- 172.21.15.119:0/1088266544 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099291093 0 0) 0x7f3a5c15e120 con 0x7f3a5c06b2b0 2024-04-25T10:17:18.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 -- 172.21.15.119:0/1088266544 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a44003520 con 0x7f3a5c06b2b0 2024-04-25T10:17:18.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 -- 172.21.15.119:0/1088266544 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1595994596 0 0) 0x7f3a44003520 con 0x7f3a5c06b2b0 2024-04-25T10:17:18.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 -- 172.21.15.119:0/1088266544 >> v2:172.21.15.132:3300/0 conn(0x7f3a5c0b36a0 msgr2=0x7f3a5c0acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 --2- 172.21.15.119:0/1088266544 >> v2:172.21.15.132:3300/0 conn(0x7f3a5c0b36a0 0x7f3a5c0acb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 -- 172.21.15.119:0/1088266544 >> v1:172.21.15.132:6789/0 conn(0x7f3a5c0ada20 legacy=0x7f3a5c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 -- 172.21.15.119:0/1088266544 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c15e6e0 con 0x7f3a5c06b2b0 2024-04-25T10:17:18.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 -- 172.21.15.119:0/1088266544 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f3a48001670 con 0x7f3a5c06b2b0 2024-04-25T10:17:18.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 -- 172.21.15.119:0/1088266544 >> v1:172.21.15.119:6789/0 conn(0x7f3a5c06b2b0 legacy=0x7f3a5c0b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 --2- 172.21.15.119:0/1088266544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a440039b0 0x7f3a44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 --2- 172.21.15.119:0/1088266544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a44004800 0x7f3a44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 --2- 172.21.15.119:0/1088266544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a440051c0 0x7f3a4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 -- 172.21.15.119:0/1088266544 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c15dec0 con 0x7f3a440051c0 2024-04-25T10:17:18.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 -- 172.21.15.119:0/1088266544 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4400c060 con 0x7f3a440039b0 2024-04-25T10:17:18.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.742+0000 7f3a5be00640 1 -- 172.21.15.119:0/1088266544 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4400c260 con 0x7f3a44004800 2024-04-25T10:17:18.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.743+0000 7f3a62400640 1 --2- 172.21.15.119:0/1088266544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a440051c0 0x7f3a4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.743+0000 7f3a61a00640 1 --2- 172.21.15.119:0/1088266544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a440039b0 0x7f3a44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.743+0000 7f3a62400640 1 -- 172.21.15.119:0/1088266544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a44004800 msgr2=0x7f3a44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.743+0000 7f3a62400640 1 --2- 172.21.15.119:0/1088266544 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a44004800 0x7f3a44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.743+0000 7f3a62400640 1 -- 172.21.15.119:0/1088266544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a440039b0 msgr2=0x7f3a44003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:18.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.743+0000 7f3a62400640 1 --2- 172.21.15.119:0/1088266544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a440039b0 0x7f3a44003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.743+0000 7f3a62400640 1 -- 172.21.15.119:0/1088266544 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a44003520 con 0x7f3a440051c0 2024-04-25T10:17:18.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.744+0000 7f3a62400640 1 -- 172.21.15.119:0/1088266544 reap_dead start 2024-04-25T10:17:18.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.744+0000 7f3a62400640 1 --2- 172.21.15.119:0/1088266544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a440051c0 0x7f3a4400b9c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009fc0 tx=0x7f3a44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29fa1f612091ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.744+0000 7f3a5be00640 1 -- 172.21.15.119:0/1088266544 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54035020 con 0x7f3a440051c0 2024-04-25T10:17:18.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.744+0000 7f3a5be00640 1 -- 172.21.15.119:0/1088266544 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a54003b90 con 0x7f3a440051c0 2024-04-25T10:17:18.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.744+0000 7f3a5be00640 1 -- 172.21.15.119:0/1088266544 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54009c10 con 0x7f3a440051c0 2024-04-25T10:17:18.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.745+0000 7f3a62e00640 1 -- 172.21.15.119:0/1088266544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a440051c0 msgr2=0x7f3a4400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.745+0000 7f3a62e00640 1 --2- 172.21.15.119:0/1088266544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a440051c0 0x7f3a4400b9c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009fc0 tx=0x7f3a44005830 comp rx=0 tx=0).stop 2024-04-25T10:17:18.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.745+0000 7f3a62e00640 1 -- 172.21.15.119:0/1088266544 shutdown_connections 2024-04-25T10:17:18.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.745+0000 7f3a62e00640 1 --2- 172.21.15.119:0/1088266544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a440051c0 0x7f3a4400b9c0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.745+0000 7f3a62e00640 1 -- 172.21.15.119:0/1088266544 >> 172.21.15.119:0/1088266544 conn(0x7f3a5c066ff0 msgr2=0x7f3a5c0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:18.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.745+0000 7f3a62e00640 1 -- 172.21.15.119:0/1088266544 shutdown_connections 2024-04-25T10:17:18.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.745+0000 7f3a62e00640 1 -- 172.21.15.119:0/1088266544 wait complete. 2024-04-25T10:17:18.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.747+0000 7f3a62e00640 1 Processor -- start 2024-04-25T10:17:18.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.747+0000 7f3a62e00640 1 -- start start 2024-04-25T10:17:18.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.748+0000 7f3a62e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a440051c0 0x7f3a5c150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.748+0000 7f3a62e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a5c0c93f0 0x7f3a5c0c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.748+0000 7f3a62e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5c0c4df0 0x7f3a5c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.748+0000 7f3a62e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0ade00 con 0x7f3a440051c0 2024-04-25T10:17:18.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.748+0000 7f3a62e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c15eca0 con 0x7f3a5c0c4df0 2024-04-25T10:17:18.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.748+0000 7f3a62e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0acb20 con 0x7f3a5c0c93f0 2024-04-25T10:17:18.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.748+0000 7f3a61a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a440051c0 0x7f3a5c150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.748+0000 7f3a61a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a440051c0 0x7f3a5c150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46960/0 (socket says 172.21.15.119:46960) 2024-04-25T10:17:18.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.748+0000 7f3a61a00640 1 -- 172.21.15.119:0/1623986362 learned_addr learned my addr 172.21.15.119:0/1623986362 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:18.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.749+0000 7f3a61000640 1 --2- 172.21.15.119:0/1623986362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a5c0c93f0 0x7f3a5c0c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.749+0000 7f3a62400640 1 --2- 172.21.15.119:0/1623986362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5c0c4df0 0x7f3a5c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.749+0000 7f3a61a00640 1 -- 172.21.15.119:0/1623986362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a5c0c93f0 msgr2=0x7f3a5c0c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.749+0000 7f3a61a00640 1 --2- 172.21.15.119:0/1623986362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a5c0c93f0 0x7f3a5c0c9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.749+0000 7f3a61a00640 1 -- 172.21.15.119:0/1623986362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5c0c4df0 msgr2=0x7f3a5c0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:18.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.749+0000 7f3a61a00640 1 --2- 172.21.15.119:0/1623986362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5c0c4df0 0x7f3a5c0c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:18.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.749+0000 7f3a61a00640 1 -- 172.21.15.119:0/1623986362 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c0c5a00 con 0x7f3a440051c0 2024-04-25T10:17:18.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.749+0000 7f3a61a00640 1 --2- 172.21.15.119:0/1623986362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a440051c0 0x7f3a5c150b30 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000e4d0 tx=0x7f3a5000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.749+0000 7f3a59600640 1 -- 172.21.15.119:0/1623986362 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a500052e0 con 0x7f3a440051c0 2024-04-25T10:17:18.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.749+0000 7f3a59600640 1 -- 172.21.15.119:0/1623986362 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a50018070 con 0x7f3a440051c0 2024-04-25T10:17:18.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.749+0000 7f3a59600640 1 -- 172.21.15.119:0/1623986362 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50003810 con 0x7f3a440051c0 2024-04-25T10:17:18.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.749+0000 7f3a62e00640 1 -- 172.21.15.119:0/1623986362 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5c0c5ce0 con 0x7f3a440051c0 2024-04-25T10:17:18.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.750+0000 7f3a62e00640 1 -- 172.21.15.119:0/1623986362 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5c0c6ea0 con 0x7f3a440051c0 2024-04-25T10:17:18.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.751+0000 7f3a62e00640 1 -- 172.21.15.119:0/1623986362 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5c0c5ee0 con 0x7f3a440051c0 2024-04-25T10:17:18.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.752+0000 7f3a59600640 1 -- 172.21.15.119:0/1623986362 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a500039b0 con 0x7f3a440051c0 2024-04-25T10:17:18.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.753+0000 7f3a59600640 1 --2- 172.21.15.119:0/1623986362 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3a3c06b500 0x7f3a3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:18.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.753+0000 7f3a59600640 1 -- 172.21.15.119:0/1623986362 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3a50089ef0 con 0x7f3a440051c0 2024-04-25T10:17:18.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.753+0000 7f3a61000640 1 --2- 172.21.15.119:0/1623986362 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3a3c06b500 0x7f3a3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:18.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.753+0000 7f3a61000640 1 --2- 172.21.15.119:0/1623986362 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3a3c06b500 0x7f3a3c06d9c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3a480011e0 tx=0x7f3a48009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:18.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.757+0000 7f3a59600640 1 -- 172.21.15.119:0/1623986362 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a50058c70 con 0x7f3a440051c0 2024-04-25T10:17:18.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:18.995+0000 7f3a62e00640 1 -- 172.21.15.119:0/1623986362 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3a5c0749b0 con 0x7f3a440051c0 2024-04-25T10:17:18.999 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:17:18.996+0000 7f8c64600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:17:19.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.000+0000 7f3a59600640 1 -- 172.21.15.119:0/1623986362 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3a50058570 con 0x7f3a440051c0 2024-04-25T10:17:19.003 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:17:19.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.003+0000 7f3a62e00640 1 -- 172.21.15.119:0/1623986362 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3a3c06b500 msgr2=0x7f3a3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.003+0000 7f3a62e00640 1 --2- 172.21.15.119:0/1623986362 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3a3c06b500 0x7f3a3c06d9c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3a480011e0 tx=0x7f3a48009040 comp rx=0 tx=0).stop 2024-04-25T10:17:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.003+0000 7f3a62e00640 1 -- 172.21.15.119:0/1623986362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a440051c0 msgr2=0x7f3a5c150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.003+0000 7f3a62e00640 1 --2- 172.21.15.119:0/1623986362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a440051c0 0x7f3a5c150b30 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000e4d0 tx=0x7f3a5000e990 comp rx=0 tx=0).stop 2024-04-25T10:17:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.003+0000 7f3a62e00640 1 -- 172.21.15.119:0/1623986362 shutdown_connections 2024-04-25T10:17:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.003+0000 7f3a62e00640 1 --2- 172.21.15.119:0/1623986362 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3a3c06b500 0x7f3a3c06d9c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.003+0000 7f3a62e00640 1 --2- 172.21.15.119:0/1623986362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a5c0c4df0 0x7f3a5c0c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.003+0000 7f3a62e00640 1 --2- 172.21.15.119:0/1623986362 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a5c0c93f0 0x7f3a5c0c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.003+0000 7f3a62e00640 1 --2- 172.21.15.119:0/1623986362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a440051c0 0x7f3a5c150b30 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.003+0000 7f3a62e00640 1 -- 172.21.15.119:0/1623986362 >> 172.21.15.119:0/1623986362 conn(0x7f3a5c066ff0 msgr2=0x7f3a5c0ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:19.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.004+0000 7f3a62e00640 1 -- 172.21.15.119:0/1623986362 shutdown_connections 2024-04-25T10:17:19.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.004+0000 7f3a62e00640 1 -- 172.21.15.119:0/1623986362 wait complete. 2024-04-25T10:17:19.019 INFO:tasks.ceph.mds.a.smithi119.stderr: -1295> 2024-04-25T10:16:47.741+0000 7f7a1dc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:17:19.019 INFO:tasks.ceph.mds.a.smithi119.stderr: -1140> 2024-04-25T10:16:47.879+0000 7f7a18c00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:17:19.019 INFO:tasks.ceph.mds.a.smithi119.stderr: -1137> 2024-04-25T10:16:47.879+0000 7f7a18c00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T10:17:19.020 INFO:tasks.ceph.mds.a.smithi119.stderr: -1121> 2024-04-25T10:16:47.879+0000 7f7a18c00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:17:19.020 INFO:tasks.ceph.mds.a.smithi119.stderr: -1118> 2024-04-25T10:16:47.879+0000 7f7a18c00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T10:17:19.034 DEBUG:teuthology.orchestra.run.smithi119:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-25T10:17:19.081 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:17:19.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.120+0000 7f4678e1d280 1 -- start start 2024-04-25T10:17:19.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.121+0000 7f4678e1d280 1 Processor -- start 2024-04-25T10:17:19.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.121+0000 7f4678e1d280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561414d78800 0x561414013b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.121+0000 7f4678e1d280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414d79c00 0x561414013080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.121+0000 7f4678e1d280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561414d79400 0x561414012b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.121+0000 7f4678e1d280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561414d7a480 con 0x561414d79c00 2024-04-25T10:17:19.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.121+0000 7f4678e1d280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561414d7a600 con 0x561414d79400 2024-04-25T10:17:19.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.121+0000 7f4678e1d280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561414d7a780 con 0x561414d78800 2024-04-25T10:17:19.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.122+0000 7f4675800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414d79c00 0x561414013080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.122+0000 7f4675800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414d79c00 0x561414013080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46992/0 (socket says 172.21.15.119:46992) 2024-04-25T10:17:19.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.122+0000 7f4675800640 1 -- 172.21.15.119:0/2650755641 learned_addr learned my addr 172.21.15.119:0/2650755641 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:19.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.122+0000 7f4676200640 1 --2- 172.21.15.119:0/2650755641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561414d78800 0x561414013b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.122+0000 7f4676c00640 1 --2- 172.21.15.119:0/2650755641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561414d79400 0x561414012b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.122+0000 7f4675800640 1 -- 172.21.15.119:0/2650755641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561414d78800 msgr2=0x561414013b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:19.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.122+0000 7f4675800640 1 --2- 172.21.15.119:0/2650755641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561414d78800 0x561414013b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.122+0000 7f4675800640 1 -- 172.21.15.119:0/2650755641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561414d79400 msgr2=0x561414012b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:19.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.122+0000 7f4675800640 1 --2- 172.21.15.119:0/2650755641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561414d79400 0x561414012b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.122+0000 7f4675800640 1 -- 172.21.15.119:0/2650755641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561413fda780 con 0x561414d79c00 2024-04-25T10:17:19.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.122+0000 7f4676c00640 1 --2- 172.21.15.119:0/2650755641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561414d79400 0x561414012b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:19.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.122+0000 7f4675800640 1 --2- 172.21.15.119:0/2650755641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414d79c00 0x561414013080 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x561414dcb200 tx=0x561414d06600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0e2d5f3b547165 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.123+0000 7f4673000640 1 -- 172.21.15.119:0/2650755641 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561413fcc9c0 con 0x561414d79c00 2024-04-25T10:17:19.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.123+0000 7f4673000640 1 -- 172.21.15.119:0/2650755641 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561413fd2700 con 0x561414d79c00 2024-04-25T10:17:19.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.123+0000 7f4673000640 1 -- 172.21.15.119:0/2650755641 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561413fccb60 con 0x561414d79c00 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.123+0000 7f4678e1d280 1 -- 172.21.15.119:0/2650755641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561413fda000 con 0x561414d79c00 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.123+0000 7f4678e1d280 1 -- 172.21.15.119:0/2650755641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x561413fdba40 con 0x561414d79c00 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.123+0000 7f4678e1d280 4 waiting for MDS map... 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.123+0000 7f4673000640 1 -- 172.21.15.119:0/2650755641 <== mon.0 v2:172.21.15.119:3300/0 4 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x561414e74400 con 0x561414d79c00 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.123+0000 7f4673000640 1 -- 172.21.15.119:0/2650755641 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 89) v1 ==== 2723+0+0 (secure 0 0 0) 0x561414001180 con 0x561414d79c00 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.123+0000 7f4678e1d280 4 Got MDS map 89 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.123+0000 7f4678e1d280 10 main: main 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.123+0000 7f4678e1d280 4 main: connecting to RADOS... 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.124+0000 7f4678e1d280 1 Processor -- start 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.124+0000 7f4678e1d280 1 -- start start 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.125+0000 7f4678e1d280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414e76c00 0x561414014100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.125+0000 7f4678e1d280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561414e77000 0x561414014680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.125+0000 7f4678e1d280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561414e77400 0x561414014c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.125+0000 7f4678e1d280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561414008180 con 0x561414e76c00 2024-04-25T10:17:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.125+0000 7f4678e1d280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561414d7af00 con 0x561414e77400 2024-04-25T10:17:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.125+0000 7f4678e1d280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561414d7b080 con 0x561414e77000 2024-04-25T10:17:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.125+0000 7f4676c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414e76c00 0x561414014100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.125+0000 7f4676c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414e76c00 0x561414014100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47000/0 (socket says 172.21.15.119:47000) 2024-04-25T10:17:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.125+0000 7f4676c00640 1 -- 172.21.15.119:0/3960017830 learned_addr learned my addr 172.21.15.119:0/3960017830 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.126+0000 7f4676c00640 1 -- 172.21.15.119:0/3960017830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561414e77000 msgr2=0x561414014680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.126+0000 7f4676c00640 1 --2- 172.21.15.119:0/3960017830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561414e77000 0x561414014680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.126+0000 7f4676c00640 1 -- 172.21.15.119:0/3960017830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561414e77400 msgr2=0x561414014c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.126+0000 7f4676c00640 1 --2- 172.21.15.119:0/3960017830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561414e77400 0x561414014c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.126+0000 7f4676c00640 1 -- 172.21.15.119:0/3960017830 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561413fdb860 con 0x561414e76c00 2024-04-25T10:17:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.126+0000 7f4676c00640 1 --2- 172.21.15.119:0/3960017830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414e76c00 0x561414014100 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x561414d24420 tx=0x561414d064e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53993bcb941e740 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.126+0000 7f4670800640 1 -- 172.21.15.119:0/3960017830 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561413fcc680 con 0x561414e76c00 2024-04-25T10:17:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.126+0000 7f4670800640 1 -- 172.21.15.119:0/3960017830 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561413fd2e00 con 0x561414e76c00 2024-04-25T10:17:19.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.126+0000 7f4670800640 1 -- 172.21.15.119:0/3960017830 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561413fcc9c0 con 0x561414e76c00 2024-04-25T10:17:19.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.127+0000 7f4678e1d280 1 -- 172.21.15.119:0/3960017830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414e76c00 msgr2=0x561414014100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.127+0000 7f4678e1d280 1 --2- 172.21.15.119:0/3960017830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414e76c00 0x561414014100 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x561414d24420 tx=0x561414d064e0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.127+0000 7f4678e1d280 1 -- 172.21.15.119:0/3960017830 shutdown_connections 2024-04-25T10:17:19.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.127+0000 7f4678e1d280 1 --2- 172.21.15.119:0/3960017830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561414e77400 0x561414014c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.127+0000 7f4678e1d280 1 --2- 172.21.15.119:0/3960017830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561414e77000 0x561414014680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.127+0000 7f4678e1d280 1 --2- 172.21.15.119:0/3960017830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414e76c00 0x561414014100 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.127+0000 7f4678e1d280 1 -- 172.21.15.119:0/3960017830 >> 172.21.15.119:0/3960017830 conn(0x561414d79800 msgr2=0x56141408b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.127+0000 7f4678e1d280 1 -- 172.21.15.119:0/3960017830 shutdown_connections 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.127+0000 7f4678e1d280 1 -- 172.21.15.119:0/3960017830 wait complete. 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.128+0000 7f4678e1d280 1 Processor -- start 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.128+0000 7f4678e1d280 1 -- start start 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4678e1d280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414e76c00 0x561414014100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4678e1d280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561414e77000 0x561414014680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4678e1d280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561414e77400 0x561414014c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4678e1d280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561414d7b380 con 0x561414e76c00 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4678e1d280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561414d7b200 con 0x561414e77400 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4678e1d280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561414d7b500 con 0x561414e77000 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4676c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414e76c00 0x561414014100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4676c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414e76c00 0x561414014100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47014/0 (socket says 172.21.15.119:47014) 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4676c00640 1 -- 172.21.15.119:0/50611420 learned_addr learned my addr 172.21.15.119:0/50611420 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4676c00640 1 -- 172.21.15.119:0/50611420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561414e77000 msgr2=0x561414014680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4676c00640 1 --2- 172.21.15.119:0/50611420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561414e77000 0x561414014680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4676c00640 1 -- 172.21.15.119:0/50611420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561414e77400 msgr2=0x561414014c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4676c00640 1 --2- 172.21.15.119:0/50611420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561414e77400 0x561414014c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4676c00640 1 -- 172.21.15.119:0/50611420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561413fdb4a0 con 0x561414e76c00 2024-04-25T10:17:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.129+0000 7f4676c00640 1 --2- 172.21.15.119:0/50611420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414e76c00 0x561414014100 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x561414ed0870 tx=0x561414d061e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.130+0000 7f466ea00640 1 -- 172.21.15.119:0/50611420 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561413fccb60 con 0x561414e76c00 2024-04-25T10:17:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.130+0000 7f466ea00640 1 -- 172.21.15.119:0/50611420 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561413fd2c40 con 0x561414e76c00 2024-04-25T10:17:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.130+0000 7f466ea00640 1 -- 172.21.15.119:0/50611420 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561413fcc820 con 0x561414e76c00 2024-04-25T10:17:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.130+0000 7f4678e1d280 1 -- 172.21.15.119:0/50611420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561413fda960 con 0x561414e76c00 2024-04-25T10:17:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.130+0000 7f4678e1d280 1 -- 172.21.15.119:0/50611420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561413fda780 con 0x561414e76c00 2024-04-25T10:17:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.130+0000 7f4678e1d280 4 main: resolving pool 14 2024-04-25T10:17:19.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.132+0000 7f466ea00640 1 -- 172.21.15.119:0/50611420 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x561414ee2c00 con 0x561414e76c00 2024-04-25T10:17:19.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.132+0000 7f466ea00640 1 --2- 172.21.15.119:0/50611420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x561414304000 0x561414015700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.132+0000 7f466ea00640 1 -- 172.21.15.119:0/50611420 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x561414ee0600 con 0x561414e76c00 2024-04-25T10:17:19.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.132+0000 7f4678e1d280 4 main: creating IoCtx.. 2024-04-25T10:17:19.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.132+0000 7f4678e1d280 1 --2- 172.21.15.119:0/50611420 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x561414e77800 0x561414015180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.132+0000 7f4678e1d280 1 -- 172.21.15.119:0/50611420 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v9 -- 0x561414e77c00 con 0x561414e77800 2024-04-25T10:17:19.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.133+0000 7f4676200640 1 --2- 172.21.15.119:0/50611420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x561414304000 0x561414015700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.133+0000 7f4675800640 1 --2- 172.21.15.119:0/50611420 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x561414e77800 0x561414015180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.133+0000 7f4676200640 1 --2- 172.21.15.119:0/50611420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x561414304000 0x561414015700 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x561414037b30 tx=0x561414ee65a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.133+0000 7f4675800640 1 --2- 172.21.15.119:0/50611420 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x561414e77800 0x561414015180 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.136+0000 7f4675800640 1 -- 172.21.15.119:0/50611420 <== osd.2 v2:172.21.15.119:6824/1001762963 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x56141409c000 con 0x561414e77800 2024-04-25T10:17:19.140 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:17:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "0": { 2024-04-25T10:17:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "data": { 2024-04-25T10:17:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "version": 5, 2024-04-25T10:17:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "inotable": { 2024-04-25T10:17:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "projected_free": [ 2024-04-25T10:17:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:19.140 INFO:teuthology.orchestra.run.smithi119.stdout: "start": 1099511627779, 2024-04-25T10:17:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "len": 1099511627773 2024-04-25T10:17:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-04-25T10:17:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "free": [ 2024-04-25T10:17:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "start": 1099511627779, 2024-04-25T10:17:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "len": 1099511627773 2024-04-25T10:17:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-25T10:17:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: "result": 0 2024-04-25T10:17:19.141 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:19.141 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:17:19.142 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:19.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.136+0000 7f4678e1d280 1 -- 172.21.15.119:0/50611420 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x561414e77800 msgr2=0x561414015180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.136+0000 7f4678e1d280 1 --2- 172.21.15.119:0/50611420 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x561414e77800 0x561414015180 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.136+0000 7f4678e1d280 1 -- 172.21.15.119:0/50611420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x561414304000 msgr2=0x561414015700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.136+0000 7f4678e1d280 1 --2- 172.21.15.119:0/50611420 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x561414304000 0x561414015700 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x561414037b30 tx=0x561414ee65a0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4678e1d280 1 -- 172.21.15.119:0/50611420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414e76c00 msgr2=0x561414014100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4678e1d280 1 --2- 172.21.15.119:0/50611420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414e76c00 0x561414014100 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x561414ed0870 tx=0x561414d061e0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4676200640 1 -- 172.21.15.119:0/50611420 reap_dead start 2024-04-25T10:17:19.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4678e1d280 1 -- 172.21.15.119:0/50611420 shutdown_connections 2024-04-25T10:17:19.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4678e1d280 1 -- 172.21.15.119:0/50611420 >> 172.21.15.119:0/50611420 conn(0x561414d79800 msgr2=0x56141408b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:19.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4678e1d280 1 -- 172.21.15.119:0/50611420 shutdown_connections 2024-04-25T10:17:19.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4678e1d280 1 -- 172.21.15.119:0/50611420 wait complete. 2024-04-25T10:17:19.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4678e1d280 1 -- 172.21.15.119:0/2650755641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414d79c00 msgr2=0x561414013080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4678e1d280 1 --2- 172.21.15.119:0/2650755641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414d79c00 0x561414013080 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x561414dcb200 tx=0x561414d06600 comp rx=0 tx=0).stop 2024-04-25T10:17:19.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4678e1d280 1 -- 172.21.15.119:0/2650755641 shutdown_connections 2024-04-25T10:17:19.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4678e1d280 1 --2- 172.21.15.119:0/2650755641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561414d79400 0x561414012b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4678e1d280 1 --2- 172.21.15.119:0/2650755641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561414d79c00 0x561414013080 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4678e1d280 1 --2- 172.21.15.119:0/2650755641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561414d78800 0x561414013b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4678e1d280 1 -- 172.21.15.119:0/2650755641 >> 172.21.15.119:0/2650755641 conn(0x561413fa7000 msgr2=0x561414089800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:19.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.137+0000 7f4678e1d280 1 -- 172.21.15.119:0/2650755641 shutdown_connections 2024-04-25T10:17:19.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.138+0000 7f4678e1d280 1 -- 172.21.15.119:0/2650755641 wait complete. 2024-04-25T10:17:19.146 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.112021, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-25T10:17:19.147 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:17:19.147 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:17:19.234 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:17:19.350 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:17:19.350 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:17:19.351 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:19.406 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:19.406 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:17:19.406 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:19.462 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:19.463 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:17:19.463 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:19.520 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:19.520 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:17:19.520 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:19.576 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:19.576 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:19.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.782+0000 7f1af8800640 1 Processor -- start 2024-04-25T10:17:19.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.783+0000 7f1af8800640 1 -- start start 2024-04-25T10:17:19.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.784+0000 7f1af8800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af006e8f0 0x7f1af015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.784+0000 7f1af8800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1af015c820 0x7f1af01654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.784+0000 7f1af8800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af00ce6c0 con 0x7f1af015bf00 2024-04-25T10:17:19.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.784+0000 7f1af8800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af01455e0 con 0x7f1af015c820 2024-04-25T10:17:19.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.784+0000 7f1af8800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af006d450 con 0x7f1af006e8f0 2024-04-25T10:17:19.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.785+0000 7f1af6800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1af015c820 0x7f1af01654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.785+0000 7f1af5400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af006e8f0 0x7f1af015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.785+0000 7f1af5400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af006e8f0 0x7f1af015c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:44766/0 (socket says 172.21.15.119:44766) 2024-04-25T10:17:19.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.785+0000 7f1af5400640 1 -- 172.21.15.119:0/3819956225 learned_addr learned my addr 172.21.15.119:0/3819956225 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:19.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.785+0000 7f1af5e00640 1 --1- 172.21.15.119:0/3819956225 >> v1:172.21.15.132:6789/0 conn(0x7f1af015bf00 0x7f1af00cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:40862/0 (socket says 172.21.15.119:40862) 2024-04-25T10:17:19.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.786+0000 7f1af5400640 1 -- 172.21.15.119:0/3819956225 >> v2:172.21.15.132:3300/0 conn(0x7f1af015c820 msgr2=0x7f1af01654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.786+0000 7f1af5400640 1 --2- 172.21.15.119:0/3819956225 >> v2:172.21.15.132:3300/0 conn(0x7f1af015c820 0x7f1af01654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.786+0000 7f1af5400640 1 -- 172.21.15.119:0/3819956225 >> v1:172.21.15.132:6789/0 conn(0x7f1af015bf00 legacy=0x7f1af00cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.786+0000 7f1af5400640 1 -- 172.21.15.119:0/3819956225 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af00cc4e0 con 0x7f1af006e8f0 2024-04-25T10:17:19.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.787+0000 7f1af5400640 1 --2- 172.21.15.119:0/3819956225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af006e8f0 0x7f1af015c2e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009870 tx=0x7f1ae002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c03b947c0e37b584 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.788+0000 7f1af4a00640 1 -- 172.21.15.119:0/3819956225 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0037070 con 0x7f1af006e8f0 2024-04-25T10:17:19.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.788+0000 7f1af4a00640 1 -- 172.21.15.119:0/3819956225 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae0032440 con 0x7f1af006e8f0 2024-04-25T10:17:19.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.788+0000 7f1af4a00640 1 -- 172.21.15.119:0/3819956225 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0032810 con 0x7f1af006e8f0 2024-04-25T10:17:19.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.789+0000 7f1af8800640 1 -- 172.21.15.119:0/3819956225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af006e8f0 msgr2=0x7f1af015c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.789+0000 7f1af8800640 1 --2- 172.21.15.119:0/3819956225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af006e8f0 0x7f1af015c2e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009870 tx=0x7f1ae002a250 comp rx=0 tx=0).stop 2024-04-25T10:17:19.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.789+0000 7f1af8800640 1 -- 172.21.15.119:0/3819956225 shutdown_connections 2024-04-25T10:17:19.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.789+0000 7f1af8800640 1 --2- 172.21.15.119:0/3819956225 >> v2:172.21.15.132:3300/0 conn(0x7f1af015c820 0x7f1af01654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.789+0000 7f1af8800640 1 --2- 172.21.15.119:0/3819956225 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af006e8f0 0x7f1af015c2e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.789+0000 7f1af8800640 1 -- 172.21.15.119:0/3819956225 >> 172.21.15.119:0/3819956225 conn(0x7f1af006aba0 msgr2=0x7f1af006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:19.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.789+0000 7f1af8800640 1 -- 172.21.15.119:0/3819956225 shutdown_connections 2024-04-25T10:17:19.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.789+0000 7f1af8800640 1 -- 172.21.15.119:0/3819956225 wait complete. 2024-04-25T10:17:19.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.791+0000 7f1af8800640 1 Processor -- start 2024-04-25T10:17:19.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.792+0000 7f1af8800640 1 -- start start 2024-04-25T10:17:19.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.793+0000 7f1af8800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af006e8f0 0x7f1af0137ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.793+0000 7f1af8800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af015bf00 0x7f1af013c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.793+0000 7f1af8800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1af015c820 0x7f1af013c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.793+0000 7f1af8800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af00ccd70 con 0x7f1af015c820 2024-04-25T10:17:19.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.793+0000 7f1af8800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af00cca90 con 0x7f1af006e8f0 2024-04-25T10:17:19.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.793+0000 7f1af8800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af00cd050 con 0x7f1af015bf00 2024-04-25T10:17:19.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.793+0000 7f1af6800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1af015c820 0x7f1af013c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.793+0000 7f1af6800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1af015c820 0x7f1af013c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47030/0 (socket says 172.21.15.119:47030) 2024-04-25T10:17:19.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.793+0000 7f1af6800640 1 -- 172.21.15.119:0/251696387 learned_addr learned my addr 172.21.15.119:0/251696387 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:19.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.793+0000 7f1af5400640 1 --2- 172.21.15.119:0/251696387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af015bf00 0x7f1af013c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.793+0000 7f1af5e00640 1 --2- 172.21.15.119:0/251696387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af006e8f0 0x7f1af0137ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.794+0000 7f1af6800640 1 -- 172.21.15.119:0/251696387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af015bf00 msgr2=0x7f1af013c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.794+0000 7f1af6800640 1 --2- 172.21.15.119:0/251696387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af015bf00 0x7f1af013c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.794+0000 7f1af6800640 1 -- 172.21.15.119:0/251696387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af006e8f0 msgr2=0x7f1af0137ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:19.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.794+0000 7f1af6800640 1 --2- 172.21.15.119:0/251696387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af006e8f0 0x7f1af0137ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:19.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.794+0000 7f1af6800640 1 -- 172.21.15.119:0/251696387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af013afa0 con 0x7f1af015c820 2024-04-25T10:17:19.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.794+0000 7f1af6800640 1 --2- 172.21.15.119:0/251696387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1af015c820 0x7f1af013c610 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00efc0 tx=0x7f1aec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.794+0000 7f1aea000640 1 -- 172.21.15.119:0/251696387 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec018020 con 0x7f1af015c820 2024-04-25T10:17:19.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.794+0000 7f1aea000640 1 -- 172.21.15.119:0/251696387 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aec014480 con 0x7f1af015c820 2024-04-25T10:17:19.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.794+0000 7f1af8800640 1 -- 172.21.15.119:0/251696387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af00c9180 con 0x7f1af015c820 2024-04-25T10:17:19.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.794+0000 7f1aea000640 1 -- 172.21.15.119:0/251696387 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec014870 con 0x7f1af015c820 2024-04-25T10:17:19.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.795+0000 7f1af8800640 1 -- 172.21.15.119:0/251696387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af00c9740 con 0x7f1af015c820 2024-04-25T10:17:19.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.796+0000 7f1af8800640 1 -- 172.21.15.119:0/251696387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af00749b0 con 0x7f1af015c820 2024-04-25T10:17:19.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.797+0000 7f1aea000640 1 -- 172.21.15.119:0/251696387 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1aec005d10 con 0x7f1af015c820 2024-04-25T10:17:19.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.798+0000 7f1aea000640 1 --2- 172.21.15.119:0/251696387 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1ad406b500 0x7f1ad406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:19.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.798+0000 7f1aea000640 1 -- 172.21.15.119:0/251696387 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f1aec08ade0 con 0x7f1af015c820 2024-04-25T10:17:19.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.798+0000 7f1af5e00640 1 --2- 172.21.15.119:0/251696387 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1ad406b500 0x7f1ad406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:19.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.799+0000 7f1af5e00640 1 --2- 172.21.15.119:0/251696387 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1ad406b500 0x7f1ad406d9c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4000f60 tx=0x7f1ae4007c80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:19.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:19.802+0000 7f1aea000640 1 -- 172.21.15.119:0/251696387 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1aec022070 con 0x7f1af015c820 2024-04-25T10:17:20.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.038+0000 7f1af8800640 1 -- 172.21.15.119:0/251696387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1af00cf450 con 0x7f1af015c820 2024-04-25T10:17:20.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.039+0000 7f1aea000640 1 -- 172.21.15.119:0/251696387 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f1aec010040 con 0x7f1af015c820 2024-04-25T10:17:20.043 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:20.043 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5459,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3768348114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3768348114},{"type":"v1","addr":"172.21.15.132:6834","nonce":3768348114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":5465,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/543190780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":543190780},{"type":"v1","addr":"172.21.15.132:6835","nonce":543190780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1145229172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1145229172},{"type":"v1","addr":"172.21.15.119:6836","nonce":1145229172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1704625876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1704625876},{"type":"v1","addr":"172.21.15.119:6837","nonce":1704625876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:51.568312+0000","modified":"2024-04-25T10:17:18.997282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:17:20.043 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 90 2024-04-25T10:17:20.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.042+0000 7f1af8800640 1 -- 172.21.15.119:0/251696387 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1ad406b500 msgr2=0x7f1ad406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.042+0000 7f1af8800640 1 --2- 172.21.15.119:0/251696387 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1ad406b500 0x7f1ad406d9c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4000f60 tx=0x7f1ae4007c80 comp rx=0 tx=0).stop 2024-04-25T10:17:20.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.042+0000 7f1af8800640 1 -- 172.21.15.119:0/251696387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1af015c820 msgr2=0x7f1af013c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.042+0000 7f1af8800640 1 --2- 172.21.15.119:0/251696387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1af015c820 0x7f1af013c610 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00efc0 tx=0x7f1aec00c490 comp rx=0 tx=0).stop 2024-04-25T10:17:20.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.042+0000 7f1af8800640 1 -- 172.21.15.119:0/251696387 shutdown_connections 2024-04-25T10:17:20.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.042+0000 7f1af8800640 1 --2- 172.21.15.119:0/251696387 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1ad406b500 0x7f1ad406d9c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.042+0000 7f1af8800640 1 --2- 172.21.15.119:0/251696387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1af015c820 0x7f1af013c610 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.042+0000 7f1af8800640 1 --2- 172.21.15.119:0/251696387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1af015bf00 0x7f1af013c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.042+0000 7f1af8800640 1 --2- 172.21.15.119:0/251696387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1af006e8f0 0x7f1af0137ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.042+0000 7f1af8800640 1 -- 172.21.15.119:0/251696387 >> 172.21.15.119:0/251696387 conn(0x7f1af006aba0 msgr2=0x7f1af00ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:20.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.042+0000 7f1af8800640 1 -- 172.21.15.119:0/251696387 shutdown_connections 2024-04-25T10:17:20.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.042+0000 7f1af8800640 1 -- 172.21.15.119:0/251696387 wait complete. 2024-04-25T10:17:20.065 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:20.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.260+0000 7f0affc00640 1 Processor -- start 2024-04-25T10:17:20.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.260+0000 7f0affc00640 1 -- start start 2024-04-25T10:17:20.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.261+0000 7f0affc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0af80aecf0 0x7f0af80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.261+0000 7f0affc00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af80712c0 con 0x7f0af815ed70 2024-04-25T10:17:20.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.262+0000 7f0affc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af80715a0 con 0x7f0af815d980 2024-04-25T10:17:20.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.262+0000 7f0affc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af8071ed0 con 0x7f0af80aecf0 2024-04-25T10:17:20.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.262+0000 7f0afdc00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0af815ed70 0x7f0af80acf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57690/0 (socket says 172.21.15.119:57690) 2024-04-25T10:17:20.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.262+0000 7f0afdc00640 1 -- 172.21.15.119:0/4233322592 learned_addr learned my addr 172.21.15.119:0/4233322592 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:20.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.262+0000 7f0aefe00640 1 --2- 172.21.15.119:0/4233322592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0af80aecf0 0x7f0af80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.262+0000 7f0af7e00640 1 -- 172.21.15.119:0/4233322592 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25296783 0 0) 0x7f0af80712c0 con 0x7f0af815ed70 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.262+0000 7f0af7e00640 1 -- 172.21.15.119:0/4233322592 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0adc003520 con 0x7f0af815ed70 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0af7e00640 1 -- 172.21.15.119:0/4233322592 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 190342993 0 0) 0x7f0adc003520 con 0x7f0af815ed70 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0af7e00640 1 -- 172.21.15.119:0/4233322592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0af80aecf0 msgr2=0x7f0af80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0af7e00640 1 --2- 172.21.15.119:0/4233322592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0af80aecf0 0x7f0af80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0af7e00640 1 -- 172.21.15.119:0/4233322592 >> v1:172.21.15.132:6789/0 conn(0x7f0af815d980 legacy=0x7f0af80ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0af7e00640 1 -- 172.21.15.119:0/4233322592 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af8071880 con 0x7f0af815ed70 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0af7e00640 1 -- 172.21.15.119:0/4233322592 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f0af00019f0 con 0x7f0af815ed70 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0af7e00640 1 -- 172.21.15.119:0/4233322592 >> v1:172.21.15.119:6789/0 conn(0x7f0af815ed70 legacy=0x7f0af80acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0af7e00640 1 --2- 172.21.15.119:0/4233322592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0adc0039e0 0x7f0adc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0af7e00640 1 --2- 172.21.15.119:0/4233322592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0adc004800 0x7f0adc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0af7e00640 1 --2- 172.21.15.119:0/4233322592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0adc005210 0x7f0adc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0af7e00640 1 -- 172.21.15.119:0/4233322592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af8071ed0 con 0x7f0adc004800 2024-04-25T10:17:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0af7e00640 1 -- 172.21.15.119:0/4233322592 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc00bf30 con 0x7f0adc0039e0 2024-04-25T10:17:20.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0af7e00640 1 -- 172.21.15.119:0/4233322592 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc00c130 con 0x7f0adc005210 2024-04-25T10:17:20.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0aefe00640 1 --2- 172.21.15.119:0/4233322592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0adc004800 0x7f0adc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.263+0000 7f0afd200640 1 --2- 172.21.15.119:0/4233322592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0adc0039e0 0x7f0adc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.264+0000 7f0afdc00640 1 --2- 172.21.15.119:0/4233322592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0adc005210 0x7f0adc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.264+0000 7f0aefe00640 1 -- 172.21.15.119:0/4233322592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0adc005210 msgr2=0x7f0adc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:20.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.264+0000 7f0aefe00640 1 --2- 172.21.15.119:0/4233322592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0adc005210 0x7f0adc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.264+0000 7f0aefe00640 1 -- 172.21.15.119:0/4233322592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0adc0039e0 msgr2=0x7f0adc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:20.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.264+0000 7f0aefe00640 1 --2- 172.21.15.119:0/4233322592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0adc0039e0 0x7f0adc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.264+0000 7f0aefe00640 1 -- 172.21.15.119:0/4233322592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0adc003520 con 0x7f0adc004800 2024-04-25T10:17:20.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.264+0000 7f0afd200640 1 --2- 172.21.15.119:0/4233322592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0adc0039e0 0x7f0adc003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:20.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.264+0000 7f0afdc00640 1 --2- 172.21.15.119:0/4233322592 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0adc005210 0x7f0adc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.265+0000 7f0aefe00640 1 --2- 172.21.15.119:0/4233322592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0adc004800 0x7f0adc004c80 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f0af8070f70 tx=0x7f0adc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6d4d25b2153958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.265+0000 7f0af7e00640 1 -- 172.21.15.119:0/4233322592 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0036070 con 0x7f0adc004800 2024-04-25T10:17:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.265+0000 7f0af7e00640 1 -- 172.21.15.119:0/4233322592 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae0031400 con 0x7f0adc004800 2024-04-25T10:17:20.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.265+0000 7f0af7e00640 1 -- 172.21.15.119:0/4233322592 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0009690 con 0x7f0adc004800 2024-04-25T10:17:20.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.265+0000 7f0affc00640 1 -- 172.21.15.119:0/4233322592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0adc004800 msgr2=0x7f0adc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.265+0000 7f0affc00640 1 --2- 172.21.15.119:0/4233322592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0adc004800 0x7f0adc004c80 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f0af8070f70 tx=0x7f0adc005830 comp rx=0 tx=0).stop 2024-04-25T10:17:20.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.266+0000 7f0afdc00640 1 -- 172.21.15.119:0/4233322592 reap_dead start 2024-04-25T10:17:20.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.266+0000 7f0affc00640 1 -- 172.21.15.119:0/4233322592 shutdown_connections 2024-04-25T10:17:20.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.266+0000 7f0affc00640 1 -- 172.21.15.119:0/4233322592 >> 172.21.15.119:0/4233322592 conn(0x7f0af806aba0 msgr2=0x7f0af806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:20.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.266+0000 7f0affc00640 1 -- 172.21.15.119:0/4233322592 shutdown_connections 2024-04-25T10:17:20.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.266+0000 7f0affc00640 1 -- 172.21.15.119:0/4233322592 wait complete. 2024-04-25T10:17:20.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.267+0000 7f0affc00640 1 Processor -- start 2024-04-25T10:17:20.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.268+0000 7f0affc00640 1 -- start start 2024-04-25T10:17:20.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0affc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0adc004800 0x7f0af81560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0affc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0af81f3520 0x7f0af80aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0affc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0af81f39a0 0x7f0af80af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0affc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80cf400 con 0x7f0af81f3520 2024-04-25T10:17:20.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0affc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80cf280 con 0x7f0adc004800 2024-04-25T10:17:20.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0affc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80cf580 con 0x7f0af81f39a0 2024-04-25T10:17:20.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0aefe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0af81f3520 0x7f0af80aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0aefe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0af81f3520 0x7f0af80aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47056/0 (socket says 172.21.15.119:47056) 2024-04-25T10:17:20.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0aefe00640 1 -- 172.21.15.119:0/35238765 learned_addr learned my addr 172.21.15.119:0/35238765 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:20.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0afd200640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0adc004800 0x7f0af81560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0aefe00640 1 -- 172.21.15.119:0/35238765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0af81f39a0 msgr2=0x7f0af80af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0afdc00640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0af81f39a0 0x7f0af80af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0aefe00640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0af81f39a0 0x7f0af80af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0aefe00640 1 -- 172.21.15.119:0/35238765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0adc004800 msgr2=0x7f0af81560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0aefe00640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0adc004800 0x7f0af81560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0aefe00640 1 -- 172.21.15.119:0/35238765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80cebd0 con 0x7f0af81f3520 2024-04-25T10:17:20.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.269+0000 7f0afd200640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0adc004800 0x7f0af81560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:20.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.270+0000 7f0aefe00640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0af81f3520 0x7f0af80aee90 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f0ae002ab40 tx=0x7f0ae0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:20.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.270+0000 7f0af5600640 1 -- 172.21.15.119:0/35238765 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0040020 con 0x7f0af81f3520 2024-04-25T10:17:20.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.270+0000 7f0af5600640 1 -- 172.21.15.119:0/35238765 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae0003940 con 0x7f0af81f3520 2024-04-25T10:17:20.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.270+0000 7f0af5600640 1 -- 172.21.15.119:0/35238765 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0036040 con 0x7f0af81f3520 2024-04-25T10:17:20.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.270+0000 7f0affc00640 1 -- 172.21.15.119:0/35238765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af81565e0 con 0x7f0af81f3520 2024-04-25T10:17:20.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.270+0000 7f0affc00640 1 -- 172.21.15.119:0/35238765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af80cc140 con 0x7f0af81f3520 2024-04-25T10:17:20.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.271+0000 7f0affc00640 1 -- 172.21.15.119:0/35238765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af80749b0 con 0x7f0af81f3520 2024-04-25T10:17:20.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.272+0000 7f0af5600640 1 -- 172.21.15.119:0/35238765 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ae0043020 con 0x7f0af81f3520 2024-04-25T10:17:20.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.273+0000 7f0af5600640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0ad406b4d0 0x7f0ad406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.273+0000 7f0af5600640 1 -- 172.21.15.119:0/35238765 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f0ae00aa880 con 0x7f0af81f3520 2024-04-25T10:17:20.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.274+0000 7f0afd200640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0ad406b4d0 0x7f0ad406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.274+0000 7f0afd200640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0ad406b4d0 0x7f0ad406d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8001690 tx=0x7f0ae8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:20.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.278+0000 7f0af5600640 1 -- 172.21.15.119:0/35238765 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ae004a030 con 0x7f0af81f3520 2024-04-25T10:17:20.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.474+0000 7f0affc00640 1 -- 172.21.15.119:0/35238765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0af806d4c0 con 0x7f0af81f3520 2024-04-25T10:17:20.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.477+0000 7f0af5600640 1 -- 172.21.15.119:0/35238765 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16905 (secure 0 0 0) 0x7f0af80cebd0 con 0x7f0af81f3520 2024-04-25T10:17:20.481 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:20.481 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":98,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:17:18.997299+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:49.348469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:50.584869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/464677621":"2024-04-26T10:17:18.997269+0000","172.21.15.119:6835/464677621":"2024-04-26T10:17:18.997269+0000","172.21.15.119:6837/1945631445":"2024-04-26T10:16:41.666750+0000","172.21.15.119:6835/1945631445":"2024-04-26T10:16:41.666750+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:20.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.480+0000 7f0affc00640 1 -- 172.21.15.119:0/35238765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0ad406b4d0 msgr2=0x7f0ad406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.480+0000 7f0affc00640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0ad406b4d0 0x7f0ad406d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8001690 tx=0x7f0ae8009040 comp rx=0 tx=0).stop 2024-04-25T10:17:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.480+0000 7f0affc00640 1 -- 172.21.15.119:0/35238765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0af81f3520 msgr2=0x7f0af80aee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.480+0000 7f0affc00640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0af81f3520 0x7f0af80aee90 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f0ae002ab40 tx=0x7f0ae0004730 comp rx=0 tx=0).stop 2024-04-25T10:17:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.480+0000 7f0affc00640 1 -- 172.21.15.119:0/35238765 shutdown_connections 2024-04-25T10:17:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.480+0000 7f0affc00640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0ad406b4d0 0x7f0ad406d990 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.480+0000 7f0affc00640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0af81f39a0 0x7f0af80af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.480+0000 7f0affc00640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0af81f3520 0x7f0af80aee90 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.480+0000 7f0affc00640 1 --2- 172.21.15.119:0/35238765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0adc004800 0x7f0af81560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.480+0000 7f0affc00640 1 -- 172.21.15.119:0/35238765 >> 172.21.15.119:0/35238765 conn(0x7f0af806aba0 msgr2=0x7f0af806dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.480+0000 7f0affc00640 1 -- 172.21.15.119:0/35238765 shutdown_connections 2024-04-25T10:17:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.480+0000 7f0affc00640 1 -- 172.21.15.119:0/35238765 wait complete. 2024-04-25T10:17:20.501 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:15:49.348469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:15:50.584869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:20.502 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:20.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.702+0000 7f5611e00640 1 Processor -- start 2024-04-25T10:17:20.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.702+0000 7f5611e00640 1 -- start start 2024-04-25T10:17:20.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.704+0000 7f5611e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f560c0ce9d0 0x7f560c0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.704+0000 7f5611e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560c06dcb0 con 0x7f560c0cc300 2024-04-25T10:17:20.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.704+0000 7f5611e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560c06df90 con 0x7f560c144e70 2024-04-25T10:17:20.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.704+0000 7f5611e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c15bfe0 con 0x7f560c0ce9d0 2024-04-25T10:17:20.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.704+0000 7f560b400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f560c0cc300 0x7f560c0cc770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57702/0 (socket says 172.21.15.119:57702) 2024-04-25T10:17:20.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.704+0000 7f560b400640 1 -- 172.21.15.119:0/2608409453 learned_addr learned my addr 172.21.15.119:0/2608409453 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:20.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f560a000640 1 --2- 172.21.15.119:0/2608409453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f560c0ce9d0 0x7f560c0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 -- 172.21.15.119:0/2608409453 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563979053 0 0) 0x7f560c06dcb0 con 0x7f560c0cc300 2024-04-25T10:17:20.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 -- 172.21.15.119:0/2608409453 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55f0003520 con 0x7f560c0cc300 2024-04-25T10:17:20.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 -- 172.21.15.119:0/2608409453 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 422982530 0 0) 0x7f55f0003520 con 0x7f560c0cc300 2024-04-25T10:17:20.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 -- 172.21.15.119:0/2608409453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f560c0ce9d0 msgr2=0x7f560c0cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:20.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 --2- 172.21.15.119:0/2608409453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f560c0ce9d0 0x7f560c0cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 -- 172.21.15.119:0/2608409453 >> v1:172.21.15.132:6789/0 conn(0x7f560c144e70 legacy=0x7f560c06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 -- 172.21.15.119:0/2608409453 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c06e270 con 0x7f560c0cc300 2024-04-25T10:17:20.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 -- 172.21.15.119:0/2608409453 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f5604001a10 con 0x7f560c0cc300 2024-04-25T10:17:20.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 -- 172.21.15.119:0/2608409453 >> v1:172.21.15.119:6789/0 conn(0x7f560c0cc300 legacy=0x7f560c0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 --2- 172.21.15.119:0/2608409453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f00039e0 0x7f55f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 --2- 172.21.15.119:0/2608409453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f560c0ce9d0 0x7f55f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 --2- 172.21.15.119:0/2608409453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55f0004d40 0x7f55f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 -- 172.21.15.119:0/2608409453 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c15bfe0 con 0x7f55f00039e0 2024-04-25T10:17:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 -- 172.21.15.119:0/2608409453 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f000bb80 con 0x7f55f0004d40 2024-04-25T10:17:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.705+0000 7f5609600640 1 -- 172.21.15.119:0/2608409453 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f000bd80 con 0x7f560c0ce9d0 2024-04-25T10:17:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.706+0000 7f560aa00640 1 --2- 172.21.15.119:0/2608409453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f00039e0 0x7f55f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.706+0000 7f560b400640 1 --2- 172.21.15.119:0/2608409453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55f0004d40 0x7f55f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.706+0000 7f560aa00640 1 -- 172.21.15.119:0/2608409453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f560c0ce9d0 msgr2=0x7f55f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.706+0000 7f560aa00640 1 --2- 172.21.15.119:0/2608409453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f560c0ce9d0 0x7f55f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.706+0000 7f560aa00640 1 -- 172.21.15.119:0/2608409453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55f0004d40 msgr2=0x7f55f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.706+0000 7f560aa00640 1 --2- 172.21.15.119:0/2608409453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55f0004d40 0x7f55f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.706+0000 7f560aa00640 1 -- 172.21.15.119:0/2608409453 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f0003520 con 0x7f55f00039e0 2024-04-25T10:17:20.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.706+0000 7f560b400640 1 --2- 172.21.15.119:0/2608409453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55f0004d40 0x7f55f00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.707+0000 7f560aa00640 1 --2- 172.21.15.119:0/2608409453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f00039e0 0x7f55f0003e00 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f55fc001750 tx=0x7f55f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b346bc9706488b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.707+0000 7f5609600640 1 -- 172.21.15.119:0/2608409453 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc001370 con 0x7f55f00039e0 2024-04-25T10:17:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.707+0000 7f5609600640 1 -- 172.21.15.119:0/2608409453 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc0052d0 con 0x7f55f00039e0 2024-04-25T10:17:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.707+0000 7f5609600640 1 -- 172.21.15.119:0/2608409453 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc02abe0 con 0x7f55f00039e0 2024-04-25T10:17:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.708+0000 7f5611e00640 1 -- 172.21.15.119:0/2608409453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f00039e0 msgr2=0x7f55f0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.708+0000 7f5611e00640 1 --2- 172.21.15.119:0/2608409453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f00039e0 0x7f55f0003e00 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f55fc001750 tx=0x7f55f0005830 comp rx=0 tx=0).stop 2024-04-25T10:17:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.708+0000 7f560b400640 1 -- 172.21.15.119:0/2608409453 reap_dead start 2024-04-25T10:17:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.708+0000 7f5611e00640 1 -- 172.21.15.119:0/2608409453 shutdown_connections 2024-04-25T10:17:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.708+0000 7f5611e00640 1 -- 172.21.15.119:0/2608409453 >> 172.21.15.119:0/2608409453 conn(0x7f560c06aba0 msgr2=0x7f560c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.708+0000 7f5611e00640 1 -- 172.21.15.119:0/2608409453 shutdown_connections 2024-04-25T10:17:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.708+0000 7f5611e00640 1 -- 172.21.15.119:0/2608409453 wait complete. 2024-04-25T10:17:20.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.710+0000 7f5611e00640 1 Processor -- start 2024-04-25T10:17:20.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.710+0000 7f5611e00640 1 -- start start 2024-04-25T10:17:20.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.711+0000 7f5611e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f00039e0 0x7f560c0ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.711+0000 7f5611e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f560c1f1e00 0x7f560c1f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.711+0000 7f5611e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f560c1e4f70 0x7f560c1e5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.711+0000 7f5611e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c1454b0 con 0x7f55f00039e0 2024-04-25T10:17:20.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.711+0000 7f5611e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c06d970 con 0x7f560c1f1e00 2024-04-25T10:17:20.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.711+0000 7f5611e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c145630 con 0x7f560c1e4f70 2024-04-25T10:17:20.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.711+0000 7f560aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f00039e0 0x7f560c0ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.711+0000 7f560aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f00039e0 0x7f560c0ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47086/0 (socket says 172.21.15.119:47086) 2024-04-25T10:17:20.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.711+0000 7f560aa00640 1 -- 172.21.15.119:0/1373154173 learned_addr learned my addr 172.21.15.119:0/1373154173 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:20.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.712+0000 7f560a000640 1 --2- 172.21.15.119:0/1373154173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f560c1f1e00 0x7f560c1f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.712+0000 7f560aa00640 1 -- 172.21.15.119:0/1373154173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f560c1e4f70 msgr2=0x7f560c1e5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.712+0000 7f560aa00640 1 --2- 172.21.15.119:0/1373154173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f560c1e4f70 0x7f560c1e5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.712+0000 7f560aa00640 1 -- 172.21.15.119:0/1373154173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f560c1f1e00 msgr2=0x7f560c1f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.712+0000 7f560aa00640 1 --2- 172.21.15.119:0/1373154173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f560c1f1e00 0x7f560c1f2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.712+0000 7f560aa00640 1 -- 172.21.15.119:0/1373154173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c1e5ce0 con 0x7f55f00039e0 2024-04-25T10:17:20.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.712+0000 7f560aa00640 1 --2- 172.21.15.119:0/1373154173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f00039e0 0x7f560c0ade50 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f55fc008fd0 tx=0x7f55fc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:20.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.712+0000 7f5603400640 1 -- 172.21.15.119:0/1373154173 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc037040 con 0x7f55f00039e0 2024-04-25T10:17:20.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.712+0000 7f5603400640 1 -- 172.21.15.119:0/1373154173 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc005480 con 0x7f55f00039e0 2024-04-25T10:17:20.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.712+0000 7f5603400640 1 -- 172.21.15.119:0/1373154173 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc007a30 con 0x7f55f00039e0 2024-04-25T10:17:20.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.712+0000 7f5611e00640 1 -- 172.21.15.119:0/1373154173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c1e5fc0 con 0x7f55f00039e0 2024-04-25T10:17:20.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.712+0000 7f5611e00640 1 -- 172.21.15.119:0/1373154173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c1e6580 con 0x7f55f00039e0 2024-04-25T10:17:20.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.713+0000 7f5611e00640 1 -- 172.21.15.119:0/1373154173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560c0750b0 con 0x7f55f00039e0 2024-04-25T10:17:20.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.714+0000 7f5603400640 1 -- 172.21.15.119:0/1373154173 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55fc044020 con 0x7f55f00039e0 2024-04-25T10:17:20.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.715+0000 7f5603400640 1 --2- 172.21.15.119:0/1373154173 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55e806b500 0x7f55e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:20.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.715+0000 7f5603400640 1 -- 172.21.15.119:0/1373154173 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f55fc0a9ff0 con 0x7f55f00039e0 2024-04-25T10:17:20.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.716+0000 7f560a000640 1 --2- 172.21.15.119:0/1373154173 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55e806b500 0x7f55e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:20.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.716+0000 7f560a000640 1 --2- 172.21.15.119:0/1373154173 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55e806b500 0x7f55e806d9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f560c15afe0 tx=0x7f55f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:20.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.720+0000 7f5603400640 1 -- 172.21.15.119:0/1373154173 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55fc078e40 con 0x7f55f00039e0 2024-04-25T10:17:20.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.917+0000 7f5611e00640 1 -- 172.21.15.119:0/1373154173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f560c13fcb0 con 0x7f55f00039e0 2024-04-25T10:17:20.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.918+0000 7f5603400640 1 -- 172.21.15.119:0/1373154173 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55fc078740 con 0x7f55f00039e0 2024-04-25T10:17:20.921 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:17:20.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.920+0000 7f5611e00640 1 -- 172.21.15.119:0/1373154173 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55e806b500 msgr2=0x7f55e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.920+0000 7f5611e00640 1 --2- 172.21.15.119:0/1373154173 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55e806b500 0x7f55e806d9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f560c15afe0 tx=0x7f55f4008040 comp rx=0 tx=0).stop 2024-04-25T10:17:20.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.920+0000 7f5611e00640 1 -- 172.21.15.119:0/1373154173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f00039e0 msgr2=0x7f560c0ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:20.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.920+0000 7f5611e00640 1 --2- 172.21.15.119:0/1373154173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f00039e0 0x7f560c0ade50 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f55fc008fd0 tx=0x7f55fc001120 comp rx=0 tx=0).stop 2024-04-25T10:17:20.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.921+0000 7f5611e00640 1 -- 172.21.15.119:0/1373154173 shutdown_connections 2024-04-25T10:17:20.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.921+0000 7f5611e00640 1 --2- 172.21.15.119:0/1373154173 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f55e806b500 0x7f55e806d9c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.921+0000 7f5611e00640 1 --2- 172.21.15.119:0/1373154173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f560c1e4f70 0x7f560c1e5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.921+0000 7f5611e00640 1 --2- 172.21.15.119:0/1373154173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f560c1f1e00 0x7f560c1f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.921+0000 7f5611e00640 1 --2- 172.21.15.119:0/1373154173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f00039e0 0x7f560c0ade50 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:20.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.921+0000 7f5611e00640 1 -- 172.21.15.119:0/1373154173 >> 172.21.15.119:0/1373154173 conn(0x7f560c06aba0 msgr2=0x7f560c0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:20.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.921+0000 7f5611e00640 1 -- 172.21.15.119:0/1373154173 shutdown_connections 2024-04-25T10:17:20.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:20.921+0000 7f5611e00640 1 -- 172.21.15.119:0/1373154173 wait complete. 2024-04-25T10:17:20.943 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:17:21.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.142+0000 7f1415200640 1 Processor -- start 2024-04-25T10:17:21.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.142+0000 7f1415200640 1 -- start start 2024-04-25T10:17:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.144+0000 7f1415200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f14100cf900 0x7f14100b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.144+0000 7f1415200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14100b2b90 0x7f1410162f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.144+0000 7f1415200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1410071350 con 0x7f14100cfce0 2024-04-25T10:17:21.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.144+0000 7f1415200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1410071630 con 0x7f14100cf900 2024-04-25T10:17:21.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.144+0000 7f1415200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1410071910 con 0x7f14100b2b90 2024-04-25T10:17:21.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.144+0000 7f140d400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f14100cfce0 0x7f1410162880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37912/0 (socket says 172.21.15.119:37912) 2024-04-25T10:17:21.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.144+0000 7f140d400640 1 -- 172.21.15.119:0/1787596794 learned_addr learned my addr 172.21.15.119:0/1787596794 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:21.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.145+0000 7f140de00640 1 --2- 172.21.15.119:0/1787596794 >> v2:172.21.15.132:3300/0 conn(0x7f14100cf900 0x7f14100b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.145+0000 7f140e800640 1 --2- 172.21.15.119:0/1787596794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14100b2b90 0x7f1410162f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.145+0000 7f140ca00640 1 -- 172.21.15.119:0/1787596794 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967159531 0 0) 0x7f1410071350 con 0x7f14100cfce0 2024-04-25T10:17:21.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.145+0000 7f140ca00640 1 -- 172.21.15.119:0/1787596794 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13f4003520 con 0x7f14100cfce0 2024-04-25T10:17:21.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.145+0000 7f140de00640 1 -- 172.21.15.119:0/1787596794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14100b2b90 msgr2=0x7f1410162f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:21.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.145+0000 7f140de00640 1 --2- 172.21.15.119:0/1787596794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14100b2b90 0x7f1410162f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.145+0000 7f140ca00640 1 -- 172.21.15.119:0/1787596794 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 228988532 0 0) 0x7f13f4003520 con 0x7f14100cfce0 2024-04-25T10:17:21.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.145+0000 7f140de00640 1 -- 172.21.15.119:0/1787596794 >> v1:172.21.15.119:6789/0 conn(0x7f14100cfce0 legacy=0x7f1410162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.145+0000 7f140de00640 1 -- 172.21.15.119:0/1787596794 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1410071bf0 con 0x7f14100cf900 2024-04-25T10:17:21.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.147+0000 7f140de00640 1 --2- 172.21.15.119:0/1787596794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14100cf900 0x7f14100b2650 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f14000097a0 tx=0x7f140002a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eb58cf7c7a01607 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.147+0000 7f140ca00640 1 -- 172.21.15.119:0/1787596794 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1400037070 con 0x7f14100cf900 2024-04-25T10:17:21.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.147+0000 7f140ca00640 1 -- 172.21.15.119:0/1787596794 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140002ab10 con 0x7f14100cf900 2024-04-25T10:17:21.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.147+0000 7f140ca00640 1 -- 172.21.15.119:0/1787596794 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc001690 con 0x7f14100cf900 2024-04-25T10:17:21.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.148+0000 7f1415200640 1 -- 172.21.15.119:0/1787596794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14100cf900 msgr2=0x7f14100b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.148+0000 7f1415200640 1 --2- 172.21.15.119:0/1787596794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14100cf900 0x7f14100b2650 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f14000097a0 tx=0x7f140002a080 comp rx=0 tx=0).stop 2024-04-25T10:17:21.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.148+0000 7f1415200640 1 -- 172.21.15.119:0/1787596794 shutdown_connections 2024-04-25T10:17:21.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.148+0000 7f1415200640 1 --2- 172.21.15.119:0/1787596794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14100b2b90 0x7f1410162f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.148+0000 7f1415200640 1 --2- 172.21.15.119:0/1787596794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14100cf900 0x7f14100b2650 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.148+0000 7f1415200640 1 -- 172.21.15.119:0/1787596794 >> 172.21.15.119:0/1787596794 conn(0x7f141006aba0 msgr2=0x7f141006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:21.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.148+0000 7f1415200640 1 -- 172.21.15.119:0/1787596794 shutdown_connections 2024-04-25T10:17:21.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.148+0000 7f1415200640 1 -- 172.21.15.119:0/1787596794 wait complete. 2024-04-25T10:17:21.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.150+0000 7f1415200640 1 Processor -- start 2024-04-25T10:17:21.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.150+0000 7f1415200640 1 -- start start 2024-04-25T10:17:21.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.151+0000 7f1415200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14100b2b90 0x7f14100afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.151+0000 7f1415200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14100cf900 0x7f1410149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.151+0000 7f1415200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14100cfce0 0x7f1410149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.151+0000 7f1415200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1410165c30 con 0x7f14100cf900 2024-04-25T10:17:21.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.151+0000 7f1415200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14100b3e00 con 0x7f14100cfce0 2024-04-25T10:17:21.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.151+0000 7f1415200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1410165db0 con 0x7f14100b2b90 2024-04-25T10:17:21.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.151+0000 7f140d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14100cf900 0x7f1410149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.152+0000 7f140d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14100cf900 0x7f1410149560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42988/0 (socket says 172.21.15.119:42988) 2024-04-25T10:17:21.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.152+0000 7f140d400640 1 -- 172.21.15.119:0/2616833781 learned_addr learned my addr 172.21.15.119:0/2616833781 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:21.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.152+0000 7f140e800640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14100cfce0 0x7f1410149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.152+0000 7f140de00640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14100b2b90 0x7f14100afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.152+0000 7f140d400640 1 -- 172.21.15.119:0/2616833781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14100b2b90 msgr2=0x7f14100afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.152+0000 7f140d400640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14100b2b90 0x7f14100afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.152+0000 7f140d400640 1 -- 172.21.15.119:0/2616833781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14100cfce0 msgr2=0x7f1410149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.152+0000 7f140d400640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14100cfce0 0x7f1410149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.152+0000 7f140d400640 1 -- 172.21.15.119:0/2616833781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14100b05d0 con 0x7f14100cf900 2024-04-25T10:17:21.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.152+0000 7f140e800640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14100cfce0 0x7f1410149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:21.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.152+0000 7f140de00640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14100b2b90 0x7f14100afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:21.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.152+0000 7f140d400640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14100cf900 0x7f1410149560 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00c500 tx=0x7f13fc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.153+0000 7f1406000640 1 -- 172.21.15.119:0/2616833781 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc018070 con 0x7f14100cf900 2024-04-25T10:17:21.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.153+0000 7f1406000640 1 -- 172.21.15.119:0/2616833781 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13fc007560 con 0x7f14100cf900 2024-04-25T10:17:21.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.153+0000 7f1406000640 1 -- 172.21.15.119:0/2616833781 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0049e0 con 0x7f14100cf900 2024-04-25T10:17:21.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.153+0000 7f1415200640 1 -- 172.21.15.119:0/2616833781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14100b08b0 con 0x7f14100cf900 2024-04-25T10:17:21.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.153+0000 7f1415200640 1 -- 172.21.15.119:0/2616833781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14100b0210 con 0x7f14100cf900 2024-04-25T10:17:21.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.154+0000 7f1415200640 1 -- 172.21.15.119:0/2616833781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14100750b0 con 0x7f14100cf900 2024-04-25T10:17:21.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.155+0000 7f1406000640 1 -- 172.21.15.119:0/2616833781 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13fc004b80 con 0x7f14100cf900 2024-04-25T10:17:21.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.156+0000 7f1406000640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f13f006b4d0 0x7f13f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.156+0000 7f1406000640 1 -- 172.21.15.119:0/2616833781 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f13fc089720 con 0x7f14100cf900 2024-04-25T10:17:21.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.156+0000 7f140de00640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f13f006b4d0 0x7f13f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.157+0000 7f140de00640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f13f006b4d0 0x7f13f006d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1400034000 tx=0x7f140002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.160+0000 7f1406000640 1 -- 172.21.15.119:0/2616833781 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f13fc04d8c0 con 0x7f14100cf900 2024-04-25T10:17:21.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.356+0000 7f1415200640 1 -- 172.21.15.119:0/2616833781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f141006b260 con 0x7f14100cf900 2024-04-25T10:17:21.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.357+0000 7f1406000640 1 -- 172.21.15.119:0/2616833781 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7f13fc0584e0 con 0x7f14100cf900 2024-04-25T10:17:21.360 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:17:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.360+0000 7f1415200640 1 -- 172.21.15.119:0/2616833781 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f13f006b4d0 msgr2=0x7f13f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.360+0000 7f1415200640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f13f006b4d0 0x7f13f006d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1400034000 tx=0x7f140002e000 comp rx=0 tx=0).stop 2024-04-25T10:17:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.360+0000 7f1415200640 1 -- 172.21.15.119:0/2616833781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14100cf900 msgr2=0x7f1410149560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.360+0000 7f1415200640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14100cf900 0x7f1410149560 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00c500 tx=0x7f13fc00c9c0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.360+0000 7f1415200640 1 -- 172.21.15.119:0/2616833781 shutdown_connections 2024-04-25T10:17:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.360+0000 7f1415200640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f13f006b4d0 0x7f13f006d990 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.360+0000 7f1415200640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14100cfce0 0x7f1410149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.360+0000 7f1415200640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14100cf900 0x7f1410149560 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.360+0000 7f1415200640 1 --2- 172.21.15.119:0/2616833781 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14100b2b90 0x7f14100afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.360+0000 7f1415200640 1 -- 172.21.15.119:0/2616833781 >> 172.21.15.119:0/2616833781 conn(0x7f141006aba0 msgr2=0x7f1410161ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:21.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.360+0000 7f1415200640 1 -- 172.21.15.119:0/2616833781 shutdown_connections 2024-04-25T10:17:21.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.360+0000 7f1415200640 1 -- 172.21.15.119:0/2616833781 wait complete. 2024-04-25T10:17:21.383 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:17:21.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.579+0000 7f96d9600640 1 Processor -- start 2024-04-25T10:17:21.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.579+0000 7f96d9600640 1 -- start start 2024-04-25T10:17:21.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.581+0000 7f96d9600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96d40ce9d0 0x7f96d40cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.581+0000 7f96d9600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96d40cc300 0x7f96d40cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.581+0000 7f96d9600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d406dcb0 con 0x7f96d4144e70 2024-04-25T10:17:21.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.581+0000 7f96d9600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d415bfe0 con 0x7f96d40cc300 2024-04-25T10:17:21.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.581+0000 7f96d9600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d406df90 con 0x7f96d40ce9d0 2024-04-25T10:17:21.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.581+0000 7f96d3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96d40cc300 0x7f96d40cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.582+0000 7f96d3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96d40cc300 0x7f96d40cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42998/0 (socket says 172.21.15.119:42998) 2024-04-25T10:17:21.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.582+0000 7f96d3400640 1 -- 172.21.15.119:0/3970088484 learned_addr learned my addr 172.21.15.119:0/3970088484 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:21.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.582+0000 7f96d2000640 1 --2- 172.21.15.119:0/3970088484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96d40ce9d0 0x7f96d40cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.582+0000 7f96d3400640 1 -- 172.21.15.119:0/3970088484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96d40ce9d0 msgr2=0x7f96d40cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:21.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.582+0000 7f96d3400640 1 --2- 172.21.15.119:0/3970088484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96d40ce9d0 0x7f96d40cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.582+0000 7f96d3400640 1 -- 172.21.15.119:0/3970088484 >> v1:172.21.15.132:6789/0 conn(0x7f96d4144e70 legacy=0x7f96d406d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:21.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.582+0000 7f96d3400640 1 -- 172.21.15.119:0/3970088484 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d406e270 con 0x7f96d40cc300 2024-04-25T10:17:21.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.583+0000 7f96d3400640 1 --2- 172.21.15.119:0/3970088484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96d40cc300 0x7f96d40cc770 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009870 tx=0x7f96cc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2974d540c511aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.584+0000 7f96d1600640 1 -- 172.21.15.119:0/3970088484 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc037070 con 0x7f96d40cc300 2024-04-25T10:17:21.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.584+0000 7f96d1600640 1 -- 172.21.15.119:0/3970088484 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc032440 con 0x7f96d40cc300 2024-04-25T10:17:21.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.584+0000 7f96d1600640 1 -- 172.21.15.119:0/3970088484 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc032810 con 0x7f96d40cc300 2024-04-25T10:17:21.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.584+0000 7f96d9600640 1 -- 172.21.15.119:0/3970088484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96d40cc300 msgr2=0x7f96d40cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.584+0000 7f96d9600640 1 --2- 172.21.15.119:0/3970088484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96d40cc300 0x7f96d40cc770 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009870 tx=0x7f96cc02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.585+0000 7f96d9600640 1 -- 172.21.15.119:0/3970088484 shutdown_connections 2024-04-25T10:17:21.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.585+0000 7f96d9600640 1 --2- 172.21.15.119:0/3970088484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96d40cc300 0x7f96d40cc770 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.585+0000 7f96d9600640 1 --2- 172.21.15.119:0/3970088484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96d40ce9d0 0x7f96d40cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.585+0000 7f96d9600640 1 -- 172.21.15.119:0/3970088484 >> 172.21.15.119:0/3970088484 conn(0x7f96d406aba0 msgr2=0x7f96d406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:21.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.585+0000 7f96d9600640 1 -- 172.21.15.119:0/3970088484 shutdown_connections 2024-04-25T10:17:21.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.585+0000 7f96d9600640 1 -- 172.21.15.119:0/3970088484 wait complete. 2024-04-25T10:17:21.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.586+0000 7f96d9600640 1 Processor -- start 2024-04-25T10:17:21.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.586+0000 7f96d9600640 1 -- start start 2024-04-25T10:17:21.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.587+0000 7f96d9600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96d40cc300 0x7f96d415d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.587+0000 7f96d9600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96d40ce9d0 0x7f96d4153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.587+0000 7f96d9600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96d4144e70 0x7f96d41535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.587+0000 7f96d9600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d406d970 con 0x7f96d40cc300 2024-04-25T10:17:21.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.587+0000 7f96d9600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40cf370 con 0x7f96d40ce9d0 2024-04-25T10:17:21.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.587+0000 7f96d9600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d41454b0 con 0x7f96d4144e70 2024-04-25T10:17:21.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.588+0000 7f96d2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96d40cc300 0x7f96d415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.588+0000 7f96d2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96d40cc300 0x7f96d415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43008/0 (socket says 172.21.15.119:43008) 2024-04-25T10:17:21.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.588+0000 7f96d2a00640 1 -- 172.21.15.119:0/12725705 learned_addr learned my addr 172.21.15.119:0/12725705 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:21.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.588+0000 7f96d2000640 1 --2- 172.21.15.119:0/12725705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96d40ce9d0 0x7f96d4153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.588+0000 7f96d3400640 1 --2- 172.21.15.119:0/12725705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96d4144e70 0x7f96d41535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.588+0000 7f96d2a00640 1 -- 172.21.15.119:0/12725705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96d4144e70 msgr2=0x7f96d41535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.588+0000 7f96d2a00640 1 --2- 172.21.15.119:0/12725705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96d4144e70 0x7f96d41535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.588+0000 7f96d2a00640 1 -- 172.21.15.119:0/12725705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96d40ce9d0 msgr2=0x7f96d4153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.588+0000 7f96d2a00640 1 --2- 172.21.15.119:0/12725705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96d40ce9d0 0x7f96d4153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.588+0000 7f96d2a00640 1 -- 172.21.15.119:0/12725705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d415c590 con 0x7f96d40cc300 2024-04-25T10:17:21.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.588+0000 7f96d2a00640 1 --2- 172.21.15.119:0/12725705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96d40cc300 0x7f96d415d630 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f96c400c340 tx=0x7f96c400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.588+0000 7f96caa00640 1 -- 172.21.15.119:0/12725705 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c4017070 con 0x7f96d40cc300 2024-04-25T10:17:21.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.589+0000 7f96caa00640 1 -- 172.21.15.119:0/12725705 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c40052d0 con 0x7f96d40cc300 2024-04-25T10:17:21.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.589+0000 7f96caa00640 1 -- 172.21.15.119:0/12725705 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c4003850 con 0x7f96d40cc300 2024-04-25T10:17:21.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.589+0000 7f96d9600640 1 -- 172.21.15.119:0/12725705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d415c870 con 0x7f96d40cc300 2024-04-25T10:17:21.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.589+0000 7f96d9600640 1 -- 172.21.15.119:0/12725705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d415ce30 con 0x7f96d40cc300 2024-04-25T10:17:21.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.590+0000 7f96d9600640 1 -- 172.21.15.119:0/12725705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d40750b0 con 0x7f96d40cc300 2024-04-25T10:17:21.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.591+0000 7f96caa00640 1 -- 172.21.15.119:0/12725705 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96c401a070 con 0x7f96d40cc300 2024-04-25T10:17:21.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.592+0000 7f96caa00640 1 --2- 172.21.15.119:0/12725705 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f96b006b4d0 0x7f96b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:21.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.592+0000 7f96caa00640 1 -- 172.21.15.119:0/12725705 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f96c40895a0 con 0x7f96d40cc300 2024-04-25T10:17:21.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.592+0000 7f96d2000640 1 --2- 172.21.15.119:0/12725705 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f96b006b4d0 0x7f96b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:21.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.592+0000 7f96d2000640 1 --2- 172.21.15.119:0/12725705 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f96b006b4d0 0x7f96b006d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f96d4155330 tx=0x7f96bc003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:21.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.596+0000 7f96caa00640 1 -- 172.21.15.119:0/12725705 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96c4058360 con 0x7f96d40cc300 2024-04-25T10:17:21.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.777+0000 7f96d9600640 1 -- 172.21.15.119:0/12725705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f96d406b020 con 0x7f96d40cc300 2024-04-25T10:17:21.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.777+0000 7f96caa00640 1 -- 172.21.15.119:0/12725705 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7f96c4057c60 con 0x7f96d40cc300 2024-04-25T10:17:21.781 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:17:21.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.780+0000 7f96d9600640 1 -- 172.21.15.119:0/12725705 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f96b006b4d0 msgr2=0x7f96b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.780+0000 7f96d9600640 1 --2- 172.21.15.119:0/12725705 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f96b006b4d0 0x7f96b006d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f96d4155330 tx=0x7f96bc003990 comp rx=0 tx=0).stop 2024-04-25T10:17:21.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.780+0000 7f96d9600640 1 -- 172.21.15.119:0/12725705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96d40cc300 msgr2=0x7f96d415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:21.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.780+0000 7f96d9600640 1 --2- 172.21.15.119:0/12725705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96d40cc300 0x7f96d415d630 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f96c400c340 tx=0x7f96c400c800 comp rx=0 tx=0).stop 2024-04-25T10:17:21.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.780+0000 7f96d9600640 1 -- 172.21.15.119:0/12725705 shutdown_connections 2024-04-25T10:17:21.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.780+0000 7f96d9600640 1 --2- 172.21.15.119:0/12725705 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f96b006b4d0 0x7f96b006d990 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.780+0000 7f96d9600640 1 --2- 172.21.15.119:0/12725705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96d4144e70 0x7f96d41535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.780+0000 7f96d9600640 1 --2- 172.21.15.119:0/12725705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96d40ce9d0 0x7f96d4153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.780+0000 7f96d9600640 1 --2- 172.21.15.119:0/12725705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96d40cc300 0x7f96d415d630 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:21.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.780+0000 7f96d9600640 1 -- 172.21.15.119:0/12725705 >> 172.21.15.119:0/12725705 conn(0x7f96d406aba0 msgr2=0x7f96d4151f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:21.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.780+0000 7f96d9600640 1 -- 172.21.15.119:0/12725705 shutdown_connections 2024-04-25T10:17:21.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.780+0000 7f96d9600640 1 -- 172.21.15.119:0/12725705 wait complete. 2024-04-25T10:17:21.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:22.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.996+0000 7fa0a3400640 1 Processor -- start 2024-04-25T10:17:22.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.996+0000 7fa0a3400640 1 -- start start 2024-04-25T10:17:22.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.998+0000 7fa0a3400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa09c06e8f0 0x7fa09c15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.998+0000 7fa0a3400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09c0ce6c0 con 0x7fa09c15bf00 2024-04-25T10:17:22.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.998+0000 7fa0a3400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09c0cc4e0 con 0x7fa09c15c820 2024-04-25T10:17:22.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.998+0000 7fa0a3400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c1455e0 con 0x7fa09c06e8f0 2024-04-25T10:17:22.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.998+0000 7fa0a0a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa09c15bf00 0x7fa09c0cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37928/0 (socket says 172.21.15.119:37928) 2024-04-25T10:17:22.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.998+0000 7fa0a0a00640 1 -- 172.21.15.119:0/1407941618 learned_addr learned my addr 172.21.15.119:0/1407941618 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa093e00640 1 --2- 172.21.15.119:0/1407941618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa09c06e8f0 0x7fa09c15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 -- 172.21.15.119:0/1407941618 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254575512 0 0) 0x7fa08c001110 con 0x7fa09c15bf00 2024-04-25T10:17:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 -- 172.21.15.119:0/1407941618 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa080003520 con 0x7fa09c15bf00 2024-04-25T10:17:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 -- 172.21.15.119:0/1407941618 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1855725942 0 0) 0x7fa080003520 con 0x7fa09c15bf00 2024-04-25T10:17:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 -- 172.21.15.119:0/1407941618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa09c06e8f0 msgr2=0x7fa09c15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 --2- 172.21.15.119:0/1407941618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa09c06e8f0 0x7fa09c15c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 -- 172.21.15.119:0/1407941618 >> v1:172.21.15.132:6789/0 conn(0x7fa09c15c820 legacy=0x7fa09c1654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 -- 172.21.15.119:0/1407941618 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c0cc7b0 con 0x7fa09c15bf00 2024-04-25T10:17:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 -- 172.21.15.119:0/1407941618 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fa08c001840 con 0x7fa09c15bf00 2024-04-25T10:17:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 -- 172.21.15.119:0/1407941618 >> v1:172.21.15.119:6789/0 conn(0x7fa09c15bf00 legacy=0x7fa09c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 --2- 172.21.15.119:0/1407941618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa09c06e8f0 0x7fa0800039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 --2- 172.21.15.119:0/1407941618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080003fb0 0x7fa080004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 --2- 172.21.15.119:0/1407941618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa080004d40 0x7fa0800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 -- 172.21.15.119:0/1407941618 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c1455e0 con 0x7fa080003fb0 2024-04-25T10:17:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 -- 172.21.15.119:0/1407941618 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08000bb00 con 0x7fa080004d40 2024-04-25T10:17:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:21.999+0000 7fa09be00640 1 -- 172.21.15.119:0/1407941618 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08000bd00 con 0x7fa09c06e8f0 2024-04-25T10:17:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.000+0000 7fa093e00640 1 --2- 172.21.15.119:0/1407941618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080003fb0 0x7fa080004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.000+0000 7fa0a0a00640 1 --2- 172.21.15.119:0/1407941618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa09c06e8f0 0x7fa0800039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.000+0000 7fa0a1400640 1 --2- 172.21.15.119:0/1407941618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa080004d40 0x7fa0800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.000+0000 7fa093e00640 1 -- 172.21.15.119:0/1407941618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa09c06e8f0 msgr2=0x7fa0800039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:22.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.000+0000 7fa093e00640 1 --2- 172.21.15.119:0/1407941618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa09c06e8f0 0x7fa0800039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.000+0000 7fa093e00640 1 -- 172.21.15.119:0/1407941618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa080004d40 msgr2=0x7fa0800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:22.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.000+0000 7fa093e00640 1 --2- 172.21.15.119:0/1407941618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa080004d40 0x7fa0800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.000+0000 7fa093e00640 1 -- 172.21.15.119:0/1407941618 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa080003520 con 0x7fa080003fb0 2024-04-25T10:17:22.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.001+0000 7fa093e00640 1 --2- 172.21.15.119:0/1407941618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080003fb0 0x7fa080004800 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa084009a90 tx=0x7fa080005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bde6bac9f92cb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.001+0000 7fa09be00640 1 -- 172.21.15.119:0/1407941618 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084036070 con 0x7fa080003fb0 2024-04-25T10:17:22.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.001+0000 7fa09be00640 1 -- 172.21.15.119:0/1407941618 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa084031400 con 0x7fa080003fb0 2024-04-25T10:17:22.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.001+0000 7fa09be00640 1 -- 172.21.15.119:0/1407941618 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084009690 con 0x7fa080003fb0 2024-04-25T10:17:22.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.002+0000 7fa0a3400640 1 -- 172.21.15.119:0/1407941618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080003fb0 msgr2=0x7fa080004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.002+0000 7fa0a3400640 1 --2- 172.21.15.119:0/1407941618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080003fb0 0x7fa080004800 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa084009a90 tx=0x7fa080005830 comp rx=0 tx=0).stop 2024-04-25T10:17:22.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.002+0000 7fa0a1400640 1 -- 172.21.15.119:0/1407941618 reap_dead start 2024-04-25T10:17:22.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.002+0000 7fa0a3400640 1 -- 172.21.15.119:0/1407941618 shutdown_connections 2024-04-25T10:17:22.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.002+0000 7fa0a3400640 1 -- 172.21.15.119:0/1407941618 >> 172.21.15.119:0/1407941618 conn(0x7fa09c06aba0 msgr2=0x7fa09c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:22.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.002+0000 7fa0a3400640 1 -- 172.21.15.119:0/1407941618 shutdown_connections 2024-04-25T10:17:22.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.002+0000 7fa0a3400640 1 -- 172.21.15.119:0/1407941618 wait complete. 2024-04-25T10:17:22.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.004+0000 7fa0a3400640 1 Processor -- start 2024-04-25T10:17:22.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.004+0000 7fa0a3400640 1 -- start start 2024-04-25T10:17:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.005+0000 7fa0a3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080003fb0 0x7fa09c155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.005+0000 7fa0a3400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa09c131280 0x7fa09c0b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.005+0000 7fa0a3400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa09c0b2670 0x7fa09c1404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.005+0000 7fa0a3400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0ccd70 con 0x7fa080003fb0 2024-04-25T10:17:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.005+0000 7fa0a3400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c06d450 con 0x7fa09c131280 2024-04-25T10:17:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.005+0000 7fa0a3400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0cd050 con 0x7fa09c0b2670 2024-04-25T10:17:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.005+0000 7fa0a0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080003fb0 0x7fa09c155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.005+0000 7fa0a0a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080003fb0 0x7fa09c155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43026/0 (socket says 172.21.15.119:43026) 2024-04-25T10:17:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.005+0000 7fa0a0a00640 1 -- 172.21.15.119:0/1216778330 learned_addr learned my addr 172.21.15.119:0/1216778330 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.006+0000 7fa0a0a00640 1 -- 172.21.15.119:0/1216778330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa09c0b2670 msgr2=0x7fa09c1404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.006+0000 7fa0a0a00640 1 --2- 172.21.15.119:0/1216778330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa09c0b2670 0x7fa09c1404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.006+0000 7fa0a0a00640 1 -- 172.21.15.119:0/1216778330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa09c131280 msgr2=0x7fa09c0b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.006+0000 7fa0a0a00640 1 --2- 172.21.15.119:0/1216778330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa09c131280 0x7fa09c0b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.006+0000 7fa0a0a00640 1 -- 172.21.15.119:0/1216778330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c156340 con 0x7fa080003fb0 2024-04-25T10:17:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.006+0000 7fa0a0a00640 1 --2- 172.21.15.119:0/1216778330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080003fb0 0x7fa09c155e00 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00e7a0 tx=0x7fa08c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.006+0000 7fa099600640 1 -- 172.21.15.119:0/1216778330 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c00c8d0 con 0x7fa080003fb0 2024-04-25T10:17:22.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.006+0000 7fa099600640 1 -- 172.21.15.119:0/1216778330 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c018070 con 0x7fa080003fb0 2024-04-25T10:17:22.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.006+0000 7fa099600640 1 -- 172.21.15.119:0/1216778330 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c00cc90 con 0x7fa080003fb0 2024-04-25T10:17:22.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.006+0000 7fa0a3400640 1 -- 172.21.15.119:0/1216778330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c0b2fe0 con 0x7fa080003fb0 2024-04-25T10:17:22.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.007+0000 7fa0a3400640 1 -- 172.21.15.119:0/1216778330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c0b35a0 con 0x7fa080003fb0 2024-04-25T10:17:22.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.008+0000 7fa0a3400640 1 -- 172.21.15.119:0/1216778330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09c06bb30 con 0x7fa080003fb0 2024-04-25T10:17:22.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.009+0000 7fa099600640 1 -- 172.21.15.119:0/1216778330 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa08c01d460 con 0x7fa080003fb0 2024-04-25T10:17:22.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.010+0000 7fa099600640 1 --2- 172.21.15.119:0/1216778330 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa07806b500 0x7fa07806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.010+0000 7fa099600640 1 -- 172.21.15.119:0/1216778330 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fa08c08a250 con 0x7fa080003fb0 2024-04-25T10:17:22.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.010+0000 7fa093e00640 1 --2- 172.21.15.119:0/1216778330 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa07806b500 0x7fa07806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.011+0000 7fa093e00640 1 --2- 172.21.15.119:0/1216778330 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa07806b500 0x7fa07806d9c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa084033000 tx=0x7fa08402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.014+0000 7fa099600640 1 -- 172.21.15.119:0/1216778330 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa09c0ce6c0 con 0x7fa080003fb0 2024-04-25T10:17:22.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.246+0000 7fa0a3400640 1 -- 172.21.15.119:0/1216778330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa09c06c170 con 0x7fa080003fb0 2024-04-25T10:17:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.247+0000 7fa099600640 1 -- 172.21.15.119:0/1216778330 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fa08c058f80 con 0x7fa080003fb0 2024-04-25T10:17:22.251 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:22.251 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5459,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3768348114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3768348114},{"type":"v1","addr":"172.21.15.132:6834","nonce":3768348114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":5465,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/543190780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":543190780},{"type":"v1","addr":"172.21.15.132:6835","nonce":543190780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1145229172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1145229172},{"type":"v1","addr":"172.21.15.119:6836","nonce":1145229172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1704625876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1704625876},{"type":"v1","addr":"172.21.15.119:6837","nonce":1704625876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:51.568312+0000","modified":"2024-04-25T10:17:18.997282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:17:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 90 2024-04-25T10:17:22.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.250+0000 7fa0a3400640 1 -- 172.21.15.119:0/1216778330 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa07806b500 msgr2=0x7fa07806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.250+0000 7fa0a3400640 1 --2- 172.21.15.119:0/1216778330 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa07806b500 0x7fa07806d9c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa084033000 tx=0x7fa08402e000 comp rx=0 tx=0).stop 2024-04-25T10:17:22.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.250+0000 7fa0a3400640 1 -- 172.21.15.119:0/1216778330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080003fb0 msgr2=0x7fa09c155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.250+0000 7fa0a3400640 1 --2- 172.21.15.119:0/1216778330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080003fb0 0x7fa09c155e00 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00e7a0 tx=0x7fa08c00ec60 comp rx=0 tx=0).stop 2024-04-25T10:17:22.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.250+0000 7fa0a3400640 1 -- 172.21.15.119:0/1216778330 shutdown_connections 2024-04-25T10:17:22.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.250+0000 7fa0a3400640 1 --2- 172.21.15.119:0/1216778330 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa07806b500 0x7fa07806d9c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.250+0000 7fa0a3400640 1 --2- 172.21.15.119:0/1216778330 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa09c0b2670 0x7fa09c1404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.250+0000 7fa0a3400640 1 --2- 172.21.15.119:0/1216778330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa09c131280 0x7fa09c0b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.250+0000 7fa0a3400640 1 --2- 172.21.15.119:0/1216778330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa080003fb0 0x7fa09c155e00 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.250+0000 7fa0a3400640 1 -- 172.21.15.119:0/1216778330 >> 172.21.15.119:0/1216778330 conn(0x7fa09c06aba0 msgr2=0x7fa09c142c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:22.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.250+0000 7fa0a3400640 1 -- 172.21.15.119:0/1216778330 shutdown_connections 2024-04-25T10:17:22.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.250+0000 7fa0a3400640 1 -- 172.21.15.119:0/1216778330 wait complete. 2024-04-25T10:17:22.272 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:22.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.469+0000 7fa236e00640 1 Processor -- start 2024-04-25T10:17:22.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.469+0000 7fa236e00640 1 -- start start 2024-04-25T10:17:22.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.471+0000 7fa236e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa23015c9f0 0x7fa2301664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.471+0000 7fa236e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2300ce6c0 con 0x7fa23006e8f0 2024-04-25T10:17:22.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.471+0000 7fa236e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2300cc4e0 con 0x7fa23015bf00 2024-04-25T10:17:22.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.471+0000 7fa236e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2301455e0 con 0x7fa23015c9f0 2024-04-25T10:17:22.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.471+0000 7fa234e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa23015c9f0 0x7fa2301664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.471+0000 7fa227400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa23006e8f0 0x7fa23015c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37930/0 (socket says 172.21.15.119:37930) 2024-04-25T10:17:22.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.471+0000 7fa227400640 1 -- 172.21.15.119:0/905994212 learned_addr learned my addr 172.21.15.119:0/905994212 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:22.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.472+0000 7fa234e00640 1 -- 172.21.15.119:0/905994212 >> v1:172.21.15.132:6789/0 conn(0x7fa23015bf00 legacy=0x7fa2300cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:22.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.472+0000 7fa234e00640 1 -- 172.21.15.119:0/905994212 >> v1:172.21.15.119:6789/0 conn(0x7fa23006e8f0 legacy=0x7fa23015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.472+0000 7fa234e00640 1 -- 172.21.15.119:0/905994212 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2300cc7b0 con 0x7fa23015c9f0 2024-04-25T10:17:22.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.472+0000 7fa22f400640 1 -- 172.21.15.119:0/905994212 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491501187 0 0) 0x7fa218001490 con 0x7fa23006e8f0 2024-04-25T10:17:22.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.473+0000 7fa234e00640 1 --2- 172.21.15.119:0/905994212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23015c9f0 0x7fa2301664c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa2280097a0 tx=0x7fa22802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9325a624cd2182 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.473+0000 7fa22f400640 1 -- 172.21.15.119:0/905994212 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228037070 con 0x7fa23015c9f0 2024-04-25T10:17:22.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.473+0000 7fa22f400640 1 -- 172.21.15.119:0/905994212 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa228032440 con 0x7fa23015c9f0 2024-04-25T10:17:22.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.473+0000 7fa22f400640 1 -- 172.21.15.119:0/905994212 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228032810 con 0x7fa23015c9f0 2024-04-25T10:17:22.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.474+0000 7fa236e00640 1 -- 172.21.15.119:0/905994212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23015c9f0 msgr2=0x7fa2301664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.474+0000 7fa236e00640 1 --2- 172.21.15.119:0/905994212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23015c9f0 0x7fa2301664c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa2280097a0 tx=0x7fa22802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.474+0000 7fa236e00640 1 -- 172.21.15.119:0/905994212 shutdown_connections 2024-04-25T10:17:22.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.474+0000 7fa236e00640 1 --2- 172.21.15.119:0/905994212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23015c9f0 0x7fa2301664c0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.474+0000 7fa236e00640 1 -- 172.21.15.119:0/905994212 >> 172.21.15.119:0/905994212 conn(0x7fa23006aba0 msgr2=0x7fa23006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:22.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.474+0000 7fa236e00640 1 -- 172.21.15.119:0/905994212 shutdown_connections 2024-04-25T10:17:22.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.474+0000 7fa236e00640 1 -- 172.21.15.119:0/905994212 wait complete. 2024-04-25T10:17:22.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.476+0000 7fa236e00640 1 Processor -- start 2024-04-25T10:17:22.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.476+0000 7fa236e00640 1 -- start start 2024-04-25T10:17:22.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.477+0000 7fa236e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa23006e8f0 0x7fa23013dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.477+0000 7fa236e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa23015bf00 0x7fa2301f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.477+0000 7fa236e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23015c9f0 0x7fa23015af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.477+0000 7fa236e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2300ccd70 con 0x7fa23015c9f0 2024-04-25T10:17:22.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.477+0000 7fa236e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23006d450 con 0x7fa23015bf00 2024-04-25T10:17:22.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.477+0000 7fa236e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2300cd050 con 0x7fa23006e8f0 2024-04-25T10:17:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.477+0000 7fa234e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23015c9f0 0x7fa23015af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.477+0000 7fa234e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23015c9f0 0x7fa23015af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43040/0 (socket says 172.21.15.119:43040) 2024-04-25T10:17:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.477+0000 7fa234e00640 1 -- 172.21.15.119:0/1952152873 learned_addr learned my addr 172.21.15.119:0/1952152873 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.477+0000 7fa22fe00640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa23006e8f0 0x7fa23013dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.478+0000 7fa234e00640 1 -- 172.21.15.119:0/1952152873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa23006e8f0 msgr2=0x7fa23013dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.478+0000 7fa227400640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa23015bf00 0x7fa2301f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.478+0000 7fa234e00640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa23006e8f0 0x7fa23013dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.478+0000 7fa234e00640 1 -- 172.21.15.119:0/1952152873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa23015bf00 msgr2=0x7fa2301f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.478+0000 7fa234e00640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa23015bf00 0x7fa2301f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.478+0000 7fa234e00640 1 -- 172.21.15.119:0/1952152873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2301f7640 con 0x7fa23015c9f0 2024-04-25T10:17:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.478+0000 7fa22fe00640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa23006e8f0 0x7fa23013dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.478+0000 7fa234e00640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23015c9f0 0x7fa23015af70 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fa2280097a0 tx=0x7fa2280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.478+0000 7fa22cc00640 1 -- 172.21.15.119:0/1952152873 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228041070 con 0x7fa23015c9f0 2024-04-25T10:17:22.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.478+0000 7fa22cc00640 1 -- 172.21.15.119:0/1952152873 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22803c870 con 0x7fa23015c9f0 2024-04-25T10:17:22.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.478+0000 7fa22cc00640 1 -- 172.21.15.119:0/1952152873 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228037040 con 0x7fa23015c9f0 2024-04-25T10:17:22.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.478+0000 7fa236e00640 1 -- 172.21.15.119:0/1952152873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2301f7920 con 0x7fa23015c9f0 2024-04-25T10:17:22.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.478+0000 7fa236e00640 1 -- 172.21.15.119:0/1952152873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2300c8a70 con 0x7fa23015c9f0 2024-04-25T10:17:22.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.479+0000 7fa236e00640 1 -- 172.21.15.119:0/1952152873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23006bb30 con 0x7fa23015c9f0 2024-04-25T10:17:22.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.480+0000 7fa22cc00640 1 -- 172.21.15.119:0/1952152873 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa228044070 con 0x7fa23015c9f0 2024-04-25T10:17:22.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.481+0000 7fa22cc00640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa20c06b4d0 0x7fa20c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.481+0000 7fa22cc00640 1 -- 172.21.15.119:0/1952152873 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fa2280aae70 con 0x7fa23015c9f0 2024-04-25T10:17:22.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.482+0000 7fa22fe00640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa20c06b4d0 0x7fa20c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.482+0000 7fa22fe00640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa20c06b4d0 0x7fa20c06d990 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa220003eb0 tx=0x7fa220009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.486+0000 7fa22cc00640 1 -- 172.21.15.119:0/1952152873 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa228079c30 con 0x7fa23015c9f0 2024-04-25T10:17:22.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.680+0000 7fa236e00640 1 -- 172.21.15.119:0/1952152873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2300749b0 con 0x7fa23015c9f0 2024-04-25T10:17:22.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.683+0000 7fa22cc00640 1 -- 172.21.15.119:0/1952152873 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16905 (secure 0 0 0) 0x7fa228079530 con 0x7fa23015c9f0 2024-04-25T10:17:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:22.688 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":98,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:17:18.997299+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:49.348469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:50.584869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/464677621":"2024-04-26T10:17:18.997269+0000","172.21.15.119:6835/464677621":"2024-04-26T10:17:18.997269+0000","172.21.15.119:6837/1945631445":"2024-04-26T10:16:41.666750+0000","172.21.15.119:6835/1945631445":"2024-04-26T10:16:41.666750+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:22.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.686+0000 7fa236e00640 1 -- 172.21.15.119:0/1952152873 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa20c06b4d0 msgr2=0x7fa20c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.686+0000 7fa236e00640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa20c06b4d0 0x7fa20c06d990 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa220003eb0 tx=0x7fa220009040 comp rx=0 tx=0).stop 2024-04-25T10:17:22.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.686+0000 7fa236e00640 1 -- 172.21.15.119:0/1952152873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23015c9f0 msgr2=0x7fa23015af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.686+0000 7fa236e00640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23015c9f0 0x7fa23015af70 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fa2280097a0 tx=0x7fa2280046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.686+0000 7fa236e00640 1 -- 172.21.15.119:0/1952152873 shutdown_connections 2024-04-25T10:17:22.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.686+0000 7fa236e00640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa20c06b4d0 0x7fa20c06d990 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.686+0000 7fa236e00640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa23015c9f0 0x7fa23015af70 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.686+0000 7fa236e00640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa23015bf00 0x7fa2301f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.686+0000 7fa236e00640 1 --2- 172.21.15.119:0/1952152873 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa23006e8f0 0x7fa23013dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.686+0000 7fa236e00640 1 -- 172.21.15.119:0/1952152873 >> 172.21.15.119:0/1952152873 conn(0x7fa23006aba0 msgr2=0x7fa23015d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:22.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.687+0000 7fa236e00640 1 -- 172.21.15.119:0/1952152873 shutdown_connections 2024-04-25T10:17:22.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.687+0000 7fa236e00640 1 -- 172.21.15.119:0/1952152873 wait complete. 2024-04-25T10:17:22.709 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:17:22.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:22.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.900+0000 7f45eb000640 1 Processor -- start 2024-04-25T10:17:22.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.900+0000 7f45eb000640 1 -- start start 2024-04-25T10:17:22.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.902+0000 7f45eb000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f45e40b5050 0x7f45e415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.902+0000 7f45eb000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e406e430 0x7f45e415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.902+0000 7f45eb000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e40cd440 con 0x7f45e4144470 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.902+0000 7f45eb000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e4145340 con 0x7f45e40b5050 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.902+0000 7f45eb000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40cd720 con 0x7f45e406e430 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.902+0000 7f45e9200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f45e40b5050 0x7f45e415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.902+0000 7f45e9c00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f45e4144470 0x7f45e40b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37944/0 (socket says 172.21.15.119:37944) 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.902+0000 7f45e9c00640 1 -- 172.21.15.119:0/164663019 learned_addr learned my addr 172.21.15.119:0/164663019 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.903+0000 7f45ea600640 1 --2- 172.21.15.119:0/164663019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e406e430 0x7f45e415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.903+0000 7f45e9200640 1 -- 172.21.15.119:0/164663019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e406e430 msgr2=0x7f45e415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.903+0000 7f45e9200640 1 --2- 172.21.15.119:0/164663019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e406e430 0x7f45e415bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.903+0000 7f45e9200640 1 -- 172.21.15.119:0/164663019 >> v1:172.21.15.119:6789/0 conn(0x7f45e4144470 legacy=0x7f45e40b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.903+0000 7f45e9200640 1 -- 172.21.15.119:0/164663019 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e40cda00 con 0x7f45e40b5050 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.903+0000 7f45e3e00640 1 -- 172.21.15.119:0/164663019 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244262876 0 0) 0x7f45e40cd440 con 0x7f45e4144470 2024-04-25T10:17:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.903+0000 7f45ea600640 1 --2- 172.21.15.119:0/164663019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e406e430 0x7f45e415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:22.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.904+0000 7f45e9200640 1 --2- 172.21.15.119:0/164663019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45e40b5050 0x7f45e415fc90 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f45d00097a0 tx=0x7f45d002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0828ca873f3eb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.904+0000 7f45e3e00640 1 -- 172.21.15.119:0/164663019 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0037070 con 0x7f45e40b5050 2024-04-25T10:17:22.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.904+0000 7f45e3e00640 1 -- 172.21.15.119:0/164663019 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d0032440 con 0x7f45e40b5050 2024-04-25T10:17:22.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.904+0000 7f45e3e00640 1 -- 172.21.15.119:0/164663019 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0032810 con 0x7f45e40b5050 2024-04-25T10:17:22.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.905+0000 7f45eb000640 1 -- 172.21.15.119:0/164663019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45e40b5050 msgr2=0x7f45e415fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.905+0000 7f45eb000640 1 --2- 172.21.15.119:0/164663019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45e40b5050 0x7f45e415fc90 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f45d00097a0 tx=0x7f45d002a180 comp rx=0 tx=0).stop 2024-04-25T10:17:22.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.905+0000 7f45eb000640 1 -- 172.21.15.119:0/164663019 shutdown_connections 2024-04-25T10:17:22.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.905+0000 7f45eb000640 1 --2- 172.21.15.119:0/164663019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e406e430 0x7f45e415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.905+0000 7f45eb000640 1 --2- 172.21.15.119:0/164663019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45e40b5050 0x7f45e415fc90 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.905+0000 7f45eb000640 1 -- 172.21.15.119:0/164663019 >> 172.21.15.119:0/164663019 conn(0x7f45e406b1d0 msgr2=0x7f45e40b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.905+0000 7f45eb000640 1 -- 172.21.15.119:0/164663019 shutdown_connections 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.905+0000 7f45eb000640 1 -- 172.21.15.119:0/164663019 wait complete. 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.907+0000 7f45eb000640 1 Processor -- start 2024-04-25T10:17:22.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.907+0000 7f45eb000640 1 -- start start 2024-04-25T10:17:22.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.908+0000 7f45eb000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45e406e430 0x7f45e415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.908+0000 7f45eb000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e40b5050 0x7f45e4130950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.908+0000 7f45eb000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e4144470 0x7f45e415f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.908+0000 7f45eb000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e40ce200 con 0x7f45e406e430 2024-04-25T10:17:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.908+0000 7f45eb000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e4144850 con 0x7f45e4144470 2024-04-25T10:17:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.908+0000 7f45eb000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e415c230 con 0x7f45e40b5050 2024-04-25T10:17:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.908+0000 7f45e9c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45e406e430 0x7f45e415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.908+0000 7f45e9c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45e406e430 0x7f45e415b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43058/0 (socket says 172.21.15.119:43058) 2024-04-25T10:17:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.908+0000 7f45e9c00640 1 -- 172.21.15.119:0/3799964920 learned_addr learned my addr 172.21.15.119:0/3799964920 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.908+0000 7f45e9c00640 1 -- 172.21.15.119:0/3799964920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e40b5050 msgr2=0x7f45e4130950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.908+0000 7f45e9c00640 1 --2- 172.21.15.119:0/3799964920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e40b5050 0x7f45e4130950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.908+0000 7f45e9c00640 1 -- 172.21.15.119:0/3799964920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e4144470 msgr2=0x7f45e415f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.908+0000 7f45e9c00640 1 --2- 172.21.15.119:0/3799964920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e4144470 0x7f45e415f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.908+0000 7f45e9c00640 1 -- 172.21.15.119:0/3799964920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e415e6b0 con 0x7f45e406e430 2024-04-25T10:17:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.909+0000 7f45e9c00640 1 --2- 172.21.15.119:0/3799964920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45e406e430 0x7f45e415b130 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c430 tx=0x7f45d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.909+0000 7f45e1600640 1 -- 172.21.15.119:0/3799964920 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d80074e0 con 0x7f45e406e430 2024-04-25T10:17:22.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.909+0000 7f45e1600640 1 -- 172.21.15.119:0/3799964920 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d80047e0 con 0x7f45e406e430 2024-04-25T10:17:22.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.909+0000 7f45eb000640 1 -- 172.21.15.119:0/3799964920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e415e990 con 0x7f45e406e430 2024-04-25T10:17:22.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.909+0000 7f45e1600640 1 -- 172.21.15.119:0/3799964920 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8004bd0 con 0x7f45e406e430 2024-04-25T10:17:22.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.909+0000 7f45eb000640 1 -- 172.21.15.119:0/3799964920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e415ef50 con 0x7f45e406e430 2024-04-25T10:17:22.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.910+0000 7f45eb000640 1 -- 172.21.15.119:0/3799964920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e40749b0 con 0x7f45e406e430 2024-04-25T10:17:22.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.911+0000 7f45e1600640 1 -- 172.21.15.119:0/3799964920 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45d8011070 con 0x7f45e406e430 2024-04-25T10:17:22.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.912+0000 7f45e1600640 1 --2- 172.21.15.119:0/3799964920 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f45c406b4d0 0x7f45c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:22.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.912+0000 7f45e1600640 1 -- 172.21.15.119:0/3799964920 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f45d8089fc0 con 0x7f45e406e430 2024-04-25T10:17:22.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.912+0000 7f45e9200640 1 --2- 172.21.15.119:0/3799964920 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f45c406b4d0 0x7f45c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:22.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.913+0000 7f45e9200640 1 --2- 172.21.15.119:0/3799964920 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f45c406b4d0 0x7f45c406d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f45d0034000 tx=0x7f45d002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:22.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:22.916+0000 7f45e1600640 1 -- 172.21.15.119:0/3799964920 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45d8058cf0 con 0x7f45e406e430 2024-04-25T10:17:23.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.135+0000 7f45eb000640 1 -- 172.21.15.119:0/3799964920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45e40b2070 con 0x7f45e406e430 2024-04-25T10:17:23.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.136+0000 7f45e1600640 1 -- 172.21.15.119:0/3799964920 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f45d80585f0 con 0x7f45e406e430 2024-04-25T10:17:23.140 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:23.140 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5459,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3768348114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3768348114},{"type":"v1","addr":"172.21.15.132:6834","nonce":3768348114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":5465,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/543190780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":543190780},{"type":"v1","addr":"172.21.15.132:6835","nonce":543190780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1145229172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1145229172},{"type":"v1","addr":"172.21.15.119:6836","nonce":1145229172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1704625876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1704625876},{"type":"v1","addr":"172.21.15.119:6837","nonce":1704625876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:51.568312+0000","modified":"2024-04-25T10:17:18.997282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:17:23.140 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 90 2024-04-25T10:17:23.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.139+0000 7f45eb000640 1 -- 172.21.15.119:0/3799964920 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f45c406b4d0 msgr2=0x7f45c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.139+0000 7f45eb000640 1 --2- 172.21.15.119:0/3799964920 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f45c406b4d0 0x7f45c406d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f45d0034000 tx=0x7f45d002c000 comp rx=0 tx=0).stop 2024-04-25T10:17:23.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.139+0000 7f45eb000640 1 -- 172.21.15.119:0/3799964920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45e406e430 msgr2=0x7f45e415b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.139+0000 7f45eb000640 1 --2- 172.21.15.119:0/3799964920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45e406e430 0x7f45e415b130 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c430 tx=0x7f45d800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.139+0000 7f45eb000640 1 -- 172.21.15.119:0/3799964920 shutdown_connections 2024-04-25T10:17:23.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.139+0000 7f45eb000640 1 --2- 172.21.15.119:0/3799964920 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f45c406b4d0 0x7f45c406d990 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.139+0000 7f45eb000640 1 --2- 172.21.15.119:0/3799964920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45e4144470 0x7f45e415f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.139+0000 7f45eb000640 1 --2- 172.21.15.119:0/3799964920 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45e40b5050 0x7f45e4130950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.139+0000 7f45eb000640 1 --2- 172.21.15.119:0/3799964920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45e406e430 0x7f45e415b130 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.139+0000 7f45eb000640 1 -- 172.21.15.119:0/3799964920 >> 172.21.15.119:0/3799964920 conn(0x7f45e406b1d0 msgr2=0x7f45e4141af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:23.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.139+0000 7f45eb000640 1 -- 172.21.15.119:0/3799964920 shutdown_connections 2024-04-25T10:17:23.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.139+0000 7f45eb000640 1 -- 172.21.15.119:0/3799964920 wait complete. 2024-04-25T10:17:23.160 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:23.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.352+0000 7f48ab200640 1 Processor -- start 2024-04-25T10:17:23.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.352+0000 7f48ab200640 1 -- start start 2024-04-25T10:17:23.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.354+0000 7f48ab200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f48a4144470 0x7f48a415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.354+0000 7f48ab200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a40cd440 con 0x7f48a406e430 2024-04-25T10:17:23.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.354+0000 7f48ab200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a40cd720 con 0x7f48a40b5050 2024-04-25T10:17:23.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.354+0000 7f48ab200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a4145340 con 0x7f48a4144470 2024-04-25T10:17:23.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.354+0000 7f48a9400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f48a406e430 0x7f48a40b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37948/0 (socket says 172.21.15.119:37948) 2024-04-25T10:17:23.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.354+0000 7f48a9400640 1 -- 172.21.15.119:0/1403454639 learned_addr learned my addr 172.21.15.119:0/1403454639 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.354+0000 7f48a9e00640 1 --2- 172.21.15.119:0/1403454639 >> v2:172.21.15.132:3300/0 conn(0x7f48a4144470 0x7f48a415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.354+0000 7f48a8a00640 1 -- 172.21.15.119:0/1403454639 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334006282 0 0) 0x7f48a40cd440 con 0x7f48a406e430 2024-04-25T10:17:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.354+0000 7f48a8a00640 1 -- 172.21.15.119:0/1403454639 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f488c003520 con 0x7f48a406e430 2024-04-25T10:17:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.355+0000 7f48a8a00640 1 -- 172.21.15.119:0/1403454639 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1048764639 0 0) 0x7f488c003520 con 0x7f48a406e430 2024-04-25T10:17:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.355+0000 7f48a8a00640 1 -- 172.21.15.119:0/1403454639 >> v2:172.21.15.132:3300/0 conn(0x7f48a4144470 msgr2=0x7f48a415fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.355+0000 7f48a8a00640 1 --2- 172.21.15.119:0/1403454639 >> v2:172.21.15.132:3300/0 conn(0x7f48a4144470 0x7f48a415fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.355+0000 7f48a8a00640 1 -- 172.21.15.119:0/1403454639 >> v1:172.21.15.132:6789/0 conn(0x7f48a40b5050 legacy=0x7f48a415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.355+0000 7f48a8a00640 1 -- 172.21.15.119:0/1403454639 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a40cda00 con 0x7f48a406e430 2024-04-25T10:17:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.355+0000 7f48a8a00640 1 -- 172.21.15.119:0/1403454639 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f4894001670 con 0x7f48a406e430 2024-04-25T10:17:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.355+0000 7f48a8a00640 1 -- 172.21.15.119:0/1403454639 >> v1:172.21.15.119:6789/0 conn(0x7f48a406e430 legacy=0x7f48a40b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.355+0000 7f48a8a00640 1 --2- 172.21.15.119:0/1403454639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f488c003a20 0x7f488c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.355+0000 7f48a8a00640 1 --2- 172.21.15.119:0/1403454639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f488c004800 0x7f488c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.355+0000 7f48a8a00640 1 --2- 172.21.15.119:0/1403454639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f488c0051c0 0x7f488c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.355+0000 7f48a8a00640 1 -- 172.21.15.119:0/1403454639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c00c0c0 con 0x7f488c003a20 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.355+0000 7f48a8a00640 1 -- 172.21.15.119:0/1403454639 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c00c270 con 0x7f488c004800 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.355+0000 7f48a8a00640 1 -- 172.21.15.119:0/1403454639 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c00c470 con 0x7f488c0051c0 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.355+0000 7f48a9e00640 1 --2- 172.21.15.119:0/1403454639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f488c003a20 0x7f488c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.356+0000 7f48a9400640 1 --2- 172.21.15.119:0/1403454639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f488c004800 0x7f488c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.356+0000 7f48a9e00640 1 -- 172.21.15.119:0/1403454639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f488c0051c0 msgr2=0x7f488c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.356+0000 7f48a9e00640 1 --2- 172.21.15.119:0/1403454639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f488c0051c0 0x7f488c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.356+0000 7f48a9e00640 1 -- 172.21.15.119:0/1403454639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f488c004800 msgr2=0x7f488c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.356+0000 7f48a9e00640 1 --2- 172.21.15.119:0/1403454639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f488c004800 0x7f488c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.356+0000 7f48a9e00640 1 -- 172.21.15.119:0/1403454639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c003520 con 0x7f488c003a20 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.356+0000 7f48a9400640 1 --2- 172.21.15.119:0/1403454639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f488c004800 0x7f488c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.356+0000 7f48aa800640 1 -- 172.21.15.119:0/1403454639 reap_dead start 2024-04-25T10:17:23.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.357+0000 7f48a9e00640 1 --2- 172.21.15.119:0/1403454639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f488c003a20 0x7f488c003e40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f4898009ac0 tx=0x7f488c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4727a63a5b870a59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.357+0000 7f48a8a00640 1 -- 172.21.15.119:0/1403454639 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898036070 con 0x7f488c003a20 2024-04-25T10:17:23.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.357+0000 7f48a8a00640 1 -- 172.21.15.119:0/1403454639 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4898031400 con 0x7f488c003a20 2024-04-25T10:17:23.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.357+0000 7f48a8a00640 1 -- 172.21.15.119:0/1403454639 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898009690 con 0x7f488c003a20 2024-04-25T10:17:23.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.358+0000 7f48ab200640 1 -- 172.21.15.119:0/1403454639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f488c003a20 msgr2=0x7f488c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.358+0000 7f48ab200640 1 --2- 172.21.15.119:0/1403454639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f488c003a20 0x7f488c003e40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f4898009ac0 tx=0x7f488c005830 comp rx=0 tx=0).stop 2024-04-25T10:17:23.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.358+0000 7f48ab200640 1 -- 172.21.15.119:0/1403454639 shutdown_connections 2024-04-25T10:17:23.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.358+0000 7f48ab200640 1 --2- 172.21.15.119:0/1403454639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f488c003a20 0x7f488c003e40 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.358+0000 7f48ab200640 1 -- 172.21.15.119:0/1403454639 >> 172.21.15.119:0/1403454639 conn(0x7f48a406b1d0 msgr2=0x7f48a40b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:23.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.358+0000 7f48ab200640 1 -- 172.21.15.119:0/1403454639 shutdown_connections 2024-04-25T10:17:23.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.358+0000 7f48ab200640 1 -- 172.21.15.119:0/1403454639 wait complete. 2024-04-25T10:17:23.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.360+0000 7f48ab200640 1 Processor -- start 2024-04-25T10:17:23.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.360+0000 7f48ab200640 1 -- start start 2024-04-25T10:17:23.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.361+0000 7f48ab200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f488c003a20 0x7f48a415d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.361+0000 7f48ab200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a40c89b0 0x7f48a415ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.361+0000 7f48ab200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48a40cbf10 0x7f48a40cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.361+0000 7f48ab200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40ce200 con 0x7f48a40c89b0 2024-04-25T10:17:23.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.361+0000 7f48ab200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a4144850 con 0x7f48a40cbf10 2024-04-25T10:17:23.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.361+0000 7f48ab200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a415fc90 con 0x7f488c003a20 2024-04-25T10:17:23.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.361+0000 7f48a9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a40c89b0 0x7f48a415ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.361+0000 7f48a9400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a40c89b0 0x7f48a415ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43072/0 (socket says 172.21.15.119:43072) 2024-04-25T10:17:23.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.361+0000 7f48a9400640 1 -- 172.21.15.119:0/3723207793 learned_addr learned my addr 172.21.15.119:0/3723207793 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:23.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.361+0000 7f48aa800640 1 --2- 172.21.15.119:0/3723207793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48a40cbf10 0x7f48a40cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.361+0000 7f48a9400640 1 -- 172.21.15.119:0/3723207793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f488c003a20 msgr2=0x7f48a415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.361+0000 7f48a9400640 1 --2- 172.21.15.119:0/3723207793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f488c003a20 0x7f48a415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.362+0000 7f48a9400640 1 -- 172.21.15.119:0/3723207793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48a40cbf10 msgr2=0x7f48a40cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.362+0000 7f48a9400640 1 --2- 172.21.15.119:0/3723207793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48a40cbf10 0x7f48a40cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.362+0000 7f48a9400640 1 -- 172.21.15.119:0/3723207793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a415dfe0 con 0x7f48a40c89b0 2024-04-25T10:17:23.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.362+0000 7f48aa800640 1 --2- 172.21.15.119:0/3723207793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48a40cbf10 0x7f48a40cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:23.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.362+0000 7f48a9400640 1 --2- 172.21.15.119:0/3723207793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a40c89b0 0x7f48a415ac40 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f4894000fb0 tx=0x7f489400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.362+0000 7f489e000640 1 -- 172.21.15.119:0/3723207793 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894019070 con 0x7f48a40c89b0 2024-04-25T10:17:23.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.362+0000 7f489e000640 1 -- 172.21.15.119:0/3723207793 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48940072d0 con 0x7f48a40c89b0 2024-04-25T10:17:23.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.362+0000 7f48ab200640 1 -- 172.21.15.119:0/3723207793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a415e2c0 con 0x7f48a40c89b0 2024-04-25T10:17:23.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.362+0000 7f489e000640 1 -- 172.21.15.119:0/3723207793 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48940047e0 con 0x7f48a40c89b0 2024-04-25T10:17:23.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.362+0000 7f48ab200640 1 -- 172.21.15.119:0/3723207793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a413b890 con 0x7f48a40c89b0 2024-04-25T10:17:23.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.364+0000 7f48ab200640 1 -- 172.21.15.119:0/3723207793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a40749b0 con 0x7f48a40c89b0 2024-04-25T10:17:23.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.364+0000 7f489e000640 1 -- 172.21.15.119:0/3723207793 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f489401c070 con 0x7f48a40c89b0 2024-04-25T10:17:23.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.365+0000 7f489e000640 1 --2- 172.21.15.119:0/3723207793 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f488806b500 0x7f488806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.365+0000 7f489e000640 1 -- 172.21.15.119:0/3723207793 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f4894089b70 con 0x7f48a40c89b0 2024-04-25T10:17:23.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.366+0000 7f48a9e00640 1 --2- 172.21.15.119:0/3723207793 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f488806b500 0x7f488806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.366+0000 7f48a9e00640 1 --2- 172.21.15.119:0/3723207793 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f488806b500 0x7f488806d9c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4898033000 tx=0x7f489802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.370+0000 7f489e000640 1 -- 172.21.15.119:0/3723207793 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4894058940 con 0x7f48a40c89b0 2024-04-25T10:17:23.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.603+0000 7f48ab200640 1 -- 172.21.15.119:0/3723207793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48a40b2090 con 0x7f48a40c89b0 2024-04-25T10:17:23.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.605+0000 7f489e000640 1 -- 172.21.15.119:0/3723207793 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f4894058240 con 0x7f48a40c89b0 2024-04-25T10:17:23.608 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:23.609 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5459,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3768348114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3768348114},{"type":"v1","addr":"172.21.15.132:6834","nonce":3768348114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":5465,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/543190780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":543190780},{"type":"v1","addr":"172.21.15.132:6835","nonce":543190780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1145229172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1145229172},{"type":"v1","addr":"172.21.15.119:6836","nonce":1145229172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1704625876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1704625876},{"type":"v1","addr":"172.21.15.119:6837","nonce":1704625876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:51.568312+0000","modified":"2024-04-25T10:17:18.997282+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:17:23.609 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 90 2024-04-25T10:17:23.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.607+0000 7f48ab200640 1 -- 172.21.15.119:0/3723207793 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f488806b500 msgr2=0x7f488806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.607+0000 7f48ab200640 1 --2- 172.21.15.119:0/3723207793 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f488806b500 0x7f488806d9c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4898033000 tx=0x7f489802e000 comp rx=0 tx=0).stop 2024-04-25T10:17:23.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.607+0000 7f48ab200640 1 -- 172.21.15.119:0/3723207793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a40c89b0 msgr2=0x7f48a415ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.607+0000 7f48ab200640 1 --2- 172.21.15.119:0/3723207793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a40c89b0 0x7f48a415ac40 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f4894000fb0 tx=0x7f489400dc00 comp rx=0 tx=0).stop 2024-04-25T10:17:23.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.607+0000 7f48ab200640 1 -- 172.21.15.119:0/3723207793 shutdown_connections 2024-04-25T10:17:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.608+0000 7f48ab200640 1 --2- 172.21.15.119:0/3723207793 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f488806b500 0x7f488806d9c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.608+0000 7f48ab200640 1 --2- 172.21.15.119:0/3723207793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48a40cbf10 0x7f48a40cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.608+0000 7f48ab200640 1 --2- 172.21.15.119:0/3723207793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a40c89b0 0x7f48a415ac40 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.608+0000 7f48ab200640 1 --2- 172.21.15.119:0/3723207793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f488c003a20 0x7f48a415d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.608+0000 7f48ab200640 1 -- 172.21.15.119:0/3723207793 >> 172.21.15.119:0/3723207793 conn(0x7f48a406b1d0 msgr2=0x7f48a4140b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.608+0000 7f48ab200640 1 -- 172.21.15.119:0/3723207793 shutdown_connections 2024-04-25T10:17:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.608+0000 7f48ab200640 1 -- 172.21.15.119:0/3723207793 wait complete. 2024-04-25T10:17:23.629 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:23.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.809+0000 7f7e8ca00640 1 Processor -- start 2024-04-25T10:17:23.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.809+0000 7f7e8ca00640 1 -- start start 2024-04-25T10:17:23.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.810+0000 7f7e8ca00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7e8815a080 0x7f7e8815e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.811+0000 7f7e8ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e880b5470 0x7f7e8806d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.811+0000 7f7e8ca00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e880cec30 con 0x7f7e88159ca0 2024-04-25T10:17:23.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.811+0000 7f7e8ca00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e880cef10 con 0x7f7e8815a080 2024-04-25T10:17:23.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.811+0000 7f7e8ca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e880cf1f0 con 0x7f7e880b5470 2024-04-25T10:17:23.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.811+0000 7f7e7fe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7e8815a080 0x7f7e8815e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.811+0000 7f7e87400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7e88159ca0 0x7f7e8815d910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37964/0 (socket says 172.21.15.119:37964) 2024-04-25T10:17:23.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.811+0000 7f7e87400640 1 -- 172.21.15.119:0/1528174627 learned_addr learned my addr 172.21.15.119:0/1528174627 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:23.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.811+0000 7f7e87e00640 1 --2- 172.21.15.119:0/1528174627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e880b5470 0x7f7e8806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.811+0000 7f7e7fe00640 1 -- 172.21.15.119:0/1528174627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e880b5470 msgr2=0x7f7e8806d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:23.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.811+0000 7f7e7fe00640 1 --2- 172.21.15.119:0/1528174627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e880b5470 0x7f7e8806d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.811+0000 7f7e7fe00640 1 -- 172.21.15.119:0/1528174627 >> v1:172.21.15.119:6789/0 conn(0x7f7e88159ca0 legacy=0x7f7e8815d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.811+0000 7f7e7fe00640 1 -- 172.21.15.119:0/1528174627 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8806d420 con 0x7f7e8815a080 2024-04-25T10:17:23.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.811+0000 7f7e87e00640 1 --2- 172.21.15.119:0/1528174627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e880b5470 0x7f7e8806d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:23.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.812+0000 7f7e7fe00640 1 --2- 172.21.15.119:0/1528174627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8815a080 0x7f7e8815e020 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7e700097a0 tx=0x7f7e7002b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b57f0b5e79b217f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.813+0000 7f7e86a00640 1 -- 172.21.15.119:0/1528174627 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70038070 con 0x7f7e8815a080 2024-04-25T10:17:23.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.813+0000 7f7e86a00640 1 -- 172.21.15.119:0/1528174627 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e70033440 con 0x7f7e8815a080 2024-04-25T10:17:23.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.813+0000 7f7e86a00640 1 -- 172.21.15.119:0/1528174627 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e700337f0 con 0x7f7e8815a080 2024-04-25T10:17:23.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.813+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1528174627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8815a080 msgr2=0x7f7e8815e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.813+0000 7f7e8ca00640 1 --2- 172.21.15.119:0/1528174627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8815a080 0x7f7e8815e020 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7e700097a0 tx=0x7f7e7002b290 comp rx=0 tx=0).stop 2024-04-25T10:17:23.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.813+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1528174627 shutdown_connections 2024-04-25T10:17:23.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.813+0000 7f7e8ca00640 1 --2- 172.21.15.119:0/1528174627 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e880b5470 0x7f7e8806d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.813+0000 7f7e8ca00640 1 --2- 172.21.15.119:0/1528174627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8815a080 0x7f7e8815e020 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.813+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1528174627 >> 172.21.15.119:0/1528174627 conn(0x7f7e8806b1d0 msgr2=0x7f7e880b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:23.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.813+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1528174627 shutdown_connections 2024-04-25T10:17:23.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.813+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1528174627 wait complete. 2024-04-25T10:17:23.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.815+0000 7f7e8ca00640 1 Processor -- start 2024-04-25T10:17:23.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.815+0000 7f7e8ca00640 1 -- start start 2024-04-25T10:17:23.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.816+0000 7f7e8ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e880b5470 0x7f7e881ef330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.816+0000 7f7e8ca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e88159ca0 0x7f7e881f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.816+0000 7f7e8ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8813ce40 0x7f7e8815b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.816+0000 7f7e8ca00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8815e6e0 con 0x7f7e8813ce40 2024-04-25T10:17:23.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.816+0000 7f7e8ca00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8815e560 con 0x7f7e88159ca0 2024-04-25T10:17:23.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.816+0000 7f7e8ca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e880b1a30 con 0x7f7e880b5470 2024-04-25T10:17:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.816+0000 7f7e87e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8813ce40 0x7f7e8815b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.816+0000 7f7e87e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8813ce40 0x7f7e8815b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43084/0 (socket says 172.21.15.119:43084) 2024-04-25T10:17:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.816+0000 7f7e87e00640 1 -- 172.21.15.119:0/1870410572 learned_addr learned my addr 172.21.15.119:0/1870410572 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.817+0000 7f7e87e00640 1 -- 172.21.15.119:0/1870410572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e880b5470 msgr2=0x7f7e881ef330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.817+0000 7f7e7fe00640 1 --2- 172.21.15.119:0/1870410572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e88159ca0 0x7f7e881f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.817+0000 7f7e87e00640 1 --2- 172.21.15.119:0/1870410572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e880b5470 0x7f7e881ef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.817+0000 7f7e87e00640 1 -- 172.21.15.119:0/1870410572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e88159ca0 msgr2=0x7f7e881f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.817+0000 7f7e87e00640 1 --2- 172.21.15.119:0/1870410572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e88159ca0 0x7f7e881f33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.817+0000 7f7e87e00640 1 -- 172.21.15.119:0/1870410572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e881f0300 con 0x7f7e8813ce40 2024-04-25T10:17:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.817+0000 7f7e87e00640 1 --2- 172.21.15.119:0/1870410572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8813ce40 0x7f7e8815b4c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000ce80 tx=0x7f7e8000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.817+0000 7f7e7f400640 1 -- 172.21.15.119:0/1870410572 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8001a070 con 0x7f7e8813ce40 2024-04-25T10:17:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.817+0000 7f7e7f400640 1 -- 172.21.15.119:0/1870410572 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8000d880 con 0x7f7e8813ce40 2024-04-25T10:17:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.817+0000 7f7e7f400640 1 -- 172.21.15.119:0/1870410572 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000dc70 con 0x7f7e8813ce40 2024-04-25T10:17:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.817+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1870410572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e881efa60 con 0x7f7e8813ce40 2024-04-25T10:17:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.817+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1870410572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e881f0020 con 0x7f7e8813ce40 2024-04-25T10:17:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.818+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1870410572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e880749b0 con 0x7f7e8813ce40 2024-04-25T10:17:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.819+0000 7f7e7f400640 1 -- 172.21.15.119:0/1870410572 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e8001d070 con 0x7f7e8813ce40 2024-04-25T10:17:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.820+0000 7f7e7f400640 1 --2- 172.21.15.119:0/1870410572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7e6406b4d0 0x7f7e6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.820+0000 7f7e7f400640 1 -- 172.21.15.119:0/1870410572 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f7e8008aad0 con 0x7f7e8813ce40 2024-04-25T10:17:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.820+0000 7f7e87400640 1 --2- 172.21.15.119:0/1870410572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7e6406b4d0 0x7f7e6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.821+0000 7f7e87400640 1 --2- 172.21.15.119:0/1870410572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7e6406b4d0 0x7f7e6406d990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7e78000f60 tx=0x7f7e78007a80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:23.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:23.824+0000 7f7e7f400640 1 -- 172.21.15.119:0/1870410572 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e800591b0 con 0x7f7e8813ce40 2024-04-25T10:17:24.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.007+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1870410572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e8806b5b0 con 0x7f7e8813ce40 2024-04-25T10:17:24.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.008+0000 7f7e7f400640 1 -- 172.21.15.119:0/1870410572 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16905 (secure 0 0 0) 0x7f7e8005d250 con 0x7f7e8813ce40 2024-04-25T10:17:24.012 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":98,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:17:18.997299+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:49.348469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:50.584869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/464677621":"2024-04-26T10:17:18.997269+0000","172.21.15.119:6835/464677621":"2024-04-26T10:17:18.997269+0000","172.21.15.119:6837/1945631445":"2024-04-26T10:16:41.666750+0000","172.21.15.119:6835/1945631445":"2024-04-26T10:16:41.666750+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:24.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.011+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1870410572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7e6406b4d0 msgr2=0x7f7e6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.011+0000 7f7e8ca00640 1 --2- 172.21.15.119:0/1870410572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7e6406b4d0 0x7f7e6406d990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7e78000f60 tx=0x7f7e78007a80 comp rx=0 tx=0).stop 2024-04-25T10:17:24.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.011+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1870410572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8813ce40 msgr2=0x7f7e8815b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.011+0000 7f7e8ca00640 1 --2- 172.21.15.119:0/1870410572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8813ce40 0x7f7e8815b4c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000ce80 tx=0x7f7e8000a710 comp rx=0 tx=0).stop 2024-04-25T10:17:24.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.011+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1870410572 shutdown_connections 2024-04-25T10:17:24.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.011+0000 7f7e8ca00640 1 --2- 172.21.15.119:0/1870410572 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7e6406b4d0 0x7f7e6406d990 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.011+0000 7f7e8ca00640 1 --2- 172.21.15.119:0/1870410572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8813ce40 0x7f7e8815b4c0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.011+0000 7f7e8ca00640 1 --2- 172.21.15.119:0/1870410572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e88159ca0 0x7f7e881f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.011+0000 7f7e8ca00640 1 --2- 172.21.15.119:0/1870410572 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e880b5470 0x7f7e881ef330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.011+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1870410572 >> 172.21.15.119:0/1870410572 conn(0x7f7e8806b1d0 msgr2=0x7f7e880b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:24.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.012+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1870410572 shutdown_connections 2024-04-25T10:17:24.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.012+0000 7f7e8ca00640 1 -- 172.21.15.119:0/1870410572 wait complete. 2024-04-25T10:17:24.033 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:17:24.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.230+0000 7f0eaf200640 1 Processor -- start 2024-04-25T10:17:24.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.230+0000 7f0eaf200640 1 -- start start 2024-04-25T10:17:24.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.231+0000 7f0eaf200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0eb00ce860 0x7f0eb00cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.231+0000 7f0eaf200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0eb00d00a0 0x7f0eb00d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.231+0000 7f0eaf200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb00d0a50 con 0x7f0eb015b7e0 2024-04-25T10:17:24.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.231+0000 7f0eaf200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb00b5710 con 0x7f0eb00d00a0 2024-04-25T10:17:24.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.232+0000 7f0eaf200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb00b59f0 con 0x7f0eb00ce860 2024-04-25T10:17:24.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.232+0000 7f0eae800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0eb00d00a0 0x7f0eb00d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.232+0000 7f0eae800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0eb00d00a0 0x7f0eb00d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43098/0 (socket says 172.21.15.119:43098) 2024-04-25T10:17:24.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.232+0000 7f0eae800640 1 -- 172.21.15.119:0/3540819507 learned_addr learned my addr 172.21.15.119:0/3540819507 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:24.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.232+0000 7f0ead400640 1 --2- 172.21.15.119:0/3540819507 >> v2:172.21.15.132:3300/0 conn(0x7f0eb00ce860 0x7f0eb00cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.232+0000 7f0eae800640 1 -- 172.21.15.119:0/3540819507 >> v2:172.21.15.132:3300/0 conn(0x7f0eb00ce860 msgr2=0x7f0eb00cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:24.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.232+0000 7f0eae800640 1 --2- 172.21.15.119:0/3540819507 >> v2:172.21.15.132:3300/0 conn(0x7f0eb00ce860 0x7f0eb00cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.232+0000 7f0eae800640 1 -- 172.21.15.119:0/3540819507 >> v1:172.21.15.132:6789/0 conn(0x7f0eb015b7e0 legacy=0x7f0eb015e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:24.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.232+0000 7f0eae800640 1 -- 172.21.15.119:0/3540819507 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb00b5cd0 con 0x7f0eb00d00a0 2024-04-25T10:17:24.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.233+0000 7f0eae800640 1 --2- 172.21.15.119:0/3540819507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb00d00a0 0x7f0eb00d0510 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00b64d0 tx=0x7f0ea802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37805dca3b1a7844 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:24.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.234+0000 7f0eaca00640 1 -- 172.21.15.119:0/3540819507 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8037070 con 0x7f0eb00d00a0 2024-04-25T10:17:24.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.234+0000 7f0eaca00640 1 -- 172.21.15.119:0/3540819507 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea802ae20 con 0x7f0eb00d00a0 2024-04-25T10:17:24.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.234+0000 7f0eaca00640 1 -- 172.21.15.119:0/3540819507 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8032660 con 0x7f0eb00d00a0 2024-04-25T10:17:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.234+0000 7f0eaf200640 1 -- 172.21.15.119:0/3540819507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb00d00a0 msgr2=0x7f0eb00d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.234+0000 7f0eaf200640 1 --2- 172.21.15.119:0/3540819507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb00d00a0 0x7f0eb00d0510 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00b64d0 tx=0x7f0ea802a110 comp rx=0 tx=0).stop 2024-04-25T10:17:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.235+0000 7f0eaf200640 1 -- 172.21.15.119:0/3540819507 shutdown_connections 2024-04-25T10:17:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.235+0000 7f0eaf200640 1 --2- 172.21.15.119:0/3540819507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb00d00a0 0x7f0eb00d0510 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.235+0000 7f0eaf200640 1 --2- 172.21.15.119:0/3540819507 >> v2:172.21.15.132:3300/0 conn(0x7f0eb00ce860 0x7f0eb00cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.235+0000 7f0eaf200640 1 -- 172.21.15.119:0/3540819507 >> 172.21.15.119:0/3540819507 conn(0x7f0eb00722d0 msgr2=0x7f0eb00726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.235+0000 7f0eaf200640 1 -- 172.21.15.119:0/3540819507 shutdown_connections 2024-04-25T10:17:24.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.235+0000 7f0eaf200640 1 -- 172.21.15.119:0/3540819507 wait complete. 2024-04-25T10:17:24.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.236+0000 7f0eaf200640 1 Processor -- start 2024-04-25T10:17:24.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.237+0000 7f0eaf200640 1 -- start start 2024-04-25T10:17:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.237+0000 7f0eaf200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0eb00ce860 0x7f0eb0144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.237+0000 7f0eaf200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb00d00a0 0x7f0eb00c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.237+0000 7f0eaf200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0eb015b7e0 0x7f0eb01f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.237+0000 7f0eaf200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb0161960 con 0x7f0eb00d00a0 2024-04-25T10:17:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.237+0000 7f0eaf200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb00b53c0 con 0x7f0eb015b7e0 2024-04-25T10:17:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.237+0000 7f0eaf200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb0161ae0 con 0x7f0eb00ce860 2024-04-25T10:17:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.238+0000 7f0ead400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb00d00a0 0x7f0eb00c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.238+0000 7f0ead400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb00d00a0 0x7f0eb00c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43114/0 (socket says 172.21.15.119:43114) 2024-04-25T10:17:24.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.238+0000 7f0ead400640 1 -- 172.21.15.119:0/235224981 learned_addr learned my addr 172.21.15.119:0/235224981 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.238+0000 7f0eae800640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0eb015b7e0 0x7f0eb01f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.238+0000 7f0ead400640 1 -- 172.21.15.119:0/235224981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0eb00ce860 msgr2=0x7f0eb0144e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.238+0000 7f0eade00640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0eb00ce860 0x7f0eb0144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.238+0000 7f0ead400640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0eb00ce860 0x7f0eb0144e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.238+0000 7f0ead400640 1 -- 172.21.15.119:0/235224981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0eb015b7e0 msgr2=0x7f0eb01f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.238+0000 7f0ead400640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0eb015b7e0 0x7f0eb01f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.238+0000 7f0ead400640 1 -- 172.21.15.119:0/235224981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb015ea40 con 0x7f0eb00d00a0 2024-04-25T10:17:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.238+0000 7f0eae800640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0eb015b7e0 0x7f0eb01f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.239+0000 7f0ead400640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb00d00a0 0x7f0eb00c98e0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f0e98007ef0 tx=0x7f0e9800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.239+0000 7f0ea6000640 1 -- 172.21.15.119:0/235224981 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e98014620 con 0x7f0eb00d00a0 2024-04-25T10:17:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.239+0000 7f0ea6000640 1 -- 172.21.15.119:0/235224981 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e98019070 con 0x7f0eb00d00a0 2024-04-25T10:17:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.239+0000 7f0ea6000640 1 -- 172.21.15.119:0/235224981 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e98014a00 con 0x7f0eb00d00a0 2024-04-25T10:17:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.239+0000 7f0eaf200640 1 -- 172.21.15.119:0/235224981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb01520a0 con 0x7f0eb00d00a0 2024-04-25T10:17:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.239+0000 7f0eaf200640 1 -- 172.21.15.119:0/235224981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb015e760 con 0x7f0eb00d00a0 2024-04-25T10:17:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.240+0000 7f0eaf200640 1 -- 172.21.15.119:0/235224981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb00749b0 con 0x7f0eb00d00a0 2024-04-25T10:17:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.241+0000 7f0ea6000640 1 -- 172.21.15.119:0/235224981 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e98014ba0 con 0x7f0eb00d00a0 2024-04-25T10:17:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.242+0000 7f0ea6000640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0e8c06b4d0 0x7f0e8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.242+0000 7f0ea6000640 1 -- 172.21.15.119:0/235224981 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f0e9808ac00 con 0x7f0eb00d00a0 2024-04-25T10:17:24.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.242+0000 7f0eade00640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0e8c06b4d0 0x7f0e8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:24.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.243+0000 7f0eade00640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0e8c06b4d0 0x7f0e8c06d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000a750 tx=0x7f0ea0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:24.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.246+0000 7f0ea6000640 1 -- 172.21.15.119:0/235224981 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e980599c0 con 0x7f0eb00d00a0 2024-04-25T10:17:24.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:24.481+0000 7f0eaf200640 1 -- 172.21.15.119:0/235224981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0eb00b7720 con 0x7f0eb00d00a0 2024-04-25T10:17:25.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.042+0000 7f0ea6000640 1 -- 172.21.15.119:0/235224981 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0e980592c0 con 0x7f0eb00d00a0 2024-04-25T10:17:25.046 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:17:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.045+0000 7f0eaf200640 1 -- 172.21.15.119:0/235224981 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0e8c06b4d0 msgr2=0x7f0e8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.045+0000 7f0eaf200640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0e8c06b4d0 0x7f0e8c06d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000a750 tx=0x7f0ea0008040 comp rx=0 tx=0).stop 2024-04-25T10:17:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.045+0000 7f0eaf200640 1 -- 172.21.15.119:0/235224981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb00d00a0 msgr2=0x7f0eb00c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.045+0000 7f0eaf200640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb00d00a0 0x7f0eb00c98e0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f0e98007ef0 tx=0x7f0e9800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.046+0000 7f0eaf200640 1 -- 172.21.15.119:0/235224981 shutdown_connections 2024-04-25T10:17:25.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.046+0000 7f0eaf200640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0e8c06b4d0 0x7f0e8c06d990 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.046+0000 7f0eaf200640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0eb015b7e0 0x7f0eb01f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.046+0000 7f0eaf200640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb00d00a0 0x7f0eb00c98e0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.046+0000 7f0eaf200640 1 --2- 172.21.15.119:0/235224981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0eb00ce860 0x7f0eb0144e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.046+0000 7f0eaf200640 1 -- 172.21.15.119:0/235224981 >> 172.21.15.119:0/235224981 conn(0x7f0eb00722d0 msgr2=0x7f0eb00726b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:25.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.046+0000 7f0eaf200640 1 -- 172.21.15.119:0/235224981 shutdown_connections 2024-04-25T10:17:25.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.046+0000 7f0eaf200640 1 -- 172.21.15.119:0/235224981 wait complete. 2024-04-25T10:17:25.070 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:17:25.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.265+0000 7f3077200640 1 Processor -- start 2024-04-25T10:17:25.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.265+0000 7f3077200640 1 -- start start 2024-04-25T10:17:25.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.266+0000 7f3077200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f307813dce0 0x7f3078071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:25.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.266+0000 7f3077200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3078071660 0x7f307815b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:25.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.266+0000 7f3077200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307813d9a0 con 0x7f307813d5c0 2024-04-25T10:17:25.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.266+0000 7f3077200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307815abf0 con 0x7f3078071660 2024-04-25T10:17:25.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.266+0000 7f3077200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3078071a90 con 0x7f307813dce0 2024-04-25T10:17:25.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.267+0000 7f3076800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3078071660 0x7f307815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:25.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.267+0000 7f3076800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3078071660 0x7f307815b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43130/0 (socket says 172.21.15.119:43130) 2024-04-25T10:17:25.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.267+0000 7f3076800640 1 -- 172.21.15.119:0/3502639831 learned_addr learned my addr 172.21.15.119:0/3502639831 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:25.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.267+0000 7f306d400640 1 --2- 172.21.15.119:0/3502639831 >> v2:172.21.15.132:3300/0 conn(0x7f307813dce0 0x7f3078071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:25.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.267+0000 7f3076800640 1 -- 172.21.15.119:0/3502639831 >> v2:172.21.15.132:3300/0 conn(0x7f307813dce0 msgr2=0x7f3078071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:25.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.267+0000 7f3076800640 1 --2- 172.21.15.119:0/3502639831 >> v2:172.21.15.132:3300/0 conn(0x7f307813dce0 0x7f3078071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.267+0000 7f3076800640 1 -- 172.21.15.119:0/3502639831 >> v1:172.21.15.132:6789/0 conn(0x7f307813d5c0 legacy=0x7f30781407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:25.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.267+0000 7f3076800640 1 -- 172.21.15.119:0/3502639831 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307815e650 con 0x7f3078071660 2024-04-25T10:17:25.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.268+0000 7f3076800640 1 --2- 172.21.15.119:0/3502639831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3078071660 0x7f307815b090 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f30780721c0 tx=0x7f307002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf7d2951a67ddb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:25.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.269+0000 7f3075400640 1 -- 172.21.15.119:0/3502639831 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070036070 con 0x7f3078071660 2024-04-25T10:17:25.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.269+0000 7f3075400640 1 -- 172.21.15.119:0/3502639831 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3070031600 con 0x7f3078071660 2024-04-25T10:17:25.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.269+0000 7f3075400640 1 -- 172.21.15.119:0/3502639831 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30700319d0 con 0x7f3078071660 2024-04-25T10:17:25.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.269+0000 7f3077200640 1 -- 172.21.15.119:0/3502639831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3078071660 msgr2=0x7f307815b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:25.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.269+0000 7f3077200640 1 --2- 172.21.15.119:0/3502639831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3078071660 0x7f307815b090 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f30780721c0 tx=0x7f307002a710 comp rx=0 tx=0).stop 2024-04-25T10:17:25.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.269+0000 7f3077200640 1 -- 172.21.15.119:0/3502639831 shutdown_connections 2024-04-25T10:17:25.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.269+0000 7f3077200640 1 --2- 172.21.15.119:0/3502639831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3078071660 0x7f307815b090 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.269+0000 7f3077200640 1 --2- 172.21.15.119:0/3502639831 >> v2:172.21.15.132:3300/0 conn(0x7f307813dce0 0x7f3078071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.270+0000 7f3077200640 1 -- 172.21.15.119:0/3502639831 >> 172.21.15.119:0/3502639831 conn(0x7f307806aba0 msgr2=0x7f307806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:25.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.270+0000 7f3077200640 1 -- 172.21.15.119:0/3502639831 shutdown_connections 2024-04-25T10:17:25.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.270+0000 7f3077200640 1 -- 172.21.15.119:0/3502639831 wait complete. 2024-04-25T10:17:25.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.271+0000 7f3077200640 1 Processor -- start 2024-04-25T10:17:25.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.272+0000 7f3077200640 1 -- start start 2024-04-25T10:17:25.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.272+0000 7f3077200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3078071660 0x7f3078141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:25.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.272+0000 7f3077200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f307813d5c0 0x7f3078143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:25.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f3077200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f307813dce0 0x7f3078138680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:25.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f3077200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307815eef0 con 0x7f307813d5c0 2024-04-25T10:17:25.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f3077200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307815ec10 con 0x7f3078071660 2024-04-25T10:17:25.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f3077200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30780b1980 con 0x7f307813dce0 2024-04-25T10:17:25.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f306d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f307813d5c0 0x7f3078143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:25.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f306d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f307813d5c0 0x7f3078143960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43132/0 (socket says 172.21.15.119:43132) 2024-04-25T10:17:25.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f306d400640 1 -- 172.21.15.119:0/1507172535 learned_addr learned my addr 172.21.15.119:0/1507172535 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:25.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f3075e00640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3078071660 0x7f3078141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:25.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f306d400640 1 -- 172.21.15.119:0/1507172535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f307813dce0 msgr2=0x7f3078138680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:25.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f3076800640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f307813dce0 0x7f3078138680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:25.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f306d400640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f307813dce0 0x7f3078138680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f306d400640 1 -- 172.21.15.119:0/1507172535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3078071660 msgr2=0x7f3078141d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:25.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f306d400640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3078071660 0x7f3078141d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:25.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f306d400640 1 -- 172.21.15.119:0/1507172535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30781f38c0 con 0x7f307813d5c0 2024-04-25T10:17:25.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.273+0000 7f3075e00640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3078071660 0x7f3078141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:25.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.274+0000 7f306d400640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f307813d5c0 0x7f3078143960 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f306000d620 tx=0x7f306000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:25.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.274+0000 7f306ea00640 1 -- 172.21.15.119:0/1507172535 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306000b620 con 0x7f307813d5c0 2024-04-25T10:17:25.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.274+0000 7f306ea00640 1 -- 172.21.15.119:0/1507172535 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3060019070 con 0x7f307813d5c0 2024-04-25T10:17:25.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.274+0000 7f306ea00640 1 -- 172.21.15.119:0/1507172535 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306000ba00 con 0x7f307813d5c0 2024-04-25T10:17:25.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.274+0000 7f3077200640 1 -- 172.21.15.119:0/1507172535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30781f3ba0 con 0x7f307813d5c0 2024-04-25T10:17:25.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.274+0000 7f3077200640 1 -- 172.21.15.119:0/1507172535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30781f4160 con 0x7f307813d5c0 2024-04-25T10:17:25.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.275+0000 7f3077200640 1 -- 172.21.15.119:0/1507172535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30780749b0 con 0x7f307813d5c0 2024-04-25T10:17:25.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.276+0000 7f306ea00640 1 -- 172.21.15.119:0/1507172535 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f306000bba0 con 0x7f307813d5c0 2024-04-25T10:17:25.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.277+0000 7f306ea00640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f305406b4d0 0x7f305406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:25.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.277+0000 7f306ea00640 1 -- 172.21.15.119:0/1507172535 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f306008a790 con 0x7f307813d5c0 2024-04-25T10:17:25.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.277+0000 7f3075e00640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f305406b4d0 0x7f305406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:25.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.278+0000 7f3075e00640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f305406b4d0 0x7f305406d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3068003eb0 tx=0x7f3068009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:25.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.281+0000 7f306ea00640 1 -- 172.21.15.119:0/1507172535 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30600594c0 con 0x7f307813d5c0 2024-04-25T10:17:25.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:25.507+0000 7f3077200640 1 -- 172.21.15.119:0/1507172535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f307813f5c0 con 0x7f307813d5c0 2024-04-25T10:17:26.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.053+0000 7f306ea00640 1 -- 172.21.15.119:0/1507172535 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3060058dc0 con 0x7f307813d5c0 2024-04-25T10:17:26.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.057+0000 7f3077200640 1 -- 172.21.15.119:0/1507172535 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f305406b4d0 msgr2=0x7f305406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:26.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.057+0000 7f3077200640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f305406b4d0 0x7f305406d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3068003eb0 tx=0x7f3068009040 comp rx=0 tx=0).stop 2024-04-25T10:17:26.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.057+0000 7f3077200640 1 -- 172.21.15.119:0/1507172535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f307813d5c0 msgr2=0x7f3078143960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:26.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.057+0000 7f3077200640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f307813d5c0 0x7f3078143960 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f306000d620 tx=0x7f306000dae0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.057+0000 7f3077200640 1 -- 172.21.15.119:0/1507172535 shutdown_connections 2024-04-25T10:17:26.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.057+0000 7f3077200640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f305406b4d0 0x7f305406d990 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.057+0000 7f3077200640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f307813dce0 0x7f3078138680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.057+0000 7f3077200640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f307813d5c0 0x7f3078143960 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.057+0000 7f3077200640 1 --2- 172.21.15.119:0/1507172535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3078071660 0x7f3078141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.057+0000 7f3077200640 1 -- 172.21.15.119:0/1507172535 >> 172.21.15.119:0/1507172535 conn(0x7f307806aba0 msgr2=0x7f307806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:26.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.057+0000 7f3077200640 1 -- 172.21.15.119:0/1507172535 shutdown_connections 2024-04-25T10:17:26.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.057+0000 7f3077200640 1 -- 172.21.15.119:0/1507172535 wait complete. 2024-04-25T10:17:26.083 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:17:26.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.285+0000 7fa433000640 1 Processor -- start 2024-04-25T10:17:26.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.285+0000 7fa433000640 1 -- start start 2024-04-25T10:17:26.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.287+0000 7fa433000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa42c15fa50 0x7fa42c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.287+0000 7fa433000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa42c0b13c0 0x7fa42c165a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.287+0000 7fa433000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa42c1625a0 con 0x7fa42c163550 2024-04-25T10:17:26.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.287+0000 7fa433000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa42c15fe30 con 0x7fa42c15fa50 2024-04-25T10:17:26.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.287+0000 7fa433000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c162880 con 0x7fa42c0b13c0 2024-04-25T10:17:26.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.287+0000 7fa431c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa42c15fa50 0x7fa42c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.287+0000 7fa431c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa42c15fa50 0x7fa42c06e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43140/0 (socket says 172.21.15.119:43140) 2024-04-25T10:17:26.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.287+0000 7fa431c00640 1 -- 172.21.15.119:0/3133222248 learned_addr learned my addr 172.21.15.119:0/3133222248 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:26.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.287+0000 7fa432600640 1 --2- 172.21.15.119:0/3133222248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa42c0b13c0 0x7fa42c165a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.288+0000 7fa431c00640 1 -- 172.21.15.119:0/3133222248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa42c0b13c0 msgr2=0x7fa42c165a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:26.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.288+0000 7fa431c00640 1 --2- 172.21.15.119:0/3133222248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa42c0b13c0 0x7fa42c165a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.288+0000 7fa431c00640 1 -- 172.21.15.119:0/3133222248 >> v1:172.21.15.132:6789/0 conn(0x7fa42c163550 legacy=0x7fa42c0b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:26.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.288+0000 7fa431c00640 1 -- 172.21.15.119:0/3133222248 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42c162b60 con 0x7fa42c15fa50 2024-04-25T10:17:26.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.289+0000 7fa431c00640 1 --2- 172.21.15.119:0/3133222248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42c15fa50 0x7fa42c06e890 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009870 tx=0x7fa41c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315130c84d6f6958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:26.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.290+0000 7fa42be00640 1 -- 172.21.15.119:0/3133222248 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c037070 con 0x7fa42c15fa50 2024-04-25T10:17:26.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.290+0000 7fa42be00640 1 -- 172.21.15.119:0/3133222248 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c032440 con 0x7fa42c15fa50 2024-04-25T10:17:26.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.290+0000 7fa42be00640 1 -- 172.21.15.119:0/3133222248 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c032810 con 0x7fa42c15fa50 2024-04-25T10:17:26.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.290+0000 7fa433000640 1 -- 172.21.15.119:0/3133222248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42c15fa50 msgr2=0x7fa42c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:26.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.290+0000 7fa433000640 1 --2- 172.21.15.119:0/3133222248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42c15fa50 0x7fa42c06e890 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009870 tx=0x7fa41c02a210 comp rx=0 tx=0).stop 2024-04-25T10:17:26.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.290+0000 7fa433000640 1 -- 172.21.15.119:0/3133222248 shutdown_connections 2024-04-25T10:17:26.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.290+0000 7fa433000640 1 --2- 172.21.15.119:0/3133222248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa42c0b13c0 0x7fa42c165a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.290+0000 7fa433000640 1 --2- 172.21.15.119:0/3133222248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42c15fa50 0x7fa42c06e890 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.290+0000 7fa433000640 1 -- 172.21.15.119:0/3133222248 >> 172.21.15.119:0/3133222248 conn(0x7fa42c06aba0 msgr2=0x7fa42c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:26.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.290+0000 7fa433000640 1 -- 172.21.15.119:0/3133222248 shutdown_connections 2024-04-25T10:17:26.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.290+0000 7fa433000640 1 -- 172.21.15.119:0/3133222248 wait complete. 2024-04-25T10:17:26.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.292+0000 7fa433000640 1 Processor -- start 2024-04-25T10:17:26.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.292+0000 7fa433000640 1 -- start start 2024-04-25T10:17:26.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.293+0000 7fa433000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa42c0b13c0 0x7fa42c152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.293+0000 7fa433000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa42c15fa50 0x7fa42c146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.293+0000 7fa433000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42c163550 0x7fa42c15c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.293+0000 7fa433000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c0cf260 con 0x7fa42c163550 2024-04-25T10:17:26.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.293+0000 7fa433000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c163120 con 0x7fa42c0b13c0 2024-04-25T10:17:26.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.293+0000 7fa433000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c0cf3e0 con 0x7fa42c15fa50 2024-04-25T10:17:26.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.293+0000 7fa432600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42c163550 0x7fa42c15c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.294+0000 7fa432600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42c163550 0x7fa42c15c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43146/0 (socket says 172.21.15.119:43146) 2024-04-25T10:17:26.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.294+0000 7fa432600640 1 -- 172.21.15.119:0/4121985451 learned_addr learned my addr 172.21.15.119:0/4121985451 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:26.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.294+0000 7fa431c00640 1 --2- 172.21.15.119:0/4121985451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa42c0b13c0 0x7fa42c152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.294+0000 7fa432600640 1 -- 172.21.15.119:0/4121985451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa42c15fa50 msgr2=0x7fa42c146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:26.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.294+0000 7fa432600640 1 --2- 172.21.15.119:0/4121985451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa42c15fa50 0x7fa42c146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.294+0000 7fa432600640 1 -- 172.21.15.119:0/4121985451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa42c0b13c0 msgr2=0x7fa42c152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:26.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.294+0000 7fa432600640 1 --2- 172.21.15.119:0/4121985451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa42c0b13c0 0x7fa42c152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.294+0000 7fa432600640 1 -- 172.21.15.119:0/4121985451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42c15ee40 con 0x7fa42c163550 2024-04-25T10:17:26.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.294+0000 7fa431c00640 1 --2- 172.21.15.119:0/4121985451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa42c0b13c0 0x7fa42c152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:26.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.294+0000 7fa432600640 1 --2- 172.21.15.119:0/4121985451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42c163550 0x7fa42c15c4a0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fa42400d830 tx=0x7fa42400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:26.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.294+0000 7fa429600640 1 -- 172.21.15.119:0/4121985451 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42400b6c0 con 0x7fa42c163550 2024-04-25T10:17:26.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.294+0000 7fa429600640 1 -- 172.21.15.119:0/4121985451 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa424019070 con 0x7fa42c163550 2024-04-25T10:17:26.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.294+0000 7fa429600640 1 -- 172.21.15.119:0/4121985451 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42400bac0 con 0x7fa42c163550 2024-04-25T10:17:26.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.294+0000 7fa433000640 1 -- 172.21.15.119:0/4121985451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42c15f120 con 0x7fa42c163550 2024-04-25T10:17:26.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.295+0000 7fa433000640 1 -- 172.21.15.119:0/4121985451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42c15ead0 con 0x7fa42c163550 2024-04-25T10:17:26.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.296+0000 7fa433000640 1 -- 172.21.15.119:0/4121985451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa42c153440 con 0x7fa42c163550 2024-04-25T10:17:26.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.297+0000 7fa429600640 1 -- 172.21.15.119:0/4121985451 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa42401e460 con 0x7fa42c163550 2024-04-25T10:17:26.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.298+0000 7fa429600640 1 --2- 172.21.15.119:0/4121985451 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa40c06b4d0 0x7fa40c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.298+0000 7fa429600640 1 -- 172.21.15.119:0/4121985451 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fa42408a960 con 0x7fa42c163550 2024-04-25T10:17:26.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.298+0000 7fa431c00640 1 --2- 172.21.15.119:0/4121985451 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa40c06b4d0 0x7fa40c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.299+0000 7fa431c00640 1 --2- 172.21.15.119:0/4121985451 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa40c06b4d0 0x7fa40c06d990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa41c034000 tx=0x7fa41c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:26.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.302+0000 7fa429600640 1 -- 172.21.15.119:0/4121985451 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa424059720 con 0x7fa42c163550 2024-04-25T10:17:26.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:26.500+0000 7fa433000640 1 -- 172.21.15.119:0/4121985451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa42c0b18e0 con 0x7fa42c163550 2024-04-25T10:17:27.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.064+0000 7fa429600640 1 -- 172.21.15.119:0/4121985451 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa424059020 con 0x7fa42c163550 2024-04-25T10:17:27.068 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:17:27.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.068+0000 7fa433000640 1 -- 172.21.15.119:0/4121985451 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa40c06b4d0 msgr2=0x7fa40c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.068+0000 7fa433000640 1 --2- 172.21.15.119:0/4121985451 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa40c06b4d0 0x7fa40c06d990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa41c034000 tx=0x7fa41c02e000 comp rx=0 tx=0).stop 2024-04-25T10:17:27.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.068+0000 7fa433000640 1 -- 172.21.15.119:0/4121985451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42c163550 msgr2=0x7fa42c15c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.068+0000 7fa433000640 1 --2- 172.21.15.119:0/4121985451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42c163550 0x7fa42c15c4a0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fa42400d830 tx=0x7fa42400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.070+0000 7fa433000640 1 -- 172.21.15.119:0/4121985451 shutdown_connections 2024-04-25T10:17:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.070+0000 7fa433000640 1 --2- 172.21.15.119:0/4121985451 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa40c06b4d0 0x7fa40c06d990 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.070+0000 7fa433000640 1 --2- 172.21.15.119:0/4121985451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa42c163550 0x7fa42c15c4a0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.070+0000 7fa433000640 1 --2- 172.21.15.119:0/4121985451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa42c15fa50 0x7fa42c146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.070+0000 7fa433000640 1 --2- 172.21.15.119:0/4121985451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa42c0b13c0 0x7fa42c152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.070+0000 7fa433000640 1 -- 172.21.15.119:0/4121985451 >> 172.21.15.119:0/4121985451 conn(0x7fa42c06aba0 msgr2=0x7fa42c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.070+0000 7fa433000640 1 -- 172.21.15.119:0/4121985451 shutdown_connections 2024-04-25T10:17:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.070+0000 7fa433000640 1 -- 172.21.15.119:0/4121985451 wait complete. 2024-04-25T10:17:27.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:17:27.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.292+0000 7f57b4a00640 1 Processor -- start 2024-04-25T10:17:27.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.293+0000 7f57b4a00640 1 -- start start 2024-04-25T10:17:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.294+0000 7f57b4a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57b0071660 0x7f57b00cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.294+0000 7f57b4a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b00cc9a0 con 0x7f57b0071200 2024-04-25T10:17:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.294+0000 7f57b4a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b00ccc80 con 0x7f57b0145260 2024-04-25T10:17:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.294+0000 7f57b4a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b00cea00 con 0x7f57b0071660 2024-04-25T10:17:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.295+0000 7f57ae800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57b0071660 0x7f57b00cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.295+0000 7f57a5400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f57b0071200 0x7f57b00cee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37974/0 (socket says 172.21.15.119:37974) 2024-04-25T10:17:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.295+0000 7f57a5400640 1 -- 172.21.15.119:0/2604166658 learned_addr learned my addr 172.21.15.119:0/2604166658 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:27.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.295+0000 7f57ae800640 1 -- 172.21.15.119:0/2604166658 >> v1:172.21.15.132:6789/0 conn(0x7f57b0145260 legacy=0x7f57b00cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:27.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.295+0000 7f57ae800640 1 -- 172.21.15.119:0/2604166658 >> v1:172.21.15.119:6789/0 conn(0x7f57b0071200 legacy=0x7f57b00cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.295+0000 7f57ae800640 1 -- 172.21.15.119:0/2604166658 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b00ccf60 con 0x7f57b0071660 2024-04-25T10:17:27.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.296+0000 7f57ae800640 1 --2- 172.21.15.119:0/2604166658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b0071660 0x7f57b00cf5a0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009d90 tx=0x7f57a802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed824d9e55665114 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.296+0000 7f57ad400640 1 -- 172.21.15.119:0/2604166658 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8007740 con 0x7f57b0071660 2024-04-25T10:17:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.296+0000 7f57ad400640 1 -- 172.21.15.119:0/2604166658 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a80078e0 con 0x7f57b0071660 2024-04-25T10:17:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.296+0000 7f57ad400640 1 -- 172.21.15.119:0/2604166658 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8007cb0 con 0x7f57b0071660 2024-04-25T10:17:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.297+0000 7f57b4a00640 1 -- 172.21.15.119:0/2604166658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b0071660 msgr2=0x7f57b00cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.297+0000 7f57b4a00640 1 --2- 172.21.15.119:0/2604166658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b0071660 0x7f57b00cf5a0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009d90 tx=0x7f57a802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.297+0000 7f57b4a00640 1 -- 172.21.15.119:0/2604166658 shutdown_connections 2024-04-25T10:17:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.297+0000 7f57b4a00640 1 --2- 172.21.15.119:0/2604166658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b0071660 0x7f57b00cf5a0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.297+0000 7f57b4a00640 1 -- 172.21.15.119:0/2604166658 >> 172.21.15.119:0/2604166658 conn(0x7f57b006aba0 msgr2=0x7f57b006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:27.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.297+0000 7f57b4a00640 1 -- 172.21.15.119:0/2604166658 shutdown_connections 2024-04-25T10:17:27.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.297+0000 7f57b4a00640 1 -- 172.21.15.119:0/2604166658 wait complete. 2024-04-25T10:17:27.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.299+0000 7f57b4a00640 1 Processor -- start 2024-04-25T10:17:27.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.300+0000 7f57b4a00640 1 -- start start 2024-04-25T10:17:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.301+0000 7f57b4a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b0071200 0x7f57b00b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.301+0000 7f57b4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b0071660 0x7f57b015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.301+0000 7f57b4a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57b0145260 0x7f57b015e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.301+0000 7f57b4a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b0071a90 con 0x7f57b0071660 2024-04-25T10:17:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.301+0000 7f57b4a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b00cc660 con 0x7f57b0071200 2024-04-25T10:17:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.301+0000 7f57b4a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b0145640 con 0x7f57b0145260 2024-04-25T10:17:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.301+0000 7f57a5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b0071660 0x7f57b015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.301+0000 7f57a5400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b0071660 0x7f57b015b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43160/0 (socket says 172.21.15.119:43160) 2024-04-25T10:17:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.301+0000 7f57a5400640 1 -- 172.21.15.119:0/2863826364 learned_addr learned my addr 172.21.15.119:0/2863826364 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.302+0000 7f57ade00640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b0071200 0x7f57b00b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.302+0000 7f57ae800640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57b0145260 0x7f57b015e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.302+0000 7f57a5400640 1 -- 172.21.15.119:0/2863826364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57b0145260 msgr2=0x7f57b015e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.302+0000 7f57a5400640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57b0145260 0x7f57b015e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.302+0000 7f57a5400640 1 -- 172.21.15.119:0/2863826364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b0071200 msgr2=0x7f57b00b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.302+0000 7f57a5400640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b0071200 0x7f57b00b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.302+0000 7f57a5400640 1 -- 172.21.15.119:0/2863826364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b00b2dd0 con 0x7f57b0071660 2024-04-25T10:17:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.302+0000 7f57ade00640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b0071200 0x7f57b00b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.302+0000 7f57ae800640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57b0145260 0x7f57b015e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.302+0000 7f57a5400640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b0071660 0x7f57b015b420 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f579800c3f0 tx=0x7f579800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.302+0000 7f57a6a00640 1 -- 172.21.15.119:0/2863826364 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798017070 con 0x7f57b0071660 2024-04-25T10:17:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.302+0000 7f57a6a00640 1 -- 172.21.15.119:0/2863826364 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798005ce0 con 0x7f57b0071660 2024-04-25T10:17:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.302+0000 7f57a6a00640 1 -- 172.21.15.119:0/2863826364 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980072f0 con 0x7f57b0071660 2024-04-25T10:17:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.302+0000 7f57b4a00640 1 -- 172.21.15.119:0/2863826364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b00b30b0 con 0x7f57b0071660 2024-04-25T10:17:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.303+0000 7f57b4a00640 1 -- 172.21.15.119:0/2863826364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b00b3670 con 0x7f57b0071660 2024-04-25T10:17:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.304+0000 7f57b4a00640 1 -- 172.21.15.119:0/2863826364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b00750b0 con 0x7f57b0071660 2024-04-25T10:17:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.305+0000 7f57a6a00640 1 -- 172.21.15.119:0/2863826364 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f579801a070 con 0x7f57b0071660 2024-04-25T10:17:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.306+0000 7f57a6a00640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f578c06b4d0 0x7f578c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.306+0000 7f57a6a00640 1 -- 172.21.15.119:0/2863826364 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f57980894a0 con 0x7f57b0071660 2024-04-25T10:17:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.306+0000 7f57ade00640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f578c06b4d0 0x7f578c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.307+0000 7f57ade00640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f578c06b4d0 0x7f578c06d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f57a0003eb0 tx=0x7f57a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:27.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.312+0000 7f57a6a00640 1 -- 172.21.15.119:0/2863826364 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5798058420 con 0x7f57b0071660 2024-04-25T10:17:27.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:27.507+0000 7f57b4a00640 1 -- 172.21.15.119:0/2863826364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f57b0141c80 con 0x7f57b0071660 2024-04-25T10:17:28.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.073+0000 7f57a6a00640 1 -- 172.21.15.119:0/2863826364 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5798057d20 con 0x7f57b0071660 2024-04-25T10:17:28.077 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2024-04-25T10:17:28.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.076+0000 7f57b4a00640 1 -- 172.21.15.119:0/2863826364 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f578c06b4d0 msgr2=0x7f578c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.076+0000 7f57b4a00640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f578c06b4d0 0x7f578c06d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f57a0003eb0 tx=0x7f57a0009040 comp rx=0 tx=0).stop 2024-04-25T10:17:28.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.077+0000 7f57b4a00640 1 -- 172.21.15.119:0/2863826364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b0071660 msgr2=0x7f57b015b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.077+0000 7f57b4a00640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b0071660 0x7f57b015b420 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f579800c3f0 tx=0x7f579800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.077+0000 7f57b4a00640 1 -- 172.21.15.119:0/2863826364 shutdown_connections 2024-04-25T10:17:28.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.077+0000 7f57b4a00640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f578c06b4d0 0x7f578c06d990 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.077+0000 7f57b4a00640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57b0145260 0x7f57b015e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.077+0000 7f57b4a00640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57b0071660 0x7f57b015b420 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.077+0000 7f57b4a00640 1 --2- 172.21.15.119:0/2863826364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57b0071200 0x7f57b00b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.077+0000 7f57b4a00640 1 -- 172.21.15.119:0/2863826364 >> 172.21.15.119:0/2863826364 conn(0x7f57b006aba0 msgr2=0x7f57b00cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:28.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.077+0000 7f57b4a00640 1 -- 172.21.15.119:0/2863826364 shutdown_connections 2024-04-25T10:17:28.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.077+0000 7f57b4a00640 1 -- 172.21.15.119:0/2863826364 wait complete. 2024-04-25T10:17:28.106 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:17:28.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.298+0000 7fdc3da00640 1 Processor -- start 2024-04-25T10:17:28.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.298+0000 7fdc3da00640 1 -- start start 2024-04-25T10:17:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.299+0000 7fdc3da00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380b5550 0x7fdc380b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.299+0000 7fdc3da00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc38161500 con 0x7fdc38161120 2024-04-25T10:17:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.299+0000 7fdc3da00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc38161eb0 con 0x7fdc380d2560 2024-04-25T10:17:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.299+0000 7fdc3da00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380b5ef0 con 0x7fdc380b5550 2024-04-25T10:17:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.299+0000 7fdc36a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdc38161120 0x7fdc381617a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37984/0 (socket says 172.21.15.119:37984) 2024-04-25T10:17:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.299+0000 7fdc36a00640 1 -- 172.21.15.119:0/503567282 learned_addr learned my addr 172.21.15.119:0/503567282 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 -- 172.21.15.119:0/503567282 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563512695 0 0) 0x7fdc38161500 con 0x7fdc38161120 2024-04-25T10:17:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 -- 172.21.15.119:0/503567282 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc1c003520 con 0x7fdc38161120 2024-04-25T10:17:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc2e000640 1 --2- 172.21.15.119:0/503567282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380b5550 0x7fdc380b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 -- 172.21.15.119:0/503567282 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 223643537 0 0) 0x7fdc1c003520 con 0x7fdc38161120 2024-04-25T10:17:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 -- 172.21.15.119:0/503567282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380b5550 msgr2=0x7fdc380b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 --2- 172.21.15.119:0/503567282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380b5550 0x7fdc380b59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 -- 172.21.15.119:0/503567282 >> v1:172.21.15.132:6789/0 conn(0x7fdc380d2560 legacy=0x7fdc380d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 -- 172.21.15.119:0/503567282 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38162180 con 0x7fdc38161120 2024-04-25T10:17:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 -- 172.21.15.119:0/503567282 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fdc28001670 con 0x7fdc38161120 2024-04-25T10:17:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 -- 172.21.15.119:0/503567282 >> v1:172.21.15.119:6789/0 conn(0x7fdc38161120 legacy=0x7fdc381617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 --2- 172.21.15.119:0/503567282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380b5550 0x7fdc1c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 --2- 172.21.15.119:0/503567282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc1c0040d0 0x7fdc1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 --2- 172.21.15.119:0/503567282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc1c004db0 0x7fdc1c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 -- 172.21.15.119:0/503567282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c00bc90 con 0x7fdc1c004db0 2024-04-25T10:17:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 -- 172.21.15.119:0/503567282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c00be40 con 0x7fdc1c0040d0 2024-04-25T10:17:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.300+0000 7fdc36000640 1 -- 172.21.15.119:0/503567282 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c00c040 con 0x7fdc380b5550 2024-04-25T10:17:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.301+0000 7fdc37400640 1 --2- 172.21.15.119:0/503567282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc1c004db0 0x7fdc1c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.301+0000 7fdc37400640 1 -- 172.21.15.119:0/503567282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380b5550 msgr2=0x7fdc1c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.301+0000 7fdc37400640 1 --2- 172.21.15.119:0/503567282 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc380b5550 0x7fdc1c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.301+0000 7fdc37400640 1 -- 172.21.15.119:0/503567282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc1c0040d0 msgr2=0x7fdc1c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.301+0000 7fdc37400640 1 --2- 172.21.15.119:0/503567282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc1c0040d0 0x7fdc1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.301+0000 7fdc37400640 1 -- 172.21.15.119:0/503567282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c003520 con 0x7fdc1c004db0 2024-04-25T10:17:28.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.302+0000 7fdc37400640 1 --2- 172.21.15.119:0/503567282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc1c004db0 0x7fdc1c005230 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fdc300044b0 tx=0x7fdc1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470550803fbc1472 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.302+0000 7fdc36000640 1 -- 172.21.15.119:0/503567282 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30037070 con 0x7fdc1c004db0 2024-04-25T10:17:28.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.302+0000 7fdc36000640 1 -- 172.21.15.119:0/503567282 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3002acd0 con 0x7fdc1c004db0 2024-04-25T10:17:28.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.302+0000 7fdc36000640 1 -- 172.21.15.119:0/503567282 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30004080 con 0x7fdc1c004db0 2024-04-25T10:17:28.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.303+0000 7fdc3da00640 1 -- 172.21.15.119:0/503567282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc1c004db0 msgr2=0x7fdc1c005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.303+0000 7fdc3da00640 1 --2- 172.21.15.119:0/503567282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc1c004db0 0x7fdc1c005230 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fdc300044b0 tx=0x7fdc1c005830 comp rx=0 tx=0).stop 2024-04-25T10:17:28.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.303+0000 7fdc37400640 1 -- 172.21.15.119:0/503567282 reap_dead start 2024-04-25T10:17:28.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.303+0000 7fdc3da00640 1 -- 172.21.15.119:0/503567282 shutdown_connections 2024-04-25T10:17:28.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.303+0000 7fdc3da00640 1 -- 172.21.15.119:0/503567282 >> 172.21.15.119:0/503567282 conn(0x7fdc380d0550 msgr2=0x7fdc380d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:28.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.303+0000 7fdc3da00640 1 -- 172.21.15.119:0/503567282 shutdown_connections 2024-04-25T10:17:28.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.303+0000 7fdc3da00640 1 -- 172.21.15.119:0/503567282 wait complete. 2024-04-25T10:17:28.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.305+0000 7fdc3da00640 1 Processor -- start 2024-04-25T10:17:28.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.305+0000 7fdc3da00640 1 -- start start 2024-04-25T10:17:28.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.306+0000 7fdc3da00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c004db0 0x7fdc3815b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.306+0000 7fdc3da00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc380c9a60 0x7fdc380c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.306+0000 7fdc3da00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc380cac10 0x7fdc380cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.306+0000 7fdc3da00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380d1d60 con 0x7fdc380cac10 2024-04-25T10:17:28.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.306+0000 7fdc3da00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc38162740 con 0x7fdc380c9a60 2024-04-25T10:17:28.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.306+0000 7fdc3da00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380d2040 con 0x7fdc1c004db0 2024-04-25T10:17:28.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.306+0000 7fdc37400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc380cac10 0x7fdc380cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.306+0000 7fdc37400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc380cac10 0x7fdc380cb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43172/0 (socket says 172.21.15.119:43172) 2024-04-25T10:17:28.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.306+0000 7fdc37400640 1 -- 172.21.15.119:0/3402603114 learned_addr learned my addr 172.21.15.119:0/3402603114 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:28.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.307+0000 7fdc2e000640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc380c9a60 0x7fdc380c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.307+0000 7fdc36a00640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c004db0 0x7fdc3815b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.307+0000 7fdc37400640 1 -- 172.21.15.119:0/3402603114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c004db0 msgr2=0x7fdc3815b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.307+0000 7fdc37400640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c004db0 0x7fdc3815b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.307+0000 7fdc37400640 1 -- 172.21.15.119:0/3402603114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc380c9a60 msgr2=0x7fdc380c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.307+0000 7fdc37400640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc380c9a60 0x7fdc380c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.307+0000 7fdc37400640 1 -- 172.21.15.119:0/3402603114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc380cb940 con 0x7fdc380cac10 2024-04-25T10:17:28.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.307+0000 7fdc2e000640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc380c9a60 0x7fdc380c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.307+0000 7fdc37400640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc380cac10 0x7fdc380cb0c0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fdc30035040 tx=0x7fdc30003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.307+0000 7fdc2f400640 1 -- 172.21.15.119:0/3402603114 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30045070 con 0x7fdc380cac10 2024-04-25T10:17:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.308+0000 7fdc2f400640 1 -- 172.21.15.119:0/3402603114 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc30007640 con 0x7fdc380cac10 2024-04-25T10:17:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.308+0000 7fdc3da00640 1 -- 172.21.15.119:0/3402603114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc380cbc20 con 0x7fdc380cac10 2024-04-25T10:17:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.308+0000 7fdc2f400640 1 -- 172.21.15.119:0/3402603114 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30037040 con 0x7fdc380cac10 2024-04-25T10:17:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.308+0000 7fdc3da00640 1 -- 172.21.15.119:0/3402603114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc380c78e0 con 0x7fdc380cac10 2024-04-25T10:17:28.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.309+0000 7fdc3da00640 1 -- 172.21.15.119:0/3402603114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc380750b0 con 0x7fdc380cac10 2024-04-25T10:17:28.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.310+0000 7fdc2f400640 1 -- 172.21.15.119:0/3402603114 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc30048070 con 0x7fdc380cac10 2024-04-25T10:17:28.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.311+0000 7fdc2f400640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdc1406b500 0x7fdc1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.311+0000 7fdc2f400640 1 -- 172.21.15.119:0/3402603114 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fdc300a9b70 con 0x7fdc380cac10 2024-04-25T10:17:28.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.311+0000 7fdc36a00640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdc1406b500 0x7fdc1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.312+0000 7fdc36a00640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdc1406b500 0x7fdc1406d9c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdc28007a90 tx=0x7fdc28009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.316+0000 7fdc2f400640 1 -- 172.21.15.119:0/3402603114 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc30078c80 con 0x7fdc380cac10 2024-04-25T10:17:28.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.524+0000 7fdc3da00640 1 -- 172.21.15.119:0/3402603114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdc380d17e0 con 0x7fdc380cac10 2024-04-25T10:17:28.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.531+0000 7fdc2f400640 1 -- 172.21.15.119:0/3402603114 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdc30078580 con 0x7fdc380cac10 2024-04-25T10:17:28.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.535+0000 7fdc3da00640 1 -- 172.21.15.119:0/3402603114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdc1406b500 msgr2=0x7fdc1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.535+0000 7fdc3da00640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdc1406b500 0x7fdc1406d9c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdc28007a90 tx=0x7fdc28009040 comp rx=0 tx=0).stop 2024-04-25T10:17:28.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.535+0000 7fdc3da00640 1 -- 172.21.15.119:0/3402603114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc380cac10 msgr2=0x7fdc380cb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.535+0000 7fdc3da00640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc380cac10 0x7fdc380cb0c0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fdc30035040 tx=0x7fdc30003a70 comp rx=0 tx=0).stop 2024-04-25T10:17:28.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.535+0000 7fdc3da00640 1 -- 172.21.15.119:0/3402603114 shutdown_connections 2024-04-25T10:17:28.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.535+0000 7fdc3da00640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdc1406b500 0x7fdc1406d9c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.535+0000 7fdc3da00640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc380cac10 0x7fdc380cb0c0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.535+0000 7fdc3da00640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc380c9a60 0x7fdc380c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.535+0000 7fdc3da00640 1 --2- 172.21.15.119:0/3402603114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdc1c004db0 0x7fdc3815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.535+0000 7fdc3da00640 1 -- 172.21.15.119:0/3402603114 >> 172.21.15.119:0/3402603114 conn(0x7fdc380d0550 msgr2=0x7fdc380d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:28.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.535+0000 7fdc3da00640 1 -- 172.21.15.119:0/3402603114 shutdown_connections 2024-04-25T10:17:28.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.535+0000 7fdc3da00640 1 -- 172.21.15.119:0/3402603114 wait complete. 2024-04-25T10:17:28.558 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T10:17:28.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.751+0000 7fa116a00640 1 Processor -- start 2024-04-25T10:17:28.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.752+0000 7fa116a00640 1 -- start start 2024-04-25T10:17:28.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.753+0000 7fa116a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11015a870 0x7fa1100cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.753+0000 7fa116a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa11006e2e0 0x7fa11015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.753+0000 7fa116a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1100cdb10 con 0x7fa1100cd580 2024-04-25T10:17:28.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.753+0000 7fa116a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1100ce720 con 0x7fa11006e2e0 2024-04-25T10:17:28.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.753+0000 7fa116a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1100cddf0 con 0x7fa11015a870 2024-04-25T10:17:28.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.753+0000 7fa114a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa1100cd580 0x7fa1100cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37998/0 (socket says 172.21.15.119:37998) 2024-04-25T10:17:28.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.753+0000 7fa114a00640 1 -- 172.21.15.119:0/1569394771 learned_addr learned my addr 172.21.15.119:0/1569394771 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:28.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10f400640 1 --2- 172.21.15.119:0/1569394771 >> v2:172.21.15.132:3300/0 conn(0x7fa11006e2e0 0x7fa11015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10fe00640 1 --2- 172.21.15.119:0/1569394771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11015a870 0x7fa1100cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 -- 172.21.15.119:0/1569394771 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28703733 0 0) 0x7fa1100cdb10 con 0x7fa1100cd580 2024-04-25T10:17:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 -- 172.21.15.119:0/1569394771 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0f4003520 con 0x7fa1100cd580 2024-04-25T10:17:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 -- 172.21.15.119:0/1569394771 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2150187723 0 0) 0x7fa0f4003520 con 0x7fa1100cd580 2024-04-25T10:17:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 -- 172.21.15.119:0/1569394771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11015a870 msgr2=0x7fa1100cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 --2- 172.21.15.119:0/1569394771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11015a870 0x7fa1100cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 -- 172.21.15.119:0/1569394771 >> v2:172.21.15.132:3300/0 conn(0x7fa11006e2e0 msgr2=0x7fa11015ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 --2- 172.21.15.119:0/1569394771 >> v2:172.21.15.132:3300/0 conn(0x7fa11006e2e0 0x7fa11015ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 -- 172.21.15.119:0/1569394771 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1100ce0d0 con 0x7fa1100cd580 2024-04-25T10:17:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 -- 172.21.15.119:0/1569394771 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fa108001670 con 0x7fa1100cd580 2024-04-25T10:17:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 -- 172.21.15.119:0/1569394771 >> v1:172.21.15.119:6789/0 conn(0x7fa1100cd580 legacy=0x7fa1100cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 --2- 172.21.15.119:0/1569394771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11015a870 0x7fa0f40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 --2- 172.21.15.119:0/1569394771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f4003f80 0x7fa0f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 --2- 172.21.15.119:0/1569394771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f4004d40 0x7fa0f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 -- 172.21.15.119:0/1569394771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1100ce720 con 0x7fa0f4003f80 2024-04-25T10:17:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 -- 172.21.15.119:0/1569394771 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1100cddf0 con 0x7fa0f4004d40 2024-04-25T10:17:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.754+0000 7fa10ea00640 1 -- 172.21.15.119:0/1569394771 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f400dbd0 con 0x7fa11015a870 2024-04-25T10:17:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.755+0000 7fa10f400640 1 --2- 172.21.15.119:0/1569394771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f4003f80 0x7fa0f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.755+0000 7fa10fe00640 1 --2- 172.21.15.119:0/1569394771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11015a870 0x7fa0f40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.755+0000 7fa10f400640 1 -- 172.21.15.119:0/1569394771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11015a870 msgr2=0x7fa0f40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.755+0000 7fa10f400640 1 --2- 172.21.15.119:0/1569394771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa11015a870 0x7fa0f40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.755+0000 7fa10f400640 1 -- 172.21.15.119:0/1569394771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f4004d40 msgr2=0x7fa0f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.755+0000 7fa10f400640 1 --2- 172.21.15.119:0/1569394771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f4004d40 0x7fa0f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.755+0000 7fa10f400640 1 -- 172.21.15.119:0/1569394771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f4003520 con 0x7fa0f4003f80 2024-04-25T10:17:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.756+0000 7fa10f400640 1 --2- 172.21.15.119:0/1569394771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f4003f80 0x7fa0f4004800 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009af0 tx=0x7fa0f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1633238015f31b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.756+0000 7fa10ea00640 1 -- 172.21.15.119:0/1569394771 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8036070 con 0x7fa0f4003f80 2024-04-25T10:17:28.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.756+0000 7fa10ea00640 1 -- 172.21.15.119:0/1569394771 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f8031400 con 0x7fa0f4003f80 2024-04-25T10:17:28.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.756+0000 7fa10ea00640 1 -- 172.21.15.119:0/1569394771 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8009690 con 0x7fa0f4003f80 2024-04-25T10:17:28.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.757+0000 7fa116a00640 1 -- 172.21.15.119:0/1569394771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f4003f80 msgr2=0x7fa0f4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.757+0000 7fa116a00640 1 --2- 172.21.15.119:0/1569394771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f4003f80 0x7fa0f4004800 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009af0 tx=0x7fa0f4005830 comp rx=0 tx=0).stop 2024-04-25T10:17:28.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.757+0000 7fa114a00640 1 -- 172.21.15.119:0/1569394771 reap_dead start 2024-04-25T10:17:28.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.757+0000 7fa116a00640 1 -- 172.21.15.119:0/1569394771 shutdown_connections 2024-04-25T10:17:28.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.757+0000 7fa116a00640 1 -- 172.21.15.119:0/1569394771 >> 172.21.15.119:0/1569394771 conn(0x7fa11006aba0 msgr2=0x7fa11006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:28.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.757+0000 7fa116a00640 1 -- 172.21.15.119:0/1569394771 shutdown_connections 2024-04-25T10:17:28.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.757+0000 7fa116a00640 1 -- 172.21.15.119:0/1569394771 wait complete. 2024-04-25T10:17:28.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.759+0000 7fa116a00640 1 Processor -- start 2024-04-25T10:17:28.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.759+0000 7fa116a00640 1 -- start start 2024-04-25T10:17:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.760+0000 7fa116a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f4003f80 0x7fa1100a88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.760+0000 7fa116a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa11014aa00 0x7fa1100ab120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.760+0000 7fa116a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1100ab660 0x7fa11015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.760+0000 7fa116a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1100cfdc0 con 0x7fa11014aa00 2024-04-25T10:17:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.760+0000 7fa116a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1100cfc40 con 0x7fa0f4003f80 2024-04-25T10:17:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.760+0000 7fa116a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1100cebb0 con 0x7fa1100ab660 2024-04-25T10:17:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.760+0000 7fa10f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa11014aa00 0x7fa1100ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.760+0000 7fa10f400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa11014aa00 0x7fa1100ab120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43182/0 (socket says 172.21.15.119:43182) 2024-04-25T10:17:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.760+0000 7fa10f400640 1 -- 172.21.15.119:0/30453213 learned_addr learned my addr 172.21.15.119:0/30453213 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.761+0000 7fa10fe00640 1 --2- 172.21.15.119:0/30453213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f4003f80 0x7fa1100a88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.761+0000 7fa114a00640 1 --2- 172.21.15.119:0/30453213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1100ab660 0x7fa11015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.761+0000 7fa10f400640 1 -- 172.21.15.119:0/30453213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1100ab660 msgr2=0x7fa11015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.761+0000 7fa10f400640 1 --2- 172.21.15.119:0/30453213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1100ab660 0x7fa11015c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.761+0000 7fa10f400640 1 -- 172.21.15.119:0/30453213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f4003f80 msgr2=0x7fa1100a88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.761+0000 7fa10f400640 1 --2- 172.21.15.119:0/30453213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f4003f80 0x7fa1100a88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.761+0000 7fa10f400640 1 -- 172.21.15.119:0/30453213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11015d570 con 0x7fa11014aa00 2024-04-25T10:17:28.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.761+0000 7fa10f400640 1 --2- 172.21.15.119:0/30453213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa11014aa00 0x7fa1100ab120 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fa0f802ac10 tx=0x7fa0f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.761+0000 7fa107e00640 1 -- 172.21.15.119:0/30453213 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8040020 con 0x7fa11014aa00 2024-04-25T10:17:28.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.761+0000 7fa107e00640 1 -- 172.21.15.119:0/30453213 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f8003810 con 0x7fa11014aa00 2024-04-25T10:17:28.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.761+0000 7fa107e00640 1 -- 172.21.15.119:0/30453213 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8036040 con 0x7fa11014aa00 2024-04-25T10:17:28.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.761+0000 7fa116a00640 1 -- 172.21.15.119:0/30453213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1100ad420 con 0x7fa11014aa00 2024-04-25T10:17:28.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.762+0000 7fa116a00640 1 -- 172.21.15.119:0/30453213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1100ad9e0 con 0x7fa11014aa00 2024-04-25T10:17:28.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.763+0000 7fa116a00640 1 -- 172.21.15.119:0/30453213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1100749b0 con 0x7fa11014aa00 2024-04-25T10:17:28.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.763+0000 7fa107e00640 1 -- 172.21.15.119:0/30453213 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0f8043020 con 0x7fa11014aa00 2024-04-25T10:17:28.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.764+0000 7fa107e00640 1 --2- 172.21.15.119:0/30453213 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa0ec06b500 0x7fa0ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.764+0000 7fa107e00640 1 -- 172.21.15.119:0/30453213 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fa0f80a9f40 con 0x7fa11014aa00 2024-04-25T10:17:28.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.765+0000 7fa10fe00640 1 --2- 172.21.15.119:0/30453213 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa0ec06b500 0x7fa0ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.765+0000 7fa10fe00640 1 --2- 172.21.15.119:0/30453213 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa0ec06b500 0x7fa0ec06d9c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa100007440 tx=0x7fa10000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:28.769+0000 7fa107e00640 1 -- 172.21.15.119:0/30453213 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0f804a070 con 0x7fa11014aa00 2024-04-25T10:17:29.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.034+0000 7fa116a00640 1 -- 172.21.15.119:0/30453213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7fa1100aba40 con 0x7fa11014aa00 2024-04-25T10:17:29.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.086+0000 7fa107e00640 1 -- 172.21.15.119:0/30453213 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v484) v1 ==== 146+0+0 (secure 0 0 0) 0x7fa0f8078fc0 con 0x7fa11014aa00 2024-04-25T10:17:29.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.089+0000 7fa116a00640 1 -- 172.21.15.119:0/30453213 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa0ec06b500 msgr2=0x7fa0ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.089+0000 7fa116a00640 1 --2- 172.21.15.119:0/30453213 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa0ec06b500 0x7fa0ec06d9c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa100007440 tx=0x7fa10000b040 comp rx=0 tx=0).stop 2024-04-25T10:17:29.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.089+0000 7fa116a00640 1 -- 172.21.15.119:0/30453213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa11014aa00 msgr2=0x7fa1100ab120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.089+0000 7fa116a00640 1 --2- 172.21.15.119:0/30453213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa11014aa00 0x7fa1100ab120 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fa0f802ac10 tx=0x7fa0f8004730 comp rx=0 tx=0).stop 2024-04-25T10:17:29.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.089+0000 7fa116a00640 1 -- 172.21.15.119:0/30453213 shutdown_connections 2024-04-25T10:17:29.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.089+0000 7fa116a00640 1 --2- 172.21.15.119:0/30453213 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa0ec06b500 0x7fa0ec06d9c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.089+0000 7fa116a00640 1 --2- 172.21.15.119:0/30453213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa1100ab660 0x7fa11015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.089+0000 7fa116a00640 1 --2- 172.21.15.119:0/30453213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa11014aa00 0x7fa1100ab120 secure :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fa0f802ac10 tx=0x7fa0f8004730 comp rx=0 tx=0).stop 2024-04-25T10:17:29.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.089+0000 7fa116a00640 1 --2- 172.21.15.119:0/30453213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0f4003f80 0x7fa1100a88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.089+0000 7fa116a00640 1 -- 172.21.15.119:0/30453213 >> 172.21.15.119:0/30453213 conn(0x7fa11006aba0 msgr2=0x7fa110159a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.089+0000 7fa116a00640 1 -- 172.21.15.119:0/30453213 shutdown_connections 2024-04-25T10:17:29.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.089+0000 7fa116a00640 1 -- 172.21.15.119:0/30453213 wait complete. 2024-04-25T10:17:29.112 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:17:29.112 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:17:29.112 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:29.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.305+0000 7fe259600640 1 Processor -- start 2024-04-25T10:17:29.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.305+0000 7fe259600640 1 -- start start 2024-04-25T10:17:29.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.307+0000 7fe259600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe254144470 0x7fe25415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.307+0000 7fe259600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2540cd440 con 0x7fe25406e430 2024-04-25T10:17:29.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.307+0000 7fe259600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2540cd720 con 0x7fe2540b5050 2024-04-25T10:17:29.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.307+0000 7fe259600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe254145340 con 0x7fe254144470 2024-04-25T10:17:29.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.307+0000 7fe253e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe254144470 0x7fe25415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.307+0000 7fe24b400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe25406e430 0x7fe2540b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38006/0 (socket says 172.21.15.119:38006) 2024-04-25T10:17:29.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.308+0000 7fe24b400640 1 -- 172.21.15.119:0/3325848168 learned_addr learned my addr 172.21.15.119:0/3325848168 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:29.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.308+0000 7fe253e00640 1 -- 172.21.15.119:0/3325848168 >> v1:172.21.15.132:6789/0 conn(0x7fe2540b5050 legacy=0x7fe25415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.308+0000 7fe253e00640 1 -- 172.21.15.119:0/3325848168 >> v1:172.21.15.119:6789/0 conn(0x7fe25406e430 legacy=0x7fe2540b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.308+0000 7fe253e00640 1 -- 172.21.15.119:0/3325848168 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2540cda00 con 0x7fe254144470 2024-04-25T10:17:29.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.309+0000 7fe253e00640 1 --2- 172.21.15.119:0/3325848168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe254144470 0x7fe25415fc90 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe244009d90 tx=0x7fe24402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef3f350680840d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.309+0000 7fe253400640 1 -- 172.21.15.119:0/3325848168 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe244007740 con 0x7fe254144470 2024-04-25T10:17:29.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.309+0000 7fe253400640 1 -- 172.21.15.119:0/3325848168 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2440078e0 con 0x7fe254144470 2024-04-25T10:17:29.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.309+0000 7fe253400640 1 -- 172.21.15.119:0/3325848168 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe244007cb0 con 0x7fe254144470 2024-04-25T10:17:29.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.310+0000 7fe259600640 1 -- 172.21.15.119:0/3325848168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe254144470 msgr2=0x7fe25415fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.310+0000 7fe259600640 1 --2- 172.21.15.119:0/3325848168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe254144470 0x7fe25415fc90 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe244009d90 tx=0x7fe24402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.310+0000 7fe259600640 1 -- 172.21.15.119:0/3325848168 shutdown_connections 2024-04-25T10:17:29.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.310+0000 7fe259600640 1 --2- 172.21.15.119:0/3325848168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe254144470 0x7fe25415fc90 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.310+0000 7fe259600640 1 -- 172.21.15.119:0/3325848168 >> 172.21.15.119:0/3325848168 conn(0x7fe25406b1d0 msgr2=0x7fe2540b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.310+0000 7fe259600640 1 -- 172.21.15.119:0/3325848168 shutdown_connections 2024-04-25T10:17:29.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.310+0000 7fe259600640 1 -- 172.21.15.119:0/3325848168 wait complete. 2024-04-25T10:17:29.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.312+0000 7fe259600640 1 Processor -- start 2024-04-25T10:17:29.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.312+0000 7fe259600640 1 -- start start 2024-04-25T10:17:29.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.313+0000 7fe259600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe25406e430 0x7fe25415b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.313+0000 7fe259600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2540b5050 0x7fe2541e6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.313+0000 7fe259600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe254144470 0x7fe2541e6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.313+0000 7fe259600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2540ce200 con 0x7fe2540b5050 2024-04-25T10:17:29.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.313+0000 7fe259600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe254144850 con 0x7fe25406e430 2024-04-25T10:17:29.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.313+0000 7fe259600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25415c400 con 0x7fe254144470 2024-04-25T10:17:29.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.313+0000 7fe24b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2540b5050 0x7fe2541e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.313+0000 7fe24b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2540b5050 0x7fe2541e6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43194/0 (socket says 172.21.15.119:43194) 2024-04-25T10:17:29.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.313+0000 7fe24b400640 1 -- 172.21.15.119:0/2555530656 learned_addr learned my addr 172.21.15.119:0/2555530656 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:29.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.314+0000 7fe253e00640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe25406e430 0x7fe25415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.314+0000 7fe24b400640 1 -- 172.21.15.119:0/2555530656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe254144470 msgr2=0x7fe2541e6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.314+0000 7fe258c00640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe254144470 0x7fe2541e6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.314+0000 7fe24b400640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe254144470 0x7fe2541e6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.314+0000 7fe24b400640 1 -- 172.21.15.119:0/2555530656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe25406e430 msgr2=0x7fe25415b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.314+0000 7fe24b400640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe25406e430 0x7fe25415b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.314+0000 7fe24b400640 1 -- 172.21.15.119:0/2555530656 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe254141af0 con 0x7fe2540b5050 2024-04-25T10:17:29.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.314+0000 7fe253e00640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe25406e430 0x7fe25415b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:29.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.314+0000 7fe24b400640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2540b5050 0x7fe2541e6840 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00c440 tx=0x7fe23c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.314+0000 7fe250c00640 1 -- 172.21.15.119:0/2555530656 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c017070 con 0x7fe2540b5050 2024-04-25T10:17:29.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.314+0000 7fe250c00640 1 -- 172.21.15.119:0/2555530656 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c005ce0 con 0x7fe2540b5050 2024-04-25T10:17:29.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.314+0000 7fe250c00640 1 -- 172.21.15.119:0/2555530656 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0072f0 con 0x7fe2540b5050 2024-04-25T10:17:29.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.314+0000 7fe259600640 1 -- 172.21.15.119:0/2555530656 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe254141dd0 con 0x7fe2540b5050 2024-04-25T10:17:29.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.314+0000 7fe259600640 1 -- 172.21.15.119:0/2555530656 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe254142390 con 0x7fe2540b5050 2024-04-25T10:17:29.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.316+0000 7fe259600640 1 -- 172.21.15.119:0/2555530656 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2540749b0 con 0x7fe2540b5050 2024-04-25T10:17:29.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.317+0000 7fe250c00640 1 -- 172.21.15.119:0/2555530656 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe23c01a070 con 0x7fe2540b5050 2024-04-25T10:17:29.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.318+0000 7fe250c00640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe23006b4d0 0x7fe23006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.318+0000 7fe250c00640 1 -- 172.21.15.119:0/2555530656 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe23c089360 con 0x7fe2540b5050 2024-04-25T10:17:29.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.318+0000 7fe253e00640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe23006b4d0 0x7fe23006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.319+0000 7fe253e00640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe23006b4d0 0x7fe23006d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe244034000 tx=0x7fe24402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.322+0000 7fe250c00640 1 -- 172.21.15.119:0/2555530656 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe23c058470 con 0x7fe2540b5050 2024-04-25T10:17:29.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.509+0000 7fe259600640 1 -- 172.21.15.119:0/2555530656 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2540b2090 con 0x7fe2540b5050 2024-04-25T10:17:29.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.512+0000 7fe250c00640 1 -- 172.21.15.119:0/2555530656 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+13190 (secure 0 0 0) 0x7fe23c057d70 con 0x7fe2540b5050 2024-04-25T10:17:29.515 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:29.516 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":100,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:17:28.066760+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/464677621":"2024-04-26T10:17:18.997269+0000","172.21.15.119:6835/464677621":"2024-04-26T10:17:18.997269+0000","172.21.15.119:6837/1945631445":"2024-04-26T10:16:41.666750+0000","172.21.15.119:6835/1945631445":"2024-04-26T10:16:41.666750+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:29.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.514+0000 7fe259600640 1 -- 172.21.15.119:0/2555530656 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe23006b4d0 msgr2=0x7fe23006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.514+0000 7fe259600640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe23006b4d0 0x7fe23006d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe244034000 tx=0x7fe24402f000 comp rx=0 tx=0).stop 2024-04-25T10:17:29.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.514+0000 7fe259600640 1 -- 172.21.15.119:0/2555530656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2540b5050 msgr2=0x7fe2541e6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.514+0000 7fe259600640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2540b5050 0x7fe2541e6840 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00c440 tx=0x7fe23c001e70 comp rx=0 tx=0).stop 2024-04-25T10:17:29.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.514+0000 7fe259600640 1 -- 172.21.15.119:0/2555530656 shutdown_connections 2024-04-25T10:17:29.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.514+0000 7fe259600640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe23006b4d0 0x7fe23006d990 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.514+0000 7fe259600640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe254144470 0x7fe2541e6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.514+0000 7fe259600640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2540b5050 0x7fe2541e6840 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.514+0000 7fe259600640 1 --2- 172.21.15.119:0/2555530656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe25406e430 0x7fe25415b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.514+0000 7fe259600640 1 -- 172.21.15.119:0/2555530656 >> 172.21.15.119:0/2555530656 conn(0x7fe25406b1d0 msgr2=0x7fe2540b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.515+0000 7fe259600640 1 -- 172.21.15.119:0/2555530656 shutdown_connections 2024-04-25T10:17:29.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.515+0000 7fe259600640 1 -- 172.21.15.119:0/2555530656 wait complete. 2024-04-25T10:17:29.535 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:29.535 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:29.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.730+0000 7fbb6b200640 1 Processor -- start 2024-04-25T10:17:29.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.730+0000 7fbb6b200640 1 -- start start 2024-04-25T10:17:29.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.731+0000 7fbb6b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb6c1450e0 0x7fbb6c06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.731+0000 7fbb6b200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbb6c0cc9e0 0x7fbb6c0b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.731+0000 7fbb6b200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6c15f400 con 0x7fbb6c06b0e0 2024-04-25T10:17:29.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.731+0000 7fbb6b200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6c1454c0 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.731+0000 7fbb6b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c06b540 con 0x7fbb6c1450e0 2024-04-25T10:17:29.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.732+0000 7fbb6a800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbb6c0cc9e0 0x7fbb6c0b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.732+0000 7fbb69400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbb6c06b0e0 0x7fbb6c0cc2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38012/0 (socket says 172.21.15.119:38012) 2024-04-25T10:17:29.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.732+0000 7fbb69400640 1 -- 172.21.15.119:0/4256722924 learned_addr learned my addr 172.21.15.119:0/4256722924 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:29.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.732+0000 7fbb69e00640 1 --2- 172.21.15.119:0/4256722924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb6c1450e0 0x7fbb6c06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.732+0000 7fbb6a800640 1 -- 172.21.15.119:0/4256722924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb6c1450e0 msgr2=0x7fbb6c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.732+0000 7fbb6a800640 1 --2- 172.21.15.119:0/4256722924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb6c1450e0 0x7fbb6c06aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.732+0000 7fbb6a800640 1 -- 172.21.15.119:0/4256722924 >> v1:172.21.15.119:6789/0 conn(0x7fbb6c06b0e0 legacy=0x7fbb6c0cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.732+0000 7fbb6a800640 1 -- 172.21.15.119:0/4256722924 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c15f6e0 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.732+0000 7fbb68a00640 1 -- 172.21.15.119:0/4256722924 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235336507 0 0) 0x7fbb6c15f400 con 0x7fbb6c06b0e0 2024-04-25T10:17:29.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.733+0000 7fbb6a800640 1 --2- 172.21.15.119:0/4256722924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb6c0cc9e0 0x7fbb6c0b2e80 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fbb640097a0 tx=0x7fbb6402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69a6c95ce37f473 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.733+0000 7fbb68a00640 1 -- 172.21.15.119:0/4256722924 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64037070 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.733+0000 7fbb68a00640 1 -- 172.21.15.119:0/4256722924 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6402adb0 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.733+0000 7fbb68a00640 1 -- 172.21.15.119:0/4256722924 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64032660 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.734+0000 7fbb6b200640 1 -- 172.21.15.119:0/4256722924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb6c0cc9e0 msgr2=0x7fbb6c0b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.734+0000 7fbb6b200640 1 --2- 172.21.15.119:0/4256722924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb6c0cc9e0 0x7fbb6c0b2e80 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fbb640097a0 tx=0x7fbb6402a060 comp rx=0 tx=0).stop 2024-04-25T10:17:29.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.734+0000 7fbb6b200640 1 -- 172.21.15.119:0/4256722924 shutdown_connections 2024-04-25T10:17:29.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.734+0000 7fbb6b200640 1 --2- 172.21.15.119:0/4256722924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb6c0cc9e0 0x7fbb6c0b2e80 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.734+0000 7fbb6b200640 1 --2- 172.21.15.119:0/4256722924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb6c1450e0 0x7fbb6c06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.734+0000 7fbb6b200640 1 -- 172.21.15.119:0/4256722924 >> 172.21.15.119:0/4256722924 conn(0x7fbb6c066ff0 msgr2=0x7fbb6c0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.734+0000 7fbb6b200640 1 -- 172.21.15.119:0/4256722924 shutdown_connections 2024-04-25T10:17:29.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.734+0000 7fbb6b200640 1 -- 172.21.15.119:0/4256722924 wait complete. 2024-04-25T10:17:29.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.736+0000 7fbb6b200640 1 Processor -- start 2024-04-25T10:17:29.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.736+0000 7fbb6b200640 1 -- start start 2024-04-25T10:17:29.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.737+0000 7fbb6b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb6c06b0e0 0x7fbb6c134b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.737+0000 7fbb6b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb6c0cc9e0 0x7fbb6c1f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.737+0000 7fbb6b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb6c1450e0 0x7fbb6c0ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.737+0000 7fbb6b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0b33c0 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.737+0000 7fbb6b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c15fca0 con 0x7fbb6c1450e0 2024-04-25T10:17:29.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.737+0000 7fbb6b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0b3540 con 0x7fbb6c06b0e0 2024-04-25T10:17:29.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.737+0000 7fbb69400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb6c0cc9e0 0x7fbb6c1f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.737+0000 7fbb69400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb6c0cc9e0 0x7fbb6c1f29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43210/0 (socket says 172.21.15.119:43210) 2024-04-25T10:17:29.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.737+0000 7fbb69400640 1 -- 172.21.15.119:0/943091852 learned_addr learned my addr 172.21.15.119:0/943091852 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:29.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.738+0000 7fbb69400640 1 -- 172.21.15.119:0/943091852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb6c06b0e0 msgr2=0x7fbb6c134b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.738+0000 7fbb69400640 1 --2- 172.21.15.119:0/943091852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb6c06b0e0 0x7fbb6c134b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.738+0000 7fbb69400640 1 -- 172.21.15.119:0/943091852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb6c1450e0 msgr2=0x7fbb6c0ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.738+0000 7fbb69400640 1 --2- 172.21.15.119:0/943091852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb6c1450e0 0x7fbb6c0ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.738+0000 7fbb69400640 1 -- 172.21.15.119:0/943091852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c06dfe0 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.738+0000 7fbb69400640 1 --2- 172.21.15.119:0/943091852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb6c0cc9e0 0x7fbb6c1f29b0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800c4d0 tx=0x7fbb5800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.738+0000 7fbb62000640 1 -- 172.21.15.119:0/943091852 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58018070 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.738+0000 7fbb62000640 1 -- 172.21.15.119:0/943091852 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb58007500 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.738+0000 7fbb62000640 1 -- 172.21.15.119:0/943091852 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58004a20 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.738+0000 7fbb6b200640 1 -- 172.21.15.119:0/943091852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c15bde0 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.738+0000 7fbb6b200640 1 -- 172.21.15.119:0/943091852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6c15c3a0 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.739+0000 7fbb6b200640 1 -- 172.21.15.119:0/943091852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6c0749b0 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.740+0000 7fbb62000640 1 -- 172.21.15.119:0/943091852 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb58004bc0 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.741+0000 7fbb62000640 1 --2- 172.21.15.119:0/943091852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbb4c06b4d0 0x7fbb4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.741+0000 7fbb62000640 1 -- 172.21.15.119:0/943091852 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fbb580892b0 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.742+0000 7fbb69e00640 1 --2- 172.21.15.119:0/943091852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbb4c06b4d0 0x7fbb4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.742+0000 7fbb69e00640 1 --2- 172.21.15.119:0/943091852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbb4c06b4d0 0x7fbb4c06d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c007dc0 tx=0x7fbb5c0072f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.745+0000 7fbb62000640 1 -- 172.21.15.119:0/943091852 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb580583c0 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.936+0000 7fbb6b200640 1 -- 172.21.15.119:0/943091852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb6c06c050 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.937+0000 7fbb62000640 1 -- 172.21.15.119:0/943091852 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb5805e0c0 con 0x7fbb6c0cc9e0 2024-04-25T10:17:29.941 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:17:29.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.940+0000 7fbb6b200640 1 -- 172.21.15.119:0/943091852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbb4c06b4d0 msgr2=0x7fbb4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.940+0000 7fbb6b200640 1 --2- 172.21.15.119:0/943091852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbb4c06b4d0 0x7fbb4c06d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c007dc0 tx=0x7fbb5c0072f0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.940+0000 7fbb6b200640 1 -- 172.21.15.119:0/943091852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb6c0cc9e0 msgr2=0x7fbb6c1f29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.940+0000 7fbb6b200640 1 --2- 172.21.15.119:0/943091852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb6c0cc9e0 0x7fbb6c1f29b0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800c4d0 tx=0x7fbb5800c990 comp rx=0 tx=0).stop 2024-04-25T10:17:29.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.940+0000 7fbb6b200640 1 -- 172.21.15.119:0/943091852 shutdown_connections 2024-04-25T10:17:29.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.940+0000 7fbb6b200640 1 --2- 172.21.15.119:0/943091852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbb4c06b4d0 0x7fbb4c06d990 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.940+0000 7fbb6b200640 1 --2- 172.21.15.119:0/943091852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb6c1450e0 0x7fbb6c0ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.940+0000 7fbb6b200640 1 --2- 172.21.15.119:0/943091852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb6c0cc9e0 0x7fbb6c1f29b0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.940+0000 7fbb6b200640 1 --2- 172.21.15.119:0/943091852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbb6c06b0e0 0x7fbb6c134b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.940+0000 7fbb6b200640 1 -- 172.21.15.119:0/943091852 >> 172.21.15.119:0/943091852 conn(0x7fbb6c066ff0 msgr2=0x7fbb6c0ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.940+0000 7fbb6b200640 1 -- 172.21.15.119:0/943091852 shutdown_connections 2024-04-25T10:17:29.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:29.940+0000 7fbb6b200640 1 -- 172.21.15.119:0/943091852 wait complete. 2024-04-25T10:17:29.961 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T10:17:30.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.158+0000 7fd272000640 1 Processor -- start 2024-04-25T10:17:30.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.159+0000 7fd272000640 1 -- start start 2024-04-25T10:17:30.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.160+0000 7fd272000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd26c15e690 0x7fd26c15ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.160+0000 7fd272000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26c0b1ed0 con 0x7fd26c0b1a30 2024-04-25T10:17:30.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.160+0000 7fd272000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26c15efb0 con 0x7fd26c0b1610 2024-04-25T10:17:30.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.160+0000 7fd272000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26c06ba70 con 0x7fd26c15e690 2024-04-25T10:17:30.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.161+0000 7fd271600640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd26c0b1a30 0x7fd26c15d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38020/0 (socket says 172.21.15.119:38020) 2024-04-25T10:17:30.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.161+0000 7fd271600640 1 -- 172.21.15.119:0/4220351182 learned_addr learned my addr 172.21.15.119:0/4220351182 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:30.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.161+0000 7fd270c00640 1 --2- 172.21.15.119:0/4220351182 >> v2:172.21.15.132:3300/0 conn(0x7fd26c15e690 0x7fd26c15ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.161+0000 7fd26be00640 1 -- 172.21.15.119:0/4220351182 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243033140 0 0) 0x7fd26c0b1ed0 con 0x7fd26c0b1a30 2024-04-25T10:17:30.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.161+0000 7fd26be00640 1 -- 172.21.15.119:0/4220351182 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd250003520 con 0x7fd26c0b1a30 2024-04-25T10:17:30.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.161+0000 7fd26be00640 1 -- 172.21.15.119:0/4220351182 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1932220157 0 0) 0x7fd250003520 con 0x7fd26c0b1a30 2024-04-25T10:17:30.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.161+0000 7fd26be00640 1 -- 172.21.15.119:0/4220351182 >> v2:172.21.15.132:3300/0 conn(0x7fd26c15e690 msgr2=0x7fd26c15ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:30.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.161+0000 7fd26be00640 1 --2- 172.21.15.119:0/4220351182 >> v2:172.21.15.132:3300/0 conn(0x7fd26c15e690 0x7fd26c15ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.162+0000 7fd26be00640 1 -- 172.21.15.119:0/4220351182 >> v1:172.21.15.132:6789/0 conn(0x7fd26c0b1610 legacy=0x7fd26c0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.162+0000 7fd26be00640 1 -- 172.21.15.119:0/4220351182 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c06acb0 con 0x7fd26c0b1a30 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.162+0000 7fd26be00640 1 -- 172.21.15.119:0/4220351182 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fd264001670 con 0x7fd26c0b1a30 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.162+0000 7fd26be00640 1 -- 172.21.15.119:0/4220351182 >> v1:172.21.15.119:6789/0 conn(0x7fd26c0b1a30 legacy=0x7fd26c15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.162+0000 7fd26be00640 1 --2- 172.21.15.119:0/4220351182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd250003a70 0x7fd250003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.162+0000 7fd26be00640 1 --2- 172.21.15.119:0/4220351182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd250004800 0x7fd250004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.162+0000 7fd26be00640 1 --2- 172.21.15.119:0/4220351182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd250005210 0x7fd25000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.162+0000 7fd26be00640 1 -- 172.21.15.119:0/4220351182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c06ba70 con 0x7fd250005210 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.162+0000 7fd26be00640 1 -- 172.21.15.119:0/4220351182 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25000c120 con 0x7fd250004800 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.162+0000 7fd26be00640 1 -- 172.21.15.119:0/4220351182 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25000c320 con 0x7fd250003a70 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.162+0000 7fd271600640 1 --2- 172.21.15.119:0/4220351182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd250005210 0x7fd25000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.162+0000 7fd263e00640 1 --2- 172.21.15.119:0/4220351182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd250004800 0x7fd250004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.162+0000 7fd270c00640 1 --2- 172.21.15.119:0/4220351182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd250003a70 0x7fd250003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.163+0000 7fd271600640 1 -- 172.21.15.119:0/4220351182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd250003a70 msgr2=0x7fd250003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.163+0000 7fd271600640 1 --2- 172.21.15.119:0/4220351182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd250003a70 0x7fd250003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.163+0000 7fd271600640 1 -- 172.21.15.119:0/4220351182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd250004800 msgr2=0x7fd250004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.163+0000 7fd271600640 1 --2- 172.21.15.119:0/4220351182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd250004800 0x7fd250004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.163+0000 7fd271600640 1 -- 172.21.15.119:0/4220351182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd250003520 con 0x7fd250005210 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.164+0000 7fd271600640 1 -- 172.21.15.119:0/4220351182 reap_dead start 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.164+0000 7fd271600640 1 --2- 172.21.15.119:0/4220351182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd250005210 0x7fd25000b9c0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd26400c190 tx=0x7fd250005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3369c149fd66b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.164+0000 7fd26be00640 1 -- 172.21.15.119:0/4220351182 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264007c60 con 0x7fd250005210 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.164+0000 7fd26be00640 1 -- 172.21.15.119:0/4220351182 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd264005af0 con 0x7fd250005210 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.164+0000 7fd26be00640 1 -- 172.21.15.119:0/4220351182 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264007070 con 0x7fd250005210 2024-04-25T10:17:30.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.165+0000 7fd272000640 1 -- 172.21.15.119:0/4220351182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd250005210 msgr2=0x7fd25000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.165+0000 7fd272000640 1 --2- 172.21.15.119:0/4220351182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd250005210 0x7fd25000b9c0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd26400c190 tx=0x7fd250005830 comp rx=0 tx=0).stop 2024-04-25T10:17:30.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.165+0000 7fd272000640 1 -- 172.21.15.119:0/4220351182 shutdown_connections 2024-04-25T10:17:30.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.165+0000 7fd272000640 1 --2- 172.21.15.119:0/4220351182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd250005210 0x7fd25000b9c0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.165+0000 7fd272000640 1 -- 172.21.15.119:0/4220351182 >> 172.21.15.119:0/4220351182 conn(0x7fd26c099a40 msgr2=0x7fd26c0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:30.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.165+0000 7fd272000640 1 -- 172.21.15.119:0/4220351182 shutdown_connections 2024-04-25T10:17:30.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.165+0000 7fd272000640 1 -- 172.21.15.119:0/4220351182 wait complete. 2024-04-25T10:17:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.167+0000 7fd272000640 1 Processor -- start 2024-04-25T10:17:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.167+0000 7fd272000640 1 -- start start 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.168+0000 7fd272000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd250005210 0x7fd26c156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.168+0000 7fd272000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd26c1e77d0 0x7fd26c1e7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.168+0000 7fd272000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd26c1580a0 0x7fd26c158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.168+0000 7fd272000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c06b550 con 0x7fd26c1580a0 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.168+0000 7fd272000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c06b270 con 0x7fd26c1e77d0 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.168+0000 7fd272000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c15ea70 con 0x7fd250005210 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.168+0000 7fd271600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd26c1580a0 0x7fd26c158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.168+0000 7fd271600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd26c1580a0 0x7fd26c158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43228/0 (socket says 172.21.15.119:43228) 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.168+0000 7fd271600640 1 -- 172.21.15.119:0/681223652 learned_addr learned my addr 172.21.15.119:0/681223652 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.169+0000 7fd270c00640 1 --2- 172.21.15.119:0/681223652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd250005210 0x7fd26c156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.169+0000 7fd263e00640 1 --2- 172.21.15.119:0/681223652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd26c1e77d0 0x7fd26c1e7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.169+0000 7fd271600640 1 -- 172.21.15.119:0/681223652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd250005210 msgr2=0x7fd26c156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.169+0000 7fd271600640 1 --2- 172.21.15.119:0/681223652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd250005210 0x7fd26c156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.169+0000 7fd271600640 1 -- 172.21.15.119:0/681223652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd26c1e77d0 msgr2=0x7fd26c1e7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.169+0000 7fd271600640 1 --2- 172.21.15.119:0/681223652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd26c1e77d0 0x7fd26c1e7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.169+0000 7fd271600640 1 -- 172.21.15.119:0/681223652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c157820 con 0x7fd26c1580a0 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.169+0000 7fd271600640 1 --2- 172.21.15.119:0/681223652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd26c1580a0 0x7fd26c158520 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fd26402ad60 tx=0x7fd264001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.169+0000 7fd269600640 1 -- 172.21.15.119:0/681223652 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264040070 con 0x7fd26c1580a0 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.169+0000 7fd269600640 1 -- 172.21.15.119:0/681223652 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2640047e0 con 0x7fd26c1580a0 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.169+0000 7fd269600640 1 -- 172.21.15.119:0/681223652 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264004bd0 con 0x7fd26c1580a0 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.170+0000 7fd272000640 1 -- 172.21.15.119:0/681223652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26c1e8380 con 0x7fd26c1580a0 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.170+0000 7fd272000640 1 -- 172.21.15.119:0/681223652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26c1e8940 con 0x7fd26c1580a0 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.171+0000 7fd272000640 1 -- 172.21.15.119:0/681223652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26c0749b0 con 0x7fd26c1580a0 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.172+0000 7fd269600640 1 -- 172.21.15.119:0/681223652 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd264043070 con 0x7fd26c1580a0 2024-04-25T10:17:30.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.173+0000 7fd269600640 1 --2- 172.21.15.119:0/681223652 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd24806b500 0x7fd24806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.173+0000 7fd269600640 1 -- 172.21.15.119:0/681223652 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd2640a91b0 con 0x7fd26c1580a0 2024-04-25T10:17:30.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.173+0000 7fd270c00640 1 --2- 172.21.15.119:0/681223652 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd24806b500 0x7fd24806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.174+0000 7fd270c00640 1 --2- 172.21.15.119:0/681223652 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd24806b500 0x7fd24806d9c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009fd0 tx=0x7fd25c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.177+0000 7fd269600640 1 -- 172.21.15.119:0/681223652 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2640781a0 con 0x7fd26c1580a0 2024-04-25T10:17:30.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:30.440+0000 7fd272000640 1 -- 172.21.15.119:0/681223652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7fd26c072430 con 0x7fd26c1580a0 2024-04-25T10:17:31.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.092+0000 7fd269600640 1 -- 172.21.15.119:0/681223652 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v486) v1 ==== 147+0+0 (secure 0 0 0) 0x7fd264077aa0 con 0x7fd26c1580a0 2024-04-25T10:17:31.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.096+0000 7fd272000640 1 -- 172.21.15.119:0/681223652 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd24806b500 msgr2=0x7fd24806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.096+0000 7fd272000640 1 --2- 172.21.15.119:0/681223652 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd24806b500 0x7fd24806d9c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009fd0 tx=0x7fd25c00b040 comp rx=0 tx=0).stop 2024-04-25T10:17:31.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.096+0000 7fd272000640 1 -- 172.21.15.119:0/681223652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd26c1580a0 msgr2=0x7fd26c158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.096+0000 7fd272000640 1 --2- 172.21.15.119:0/681223652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd26c1580a0 0x7fd26c158520 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fd26402ad60 tx=0x7fd264001970 comp rx=0 tx=0).stop 2024-04-25T10:17:31.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.096+0000 7fd272000640 1 -- 172.21.15.119:0/681223652 shutdown_connections 2024-04-25T10:17:31.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.096+0000 7fd272000640 1 --2- 172.21.15.119:0/681223652 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd24806b500 0x7fd24806d9c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.096+0000 7fd272000640 1 --2- 172.21.15.119:0/681223652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd26c1580a0 0x7fd26c158520 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.096+0000 7fd272000640 1 --2- 172.21.15.119:0/681223652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd26c1e77d0 0x7fd26c1e7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.096+0000 7fd272000640 1 --2- 172.21.15.119:0/681223652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd250005210 0x7fd26c156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.096+0000 7fd272000640 1 -- 172.21.15.119:0/681223652 >> 172.21.15.119:0/681223652 conn(0x7fd26c099a40 msgr2=0x7fd26c0d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:31.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.096+0000 7fd272000640 1 -- 172.21.15.119:0/681223652 shutdown_connections 2024-04-25T10:17:31.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.096+0000 7fd272000640 1 -- 172.21.15.119:0/681223652 wait complete. 2024-04-25T10:17:31.120 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:17:31.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.318+0000 7f34aa600640 1 Processor -- start 2024-04-25T10:17:31.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.318+0000 7f34aa600640 1 -- start start 2024-04-25T10:17:31.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.319+0000 7f34aa600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a4161d60 0x7f34a4166380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.320+0000 7f34aa600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f34a40cea30 0x7f34a40b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.320+0000 7f34aa600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a4162140 con 0x7f34a406f0c0 2024-04-25T10:17:31.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.320+0000 7f34aa600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a40b2470 con 0x7f34a40cea30 2024-04-25T10:17:31.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.320+0000 7f34aa600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a4165ef0 con 0x7f34a4161d60 2024-04-25T10:17:31.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.320+0000 7f34a9200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a4161d60 0x7f34a4166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.320+0000 7f34a3e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f34a40cea30 0x7f34a40b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.320+0000 7f34a9200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a4161d60 0x7f34a4166380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:46382/0 (socket says 172.21.15.119:46382) 2024-04-25T10:17:31.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.320+0000 7f34a9200640 1 -- 172.21.15.119:0/899722405 learned_addr learned my addr 172.21.15.119:0/899722405 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:31.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.321+0000 7f34a9200640 1 -- 172.21.15.119:0/899722405 >> v2:172.21.15.132:3300/0 conn(0x7f34a40cea30 msgr2=0x7f34a40b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:31.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.321+0000 7f34a9200640 1 --2- 172.21.15.119:0/899722405 >> v2:172.21.15.132:3300/0 conn(0x7f34a40cea30 0x7f34a40b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.321+0000 7f34a9200640 1 -- 172.21.15.119:0/899722405 >> v1:172.21.15.132:6789/0 conn(0x7f34a406f0c0 legacy=0x7f34a406dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:31.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.321+0000 7f34a9200640 1 -- 172.21.15.119:0/899722405 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a40cf210 con 0x7f34a4161d60 2024-04-25T10:17:31.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.322+0000 7f34a9200640 1 --2- 172.21.15.119:0/899722405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a4161d60 0x7f34a4166380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3498009870 tx=0x7f349802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4ea13357381aae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.323+0000 7f34a3400640 1 -- 172.21.15.119:0/899722405 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3498037070 con 0x7f34a4161d60 2024-04-25T10:17:31.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.323+0000 7f34a3400640 1 -- 172.21.15.119:0/899722405 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3498032440 con 0x7f34a4161d60 2024-04-25T10:17:31.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.323+0000 7f34a3400640 1 -- 172.21.15.119:0/899722405 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3498032810 con 0x7f34a4161d60 2024-04-25T10:17:31.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.323+0000 7f34aa600640 1 -- 172.21.15.119:0/899722405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a4161d60 msgr2=0x7f34a4166380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.323+0000 7f34aa600640 1 --2- 172.21.15.119:0/899722405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a4161d60 0x7f34a4166380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3498009870 tx=0x7f349802a180 comp rx=0 tx=0).stop 2024-04-25T10:17:31.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.324+0000 7f34aa600640 1 -- 172.21.15.119:0/899722405 shutdown_connections 2024-04-25T10:17:31.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.324+0000 7f34aa600640 1 --2- 172.21.15.119:0/899722405 >> v2:172.21.15.132:3300/0 conn(0x7f34a40cea30 0x7f34a40b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.324+0000 7f34aa600640 1 --2- 172.21.15.119:0/899722405 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a4161d60 0x7f34a4166380 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.324+0000 7f34aa600640 1 -- 172.21.15.119:0/899722405 >> 172.21.15.119:0/899722405 conn(0x7f34a4066ff0 msgr2=0x7f34a40b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:31.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.324+0000 7f34aa600640 1 -- 172.21.15.119:0/899722405 shutdown_connections 2024-04-25T10:17:31.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.324+0000 7f34aa600640 1 -- 172.21.15.119:0/899722405 wait complete. 2024-04-25T10:17:31.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.326+0000 7f34aa600640 1 Processor -- start 2024-04-25T10:17:31.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.326+0000 7f34aa600640 1 -- start start 2024-04-25T10:17:31.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.327+0000 7f34aa600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34a406f0c0 0x7f34a40ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.327+0000 7f34aa600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34a40cea30 0x7f34a415f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.327+0000 7f34aa600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a4161d60 0x7f34a4160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.327+0000 7f34aa600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a41658a0 con 0x7f34a40cea30 2024-04-25T10:17:31.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.327+0000 7f34aa600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40b29a0 con 0x7f34a406f0c0 2024-04-25T10:17:31.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.327+0000 7f34aa600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a4165b80 con 0x7f34a4161d60 2024-04-25T10:17:31.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.328+0000 7f34a3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34a40cea30 0x7f34a415f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.328+0000 7f34a3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34a40cea30 0x7f34a415f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45592/0 (socket says 172.21.15.119:45592) 2024-04-25T10:17:31.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.328+0000 7f34a3e00640 1 -- 172.21.15.119:0/2491214083 learned_addr learned my addr 172.21.15.119:0/2491214083 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:31.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.328+0000 7f34a9200640 1 --2- 172.21.15.119:0/2491214083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34a406f0c0 0x7f34a40ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.328+0000 7f34a9c00640 1 --2- 172.21.15.119:0/2491214083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a4161d60 0x7f34a4160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.328+0000 7f34a3e00640 1 -- 172.21.15.119:0/2491214083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a4161d60 msgr2=0x7f34a4160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.328+0000 7f34a3e00640 1 --2- 172.21.15.119:0/2491214083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a4161d60 0x7f34a4160a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.328+0000 7f34a3e00640 1 -- 172.21.15.119:0/2491214083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34a406f0c0 msgr2=0x7f34a40ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.328+0000 7f34a3e00640 1 --2- 172.21.15.119:0/2491214083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34a406f0c0 0x7f34a40ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.328+0000 7f34a3e00640 1 -- 172.21.15.119:0/2491214083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a40ab610 con 0x7f34a40cea30 2024-04-25T10:17:31.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.329+0000 7f34a3e00640 1 --2- 172.21.15.119:0/2491214083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34a40cea30 0x7f34a415f680 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f349000e980 tx=0x7f349000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.329+0000 7f34a0c00640 1 -- 172.21.15.119:0/2491214083 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490017070 con 0x7f34a40cea30 2024-04-25T10:17:31.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.329+0000 7f34a0c00640 1 -- 172.21.15.119:0/2491214083 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349000ca00 con 0x7f34a40cea30 2024-04-25T10:17:31.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.329+0000 7f34a0c00640 1 -- 172.21.15.119:0/2491214083 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349000cdd0 con 0x7f34a40cea30 2024-04-25T10:17:31.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.329+0000 7f34aa600640 1 -- 172.21.15.119:0/2491214083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a40a91e0 con 0x7f34a40cea30 2024-04-25T10:17:31.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.329+0000 7f34aa600640 1 -- 172.21.15.119:0/2491214083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a40a97a0 con 0x7f34a40cea30 2024-04-25T10:17:31.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.330+0000 7f34aa600640 1 -- 172.21.15.119:0/2491214083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a40a9b80 con 0x7f34a40cea30 2024-04-25T10:17:31.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.331+0000 7f34a0c00640 1 -- 172.21.15.119:0/2491214083 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f349001a070 con 0x7f34a40cea30 2024-04-25T10:17:31.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.332+0000 7f34a0c00640 1 --2- 172.21.15.119:0/2491214083 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f348406b4d0 0x7f348406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.332+0000 7f34a0c00640 1 -- 172.21.15.119:0/2491214083 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3490089f60 con 0x7f34a40cea30 2024-04-25T10:17:31.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.332+0000 7f34a9200640 1 --2- 172.21.15.119:0/2491214083 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f348406b4d0 0x7f348406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.333+0000 7f34a9200640 1 --2- 172.21.15.119:0/2491214083 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f348406b4d0 0x7f348406d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3498034000 tx=0x7f349802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.336+0000 7f34a0c00640 1 -- 172.21.15.119:0/2491214083 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34a40a9b80 con 0x7f34a40cea30 2024-04-25T10:17:31.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.514+0000 7f34aa600640 1 -- 172.21.15.119:0/2491214083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f34a40749b0 con 0x7f34a40cea30 2024-04-25T10:17:31.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.522+0000 7f34a0c00640 1 -- 172.21.15.119:0/2491214083 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3490059070 con 0x7f34a40cea30 2024-04-25T10:17:31.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.525+0000 7f34aa600640 1 -- 172.21.15.119:0/2491214083 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f348406b4d0 msgr2=0x7f348406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.525+0000 7f34aa600640 1 --2- 172.21.15.119:0/2491214083 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f348406b4d0 0x7f348406d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3498034000 tx=0x7f349802e000 comp rx=0 tx=0).stop 2024-04-25T10:17:31.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.525+0000 7f34aa600640 1 -- 172.21.15.119:0/2491214083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34a40cea30 msgr2=0x7f34a415f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.525+0000 7f34aa600640 1 --2- 172.21.15.119:0/2491214083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34a40cea30 0x7f34a415f680 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f349000e980 tx=0x7f349000ee40 comp rx=0 tx=0).stop 2024-04-25T10:17:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.525+0000 7f34aa600640 1 -- 172.21.15.119:0/2491214083 shutdown_connections 2024-04-25T10:17:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.525+0000 7f34aa600640 1 --2- 172.21.15.119:0/2491214083 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f348406b4d0 0x7f348406d990 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.525+0000 7f34aa600640 1 --2- 172.21.15.119:0/2491214083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34a4161d60 0x7f34a4160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.525+0000 7f34aa600640 1 --2- 172.21.15.119:0/2491214083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34a40cea30 0x7f34a415f680 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.525+0000 7f34aa600640 1 --2- 172.21.15.119:0/2491214083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34a406f0c0 0x7f34a40ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.525+0000 7f34aa600640 1 -- 172.21.15.119:0/2491214083 >> 172.21.15.119:0/2491214083 conn(0x7f34a4066ff0 msgr2=0x7f34a406dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.525+0000 7f34aa600640 1 -- 172.21.15.119:0/2491214083 shutdown_connections 2024-04-25T10:17:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:31.525+0000 7f34aa600640 1 -- 172.21.15.119:0/2491214083 wait complete. 2024-04-25T10:17:31.547 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:17:31.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:17:31.654 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:17:31.766 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:17:31.767 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:17:31.767 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:31.825 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:31.825 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:17:31.826 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:31.882 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:31.882 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:17:31.883 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:31.939 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:31.939 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:17:31.939 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:31.995 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:31.995 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:17:31.996 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:32.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.206+0000 7f3b1da00640 1 Processor -- start 2024-04-25T10:17:32.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.206+0000 7f3b1da00640 1 -- start start 2024-04-25T10:17:32.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.207+0000 7f3b1da00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b180b5050 0x7f3b1815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.207+0000 7f3b1da00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b180cd440 con 0x7f3b18144470 2024-04-25T10:17:32.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.207+0000 7f3b1da00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b180cd720 con 0x7f3b1806e430 2024-04-25T10:17:32.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.207+0000 7f3b1da00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b18145340 con 0x7f3b180b5050 2024-04-25T10:17:32.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.208+0000 7f3b17e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3b18144470 0x7f3b180b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50504/0 (socket says 172.21.15.119:50504) 2024-04-25T10:17:32.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.208+0000 7f3b17e00640 1 -- 172.21.15.119:0/1722700529 learned_addr learned my addr 172.21.15.119:0/1722700529 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:32.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.208+0000 7f3b16a00640 1 -- 172.21.15.119:0/1722700529 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931089485 0 0) 0x7f3b180cd440 con 0x7f3b18144470 2024-04-25T10:17:32.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.208+0000 7f3b16a00640 1 -- 172.21.15.119:0/1722700529 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3afc003520 con 0x7f3b18144470 2024-04-25T10:17:32.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.208+0000 7f3b17400640 1 --2- 172.21.15.119:0/1722700529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b180b5050 0x7f3b1815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.208+0000 7f3b16a00640 1 -- 172.21.15.119:0/1722700529 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 812107943 0 0) 0x7f3afc003520 con 0x7f3b18144470 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.208+0000 7f3b16a00640 1 -- 172.21.15.119:0/1722700529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b180b5050 msgr2=0x7f3b1815fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.208+0000 7f3b16a00640 1 --2- 172.21.15.119:0/1722700529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b180b5050 0x7f3b1815fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.208+0000 7f3b16a00640 1 -- 172.21.15.119:0/1722700529 >> v1:172.21.15.132:6789/0 conn(0x7f3b1806e430 legacy=0x7f3b1815bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.208+0000 7f3b16a00640 1 -- 172.21.15.119:0/1722700529 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b180cda00 con 0x7f3b18144470 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.208+0000 7f3b16a00640 1 -- 172.21.15.119:0/1722700529 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f3b08001670 con 0x7f3b18144470 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b16a00640 1 -- 172.21.15.119:0/1722700529 >> v1:172.21.15.119:6789/0 conn(0x7f3b18144470 legacy=0x7f3b180b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b16a00640 1 --2- 172.21.15.119:0/1722700529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3afc003a50 0x7f3afc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b16a00640 1 --2- 172.21.15.119:0/1722700529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3afc004800 0x7f3afc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b16a00640 1 --2- 172.21.15.119:0/1722700529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b180b5050 0x7f3afc005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b16a00640 1 -- 172.21.15.119:0/1722700529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc00bb50 con 0x7f3afc004800 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b16a00640 1 -- 172.21.15.119:0/1722700529 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc00bcd0 con 0x7f3afc003a50 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b16a00640 1 -- 172.21.15.119:0/1722700529 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc00bed0 con 0x7f3b180b5050 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b17400640 1 --2- 172.21.15.119:0/1722700529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3afc004800 0x7f3afc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b17e00640 1 --2- 172.21.15.119:0/1722700529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3afc003a50 0x7f3afc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b17400640 1 -- 172.21.15.119:0/1722700529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b180b5050 msgr2=0x7f3afc005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:32.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b1d000640 1 --2- 172.21.15.119:0/1722700529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b180b5050 0x7f3afc005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b17400640 1 --2- 172.21.15.119:0/1722700529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b180b5050 0x7f3afc005250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b17400640 1 -- 172.21.15.119:0/1722700529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3afc003a50 msgr2=0x7f3afc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:32.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b17400640 1 --2- 172.21.15.119:0/1722700529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3afc003a50 0x7f3afc003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b17400640 1 -- 172.21.15.119:0/1722700529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc003520 con 0x7f3afc004800 2024-04-25T10:17:32.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.209+0000 7f3b17e00640 1 --2- 172.21.15.119:0/1722700529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3afc003a50 0x7f3afc003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:32.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.210+0000 7f3b17400640 1 --2- 172.21.15.119:0/1722700529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3afc004800 0x7f3afc004c80 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400ba30 tx=0x7f3afc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c629b20e1e2dfdcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:32.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.211+0000 7f3b16a00640 1 -- 172.21.15.119:0/1722700529 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04030070 con 0x7f3afc004800 2024-04-25T10:17:32.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.211+0000 7f3b16a00640 1 -- 172.21.15.119:0/1722700529 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0402ade0 con 0x7f3afc004800 2024-04-25T10:17:32.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.211+0000 7f3b16a00640 1 -- 172.21.15.119:0/1722700529 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400b600 con 0x7f3afc004800 2024-04-25T10:17:32.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.211+0000 7f3b1da00640 1 -- 172.21.15.119:0/1722700529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3afc004800 msgr2=0x7f3afc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.211+0000 7f3b1da00640 1 --2- 172.21.15.119:0/1722700529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3afc004800 0x7f3afc004c80 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400ba30 tx=0x7f3afc005830 comp rx=0 tx=0).stop 2024-04-25T10:17:32.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.211+0000 7f3b1d000640 1 -- 172.21.15.119:0/1722700529 reap_dead start 2024-04-25T10:17:32.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.211+0000 7f3b1da00640 1 -- 172.21.15.119:0/1722700529 shutdown_connections 2024-04-25T10:17:32.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.211+0000 7f3b1da00640 1 -- 172.21.15.119:0/1722700529 >> 172.21.15.119:0/1722700529 conn(0x7f3b1806b1d0 msgr2=0x7f3b180b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:32.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.211+0000 7f3b1da00640 1 -- 172.21.15.119:0/1722700529 shutdown_connections 2024-04-25T10:17:32.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.211+0000 7f3b1da00640 1 -- 172.21.15.119:0/1722700529 wait complete. 2024-04-25T10:17:32.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.213+0000 7f3b1da00640 1 Processor -- start 2024-04-25T10:17:32.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.213+0000 7f3b1da00640 1 -- start start 2024-04-25T10:17:32.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.214+0000 7f3b1da00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3afc004800 0x7f3b1815b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.214+0000 7f3b1da00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b1815f070 0x7f3b180a9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.214+0000 7f3b1da00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b181f23f0 0x7f3b1815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.214+0000 7f3b1da00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b180ce200 con 0x7f3b181f23f0 2024-04-25T10:17:32.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.214+0000 7f3b1da00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b18144850 con 0x7f3afc004800 2024-04-25T10:17:32.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.214+0000 7f3b1da00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1815fc90 con 0x7f3b1815f070 2024-04-25T10:17:32.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.215+0000 7f3b1d000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b181f23f0 0x7f3b1815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.215+0000 7f3b1d000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b181f23f0 0x7f3b1815cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45610/0 (socket says 172.21.15.119:45610) 2024-04-25T10:17:32.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.215+0000 7f3b1d000640 1 -- 172.21.15.119:0/2134133210 learned_addr learned my addr 172.21.15.119:0/2134133210 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:32.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.215+0000 7f3b17e00640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3afc004800 0x7f3b1815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.215+0000 7f3b17400640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b1815f070 0x7f3b180a9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.215+0000 7f3b1d000640 1 -- 172.21.15.119:0/2134133210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b1815f070 msgr2=0x7f3b180a9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.215+0000 7f3b1d000640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b1815f070 0x7f3b180a9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.215+0000 7f3b1d000640 1 -- 172.21.15.119:0/2134133210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3afc004800 msgr2=0x7f3b1815b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.215+0000 7f3b1d000640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3afc004800 0x7f3b1815b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.215+0000 7f3b1d000640 1 -- 172.21.15.119:0/2134133210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1815bc90 con 0x7f3b181f23f0 2024-04-25T10:17:32.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.215+0000 7f3b17e00640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3afc004800 0x7f3b1815b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:32.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.216+0000 7f3b1d000640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b181f23f0 0x7f3b1815cd30 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f3b180b23d0 tx=0x7f3b1000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:32.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.216+0000 7f3b0fe00640 1 -- 172.21.15.119:0/2134133210 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10004b70 con 0x7f3b181f23f0 2024-04-25T10:17:32.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.216+0000 7f3b0fe00640 1 -- 172.21.15.119:0/2134133210 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b10004d10 con 0x7f3b181f23f0 2024-04-25T10:17:32.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.216+0000 7f3b0fe00640 1 -- 172.21.15.119:0/2134133210 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10005270 con 0x7f3b181f23f0 2024-04-25T10:17:32.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.216+0000 7f3b1da00640 1 -- 172.21.15.119:0/2134133210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1815bf70 con 0x7f3b181f23f0 2024-04-25T10:17:32.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.216+0000 7f3b1da00640 1 -- 172.21.15.119:0/2134133210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1815c530 con 0x7f3b181f23f0 2024-04-25T10:17:32.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.217+0000 7f3b1da00640 1 -- 172.21.15.119:0/2134133210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b180749b0 con 0x7f3b181f23f0 2024-04-25T10:17:32.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.218+0000 7f3b0fe00640 1 -- 172.21.15.119:0/2134133210 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b10005410 con 0x7f3b181f23f0 2024-04-25T10:17:32.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.219+0000 7f3b0fe00640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3af806b500 0x7f3af806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.219+0000 7f3b0fe00640 1 -- 172.21.15.119:0/2134133210 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3b10089d90 con 0x7f3b181f23f0 2024-04-25T10:17:32.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.219+0000 7f3b17e00640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3af806b500 0x7f3af806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.220+0000 7f3b17e00640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3af806b500 0x7f3af806d9c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3b08005b90 tx=0x7f3b08009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:32.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.223+0000 7f3b0fe00640 1 -- 172.21.15.119:0/2134133210 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b10016020 con 0x7f3b181f23f0 2024-04-25T10:17:32.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.461+0000 7f3b1da00640 1 -- 172.21.15.119:0/2134133210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b1815fa50 con 0x7f3b181f23f0 2024-04-25T10:17:32.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.463+0000 7f3b0fe00640 1 -- 172.21.15.119:0/2134133210 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f3b10058ea0 con 0x7f3b181f23f0 2024-04-25T10:17:32.465 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:32.465 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5459,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3768348114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3768348114},{"type":"v1","addr":"172.21.15.132:6834","nonce":3768348114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":5465,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/543190780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":543190780},{"type":"v1","addr":"172.21.15.132:6835","nonce":543190780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/1145229172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1145229172},{"type":"v1","addr":"172.21.15.119:6836","nonce":1145229172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1704625876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1704625876},{"type":"v1","addr":"172.21.15.119:6837","nonce":1704625876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[]} 2024-04-25T10:17:32.465 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 92 2024-04-25T10:17:32.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.465+0000 7f3b1da00640 1 -- 172.21.15.119:0/2134133210 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3af806b500 msgr2=0x7f3af806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.465+0000 7f3b1da00640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3af806b500 0x7f3af806d9c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3b08005b90 tx=0x7f3b08009040 comp rx=0 tx=0).stop 2024-04-25T10:17:32.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.465+0000 7f3b1da00640 1 -- 172.21.15.119:0/2134133210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b181f23f0 msgr2=0x7f3b1815cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.465+0000 7f3b1da00640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b181f23f0 0x7f3b1815cd30 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f3b180b23d0 tx=0x7f3b1000d9d0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.465+0000 7f3b1da00640 1 -- 172.21.15.119:0/2134133210 shutdown_connections 2024-04-25T10:17:32.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.465+0000 7f3b1da00640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3af806b500 0x7f3af806d9c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.465+0000 7f3b1da00640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b181f23f0 0x7f3b1815cd30 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.465+0000 7f3b1da00640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3b1815f070 0x7f3b180a9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.465+0000 7f3b1da00640 1 --2- 172.21.15.119:0/2134133210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3afc004800 0x7f3b1815b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.465+0000 7f3b1da00640 1 -- 172.21.15.119:0/2134133210 >> 172.21.15.119:0/2134133210 conn(0x7f3b1806b1d0 msgr2=0x7f3b18140e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:32.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.466+0000 7f3b1da00640 1 -- 172.21.15.119:0/2134133210 shutdown_connections 2024-04-25T10:17:32.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.466+0000 7f3b1da00640 1 -- 172.21.15.119:0/2134133210 wait complete. 2024-04-25T10:17:32.488 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:17:32.488 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:17:32.488 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:17:32.489 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:32.489 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:17:32.489 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:17:32.489 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:17:32.489 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:32.489 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:17:32.489 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:17:32.489 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:17:32.490 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:32.490 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:17:32.490 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:17:32.490 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:17:32.490 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:32.506 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:17:32.506 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:17:32.510 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:17:32.510 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:17:32.511 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:17:32.511 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:17:32.515 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:17:32.516 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:17:32.555 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:17:32.555 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:32.561 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:17:32.561 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:32.562 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:17:32.562 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:32.568 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:17:32.568 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:32.568 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:17:32.701 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2024-04-25T10:17:32.718 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-04-25T10:17:32.718 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-04-25T10:17:32.720 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2024-04-25T10:17:32.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.755+0000 7f23f4200640 1 Processor -- start 2024-04-25T10:17:32.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.755+0000 7f23f4200640 1 -- start start 2024-04-25T10:17:32.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.756+0000 7f23f4200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23ec15f0b0 0x7f23ec0d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.757+0000 7f23f4200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f23ec0cc240 0x7f23ec0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.757+0000 7f23f4200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ec15f490 con 0x7f23ec0ce1e0 2024-04-25T10:17:32.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.757+0000 7f23f4200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23ec0cf8f0 con 0x7f23ec0cc240 2024-04-25T10:17:32.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.757+0000 7f23f4200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0cfbd0 con 0x7f23ec15f0b0 2024-04-25T10:17:32.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.757+0000 7f23f0e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f23ec0ce1e0 0x7f23ec0ce640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50520/0 (socket says 172.21.15.119:50520) 2024-04-25T10:17:32.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.757+0000 7f23f0e00640 1 -- 172.21.15.119:0/1968091554 learned_addr learned my addr 172.21.15.119:0/1968091554 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:32.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.757+0000 7f23f1800640 1 --2- 172.21.15.119:0/1968091554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23ec15f0b0 0x7f23ec0d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.757+0000 7f23f2200640 1 --2- 172.21.15.119:0/1968091554 >> v2:172.21.15.132:3300/0 conn(0x7f23ec0cc240 0x7f23ec0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.757+0000 7f23ebe00640 1 -- 172.21.15.119:0/1968091554 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2327260707 0 0) 0x7f23ec15f490 con 0x7f23ec0ce1e0 2024-04-25T10:17:32.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.757+0000 7f23ebe00640 1 -- 172.21.15.119:0/1968091554 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23d4003520 con 0x7f23ec0ce1e0 2024-04-25T10:17:32.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.758+0000 7f23f1800640 1 -- 172.21.15.119:0/1968091554 >> v2:172.21.15.132:3300/0 conn(0x7f23ec0cc240 msgr2=0x7f23ec0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:32.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.758+0000 7f23f1800640 1 --2- 172.21.15.119:0/1968091554 >> v2:172.21.15.132:3300/0 conn(0x7f23ec0cc240 0x7f23ec0cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.758+0000 7f23f1800640 1 -- 172.21.15.119:0/1968091554 >> v1:172.21.15.119:6789/0 conn(0x7f23ec0ce1e0 legacy=0x7f23ec0ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.758+0000 7f23f1800640 1 -- 172.21.15.119:0/1968091554 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ec0cfeb0 con 0x7f23ec15f0b0 2024-04-25T10:17:32.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.759+0000 7f23f1800640 1 --2- 172.21.15.119:0/1968091554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23ec15f0b0 0x7f23ec0d0990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f23e00097a0 tx=0x7f23e002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cbd6f299ea5b3cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:32.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.759+0000 7f23ebe00640 1 -- 172.21.15.119:0/1968091554 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e0037070 con 0x7f23ec15f0b0 2024-04-25T10:17:32.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.759+0000 7f23ebe00640 1 -- 172.21.15.119:0/1968091554 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e0032440 con 0x7f23ec15f0b0 2024-04-25T10:17:32.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.759+0000 7f23ebe00640 1 -- 172.21.15.119:0/1968091554 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e0032810 con 0x7f23ec15f0b0 2024-04-25T10:17:32.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.760+0000 7f23f4200640 1 -- 172.21.15.119:0/1968091554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23ec15f0b0 msgr2=0x7f23ec0d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.760+0000 7f23f4200640 1 --2- 172.21.15.119:0/1968091554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23ec15f0b0 0x7f23ec0d0990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f23e00097a0 tx=0x7f23e002a230 comp rx=0 tx=0).stop 2024-04-25T10:17:32.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.760+0000 7f23f4200640 1 -- 172.21.15.119:0/1968091554 shutdown_connections 2024-04-25T10:17:32.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.760+0000 7f23f4200640 1 --2- 172.21.15.119:0/1968091554 >> v2:172.21.15.132:3300/0 conn(0x7f23ec0cc240 0x7f23ec0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.760+0000 7f23f4200640 1 --2- 172.21.15.119:0/1968091554 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23ec15f0b0 0x7f23ec0d0990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.760+0000 7f23f4200640 1 -- 172.21.15.119:0/1968091554 >> 172.21.15.119:0/1968091554 conn(0x7f23ec06f430 msgr2=0x7f23ec06f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:32.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.760+0000 7f23f4200640 1 -- 172.21.15.119:0/1968091554 shutdown_connections 2024-04-25T10:17:32.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.760+0000 7f23f4200640 1 -- 172.21.15.119:0/1968091554 wait complete. 2024-04-25T10:17:32.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.762+0000 7f23f4200640 1 Processor -- start 2024-04-25T10:17:32.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.762+0000 7f23f4200640 1 -- start start 2024-04-25T10:17:32.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.763+0000 7f23f4200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23ec0cc240 0x7f23ec0c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.763+0000 7f23f4200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23ec0ce1e0 0x7f23ec1f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.763+0000 7f23f4200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23ec15f0b0 0x7f23ec0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.763+0000 7f23f4200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0ccc10 con 0x7f23ec15f0b0 2024-04-25T10:17:32.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.763+0000 7f23f4200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0d0470 con 0x7f23ec0cc240 2024-04-25T10:17:32.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.763+0000 7f23f4200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0d0ed0 con 0x7f23ec0ce1e0 2024-04-25T10:17:32.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.763+0000 7f23f2200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23ec15f0b0 0x7f23ec0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.763+0000 7f23f2200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23ec15f0b0 0x7f23ec0ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45638/0 (socket says 172.21.15.119:45638) 2024-04-25T10:17:32.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.763+0000 7f23f2200640 1 -- 172.21.15.119:0/2592645205 learned_addr learned my addr 172.21.15.119:0/2592645205 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:32.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.763+0000 7f23f0e00640 1 --2- 172.21.15.119:0/2592645205 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23ec0ce1e0 0x7f23ec1f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.763+0000 7f23f1800640 1 --2- 172.21.15.119:0/2592645205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23ec0cc240 0x7f23ec0c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.764+0000 7f23f2200640 1 -- 172.21.15.119:0/2592645205 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23ec0ce1e0 msgr2=0x7f23ec1f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.764+0000 7f23f2200640 1 --2- 172.21.15.119:0/2592645205 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23ec0ce1e0 0x7f23ec1f4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.764+0000 7f23f2200640 1 -- 172.21.15.119:0/2592645205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23ec0cc240 msgr2=0x7f23ec0c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.764+0000 7f23f2200640 1 --2- 172.21.15.119:0/2592645205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23ec0cc240 0x7f23ec0c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.764+0000 7f23f2200640 1 -- 172.21.15.119:0/2592645205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ec1f49c0 con 0x7f23ec15f0b0 2024-04-25T10:17:32.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.764+0000 7f23f2200640 1 --2- 172.21.15.119:0/2592645205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23ec15f0b0 0x7f23ec0ca640 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009dd0 tx=0x7f23e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:32.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.764+0000 7f23e9600640 1 -- 172.21.15.119:0/2592645205 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4018020 con 0x7f23ec15f0b0 2024-04-25T10:17:32.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.764+0000 7f23e9600640 1 -- 172.21.15.119:0/2592645205 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e4014480 con 0x7f23ec15f0b0 2024-04-25T10:17:32.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.764+0000 7f23e9600640 1 -- 172.21.15.119:0/2592645205 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4014870 con 0x7f23ec15f0b0 2024-04-25T10:17:32.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.764+0000 7f23f4200640 1 -- 172.21.15.119:0/2592645205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ec0c8e20 con 0x7f23ec15f0b0 2024-04-25T10:17:32.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.764+0000 7f23f4200640 1 -- 172.21.15.119:0/2592645205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ec0c93e0 con 0x7f23ec15f0b0 2024-04-25T10:17:32.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.766+0000 7f23f4200640 1 -- 172.21.15.119:0/2592645205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ec0b4460 con 0x7f23ec15f0b0 2024-04-25T10:17:32.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.767+0000 7f23e9600640 1 -- 172.21.15.119:0/2592645205 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23e4005d10 con 0x7f23ec15f0b0 2024-04-25T10:17:32.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.768+0000 7f23e9600640 1 --2- 172.21.15.119:0/2592645205 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f23cc06b500 0x7f23cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.768+0000 7f23e9600640 1 -- 172.21.15.119:0/2592645205 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f23e408aa30 con 0x7f23ec15f0b0 2024-04-25T10:17:32.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.768+0000 7f23f1800640 1 --2- 172.21.15.119:0/2592645205 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f23cc06b500 0x7f23cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.769+0000 7f23f1800640 1 --2- 172.21.15.119:0/2592645205 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f23cc06b500 0x7f23cc06d9c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f23e0034000 tx=0x7f23e002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:32.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.773+0000 7f23e9600640 1 -- 172.21.15.119:0/2592645205 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f23ec0b4460 con 0x7f23ec15f0b0 2024-04-25T10:17:32.780 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:32.784 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:32.982+0000 7f23f4200640 1 -- 172.21.15.119:0/2592645205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f23ec0750b0 con 0x7f23ec15f0b0 2024-04-25T10:17:33.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.119+0000 7f23e9600640 1 -- 172.21.15.119:0/2592645205 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v102) v1 ==== 97+0+0 (secure 0 0 0) 0x7f23e4022070 con 0x7f23ec15f0b0 2024-04-25T10:17:33.123 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2024-04-25T10:17:33.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.123+0000 7f23f4200640 1 -- 172.21.15.119:0/2592645205 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f23cc06b500 msgr2=0x7f23cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.123+0000 7f23f4200640 1 --2- 172.21.15.119:0/2592645205 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f23cc06b500 0x7f23cc06d9c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f23e0034000 tx=0x7f23e002e000 comp rx=0 tx=0).stop 2024-04-25T10:17:33.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.123+0000 7f23f4200640 1 -- 172.21.15.119:0/2592645205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23ec15f0b0 msgr2=0x7f23ec0ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.123+0000 7f23f4200640 1 --2- 172.21.15.119:0/2592645205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23ec15f0b0 0x7f23ec0ca640 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009dd0 tx=0x7f23e400c490 comp rx=0 tx=0).stop 2024-04-25T10:17:33.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.123+0000 7f23f4200640 1 -- 172.21.15.119:0/2592645205 shutdown_connections 2024-04-25T10:17:33.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.123+0000 7f23f4200640 1 --2- 172.21.15.119:0/2592645205 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f23cc06b500 0x7f23cc06d9c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.123+0000 7f23f4200640 1 --2- 172.21.15.119:0/2592645205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23ec15f0b0 0x7f23ec0ca640 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.123+0000 7f23f4200640 1 --2- 172.21.15.119:0/2592645205 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23ec0ce1e0 0x7f23ec1f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.123+0000 7f23f4200640 1 --2- 172.21.15.119:0/2592645205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23ec0cc240 0x7f23ec0c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.123+0000 7f23f4200640 1 -- 172.21.15.119:0/2592645205 >> 172.21.15.119:0/2592645205 conn(0x7f23ec06f430 msgr2=0x7f23ec06d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:33.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.123+0000 7f23f4200640 1 -- 172.21.15.119:0/2592645205 shutdown_connections 2024-04-25T10:17:33.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.124+0000 7f23f4200640 1 -- 172.21.15.119:0/2592645205 wait complete. 2024-04-25T10:17:33.150 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:17:33.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.341+0000 7f5376e00640 1 Processor -- start 2024-04-25T10:17:33.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.341+0000 7f5376e00640 1 -- start start 2024-04-25T10:17:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.343+0000 7f5376e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53700cd6a0 0x7f53700cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.343+0000 7f5376e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f53700b4cd0 0x7f53700b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.343+0000 7f5376e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53700cb150 con 0x7f537015a850 2024-04-25T10:17:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.343+0000 7f5376e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53700b5660 con 0x7f53700b4cd0 2024-04-25T10:17:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.343+0000 7f5376e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53700cb420 con 0x7f53700cd6a0 2024-04-25T10:17:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.343+0000 7f536fe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f537015a850 0x7f53700ceb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50522/0 (socket says 172.21.15.119:50522) 2024-04-25T10:17:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.343+0000 7f536fe00640 1 -- 172.21.15.119:0/3141392156 learned_addr learned my addr 172.21.15.119:0/3141392156 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.343+0000 7f5374e00640 1 --2- 172.21.15.119:0/3141392156 >> v2:172.21.15.119:3300/0 conn(0x7f53700b4cd0 0x7f53700b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.343+0000 7f536f400640 1 -- 172.21.15.119:0/3141392156 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535762277 0 0) 0x7f53700cb150 con 0x7f537015a850 2024-04-25T10:17:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.344+0000 7f536f400640 1 -- 172.21.15.119:0/3141392156 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5354003520 con 0x7f537015a850 2024-04-25T10:17:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.344+0000 7f5374e00640 1 -- 172.21.15.119:0/3141392156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53700cd6a0 msgr2=0x7f53700cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.344+0000 7f5374e00640 1 --2- 172.21.15.119:0/3141392156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53700cd6a0 0x7f53700cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.344+0000 7f5374e00640 1 -- 172.21.15.119:0/3141392156 >> v1:172.21.15.119:6789/0 conn(0x7f537015a850 legacy=0x7f53700ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.344+0000 7f5374e00640 1 -- 172.21.15.119:0/3141392156 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53700cb700 con 0x7f53700b4cd0 2024-04-25T10:17:33.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.345+0000 7f5374e00640 1 --2- 172.21.15.119:0/3141392156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53700b4cd0 0x7f53700b5120 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f5368009d90 tx=0x7f536802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9107d865f0031fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.345+0000 7f536f400640 1 -- 172.21.15.119:0/3141392156 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368007740 con 0x7f53700b4cd0 2024-04-25T10:17:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.345+0000 7f536f400640 1 -- 172.21.15.119:0/3141392156 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53680078e0 con 0x7f53700b4cd0 2024-04-25T10:17:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.345+0000 7f536f400640 1 -- 172.21.15.119:0/3141392156 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368007cb0 con 0x7f53700b4cd0 2024-04-25T10:17:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.346+0000 7f5376e00640 1 -- 172.21.15.119:0/3141392156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53700b4cd0 msgr2=0x7f53700b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.346+0000 7f5376e00640 1 --2- 172.21.15.119:0/3141392156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53700b4cd0 0x7f53700b5120 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f5368009d90 tx=0x7f536802a660 comp rx=0 tx=0).stop 2024-04-25T10:17:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.346+0000 7f5376e00640 1 -- 172.21.15.119:0/3141392156 shutdown_connections 2024-04-25T10:17:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.346+0000 7f5376e00640 1 --2- 172.21.15.119:0/3141392156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53700b4cd0 0x7f53700b5120 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.346+0000 7f5376e00640 1 --2- 172.21.15.119:0/3141392156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53700cd6a0 0x7f53700cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.346+0000 7f5376e00640 1 -- 172.21.15.119:0/3141392156 >> 172.21.15.119:0/3141392156 conn(0x7f53700cf5a0 msgr2=0x7f53700cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.346+0000 7f5376e00640 1 -- 172.21.15.119:0/3141392156 shutdown_connections 2024-04-25T10:17:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.346+0000 7f5376e00640 1 -- 172.21.15.119:0/3141392156 wait complete. 2024-04-25T10:17:33.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.348+0000 7f5376e00640 1 Processor -- start 2024-04-25T10:17:33.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.348+0000 7f5376e00640 1 -- start start 2024-04-25T10:17:33.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.349+0000 7f5376e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53700b4cd0 0x7f53700b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.349+0000 7f5376e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53700cd6a0 0x7f5370151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.349+0000 7f5376e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f537015a850 0x7f53700c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.349+0000 7f5376e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53700cbcc0 con 0x7f537015a850 2024-04-25T10:17:33.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.349+0000 7f5376e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53700cd180 con 0x7f53700cd6a0 2024-04-25T10:17:33.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.349+0000 7f5376e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53700cf240 con 0x7f53700b4cd0 2024-04-25T10:17:33.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.349+0000 7f5374e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f537015a850 0x7f53700c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.349+0000 7f5374e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f537015a850 0x7f53700c88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45666/0 (socket says 172.21.15.119:45666) 2024-04-25T10:17:33.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.349+0000 7f5374e00640 1 -- 172.21.15.119:0/4002204163 learned_addr learned my addr 172.21.15.119:0/4002204163 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.349+0000 7f5374e00640 1 -- 172.21.15.119:0/4002204163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53700b4cd0 msgr2=0x7f53700b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.349+0000 7f5374e00640 1 --2- 172.21.15.119:0/4002204163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53700b4cd0 0x7f53700b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.349+0000 7f5374e00640 1 -- 172.21.15.119:0/4002204163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53700cd6a0 msgr2=0x7f5370151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.349+0000 7f5367400640 1 --2- 172.21.15.119:0/4002204163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53700cd6a0 0x7f5370151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.350+0000 7f5374e00640 1 --2- 172.21.15.119:0/4002204163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53700cd6a0 0x7f5370151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.350+0000 7f5374e00640 1 -- 172.21.15.119:0/4002204163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53700b3e20 con 0x7f537015a850 2024-04-25T10:17:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.350+0000 7f5367400640 1 --2- 172.21.15.119:0/4002204163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53700cd6a0 0x7f5370151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.350+0000 7f5374e00640 1 --2- 172.21.15.119:0/4002204163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f537015a850 0x7f53700c88d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5368009d90 tx=0x7f5368004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.350+0000 7f536cc00640 1 -- 172.21.15.119:0/4002204163 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536803f070 con 0x7f537015a850 2024-04-25T10:17:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.350+0000 7f536cc00640 1 -- 172.21.15.119:0/4002204163 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5368031bc0 con 0x7f537015a850 2024-04-25T10:17:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.350+0000 7f536cc00640 1 -- 172.21.15.119:0/4002204163 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368009450 con 0x7f537015a850 2024-04-25T10:17:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.350+0000 7f5376e00640 1 -- 172.21.15.119:0/4002204163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53700b4100 con 0x7f537015a850 2024-04-25T10:17:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.350+0000 7f5376e00640 1 -- 172.21.15.119:0/4002204163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53700af5a0 con 0x7f537015a850 2024-04-25T10:17:33.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.352+0000 7f5376e00640 1 -- 172.21.15.119:0/4002204163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53700b42d0 con 0x7f537015a850 2024-04-25T10:17:33.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.353+0000 7f536cc00640 1 -- 172.21.15.119:0/4002204163 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f536803b410 con 0x7f537015a850 2024-04-25T10:17:33.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.353+0000 7f536cc00640 1 --2- 172.21.15.119:0/4002204163 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f534c06b4d0 0x7f534c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.354+0000 7f536cc00640 1 -- 172.21.15.119:0/4002204163 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f53680aaf50 con 0x7f537015a850 2024-04-25T10:17:33.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.354+0000 7f536fe00640 1 --2- 172.21.15.119:0/4002204163 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f534c06b4d0 0x7f534c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.354+0000 7f536fe00640 1 --2- 172.21.15.119:0/4002204163 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f534c06b4d0 0x7f534c06d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5360000f60 tx=0x7f53600078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.358+0000 7f536cc00640 1 -- 172.21.15.119:0/4002204163 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f536807a080 con 0x7f537015a850 2024-04-25T10:17:33.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.624+0000 7f5376e00640 1 -- 172.21.15.119:0/4002204163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f53700b1160 con 0x7f537015a850 2024-04-25T10:17:33.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.625+0000 7f536cc00640 1 -- 172.21.15.119:0/4002204163 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5368079980 con 0x7f537015a850 2024-04-25T10:17:33.629 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:33.629 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:17:33.629 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2024-04-25T10:17:33.629 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.629 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2024-04-25T10:17:33.629 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmFDz9FBAAdC/N+8UnkYkK5wrDlgpkgA==", 2024-04-25T10:17:33.629 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.630 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:17:33.630 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:17:33.630 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:17:33.630 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.630 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.630 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.630 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.630 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2024-04-25T10:17:33.630 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmmh7lLhAAkzsFzTL6Ko/J/0ZJOpvR4A==", 2024-04-25T10:17:33.630 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.630 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:17:33.630 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:17:33.631 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:17:33.631 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.631 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.631 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.631 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2024-04-25T10:17:33.631 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmojotIhAAdSIhFOXoYZN+01nbCs3e+Q==", 2024-04-25T10:17:33.631 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.631 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:17:33.631 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:17:33.631 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:17:33.631 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.631 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypm1gp7ORAAvtG28IuhGtR1/Vj6JCPq5A==", 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAPKypmOI9YMRAAmMrg0xRUG+GG+waJ12llJQ==", 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.632 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.633 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.633 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.633 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.633 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2024-04-25T10:17:33.633 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQATKypmHV5GDxAAW0XzpAxF7oxgVBKy5yzavw==", 2024-04-25T10:17:33.633 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.633 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.633 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.633 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.633 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.633 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.633 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.634 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2024-04-25T10:17:33.634 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAWKypmXTiRGhAAZxct/Ck3A3DMBjqNW3HXrg==", 2024-04-25T10:17:33.634 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.634 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.634 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.634 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.634 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.634 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.634 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2024-04-25T10:17:33.634 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAZKypmEeGSMhAAwVNTny5OmP/KONYhIXzrrQ==", 2024-04-25T10:17:33.634 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.634 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAgKypmh7gPBRAApx1VM3WpYW7hIQzu7+6J7w==", 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.635 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAjKypmQ2EaEBAAoFYMlnh5/ar0pAlynwjYOA==", 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAmKypmDATkKRAAsIDViOBeDrd5N91wf8zGLA==", 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAqKypmNskbCBAAQIYhJ9lfHHr9EP/n5QJoRw==", 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA==", 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.637 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:17:33.638 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2024-04-25T10:17:33.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:17:33.638 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.638 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.638 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.638 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2024-04-25T10:17:33.638 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmm0DLGhAApkxEcc1jUU5T/9ErkTkT4A==", 2024-04-25T10:17:33.638 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.638 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:17:33.638 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:17:33.638 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:17:33.638 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:17:33.638 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmMZ89ERAAICfWm/ZgOLQMVSv5xG0Crg==", 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAMKypmrokCIxAAwi8FtvYSVhMguvZ0OyYt+g==", 2024-04-25T10:17:33.639 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.640 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2024-04-25T10:17:33.640 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2024-04-25T10:17:33.640 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2024-04-25T10:17:33.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2024-04-25T10:17:33.640 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.640 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.640 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.640 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2024-04-25T10:17:33.640 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQAKKypmN9WFGRAAi+stjuG2ZDBjafeR9Ygrug==", 2024-04-25T10:17:33.640 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.640 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:17:33.640 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2024-04-25T10:17:33.640 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2024-04-25T10:17:33.640 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypm/KyKDBAA646fW6qrcXB31usxeIApaA==", 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQALKypmr8SqBBAA8l4CjUEidXz1obWJFM4DHA==", 2024-04-25T10:17:33.641 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2024-04-25T10:17:33.642 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2024-04-25T10:17:33.642 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2024-04-25T10:17:33.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2024-04-25T10:17:33.642 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.642 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:33.642 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-25T10:17:33.642 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:17:33.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.628+0000 7f5376e00640 1 -- 172.21.15.119:0/4002204163 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f534c06b4d0 msgr2=0x7f534c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.628+0000 7f5376e00640 1 --2- 172.21.15.119:0/4002204163 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f534c06b4d0 0x7f534c06d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5360000f60 tx=0x7f53600078f0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.628+0000 7f5376e00640 1 -- 172.21.15.119:0/4002204163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f537015a850 msgr2=0x7f53700c88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.628+0000 7f5376e00640 1 --2- 172.21.15.119:0/4002204163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f537015a850 0x7f53700c88d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5368009d90 tx=0x7f5368004a40 comp rx=0 tx=0).stop 2024-04-25T10:17:33.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.628+0000 7f5376e00640 1 -- 172.21.15.119:0/4002204163 shutdown_connections 2024-04-25T10:17:33.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.628+0000 7f5376e00640 1 --2- 172.21.15.119:0/4002204163 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f534c06b4d0 0x7f534c06d990 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.628+0000 7f5376e00640 1 --2- 172.21.15.119:0/4002204163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f537015a850 0x7f53700c88d0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.628+0000 7f5376e00640 1 --2- 172.21.15.119:0/4002204163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53700cd6a0 0x7f5370151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.628+0000 7f5376e00640 1 --2- 172.21.15.119:0/4002204163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53700b4cd0 0x7f53700b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.628+0000 7f5376e00640 1 -- 172.21.15.119:0/4002204163 >> 172.21.15.119:0/4002204163 conn(0x7f53700cf5a0 msgr2=0x7f53700cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:33.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.628+0000 7f5376e00640 1 -- 172.21.15.119:0/4002204163 shutdown_connections 2024-04-25T10:17:33.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.628+0000 7f5376e00640 1 -- 172.21.15.119:0/4002204163 wait complete. 2024-04-25T10:17:33.651 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:33.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.847+0000 7f5df2e00640 1 Processor -- start 2024-04-25T10:17:33.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.848+0000 7f5df2e00640 1 -- start start 2024-04-25T10:17:33.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.849+0000 7f5df2e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5dec15dc30 0x7f5dec06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.849+0000 7f5df2e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5dec06cc00 0x7f5dec0b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.849+0000 7f5df2e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dec0b1960 0x7f5dec099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.849+0000 7f5df2e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dec06c1f0 con 0x7f5dec15dc30 2024-04-25T10:17:33.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.849+0000 7f5df2e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dec0cfb40 con 0x7f5dec06cc00 2024-04-25T10:17:33.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.849+0000 7f5df2e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec0b1db0 con 0x7f5dec0b1960 2024-04-25T10:17:33.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.849+0000 7f5debe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5dec15dc30 0x7f5dec06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.850+0000 7f5debe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5dec15dc30 0x7f5dec06c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45670/0 (socket says 172.21.15.119:45670) 2024-04-25T10:17:33.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.850+0000 7f5debe00640 1 -- 172.21.15.119:0/3072005907 learned_addr learned my addr 172.21.15.119:0/3072005907 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:33.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.850+0000 7f5de3e00640 1 --2- 172.21.15.119:0/3072005907 >> v2:172.21.15.132:3300/0 conn(0x7f5dec06cc00 0x7f5dec0b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.850+0000 7f5debe00640 1 -- 172.21.15.119:0/3072005907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dec0b1960 msgr2=0x7f5dec099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:33.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.850+0000 7f5df0e00640 1 --2- 172.21.15.119:0/3072005907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dec0b1960 0x7f5dec099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.850+0000 7f5debe00640 1 --2- 172.21.15.119:0/3072005907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dec0b1960 0x7f5dec099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.850+0000 7f5debe00640 1 -- 172.21.15.119:0/3072005907 >> v2:172.21.15.132:3300/0 conn(0x7f5dec06cc00 msgr2=0x7f5dec0b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:33.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.850+0000 7f5debe00640 1 --2- 172.21.15.119:0/3072005907 >> v2:172.21.15.132:3300/0 conn(0x7f5dec06cc00 0x7f5dec0b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.850+0000 7f5debe00640 1 -- 172.21.15.119:0/3072005907 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec0b2060 con 0x7f5dec15dc30 2024-04-25T10:17:33.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.850+0000 7f5de3e00640 1 --2- 172.21.15.119:0/3072005907 >> v2:172.21.15.132:3300/0 conn(0x7f5dec06cc00 0x7f5dec0b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:33.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.850+0000 7f5df0e00640 1 --2- 172.21.15.119:0/3072005907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dec0b1960 0x7f5dec099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:33.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.851+0000 7f5debe00640 1 --2- 172.21.15.119:0/3072005907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dec15dc30 0x7f5dec06c6c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0d27a0 tx=0x7f5ddc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d2518c5a88db49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.851+0000 7f5deb400640 1 -- 172.21.15.119:0/3072005907 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc037070 con 0x7f5dec15dc30 2024-04-25T10:17:33.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.852+0000 7f5deb400640 1 -- 172.21.15.119:0/3072005907 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ddc032440 con 0x7f5dec15dc30 2024-04-25T10:17:33.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.852+0000 7f5deb400640 1 -- 172.21.15.119:0/3072005907 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc032810 con 0x7f5dec15dc30 2024-04-25T10:17:33.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.852+0000 7f5df2e00640 1 -- 172.21.15.119:0/3072005907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dec15dc30 msgr2=0x7f5dec06c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.852+0000 7f5df2e00640 1 --2- 172.21.15.119:0/3072005907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dec15dc30 0x7f5dec06c6c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0d27a0 tx=0x7f5ddc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.852+0000 7f5df2e00640 1 -- 172.21.15.119:0/3072005907 shutdown_connections 2024-04-25T10:17:33.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.852+0000 7f5df2e00640 1 --2- 172.21.15.119:0/3072005907 >> v2:172.21.15.132:3300/0 conn(0x7f5dec06cc00 0x7f5dec0b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.852+0000 7f5df2e00640 1 --2- 172.21.15.119:0/3072005907 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dec0b1960 0x7f5dec099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.852+0000 7f5df2e00640 1 --2- 172.21.15.119:0/3072005907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dec15dc30 0x7f5dec06c6c0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.852+0000 7f5df2e00640 1 -- 172.21.15.119:0/3072005907 >> 172.21.15.119:0/3072005907 conn(0x7f5dec0d1900 msgr2=0x7f5dec0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:33.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.853+0000 7f5df2e00640 1 -- 172.21.15.119:0/3072005907 shutdown_connections 2024-04-25T10:17:33.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.853+0000 7f5df2e00640 1 -- 172.21.15.119:0/3072005907 wait complete. 2024-04-25T10:17:33.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.854+0000 7f5df2e00640 1 Processor -- start 2024-04-25T10:17:33.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.854+0000 7f5df2e00640 1 -- start start 2024-04-25T10:17:33.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.855+0000 7f5df2e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dec06cc00 0x7f5dec157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.855+0000 7f5df2e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dec0b1960 0x7f5dec0b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.855+0000 7f5df2e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dec15dc30 0x7f5dec099a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.855+0000 7f5df2e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec0cfde0 con 0x7f5dec15dc30 2024-04-25T10:17:33.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.855+0000 7f5df2e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec0b2620 con 0x7f5dec06cc00 2024-04-25T10:17:33.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.855+0000 7f5df2e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec15e010 con 0x7f5dec0b1960 2024-04-25T10:17:33.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.856+0000 7f5df0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dec15dc30 0x7f5dec099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.856+0000 7f5df0e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dec15dc30 0x7f5dec099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45672/0 (socket says 172.21.15.119:45672) 2024-04-25T10:17:33.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.856+0000 7f5df0e00640 1 -- 172.21.15.119:0/3886302131 learned_addr learned my addr 172.21.15.119:0/3886302131 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:33.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.856+0000 7f5debe00640 1 --2- 172.21.15.119:0/3886302131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dec06cc00 0x7f5dec157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.856+0000 7f5de3e00640 1 --2- 172.21.15.119:0/3886302131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dec0b1960 0x7f5dec0b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.856+0000 7f5df0e00640 1 -- 172.21.15.119:0/3886302131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dec0b1960 msgr2=0x7f5dec0b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.856+0000 7f5df0e00640 1 --2- 172.21.15.119:0/3886302131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dec0b1960 0x7f5dec0b6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.856+0000 7f5df0e00640 1 -- 172.21.15.119:0/3886302131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dec06cc00 msgr2=0x7f5dec157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.856+0000 7f5df0e00640 1 --2- 172.21.15.119:0/3886302131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dec06cc00 0x7f5dec157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.856+0000 7f5df0e00640 1 -- 172.21.15.119:0/3886302131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec157bc0 con 0x7f5dec15dc30 2024-04-25T10:17:33.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.857+0000 7f5df0e00640 1 --2- 172.21.15.119:0/3886302131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dec15dc30 0x7f5dec099a40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5de400cd50 tx=0x7f5de400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.857+0000 7f5de8c00640 1 -- 172.21.15.119:0/3886302131 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de400f680 con 0x7f5dec15dc30 2024-04-25T10:17:33.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.857+0000 7f5de8c00640 1 -- 172.21.15.119:0/3886302131 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de400f840 con 0x7f5dec15dc30 2024-04-25T10:17:33.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.857+0000 7f5de8c00640 1 -- 172.21.15.119:0/3886302131 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de400fc30 con 0x7f5dec15dc30 2024-04-25T10:17:33.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.857+0000 7f5df2e00640 1 -- 172.21.15.119:0/3886302131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dec0b6410 con 0x7f5dec15dc30 2024-04-25T10:17:33.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.857+0000 7f5df2e00640 1 -- 172.21.15.119:0/3886302131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dec156ba0 con 0x7f5dec15dc30 2024-04-25T10:17:33.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.858+0000 7f5df2e00640 1 -- 172.21.15.119:0/3886302131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dec0749b0 con 0x7f5dec15dc30 2024-04-25T10:17:33.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.859+0000 7f5de8c00640 1 -- 172.21.15.119:0/3886302131 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5de400e3e0 con 0x7f5dec15dc30 2024-04-25T10:17:33.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.860+0000 7f5de8c00640 1 --2- 172.21.15.119:0/3886302131 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5dc806b4d0 0x7f5dc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.860+0000 7f5de8c00640 1 -- 172.21.15.119:0/3886302131 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5de408ad50 con 0x7f5dec15dc30 2024-04-25T10:17:33.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.860+0000 7f5debe00640 1 --2- 172.21.15.119:0/3886302131 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5dc806b4d0 0x7f5dc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.861+0000 7f5debe00640 1 --2- 172.21.15.119:0/3886302131 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5dc806b4d0 0x7f5dc806d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc034000 tx=0x7f5ddc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:33.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:33.864+0000 7f5de8c00640 1 -- 172.21.15.119:0/3886302131 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5de4014030 con 0x7f5dec15dc30 2024-04-25T10:17:34.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.063+0000 7f5df2e00640 1 -- 172.21.15.119:0/3886302131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dec0b1370 con 0x7f5dec15dc30 2024-04-25T10:17:34.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.065+0000 7f5de8c00640 1 -- 172.21.15.119:0/3886302131 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+12929 (secure 0 0 0) 0x7f5de4059e80 con 0x7f5dec15dc30 2024-04-25T10:17:34.068 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:34.069 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":104,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:17:33.131751+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:34.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.067+0000 7f5df2e00640 1 -- 172.21.15.119:0/3886302131 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5dc806b4d0 msgr2=0x7f5dc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.067+0000 7f5df2e00640 1 --2- 172.21.15.119:0/3886302131 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5dc806b4d0 0x7f5dc806d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc034000 tx=0x7f5ddc02e000 comp rx=0 tx=0).stop 2024-04-25T10:17:34.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.068+0000 7f5df2e00640 1 -- 172.21.15.119:0/3886302131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dec15dc30 msgr2=0x7f5dec099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.068+0000 7f5df2e00640 1 --2- 172.21.15.119:0/3886302131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dec15dc30 0x7f5dec099a40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5de400cd50 tx=0x7f5de400a650 comp rx=0 tx=0).stop 2024-04-25T10:17:34.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.068+0000 7f5df2e00640 1 -- 172.21.15.119:0/3886302131 shutdown_connections 2024-04-25T10:17:34.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.068+0000 7f5df2e00640 1 --2- 172.21.15.119:0/3886302131 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5dc806b4d0 0x7f5dc806d990 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.068+0000 7f5df2e00640 1 --2- 172.21.15.119:0/3886302131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dec15dc30 0x7f5dec099a40 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.068+0000 7f5df2e00640 1 --2- 172.21.15.119:0/3886302131 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5dec0b1960 0x7f5dec0b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.068+0000 7f5df2e00640 1 --2- 172.21.15.119:0/3886302131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5dec06cc00 0x7f5dec157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.068+0000 7f5df2e00640 1 -- 172.21.15.119:0/3886302131 >> 172.21.15.119:0/3886302131 conn(0x7f5dec0d1900 msgr2=0x7f5dec0cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:34.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.068+0000 7f5df2e00640 1 -- 172.21.15.119:0/3886302131 shutdown_connections 2024-04-25T10:17:34.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.068+0000 7f5df2e00640 1 -- 172.21.15.119:0/3886302131 wait complete. 2024-04-25T10:17:34.089 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:34.089 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:34.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.279+0000 7fd923200640 1 Processor -- start 2024-04-25T10:17:34.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.279+0000 7fd923200640 1 -- start start 2024-04-25T10:17:34.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.280+0000 7fd923200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd91c0d07b0 0x7fd91c0cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.280+0000 7fd923200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91c0b5710 con 0x7fd91c0ce860 2024-04-25T10:17:34.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.280+0000 7fd923200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91c0b59f0 con 0x7fd91c15b7e0 2024-04-25T10:17:34.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.280+0000 7fd923200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91c0b5cd0 con 0x7fd91c0d07b0 2024-04-25T10:17:34.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.280+0000 7fd922800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd91c0d07b0 0x7fd91c0cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.281+0000 7fd921400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd91c0ce860 0x7fd91c0d00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50536/0 (socket says 172.21.15.119:50536) 2024-04-25T10:17:34.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.281+0000 7fd921400640 1 -- 172.21.15.119:0/1165419155 learned_addr learned my addr 172.21.15.119:0/1165419155 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:34.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.281+0000 7fd922800640 1 -- 172.21.15.119:0/1165419155 >> v1:172.21.15.132:6789/0 conn(0x7fd91c15b7e0 legacy=0x7fd91c15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:34.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.281+0000 7fd922800640 1 -- 172.21.15.119:0/1165419155 >> v1:172.21.15.119:6789/0 conn(0x7fd91c0ce860 legacy=0x7fd91c0d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.281+0000 7fd922800640 1 -- 172.21.15.119:0/1165419155 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c0b5fb0 con 0x7fd91c0d07b0 2024-04-25T10:17:34.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.282+0000 7fd922800640 1 --2- 172.21.15.119:0/1165419155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd91c0d07b0 0x7fd91c0cecf0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fd918009d90 tx=0x7fd91802a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6085f744751f39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.282+0000 7fd920a00640 1 -- 172.21.15.119:0/1165419155 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918007740 con 0x7fd91c0d07b0 2024-04-25T10:17:34.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.282+0000 7fd920a00640 1 -- 172.21.15.119:0/1165419155 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9180078e0 con 0x7fd91c0d07b0 2024-04-25T10:17:34.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.282+0000 7fd920a00640 1 -- 172.21.15.119:0/1165419155 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918007cb0 con 0x7fd91c0d07b0 2024-04-25T10:17:34.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.283+0000 7fd923200640 1 -- 172.21.15.119:0/1165419155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd91c0d07b0 msgr2=0x7fd91c0cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.283+0000 7fd923200640 1 --2- 172.21.15.119:0/1165419155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd91c0d07b0 0x7fd91c0cecf0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fd918009d90 tx=0x7fd91802a590 comp rx=0 tx=0).stop 2024-04-25T10:17:34.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.283+0000 7fd923200640 1 -- 172.21.15.119:0/1165419155 shutdown_connections 2024-04-25T10:17:34.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.283+0000 7fd923200640 1 --2- 172.21.15.119:0/1165419155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd91c0d07b0 0x7fd91c0cecf0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.283+0000 7fd923200640 1 -- 172.21.15.119:0/1165419155 >> 172.21.15.119:0/1165419155 conn(0x7fd91c0722d0 msgr2=0x7fd91c0726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:34.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.283+0000 7fd923200640 1 -- 172.21.15.119:0/1165419155 shutdown_connections 2024-04-25T10:17:34.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.283+0000 7fd923200640 1 -- 172.21.15.119:0/1165419155 wait complete. 2024-04-25T10:17:34.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.285+0000 7fd923200640 1 Processor -- start 2024-04-25T10:17:34.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.285+0000 7fd923200640 1 -- start start 2024-04-25T10:17:34.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.286+0000 7fd923200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd91c0ce860 0x7fd91c15bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.286+0000 7fd923200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd91c0d07b0 0x7fd91c142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.286+0000 7fd923200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd91c15b7e0 0x7fd91c1432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.286+0000 7fd923200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c161ae0 con 0x7fd91c0ce860 2024-04-25T10:17:34.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.286+0000 7fd923200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c161960 con 0x7fd91c0d07b0 2024-04-25T10:17:34.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.286+0000 7fd923200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c161c60 con 0x7fd91c15b7e0 2024-04-25T10:17:34.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.286+0000 7fd921e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd91c0ce860 0x7fd91c15bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.286+0000 7fd921e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd91c0ce860 0x7fd91c15bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45682/0 (socket says 172.21.15.119:45682) 2024-04-25T10:17:34.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.286+0000 7fd921e00640 1 -- 172.21.15.119:0/3424866469 learned_addr learned my addr 172.21.15.119:0/3424866469 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:34.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.287+0000 7fd921400640 1 --2- 172.21.15.119:0/3424866469 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd91c0d07b0 0x7fd91c142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.287+0000 7fd922800640 1 --2- 172.21.15.119:0/3424866469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd91c15b7e0 0x7fd91c1432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.287+0000 7fd921e00640 1 -- 172.21.15.119:0/3424866469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd91c15b7e0 msgr2=0x7fd91c1432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.287+0000 7fd921e00640 1 --2- 172.21.15.119:0/3424866469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd91c15b7e0 0x7fd91c1432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.287+0000 7fd921e00640 1 -- 172.21.15.119:0/3424866469 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd91c0d07b0 msgr2=0x7fd91c142da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.287+0000 7fd921e00640 1 --2- 172.21.15.119:0/3424866469 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd91c0d07b0 0x7fd91c142da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.287+0000 7fd921e00640 1 -- 172.21.15.119:0/3424866469 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c152750 con 0x7fd91c0ce860 2024-04-25T10:17:34.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.287+0000 7fd921e00640 1 --2- 172.21.15.119:0/3424866469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd91c0ce860 0x7fd91c15bc20 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd91000b3f0 tx=0x7fd91000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.287+0000 7fd916000640 1 -- 172.21.15.119:0/3424866469 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910015020 con 0x7fd91c0ce860 2024-04-25T10:17:34.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.287+0000 7fd916000640 1 -- 172.21.15.119:0/3424866469 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd910011830 con 0x7fd91c0ce860 2024-04-25T10:17:34.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.287+0000 7fd923200640 1 -- 172.21.15.119:0/3424866469 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91c158430 con 0x7fd91c0ce860 2024-04-25T10:17:34.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.287+0000 7fd916000640 1 -- 172.21.15.119:0/3424866469 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910011c20 con 0x7fd91c0ce860 2024-04-25T10:17:34.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.288+0000 7fd923200640 1 -- 172.21.15.119:0/3424866469 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c150070 con 0x7fd91c0ce860 2024-04-25T10:17:34.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.289+0000 7fd923200640 1 -- 172.21.15.119:0/3424866469 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91c0cecf0 con 0x7fd91c0ce860 2024-04-25T10:17:34.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.290+0000 7fd916000640 1 -- 172.21.15.119:0/3424866469 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd910018020 con 0x7fd91c0ce860 2024-04-25T10:17:34.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.291+0000 7fd916000640 1 --2- 172.21.15.119:0/3424866469 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd90006b4d0 0x7fd90006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.291+0000 7fd916000640 1 -- 172.21.15.119:0/3424866469 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd910089b00 con 0x7fd91c0ce860 2024-04-25T10:17:34.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.291+0000 7fd921400640 1 --2- 172.21.15.119:0/3424866469 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd90006b4d0 0x7fd90006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.292+0000 7fd921400640 1 --2- 172.21.15.119:0/3424866469 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd90006b4d0 0x7fd90006d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd91c1f7870 tx=0x7fd90c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.295+0000 7fd916000640 1 -- 172.21.15.119:0/3424866469 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd91c0cecf0 con 0x7fd91c0ce860 2024-04-25T10:17:34.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.487+0000 7fd923200640 1 -- 172.21.15.119:0/3424866469 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd91c0749b0 con 0x7fd91c0ce860 2024-04-25T10:17:34.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.488+0000 7fd916000640 1 -- 172.21.15.119:0/3424866469 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd910089e20 con 0x7fd91c0ce860 2024-04-25T10:17:34.491 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:17:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.490+0000 7fd923200640 1 -- 172.21.15.119:0/3424866469 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd90006b4d0 msgr2=0x7fd90006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.490+0000 7fd923200640 1 --2- 172.21.15.119:0/3424866469 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd90006b4d0 0x7fd90006d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd91c1f7870 tx=0x7fd90c009040 comp rx=0 tx=0).stop 2024-04-25T10:17:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.490+0000 7fd923200640 1 -- 172.21.15.119:0/3424866469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd91c0ce860 msgr2=0x7fd91c15bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.490+0000 7fd923200640 1 --2- 172.21.15.119:0/3424866469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd91c0ce860 0x7fd91c15bc20 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd91000b3f0 tx=0x7fd91000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.490+0000 7fd923200640 1 -- 172.21.15.119:0/3424866469 shutdown_connections 2024-04-25T10:17:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.491+0000 7fd923200640 1 --2- 172.21.15.119:0/3424866469 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd90006b4d0 0x7fd90006d990 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.491+0000 7fd923200640 1 --2- 172.21.15.119:0/3424866469 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd91c15b7e0 0x7fd91c1432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.491+0000 7fd923200640 1 --2- 172.21.15.119:0/3424866469 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd91c0d07b0 0x7fd91c142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.491+0000 7fd923200640 1 --2- 172.21.15.119:0/3424866469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd91c0ce860 0x7fd91c15bc20 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.491+0000 7fd923200640 1 -- 172.21.15.119:0/3424866469 >> 172.21.15.119:0/3424866469 conn(0x7fd91c0722d0 msgr2=0x7fd91c0b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.491+0000 7fd923200640 1 -- 172.21.15.119:0/3424866469 shutdown_connections 2024-04-25T10:17:34.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.491+0000 7fd923200640 1 -- 172.21.15.119:0/3424866469 wait complete. 2024-04-25T10:17:34.513 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:17:34.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.712+0000 7f5e7ea00640 1 Processor -- start 2024-04-25T10:17:34.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.712+0000 7f5e7ea00640 1 -- start start 2024-04-25T10:17:34.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.714+0000 7f5e7ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e780cc2d0 0x7f5e780cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.714+0000 7f5e7ea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5e780b1be0 0x7f5e780cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.714+0000 7f5e7ea00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e780ce660 con 0x7f5e780cdd70 2024-04-25T10:17:34.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.714+0000 7f5e7ea00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7806b7f0 con 0x7f5e780b1be0 2024-04-25T10:17:34.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.714+0000 7f5e7ea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780cca70 con 0x7f5e780cc2d0 2024-04-25T10:17:34.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.714+0000 7f5e6f400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5e780cdd70 0x7f5e780b14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50544/0 (socket says 172.21.15.119:50544) 2024-04-25T10:17:34.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.714+0000 7f5e6f400640 1 -- 172.21.15.119:0/3002908731 learned_addr learned my addr 172.21.15.119:0/3002908731 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:34.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.715+0000 7f5e77e00640 1 --2- 172.21.15.119:0/3002908731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e780cc2d0 0x7f5e780cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.715+0000 7f5e7ca00640 1 --2- 172.21.15.119:0/3002908731 >> v2:172.21.15.132:3300/0 conn(0x7f5e780b1be0 0x7f5e780cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.715+0000 7f5e77400640 1 -- 172.21.15.119:0/3002908731 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813960215 0 0) 0x7f5e780ce660 con 0x7f5e780cdd70 2024-04-25T10:17:34.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.715+0000 7f5e77400640 1 -- 172.21.15.119:0/3002908731 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e5c003520 con 0x7f5e780cdd70 2024-04-25T10:17:34.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.715+0000 7f5e77e00640 1 -- 172.21.15.119:0/3002908731 >> v2:172.21.15.132:3300/0 conn(0x7f5e780b1be0 msgr2=0x7f5e780cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:34.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.715+0000 7f5e77e00640 1 --2- 172.21.15.119:0/3002908731 >> v2:172.21.15.132:3300/0 conn(0x7f5e780b1be0 0x7f5e780cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.715+0000 7f5e77e00640 1 -- 172.21.15.119:0/3002908731 >> v1:172.21.15.119:6789/0 conn(0x7f5e780cdd70 legacy=0x7f5e780b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.715+0000 7f5e77e00640 1 -- 172.21.15.119:0/3002908731 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e780ccd50 con 0x7f5e780cc2d0 2024-04-25T10:17:34.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.717+0000 7f5e77e00640 1 --2- 172.21.15.119:0/3002908731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e780cc2d0 0x7f5e780cd830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e680097a0 tx=0x7f5e6802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70b6e1d6aa6fc75d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.717+0000 7f5e77400640 1 -- 172.21.15.119:0/3002908731 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e68037070 con 0x7f5e780cc2d0 2024-04-25T10:17:34.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.718+0000 7f5e77400640 1 -- 172.21.15.119:0/3002908731 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e68032440 con 0x7f5e780cc2d0 2024-04-25T10:17:34.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.718+0000 7f5e77400640 1 -- 172.21.15.119:0/3002908731 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e680327f0 con 0x7f5e780cc2d0 2024-04-25T10:17:34.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.718+0000 7f5e7ea00640 1 -- 172.21.15.119:0/3002908731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e780cc2d0 msgr2=0x7f5e780cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.718+0000 7f5e7ea00640 1 --2- 172.21.15.119:0/3002908731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e780cc2d0 0x7f5e780cd830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e680097a0 tx=0x7f5e6802a180 comp rx=0 tx=0).stop 2024-04-25T10:17:34.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.718+0000 7f5e7ea00640 1 -- 172.21.15.119:0/3002908731 shutdown_connections 2024-04-25T10:17:34.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.718+0000 7f5e7ea00640 1 --2- 172.21.15.119:0/3002908731 >> v2:172.21.15.132:3300/0 conn(0x7f5e780b1be0 0x7f5e780cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.718+0000 7f5e7ea00640 1 --2- 172.21.15.119:0/3002908731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e780cc2d0 0x7f5e780cd830 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.718+0000 7f5e7ea00640 1 -- 172.21.15.119:0/3002908731 >> 172.21.15.119:0/3002908731 conn(0x7f5e780b2220 msgr2=0x7f5e780b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:34.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.718+0000 7f5e7ea00640 1 -- 172.21.15.119:0/3002908731 shutdown_connections 2024-04-25T10:17:34.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.718+0000 7f5e7ea00640 1 -- 172.21.15.119:0/3002908731 wait complete. 2024-04-25T10:17:34.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.720+0000 7f5e7ea00640 1 Processor -- start 2024-04-25T10:17:34.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.720+0000 7f5e7ea00640 1 -- start start 2024-04-25T10:17:34.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.721+0000 7f5e7ea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e780b1be0 0x7f5e78152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.721+0000 7f5e7ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e780cc2d0 0x7f5e7815b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.721+0000 7f5e7ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e7815bd00 0x7f5e780b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.721+0000 7f5e7ea00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780cb260 con 0x7f5e7815bd00 2024-04-25T10:17:34.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.721+0000 7f5e7ea00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780cd310 con 0x7f5e780b1be0 2024-04-25T10:17:34.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.721+0000 7f5e7ea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780cb3e0 con 0x7f5e780cc2d0 2024-04-25T10:17:34.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.722+0000 7f5e7ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e7815bd00 0x7f5e780b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.722+0000 7f5e7ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e7815bd00 0x7f5e780b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45684/0 (socket says 172.21.15.119:45684) 2024-04-25T10:17:34.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.722+0000 7f5e7ca00640 1 -- 172.21.15.119:0/1403337925 learned_addr learned my addr 172.21.15.119:0/1403337925 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:34.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.722+0000 7f5e6f400640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e780cc2d0 0x7f5e7815b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.722+0000 7f5e77e00640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e780b1be0 0x7f5e78152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.722+0000 7f5e7ca00640 1 -- 172.21.15.119:0/1403337925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e780cc2d0 msgr2=0x7f5e7815b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.722+0000 7f5e7ca00640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e780cc2d0 0x7f5e7815b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.722+0000 7f5e7ca00640 1 -- 172.21.15.119:0/1403337925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e780b1be0 msgr2=0x7f5e78152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.722+0000 7f5e7ca00640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e780b1be0 0x7f5e78152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.722+0000 7f5e7ca00640 1 -- 172.21.15.119:0/1403337925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e78152da0 con 0x7f5e7815bd00 2024-04-25T10:17:34.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.722+0000 7f5e77e00640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e780b1be0 0x7f5e78152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:34.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.722+0000 7f5e7ca00640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e7815bd00 0x7f5e780b3410 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000efc0 tx=0x7f5e7000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.723+0000 7f5e74c00640 1 -- 172.21.15.119:0/1403337925 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70018020 con 0x7f5e7815bd00 2024-04-25T10:17:34.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.723+0000 7f5e74c00640 1 -- 172.21.15.119:0/1403337925 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e70014480 con 0x7f5e7815bd00 2024-04-25T10:17:34.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.723+0000 7f5e74c00640 1 -- 172.21.15.119:0/1403337925 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70014870 con 0x7f5e7815bd00 2024-04-25T10:17:34.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.723+0000 7f5e7ea00640 1 -- 172.21.15.119:0/1403337925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e781f7680 con 0x7f5e7815bd00 2024-04-25T10:17:34.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.723+0000 7f5e7ea00640 1 -- 172.21.15.119:0/1403337925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e781f7c40 con 0x7f5e7815bd00 2024-04-25T10:17:34.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.724+0000 7f5e7ea00640 1 -- 172.21.15.119:0/1403337925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e780749b0 con 0x7f5e7815bd00 2024-04-25T10:17:34.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.725+0000 7f5e74c00640 1 -- 172.21.15.119:0/1403337925 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e70005d10 con 0x7f5e7815bd00 2024-04-25T10:17:34.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.726+0000 7f5e74c00640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e5406b520 0x7f5e5406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.727+0000 7f5e74c00640 1 -- 172.21.15.119:0/1403337925 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e7008acd0 con 0x7f5e7815bd00 2024-04-25T10:17:34.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.727+0000 7f5e77e00640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e5406b520 0x7f5e5406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.727+0000 7f5e77e00640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e5406b520 0x7f5e5406d9e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5e68034000 tx=0x7f5e6802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.731+0000 7f5e74c00640 1 -- 172.21.15.119:0/1403337925 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e70022030 con 0x7f5e7815bd00 2024-04-25T10:17:34.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.928+0000 7f5e7ea00640 1 -- 172.21.15.119:0/1403337925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5e780c6f50 con 0x7f5e7815bd00 2024-04-25T10:17:34.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.929+0000 7f5e74c00640 1 -- 172.21.15.119:0/1403337925 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v104) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5e70010040 con 0x7f5e7815bd00 2024-04-25T10:17:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:34.933 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2024-04-25T10:17:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: { 2024-04-25T10:17:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2024-04-25T10:17:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2024-04-25T10:17:34.933 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-04-25T10:17:34.933 INFO:teuthology.orchestra.run.smithi119.stdout:] 2024-04-25T10:17:34.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.932+0000 7f5e7ea00640 1 -- 172.21.15.119:0/1403337925 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e5406b520 msgr2=0x7f5e5406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.932+0000 7f5e7ea00640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e5406b520 0x7f5e5406d9e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5e68034000 tx=0x7f5e6802e000 comp rx=0 tx=0).stop 2024-04-25T10:17:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.932+0000 7f5e7ea00640 1 -- 172.21.15.119:0/1403337925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e7815bd00 msgr2=0x7f5e780b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.932+0000 7f5e7ea00640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e7815bd00 0x7f5e780b3410 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000efc0 tx=0x7f5e7000c490 comp rx=0 tx=0).stop 2024-04-25T10:17:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.932+0000 7f5e7ea00640 1 -- 172.21.15.119:0/1403337925 shutdown_connections 2024-04-25T10:17:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.932+0000 7f5e7ea00640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5e5406b520 0x7f5e5406d9e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.932+0000 7f5e7ea00640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e7815bd00 0x7f5e780b3410 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.932+0000 7f5e7ea00640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e780cc2d0 0x7f5e7815b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.932+0000 7f5e7ea00640 1 --2- 172.21.15.119:0/1403337925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e780b1be0 0x7f5e78152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.932+0000 7f5e7ea00640 1 -- 172.21.15.119:0/1403337925 >> 172.21.15.119:0/1403337925 conn(0x7f5e780b2220 msgr2=0x7f5e7815db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.932+0000 7f5e7ea00640 1 -- 172.21.15.119:0/1403337925 shutdown_connections 2024-04-25T10:17:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:34.932+0000 7f5e7ea00640 1 -- 172.21.15.119:0/1403337925 wait complete. 2024-04-25T10:17:34.958 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:17:34.959 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:17:35.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.166+0000 7fcd5ae00640 1 Processor -- start 2024-04-25T10:17:35.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.166+0000 7fcd5ae00640 1 -- start start 2024-04-25T10:17:35.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.167+0000 7fcd5ae00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcd54145260 0x7fcd540cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.167+0000 7fcd5ae00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd54071910 0x7fcd540cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.167+0000 7fcd5ae00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd540cc9a0 con 0x7fcd540cc490 2024-04-25T10:17:35.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.167+0000 7fcd5ae00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd540cea00 con 0x7fcd54071910 2024-04-25T10:17:35.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.167+0000 7fcd5ae00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd540ccc80 con 0x7fcd54145260 2024-04-25T10:17:35.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.168+0000 7fcd58e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd54071910 0x7fcd540cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.168+0000 7fcd53400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcd540cc490 0x7fcd54071200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50548/0 (socket says 172.21.15.119:50548) 2024-04-25T10:17:35.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.168+0000 7fcd53400640 1 -- 172.21.15.119:0/2970940147 learned_addr learned my addr 172.21.15.119:0/2970940147 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:35.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.168+0000 7fcd53e00640 1 --2- 172.21.15.119:0/2970940147 >> v2:172.21.15.132:3300/0 conn(0x7fcd54145260 0x7fcd540cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.168+0000 7fcd58e00640 1 -- 172.21.15.119:0/2970940147 >> v2:172.21.15.132:3300/0 conn(0x7fcd54145260 msgr2=0x7fcd540cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:35.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.168+0000 7fcd58e00640 1 --2- 172.21.15.119:0/2970940147 >> v2:172.21.15.132:3300/0 conn(0x7fcd54145260 0x7fcd540cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.168+0000 7fcd58e00640 1 -- 172.21.15.119:0/2970940147 >> v1:172.21.15.119:6789/0 conn(0x7fcd540cc490 legacy=0x7fcd54071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.168+0000 7fcd58e00640 1 -- 172.21.15.119:0/2970940147 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd540ccf60 con 0x7fcd54071910 2024-04-25T10:17:35.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.168+0000 7fcd52a00640 1 -- 172.21.15.119:0/2970940147 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290666463 0 0) 0x7fcd540cc9a0 con 0x7fcd540cc490 2024-04-25T10:17:35.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.169+0000 7fcd58e00640 1 --2- 172.21.15.119:0/2970940147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54071910 0x7fcd540cee90 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fcd5406d0b0 tx=0x7fcd4c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584043bae53da62d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.169+0000 7fcd52a00640 1 -- 172.21.15.119:0/2970940147 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c007740 con 0x7fcd54071910 2024-04-25T10:17:35.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.170+0000 7fcd52a00640 1 -- 172.21.15.119:0/2970940147 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c0078e0 con 0x7fcd54071910 2024-04-25T10:17:35.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.170+0000 7fcd52a00640 1 -- 172.21.15.119:0/2970940147 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c007cb0 con 0x7fcd54071910 2024-04-25T10:17:35.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.170+0000 7fcd5ae00640 1 -- 172.21.15.119:0/2970940147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54071910 msgr2=0x7fcd540cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.170+0000 7fcd5ae00640 1 --2- 172.21.15.119:0/2970940147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54071910 0x7fcd540cee90 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fcd5406d0b0 tx=0x7fcd4c02a680 comp rx=0 tx=0).stop 2024-04-25T10:17:35.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.170+0000 7fcd5ae00640 1 -- 172.21.15.119:0/2970940147 shutdown_connections 2024-04-25T10:17:35.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.170+0000 7fcd5ae00640 1 --2- 172.21.15.119:0/2970940147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54071910 0x7fcd540cee90 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.170+0000 7fcd5ae00640 1 --2- 172.21.15.119:0/2970940147 >> v2:172.21.15.132:3300/0 conn(0x7fcd54145260 0x7fcd540cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.170+0000 7fcd5ae00640 1 -- 172.21.15.119:0/2970940147 >> 172.21.15.119:0/2970940147 conn(0x7fcd5406aba0 msgr2=0x7fcd5406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:35.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.170+0000 7fcd5ae00640 1 -- 172.21.15.119:0/2970940147 shutdown_connections 2024-04-25T10:17:35.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.170+0000 7fcd5ae00640 1 -- 172.21.15.119:0/2970940147 wait complete. 2024-04-25T10:17:35.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.172+0000 7fcd5ae00640 1 Processor -- start 2024-04-25T10:17:35.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.172+0000 7fcd5ae00640 1 -- start start 2024-04-25T10:17:35.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.173+0000 7fcd5ae00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd54071910 0x7fcd5415e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.173+0000 7fcd5ae00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd540cc490 0x7fcd5413e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.173+0000 7fcd5ae00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54145260 0x7fcd5413ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.173+0000 7fcd5ae00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5415c270 con 0x7fcd54145260 2024-04-25T10:17:35.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.173+0000 7fcd5ae00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd54145640 con 0x7fcd540cc490 2024-04-25T10:17:35.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.173+0000 7fcd5ae00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5415c3f0 con 0x7fcd54071910 2024-04-25T10:17:35.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.174+0000 7fcd58e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54145260 0x7fcd5413ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.174+0000 7fcd58e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54145260 0x7fcd5413ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45714/0 (socket says 172.21.15.119:45714) 2024-04-25T10:17:35.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.174+0000 7fcd58e00640 1 -- 172.21.15.119:0/3713597114 learned_addr learned my addr 172.21.15.119:0/3713597114 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:35.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.174+0000 7fcd53e00640 1 --2- 172.21.15.119:0/3713597114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd54071910 0x7fcd5415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.174+0000 7fcd53400640 1 --2- 172.21.15.119:0/3713597114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd540cc490 0x7fcd5413e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.174+0000 7fcd58e00640 1 -- 172.21.15.119:0/3713597114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd54071910 msgr2=0x7fcd5415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.174+0000 7fcd58e00640 1 --2- 172.21.15.119:0/3713597114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd54071910 0x7fcd5415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.174+0000 7fcd58e00640 1 -- 172.21.15.119:0/3713597114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd540cc490 msgr2=0x7fcd5413e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.174+0000 7fcd58e00640 1 --2- 172.21.15.119:0/3713597114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd540cc490 0x7fcd5413e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.174+0000 7fcd58e00640 1 -- 172.21.15.119:0/3713597114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5413f3f0 con 0x7fcd54145260 2024-04-25T10:17:35.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.174+0000 7fcd58e00640 1 --2- 172.21.15.119:0/3713597114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54145260 0x7fcd5413ede0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c02ac70 tx=0x7fcd4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.175+0000 7fcd4be00640 1 -- 172.21.15.119:0/3713597114 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c040070 con 0x7fcd54145260 2024-04-25T10:17:35.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.175+0000 7fcd4be00640 1 -- 172.21.15.119:0/3713597114 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c031bc0 con 0x7fcd54145260 2024-04-25T10:17:35.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.175+0000 7fcd4be00640 1 -- 172.21.15.119:0/3713597114 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c045460 con 0x7fcd54145260 2024-04-25T10:17:35.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.175+0000 7fcd5ae00640 1 -- 172.21.15.119:0/3713597114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5413f6d0 con 0x7fcd54145260 2024-04-25T10:17:35.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.175+0000 7fcd5ae00640 1 -- 172.21.15.119:0/3713597114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5413fc90 con 0x7fcd54145260 2024-04-25T10:17:35.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.176+0000 7fcd5ae00640 1 -- 172.21.15.119:0/3713597114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd540750b0 con 0x7fcd54145260 2024-04-25T10:17:35.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.177+0000 7fcd4be00640 1 -- 172.21.15.119:0/3713597114 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd4c039040 con 0x7fcd54145260 2024-04-25T10:17:35.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.178+0000 7fcd4be00640 1 --2- 172.21.15.119:0/3713597114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd3406b4d0 0x7fcd3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.178+0000 7fcd4be00640 1 -- 172.21.15.119:0/3713597114 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcd4c0ab900 con 0x7fcd54145260 2024-04-25T10:17:35.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.178+0000 7fcd53e00640 1 --2- 172.21.15.119:0/3713597114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd3406b4d0 0x7fcd3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.179+0000 7fcd53e00640 1 --2- 172.21.15.119:0/3713597114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd3406b4d0 0x7fcd3406d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcd44006fd0 tx=0x7fcd44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.182+0000 7fcd4be00640 1 -- 172.21.15.119:0/3713597114 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd4c07a350 con 0x7fcd54145260 2024-04-25T10:17:35.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:35.370+0000 7fcd5ae00640 1 -- 172.21.15.119:0/3713597114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcd540cbca0 con 0x7fcd54145260 2024-04-25T10:17:36.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.146+0000 7fcd4be00640 1 -- 172.21.15.119:0/3713597114 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v105) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcd4c07e3f0 con 0x7fcd54145260 2024-04-25T10:17:36.150 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2024-04-25T10:17:36.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.150+0000 7fcd5ae00640 1 -- 172.21.15.119:0/3713597114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd3406b4d0 msgr2=0x7fcd3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.150+0000 7fcd5ae00640 1 --2- 172.21.15.119:0/3713597114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd3406b4d0 0x7fcd3406d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcd44006fd0 tx=0x7fcd44008040 comp rx=0 tx=0).stop 2024-04-25T10:17:36.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.150+0000 7fcd5ae00640 1 -- 172.21.15.119:0/3713597114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54145260 msgr2=0x7fcd5413ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.150+0000 7fcd5ae00640 1 --2- 172.21.15.119:0/3713597114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54145260 0x7fcd5413ede0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c02ac70 tx=0x7fcd4c004a40 comp rx=0 tx=0).stop 2024-04-25T10:17:36.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.150+0000 7fcd5ae00640 1 -- 172.21.15.119:0/3713597114 shutdown_connections 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.150+0000 7fcd5ae00640 1 --2- 172.21.15.119:0/3713597114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd3406b4d0 0x7fcd3406d990 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.150+0000 7fcd5ae00640 1 --2- 172.21.15.119:0/3713597114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd54145260 0x7fcd5413ede0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.150+0000 7fcd5ae00640 1 --2- 172.21.15.119:0/3713597114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd540cc490 0x7fcd5413e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.150+0000 7fcd5ae00640 1 --2- 172.21.15.119:0/3713597114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd54071910 0x7fcd5415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.150+0000 7fcd5ae00640 1 -- 172.21.15.119:0/3713597114 >> 172.21.15.119:0/3713597114 conn(0x7fcd5406aba0 msgr2=0x7fcd54071200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.151+0000 7fcd5ae00640 1 -- 172.21.15.119:0/3713597114 shutdown_connections 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.151+0000 7fcd5ae00640 1 -- 172.21.15.119:0/3713597114 wait complete. 2024-04-25T10:17:36.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:17:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.364+0000 7f9937e00640 1 Processor -- start 2024-04-25T10:17:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.365+0000 7f9937e00640 1 -- start start 2024-04-25T10:17:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.366+0000 7f9937e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f993815daf0 0x7f99380b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.366+0000 7f9937e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99380ce760 con 0x7f99380b2c90 2024-04-25T10:17:36.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.366+0000 7f9937e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99380cea40 con 0x7f99380d0b80 2024-04-25T10:17:36.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.366+0000 7f9937e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99380d17b0 con 0x7f993815daf0 2024-04-25T10:17:36.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.366+0000 7f992e000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f99380b2c90 0x7f99380d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50556/0 (socket says 172.21.15.119:50556) 2024-04-25T10:17:36.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.367+0000 7f992e000640 1 -- 172.21.15.119:0/2124346026 learned_addr learned my addr 172.21.15.119:0/2124346026 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:36.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.367+0000 7f9936a00640 1 --2- 172.21.15.119:0/2124346026 >> v2:172.21.15.132:3300/0 conn(0x7f993815daf0 0x7f99380b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.367+0000 7f9936000640 1 -- 172.21.15.119:0/2124346026 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 935590258 0 0) 0x7f99380ce760 con 0x7f99380b2c90 2024-04-25T10:17:36.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.367+0000 7f9936000640 1 -- 172.21.15.119:0/2124346026 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f991c003520 con 0x7f99380b2c90 2024-04-25T10:17:36.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.367+0000 7f9936000640 1 -- 172.21.15.119:0/2124346026 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1197957273 0 0) 0x7f991c003520 con 0x7f99380b2c90 2024-04-25T10:17:36.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.367+0000 7f9936000640 1 -- 172.21.15.119:0/2124346026 >> v2:172.21.15.132:3300/0 conn(0x7f993815daf0 msgr2=0x7f99380b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:36.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.367+0000 7f9936000640 1 --2- 172.21.15.119:0/2124346026 >> v2:172.21.15.132:3300/0 conn(0x7f993815daf0 0x7f99380b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.367+0000 7f9936000640 1 -- 172.21.15.119:0/2124346026 >> v1:172.21.15.132:6789/0 conn(0x7f99380d0b80 legacy=0x7f99380d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.367+0000 7f9936000640 1 -- 172.21.15.119:0/2124346026 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99380ced20 con 0x7f99380b2c90 2024-04-25T10:17:36.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.367+0000 7f9936000640 1 -- 172.21.15.119:0/2124346026 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f9920001670 con 0x7f99380b2c90 2024-04-25T10:17:36.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.367+0000 7f9936000640 1 -- 172.21.15.119:0/2124346026 >> v1:172.21.15.119:6789/0 conn(0x7f99380b2c90 legacy=0x7f99380d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.368+0000 7f9936000640 1 --2- 172.21.15.119:0/2124346026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f991c0039b0 0x7f991c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.368+0000 7f9936000640 1 --2- 172.21.15.119:0/2124346026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f991c004800 0x7f991c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.368+0000 7f9936000640 1 --2- 172.21.15.119:0/2124346026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c0051c0 0x7f991c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.368+0000 7f9936000640 1 -- 172.21.15.119:0/2124346026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99380d17b0 con 0x7f991c0051c0 2024-04-25T10:17:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.368+0000 7f9936000640 1 -- 172.21.15.119:0/2124346026 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c00c070 con 0x7f991c004800 2024-04-25T10:17:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.368+0000 7f9936000640 1 -- 172.21.15.119:0/2124346026 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c00c270 con 0x7f991c0039b0 2024-04-25T10:17:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.368+0000 7f9937400640 1 --2- 172.21.15.119:0/2124346026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c0051c0 0x7f991c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.368+0000 7f9937400640 1 -- 172.21.15.119:0/2124346026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f991c0039b0 msgr2=0x7f991c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.368+0000 7f9937400640 1 --2- 172.21.15.119:0/2124346026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f991c0039b0 0x7f991c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.368+0000 7f9937400640 1 -- 172.21.15.119:0/2124346026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f991c004800 msgr2=0x7f991c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.368+0000 7f9937400640 1 --2- 172.21.15.119:0/2124346026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f991c004800 0x7f991c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.368+0000 7f9937400640 1 -- 172.21.15.119:0/2124346026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c003520 con 0x7f991c0051c0 2024-04-25T10:17:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.369+0000 7f9937400640 1 -- 172.21.15.119:0/2124346026 reap_dead start 2024-04-25T10:17:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.369+0000 7f9937400640 1 --2- 172.21.15.119:0/2124346026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c0051c0 0x7f991c00b9c0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f993808c3e0 tx=0x7f991c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5e94a6c36b8ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.369+0000 7f9936000640 1 -- 172.21.15.119:0/2124346026 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930001370 con 0x7f991c0051c0 2024-04-25T10:17:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.369+0000 7f9936000640 1 -- 172.21.15.119:0/2124346026 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99300052d0 con 0x7f991c0051c0 2024-04-25T10:17:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.369+0000 7f9936000640 1 -- 172.21.15.119:0/2124346026 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993002ab90 con 0x7f991c0051c0 2024-04-25T10:17:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.370+0000 7f9937e00640 1 -- 172.21.15.119:0/2124346026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c0051c0 msgr2=0x7f991c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.370+0000 7f9937e00640 1 --2- 172.21.15.119:0/2124346026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c0051c0 0x7f991c00b9c0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f993808c3e0 tx=0x7f991c005830 comp rx=0 tx=0).stop 2024-04-25T10:17:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.370+0000 7f9937e00640 1 -- 172.21.15.119:0/2124346026 shutdown_connections 2024-04-25T10:17:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.370+0000 7f9937e00640 1 --2- 172.21.15.119:0/2124346026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991c0051c0 0x7f991c00b9c0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.370+0000 7f9937e00640 1 -- 172.21.15.119:0/2124346026 >> 172.21.15.119:0/2124346026 conn(0x7f99380b55c0 msgr2=0x7f99380b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:36.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.370+0000 7f9937e00640 1 -- 172.21.15.119:0/2124346026 shutdown_connections 2024-04-25T10:17:36.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.370+0000 7f9937e00640 1 -- 172.21.15.119:0/2124346026 wait complete. 2024-04-25T10:17:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.372+0000 7f9937e00640 1 Processor -- start 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.372+0000 7f9937e00640 1 -- start start 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.373+0000 7f9937e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f991c0051c0 0x7f99380cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.373+0000 7f9937e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99380b7e00 0x7f99381607b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.373+0000 7f9937e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99380b8280 0x7f9938160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.373+0000 7f9937e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99380ce310 con 0x7f99380b7e00 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.373+0000 7f9937e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99380cf000 con 0x7f99380b8280 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.373+0000 7f9937e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99380ce490 con 0x7f991c0051c0 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.373+0000 7f992e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99380b7e00 0x7f99381607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.373+0000 7f992e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99380b7e00 0x7f99381607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45726/0 (socket says 172.21.15.119:45726) 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.373+0000 7f992e000640 1 -- 172.21.15.119:0/3251302568 learned_addr learned my addr 172.21.15.119:0/3251302568 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.374+0000 7f992e000640 1 -- 172.21.15.119:0/3251302568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f991c0051c0 msgr2=0x7f99380cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.374+0000 7f992e000640 1 --2- 172.21.15.119:0/3251302568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f991c0051c0 0x7f99380cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.374+0000 7f992e000640 1 -- 172.21.15.119:0/3251302568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99380b8280 msgr2=0x7f9938160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.374+0000 7f992e000640 1 --2- 172.21.15.119:0/3251302568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99380b8280 0x7f9938160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.374+0000 7f992e000640 1 -- 172.21.15.119:0/3251302568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99381549e0 con 0x7f99380b7e00 2024-04-25T10:17:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.374+0000 7f992e000640 1 --2- 172.21.15.119:0/3251302568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99380b7e00 0x7f99381607b0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f992000c700 tx=0x7f992000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.374+0000 7f992f400640 1 -- 172.21.15.119:0/3251302568 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920018070 con 0x7f99380b7e00 2024-04-25T10:17:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.374+0000 7f992f400640 1 -- 172.21.15.119:0/3251302568 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9920007500 con 0x7f99380b7e00 2024-04-25T10:17:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.374+0000 7f992f400640 1 -- 172.21.15.119:0/3251302568 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920004a20 con 0x7f99380b7e00 2024-04-25T10:17:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.374+0000 7f9937e00640 1 -- 172.21.15.119:0/3251302568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9938154cc0 con 0x7f99380b7e00 2024-04-25T10:17:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.374+0000 7f9937e00640 1 -- 172.21.15.119:0/3251302568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9938155280 con 0x7f99380b7e00 2024-04-25T10:17:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.375+0000 7f9937e00640 1 -- 172.21.15.119:0/3251302568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9938067490 con 0x7f99380b7e00 2024-04-25T10:17:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.376+0000 7f992f400640 1 -- 172.21.15.119:0/3251302568 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9920004bc0 con 0x7f99380b7e00 2024-04-25T10:17:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.377+0000 7f992f400640 1 --2- 172.21.15.119:0/3251302568 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f991406b4d0 0x7f991406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.377+0000 7f992f400640 1 -- 172.21.15.119:0/3251302568 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9920089590 con 0x7f99380b7e00 2024-04-25T10:17:36.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.378+0000 7f9936a00640 1 --2- 172.21.15.119:0/3251302568 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f991406b4d0 0x7f991406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.378+0000 7f9936a00640 1 --2- 172.21.15.119:0/3251302568 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f991406b4d0 0x7f991406d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9928004b30 tx=0x7f992800d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.383+0000 7f992f400640 1 -- 172.21.15.119:0/3251302568 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9938067490 con 0x7f99380b7e00 2024-04-25T10:17:36.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:36.580+0000 7f9937e00640 1 -- 172.21.15.119:0/3251302568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f99380749b0 con 0x7f99380b7e00 2024-04-25T10:17:37.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.152+0000 7f992f400640 1 -- 172.21.15.119:0/3251302568 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v106) v1 ==== 170+0+0 (secure 0 0 0) 0x7f99200585e0 con 0x7f99380b7e00 2024-04-25T10:17:37.156 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2024-04-25T10:17:37.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.156+0000 7f9937e00640 1 -- 172.21.15.119:0/3251302568 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f991406b4d0 msgr2=0x7f991406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.156+0000 7f9937e00640 1 --2- 172.21.15.119:0/3251302568 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f991406b4d0 0x7f991406d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9928004b30 tx=0x7f992800d040 comp rx=0 tx=0).stop 2024-04-25T10:17:37.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.156+0000 7f9937e00640 1 -- 172.21.15.119:0/3251302568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99380b7e00 msgr2=0x7f99381607b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.156+0000 7f9937e00640 1 --2- 172.21.15.119:0/3251302568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99380b7e00 0x7f99381607b0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f992000c700 tx=0x7f992000cbc0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.157+0000 7f9937e00640 1 -- 172.21.15.119:0/3251302568 shutdown_connections 2024-04-25T10:17:37.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.157+0000 7f9937e00640 1 --2- 172.21.15.119:0/3251302568 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f991406b4d0 0x7f991406d990 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.157+0000 7f9937e00640 1 --2- 172.21.15.119:0/3251302568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99380b8280 0x7f9938160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.157+0000 7f9937e00640 1 --2- 172.21.15.119:0/3251302568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99380b7e00 0x7f99381607b0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.157+0000 7f9937e00640 1 --2- 172.21.15.119:0/3251302568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f991c0051c0 0x7f99380cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.157+0000 7f9937e00640 1 -- 172.21.15.119:0/3251302568 >> 172.21.15.119:0/3251302568 conn(0x7f99380b55c0 msgr2=0x7f99380d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.157+0000 7f9937e00640 1 -- 172.21.15.119:0/3251302568 shutdown_connections 2024-04-25T10:17:37.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.157+0000 7f9937e00640 1 -- 172.21.15.119:0/3251302568 wait complete. 2024-04-25T10:17:37.194 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:17:37.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.365+0000 7f418fe00640 1 Processor -- start 2024-04-25T10:17:37.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.365+0000 7f418fe00640 1 -- start start 2024-04-25T10:17:37.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.367+0000 7f418fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f41901450e0 0x7f419006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.367+0000 7f418fe00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f419015f400 con 0x7f41900cc9e0 2024-04-25T10:17:37.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.367+0000 7f418fe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f419015f6e0 con 0x7f419006b0e0 2024-04-25T10:17:37.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.367+0000 7f418fe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41901454c0 con 0x7f41901450e0 2024-04-25T10:17:37.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.367+0000 7f418f400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f41900cc9e0 0x7f41900b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50572/0 (socket says 172.21.15.119:50572) 2024-04-25T10:17:37.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.367+0000 7f418f400640 1 -- 172.21.15.119:0/1774431725 learned_addr learned my addr 172.21.15.119:0/1774431725 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:37.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418ea00640 1 --2- 172.21.15.119:0/1774431725 >> v2:172.21.15.132:3300/0 conn(0x7f41901450e0 0x7f419006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 -- 172.21.15.119:0/1774431725 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131616009 0 0) 0x7f419015f400 con 0x7f41900cc9e0 2024-04-25T10:17:37.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 -- 172.21.15.119:0/1774431725 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4174003520 con 0x7f41900cc9e0 2024-04-25T10:17:37.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 -- 172.21.15.119:0/1774431725 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3654487734 0 0) 0x7f4174003520 con 0x7f41900cc9e0 2024-04-25T10:17:37.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 -- 172.21.15.119:0/1774431725 >> v2:172.21.15.132:3300/0 conn(0x7f41901450e0 msgr2=0x7f419006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:37.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 --2- 172.21.15.119:0/1774431725 >> v2:172.21.15.132:3300/0 conn(0x7f41901450e0 0x7f419006aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 -- 172.21.15.119:0/1774431725 >> v1:172.21.15.132:6789/0 conn(0x7f419006b0e0 legacy=0x7f41900cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 -- 172.21.15.119:0/1774431725 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419015f9c0 con 0x7f41900cc9e0 2024-04-25T10:17:37.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 -- 172.21.15.119:0/1774431725 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f4188001670 con 0x7f41900cc9e0 2024-04-25T10:17:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 -- 172.21.15.119:0/1774431725 >> v1:172.21.15.119:6789/0 conn(0x7f41900cc9e0 legacy=0x7f41900b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 --2- 172.21.15.119:0/1774431725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41740039b0 0x7f4174003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 --2- 172.21.15.119:0/1774431725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4174004800 0x7f4174004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 --2- 172.21.15.119:0/1774431725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41740052e0 0x7f417400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 -- 172.21.15.119:0/1774431725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417400c180 con 0x7f41740052e0 2024-04-25T10:17:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 -- 172.21.15.119:0/1774431725 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417400c330 con 0x7f41740039b0 2024-04-25T10:17:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.368+0000 7f418e000640 1 -- 172.21.15.119:0/1774431725 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417400c530 con 0x7f4174004800 2024-04-25T10:17:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.369+0000 7f418f400640 1 --2- 172.21.15.119:0/1774431725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41740052e0 0x7f417400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.369+0000 7f418ea00640 1 --2- 172.21.15.119:0/1774431725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41740039b0 0x7f4174003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.369+0000 7f4186000640 1 --2- 172.21.15.119:0/1774431725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4174004800 0x7f4174004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.369+0000 7f418f400640 1 -- 172.21.15.119:0/1774431725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4174004800 msgr2=0x7f4174004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.369+0000 7f418f400640 1 --2- 172.21.15.119:0/1774431725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4174004800 0x7f4174004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.369+0000 7f418f400640 1 -- 172.21.15.119:0/1774431725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41740039b0 msgr2=0x7f4174003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:37.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.369+0000 7f418f400640 1 --2- 172.21.15.119:0/1774431725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41740039b0 0x7f4174003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.369+0000 7f418f400640 1 -- 172.21.15.119:0/1774431725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4174003520 con 0x7f41740052e0 2024-04-25T10:17:37.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.369+0000 7f418ea00640 1 --2- 172.21.15.119:0/1774431725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41740039b0 0x7f4174003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:37.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.370+0000 7f418f400640 1 -- 172.21.15.119:0/1774431725 reap_dead start 2024-04-25T10:17:37.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.370+0000 7f418f400640 1 --2- 172.21.15.119:0/1774431725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41740052e0 0x7f417400b9c0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f418800c210 tx=0x7f4174005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e01512678e9e1b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.370+0000 7f418e000640 1 -- 172.21.15.119:0/1774431725 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188007d10 con 0x7f41740052e0 2024-04-25T10:17:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.371+0000 7f418e000640 1 -- 172.21.15.119:0/1774431725 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41880058d0 con 0x7f41740052e0 2024-04-25T10:17:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.371+0000 7f418e000640 1 -- 172.21.15.119:0/1774431725 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188005ca0 con 0x7f41740052e0 2024-04-25T10:17:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.371+0000 7f418fe00640 1 -- 172.21.15.119:0/1774431725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41740052e0 msgr2=0x7f417400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.371+0000 7f418fe00640 1 --2- 172.21.15.119:0/1774431725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41740052e0 0x7f417400b9c0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f418800c210 tx=0x7f4174005830 comp rx=0 tx=0).stop 2024-04-25T10:17:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.371+0000 7f418fe00640 1 -- 172.21.15.119:0/1774431725 shutdown_connections 2024-04-25T10:17:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.371+0000 7f418fe00640 1 --2- 172.21.15.119:0/1774431725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41740052e0 0x7f417400b9c0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.371+0000 7f418fe00640 1 -- 172.21.15.119:0/1774431725 >> 172.21.15.119:0/1774431725 conn(0x7f4190066ff0 msgr2=0x7f41900b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.371+0000 7f418fe00640 1 -- 172.21.15.119:0/1774431725 shutdown_connections 2024-04-25T10:17:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.371+0000 7f418fe00640 1 -- 172.21.15.119:0/1774431725 wait complete. 2024-04-25T10:17:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.373+0000 7f418fe00640 1 Processor -- start 2024-04-25T10:17:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.373+0000 7f418fe00640 1 -- start start 2024-04-25T10:17:37.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.374+0000 7f418fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41740052e0 0x7f419015b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.374+0000 7f418fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4190143590 0x7f41900b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.374+0000 7f418fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4190143a10 0x7f41900b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.374+0000 7f418fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41900cce00 con 0x7f41740052e0 2024-04-25T10:17:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.374+0000 7f418fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419006aba0 con 0x7f4190143590 2024-04-25T10:17:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.374+0000 7f418fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419006ad20 con 0x7f4190143a10 2024-04-25T10:17:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.374+0000 7f418ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41740052e0 0x7f419015b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.374+0000 7f418ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41740052e0 0x7f419015b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45750/0 (socket says 172.21.15.119:45750) 2024-04-25T10:17:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.374+0000 7f418ea00640 1 -- 172.21.15.119:0/1668782898 learned_addr learned my addr 172.21.15.119:0/1668782898 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.375+0000 7f418ea00640 1 -- 172.21.15.119:0/1668782898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4190143a10 msgr2=0x7f41900b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.375+0000 7f418f400640 1 --2- 172.21.15.119:0/1668782898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4190143a10 0x7f41900b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.375+0000 7f4186000640 1 --2- 172.21.15.119:0/1668782898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4190143590 0x7f41900b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.375+0000 7f418ea00640 1 --2- 172.21.15.119:0/1668782898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4190143a10 0x7f41900b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.375+0000 7f418ea00640 1 -- 172.21.15.119:0/1668782898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4190143590 msgr2=0x7f41900b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.375+0000 7f418ea00640 1 --2- 172.21.15.119:0/1668782898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4190143590 0x7f41900b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.375+0000 7f418ea00640 1 -- 172.21.15.119:0/1668782898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419006d080 con 0x7f41740052e0 2024-04-25T10:17:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.375+0000 7f418ea00640 1 --2- 172.21.15.119:0/1668782898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41740052e0 0x7f419015b090 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f4180007f30 tx=0x7f4180010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.375+0000 7f4187400640 1 -- 172.21.15.119:0/1668782898 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418001a030 con 0x7f41740052e0 2024-04-25T10:17:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.375+0000 7f4187400640 1 -- 172.21.15.119:0/1668782898 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4180003990 con 0x7f41740052e0 2024-04-25T10:17:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.376+0000 7f418fe00640 1 -- 172.21.15.119:0/1668782898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419006d360 con 0x7f41740052e0 2024-04-25T10:17:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.376+0000 7f4187400640 1 -- 172.21.15.119:0/1668782898 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180003d60 con 0x7f41740052e0 2024-04-25T10:17:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.376+0000 7f418fe00640 1 -- 172.21.15.119:0/1668782898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419006d920 con 0x7f41740052e0 2024-04-25T10:17:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.377+0000 7f418fe00640 1 -- 172.21.15.119:0/1668782898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41900749b0 con 0x7f41740052e0 2024-04-25T10:17:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.378+0000 7f4187400640 1 -- 172.21.15.119:0/1668782898 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4180005d10 con 0x7f41740052e0 2024-04-25T10:17:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.379+0000 7f4187400640 1 --2- 172.21.15.119:0/1668782898 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f416c06b4d0 0x7f416c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.379+0000 7f4187400640 1 -- 172.21.15.119:0/1668782898 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f418008b160 con 0x7f41740052e0 2024-04-25T10:17:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.379+0000 7f4186000640 1 --2- 172.21.15.119:0/1668782898 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f416c06b4d0 0x7f416c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.380+0000 7f4186000640 1 --2- 172.21.15.119:0/1668782898 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f416c06b4d0 0x7f416c06d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4178007fd0 tx=0x7f4178009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.384+0000 7f4187400640 1 -- 172.21.15.119:0/1668782898 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4180024070 con 0x7f41740052e0 2024-04-25T10:17:37.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:37.615+0000 7f418fe00640 1 -- 172.21.15.119:0/1668782898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f41900b1430 con 0x7f41740052e0 2024-04-25T10:17:38.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.175+0000 7f4187400640 1 -- 172.21.15.119:0/1668782898 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v95) v1 ==== 174+0+0 (secure 0 0 0) 0x7f418005a0d0 con 0x7f41740052e0 2024-04-25T10:17:38.179 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 16 and data pool 17 2024-04-25T10:17:38.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.180+0000 7f418fe00640 1 -- 172.21.15.119:0/1668782898 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f416c06b4d0 msgr2=0x7f416c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.180+0000 7f418fe00640 1 --2- 172.21.15.119:0/1668782898 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f416c06b4d0 0x7f416c06d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4178007fd0 tx=0x7f4178009040 comp rx=0 tx=0).stop 2024-04-25T10:17:38.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.180+0000 7f418fe00640 1 -- 172.21.15.119:0/1668782898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41740052e0 msgr2=0x7f419015b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.180+0000 7f418fe00640 1 --2- 172.21.15.119:0/1668782898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41740052e0 0x7f419015b090 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f4180007f30 tx=0x7f4180010650 comp rx=0 tx=0).stop 2024-04-25T10:17:38.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.181+0000 7f418fe00640 1 -- 172.21.15.119:0/1668782898 shutdown_connections 2024-04-25T10:17:38.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.181+0000 7f418fe00640 1 --2- 172.21.15.119:0/1668782898 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f416c06b4d0 0x7f416c06d990 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.181+0000 7f418fe00640 1 --2- 172.21.15.119:0/1668782898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4190143a10 0x7f41900b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.181+0000 7f418fe00640 1 --2- 172.21.15.119:0/1668782898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4190143590 0x7f41900b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.181+0000 7f418fe00640 1 --2- 172.21.15.119:0/1668782898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41740052e0 0x7f419015b090 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.181+0000 7f418fe00640 1 -- 172.21.15.119:0/1668782898 >> 172.21.15.119:0/1668782898 conn(0x7f4190066ff0 msgr2=0x7f4190071050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.182+0000 7f418fe00640 1 -- 172.21.15.119:0/1668782898 shutdown_connections 2024-04-25T10:17:38.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.182+0000 7f418fe00640 1 -- 172.21.15.119:0/1668782898 wait complete. 2024-04-25T10:17:38.226 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:38.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.393+0000 7f9277e00640 1 Processor -- start 2024-04-25T10:17:38.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.393+0000 7f9277e00640 1 -- start start 2024-04-25T10:17:38.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.395+0000 7f9277e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9270071190 0x7f92700ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.395+0000 7f9277e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92700b3730 0x7f927015e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.395+0000 7f9277e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927006d550 con 0x7f9270071570 2024-04-25T10:17:38.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.395+0000 7f9277e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92700aeee0 con 0x7f9270071190 2024-04-25T10:17:38.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.395+0000 7f9277e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927006d830 con 0x7f92700b3730 2024-04-25T10:17:38.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.395+0000 7f9275400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9270071190 0x7f92700ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.395+0000 7f9275400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9270071190 0x7f92700ae9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45764/0 (socket says 172.21.15.119:45764) 2024-04-25T10:17:38.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.395+0000 7f9275400640 1 -- 172.21.15.119:0/3337642300 learned_addr learned my addr 172.21.15.119:0/3337642300 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:38.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.396+0000 7f9275400640 1 -- 172.21.15.119:0/3337642300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92700b3730 msgr2=0x7f927015e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.396+0000 7f9275e00640 1 --2- 172.21.15.119:0/3337642300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92700b3730 0x7f927015e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.396+0000 7f9275400640 1 --2- 172.21.15.119:0/3337642300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92700b3730 0x7f927015e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.396+0000 7f9275400640 1 -- 172.21.15.119:0/3337642300 >> v1:172.21.15.132:6789/0 conn(0x7f9270071570 legacy=0x7f927015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.396+0000 7f9275400640 1 -- 172.21.15.119:0/3337642300 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927006db10 con 0x7f9270071190 2024-04-25T10:17:38.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.397+0000 7f9275400640 1 --2- 172.21.15.119:0/3337642300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9270071190 0x7f92700ae9a0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f92640098a0 tx=0x7f926402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363b684593886f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.397+0000 7f926fe00640 1 -- 172.21.15.119:0/3337642300 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264037070 con 0x7f9270071190 2024-04-25T10:17:38.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.397+0000 7f926fe00640 1 -- 172.21.15.119:0/3337642300 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9264032440 con 0x7f9270071190 2024-04-25T10:17:38.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.397+0000 7f926fe00640 1 -- 172.21.15.119:0/3337642300 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264032810 con 0x7f9270071190 2024-04-25T10:17:38.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.398+0000 7f9277e00640 1 -- 172.21.15.119:0/3337642300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9270071190 msgr2=0x7f92700ae9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.398+0000 7f9277e00640 1 --2- 172.21.15.119:0/3337642300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9270071190 0x7f92700ae9a0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f92640098a0 tx=0x7f926402a250 comp rx=0 tx=0).stop 2024-04-25T10:17:38.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.398+0000 7f9277e00640 1 -- 172.21.15.119:0/3337642300 shutdown_connections 2024-04-25T10:17:38.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.398+0000 7f9277e00640 1 --2- 172.21.15.119:0/3337642300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92700b3730 0x7f927015e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.398+0000 7f9277e00640 1 --2- 172.21.15.119:0/3337642300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9270071190 0x7f92700ae9a0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.398+0000 7f9277e00640 1 -- 172.21.15.119:0/3337642300 >> 172.21.15.119:0/3337642300 conn(0x7f927006aba0 msgr2=0x7f927006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.398+0000 7f9277e00640 1 -- 172.21.15.119:0/3337642300 shutdown_connections 2024-04-25T10:17:38.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.398+0000 7f9277e00640 1 -- 172.21.15.119:0/3337642300 wait complete. 2024-04-25T10:17:38.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.399+0000 7f9277e00640 1 Processor -- start 2024-04-25T10:17:38.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.400+0000 7f9277e00640 1 -- start start 2024-04-25T10:17:38.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.400+0000 7f9277e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9270071190 0x7f9270150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.400+0000 7f9277e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9270071570 0x7f927014a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.400+0000 7f9277e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92700b3730 0x7f927014f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.400+0000 7f9277e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9270071a00 con 0x7f9270071190 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.400+0000 7f9277e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927006e0d0 con 0x7f92700b3730 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.400+0000 7f9277e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927006e8b0 con 0x7f9270071570 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.401+0000 7f9275400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9270071190 0x7f9270150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.401+0000 7f9275400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9270071190 0x7f9270150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45780/0 (socket says 172.21.15.119:45780) 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.401+0000 7f9275400640 1 -- 172.21.15.119:0/1957705935 learned_addr learned my addr 172.21.15.119:0/1957705935 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.401+0000 7f9275400640 1 -- 172.21.15.119:0/1957705935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9270071570 msgr2=0x7f927014a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.401+0000 7f9275400640 1 --2- 172.21.15.119:0/1957705935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9270071570 0x7f927014a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.401+0000 7f9275400640 1 -- 172.21.15.119:0/1957705935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92700b3730 msgr2=0x7f927014f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.401+0000 7f9275400640 1 --2- 172.21.15.119:0/1957705935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92700b3730 0x7f927014f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.401+0000 7f9275400640 1 -- 172.21.15.119:0/1957705935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9270136350 con 0x7f9270071190 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.401+0000 7f9275400640 1 --2- 172.21.15.119:0/1957705935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9270071190 0x7f9270150f50 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f92640098a0 tx=0x7f92640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.402+0000 7f926d600640 1 -- 172.21.15.119:0/1957705935 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264037040 con 0x7f9270071190 2024-04-25T10:17:38.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.402+0000 7f926d600640 1 -- 172.21.15.119:0/1957705935 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9264004230 con 0x7f9270071190 2024-04-25T10:17:38.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.402+0000 7f926d600640 1 -- 172.21.15.119:0/1957705935 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264030620 con 0x7f9270071190 2024-04-25T10:17:38.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.402+0000 7f9277e00640 1 -- 172.21.15.119:0/1957705935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9270136630 con 0x7f9270071190 2024-04-25T10:17:38.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.402+0000 7f9277e00640 1 -- 172.21.15.119:0/1957705935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9270136bf0 con 0x7f9270071190 2024-04-25T10:17:38.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.403+0000 7f9277e00640 1 -- 172.21.15.119:0/1957705935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92700749b0 con 0x7f9270071190 2024-04-25T10:17:38.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.404+0000 7f926d600640 1 -- 172.21.15.119:0/1957705935 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92640307c0 con 0x7f9270071190 2024-04-25T10:17:38.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.405+0000 7f926d600640 1 --2- 172.21.15.119:0/1957705935 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f925006b500 0x7f925006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.405+0000 7f926d600640 1 -- 172.21.15.119:0/1957705935 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92640ab140 con 0x7f9270071190 2024-04-25T10:17:38.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.405+0000 7f9274a00640 1 --2- 172.21.15.119:0/1957705935 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f925006b500 0x7f925006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.406+0000 7f9274a00640 1 --2- 172.21.15.119:0/1957705935 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f925006b500 0x7f925006d9c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f927006b600 tx=0x7f925c0079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.410+0000 7f926d600640 1 -- 172.21.15.119:0/1957705935 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f926407a040 con 0x7f9270071190 2024-04-25T10:17:38.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.593+0000 7f9277e00640 1 -- 172.21.15.119:0/1957705935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f927006e470 con 0x7f9270071190 2024-04-25T10:17:38.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.596+0000 7f926d600640 1 -- 172.21.15.119:0/1957705935 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16676 (secure 0 0 0) 0x7f9264079940 con 0x7f9270071190 2024-04-25T10:17:38.600 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:38.600 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":107,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:17:37.617468+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:35.422922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:36.634584+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:38.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.598+0000 7f9277e00640 1 -- 172.21.15.119:0/1957705935 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f925006b500 msgr2=0x7f925006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.598+0000 7f9277e00640 1 --2- 172.21.15.119:0/1957705935 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f925006b500 0x7f925006d9c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f927006b600 tx=0x7f925c0079b0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.598+0000 7f9277e00640 1 -- 172.21.15.119:0/1957705935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9270071190 msgr2=0x7f9270150f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.598+0000 7f9277e00640 1 --2- 172.21.15.119:0/1957705935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9270071190 0x7f9270150f50 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f92640098a0 tx=0x7f92640046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.598+0000 7f9277e00640 1 -- 172.21.15.119:0/1957705935 shutdown_connections 2024-04-25T10:17:38.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.598+0000 7f9277e00640 1 --2- 172.21.15.119:0/1957705935 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f925006b500 0x7f925006d9c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.598+0000 7f9277e00640 1 --2- 172.21.15.119:0/1957705935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92700b3730 0x7f927014f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.598+0000 7f9277e00640 1 --2- 172.21.15.119:0/1957705935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9270071570 0x7f927014a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.598+0000 7f9277e00640 1 --2- 172.21.15.119:0/1957705935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9270071190 0x7f9270150f50 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.598+0000 7f9277e00640 1 -- 172.21.15.119:0/1957705935 >> 172.21.15.119:0/1957705935 conn(0x7f927006aba0 msgr2=0x7f92700ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.599+0000 7f9277e00640 1 -- 172.21.15.119:0/1957705935 shutdown_connections 2024-04-25T10:17:38.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.599+0000 7f9277e00640 1 -- 172.21.15.119:0/1957705935 wait complete. 2024-04-25T10:17:38.619 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:38.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.802+0000 7f84d3200640 1 Processor -- start 2024-04-25T10:17:38.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.802+0000 7f84d3200640 1 -- start start 2024-04-25T10:17:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.803+0000 7f84d3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d40b5050 0x7f84d415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.803+0000 7f84d3200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f84d406e430 0x7f84d415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.803+0000 7f84d3200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d40cd440 con 0x7f84d4144470 2024-04-25T10:17:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.803+0000 7f84d3200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d4145340 con 0x7f84d406e430 2024-04-25T10:17:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.803+0000 7f84d3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d40cd720 con 0x7f84d40b5050 2024-04-25T10:17:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.804+0000 7f84d2800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f84d406e430 0x7f84d415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.804+0000 7f84d2800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f84d406e430 0x7f84d415bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45788/0 (socket says 172.21.15.119:45788) 2024-04-25T10:17:38.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.804+0000 7f84d2800640 1 -- 172.21.15.119:0/1036163556 learned_addr learned my addr 172.21.15.119:0/1036163556 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.804+0000 7f84cbe00640 1 --2- 172.21.15.119:0/1036163556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d40b5050 0x7f84d415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.804+0000 7f84d2800640 1 -- 172.21.15.119:0/1036163556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d40b5050 msgr2=0x7f84d415fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.804+0000 7f84d2800640 1 --2- 172.21.15.119:0/1036163556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d40b5050 0x7f84d415fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.804+0000 7f84d2800640 1 -- 172.21.15.119:0/1036163556 >> v1:172.21.15.132:6789/0 conn(0x7f84d4144470 legacy=0x7f84d40b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.804+0000 7f84d2800640 1 -- 172.21.15.119:0/1036163556 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d40cda00 con 0x7f84d406e430 2024-04-25T10:17:38.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.804+0000 7f84cbe00640 1 --2- 172.21.15.119:0/1036163556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d40b5050 0x7f84d415fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:38.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.806+0000 7f84d2800640 1 --2- 172.21.15.119:0/1036163556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84d406e430 0x7f84d415bcf0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009e50 tx=0x7f84cc02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd81c3446e71f300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.806+0000 7f84d1400640 1 -- 172.21.15.119:0/1036163556 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc036070 con 0x7f84d406e430 2024-04-25T10:17:38.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.806+0000 7f84d1400640 1 -- 172.21.15.119:0/1036163556 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84cc031600 con 0x7f84d406e430 2024-04-25T10:17:38.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.806+0000 7f84d1400640 1 -- 172.21.15.119:0/1036163556 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc0319d0 con 0x7f84d406e430 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.807+0000 7f84d3200640 1 -- 172.21.15.119:0/1036163556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84d406e430 msgr2=0x7f84d415bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.807+0000 7f84d3200640 1 --2- 172.21.15.119:0/1036163556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84d406e430 0x7f84d415bcf0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009e50 tx=0x7f84cc02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.807+0000 7f84d3200640 1 -- 172.21.15.119:0/1036163556 shutdown_connections 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.807+0000 7f84d3200640 1 --2- 172.21.15.119:0/1036163556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84d406e430 0x7f84d415bcf0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.807+0000 7f84d3200640 1 --2- 172.21.15.119:0/1036163556 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d40b5050 0x7f84d415fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.807+0000 7f84d3200640 1 -- 172.21.15.119:0/1036163556 >> 172.21.15.119:0/1036163556 conn(0x7f84d406b1d0 msgr2=0x7f84d40b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.807+0000 7f84d3200640 1 -- 172.21.15.119:0/1036163556 shutdown_connections 2024-04-25T10:17:38.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.807+0000 7f84d3200640 1 -- 172.21.15.119:0/1036163556 wait complete. 2024-04-25T10:17:38.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.808+0000 7f84d3200640 1 Processor -- start 2024-04-25T10:17:38.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.809+0000 7f84d3200640 1 -- start start 2024-04-25T10:17:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.809+0000 7f84d3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d406e430 0x7f84d413b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84d3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84d40b5050 0x7f84d4138f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84d3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d4144470 0x7f84d40b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84d3200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d40ce200 con 0x7f84d40b5050 2024-04-25T10:17:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84d3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d4144850 con 0x7f84d406e430 2024-04-25T10:17:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84d3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d415c230 con 0x7f84d4144470 2024-04-25T10:17:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84cbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84d40b5050 0x7f84d4138f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84cbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84d40b5050 0x7f84d4138f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45794/0 (socket says 172.21.15.119:45794) 2024-04-25T10:17:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84cbe00640 1 -- 172.21.15.119:0/2243045429 learned_addr learned my addr 172.21.15.119:0/2243045429 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84d2800640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d4144470 0x7f84d40b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84d1e00640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d406e430 0x7f84d413b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84cbe00640 1 -- 172.21.15.119:0/2243045429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d4144470 msgr2=0x7f84d40b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84cbe00640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d4144470 0x7f84d40b3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84cbe00640 1 -- 172.21.15.119:0/2243045429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d406e430 msgr2=0x7f84d413b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84cbe00640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d406e430 0x7f84d413b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84cbe00640 1 -- 172.21.15.119:0/2243045429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d415b4b0 con 0x7f84d40b5050 2024-04-25T10:17:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.810+0000 7f84d1e00640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d406e430 0x7f84d413b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:38.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.811+0000 7f84cbe00640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84d40b5050 0x7f84d4138f90 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00c7f0 tx=0x7f84bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.811+0000 7f84ca000640 1 -- 172.21.15.119:0/2243045429 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc015070 con 0x7f84d40b5050 2024-04-25T10:17:38.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.811+0000 7f84ca000640 1 -- 172.21.15.119:0/2243045429 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84bc007a30 con 0x7f84d40b5050 2024-04-25T10:17:38.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.811+0000 7f84ca000640 1 -- 172.21.15.119:0/2243045429 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc007e00 con 0x7f84d40b5050 2024-04-25T10:17:38.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.811+0000 7f84d3200640 1 -- 172.21.15.119:0/2243045429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d40b3b40 con 0x7f84d40b5050 2024-04-25T10:17:38.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.811+0000 7f84d3200640 1 -- 172.21.15.119:0/2243045429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d413a720 con 0x7f84d40b5050 2024-04-25T10:17:38.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.812+0000 7f84d3200640 1 -- 172.21.15.119:0/2243045429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d40749b0 con 0x7f84d40b5050 2024-04-25T10:17:38.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.813+0000 7f84ca000640 1 -- 172.21.15.119:0/2243045429 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84bc018070 con 0x7f84d40b5050 2024-04-25T10:17:38.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.814+0000 7f84ca000640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f84b006b4d0 0x7f84b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.814+0000 7f84ca000640 1 -- 172.21.15.119:0/2243045429 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84bc08a200 con 0x7f84d40b5050 2024-04-25T10:17:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.814+0000 7f84d1e00640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f84b006b4d0 0x7f84b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.815+0000 7f84d1e00640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f84b006b4d0 0x7f84b006d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f84c400a830 tx=0x7f84c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:38.818+0000 7f84ca000640 1 -- 172.21.15.119:0/2243045429 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84bc058ff0 con 0x7f84d40b5050 2024-04-25T10:17:39.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.002+0000 7f84d3200640 1 -- 172.21.15.119:0/2243045429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84d40b3d70 con 0x7f84d40b5050 2024-04-25T10:17:39.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.004+0000 7f84ca000640 1 -- 172.21.15.119:0/2243045429 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16676 (secure 0 0 0) 0x7f84bc0588f0 con 0x7f84d40b5050 2024-04-25T10:17:39.008 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:39.008 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":107,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:17:37.617468+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:35.422922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:36.634584+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:39.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.006+0000 7f84d3200640 1 -- 172.21.15.119:0/2243045429 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f84b006b4d0 msgr2=0x7f84b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.006+0000 7f84d3200640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f84b006b4d0 0x7f84b006d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f84c400a830 tx=0x7f84c4008040 comp rx=0 tx=0).stop 2024-04-25T10:17:39.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.006+0000 7f84d3200640 1 -- 172.21.15.119:0/2243045429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84d40b5050 msgr2=0x7f84d4138f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.006+0000 7f84d3200640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84d40b5050 0x7f84d4138f90 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00c7f0 tx=0x7f84bc00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.007+0000 7f84d3200640 1 -- 172.21.15.119:0/2243045429 shutdown_connections 2024-04-25T10:17:39.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.007+0000 7f84d3200640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f84b006b4d0 0x7f84b006d990 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.007+0000 7f84d3200640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f84d4144470 0x7f84d40b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.007+0000 7f84d3200640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84d40b5050 0x7f84d4138f90 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.007+0000 7f84d3200640 1 --2- 172.21.15.119:0/2243045429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84d406e430 0x7f84d413b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.007+0000 7f84d3200640 1 -- 172.21.15.119:0/2243045429 >> 172.21.15.119:0/2243045429 conn(0x7f84d406b1d0 msgr2=0x7f84d40af600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:39.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.007+0000 7f84d3200640 1 -- 172.21.15.119:0/2243045429 shutdown_connections 2024-04-25T10:17:39.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.007+0000 7f84d3200640 1 -- 172.21.15.119:0/2243045429 wait complete. 2024-04-25T10:17:39.030 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:17:39.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.224+0000 7f9cf1400640 1 Processor -- start 2024-04-25T10:17:39.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.224+0000 7f9cf1400640 1 -- start start 2024-04-25T10:17:39.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.226+0000 7f9cf1400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cec15f3c0 0x7f9cec15f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.226+0000 7f9cf1400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9cec0cfc10 0x7f9cec0d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.226+0000 7f9cf1400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cec15fe80 con 0x7f9cec15b740 2024-04-25T10:17:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.226+0000 7f9cf1400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cec0b1660 con 0x7f9cec0cfc10 2024-04-25T10:17:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.226+0000 7f9cf1400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec160160 con 0x7f9cec15f3c0 2024-04-25T10:17:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.226+0000 7f9cf0a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9cec0cfc10 0x7f9cec0d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.226+0000 7f9cf0a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9cec0cfc10 0x7f9cec0d00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45804/0 (socket says 172.21.15.119:45804) 2024-04-25T10:17:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.226+0000 7f9cf0a00640 1 -- 172.21.15.119:0/4189922042 learned_addr learned my addr 172.21.15.119:0/4189922042 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.227+0000 7f9ceb400640 1 --2- 172.21.15.119:0/4189922042 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cec15f3c0 0x7f9cec15f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.227+0000 7f9cf0a00640 1 -- 172.21.15.119:0/4189922042 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cec15f3c0 msgr2=0x7f9cec15f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.227+0000 7f9cf0a00640 1 --2- 172.21.15.119:0/4189922042 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cec15f3c0 0x7f9cec15f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.227+0000 7f9cf0a00640 1 -- 172.21.15.119:0/4189922042 >> v1:172.21.15.132:6789/0 conn(0x7f9cec15b740 legacy=0x7f9cec0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:39.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.227+0000 7f9cf0a00640 1 -- 172.21.15.119:0/4189922042 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec0b18b0 con 0x7f9cec0cfc10 2024-04-25T10:17:39.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.228+0000 7f9cf0a00640 1 --2- 172.21.15.119:0/4189922042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cec0cfc10 0x7f9cec0d00a0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009e80 tx=0x7f9ce402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178c928bbda225e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:39.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.228+0000 7f9ceaa00640 1 -- 172.21.15.119:0/4189922042 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4036070 con 0x7f9cec0cfc10 2024-04-25T10:17:39.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.228+0000 7f9ceaa00640 1 -- 172.21.15.119:0/4189922042 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce4031600 con 0x7f9cec0cfc10 2024-04-25T10:17:39.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.228+0000 7f9ceaa00640 1 -- 172.21.15.119:0/4189922042 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce40319d0 con 0x7f9cec0cfc10 2024-04-25T10:17:39.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.229+0000 7f9cf1400640 1 -- 172.21.15.119:0/4189922042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cec0cfc10 msgr2=0x7f9cec0d00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.229+0000 7f9cf1400640 1 --2- 172.21.15.119:0/4189922042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cec0cfc10 0x7f9cec0d00a0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009e80 tx=0x7f9ce402a660 comp rx=0 tx=0).stop 2024-04-25T10:17:39.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.229+0000 7f9cf1400640 1 -- 172.21.15.119:0/4189922042 shutdown_connections 2024-04-25T10:17:39.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.229+0000 7f9cf1400640 1 --2- 172.21.15.119:0/4189922042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cec0cfc10 0x7f9cec0d00a0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.229+0000 7f9cf1400640 1 --2- 172.21.15.119:0/4189922042 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cec15f3c0 0x7f9cec15f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.229+0000 7f9cf1400640 1 -- 172.21.15.119:0/4189922042 >> 172.21.15.119:0/4189922042 conn(0x7f9cec0d1650 msgr2=0x7f9cec0d1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.229+0000 7f9cf1400640 1 -- 172.21.15.119:0/4189922042 shutdown_connections 2024-04-25T10:17:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.229+0000 7f9cf1400640 1 -- 172.21.15.119:0/4189922042 wait complete. 2024-04-25T10:17:39.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.231+0000 7f9cf1400640 1 Processor -- start 2024-04-25T10:17:39.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.232+0000 7f9cf1400640 1 -- start start 2024-04-25T10:17:39.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9cf1400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cec0cfc10 0x7f9cec158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9cf1400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cec15b740 0x7f9cec0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9cf1400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cec15f3c0 0x7f9cec0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9cf1400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec15bb20 con 0x7f9cec15b740 2024-04-25T10:17:39.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9cf1400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0cf420 con 0x7f9cec15f3c0 2024-04-25T10:17:39.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9cf1400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec15bca0 con 0x7f9cec0cfc10 2024-04-25T10:17:39.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9ceb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cec15b740 0x7f9cec0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9ceb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cec15b740 0x7f9cec0b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45816/0 (socket says 172.21.15.119:45816) 2024-04-25T10:17:39.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9ceb400640 1 -- 172.21.15.119:0/2345248502 learned_addr learned my addr 172.21.15.119:0/2345248502 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:39.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9cf0a00640 1 --2- 172.21.15.119:0/2345248502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cec15f3c0 0x7f9cec0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9ceb400640 1 -- 172.21.15.119:0/2345248502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cec0cfc10 msgr2=0x7f9cec158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9ceb400640 1 --2- 172.21.15.119:0/2345248502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cec0cfc10 0x7f9cec158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9ceb400640 1 -- 172.21.15.119:0/2345248502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cec15f3c0 msgr2=0x7f9cec0b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9ceb400640 1 --2- 172.21.15.119:0/2345248502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cec15f3c0 0x7f9cec0b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.233+0000 7f9ceb400640 1 -- 172.21.15.119:0/2345248502 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec1496e0 con 0x7f9cec15b740 2024-04-25T10:17:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.234+0000 7f9ceb400640 1 --2- 172.21.15.119:0/2345248502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cec15b740 0x7f9cec0b02b0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800d800 tx=0x7f9cd800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.234+0000 7f9ce3e00640 1 -- 172.21.15.119:0/2345248502 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd800b6c0 con 0x7f9cec15b740 2024-04-25T10:17:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.234+0000 7f9ce3e00640 1 -- 172.21.15.119:0/2345248502 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd8019070 con 0x7f9cec15b740 2024-04-25T10:17:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.234+0000 7f9cf1400640 1 -- 172.21.15.119:0/2345248502 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cec15a8b0 con 0x7f9cec15b740 2024-04-25T10:17:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.234+0000 7f9ce3e00640 1 -- 172.21.15.119:0/2345248502 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd800ba80 con 0x7f9cec15b740 2024-04-25T10:17:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.234+0000 7f9cf1400640 1 -- 172.21.15.119:0/2345248502 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cec158550 con 0x7f9cec15b740 2024-04-25T10:17:39.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.235+0000 7f9cf1400640 1 -- 172.21.15.119:0/2345248502 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cec0749b0 con 0x7f9cec15b740 2024-04-25T10:17:39.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.236+0000 7f9ce3e00640 1 -- 172.21.15.119:0/2345248502 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9cd800bc20 con 0x7f9cec15b740 2024-04-25T10:17:39.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.237+0000 7f9ce3e00640 1 --2- 172.21.15.119:0/2345248502 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ccc06b4d0 0x7f9ccc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.237+0000 7f9ce3e00640 1 -- 172.21.15.119:0/2345248502 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9cd808a780 con 0x7f9cec15b740 2024-04-25T10:17:39.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.238+0000 7f9cebe00640 1 --2- 172.21.15.119:0/2345248502 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ccc06b4d0 0x7f9ccc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.238+0000 7f9cebe00640 1 --2- 172.21.15.119:0/2345248502 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ccc06b4d0 0x7f9ccc06d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc006fd0 tx=0x7f9cdc006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:39.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.242+0000 7f9ce3e00640 1 -- 172.21.15.119:0/2345248502 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9cd8059560 con 0x7f9cec15b740 2024-04-25T10:17:39.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:39.477+0000 7f9cf1400640 1 -- 172.21.15.119:0/2345248502 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9cec0b1180 con 0x7f9cec15b740 2024-04-25T10:17:40.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.222+0000 7f9ce3e00640 1 -- 172.21.15.119:0/2345248502 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9cd8058e60 con 0x7f9cec15b740 2024-04-25T10:17:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.225+0000 7f9cf1400640 1 -- 172.21.15.119:0/2345248502 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ccc06b4d0 msgr2=0x7f9ccc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.225+0000 7f9cf1400640 1 --2- 172.21.15.119:0/2345248502 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ccc06b4d0 0x7f9ccc06d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc006fd0 tx=0x7f9cdc006bc0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.225+0000 7f9cf1400640 1 -- 172.21.15.119:0/2345248502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cec15b740 msgr2=0x7f9cec0b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.225+0000 7f9cf1400640 1 --2- 172.21.15.119:0/2345248502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cec15b740 0x7f9cec0b02b0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800d800 tx=0x7f9cd800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.225+0000 7f9cf1400640 1 -- 172.21.15.119:0/2345248502 shutdown_connections 2024-04-25T10:17:40.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.225+0000 7f9cf1400640 1 --2- 172.21.15.119:0/2345248502 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9ccc06b4d0 0x7f9ccc06d990 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.225+0000 7f9cf1400640 1 --2- 172.21.15.119:0/2345248502 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cec15f3c0 0x7f9cec0b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.225+0000 7f9cf1400640 1 --2- 172.21.15.119:0/2345248502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cec15b740 0x7f9cec0b02b0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.225+0000 7f9cf1400640 1 --2- 172.21.15.119:0/2345248502 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9cec0cfc10 0x7f9cec158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.225+0000 7f9cf1400640 1 -- 172.21.15.119:0/2345248502 >> 172.21.15.119:0/2345248502 conn(0x7f9cec0d1650 msgr2=0x7f9cec06d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:40.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.226+0000 7f9cf1400640 1 -- 172.21.15.119:0/2345248502 shutdown_connections 2024-04-25T10:17:40.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.226+0000 7f9cf1400640 1 -- 172.21.15.119:0/2345248502 wait complete. 2024-04-25T10:17:40.253 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:17:40.253 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:17:40.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.457+0000 7faa7e000640 1 Processor -- start 2024-04-25T10:17:40.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.457+0000 7faa7e000640 1 -- start start 2024-04-25T10:17:40.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.458+0000 7faa7e000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faa7806aba0 0x7faa7806af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.458+0000 7faa7e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa7806f040 0x7faa7815e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.458+0000 7faa7e000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7806b4c0 con 0x7faa780cec60 2024-04-25T10:17:40.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.458+0000 7faa7e000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7806d080 con 0x7faa7806aba0 2024-04-25T10:17:40.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.458+0000 7faa7e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7806d360 con 0x7faa7806f040 2024-04-25T10:17:40.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.459+0000 7faa7cc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faa7806aba0 0x7faa7806af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.459+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa7806f040 0x7faa7815e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.459+0000 7faa7cc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faa7806aba0 0x7faa7806af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:51704/0 (socket says 172.21.15.119:51704) 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.459+0000 7faa7cc00640 1 -- 172.21.15.119:0/2465728800 learned_addr learned my addr 172.21.15.119:0/2465728800 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.460+0000 7faa7cc00640 1 -- 172.21.15.119:0/2465728800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa7806f040 msgr2=0x7faa7815e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:40.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.460+0000 7faa7cc00640 1 --2- 172.21.15.119:0/2465728800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa7806f040 0x7faa7815e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.460+0000 7faa7cc00640 1 -- 172.21.15.119:0/2465728800 >> v1:172.21.15.132:6789/0 conn(0x7faa780cec60 legacy=0x7faa7815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.460+0000 7faa7cc00640 1 -- 172.21.15.119:0/2465728800 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7806d640 con 0x7faa7806aba0 2024-04-25T10:17:40.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.461+0000 7faa7cc00640 1 --2- 172.21.15.119:0/2465728800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa7806aba0 0x7faa7806af80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7faa68009870 tx=0x7faa6802c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcaee36e88f31ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.461+0000 7faa77e00640 1 -- 172.21.15.119:0/2465728800 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68038070 con 0x7faa7806aba0 2024-04-25T10:17:40.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.462+0000 7faa77e00640 1 -- 172.21.15.119:0/2465728800 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa68031710 con 0x7faa7806aba0 2024-04-25T10:17:40.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.462+0000 7faa77e00640 1 -- 172.21.15.119:0/2465728800 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68031ae0 con 0x7faa7806aba0 2024-04-25T10:17:40.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.462+0000 7faa7e000640 1 -- 172.21.15.119:0/2465728800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa7806aba0 msgr2=0x7faa7806af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.462+0000 7faa7e000640 1 --2- 172.21.15.119:0/2465728800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa7806aba0 0x7faa7806af80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7faa68009870 tx=0x7faa6802c460 comp rx=0 tx=0).stop 2024-04-25T10:17:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.462+0000 7faa7e000640 1 -- 172.21.15.119:0/2465728800 shutdown_connections 2024-04-25T10:17:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.462+0000 7faa7e000640 1 --2- 172.21.15.119:0/2465728800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa7806f040 0x7faa7815e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.462+0000 7faa7e000640 1 --2- 172.21.15.119:0/2465728800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa7806aba0 0x7faa7806af80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.462+0000 7faa7e000640 1 -- 172.21.15.119:0/2465728800 >> 172.21.15.119:0/2465728800 conn(0x7faa78066ff0 msgr2=0x7faa780b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.462+0000 7faa7e000640 1 -- 172.21.15.119:0/2465728800 shutdown_connections 2024-04-25T10:17:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.462+0000 7faa7e000640 1 -- 172.21.15.119:0/2465728800 wait complete. 2024-04-25T10:17:40.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.464+0000 7faa7e000640 1 Processor -- start 2024-04-25T10:17:40.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.464+0000 7faa7e000640 1 -- start start 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.465+0000 7faa7e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa7806aba0 0x7faa78155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.465+0000 7faa7e000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa7806f040 0x7faa7815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.465+0000 7faa7e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa780cec60 0x7faa78144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.465+0000 7faa7e000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7815a980 con 0x7faa780cec60 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.465+0000 7faa7e000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7806dc00 con 0x7faa7806f040 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.465+0000 7faa7e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa780aed40 con 0x7faa7806aba0 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.465+0000 7faa7d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa780cec60 0x7faa78144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.465+0000 7faa7d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa780cec60 0x7faa78144e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45826/0 (socket says 172.21.15.119:45826) 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.465+0000 7faa7d600640 1 -- 172.21.15.119:0/546003000 learned_addr learned my addr 172.21.15.119:0/546003000 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.466+0000 7faa7cc00640 1 --2- 172.21.15.119:0/546003000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa7806aba0 0x7faa78155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.466+0000 7faa6fe00640 1 --2- 172.21.15.119:0/546003000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa7806f040 0x7faa7815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.466+0000 7faa7d600640 1 -- 172.21.15.119:0/546003000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa7806aba0 msgr2=0x7faa78155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.466+0000 7faa7d600640 1 --2- 172.21.15.119:0/546003000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa7806aba0 0x7faa78155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.466+0000 7faa7d600640 1 -- 172.21.15.119:0/546003000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa7806f040 msgr2=0x7faa7815bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.466+0000 7faa7d600640 1 --2- 172.21.15.119:0/546003000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa7806f040 0x7faa7815bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.466+0000 7faa7d600640 1 -- 172.21.15.119:0/546003000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7815c4d0 con 0x7faa780cec60 2024-04-25T10:17:40.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.466+0000 7faa7d600640 1 --2- 172.21.15.119:0/546003000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa780cec60 0x7faa78144e50 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7faa7000c6b0 tx=0x7faa7000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.466+0000 7faa75600640 1 -- 172.21.15.119:0/546003000 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70003b70 con 0x7faa780cec60 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.466+0000 7faa75600640 1 -- 172.21.15.119:0/546003000 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa70003d30 con 0x7faa780cec60 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.466+0000 7faa75600640 1 -- 172.21.15.119:0/546003000 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70005310 con 0x7faa780cec60 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.466+0000 7faa7e000640 1 -- 172.21.15.119:0/546003000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7815c770 con 0x7faa780cec60 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.467+0000 7faa7e000640 1 -- 172.21.15.119:0/546003000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7815cd30 con 0x7faa780cec60 2024-04-25T10:17:40.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.468+0000 7faa7e000640 1 -- 172.21.15.119:0/546003000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa780749b0 con 0x7faa780cec60 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.469+0000 7faa75600640 1 -- 172.21.15.119:0/546003000 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa700054b0 con 0x7faa780cec60 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.470+0000 7faa75600640 1 --2- 172.21.15.119:0/546003000 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa5406b4d0 0x7faa5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.470+0000 7faa75600640 1 -- 172.21.15.119:0/546003000 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa70089fd0 con 0x7faa780cec60 2024-04-25T10:17:40.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.470+0000 7faa7cc00640 1 --2- 172.21.15.119:0/546003000 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa5406b4d0 0x7faa5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.471+0000 7faa7cc00640 1 --2- 172.21.15.119:0/546003000 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa5406b4d0 0x7faa5406d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7faa68035000 tx=0x7faa68008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.474+0000 7faa75600640 1 -- 172.21.15.119:0/546003000 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa70058ee0 con 0x7faa780cec60 2024-04-25T10:17:40.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:40.690+0000 7faa7e000640 1 -- 172.21.15.119:0/546003000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7faa7806e6c0 con 0x7faa780cec60 2024-04-25T10:17:41.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.234+0000 7faa75600640 1 -- 172.21.15.119:0/546003000 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) v1 ==== 122+0+0 (secure 0 0 0) 0x7faa700587e0 con 0x7faa780cec60 2024-04-25T10:17:41.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.239+0000 7faa7e000640 1 -- 172.21.15.119:0/546003000 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa5406b4d0 msgr2=0x7faa5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.239+0000 7faa7e000640 1 --2- 172.21.15.119:0/546003000 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa5406b4d0 0x7faa5406d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7faa68035000 tx=0x7faa68008030 comp rx=0 tx=0).stop 2024-04-25T10:17:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.239+0000 7faa7e000640 1 -- 172.21.15.119:0/546003000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa780cec60 msgr2=0x7faa78144e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.239+0000 7faa7e000640 1 --2- 172.21.15.119:0/546003000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa780cec60 0x7faa78144e50 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7faa7000c6b0 tx=0x7faa7000cb70 comp rx=0 tx=0).stop 2024-04-25T10:17:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.239+0000 7faa7e000640 1 -- 172.21.15.119:0/546003000 shutdown_connections 2024-04-25T10:17:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.239+0000 7faa7e000640 1 --2- 172.21.15.119:0/546003000 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faa5406b4d0 0x7faa5406d990 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.239+0000 7faa7e000640 1 --2- 172.21.15.119:0/546003000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa780cec60 0x7faa78144e50 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.239+0000 7faa7e000640 1 --2- 172.21.15.119:0/546003000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa7806f040 0x7faa7815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.239+0000 7faa7e000640 1 --2- 172.21.15.119:0/546003000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faa7806aba0 0x7faa78155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.239+0000 7faa7e000640 1 -- 172.21.15.119:0/546003000 >> 172.21.15.119:0/546003000 conn(0x7faa78066ff0 msgr2=0x7faa78071050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.239+0000 7faa7e000640 1 -- 172.21.15.119:0/546003000 shutdown_connections 2024-04-25T10:17:41.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.239+0000 7faa7e000640 1 -- 172.21.15.119:0/546003000 wait complete. 2024-04-25T10:17:41.264 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:17:41.264 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:41.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.456+0000 7f9a9ee00640 1 Processor -- start 2024-04-25T10:17:41.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.457+0000 7f9a9ee00640 1 -- start start 2024-04-25T10:17:41.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.458+0000 7f9a9ee00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9a98161ee0 0x7f9a980b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.458+0000 7f9a9ee00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9a981664d0 0x7f9a980b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.458+0000 7f9a9ee00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a98071330 con 0x7f9a980b2570 2024-04-25T10:17:41.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.458+0000 7f9a9ee00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a980720f0 con 0x7f9a981664d0 2024-04-25T10:17:41.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.458+0000 7f9a9ee00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a98071610 con 0x7f9a98161ee0 2024-04-25T10:17:41.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.458+0000 7f9a9ce00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9a980b2570 0x7f9a9806dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56840/0 (socket says 172.21.15.119:56840) 2024-04-25T10:17:41.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.458+0000 7f9a8f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9a981664d0 0x7f9a980b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.458+0000 7f9a9ce00640 1 -- 172.21.15.119:0/2620024118 learned_addr learned my addr 172.21.15.119:0/2620024118 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:41.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.459+0000 7f9a97e00640 1 --2- 172.21.15.119:0/2620024118 >> v2:172.21.15.132:3300/0 conn(0x7f9a98161ee0 0x7f9a980b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.459+0000 7f9a97400640 1 -- 172.21.15.119:0/2620024118 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544244056 0 0) 0x7f9a98071330 con 0x7f9a980b2570 2024-04-25T10:17:41.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.459+0000 7f9a8f400640 1 -- 172.21.15.119:0/2620024118 >> v2:172.21.15.132:3300/0 conn(0x7f9a98161ee0 msgr2=0x7f9a980b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:41.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.459+0000 7f9a8f400640 1 --2- 172.21.15.119:0/2620024118 >> v2:172.21.15.132:3300/0 conn(0x7f9a98161ee0 0x7f9a980b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.459+0000 7f9a8f400640 1 -- 172.21.15.119:0/2620024118 >> v1:172.21.15.119:6789/0 conn(0x7f9a980b2570 legacy=0x7f9a9806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.459+0000 7f9a8f400640 1 -- 172.21.15.119:0/2620024118 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a980718f0 con 0x7f9a981664d0 2024-04-25T10:17:41.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.460+0000 7f9a8f400640 1 --2- 172.21.15.119:0/2620024118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a981664d0 0x7f9a980b34a0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f9a98067460 tx=0x7f9a8002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c67a96b41ac6e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.460+0000 7f9a97400640 1 -- 172.21.15.119:0/2620024118 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80037070 con 0x7f9a981664d0 2024-04-25T10:17:41.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.461+0000 7f9a97400640 1 -- 172.21.15.119:0/2620024118 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a80032440 con 0x7f9a981664d0 2024-04-25T10:17:41.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.461+0000 7f9a97400640 1 -- 172.21.15.119:0/2620024118 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80032810 con 0x7f9a981664d0 2024-04-25T10:17:41.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.461+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2620024118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a981664d0 msgr2=0x7f9a980b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.461+0000 7f9a9ee00640 1 --2- 172.21.15.119:0/2620024118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a981664d0 0x7f9a980b34a0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f9a98067460 tx=0x7f9a8002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.461+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2620024118 shutdown_connections 2024-04-25T10:17:41.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.461+0000 7f9a9ee00640 1 --2- 172.21.15.119:0/2620024118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a981664d0 0x7f9a980b34a0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.461+0000 7f9a9ee00640 1 --2- 172.21.15.119:0/2620024118 >> v2:172.21.15.132:3300/0 conn(0x7f9a98161ee0 0x7f9a980b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.461+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2620024118 >> 172.21.15.119:0/2620024118 conn(0x7f9a9806aba0 msgr2=0x7f9a9806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:41.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.461+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2620024118 shutdown_connections 2024-04-25T10:17:41.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.461+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2620024118 wait complete. 2024-04-25T10:17:41.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.463+0000 7f9a9ee00640 1 Processor -- start 2024-04-25T10:17:41.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.463+0000 7f9a9ee00640 1 -- start start 2024-04-25T10:17:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.464+0000 7f9a9ee00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a980b2570 0x7f9a980adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.464+0000 7f9a9ee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a98161ee0 0x7f9a980b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.464+0000 7f9a9ee00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a981664d0 0x7f9a980afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.464+0000 7f9a9ee00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a980b3d30 con 0x7f9a981664d0 2024-04-25T10:17:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.464+0000 7f9a9ee00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a980b3bb0 con 0x7f9a980b2570 2024-04-25T10:17:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.464+0000 7f9a9ee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a981632d0 con 0x7f9a98161ee0 2024-04-25T10:17:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.464+0000 7f9a9ce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a981664d0 0x7f9a980afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.464+0000 7f9a9ce00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a981664d0 0x7f9a980afce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36914/0 (socket says 172.21.15.119:36914) 2024-04-25T10:17:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.464+0000 7f9a9ce00640 1 -- 172.21.15.119:0/2709422663 learned_addr learned my addr 172.21.15.119:0/2709422663 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:41.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.465+0000 7f9a8f400640 1 --2- 172.21.15.119:0/2709422663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a98161ee0 0x7f9a980b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.465+0000 7f9a97e00640 1 --2- 172.21.15.119:0/2709422663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a980b2570 0x7f9a980adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.465+0000 7f9a9ce00640 1 -- 172.21.15.119:0/2709422663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a98161ee0 msgr2=0x7f9a980b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.465+0000 7f9a9ce00640 1 --2- 172.21.15.119:0/2709422663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a98161ee0 0x7f9a980b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.465+0000 7f9a9ce00640 1 -- 172.21.15.119:0/2709422663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a980b2570 msgr2=0x7f9a980adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.465+0000 7f9a9ce00640 1 --2- 172.21.15.119:0/2709422663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a980b2570 0x7f9a980adbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.465+0000 7f9a9ce00640 1 -- 172.21.15.119:0/2709422663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a980b0a50 con 0x7f9a981664d0 2024-04-25T10:17:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.465+0000 7f9a9ce00640 1 --2- 172.21.15.119:0/2709422663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a981664d0 0x7f9a980afce0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000caf0 tx=0x7f9a9000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.465+0000 7f9a94c00640 1 -- 172.21.15.119:0/2709422663 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9000aca0 con 0x7f9a981664d0 2024-04-25T10:17:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.465+0000 7f9a94c00640 1 -- 172.21.15.119:0/2709422663 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9000ae40 con 0x7f9a981664d0 2024-04-25T10:17:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.465+0000 7f9a94c00640 1 -- 172.21.15.119:0/2709422663 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90012640 con 0x7f9a981664d0 2024-04-25T10:17:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.465+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2709422663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a980b0d30 con 0x7f9a981664d0 2024-04-25T10:17:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.465+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2709422663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a980b12f0 con 0x7f9a981664d0 2024-04-25T10:17:41.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.466+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2709422663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a980749b0 con 0x7f9a981664d0 2024-04-25T10:17:41.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.467+0000 7f9a94c00640 1 -- 172.21.15.119:0/2709422663 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a900127e0 con 0x7f9a981664d0 2024-04-25T10:17:41.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.468+0000 7f9a94c00640 1 --2- 172.21.15.119:0/2709422663 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a7406b500 0x7f9a7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.468+0000 7f9a94c00640 1 -- 172.21.15.119:0/2709422663 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a9008a880 con 0x7f9a981664d0 2024-04-25T10:17:41.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.468+0000 7f9a97e00640 1 --2- 172.21.15.119:0/2709422663 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a7406b500 0x7f9a7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.469+0000 7f9a97e00640 1 --2- 172.21.15.119:0/2709422663 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a7406b500 0x7f9a7406d9c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9a88006fd0 tx=0x7f9a88008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.473+0000 7f9a94c00640 1 -- 172.21.15.119:0/2709422663 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a900596f0 con 0x7f9a981664d0 2024-04-25T10:17:41.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.705+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2709422663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a9806d4c0 con 0x7f9a981664d0 2024-04-25T10:17:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.706+0000 7f9a94c00640 1 -- 172.21.15.119:0/2709422663 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f9a90058ff0 con 0x7f9a981664d0 2024-04-25T10:17:41.709 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:41.709 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3394261143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3394261143},{"type":"v1","addr":"172.21.15.132:6834","nonce":3394261143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2502874931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2502874931},{"type":"v1","addr":"172.21.15.119:6835","nonce":2502874931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":8694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2305428003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2305428003},{"type":"v1","addr":"172.21.15.132:6835","nonce":2305428003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:37.617787+0000","modified":"2024-04-25T10:17:40.692459+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8697":{"gid":8697,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1849584205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":1849584205},{"type":"v1","addr":"172.21.15.119:6837","nonce":1849584205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:17:41.709 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 99 2024-04-25T10:17:41.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.708+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2709422663 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a7406b500 msgr2=0x7f9a7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.708+0000 7f9a9ee00640 1 --2- 172.21.15.119:0/2709422663 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a7406b500 0x7f9a7406d9c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9a88006fd0 tx=0x7f9a88008040 comp rx=0 tx=0).stop 2024-04-25T10:17:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.708+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2709422663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a981664d0 msgr2=0x7f9a980afce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.708+0000 7f9a9ee00640 1 --2- 172.21.15.119:0/2709422663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a981664d0 0x7f9a980afce0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000caf0 tx=0x7f9a9000a060 comp rx=0 tx=0).stop 2024-04-25T10:17:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.708+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2709422663 shutdown_connections 2024-04-25T10:17:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.708+0000 7f9a9ee00640 1 --2- 172.21.15.119:0/2709422663 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a7406b500 0x7f9a7406d9c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.708+0000 7f9a9ee00640 1 --2- 172.21.15.119:0/2709422663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a981664d0 0x7f9a980afce0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.708+0000 7f9a9ee00640 1 --2- 172.21.15.119:0/2709422663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a98161ee0 0x7f9a980b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.708+0000 7f9a9ee00640 1 --2- 172.21.15.119:0/2709422663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a980b2570 0x7f9a980adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.708+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2709422663 >> 172.21.15.119:0/2709422663 conn(0x7f9a9806aba0 msgr2=0x7f9a981622c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.708+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2709422663 shutdown_connections 2024-04-25T10:17:41.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.708+0000 7f9a9ee00640 1 -- 172.21.15.119:0/2709422663 wait complete. 2024-04-25T10:17:41.730 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:41.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.925+0000 7fa94ac00640 1 Processor -- start 2024-04-25T10:17:41.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.925+0000 7fa94ac00640 1 -- start start 2024-04-25T10:17:41.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.926+0000 7fa94ac00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa9440d3f80 0x7fa9440d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.926+0000 7fa94ac00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94415ef10 con 0x7fa9440b9270 2024-04-25T10:17:41.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.926+0000 7fa94ac00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94415e390 con 0x7fa94415db60 2024-04-25T10:17:41.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.926+0000 7fa94ac00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94415e670 con 0x7fa9440d3f80 2024-04-25T10:17:41.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.927+0000 7fa94a200640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa9440b9270 0x7fa9440d24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56842/0 (socket says 172.21.15.119:56842) 2024-04-25T10:17:41.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.927+0000 7fa94a200640 1 -- 172.21.15.119:0/1218664790 learned_addr learned my addr 172.21.15.119:0/1218664790 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:41.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.927+0000 7fa948e00640 1 --2- 172.21.15.119:0/1218664790 >> v2:172.21.15.132:3300/0 conn(0x7fa9440d3f80 0x7fa9440d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.927+0000 7fa943e00640 1 -- 172.21.15.119:0/1218664790 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460092799 0 0) 0x7fa94415ef10 con 0x7fa9440b9270 2024-04-25T10:17:41.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.927+0000 7fa943e00640 1 -- 172.21.15.119:0/1218664790 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa92c003520 con 0x7fa9440b9270 2024-04-25T10:17:41.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.927+0000 7fa943e00640 1 -- 172.21.15.119:0/1218664790 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1969480299 0 0) 0x7fa92c003520 con 0x7fa9440b9270 2024-04-25T10:17:41.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.927+0000 7fa943e00640 1 -- 172.21.15.119:0/1218664790 >> v2:172.21.15.132:3300/0 conn(0x7fa9440d3f80 msgr2=0x7fa9440d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:41.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.927+0000 7fa943e00640 1 --2- 172.21.15.119:0/1218664790 >> v2:172.21.15.132:3300/0 conn(0x7fa9440d3f80 0x7fa9440d1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.927+0000 7fa943e00640 1 -- 172.21.15.119:0/1218664790 >> v1:172.21.15.132:6789/0 conn(0x7fa94415db60 legacy=0x7fa9440d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:41.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.927+0000 7fa943e00640 1 -- 172.21.15.119:0/1218664790 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94415e950 con 0x7fa9440b9270 2024-04-25T10:17:41.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa943e00640 1 -- 172.21.15.119:0/1218664790 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fa93c0019f0 con 0x7fa9440b9270 2024-04-25T10:17:41.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa943e00640 1 -- 172.21.15.119:0/1218664790 >> v1:172.21.15.119:6789/0 conn(0x7fa9440b9270 legacy=0x7fa9440d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa943e00640 1 --2- 172.21.15.119:0/1218664790 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92c003a70 0x7fa92c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa943e00640 1 --2- 172.21.15.119:0/1218664790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa92c004800 0x7fa92c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa943e00640 1 --2- 172.21.15.119:0/1218664790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa92c0051c0 0x7fa92c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa943e00640 1 -- 172.21.15.119:0/1218664790 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94415e670 con 0x7fa92c004800 2024-04-25T10:17:41.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa943e00640 1 -- 172.21.15.119:0/1218664790 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c00c160 con 0x7fa92c003a70 2024-04-25T10:17:41.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa943e00640 1 -- 172.21.15.119:0/1218664790 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c00c360 con 0x7fa92c0051c0 2024-04-25T10:17:41.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa948e00640 1 --2- 172.21.15.119:0/1218664790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa92c004800 0x7fa92c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa948e00640 1 -- 172.21.15.119:0/1218664790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa92c0051c0 msgr2=0x7fa92c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:41.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa94a200640 1 --2- 172.21.15.119:0/1218664790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa92c0051c0 0x7fa92c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa948e00640 1 --2- 172.21.15.119:0/1218664790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa92c0051c0 0x7fa92c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa948e00640 1 -- 172.21.15.119:0/1218664790 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92c003a70 msgr2=0x7fa92c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:41.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa948e00640 1 --2- 172.21.15.119:0/1218664790 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92c003a70 0x7fa92c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa948e00640 1 -- 172.21.15.119:0/1218664790 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92c003520 con 0x7fa92c004800 2024-04-25T10:17:41.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.928+0000 7fa94a200640 1 -- 172.21.15.119:0/1218664790 reap_dead start 2024-04-25T10:17:41.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.929+0000 7fa948e00640 1 --2- 172.21.15.119:0/1218664790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa92c004800 0x7fa92c004c80 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fa930009ab0 tx=0x7fa92c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac5bdd58d4fdc93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.930+0000 7fa943e00640 1 -- 172.21.15.119:0/1218664790 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930036070 con 0x7fa92c004800 2024-04-25T10:17:41.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.930+0000 7fa943e00640 1 -- 172.21.15.119:0/1218664790 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa930005990 con 0x7fa92c004800 2024-04-25T10:17:41.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.930+0000 7fa943e00640 1 -- 172.21.15.119:0/1218664790 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930009680 con 0x7fa92c004800 2024-04-25T10:17:41.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.930+0000 7fa94ac00640 1 -- 172.21.15.119:0/1218664790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa92c004800 msgr2=0x7fa92c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.930+0000 7fa94ac00640 1 --2- 172.21.15.119:0/1218664790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa92c004800 0x7fa92c004c80 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fa930009ab0 tx=0x7fa92c005830 comp rx=0 tx=0).stop 2024-04-25T10:17:41.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.930+0000 7fa94ac00640 1 -- 172.21.15.119:0/1218664790 shutdown_connections 2024-04-25T10:17:41.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.930+0000 7fa94ac00640 1 --2- 172.21.15.119:0/1218664790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa92c004800 0x7fa92c004c80 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.930+0000 7fa94ac00640 1 -- 172.21.15.119:0/1218664790 >> 172.21.15.119:0/1218664790 conn(0x7fa9440726c0 msgr2=0x7fa944072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:41.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.930+0000 7fa94ac00640 1 -- 172.21.15.119:0/1218664790 shutdown_connections 2024-04-25T10:17:41.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.930+0000 7fa94ac00640 1 -- 172.21.15.119:0/1218664790 wait complete. 2024-04-25T10:17:41.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.932+0000 7fa94ac00640 1 Processor -- start 2024-04-25T10:17:41.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.932+0000 7fa94ac00640 1 -- start start 2024-04-25T10:17:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.933+0000 7fa94ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa92c004800 0x7fa94415aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.933+0000 7fa94ac00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9440b4f70 0x7fa9440d2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.933+0000 7fa94ac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9440d2ab0 0x7fa9441f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.933+0000 7fa94ac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9440b3120 con 0x7fa9440d2ab0 2024-04-25T10:17:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.933+0000 7fa94ac00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9440b2fa0 con 0x7fa9440b4f70 2024-04-25T10:17:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.933+0000 7fa94ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94415df40 con 0x7fa92c004800 2024-04-25T10:17:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.933+0000 7fa94a200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9440d2ab0 0x7fa9441f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.933+0000 7fa94a200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9440d2ab0 0x7fa9441f3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36922/0 (socket says 172.21.15.119:36922) 2024-04-25T10:17:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.933+0000 7fa94a200640 1 -- 172.21.15.119:0/2161346201 learned_addr learned my addr 172.21.15.119:0/2161346201 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.934+0000 7fa948e00640 1 --2- 172.21.15.119:0/2161346201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9440b4f70 0x7fa9440d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.934+0000 7fa949800640 1 --2- 172.21.15.119:0/2161346201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa92c004800 0x7fa94415aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.934+0000 7fa94a200640 1 -- 172.21.15.119:0/2161346201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa92c004800 msgr2=0x7fa94415aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.934+0000 7fa94a200640 1 --2- 172.21.15.119:0/2161346201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa92c004800 0x7fa94415aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.934+0000 7fa94a200640 1 -- 172.21.15.119:0/2161346201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9440b4f70 msgr2=0x7fa9440d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.934+0000 7fa94a200640 1 --2- 172.21.15.119:0/2161346201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9440b4f70 0x7fa9440d2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.934+0000 7fa94a200640 1 -- 172.21.15.119:0/2161346201 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9441f3f50 con 0x7fa9440d2ab0 2024-04-25T10:17:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.934+0000 7fa94a200640 1 --2- 172.21.15.119:0/2161346201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9440d2ab0 0x7fa9441f3220 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa9440d13b0 tx=0x7fa93c00b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.934+0000 7fa941600640 1 -- 172.21.15.119:0/2161346201 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00b870 con 0x7fa9440d2ab0 2024-04-25T10:17:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.934+0000 7fa941600640 1 -- 172.21.15.119:0/2161346201 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c00ba30 con 0x7fa9440d2ab0 2024-04-25T10:17:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.934+0000 7fa941600640 1 -- 172.21.15.119:0/2161346201 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00be20 con 0x7fa9440d2ab0 2024-04-25T10:17:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.935+0000 7fa94ac00640 1 -- 172.21.15.119:0/2161346201 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9440b4490 con 0x7fa9440d2ab0 2024-04-25T10:17:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.935+0000 7fa94ac00640 1 -- 172.21.15.119:0/2161346201 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94415b3e0 con 0x7fa9440d2ab0 2024-04-25T10:17:41.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.936+0000 7fa94ac00640 1 -- 172.21.15.119:0/2161346201 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9440d2f30 con 0x7fa9440d2ab0 2024-04-25T10:17:41.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.937+0000 7fa941600640 1 -- 172.21.15.119:0/2161346201 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa93c01b020 con 0x7fa9440d2ab0 2024-04-25T10:17:41.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.937+0000 7fa941600640 1 --2- 172.21.15.119:0/2161346201 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa92406b500 0x7fa92406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.938+0000 7fa941600640 1 -- 172.21.15.119:0/2161346201 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa93c08a270 con 0x7fa9440d2ab0 2024-04-25T10:17:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.938+0000 7fa949800640 1 --2- 172.21.15.119:0/2161346201 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa92406b500 0x7fa92406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.938+0000 7fa949800640 1 --2- 172.21.15.119:0/2161346201 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa92406b500 0x7fa92406d9c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa9441f0d80 tx=0x7fa938009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:41.942+0000 7fa941600640 1 -- 172.21.15.119:0/2161346201 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa93c059180 con 0x7fa9440d2ab0 2024-04-25T10:17:42.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.124+0000 7fa94ac00640 1 -- 172.21.15.119:0/2161346201 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9440749b0 con 0x7fa9440d2ab0 2024-04-25T10:17:42.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.127+0000 7fa941600640 1 -- 172.21.15.119:0/2161346201 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16663 (secure 0 0 0) 0x7fa93c058a80 con 0x7fa9440d2ab0 2024-04-25T10:17:42.131 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:42.131 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":108,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:17:39.194687+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:35.422922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:36.634584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:42.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.129+0000 7fa94ac00640 1 -- 172.21.15.119:0/2161346201 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa92406b500 msgr2=0x7fa92406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.129+0000 7fa94ac00640 1 --2- 172.21.15.119:0/2161346201 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa92406b500 0x7fa92406d9c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa9441f0d80 tx=0x7fa938009040 comp rx=0 tx=0).stop 2024-04-25T10:17:42.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.129+0000 7fa94ac00640 1 -- 172.21.15.119:0/2161346201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9440d2ab0 msgr2=0x7fa9441f3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.129+0000 7fa94ac00640 1 --2- 172.21.15.119:0/2161346201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9440d2ab0 0x7fa9441f3220 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa9440d13b0 tx=0x7fa93c00b0d0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.129+0000 7fa94ac00640 1 -- 172.21.15.119:0/2161346201 shutdown_connections 2024-04-25T10:17:42.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.129+0000 7fa94ac00640 1 --2- 172.21.15.119:0/2161346201 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa92406b500 0x7fa92406d9c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.129+0000 7fa94ac00640 1 --2- 172.21.15.119:0/2161346201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9440d2ab0 0x7fa9441f3220 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.129+0000 7fa94ac00640 1 --2- 172.21.15.119:0/2161346201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9440b4f70 0x7fa9440d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.129+0000 7fa94ac00640 1 --2- 172.21.15.119:0/2161346201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa92c004800 0x7fa94415aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.129+0000 7fa94ac00640 1 -- 172.21.15.119:0/2161346201 >> 172.21.15.119:0/2161346201 conn(0x7fa9440726c0 msgr2=0x7fa9440b7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.130+0000 7fa94ac00640 1 -- 172.21.15.119:0/2161346201 shutdown_connections 2024-04-25T10:17:42.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.130+0000 7fa94ac00640 1 -- 172.21.15.119:0/2161346201 wait complete. 2024-04-25T10:17:42.150 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:17:42.151 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:42.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.336+0000 7f99fc600640 1 Processor -- start 2024-04-25T10:17:42.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.336+0000 7f99fc600640 1 -- start start 2024-04-25T10:17:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.337+0000 7f99fc600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99f406e8f0 0x7f99f415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.337+0000 7f99fc600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f99f415c820 0x7f99f41654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.337+0000 7f99fc600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f40ce6c0 con 0x7f99f415bf00 2024-04-25T10:17:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.337+0000 7f99fc600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f41455e0 con 0x7f99f415c820 2024-04-25T10:17:42.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.337+0000 7f99fc600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f406d450 con 0x7f99f406e8f0 2024-04-25T10:17:42.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.338+0000 7f99fa600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f99f415c820 0x7f99f41654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.338+0000 7f99f9200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99f406e8f0 0x7f99f415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.338+0000 7f99fa600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f99f415c820 0x7f99f41654b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:41868/0 (socket says 172.21.15.119:41868) 2024-04-25T10:17:42.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.338+0000 7f99fa600640 1 -- 172.21.15.119:0/2829699148 learned_addr learned my addr 172.21.15.119:0/2829699148 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:42.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.339+0000 7f99fa600640 1 -- 172.21.15.119:0/2829699148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99f406e8f0 msgr2=0x7f99f415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.339+0000 7f99fa600640 1 --2- 172.21.15.119:0/2829699148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99f406e8f0 0x7f99f415c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.339+0000 7f99fa600640 1 -- 172.21.15.119:0/2829699148 >> v1:172.21.15.132:6789/0 conn(0x7f99f415bf00 legacy=0x7f99f40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.339+0000 7f99fa600640 1 -- 172.21.15.119:0/2829699148 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f40cc4e0 con 0x7f99f415c820 2024-04-25T10:17:42.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.339+0000 7f99f9200640 1 --2- 172.21.15.119:0/2829699148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99f406e8f0 0x7f99f415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:42.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.341+0000 7f99fa600640 1 --2- 172.21.15.119:0/2829699148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f415c820 0x7f99f41654b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009e50 tx=0x7f99ec02a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2062bd2be35a613 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.341+0000 7f99f3e00640 1 -- 172.21.15.119:0/2829699148 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec036070 con 0x7f99f415c820 2024-04-25T10:17:42.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.341+0000 7f99f3e00640 1 -- 172.21.15.119:0/2829699148 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec031600 con 0x7f99f415c820 2024-04-25T10:17:42.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.341+0000 7f99f3e00640 1 -- 172.21.15.119:0/2829699148 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0319d0 con 0x7f99f415c820 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.342+0000 7f99fc600640 1 -- 172.21.15.119:0/2829699148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f415c820 msgr2=0x7f99f41654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.342+0000 7f99fc600640 1 --2- 172.21.15.119:0/2829699148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f415c820 0x7f99f41654b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009e50 tx=0x7f99ec02a730 comp rx=0 tx=0).stop 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.342+0000 7f99fc600640 1 -- 172.21.15.119:0/2829699148 shutdown_connections 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.342+0000 7f99fc600640 1 --2- 172.21.15.119:0/2829699148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f415c820 0x7f99f41654b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.342+0000 7f99fc600640 1 --2- 172.21.15.119:0/2829699148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99f406e8f0 0x7f99f415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.342+0000 7f99fc600640 1 -- 172.21.15.119:0/2829699148 >> 172.21.15.119:0/2829699148 conn(0x7f99f406aba0 msgr2=0x7f99f406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.342+0000 7f99fc600640 1 -- 172.21.15.119:0/2829699148 shutdown_connections 2024-04-25T10:17:42.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.342+0000 7f99fc600640 1 -- 172.21.15.119:0/2829699148 wait complete. 2024-04-25T10:17:42.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.344+0000 7f99fc600640 1 Processor -- start 2024-04-25T10:17:42.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.344+0000 7f99fc600640 1 -- start start 2024-04-25T10:17:42.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.346+0000 7f99fc600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99f406e8f0 0x7f99f413eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.346+0000 7f99fc600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99f415bf00 0x7f99f4142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.346+0000 7f99fc600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f415c820 0x7f99f4136e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.346+0000 7f99fc600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f40ccd70 con 0x7f99f415bf00 2024-04-25T10:17:42.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.346+0000 7f99fc600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f40cca90 con 0x7f99f415c820 2024-04-25T10:17:42.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.346+0000 7f99fc600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f40cd050 con 0x7f99f406e8f0 2024-04-25T10:17:42.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.346+0000 7f99f9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99f415bf00 0x7f99f4142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.346+0000 7f99f9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99f415bf00 0x7f99f4142240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36934/0 (socket says 172.21.15.119:36934) 2024-04-25T10:17:42.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.346+0000 7f99f9200640 1 -- 172.21.15.119:0/370945690 learned_addr learned my addr 172.21.15.119:0/370945690 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:42.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.346+0000 7f99f9c00640 1 --2- 172.21.15.119:0/370945690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99f406e8f0 0x7f99f413eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.346+0000 7f99fa600640 1 --2- 172.21.15.119:0/370945690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f415c820 0x7f99f4136e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.347+0000 7f99f9200640 1 -- 172.21.15.119:0/370945690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99f406e8f0 msgr2=0x7f99f413eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.347+0000 7f99f9200640 1 --2- 172.21.15.119:0/370945690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99f406e8f0 0x7f99f413eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.347+0000 7f99f9200640 1 -- 172.21.15.119:0/370945690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f415c820 msgr2=0x7f99f4136e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.347+0000 7f99f9200640 1 --2- 172.21.15.119:0/370945690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f415c820 0x7f99f4136e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.347+0000 7f99f9200640 1 -- 172.21.15.119:0/370945690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f413da80 con 0x7f99f415bf00 2024-04-25T10:17:42.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.347+0000 7f99f9200640 1 --2- 172.21.15.119:0/370945690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99f415bf00 0x7f99f4142240 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f99e800ebb0 tx=0x7f99e800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.347+0000 7f99f1600640 1 -- 172.21.15.119:0/370945690 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8017070 con 0x7f99f415bf00 2024-04-25T10:17:42.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.347+0000 7f99f1600640 1 -- 172.21.15.119:0/370945690 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e800cc30 con 0x7f99f415bf00 2024-04-25T10:17:42.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.347+0000 7f99f1600640 1 -- 172.21.15.119:0/370945690 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8013470 con 0x7f99f415bf00 2024-04-25T10:17:42.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.347+0000 7f99fc600640 1 -- 172.21.15.119:0/370945690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f413dd60 con 0x7f99f415bf00 2024-04-25T10:17:42.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.347+0000 7f99fc600640 1 -- 172.21.15.119:0/370945690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f413e320 con 0x7f99f415bf00 2024-04-25T10:17:42.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.349+0000 7f99fc600640 1 -- 172.21.15.119:0/370945690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f40749b0 con 0x7f99f415bf00 2024-04-25T10:17:42.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.350+0000 7f99f1600640 1 -- 172.21.15.119:0/370945690 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99e801a070 con 0x7f99f415bf00 2024-04-25T10:17:42.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.351+0000 7f99f1600640 1 --2- 172.21.15.119:0/370945690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f99d806b4d0 0x7f99d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.351+0000 7f99f1600640 1 -- 172.21.15.119:0/370945690 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99e808a560 con 0x7f99f415bf00 2024-04-25T10:17:42.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.351+0000 7f99f9c00640 1 --2- 172.21.15.119:0/370945690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f99d806b4d0 0x7f99d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.352+0000 7f99f9c00640 1 --2- 172.21.15.119:0/370945690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f99d806b4d0 0x7f99d806d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f99e4001640 tx=0x7f99e4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.355+0000 7f99f1600640 1 -- 172.21.15.119:0/370945690 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99e8059340 con 0x7f99f415bf00 2024-04-25T10:17:42.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.546+0000 7f99fc600640 1 -- 172.21.15.119:0/370945690 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f99f406b120 con 0x7f99d806b4d0 2024-04-25T10:17:42.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.560+0000 7f99f1600640 1 -- 172.21.15.119:0/370945690 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252482 (secure 0 0 0) 0x7f99f406b120 con 0x7f99d806b4d0 2024-04-25T10:17:42.568 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:42.570 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:17:42.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.567+0000 7f99fc600640 1 -- 172.21.15.119:0/370945690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f99d806b4d0 msgr2=0x7f99d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.567+0000 7f99fc600640 1 --2- 172.21.15.119:0/370945690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f99d806b4d0 0x7f99d806d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f99e4001640 tx=0x7f99e4009040 comp rx=0 tx=0).stop 2024-04-25T10:17:42.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.567+0000 7f99fc600640 1 -- 172.21.15.119:0/370945690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99f415bf00 msgr2=0x7f99f4142240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.567+0000 7f99fc600640 1 --2- 172.21.15.119:0/370945690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99f415bf00 0x7f99f4142240 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f99e800ebb0 tx=0x7f99e800c520 comp rx=0 tx=0).stop 2024-04-25T10:17:42.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.567+0000 7f99fc600640 1 -- 172.21.15.119:0/370945690 shutdown_connections 2024-04-25T10:17:42.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.567+0000 7f99fc600640 1 --2- 172.21.15.119:0/370945690 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f99d806b4d0 0x7f99d806d990 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.567+0000 7f99fc600640 1 --2- 172.21.15.119:0/370945690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f415c820 0x7f99f4136e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.567+0000 7f99fc600640 1 --2- 172.21.15.119:0/370945690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99f415bf00 0x7f99f4142240 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.567+0000 7f99fc600640 1 --2- 172.21.15.119:0/370945690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f99f406e8f0 0x7f99f413eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.567+0000 7f99fc600640 1 -- 172.21.15.119:0/370945690 >> 172.21.15.119:0/370945690 conn(0x7f99f406aba0 msgr2=0x7f99f415d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.567+0000 7f99fc600640 1 -- 172.21.15.119:0/370945690 shutdown_connections 2024-04-25T10:17:42.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.567+0000 7f99fc600640 1 -- 172.21.15.119:0/370945690 wait complete. 2024-04-25T10:17:42.589 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-04-25T10:17:42.140830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261208,"kb_used_data":11720,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713512,"statfs":{"total":773094113280,"available":772826636288,"internally_reserved":0,"allocated":12001280,"data_stored":5838052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.992017"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402443+0000","last_change":"2024-04-25T10:17:38.219576+0000","last_active":"2024-04-25T10:17:39.402443+0000","last_peered":"2024-04-25T10:17:39.402443+0000","last_clean":"2024-04-25T10:17:39.402443+0000","last_became_active":"2024-04-25T10:17:38.218662+0000","last_became_peered":"2024-04-25T10:17:38.218662+0000","last_unstale":"2024-04-25T10:17:39.402443+0000","last_undegraded":"2024-04-25T10:17:39.402443+0000","last_fullsized":"2024-04-25T10:17:39.402443+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:30.593455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.207426+0000","last_change":"2024-04-25T10:17:38.209218+0000","last_active":"2024-04-25T10:17:39.207426+0000","last_peered":"2024-04-25T10:17:39.207426+0000","last_clean":"2024-04-25T10:17:39.207426+0000","last_became_active":"2024-04-25T10:17:38.208169+0000","last_became_peered":"2024-04-25T10:17:38.208169+0000","last_unstale":"2024-04-25T10:17:39.207426+0000","last_undegraded":"2024-04-25T10:17:39.207426+0000","last_fullsized":"2024-04-25T10:17:39.207426+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:46.798286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403563+0000","last_change":"2024-04-25T10:17:38.222469+0000","last_active":"2024-04-25T10:17:39.403563+0000","last_peered":"2024-04-25T10:17:39.403563+0000","last_clean":"2024-04-25T10:17:39.403563+0000","last_became_active":"2024-04-25T10:17:38.221842+0000","last_became_peered":"2024-04-25T10:17:38.221842+0000","last_unstale":"2024-04-25T10:17:39.403563+0000","last_undegraded":"2024-04-25T10:17:39.403563+0000","last_fullsized":"2024-04-25T10:17:39.403563+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:59.194994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402120+0000","last_change":"2024-04-25T10:17:38.218561+0000","last_active":"2024-04-25T10:17:39.402120+0000","last_peered":"2024-04-25T10:17:39.402120+0000","last_clean":"2024-04-25T10:17:39.402120+0000","last_became_active":"2024-04-25T10:17:38.218060+0000","last_became_peered":"2024-04-25T10:17:38.218060+0000","last_unstale":"2024-04-25T10:17:39.402120+0000","last_undegraded":"2024-04-25T10:17:39.402120+0000","last_fullsized":"2024-04-25T10:17:39.402120+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:13.711019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.204671+0000","last_change":"2024-04-25T10:17:38.202476+0000","last_active":"2024-04-25T10:17:39.204671+0000","last_peered":"2024-04-25T10:17:39.204671+0000","last_clean":"2024-04-25T10:17:39.204671+0000","last_became_active":"2024-04-25T10:17:38.200141+0000","last_became_peered":"2024-04-25T10:17:38.200141+0000","last_unstale":"2024-04-25T10:17:39.204671+0000","last_undegraded":"2024-04-25T10:17:39.204671+0000","last_fullsized":"2024-04-25T10:17:39.204671+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:09.360593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403492+0000","last_change":"2024-04-25T10:17:38.209349+0000","last_active":"2024-04-25T10:17:39.403492+0000","last_peered":"2024-04-25T10:17:39.403492+0000","last_clean":"2024-04-25T10:17:39.403492+0000","last_became_active":"2024-04-25T10:17:38.209028+0000","last_became_peered":"2024-04-25T10:17:38.209028+0000","last_unstale":"2024-04-25T10:17:39.403492+0000","last_undegraded":"2024-04-25T10:17:39.403492+0000","last_fullsized":"2024-04-25T10:17:39.403492+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:54.663639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.207430+0000","last_change":"2024-04-25T10:17:38.209694+0000","last_active":"2024-04-25T10:17:39.207430+0000","last_peered":"2024-04-25T10:17:39.207430+0000","last_clean":"2024-04-25T10:17:39.207430+0000","last_became_active":"2024-04-25T10:17:38.209415+0000","last_became_peered":"2024-04-25T10:17:38.209415+0000","last_unstale":"2024-04-25T10:17:39.207430+0000","last_undegraded":"2024-04-25T10:17:39.207430+0000","last_fullsized":"2024-04-25T10:17:39.207430+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:11.558704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401779+0000","last_change":"2024-04-25T10:17:38.220327+0000","last_active":"2024-04-25T10:17:39.401779+0000","last_peered":"2024-04-25T10:17:39.401779+0000","last_clean":"2024-04-25T10:17:39.401779+0000","last_became_active":"2024-04-25T10:17:38.216829+0000","last_became_peered":"2024-04-25T10:17:38.216829+0000","last_unstale":"2024-04-25T10:17:39.401779+0000","last_undegraded":"2024-04-25T10:17:39.401779+0000","last_fullsized":"2024-04-25T10:17:39.401779+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:08.851641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402602+0000","last_change":"2024-04-25T10:17:38.197762+0000","last_active":"2024-04-25T10:17:39.402602+0000","last_peered":"2024-04-25T10:17:39.402602+0000","last_clean":"2024-04-25T10:17:39.402602+0000","last_became_active":"2024-04-25T10:17:38.197498+0000","last_became_peered":"2024-04-25T10:17:38.197498+0000","last_unstale":"2024-04-25T10:17:39.402602+0000","last_undegraded":"2024-04-25T10:17:39.402602+0000","last_fullsized":"2024-04-25T10:17:39.402602+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:18.242394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403028+0000","last_change":"2024-04-25T10:17:38.210396+0000","last_active":"2024-04-25T10:17:39.403028+0000","last_peered":"2024-04-25T10:17:39.403028+0000","last_clean":"2024-04-25T10:17:39.403028+0000","last_became_active":"2024-04-25T10:17:38.208324+0000","last_became_peered":"2024-04-25T10:17:38.208324+0000","last_unstale":"2024-04-25T10:17:39.403028+0000","last_undegraded":"2024-04-25T10:17:39.403028+0000","last_fullsized":"2024-04-25T10:17:39.403028+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:40.290291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.206561+0000","last_change":"2024-04-25T10:17:38.212504+0000","last_active":"2024-04-25T10:17:39.206561+0000","last_peered":"2024-04-25T10:17:39.206561+0000","last_clean":"2024-04-25T10:17:39.206561+0000","last_became_active":"2024-04-25T10:17:38.212196+0000","last_became_peered":"2024-04-25T10:17:38.212196+0000","last_unstale":"2024-04-25T10:17:39.206561+0000","last_undegraded":"2024-04-25T10:17:39.206561+0000","last_fullsized":"2024-04-25T10:17:39.206561+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:38.235200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402409+0000","last_change":"2024-04-25T10:17:38.216559+0000","last_active":"2024-04-25T10:17:39.402409+0000","last_peered":"2024-04-25T10:17:39.402409+0000","last_clean":"2024-04-25T10:17:39.402409+0000","last_became_active":"2024-04-25T10:17:38.214421+0000","last_became_peered":"2024-04-25T10:17:38.214421+0000","last_unstale":"2024-04-25T10:17:39.402409+0000","last_undegraded":"2024-04-25T10:17:39.402409+0000","last_fullsized":"2024-04-25T10:17:39.402409+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:23.758403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403496+0000","last_change":"2024-04-25T10:17:38.214026+0000","last_active":"2024-04-25T10:17:39.403496+0000","last_peered":"2024-04-25T10:17:39.403496+0000","last_clean":"2024-04-25T10:17:39.403496+0000","last_became_active":"2024-04-25T10:17:38.213630+0000","last_became_peered":"2024-04-25T10:17:38.213630+0000","last_unstale":"2024-04-25T10:17:39.403496+0000","last_undegraded":"2024-04-25T10:17:39.403496+0000","last_fullsized":"2024-04-25T10:17:39.403496+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:19.386704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402234+0000","last_change":"2024-04-25T10:17:38.218642+0000","last_active":"2024-04-25T10:17:39.402234+0000","last_peered":"2024-04-25T10:17:39.402234+0000","last_clean":"2024-04-25T10:17:39.402234+0000","last_became_active":"2024-04-25T10:17:38.218293+0000","last_became_peered":"2024-04-25T10:17:38.218293+0000","last_unstale":"2024-04-25T10:17:39.402234+0000","last_undegraded":"2024-04-25T10:17:39.402234+0000","last_fullsized":"2024-04-25T10:17:39.402234+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:17.188112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.400936+0000","last_change":"2024-04-25T10:17:38.210930+0000","last_active":"2024-04-25T10:17:39.400936+0000","last_peered":"2024-04-25T10:17:39.400936+0000","last_clean":"2024-04-25T10:17:39.400936+0000","last_became_active":"2024-04-25T10:17:38.210672+0000","last_became_peered":"2024-04-25T10:17:38.210672+0000","last_unstale":"2024-04-25T10:17:39.400936+0000","last_undegraded":"2024-04-25T10:17:39.400936+0000","last_fullsized":"2024-04-25T10:17:39.400936+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:59.916801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401005+0000","last_change":"2024-04-25T10:17:38.215806+0000","last_active":"2024-04-25T10:17:39.401005+0000","last_peered":"2024-04-25T10:17:39.401005+0000","last_clean":"2024-04-25T10:17:39.401005+0000","last_became_active":"2024-04-25T10:17:38.214746+0000","last_became_peered":"2024-04-25T10:17:38.214746+0000","last_unstale":"2024-04-25T10:17:39.401005+0000","last_undegraded":"2024-04-25T10:17:39.401005+0000","last_fullsized":"2024-04-25T10:17:39.401005+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:18.847917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.206536+0000","last_change":"2024-04-25T10:17:38.214985+0000","last_active":"2024-04-25T10:17:39.206536+0000","last_peered":"2024-04-25T10:17:39.206536+0000","last_clean":"2024-04-25T10:17:39.206536+0000","last_became_active":"2024-04-25T10:17:38.214590+0000","last_became_peered":"2024-04-25T10:17:38.214590+0000","last_unstale":"2024-04-25T10:17:39.206536+0000","last_undegraded":"2024-04-25T10:17:39.206536+0000","last_fullsized":"2024-04-25T10:17:39.206536+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:45.676668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401514+0000","last_change":"2024-04-25T10:17:38.220631+0000","last_active":"2024-04-25T10:17:39.401514+0000","last_peered":"2024-04-25T10:17:39.401514+0000","last_clean":"2024-04-25T10:17:39.401514+0000","last_became_active":"2024-04-25T10:17:38.220228+0000","last_became_peered":"2024-04-25T10:17:38.220228+0000","last_unstale":"2024-04-25T10:17:39.401514+0000","last_undegraded":"2024-04-25T10:17:39.401514+0000","last_fullsized":"2024-04-25T10:17:39.401514+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:22.321963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403618+0000","last_change":"2024-04-25T10:17:38.218236+0000","last_active":"2024-04-25T10:17:39.403618+0000","last_peered":"2024-04-25T10:17:39.403618+0000","last_clean":"2024-04-25T10:17:39.403618+0000","last_became_active":"2024-04-25T10:17:38.216333+0000","last_became_peered":"2024-04-25T10:17:38.216333+0000","last_unstale":"2024-04-25T10:17:39.403618+0000","last_undegraded":"2024-04-25T10:17:39.403618+0000","last_fullsized":"2024-04-25T10:17:39.403618+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:45.676668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.206273+0000","last_change":"2024-04-25T10:17:38.219695+0000","last_active":"2024-04-25T10:17:39.206273+0000","last_peered":"2024-04-25T10:17:39.206273+0000","last_clean":"2024-04-25T10:17:39.206273+0000","last_became_active":"2024-04-25T10:17:38.215990+0000","last_became_peered":"2024-04-25T10:17:38.215990+0000","last_unstale":"2024-04-25T10:17:39.206273+0000","last_undegraded":"2024-04-25T10:17:39.206273+0000","last_fullsized":"2024-04-25T10:17:39.206273+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:19.386704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.206364+0000","last_change":"2024-04-25T10:17:38.204217+0000","last_active":"2024-04-25T10:17:39.206364+0000","last_peered":"2024-04-25T10:17:39.206364+0000","last_clean":"2024-04-25T10:17:39.206364+0000","last_became_active":"2024-04-25T10:17:38.203910+0000","last_became_peered":"2024-04-25T10:17:38.203910+0000","last_unstale":"2024-04-25T10:17:39.206364+0000","last_undegraded":"2024-04-25T10:17:39.206364+0000","last_fullsized":"2024-04-25T10:17:39.206364+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:59.194994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401441+0000","last_change":"2024-04-25T10:17:38.214776+0000","last_active":"2024-04-25T10:17:39.401441+0000","last_peered":"2024-04-25T10:17:39.401441+0000","last_clean":"2024-04-25T10:17:39.401441+0000","last_became_active":"2024-04-25T10:17:38.214511+0000","last_became_peered":"2024-04-25T10:17:38.214511+0000","last_unstale":"2024-04-25T10:17:39.401441+0000","last_undegraded":"2024-04-25T10:17:39.401441+0000","last_fullsized":"2024-04-25T10:17:39.401441+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:05.814548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205348+0000","last_change":"2024-04-25T10:17:38.210903+0000","last_active":"2024-04-25T10:17:39.205348+0000","last_peered":"2024-04-25T10:17:39.205348+0000","last_clean":"2024-04-25T10:17:39.205348+0000","last_became_active":"2024-04-25T10:17:38.210507+0000","last_became_peered":"2024-04-25T10:17:38.210507+0000","last_unstale":"2024-04-25T10:17:39.205348+0000","last_undegraded":"2024-04-25T10:17:39.205348+0000","last_fullsized":"2024-04-25T10:17:39.205348+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:17.188112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.400371+0000","last_change":"2024-04-25T10:17:38.211219+0000","last_active":"2024-04-25T10:17:39.400371+0000","last_peered":"2024-04-25T10:17:39.400371+0000","last_clean":"2024-04-25T10:17:39.400371+0000","last_became_active":"2024-04-25T10:17:38.210057+0000","last_became_peered":"2024-04-25T10:17:38.210057+0000","last_unstale":"2024-04-25T10:17:39.400371+0000","last_undegraded":"2024-04-25T10:17:39.400371+0000","last_fullsized":"2024-04-25T10:17:39.400371+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:39.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.206647+0000","last_change":"2024-04-25T10:17:38.214429+0000","last_active":"2024-04-25T10:17:39.206647+0000","last_peered":"2024-04-25T10:17:39.206647+0000","last_clean":"2024-04-25T10:17:39.206647+0000","last_became_active":"2024-04-25T10:17:38.213642+0000","last_became_peered":"2024-04-25T10:17:38.213642+0000","last_unstale":"2024-04-25T10:17:39.206647+0000","last_undegraded":"2024-04-25T10:17:39.206647+0000","last_fullsized":"2024-04-25T10:17:39.206647+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:39.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403197+0000","last_change":"2024-04-25T10:17:38.198663+0000","last_active":"2024-04-25T10:17:39.403197+0000","last_peered":"2024-04-25T10:17:39.403197+0000","last_clean":"2024-04-25T10:17:39.403197+0000","last_became_active":"2024-04-25T10:17:38.198358+0000","last_became_peered":"2024-04-25T10:17:38.198358+0000","last_unstale":"2024-04-25T10:17:39.403197+0000","last_undegraded":"2024-04-25T10:17:39.403197+0000","last_fullsized":"2024-04-25T10:17:39.403197+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:06.405918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.405182+0000","last_change":"2024-04-25T10:17:38.218539+0000","last_active":"2024-04-25T10:17:39.405182+0000","last_peered":"2024-04-25T10:17:39.405182+0000","last_clean":"2024-04-25T10:17:39.405182+0000","last_became_active":"2024-04-25T10:17:38.218282+0000","last_became_peered":"2024-04-25T10:17:38.218282+0000","last_unstale":"2024-04-25T10:17:39.405182+0000","last_undegraded":"2024-04-25T10:17:39.405182+0000","last_fullsized":"2024-04-25T10:17:39.405182+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:11.558704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.404740+0000","last_change":"2024-04-25T10:17:38.203527+0000","last_active":"2024-04-25T10:17:39.404740+0000","last_peered":"2024-04-25T10:17:39.404740+0000","last_clean":"2024-04-25T10:17:39.404740+0000","last_became_active":"2024-04-25T10:17:38.203261+0000","last_became_peered":"2024-04-25T10:17:38.203261+0000","last_unstale":"2024-04-25T10:17:39.404740+0000","last_undegraded":"2024-04-25T10:17:39.404740+0000","last_fullsized":"2024-04-25T10:17:39.404740+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:57.970904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205277+0000","last_change":"2024-04-25T10:17:38.216263+0000","last_active":"2024-04-25T10:17:39.205277+0000","last_peered":"2024-04-25T10:17:39.205277+0000","last_clean":"2024-04-25T10:17:39.205277+0000","last_became_active":"2024-04-25T10:17:38.215959+0000","last_became_peered":"2024-04-25T10:17:38.215959+0000","last_unstale":"2024-04-25T10:17:39.205277+0000","last_undegraded":"2024-04-25T10:17:39.205277+0000","last_fullsized":"2024-04-25T10:17:39.205277+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.703896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.208048+0000","last_change":"2024-04-25T10:17:38.194952+0000","last_active":"2024-04-25T10:17:39.208048+0000","last_peered":"2024-04-25T10:17:39.208048+0000","last_clean":"2024-04-25T10:17:39.208048+0000","last_became_active":"2024-04-25T10:17:38.194122+0000","last_became_peered":"2024-04-25T10:17:38.194122+0000","last_unstale":"2024-04-25T10:17:39.208048+0000","last_undegraded":"2024-04-25T10:17:39.208048+0000","last_fullsized":"2024-04-25T10:17:39.208048+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:55.995614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403362+0000","last_change":"2024-04-25T10:17:38.202503+0000","last_active":"2024-04-25T10:17:39.403362+0000","last_peered":"2024-04-25T10:17:39.403362+0000","last_clean":"2024-04-25T10:17:39.403362+0000","last_became_active":"2024-04-25T10:17:38.202192+0000","last_became_peered":"2024-04-25T10:17:38.202192+0000","last_unstale":"2024-04-25T10:17:39.403362+0000","last_undegraded":"2024-04-25T10:17:39.403362+0000","last_fullsized":"2024-04-25T10:17:39.403362+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:13.921165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402388+0000","last_change":"2024-04-25T10:17:38.204953+0000","last_active":"2024-04-25T10:17:39.402388+0000","last_peered":"2024-04-25T10:17:39.402388+0000","last_clean":"2024-04-25T10:17:39.402388+0000","last_became_active":"2024-04-25T10:17:38.204757+0000","last_became_peered":"2024-04-25T10:17:38.204757+0000","last_unstale":"2024-04-25T10:17:39.402388+0000","last_undegraded":"2024-04-25T10:17:39.402388+0000","last_fullsized":"2024-04-25T10:17:39.402388+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:39.345689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.405849+0000","last_change":"2024-04-25T10:17:38.210354+0000","last_active":"2024-04-25T10:17:39.405849+0000","last_peered":"2024-04-25T10:17:39.405849+0000","last_clean":"2024-04-25T10:17:39.405849+0000","last_became_active":"2024-04-25T10:17:38.208585+0000","last_became_peered":"2024-04-25T10:17:38.208585+0000","last_unstale":"2024-04-25T10:17:39.405849+0000","last_undegraded":"2024-04-25T10:17:39.405849+0000","last_fullsized":"2024-04-25T10:17:39.405849+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:17.015596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"107'4","reported_seq":18,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.208583+0000","last_change":"2024-04-25T10:17:37.187966+0000","last_active":"2024-04-25T10:17:39.208583+0000","last_peered":"2024-04-25T10:17:39.208583+0000","last_clean":"2024-04-25T10:17:39.208583+0000","last_became_active":"2024-04-25T10:17:37.187662+0000","last_became_peered":"2024-04-25T10:17:37.187662+0000","last_unstale":"2024-04-25T10:17:39.208583+0000","last_undegraded":"2024-04-25T10:17:39.208583+0000","last_fullsized":"2024-04-25T10:17:39.208583+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:23.899400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402667+0000","last_change":"2024-04-25T10:17:38.220233+0000","last_active":"2024-04-25T10:17:39.402667+0000","last_peered":"2024-04-25T10:17:39.402667+0000","last_clean":"2024-04-25T10:17:39.402667+0000","last_became_active":"2024-04-25T10:17:38.219933+0000","last_became_peered":"2024-04-25T10:17:38.219933+0000","last_unstale":"2024-04-25T10:17:39.402667+0000","last_undegraded":"2024-04-25T10:17:39.402667+0000","last_fullsized":"2024-04-25T10:17:39.402667+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:17.188112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403322+0000","last_change":"2024-04-25T10:17:37.183153+0000","last_active":"2024-04-25T10:17:39.403322+0000","last_peered":"2024-04-25T10:17:39.403322+0000","last_clean":"2024-04-25T10:17:39.403322+0000","last_became_active":"2024-04-25T10:17:37.182528+0000","last_became_peered":"2024-04-25T10:17:37.182528+0000","last_unstale":"2024-04-25T10:17:39.403322+0000","last_undegraded":"2024-04-25T10:17:39.403322+0000","last_fullsized":"2024-04-25T10:17:39.403322+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:47.774882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.405425+0000","last_change":"2024-04-25T10:17:38.216108+0000","last_active":"2024-04-25T10:17:39.405425+0000","last_peered":"2024-04-25T10:17:39.405425+0000","last_clean":"2024-04-25T10:17:39.405425+0000","last_became_active":"2024-04-25T10:17:38.215831+0000","last_became_peered":"2024-04-25T10:17:38.215831+0000","last_unstale":"2024-04-25T10:17:39.405425+0000","last_undegraded":"2024-04-25T10:17:39.405425+0000","last_fullsized":"2024-04-25T10:17:39.405425+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:55.995614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205096+0000","last_change":"2024-04-25T10:17:37.193464+0000","last_active":"2024-04-25T10:17:39.205096+0000","last_peered":"2024-04-25T10:17:39.205096+0000","last_clean":"2024-04-25T10:17:39.205096+0000","last_became_active":"2024-04-25T10:17:37.193022+0000","last_became_peered":"2024-04-25T10:17:37.193022+0000","last_unstale":"2024-04-25T10:17:39.205096+0000","last_undegraded":"2024-04-25T10:17:39.205096+0000","last_fullsized":"2024-04-25T10:17:39.205096+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:06.291123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403983+0000","last_change":"2024-04-25T10:17:38.220558+0000","last_active":"2024-04-25T10:17:39.403983+0000","last_peered":"2024-04-25T10:17:39.403983+0000","last_clean":"2024-04-25T10:17:39.403983+0000","last_became_active":"2024-04-25T10:17:38.216848+0000","last_became_peered":"2024-04-25T10:17:38.216848+0000","last_unstale":"2024-04-25T10:17:39.403983+0000","last_undegraded":"2024-04-25T10:17:39.403983+0000","last_fullsized":"2024-04-25T10:17:39.403983+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:39.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.204770+0000","last_change":"2024-04-25T10:17:37.189460+0000","last_active":"2024-04-25T10:17:39.204770+0000","last_peered":"2024-04-25T10:17:39.204770+0000","last_clean":"2024-04-25T10:17:39.204770+0000","last_became_active":"2024-04-25T10:17:37.189221+0000","last_became_peered":"2024-04-25T10:17:37.189221+0000","last_unstale":"2024-04-25T10:17:39.204770+0000","last_undegraded":"2024-04-25T10:17:39.204770+0000","last_fullsized":"2024-04-25T10:17:39.204770+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:10.284592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401484+0000","last_change":"2024-04-25T10:17:38.196915+0000","last_active":"2024-04-25T10:17:39.401484+0000","last_peered":"2024-04-25T10:17:39.401484+0000","last_clean":"2024-04-25T10:17:39.401484+0000","last_became_active":"2024-04-25T10:17:38.196634+0000","last_became_peered":"2024-04-25T10:17:38.196634+0000","last_unstale":"2024-04-25T10:17:39.401484+0000","last_undegraded":"2024-04-25T10:17:39.401484+0000","last_fullsized":"2024-04-25T10:17:39.401484+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:12.410162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.207769+0000","last_change":"2024-04-25T10:17:37.189092+0000","last_active":"2024-04-25T10:17:39.207769+0000","last_peered":"2024-04-25T10:17:39.207769+0000","last_clean":"2024-04-25T10:17:39.207769+0000","last_became_active":"2024-04-25T10:17:37.188753+0000","last_became_peered":"2024-04-25T10:17:37.188753+0000","last_unstale":"2024-04-25T10:17:39.207769+0000","last_undegraded":"2024-04-25T10:17:39.207769+0000","last_fullsized":"2024-04-25T10:17:39.207769+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:45.834637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205754+0000","last_change":"2024-04-25T10:17:38.208949+0000","last_active":"2024-04-25T10:17:39.205754+0000","last_peered":"2024-04-25T10:17:39.205754+0000","last_clean":"2024-04-25T10:17:39.205754+0000","last_became_active":"2024-04-25T10:17:38.208596+0000","last_became_peered":"2024-04-25T10:17:38.208596+0000","last_unstale":"2024-04-25T10:17:39.205754+0000","last_undegraded":"2024-04-25T10:17:39.205754+0000","last_fullsized":"2024-04-25T10:17:39.205754+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:41.678472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403421+0000","last_change":"2024-04-25T10:17:37.186303+0000","last_active":"2024-04-25T10:17:39.403421+0000","last_peered":"2024-04-25T10:17:39.403421+0000","last_clean":"2024-04-25T10:17:39.403421+0000","last_became_active":"2024-04-25T10:17:37.185980+0000","last_became_peered":"2024-04-25T10:17:37.185980+0000","last_unstale":"2024-04-25T10:17:39.403421+0000","last_undegraded":"2024-04-25T10:17:39.403421+0000","last_fullsized":"2024-04-25T10:17:39.403421+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:56.567178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401436+0000","last_change":"2024-04-25T10:17:38.214768+0000","last_active":"2024-04-25T10:17:39.401436+0000","last_peered":"2024-04-25T10:17:39.401436+0000","last_clean":"2024-04-25T10:17:39.401436+0000","last_became_active":"2024-04-25T10:17:38.213620+0000","last_became_peered":"2024-04-25T10:17:38.213620+0000","last_unstale":"2024-04-25T10:17:39.401436+0000","last_undegraded":"2024-04-25T10:17:39.401436+0000","last_fullsized":"2024-04-25T10:17:39.401436+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:46.154221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205149+0000","last_change":"2024-04-25T10:17:37.177115+0000","last_active":"2024-04-25T10:17:39.205149+0000","last_peered":"2024-04-25T10:17:39.205149+0000","last_clean":"2024-04-25T10:17:39.205149+0000","last_became_active":"2024-04-25T10:17:37.176679+0000","last_became_peered":"2024-04-25T10:17:37.176679+0000","last_unstale":"2024-04-25T10:17:39.205149+0000","last_undegraded":"2024-04-25T10:17:39.205149+0000","last_fullsized":"2024-04-25T10:17:39.205149+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:58.089525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.206915+0000","last_change":"2024-04-25T10:17:38.209909+0000","last_active":"2024-04-25T10:17:39.206915+0000","last_peered":"2024-04-25T10:17:39.206915+0000","last_clean":"2024-04-25T10:17:39.206915+0000","last_became_active":"2024-04-25T10:17:38.209593+0000","last_became_peered":"2024-04-25T10:17:38.209593+0000","last_unstale":"2024-04-25T10:17:39.206915+0000","last_undegraded":"2024-04-25T10:17:39.206915+0000","last_fullsized":"2024-04-25T10:17:39.206915+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:54.663639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401879+0000","last_change":"2024-04-25T10:17:37.180701+0000","last_active":"2024-04-25T10:17:39.401879+0000","last_peered":"2024-04-25T10:17:39.401879+0000","last_clean":"2024-04-25T10:17:39.401879+0000","last_became_active":"2024-04-25T10:17:37.180439+0000","last_became_peered":"2024-04-25T10:17:37.180439+0000","last_unstale":"2024-04-25T10:17:39.401879+0000","last_undegraded":"2024-04-25T10:17:39.401879+0000","last_fullsized":"2024-04-25T10:17:39.401879+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:38.048694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.209130+0000","last_change":"2024-04-25T10:17:38.210858+0000","last_active":"2024-04-25T10:17:39.209130+0000","last_peered":"2024-04-25T10:17:39.209130+0000","last_clean":"2024-04-25T10:17:39.209130+0000","last_became_active":"2024-04-25T10:17:38.208935+0000","last_became_peered":"2024-04-25T10:17:38.208935+0000","last_unstale":"2024-04-25T10:17:39.209130+0000","last_undegraded":"2024-04-25T10:17:39.209130+0000","last_fullsized":"2024-04-25T10:17:39.209130+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:38.268735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205300+0000","last_change":"2024-04-25T10:17:37.190960+0000","last_active":"2024-04-25T10:17:39.205300+0000","last_peered":"2024-04-25T10:17:39.205300+0000","last_clean":"2024-04-25T10:17:39.205300+0000","last_became_active":"2024-04-25T10:17:37.189950+0000","last_became_peered":"2024-04-25T10:17:37.189950+0000","last_unstale":"2024-04-25T10:17:39.205300+0000","last_undegraded":"2024-04-25T10:17:39.205300+0000","last_fullsized":"2024-04-25T10:17:39.205300+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:45.683871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402477+0000","last_change":"2024-04-25T10:17:38.219954+0000","last_active":"2024-04-25T10:17:39.402477+0000","last_peered":"2024-04-25T10:17:39.402477+0000","last_clean":"2024-04-25T10:17:39.402477+0000","last_became_active":"2024-04-25T10:17:38.219608+0000","last_became_peered":"2024-04-25T10:17:38.219608+0000","last_unstale":"2024-04-25T10:17:39.402477+0000","last_undegraded":"2024-04-25T10:17:39.402477+0000","last_fullsized":"2024-04-25T10:17:39.402477+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:48.495338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.405481+0000","last_change":"2024-04-25T10:17:37.185651+0000","last_active":"2024-04-25T10:17:39.405481+0000","last_peered":"2024-04-25T10:17:39.405481+0000","last_clean":"2024-04-25T10:17:39.405481+0000","last_became_active":"2024-04-25T10:17:37.185320+0000","last_became_peered":"2024-04-25T10:17:37.185320+0000","last_unstale":"2024-04-25T10:17:39.405481+0000","last_undegraded":"2024-04-25T10:17:39.405481+0000","last_fullsized":"2024-04-25T10:17:39.405481+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:26.517389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.404730+0000","last_change":"2024-04-25T10:17:38.209654+0000","last_active":"2024-04-25T10:17:39.404730+0000","last_peered":"2024-04-25T10:17:39.404730+0000","last_clean":"2024-04-25T10:17:39.404730+0000","last_became_active":"2024-04-25T10:17:38.207818+0000","last_became_peered":"2024-04-25T10:17:38.207818+0000","last_unstale":"2024-04-25T10:17:39.404730+0000","last_undegraded":"2024-04-25T10:17:39.404730+0000","last_fullsized":"2024-04-25T10:17:39.404730+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:02.724367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"107'3","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402753+0000","last_change":"2024-04-25T10:17:37.193973+0000","last_active":"2024-04-25T10:17:39.402753+0000","last_peered":"2024-04-25T10:17:39.402753+0000","last_clean":"2024-04-25T10:17:39.402753+0000","last_became_active":"2024-04-25T10:17:37.193581+0000","last_became_peered":"2024-04-25T10:17:37.193581+0000","last_unstale":"2024-04-25T10:17:39.402753+0000","last_undegraded":"2024-04-25T10:17:39.402753+0000","last_fullsized":"2024-04-25T10:17:39.402753+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:37.220423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.208393+0000","last_change":"2024-04-25T10:17:38.218063+0000","last_active":"2024-04-25T10:17:39.208393+0000","last_peered":"2024-04-25T10:17:39.208393+0000","last_clean":"2024-04-25T10:17:39.208393+0000","last_became_active":"2024-04-25T10:17:38.216415+0000","last_became_peered":"2024-04-25T10:17:38.216415+0000","last_unstale":"2024-04-25T10:17:39.208393+0000","last_undegraded":"2024-04-25T10:17:39.208393+0000","last_fullsized":"2024-04-25T10:17:39.208393+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:17.015596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"107'6","reported_seq":20,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403090+0000","last_change":"2024-04-25T10:17:37.175580+0000","last_active":"2024-04-25T10:17:39.403090+0000","last_peered":"2024-04-25T10:17:39.403090+0000","last_clean":"2024-04-25T10:17:39.403090+0000","last_became_active":"2024-04-25T10:17:37.175291+0000","last_became_peered":"2024-04-25T10:17:37.175291+0000","last_unstale":"2024-04-25T10:17:39.403090+0000","last_undegraded":"2024-04-25T10:17:39.403090+0000","last_fullsized":"2024-04-25T10:17:39.403090+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.516817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401531+0000","last_change":"2024-04-25T10:17:38.197921+0000","last_active":"2024-04-25T10:17:39.401531+0000","last_peered":"2024-04-25T10:17:39.401531+0000","last_clean":"2024-04-25T10:17:39.401531+0000","last_became_active":"2024-04-25T10:17:38.197593+0000","last_became_peered":"2024-04-25T10:17:38.197593+0000","last_unstale":"2024-04-25T10:17:39.401531+0000","last_undegraded":"2024-04-25T10:17:39.401531+0000","last_fullsized":"2024-04-25T10:17:39.401531+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:47.608829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.404328+0000","last_change":"2024-04-25T10:17:37.187413+0000","last_active":"2024-04-25T10:17:39.404328+0000","last_peered":"2024-04-25T10:17:39.404328+0000","last_clean":"2024-04-25T10:17:39.404328+0000","last_became_active":"2024-04-25T10:17:37.187149+0000","last_became_peered":"2024-04-25T10:17:37.187149+0000","last_unstale":"2024-04-25T10:17:39.404328+0000","last_undegraded":"2024-04-25T10:17:39.404328+0000","last_fullsized":"2024-04-25T10:17:39.404328+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:26.239232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403730+0000","last_change":"2024-04-25T10:17:38.212733+0000","last_active":"2024-04-25T10:17:39.403730+0000","last_peered":"2024-04-25T10:17:39.403730+0000","last_clean":"2024-04-25T10:17:39.403730+0000","last_became_active":"2024-04-25T10:17:38.209988+0000","last_became_peered":"2024-04-25T10:17:38.209988+0000","last_unstale":"2024-04-25T10:17:39.403730+0000","last_undegraded":"2024-04-25T10:17:39.403730+0000","last_fullsized":"2024-04-25T10:17:39.403730+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:24.980261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401978+0000","last_change":"2024-04-25T10:17:37.194785+0000","last_active":"2024-04-25T10:17:39.401978+0000","last_peered":"2024-04-25T10:17:39.401978+0000","last_clean":"2024-04-25T10:17:39.401978+0000","last_became_active":"2024-04-25T10:17:37.194494+0000","last_became_peered":"2024-04-25T10:17:37.194494+0000","last_unstale":"2024-04-25T10:17:39.401978+0000","last_undegraded":"2024-04-25T10:17:39.401978+0000","last_fullsized":"2024-04-25T10:17:39.401978+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:22.440679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.400607+0000","last_change":"2024-04-25T10:17:38.206028+0000","last_active":"2024-04-25T10:17:39.400607+0000","last_peered":"2024-04-25T10:17:39.400607+0000","last_clean":"2024-04-25T10:17:39.400607+0000","last_became_active":"2024-04-25T10:17:38.205789+0000","last_became_peered":"2024-04-25T10:17:38.205789+0000","last_unstale":"2024-04-25T10:17:39.400607+0000","last_undegraded":"2024-04-25T10:17:39.400607+0000","last_fullsized":"2024-04-25T10:17:39.400607+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:46.798286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"107'8","reported_seq":22,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.204418+0000","last_change":"2024-04-25T10:17:37.181066+0000","last_active":"2024-04-25T10:17:39.204418+0000","last_peered":"2024-04-25T10:17:39.204418+0000","last_clean":"2024-04-25T10:17:39.204418+0000","last_became_active":"2024-04-25T10:17:37.180791+0000","last_became_peered":"2024-04-25T10:17:37.180791+0000","last_unstale":"2024-04-25T10:17:39.204418+0000","last_undegraded":"2024-04-25T10:17:39.204418+0000","last_fullsized":"2024-04-25T10:17:39.204418+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:53.580198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205393+0000","last_change":"2024-04-25T10:17:38.205651+0000","last_active":"2024-04-25T10:17:39.205393+0000","last_peered":"2024-04-25T10:17:39.205393+0000","last_clean":"2024-04-25T10:17:39.205393+0000","last_became_active":"2024-04-25T10:17:38.205300+0000","last_became_peered":"2024-04-25T10:17:38.205300+0000","last_unstale":"2024-04-25T10:17:39.205393+0000","last_undegraded":"2024-04-25T10:17:39.205393+0000","last_fullsized":"2024-04-25T10:17:39.205393+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:11.812895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402945+0000","last_change":"2024-04-25T10:17:37.180200+0000","last_active":"2024-04-25T10:17:39.402945+0000","last_peered":"2024-04-25T10:17:39.402945+0000","last_clean":"2024-04-25T10:17:39.402945+0000","last_became_active":"2024-04-25T10:17:37.179916+0000","last_became_peered":"2024-04-25T10:17:37.179916+0000","last_unstale":"2024-04-25T10:17:39.402945+0000","last_undegraded":"2024-04-25T10:17:39.402945+0000","last_fullsized":"2024-04-25T10:17:39.402945+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:44.661891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.406014+0000","last_change":"2024-04-25T10:17:38.214695+0000","last_active":"2024-04-25T10:17:39.406014+0000","last_peered":"2024-04-25T10:17:39.406014+0000","last_clean":"2024-04-25T10:17:39.406014+0000","last_became_active":"2024-04-25T10:17:38.214425+0000","last_became_peered":"2024-04-25T10:17:38.214425+0000","last_unstale":"2024-04-25T10:17:39.406014+0000","last_undegraded":"2024-04-25T10:17:39.406014+0000","last_fullsized":"2024-04-25T10:17:39.406014+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:38.268735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205289+0000","last_change":"2024-04-25T10:17:37.182981+0000","last_active":"2024-04-25T10:17:39.205289+0000","last_peered":"2024-04-25T10:17:39.205289+0000","last_clean":"2024-04-25T10:17:39.205289+0000","last_became_active":"2024-04-25T10:17:37.182698+0000","last_became_peered":"2024-04-25T10:17:37.182698+0000","last_unstale":"2024-04-25T10:17:39.205289+0000","last_undegraded":"2024-04-25T10:17:39.205289+0000","last_fullsized":"2024-04-25T10:17:39.205289+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:46.896358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403248+0000","last_change":"2024-04-25T10:17:38.223133+0000","last_active":"2024-04-25T10:17:39.403248+0000","last_peered":"2024-04-25T10:17:39.403248+0000","last_clean":"2024-04-25T10:17:39.403248+0000","last_became_active":"2024-04-25T10:17:38.222857+0000","last_became_peered":"2024-04-25T10:17:38.222857+0000","last_unstale":"2024-04-25T10:17:39.403248+0000","last_undegraded":"2024-04-25T10:17:39.403248+0000","last_fullsized":"2024-04-25T10:17:39.403248+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:11.265042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401683+0000","last_change":"2024-04-25T10:17:37.184886+0000","last_active":"2024-04-25T10:17:39.401683+0000","last_peered":"2024-04-25T10:17:39.401683+0000","last_clean":"2024-04-25T10:17:39.401683+0000","last_became_active":"2024-04-25T10:17:37.184114+0000","last_became_peered":"2024-04-25T10:17:37.184114+0000","last_unstale":"2024-04-25T10:17:39.401683+0000","last_undegraded":"2024-04-25T10:17:39.401683+0000","last_fullsized":"2024-04-25T10:17:39.401683+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:29.558500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403886+0000","last_change":"2024-04-25T10:17:38.221129+0000","last_active":"2024-04-25T10:17:39.403886+0000","last_peered":"2024-04-25T10:17:39.403886+0000","last_clean":"2024-04-25T10:17:39.403886+0000","last_became_active":"2024-04-25T10:17:38.218848+0000","last_became_peered":"2024-04-25T10:17:38.218848+0000","last_unstale":"2024-04-25T10:17:39.403886+0000","last_undegraded":"2024-04-25T10:17:39.403886+0000","last_fullsized":"2024-04-25T10:17:39.403886+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:38.235200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.208064+0000","last_change":"2024-04-25T10:17:37.180225+0000","last_active":"2024-04-25T10:17:39.208064+0000","last_peered":"2024-04-25T10:17:39.208064+0000","last_clean":"2024-04-25T10:17:39.208064+0000","last_became_active":"2024-04-25T10:17:37.179313+0000","last_became_peered":"2024-04-25T10:17:37.179313+0000","last_unstale":"2024-04-25T10:17:39.208064+0000","last_undegraded":"2024-04-25T10:17:39.208064+0000","last_fullsized":"2024-04-25T10:17:39.208064+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:00.872347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402630+0000","last_change":"2024-04-25T10:17:38.221613+0000","last_active":"2024-04-25T10:17:39.402630+0000","last_peered":"2024-04-25T10:17:39.402630+0000","last_clean":"2024-04-25T10:17:39.402630+0000","last_became_active":"2024-04-25T10:17:38.221327+0000","last_became_peered":"2024-04-25T10:17:38.221327+0000","last_unstale":"2024-04-25T10:17:39.402630+0000","last_undegraded":"2024-04-25T10:17:39.402630+0000","last_fullsized":"2024-04-25T10:17:39.402630+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:41.678472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.404550+0000","last_change":"2024-04-25T10:17:37.190849+0000","last_active":"2024-04-25T10:17:39.404550+0000","last_peered":"2024-04-25T10:17:39.404550+0000","last_clean":"2024-04-25T10:17:39.404550+0000","last_became_active":"2024-04-25T10:17:37.190468+0000","last_became_peered":"2024-04-25T10:17:37.190468+0000","last_unstale":"2024-04-25T10:17:39.404550+0000","last_undegraded":"2024-04-25T10:17:39.404550+0000","last_fullsized":"2024-04-25T10:17:39.404550+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:29.385847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401961+0000","last_change":"2024-04-25T10:17:38.215758+0000","last_active":"2024-04-25T10:17:39.401961+0000","last_peered":"2024-04-25T10:17:39.401961+0000","last_clean":"2024-04-25T10:17:39.401961+0000","last_became_active":"2024-04-25T10:17:38.214126+0000","last_became_peered":"2024-04-25T10:17:38.214126+0000","last_unstale":"2024-04-25T10:17:39.401961+0000","last_undegraded":"2024-04-25T10:17:39.401961+0000","last_fullsized":"2024-04-25T10:17:39.401961+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:50.552440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.204318+0000","last_change":"2024-04-25T10:17:37.182941+0000","last_active":"2024-04-25T10:17:39.204318+0000","last_peered":"2024-04-25T10:17:39.204318+0000","last_clean":"2024-04-25T10:17:39.204318+0000","last_became_active":"2024-04-25T10:17:37.182664+0000","last_became_peered":"2024-04-25T10:17:37.182664+0000","last_unstale":"2024-04-25T10:17:39.204318+0000","last_undegraded":"2024-04-25T10:17:39.204318+0000","last_fullsized":"2024-04-25T10:17:39.204318+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:23.014898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402137+0000","last_change":"2024-04-25T10:17:38.218460+0000","last_active":"2024-04-25T10:17:39.402137+0000","last_peered":"2024-04-25T10:17:39.402137+0000","last_clean":"2024-04-25T10:17:39.402137+0000","last_became_active":"2024-04-25T10:17:38.216286+0000","last_became_peered":"2024-04-25T10:17:38.216286+0000","last_unstale":"2024-04-25T10:17:39.402137+0000","last_undegraded":"2024-04-25T10:17:39.402137+0000","last_fullsized":"2024-04-25T10:17:39.402137+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.703896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401099+0000","last_change":"2024-04-25T10:17:37.185649+0000","last_active":"2024-04-25T10:17:39.401099+0000","last_peered":"2024-04-25T10:17:39.401099+0000","last_clean":"2024-04-25T10:17:39.401099+0000","last_became_active":"2024-04-25T10:17:37.185131+0000","last_became_peered":"2024-04-25T10:17:37.185131+0000","last_unstale":"2024-04-25T10:17:39.401099+0000","last_undegraded":"2024-04-25T10:17:39.401099+0000","last_fullsized":"2024-04-25T10:17:39.401099+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:24.059107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.400701+0000","last_change":"2024-04-25T10:17:38.205960+0000","last_active":"2024-04-25T10:17:39.400701+0000","last_peered":"2024-04-25T10:17:39.400701+0000","last_clean":"2024-04-25T10:17:39.400701+0000","last_became_active":"2024-04-25T10:17:38.205692+0000","last_became_peered":"2024-04-25T10:17:38.205692+0000","last_unstale":"2024-04-25T10:17:39.400701+0000","last_undegraded":"2024-04-25T10:17:39.400701+0000","last_fullsized":"2024-04-25T10:17:39.400701+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:14.531595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401916+0000","last_change":"2024-04-25T10:17:37.189790+0000","last_active":"2024-04-25T10:17:39.401916+0000","last_peered":"2024-04-25T10:17:39.401916+0000","last_clean":"2024-04-25T10:17:39.401916+0000","last_became_active":"2024-04-25T10:17:37.189293+0000","last_became_peered":"2024-04-25T10:17:37.189293+0000","last_unstale":"2024-04-25T10:17:39.401916+0000","last_undegraded":"2024-04-25T10:17:39.401916+0000","last_fullsized":"2024-04-25T10:17:39.401916+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:18.371926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401601+0000","last_change":"2024-04-25T10:17:38.221000+0000","last_active":"2024-04-25T10:17:39.401601+0000","last_peered":"2024-04-25T10:17:39.401601+0000","last_clean":"2024-04-25T10:17:39.401601+0000","last_became_active":"2024-04-25T10:17:38.220643+0000","last_became_peered":"2024-04-25T10:17:38.220643+0000","last_unstale":"2024-04-25T10:17:39.401601+0000","last_undegraded":"2024-04-25T10:17:39.401601+0000","last_fullsized":"2024-04-25T10:17:39.401601+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.703896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.405081+0000","last_change":"2024-04-25T10:17:37.179612+0000","last_active":"2024-04-25T10:17:39.405081+0000","last_peered":"2024-04-25T10:17:39.405081+0000","last_clean":"2024-04-25T10:17:39.405081+0000","last_became_active":"2024-04-25T10:17:37.178581+0000","last_became_peered":"2024-04-25T10:17:37.178581+0000","last_unstale":"2024-04-25T10:17:39.405081+0000","last_undegraded":"2024-04-25T10:17:39.405081+0000","last_fullsized":"2024-04-25T10:17:39.405081+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:21.307185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402762+0000","last_change":"2024-04-25T10:17:38.207880+0000","last_active":"2024-04-25T10:17:39.402762+0000","last_peered":"2024-04-25T10:17:39.402762+0000","last_clean":"2024-04-25T10:17:39.402762+0000","last_became_active":"2024-04-25T10:17:38.207508+0000","last_became_peered":"2024-04-25T10:17:38.207508+0000","last_unstale":"2024-04-25T10:17:39.402762+0000","last_undegraded":"2024-04-25T10:17:39.402762+0000","last_fullsized":"2024-04-25T10:17:39.402762+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:57.876347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"108'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:40.192635+0000","last_change":"2024-04-25T10:17:37.190109+0000","last_active":"2024-04-25T10:17:40.192635+0000","last_peered":"2024-04-25T10:17:40.192635+0000","last_clean":"2024-04-25T10:17:40.192635+0000","last_became_active":"2024-04-25T10:17:37.189636+0000","last_became_peered":"2024-04-25T10:17:37.189636+0000","last_unstale":"2024-04-25T10:17:40.192635+0000","last_undegraded":"2024-04-25T10:17:40.192635+0000","last_fullsized":"2024-04-25T10:17:40.192635+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:28.999385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402804+0000","last_change":"2024-04-25T10:17:38.216837+0000","last_active":"2024-04-25T10:17:39.402804+0000","last_peered":"2024-04-25T10:17:39.402804+0000","last_clean":"2024-04-25T10:17:39.402804+0000","last_became_active":"2024-04-25T10:17:38.214897+0000","last_became_peered":"2024-04-25T10:17:38.214897+0000","last_unstale":"2024-04-25T10:17:39.402804+0000","last_undegraded":"2024-04-25T10:17:39.402804+0000","last_fullsized":"2024-04-25T10:17:39.402804+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:12.833042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403062+0000","last_change":"2024-04-25T10:17:37.190370+0000","last_active":"2024-04-25T10:17:39.403062+0000","last_peered":"2024-04-25T10:17:39.403062+0000","last_clean":"2024-04-25T10:17:39.403062+0000","last_became_active":"2024-04-25T10:17:37.190014+0000","last_became_peered":"2024-04-25T10:17:37.190014+0000","last_unstale":"2024-04-25T10:17:39.403062+0000","last_undegraded":"2024-04-25T10:17:39.403062+0000","last_fullsized":"2024-04-25T10:17:39.403062+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:45.783509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401565+0000","last_change":"2024-04-25T10:17:38.221421+0000","last_active":"2024-04-25T10:17:39.401565+0000","last_peered":"2024-04-25T10:17:39.401565+0000","last_clean":"2024-04-25T10:17:39.401565+0000","last_became_active":"2024-04-25T10:17:38.221021+0000","last_became_peered":"2024-04-25T10:17:38.221021+0000","last_unstale":"2024-04-25T10:17:39.401565+0000","last_undegraded":"2024-04-25T10:17:39.401565+0000","last_fullsized":"2024-04-25T10:17:39.401565+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:08.851641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.405090+0000","last_change":"2024-04-25T10:17:37.189210+0000","last_active":"2024-04-25T10:17:39.405090+0000","last_peered":"2024-04-25T10:17:39.405090+0000","last_clean":"2024-04-25T10:17:39.405090+0000","last_became_active":"2024-04-25T10:17:37.188916+0000","last_became_peered":"2024-04-25T10:17:37.188916+0000","last_unstale":"2024-04-25T10:17:39.405090+0000","last_undegraded":"2024-04-25T10:17:39.405090+0000","last_fullsized":"2024-04-25T10:17:39.405090+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:20.069783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205290+0000","last_change":"2024-04-25T10:17:38.201870+0000","last_active":"2024-04-25T10:17:39.205290+0000","last_peered":"2024-04-25T10:17:39.205290+0000","last_clean":"2024-04-25T10:17:39.205290+0000","last_became_active":"2024-04-25T10:17:38.201382+0000","last_became_peered":"2024-04-25T10:17:38.201382+0000","last_unstale":"2024-04-25T10:17:39.205290+0000","last_undegraded":"2024-04-25T10:17:39.205290+0000","last_fullsized":"2024-04-25T10:17:39.205290+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:48.495338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.207796+0000","last_change":"2024-04-25T10:17:37.181661+0000","last_active":"2024-04-25T10:17:39.207796+0000","last_peered":"2024-04-25T10:17:39.207796+0000","last_clean":"2024-04-25T10:17:39.207796+0000","last_became_active":"2024-04-25T10:17:37.181318+0000","last_became_peered":"2024-04-25T10:17:37.181318+0000","last_unstale":"2024-04-25T10:17:39.207796+0000","last_undegraded":"2024-04-25T10:17:39.207796+0000","last_fullsized":"2024-04-25T10:17:39.207796+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:43.543431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.206049+0000","last_change":"2024-04-25T10:17:38.206193+0000","last_active":"2024-04-25T10:17:39.206049+0000","last_peered":"2024-04-25T10:17:39.206049+0000","last_clean":"2024-04-25T10:17:39.206049+0000","last_became_active":"2024-04-25T10:17:38.205880+0000","last_became_peered":"2024-04-25T10:17:38.205880+0000","last_unstale":"2024-04-25T10:17:39.206049+0000","last_undegraded":"2024-04-25T10:17:39.206049+0000","last_fullsized":"2024-04-25T10:17:39.206049+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:11.265042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205299+0000","last_change":"2024-04-25T10:17:37.186571+0000","last_active":"2024-04-25T10:17:39.205299+0000","last_peered":"2024-04-25T10:17:39.205299+0000","last_clean":"2024-04-25T10:17:39.205299+0000","last_became_active":"2024-04-25T10:17:37.186241+0000","last_became_peered":"2024-04-25T10:17:37.186241+0000","last_unstale":"2024-04-25T10:17:39.205299+0000","last_undegraded":"2024-04-25T10:17:39.205299+0000","last_fullsized":"2024-04-25T10:17:39.205299+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:47.774882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402198+0000","last_change":"2024-04-25T10:17:38.219212+0000","last_active":"2024-04-25T10:17:39.402198+0000","last_peered":"2024-04-25T10:17:39.402198+0000","last_clean":"2024-04-25T10:17:39.402198+0000","last_became_active":"2024-04-25T10:17:38.218921+0000","last_became_peered":"2024-04-25T10:17:38.218921+0000","last_unstale":"2024-04-25T10:17:39.402198+0000","last_undegraded":"2024-04-25T10:17:39.402198+0000","last_fullsized":"2024-04-25T10:17:39.402198+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:48.495338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.204241+0000","last_change":"2024-04-25T10:17:37.186109+0000","last_active":"2024-04-25T10:17:39.204241+0000","last_peered":"2024-04-25T10:17:39.204241+0000","last_clean":"2024-04-25T10:17:39.204241+0000","last_became_active":"2024-04-25T10:17:37.185631+0000","last_became_peered":"2024-04-25T10:17:37.185631+0000","last_unstale":"2024-04-25T10:17:39.204241+0000","last_undegraded":"2024-04-25T10:17:39.204241+0000","last_fullsized":"2024-04-25T10:17:39.204241+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:35.821245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403395+0000","last_change":"2024-04-25T10:17:38.214549+0000","last_active":"2024-04-25T10:17:39.403395+0000","last_peered":"2024-04-25T10:17:39.403395+0000","last_clean":"2024-04-25T10:17:39.403395+0000","last_became_active":"2024-04-25T10:17:38.213672+0000","last_became_peered":"2024-04-25T10:17:38.213672+0000","last_unstale":"2024-04-25T10:17:39.403395+0000","last_undegraded":"2024-04-25T10:17:39.403395+0000","last_fullsized":"2024-04-25T10:17:39.403395+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:42.206645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402207+0000","last_change":"2024-04-25T10:17:37.189569+0000","last_active":"2024-04-25T10:17:39.402207+0000","last_peered":"2024-04-25T10:17:39.402207+0000","last_clean":"2024-04-25T10:17:39.402207+0000","last_became_active":"2024-04-25T10:17:37.188924+0000","last_became_peered":"2024-04-25T10:17:37.188924+0000","last_unstale":"2024-04-25T10:17:39.402207+0000","last_undegraded":"2024-04-25T10:17:39.402207+0000","last_fullsized":"2024-04-25T10:17:39.402207+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:58.180216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.400118+0000","last_change":"2024-04-25T10:17:38.211104+0000","last_active":"2024-04-25T10:17:39.400118+0000","last_peered":"2024-04-25T10:17:39.400118+0000","last_clean":"2024-04-25T10:17:39.400118+0000","last_became_active":"2024-04-25T10:17:38.209662+0000","last_became_peered":"2024-04-25T10:17:38.209662+0000","last_unstale":"2024-04-25T10:17:39.400118+0000","last_undegraded":"2024-04-25T10:17:39.400118+0000","last_fullsized":"2024-04-25T10:17:39.400118+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:38.235200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.207252+0000","last_change":"2024-04-25T10:17:37.185032+0000","last_active":"2024-04-25T10:17:39.207252+0000","last_peered":"2024-04-25T10:17:39.207252+0000","last_clean":"2024-04-25T10:17:39.207252+0000","last_became_active":"2024-04-25T10:17:37.184715+0000","last_became_peered":"2024-04-25T10:17:37.184715+0000","last_unstale":"2024-04-25T10:17:39.207252+0000","last_undegraded":"2024-04-25T10:17:39.207252+0000","last_fullsized":"2024-04-25T10:17:39.207252+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:13.304049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":337,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402421+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:17:39.402421+0000","last_peered":"2024-04-25T10:17:39.402421+0000","last_clean":"2024-04-25T10:17:39.402421+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:17:39.402421+0000","last_undegraded":"2024-04-25T10:17:39.402421+0000","last_fullsized":"2024-04-25T10:17:39.402421+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1945600,"data_stored":1172546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1347584,"data_stored":581632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":582166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1343488,"data_stored":581562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1359872,"data_stored":583244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1343488,"data_stored":581562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":581686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1957888,"data_stored":1173654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:17:42.595 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:42.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.833+0000 7f06ee000640 1 Processor -- start 2024-04-25T10:17:42.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.833+0000 7f06ee000640 1 -- start start 2024-04-25T10:17:42.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.835+0000 7f06ee000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f06e815d980 0x7f06e80ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.835+0000 7f06ee000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06e815ed70 0x7f06e80acf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.835+0000 7f06ee000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e80712c0 con 0x7f06e80aeb20 2024-04-25T10:17:42.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.835+0000 7f06ee000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e8071ed0 con 0x7f06e815d980 2024-04-25T10:17:42.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.835+0000 7f06ee000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80715a0 con 0x7f06e815ed70 2024-04-25T10:17:42.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.835+0000 7f06e7400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f06e815d980 0x7f06e80ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.835+0000 7f06e6a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f06e80aeb20 0x7f06e80cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56850/0 (socket says 172.21.15.119:56850) 2024-04-25T10:17:42.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.835+0000 7f06e6a00640 1 -- 172.21.15.119:0/3597629 learned_addr learned my addr 172.21.15.119:0/3597629 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:42.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.836+0000 7f06e6000640 1 -- 172.21.15.119:0/3597629 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782187964 0 0) 0x7f06e80712c0 con 0x7f06e80aeb20 2024-04-25T10:17:42.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.836+0000 7f06e7e00640 1 --2- 172.21.15.119:0/3597629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06e815ed70 0x7f06e80acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.836+0000 7f06e6000640 1 -- 172.21.15.119:0/3597629 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06cc003520 con 0x7f06e80aeb20 2024-04-25T10:17:42.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.836+0000 7f06e7400640 1 -- 172.21.15.119:0/3597629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06e815ed70 msgr2=0x7f06e80acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.836+0000 7f06e7400640 1 --2- 172.21.15.119:0/3597629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06e815ed70 0x7f06e80acf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.836+0000 7f06e7400640 1 -- 172.21.15.119:0/3597629 >> v1:172.21.15.119:6789/0 conn(0x7f06e80aeb20 legacy=0x7f06e80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.836+0000 7f06e7400640 1 -- 172.21.15.119:0/3597629 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e8071880 con 0x7f06e815d980 2024-04-25T10:17:42.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.837+0000 7f06e7400640 1 --2- 172.21.15.119:0/3597629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e815d980 0x7f06e80ae5e0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f06d80097a0 tx=0x7f06d802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2a4b9dc6ec695a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.837+0000 7f06e6000640 1 -- 172.21.15.119:0/3597629 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d8037070 con 0x7f06e815d980 2024-04-25T10:17:42.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.838+0000 7f06e6000640 1 -- 172.21.15.119:0/3597629 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d802ae50 con 0x7f06e815d980 2024-04-25T10:17:42.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.838+0000 7f06e6000640 1 -- 172.21.15.119:0/3597629 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d8032640 con 0x7f06e815d980 2024-04-25T10:17:42.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.838+0000 7f06ee000640 1 -- 172.21.15.119:0/3597629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e815d980 msgr2=0x7f06e80ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.838+0000 7f06ee000640 1 --2- 172.21.15.119:0/3597629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e815d980 0x7f06e80ae5e0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f06d80097a0 tx=0x7f06d802a100 comp rx=0 tx=0).stop 2024-04-25T10:17:42.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.838+0000 7f06ee000640 1 -- 172.21.15.119:0/3597629 shutdown_connections 2024-04-25T10:17:42.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.838+0000 7f06ee000640 1 --2- 172.21.15.119:0/3597629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06e815ed70 0x7f06e80acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.838+0000 7f06ee000640 1 --2- 172.21.15.119:0/3597629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e815d980 0x7f06e80ae5e0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.838+0000 7f06ee000640 1 -- 172.21.15.119:0/3597629 >> 172.21.15.119:0/3597629 conn(0x7f06e806aba0 msgr2=0x7f06e806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.838+0000 7f06ee000640 1 -- 172.21.15.119:0/3597629 shutdown_connections 2024-04-25T10:17:42.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.838+0000 7f06ee000640 1 -- 172.21.15.119:0/3597629 wait complete. 2024-04-25T10:17:42.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.840+0000 7f06ee000640 1 Processor -- start 2024-04-25T10:17:42.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.840+0000 7f06ee000640 1 -- start start 2024-04-25T10:17:42.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.841+0000 7f06ee000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06e80aeb20 0x7f06e80cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.841+0000 7f06ee000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e815d980 0x7f06e8149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.841+0000 7f06ee000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06e814a3c0 0x7f06e8155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.841+0000 7f06ee000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80cfb10 con 0x7f06e815d980 2024-04-25T10:17:42.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.841+0000 7f06ee000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80cf990 con 0x7f06e814a3c0 2024-04-25T10:17:42.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.841+0000 7f06ee000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80ad4b0 con 0x7f06e80aeb20 2024-04-25T10:17:42.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.842+0000 7f06e6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e815d980 0x7f06e8149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.842+0000 7f06e6a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e815d980 0x7f06e8149b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36950/0 (socket says 172.21.15.119:36950) 2024-04-25T10:17:42.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.842+0000 7f06e6a00640 1 -- 172.21.15.119:0/3858628703 learned_addr learned my addr 172.21.15.119:0/3858628703 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:42.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.842+0000 7f06e7400640 1 --2- 172.21.15.119:0/3858628703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06e80aeb20 0x7f06e80cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.842+0000 7f06e6a00640 1 -- 172.21.15.119:0/3858628703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06e80aeb20 msgr2=0x7f06e80cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.842+0000 7f06e7e00640 1 --2- 172.21.15.119:0/3858628703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06e814a3c0 0x7f06e8155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.842+0000 7f06e6a00640 1 --2- 172.21.15.119:0/3858628703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06e80aeb20 0x7f06e80cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.842+0000 7f06e6a00640 1 -- 172.21.15.119:0/3858628703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06e814a3c0 msgr2=0x7f06e8155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.842+0000 7f06e6a00640 1 --2- 172.21.15.119:0/3858628703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06e814a3c0 0x7f06e8155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.842+0000 7f06e6a00640 1 -- 172.21.15.119:0/3858628703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e80cca50 con 0x7f06e815d980 2024-04-25T10:17:42.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.842+0000 7f06e6a00640 1 --2- 172.21.15.119:0/3858628703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e815d980 0x7f06e8149b10 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f06d400c4d0 tx=0x7f06d400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.843+0000 7f06df400640 1 -- 172.21.15.119:0/3858628703 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4017070 con 0x7f06e815d980 2024-04-25T10:17:42.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.843+0000 7f06df400640 1 -- 172.21.15.119:0/3858628703 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d4005d00 con 0x7f06e815d980 2024-04-25T10:17:42.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.843+0000 7f06df400640 1 -- 172.21.15.119:0/3858628703 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d40072b0 con 0x7f06e815d980 2024-04-25T10:17:42.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.843+0000 7f06ee000640 1 -- 172.21.15.119:0/3858628703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e80ccd30 con 0x7f06e815d980 2024-04-25T10:17:42.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.843+0000 7f06ee000640 1 -- 172.21.15.119:0/3858628703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e80cd2f0 con 0x7f06e815d980 2024-04-25T10:17:42.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.844+0000 7f06ee000640 1 -- 172.21.15.119:0/3858628703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e80b4590 con 0x7f06e815d980 2024-04-25T10:17:42.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.845+0000 7f06df400640 1 -- 172.21.15.119:0/3858628703 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06d401a070 con 0x7f06e815d980 2024-04-25T10:17:42.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.846+0000 7f06df400640 1 --2- 172.21.15.119:0/3858628703 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f06c806b4d0 0x7f06c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.846+0000 7f06df400640 1 -- 172.21.15.119:0/3858628703 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06d4089610 con 0x7f06e815d980 2024-04-25T10:17:42.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.846+0000 7f06e7400640 1 --2- 172.21.15.119:0/3858628703 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f06c806b4d0 0x7f06c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.847+0000 7f06e7400640 1 --2- 172.21.15.119:0/3858628703 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f06c806b4d0 0x7f06c806d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f06d8034000 tx=0x7f06d802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:42.851+0000 7f06df400640 1 -- 172.21.15.119:0/3858628703 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06e80b4590 con 0x7f06e815d980 2024-04-25T10:17:43.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.042+0000 7f06ee000640 1 -- 172.21.15.119:0/3858628703 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f06e806d4c0 con 0x7f06c806b4d0 2024-04-25T10:17:43.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.060+0000 7f06df400640 1 -- 172.21.15.119:0/3858628703 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252482 (secure 0 0 0) 0x7f06d807fa60 con 0x7f06c806b4d0 2024-04-25T10:17:43.066 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:43.068 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:17:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.066+0000 7f06ee000640 1 -- 172.21.15.119:0/3858628703 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f06c806b4d0 msgr2=0x7f06c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.066+0000 7f06ee000640 1 --2- 172.21.15.119:0/3858628703 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f06c806b4d0 0x7f06c806d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f06d8034000 tx=0x7f06d802e000 comp rx=0 tx=0).stop 2024-04-25T10:17:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.066+0000 7f06ee000640 1 -- 172.21.15.119:0/3858628703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e815d980 msgr2=0x7f06e8149b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.066+0000 7f06ee000640 1 --2- 172.21.15.119:0/3858628703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e815d980 0x7f06e8149b10 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f06d400c4d0 tx=0x7f06d400c990 comp rx=0 tx=0).stop 2024-04-25T10:17:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.066+0000 7f06ee000640 1 -- 172.21.15.119:0/3858628703 shutdown_connections 2024-04-25T10:17:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.066+0000 7f06ee000640 1 --2- 172.21.15.119:0/3858628703 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f06c806b4d0 0x7f06c806d990 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.066+0000 7f06ee000640 1 --2- 172.21.15.119:0/3858628703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06e814a3c0 0x7f06e8155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.066+0000 7f06ee000640 1 --2- 172.21.15.119:0/3858628703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06e815d980 0x7f06e8149b10 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.066+0000 7f06ee000640 1 --2- 172.21.15.119:0/3858628703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f06e80aeb20 0x7f06e80cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.066+0000 7f06ee000640 1 -- 172.21.15.119:0/3858628703 >> 172.21.15.119:0/3858628703 conn(0x7f06e806aba0 msgr2=0x7f06e815f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:43.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.066+0000 7f06ee000640 1 -- 172.21.15.119:0/3858628703 shutdown_connections 2024-04-25T10:17:43.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.066+0000 7f06ee000640 1 -- 172.21.15.119:0/3858628703 wait complete. 2024-04-25T10:17:43.089 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-04-25T10:17:42.140830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261208,"kb_used_data":11720,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713512,"statfs":{"total":773094113280,"available":772826636288,"internally_reserved":0,"allocated":12001280,"data_stored":5838052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.992017"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402443+0000","last_change":"2024-04-25T10:17:38.219576+0000","last_active":"2024-04-25T10:17:39.402443+0000","last_peered":"2024-04-25T10:17:39.402443+0000","last_clean":"2024-04-25T10:17:39.402443+0000","last_became_active":"2024-04-25T10:17:38.218662+0000","last_became_peered":"2024-04-25T10:17:38.218662+0000","last_unstale":"2024-04-25T10:17:39.402443+0000","last_undegraded":"2024-04-25T10:17:39.402443+0000","last_fullsized":"2024-04-25T10:17:39.402443+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:30.593455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.207426+0000","last_change":"2024-04-25T10:17:38.209218+0000","last_active":"2024-04-25T10:17:39.207426+0000","last_peered":"2024-04-25T10:17:39.207426+0000","last_clean":"2024-04-25T10:17:39.207426+0000","last_became_active":"2024-04-25T10:17:38.208169+0000","last_became_peered":"2024-04-25T10:17:38.208169+0000","last_unstale":"2024-04-25T10:17:39.207426+0000","last_undegraded":"2024-04-25T10:17:39.207426+0000","last_fullsized":"2024-04-25T10:17:39.207426+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:46.798286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403563+0000","last_change":"2024-04-25T10:17:38.222469+0000","last_active":"2024-04-25T10:17:39.403563+0000","last_peered":"2024-04-25T10:17:39.403563+0000","last_clean":"2024-04-25T10:17:39.403563+0000","last_became_active":"2024-04-25T10:17:38.221842+0000","last_became_peered":"2024-04-25T10:17:38.221842+0000","last_unstale":"2024-04-25T10:17:39.403563+0000","last_undegraded":"2024-04-25T10:17:39.403563+0000","last_fullsized":"2024-04-25T10:17:39.403563+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:59.194994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402120+0000","last_change":"2024-04-25T10:17:38.218561+0000","last_active":"2024-04-25T10:17:39.402120+0000","last_peered":"2024-04-25T10:17:39.402120+0000","last_clean":"2024-04-25T10:17:39.402120+0000","last_became_active":"2024-04-25T10:17:38.218060+0000","last_became_peered":"2024-04-25T10:17:38.218060+0000","last_unstale":"2024-04-25T10:17:39.402120+0000","last_undegraded":"2024-04-25T10:17:39.402120+0000","last_fullsized":"2024-04-25T10:17:39.402120+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:13.711019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.204671+0000","last_change":"2024-04-25T10:17:38.202476+0000","last_active":"2024-04-25T10:17:39.204671+0000","last_peered":"2024-04-25T10:17:39.204671+0000","last_clean":"2024-04-25T10:17:39.204671+0000","last_became_active":"2024-04-25T10:17:38.200141+0000","last_became_peered":"2024-04-25T10:17:38.200141+0000","last_unstale":"2024-04-25T10:17:39.204671+0000","last_undegraded":"2024-04-25T10:17:39.204671+0000","last_fullsized":"2024-04-25T10:17:39.204671+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:09.360593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403492+0000","last_change":"2024-04-25T10:17:38.209349+0000","last_active":"2024-04-25T10:17:39.403492+0000","last_peered":"2024-04-25T10:17:39.403492+0000","last_clean":"2024-04-25T10:17:39.403492+0000","last_became_active":"2024-04-25T10:17:38.209028+0000","last_became_peered":"2024-04-25T10:17:38.209028+0000","last_unstale":"2024-04-25T10:17:39.403492+0000","last_undegraded":"2024-04-25T10:17:39.403492+0000","last_fullsized":"2024-04-25T10:17:39.403492+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:54.663639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.207430+0000","last_change":"2024-04-25T10:17:38.209694+0000","last_active":"2024-04-25T10:17:39.207430+0000","last_peered":"2024-04-25T10:17:39.207430+0000","last_clean":"2024-04-25T10:17:39.207430+0000","last_became_active":"2024-04-25T10:17:38.209415+0000","last_became_peered":"2024-04-25T10:17:38.209415+0000","last_unstale":"2024-04-25T10:17:39.207430+0000","last_undegraded":"2024-04-25T10:17:39.207430+0000","last_fullsized":"2024-04-25T10:17:39.207430+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:11.558704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401779+0000","last_change":"2024-04-25T10:17:38.220327+0000","last_active":"2024-04-25T10:17:39.401779+0000","last_peered":"2024-04-25T10:17:39.401779+0000","last_clean":"2024-04-25T10:17:39.401779+0000","last_became_active":"2024-04-25T10:17:38.216829+0000","last_became_peered":"2024-04-25T10:17:38.216829+0000","last_unstale":"2024-04-25T10:17:39.401779+0000","last_undegraded":"2024-04-25T10:17:39.401779+0000","last_fullsized":"2024-04-25T10:17:39.401779+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:08.851641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402602+0000","last_change":"2024-04-25T10:17:38.197762+0000","last_active":"2024-04-25T10:17:39.402602+0000","last_peered":"2024-04-25T10:17:39.402602+0000","last_clean":"2024-04-25T10:17:39.402602+0000","last_became_active":"2024-04-25T10:17:38.197498+0000","last_became_peered":"2024-04-25T10:17:38.197498+0000","last_unstale":"2024-04-25T10:17:39.402602+0000","last_undegraded":"2024-04-25T10:17:39.402602+0000","last_fullsized":"2024-04-25T10:17:39.402602+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:18.242394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403028+0000","last_change":"2024-04-25T10:17:38.210396+0000","last_active":"2024-04-25T10:17:39.403028+0000","last_peered":"2024-04-25T10:17:39.403028+0000","last_clean":"2024-04-25T10:17:39.403028+0000","last_became_active":"2024-04-25T10:17:38.208324+0000","last_became_peered":"2024-04-25T10:17:38.208324+0000","last_unstale":"2024-04-25T10:17:39.403028+0000","last_undegraded":"2024-04-25T10:17:39.403028+0000","last_fullsized":"2024-04-25T10:17:39.403028+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:40.290291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.206561+0000","last_change":"2024-04-25T10:17:38.212504+0000","last_active":"2024-04-25T10:17:39.206561+0000","last_peered":"2024-04-25T10:17:39.206561+0000","last_clean":"2024-04-25T10:17:39.206561+0000","last_became_active":"2024-04-25T10:17:38.212196+0000","last_became_peered":"2024-04-25T10:17:38.212196+0000","last_unstale":"2024-04-25T10:17:39.206561+0000","last_undegraded":"2024-04-25T10:17:39.206561+0000","last_fullsized":"2024-04-25T10:17:39.206561+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:38.235200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402409+0000","last_change":"2024-04-25T10:17:38.216559+0000","last_active":"2024-04-25T10:17:39.402409+0000","last_peered":"2024-04-25T10:17:39.402409+0000","last_clean":"2024-04-25T10:17:39.402409+0000","last_became_active":"2024-04-25T10:17:38.214421+0000","last_became_peered":"2024-04-25T10:17:38.214421+0000","last_unstale":"2024-04-25T10:17:39.402409+0000","last_undegraded":"2024-04-25T10:17:39.402409+0000","last_fullsized":"2024-04-25T10:17:39.402409+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:23.758403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403496+0000","last_change":"2024-04-25T10:17:38.214026+0000","last_active":"2024-04-25T10:17:39.403496+0000","last_peered":"2024-04-25T10:17:39.403496+0000","last_clean":"2024-04-25T10:17:39.403496+0000","last_became_active":"2024-04-25T10:17:38.213630+0000","last_became_peered":"2024-04-25T10:17:38.213630+0000","last_unstale":"2024-04-25T10:17:39.403496+0000","last_undegraded":"2024-04-25T10:17:39.403496+0000","last_fullsized":"2024-04-25T10:17:39.403496+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:19.386704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402234+0000","last_change":"2024-04-25T10:17:38.218642+0000","last_active":"2024-04-25T10:17:39.402234+0000","last_peered":"2024-04-25T10:17:39.402234+0000","last_clean":"2024-04-25T10:17:39.402234+0000","last_became_active":"2024-04-25T10:17:38.218293+0000","last_became_peered":"2024-04-25T10:17:38.218293+0000","last_unstale":"2024-04-25T10:17:39.402234+0000","last_undegraded":"2024-04-25T10:17:39.402234+0000","last_fullsized":"2024-04-25T10:17:39.402234+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:17.188112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.400936+0000","last_change":"2024-04-25T10:17:38.210930+0000","last_active":"2024-04-25T10:17:39.400936+0000","last_peered":"2024-04-25T10:17:39.400936+0000","last_clean":"2024-04-25T10:17:39.400936+0000","last_became_active":"2024-04-25T10:17:38.210672+0000","last_became_peered":"2024-04-25T10:17:38.210672+0000","last_unstale":"2024-04-25T10:17:39.400936+0000","last_undegraded":"2024-04-25T10:17:39.400936+0000","last_fullsized":"2024-04-25T10:17:39.400936+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:59.916801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401005+0000","last_change":"2024-04-25T10:17:38.215806+0000","last_active":"2024-04-25T10:17:39.401005+0000","last_peered":"2024-04-25T10:17:39.401005+0000","last_clean":"2024-04-25T10:17:39.401005+0000","last_became_active":"2024-04-25T10:17:38.214746+0000","last_became_peered":"2024-04-25T10:17:38.214746+0000","last_unstale":"2024-04-25T10:17:39.401005+0000","last_undegraded":"2024-04-25T10:17:39.401005+0000","last_fullsized":"2024-04-25T10:17:39.401005+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:18.847917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.206536+0000","last_change":"2024-04-25T10:17:38.214985+0000","last_active":"2024-04-25T10:17:39.206536+0000","last_peered":"2024-04-25T10:17:39.206536+0000","last_clean":"2024-04-25T10:17:39.206536+0000","last_became_active":"2024-04-25T10:17:38.214590+0000","last_became_peered":"2024-04-25T10:17:38.214590+0000","last_unstale":"2024-04-25T10:17:39.206536+0000","last_undegraded":"2024-04-25T10:17:39.206536+0000","last_fullsized":"2024-04-25T10:17:39.206536+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:45.676668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401514+0000","last_change":"2024-04-25T10:17:38.220631+0000","last_active":"2024-04-25T10:17:39.401514+0000","last_peered":"2024-04-25T10:17:39.401514+0000","last_clean":"2024-04-25T10:17:39.401514+0000","last_became_active":"2024-04-25T10:17:38.220228+0000","last_became_peered":"2024-04-25T10:17:38.220228+0000","last_unstale":"2024-04-25T10:17:39.401514+0000","last_undegraded":"2024-04-25T10:17:39.401514+0000","last_fullsized":"2024-04-25T10:17:39.401514+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:22.321963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403618+0000","last_change":"2024-04-25T10:17:38.218236+0000","last_active":"2024-04-25T10:17:39.403618+0000","last_peered":"2024-04-25T10:17:39.403618+0000","last_clean":"2024-04-25T10:17:39.403618+0000","last_became_active":"2024-04-25T10:17:38.216333+0000","last_became_peered":"2024-04-25T10:17:38.216333+0000","last_unstale":"2024-04-25T10:17:39.403618+0000","last_undegraded":"2024-04-25T10:17:39.403618+0000","last_fullsized":"2024-04-25T10:17:39.403618+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:45.676668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.206273+0000","last_change":"2024-04-25T10:17:38.219695+0000","last_active":"2024-04-25T10:17:39.206273+0000","last_peered":"2024-04-25T10:17:39.206273+0000","last_clean":"2024-04-25T10:17:39.206273+0000","last_became_active":"2024-04-25T10:17:38.215990+0000","last_became_peered":"2024-04-25T10:17:38.215990+0000","last_unstale":"2024-04-25T10:17:39.206273+0000","last_undegraded":"2024-04-25T10:17:39.206273+0000","last_fullsized":"2024-04-25T10:17:39.206273+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:19.386704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.206364+0000","last_change":"2024-04-25T10:17:38.204217+0000","last_active":"2024-04-25T10:17:39.206364+0000","last_peered":"2024-04-25T10:17:39.206364+0000","last_clean":"2024-04-25T10:17:39.206364+0000","last_became_active":"2024-04-25T10:17:38.203910+0000","last_became_peered":"2024-04-25T10:17:38.203910+0000","last_unstale":"2024-04-25T10:17:39.206364+0000","last_undegraded":"2024-04-25T10:17:39.206364+0000","last_fullsized":"2024-04-25T10:17:39.206364+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:59.194994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401441+0000","last_change":"2024-04-25T10:17:38.214776+0000","last_active":"2024-04-25T10:17:39.401441+0000","last_peered":"2024-04-25T10:17:39.401441+0000","last_clean":"2024-04-25T10:17:39.401441+0000","last_became_active":"2024-04-25T10:17:38.214511+0000","last_became_peered":"2024-04-25T10:17:38.214511+0000","last_unstale":"2024-04-25T10:17:39.401441+0000","last_undegraded":"2024-04-25T10:17:39.401441+0000","last_fullsized":"2024-04-25T10:17:39.401441+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:05.814548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205348+0000","last_change":"2024-04-25T10:17:38.210903+0000","last_active":"2024-04-25T10:17:39.205348+0000","last_peered":"2024-04-25T10:17:39.205348+0000","last_clean":"2024-04-25T10:17:39.205348+0000","last_became_active":"2024-04-25T10:17:38.210507+0000","last_became_peered":"2024-04-25T10:17:38.210507+0000","last_unstale":"2024-04-25T10:17:39.205348+0000","last_undegraded":"2024-04-25T10:17:39.205348+0000","last_fullsized":"2024-04-25T10:17:39.205348+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:17.188112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.400371+0000","last_change":"2024-04-25T10:17:38.211219+0000","last_active":"2024-04-25T10:17:39.400371+0000","last_peered":"2024-04-25T10:17:39.400371+0000","last_clean":"2024-04-25T10:17:39.400371+0000","last_became_active":"2024-04-25T10:17:38.210057+0000","last_became_peered":"2024-04-25T10:17:38.210057+0000","last_unstale":"2024-04-25T10:17:39.400371+0000","last_undegraded":"2024-04-25T10:17:39.400371+0000","last_fullsized":"2024-04-25T10:17:39.400371+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:39.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.206647+0000","last_change":"2024-04-25T10:17:38.214429+0000","last_active":"2024-04-25T10:17:39.206647+0000","last_peered":"2024-04-25T10:17:39.206647+0000","last_clean":"2024-04-25T10:17:39.206647+0000","last_became_active":"2024-04-25T10:17:38.213642+0000","last_became_peered":"2024-04-25T10:17:38.213642+0000","last_unstale":"2024-04-25T10:17:39.206647+0000","last_undegraded":"2024-04-25T10:17:39.206647+0000","last_fullsized":"2024-04-25T10:17:39.206647+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:39.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403197+0000","last_change":"2024-04-25T10:17:38.198663+0000","last_active":"2024-04-25T10:17:39.403197+0000","last_peered":"2024-04-25T10:17:39.403197+0000","last_clean":"2024-04-25T10:17:39.403197+0000","last_became_active":"2024-04-25T10:17:38.198358+0000","last_became_peered":"2024-04-25T10:17:38.198358+0000","last_unstale":"2024-04-25T10:17:39.403197+0000","last_undegraded":"2024-04-25T10:17:39.403197+0000","last_fullsized":"2024-04-25T10:17:39.403197+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:06.405918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.405182+0000","last_change":"2024-04-25T10:17:38.218539+0000","last_active":"2024-04-25T10:17:39.405182+0000","last_peered":"2024-04-25T10:17:39.405182+0000","last_clean":"2024-04-25T10:17:39.405182+0000","last_became_active":"2024-04-25T10:17:38.218282+0000","last_became_peered":"2024-04-25T10:17:38.218282+0000","last_unstale":"2024-04-25T10:17:39.405182+0000","last_undegraded":"2024-04-25T10:17:39.405182+0000","last_fullsized":"2024-04-25T10:17:39.405182+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:11.558704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.404740+0000","last_change":"2024-04-25T10:17:38.203527+0000","last_active":"2024-04-25T10:17:39.404740+0000","last_peered":"2024-04-25T10:17:39.404740+0000","last_clean":"2024-04-25T10:17:39.404740+0000","last_became_active":"2024-04-25T10:17:38.203261+0000","last_became_peered":"2024-04-25T10:17:38.203261+0000","last_unstale":"2024-04-25T10:17:39.404740+0000","last_undegraded":"2024-04-25T10:17:39.404740+0000","last_fullsized":"2024-04-25T10:17:39.404740+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:57.970904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205277+0000","last_change":"2024-04-25T10:17:38.216263+0000","last_active":"2024-04-25T10:17:39.205277+0000","last_peered":"2024-04-25T10:17:39.205277+0000","last_clean":"2024-04-25T10:17:39.205277+0000","last_became_active":"2024-04-25T10:17:38.215959+0000","last_became_peered":"2024-04-25T10:17:38.215959+0000","last_unstale":"2024-04-25T10:17:39.205277+0000","last_undegraded":"2024-04-25T10:17:39.205277+0000","last_fullsized":"2024-04-25T10:17:39.205277+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.703896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.208048+0000","last_change":"2024-04-25T10:17:38.194952+0000","last_active":"2024-04-25T10:17:39.208048+0000","last_peered":"2024-04-25T10:17:39.208048+0000","last_clean":"2024-04-25T10:17:39.208048+0000","last_became_active":"2024-04-25T10:17:38.194122+0000","last_became_peered":"2024-04-25T10:17:38.194122+0000","last_unstale":"2024-04-25T10:17:39.208048+0000","last_undegraded":"2024-04-25T10:17:39.208048+0000","last_fullsized":"2024-04-25T10:17:39.208048+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:55.995614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403362+0000","last_change":"2024-04-25T10:17:38.202503+0000","last_active":"2024-04-25T10:17:39.403362+0000","last_peered":"2024-04-25T10:17:39.403362+0000","last_clean":"2024-04-25T10:17:39.403362+0000","last_became_active":"2024-04-25T10:17:38.202192+0000","last_became_peered":"2024-04-25T10:17:38.202192+0000","last_unstale":"2024-04-25T10:17:39.403362+0000","last_undegraded":"2024-04-25T10:17:39.403362+0000","last_fullsized":"2024-04-25T10:17:39.403362+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:13.921165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402388+0000","last_change":"2024-04-25T10:17:38.204953+0000","last_active":"2024-04-25T10:17:39.402388+0000","last_peered":"2024-04-25T10:17:39.402388+0000","last_clean":"2024-04-25T10:17:39.402388+0000","last_became_active":"2024-04-25T10:17:38.204757+0000","last_became_peered":"2024-04-25T10:17:38.204757+0000","last_unstale":"2024-04-25T10:17:39.402388+0000","last_undegraded":"2024-04-25T10:17:39.402388+0000","last_fullsized":"2024-04-25T10:17:39.402388+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:39.345689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.405849+0000","last_change":"2024-04-25T10:17:38.210354+0000","last_active":"2024-04-25T10:17:39.405849+0000","last_peered":"2024-04-25T10:17:39.405849+0000","last_clean":"2024-04-25T10:17:39.405849+0000","last_became_active":"2024-04-25T10:17:38.208585+0000","last_became_peered":"2024-04-25T10:17:38.208585+0000","last_unstale":"2024-04-25T10:17:39.405849+0000","last_undegraded":"2024-04-25T10:17:39.405849+0000","last_fullsized":"2024-04-25T10:17:39.405849+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:17.015596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"107'4","reported_seq":18,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.208583+0000","last_change":"2024-04-25T10:17:37.187966+0000","last_active":"2024-04-25T10:17:39.208583+0000","last_peered":"2024-04-25T10:17:39.208583+0000","last_clean":"2024-04-25T10:17:39.208583+0000","last_became_active":"2024-04-25T10:17:37.187662+0000","last_became_peered":"2024-04-25T10:17:37.187662+0000","last_unstale":"2024-04-25T10:17:39.208583+0000","last_undegraded":"2024-04-25T10:17:39.208583+0000","last_fullsized":"2024-04-25T10:17:39.208583+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:23.899400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402667+0000","last_change":"2024-04-25T10:17:38.220233+0000","last_active":"2024-04-25T10:17:39.402667+0000","last_peered":"2024-04-25T10:17:39.402667+0000","last_clean":"2024-04-25T10:17:39.402667+0000","last_became_active":"2024-04-25T10:17:38.219933+0000","last_became_peered":"2024-04-25T10:17:38.219933+0000","last_unstale":"2024-04-25T10:17:39.402667+0000","last_undegraded":"2024-04-25T10:17:39.402667+0000","last_fullsized":"2024-04-25T10:17:39.402667+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:17.188112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403322+0000","last_change":"2024-04-25T10:17:37.183153+0000","last_active":"2024-04-25T10:17:39.403322+0000","last_peered":"2024-04-25T10:17:39.403322+0000","last_clean":"2024-04-25T10:17:39.403322+0000","last_became_active":"2024-04-25T10:17:37.182528+0000","last_became_peered":"2024-04-25T10:17:37.182528+0000","last_unstale":"2024-04-25T10:17:39.403322+0000","last_undegraded":"2024-04-25T10:17:39.403322+0000","last_fullsized":"2024-04-25T10:17:39.403322+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:47.774882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.405425+0000","last_change":"2024-04-25T10:17:38.216108+0000","last_active":"2024-04-25T10:17:39.405425+0000","last_peered":"2024-04-25T10:17:39.405425+0000","last_clean":"2024-04-25T10:17:39.405425+0000","last_became_active":"2024-04-25T10:17:38.215831+0000","last_became_peered":"2024-04-25T10:17:38.215831+0000","last_unstale":"2024-04-25T10:17:39.405425+0000","last_undegraded":"2024-04-25T10:17:39.405425+0000","last_fullsized":"2024-04-25T10:17:39.405425+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:55.995614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205096+0000","last_change":"2024-04-25T10:17:37.193464+0000","last_active":"2024-04-25T10:17:39.205096+0000","last_peered":"2024-04-25T10:17:39.205096+0000","last_clean":"2024-04-25T10:17:39.205096+0000","last_became_active":"2024-04-25T10:17:37.193022+0000","last_became_peered":"2024-04-25T10:17:37.193022+0000","last_unstale":"2024-04-25T10:17:39.205096+0000","last_undegraded":"2024-04-25T10:17:39.205096+0000","last_fullsized":"2024-04-25T10:17:39.205096+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:06.291123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403983+0000","last_change":"2024-04-25T10:17:38.220558+0000","last_active":"2024-04-25T10:17:39.403983+0000","last_peered":"2024-04-25T10:17:39.403983+0000","last_clean":"2024-04-25T10:17:39.403983+0000","last_became_active":"2024-04-25T10:17:38.216848+0000","last_became_peered":"2024-04-25T10:17:38.216848+0000","last_unstale":"2024-04-25T10:17:39.403983+0000","last_undegraded":"2024-04-25T10:17:39.403983+0000","last_fullsized":"2024-04-25T10:17:39.403983+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:39.063472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.204770+0000","last_change":"2024-04-25T10:17:37.189460+0000","last_active":"2024-04-25T10:17:39.204770+0000","last_peered":"2024-04-25T10:17:39.204770+0000","last_clean":"2024-04-25T10:17:39.204770+0000","last_became_active":"2024-04-25T10:17:37.189221+0000","last_became_peered":"2024-04-25T10:17:37.189221+0000","last_unstale":"2024-04-25T10:17:39.204770+0000","last_undegraded":"2024-04-25T10:17:39.204770+0000","last_fullsized":"2024-04-25T10:17:39.204770+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:10.284592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401484+0000","last_change":"2024-04-25T10:17:38.196915+0000","last_active":"2024-04-25T10:17:39.401484+0000","last_peered":"2024-04-25T10:17:39.401484+0000","last_clean":"2024-04-25T10:17:39.401484+0000","last_became_active":"2024-04-25T10:17:38.196634+0000","last_became_peered":"2024-04-25T10:17:38.196634+0000","last_unstale":"2024-04-25T10:17:39.401484+0000","last_undegraded":"2024-04-25T10:17:39.401484+0000","last_fullsized":"2024-04-25T10:17:39.401484+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:12.410162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.207769+0000","last_change":"2024-04-25T10:17:37.189092+0000","last_active":"2024-04-25T10:17:39.207769+0000","last_peered":"2024-04-25T10:17:39.207769+0000","last_clean":"2024-04-25T10:17:39.207769+0000","last_became_active":"2024-04-25T10:17:37.188753+0000","last_became_peered":"2024-04-25T10:17:37.188753+0000","last_unstale":"2024-04-25T10:17:39.207769+0000","last_undegraded":"2024-04-25T10:17:39.207769+0000","last_fullsized":"2024-04-25T10:17:39.207769+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:45.834637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205754+0000","last_change":"2024-04-25T10:17:38.208949+0000","last_active":"2024-04-25T10:17:39.205754+0000","last_peered":"2024-04-25T10:17:39.205754+0000","last_clean":"2024-04-25T10:17:39.205754+0000","last_became_active":"2024-04-25T10:17:38.208596+0000","last_became_peered":"2024-04-25T10:17:38.208596+0000","last_unstale":"2024-04-25T10:17:39.205754+0000","last_undegraded":"2024-04-25T10:17:39.205754+0000","last_fullsized":"2024-04-25T10:17:39.205754+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:41.678472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403421+0000","last_change":"2024-04-25T10:17:37.186303+0000","last_active":"2024-04-25T10:17:39.403421+0000","last_peered":"2024-04-25T10:17:39.403421+0000","last_clean":"2024-04-25T10:17:39.403421+0000","last_became_active":"2024-04-25T10:17:37.185980+0000","last_became_peered":"2024-04-25T10:17:37.185980+0000","last_unstale":"2024-04-25T10:17:39.403421+0000","last_undegraded":"2024-04-25T10:17:39.403421+0000","last_fullsized":"2024-04-25T10:17:39.403421+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:56.567178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401436+0000","last_change":"2024-04-25T10:17:38.214768+0000","last_active":"2024-04-25T10:17:39.401436+0000","last_peered":"2024-04-25T10:17:39.401436+0000","last_clean":"2024-04-25T10:17:39.401436+0000","last_became_active":"2024-04-25T10:17:38.213620+0000","last_became_peered":"2024-04-25T10:17:38.213620+0000","last_unstale":"2024-04-25T10:17:39.401436+0000","last_undegraded":"2024-04-25T10:17:39.401436+0000","last_fullsized":"2024-04-25T10:17:39.401436+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:46.154221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205149+0000","last_change":"2024-04-25T10:17:37.177115+0000","last_active":"2024-04-25T10:17:39.205149+0000","last_peered":"2024-04-25T10:17:39.205149+0000","last_clean":"2024-04-25T10:17:39.205149+0000","last_became_active":"2024-04-25T10:17:37.176679+0000","last_became_peered":"2024-04-25T10:17:37.176679+0000","last_unstale":"2024-04-25T10:17:39.205149+0000","last_undegraded":"2024-04-25T10:17:39.205149+0000","last_fullsized":"2024-04-25T10:17:39.205149+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:58.089525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.206915+0000","last_change":"2024-04-25T10:17:38.209909+0000","last_active":"2024-04-25T10:17:39.206915+0000","last_peered":"2024-04-25T10:17:39.206915+0000","last_clean":"2024-04-25T10:17:39.206915+0000","last_became_active":"2024-04-25T10:17:38.209593+0000","last_became_peered":"2024-04-25T10:17:38.209593+0000","last_unstale":"2024-04-25T10:17:39.206915+0000","last_undegraded":"2024-04-25T10:17:39.206915+0000","last_fullsized":"2024-04-25T10:17:39.206915+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:54.663639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401879+0000","last_change":"2024-04-25T10:17:37.180701+0000","last_active":"2024-04-25T10:17:39.401879+0000","last_peered":"2024-04-25T10:17:39.401879+0000","last_clean":"2024-04-25T10:17:39.401879+0000","last_became_active":"2024-04-25T10:17:37.180439+0000","last_became_peered":"2024-04-25T10:17:37.180439+0000","last_unstale":"2024-04-25T10:17:39.401879+0000","last_undegraded":"2024-04-25T10:17:39.401879+0000","last_fullsized":"2024-04-25T10:17:39.401879+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:38.048694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.209130+0000","last_change":"2024-04-25T10:17:38.210858+0000","last_active":"2024-04-25T10:17:39.209130+0000","last_peered":"2024-04-25T10:17:39.209130+0000","last_clean":"2024-04-25T10:17:39.209130+0000","last_became_active":"2024-04-25T10:17:38.208935+0000","last_became_peered":"2024-04-25T10:17:38.208935+0000","last_unstale":"2024-04-25T10:17:39.209130+0000","last_undegraded":"2024-04-25T10:17:39.209130+0000","last_fullsized":"2024-04-25T10:17:39.209130+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:38.268735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205300+0000","last_change":"2024-04-25T10:17:37.190960+0000","last_active":"2024-04-25T10:17:39.205300+0000","last_peered":"2024-04-25T10:17:39.205300+0000","last_clean":"2024-04-25T10:17:39.205300+0000","last_became_active":"2024-04-25T10:17:37.189950+0000","last_became_peered":"2024-04-25T10:17:37.189950+0000","last_unstale":"2024-04-25T10:17:39.205300+0000","last_undegraded":"2024-04-25T10:17:39.205300+0000","last_fullsized":"2024-04-25T10:17:39.205300+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:45.683871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402477+0000","last_change":"2024-04-25T10:17:38.219954+0000","last_active":"2024-04-25T10:17:39.402477+0000","last_peered":"2024-04-25T10:17:39.402477+0000","last_clean":"2024-04-25T10:17:39.402477+0000","last_became_active":"2024-04-25T10:17:38.219608+0000","last_became_peered":"2024-04-25T10:17:38.219608+0000","last_unstale":"2024-04-25T10:17:39.402477+0000","last_undegraded":"2024-04-25T10:17:39.402477+0000","last_fullsized":"2024-04-25T10:17:39.402477+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:48.495338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.405481+0000","last_change":"2024-04-25T10:17:37.185651+0000","last_active":"2024-04-25T10:17:39.405481+0000","last_peered":"2024-04-25T10:17:39.405481+0000","last_clean":"2024-04-25T10:17:39.405481+0000","last_became_active":"2024-04-25T10:17:37.185320+0000","last_became_peered":"2024-04-25T10:17:37.185320+0000","last_unstale":"2024-04-25T10:17:39.405481+0000","last_undegraded":"2024-04-25T10:17:39.405481+0000","last_fullsized":"2024-04-25T10:17:39.405481+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:26.517389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.404730+0000","last_change":"2024-04-25T10:17:38.209654+0000","last_active":"2024-04-25T10:17:39.404730+0000","last_peered":"2024-04-25T10:17:39.404730+0000","last_clean":"2024-04-25T10:17:39.404730+0000","last_became_active":"2024-04-25T10:17:38.207818+0000","last_became_peered":"2024-04-25T10:17:38.207818+0000","last_unstale":"2024-04-25T10:17:39.404730+0000","last_undegraded":"2024-04-25T10:17:39.404730+0000","last_fullsized":"2024-04-25T10:17:39.404730+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:02.724367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"107'3","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402753+0000","last_change":"2024-04-25T10:17:37.193973+0000","last_active":"2024-04-25T10:17:39.402753+0000","last_peered":"2024-04-25T10:17:39.402753+0000","last_clean":"2024-04-25T10:17:39.402753+0000","last_became_active":"2024-04-25T10:17:37.193581+0000","last_became_peered":"2024-04-25T10:17:37.193581+0000","last_unstale":"2024-04-25T10:17:39.402753+0000","last_undegraded":"2024-04-25T10:17:39.402753+0000","last_fullsized":"2024-04-25T10:17:39.402753+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:37.220423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.208393+0000","last_change":"2024-04-25T10:17:38.218063+0000","last_active":"2024-04-25T10:17:39.208393+0000","last_peered":"2024-04-25T10:17:39.208393+0000","last_clean":"2024-04-25T10:17:39.208393+0000","last_became_active":"2024-04-25T10:17:38.216415+0000","last_became_peered":"2024-04-25T10:17:38.216415+0000","last_unstale":"2024-04-25T10:17:39.208393+0000","last_undegraded":"2024-04-25T10:17:39.208393+0000","last_fullsized":"2024-04-25T10:17:39.208393+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:17.015596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"107'6","reported_seq":20,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403090+0000","last_change":"2024-04-25T10:17:37.175580+0000","last_active":"2024-04-25T10:17:39.403090+0000","last_peered":"2024-04-25T10:17:39.403090+0000","last_clean":"2024-04-25T10:17:39.403090+0000","last_became_active":"2024-04-25T10:17:37.175291+0000","last_became_peered":"2024-04-25T10:17:37.175291+0000","last_unstale":"2024-04-25T10:17:39.403090+0000","last_undegraded":"2024-04-25T10:17:39.403090+0000","last_fullsized":"2024-04-25T10:17:39.403090+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.516817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401531+0000","last_change":"2024-04-25T10:17:38.197921+0000","last_active":"2024-04-25T10:17:39.401531+0000","last_peered":"2024-04-25T10:17:39.401531+0000","last_clean":"2024-04-25T10:17:39.401531+0000","last_became_active":"2024-04-25T10:17:38.197593+0000","last_became_peered":"2024-04-25T10:17:38.197593+0000","last_unstale":"2024-04-25T10:17:39.401531+0000","last_undegraded":"2024-04-25T10:17:39.401531+0000","last_fullsized":"2024-04-25T10:17:39.401531+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:47.608829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.404328+0000","last_change":"2024-04-25T10:17:37.187413+0000","last_active":"2024-04-25T10:17:39.404328+0000","last_peered":"2024-04-25T10:17:39.404328+0000","last_clean":"2024-04-25T10:17:39.404328+0000","last_became_active":"2024-04-25T10:17:37.187149+0000","last_became_peered":"2024-04-25T10:17:37.187149+0000","last_unstale":"2024-04-25T10:17:39.404328+0000","last_undegraded":"2024-04-25T10:17:39.404328+0000","last_fullsized":"2024-04-25T10:17:39.404328+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:26.239232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403730+0000","last_change":"2024-04-25T10:17:38.212733+0000","last_active":"2024-04-25T10:17:39.403730+0000","last_peered":"2024-04-25T10:17:39.403730+0000","last_clean":"2024-04-25T10:17:39.403730+0000","last_became_active":"2024-04-25T10:17:38.209988+0000","last_became_peered":"2024-04-25T10:17:38.209988+0000","last_unstale":"2024-04-25T10:17:39.403730+0000","last_undegraded":"2024-04-25T10:17:39.403730+0000","last_fullsized":"2024-04-25T10:17:39.403730+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:24.980261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401978+0000","last_change":"2024-04-25T10:17:37.194785+0000","last_active":"2024-04-25T10:17:39.401978+0000","last_peered":"2024-04-25T10:17:39.401978+0000","last_clean":"2024-04-25T10:17:39.401978+0000","last_became_active":"2024-04-25T10:17:37.194494+0000","last_became_peered":"2024-04-25T10:17:37.194494+0000","last_unstale":"2024-04-25T10:17:39.401978+0000","last_undegraded":"2024-04-25T10:17:39.401978+0000","last_fullsized":"2024-04-25T10:17:39.401978+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:22.440679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.400607+0000","last_change":"2024-04-25T10:17:38.206028+0000","last_active":"2024-04-25T10:17:39.400607+0000","last_peered":"2024-04-25T10:17:39.400607+0000","last_clean":"2024-04-25T10:17:39.400607+0000","last_became_active":"2024-04-25T10:17:38.205789+0000","last_became_peered":"2024-04-25T10:17:38.205789+0000","last_unstale":"2024-04-25T10:17:39.400607+0000","last_undegraded":"2024-04-25T10:17:39.400607+0000","last_fullsized":"2024-04-25T10:17:39.400607+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:46.798286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"107'8","reported_seq":22,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.204418+0000","last_change":"2024-04-25T10:17:37.181066+0000","last_active":"2024-04-25T10:17:39.204418+0000","last_peered":"2024-04-25T10:17:39.204418+0000","last_clean":"2024-04-25T10:17:39.204418+0000","last_became_active":"2024-04-25T10:17:37.180791+0000","last_became_peered":"2024-04-25T10:17:37.180791+0000","last_unstale":"2024-04-25T10:17:39.204418+0000","last_undegraded":"2024-04-25T10:17:39.204418+0000","last_fullsized":"2024-04-25T10:17:39.204418+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:53.580198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205393+0000","last_change":"2024-04-25T10:17:38.205651+0000","last_active":"2024-04-25T10:17:39.205393+0000","last_peered":"2024-04-25T10:17:39.205393+0000","last_clean":"2024-04-25T10:17:39.205393+0000","last_became_active":"2024-04-25T10:17:38.205300+0000","last_became_peered":"2024-04-25T10:17:38.205300+0000","last_unstale":"2024-04-25T10:17:39.205393+0000","last_undegraded":"2024-04-25T10:17:39.205393+0000","last_fullsized":"2024-04-25T10:17:39.205393+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:11.812895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402945+0000","last_change":"2024-04-25T10:17:37.180200+0000","last_active":"2024-04-25T10:17:39.402945+0000","last_peered":"2024-04-25T10:17:39.402945+0000","last_clean":"2024-04-25T10:17:39.402945+0000","last_became_active":"2024-04-25T10:17:37.179916+0000","last_became_peered":"2024-04-25T10:17:37.179916+0000","last_unstale":"2024-04-25T10:17:39.402945+0000","last_undegraded":"2024-04-25T10:17:39.402945+0000","last_fullsized":"2024-04-25T10:17:39.402945+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:44.661891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.406014+0000","last_change":"2024-04-25T10:17:38.214695+0000","last_active":"2024-04-25T10:17:39.406014+0000","last_peered":"2024-04-25T10:17:39.406014+0000","last_clean":"2024-04-25T10:17:39.406014+0000","last_became_active":"2024-04-25T10:17:38.214425+0000","last_became_peered":"2024-04-25T10:17:38.214425+0000","last_unstale":"2024-04-25T10:17:39.406014+0000","last_undegraded":"2024-04-25T10:17:39.406014+0000","last_fullsized":"2024-04-25T10:17:39.406014+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:38.268735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205289+0000","last_change":"2024-04-25T10:17:37.182981+0000","last_active":"2024-04-25T10:17:39.205289+0000","last_peered":"2024-04-25T10:17:39.205289+0000","last_clean":"2024-04-25T10:17:39.205289+0000","last_became_active":"2024-04-25T10:17:37.182698+0000","last_became_peered":"2024-04-25T10:17:37.182698+0000","last_unstale":"2024-04-25T10:17:39.205289+0000","last_undegraded":"2024-04-25T10:17:39.205289+0000","last_fullsized":"2024-04-25T10:17:39.205289+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:46.896358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403248+0000","last_change":"2024-04-25T10:17:38.223133+0000","last_active":"2024-04-25T10:17:39.403248+0000","last_peered":"2024-04-25T10:17:39.403248+0000","last_clean":"2024-04-25T10:17:39.403248+0000","last_became_active":"2024-04-25T10:17:38.222857+0000","last_became_peered":"2024-04-25T10:17:38.222857+0000","last_unstale":"2024-04-25T10:17:39.403248+0000","last_undegraded":"2024-04-25T10:17:39.403248+0000","last_fullsized":"2024-04-25T10:17:39.403248+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:11.265042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401683+0000","last_change":"2024-04-25T10:17:37.184886+0000","last_active":"2024-04-25T10:17:39.401683+0000","last_peered":"2024-04-25T10:17:39.401683+0000","last_clean":"2024-04-25T10:17:39.401683+0000","last_became_active":"2024-04-25T10:17:37.184114+0000","last_became_peered":"2024-04-25T10:17:37.184114+0000","last_unstale":"2024-04-25T10:17:39.401683+0000","last_undegraded":"2024-04-25T10:17:39.401683+0000","last_fullsized":"2024-04-25T10:17:39.401683+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:29.558500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403886+0000","last_change":"2024-04-25T10:17:38.221129+0000","last_active":"2024-04-25T10:17:39.403886+0000","last_peered":"2024-04-25T10:17:39.403886+0000","last_clean":"2024-04-25T10:17:39.403886+0000","last_became_active":"2024-04-25T10:17:38.218848+0000","last_became_peered":"2024-04-25T10:17:38.218848+0000","last_unstale":"2024-04-25T10:17:39.403886+0000","last_undegraded":"2024-04-25T10:17:39.403886+0000","last_fullsized":"2024-04-25T10:17:39.403886+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:38.235200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.208064+0000","last_change":"2024-04-25T10:17:37.180225+0000","last_active":"2024-04-25T10:17:39.208064+0000","last_peered":"2024-04-25T10:17:39.208064+0000","last_clean":"2024-04-25T10:17:39.208064+0000","last_became_active":"2024-04-25T10:17:37.179313+0000","last_became_peered":"2024-04-25T10:17:37.179313+0000","last_unstale":"2024-04-25T10:17:39.208064+0000","last_undegraded":"2024-04-25T10:17:39.208064+0000","last_fullsized":"2024-04-25T10:17:39.208064+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:00.872347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402630+0000","last_change":"2024-04-25T10:17:38.221613+0000","last_active":"2024-04-25T10:17:39.402630+0000","last_peered":"2024-04-25T10:17:39.402630+0000","last_clean":"2024-04-25T10:17:39.402630+0000","last_became_active":"2024-04-25T10:17:38.221327+0000","last_became_peered":"2024-04-25T10:17:38.221327+0000","last_unstale":"2024-04-25T10:17:39.402630+0000","last_undegraded":"2024-04-25T10:17:39.402630+0000","last_fullsized":"2024-04-25T10:17:39.402630+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:41.678472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.404550+0000","last_change":"2024-04-25T10:17:37.190849+0000","last_active":"2024-04-25T10:17:39.404550+0000","last_peered":"2024-04-25T10:17:39.404550+0000","last_clean":"2024-04-25T10:17:39.404550+0000","last_became_active":"2024-04-25T10:17:37.190468+0000","last_became_peered":"2024-04-25T10:17:37.190468+0000","last_unstale":"2024-04-25T10:17:39.404550+0000","last_undegraded":"2024-04-25T10:17:39.404550+0000","last_fullsized":"2024-04-25T10:17:39.404550+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:29.385847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401961+0000","last_change":"2024-04-25T10:17:38.215758+0000","last_active":"2024-04-25T10:17:39.401961+0000","last_peered":"2024-04-25T10:17:39.401961+0000","last_clean":"2024-04-25T10:17:39.401961+0000","last_became_active":"2024-04-25T10:17:38.214126+0000","last_became_peered":"2024-04-25T10:17:38.214126+0000","last_unstale":"2024-04-25T10:17:39.401961+0000","last_undegraded":"2024-04-25T10:17:39.401961+0000","last_fullsized":"2024-04-25T10:17:39.401961+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:50.552440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.204318+0000","last_change":"2024-04-25T10:17:37.182941+0000","last_active":"2024-04-25T10:17:39.204318+0000","last_peered":"2024-04-25T10:17:39.204318+0000","last_clean":"2024-04-25T10:17:39.204318+0000","last_became_active":"2024-04-25T10:17:37.182664+0000","last_became_peered":"2024-04-25T10:17:37.182664+0000","last_unstale":"2024-04-25T10:17:39.204318+0000","last_undegraded":"2024-04-25T10:17:39.204318+0000","last_fullsized":"2024-04-25T10:17:39.204318+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:23.014898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402137+0000","last_change":"2024-04-25T10:17:38.218460+0000","last_active":"2024-04-25T10:17:39.402137+0000","last_peered":"2024-04-25T10:17:39.402137+0000","last_clean":"2024-04-25T10:17:39.402137+0000","last_became_active":"2024-04-25T10:17:38.216286+0000","last_became_peered":"2024-04-25T10:17:38.216286+0000","last_unstale":"2024-04-25T10:17:39.402137+0000","last_undegraded":"2024-04-25T10:17:39.402137+0000","last_fullsized":"2024-04-25T10:17:39.402137+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.703896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401099+0000","last_change":"2024-04-25T10:17:37.185649+0000","last_active":"2024-04-25T10:17:39.401099+0000","last_peered":"2024-04-25T10:17:39.401099+0000","last_clean":"2024-04-25T10:17:39.401099+0000","last_became_active":"2024-04-25T10:17:37.185131+0000","last_became_peered":"2024-04-25T10:17:37.185131+0000","last_unstale":"2024-04-25T10:17:39.401099+0000","last_undegraded":"2024-04-25T10:17:39.401099+0000","last_fullsized":"2024-04-25T10:17:39.401099+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:24.059107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.400701+0000","last_change":"2024-04-25T10:17:38.205960+0000","last_active":"2024-04-25T10:17:39.400701+0000","last_peered":"2024-04-25T10:17:39.400701+0000","last_clean":"2024-04-25T10:17:39.400701+0000","last_became_active":"2024-04-25T10:17:38.205692+0000","last_became_peered":"2024-04-25T10:17:38.205692+0000","last_unstale":"2024-04-25T10:17:39.400701+0000","last_undegraded":"2024-04-25T10:17:39.400701+0000","last_fullsized":"2024-04-25T10:17:39.400701+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:14.531595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401916+0000","last_change":"2024-04-25T10:17:37.189790+0000","last_active":"2024-04-25T10:17:39.401916+0000","last_peered":"2024-04-25T10:17:39.401916+0000","last_clean":"2024-04-25T10:17:39.401916+0000","last_became_active":"2024-04-25T10:17:37.189293+0000","last_became_peered":"2024-04-25T10:17:37.189293+0000","last_unstale":"2024-04-25T10:17:39.401916+0000","last_undegraded":"2024-04-25T10:17:39.401916+0000","last_fullsized":"2024-04-25T10:17:39.401916+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:18.371926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401601+0000","last_change":"2024-04-25T10:17:38.221000+0000","last_active":"2024-04-25T10:17:39.401601+0000","last_peered":"2024-04-25T10:17:39.401601+0000","last_clean":"2024-04-25T10:17:39.401601+0000","last_became_active":"2024-04-25T10:17:38.220643+0000","last_became_peered":"2024-04-25T10:17:38.220643+0000","last_unstale":"2024-04-25T10:17:39.401601+0000","last_undegraded":"2024-04-25T10:17:39.401601+0000","last_fullsized":"2024-04-25T10:17:39.401601+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:04.703896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.405081+0000","last_change":"2024-04-25T10:17:37.179612+0000","last_active":"2024-04-25T10:17:39.405081+0000","last_peered":"2024-04-25T10:17:39.405081+0000","last_clean":"2024-04-25T10:17:39.405081+0000","last_became_active":"2024-04-25T10:17:37.178581+0000","last_became_peered":"2024-04-25T10:17:37.178581+0000","last_unstale":"2024-04-25T10:17:39.405081+0000","last_undegraded":"2024-04-25T10:17:39.405081+0000","last_fullsized":"2024-04-25T10:17:39.405081+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:21.307185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402762+0000","last_change":"2024-04-25T10:17:38.207880+0000","last_active":"2024-04-25T10:17:39.402762+0000","last_peered":"2024-04-25T10:17:39.402762+0000","last_clean":"2024-04-25T10:17:39.402762+0000","last_became_active":"2024-04-25T10:17:38.207508+0000","last_became_peered":"2024-04-25T10:17:38.207508+0000","last_unstale":"2024-04-25T10:17:39.402762+0000","last_undegraded":"2024-04-25T10:17:39.402762+0000","last_fullsized":"2024-04-25T10:17:39.402762+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:57.876347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"108'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:40.192635+0000","last_change":"2024-04-25T10:17:37.190109+0000","last_active":"2024-04-25T10:17:40.192635+0000","last_peered":"2024-04-25T10:17:40.192635+0000","last_clean":"2024-04-25T10:17:40.192635+0000","last_became_active":"2024-04-25T10:17:37.189636+0000","last_became_peered":"2024-04-25T10:17:37.189636+0000","last_unstale":"2024-04-25T10:17:40.192635+0000","last_undegraded":"2024-04-25T10:17:40.192635+0000","last_fullsized":"2024-04-25T10:17:40.192635+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:28.999385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402804+0000","last_change":"2024-04-25T10:17:38.216837+0000","last_active":"2024-04-25T10:17:39.402804+0000","last_peered":"2024-04-25T10:17:39.402804+0000","last_clean":"2024-04-25T10:17:39.402804+0000","last_became_active":"2024-04-25T10:17:38.214897+0000","last_became_peered":"2024-04-25T10:17:38.214897+0000","last_unstale":"2024-04-25T10:17:39.402804+0000","last_undegraded":"2024-04-25T10:17:39.402804+0000","last_fullsized":"2024-04-25T10:17:39.402804+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:12.833042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403062+0000","last_change":"2024-04-25T10:17:37.190370+0000","last_active":"2024-04-25T10:17:39.403062+0000","last_peered":"2024-04-25T10:17:39.403062+0000","last_clean":"2024-04-25T10:17:39.403062+0000","last_became_active":"2024-04-25T10:17:37.190014+0000","last_became_peered":"2024-04-25T10:17:37.190014+0000","last_unstale":"2024-04-25T10:17:39.403062+0000","last_undegraded":"2024-04-25T10:17:39.403062+0000","last_fullsized":"2024-04-25T10:17:39.403062+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:45.783509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.401565+0000","last_change":"2024-04-25T10:17:38.221421+0000","last_active":"2024-04-25T10:17:39.401565+0000","last_peered":"2024-04-25T10:17:39.401565+0000","last_clean":"2024-04-25T10:17:39.401565+0000","last_became_active":"2024-04-25T10:17:38.221021+0000","last_became_peered":"2024-04-25T10:17:38.221021+0000","last_unstale":"2024-04-25T10:17:39.401565+0000","last_undegraded":"2024-04-25T10:17:39.401565+0000","last_fullsized":"2024-04-25T10:17:39.401565+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:08.851641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.405090+0000","last_change":"2024-04-25T10:17:37.189210+0000","last_active":"2024-04-25T10:17:39.405090+0000","last_peered":"2024-04-25T10:17:39.405090+0000","last_clean":"2024-04-25T10:17:39.405090+0000","last_became_active":"2024-04-25T10:17:37.188916+0000","last_became_peered":"2024-04-25T10:17:37.188916+0000","last_unstale":"2024-04-25T10:17:39.405090+0000","last_undegraded":"2024-04-25T10:17:39.405090+0000","last_fullsized":"2024-04-25T10:17:39.405090+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:20.069783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205290+0000","last_change":"2024-04-25T10:17:38.201870+0000","last_active":"2024-04-25T10:17:39.205290+0000","last_peered":"2024-04-25T10:17:39.205290+0000","last_clean":"2024-04-25T10:17:39.205290+0000","last_became_active":"2024-04-25T10:17:38.201382+0000","last_became_peered":"2024-04-25T10:17:38.201382+0000","last_unstale":"2024-04-25T10:17:39.205290+0000","last_undegraded":"2024-04-25T10:17:39.205290+0000","last_fullsized":"2024-04-25T10:17:39.205290+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:48.495338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.207796+0000","last_change":"2024-04-25T10:17:37.181661+0000","last_active":"2024-04-25T10:17:39.207796+0000","last_peered":"2024-04-25T10:17:39.207796+0000","last_clean":"2024-04-25T10:17:39.207796+0000","last_became_active":"2024-04-25T10:17:37.181318+0000","last_became_peered":"2024-04-25T10:17:37.181318+0000","last_unstale":"2024-04-25T10:17:39.207796+0000","last_undegraded":"2024-04-25T10:17:39.207796+0000","last_fullsized":"2024-04-25T10:17:39.207796+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:43.543431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.206049+0000","last_change":"2024-04-25T10:17:38.206193+0000","last_active":"2024-04-25T10:17:39.206049+0000","last_peered":"2024-04-25T10:17:39.206049+0000","last_clean":"2024-04-25T10:17:39.206049+0000","last_became_active":"2024-04-25T10:17:38.205880+0000","last_became_peered":"2024-04-25T10:17:38.205880+0000","last_unstale":"2024-04-25T10:17:39.206049+0000","last_undegraded":"2024-04-25T10:17:39.206049+0000","last_fullsized":"2024-04-25T10:17:39.206049+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:11.265042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.205299+0000","last_change":"2024-04-25T10:17:37.186571+0000","last_active":"2024-04-25T10:17:39.205299+0000","last_peered":"2024-04-25T10:17:39.205299+0000","last_clean":"2024-04-25T10:17:39.205299+0000","last_became_active":"2024-04-25T10:17:37.186241+0000","last_became_peered":"2024-04-25T10:17:37.186241+0000","last_unstale":"2024-04-25T10:17:39.205299+0000","last_undegraded":"2024-04-25T10:17:39.205299+0000","last_fullsized":"2024-04-25T10:17:39.205299+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:47.774882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402198+0000","last_change":"2024-04-25T10:17:38.219212+0000","last_active":"2024-04-25T10:17:39.402198+0000","last_peered":"2024-04-25T10:17:39.402198+0000","last_clean":"2024-04-25T10:17:39.402198+0000","last_became_active":"2024-04-25T10:17:38.218921+0000","last_became_peered":"2024-04-25T10:17:38.218921+0000","last_unstale":"2024-04-25T10:17:39.402198+0000","last_undegraded":"2024-04-25T10:17:39.402198+0000","last_fullsized":"2024-04-25T10:17:39.402198+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:48.495338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.204241+0000","last_change":"2024-04-25T10:17:37.186109+0000","last_active":"2024-04-25T10:17:39.204241+0000","last_peered":"2024-04-25T10:17:39.204241+0000","last_clean":"2024-04-25T10:17:39.204241+0000","last_became_active":"2024-04-25T10:17:37.185631+0000","last_became_peered":"2024-04-25T10:17:37.185631+0000","last_unstale":"2024-04-25T10:17:39.204241+0000","last_undegraded":"2024-04-25T10:17:39.204241+0000","last_fullsized":"2024-04-25T10:17:39.204241+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:35.821245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.403395+0000","last_change":"2024-04-25T10:17:38.214549+0000","last_active":"2024-04-25T10:17:39.403395+0000","last_peered":"2024-04-25T10:17:39.403395+0000","last_clean":"2024-04-25T10:17:39.403395+0000","last_became_active":"2024-04-25T10:17:38.213672+0000","last_became_peered":"2024-04-25T10:17:38.213672+0000","last_unstale":"2024-04-25T10:17:39.403395+0000","last_undegraded":"2024-04-25T10:17:39.403395+0000","last_fullsized":"2024-04-25T10:17:39.403395+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:42.206645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402207+0000","last_change":"2024-04-25T10:17:37.189569+0000","last_active":"2024-04-25T10:17:39.402207+0000","last_peered":"2024-04-25T10:17:39.402207+0000","last_clean":"2024-04-25T10:17:39.402207+0000","last_became_active":"2024-04-25T10:17:37.188924+0000","last_became_peered":"2024-04-25T10:17:37.188924+0000","last_unstale":"2024-04-25T10:17:39.402207+0000","last_undegraded":"2024-04-25T10:17:39.402207+0000","last_fullsized":"2024-04-25T10:17:39.402207+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:58.180216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.400118+0000","last_change":"2024-04-25T10:17:38.211104+0000","last_active":"2024-04-25T10:17:39.400118+0000","last_peered":"2024-04-25T10:17:39.400118+0000","last_clean":"2024-04-25T10:17:39.400118+0000","last_became_active":"2024-04-25T10:17:38.209662+0000","last_became_peered":"2024-04-25T10:17:38.209662+0000","last_unstale":"2024-04-25T10:17:39.400118+0000","last_undegraded":"2024-04-25T10:17:39.400118+0000","last_fullsized":"2024-04-25T10:17:39.400118+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:37.147097+0000","last_clean_scrub_stamp":"2024-04-25T10:17:37.147097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:38.235200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.207252+0000","last_change":"2024-04-25T10:17:37.185032+0000","last_active":"2024-04-25T10:17:39.207252+0000","last_peered":"2024-04-25T10:17:39.207252+0000","last_clean":"2024-04-25T10:17:39.207252+0000","last_became_active":"2024-04-25T10:17:37.184715+0000","last_became_peered":"2024-04-25T10:17:37.184715+0000","last_unstale":"2024-04-25T10:17:39.207252+0000","last_undegraded":"2024-04-25T10:17:39.207252+0000","last_fullsized":"2024-04-25T10:17:39.207252+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:36.132320+0000","last_clean_scrub_stamp":"2024-04-25T10:17:36.132320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:13.304049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":337,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:17:39.402421+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:17:39.402421+0000","last_peered":"2024-04-25T10:17:39.402421+0000","last_clean":"2024-04-25T10:17:39.402421+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:17:39.402421+0000","last_undegraded":"2024-04-25T10:17:39.402421+0000","last_fullsized":"2024-04-25T10:17:39.402421+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1945600,"data_stored":1172546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1347584,"data_stored":581632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":582166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1343488,"data_stored":581562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1359872,"data_stored":583244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1343488,"data_stored":581562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":581686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1957888,"data_stored":1173654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:17:43.093 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:17:43.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:17:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.342+0000 7f3beb400640 1 Processor -- start 2024-04-25T10:17:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.342+0000 7f3beb400640 1 -- start start 2024-04-25T10:17:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.344+0000 7f3beb400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3be4165480 0x7f3be40b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.344+0000 7f3beb400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3be41636d0 0x7f3be40b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.344+0000 7f3beb400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be40b3610 0x7f3be4071300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.344+0000 7f3beb400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be4165ce0 con 0x7f3be41636d0 2024-04-25T10:17:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.344+0000 7f3beb400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be4165fc0 con 0x7f3be4165480 2024-04-25T10:17:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.344+0000 7f3beb400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be41662a0 con 0x7f3be40b3610 2024-04-25T10:17:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.344+0000 7f3bdbe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3be41636d0 0x7f3be40b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.344+0000 7f3bdbe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3be41636d0 0x7f3be40b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36958/0 (socket says 172.21.15.119:36958) 2024-04-25T10:17:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.344+0000 7f3bdbe00640 1 -- 172.21.15.119:0/2726220758 learned_addr learned my addr 172.21.15.119:0/2726220758 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.345+0000 7f3be8a00640 1 --2- 172.21.15.119:0/2726220758 >> v2:172.21.15.132:3300/0 conn(0x7f3be4165480 0x7f3be40b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.345+0000 7f3be9400640 1 --2- 172.21.15.119:0/2726220758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be40b3610 0x7f3be4071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.345+0000 7f3bdbe00640 1 -- 172.21.15.119:0/2726220758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be40b3610 msgr2=0x7f3be4071300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.345+0000 7f3bdbe00640 1 --2- 172.21.15.119:0/2726220758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be40b3610 0x7f3be4071300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.345+0000 7f3bdbe00640 1 -- 172.21.15.119:0/2726220758 >> v2:172.21.15.132:3300/0 conn(0x7f3be4165480 msgr2=0x7f3be40b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.345+0000 7f3bdbe00640 1 --2- 172.21.15.119:0/2726220758 >> v2:172.21.15.132:3300/0 conn(0x7f3be4165480 0x7f3be40b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.345+0000 7f3bdbe00640 1 -- 172.21.15.119:0/2726220758 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be4166580 con 0x7f3be41636d0 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.345+0000 7f3be9400640 1 --2- 172.21.15.119:0/2726220758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be40b3610 0x7f3be4071300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.346+0000 7f3bdbe00640 1 --2- 172.21.15.119:0/2726220758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be41636d0 0x7f3be40b30d0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f3be41661d0 tx=0x7f3bcc02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704c5b9108ef6659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.346+0000 7f3be3e00640 1 -- 172.21.15.119:0/2726220758 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc037070 con 0x7f3be41636d0 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.346+0000 7f3be3e00640 1 -- 172.21.15.119:0/2726220758 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc032440 con 0x7f3be41636d0 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.346+0000 7f3be3e00640 1 -- 172.21.15.119:0/2726220758 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc032810 con 0x7f3be41636d0 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.347+0000 7f3beb400640 1 -- 172.21.15.119:0/2726220758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be41636d0 msgr2=0x7f3be40b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.347+0000 7f3beb400640 1 --2- 172.21.15.119:0/2726220758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be41636d0 0x7f3be40b30d0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f3be41661d0 tx=0x7f3bcc02a130 comp rx=0 tx=0).stop 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.347+0000 7f3beb400640 1 -- 172.21.15.119:0/2726220758 shutdown_connections 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.347+0000 7f3beb400640 1 --2- 172.21.15.119:0/2726220758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be40b3610 0x7f3be4071300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.347+0000 7f3beb400640 1 --2- 172.21.15.119:0/2726220758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be41636d0 0x7f3be40b30d0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.347+0000 7f3beb400640 1 --2- 172.21.15.119:0/2726220758 >> v2:172.21.15.132:3300/0 conn(0x7f3be4165480 0x7f3be40b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.347+0000 7f3beb400640 1 -- 172.21.15.119:0/2726220758 >> 172.21.15.119:0/2726220758 conn(0x7f3be406aba0 msgr2=0x7f3be406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.347+0000 7f3beb400640 1 -- 172.21.15.119:0/2726220758 shutdown_connections 2024-04-25T10:17:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.348+0000 7f3beb400640 1 -- 172.21.15.119:0/2726220758 wait complete. 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.350+0000 7f3beb400640 1 Processor -- start 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.350+0000 7f3beb400640 1 -- start start 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.351+0000 7f3beb400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be40b3610 0x7f3be4156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.351+0000 7f3beb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be41636d0 0x7f3be40adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.351+0000 7f3beb400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be4165480 0x7f3be40ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.351+0000 7f3beb400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be406e980 con 0x7f3be41636d0 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.351+0000 7f3beb400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be4165860 con 0x7f3be4165480 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.351+0000 7f3beb400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be406eb00 con 0x7f3be40b3610 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.351+0000 7f3bdbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be41636d0 0x7f3be40adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.351+0000 7f3bdbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be41636d0 0x7f3be40adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36974/0 (socket says 172.21.15.119:36974) 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.351+0000 7f3bdbe00640 1 -- 172.21.15.119:0/1909752655 learned_addr learned my addr 172.21.15.119:0/1909752655 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.352+0000 7f3bdbe00640 1 -- 172.21.15.119:0/1909752655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be40b3610 msgr2=0x7f3be4156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.352+0000 7f3bdbe00640 1 --2- 172.21.15.119:0/1909752655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be40b3610 0x7f3be4156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.352+0000 7f3bdbe00640 1 -- 172.21.15.119:0/1909752655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be4165480 msgr2=0x7f3be40ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.352+0000 7f3bdbe00640 1 --2- 172.21.15.119:0/1909752655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be4165480 0x7f3be40ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.352+0000 7f3bdbe00640 1 -- 172.21.15.119:0/1909752655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be40aae30 con 0x7f3be41636d0 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.352+0000 7f3bdbe00640 1 --2- 172.21.15.119:0/1909752655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be41636d0 0x7f3be40adb90 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc02a170 tx=0x7f3bcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.352+0000 7f3be1600640 1 -- 172.21.15.119:0/1909752655 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc041020 con 0x7f3be41636d0 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.352+0000 7f3be1600640 1 -- 172.21.15.119:0/1909752655 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc0306c0 con 0x7f3be41636d0 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.352+0000 7f3beb400640 1 -- 172.21.15.119:0/1909752655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be40ab3f0 con 0x7f3be41636d0 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.352+0000 7f3be1600640 1 -- 172.21.15.119:0/1909752655 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc037040 con 0x7f3be41636d0 2024-04-25T10:17:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.352+0000 7f3beb400640 1 -- 172.21.15.119:0/1909752655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be40ab9b0 con 0x7f3be41636d0 2024-04-25T10:17:43.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.354+0000 7f3beb400640 1 -- 172.21.15.119:0/1909752655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be40749b0 con 0x7f3be41636d0 2024-04-25T10:17:43.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.355+0000 7f3be1600640 1 -- 172.21.15.119:0/1909752655 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bcc044020 con 0x7f3be41636d0 2024-04-25T10:17:43.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.356+0000 7f3be1600640 1 --2- 172.21.15.119:0/1909752655 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bc006b4d0 0x7f3bc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.356+0000 7f3be1600640 1 -- 172.21.15.119:0/1909752655 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bcc0aa980 con 0x7f3be41636d0 2024-04-25T10:17:43.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.356+0000 7f3be8a00640 1 --2- 172.21.15.119:0/1909752655 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bc006b4d0 0x7f3bc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.357+0000 7f3be8a00640 1 --2- 172.21.15.119:0/1909752655 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bc006b4d0 0x7f3bc006d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40098f0 tx=0x7f3bd4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:43.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.361+0000 7f3be1600640 1 -- 172.21.15.119:0/1909752655 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3bcc04b030 con 0x7f3be41636d0 2024-04-25T10:17:43.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.624+0000 7f3beb400640 1 -- 172.21.15.119:0/1909752655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3be40b3d90 con 0x7f3be41636d0 2024-04-25T10:17:43.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.628+0000 7f3be1600640 1 -- 172.21.15.119:0/1909752655 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3bcc079890 con 0x7f3be41636d0 2024-04-25T10:17:43.631 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-04-25T10:17:43.632 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQAMKypm3kXKCBAA/QMcKcvvGvN94xOBTeTAKA== 2024-04-25T10:17:43.632 INFO:teuthology.orchestra.run.smithi119.stdout: caps mds = "allow" 2024-04-25T10:17:43.632 INFO:teuthology.orchestra.run.smithi119.stdout: caps mon = "allow r" 2024-04-25T10:17:43.632 INFO:teuthology.orchestra.run.smithi119.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:17:43.632 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2024-04-25T10:17:43.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.631+0000 7f3beb400640 1 -- 172.21.15.119:0/1909752655 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bc006b4d0 msgr2=0x7f3bc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.631+0000 7f3beb400640 1 --2- 172.21.15.119:0/1909752655 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bc006b4d0 0x7f3bc006d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40098f0 tx=0x7f3bd4008040 comp rx=0 tx=0).stop 2024-04-25T10:17:43.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.631+0000 7f3beb400640 1 -- 172.21.15.119:0/1909752655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be41636d0 msgr2=0x7f3be40adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.631+0000 7f3beb400640 1 --2- 172.21.15.119:0/1909752655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be41636d0 0x7f3be40adb90 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc02a170 tx=0x7f3bcc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.631+0000 7f3beb400640 1 -- 172.21.15.119:0/1909752655 shutdown_connections 2024-04-25T10:17:43.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.631+0000 7f3beb400640 1 --2- 172.21.15.119:0/1909752655 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3bc006b4d0 0x7f3bc006d990 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.631+0000 7f3beb400640 1 --2- 172.21.15.119:0/1909752655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3be4165480 0x7f3be40ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.631+0000 7f3beb400640 1 --2- 172.21.15.119:0/1909752655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3be41636d0 0x7f3be40adb90 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.631+0000 7f3beb400640 1 --2- 172.21.15.119:0/1909752655 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3be40b3610 0x7f3be4156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.631+0000 7f3beb400640 1 -- 172.21.15.119:0/1909752655 >> 172.21.15.119:0/1909752655 conn(0x7f3be406aba0 msgr2=0x7f3be4161980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:43.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.631+0000 7f3beb400640 1 -- 172.21.15.119:0/1909752655 shutdown_connections 2024-04-25T10:17:43.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.631+0000 7f3beb400640 1 -- 172.21.15.119:0/1909752655 wait complete. 2024-04-25T10:17:43.653 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:43.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.841+0000 7f9a92c00640 1 Processor -- start 2024-04-25T10:17:43.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.841+0000 7f9a92c00640 1 -- start start 2024-04-25T10:17:43.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.843+0000 7f9a92c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9a8c0d3f80 0x7f9a8c0d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.843+0000 7f9a92c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a8c0b9270 0x7f9a8c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.843+0000 7f9a92c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a8c15ef10 con 0x7f9a8c15db60 2024-04-25T10:17:43.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.843+0000 7f9a92c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8c15e390 con 0x7f9a8c0d3f80 2024-04-25T10:17:43.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.843+0000 7f9a92c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c15e670 con 0x7f9a8c0b9270 2024-04-25T10:17:43.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.843+0000 7f9a91800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9a8c15db60 0x7f9a8c0d3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56860/0 (socket says 172.21.15.119:56860) 2024-04-25T10:17:43.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.843+0000 7f9a91800640 1 -- 172.21.15.119:0/384754320 learned_addr learned my addr 172.21.15.119:0/384754320 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:43.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.843+0000 7f9a8be00640 1 -- 172.21.15.119:0/384754320 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802204465 0 0) 0x7f9a8c15ef10 con 0x7f9a8c15db60 2024-04-25T10:17:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.843+0000 7f9a8be00640 1 -- 172.21.15.119:0/384754320 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a74003520 con 0x7f9a8c15db60 2024-04-25T10:17:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.843+0000 7f9a90e00640 1 --2- 172.21.15.119:0/384754320 >> v2:172.21.15.132:3300/0 conn(0x7f9a8c0d3f80 0x7f9a8c0d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.843+0000 7f9a92200640 1 --2- 172.21.15.119:0/384754320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a8c0b9270 0x7f9a8c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a8be00640 1 -- 172.21.15.119:0/384754320 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1188227013 0 0) 0x7f9a74003520 con 0x7f9a8c15db60 2024-04-25T10:17:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a8be00640 1 -- 172.21.15.119:0/384754320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a8c0b9270 msgr2=0x7f9a8c0d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a8be00640 1 --2- 172.21.15.119:0/384754320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a8c0b9270 0x7f9a8c0d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a8be00640 1 -- 172.21.15.119:0/384754320 >> v2:172.21.15.132:3300/0 conn(0x7f9a8c0d3f80 msgr2=0x7f9a8c0d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a8be00640 1 --2- 172.21.15.119:0/384754320 >> v2:172.21.15.132:3300/0 conn(0x7f9a8c0d3f80 0x7f9a8c0d1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a8be00640 1 -- 172.21.15.119:0/384754320 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c15e950 con 0x7f9a8c15db60 2024-04-25T10:17:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a8be00640 1 -- 172.21.15.119:0/384754320 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f9a80001690 con 0x7f9a8c15db60 2024-04-25T10:17:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a8be00640 1 -- 172.21.15.119:0/384754320 >> v1:172.21.15.119:6789/0 conn(0x7f9a8c15db60 legacy=0x7f9a8c0d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a8be00640 1 --2- 172.21.15.119:0/384754320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a74003a20 0x7f9a74003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a8be00640 1 --2- 172.21.15.119:0/384754320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a8c0b9270 0x7f9a74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a8be00640 1 --2- 172.21.15.119:0/384754320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74004d40 0x7f9a740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a8be00640 1 -- 172.21.15.119:0/384754320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c15e390 con 0x7f9a74004d40 2024-04-25T10:17:43.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a8be00640 1 -- 172.21.15.119:0/384754320 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c15e670 con 0x7f9a74003a20 2024-04-25T10:17:43.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a8be00640 1 -- 172.21.15.119:0/384754320 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7400bb20 con 0x7f9a8c0b9270 2024-04-25T10:17:43.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a92200640 1 --2- 172.21.15.119:0/384754320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74004d40 0x7f9a740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.844+0000 7f9a90e00640 1 --2- 172.21.15.119:0/384754320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a8c0b9270 0x7f9a74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.845+0000 7f9a92200640 1 -- 172.21.15.119:0/384754320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a8c0b9270 msgr2=0x7f9a74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:43.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.845+0000 7f9a92200640 1 --2- 172.21.15.119:0/384754320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a8c0b9270 0x7f9a74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.845+0000 7f9a92200640 1 -- 172.21.15.119:0/384754320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a74003a20 msgr2=0x7f9a74003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:43.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.845+0000 7f9a91800640 1 --2- 172.21.15.119:0/384754320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a74003a20 0x7f9a74003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.845+0000 7f9a92200640 1 --2- 172.21.15.119:0/384754320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a74003a20 0x7f9a74003e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.845+0000 7f9a92200640 1 -- 172.21.15.119:0/384754320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a74003520 con 0x7f9a74004d40 2024-04-25T10:17:43.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.846+0000 7f9a92200640 1 --2- 172.21.15.119:0/384754320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74004d40 0x7f9a740051c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f9a84007330 tx=0x7f9a74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f66b2bb8127fb56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:43.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.846+0000 7f9a8be00640 1 -- 172.21.15.119:0/384754320 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84033070 con 0x7f9a74004d40 2024-04-25T10:17:43.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.846+0000 7f9a8be00640 1 -- 172.21.15.119:0/384754320 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8402f650 con 0x7f9a74004d40 2024-04-25T10:17:43.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.846+0000 7f9a8be00640 1 -- 172.21.15.119:0/384754320 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84009be0 con 0x7f9a74004d40 2024-04-25T10:17:43.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.846+0000 7f9a92c00640 1 -- 172.21.15.119:0/384754320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74004d40 msgr2=0x7f9a740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.846+0000 7f9a92c00640 1 --2- 172.21.15.119:0/384754320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74004d40 0x7f9a740051c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f9a84007330 tx=0x7f9a74005830 comp rx=0 tx=0).stop 2024-04-25T10:17:43.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.847+0000 7f9a92200640 1 -- 172.21.15.119:0/384754320 reap_dead start 2024-04-25T10:17:43.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.847+0000 7f9a92c00640 1 -- 172.21.15.119:0/384754320 shutdown_connections 2024-04-25T10:17:43.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.847+0000 7f9a92c00640 1 -- 172.21.15.119:0/384754320 >> 172.21.15.119:0/384754320 conn(0x7f9a8c0726c0 msgr2=0x7f9a8c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:43.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.847+0000 7f9a92c00640 1 -- 172.21.15.119:0/384754320 shutdown_connections 2024-04-25T10:17:43.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.847+0000 7f9a92c00640 1 -- 172.21.15.119:0/384754320 wait complete. 2024-04-25T10:17:43.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.848+0000 7f9a92c00640 1 Processor -- start 2024-04-25T10:17:43.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.848+0000 7f9a92c00640 1 -- start start 2024-04-25T10:17:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.849+0000 7f9a92c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a74004d40 0x7f9a8c14e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.849+0000 7f9a92c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a8c14dc90 0x7f9a8c0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.849+0000 7f9a92c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a8c0cbce0 0x7f9a8c0c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.849+0000 7f9a92c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c0b3120 con 0x7f9a8c14dc90 2024-04-25T10:17:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.849+0000 7f9a92c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c0b2fa0 con 0x7f9a74004d40 2024-04-25T10:17:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.849+0000 7f9a92c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c15df40 con 0x7f9a8c0cbce0 2024-04-25T10:17:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.850+0000 7f9a90e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a8c14dc90 0x7f9a8c0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.850+0000 7f9a90e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a8c14dc90 0x7f9a8c0cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36998/0 (socket says 172.21.15.119:36998) 2024-04-25T10:17:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.850+0000 7f9a90e00640 1 -- 172.21.15.119:0/3045819606 learned_addr learned my addr 172.21.15.119:0/3045819606 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.850+0000 7f9a91800640 1 --2- 172.21.15.119:0/3045819606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a74004d40 0x7f9a8c14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.850+0000 7f9a90e00640 1 -- 172.21.15.119:0/3045819606 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a8c0cbce0 msgr2=0x7f9a8c0c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.850+0000 7f9a90e00640 1 --2- 172.21.15.119:0/3045819606 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a8c0cbce0 0x7f9a8c0c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.850+0000 7f9a90e00640 1 -- 172.21.15.119:0/3045819606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a74004d40 msgr2=0x7f9a8c14e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.850+0000 7f9a90e00640 1 --2- 172.21.15.119:0/3045819606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a74004d40 0x7f9a8c14e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.850+0000 7f9a90e00640 1 -- 172.21.15.119:0/3045819606 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c0cc160 con 0x7f9a8c14dc90 2024-04-25T10:17:43.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.850+0000 7f9a90e00640 1 --2- 172.21.15.119:0/3045819606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a8c14dc90 0x7f9a8c0cb7a0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800f610 tx=0x7f9a7800fad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:43.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.850+0000 7f9a89600640 1 -- 172.21.15.119:0/3045819606 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78019030 con 0x7f9a8c14dc90 2024-04-25T10:17:43.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.851+0000 7f9a89600640 1 -- 172.21.15.119:0/3045819606 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a78005990 con 0x7f9a8c14dc90 2024-04-25T10:17:43.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.851+0000 7f9a89600640 1 -- 172.21.15.119:0/3045819606 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78005d60 con 0x7f9a8c14dc90 2024-04-25T10:17:43.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.851+0000 7f9a92c00640 1 -- 172.21.15.119:0/3045819606 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c0b4ab0 con 0x7f9a8c14dc90 2024-04-25T10:17:43.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.851+0000 7f9a92c00640 1 -- 172.21.15.119:0/3045819606 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8c0b5070 con 0x7f9a8c14dc90 2024-04-25T10:17:43.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.852+0000 7f9a92c00640 1 -- 172.21.15.119:0/3045819606 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8c0749b0 con 0x7f9a8c14dc90 2024-04-25T10:17:43.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.853+0000 7f9a89600640 1 -- 172.21.15.119:0/3045819606 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a7801c070 con 0x7f9a8c14dc90 2024-04-25T10:17:43.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.854+0000 7f9a89600640 1 --2- 172.21.15.119:0/3045819606 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a6c06b4d0 0x7f9a6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.854+0000 7f9a89600640 1 -- 172.21.15.119:0/3045819606 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a7808a790 con 0x7f9a8c14dc90 2024-04-25T10:17:43.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.854+0000 7f9a91800640 1 --2- 172.21.15.119:0/3045819606 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a6c06b4d0 0x7f9a6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.855+0000 7f9a91800640 1 --2- 172.21.15.119:0/3045819606 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a6c06b4d0 0x7f9a6c06d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9a80001900 tx=0x7f9a80009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:43.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:43.858+0000 7f9a89600640 1 -- 172.21.15.119:0/3045819606 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a780594e0 con 0x7f9a8c14dc90 2024-04-25T10:17:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.073+0000 7f9a92c00640 1 -- 172.21.15.119:0/3045819606 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a8c0d1a60 con 0x7f9a8c14dc90 2024-04-25T10:17:44.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.074+0000 7f9a89600640 1 -- 172.21.15.119:0/3045819606 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f9a78058de0 con 0x7f9a8c14dc90 2024-04-25T10:17:44.078 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:44.078 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3394261143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3394261143},{"type":"v1","addr":"172.21.15.132:6834","nonce":3394261143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2502874931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2502874931},{"type":"v1","addr":"172.21.15.119:6835","nonce":2502874931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":8694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2305428003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2305428003},{"type":"v1","addr":"172.21.15.132:6835","nonce":2305428003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:37.617787+0000","modified":"2024-04-25T10:17:40.692459+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8697":{"gid":8697,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/1849584205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":1849584205},{"type":"v1","addr":"172.21.15.119:6837","nonce":1849584205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:17:44.078 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 99 2024-04-25T10:17:44.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.077+0000 7f9a92c00640 1 -- 172.21.15.119:0/3045819606 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a6c06b4d0 msgr2=0x7f9a6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.077+0000 7f9a92c00640 1 --2- 172.21.15.119:0/3045819606 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a6c06b4d0 0x7f9a6c06d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9a80001900 tx=0x7f9a80009040 comp rx=0 tx=0).stop 2024-04-25T10:17:44.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.077+0000 7f9a92c00640 1 -- 172.21.15.119:0/3045819606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a8c14dc90 msgr2=0x7f9a8c0cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.077+0000 7f9a92c00640 1 --2- 172.21.15.119:0/3045819606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a8c14dc90 0x7f9a8c0cb7a0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800f610 tx=0x7f9a7800fad0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.077+0000 7f9a92c00640 1 -- 172.21.15.119:0/3045819606 shutdown_connections 2024-04-25T10:17:44.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.077+0000 7f9a92c00640 1 --2- 172.21.15.119:0/3045819606 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9a6c06b4d0 0x7f9a6c06d990 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.077+0000 7f9a92c00640 1 --2- 172.21.15.119:0/3045819606 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9a8c0cbce0 0x7f9a8c0c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.077+0000 7f9a92c00640 1 --2- 172.21.15.119:0/3045819606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a8c14dc90 0x7f9a8c0cb7a0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.077+0000 7f9a92c00640 1 --2- 172.21.15.119:0/3045819606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a74004d40 0x7f9a8c14e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.077+0000 7f9a92c00640 1 -- 172.21.15.119:0/3045819606 >> 172.21.15.119:0/3045819606 conn(0x7f9a8c0726c0 msgr2=0x7f9a8c0b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:44.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.077+0000 7f9a92c00640 1 -- 172.21.15.119:0/3045819606 shutdown_connections 2024-04-25T10:17:44.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.077+0000 7f9a92c00640 1 -- 172.21.15.119:0/3045819606 wait complete. 2024-04-25T10:17:44.130 INFO:tasks.ceph.mds.c.smithi119.stderr:2024-04-25T10:17:44.126+0000 7fd6f4400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:17:44.502 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:17:37.617787+0000', 'modified': '2024-04-25T10:17:40.692459+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8697}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8697': {'gid': 8697, 'name': 'c', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6837/1849584205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 1849584205}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 1849584205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:17:44.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:17:44.503 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:17:44.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.707+0000 7fdba7c00640 1 Processor -- start 2024-04-25T10:17:44.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.708+0000 7fdba7c00640 1 -- start start 2024-04-25T10:17:44.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.709+0000 7fdba7c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdba0161090 0x7fdba0161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.709+0000 7fdba7c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdba00ceeb0 0x7fdba00d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.709+0000 7fdba7c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba0161f80 con 0x7fdba00b9720 2024-04-25T10:17:44.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.709+0000 7fdba7c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba00d21a0 con 0x7fdba0161090 2024-04-25T10:17:44.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.709+0000 7fdba7c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba0162260 con 0x7fdba00ceeb0 2024-04-25T10:17:44.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.709+0000 7fdba5200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdba0161090 0x7fdba0161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.709+0000 7fdba5200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdba0161090 0x7fdba0161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37004/0 (socket says 172.21.15.119:37004) 2024-04-25T10:17:44.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.709+0000 7fdba5200640 1 -- 172.21.15.119:0/1053594500 learned_addr learned my addr 172.21.15.119:0/1053594500 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:44.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.710+0000 7fdba5c00640 1 --2- 172.21.15.119:0/1053594500 >> v2:172.21.15.132:3300/0 conn(0x7fdba00ceeb0 0x7fdba00d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.710+0000 7fdba5200640 1 -- 172.21.15.119:0/1053594500 >> v2:172.21.15.132:3300/0 conn(0x7fdba00ceeb0 msgr2=0x7fdba00d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.710+0000 7fdba5200640 1 --2- 172.21.15.119:0/1053594500 >> v2:172.21.15.132:3300/0 conn(0x7fdba00ceeb0 0x7fdba00d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.710+0000 7fdba5200640 1 -- 172.21.15.119:0/1053594500 >> v1:172.21.15.132:6789/0 conn(0x7fdba00b9720 legacy=0x7fdba00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.710+0000 7fdba5200640 1 -- 172.21.15.119:0/1053594500 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba0162540 con 0x7fdba0161090 2024-04-25T10:17:44.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.711+0000 7fdba5200640 1 --2- 172.21.15.119:0/1053594500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba0161090 0x7fdba0161730 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fdba00b69d0 tx=0x7fdb9402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d77c03b6c140bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:44.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.711+0000 7fdb9f400640 1 -- 172.21.15.119:0/1053594500 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94037070 con 0x7fdba0161090 2024-04-25T10:17:44.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.711+0000 7fdb9f400640 1 -- 172.21.15.119:0/1053594500 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb94032440 con 0x7fdba0161090 2024-04-25T10:17:44.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.712+0000 7fdb9f400640 1 -- 172.21.15.119:0/1053594500 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94032810 con 0x7fdba0161090 2024-04-25T10:17:44.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.712+0000 7fdba7c00640 1 -- 172.21.15.119:0/1053594500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba0161090 msgr2=0x7fdba0161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.712+0000 7fdba7c00640 1 --2- 172.21.15.119:0/1053594500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba0161090 0x7fdba0161730 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fdba00b69d0 tx=0x7fdb9402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.712+0000 7fdba7c00640 1 -- 172.21.15.119:0/1053594500 shutdown_connections 2024-04-25T10:17:44.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.712+0000 7fdba7c00640 1 --2- 172.21.15.119:0/1053594500 >> v2:172.21.15.132:3300/0 conn(0x7fdba00ceeb0 0x7fdba00d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.712+0000 7fdba7c00640 1 --2- 172.21.15.119:0/1053594500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba0161090 0x7fdba0161730 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.712+0000 7fdba7c00640 1 -- 172.21.15.119:0/1053594500 >> 172.21.15.119:0/1053594500 conn(0x7fdba00b55e0 msgr2=0x7fdba00b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:44.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.712+0000 7fdba7c00640 1 -- 172.21.15.119:0/1053594500 shutdown_connections 2024-04-25T10:17:44.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.712+0000 7fdba7c00640 1 -- 172.21.15.119:0/1053594500 wait complete. 2024-04-25T10:17:44.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.714+0000 7fdba7c00640 1 Processor -- start 2024-04-25T10:17:44.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.714+0000 7fdba7c00640 1 -- start start 2024-04-25T10:17:44.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.715+0000 7fdba7c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba00b9720 0x7fdba015b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.715+0000 7fdba7c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba00ceeb0 0x7fdba01f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.715+0000 7fdba7c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba0161090 0x7fdba00aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.715+0000 7fdba7c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba0161c70 con 0x7fdba00b9720 2024-04-25T10:17:44.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.715+0000 7fdba7c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba0161470 con 0x7fdba0161090 2024-04-25T10:17:44.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.715+0000 7fdba7c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00d2b70 con 0x7fdba00ceeb0 2024-04-25T10:17:44.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.715+0000 7fdba5200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba00b9720 0x7fdba015b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.715+0000 7fdba5200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba00b9720 0x7fdba015b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37016/0 (socket says 172.21.15.119:37016) 2024-04-25T10:17:44.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.715+0000 7fdba5200640 1 -- 172.21.15.119:0/3122053696 learned_addr learned my addr 172.21.15.119:0/3122053696 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:44.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.716+0000 7fdba5200640 1 -- 172.21.15.119:0/3122053696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba00ceeb0 msgr2=0x7fdba01f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.716+0000 7fdba5200640 1 --2- 172.21.15.119:0/3122053696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba00ceeb0 0x7fdba01f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.716+0000 7fdba5200640 1 -- 172.21.15.119:0/3122053696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba0161090 msgr2=0x7fdba00aaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.716+0000 7fdba5200640 1 --2- 172.21.15.119:0/3122053696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba0161090 0x7fdba00aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.716+0000 7fdba5200640 1 -- 172.21.15.119:0/3122053696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba00ab9a0 con 0x7fdba00b9720 2024-04-25T10:17:44.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.716+0000 7fdba5200640 1 --2- 172.21.15.119:0/3122053696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba00b9720 0x7fdba015b1e0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fdb9402a1e0 tx=0x7fdb940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:44.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.716+0000 7fdb9cc00640 1 -- 172.21.15.119:0/3122053696 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94041020 con 0x7fdba00b9720 2024-04-25T10:17:44.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.716+0000 7fdb9cc00640 1 -- 172.21.15.119:0/3122053696 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb940306c0 con 0x7fdba00b9720 2024-04-25T10:17:44.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.716+0000 7fdb9cc00640 1 -- 172.21.15.119:0/3122053696 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94037040 con 0x7fdba00b9720 2024-04-25T10:17:44.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.716+0000 7fdba7c00640 1 -- 172.21.15.119:0/3122053696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba00cc030 con 0x7fdba00b9720 2024-04-25T10:17:44.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.716+0000 7fdba7c00640 1 -- 172.21.15.119:0/3122053696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba00cc5f0 con 0x7fdba00b9720 2024-04-25T10:17:44.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.718+0000 7fdb9cc00640 1 -- 172.21.15.119:0/3122053696 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb94044020 con 0x7fdba00b9720 2024-04-25T10:17:44.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.719+0000 7fdb9cc00640 1 --2- 172.21.15.119:0/3122053696 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb8006b500 0x7fdb8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.719+0000 7fdb9cc00640 1 -- 172.21.15.119:0/3122053696 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb940aaaa0 con 0x7fdba00b9720 2024-04-25T10:17:44.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.720+0000 7fdb9fe00640 1 --2- 172.21.15.119:0/3122053696 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb8006b500 0x7fdb8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.720+0000 7fdb9fe00640 1 --2- 172.21.15.119:0/3122053696 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb8006b500 0x7fdb8006d9c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdba00680b0 tx=0x7fdb8c0079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:44.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.720+0000 7fdbaa99c740 1 Processor -- start 2024-04-25T10:17:44.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.721+0000 7fdbaa99c740 1 -- start start 2024-04-25T10:17:44.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.721+0000 7fdbaa99c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a181c950 0x5639a1844790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.721+0000 7fdbaa99c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1844cd0 0x5639a181cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.721+0000 7fdbaa99c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a18655c0 0x5639a181b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.721+0000 7fdbaa99c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a1780620 con 0x5639a181c950 2024-04-25T10:17:44.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.721+0000 7fdbaa99c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a1729ca0 con 0x5639a1844cd0 2024-04-25T10:17:44.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.721+0000 7fdbaa99c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639a177c6c0 con 0x5639a18655c0 2024-04-25T10:17:44.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.722+0000 7fdba5200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a181c950 0x5639a1844790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.722+0000 7fdba5200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a181c950 0x5639a1844790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37024/0 (socket says 172.21.15.119:37024) 2024-04-25T10:17:44.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.722+0000 7fdba5200640 1 -- 172.21.15.119:0/613879381 learned_addr learned my addr 172.21.15.119:0/613879381 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:44.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.722+0000 7fdba5200640 1 -- 172.21.15.119:0/613879381 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a18655c0 msgr2=0x5639a181b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.722+0000 7fdba5200640 1 --2- 172.21.15.119:0/613879381 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a18655c0 0x5639a181b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.722+0000 7fdba5200640 1 -- 172.21.15.119:0/613879381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1844cd0 msgr2=0x5639a181cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.722+0000 7fdba5200640 1 --2- 172.21.15.119:0/613879381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1844cd0 0x5639a181cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.722+0000 7fdba5200640 1 -- 172.21.15.119:0/613879381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639a1730f40 con 0x5639a181c950 2024-04-25T10:17:44.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.722+0000 7fdba5200640 1 --2- 172.21.15.119:0/613879381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a181c950 0x5639a1844790 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fdb94081360 tx=0x7fdb94079880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f422ba8ad782d9bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:44.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.722+0000 7fdb91600640 1 -- 172.21.15.119:0/613879381 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94079180 con 0x5639a181c950 2024-04-25T10:17:44.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.723+0000 7fdb91600640 1 -- 172.21.15.119:0/613879381 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9407d2d0 con 0x5639a181c950 2024-04-25T10:17:44.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.723+0000 7fdb91600640 1 -- 172.21.15.119:0/613879381 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94081890 con 0x5639a181c950 2024-04-25T10:17:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.723+0000 7fdbaa99c740 1 -- 172.21.15.119:0/613879381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a181c950 msgr2=0x5639a1844790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.723+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/613879381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a181c950 0x5639a1844790 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fdb94081360 tx=0x7fdb94079880 comp rx=0 tx=0).stop 2024-04-25T10:17:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.723+0000 7fdbaa99c740 1 -- 172.21.15.119:0/613879381 shutdown_connections 2024-04-25T10:17:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.723+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/613879381 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a18655c0 0x5639a181b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.723+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/613879381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1844cd0 0x5639a181cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.723+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/613879381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a181c950 0x5639a1844790 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.723+0000 7fdbaa99c740 1 -- 172.21.15.119:0/613879381 >> 172.21.15.119:0/613879381 conn(0x5639a1819b70 msgr2=0x5639a1862550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.723+0000 7fdbaa99c740 1 -- 172.21.15.119:0/613879381 shutdown_connections 2024-04-25T10:17:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.723+0000 7fdbaa99c740 1 -- 172.21.15.119:0/613879381 wait complete. 2024-04-25T10:17:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.724+0000 7fdbaa99c740 1 -- start start 2024-04-25T10:17:44.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.724+0000 7fdbaa99c740 1 Processor -- start 2024-04-25T10:17:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.725+0000 7fdbaa99c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1844cd0 0x5639a184a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.725+0000 7fdbaa99c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a18655c0 0x5639a183fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.725+0000 7fdbaa99c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1847240 0x5639a1872b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.725+0000 7fdbaa99c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a1780620 con 0x5639a18655c0 2024-04-25T10:17:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.725+0000 7fdbaa99c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a171f1f0 con 0x5639a1847240 2024-04-25T10:17:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.725+0000 7fdbaa99c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639a1844300 con 0x5639a1844cd0 2024-04-25T10:17:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.725+0000 7fdbaa99c740 10 client.0 _ll_register_callbacks cb 0x5639a173cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.725+0000 7fdb9fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a18655c0 0x5639a183fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.726+0000 7fdb9fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a18655c0 0x5639a183fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37028/0 (socket says 172.21.15.119:37028) 2024-04-25T10:17:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.726+0000 7fdb9fe00640 1 -- 172.21.15.119:0/4060715589 learned_addr learned my addr 172.21.15.119:0/4060715589 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.726+0000 7fdba5200640 1 --2- 172.21.15.119:0/4060715589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1844cd0 0x5639a184a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.726+0000 7fdba5c00640 1 --2- 172.21.15.119:0/4060715589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1847240 0x5639a1872b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.726+0000 7fdb9fe00640 1 -- 172.21.15.119:0/4060715589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1844cd0 msgr2=0x5639a184a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.726+0000 7fdb9fe00640 1 --2- 172.21.15.119:0/4060715589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1844cd0 0x5639a184a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.726+0000 7fdb9fe00640 1 -- 172.21.15.119:0/4060715589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1847240 msgr2=0x5639a1872b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.726+0000 7fdb9fe00640 1 --2- 172.21.15.119:0/4060715589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1847240 0x5639a1872b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.726+0000 7fdb9fe00640 1 -- 172.21.15.119:0/4060715589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639a173ec50 con 0x5639a18655c0 2024-04-25T10:17:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.726+0000 7fdba5c00640 1 --2- 172.21.15.119:0/4060715589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1847240 0x5639a1872b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.726+0000 7fdb9fe00640 1 --2- 172.21.15.119:0/4060715589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a18655c0 0x5639a183fde0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c016ee0 tx=0x7fdb8c013780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f2373888493e71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.726+0000 7fdb86000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:17:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.727+0000 7fdb86000640 1 -- 172.21.15.119:0/4060715589 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c024060 con 0x5639a18655c0 2024-04-25T10:17:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.727+0000 7fdb86000640 1 -- 172.21.15.119:0/4060715589 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb8c0160d0 con 0x5639a18655c0 2024-04-25T10:17:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.727+0000 7fdbaa99c740 1 -- 172.21.15.119:0/4060715589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb9407d2d0 con 0x5639a18655c0 2024-04-25T10:17:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.727+0000 7fdb86000640 1 -- 172.21.15.119:0/4060715589 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb8c02a050 con 0x5639a18655c0 2024-04-25T10:17:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.727+0000 7fdbaa99c740 10 client.8826 fetch_fsmap learned FSMap version 99 2024-04-25T10:17:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.727+0000 7fdbaa99c740 1 -- 172.21.15.119:0/4060715589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5639a14952d0 con 0x5639a18655c0 2024-04-25T10:17:44.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.727+0000 7fdb86000640 1 -- 172.21.15.119:0/4060715589 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9407d2d0 con 0x5639a18655c0 2024-04-25T10:17:44.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.727+0000 7fdb86000640 1 -- 172.21.15.119:0/4060715589 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb8c02d020 con 0x5639a18655c0 2024-04-25T10:17:44.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.727+0000 7fdbaa99c740 10 client.8826 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:17:44.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.727+0000 7fdbaa99c740 10 client.8826 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:17:44.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.727+0000 7fdbaa99c740 20 client.8826 populate_metadata read hostname 'smithi119' 2024-04-25T10:17:44.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.727+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/4060715589 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x5639a1845610 0x5639a18459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.727+0000 7fdbaa99c740 4 client.8826 mds_command: new command op to 8697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:44.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.727+0000 7fdbaa99c740 1 -- 172.21.15.119:0/4060715589 --> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639a178db60 con 0x5639a1845610 2024-04-25T10:17:44.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.728+0000 7fdba5200640 1 --2- 172.21.15.119:0/4060715589 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x5639a1845610 0x5639a18459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.728+0000 7fdba5200640 1 --2- 172.21.15.119:0/4060715589 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x5639a1845610 0x5639a18459f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c8d19bad4f5762d server_cookie=7831cace392e0eba in_seq=0 out_seq=0 2024-04-25T10:17:44.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.728+0000 7fdb86000640 10 client.8826 ms_handle_connect on v2:172.21.15.119:6836/1849584205 2024-04-25T10:17:44.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.729+0000 7fdb86000640 1 -- 172.21.15.119:0/4060715589 <== mds.0 v2:172.21.15.119:6836/1849584205 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639a178db60 con 0x5639a1845610 2024-04-25T10:17:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.729+0000 7fdb86000640 10 client.8826 handle_command_reply: tid=0 2024-04-25T10:17:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.729+0000 7fdbaa99c740 1 client.8826 shutdown 2024-04-25T10:17:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.729+0000 7fdbaa99c740 1 -- 172.21.15.119:0/4060715589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a18655c0 msgr2=0x5639a183fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.729+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/4060715589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a18655c0 0x5639a183fde0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c016ee0 tx=0x7fdb8c013780 comp rx=0 tx=0).stop 2024-04-25T10:17:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.729+0000 7fdbaa99c740 1 -- 172.21.15.119:0/4060715589 shutdown_connections 2024-04-25T10:17:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.729+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/4060715589 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x5639a1845610 0x5639a18459f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.729+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/4060715589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1847240 0x5639a1872b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.729+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/4060715589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a18655c0 0x5639a183fde0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.729+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/4060715589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1844cd0 0x5639a184a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.729+0000 7fdbaa99c740 1 -- 172.21.15.119:0/4060715589 >> 172.21.15.119:0/4060715589 conn(0x5639a181c950 msgr2=0x5639a18168b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.729+0000 7fdb86000640 0 client.8826 ms_handle_reset on v2:172.21.15.119:6836/1849584205 2024-04-25T10:17:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.729+0000 7fdbaa99c740 1 -- 172.21.15.119:0/4060715589 shutdown_connections 2024-04-25T10:17:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.729+0000 7fdbaa99c740 1 -- 172.21.15.119:0/4060715589 wait complete. 2024-04-25T10:17:44.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.730+0000 7fdbaa99c740 20 client.8826 trim_cache size 0 max 16384 2024-04-25T10:17:44.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.746+0000 7fdbaa99c740 1 Processor -- start 2024-04-25T10:17:44.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.746+0000 7fdbaa99c740 1 -- start start 2024-04-25T10:17:44.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.747+0000 7fdbaa99c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1844cd0 0x5639a1847fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.747+0000 7fdbaa99c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1847240 0x5639a1875a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.747+0000 7fdbaa99c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a18655c0 0x5639a1816aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.747+0000 7fdbaa99c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba01ff130 con 0x5639a18655c0 2024-04-25T10:17:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.747+0000 7fdbaa99c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00cdaf0 con 0x5639a1844cd0 2024-04-25T10:17:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.747+0000 7fdbaa99c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00cb570 con 0x5639a1847240 2024-04-25T10:17:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.748+0000 7fdba5c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a18655c0 0x5639a1816aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.748+0000 7fdba5c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a18655c0 0x5639a1816aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37038/0 (socket says 172.21.15.119:37038) 2024-04-25T10:17:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.748+0000 7fdba5c00640 1 -- 172.21.15.119:0/1507368504 learned_addr learned my addr 172.21.15.119:0/1507368504 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.748+0000 7fdba5200640 1 --2- 172.21.15.119:0/1507368504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1844cd0 0x5639a1847fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.748+0000 7fdb9fe00640 1 --2- 172.21.15.119:0/1507368504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1847240 0x5639a1875a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.748+0000 7fdba5c00640 1 -- 172.21.15.119:0/1507368504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1847240 msgr2=0x5639a1875a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.748+0000 7fdba5c00640 1 --2- 172.21.15.119:0/1507368504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1847240 0x5639a1875a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.748+0000 7fdba5c00640 1 -- 172.21.15.119:0/1507368504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1844cd0 msgr2=0x5639a1847fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.748+0000 7fdba5c00640 1 --2- 172.21.15.119:0/1507368504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1844cd0 0x5639a1847fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.748+0000 7fdba5c00640 1 -- 172.21.15.119:0/1507368504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639a155f6b0 con 0x5639a18655c0 2024-04-25T10:17:44.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.748+0000 7fdba5c00640 1 --2- 172.21.15.119:0/1507368504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a18655c0 0x5639a1816aa0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009f80 tx=0x7fdb980077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8408f4283f895e8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:44.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.749+0000 7fdb90c00640 1 -- 172.21.15.119:0/1507368504 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98013780 con 0x5639a18655c0 2024-04-25T10:17:44.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.749+0000 7fdb90c00640 1 -- 172.21.15.119:0/1507368504 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb98019070 con 0x5639a18655c0 2024-04-25T10:17:44.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.749+0000 7fdb90c00640 1 -- 172.21.15.119:0/1507368504 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98013b80 con 0x5639a18655c0 2024-04-25T10:17:44.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.749+0000 7fdbaa99c740 1 -- 172.21.15.119:0/1507368504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a18655c0 msgr2=0x5639a1816aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.749+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/1507368504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a18655c0 0x5639a1816aa0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009f80 tx=0x7fdb980077c0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.749+0000 7fdbaa99c740 1 -- 172.21.15.119:0/1507368504 shutdown_connections 2024-04-25T10:17:44.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.749+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/1507368504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a18655c0 0x5639a1816aa0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.749+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/1507368504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1847240 0x5639a1875a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.749+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/1507368504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a1844cd0 0x5639a1847fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.749+0000 7fdbaa99c740 1 -- 172.21.15.119:0/1507368504 >> 172.21.15.119:0/1507368504 conn(0x5639a181c950 msgr2=0x5639a1846b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:44.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.749+0000 7fdbaa99c740 1 -- 172.21.15.119:0/1507368504 shutdown_connections 2024-04-25T10:17:44.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.749+0000 7fdbaa99c740 1 -- 172.21.15.119:0/1507368504 wait complete. 2024-04-25T10:17:44.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.750+0000 7fdbaa99c740 1 -- start start 2024-04-25T10:17:44.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.750+0000 7fdbaa99c740 1 Processor -- start 2024-04-25T10:17:44.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.751+0000 7fdbaa99c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a1844cd0 0x5639a18c0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.751+0000 7fdbaa99c740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1847240 0x5639a1857ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.751+0000 7fdbaa99c740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a18655c0 0x5639a181b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.751+0000 7fdbaa99c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a1847710 con 0x5639a1844cd0 2024-04-25T10:17:44.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.751+0000 7fdbaa99c740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a1847c00 con 0x5639a18655c0 2024-04-25T10:17:44.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.751+0000 7fdbaa99c740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639a1864770 con 0x5639a1847240 2024-04-25T10:17:44.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.751+0000 7fdbaa99c740 10 client.0 _ll_register_callbacks cb 0x5639a184a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:44.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.752+0000 7fdba5200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a1844cd0 0x5639a18c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.752+0000 7fdba5200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a1844cd0 0x5639a18c0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37040/0 (socket says 172.21.15.119:37040) 2024-04-25T10:17:44.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.752+0000 7fdba5200640 1 -- 172.21.15.119:0/3377261374 learned_addr learned my addr 172.21.15.119:0/3377261374 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:44.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.752+0000 7fdb9fe00640 1 --2- 172.21.15.119:0/3377261374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1847240 0x5639a1857ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.752+0000 7fdba5200640 1 -- 172.21.15.119:0/3377261374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1847240 msgr2=0x5639a1857ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.752+0000 7fdba5c00640 1 --2- 172.21.15.119:0/3377261374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a18655c0 0x5639a181b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.752+0000 7fdba5200640 1 --2- 172.21.15.119:0/3377261374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1847240 0x5639a1857ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.752+0000 7fdba5200640 1 -- 172.21.15.119:0/3377261374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a18655c0 msgr2=0x5639a181b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.752+0000 7fdba5200640 1 --2- 172.21.15.119:0/3377261374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a18655c0 0x5639a181b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.752+0000 7fdba5200640 1 -- 172.21.15.119:0/3377261374 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639a18c4170 con 0x5639a1844cd0 2024-04-25T10:17:44.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.752+0000 7fdba5200640 1 --2- 172.21.15.119:0/3377261374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a1844cd0 0x5639a18c0150 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fdb94033fd0 tx=0x7fdb94008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5804bde5dee843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:44.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.752+0000 7fdb86000640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:17:44.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.752+0000 7fdb86000640 1 -- 172.21.15.119:0/3377261374 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9407d2d0 con 0x5639a1844cd0 2024-04-25T10:17:44.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.752+0000 7fdb86000640 1 -- 172.21.15.119:0/3377261374 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb94077480 con 0x5639a1844cd0 2024-04-25T10:17:44.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.753+0000 7fdbaa99c740 1 -- 172.21.15.119:0/3377261374 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb98019070 con 0x5639a1844cd0 2024-04-25T10:17:44.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.753+0000 7fdb86000640 1 -- 172.21.15.119:0/3377261374 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb940757f0 con 0x5639a1844cd0 2024-04-25T10:17:44.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.753+0000 7fdbaa99c740 10 client.8832 fetch_fsmap learned FSMap version 99 2024-04-25T10:17:44.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.753+0000 7fdbaa99c740 1 -- 172.21.15.119:0/3377261374 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5639a181b320 con 0x5639a1844cd0 2024-04-25T10:17:44.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.753+0000 7fdb86000640 1 -- 172.21.15.119:0/3377261374 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb98019070 con 0x5639a1844cd0 2024-04-25T10:17:44.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.753+0000 7fdb86000640 1 -- 172.21.15.119:0/3377261374 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb9403a030 con 0x5639a1844cd0 2024-04-25T10:17:44.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.753+0000 7fdbaa99c740 10 client.8832 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:17:44.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.753+0000 7fdbaa99c740 10 client.8832 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:17:44.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.753+0000 7fdbaa99c740 20 client.8832 populate_metadata read hostname 'smithi119' 2024-04-25T10:17:44.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.753+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/3377261374 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x5639a181d250 0x5639a181d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.753+0000 7fdbaa99c740 4 client.8832 mds_command: new command op to 8697 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:17:44.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.753+0000 7fdbaa99c740 1 -- 172.21.15.119:0/3377261374 --> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5639a1787eb0 con 0x5639a181d250 2024-04-25T10:17:44.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.753+0000 7fdb9fe00640 1 --2- 172.21.15.119:0/3377261374 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x5639a181d250 0x5639a181d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdb9fe00640 1 --2- 172.21.15.119:0/3377261374 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x5639a181d250 0x5639a181d630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f73aa82e7b0838ac server_cookie=4cb228c2565ca4fe in_seq=0 out_seq=0 2024-04-25T10:17:44.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdb86000640 10 client.8832 ms_handle_connect on v2:172.21.15.119:6836/1849584205 2024-04-25T10:17:44.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdb86000640 1 -- 172.21.15.119:0/3377261374 <== mds.0 v2:172.21.15.119:6836/1849584205 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5639a1787eb0 con 0x5639a181d250 2024-04-25T10:17:44.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdb86000640 10 client.8832 handle_command_reply: tid=0 2024-04-25T10:17:44.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdbaa99c740 1 client.8832 shutdown 2024-04-25T10:17:44.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdbaa99c740 1 -- 172.21.15.119:0/3377261374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a1844cd0 msgr2=0x5639a18c0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/3377261374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a1844cd0 0x5639a18c0150 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fdb94033fd0 tx=0x7fdb94008070 comp rx=0 tx=0).stop 2024-04-25T10:17:44.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdbaa99c740 1 -- 172.21.15.119:0/3377261374 shutdown_connections 2024-04-25T10:17:44.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/3377261374 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x5639a181d250 0x5639a181d630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/3377261374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639a18655c0 0x5639a181b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/3377261374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5639a1847240 0x5639a1857ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdb86000640 0 client.8832 ms_handle_reset on v2:172.21.15.119:6836/1849584205 2024-04-25T10:17:44.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdbaa99c740 1 --2- 172.21.15.119:0/3377261374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5639a1844cd0 0x5639a18c0150 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdbaa99c740 1 -- 172.21.15.119:0/3377261374 >> 172.21.15.119:0/3377261374 conn(0x5639a181c950 msgr2=0x5639a1875b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:44.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdbaa99c740 1 -- 172.21.15.119:0/3377261374 shutdown_connections 2024-04-25T10:17:44.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.754+0000 7fdbaa99c740 1 -- 172.21.15.119:0/3377261374 wait complete. 2024-04-25T10:17:44.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.755+0000 7fdbaa99c740 20 client.8832 trim_cache size 0 max 16384 2024-04-25T10:17:44.763 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:17:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "e0b0f428-b441-4237-a4d1-ed53321f87a5", 2024-04-25T10:17:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-04-25T10:17:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 8697, 2024-04-25T10:17:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-04-25T10:17:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-04-25T10:17:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-04-25T10:17:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.5777706540000001, 2024-04-25T10:17:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 99, 2024-04-25T10:17:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 108, 2024-04-25T10:17:44.763 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 108, 2024-04-25T10:17:44.764 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 12.033262948999999 2024-04-25T10:17:44.764 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:17:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.755+0000 7fdba7c00640 1 -- 172.21.15.119:0/3122053696 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb8006b500 msgr2=0x7fdb8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.755+0000 7fdba7c00640 1 --2- 172.21.15.119:0/3122053696 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb8006b500 0x7fdb8006d9c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdba00680b0 tx=0x7fdb8c0079b0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.755+0000 7fdba7c00640 1 -- 172.21.15.119:0/3122053696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba00b9720 msgr2=0x7fdba015b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.755+0000 7fdba7c00640 1 --2- 172.21.15.119:0/3122053696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba00b9720 0x7fdba015b1e0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fdb9402a1e0 tx=0x7fdb940046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.755+0000 7fdba7c00640 1 -- 172.21.15.119:0/3122053696 shutdown_connections 2024-04-25T10:17:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.755+0000 7fdba7c00640 1 --2- 172.21.15.119:0/3122053696 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdb8006b500 0x7fdb8006d9c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.755+0000 7fdba7c00640 1 --2- 172.21.15.119:0/3122053696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba0161090 0x7fdba00aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.755+0000 7fdba7c00640 1 --2- 172.21.15.119:0/3122053696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba00ceeb0 0x7fdba01f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.755+0000 7fdba7c00640 1 --2- 172.21.15.119:0/3122053696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba00b9720 0x7fdba015b1e0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.755+0000 7fdba7c00640 1 -- 172.21.15.119:0/3122053696 >> 172.21.15.119:0/3122053696 conn(0x7fdba00b55e0 msgr2=0x7fdba00b7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.755+0000 7fdba7c00640 1 -- 172.21.15.119:0/3122053696 shutdown_connections 2024-04-25T10:17:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.755+0000 7fdba7c00640 1 -- 172.21.15.119:0/3122053696 wait complete. 2024-04-25T10:17:44.775 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:17:44.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.963+0000 7fb4c8c00640 1 Processor -- start 2024-04-25T10:17:44.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.963+0000 7fb4c8c00640 1 -- start start 2024-04-25T10:17:44.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.965+0000 7fb4c8c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4c415a870 0x7fb4c40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.965+0000 7fb4c8c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c40cdb10 con 0x7fb4c406e2e0 2024-04-25T10:17:44.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.965+0000 7fb4c8c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c40cddf0 con 0x7fb4c40cfc40 2024-04-25T10:17:44.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.965+0000 7fb4c8c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40ce720 con 0x7fb4c415a870 2024-04-25T10:17:44.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.965+0000 7fb4bbe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb4c406e2e0 0x7fb4c40cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56876/0 (socket says 172.21.15.119:56876) 2024-04-25T10:17:44.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.965+0000 7fb4bbe00640 1 -- 172.21.15.119:0/631802495 learned_addr learned my addr 172.21.15.119:0/631802495 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:44.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.965+0000 7fb4c1e00640 1 --2- 172.21.15.119:0/631802495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4c415a870 0x7fb4c40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 -- 172.21.15.119:0/631802495 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758205886 0 0) 0x7fb4c40cdb10 con 0x7fb4c406e2e0 2024-04-25T10:17:44.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 -- 172.21.15.119:0/631802495 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4a8003520 con 0x7fb4c406e2e0 2024-04-25T10:17:44.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 -- 172.21.15.119:0/631802495 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1840345323 0 0) 0x7fb4a8003520 con 0x7fb4c406e2e0 2024-04-25T10:17:44.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 -- 172.21.15.119:0/631802495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4c415a870 msgr2=0x7fb4c40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 --2- 172.21.15.119:0/631802495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4c415a870 0x7fb4c40cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 -- 172.21.15.119:0/631802495 >> v1:172.21.15.132:6789/0 conn(0x7fb4c40cfc40 legacy=0x7fb4c415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 -- 172.21.15.119:0/631802495 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c40ce0d0 con 0x7fb4c406e2e0 2024-04-25T10:17:44.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 -- 172.21.15.119:0/631802495 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fb4ac001670 con 0x7fb4c406e2e0 2024-04-25T10:17:44.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 -- 172.21.15.119:0/631802495 >> v1:172.21.15.119:6789/0 conn(0x7fb4c406e2e0 legacy=0x7fb4c40cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 --2- 172.21.15.119:0/631802495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4a80039e0 0x7fb4a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 --2- 172.21.15.119:0/631802495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a8004800 0x7fb4a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 --2- 172.21.15.119:0/631802495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4c415a870 0x7fb4a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 -- 172.21.15.119:0/631802495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40ce720 con 0x7fb4a8004800 2024-04-25T10:17:44.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 -- 172.21.15.119:0/631802495 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a800ba70 con 0x7fb4a80039e0 2024-04-25T10:17:44.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.966+0000 7fb4c1400640 1 -- 172.21.15.119:0/631802495 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a800bc70 con 0x7fb4c415a870 2024-04-25T10:17:44.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.967+0000 7fb4bbe00640 1 --2- 172.21.15.119:0/631802495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a8004800 0x7fb4a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.967+0000 7fb4c1e00640 1 --2- 172.21.15.119:0/631802495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4a80039e0 0x7fb4a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.967+0000 7fb4bbe00640 1 -- 172.21.15.119:0/631802495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4c415a870 msgr2=0x7fb4a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.967+0000 7fb4bbe00640 1 --2- 172.21.15.119:0/631802495 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4c415a870 0x7fb4a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.967+0000 7fb4bbe00640 1 -- 172.21.15.119:0/631802495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4a80039e0 msgr2=0x7fb4a8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.967+0000 7fb4bbe00640 1 --2- 172.21.15.119:0/631802495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4a80039e0 0x7fb4a8003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.967+0000 7fb4bbe00640 1 -- 172.21.15.119:0/631802495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a8003520 con 0x7fb4a8004800 2024-04-25T10:17:44.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.968+0000 7fb4bbe00640 1 --2- 172.21.15.119:0/631802495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a8004800 0x7fb4a8004c80 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00c210 tx=0x7fb4a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b88ad0966104c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:44.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.968+0000 7fb4c1400640 1 -- 172.21.15.119:0/631802495 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac007cf0 con 0x7fb4a8004800 2024-04-25T10:17:44.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.968+0000 7fb4c1400640 1 -- 172.21.15.119:0/631802495 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac005ad0 con 0x7fb4a8004800 2024-04-25T10:17:44.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.968+0000 7fb4c1400640 1 -- 172.21.15.119:0/631802495 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac007070 con 0x7fb4a8004800 2024-04-25T10:17:44.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.969+0000 7fb4c8c00640 1 -- 172.21.15.119:0/631802495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a8004800 msgr2=0x7fb4a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.969+0000 7fb4c8c00640 1 --2- 172.21.15.119:0/631802495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a8004800 0x7fb4a8004c80 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00c210 tx=0x7fb4a8005830 comp rx=0 tx=0).stop 2024-04-25T10:17:44.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.969+0000 7fb4c2800640 1 -- 172.21.15.119:0/631802495 reap_dead start 2024-04-25T10:17:44.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.969+0000 7fb4c8c00640 1 -- 172.21.15.119:0/631802495 shutdown_connections 2024-04-25T10:17:44.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.969+0000 7fb4c8c00640 1 -- 172.21.15.119:0/631802495 >> 172.21.15.119:0/631802495 conn(0x7fb4c406aba0 msgr2=0x7fb4c406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:44.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.969+0000 7fb4c8c00640 1 -- 172.21.15.119:0/631802495 shutdown_connections 2024-04-25T10:17:44.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.969+0000 7fb4c8c00640 1 -- 172.21.15.119:0/631802495 wait complete. 2024-04-25T10:17:44.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.971+0000 7fb4c8c00640 1 Processor -- start 2024-04-25T10:17:44.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.971+0000 7fb4c8c00640 1 -- start start 2024-04-25T10:17:44.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4c8c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4a8004800 0x7fb4c41449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4c8c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c4145060 0x7fb4c4147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4c8c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4c41466a0 0x7fb4c415b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4c8c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40cd700 con 0x7fb4c4145060 2024-04-25T10:17:44.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4c8c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40cd580 con 0x7fb4c41466a0 2024-04-25T10:17:44.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4c8c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40cd880 con 0x7fb4a8004800 2024-04-25T10:17:44.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4bbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c4145060 0x7fb4c4147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4bbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c4145060 0x7fb4c4147ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37060/0 (socket says 172.21.15.119:37060) 2024-04-25T10:17:44.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4bbe00640 1 -- 172.21.15.119:0/43109148 learned_addr learned my addr 172.21.15.119:0/43109148 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:44.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4c1e00640 1 --2- 172.21.15.119:0/43109148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4a8004800 0x7fb4c41449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4bbe00640 1 -- 172.21.15.119:0/43109148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4a8004800 msgr2=0x7fb4c41449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4bbe00640 1 --2- 172.21.15.119:0/43109148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4a8004800 0x7fb4c41449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4bbe00640 1 -- 172.21.15.119:0/43109148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4c41466a0 msgr2=0x7fb4c415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4bbe00640 1 --2- 172.21.15.119:0/43109148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4c41466a0 0x7fb4c415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.972+0000 7fb4bbe00640 1 -- 172.21.15.119:0/43109148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c4147ff0 con 0x7fb4c4145060 2024-04-25T10:17:44.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.973+0000 7fb4bbe00640 1 --2- 172.21.15.119:0/43109148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c4145060 0x7fb4c4147ab0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00c330 tx=0x7fb4c40cdce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:44.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.973+0000 7fb4ba000640 1 -- 172.21.15.119:0/43109148 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac041020 con 0x7fb4c4145060 2024-04-25T10:17:44.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.973+0000 7fb4ba000640 1 -- 172.21.15.119:0/43109148 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac001430 con 0x7fb4c4145060 2024-04-25T10:17:44.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.973+0000 7fb4ba000640 1 -- 172.21.15.119:0/43109148 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac007070 con 0x7fb4c4145060 2024-04-25T10:17:44.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.973+0000 7fb4c8c00640 1 -- 172.21.15.119:0/43109148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c4145600 con 0x7fb4c4145060 2024-04-25T10:17:44.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.973+0000 7fb4c8c00640 1 -- 172.21.15.119:0/43109148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c4145bc0 con 0x7fb4c4145060 2024-04-25T10:17:44.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.974+0000 7fb4c8c00640 1 -- 172.21.15.119:0/43109148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c40749b0 con 0x7fb4c4145060 2024-04-25T10:17:44.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.975+0000 7fb4ba000640 1 -- 172.21.15.119:0/43109148 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4ac044020 con 0x7fb4c4145060 2024-04-25T10:17:44.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.976+0000 7fb4ba000640 1 --2- 172.21.15.119:0/43109148 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb4a006b4d0 0x7fb4a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.976+0000 7fb4ba000640 1 -- 172.21.15.119:0/43109148 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4ac0aa5c0 con 0x7fb4c4145060 2024-04-25T10:17:44.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.976+0000 7fb4c1e00640 1 --2- 172.21.15.119:0/43109148 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb4a006b4d0 0x7fb4a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.977+0000 7fb4c1e00640 1 --2- 172.21.15.119:0/43109148 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb4a006b4d0 0x7fb4a006d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40b1000 tx=0x7fb4b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:44.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:44.980+0000 7fb4ba000640 1 -- 172.21.15.119:0/43109148 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4ac005aa0 con 0x7fb4c4145060 2024-04-25T10:17:45.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:45.202+0000 7fb4c8c00640 1 -- 172.21.15.119:0/43109148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb4c406b330 con 0x7fb4c4145060 2024-04-25T10:17:45.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:45.203+0000 7fb4ba000640 1 -- 172.21.15.119:0/43109148 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb4ac079440 con 0x7fb4c4145060 2024-04-25T10:17:45.206 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:17:45.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:45.206+0000 7fb4c8c00640 1 -- 172.21.15.119:0/43109148 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb4a006b4d0 msgr2=0x7fb4a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:45.206+0000 7fb4c8c00640 1 --2- 172.21.15.119:0/43109148 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb4a006b4d0 0x7fb4a006d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40b1000 tx=0x7fb4b4008040 comp rx=0 tx=0).stop 2024-04-25T10:17:45.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:45.206+0000 7fb4c8c00640 1 -- 172.21.15.119:0/43109148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c4145060 msgr2=0x7fb4c4147ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:45.206+0000 7fb4c8c00640 1 --2- 172.21.15.119:0/43109148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c4145060 0x7fb4c4147ab0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00c330 tx=0x7fb4c40cdce0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:45.206+0000 7fb4c8c00640 1 -- 172.21.15.119:0/43109148 shutdown_connections 2024-04-25T10:17:45.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:45.206+0000 7fb4c8c00640 1 --2- 172.21.15.119:0/43109148 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb4a006b4d0 0x7fb4a006d990 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:45.206+0000 7fb4c8c00640 1 --2- 172.21.15.119:0/43109148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4c41466a0 0x7fb4c415b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:45.206+0000 7fb4c8c00640 1 --2- 172.21.15.119:0/43109148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c4145060 0x7fb4c4147ab0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:45.206+0000 7fb4c8c00640 1 --2- 172.21.15.119:0/43109148 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb4a8004800 0x7fb4c41449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:45.206+0000 7fb4c8c00640 1 -- 172.21.15.119:0/43109148 >> 172.21.15.119:0/43109148 conn(0x7fb4c406aba0 msgr2=0x7fb4c4071990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:45.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:45.206+0000 7fb4c8c00640 1 -- 172.21.15.119:0/43109148 shutdown_connections 2024-04-25T10:17:45.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:45.206+0000 7fb4c8c00640 1 -- 172.21.15.119:0/43109148 wait complete. 2024-04-25T10:17:45.228 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:17:45.228 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:17:45.228 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:17:45.228 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:17:45.228 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:45.228 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:17:45.228 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:17:45.228 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:17:45.228 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:17:45.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:45.229 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:17:45.277 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:17:45.277 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:17:45.277 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:17:45.277 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:45.277 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:17:45.277 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:45.277 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:17:45.278 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:45.278 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:17:45.278 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:17:45.278 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:45.278 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:17:45.278 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:17:45.278 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:45.278 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:17:45.278 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:45.278 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:17:45.278 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42024sec preferred_lft 42024sec 2024-04-25T10:17:45.278 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:17:45.278 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:17:45.280 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:45.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:45.281 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:17:45.394 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:17:45.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:45.398 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:17:45.450 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:17:45.451 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:17:45.451 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:17:45.451 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:17:45.451 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:17:45.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:45.452 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:17:45.452 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:17:45.452 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:17:45.452 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:17:45.452 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:17:45.452 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:17:45.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:45.597 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:17:45.647 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:17:45.647 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:17:45.648 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:17:45.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:45.648 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:17:45.731 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:17:45.731 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:45.785 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:17:45.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:17:45.786 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:45.836 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:17:45.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:45.836 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:17:45.963 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:17:45.964 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:17:45.964 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:17:45.985 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:17:45.985 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:17:45.985 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:17:45.985 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:17:45.985 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:17:46.011 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:17:46.011 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:17:46.011 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:17:46.011 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:17:46.011 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:17:46.011 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:17:46.011 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:17:46.011 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:17:46.014 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:17:46.014 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:17:46.014 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:17:46.014 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:17:46.014 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:17:46.018 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:17:46.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:46.018 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:46.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:46.092 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:17:46.163 INFO:teuthology.orchestra.run.smithi119.stdout:client6310 2024-04-25T10:17:46.163 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:17:46.163 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:46.215 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:17:46.216 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:17:46.216 DEBUG:teuthology.orchestra.run.smithi119:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:17:46.216 DEBUG:teuthology.orchestra.run.smithi119:> parent = os.path.join(mount_point, "parent") 2024-04-25T10:17:46.216 DEBUG:teuthology.orchestra.run.smithi119:> os.mkdir(parent) 2024-04-25T10:17:46.216 DEBUG:teuthology.orchestra.run.smithi119:> flushed = os.path.join(parent, "flushed") 2024-04-25T10:17:46.216 DEBUG:teuthology.orchestra.run.smithi119:> os.mkdir(flushed) 2024-04-25T10:17:46.216 DEBUG:teuthology.orchestra.run.smithi119:> for f in ["alpha", "bravo", "charlie"]: 2024-04-25T10:17:46.216 DEBUG:teuthology.orchestra.run.smithi119:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-25T10:17:46.216 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:17:46.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:17:46.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:46.435 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:17:46.538 INFO:teuthology.orchestra.run.smithi119.stdout:./ 2024-04-25T10:17:46.538 INFO:teuthology.orchestra.run.smithi119.stdout:./parent 2024-04-25T10:17:46.538 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/flushed 2024-04-25T10:17:46.538 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/flushed/charlie 2024-04-25T10:17:46.538 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/flushed/alpha 2024-04-25T10:17:46.538 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/flushed/bravo 2024-04-25T10:17:46.539 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:17:46.540 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:17:46.540 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:17:46.540 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:17:46.540 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:17:46.540 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:17:46.677 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:17:46.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:17:46.690 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:17:46.691 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:17:46.691 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:17:46.691 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:17:46.691 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:17:46.691 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:17:46.875 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627776 2024-04-25T10:17:46.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:17:46.889 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:17:46.889 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:17:46.889 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:17:46.889 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:17:46.889 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:17:46.889 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:17:47.073 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627777 2024-04-25T10:17:47.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:17:47.088 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:17:47.088 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:17:47.088 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:17:47.088 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:17:47.089 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:17:47.089 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:17:47.271 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627780 2024-04-25T10:17:47.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:17:47.285 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:17:47.285 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:17:47.285 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:17:47.286 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:17:47.286 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:17:47.286 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:17:47.473 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627778 2024-04-25T10:17:47.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:17:47.487 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:17:47.487 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:17:47.487 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:17:47.487 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:17:47.487 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T10:17:47.487 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:17:47.671 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627779 2024-04-25T10:17:47.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:17:47.685 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:17:47.685 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:47.782 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:17:47.782 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:47.833 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:48.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.030+0000 7f0b32c00640 1 Processor -- start 2024-04-25T10:17:48.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.030+0000 7f0b32c00640 1 -- start start 2024-04-25T10:17:48.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.031+0000 7f0b32c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0b2c15bf00 0x7f0b2c0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.031+0000 7f0b32c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2c0ce6c0 con 0x7f0b2c0cc310 2024-04-25T10:17:48.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.031+0000 7f0b32c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2c0cc7b0 con 0x7f0b2c06e8f0 2024-04-25T10:17:48.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.031+0000 7f0b32c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b2c1455e0 con 0x7f0b2c15bf00 2024-04-25T10:17:48.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.031+0000 7f0b2be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0b2c15bf00 0x7f0b2c0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.031+0000 7f0b2be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0b2c15bf00 0x7f0b2c0cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37068/0 (socket says 172.21.15.119:37068) 2024-04-25T10:17:48.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.031+0000 7f0b2be00640 1 -- 172.21.15.119:0/1131817022 learned_addr learned my addr 172.21.15.119:0/1131817022 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:48.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.032+0000 7f0b2be00640 1 -- 172.21.15.119:0/1131817022 >> v1:172.21.15.132:6789/0 conn(0x7f0b2c06e8f0 legacy=0x7f0b2c1654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:48.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.032+0000 7f0b2be00640 1 -- 172.21.15.119:0/1131817022 >> v1:172.21.15.119:6789/0 conn(0x7f0b2c0cc310 legacy=0x7f0b2c15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.032+0000 7f0b2be00640 1 -- 172.21.15.119:0/1131817022 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c0cca90 con 0x7f0b2c15bf00 2024-04-25T10:17:48.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.033+0000 7f0b2be00640 1 --2- 172.21.15.119:0/1131817022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b2c15bf00 0x7f0b2c0cbdd0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009870 tx=0x7f0b1c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3e431570551d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.033+0000 7f0b2aa00640 1 -- 172.21.15.119:0/1131817022 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c037070 con 0x7f0b2c15bf00 2024-04-25T10:17:48.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.033+0000 7f0b2aa00640 1 -- 172.21.15.119:0/1131817022 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c032440 con 0x7f0b2c15bf00 2024-04-25T10:17:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.033+0000 7f0b2aa00640 1 -- 172.21.15.119:0/1131817022 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c032810 con 0x7f0b2c15bf00 2024-04-25T10:17:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.034+0000 7f0b32c00640 1 -- 172.21.15.119:0/1131817022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b2c15bf00 msgr2=0x7f0b2c0cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.034+0000 7f0b32c00640 1 --2- 172.21.15.119:0/1131817022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b2c15bf00 0x7f0b2c0cbdd0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009870 tx=0x7f0b1c02a2a0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.034+0000 7f0b32c00640 1 -- 172.21.15.119:0/1131817022 shutdown_connections 2024-04-25T10:17:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.034+0000 7f0b32c00640 1 --2- 172.21.15.119:0/1131817022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b2c15bf00 0x7f0b2c0cbdd0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.034+0000 7f0b32c00640 1 -- 172.21.15.119:0/1131817022 >> 172.21.15.119:0/1131817022 conn(0x7f0b2c06aba0 msgr2=0x7f0b2c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.034+0000 7f0b32c00640 1 -- 172.21.15.119:0/1131817022 shutdown_connections 2024-04-25T10:17:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.034+0000 7f0b32c00640 1 -- 172.21.15.119:0/1131817022 wait complete. 2024-04-25T10:17:48.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.036+0000 7f0b32c00640 1 Processor -- start 2024-04-25T10:17:48.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.036+0000 7f0b32c00640 1 -- start start 2024-04-25T10:17:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.037+0000 7f0b32c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2c06e8f0 0x7f0b2c0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.037+0000 7f0b32c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b2c0cc310 0x7f0b2c0b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.037+0000 7f0b32c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b2c15bf00 0x7f0b2c15a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.037+0000 7f0b32c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c0cd050 con 0x7f0b2c0cc310 2024-04-25T10:17:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.037+0000 7f0b32c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c06d450 con 0x7f0b2c06e8f0 2024-04-25T10:17:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.037+0000 7f0b32c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c06ee60 con 0x7f0b2c15bf00 2024-04-25T10:17:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.037+0000 7f0b2b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b2c0cc310 0x7f0b2c0b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.037+0000 7f0b2b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b2c0cc310 0x7f0b2c0b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37074/0 (socket says 172.21.15.119:37074) 2024-04-25T10:17:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.037+0000 7f0b2b400640 1 -- 172.21.15.119:0/1306243509 learned_addr learned my addr 172.21.15.119:0/1306243509 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.037+0000 7f0b2b400640 1 -- 172.21.15.119:0/1306243509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b2c15bf00 msgr2=0x7f0b2c15a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.037+0000 7f0b2b400640 1 --2- 172.21.15.119:0/1306243509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b2c15bf00 0x7f0b2c15a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.037+0000 7f0b2b400640 1 -- 172.21.15.119:0/1306243509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2c06e8f0 msgr2=0x7f0b2c0cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.037+0000 7f0b2b400640 1 --2- 172.21.15.119:0/1306243509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2c06e8f0 0x7f0b2c0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.037+0000 7f0b2b400640 1 -- 172.21.15.119:0/1306243509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c1f3b80 con 0x7f0b2c0cc310 2024-04-25T10:17:48.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.038+0000 7f0b2b400640 1 --2- 172.21.15.119:0/1306243509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b2c0cc310 0x7f0b2c0b18b0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800c7a0 tx=0x7f0b1800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.038+0000 7f0b23e00640 1 -- 172.21.15.119:0/1306243509 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18016020 con 0x7f0b2c0cc310 2024-04-25T10:17:48.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.038+0000 7f0b23e00640 1 -- 172.21.15.119:0/1306243509 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b18006d50 con 0x7f0b2c0cc310 2024-04-25T10:17:48.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.038+0000 7f0b23e00640 1 -- 172.21.15.119:0/1306243509 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18003bb0 con 0x7f0b2c0cc310 2024-04-25T10:17:48.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.038+0000 7f0b32c00640 1 -- 172.21.15.119:0/1306243509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b2c1f3e60 con 0x7f0b2c0cc310 2024-04-25T10:17:48.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.038+0000 7f0b32c00640 1 -- 172.21.15.119:0/1306243509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2c0cb090 con 0x7f0b2c0cc310 2024-04-25T10:17:48.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.039+0000 7f0b32c00640 1 -- 172.21.15.119:0/1306243509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b2c06bb30 con 0x7f0b2c0cc310 2024-04-25T10:17:48.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.040+0000 7f0b23e00640 1 -- 172.21.15.119:0/1306243509 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b1800a850 con 0x7f0b2c0cc310 2024-04-25T10:17:48.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.041+0000 7f0b23e00640 1 --2- 172.21.15.119:0/1306243509 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b0c06b4d0 0x7f0b0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.041+0000 7f0b23e00640 1 -- 172.21.15.119:0/1306243509 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b18089ef0 con 0x7f0b2c0cc310 2024-04-25T10:17:48.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.041+0000 7f0b2be00640 1 --2- 172.21.15.119:0/1306243509 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b0c06b4d0 0x7f0b0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.042+0000 7f0b2be00640 1 --2- 172.21.15.119:0/1306243509 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b0c06b4d0 0x7f0b0c06d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c034000 tx=0x7f0b1c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.046+0000 7f0b23e00640 1 -- 172.21.15.119:0/1306243509 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b18020020 con 0x7f0b2c0cc310 2024-04-25T10:17:48.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.247+0000 7f0b32c00640 1 -- 172.21.15.119:0/1306243509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b2c0749b0 con 0x7f0b2c0cc310 2024-04-25T10:17:48.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.249+0000 7f0b23e00640 1 -- 172.21.15.119:0/1306243509 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16663 (secure 0 0 0) 0x7f0b2c0ce6c0 con 0x7f0b2c0cc310 2024-04-25T10:17:48.253 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:48.253 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":108,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:17:39.194687+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:35.422922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:36.634584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:48.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.252+0000 7f0b32c00640 1 -- 172.21.15.119:0/1306243509 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b0c06b4d0 msgr2=0x7f0b0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.252+0000 7f0b32c00640 1 --2- 172.21.15.119:0/1306243509 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b0c06b4d0 0x7f0b0c06d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c034000 tx=0x7f0b1c02c000 comp rx=0 tx=0).stop 2024-04-25T10:17:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.252+0000 7f0b32c00640 1 -- 172.21.15.119:0/1306243509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b2c0cc310 msgr2=0x7f0b2c0b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.252+0000 7f0b32c00640 1 --2- 172.21.15.119:0/1306243509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b2c0cc310 0x7f0b2c0b18b0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800c7a0 tx=0x7f0b1800cc60 comp rx=0 tx=0).stop 2024-04-25T10:17:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.252+0000 7f0b32c00640 1 -- 172.21.15.119:0/1306243509 shutdown_connections 2024-04-25T10:17:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.252+0000 7f0b32c00640 1 --2- 172.21.15.119:0/1306243509 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0b0c06b4d0 0x7f0b0c06d990 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.252+0000 7f0b32c00640 1 --2- 172.21.15.119:0/1306243509 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b2c15bf00 0x7f0b2c15a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.252+0000 7f0b32c00640 1 --2- 172.21.15.119:0/1306243509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b2c0cc310 0x7f0b2c0b18b0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.252+0000 7f0b32c00640 1 --2- 172.21.15.119:0/1306243509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b2c06e8f0 0x7f0b2c0cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.252+0000 7f0b32c00640 1 -- 172.21.15.119:0/1306243509 >> 172.21.15.119:0/1306243509 conn(0x7f0b2c06aba0 msgr2=0x7f0b2c0cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.252+0000 7f0b32c00640 1 -- 172.21.15.119:0/1306243509 shutdown_connections 2024-04-25T10:17:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.252+0000 7f0b32c00640 1 -- 172.21.15.119:0/1306243509 wait complete. 2024-04-25T10:17:48.274 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:35.422922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:17:36.634584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:48.274 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:48.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.472+0000 7f7291c00640 1 Processor -- start 2024-04-25T10:17:48.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.472+0000 7f7291c00640 1 -- start start 2024-04-25T10:17:48.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.474+0000 7f7291c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f728c0d3b90 0x7f728c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.474+0000 7f7291c00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f728c0d0f60 0x7f728c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.474+0000 7f7291c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728c0d4090 con 0x7f728c15dca0 2024-04-25T10:17:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.474+0000 7f7291c00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728c0b6b20 con 0x7f728c0d0f60 2024-04-25T10:17:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.474+0000 7f7291c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728c0d1390 con 0x7f728c0d3b90 2024-04-25T10:17:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.474+0000 7f728aa00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f728c15dca0 0x7f728c0ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56898/0 (socket says 172.21.15.119:56898) 2024-04-25T10:17:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.474+0000 7f728aa00640 1 -- 172.21.15.119:0/1870584940 learned_addr learned my addr 172.21.15.119:0/1870584940 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.474+0000 7f728b400640 1 --2- 172.21.15.119:0/1870584940 >> v2:172.21.15.119:3300/0 conn(0x7f728c0d0f60 0x7f728c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.475+0000 7f728a000640 1 --2- 172.21.15.119:0/1870584940 >> v2:172.21.15.132:3300/0 conn(0x7f728c0d3b90 0x7f728c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.475+0000 7f7289600640 1 -- 172.21.15.119:0/1870584940 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862758386 0 0) 0x7f728c0d4090 con 0x7f728c15dca0 2024-04-25T10:17:48.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.475+0000 7f7289600640 1 -- 172.21.15.119:0/1870584940 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7270003520 con 0x7f728c15dca0 2024-04-25T10:17:48.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.475+0000 7f728b400640 1 -- 172.21.15.119:0/1870584940 >> v2:172.21.15.132:3300/0 conn(0x7f728c0d3b90 msgr2=0x7f728c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:48.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.475+0000 7f728b400640 1 --2- 172.21.15.119:0/1870584940 >> v2:172.21.15.132:3300/0 conn(0x7f728c0d3b90 0x7f728c0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.475+0000 7f728b400640 1 -- 172.21.15.119:0/1870584940 >> v1:172.21.15.119:6789/0 conn(0x7f728c15dca0 legacy=0x7f728c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.475+0000 7f728b400640 1 -- 172.21.15.119:0/1870584940 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c0d4370 con 0x7f728c0d0f60 2024-04-25T10:17:48.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.476+0000 7f728b400640 1 --2- 172.21.15.119:0/1870584940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f728c0d0f60 0x7f728c0b2500 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f728c0b6560 tx=0x7f728402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba4dd4e1ae9f1dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.476+0000 7f7289600640 1 -- 172.21.15.119:0/1870584940 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284007740 con 0x7f728c0d0f60 2024-04-25T10:17:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.476+0000 7f7289600640 1 -- 172.21.15.119:0/1870584940 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72840078e0 con 0x7f728c0d0f60 2024-04-25T10:17:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.476+0000 7f7289600640 1 -- 172.21.15.119:0/1870584940 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284007cb0 con 0x7f728c0d0f60 2024-04-25T10:17:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.477+0000 7f7291c00640 1 -- 172.21.15.119:0/1870584940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f728c0d0f60 msgr2=0x7f728c0b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.477+0000 7f7291c00640 1 --2- 172.21.15.119:0/1870584940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f728c0d0f60 0x7f728c0b2500 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f728c0b6560 tx=0x7f728402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.477+0000 7f7291c00640 1 -- 172.21.15.119:0/1870584940 shutdown_connections 2024-04-25T10:17:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.477+0000 7f7291c00640 1 --2- 172.21.15.119:0/1870584940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f728c0d0f60 0x7f728c0b2500 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.477+0000 7f7291c00640 1 --2- 172.21.15.119:0/1870584940 >> v2:172.21.15.132:3300/0 conn(0x7f728c0d3b90 0x7f728c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.477+0000 7f7291c00640 1 -- 172.21.15.119:0/1870584940 >> 172.21.15.119:0/1870584940 conn(0x7f728c0726c0 msgr2=0x7f728c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.477+0000 7f7291c00640 1 -- 172.21.15.119:0/1870584940 shutdown_connections 2024-04-25T10:17:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.477+0000 7f7291c00640 1 -- 172.21.15.119:0/1870584940 wait complete. 2024-04-25T10:17:48.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.479+0000 7f7291c00640 1 Processor -- start 2024-04-25T10:17:48.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.479+0000 7f7291c00640 1 -- start start 2024-04-25T10:17:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.480+0000 7f7291c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f728c0d0f60 0x7f728c15ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.480+0000 7f7291c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f728c0d3b90 0x7f728c1f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.480+0000 7f7291c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f728c15dca0 0x7f728c1f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.480+0000 7f7291c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c0cf050 con 0x7f728c0d3b90 2024-04-25T10:17:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.480+0000 7f7291c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c0b6d50 con 0x7f728c0d0f60 2024-04-25T10:17:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.480+0000 7f7291c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728c15e080 con 0x7f728c15dca0 2024-04-25T10:17:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.480+0000 7f728a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f728c0d3b90 0x7f728c1f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.480+0000 7f728a000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f728c0d3b90 0x7f728c1f6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37088/0 (socket says 172.21.15.119:37088) 2024-04-25T10:17:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.480+0000 7f728a000640 1 -- 172.21.15.119:0/4182862348 learned_addr learned my addr 172.21.15.119:0/4182862348 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.481+0000 7f728a000640 1 -- 172.21.15.119:0/4182862348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f728c15dca0 msgr2=0x7f728c1f2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.481+0000 7f728a000640 1 --2- 172.21.15.119:0/4182862348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f728c15dca0 0x7f728c1f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.481+0000 7f728aa00640 1 --2- 172.21.15.119:0/4182862348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f728c0d0f60 0x7f728c15ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.481+0000 7f728a000640 1 -- 172.21.15.119:0/4182862348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f728c0d0f60 msgr2=0x7f728c15ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.481+0000 7f728a000640 1 --2- 172.21.15.119:0/4182862348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f728c0d0f60 0x7f728c15ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.481+0000 7f728a000640 1 -- 172.21.15.119:0/4182862348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c1f7340 con 0x7f728c0d3b90 2024-04-25T10:17:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.481+0000 7f728aa00640 1 --2- 172.21.15.119:0/4182862348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f728c0d0f60 0x7f728c15ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.481+0000 7f728a000640 1 --2- 172.21.15.119:0/4182862348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f728c0d3b90 0x7f728c1f6bd0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f727800d7d0 tx=0x7f727800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.481+0000 7f7282a00640 1 -- 172.21.15.119:0/4182862348 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800b6c0 con 0x7f728c0d3b90 2024-04-25T10:17:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.481+0000 7f7282a00640 1 -- 172.21.15.119:0/4182862348 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7278019070 con 0x7f728c0d3b90 2024-04-25T10:17:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.481+0000 7f7282a00640 1 -- 172.21.15.119:0/4182862348 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800ba80 con 0x7f728c0d3b90 2024-04-25T10:17:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.481+0000 7f7291c00640 1 -- 172.21.15.119:0/4182862348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728c1f7620 con 0x7f728c0d3b90 2024-04-25T10:17:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.482+0000 7f7291c00640 1 -- 172.21.15.119:0/4182862348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728c1f32d0 con 0x7f728c0d3b90 2024-04-25T10:17:48.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.483+0000 7f7291c00640 1 -- 172.21.15.119:0/4182862348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728c0750b0 con 0x7f728c0d3b90 2024-04-25T10:17:48.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.484+0000 7f7282a00640 1 -- 172.21.15.119:0/4182862348 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f727800bc20 con 0x7f728c0d3b90 2024-04-25T10:17:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.484+0000 7f7282a00640 1 --2- 172.21.15.119:0/4182862348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f726c06b4d0 0x7f726c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.485+0000 7f7282a00640 1 -- 172.21.15.119:0/4182862348 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f727808a670 con 0x7f728c0d3b90 2024-04-25T10:17:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.485+0000 7f728aa00640 1 --2- 172.21.15.119:0/4182862348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f726c06b4d0 0x7f726c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.485+0000 7f728aa00640 1 --2- 172.21.15.119:0/4182862348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f726c06b4d0 0x7f726c06d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f727c005880 tx=0x7f727c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.489+0000 7f7282a00640 1 -- 172.21.15.119:0/4182862348 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f727801c360 con 0x7f728c0d3b90 2024-04-25T10:17:48.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.682+0000 7f7291c00640 1 -- 172.21.15.119:0/4182862348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f728c0d1e30 con 0x7f728c0d3b90 2024-04-25T10:17:48.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.683+0000 7f7282a00640 1 -- 172.21.15.119:0/4182862348 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72780594e0 con 0x7f728c0d3b90 2024-04-25T10:17:48.686 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:17:48.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.685+0000 7f7291c00640 1 -- 172.21.15.119:0/4182862348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f726c06b4d0 msgr2=0x7f726c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.685+0000 7f7291c00640 1 --2- 172.21.15.119:0/4182862348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f726c06b4d0 0x7f726c06d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f727c005880 tx=0x7f727c009040 comp rx=0 tx=0).stop 2024-04-25T10:17:48.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.685+0000 7f7291c00640 1 -- 172.21.15.119:0/4182862348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f728c0d3b90 msgr2=0x7f728c1f6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.685+0000 7f7291c00640 1 --2- 172.21.15.119:0/4182862348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f728c0d3b90 0x7f728c1f6bd0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f727800d7d0 tx=0x7f727800dc90 comp rx=0 tx=0).stop 2024-04-25T10:17:48.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.685+0000 7f7291c00640 1 -- 172.21.15.119:0/4182862348 shutdown_connections 2024-04-25T10:17:48.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.685+0000 7f7291c00640 1 --2- 172.21.15.119:0/4182862348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f726c06b4d0 0x7f726c06d990 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.685+0000 7f7291c00640 1 --2- 172.21.15.119:0/4182862348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f728c15dca0 0x7f728c1f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.685+0000 7f7291c00640 1 --2- 172.21.15.119:0/4182862348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f728c0d3b90 0x7f728c1f6bd0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.685+0000 7f7291c00640 1 --2- 172.21.15.119:0/4182862348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f728c0d0f60 0x7f728c15ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.685+0000 7f7291c00640 1 -- 172.21.15.119:0/4182862348 >> 172.21.15.119:0/4182862348 conn(0x7f728c0726c0 msgr2=0x7f728c0b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.685+0000 7f7291c00640 1 -- 172.21.15.119:0/4182862348 shutdown_connections 2024-04-25T10:17:48.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.685+0000 7f7291c00640 1 -- 172.21.15.119:0/4182862348 wait complete. 2024-04-25T10:17:48.708 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:17:48.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.905+0000 7f4322a00640 1 Processor -- start 2024-04-25T10:17:48.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.905+0000 7f4322a00640 1 -- start start 2024-04-25T10:17:48.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.906+0000 7f4322a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f431c0b42c0 0x7f431c15ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.907+0000 7f4322a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f431c0b46a0 0x7f431c1549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.907+0000 7f4322a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431c15b370 con 0x7f431c15aef0 2024-04-25T10:17:48.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.907+0000 7f4322a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431c0d05d0 con 0x7f431c0b42c0 2024-04-25T10:17:48.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.907+0000 7f4322a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c15b650 con 0x7f431c0b46a0 2024-04-25T10:17:48.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.907+0000 7f431b400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f431c15aef0 0x7f431c0cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56900/0 (socket says 172.21.15.119:56900) 2024-04-25T10:17:48.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.907+0000 7f431b400640 1 -- 172.21.15.119:0/1581179679 learned_addr learned my addr 172.21.15.119:0/1581179679 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:48.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.907+0000 7f431be00640 1 --2- 172.21.15.119:0/1581179679 >> v2:172.21.15.119:3300/0 conn(0x7f431c0b42c0 0x7f431c15ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.907+0000 7f4320a00640 1 --2- 172.21.15.119:0/1581179679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f431c0b46a0 0x7f431c1549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.907+0000 7f431aa00640 1 -- 172.21.15.119:0/1581179679 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327547065 0 0) 0x7f431c15b370 con 0x7f431c15aef0 2024-04-25T10:17:48.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.907+0000 7f431aa00640 1 -- 172.21.15.119:0/1581179679 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4300005730 con 0x7f431c15aef0 2024-04-25T10:17:48.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.908+0000 7f431be00640 1 -- 172.21.15.119:0/1581179679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f431c0b46a0 msgr2=0x7f431c1549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:48.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.908+0000 7f431be00640 1 --2- 172.21.15.119:0/1581179679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f431c0b46a0 0x7f431c1549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.908+0000 7f431be00640 1 -- 172.21.15.119:0/1581179679 >> v1:172.21.15.119:6789/0 conn(0x7f431c15aef0 legacy=0x7f431c0cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.908+0000 7f431be00640 1 -- 172.21.15.119:0/1581179679 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c15b930 con 0x7f431c0b42c0 2024-04-25T10:17:48.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.909+0000 7f431be00640 1 --2- 172.21.15.119:0/1581179679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f431c0b42c0 0x7f431c15ef40 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f430c0097a0 tx=0x7f430c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254744de3ed82bb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.909+0000 7f431aa00640 1 -- 172.21.15.119:0/1581179679 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c037070 con 0x7f431c0b42c0 2024-04-25T10:17:48.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.909+0000 7f431aa00640 1 -- 172.21.15.119:0/1581179679 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c02adb0 con 0x7f431c0b42c0 2024-04-25T10:17:48.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.909+0000 7f431aa00640 1 -- 172.21.15.119:0/1581179679 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c032660 con 0x7f431c0b42c0 2024-04-25T10:17:48.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.909+0000 7f4322a00640 1 -- 172.21.15.119:0/1581179679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f431c0b42c0 msgr2=0x7f431c15ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.909+0000 7f4322a00640 1 --2- 172.21.15.119:0/1581179679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f431c0b42c0 0x7f431c15ef40 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f430c0097a0 tx=0x7f430c02a060 comp rx=0 tx=0).stop 2024-04-25T10:17:48.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.910+0000 7f4322a00640 1 -- 172.21.15.119:0/1581179679 shutdown_connections 2024-04-25T10:17:48.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.910+0000 7f4322a00640 1 --2- 172.21.15.119:0/1581179679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f431c0b46a0 0x7f431c1549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.910+0000 7f4322a00640 1 --2- 172.21.15.119:0/1581179679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f431c0b42c0 0x7f431c15ef40 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.910+0000 7f4322a00640 1 -- 172.21.15.119:0/1581179679 >> 172.21.15.119:0/1581179679 conn(0x7f431c06aba0 msgr2=0x7f431c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.910+0000 7f4322a00640 1 -- 172.21.15.119:0/1581179679 shutdown_connections 2024-04-25T10:17:48.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.910+0000 7f4322a00640 1 -- 172.21.15.119:0/1581179679 wait complete. 2024-04-25T10:17:48.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.911+0000 7f4322a00640 1 Processor -- start 2024-04-25T10:17:48.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.912+0000 7f4322a00640 1 -- start start 2024-04-25T10:17:48.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.912+0000 7f4322a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f431c0b42c0 0x7f431c0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.912+0000 7f4322a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f431c0b46a0 0x7f431c0cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.912+0000 7f4322a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f431c15aef0 0x7f431c1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.913+0000 7f4322a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c06ec60 con 0x7f431c0b42c0 2024-04-25T10:17:48.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.913+0000 7f4322a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c06eae0 con 0x7f431c0b46a0 2024-04-25T10:17:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.913+0000 7f4322a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c154f10 con 0x7f431c15aef0 2024-04-25T10:17:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.913+0000 7f431be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f431c0b42c0 0x7f431c0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.913+0000 7f431be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f431c0b42c0 0x7f431c0c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37102/0 (socket says 172.21.15.119:37102) 2024-04-25T10:17:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.913+0000 7f431be00640 1 -- 172.21.15.119:0/618096410 learned_addr learned my addr 172.21.15.119:0/618096410 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.913+0000 7f431be00640 1 -- 172.21.15.119:0/618096410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f431c15aef0 msgr2=0x7f431c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.913+0000 7f431be00640 1 --2- 172.21.15.119:0/618096410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f431c15aef0 0x7f431c1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.913+0000 7f431be00640 1 -- 172.21.15.119:0/618096410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f431c0b46a0 msgr2=0x7f431c0cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.913+0000 7f431be00640 1 --2- 172.21.15.119:0/618096410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f431c0b46a0 0x7f431c0cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.913+0000 7f431be00640 1 -- 172.21.15.119:0/618096410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c0c9ff0 con 0x7f431c0b42c0 2024-04-25T10:17:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.914+0000 7f431be00640 1 --2- 172.21.15.119:0/618096410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f431c0b42c0 0x7f431c0c7e60 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f430c035040 tx=0x7f430c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.914+0000 7f4313e00640 1 -- 172.21.15.119:0/618096410 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c041070 con 0x7f431c0b42c0 2024-04-25T10:17:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.914+0000 7f4313e00640 1 -- 172.21.15.119:0/618096410 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c03c680 con 0x7f431c0b42c0 2024-04-25T10:17:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.914+0000 7f4322a00640 1 -- 172.21.15.119:0/618096410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c0ca2d0 con 0x7f431c0b42c0 2024-04-25T10:17:48.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.914+0000 7f4313e00640 1 -- 172.21.15.119:0/618096410 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c037040 con 0x7f431c0b42c0 2024-04-25T10:17:48.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.914+0000 7f4322a00640 1 -- 172.21.15.119:0/618096410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c0ca890 con 0x7f431c0b42c0 2024-04-25T10:17:48.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.915+0000 7f4322a00640 1 -- 172.21.15.119:0/618096410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c0750b0 con 0x7f431c0b42c0 2024-04-25T10:17:48.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.916+0000 7f4313e00640 1 -- 172.21.15.119:0/618096410 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f430c03a070 con 0x7f431c0b42c0 2024-04-25T10:17:48.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.917+0000 7f4313e00640 1 --2- 172.21.15.119:0/618096410 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42fc06b4d0 0x7f42fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.917+0000 7f4313e00640 1 -- 172.21.15.119:0/618096410 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f430c0aa510 con 0x7f431c0b42c0 2024-04-25T10:17:48.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.917+0000 7f431b400640 1 --2- 172.21.15.119:0/618096410 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42fc06b4d0 0x7f42fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.918+0000 7f431b400640 1 --2- 172.21.15.119:0/618096410 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42fc06b4d0 0x7f42fc06d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4308000f60 tx=0x7f43080078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:48.921+0000 7f4313e00640 1 -- 172.21.15.119:0/618096410 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f430c079390 con 0x7f431c0b42c0 2024-04-25T10:17:49.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.114+0000 7f4322a00640 1 -- 172.21.15.119:0/618096410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f431c06e700 con 0x7f431c0b42c0 2024-04-25T10:17:49.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.115+0000 7f4313e00640 1 -- 172.21.15.119:0/618096410 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7f430c078c90 con 0x7f431c0b42c0 2024-04-25T10:17:49.119 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:17:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.118+0000 7f4322a00640 1 -- 172.21.15.119:0/618096410 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42fc06b4d0 msgr2=0x7f42fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.118+0000 7f4322a00640 1 --2- 172.21.15.119:0/618096410 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42fc06b4d0 0x7f42fc06d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4308000f60 tx=0x7f43080078f0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.118+0000 7f4322a00640 1 -- 172.21.15.119:0/618096410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f431c0b42c0 msgr2=0x7f431c0c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.118+0000 7f4322a00640 1 --2- 172.21.15.119:0/618096410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f431c0b42c0 0x7f431c0c7e60 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f430c035040 tx=0x7f430c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.118+0000 7f4322a00640 1 -- 172.21.15.119:0/618096410 shutdown_connections 2024-04-25T10:17:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.118+0000 7f4322a00640 1 --2- 172.21.15.119:0/618096410 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42fc06b4d0 0x7f42fc06d990 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.118+0000 7f4322a00640 1 --2- 172.21.15.119:0/618096410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f431c15aef0 0x7f431c1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.118+0000 7f4322a00640 1 --2- 172.21.15.119:0/618096410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f431c0b46a0 0x7f431c0cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.118+0000 7f4322a00640 1 --2- 172.21.15.119:0/618096410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f431c0b42c0 0x7f431c0c7e60 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.118+0000 7f4322a00640 1 -- 172.21.15.119:0/618096410 >> 172.21.15.119:0/618096410 conn(0x7f431c06aba0 msgr2=0x7f431c0cb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:49.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.118+0000 7f4322a00640 1 -- 172.21.15.119:0/618096410 shutdown_connections 2024-04-25T10:17:49.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.118+0000 7f4322a00640 1 -- 172.21.15.119:0/618096410 wait complete. 2024-04-25T10:17:49.140 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:17:49.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.342+0000 7fa646000640 1 Processor -- start 2024-04-25T10:17:49.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.342+0000 7fa646000640 1 -- start start 2024-04-25T10:17:49.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.343+0000 7fa646000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa6401450e0 0x7fa64006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.343+0000 7fa646000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6400cc9e0 0x7fa6400b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.343+0000 7fa646000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64015f400 con 0x7fa64006b0e0 2024-04-25T10:17:49.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.343+0000 7fa646000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6401454c0 con 0x7fa6401450e0 2024-04-25T10:17:49.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.344+0000 7fa646000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64006b540 con 0x7fa6400cc9e0 2024-04-25T10:17:49.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.344+0000 7fa645600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6400cc9e0 0x7fa6400b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.344+0000 7fa645600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6400cc9e0 0x7fa6400b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:37252/0 (socket says 172.21.15.119:37252) 2024-04-25T10:17:49.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.344+0000 7fa645600640 1 -- 172.21.15.119:0/58487729 learned_addr learned my addr 172.21.15.119:0/58487729 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:49.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.344+0000 7fa644c00640 1 --2- 172.21.15.119:0/58487729 >> v2:172.21.15.132:3300/0 conn(0x7fa6401450e0 0x7fa64006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.345+0000 7fa645600640 1 -- 172.21.15.119:0/58487729 >> v2:172.21.15.132:3300/0 conn(0x7fa6401450e0 msgr2=0x7fa64006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.345+0000 7fa645600640 1 --2- 172.21.15.119:0/58487729 >> v2:172.21.15.132:3300/0 conn(0x7fa6401450e0 0x7fa64006aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.345+0000 7fa645600640 1 -- 172.21.15.119:0/58487729 >> v1:172.21.15.132:6789/0 conn(0x7fa64006b0e0 legacy=0x7fa6400cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.345+0000 7fa645600640 1 -- 172.21.15.119:0/58487729 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64015f6e0 con 0x7fa6400cc9e0 2024-04-25T10:17:49.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.347+0000 7fa645600640 1 --2- 172.21.15.119:0/58487729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6400cc9e0 0x7fa6400b2e80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa638009870 tx=0x7fa63802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b462e1d7be2996a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.347+0000 7fa63f400640 1 -- 172.21.15.119:0/58487729 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638037070 con 0x7fa6400cc9e0 2024-04-25T10:17:49.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.347+0000 7fa63f400640 1 -- 172.21.15.119:0/58487729 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa638032440 con 0x7fa6400cc9e0 2024-04-25T10:17:49.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.347+0000 7fa63f400640 1 -- 172.21.15.119:0/58487729 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638032810 con 0x7fa6400cc9e0 2024-04-25T10:17:49.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.348+0000 7fa646000640 1 -- 172.21.15.119:0/58487729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6400cc9e0 msgr2=0x7fa6400b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.348+0000 7fa646000640 1 --2- 172.21.15.119:0/58487729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6400cc9e0 0x7fa6400b2e80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa638009870 tx=0x7fa63802a130 comp rx=0 tx=0).stop 2024-04-25T10:17:49.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.348+0000 7fa646000640 1 -- 172.21.15.119:0/58487729 shutdown_connections 2024-04-25T10:17:49.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.348+0000 7fa646000640 1 --2- 172.21.15.119:0/58487729 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6400cc9e0 0x7fa6400b2e80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.348+0000 7fa646000640 1 --2- 172.21.15.119:0/58487729 >> v2:172.21.15.132:3300/0 conn(0x7fa6401450e0 0x7fa64006aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.348+0000 7fa646000640 1 -- 172.21.15.119:0/58487729 >> 172.21.15.119:0/58487729 conn(0x7fa640066ff0 msgr2=0x7fa6400b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.348+0000 7fa646000640 1 -- 172.21.15.119:0/58487729 shutdown_connections 2024-04-25T10:17:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.348+0000 7fa646000640 1 -- 172.21.15.119:0/58487729 wait complete. 2024-04-25T10:17:49.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.350+0000 7fa646000640 1 Processor -- start 2024-04-25T10:17:49.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.350+0000 7fa646000640 1 -- start start 2024-04-25T10:17:49.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.351+0000 7fa646000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa64006b0e0 0x7fa6401f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.351+0000 7fa646000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6400cc9e0 0x7fa6400c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.351+0000 7fa646000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6401450e0 0x7fa6400b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.351+0000 7fa646000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6400b33c0 con 0x7fa6400cc9e0 2024-04-25T10:17:49.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.351+0000 7fa646000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64015fca0 con 0x7fa64006b0e0 2024-04-25T10:17:49.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.351+0000 7fa646000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6400b3540 con 0x7fa6401450e0 2024-04-25T10:17:49.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.351+0000 7fa63fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6400cc9e0 0x7fa6400c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.351+0000 7fa63fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6400cc9e0 0x7fa6400c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37112/0 (socket says 172.21.15.119:37112) 2024-04-25T10:17:49.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.351+0000 7fa63fe00640 1 -- 172.21.15.119:0/1679050595 learned_addr learned my addr 172.21.15.119:0/1679050595 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.351+0000 7fa644c00640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa64006b0e0 0x7fa6401f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.351+0000 7fa645600640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6401450e0 0x7fa6400b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.352+0000 7fa63fe00640 1 -- 172.21.15.119:0/1679050595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6401450e0 msgr2=0x7fa6400b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.352+0000 7fa63fe00640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6401450e0 0x7fa6400b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.352+0000 7fa63fe00640 1 -- 172.21.15.119:0/1679050595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa64006b0e0 msgr2=0x7fa6401f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.352+0000 7fa63fe00640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa64006b0e0 0x7fa6401f3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.352+0000 7fa63fe00640 1 -- 172.21.15.119:0/1679050595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64015d7f0 con 0x7fa6400cc9e0 2024-04-25T10:17:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.352+0000 7fa645600640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6401450e0 0x7fa6400b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.352+0000 7fa644c00640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa64006b0e0 0x7fa6401f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.352+0000 7fa63fe00640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6400cc9e0 0x7fa6400c4960 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fa62800c430 tx=0x7fa62800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.352+0000 7fa63cc00640 1 -- 172.21.15.119:0/1679050595 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628017070 con 0x7fa6400cc9e0 2024-04-25T10:17:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.352+0000 7fa63cc00640 1 -- 172.21.15.119:0/1679050595 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6280052d0 con 0x7fa6400cc9e0 2024-04-25T10:17:49.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.352+0000 7fa646000640 1 -- 172.21.15.119:0/1679050595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6401f4a70 con 0x7fa6400cc9e0 2024-04-25T10:17:49.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.352+0000 7fa63cc00640 1 -- 172.21.15.119:0/1679050595 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628003850 con 0x7fa6400cc9e0 2024-04-25T10:17:49.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.353+0000 7fa646000640 1 -- 172.21.15.119:0/1679050595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6401f41e0 con 0x7fa6400cc9e0 2024-04-25T10:17:49.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.353+0000 7fa646000640 1 -- 172.21.15.119:0/1679050595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6400749b0 con 0x7fa6400cc9e0 2024-04-25T10:17:49.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.355+0000 7fa63cc00640 1 -- 172.21.15.119:0/1679050595 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa62801a070 con 0x7fa6400cc9e0 2024-04-25T10:17:49.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.356+0000 7fa63cc00640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa61c06b4d0 0x7fa61c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.356+0000 7fa63cc00640 1 -- 172.21.15.119:0/1679050595 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa628089640 con 0x7fa6400cc9e0 2024-04-25T10:17:49.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.356+0000 7fa644c00640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa61c06b4d0 0x7fa61c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.357+0000 7fa644c00640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa61c06b4d0 0x7fa61c06d990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa630007440 tx=0x7fa63000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.362+0000 7fa63cc00640 1 -- 172.21.15.119:0/1679050595 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa628058420 con 0x7fa6400cc9e0 2024-04-25T10:17:49.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.560+0000 7fa646000640 1 -- 172.21.15.119:0/1679050595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa640142df0 con 0x7fa6400cc9e0 2024-04-25T10:17:49.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.561+0000 7fa63cc00640 1 -- 172.21.15.119:0/1679050595 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa628057d20 con 0x7fa6400cc9e0 2024-04-25T10:17:49.564 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:17:49.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.563+0000 7fa646000640 1 -- 172.21.15.119:0/1679050595 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa61c06b4d0 msgr2=0x7fa61c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.563+0000 7fa646000640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa61c06b4d0 0x7fa61c06d990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa630007440 tx=0x7fa63000b040 comp rx=0 tx=0).stop 2024-04-25T10:17:49.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.563+0000 7fa646000640 1 -- 172.21.15.119:0/1679050595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6400cc9e0 msgr2=0x7fa6400c4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.563+0000 7fa646000640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6400cc9e0 0x7fa6400c4960 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fa62800c430 tx=0x7fa62800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.563+0000 7fa646000640 1 -- 172.21.15.119:0/1679050595 shutdown_connections 2024-04-25T10:17:49.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.563+0000 7fa646000640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa61c06b4d0 0x7fa61c06d990 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.563+0000 7fa646000640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6401450e0 0x7fa6400b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.563+0000 7fa646000640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6400cc9e0 0x7fa6400c4960 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.563+0000 7fa646000640 1 --2- 172.21.15.119:0/1679050595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa64006b0e0 0x7fa6401f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.563+0000 7fa646000640 1 -- 172.21.15.119:0/1679050595 >> 172.21.15.119:0/1679050595 conn(0x7fa640066ff0 msgr2=0x7fa64006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:49.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.564+0000 7fa646000640 1 -- 172.21.15.119:0/1679050595 shutdown_connections 2024-04-25T10:17:49.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.564+0000 7fa646000640 1 -- 172.21.15.119:0/1679050595 wait complete. 2024-04-25T10:17:49.584 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:17:49.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.779+0000 7f445f200640 1 Processor -- start 2024-04-25T10:17:49.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.779+0000 7f445f200640 1 -- start start 2024-04-25T10:17:49.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.781+0000 7f445f200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f44580ce860 0x7f44580cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.781+0000 7f445f200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44580b5710 con 0x7f44580d00a0 2024-04-25T10:17:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.781+0000 7f445f200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44580b59f0 con 0x7f445815b7e0 2024-04-25T10:17:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.781+0000 7f445f200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44580b5cd0 con 0x7f44580ce860 2024-04-25T10:17:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.781+0000 7f445e800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f44580d00a0 0x7f44580d0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56904/0 (socket says 172.21.15.119:56904) 2024-04-25T10:17:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.781+0000 7f445e800640 1 -- 172.21.15.119:0/4019767334 learned_addr learned my addr 172.21.15.119:0/4019767334 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445d400640 1 --2- 172.21.15.119:0/4019767334 >> v2:172.21.15.132:3300/0 conn(0x7f44580ce860 0x7f44580cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 -- 172.21.15.119:0/4019767334 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993512766 0 0) 0x7f44580b5710 con 0x7f44580d00a0 2024-04-25T10:17:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 -- 172.21.15.119:0/4019767334 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4444003520 con 0x7f44580d00a0 2024-04-25T10:17:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 -- 172.21.15.119:0/4019767334 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2357099695 0 0) 0x7f4444003520 con 0x7f44580d00a0 2024-04-25T10:17:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 -- 172.21.15.119:0/4019767334 >> v2:172.21.15.132:3300/0 conn(0x7f44580ce860 msgr2=0x7f44580cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 --2- 172.21.15.119:0/4019767334 >> v2:172.21.15.132:3300/0 conn(0x7f44580ce860 0x7f44580cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 -- 172.21.15.119:0/4019767334 >> v1:172.21.15.132:6789/0 conn(0x7f445815b7e0 legacy=0x7f445815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 -- 172.21.15.119:0/4019767334 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44580b5fb0 con 0x7f44580d00a0 2024-04-25T10:17:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 -- 172.21.15.119:0/4019767334 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f4454001670 con 0x7f44580d00a0 2024-04-25T10:17:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 -- 172.21.15.119:0/4019767334 >> v1:172.21.15.119:6789/0 conn(0x7f44580d00a0 legacy=0x7f44580d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 --2- 172.21.15.119:0/4019767334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44440039b0 0x7f4444003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 --2- 172.21.15.119:0/4019767334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4444004800 0x7f4444004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 --2- 172.21.15.119:0/4019767334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44440052e0 0x7f444400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 -- 172.21.15.119:0/4019767334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44580b5cd0 con 0x7f4444004800 2024-04-25T10:17:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 -- 172.21.15.119:0/4019767334 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444400c1b0 con 0x7f44440039b0 2024-04-25T10:17:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.782+0000 7f445ca00640 1 -- 172.21.15.119:0/4019767334 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444400c3b0 con 0x7f44440052e0 2024-04-25T10:17:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.783+0000 7f445d400640 1 --2- 172.21.15.119:0/4019767334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4444004800 0x7f4444004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.783+0000 7f445d400640 1 -- 172.21.15.119:0/4019767334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44440052e0 msgr2=0x7f444400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.783+0000 7f445d400640 1 --2- 172.21.15.119:0/4019767334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f44440052e0 0x7f444400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.783+0000 7f445d400640 1 -- 172.21.15.119:0/4019767334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44440039b0 msgr2=0x7f4444003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.783+0000 7f445d400640 1 --2- 172.21.15.119:0/4019767334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44440039b0 0x7f4444003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.783+0000 7f445d400640 1 -- 172.21.15.119:0/4019767334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4444003520 con 0x7f4444004800 2024-04-25T10:17:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.783+0000 7f445e800640 1 -- 172.21.15.119:0/4019767334 reap_dead start 2024-04-25T10:17:49.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.784+0000 7f445d400640 1 --2- 172.21.15.119:0/4019767334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4444004800 0x7f4444004c80 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f4448009a90 tx=0x7f4444005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439d721e9ce9e445 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.784+0000 7f445ca00640 1 -- 172.21.15.119:0/4019767334 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448036070 con 0x7f4444004800 2024-04-25T10:17:49.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.784+0000 7f445ca00640 1 -- 172.21.15.119:0/4019767334 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4448031400 con 0x7f4444004800 2024-04-25T10:17:49.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.784+0000 7f445ca00640 1 -- 172.21.15.119:0/4019767334 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448009690 con 0x7f4444004800 2024-04-25T10:17:49.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.785+0000 7f445f200640 1 -- 172.21.15.119:0/4019767334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4444004800 msgr2=0x7f4444004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.785+0000 7f445f200640 1 --2- 172.21.15.119:0/4019767334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4444004800 0x7f4444004c80 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f4448009a90 tx=0x7f4444005830 comp rx=0 tx=0).stop 2024-04-25T10:17:49.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.785+0000 7f445f200640 1 -- 172.21.15.119:0/4019767334 shutdown_connections 2024-04-25T10:17:49.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.785+0000 7f445f200640 1 --2- 172.21.15.119:0/4019767334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4444004800 0x7f4444004c80 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.785+0000 7f445f200640 1 -- 172.21.15.119:0/4019767334 >> 172.21.15.119:0/4019767334 conn(0x7f44580722d0 msgr2=0x7f44580726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:49.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.785+0000 7f445f200640 1 -- 172.21.15.119:0/4019767334 shutdown_connections 2024-04-25T10:17:49.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.785+0000 7f445f200640 1 -- 172.21.15.119:0/4019767334 wait complete. 2024-04-25T10:17:49.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.787+0000 7f445f200640 1 Processor -- start 2024-04-25T10:17:49.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.787+0000 7f445f200640 1 -- start start 2024-04-25T10:17:49.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.788+0000 7f445f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4444004800 0x7f44580c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.788+0000 7f445f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44580cba60 0x7f44580cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.788+0000 7f445f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445815f960 0x7f445815fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.788+0000 7f445f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44580cee70 con 0x7f445815f960 2024-04-25T10:17:49.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.788+0000 7f445f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44580cecf0 con 0x7f44580cba60 2024-04-25T10:17:49.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.788+0000 7f445f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44580ceff0 con 0x7f4444004800 2024-04-25T10:17:49.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.788+0000 7f445e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445815f960 0x7f445815fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.788+0000 7f445e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445815f960 0x7f445815fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37122/0 (socket says 172.21.15.119:37122) 2024-04-25T10:17:49.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.788+0000 7f445e800640 1 -- 172.21.15.119:0/476870028 learned_addr learned my addr 172.21.15.119:0/476870028 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:49.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.789+0000 7f445de00640 1 --2- 172.21.15.119:0/476870028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4444004800 0x7f44580c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.789+0000 7f445e800640 1 -- 172.21.15.119:0/476870028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4444004800 msgr2=0x7f44580c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.789+0000 7f445e800640 1 --2- 172.21.15.119:0/476870028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4444004800 0x7f44580c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.789+0000 7f445e800640 1 -- 172.21.15.119:0/476870028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44580cba60 msgr2=0x7f44580cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.789+0000 7f445e800640 1 --2- 172.21.15.119:0/476870028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44580cba60 0x7f44580cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.789+0000 7f445e800640 1 -- 172.21.15.119:0/476870028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44580c96c0 con 0x7f445815f960 2024-04-25T10:17:49.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.789+0000 7f445de00640 1 --2- 172.21.15.119:0/476870028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4444004800 0x7f44580c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:49.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.789+0000 7f445e800640 1 --2- 172.21.15.119:0/476870028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445815f960 0x7f445815fde0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f445808bf10 tx=0x7f445400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.789+0000 7f444e000640 1 -- 172.21.15.119:0/476870028 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4454007440 con 0x7f445815f960 2024-04-25T10:17:49.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.789+0000 7f444e000640 1 -- 172.21.15.119:0/476870028 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4454007600 con 0x7f445815f960 2024-04-25T10:17:49.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.789+0000 7f445f200640 1 -- 172.21.15.119:0/476870028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44580cc5d0 con 0x7f445815f960 2024-04-25T10:17:49.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.789+0000 7f444e000640 1 -- 172.21.15.119:0/476870028 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4454004a20 con 0x7f445815f960 2024-04-25T10:17:49.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.789+0000 7f445f200640 1 -- 172.21.15.119:0/476870028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44580ccb90 con 0x7f445815f960 2024-04-25T10:17:49.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.790+0000 7f445f200640 1 -- 172.21.15.119:0/476870028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4458160320 con 0x7f445815f960 2024-04-25T10:17:49.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.792+0000 7f444e000640 1 -- 172.21.15.119:0/476870028 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4454004bc0 con 0x7f445815f960 2024-04-25T10:17:49.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.793+0000 7f444e000640 1 --2- 172.21.15.119:0/476870028 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f443c06b500 0x7f443c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.793+0000 7f444e000640 1 -- 172.21.15.119:0/476870028 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4454089c50 con 0x7f445815f960 2024-04-25T10:17:49.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.793+0000 7f445de00640 1 --2- 172.21.15.119:0/476870028 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f443c06b500 0x7f443c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.794+0000 7f445de00640 1 --2- 172.21.15.119:0/476870028 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f443c06b500 0x7f443c06d9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4450007fd0 tx=0x7f4450009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:49.797+0000 7f444e000640 1 -- 172.21.15.119:0/476870028 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f445408f020 con 0x7f445815f960 2024-04-25T10:17:50.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.004+0000 7f445f200640 1 -- 172.21.15.119:0/476870028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f44580749b0 con 0x7f445815f960 2024-04-25T10:17:50.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.004+0000 7f444e000640 1 -- 172.21.15.119:0/476870028 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v108) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4454058a40 con 0x7f445815f960 2024-04-25T10:17:50.008 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2024-04-25T10:17:50.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.007+0000 7f445f200640 1 -- 172.21.15.119:0/476870028 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f443c06b500 msgr2=0x7f443c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.007+0000 7f445f200640 1 --2- 172.21.15.119:0/476870028 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f443c06b500 0x7f443c06d9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4450007fd0 tx=0x7f4450009040 comp rx=0 tx=0).stop 2024-04-25T10:17:50.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.007+0000 7f445f200640 1 -- 172.21.15.119:0/476870028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445815f960 msgr2=0x7f445815fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.007+0000 7f445f200640 1 --2- 172.21.15.119:0/476870028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445815f960 0x7f445815fde0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f445808bf10 tx=0x7f445400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.007+0000 7f445f200640 1 -- 172.21.15.119:0/476870028 shutdown_connections 2024-04-25T10:17:50.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.007+0000 7f445f200640 1 --2- 172.21.15.119:0/476870028 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f443c06b500 0x7f443c06d9c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.007+0000 7f445f200640 1 --2- 172.21.15.119:0/476870028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f445815f960 0x7f445815fde0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.007+0000 7f445f200640 1 --2- 172.21.15.119:0/476870028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44580cba60 0x7f44580cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.007+0000 7f445f200640 1 --2- 172.21.15.119:0/476870028 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4444004800 0x7f44580c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.007+0000 7f445f200640 1 -- 172.21.15.119:0/476870028 >> 172.21.15.119:0/476870028 conn(0x7f44580722d0 msgr2=0x7f44580d1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.007+0000 7f445f200640 1 -- 172.21.15.119:0/476870028 shutdown_connections 2024-04-25T10:17:50.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.007+0000 7f445f200640 1 -- 172.21.15.119:0/476870028 wait complete. 2024-04-25T10:17:50.030 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:17:50.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:50.030 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:50.148 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:17:50.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:50.148 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:17:50.199 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:17:50.199 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T10:17:50.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.366+0000 7f27af200640 1 Processor -- start 2024-04-25T10:17:50.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.367+0000 7f27af200640 1 -- start start 2024-04-25T10:17:50.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.368+0000 7f27af200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27b006ae70 0x7f27b0162040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.368+0000 7f27af200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f27b006b250 0x7f27b00ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.368+0000 7f27af200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b00b5600 con 0x7f27b00add10 2024-04-25T10:17:50.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.368+0000 7f27af200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b0162580 con 0x7f27b006b250 2024-04-25T10:17:50.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.368+0000 7f27af200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00b34a0 con 0x7f27b006ae70 2024-04-25T10:17:50.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.369+0000 7f27ae800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f27b00add10 0x7f27b00ae180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56910/0 (socket says 172.21.15.119:56910) 2024-04-25T10:17:50.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.369+0000 7f27ae800640 1 -- 172.21.15.119:0/1024441650 learned_addr learned my addr 172.21.15.119:0/1024441650 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:50.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.369+0000 7f27aca00640 1 -- 172.21.15.119:0/1024441650 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590717257 0 0) 0x7f27b00b5600 con 0x7f27b00add10 2024-04-25T10:17:50.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.369+0000 7f27ade00640 1 --2- 172.21.15.119:0/1024441650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27b006ae70 0x7f27b0162040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.369+0000 7f27ad400640 1 --2- 172.21.15.119:0/1024441650 >> v2:172.21.15.132:3300/0 conn(0x7f27b006b250 0x7f27b00ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.369+0000 7f27aca00640 1 -- 172.21.15.119:0/1024441650 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2794003520 con 0x7f27b00add10 2024-04-25T10:17:50.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.369+0000 7f27aca00640 1 -- 172.21.15.119:0/1024441650 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1248519138 0 0) 0x7f2794003520 con 0x7f27b00add10 2024-04-25T10:17:50.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.369+0000 7f27aca00640 1 -- 172.21.15.119:0/1024441650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27b006ae70 msgr2=0x7f27b0162040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.369+0000 7f27aca00640 1 --2- 172.21.15.119:0/1024441650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27b006ae70 0x7f27b0162040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.369+0000 7f27aca00640 1 -- 172.21.15.119:0/1024441650 >> v2:172.21.15.132:3300/0 conn(0x7f27b006b250 msgr2=0x7f27b00ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.369+0000 7f27aca00640 1 --2- 172.21.15.119:0/1024441650 >> v2:172.21.15.132:3300/0 conn(0x7f27b006b250 0x7f27b00ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.369+0000 7f27aca00640 1 -- 172.21.15.119:0/1024441650 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b00b36f0 con 0x7f27b00add10 2024-04-25T10:17:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.369+0000 7f27aca00640 1 -- 172.21.15.119:0/1024441650 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f27a0001670 con 0x7f27b00add10 2024-04-25T10:17:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.369+0000 7f27aca00640 1 -- 172.21.15.119:0/1024441650 >> v1:172.21.15.119:6789/0 conn(0x7f27b00add10 legacy=0x7f27b00ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.370+0000 7f27aca00640 1 --2- 172.21.15.119:0/1024441650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27b006ae70 0x7f27940039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.370+0000 7f27aca00640 1 --2- 172.21.15.119:0/1024441650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2794003fe0 0x7f2794004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.370+0000 7f27aca00640 1 --2- 172.21.15.119:0/1024441650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2794004d40 0x7f27940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.370+0000 7f27aca00640 1 -- 172.21.15.119:0/1024441650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b0162580 con 0x7f2794003fe0 2024-04-25T10:17:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.370+0000 7f27aca00640 1 -- 172.21.15.119:0/1024441650 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00b34a0 con 0x7f2794004d40 2024-04-25T10:17:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.370+0000 7f27aca00640 1 -- 172.21.15.119:0/1024441650 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279400bb80 con 0x7f27b006ae70 2024-04-25T10:17:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.370+0000 7f27ad400640 1 --2- 172.21.15.119:0/1024441650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2794003fe0 0x7f2794004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.370+0000 7f27ad400640 1 -- 172.21.15.119:0/1024441650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27b006ae70 msgr2=0x7f27940039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.370+0000 7f27ad400640 1 --2- 172.21.15.119:0/1024441650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27b006ae70 0x7f27940039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.370+0000 7f27ae800640 1 --2- 172.21.15.119:0/1024441650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2794004d40 0x7f27940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.370+0000 7f27ad400640 1 -- 172.21.15.119:0/1024441650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2794004d40 msgr2=0x7f27940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.370+0000 7f27ad400640 1 --2- 172.21.15.119:0/1024441650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2794004d40 0x7f27940051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.370+0000 7f27ad400640 1 -- 172.21.15.119:0/1024441650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2794003520 con 0x7f2794003fe0 2024-04-25T10:17:50.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.371+0000 7f27ad400640 1 --2- 172.21.15.119:0/1024441650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2794003fe0 0x7f2794004800 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f2798007360 tx=0x7f2794005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514c068ae7545f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.372+0000 7f27aca00640 1 -- 172.21.15.119:0/1024441650 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2798009be0 con 0x7f2794003fe0 2024-04-25T10:17:50.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.372+0000 7f27aca00640 1 -- 172.21.15.119:0/1024441650 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279802e620 con 0x7f2794003fe0 2024-04-25T10:17:50.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.372+0000 7f27aca00640 1 -- 172.21.15.119:0/1024441650 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279802e9f0 con 0x7f2794003fe0 2024-04-25T10:17:50.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.372+0000 7f27af200640 1 -- 172.21.15.119:0/1024441650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2794003fe0 msgr2=0x7f2794004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.372+0000 7f27af200640 1 --2- 172.21.15.119:0/1024441650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2794003fe0 0x7f2794004800 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f2798007360 tx=0x7f2794005830 comp rx=0 tx=0).stop 2024-04-25T10:17:50.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.372+0000 7f27ae800640 1 -- 172.21.15.119:0/1024441650 reap_dead start 2024-04-25T10:17:50.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.372+0000 7f27af200640 1 -- 172.21.15.119:0/1024441650 shutdown_connections 2024-04-25T10:17:50.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.372+0000 7f27af200640 1 -- 172.21.15.119:0/1024441650 >> 172.21.15.119:0/1024441650 conn(0x7f27b00b59c0 msgr2=0x7f27b00b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.373+0000 7f27af200640 1 -- 172.21.15.119:0/1024441650 shutdown_connections 2024-04-25T10:17:50.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.373+0000 7f27af200640 1 -- 172.21.15.119:0/1024441650 wait complete. 2024-04-25T10:17:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.374+0000 7f27af200640 1 Processor -- start 2024-04-25T10:17:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.374+0000 7f27af200640 1 -- start start 2024-04-25T10:17:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.375+0000 7f27af200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2794003fe0 0x7f27b01572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.375+0000 7f27af200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27b01408f0 0x7f27b01410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.375+0000 7f27af200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b0141630 0x7f27b0155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.375+0000 7f27af200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00b1c50 con 0x7f27b0141630 2024-04-25T10:17:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.375+0000 7f27af200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00b3cb0 con 0x7f2794003fe0 2024-04-25T10:17:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.375+0000 7f27af200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00b1dd0 con 0x7f27b01408f0 2024-04-25T10:17:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.376+0000 7f27ae800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b0141630 0x7f27b0155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.376+0000 7f27ae800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b0141630 0x7f27b0155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37130/0 (socket says 172.21.15.119:37130) 2024-04-25T10:17:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.376+0000 7f27ae800640 1 -- 172.21.15.119:0/419601719 learned_addr learned my addr 172.21.15.119:0/419601719 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.376+0000 7f27ae800640 1 -- 172.21.15.119:0/419601719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27b01408f0 msgr2=0x7f27b01410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.376+0000 7f27ae800640 1 --2- 172.21.15.119:0/419601719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27b01408f0 0x7f27b01410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.376+0000 7f27ae800640 1 -- 172.21.15.119:0/419601719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2794003fe0 msgr2=0x7f27b01572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.376+0000 7f27ade00640 1 --2- 172.21.15.119:0/419601719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2794003fe0 0x7f27b01572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.376+0000 7f27ae800640 1 --2- 172.21.15.119:0/419601719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2794003fe0 0x7f27b01572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.376+0000 7f27ae800640 1 -- 172.21.15.119:0/419601719 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b0141cc0 con 0x7f27b0141630 2024-04-25T10:17:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.376+0000 7f27ae800640 1 --2- 172.21.15.119:0/419601719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b0141630 0x7f27b0155030 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f27b00ae0f0 tx=0x7f27a000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.377+0000 7f27a6000640 1 -- 172.21.15.119:0/419601719 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a00072d0 con 0x7f27b0141630 2024-04-25T10:17:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.377+0000 7f27a6000640 1 -- 172.21.15.119:0/419601719 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a0007470 con 0x7f27b0141630 2024-04-25T10:17:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.377+0000 7f27af200640 1 -- 172.21.15.119:0/419601719 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b01e8220 con 0x7f27b0141630 2024-04-25T10:17:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.377+0000 7f27a6000640 1 -- 172.21.15.119:0/419601719 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a00049e0 con 0x7f27b0141630 2024-04-25T10:17:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.377+0000 7f27af200640 1 -- 172.21.15.119:0/419601719 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b0156ac0 con 0x7f27b0141630 2024-04-25T10:17:50.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.380+0000 7f27a6000640 1 -- 172.21.15.119:0/419601719 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27a001b020 con 0x7f27b0141630 2024-04-25T10:17:50.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.381+0000 7f27a6000640 1 --2- 172.21.15.119:0/419601719 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f278c06b4d0 0x7f278c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.381+0000 7f27a6000640 1 -- 172.21.15.119:0/419601719 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27a0089cf0 con 0x7f27b0141630 2024-04-25T10:17:50.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.381+0000 7f27ade00640 1 --2- 172.21.15.119:0/419601719 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f278c06b4d0 0x7f278c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.381+0000 7f27b7df3740 1 Processor -- start 2024-04-25T10:17:50.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.381+0000 7f27b7df3740 1 -- start start 2024-04-25T10:17:50.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.382+0000 7f27ade00640 1 --2- 172.21.15.119:0/419601719 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f278c06b4d0 0x7f278c06d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f27a8006a90 tx=0x7f27a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.382+0000 7f27b7df3740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd70ed70 0x558afd710ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.382+0000 7f27b7df3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd74b440 0x558afd70f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.382+0000 7f27b7df3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd739830 0x558afd739cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.382+0000 7f27b7df3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afd672c70 con 0x558afd74b440 2024-04-25T10:17:50.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.382+0000 7f27b7df3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afd61bcf0 con 0x558afd739830 2024-04-25T10:17:50.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.382+0000 7f27b7df3740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558afd66ed10 con 0x558afd70ed70 2024-04-25T10:17:50.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.382+0000 7f27ad400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd74b440 0x558afd70f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.382+0000 7f27ad400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd74b440 0x558afd70f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37142/0 (socket says 172.21.15.119:37142) 2024-04-25T10:17:50.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.382+0000 7f27ad400640 1 -- 172.21.15.119:0/1228787632 learned_addr learned my addr 172.21.15.119:0/1228787632 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:50.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.383+0000 7f27ad400640 1 -- 172.21.15.119:0/1228787632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd70ed70 msgr2=0x558afd710ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.383+0000 7f27ae800640 1 --2- 172.21.15.119:0/1228787632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd739830 0x558afd739cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.383+0000 7f27ad400640 1 --2- 172.21.15.119:0/1228787632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd70ed70 0x558afd710ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.383+0000 7f27ad400640 1 -- 172.21.15.119:0/1228787632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd739830 msgr2=0x558afd739cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.383+0000 7f27ad400640 1 --2- 172.21.15.119:0/1228787632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd739830 0x558afd739cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.383+0000 7f27ad400640 1 -- 172.21.15.119:0/1228787632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558afd4517d0 con 0x558afd74b440 2024-04-25T10:17:50.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.383+0000 7f27ad400640 1 --2- 172.21.15.119:0/1228787632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd74b440 0x558afd70f150 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f279802b0a0 tx=0x7f279802d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26142c1ad466f984 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.383+0000 7f279ea00640 1 -- 172.21.15.119:0/1228787632 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2798045070 con 0x558afd74b440 2024-04-25T10:17:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.383+0000 7f279ea00640 1 -- 172.21.15.119:0/1228787632 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2798003b90 con 0x558afd74b440 2024-04-25T10:17:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.383+0000 7f279ea00640 1 -- 172.21.15.119:0/1228787632 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2798004030 con 0x558afd74b440 2024-04-25T10:17:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.384+0000 7f27b7df3740 1 -- 172.21.15.119:0/1228787632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd74b440 msgr2=0x558afd70f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.384+0000 7f27b7df3740 1 --2- 172.21.15.119:0/1228787632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd74b440 0x558afd70f150 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f279802b0a0 tx=0x7f279802d000 comp rx=0 tx=0).stop 2024-04-25T10:17:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.384+0000 7f27b7df3740 1 -- 172.21.15.119:0/1228787632 shutdown_connections 2024-04-25T10:17:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.384+0000 7f27b7df3740 1 --2- 172.21.15.119:0/1228787632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd739830 0x558afd739cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.384+0000 7f27b7df3740 1 --2- 172.21.15.119:0/1228787632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd74b440 0x558afd70f150 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.384+0000 7f27b7df3740 1 --2- 172.21.15.119:0/1228787632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd70ed70 0x558afd710ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.384+0000 7f27b7df3740 1 -- 172.21.15.119:0/1228787632 >> 172.21.15.119:0/1228787632 conn(0x558afd6d6a30 msgr2=0x558afd754830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.384+0000 7f27b7df3740 1 -- 172.21.15.119:0/1228787632 shutdown_connections 2024-04-25T10:17:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.384+0000 7f27b7df3740 1 -- 172.21.15.119:0/1228787632 wait complete. 2024-04-25T10:17:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.385+0000 7f27b7df3740 1 -- start start 2024-04-25T10:17:50.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.385+0000 7f27b7df3740 1 Processor -- start 2024-04-25T10:17:50.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.386+0000 7f27b7df3740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd70ed70 0x558afd74c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.386+0000 7f27b7df3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd739830 0x558afd74cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.386+0000 7f27b7df3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd74b440 0x558afd73b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.386+0000 7f27b7df3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afd6113a0 con 0x558afd74b440 2024-04-25T10:17:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.386+0000 7f27b7df3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afd737380 con 0x558afd739830 2024-04-25T10:17:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.386+0000 7f27b7df3740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558afd727260 con 0x558afd70ed70 2024-04-25T10:17:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.386+0000 7f27b7df3740 10 client.0 _ll_register_callbacks cb 0x558afd62eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.386+0000 7f27ae800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd74b440 0x558afd73b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.386+0000 7f27ae800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd74b440 0x558afd73b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37152/0 (socket says 172.21.15.119:37152) 2024-04-25T10:17:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.386+0000 7f27ae800640 1 -- 172.21.15.119:0/1784616047 learned_addr learned my addr 172.21.15.119:0/1784616047 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.387+0000 7f27ae800640 1 -- 172.21.15.119:0/1784616047 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd70ed70 msgr2=0x558afd74c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.387+0000 7f27ae800640 1 --2- 172.21.15.119:0/1784616047 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd70ed70 0x558afd74c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.387+0000 7f27ae800640 1 -- 172.21.15.119:0/1784616047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd739830 msgr2=0x558afd74cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.387+0000 7f27ae800640 1 --2- 172.21.15.119:0/1784616047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd739830 0x558afd74cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.387+0000 7f27ae800640 1 -- 172.21.15.119:0/1784616047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558afd3873f0 con 0x558afd74b440 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.387+0000 7f27ae800640 1 --2- 172.21.15.119:0/1784616047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd74b440 0x558afd73b8e0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f27a0000fb0 tx=0x7f27a0058cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fbc128dca1481e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.387+0000 7f2793400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.387+0000 7f2793400640 1 -- 172.21.15.119:0/1784616047 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0060980 con 0x558afd74b440 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.387+0000 7f2793400640 1 -- 172.21.15.119:0/1784616047 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27a0004b80 con 0x558afd74b440 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.387+0000 7f27b7df3740 1 -- 172.21.15.119:0/1784616047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2798003b90 con 0x558afd74b440 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.387+0000 7f2793400640 1 -- 172.21.15.119:0/1784616047 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27a0086050 con 0x558afd74b440 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.387+0000 7f27b7df3740 10 client.8886 fetch_fsmap learned FSMap version 99 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.387+0000 7f27b7df3740 1 -- 172.21.15.119:0/1784616047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x558afd3470f0 con 0x558afd74b440 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.388+0000 7f2793400640 1 -- 172.21.15.119:0/1784616047 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2798003b90 con 0x558afd74b440 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.388+0000 7f2793400640 1 -- 172.21.15.119:0/1784616047 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f27a005c160 con 0x558afd74b440 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.388+0000 7f27b7df3740 10 client.8886 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.388+0000 7f27b7df3740 10 client.8886 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:17:50.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.388+0000 7f27b7df3740 20 client.8886 populate_metadata read hostname 'smithi119' 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.388+0000 7f27b7df3740 1 --2- 172.21.15.119:0/1784616047 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x558afd726a80 0x558afd73be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.388+0000 7f27b7df3740 4 client.8886 mds_command: new command op to 8697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.388+0000 7f27b7df3740 1 -- 172.21.15.119:0/1784616047 --> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558afd6801b0 con 0x558afd726a80 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.388+0000 7f27ade00640 1 --2- 172.21.15.119:0/1784616047 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x558afd726a80 0x558afd73be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.389+0000 7f27ade00640 1 --2- 172.21.15.119:0/1784616047 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x558afd726a80 0x558afd73be20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77f227de96682518 server_cookie=f6161b1272421968 in_seq=0 out_seq=0 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.389+0000 7f2793400640 10 client.8886 ms_handle_connect on v2:172.21.15.119:6836/1849584205 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.389+0000 7f2793400640 1 -- 172.21.15.119:0/1784616047 <== mds.0 v2:172.21.15.119:6836/1849584205 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558afd6801b0 con 0x558afd726a80 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.389+0000 7f2793400640 10 client.8886 handle_command_reply: tid=0 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.389+0000 7f27b7df3740 1 client.8886 shutdown 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.389+0000 7f27b7df3740 1 -- 172.21.15.119:0/1784616047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd74b440 msgr2=0x558afd73b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.389+0000 7f27b7df3740 1 --2- 172.21.15.119:0/1784616047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd74b440 0x558afd73b8e0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f27a0000fb0 tx=0x7f27a0058cc0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.390+0000 7f27b7df3740 1 -- 172.21.15.119:0/1784616047 shutdown_connections 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.390+0000 7f27b7df3740 1 --2- 172.21.15.119:0/1784616047 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x558afd726a80 0x558afd73be20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.390+0000 7f27b7df3740 1 --2- 172.21.15.119:0/1784616047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd74b440 0x558afd73b8e0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.390+0000 7f27b7df3740 1 --2- 172.21.15.119:0/1784616047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd739830 0x558afd74cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.390+0000 7f2793400640 0 client.8886 ms_handle_reset on v2:172.21.15.119:6836/1849584205 2024-04-25T10:17:50.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.390+0000 7f27b7df3740 1 --2- 172.21.15.119:0/1784616047 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd70ed70 0x558afd74c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.390+0000 7f27b7df3740 1 -- 172.21.15.119:0/1784616047 >> 172.21.15.119:0/1784616047 conn(0x558afd6d6a30 msgr2=0x558afd7349f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.390+0000 7f27b7df3740 1 -- 172.21.15.119:0/1784616047 shutdown_connections 2024-04-25T10:17:50.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.390+0000 7f27b7df3740 1 -- 172.21.15.119:0/1784616047 wait complete. 2024-04-25T10:17:50.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.390+0000 7f27b7df3740 20 client.8886 trim_cache size 0 max 16384 2024-04-25T10:17:50.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.404+0000 7f27b7df3740 1 Processor -- start 2024-04-25T10:17:50.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.404+0000 7f27b7df3740 1 -- start start 2024-04-25T10:17:50.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.405+0000 7f27b7df3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd70ed70 0x558afd74e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.405+0000 7f27b7df3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd739830 0x558afd74e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.405+0000 7f27b7df3740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd74b440 0x558afd733ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.405+0000 7f27b7df3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b02004e0 con 0x558afd739830 2024-04-25T10:17:50.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.405+0000 7f27b7df3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b01ff660 con 0x558afd70ed70 2024-04-25T10:17:50.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.405+0000 7f27b7df3740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b01e7e70 con 0x558afd74b440 2024-04-25T10:17:50.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.405+0000 7f27ad400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd739830 0x558afd74e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.406+0000 7f27ad400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd739830 0x558afd74e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37164/0 (socket says 172.21.15.119:37164) 2024-04-25T10:17:50.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.406+0000 7f27ad400640 1 -- 172.21.15.119:0/1833482305 learned_addr learned my addr 172.21.15.119:0/1833482305 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:50.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.406+0000 7f27ae800640 1 --2- 172.21.15.119:0/1833482305 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd74b440 0x558afd733ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.406+0000 7f27ad400640 1 -- 172.21.15.119:0/1833482305 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd74b440 msgr2=0x558afd733ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.406+0000 7f27ad400640 1 --2- 172.21.15.119:0/1833482305 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd74b440 0x558afd733ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.406+0000 7f27ad400640 1 -- 172.21.15.119:0/1833482305 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd70ed70 msgr2=0x558afd74e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.406+0000 7f27ade00640 1 --2- 172.21.15.119:0/1833482305 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd70ed70 0x558afd74e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.406+0000 7f27ad400640 1 --2- 172.21.15.119:0/1833482305 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd70ed70 0x558afd74e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.406+0000 7f27ad400640 1 -- 172.21.15.119:0/1833482305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558afd725a10 con 0x558afd739830 2024-04-25T10:17:50.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.406+0000 7f27ae800640 1 --2- 172.21.15.119:0/1833482305 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd74b440 0x558afd733ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:50.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.406+0000 7f27ad400640 1 --2- 172.21.15.119:0/1833482305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd739830 0x558afd74e6d0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f2798007450 tx=0x7f2798030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404d5427a86bd279 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.406+0000 7f279e000640 1 -- 172.21.15.119:0/1833482305 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279804f070 con 0x558afd739830 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.406+0000 7f279e000640 1 -- 172.21.15.119:0/1833482305 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2798046040 con 0x558afd739830 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.407+0000 7f279e000640 1 -- 172.21.15.119:0/1833482305 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2798045070 con 0x558afd739830 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.407+0000 7f27b7df3740 1 -- 172.21.15.119:0/1833482305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd739830 msgr2=0x558afd74e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.407+0000 7f27b7df3740 1 --2- 172.21.15.119:0/1833482305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd739830 0x558afd74e6d0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f2798007450 tx=0x7f2798030000 comp rx=0 tx=0).stop 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.407+0000 7f27b7df3740 1 -- 172.21.15.119:0/1833482305 shutdown_connections 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.407+0000 7f27b7df3740 1 --2- 172.21.15.119:0/1833482305 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd74b440 0x558afd733ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.407+0000 7f27b7df3740 1 --2- 172.21.15.119:0/1833482305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd739830 0x558afd74e6d0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.407+0000 7f27b7df3740 1 --2- 172.21.15.119:0/1833482305 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd70ed70 0x558afd74e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.407+0000 7f27b7df3740 1 -- 172.21.15.119:0/1833482305 >> 172.21.15.119:0/1833482305 conn(0x558afd6d6a30 msgr2=0x558afd735b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.407+0000 7f27b7df3740 1 -- 172.21.15.119:0/1833482305 shutdown_connections 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.407+0000 7f27b7df3740 1 -- 172.21.15.119:0/1833482305 wait complete. 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.408+0000 7f27b7df3740 1 -- start start 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.408+0000 7f27b7df3740 1 Processor -- start 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.409+0000 7f27b7df3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd70ed70 0x558afd7bbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.409+0000 7f27b7df3740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd739830 0x558afd737120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.409+0000 7f27b7df3740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd74b440 0x558afd7574d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.409+0000 7f27b7df3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b0143d60 con 0x558afd70ed70 2024-04-25T10:17:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.409+0000 7f27b7df3740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558afd7052e0 con 0x558afd74b440 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.409+0000 7f27b7df3740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x558afd74a930 con 0x558afd739830 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.409+0000 7f27b7df3740 10 client.0 _ll_register_callbacks cb 0x7f27a0057ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.409+0000 7f27ade00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd70ed70 0x558afd7bbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.409+0000 7f27ade00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd70ed70 0x558afd7bbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37176/0 (socket says 172.21.15.119:37176) 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.409+0000 7f27ade00640 1 -- 172.21.15.119:0/2753810279 learned_addr learned my addr 172.21.15.119:0/2753810279 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.410+0000 7f27ae800640 1 --2- 172.21.15.119:0/2753810279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd74b440 0x558afd7574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.410+0000 7f27ade00640 1 -- 172.21.15.119:0/2753810279 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd739830 msgr2=0x558afd737120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.410+0000 7f27ade00640 1 --2- 172.21.15.119:0/2753810279 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd739830 0x558afd737120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.410+0000 7f27ade00640 1 -- 172.21.15.119:0/2753810279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd74b440 msgr2=0x558afd7574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.410+0000 7f27ade00640 1 --2- 172.21.15.119:0/2753810279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd74b440 0x558afd7574d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.410+0000 7f27ade00640 1 -- 172.21.15.119:0/2753810279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558afd74c810 con 0x558afd70ed70 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.410+0000 7f27ade00640 1 --2- 172.21.15.119:0/2753810279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd70ed70 0x558afd7bbe40 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x558afd73b0c0 tx=0x7f27a80155a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595ccd9f520d70de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.410+0000 7f2793400640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:17:50.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.410+0000 7f2793400640 1 -- 172.21.15.119:0/2753810279 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a800f040 con 0x558afd70ed70 2024-04-25T10:17:50.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.410+0000 7f2793400640 1 -- 172.21.15.119:0/2753810279 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27a8023480 con 0x558afd70ed70 2024-04-25T10:17:50.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.410+0000 7f27b7df3740 1 -- 172.21.15.119:0/2753810279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2798046040 con 0x558afd70ed70 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.410+0000 7f2793400640 1 -- 172.21.15.119:0/2753810279 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27a8033050 con 0x558afd70ed70 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.410+0000 7f27b7df3740 10 client.8892 fetch_fsmap learned FSMap version 99 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.410+0000 7f27b7df3740 1 -- 172.21.15.119:0/2753810279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x558afd74a1c0 con 0x558afd70ed70 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.411+0000 7f2793400640 1 -- 172.21.15.119:0/2753810279 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2798046040 con 0x558afd70ed70 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.411+0000 7f2793400640 1 -- 172.21.15.119:0/2753810279 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f27a8036020 con 0x558afd70ed70 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.411+0000 7f27b7df3740 10 client.8892 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.411+0000 7f27b7df3740 10 client.8892 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:17:50.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.411+0000 7f27b7df3740 20 client.8892 populate_metadata read hostname 'smithi119' 2024-04-25T10:17:50.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.411+0000 7f27b7df3740 1 --2- 172.21.15.119:0/2753810279 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x558afd757a10 0x558afd7bed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.411+0000 7f27b7df3740 4 client.8892 mds_command: new command op to 8697 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:17:50.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.411+0000 7f27b7df3740 1 -- 172.21.15.119:0/2753810279 --> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558afd67a500 con 0x558afd757a10 2024-04-25T10:17:50.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.411+0000 7f27ad400640 1 --2- 172.21.15.119:0/2753810279 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x558afd757a10 0x558afd7bed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.412+0000 7f27ad400640 1 --2- 172.21.15.119:0/2753810279 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x558afd757a10 0x558afd7bed40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2058fdaf638a8610 server_cookie=d74b3dd2bc664552 in_seq=0 out_seq=0 2024-04-25T10:17:50.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.412+0000 7f2793400640 10 client.8892 ms_handle_connect on v2:172.21.15.119:6836/1849584205 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.425+0000 7f2793400640 1 -- 172.21.15.119:0/2753810279 <== mds.0 v2:172.21.15.119:6836/1849584205 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558afd67a500 con 0x558afd757a10 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.425+0000 7f2793400640 10 client.8892 handle_command_reply: tid=0 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.425+0000 7f27b7df3740 1 client.8892 shutdown 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.425+0000 7f27b7df3740 1 -- 172.21.15.119:0/2753810279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd70ed70 msgr2=0x558afd7bbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.425+0000 7f27b7df3740 1 --2- 172.21.15.119:0/2753810279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd70ed70 0x558afd7bbe40 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x558afd73b0c0 tx=0x7f27a80155a0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.425+0000 7f27b7df3740 1 -- 172.21.15.119:0/2753810279 shutdown_connections 2024-04-25T10:17:50.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.425+0000 7f27b7df3740 1 --2- 172.21.15.119:0/2753810279 >> [v2:172.21.15.119:6836/1849584205,v1:172.21.15.119:6837/1849584205] conn(0x558afd757a10 0x558afd7bed40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.425+0000 7f27b7df3740 1 --2- 172.21.15.119:0/2753810279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558afd74b440 0x558afd7574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.425+0000 7f27b7df3740 1 --2- 172.21.15.119:0/2753810279 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x558afd739830 0x558afd737120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.425+0000 7f27b7df3740 1 --2- 172.21.15.119:0/2753810279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558afd70ed70 0x558afd7bbe40 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.425+0000 7f2793400640 0 client.8892 ms_handle_reset on v2:172.21.15.119:6836/1849584205 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.425+0000 7f27b7df3740 1 -- 172.21.15.119:0/2753810279 >> 172.21.15.119:0/2753810279 conn(0x558afd6d6a30 msgr2=0x558afd74e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.426+0000 7f27b7df3740 1 -- 172.21.15.119:0/2753810279 shutdown_connections 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.426+0000 7f27b7df3740 1 -- 172.21.15.119:0/2753810279 wait complete. 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.426+0000 7f27b7df3740 20 client.8892 trim_cache size 0 max 16384 2024-04-25T10:17:50.430 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:17:50.431 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "", 2024-04-25T10:17:50.431 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0 2024-04-25T10:17:50.431 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:17:50.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.426+0000 7f27af200640 1 -- 172.21.15.119:0/419601719 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f278c06b4d0 msgr2=0x7f278c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.426+0000 7f27af200640 1 --2- 172.21.15.119:0/419601719 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f278c06b4d0 0x7f278c06d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f27a8006a90 tx=0x7f27a8008040 comp rx=0 tx=0).stop 2024-04-25T10:17:50.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.426+0000 7f27af200640 1 -- 172.21.15.119:0/419601719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b0141630 msgr2=0x7f27b0155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.426+0000 7f27af200640 1 --2- 172.21.15.119:0/419601719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b0141630 0x7f27b0155030 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f27b00ae0f0 tx=0x7f27a000dc00 comp rx=0 tx=0).stop 2024-04-25T10:17:50.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.427+0000 7f27af200640 1 -- 172.21.15.119:0/419601719 shutdown_connections 2024-04-25T10:17:50.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.427+0000 7f27af200640 1 --2- 172.21.15.119:0/419601719 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f278c06b4d0 0x7f278c06d990 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.427+0000 7f27af200640 1 --2- 172.21.15.119:0/419601719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b0141630 0x7f27b0155030 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.427+0000 7f27af200640 1 --2- 172.21.15.119:0/419601719 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27b01408f0 0x7f27b01410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.427+0000 7f27af200640 1 --2- 172.21.15.119:0/419601719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2794003fe0 0x7f27b01572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.427+0000 7f27af200640 1 -- 172.21.15.119:0/419601719 >> 172.21.15.119:0/419601719 conn(0x7f27b00b59c0 msgr2=0x7f27b00b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.427+0000 7f27af200640 1 -- 172.21.15.119:0/419601719 shutdown_connections 2024-04-25T10:17:50.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.427+0000 7f27af200640 1 -- 172.21.15.119:0/419601719 wait complete. 2024-04-25T10:17:50.447 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:17:50.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.632+0000 7efce0800640 1 Processor -- start 2024-04-25T10:17:50.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.632+0000 7efce0800640 1 -- start start 2024-04-25T10:17:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.634+0000 7efce0800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efcd815dce0 0x7efcd80d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.634+0000 7efce0800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efcd80b5b60 0x7efcd80b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.634+0000 7efce0800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd815e0c0 con 0x7efcd80d35b0 2024-04-25T10:17:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.634+0000 7efce0800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd80d2510 con 0x7efcd815dce0 2024-04-25T10:17:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.634+0000 7efce0800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd80d27f0 con 0x7efcd80b5b60 2024-04-25T10:17:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.634+0000 7efcdde00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efcd815dce0 0x7efcd80d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.634+0000 7efcdde00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efcd815dce0 0x7efcd80d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37180/0 (socket says 172.21.15.119:37180) 2024-04-25T10:17:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.634+0000 7efcdde00640 1 -- 172.21.15.119:0/3744348963 learned_addr learned my addr 172.21.15.119:0/3744348963 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.635+0000 7efcde800640 1 --2- 172.21.15.119:0/3744348963 >> v2:172.21.15.132:3300/0 conn(0x7efcd80b5b60 0x7efcd80b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.635+0000 7efcdde00640 1 -- 172.21.15.119:0/3744348963 >> v2:172.21.15.132:3300/0 conn(0x7efcd80b5b60 msgr2=0x7efcd80b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.635+0000 7efcdde00640 1 --2- 172.21.15.119:0/3744348963 >> v2:172.21.15.132:3300/0 conn(0x7efcd80b5b60 0x7efcd80b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.635+0000 7efcdde00640 1 -- 172.21.15.119:0/3744348963 >> v1:172.21.15.119:6789/0 conn(0x7efcd80d35b0 legacy=0x7efcd80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.635+0000 7efcdde00640 1 -- 172.21.15.119:0/3744348963 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd80d2ad0 con 0x7efcd815dce0 2024-04-25T10:17:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.635+0000 7efcde800640 1 --2- 172.21.15.119:0/3744348963 >> v2:172.21.15.132:3300/0 conn(0x7efcd80b5b60 0x7efcd80b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.636+0000 7efcdde00640 1 --2- 172.21.15.119:0/3744348963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcd815dce0 0x7efcd80d1a50 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7efcd80b3390 tx=0x7efcd002a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fe92670084625b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.636+0000 7efcdca00640 1 -- 172.21.15.119:0/3744348963 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0036070 con 0x7efcd815dce0 2024-04-25T10:17:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.636+0000 7efcdca00640 1 -- 172.21.15.119:0/3744348963 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd0031600 con 0x7efcd815dce0 2024-04-25T10:17:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.636+0000 7efcdca00640 1 -- 172.21.15.119:0/3744348963 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd00319d0 con 0x7efcd815dce0 2024-04-25T10:17:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.637+0000 7efce0800640 1 -- 172.21.15.119:0/3744348963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcd815dce0 msgr2=0x7efcd80d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.637+0000 7efce0800640 1 --2- 172.21.15.119:0/3744348963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcd815dce0 0x7efcd80d1a50 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7efcd80b3390 tx=0x7efcd002a740 comp rx=0 tx=0).stop 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.637+0000 7efce0800640 1 -- 172.21.15.119:0/3744348963 shutdown_connections 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.637+0000 7efce0800640 1 --2- 172.21.15.119:0/3744348963 >> v2:172.21.15.132:3300/0 conn(0x7efcd80b5b60 0x7efcd80b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.637+0000 7efce0800640 1 --2- 172.21.15.119:0/3744348963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcd815dce0 0x7efcd80d1a50 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.637+0000 7efce0800640 1 -- 172.21.15.119:0/3744348963 >> 172.21.15.119:0/3744348963 conn(0x7efcd8072900 msgr2=0x7efcd80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.637+0000 7efce0800640 1 -- 172.21.15.119:0/3744348963 shutdown_connections 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.637+0000 7efce0800640 1 -- 172.21.15.119:0/3744348963 wait complete. 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.639+0000 7efce0800640 1 Processor -- start 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.639+0000 7efce0800640 1 -- start start 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.640+0000 7efce0800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcd80b5b60 0x7efcd815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.640+0000 7efce0800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcd80d35b0 0x7efcd80c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.640+0000 7efce0800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcd815dce0 0x7efcd80ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.640+0000 7efce0800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd80b9820 con 0x7efcd80d35b0 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.640+0000 7efce0800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd80d3090 con 0x7efcd80b5b60 2024-04-25T10:17:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.640+0000 7efce0800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd80b99a0 con 0x7efcd815dce0 2024-04-25T10:17:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.640+0000 7efcdd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcd80d35b0 0x7efcd80c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.640+0000 7efcdd400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcd80d35b0 0x7efcd80c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37186/0 (socket says 172.21.15.119:37186) 2024-04-25T10:17:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.640+0000 7efcdd400640 1 -- 172.21.15.119:0/663607441 learned_addr learned my addr 172.21.15.119:0/663607441 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.641+0000 7efcde800640 1 --2- 172.21.15.119:0/663607441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcd815dce0 0x7efcd80ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.641+0000 7efcdd400640 1 -- 172.21.15.119:0/663607441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcd815dce0 msgr2=0x7efcd80ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.641+0000 7efcdde00640 1 --2- 172.21.15.119:0/663607441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcd80b5b60 0x7efcd815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.641+0000 7efcdd400640 1 --2- 172.21.15.119:0/663607441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcd815dce0 0x7efcd80ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.641+0000 7efcdd400640 1 -- 172.21.15.119:0/663607441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcd80b5b60 msgr2=0x7efcd815e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.641+0000 7efcdd400640 1 --2- 172.21.15.119:0/663607441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcd80b5b60 0x7efcd815e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.641+0000 7efcdd400640 1 -- 172.21.15.119:0/663607441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd8160770 con 0x7efcd80d35b0 2024-04-25T10:17:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.641+0000 7efcdd400640 1 --2- 172.21.15.119:0/663607441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcd80d35b0 0x7efcd80c8860 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7efcc800c470 tx=0x7efcc8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.641+0000 7efcce000640 1 -- 172.21.15.119:0/663607441 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc80052d0 con 0x7efcd80d35b0 2024-04-25T10:17:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.641+0000 7efcce000640 1 -- 172.21.15.119:0/663607441 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc8005470 con 0x7efcd80d35b0 2024-04-25T10:17:50.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.641+0000 7efce0800640 1 -- 172.21.15.119:0/663607441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd815e7f0 con 0x7efcd80d35b0 2024-04-25T10:17:50.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.641+0000 7efcce000640 1 -- 172.21.15.119:0/663607441 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8007260 con 0x7efcd80d35b0 2024-04-25T10:17:50.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.641+0000 7efce0800640 1 -- 172.21.15.119:0/663607441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd8160cc0 con 0x7efcd80d35b0 2024-04-25T10:17:50.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.643+0000 7efce0800640 1 -- 172.21.15.119:0/663607441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd80749b0 con 0x7efcd80d35b0 2024-04-25T10:17:50.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.643+0000 7efcce000640 1 -- 172.21.15.119:0/663607441 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efcc8007400 con 0x7efcd80d35b0 2024-04-25T10:17:50.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.645+0000 7efcce000640 1 --2- 172.21.15.119:0/663607441 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efcbc06b4d0 0x7efcbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.646+0000 7efcce000640 1 -- 172.21.15.119:0/663607441 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efcc8089710 con 0x7efcd80d35b0 2024-04-25T10:17:50.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.646+0000 7efcdde00640 1 --2- 172.21.15.119:0/663607441 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efcbc06b4d0 0x7efcbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.646+0000 7efcdde00640 1 --2- 172.21.15.119:0/663607441 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efcbc06b4d0 0x7efcbc06d990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7efcd0033000 tx=0x7efcd0009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.650+0000 7efcce000640 1 -- 172.21.15.119:0/663607441 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efcc8058580 con 0x7efcd80d35b0 2024-04-25T10:17:50.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.885+0000 7efce0800640 1 -- 172.21.15.119:0/663607441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efcd80b5110 con 0x7efcd80d35b0 2024-04-25T10:17:50.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.886+0000 7efcce000640 1 -- 172.21.15.119:0/663607441 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+73 (secure 0 0 0) 0x7efcc8057e80 con 0x7efcd80d35b0 2024-04-25T10:17:50.889 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:17:50.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.888+0000 7efce0800640 1 -- 172.21.15.119:0/663607441 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efcbc06b4d0 msgr2=0x7efcbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.888+0000 7efce0800640 1 --2- 172.21.15.119:0/663607441 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efcbc06b4d0 0x7efcbc06d990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7efcd0033000 tx=0x7efcd0009000 comp rx=0 tx=0).stop 2024-04-25T10:17:50.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.888+0000 7efce0800640 1 -- 172.21.15.119:0/663607441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcd80d35b0 msgr2=0x7efcd80c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.888+0000 7efce0800640 1 --2- 172.21.15.119:0/663607441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcd80d35b0 0x7efcd80c8860 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7efcc800c470 tx=0x7efcc8001e70 comp rx=0 tx=0).stop 2024-04-25T10:17:50.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.888+0000 7efce0800640 1 -- 172.21.15.119:0/663607441 shutdown_connections 2024-04-25T10:17:50.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.888+0000 7efce0800640 1 --2- 172.21.15.119:0/663607441 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efcbc06b4d0 0x7efcbc06d990 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.888+0000 7efce0800640 1 --2- 172.21.15.119:0/663607441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcd815dce0 0x7efcd80ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.888+0000 7efce0800640 1 --2- 172.21.15.119:0/663607441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcd80d35b0 0x7efcd80c8860 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.889+0000 7efce0800640 1 --2- 172.21.15.119:0/663607441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcd80b5b60 0x7efcd815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.889+0000 7efce0800640 1 -- 172.21.15.119:0/663607441 >> 172.21.15.119:0/663607441 conn(0x7efcd8072900 msgr2=0x7efcd80b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.889+0000 7efce0800640 1 -- 172.21.15.119:0/663607441 shutdown_connections 2024-04-25T10:17:50.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:50.889+0000 7efce0800640 1 -- 172.21.15.119:0/663607441 wait complete. 2024-04-25T10:17:50.910 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:17:50.910 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:17:50.910 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:17:50.910 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:17:50.910 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:50.910 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:17:50.910 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:17:50.910 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:17:50.910 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:17:50.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:50.911 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:17:50.959 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:17:50.959 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:17:50.959 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:17:50.959 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:50.960 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:17:50.960 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:50.960 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:17:50.960 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:50.960 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:17:50.960 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:17:50.960 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:50.960 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:17:50.960 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:17:50.960 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:50.960 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:17:50.960 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:50.960 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:17:50.961 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42018sec preferred_lft 42018sec 2024-04-25T10:17:50.961 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:17:50.961 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:50.961 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:17:50.961 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:17:50.961 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:17:50.961 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:50.961 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:17:50.961 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:50.961 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:17:50.961 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:17:50.961 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:17:50.961 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:50.961 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:17:50.962 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:17:50.962 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:17:50.962 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:17:50.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:50.962 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:17:51.035 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:17:51.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:51.039 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:17:51.087 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:17:51.087 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:17:51.088 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:17:51.088 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:17:51.088 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:17:51.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:51.088 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:17:51.088 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:17:51.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:17:51.089 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:17:51.089 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:17:51.089 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:17:51.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:51.283 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:17:51.332 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:17:51.332 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:17:51.333 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:17:51.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:51.334 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:17:51.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:17:51.409 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:51.460 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:17:51.460 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:17:51.460 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:51.512 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:17:51.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:51.513 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:17:51.645 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:17:51.645 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:17:51.645 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:17:51.666 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:17:51.666 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:17:51.666 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:17:51.666 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:17:51.666 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:17:51.714 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:17:51.714 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:17:51.714 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:17:51.714 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:17:51.714 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:17:51.714 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:17:51.714 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:17:51.714 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:17:51.719 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:17:51.719 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:17:51.719 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:17:51.719 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:17:51.720 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:17:51.724 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:17:51.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:51.725 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:51.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:51.803 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:17:51.857 INFO:teuthology.orchestra.run.smithi119.stdout:client6331 2024-04-25T10:17:51.857 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:17:51.858 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:51.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:51.909 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-25T10:17:51.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:51.969 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-25T10:17:52.788 INFO:teuthology.orchestra.run.smithi119.stderr:8+0 records in 2024-04-25T10:17:52.788 INFO:teuthology.orchestra.run.smithi119.stderr:8+0 records out 2024-04-25T10:17:52.788 INFO:teuthology.orchestra.run.smithi119.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.765226 s, 11.0 MB/s 2024-04-25T10:17:52.790 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:17:52.790 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:17:52.790 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:17:52.790 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:17:52.790 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:17:52.790 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:17:52.924 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627781 2024-04-25T10:17:52.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:17:52.938 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:17:52.938 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:17:52.938 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:17:52.938 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:17:52.938 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:17:52.938 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:17:53.122 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627782 2024-04-25T10:17:53.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:17:53.135 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:17:53.135 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:53.232 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:17:53.233 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:53.282 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:53.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.527+0000 7f148ca00640 1 Processor -- start 2024-04-25T10:17:53.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.527+0000 7f148ca00640 1 -- start start 2024-04-25T10:17:53.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.529+0000 7f148ca00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f14880b9270 0x7f14880d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.529+0000 7f148ca00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f148815ef10 con 0x7f148815db60 2024-04-25T10:17:53.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.529+0000 7f148ca00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f148815e390 con 0x7f14880d3f80 2024-04-25T10:17:53.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.529+0000 7f148ca00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148815e670 con 0x7f14880b9270 2024-04-25T10:17:53.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.529+0000 7f1487400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f148815db60 0x7f14880d3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50494/0 (socket says 172.21.15.119:50494) 2024-04-25T10:17:53.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.529+0000 7f1487400640 1 -- 172.21.15.119:0/129617638 learned_addr learned my addr 172.21.15.119:0/129617638 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:53.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.529+0000 7f1486a00640 1 -- 172.21.15.119:0/129617638 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720011260 0 0) 0x7f148815ef10 con 0x7f148815db60 2024-04-25T10:17:53.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.529+0000 7f1486a00640 1 -- 172.21.15.119:0/129617638 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f146c003520 con 0x7f148815db60 2024-04-25T10:17:53.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1487e00640 1 --2- 172.21.15.119:0/129617638 >> v2:172.21.15.132:3300/0 conn(0x7f14880b9270 0x7f14880d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1486a00640 1 -- 172.21.15.119:0/129617638 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 44449353 0 0) 0x7f146c003520 con 0x7f148815db60 2024-04-25T10:17:53.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1486a00640 1 -- 172.21.15.119:0/129617638 >> v2:172.21.15.132:3300/0 conn(0x7f14880b9270 msgr2=0x7f14880d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:53.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1486a00640 1 --2- 172.21.15.119:0/129617638 >> v2:172.21.15.132:3300/0 conn(0x7f14880b9270 0x7f14880d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1486a00640 1 -- 172.21.15.119:0/129617638 >> v1:172.21.15.132:6789/0 conn(0x7f14880d3f80 legacy=0x7f14880d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:53.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1486a00640 1 -- 172.21.15.119:0/129617638 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148815e950 con 0x7f148815db60 2024-04-25T10:17:53.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1486a00640 1 -- 172.21.15.119:0/129617638 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f1478001670 con 0x7f148815db60 2024-04-25T10:17:53.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1486a00640 1 -- 172.21.15.119:0/129617638 >> v1:172.21.15.119:6789/0 conn(0x7f148815db60 legacy=0x7f14880d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1486a00640 1 --2- 172.21.15.119:0/129617638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f146c003a20 0x7f146c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1486a00640 1 --2- 172.21.15.119:0/129617638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f146c004800 0x7f146c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1486a00640 1 --2- 172.21.15.119:0/129617638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f146c005210 0x7f146c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1486a00640 1 -- 172.21.15.119:0/129617638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148815e670 con 0x7f146c003a20 2024-04-25T10:17:53.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1486a00640 1 -- 172.21.15.119:0/129617638 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c00c160 con 0x7f146c004800 2024-04-25T10:17:53.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1486a00640 1 -- 172.21.15.119:0/129617638 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c00c360 con 0x7f146c005210 2024-04-25T10:17:53.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.530+0000 7f1487400640 1 --2- 172.21.15.119:0/129617638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f146c003a20 0x7f146c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.531+0000 7f147fe00640 1 --2- 172.21.15.119:0/129617638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f146c004800 0x7f146c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.531+0000 7f1487400640 1 -- 172.21.15.119:0/129617638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f146c005210 msgr2=0x7f146c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:53.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.531+0000 7f1487400640 1 --2- 172.21.15.119:0/129617638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f146c005210 0x7f146c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.531+0000 7f1487400640 1 -- 172.21.15.119:0/129617638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f146c004800 msgr2=0x7f146c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:53.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.531+0000 7f1487400640 1 --2- 172.21.15.119:0/129617638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f146c004800 0x7f146c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.531+0000 7f1487400640 1 -- 172.21.15.119:0/129617638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c003520 con 0x7f146c003a20 2024-04-25T10:17:53.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.531+0000 7f1487e00640 1 -- 172.21.15.119:0/129617638 reap_dead start 2024-04-25T10:17:53.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.532+0000 7f1487400640 1 --2- 172.21.15.119:0/129617638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f146c003a20 0x7f146c003e20 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f147800c240 tx=0x7f146c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4193a7ed839e5d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:53.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.532+0000 7f1486a00640 1 -- 172.21.15.119:0/129617638 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478007d10 con 0x7f146c003a20 2024-04-25T10:17:53.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.532+0000 7f1486a00640 1 -- 172.21.15.119:0/129617638 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1478005ad0 con 0x7f146c003a20 2024-04-25T10:17:53.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.532+0000 7f1486a00640 1 -- 172.21.15.119:0/129617638 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478007070 con 0x7f146c003a20 2024-04-25T10:17:53.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.533+0000 7f148ca00640 1 -- 172.21.15.119:0/129617638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f146c003a20 msgr2=0x7f146c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.533+0000 7f148ca00640 1 --2- 172.21.15.119:0/129617638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f146c003a20 0x7f146c003e20 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f147800c240 tx=0x7f146c005830 comp rx=0 tx=0).stop 2024-04-25T10:17:53.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.533+0000 7f148ca00640 1 -- 172.21.15.119:0/129617638 shutdown_connections 2024-04-25T10:17:53.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.533+0000 7f148ca00640 1 --2- 172.21.15.119:0/129617638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f146c003a20 0x7f146c003e20 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.533+0000 7f148ca00640 1 -- 172.21.15.119:0/129617638 >> 172.21.15.119:0/129617638 conn(0x7f14880726c0 msgr2=0x7f1488072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:53.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.533+0000 7f148ca00640 1 -- 172.21.15.119:0/129617638 shutdown_connections 2024-04-25T10:17:53.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.533+0000 7f148ca00640 1 -- 172.21.15.119:0/129617638 wait complete. 2024-04-25T10:17:53.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.535+0000 7f148ca00640 1 Processor -- start 2024-04-25T10:17:53.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.535+0000 7f148ca00640 1 -- start start 2024-04-25T10:17:53.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.536+0000 7f148ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f146c003a20 0x7f148814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.536+0000 7f148ca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1488160890 0x7f1488160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.536+0000 7f148ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881f6e80 0x7f14881f7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.536+0000 7f148ca00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14880b3120 con 0x7f14881f6e80 2024-04-25T10:17:53.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.536+0000 7f148ca00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14880b2fa0 con 0x7f1488160890 2024-04-25T10:17:53.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.536+0000 7f148ca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148815df40 con 0x7f146c003a20 2024-04-25T10:17:53.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.536+0000 7f1487e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881f6e80 0x7f14881f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.536+0000 7f1487e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881f6e80 0x7f14881f7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46470/0 (socket says 172.21.15.119:46470) 2024-04-25T10:17:53.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.536+0000 7f1487e00640 1 -- 172.21.15.119:0/3317163603 learned_addr learned my addr 172.21.15.119:0/3317163603 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:53.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.536+0000 7f1487e00640 1 -- 172.21.15.119:0/3317163603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f146c003a20 msgr2=0x7f148814cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.536+0000 7f1487e00640 1 --2- 172.21.15.119:0/3317163603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f146c003a20 0x7f148814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.536+0000 7f1487e00640 1 -- 172.21.15.119:0/3317163603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1488160890 msgr2=0x7f1488160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.536+0000 7f1487e00640 1 --2- 172.21.15.119:0/3317163603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1488160890 0x7f1488160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.536+0000 7f1487e00640 1 -- 172.21.15.119:0/3317163603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14880cabc0 con 0x7f14881f6e80 2024-04-25T10:17:53.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.537+0000 7f1487e00640 1 --2- 172.21.15.119:0/3317163603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881f6e80 0x7f14881f7300 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f148808cbf0 tx=0x7f148000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:53.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.537+0000 7f147f400640 1 -- 172.21.15.119:0/3317163603 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480007e40 con 0x7f14881f6e80 2024-04-25T10:17:53.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.537+0000 7f147f400640 1 -- 172.21.15.119:0/3317163603 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1480019070 con 0x7f14881f6e80 2024-04-25T10:17:53.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.537+0000 7f147f400640 1 -- 172.21.15.119:0/3317163603 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480014640 con 0x7f14881f6e80 2024-04-25T10:17:53.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.537+0000 7f148ca00640 1 -- 172.21.15.119:0/3317163603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14880caea0 con 0x7f14881f6e80 2024-04-25T10:17:53.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.537+0000 7f148ca00640 1 -- 172.21.15.119:0/3317163603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14881f7840 con 0x7f14881f6e80 2024-04-25T10:17:53.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.538+0000 7f148ca00640 1 -- 172.21.15.119:0/3317163603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14880749b0 con 0x7f14881f6e80 2024-04-25T10:17:53.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.539+0000 7f147f400640 1 -- 172.21.15.119:0/3317163603 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1480005d10 con 0x7f14881f6e80 2024-04-25T10:17:53.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.540+0000 7f147f400640 1 --2- 172.21.15.119:0/3317163603 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f146406b500 0x7f146406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.540+0000 7f147f400640 1 -- 172.21.15.119:0/3317163603 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f148008a440 con 0x7f14881f6e80 2024-04-25T10:17:53.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.540+0000 7f1487400640 1 --2- 172.21.15.119:0/3317163603 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f146406b500 0x7f146406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.541+0000 7f1487400640 1 --2- 172.21.15.119:0/3317163603 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f146406b500 0x7f146406d9c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1478033000 tx=0x7f1478007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:53.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.545+0000 7f147f400640 1 -- 172.21.15.119:0/3317163603 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1480023020 con 0x7f14881f6e80 2024-04-25T10:17:53.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.741+0000 7f148ca00640 1 -- 172.21.15.119:0/3317163603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14880d1a60 con 0x7f14881f6e80 2024-04-25T10:17:53.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.743+0000 7f147f400640 1 -- 172.21.15.119:0/3317163603 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16816 (secure 0 0 0) 0x7f1480059280 con 0x7f14881f6e80 2024-04-25T10:17:53.747 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:53.747 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":110,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:17:52.677536+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:35.422922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:36.634584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.25","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:53.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.747+0000 7f148ca00640 1 -- 172.21.15.119:0/3317163603 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f146406b500 msgr2=0x7f146406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.747+0000 7f148ca00640 1 --2- 172.21.15.119:0/3317163603 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f146406b500 0x7f146406d9c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1478033000 tx=0x7f1478007000 comp rx=0 tx=0).stop 2024-04-25T10:17:53.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.747+0000 7f148ca00640 1 -- 172.21.15.119:0/3317163603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881f6e80 msgr2=0x7f14881f7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.747+0000 7f148ca00640 1 --2- 172.21.15.119:0/3317163603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881f6e80 0x7f14881f7300 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f148808cbf0 tx=0x7f148000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.747+0000 7f148ca00640 1 -- 172.21.15.119:0/3317163603 shutdown_connections 2024-04-25T10:17:53.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.747+0000 7f148ca00640 1 --2- 172.21.15.119:0/3317163603 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f146406b500 0x7f146406d9c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.747+0000 7f148ca00640 1 --2- 172.21.15.119:0/3317163603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14881f6e80 0x7f14881f7300 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.747+0000 7f148ca00640 1 --2- 172.21.15.119:0/3317163603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1488160890 0x7f1488160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.747+0000 7f148ca00640 1 --2- 172.21.15.119:0/3317163603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f146c003a20 0x7f148814cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.747+0000 7f148ca00640 1 -- 172.21.15.119:0/3317163603 >> 172.21.15.119:0/3317163603 conn(0x7f14880726c0 msgr2=0x7f14880b7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:53.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.747+0000 7f148ca00640 1 -- 172.21.15.119:0/3317163603 shutdown_connections 2024-04-25T10:17:53.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.747+0000 7f148ca00640 1 -- 172.21.15.119:0/3317163603 wait complete. 2024-04-25T10:17:53.771 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:35.422922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:17:36.634584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:53.771 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.969+0000 7f89d2400640 1 Processor -- start 2024-04-25T10:17:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.969+0000 7f89d2400640 1 -- start start 2024-04-25T10:17:53.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.971+0000 7f89d2400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89cc06b2b0 0x7f89cc0acb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.971+0000 7f89d2400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89cc15e120 con 0x7f89cc0b2f90 2024-04-25T10:17:53.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.971+0000 7f89d2400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89cc15e400 con 0x7f89cc0ada20 2024-04-25T10:17:53.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.971+0000 7f89d2400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc15dec0 con 0x7f89cc06b2b0 2024-04-25T10:17:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.971+0000 7f89d1a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f89cc0b2f90 0x7f89cc0b3400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50506/0 (socket says 172.21.15.119:50506) 2024-04-25T10:17:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.971+0000 7f89d1a00640 1 -- 172.21.15.119:0/4143090442 learned_addr learned my addr 172.21.15.119:0/4143090442 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.971+0000 7f89cbe00640 1 --2- 172.21.15.119:0/4143090442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89cc06b2b0 0x7f89cc0acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 -- 172.21.15.119:0/4143090442 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2444765280 0 0) 0x7f89cc15e120 con 0x7f89cc0b2f90 2024-04-25T10:17:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 -- 172.21.15.119:0/4143090442 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89b4003520 con 0x7f89cc0b2f90 2024-04-25T10:17:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 -- 172.21.15.119:0/4143090442 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4176656967 0 0) 0x7f89b4003520 con 0x7f89cc0b2f90 2024-04-25T10:17:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 -- 172.21.15.119:0/4143090442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89cc06b2b0 msgr2=0x7f89cc0acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 --2- 172.21.15.119:0/4143090442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89cc06b2b0 0x7f89cc0acb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 -- 172.21.15.119:0/4143090442 >> v1:172.21.15.132:6789/0 conn(0x7f89cc0ada20 legacy=0x7f89cc06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 -- 172.21.15.119:0/4143090442 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc15e6e0 con 0x7f89cc0b2f90 2024-04-25T10:17:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 -- 172.21.15.119:0/4143090442 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f89c40019f0 con 0x7f89cc0b2f90 2024-04-25T10:17:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 -- 172.21.15.119:0/4143090442 >> v1:172.21.15.119:6789/0 conn(0x7f89cc0b2f90 legacy=0x7f89cc0b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 --2- 172.21.15.119:0/4143090442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89b40039e0 0x7f89b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 --2- 172.21.15.119:0/4143090442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89cc06b2b0 0x7f89b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 --2- 172.21.15.119:0/4143090442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89b4004d40 0x7f89b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 -- 172.21.15.119:0/4143090442 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc15dec0 con 0x7f89b4004d40 2024-04-25T10:17:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 -- 172.21.15.119:0/4143090442 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b400ba70 con 0x7f89b40039e0 2024-04-25T10:17:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.972+0000 7f89cb400640 1 -- 172.21.15.119:0/4143090442 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b400bc70 con 0x7f89cc06b2b0 2024-04-25T10:17:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.973+0000 7f89d1a00640 1 --2- 172.21.15.119:0/4143090442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89b4004d40 0x7f89b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.973+0000 7f89d1000640 1 --2- 172.21.15.119:0/4143090442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89b40039e0 0x7f89b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.973+0000 7f89d1a00640 1 -- 172.21.15.119:0/4143090442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89cc06b2b0 msgr2=0x7f89b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.973+0000 7f89cbe00640 1 --2- 172.21.15.119:0/4143090442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89cc06b2b0 0x7f89b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.973+0000 7f89d1a00640 1 --2- 172.21.15.119:0/4143090442 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89cc06b2b0 0x7f89b4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.973+0000 7f89d1a00640 1 -- 172.21.15.119:0/4143090442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89b40039e0 msgr2=0x7f89b4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:53.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.973+0000 7f89d1a00640 1 --2- 172.21.15.119:0/4143090442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89b40039e0 0x7f89b4003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.973+0000 7f89d1a00640 1 -- 172.21.15.119:0/4143090442 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b4003520 con 0x7f89b4004d40 2024-04-25T10:17:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.974+0000 7f89d1a00640 1 --2- 172.21.15.119:0/4143090442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89b4004d40 0x7f89b40051c0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f89c400a830 tx=0x7f89b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5bb4fbbe210fba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.974+0000 7f89cb400640 1 -- 172.21.15.119:0/4143090442 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c400a480 con 0x7f89b4004d40 2024-04-25T10:17:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.974+0000 7f89cb400640 1 -- 172.21.15.119:0/4143090442 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c402f070 con 0x7f89b4004d40 2024-04-25T10:17:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.974+0000 7f89cb400640 1 -- 172.21.15.119:0/4143090442 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c402bcd0 con 0x7f89b4004d40 2024-04-25T10:17:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.975+0000 7f89d2400640 1 -- 172.21.15.119:0/4143090442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89b4004d40 msgr2=0x7f89b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.975+0000 7f89d2400640 1 --2- 172.21.15.119:0/4143090442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89b4004d40 0x7f89b40051c0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f89c400a830 tx=0x7f89b4005830 comp rx=0 tx=0).stop 2024-04-25T10:17:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.975+0000 7f89d1a00640 1 -- 172.21.15.119:0/4143090442 reap_dead start 2024-04-25T10:17:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.975+0000 7f89d2400640 1 -- 172.21.15.119:0/4143090442 shutdown_connections 2024-04-25T10:17:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.975+0000 7f89d2400640 1 -- 172.21.15.119:0/4143090442 >> 172.21.15.119:0/4143090442 conn(0x7f89cc066ff0 msgr2=0x7f89cc0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.975+0000 7f89d2400640 1 -- 172.21.15.119:0/4143090442 shutdown_connections 2024-04-25T10:17:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.975+0000 7f89d2400640 1 -- 172.21.15.119:0/4143090442 wait complete. 2024-04-25T10:17:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.977+0000 7f89d2400640 1 Processor -- start 2024-04-25T10:17:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.977+0000 7f89d2400640 1 -- start start 2024-04-25T10:17:53.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.978+0000 7f89d2400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89b4004d40 0x7f89cc147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.978+0000 7f89d2400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89cc152b00 0x7f89cc152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.978+0000 7f89d2400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc0cf2b0 0x7f89cc0cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.978+0000 7f89d2400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc0ade00 con 0x7f89cc0cf2b0 2024-04-25T10:17:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.978+0000 7f89d2400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc15eca0 con 0x7f89b4004d40 2024-04-25T10:17:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.978+0000 7f89d2400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc0acb20 con 0x7f89cc152b00 2024-04-25T10:17:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.978+0000 7f89d1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc0cf2b0 0x7f89cc0cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.978+0000 7f89d1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc0cf2b0 0x7f89cc0cf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46480/0 (socket says 172.21.15.119:46480) 2024-04-25T10:17:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.978+0000 7f89d1a00640 1 -- 172.21.15.119:0/4244485961 learned_addr learned my addr 172.21.15.119:0/4244485961 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.979+0000 7f89d1000640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89b4004d40 0x7f89cc147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.979+0000 7f89cbe00640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89cc152b00 0x7f89cc152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.979+0000 7f89d1a00640 1 -- 172.21.15.119:0/4244485961 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89cc152b00 msgr2=0x7f89cc152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.979+0000 7f89d1a00640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89cc152b00 0x7f89cc152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.979+0000 7f89d1a00640 1 -- 172.21.15.119:0/4244485961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89b4004d40 msgr2=0x7f89cc147dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.979+0000 7f89d1a00640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89b4004d40 0x7f89cc147dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.979+0000 7f89d1a00640 1 -- 172.21.15.119:0/4244485961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc148300 con 0x7f89cc0cf2b0 2024-04-25T10:17:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.979+0000 7f89d1000640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89b4004d40 0x7f89cc147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.979+0000 7f89d1a00640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc0cf2b0 0x7f89cc0cf730 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f89c400a950 tx=0x7f89cc06dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.979+0000 7f89c8c00640 1 -- 172.21.15.119:0/4244485961 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c400a480 con 0x7f89cc0cf2b0 2024-04-25T10:17:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.979+0000 7f89c8c00640 1 -- 172.21.15.119:0/4244485961 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c4004b60 con 0x7f89cc0cf2b0 2024-04-25T10:17:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.979+0000 7f89c8c00640 1 -- 172.21.15.119:0/4244485961 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4034470 con 0x7f89cc0cf2b0 2024-04-25T10:17:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.979+0000 7f89d2400640 1 -- 172.21.15.119:0/4244485961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89cc153670 con 0x7f89cc0cf2b0 2024-04-25T10:17:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.980+0000 7f89d2400640 1 -- 172.21.15.119:0/4244485961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc153c30 con 0x7f89cc0cf2b0 2024-04-25T10:17:53.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.981+0000 7f89d2400640 1 -- 172.21.15.119:0/4244485961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89cc0749b0 con 0x7f89cc0cf2b0 2024-04-25T10:17:53.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.982+0000 7f89c8c00640 1 -- 172.21.15.119:0/4244485961 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89c4034610 con 0x7f89cc0cf2b0 2024-04-25T10:17:53.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.983+0000 7f89c8c00640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f89ac06b4d0 0x7f89ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.983+0000 7f89c8c00640 1 -- 172.21.15.119:0/4244485961 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f89c40aa440 con 0x7f89cc0cf2b0 2024-04-25T10:17:53.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.983+0000 7f89d1000640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f89ac06b4d0 0x7f89ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.984+0000 7f89d1000640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f89ac06b4d0 0x7f89ac06d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f89c00016f0 tx=0x7f89c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:53.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:53.987+0000 7f89c8c00640 1 -- 172.21.15.119:0/4244485961 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89c40791e0 con 0x7f89cc0cf2b0 2024-04-25T10:17:54.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.183+0000 7f89d2400640 1 -- 172.21.15.119:0/4244485961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89cc06d7c0 con 0x7f89cc0cf2b0 2024-04-25T10:17:54.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.184+0000 7f89c8c00640 1 -- 172.21.15.119:0/4244485961 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f89c4078ae0 con 0x7f89cc0cf2b0 2024-04-25T10:17:54.188 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:17:54.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.187+0000 7f89d2400640 1 -- 172.21.15.119:0/4244485961 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f89ac06b4d0 msgr2=0x7f89ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.187+0000 7f89d2400640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f89ac06b4d0 0x7f89ac06d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f89c00016f0 tx=0x7f89c0009040 comp rx=0 tx=0).stop 2024-04-25T10:17:54.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.187+0000 7f89d2400640 1 -- 172.21.15.119:0/4244485961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc0cf2b0 msgr2=0x7f89cc0cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.187+0000 7f89d2400640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc0cf2b0 0x7f89cc0cf730 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f89c400a950 tx=0x7f89cc06dbd0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.187+0000 7f89d2400640 1 -- 172.21.15.119:0/4244485961 shutdown_connections 2024-04-25T10:17:54.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.187+0000 7f89d2400640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f89ac06b4d0 0x7f89ac06d990 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.187+0000 7f89d2400640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89cc0cf2b0 0x7f89cc0cf730 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.187+0000 7f89d2400640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f89cc152b00 0x7f89cc152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.187+0000 7f89d2400640 1 --2- 172.21.15.119:0/4244485961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89b4004d40 0x7f89cc147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.187+0000 7f89d2400640 1 -- 172.21.15.119:0/4244485961 >> 172.21.15.119:0/4244485961 conn(0x7f89cc066ff0 msgr2=0x7f89cc15ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:54.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.187+0000 7f89d2400640 1 -- 172.21.15.119:0/4244485961 shutdown_connections 2024-04-25T10:17:54.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.187+0000 7f89d2400640 1 -- 172.21.15.119:0/4244485961 wait complete. 2024-04-25T10:17:54.210 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:17:54.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.409+0000 7f4167000640 1 Processor -- start 2024-04-25T10:17:54.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.410+0000 7f4167000640 1 -- start start 2024-04-25T10:17:54.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.411+0000 7f4167000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f416015bb60 0x7f41600d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.411+0000 7f4167000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f41600cf590 0x7f41600cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.411+0000 7f4167000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4160099b20 0x7f4160099fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.411+0000 7f4167000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41600cd430 con 0x7f4160099b20 2024-04-25T10:17:54.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.411+0000 7f4167000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41600cc820 con 0x7f41600cf590 2024-04-25T10:17:54.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.411+0000 7f4167000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41600ccb00 con 0x7f416015bb60 2024-04-25T10:17:54.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.412+0000 7f4165000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4160099b20 0x7f4160099fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.412+0000 7f4165000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4160099b20 0x7f4160099fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46492/0 (socket says 172.21.15.119:46492) 2024-04-25T10:17:54.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.412+0000 7f4165000640 1 -- 172.21.15.119:0/2969666840 learned_addr learned my addr 172.21.15.119:0/2969666840 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:54.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.412+0000 7f4165000640 1 -- 172.21.15.119:0/2969666840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f416015bb60 msgr2=0x7f41600d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:54.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.412+0000 7f415fe00640 1 --2- 172.21.15.119:0/2969666840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f416015bb60 0x7f41600d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.412+0000 7f415f400640 1 --2- 172.21.15.119:0/2969666840 >> v2:172.21.15.132:3300/0 conn(0x7f41600cf590 0x7f41600cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.412+0000 7f4165000640 1 --2- 172.21.15.119:0/2969666840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f416015bb60 0x7f41600d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.412+0000 7f4165000640 1 -- 172.21.15.119:0/2969666840 >> v2:172.21.15.132:3300/0 conn(0x7f41600cf590 msgr2=0x7f41600cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:54.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.412+0000 7f4165000640 1 --2- 172.21.15.119:0/2969666840 >> v2:172.21.15.132:3300/0 conn(0x7f41600cf590 0x7f41600cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.412+0000 7f4165000640 1 -- 172.21.15.119:0/2969666840 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41600ccde0 con 0x7f4160099b20 2024-04-25T10:17:54.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.412+0000 7f415f400640 1 --2- 172.21.15.119:0/2969666840 >> v2:172.21.15.132:3300/0 conn(0x7f41600cf590 0x7f41600cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:54.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.412+0000 7f415fe00640 1 --2- 172.21.15.119:0/2969666840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f416015bb60 0x7f41600d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:54.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.413+0000 7f4165000640 1 --2- 172.21.15.119:0/2969666840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4160099b20 0x7f4160099fa0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f4160072e40 tx=0x7f415802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5261c735dac28b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:54.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.414+0000 7f415ea00640 1 -- 172.21.15.119:0/2969666840 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158037070 con 0x7f4160099b20 2024-04-25T10:17:54.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.414+0000 7f415ea00640 1 -- 172.21.15.119:0/2969666840 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4158032440 con 0x7f4160099b20 2024-04-25T10:17:54.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.414+0000 7f415ea00640 1 -- 172.21.15.119:0/2969666840 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158032810 con 0x7f4160099b20 2024-04-25T10:17:54.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.414+0000 7f4167000640 1 -- 172.21.15.119:0/2969666840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4160099b20 msgr2=0x7f4160099fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.414+0000 7f4167000640 1 --2- 172.21.15.119:0/2969666840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4160099b20 0x7f4160099fa0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f4160072e40 tx=0x7f415802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.414+0000 7f4167000640 1 -- 172.21.15.119:0/2969666840 shutdown_connections 2024-04-25T10:17:54.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.414+0000 7f4167000640 1 --2- 172.21.15.119:0/2969666840 >> v2:172.21.15.132:3300/0 conn(0x7f41600cf590 0x7f41600cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.415+0000 7f4167000640 1 --2- 172.21.15.119:0/2969666840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4160099b20 0x7f4160099fa0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.415+0000 7f4167000640 1 --2- 172.21.15.119:0/2969666840 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f416015bb60 0x7f41600d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.415+0000 7f4167000640 1 -- 172.21.15.119:0/2969666840 >> 172.21.15.119:0/2969666840 conn(0x7f41600b33b0 msgr2=0x7f41600b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:54.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.415+0000 7f4167000640 1 -- 172.21.15.119:0/2969666840 shutdown_connections 2024-04-25T10:17:54.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.415+0000 7f4167000640 1 -- 172.21.15.119:0/2969666840 wait complete. 2024-04-25T10:17:54.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.417+0000 7f4167000640 1 Processor -- start 2024-04-25T10:17:54.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.417+0000 7f4167000640 1 -- start start 2024-04-25T10:17:54.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.418+0000 7f4167000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4160099b20 0x7f416015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.418+0000 7f4167000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41600cf590 0x7f416014c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.418+0000 7f4167000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f416015bb60 0x7f41600b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.418+0000 7f4167000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41600cde00 con 0x7f416015bb60 2024-04-25T10:17:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.418+0000 7f4167000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41600d08c0 con 0x7f41600cf590 2024-04-25T10:17:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.418+0000 7f4167000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41600cc440 con 0x7f4160099b20 2024-04-25T10:17:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.418+0000 7f4165000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f416015bb60 0x7f41600b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.418+0000 7f4165000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f416015bb60 0x7f41600b71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46494/0 (socket says 172.21.15.119:46494) 2024-04-25T10:17:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.418+0000 7f4165000640 1 -- 172.21.15.119:0/2796538286 learned_addr learned my addr 172.21.15.119:0/2796538286 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.419+0000 7f415fe00640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4160099b20 0x7f416015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.419+0000 7f415f400640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41600cf590 0x7f416014c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.419+0000 7f4165000640 1 -- 172.21.15.119:0/2796538286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4160099b20 msgr2=0x7f416015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.419+0000 7f4165000640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4160099b20 0x7f416015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.419+0000 7f4165000640 1 -- 172.21.15.119:0/2796538286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41600cf590 msgr2=0x7f416014c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.419+0000 7f4165000640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41600cf590 0x7f416014c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.419+0000 7f4165000640 1 -- 172.21.15.119:0/2796538286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416015d570 con 0x7f416015bb60 2024-04-25T10:17:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.419+0000 7f415f400640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41600cf590 0x7f416014c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:54.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.419+0000 7f4165000640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f416015bb60 0x7f41600b71b0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f415802a220 tx=0x7f41580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:54.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.420+0000 7f4157e00640 1 -- 172.21.15.119:0/2796538286 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158037040 con 0x7f416015bb60 2024-04-25T10:17:54.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.420+0000 7f4157e00640 1 -- 172.21.15.119:0/2796538286 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4158042070 con 0x7f416015bb60 2024-04-25T10:17:54.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.420+0000 7f4157e00640 1 -- 172.21.15.119:0/2796538286 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158030bd0 con 0x7f416015bb60 2024-04-25T10:17:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.420+0000 7f4167000640 1 -- 172.21.15.119:0/2796538286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416015d850 con 0x7f416015bb60 2024-04-25T10:17:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.420+0000 7f4167000640 1 -- 172.21.15.119:0/2796538286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416015de10 con 0x7f416015bb60 2024-04-25T10:17:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.421+0000 7f4167000640 1 -- 172.21.15.119:0/2796538286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41600750b0 con 0x7f416015bb60 2024-04-25T10:17:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.422+0000 7f4157e00640 1 -- 172.21.15.119:0/2796538286 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4158047460 con 0x7f416015bb60 2024-04-25T10:17:54.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.424+0000 7f4157e00640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f413c06b4d0 0x7f413c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.424+0000 7f4157e00640 1 -- 172.21.15.119:0/2796538286 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f41580aafa0 con 0x7f416015bb60 2024-04-25T10:17:54.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.424+0000 7f415fe00640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f413c06b4d0 0x7f413c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.425+0000 7f415fe00640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f413c06b4d0 0x7f413c06d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4150007fb0 tx=0x7f4150007990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:54.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.428+0000 7f4157e00640 1 -- 172.21.15.119:0/2796538286 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4158079cb0 con 0x7f416015bb60 2024-04-25T10:17:54.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.623+0000 7f4167000640 1 -- 172.21.15.119:0/2796538286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4160070b40 con 0x7f416015bb60 2024-04-25T10:17:54.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.624+0000 7f4157e00640 1 -- 172.21.15.119:0/2796538286 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f41580795b0 con 0x7f416015bb60 2024-04-25T10:17:54.628 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:17:54.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.627+0000 7f4167000640 1 -- 172.21.15.119:0/2796538286 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f413c06b4d0 msgr2=0x7f413c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.627+0000 7f4167000640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f413c06b4d0 0x7f413c06d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4150007fb0 tx=0x7f4150007990 comp rx=0 tx=0).stop 2024-04-25T10:17:54.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.627+0000 7f4167000640 1 -- 172.21.15.119:0/2796538286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f416015bb60 msgr2=0x7f41600b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.627+0000 7f4167000640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f416015bb60 0x7f41600b71b0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f415802a220 tx=0x7f41580046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.628+0000 7f4167000640 1 -- 172.21.15.119:0/2796538286 shutdown_connections 2024-04-25T10:17:54.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.628+0000 7f4167000640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f413c06b4d0 0x7f413c06d990 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.628+0000 7f4167000640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f416015bb60 0x7f41600b71b0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.628+0000 7f4167000640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41600cf590 0x7f416014c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.628+0000 7f4167000640 1 --2- 172.21.15.119:0/2796538286 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4160099b20 0x7f416015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.628+0000 7f4167000640 1 -- 172.21.15.119:0/2796538286 >> 172.21.15.119:0/2796538286 conn(0x7f41600b33b0 msgr2=0x7f416015fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:54.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.628+0000 7f4167000640 1 -- 172.21.15.119:0/2796538286 shutdown_connections 2024-04-25T10:17:54.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.628+0000 7f4167000640 1 -- 172.21.15.119:0/2796538286 wait complete. 2024-04-25T10:17:54.649 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:17:54.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.848+0000 7f4f07e00640 1 Processor -- start 2024-04-25T10:17:54.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.848+0000 7f4f07e00640 1 -- start start 2024-04-25T10:17:54.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.849+0000 7f4f07e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f0806b2b0 0x7f4f080cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.850+0000 7f4f07e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f0815f400 con 0x7f4f080cc810 2024-04-25T10:17:54.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.850+0000 7f4f07e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f0815f6e0 con 0x7f4f081450e0 2024-04-25T10:17:54.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.850+0000 7f4f07e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f081454c0 con 0x7f4f0806b2b0 2024-04-25T10:17:54.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.850+0000 7f4f07400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4f080cc810 0x7f4f080b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50518/0 (socket says 172.21.15.119:50518) 2024-04-25T10:17:54.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.850+0000 7f4f07400640 1 -- 172.21.15.119:0/1033246043 learned_addr learned my addr 172.21.15.119:0/1033246043 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:54.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.850+0000 7f4effe00640 1 --2- 172.21.15.119:0/1033246043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f0806b2b0 0x7f4f080cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.850+0000 7f4f06000640 1 -- 172.21.15.119:0/1033246043 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931569506 0 0) 0x7f4f0815f400 con 0x7f4f080cc810 2024-04-25T10:17:54.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.850+0000 7f4f06000640 1 -- 172.21.15.119:0/1033246043 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eec003520 con 0x7f4f080cc810 2024-04-25T10:17:54.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.851+0000 7f4f06000640 1 -- 172.21.15.119:0/1033246043 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 638773571 0 0) 0x7f4eec003520 con 0x7f4f080cc810 2024-04-25T10:17:54.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.851+0000 7f4f06000640 1 -- 172.21.15.119:0/1033246043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f0806b2b0 msgr2=0x7f4f080cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:54.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.851+0000 7f4f06000640 1 --2- 172.21.15.119:0/1033246043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f0806b2b0 0x7f4f080cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.851+0000 7f4f06000640 1 -- 172.21.15.119:0/1033246043 >> v1:172.21.15.132:6789/0 conn(0x7f4f081450e0 legacy=0x7f4f0806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.851+0000 7f4f06000640 1 -- 172.21.15.119:0/1033246043 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0815f9c0 con 0x7f4f080cc810 2024-04-25T10:17:54.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.851+0000 7f4f06000640 1 -- 172.21.15.119:0/1033246043 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f4f000019f0 con 0x7f4f080cc810 2024-04-25T10:17:54.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.851+0000 7f4f06000640 1 -- 172.21.15.119:0/1033246043 >> v1:172.21.15.119:6789/0 conn(0x7f4f080cc810 legacy=0x7f4f080b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.851+0000 7f4f06000640 1 --2- 172.21.15.119:0/1033246043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eec0039e0 0x7f4eec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.851+0000 7f4f06000640 1 --2- 172.21.15.119:0/1033246043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f0806b2b0 0x7f4eec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.851+0000 7f4f06000640 1 --2- 172.21.15.119:0/1033246043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eec004d40 0x7f4eec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.851+0000 7f4f06000640 1 -- 172.21.15.119:0/1033246043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec00bb50 con 0x7f4eec0039e0 2024-04-25T10:17:54.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.851+0000 7f4f06000640 1 -- 172.21.15.119:0/1033246043 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec00bd00 con 0x7f4eec004d40 2024-04-25T10:17:54.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.851+0000 7f4f06000640 1 -- 172.21.15.119:0/1033246043 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec00bf00 con 0x7f4f0806b2b0 2024-04-25T10:17:54.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.851+0000 7f4f06a00640 1 --2- 172.21.15.119:0/1033246043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eec0039e0 0x7f4eec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.852+0000 7f4f07400640 1 --2- 172.21.15.119:0/1033246043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eec004d40 0x7f4eec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.852+0000 7f4f06a00640 1 -- 172.21.15.119:0/1033246043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f0806b2b0 msgr2=0x7f4eec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:54.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.852+0000 7f4f06a00640 1 --2- 172.21.15.119:0/1033246043 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f0806b2b0 0x7f4eec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.852+0000 7f4f06a00640 1 -- 172.21.15.119:0/1033246043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eec004d40 msgr2=0x7f4eec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:54.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.852+0000 7f4f06a00640 1 --2- 172.21.15.119:0/1033246043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eec004d40 0x7f4eec0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.852+0000 7f4f06a00640 1 -- 172.21.15.119:0/1033246043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec003520 con 0x7f4eec0039e0 2024-04-25T10:17:54.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.853+0000 7f4f06a00640 1 --2- 172.21.15.119:0/1033246043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eec0039e0 0x7f4eec003e00 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8001750 tx=0x7f4eec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9020f453068f7860 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:54.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.853+0000 7f4f06000640 1 -- 172.21.15.119:0/1033246043 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8001370 con 0x7f4eec0039e0 2024-04-25T10:17:54.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.853+0000 7f4f06000640 1 -- 172.21.15.119:0/1033246043 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef80052d0 con 0x7f4eec0039e0 2024-04-25T10:17:54.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.853+0000 7f4f06000640 1 -- 172.21.15.119:0/1033246043 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef802abe0 con 0x7f4eec0039e0 2024-04-25T10:17:54.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.854+0000 7f4f07e00640 1 -- 172.21.15.119:0/1033246043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eec0039e0 msgr2=0x7f4eec003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.854+0000 7f4f07e00640 1 --2- 172.21.15.119:0/1033246043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eec0039e0 0x7f4eec003e00 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8001750 tx=0x7f4eec005830 comp rx=0 tx=0).stop 2024-04-25T10:17:54.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.854+0000 7f4f07400640 1 -- 172.21.15.119:0/1033246043 reap_dead start 2024-04-25T10:17:54.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.854+0000 7f4f07e00640 1 -- 172.21.15.119:0/1033246043 shutdown_connections 2024-04-25T10:17:54.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.854+0000 7f4f07e00640 1 -- 172.21.15.119:0/1033246043 >> 172.21.15.119:0/1033246043 conn(0x7f4f08066ff0 msgr2=0x7f4f080b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:54.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.854+0000 7f4f07e00640 1 -- 172.21.15.119:0/1033246043 shutdown_connections 2024-04-25T10:17:54.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.854+0000 7f4f07e00640 1 -- 172.21.15.119:0/1033246043 wait complete. 2024-04-25T10:17:54.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.856+0000 7f4f07e00640 1 Processor -- start 2024-04-25T10:17:54.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.856+0000 7f4f07e00640 1 -- start start 2024-04-25T10:17:54.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.857+0000 7f4f07e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eec0039e0 0x7f4f0813e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.857+0000 7f4f07e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f0815b320 0x7f4f0813f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.857+0000 7f4f07e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f0815ab60 0x7f4f08141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.857+0000 7f4f07e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f080cce00 con 0x7f4eec0039e0 2024-04-25T10:17:54.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.857+0000 7f4f07e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f080ccc80 con 0x7f4f0815ab60 2024-04-25T10:17:54.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.857+0000 7f4f07e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f080cc2d0 con 0x7f4f0815b320 2024-04-25T10:17:54.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.857+0000 7f4f06a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eec0039e0 0x7f4f0813e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.857+0000 7f4f06a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eec0039e0 0x7f4f0813e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46508/0 (socket says 172.21.15.119:46508) 2024-04-25T10:17:54.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.857+0000 7f4f06a00640 1 -- 172.21.15.119:0/76720563 learned_addr learned my addr 172.21.15.119:0/76720563 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:54.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.857+0000 7f4f06a00640 1 -- 172.21.15.119:0/76720563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f0815b320 msgr2=0x7f4f0813f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.857+0000 7f4f06a00640 1 --2- 172.21.15.119:0/76720563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f0815b320 0x7f4f0813f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.857+0000 7f4f06a00640 1 -- 172.21.15.119:0/76720563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f0815ab60 msgr2=0x7f4f08141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.857+0000 7f4f06a00640 1 --2- 172.21.15.119:0/76720563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f0815ab60 0x7f4f08141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.857+0000 7f4f06a00640 1 -- 172.21.15.119:0/76720563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0813d3b0 con 0x7f4eec0039e0 2024-04-25T10:17:54.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.858+0000 7f4f06a00640 1 --2- 172.21.15.119:0/76720563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eec0039e0 0x7f4f0813e020 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8008fd0 tx=0x7f4ef8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:54.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.858+0000 7f4efea00640 1 -- 172.21.15.119:0/76720563 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8037040 con 0x7f4eec0039e0 2024-04-25T10:17:54.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.858+0000 7f4efea00640 1 -- 172.21.15.119:0/76720563 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef8005480 con 0x7f4eec0039e0 2024-04-25T10:17:54.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.858+0000 7f4efea00640 1 -- 172.21.15.119:0/76720563 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8007a30 con 0x7f4eec0039e0 2024-04-25T10:17:54.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.858+0000 7f4f07e00640 1 -- 172.21.15.119:0/76720563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f0813e950 con 0x7f4eec0039e0 2024-04-25T10:17:54.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.858+0000 7f4f07e00640 1 -- 172.21.15.119:0/76720563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f0813eed0 con 0x7f4eec0039e0 2024-04-25T10:17:54.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.859+0000 7f4f07e00640 1 -- 172.21.15.119:0/76720563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f080749b0 con 0x7f4eec0039e0 2024-04-25T10:17:54.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.860+0000 7f4efea00640 1 -- 172.21.15.119:0/76720563 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ef8044020 con 0x7f4eec0039e0 2024-04-25T10:17:54.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.861+0000 7f4efea00640 1 --2- 172.21.15.119:0/76720563 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ee406b500 0x7f4ee406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.861+0000 7f4efea00640 1 -- 172.21.15.119:0/76720563 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4ef80a9c60 con 0x7f4eec0039e0 2024-04-25T10:17:54.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.862+0000 7f4effe00640 1 --2- 172.21.15.119:0/76720563 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ee406b500 0x7f4ee406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.863+0000 7f4effe00640 1 --2- 172.21.15.119:0/76720563 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ee406b500 0x7f4ee406d9c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0006e80 tx=0x7f4ef0006dd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:54.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:54.865+0000 7f4efea00640 1 -- 172.21.15.119:0/76720563 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ef804b020 con 0x7f4eec0039e0 2024-04-25T10:17:55.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.063+0000 7f4f07e00640 1 -- 172.21.15.119:0/76720563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4f08143880 con 0x7f4eec0039e0 2024-04-25T10:17:55.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.064+0000 7f4efea00640 1 -- 172.21.15.119:0/76720563 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ef8078aa0 con 0x7f4eec0039e0 2024-04-25T10:17:55.067 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:17:55.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.067+0000 7f4f07e00640 1 -- 172.21.15.119:0/76720563 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ee406b500 msgr2=0x7f4ee406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.067+0000 7f4f07e00640 1 --2- 172.21.15.119:0/76720563 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ee406b500 0x7f4ee406d9c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0006e80 tx=0x7f4ef0006dd0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.067+0000 7f4f07e00640 1 -- 172.21.15.119:0/76720563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eec0039e0 msgr2=0x7f4f0813e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.067+0000 7f4f07e00640 1 --2- 172.21.15.119:0/76720563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eec0039e0 0x7f4f0813e020 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8008fd0 tx=0x7f4ef8001120 comp rx=0 tx=0).stop 2024-04-25T10:17:55.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.067+0000 7f4f07e00640 1 -- 172.21.15.119:0/76720563 shutdown_connections 2024-04-25T10:17:55.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.067+0000 7f4f07e00640 1 --2- 172.21.15.119:0/76720563 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4ee406b500 0x7f4ee406d9c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.067+0000 7f4f07e00640 1 --2- 172.21.15.119:0/76720563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f0815ab60 0x7f4f08141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.067+0000 7f4f07e00640 1 --2- 172.21.15.119:0/76720563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f0815b320 0x7f4f0813f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.067+0000 7f4f07e00640 1 --2- 172.21.15.119:0/76720563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eec0039e0 0x7f4f0813e020 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.067+0000 7f4f07e00640 1 -- 172.21.15.119:0/76720563 >> 172.21.15.119:0/76720563 conn(0x7f4f08066ff0 msgr2=0x7f4f080b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.067+0000 7f4f07e00640 1 -- 172.21.15.119:0/76720563 shutdown_connections 2024-04-25T10:17:55.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.067+0000 7f4f07e00640 1 -- 172.21.15.119:0/76720563 wait complete. 2024-04-25T10:17:55.089 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:17:55.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.290+0000 7f90c0000640 1 Processor -- start 2024-04-25T10:17:55.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.290+0000 7f90c0000640 1 -- start start 2024-04-25T10:17:55.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.292+0000 7f90c0000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90b80b3240 0x7f90b80b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.292+0000 7f90c0000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f90b8162e30 0x7f90b81632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.292+0000 7f90c0000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b806e3f0 con 0x7f90b8071fc0 2024-04-25T10:17:55.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.292+0000 7f90c0000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b806e6d0 con 0x7f90b8162e30 2024-04-25T10:17:55.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.292+0000 7f90c0000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b806e9b0 con 0x7f90b80b3240 2024-04-25T10:17:55.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.292+0000 7f90be000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f90b8162e30 0x7f90b81632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.293+0000 7f90be000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f90b8162e30 0x7f90b81632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46520/0 (socket says 172.21.15.119:46520) 2024-04-25T10:17:55.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.293+0000 7f90be000640 1 -- 172.21.15.119:0/3318243016 learned_addr learned my addr 172.21.15.119:0/3318243016 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:55.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.293+0000 7f90bcc00640 1 --2- 172.21.15.119:0/3318243016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90b80b3240 0x7f90b80b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.293+0000 7f90be000640 1 -- 172.21.15.119:0/3318243016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90b80b3240 msgr2=0x7f90b80b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:55.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.293+0000 7f90be000640 1 --2- 172.21.15.119:0/3318243016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90b80b3240 0x7f90b80b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.293+0000 7f90be000640 1 -- 172.21.15.119:0/3318243016 >> v1:172.21.15.132:6789/0 conn(0x7f90b8071fc0 legacy=0x7f90b806d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.293+0000 7f90be000640 1 -- 172.21.15.119:0/3318243016 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b8162910 con 0x7f90b8162e30 2024-04-25T10:17:55.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.294+0000 7f90be000640 1 --2- 172.21.15.119:0/3318243016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b8162e30 0x7f90b81632c0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009e50 tx=0x7f90b002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9dffe7559f980e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.294+0000 7f90b7e00640 1 -- 172.21.15.119:0/3318243016 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0036070 con 0x7f90b8162e30 2024-04-25T10:17:55.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.295+0000 7f90b7e00640 1 -- 172.21.15.119:0/3318243016 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b0031600 con 0x7f90b8162e30 2024-04-25T10:17:55.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.295+0000 7f90b7e00640 1 -- 172.21.15.119:0/3318243016 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00319d0 con 0x7f90b8162e30 2024-04-25T10:17:55.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.295+0000 7f90c0000640 1 -- 172.21.15.119:0/3318243016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b8162e30 msgr2=0x7f90b81632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.295+0000 7f90c0000640 1 --2- 172.21.15.119:0/3318243016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b8162e30 0x7f90b81632c0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009e50 tx=0x7f90b002a710 comp rx=0 tx=0).stop 2024-04-25T10:17:55.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.295+0000 7f90c0000640 1 -- 172.21.15.119:0/3318243016 shutdown_connections 2024-04-25T10:17:55.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.295+0000 7f90c0000640 1 --2- 172.21.15.119:0/3318243016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b8162e30 0x7f90b81632c0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.295+0000 7f90c0000640 1 --2- 172.21.15.119:0/3318243016 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90b80b3240 0x7f90b80b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.295+0000 7f90c0000640 1 -- 172.21.15.119:0/3318243016 >> 172.21.15.119:0/3318243016 conn(0x7f90b806aba0 msgr2=0x7f90b806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.296+0000 7f90c0000640 1 -- 172.21.15.119:0/3318243016 shutdown_connections 2024-04-25T10:17:55.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.296+0000 7f90c0000640 1 -- 172.21.15.119:0/3318243016 wait complete. 2024-04-25T10:17:55.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.297+0000 7f90c0000640 1 Processor -- start 2024-04-25T10:17:55.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.297+0000 7f90c0000640 1 -- start start 2024-04-25T10:17:55.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.298+0000 7f90c0000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90b8071fc0 0x7f90b81f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.298+0000 7f90c0000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90b80b3240 0x7f90b814cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.298+0000 7f90c0000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b8162e30 0x7f90b815d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.298+0000 7f90c0000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b8162590 con 0x7f90b8162e30 2024-04-25T10:17:55.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.298+0000 7f90c0000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b806e0b0 con 0x7f90b80b3240 2024-04-25T10:17:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.298+0000 7f90c0000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b80713c0 con 0x7f90b8071fc0 2024-04-25T10:17:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.298+0000 7f90be000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b8162e30 0x7f90b815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.299+0000 7f90be000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b8162e30 0x7f90b815d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46534/0 (socket says 172.21.15.119:46534) 2024-04-25T10:17:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.299+0000 7f90be000640 1 -- 172.21.15.119:0/3682694273 learned_addr learned my addr 172.21.15.119:0/3682694273 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.299+0000 7f90bcc00640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90b80b3240 0x7f90b814cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.299+0000 7f90bd600640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90b8071fc0 0x7f90b81f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.299+0000 7f90be000640 1 -- 172.21.15.119:0/3682694273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90b8071fc0 msgr2=0x7f90b81f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.299+0000 7f90be000640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90b8071fc0 0x7f90b81f7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.299+0000 7f90be000640 1 -- 172.21.15.119:0/3682694273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90b80b3240 msgr2=0x7f90b814cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.299+0000 7f90be000640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90b80b3240 0x7f90b814cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.299+0000 7f90be000640 1 -- 172.21.15.119:0/3682694273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b81f7880 con 0x7f90b8162e30 2024-04-25T10:17:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.299+0000 7f90bcc00640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90b80b3240 0x7f90b814cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.299+0000 7f90be000640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b8162e30 0x7f90b815d830 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f90b0034040 tx=0x7f90b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.300+0000 7f90b5600640 1 -- 172.21.15.119:0/3682694273 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0036050 con 0x7f90b8162e30 2024-04-25T10:17:55.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.300+0000 7f90b5600640 1 -- 172.21.15.119:0/3682694273 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b0042070 con 0x7f90b8162e30 2024-04-25T10:17:55.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.300+0000 7f90b5600640 1 -- 172.21.15.119:0/3682694273 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0009510 con 0x7f90b8162e30 2024-04-25T10:17:55.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.300+0000 7f90c0000640 1 -- 172.21.15.119:0/3682694273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b814d4f0 con 0x7f90b8162e30 2024-04-25T10:17:55.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.300+0000 7f90c0000640 1 -- 172.21.15.119:0/3682694273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b815c790 con 0x7f90b8162e30 2024-04-25T10:17:55.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.301+0000 7f90c0000640 1 -- 172.21.15.119:0/3682694273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b8165900 con 0x7f90b8162e30 2024-04-25T10:17:55.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.302+0000 7f90b5600640 1 -- 172.21.15.119:0/3682694273 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90b0007bf0 con 0x7f90b8162e30 2024-04-25T10:17:55.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.303+0000 7f90b5600640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f909806b4d0 0x7f909806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.303+0000 7f90b5600640 1 -- 172.21.15.119:0/3682694273 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f90b00ab190 con 0x7f90b8162e30 2024-04-25T10:17:55.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.303+0000 7f90bd600640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f909806b4d0 0x7f909806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.304+0000 7f90bd600640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f909806b4d0 0x7f909806d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f90b815e1b0 tx=0x7f90a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.307+0000 7f90b5600640 1 -- 172.21.15.119:0/3682694273 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90b81f7880 con 0x7f90b8162e30 2024-04-25T10:17:55.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.513+0000 7f90c0000640 1 -- 172.21.15.119:0/3682694273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f90b806bab0 con 0x7f90b8162e30 2024-04-25T10:17:55.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.514+0000 7f90b5600640 1 -- 172.21.15.119:0/3682694273 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) v1 ==== 80+0+0 (secure 0 0 0) 0x7f90b0045360 con 0x7f90b8162e30 2024-04-25T10:17:55.518 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2024-04-25T10:17:55.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.517+0000 7f90c0000640 1 -- 172.21.15.119:0/3682694273 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f909806b4d0 msgr2=0x7f909806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.517+0000 7f90c0000640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f909806b4d0 0x7f909806d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f90b815e1b0 tx=0x7f90a8009040 comp rx=0 tx=0).stop 2024-04-25T10:17:55.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.517+0000 7f90c0000640 1 -- 172.21.15.119:0/3682694273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b8162e30 msgr2=0x7f90b815d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.517+0000 7f90c0000640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b8162e30 0x7f90b815d830 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f90b0034040 tx=0x7f90b0004a40 comp rx=0 tx=0).stop 2024-04-25T10:17:55.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.517+0000 7f90c0000640 1 -- 172.21.15.119:0/3682694273 shutdown_connections 2024-04-25T10:17:55.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.517+0000 7f90c0000640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f909806b4d0 0x7f909806d990 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.517+0000 7f90c0000640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90b8162e30 0x7f90b815d830 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.517+0000 7f90c0000640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90b80b3240 0x7f90b814cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.517+0000 7f90c0000640 1 --2- 172.21.15.119:0/3682694273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f90b8071fc0 0x7f90b81f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.517+0000 7f90c0000640 1 -- 172.21.15.119:0/3682694273 >> 172.21.15.119:0/3682694273 conn(0x7f90b806aba0 msgr2=0x7f90b80af180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.517+0000 7f90c0000640 1 -- 172.21.15.119:0/3682694273 shutdown_connections 2024-04-25T10:17:55.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:55.517+0000 7f90c0000640 1 -- 172.21.15.119:0/3682694273 wait complete. 2024-04-25T10:17:55.540 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:17:55.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:55.540 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:55.767 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:17:55.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:55.767 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:17:55.818 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:17:55.818 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:55.899 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:55.899 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:17:55.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:55.899 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:17:55.989 INFO:teuthology.orchestra.run.smithi119.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:17:55.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:17:55.991 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:17:56.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.185+0000 7f7bd4a00640 1 Processor -- start 2024-04-25T10:17:56.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.185+0000 7f7bd4a00640 1 -- start start 2024-04-25T10:17:56.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.187+0000 7f7bd4a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7bd00b3e90 0x7f7bd00cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.187+0000 7f7bd4a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd015f400 con 0x7f7bd006b2b0 2024-04-25T10:17:56.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.187+0000 7f7bd4a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd015f6e0 con 0x7f7bd01450e0 2024-04-25T10:17:56.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.187+0000 7f7bd4a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd01454c0 con 0x7f7bd00b3e90 2024-04-25T10:17:56.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.187+0000 7f7bc7e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7bd006b2b0 0x7f7bd00cc2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50526/0 (socket says 172.21.15.119:50526) 2024-04-25T10:17:56.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.187+0000 7f7bc7e00640 1 -- 172.21.15.119:0/1535511230 learned_addr learned my addr 172.21.15.119:0/1535511230 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:56.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.187+0000 7f7bcfe00640 1 --2- 172.21.15.119:0/1535511230 >> v2:172.21.15.132:3300/0 conn(0x7f7bd00b3e90 0x7f7bd00cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 -- 172.21.15.119:0/1535511230 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221974514 0 0) 0x7f7bd015f400 con 0x7f7bd006b2b0 2024-04-25T10:17:56.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 -- 172.21.15.119:0/1535511230 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb4003520 con 0x7f7bd006b2b0 2024-04-25T10:17:56.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 -- 172.21.15.119:0/1535511230 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2745347889 0 0) 0x7f7bb4003520 con 0x7f7bd006b2b0 2024-04-25T10:17:56.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 -- 172.21.15.119:0/1535511230 >> v2:172.21.15.132:3300/0 conn(0x7f7bd00b3e90 msgr2=0x7f7bd00cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:56.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 --2- 172.21.15.119:0/1535511230 >> v2:172.21.15.132:3300/0 conn(0x7f7bd00b3e90 0x7f7bd00cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 -- 172.21.15.119:0/1535511230 >> v1:172.21.15.132:6789/0 conn(0x7f7bd01450e0 legacy=0x7f7bd006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 -- 172.21.15.119:0/1535511230 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd015f9c0 con 0x7f7bd006b2b0 2024-04-25T10:17:56.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 -- 172.21.15.119:0/1535511230 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f7bb8001670 con 0x7f7bd006b2b0 2024-04-25T10:17:56.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 -- 172.21.15.119:0/1535511230 >> v1:172.21.15.119:6789/0 conn(0x7f7bd006b2b0 legacy=0x7f7bd00cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 --2- 172.21.15.119:0/1535511230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bb40039b0 0x7f7bb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 --2- 172.21.15.119:0/1535511230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bb4004800 0x7f7bb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 --2- 172.21.15.119:0/1535511230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7bb40052e0 0x7f7bb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 -- 172.21.15.119:0/1535511230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb400c180 con 0x7f7bb40039b0 2024-04-25T10:17:56.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 -- 172.21.15.119:0/1535511230 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb400c330 con 0x7f7bb4004800 2024-04-25T10:17:56.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcea00640 1 -- 172.21.15.119:0/1535511230 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb400c530 con 0x7f7bb40052e0 2024-04-25T10:17:56.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.188+0000 7f7bcf400640 1 --2- 172.21.15.119:0/1535511230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bb40039b0 0x7f7bb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.189+0000 7f7bc7e00640 1 --2- 172.21.15.119:0/1535511230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bb4004800 0x7f7bb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.189+0000 7f7bcf400640 1 -- 172.21.15.119:0/1535511230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7bb40052e0 msgr2=0x7f7bb400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:56.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.189+0000 7f7bcf400640 1 --2- 172.21.15.119:0/1535511230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7bb40052e0 0x7f7bb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.189+0000 7f7bcf400640 1 -- 172.21.15.119:0/1535511230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bb4004800 msgr2=0x7f7bb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:56.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.189+0000 7f7bcf400640 1 --2- 172.21.15.119:0/1535511230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bb4004800 0x7f7bb4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.189+0000 7f7bcf400640 1 -- 172.21.15.119:0/1535511230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb4003520 con 0x7f7bb40039b0 2024-04-25T10:17:56.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.189+0000 7f7bcfe00640 1 -- 172.21.15.119:0/1535511230 reap_dead start 2024-04-25T10:17:56.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.190+0000 7f7bcf400640 1 --2- 172.21.15.119:0/1535511230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bb40039b0 0x7f7bb4003dd0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0001720 tx=0x7f7bb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09aa746f05115fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.190+0000 7f7bcea00640 1 -- 172.21.15.119:0/1535511230 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0001370 con 0x7f7bb40039b0 2024-04-25T10:17:56.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.190+0000 7f7bcea00640 1 -- 172.21.15.119:0/1535511230 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc00052d0 con 0x7f7bb40039b0 2024-04-25T10:17:56.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.190+0000 7f7bcea00640 1 -- 172.21.15.119:0/1535511230 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc002ab90 con 0x7f7bb40039b0 2024-04-25T10:17:56.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.191+0000 7f7bd4a00640 1 -- 172.21.15.119:0/1535511230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bb40039b0 msgr2=0x7f7bb4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.191+0000 7f7bd4a00640 1 --2- 172.21.15.119:0/1535511230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bb40039b0 0x7f7bb4003dd0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0001720 tx=0x7f7bb4005830 comp rx=0 tx=0).stop 2024-04-25T10:17:56.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.191+0000 7f7bd4a00640 1 -- 172.21.15.119:0/1535511230 shutdown_connections 2024-04-25T10:17:56.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.191+0000 7f7bd4a00640 1 --2- 172.21.15.119:0/1535511230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bb40039b0 0x7f7bb4003dd0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.191+0000 7f7bd4a00640 1 -- 172.21.15.119:0/1535511230 >> 172.21.15.119:0/1535511230 conn(0x7f7bd0066ff0 msgr2=0x7f7bd00b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:56.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.191+0000 7f7bd4a00640 1 -- 172.21.15.119:0/1535511230 shutdown_connections 2024-04-25T10:17:56.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.191+0000 7f7bd4a00640 1 -- 172.21.15.119:0/1535511230 wait complete. 2024-04-25T10:17:56.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.193+0000 7f7bd4a00640 1 Processor -- start 2024-04-25T10:17:56.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.193+0000 7f7bd4a00640 1 -- start start 2024-04-25T10:17:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.194+0000 7f7bd4a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7bb40039b0 0x7f7bd013ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.194+0000 7f7bd4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bd015bd40 0x7f7bd00ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.194+0000 7f7bd4a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bd013cf40 0x7f7bd01f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.194+0000 7f7bd4a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd006ad20 con 0x7f7bd015bd40 2024-04-25T10:17:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.194+0000 7f7bd4a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd006aba0 con 0x7f7bd013cf40 2024-04-25T10:17:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.194+0000 7f7bd4a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd006aea0 con 0x7f7bb40039b0 2024-04-25T10:17:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.194+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bd015bd40 0x7f7bd00ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.194+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bd015bd40 0x7f7bd00ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46548/0 (socket says 172.21.15.119:46548) 2024-04-25T10:17:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.194+0000 7f7bc7e00640 1 -- 172.21.15.119:0/262397924 learned_addr learned my addr 172.21.15.119:0/262397924 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.195+0000 7f7bcf400640 1 --2- 172.21.15.119:0/262397924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7bb40039b0 0x7f7bd013ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.195+0000 7f7bcfe00640 1 --2- 172.21.15.119:0/262397924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bd013cf40 0x7f7bd01f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.195+0000 7f7bc7e00640 1 -- 172.21.15.119:0/262397924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7bb40039b0 msgr2=0x7f7bd013ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.195+0000 7f7bc7e00640 1 --2- 172.21.15.119:0/262397924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7bb40039b0 0x7f7bd013ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.195+0000 7f7bc7e00640 1 -- 172.21.15.119:0/262397924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bd013cf40 msgr2=0x7f7bd01f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.195+0000 7f7bc7e00640 1 --2- 172.21.15.119:0/262397924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bd013cf40 0x7f7bd01f6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.195+0000 7f7bc7e00640 1 -- 172.21.15.119:0/262397924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd00acdf0 con 0x7f7bd015bd40 2024-04-25T10:17:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.195+0000 7f7bc7e00640 1 --2- 172.21.15.119:0/262397924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bd015bd40 0x7f7bd00ac8b0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800e830 tx=0x7f7bb800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.195+0000 7f7bc7400640 1 -- 172.21.15.119:0/262397924 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800c660 con 0x7f7bd015bd40 2024-04-25T10:17:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.195+0000 7f7bc7400640 1 -- 172.21.15.119:0/262397924 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb8018070 con 0x7f7bd015bd40 2024-04-25T10:17:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.195+0000 7f7bd4a00640 1 -- 172.21.15.119:0/262397924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd015d670 con 0x7f7bd015bd40 2024-04-25T10:17:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.196+0000 7f7bc7400640 1 -- 172.21.15.119:0/262397924 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800ca40 con 0x7f7bd015bd40 2024-04-25T10:17:56.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.196+0000 7f7bd4a00640 1 -- 172.21.15.119:0/262397924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd015d930 con 0x7f7bd015bd40 2024-04-25T10:17:56.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.197+0000 7f7bd4a00640 1 -- 172.21.15.119:0/262397924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd00749b0 con 0x7f7bd015bd40 2024-04-25T10:17:56.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.198+0000 7f7bc7400640 1 -- 172.21.15.119:0/262397924 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7bb801d460 con 0x7f7bd015bd40 2024-04-25T10:17:56.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.199+0000 7f7bc7400640 1 --2- 172.21.15.119:0/262397924 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7bac06b4d0 0x7f7bac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.199+0000 7f7bc7400640 1 -- 172.21.15.119:0/262397924 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7bb808a0b0 con 0x7f7bd015bd40 2024-04-25T10:17:56.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.199+0000 7f7bcf400640 1 --2- 172.21.15.119:0/262397924 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7bac06b4d0 0x7f7bac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.200+0000 7f7bcf400640 1 --2- 172.21.15.119:0/262397924 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7bac06b4d0 0x7f7bac06d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0032000 tx=0x7f7bc002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.203+0000 7f7bc7400640 1 -- 172.21.15.119:0/262397924 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7bb8058ee0 con 0x7f7bd015bd40 2024-04-25T10:17:56.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.437+0000 7f7bd4a00640 1 -- 172.21.15.119:0/262397924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7bd00cff50 con 0x7f7bd015bd40 2024-04-25T10:17:56.441 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:17:56.438+0000 7f8c64600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:17:56.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.441+0000 7f7bc7400640 1 -- 172.21.15.119:0/262397924 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7bb80587e0 con 0x7f7bd015bd40 2024-04-25T10:17:56.445 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:17:56.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.445+0000 7f7bd4a00640 1 -- 172.21.15.119:0/262397924 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7bac06b4d0 msgr2=0x7f7bac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.445+0000 7f7bd4a00640 1 --2- 172.21.15.119:0/262397924 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7bac06b4d0 0x7f7bac06d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0032000 tx=0x7f7bc002e000 comp rx=0 tx=0).stop 2024-04-25T10:17:56.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.445+0000 7f7bd4a00640 1 -- 172.21.15.119:0/262397924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bd015bd40 msgr2=0x7f7bd00ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.445+0000 7f7bd4a00640 1 --2- 172.21.15.119:0/262397924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bd015bd40 0x7f7bd00ac8b0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800e830 tx=0x7f7bb800ecf0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.446+0000 7f7bd4a00640 1 -- 172.21.15.119:0/262397924 shutdown_connections 2024-04-25T10:17:56.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.446+0000 7f7bd4a00640 1 --2- 172.21.15.119:0/262397924 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f7bac06b4d0 0x7f7bac06d990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.446+0000 7f7bd4a00640 1 --2- 172.21.15.119:0/262397924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bd013cf40 0x7f7bd01f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.446+0000 7f7bd4a00640 1 --2- 172.21.15.119:0/262397924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bd015bd40 0x7f7bd00ac8b0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.446+0000 7f7bd4a00640 1 --2- 172.21.15.119:0/262397924 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7bb40039b0 0x7f7bd013ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.446+0000 7f7bd4a00640 1 -- 172.21.15.119:0/262397924 >> 172.21.15.119:0/262397924 conn(0x7f7bd0066ff0 msgr2=0x7f7bd01433a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:56.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.446+0000 7f7bd4a00640 1 -- 172.21.15.119:0/262397924 shutdown_connections 2024-04-25T10:17:56.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.446+0000 7f7bd4a00640 1 -- 172.21.15.119:0/262397924 wait complete. 2024-04-25T10:17:56.467 INFO:tasks.ceph.mds.c.smithi119.stderr: -8323> 2024-04-25T10:17:44.126+0000 7fd6f4400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:17:56.469 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:17:56.473 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:17:56.540 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2024-04-25T10:17:56.582 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-04-25T10:17:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2024-04-25T10:17:56.582 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:17:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2024-04-25T10:17:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2024-04-25T10:17:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2024-04-25T10:17:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2024-04-25T10:17:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2024-04-25T10:17:56.583 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2024-04-25T10:17:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2024-04-25T10:17:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:17:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2024-04-25T10:17:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:17:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:17:56.584 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2024-04-25T10:17:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2024-04-25T10:17:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2024-04-25T10:17:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:17:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:17:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2024-04-25T10:17:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:17:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2024-04-25T10:17:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:17:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:17:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2024-04-25T10:17:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.585 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:17:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2024-04-25T10:17:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2024-04-25T10:17:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2024-04-25T10:17:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2024-04-25T10:17:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2024-04-25T10:17:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.119,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2024-04-25T10:17:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on skipped message = False 2024-04-25T10:17:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:17:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.586 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2024-04-25T10:17:56.586 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2024-04-25T10:17:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2024-04-25T10:17:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2024-04-25T10:17:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:17:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2024-04-25T10:17:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2024-04-25T10:17:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2024-04-25T10:17:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2024-04-25T10:17:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2024-04-25T10:17:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2024-04-25T10:17:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.587 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: bdev async discard = True 2024-04-25T10:17:56.588 INFO:teuthology.orchestra.run.smithi119.stdout: bdev enable discard = True 2024-04-25T10:17:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore allocator = bitmap 2024-04-25T10:17:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2024-04-25T10:17:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2024-04-25T10:17:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 1/20 2024-04-25T10:17:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 1/20 2024-04-25T10:17:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:17:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2024-04-25T10:17:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 4/10 2024-04-25T10:17:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:17:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2024-04-25T10:17:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:17:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:17:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2024-04-25T10:17:56.589 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2024-04-25T10:17:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.590 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2024-04-25T10:17:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:17:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2024-04-25T10:17:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-04-25T10:17:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-04-25T10:17:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2024-04-25T10:17:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.590 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2024-04-25T10:17:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:17:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-04-25T10:17:56.590 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2024-04-25T10:17:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-04-25T10:17:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2024-04-25T10:17:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2024-04-25T10:17:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2024-04-25T10:17:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2024-04-25T10:17:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:17:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:17:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:17:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:17:56.591 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: mon op complaint time = 120 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:17:56.592 INFO:teuthology.orchestra.run.smithi119.stdout: client mount timeout = 600 2024-04-25T10:17:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: debug client = 20 2024-04-25T10:17:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:17:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = "" 2024-04-25T10:17:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2024-04-25T10:17:56.593 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2024-04-25T10:17:56.593 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2024-04-25T10:17:56.594 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2024-04-25T10:17:56.594 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2024-04-25T10:17:56.594 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2024-04-25T10:17:56.594 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2024-04-25T10:17:56.594 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2024-04-25T10:17:56.594 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2024-04-25T10:17:56.595 INFO:teuthology.orchestra.run.smithi119.stdout:[mds] 2024-04-25T10:17:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds = 20 2024-04-25T10:17:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds balancer = 20 2024-04-25T10:17:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:17:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug frag = True 2024-04-25T10:17:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug scatterstat = True 2024-04-25T10:17:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: mds op complaint time = 180 2024-04-25T10:17:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: mds verify scatter = False 2024-04-25T10:17:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2024-04-25T10:17:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2024-04-25T10:17:56.595 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2024-04-25T10:17:56.611 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2024-04-25T10:17:56.611 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2024-04-25T10:17:56.611 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:17:56.612 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2024-04-25T10:17:56.612 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.612 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2024-04-25T10:17:56.612 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.612 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2024-04-25T10:17:56.612 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.612 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2024-04-25T10:17:56.612 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2024-04-25T10:17:56.612 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.612 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2024-04-25T10:17:56.612 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2024-04-25T10:17:56.612 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.612 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2024-04-25T10:17:56.613 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:17:56.614 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:17:56.614 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2024-04-25T10:17:56.614 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.614 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:17:56.614 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.614 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2024-04-25T10:17:56.614 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.614 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2024-04-25T10:17:56.614 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.614 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2024-04-25T10:17:56.614 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2024-04-25T10:17:56.615 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2024-04-25T10:17:56.615 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.119,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2024-04-25T10:17:56.615 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2024-04-25T10:17:56.615 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:17:56.615 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.615 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2024-04-25T10:17:56.615 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2024-04-25T10:17:56.615 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.615 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2024-04-25T10:17:56.615 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2024-04-25T10:17:56.615 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:17:56.615 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.615 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2024-04-25T10:17:56.616 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2024-04-25T10:17:56.616 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.616 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2024-04-25T10:17:56.616 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2024-04-25T10:17:56.616 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2024-04-25T10:17:56.616 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug trim objects = true 2024-04-25T10:17:56.616 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.616 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2024-04-25T10:17:56.616 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2024-04-25T10:17:56.616 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.616 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:17:56.616 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.616 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2024-04-25T10:17:56.616 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.617 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2024-04-25T10:17:56.617 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.617 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:17:56.617 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2024-04-25T10:17:56.617 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.617 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2024-04-25T10:17:56.617 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2024-04-25T10:17:56.617 INFO:teuthology.orchestra.run.smithi132.stdout: bdev async discard = True 2024-04-25T10:17:56.617 INFO:teuthology.orchestra.run.smithi132.stdout: bdev enable discard = True 2024-04-25T10:17:56.617 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore allocator = bitmap 2024-04-25T10:17:56.617 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2024-04-25T10:17:56.617 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2024-04-25T10:17:56.617 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 1/20 2024-04-25T10:17:56.617 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 1/20 2024-04-25T10:17:56.618 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:17:56.618 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2024-04-25T10:17:56.618 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 4/10 2024-04-25T10:17:56.618 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:17:56.618 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2024-04-25T10:17:56.618 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:17:56.618 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:17:56.618 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2024-04-25T10:17:56.618 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2024-04-25T10:17:56.618 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.618 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2024-04-25T10:17:56.618 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:17:56.618 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2024-04-25T10:17:56.618 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-04-25T10:17:56.619 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-04-25T10:17:56.619 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:56.619 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:56.619 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2024-04-25T10:17:56.619 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.619 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2024-04-25T10:17:56.619 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:17:56.619 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-04-25T10:17:56.619 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2024-04-25T10:17:56.619 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-04-25T10:17:56.619 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2024-04-25T10:17:56.619 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2024-04-25T10:17:56.619 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:56.619 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2024-04-25T10:17:56.620 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2024-04-25T10:17:56.620 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:56.620 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.620 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:17:56.620 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:17:56.620 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:17:56.620 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.620 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:17:56.620 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:17:56.620 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:17:56.620 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.620 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2024-04-25T10:17:56.620 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:17:56.620 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.621 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:17:56.621 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2024-04-25T10:17:56.621 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.621 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2024-04-25T10:17:56.621 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2024-04-25T10:17:56.621 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2024-04-25T10:17:56.621 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2024-04-25T10:17:56.621 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:17:56.621 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:17:56.621 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2024-04-25T10:17:56.621 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2024-04-25T10:17:56.622 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:17:56.622 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2024-04-25T10:17:56.622 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2024-04-25T10:17:56.622 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2024-04-25T10:17:56.622 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2024-04-25T10:17:56.622 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2024-04-25T10:17:56.622 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2024-04-25T10:17:56.622 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2024-04-25T10:17:56.622 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2024-04-25T10:17:56.622 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2024-04-25T10:17:56.622 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2024-04-25T10:17:56.622 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2024-04-25T10:17:56.622 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2024-04-25T10:17:56.622 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2024-04-25T10:17:56.623 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:17:56.623 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2024-04-25T10:17:56.623 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = True 2024-04-25T10:17:56.623 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2024-04-25T10:17:56.623 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = False 2024-04-25T10:17:56.623 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2024-04-25T10:17:56.623 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2024-04-25T10:17:56.623 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2024-04-25T10:17:56.646 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:17:56.656 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:17:56.782 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-04-25T10:17:56.782 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2024-04-25T10:17:56.782 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:17:56.783 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2024-04-25T10:17:56.783 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.783 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2024-04-25T10:17:56.783 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.783 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2024-04-25T10:17:56.783 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.783 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2024-04-25T10:17:56.783 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2024-04-25T10:17:56.783 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.783 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2024-04-25T10:17:56.783 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2024-04-25T10:17:56.783 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.783 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2024-04-25T10:17:56.785 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:17:56.785 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:17:56.785 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2024-04-25T10:17:56.785 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.785 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:17:56.785 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.785 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2024-04-25T10:17:56.785 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.785 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2024-04-25T10:17:56.785 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.785 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2024-04-25T10:17:56.785 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2024-04-25T10:17:56.785 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2024-04-25T10:17:56.785 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.119,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on skipped message = False 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:17:56.787 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi119.stdout: bdev async discard = True 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi119.stdout: bdev enable discard = True 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore allocator = bitmap 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 1/20 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 1/20 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 4/10 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:17:56.789 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2024-04-25T10:17:56.789 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:17:56.789 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:17:56.789 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2024-04-25T10:17:56.789 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2024-04-25T10:17:56.789 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.789 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2024-04-25T10:17:56.789 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:17:56.789 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2024-04-25T10:17:56.789 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-04-25T10:17:56.789 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-04-25T10:17:56.789 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:56.789 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:56.789 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2024-04-25T10:17:56.790 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.790 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2024-04-25T10:17:56.790 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:17:56.790 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-04-25T10:17:56.790 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2024-04-25T10:17:56.790 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-04-25T10:17:56.790 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2024-04-25T10:17:56.790 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2024-04-25T10:17:56.790 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:56.790 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2024-04-25T10:17:56.790 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2024-04-25T10:17:56.790 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:56.790 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.790 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:17:56.791 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:17:56.791 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:17:56.791 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.791 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:17:56.791 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:17:56.791 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:17:56.791 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.791 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2024-04-25T10:17:56.791 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:17:56.791 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.791 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:17:56.791 INFO:teuthology.orchestra.run.smithi119.stdout: mon op complaint time = 120 2024-04-25T10:17:56.791 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:56.791 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2024-04-25T10:17:56.792 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2024-04-25T10:17:56.792 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2024-04-25T10:17:56.792 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2024-04-25T10:17:56.792 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:17:56.792 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:17:56.792 INFO:teuthology.orchestra.run.smithi119.stdout: client mount timeout = 600 2024-04-25T10:17:56.792 INFO:teuthology.orchestra.run.smithi119.stdout: debug client = 20 2024-04-25T10:17:56.792 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:17:56.792 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = "" 2024-04-25T10:17:56.792 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2024-04-25T10:17:56.792 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2024-04-25T10:17:56.792 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2024-04-25T10:17:56.792 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2024-04-25T10:17:56.792 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout:[mds] 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds = 20 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds balancer = 20 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug frag = True 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug scatterstat = False 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout: mds op complaint time = 180 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout: mds verify scatter = False 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2024-04-25T10:17:56.793 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2024-04-25T10:17:56.798 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2024-04-25T10:17:56.798 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2024-04-25T10:17:56.798 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:17:56.798 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2024-04-25T10:17:56.798 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.798 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:17:56.799 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.800 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2024-04-25T10:17:56.800 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.800 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2024-04-25T10:17:56.800 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2024-04-25T10:17:56.800 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:17:56.800 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:17:56.800 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2024-04-25T10:17:56.800 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:17:56.800 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2024-04-25T10:17:56.800 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:17:56.800 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:17:56.800 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2024-04-25T10:17:56.800 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.800 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.119,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.801 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2024-04-25T10:17:56.802 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2024-04-25T10:17:56.802 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:17:56.802 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.802 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2024-04-25T10:17:56.802 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2024-04-25T10:17:56.802 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.802 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2024-04-25T10:17:56.802 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2024-04-25T10:17:56.802 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2024-04-25T10:17:56.802 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug trim objects = true 2024-04-25T10:17:56.802 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.802 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2024-04-25T10:17:56.802 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2024-04-25T10:17:56.802 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.803 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:17:56.803 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.803 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2024-04-25T10:17:56.803 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.803 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2024-04-25T10:17:56.803 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.803 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:17:56.803 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2024-04-25T10:17:56.803 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.803 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2024-04-25T10:17:56.803 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2024-04-25T10:17:56.803 INFO:teuthology.orchestra.run.smithi132.stdout: bdev async discard = True 2024-04-25T10:17:56.803 INFO:teuthology.orchestra.run.smithi132.stdout: bdev enable discard = True 2024-04-25T10:17:56.803 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore allocator = bitmap 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 1/20 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 1/20 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 4/10 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.804 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2024-04-25T10:17:56.805 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:17:56.805 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2024-04-25T10:17:56.805 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-04-25T10:17:56.805 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-04-25T10:17:56.805 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:56.805 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:56.805 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2024-04-25T10:17:56.805 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.805 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2024-04-25T10:17:56.805 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:17:56.805 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-04-25T10:17:56.805 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2024-04-25T10:17:56.805 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-04-25T10:17:56.805 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.807 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2024-04-25T10:17:56.807 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:17:56.807 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.807 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:17:56.807 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2024-04-25T10:17:56.807 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:17:56.807 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2024-04-25T10:17:56.807 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2024-04-25T10:17:56.807 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2024-04-25T10:17:56.807 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2024-04-25T10:17:56.807 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:17:56.807 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:17:56.807 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2024-04-25T10:17:56.807 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2024-04-25T10:17:56.808 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:17:56.809 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2024-04-25T10:17:56.809 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = False 2024-04-25T10:17:56.809 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2024-04-25T10:17:56.809 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = False 2024-04-25T10:17:56.809 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2024-04-25T10:17:56.809 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2024-04-25T10:17:56.809 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2024-04-25T10:17:56.833 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-25T10:17:56.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.928+0000 7fa1bd52cf40 1 Processor -- start 2024-04-25T10:17:56.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.928+0000 7fa1bd52cf40 1 -- start start 2024-04-25T10:17:56.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.929+0000 7fa1bd52cf40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b7d493220 0x556b7d55d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.929+0000 7fa1bd52cf40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b7d4de080 0x556b7d56dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.929+0000 7fa1bd52cf40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b7d56ac70 0x556b7d492410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.929+0000 7fa1bd52cf40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b7d4c62f0 con 0x556b7d56ac70 2024-04-25T10:17:56.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.929+0000 7fa1bd52cf40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b7d43add0 con 0x556b7d493220 2024-04-25T10:17:56.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.929+0000 7fa1bd52cf40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b7d493960 con 0x556b7d4de080 2024-04-25T10:17:56.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.930+0000 7fa1ba600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b7d56ac70 0x556b7d492410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.930+0000 7fa1ba600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b7d56ac70 0x556b7d492410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46582/0 (socket says 172.21.15.119:46582) 2024-04-25T10:17:56.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.930+0000 7fa1ba600640 1 -- 172.21.15.119:0/778713869 learned_addr learned my addr 172.21.15.119:0/778713869 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:56.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.930+0000 7fa1b9200640 1 --2- 172.21.15.119:0/778713869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b7d4de080 0x556b7d56dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.930+0000 7fa1b9c00640 1 --2- 172.21.15.119:0/778713869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b7d493220 0x556b7d55d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.930+0000 7fa1ba600640 1 -- 172.21.15.119:0/778713869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b7d4de080 msgr2=0x556b7d56dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:56.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.930+0000 7fa1ba600640 1 --2- 172.21.15.119:0/778713869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b7d4de080 0x556b7d56dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.930+0000 7fa1ba600640 1 -- 172.21.15.119:0/778713869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b7d493220 msgr2=0x556b7d55d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:56.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.930+0000 7fa1ba600640 1 --2- 172.21.15.119:0/778713869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b7d493220 0x556b7d55d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.930+0000 7fa1ba600640 1 -- 172.21.15.119:0/778713869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b7d4dde90 con 0x556b7d56ac70 2024-04-25T10:17:56.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.930+0000 7fa1b9c00640 1 --2- 172.21.15.119:0/778713869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b7d493220 0x556b7d55d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:56.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.930+0000 7fa1ba600640 1 --2- 172.21.15.119:0/778713869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b7d56ac70 0x556b7d492410 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40098a0 tx=0x7fa1b4005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c38aa2f87b4d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.931+0000 7fa1b3400640 1 -- 172.21.15.119:0/778713869 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4008870 con 0x556b7d56ac70 2024-04-25T10:17:56.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.931+0000 7fa1b3400640 1 -- 172.21.15.119:0/778713869 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b4018070 con 0x556b7d56ac70 2024-04-25T10:17:56.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.931+0000 7fa1b3400640 1 -- 172.21.15.119:0/778713869 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4008c50 con 0x556b7d56ac70 2024-04-25T10:17:56.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.931+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/778713869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b7d56ac70 msgr2=0x556b7d492410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.931+0000 7fa1bd52cf40 1 --2- 172.21.15.119:0/778713869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b7d56ac70 0x556b7d492410 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40098a0 tx=0x7fa1b4005a00 comp rx=0 tx=0).stop 2024-04-25T10:17:56.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.931+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/778713869 shutdown_connections 2024-04-25T10:17:56.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.931+0000 7fa1bd52cf40 1 --2- 172.21.15.119:0/778713869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b7d56ac70 0x556b7d492410 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.931+0000 7fa1bd52cf40 1 --2- 172.21.15.119:0/778713869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b7d4de080 0x556b7d56dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.931+0000 7fa1bd52cf40 1 --2- 172.21.15.119:0/778713869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b7d493220 0x556b7d55d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.931+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/778713869 >> 172.21.15.119:0/778713869 conn(0x556b7d4e1070 msgr2=0x556b7d4e1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:56.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.931+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/778713869 shutdown_connections 2024-04-25T10:17:56.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.931+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/778713869 wait complete. 2024-04-25T10:17:56.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.932+0000 7fa1bd52cf40 1 Processor -- start 2024-04-25T10:17:56.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.932+0000 7fa1bd52cf40 1 -- start start 2024-04-25T10:17:56.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.933+0000 7fa1bd52cf40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b7d493220 0x556b7d559f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.933+0000 7fa1bd52cf40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b7d4de080 0x556b7d550db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.933+0000 7fa1bd52cf40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b7d56ac70 0x556b7d492410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.933+0000 7fa1bd52cf40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b7d56e230 con 0x556b7d4de080 2024-04-25T10:17:56.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.933+0000 7fa1bd52cf40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b7d55d6b0 con 0x556b7d493220 2024-04-25T10:17:56.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.933+0000 7fa1bd52cf40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b7d56d400 con 0x556b7d56ac70 2024-04-25T10:17:56.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.933+0000 7fa1b9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b7d4de080 0x556b7d550db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.933+0000 7fa1b9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b7d4de080 0x556b7d550db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46592/0 (socket says 172.21.15.119:46592) 2024-04-25T10:17:56.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.933+0000 7fa1b9200640 1 -- 172.21.15.119:0/799455022 learned_addr learned my addr 172.21.15.119:0/799455022 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:56.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.934+0000 7fa1b9c00640 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b7d493220 0x556b7d559f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.934+0000 7fa1ba600640 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b7d56ac70 0x556b7d492410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.934+0000 7fa1b9200640 1 -- 172.21.15.119:0/799455022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b7d56ac70 msgr2=0x556b7d492410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.934+0000 7fa1b9200640 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b7d56ac70 0x556b7d492410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.934+0000 7fa1b9200640 1 -- 172.21.15.119:0/799455022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b7d493220 msgr2=0x556b7d559f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.934+0000 7fa1b9200640 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b7d493220 0x556b7d559f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.934+0000 7fa1b9200640 1 -- 172.21.15.119:0/799455022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b7d559a50 con 0x556b7d4de080 2024-04-25T10:17:56.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.934+0000 7fa1ba600640 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556b7d56ac70 0x556b7d492410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:56.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.934+0000 7fa1b9c00640 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556b7d493220 0x556b7d559f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:56.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.934+0000 7fa1b9200640 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b7d4de080 0x556b7d550db0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8000f60 tx=0x7fa1a800a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.934+0000 7fa1b2000640 1 -- 172.21.15.119:0/799455022 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800e640 con 0x556b7d4de080 2024-04-25T10:17:56.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.934+0000 7fa1b2000640 1 -- 172.21.15.119:0/799455022 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a800e7e0 con 0x556b7d4de080 2024-04-25T10:17:56.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.935+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/799455022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556b7d56bf50 con 0x556b7d4de080 2024-04-25T10:17:56.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.935+0000 7fa1b2000640 1 -- 172.21.15.119:0/799455022 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800ebb0 con 0x556b7d4de080 2024-04-25T10:17:56.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.935+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/799455022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b7d56c510 con 0x556b7d4de080 2024-04-25T10:17:56.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.937+0000 7fa1b2000640 1 -- 172.21.15.119:0/799455022 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1a8009030 con 0x556b7d4de080 2024-04-25T10:17:56.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.937+0000 7fa1b2000640 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa19c069ee0 0x7fa19c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.938+0000 7fa1b2000640 1 -- 172.21.15.119:0/799455022 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa1a80891f0 con 0x556b7d4de080 2024-04-25T10:17:56.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.938+0000 7fa1b9c00640 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa19c069ee0 0x7fa19c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.938+0000 7fa1bd52cf40 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x556b7d5586f0 0x556b7d558b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.938+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/799455022 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) v9 -- 0x556b7d4c5d40 con 0x556b7d5586f0 2024-04-25T10:17:56.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.938+0000 7fa1ba600640 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x556b7d5586f0 0x556b7d558b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.938+0000 7fa1b9c00640 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa19c069ee0 0x7fa19c06c3a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac057000 tx=0x7fa1ac05e040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.939+0000 7fa1ba600640 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x556b7d5586f0 0x556b7d558b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.940+0000 7fa1ba600640 1 -- 172.21.15.119:0/799455022 <== osd.3 v2:172.21.15.119:6816/1142971786 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v111'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa1b40088a0 con 0x556b7d5586f0 2024-04-25T10:17:56.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.940+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/799455022 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x556b7d5586f0 msgr2=0x556b7d558b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.940+0000 7fa1bd52cf40 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x556b7d5586f0 0x556b7d558b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.940+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/799455022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa19c069ee0 msgr2=0x7fa19c06c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.940+0000 7fa1bd52cf40 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa19c069ee0 0x7fa19c06c3a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac057000 tx=0x7fa1ac05e040 comp rx=0 tx=0).stop 2024-04-25T10:17:56.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.940+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/799455022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b7d4de080 msgr2=0x556b7d550db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.940+0000 7fa1bd52cf40 1 --2- 172.21.15.119:0/799455022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b7d4de080 0x556b7d550db0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8000f60 tx=0x7fa1a800a6b0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.940+0000 7fa1ba600640 1 -- 172.21.15.119:0/799455022 reap_dead start 2024-04-25T10:17:56.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.941+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/799455022 shutdown_connections 2024-04-25T10:17:56.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.941+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/799455022 >> 172.21.15.119:0/799455022 conn(0x556b7d4e1070 msgr2=0x556b7d4e1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:56.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.941+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/799455022 shutdown_connections 2024-04-25T10:17:56.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:56.941+0000 7fa1bd52cf40 1 -- 172.21.15.119:0/799455022 wait complete. 2024-04-25T10:17:56.949 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:17:57.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.137+0000 7fb65b200640 1 Processor -- start 2024-04-25T10:17:57.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.137+0000 7fb65b200640 1 -- start start 2024-04-25T10:17:57.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.139+0000 7fb65b200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb65c06b2b0 0x7fb65c0acb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.139+0000 7fb65b200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65c15e120 con 0x7fb65c0b2f90 2024-04-25T10:17:57.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.139+0000 7fb65b200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65c15e400 con 0x7fb65c0ada20 2024-04-25T10:17:57.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.139+0000 7fb65b200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65c15dec0 con 0x7fb65c06b2b0 2024-04-25T10:17:57.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.140+0000 7fb65a800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb65c0b2f90 0x7fb65c0b33a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50550/0 (socket says 172.21.15.119:50550) 2024-04-25T10:17:57.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.140+0000 7fb65a800640 1 -- 172.21.15.119:0/1527001262 learned_addr learned my addr 172.21.15.119:0/1527001262 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:57.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.140+0000 7fb659400640 1 --2- 172.21.15.119:0/1527001262 >> v2:172.21.15.132:3300/0 conn(0x7fb65c06b2b0 0x7fb65c0acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.140+0000 7fb658a00640 1 -- 172.21.15.119:0/1527001262 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677924339 0 0) 0x7fb65c15e120 con 0x7fb65c0b2f90 2024-04-25T10:17:57.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.140+0000 7fb658a00640 1 -- 172.21.15.119:0/1527001262 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb640003520 con 0x7fb65c0b2f90 2024-04-25T10:17:57.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.140+0000 7fb659400640 1 -- 172.21.15.119:0/1527001262 >> v1:172.21.15.132:6789/0 conn(0x7fb65c0ada20 legacy=0x7fb65c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.140+0000 7fb659400640 1 -- 172.21.15.119:0/1527001262 >> v1:172.21.15.119:6789/0 conn(0x7fb65c0b2f90 legacy=0x7fb65c0b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.140+0000 7fb659400640 1 -- 172.21.15.119:0/1527001262 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c15e6e0 con 0x7fb65c06b2b0 2024-04-25T10:17:57.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.142+0000 7fb659400640 1 --2- 172.21.15.119:0/1527001262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb65c06b2b0 0x7fb65c0acb20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb6440097a0 tx=0x7fb64402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=567f6a2e342bdc44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:57.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.142+0000 7fb658a00640 1 -- 172.21.15.119:0/1527001262 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644037070 con 0x7fb65c06b2b0 2024-04-25T10:17:57.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.142+0000 7fb658a00640 1 -- 172.21.15.119:0/1527001262 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb644032440 con 0x7fb65c06b2b0 2024-04-25T10:17:57.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.143+0000 7fb658a00640 1 -- 172.21.15.119:0/1527001262 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6440327f0 con 0x7fb65c06b2b0 2024-04-25T10:17:57.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.143+0000 7fb65b200640 1 -- 172.21.15.119:0/1527001262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb65c06b2b0 msgr2=0x7fb65c0acb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.143+0000 7fb65b200640 1 --2- 172.21.15.119:0/1527001262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb65c06b2b0 0x7fb65c0acb20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb6440097a0 tx=0x7fb64402a230 comp rx=0 tx=0).stop 2024-04-25T10:17:57.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.143+0000 7fb65b200640 1 -- 172.21.15.119:0/1527001262 shutdown_connections 2024-04-25T10:17:57.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.143+0000 7fb65b200640 1 --2- 172.21.15.119:0/1527001262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb65c06b2b0 0x7fb65c0acb20 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.143+0000 7fb65b200640 1 -- 172.21.15.119:0/1527001262 >> 172.21.15.119:0/1527001262 conn(0x7fb65c066ff0 msgr2=0x7fb65c0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:57.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.144+0000 7fb65b200640 1 -- 172.21.15.119:0/1527001262 shutdown_connections 2024-04-25T10:17:57.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.144+0000 7fb65b200640 1 -- 172.21.15.119:0/1527001262 wait complete. 2024-04-25T10:17:57.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.146+0000 7fb65b200640 1 Processor -- start 2024-04-25T10:17:57.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.146+0000 7fb65b200640 1 -- start start 2024-04-25T10:17:57.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.147+0000 7fb65b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb65c06b2b0 0x7fb65c1500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.147+0000 7fb65b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb65c0ada20 0x7fb65c143d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.147+0000 7fb65b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb65c0b2f90 0x7fb65c06e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.147+0000 7fb65b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c0ade00 con 0x7fb65c0ada20 2024-04-25T10:17:57.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.147+0000 7fb65b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c15eca0 con 0x7fb65c0b2f90 2024-04-25T10:17:57.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.147+0000 7fb65b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c157700 con 0x7fb65c06b2b0 2024-04-25T10:17:57.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.147+0000 7fb659400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb65c0ada20 0x7fb65c143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.147+0000 7fb659400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb65c0ada20 0x7fb65c143d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46596/0 (socket says 172.21.15.119:46596) 2024-04-25T10:17:57.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.147+0000 7fb659400640 1 -- 172.21.15.119:0/2644953638 learned_addr learned my addr 172.21.15.119:0/2644953638 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:57.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.148+0000 7fb659e00640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb65c06b2b0 0x7fb65c1500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.148+0000 7fb65a800640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb65c0b2f90 0x7fb65c06e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.148+0000 7fb659400640 1 -- 172.21.15.119:0/2644953638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb65c06b2b0 msgr2=0x7fb65c1500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.148+0000 7fb659400640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb65c06b2b0 0x7fb65c1500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.148+0000 7fb659400640 1 -- 172.21.15.119:0/2644953638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb65c0b2f90 msgr2=0x7fb65c06e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.148+0000 7fb659400640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb65c0b2f90 0x7fb65c06e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.148+0000 7fb659400640 1 -- 172.21.15.119:0/2644953638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c14ec90 con 0x7fb65c0ada20 2024-04-25T10:17:57.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.148+0000 7fb65a800640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb65c0b2f90 0x7fb65c06e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:57.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.148+0000 7fb659400640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb65c0ada20 0x7fb65c143d80 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb644035040 tx=0x7fb6440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:57.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.149+0000 7fb652000640 1 -- 172.21.15.119:0/2644953638 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644037050 con 0x7fb65c0ada20 2024-04-25T10:17:57.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.149+0000 7fb652000640 1 -- 172.21.15.119:0/2644953638 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb644038040 con 0x7fb65c0ada20 2024-04-25T10:17:57.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.149+0000 7fb65b200640 1 -- 172.21.15.119:0/2644953638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65c14ef70 con 0x7fb65c0ada20 2024-04-25T10:17:57.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.149+0000 7fb652000640 1 -- 172.21.15.119:0/2644953638 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644004470 con 0x7fb65c0ada20 2024-04-25T10:17:57.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.149+0000 7fb65b200640 1 -- 172.21.15.119:0/2644953638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c14f530 con 0x7fb65c0ada20 2024-04-25T10:17:57.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.150+0000 7fb65b200640 1 -- 172.21.15.119:0/2644953638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65c0749b0 con 0x7fb65c0ada20 2024-04-25T10:17:57.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.151+0000 7fb652000640 1 -- 172.21.15.119:0/2644953638 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb64403c6d0 con 0x7fb65c0ada20 2024-04-25T10:17:57.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.152+0000 7fb652000640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb63806b4d0 0x7fb63806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.152+0000 7fb652000640 1 -- 172.21.15.119:0/2644953638 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb6440aae00 con 0x7fb65c0ada20 2024-04-25T10:17:57.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.152+0000 7fb659e00640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb63806b4d0 0x7fb63806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.153+0000 7fb659e00640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb63806b4d0 0x7fb63806d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb64c007fd0 tx=0x7fb64c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:57.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.158+0000 7fb652000640 1 -- 172.21.15.119:0/2644953638 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb644079ad0 con 0x7fb65c0ada20 2024-04-25T10:17:57.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.395+0000 7fb65b200640 1 -- 172.21.15.119:0/2644953638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fb65c06b060 con 0x7fb65c0ada20 2024-04-25T10:17:57.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.445+0000 7fb652000640 1 -- 172.21.15.119:0/2644953638 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v101) v1 ==== 162+0+0 (secure 0 0 0) 0x7fb6440793d0 con 0x7fb65c0ada20 2024-04-25T10:17:57.448 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:17:57.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.448+0000 7fb65b200640 1 -- 172.21.15.119:0/2644953638 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb63806b4d0 msgr2=0x7fb63806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.448+0000 7fb65b200640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb63806b4d0 0x7fb63806d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb64c007fd0 tx=0x7fb64c009040 comp rx=0 tx=0).stop 2024-04-25T10:17:57.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.448+0000 7fb65b200640 1 -- 172.21.15.119:0/2644953638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb65c0ada20 msgr2=0x7fb65c143d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.448+0000 7fb65b200640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb65c0ada20 0x7fb65c143d80 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb644035040 tx=0x7fb6440046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.448+0000 7fb65b200640 1 -- 172.21.15.119:0/2644953638 shutdown_connections 2024-04-25T10:17:57.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.448+0000 7fb65b200640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fb63806b4d0 0x7fb63806d990 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.448+0000 7fb65b200640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb65c0b2f90 0x7fb65c06e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.448+0000 7fb65b200640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb65c0ada20 0x7fb65c143d80 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.448+0000 7fb65b200640 1 --2- 172.21.15.119:0/2644953638 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb65c06b2b0 0x7fb65c1500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.448+0000 7fb65b200640 1 -- 172.21.15.119:0/2644953638 >> 172.21.15.119:0/2644953638 conn(0x7fb65c066ff0 msgr2=0x7fb65c0b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:57.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.448+0000 7fb65b200640 1 -- 172.21.15.119:0/2644953638 shutdown_connections 2024-04-25T10:17:57.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.448+0000 7fb65b200640 1 -- 172.21.15.119:0/2644953638 wait complete. 2024-04-25T10:17:57.475 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:57.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.649+0000 7fbfc4c00640 1 Processor -- start 2024-04-25T10:17:57.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.649+0000 7fbfc4c00640 1 -- start start 2024-04-25T10:17:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.650+0000 7fbfc4c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbfc015acd0 0x7fbfc006e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.650+0000 7fbfc4c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc006ec70 con 0x7fbfc006e860 2024-04-25T10:17:57.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.651+0000 7fbfc4c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc0144420 con 0x7fbfc0144040 2024-04-25T10:17:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.651+0000 7fbfc4c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc00b3d80 con 0x7fbfc015acd0 2024-04-25T10:17:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.651+0000 7fbfbfe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbfc006e860 0x7fbfc00b26c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50552/0 (socket says 172.21.15.119:50552) 2024-04-25T10:17:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.651+0000 7fbfbfe00640 1 -- 172.21.15.119:0/1056985377 learned_addr learned my addr 172.21.15.119:0/1056985377 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:17:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.651+0000 7fbfbea00640 1 --2- 172.21.15.119:0/1056985377 >> v2:172.21.15.132:3300/0 conn(0x7fbfc015acd0 0x7fbfc006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.651+0000 7fbfbe000640 1 -- 172.21.15.119:0/1056985377 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252268544 0 0) 0x7fbfb8001490 con 0x7fbfc006e860 2024-04-25T10:17:57.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.651+0000 7fbfbe000640 1 -- 172.21.15.119:0/1056985377 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfa4003520 con 0x7fbfc006e860 2024-04-25T10:17:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.652+0000 7fbfbe000640 1 -- 172.21.15.119:0/1056985377 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4290209910 0 0) 0x7fbfa4003520 con 0x7fbfc006e860 2024-04-25T10:17:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.652+0000 7fbfbe000640 1 -- 172.21.15.119:0/1056985377 >> v2:172.21.15.132:3300/0 conn(0x7fbfc015acd0 msgr2=0x7fbfc006e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.652+0000 7fbfbe000640 1 --2- 172.21.15.119:0/1056985377 >> v2:172.21.15.132:3300/0 conn(0x7fbfc015acd0 0x7fbfc006e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.652+0000 7fbfbe000640 1 -- 172.21.15.119:0/1056985377 >> v1:172.21.15.132:6789/0 conn(0x7fbfc0144040 legacy=0x7fbfc00cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.652+0000 7fbfbe000640 1 -- 172.21.15.119:0/1056985377 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc00b4060 con 0x7fbfc006e860 2024-04-25T10:17:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.652+0000 7fbfbe000640 1 -- 172.21.15.119:0/1056985377 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fbfb8001bc0 con 0x7fbfc006e860 2024-04-25T10:17:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.652+0000 7fbfbe000640 1 -- 172.21.15.119:0/1056985377 >> v1:172.21.15.119:6789/0 conn(0x7fbfc006e860 legacy=0x7fbfc00b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.652+0000 7fbfbe000640 1 --2- 172.21.15.119:0/1056985377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa4003a70 0x7fbfa4003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.652+0000 7fbfbe000640 1 --2- 172.21.15.119:0/1056985377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfa4004800 0x7fbfa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.652+0000 7fbfbe000640 1 --2- 172.21.15.119:0/1056985377 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbfa40052e0 0x7fbfa400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.652+0000 7fbfbe000640 1 -- 172.21.15.119:0/1056985377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00b3d80 con 0x7fbfa4004800 2024-04-25T10:17:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.652+0000 7fbfbe000640 1 -- 172.21.15.119:0/1056985377 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa400c1b0 con 0x7fbfa4003a70 2024-04-25T10:17:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.652+0000 7fbfbe000640 1 -- 172.21.15.119:0/1056985377 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa400c3b0 con 0x7fbfa40052e0 2024-04-25T10:17:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.652+0000 7fbfbea00640 1 --2- 172.21.15.119:0/1056985377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfa4004800 0x7fbfa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.653+0000 7fbfbf400640 1 --2- 172.21.15.119:0/1056985377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa4003a70 0x7fbfa4003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.653+0000 7fbfbea00640 1 -- 172.21.15.119:0/1056985377 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbfa40052e0 msgr2=0x7fbfa400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.653+0000 7fbfbea00640 1 --2- 172.21.15.119:0/1056985377 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbfa40052e0 0x7fbfa400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.653+0000 7fbfbea00640 1 -- 172.21.15.119:0/1056985377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa4003a70 msgr2=0x7fbfa4003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.653+0000 7fbfbea00640 1 --2- 172.21.15.119:0/1056985377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa4003a70 0x7fbfa4003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.653+0000 7fbfbea00640 1 -- 172.21.15.119:0/1056985377 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa4003520 con 0x7fbfa4004800 2024-04-25T10:17:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.653+0000 7fbfbfe00640 1 -- 172.21.15.119:0/1056985377 reap_dead start 2024-04-25T10:17:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.654+0000 7fbfbea00640 1 --2- 172.21.15.119:0/1056985377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfa4004800 0x7fbfa4004c80 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009ac0 tx=0x7fbfa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef99f89d0cc6f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:57.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.654+0000 7fbfbe000640 1 -- 172.21.15.119:0/1056985377 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac036070 con 0x7fbfa4004800 2024-04-25T10:17:57.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.654+0000 7fbfbe000640 1 -- 172.21.15.119:0/1056985377 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac031400 con 0x7fbfa4004800 2024-04-25T10:17:57.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.654+0000 7fbfbe000640 1 -- 172.21.15.119:0/1056985377 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac009690 con 0x7fbfa4004800 2024-04-25T10:17:57.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.654+0000 7fbfc4c00640 1 -- 172.21.15.119:0/1056985377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfa4004800 msgr2=0x7fbfa4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.654+0000 7fbfc4c00640 1 --2- 172.21.15.119:0/1056985377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfa4004800 0x7fbfa4004c80 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009ac0 tx=0x7fbfa4005830 comp rx=0 tx=0).stop 2024-04-25T10:17:57.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.655+0000 7fbfc4c00640 1 -- 172.21.15.119:0/1056985377 shutdown_connections 2024-04-25T10:17:57.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.655+0000 7fbfc4c00640 1 --2- 172.21.15.119:0/1056985377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfa4004800 0x7fbfa4004c80 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.655+0000 7fbfc4c00640 1 -- 172.21.15.119:0/1056985377 >> 172.21.15.119:0/1056985377 conn(0x7fbfc006b1d0 msgr2=0x7fbfc00b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:57.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.655+0000 7fbfc4c00640 1 -- 172.21.15.119:0/1056985377 shutdown_connections 2024-04-25T10:17:57.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.655+0000 7fbfc4c00640 1 -- 172.21.15.119:0/1056985377 wait complete. 2024-04-25T10:17:57.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.656+0000 7fbfc4c00640 1 Processor -- start 2024-04-25T10:17:57.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.657+0000 7fbfc4c00640 1 -- start start 2024-04-25T10:17:57.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.657+0000 7fbfc4c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfa4004800 0x7fbfc013a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.657+0000 7fbfc4c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbfc015d390 0x7fbfc013c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.657+0000 7fbfc4c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfc0159840 0x7fbfc00b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.658+0000 7fbfc4c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00b4900 con 0x7fbfa4004800 2024-04-25T10:17:57.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.658+0000 7fbfc4c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00b4620 con 0x7fbfc0159840 2024-04-25T10:17:57.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.658+0000 7fbfc4c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00b5180 con 0x7fbfc015d390 2024-04-25T10:17:57.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.658+0000 7fbfbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfa4004800 0x7fbfc013a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.658+0000 7fbfbf400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfa4004800 0x7fbfc013a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46610/0 (socket says 172.21.15.119:46610) 2024-04-25T10:17:57.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.658+0000 7fbfbf400640 1 -- 172.21.15.119:0/116579186 learned_addr learned my addr 172.21.15.119:0/116579186 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:57.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.658+0000 7fbfbfe00640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfc0159840 0x7fbfc00b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.658+0000 7fbfbea00640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbfc015d390 0x7fbfc013c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.658+0000 7fbfbf400640 1 -- 172.21.15.119:0/116579186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbfc015d390 msgr2=0x7fbfc013c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.658+0000 7fbfbf400640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbfc015d390 0x7fbfc013c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.658+0000 7fbfbf400640 1 -- 172.21.15.119:0/116579186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfc0159840 msgr2=0x7fbfc00b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.658+0000 7fbfbf400640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfc0159840 0x7fbfc00b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.658+0000 7fbfbf400640 1 -- 172.21.15.119:0/116579186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc015d890 con 0x7fbfa4004800 2024-04-25T10:17:57.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.658+0000 7fbfbfe00640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfc0159840 0x7fbfc00b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:57.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.659+0000 7fbfbf400640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfa4004800 0x7fbfc013a9d0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000e580 tx=0x7fbfb000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:57.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.659+0000 7fbfb7400640 1 -- 172.21.15.119:0/116579186 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb00052e0 con 0x7fbfa4004800 2024-04-25T10:17:57.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.659+0000 7fbfb7400640 1 -- 172.21.15.119:0/116579186 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb0018070 con 0x7fbfa4004800 2024-04-25T10:17:57.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.659+0000 7fbfc4c00640 1 -- 172.21.15.119:0/116579186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc013b5c0 con 0x7fbfa4004800 2024-04-25T10:17:57.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.659+0000 7fbfb7400640 1 -- 172.21.15.119:0/116579186 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0003810 con 0x7fbfa4004800 2024-04-25T10:17:57.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.659+0000 7fbfc4c00640 1 -- 172.21.15.119:0/116579186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc013af10 con 0x7fbfa4004800 2024-04-25T10:17:57.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.660+0000 7fbfc4c00640 1 -- 172.21.15.119:0/116579186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc006bb30 con 0x7fbfa4004800 2024-04-25T10:17:57.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.661+0000 7fbfb7400640 1 -- 172.21.15.119:0/116579186 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbfb00039b0 con 0x7fbfa4004800 2024-04-25T10:17:57.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.662+0000 7fbfb7400640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbfa006b500 0x7fbfa006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.662+0000 7fbfb7400640 1 -- 172.21.15.119:0/116579186 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fbfb008a060 con 0x7fbfa4004800 2024-04-25T10:17:57.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.662+0000 7fbfbea00640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbfa006b500 0x7fbfa006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.663+0000 7fbfbea00640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbfa006b500 0x7fbfa006d9c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbfac033000 tx=0x7fbfac008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:57.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.666+0000 7fbfb7400640 1 -- 172.21.15.119:0/116579186 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbfb0058d30 con 0x7fbfa4004800 2024-04-25T10:17:57.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.893+0000 7fbfc4c00640 1 -- 172.21.15.119:0/116579186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfc00749b0 con 0x7fbfa4004800 2024-04-25T10:17:57.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.894+0000 7fbfb7400640 1 -- 172.21.15.119:0/116579186 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4957 (secure 0 0 0) 0x7fbfb0058630 con 0x7fbfa4004800 2024-04-25T10:17:57.897 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:57.898 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":102,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3394261143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3394261143},{"type":"v1","addr":"172.21.15.132:6834","nonce":3394261143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2502874931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2502874931},{"type":"v1","addr":"172.21.15.119:6835","nonce":2502874931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":8694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2305428003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2305428003},{"type":"v1","addr":"172.21.15.132:6835","nonce":2305428003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:37.617787+0000","modified":"2024-04-25T10:17:57.451117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8943":{"gid":8943,"name":"c","rank":0,"incarnation":102,"state":"up:replay","state_seq":1,"addr":"172.21.15.119:6837/2864575636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2864575636},{"type":"v1","addr":"172.21.15.119:6837","nonce":2864575636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:17:57.898 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 102 2024-04-25T10:17:57.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.896+0000 7fbfc4c00640 1 -- 172.21.15.119:0/116579186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbfa006b500 msgr2=0x7fbfa006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.896+0000 7fbfc4c00640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbfa006b500 0x7fbfa006d9c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbfac033000 tx=0x7fbfac008030 comp rx=0 tx=0).stop 2024-04-25T10:17:57.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.896+0000 7fbfc4c00640 1 -- 172.21.15.119:0/116579186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfa4004800 msgr2=0x7fbfc013a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.896+0000 7fbfc4c00640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfa4004800 0x7fbfc013a9d0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000e580 tx=0x7fbfb000ea40 comp rx=0 tx=0).stop 2024-04-25T10:17:57.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.896+0000 7fbfc4c00640 1 -- 172.21.15.119:0/116579186 shutdown_connections 2024-04-25T10:17:57.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.896+0000 7fbfc4c00640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fbfa006b500 0x7fbfa006d9c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.896+0000 7fbfc4c00640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfc0159840 0x7fbfc00b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.896+0000 7fbfc4c00640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbfc015d390 0x7fbfc013c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.896+0000 7fbfc4c00640 1 --2- 172.21.15.119:0/116579186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfa4004800 0x7fbfc013a9d0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.896+0000 7fbfc4c00640 1 -- 172.21.15.119:0/116579186 >> 172.21.15.119:0/116579186 conn(0x7fbfc006b1d0 msgr2=0x7fbfc015b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:57.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.896+0000 7fbfc4c00640 1 -- 172.21.15.119:0/116579186 shutdown_connections 2024-04-25T10:17:57.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:57.896+0000 7fbfc4c00640 1 -- 172.21.15.119:0/116579186 wait complete. 2024-04-25T10:17:58.321 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:17:37.617787+0000', 'modified': '2024-04-25T10:17:57.451117+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 111, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8943}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8943': {'gid': 8943, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.119:6837/2864575636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 2864575636}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2864575636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:17:58.321 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8943:up:replay 2024-04-25T10:17:59.322 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:59.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.526+0000 7f1f3b800640 1 Processor -- start 2024-04-25T10:17:59.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.526+0000 7f1f3b800640 1 -- start start 2024-04-25T10:17:59.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.527+0000 7f1f3b800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1f3415bf00 0x7f1f340cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.528+0000 7f1f3b800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1f340cc310 0x7f1f3415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.528+0000 7f1f3b800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f340ce6c0 con 0x7f1f3415c820 2024-04-25T10:17:59.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.528+0000 7f1f3b800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f341455e0 con 0x7f1f340cc310 2024-04-25T10:17:59.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.528+0000 7f1f3b800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f3406d450 con 0x7f1f3415bf00 2024-04-25T10:17:59.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.528+0000 7f1f33e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1f340cc310 0x7f1f3415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.528+0000 7f1f33e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1f340cc310 0x7f1f3415c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46624/0 (socket says 172.21.15.119:46624) 2024-04-25T10:17:59.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.528+0000 7f1f33e00640 1 -- 172.21.15.119:0/3400800721 learned_addr learned my addr 172.21.15.119:0/3400800721 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:59.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.528+0000 7f1f38e00640 1 --2- 172.21.15.119:0/3400800721 >> v2:172.21.15.132:3300/0 conn(0x7f1f3415bf00 0x7f1f340cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.528+0000 7f1f33e00640 1 -- 172.21.15.119:0/3400800721 >> v2:172.21.15.132:3300/0 conn(0x7f1f3415bf00 msgr2=0x7f1f340cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:59.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.528+0000 7f1f33400640 1 -- 172.21.15.119:0/3400800721 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1355839980 0 0) 0x7f1f2c001490 con 0x7f1f3415c820 2024-04-25T10:17:59.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.528+0000 7f1f33e00640 1 --2- 172.21.15.119:0/3400800721 >> v2:172.21.15.132:3300/0 conn(0x7f1f3415bf00 0x7f1f340cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.529+0000 7f1f33e00640 1 -- 172.21.15.119:0/3400800721 >> v1:172.21.15.119:6789/0 conn(0x7f1f3415c820 legacy=0x7f1f3406d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.529+0000 7f1f33e00640 1 -- 172.21.15.119:0/3400800721 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f340cc7b0 con 0x7f1f340cc310 2024-04-25T10:17:59.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.529+0000 7f1f38e00640 1 --2- 172.21.15.119:0/3400800721 >> v2:172.21.15.132:3300/0 conn(0x7f1f3415bf00 0x7f1f340cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.530+0000 7f1f33e00640 1 --2- 172.21.15.119:0/3400800721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f340cc310 0x7f1f3415c2e0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f1f3406ba10 tx=0x7f1f2002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cff55c44ab420c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.530+0000 7f1f33400640 1 -- 172.21.15.119:0/3400800721 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f20037070 con 0x7f1f340cc310 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.530+0000 7f1f33400640 1 -- 172.21.15.119:0/3400800721 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f20032440 con 0x7f1f340cc310 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.530+0000 7f1f33400640 1 -- 172.21.15.119:0/3400800721 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f20032810 con 0x7f1f340cc310 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.531+0000 7f1f3b800640 1 -- 172.21.15.119:0/3400800721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f340cc310 msgr2=0x7f1f3415c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.531+0000 7f1f3b800640 1 --2- 172.21.15.119:0/3400800721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f340cc310 0x7f1f3415c2e0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f1f3406ba10 tx=0x7f1f2002a270 comp rx=0 tx=0).stop 2024-04-25T10:17:59.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.531+0000 7f1f3b800640 1 -- 172.21.15.119:0/3400800721 shutdown_connections 2024-04-25T10:17:59.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.531+0000 7f1f3b800640 1 --2- 172.21.15.119:0/3400800721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f340cc310 0x7f1f3415c2e0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.531+0000 7f1f3b800640 1 --2- 172.21.15.119:0/3400800721 >> v2:172.21.15.132:3300/0 conn(0x7f1f3415bf00 0x7f1f340cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.531+0000 7f1f3b800640 1 -- 172.21.15.119:0/3400800721 >> 172.21.15.119:0/3400800721 conn(0x7f1f3406aba0 msgr2=0x7f1f3406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:59.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.531+0000 7f1f3b800640 1 -- 172.21.15.119:0/3400800721 shutdown_connections 2024-04-25T10:17:59.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.531+0000 7f1f3b800640 1 -- 172.21.15.119:0/3400800721 wait complete. 2024-04-25T10:17:59.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.533+0000 7f1f3b800640 1 Processor -- start 2024-04-25T10:17:59.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.533+0000 7f1f3b800640 1 -- start start 2024-04-25T10:17:59.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.534+0000 7f1f3b800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f340cc310 0x7f1f3413aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.534+0000 7f1f3b800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f3415bf00 0x7f1f3413af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.534+0000 7f1f3b800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f3415c820 0x7f1f3413b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.534+0000 7f1f3b800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340cd050 con 0x7f1f3415c820 2024-04-25T10:17:59.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.534+0000 7f1f3b800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340ccd70 con 0x7f1f340cc310 2024-04-25T10:17:59.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.534+0000 7f1f3b800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3415ccd0 con 0x7f1f3415bf00 2024-04-25T10:17:59.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.534+0000 7f1f39800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f3415c820 0x7f1f3413b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.534+0000 7f1f39800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f3415c820 0x7f1f3413b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46636/0 (socket says 172.21.15.119:46636) 2024-04-25T10:17:59.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.534+0000 7f1f39800640 1 -- 172.21.15.119:0/549849549 learned_addr learned my addr 172.21.15.119:0/549849549 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:17:59.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.535+0000 7f1f39800640 1 -- 172.21.15.119:0/549849549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f3415bf00 msgr2=0x7f1f3413af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.535+0000 7f1f39800640 1 --2- 172.21.15.119:0/549849549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f3415bf00 0x7f1f3413af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.535+0000 7f1f39800640 1 -- 172.21.15.119:0/549849549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f340cc310 msgr2=0x7f1f3413aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.535+0000 7f1f39800640 1 --2- 172.21.15.119:0/549849549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f340cc310 0x7f1f3413aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.535+0000 7f1f39800640 1 -- 172.21.15.119:0/549849549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3413bea0 con 0x7f1f3415c820 2024-04-25T10:17:59.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.535+0000 7f1f39800640 1 --2- 172.21.15.119:0/549849549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f3415c820 0x7f1f3413b4a0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00c780 tx=0x7f1f2c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:59.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.535+0000 7f1f30c00640 1 -- 172.21.15.119:0/549849549 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c003b70 con 0x7f1f3415c820 2024-04-25T10:17:59.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.535+0000 7f1f30c00640 1 -- 172.21.15.119:0/549849549 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c003d10 con 0x7f1f3415c820 2024-04-25T10:17:59.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.535+0000 7f1f3b800640 1 -- 172.21.15.119:0/549849549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3413bbc0 con 0x7f1f3415c820 2024-04-25T10:17:59.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.535+0000 7f1f30c00640 1 -- 172.21.15.119:0/549849549 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c005270 con 0x7f1f3415c820 2024-04-25T10:17:59.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.535+0000 7f1f3b800640 1 -- 172.21.15.119:0/549849549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3413fd50 con 0x7f1f3415c820 2024-04-25T10:17:59.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.537+0000 7f1f3b800640 1 -- 172.21.15.119:0/549849549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3406bb30 con 0x7f1f3415c820 2024-04-25T10:17:59.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.537+0000 7f1f30c00640 1 -- 172.21.15.119:0/549849549 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f2c005410 con 0x7f1f3415c820 2024-04-25T10:17:59.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.539+0000 7f1f30c00640 1 --2- 172.21.15.119:0/549849549 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f1406b4d0 0x7f1f1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.539+0000 7f1f30c00640 1 -- 172.21.15.119:0/549849549 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1f2c08a1b0 con 0x7f1f3415c820 2024-04-25T10:17:59.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.539+0000 7f1f38e00640 1 --2- 172.21.15.119:0/549849549 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f1406b4d0 0x7f1f1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.540+0000 7f1f38e00640 1 --2- 172.21.15.119:0/549849549 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f1406b4d0 0x7f1f1406d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1f28006fd0 tx=0x7f1f28008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:59.546 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.543+0000 7f1f30c00640 1 -- 172.21.15.119:0/549849549 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f340ce6c0 con 0x7f1f3415c820 2024-04-25T10:17:59.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.775+0000 7f1f3b800640 1 -- 172.21.15.119:0/549849549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f34144be0 con 0x7f1f3415c820 2024-04-25T10:17:59.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.776+0000 7f1f30c00640 1 -- 172.21.15.119:0/549849549 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4957 (secure 0 0 0) 0x7f1f2c058f00 con 0x7f1f3415c820 2024-04-25T10:17:59.779 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:17:59.779 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3394261143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3394261143},{"type":"v1","addr":"172.21.15.132:6834","nonce":3394261143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2502874931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2502874931},{"type":"v1","addr":"172.21.15.119:6835","nonce":2502874931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":8694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2305428003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2305428003},{"type":"v1","addr":"172.21.15.132:6835","nonce":2305428003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:37.617787+0000","modified":"2024-04-25T10:17:58.462065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8943":{"gid":8943,"name":"c","rank":0,"incarnation":102,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.119:6837/2864575636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2864575636},{"type":"v1","addr":"172.21.15.119:6837","nonce":2864575636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:17:59.780 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 104 2024-04-25T10:17:59.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.779+0000 7f1f3b800640 1 -- 172.21.15.119:0/549849549 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f1406b4d0 msgr2=0x7f1f1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.779+0000 7f1f3b800640 1 --2- 172.21.15.119:0/549849549 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f1406b4d0 0x7f1f1406d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1f28006fd0 tx=0x7f1f28008040 comp rx=0 tx=0).stop 2024-04-25T10:17:59.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.779+0000 7f1f3b800640 1 -- 172.21.15.119:0/549849549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f3415c820 msgr2=0x7f1f3413b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.779+0000 7f1f3b800640 1 --2- 172.21.15.119:0/549849549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f3415c820 0x7f1f3413b4a0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00c780 tx=0x7f1f2c00cb80 comp rx=0 tx=0).stop 2024-04-25T10:17:59.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.779+0000 7f1f3b800640 1 -- 172.21.15.119:0/549849549 shutdown_connections 2024-04-25T10:17:59.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.779+0000 7f1f3b800640 1 --2- 172.21.15.119:0/549849549 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1f1406b4d0 0x7f1f1406d990 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.779+0000 7f1f3b800640 1 --2- 172.21.15.119:0/549849549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f3415c820 0x7f1f3413b4a0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.779+0000 7f1f3b800640 1 --2- 172.21.15.119:0/549849549 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f3415bf00 0x7f1f3413af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.779+0000 7f1f3b800640 1 --2- 172.21.15.119:0/549849549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f340cc310 0x7f1f3413aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.779+0000 7f1f3b800640 1 -- 172.21.15.119:0/549849549 >> 172.21.15.119:0/549849549 conn(0x7f1f3406aba0 msgr2=0x7f1f3415d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:59.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.779+0000 7f1f3b800640 1 -- 172.21.15.119:0/549849549 shutdown_connections 2024-04-25T10:17:59.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:17:59.779+0000 7f1f3b800640 1 -- 172.21.15.119:0/549849549 wait complete. 2024-04-25T10:18:00.201 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:17:37.617787+0000', 'modified': '2024-04-25T10:17:58.462065+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 111, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8943}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8943': {'gid': 8943, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.119:6837/2864575636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 2864575636}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2864575636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:18:00.201 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8943:up:rejoin 2024-04-25T10:18:01.202 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:01.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.407+0000 7fdf53400640 1 Processor -- start 2024-04-25T10:18:01.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.408+0000 7fdf53400640 1 -- start start 2024-04-25T10:18:01.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.409+0000 7fdf53400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdf4c15dce0 0x7fdf4c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.409+0000 7fdf53400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf4c0b5b60 0x7fdf4c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.409+0000 7fdf53400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4c15e0c0 con 0x7fdf4c0d35b0 2024-04-25T10:18:01.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.409+0000 7fdf53400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4c0d2510 con 0x7fdf4c0b5b60 2024-04-25T10:18:01.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.409+0000 7fdf53400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4c0d27f0 con 0x7fdf4c15dce0 2024-04-25T10:18:01.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.409+0000 7fdf51400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf4c0b5b60 0x7fdf4c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.410+0000 7fdf51400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf4c0b5b60 0x7fdf4c0b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43806/0 (socket says 172.21.15.119:43806) 2024-04-25T10:18:01.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.410+0000 7fdf51400640 1 -- 172.21.15.119:0/3999692462 learned_addr learned my addr 172.21.15.119:0/3999692462 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:01.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.410+0000 7fdf50a00640 1 --2- 172.21.15.119:0/3999692462 >> v2:172.21.15.132:3300/0 conn(0x7fdf4c15dce0 0x7fdf4c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.410+0000 7fdf51400640 1 -- 172.21.15.119:0/3999692462 >> v2:172.21.15.132:3300/0 conn(0x7fdf4c15dce0 msgr2=0x7fdf4c0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.410+0000 7fdf51400640 1 --2- 172.21.15.119:0/3999692462 >> v2:172.21.15.132:3300/0 conn(0x7fdf4c15dce0 0x7fdf4c0d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.410+0000 7fdf51400640 1 -- 172.21.15.119:0/3999692462 >> v1:172.21.15.132:6789/0 conn(0x7fdf4c0d35b0 legacy=0x7fdf4c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.414 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.410+0000 7fdf51400640 1 -- 172.21.15.119:0/3999692462 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c0d2ad0 con 0x7fdf4c0b5b60 2024-04-25T10:18:01.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.411+0000 7fdf51400640 1 --2- 172.21.15.119:0/3999692462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf4c0b5b60 0x7fdf4c0b6b00 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0ce220 tx=0x7fdf4402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a9effd4cee71b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.411+0000 7fdf4b400640 1 -- 172.21.15.119:0/3999692462 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44036070 con 0x7fdf4c0b5b60 2024-04-25T10:18:01.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.411+0000 7fdf4b400640 1 -- 172.21.15.119:0/3999692462 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf44031600 con 0x7fdf4c0b5b60 2024-04-25T10:18:01.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.411+0000 7fdf4b400640 1 -- 172.21.15.119:0/3999692462 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf440319d0 con 0x7fdf4c0b5b60 2024-04-25T10:18:01.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.412+0000 7fdf53400640 1 -- 172.21.15.119:0/3999692462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf4c0b5b60 msgr2=0x7fdf4c0b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.412+0000 7fdf53400640 1 --2- 172.21.15.119:0/3999692462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf4c0b5b60 0x7fdf4c0b6b00 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0ce220 tx=0x7fdf4402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.412+0000 7fdf53400640 1 -- 172.21.15.119:0/3999692462 shutdown_connections 2024-04-25T10:18:01.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.412+0000 7fdf53400640 1 --2- 172.21.15.119:0/3999692462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf4c0b5b60 0x7fdf4c0b6b00 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.412+0000 7fdf53400640 1 --2- 172.21.15.119:0/3999692462 >> v2:172.21.15.132:3300/0 conn(0x7fdf4c15dce0 0x7fdf4c0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.412+0000 7fdf53400640 1 -- 172.21.15.119:0/3999692462 >> 172.21.15.119:0/3999692462 conn(0x7fdf4c072900 msgr2=0x7fdf4c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.412+0000 7fdf53400640 1 -- 172.21.15.119:0/3999692462 shutdown_connections 2024-04-25T10:18:01.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.412+0000 7fdf53400640 1 -- 172.21.15.119:0/3999692462 wait complete. 2024-04-25T10:18:01.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.414+0000 7fdf53400640 1 Processor -- start 2024-04-25T10:18:01.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.414+0000 7fdf53400640 1 -- start start 2024-04-25T10:18:01.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.415+0000 7fdf53400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf4c0b5b60 0x7fdf4c149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.415+0000 7fdf53400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf4c0d35b0 0x7fdf4c0c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.415+0000 7fdf53400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf4c15dce0 0x7fdf4c14c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.415+0000 7fdf53400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c0b9820 con 0x7fdf4c15dce0 2024-04-25T10:18:01.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.415+0000 7fdf53400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c0d3090 con 0x7fdf4c0d35b0 2024-04-25T10:18:01.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.415+0000 7fdf53400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c0b99a0 con 0x7fdf4c0b5b60 2024-04-25T10:18:01.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.415+0000 7fdf51400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf4c15dce0 0x7fdf4c14c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.415+0000 7fdf51400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf4c15dce0 0x7fdf4c14c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43814/0 (socket says 172.21.15.119:43814) 2024-04-25T10:18:01.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.415+0000 7fdf51400640 1 -- 172.21.15.119:0/2354663982 learned_addr learned my addr 172.21.15.119:0/2354663982 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:01.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.416+0000 7fdf4be00640 1 --2- 172.21.15.119:0/2354663982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf4c0d35b0 0x7fdf4c0c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.416+0000 7fdf51400640 1 -- 172.21.15.119:0/2354663982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf4c0b5b60 msgr2=0x7fdf4c149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.416+0000 7fdf51400640 1 --2- 172.21.15.119:0/2354663982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf4c0b5b60 0x7fdf4c149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.416+0000 7fdf51400640 1 -- 172.21.15.119:0/2354663982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf4c0d35b0 msgr2=0x7fdf4c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.416+0000 7fdf51400640 1 --2- 172.21.15.119:0/2354663982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf4c0d35b0 0x7fdf4c0c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.416+0000 7fdf51400640 1 -- 172.21.15.119:0/2354663982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c0c9b10 con 0x7fdf4c15dce0 2024-04-25T10:18:01.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.416+0000 7fdf51400640 1 --2- 172.21.15.119:0/2354663982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf4c15dce0 0x7fdf4c14c210 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fdf4402ace0 tx=0x7fdf44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.417+0000 7fdf48c00640 1 -- 172.21.15.119:0/2354663982 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44036050 con 0x7fdf4c15dce0 2024-04-25T10:18:01.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.417+0000 7fdf48c00640 1 -- 172.21.15.119:0/2354663982 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf44042070 con 0x7fdf4c15dce0 2024-04-25T10:18:01.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.417+0000 7fdf48c00640 1 -- 172.21.15.119:0/2354663982 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44009510 con 0x7fdf4c15dce0 2024-04-25T10:18:01.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.417+0000 7fdf53400640 1 -- 172.21.15.119:0/2354663982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4c0c9df0 con 0x7fdf4c15dce0 2024-04-25T10:18:01.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.417+0000 7fdf53400640 1 -- 172.21.15.119:0/2354663982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4c14cf10 con 0x7fdf4c15dce0 2024-04-25T10:18:01.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.418+0000 7fdf53400640 1 -- 172.21.15.119:0/2354663982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4c0749b0 con 0x7fdf4c15dce0 2024-04-25T10:18:01.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.419+0000 7fdf48c00640 1 -- 172.21.15.119:0/2354663982 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf44007bd0 con 0x7fdf4c15dce0 2024-04-25T10:18:01.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.420+0000 7fdf48c00640 1 --2- 172.21.15.119:0/2354663982 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdf2806b4d0 0x7fdf2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.420+0000 7fdf48c00640 1 -- 172.21.15.119:0/2354663982 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fdf440ab200 con 0x7fdf4c15dce0 2024-04-25T10:18:01.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.420+0000 7fdf50a00640 1 --2- 172.21.15.119:0/2354663982 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdf2806b4d0 0x7fdf2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.421+0000 7fdf50a00640 1 --2- 172.21.15.119:0/2354663982 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdf2806b4d0 0x7fdf2806d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c006fd0 tx=0x7fdf3c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.424+0000 7fdf48c00640 1 -- 172.21.15.119:0/2354663982 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf44045360 con 0x7fdf4c15dce0 2024-04-25T10:18:01.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.663+0000 7fdf53400640 1 -- 172.21.15.119:0/2354663982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf4c0b7360 con 0x7fdf4c15dce0 2024-04-25T10:18:01.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.664+0000 7fdf48c00640 1 -- 172.21.15.119:0/2354663982 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4957 (secure 0 0 0) 0x7fdf44079f50 con 0x7fdf4c15dce0 2024-04-25T10:18:01.668 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:01.668 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3394261143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3394261143},{"type":"v1","addr":"172.21.15.132:6834","nonce":3394261143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2502874931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2502874931},{"type":"v1","addr":"172.21.15.119:6835","nonce":2502874931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":8694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2305428003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2305428003},{"type":"v1","addr":"172.21.15.132:6835","nonce":2305428003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:37.617787+0000","modified":"2024-04-25T10:17:59.464926+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8943":{"gid":8943,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/2864575636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2864575636},{"type":"v1","addr":"172.21.15.119:6837","nonce":2864575636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:18:01.668 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 105 2024-04-25T10:18:01.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.667+0000 7fdf53400640 1 -- 172.21.15.119:0/2354663982 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdf2806b4d0 msgr2=0x7fdf2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.667+0000 7fdf53400640 1 --2- 172.21.15.119:0/2354663982 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdf2806b4d0 0x7fdf2806d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c006fd0 tx=0x7fdf3c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:01.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.667+0000 7fdf53400640 1 -- 172.21.15.119:0/2354663982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf4c15dce0 msgr2=0x7fdf4c14c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.667+0000 7fdf53400640 1 --2- 172.21.15.119:0/2354663982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf4c15dce0 0x7fdf4c14c210 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fdf4402ace0 tx=0x7fdf44004a40 comp rx=0 tx=0).stop 2024-04-25T10:18:01.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.667+0000 7fdf53400640 1 -- 172.21.15.119:0/2354663982 shutdown_connections 2024-04-25T10:18:01.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.667+0000 7fdf53400640 1 --2- 172.21.15.119:0/2354663982 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fdf2806b4d0 0x7fdf2806d990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.667+0000 7fdf53400640 1 --2- 172.21.15.119:0/2354663982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf4c15dce0 0x7fdf4c14c210 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.667+0000 7fdf53400640 1 --2- 172.21.15.119:0/2354663982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf4c0d35b0 0x7fdf4c0c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.667+0000 7fdf53400640 1 --2- 172.21.15.119:0/2354663982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf4c0b5b60 0x7fdf4c149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.667+0000 7fdf53400640 1 -- 172.21.15.119:0/2354663982 >> 172.21.15.119:0/2354663982 conn(0x7fdf4c072900 msgr2=0x7fdf4c0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.667+0000 7fdf53400640 1 -- 172.21.15.119:0/2354663982 shutdown_connections 2024-04-25T10:18:01.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:01.667+0000 7fdf53400640 1 -- 172.21.15.119:0/2354663982 wait complete. 2024-04-25T10:18:02.093 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:17:37.617787+0000', 'modified': '2024-04-25T10:17:59.464926+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 111, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8943}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8943': {'gid': 8943, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.119:6837/2864575636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 2864575636}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2864575636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:18:02.093 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:18:02.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:18:02.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.303+0000 7f2ae1a00640 1 Processor -- start 2024-04-25T10:18:02.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.303+0000 7f2ae1a00640 1 -- start start 2024-04-25T10:18:02.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.304+0000 7f2ae1a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2adc145260 0x7f2adc0cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.304+0000 7f2ae1a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2adc0cc490 0x7f2adc071200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.304+0000 7f2ae1a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2adc0cc9a0 con 0x7f2adc071740 2024-04-25T10:18:02.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.304+0000 7f2ae1a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2adc0cea00 con 0x7f2adc145260 2024-04-25T10:18:02.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.304+0000 7f2ae1a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2adc071b70 con 0x7f2adc0cc490 2024-04-25T10:18:02.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.305+0000 7f2adaa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2adc145260 0x7f2adc0cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.305+0000 7f2adaa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2adc145260 0x7f2adc0cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43824/0 (socket says 172.21.15.119:43824) 2024-04-25T10:18:02.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.305+0000 7f2adaa00640 1 -- 172.21.15.119:0/3161908930 learned_addr learned my addr 172.21.15.119:0/3161908930 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:02.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.305+0000 7f2ad2000640 1 --2- 172.21.15.119:0/3161908930 >> v2:172.21.15.132:3300/0 conn(0x7f2adc0cc490 0x7f2adc071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.305+0000 7f2adaa00640 1 -- 172.21.15.119:0/3161908930 >> v2:172.21.15.132:3300/0 conn(0x7f2adc0cc490 msgr2=0x7f2adc071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.305+0000 7f2adaa00640 1 --2- 172.21.15.119:0/3161908930 >> v2:172.21.15.132:3300/0 conn(0x7f2adc0cc490 0x7f2adc071200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.305+0000 7f2adaa00640 1 -- 172.21.15.119:0/3161908930 >> v1:172.21.15.132:6789/0 conn(0x7f2adc071740 legacy=0x7f2adc15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.305+0000 7f2adaa00640 1 -- 172.21.15.119:0/3161908930 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc0ccc80 con 0x7f2adc145260 2024-04-25T10:18:02.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.306+0000 7f2adaa00640 1 --2- 172.21.15.119:0/3161908930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc145260 0x7f2adc0cbf50 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0cb960 tx=0x7f2acc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8013c5e3a3fced2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.306+0000 7f2ada000640 1 -- 172.21.15.119:0/3161908930 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc037070 con 0x7f2adc145260 2024-04-25T10:18:02.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.307+0000 7f2ada000640 1 -- 172.21.15.119:0/3161908930 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2acc032440 con 0x7f2adc145260 2024-04-25T10:18:02.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.307+0000 7f2ada000640 1 -- 172.21.15.119:0/3161908930 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc032810 con 0x7f2adc145260 2024-04-25T10:18:02.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.307+0000 7f2ae1a00640 1 -- 172.21.15.119:0/3161908930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc145260 msgr2=0x7f2adc0cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.307+0000 7f2ae1a00640 1 --2- 172.21.15.119:0/3161908930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc145260 0x7f2adc0cbf50 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0cb960 tx=0x7f2acc02a230 comp rx=0 tx=0).stop 2024-04-25T10:18:02.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.307+0000 7f2ae1a00640 1 -- 172.21.15.119:0/3161908930 shutdown_connections 2024-04-25T10:18:02.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.307+0000 7f2ae1a00640 1 --2- 172.21.15.119:0/3161908930 >> v2:172.21.15.132:3300/0 conn(0x7f2adc0cc490 0x7f2adc071200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.307+0000 7f2ae1a00640 1 --2- 172.21.15.119:0/3161908930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc145260 0x7f2adc0cbf50 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.307+0000 7f2ae1a00640 1 -- 172.21.15.119:0/3161908930 >> 172.21.15.119:0/3161908930 conn(0x7f2adc06aba0 msgr2=0x7f2adc06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:02.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.307+0000 7f2ae1a00640 1 -- 172.21.15.119:0/3161908930 shutdown_connections 2024-04-25T10:18:02.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.307+0000 7f2ae1a00640 1 -- 172.21.15.119:0/3161908930 wait complete. 2024-04-25T10:18:02.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.309+0000 7f2ae1a00640 1 Processor -- start 2024-04-25T10:18:02.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.309+0000 7f2ae1a00640 1 -- start start 2024-04-25T10:18:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.310+0000 7f2ae1a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2adc071740 0x7f2adc15b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.310+0000 7f2ae1a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2adc0cc490 0x7f2adc131600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.310+0000 7f2ae1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc145260 0x7f2adc0acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.310+0000 7f2ae1a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc145640 con 0x7f2adc145260 2024-04-25T10:18:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.310+0000 7f2ae1a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc0cd240 con 0x7f2adc0cc490 2024-04-25T10:18:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.310+0000 7f2ae1a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc15c270 con 0x7f2adc071740 2024-04-25T10:18:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.310+0000 7f2adb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc145260 0x7f2adc0acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.310+0000 7f2adb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc145260 0x7f2adc0acef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43834/0 (socket says 172.21.15.119:43834) 2024-04-25T10:18:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.310+0000 7f2adb400640 1 -- 172.21.15.119:0/2776249947 learned_addr learned my addr 172.21.15.119:0/2776249947 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:02.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.311+0000 7f2adb400640 1 -- 172.21.15.119:0/2776249947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2adc071740 msgr2=0x7f2adc15b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.311+0000 7f2adb400640 1 --2- 172.21.15.119:0/2776249947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2adc071740 0x7f2adc15b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.311+0000 7f2adb400640 1 -- 172.21.15.119:0/2776249947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2adc0cc490 msgr2=0x7f2adc131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.311+0000 7f2adb400640 1 --2- 172.21.15.119:0/2776249947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2adc0cc490 0x7f2adc131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.311+0000 7f2adb400640 1 -- 172.21.15.119:0/2776249947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc15b720 con 0x7f2adc145260 2024-04-25T10:18:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.311+0000 7f2adb400640 1 --2- 172.21.15.119:0/2776249947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc145260 0x7f2adc0acef0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400c780 tx=0x7f2ad400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.311+0000 7f2ad3400640 1 -- 172.21.15.119:0/2776249947 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4017070 con 0x7f2adc145260 2024-04-25T10:18:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.311+0000 7f2ad3400640 1 -- 172.21.15.119:0/2776249947 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad4003da0 con 0x7f2adc145260 2024-04-25T10:18:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.311+0000 7f2ad3400640 1 -- 172.21.15.119:0/2776249947 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40052b0 con 0x7f2adc145260 2024-04-25T10:18:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.311+0000 7f2ae1a00640 1 -- 172.21.15.119:0/2776249947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2adc13fc00 con 0x7f2adc145260 2024-04-25T10:18:02.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.312+0000 7f2ae1a00640 1 -- 172.21.15.119:0/2776249947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2adc1401c0 con 0x7f2adc145260 2024-04-25T10:18:02.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.314+0000 7f2ad3400640 1 -- 172.21.15.119:0/2776249947 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ad4005450 con 0x7f2adc145260 2024-04-25T10:18:02.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.315+0000 7f2ad3400640 1 --2- 172.21.15.119:0/2776249947 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ab806b4d0 0x7f2ab806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.315+0000 7f2ad3400640 1 -- 172.21.15.119:0/2776249947 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2ad4089880 con 0x7f2adc145260 2024-04-25T10:18:02.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.315+0000 7f2adaa00640 1 --2- 172.21.15.119:0/2776249947 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ab806b4d0 0x7f2ab806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.315+0000 7f2ae49c0740 1 Processor -- start 2024-04-25T10:18:02.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.315+0000 7f2ae49c0740 1 -- start start 2024-04-25T10:18:02.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.316+0000 7f2adaa00640 1 --2- 172.21.15.119:0/2776249947 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ab806b4d0 0x7f2ab806d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2acc034000 tx=0x7f2acc02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.316+0000 7f2ae49c0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d0173d950 0x562d01765790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.316+0000 7f2ae49c0740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d01765cd0 0x562d0173dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.316+0000 7f2ae49c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017865c0 0x562d0173c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.316+0000 7f2ae49c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d016a1620 con 0x562d017865c0 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.316+0000 7f2ae49c0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0164aca0 con 0x562d0173d950 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.316+0000 7f2ae49c0740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d0169d6c0 con 0x562d01765cd0 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.317+0000 7f2adb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017865c0 0x562d0173c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.317+0000 7f2adb400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017865c0 0x562d0173c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43842/0 (socket says 172.21.15.119:43842) 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.317+0000 7f2adb400640 1 -- 172.21.15.119:0/3566941115 learned_addr learned my addr 172.21.15.119:0/3566941115 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.317+0000 7f2ad2000640 1 --2- 172.21.15.119:0/3566941115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d01765cd0 0x562d0173dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.317+0000 7f2adaa00640 1 --2- 172.21.15.119:0/3566941115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d0173d950 0x562d01765790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.317+0000 7f2adb400640 1 -- 172.21.15.119:0/3566941115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d01765cd0 msgr2=0x562d0173dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.317+0000 7f2adb400640 1 --2- 172.21.15.119:0/3566941115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d01765cd0 0x562d0173dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.317+0000 7f2adb400640 1 -- 172.21.15.119:0/3566941115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d0173d950 msgr2=0x562d01765790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.317+0000 7f2adb400640 1 --2- 172.21.15.119:0/3566941115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d0173d950 0x562d01765790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.317+0000 7f2adb400640 1 -- 172.21.15.119:0/3566941115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d01651f40 con 0x562d017865c0 2024-04-25T10:18:02.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.317+0000 7f2adb400640 1 --2- 172.21.15.119:0/3566941115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017865c0 0x562d0173c110 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40600b0 tx=0x7f2ad4058650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d123d5ebcda172 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.318+0000 7f2acb400640 1 -- 172.21.15.119:0/3566941115 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40605e0 con 0x562d017865c0 2024-04-25T10:18:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.318+0000 7f2acb400640 1 -- 172.21.15.119:0/3566941115 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad4005740 con 0x562d017865c0 2024-04-25T10:18:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.318+0000 7f2acb400640 1 -- 172.21.15.119:0/3566941115 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4054540 con 0x562d017865c0 2024-04-25T10:18:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.318+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3566941115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017865c0 msgr2=0x562d0173c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.318+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/3566941115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017865c0 0x562d0173c110 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40600b0 tx=0x7f2ad4058650 comp rx=0 tx=0).stop 2024-04-25T10:18:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.318+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3566941115 shutdown_connections 2024-04-25T10:18:02.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.318+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/3566941115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017865c0 0x562d0173c110 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.318+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/3566941115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d01765cd0 0x562d0173dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.318+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/3566941115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d0173d950 0x562d01765790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.318+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3566941115 >> 172.21.15.119:0/3566941115 conn(0x562d0173ab70 msgr2=0x562d01783550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:02.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.319+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3566941115 shutdown_connections 2024-04-25T10:18:02.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.319+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3566941115 wait complete. 2024-04-25T10:18:02.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.319+0000 7f2ae49c0740 1 -- start start 2024-04-25T10:18:02.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.319+0000 7f2ae49c0740 1 Processor -- start 2024-04-25T10:18:02.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.320+0000 7f2ae49c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d01765cd0 0x562d01767fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.320+0000 7f2ae49c0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d017865c0 0x562d01760de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.320+0000 7f2ae49c0740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d017664f0 0x562d0176a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.320+0000 7f2ae49c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d01765300 con 0x562d01765cd0 2024-04-25T10:18:02.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.320+0000 7f2ae49c0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d016401f0 con 0x562d017865c0 2024-04-25T10:18:02.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.321+0000 7f2ae49c0740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d01764cf0 con 0x562d017664f0 2024-04-25T10:18:02.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.321+0000 7f2ae49c0740 10 client.0 _ll_register_callbacks cb 0x562d0165dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:02.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.321+0000 7f2adaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d01765cd0 0x562d01767fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.321+0000 7f2adaa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d01765cd0 0x562d01767fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43858/0 (socket says 172.21.15.119:43858) 2024-04-25T10:18:02.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.321+0000 7f2adaa00640 1 -- 172.21.15.119:0/2445183991 learned_addr learned my addr 172.21.15.119:0/2445183991 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:02.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.321+0000 7f2ad2000640 1 --2- 172.21.15.119:0/2445183991 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d017865c0 0x562d01760de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.321+0000 7f2adaa00640 1 -- 172.21.15.119:0/2445183991 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d017664f0 msgr2=0x562d0176a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.321+0000 7f2adaa00640 1 --2- 172.21.15.119:0/2445183991 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d017664f0 0x562d0176a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.321+0000 7f2adaa00640 1 -- 172.21.15.119:0/2445183991 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d017865c0 msgr2=0x562d01760de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.321+0000 7f2adaa00640 1 --2- 172.21.15.119:0/2445183991 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d017865c0 0x562d01760de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.321+0000 7f2adaa00640 1 -- 172.21.15.119:0/2445183991 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d0165fc50 con 0x562d01765cd0 2024-04-25T10:18:02.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.322+0000 7f2adaa00640 1 --2- 172.21.15.119:0/2445183991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d01765cd0 0x562d01767fb0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f2acc02a270 tx=0x7f2acc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e28f085eaebe263 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.322+0000 7f2abfe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:18:02.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.322+0000 7f2abfe00640 1 -- 172.21.15.119:0/2445183991 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc04c070 con 0x562d01765cd0 2024-04-25T10:18:02.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.322+0000 7f2abfe00640 1 -- 172.21.15.119:0/2445183991 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2acc03ca30 con 0x562d01765cd0 2024-04-25T10:18:02.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.322+0000 7f2ae49c0740 1 -- 172.21.15.119:0/2445183991 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ad4005740 con 0x562d01765cd0 2024-04-25T10:18:02.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.322+0000 7f2abfe00640 1 -- 172.21.15.119:0/2445183991 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2acc057050 con 0x562d01765cd0 2024-04-25T10:18:02.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.322+0000 7f2ae49c0740 10 client.8994 fetch_fsmap learned FSMap version 105 2024-04-25T10:18:02.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.322+0000 7f2ae49c0740 1 -- 172.21.15.119:0/2445183991 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x562d013b62d0 con 0x562d01765cd0 2024-04-25T10:18:02.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.322+0000 7f2abfe00640 1 -- 172.21.15.119:0/2445183991 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad4005740 con 0x562d01765cd0 2024-04-25T10:18:02.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.322+0000 7f2abfe00640 1 -- 172.21.15.119:0/2445183991 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2acc05a020 con 0x562d01765cd0 2024-04-25T10:18:02.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.323+0000 7f2ae49c0740 10 client.8994 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:18:02.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.323+0000 7f2ae49c0740 10 client.8994 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:18:02.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.323+0000 7f2ae49c0740 20 client.8994 populate_metadata read hostname 'smithi119' 2024-04-25T10:18:02.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.323+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/2445183991 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x562d01797070 0x562d01797450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.323+0000 7f2ae49c0740 4 client.8994 mds_command: new command op to 8943 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:02.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.323+0000 7f2ae49c0740 1 -- 172.21.15.119:0/2445183991 --> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d016aeb60 con 0x562d01797070 2024-04-25T10:18:02.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.323+0000 7f2ad2000640 1 --2- 172.21.15.119:0/2445183991 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x562d01797070 0x562d01797450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.323+0000 7f2ad2000640 1 --2- 172.21.15.119:0/2445183991 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x562d01797070 0x562d01797450 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=238cceccd4871460 server_cookie=f6e2fae78404cccc in_seq=0 out_seq=0 2024-04-25T10:18:02.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.323+0000 7f2abfe00640 10 client.8994 ms_handle_connect on v2:172.21.15.119:6836/2864575636 2024-04-25T10:18:02.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.324+0000 7f2abfe00640 1 -- 172.21.15.119:0/2445183991 <== mds.0 v2:172.21.15.119:6836/2864575636 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d016aeb60 con 0x562d01797070 2024-04-25T10:18:02.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.324+0000 7f2abfe00640 10 client.8994 handle_command_reply: tid=0 2024-04-25T10:18:02.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.324+0000 7f2ae49c0740 1 client.8994 shutdown 2024-04-25T10:18:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.324+0000 7f2ae49c0740 1 -- 172.21.15.119:0/2445183991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d01765cd0 msgr2=0x562d01767fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.324+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/2445183991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d01765cd0 0x562d01767fb0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f2acc02a270 tx=0x7f2acc004730 comp rx=0 tx=0).stop 2024-04-25T10:18:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.325+0000 7f2ae49c0740 1 -- 172.21.15.119:0/2445183991 shutdown_connections 2024-04-25T10:18:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.325+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/2445183991 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x562d01797070 0x562d01797450 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.325+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/2445183991 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d017664f0 0x562d0176a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.325+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/2445183991 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d017865c0 0x562d01760de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.325+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/2445183991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d01765cd0 0x562d01767fb0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.325+0000 7f2abfe00640 0 client.8994 ms_handle_reset on v2:172.21.15.119:6836/2864575636 2024-04-25T10:18:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.325+0000 7f2ae49c0740 1 -- 172.21.15.119:0/2445183991 >> 172.21.15.119:0/2445183991 conn(0x562d0173d950 msgr2=0x562d0173c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.325+0000 7f2ae49c0740 1 -- 172.21.15.119:0/2445183991 shutdown_connections 2024-04-25T10:18:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.325+0000 7f2ae49c0740 1 -- 172.21.15.119:0/2445183991 wait complete. 2024-04-25T10:18:02.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.325+0000 7f2ae49c0740 20 client.8994 trim_cache size 0 max 16384 2024-04-25T10:18:02.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.342+0000 7f2ae49c0740 1 Processor -- start 2024-04-25T10:18:02.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.342+0000 7f2ae49c0740 1 -- start start 2024-04-25T10:18:02.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.343+0000 7f2ae49c0740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d01765cd0 0x562d0179d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.343+0000 7f2ae49c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017664f0 0x562d01760bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.343+0000 7f2ae49c0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d017865c0 0x562d01761130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.343+0000 7f2ae49c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc1fd0c0 con 0x562d017664f0 2024-04-25T10:18:02.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.343+0000 7f2ae49c0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc1416f0 con 0x562d017865c0 2024-04-25T10:18:02.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.343+0000 7f2ae49c0740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc1f21e0 con 0x562d01765cd0 2024-04-25T10:18:02.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.343+0000 7f2ad2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017664f0 0x562d01760bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.343+0000 7f2ad2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017664f0 0x562d01760bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43868/0 (socket says 172.21.15.119:43868) 2024-04-25T10:18:02.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.343+0000 7f2ad2000640 1 -- 172.21.15.119:0/3891369856 learned_addr learned my addr 172.21.15.119:0/3891369856 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:02.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.343+0000 7f2ad2000640 1 -- 172.21.15.119:0/3891369856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d01765cd0 msgr2=0x562d0179d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.343+0000 7f2ad2000640 1 --2- 172.21.15.119:0/3891369856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d01765cd0 0x562d0179d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.344+0000 7f2ad2000640 1 -- 172.21.15.119:0/3891369856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d017865c0 msgr2=0x562d01761130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.344+0000 7f2ad2000640 1 --2- 172.21.15.119:0/3891369856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d017865c0 0x562d01761130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.344+0000 7f2ad2000640 1 -- 172.21.15.119:0/3891369856 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d014806b0 con 0x562d017664f0 2024-04-25T10:18:02.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.344+0000 7f2ad2000640 1 --2- 172.21.15.119:0/3891369856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017664f0 0x562d01760bf0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400a9b0 tx=0x7f2ac400a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f5dfab1e8878d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.344+0000 7f2acaa00640 1 -- 172.21.15.119:0/3891369856 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4018e60 con 0x562d017664f0 2024-04-25T10:18:02.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.344+0000 7f2acaa00640 1 -- 172.21.15.119:0/3891369856 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac401c070 con 0x562d017664f0 2024-04-25T10:18:02.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.344+0000 7f2acaa00640 1 -- 172.21.15.119:0/3891369856 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4018490 con 0x562d017664f0 2024-04-25T10:18:02.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.345+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3891369856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017664f0 msgr2=0x562d01760bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.345+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/3891369856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017664f0 0x562d01760bf0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400a9b0 tx=0x7f2ac400a820 comp rx=0 tx=0).stop 2024-04-25T10:18:02.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.345+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3891369856 shutdown_connections 2024-04-25T10:18:02.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.345+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/3891369856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d017865c0 0x562d01761130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.345+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/3891369856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017664f0 0x562d01760bf0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.345+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/3891369856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d01765cd0 0x562d0179d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.345+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3891369856 >> 172.21.15.119:0/3891369856 conn(0x562d0173d950 msgr2=0x562d01764f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:02.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.345+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3891369856 shutdown_connections 2024-04-25T10:18:02.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.345+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3891369856 wait complete. 2024-04-25T10:18:02.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.346+0000 7f2ae49c0740 1 -- start start 2024-04-25T10:18:02.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.346+0000 7f2ae49c0740 1 Processor -- start 2024-04-25T10:18:02.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ae49c0740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d01765cd0 0x562d017ee300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ae49c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017664f0 0x562d01795d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ae49c0740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d017865c0 0x562d01796260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ae49c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d01766970 con 0x562d017664f0 2024-04-25T10:18:02.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ae49c0740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0176a0f0 con 0x562d01765cd0 2024-04-25T10:18:02.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ae49c0740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d01763bb0 con 0x562d017865c0 2024-04-25T10:18:02.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ae49c0740 10 client.0 _ll_register_callbacks cb 0x562d01765c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:02.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ad2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017664f0 0x562d01795d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ad2000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017664f0 0x562d01795d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43872/0 (socket says 172.21.15.119:43872) 2024-04-25T10:18:02.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ad2000640 1 -- 172.21.15.119:0/3818392306 learned_addr learned my addr 172.21.15.119:0/3818392306 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:02.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2adaa00640 1 --2- 172.21.15.119:0/3818392306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d01765cd0 0x562d017ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ad2000640 1 -- 172.21.15.119:0/3818392306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d017865c0 msgr2=0x562d01796260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ad2000640 1 --2- 172.21.15.119:0/3818392306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d017865c0 0x562d01796260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ad2000640 1 -- 172.21.15.119:0/3818392306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d01765cd0 msgr2=0x562d017ee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ad2000640 1 --2- 172.21.15.119:0/3818392306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d01765cd0 0x562d017ee300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.347+0000 7f2ad2000640 1 -- 172.21.15.119:0/3818392306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d0179d5a0 con 0x562d017664f0 2024-04-25T10:18:02.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.348+0000 7f2ad2000640 1 --2- 172.21.15.119:0/3818392306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017664f0 0x562d01795d20 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40183f0 tx=0x7f2ac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6bf86d75568c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.348+0000 7f2abfe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:18:02.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.348+0000 7f2abfe00640 1 -- 172.21.15.119:0/3818392306 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4018e60 con 0x562d017664f0 2024-04-25T10:18:02.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.348+0000 7f2abfe00640 1 -- 172.21.15.119:0/3818392306 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2ac4016590 con 0x562d017664f0 2024-04-25T10:18:02.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.348+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3818392306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ac401c070 con 0x562d017664f0 2024-04-25T10:18:02.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.348+0000 7f2abfe00640 1 -- 172.21.15.119:0/3818392306 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ac4030050 con 0x562d017664f0 2024-04-25T10:18:02.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.348+0000 7f2ae49c0740 10 client.9000 fetch_fsmap learned FSMap version 105 2024-04-25T10:18:02.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.348+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3818392306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x562d017ed700 con 0x562d017664f0 2024-04-25T10:18:02.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.348+0000 7f2abfe00640 1 -- 172.21.15.119:0/3818392306 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac401c070 con 0x562d017664f0 2024-04-25T10:18:02.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.349+0000 7f2abfe00640 1 -- 172.21.15.119:0/3818392306 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2ac4033020 con 0x562d017664f0 2024-04-25T10:18:02.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.349+0000 7f2ae49c0740 10 client.9000 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:18:02.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.349+0000 7f2ae49c0740 10 client.9000 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:18:02.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.349+0000 7f2ae49c0740 20 client.9000 populate_metadata read hostname 'smithi119' 2024-04-25T10:18:02.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.349+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/3818392306 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x562d01782430 0x562d01768bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.349+0000 7f2ae49c0740 4 client.9000 mds_command: new command op to 8943 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:18:02.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.349+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3818392306 --> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562d016a8eb0 con 0x562d01782430 2024-04-25T10:18:02.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.349+0000 7f2adaa00640 1 --2- 172.21.15.119:0/3818392306 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x562d01782430 0x562d01768bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.349+0000 7f2adaa00640 1 --2- 172.21.15.119:0/3818392306 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x562d01782430 0x562d01768bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6412f921a2f9d714 server_cookie=f570fe5fc4e60e in_seq=0 out_seq=0 2024-04-25T10:18:02.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.349+0000 7f2abfe00640 10 client.9000 ms_handle_connect on v2:172.21.15.119:6836/2864575636 2024-04-25T10:18:02.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2abfe00640 1 -- 172.21.15.119:0/3818392306 <== mds.0 v2:172.21.15.119:6836/2864575636 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x562d016a8eb0 con 0x562d01782430 2024-04-25T10:18:02.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2abfe00640 10 client.9000 handle_command_reply: tid=0 2024-04-25T10:18:02.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2ae49c0740 1 client.9000 shutdown 2024-04-25T10:18:02.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3818392306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017664f0 msgr2=0x562d01795d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/3818392306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017664f0 0x562d01795d20 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40183f0 tx=0x7f2ac40046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3818392306 shutdown_connections 2024-04-25T10:18:02.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/3818392306 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x562d01782430 0x562d01768bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/3818392306 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x562d017865c0 0x562d01796260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/3818392306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562d017664f0 0x562d01795d20 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2ae49c0740 1 --2- 172.21.15.119:0/3818392306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562d01765cd0 0x562d017ee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3818392306 >> 172.21.15.119:0/3818392306 conn(0x562d0173d950 msgr2=0x562d01792e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:02.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2abfe00640 0 client.9000 ms_handle_reset on v2:172.21.15.119:6836/2864575636 2024-04-25T10:18:02.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3818392306 shutdown_connections 2024-04-25T10:18:02.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2ae49c0740 1 -- 172.21.15.119:0/3818392306 wait complete. 2024-04-25T10:18:02.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.350+0000 7f2ae49c0740 20 client.9000 trim_cache size 0 max 16384 2024-04-25T10:18:02.358 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:18:02.358 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "e0b0f428-b441-4237-a4d1-ed53321f87a5", 2024-04-25T10:18:02.358 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-04-25T10:18:02.358 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 8943, 2024-04-25T10:18:02.358 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-04-25T10:18:02.358 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-04-25T10:18:02.358 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-04-25T10:18:02.358 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 4.8921266279999998, 2024-04-25T10:18:02.359 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 105, 2024-04-25T10:18:02.359 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 111, 2024-04-25T10:18:02.359 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 111, 2024-04-25T10:18:02.359 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 5.8081338310000001 2024-04-25T10:18:02.359 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:18:02.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.351+0000 7f2ae1a00640 1 -- 172.21.15.119:0/2776249947 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ab806b4d0 msgr2=0x7f2ab806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.351+0000 7f2ae1a00640 1 --2- 172.21.15.119:0/2776249947 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ab806b4d0 0x7f2ab806d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2acc034000 tx=0x7f2acc02c000 comp rx=0 tx=0).stop 2024-04-25T10:18:02.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.351+0000 7f2ae1a00640 1 -- 172.21.15.119:0/2776249947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc145260 msgr2=0x7f2adc0acef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.351+0000 7f2ae1a00640 1 --2- 172.21.15.119:0/2776249947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc145260 0x7f2adc0acef0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400c780 tx=0x7f2ad400cc40 comp rx=0 tx=0).stop 2024-04-25T10:18:02.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.351+0000 7f2ae1a00640 1 -- 172.21.15.119:0/2776249947 shutdown_connections 2024-04-25T10:18:02.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.351+0000 7f2ae1a00640 1 --2- 172.21.15.119:0/2776249947 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2ab806b4d0 0x7f2ab806d990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.351+0000 7f2ae1a00640 1 --2- 172.21.15.119:0/2776249947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2adc145260 0x7f2adc0acef0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.351+0000 7f2ae1a00640 1 --2- 172.21.15.119:0/2776249947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2adc0cc490 0x7f2adc131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.351+0000 7f2ae1a00640 1 --2- 172.21.15.119:0/2776249947 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2adc071740 0x7f2adc15b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.351+0000 7f2ae1a00640 1 -- 172.21.15.119:0/2776249947 >> 172.21.15.119:0/2776249947 conn(0x7f2adc06aba0 msgr2=0x7f2adc0af360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:02.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.351+0000 7f2ae1a00640 1 -- 172.21.15.119:0/2776249947 shutdown_connections 2024-04-25T10:18:02.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.351+0000 7f2ae1a00640 1 -- 172.21.15.119:0/2776249947 wait complete. 2024-04-25T10:18:02.373 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:18:02.463 INFO:tasks.ceph.mds.c.smithi119.stderr:2024-04-25T10:18:02.460+0000 7ff8eec00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:18:02.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.565+0000 7ffa22600640 1 Processor -- start 2024-04-25T10:18:02.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.565+0000 7ffa22600640 1 -- start start 2024-04-25T10:18:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.566+0000 7ffa22600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffa1c06e8f0 0x7ffa1c15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.566+0000 7ffa22600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1c0ce6c0 con 0x7ffa1c15bf00 2024-04-25T10:18:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.566+0000 7ffa22600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1c0cc4e0 con 0x7ffa1c15c820 2024-04-25T10:18:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.566+0000 7ffa22600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa1c1455e0 con 0x7ffa1c06e8f0 2024-04-25T10:18:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.567+0000 7ffa1b400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ffa1c15bf00 0x7ffa1c0cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38304/0 (socket says 172.21.15.119:38304) 2024-04-25T10:18:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.567+0000 7ffa1b400640 1 -- 172.21.15.119:0/2916531829 learned_addr learned my addr 172.21.15.119:0/2916531829 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.567+0000 7ffa12a00640 1 --2- 172.21.15.119:0/2916531829 >> v2:172.21.15.132:3300/0 conn(0x7ffa1c06e8f0 0x7ffa1c15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.567+0000 7ffa1aa00640 1 -- 172.21.15.119:0/2916531829 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730255040 0 0) 0x7ffa0c001110 con 0x7ffa1c15bf00 2024-04-25T10:18:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.567+0000 7ffa1aa00640 1 -- 172.21.15.119:0/2916531829 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa00003520 con 0x7ffa1c15bf00 2024-04-25T10:18:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1aa00640 1 -- 172.21.15.119:0/2916531829 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 701186952 0 0) 0x7ffa00003520 con 0x7ffa1c15bf00 2024-04-25T10:18:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1aa00640 1 -- 172.21.15.119:0/2916531829 >> v2:172.21.15.132:3300/0 conn(0x7ffa1c06e8f0 msgr2=0x7ffa1c15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1aa00640 1 --2- 172.21.15.119:0/2916531829 >> v2:172.21.15.132:3300/0 conn(0x7ffa1c06e8f0 0x7ffa1c15c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1aa00640 1 -- 172.21.15.119:0/2916531829 >> v1:172.21.15.132:6789/0 conn(0x7ffa1c15c820 legacy=0x7ffa1c1654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1aa00640 1 -- 172.21.15.119:0/2916531829 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c0cc7b0 con 0x7ffa1c15bf00 2024-04-25T10:18:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1aa00640 1 -- 172.21.15.119:0/2916531829 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7ffa0c001840 con 0x7ffa1c15bf00 2024-04-25T10:18:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1aa00640 1 -- 172.21.15.119:0/2916531829 >> v1:172.21.15.119:6789/0 conn(0x7ffa1c15bf00 legacy=0x7ffa1c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1aa00640 1 --2- 172.21.15.119:0/2916531829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa00003a00 0x7ffa00003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1aa00640 1 --2- 172.21.15.119:0/2916531829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa00004800 0x7ffa00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1aa00640 1 --2- 172.21.15.119:0/2916531829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa000051c0 0x7ffa0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1aa00640 1 -- 172.21.15.119:0/2916531829 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c1455e0 con 0x7ffa000051c0 2024-04-25T10:18:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1aa00640 1 -- 172.21.15.119:0/2916531829 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0000c0c0 con 0x7ffa00003a00 2024-04-25T10:18:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1aa00640 1 -- 172.21.15.119:0/2916531829 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0000c2c0 con 0x7ffa00004800 2024-04-25T10:18:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1be00640 1 --2- 172.21.15.119:0/2916531829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa000051c0 0x7ffa0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.568+0000 7ffa1b400640 1 --2- 172.21.15.119:0/2916531829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa00003a00 0x7ffa00003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.569+0000 7ffa1be00640 1 -- 172.21.15.119:0/2916531829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa00004800 msgr2=0x7ffa00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.569+0000 7ffa12a00640 1 --2- 172.21.15.119:0/2916531829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa00004800 0x7ffa00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.569+0000 7ffa1be00640 1 --2- 172.21.15.119:0/2916531829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa00004800 0x7ffa00004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.569+0000 7ffa1be00640 1 -- 172.21.15.119:0/2916531829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa00003a00 msgr2=0x7ffa00003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.569+0000 7ffa1be00640 1 --2- 172.21.15.119:0/2916531829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa00003a00 0x7ffa00003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.569+0000 7ffa1be00640 1 -- 172.21.15.119:0/2916531829 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa00003520 con 0x7ffa000051c0 2024-04-25T10:18:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.569+0000 7ffa1b400640 1 --2- 172.21.15.119:0/2916531829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa00003a00 0x7ffa00003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.569+0000 7ffa1be00640 1 -- 172.21.15.119:0/2916531829 reap_dead start 2024-04-25T10:18:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.570+0000 7ffa1be00640 1 --2- 172.21.15.119:0/2916531829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa000051c0 0x7ffa0000b9c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ffa14001e50 tx=0x7ffa00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6abf4e54c090368 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.570+0000 7ffa1aa00640 1 -- 172.21.15.119:0/2916531829 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa14001a70 con 0x7ffa000051c0 2024-04-25T10:18:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.570+0000 7ffa1aa00640 1 -- 172.21.15.119:0/2916531829 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa14003970 con 0x7ffa000051c0 2024-04-25T10:18:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.570+0000 7ffa1aa00640 1 -- 172.21.15.119:0/2916531829 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa14003d60 con 0x7ffa000051c0 2024-04-25T10:18:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.570+0000 7ffa22600640 1 -- 172.21.15.119:0/2916531829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa000051c0 msgr2=0x7ffa0000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.570+0000 7ffa22600640 1 --2- 172.21.15.119:0/2916531829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa000051c0 0x7ffa0000b9c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ffa14001e50 tx=0x7ffa00005830 comp rx=0 tx=0).stop 2024-04-25T10:18:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.571+0000 7ffa22600640 1 -- 172.21.15.119:0/2916531829 shutdown_connections 2024-04-25T10:18:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.571+0000 7ffa22600640 1 --2- 172.21.15.119:0/2916531829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa000051c0 0x7ffa0000b9c0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.571+0000 7ffa22600640 1 -- 172.21.15.119:0/2916531829 >> 172.21.15.119:0/2916531829 conn(0x7ffa1c06aba0 msgr2=0x7ffa1c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.571+0000 7ffa22600640 1 -- 172.21.15.119:0/2916531829 shutdown_connections 2024-04-25T10:18:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.571+0000 7ffa22600640 1 -- 172.21.15.119:0/2916531829 wait complete. 2024-04-25T10:18:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.572+0000 7ffa22600640 1 Processor -- start 2024-04-25T10:18:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.572+0000 7ffa22600640 1 -- start start 2024-04-25T10:18:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.573+0000 7ffa22600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa000051c0 0x7ffa1c153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.573+0000 7ffa22600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa1c153940 0x7ffa1c144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.573+0000 7ffa22600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa1c071190 0x7ffa1c071640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.573+0000 7ffa22600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c0ccd70 con 0x7ffa1c153940 2024-04-25T10:18:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.573+0000 7ffa22600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c06d450 con 0x7ffa1c071190 2024-04-25T10:18:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.573+0000 7ffa22600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c0cd050 con 0x7ffa000051c0 2024-04-25T10:18:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.574+0000 7ffa12a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa1c153940 0x7ffa1c144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.574+0000 7ffa12a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa1c153940 0x7ffa1c144800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43890/0 (socket says 172.21.15.119:43890) 2024-04-25T10:18:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.574+0000 7ffa12a00640 1 -- 172.21.15.119:0/2264258018 learned_addr learned my addr 172.21.15.119:0/2264258018 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.574+0000 7ffa1be00640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa1c071190 0x7ffa1c071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.574+0000 7ffa1b400640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa000051c0 0x7ffa1c153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.574+0000 7ffa12a00640 1 -- 172.21.15.119:0/2264258018 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa000051c0 msgr2=0x7ffa1c153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.574+0000 7ffa12a00640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa000051c0 0x7ffa1c153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.574+0000 7ffa12a00640 1 -- 172.21.15.119:0/2264258018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa1c071190 msgr2=0x7ffa1c071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.574+0000 7ffa12a00640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa1c071190 0x7ffa1c071640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.574+0000 7ffa12a00640 1 -- 172.21.15.119:0/2264258018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c13b3c0 con 0x7ffa1c153940 2024-04-25T10:18:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.574+0000 7ffa1be00640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa1c071190 0x7ffa1c071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.574+0000 7ffa1b400640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa000051c0 0x7ffa1c153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.574+0000 7ffa12a00640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa1c153940 0x7ffa1c144800 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ffa040107f0 tx=0x7ffa04010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.575+0000 7ffa13e00640 1 -- 172.21.15.119:0/2264258018 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400f040 con 0x7ffa1c153940 2024-04-25T10:18:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.575+0000 7ffa13e00640 1 -- 172.21.15.119:0/2264258018 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0400eb60 con 0x7ffa1c153940 2024-04-25T10:18:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.575+0000 7ffa22600640 1 -- 172.21.15.119:0/2264258018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1c153dd0 con 0x7ffa1c153940 2024-04-25T10:18:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.575+0000 7ffa13e00640 1 -- 172.21.15.119:0/2264258018 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0401c400 con 0x7ffa1c153940 2024-04-25T10:18:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.575+0000 7ffa22600640 1 -- 172.21.15.119:0/2264258018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1c154390 con 0x7ffa1c153940 2024-04-25T10:18:02.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.576+0000 7ffa22600640 1 -- 172.21.15.119:0/2264258018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1c06bb30 con 0x7ffa1c153940 2024-04-25T10:18:02.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.577+0000 7ffa13e00640 1 -- 172.21.15.119:0/2264258018 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa0401c5a0 con 0x7ffa1c153940 2024-04-25T10:18:02.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.578+0000 7ffa13e00640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff9f806b4d0 0x7ff9f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.578+0000 7ffa13e00640 1 -- 172.21.15.119:0/2264258018 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ffa0408a8d0 con 0x7ffa1c153940 2024-04-25T10:18:02.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.578+0000 7ffa1b400640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff9f806b4d0 0x7ff9f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.579+0000 7ffa1b400640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff9f806b4d0 0x7ff9f806d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c000c00 tx=0x7ffa0c000ef0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.582+0000 7ffa13e00640 1 -- 172.21.15.119:0/2264258018 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa04018030 con 0x7ffa1c153940 2024-04-25T10:18:02.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.818+0000 7ffa22600640 1 -- 172.21.15.119:0/2264258018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffa1c0749b0 con 0x7ffa1c153940 2024-04-25T10:18:02.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.818+0000 7ffa13e00640 1 -- 172.21.15.119:0/2264258018 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffa040596c0 con 0x7ffa1c153940 2024-04-25T10:18:02.822 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:18:02.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.821+0000 7ffa22600640 1 -- 172.21.15.119:0/2264258018 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff9f806b4d0 msgr2=0x7ff9f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.821+0000 7ffa22600640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff9f806b4d0 0x7ff9f806d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c000c00 tx=0x7ffa0c000ef0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.821+0000 7ffa22600640 1 -- 172.21.15.119:0/2264258018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa1c153940 msgr2=0x7ffa1c144800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.821+0000 7ffa22600640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa1c153940 0x7ffa1c144800 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ffa040107f0 tx=0x7ffa04010cb0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.821+0000 7ffa22600640 1 -- 172.21.15.119:0/2264258018 shutdown_connections 2024-04-25T10:18:02.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.821+0000 7ffa22600640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff9f806b4d0 0x7ff9f806d990 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.821+0000 7ffa22600640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa1c071190 0x7ffa1c071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.821+0000 7ffa22600640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa1c153940 0x7ffa1c144800 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.821+0000 7ffa22600640 1 --2- 172.21.15.119:0/2264258018 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa000051c0 0x7ffa1c153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.821+0000 7ffa22600640 1 -- 172.21.15.119:0/2264258018 >> 172.21.15.119:0/2264258018 conn(0x7ffa1c06aba0 msgr2=0x7ffa1c13b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:02.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.821+0000 7ffa22600640 1 -- 172.21.15.119:0/2264258018 shutdown_connections 2024-04-25T10:18:02.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:02.821+0000 7ffa22600640 1 -- 172.21.15.119:0/2264258018 wait complete. 2024-04-25T10:18:02.843 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:18:02.843 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:18:02.843 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:18:02.843 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:18:02.843 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:02.843 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:18:02.843 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:18:02.844 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:18:02.844 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:18:02.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:02.844 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:18:02.888 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:02.896 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:02.897 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:18:02.897 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:18:02.897 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:18:02.897 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:02.897 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:18:02.897 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:02.897 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:18:02.897 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:02.898 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:18:02.898 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:18:02.898 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:02.898 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:18:02.898 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:18:02.898 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:02.898 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:18:02.898 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:02.898 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:18:02.898 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42006sec preferred_lft 42006sec 2024-04-25T10:18:02.898 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:18:02.898 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:02.899 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:18:02.899 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:02.899 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:18:02.899 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:02.899 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:18:02.899 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:02.899 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:18:02.899 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:18:02.899 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:18:02.899 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:02.899 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:18:02.899 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:18:02.899 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:18:02.900 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:02.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:02.900 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:18:03.003 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:18:03.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:03.007 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:18:03.102 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:18:03.103 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:18:03.103 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:18:03.103 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:18:03.103 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:18:03.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:03.103 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:18:03.103 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:18:03.103 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:18:03.103 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:18:03.104 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:18:03.104 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:18:03.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:03.247 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:18:03.295 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:18:03.295 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:18:03.296 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:18:03.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:03.296 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:18:03.382 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:03.383 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:03.434 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:18:03.434 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:03.435 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:03.483 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:18:03.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:03.483 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:03.598 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:03.599 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:18:03.599 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:18:03.620 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:03.620 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:18:03.620 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:18:03.620 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:18:03.620 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:18:03.646 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:03.646 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:18:03.647 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:18:03.647 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:18:03.647 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:18:03.647 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:18:03.647 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:18:03.647 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:18:03.651 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:18:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:18:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:18:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:18:03.651 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:18:03.655 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:18:03.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:03.655 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:03.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:03.726 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:03.778 INFO:teuthology.orchestra.run.smithi119.stdout:client6382 2024-04-25T10:18:03.778 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:18:03.778 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:03.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:03.823 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:18:03.933 INFO:teuthology.orchestra.run.smithi119.stdout:./ 2024-04-25T10:18:03.933 INFO:teuthology.orchestra.run.smithi119.stdout:./parent 2024-04-25T10:18:03.934 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/flushed 2024-04-25T10:18:03.934 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/flushed/charlie 2024-04-25T10:18:03.934 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/flushed/alpha 2024-04-25T10:18:03.934 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/unflushed 2024-04-25T10:18:03.934 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/unflushed/jfile 2024-04-25T10:18:03.935 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:03.935 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:03.935 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:03.935 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:03.935 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:18:03.935 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:04.084 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:18:04.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:04.097 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:04.097 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:04.098 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:04.098 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:04.098 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:18:04.098 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:04.279 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627776 2024-04-25T10:18:04.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:04.293 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:04.293 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:04.293 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:04.294 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:04.294 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:18:04.294 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:04.464 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627777 2024-04-25T10:18:04.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:04.477 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:04.477 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:04.478 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:04.478 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:04.478 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:18:04.478 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:04.661 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627780 2024-04-25T10:18:04.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:04.674 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:04.674 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:04.674 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:04.675 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:04.675 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:18:04.675 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:04.858 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627778 2024-04-25T10:18:04.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:04.872 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:04.872 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:04.872 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:04.872 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:04.872 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:18:04.872 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:05.055 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627781 2024-04-25T10:18:05.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:05.069 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:05.069 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:05.069 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:05.069 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:05.069 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:18:05.069 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:05.254 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627782 2024-04-25T10:18:05.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:05.267 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:18:05.267 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:05.363 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:18:05.363 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:05.413 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.659+0000 7f1db4600640 1 Processor -- start 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.659+0000 7f1db4600640 1 -- start start 2024-04-25T10:18:05.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.660+0000 7f1db4600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1dac15bf00 0x7f1dac0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.660+0000 7f1db4600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dac0cc310 0x7f1dac15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.661+0000 7f1db4600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dac0ce6c0 con 0x7f1dac15c820 2024-04-25T10:18:05.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.661+0000 7f1db4600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dac1455e0 con 0x7f1dac15bf00 2024-04-25T10:18:05.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.661+0000 7f1db4600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac06d450 con 0x7f1dac0cc310 2024-04-25T10:18:05.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.661+0000 7f1db1200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dac0cc310 0x7f1dac15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.662+0000 7f1db1c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1dac15bf00 0x7f1dac0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.662+0000 7f1db2600640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f1dac15c820 0x7f1dac06d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:59004/0 (socket says 172.21.15.119:59004) 2024-04-25T10:18:05.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.662+0000 7f1db2600640 1 -- 172.21.15.119:0/3870605643 learned_addr learned my addr 172.21.15.119:0/3870605643 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:05.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.662+0000 7f1db1200640 1 --2- 172.21.15.119:0/3870605643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dac0cc310 0x7f1dac15c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:48760/0 (socket says 172.21.15.119:48760) 2024-04-25T10:18:05.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.662+0000 7f1db1200640 1 -- 172.21.15.119:0/3870605643 >> v2:172.21.15.132:3300/0 conn(0x7f1dac15bf00 msgr2=0x7f1dac0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:05.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.662+0000 7f1db1200640 1 --2- 172.21.15.119:0/3870605643 >> v2:172.21.15.132:3300/0 conn(0x7f1dac15bf00 0x7f1dac0cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.662+0000 7f1db1200640 1 -- 172.21.15.119:0/3870605643 >> v1:172.21.15.132:6789/0 conn(0x7f1dac15c820 legacy=0x7f1dac06d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.662+0000 7f1db1200640 1 -- 172.21.15.119:0/3870605643 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac0cc7b0 con 0x7f1dac0cc310 2024-04-25T10:18:05.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.664+0000 7f1db1200640 1 --2- 172.21.15.119:0/3870605643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dac0cc310 0x7f1dac15c2e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1d980097a0 tx=0x7f1d9802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da41f9b6da01248 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.664+0000 7f1dabe00640 1 -- 172.21.15.119:0/3870605643 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d98037070 con 0x7f1dac0cc310 2024-04-25T10:18:05.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.664+0000 7f1dabe00640 1 -- 172.21.15.119:0/3870605643 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d98032440 con 0x7f1dac0cc310 2024-04-25T10:18:05.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.664+0000 7f1dabe00640 1 -- 172.21.15.119:0/3870605643 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d98032810 con 0x7f1dac0cc310 2024-04-25T10:18:05.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.665+0000 7f1db4600640 1 -- 172.21.15.119:0/3870605643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dac0cc310 msgr2=0x7f1dac15c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.665+0000 7f1db4600640 1 --2- 172.21.15.119:0/3870605643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dac0cc310 0x7f1dac15c2e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1d980097a0 tx=0x7f1d9802a180 comp rx=0 tx=0).stop 2024-04-25T10:18:05.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.665+0000 7f1db4600640 1 -- 172.21.15.119:0/3870605643 shutdown_connections 2024-04-25T10:18:05.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.665+0000 7f1db4600640 1 --2- 172.21.15.119:0/3870605643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dac0cc310 0x7f1dac15c2e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.665+0000 7f1db4600640 1 --2- 172.21.15.119:0/3870605643 >> v2:172.21.15.132:3300/0 conn(0x7f1dac15bf00 0x7f1dac0cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.665+0000 7f1db4600640 1 -- 172.21.15.119:0/3870605643 >> 172.21.15.119:0/3870605643 conn(0x7f1dac06aba0 msgr2=0x7f1dac06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:05.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.665+0000 7f1db4600640 1 -- 172.21.15.119:0/3870605643 shutdown_connections 2024-04-25T10:18:05.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.665+0000 7f1db4600640 1 -- 172.21.15.119:0/3870605643 wait complete. 2024-04-25T10:18:05.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.667+0000 7f1db4600640 1 Processor -- start 2024-04-25T10:18:05.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.667+0000 7f1db4600640 1 -- start start 2024-04-25T10:18:05.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.668+0000 7f1db4600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1dac0cc310 0x7f1dac14db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.668+0000 7f1db4600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dac15bf00 0x7f1dac1e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.668+0000 7f1db4600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dac15c820 0x7f1dac140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.668+0000 7f1db4600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac0cd050 con 0x7f1dac0cc310 2024-04-25T10:18:05.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.668+0000 7f1db4600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac0ccd70 con 0x7f1dac15bf00 2024-04-25T10:18:05.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.668+0000 7f1db4600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac15cca0 con 0x7f1dac15c820 2024-04-25T10:18:05.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.668+0000 7f1db1c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1dac0cc310 0x7f1dac14db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.668+0000 7f1db1c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1dac0cc310 0x7f1dac14db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43896/0 (socket says 172.21.15.119:43896) 2024-04-25T10:18:05.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.668+0000 7f1db1c00640 1 -- 172.21.15.119:0/1984459970 learned_addr learned my addr 172.21.15.119:0/1984459970 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:05.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.669+0000 7f1db2600640 1 --2- 172.21.15.119:0/1984459970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dac15c820 0x7f1dac140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.669+0000 7f1db1200640 1 --2- 172.21.15.119:0/1984459970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dac15bf00 0x7f1dac1e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.669+0000 7f1db1c00640 1 -- 172.21.15.119:0/1984459970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dac15c820 msgr2=0x7f1dac140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.669+0000 7f1db1c00640 1 --2- 172.21.15.119:0/1984459970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dac15c820 0x7f1dac140820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.669+0000 7f1db1c00640 1 -- 172.21.15.119:0/1984459970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dac15bf00 msgr2=0x7f1dac1e49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.669+0000 7f1db1c00640 1 --2- 172.21.15.119:0/1984459970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dac15bf00 0x7f1dac1e49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.669+0000 7f1db1c00640 1 -- 172.21.15.119:0/1984459970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac1f3690 con 0x7f1dac0cc310 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.669+0000 7f1db1c00640 1 --2- 172.21.15.119:0/1984459970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1dac0cc310 0x7f1dac14db10 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f1da000e9e0 tx=0x7f1da000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.669+0000 7f1da9600640 1 -- 172.21.15.119:0/1984459970 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0017070 con 0x7f1dac0cc310 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.670+0000 7f1da9600640 1 -- 172.21.15.119:0/1984459970 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da000c9d0 con 0x7f1dac0cc310 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.670+0000 7f1da9600640 1 -- 172.21.15.119:0/1984459970 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da000cdc0 con 0x7f1dac0cc310 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.670+0000 7f1db4600640 1 -- 172.21.15.119:0/1984459970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dac13f780 con 0x7f1dac0cc310 2024-04-25T10:18:05.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.670+0000 7f1db4600640 1 -- 172.21.15.119:0/1984459970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dac140020 con 0x7f1dac0cc310 2024-04-25T10:18:05.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.671+0000 7f1db4600640 1 -- 172.21.15.119:0/1984459970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dac06bb30 con 0x7f1dac0cc310 2024-04-25T10:18:05.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.672+0000 7f1da9600640 1 -- 172.21.15.119:0/1984459970 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1da001a070 con 0x7f1dac0cc310 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.673+0000 7f1da9600640 1 --2- 172.21.15.119:0/1984459970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d8c06b4d0 0x7f1d8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.673+0000 7f1da9600640 1 -- 172.21.15.119:0/1984459970 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1da008a2e0 con 0x7f1dac0cc310 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.673+0000 7f1db1200640 1 --2- 172.21.15.119:0/1984459970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d8c06b4d0 0x7f1d8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.674+0000 7f1db1200640 1 --2- 172.21.15.119:0/1984459970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d8c06b4d0 0x7f1d8c06d990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1d98034000 tx=0x7f1d9802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.678+0000 7f1da9600640 1 -- 172.21.15.119:0/1984459970 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1dac1f3690 con 0x7f1dac0cc310 2024-04-25T10:18:05.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.877+0000 7f1db4600640 1 -- 172.21.15.119:0/1984459970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1dac0cea20 con 0x7f1dac0cc310 2024-04-25T10:18:05.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.879+0000 7f1da9600640 1 -- 172.21.15.119:0/1984459970 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16947 (secure 0 0 0) 0x7f1da00590c0 con 0x7f1dac0cc310 2024-04-25T10:18:05.882 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:05.882 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":111,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:17:56.438907+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:35.422922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:36.634584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.25","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/1849584205":"2024-04-26T10:17:56.438859+0000","172.21.15.119:6836/1849584205":"2024-04-26T10:17:56.438859+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.881+0000 7f1db4600640 1 -- 172.21.15.119:0/1984459970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d8c06b4d0 msgr2=0x7f1d8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.881+0000 7f1db4600640 1 --2- 172.21.15.119:0/1984459970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d8c06b4d0 0x7f1d8c06d990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1d98034000 tx=0x7f1d9802e000 comp rx=0 tx=0).stop 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.881+0000 7f1db4600640 1 -- 172.21.15.119:0/1984459970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1dac0cc310 msgr2=0x7f1dac14db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.881+0000 7f1db4600640 1 --2- 172.21.15.119:0/1984459970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1dac0cc310 0x7f1dac14db10 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f1da000e9e0 tx=0x7f1da000eea0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.882+0000 7f1db4600640 1 -- 172.21.15.119:0/1984459970 shutdown_connections 2024-04-25T10:18:05.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.882+0000 7f1db4600640 1 --2- 172.21.15.119:0/1984459970 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1d8c06b4d0 0x7f1d8c06d990 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.882+0000 7f1db4600640 1 --2- 172.21.15.119:0/1984459970 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dac15c820 0x7f1dac140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.882+0000 7f1db4600640 1 --2- 172.21.15.119:0/1984459970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dac15bf00 0x7f1dac1e49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.882+0000 7f1db4600640 1 --2- 172.21.15.119:0/1984459970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1dac0cc310 0x7f1dac14db10 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.882+0000 7f1db4600640 1 -- 172.21.15.119:0/1984459970 >> 172.21.15.119:0/1984459970 conn(0x7f1dac06aba0 msgr2=0x7f1dac15e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:05.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.882+0000 7f1db4600640 1 -- 172.21.15.119:0/1984459970 shutdown_connections 2024-04-25T10:18:05.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:05.882+0000 7f1db4600640 1 -- 172.21.15.119:0/1984459970 wait complete. 2024-04-25T10:18:05.905 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:35.422922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:17:36.634584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:05.906 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:06.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.107+0000 7f2114600640 1 Processor -- start 2024-04-25T10:18:06.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.108+0000 7f2114600640 1 -- start start 2024-04-25T10:18:06.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.109+0000 7f2114600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f210c0ce9d0 0x7f210c0cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.109+0000 7f2114600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210c06dcb0 con 0x7f210c144e70 2024-04-25T10:18:06.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.109+0000 7f2114600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210c06df90 con 0x7f210c0cc300 2024-04-25T10:18:06.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.109+0000 7f2114600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210c15bfe0 con 0x7f210c0ce9d0 2024-04-25T10:18:06.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.109+0000 7f2111c00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f210c144e70 0x7f210c06d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38308/0 (socket says 172.21.15.119:38308) 2024-04-25T10:18:06.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.110+0000 7f2111c00640 1 -- 172.21.15.119:0/1460270768 learned_addr learned my addr 172.21.15.119:0/1460270768 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:06.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.109+0000 7f2111200640 1 --2- 172.21.15.119:0/1460270768 >> v2:172.21.15.119:3300/0 conn(0x7f210c0ce9d0 0x7f210c0cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.110+0000 7f2111200640 1 -- 172.21.15.119:0/1460270768 >> v1:172.21.15.132:6789/0 conn(0x7f210c0cc300 legacy=0x7f210c0cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.110+0000 7f2111200640 1 -- 172.21.15.119:0/1460270768 >> v1:172.21.15.119:6789/0 conn(0x7f210c144e70 legacy=0x7f210c06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.110+0000 7f2111200640 1 -- 172.21.15.119:0/1460270768 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c06e270 con 0x7f210c0ce9d0 2024-04-25T10:18:06.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.110+0000 7f210be00640 1 -- 172.21.15.119:0/1460270768 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860724496 0 0) 0x7f210c06dcb0 con 0x7f210c144e70 2024-04-25T10:18:06.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.111+0000 7f2111200640 1 --2- 172.21.15.119:0/1460270768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c0ce9d0 0x7f210c0cee60 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f210c15c160 tx=0x7f210002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8440a1b750b367 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.112+0000 7f210be00640 1 -- 172.21.15.119:0/1460270768 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100037070 con 0x7f210c0ce9d0 2024-04-25T10:18:06.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.112+0000 7f210be00640 1 -- 172.21.15.119:0/1460270768 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210002ae40 con 0x7f210c0ce9d0 2024-04-25T10:18:06.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.112+0000 7f210be00640 1 -- 172.21.15.119:0/1460270768 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100032660 con 0x7f210c0ce9d0 2024-04-25T10:18:06.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.112+0000 7f2114600640 1 -- 172.21.15.119:0/1460270768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c0ce9d0 msgr2=0x7f210c0cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.112+0000 7f2114600640 1 --2- 172.21.15.119:0/1460270768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c0ce9d0 0x7f210c0cee60 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f210c15c160 tx=0x7f210002a130 comp rx=0 tx=0).stop 2024-04-25T10:18:06.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.112+0000 7f2114600640 1 -- 172.21.15.119:0/1460270768 shutdown_connections 2024-04-25T10:18:06.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.112+0000 7f2114600640 1 --2- 172.21.15.119:0/1460270768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c0ce9d0 0x7f210c0cee60 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.112+0000 7f2114600640 1 -- 172.21.15.119:0/1460270768 >> 172.21.15.119:0/1460270768 conn(0x7f210c06aba0 msgr2=0x7f210c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.112+0000 7f2114600640 1 -- 172.21.15.119:0/1460270768 shutdown_connections 2024-04-25T10:18:06.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.112+0000 7f2114600640 1 -- 172.21.15.119:0/1460270768 wait complete. 2024-04-25T10:18:06.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.114+0000 7f2114600640 1 Processor -- start 2024-04-25T10:18:06.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.114+0000 7f2114600640 1 -- start start 2024-04-25T10:18:06.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.115+0000 7f2114600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f210c0cc300 0x7f210c0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.115+0000 7f2114600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f210c0ce9d0 0x7f210c13c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.115+0000 7f2114600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c144e70 0x7f210c1f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.115+0000 7f2114600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c06d970 con 0x7f210c144e70 2024-04-25T10:18:06.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.115+0000 7f2114600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0cf3a0 con 0x7f210c0ce9d0 2024-04-25T10:18:06.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.115+0000 7f2114600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c1454b0 con 0x7f210c0cc300 2024-04-25T10:18:06.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.115+0000 7f2112600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c144e70 0x7f210c1f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.115+0000 7f2112600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c144e70 0x7f210c1f3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43918/0 (socket says 172.21.15.119:43918) 2024-04-25T10:18:06.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.115+0000 7f2112600640 1 -- 172.21.15.119:0/4036866941 learned_addr learned my addr 172.21.15.119:0/4036866941 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:06.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.116+0000 7f2112600640 1 -- 172.21.15.119:0/4036866941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f210c0cc300 msgr2=0x7f210c0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.116+0000 7f2112600640 1 --2- 172.21.15.119:0/4036866941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f210c0cc300 0x7f210c0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.116+0000 7f2112600640 1 -- 172.21.15.119:0/4036866941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f210c0ce9d0 msgr2=0x7f210c13c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.116+0000 7f2112600640 1 --2- 172.21.15.119:0/4036866941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f210c0ce9d0 0x7f210c13c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.116+0000 7f2112600640 1 -- 172.21.15.119:0/4036866941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c0b06f0 con 0x7f210c144e70 2024-04-25T10:18:06.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.116+0000 7f2112600640 1 --2- 172.21.15.119:0/4036866941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c144e70 0x7f210c1f3840 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f210400c7b0 tx=0x7f210400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.116+0000 7f2109600640 1 -- 172.21.15.119:0/4036866941 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2104017070 con 0x7f210c144e70 2024-04-25T10:18:06.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.116+0000 7f2109600640 1 -- 172.21.15.119:0/4036866941 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2104003da0 con 0x7f210c144e70 2024-04-25T10:18:06.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.116+0000 7f2114600640 1 -- 172.21.15.119:0/4036866941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c0b09a0 con 0x7f210c144e70 2024-04-25T10:18:06.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.116+0000 7f2109600640 1 -- 172.21.15.119:0/4036866941 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21040052b0 con 0x7f210c144e70 2024-04-25T10:18:06.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.116+0000 7f2114600640 1 -- 172.21.15.119:0/4036866941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c0b0f60 con 0x7f210c144e70 2024-04-25T10:18:06.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.118+0000 7f2114600640 1 -- 172.21.15.119:0/4036866941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c0750b0 con 0x7f210c144e70 2024-04-25T10:18:06.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.119+0000 7f2109600640 1 -- 172.21.15.119:0/4036866941 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f210401a070 con 0x7f210c144e70 2024-04-25T10:18:06.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.120+0000 7f2109600640 1 --2- 172.21.15.119:0/4036866941 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f20f006b4d0 0x7f20f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.120+0000 7f2109600640 1 -- 172.21.15.119:0/4036866941 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2104089970 con 0x7f210c144e70 2024-04-25T10:18:06.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.120+0000 7f2111c00640 1 --2- 172.21.15.119:0/4036866941 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f20f006b4d0 0x7f20f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.121+0000 7f2111c00640 1 --2- 172.21.15.119:0/4036866941 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f20f006b4d0 0x7f20f006d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f20fc007fd0 tx=0x7f20fc007960 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.124+0000 7f2109600640 1 -- 172.21.15.119:0/4036866941 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21040586c0 con 0x7f210c144e70 2024-04-25T10:18:06.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.318+0000 7f2114600640 1 -- 172.21.15.119:0/4036866941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f210c143880 con 0x7f210c144e70 2024-04-25T10:18:06.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.319+0000 7f2109600640 1 -- 172.21.15.119:0/4036866941 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2104057fc0 con 0x7f210c144e70 2024-04-25T10:18:06.322 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:18:06.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.321+0000 7f2114600640 1 -- 172.21.15.119:0/4036866941 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f20f006b4d0 msgr2=0x7f20f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.321+0000 7f2114600640 1 --2- 172.21.15.119:0/4036866941 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f20f006b4d0 0x7f20f006d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f20fc007fd0 tx=0x7f20fc007960 comp rx=0 tx=0).stop 2024-04-25T10:18:06.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.321+0000 7f2114600640 1 -- 172.21.15.119:0/4036866941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c144e70 msgr2=0x7f210c1f3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.321+0000 7f2114600640 1 --2- 172.21.15.119:0/4036866941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c144e70 0x7f210c1f3840 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f210400c7b0 tx=0x7f210400cc70 comp rx=0 tx=0).stop 2024-04-25T10:18:06.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.322+0000 7f2114600640 1 -- 172.21.15.119:0/4036866941 shutdown_connections 2024-04-25T10:18:06.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.322+0000 7f2114600640 1 --2- 172.21.15.119:0/4036866941 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f20f006b4d0 0x7f20f006d990 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.322+0000 7f2114600640 1 --2- 172.21.15.119:0/4036866941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c144e70 0x7f210c1f3840 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.322+0000 7f2114600640 1 --2- 172.21.15.119:0/4036866941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f210c0ce9d0 0x7f210c13c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.322+0000 7f2114600640 1 --2- 172.21.15.119:0/4036866941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f210c0cc300 0x7f210c0b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.322+0000 7f2114600640 1 -- 172.21.15.119:0/4036866941 >> 172.21.15.119:0/4036866941 conn(0x7f210c06aba0 msgr2=0x7f210c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.322+0000 7f2114600640 1 -- 172.21.15.119:0/4036866941 shutdown_connections 2024-04-25T10:18:06.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.322+0000 7f2114600640 1 -- 172.21.15.119:0/4036866941 wait complete. 2024-04-25T10:18:06.343 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:06.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.547+0000 7f04c3a00640 1 Processor -- start 2024-04-25T10:18:06.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.547+0000 7f04c3a00640 1 -- start start 2024-04-25T10:18:06.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.549+0000 7f04c3a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f04bc0b43b0 0x7f04bc06ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.549+0000 7f04c3a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04bc09dd00 0x7f04bc06b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.549+0000 7f04c3a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04bc06efd0 con 0x7f04bc15a120 2024-04-25T10:18:06.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.549+0000 7f04c3a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04bc0b31d0 con 0x7f04bc0b43b0 2024-04-25T10:18:06.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.549+0000 7f04c3a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc06f2b0 con 0x7f04bc09dd00 2024-04-25T10:18:06.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.549+0000 7f04bbe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f04bc0b43b0 0x7f04bc06ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.550+0000 7f04bbe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f04bc0b43b0 0x7f04bc06ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43932/0 (socket says 172.21.15.119:43932) 2024-04-25T10:18:06.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.550+0000 7f04bbe00640 1 -- 172.21.15.119:0/2658971222 learned_addr learned my addr 172.21.15.119:0/2658971222 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:06.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.550+0000 7f04c1a00640 1 --2- 172.21.15.119:0/2658971222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04bc09dd00 0x7f04bc06b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.550+0000 7f04bbe00640 1 -- 172.21.15.119:0/2658971222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04bc09dd00 msgr2=0x7f04bc06b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.550+0000 7f04bbe00640 1 --2- 172.21.15.119:0/2658971222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04bc09dd00 0x7f04bc06b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.550+0000 7f04bbe00640 1 -- 172.21.15.119:0/2658971222 >> v1:172.21.15.119:6789/0 conn(0x7f04bc15a120 legacy=0x7f04bc0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.550+0000 7f04bbe00640 1 -- 172.21.15.119:0/2658971222 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc06f590 con 0x7f04bc0b43b0 2024-04-25T10:18:06.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.550+0000 7f04c1a00640 1 --2- 172.21.15.119:0/2658971222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04bc09dd00 0x7f04bc06b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:06.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.551+0000 7f04bbe00640 1 --2- 172.21.15.119:0/2658971222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04bc0b43b0 0x7f04bc06ad50 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009870 tx=0x7f04a802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac45efc915ede900 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.551+0000 7f04bb400640 1 -- 172.21.15.119:0/2658971222 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8037070 con 0x7f04bc0b43b0 2024-04-25T10:18:06.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.551+0000 7f04bb400640 1 -- 172.21.15.119:0/2658971222 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a8032440 con 0x7f04bc0b43b0 2024-04-25T10:18:06.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.551+0000 7f04bb400640 1 -- 172.21.15.119:0/2658971222 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8032810 con 0x7f04bc0b43b0 2024-04-25T10:18:06.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.552+0000 7f04c3a00640 1 -- 172.21.15.119:0/2658971222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04bc0b43b0 msgr2=0x7f04bc06ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.552+0000 7f04c3a00640 1 --2- 172.21.15.119:0/2658971222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04bc0b43b0 0x7f04bc06ad50 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009870 tx=0x7f04a802a1d0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.552+0000 7f04c3a00640 1 -- 172.21.15.119:0/2658971222 shutdown_connections 2024-04-25T10:18:06.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.552+0000 7f04c3a00640 1 --2- 172.21.15.119:0/2658971222 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04bc09dd00 0x7f04bc06b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.552+0000 7f04c3a00640 1 --2- 172.21.15.119:0/2658971222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04bc0b43b0 0x7f04bc06ad50 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.552+0000 7f04c3a00640 1 -- 172.21.15.119:0/2658971222 >> 172.21.15.119:0/2658971222 conn(0x7f04bc0cff20 msgr2=0x7f04bc0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.552+0000 7f04c3a00640 1 -- 172.21.15.119:0/2658971222 shutdown_connections 2024-04-25T10:18:06.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.552+0000 7f04c3a00640 1 -- 172.21.15.119:0/2658971222 wait complete. 2024-04-25T10:18:06.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.554+0000 7f04c3a00640 1 Processor -- start 2024-04-25T10:18:06.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.554+0000 7f04c3a00640 1 -- start start 2024-04-25T10:18:06.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.555+0000 7f04c3a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04bc09dd00 0x7f04bc144b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.555+0000 7f04c3a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04bc0b43b0 0x7f04bc0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.555+0000 7f04c3a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04bc15a120 0x7f04bc1f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.555+0000 7f04c3a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc0b3520 con 0x7f04bc15a120 2024-04-25T10:18:06.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.555+0000 7f04c3a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc0b33a0 con 0x7f04bc09dd00 2024-04-25T10:18:06.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.555+0000 7f04c3a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc0b36a0 con 0x7f04bc0b43b0 2024-04-25T10:18:06.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.556+0000 7f04c1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04bc15a120 0x7f04bc1f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.556+0000 7f04c1a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04bc15a120 0x7f04bc1f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43944/0 (socket says 172.21.15.119:43944) 2024-04-25T10:18:06.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.556+0000 7f04c1a00640 1 -- 172.21.15.119:0/2948493535 learned_addr learned my addr 172.21.15.119:0/2948493535 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:06.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.556+0000 7f04bbe00640 1 --2- 172.21.15.119:0/2948493535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04bc0b43b0 0x7f04bc0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.556+0000 7f04c1000640 1 --2- 172.21.15.119:0/2948493535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04bc09dd00 0x7f04bc144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.556+0000 7f04c1a00640 1 -- 172.21.15.119:0/2948493535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04bc0b43b0 msgr2=0x7f04bc0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.556+0000 7f04c1a00640 1 --2- 172.21.15.119:0/2948493535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04bc0b43b0 0x7f04bc0cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.556+0000 7f04c1a00640 1 -- 172.21.15.119:0/2948493535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04bc09dd00 msgr2=0x7f04bc144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.556+0000 7f04c1a00640 1 --2- 172.21.15.119:0/2948493535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04bc09dd00 0x7f04bc144b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.556+0000 7f04c1a00640 1 -- 172.21.15.119:0/2948493535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc09c130 con 0x7f04bc15a120 2024-04-25T10:18:06.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.556+0000 7f04c1a00640 1 --2- 172.21.15.119:0/2948493535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04bc15a120 0x7f04bc1f3750 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f04b400cd50 tx=0x7f04b400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.557+0000 7f04b8c00640 1 -- 172.21.15.119:0/2948493535 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b400f680 con 0x7f04bc15a120 2024-04-25T10:18:06.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.557+0000 7f04b8c00640 1 -- 172.21.15.119:0/2948493535 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b400f840 con 0x7f04bc15a120 2024-04-25T10:18:06.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.557+0000 7f04b8c00640 1 -- 172.21.15.119:0/2948493535 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b400fc30 con 0x7f04bc15a120 2024-04-25T10:18:06.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.557+0000 7f04c3a00640 1 -- 172.21.15.119:0/2948493535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04bc1f3ea0 con 0x7f04bc15a120 2024-04-25T10:18:06.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.557+0000 7f04c3a00640 1 -- 172.21.15.119:0/2948493535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04bc1f4460 con 0x7f04bc15a120 2024-04-25T10:18:06.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.558+0000 7f04c3a00640 1 -- 172.21.15.119:0/2948493535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04bc1f46a0 con 0x7f04bc15a120 2024-04-25T10:18:06.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.559+0000 7f04b8c00640 1 -- 172.21.15.119:0/2948493535 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04b40213f0 con 0x7f04bc15a120 2024-04-25T10:18:06.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.560+0000 7f04b8c00640 1 --2- 172.21.15.119:0/2948493535 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f049c06b4d0 0x7f049c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.560+0000 7f04b8c00640 1 -- 172.21.15.119:0/2948493535 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f04b408b0d0 con 0x7f04bc15a120 2024-04-25T10:18:06.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.560+0000 7f04c1000640 1 --2- 172.21.15.119:0/2948493535 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f049c06b4d0 0x7f049c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.561+0000 7f04c1000640 1 --2- 172.21.15.119:0/2948493535 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f049c06b4d0 0x7f049c06d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f04b0001640 tx=0x7f04b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.565+0000 7f04b8c00640 1 -- 172.21.15.119:0/2948493535 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04bc1f46a0 con 0x7f04bc15a120 2024-04-25T10:18:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.758+0000 7f04c3a00640 1 -- 172.21.15.119:0/2948493535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f04bc0750b0 con 0x7f04bc15a120 2024-04-25T10:18:06.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.759+0000 7f04b8c00640 1 -- 172.21.15.119:0/2948493535 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+11 (secure 0 0 0) 0x7f04b4014030 con 0x7f04bc15a120 2024-04-25T10:18:06.763 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:18:06.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.762+0000 7f04c3a00640 1 -- 172.21.15.119:0/2948493535 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f049c06b4d0 msgr2=0x7f049c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.762+0000 7f04c3a00640 1 --2- 172.21.15.119:0/2948493535 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f049c06b4d0 0x7f049c06d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f04b0001640 tx=0x7f04b0009040 comp rx=0 tx=0).stop 2024-04-25T10:18:06.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.762+0000 7f04c3a00640 1 -- 172.21.15.119:0/2948493535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04bc15a120 msgr2=0x7f04bc1f3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.762+0000 7f04c3a00640 1 --2- 172.21.15.119:0/2948493535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04bc15a120 0x7f04bc1f3750 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f04b400cd50 tx=0x7f04b400a650 comp rx=0 tx=0).stop 2024-04-25T10:18:06.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.762+0000 7f04c3a00640 1 -- 172.21.15.119:0/2948493535 shutdown_connections 2024-04-25T10:18:06.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.762+0000 7f04c3a00640 1 --2- 172.21.15.119:0/2948493535 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f049c06b4d0 0x7f049c06d990 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.762+0000 7f04c3a00640 1 --2- 172.21.15.119:0/2948493535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04bc15a120 0x7f04bc1f3750 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.762+0000 7f04c3a00640 1 --2- 172.21.15.119:0/2948493535 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04bc0b43b0 0x7f04bc0cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.762+0000 7f04c3a00640 1 --2- 172.21.15.119:0/2948493535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04bc09dd00 0x7f04bc144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.762+0000 7f04c3a00640 1 -- 172.21.15.119:0/2948493535 >> 172.21.15.119:0/2948493535 conn(0x7f04bc0cff20 msgr2=0x7f04bc152b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.763+0000 7f04c3a00640 1 -- 172.21.15.119:0/2948493535 shutdown_connections 2024-04-25T10:18:06.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.763+0000 7f04c3a00640 1 -- 172.21.15.119:0/2948493535 wait complete. 2024-04-25T10:18:06.787 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:06.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.988+0000 7f105fe00640 1 Processor -- start 2024-04-25T10:18:06.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.988+0000 7f105fe00640 1 -- start start 2024-04-25T10:18:06.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.990+0000 7f105fe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f105815dc80 0x7f1058161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.990+0000 7f105fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10580b5c00 0x7f10580d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.990+0000 7f105fe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10581627a0 con 0x7f10580b97c0 2024-04-25T10:18:06.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.990+0000 7f105fe00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10581621e0 con 0x7f105815dc80 2024-04-25T10:18:06.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.990+0000 7f105fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10581624c0 con 0x7f10580b5c00 2024-04-25T10:18:06.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.991+0000 7f105d400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f105815dc80 0x7f1058161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.991+0000 7f105d400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f105815dc80 0x7f1058161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43948/0 (socket says 172.21.15.119:43948) 2024-04-25T10:18:06.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.991+0000 7f105d400640 1 -- 172.21.15.119:0/2184406796 learned_addr learned my addr 172.21.15.119:0/2184406796 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:06.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.991+0000 7f105de00640 1 --2- 172.21.15.119:0/2184406796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10580b5c00 0x7f10580d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.991+0000 7f105d400640 1 -- 172.21.15.119:0/2184406796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10580b5c00 msgr2=0x7f10580d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.991+0000 7f105d400640 1 --2- 172.21.15.119:0/2184406796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10580b5c00 0x7f10580d2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.991+0000 7f105d400640 1 -- 172.21.15.119:0/2184406796 >> v1:172.21.15.132:6789/0 conn(0x7f10580b97c0 legacy=0x7f10580b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.991+0000 7f105d400640 1 -- 172.21.15.119:0/2184406796 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10580d1c70 con 0x7f105815dc80 2024-04-25T10:18:06.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.991+0000 7f105de00640 1 --2- 172.21.15.119:0/2184406796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10580b5c00 0x7f10580d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.992+0000 7f105d400640 1 --2- 172.21.15.119:0/2184406796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f105815dc80 0x7f1058161870 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f104c009e50 tx=0x7f104c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752230d25adf94be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.993+0000 7f1057e00640 1 -- 172.21.15.119:0/2184406796 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c036070 con 0x7f105815dc80 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.993+0000 7f1057e00640 1 -- 172.21.15.119:0/2184406796 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c031600 con 0x7f105815dc80 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.993+0000 7f1057e00640 1 -- 172.21.15.119:0/2184406796 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c0319d0 con 0x7f105815dc80 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.993+0000 7f105fe00640 1 -- 172.21.15.119:0/2184406796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f105815dc80 msgr2=0x7f1058161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.993+0000 7f105fe00640 1 --2- 172.21.15.119:0/2184406796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f105815dc80 0x7f1058161870 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f104c009e50 tx=0x7f104c02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.993+0000 7f105fe00640 1 -- 172.21.15.119:0/2184406796 shutdown_connections 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.993+0000 7f105fe00640 1 --2- 172.21.15.119:0/2184406796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10580b5c00 0x7f10580d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.993+0000 7f105fe00640 1 --2- 172.21.15.119:0/2184406796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f105815dc80 0x7f1058161870 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.994+0000 7f105fe00640 1 -- 172.21.15.119:0/2184406796 >> 172.21.15.119:0/2184406796 conn(0x7f10580d04f0 msgr2=0x7f10580d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.994+0000 7f105fe00640 1 -- 172.21.15.119:0/2184406796 shutdown_connections 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.994+0000 7f105fe00640 1 -- 172.21.15.119:0/2184406796 wait complete. 2024-04-25T10:18:06.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.995+0000 7f105fe00640 1 Processor -- start 2024-04-25T10:18:06.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.995+0000 7f105fe00640 1 -- start start 2024-04-25T10:18:07.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.996+0000 7f105fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10580b5c00 0x7f105815a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.996+0000 7f105fe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10580b97c0 0x7f10580cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.996+0000 7f105fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f105815dc80 0x7f10580aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.996+0000 7f105fe00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1058161f30 con 0x7f10580b97c0 2024-04-25T10:18:07.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.996+0000 7f105fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1058161db0 con 0x7f105815dc80 2024-04-25T10:18:07.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.996+0000 7f105fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10580d29b0 con 0x7f10580b5c00 2024-04-25T10:18:07.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.997+0000 7f105ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10580b97c0 0x7f10580cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.997+0000 7f105ca00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10580b97c0 0x7f10580cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43964/0 (socket says 172.21.15.119:43964) 2024-04-25T10:18:07.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.997+0000 7f105ca00640 1 -- 172.21.15.119:0/3499935040 learned_addr learned my addr 172.21.15.119:0/3499935040 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:07.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.997+0000 7f105d400640 1 --2- 172.21.15.119:0/3499935040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10580b5c00 0x7f105815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.997+0000 7f105de00640 1 --2- 172.21.15.119:0/3499935040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f105815dc80 0x7f10580aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.997+0000 7f105ca00640 1 -- 172.21.15.119:0/3499935040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10580b5c00 msgr2=0x7f105815a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.997+0000 7f105ca00640 1 --2- 172.21.15.119:0/3499935040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10580b5c00 0x7f105815a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.997+0000 7f105ca00640 1 -- 172.21.15.119:0/3499935040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f105815dc80 msgr2=0x7f10580aa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.997+0000 7f105ca00640 1 --2- 172.21.15.119:0/3499935040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f105815dc80 0x7f10580aa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.997+0000 7f105ca00640 1 -- 172.21.15.119:0/3499935040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105814a300 con 0x7f10580b97c0 2024-04-25T10:18:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.997+0000 7f105ca00640 1 --2- 172.21.15.119:0/3499935040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10580b97c0 0x7f10580cb1b0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f104400c440 tx=0x7f104400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.998+0000 7f1055600640 1 -- 172.21.15.119:0/3499935040 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044017070 con 0x7f10580b97c0 2024-04-25T10:18:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.998+0000 7f1055600640 1 -- 172.21.15.119:0/3499935040 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10440035f0 con 0x7f10580b97c0 2024-04-25T10:18:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.998+0000 7f1055600640 1 -- 172.21.15.119:0/3499935040 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10440039e0 con 0x7f10580b97c0 2024-04-25T10:18:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.998+0000 7f105fe00640 1 -- 172.21.15.119:0/3499935040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105814a5e0 con 0x7f10580b97c0 2024-04-25T10:18:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.998+0000 7f105fe00640 1 -- 172.21.15.119:0/3499935040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105814aba0 con 0x7f10580b97c0 2024-04-25T10:18:07.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:06.999+0000 7f105fe00640 1 -- 172.21.15.119:0/3499935040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10580750b0 con 0x7f10580b97c0 2024-04-25T10:18:07.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.000+0000 7f1055600640 1 -- 172.21.15.119:0/3499935040 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1044003b80 con 0x7f10580b97c0 2024-04-25T10:18:07.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.001+0000 7f1055600640 1 --2- 172.21.15.119:0/3499935040 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f103806b500 0x7f103806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.001+0000 7f1055600640 1 -- 172.21.15.119:0/3499935040 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1044089500 con 0x7f10580b97c0 2024-04-25T10:18:07.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.001+0000 7f105d400640 1 --2- 172.21.15.119:0/3499935040 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f103806b500 0x7f103806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.002+0000 7f105d400640 1 --2- 172.21.15.119:0/3499935040 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f103806b500 0x7f103806d9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f104c033000 tx=0x7f104c009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:07.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.006+0000 7f1055600640 1 -- 172.21.15.119:0/3499935040 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1044058250 con 0x7f10580b97c0 2024-04-25T10:18:07.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.207+0000 7f105fe00640 1 -- 172.21.15.119:0/3499935040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10580d1780 con 0x7f10580b97c0 2024-04-25T10:18:07.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.208+0000 7f1055600640 1 -- 172.21.15.119:0/3499935040 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+11 (secure 0 0 0) 0x7f10440010c0 con 0x7f10580b97c0 2024-04-25T10:18:07.211 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:18:07.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.210+0000 7f105fe00640 1 -- 172.21.15.119:0/3499935040 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f103806b500 msgr2=0x7f103806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.210+0000 7f105fe00640 1 --2- 172.21.15.119:0/3499935040 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f103806b500 0x7f103806d9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f104c033000 tx=0x7f104c009000 comp rx=0 tx=0).stop 2024-04-25T10:18:07.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.210+0000 7f105fe00640 1 -- 172.21.15.119:0/3499935040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10580b97c0 msgr2=0x7f10580cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.210+0000 7f105fe00640 1 --2- 172.21.15.119:0/3499935040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10580b97c0 0x7f10580cb1b0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f104400c440 tx=0x7f104400c900 comp rx=0 tx=0).stop 2024-04-25T10:18:07.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.210+0000 7f105fe00640 1 -- 172.21.15.119:0/3499935040 shutdown_connections 2024-04-25T10:18:07.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.210+0000 7f105fe00640 1 --2- 172.21.15.119:0/3499935040 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f103806b500 0x7f103806d9c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.210+0000 7f105fe00640 1 --2- 172.21.15.119:0/3499935040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f105815dc80 0x7f10580aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.210+0000 7f105fe00640 1 --2- 172.21.15.119:0/3499935040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10580b97c0 0x7f10580cb1b0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.210+0000 7f105fe00640 1 --2- 172.21.15.119:0/3499935040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10580b5c00 0x7f105815a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.210+0000 7f105fe00640 1 -- 172.21.15.119:0/3499935040 >> 172.21.15.119:0/3499935040 conn(0x7f10580d04f0 msgr2=0x7f10580b8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:07.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.211+0000 7f105fe00640 1 -- 172.21.15.119:0/3499935040 shutdown_connections 2024-04-25T10:18:07.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.211+0000 7f105fe00640 1 -- 172.21.15.119:0/3499935040 wait complete. 2024-04-25T10:18:07.234 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:18:07.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.429+0000 7f1b54c00640 1 Processor -- start 2024-04-25T10:18:07.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.430+0000 7f1b54c00640 1 -- start start 2024-04-25T10:18:07.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.431+0000 7f1b54c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b5006b2b0 0x7f1b500cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.431+0000 7f1b54c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1b500cc810 0x7f1b500b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.431+0000 7f1b54c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5015f400 con 0x7f1b501450e0 2024-04-25T10:18:07.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.431+0000 7f1b54c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b501454c0 con 0x7f1b500cc810 2024-04-25T10:18:07.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.431+0000 7f1b54c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5015f6e0 con 0x7f1b5006b2b0 2024-04-25T10:18:07.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.432+0000 7f1b4f400640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f1b501450e0 0x7f1b5006aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:59030/0 (socket says 172.21.15.119:59030) 2024-04-25T10:18:07.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.432+0000 7f1b4f400640 1 -- 172.21.15.119:0/137313151 learned_addr learned my addr 172.21.15.119:0/137313151 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:07.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.432+0000 7f1b4ea00640 1 --2- 172.21.15.119:0/137313151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b5006b2b0 0x7f1b500cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.432+0000 7f1b4fe00640 1 --2- 172.21.15.119:0/137313151 >> v2:172.21.15.132:3300/0 conn(0x7f1b500cc810 0x7f1b500b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.433+0000 7f1b4ea00640 1 -- 172.21.15.119:0/137313151 >> v2:172.21.15.132:3300/0 conn(0x7f1b500cc810 msgr2=0x7f1b500b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:07.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.433+0000 7f1b4ea00640 1 --2- 172.21.15.119:0/137313151 >> v2:172.21.15.132:3300/0 conn(0x7f1b500cc810 0x7f1b500b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.433+0000 7f1b4ea00640 1 -- 172.21.15.119:0/137313151 >> v1:172.21.15.132:6789/0 conn(0x7f1b501450e0 legacy=0x7f1b5006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.433+0000 7f1b4ea00640 1 -- 172.21.15.119:0/137313151 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5015f9c0 con 0x7f1b5006b2b0 2024-04-25T10:18:07.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.434+0000 7f1b4ea00640 1 --2- 172.21.15.119:0/137313151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b5006b2b0 0x7f1b500cc2d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0097a0 tx=0x7f1b3c02a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c9a5417fda65efe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:07.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.434+0000 7f1b4e000640 1 -- 172.21.15.119:0/137313151 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c037070 con 0x7f1b5006b2b0 2024-04-25T10:18:07.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.435+0000 7f1b4e000640 1 -- 172.21.15.119:0/137313151 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c02adb0 con 0x7f1b5006b2b0 2024-04-25T10:18:07.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.435+0000 7f1b4e000640 1 -- 172.21.15.119:0/137313151 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c032660 con 0x7f1b5006b2b0 2024-04-25T10:18:07.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.435+0000 7f1b54c00640 1 -- 172.21.15.119:0/137313151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b5006b2b0 msgr2=0x7f1b500cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.435+0000 7f1b54c00640 1 --2- 172.21.15.119:0/137313151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b5006b2b0 0x7f1b500cc2d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0097a0 tx=0x7f1b3c02a060 comp rx=0 tx=0).stop 2024-04-25T10:18:07.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.436+0000 7f1b54c00640 1 -- 172.21.15.119:0/137313151 shutdown_connections 2024-04-25T10:18:07.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.436+0000 7f1b54c00640 1 --2- 172.21.15.119:0/137313151 >> v2:172.21.15.132:3300/0 conn(0x7f1b500cc810 0x7f1b500b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.436+0000 7f1b54c00640 1 --2- 172.21.15.119:0/137313151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b5006b2b0 0x7f1b500cc2d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.436+0000 7f1b54c00640 1 -- 172.21.15.119:0/137313151 >> 172.21.15.119:0/137313151 conn(0x7f1b50066ff0 msgr2=0x7f1b500b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:07.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.436+0000 7f1b54c00640 1 -- 172.21.15.119:0/137313151 shutdown_connections 2024-04-25T10:18:07.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.436+0000 7f1b54c00640 1 -- 172.21.15.119:0/137313151 wait complete. 2024-04-25T10:18:07.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.437+0000 7f1b54c00640 1 Processor -- start 2024-04-25T10:18:07.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.437+0000 7f1b54c00640 1 -- start start 2024-04-25T10:18:07.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.438+0000 7f1b54c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b5006b2b0 0x7f1b5015b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.438+0000 7f1b54c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b500cc810 0x7f1b5006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.438+0000 7f1b54c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b501450e0 0x7f1b5006e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.438+0000 7f1b54c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500cce00 con 0x7f1b500cc810 2024-04-25T10:18:07.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.438+0000 7f1b54c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500ccc80 con 0x7f1b501450e0 2024-04-25T10:18:07.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.438+0000 7f1b54c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500b33c0 con 0x7f1b5006b2b0 2024-04-25T10:18:07.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.439+0000 7f1b4ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b500cc810 0x7f1b5006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.439+0000 7f1b4ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b500cc810 0x7f1b5006e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43978/0 (socket says 172.21.15.119:43978) 2024-04-25T10:18:07.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.439+0000 7f1b4ea00640 1 -- 172.21.15.119:0/363651273 learned_addr learned my addr 172.21.15.119:0/363651273 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:07.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.439+0000 7f1b4f400640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b5006b2b0 0x7f1b5015b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.439+0000 7f1b4fe00640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b501450e0 0x7f1b5006e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.439+0000 7f1b4ea00640 1 -- 172.21.15.119:0/363651273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b5006b2b0 msgr2=0x7f1b5015b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.439+0000 7f1b4ea00640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b5006b2b0 0x7f1b5015b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.439+0000 7f1b4ea00640 1 -- 172.21.15.119:0/363651273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b501450e0 msgr2=0x7f1b5006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.439+0000 7f1b4ea00640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b501450e0 0x7f1b5006e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.439+0000 7f1b4ea00640 1 -- 172.21.15.119:0/363651273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b501f71d0 con 0x7f1b500cc810 2024-04-25T10:18:07.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.439+0000 7f1b4fe00640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b501450e0 0x7f1b5006e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:07.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.440+0000 7f1b4ea00640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b500cc810 0x7f1b5006e0d0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c035040 tx=0x7f1b3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:07.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.440+0000 7f1b47400640 1 -- 172.21.15.119:0/363651273 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c041070 con 0x7f1b500cc810 2024-04-25T10:18:07.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.440+0000 7f1b47400640 1 -- 172.21.15.119:0/363651273 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c03c680 con 0x7f1b500cc810 2024-04-25T10:18:07.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.440+0000 7f1b47400640 1 -- 172.21.15.119:0/363651273 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c037040 con 0x7f1b500cc810 2024-04-25T10:18:07.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.440+0000 7f1b54c00640 1 -- 172.21.15.119:0/363651273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b501f7470 con 0x7f1b500cc810 2024-04-25T10:18:07.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.440+0000 7f1b54c00640 1 -- 172.21.15.119:0/363651273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b501f7a30 con 0x7f1b500cc810 2024-04-25T10:18:07.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.441+0000 7f1b54c00640 1 -- 172.21.15.119:0/363651273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b500749b0 con 0x7f1b500cc810 2024-04-25T10:18:07.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.442+0000 7f1b47400640 1 -- 172.21.15.119:0/363651273 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b3c03a070 con 0x7f1b500cc810 2024-04-25T10:18:07.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.443+0000 7f1b47400640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b3006b4d0 0x7f1b3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.443+0000 7f1b47400640 1 -- 172.21.15.119:0/363651273 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1b3c0aa600 con 0x7f1b500cc810 2024-04-25T10:18:07.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.443+0000 7f1b4f400640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b3006b4d0 0x7f1b3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.444+0000 7f1b4f400640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b3006b4d0 0x7f1b3006d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1b40000f60 tx=0x7f1b40009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:07.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.447+0000 7f1b47400640 1 -- 172.21.15.119:0/363651273 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b3c079350 con 0x7f1b500cc810 2024-04-25T10:18:07.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.653+0000 7f1b54c00640 1 -- 172.21.15.119:0/363651273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1b500cbc50 con 0x7f1b500cc810 2024-04-25T10:18:07.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.654+0000 7f1b47400640 1 -- 172.21.15.119:0/363651273 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v111) v1 ==== 80+0+124 (secure 0 0 0) 0x7f1b3c0a8e20 con 0x7f1b500cc810 2024-04-25T10:18:07.657 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6837/1849584205 2024-04-26T10:17:56.438859+0000 2024-04-25T10:18:07.657 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6836/1849584205 2024-04-26T10:17:56.438859+0000 2024-04-25T10:18:07.658 INFO:teuthology.orchestra.run.smithi119.stderr:listed 2 entries 2024-04-25T10:18:07.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.657+0000 7f1b54c00640 1 -- 172.21.15.119:0/363651273 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b3006b4d0 msgr2=0x7f1b3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.657+0000 7f1b54c00640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b3006b4d0 0x7f1b3006d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1b40000f60 tx=0x7f1b40009040 comp rx=0 tx=0).stop 2024-04-25T10:18:07.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.657+0000 7f1b54c00640 1 -- 172.21.15.119:0/363651273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b500cc810 msgr2=0x7f1b5006e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.657+0000 7f1b54c00640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b500cc810 0x7f1b5006e0d0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c035040 tx=0x7f1b3c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.657+0000 7f1b54c00640 1 -- 172.21.15.119:0/363651273 shutdown_connections 2024-04-25T10:18:07.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.657+0000 7f1b54c00640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1b3006b4d0 0x7f1b3006d990 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.657+0000 7f1b54c00640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b501450e0 0x7f1b5006e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.657+0000 7f1b54c00640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b500cc810 0x7f1b5006e0d0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.657+0000 7f1b54c00640 1 --2- 172.21.15.119:0/363651273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1b5006b2b0 0x7f1b5015b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.657+0000 7f1b54c00640 1 -- 172.21.15.119:0/363651273 >> 172.21.15.119:0/363651273 conn(0x7f1b50066ff0 msgr2=0x7f1b500af560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:07.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.657+0000 7f1b54c00640 1 -- 172.21.15.119:0/363651273 shutdown_connections 2024-04-25T10:18:07.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:07.657+0000 7f1b54c00640 1 -- 172.21.15.119:0/363651273 wait complete. 2024-04-25T10:18:07.680 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:18:07.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:07.680 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:07.766 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:18:07.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:07.766 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:18:07.816 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:08.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.009+0000 7fc637200640 1 Processor -- start 2024-04-25T10:18:08.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.010+0000 7fc637200640 1 -- start start 2024-04-25T10:18:08.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.011+0000 7fc637200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6380ce560 0x7fc6380ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.011+0000 7fc637200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6380d3340 0x7fc6380b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.011+0000 7fc637200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6380cee80 con 0x7fc63815dc80 2024-04-25T10:18:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.011+0000 7fc637200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6380d0e30 con 0x7fc6380d3340 2024-04-25T10:18:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.011+0000 7fc637200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6380d2b50 con 0x7fc6380ce560 2024-04-25T10:18:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.011+0000 7fc636800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6380d3340 0x7fc6380b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.011+0000 7fc636800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6380d3340 0x7fc6380b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43980/0 (socket says 172.21.15.119:43980) 2024-04-25T10:18:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.011+0000 7fc636800640 1 -- 172.21.15.119:0/3444901019 learned_addr learned my addr 172.21.15.119:0/3444901019 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:08.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.012+0000 7fc635e00640 1 --2- 172.21.15.119:0/3444901019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6380ce560 0x7fc6380ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.012+0000 7fc636800640 1 -- 172.21.15.119:0/3444901019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6380ce560 msgr2=0x7fc6380ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.012+0000 7fc636800640 1 --2- 172.21.15.119:0/3444901019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6380ce560 0x7fc6380ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.012+0000 7fc636800640 1 -- 172.21.15.119:0/3444901019 >> v1:172.21.15.119:6789/0 conn(0x7fc63815dc80 legacy=0x7fc6380b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.012+0000 7fc636800640 1 -- 172.21.15.119:0/3444901019 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6380d1c80 con 0x7fc6380d3340 2024-04-25T10:18:08.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.013+0000 7fc636800640 1 --2- 172.21.15.119:0/3444901019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6380d3340 0x7fc6380b5ab0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fc630009e50 tx=0x7fc63002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26eb99becfc797c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.013+0000 7fc634a00640 1 -- 172.21.15.119:0/3444901019 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc630036070 con 0x7fc6380d3340 2024-04-25T10:18:08.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.013+0000 7fc634a00640 1 -- 172.21.15.119:0/3444901019 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc630031600 con 0x7fc6380d3340 2024-04-25T10:18:08.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.013+0000 7fc634a00640 1 -- 172.21.15.119:0/3444901019 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6300319d0 con 0x7fc6380d3340 2024-04-25T10:18:08.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.014+0000 7fc637200640 1 -- 172.21.15.119:0/3444901019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6380d3340 msgr2=0x7fc6380b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.014+0000 7fc637200640 1 --2- 172.21.15.119:0/3444901019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6380d3340 0x7fc6380b5ab0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fc630009e50 tx=0x7fc63002a730 comp rx=0 tx=0).stop 2024-04-25T10:18:08.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.014+0000 7fc637200640 1 -- 172.21.15.119:0/3444901019 shutdown_connections 2024-04-25T10:18:08.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.014+0000 7fc637200640 1 --2- 172.21.15.119:0/3444901019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6380d3340 0x7fc6380b5ab0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.014+0000 7fc637200640 1 --2- 172.21.15.119:0/3444901019 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6380ce560 0x7fc6380ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.014+0000 7fc637200640 1 -- 172.21.15.119:0/3444901019 >> 172.21.15.119:0/3444901019 conn(0x7fc63806bde0 msgr2=0x7fc63806c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.014+0000 7fc637200640 1 -- 172.21.15.119:0/3444901019 shutdown_connections 2024-04-25T10:18:08.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.014+0000 7fc637200640 1 -- 172.21.15.119:0/3444901019 wait complete. 2024-04-25T10:18:08.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.016+0000 7fc637200640 1 Processor -- start 2024-04-25T10:18:08.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.016+0000 7fc637200640 1 -- start start 2024-04-25T10:18:08.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.017+0000 7fc637200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6380ce560 0x7fc6380b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.017+0000 7fc637200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6380d3340 0x7fc6380b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.017+0000 7fc637200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63815dc80 0x7fc6381509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.017+0000 7fc637200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6380d2500 con 0x7fc6380ce560 2024-04-25T10:18:08.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.017+0000 7fc637200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6380d2220 con 0x7fc63815dc80 2024-04-25T10:18:08.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.017+0000 7fc637200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6380d27e0 con 0x7fc6380d3340 2024-04-25T10:18:08.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.017+0000 7fc635e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6380ce560 0x7fc6380b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.017+0000 7fc635e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6380ce560 0x7fc6380b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43990/0 (socket says 172.21.15.119:43990) 2024-04-25T10:18:08.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.017+0000 7fc635e00640 1 -- 172.21.15.119:0/2415226143 learned_addr learned my addr 172.21.15.119:0/2415226143 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:08.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.018+0000 7fc635400640 1 --2- 172.21.15.119:0/2415226143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6380d3340 0x7fc6380b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.018+0000 7fc635e00640 1 -- 172.21.15.119:0/2415226143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6380d3340 msgr2=0x7fc6380b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.018+0000 7fc636800640 1 --2- 172.21.15.119:0/2415226143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63815dc80 0x7fc6381509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.018+0000 7fc635e00640 1 --2- 172.21.15.119:0/2415226143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6380d3340 0x7fc6380b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.018+0000 7fc635e00640 1 -- 172.21.15.119:0/2415226143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63815dc80 msgr2=0x7fc6381509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.018+0000 7fc635e00640 1 --2- 172.21.15.119:0/2415226143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63815dc80 0x7fc6381509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.018+0000 7fc635e00640 1 -- 172.21.15.119:0/2415226143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6380b38c0 con 0x7fc6380ce560 2024-04-25T10:18:08.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.018+0000 7fc635e00640 1 --2- 172.21.15.119:0/2415226143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6380ce560 0x7fc6380b3380 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fc62800d830 tx=0x7fc62800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.019+0000 7fc62e000640 1 -- 172.21.15.119:0/2415226143 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62800b6c0 con 0x7fc6380ce560 2024-04-25T10:18:08.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.019+0000 7fc62e000640 1 -- 172.21.15.119:0/2415226143 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc628019070 con 0x7fc6380ce560 2024-04-25T10:18:08.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.019+0000 7fc62e000640 1 -- 172.21.15.119:0/2415226143 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62800ba80 con 0x7fc6380ce560 2024-04-25T10:18:08.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.019+0000 7fc637200640 1 -- 172.21.15.119:0/2415226143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6380b3bf0 con 0x7fc6380ce560 2024-04-25T10:18:08.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.019+0000 7fc637200640 1 -- 172.21.15.119:0/2415226143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6380b4430 con 0x7fc6380ce560 2024-04-25T10:18:08.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.020+0000 7fc637200640 1 -- 172.21.15.119:0/2415226143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc63815ac30 con 0x7fc6380ce560 2024-04-25T10:18:08.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.021+0000 7fc62e000640 1 -- 172.21.15.119:0/2415226143 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc62801e460 con 0x7fc6380ce560 2024-04-25T10:18:08.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.022+0000 7fc62e000640 1 --2- 172.21.15.119:0/2415226143 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc61406b4d0 0x7fc61406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.022+0000 7fc62e000640 1 -- 172.21.15.119:0/2415226143 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc62808a8b0 con 0x7fc6380ce560 2024-04-25T10:18:08.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.022+0000 7fc635400640 1 --2- 172.21.15.119:0/2415226143 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc61406b4d0 0x7fc61406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.023+0000 7fc635400640 1 --2- 172.21.15.119:0/2415226143 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc61406b4d0 0x7fc61406d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc6381f6fa0 tx=0x7fc620009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.026+0000 7fc62e000640 1 -- 172.21.15.119:0/2415226143 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc63815ac30 con 0x7fc6380ce560 2024-04-25T10:18:08.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.257+0000 7fc637200640 1 -- 172.21.15.119:0/2415226143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6380cf880 con 0x7fc6380ce560 2024-04-25T10:18:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.258+0000 7fc62e000640 1 -- 172.21.15.119:0/2415226143 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4957 (secure 0 0 0) 0x7fc6380b3bf0 con 0x7fc6380ce560 2024-04-25T10:18:08.262 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:08.262 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3394261143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3394261143},{"type":"v1","addr":"172.21.15.132:6834","nonce":3394261143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2502874931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2502874931},{"type":"v1","addr":"172.21.15.119:6835","nonce":2502874931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":8694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2305428003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2305428003},{"type":"v1","addr":"172.21.15.132:6835","nonce":2305428003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:37.617787+0000","modified":"2024-04-25T10:17:59.464926+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8943":{"gid":8943,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/2864575636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2864575636},{"type":"v1","addr":"172.21.15.119:6837","nonce":2864575636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:18:08.262 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 105 2024-04-25T10:18:08.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.261+0000 7fc637200640 1 -- 172.21.15.119:0/2415226143 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc61406b4d0 msgr2=0x7fc61406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.261+0000 7fc637200640 1 --2- 172.21.15.119:0/2415226143 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc61406b4d0 0x7fc61406d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc6381f6fa0 tx=0x7fc620009040 comp rx=0 tx=0).stop 2024-04-25T10:18:08.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.261+0000 7fc637200640 1 -- 172.21.15.119:0/2415226143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6380ce560 msgr2=0x7fc6380b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.261+0000 7fc637200640 1 --2- 172.21.15.119:0/2415226143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6380ce560 0x7fc6380b3380 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fc62800d830 tx=0x7fc62800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.261+0000 7fc637200640 1 -- 172.21.15.119:0/2415226143 shutdown_connections 2024-04-25T10:18:08.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.261+0000 7fc637200640 1 --2- 172.21.15.119:0/2415226143 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc61406b4d0 0x7fc61406d990 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.261+0000 7fc637200640 1 --2- 172.21.15.119:0/2415226143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63815dc80 0x7fc6381509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.261+0000 7fc637200640 1 --2- 172.21.15.119:0/2415226143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc6380d3340 0x7fc6380b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.261+0000 7fc637200640 1 --2- 172.21.15.119:0/2415226143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6380ce560 0x7fc6380b3380 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.261+0000 7fc637200640 1 -- 172.21.15.119:0/2415226143 >> 172.21.15.119:0/2415226143 conn(0x7fc63806bde0 msgr2=0x7fc6380b7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.261+0000 7fc637200640 1 -- 172.21.15.119:0/2415226143 shutdown_connections 2024-04-25T10:18:08.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.261+0000 7fc637200640 1 -- 172.21.15.119:0/2415226143 wait complete. 2024-04-25T10:18:08.284 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2024-04-25T10:18:08.499 INFO:teuthology.orchestra.run.smithi119.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-25T10:18:08.499 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-25T10:18:08.500 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000003.00000000 scrub_tag 2024-04-25T10:18:08.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.600+0000 7fd8b558ef40 1 Processor -- start 2024-04-25T10:18:08.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.600+0000 7fd8b558ef40 1 -- start start 2024-04-25T10:18:08.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.601+0000 7fd8b558ef40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5584c066cff0 0x5584c07176e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.601+0000 7fd8b558ef40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584c066f020 0x5584c0710460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.601+0000 7fd8b558ef40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584c070fff0 0x5584c070ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.601+0000 7fd8b558ef40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c066f5e0 con 0x5584c070fff0 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.602+0000 7fd8b558ef40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c05e7dd0 con 0x5584c066f020 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.602+0000 7fd8b558ef40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584c066f880 con 0x5584c066cff0 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.602+0000 7fd8b3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584c070fff0 0x5584c070ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.602+0000 7fd8b3200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584c070fff0 0x5584c070ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44006/0 (socket says 172.21.15.119:44006) 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.602+0000 7fd8b3200640 1 -- 172.21.15.119:0/3245966329 learned_addr learned my addr 172.21.15.119:0/3245966329 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.602+0000 7fd8b2800640 1 --2- 172.21.15.119:0/3245966329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5584c066cff0 0x5584c07176e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.602+0000 7fd8b3200640 1 -- 172.21.15.119:0/3245966329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5584c066cff0 msgr2=0x5584c07176e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.602+0000 7fd8b3200640 1 --2- 172.21.15.119:0/3245966329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5584c066cff0 0x5584c07176e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.602+0000 7fd8b1e00640 1 --2- 172.21.15.119:0/3245966329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584c066f020 0x5584c0710460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.602+0000 7fd8b3200640 1 -- 172.21.15.119:0/3245966329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584c066f020 msgr2=0x5584c0710460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.602+0000 7fd8b3200640 1 --2- 172.21.15.119:0/3245966329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584c066f020 0x5584c0710460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.602+0000 7fd8b3200640 1 -- 172.21.15.119:0/3245966329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584c0715960 con 0x5584c070fff0 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.603+0000 7fd8b3200640 1 --2- 172.21.15.119:0/3245966329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584c070fff0 0x5584c070ddb0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00a7a0 tx=0x7fd8ac00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9413fd8308bad6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.603+0000 7fd8b0a00640 1 -- 172.21.15.119:0/3245966329 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac019070 con 0x5584c070fff0 2024-04-25T10:18:08.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.603+0000 7fd8b0a00640 1 -- 172.21.15.119:0/3245966329 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac007600 con 0x5584c070fff0 2024-04-25T10:18:08.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.603+0000 7fd8b0a00640 1 -- 172.21.15.119:0/3245966329 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0079d0 con 0x5584c070fff0 2024-04-25T10:18:08.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.603+0000 7fd8b558ef40 1 -- 172.21.15.119:0/3245966329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584c070fff0 msgr2=0x5584c070ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.604+0000 7fd8b558ef40 1 --2- 172.21.15.119:0/3245966329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584c070fff0 0x5584c070ddb0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00a7a0 tx=0x7fd8ac00a890 comp rx=0 tx=0).stop 2024-04-25T10:18:08.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.604+0000 7fd8b558ef40 1 -- 172.21.15.119:0/3245966329 shutdown_connections 2024-04-25T10:18:08.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.604+0000 7fd8b558ef40 1 --2- 172.21.15.119:0/3245966329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584c070fff0 0x5584c070ddb0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.604+0000 7fd8b558ef40 1 --2- 172.21.15.119:0/3245966329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584c066f020 0x5584c0710460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.604+0000 7fd8b558ef40 1 --2- 172.21.15.119:0/3245966329 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5584c066cff0 0x5584c07176e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.604+0000 7fd8b558ef40 1 -- 172.21.15.119:0/3245966329 >> 172.21.15.119:0/3245966329 conn(0x5584c0643980 msgr2=0x5584c0643d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.604+0000 7fd8b558ef40 1 -- 172.21.15.119:0/3245966329 shutdown_connections 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.604+0000 7fd8b558ef40 1 -- 172.21.15.119:0/3245966329 wait complete. 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.604+0000 7fd8b558ef40 1 Processor -- start 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.605+0000 7fd8b558ef40 1 -- start start 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b558ef40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5584c066cff0 0x5584c0712e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b558ef40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584c066f020 0x5584c070e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b558ef40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584c070fff0 0x5584c0708a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b558ef40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c070d630 con 0x5584c066f020 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b558ef40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c066ac30 con 0x5584c070fff0 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b558ef40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584c070d8d0 con 0x5584c066cff0 2024-04-25T10:18:08.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584c066f020 0x5584c070e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584c066f020 0x5584c070e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44020/0 (socket says 172.21.15.119:44020) 2024-04-25T10:18:08.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b1e00640 1 -- 172.21.15.119:0/600865323 learned_addr learned my addr 172.21.15.119:0/600865323 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:08.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b2800640 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5584c066cff0 0x5584c0712e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b3200640 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584c070fff0 0x5584c0708a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b1e00640 1 -- 172.21.15.119:0/600865323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5584c066cff0 msgr2=0x5584c0712e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b1e00640 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5584c066cff0 0x5584c0712e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b1e00640 1 -- 172.21.15.119:0/600865323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584c070fff0 msgr2=0x5584c0708a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b1e00640 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584c070fff0 0x5584c0708a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b1e00640 1 -- 172.21.15.119:0/600865323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584c0708f70 con 0x5584c066f020 2024-04-25T10:18:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.606+0000 7fd8b3200640 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584c070fff0 0x5584c0708a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.607+0000 7fd8b1e00640 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584c066f020 0x5584c070e2c0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00259e0 tx=0x7fd8a0003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.607+0000 7fd8ab400640 1 -- 172.21.15.119:0/600865323 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0056040 con 0x5584c066f020 2024-04-25T10:18:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.607+0000 7fd8ab400640 1 -- 172.21.15.119:0/600865323 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a0065630 con 0x5584c066f020 2024-04-25T10:18:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.607+0000 7fd8ab400640 1 -- 172.21.15.119:0/600865323 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0005a80 con 0x5584c066f020 2024-04-25T10:18:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.607+0000 7fd8b558ef40 1 -- 172.21.15.119:0/600865323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5584c06683f0 con 0x5584c066f020 2024-04-25T10:18:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.607+0000 7fd8b558ef40 1 -- 172.21.15.119:0/600865323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584c06689b0 con 0x5584c066f020 2024-04-25T10:18:08.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.609+0000 7fd8ab400640 1 -- 172.21.15.119:0/600865323 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8a0059020 con 0x5584c066f020 2024-04-25T10:18:08.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.610+0000 7fd8ab400640 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd894069eb0 0x7fd89406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.610+0000 7fd8ab400640 1 -- 172.21.15.119:0/600865323 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd8a00ccff0 con 0x5584c066f020 2024-04-25T10:18:08.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.611+0000 7fd8b2800640 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd894069eb0 0x7fd89406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.611+0000 7fd8b558ef40 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5584c07ae280 0x5584c07ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.611+0000 7fd8b558ef40 1 -- 172.21.15.119:0/600865323 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:1 17.32 17:4def72d4:::10000000003.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) v9 -- 0x5584c07aebd0 con 0x5584c07ae280 2024-04-25T10:18:08.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.611+0000 7fd8b2800640 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd894069eb0 0x7fd89406c370 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4004170 tx=0x7fd8a4000bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.611+0000 7fd8b3200640 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5584c07ae280 0x5584c07ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.613+0000 7fd8b3200640 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5584c07ae280 0x5584c07ae690 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.613+0000 7fd8b3200640 1 -- 172.21.15.119:0/600865323 <== osd.7 v2:172.21.15.132:6821/2727385616 1 ==== osd_op_reply(1 10000000003.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd8ac01c040 con 0x5584c07ae280 2024-04-25T10:18:08.618 INFO:teuthology.orchestra.run.smithi119.stderr:error getting xattr cephfs_data/10000000003.00000000/scrub_tag: (61) No data available 2024-04-25T10:18:08.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.613+0000 7fd8b558ef40 1 -- 172.21.15.119:0/600865323 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5584c07ae280 msgr2=0x5584c07ae690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.613+0000 7fd8b558ef40 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5584c07ae280 0x5584c07ae690 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.613+0000 7fd8b558ef40 1 -- 172.21.15.119:0/600865323 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd894069eb0 msgr2=0x7fd89406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.613+0000 7fd8b558ef40 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd894069eb0 0x7fd89406c370 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4004170 tx=0x7fd8a4000bc0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.613+0000 7fd8b558ef40 1 -- 172.21.15.119:0/600865323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584c066f020 msgr2=0x5584c070e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.613+0000 7fd8b558ef40 1 --2- 172.21.15.119:0/600865323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584c066f020 0x5584c070e2c0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00259e0 tx=0x7fd8a0003330 comp rx=0 tx=0).stop 2024-04-25T10:18:08.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.614+0000 7fd8b3200640 1 -- 172.21.15.119:0/600865323 reap_dead start 2024-04-25T10:18:08.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.614+0000 7fd8b558ef40 1 -- 172.21.15.119:0/600865323 shutdown_connections 2024-04-25T10:18:08.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.614+0000 7fd8b558ef40 1 -- 172.21.15.119:0/600865323 >> 172.21.15.119:0/600865323 conn(0x5584c0643980 msgr2=0x5584c070ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.614+0000 7fd8b558ef40 1 -- 172.21.15.119:0/600865323 shutdown_connections 2024-04-25T10:18:08.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.614+0000 7fd8b558ef40 1 -- 172.21.15.119:0/600865323 wait complete. 2024-04-25T10:18:08.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:18:08.622 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T10:18:08.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.719+0000 7f5f21db3f40 1 Processor -- start 2024-04-25T10:18:08.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.719+0000 7f5f21db3f40 1 -- start start 2024-04-25T10:18:08.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.720+0000 7f5f21db3f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d94aab220 0x555d94aaa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.720+0000 7f5f21db3f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d94af6080 0x555d94b76a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.720+0000 7f5f21db3f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555d94b82c70 0x555d94b76fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.720+0000 7f5f21db3f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d94ade2f0 con 0x555d94af6080 2024-04-25T10:18:08.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.720+0000 7f5f21db3f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d94a52dd0 con 0x555d94aab220 2024-04-25T10:18:08.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.720+0000 7f5f21db3f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d94aab960 con 0x555d94b82c70 2024-04-25T10:18:08.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.721+0000 7f5f1e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d94af6080 0x555d94b76a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.721+0000 7f5f1e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d94af6080 0x555d94b76a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44026/0 (socket says 172.21.15.119:44026) 2024-04-25T10:18:08.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.721+0000 7f5f1e600640 1 -- 172.21.15.119:0/2983218088 learned_addr learned my addr 172.21.15.119:0/2983218088 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:08.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.721+0000 7f5f1fa00640 1 --2- 172.21.15.119:0/2983218088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555d94b82c70 0x555d94b76fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.721+0000 7f5f1f000640 1 --2- 172.21.15.119:0/2983218088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d94aab220 0x555d94aaa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.721+0000 7f5f1e600640 1 -- 172.21.15.119:0/2983218088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555d94b82c70 msgr2=0x555d94b76fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.721+0000 7f5f1e600640 1 --2- 172.21.15.119:0/2983218088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555d94b82c70 0x555d94b76fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.721+0000 7f5f1e600640 1 -- 172.21.15.119:0/2983218088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d94aab220 msgr2=0x555d94aaa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.721+0000 7f5f1e600640 1 --2- 172.21.15.119:0/2983218088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d94aab220 0x555d94aaa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.721+0000 7f5f1e600640 1 -- 172.21.15.119:0/2983218088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d94b77500 con 0x555d94af6080 2024-04-25T10:18:08.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.721+0000 7f5f1e600640 1 --2- 172.21.15.119:0/2983218088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d94af6080 0x555d94b76a80 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c057040 tx=0x7f5f0c04e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e99b4413fd6715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.722+0000 7f5f1d200640 1 -- 172.21.15.119:0/2983218088 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c059040 con 0x555d94af6080 2024-04-25T10:18:08.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.722+0000 7f5f1d200640 1 -- 172.21.15.119:0/2983218088 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c0547f0 con 0x555d94af6080 2024-04-25T10:18:08.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.722+0000 7f5f1d200640 1 -- 172.21.15.119:0/2983218088 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c054bc0 con 0x555d94af6080 2024-04-25T10:18:08.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.722+0000 7f5f21db3f40 1 -- 172.21.15.119:0/2983218088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d94af6080 msgr2=0x555d94b76a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.722+0000 7f5f21db3f40 1 --2- 172.21.15.119:0/2983218088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d94af6080 0x555d94b76a80 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c057040 tx=0x7f5f0c04e000 comp rx=0 tx=0).stop 2024-04-25T10:18:08.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.722+0000 7f5f21db3f40 1 -- 172.21.15.119:0/2983218088 shutdown_connections 2024-04-25T10:18:08.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.722+0000 7f5f21db3f40 1 --2- 172.21.15.119:0/2983218088 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555d94b82c70 0x555d94b76fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.722+0000 7f5f21db3f40 1 --2- 172.21.15.119:0/2983218088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d94af6080 0x555d94b76a80 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.722+0000 7f5f21db3f40 1 --2- 172.21.15.119:0/2983218088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d94aab220 0x555d94aaa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.722+0000 7f5f21db3f40 1 -- 172.21.15.119:0/2983218088 >> 172.21.15.119:0/2983218088 conn(0x555d94af9070 msgr2=0x555d94af5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.722+0000 7f5f21db3f40 1 -- 172.21.15.119:0/2983218088 shutdown_connections 2024-04-25T10:18:08.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.723+0000 7f5f21db3f40 1 -- 172.21.15.119:0/2983218088 wait complete. 2024-04-25T10:18:08.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.723+0000 7f5f21db3f40 1 Processor -- start 2024-04-25T10:18:08.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.723+0000 7f5f21db3f40 1 -- start start 2024-04-25T10:18:08.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.724+0000 7f5f21db3f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d94aab220 0x555d94b72780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.724+0000 7f5f21db3f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555d94af6080 0x555d94b76850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.724+0000 7f5f21db3f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d94b82c70 0x555d94b76d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.724+0000 7f5f21db3f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d94b83f50 con 0x555d94aab220 2024-04-25T10:18:08.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.724+0000 7f5f21db3f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d94b841a0 con 0x555d94b82c70 2024-04-25T10:18:08.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.724+0000 7f5f21db3f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d94b83c70 con 0x555d94af6080 2024-04-25T10:18:08.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.725+0000 7f5f1f000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d94aab220 0x555d94b72780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.725+0000 7f5f1f000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d94aab220 0x555d94b72780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44038/0 (socket says 172.21.15.119:44038) 2024-04-25T10:18:08.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.725+0000 7f5f1f000640 1 -- 172.21.15.119:0/487932251 learned_addr learned my addr 172.21.15.119:0/487932251 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:08.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.725+0000 7f5f1e600640 1 --2- 172.21.15.119:0/487932251 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555d94af6080 0x555d94b76850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.725+0000 7f5f1fa00640 1 --2- 172.21.15.119:0/487932251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d94b82c70 0x555d94b76d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.725+0000 7f5f1f000640 1 -- 172.21.15.119:0/487932251 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555d94af6080 msgr2=0x555d94b76850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.725+0000 7f5f1f000640 1 --2- 172.21.15.119:0/487932251 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x555d94af6080 0x555d94b76850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.725+0000 7f5f1f000640 1 -- 172.21.15.119:0/487932251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d94b82c70 msgr2=0x555d94b76d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.725+0000 7f5f1f000640 1 --2- 172.21.15.119:0/487932251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d94b82c70 0x555d94b76d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.725+0000 7f5f1f000640 1 -- 172.21.15.119:0/487932251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d94b72ef0 con 0x555d94aab220 2024-04-25T10:18:08.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.726+0000 7f5f1f000640 1 --2- 172.21.15.119:0/487932251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d94aab220 0x555d94b72780 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f5f10008430 tx=0x7f5f10008520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.726+0000 7f5f17400640 1 -- 172.21.15.119:0/487932251 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10015660 con 0x555d94aab220 2024-04-25T10:18:08.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.726+0000 7f5f17400640 1 -- 172.21.15.119:0/487932251 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1001a070 con 0x555d94aab220 2024-04-25T10:18:08.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.726+0000 7f5f17400640 1 -- 172.21.15.119:0/487932251 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10015a20 con 0x555d94aab220 2024-04-25T10:18:08.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.726+0000 7f5f21db3f40 1 -- 172.21.15.119:0/487932251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555d94b731d0 con 0x555d94aab220 2024-04-25T10:18:08.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.726+0000 7f5f21db3f40 1 -- 172.21.15.119:0/487932251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d94b73790 con 0x555d94aab220 2024-04-25T10:18:08.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.728+0000 7f5f17400640 1 -- 172.21.15.119:0/487932251 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f10013030 con 0x555d94aab220 2024-04-25T10:18:08.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.729+0000 7f5f17400640 1 --2- 172.21.15.119:0/487932251 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5f00069ee0 0x7f5f0006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.729+0000 7f5f17400640 1 -- 172.21.15.119:0/487932251 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5f100891d0 con 0x555d94aab220 2024-04-25T10:18:08.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.729+0000 7f5f1e600640 1 --2- 172.21.15.119:0/487932251 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5f00069ee0 0x7f5f0006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.729+0000 7f5f21db3f40 1 --2- 172.21.15.119:0/487932251 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x555d94b7a770 0x555d94b78eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.729+0000 7f5f21db3f40 1 -- 172.21.15.119:0/487932251 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:1 17.12 17:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) v9 -- 0x555d94b793f0 con 0x555d94b7a770 2024-04-25T10:18:08.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.730+0000 7f5f1fa00640 1 --2- 172.21.15.119:0/487932251 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x555d94b7a770 0x555d94b78eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.730+0000 7f5f1e600640 1 --2- 172.21.15.119:0/487932251 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5f00069ee0 0x7f5f0006c3a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c063000 tx=0x7f5f0c056040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.730+0000 7f5f1fa00640 1 --2- 172.21.15.119:0/487932251 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x555d94b7a770 0x555d94b78eb0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.731+0000 7f5f1fa00640 1 -- 172.21.15.119:0/487932251 <== osd.3 v2:172.21.15.119:6816/1142971786 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f5f18001620 con 0x555d94b7a770 2024-04-25T10:18:08.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.731+0000 7f5f21db3f40 1 -- 172.21.15.119:0/487932251 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x555d94b7a770 msgr2=0x555d94b78eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.731+0000 7f5f21db3f40 1 --2- 172.21.15.119:0/487932251 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x555d94b7a770 0x555d94b78eb0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.731+0000 7f5f21db3f40 1 -- 172.21.15.119:0/487932251 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5f00069ee0 msgr2=0x7f5f0006c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.731+0000 7f5f21db3f40 1 --2- 172.21.15.119:0/487932251 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f5f00069ee0 0x7f5f0006c3a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c063000 tx=0x7f5f0c056040 comp rx=0 tx=0).stop 2024-04-25T10:18:08.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.731+0000 7f5f21db3f40 1 -- 172.21.15.119:0/487932251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d94aab220 msgr2=0x555d94b72780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.731+0000 7f5f21db3f40 1 --2- 172.21.15.119:0/487932251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d94aab220 0x555d94b72780 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f5f10008430 tx=0x7f5f10008520 comp rx=0 tx=0).stop 2024-04-25T10:18:08.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.731+0000 7f5f1fa00640 1 -- 172.21.15.119:0/487932251 reap_dead start 2024-04-25T10:18:08.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.731+0000 7f5f21db3f40 1 -- 172.21.15.119:0/487932251 shutdown_connections 2024-04-25T10:18:08.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.731+0000 7f5f21db3f40 1 -- 172.21.15.119:0/487932251 >> 172.21.15.119:0/487932251 conn(0x555d94af9070 msgr2=0x555d94b811d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.731+0000 7f5f21db3f40 1 -- 172.21.15.119:0/487932251 shutdown_connections 2024-04-25T10:18:08.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.731+0000 7f5f21db3f40 1 -- 172.21.15.119:0/487932251 wait complete. 2024-04-25T10:18:08.739 INFO:teuthology.orchestra.run.smithi119.stdout:mytag123 2024-04-25T10:18:08.739 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-25T10:18:08.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.839+0000 7fabe541af40 1 Processor -- start 2024-04-25T10:18:08.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.839+0000 7fabe541af40 1 -- start start 2024-04-25T10:18:08.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.840+0000 7fabe541af40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0dcede220 0x55d0dcfb6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.840+0000 7fabe541af40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0dcf29080 0x55d0dcfa99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.840+0000 7fabe541af40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d0dcfb5c70 0x55d0dcf28340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.840+0000 7fabe541af40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0dcf112f0 con 0x55d0dcf29080 2024-04-25T10:18:08.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.840+0000 7fabe541af40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0dce85dd0 con 0x55d0dcede220 2024-04-25T10:18:08.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.840+0000 7fabe541af40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0dcede960 con 0x55d0dcfb5c70 2024-04-25T10:18:08.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.840+0000 7fabdbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0dcf29080 0x55d0dcfa99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.840+0000 7fabdbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0dcf29080 0x55d0dcfa99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44056/0 (socket says 172.21.15.119:44056) 2024-04-25T10:18:08.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.840+0000 7fabdbe00640 1 -- 172.21.15.119:0/71202453 learned_addr learned my addr 172.21.15.119:0/71202453 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:08.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.841+0000 7fabdbe00640 1 -- 172.21.15.119:0/71202453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d0dcfb5c70 msgr2=0x55d0dcf28340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.841+0000 7fabe2400640 1 --2- 172.21.15.119:0/71202453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d0dcfb5c70 0x55d0dcf28340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.841+0000 7fabe1a00640 1 --2- 172.21.15.119:0/71202453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0dcede220 0x55d0dcfb6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.841+0000 7fabdbe00640 1 --2- 172.21.15.119:0/71202453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d0dcfb5c70 0x55d0dcf28340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.841+0000 7fabdbe00640 1 -- 172.21.15.119:0/71202453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0dcede220 msgr2=0x55d0dcfb6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.841+0000 7fabdbe00640 1 --2- 172.21.15.119:0/71202453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0dcede220 0x55d0dcfb6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.841+0000 7fabdbe00640 1 -- 172.21.15.119:0/71202453 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0dcfaa410 con 0x55d0dcf29080 2024-04-25T10:18:08.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.841+0000 7fabdbe00640 1 --2- 172.21.15.119:0/71202453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0dcf29080 0x55d0dcfa99b0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fabcc057040 tx=0x7fabcc04e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e398be2cf3951ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.841+0000 7fabdb400640 1 -- 172.21.15.119:0/71202453 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc059040 con 0x55d0dcf29080 2024-04-25T10:18:08.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.842+0000 7fabdb400640 1 -- 172.21.15.119:0/71202453 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc0548f0 con 0x55d0dcf29080 2024-04-25T10:18:08.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.842+0000 7fabdb400640 1 -- 172.21.15.119:0/71202453 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc054ce0 con 0x55d0dcf29080 2024-04-25T10:18:08.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.842+0000 7fabe541af40 1 -- 172.21.15.119:0/71202453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0dcf29080 msgr2=0x55d0dcfa99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.842+0000 7fabe541af40 1 --2- 172.21.15.119:0/71202453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0dcf29080 0x55d0dcfa99b0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fabcc057040 tx=0x7fabcc04e000 comp rx=0 tx=0).stop 2024-04-25T10:18:08.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.842+0000 7fabe541af40 1 -- 172.21.15.119:0/71202453 shutdown_connections 2024-04-25T10:18:08.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.842+0000 7fabe541af40 1 --2- 172.21.15.119:0/71202453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d0dcfb5c70 0x55d0dcf28340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.842+0000 7fabe541af40 1 --2- 172.21.15.119:0/71202453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0dcf29080 0x55d0dcfa99b0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.842+0000 7fabe541af40 1 --2- 172.21.15.119:0/71202453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0dcede220 0x55d0dcfb6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.842+0000 7fabe541af40 1 -- 172.21.15.119:0/71202453 >> 172.21.15.119:0/71202453 conn(0x55d0dcf2c070 msgr2=0x55d0dcedd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.842+0000 7fabe541af40 1 -- 172.21.15.119:0/71202453 shutdown_connections 2024-04-25T10:18:08.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.842+0000 7fabe541af40 1 -- 172.21.15.119:0/71202453 wait complete. 2024-04-25T10:18:08.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.843+0000 7fabe541af40 1 Processor -- start 2024-04-25T10:18:08.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.843+0000 7fabe541af40 1 -- start start 2024-04-25T10:18:08.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.844+0000 7fabe541af40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d0dcede220 0x55d0dcedd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.844+0000 7fabe541af40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0dcf29080 0x55d0dd050080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.844+0000 7fabe541af40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0dcfb5c70 0x55d0dcf28750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.844+0000 7fabe541af40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0dcfb60f0 con 0x55d0dcf29080 2024-04-25T10:18:08.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.844+0000 7fabe541af40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0dcfb63d0 con 0x55d0dcfb5c70 2024-04-25T10:18:08.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.844+0000 7fabe541af40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0dcfb6a30 con 0x55d0dcede220 2024-04-25T10:18:08.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.844+0000 7fabdbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0dcf29080 0x55d0dd050080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.844+0000 7fabdbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0dcf29080 0x55d0dd050080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44072/0 (socket says 172.21.15.119:44072) 2024-04-25T10:18:08.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.845+0000 7fabdbe00640 1 -- 172.21.15.119:0/1325582649 learned_addr learned my addr 172.21.15.119:0/1325582649 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:08.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.845+0000 7fabe1a00640 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d0dcede220 0x55d0dcedd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.845+0000 7fabe2400640 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0dcfb5c70 0x55d0dcf28750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.845+0000 7fabdbe00640 1 -- 172.21.15.119:0/1325582649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d0dcede220 msgr2=0x55d0dcedd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.845+0000 7fabdbe00640 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55d0dcede220 0x55d0dcedd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.845+0000 7fabdbe00640 1 -- 172.21.15.119:0/1325582649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0dcfb5c70 msgr2=0x55d0dcf28750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.845+0000 7fabdbe00640 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0dcfb5c70 0x55d0dcf28750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.845+0000 7fabdbe00640 1 -- 172.21.15.119:0/1325582649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0dcf28c90 con 0x55d0dcf29080 2024-04-25T10:18:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.845+0000 7fabe2400640 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0dcfb5c70 0x55d0dcf28750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.845+0000 7fabdbe00640 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0dcf29080 0x55d0dd050080 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fabcc045d50 tx=0x7fabcc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.846+0000 7fabda000640 1 -- 172.21.15.119:0/1325582649 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc06e070 con 0x55d0dcf29080 2024-04-25T10:18:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.846+0000 7fabda000640 1 -- 172.21.15.119:0/1325582649 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc05f470 con 0x55d0dcf29080 2024-04-25T10:18:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.846+0000 7fabda000640 1 -- 172.21.15.119:0/1325582649 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc059050 con 0x55d0dcf29080 2024-04-25T10:18:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.846+0000 7fabe541af40 1 -- 172.21.15.119:0/1325582649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d0dcfaed10 con 0x55d0dcf29080 2024-04-25T10:18:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.846+0000 7fabe541af40 1 -- 172.21.15.119:0/1325582649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0dcfaefb0 con 0x55d0dcf29080 2024-04-25T10:18:08.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.848+0000 7fabda000640 1 -- 172.21.15.119:0/1325582649 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabcc05f860 con 0x55d0dcf29080 2024-04-25T10:18:08.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.849+0000 7fabda000640 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fabc0069eb0 0x7fabc006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.849+0000 7fabda000640 1 -- 172.21.15.119:0/1325582649 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fabcc0cd890 con 0x55d0dcf29080 2024-04-25T10:18:08.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.849+0000 7fabe1a00640 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fabc0069eb0 0x7fabc006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.849+0000 7fabe541af40 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55d0dcfafc60 0x55d0dcfa7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.849+0000 7fabe541af40 1 -- 172.21.15.119:0/1325582649 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:1 17.5 17:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) v9 -- 0x55d0dcfa7fa0 con 0x55d0dcfafc60 2024-04-25T10:18:08.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.850+0000 7fabe1a00640 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fabc0069eb0 0x7fabc006c370 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fabd4005380 tx=0x7fabd4005310 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.850+0000 7fabe2400640 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55d0dcfafc60 0x55d0dcfa7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.851+0000 7fabe2400640 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55d0dcfafc60 0x55d0dcfa7380 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.852+0000 7fabe2400640 1 -- 172.21.15.119:0/1325582649 <== osd.6 v2:172.21.15.132:6806/353653608 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7fabdc001790 con 0x55d0dcfafc60 2024-04-25T10:18:08.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.852+0000 7fabe541af40 1 -- 172.21.15.119:0/1325582649 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55d0dcfafc60 msgr2=0x55d0dcfa7380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.852+0000 7fabe541af40 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55d0dcfafc60 0x55d0dcfa7380 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.852+0000 7fabe541af40 1 -- 172.21.15.119:0/1325582649 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fabc0069eb0 msgr2=0x7fabc006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.852+0000 7fabe541af40 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fabc0069eb0 0x7fabc006c370 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fabd4005380 tx=0x7fabd4005310 comp rx=0 tx=0).stop 2024-04-25T10:18:08.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.852+0000 7fabe541af40 1 -- 172.21.15.119:0/1325582649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0dcf29080 msgr2=0x55d0dd050080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.852+0000 7fabe541af40 1 --2- 172.21.15.119:0/1325582649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0dcf29080 0x55d0dd050080 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fabcc045d50 tx=0x7fabcc002fb0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.852+0000 7fabe2400640 1 -- 172.21.15.119:0/1325582649 reap_dead start 2024-04-25T10:18:08.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.852+0000 7fabe541af40 1 -- 172.21.15.119:0/1325582649 shutdown_connections 2024-04-25T10:18:08.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.852+0000 7fabe541af40 1 -- 172.21.15.119:0/1325582649 >> 172.21.15.119:0/1325582649 conn(0x55d0dcf2c070 msgr2=0x55d0dcf0b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.852+0000 7fabe541af40 1 -- 172.21.15.119:0/1325582649 shutdown_connections 2024-04-25T10:18:08.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.852+0000 7fabe541af40 1 -- 172.21.15.119:0/1325582649 wait complete. 2024-04-25T10:18:08.860 INFO:teuthology.orchestra.run.smithi119.stdout:mytag123 2024-04-25T10:18:08.860 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000006.00000000 scrub_tag 2024-04-25T10:18:08.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.944+0000 7f0036423f40 1 Processor -- start 2024-04-25T10:18:08.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.944+0000 7f0036423f40 1 -- start start 2024-04-25T10:18:08.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.945+0000 7f0036423f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55aba01d2220 0x55aba029da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.945+0000 7f0036423f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aba021d080 0x55aba021f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.945+0000 7f0036423f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba02a9c70 0x55aba021f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.945+0000 7f0036423f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba02052f0 con 0x55aba021d080 2024-04-25T10:18:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.945+0000 7f0036423f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba0179dd0 con 0x55aba02a9c70 2024-04-25T10:18:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.945+0000 7f0036423f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aba01d2960 con 0x55aba01d2220 2024-04-25T10:18:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.945+0000 7f0032c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aba021d080 0x55aba021f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.945+0000 7f0032c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aba021d080 0x55aba021f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44086/0 (socket says 172.21.15.119:44086) 2024-04-25T10:18:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.945+0000 7f0032c00640 1 -- 172.21.15.119:0/4118705161 learned_addr learned my addr 172.21.15.119:0/4118705161 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.945+0000 7f0032c00640 1 -- 172.21.15.119:0/4118705161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55aba01d2220 msgr2=0x55aba029da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.945+0000 7f0033600640 1 --2- 172.21.15.119:0/4118705161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55aba01d2220 0x55aba029da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.946+0000 7f0032c00640 1 --2- 172.21.15.119:0/4118705161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55aba01d2220 0x55aba029da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.946+0000 7f0034000640 1 --2- 172.21.15.119:0/4118705161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba02a9c70 0x55aba021f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.946+0000 7f0032c00640 1 -- 172.21.15.119:0/4118705161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba02a9c70 msgr2=0x55aba021f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.946+0000 7f0032c00640 1 --2- 172.21.15.119:0/4118705161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba02a9c70 0x55aba021f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.946+0000 7f0032c00640 1 -- 172.21.15.119:0/4118705161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aba0220610 con 0x55aba021d080 2024-04-25T10:18:08.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.946+0000 7f0033600640 1 --2- 172.21.15.119:0/4118705161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55aba01d2220 0x55aba029da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:08.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.946+0000 7f0032c00640 1 --2- 172.21.15.119:0/4118705161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aba021d080 0x55aba021f310 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f002000b310 tx=0x7f002000b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185929b52a1e4998 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.946+0000 7f0031800640 1 -- 172.21.15.119:0/4118705161 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020008070 con 0x55aba021d080 2024-04-25T10:18:08.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.946+0000 7f0031800640 1 -- 172.21.15.119:0/4118705161 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002000d500 con 0x55aba021d080 2024-04-25T10:18:08.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.946+0000 7f0031800640 1 -- 172.21.15.119:0/4118705161 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002000d8b0 con 0x55aba021d080 2024-04-25T10:18:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.947+0000 7f0036423f40 1 -- 172.21.15.119:0/4118705161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aba021d080 msgr2=0x55aba021f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.947+0000 7f0036423f40 1 --2- 172.21.15.119:0/4118705161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aba021d080 0x55aba021f310 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f002000b310 tx=0x7f002000b7d0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.947+0000 7f0036423f40 1 -- 172.21.15.119:0/4118705161 shutdown_connections 2024-04-25T10:18:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.947+0000 7f0036423f40 1 --2- 172.21.15.119:0/4118705161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba02a9c70 0x55aba021f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.947+0000 7f0036423f40 1 --2- 172.21.15.119:0/4118705161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aba021d080 0x55aba021f310 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.947+0000 7f0036423f40 1 --2- 172.21.15.119:0/4118705161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55aba01d2220 0x55aba029da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.947+0000 7f0036423f40 1 -- 172.21.15.119:0/4118705161 >> 172.21.15.119:0/4118705161 conn(0x55aba0220070 msgr2=0x55aba01d15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.947+0000 7f0036423f40 1 -- 172.21.15.119:0/4118705161 shutdown_connections 2024-04-25T10:18:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.947+0000 7f0036423f40 1 -- 172.21.15.119:0/4118705161 wait complete. 2024-04-25T10:18:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.948+0000 7f0036423f40 1 Processor -- start 2024-04-25T10:18:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.948+0000 7f0036423f40 1 -- start start 2024-04-25T10:18:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.949+0000 7f0036423f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aba01d2220 0x55aba033a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.949+0000 7f0036423f40 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55aba021d080 0x55aba021c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.949+0000 7f0036423f40 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba02a9c70 0x55aba0298ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.949+0000 7f0036423f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba02aa930 con 0x55aba01d2220 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.949+0000 7f0036423f40 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba0338630 con 0x55aba02a9c70 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.949+0000 7f0036423f40 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aba02aac10 con 0x55aba021d080 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.949+0000 7f0033600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aba01d2220 0x55aba033a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.949+0000 7f0033600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aba01d2220 0x55aba033a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44102/0 (socket says 172.21.15.119:44102) 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.949+0000 7f0033600640 1 -- 172.21.15.119:0/2536886679 learned_addr learned my addr 172.21.15.119:0/2536886679 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.949+0000 7f0034000640 1 --2- 172.21.15.119:0/2536886679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba02a9c70 0x55aba0298ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.950+0000 7f0033600640 1 -- 172.21.15.119:0/2536886679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55aba021d080 msgr2=0x55aba021c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.950+0000 7f0033600640 1 --2- 172.21.15.119:0/2536886679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55aba021d080 0x55aba021c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.950+0000 7f0033600640 1 -- 172.21.15.119:0/2536886679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba02a9c70 msgr2=0x55aba0298ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.950+0000 7f0033600640 1 --2- 172.21.15.119:0/2536886679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba02a9c70 0x55aba0298ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.950+0000 7f0033600640 1 -- 172.21.15.119:0/2536886679 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aba033b510 con 0x55aba01d2220 2024-04-25T10:18:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.950+0000 7f0034000640 1 --2- 172.21.15.119:0/2536886679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba02a9c70 0x55aba0298ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.950+0000 7f0033600640 1 --2- 172.21.15.119:0/2536886679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aba01d2220 0x55aba033a2d0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f002400a4d0 tx=0x7f002400a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.950+0000 7f002be00640 1 -- 172.21.15.119:0/2536886679 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024015030 con 0x55aba01d2220 2024-04-25T10:18:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.950+0000 7f002be00640 1 -- 172.21.15.119:0/2536886679 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0024011670 con 0x55aba01d2220 2024-04-25T10:18:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.950+0000 7f002be00640 1 -- 172.21.15.119:0/2536886679 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024011a40 con 0x55aba01d2220 2024-04-25T10:18:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.950+0000 7f0036423f40 1 -- 172.21.15.119:0/2536886679 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55aba033b7f0 con 0x55aba01d2220 2024-04-25T10:18:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.950+0000 7f0036423f40 1 -- 172.21.15.119:0/2536886679 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aba0339db0 con 0x55aba01d2220 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.953+0000 7f002be00640 1 -- 172.21.15.119:0/2536886679 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0024018070 con 0x55aba01d2220 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.953+0000 7f002be00640 1 --2- 172.21.15.119:0/2536886679 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0014069eb0 0x7f001406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.954+0000 7f002be00640 1 -- 172.21.15.119:0/2536886679 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f00240892e0 con 0x55aba01d2220 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.954+0000 7f0032c00640 1 --2- 172.21.15.119:0/2536886679 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0014069eb0 0x7f001406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.954+0000 7f0036423f40 1 --2- 172.21.15.119:0/2536886679 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55aba021ca00 0x55aba03387e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.954+0000 7f0036423f40 1 -- 172.21.15.119:0/2536886679 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:1 17.25 17:a5feb6f3:::10000000006.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) v9 -- 0x55aba021ee40 con 0x55aba021ca00 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.954+0000 7f0034000640 1 --2- 172.21.15.119:0/2536886679 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55aba021ca00 0x55aba03387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.955+0000 7f0032c00640 1 --2- 172.21.15.119:0/2536886679 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0014069eb0 0x7f001406c370 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x55aba0221300 tx=0x7f0020005000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.955+0000 7f0034000640 1 --2- 172.21.15.119:0/2536886679 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55aba021ca00 0x55aba03387e0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.955+0000 7f0034000640 1 -- 172.21.15.119:0/2536886679 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== osd_op_reply(1 10000000006.00000000 [getxattr (12) out=12b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f002c050050 con 0x55aba021ca00 2024-04-25T10:18:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.955+0000 7f0036423f40 1 -- 172.21.15.119:0/2536886679 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55aba021ca00 msgr2=0x55aba03387e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.955+0000 7f0036423f40 1 --2- 172.21.15.119:0/2536886679 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55aba021ca00 0x55aba03387e0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.955+0000 7f0036423f40 1 -- 172.21.15.119:0/2536886679 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0014069eb0 msgr2=0x7f001406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.955+0000 7f0036423f40 1 --2- 172.21.15.119:0/2536886679 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0014069eb0 0x7f001406c370 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x55aba0221300 tx=0x7f0020005000 comp rx=0 tx=0).stop 2024-04-25T10:18:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.955+0000 7f0036423f40 1 -- 172.21.15.119:0/2536886679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aba01d2220 msgr2=0x55aba033a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.955+0000 7f0036423f40 1 --2- 172.21.15.119:0/2536886679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aba01d2220 0x55aba033a2d0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f002400a4d0 tx=0x7f002400a990 comp rx=0 tx=0).stop 2024-04-25T10:18:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.955+0000 7f0034000640 1 -- 172.21.15.119:0/2536886679 reap_dead start 2024-04-25T10:18:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.956+0000 7f0036423f40 1 -- 172.21.15.119:0/2536886679 shutdown_connections 2024-04-25T10:18:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.956+0000 7f0036423f40 1 -- 172.21.15.119:0/2536886679 >> 172.21.15.119:0/2536886679 conn(0x55aba0220070 msgr2=0x55aba02a7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.956+0000 7f0036423f40 1 -- 172.21.15.119:0/2536886679 shutdown_connections 2024-04-25T10:18:08.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:08.956+0000 7f0036423f40 1 -- 172.21.15.119:0/2536886679 wait complete. 2024-04-25T10:18:08.963 INFO:teuthology.orchestra.run.smithi119.stdout:mytag123 2024-04-25T10:18:08.964 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:18:08.964 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T10:18:09.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.150+0000 7f280be00640 1 Processor -- start 2024-04-25T10:18:09.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.150+0000 7f280be00640 1 -- start start 2024-04-25T10:18:09.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.151+0000 7f280be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c15d980 0x7f280c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.151+0000 7f280be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f280c0b2a00 0x7f280c0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.151+0000 7f280be00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280c0b6ac0 con 0x7f280c0b2440 2024-04-25T10:18:09.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.151+0000 7f280be00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280c0cefe0 con 0x7f280c0b2a00 2024-04-25T10:18:09.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.151+0000 7f280be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0b70d0 con 0x7f280c15d980 2024-04-25T10:18:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.152+0000 7f2802000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f280c0b2a00 0x7f280c0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.152+0000 7f280aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c15d980 0x7f280c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.152+0000 7f2802000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f280c0b2a00 0x7f280c0ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:36028/0 (socket says 172.21.15.119:36028) 2024-04-25T10:18:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.152+0000 7f2802000640 1 -- 172.21.15.119:0/3330119273 learned_addr learned my addr 172.21.15.119:0/3330119273 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.153+0000 7f2802000640 1 -- 172.21.15.119:0/3330119273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c15d980 msgr2=0x7f280c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.153+0000 7f2802000640 1 --2- 172.21.15.119:0/3330119273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c15d980 0x7f280c0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.153+0000 7f2802000640 1 -- 172.21.15.119:0/3330119273 >> v1:172.21.15.132:6789/0 conn(0x7f280c0b2440 legacy=0x7f280c0b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.153+0000 7f2802000640 1 -- 172.21.15.119:0/3330119273 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c0b94f0 con 0x7f280c0b2a00 2024-04-25T10:18:09.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.154+0000 7f2802000640 1 --2- 172.21.15.119:0/3330119273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f280c0b2a00 0x7f280c0ceaa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009870 tx=0x7f27f402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc13f4ec541d4419 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.155+0000 7f280a000640 1 -- 172.21.15.119:0/3330119273 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4037070 con 0x7f280c0b2a00 2024-04-25T10:18:09.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.155+0000 7f280a000640 1 -- 172.21.15.119:0/3330119273 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f4032440 con 0x7f280c0b2a00 2024-04-25T10:18:09.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.155+0000 7f280a000640 1 -- 172.21.15.119:0/3330119273 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4032810 con 0x7f280c0b2a00 2024-04-25T10:18:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.156+0000 7f280be00640 1 -- 172.21.15.119:0/3330119273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f280c0b2a00 msgr2=0x7f280c0ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.156+0000 7f280be00640 1 --2- 172.21.15.119:0/3330119273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f280c0b2a00 0x7f280c0ceaa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009870 tx=0x7f27f402a270 comp rx=0 tx=0).stop 2024-04-25T10:18:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.156+0000 7f280be00640 1 -- 172.21.15.119:0/3330119273 shutdown_connections 2024-04-25T10:18:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.156+0000 7f280be00640 1 --2- 172.21.15.119:0/3330119273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f280c0b2a00 0x7f280c0ceaa0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.156+0000 7f280be00640 1 --2- 172.21.15.119:0/3330119273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c15d980 0x7f280c0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.156+0000 7f280be00640 1 -- 172.21.15.119:0/3330119273 >> 172.21.15.119:0/3330119273 conn(0x7f280c0726c0 msgr2=0x7f280c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.156+0000 7f280be00640 1 -- 172.21.15.119:0/3330119273 shutdown_connections 2024-04-25T10:18:09.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.156+0000 7f280be00640 1 -- 172.21.15.119:0/3330119273 wait complete. 2024-04-25T10:18:09.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.158+0000 7f280be00640 1 Processor -- start 2024-04-25T10:18:09.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.158+0000 7f280be00640 1 -- start start 2024-04-25T10:18:09.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.159+0000 7f280be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0b2440 0x7f280c0cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.159+0000 7f280be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f280c0b2a00 0x7f280c0ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.159+0000 7f280be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c15d980 0x7f280c151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.159+0000 7f280be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c15dd60 con 0x7f280c15d980 2024-04-25T10:18:09.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.159+0000 7f280be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0b97d0 con 0x7f280c0b2a00 2024-04-25T10:18:09.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.159+0000 7f280be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0b9110 con 0x7f280c0b2440 2024-04-25T10:18:09.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.159+0000 7f280b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c15d980 0x7f280c151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.159+0000 7f280b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c15d980 0x7f280c151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44114/0 (socket says 172.21.15.119:44114) 2024-04-25T10:18:09.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.159+0000 7f280b400640 1 -- 172.21.15.119:0/4056542899 learned_addr learned my addr 172.21.15.119:0/4056542899 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:09.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.160+0000 7f2802000640 1 --2- 172.21.15.119:0/4056542899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f280c0b2a00 0x7f280c0ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.160+0000 7f280aa00640 1 --2- 172.21.15.119:0/4056542899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0b2440 0x7f280c0cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.160+0000 7f280b400640 1 -- 172.21.15.119:0/4056542899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0b2440 msgr2=0x7f280c0cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.160+0000 7f280b400640 1 --2- 172.21.15.119:0/4056542899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0b2440 0x7f280c0cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.160+0000 7f280b400640 1 -- 172.21.15.119:0/4056542899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f280c0b2a00 msgr2=0x7f280c0ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.160+0000 7f280b400640 1 --2- 172.21.15.119:0/4056542899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f280c0b2a00 0x7f280c0ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.160+0000 7f280b400640 1 -- 172.21.15.119:0/4056542899 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c0cc370 con 0x7f280c15d980 2024-04-25T10:18:09.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.160+0000 7f280b400640 1 --2- 172.21.15.119:0/4056542899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c15d980 0x7f280c151900 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f280400c6e0 tx=0x7f280400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.160+0000 7f2803400640 1 -- 172.21.15.119:0/4056542899 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804003b70 con 0x7f280c15d980 2024-04-25T10:18:09.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.160+0000 7f2803400640 1 -- 172.21.15.119:0/4056542899 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2804003d30 con 0x7f280c15d980 2024-04-25T10:18:09.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.160+0000 7f2803400640 1 -- 172.21.15.119:0/4056542899 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804005310 con 0x7f280c15d980 2024-04-25T10:18:09.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.160+0000 7f280be00640 1 -- 172.21.15.119:0/4056542899 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280c0cb2d0 con 0x7f280c15d980 2024-04-25T10:18:09.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.161+0000 7f280be00640 1 -- 172.21.15.119:0/4056542899 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c0cbb70 con 0x7f280c15d980 2024-04-25T10:18:09.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.163+0000 7f2803400640 1 -- 172.21.15.119:0/4056542899 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28040054b0 con 0x7f280c15d980 2024-04-25T10:18:09.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.164+0000 7f2803400640 1 --2- 172.21.15.119:0/4056542899 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f27e806b4d0 0x7f27e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.164+0000 7f2803400640 1 -- 172.21.15.119:0/4056542899 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f280408a0e0 con 0x7f280c15d980 2024-04-25T10:18:09.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.164+0000 7f280aa00640 1 --2- 172.21.15.119:0/4056542899 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f27e806b4d0 0x7f27e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.165+0000 7f2814574740 1 Processor -- start 2024-04-25T10:18:09.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.165+0000 7f280aa00640 1 --2- 172.21.15.119:0/4056542899 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f27e806b4d0 0x7f27e806d990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009ad0 tx=0x7f27fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.165+0000 7f2814574740 1 -- start start 2024-04-25T10:18:09.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.166+0000 7f2814574740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd9691d70 0x556cd9693ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.166+0000 7f2814574740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96ce440 0x556cd9692150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.166+0000 7f2814574740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96bc830 0x556cd96bccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.166+0000 7f2814574740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd95f5c70 con 0x556cd96ce440 2024-04-25T10:18:09.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.166+0000 7f2814574740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd959ecf0 con 0x556cd9691d70 2024-04-25T10:18:09.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.166+0000 7f2814574740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556cd95f1d10 con 0x556cd96bc830 2024-04-25T10:18:09.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.166+0000 7f2802000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96ce440 0x556cd9692150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.166+0000 7f2802000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96ce440 0x556cd9692150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44116/0 (socket says 172.21.15.119:44116) 2024-04-25T10:18:09.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.166+0000 7f2802000640 1 -- 172.21.15.119:0/1216872027 learned_addr learned my addr 172.21.15.119:0/1216872027 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:09.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.166+0000 7f2802000640 1 -- 172.21.15.119:0/1216872027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96bc830 msgr2=0x556cd96bccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.166+0000 7f280b400640 1 --2- 172.21.15.119:0/1216872027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96bc830 0x556cd96bccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.166+0000 7f280aa00640 1 --2- 172.21.15.119:0/1216872027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd9691d70 0x556cd9693ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.166+0000 7f2802000640 1 --2- 172.21.15.119:0/1216872027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96bc830 0x556cd96bccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.166+0000 7f2802000640 1 -- 172.21.15.119:0/1216872027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd9691d70 msgr2=0x556cd9693ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.167+0000 7f2802000640 1 --2- 172.21.15.119:0/1216872027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd9691d70 0x556cd9693ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.167+0000 7f2802000640 1 -- 172.21.15.119:0/1216872027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cd93d47d0 con 0x556cd96ce440 2024-04-25T10:18:09.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.167+0000 7f2802000640 1 --2- 172.21.15.119:0/1216872027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96ce440 0x556cd9692150 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f27f402a2b0 tx=0x7f27f402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6062a5a657493d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.167+0000 7f27fb400640 1 -- 172.21.15.119:0/1216872027 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4037040 con 0x556cd96ce440 2024-04-25T10:18:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.167+0000 7f27fb400640 1 -- 172.21.15.119:0/1216872027 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f4032740 con 0x556cd96ce440 2024-04-25T10:18:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.167+0000 7f27fb400640 1 -- 172.21.15.119:0/1216872027 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4032af0 con 0x556cd96ce440 2024-04-25T10:18:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.168+0000 7f2814574740 1 -- 172.21.15.119:0/1216872027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96ce440 msgr2=0x556cd9692150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.168+0000 7f2814574740 1 --2- 172.21.15.119:0/1216872027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96ce440 0x556cd9692150 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f27f402a2b0 tx=0x7f27f402c000 comp rx=0 tx=0).stop 2024-04-25T10:18:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.168+0000 7f2814574740 1 -- 172.21.15.119:0/1216872027 shutdown_connections 2024-04-25T10:18:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.168+0000 7f2814574740 1 --2- 172.21.15.119:0/1216872027 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96bc830 0x556cd96bccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.168+0000 7f2814574740 1 --2- 172.21.15.119:0/1216872027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96ce440 0x556cd9692150 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.168+0000 7f2814574740 1 --2- 172.21.15.119:0/1216872027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd9691d70 0x556cd9693ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.168+0000 7f2814574740 1 -- 172.21.15.119:0/1216872027 >> 172.21.15.119:0/1216872027 conn(0x556cd9659a30 msgr2=0x556cd96d7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.168+0000 7f2814574740 1 -- 172.21.15.119:0/1216872027 shutdown_connections 2024-04-25T10:18:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.168+0000 7f2814574740 1 -- 172.21.15.119:0/1216872027 wait complete. 2024-04-25T10:18:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.169+0000 7f2814574740 1 -- start start 2024-04-25T10:18:09.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.169+0000 7f2814574740 1 Processor -- start 2024-04-25T10:18:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.170+0000 7f2814574740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd9691d70 0x556cd96be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.170+0000 7f2814574740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96bc830 0x556cd96bb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.170+0000 7f2814574740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96ce440 0x556cd96bbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.170+0000 7f2814574740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd95943a0 con 0x556cd96bc830 2024-04-25T10:18:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.170+0000 7f2814574740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd96ba380 con 0x556cd9691d70 2024-04-25T10:18:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.170+0000 7f2814574740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556cd96ea8c0 con 0x556cd96ce440 2024-04-25T10:18:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.170+0000 7f2814574740 10 client.0 _ll_register_callbacks cb 0x556cd95b1b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.170+0000 7f2802000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96bc830 0x556cd96bb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.170+0000 7f2802000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96bc830 0x556cd96bb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44132/0 (socket says 172.21.15.119:44132) 2024-04-25T10:18:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.170+0000 7f2802000640 1 -- 172.21.15.119:0/1497608587 learned_addr learned my addr 172.21.15.119:0/1497608587 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.171+0000 7f280aa00640 1 --2- 172.21.15.119:0/1497608587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd9691d70 0x556cd96be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.171+0000 7f280b400640 1 --2- 172.21.15.119:0/1497608587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96ce440 0x556cd96bbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.171+0000 7f2802000640 1 -- 172.21.15.119:0/1497608587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96ce440 msgr2=0x556cd96bbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.171+0000 7f2802000640 1 --2- 172.21.15.119:0/1497608587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96ce440 0x556cd96bbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.171+0000 7f2802000640 1 -- 172.21.15.119:0/1497608587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd9691d70 msgr2=0x556cd96be9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.171+0000 7f2802000640 1 --2- 172.21.15.119:0/1497608587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd9691d70 0x556cd96be9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.171+0000 7f2802000640 1 -- 172.21.15.119:0/1497608587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556cd930a3f0 con 0x556cd96bc830 2024-04-25T10:18:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.171+0000 7f280aa00640 1 --2- 172.21.15.119:0/1497608587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd9691d70 0x556cd96be9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.171+0000 7f2802000640 1 --2- 172.21.15.119:0/1497608587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96bc830 0x556cd96bb7b0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f27f402a860 tx=0x7f27f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a52d388fe32aa71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.171+0000 7f27efe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:18:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.171+0000 7f27efe00640 1 -- 172.21.15.119:0/1497608587 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f404a070 con 0x556cd96bc830 2024-04-25T10:18:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.171+0000 7f27efe00640 1 -- 172.21.15.119:0/1497608587 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f27f4032b00 con 0x556cd96bc830 2024-04-25T10:18:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.171+0000 7f2814574740 1 -- 172.21.15.119:0/1497608587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27f4032740 con 0x556cd96bc830 2024-04-25T10:18:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.172+0000 7f27efe00640 1 -- 172.21.15.119:0/1497608587 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27f4052050 con 0x556cd96bc830 2024-04-25T10:18:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.172+0000 7f2814574740 10 client.9096 fetch_fsmap learned FSMap version 105 2024-04-25T10:18:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.172+0000 7f2814574740 1 -- 172.21.15.119:0/1497608587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x556cd92ca0f0 con 0x556cd96bc830 2024-04-25T10:18:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.172+0000 7f27efe00640 1 -- 172.21.15.119:0/1497608587 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f4032740 con 0x556cd96bc830 2024-04-25T10:18:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.172+0000 7f27efe00640 1 -- 172.21.15.119:0/1497608587 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f27f4055020 con 0x556cd96bc830 2024-04-25T10:18:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.172+0000 7f2814574740 10 client.9096 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:18:09.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.172+0000 7f2814574740 10 client.9096 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:18:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.172+0000 7f2814574740 20 client.9096 populate_metadata read hostname 'smithi119' 2024-04-25T10:18:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.172+0000 7f2814574740 1 --2- 172.21.15.119:0/1497608587 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x556cd96bccb0 0x556cd96bd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.172+0000 7f2814574740 4 client.9096 mds_command: new command op to 8943 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.172+0000 7f2814574740 1 -- 172.21.15.119:0/1497608587 --> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556cd96031b0 con 0x556cd96bccb0 2024-04-25T10:18:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.172+0000 7f280aa00640 1 --2- 172.21.15.119:0/1497608587 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x556cd96bccb0 0x556cd96bd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.173+0000 7f280aa00640 1 --2- 172.21.15.119:0/1497608587 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x556cd96bccb0 0x556cd96bd0f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f87ff78350ae454 server_cookie=ae9ebaa733682994 in_seq=0 out_seq=0 2024-04-25T10:18:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.173+0000 7f27efe00640 10 client.9096 ms_handle_connect on v2:172.21.15.119:6836/2864575636 2024-04-25T10:18:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f27efe00640 1 -- 172.21.15.119:0/1497608587 <== mds.0 v2:172.21.15.119:6836/2864575636 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556cd96031b0 con 0x556cd96bccb0 2024-04-25T10:18:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f27efe00640 10 client.9096 handle_command_reply: tid=0 2024-04-25T10:18:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f2814574740 1 client.9096 shutdown 2024-04-25T10:18:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f2814574740 1 -- 172.21.15.119:0/1497608587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96bc830 msgr2=0x556cd96bb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f2814574740 1 --2- 172.21.15.119:0/1497608587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96bc830 0x556cd96bb7b0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f27f402a860 tx=0x7f27f4004730 comp rx=0 tx=0).stop 2024-04-25T10:18:09.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f2814574740 1 -- 172.21.15.119:0/1497608587 shutdown_connections 2024-04-25T10:18:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f2814574740 1 --2- 172.21.15.119:0/1497608587 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x556cd96bccb0 0x556cd96bd0f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f2814574740 1 --2- 172.21.15.119:0/1497608587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96ce440 0x556cd96bbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f2814574740 1 --2- 172.21.15.119:0/1497608587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96bc830 0x556cd96bb7b0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f2814574740 1 --2- 172.21.15.119:0/1497608587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd9691d70 0x556cd96be9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f2814574740 1 -- 172.21.15.119:0/1497608587 >> 172.21.15.119:0/1497608587 conn(0x556cd9659a30 msgr2=0x556cd96b79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f27efe00640 0 client.9096 ms_handle_reset on v2:172.21.15.119:6836/2864575636 2024-04-25T10:18:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f2814574740 1 -- 172.21.15.119:0/1497608587 shutdown_connections 2024-04-25T10:18:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f2814574740 1 -- 172.21.15.119:0/1497608587 wait complete. 2024-04-25T10:18:09.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.174+0000 7f2814574740 20 client.9096 trim_cache size 0 max 16384 2024-04-25T10:18:09.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.190+0000 7f2814574740 1 Processor -- start 2024-04-25T10:18:09.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.190+0000 7f2814574740 1 -- start start 2024-04-25T10:18:09.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.191+0000 7f2814574740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd9691d70 0x556cd96bab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.191+0000 7f2814574740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96bc830 0x556cd96bb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.191+0000 7f2814574740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd96ce440 0x556cd96da970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.191+0000 7f2814574740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c1fff70 con 0x556cd9691d70 2024-04-25T10:18:09.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.191+0000 7f2814574740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c1ff0b0 con 0x556cd96ce440 2024-04-25T10:18:09.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.191+0000 7f2814574740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c155de0 con 0x556cd96bc830 2024-04-25T10:18:09.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.191+0000 7f280aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd9691d70 0x556cd96bab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.192+0000 7f280aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd9691d70 0x556cd96bab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44144/0 (socket says 172.21.15.119:44144) 2024-04-25T10:18:09.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.192+0000 7f280aa00640 1 -- 172.21.15.119:0/1914184519 learned_addr learned my addr 172.21.15.119:0/1914184519 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:09.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.192+0000 7f2802000640 1 --2- 172.21.15.119:0/1914184519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96bc830 0x556cd96bb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.192+0000 7f280aa00640 1 -- 172.21.15.119:0/1914184519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96bc830 msgr2=0x556cd96bb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.192+0000 7f280aa00640 1 --2- 172.21.15.119:0/1914184519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96bc830 0x556cd96bb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.192+0000 7f280aa00640 1 -- 172.21.15.119:0/1914184519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd96ce440 msgr2=0x556cd96da970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.192+0000 7f280b400640 1 --2- 172.21.15.119:0/1914184519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd96ce440 0x556cd96da970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.192+0000 7f280aa00640 1 --2- 172.21.15.119:0/1914184519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd96ce440 0x556cd96da970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.192+0000 7f280aa00640 1 -- 172.21.15.119:0/1914184519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cd9692670 con 0x556cd9691d70 2024-04-25T10:18:09.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.192+0000 7f280aa00640 1 --2- 172.21.15.119:0/1914184519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd9691d70 0x556cd96bab30 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x556cd96b7760 tx=0x7f27fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718a10abf872926d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.193+0000 7f27faa00640 1 -- 172.21.15.119:0/1914184519 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc022d90 con 0x556cd9691d70 2024-04-25T10:18:09.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.193+0000 7f27faa00640 1 -- 172.21.15.119:0/1914184519 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc01a040 con 0x556cd9691d70 2024-04-25T10:18:09.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.193+0000 7f27faa00640 1 -- 172.21.15.119:0/1914184519 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc015640 con 0x556cd9691d70 2024-04-25T10:18:09.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.194+0000 7f2814574740 1 -- 172.21.15.119:0/1914184519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd9691d70 msgr2=0x556cd96bab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.194+0000 7f2814574740 1 --2- 172.21.15.119:0/1914184519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd9691d70 0x556cd96bab30 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x556cd96b7760 tx=0x7f27fc014000 comp rx=0 tx=0).stop 2024-04-25T10:18:09.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.194+0000 7f2814574740 1 -- 172.21.15.119:0/1914184519 shutdown_connections 2024-04-25T10:18:09.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.194+0000 7f2814574740 1 --2- 172.21.15.119:0/1914184519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd96ce440 0x556cd96da970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.194+0000 7f2814574740 1 --2- 172.21.15.119:0/1914184519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd96bc830 0x556cd96bb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.194+0000 7f2814574740 1 --2- 172.21.15.119:0/1914184519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd9691d70 0x556cd96bab30 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.194+0000 7f2814574740 1 -- 172.21.15.119:0/1914184519 >> 172.21.15.119:0/1914184519 conn(0x556cd9659a30 msgr2=0x556cd96eaa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.194+0000 7f2814574740 1 -- 172.21.15.119:0/1914184519 shutdown_connections 2024-04-25T10:18:09.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.194+0000 7f2814574740 1 -- 172.21.15.119:0/1914184519 wait complete. 2024-04-25T10:18:09.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.194+0000 7f2814574740 1 -- start start 2024-04-25T10:18:09.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.195+0000 7f2814574740 1 Processor -- start 2024-04-25T10:18:09.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f2814574740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd9691d70 0x556cd974d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f2814574740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd96bc830 0x556cd96fa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f2814574740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96ce440 0x556cd96bab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f2814574740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd968ff00 con 0x556cd96ce440 2024-04-25T10:18:09.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f2814574740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c1563e0 con 0x556cd96bc830 2024-04-25T10:18:09.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f2814574740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x556cd95990d0 con 0x556cd9691d70 2024-04-25T10:18:09.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f2814574740 10 client.0 _ll_register_callbacks cb 0x7f27fc009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:09.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f280b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96ce440 0x556cd96bab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f280b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96ce440 0x556cd96bab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44150/0 (socket says 172.21.15.119:44150) 2024-04-25T10:18:09.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f280b400640 1 -- 172.21.15.119:0/1377106523 learned_addr learned my addr 172.21.15.119:0/1377106523 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:09.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f2802000640 1 --2- 172.21.15.119:0/1377106523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd96bc830 0x556cd96fa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f280aa00640 1 --2- 172.21.15.119:0/1377106523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd9691d70 0x556cd974d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f280b400640 1 -- 172.21.15.119:0/1377106523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd9691d70 msgr2=0x556cd974d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f280b400640 1 --2- 172.21.15.119:0/1377106523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd9691d70 0x556cd974d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f280b400640 1 -- 172.21.15.119:0/1377106523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd96bc830 msgr2=0x556cd96fa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f280b400640 1 --2- 172.21.15.119:0/1377106523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd96bc830 0x556cd96fa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f280b400640 1 -- 172.21.15.119:0/1377106523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556cd96d3fd0 con 0x556cd96ce440 2024-04-25T10:18:09.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.196+0000 7f2802000640 1 --2- 172.21.15.119:0/1377106523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd96bc830 0x556cd96fa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:09.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.197+0000 7f280b400640 1 --2- 172.21.15.119:0/1377106523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96ce440 0x556cd96bab30 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f2804054c20 tx=0x7f2804059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbe288627b860cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.197+0000 7f27efe00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:18:09.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.197+0000 7f27efe00640 1 -- 172.21.15.119:0/1377106523 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804086060 con 0x556cd96ce440 2024-04-25T10:18:09.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.197+0000 7f27efe00640 1 -- 172.21.15.119:0/1377106523 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2804070210 con 0x556cd96ce440 2024-04-25T10:18:09.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.197+0000 7f2814574740 1 -- 172.21.15.119:0/1377106523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27fc01a040 con 0x556cd96ce440 2024-04-25T10:18:09.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.197+0000 7f27efe00640 1 -- 172.21.15.119:0/1377106523 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2804075050 con 0x556cd96ce440 2024-04-25T10:18:09.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.197+0000 7f2814574740 10 client.9102 fetch_fsmap learned FSMap version 105 2024-04-25T10:18:09.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.197+0000 7f2814574740 1 -- 172.21.15.119:0/1377106523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x556cd96fac80 con 0x556cd96ce440 2024-04-25T10:18:09.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.197+0000 7f27efe00640 1 -- 172.21.15.119:0/1377106523 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc01a040 con 0x556cd96ce440 2024-04-25T10:18:09.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.197+0000 7f27efe00640 1 -- 172.21.15.119:0/1377106523 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2804054e30 con 0x556cd96ce440 2024-04-25T10:18:09.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.197+0000 7f2814574740 10 client.9102 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:18:09.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.197+0000 7f2814574740 10 client.9102 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:18:09.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.197+0000 7f2814574740 20 client.9102 populate_metadata read hostname 'smithi119' 2024-04-25T10:18:09.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.198+0000 7f2814574740 1 --2- 172.21.15.119:0/1377106523 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x556cd96bb070 0x556cd96bfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.198+0000 7f2814574740 4 client.9102 mds_command: new command op to 8943 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:18:09.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.198+0000 7f2814574740 1 -- 172.21.15.119:0/1377106523 --> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556cd95fd500 con 0x556cd96bb070 2024-04-25T10:18:09.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.198+0000 7f280aa00640 1 --2- 172.21.15.119:0/1377106523 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x556cd96bb070 0x556cd96bfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.198+0000 7f280aa00640 1 --2- 172.21.15.119:0/1377106523 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x556cd96bb070 0x556cd96bfab0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73c55b2074afb7e4 server_cookie=fe25bda8c6fa24ae in_seq=0 out_seq=0 2024-04-25T10:18:09.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.198+0000 7f27efe00640 10 client.9102 ms_handle_connect on v2:172.21.15.119:6836/2864575636 2024-04-25T10:18:09.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.212+0000 7f27efe00640 1 -- 172.21.15.119:0/1377106523 <== mds.0 v2:172.21.15.119:6836/2864575636 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556cd95fd500 con 0x556cd96bb070 2024-04-25T10:18:09.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.212+0000 7f27efe00640 10 client.9102 handle_command_reply: tid=0 2024-04-25T10:18:09.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.212+0000 7f2814574740 1 client.9102 shutdown 2024-04-25T10:18:09.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.212+0000 7f2814574740 1 -- 172.21.15.119:0/1377106523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96ce440 msgr2=0x556cd96bab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.212+0000 7f2814574740 1 --2- 172.21.15.119:0/1377106523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96ce440 0x556cd96bab30 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f2804054c20 tx=0x7f2804059000 comp rx=0 tx=0).stop 2024-04-25T10:18:09.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.212+0000 7f2814574740 1 -- 172.21.15.119:0/1377106523 shutdown_connections 2024-04-25T10:18:09.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.212+0000 7f2814574740 1 --2- 172.21.15.119:0/1377106523 >> [v2:172.21.15.119:6836/2864575636,v1:172.21.15.119:6837/2864575636] conn(0x556cd96bb070 0x556cd96bfab0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.212+0000 7f2814574740 1 --2- 172.21.15.119:0/1377106523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556cd96ce440 0x556cd96bab30 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.212+0000 7f2814574740 1 --2- 172.21.15.119:0/1377106523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556cd96bc830 0x556cd96fa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.212+0000 7f2814574740 1 --2- 172.21.15.119:0/1377106523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x556cd9691d70 0x556cd974d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.212+0000 7f27efe00640 0 client.9102 ms_handle_reset on v2:172.21.15.119:6836/2864575636 2024-04-25T10:18:09.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.212+0000 7f2814574740 1 -- 172.21.15.119:0/1377106523 >> 172.21.15.119:0/1377106523 conn(0x556cd9659a30 msgr2=0x556cd96bbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.213+0000 7f2814574740 1 -- 172.21.15.119:0/1377106523 shutdown_connections 2024-04-25T10:18:09.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.213+0000 7f2814574740 1 -- 172.21.15.119:0/1377106523 wait complete. 2024-04-25T10:18:09.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.213+0000 7f2814574740 20 client.9102 trim_cache size 0 max 16384 2024-04-25T10:18:09.217 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:18:09.217 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "", 2024-04-25T10:18:09.217 INFO:teuthology.orchestra.run.smithi119.stdout: "return_code": 0 2024-04-25T10:18:09.217 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:18:09.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.213+0000 7f280be00640 1 -- 172.21.15.119:0/4056542899 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f27e806b4d0 msgr2=0x7f27e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.213+0000 7f280be00640 1 --2- 172.21.15.119:0/4056542899 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f27e806b4d0 0x7f27e806d990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009ad0 tx=0x7f27fc008040 comp rx=0 tx=0).stop 2024-04-25T10:18:09.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.213+0000 7f280be00640 1 -- 172.21.15.119:0/4056542899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c15d980 msgr2=0x7f280c151900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.213+0000 7f280be00640 1 --2- 172.21.15.119:0/4056542899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c15d980 0x7f280c151900 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f280400c6e0 tx=0x7f280400cba0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.214+0000 7f280be00640 1 -- 172.21.15.119:0/4056542899 shutdown_connections 2024-04-25T10:18:09.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.214+0000 7f280be00640 1 --2- 172.21.15.119:0/4056542899 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f27e806b4d0 0x7f27e806d990 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.214+0000 7f280be00640 1 --2- 172.21.15.119:0/4056542899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280c15d980 0x7f280c151900 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.214+0000 7f280be00640 1 --2- 172.21.15.119:0/4056542899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f280c0b2a00 0x7f280c0ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.214+0000 7f280be00640 1 --2- 172.21.15.119:0/4056542899 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f280c0b2440 0x7f280c0cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.214+0000 7f280be00640 1 -- 172.21.15.119:0/4056542899 >> 172.21.15.119:0/4056542899 conn(0x7f280c0726c0 msgr2=0x7f280c0b86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.214+0000 7f280be00640 1 -- 172.21.15.119:0/4056542899 shutdown_connections 2024-04-25T10:18:09.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.214+0000 7f280be00640 1 -- 172.21.15.119:0/4056542899 wait complete. 2024-04-25T10:18:09.235 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:18:09.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.411+0000 7fe762e00640 1 Processor -- start 2024-04-25T10:18:09.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.411+0000 7fe762e00640 1 -- start start 2024-04-25T10:18:09.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.413+0000 7fe762e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe75c0aedc0 0x7fe75c15ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.413+0000 7fe762e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe75c06d260 0x7fe75c0b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.413+0000 7fe762e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe75c06da40 con 0x7fe75c0b4730 2024-04-25T10:18:09.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.413+0000 7fe762e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75c0b4ba0 con 0x7fe75c06d260 2024-04-25T10:18:09.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.413+0000 7fe762e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c15c050 con 0x7fe75c0aedc0 2024-04-25T10:18:09.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.414+0000 7fe760e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe75c0b4730 0x7fe75c0b26f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38332/0 (socket says 172.21.15.119:38332) 2024-04-25T10:18:09.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.414+0000 7fe760e00640 1 -- 172.21.15.119:0/3845066692 learned_addr learned my addr 172.21.15.119:0/3845066692 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:09.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.414+0000 7fe75be00640 1 --2- 172.21.15.119:0/3845066692 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe75c0aedc0 0x7fe75c15ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.414+0000 7fe75b400640 1 --2- 172.21.15.119:0/3845066692 >> v2:172.21.15.132:3300/0 conn(0x7fe75c06d260 0x7fe75c0b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.414+0000 7fe75aa00640 1 -- 172.21.15.119:0/3845066692 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307185491 0 0) 0x7fe75c06da40 con 0x7fe75c0b4730 2024-04-25T10:18:09.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.414+0000 7fe75aa00640 1 -- 172.21.15.119:0/3845066692 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe740003520 con 0x7fe75c0b4730 2024-04-25T10:18:09.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.414+0000 7fe75aa00640 1 -- 172.21.15.119:0/3845066692 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 312642830 0 0) 0x7fe740003520 con 0x7fe75c0b4730 2024-04-25T10:18:09.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.414+0000 7fe75be00640 1 -- 172.21.15.119:0/3845066692 >> v2:172.21.15.132:3300/0 conn(0x7fe75c06d260 msgr2=0x7fe75c0b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.414+0000 7fe75be00640 1 --2- 172.21.15.119:0/3845066692 >> v2:172.21.15.132:3300/0 conn(0x7fe75c06d260 0x7fe75c0b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.414+0000 7fe75be00640 1 -- 172.21.15.119:0/3845066692 >> v1:172.21.15.119:6789/0 conn(0x7fe75c0b4730 legacy=0x7fe75c0b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.414+0000 7fe75be00640 1 -- 172.21.15.119:0/3845066692 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c06dd20 con 0x7fe75c0aedc0 2024-04-25T10:18:09.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.416+0000 7fe75be00640 1 --2- 172.21.15.119:0/3845066692 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe75c0aedc0 0x7fe75c15ae20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009d90 tx=0x7fe74c02a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7562c7bb7ac5760 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.416+0000 7fe75aa00640 1 -- 172.21.15.119:0/3845066692 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754001670 con 0x7fe75c0aedc0 2024-04-25T10:18:09.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.416+0000 7fe75aa00640 1 -- 172.21.15.119:0/3845066692 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c007540 con 0x7fe75c0aedc0 2024-04-25T10:18:09.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.416+0000 7fe75aa00640 1 -- 172.21.15.119:0/3845066692 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c007910 con 0x7fe75c0aedc0 2024-04-25T10:18:09.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.417+0000 7fe762e00640 1 -- 172.21.15.119:0/3845066692 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe75c0aedc0 msgr2=0x7fe75c15ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.417+0000 7fe762e00640 1 --2- 172.21.15.119:0/3845066692 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe75c0aedc0 0x7fe75c15ae20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009d90 tx=0x7fe74c02a560 comp rx=0 tx=0).stop 2024-04-25T10:18:09.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.417+0000 7fe762e00640 1 -- 172.21.15.119:0/3845066692 shutdown_connections 2024-04-25T10:18:09.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.417+0000 7fe762e00640 1 --2- 172.21.15.119:0/3845066692 >> v2:172.21.15.132:3300/0 conn(0x7fe75c06d260 0x7fe75c0b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.417+0000 7fe762e00640 1 --2- 172.21.15.119:0/3845066692 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe75c0aedc0 0x7fe75c15ae20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.417+0000 7fe762e00640 1 -- 172.21.15.119:0/3845066692 >> 172.21.15.119:0/3845066692 conn(0x7fe75c06aba0 msgr2=0x7fe75c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.417+0000 7fe762e00640 1 -- 172.21.15.119:0/3845066692 shutdown_connections 2024-04-25T10:18:09.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.417+0000 7fe762e00640 1 -- 172.21.15.119:0/3845066692 wait complete. 2024-04-25T10:18:09.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.419+0000 7fe762e00640 1 Processor -- start 2024-04-25T10:18:09.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.419+0000 7fe762e00640 1 -- start start 2024-04-25T10:18:09.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.420+0000 7fe762e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe75c06d260 0x7fe75c1467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.420+0000 7fe762e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe75c0aedc0 0x7fe75c0acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.420+0000 7fe762e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe75c0b4730 0x7fe75c0ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.420+0000 7fe762e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c15c2a0 con 0x7fe75c06d260 2024-04-25T10:18:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.420+0000 7fe762e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c0af1a0 con 0x7fe75c0b4730 2024-04-25T10:18:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.420+0000 7fe762e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c06d6c0 con 0x7fe75c0aedc0 2024-04-25T10:18:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.420+0000 7fe75be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe75c06d260 0x7fe75c1467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.420+0000 7fe75be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe75c06d260 0x7fe75c1467c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44154/0 (socket says 172.21.15.119:44154) 2024-04-25T10:18:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.420+0000 7fe75be00640 1 -- 172.21.15.119:0/1390773343 learned_addr learned my addr 172.21.15.119:0/1390773343 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:09.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.421+0000 7fe760e00640 1 --2- 172.21.15.119:0/1390773343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe75c0b4730 0x7fe75c0ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.421+0000 7fe75b400640 1 --2- 172.21.15.119:0/1390773343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe75c0aedc0 0x7fe75c0acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.421+0000 7fe75be00640 1 -- 172.21.15.119:0/1390773343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe75c0aedc0 msgr2=0x7fe75c0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.421+0000 7fe75be00640 1 --2- 172.21.15.119:0/1390773343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe75c0aedc0 0x7fe75c0acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.421+0000 7fe75be00640 1 -- 172.21.15.119:0/1390773343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe75c0b4730 msgr2=0x7fe75c0ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.421+0000 7fe75be00640 1 --2- 172.21.15.119:0/1390773343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe75c0b4730 0x7fe75c0ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.421+0000 7fe75be00640 1 -- 172.21.15.119:0/1390773343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c0a9be0 con 0x7fe75c06d260 2024-04-25T10:18:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.421+0000 7fe75be00640 1 --2- 172.21.15.119:0/1390773343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe75c06d260 0x7fe75c1467c0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009d90 tx=0x7fe74c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.421+0000 7fe753e00640 1 -- 172.21.15.119:0/1390773343 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c040070 con 0x7fe75c06d260 2024-04-25T10:18:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.421+0000 7fe753e00640 1 -- 172.21.15.119:0/1390773343 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c031c40 con 0x7fe75c06d260 2024-04-25T10:18:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.421+0000 7fe753e00640 1 -- 172.21.15.119:0/1390773343 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c009450 con 0x7fe75c06d260 2024-04-25T10:18:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.422+0000 7fe762e00640 1 -- 172.21.15.119:0/1390773343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75c0a9ec0 con 0x7fe75c06d260 2024-04-25T10:18:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.422+0000 7fe762e00640 1 -- 172.21.15.119:0/1390773343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75c147250 con 0x7fe75c06d260 2024-04-25T10:18:09.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.423+0000 7fe762e00640 1 -- 172.21.15.119:0/1390773343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75c0750b0 con 0x7fe75c06d260 2024-04-25T10:18:09.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.424+0000 7fe753e00640 1 -- 172.21.15.119:0/1390773343 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe74c039040 con 0x7fe75c06d260 2024-04-25T10:18:09.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.425+0000 7fe753e00640 1 --2- 172.21.15.119:0/1390773343 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe73806b4d0 0x7fe73806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.425+0000 7fe753e00640 1 -- 172.21.15.119:0/1390773343 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe74c0abbc0 con 0x7fe75c06d260 2024-04-25T10:18:09.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.425+0000 7fe75b400640 1 --2- 172.21.15.119:0/1390773343 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe73806b4d0 0x7fe73806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.425+0000 7fe75b400640 1 --2- 172.21.15.119:0/1390773343 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe73806b4d0 0x7fe73806d990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0b0c90 tx=0x7fe744008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.429+0000 7fe753e00640 1 -- 172.21.15.119:0/1390773343 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe74c043360 con 0x7fe75c06d260 2024-04-25T10:18:09.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.664+0000 7fe762e00640 1 -- 172.21.15.119:0/1390773343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe75c067d30 con 0x7fe75c06d260 2024-04-25T10:18:09.669 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:18:09.666+0000 7f8c64600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:18:09.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.669+0000 7fe753e00640 1 -- 172.21.15.119:0/1390773343 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe74c07a910 con 0x7fe75c06d260 2024-04-25T10:18:09.673 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:18:09.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.673+0000 7fe762e00640 1 -- 172.21.15.119:0/1390773343 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe73806b4d0 msgr2=0x7fe73806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.673+0000 7fe762e00640 1 --2- 172.21.15.119:0/1390773343 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe73806b4d0 0x7fe73806d990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0b0c90 tx=0x7fe744008040 comp rx=0 tx=0).stop 2024-04-25T10:18:09.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.673+0000 7fe762e00640 1 -- 172.21.15.119:0/1390773343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe75c06d260 msgr2=0x7fe75c1467c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.673+0000 7fe762e00640 1 --2- 172.21.15.119:0/1390773343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe75c06d260 0x7fe75c1467c0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009d90 tx=0x7fe74c004a40 comp rx=0 tx=0).stop 2024-04-25T10:18:09.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.673+0000 7fe762e00640 1 -- 172.21.15.119:0/1390773343 shutdown_connections 2024-04-25T10:18:09.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.673+0000 7fe762e00640 1 --2- 172.21.15.119:0/1390773343 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe73806b4d0 0x7fe73806d990 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.673+0000 7fe762e00640 1 --2- 172.21.15.119:0/1390773343 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe75c0b4730 0x7fe75c0ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.673+0000 7fe762e00640 1 --2- 172.21.15.119:0/1390773343 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe75c0aedc0 0x7fe75c0acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.673+0000 7fe762e00640 1 --2- 172.21.15.119:0/1390773343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe75c06d260 0x7fe75c1467c0 secure :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009d90 tx=0x7fe74c004a40 comp rx=0 tx=0).stop 2024-04-25T10:18:09.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.673+0000 7fe762e00640 1 -- 172.21.15.119:0/1390773343 >> 172.21.15.119:0/1390773343 conn(0x7fe75c06aba0 msgr2=0x7fe75c0b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.674+0000 7fe762e00640 1 -- 172.21.15.119:0/1390773343 shutdown_connections 2024-04-25T10:18:09.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.675+0000 7fe762e00640 1 -- 172.21.15.119:0/1390773343 wait complete. 2024-04-25T10:18:09.692 INFO:tasks.ceph.mds.c.smithi119.stderr: -1829> 2024-04-25T10:18:02.460+0000 7ff8eec00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:18:09.697 DEBUG:teuthology.orchestra.run.smithi119:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-25T10:18:09.752 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2024-04-25T10:18:09.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.785+0000 7f80b7695280 1 -- start start 2024-04-25T10:18:09.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.785+0000 7f80b7695280 1 Processor -- start 2024-04-25T10:18:09.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.786+0000 7f80b7695280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780b44c00 0x557780bb0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.786+0000 7f80b7695280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780b45800 0x557780bb0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.786+0000 7f80b7695280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780b45c00 0x557780bb1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.787+0000 7f80b7695280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557781914180 con 0x557780b45800 2024-04-25T10:18:09.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.787+0000 7f80b7695280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557781914300 con 0x557780b45c00 2024-04-25T10:18:09.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.787+0000 7f80b7695280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557781914480 con 0x557780b44c00 2024-04-25T10:18:09.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.787+0000 7f80b4e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780b45800 0x557780bb0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.787+0000 7f80b4e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780b45800 0x557780bb0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44198/0 (socket says 172.21.15.119:44198) 2024-04-25T10:18:09.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.787+0000 7f80b4e00640 1 -- 172.21.15.119:0/3808652823 learned_addr learned my addr 172.21.15.119:0/3808652823 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:09.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.787+0000 7f80b5800640 1 --2- 172.21.15.119:0/3808652823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780b44c00 0x557780bb0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.787+0000 7f80b6200640 1 --2- 172.21.15.119:0/3808652823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780b45c00 0x557780bb1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.787+0000 7f80b4e00640 1 -- 172.21.15.119:0/3808652823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780b44c00 msgr2=0x557780bb0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.787+0000 7f80b4e00640 1 --2- 172.21.15.119:0/3808652823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780b44c00 0x557780bb0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.787+0000 7f80b4e00640 1 -- 172.21.15.119:0/3808652823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780b45c00 msgr2=0x557780bb1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.787+0000 7f80b4e00640 1 --2- 172.21.15.119:0/3808652823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780b45c00 0x557780bb1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.787+0000 7f80b4e00640 1 -- 172.21.15.119:0/3808652823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557780b78780 con 0x557780b45800 2024-04-25T10:18:09.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.788+0000 7f80b4e00640 1 --2- 172.21.15.119:0/3808652823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780b45800 0x557780bb0b00 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x5577819593b0 tx=0x557780e36480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06ea7919fbfad51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.789+0000 7f80b2600640 1 -- 172.21.15.119:0/3808652823 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557780b6a680 con 0x557780b45800 2024-04-25T10:18:09.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.789+0000 7f80b2600640 1 -- 172.21.15.119:0/3808652823 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557780b70700 con 0x557780b45800 2024-04-25T10:18:09.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.789+0000 7f80b2600640 1 -- 172.21.15.119:0/3808652823 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557780b6ab60 con 0x557780b45800 2024-04-25T10:18:09.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.789+0000 7f80b7695280 1 -- 172.21.15.119:0/3808652823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557780b794a0 con 0x557780b45800 2024-04-25T10:18:09.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.789+0000 7f80b7695280 1 -- 172.21.15.119:0/3808652823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557780b78000 con 0x557780b45800 2024-04-25T10:18:09.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.789+0000 7f80b7695280 4 waiting for MDS map... 2024-04-25T10:18:09.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.789+0000 7f80b2600640 1 -- 172.21.15.119:0/3808652823 <== mon.0 v2:172.21.15.119:3300/0 4 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x557780e40a00 con 0x557780b45800 2024-04-25T10:18:09.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.790+0000 7f80b2600640 1 -- 172.21.15.119:0/3808652823 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x557780b9f180 con 0x557780b45800 2024-04-25T10:18:09.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.790+0000 7f80b7695280 4 Got MDS map 106 2024-04-25T10:18:09.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.790+0000 7f80b7695280 10 main: JournalTool::main 2024-04-25T10:18:09.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.790+0000 7f80b7695280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:18:09.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.791+0000 7f80b7695280 1 Processor -- start 2024-04-25T10:18:09.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.791+0000 7f80b7695280 1 -- start start 2024-04-25T10:18:09.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.791+0000 7f80b7695280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780e43000 0x557780bb1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.791+0000 7f80b7695280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780e43400 0x557780bb1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.791+0000 7f80b7695280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780e43800 0x557780bb2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.792+0000 7f80b7695280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557781914600 con 0x557780e43000 2024-04-25T10:18:09.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.792+0000 7f80b7695280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557781914780 con 0x557780e43800 2024-04-25T10:18:09.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.792+0000 7f80b7695280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557781914900 con 0x557780e43400 2024-04-25T10:18:09.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.792+0000 7f80b6200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780e43000 0x557780bb1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.792+0000 7f80b6200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780e43000 0x557780bb1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44212/0 (socket says 172.21.15.119:44212) 2024-04-25T10:18:09.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.792+0000 7f80b6200640 1 -- 172.21.15.119:0/253098189 learned_addr learned my addr 172.21.15.119:0/253098189 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:09.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.792+0000 7f80b6200640 1 -- 172.21.15.119:0/253098189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780e43400 msgr2=0x557780bb1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.792+0000 7f80b6200640 1 --2- 172.21.15.119:0/253098189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780e43400 0x557780bb1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.792+0000 7f80b6200640 1 -- 172.21.15.119:0/253098189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780e43800 msgr2=0x557780bb2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.792+0000 7f80b4e00640 1 --2- 172.21.15.119:0/253098189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780e43800 0x557780bb2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.792+0000 7f80b6200640 1 --2- 172.21.15.119:0/253098189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780e43800 0x557780bb2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.792+0000 7f80b6200640 1 -- 172.21.15.119:0/253098189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557780b783c0 con 0x557780e43000 2024-04-25T10:18:09.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.793+0000 7f80b6200640 1 --2- 172.21.15.119:0/253098189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780e43000 0x557780bb1600 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x557780e5c660 tx=0x557780eb60c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1607f842725822f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.793+0000 7f80afe00640 1 -- 172.21.15.119:0/253098189 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557780b6a680 con 0x557780e43000 2024-04-25T10:18:09.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.793+0000 7f80afe00640 1 -- 172.21.15.119:0/253098189 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557780b70fc0 con 0x557780e43000 2024-04-25T10:18:09.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.793+0000 7f80afe00640 1 -- 172.21.15.119:0/253098189 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557780b6ab60 con 0x557780e43000 2024-04-25T10:18:09.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.793+0000 7f80b7695280 1 -- 172.21.15.119:0/253098189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780e43000 msgr2=0x557780bb1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.793+0000 7f80b7695280 1 --2- 172.21.15.119:0/253098189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780e43000 0x557780bb1600 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x557780e5c660 tx=0x557780eb60c0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.793+0000 7f80b7695280 1 -- 172.21.15.119:0/253098189 shutdown_connections 2024-04-25T10:18:09.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.793+0000 7f80b7695280 1 --2- 172.21.15.119:0/253098189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780e43800 0x557780bb2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.793+0000 7f80b7695280 1 --2- 172.21.15.119:0/253098189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780e43400 0x557780bb1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.793+0000 7f80b7695280 1 --2- 172.21.15.119:0/253098189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780e43000 0x557780bb1600 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.794+0000 7f80b7695280 1 -- 172.21.15.119:0/253098189 >> 172.21.15.119:0/253098189 conn(0x557780e42c00 msgr2=0x557780c4b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.794+0000 7f80b7695280 1 -- 172.21.15.119:0/253098189 shutdown_connections 2024-04-25T10:18:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.794+0000 7f80b7695280 1 -- 172.21.15.119:0/253098189 wait complete. 2024-04-25T10:18:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.794+0000 7f80b7695280 1 Processor -- start 2024-04-25T10:18:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.795+0000 7f80b7695280 1 -- start start 2024-04-25T10:18:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.795+0000 7f80b7695280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780e43000 0x557780bb3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.795+0000 7f80b7695280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780e43400 0x557780bb2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.795+0000 7f80b7695280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780e43800 0x557780bb2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.795+0000 7f80b7695280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557781915c80 con 0x557780e43000 2024-04-25T10:18:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.795+0000 7f80b7695280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557781915b00 con 0x557780e43400 2024-04-25T10:18:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.795+0000 7f80b7695280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557781915980 con 0x557780e43800 2024-04-25T10:18:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.796+0000 7f80b6200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780e43000 0x557780bb3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.796+0000 7f80b6200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780e43000 0x557780bb3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44218/0 (socket says 172.21.15.119:44218) 2024-04-25T10:18:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.796+0000 7f80b6200640 1 -- 172.21.15.119:0/1816505997 learned_addr learned my addr 172.21.15.119:0/1816505997 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:09.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.796+0000 7f80b6200640 1 -- 172.21.15.119:0/1816505997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780e43800 msgr2=0x557780bb2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.796+0000 7f80b6200640 1 --2- 172.21.15.119:0/1816505997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780e43800 0x557780bb2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.796+0000 7f80b5800640 1 --2- 172.21.15.119:0/1816505997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780e43400 0x557780bb2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.796+0000 7f80b6200640 1 -- 172.21.15.119:0/1816505997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780e43400 msgr2=0x557780bb2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.796+0000 7f80b6200640 1 --2- 172.21.15.119:0/1816505997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780e43400 0x557780bb2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.796+0000 7f80b6200640 1 -- 172.21.15.119:0/1816505997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557780b78780 con 0x557780e43000 2024-04-25T10:18:09.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.796+0000 7f80b6200640 1 --2- 172.21.15.119:0/1816505997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780e43000 0x557780bb3180 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x557780e5d320 tx=0x557780eb6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.796+0000 7f80ae000640 1 -- 172.21.15.119:0/1816505997 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557780b6a4e0 con 0x557780e43000 2024-04-25T10:18:09.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.796+0000 7f80ae000640 1 -- 172.21.15.119:0/1816505997 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557780b70e00 con 0x557780e43000 2024-04-25T10:18:09.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.796+0000 7f80b7695280 1 -- 172.21.15.119:0/1816505997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557780b794a0 con 0x557780e43000 2024-04-25T10:18:09.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.796+0000 7f80ae000640 1 -- 172.21.15.119:0/1816505997 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557780b6a680 con 0x557780e43000 2024-04-25T10:18:09.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.797+0000 7f80b7695280 1 -- 172.21.15.119:0/1816505997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557780b78960 con 0x557780e43000 2024-04-25T10:18:09.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.797+0000 7f80b7695280 4 main: JournalTool: resolving pool 16 2024-04-25T10:18:09.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.798+0000 7f80ae000640 1 -- 172.21.15.119:0/1816505997 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x557780ec0c00 con 0x557780e43000 2024-04-25T10:18:09.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.799+0000 7f80ae000640 1 --2- 172.21.15.119:0/1816505997 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x557780eb8800 0x557780bb3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.799+0000 7f80ae000640 1 -- 172.21.15.119:0/1816505997 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x557780ebaa00 con 0x557780e43000 2024-04-25T10:18:09.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.799+0000 7f80b7695280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:18:09.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.799+0000 7f80b7695280 4 main: Executing for rank 0 2024-04-25T10:18:09.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.799+0000 7f80b5800640 1 --2- 172.21.15.119:0/1816505997 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x557780eb8800 0x557780bb3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.800+0000 7f80b5800640 1 --2- 172.21.15.119:0/1816505997 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x557780eb8800 0x557780bb3700 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x557780b8def0 tx=0x557780e36240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.800+0000 7f80b7695280 1 -- start start 2024-04-25T10:18:09.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.800+0000 7f80b7695280 1 Processor -- start 2024-04-25T10:18:09.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.801+0000 7f80b7695280 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780eb9000 0x557780bb2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.801+0000 7f80b7695280 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780eb9400 0x557780bb1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.801+0000 7f80b7695280 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780eb9800 0x557780bb1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.801+0000 7f80b7695280 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557781915680 con 0x557780eb9800 2024-04-25T10:18:09.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.801+0000 7f80b7695280 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557781914600 con 0x557780eb9000 2024-04-25T10:18:09.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.801+0000 7f80b7695280 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x557781914780 con 0x557780eb9400 2024-04-25T10:18:09.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.801+0000 7f80b4e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780eb9800 0x557780bb1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.801+0000 7f80b4e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780eb9800 0x557780bb1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44232/0 (socket says 172.21.15.119:44232) 2024-04-25T10:18:09.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.801+0000 7f80b4e00640 1 -- 172.21.15.119:0/3337018985 learned_addr learned my addr 172.21.15.119:0/3337018985 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:09.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.802+0000 7f80b6200640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780eb9000 0x557780bb2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.802+0000 7f80b4e00640 1 -- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780eb9400 msgr2=0x557780bb1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.802+0000 7f80b4e00640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780eb9400 0x557780bb1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.802+0000 7f80b4e00640 1 -- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780eb9000 msgr2=0x557780bb2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.802+0000 7f80b4e00640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780eb9000 0x557780bb2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.802+0000 7f80b4e00640 1 -- 172.21.15.119:0/3337018985 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557780b79680 con 0x557780eb9800 2024-04-25T10:18:09.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.802+0000 7f80b6200640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780eb9000 0x557780bb2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:09.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.802+0000 7f80b4e00640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780eb9800 0x557780bb1600 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x557780f77620 tx=0x557780e36060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75898bfd5c930c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.802+0000 7f80aa400640 1 -- 172.21.15.119:0/3337018985 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557780b6a820 con 0x557780eb9800 2024-04-25T10:18:09.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.802+0000 7f80aa400640 1 -- 172.21.15.119:0/3337018985 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557780b701c0 con 0x557780eb9800 2024-04-25T10:18:09.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.802+0000 7f80aa400640 1 -- 172.21.15.119:0/3337018985 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557780ef9860 con 0x557780eb9800 2024-04-25T10:18:09.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.802+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557780b783c0 con 0x557780eb9800 2024-04-25T10:18:09.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.802+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557780b78780 con 0x557780eb9800 2024-04-25T10:18:09.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.802+0000 7f80b7695280 4 waiting for MDS map... 2024-04-25T10:18:09.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.803+0000 7f80aa400640 1 -- 172.21.15.119:0/3337018985 <== mon.0 v2:172.21.15.119:3300/0 4 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x55778100c200 con 0x557780eb9800 2024-04-25T10:18:09.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.803+0000 7f80aa400640 1 -- 172.21.15.119:0/3337018985 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x557780b9f880 con 0x557780eb9800 2024-04-25T10:18:09.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.803+0000 7f80b7695280 4 Got MDS map 106 2024-04-25T10:18:09.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.803+0000 7f80b7695280 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x557780eb9c00 0x55778101e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.803+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x55778100f400 con 0x557780eb9c00 2024-04-25T10:18:09.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.803+0000 7f80b6200640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x557780eb9c00 0x55778101e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.804+0000 7f80b6200640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x557780eb9c00 0x55778101e000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=c886743b445b283b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.809+0000 7f80b6200640 1 -- 172.21.15.119:0/3337018985 <== osd.0 v2:172.21.15.119:6808/2890625333 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v112'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5577818c2000 con 0x557780eb9c00 2024-04-25T10:18:09.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.810+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x55778100f800 con 0x557780eb9c00 2024-04-25T10:18:09.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.812+0000 7f80b6200640 1 -- 172.21.15.119:0/3337018985 <== osd.0 v2:172.21.15.119:6808/2890625333 2 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v112'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5577818c2000 con 0x557780eb9c00 2024-04-25T10:18:09.815 INFO:teuthology.orchestra.run.smithi119.stdout:writing EResetJournal entry 2024-04-25T10:18:09.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.812+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x55778100fc00 con 0x557780eb9c00 2024-04-25T10:18:09.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.812+0000 7f80b7695280 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55778102a000 0x55778101e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.812+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x55778102a400 con 0x55778102a000 2024-04-25T10:18:09.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.812+0000 7f80b7695280 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55778102a800 0x55778101eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.812+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:5 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x55778102ac00 con 0x55778102a800 2024-04-25T10:18:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.812+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:6 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x55778102b000 con 0x55778102a000 2024-04-25T10:18:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.812+0000 7f80b7695280 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55778102b400 0x55778101f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.812+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:7 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x55778102b800 con 0x55778102b400 2024-04-25T10:18:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.812+0000 7f80b7695280 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55778102bc00 0x55778101f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.812+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:8 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x557781034000 con 0x55778102bc00 2024-04-25T10:18:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.812+0000 7f80b6200640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55778102a800 0x55778101eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.812+0000 7f80b5800640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55778102b400 0x55778101f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.813+0000 7f80b4e00640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55778102bc00 0x55778101f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.813+0000 7f80b5800640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55778102a000 0x55778101e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.813+0000 7f80b6200640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55778102a800 0x55778101eb00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=15820737c14beeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.813+0000 7f80b5800640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55778102b400 0x55778101f080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=78018659783265b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.813+0000 7f80b6200640 1 -- 172.21.15.119:0/3337018985 <== osd.0 v2:172.21.15.119:6808/2890625333 3 ==== osd_op_reply(3 200.00000001 [delete] v112'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5577818c2000 con 0x557780eb9c00 2024-04-25T10:18:09.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.814+0000 7f80b5800640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55778102a000 0x55778101e580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=e6020cf6093d5f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.814+0000 7f80b4e00640 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55778102bc00 0x55778101f600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=b3f1ba95365d357c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.817+0000 7f80b5800640 1 -- 172.21.15.119:0/3337018985 <== osd.3 v2:172.21.15.119:6816/1142971786 1 ==== osd_op_reply(7 200.00000005 [delete] v112'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5577818c2500 con 0x55778102b400 2024-04-25T10:18:09.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.817+0000 7f80b6200640 1 -- 172.21.15.119:0/3337018985 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== osd_op_reply(5 200.00000003 [delete] v112'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5577818c2000 con 0x55778102a800 2024-04-25T10:18:09.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.818+0000 7f80b5800640 1 -- 172.21.15.119:0/3337018985 <== osd.5 v2:172.21.15.132:6815/2978152964 1 ==== osd_op_reply(4 200.00000002 [delete] v112'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5577818c2500 con 0x55778102a000 2024-04-25T10:18:09.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.818+0000 7f80a8600640 1 -- 172.21.15.119:0/3337018985 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:9 16.14 16:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x557781917400 con 0x557780eb9c00 2024-04-25T10:18:09.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.819+0000 7f80b5800640 1 -- 172.21.15.119:0/3337018985 <== osd.5 v2:172.21.15.132:6815/2978152964 2 ==== osd_op_reply(6 200.00000004 [delete] v112'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5577818c2500 con 0x55778102a000 2024-04-25T10:18:09.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.820+0000 7f80b6200640 1 -- 172.21.15.119:0/3337018985 <== osd.0 v2:172.21.15.119:6808/2890625333 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v112'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5577818c2000 con 0x557780eb9c00 2024-04-25T10:18:09.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b4e00640 1 -- 172.21.15.119:0/3337018985 <== osd.7 v2:172.21.15.132:6821/2727385616 1 ==== osd_op_reply(8 200.00000006 [delete] v112'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5577818c2780 con 0x55778102bc00 2024-04-25T10:18:09.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-25T10:18:09.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x557780eb9c00 msgr2=0x55778101e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x557780eb9c00 0x55778101e000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55778102a800 msgr2=0x55778101eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55778102a800 0x55778101eb00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55778102b400 msgr2=0x55778101f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55778102b400 0x55778101f080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55778102a000 msgr2=0x55778101e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55778102a000 0x55778101e580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55778102bc00 msgr2=0x55778101f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55778102bc00 0x55778101f600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b4e00640 1 -- 172.21.15.119:0/3337018985 reap_dead start 2024-04-25T10:18:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780eb9800 msgr2=0x557780bb1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780eb9800 0x557780bb1600 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x557780f77620 tx=0x557780e36060 comp rx=0 tx=0).stop 2024-04-25T10:18:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 shutdown_connections 2024-04-25T10:18:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55778102bc00 0x55778101f600 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 --2- 172.21.15.119:0/3337018985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780eb9800 0x557780bb1600 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 >> 172.21.15.119:0/3337018985 conn(0x557780eb8c00 msgr2=0x557780c4b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 shutdown_connections 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.825+0000 7f80b7695280 1 -- 172.21.15.119:0/3337018985 wait complete. 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.826+0000 7f80b7695280 1 -- 172.21.15.119:0/1816505997 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x557780eb8800 msgr2=0x557780bb3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.826+0000 7f80b7695280 1 --2- 172.21.15.119:0/1816505997 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x557780eb8800 0x557780bb3700 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x557780b8def0 tx=0x557780e36240 comp rx=0 tx=0).stop 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.826+0000 7f80b7695280 1 -- 172.21.15.119:0/1816505997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780e43000 msgr2=0x557780bb3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.826+0000 7f80b7695280 1 --2- 172.21.15.119:0/1816505997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780e43000 0x557780bb3180 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x557780e5d320 tx=0x557780eb6000 comp rx=0 tx=0).stop 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.826+0000 7f80b7695280 1 -- 172.21.15.119:0/1816505997 shutdown_connections 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.826+0000 7f80b7695280 1 --2- 172.21.15.119:0/1816505997 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x557780eb8800 0x557780bb3700 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.826+0000 7f80b7695280 1 --2- 172.21.15.119:0/1816505997 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780e43800 0x557780bb2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.826+0000 7f80b7695280 1 --2- 172.21.15.119:0/1816505997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780e43400 0x557780bb2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.826+0000 7f80b7695280 1 --2- 172.21.15.119:0/1816505997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780e43000 0x557780bb3180 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.826+0000 7f80b7695280 1 -- 172.21.15.119:0/1816505997 >> 172.21.15.119:0/1816505997 conn(0x557780e42c00 msgr2=0x557780c4b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.826+0000 7f80b7695280 1 -- 172.21.15.119:0/1816505997 shutdown_connections 2024-04-25T10:18:09.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.826+0000 7f80b7695280 1 -- 172.21.15.119:0/1816505997 wait complete. 2024-04-25T10:18:09.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.827+0000 7f80b7695280 1 -- 172.21.15.119:0/3808652823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780b45800 msgr2=0x557780bb0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.827+0000 7f80b7695280 1 --2- 172.21.15.119:0/3808652823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780b45800 0x557780bb0b00 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x5577819593b0 tx=0x557780e36480 comp rx=0 tx=0).stop 2024-04-25T10:18:09.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.827+0000 7f80b7695280 1 -- 172.21.15.119:0/3808652823 shutdown_connections 2024-04-25T10:18:09.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.827+0000 7f80b7695280 1 --2- 172.21.15.119:0/3808652823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557780b45c00 0x557780bb1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.827+0000 7f80b7695280 1 --2- 172.21.15.119:0/3808652823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557780b45800 0x557780bb0b00 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.827+0000 7f80b7695280 1 --2- 172.21.15.119:0/3808652823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557780b44c00 0x557780bb0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.827+0000 7f80b7695280 1 -- 172.21.15.119:0/3808652823 >> 172.21.15.119:0/3808652823 conn(0x557780b45000 msgr2=0x557780c4a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.827+0000 7f80b7695280 1 -- 172.21.15.119:0/3808652823 shutdown_connections 2024-04-25T10:18:09.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.827+0000 7f80b7695280 1 -- 172.21.15.119:0/3808652823 wait complete. 2024-04-25T10:18:09.836 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.138824, result: writing EResetJournal entry 2024-04-25T10:18:09.836 DEBUG:teuthology.orchestra.run.smithi119:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T10:18:09.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.916+0000 7f0d5400d740 1 -- start start 2024-04-25T10:18:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.916+0000 7f0d5400d740 1 Processor -- start 2024-04-25T10:18:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.917+0000 7f0d5400d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5634bc16c400 0x5634bb437600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.917+0000 7f0d5400d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bb397c00 0x5634bb436b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.917+0000 7f0d5400d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5634bc16c800 0x5634bb437b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.917+0000 7f0d5400d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bc160000 con 0x5634bb397c00 2024-04-25T10:18:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.917+0000 7f0d5400d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bc160180 con 0x5634bc16c800 2024-04-25T10:18:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.917+0000 7f0d5400d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634bc160300 con 0x5634bc16c400 2024-04-25T10:18:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.917+0000 7f0d50a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bb397c00 0x5634bb436b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.918+0000 7f0d50a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bb397c00 0x5634bb436b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44254/0 (socket says 172.21.15.119:44254) 2024-04-25T10:18:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.918+0000 7f0d50a00640 1 -- 172.21.15.119:0/936984724 learned_addr learned my addr 172.21.15.119:0/936984724 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.918+0000 7f0d51e00640 1 --2- 172.21.15.119:0/936984724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5634bc16c800 0x5634bb437b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.918+0000 7f0d51400640 1 --2- 172.21.15.119:0/936984724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5634bc16c400 0x5634bb437600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.918+0000 7f0d50a00640 1 -- 172.21.15.119:0/936984724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5634bc16c400 msgr2=0x5634bb437600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.918+0000 7f0d50a00640 1 --2- 172.21.15.119:0/936984724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5634bc16c400 0x5634bb437600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.918+0000 7f0d50a00640 1 -- 172.21.15.119:0/936984724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5634bc16c800 msgr2=0x5634bb437b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.918+0000 7f0d50a00640 1 --2- 172.21.15.119:0/936984724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5634bc16c800 0x5634bb437b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.918+0000 7f0d50a00640 1 -- 172.21.15.119:0/936984724 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634bb3c3680 con 0x5634bb397c00 2024-04-25T10:18:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.918+0000 7f0d51400640 1 --2- 172.21.15.119:0/936984724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5634bc16c400 0x5634bb437600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.918+0000 7f0d50a00640 1 --2- 172.21.15.119:0/936984724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bb397c00 0x5634bb436b00 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x5634bc1b14a0 tx=0x5634bc127c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224a954e13f73c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.919+0000 7f0d4e200640 1 -- 172.21.15.119:0/936984724 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5634bb3bcd00 con 0x5634bb397c00 2024-04-25T10:18:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.919+0000 7f0d4e200640 1 -- 172.21.15.119:0/936984724 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5634bb3e0c40 con 0x5634bb397c00 2024-04-25T10:18:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.919+0000 7f0d4e200640 1 -- 172.21.15.119:0/936984724 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5634bb3bc680 con 0x5634bb397c00 2024-04-25T10:18:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.919+0000 7f0d5400d740 1 -- 172.21.15.119:0/936984724 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634bb3c3860 con 0x5634bb397c00 2024-04-25T10:18:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.919+0000 7f0d5400d740 1 -- 172.21.15.119:0/936984724 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5634bb3c30e0 con 0x5634bb397c00 2024-04-25T10:18:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.919+0000 7f0d5400d740 4 waiting for MDS map... 2024-04-25T10:18:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.919+0000 7f0d4e200640 1 -- 172.21.15.119:0/936984724 <== mon.0 v2:172.21.15.119:3300/0 4 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x5634bc260400 con 0x5634bb397c00 2024-04-25T10:18:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.919+0000 7f0d4e200640 1 -- 172.21.15.119:0/936984724 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x5634bb3ef180 con 0x5634bb397c00 2024-04-25T10:18:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.919+0000 7f0d5400d740 4 Got MDS map 106 2024-04-25T10:18:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.919+0000 7f0d5400d740 4 datascan.main: Using metadata pool output 2024-04-25T10:18:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.919+0000 7f0d5400d740 4 datascan.main: connecting to RADOS... 2024-04-25T10:18:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.920+0000 7f0d5400d740 1 Processor -- start 2024-04-25T10:18:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.920+0000 7f0d5400d740 1 -- start start 2024-04-25T10:18:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.921+0000 7f0d5400d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5634bc2ba000 0x5634bb437080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.921+0000 7f0d5400d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5634bc2ba400 0x5634bb436580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.921+0000 7f0d5400d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bc2ba800 0x5634bb438100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.921+0000 7f0d5400d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bc160480 con 0x5634bc2ba800 2024-04-25T10:18:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.921+0000 7f0d5400d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bc160600 con 0x5634bc2ba400 2024-04-25T10:18:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.921+0000 7f0d5400d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634bc160780 con 0x5634bc2ba000 2024-04-25T10:18:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.921+0000 7f0d50a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bc2ba800 0x5634bb438100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.921+0000 7f0d50a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bc2ba800 0x5634bb438100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44268/0 (socket says 172.21.15.119:44268) 2024-04-25T10:18:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.921+0000 7f0d50a00640 1 -- 172.21.15.119:0/1798194098 learned_addr learned my addr 172.21.15.119:0/1798194098 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.921+0000 7f0d50a00640 1 -- 172.21.15.119:0/1798194098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5634bc2ba000 msgr2=0x5634bb437080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.921+0000 7f0d50a00640 1 --2- 172.21.15.119:0/1798194098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5634bc2ba000 0x5634bb437080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.921+0000 7f0d50a00640 1 -- 172.21.15.119:0/1798194098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5634bc2ba400 msgr2=0x5634bb436580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.921+0000 7f0d50a00640 1 --2- 172.21.15.119:0/1798194098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5634bc2ba400 0x5634bb436580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.922+0000 7f0d50a00640 1 -- 172.21.15.119:0/1798194098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634bb3c34a0 con 0x5634bc2ba800 2024-04-25T10:18:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.922+0000 7f0d50a00640 1 --2- 172.21.15.119:0/1798194098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bc2ba800 0x5634bb438100 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x5634bc1b1a10 tx=0x5634bc126000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294cced33e0a6fa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.922+0000 7f0d4ba00640 1 -- 172.21.15.119:0/1798194098 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5634bb3bcb60 con 0x5634bc2ba800 2024-04-25T10:18:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.922+0000 7f0d4ba00640 1 -- 172.21.15.119:0/1798194098 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5634bb3e0a80 con 0x5634bc2ba800 2024-04-25T10:18:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.922+0000 7f0d4ba00640 1 -- 172.21.15.119:0/1798194098 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5634bb3bcd00 con 0x5634bc2ba800 2024-04-25T10:18:09.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.922+0000 7f0d5400d740 1 -- 172.21.15.119:0/1798194098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bc2ba800 msgr2=0x5634bb438100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.922+0000 7f0d5400d740 1 --2- 172.21.15.119:0/1798194098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bc2ba800 0x5634bb438100 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x5634bc1b1a10 tx=0x5634bc126000 comp rx=0 tx=0).stop 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.923+0000 7f0d5400d740 1 -- 172.21.15.119:0/1798194098 shutdown_connections 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.923+0000 7f0d5400d740 1 --2- 172.21.15.119:0/1798194098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bc2ba800 0x5634bb438100 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.923+0000 7f0d5400d740 1 --2- 172.21.15.119:0/1798194098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5634bc2ba400 0x5634bb436580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.923+0000 7f0d5400d740 1 --2- 172.21.15.119:0/1798194098 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5634bc2ba000 0x5634bb437080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.923+0000 7f0d5400d740 1 -- 172.21.15.119:0/1798194098 >> 172.21.15.119:0/1798194098 conn(0x5634bc16c000 msgr2=0x5634bc111000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.923+0000 7f0d5400d740 1 -- 172.21.15.119:0/1798194098 shutdown_connections 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.923+0000 7f0d5400d740 1 -- 172.21.15.119:0/1798194098 wait complete. 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.923+0000 7f0d5400d740 1 Processor -- start 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.924+0000 7f0d5400d740 1 -- start start 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.924+0000 7f0d5400d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5634bc2ba000 0x5634bb439180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.924+0000 7f0d5400d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5634bc2ba400 0x5634bb438c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.924+0000 7f0d5400d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bc2ba800 0x5634bb438680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.924+0000 7f0d5400d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bc160a80 con 0x5634bc2ba800 2024-04-25T10:18:09.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.924+0000 7f0d5400d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bc160900 con 0x5634bc2ba400 2024-04-25T10:18:09.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.924+0000 7f0d5400d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634bc161500 con 0x5634bc2ba000 2024-04-25T10:18:09.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.924+0000 7f0d50a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bc2ba800 0x5634bb438680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.925+0000 7f0d50a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bc2ba800 0x5634bb438680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44282/0 (socket says 172.21.15.119:44282) 2024-04-25T10:18:09.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.925+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 learned_addr learned my addr 172.21.15.119:0/22485321 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:09.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.925+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5634bc2ba000 msgr2=0x5634bb439180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.925+0000 7f0d50a00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5634bc2ba000 0x5634bb439180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.925+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5634bc2ba400 msgr2=0x5634bb438c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.925+0000 7f0d51400640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5634bc2ba400 0x5634bb438c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.925+0000 7f0d50a00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5634bc2ba400 0x5634bb438c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.925+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634bb3c3680 con 0x5634bc2ba800 2024-04-25T10:18:09.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.925+0000 7f0d51400640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5634bc2ba400 0x5634bb438c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:09.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.925+0000 7f0d50a00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bc2ba800 0x5634bb438680 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x5634bc26bb00 tx=0x5634bc119d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.925+0000 7f0d49c00640 1 -- 172.21.15.119:0/22485321 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5634bb3bc680 con 0x5634bc2ba800 2024-04-25T10:18:09.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.926+0000 7f0d49c00640 1 -- 172.21.15.119:0/22485321 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5634bb3e1340 con 0x5634bc2ba800 2024-04-25T10:18:09.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.926+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5634bb3c32c0 con 0x5634bc2ba800 2024-04-25T10:18:09.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.926+0000 7f0d49c00640 1 -- 172.21.15.119:0/22485321 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5634bb3bc820 con 0x5634bc2ba800 2024-04-25T10:18:09.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.926+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634bb3c3a40 con 0x5634bc2ba800 2024-04-25T10:18:09.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.926+0000 7f0d5400d740 4 datascan.init: resolving metadata pool 16 2024-04-25T10:18:09.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.927+0000 7f0d49c00640 1 -- 172.21.15.119:0/22485321 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x5634bc2c0c00 con 0x5634bc2ba800 2024-04-25T10:18:09.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.928+0000 7f0d49c00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x5634bb6e8000 0x5634bb439700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.928+0000 7f0d49c00640 1 -- 172.21.15.119:0/22485321 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x5634bc2c4400 con 0x5634bc2ba800 2024-04-25T10:18:09.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.928+0000 7f0d5400d740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:18:09.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.928+0000 7f0d5400d740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-25T10:18:09.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.928+0000 7f0d5400d740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:18:09.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.928+0000 7f0d5400d740 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5634bc2bac00 0x5634bb438100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.928+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bc2bb000 con 0x5634bc2bac00 2024-04-25T10:18:09.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.928+0000 7f0d51e00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x5634bb6e8000 0x5634bb439700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.929+0000 7f0d51e00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x5634bb6e8000 0x5634bb439700 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x5634bb3cb4a0 tx=0x5634bc2c6180 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.929+0000 7f0d51400640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5634bc2bac00 0x5634bb438100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.930+0000 7f0d51400640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5634bc2bac00 0x5634bb438100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.931+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.7 v2:172.21.15.132:6821/2727385616 1 ==== osd_op_reply(1 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bc2bac00 2024-04-25T10:18:09.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.931+0000 7f0d4b000640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5634bb6e8400 0x5634bb706000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.931+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:2 17.20 17:04000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb6e8800 con 0x5634bb6e8400 2024-04-25T10:18:09.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.931+0000 7f0d51e00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5634bb6e8400 0x5634bb706000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.932+0000 7f0d51e00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5634bb6e8400 0x5634bb706000 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.932+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== osd_op_reply(2 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e8400 2024-04-25T10:18:09.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.932+0000 7f0d4ba00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x5634bb6e9000 0x5634bb706580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.932+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:3 17.10 17:08000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb6e9400 con 0x5634bb6e9000 2024-04-25T10:18:09.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.933+0000 7f0d51400640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x5634bb6e9000 0x5634bb706580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.933+0000 7f0d51400640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x5634bb6e9000 0x5634bb706580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.933+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.3 v2:172.21.15.119:6816/1142971786 1 ==== osd_op_reply(3 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bb6e9000 2024-04-25T10:18:09.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.933+0000 7f0d4b000640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x5634bb6e8c00 0x5634bb706b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.934+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:4 17.30 17:0c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb6e9400 con 0x5634bb6e8c00 2024-04-25T10:18:09.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.934+0000 7f0d50a00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x5634bb6e8c00 0x5634bb706b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.934+0000 7f0d50a00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x5634bb6e8c00 0x5634bb706b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.935+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.0 v2:172.21.15.119:6808/2890625333 1 ==== osd_op_reply(4 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb6e8c00 2024-04-25T10:18:09.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.935+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:5 17.8 17:10000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb6e9800 con 0x5634bc2bac00 2024-04-25T10:18:09.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.935+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.7 v2:172.21.15.132:6821/2727385616 2 ==== osd_op_reply(5 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bc2bac00 2024-04-25T10:18:09.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.935+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:6 17.28 17:14000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb6e9c00 con 0x5634bb6e9000 2024-04-25T10:18:09.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.936+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.3 v2:172.21.15.119:6816/1142971786 2 ==== osd_op_reply(6 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bb6e9000 2024-04-25T10:18:09.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.936+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:7 17.18 17:18000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71a000 con 0x5634bb6e8c00 2024-04-25T10:18:09.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.936+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.0 v2:172.21.15.119:6808/2890625333 2 ==== osd_op_reply(7 [pgnls start_epoch 112 out=81b] v110'17 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5634bc114780 con 0x5634bb6e8c00 2024-04-25T10:18:09.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.936+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:8 17.18 17:1ad8d55b:::10000000006.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71ac00 con 0x5634bb6e8c00 2024-04-25T10:18:09.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.936+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.0 v2:172.21.15.119:6808/2890625333 3 ==== osd_op_reply(8 10000000006.00000001 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5634bc114780 con 0x5634bb6e8c00 2024-04-25T10:18:09.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.937+0000 7f0d5400d740 10 datascan.operator(): handling object 1099511627782.1 2024-04-25T10:18:09.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.937+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:9 17.25 17:a5feb6f3:::10000000006.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71b000 con 0x5634bb6e8400 2024-04-25T10:18:09.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.938+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.1 v2:172.21.15.119:6800/298545098 2 ==== osd_op_reply(9 10000000006.00000000 [call] v112'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e8400 2024-04-25T10:18:09.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.938+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71b400 con 0x5634bc2bac00 2024-04-25T10:18:09.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.939+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.7 v2:172.21.15.132:6821/2727385616 3 ==== osd_op_reply(10 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bc2bac00 2024-04-25T10:18:09.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.939+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71a400 con 0x5634bb6e8c00 2024-04-25T10:18:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.939+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.0 v2:172.21.15.119:6808/2890625333 4 ==== osd_op_reply(11 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb6e8c00 2024-04-25T10:18:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.939+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bc2bb000 con 0x5634bb6e9000 2024-04-25T10:18:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.940+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.3 v2:172.21.15.119:6816/1142971786 3 ==== osd_op_reply(12 [pgnls start_epoch 112 out=49b] v110'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bb6e9000 2024-04-25T10:18:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.940+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71a800 con 0x5634bb6e9000 2024-04-25T10:18:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.940+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.3 v2:172.21.15.119:6816/1142971786 4 ==== osd_op_reply(13 [pgnls start_epoch 112 out=49b] v110'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bb6e9000 2024-04-25T10:18:09.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.940+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71b400 con 0x5634bb6e8c00 2024-04-25T10:18:09.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.940+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.0 v2:172.21.15.119:6808/2890625333 5 ==== osd_op_reply(14 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb6e8c00 2024-04-25T10:18:09.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.940+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71e000 con 0x5634bc2bac00 2024-04-25T10:18:09.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.941+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.7 v2:172.21.15.132:6821/2727385616 4 ==== osd_op_reply(15 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bc2bac00 2024-04-25T10:18:09.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.942+0000 7f0d4b000640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5634bb6e9800 0x5634bb707080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.942+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb6e9c00 con 0x5634bb6e9800 2024-04-25T10:18:09.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.942+0000 7f0d51e00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5634bb6e9800 0x5634bb707080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.943+0000 7f0d51e00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5634bb6e9800 0x5634bb707080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.944+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.4 v2:172.21.15.132:6800/1740646323 1 ==== osd_op_reply(16 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e9800 2024-04-25T10:18:09.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.944+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71e400 con 0x5634bb6e8400 2024-04-25T10:18:09.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.944+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.1 v2:172.21.15.119:6800/298545098 3 ==== osd_op_reply(17 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e8400 2024-04-25T10:18:09.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.944+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71f000 con 0x5634bb6e8c00 2024-04-25T10:18:09.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.945+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.0 v2:172.21.15.119:6808/2890625333 6 ==== osd_op_reply(18 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb6e8c00 2024-04-25T10:18:09.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.945+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71e800 con 0x5634bb6e9800 2024-04-25T10:18:09.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.946+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.4 v2:172.21.15.132:6800/1740646323 2 ==== osd_op_reply(19 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e9800 2024-04-25T10:18:09.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.946+0000 7f0d4b000640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5634bb71f400 0x5634bb707600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.946+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb6e8800 con 0x5634bb71f400 2024-04-25T10:18:09.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.947+0000 7f0d51400640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5634bb71f400 0x5634bb707600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.948+0000 7f0d51400640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5634bb71f400 0x5634bb707600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.949+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.5 v2:172.21.15.132:6815/2978152964 1 ==== osd_op_reply(20 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bb71f400 2024-04-25T10:18:09.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.949+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71ec00 con 0x5634bb6e9000 2024-04-25T10:18:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.949+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.3 v2:172.21.15.119:6816/1142971786 5 ==== osd_op_reply(21 [pgnls start_epoch 112 out=81b] v111'22 uv22 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5634bc114000 con 0x5634bb6e9000 2024-04-25T10:18:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.950+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:22 17.12 17:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71b800 con 0x5634bb6e9000 2024-04-25T10:18:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.950+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.3 v2:172.21.15.119:6816/1142971786 6 ==== osd_op_reply(22 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5634bc114000 con 0x5634bb6e9000 2024-04-25T10:18:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.950+0000 7f0d5400d740 10 datascan.operator(): handling object 1099511627778.0 2024-04-25T10:18:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.950+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:23 17.12 17:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71bc00 con 0x5634bb6e9000 2024-04-25T10:18:09.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.953+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.3 v2:172.21.15.119:6816/1142971786 7 ==== osd_op_reply(23 10000000002.00000000 [call] v112'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5634bc114000 con 0x5634bb6e9000 2024-04-25T10:18:09.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.953+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:24 17.32 17:4c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71a800 con 0x5634bc2bac00 2024-04-25T10:18:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.954+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.7 v2:172.21.15.132:6821/2727385616 5 ==== osd_op_reply(24 [pgnls start_epoch 112 out=81b] v110'16 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5634bc114000 con 0x5634bc2bac00 2024-04-25T10:18:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.954+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:25 17.32 17:4def72d4:::10000000003.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71a000 con 0x5634bc2bac00 2024-04-25T10:18:09.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.955+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.7 v2:172.21.15.132:6821/2727385616 6 ==== osd_op_reply(25 10000000003.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5634bc114000 con 0x5634bc2bac00 2024-04-25T10:18:09.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.955+0000 7f0d5400d740 10 datascan.operator(): handling object 1099511627779.0 2024-04-25T10:18:09.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.955+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:26 17.32 17:4def72d4:::10000000003.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71ac00 con 0x5634bc2bac00 2024-04-25T10:18:09.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.960+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.7 v2:172.21.15.132:6821/2727385616 7 ==== osd_op_reply(26 10000000003.00000000 [call] v112'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5634bc114000 con 0x5634bc2bac00 2024-04-25T10:18:09.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.960+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:27 17.a 17:50000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71a400 con 0x5634bc2bac00 2024-04-25T10:18:09.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.961+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.7 v2:172.21.15.132:6821/2727385616 8 ==== osd_op_reply(27 [pgnls start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bc2bac00 2024-04-25T10:18:09.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.961+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:28 17.2a 17:54000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb6e9c00 con 0x5634bb6e9000 2024-04-25T10:18:09.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.961+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.3 v2:172.21.15.119:6816/1142971786 8 ==== osd_op_reply(28 [pgnls start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bb6e9000 2024-04-25T10:18:09.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.961+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:29 17.1a 17:58000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71b800 con 0x5634bb6e8c00 2024-04-25T10:18:09.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.962+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.0 v2:172.21.15.119:6808/2890625333 7 ==== osd_op_reply(29 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb6e8c00 2024-04-25T10:18:09.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.962+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:30 17.3a 17:5c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71b000 con 0x5634bb6e8400 2024-04-25T10:18:09.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.962+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.1 v2:172.21.15.119:6800/298545098 4 ==== osd_op_reply(30 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e8400 2024-04-25T10:18:09.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.962+0000 7f0d4ba00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5634bb71bc00 0x5634bb708680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.963+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:31 17.6 17:60000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71a800 con 0x5634bb71bc00 2024-04-25T10:18:09.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.964+0000 7f0d50a00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5634bb71bc00 0x5634bb708680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.965+0000 7f0d50a00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5634bb71bc00 0x5634bb708680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.966+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.6 v2:172.21.15.132:6806/353653608 1 ==== osd_op_reply(31 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb71bc00 2024-04-25T10:18:09.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.966+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:32 17.26 17:64000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71ec00 con 0x5634bb6e8c00 2024-04-25T10:18:09.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.967+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.0 v2:172.21.15.119:6808/2890625333 8 ==== osd_op_reply(32 [pgnls start_epoch 112 out=49b] v110'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb6e8c00 2024-04-25T10:18:09.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.967+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:33 17.16 17:68000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71a000 con 0x5634bb71f400 2024-04-25T10:18:09.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.968+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.5 v2:172.21.15.132:6815/2978152964 2 ==== osd_op_reply(33 [pgnls start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bb71f400 2024-04-25T10:18:09.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.968+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:34 17.36 17:6c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71e000 con 0x5634bb6e8c00 2024-04-25T10:18:09.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.968+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.0 v2:172.21.15.119:6808/2890625333 9 ==== osd_op_reply(34 [pgnls start_epoch 112 out=49b] v110'32 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb6e8c00 2024-04-25T10:18:09.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.968+0000 7f0d4ba00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x5634bb71b400 0x5634bb708c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.968+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:35 17.e 17:70000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73c000 con 0x5634bb71b400 2024-04-25T10:18:09.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.969+0000 7f0d51e00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x5634bb71b400 0x5634bb708c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.970+0000 7f0d51e00640 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x5634bb71b400 0x5634bb708c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.970+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.2 v2:172.21.15.119:6824/1001762963 1 ==== osd_op_reply(35 [pgnls start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb71b400 2024-04-25T10:18:09.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.970+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:36 17.2e 17:74000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71f000 con 0x5634bb71bc00 2024-04-25T10:18:09.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.971+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.6 v2:172.21.15.132:6806/353653608 2 ==== osd_op_reply(36 [pgnls start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb71bc00 2024-04-25T10:18:09.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.971+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:37 17.1e 17:78000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73c400 con 0x5634bb6e8400 2024-04-25T10:18:09.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.972+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.1 v2:172.21.15.119:6800/298545098 5 ==== osd_op_reply(37 [pgnls start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e8400 2024-04-25T10:18:09.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.972+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:38 17.3e 17:7c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71f800 con 0x5634bb6e9800 2024-04-25T10:18:09.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.973+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.4 v2:172.21.15.132:6800/1740646323 3 ==== osd_op_reply(38 [pgnls start_epoch 112 out=49b] v110'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e9800 2024-04-25T10:18:09.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.973+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:39 17.1 17:80000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73c800 con 0x5634bb6e8c00 2024-04-25T10:18:09.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.974+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.0 v2:172.21.15.119:6808/2890625333 10 ==== osd_op_reply(39 [pgnls start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb6e8c00 2024-04-25T10:18:09.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.974+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:40 17.21 17:84000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71fc00 con 0x5634bb6e8c00 2024-04-25T10:18:09.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.974+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.0 v2:172.21.15.119:6808/2890625333 11 ==== osd_op_reply(40 [pgnls start_epoch 112 out=49b] v110'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb6e8c00 2024-04-25T10:18:09.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.974+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:41 17.11 17:88000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73cc00 con 0x5634bb6e9000 2024-04-25T10:18:09.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.975+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.3 v2:172.21.15.119:6816/1142971786 9 ==== osd_op_reply(41 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bb6e9000 2024-04-25T10:18:09.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.975+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:42 17.31 17:8c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71b800 con 0x5634bb6e8400 2024-04-25T10:18:09.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.976+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.1 v2:172.21.15.119:6800/298545098 6 ==== osd_op_reply(42 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e8400 2024-04-25T10:18:09.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.976+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:43 17.9 17:90000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73d000 con 0x5634bb6e9000 2024-04-25T10:18:09.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.976+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.3 v2:172.21.15.119:6816/1142971786 10 ==== osd_op_reply(43 [pgnls start_epoch 112 out=49b] v110'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bb6e9000 2024-04-25T10:18:09.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.976+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:44 17.29 17:94000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71ec00 con 0x5634bb6e9800 2024-04-25T10:18:09.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.978+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.4 v2:172.21.15.132:6800/1740646323 4 ==== osd_op_reply(44 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e9800 2024-04-25T10:18:09.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.978+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:45 17.19 17:98000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73d400 con 0x5634bb71f400 2024-04-25T10:18:09.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.979+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.5 v2:172.21.15.132:6815/2978152964 3 ==== osd_op_reply(45 [pgnls start_epoch 112 out=49b] v110'21 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bb71f400 2024-04-25T10:18:09.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.979+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:46 17.39 17:9c000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71e000 con 0x5634bb71f400 2024-04-25T10:18:09.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.980+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.5 v2:172.21.15.132:6815/2978152964 4 ==== osd_op_reply(46 [pgnls start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bb71f400 2024-04-25T10:18:09.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.980+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:47 17.5 17:a0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73cc00 con 0x5634bb71bc00 2024-04-25T10:18:09.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.981+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.6 v2:172.21.15.132:6806/353653608 3 ==== osd_op_reply(47 [pgnls start_epoch 112 out=81b] v111'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5634bc114780 con 0x5634bb71bc00 2024-04-25T10:18:09.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.981+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:48 17.5 17:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb6e8800 con 0x5634bb71bc00 2024-04-25T10:18:09.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.982+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.6 v2:172.21.15.132:6806/353653608 4 ==== osd_op_reply(48 10000000004.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5634bc114780 con 0x5634bb71bc00 2024-04-25T10:18:09.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.982+0000 7f0d5400d740 10 datascan.operator(): handling object 1099511627780.0 2024-04-25T10:18:09.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.982+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:49 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb6e9400 con 0x5634bb71bc00 2024-04-25T10:18:09.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.986+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.6 v2:172.21.15.132:6806/353653608 5 ==== osd_op_reply(49 10000000004.00000000 [call] v112'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5634bc114780 con 0x5634bb71bc00 2024-04-25T10:18:09.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.986+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:50 17.25 17:a4000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bc2bb000 con 0x5634bb6e8400 2024-04-25T10:18:09.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.987+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.1 v2:172.21.15.119:6800/298545098 7 ==== osd_op_reply(50 [pgnls start_epoch 112 out=81b] v112'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e8400 2024-04-25T10:18:09.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.987+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:51 17.25 17:a5feb6f3:::10000000006.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73c400 con 0x5634bb6e8400 2024-04-25T10:18:09.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.988+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.1 v2:172.21.15.119:6800/298545098 8 ==== osd_op_reply(51 10000000006.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e8400 2024-04-25T10:18:09.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.988+0000 7f0d5400d740 10 datascan.operator(): handling object 1099511627782.0 2024-04-25T10:18:09.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.988+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:52 17.25 17:a5feb6f3:::10000000006.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73d000 con 0x5634bb6e8400 2024-04-25T10:18:09.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.991+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.1 v2:172.21.15.119:6800/298545098 9 ==== osd_op_reply(52 10000000006.00000000 [call] v112'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e8400 2024-04-25T10:18:09.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.991+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:53 17.15 17:a8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73c800 con 0x5634bb71f400 2024-04-25T10:18:09.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.992+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.5 v2:172.21.15.132:6815/2978152964 5 ==== osd_op_reply(53 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bb71f400 2024-04-25T10:18:09.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.992+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:54 17.35 17:ac000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71f000 con 0x5634bb6e9800 2024-04-25T10:18:09.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.993+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.4 v2:172.21.15.132:6800/1740646323 5 ==== osd_op_reply(54 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e9800 2024-04-25T10:18:09.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.994+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:55 17.d 17:b0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71ac00 con 0x5634bc2bac00 2024-04-25T10:18:09.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.995+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.7 v2:172.21.15.132:6821/2727385616 9 ==== osd_op_reply(55 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bc2bac00 2024-04-25T10:18:09.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.995+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:56 17.2d 17:b4000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71fc00 con 0x5634bc2bac00 2024-04-25T10:18:09.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.996+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.7 v2:172.21.15.132:6821/2727385616 10 ==== osd_op_reply(56 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bc2bac00 2024-04-25T10:18:09.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.996+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:57 17.1d 17:b8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71a400 con 0x5634bc2bac00 2024-04-25T10:18:10.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.997+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.7 v2:172.21.15.132:6821/2727385616 11 ==== osd_op_reply(57 [pgnls start_epoch 112 out=49b] v110'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bc2bac00 2024-04-25T10:18:10.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.997+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:58 17.3d 17:bc000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71e400 con 0x5634bc2bac00 2024-04-25T10:18:10.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.998+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.7 v2:172.21.15.132:6821/2727385616 12 ==== osd_op_reply(58 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bc2bac00 2024-04-25T10:18:10.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.998+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:59 17.3 17:c0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71a000 con 0x5634bb6e9800 2024-04-25T10:18:10.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.999+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.4 v2:172.21.15.132:6800/1740646323 6 ==== osd_op_reply(59 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e9800 2024-04-25T10:18:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:09.999+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:60 17.23 17:c4000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71e800 con 0x5634bb6e9800 2024-04-25T10:18:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.000+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.4 v2:172.21.15.132:6800/1740646323 7 ==== osd_op_reply(60 [pgnls start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e9800 2024-04-25T10:18:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.001+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:61 17.13 17:c8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71a800 con 0x5634bb6e9000 2024-04-25T10:18:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.002+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.3 v2:172.21.15.119:6816/1142971786 11 ==== osd_op_reply(61 [pgnls start_epoch 112 out=49b] v110'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bb6e9000 2024-04-25T10:18:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.002+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:62 17.33 17:cc000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73d400 con 0x5634bc2bac00 2024-04-25T10:18:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.003+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.7 v2:172.21.15.132:6821/2727385616 13 ==== osd_op_reply(62 [pgnls start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bc2bac00 2024-04-25T10:18:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.003+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:63 17.b 17:d0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71b000 con 0x5634bb6e8c00 2024-04-25T10:18:10.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.004+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.0 v2:172.21.15.119:6808/2890625333 12 ==== osd_op_reply(63 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb6e8c00 2024-04-25T10:18:10.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.004+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:64 17.2b 17:d4000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73c800 con 0x5634bb6e9800 2024-04-25T10:18:10.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.006+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.4 v2:172.21.15.132:6800/1740646323 8 ==== osd_op_reply(64 [pgnls start_epoch 112 out=49b] v110'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e9800 2024-04-25T10:18:10.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.006+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:65 17.1b 17:d8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb6e9c00 con 0x5634bb6e9800 2024-04-25T10:18:10.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.008+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.4 v2:172.21.15.132:6800/1740646323 9 ==== osd_op_reply(65 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e9800 2024-04-25T10:18:10.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.008+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:66 17.3b 17:dc000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73d800 con 0x5634bb71b400 2024-04-25T10:18:10.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.008+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.2 v2:172.21.15.119:6824/1001762963 2 ==== osd_op_reply(66 [pgnls start_epoch 112 out=49b] v110'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb71b400 2024-04-25T10:18:10.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.008+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:67 17.7 17:e0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73c400 con 0x5634bb71f400 2024-04-25T10:18:10.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.009+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 <== osd.5 v2:172.21.15.132:6815/2978152964 6 ==== osd_op_reply(67 [pgnls start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114000 con 0x5634bb71f400 2024-04-25T10:18:10.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.009+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:68 17.27 17:e4000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71e000 con 0x5634bb6e9800 2024-04-25T10:18:10.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.010+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.4 v2:172.21.15.132:6800/1740646323 10 ==== osd_op_reply(68 [pgnls start_epoch 112 out=49b] v110'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e9800 2024-04-25T10:18:10.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.011+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:69 17.17 17:e8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73d000 con 0x5634bb71bc00 2024-04-25T10:18:10.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.012+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.6 v2:172.21.15.132:6806/353653608 6 ==== osd_op_reply(69 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb71bc00 2024-04-25T10:18:10.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.012+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:70 17.37 17:ec000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71fc00 con 0x5634bb6e8c00 2024-04-25T10:18:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.013+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.0 v2:172.21.15.119:6808/2890625333 13 ==== osd_op_reply(70 [pgnls start_epoch 112 out=49b] v110'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb6e8c00 2024-04-25T10:18:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.013+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:71 17.f 17:f0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73c800 con 0x5634bb6e8400 2024-04-25T10:18:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.014+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.1 v2:172.21.15.119:6800/298545098 10 ==== osd_op_reply(71 [pgnls start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e8400 2024-04-25T10:18:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.014+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:72 17.2f 17:f4000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71e400 con 0x5634bb6e9800 2024-04-25T10:18:10.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.015+0000 7f0d51e00640 1 -- 172.21.15.119:0/22485321 <== osd.4 v2:172.21.15.132:6800/1740646323 11 ==== osd_op_reply(72 [pgnls start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114500 con 0x5634bb6e9800 2024-04-25T10:18:10.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.015+0000 7f0d4ba00640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:73 17.1f 17:f8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb73dc00 con 0x5634bb71bc00 2024-04-25T10:18:10.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.016+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.6 v2:172.21.15.132:6806/353653608 7 ==== osd_op_reply(73 [pgnls start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb71bc00 2024-04-25T10:18:10.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.016+0000 7f0d4b000640 1 -- 172.21.15.119:0/22485321 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:74 17.3f 17:fc000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x5634bb71ac00 con 0x5634bb6e8c00 2024-04-25T10:18:10.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.017+0000 7f0d50a00640 1 -- 172.21.15.119:0/22485321 <== osd.0 v2:172.21.15.119:6808/2890625333 14 ==== osd_op_reply(74 [pgnls start_epoch 112 out=49b] v110'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bc114780 con 0x5634bb6e8c00 2024-04-25T10:18:10.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.017+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x5634bb6e8c00 msgr2=0x5634bb706b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.017+0000 7f0d5400d740 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x5634bb6e8c00 0x5634bb706b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.017+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5634bb6e8400 msgr2=0x5634bb706000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.017+0000 7f0d5400d740 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x5634bb6e8400 0x5634bb706000 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.017+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x5634bb71b400 msgr2=0x5634bb708c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.017+0000 7f0d5400d740 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x5634bb71b400 0x5634bb708c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.017+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x5634bb6e9000 msgr2=0x5634bb706580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d5400d740 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x5634bb6e9000 0x5634bb706580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5634bb6e9800 msgr2=0x5634bb707080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d5400d740 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x5634bb6e9800 0x5634bb707080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 reap_dead start 2024-04-25T10:18:10.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 reap_dead start 2024-04-25T10:18:10.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5634bb71f400 msgr2=0x5634bb707600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d5400d740 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x5634bb71f400 0x5634bb707600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5634bb71bc00 msgr2=0x5634bb708680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d5400d740 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x5634bb71bc00 0x5634bb708680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5634bc2bac00 msgr2=0x5634bb438100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d5400d740 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x5634bc2bac00 0x5634bb438100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x5634bb6e8000 msgr2=0x5634bb439700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d5400d740 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x5634bb6e8000 0x5634bb439700 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x5634bb3cb4a0 tx=0x5634bc2c6180 comp rx=0 tx=0).stop 2024-04-25T10:18:10.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bc2ba800 msgr2=0x5634bb438680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d5400d740 1 --2- 172.21.15.119:0/22485321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bc2ba800 0x5634bb438680 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x5634bc26bb00 tx=0x5634bc119d40 comp rx=0 tx=0).stop 2024-04-25T10:18:10.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.018+0000 7f0d51400640 1 -- 172.21.15.119:0/22485321 reap_dead start 2024-04-25T10:18:10.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.019+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 shutdown_connections 2024-04-25T10:18:10.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.019+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 >> 172.21.15.119:0/22485321 conn(0x5634bc16c000 msgr2=0x5634bc111000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.019+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 shutdown_connections 2024-04-25T10:18:10.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.019+0000 7f0d5400d740 1 -- 172.21.15.119:0/22485321 wait complete. 2024-04-25T10:18:10.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.020+0000 7f0d5400d740 1 -- 172.21.15.119:0/936984724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bb397c00 msgr2=0x5634bb436b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.020+0000 7f0d5400d740 1 --2- 172.21.15.119:0/936984724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bb397c00 0x5634bb436b00 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x5634bc1b14a0 tx=0x5634bc127c20 comp rx=0 tx=0).stop 2024-04-25T10:18:10.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.020+0000 7f0d5400d740 1 -- 172.21.15.119:0/936984724 shutdown_connections 2024-04-25T10:18:10.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.020+0000 7f0d5400d740 1 --2- 172.21.15.119:0/936984724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5634bc16c800 0x5634bb437b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.020+0000 7f0d5400d740 1 --2- 172.21.15.119:0/936984724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5634bb397c00 0x5634bb436b00 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.020+0000 7f0d5400d740 1 --2- 172.21.15.119:0/936984724 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x5634bc16c400 0x5634bb437600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.020+0000 7f0d5400d740 1 -- 172.21.15.119:0/936984724 >> 172.21.15.119:0/936984724 conn(0x5634bb397000 msgr2=0x5634bc112000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.021+0000 7f0d5400d740 1 -- 172.21.15.119:0/936984724 shutdown_connections 2024-04-25T10:18:10.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.021+0000 7f0d5400d740 1 -- 172.21.15.119:0/936984724 wait complete. 2024-04-25T10:18:10.036 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.200606, result: 2024-04-25T10:18:10.037 DEBUG:teuthology.orchestra.run.smithi119:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-25T10:18:10.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.144+0000 7fb882322740 1 -- start start 2024-04-25T10:18:10.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.144+0000 7fb882322740 1 Processor -- start 2024-04-25T10:18:10.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.145+0000 7fb882322740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8492ac00 0x561e849ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.145+0000 7fb882322740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e8492b800 0x561e849cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.145+0000 7fb882322740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561e8492bc00 0x561e849cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.145+0000 7fb882322740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e857e4180 con 0x561e8492ac00 2024-04-25T10:18:10.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.145+0000 7fb882322740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e857e4300 con 0x561e8492b800 2024-04-25T10:18:10.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.145+0000 7fb882322740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e857e4480 con 0x561e8492bc00 2024-04-25T10:18:10.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.146+0000 7fb87fc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8492ac00 0x561e849ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.146+0000 7fb87fc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8492ac00 0x561e849ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44290/0 (socket says 172.21.15.119:44290) 2024-04-25T10:18:10.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.146+0000 7fb87fc00640 1 -- 172.21.15.119:0/147662422 learned_addr learned my addr 172.21.15.119:0/147662422 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:10.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.146+0000 7fb87f200640 1 --2- 172.21.15.119:0/147662422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e8492b800 0x561e849cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.146+0000 7fb880600640 1 --2- 172.21.15.119:0/147662422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561e8492bc00 0x561e849cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.146+0000 7fb87fc00640 1 -- 172.21.15.119:0/147662422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561e8492bc00 msgr2=0x561e849cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.146+0000 7fb87fc00640 1 --2- 172.21.15.119:0/147662422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561e8492bc00 0x561e849cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.146+0000 7fb87fc00640 1 -- 172.21.15.119:0/147662422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e8492b800 msgr2=0x561e849cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.146+0000 7fb87fc00640 1 --2- 172.21.15.119:0/147662422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e8492b800 0x561e849cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.146+0000 7fb87fc00640 1 -- 172.21.15.119:0/147662422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e84957680 con 0x561e8492ac00 2024-04-25T10:18:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.147+0000 7fb87fc00640 1 --2- 172.21.15.119:0/147662422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8492ac00 0x561e849ca580 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x561e85737c50 tx=0x561e857f8420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e98499745af33ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.147+0000 7fb87ca00640 1 -- 172.21.15.119:0/147662422 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561e84950680 con 0x561e8492ac00 2024-04-25T10:18:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.147+0000 7fb87ca00640 1 -- 172.21.15.119:0/147662422 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e84974e00 con 0x561e8492ac00 2024-04-25T10:18:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.147+0000 7fb87ca00640 1 -- 172.21.15.119:0/147662422 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561e84950b60 con 0x561e8492ac00 2024-04-25T10:18:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.147+0000 7fb882322740 1 -- 172.21.15.119:0/147662422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e84957860 con 0x561e8492ac00 2024-04-25T10:18:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.147+0000 7fb882322740 1 -- 172.21.15.119:0/147662422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x561e849572c0 con 0x561e8492ac00 2024-04-25T10:18:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.147+0000 7fb882322740 4 waiting for MDS map... 2024-04-25T10:18:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.147+0000 7fb87ca00640 1 -- 172.21.15.119:0/147662422 <== mon.0 v2:172.21.15.119:3300/0 4 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x561e857fe600 con 0x561e8492ac00 2024-04-25T10:18:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.147+0000 7fb87ca00640 1 -- 172.21.15.119:0/147662422 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x561e84983180 con 0x561e8492ac00 2024-04-25T10:18:10.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.147+0000 7fb882322740 4 Got MDS map 106 2024-04-25T10:18:10.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.148+0000 7fb882322740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-25T10:18:10.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.148+0000 7fb882322740 4 datascan.main: Using metadata pool output 2024-04-25T10:18:10.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.148+0000 7fb882322740 4 datascan.main: connecting to RADOS... 2024-04-25T10:18:10.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.148+0000 7fb882322740 1 Processor -- start 2024-04-25T10:18:10.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.149+0000 7fb882322740 1 -- start start 2024-04-25T10:18:10.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.149+0000 7fb882322740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561e857ed000 0x561e849cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.149+0000 7fb882322740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e857ed400 0x561e849cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.149+0000 7fb882322740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e857ed800 0x561e849cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.149+0000 7fb882322740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e857e4600 con 0x561e857ed800 2024-04-25T10:18:10.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.149+0000 7fb882322740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e857e4780 con 0x561e857ed400 2024-04-25T10:18:10.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.149+0000 7fb882322740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e857e4900 con 0x561e857ed000 2024-04-25T10:18:10.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.150+0000 7fb87f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e857ed800 0x561e849cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.150+0000 7fb87f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e857ed800 0x561e849cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44300/0 (socket says 172.21.15.119:44300) 2024-04-25T10:18:10.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.150+0000 7fb87f200640 1 -- 172.21.15.119:0/276037073 learned_addr learned my addr 172.21.15.119:0/276037073 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:10.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.150+0000 7fb87f200640 1 -- 172.21.15.119:0/276037073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561e857ed000 msgr2=0x561e849cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.150+0000 7fb87f200640 1 --2- 172.21.15.119:0/276037073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561e857ed000 0x561e849cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.150+0000 7fb87f200640 1 -- 172.21.15.119:0/276037073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e857ed400 msgr2=0x561e849cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.150+0000 7fb87f200640 1 --2- 172.21.15.119:0/276037073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e857ed400 0x561e849cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.150+0000 7fb87f200640 1 -- 172.21.15.119:0/276037073 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e849574a0 con 0x561e857ed800 2024-04-25T10:18:10.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.150+0000 7fb87f200640 1 --2- 172.21.15.119:0/276037073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e857ed800 0x561e849cc100 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x561e8580aa80 tx=0x561e85812c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6346882bcb514a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.151+0000 7fb87a200640 1 -- 172.21.15.119:0/276037073 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561e84950680 con 0x561e857ed800 2024-04-25T10:18:10.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.151+0000 7fb87a200640 1 -- 172.21.15.119:0/276037073 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e84974540 con 0x561e857ed800 2024-04-25T10:18:10.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.151+0000 7fb87a200640 1 -- 172.21.15.119:0/276037073 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561e84950b60 con 0x561e857ed800 2024-04-25T10:18:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.151+0000 7fb882322740 1 -- 172.21.15.119:0/276037073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e857ed800 msgr2=0x561e849cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.151+0000 7fb882322740 1 --2- 172.21.15.119:0/276037073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e857ed800 0x561e849cc100 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x561e8580aa80 tx=0x561e85812c00 comp rx=0 tx=0).stop 2024-04-25T10:18:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.151+0000 7fb882322740 1 -- 172.21.15.119:0/276037073 shutdown_connections 2024-04-25T10:18:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.151+0000 7fb882322740 1 --2- 172.21.15.119:0/276037073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e857ed800 0x561e849cc100 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.151+0000 7fb882322740 1 --2- 172.21.15.119:0/276037073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e857ed400 0x561e849cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.151+0000 7fb882322740 1 --2- 172.21.15.119:0/276037073 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561e857ed000 0x561e849cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.151+0000 7fb882322740 1 -- 172.21.15.119:0/276037073 >> 172.21.15.119:0/276037073 conn(0x561e857ecc00 msgr2=0x561e856a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.152+0000 7fb882322740 1 -- 172.21.15.119:0/276037073 shutdown_connections 2024-04-25T10:18:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.152+0000 7fb882322740 1 -- 172.21.15.119:0/276037073 wait complete. 2024-04-25T10:18:10.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.152+0000 7fb882322740 1 Processor -- start 2024-04-25T10:18:10.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.152+0000 7fb882322740 1 -- start start 2024-04-25T10:18:10.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.153+0000 7fb882322740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e857ed000 0x561e849cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.153+0000 7fb882322740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e857ed400 0x561e849ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.153+0000 7fb882322740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561e857ed800 0x561e849cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.153+0000 7fb882322740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e857e5c80 con 0x561e857ed400 2024-04-25T10:18:10.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.153+0000 7fb882322740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e857e5b00 con 0x561e857ed000 2024-04-25T10:18:10.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.153+0000 7fb882322740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e857e5980 con 0x561e857ed800 2024-04-25T10:18:10.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.153+0000 7fb87fc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e857ed400 0x561e849ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.153+0000 7fb87fc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e857ed400 0x561e849ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44308/0 (socket says 172.21.15.119:44308) 2024-04-25T10:18:10.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.153+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 learned_addr learned my addr 172.21.15.119:0/1988324374 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:10.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.154+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561e857ed800 msgr2=0x561e849cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.154+0000 7fb87fc00640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561e857ed800 0x561e849cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.154+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e857ed000 msgr2=0x561e849cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.154+0000 7fb87fc00640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e857ed000 0x561e849cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.154+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e84957680 con 0x561e857ed400 2024-04-25T10:18:10.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.154+0000 7fb87fc00640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e857ed400 0x561e849ccc00 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x561e8580bd40 tx=0x561e856bb5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.154+0000 7fb878400640 1 -- 172.21.15.119:0/1988324374 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561e84950820 con 0x561e857ed400 2024-04-25T10:18:10.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.154+0000 7fb878400640 1 -- 172.21.15.119:0/1988324374 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e84974c40 con 0x561e857ed400 2024-04-25T10:18:10.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.154+0000 7fb878400640 1 -- 172.21.15.119:0/1988324374 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561e84950680 con 0x561e857ed400 2024-04-25T10:18:10.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.154+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561e84957860 con 0x561e857ed400 2024-04-25T10:18:10.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.155+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e849570e0 con 0x561e857ed400 2024-04-25T10:18:10.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.155+0000 7fb882322740 4 datascan.init: resolving metadata pool 16 2024-04-25T10:18:10.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.156+0000 7fb878400640 1 -- 172.21.15.119:0/1988324374 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x561e85868c00 con 0x561e857ed400 2024-04-25T10:18:10.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.157+0000 7fb878400640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x561e85861800 0x561e849cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.157+0000 7fb878400640 1 -- 172.21.15.119:0/1988324374 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x561e85862c00 con 0x561e857ed400 2024-04-25T10:18:10.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.157+0000 7fb882322740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:18:10.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.157+0000 7fb882322740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-25T10:18:10.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.157+0000 7fb882322740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:18:10.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.157+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x561e85861c00 0x561e849cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.157+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c98000 con 0x561e85861c00 2024-04-25T10:18:10.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.157+0000 7fb880600640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x561e85861800 0x561e849cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.157+0000 7fb87f200640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x561e85861c00 0x561e849cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.158+0000 7fb880600640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x561e85861800 0x561e849cd700 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x561e856bc780 tx=0x561e857f8180 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.158+0000 7fb87f200640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x561e85861c00 0x561e849cc100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.158+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv35 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.158+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x561e84c98400 0x561e849cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.158+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c98800 con 0x561e84c98400 2024-04-25T10:18:10.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.159+0000 7fb880600640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x561e84c98400 0x561e849cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.160+0000 7fb880600640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x561e84c98400 0x561e849cbb80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.161+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.5 v2:172.21.15.132:6815/2978152964 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98400 2024-04-25T10:18:10.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.161+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x561e84c98c00 0x561e849cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.161+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99000 con 0x561e84c98c00 2024-04-25T10:18:10.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.162+0000 7fb87fc00640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x561e84c98c00 0x561e849cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.163+0000 7fb87fc00640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x561e84c98c00 0x561e849cb600 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.164+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.164+0000 7fb879800640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x561e857edc00 0x561e84cb0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.164+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85860400 con 0x561e857edc00 2024-04-25T10:18:10.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.164+0000 7fb87f200640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x561e857edc00 0x561e84cb0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.165+0000 7fb87f200640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x561e857edc00 0x561e84cb0000 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.165+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e857edc00 2024-04-25T10:18:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.165+0000 7fb87a200640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x561e84c98000 0x561e84cb0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.165+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:5 17.10 17:08000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99800 con 0x561e84c98000 2024-04-25T10:18:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.165+0000 7fb880600640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x561e84c98000 0x561e84cb0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.166+0000 7fb880600640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x561e84c98000 0x561e84cb0580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.166+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.166+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:6 17.30 17:0c000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85860000 con 0x561e85861c00 2024-04-25T10:18:10.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.167+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.167+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:7 17.8 17:10000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99c00 con 0x561e84c98c00 2024-04-25T10:18:10.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.168+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.168+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:8 17.28 17:14000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb8000 con 0x561e84c98000 2024-04-25T10:18:10.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.168+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.168+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:9 17.18 17:18000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c98800 con 0x561e85861c00 2024-04-25T10:18:10.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.168+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 112 out=49b] v110'17 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.168+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb8400 con 0x561e84c98c00 2024-04-25T10:18:10.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.169+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.169+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85705400 con 0x561e85861c00 2024-04-25T10:18:10.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.169+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.169+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb8800 con 0x561e84c98000 2024-04-25T10:18:10.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.170+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 112 out=49b] v110'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.170+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85705000 con 0x561e84c98000 2024-04-25T10:18:10.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.170+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 4 ==== osd_op_reply(13 [pgnls-filter start_epoch 112 out=49b] v110'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.170+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99000 con 0x561e85861c00 2024-04-25T10:18:10.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.171+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.171+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c98800 con 0x561e84c98c00 2024-04-25T10:18:10.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.171+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 4 ==== osd_op_reply(15 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.172+0000 7fb879800640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x561e85861400 0x561e84cb0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.172+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85861000 con 0x561e85861400 2024-04-25T10:18:10.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.172+0000 7fb87fc00640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x561e85861400 0x561e84cb0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.173+0000 7fb87fc00640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x561e85861400 0x561e84cb0b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.174+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.4 v2:172.21.15.132:6800/1740646323 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e85861400 2024-04-25T10:18:10.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.174+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85705400 con 0x561e857edc00 2024-04-25T10:18:10.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.175+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.1 v2:172.21.15.119:6800/298545098 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e857edc00 2024-04-25T10:18:10.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.175+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85860c00 con 0x561e85861c00 2024-04-25T10:18:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.175+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 6 ==== osd_op_reply(18 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.175+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85860400 con 0x561e85861400 2024-04-25T10:18:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.176+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.4 v2:172.21.15.132:6800/1740646323 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e85861400 2024-04-25T10:18:10.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.176+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb8400 con 0x561e84c98400 2024-04-25T10:18:10.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.177+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.5 v2:172.21.15.132:6815/2978152964 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98400 2024-04-25T10:18:10.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.177+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls-filter start_epoch 112 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85860000 con 0x561e84c98000 2024-04-25T10:18:10.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.178+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 112 out=81b] v112'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.178+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:22 17.0 17:00000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99400 con 0x561e84c98c00 2024-04-25T10:18:10.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.179+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.179+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:23 17.20 17:04000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb8c00 con 0x561e857edc00 2024-04-25T10:18:10.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.179+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.1 v2:172.21.15.119:6800/298545098 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e857edc00 2024-04-25T10:18:10.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.179+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:24 17.10 17:08000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb8000 con 0x561e84c98000 2024-04-25T10:18:10.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.180+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.180+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:25 17.30 17:0c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb9000 con 0x561e85861c00 2024-04-25T10:18:10.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.180+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.180+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:26 17.8 17:10000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99000 con 0x561e84c98c00 2024-04-25T10:18:10.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.181+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 6 ==== osd_op_reply(26 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.181+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:27 17.28 17:14000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb9400 con 0x561e84c98000 2024-04-25T10:18:10.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.182+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 7 ==== osd_op_reply(27 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.182+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:28 17.18 17:18000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99c00 con 0x561e85861c00 2024-04-25T10:18:10.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.183+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 8 ==== osd_op_reply(28 [pgnls-filter start_epoch 112 out=49b] v110'17 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.183+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:29 17.38 17:1c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb9800 con 0x561e84c98c00 2024-04-25T10:18:10.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.184+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 7 ==== osd_op_reply(29 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.184+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:30 17.4 17:20000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99800 con 0x561e85861c00 2024-04-25T10:18:10.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.184+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 9 ==== osd_op_reply(30 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.184+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:31 17.24 17:24000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb9c00 con 0x561e84c98000 2024-04-25T10:18:10.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.185+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 8 ==== osd_op_reply(31 [pgnls-filter start_epoch 112 out=49b] v110'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.185+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:32 17.14 17:28000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85860c00 con 0x561e84c98000 2024-04-25T10:18:10.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.185+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 9 ==== osd_op_reply(32 [pgnls-filter start_epoch 112 out=49b] v110'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.185+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:33 17.34 17:2c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb8c00 con 0x561e85861c00 2024-04-25T10:18:10.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.186+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 10 ==== osd_op_reply(33 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.186+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:34 17.c 17:30000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85860000 con 0x561e84c98c00 2024-04-25T10:18:10.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.187+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 8 ==== osd_op_reply(34 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.187+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:35 17.2c 17:34000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb9000 con 0x561e85861400 2024-04-25T10:18:10.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.188+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.4 v2:172.21.15.132:6800/1740646323 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e85861400 2024-04-25T10:18:10.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.188+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:36 17.1c 17:38000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85705400 con 0x561e857edc00 2024-04-25T10:18:10.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.188+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.1 v2:172.21.15.119:6800/298545098 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e857edc00 2024-04-25T10:18:10.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.188+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:37 17.3c 17:3c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb8800 con 0x561e85861c00 2024-04-25T10:18:10.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.189+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 11 ==== osd_op_reply(37 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.189+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:38 17.2 17:40000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85705000 con 0x561e85861400 2024-04-25T10:18:10.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.190+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.4 v2:172.21.15.132:6800/1740646323 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e85861400 2024-04-25T10:18:10.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.190+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:39 17.22 17:44000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb8400 con 0x561e84c98400 2024-04-25T10:18:10.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.191+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.5 v2:172.21.15.132:6815/2978152964 3 ==== osd_op_reply(39 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98400 2024-04-25T10:18:10.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.191+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:40 17.12 17:48000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99c00 con 0x561e84c98000 2024-04-25T10:18:10.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.192+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 112 out=49b] v112'23 uv23 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.192+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:41 17.32 17:4c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb8000 con 0x561e84c98c00 2024-04-25T10:18:10.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.193+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 9 ==== osd_op_reply(41 [pgnls-filter start_epoch 112 out=81b] v112'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.193+0000 7fb882322740 20 datascan.forall_objects: OSD matched oid 10000000003.00000000 2024-04-25T10:18:10.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.193+0000 7fb882322740 10 datascan.operator(): handling object 10000000003.0 2024-04-25T10:18:10.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.193+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:42 17.32 17:4def72d4:::10000000003.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85860400 con 0x561e84c98c00 2024-04-25T10:18:10.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.195+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 10 ==== osd_op_reply(42 10000000003.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+194 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.195+0000 7fb882322740 10 datascan.inject_with_backtrace: inode: 0x0x10000000003 2024-04-25T10:18:10.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.195+0000 7fb882322740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-25T10:18:10.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.195+0000 7fb882322740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-25T10:18:10.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.195+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:43 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85860000 con 0x561e84c98000 2024-04-25T10:18:10.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.195+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 11 ==== osd_op_reply(43 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.195+0000 7fb882322740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-25T10:18:10.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.196+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x561e84cb9800 0x561e84cb1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.196+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:44 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c98800 con 0x561e84cb9800 2024-04-25T10:18:10.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.196+0000 7fb87f200640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x561e84cb9800 0x561e84cb1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.197+0000 7fb87f200640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x561e84cb9800 0x561e84cb1600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.197+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.2 v2:172.21.15.119:6824/1001762963 1 ==== osd_op_reply(44 10000000000.00000000 [omap-get-vals-by-keys out=585b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+585 (crc 0 0 0) 0x561e856a8000 con 0x561e84cb9800 2024-04-25T10:18:10.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.197+0000 7fb882322740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-25T10:18:10.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.198+0000 7fb882322740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T10:18:10.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.198+0000 7fb882322740 20 datascan.get_frag_of: frag is * 2024-04-25T10:18:10.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.198+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:45 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99400 con 0x561e84c98000 2024-04-25T10:18:10.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.198+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 12 ==== osd_op_reply(45 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.198+0000 7fb882322740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-04-25T10:18:10.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.198+0000 7fb882322740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-25T10:18:10.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.198+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99000 con 0x561e84c98000 2024-04-25T10:18:10.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.199+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 13 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.199+0000 7fb882322740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T10:18:10.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.199+0000 7fb882322740 20 datascan.read_dentry: bravo_head not found in result 2024-04-25T10:18:10.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.199+0000 7fb882322740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000003 at 0x0x10000000001/bravo with size=5 bytes 2024-04-25T10:18:10.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.199+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:47 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99c00 con 0x561e84c98000 2024-04-25T10:18:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.201+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 14 ==== osd_op_reply(47 10000000001.00000000 [omap-set-vals] v112'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.201+0000 7fb882322740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x10000000003 2024-04-25T10:18:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.201+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:48 17.a 17:50000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99400 con 0x561e84c98c00 2024-04-25T10:18:10.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.202+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 11 ==== osd_op_reply(48 [pgnls-filter start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.202+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:49 17.2a 17:54000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb9400 con 0x561e84c98000 2024-04-25T10:18:10.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.202+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 15 ==== osd_op_reply(49 [pgnls-filter start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.202+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:50 17.1a 17:58000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99800 con 0x561e85861c00 2024-04-25T10:18:10.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.203+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 12 ==== osd_op_reply(50 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.203+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:51 17.3a 17:5c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cda000 con 0x561e857edc00 2024-04-25T10:18:10.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.204+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.1 v2:172.21.15.119:6800/298545098 5 ==== osd_op_reply(51 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e857edc00 2024-04-25T10:18:10.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.204+0000 7fb879800640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x561e84cdbc00 0x561e84cb1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.204+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:52 17.6 17:60000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdc000 con 0x561e84cdbc00 2024-04-25T10:18:10.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.205+0000 7fb880600640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x561e84cdbc00 0x561e84cb1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.206+0000 7fb880600640 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x561e84cdbc00 0x561e84cb1080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.206+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.6 v2:172.21.15.132:6806/353653608 1 ==== osd_op_reply(52 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84cdbc00 2024-04-25T10:18:10.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.207+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:53 17.26 17:64000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cda400 con 0x561e85861c00 2024-04-25T10:18:10.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.207+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 13 ==== osd_op_reply(53 [pgnls-filter start_epoch 112 out=49b] v110'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.207+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:54 17.16 17:68000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdc400 con 0x561e84c98400 2024-04-25T10:18:10.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.208+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.5 v2:172.21.15.132:6815/2978152964 4 ==== osd_op_reply(54 [pgnls-filter start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98400 2024-04-25T10:18:10.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.208+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:55 17.36 17:6c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cda800 con 0x561e85861c00 2024-04-25T10:18:10.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.209+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 14 ==== osd_op_reply(55 [pgnls-filter start_epoch 112 out=49b] v110'32 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.209+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:56 17.e 17:70000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdc800 con 0x561e84cb9800 2024-04-25T10:18:10.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.209+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.2 v2:172.21.15.119:6824/1001762963 2 ==== osd_op_reply(56 [pgnls-filter start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e84cb9800 2024-04-25T10:18:10.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.209+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:57 17.2e 17:74000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdac00 con 0x561e84cdbc00 2024-04-25T10:18:10.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.210+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.6 v2:172.21.15.132:6806/353653608 2 ==== osd_op_reply(57 [pgnls-filter start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84cdbc00 2024-04-25T10:18:10.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.210+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:58 17.1e 17:78000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdcc00 con 0x561e857edc00 2024-04-25T10:18:10.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.211+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.1 v2:172.21.15.119:6800/298545098 6 ==== osd_op_reply(58 [pgnls-filter start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e857edc00 2024-04-25T10:18:10.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.211+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:59 17.3e 17:7c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdb000 con 0x561e85861400 2024-04-25T10:18:10.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.212+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.4 v2:172.21.15.132:6800/1740646323 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 112 out=49b] v110'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e85861400 2024-04-25T10:18:10.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.212+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:60 17.1 17:80000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdd000 con 0x561e85861c00 2024-04-25T10:18:10.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.213+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 15 ==== osd_op_reply(60 [pgnls-filter start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.213+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:61 17.21 17:84000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdb400 con 0x561e85861c00 2024-04-25T10:18:10.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.214+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 16 ==== osd_op_reply(61 [pgnls-filter start_epoch 112 out=49b] v110'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.214+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:62 17.11 17:88000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdd400 con 0x561e84c98000 2024-04-25T10:18:10.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.214+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 16 ==== osd_op_reply(62 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.215+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:63 17.31 17:8c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdb800 con 0x561e857edc00 2024-04-25T10:18:10.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.215+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.1 v2:172.21.15.119:6800/298545098 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e857edc00 2024-04-25T10:18:10.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.215+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:64 17.9 17:90000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdd800 con 0x561e84c98000 2024-04-25T10:18:10.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.216+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 17 ==== osd_op_reply(64 [pgnls-filter start_epoch 112 out=49b] v110'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.216+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:65 17.29 17:94000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cda000 con 0x561e85861400 2024-04-25T10:18:10.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.217+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.4 v2:172.21.15.132:6800/1740646323 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e85861400 2024-04-25T10:18:10.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.217+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:66 17.19 17:98000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cddc00 con 0x561e84c98400 2024-04-25T10:18:10.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.218+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.5 v2:172.21.15.132:6815/2978152964 5 ==== osd_op_reply(66 [pgnls-filter start_epoch 112 out=49b] v110'21 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98400 2024-04-25T10:18:10.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.218+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:67 17.39 17:9c000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cda400 con 0x561e84c98400 2024-04-25T10:18:10.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.219+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.5 v2:172.21.15.132:6815/2978152964 6 ==== osd_op_reply(67 [pgnls-filter start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98400 2024-04-25T10:18:10.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.220+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb9400 con 0x561e84cdbc00 2024-04-25T10:18:10.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.222+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.6 v2:172.21.15.132:6806/353653608 3 ==== osd_op_reply(68 [pgnls-filter start_epoch 112 out=49b] v112'12 uv12 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84cdbc00 2024-04-25T10:18:10.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.222+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:69 17.25 17:a4000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c98800 con 0x561e857edc00 2024-04-25T10:18:10.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.222+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.1 v2:172.21.15.119:6800/298545098 8 ==== osd_op_reply(69 [pgnls-filter start_epoch 112 out=49b] v112'17 uv17 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e857edc00 2024-04-25T10:18:10.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.223+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:70 17.15 17:a8000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb9000 con 0x561e84c98400 2024-04-25T10:18:10.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.223+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.5 v2:172.21.15.132:6815/2978152964 7 ==== osd_op_reply(70 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98400 2024-04-25T10:18:10.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.224+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:71 17.35 17:ac000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99800 con 0x561e85861400 2024-04-25T10:18:10.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.225+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.4 v2:172.21.15.132:6800/1740646323 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e85861400 2024-04-25T10:18:10.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.225+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:72 17.d 17:b0000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb8000 con 0x561e84c98c00 2024-04-25T10:18:10.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.226+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 12 ==== osd_op_reply(72 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.226+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:73 17.2d 17:b4000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb8c00 con 0x561e84c98c00 2024-04-25T10:18:10.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.227+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 13 ==== osd_op_reply(73 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.227+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:74 17.1d 17:b8000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85705000 con 0x561e84c98c00 2024-04-25T10:18:10.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.228+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 14 ==== osd_op_reply(74 [pgnls-filter start_epoch 112 out=49b] v110'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.228+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:75 17.3d 17:bc000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cb8800 con 0x561e84c98c00 2024-04-25T10:18:10.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.230+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 15 ==== osd_op_reply(75 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.230+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:76 17.3 17:c0000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdc000 con 0x561e85861400 2024-04-25T10:18:10.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.231+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.4 v2:172.21.15.132:6800/1740646323 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e85861400 2024-04-25T10:18:10.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.231+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:77 17.23 17:c4000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85705400 con 0x561e85861400 2024-04-25T10:18:10.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.232+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.4 v2:172.21.15.132:6800/1740646323 9 ==== osd_op_reply(77 [pgnls-filter start_epoch 112 out=49b] v110'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e85861400 2024-04-25T10:18:10.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.232+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:78 17.13 17:c8000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdc400 con 0x561e84c98000 2024-04-25T10:18:10.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.233+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.3 v2:172.21.15.119:6816/1142971786 18 ==== osd_op_reply(78 [pgnls-filter start_epoch 112 out=49b] v110'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98000 2024-04-25T10:18:10.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.233+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:79 17.33 17:cc000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdac00 con 0x561e84c98c00 2024-04-25T10:18:10.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.234+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.7 v2:172.21.15.132:6821/2727385616 16 ==== osd_op_reply(79 [pgnls-filter start_epoch 112 out=49b] v110'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e84c98c00 2024-04-25T10:18:10.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.234+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:80 17.b 17:d0000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdd400 con 0x561e85861c00 2024-04-25T10:18:10.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.235+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 17 ==== osd_op_reply(80 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.235+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:81 17.2b 17:d4000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdb000 con 0x561e85861400 2024-04-25T10:18:10.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.236+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.4 v2:172.21.15.132:6800/1740646323 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 112 out=49b] v110'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e85861400 2024-04-25T10:18:10.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.236+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:82 17.1b 17:d8000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdd800 con 0x561e85861400 2024-04-25T10:18:10.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.238+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.4 v2:172.21.15.132:6800/1740646323 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e85861400 2024-04-25T10:18:10.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.238+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:83 17.3b 17:dc000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cda000 con 0x561e84cb9800 2024-04-25T10:18:10.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.238+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.2 v2:172.21.15.119:6824/1001762963 3 ==== osd_op_reply(83 [pgnls-filter start_epoch 112 out=49b] v110'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e84cb9800 2024-04-25T10:18:10.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.238+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:84 17.7 17:e0000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cddc00 con 0x561e84c98400 2024-04-25T10:18:10.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.239+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.5 v2:172.21.15.132:6815/2978152964 8 ==== osd_op_reply(84 [pgnls-filter start_epoch 112 out=49b] v110'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84c98400 2024-04-25T10:18:10.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.239+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:85 17.27 17:e4000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99c00 con 0x561e85861400 2024-04-25T10:18:10.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.240+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.4 v2:172.21.15.132:6800/1740646323 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 112 out=49b] v110'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e85861400 2024-04-25T10:18:10.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.240+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:86 17.17 17:e8000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdac00 con 0x561e84cdbc00 2024-04-25T10:18:10.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.242+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.6 v2:172.21.15.132:6806/353653608 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84cdbc00 2024-04-25T10:18:10.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.242+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:87 17.37 17:ec000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84c99400 con 0x561e85861c00 2024-04-25T10:18:10.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.243+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 18 ==== osd_op_reply(87 [pgnls-filter start_epoch 112 out=49b] v110'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.243+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:88 17.f 17:f0000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cdb000 con 0x561e857edc00 2024-04-25T10:18:10.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.244+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.1 v2:172.21.15.119:6800/298545098 9 ==== osd_op_reply(88 [pgnls-filter start_epoch 112 out=49b] v110'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e857edc00 2024-04-25T10:18:10.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.244+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:89 17.2f 17:f4000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85860400 con 0x561e85861400 2024-04-25T10:18:10.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.245+0000 7fb87fc00640 1 -- 172.21.15.119:0/1988324374 <== osd.4 v2:172.21.15.132:6800/1740646323 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 112 out=49b] v110'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8780 con 0x561e85861400 2024-04-25T10:18:10.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.245+0000 7fb879800640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:90 17.1f 17:f8000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e84cda800 con 0x561e84cdbc00 2024-04-25T10:18:10.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb880600640 1 -- 172.21.15.119:0/1988324374 <== osd.6 v2:172.21.15.132:6806/353653608 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 112 out=49b] v110'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8500 con 0x561e84cdbc00 2024-04-25T10:18:10.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb87a200640 1 -- 172.21.15.119:0/1988324374 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:91 17.3f 17:fc000000::::head [pgnls-filter start_epoch 112 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x561e85861000 con 0x561e85861c00 2024-04-25T10:18:10.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 <== osd.0 v2:172.21.15.119:6808/2890625333 19 ==== osd_op_reply(91 [pgnls-filter start_epoch 112 out=49b] v110'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e856a8000 con 0x561e85861c00 2024-04-25T10:18:10.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x561e85861c00 msgr2=0x561e849cc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x561e85861c00 0x561e849cc100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x561e857edc00 msgr2=0x561e84cb0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x561e857edc00 0x561e84cb0000 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x561e84cb9800 msgr2=0x561e84cb1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x561e84cb9800 0x561e84cb1600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x561e84c98000 msgr2=0x561e84cb0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x561e84c98000 0x561e84cb0580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x561e85861400 msgr2=0x561e84cb0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x561e85861400 0x561e84cb0b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x561e84c98400 msgr2=0x561e849cbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x561e84c98400 0x561e849cbb80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x561e84cdbc00 msgr2=0x561e84cb1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x561e84cdbc00 0x561e84cb1080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x561e84c98c00 msgr2=0x561e849cb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x561e84c98c00 0x561e849cb600 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 reap_dead start 2024-04-25T10:18:10.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 reap_dead start 2024-04-25T10:18:10.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.247+0000 7fb87f200640 1 -- 172.21.15.119:0/1988324374 reap_dead start 2024-04-25T10:18:10.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.248+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x561e85861800 msgr2=0x561e849cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.248+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x561e85861800 0x561e849cd700 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x561e856bc780 tx=0x561e857f8180 comp rx=0 tx=0).stop 2024-04-25T10:18:10.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.248+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e857ed400 msgr2=0x561e849ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.248+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e857ed400 0x561e849ccc00 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x561e8580bd40 tx=0x561e856bb5c0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.248+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 shutdown_connections 2024-04-25T10:18:10.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.248+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x561e85861800 0x561e849cd700 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.248+0000 7fb882322740 1 --2- 172.21.15.119:0/1988324374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e857ed400 0x561e849ccc00 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.248+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 >> 172.21.15.119:0/1988324374 conn(0x561e857ecc00 msgr2=0x561e856a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.248+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 shutdown_connections 2024-04-25T10:18:10.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.249+0000 7fb882322740 1 -- 172.21.15.119:0/1988324374 wait complete. 2024-04-25T10:18:10.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.249+0000 7fb882322740 1 -- 172.21.15.119:0/147662422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8492ac00 msgr2=0x561e849ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.249+0000 7fb882322740 1 --2- 172.21.15.119:0/147662422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8492ac00 0x561e849ca580 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x561e85737c50 tx=0x561e857f8420 comp rx=0 tx=0).stop 2024-04-25T10:18:10.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.250+0000 7fb882322740 1 -- 172.21.15.119:0/147662422 shutdown_connections 2024-04-25T10:18:10.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.250+0000 7fb882322740 1 --2- 172.21.15.119:0/147662422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x561e8492bc00 0x561e849cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.250+0000 7fb882322740 1 --2- 172.21.15.119:0/147662422 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e8492b800 0x561e849cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.250+0000 7fb882322740 1 --2- 172.21.15.119:0/147662422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e8492ac00 0x561e849ca580 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.250+0000 7fb882322740 1 -- 172.21.15.119:0/147662422 >> 172.21.15.119:0/147662422 conn(0x561e8492b000 msgr2=0x561e856a6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.250+0000 7fb882322740 1 -- 172.21.15.119:0/147662422 shutdown_connections 2024-04-25T10:18:10.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.250+0000 7fb882322740 1 -- 172.21.15.119:0/147662422 wait complete. 2024-04-25T10:18:10.265 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.228744, result: 2024-04-25T10:18:10.266 DEBUG:teuthology.orchestra.run.smithi119:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T10:18:10.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.365+0000 7f4d539dc740 1 -- start start 2024-04-25T10:18:10.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.365+0000 7f4d539dc740 1 Processor -- start 2024-04-25T10:18:10.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.366+0000 7f4d539dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7278e6c00 0x55b727986580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.366+0000 7f4d539dc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b7278e7800 0x55b727986b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.366+0000 7f4d539dc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7278e7c00 0x55b727987080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.366+0000 7f4d539dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7286ba180 con 0x55b7278e6c00 2024-04-25T10:18:10.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.366+0000 7f4d539dc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7286ba300 con 0x55b7278e7c00 2024-04-25T10:18:10.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.366+0000 7f4d539dc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7286ba480 con 0x55b7278e7800 2024-04-25T10:18:10.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.366+0000 7f4d51a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7278e6c00 0x55b727986580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.366+0000 7f4d51a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7278e6c00 0x55b727986580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44318/0 (socket says 172.21.15.119:44318) 2024-04-25T10:18:10.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.366+0000 7f4d51a00640 1 -- 172.21.15.119:0/1935336689 learned_addr learned my addr 172.21.15.119:0/1935336689 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.367+0000 7f4d52400640 1 --2- 172.21.15.119:0/1935336689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7278e7c00 0x55b727987080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.367+0000 7f4d51a00640 1 -- 172.21.15.119:0/1935336689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b7278e7800 msgr2=0x55b727986b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.367+0000 7f4d51a00640 1 --2- 172.21.15.119:0/1935336689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b7278e7800 0x55b727986b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.367+0000 7f4d51a00640 1 -- 172.21.15.119:0/1935336689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7278e7c00 msgr2=0x55b727987080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.367+0000 7f4d51a00640 1 --2- 172.21.15.119:0/1935336689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7278e7c00 0x55b727987080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.367+0000 7f4d51a00640 1 -- 172.21.15.119:0/1935336689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b727913680 con 0x55b7278e6c00 2024-04-25T10:18:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.367+0000 7f4d52400640 1 --2- 172.21.15.119:0/1935336689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7278e7c00 0x55b727987080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.367+0000 7f4d51a00640 1 --2- 172.21.15.119:0/1935336689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7278e6c00 0x55b727986580 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x55b7286fbc80 tx=0x55b7287a2120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8257e70aa8dcdde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.367+0000 7f4d4e800640 1 -- 172.21.15.119:0/1935336689 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b72790c680 con 0x55b7278e6c00 2024-04-25T10:18:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.367+0000 7f4d4e800640 1 -- 172.21.15.119:0/1935336689 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b727930540 con 0x55b7278e6c00 2024-04-25T10:18:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.367+0000 7f4d4e800640 1 -- 172.21.15.119:0/1935336689 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b72790cb60 con 0x55b7278e6c00 2024-04-25T10:18:10.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.367+0000 7f4d539dc740 1 -- 172.21.15.119:0/1935336689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7279134a0 con 0x55b7278e6c00 2024-04-25T10:18:10.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.367+0000 7f4d539dc740 1 -- 172.21.15.119:0/1935336689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b727913a40 con 0x55b7278e6c00 2024-04-25T10:18:10.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.367+0000 7f4d539dc740 4 waiting for MDS map... 2024-04-25T10:18:10.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.368+0000 7f4d4e800640 1 -- 172.21.15.119:0/1935336689 <== mon.0 v2:172.21.15.119:3300/0 4 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x55b7287ae600 con 0x55b7278e6c00 2024-04-25T10:18:10.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.368+0000 7f4d4e800640 1 -- 172.21.15.119:0/1935336689 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x55b72793f180 con 0x55b7278e6c00 2024-04-25T10:18:10.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.368+0000 7f4d539dc740 4 Got MDS map 106 2024-04-25T10:18:10.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.368+0000 7f4d539dc740 4 datascan.main: Using metadata pool output 2024-04-25T10:18:10.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.368+0000 7f4d539dc740 4 datascan.main: connecting to RADOS... 2024-04-25T10:18:10.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.369+0000 7f4d539dc740 1 Processor -- start 2024-04-25T10:18:10.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.369+0000 7f4d539dc740 1 -- start start 2024-04-25T10:18:10.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.370+0000 7f4d539dc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b7287b1000 0x55b727987600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.370+0000 7f4d539dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7287b1400 0x55b727987b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.370+0000 7f4d539dc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7287b1800 0x55b727988100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.370+0000 7f4d539dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7286ba600 con 0x55b7287b1400 2024-04-25T10:18:10.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.370+0000 7f4d539dc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7286ba780 con 0x55b7287b1800 2024-04-25T10:18:10.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.370+0000 7f4d539dc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7286ba900 con 0x55b7287b1000 2024-04-25T10:18:10.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.370+0000 7f4d51a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7287b1400 0x55b727987b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.370+0000 7f4d51a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7287b1400 0x55b727987b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44330/0 (socket says 172.21.15.119:44330) 2024-04-25T10:18:10.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.370+0000 7f4d51a00640 1 -- 172.21.15.119:0/4014160856 learned_addr learned my addr 172.21.15.119:0/4014160856 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:10.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.370+0000 7f4d51000640 1 --2- 172.21.15.119:0/4014160856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7287b1800 0x55b727988100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.370+0000 7f4d51a00640 1 -- 172.21.15.119:0/4014160856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b7287b1000 msgr2=0x55b727987600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.370+0000 7f4d51a00640 1 --2- 172.21.15.119:0/4014160856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b7287b1000 0x55b727987600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.371+0000 7f4d51a00640 1 -- 172.21.15.119:0/4014160856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7287b1800 msgr2=0x55b727988100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.371+0000 7f4d51a00640 1 --2- 172.21.15.119:0/4014160856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7287b1800 0x55b727988100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.371+0000 7f4d51a00640 1 -- 172.21.15.119:0/4014160856 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b727913860 con 0x55b7287b1400 2024-04-25T10:18:10.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.371+0000 7f4d51000640 1 --2- 172.21.15.119:0/4014160856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7287b1800 0x55b727988100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:10.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.371+0000 7f4d51a00640 1 --2- 172.21.15.119:0/4014160856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7287b1400 0x55b727987b80 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x55b7287ad110 tx=0x55b7287a2060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59130f0b00365c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.371+0000 7f4d4c000640 1 -- 172.21.15.119:0/4014160856 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b72790c820 con 0x55b7287b1400 2024-04-25T10:18:10.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.371+0000 7f4d4c000640 1 -- 172.21.15.119:0/4014160856 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b727930e00 con 0x55b7287b1400 2024-04-25T10:18:10.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.371+0000 7f4d4c000640 1 -- 172.21.15.119:0/4014160856 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b72790c680 con 0x55b7287b1400 2024-04-25T10:18:10.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.372+0000 7f4d539dc740 1 -- 172.21.15.119:0/4014160856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7287b1400 msgr2=0x55b727987b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.372+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4014160856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7287b1400 0x55b727987b80 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x55b7287ad110 tx=0x55b7287a2060 comp rx=0 tx=0).stop 2024-04-25T10:18:10.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.372+0000 7f4d539dc740 1 -- 172.21.15.119:0/4014160856 shutdown_connections 2024-04-25T10:18:10.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.372+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4014160856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7287b1800 0x55b727988100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.372+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4014160856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7287b1400 0x55b727987b80 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.372+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4014160856 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b7287b1000 0x55b727987600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.372+0000 7f4d539dc740 1 -- 172.21.15.119:0/4014160856 >> 172.21.15.119:0/4014160856 conn(0x55b7287b0c00 msgr2=0x55b7279d7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.372+0000 7f4d539dc740 1 -- 172.21.15.119:0/4014160856 shutdown_connections 2024-04-25T10:18:10.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.372+0000 7f4d539dc740 1 -- 172.21.15.119:0/4014160856 wait complete. 2024-04-25T10:18:10.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.373+0000 7f4d539dc740 1 Processor -- start 2024-04-25T10:18:10.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.373+0000 7f4d539dc740 1 -- start start 2024-04-25T10:18:10.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.373+0000 7f4d539dc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b7287b1000 0x55b727989180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.373+0000 7f4d539dc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7287b1400 0x55b727988c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.373+0000 7f4d539dc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7287b1800 0x55b727988680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.373+0000 7f4d539dc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7286bbc80 con 0x55b7287b1400 2024-04-25T10:18:10.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.373+0000 7f4d539dc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7286bbb00 con 0x55b7287b1800 2024-04-25T10:18:10.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.373+0000 7f4d539dc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7286bb980 con 0x55b7287b1000 2024-04-25T10:18:10.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.374+0000 7f4d51a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7287b1400 0x55b727988c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.374+0000 7f4d51a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7287b1400 0x55b727988c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44344/0 (socket says 172.21.15.119:44344) 2024-04-25T10:18:10.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.374+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 learned_addr learned my addr 172.21.15.119:0/4120481699 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:10.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.374+0000 7f4d51000640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7287b1800 0x55b727988680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.374+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b7287b1000 msgr2=0x55b727989180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.374+0000 7f4d51a00640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b7287b1000 0x55b727989180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.374+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7287b1800 msgr2=0x55b727988680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.374+0000 7f4d51a00640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7287b1800 0x55b727988680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.374+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b727913680 con 0x55b7287b1400 2024-04-25T10:18:10.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.374+0000 7f4d51000640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7287b1800 0x55b727988680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:10.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.374+0000 7f4d51a00640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7287b1400 0x55b727988c00 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x55b7287ad380 tx=0x55b728820300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.375+0000 7f4d4a200640 1 -- 172.21.15.119:0/4120481699 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b72790cb60 con 0x55b7287b1400 2024-04-25T10:18:10.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.375+0000 7f4d4a200640 1 -- 172.21.15.119:0/4120481699 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b727930c40 con 0x55b7287b1400 2024-04-25T10:18:10.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.375+0000 7f4d4a200640 1 -- 172.21.15.119:0/4120481699 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b72790c4e0 con 0x55b7287b1400 2024-04-25T10:18:10.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.375+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b7279134a0 con 0x55b7287b1400 2024-04-25T10:18:10.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.375+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b727913a40 con 0x55b7287b1400 2024-04-25T10:18:10.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.375+0000 7f4d539dc740 4 datascan.init: resolving metadata pool 16 2024-04-25T10:18:10.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.377+0000 7f4d4a200640 1 -- 172.21.15.119:0/4120481699 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55b728822c00 con 0x55b7287b1400 2024-04-25T10:18:10.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.377+0000 7f4d4a200640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x55b727c50000 0x55b727989700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.377+0000 7f4d4a200640 1 -- 172.21.15.119:0/4120481699 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x55b728826600 con 0x55b7287b1400 2024-04-25T10:18:10.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.377+0000 7f4d539dc740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:18:10.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.377+0000 7f4d539dc740 4 datascan.main: resolving metadata pool 16 2024-04-25T10:18:10.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.377+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x55b727c50400 0x55b727988100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.377+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c50800 con 0x55b727c50400 2024-04-25T10:18:10.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.378+0000 7f4d52400640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x55b727c50000 0x55b727989700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.378+0000 7f4d51000640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x55b727c50400 0x55b727988100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.378+0000 7f4d52400640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x55b727c50000 0x55b727989700 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x55b728678e40 tx=0x55b728820780 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.378+0000 7f4d51000640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x55b727c50400 0x55b727988100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.379+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 1 ==== osd_op_reply(1 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.379+0000 7f4d51000640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55b727c50800 0x55b727c60000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.379+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c1400 con 0x55b727c50800 2024-04-25T10:18:10.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.379+0000 7f4d52400640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55b727c50800 0x55b727c60000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.380+0000 7f4d52400640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55b727c50800 0x55b727c60000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.380+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 1 ==== osd_op_reply(2 [pgnls start_epoch 112 out=177b] v111'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.380+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c1400 con 0x55b727c50400 2024-04-25T10:18:10.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.380+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 2 ==== osd_op_reply(3 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.381+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c1400 con 0x55b727c50400 2024-04-25T10:18:10.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.381+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 3 ==== osd_op_reply(4 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.381+0000 7f4d51000640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55b7286c1400 0x55b727c60580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.381+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c1000 con 0x55b7286c1400 2024-04-25T10:18:10.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.381+0000 7f4d51a00640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55b7286c1400 0x55b727c60580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.382+0000 7f4d51a00640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55b7286c1400 0x55b727c60580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.382+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.3 v2:172.21.15.119:6816/1142971786 1 ==== osd_op_reply(5 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660780 con 0x55b7286c1400 2024-04-25T10:18:10.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.382+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c1000 con 0x55b727c50400 2024-04-25T10:18:10.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.382+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 4 ==== osd_op_reply(6 [pgnls start_epoch 112 out=149b] v112'42 uv42 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.383+0000 7f4d51000640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55b7286c1000 0x55b727c60b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.383+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0800 con 0x55b7286c1000 2024-04-25T10:18:10.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.383+0000 7f4d51000640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55b7286c1000 0x55b727c60b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.384+0000 7f4d51000640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55b7286c1000 0x55b727c60b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.385+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.5 v2:172.21.15.132:6815/2978152964 1 ==== osd_op_reply(7 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b7286c1000 2024-04-25T10:18:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.385+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0800 con 0x55b727c50800 2024-04-25T10:18:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.385+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 2 ==== osd_op_reply(8 [pgnls start_epoch 112 out=73b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.386+0000 7f4d52400640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x55b7286c0800 0x55b727c61080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.386+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c64000 con 0x55b7286c0800 2024-04-25T10:18:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.386+0000 7f4d52400640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x55b7286c0800 0x55b727c61080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.387+0000 7f4d52400640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x55b7286c0800 0x55b727c61080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.388+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.4 v2:172.21.15.132:6800/1740646323 1 ==== osd_op_reply(9 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660500 con 0x55b7286c0800 2024-04-25T10:18:10.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.388+0000 7f4d52400640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55b727c64000 0x55b727c61600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.388+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c64400 con 0x55b727c64000 2024-04-25T10:18:10.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.388+0000 7f4d51a00640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55b727c64000 0x55b727c61600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.389+0000 7f4d51a00640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55b727c64000 0x55b727c61600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.390+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.6 v2:172.21.15.132:6806/353653608 1 ==== osd_op_reply(10 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660780 con 0x55b727c64000 2024-04-25T10:18:10.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.390+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c64400 con 0x55b727c50800 2024-04-25T10:18:10.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.391+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 3 ==== osd_op_reply(11 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.391+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c64400 con 0x55b7286c1400 2024-04-25T10:18:10.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.391+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.3 v2:172.21.15.119:6816/1142971786 2 ==== osd_op_reply(12 [pgnls start_epoch 112 out=49b] v112'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660780 con 0x55b7286c1400 2024-04-25T10:18:10.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.391+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c64400 con 0x55b727c64000 2024-04-25T10:18:10.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.392+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.6 v2:172.21.15.132:6806/353653608 2 ==== osd_op_reply(13 [pgnls start_epoch 112 out=77b] v111'11 uv11 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55b728660780 con 0x55b727c64000 2024-04-25T10:18:10.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.392+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c64400 con 0x55b7286c1400 2024-04-25T10:18:10.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.392+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.3 v2:172.21.15.119:6816/1142971786 3 ==== osd_op_reply(14 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660780 con 0x55b7286c1400 2024-04-25T10:18:10.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.392+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c64400 con 0x55b727c50800 2024-04-25T10:18:10.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.393+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 4 ==== osd_op_reply(15 [pgnls start_epoch 112 out=73b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.393+0000 7f4d52400640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55b727c64400 0x55b727c61b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.393+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c64800 con 0x55b727c64400 2024-04-25T10:18:10.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.394+0000 7f4d51000640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55b727c64400 0x55b727c61b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.394+0000 7f4d51000640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55b727c64400 0x55b727c61b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.395+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.7 v2:172.21.15.132:6821/2727385616 1 ==== osd_op_reply(16 [pgnls start_epoch 112 out=49b] v112'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b727c64400 2024-04-25T10:18:10.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.395+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c64800 con 0x55b727c50800 2024-04-25T10:18:10.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.396+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 5 ==== osd_op_reply(17 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.396+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c64800 con 0x55b7286c1000 2024-04-25T10:18:10.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.397+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.5 v2:172.21.15.132:6815/2978152964 2 ==== osd_op_reply(18 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b7286c1000 2024-04-25T10:18:10.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.397+0000 7f4d51000640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55b727c64800 0x55b727c62100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.397+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0c00 con 0x55b727c64800 2024-04-25T10:18:10.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.397+0000 7f4d52400640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55b727c64800 0x55b727c62100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.397+0000 7f4d52400640 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55b727c64800 0x55b727c62100 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.398+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== osd_op_reply(19 [pgnls start_epoch 112 out=81b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b728660500 con 0x55b727c64800 2024-04-25T10:18:10.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.398+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0c00 con 0x55b7286c0800 2024-04-25T10:18:10.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.399+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.4 v2:172.21.15.132:6800/1740646323 2 ==== osd_op_reply(20 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660500 con 0x55b7286c0800 2024-04-25T10:18:10.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.399+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0c00 con 0x55b727c50400 2024-04-25T10:18:10.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.399+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 5 ==== osd_op_reply(21 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.399+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0c00 con 0x55b727c50400 2024-04-25T10:18:10.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.400+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 6 ==== osd_op_reply(22 [pgnls start_epoch 112 out=97b] v111'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.400+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0c00 con 0x55b727c64000 2024-04-25T10:18:10.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.401+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.6 v2:172.21.15.132:6806/353653608 3 ==== osd_op_reply(23 [pgnls start_epoch 112 out=74b] v107'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55b728660780 con 0x55b727c64000 2024-04-25T10:18:10.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.401+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0c00 con 0x55b727c50800 2024-04-25T10:18:10.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.402+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 6 ==== osd_op_reply(24 [pgnls start_epoch 112 out=73b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.402+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0c00 con 0x55b7286c1000 2024-04-25T10:18:10.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.403+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.5 v2:172.21.15.132:6815/2978152964 3 ==== osd_op_reply(25 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b7286c1000 2024-04-25T10:18:10.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.403+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0c00 con 0x55b727c64800 2024-04-25T10:18:10.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.403+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.1 v2:172.21.15.119:6800/298545098 2 ==== osd_op_reply(26 [pgnls start_epoch 112 out=98b] v111'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55b728660500 con 0x55b727c64800 2024-04-25T10:18:10.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.403+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0c00 con 0x55b727c64800 2024-04-25T10:18:10.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.403+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.1 v2:172.21.15.119:6800/298545098 3 ==== osd_op_reply(27 [pgnls start_epoch 112 out=76b] v112'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55b728660500 con 0x55b727c64800 2024-04-25T10:18:10.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.403+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0c00 con 0x55b7286c1000 2024-04-25T10:18:10.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.404+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.5 v2:172.21.15.132:6815/2978152964 4 ==== osd_op_reply(28 [pgnls start_epoch 112 out=49b] v112'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b7286c1000 2024-04-25T10:18:10.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.405+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0c00 con 0x55b7286c1400 2024-04-25T10:18:10.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.405+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.3 v2:172.21.15.119:6816/1142971786 4 ==== osd_op_reply(29 [pgnls start_epoch 112 out=105b] v112'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x55b728660780 con 0x55b7286c1400 2024-04-25T10:18:10.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.405+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0c00 con 0x55b727c50800 2024-04-25T10:18:10.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.406+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 7 ==== osd_op_reply(30 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.406+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0c00 con 0x55b727c50400 2024-04-25T10:18:10.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.406+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 7 ==== osd_op_reply(31 [pgnls start_epoch 112 out=73b] v112'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.406+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b7286c0c00 con 0x55b7286c1000 2024-04-25T10:18:10.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.408+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.5 v2:172.21.15.132:6815/2978152964 5 ==== osd_op_reply(32 [pgnls start_epoch 112 out=101b] v112'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55b728660000 con 0x55b7286c1000 2024-04-25T10:18:10.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.408+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T10:18:10.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.408+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c50c00 con 0x55b727c50800 2024-04-25T10:18:10.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.408+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.408+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T10:18:10.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.408+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c51000 con 0x55b727c50800 2024-04-25T10:18:10.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.409+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.409+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T10:18:10.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.409+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c51400 con 0x55b727c50800 2024-04-25T10:18:10.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.409+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1169 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.409+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T10:18:10.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.409+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c64c00 con 0x55b727c50800 2024-04-25T10:18:10.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T10:18:10.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c65000 con 0x55b727c50800 2024-04-25T10:18:10.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T10:18:10.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:18:10.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T10:18:10.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:18:10.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T10:18:10.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:18:10.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T10:18:10.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c65400 con 0x55b727c50400 2024-04-25T10:18:10.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 8 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv37 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T10:18:10.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.410+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c65800 con 0x55b727c50800 2024-04-25T10:18:10.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.411+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.411+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T10:18:10.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.411+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:18:10.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.411+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T10:18:10.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.411+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:40 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c65c00 con 0x55b727c50800 2024-04-25T10:18:10.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.411+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 14 ==== osd_op_reply(40 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.411+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 10000000005.00000000 2024-04-25T10:18:10.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.411+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:41 16.9 16:9258b67d:::10000000005.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c64c00 con 0x55b727c64800 2024-04-25T10:18:10.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.412+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.1 v2:172.21.15.119:6800/298545098 4 ==== osd_op_reply(41 10000000005.00000000 [omap-get-vals out=592b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+592 (crc 0 0 0) 0x55b728660500 con 0x55b727c64800 2024-04-25T10:18:10.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.412+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T10:18:10.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.412+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c65000 con 0x55b727c50400 2024-04-25T10:18:10.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.412+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.412+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T10:18:10.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.412+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:18:10.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.412+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T10:18:10.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.412+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:18:10.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.412+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T10:18:10.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.412+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c50c00 con 0x55b727c50800 2024-04-25T10:18:10.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.413+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.413+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T10:18:10.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.413+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:18:10.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.413+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T10:18:10.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.413+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c51000 con 0x55b727c64800 2024-04-25T10:18:10.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.414+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.1 v2:172.21.15.119:6800/298545098 5 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55b728660500 con 0x55b727c64800 2024-04-25T10:18:10.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.414+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T10:18:10.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.414+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:18:10.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.414+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T10:18:10.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.414+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:45 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c51400 con 0x55b7286c1400 2024-04-25T10:18:10.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.414+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.3 v2:172.21.15.119:6816/1142971786 5 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660780 con 0x55b7286c1400 2024-04-25T10:18:10.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.414+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-25T10:18:10.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.414+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c000 con 0x55b7286c1400 2024-04-25T10:18:10.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.415+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.3 v2:172.21.15.119:6816/1142971786 6 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1683b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1683 (crc 0 0 0) 0x55b728660780 con 0x55b7286c1400 2024-04-25T10:18:10.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.415+0000 7f4d539dc740 20 datascan.scan_links: injected ino detected 2024-04-25T10:18:10.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.415+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T10:18:10.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.415+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:18:10.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.415+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T10:18:10.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.415+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:18:10.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.415+0000 7f4d539dc740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T10:18:10.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.415+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c400 con 0x55b7286c1000 2024-04-25T10:18:10.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.416+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.5 v2:172.21.15.132:6815/2978152964 6 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+585 (crc 0 0 0) 0x55b728660000 con 0x55b7286c1000 2024-04-25T10:18:10.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.416+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c50400 2024-04-25T10:18:10.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.417+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 10 ==== osd_op_reply(48 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.417+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c50800 2024-04-25T10:18:10.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.417+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 16 ==== osd_op_reply(49 [pgnls start_epoch 112 out=177b] v111'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.417+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c50400 2024-04-25T10:18:10.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.418+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 11 ==== osd_op_reply(50 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.418+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c50400 2024-04-25T10:18:10.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.418+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 12 ==== osd_op_reply(51 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.418+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b7286c1400 2024-04-25T10:18:10.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.418+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.3 v2:172.21.15.119:6816/1142971786 7 ==== osd_op_reply(52 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660780 con 0x55b7286c1400 2024-04-25T10:18:10.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.419+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c50400 2024-04-25T10:18:10.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.419+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 13 ==== osd_op_reply(53 [pgnls start_epoch 112 out=149b] v112'42 uv42 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.419+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b7286c1000 2024-04-25T10:18:10.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.420+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.5 v2:172.21.15.132:6815/2978152964 7 ==== osd_op_reply(54 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b7286c1000 2024-04-25T10:18:10.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.420+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c50800 2024-04-25T10:18:10.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.421+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 17 ==== osd_op_reply(55 [pgnls start_epoch 112 out=73b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.421+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b7286c0800 2024-04-25T10:18:10.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.422+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.4 v2:172.21.15.132:6800/1740646323 3 ==== osd_op_reply(56 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660500 con 0x55b7286c0800 2024-04-25T10:18:10.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.422+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c64000 2024-04-25T10:18:10.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.423+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.6 v2:172.21.15.132:6806/353653608 4 ==== osd_op_reply(57 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660780 con 0x55b727c64000 2024-04-25T10:18:10.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.423+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c50800 2024-04-25T10:18:10.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.424+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 18 ==== osd_op_reply(58 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.424+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b7286c1400 2024-04-25T10:18:10.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.424+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.3 v2:172.21.15.119:6816/1142971786 8 ==== osd_op_reply(59 [pgnls start_epoch 112 out=49b] v112'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660780 con 0x55b7286c1400 2024-04-25T10:18:10.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.424+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c64000 2024-04-25T10:18:10.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.425+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.6 v2:172.21.15.132:6806/353653608 5 ==== osd_op_reply(60 [pgnls start_epoch 112 out=77b] v111'11 uv11 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55b728660780 con 0x55b727c64000 2024-04-25T10:18:10.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.425+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b7286c1400 2024-04-25T10:18:10.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.426+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.3 v2:172.21.15.119:6816/1142971786 9 ==== osd_op_reply(61 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660780 con 0x55b7286c1400 2024-04-25T10:18:10.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.426+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c50800 2024-04-25T10:18:10.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.427+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 19 ==== osd_op_reply(62 [pgnls start_epoch 112 out=73b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.427+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c64400 2024-04-25T10:18:10.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.428+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.7 v2:172.21.15.132:6821/2727385616 2 ==== osd_op_reply(63 [pgnls start_epoch 112 out=49b] v112'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b727c64400 2024-04-25T10:18:10.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.428+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c50800 2024-04-25T10:18:10.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.429+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 20 ==== osd_op_reply(64 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.429+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b7286c1000 2024-04-25T10:18:10.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.429+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.5 v2:172.21.15.132:6815/2978152964 8 ==== osd_op_reply(65 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b7286c1000 2024-04-25T10:18:10.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.429+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c64800 2024-04-25T10:18:10.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.430+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.1 v2:172.21.15.119:6800/298545098 6 ==== osd_op_reply(66 [pgnls start_epoch 112 out=81b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b728660500 con 0x55b727c64800 2024-04-25T10:18:10.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.430+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b7286c0800 2024-04-25T10:18:10.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.431+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.4 v2:172.21.15.132:6800/1740646323 4 ==== osd_op_reply(67 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660500 con 0x55b7286c0800 2024-04-25T10:18:10.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.431+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c50400 2024-04-25T10:18:10.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.432+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 14 ==== osd_op_reply(68 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.432+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c50400 2024-04-25T10:18:10.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.432+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 15 ==== osd_op_reply(69 [pgnls start_epoch 112 out=97b] v111'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.432+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c64000 2024-04-25T10:18:10.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.433+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.6 v2:172.21.15.132:6806/353653608 6 ==== osd_op_reply(70 [pgnls start_epoch 112 out=74b] v107'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55b728660780 con 0x55b727c64000 2024-04-25T10:18:10.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.433+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c50800 2024-04-25T10:18:10.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.434+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 21 ==== osd_op_reply(71 [pgnls start_epoch 112 out=73b] v111'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.434+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b7286c1000 2024-04-25T10:18:10.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.435+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.5 v2:172.21.15.132:6815/2978152964 9 ==== osd_op_reply(72 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b7286c1000 2024-04-25T10:18:10.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.435+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c64800 2024-04-25T10:18:10.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.436+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.1 v2:172.21.15.119:6800/298545098 7 ==== osd_op_reply(73 [pgnls start_epoch 112 out=98b] v111'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55b728660500 con 0x55b727c64800 2024-04-25T10:18:10.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.436+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c64800 2024-04-25T10:18:10.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.437+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.1 v2:172.21.15.119:6800/298545098 8 ==== osd_op_reply(74 [pgnls start_epoch 112 out=76b] v112'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55b728660500 con 0x55b727c64800 2024-04-25T10:18:10.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.437+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b7286c1000 2024-04-25T10:18:10.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.438+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.5 v2:172.21.15.132:6815/2978152964 10 ==== osd_op_reply(75 [pgnls start_epoch 112 out=49b] v112'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660000 con 0x55b7286c1000 2024-04-25T10:18:10.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.438+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b7286c1400 2024-04-25T10:18:10.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.439+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.3 v2:172.21.15.119:6816/1142971786 10 ==== osd_op_reply(76 [pgnls start_epoch 112 out=105b] v112'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x55b728660780 con 0x55b7286c1400 2024-04-25T10:18:10.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.439+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c50800 2024-04-25T10:18:10.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.440+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 22 ==== osd_op_reply(77 [pgnls start_epoch 112 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.440+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b727c50400 2024-04-25T10:18:10.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.441+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 16 ==== osd_op_reply(78 [pgnls start_epoch 112 out=73b] v112'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.441+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 112 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b7286c1000 2024-04-25T10:18:10.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.442+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.5 v2:172.21.15.132:6815/2978152964 11 ==== osd_op_reply(79 [pgnls start_epoch 112 out=101b] v112'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55b728660000 con 0x55b7286c1000 2024-04-25T10:18:10.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.442+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T10:18:10.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.442+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8cc00 con 0x55b727c50800 2024-04-25T10:18:10.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.443+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 23 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.443+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T10:18:10.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.443+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8d000 con 0x55b727c50800 2024-04-25T10:18:10.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.443+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 24 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.443+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T10:18:10.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.443+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8d400 con 0x55b727c50800 2024-04-25T10:18:10.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.444+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 25 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1169 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.444+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T10:18:10.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.444+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8d800 con 0x55b727c50800 2024-04-25T10:18:10.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.445+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 26 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.445+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T10:18:10.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.445+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8dc00 con 0x55b727c50800 2024-04-25T10:18:10.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.445+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 27 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.446+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T10:18:10.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.446+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:18:10.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.446+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T10:18:10.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.446+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:18:10.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.446+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T10:18:10.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.446+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:18:10.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.446+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T10:18:10.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.446+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8cc00 con 0x55b727c50400 2024-04-25T10:18:10.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.446+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 17 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv37 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.446+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T10:18:10.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.446+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8d000 con 0x55b727c50800 2024-04-25T10:18:10.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.447+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 28 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.447+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T10:18:10.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.447+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:18:10.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.447+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T10:18:10.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.447+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:87 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8d400 con 0x55b727c50800 2024-04-25T10:18:10.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.448+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 29 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.448+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 10000000005.00000000 2024-04-25T10:18:10.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.448+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:88 16.9 16:9258b67d:::10000000005.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c51000 con 0x55b727c64800 2024-04-25T10:18:10.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.449+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.1 v2:172.21.15.119:6800/298545098 9 ==== osd_op_reply(88 10000000005.00000000 [omap-get-vals out=592b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+592 (crc 0 0 0) 0x55b728660500 con 0x55b727c64800 2024-04-25T10:18:10.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.449+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T10:18:10.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.449+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c50c00 con 0x55b727c50400 2024-04-25T10:18:10.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.449+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.0 v2:172.21.15.119:6808/2890625333 18 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660000 con 0x55b727c50400 2024-04-25T10:18:10.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.450+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T10:18:10.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.450+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:18:10.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.450+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T10:18:10.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.450+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:18:10.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.450+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T10:18:10.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.450+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c64c00 con 0x55b727c50800 2024-04-25T10:18:10.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.450+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.2 v2:172.21.15.119:6824/1001762963 30 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660500 con 0x55b727c50800 2024-04-25T10:18:10.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.450+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T10:18:10.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.450+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:18:10.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.450+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T10:18:10.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.450+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c65c00 con 0x55b727c64800 2024-04-25T10:18:10.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.451+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.1 v2:172.21.15.119:6800/298545098 10 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55b728660500 con 0x55b727c64800 2024-04-25T10:18:10.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.451+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T10:18:10.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.451+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:18:10.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.451+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T10:18:10.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.451+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:92 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c65800 con 0x55b7286c1400 2024-04-25T10:18:10.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.452+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.3 v2:172.21.15.119:6816/1142971786 11 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b728660780 con 0x55b7286c1400 2024-04-25T10:18:10.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.452+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-25T10:18:10.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.452+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:93 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c50c00 con 0x55b7286c1400 2024-04-25T10:18:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.453+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.3 v2:172.21.15.119:6816/1142971786 12 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1683b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1683 (crc 0 0 0) 0x55b728660780 con 0x55b7286c1400 2024-04-25T10:18:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.453+0000 7f4d539dc740 20 datascan.scan_links: injected ino detected 2024-04-25T10:18:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.453+0000 7f4d539dc740 20 datascan.scan_links: adding 0x10000000003 for future processing to fix dnfirst 2024-04-25T10:18:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.453+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T10:18:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.453+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:18:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.453+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T10:18:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.453+0000 7f4d539dc740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:18:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.453+0000 7f4d539dc740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T10:18:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.453+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c400 con 0x55b7286c1000 2024-04-25T10:18:10.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.455+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 <== osd.5 v2:172.21.15.132:6815/2978152964 12 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+585 (crc 0 0 0) 0x55b728660000 con 0x55b7286c1000 2024-04-25T10:18:10.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.455+0000 7f4d539dc740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-25T10:18:10.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.455+0000 7f4d539dc740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T10:18:10.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.455+0000 7f4d539dc740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-25T10:18:10.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.455+0000 7f4d539dc740 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T10:18:10.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.455+0000 7f4d539dc740 10 datascan.scan_links: handling injected_ino 0x10000000003 2024-04-25T10:18:10.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.455+0000 7f4d539dc740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-25T10:18:10.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.455+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:95 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8c800 con 0x55b7286c1400 2024-04-25T10:18:10.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.456+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.3 v2:172.21.15.119:6816/1142971786 13 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55b728660780 con 0x55b7286c1400 2024-04-25T10:18:10.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.456+0000 7f4d539dc740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T10:18:10.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.456+0000 7f4d539dc740 20 datascan.scan_links: first is now 2 2024-04-25T10:18:10.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.456+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- osd_op(unknown.0.0:96 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c8cc00 con 0x55b7286c1400 2024-04-25T10:18:10.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.459+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.3 v2:172.21.15.119:6816/1142971786 14 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v112'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b728660780 con 0x55b7286c1400 2024-04-25T10:18:10.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.459+0000 7f4d539dc740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x10000000003 2024-04-25T10:18:10.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.459+0000 7f4d539dc740 10 datascan.scan_links: updating inotable 2024-04-25T10:18:10.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.459+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- osd_op(unknown.0.0:97 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c65400 con 0x55b727c64800 2024-04-25T10:18:10.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.460+0000 7f4d52400640 1 -- 172.21.15.119:0/4120481699 <== osd.1 v2:172.21.15.119:6800/298545098 11 ==== osd_op_reply(97 mds0_inotable [read 0~34 out=34b] v0'0 uv4 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x55b728660500 con 0x55b727c64800 2024-04-25T10:18:10.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.460+0000 7f4d539dc740 10 datascan.scan_links: updating snaptable 2024-04-25T10:18:10.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.460+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- osd_op(unknown.0.0:98 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) v9 -- 0x55b727c65000 con 0x55b727c64000 2024-04-25T10:18:10.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.461+0000 7f4d51a00640 1 -- 172.21.15.119:0/4120481699 <== osd.6 v2:172.21.15.132:6806/353653608 7 ==== osd_op_reply(98 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55b728660780 con 0x55b727c64000 2024-04-25T10:18:10.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x55b727c50400 msgr2=0x55b727988100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x55b727c50400 0x55b727988100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55b727c64800 msgr2=0x55b727c62100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x55b727c64800 0x55b727c62100 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55b727c50800 msgr2=0x55b727c60000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x55b727c50800 0x55b727c60000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55b7286c1400 msgr2=0x55b727c60580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x55b7286c1400 0x55b727c60580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x55b7286c0800 msgr2=0x55b727c61080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x55b7286c0800 0x55b727c61080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55b7286c1000 msgr2=0x55b727c60b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x55b7286c1000 0x55b727c60b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55b727c64000 msgr2=0x55b727c61600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x55b727c64000 0x55b727c61600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55b727c64400 msgr2=0x55b727c61b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x55b727c64400 0x55b727c61b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 reap_dead start 2024-04-25T10:18:10.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 reap_dead start 2024-04-25T10:18:10.466 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 reap_dead start 2024-04-25T10:18:10.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d51000640 1 -- 172.21.15.119:0/4120481699 reap_dead start 2024-04-25T10:18:10.467 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x55b727c50000 msgr2=0x55b727989700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x55b727c50000 0x55b727989700 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x55b728678e40 tx=0x55b728820780 comp rx=0 tx=0).stop 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7287b1400 msgr2=0x55b727988c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7287b1400 0x55b727988c00 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x55b7287ad380 tx=0x55b728820300 comp rx=0 tx=0).stop 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.462+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 shutdown_connections 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.463+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x55b727c50000 0x55b727989700 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.463+0000 7f4d539dc740 1 --2- 172.21.15.119:0/4120481699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7287b1400 0x55b727988c00 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.463+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 >> 172.21.15.119:0/4120481699 conn(0x55b7287b0c00 msgr2=0x55b7279d7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.463+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 shutdown_connections 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.463+0000 7f4d539dc740 1 -- 172.21.15.119:0/4120481699 wait complete. 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.464+0000 7f4d539dc740 1 -- 172.21.15.119:0/1935336689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7278e6c00 msgr2=0x55b727986580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.464+0000 7f4d539dc740 1 --2- 172.21.15.119:0/1935336689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7278e6c00 0x55b727986580 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x55b7286fbc80 tx=0x55b7287a2120 comp rx=0 tx=0).stop 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.464+0000 7f4d539dc740 1 -- 172.21.15.119:0/1935336689 shutdown_connections 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.464+0000 7f4d539dc740 1 --2- 172.21.15.119:0/1935336689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b7278e7c00 0x55b727987080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.464+0000 7f4d539dc740 1 --2- 172.21.15.119:0/1935336689 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55b7278e7800 0x55b727986b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.464+0000 7f4d539dc740 1 --2- 172.21.15.119:0/1935336689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7278e6c00 0x55b727986580 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.464+0000 7f4d539dc740 1 -- 172.21.15.119:0/1935336689 >> 172.21.15.119:0/1935336689 conn(0x55b7278e7000 msgr2=0x55b7279d6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.465+0000 7f4d539dc740 1 -- 172.21.15.119:0/1935336689 shutdown_connections 2024-04-25T10:18:10.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:10.465+0000 7f4d539dc740 1 -- 172.21.15.119:0/1935336689 wait complete. 2024-04-25T10:18:10.480 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.214815, result: 2024-04-25T10:18:10.482 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:18:10.489 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:18:10.623 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-04-25T10:18:10.623 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2024-04-25T10:18:10.623 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:18:10.623 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2024-04-25T10:18:10.623 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.623 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2024-04-25T10:18:10.623 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.623 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2024-04-25T10:18:10.623 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.623 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2024-04-25T10:18:10.623 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2024-04-25T10:18:10.624 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.624 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2024-04-25T10:18:10.624 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2024-04-25T10:18:10.624 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.624 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:18:10.624 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2024-04-25T10:18:10.624 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.624 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:18:10.624 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:18:10.624 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.624 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2024-04-25T10:18:10.624 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.624 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2024-04-25T10:18:10.624 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2024-04-25T10:18:10.625 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.119,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on skipped message = False 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: bdev async discard = True 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: bdev enable discard = True 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore allocator = bitmap 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 1/20 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 1/20 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 4/10 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: mon op complaint time = 120 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: client mount timeout = 600 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: debug client = 20 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = "" 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout:[mds] 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds = 20 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds balancer = 20 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug frag = True 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug scatterstat = False 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout: mds op complaint time = 180 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout: mds verify scatter = True 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2024-04-25T10:18:10.634 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2024-04-25T10:18:10.634 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2024-04-25T10:18:10.634 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2024-04-25T10:18:10.634 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:18:10.634 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2024-04-25T10:18:10.634 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.634 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.119,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug trim objects = true 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: bdev async discard = True 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: bdev enable discard = True 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore allocator = bitmap 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 1/20 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 1/20 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 4/10 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = False 2024-04-25T10:18:10.645 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2024-04-25T10:18:10.645 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = True 2024-04-25T10:18:10.645 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2024-04-25T10:18:10.645 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2024-04-25T10:18:10.645 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2024-04-25T10:18:10.662 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:18:10.667 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:18:10.790 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-04-25T10:18:10.791 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2024-04-25T10:18:10.791 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:18:10.791 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2024-04-25T10:18:10.791 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.791 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2024-04-25T10:18:10.791 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.791 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2024-04-25T10:18:10.791 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.791 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2024-04-25T10:18:10.791 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2024-04-25T10:18:10.791 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.791 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2024-04-25T10:18:10.791 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2024-04-25T10:18:10.792 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.792 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:18:10.792 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2024-04-25T10:18:10.792 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.792 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:18:10.792 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:18:10.792 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.792 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2024-04-25T10:18:10.792 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.792 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2024-04-25T10:18:10.792 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2024-04-25T10:18:10.793 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:18:10.793 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:18:10.793 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2024-04-25T10:18:10.793 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:18:10.793 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2024-04-25T10:18:10.793 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:18:10.793 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:18:10.793 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2024-04-25T10:18:10.793 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.793 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:18:10.793 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.793 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2024-04-25T10:18:10.794 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.794 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2024-04-25T10:18:10.794 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.794 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2024-04-25T10:18:10.794 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2024-04-25T10:18:10.794 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2024-04-25T10:18:10.794 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.119,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2024-04-25T10:18:10.794 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on skipped message = False 2024-04-25T10:18:10.794 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:18:10.794 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.794 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2024-04-25T10:18:10.794 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2024-04-25T10:18:10.794 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.795 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2024-04-25T10:18:10.795 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2024-04-25T10:18:10.795 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:18:10.795 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.795 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2024-04-25T10:18:10.795 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2024-04-25T10:18:10.795 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.795 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2024-04-25T10:18:10.795 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2024-04-25T10:18:10.795 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2024-04-25T10:18:10.795 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2024-04-25T10:18:10.795 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.795 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2024-04-25T10:18:10.795 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2024-04-25T10:18:10.796 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.796 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:18:10.796 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.796 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2024-04-25T10:18:10.796 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.796 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2024-04-25T10:18:10.796 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.796 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:18:10.796 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2024-04-25T10:18:10.796 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.796 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2024-04-25T10:18:10.796 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: bdev async discard = True 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: bdev enable discard = True 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore allocator = bitmap 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 1/20 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 1/20 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 4/10 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2024-04-25T10:18:10.797 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2024-04-25T10:18:10.798 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.798 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2024-04-25T10:18:10.798 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:18:10.798 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2024-04-25T10:18:10.798 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-04-25T10:18:10.798 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-04-25T10:18:10.798 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:18:10.798 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:18:10.798 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2024-04-25T10:18:10.798 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.798 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2024-04-25T10:18:10.798 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:18:10.798 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-04-25T10:18:10.798 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2024-04-25T10:18:10.799 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-04-25T10:18:10.799 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2024-04-25T10:18:10.799 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2024-04-25T10:18:10.799 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:18:10.799 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2024-04-25T10:18:10.799 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2024-04-25T10:18:10.799 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:18:10.799 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.799 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:18:10.799 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:18:10.799 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:18:10.799 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.800 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:18:10.800 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:18:10.800 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:18:10.800 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.800 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2024-04-25T10:18:10.800 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:18:10.800 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.800 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:18:10.800 INFO:teuthology.orchestra.run.smithi119.stdout: mon op complaint time = 120 2024-04-25T10:18:10.800 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:10.800 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2024-04-25T10:18:10.800 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2024-04-25T10:18:10.800 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2024-04-25T10:18:10.801 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2024-04-25T10:18:10.801 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:18:10.801 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:18:10.801 INFO:teuthology.orchestra.run.smithi119.stdout: client mount timeout = 600 2024-04-25T10:18:10.801 INFO:teuthology.orchestra.run.smithi119.stdout: debug client = 20 2024-04-25T10:18:10.801 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:18:10.801 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = "" 2024-04-25T10:18:10.801 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2024-04-25T10:18:10.801 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2024-04-25T10:18:10.801 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2024-04-25T10:18:10.801 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2024-04-25T10:18:10.801 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2024-04-25T10:18:10.801 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2024-04-25T10:18:10.802 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2024-04-25T10:18:10.802 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2024-04-25T10:18:10.802 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2024-04-25T10:18:10.802 INFO:teuthology.orchestra.run.smithi119.stdout:[mds] 2024-04-25T10:18:10.802 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds = 20 2024-04-25T10:18:10.802 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds balancer = 20 2024-04-25T10:18:10.802 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-04-25T10:18:10.802 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug frag = True 2024-04-25T10:18:10.802 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug scatterstat = True 2024-04-25T10:18:10.802 INFO:teuthology.orchestra.run.smithi119.stdout: mds op complaint time = 180 2024-04-25T10:18:10.802 INFO:teuthology.orchestra.run.smithi119.stdout: mds verify scatter = True 2024-04-25T10:18:10.802 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2024-04-25T10:18:10.802 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2024-04-25T10:18:10.802 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2024-04-25T10:18:10.803 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2024-04-25T10:18:10.803 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2024-04-25T10:18:10.803 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:18:10.803 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2024-04-25T10:18:10.803 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.803 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2024-04-25T10:18:10.803 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2024-04-25T10:18:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2024-04-25T10:18:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2024-04-25T10:18:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2024-04-25T10:18:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2024-04-25T10:18:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:18:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2024-04-25T10:18:10.804 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:18:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:18:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2024-04-25T10:18:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2024-04-25T10:18:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2024-04-25T10:18:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:18:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:18:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2024-04-25T10:18:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:18:10.805 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2024-04-25T10:18:10.806 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:18:10.806 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:18:10.806 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2024-04-25T10:18:10.806 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.806 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:18:10.806 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.806 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2024-04-25T10:18:10.806 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.806 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2024-04-25T10:18:10.806 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.806 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2024-04-25T10:18:10.806 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2024-04-25T10:18:10.807 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2024-04-25T10:18:10.807 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.119,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2024-04-25T10:18:10.807 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on skipped message = False 2024-04-25T10:18:10.807 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = e0b0f428-b441-4237-a4d1-ed53321f87a5 2024-04-25T10:18:10.807 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.807 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2024-04-25T10:18:10.807 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2024-04-25T10:18:10.807 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.807 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2024-04-25T10:18:10.807 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2024-04-25T10:18:10.807 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:18:10.808 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.808 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2024-04-25T10:18:10.808 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2024-04-25T10:18:10.808 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.808 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2024-04-25T10:18:10.808 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2024-04-25T10:18:10.808 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2024-04-25T10:18:10.808 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug trim objects = true 2024-04-25T10:18:10.808 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.808 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2024-04-25T10:18:10.808 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2024-04-25T10:18:10.808 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.808 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:18:10.809 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.809 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2024-04-25T10:18:10.809 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.809 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2024-04-25T10:18:10.809 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.809 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:18:10.809 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2024-04-25T10:18:10.809 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.809 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2024-04-25T10:18:10.809 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2024-04-25T10:18:10.809 INFO:teuthology.orchestra.run.smithi132.stdout: bdev async discard = True 2024-04-25T10:18:10.809 INFO:teuthology.orchestra.run.smithi132.stdout: bdev enable discard = True 2024-04-25T10:18:10.810 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore allocator = bitmap 2024-04-25T10:18:10.810 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2024-04-25T10:18:10.810 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2024-04-25T10:18:10.810 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 1/20 2024-04-25T10:18:10.810 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 1/20 2024-04-25T10:18:10.810 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:18:10.810 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2024-04-25T10:18:10.810 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 4/10 2024-04-25T10:18:10.810 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:18:10.810 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2024-04-25T10:18:10.810 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:18:10.810 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:18:10.811 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2024-04-25T10:18:10.811 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2024-04-25T10:18:10.811 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.811 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2024-04-25T10:18:10.811 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:18:10.811 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2024-04-25T10:18:10.811 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-04-25T10:18:10.811 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-04-25T10:18:10.811 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:18:10.811 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:18:10.811 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2024-04-25T10:18:10.811 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.811 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2024-04-25T10:18:10.812 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:18:10.812 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-04-25T10:18:10.812 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2024-04-25T10:18:10.812 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-04-25T10:18:10.812 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2024-04-25T10:18:10.812 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2024-04-25T10:18:10.812 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:18:10.812 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2024-04-25T10:18:10.812 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2024-04-25T10:18:10.812 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:18:10.812 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.812 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:18:10.813 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:18:10.813 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:18:10.813 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.813 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:18:10.813 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:18:10.813 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:18:10.813 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.813 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2024-04-25T10:18:10.813 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:18:10.813 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.813 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:18:10.813 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2024-04-25T10:18:10.814 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-25T10:18:10.814 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2024-04-25T10:18:10.814 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2024-04-25T10:18:10.814 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2024-04-25T10:18:10.814 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2024-04-25T10:18:10.814 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:18:10.814 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:18:10.814 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2024-04-25T10:18:10.814 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2024-04-25T10:18:10.814 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:18:10.814 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2024-04-25T10:18:10.815 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2024-04-25T10:18:10.815 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2024-04-25T10:18:10.815 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2024-04-25T10:18:10.815 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2024-04-25T10:18:10.815 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2024-04-25T10:18:10.815 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2024-04-25T10:18:10.815 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2024-04-25T10:18:10.815 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2024-04-25T10:18:10.815 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2024-04-25T10:18:10.815 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2024-04-25T10:18:10.815 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2024-04-25T10:18:10.815 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2024-04-25T10:18:10.815 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-25T10:18:10.816 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2024-04-25T10:18:10.816 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = True 2024-04-25T10:18:10.816 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2024-04-25T10:18:10.816 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = True 2024-04-25T10:18:10.816 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2024-04-25T10:18:10.816 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2024-04-25T10:18:10.816 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2024-04-25T10:18:10.835 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:18:11.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.035+0000 7ff729000640 1 Processor -- start 2024-04-25T10:18:11.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.035+0000 7ff729000640 1 -- start start 2024-04-25T10:18:11.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.037+0000 7ff729000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff72415aef0 0x7ff72415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:11.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.037+0000 7ff729000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72415b370 con 0x7ff7240b42c0 2024-04-25T10:18:11.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.037+0000 7ff729000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72415b650 con 0x7ff7240b46a0 2024-04-25T10:18:11.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.037+0000 7ff729000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7240d05d0 con 0x7ff72415aef0 2024-04-25T10:18:11.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.037+0000 7ff721e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff7240b42c0 0x7ff7240cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38060/0 (socket says 172.21.15.119:38060) 2024-04-25T10:18:11.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.037+0000 7ff721e00640 1 -- 172.21.15.119:0/4192424096 learned_addr learned my addr 172.21.15.119:0/4192424096 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:11.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.037+0000 7ff721400640 1 --2- 172.21.15.119:0/4192424096 >> v2:172.21.15.119:3300/0 conn(0x7ff72415aef0 0x7ff72415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:11.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.038+0000 7ff720a00640 1 -- 172.21.15.119:0/4192424096 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208318783 0 0) 0x7ff72415b370 con 0x7ff7240b42c0 2024-04-25T10:18:11.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.038+0000 7ff720a00640 1 -- 172.21.15.119:0/4192424096 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff708005730 con 0x7ff7240b42c0 2024-04-25T10:18:11.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.038+0000 7ff721400640 1 -- 172.21.15.119:0/4192424096 >> v1:172.21.15.132:6789/0 conn(0x7ff7240b46a0 legacy=0x7ff7241549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:11.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.038+0000 7ff721400640 1 -- 172.21.15.119:0/4192424096 >> v1:172.21.15.119:6789/0 conn(0x7ff7240b42c0 legacy=0x7ff7240cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.038+0000 7ff721400640 1 -- 172.21.15.119:0/4192424096 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72415b930 con 0x7ff72415aef0 2024-04-25T10:18:11.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.039+0000 7ff721400640 1 --2- 172.21.15.119:0/4192424096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72415aef0 0x7ff72415ef40 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7ff7100097a0 tx=0x7ff71002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c29a5b3d3d75d59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:11.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.039+0000 7ff720a00640 1 -- 172.21.15.119:0/4192424096 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710037070 con 0x7ff72415aef0 2024-04-25T10:18:11.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.039+0000 7ff720a00640 1 -- 172.21.15.119:0/4192424096 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71002adc0 con 0x7ff72415aef0 2024-04-25T10:18:11.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.039+0000 7ff720a00640 1 -- 172.21.15.119:0/4192424096 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710032640 con 0x7ff72415aef0 2024-04-25T10:18:11.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.040+0000 7ff729000640 1 -- 172.21.15.119:0/4192424096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72415aef0 msgr2=0x7ff72415ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.040+0000 7ff729000640 1 --2- 172.21.15.119:0/4192424096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72415aef0 0x7ff72415ef40 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7ff7100097a0 tx=0x7ff71002a0b0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.040+0000 7ff729000640 1 -- 172.21.15.119:0/4192424096 shutdown_connections 2024-04-25T10:18:11.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.040+0000 7ff729000640 1 --2- 172.21.15.119:0/4192424096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72415aef0 0x7ff72415ef40 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.040+0000 7ff729000640 1 -- 172.21.15.119:0/4192424096 >> 172.21.15.119:0/4192424096 conn(0x7ff72406aba0 msgr2=0x7ff72406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:11.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.040+0000 7ff729000640 1 -- 172.21.15.119:0/4192424096 shutdown_connections 2024-04-25T10:18:11.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.040+0000 7ff729000640 1 -- 172.21.15.119:0/4192424096 wait complete. 2024-04-25T10:18:11.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.042+0000 7ff729000640 1 Processor -- start 2024-04-25T10:18:11.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.042+0000 7ff729000640 1 -- start start 2024-04-25T10:18:11.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.043+0000 7ff729000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7240b42c0 0x7ff724152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:11.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.043+0000 7ff729000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7240b46a0 0x7ff7240cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:11.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.043+0000 7ff729000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff72415aef0 0x7ff7240ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:11.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.043+0000 7ff729000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72406ec60 con 0x7ff7240b42c0 2024-04-25T10:18:11.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.043+0000 7ff729000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72406eae0 con 0x7ff7240b46a0 2024-04-25T10:18:11.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.043+0000 7ff729000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7241550e0 con 0x7ff72415aef0 2024-04-25T10:18:11.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.043+0000 7ff721e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7240b42c0 0x7ff724152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:11.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.043+0000 7ff721e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7240b42c0 0x7ff724152400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55810/0 (socket says 172.21.15.119:55810) 2024-04-25T10:18:11.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.043+0000 7ff721e00640 1 -- 172.21.15.119:0/629049662 learned_addr learned my addr 172.21.15.119:0/629049662 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:11.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.044+0000 7ff721e00640 1 -- 172.21.15.119:0/629049662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff72415aef0 msgr2=0x7ff7240ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.044+0000 7ff721e00640 1 --2- 172.21.15.119:0/629049662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff72415aef0 0x7ff7240ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.044+0000 7ff721400640 1 --2- 172.21.15.119:0/629049662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7240b46a0 0x7ff7240cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:11.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.044+0000 7ff721e00640 1 -- 172.21.15.119:0/629049662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7240b46a0 msgr2=0x7ff7240cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.044+0000 7ff721e00640 1 --2- 172.21.15.119:0/629049662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7240b46a0 0x7ff7240cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.044+0000 7ff721e00640 1 -- 172.21.15.119:0/629049662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7240c6770 con 0x7ff7240b42c0 2024-04-25T10:18:11.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.044+0000 7ff721e00640 1 --2- 172.21.15.119:0/629049662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7240b42c0 0x7ff724152400 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7ff71400c430 tx=0x7ff71400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:11.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.044+0000 7ff71a000640 1 -- 172.21.15.119:0/629049662 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff714017070 con 0x7ff7240b42c0 2024-04-25T10:18:11.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.044+0000 7ff71a000640 1 -- 172.21.15.119:0/629049662 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff714005ce0 con 0x7ff7240b42c0 2024-04-25T10:18:11.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.044+0000 7ff71a000640 1 -- 172.21.15.119:0/629049662 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7140072f0 con 0x7ff7240b42c0 2024-04-25T10:18:11.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.044+0000 7ff729000640 1 -- 172.21.15.119:0/629049662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7240c6a50 con 0x7ff7240b42c0 2024-04-25T10:18:11.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.044+0000 7ff729000640 1 -- 172.21.15.119:0/629049662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7240c7010 con 0x7ff7240b42c0 2024-04-25T10:18:11.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.045+0000 7ff729000640 1 -- 172.21.15.119:0/629049662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72415deb0 con 0x7ff7240b42c0 2024-04-25T10:18:11.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.047+0000 7ff71a000640 1 -- 172.21.15.119:0/629049662 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff71401a070 con 0x7ff7240b42c0 2024-04-25T10:18:11.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.047+0000 7ff71a000640 1 --2- 172.21.15.119:0/629049662 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff70406b4d0 0x7ff70406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:11.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.048+0000 7ff71a000640 1 -- 172.21.15.119:0/629049662 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff7140897e0 con 0x7ff7240b42c0 2024-04-25T10:18:11.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.048+0000 7ff721400640 1 --2- 172.21.15.119:0/629049662 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff70406b4d0 0x7ff70406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:11.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.049+0000 7ff721400640 1 --2- 172.21.15.119:0/629049662 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff70406b4d0 0x7ff70406d990 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff710034000 tx=0x7ff71002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.052+0000 7ff71a000640 1 -- 172.21.15.119:0/629049662 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff72415deb0 con 0x7ff7240b42c0 2024-04-25T10:18:11.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.288+0000 7ff729000640 1 -- 172.21.15.119:0/629049662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7ff7240750b0 con 0x7ff7240b42c0 2024-04-25T10:18:11.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.792+0000 7ff71a000640 1 -- 172.21.15.119:0/629049662 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v108) v1 ==== 162+0+0 (secure 0 0 0) 0x7ff714058560 con 0x7ff7240b42c0 2024-04-25T10:18:11.796 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:18:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.795+0000 7ff729000640 1 -- 172.21.15.119:0/629049662 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff70406b4d0 msgr2=0x7ff70406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.795+0000 7ff729000640 1 --2- 172.21.15.119:0/629049662 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff70406b4d0 0x7ff70406d990 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff710034000 tx=0x7ff71002e000 comp rx=0 tx=0).stop 2024-04-25T10:18:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.795+0000 7ff729000640 1 -- 172.21.15.119:0/629049662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7240b42c0 msgr2=0x7ff724152400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.795+0000 7ff729000640 1 --2- 172.21.15.119:0/629049662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7240b42c0 0x7ff724152400 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7ff71400c430 tx=0x7ff71400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.796+0000 7ff729000640 1 -- 172.21.15.119:0/629049662 shutdown_connections 2024-04-25T10:18:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.796+0000 7ff729000640 1 --2- 172.21.15.119:0/629049662 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff70406b4d0 0x7ff70406d990 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.796+0000 7ff729000640 1 --2- 172.21.15.119:0/629049662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff72415aef0 0x7ff7240ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.796+0000 7ff729000640 1 --2- 172.21.15.119:0/629049662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7240b46a0 0x7ff7240cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.796+0000 7ff729000640 1 --2- 172.21.15.119:0/629049662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7240b42c0 0x7ff724152400 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.796+0000 7ff729000640 1 -- 172.21.15.119:0/629049662 >> 172.21.15.119:0/629049662 conn(0x7ff72406aba0 msgr2=0x7ff7241542f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:11.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.796+0000 7ff729000640 1 -- 172.21.15.119:0/629049662 shutdown_connections 2024-04-25T10:18:11.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:11.796+0000 7ff729000640 1 -- 172.21.15.119:0/629049662 wait complete. 2024-04-25T10:18:11.823 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:12.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.004+0000 7fa6a8e00640 1 Processor -- start 2024-04-25T10:18:12.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.004+0000 7fa6a8e00640 1 -- start start 2024-04-25T10:18:12.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.005+0000 7fa6a8e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a406e2e0 0x7fa6a40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.005+0000 7fa6a8e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa6a40cfc40 0x7fa6a415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.005+0000 7fa6a8e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a40cdb10 con 0x7fa6a415a870 2024-04-25T10:18:12.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.005+0000 7fa6a8e00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a40ce720 con 0x7fa6a40cfc40 2024-04-25T10:18:12.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.005+0000 7fa6a8e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40cddf0 con 0x7fa6a406e2e0 2024-04-25T10:18:12.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.005+0000 7fa6a2800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa6a40cfc40 0x7fa6a415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.005+0000 7fa6a2800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa6a40cfc40 0x7fa6a415ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55812/0 (socket says 172.21.15.119:55812) 2024-04-25T10:18:12.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.005+0000 7fa6a2800640 1 -- 172.21.15.119:0/3167643342 learned_addr learned my addr 172.21.15.119:0/3167643342 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:12.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.005+0000 7fa6a1400640 1 --2- 172.21.15.119:0/3167643342 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a406e2e0 0x7fa6a40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.005+0000 7fa6a2800640 1 -- 172.21.15.119:0/3167643342 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a406e2e0 msgr2=0x7fa6a40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:12.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.005+0000 7fa6a2800640 1 --2- 172.21.15.119:0/3167643342 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a406e2e0 0x7fa6a40cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.005+0000 7fa6a2800640 1 -- 172.21.15.119:0/3167643342 >> v1:172.21.15.132:6789/0 conn(0x7fa6a415a870 legacy=0x7fa6a40cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:12.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.005+0000 7fa6a2800640 1 -- 172.21.15.119:0/3167643342 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a40ce0d0 con 0x7fa6a40cfc40 2024-04-25T10:18:12.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.007+0000 7fa6a2800640 1 --2- 172.21.15.119:0/3167643342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a40cfc40 0x7fa6a415ac50 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009e50 tx=0x7fa69c02c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1fa3d171fb4636 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:12.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.007+0000 7fa6a0a00640 1 -- 172.21.15.119:0/3167643342 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c039070 con 0x7fa6a40cfc40 2024-04-25T10:18:12.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.007+0000 7fa6a0a00640 1 -- 172.21.15.119:0/3167643342 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c034680 con 0x7fa6a40cfc40 2024-04-25T10:18:12.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.007+0000 7fa6a0a00640 1 -- 172.21.15.119:0/3167643342 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c034a50 con 0x7fa6a40cfc40 2024-04-25T10:18:12.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.007+0000 7fa6a8e00640 1 -- 172.21.15.119:0/3167643342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a40cfc40 msgr2=0x7fa6a415ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.007+0000 7fa6a8e00640 1 --2- 172.21.15.119:0/3167643342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a40cfc40 0x7fa6a415ac50 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009e50 tx=0x7fa69c02c8d0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.008+0000 7fa6a8e00640 1 -- 172.21.15.119:0/3167643342 shutdown_connections 2024-04-25T10:18:12.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.008+0000 7fa6a8e00640 1 --2- 172.21.15.119:0/3167643342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a40cfc40 0x7fa6a415ac50 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.008+0000 7fa6a8e00640 1 --2- 172.21.15.119:0/3167643342 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a406e2e0 0x7fa6a40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.008+0000 7fa6a8e00640 1 -- 172.21.15.119:0/3167643342 >> 172.21.15.119:0/3167643342 conn(0x7fa6a406aba0 msgr2=0x7fa6a406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:12.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.008+0000 7fa6a8e00640 1 -- 172.21.15.119:0/3167643342 shutdown_connections 2024-04-25T10:18:12.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.008+0000 7fa6a8e00640 1 -- 172.21.15.119:0/3167643342 wait complete. 2024-04-25T10:18:12.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.009+0000 7fa6a8e00640 1 Processor -- start 2024-04-25T10:18:12.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.010+0000 7fa6a8e00640 1 -- start start 2024-04-25T10:18:12.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.011+0000 7fa6a8e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a406e2e0 0x7fa6a40cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.011+0000 7fa6a8e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a40cfc40 0x7fa6a406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.011+0000 7fa6a8e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a415a870 0x7fa6a406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.011+0000 7fa6a8e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40cd700 con 0x7fa6a406e2e0 2024-04-25T10:18:12.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.011+0000 7fa6a8e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40cd580 con 0x7fa6a415a870 2024-04-25T10:18:12.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.011+0000 7fa6a8e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a40cd880 con 0x7fa6a40cfc40 2024-04-25T10:18:12.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.011+0000 7fa6a1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a406e2e0 0x7fa6a40cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.011+0000 7fa6a1e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a406e2e0 0x7fa6a40cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55824/0 (socket says 172.21.15.119:55824) 2024-04-25T10:18:12.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.011+0000 7fa6a1e00640 1 -- 172.21.15.119:0/4080466629 learned_addr learned my addr 172.21.15.119:0/4080466629 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:12.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.011+0000 7fa6a1e00640 1 -- 172.21.15.119:0/4080466629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a40cfc40 msgr2=0x7fa6a406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.011+0000 7fa6a1e00640 1 --2- 172.21.15.119:0/4080466629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a40cfc40 0x7fa6a406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.011+0000 7fa6a1e00640 1 -- 172.21.15.119:0/4080466629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a415a870 msgr2=0x7fa6a406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.011+0000 7fa6a1e00640 1 --2- 172.21.15.119:0/4080466629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a415a870 0x7fa6a406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.011+0000 7fa6a1e00640 1 -- 172.21.15.119:0/4080466629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a40ab630 con 0x7fa6a406e2e0 2024-04-25T10:18:12.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.012+0000 7fa6a1e00640 1 --2- 172.21.15.119:0/4080466629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a406e2e0 0x7fa6a40cb740 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fa69400b420 tx=0x7fa69400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:12.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.012+0000 7fa69a000640 1 -- 172.21.15.119:0/4080466629 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694011830 con 0x7fa6a406e2e0 2024-04-25T10:18:12.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.012+0000 7fa69a000640 1 -- 172.21.15.119:0/4080466629 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa694016070 con 0x7fa6a406e2e0 2024-04-25T10:18:12.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.012+0000 7fa69a000640 1 -- 172.21.15.119:0/4080466629 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694011c10 con 0x7fa6a406e2e0 2024-04-25T10:18:12.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.012+0000 7fa6a8e00640 1 -- 172.21.15.119:0/4080466629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a40ab910 con 0x7fa6a406e2e0 2024-04-25T10:18:12.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.012+0000 7fa6a8e00640 1 -- 172.21.15.119:0/4080466629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a40abed0 con 0x7fa6a406e2e0 2024-04-25T10:18:12.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.013+0000 7fa6a8e00640 1 -- 172.21.15.119:0/4080466629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a40749b0 con 0x7fa6a406e2e0 2024-04-25T10:18:12.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.015+0000 7fa69a000640 1 -- 172.21.15.119:0/4080466629 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa694004020 con 0x7fa6a406e2e0 2024-04-25T10:18:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.016+0000 7fa69a000640 1 --2- 172.21.15.119:0/4080466629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa68406b4d0 0x7fa68406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.016+0000 7fa69a000640 1 -- 172.21.15.119:0/4080466629 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fa69408a750 con 0x7fa6a406e2e0 2024-04-25T10:18:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.016+0000 7fa6a1400640 1 --2- 172.21.15.119:0/4080466629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa68406b4d0 0x7fa68406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.016+0000 7fa6a1400640 1 --2- 172.21.15.119:0/4080466629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa68406b4d0 0x7fa68406d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa690006fd0 tx=0x7fa690008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:12.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.019+0000 7fa69a000640 1 -- 172.21.15.119:0/4080466629 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa694020020 con 0x7fa6a406e2e0 2024-04-25T10:18:12.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.254+0000 7fa6a8e00640 1 -- 172.21.15.119:0/4080466629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6a406c170 con 0x7fa6a406e2e0 2024-04-25T10:18:12.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.255+0000 7fa69a000640 1 -- 172.21.15.119:0/4080466629 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4957 (secure 0 0 0) 0x7fa694059550 con 0x7fa6a406e2e0 2024-04-25T10:18:12.259 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:12.259 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":109,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3394261143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3394261143},{"type":"v1","addr":"172.21.15.132:6834","nonce":3394261143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2502874931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2502874931},{"type":"v1","addr":"172.21.15.119:6835","nonce":2502874931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":8694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2305428003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2305428003},{"type":"v1","addr":"172.21.15.132:6835","nonce":2305428003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:37.617787+0000","modified":"2024-04-25T10:18:11.797013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9117},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9117":{"gid":9117,"name":"c","rank":0,"incarnation":109,"state":"up:replay","state_seq":1,"addr":"172.21.15.119:6837/2618740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2618740513},{"type":"v1","addr":"172.21.15.119:6837","nonce":2618740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:18:12.259 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 109 2024-04-25T10:18:12.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.258+0000 7fa6a8e00640 1 -- 172.21.15.119:0/4080466629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa68406b4d0 msgr2=0x7fa68406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.258+0000 7fa6a8e00640 1 --2- 172.21.15.119:0/4080466629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa68406b4d0 0x7fa68406d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa690006fd0 tx=0x7fa690008040 comp rx=0 tx=0).stop 2024-04-25T10:18:12.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.258+0000 7fa6a8e00640 1 -- 172.21.15.119:0/4080466629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a406e2e0 msgr2=0x7fa6a40cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.258+0000 7fa6a8e00640 1 --2- 172.21.15.119:0/4080466629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a406e2e0 0x7fa6a40cb740 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fa69400b420 tx=0x7fa69400b8e0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.258+0000 7fa6a8e00640 1 -- 172.21.15.119:0/4080466629 shutdown_connections 2024-04-25T10:18:12.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.258+0000 7fa6a8e00640 1 --2- 172.21.15.119:0/4080466629 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa68406b4d0 0x7fa68406d990 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.258+0000 7fa6a8e00640 1 --2- 172.21.15.119:0/4080466629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a415a870 0x7fa6a406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.258+0000 7fa6a8e00640 1 --2- 172.21.15.119:0/4080466629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a40cfc40 0x7fa6a406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.258+0000 7fa6a8e00640 1 --2- 172.21.15.119:0/4080466629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a406e2e0 0x7fa6a40cb740 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.258+0000 7fa6a8e00640 1 -- 172.21.15.119:0/4080466629 >> 172.21.15.119:0/4080466629 conn(0x7fa6a406aba0 msgr2=0x7fa6a40af150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:12.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.258+0000 7fa6a8e00640 1 -- 172.21.15.119:0/4080466629 shutdown_connections 2024-04-25T10:18:12.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:12.259+0000 7fa6a8e00640 1 -- 172.21.15.119:0/4080466629 wait complete. 2024-04-25T10:18:12.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:17:37.617787+0000', 'modified': '2024-04-25T10:18:11.797013+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9117}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9117': {'gid': 9117, 'name': 'c', 'rank': 0, 'incarnation': 109, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.119:6837/2618740513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 2618740513}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2618740513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:18:12.682 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9117:up:replay 2024-04-25T10:18:13.683 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:13.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.890+0000 7f8f0a200640 1 Processor -- start 2024-04-25T10:18:13.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.890+0000 7f8f0a200640 1 -- start start 2024-04-25T10:18:13.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.891+0000 7f8f0a200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f04144470 0x7f8f0415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.891+0000 7f8f0a200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8f0406e430 0x7f8f040b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.891+0000 7f8f0a200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f040cd440 con 0x7f8f040b4e80 2024-04-25T10:18:13.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.891+0000 7f8f0a200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f04145340 con 0x7f8f0406e430 2024-04-25T10:18:13.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.891+0000 7f8f0a200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040cd720 con 0x7f8f04144470 2024-04-25T10:18:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.892+0000 7f8f03e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8f0406e430 0x7f8f040b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.892+0000 7f8f08e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f04144470 0x7f8f0415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.892+0000 7f8f03e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8f0406e430 0x7f8f040b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:56920/0 (socket says 172.21.15.119:56920) 2024-04-25T10:18:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.892+0000 7f8f03e00640 1 -- 172.21.15.119:0/3667656387 learned_addr learned my addr 172.21.15.119:0/3667656387 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.893+0000 7f8f03e00640 1 -- 172.21.15.119:0/3667656387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f04144470 msgr2=0x7f8f0415fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.893+0000 7f8f03e00640 1 --2- 172.21.15.119:0/3667656387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f04144470 0x7f8f0415fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.893+0000 7f8f03e00640 1 -- 172.21.15.119:0/3667656387 >> v1:172.21.15.132:6789/0 conn(0x7f8f040b4e80 legacy=0x7f8f0406d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.893+0000 7f8f03e00640 1 -- 172.21.15.119:0/3667656387 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f040cda00 con 0x7f8f0406e430 2024-04-25T10:18:13.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.894+0000 7f8f03e00640 1 --2- 172.21.15.119:0/3667656387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f0406e430 0x7f8f040b4940 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009870 tx=0x7f8eec02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=245ff2ac9ceb13f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:13.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.895+0000 7f8f03400640 1 -- 172.21.15.119:0/3667656387 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec037070 con 0x7f8f0406e430 2024-04-25T10:18:13.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.895+0000 7f8f03400640 1 -- 172.21.15.119:0/3667656387 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eec032440 con 0x7f8f0406e430 2024-04-25T10:18:13.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.895+0000 7f8f03400640 1 -- 172.21.15.119:0/3667656387 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec032810 con 0x7f8f0406e430 2024-04-25T10:18:13.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.896+0000 7f8f0a200640 1 -- 172.21.15.119:0/3667656387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f0406e430 msgr2=0x7f8f040b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.896+0000 7f8f0a200640 1 --2- 172.21.15.119:0/3667656387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f0406e430 0x7f8f040b4940 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009870 tx=0x7f8eec02a210 comp rx=0 tx=0).stop 2024-04-25T10:18:13.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.896+0000 7f8f0a200640 1 -- 172.21.15.119:0/3667656387 shutdown_connections 2024-04-25T10:18:13.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.896+0000 7f8f0a200640 1 --2- 172.21.15.119:0/3667656387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f0406e430 0x7f8f040b4940 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.896+0000 7f8f0a200640 1 --2- 172.21.15.119:0/3667656387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f04144470 0x7f8f0415fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.896+0000 7f8f0a200640 1 -- 172.21.15.119:0/3667656387 >> 172.21.15.119:0/3667656387 conn(0x7f8f0406b1d0 msgr2=0x7f8f040b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:13.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.896+0000 7f8f0a200640 1 -- 172.21.15.119:0/3667656387 shutdown_connections 2024-04-25T10:18:13.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.896+0000 7f8f0a200640 1 -- 172.21.15.119:0/3667656387 wait complete. 2024-04-25T10:18:13.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.897+0000 7f8f0a200640 1 Processor -- start 2024-04-25T10:18:13.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.898+0000 7f8f0a200640 1 -- start start 2024-04-25T10:18:13.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.898+0000 7f8f0a200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f0406e430 0x7f8f0415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.898+0000 7f8f0a200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f040b4e80 0x7f8f0415f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.898+0000 7f8f0a200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f04144470 0x7f8f0415ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.898+0000 7f8f0a200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040ce200 con 0x7f8f0406e430 2024-04-25T10:18:13.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.898+0000 7f8f0a200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f04144850 con 0x7f8f04144470 2024-04-25T10:18:13.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.898+0000 7f8f0a200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0415cd00 con 0x7f8f040b4e80 2024-04-25T10:18:13.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.899+0000 7f8f08e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f0406e430 0x7f8f0415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.899+0000 7f8f08e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f0406e430 0x7f8f0415d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55834/0 (socket says 172.21.15.119:55834) 2024-04-25T10:18:13.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.899+0000 7f8f08e00640 1 -- 172.21.15.119:0/2098918712 learned_addr learned my addr 172.21.15.119:0/2098918712 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:13.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.899+0000 7f8f03e00640 1 --2- 172.21.15.119:0/2098918712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f040b4e80 0x7f8f0415f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.899+0000 7f8f09800640 1 --2- 172.21.15.119:0/2098918712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f04144470 0x7f8f0415ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.899+0000 7f8f08e00640 1 -- 172.21.15.119:0/2098918712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f040b4e80 msgr2=0x7f8f0415f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.899+0000 7f8f08e00640 1 --2- 172.21.15.119:0/2098918712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f040b4e80 0x7f8f0415f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.899+0000 7f8f08e00640 1 -- 172.21.15.119:0/2098918712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f04144470 msgr2=0x7f8f0415ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.899+0000 7f8f08e00640 1 --2- 172.21.15.119:0/2098918712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f04144470 0x7f8f0415ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.899+0000 7f8f08e00640 1 -- 172.21.15.119:0/2098918712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0415ea40 con 0x7f8f0406e430 2024-04-25T10:18:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.900+0000 7f8f08e00640 1 --2- 172.21.15.119:0/2098918712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f0406e430 0x7f8f0415d0a0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400e870 tx=0x7f8ef400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.900+0000 7f8f00c00640 1 -- 172.21.15.119:0/2098918712 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4017070 con 0x7f8f0406e430 2024-04-25T10:18:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.900+0000 7f8f00c00640 1 -- 172.21.15.119:0/2098918712 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef400c8e0 con 0x7f8f0406e430 2024-04-25T10:18:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.900+0000 7f8f00c00640 1 -- 172.21.15.119:0/2098918712 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef400ccd0 con 0x7f8f0406e430 2024-04-25T10:18:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.900+0000 7f8f0a200640 1 -- 172.21.15.119:0/2098918712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0415d9a0 con 0x7f8f0406e430 2024-04-25T10:18:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.900+0000 7f8f0a200640 1 -- 172.21.15.119:0/2098918712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0415e240 con 0x7f8f0406e430 2024-04-25T10:18:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.901+0000 7f8f0a200640 1 -- 172.21.15.119:0/2098918712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f040749b0 con 0x7f8f0406e430 2024-04-25T10:18:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.902+0000 7f8f00c00640 1 -- 172.21.15.119:0/2098918712 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ef401a070 con 0x7f8f0406e430 2024-04-25T10:18:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.903+0000 7f8f00c00640 1 --2- 172.21.15.119:0/2098918712 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8ee006b500 0x7f8ee006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.903+0000 7f8f00c00640 1 -- 172.21.15.119:0/2098918712 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f8ef408a060 con 0x7f8f0406e430 2024-04-25T10:18:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.903+0000 7f8f03e00640 1 --2- 172.21.15.119:0/2098918712 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8ee006b500 0x7f8ee006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.904+0000 7f8f03e00640 1 --2- 172.21.15.119:0/2098918712 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8ee006b500 0x7f8ee006d9c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8eec034000 tx=0x7f8eec02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:13.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:13.908+0000 7f8f00c00640 1 -- 172.21.15.119:0/2098918712 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ef4058e80 con 0x7f8f0406e430 2024-04-25T10:18:14.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:14.140+0000 7f8f0a200640 1 -- 172.21.15.119:0/2098918712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f040b2d90 con 0x7f8f0406e430 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:14.141+0000 7f8f00c00640 1 -- 172.21.15.119:0/2098918712 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4957 (secure 0 0 0) 0x7f8ef4058780 con 0x7f8f0406e430 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:14.145 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3394261143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3394261143},{"type":"v1","addr":"172.21.15.132:6834","nonce":3394261143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2502874931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2502874931},{"type":"v1","addr":"172.21.15.119:6835","nonce":2502874931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":8694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2305428003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2305428003},{"type":"v1","addr":"172.21.15.132:6835","nonce":2305428003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:37.617787+0000","modified":"2024-04-25T10:18:12.820935+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9117},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9117":{"gid":9117,"name":"c","rank":0,"incarnation":109,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.119:6837/2618740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2618740513},{"type":"v1","addr":"172.21.15.119:6837","nonce":2618740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:18:14.145 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 111 2024-04-25T10:18:14.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:14.143+0000 7f8f0a200640 1 -- 172.21.15.119:0/2098918712 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8ee006b500 msgr2=0x7f8ee006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:14.143+0000 7f8f0a200640 1 --2- 172.21.15.119:0/2098918712 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8ee006b500 0x7f8ee006d9c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8eec034000 tx=0x7f8eec02e000 comp rx=0 tx=0).stop 2024-04-25T10:18:14.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:14.143+0000 7f8f0a200640 1 -- 172.21.15.119:0/2098918712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f0406e430 msgr2=0x7f8f0415d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:14.143+0000 7f8f0a200640 1 --2- 172.21.15.119:0/2098918712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f0406e430 0x7f8f0415d0a0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400e870 tx=0x7f8ef400ed30 comp rx=0 tx=0).stop 2024-04-25T10:18:14.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:14.144+0000 7f8f0a200640 1 -- 172.21.15.119:0/2098918712 shutdown_connections 2024-04-25T10:18:14.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:14.144+0000 7f8f0a200640 1 --2- 172.21.15.119:0/2098918712 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f8ee006b500 0x7f8ee006d9c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:14.144+0000 7f8f0a200640 1 --2- 172.21.15.119:0/2098918712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f04144470 0x7f8f0415ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:14.144+0000 7f8f0a200640 1 --2- 172.21.15.119:0/2098918712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f040b4e80 0x7f8f0415f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:14.144+0000 7f8f0a200640 1 --2- 172.21.15.119:0/2098918712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f0406e430 0x7f8f0415d0a0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:14.144+0000 7f8f0a200640 1 -- 172.21.15.119:0/2098918712 >> 172.21.15.119:0/2098918712 conn(0x7f8f0406b1d0 msgr2=0x7f8f04143430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:14.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:14.144+0000 7f8f0a200640 1 -- 172.21.15.119:0/2098918712 shutdown_connections 2024-04-25T10:18:14.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:14.144+0000 7f8f0a200640 1 -- 172.21.15.119:0/2098918712 wait complete. 2024-04-25T10:18:14.568 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:17:37.617787+0000', 'modified': '2024-04-25T10:18:12.820935+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9117}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9117': {'gid': 9117, 'name': 'c', 'rank': 0, 'incarnation': 109, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.119:6837/2618740513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 2618740513}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2618740513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:18:14.569 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9117:up:rejoin 2024-04-25T10:18:15.570 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:15.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.780+0000 7f2f41a00640 1 Processor -- start 2024-04-25T10:18:15.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.780+0000 7f2f41a00640 1 -- start start 2024-04-25T10:18:15.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.782+0000 7f2f41a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2f3c0b3db0 0x7f2f3c159b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.782+0000 7f2f41a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c0cc8e0 con 0x7f2f3c15d690 2024-04-25T10:18:15.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.782+0000 7f2f41a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c071fb0 con 0x7f2f3c06b2b0 2024-04-25T10:18:15.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.782+0000 7f2f41a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f3c15dbe0 con 0x7f2f3c0b3db0 2024-04-25T10:18:15.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.782+0000 7f2f3a000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2f3c15d690 0x7f2f3c0718a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38070/0 (socket says 172.21.15.119:38070) 2024-04-25T10:18:15.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.782+0000 7f2f3a000640 1 -- 172.21.15.119:0/1718125056 learned_addr learned my addr 172.21.15.119:0/1718125056 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:15.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.783+0000 7f2f3b400640 1 --2- 172.21.15.119:0/1718125056 >> v2:172.21.15.132:3300/0 conn(0x7f2f3c0b3db0 0x7f2f3c159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.783+0000 7f2f39600640 1 -- 172.21.15.119:0/1718125056 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39339412 0 0) 0x7f2f28001110 con 0x7f2f3c15d690 2024-04-25T10:18:15.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.783+0000 7f2f39600640 1 -- 172.21.15.119:0/1718125056 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f20003440 con 0x7f2f3c15d690 2024-04-25T10:18:15.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.783+0000 7f2f39600640 1 -- 172.21.15.119:0/1718125056 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2126610123 0 0) 0x7f2f20003440 con 0x7f2f3c15d690 2024-04-25T10:18:15.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.783+0000 7f2f39600640 1 -- 172.21.15.119:0/1718125056 >> v2:172.21.15.132:3300/0 conn(0x7f2f3c0b3db0 msgr2=0x7f2f3c159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:15.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.783+0000 7f2f39600640 1 --2- 172.21.15.119:0/1718125056 >> v2:172.21.15.132:3300/0 conn(0x7f2f3c0b3db0 0x7f2f3c159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.783+0000 7f2f39600640 1 -- 172.21.15.119:0/1718125056 >> v1:172.21.15.132:6789/0 conn(0x7f2f3c06b2b0 legacy=0x7f2f3c071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.783+0000 7f2f39600640 1 -- 172.21.15.119:0/1718125056 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c15dec0 con 0x7f2f3c15d690 2024-04-25T10:18:15.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.783+0000 7f2f39600640 1 -- 172.21.15.119:0/1718125056 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f2f28001840 con 0x7f2f3c15d690 2024-04-25T10:18:15.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.783+0000 7f2f39600640 1 -- 172.21.15.119:0/1718125056 >> v1:172.21.15.119:6789/0 conn(0x7f2f3c15d690 legacy=0x7f2f3c0718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.783+0000 7f2f39600640 1 --2- 172.21.15.119:0/1718125056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f20003af0 0x7f2f20003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.784+0000 7f2f39600640 1 --2- 172.21.15.119:0/1718125056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f200088a0 0x7f2f2000ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.784+0000 7f2f39600640 1 --2- 172.21.15.119:0/1718125056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2000b5b0 0x7f2f2000da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.784+0000 7f2f39600640 1 -- 172.21.15.119:0/1718125056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c15dbe0 con 0x7f2f200088a0 2024-04-25T10:18:15.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.784+0000 7f2f39600640 1 -- 172.21.15.119:0/1718125056 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2000e350 con 0x7f2f2000b5b0 2024-04-25T10:18:15.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.784+0000 7f2f39600640 1 -- 172.21.15.119:0/1718125056 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2000e550 con 0x7f2f20003af0 2024-04-25T10:18:15.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.784+0000 7f2f3a000640 1 --2- 172.21.15.119:0/1718125056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f200088a0 0x7f2f2000ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.784+0000 7f2f3a000640 1 -- 172.21.15.119:0/1718125056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f20003af0 msgr2=0x7f2f20003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:15.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.784+0000 7f2f3aa00640 1 --2- 172.21.15.119:0/1718125056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f20003af0 0x7f2f20003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.784+0000 7f2f3a000640 1 --2- 172.21.15.119:0/1718125056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f20003af0 0x7f2f20003f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.784+0000 7f2f3a000640 1 -- 172.21.15.119:0/1718125056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2000b5b0 msgr2=0x7f2f2000da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:15.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.784+0000 7f2f3a000640 1 --2- 172.21.15.119:0/1718125056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2000b5b0 0x7f2f2000da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.784+0000 7f2f3a000640 1 -- 172.21.15.119:0/1718125056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f20003440 con 0x7f2f200088a0 2024-04-25T10:18:15.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.784+0000 7f2f3b400640 1 -- 172.21.15.119:0/1718125056 reap_dead start 2024-04-25T10:18:15.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.785+0000 7f2f3a000640 1 --2- 172.21.15.119:0/1718125056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f200088a0 0x7f2f2000ad10 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800c320 tx=0x7f2f20003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7613939da6340563 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:15.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.785+0000 7f2f39600640 1 -- 172.21.15.119:0/1718125056 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28007e10 con 0x7f2f200088a0 2024-04-25T10:18:15.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.786+0000 7f2f39600640 1 -- 172.21.15.119:0/1718125056 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f28005ad0 con 0x7f2f200088a0 2024-04-25T10:18:15.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.786+0000 7f2f39600640 1 -- 172.21.15.119:0/1718125056 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28007070 con 0x7f2f200088a0 2024-04-25T10:18:15.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.786+0000 7f2f41a00640 1 -- 172.21.15.119:0/1718125056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f200088a0 msgr2=0x7f2f2000ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.786+0000 7f2f41a00640 1 --2- 172.21.15.119:0/1718125056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f200088a0 0x7f2f2000ad10 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800c320 tx=0x7f2f20003730 comp rx=0 tx=0).stop 2024-04-25T10:18:15.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.786+0000 7f2f41a00640 1 -- 172.21.15.119:0/1718125056 shutdown_connections 2024-04-25T10:18:15.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.786+0000 7f2f41a00640 1 --2- 172.21.15.119:0/1718125056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f200088a0 0x7f2f2000ad10 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.786+0000 7f2f41a00640 1 -- 172.21.15.119:0/1718125056 >> 172.21.15.119:0/1718125056 conn(0x7f2f3c066d80 msgr2=0x7f2f3c06aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:15.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.786+0000 7f2f41a00640 1 -- 172.21.15.119:0/1718125056 shutdown_connections 2024-04-25T10:18:15.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.786+0000 7f2f41a00640 1 -- 172.21.15.119:0/1718125056 wait complete. 2024-04-25T10:18:15.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.788+0000 7f2f41a00640 1 Processor -- start 2024-04-25T10:18:15.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.788+0000 7f2f41a00640 1 -- start start 2024-04-25T10:18:15.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.789+0000 7f2f41a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f200088a0 0x7f2f3c1f0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.789+0000 7f2f41a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f3c15c7f0 0x7f2f3c15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.789+0000 7f2f41a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f3c14cf30 0x7f2f3c14d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.789+0000 7f2f41a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c0b2880 con 0x7f2f200088a0 2024-04-25T10:18:15.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.789+0000 7f2f41a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c15e480 con 0x7f2f3c14cf30 2024-04-25T10:18:15.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.789+0000 7f2f41a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c06ecd0 con 0x7f2f3c15c7f0 2024-04-25T10:18:15.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.789+0000 7f2f3aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f200088a0 0x7f2f3c1f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.789+0000 7f2f3aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f200088a0 0x7f2f3c1f0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55856/0 (socket says 172.21.15.119:55856) 2024-04-25T10:18:15.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.789+0000 7f2f3aa00640 1 -- 172.21.15.119:0/4244732447 learned_addr learned my addr 172.21.15.119:0/4244732447 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:15.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.790+0000 7f2f3aa00640 1 -- 172.21.15.119:0/4244732447 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f3c15c7f0 msgr2=0x7f2f3c15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.790+0000 7f2f3aa00640 1 --2- 172.21.15.119:0/4244732447 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f3c15c7f0 0x7f2f3c15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.790+0000 7f2f3aa00640 1 -- 172.21.15.119:0/4244732447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f3c14cf30 msgr2=0x7f2f3c14d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.790+0000 7f2f3b400640 1 --2- 172.21.15.119:0/4244732447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f3c14cf30 0x7f2f3c14d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.790+0000 7f2f3aa00640 1 --2- 172.21.15.119:0/4244732447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f3c14cf30 0x7f2f3c14d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.790+0000 7f2f3aa00640 1 -- 172.21.15.119:0/4244732447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c1f06a0 con 0x7f2f200088a0 2024-04-25T10:18:15.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.790+0000 7f2f3aa00640 1 --2- 172.21.15.119:0/4244732447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f200088a0 0x7f2f3c1f0160 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00d470 tx=0x7f2f2c00d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:15.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.790+0000 7f2f32a00640 1 -- 172.21.15.119:0/4244732447 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c005480 con 0x7f2f200088a0 2024-04-25T10:18:15.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.790+0000 7f2f32a00640 1 -- 172.21.15.119:0/4244732447 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c017070 con 0x7f2f200088a0 2024-04-25T10:18:15.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.790+0000 7f2f32a00640 1 -- 172.21.15.119:0/4244732447 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c003810 con 0x7f2f200088a0 2024-04-25T10:18:15.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.790+0000 7f2f41a00640 1 -- 172.21.15.119:0/4244732447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c14b380 con 0x7f2f200088a0 2024-04-25T10:18:15.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.791+0000 7f2f41a00640 1 -- 172.21.15.119:0/4244732447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c14b940 con 0x7f2f200088a0 2024-04-25T10:18:15.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.792+0000 7f2f41a00640 1 -- 172.21.15.119:0/4244732447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3c0749b0 con 0x7f2f200088a0 2024-04-25T10:18:15.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.792+0000 7f2f32a00640 1 -- 172.21.15.119:0/4244732447 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f2c0039b0 con 0x7f2f200088a0 2024-04-25T10:18:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.793+0000 7f2f32a00640 1 --2- 172.21.15.119:0/4244732447 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2f1c06b500 0x7f2f1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.794+0000 7f2f32a00640 1 -- 172.21.15.119:0/4244732447 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f2f2c08a0b0 con 0x7f2f200088a0 2024-04-25T10:18:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.794+0000 7f2f3a000640 1 --2- 172.21.15.119:0/4244732447 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2f1c06b500 0x7f2f1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.795+0000 7f2f3a000640 1 --2- 172.21.15.119:0/4244732447 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2f1c06b500 0x7f2f1c06d9c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2f28033000 tx=0x7f2f28009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:15.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:15.798+0000 7f2f32a00640 1 -- 172.21.15.119:0/4244732447 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f2c058da0 con 0x7f2f200088a0 2024-04-25T10:18:16.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.030+0000 7f2f41a00640 1 -- 172.21.15.119:0/4244732447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f3c06d830 con 0x7f2f200088a0 2024-04-25T10:18:16.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.030+0000 7f2f32a00640 1 -- 172.21.15.119:0/4244732447 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4957 (secure 0 0 0) 0x7f2f2c0586a0 con 0x7f2f200088a0 2024-04-25T10:18:16.034 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:16.034 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3394261143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3394261143},{"type":"v1","addr":"172.21.15.132:6834","nonce":3394261143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2502874931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2502874931},{"type":"v1","addr":"172.21.15.119:6835","nonce":2502874931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":8694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2305428003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2305428003},{"type":"v1","addr":"172.21.15.132:6835","nonce":2305428003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:37.617787+0000","modified":"2024-04-25T10:18:13.835772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9117},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9117":{"gid":9117,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/2618740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2618740513},{"type":"v1","addr":"172.21.15.119:6837","nonce":2618740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:18:16.034 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 112 2024-04-25T10:18:16.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.033+0000 7f2f41a00640 1 -- 172.21.15.119:0/4244732447 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2f1c06b500 msgr2=0x7f2f1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.033+0000 7f2f41a00640 1 --2- 172.21.15.119:0/4244732447 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2f1c06b500 0x7f2f1c06d9c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2f28033000 tx=0x7f2f28009030 comp rx=0 tx=0).stop 2024-04-25T10:18:16.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.033+0000 7f2f41a00640 1 -- 172.21.15.119:0/4244732447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f200088a0 msgr2=0x7f2f3c1f0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.033+0000 7f2f41a00640 1 --2- 172.21.15.119:0/4244732447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f200088a0 0x7f2f3c1f0160 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00d470 tx=0x7f2f2c00d930 comp rx=0 tx=0).stop 2024-04-25T10:18:16.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.033+0000 7f2f41a00640 1 -- 172.21.15.119:0/4244732447 shutdown_connections 2024-04-25T10:18:16.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.033+0000 7f2f41a00640 1 --2- 172.21.15.119:0/4244732447 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f2f1c06b500 0x7f2f1c06d9c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.033+0000 7f2f41a00640 1 --2- 172.21.15.119:0/4244732447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f3c14cf30 0x7f2f3c14d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.033+0000 7f2f41a00640 1 --2- 172.21.15.119:0/4244732447 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2f3c15c7f0 0x7f2f3c15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.033+0000 7f2f41a00640 1 --2- 172.21.15.119:0/4244732447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f200088a0 0x7f2f3c1f0160 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.033+0000 7f2f41a00640 1 -- 172.21.15.119:0/4244732447 >> 172.21.15.119:0/4244732447 conn(0x7f2f3c066d80 msgr2=0x7f2f3c0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:16.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.033+0000 7f2f41a00640 1 -- 172.21.15.119:0/4244732447 shutdown_connections 2024-04-25T10:18:16.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.033+0000 7f2f41a00640 1 -- 172.21.15.119:0/4244732447 wait complete. 2024-04-25T10:18:16.458 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:17:37.617787+0000', 'modified': '2024-04-25T10:18:13.835772+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9117}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9117': {'gid': 9117, 'name': 'c', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.119:6837/2618740513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 2618740513}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2618740513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:18:16.458 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:18:16.458 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:18:16.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.665+0000 7f599f200640 1 Processor -- start 2024-04-25T10:18:16.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.665+0000 7f599f200640 1 -- start start 2024-04-25T10:18:16.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.666+0000 7f599f200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59980cd630 0x7f599809a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.666+0000 7f599f200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59980cfef0 con 0x7f599806cdd0 2024-04-25T10:18:16.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.666+0000 7f599f200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59980d01d0 con 0x7f59980cf880 2024-04-25T10:18:16.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.666+0000 7f599f200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59980d04b0 con 0x7f59980cd630 2024-04-25T10:18:16.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.667+0000 7f599de00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59980cd630 0x7f599809a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.667+0000 7f599de00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59980cd630 0x7f599809a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55866/0 (socket says 172.21.15.119:55866) 2024-04-25T10:18:16.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.667+0000 7f599de00640 1 -- 172.21.15.119:0/2563519664 learned_addr learned my addr 172.21.15.119:0/2563519664 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:16.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.667+0000 7f599e800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f599806cdd0 0x7f599815d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38072/0 (socket says 172.21.15.119:38072) 2024-04-25T10:18:16.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.667+0000 7f599de00640 1 -- 172.21.15.119:0/2563519664 >> v1:172.21.15.132:6789/0 conn(0x7f59980cf880 legacy=0x7f599806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:16.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.667+0000 7f599de00640 1 -- 172.21.15.119:0/2563519664 >> v1:172.21.15.119:6789/0 conn(0x7f599806cdd0 legacy=0x7f599815d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.667+0000 7f599de00640 1 -- 172.21.15.119:0/2563519664 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59980d0790 con 0x7f59980cd630 2024-04-25T10:18:16.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.668+0000 7f599de00640 1 --2- 172.21.15.119:0/2563519664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59980cd630 0x7f599809a450 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f598c009870 tx=0x7f598c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8971d014f8f6b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.668+0000 7f599ca00640 1 -- 172.21.15.119:0/2563519664 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c037070 con 0x7f59980cd630 2024-04-25T10:18:16.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.669+0000 7f599ca00640 1 -- 172.21.15.119:0/2563519664 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598c032440 con 0x7f59980cd630 2024-04-25T10:18:16.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.669+0000 7f599ca00640 1 -- 172.21.15.119:0/2563519664 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c032810 con 0x7f59980cd630 2024-04-25T10:18:16.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.669+0000 7f599f200640 1 -- 172.21.15.119:0/2563519664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59980cd630 msgr2=0x7f599809a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.669+0000 7f599f200640 1 --2- 172.21.15.119:0/2563519664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59980cd630 0x7f599809a450 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f598c009870 tx=0x7f598c02a2a0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.669+0000 7f599f200640 1 -- 172.21.15.119:0/2563519664 shutdown_connections 2024-04-25T10:18:16.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.669+0000 7f599f200640 1 --2- 172.21.15.119:0/2563519664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59980cd630 0x7f599809a450 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.669+0000 7f599f200640 1 -- 172.21.15.119:0/2563519664 >> 172.21.15.119:0/2563519664 conn(0x7f59980b3130 msgr2=0x7f59980b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:16.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.669+0000 7f599f200640 1 -- 172.21.15.119:0/2563519664 shutdown_connections 2024-04-25T10:18:16.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.669+0000 7f599f200640 1 -- 172.21.15.119:0/2563519664 wait complete. 2024-04-25T10:18:16.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.671+0000 7f599f200640 1 Processor -- start 2024-04-25T10:18:16.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.671+0000 7f599f200640 1 -- start start 2024-04-25T10:18:16.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.672+0000 7f599f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599806cdd0 0x7f59981f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.672+0000 7f599f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59980cd630 0x7f59981594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.672+0000 7f599f200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59980cf880 0x7f5998159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.672+0000 7f599f200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599806c0f0 con 0x7f59980cf880 2024-04-25T10:18:16.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.672+0000 7f599f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59980cda10 con 0x7f59980cd630 2024-04-25T10:18:16.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.672+0000 7f599f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59980d0f90 con 0x7f599806cdd0 2024-04-25T10:18:16.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.673+0000 7f599e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59980cf880 0x7f5998159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.673+0000 7f599e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59980cf880 0x7f5998159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55870/0 (socket says 172.21.15.119:55870) 2024-04-25T10:18:16.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.673+0000 7f599e800640 1 -- 172.21.15.119:0/2917066160 learned_addr learned my addr 172.21.15.119:0/2917066160 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:16.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.673+0000 7f599d400640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59980cd630 0x7f59981594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.673+0000 7f599de00640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599806cdd0 0x7f59981f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.673+0000 7f599e800640 1 -- 172.21.15.119:0/2917066160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599806cdd0 msgr2=0x7f59981f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.673+0000 7f599e800640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599806cdd0 0x7f59981f36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.673+0000 7f599e800640 1 -- 172.21.15.119:0/2917066160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59980cd630 msgr2=0x7f59981594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.673+0000 7f599e800640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59980cd630 0x7f59981594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.673+0000 7f599e800640 1 -- 172.21.15.119:0/2917066160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59981588b0 con 0x7f59980cf880 2024-04-25T10:18:16.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.673+0000 7f599d400640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59980cd630 0x7f59981594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:16.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.673+0000 7f599e800640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59980cf880 0x7f5998159a20 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f599400c730 tx=0x7f599400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.673+0000 7f5992000640 1 -- 172.21.15.119:0/2917066160 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994017070 con 0x7f59980cf880 2024-04-25T10:18:16.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.674+0000 7f5992000640 1 -- 172.21.15.119:0/2917066160 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599400ce40 con 0x7f59980cf880 2024-04-25T10:18:16.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.674+0000 7f599f200640 1 -- 172.21.15.119:0/2917066160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599809dd40 con 0x7f59980cf880 2024-04-25T10:18:16.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.674+0000 7f5992000640 1 -- 172.21.15.119:0/2917066160 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599400a840 con 0x7f59980cf880 2024-04-25T10:18:16.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.674+0000 7f599f200640 1 -- 172.21.15.119:0/2917066160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599809e2c0 con 0x7f59980cf880 2024-04-25T10:18:16.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.676+0000 7f5992000640 1 -- 172.21.15.119:0/2917066160 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f599401a070 con 0x7f59980cf880 2024-04-25T10:18:16.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.677+0000 7f5992000640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f597806b4d0 0x7f597806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.678+0000 7f5992000640 1 -- 172.21.15.119:0/2917066160 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f59940897f0 con 0x7f59980cf880 2024-04-25T10:18:16.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.678+0000 7f599de00640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f597806b4d0 0x7f597806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.678+0000 7f59a3580740 1 Processor -- start 2024-04-25T10:18:16.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.678+0000 7f59a3580740 1 -- start start 2024-04-25T10:18:16.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.678+0000 7f599de00640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f597806b4d0 0x7f597806d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f598c034000 tx=0x7f598c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.679+0000 7f59a3580740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de821eeb30 0x55de82216970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.679+0000 7f59a3580740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de82216eb0 0x55de821eef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.679+0000 7f59a3580740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de822377a0 0x55de821ed2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.679+0000 7f59a3580740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de82152620 con 0x55de822377a0 2024-04-25T10:18:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.679+0000 7f59a3580740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de820fbca0 con 0x55de82216eb0 2024-04-25T10:18:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.679+0000 7f59a3580740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de8214e6c0 con 0x55de821eeb30 2024-04-25T10:18:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.679+0000 7f599e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de822377a0 0x55de821ed2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.679+0000 7f599e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de822377a0 0x55de821ed2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55882/0 (socket says 172.21.15.119:55882) 2024-04-25T10:18:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.679+0000 7f599e800640 1 -- 172.21.15.119:0/3474656559 learned_addr learned my addr 172.21.15.119:0/3474656559 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.680+0000 7f599de00640 1 --2- 172.21.15.119:0/3474656559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de821eeb30 0x55de82216970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.680+0000 7f599e800640 1 -- 172.21.15.119:0/3474656559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de821eeb30 msgr2=0x55de82216970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.680+0000 7f599e800640 1 --2- 172.21.15.119:0/3474656559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de821eeb30 0x55de82216970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.680+0000 7f599e800640 1 -- 172.21.15.119:0/3474656559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de82216eb0 msgr2=0x55de821eef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.680+0000 7f599d400640 1 --2- 172.21.15.119:0/3474656559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de82216eb0 0x55de821eef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.680+0000 7f599e800640 1 --2- 172.21.15.119:0/3474656559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de82216eb0 0x55de821eef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.680+0000 7f599e800640 1 -- 172.21.15.119:0/3474656559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de82102f40 con 0x55de822377a0 2024-04-25T10:18:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.680+0000 7f599e800640 1 --2- 172.21.15.119:0/3474656559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de822377a0 0x55de821ed2f0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f5994057b90 tx=0x7f5994057920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c07515bb8cd3c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.680+0000 7f598a600640 1 -- 172.21.15.119:0/3474656559 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599406f020 con 0x55de822377a0 2024-04-25T10:18:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.680+0000 7f598a600640 1 -- 172.21.15.119:0/3474656559 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599405ce50 con 0x55de822377a0 2024-04-25T10:18:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.680+0000 7f598a600640 1 -- 172.21.15.119:0/3474656559 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599405b070 con 0x55de822377a0 2024-04-25T10:18:16.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.681+0000 7f59a3580740 1 -- 172.21.15.119:0/3474656559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de822377a0 msgr2=0x55de821ed2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.681+0000 7f59a3580740 1 --2- 172.21.15.119:0/3474656559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de822377a0 0x55de821ed2f0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f5994057b90 tx=0x7f5994057920 comp rx=0 tx=0).stop 2024-04-25T10:18:16.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.681+0000 7f59a3580740 1 -- 172.21.15.119:0/3474656559 shutdown_connections 2024-04-25T10:18:16.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.681+0000 7f59a3580740 1 --2- 172.21.15.119:0/3474656559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de822377a0 0x55de821ed2f0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.681+0000 7f59a3580740 1 --2- 172.21.15.119:0/3474656559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de82216eb0 0x55de821eef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.681+0000 7f59a3580740 1 --2- 172.21.15.119:0/3474656559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de821eeb30 0x55de82216970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.681+0000 7f59a3580740 1 -- 172.21.15.119:0/3474656559 >> 172.21.15.119:0/3474656559 conn(0x55de821ebd50 msgr2=0x55de82234730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.681+0000 7f59a3580740 1 -- 172.21.15.119:0/3474656559 shutdown_connections 2024-04-25T10:18:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.681+0000 7f59a3580740 1 -- 172.21.15.119:0/3474656559 wait complete. 2024-04-25T10:18:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.682+0000 7f59a3580740 1 -- start start 2024-04-25T10:18:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.682+0000 7f59a3580740 1 Processor -- start 2024-04-25T10:18:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.683+0000 7f59a3580740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de82216eb0 0x55de82203d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.683+0000 7f59a3580740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de822377a0 0x55de822180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.683+0000 7f59a3580740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82204270 0x55de82237de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.683+0000 7f59a3580740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de822164e0 con 0x55de82204270 2024-04-25T10:18:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.683+0000 7f59a3580740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de820f11f0 con 0x55de822377a0 2024-04-25T10:18:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.683+0000 7f59a3580740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de82215ed0 con 0x55de82216eb0 2024-04-25T10:18:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.683+0000 7f59a3580740 10 client.0 _ll_register_callbacks cb 0x55de8210ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.683+0000 7f599e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82204270 0x55de82237de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.683+0000 7f599e800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82204270 0x55de82237de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55888/0 (socket says 172.21.15.119:55888) 2024-04-25T10:18:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.683+0000 7f599e800640 1 -- 172.21.15.119:0/73152659 learned_addr learned my addr 172.21.15.119:0/73152659 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f599de00640 1 --2- 172.21.15.119:0/73152659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de82216eb0 0x55de82203d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f599e800640 1 -- 172.21.15.119:0/73152659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de82216eb0 msgr2=0x55de82203d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f599d400640 1 --2- 172.21.15.119:0/73152659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de822377a0 0x55de822180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f599e800640 1 --2- 172.21.15.119:0/73152659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de82216eb0 0x55de82203d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f599e800640 1 -- 172.21.15.119:0/73152659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de822377a0 msgr2=0x55de822180e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f599e800640 1 --2- 172.21.15.119:0/73152659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de822377a0 0x55de822180e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f599e800640 1 -- 172.21.15.119:0/73152659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de82110c50 con 0x55de82204270 2024-04-25T10:18:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f599e800640 1 --2- 172.21.15.119:0/73152659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82204270 0x55de82237de0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f59940606b0 tx=0x7f5994001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7674aee9f33e0f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f597ea00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:18:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f597ea00640 1 -- 172.21.15.119:0/73152659 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599406f040 con 0x55de82204270 2024-04-25T10:18:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f597ea00640 1 -- 172.21.15.119:0/73152659 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f5994060270 con 0x55de82204270 2024-04-25T10:18:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f59a3580740 1 -- 172.21.15.119:0/73152659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f599405ce50 con 0x55de82204270 2024-04-25T10:18:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f597ea00640 1 -- 172.21.15.119:0/73152659 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7f599405e200 con 0x55de82204270 2024-04-25T10:18:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f59a3580740 10 client.9207 fetch_fsmap learned FSMap version 112 2024-04-25T10:18:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.684+0000 7f59a3580740 1 -- 172.21.15.119:0/73152659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55de81e672d0 con 0x55de82204270 2024-04-25T10:18:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.685+0000 7f597ea00640 1 -- 172.21.15.119:0/73152659 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599405ce50 con 0x55de82204270 2024-04-25T10:18:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.685+0000 7f597ea00640 1 -- 172.21.15.119:0/73152659 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 112) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f599406fb70 con 0x55de82204270 2024-04-25T10:18:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.685+0000 7f59a3580740 10 client.9207 fetch_fsmap finished waiting for FSMap version 112 2024-04-25T10:18:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.685+0000 7f59a3580740 10 client.9207 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:18:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.685+0000 7f59a3580740 20 client.9207 populate_metadata read hostname 'smithi119' 2024-04-25T10:18:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.685+0000 7f59a3580740 1 --2- 172.21.15.119:0/73152659 >> [v2:172.21.15.119:6836/2618740513,v1:172.21.15.119:6837/2618740513] conn(0x55de8224fe30 0x55de82252220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.685+0000 7f59a3580740 4 client.9207 mds_command: new command op to 9117 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.685+0000 7f59a3580740 1 -- 172.21.15.119:0/73152659 --> [v2:172.21.15.119:6836/2618740513,v1:172.21.15.119:6837/2618740513] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de8215fb60 con 0x55de8224fe30 2024-04-25T10:18:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.685+0000 7f599de00640 1 --2- 172.21.15.119:0/73152659 >> [v2:172.21.15.119:6836/2618740513,v1:172.21.15.119:6837/2618740513] conn(0x55de8224fe30 0x55de82252220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.686+0000 7f599de00640 1 --2- 172.21.15.119:0/73152659 >> [v2:172.21.15.119:6836/2618740513,v1:172.21.15.119:6837/2618740513] conn(0x55de8224fe30 0x55de82252220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b186a4dfb74cca64 server_cookie=41f20817ea1ba788 in_seq=0 out_seq=0 2024-04-25T10:18:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.686+0000 7f597ea00640 10 client.9207 ms_handle_connect on v2:172.21.15.119:6836/2618740513 2024-04-25T10:18:16.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.686+0000 7f597ea00640 1 -- 172.21.15.119:0/73152659 <== mds.0 v2:172.21.15.119:6836/2618740513 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55de8215fb60 con 0x55de8224fe30 2024-04-25T10:18:16.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.686+0000 7f597ea00640 10 client.9207 handle_command_reply: tid=0 2024-04-25T10:18:16.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.686+0000 7f59a3580740 1 client.9207 shutdown 2024-04-25T10:18:16.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.687+0000 7f59a3580740 1 -- 172.21.15.119:0/73152659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82204270 msgr2=0x55de82237de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.687+0000 7f59a3580740 1 --2- 172.21.15.119:0/73152659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82204270 0x55de82237de0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f59940606b0 tx=0x7f5994001250 comp rx=0 tx=0).stop 2024-04-25T10:18:16.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.687+0000 7f59a3580740 1 -- 172.21.15.119:0/73152659 shutdown_connections 2024-04-25T10:18:16.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.687+0000 7f59a3580740 1 --2- 172.21.15.119:0/73152659 >> [v2:172.21.15.119:6836/2618740513,v1:172.21.15.119:6837/2618740513] conn(0x55de8224fe30 0x55de82252220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.687+0000 7f59a3580740 1 --2- 172.21.15.119:0/73152659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82204270 0x55de82237de0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.687+0000 7f59a3580740 1 --2- 172.21.15.119:0/73152659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de822377a0 0x55de822180e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.687+0000 7f59a3580740 1 --2- 172.21.15.119:0/73152659 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de82216eb0 0x55de82203d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.687+0000 7f597ea00640 0 client.9207 ms_handle_reset on v2:172.21.15.119:6836/2618740513 2024-04-25T10:18:16.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.687+0000 7f59a3580740 1 -- 172.21.15.119:0/73152659 >> 172.21.15.119:0/73152659 conn(0x55de821eeb30 msgr2=0x55de82234530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:16.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.687+0000 7f59a3580740 1 -- 172.21.15.119:0/73152659 shutdown_connections 2024-04-25T10:18:16.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.687+0000 7f59a3580740 1 -- 172.21.15.119:0/73152659 wait complete. 2024-04-25T10:18:16.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.687+0000 7f59a3580740 20 client.9207 trim_cache size 0 max 16384 2024-04-25T10:18:16.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.703+0000 7f59a3580740 1 Processor -- start 2024-04-25T10:18:16.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.703+0000 7f59a3580740 1 -- start start 2024-04-25T10:18:16.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.704+0000 7f59a3580740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de82204270 0x55de82237c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.704+0000 7f59a3580740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82216eb0 0x55de822033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.704+0000 7f59a3580740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de822377a0 0x55de82233400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.704+0000 7f59a3580740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59981f8200 con 0x55de82216eb0 2024-04-25T10:18:16.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.704+0000 7f59a3580740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59981f7600 con 0x55de822377a0 2024-04-25T10:18:16.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.704+0000 7f59a3580740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59981f7c00 con 0x55de82204270 2024-04-25T10:18:16.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.705+0000 7f599d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82216eb0 0x55de822033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.705+0000 7f599d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82216eb0 0x55de822033e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55894/0 (socket says 172.21.15.119:55894) 2024-04-25T10:18:16.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.705+0000 7f599d400640 1 -- 172.21.15.119:0/536611770 learned_addr learned my addr 172.21.15.119:0/536611770 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:16.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.705+0000 7f599de00640 1 --2- 172.21.15.119:0/536611770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de82204270 0x55de82237c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.705+0000 7f599e800640 1 --2- 172.21.15.119:0/536611770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de822377a0 0x55de82233400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.705+0000 7f599d400640 1 -- 172.21.15.119:0/536611770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de82204270 msgr2=0x55de82237c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:16.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.705+0000 7f599d400640 1 --2- 172.21.15.119:0/536611770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de82204270 0x55de82237c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.705+0000 7f599d400640 1 -- 172.21.15.119:0/536611770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de822377a0 msgr2=0x55de82233400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:16.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.705+0000 7f599d400640 1 --2- 172.21.15.119:0/536611770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de822377a0 0x55de82233400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.705+0000 7f599d400640 1 -- 172.21.15.119:0/536611770 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de81f316b0 con 0x55de82216eb0 2024-04-25T10:18:16.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.705+0000 7f599de00640 1 --2- 172.21.15.119:0/536611770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de82204270 0x55de82237c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:16.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.705+0000 7f599d400640 1 --2- 172.21.15.119:0/536611770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82216eb0 0x55de822033e0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f599815c250 tx=0x7f59840019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede119d6734e01a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.706+0000 7f5989c00640 1 -- 172.21.15.119:0/536611770 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59840052e0 con 0x55de82216eb0 2024-04-25T10:18:16.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.706+0000 7f5989c00640 1 -- 172.21.15.119:0/536611770 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5984007030 con 0x55de82216eb0 2024-04-25T10:18:16.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.706+0000 7f5989c00640 1 -- 172.21.15.119:0/536611770 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984007420 con 0x55de82216eb0 2024-04-25T10:18:16.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.706+0000 7f59a3580740 1 -- 172.21.15.119:0/536611770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82216eb0 msgr2=0x55de822033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.706+0000 7f59a3580740 1 --2- 172.21.15.119:0/536611770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82216eb0 0x55de822033e0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f599815c250 tx=0x7f59840019b0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.706+0000 7f59a3580740 1 -- 172.21.15.119:0/536611770 shutdown_connections 2024-04-25T10:18:16.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.706+0000 7f59a3580740 1 --2- 172.21.15.119:0/536611770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de822377a0 0x55de82233400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.706+0000 7f59a3580740 1 --2- 172.21.15.119:0/536611770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82216eb0 0x55de822033e0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.706+0000 7f59a3580740 1 --2- 172.21.15.119:0/536611770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de82204270 0x55de82237c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.706+0000 7f59a3580740 1 -- 172.21.15.119:0/536611770 >> 172.21.15.119:0/536611770 conn(0x55de821eeb30 msgr2=0x55de8221b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:16.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.706+0000 7f59a3580740 1 -- 172.21.15.119:0/536611770 shutdown_connections 2024-04-25T10:18:16.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.706+0000 7f59a3580740 1 -- 172.21.15.119:0/536611770 wait complete. 2024-04-25T10:18:16.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.707+0000 7f59a3580740 1 -- start start 2024-04-25T10:18:16.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.707+0000 7f59a3580740 1 Processor -- start 2024-04-25T10:18:16.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.708+0000 7f59a3580740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de82204270 0x55de82203cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.708+0000 7f59a3580740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82216eb0 0x55de82233400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.708+0000 7f59a3580740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de822377a0 0x55de82233940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.708+0000 7f59a3580740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de822164e0 con 0x55de82216eb0 2024-04-25T10:18:16.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.708+0000 7f59a3580740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de820f11f0 con 0x55de82204270 2024-04-25T10:18:16.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.708+0000 7f59a3580740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de8224e0e0 con 0x55de822377a0 2024-04-25T10:18:16.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.708+0000 7f59a3580740 10 client.0 _ll_register_callbacks cb 0x55de82216e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:16.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.708+0000 7f599d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82216eb0 0x55de82233400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.708+0000 7f599d400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82216eb0 0x55de82233400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55908/0 (socket says 172.21.15.119:55908) 2024-04-25T10:18:16.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.708+0000 7f599d400640 1 -- 172.21.15.119:0/2212804171 learned_addr learned my addr 172.21.15.119:0/2212804171 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:16.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.709+0000 7f599e800640 1 --2- 172.21.15.119:0/2212804171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de822377a0 0x55de82233940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.709+0000 7f599d400640 1 -- 172.21.15.119:0/2212804171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de822377a0 msgr2=0x55de82233940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:16.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.709+0000 7f599de00640 1 --2- 172.21.15.119:0/2212804171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de82204270 0x55de82203cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.709+0000 7f599d400640 1 --2- 172.21.15.119:0/2212804171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de822377a0 0x55de82233940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.709+0000 7f599d400640 1 -- 172.21.15.119:0/2212804171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de82204270 msgr2=0x55de82203cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:16.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.709+0000 7f599d400640 1 --2- 172.21.15.119:0/2212804171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de82204270 0x55de82203cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.709+0000 7f599d400640 1 -- 172.21.15.119:0/2212804171 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de822033e0 con 0x55de82216eb0 2024-04-25T10:18:16.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.709+0000 7f599d400640 1 --2- 172.21.15.119:0/2212804171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82216eb0 0x55de82233400 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f59840019f0 tx=0x7f5984000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671f86a971075b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.709+0000 7f597ea00640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-25T10:18:16.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.709+0000 7f597ea00640 1 -- 172.21.15.119:0/2212804171 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984022070 con 0x55de82216eb0 2024-04-25T10:18:16.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.709+0000 7f597ea00640 1 -- 172.21.15.119:0/2212804171 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f59840069a0 con 0x55de82216eb0 2024-04-25T10:18:16.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.709+0000 7f59a3580740 1 -- 172.21.15.119:0/2212804171 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5984007030 con 0x55de82216eb0 2024-04-25T10:18:16.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.710+0000 7f597ea00640 1 -- 172.21.15.119:0/2212804171 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7f598402d050 con 0x55de82216eb0 2024-04-25T10:18:16.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.710+0000 7f59a3580740 10 client.9213 fetch_fsmap learned FSMap version 112 2024-04-25T10:18:16.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.710+0000 7f59a3580740 1 -- 172.21.15.119:0/2212804171 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55de8229fb80 con 0x55de82216eb0 2024-04-25T10:18:16.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.710+0000 7f597ea00640 1 -- 172.21.15.119:0/2212804171 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5984007030 con 0x55de82216eb0 2024-04-25T10:18:16.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.710+0000 7f597ea00640 1 -- 172.21.15.119:0/2212804171 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 112) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5984030020 con 0x55de82216eb0 2024-04-25T10:18:16.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.710+0000 7f59a3580740 10 client.9213 fetch_fsmap finished waiting for FSMap version 112 2024-04-25T10:18:16.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.710+0000 7f59a3580740 10 client.9213 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:18:16.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.710+0000 7f59a3580740 20 client.9213 populate_metadata read hostname 'smithi119' 2024-04-25T10:18:16.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.710+0000 7f59a3580740 1 --2- 172.21.15.119:0/2212804171 >> [v2:172.21.15.119:6836/2618740513,v1:172.21.15.119:6837/2618740513] conn(0x55de8229eb00 0x55de8229eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.710+0000 7f59a3580740 4 client.9213 mds_command: new command op to 9117 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:18:16.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.710+0000 7f59a3580740 1 -- 172.21.15.119:0/2212804171 --> [v2:172.21.15.119:6836/2618740513,v1:172.21.15.119:6837/2618740513] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55de822198f0 con 0x55de8229eb00 2024-04-25T10:18:16.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.711+0000 7f599de00640 1 --2- 172.21.15.119:0/2212804171 >> [v2:172.21.15.119:6836/2618740513,v1:172.21.15.119:6837/2618740513] conn(0x55de8229eb00 0x55de8229eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.711+0000 7f599de00640 1 --2- 172.21.15.119:0/2212804171 >> [v2:172.21.15.119:6836/2618740513,v1:172.21.15.119:6837/2618740513] conn(0x55de8229eb00 0x55de8229eee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26aed8bba77343ad server_cookie=4fa8a6c2d41755ad in_seq=0 out_seq=0 2024-04-25T10:18:16.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.711+0000 7f597ea00640 10 client.9213 ms_handle_connect on v2:172.21.15.119:6836/2618740513 2024-04-25T10:18:16.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.711+0000 7f597ea00640 1 -- 172.21.15.119:0/2212804171 <== mds.0 v2:172.21.15.119:6836/2618740513 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55de822198f0 con 0x55de8229eb00 2024-04-25T10:18:16.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.711+0000 7f597ea00640 10 client.9213 handle_command_reply: tid=0 2024-04-25T10:18:16.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.711+0000 7f59a3580740 1 client.9213 shutdown 2024-04-25T10:18:16.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.711+0000 7f59a3580740 1 -- 172.21.15.119:0/2212804171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82216eb0 msgr2=0x55de82233400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.711+0000 7f59a3580740 1 --2- 172.21.15.119:0/2212804171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82216eb0 0x55de82233400 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f59840019f0 tx=0x7f5984000fb0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.712+0000 7f59a3580740 1 -- 172.21.15.119:0/2212804171 shutdown_connections 2024-04-25T10:18:16.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.712+0000 7f59a3580740 1 --2- 172.21.15.119:0/2212804171 >> [v2:172.21.15.119:6836/2618740513,v1:172.21.15.119:6837/2618740513] conn(0x55de8229eb00 0x55de8229eee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.712+0000 7f59a3580740 1 --2- 172.21.15.119:0/2212804171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de822377a0 0x55de82233940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.712+0000 7f59a3580740 1 --2- 172.21.15.119:0/2212804171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de82216eb0 0x55de82233400 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.712+0000 7f59a3580740 1 --2- 172.21.15.119:0/2212804171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de82204270 0x55de82203cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.712+0000 7f597ea00640 0 client.9213 ms_handle_reset on v2:172.21.15.119:6836/2618740513 2024-04-25T10:18:16.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.712+0000 7f59a3580740 1 -- 172.21.15.119:0/2212804171 >> 172.21.15.119:0/2212804171 conn(0x55de821eeb30 msgr2=0x55de82211940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:16.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.712+0000 7f59a3580740 1 -- 172.21.15.119:0/2212804171 shutdown_connections 2024-04-25T10:18:16.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.712+0000 7f59a3580740 1 -- 172.21.15.119:0/2212804171 wait complete. 2024-04-25T10:18:16.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.712+0000 7f59a3580740 20 client.9213 trim_cache size 0 max 16384 2024-04-25T10:18:16.721 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:18:16.721 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "e0b0f428-b441-4237-a4d1-ed53321f87a5", 2024-04-25T10:18:16.721 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2024-04-25T10:18:16.721 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 9117, 2024-04-25T10:18:16.721 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2024-04-25T10:18:16.721 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2024-04-25T10:18:16.721 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2024-04-25T10:18:16.722 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 4.9097673520000003, 2024-04-25T10:18:16.722 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 112, 2024-04-25T10:18:16.722 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 112, 2024-04-25T10:18:16.722 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 112, 2024-04-25T10:18:16.722 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 6.958126247 2024-04-25T10:18:16.722 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:18:16.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.712+0000 7f599f200640 1 -- 172.21.15.119:0/2917066160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f597806b4d0 msgr2=0x7f597806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.712+0000 7f599f200640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f597806b4d0 0x7f597806d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f598c034000 tx=0x7f598c02e000 comp rx=0 tx=0).stop 2024-04-25T10:18:16.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.712+0000 7f599f200640 1 -- 172.21.15.119:0/2917066160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59980cf880 msgr2=0x7f5998159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.712+0000 7f599f200640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59980cf880 0x7f5998159a20 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f599400c730 tx=0x7f599400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.713+0000 7f599f200640 1 -- 172.21.15.119:0/2917066160 shutdown_connections 2024-04-25T10:18:16.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.713+0000 7f599f200640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f597806b4d0 0x7f597806d990 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.713+0000 7f599f200640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59980cf880 0x7f5998159a20 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.713+0000 7f599f200640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59980cd630 0x7f59981594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.713+0000 7f599f200640 1 --2- 172.21.15.119:0/2917066160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f599806cdd0 0x7f59981f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.713+0000 7f599f200640 1 -- 172.21.15.119:0/2917066160 >> 172.21.15.119:0/2917066160 conn(0x7f59980b3130 msgr2=0x7f599815ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:16.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.713+0000 7f599f200640 1 -- 172.21.15.119:0/2917066160 shutdown_connections 2024-04-25T10:18:16.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.713+0000 7f599f200640 1 -- 172.21.15.119:0/2917066160 wait complete. 2024-04-25T10:18:16.731 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:18:16.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.902+0000 7f6317800640 1 Processor -- start 2024-04-25T10:18:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.902+0000 7f6317800640 1 -- start start 2024-04-25T10:18:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.903+0000 7f6317800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f631015bf00 0x7f63100cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.903+0000 7f6317800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63100ce6c0 con 0x7f631006e8f0 2024-04-25T10:18:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.903+0000 7f6317800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63100cc7b0 con 0x7f63100cc310 2024-04-25T10:18:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.903+0000 7f6317800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63101455e0 con 0x7f631015bf00 2024-04-25T10:18:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.904+0000 7f6314e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f631015bf00 0x7f63100cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.904+0000 7f6315800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f631006e8f0 0x7f63101654b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38082/0 (socket says 172.21.15.119:38082) 2024-04-25T10:18:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.904+0000 7f6315800640 1 -- 172.21.15.119:0/1513294692 learned_addr learned my addr 172.21.15.119:0/1513294692 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.904+0000 7f6314e00640 1 -- 172.21.15.119:0/1513294692 >> v1:172.21.15.132:6789/0 conn(0x7f63100cc310 legacy=0x7f631015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.904+0000 7f6314e00640 1 -- 172.21.15.119:0/1513294692 >> v1:172.21.15.119:6789/0 conn(0x7f631006e8f0 legacy=0x7f63101654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.904+0000 7f6314e00640 1 -- 172.21.15.119:0/1513294692 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63100cca90 con 0x7f631015bf00 2024-04-25T10:18:16.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.905+0000 7f6314e00640 1 --2- 172.21.15.119:0/1513294692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631015bf00 0x7f63100cbdd0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f63000097a0 tx=0x7f630002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb96de7c6d78918 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.905+0000 7f630f400640 1 -- 172.21.15.119:0/1513294692 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300037070 con 0x7f631015bf00 2024-04-25T10:18:16.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.905+0000 7f630f400640 1 -- 172.21.15.119:0/1513294692 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6300032440 con 0x7f631015bf00 2024-04-25T10:18:16.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.905+0000 7f630f400640 1 -- 172.21.15.119:0/1513294692 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300032810 con 0x7f631015bf00 2024-04-25T10:18:16.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.906+0000 7f6317800640 1 -- 172.21.15.119:0/1513294692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631015bf00 msgr2=0x7f63100cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.906+0000 7f6317800640 1 --2- 172.21.15.119:0/1513294692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631015bf00 0x7f63100cbdd0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f63000097a0 tx=0x7f630002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.906+0000 7f6317800640 1 -- 172.21.15.119:0/1513294692 shutdown_connections 2024-04-25T10:18:16.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.906+0000 7f6317800640 1 --2- 172.21.15.119:0/1513294692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631015bf00 0x7f63100cbdd0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.906+0000 7f6317800640 1 -- 172.21.15.119:0/1513294692 >> 172.21.15.119:0/1513294692 conn(0x7f631006aba0 msgr2=0x7f631006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:16.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.906+0000 7f6317800640 1 -- 172.21.15.119:0/1513294692 shutdown_connections 2024-04-25T10:18:16.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.906+0000 7f6317800640 1 -- 172.21.15.119:0/1513294692 wait complete. 2024-04-25T10:18:16.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.908+0000 7f6317800640 1 Processor -- start 2024-04-25T10:18:16.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.908+0000 7f6317800640 1 -- start start 2024-04-25T10:18:16.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.909+0000 7f6317800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631006e8f0 0x7f6310156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.909+0000 7f6317800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63100cc310 0x7f6310159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.909+0000 7f6317800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f631015bf00 0x7f63101598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.909+0000 7f6317800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63100cd050 con 0x7f631006e8f0 2024-04-25T10:18:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.909+0000 7f6317800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631006d450 con 0x7f631015bf00 2024-04-25T10:18:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.909+0000 7f6317800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631006ee60 con 0x7f63100cc310 2024-04-25T10:18:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.909+0000 7f6314e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631006e8f0 0x7f6310156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.909+0000 7f6314e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631006e8f0 0x7f6310156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55930/0 (socket says 172.21.15.119:55930) 2024-04-25T10:18:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.909+0000 7f6314e00640 1 -- 172.21.15.119:0/2223625517 learned_addr learned my addr 172.21.15.119:0/2223625517 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.909+0000 7f6314e00640 1 -- 172.21.15.119:0/2223625517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63100cc310 msgr2=0x7f6310159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.909+0000 7f6314e00640 1 --2- 172.21.15.119:0/2223625517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63100cc310 0x7f6310159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.909+0000 7f6314e00640 1 -- 172.21.15.119:0/2223625517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f631015bf00 msgr2=0x7f63101598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.909+0000 7f6314e00640 1 --2- 172.21.15.119:0/2223625517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f631015bf00 0x7f63101598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.909+0000 7f6314e00640 1 -- 172.21.15.119:0/2223625517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6310156790 con 0x7f631006e8f0 2024-04-25T10:18:16.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.910+0000 7f6314e00640 1 --2- 172.21.15.119:0/2223625517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631006e8f0 0x7f6310156250 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f6300000c00 tx=0x7f63000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.910+0000 7f630cc00640 1 -- 172.21.15.119:0/2223625517 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300041070 con 0x7f631006e8f0 2024-04-25T10:18:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.910+0000 7f630cc00640 1 -- 172.21.15.119:0/2223625517 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630003c930 con 0x7f631006e8f0 2024-04-25T10:18:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.910+0000 7f6317800640 1 -- 172.21.15.119:0/2223625517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63101328e0 con 0x7f631006e8f0 2024-04-25T10:18:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.910+0000 7f630cc00640 1 -- 172.21.15.119:0/2223625517 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300037040 con 0x7f631006e8f0 2024-04-25T10:18:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.910+0000 7f6317800640 1 -- 172.21.15.119:0/2223625517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6310132ea0 con 0x7f631006e8f0 2024-04-25T10:18:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.911+0000 7f6317800640 1 -- 172.21.15.119:0/2223625517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63100749b0 con 0x7f631006e8f0 2024-04-25T10:18:16.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.912+0000 7f630cc00640 1 -- 172.21.15.119:0/2223625517 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6300044070 con 0x7f631006e8f0 2024-04-25T10:18:16.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.914+0000 7f630cc00640 1 --2- 172.21.15.119:0/2223625517 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62f006b4d0 0x7f62f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.914+0000 7f630cc00640 1 -- 172.21.15.119:0/2223625517 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f63000ab200 con 0x7f631006e8f0 2024-04-25T10:18:16.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.914+0000 7f630fe00640 1 --2- 172.21.15.119:0/2223625517 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62f006b4d0 0x7f62f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.914+0000 7f630fe00640 1 --2- 172.21.15.119:0/2223625517 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62f006b4d0 0x7f62f006d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f63100cfae0 tx=0x7f62fc007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:16.918+0000 7f630cc00640 1 -- 172.21.15.119:0/2223625517 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6310156790 con 0x7f631006e8f0 2024-04-25T10:18:17.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:17.151+0000 7f6317800640 1 -- 172.21.15.119:0/2223625517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6310133980 con 0x7f631006e8f0 2024-04-25T10:18:17.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:17.152+0000 7f630cc00640 1 -- 172.21.15.119:0/2223625517 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) v1 ==== 53+0+73 (secure 0 0 0) 0x7f630007a0a0 con 0x7f631006e8f0 2024-04-25T10:18:17.155 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:18:17.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:17.154+0000 7f6317800640 1 -- 172.21.15.119:0/2223625517 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62f006b4d0 msgr2=0x7f62f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:17.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:17.154+0000 7f6317800640 1 --2- 172.21.15.119:0/2223625517 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62f006b4d0 0x7f62f006d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f63100cfae0 tx=0x7f62fc007d30 comp rx=0 tx=0).stop 2024-04-25T10:18:17.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:17.154+0000 7f6317800640 1 -- 172.21.15.119:0/2223625517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631006e8f0 msgr2=0x7f6310156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:17.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:17.154+0000 7f6317800640 1 --2- 172.21.15.119:0/2223625517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631006e8f0 0x7f6310156250 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f6300000c00 tx=0x7f63000046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:17.154+0000 7f6317800640 1 -- 172.21.15.119:0/2223625517 shutdown_connections 2024-04-25T10:18:17.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:17.154+0000 7f6317800640 1 --2- 172.21.15.119:0/2223625517 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f62f006b4d0 0x7f62f006d990 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:17.154+0000 7f6317800640 1 --2- 172.21.15.119:0/2223625517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f631015bf00 0x7f63101598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:17.154+0000 7f6317800640 1 --2- 172.21.15.119:0/2223625517 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63100cc310 0x7f6310159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:17.154+0000 7f6317800640 1 --2- 172.21.15.119:0/2223625517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f631006e8f0 0x7f6310156250 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:17.154+0000 7f6317800640 1 -- 172.21.15.119:0/2223625517 >> 172.21.15.119:0/2223625517 conn(0x7f631006aba0 msgr2=0x7f6310143bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:17.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:17.155+0000 7f6317800640 1 -- 172.21.15.119:0/2223625517 shutdown_connections 2024-04-25T10:18:17.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:17.155+0000 7f6317800640 1 -- 172.21.15.119:0/2223625517 wait complete. 2024-04-25T10:18:17.176 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:18:17.176 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:18:17.177 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-04-25T10:18:17.177 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:18:17.177 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:17.177 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:18:17.177 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:18:17.177 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:18:17.177 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:18:17.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.177 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-04-25T10:18:17.229 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:18:17.229 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:18:17.229 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:18:17.229 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:17.229 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-04-25T10:18:17.229 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:17.229 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:18:17.230 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:17.230 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-04-25T10:18:17.230 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:18:17.230 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:17.230 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-04-25T10:18:17.230 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:18:17.230 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:17.230 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:18:17.230 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:17.230 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:18:17.230 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 41992sec preferred_lft 41992sec 2024-04-25T10:18:17.230 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-04-25T10:18:17.230 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:17.231 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:18:17.231 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:17.231 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:18:17.231 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:17.231 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::f834:f3ff:fe1f:222a/64 scope link 2024-04-25T10:18:17.231 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:17.231 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:18:17.231 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 82:52:48:a0:97:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:18:17.231 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::8052:48ff:fea0:970a/64 scope link 2024-04-25T10:18:17.231 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:17.231 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:18:17.231 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether aa:57:c3:49:4d:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:18:17.231 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a857:c3ff:fe49:4d42/64 scope link 2024-04-25T10:18:17.231 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:17.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.232 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:18:17.303 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:18:17.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.307 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-04-25T10:18:17.353 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-04-25T10:18:17.353 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:18:17.353 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:18:17.353 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:18:17.353 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:18:17.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.354 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2024-04-25T10:18:17.354 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-04-25T10:18:17.354 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:18:17.354 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:18:17.354 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:18:17.354 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-04-25T10:18:17.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.541 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:18:17.589 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:18:17.590 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:18:17.591 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:18:17.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.591 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-04-25T10:18:17.674 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:17.674 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:17.728 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:18:17.728 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:17.728 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:17.779 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:18:17.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.780 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:17.902 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:17.902 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:18:17.902 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:18:17.922 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:17.922 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:18:17.922 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:18:17.922 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:18:17.922 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:18:17.948 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:17.948 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:18:17.948 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-04-25T10:18:17.948 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.132:3300,172.21.15.132:3301" 2024-04-25T10:18:17.948 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:18:17.948 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790" 2024-04-25T10:18:17.948 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.132:6789,172.21.15.132:6790:/ 2024-04-25T10:18:17.948 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e0b0f428-b441-4237-a4d1-ed53321f87a5" will pass to kernel 2024-04-25T10:18:17.951 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:18:17.951 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:18:17.951 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:18:17.951 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:18:17.951 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:18:17.955 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:18:17.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.955 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:18.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:18.026 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:18.077 INFO:teuthology.orchestra.run.smithi119.stdout:client9225 2024-04-25T10:18:18.077 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:18:18.077 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:18.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:18.126 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:18:18.181 INFO:teuthology.orchestra.run.smithi119.stdout:./ 2024-04-25T10:18:18.181 INFO:teuthology.orchestra.run.smithi119.stdout:./parent 2024-04-25T10:18:18.181 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/flushed 2024-04-25T10:18:18.181 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/flushed/charlie 2024-04-25T10:18:18.181 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/flushed/alpha 2024-04-25T10:18:18.181 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/flushed/bravo 2024-04-25T10:18:18.181 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/unflushed 2024-04-25T10:18:18.181 INFO:teuthology.orchestra.run.smithi119.stdout:./parent/unflushed/jfile 2024-04-25T10:18:18.182 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:18.183 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:18.183 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:18.183 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:18.183 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:18:18.183 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:18.310 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-04-25T10:18:18.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:18.323 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:18.323 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:18.323 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:18.323 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:18.323 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:18:18.323 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:18.504 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627776 2024-04-25T10:18:18.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:18.518 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:18.518 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:18.518 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:18.518 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:18.518 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:18:18.518 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:18.700 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627777 2024-04-25T10:18:18.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:18.714 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:18.714 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:18.714 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:18.714 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:18.714 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:18:18.715 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:18.896 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627780 2024-04-25T10:18:18.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:18.909 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:18.909 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:18.910 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:18.910 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:18.910 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:18:18.910 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:19.093 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627778 2024-04-25T10:18:19.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:19.106 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:19.106 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:19.106 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:19.106 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:19.107 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T10:18:19.107 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:19.311 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627779 2024-04-25T10:18:19.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:19.326 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:19.327 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:19.327 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:19.327 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:19.327 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:18:19.327 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:19.511 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627781 2024-04-25T10:18:19.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:19.524 DEBUG:teuthology.orchestra.run.smithi119:> stdin-killer -- python3 -c ' 2024-04-25T10:18:19.524 DEBUG:teuthology.orchestra.run.smithi119:> import os 2024-04-25T10:18:19.524 DEBUG:teuthology.orchestra.run.smithi119:> import stat 2024-04-25T10:18:19.524 DEBUG:teuthology.orchestra.run.smithi119:> 2024-04-25T10:18:19.524 DEBUG:teuthology.orchestra.run.smithi119:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:18:19.524 DEBUG:teuthology.orchestra.run.smithi119:> ' 2024-04-25T10:18:19.707 INFO:teuthology.orchestra.run.smithi119.stdout:1099511627782 2024-04-25T10:18:19.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:18:19.721 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:18:19.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:18:19.874 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:18:19.986 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:18:19.986 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:18:19.987 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:20.044 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:18:20.044 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:20.096 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:20.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.298+0000 7f42ce000640 1 Processor -- start 2024-04-25T10:18:20.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.298+0000 7f42ce000640 1 -- start start 2024-04-25T10:18:20.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.299+0000 7f42ce000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f42c8144470 0x7f42c815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.300+0000 7f42ce000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c80cd440 con 0x7f42c80b5050 2024-04-25T10:18:20.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.300+0000 7f42ce000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c80cd720 con 0x7f42c806e430 2024-04-25T10:18:20.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.300+0000 7f42ce000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c8145340 con 0x7f42c8144470 2024-04-25T10:18:20.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.300+0000 7f42cd600640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f42c80b5050 0x7f42c815bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38084/0 (socket says 172.21.15.119:38084) 2024-04-25T10:18:20.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.300+0000 7f42cd600640 1 -- 172.21.15.119:0/2373807490 learned_addr learned my addr 172.21.15.119:0/2373807490 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:20.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.300+0000 7f42ccc00640 1 --2- 172.21.15.119:0/2373807490 >> v2:172.21.15.132:3300/0 conn(0x7f42c8144470 0x7f42c815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.300+0000 7f42c7400640 1 -- 172.21.15.119:0/2373807490 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2391177283 0 0) 0x7f42c80cd440 con 0x7f42c80b5050 2024-04-25T10:18:20.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.300+0000 7f42c7400640 1 -- 172.21.15.119:0/2373807490 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42b0003520 con 0x7f42c80b5050 2024-04-25T10:18:20.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.301+0000 7f42c7400640 1 -- 172.21.15.119:0/2373807490 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 609562276 0 0) 0x7f42b0003520 con 0x7f42c80b5050 2024-04-25T10:18:20.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.301+0000 7f42c7400640 1 -- 172.21.15.119:0/2373807490 >> v2:172.21.15.132:3300/0 conn(0x7f42c8144470 msgr2=0x7f42c815fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.301+0000 7f42c7400640 1 --2- 172.21.15.119:0/2373807490 >> v2:172.21.15.132:3300/0 conn(0x7f42c8144470 0x7f42c815fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.301+0000 7f42c7400640 1 -- 172.21.15.119:0/2373807490 >> v1:172.21.15.132:6789/0 conn(0x7f42c806e430 legacy=0x7f42c80b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.301+0000 7f42c7400640 1 -- 172.21.15.119:0/2373807490 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c80cda00 con 0x7f42c80b5050 2024-04-25T10:18:20.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.301+0000 7f42c7400640 1 -- 172.21.15.119:0/2373807490 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f42c00019f0 con 0x7f42c80b5050 2024-04-25T10:18:20.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.301+0000 7f42c7400640 1 -- 172.21.15.119:0/2373807490 >> v1:172.21.15.119:6789/0 conn(0x7f42c80b5050 legacy=0x7f42c815bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.301+0000 7f42c7400640 1 --2- 172.21.15.119:0/2373807490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42b0003a20 0x7f42b0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.301+0000 7f42c7400640 1 --2- 172.21.15.119:0/2373807490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42b0004800 0x7f42b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.301+0000 7f42c7400640 1 --2- 172.21.15.119:0/2373807490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b00051c0 0x7f42b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.301+0000 7f42c7400640 1 -- 172.21.15.119:0/2373807490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b000c0c0 con 0x7f42b0004800 2024-04-25T10:18:20.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.301+0000 7f42c7400640 1 -- 172.21.15.119:0/2373807490 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b000c270 con 0x7f42b00051c0 2024-04-25T10:18:20.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.301+0000 7f42c7400640 1 -- 172.21.15.119:0/2373807490 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b000c470 con 0x7f42b0003a20 2024-04-25T10:18:20.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.301+0000 7f42c7e00640 1 --2- 172.21.15.119:0/2373807490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42b0004800 0x7f42b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.302+0000 7f42cd600640 1 --2- 172.21.15.119:0/2373807490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b00051c0 0x7f42b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.302+0000 7f42c7e00640 1 -- 172.21.15.119:0/2373807490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42b0003a20 msgr2=0x7f42b0003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.302+0000 7f42c7e00640 1 --2- 172.21.15.119:0/2373807490 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42b0003a20 0x7f42b0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.302+0000 7f42c7e00640 1 -- 172.21.15.119:0/2373807490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b00051c0 msgr2=0x7f42b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.302+0000 7f42c7e00640 1 --2- 172.21.15.119:0/2373807490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b00051c0 0x7f42b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.302+0000 7f42c7e00640 1 -- 172.21.15.119:0/2373807490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b0003520 con 0x7f42b0004800 2024-04-25T10:18:20.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.302+0000 7f42cd600640 1 --2- 172.21.15.119:0/2373807490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42b00051c0 0x7f42b000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:20.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.302+0000 7f42cd600640 1 -- 172.21.15.119:0/2373807490 reap_dead start 2024-04-25T10:18:20.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.303+0000 7f42c7e00640 1 --2- 172.21.15.119:0/2373807490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42b0004800 0x7f42b0004c80 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f42b4001720 tx=0x7f42b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd580da28a718c01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.303+0000 7f42c7400640 1 -- 172.21.15.119:0/2373807490 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4001370 con 0x7f42b0004800 2024-04-25T10:18:20.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.303+0000 7f42c7400640 1 -- 172.21.15.119:0/2373807490 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b40052d0 con 0x7f42b0004800 2024-04-25T10:18:20.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.303+0000 7f42c7400640 1 -- 172.21.15.119:0/2373807490 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b402ab90 con 0x7f42b0004800 2024-04-25T10:18:20.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.304+0000 7f42ce000640 1 -- 172.21.15.119:0/2373807490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42b0004800 msgr2=0x7f42b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.304+0000 7f42ce000640 1 --2- 172.21.15.119:0/2373807490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42b0004800 0x7f42b0004c80 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f42b4001720 tx=0x7f42b0005830 comp rx=0 tx=0).stop 2024-04-25T10:18:20.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.304+0000 7f42ce000640 1 -- 172.21.15.119:0/2373807490 shutdown_connections 2024-04-25T10:18:20.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.304+0000 7f42ce000640 1 --2- 172.21.15.119:0/2373807490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42b0004800 0x7f42b0004c80 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.304+0000 7f42ce000640 1 -- 172.21.15.119:0/2373807490 >> 172.21.15.119:0/2373807490 conn(0x7f42c806b1d0 msgr2=0x7f42c80b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.304+0000 7f42ce000640 1 -- 172.21.15.119:0/2373807490 shutdown_connections 2024-04-25T10:18:20.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.304+0000 7f42ce000640 1 -- 172.21.15.119:0/2373807490 wait complete. 2024-04-25T10:18:20.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.306+0000 7f42ce000640 1 Processor -- start 2024-04-25T10:18:20.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.306+0000 7f42ce000640 1 -- start start 2024-04-25T10:18:20.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42ce000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42b0004800 0x7f42c80cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42ce000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42c813e1e0 0x7f42c81f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42ce000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42c815da90 0x7f42c815df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42ce000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c80ce200 con 0x7f42c815da90 2024-04-25T10:18:20.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42ce000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c8144850 con 0x7f42c813e1e0 2024-04-25T10:18:20.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42ce000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c815fc90 con 0x7f42b0004800 2024-04-25T10:18:20.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42cd600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42c815da90 0x7f42c815df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42cd600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42c815da90 0x7f42c815df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55948/0 (socket says 172.21.15.119:55948) 2024-04-25T10:18:20.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42cd600640 1 -- 172.21.15.119:0/3726261889 learned_addr learned my addr 172.21.15.119:0/3726261889 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:20.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42c7e00640 1 --2- 172.21.15.119:0/3726261889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42c813e1e0 0x7f42c81f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42cd600640 1 -- 172.21.15.119:0/3726261889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42b0004800 msgr2=0x7f42c80cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42cd600640 1 --2- 172.21.15.119:0/3726261889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42b0004800 0x7f42c80cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42cd600640 1 -- 172.21.15.119:0/3726261889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42c813e1e0 msgr2=0x7f42c81f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42cd600640 1 --2- 172.21.15.119:0/3726261889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42c813e1e0 0x7f42c81f7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.307+0000 7f42cd600640 1 -- 172.21.15.119:0/3726261889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c81f8250 con 0x7f42c815da90 2024-04-25T10:18:20.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.308+0000 7f42cd600640 1 --2- 172.21.15.119:0/3726261889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42c815da90 0x7f42c815df20 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f42c0001580 tx=0x7f42c000b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.308+0000 7f42c4c00640 1 -- 172.21.15.119:0/3726261889 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c000b8d0 con 0x7f42c815da90 2024-04-25T10:18:20.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.308+0000 7f42c4c00640 1 -- 172.21.15.119:0/3726261889 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c000ba90 con 0x7f42c815da90 2024-04-25T10:18:20.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.308+0000 7f42c4c00640 1 -- 172.21.15.119:0/3726261889 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c000be80 con 0x7f42c815da90 2024-04-25T10:18:20.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.308+0000 7f42ce000640 1 -- 172.21.15.119:0/3726261889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c81f3d10 con 0x7f42c815da90 2024-04-25T10:18:20.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.308+0000 7f42ce000640 1 -- 172.21.15.119:0/3726261889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c81f3fd0 con 0x7f42c815da90 2024-04-25T10:18:20.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.309+0000 7f42ce000640 1 -- 172.21.15.119:0/3726261889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c813ea80 con 0x7f42c815da90 2024-04-25T10:18:20.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.310+0000 7f42c4c00640 1 -- 172.21.15.119:0/3726261889 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42c001b060 con 0x7f42c815da90 2024-04-25T10:18:20.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.311+0000 7f42c4c00640 1 --2- 172.21.15.119:0/3726261889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42a806b500 0x7f42a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.311+0000 7f42c4c00640 1 -- 172.21.15.119:0/3726261889 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f42c008a190 con 0x7f42c815da90 2024-04-25T10:18:20.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.311+0000 7f42ccc00640 1 --2- 172.21.15.119:0/3726261889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42a806b500 0x7f42a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.312+0000 7f42ccc00640 1 --2- 172.21.15.119:0/3726261889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42a806b500 0x7f42a806d9c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f42bc006a90 tx=0x7f42bc006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.316+0000 7f42c4c00640 1 -- 172.21.15.119:0/3726261889 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42c81f8250 con 0x7f42c815da90 2024-04-25T10:18:20.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.513+0000 7f42ce000640 1 -- 172.21.15.119:0/3726261889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42c80749b0 con 0x7f42c815da90 2024-04-25T10:18:20.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.516+0000 7f42c4c00640 1 -- 172.21.15.119:0/3726261889 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+17079 (secure 0 0 0) 0x7f42c0058e80 con 0x7f42c815da90 2024-04-25T10:18:20.519 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:20.519 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":112,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:18:09.666428+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:35.422922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:36.634584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.25","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/2864575636":"2024-04-26T10:18:09.666363+0000","172.21.15.119:6836/2864575636":"2024-04-26T10:18:09.666363+0000","172.21.15.119:6837/1849584205":"2024-04-26T10:17:56.438859+0000","172.21.15.119:6836/1849584205":"2024-04-26T10:17:56.438859+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:20.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.518+0000 7f42ce000640 1 -- 172.21.15.119:0/3726261889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42a806b500 msgr2=0x7f42a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.518+0000 7f42ce000640 1 --2- 172.21.15.119:0/3726261889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42a806b500 0x7f42a806d9c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f42bc006a90 tx=0x7f42bc006a20 comp rx=0 tx=0).stop 2024-04-25T10:18:20.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.519+0000 7f42ce000640 1 -- 172.21.15.119:0/3726261889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42c815da90 msgr2=0x7f42c815df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.519+0000 7f42ce000640 1 --2- 172.21.15.119:0/3726261889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42c815da90 0x7f42c815df20 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f42c0001580 tx=0x7f42c000b0d0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.519+0000 7f42ce000640 1 -- 172.21.15.119:0/3726261889 shutdown_connections 2024-04-25T10:18:20.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.519+0000 7f42ce000640 1 --2- 172.21.15.119:0/3726261889 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f42a806b500 0x7f42a806d9c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.519+0000 7f42ce000640 1 --2- 172.21.15.119:0/3726261889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42c815da90 0x7f42c815df20 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.519+0000 7f42ce000640 1 --2- 172.21.15.119:0/3726261889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42c813e1e0 0x7f42c81f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.519+0000 7f42ce000640 1 --2- 172.21.15.119:0/3726261889 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42b0004800 0x7f42c80cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.519+0000 7f42ce000640 1 -- 172.21.15.119:0/3726261889 >> 172.21.15.119:0/3726261889 conn(0x7f42c806b1d0 msgr2=0x7f42c8142340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.519+0000 7f42ce000640 1 -- 172.21.15.119:0/3726261889 shutdown_connections 2024-04-25T10:18:20.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.519+0000 7f42ce000640 1 -- 172.21.15.119:0/3726261889 wait complete. 2024-04-25T10:18:20.542 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:35.422922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:17:36.634584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:20.542 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:20.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.746+0000 7f3dea800640 1 Processor -- start 2024-04-25T10:18:20.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.747+0000 7f3dea800640 1 -- start start 2024-04-25T10:18:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.748+0000 7f3dea800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de415dc80 0x7f3de4161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.748+0000 7f3dea800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3de40b97c0 0x7f3de40b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.748+0000 7f3dea800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de41627a0 con 0x7f3de40b5a30 2024-04-25T10:18:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.748+0000 7f3dea800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de40b5ea0 con 0x7f3de40b97c0 2024-04-25T10:18:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.748+0000 7f3dea800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de41621e0 con 0x7f3de415dc80 2024-04-25T10:18:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.748+0000 7f3de3e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3de40b5a30 0x7f3de406bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38094/0 (socket says 172.21.15.119:38094) 2024-04-25T10:18:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.749+0000 7f3de3e00640 1 -- 172.21.15.119:0/3638389822 learned_addr learned my addr 172.21.15.119:0/3638389822 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.749+0000 7f3de3400640 1 --2- 172.21.15.119:0/3638389822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de415dc80 0x7f3de4161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.749+0000 7f3de2a00640 1 --2- 172.21.15.119:0/3638389822 >> v2:172.21.15.132:3300/0 conn(0x7f3de40b97c0 0x7f3de40b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.749+0000 7f3de2000640 1 -- 172.21.15.119:0/3638389822 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733361971 0 0) 0x7f3de41627a0 con 0x7f3de40b5a30 2024-04-25T10:18:20.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.749+0000 7f3de2000640 1 -- 172.21.15.119:0/3638389822 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dc8003520 con 0x7f3de40b5a30 2024-04-25T10:18:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.749+0000 7f3de2000640 1 -- 172.21.15.119:0/3638389822 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3055019139 0 0) 0x7f3dc8003520 con 0x7f3de40b5a30 2024-04-25T10:18:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.749+0000 7f3de2000640 1 -- 172.21.15.119:0/3638389822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de415dc80 msgr2=0x7f3de4161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.749+0000 7f3de2000640 1 --2- 172.21.15.119:0/3638389822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de415dc80 0x7f3de4161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.749+0000 7f3de2000640 1 -- 172.21.15.119:0/3638389822 >> v2:172.21.15.132:3300/0 conn(0x7f3de40b97c0 msgr2=0x7f3de40b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.749+0000 7f3de2000640 1 --2- 172.21.15.119:0/3638389822 >> v2:172.21.15.132:3300/0 conn(0x7f3de40b97c0 0x7f3de40b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.749+0000 7f3de2000640 1 -- 172.21.15.119:0/3638389822 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de41624c0 con 0x7f3de40b5a30 2024-04-25T10:18:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.749+0000 7f3de2000640 1 -- 172.21.15.119:0/3638389822 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f3ddc001670 con 0x7f3de40b5a30 2024-04-25T10:18:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.749+0000 7f3de2000640 1 -- 172.21.15.119:0/3638389822 >> v1:172.21.15.119:6789/0 conn(0x7f3de40b5a30 legacy=0x7f3de406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.749+0000 7f3de2000640 1 --2- 172.21.15.119:0/3638389822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc80039b0 0x7f3dc8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.750+0000 7f3de2000640 1 --2- 172.21.15.119:0/3638389822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc8004800 0x7f3dc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.750+0000 7f3de2000640 1 --2- 172.21.15.119:0/3638389822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de415dc80 0x7f3dc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.750+0000 7f3de2000640 1 -- 172.21.15.119:0/3638389822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40b5ea0 con 0x7f3dc8004800 2024-04-25T10:18:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.750+0000 7f3de2000640 1 -- 172.21.15.119:0/3638389822 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de41621e0 con 0x7f3dc80039b0 2024-04-25T10:18:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.750+0000 7f3de2000640 1 -- 172.21.15.119:0/3638389822 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc800b9c0 con 0x7f3de415dc80 2024-04-25T10:18:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.750+0000 7f3de2a00640 1 --2- 172.21.15.119:0/3638389822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc8004800 0x7f3dc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.750+0000 7f3de3400640 1 --2- 172.21.15.119:0/3638389822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc80039b0 0x7f3dc8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.750+0000 7f3de3e00640 1 --2- 172.21.15.119:0/3638389822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de415dc80 0x7f3dc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.750+0000 7f3de2a00640 1 -- 172.21.15.119:0/3638389822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de415dc80 msgr2=0x7f3dc80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.750+0000 7f3de2a00640 1 --2- 172.21.15.119:0/3638389822 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de415dc80 0x7f3dc80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.750+0000 7f3de2a00640 1 -- 172.21.15.119:0/3638389822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc80039b0 msgr2=0x7f3dc8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.750+0000 7f3de2a00640 1 --2- 172.21.15.119:0/3638389822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc80039b0 0x7f3dc8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.750+0000 7f3de2a00640 1 -- 172.21.15.119:0/3638389822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc8003520 con 0x7f3dc8004800 2024-04-25T10:18:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.750+0000 7f3de3400640 1 --2- 172.21.15.119:0/3638389822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc80039b0 0x7f3dc8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.751+0000 7f3de2a00640 1 --2- 172.21.15.119:0/3638389822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc8004800 0x7f3dc8004c80 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009f90 tx=0x7f3dc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18d48feff34499d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.752+0000 7f3de2000640 1 -- 172.21.15.119:0/3638389822 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4009c10 con 0x7f3dc8004800 2024-04-25T10:18:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.752+0000 7f3de2000640 1 -- 172.21.15.119:0/3638389822 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd4036070 con 0x7f3dc8004800 2024-04-25T10:18:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.752+0000 7f3de2000640 1 -- 172.21.15.119:0/3638389822 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4031670 con 0x7f3dc8004800 2024-04-25T10:18:20.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.752+0000 7f3dea800640 1 -- 172.21.15.119:0/3638389822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc8004800 msgr2=0x7f3dc8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.752+0000 7f3dea800640 1 --2- 172.21.15.119:0/3638389822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc8004800 0x7f3dc8004c80 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009f90 tx=0x7f3dc8005830 comp rx=0 tx=0).stop 2024-04-25T10:18:20.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.752+0000 7f3de3e00640 1 -- 172.21.15.119:0/3638389822 reap_dead start 2024-04-25T10:18:20.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.752+0000 7f3dea800640 1 -- 172.21.15.119:0/3638389822 shutdown_connections 2024-04-25T10:18:20.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.752+0000 7f3dea800640 1 -- 172.21.15.119:0/3638389822 >> 172.21.15.119:0/3638389822 conn(0x7f3de40d04f0 msgr2=0x7f3de40d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.752+0000 7f3dea800640 1 -- 172.21.15.119:0/3638389822 shutdown_connections 2024-04-25T10:18:20.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.752+0000 7f3dea800640 1 -- 172.21.15.119:0/3638389822 wait complete. 2024-04-25T10:18:20.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.754+0000 7f3dea800640 1 Processor -- start 2024-04-25T10:18:20.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.754+0000 7f3dea800640 1 -- start start 2024-04-25T10:18:20.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.755+0000 7f3dea800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc8004800 0x7f3de40d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.755+0000 7f3dea800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de40c6350 0x7f3de40c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.755+0000 7f3dea800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3de4146970 0x7f3de4146dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.755+0000 7f3dea800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40d3480 con 0x7f3de4146970 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.755+0000 7f3dea800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de415e060 con 0x7f3dc8004800 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.755+0000 7f3dea800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40d3600 con 0x7f3de40c6350 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.756+0000 7f3de3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3de4146970 0x7f3de4146dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.756+0000 7f3de3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3de4146970 0x7f3de4146dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55968/0 (socket says 172.21.15.119:55968) 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.756+0000 7f3de3e00640 1 -- 172.21.15.119:0/836143722 learned_addr learned my addr 172.21.15.119:0/836143722 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.756+0000 7f3de2a00640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de40c6350 0x7f3de40c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.756+0000 7f3de3400640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc8004800 0x7f3de40d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.756+0000 7f3de3e00640 1 -- 172.21.15.119:0/836143722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de40c6350 msgr2=0x7f3de40c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.756+0000 7f3de3e00640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de40c6350 0x7f3de40c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.756+0000 7f3de3e00640 1 -- 172.21.15.119:0/836143722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc8004800 msgr2=0x7f3de40d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.756+0000 7f3de3e00640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc8004800 0x7f3de40d3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.756+0000 7f3de3e00640 1 -- 172.21.15.119:0/836143722 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40c6cf0 con 0x7f3de4146970 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.756+0000 7f3de2a00640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de40c6350 0x7f3de40c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.756+0000 7f3de3400640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc8004800 0x7f3de40d3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:20.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.756+0000 7f3de3e00640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3de4146970 0x7f3de4146dd0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f3de406dab0 tx=0x7f3ddc00d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.757+0000 7f3ddb400640 1 -- 172.21.15.119:0/836143722 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc007500 con 0x7f3de4146970 2024-04-25T10:18:20.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.757+0000 7f3ddb400640 1 -- 172.21.15.119:0/836143722 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc0047e0 con 0x7f3de4146970 2024-04-25T10:18:20.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.757+0000 7f3ddb400640 1 -- 172.21.15.119:0/836143722 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc004bd0 con 0x7f3de4146970 2024-04-25T10:18:20.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.757+0000 7f3dea800640 1 -- 172.21.15.119:0/836143722 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de4147740 con 0x7f3de4146970 2024-04-25T10:18:20.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.757+0000 7f3dea800640 1 -- 172.21.15.119:0/836143722 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de4147d00 con 0x7f3de4146970 2024-04-25T10:18:20.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.758+0000 7f3dea800640 1 -- 172.21.15.119:0/836143722 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de40750b0 con 0x7f3de4146970 2024-04-25T10:18:20.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.759+0000 7f3ddb400640 1 -- 172.21.15.119:0/836143722 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ddc014430 con 0x7f3de4146970 2024-04-25T10:18:20.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.760+0000 7f3ddb400640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3dc006b500 0x7f3dc006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.760+0000 7f3ddb400640 1 -- 172.21.15.119:0/836143722 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f3ddc089c40 con 0x7f3de4146970 2024-04-25T10:18:20.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.760+0000 7f3de3400640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3dc006b500 0x7f3dc006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.761+0000 7f3de3400640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3dc006b500 0x7f3dc006d9c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0099c0 tx=0x7f3dcc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.764+0000 7f3ddb400640 1 -- 172.21.15.119:0/836143722 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ddc0582e0 con 0x7f3de4146970 2024-04-25T10:18:20.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.958+0000 7f3dea800640 1 -- 172.21.15.119:0/836143722 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3de40b4460 con 0x7f3de4146970 2024-04-25T10:18:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.959+0000 7f3ddb400640 1 -- 172.21.15.119:0/836143722 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ddc05c380 con 0x7f3de4146970 2024-04-25T10:18:20.962 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:18:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.961+0000 7f3dea800640 1 -- 172.21.15.119:0/836143722 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3dc006b500 msgr2=0x7f3dc006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.962+0000 7f3dea800640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3dc006b500 0x7f3dc006d9c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0099c0 tx=0x7f3dcc008040 comp rx=0 tx=0).stop 2024-04-25T10:18:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.962+0000 7f3dea800640 1 -- 172.21.15.119:0/836143722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3de4146970 msgr2=0x7f3de4146dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.962+0000 7f3dea800640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3de4146970 0x7f3de4146dd0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f3de406dab0 tx=0x7f3ddc00d900 comp rx=0 tx=0).stop 2024-04-25T10:18:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.962+0000 7f3dea800640 1 -- 172.21.15.119:0/836143722 shutdown_connections 2024-04-25T10:18:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.962+0000 7f3dea800640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3dc006b500 0x7f3dc006d9c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.962+0000 7f3dea800640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3de4146970 0x7f3de4146dd0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.962+0000 7f3dea800640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3de40c6350 0x7f3de40c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.962+0000 7f3dea800640 1 --2- 172.21.15.119:0/836143722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3dc8004800 0x7f3de40d3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.962+0000 7f3dea800640 1 -- 172.21.15.119:0/836143722 >> 172.21.15.119:0/836143722 conn(0x7f3de40d04f0 msgr2=0x7f3de406bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.962+0000 7f3dea800640 1 -- 172.21.15.119:0/836143722 shutdown_connections 2024-04-25T10:18:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:20.962+0000 7f3dea800640 1 -- 172.21.15.119:0/836143722 wait complete. 2024-04-25T10:18:20.984 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:21.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.183+0000 7f4249600640 1 Processor -- start 2024-04-25T10:18:21.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.183+0000 7f4249600640 1 -- start start 2024-04-25T10:18:21.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.185+0000 7f4249600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424406aba0 0x7f42440b1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.185+0000 7f4249600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f42440ce3a0 0x7f424406af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.185+0000 7f4249600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4244161930 0x7f4244161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.185+0000 7f4249600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42440b0130 con 0x7f4244161930 2024-04-25T10:18:21.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.185+0000 7f4249600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42440b09d0 con 0x7f42440ce3a0 2024-04-25T10:18:21.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.185+0000 7f4249600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424406b4c0 con 0x7f424406aba0 2024-04-25T10:18:21.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.185+0000 7f4243400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4244161930 0x7f4244161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.185+0000 7f4243400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4244161930 0x7f4244161dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60872/0 (socket says 172.21.15.119:60872) 2024-04-25T10:18:21.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.185+0000 7f4243400640 1 -- 172.21.15.119:0/39802091 learned_addr learned my addr 172.21.15.119:0/39802091 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:21.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.186+0000 7f423be00640 1 --2- 172.21.15.119:0/39802091 >> v2:172.21.15.132:3300/0 conn(0x7f42440ce3a0 0x7f424406af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.186+0000 7f4242a00640 1 --2- 172.21.15.119:0/39802091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424406aba0 0x7f42440b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.186+0000 7f4243400640 1 -- 172.21.15.119:0/39802091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424406aba0 msgr2=0x7f42440b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:21.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.186+0000 7f4243400640 1 --2- 172.21.15.119:0/39802091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424406aba0 0x7f42440b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.186+0000 7f4243400640 1 -- 172.21.15.119:0/39802091 >> v2:172.21.15.132:3300/0 conn(0x7f42440ce3a0 msgr2=0x7f424406af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:21.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.186+0000 7f4243400640 1 --2- 172.21.15.119:0/39802091 >> v2:172.21.15.132:3300/0 conn(0x7f42440ce3a0 0x7f424406af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.186+0000 7f4243400640 1 -- 172.21.15.119:0/39802091 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440b1f60 con 0x7f4244161930 2024-04-25T10:18:21.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.186+0000 7f423be00640 1 --2- 172.21.15.119:0/39802091 >> v2:172.21.15.132:3300/0 conn(0x7f42440ce3a0 0x7f424406af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:21.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.186+0000 7f4242a00640 1 --2- 172.21.15.119:0/39802091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424406aba0 0x7f42440b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:21.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.187+0000 7f4243400640 1 --2- 172.21.15.119:0/39802091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244161930 0x7f4244161dc0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f423c009e50 tx=0x7f423c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7130ac92a0629632 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.187+0000 7f4242000640 1 -- 172.21.15.119:0/39802091 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c036070 con 0x7f4244161930 2024-04-25T10:18:21.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.187+0000 7f4242000640 1 -- 172.21.15.119:0/39802091 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c031600 con 0x7f4244161930 2024-04-25T10:18:21.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.187+0000 7f4242000640 1 -- 172.21.15.119:0/39802091 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c0319d0 con 0x7f4244161930 2024-04-25T10:18:21.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.188+0000 7f4249600640 1 -- 172.21.15.119:0/39802091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244161930 msgr2=0x7f4244161dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.188+0000 7f4249600640 1 --2- 172.21.15.119:0/39802091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244161930 0x7f4244161dc0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f423c009e50 tx=0x7f423c02a6d0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.188+0000 7f4249600640 1 -- 172.21.15.119:0/39802091 shutdown_connections 2024-04-25T10:18:21.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.188+0000 7f4249600640 1 --2- 172.21.15.119:0/39802091 >> v2:172.21.15.132:3300/0 conn(0x7f42440ce3a0 0x7f424406af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.188+0000 7f4249600640 1 --2- 172.21.15.119:0/39802091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4244161930 0x7f4244161dc0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.188+0000 7f4249600640 1 --2- 172.21.15.119:0/39802091 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424406aba0 0x7f42440b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.188+0000 7f4249600640 1 -- 172.21.15.119:0/39802091 >> 172.21.15.119:0/39802091 conn(0x7f42440d07d0 msgr2=0x7f42440d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:21.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.188+0000 7f4249600640 1 -- 172.21.15.119:0/39802091 shutdown_connections 2024-04-25T10:18:21.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.188+0000 7f4249600640 1 -- 172.21.15.119:0/39802091 wait complete. 2024-04-25T10:18:21.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.190+0000 7f4249600640 1 Processor -- start 2024-04-25T10:18:21.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.190+0000 7f4249600640 1 -- start start 2024-04-25T10:18:21.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.191+0000 7f4249600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f424406aba0 0x7f424414baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.191+0000 7f4249600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42440ce3a0 0x7f42440ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.191+0000 7f4249600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4244161930 0x7f42440aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.191+0000 7f4249600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424406b880 con 0x7f424406aba0 2024-04-25T10:18:21.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.191+0000 7f4249600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42440b0660 con 0x7f4244161930 2024-04-25T10:18:21.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.191+0000 7f4249600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42440b0e60 con 0x7f42440ce3a0 2024-04-25T10:18:21.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.191+0000 7f4242a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f424406aba0 0x7f424414baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.191+0000 7f4242a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f424406aba0 0x7f424414baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60882/0 (socket says 172.21.15.119:60882) 2024-04-25T10:18:21.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.191+0000 7f4242a00640 1 -- 172.21.15.119:0/3009615820 learned_addr learned my addr 172.21.15.119:0/3009615820 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:21.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.191+0000 7f4243400640 1 --2- 172.21.15.119:0/3009615820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4244161930 0x7f42440aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.192+0000 7f4242a00640 1 -- 172.21.15.119:0/3009615820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42440ce3a0 msgr2=0x7f42440ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.192+0000 7f4242a00640 1 --2- 172.21.15.119:0/3009615820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42440ce3a0 0x7f42440ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.192+0000 7f4242a00640 1 -- 172.21.15.119:0/3009615820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4244161930 msgr2=0x7f42440aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.192+0000 7f4242a00640 1 --2- 172.21.15.119:0/3009615820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4244161930 0x7f42440aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.192+0000 7f4242a00640 1 -- 172.21.15.119:0/3009615820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440c6290 con 0x7f424406aba0 2024-04-25T10:18:21.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.192+0000 7f4243400640 1 --2- 172.21.15.119:0/3009615820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4244161930 0x7f42440aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:21.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.192+0000 7f4242a00640 1 --2- 172.21.15.119:0/3009615820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f424406aba0 0x7f424414baf0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f423400c7c0 tx=0x7f423400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.192+0000 7f423aa00640 1 -- 172.21.15.119:0/3009615820 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234015070 con 0x7f424406aba0 2024-04-25T10:18:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.192+0000 7f423aa00640 1 -- 172.21.15.119:0/3009615820 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4234007a30 con 0x7f424406aba0 2024-04-25T10:18:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.192+0000 7f4249600640 1 -- 172.21.15.119:0/3009615820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42440c6570 con 0x7f424406aba0 2024-04-25T10:18:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.192+0000 7f423aa00640 1 -- 172.21.15.119:0/3009615820 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234007e00 con 0x7f424406aba0 2024-04-25T10:18:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.192+0000 7f4249600640 1 -- 172.21.15.119:0/3009615820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42440c6b30 con 0x7f424406aba0 2024-04-25T10:18:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.194+0000 7f4249600640 1 -- 172.21.15.119:0/3009615820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42440750b0 con 0x7f424406aba0 2024-04-25T10:18:21.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.195+0000 7f423aa00640 1 -- 172.21.15.119:0/3009615820 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4234018070 con 0x7f424406aba0 2024-04-25T10:18:21.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.195+0000 7f423aa00640 1 --2- 172.21.15.119:0/3009615820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f422006b4d0 0x7f422006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.196+0000 7f423aa00640 1 -- 172.21.15.119:0/3009615820 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f423408a270 con 0x7f424406aba0 2024-04-25T10:18:21.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.196+0000 7f423be00640 1 --2- 172.21.15.119:0/3009615820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f422006b4d0 0x7f422006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.199 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.196+0000 7f423be00640 1 --2- 172.21.15.119:0/3009615820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f422006b4d0 0x7f422006d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f422c006fd0 tx=0x7f422c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.200+0000 7f423aa00640 1 -- 172.21.15.119:0/3009615820 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4234058ff0 con 0x7f424406aba0 2024-04-25T10:18:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.384+0000 7f4249600640 1 -- 172.21.15.119:0/3009615820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4244072f40 con 0x7f424406aba0 2024-04-25T10:18:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.385+0000 7f423aa00640 1 -- 172.21.15.119:0/3009615820 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f42340588f0 con 0x7f424406aba0 2024-04-25T10:18:21.388 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:18:21.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.387+0000 7f4249600640 1 -- 172.21.15.119:0/3009615820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f422006b4d0 msgr2=0x7f422006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.387+0000 7f4249600640 1 --2- 172.21.15.119:0/3009615820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f422006b4d0 0x7f422006d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f422c006fd0 tx=0x7f422c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.387+0000 7f4249600640 1 -- 172.21.15.119:0/3009615820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f424406aba0 msgr2=0x7f424414baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.387+0000 7f4249600640 1 --2- 172.21.15.119:0/3009615820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f424406aba0 0x7f424414baf0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f423400c7c0 tx=0x7f423400cc80 comp rx=0 tx=0).stop 2024-04-25T10:18:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.387+0000 7f4249600640 1 -- 172.21.15.119:0/3009615820 shutdown_connections 2024-04-25T10:18:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.387+0000 7f4249600640 1 --2- 172.21.15.119:0/3009615820 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f422006b4d0 0x7f422006d990 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.387+0000 7f4249600640 1 --2- 172.21.15.119:0/3009615820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4244161930 0x7f42440aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.387+0000 7f4249600640 1 --2- 172.21.15.119:0/3009615820 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42440ce3a0 0x7f42440ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.387+0000 7f4249600640 1 --2- 172.21.15.119:0/3009615820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f424406aba0 0x7f424414baf0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.387+0000 7f4249600640 1 -- 172.21.15.119:0/3009615820 >> 172.21.15.119:0/3009615820 conn(0x7f42440d07d0 msgr2=0x7f42440cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.387+0000 7f4249600640 1 -- 172.21.15.119:0/3009615820 shutdown_connections 2024-04-25T10:18:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.387+0000 7f4249600640 1 -- 172.21.15.119:0/3009615820 wait complete. 2024-04-25T10:18:21.408 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:21.609 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.606+0000 7f6e12a00640 1 Processor -- start 2024-04-25T10:18:21.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.606+0000 7f6e12a00640 1 -- start start 2024-04-25T10:18:21.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.607+0000 7f6e12a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e0c1450e0 0x7f6e0c06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.607+0000 7f6e12a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6e0c06b0e0 0x7f6e0c0cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.607+0000 7f6e12a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6e0c0cc810 0x7f6e0c071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.607+0000 7f6e12a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e0c1454c0 con 0x7f6e0c0cc810 2024-04-25T10:18:21.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.607+0000 7f6e12a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e0c06b540 con 0x7f6e0c06b0e0 2024-04-25T10:18:21.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.607+0000 7f6e12a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c15f400 con 0x7f6e0c1450e0 2024-04-25T10:18:21.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.608+0000 7f6e12000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6e0c0cc810 0x7f6e0c071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.608+0000 7f6e12000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6e0c0cc810 0x7f6e0c071050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60884/0 (socket says 172.21.15.119:60884) 2024-04-25T10:18:21.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.608+0000 7f6e12000640 1 -- 172.21.15.119:0/805705887 learned_addr learned my addr 172.21.15.119:0/805705887 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:21.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.608+0000 7f6e10c00640 1 --2- 172.21.15.119:0/805705887 >> v2:172.21.15.132:3300/0 conn(0x7f6e0c06b0e0 0x7f6e0c0cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.608+0000 7f6e11600640 1 --2- 172.21.15.119:0/805705887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e0c1450e0 0x7f6e0c06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.608+0000 7f6e12000640 1 -- 172.21.15.119:0/805705887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e0c1450e0 msgr2=0x7f6e0c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:21.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.608+0000 7f6e12000640 1 --2- 172.21.15.119:0/805705887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e0c1450e0 0x7f6e0c06aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.608+0000 7f6e12000640 1 -- 172.21.15.119:0/805705887 >> v2:172.21.15.132:3300/0 conn(0x7f6e0c06b0e0 msgr2=0x7f6e0c0cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:21.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.608+0000 7f6e12000640 1 --2- 172.21.15.119:0/805705887 >> v2:172.21.15.132:3300/0 conn(0x7f6e0c06b0e0 0x7f6e0c0cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.608+0000 7f6e12000640 1 -- 172.21.15.119:0/805705887 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c15f6e0 con 0x7f6e0c0cc810 2024-04-25T10:18:21.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.609+0000 7f6e12000640 1 --2- 172.21.15.119:0/805705887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e0c0cc810 0x7f6e0c071050 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c06f4c0 tx=0x7f6e0402a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9255cfd8b62c3fba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.610+0000 7f6e0be00640 1 -- 172.21.15.119:0/805705887 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04036070 con 0x7f6e0c0cc810 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.610+0000 7f6e0be00640 1 -- 172.21.15.119:0/805705887 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e04031600 con 0x7f6e0c0cc810 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.610+0000 7f6e0be00640 1 -- 172.21.15.119:0/805705887 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e040319d0 con 0x7f6e0c0cc810 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.610+0000 7f6e12a00640 1 -- 172.21.15.119:0/805705887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e0c0cc810 msgr2=0x7f6e0c071050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.610+0000 7f6e12a00640 1 --2- 172.21.15.119:0/805705887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e0c0cc810 0x7f6e0c071050 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c06f4c0 tx=0x7f6e0402a570 comp rx=0 tx=0).stop 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.611+0000 7f6e12a00640 1 -- 172.21.15.119:0/805705887 shutdown_connections 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.611+0000 7f6e12a00640 1 --2- 172.21.15.119:0/805705887 >> v2:172.21.15.132:3300/0 conn(0x7f6e0c06b0e0 0x7f6e0c0cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.611+0000 7f6e12a00640 1 --2- 172.21.15.119:0/805705887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e0c0cc810 0x7f6e0c071050 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.611+0000 7f6e12a00640 1 --2- 172.21.15.119:0/805705887 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e0c1450e0 0x7f6e0c06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.611+0000 7f6e12a00640 1 -- 172.21.15.119:0/805705887 >> 172.21.15.119:0/805705887 conn(0x7f6e0c066ff0 msgr2=0x7f6e0c0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.611+0000 7f6e12a00640 1 -- 172.21.15.119:0/805705887 shutdown_connections 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.611+0000 7f6e12a00640 1 -- 172.21.15.119:0/805705887 wait complete. 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.612+0000 7f6e12a00640 1 Processor -- start 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.612+0000 7f6e12a00640 1 -- start start 2024-04-25T10:18:21.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.613+0000 7f6e12a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e0c06b0e0 0x7f6e0c1f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.613+0000 7f6e12a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e0c0cc810 0x7f6e0c0ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.613+0000 7f6e12a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e0c1450e0 0x7f6e0c0ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.613+0000 7f6e12a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c0ccca0 con 0x7f6e0c06b0e0 2024-04-25T10:18:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.613+0000 7f6e12a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c15fca0 con 0x7f6e0c0cc810 2024-04-25T10:18:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.613+0000 7f6e12a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c0ac2d0 con 0x7f6e0c1450e0 2024-04-25T10:18:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.614+0000 7f6e11600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e0c06b0e0 0x7f6e0c1f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.614+0000 7f6e11600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e0c06b0e0 0x7f6e0c1f2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60892/0 (socket says 172.21.15.119:60892) 2024-04-25T10:18:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.614+0000 7f6e11600640 1 -- 172.21.15.119:0/2162495003 learned_addr learned my addr 172.21.15.119:0/2162495003 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.614+0000 7f6e12000640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e0c1450e0 0x7f6e0c0ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.614+0000 7f6e10c00640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e0c0cc810 0x7f6e0c0ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.614+0000 7f6e11600640 1 -- 172.21.15.119:0/2162495003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e0c1450e0 msgr2=0x7f6e0c0ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.614+0000 7f6e11600640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e0c1450e0 0x7f6e0c0ad5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.614+0000 7f6e11600640 1 -- 172.21.15.119:0/2162495003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e0c0cc810 msgr2=0x7f6e0c0ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.614+0000 7f6e11600640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e0c0cc810 0x7f6e0c0ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.614+0000 7f6e11600640 1 -- 172.21.15.119:0/2162495003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c1f2960 con 0x7f6e0c06b0e0 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.614+0000 7f6e10c00640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e0c0cc810 0x7f6e0c0ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.614+0000 7f6e11600640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e0c06b0e0 0x7f6e0c1f2420 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000d800 tx=0x7f6e0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.615+0000 7f6e09600640 1 -- 172.21.15.119:0/2162495003 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0000b6c0 con 0x7f6e0c06b0e0 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.615+0000 7f6e09600640 1 -- 172.21.15.119:0/2162495003 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e00019070 con 0x7f6e0c06b0e0 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.615+0000 7f6e09600640 1 -- 172.21.15.119:0/2162495003 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0000ba80 con 0x7f6e0c06b0e0 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.615+0000 7f6e12a00640 1 -- 172.21.15.119:0/2162495003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0c0ab6c0 con 0x7f6e0c06b0e0 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.615+0000 7f6e12a00640 1 -- 172.21.15.119:0/2162495003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0c0abc80 con 0x7f6e0c06b0e0 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.616+0000 7f6e12a00640 1 -- 172.21.15.119:0/2162495003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0c067490 con 0x7f6e0c06b0e0 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.617+0000 7f6e09600640 1 -- 172.21.15.119:0/2162495003 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e0000bc20 con 0x7f6e0c06b0e0 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.618+0000 7f6e09600640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6dec06b4d0 0x7f6dec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.618+0000 7f6e09600640 1 -- 172.21.15.119:0/2162495003 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f6e0008a840 con 0x7f6e0c06b0e0 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.618+0000 7f6e10c00640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6dec06b4d0 0x7f6dec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.619+0000 7f6e10c00640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6dec06b4d0 0x7f6dec06d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6df8006fd0 tx=0x7f6df8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.622+0000 7f6e09600640 1 -- 172.21.15.119:0/2162495003 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e0c067490 con 0x7f6e0c06b0e0 2024-04-25T10:18:21.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.822+0000 7f6e12a00640 1 -- 172.21.15.119:0/2162495003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e0c0749b0 con 0x7f6e0c06b0e0 2024-04-25T10:18:21.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.823+0000 7f6e09600640 1 -- 172.21.15.119:0/2162495003 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e00059530 con 0x7f6e0c06b0e0 2024-04-25T10:18:21.826 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:18:21.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.826+0000 7f6e12a00640 1 -- 172.21.15.119:0/2162495003 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6dec06b4d0 msgr2=0x7f6dec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.826+0000 7f6e12a00640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6dec06b4d0 0x7f6dec06d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6df8006fd0 tx=0x7f6df8008040 comp rx=0 tx=0).stop 2024-04-25T10:18:21.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.826+0000 7f6e12a00640 1 -- 172.21.15.119:0/2162495003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e0c06b0e0 msgr2=0x7f6e0c1f2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.826+0000 7f6e12a00640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e0c06b0e0 0x7f6e0c1f2420 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000d800 tx=0x7f6e0000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.826+0000 7f6e12a00640 1 -- 172.21.15.119:0/2162495003 shutdown_connections 2024-04-25T10:18:21.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.826+0000 7f6e12a00640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6dec06b4d0 0x7f6dec06d990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.826+0000 7f6e12a00640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6e0c1450e0 0x7f6e0c0ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.826+0000 7f6e12a00640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e0c0cc810 0x7f6e0c0ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.826+0000 7f6e12a00640 1 --2- 172.21.15.119:0/2162495003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e0c06b0e0 0x7f6e0c1f2420 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.826+0000 7f6e12a00640 1 -- 172.21.15.119:0/2162495003 >> 172.21.15.119:0/2162495003 conn(0x7f6e0c066ff0 msgr2=0x7f6e0c15e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:21.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.826+0000 7f6e12a00640 1 -- 172.21.15.119:0/2162495003 shutdown_connections 2024-04-25T10:18:21.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:21.826+0000 7f6e12a00640 1 -- 172.21.15.119:0/2162495003 wait complete. 2024-04-25T10:18:21.847 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:18:22.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.043+0000 7f3c3ac00640 1 Processor -- start 2024-04-25T10:18:22.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.043+0000 7f3c3ac00640 1 -- start start 2024-04-25T10:18:22.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.045+0000 7f3c3ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c3415dbe0 0x7f3c34071c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.045+0000 7f3c3ac00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3c340cfad0 0x7f3c3406d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.045+0000 7f3c3ac00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c340ace70 con 0x7f3c340aedc0 2024-04-25T10:18:22.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.045+0000 7f3c3ac00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c34072150 con 0x7f3c340cfad0 2024-04-25T10:18:22.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.045+0000 7f3c3ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c340ad110 con 0x7f3c3415dbe0 2024-04-25T10:18:22.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.045+0000 7f3c2be00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3c340aedc0 0x7f3c340cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42548/0 (socket says 172.21.15.119:42548) 2024-04-25T10:18:22.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.045+0000 7f3c2be00640 1 -- 172.21.15.119:0/906016830 learned_addr learned my addr 172.21.15.119:0/906016830 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:22.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.045+0000 7f3c38c00640 1 --2- 172.21.15.119:0/906016830 >> v2:172.21.15.119:3300/0 conn(0x7f3c340cfad0 0x7f3c3406d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.045+0000 7f3c33e00640 1 --2- 172.21.15.119:0/906016830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c3415dbe0 0x7f3c34071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.046+0000 7f3c33400640 1 -- 172.21.15.119:0/906016830 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204098043 0 0) 0x7f3c340ace70 con 0x7f3c340aedc0 2024-04-25T10:18:22.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.046+0000 7f3c33400640 1 -- 172.21.15.119:0/906016830 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c18003520 con 0x7f3c340aedc0 2024-04-25T10:18:22.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.046+0000 7f3c38c00640 1 -- 172.21.15.119:0/906016830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c3415dbe0 msgr2=0x7f3c34071c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:22.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.046+0000 7f3c38c00640 1 --2- 172.21.15.119:0/906016830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c3415dbe0 0x7f3c34071c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.046+0000 7f3c38c00640 1 -- 172.21.15.119:0/906016830 >> v1:172.21.15.119:6789/0 conn(0x7f3c340aedc0 legacy=0x7f3c340cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.046+0000 7f3c38c00640 1 -- 172.21.15.119:0/906016830 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c340ad3f0 con 0x7f3c340cfad0 2024-04-25T10:18:22.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.047+0000 7f3c38c00640 1 --2- 172.21.15.119:0/906016830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c340cfad0 0x7f3c3406d9a0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009dc0 tx=0x7f3c2c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c65f005df4bd91c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.047+0000 7f3c33400640 1 -- 172.21.15.119:0/906016830 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c02ae60 con 0x7f3c340cfad0 2024-04-25T10:18:22.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.047+0000 7f3c33400640 1 -- 172.21.15.119:0/906016830 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c007740 con 0x7f3c340cfad0 2024-04-25T10:18:22.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.047+0000 7f3c33400640 1 -- 172.21.15.119:0/906016830 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c007b10 con 0x7f3c340cfad0 2024-04-25T10:18:22.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.048+0000 7f3c3ac00640 1 -- 172.21.15.119:0/906016830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c340cfad0 msgr2=0x7f3c3406d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.048+0000 7f3c3ac00640 1 --2- 172.21.15.119:0/906016830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c340cfad0 0x7f3c3406d9a0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009dc0 tx=0x7f3c2c02a680 comp rx=0 tx=0).stop 2024-04-25T10:18:22.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.048+0000 7f3c3ac00640 1 -- 172.21.15.119:0/906016830 shutdown_connections 2024-04-25T10:18:22.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.048+0000 7f3c3ac00640 1 --2- 172.21.15.119:0/906016830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c340cfad0 0x7f3c3406d9a0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.048+0000 7f3c3ac00640 1 --2- 172.21.15.119:0/906016830 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c3415dbe0 0x7f3c34071c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.048+0000 7f3c3ac00640 1 -- 172.21.15.119:0/906016830 >> 172.21.15.119:0/906016830 conn(0x7f3c3406aba0 msgr2=0x7f3c3406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:22.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.048+0000 7f3c3ac00640 1 -- 172.21.15.119:0/906016830 shutdown_connections 2024-04-25T10:18:22.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.048+0000 7f3c3ac00640 1 -- 172.21.15.119:0/906016830 wait complete. 2024-04-25T10:18:22.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.050+0000 7f3c3ac00640 1 Processor -- start 2024-04-25T10:18:22.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.050+0000 7f3c3ac00640 1 -- start start 2024-04-25T10:18:22.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.051+0000 7f3c3ac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c340aedc0 0x7f3c3414c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.051+0000 7f3c3ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c340cfad0 0x7f3c34156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.051+0000 7f3c3ac00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c3415dbe0 0x7f3c341572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.051+0000 7f3c3ac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c340adc90 con 0x7f3c340aedc0 2024-04-25T10:18:22.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.051+0000 7f3c3ac00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c340ad9b0 con 0x7f3c3415dbe0 2024-04-25T10:18:22.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.051+0000 7f3c3ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3415dfc0 con 0x7f3c340cfad0 2024-04-25T10:18:22.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.051+0000 7f3c33e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c340aedc0 0x7f3c3414c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.051+0000 7f3c33e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c340aedc0 0x7f3c3414c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60918/0 (socket says 172.21.15.119:60918) 2024-04-25T10:18:22.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.051+0000 7f3c33e00640 1 -- 172.21.15.119:0/4226776182 learned_addr learned my addr 172.21.15.119:0/4226776182 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:22.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.051+0000 7f3c33e00640 1 -- 172.21.15.119:0/4226776182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c340cfad0 msgr2=0x7f3c34156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.051+0000 7f3c33e00640 1 --2- 172.21.15.119:0/4226776182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c340cfad0 0x7f3c34156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.051+0000 7f3c33e00640 1 -- 172.21.15.119:0/4226776182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c3415dbe0 msgr2=0x7f3c341572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.051+0000 7f3c33e00640 1 --2- 172.21.15.119:0/4226776182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c3415dbe0 0x7f3c341572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.051+0000 7f3c33e00640 1 -- 172.21.15.119:0/4226776182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c340b0fd0 con 0x7f3c340aedc0 2024-04-25T10:18:22.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.052+0000 7f3c33e00640 1 --2- 172.21.15.119:0/4226776182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c340aedc0 0x7f3c3414c2a0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400d5f0 tx=0x7f3c2400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.052+0000 7f3c30c00640 1 -- 172.21.15.119:0/4226776182 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c240125f0 con 0x7f3c340aedc0 2024-04-25T10:18:22.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.052+0000 7f3c30c00640 1 -- 172.21.15.119:0/4226776182 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c24019070 con 0x7f3c340aedc0 2024-04-25T10:18:22.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.052+0000 7f3c3ac00640 1 -- 172.21.15.119:0/4226776182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c340b12b0 con 0x7f3c340aedc0 2024-04-25T10:18:22.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.052+0000 7f3c30c00640 1 -- 172.21.15.119:0/4226776182 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c240129d0 con 0x7f3c340aedc0 2024-04-25T10:18:22.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.052+0000 7f3c3ac00640 1 -- 172.21.15.119:0/4226776182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c340b1870 con 0x7f3c340aedc0 2024-04-25T10:18:22.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.053+0000 7f3c3ac00640 1 -- 172.21.15.119:0/4226776182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c340b4450 con 0x7f3c340aedc0 2024-04-25T10:18:22.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.054+0000 7f3c30c00640 1 -- 172.21.15.119:0/4226776182 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3c24012b70 con 0x7f3c340aedc0 2024-04-25T10:18:22.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.055+0000 7f3c30c00640 1 --2- 172.21.15.119:0/4226776182 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3c1006b520 0x7f3c1006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.055+0000 7f3c30c00640 1 -- 172.21.15.119:0/4226776182 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f3c2408a630 con 0x7f3c340aedc0 2024-04-25T10:18:22.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.055+0000 7f3c2be00640 1 --2- 172.21.15.119:0/4226776182 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3c1006b520 0x7f3c1006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.056+0000 7f3c2be00640 1 --2- 172.21.15.119:0/4226776182 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3c1006b520 0x7f3c1006d9e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c000f60 tx=0x7f3c1c0078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.060+0000 7f3c30c00640 1 -- 172.21.15.119:0/4226776182 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3c340b4450 con 0x7f3c340aedc0 2024-04-25T10:18:22.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.251+0000 7f3c3ac00640 1 -- 172.21.15.119:0/4226776182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3c340750b0 con 0x7f3c340aedc0 2024-04-25T10:18:22.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.252+0000 7f3c30c00640 1 -- 172.21.15.119:0/4226776182 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v112) v1 ==== 80+0+248 (secure 0 0 0) 0x7f3c24059430 con 0x7f3c340aedc0 2024-04-25T10:18:22.255 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6837/2864575636 2024-04-26T10:18:09.666363+0000 2024-04-25T10:18:22.255 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6836/2864575636 2024-04-26T10:18:09.666363+0000 2024-04-25T10:18:22.255 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6837/1849584205 2024-04-26T10:17:56.438859+0000 2024-04-25T10:18:22.255 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6836/1849584205 2024-04-26T10:17:56.438859+0000 2024-04-25T10:18:22.256 INFO:teuthology.orchestra.run.smithi119.stderr:listed 4 entries 2024-04-25T10:18:22.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.254+0000 7f3c3ac00640 1 -- 172.21.15.119:0/4226776182 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3c1006b520 msgr2=0x7f3c1006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.255+0000 7f3c3ac00640 1 --2- 172.21.15.119:0/4226776182 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3c1006b520 0x7f3c1006d9e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c000f60 tx=0x7f3c1c0078f0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.255+0000 7f3c3ac00640 1 -- 172.21.15.119:0/4226776182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c340aedc0 msgr2=0x7f3c3414c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.255+0000 7f3c3ac00640 1 --2- 172.21.15.119:0/4226776182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c340aedc0 0x7f3c3414c2a0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400d5f0 tx=0x7f3c2400dab0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.255+0000 7f3c3ac00640 1 -- 172.21.15.119:0/4226776182 shutdown_connections 2024-04-25T10:18:22.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.255+0000 7f3c3ac00640 1 --2- 172.21.15.119:0/4226776182 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f3c1006b520 0x7f3c1006d9e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.255+0000 7f3c3ac00640 1 --2- 172.21.15.119:0/4226776182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c3415dbe0 0x7f3c341572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.255+0000 7f3c3ac00640 1 --2- 172.21.15.119:0/4226776182 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c340cfad0 0x7f3c34156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.255+0000 7f3c3ac00640 1 --2- 172.21.15.119:0/4226776182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c340aedc0 0x7f3c3414c2a0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.255+0000 7f3c3ac00640 1 -- 172.21.15.119:0/4226776182 >> 172.21.15.119:0/4226776182 conn(0x7f3c3406aba0 msgr2=0x7f3c3406e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:22.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.255+0000 7f3c3ac00640 1 -- 172.21.15.119:0/4226776182 shutdown_connections 2024-04-25T10:18:22.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.255+0000 7f3c3ac00640 1 -- 172.21.15.119:0/4226776182 wait complete. 2024-04-25T10:18:22.276 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:18:22.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.276 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:22.369 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:18:22.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.369 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:18:22.419 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:18:22.419 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:22.477 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:18:22.478 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:18:22.478 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:22.530 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:18:22.531 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:18:22.531 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:22.588 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:18:22.589 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:22.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.790+0000 7f256be00640 1 Processor -- start 2024-04-25T10:18:22.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.791+0000 7f256be00640 1 -- start start 2024-04-25T10:18:22.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.792+0000 7f256be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f256c0d3f80 0x7f256c0d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.792+0000 7f256be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f256c0b9270 0x7f256c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.792+0000 7f256be00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256c15ef10 con 0x7f256c15db60 2024-04-25T10:18:22.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.792+0000 7f256be00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256c15e390 con 0x7f256c0d3f80 2024-04-25T10:18:22.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.792+0000 7f256be00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256c15e670 con 0x7f256c0b9270 2024-04-25T10:18:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.792+0000 7f256aa00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f256c15db60 0x7f256c0d3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42550/0 (socket says 172.21.15.119:42550) 2024-04-25T10:18:22.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.792+0000 7f256a000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f256c0d3f80 0x7f256c0d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.792+0000 7f256aa00640 1 -- 172.21.15.119:0/2907708820 learned_addr learned my addr 172.21.15.119:0/2907708820 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:22.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.793+0000 7f256b400640 1 --2- 172.21.15.119:0/2907708820 >> v2:172.21.15.132:3300/0 conn(0x7f256c0b9270 0x7f256c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.793+0000 7f256a000640 1 -- 172.21.15.119:0/2907708820 >> v2:172.21.15.132:3300/0 conn(0x7f256c0b9270 msgr2=0x7f256c0d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:22.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.793+0000 7f256a000640 1 --2- 172.21.15.119:0/2907708820 >> v2:172.21.15.132:3300/0 conn(0x7f256c0b9270 0x7f256c0d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.793+0000 7f256a000640 1 -- 172.21.15.119:0/2907708820 >> v1:172.21.15.119:6789/0 conn(0x7f256c15db60 legacy=0x7f256c0d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.793+0000 7f256a000640 1 -- 172.21.15.119:0/2907708820 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c15e950 con 0x7f256c0d3f80 2024-04-25T10:18:22.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.793+0000 7f2569600640 1 -- 172.21.15.119:0/2907708820 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432624639 0 0) 0x7f256c15ef10 con 0x7f256c15db60 2024-04-25T10:18:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.794+0000 7f256a000640 1 --2- 172.21.15.119:0/2907708820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256c0d3f80 0x7f256c0d1f90 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f256c0d1760 tx=0x7f255402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b707682db289ed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.795+0000 7f2569600640 1 -- 172.21.15.119:0/2907708820 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554007740 con 0x7f256c0d3f80 2024-04-25T10:18:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.795+0000 7f2569600640 1 -- 172.21.15.119:0/2907708820 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25540078e0 con 0x7f256c0d3f80 2024-04-25T10:18:22.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.795+0000 7f2569600640 1 -- 172.21.15.119:0/2907708820 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554007cb0 con 0x7f256c0d3f80 2024-04-25T10:18:22.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.795+0000 7f256be00640 1 -- 172.21.15.119:0/2907708820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256c0d3f80 msgr2=0x7f256c0d1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.795+0000 7f256be00640 1 --2- 172.21.15.119:0/2907708820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256c0d3f80 0x7f256c0d1f90 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f256c0d1760 tx=0x7f255402a660 comp rx=0 tx=0).stop 2024-04-25T10:18:22.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.795+0000 7f256be00640 1 -- 172.21.15.119:0/2907708820 shutdown_connections 2024-04-25T10:18:22.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.795+0000 7f256be00640 1 --2- 172.21.15.119:0/2907708820 >> v2:172.21.15.132:3300/0 conn(0x7f256c0b9270 0x7f256c0d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.795+0000 7f256be00640 1 --2- 172.21.15.119:0/2907708820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256c0d3f80 0x7f256c0d1f90 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.795+0000 7f256be00640 1 -- 172.21.15.119:0/2907708820 >> 172.21.15.119:0/2907708820 conn(0x7f256c0726c0 msgr2=0x7f256c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:22.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.796+0000 7f256be00640 1 -- 172.21.15.119:0/2907708820 shutdown_connections 2024-04-25T10:18:22.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.796+0000 7f256be00640 1 -- 172.21.15.119:0/2907708820 wait complete. 2024-04-25T10:18:22.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.797+0000 7f256be00640 1 Processor -- start 2024-04-25T10:18:22.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.798+0000 7f256be00640 1 -- start start 2024-04-25T10:18:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.798+0000 7f256be00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256c0b9270 0x7f256c1f3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.798+0000 7f256be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f256c0d3f80 0x7f256c14fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.799+0000 7f256be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f256c15db60 0x7f256c0ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.799+0000 7f256be00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c0b3120 con 0x7f256c0b9270 2024-04-25T10:18:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.799+0000 7f256be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c0b2fa0 con 0x7f256c0d3f80 2024-04-25T10:18:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.799+0000 7f256be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256c15df40 con 0x7f256c15db60 2024-04-25T10:18:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.799+0000 7f256aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256c0b9270 0x7f256c1f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.799+0000 7f256aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256c0b9270 0x7f256c1f3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60938/0 (socket says 172.21.15.119:60938) 2024-04-25T10:18:22.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.799+0000 7f256aa00640 1 -- 172.21.15.119:0/1357417341 learned_addr learned my addr 172.21.15.119:0/1357417341 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:22.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.799+0000 7f256a000640 1 --2- 172.21.15.119:0/1357417341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f256c0d3f80 0x7f256c14fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.799+0000 7f256aa00640 1 -- 172.21.15.119:0/1357417341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f256c15db60 msgr2=0x7f256c0ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.799+0000 7f256aa00640 1 --2- 172.21.15.119:0/1357417341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f256c15db60 0x7f256c0ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.799+0000 7f256aa00640 1 -- 172.21.15.119:0/1357417341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f256c0d3f80 msgr2=0x7f256c14fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.799+0000 7f256aa00640 1 --2- 172.21.15.119:0/1357417341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f256c0d3f80 0x7f256c14fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.799+0000 7f256aa00640 1 -- 172.21.15.119:0/1357417341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c0b5bf0 con 0x7f256c0b9270 2024-04-25T10:18:22.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.800+0000 7f256aa00640 1 --2- 172.21.15.119:0/1357417341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256c0b9270 0x7f256c1f3800 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f255c00c4d0 tx=0x7f255c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.800+0000 7f2563400640 1 -- 172.21.15.119:0/1357417341 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c018070 con 0x7f256c0b9270 2024-04-25T10:18:22.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.800+0000 7f2563400640 1 -- 172.21.15.119:0/1357417341 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255c007500 con 0x7f256c0b9270 2024-04-25T10:18:22.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.800+0000 7f256be00640 1 -- 172.21.15.119:0/1357417341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256c1e4700 con 0x7f256c0b9270 2024-04-25T10:18:22.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.800+0000 7f2563400640 1 -- 172.21.15.119:0/1357417341 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c004a20 con 0x7f256c0b9270 2024-04-25T10:18:22.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.800+0000 7f256be00640 1 -- 172.21.15.119:0/1357417341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256c1e4cc0 con 0x7f256c0b9270 2024-04-25T10:18:22.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.801+0000 7f256be00640 1 -- 172.21.15.119:0/1357417341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256c0749b0 con 0x7f256c0b9270 2024-04-25T10:18:22.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.802+0000 7f2563400640 1 -- 172.21.15.119:0/1357417341 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f255c01b070 con 0x7f256c0b9270 2024-04-25T10:18:22.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.803+0000 7f2563400640 1 --2- 172.21.15.119:0/1357417341 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f254806b4d0 0x7f254806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:22.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.803+0000 7f2563400640 1 -- 172.21.15.119:0/1357417341 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f255c0896f0 con 0x7f256c0b9270 2024-04-25T10:18:22.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.803+0000 7f256a000640 1 --2- 172.21.15.119:0/1357417341 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f254806b4d0 0x7f254806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:22.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.804+0000 7f256a000640 1 --2- 172.21.15.119:0/1357417341 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f254806b4d0 0x7f254806d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2554034000 tx=0x7f255402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:22.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:22.809+0000 7f2563400640 1 -- 172.21.15.119:0/1357417341 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f255c0583e0 con 0x7f256c0b9270 2024-04-25T10:18:23.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.055+0000 7f256be00640 1 -- 172.21.15.119:0/1357417341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f256c0d1a60 con 0x7f256c0b9270 2024-04-25T10:18:23.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.056+0000 7f2563400640 1 -- 172.21.15.119:0/1357417341 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4957 (secure 0 0 0) 0x7f255c057ce0 con 0x7f256c0b9270 2024-04-25T10:18:23.060 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:23.060 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3394261143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3394261143},{"type":"v1","addr":"172.21.15.132:6834","nonce":3394261143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2502874931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2502874931},{"type":"v1","addr":"172.21.15.119:6835","nonce":2502874931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":8694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2305428003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2305428003},{"type":"v1","addr":"172.21.15.132:6835","nonce":2305428003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:37.617787+0000","modified":"2024-04-25T10:18:13.835772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9117},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9117":{"gid":9117,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/2618740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2618740513},{"type":"v1","addr":"172.21.15.119:6837","nonce":2618740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:18:23.060 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 112 2024-04-25T10:18:23.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.059+0000 7f256be00640 1 -- 172.21.15.119:0/1357417341 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f254806b4d0 msgr2=0x7f254806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.059+0000 7f256be00640 1 --2- 172.21.15.119:0/1357417341 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f254806b4d0 0x7f254806d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2554034000 tx=0x7f255402f000 comp rx=0 tx=0).stop 2024-04-25T10:18:23.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.059+0000 7f256be00640 1 -- 172.21.15.119:0/1357417341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256c0b9270 msgr2=0x7f256c1f3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.059+0000 7f256be00640 1 --2- 172.21.15.119:0/1357417341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256c0b9270 0x7f256c1f3800 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f255c00c4d0 tx=0x7f255c00c990 comp rx=0 tx=0).stop 2024-04-25T10:18:23.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.059+0000 7f256be00640 1 -- 172.21.15.119:0/1357417341 shutdown_connections 2024-04-25T10:18:23.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.059+0000 7f256be00640 1 --2- 172.21.15.119:0/1357417341 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f254806b4d0 0x7f254806d990 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.059+0000 7f256be00640 1 --2- 172.21.15.119:0/1357417341 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f256c15db60 0x7f256c0ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.059+0000 7f256be00640 1 --2- 172.21.15.119:0/1357417341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f256c0d3f80 0x7f256c14fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.059+0000 7f256be00640 1 --2- 172.21.15.119:0/1357417341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256c0b9270 0x7f256c1f3800 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.059+0000 7f256be00640 1 -- 172.21.15.119:0/1357417341 >> 172.21.15.119:0/1357417341 conn(0x7f256c0726c0 msgr2=0x7f256c1e6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:23.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.060+0000 7f256be00640 1 -- 172.21.15.119:0/1357417341 shutdown_connections 2024-04-25T10:18:23.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.060+0000 7f256be00640 1 -- 172.21.15.119:0/1357417341 wait complete. 2024-04-25T10:18:23.081 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:23.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.279+0000 7f4b83200640 1 Processor -- start 2024-04-25T10:18:23.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.279+0000 7f4b83200640 1 -- start start 2024-04-25T10:18:23.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.281+0000 7f4b83200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4b84144470 0x7f4b8415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.281+0000 7f4b83200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4b840b5050 0x7f4b8415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.281+0000 7f4b83200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b840cd440 con 0x7f4b8406e430 2024-04-25T10:18:23.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.281+0000 7f4b83200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b84145340 con 0x7f4b84144470 2024-04-25T10:18:23.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.281+0000 7f4b83200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b840cd720 con 0x7f4b840b5050 2024-04-25T10:18:23.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.281+0000 7f4b81e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4b84144470 0x7f4b8415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.281+0000 7f4b81e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4b84144470 0x7f4b8415fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60954/0 (socket says 172.21.15.119:60954) 2024-04-25T10:18:23.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.281+0000 7f4b81400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4b8406e430 0x7f4b840b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42554/0 (socket says 172.21.15.119:42554) 2024-04-25T10:18:23.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.281+0000 7f4b81e00640 1 -- 172.21.15.119:0/1155537000 learned_addr learned my addr 172.21.15.119:0/1155537000 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:23.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.281+0000 7f4b81e00640 1 -- 172.21.15.119:0/1155537000 >> v2:172.21.15.132:3300/0 conn(0x7f4b840b5050 msgr2=0x7f4b8415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:23.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.281+0000 7f4b82800640 1 --2- 172.21.15.119:0/1155537000 >> v2:172.21.15.132:3300/0 conn(0x7f4b840b5050 0x7f4b8415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.281+0000 7f4b80a00640 1 -- 172.21.15.119:0/1155537000 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493803165 0 0) 0x7f4b840cd440 con 0x7f4b8406e430 2024-04-25T10:18:23.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.282+0000 7f4b81e00640 1 --2- 172.21.15.119:0/1155537000 >> v2:172.21.15.132:3300/0 conn(0x7f4b840b5050 0x7f4b8415bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.282+0000 7f4b81e00640 1 -- 172.21.15.119:0/1155537000 >> v1:172.21.15.119:6789/0 conn(0x7f4b8406e430 legacy=0x7f4b840b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.282+0000 7f4b81e00640 1 -- 172.21.15.119:0/1155537000 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840cda00 con 0x7f4b84144470 2024-04-25T10:18:23.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.283+0000 7f4b81e00640 1 --2- 172.21.15.119:0/1155537000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b84144470 0x7f4b8415fc90 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f4b841455d0 tx=0x7f4b7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdd42c831235db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.283+0000 7f4b80a00640 1 -- 172.21.15.119:0/1155537000 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74037070 con 0x7f4b84144470 2024-04-25T10:18:23.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.283+0000 7f4b80a00640 1 -- 172.21.15.119:0/1155537000 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b74032440 con 0x7f4b84144470 2024-04-25T10:18:23.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.283+0000 7f4b80a00640 1 -- 172.21.15.119:0/1155537000 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74032810 con 0x7f4b84144470 2024-04-25T10:18:23.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.283+0000 7f4b83200640 1 -- 172.21.15.119:0/1155537000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b84144470 msgr2=0x7f4b8415fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.283+0000 7f4b83200640 1 --2- 172.21.15.119:0/1155537000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b84144470 0x7f4b8415fc90 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f4b841455d0 tx=0x7f4b7402a270 comp rx=0 tx=0).stop 2024-04-25T10:18:23.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.284+0000 7f4b83200640 1 -- 172.21.15.119:0/1155537000 shutdown_connections 2024-04-25T10:18:23.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.284+0000 7f4b83200640 1 --2- 172.21.15.119:0/1155537000 >> v2:172.21.15.132:3300/0 conn(0x7f4b840b5050 0x7f4b8415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.284+0000 7f4b83200640 1 --2- 172.21.15.119:0/1155537000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b84144470 0x7f4b8415fc90 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.284+0000 7f4b83200640 1 -- 172.21.15.119:0/1155537000 >> 172.21.15.119:0/1155537000 conn(0x7f4b8406b1d0 msgr2=0x7f4b840b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:23.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.284+0000 7f4b83200640 1 -- 172.21.15.119:0/1155537000 shutdown_connections 2024-04-25T10:18:23.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.284+0000 7f4b83200640 1 -- 172.21.15.119:0/1155537000 wait complete. 2024-04-25T10:18:23.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.285+0000 7f4b83200640 1 Processor -- start 2024-04-25T10:18:23.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.286+0000 7f4b83200640 1 -- start start 2024-04-25T10:18:23.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.286+0000 7f4b83200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b8406e430 0x7f4b8415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.286+0000 7f4b83200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b840b5050 0x7f4b841e5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.286+0000 7f4b83200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b84144470 0x7f4b8415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.286+0000 7f4b83200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b840ce200 con 0x7f4b84144470 2024-04-25T10:18:23.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.286+0000 7f4b83200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b84144850 con 0x7f4b840b5050 2024-04-25T10:18:23.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.286+0000 7f4b83200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8415c230 con 0x7f4b8406e430 2024-04-25T10:18:23.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.287+0000 7f4b82800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b84144470 0x7f4b8415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.287+0000 7f4b82800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b84144470 0x7f4b8415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60964/0 (socket says 172.21.15.119:60964) 2024-04-25T10:18:23.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.287+0000 7f4b82800640 1 -- 172.21.15.119:0/1320289276 learned_addr learned my addr 172.21.15.119:0/1320289276 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:23.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.287+0000 7f4b81400640 1 --2- 172.21.15.119:0/1320289276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b840b5050 0x7f4b841e5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.287+0000 7f4b82800640 1 -- 172.21.15.119:0/1320289276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b8406e430 msgr2=0x7f4b8415c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.287+0000 7f4b82800640 1 --2- 172.21.15.119:0/1320289276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b8406e430 0x7f4b8415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.287+0000 7f4b82800640 1 -- 172.21.15.119:0/1320289276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b840b5050 msgr2=0x7f4b841e5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.287+0000 7f4b82800640 1 --2- 172.21.15.119:0/1320289276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b840b5050 0x7f4b841e5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.287+0000 7f4b82800640 1 -- 172.21.15.119:0/1320289276 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8415cca0 con 0x7f4b84144470 2024-04-25T10:18:23.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.287+0000 7f4b82800640 1 --2- 172.21.15.119:0/1320289276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b84144470 0x7f4b8415b1f0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00cd50 tx=0x7f4b7c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.288+0000 7f4b7a000640 1 -- 172.21.15.119:0/1320289276 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c003b90 con 0x7f4b84144470 2024-04-25T10:18:23.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.288+0000 7f4b7a000640 1 -- 172.21.15.119:0/1320289276 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c003d30 con 0x7f4b84144470 2024-04-25T10:18:23.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.288+0000 7f4b83200640 1 -- 172.21.15.119:0/1320289276 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8415cf80 con 0x7f4b84144470 2024-04-25T10:18:23.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.288+0000 7f4b7a000640 1 -- 172.21.15.119:0/1320289276 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c004240 con 0x7f4b84144470 2024-04-25T10:18:23.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.288+0000 7f4b83200640 1 -- 172.21.15.119:0/1320289276 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8415d540 con 0x7f4b84144470 2024-04-25T10:18:23.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.289+0000 7f4b83200640 1 -- 172.21.15.119:0/1320289276 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b840b1e50 con 0x7f4b84144470 2024-04-25T10:18:23.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.290+0000 7f4b7a000640 1 -- 172.21.15.119:0/1320289276 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b7c0043e0 con 0x7f4b84144470 2024-04-25T10:18:23.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.291+0000 7f4b7a000640 1 --2- 172.21.15.119:0/1320289276 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b6406b500 0x7f4b6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.291+0000 7f4b7a000640 1 -- 172.21.15.119:0/1320289276 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f4b7c08ad40 con 0x7f4b84144470 2024-04-25T10:18:23.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.291+0000 7f4b81e00640 1 --2- 172.21.15.119:0/1320289276 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b6406b500 0x7f4b6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.292+0000 7f4b81e00640 1 --2- 172.21.15.119:0/1320289276 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b6406b500 0x7f4b6406d9c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4b74034000 tx=0x7f4b7402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.295+0000 7f4b7a000640 1 -- 172.21.15.119:0/1320289276 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b840b1e50 con 0x7f4b84144470 2024-04-25T10:18:23.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.490+0000 7f4b83200640 1 -- 172.21.15.119:0/1320289276 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b840749b0 con 0x7f4b84144470 2024-04-25T10:18:23.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.491+0000 7f4b7a000640 1 -- 172.21.15.119:0/1320289276 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+17079 (secure 0 0 0) 0x7f4b7c059ac0 con 0x7f4b84144470 2024-04-25T10:18:23.495 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:23.495 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":112,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:18:09.666428+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:35.422922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:36.634584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.25","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/2864575636":"2024-04-26T10:18:09.666363+0000","172.21.15.119:6836/2864575636":"2024-04-26T10:18:09.666363+0000","172.21.15.119:6837/1849584205":"2024-04-26T10:17:56.438859+0000","172.21.15.119:6836/1849584205":"2024-04-26T10:17:56.438859+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:23.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.494+0000 7f4b83200640 1 -- 172.21.15.119:0/1320289276 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b6406b500 msgr2=0x7f4b6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.494+0000 7f4b83200640 1 --2- 172.21.15.119:0/1320289276 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b6406b500 0x7f4b6406d9c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4b74034000 tx=0x7f4b7402c000 comp rx=0 tx=0).stop 2024-04-25T10:18:23.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.494+0000 7f4b83200640 1 -- 172.21.15.119:0/1320289276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b84144470 msgr2=0x7f4b8415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.494+0000 7f4b83200640 1 --2- 172.21.15.119:0/1320289276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b84144470 0x7f4b8415b1f0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00cd50 tx=0x7f4b7c00a660 comp rx=0 tx=0).stop 2024-04-25T10:18:23.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.494+0000 7f4b83200640 1 -- 172.21.15.119:0/1320289276 shutdown_connections 2024-04-25T10:18:23.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.494+0000 7f4b83200640 1 --2- 172.21.15.119:0/1320289276 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4b6406b500 0x7f4b6406d9c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.494+0000 7f4b83200640 1 --2- 172.21.15.119:0/1320289276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b84144470 0x7f4b8415b1f0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.494+0000 7f4b83200640 1 --2- 172.21.15.119:0/1320289276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b840b5050 0x7f4b841e5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.494+0000 7f4b83200640 1 --2- 172.21.15.119:0/1320289276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b8406e430 0x7f4b8415c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.495+0000 7f4b83200640 1 -- 172.21.15.119:0/1320289276 >> 172.21.15.119:0/1320289276 conn(0x7f4b8406b1d0 msgr2=0x7f4b840af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:23.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.495+0000 7f4b83200640 1 -- 172.21.15.119:0/1320289276 shutdown_connections 2024-04-25T10:18:23.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.495+0000 7f4b83200640 1 -- 172.21.15.119:0/1320289276 wait complete. 2024-04-25T10:18:23.517 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:06:55.994187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:35.422922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:17:36.634584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:23.518 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:23.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.721+0000 7f005b200640 1 Processor -- start 2024-04-25T10:18:23.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.722+0000 7f005b200640 1 -- start start 2024-04-25T10:18:23.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.723+0000 7f005b200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f005c06aba0 0x7f005c06af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.723+0000 7f005b200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f005c15f000 con 0x7f005c0ce2f0 2024-04-25T10:18:23.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.723+0000 7f005b200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f005c06b4c0 con 0x7f005c06f040 2024-04-25T10:18:23.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.723+0000 7f005b200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f005c06d080 con 0x7f005c06aba0 2024-04-25T10:18:23.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.723+0000 7f0059e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f005c06aba0 0x7f005c06af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.723+0000 7f0059e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f005c06aba0 0x7f005c06af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60968/0 (socket says 172.21.15.119:60968) 2024-04-25T10:18:23.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.723+0000 7f0059e00640 1 -- 172.21.15.119:0/1063750962 learned_addr learned my addr 172.21.15.119:0/1063750962 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:23.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.723+0000 7f005a800640 1 --1- 172.21.15.119:0/1063750962 >> v1:172.21.15.119:6789/0 conn(0x7f005c0ce2f0 0x7f005c15e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42566/0 (socket says 172.21.15.119:42566) 2024-04-25T10:18:23.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.724+0000 7f0059e00640 1 -- 172.21.15.119:0/1063750962 >> v1:172.21.15.132:6789/0 conn(0x7f005c06f040 legacy=0x7f005c15e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:23.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.724+0000 7f0059e00640 1 -- 172.21.15.119:0/1063750962 >> v1:172.21.15.119:6789/0 conn(0x7f005c0ce2f0 legacy=0x7f005c15e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.724+0000 7f0059e00640 1 -- 172.21.15.119:0/1063750962 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005c06d360 con 0x7f005c06aba0 2024-04-25T10:18:23.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.724+0000 7f0059400640 1 -- 172.21.15.119:0/1063750962 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22781907 0 0) 0x7f005c15f000 con 0x7f005c0ce2f0 2024-04-25T10:18:23.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.725+0000 7f0059e00640 1 --2- 172.21.15.119:0/1063750962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f005c06aba0 0x7f005c06af80 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f004c009e50 tx=0x7f004c02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210e86377ce1d7c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.725+0000 7f0059400640 1 -- 172.21.15.119:0/1063750962 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c036070 con 0x7f005c06aba0 2024-04-25T10:18:23.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.725+0000 7f0059400640 1 -- 172.21.15.119:0/1063750962 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c031600 con 0x7f005c06aba0 2024-04-25T10:18:23.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.725+0000 7f0059400640 1 -- 172.21.15.119:0/1063750962 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c0319d0 con 0x7f005c06aba0 2024-04-25T10:18:23.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.726+0000 7f005b200640 1 -- 172.21.15.119:0/1063750962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f005c06aba0 msgr2=0x7f005c06af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.726+0000 7f005b200640 1 --2- 172.21.15.119:0/1063750962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f005c06aba0 0x7f005c06af80 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f004c009e50 tx=0x7f004c02a760 comp rx=0 tx=0).stop 2024-04-25T10:18:23.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.726+0000 7f005b200640 1 -- 172.21.15.119:0/1063750962 shutdown_connections 2024-04-25T10:18:23.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.726+0000 7f005b200640 1 --2- 172.21.15.119:0/1063750962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f005c06aba0 0x7f005c06af80 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.726+0000 7f005b200640 1 -- 172.21.15.119:0/1063750962 >> 172.21.15.119:0/1063750962 conn(0x7f005c066ff0 msgr2=0x7f005c0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:23.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.726+0000 7f005b200640 1 -- 172.21.15.119:0/1063750962 shutdown_connections 2024-04-25T10:18:23.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.726+0000 7f005b200640 1 -- 172.21.15.119:0/1063750962 wait complete. 2024-04-25T10:18:23.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.728+0000 7f005b200640 1 Processor -- start 2024-04-25T10:18:23.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.728+0000 7f005b200640 1 -- start start 2024-04-25T10:18:23.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.729+0000 7f005b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f005c06aba0 0x7f005c1e7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.729+0000 7f005b200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f005c06f040 0x7f005c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.729+0000 7f005b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f005c0ce2f0 0x7f005c14e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.729+0000 7f005b200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c06dc00 con 0x7f005c06f040 2024-04-25T10:18:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.729+0000 7f005b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c06d920 con 0x7f005c0ce2f0 2024-04-25T10:18:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.729+0000 7f005b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005c15a980 con 0x7f005c06aba0 2024-04-25T10:18:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.729+0000 7f0051400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f005c06f040 0x7f005c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.729+0000 7f0051400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f005c06f040 0x7f005c152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60978/0 (socket says 172.21.15.119:60978) 2024-04-25T10:18:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.729+0000 7f0051400640 1 -- 172.21.15.119:0/4007367492 learned_addr learned my addr 172.21.15.119:0/4007367492 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.729+0000 7f0051400640 1 -- 172.21.15.119:0/4007367492 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f005c06aba0 msgr2=0x7f005c1e7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.730+0000 7f0051400640 1 --2- 172.21.15.119:0/4007367492 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f005c06aba0 0x7f005c1e7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.730+0000 7f0051400640 1 -- 172.21.15.119:0/4007367492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f005c0ce2f0 msgr2=0x7f005c14e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.730+0000 7f0051400640 1 --2- 172.21.15.119:0/4007367492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f005c0ce2f0 0x7f005c14e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.730+0000 7f0051400640 1 -- 172.21.15.119:0/4007367492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005c1e6bb0 con 0x7f005c06f040 2024-04-25T10:18:23.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.730+0000 7f0051400640 1 --2- 172.21.15.119:0/4007367492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f005c06f040 0x7f005c152880 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f004400c430 tx=0x7f004400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.730+0000 7f0052a00640 1 -- 172.21.15.119:0/4007367492 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044011790 con 0x7f005c06f040 2024-04-25T10:18:23.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.730+0000 7f0052a00640 1 -- 172.21.15.119:0/4007367492 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0044017070 con 0x7f005c06f040 2024-04-25T10:18:23.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.730+0000 7f0052a00640 1 -- 172.21.15.119:0/4007367492 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044011b70 con 0x7f005c06f040 2024-04-25T10:18:23.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.730+0000 7f005b200640 1 -- 172.21.15.119:0/4007367492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005c1e51e0 con 0x7f005c06f040 2024-04-25T10:18:23.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.730+0000 7f005b200640 1 -- 172.21.15.119:0/4007367492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005c1e5830 con 0x7f005c06f040 2024-04-25T10:18:23.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.731+0000 7f005b200640 1 -- 172.21.15.119:0/4007367492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f005c0749b0 con 0x7f005c06f040 2024-04-25T10:18:23.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.732+0000 7f0052a00640 1 -- 172.21.15.119:0/4007367492 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0044011d10 con 0x7f005c06f040 2024-04-25T10:18:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.733+0000 7f0052a00640 1 --2- 172.21.15.119:0/4007367492 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f003806b500 0x7f003806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.733+0000 7f0052a00640 1 -- 172.21.15.119:0/4007367492 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f004408a130 con 0x7f005c06f040 2024-04-25T10:18:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.733+0000 7f0059e00640 1 --2- 172.21.15.119:0/4007367492 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f003806b500 0x7f003806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.734+0000 7f0059e00640 1 --2- 172.21.15.119:0/4007367492 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f003806b500 0x7f003806d9c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f004c033000 tx=0x7f004c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.738+0000 7f0052a00640 1 -- 172.21.15.119:0/4007367492 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0044058ec0 con 0x7f005c06f040 2024-04-25T10:18:23.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.932+0000 7f005b200640 1 -- 172.21.15.119:0/4007367492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f005c0cf1b0 con 0x7f005c06f040 2024-04-25T10:18:23.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.933+0000 7f0052a00640 1 -- 172.21.15.119:0/4007367492 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f00440587c0 con 0x7f005c06f040 2024-04-25T10:18:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-04-25T10:18:23.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.936+0000 7f005b200640 1 -- 172.21.15.119:0/4007367492 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f003806b500 msgr2=0x7f003806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.936+0000 7f005b200640 1 --2- 172.21.15.119:0/4007367492 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f003806b500 0x7f003806d9c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f004c033000 tx=0x7f004c02e000 comp rx=0 tx=0).stop 2024-04-25T10:18:23.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.936+0000 7f005b200640 1 -- 172.21.15.119:0/4007367492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f005c06f040 msgr2=0x7f005c152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.936+0000 7f005b200640 1 --2- 172.21.15.119:0/4007367492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f005c06f040 0x7f005c152880 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f004400c430 tx=0x7f004400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.936+0000 7f005b200640 1 -- 172.21.15.119:0/4007367492 shutdown_connections 2024-04-25T10:18:23.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.936+0000 7f005b200640 1 --2- 172.21.15.119:0/4007367492 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f003806b500 0x7f003806d9c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.936+0000 7f005b200640 1 --2- 172.21.15.119:0/4007367492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f005c0ce2f0 0x7f005c14e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.936+0000 7f005b200640 1 --2- 172.21.15.119:0/4007367492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f005c06f040 0x7f005c152880 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.936+0000 7f005b200640 1 --2- 172.21.15.119:0/4007367492 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f005c06aba0 0x7f005c1e7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.936+0000 7f005b200640 1 -- 172.21.15.119:0/4007367492 >> 172.21.15.119:0/4007367492 conn(0x7f005c066ff0 msgr2=0x7f005c0b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:23.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.936+0000 7f005b200640 1 -- 172.21.15.119:0/4007367492 shutdown_connections 2024-04-25T10:18:23.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:23.936+0000 7f005b200640 1 -- 172.21.15.119:0/4007367492 wait complete. 2024-04-25T10:18:23.959 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:24.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.141+0000 7f48b9a00640 1 Processor -- start 2024-04-25T10:18:24.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.141+0000 7f48b9a00640 1 -- start start 2024-04-25T10:18:24.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.143+0000 7f48b9a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b40ce860 0x7f48b40cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.143+0000 7f48b9a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f48b40d00a0 0x7f48b40d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.143+0000 7f48b9a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b40d0a70 con 0x7f48b415b7e0 2024-04-25T10:18:24.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.143+0000 7f48b9a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b40b5710 con 0x7f48b40d00a0 2024-04-25T10:18:24.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.143+0000 7f48b9a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b40b59f0 con 0x7f48b40ce860 2024-04-25T10:18:24.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.144+0000 7f48b9000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f48b40d00a0 0x7f48b40d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.144+0000 7f48b3400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b40ce860 0x7f48b40cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.144+0000 7f48b3400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b40ce860 0x7f48b40cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.119:49862/0 (socket says 172.21.15.119:49862) 2024-04-25T10:18:24.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.144+0000 7f48b3400640 1 -- 172.21.15.119:0/3643678923 learned_addr learned my addr 172.21.15.119:0/3643678923 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:24.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.145+0000 7f48b3400640 1 -- 172.21.15.119:0/3643678923 >> v2:172.21.15.132:3300/0 conn(0x7f48b40d00a0 msgr2=0x7f48b40d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:24.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.145+0000 7f48b3400640 1 --2- 172.21.15.119:0/3643678923 >> v2:172.21.15.132:3300/0 conn(0x7f48b40d00a0 0x7f48b40d0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.145+0000 7f48b3400640 1 -- 172.21.15.119:0/3643678923 >> v1:172.21.15.132:6789/0 conn(0x7f48b415b7e0 legacy=0x7f48b415e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.145+0000 7f48b3400640 1 -- 172.21.15.119:0/3643678923 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b40b5cd0 con 0x7f48b40ce860 2024-04-25T10:18:24.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.145+0000 7f48b9000640 1 --2- 172.21.15.119:0/3643678923 >> v2:172.21.15.132:3300/0 conn(0x7f48b40d00a0 0x7f48b40d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:24.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.146+0000 7f48b3400640 1 --2- 172.21.15.119:0/3643678923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b40ce860 0x7f48b40cecc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f489c009e50 tx=0x7f489c02a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb4b8fe0ce7a8759 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.146+0000 7f48b2a00640 1 -- 172.21.15.119:0/3643678923 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c036070 con 0x7f48b40ce860 2024-04-25T10:18:24.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.146+0000 7f48b2a00640 1 -- 172.21.15.119:0/3643678923 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c031600 con 0x7f48b40ce860 2024-04-25T10:18:24.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.146+0000 7f48b2a00640 1 -- 172.21.15.119:0/3643678923 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c0319d0 con 0x7f48b40ce860 2024-04-25T10:18:24.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.147+0000 7f48b9a00640 1 -- 172.21.15.119:0/3643678923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b40ce860 msgr2=0x7f48b40cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.147+0000 7f48b9a00640 1 --2- 172.21.15.119:0/3643678923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b40ce860 0x7f48b40cecc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f489c009e50 tx=0x7f489c02a610 comp rx=0 tx=0).stop 2024-04-25T10:18:24.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.147+0000 7f48b9a00640 1 -- 172.21.15.119:0/3643678923 shutdown_connections 2024-04-25T10:18:24.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.147+0000 7f48b9a00640 1 --2- 172.21.15.119:0/3643678923 >> v2:172.21.15.132:3300/0 conn(0x7f48b40d00a0 0x7f48b40d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.147+0000 7f48b9a00640 1 --2- 172.21.15.119:0/3643678923 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b40ce860 0x7f48b40cecc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.147+0000 7f48b9a00640 1 -- 172.21.15.119:0/3643678923 >> 172.21.15.119:0/3643678923 conn(0x7f48b40722d0 msgr2=0x7f48b40726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:24.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.147+0000 7f48b9a00640 1 -- 172.21.15.119:0/3643678923 shutdown_connections 2024-04-25T10:18:24.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.147+0000 7f48b9a00640 1 -- 172.21.15.119:0/3643678923 wait complete. 2024-04-25T10:18:24.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.149+0000 7f48b9a00640 1 Processor -- start 2024-04-25T10:18:24.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.149+0000 7f48b9a00640 1 -- start start 2024-04-25T10:18:24.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b9a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48b40ce860 0x7f48b41408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b9a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b40d00a0 0x7f48b4142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b9a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48b415b7e0 0x7f48b4158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b9a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4161960 con 0x7f48b415b7e0 2024-04-25T10:18:24.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b9a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b40b53c0 con 0x7f48b40ce860 2024-04-25T10:18:24.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b9a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4161ae0 con 0x7f48b40d00a0 2024-04-25T10:18:24.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b9000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48b415b7e0 0x7f48b4158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b9000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48b415b7e0 0x7f48b4158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60982/0 (socket says 172.21.15.119:60982) 2024-04-25T10:18:24.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b9000640 1 -- 172.21.15.119:0/2838652771 learned_addr learned my addr 172.21.15.119:0/2838652771 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:24.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b9000640 1 -- 172.21.15.119:0/2838652771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b40d00a0 msgr2=0x7f48b4142150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b9000640 1 --2- 172.21.15.119:0/2838652771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b40d00a0 0x7f48b4142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b9000640 1 -- 172.21.15.119:0/2838652771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48b40ce860 msgr2=0x7f48b41408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b3e00640 1 --2- 172.21.15.119:0/2838652771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48b40ce860 0x7f48b41408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b9000640 1 --2- 172.21.15.119:0/2838652771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48b40ce860 0x7f48b41408a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.150+0000 7f48b9000640 1 -- 172.21.15.119:0/2838652771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b4140de0 con 0x7f48b415b7e0 2024-04-25T10:18:24.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.151+0000 7f48b9000640 1 --2- 172.21.15.119:0/2838652771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48b415b7e0 0x7f48b4158f90 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00e960 tx=0x7f48ac00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.151+0000 7f48abe00640 1 -- 172.21.15.119:0/2838652771 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac017070 con 0x7f48b415b7e0 2024-04-25T10:18:24.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.151+0000 7f48abe00640 1 -- 172.21.15.119:0/2838652771 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac004020 con 0x7f48b415b7e0 2024-04-25T10:18:24.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.151+0000 7f48abe00640 1 -- 172.21.15.119:0/2838652771 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac0043f0 con 0x7f48b415b7e0 2024-04-25T10:18:24.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.151+0000 7f48b9a00640 1 -- 172.21.15.119:0/2838652771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b4142690 con 0x7f48b415b7e0 2024-04-25T10:18:24.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.151+0000 7f48b9a00640 1 -- 172.21.15.119:0/2838652771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b4141390 con 0x7f48b415b7e0 2024-04-25T10:18:24.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.152+0000 7f48b9a00640 1 -- 172.21.15.119:0/2838652771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b40749b0 con 0x7f48b415b7e0 2024-04-25T10:18:24.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.153+0000 7f48abe00640 1 -- 172.21.15.119:0/2838652771 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48ac00ca00 con 0x7f48b415b7e0 2024-04-25T10:18:24.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.154+0000 7f48abe00640 1 --2- 172.21.15.119:0/2838652771 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f489006b500 0x7f489006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.154+0000 7f48abe00640 1 -- 172.21.15.119:0/2838652771 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f48ac08a1f0 con 0x7f48b415b7e0 2024-04-25T10:18:24.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.154+0000 7f48b3e00640 1 --2- 172.21.15.119:0/2838652771 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f489006b500 0x7f489006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.155+0000 7f48b3e00640 1 --2- 172.21.15.119:0/2838652771 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f489006b500 0x7f489006d9c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f48b40b91c0 tx=0x7f48a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.158+0000 7f48abe00640 1 -- 172.21.15.119:0/2838652771 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48ac058ee0 con 0x7f48b415b7e0 2024-04-25T10:18:24.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.339+0000 7f48b9a00640 1 -- 172.21.15.119:0/2838652771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48b40d1650 con 0x7f48b415b7e0 2024-04-25T10:18:24.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.340+0000 7f48abe00640 1 -- 172.21.15.119:0/2838652771 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f48ac0587e0 con 0x7f48b415b7e0 2024-04-25T10:18:24.343 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2024-04-25T10:18:24.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.342+0000 7f48b9a00640 1 -- 172.21.15.119:0/2838652771 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f489006b500 msgr2=0x7f489006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.342+0000 7f48b9a00640 1 --2- 172.21.15.119:0/2838652771 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f489006b500 0x7f489006d9c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f48b40b91c0 tx=0x7f48a4009040 comp rx=0 tx=0).stop 2024-04-25T10:18:24.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.342+0000 7f48b9a00640 1 -- 172.21.15.119:0/2838652771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48b415b7e0 msgr2=0x7f48b4158f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.342+0000 7f48b9a00640 1 --2- 172.21.15.119:0/2838652771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48b415b7e0 0x7f48b4158f90 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00e960 tx=0x7f48ac00ee20 comp rx=0 tx=0).stop 2024-04-25T10:18:24.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.342+0000 7f48b9a00640 1 -- 172.21.15.119:0/2838652771 shutdown_connections 2024-04-25T10:18:24.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.342+0000 7f48b9a00640 1 --2- 172.21.15.119:0/2838652771 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f489006b500 0x7f489006d9c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.342+0000 7f48b9a00640 1 --2- 172.21.15.119:0/2838652771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48b415b7e0 0x7f48b4158f90 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.342+0000 7f48b9a00640 1 --2- 172.21.15.119:0/2838652771 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b40d00a0 0x7f48b4142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.342+0000 7f48b9a00640 1 --2- 172.21.15.119:0/2838652771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48b40ce860 0x7f48b41408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.342+0000 7f48b9a00640 1 -- 172.21.15.119:0/2838652771 >> 172.21.15.119:0/2838652771 conn(0x7f48b40722d0 msgr2=0x7f48b4150b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:24.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.343+0000 7f48b9a00640 1 -- 172.21.15.119:0/2838652771 shutdown_connections 2024-04-25T10:18:24.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.343+0000 7f48b9a00640 1 -- 172.21.15.119:0/2838652771 wait complete. 2024-04-25T10:18:24.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:24.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.554+0000 7ff0db000640 1 Processor -- start 2024-04-25T10:18:24.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.554+0000 7ff0db000640 1 -- start start 2024-04-25T10:18:24.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.556+0000 7ff0db000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0d406aba0 0x7ff0d406af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.556+0000 7ff0db000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0d40ce2f0 0x7ff0d415e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.556+0000 7ff0db000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d406b4c0 con 0x7ff0d406f040 2024-04-25T10:18:24.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.556+0000 7ff0db000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d406d080 con 0x7ff0d40ce2f0 2024-04-25T10:18:24.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.556+0000 7ff0db000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d406d360 con 0x7ff0d406aba0 2024-04-25T10:18:24.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.556+0000 7ff0da600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0d40ce2f0 0x7ff0d415e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.556+0000 7ff0da600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0d40ce2f0 0x7ff0d415e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60994/0 (socket says 172.21.15.119:60994) 2024-04-25T10:18:24.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.556+0000 7ff0da600640 1 -- 172.21.15.119:0/117568875 learned_addr learned my addr 172.21.15.119:0/117568875 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:24.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.556+0000 7ff0d9c00640 1 --2- 172.21.15.119:0/117568875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0d406aba0 0x7ff0d406af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.556+0000 7ff0da600640 1 -- 172.21.15.119:0/117568875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0d406aba0 msgr2=0x7ff0d406af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:24.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.556+0000 7ff0da600640 1 --2- 172.21.15.119:0/117568875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0d406aba0 0x7ff0d406af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.557+0000 7ff0da600640 1 -- 172.21.15.119:0/117568875 >> v1:172.21.15.132:6789/0 conn(0x7ff0d406f040 legacy=0x7ff0d415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:24.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.557+0000 7ff0da600640 1 -- 172.21.15.119:0/117568875 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d406d640 con 0x7ff0d40ce2f0 2024-04-25T10:18:24.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.557+0000 7ff0da600640 1 --2- 172.21.15.119:0/117568875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d40ce2f0 0x7ff0d415e8f0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00d150 tx=0x7ff0cc02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c335241ca0d67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.558+0000 7ff0d3e00640 1 -- 172.21.15.119:0/117568875 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc00b2e0 con 0x7ff0d40ce2f0 2024-04-25T10:18:24.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.558+0000 7ff0d3e00640 1 -- 172.21.15.119:0/117568875 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc009600 con 0x7ff0d40ce2f0 2024-04-25T10:18:24.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.558+0000 7ff0d3e00640 1 -- 172.21.15.119:0/117568875 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc00b480 con 0x7ff0d40ce2f0 2024-04-25T10:18:24.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.558+0000 7ff0db000640 1 -- 172.21.15.119:0/117568875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d40ce2f0 msgr2=0x7ff0d415e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.558+0000 7ff0db000640 1 --2- 172.21.15.119:0/117568875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d40ce2f0 0x7ff0d415e8f0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00d150 tx=0x7ff0cc02c8e0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.559+0000 7ff0db000640 1 -- 172.21.15.119:0/117568875 shutdown_connections 2024-04-25T10:18:24.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.559+0000 7ff0db000640 1 --2- 172.21.15.119:0/117568875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d40ce2f0 0x7ff0d415e8f0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.559+0000 7ff0db000640 1 --2- 172.21.15.119:0/117568875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0d406aba0 0x7ff0d406af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.559+0000 7ff0db000640 1 -- 172.21.15.119:0/117568875 >> 172.21.15.119:0/117568875 conn(0x7ff0d4066ff0 msgr2=0x7ff0d40b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:24.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.559+0000 7ff0db000640 1 -- 172.21.15.119:0/117568875 shutdown_connections 2024-04-25T10:18:24.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.559+0000 7ff0db000640 1 -- 172.21.15.119:0/117568875 wait complete. 2024-04-25T10:18:24.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.560+0000 7ff0db000640 1 Processor -- start 2024-04-25T10:18:24.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.560+0000 7ff0db000640 1 -- start start 2024-04-25T10:18:24.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.561+0000 7ff0db000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0d406aba0 0x7ff0d414efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.561+0000 7ff0db000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d406f040 0x7ff0d415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.561+0000 7ff0db000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0d40ce2f0 0x7ff0d415d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.561+0000 7ff0db000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d415a980 con 0x7ff0d406f040 2024-04-25T10:18:24.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.561+0000 7ff0db000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d406dc00 con 0x7ff0d406aba0 2024-04-25T10:18:24.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.561+0000 7ff0db000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d40aed40 con 0x7ff0d40ce2f0 2024-04-25T10:18:24.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.562+0000 7ff0d9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d406f040 0x7ff0d415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.562+0000 7ff0d9200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d406f040 0x7ff0d415d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32774/0 (socket says 172.21.15.119:32774) 2024-04-25T10:18:24.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.562+0000 7ff0d9200640 1 -- 172.21.15.119:0/4024458849 learned_addr learned my addr 172.21.15.119:0/4024458849 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:24.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.562+0000 7ff0d9c00640 1 --2- 172.21.15.119:0/4024458849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0d406aba0 0x7ff0d414efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.562+0000 7ff0d9200640 1 -- 172.21.15.119:0/4024458849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0d40ce2f0 msgr2=0x7ff0d415d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.562+0000 7ff0d9200640 1 --2- 172.21.15.119:0/4024458849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0d40ce2f0 0x7ff0d415d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.562+0000 7ff0d9200640 1 -- 172.21.15.119:0/4024458849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0d406aba0 msgr2=0x7ff0d414efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.562+0000 7ff0d9200640 1 --2- 172.21.15.119:0/4024458849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0d406aba0 0x7ff0d414efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.562+0000 7ff0d9200640 1 -- 172.21.15.119:0/4024458849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d41e6200 con 0x7ff0d406f040 2024-04-25T10:18:24.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.562+0000 7ff0d9200640 1 --2- 172.21.15.119:0/4024458849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d406f040 0x7ff0d415d1f0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000c430 tx=0x7ff0c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.562+0000 7ff0d1600640 1 -- 172.21.15.119:0/4024458849 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0011790 con 0x7ff0d406f040 2024-04-25T10:18:24.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.563+0000 7ff0d1600640 1 -- 172.21.15.119:0/4024458849 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c0017070 con 0x7ff0d406f040 2024-04-25T10:18:24.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.563+0000 7ff0d1600640 1 -- 172.21.15.119:0/4024458849 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0011b70 con 0x7ff0d406f040 2024-04-25T10:18:24.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.563+0000 7ff0db000640 1 -- 172.21.15.119:0/4024458849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d41e64e0 con 0x7ff0d406f040 2024-04-25T10:18:24.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.563+0000 7ff0db000640 1 -- 172.21.15.119:0/4024458849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d41f3970 con 0x7ff0d406f040 2024-04-25T10:18:24.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.564+0000 7ff0db000640 1 -- 172.21.15.119:0/4024458849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d4150a80 con 0x7ff0d406f040 2024-04-25T10:18:24.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.565+0000 7ff0d1600640 1 -- 172.21.15.119:0/4024458849 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0c0011d10 con 0x7ff0d406f040 2024-04-25T10:18:24.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.566+0000 7ff0d1600640 1 --2- 172.21.15.119:0/4024458849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff0b406b4d0 0x7ff0b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.566+0000 7ff0d1600640 1 -- 172.21.15.119:0/4024458849 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff0c008a1c0 con 0x7ff0d406f040 2024-04-25T10:18:24.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.566+0000 7ff0d9c00640 1 --2- 172.21.15.119:0/4024458849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff0b406b4d0 0x7ff0b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.567+0000 7ff0d9c00640 1 --2- 172.21.15.119:0/4024458849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff0b406b4d0 0x7ff0b406d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8006fd0 tx=0x7ff0c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.571+0000 7ff0d1600640 1 -- 172.21.15.119:0/4024458849 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0c0058f40 con 0x7ff0d406f040 2024-04-25T10:18:24.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.771+0000 7ff0db000640 1 -- 172.21.15.119:0/4024458849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0d40749b0 con 0x7ff0d406f040 2024-04-25T10:18:24.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.772+0000 7ff0d1600640 1 -- 172.21.15.119:0/4024458849 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff0c0058840 con 0x7ff0d406f040 2024-04-25T10:18:24.775 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2024-04-25T10:18:24.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.775+0000 7ff0db000640 1 -- 172.21.15.119:0/4024458849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff0b406b4d0 msgr2=0x7ff0b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.775+0000 7ff0db000640 1 --2- 172.21.15.119:0/4024458849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff0b406b4d0 0x7ff0b406d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8006fd0 tx=0x7ff0c8008040 comp rx=0 tx=0).stop 2024-04-25T10:18:24.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.775+0000 7ff0db000640 1 -- 172.21.15.119:0/4024458849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d406f040 msgr2=0x7ff0d415d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.775+0000 7ff0db000640 1 --2- 172.21.15.119:0/4024458849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d406f040 0x7ff0d415d1f0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000c430 tx=0x7ff0c000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.775+0000 7ff0db000640 1 -- 172.21.15.119:0/4024458849 shutdown_connections 2024-04-25T10:18:24.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.775+0000 7ff0db000640 1 --2- 172.21.15.119:0/4024458849 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff0b406b4d0 0x7ff0b406d990 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.775+0000 7ff0db000640 1 --2- 172.21.15.119:0/4024458849 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0d40ce2f0 0x7ff0d415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.775+0000 7ff0db000640 1 --2- 172.21.15.119:0/4024458849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d406f040 0x7ff0d415d1f0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.775+0000 7ff0db000640 1 --2- 172.21.15.119:0/4024458849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0d406aba0 0x7ff0d414efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.775+0000 7ff0db000640 1 -- 172.21.15.119:0/4024458849 >> 172.21.15.119:0/4024458849 conn(0x7ff0d4066ff0 msgr2=0x7ff0d40b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:24.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.775+0000 7ff0db000640 1 -- 172.21.15.119:0/4024458849 shutdown_connections 2024-04-25T10:18:24.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.775+0000 7ff0db000640 1 -- 172.21.15.119:0/4024458849 wait complete. 2024-04-25T10:18:24.797 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:24.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.986+0000 7efc5ea00640 1 Processor -- start 2024-04-25T10:18:24.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.986+0000 7efc5ea00640 1 -- start start 2024-04-25T10:18:24.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.988+0000 7efc5ea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efc5815bf00 0x7efc580cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.988+0000 7efc5ea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc5806e8f0 0x7efc581654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.988+0000 7efc5ea00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc580ce6c0 con 0x7efc580cc310 2024-04-25T10:18:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.988+0000 7efc5ea00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc581455e0 con 0x7efc5806e8f0 2024-04-25T10:18:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.988+0000 7efc5ea00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5806d450 con 0x7efc5815bf00 2024-04-25T10:18:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.988+0000 7efc5ca00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc5806e8f0 0x7efc581654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.988+0000 7efc5ca00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc5806e8f0 0x7efc581654b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32776/0 (socket says 172.21.15.119:32776) 2024-04-25T10:18:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.988+0000 7efc5ca00640 1 -- 172.21.15.119:0/2968192071 learned_addr learned my addr 172.21.15.119:0/2968192071 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.988+0000 7efc57400640 1 --1- 172.21.15.119:0/2968192071 >> v1:172.21.15.119:6789/0 conn(0x7efc580cc310 0x7efc5815c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42576/0 (socket says 172.21.15.119:42576) 2024-04-25T10:18:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.989+0000 7efc57e00640 1 --2- 172.21.15.119:0/2968192071 >> v2:172.21.15.132:3300/0 conn(0x7efc5815bf00 0x7efc580cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.989+0000 7efc5ca00640 1 -- 172.21.15.119:0/2968192071 >> v2:172.21.15.132:3300/0 conn(0x7efc5815bf00 msgr2=0x7efc580cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.989+0000 7efc5ca00640 1 --2- 172.21.15.119:0/2968192071 >> v2:172.21.15.132:3300/0 conn(0x7efc5815bf00 0x7efc580cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.989+0000 7efc5ca00640 1 -- 172.21.15.119:0/2968192071 >> v1:172.21.15.119:6789/0 conn(0x7efc580cc310 legacy=0x7efc5815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.989+0000 7efc5ca00640 1 -- 172.21.15.119:0/2968192071 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc580cc7b0 con 0x7efc5806e8f0 2024-04-25T10:18:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.990+0000 7efc5ca00640 1 --2- 172.21.15.119:0/2968192071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5806e8f0 0x7efc581654b0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7efc5806ba10 tx=0x7efc5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87948465b0b08c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:24.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.990+0000 7efc56a00640 1 -- 172.21.15.119:0/2968192071 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50037070 con 0x7efc5806e8f0 2024-04-25T10:18:24.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.990+0000 7efc56a00640 1 -- 172.21.15.119:0/2968192071 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc50032440 con 0x7efc5806e8f0 2024-04-25T10:18:24.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.990+0000 7efc56a00640 1 -- 172.21.15.119:0/2968192071 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50032810 con 0x7efc5806e8f0 2024-04-25T10:18:24.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.991+0000 7efc5ea00640 1 -- 172.21.15.119:0/2968192071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5806e8f0 msgr2=0x7efc581654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.991+0000 7efc5ea00640 1 --2- 172.21.15.119:0/2968192071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5806e8f0 0x7efc581654b0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7efc5806ba10 tx=0x7efc5002a270 comp rx=0 tx=0).stop 2024-04-25T10:18:24.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.991+0000 7efc5ea00640 1 -- 172.21.15.119:0/2968192071 shutdown_connections 2024-04-25T10:18:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.991+0000 7efc5ea00640 1 --2- 172.21.15.119:0/2968192071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5806e8f0 0x7efc581654b0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.991+0000 7efc5ea00640 1 --2- 172.21.15.119:0/2968192071 >> v2:172.21.15.132:3300/0 conn(0x7efc5815bf00 0x7efc580cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.991+0000 7efc5ea00640 1 -- 172.21.15.119:0/2968192071 >> 172.21.15.119:0/2968192071 conn(0x7efc5806aba0 msgr2=0x7efc5806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.991+0000 7efc5ea00640 1 -- 172.21.15.119:0/2968192071 shutdown_connections 2024-04-25T10:18:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.991+0000 7efc5ea00640 1 -- 172.21.15.119:0/2968192071 wait complete. 2024-04-25T10:18:24.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.993+0000 7efc5ea00640 1 Processor -- start 2024-04-25T10:18:24.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.993+0000 7efc5ea00640 1 -- start start 2024-04-25T10:18:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.994+0000 7efc5ea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc5806e8f0 0x7efc58140660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.994+0000 7efc5ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cc310 0x7efc581f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.994+0000 7efc5ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc5815bf00 0x7efc581f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.994+0000 7efc5ea00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc580cd050 con 0x7efc580cc310 2024-04-25T10:18:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.994+0000 7efc5ea00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc580ccd70 con 0x7efc5806e8f0 2024-04-25T10:18:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.994+0000 7efc5ea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5806ee60 con 0x7efc5815bf00 2024-04-25T10:18:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.994+0000 7efc57400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cc310 0x7efc581f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.994+0000 7efc57400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cc310 0x7efc581f7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32790/0 (socket says 172.21.15.119:32790) 2024-04-25T10:18:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.994+0000 7efc57400640 1 -- 172.21.15.119:0/1837087846 learned_addr learned my addr 172.21.15.119:0/1837087846 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.995+0000 7efc57e00640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc5806e8f0 0x7efc58140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.995+0000 7efc5ca00640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc5815bf00 0x7efc581f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.995+0000 7efc57400640 1 -- 172.21.15.119:0/1837087846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc5815bf00 msgr2=0x7efc581f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.995+0000 7efc57400640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc5815bf00 0x7efc581f7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.995+0000 7efc57400640 1 -- 172.21.15.119:0/1837087846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc5806e8f0 msgr2=0x7efc58140660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.995+0000 7efc57400640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc5806e8f0 0x7efc58140660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.995+0000 7efc57400640 1 -- 172.21.15.119:0/1837087846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5813d920 con 0x7efc580cc310 2024-04-25T10:18:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.995+0000 7efc5ca00640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc5815bf00 0x7efc581f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.995+0000 7efc57400640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cc310 0x7efc581f7430 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7efc4000c7d0 tx=0x7efc4000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.995+0000 7efc4fe00640 1 -- 172.21.15.119:0/1837087846 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40017070 con 0x7efc580cc310 2024-04-25T10:18:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.995+0000 7efc4fe00640 1 -- 172.21.15.119:0/1837087846 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4000a640 con 0x7efc580cc310 2024-04-25T10:18:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.995+0000 7efc5ea00640 1 -- 172.21.15.119:0/1837087846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5813dc00 con 0x7efc580cc310 2024-04-25T10:18:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.996+0000 7efc4fe00640 1 -- 172.21.15.119:0/1837087846 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4000aa10 con 0x7efc580cc310 2024-04-25T10:18:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.996+0000 7efc5ea00640 1 -- 172.21.15.119:0/1837087846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5813e6e0 con 0x7efc580cc310 2024-04-25T10:18:25.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.997+0000 7efc5ea00640 1 -- 172.21.15.119:0/1837087846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5806bb30 con 0x7efc580cc310 2024-04-25T10:18:25.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.998+0000 7efc4fe00640 1 -- 172.21.15.119:0/1837087846 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc4001a070 con 0x7efc580cc310 2024-04-25T10:18:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.999+0000 7efc4fe00640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc3406b4d0 0x7efc3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.999+0000 7efc4fe00640 1 -- 172.21.15.119:0/1837087846 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7efc40089950 con 0x7efc580cc310 2024-04-25T10:18:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:24.999+0000 7efc57e00640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc3406b4d0 0x7efc3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.000+0000 7efc57e00640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc3406b4d0 0x7efc3406d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7efc48006fd0 tx=0x7efc48008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.003+0000 7efc4fe00640 1 -- 172.21.15.119:0/1837087846 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc580ce6c0 con 0x7efc580cc310 2024-04-25T10:18:25.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.238+0000 7efc5ea00640 1 -- 172.21.15.119:0/1837087846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc580749b0 con 0x7efc580cc310 2024-04-25T10:18:25.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.239+0000 7efc4fe00640 1 -- 172.21.15.119:0/1837087846 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4957 (secure 0 0 0) 0x7efc400586d0 con 0x7efc580cc310 2024-04-25T10:18:25.243 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:25.243 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3394261143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3394261143},{"type":"v1","addr":"172.21.15.132:6834","nonce":3394261143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2502874931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2502874931},{"type":"v1","addr":"172.21.15.119:6835","nonce":2502874931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":8694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2305428003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2305428003},{"type":"v1","addr":"172.21.15.132:6835","nonce":2305428003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:37.617787+0000","modified":"2024-04-25T10:18:13.835772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9117},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9117":{"gid":9117,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/2618740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2618740513},{"type":"v1","addr":"172.21.15.119:6837","nonce":2618740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:18:25.243 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 112 2024-04-25T10:18:25.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.242+0000 7efc5ea00640 1 -- 172.21.15.119:0/1837087846 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc3406b4d0 msgr2=0x7efc3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.242+0000 7efc5ea00640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc3406b4d0 0x7efc3406d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7efc48006fd0 tx=0x7efc48008040 comp rx=0 tx=0).stop 2024-04-25T10:18:25.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.242+0000 7efc5ea00640 1 -- 172.21.15.119:0/1837087846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cc310 msgr2=0x7efc581f7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.242+0000 7efc5ea00640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cc310 0x7efc581f7430 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7efc4000c7d0 tx=0x7efc4000cc90 comp rx=0 tx=0).stop 2024-04-25T10:18:25.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.242+0000 7efc5ea00640 1 -- 172.21.15.119:0/1837087846 shutdown_connections 2024-04-25T10:18:25.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.242+0000 7efc5ea00640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc3406b4d0 0x7efc3406d990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.242+0000 7efc5ea00640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc5815bf00 0x7efc581f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.242+0000 7efc5ea00640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc580cc310 0x7efc581f7430 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.242+0000 7efc5ea00640 1 --2- 172.21.15.119:0/1837087846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc5806e8f0 0x7efc58140660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.242+0000 7efc5ea00640 1 -- 172.21.15.119:0/1837087846 >> 172.21.15.119:0/1837087846 conn(0x7efc5806aba0 msgr2=0x7efc5806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:25.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.242+0000 7efc5ea00640 1 -- 172.21.15.119:0/1837087846 shutdown_connections 2024-04-25T10:18:25.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.242+0000 7efc5ea00640 1 -- 172.21.15.119:0/1837087846 wait complete. 2024-04-25T10:18:25.266 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:25.468 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.465+0000 7fe6e7800640 1 Processor -- start 2024-04-25T10:18:25.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.465+0000 7fe6e7800640 1 -- start start 2024-04-25T10:18:25.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.467+0000 7fe6e7800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe6e015bf00 0x7fe6e00cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.467+0000 7fe6e7800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe6e00cc310 0x7fe6e015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.467+0000 7fe6e7800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e00ce6c0 con 0x7fe6e015c820 2024-04-25T10:18:25.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.467+0000 7fe6e7800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e01455e0 con 0x7fe6e00cc310 2024-04-25T10:18:25.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.467+0000 7fe6e7800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e006d450 con 0x7fe6e015bf00 2024-04-25T10:18:25.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.467+0000 7fe6d7e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe6e00cc310 0x7fe6e015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.467+0000 7fe6d7e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe6e00cc310 0x7fe6e015c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32802/0 (socket says 172.21.15.119:32802) 2024-04-25T10:18:25.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.467+0000 7fe6d7e00640 1 -- 172.21.15.119:0/3175850906 learned_addr learned my addr 172.21.15.119:0/3175850906 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:25.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.468+0000 7fe6e4e00640 1 --2- 172.21.15.119:0/3175850906 >> v2:172.21.15.132:3300/0 conn(0x7fe6e015bf00 0x7fe6e00cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.468+0000 7fe6d7e00640 1 -- 172.21.15.119:0/3175850906 >> v2:172.21.15.132:3300/0 conn(0x7fe6e015bf00 msgr2=0x7fe6e00cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:25.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.468+0000 7fe6d7e00640 1 --2- 172.21.15.119:0/3175850906 >> v2:172.21.15.132:3300/0 conn(0x7fe6e015bf00 0x7fe6e00cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.468+0000 7fe6d7e00640 1 -- 172.21.15.119:0/3175850906 >> v1:172.21.15.132:6789/0 conn(0x7fe6e015c820 legacy=0x7fe6e006d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:25.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.468+0000 7fe6d7e00640 1 -- 172.21.15.119:0/3175850906 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e00cc7b0 con 0x7fe6e00cc310 2024-04-25T10:18:25.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.469+0000 7fe6d7e00640 1 --2- 172.21.15.119:0/3175850906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e00cc310 0x7fe6e015c2e0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fe6e006ba10 tx=0x7fe6c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874e84474a64e7db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.469+0000 7fe6dfe00640 1 -- 172.21.15.119:0/3175850906 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8037070 con 0x7fe6e00cc310 2024-04-25T10:18:25.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.469+0000 7fe6dfe00640 1 -- 172.21.15.119:0/3175850906 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c8032440 con 0x7fe6e00cc310 2024-04-25T10:18:25.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.469+0000 7fe6dfe00640 1 -- 172.21.15.119:0/3175850906 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8032810 con 0x7fe6e00cc310 2024-04-25T10:18:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.470+0000 7fe6e7800640 1 -- 172.21.15.119:0/3175850906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e00cc310 msgr2=0x7fe6e015c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.470+0000 7fe6e7800640 1 --2- 172.21.15.119:0/3175850906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e00cc310 0x7fe6e015c2e0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fe6e006ba10 tx=0x7fe6c802a230 comp rx=0 tx=0).stop 2024-04-25T10:18:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.470+0000 7fe6e7800640 1 -- 172.21.15.119:0/3175850906 shutdown_connections 2024-04-25T10:18:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.470+0000 7fe6e7800640 1 --2- 172.21.15.119:0/3175850906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e00cc310 0x7fe6e015c2e0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.470+0000 7fe6e7800640 1 --2- 172.21.15.119:0/3175850906 >> v2:172.21.15.132:3300/0 conn(0x7fe6e015bf00 0x7fe6e00cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.470+0000 7fe6e7800640 1 -- 172.21.15.119:0/3175850906 >> 172.21.15.119:0/3175850906 conn(0x7fe6e006aba0 msgr2=0x7fe6e006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.470+0000 7fe6e7800640 1 -- 172.21.15.119:0/3175850906 shutdown_connections 2024-04-25T10:18:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.470+0000 7fe6e7800640 1 -- 172.21.15.119:0/3175850906 wait complete. 2024-04-25T10:18:25.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.472+0000 7fe6e7800640 1 Processor -- start 2024-04-25T10:18:25.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.472+0000 7fe6e7800640 1 -- start start 2024-04-25T10:18:25.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.473+0000 7fe6e7800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e00cc310 0x7fe6e015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.473+0000 7fe6e7800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e015bf00 0x7fe6e01f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.473+0000 7fe6e7800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6e015c820 0x7fe6e0140510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.473+0000 7fe6e7800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e00cd050 con 0x7fe6e015bf00 2024-04-25T10:18:25.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.473+0000 7fe6e7800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e00ccd70 con 0x7fe6e00cc310 2024-04-25T10:18:25.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.473+0000 7fe6e7800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e015ccd0 con 0x7fe6e015c820 2024-04-25T10:18:25.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.473+0000 7fe6d7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e015bf00 0x7fe6e01f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.473+0000 7fe6d7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e015bf00 0x7fe6e01f3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32806/0 (socket says 172.21.15.119:32806) 2024-04-25T10:18:25.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.473+0000 7fe6d7e00640 1 -- 172.21.15.119:0/3254436230 learned_addr learned my addr 172.21.15.119:0/3254436230 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:25.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.474+0000 7fe6d7e00640 1 -- 172.21.15.119:0/3254436230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6e015c820 msgr2=0x7fe6e0140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.474+0000 7fe6d7e00640 1 --2- 172.21.15.119:0/3254436230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6e015c820 0x7fe6e0140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.474+0000 7fe6e4e00640 1 --2- 172.21.15.119:0/3254436230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e00cc310 0x7fe6e015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.474+0000 7fe6d7e00640 1 -- 172.21.15.119:0/3254436230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e00cc310 msgr2=0x7fe6e015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.474+0000 7fe6d7e00640 1 --2- 172.21.15.119:0/3254436230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e00cc310 0x7fe6e015afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.474+0000 7fe6d7e00640 1 -- 172.21.15.119:0/3254436230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e01e41c0 con 0x7fe6e015bf00 2024-04-25T10:18:25.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.474+0000 7fe6d7e00640 1 --2- 172.21.15.119:0/3254436230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e015bf00 0x7fe6e01f3ba0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8000c00 tx=0x7fe6c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.474+0000 7fe6dd600640 1 -- 172.21.15.119:0/3254436230 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8041020 con 0x7fe6e015bf00 2024-04-25T10:18:25.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.474+0000 7fe6dd600640 1 -- 172.21.15.119:0/3254436230 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c80306c0 con 0x7fe6e015bf00 2024-04-25T10:18:25.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.475+0000 7fe6dd600640 1 -- 172.21.15.119:0/3254436230 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8037040 con 0x7fe6e015bf00 2024-04-25T10:18:25.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.475+0000 7fe6e7800640 1 -- 172.21.15.119:0/3254436230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e01f40e0 con 0x7fe6e015bf00 2024-04-25T10:18:25.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.475+0000 7fe6e7800640 1 -- 172.21.15.119:0/3254436230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e01e4710 con 0x7fe6e015bf00 2024-04-25T10:18:25.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.476+0000 7fe6e7800640 1 -- 172.21.15.119:0/3254436230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e006bb30 con 0x7fe6e015bf00 2024-04-25T10:18:25.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.477+0000 7fe6dd600640 1 -- 172.21.15.119:0/3254436230 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe6c8044020 con 0x7fe6e015bf00 2024-04-25T10:18:25.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.478+0000 7fe6dd600640 1 --2- 172.21.15.119:0/3254436230 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe6bc06b4d0 0x7fe6bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.478+0000 7fe6dd600640 1 -- 172.21.15.119:0/3254436230 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fe6c80aabc0 con 0x7fe6e015bf00 2024-04-25T10:18:25.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.478+0000 7fe6e4e00640 1 --2- 172.21.15.119:0/3254436230 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe6bc06b4d0 0x7fe6bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.479+0000 7fe6e4e00640 1 --2- 172.21.15.119:0/3254436230 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe6bc06b4d0 0x7fe6bc06d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0006fd0 tx=0x7fe6d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.482+0000 7fe6dd600640 1 -- 172.21.15.119:0/3254436230 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe6c804b030 con 0x7fe6e015bf00 2024-04-25T10:18:25.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.677+0000 7fe6e7800640 1 -- 172.21.15.119:0/3254436230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6e0142790 con 0x7fe6e015bf00 2024-04-25T10:18:25.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.679+0000 7fe6dd600640 1 -- 172.21.15.119:0/3254436230 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+17079 (secure 0 0 0) 0x7fe6e0142790 con 0x7fe6e015bf00 2024-04-25T10:18:25.683 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:25.683 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":112,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:18:09.666428+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:35.422922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:36.634584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.25","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/2864575636":"2024-04-26T10:18:09.666363+0000","172.21.15.119:6836/2864575636":"2024-04-26T10:18:09.666363+0000","172.21.15.119:6837/1849584205":"2024-04-26T10:17:56.438859+0000","172.21.15.119:6836/1849584205":"2024-04-26T10:17:56.438859+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:25.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.682+0000 7fe6e7800640 1 -- 172.21.15.119:0/3254436230 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe6bc06b4d0 msgr2=0x7fe6bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.682+0000 7fe6e7800640 1 --2- 172.21.15.119:0/3254436230 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe6bc06b4d0 0x7fe6bc06d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0006fd0 tx=0x7fe6d0008040 comp rx=0 tx=0).stop 2024-04-25T10:18:25.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.682+0000 7fe6e7800640 1 -- 172.21.15.119:0/3254436230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e015bf00 msgr2=0x7fe6e01f3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.682+0000 7fe6e7800640 1 --2- 172.21.15.119:0/3254436230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e015bf00 0x7fe6e01f3ba0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8000c00 tx=0x7fe6c80046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.682+0000 7fe6e7800640 1 -- 172.21.15.119:0/3254436230 shutdown_connections 2024-04-25T10:18:25.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.682+0000 7fe6e7800640 1 --2- 172.21.15.119:0/3254436230 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe6bc06b4d0 0x7fe6bc06d990 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.682+0000 7fe6e7800640 1 --2- 172.21.15.119:0/3254436230 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6e015c820 0x7fe6e0140510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.682+0000 7fe6e7800640 1 --2- 172.21.15.119:0/3254436230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e015bf00 0x7fe6e01f3ba0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.682+0000 7fe6e7800640 1 --2- 172.21.15.119:0/3254436230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6e00cc310 0x7fe6e015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.682+0000 7fe6e7800640 1 -- 172.21.15.119:0/3254436230 >> 172.21.15.119:0/3254436230 conn(0x7fe6e006aba0 msgr2=0x7fe6e015d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:25.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.683+0000 7fe6e7800640 1 -- 172.21.15.119:0/3254436230 shutdown_connections 2024-04-25T10:18:25.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.683+0000 7fe6e7800640 1 -- 172.21.15.119:0/3254436230 wait complete. 2024-04-25T10:18:25.705 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:18:25.705 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:25.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.890+0000 7f03a3e00640 1 Processor -- start 2024-04-25T10:18:25.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.890+0000 7f03a3e00640 1 -- start start 2024-04-25T10:18:25.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.891+0000 7f03a3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03a4159ca0 0x7f03a415d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.892+0000 7f03a3e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a40cec30 con 0x7f03a415a080 2024-04-25T10:18:25.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.892+0000 7f03a3e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a40cef10 con 0x7f03a40b5470 2024-04-25T10:18:25.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.892+0000 7f03a3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40cf1f0 con 0x7f03a4159ca0 2024-04-25T10:18:25.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.892+0000 7f03a2000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f03a415a080 0x7f03a415de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42592/0 (socket says 172.21.15.119:42592) 2024-04-25T10:18:25.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.892+0000 7f03a2000640 1 -- 172.21.15.119:0/3475676734 learned_addr learned my addr 172.21.15.119:0/3475676734 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:25.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.892+0000 7f03a2a00640 1 --2- 172.21.15.119:0/3475676734 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03a4159ca0 0x7f03a415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.893+0000 7f03a2a00640 1 -- 172.21.15.119:0/3475676734 >> v1:172.21.15.132:6789/0 conn(0x7f03a40b5470 legacy=0x7f03a406d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.893+0000 7f03a2a00640 1 -- 172.21.15.119:0/3475676734 >> v1:172.21.15.119:6789/0 conn(0x7f03a415a080 legacy=0x7f03a415de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.893+0000 7f03a2a00640 1 -- 172.21.15.119:0/3475676734 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a406d420 con 0x7f03a4159ca0 2024-04-25T10:18:25.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.894+0000 7f03a2a00640 1 --2- 172.21.15.119:0/3475676734 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03a4159ca0 0x7f03a415d910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03940097d0 tx=0x7f039402c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d887f6562a72a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.900+0000 7f03a1600640 1 -- 172.21.15.119:0/3475676734 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394038070 con 0x7f03a4159ca0 2024-04-25T10:18:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.900+0000 7f03a1600640 1 -- 172.21.15.119:0/3475676734 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0394031710 con 0x7f03a4159ca0 2024-04-25T10:18:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.900+0000 7f03a1600640 1 -- 172.21.15.119:0/3475676734 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394031ae0 con 0x7f03a4159ca0 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.900+0000 7f03a3e00640 1 -- 172.21.15.119:0/3475676734 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03a4159ca0 msgr2=0x7f03a415d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.900+0000 7f03a3e00640 1 --2- 172.21.15.119:0/3475676734 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03a4159ca0 0x7f03a415d910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03940097d0 tx=0x7f039402c420 comp rx=0 tx=0).stop 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.900+0000 7f03a3e00640 1 -- 172.21.15.119:0/3475676734 shutdown_connections 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.900+0000 7f03a3e00640 1 --2- 172.21.15.119:0/3475676734 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03a4159ca0 0x7f03a415d910 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.901+0000 7f03a3e00640 1 -- 172.21.15.119:0/3475676734 >> 172.21.15.119:0/3475676734 conn(0x7f03a406b1d0 msgr2=0x7f03a40b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.901+0000 7f03a3e00640 1 -- 172.21.15.119:0/3475676734 shutdown_connections 2024-04-25T10:18:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.901+0000 7f03a3e00640 1 -- 172.21.15.119:0/3475676734 wait complete. 2024-04-25T10:18:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.902+0000 7f03a3e00640 1 Processor -- start 2024-04-25T10:18:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.902+0000 7f03a3e00640 1 -- start start 2024-04-25T10:18:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.903+0000 7f03a3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a40b5470 0x7f03a413f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.903+0000 7f03a3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03a4159ca0 0x7f03a40a9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.903+0000 7f03a3e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03a413bd00 0x7f03a415c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.903+0000 7f03a3e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a415e6e0 con 0x7f03a40b5470 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.903+0000 7f03a3e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a415e560 con 0x7f03a413bd00 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.903+0000 7f03a3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40b1a30 con 0x7f03a4159ca0 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.904+0000 7f03a2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a40b5470 0x7f03a413f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.904+0000 7f03a2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a40b5470 0x7f03a413f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32820/0 (socket says 172.21.15.119:32820) 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.904+0000 7f03a2a00640 1 -- 172.21.15.119:0/4141723206 learned_addr learned my addr 172.21.15.119:0/4141723206 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.904+0000 7f03a3400640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03a413bd00 0x7f03a415c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.904+0000 7f03a2000640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03a4159ca0 0x7f03a40a9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.904+0000 7f03a2a00640 1 -- 172.21.15.119:0/4141723206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03a4159ca0 msgr2=0x7f03a40a9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.904+0000 7f03a2a00640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03a4159ca0 0x7f03a40a9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.904+0000 7f03a2a00640 1 -- 172.21.15.119:0/4141723206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03a413bd00 msgr2=0x7f03a415c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.904+0000 7f03a2a00640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03a413bd00 0x7f03a415c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.904+0000 7f03a2a00640 1 -- 172.21.15.119:0/4141723206 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a40a9dc0 con 0x7f03a40b5470 2024-04-25T10:18:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.904+0000 7f03a2000640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03a4159ca0 0x7f03a40a9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.904+0000 7f03a3400640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03a413bd00 0x7f03a415c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.904+0000 7f03a2a00640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a40b5470 0x7f03a413f210 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f03940098f0 tx=0x7f03940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.905+0000 7f039b400640 1 -- 172.21.15.119:0/4141723206 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394042020 con 0x7f03a40b5470 2024-04-25T10:18:25.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.905+0000 7f039b400640 1 -- 172.21.15.119:0/4141723206 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0394033a40 con 0x7f03a40b5470 2024-04-25T10:18:25.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.905+0000 7f039b400640 1 -- 172.21.15.119:0/4141723206 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394038040 con 0x7f03a40b5470 2024-04-25T10:18:25.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.905+0000 7f03a3e00640 1 -- 172.21.15.119:0/4141723206 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a40aa0a0 con 0x7f03a40b5470 2024-04-25T10:18:25.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.905+0000 7f03a3e00640 1 -- 172.21.15.119:0/4141723206 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a413c1b0 con 0x7f03a40b5470 2024-04-25T10:18:25.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.906+0000 7f03a3e00640 1 -- 172.21.15.119:0/4141723206 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a40749b0 con 0x7f03a40b5470 2024-04-25T10:18:25.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.907+0000 7f039b400640 1 -- 172.21.15.119:0/4141723206 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0394045020 con 0x7f03a40b5470 2024-04-25T10:18:25.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.908+0000 7f039b400640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f038406b500 0x7f038406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.908+0000 7f039b400640 1 -- 172.21.15.119:0/4141723206 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f03940ace40 con 0x7f03a40b5470 2024-04-25T10:18:25.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.908+0000 7f03a2000640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f038406b500 0x7f038406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.909+0000 7f03a2000640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f038406b500 0x7f038406d9c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f03a40aaaa0 tx=0x7f0390009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:25.912+0000 7f039b400640 1 -- 172.21.15.119:0/4141723206 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f039404c030 con 0x7f03a40b5470 2024-04-25T10:18:26.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.137+0000 7f03a3e00640 1 -- 172.21.15.119:0/4141723206 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03a40b2d20 con 0x7f03a40b5470 2024-04-25T10:18:26.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.139+0000 7f039b400640 1 -- 172.21.15.119:0/4141723206 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4957 (secure 0 0 0) 0x7f039407bbc0 con 0x7f03a40b5470 2024-04-25T10:18:26.142 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:26.142 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3394261143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3394261143},{"type":"v1","addr":"172.21.15.132:6834","nonce":3394261143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2502874931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2502874931},{"type":"v1","addr":"172.21.15.119:6835","nonce":2502874931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":8694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2305428003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2305428003},{"type":"v1","addr":"172.21.15.132:6835","nonce":2305428003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:37.617787+0000","modified":"2024-04-25T10:18:13.835772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9117},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9117":{"gid":9117,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/2618740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2618740513},{"type":"v1","addr":"172.21.15.119:6837","nonce":2618740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:18:26.143 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 112 2024-04-25T10:18:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.141+0000 7f03a3e00640 1 -- 172.21.15.119:0/4141723206 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f038406b500 msgr2=0x7f038406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.141+0000 7f03a3e00640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f038406b500 0x7f038406d9c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f03a40aaaa0 tx=0x7f0390009040 comp rx=0 tx=0).stop 2024-04-25T10:18:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.141+0000 7f03a3e00640 1 -- 172.21.15.119:0/4141723206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a40b5470 msgr2=0x7f03a413f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.141+0000 7f03a3e00640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a40b5470 0x7f03a413f210 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f03940098f0 tx=0x7f03940046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.141+0000 7f03a3e00640 1 -- 172.21.15.119:0/4141723206 shutdown_connections 2024-04-25T10:18:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.141+0000 7f03a3e00640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f038406b500 0x7f038406d9c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.141+0000 7f03a3e00640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03a413bd00 0x7f03a415c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.141+0000 7f03a3e00640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03a4159ca0 0x7f03a40a9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.141+0000 7f03a3e00640 1 --2- 172.21.15.119:0/4141723206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a40b5470 0x7f03a413f210 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.141+0000 7f03a3e00640 1 -- 172.21.15.119:0/4141723206 >> 172.21.15.119:0/4141723206 conn(0x7f03a406b1d0 msgr2=0x7f03a40b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.141+0000 7f03a3e00640 1 -- 172.21.15.119:0/4141723206 shutdown_connections 2024-04-25T10:18:26.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.142+0000 7f03a3e00640 1 -- 172.21.15.119:0/4141723206 wait complete. 2024-04-25T10:18:26.163 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:26.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.355+0000 7f838e200640 1 Processor -- start 2024-04-25T10:18:26.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.355+0000 7f838e200640 1 -- start start 2024-04-25T10:18:26.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.356+0000 7f838e200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f838815ba10 0x7f8388099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.356+0000 7f838e200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8388099f80 0x7f838806bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.356+0000 7f838e200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f838809a3a0 con 0x7f838806c1f0 2024-04-25T10:18:26.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.356+0000 7f838e200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83880cbe50 con 0x7f8388099f80 2024-04-25T10:18:26.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.356+0000 7f838e200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83880cfe80 con 0x7f838815ba10 2024-04-25T10:18:26.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.357+0000 7f837ea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8388099f80 0x7f838806bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.357+0000 7f8387400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f838815ba10 0x7f8388099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.357+0000 7f8387e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f838806c1f0 0x7f83880cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.119:47436/0 (socket says 172.21.15.119:47436) 2024-04-25T10:18:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.357+0000 7f8387e00640 1 -- 172.21.15.119:0/3567255446 learned_addr learned my addr 172.21.15.119:0/3567255446 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.358+0000 7f837ea00640 1 -- 172.21.15.119:0/3567255446 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f838815ba10 msgr2=0x7f8388099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.358+0000 7f837ea00640 1 --2- 172.21.15.119:0/3567255446 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f838815ba10 0x7f8388099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.358+0000 7f837ea00640 1 -- 172.21.15.119:0/3567255446 >> v1:172.21.15.132:6789/0 conn(0x7f838806c1f0 legacy=0x7f83880cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.358+0000 7f837ea00640 1 -- 172.21.15.119:0/3567255446 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838806c680 con 0x7f8388099f80 2024-04-25T10:18:26.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.359+0000 7f837ea00640 1 --2- 172.21.15.119:0/3567255446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8388099f80 0x7f838806bcb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f83700097a0 tx=0x7f837002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0206c9940f556d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.360+0000 7f8386a00640 1 -- 172.21.15.119:0/3567255446 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370037070 con 0x7f8388099f80 2024-04-25T10:18:26.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.360+0000 7f8386a00640 1 -- 172.21.15.119:0/3567255446 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8370032440 con 0x7f8388099f80 2024-04-25T10:18:26.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.360+0000 7f8386a00640 1 -- 172.21.15.119:0/3567255446 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370032810 con 0x7f8388099f80 2024-04-25T10:18:26.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.361+0000 7f838e200640 1 -- 172.21.15.119:0/3567255446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8388099f80 msgr2=0x7f838806bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.361+0000 7f838e200640 1 --2- 172.21.15.119:0/3567255446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8388099f80 0x7f838806bcb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f83700097a0 tx=0x7f837002a180 comp rx=0 tx=0).stop 2024-04-25T10:18:26.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.361+0000 7f838e200640 1 -- 172.21.15.119:0/3567255446 shutdown_connections 2024-04-25T10:18:26.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.361+0000 7f838e200640 1 --2- 172.21.15.119:0/3567255446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8388099f80 0x7f838806bcb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.361+0000 7f838e200640 1 --2- 172.21.15.119:0/3567255446 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f838815ba10 0x7f8388099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.361+0000 7f838e200640 1 -- 172.21.15.119:0/3567255446 >> 172.21.15.119:0/3567255446 conn(0x7f83880b32f0 msgr2=0x7f83880b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.361+0000 7f838e200640 1 -- 172.21.15.119:0/3567255446 shutdown_connections 2024-04-25T10:18:26.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.361+0000 7f838e200640 1 -- 172.21.15.119:0/3567255446 wait complete. 2024-04-25T10:18:26.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.362+0000 7f838e200640 1 Processor -- start 2024-04-25T10:18:26.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.363+0000 7f838e200640 1 -- start start 2024-04-25T10:18:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.363+0000 7f838e200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f838806c1f0 0x7f83880b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.363+0000 7f838e200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8388099f80 0x7f83880c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.363+0000 7f838e200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f838815ba10 0x7f83880af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.364+0000 7f838e200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838806cec0 con 0x7f8388099f80 2024-04-25T10:18:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.364+0000 7f838e200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838806cbe0 con 0x7f838806c1f0 2024-04-25T10:18:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.364+0000 7f838e200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83880cd930 con 0x7f838815ba10 2024-04-25T10:18:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.364+0000 7f837ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8388099f80 0x7f83880c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.364+0000 7f837ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8388099f80 0x7f83880c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32836/0 (socket says 172.21.15.119:32836) 2024-04-25T10:18:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.364+0000 7f837ea00640 1 -- 172.21.15.119:0/2824281185 learned_addr learned my addr 172.21.15.119:0/2824281185 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.364+0000 7f8387400640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f838806c1f0 0x7f83880b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.364+0000 7f8387e00640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f838815ba10 0x7f83880af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.364+0000 7f837ea00640 1 -- 172.21.15.119:0/2824281185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f838815ba10 msgr2=0x7f83880af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.364+0000 7f837ea00640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f838815ba10 0x7f83880af500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.364+0000 7f837ea00640 1 -- 172.21.15.119:0/2824281185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f838806c1f0 msgr2=0x7f83880b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.364+0000 7f837ea00640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f838806c1f0 0x7f83880b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.364+0000 7f837ea00640 1 -- 172.21.15.119:0/2824281185 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83880c6fc0 con 0x7f8388099f80 2024-04-25T10:18:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.364+0000 7f8387400640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f838806c1f0 0x7f83880b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.365+0000 7f837ea00640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8388099f80 0x7f83880c65e0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f83700097a0 tx=0x7f83700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.365+0000 7f837fe00640 1 -- 172.21.15.119:0/2824281185 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370041070 con 0x7f8388099f80 2024-04-25T10:18:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.365+0000 7f837fe00640 1 -- 172.21.15.119:0/2824281185 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837003c870 con 0x7f8388099f80 2024-04-25T10:18:26.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.365+0000 7f838e200640 1 -- 172.21.15.119:0/2824281185 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83880c72a0 con 0x7f8388099f80 2024-04-25T10:18:26.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.365+0000 7f837fe00640 1 -- 172.21.15.119:0/2824281185 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370037040 con 0x7f8388099f80 2024-04-25T10:18:26.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.365+0000 7f838e200640 1 -- 172.21.15.119:0/2824281185 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83880c7860 con 0x7f8388099f80 2024-04-25T10:18:26.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.366+0000 7f838e200640 1 -- 172.21.15.119:0/2824281185 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83880749b0 con 0x7f8388099f80 2024-04-25T10:18:26.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.367+0000 7f837fe00640 1 -- 172.21.15.119:0/2824281185 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8370044070 con 0x7f8388099f80 2024-04-25T10:18:26.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.368+0000 7f837fe00640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f836406b4d0 0x7f836406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.369+0000 7f837fe00640 1 -- 172.21.15.119:0/2824281185 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f83700ab100 con 0x7f8388099f80 2024-04-25T10:18:26.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.369+0000 7f8387400640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f836406b4d0 0x7f836406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.369+0000 7f8387400640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f836406b4d0 0x7f836406d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8378009a20 tx=0x7f8378008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.373+0000 7f837fe00640 1 -- 172.21.15.119:0/2824281185 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f838806c680 con 0x7f8388099f80 2024-04-25T10:18:26.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.610+0000 7f838e200640 1 -- 172.21.15.119:0/2824281185 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f838809a570 con 0x7f8388099f80 2024-04-25T10:18:26.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.611+0000 7f837fe00640 1 -- 172.21.15.119:0/2824281185 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4957 (secure 0 0 0) 0x7f8370079f10 con 0x7f8388099f80 2024-04-25T10:18:26.615 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:26.615 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3394261143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3394261143},{"type":"v1","addr":"172.21.15.132:6834","nonce":3394261143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2502874931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2502874931},{"type":"v1","addr":"172.21.15.119:6835","nonce":2502874931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":8694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2305428003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2305428003},{"type":"v1","addr":"172.21.15.132:6835","nonce":2305428003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:37.617787+0000","modified":"2024-04-25T10:18:13.835772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9117},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9117":{"gid":9117,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6837/2618740513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2618740513},{"type":"v1","addr":"172.21.15.119:6837","nonce":2618740513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:18:26.615 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 112 2024-04-25T10:18:26.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.614+0000 7f838e200640 1 -- 172.21.15.119:0/2824281185 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f836406b4d0 msgr2=0x7f836406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.614+0000 7f838e200640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f836406b4d0 0x7f836406d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8378009a20 tx=0x7f8378008040 comp rx=0 tx=0).stop 2024-04-25T10:18:26.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.614+0000 7f838e200640 1 -- 172.21.15.119:0/2824281185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8388099f80 msgr2=0x7f83880c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.614+0000 7f838e200640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8388099f80 0x7f83880c65e0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f83700097a0 tx=0x7f83700046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.614+0000 7f838e200640 1 -- 172.21.15.119:0/2824281185 shutdown_connections 2024-04-25T10:18:26.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.614+0000 7f838e200640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f836406b4d0 0x7f836406d990 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.614+0000 7f838e200640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f838815ba10 0x7f83880af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.614+0000 7f838e200640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8388099f80 0x7f83880c65e0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.614+0000 7f838e200640 1 --2- 172.21.15.119:0/2824281185 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f838806c1f0 0x7f83880b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.614+0000 7f838e200640 1 -- 172.21.15.119:0/2824281185 >> 172.21.15.119:0/2824281185 conn(0x7f83880b32f0 msgr2=0x7f83880708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.615+0000 7f838e200640 1 -- 172.21.15.119:0/2824281185 shutdown_connections 2024-04-25T10:18:26.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.615+0000 7f838e200640 1 -- 172.21.15.119:0/2824281185 wait complete. 2024-04-25T10:18:26.636 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:26.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.826+0000 7ff36d600640 1 Processor -- start 2024-04-25T10:18:26.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.826+0000 7ff36d600640 1 -- start start 2024-04-25T10:18:26.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.828+0000 7ff36d600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff3680aecf0 0x7ff3680cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.828+0000 7ff36d600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36815ed70 0x7ff3680cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.828+0000 7ff36d600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3680712c0 con 0x7ff36815d980 2024-04-25T10:18:26.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.828+0000 7ff36d600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff368071ed0 con 0x7ff3680aecf0 2024-04-25T10:18:26.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.828+0000 7ff36d600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3680715a0 con 0x7ff36815ed70 2024-04-25T10:18:26.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.828+0000 7ff366a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff36815d980 0x7ff3680ae5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42606/0 (socket says 172.21.15.119:42606) 2024-04-25T10:18:26.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.828+0000 7ff366a00640 1 -- 172.21.15.119:0/2435591273 learned_addr learned my addr 172.21.15.119:0/2435591273 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:26.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.829+0000 7ff35e000640 1 --2- 172.21.15.119:0/2435591273 >> v2:172.21.15.119:3300/0 conn(0x7ff3680aecf0 0x7ff3680cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.829+0000 7ff367400640 1 --2- 172.21.15.119:0/2435591273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36815ed70 0x7ff3680cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.829+0000 7ff366000640 1 -- 172.21.15.119:0/2435591273 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558383610 0 0) 0x7ff3680712c0 con 0x7ff36815d980 2024-04-25T10:18:26.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.829+0000 7ff366000640 1 -- 172.21.15.119:0/2435591273 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff34c003520 con 0x7ff36815d980 2024-04-25T10:18:26.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.829+0000 7ff35e000640 1 -- 172.21.15.119:0/2435591273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36815ed70 msgr2=0x7ff3680cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:26.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.829+0000 7ff35e000640 1 --2- 172.21.15.119:0/2435591273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36815ed70 0x7ff3680cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.829+0000 7ff35e000640 1 -- 172.21.15.119:0/2435591273 >> v1:172.21.15.119:6789/0 conn(0x7ff36815d980 legacy=0x7ff3680ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.829+0000 7ff35e000640 1 -- 172.21.15.119:0/2435591273 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff368071880 con 0x7ff3680aecf0 2024-04-25T10:18:26.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.830+0000 7ff35e000640 1 --2- 172.21.15.119:0/2435591273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3680aecf0 0x7ff3680cf280 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7ff3500097a0 tx=0x7ff35002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926c31c130be8686 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.830+0000 7ff366000640 1 -- 172.21.15.119:0/2435591273 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff350037070 con 0x7ff3680aecf0 2024-04-25T10:18:26.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.830+0000 7ff366000640 1 -- 172.21.15.119:0/2435591273 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35002ae50 con 0x7ff3680aecf0 2024-04-25T10:18:26.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.830+0000 7ff366000640 1 -- 172.21.15.119:0/2435591273 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff350032660 con 0x7ff3680aecf0 2024-04-25T10:18:26.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.831+0000 7ff36d600640 1 -- 172.21.15.119:0/2435591273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3680aecf0 msgr2=0x7ff3680cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.831+0000 7ff36d600640 1 --2- 172.21.15.119:0/2435591273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3680aecf0 0x7ff3680cf280 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7ff3500097a0 tx=0x7ff35002a100 comp rx=0 tx=0).stop 2024-04-25T10:18:26.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.831+0000 7ff36d600640 1 -- 172.21.15.119:0/2435591273 shutdown_connections 2024-04-25T10:18:26.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.831+0000 7ff36d600640 1 --2- 172.21.15.119:0/2435591273 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36815ed70 0x7ff3680cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.831+0000 7ff36d600640 1 --2- 172.21.15.119:0/2435591273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3680aecf0 0x7ff3680cf280 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.831+0000 7ff36d600640 1 -- 172.21.15.119:0/2435591273 >> 172.21.15.119:0/2435591273 conn(0x7ff36806aba0 msgr2=0x7ff36806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.831+0000 7ff36d600640 1 -- 172.21.15.119:0/2435591273 shutdown_connections 2024-04-25T10:18:26.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.831+0000 7ff36d600640 1 -- 172.21.15.119:0/2435591273 wait complete. 2024-04-25T10:18:26.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.833+0000 7ff36d600640 1 Processor -- start 2024-04-25T10:18:26.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.833+0000 7ff36d600640 1 -- start start 2024-04-25T10:18:26.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.834+0000 7ff36d600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36815d980 0x7ff36814e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.834+0000 7ff36d600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff368156430 0x7ff368160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.834+0000 7ff36d600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3681568b0 0x7ff368160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.834+0000 7ff36d600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3680ad0f0 con 0x7ff3681568b0 2024-04-25T10:18:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.834+0000 7ff36d600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3680acf70 con 0x7ff368156430 2024-04-25T10:18:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.834+0000 7ff36d600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3680ad270 con 0x7ff36815d980 2024-04-25T10:18:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.834+0000 7ff367400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3681568b0 0x7ff368160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.834+0000 7ff367400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3681568b0 0x7ff368160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32852/0 (socket says 172.21.15.119:32852) 2024-04-25T10:18:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.834+0000 7ff367400640 1 -- 172.21.15.119:0/2178671370 learned_addr learned my addr 172.21.15.119:0/2178671370 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.835+0000 7ff35e000640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff368156430 0x7ff368160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.835+0000 7ff366a00640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36815d980 0x7ff36814e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.835+0000 7ff367400640 1 -- 172.21.15.119:0/2178671370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36815d980 msgr2=0x7ff36814e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.835+0000 7ff367400640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36815d980 0x7ff36814e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.835+0000 7ff367400640 1 -- 172.21.15.119:0/2178671370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff368156430 msgr2=0x7ff368160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.835+0000 7ff367400640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff368156430 0x7ff368160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.835+0000 7ff367400640 1 -- 172.21.15.119:0/2178671370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3680cdc70 con 0x7ff3681568b0 2024-04-25T10:18:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.835+0000 7ff35e000640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff368156430 0x7ff368160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.835+0000 7ff367400640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3681568b0 0x7ff368160790 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7ff360007d80 tx=0x7ff36000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.835+0000 7ff35f400640 1 -- 172.21.15.119:0/2178671370 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360014420 con 0x7ff3681568b0 2024-04-25T10:18:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.835+0000 7ff35f400640 1 -- 172.21.15.119:0/2178671370 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff360019070 con 0x7ff3681568b0 2024-04-25T10:18:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.835+0000 7ff35f400640 1 -- 172.21.15.119:0/2178671370 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360014800 con 0x7ff3681568b0 2024-04-25T10:18:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.835+0000 7ff36d600640 1 -- 172.21.15.119:0/2178671370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3680cdf50 con 0x7ff3681568b0 2024-04-25T10:18:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.836+0000 7ff36d600640 1 -- 172.21.15.119:0/2178671370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3680ce510 con 0x7ff3681568b0 2024-04-25T10:18:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.837+0000 7ff36d600640 1 -- 172.21.15.119:0/2178671370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3680749b0 con 0x7ff3681568b0 2024-04-25T10:18:26.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.838+0000 7ff35f400640 1 -- 172.21.15.119:0/2178671370 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3600149a0 con 0x7ff3681568b0 2024-04-25T10:18:26.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.839+0000 7ff35f400640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff34406b500 0x7ff34406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.839+0000 7ff35f400640 1 -- 172.21.15.119:0/2178671370 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff36008aa50 con 0x7ff3681568b0 2024-04-25T10:18:26.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.840+0000 7ff366a00640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff34406b500 0x7ff34406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.840+0000 7ff366a00640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff34406b500 0x7ff34406d9c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff358000f60 tx=0x7ff358009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:26.844+0000 7ff35f400640 1 -- 172.21.15.119:0/2178671370 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff360059870 con 0x7ff3681568b0 2024-04-25T10:18:27.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.044+0000 7ff36d600640 1 -- 172.21.15.119:0/2178671370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff36806d4c0 con 0x7ff3681568b0 2024-04-25T10:18:27.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.046+0000 7ff35f400640 1 -- 172.21.15.119:0/2178671370 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+17079 (secure 0 0 0) 0x7ff360059170 con 0x7ff3681568b0 2024-04-25T10:18:27.050 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:27.050 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":112,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:18:09.666428+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:35.422922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:36.634584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.25","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/2864575636":"2024-04-26T10:18:09.666363+0000","172.21.15.119:6836/2864575636":"2024-04-26T10:18:09.666363+0000","172.21.15.119:6837/1849584205":"2024-04-26T10:17:56.438859+0000","172.21.15.119:6836/1849584205":"2024-04-26T10:17:56.438859+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:27.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.049+0000 7ff36d600640 1 -- 172.21.15.119:0/2178671370 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff34406b500 msgr2=0x7ff34406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.049+0000 7ff36d600640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff34406b500 0x7ff34406d9c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff358000f60 tx=0x7ff358009040 comp rx=0 tx=0).stop 2024-04-25T10:18:27.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.049+0000 7ff36d600640 1 -- 172.21.15.119:0/2178671370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3681568b0 msgr2=0x7ff368160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.049+0000 7ff36d600640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3681568b0 0x7ff368160790 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7ff360007d80 tx=0x7ff36000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.049+0000 7ff36d600640 1 -- 172.21.15.119:0/2178671370 shutdown_connections 2024-04-25T10:18:27.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.049+0000 7ff36d600640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff34406b500 0x7ff34406d9c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.049+0000 7ff36d600640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3681568b0 0x7ff368160790 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.049+0000 7ff36d600640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff368156430 0x7ff368160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.049+0000 7ff36d600640 1 --2- 172.21.15.119:0/2178671370 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36815d980 0x7ff36814e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.049+0000 7ff36d600640 1 -- 172.21.15.119:0/2178671370 >> 172.21.15.119:0/2178671370 conn(0x7ff36806aba0 msgr2=0x7ff3680b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:27.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.049+0000 7ff36d600640 1 -- 172.21.15.119:0/2178671370 shutdown_connections 2024-04-25T10:18:27.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.049+0000 7ff36d600640 1 -- 172.21.15.119:0/2178671370 wait complete. 2024-04-25T10:18:27.071 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:18:27.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.268+0000 7ff0c0400640 1 Processor -- start 2024-04-25T10:18:27.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.268+0000 7ff0c0400640 1 -- start start 2024-04-25T10:18:27.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.270+0000 7ff0c0400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0b815dbe0 0x7ff0b8071c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.270+0000 7ff0c0400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b80ace70 con 0x7ff0b80cfad0 2024-04-25T10:18:27.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.270+0000 7ff0c0400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b8072150 con 0x7ff0b80aedc0 2024-04-25T10:18:27.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.270+0000 7ff0c0400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b80ad110 con 0x7ff0b815dbe0 2024-04-25T10:18:27.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.270+0000 7ff0bda00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0b815dbe0 0x7ff0b8071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.270+0000 7ff0bda00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0b815dbe0 0x7ff0b8071c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32858/0 (socket says 172.21.15.119:32858) 2024-04-25T10:18:27.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.270+0000 7ff0bda00640 1 -- 172.21.15.119:0/4260223025 learned_addr learned my addr 172.21.15.119:0/4260223025 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:27.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.270+0000 7ff0be400640 1 --1- 172.21.15.119:0/4260223025 >> v1:172.21.15.119:6789/0 conn(0x7ff0b80cfad0 0x7ff0b806d9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42622/0 (socket says 172.21.15.119:42622) 2024-04-25T10:18:27.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.271+0000 7ff0bda00640 1 -- 172.21.15.119:0/4260223025 >> v1:172.21.15.132:6789/0 conn(0x7ff0b80aedc0 legacy=0x7ff0b80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:27.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.271+0000 7ff0bda00640 1 -- 172.21.15.119:0/4260223025 >> v1:172.21.15.119:6789/0 conn(0x7ff0b80cfad0 legacy=0x7ff0b806d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.271+0000 7ff0bda00640 1 -- 172.21.15.119:0/4260223025 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b80ad3f0 con 0x7ff0b815dbe0 2024-04-25T10:18:27.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.271+0000 7ff0b7e00640 1 -- 172.21.15.119:0/4260223025 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095432734 0 0) 0x7ff0b80ace70 con 0x7ff0b80cfad0 2024-04-25T10:18:27.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.272+0000 7ff0bda00640 1 --2- 172.21.15.119:0/4260223025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b815dbe0 0x7ff0b8071c10 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009870 tx=0x7ff0ac02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82772f24b3cd038 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.272+0000 7ff0b7e00640 1 -- 172.21.15.119:0/4260223025 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac037070 con 0x7ff0b815dbe0 2024-04-25T10:18:27.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.272+0000 7ff0b7e00640 1 -- 172.21.15.119:0/4260223025 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ac032440 con 0x7ff0b815dbe0 2024-04-25T10:18:27.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.272+0000 7ff0b7e00640 1 -- 172.21.15.119:0/4260223025 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac032810 con 0x7ff0b815dbe0 2024-04-25T10:18:27.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.273+0000 7ff0c0400640 1 -- 172.21.15.119:0/4260223025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b815dbe0 msgr2=0x7ff0b8071c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.273+0000 7ff0c0400640 1 --2- 172.21.15.119:0/4260223025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b815dbe0 0x7ff0b8071c10 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009870 tx=0x7ff0ac02a2a0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.273+0000 7ff0c0400640 1 -- 172.21.15.119:0/4260223025 shutdown_connections 2024-04-25T10:18:27.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.273+0000 7ff0c0400640 1 --2- 172.21.15.119:0/4260223025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b815dbe0 0x7ff0b8071c10 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.273+0000 7ff0c0400640 1 -- 172.21.15.119:0/4260223025 >> 172.21.15.119:0/4260223025 conn(0x7ff0b806aba0 msgr2=0x7ff0b806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:27.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.273+0000 7ff0c0400640 1 -- 172.21.15.119:0/4260223025 shutdown_connections 2024-04-25T10:18:27.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.273+0000 7ff0c0400640 1 -- 172.21.15.119:0/4260223025 wait complete. 2024-04-25T10:18:27.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.274+0000 7ff0c0400640 1 Processor -- start 2024-04-25T10:18:27.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.275+0000 7ff0c0400640 1 -- start start 2024-04-25T10:18:27.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.275+0000 7ff0c0400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b80aedc0 0x7ff0b81f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.275+0000 7ff0c0400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b80cfad0 0x7ff0b815ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.275+0000 7ff0c0400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b815dbe0 0x7ff0b815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.276+0000 7ff0c0400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80adc90 con 0x7ff0b815dbe0 2024-04-25T10:18:27.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.276+0000 7ff0c0400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80ad9b0 con 0x7ff0b80aedc0 2024-04-25T10:18:27.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.276+0000 7ff0c0400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b815dfc0 con 0x7ff0b80cfad0 2024-04-25T10:18:27.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.276+0000 7ff0be400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b815dbe0 0x7ff0b815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.276+0000 7ff0be400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b815dbe0 0x7ff0b815f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32862/0 (socket says 172.21.15.119:32862) 2024-04-25T10:18:27.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.276+0000 7ff0be400640 1 -- 172.21.15.119:0/3106395813 learned_addr learned my addr 172.21.15.119:0/3106395813 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:27.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.276+0000 7ff0bda00640 1 --2- 172.21.15.119:0/3106395813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b80aedc0 0x7ff0b81f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.276+0000 7ff0be400640 1 -- 172.21.15.119:0/3106395813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b80cfad0 msgr2=0x7ff0b815ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.276+0000 7ff0be400640 1 --2- 172.21.15.119:0/3106395813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b80cfad0 0x7ff0b815ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.276+0000 7ff0be400640 1 -- 172.21.15.119:0/3106395813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b80aedc0 msgr2=0x7ff0b81f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.276+0000 7ff0be400640 1 --2- 172.21.15.119:0/3106395813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b80aedc0 0x7ff0b81f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.276+0000 7ff0be400640 1 -- 172.21.15.119:0/3106395813 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b814bb50 con 0x7ff0b815dbe0 2024-04-25T10:18:27.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.276+0000 7ff0bda00640 1 --2- 172.21.15.119:0/3106395813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b80aedc0 0x7ff0b81f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:27.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.277+0000 7ff0be400640 1 --2- 172.21.15.119:0/3106395813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b815dbe0 0x7ff0b815f4b0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000c7e0 tx=0x7ff0b000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.277+0000 7ff0b5600640 1 -- 172.21.15.119:0/3106395813 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0018070 con 0x7ff0b815dbe0 2024-04-25T10:18:27.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.277+0000 7ff0b5600640 1 -- 172.21.15.119:0/3106395813 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b000a640 con 0x7ff0b815dbe0 2024-04-25T10:18:27.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.277+0000 7ff0c0400640 1 -- 172.21.15.119:0/3106395813 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b81f6bc0 con 0x7ff0b815dbe0 2024-04-25T10:18:27.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.277+0000 7ff0b5600640 1 -- 172.21.15.119:0/3106395813 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b000aa30 con 0x7ff0b815dbe0 2024-04-25T10:18:27.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.277+0000 7ff0c0400640 1 -- 172.21.15.119:0/3106395813 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b81f7180 con 0x7ff0b815dbe0 2024-04-25T10:18:27.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.278+0000 7ff0c0400640 1 -- 172.21.15.119:0/3106395813 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b80750b0 con 0x7ff0b815dbe0 2024-04-25T10:18:27.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.279+0000 7ff0b5600640 1 -- 172.21.15.119:0/3106395813 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0b001b070 con 0x7ff0b815dbe0 2024-04-25T10:18:27.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.280+0000 7ff0b5600640 1 --2- 172.21.15.119:0/3106395813 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff09c06b4d0 0x7ff09c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.280+0000 7ff0b5600640 1 -- 172.21.15.119:0/3106395813 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff0b0089950 con 0x7ff0b815dbe0 2024-04-25T10:18:27.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.280+0000 7ff0bda00640 1 --2- 172.21.15.119:0/3106395813 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff09c06b4d0 0x7ff09c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.281+0000 7ff0bda00640 1 --2- 172.21.15.119:0/3106395813 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff09c06b4d0 0x7ff09c06d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac034000 tx=0x7ff0ac02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.284+0000 7ff0b5600640 1 -- 172.21.15.119:0/3106395813 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0b00586d0 con 0x7ff0b815dbe0 2024-04-25T10:18:27.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.517+0000 7ff0c0400640 1 -- 172.21.15.119:0/3106395813 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff0b80b2680 con 0x7ff0b815dbe0 2024-04-25T10:18:27.521 INFO:tasks.ceph.mon.a.smithi119.stderr:2024-04-25T10:18:27.518+0000 7f8c64600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:18:27.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.521+0000 7ff0b5600640 1 -- 172.21.15.119:0/3106395813 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff0b0057fd0 con 0x7ff0b815dbe0 2024-04-25T10:18:27.525 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:18:27.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.525+0000 7ff0c0400640 1 -- 172.21.15.119:0/3106395813 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff09c06b4d0 msgr2=0x7ff09c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.525+0000 7ff0c0400640 1 --2- 172.21.15.119:0/3106395813 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff09c06b4d0 0x7ff09c06d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac034000 tx=0x7ff0ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:18:27.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.525+0000 7ff0c0400640 1 -- 172.21.15.119:0/3106395813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b815dbe0 msgr2=0x7ff0b815f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.525+0000 7ff0c0400640 1 --2- 172.21.15.119:0/3106395813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b815dbe0 0x7ff0b815f4b0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000c7e0 tx=0x7ff0b000cca0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.525+0000 7ff0c0400640 1 -- 172.21.15.119:0/3106395813 shutdown_connections 2024-04-25T10:18:27.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.525+0000 7ff0c0400640 1 --2- 172.21.15.119:0/3106395813 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ff09c06b4d0 0x7ff09c06d990 secure :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac034000 tx=0x7ff0ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:18:27.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.525+0000 7ff0c0400640 1 --2- 172.21.15.119:0/3106395813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b815dbe0 0x7ff0b815f4b0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.525+0000 7ff0c0400640 1 --2- 172.21.15.119:0/3106395813 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b80cfad0 0x7ff0b815ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.525+0000 7ff0c0400640 1 --2- 172.21.15.119:0/3106395813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b80aedc0 0x7ff0b81f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.525+0000 7ff0c0400640 1 -- 172.21.15.119:0/3106395813 >> 172.21.15.119:0/3106395813 conn(0x7ff0b806aba0 msgr2=0x7ff0b815e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:27.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.525+0000 7ff0c0400640 1 -- 172.21.15.119:0/3106395813 shutdown_connections 2024-04-25T10:18:27.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.526+0000 7ff0c0400640 1 -- 172.21.15.119:0/3106395813 wait complete. 2024-04-25T10:18:27.548 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:18:27.597 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2024-04-25T10:18:27.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.727+0000 7fe2e5800640 1 Processor -- start 2024-04-25T10:18:27.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.727+0000 7fe2e5800640 1 -- start start 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.729+0000 7fe2e5800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe2e015a260 0x7fe2e0071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.729+0000 7fe2e5800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2e0071760 0x7fe2e006e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.729+0000 7fe2e5800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e015dc30 con 0x7fe2e006ea30 2024-04-25T10:18:27.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.729+0000 7fe2e5800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e0071bc0 con 0x7fe2e015a260 2024-04-25T10:18:27.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.729+0000 7fe2e5800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e015df10 con 0x7fe2e0071760 2024-04-25T10:18:27.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.729+0000 7fe2df400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe2e006ea30 0x7fe2e006d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42632/0 (socket says 172.21.15.119:42632) 2024-04-25T10:18:27.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.729+0000 7fe2df400640 1 -- 172.21.15.119:0/170506678 learned_addr learned my addr 172.21.15.119:0/170506678 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:27.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.729+0000 7fe2dea00640 1 --2- 172.21.15.119:0/170506678 >> v2:172.21.15.132:3300/0 conn(0x7fe2e015a260 0x7fe2e0071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.729+0000 7fe2d6000640 1 --2- 172.21.15.119:0/170506678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2e0071760 0x7fe2e006e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.729+0000 7fe2de000640 1 -- 172.21.15.119:0/170506678 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185682198 0 0) 0x7fe2e015dc30 con 0x7fe2e006ea30 2024-04-25T10:18:27.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.729+0000 7fe2de000640 1 -- 172.21.15.119:0/170506678 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2c4003520 con 0x7fe2e006ea30 2024-04-25T10:18:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2de000640 1 -- 172.21.15.119:0/170506678 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3449990619 0 0) 0x7fe2c4003520 con 0x7fe2e006ea30 2024-04-25T10:18:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2de000640 1 -- 172.21.15.119:0/170506678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2e0071760 msgr2=0x7fe2e006e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2de000640 1 --2- 172.21.15.119:0/170506678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2e0071760 0x7fe2e006e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2de000640 1 -- 172.21.15.119:0/170506678 >> v2:172.21.15.132:3300/0 conn(0x7fe2e015a260 msgr2=0x7fe2e0071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2de000640 1 --2- 172.21.15.119:0/170506678 >> v2:172.21.15.132:3300/0 conn(0x7fe2e015a260 0x7fe2e0071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2de000640 1 -- 172.21.15.119:0/170506678 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e015e1f0 con 0x7fe2e006ea30 2024-04-25T10:18:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2de000640 1 -- 172.21.15.119:0/170506678 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fe2d80019f0 con 0x7fe2e006ea30 2024-04-25T10:18:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2de000640 1 -- 172.21.15.119:0/170506678 >> v1:172.21.15.119:6789/0 conn(0x7fe2e006ea30 legacy=0x7fe2e006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2de000640 1 --2- 172.21.15.119:0/170506678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2e0071760 0x7fe2c40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2de000640 1 --2- 172.21.15.119:0/170506678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c4003f30 0x7fe2c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2de000640 1 --2- 172.21.15.119:0/170506678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2c4004d40 0x7fe2c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2de000640 1 -- 172.21.15.119:0/170506678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e015df10 con 0x7fe2c4003f30 2024-04-25T10:18:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2de000640 1 -- 172.21.15.119:0/170506678 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c400b9c0 con 0x7fe2c4004d40 2024-04-25T10:18:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2de000640 1 -- 172.21.15.119:0/170506678 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c400bb40 con 0x7fe2e0071760 2024-04-25T10:18:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.730+0000 7fe2d6000640 1 --2- 172.21.15.119:0/170506678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c4003f30 0x7fe2c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.731+0000 7fe2d6000640 1 -- 172.21.15.119:0/170506678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2e0071760 msgr2=0x7fe2c40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.731+0000 7fe2d6000640 1 --2- 172.21.15.119:0/170506678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2e0071760 0x7fe2c40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.731+0000 7fe2df400640 1 --2- 172.21.15.119:0/170506678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2c4004d40 0x7fe2c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.731+0000 7fe2d6000640 1 -- 172.21.15.119:0/170506678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2c4004d40 msgr2=0x7fe2c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.731+0000 7fe2d6000640 1 --2- 172.21.15.119:0/170506678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2c4004d40 0x7fe2c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.731+0000 7fe2d6000640 1 -- 172.21.15.119:0/170506678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c4003520 con 0x7fe2c4003f30 2024-04-25T10:18:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.731+0000 7fe2df400640 1 --2- 172.21.15.119:0/170506678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2c4004d40 0x7fe2c40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.732+0000 7fe2d6000640 1 --2- 172.21.15.119:0/170506678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c4003f30 0x7fe2c4004800 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800aac0 tx=0x7fe2c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbd2afc85b1687c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.732+0000 7fe2de000640 1 -- 172.21.15.119:0/170506678 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8039070 con 0x7fe2c4003f30 2024-04-25T10:18:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.732+0000 7fe2de000640 1 -- 172.21.15.119:0/170506678 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c802c750 con 0x7fe2c4003f30 2024-04-25T10:18:27.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.732+0000 7fe2de000640 1 -- 172.21.15.119:0/170506678 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c800b040 con 0x7fe2c4003f30 2024-04-25T10:18:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.733+0000 7fe2e5800640 1 -- 172.21.15.119:0/170506678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c4003f30 msgr2=0x7fe2c4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.733+0000 7fe2e5800640 1 --2- 172.21.15.119:0/170506678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2c4003f30 0x7fe2c4004800 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800aac0 tx=0x7fe2c4005830 comp rx=0 tx=0).stop 2024-04-25T10:18:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.733+0000 7fe2df400640 1 -- 172.21.15.119:0/170506678 reap_dead start 2024-04-25T10:18:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.733+0000 7fe2e5800640 1 -- 172.21.15.119:0/170506678 shutdown_connections 2024-04-25T10:18:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.733+0000 7fe2e5800640 1 -- 172.21.15.119:0/170506678 >> 172.21.15.119:0/170506678 conn(0x7fe2e006aba0 msgr2=0x7fe2e006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.733+0000 7fe2e5800640 1 -- 172.21.15.119:0/170506678 shutdown_connections 2024-04-25T10:18:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.733+0000 7fe2e5800640 1 -- 172.21.15.119:0/170506678 wait complete. 2024-04-25T10:18:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.735+0000 7fe2e5800640 1 Processor -- start 2024-04-25T10:18:27.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.735+0000 7fe2e5800640 1 -- start start 2024-04-25T10:18:27.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.736+0000 7fe2e5800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2c4003f30 0x7fe2e00b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.736+0000 7fe2e5800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2e014cbc0 0x7fe2e0152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.736+0000 7fe2e5800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2e01527a0 0x7fe2e00af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.736+0000 7fe2e5800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e006eec0 con 0x7fe2e014cbc0 2024-04-25T10:18:27.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.736+0000 7fe2e5800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e015e7b0 con 0x7fe2c4003f30 2024-04-25T10:18:27.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.736+0000 7fe2e5800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e006f040 con 0x7fe2e01527a0 2024-04-25T10:18:27.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.736+0000 7fe2d6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2e014cbc0 0x7fe2e0152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.736+0000 7fe2d6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2e014cbc0 0x7fe2e0152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32902/0 (socket says 172.21.15.119:32902) 2024-04-25T10:18:27.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.736+0000 7fe2d6000640 1 -- 172.21.15.119:0/3984407114 learned_addr learned my addr 172.21.15.119:0/3984407114 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:27.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.736+0000 7fe2d6000640 1 -- 172.21.15.119:0/3984407114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2e01527a0 msgr2=0x7fe2e00af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.736+0000 7fe2d6000640 1 --2- 172.21.15.119:0/3984407114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2e01527a0 0x7fe2e00af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.737+0000 7fe2d6000640 1 -- 172.21.15.119:0/3984407114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2c4003f30 msgr2=0x7fe2e00b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.737+0000 7fe2d6000640 1 --2- 172.21.15.119:0/3984407114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2c4003f30 0x7fe2e00b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.737+0000 7fe2d6000640 1 -- 172.21.15.119:0/3984407114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e01f6fb0 con 0x7fe2e014cbc0 2024-04-25T10:18:27.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.737+0000 7fe2d6000640 1 --2- 172.21.15.119:0/3984407114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2e014cbc0 0x7fe2e0152260 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800a9e0 tx=0x7fe2c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.737+0000 7fe2d7400640 1 -- 172.21.15.119:0/3984407114 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8039040 con 0x7fe2e014cbc0 2024-04-25T10:18:27.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.737+0000 7fe2d7400640 1 -- 172.21.15.119:0/3984407114 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c802c550 con 0x7fe2e014cbc0 2024-04-25T10:18:27.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.737+0000 7fe2d7400640 1 -- 172.21.15.119:0/3984407114 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c800b040 con 0x7fe2e014cbc0 2024-04-25T10:18:27.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.737+0000 7fe2e5800640 1 -- 172.21.15.119:0/3984407114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e01f7290 con 0x7fe2e014cbc0 2024-04-25T10:18:27.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.737+0000 7fe2e5800640 1 -- 172.21.15.119:0/3984407114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e01f7850 con 0x7fe2e014cbc0 2024-04-25T10:18:27.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.738+0000 7fe2e5800640 1 -- 172.21.15.119:0/3984407114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e01f3970 con 0x7fe2e014cbc0 2024-04-25T10:18:27.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.739+0000 7fe2d7400640 1 -- 172.21.15.119:0/3984407114 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2c8044030 con 0x7fe2e014cbc0 2024-04-25T10:18:27.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.741+0000 7fe2d7400640 1 --2- 172.21.15.119:0/3984407114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2bc06b500 0x7fe2bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.741+0000 7fe2d7400640 1 -- 172.21.15.119:0/3984407114 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6025+0+0 (secure 0 0 0) 0x7fe2c80a9ef0 con 0x7fe2e014cbc0 2024-04-25T10:18:27.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.741+0000 7fe2dea00640 1 --2- 172.21.15.119:0/3984407114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2bc06b500 0x7fe2bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.742+0000 7fe2dea00640 1 --2- 172.21.15.119:0/3984407114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2bc06b500 0x7fe2bc06d9c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0006e80 tx=0x7fe2d0006dd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.745+0000 7fe2d7400640 1 -- 172.21.15.119:0/3984407114 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2c804b060 con 0x7fe2e014cbc0 2024-04-25T10:18:27.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:27.980+0000 7fe2e5800640 1 -- 172.21.15.119:0/3984407114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe2e00750b0 con 0x7fe2e014cbc0 2024-04-25T10:18:28.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.525+0000 7fe2d7400640 1 -- 172.21.15.119:0/3984407114 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v114) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe2c8078c20 con 0x7fe2e014cbc0 2024-04-25T10:18:28.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.530+0000 7fe2e5800640 1 -- 172.21.15.119:0/3984407114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2bc06b500 msgr2=0x7fe2bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.530+0000 7fe2e5800640 1 --2- 172.21.15.119:0/3984407114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2bc06b500 0x7fe2bc06d9c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0006e80 tx=0x7fe2d0006dd0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.530+0000 7fe2e5800640 1 -- 172.21.15.119:0/3984407114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2e014cbc0 msgr2=0x7fe2e0152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.530+0000 7fe2e5800640 1 --2- 172.21.15.119:0/3984407114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2e014cbc0 0x7fe2e0152260 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800a9e0 tx=0x7fe2c8004730 comp rx=0 tx=0).stop 2024-04-25T10:18:28.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.530+0000 7fe2e5800640 1 -- 172.21.15.119:0/3984407114 shutdown_connections 2024-04-25T10:18:28.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.530+0000 7fe2e5800640 1 --2- 172.21.15.119:0/3984407114 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe2bc06b500 0x7fe2bc06d9c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.530+0000 7fe2e5800640 1 --2- 172.21.15.119:0/3984407114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2e01527a0 0x7fe2e00af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.530+0000 7fe2e5800640 1 --2- 172.21.15.119:0/3984407114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2e014cbc0 0x7fe2e0152260 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.530+0000 7fe2e5800640 1 --2- 172.21.15.119:0/3984407114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2c4003f30 0x7fe2e00b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.530+0000 7fe2e5800640 1 -- 172.21.15.119:0/3984407114 >> 172.21.15.119:0/3984407114 conn(0x7fe2e006aba0 msgr2=0x7fe2e00b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:28.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.530+0000 7fe2e5800640 1 -- 172.21.15.119:0/3984407114 shutdown_connections 2024-04-25T10:18:28.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.531+0000 7fe2e5800640 1 -- 172.21.15.119:0/3984407114 wait complete. 2024-04-25T10:18:28.555 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:18:28.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.759+0000 7f035e000640 1 Processor -- start 2024-04-25T10:18:28.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.759+0000 7f035e000640 1 -- start start 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.761+0000 7f035e000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f035815ae20 0x7f03580b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.761+0000 7f035e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03580b4730 0x7f035815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.761+0000 7f035e000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035806da40 con 0x7f03580aedc0 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.761+0000 7f035e000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035815c050 con 0x7f035815ae20 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.761+0000 7f035e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03580af1a0 con 0x7f03580b4730 2024-04-25T10:18:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.762+0000 7f0356a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f035815ae20 0x7f03580b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.762+0000 7f0357e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03580b4730 0x7f035815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.762+0000 7f0356a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f035815ae20 0x7f03580b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:43716/0 (socket says 172.21.15.119:43716) 2024-04-25T10:18:28.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.762+0000 7f0356a00640 1 -- 172.21.15.119:0/848281570 learned_addr learned my addr 172.21.15.119:0/848281570 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:28.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.763+0000 7f0357e00640 1 -- 172.21.15.119:0/848281570 >> v2:172.21.15.132:3300/0 conn(0x7f035815ae20 msgr2=0x7f03580b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:28.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.763+0000 7f0357e00640 1 --2- 172.21.15.119:0/848281570 >> v2:172.21.15.132:3300/0 conn(0x7f035815ae20 0x7f03580b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.763+0000 7f0357e00640 1 -- 172.21.15.119:0/848281570 >> v1:172.21.15.132:6789/0 conn(0x7f03580aedc0 legacy=0x7f035806d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.763+0000 7f0357e00640 1 -- 172.21.15.119:0/848281570 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035806dd20 con 0x7f03580b4730 2024-04-25T10:18:28.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.763+0000 7f0356a00640 1 --2- 172.21.15.119:0/848281570 >> v2:172.21.15.132:3300/0 conn(0x7f035815ae20 0x7f03580b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:28.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.764+0000 7f0357e00640 1 --2- 172.21.15.119:0/848281570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03580b4730 0x7f035815c7c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f035000ba40 tx=0x7f035002c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a231451218a00ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:28.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.765+0000 7f0356000640 1 -- 172.21.15.119:0/848281570 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350031710 con 0x7f03580b4730 2024-04-25T10:18:28.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.765+0000 7f0356000640 1 -- 172.21.15.119:0/848281570 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03500318b0 con 0x7f03580b4730 2024-04-25T10:18:28.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.765+0000 7f0356000640 1 -- 172.21.15.119:0/848281570 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350031c80 con 0x7f03580b4730 2024-04-25T10:18:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.766+0000 7f035e000640 1 -- 172.21.15.119:0/848281570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03580b4730 msgr2=0x7f035815c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.766+0000 7f035e000640 1 --2- 172.21.15.119:0/848281570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03580b4730 0x7f035815c7c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f035000ba40 tx=0x7f035002c7b0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.766+0000 7f035e000640 1 -- 172.21.15.119:0/848281570 shutdown_connections 2024-04-25T10:18:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.766+0000 7f035e000640 1 --2- 172.21.15.119:0/848281570 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03580b4730 0x7f035815c7c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.766+0000 7f035e000640 1 --2- 172.21.15.119:0/848281570 >> v2:172.21.15.132:3300/0 conn(0x7f035815ae20 0x7f03580b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.766+0000 7f035e000640 1 -- 172.21.15.119:0/848281570 >> 172.21.15.119:0/848281570 conn(0x7f035806aba0 msgr2=0x7f035806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.766+0000 7f035e000640 1 -- 172.21.15.119:0/848281570 shutdown_connections 2024-04-25T10:18:28.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.766+0000 7f035e000640 1 -- 172.21.15.119:0/848281570 wait complete. 2024-04-25T10:18:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.768+0000 7f035e000640 1 Processor -- start 2024-04-25T10:18:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.768+0000 7f035e000640 1 -- start start 2024-04-25T10:18:28.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.769+0000 7f035e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03580aedc0 0x7f035814cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.769+0000 7f035e000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03580b4730 0x7f0358154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.769+0000 7f035e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f035815ae20 0x7f0358154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.769+0000 7f035e000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035815c2a0 con 0x7f03580aedc0 2024-04-25T10:18:28.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.769+0000 7f035e000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035815b230 con 0x7f03580b4730 2024-04-25T10:18:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.769+0000 7f035e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035815cd00 con 0x7f035815ae20 2024-04-25T10:18:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.769+0000 7f0357400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03580aedc0 0x7f035814cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.769+0000 7f0357400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03580aedc0 0x7f035814cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32914/0 (socket says 172.21.15.119:32914) 2024-04-25T10:18:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.769+0000 7f0357400640 1 -- 172.21.15.119:0/1075348129 learned_addr learned my addr 172.21.15.119:0/1075348129 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:28.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.769+0000 7f0356a00640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03580b4730 0x7f0358154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.770+0000 7f0357400640 1 -- 172.21.15.119:0/1075348129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f035815ae20 msgr2=0x7f0358154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.770+0000 7f0357e00640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f035815ae20 0x7f0358154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.770+0000 7f0357400640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f035815ae20 0x7f0358154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.770+0000 7f0357400640 1 -- 172.21.15.119:0/1075348129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03580b4730 msgr2=0x7f0358154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.770+0000 7f0357400640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03580b4730 0x7f0358154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.770+0000 7f0357400640 1 -- 172.21.15.119:0/1075348129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035814fd70 con 0x7f03580aedc0 2024-04-25T10:18:28.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.770+0000 7f0356a00640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03580b4730 0x7f0358154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:28.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.770+0000 7f0357400640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03580aedc0 0x7f035814cc00 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f034800c7d0 tx=0x7f034800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:28.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.770+0000 7f034f400640 1 -- 172.21.15.119:0/1075348129 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348003b90 con 0x7f03580aedc0 2024-04-25T10:18:28.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.770+0000 7f034f400640 1 -- 172.21.15.119:0/1075348129 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0348003d50 con 0x7f03580aedc0 2024-04-25T10:18:28.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.770+0000 7f035e000640 1 -- 172.21.15.119:0/1075348129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035814ecd0 con 0x7f03580aedc0 2024-04-25T10:18:28.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.770+0000 7f034f400640 1 -- 172.21.15.119:0/1075348129 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034800aa90 con 0x7f03580aedc0 2024-04-25T10:18:28.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.770+0000 7f035e000640 1 -- 172.21.15.119:0/1075348129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035814f290 con 0x7f03580aedc0 2024-04-25T10:18:28.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.771+0000 7f035e000640 1 -- 172.21.15.119:0/1075348129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03580750b0 con 0x7f03580aedc0 2024-04-25T10:18:28.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.772+0000 7f034f400640 1 -- 172.21.15.119:0/1075348129 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f034800ac30 con 0x7f03580aedc0 2024-04-25T10:18:28.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.773+0000 7f034f400640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f033406b4d0 0x7f033406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.773+0000 7f034f400640 1 -- 172.21.15.119:0/1075348129 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f034808a330 con 0x7f03580aedc0 2024-04-25T10:18:28.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.774+0000 7f0356a00640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f033406b4d0 0x7f033406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.774+0000 7f0356a00640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f033406b4d0 0x7f033406d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0358152b60 tx=0x7f0340008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:28.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.778+0000 7f034f400640 1 -- 172.21.15.119:0/1075348129 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0348058fd0 con 0x7f03580aedc0 2024-04-25T10:18:28.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:28.973+0000 7f035e000640 1 -- 172.21.15.119:0/1075348129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f03580cff90 con 0x7f03580aedc0 2024-04-25T10:18:29.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:29.929+0000 7f034f400640 1 -- 172.21.15.119:0/1075348129 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) v1 ==== 181+0+0 (secure 0 0 0) 0x7f03480588d0 con 0x7f03580aedc0 2024-04-25T10:18:29.932 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:18:29.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:29.933+0000 7f035e000640 1 -- 172.21.15.119:0/1075348129 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f033406b4d0 msgr2=0x7f033406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:29.933+0000 7f035e000640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f033406b4d0 0x7f033406d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0358152b60 tx=0x7f0340008040 comp rx=0 tx=0).stop 2024-04-25T10:18:29.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:29.933+0000 7f035e000640 1 -- 172.21.15.119:0/1075348129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03580aedc0 msgr2=0x7f035814cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:29.933+0000 7f035e000640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03580aedc0 0x7f035814cc00 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f034800c7d0 tx=0x7f034800cc90 comp rx=0 tx=0).stop 2024-04-25T10:18:29.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:29.934+0000 7f035e000640 1 -- 172.21.15.119:0/1075348129 shutdown_connections 2024-04-25T10:18:29.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:29.934+0000 7f035e000640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f033406b4d0 0x7f033406d990 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:29.934+0000 7f035e000640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f035815ae20 0x7f0358154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:29.934+0000 7f035e000640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03580b4730 0x7f0358154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:29.934+0000 7f035e000640 1 --2- 172.21.15.119:0/1075348129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03580aedc0 0x7f035814cc00 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:29.934+0000 7f035e000640 1 -- 172.21.15.119:0/1075348129 >> 172.21.15.119:0/1075348129 conn(0x7f035806aba0 msgr2=0x7f035806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:29.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:29.934+0000 7f035e000640 1 -- 172.21.15.119:0/1075348129 shutdown_connections 2024-04-25T10:18:29.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:29.934+0000 7f035e000640 1 -- 172.21.15.119:0/1075348129 wait complete. 2024-04-25T10:18:29.961 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:18:30.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.160+0000 7fd83de00640 1 Processor -- start 2024-04-25T10:18:30.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.160+0000 7fd83de00640 1 -- start start 2024-04-25T10:18:30.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.161+0000 7fd83de00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd8380d44c0 0x7fd8380d33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.161+0000 7fd83de00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8380d1f80 con 0x7fd83815db70 2024-04-25T10:18:30.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.161+0000 7fd83de00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8380d2260 con 0x7fd8380b6e80 2024-04-25T10:18:30.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.161+0000 7fd83de00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8380b69f0 con 0x7fd8380d44c0 2024-04-25T10:18:30.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.162+0000 7fd83ca00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd83815db70 0x7fd8380d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42648/0 (socket says 172.21.15.119:42648) 2024-04-25T10:18:30.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.162+0000 7fd83ca00640 1 -- 172.21.15.119:0/2914452192 learned_addr learned my addr 172.21.15.119:0/2914452192 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:30.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.162+0000 7fd837e00640 1 --2- 172.21.15.119:0/2914452192 >> v2:172.21.15.119:3300/0 conn(0x7fd8380d44c0 0x7fd8380d33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.162+0000 7fd837400640 1 -- 172.21.15.119:0/2914452192 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2423177386 0 0) 0x7fd8380d1f80 con 0x7fd83815db70 2024-04-25T10:18:30.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.162+0000 7fd837400640 1 -- 172.21.15.119:0/2914452192 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd820003520 con 0x7fd83815db70 2024-04-25T10:18:30.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.162+0000 7fd837e00640 1 -- 172.21.15.119:0/2914452192 >> v1:172.21.15.132:6789/0 conn(0x7fd8380b6e80 legacy=0x7fd8380b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:30.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.162+0000 7fd837e00640 1 -- 172.21.15.119:0/2914452192 >> v1:172.21.15.119:6789/0 conn(0x7fd83815db70 legacy=0x7fd8380d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.162+0000 7fd837e00640 1 -- 172.21.15.119:0/2914452192 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8380d2540 con 0x7fd8380d44c0 2024-04-25T10:18:30.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.164+0000 7fd837e00640 1 --2- 172.21.15.119:0/2914452192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8380d44c0 0x7fd8380d33b0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fd8380b6b70 tx=0x7fd82402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e8576be3e79d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.164+0000 7fd837400640 1 -- 172.21.15.119:0/2914452192 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824037070 con 0x7fd8380d44c0 2024-04-25T10:18:30.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.164+0000 7fd837400640 1 -- 172.21.15.119:0/2914452192 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82402ae40 con 0x7fd8380d44c0 2024-04-25T10:18:30.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.164+0000 7fd837400640 1 -- 172.21.15.119:0/2914452192 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824032660 con 0x7fd8380d44c0 2024-04-25T10:18:30.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.164+0000 7fd83de00640 1 -- 172.21.15.119:0/2914452192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8380d44c0 msgr2=0x7fd8380d33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.164+0000 7fd83de00640 1 --2- 172.21.15.119:0/2914452192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8380d44c0 0x7fd8380d33b0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fd8380b6b70 tx=0x7fd82402a130 comp rx=0 tx=0).stop 2024-04-25T10:18:30.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.165+0000 7fd83de00640 1 -- 172.21.15.119:0/2914452192 shutdown_connections 2024-04-25T10:18:30.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.165+0000 7fd83de00640 1 --2- 172.21.15.119:0/2914452192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8380d44c0 0x7fd8380d33b0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.165+0000 7fd83de00640 1 -- 172.21.15.119:0/2914452192 >> 172.21.15.119:0/2914452192 conn(0x7fd8380724b0 msgr2=0x7fd8380728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:30.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.165+0000 7fd83de00640 1 -- 172.21.15.119:0/2914452192 shutdown_connections 2024-04-25T10:18:30.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.165+0000 7fd83de00640 1 -- 172.21.15.119:0/2914452192 wait complete. 2024-04-25T10:18:30.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.166+0000 7fd83de00640 1 Processor -- start 2024-04-25T10:18:30.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.167+0000 7fd83de00640 1 -- start start 2024-04-25T10:18:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.167+0000 7fd83de00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8380b6e80 0x7fd83814ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.167+0000 7fd83de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8380d44c0 0x7fd8381e6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.167+0000 7fd83de00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd83815db70 0x7fd8381e6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.167+0000 7fd83de00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380d3060 con 0x7fd8380d44c0 2024-04-25T10:18:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.167+0000 7fd83de00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380d38f0 con 0x7fd83815db70 2024-04-25T10:18:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.167+0000 7fd83de00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380d1b30 con 0x7fd8380b6e80 2024-04-25T10:18:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.168+0000 7fd837e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8380d44c0 0x7fd8381e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.168+0000 7fd837e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8380d44c0 0x7fd8381e6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32940/0 (socket says 172.21.15.119:32940) 2024-04-25T10:18:30.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.168+0000 7fd837e00640 1 -- 172.21.15.119:0/2628736908 learned_addr learned my addr 172.21.15.119:0/2628736908 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:30.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.168+0000 7fd83d400640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd83815db70 0x7fd8381e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.168+0000 7fd837e00640 1 -- 172.21.15.119:0/2628736908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8380b6e80 msgr2=0x7fd83814ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.168+0000 7fd83ca00640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8380b6e80 0x7fd83814ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.168+0000 7fd837e00640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8380b6e80 0x7fd83814ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.168+0000 7fd837e00640 1 -- 172.21.15.119:0/2628736908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd83815db70 msgr2=0x7fd8381e6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.168+0000 7fd837e00640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd83815db70 0x7fd8381e6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.168+0000 7fd837e00640 1 -- 172.21.15.119:0/2628736908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8380c9d20 con 0x7fd8380d44c0 2024-04-25T10:18:30.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.168+0000 7fd83d400640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd83815db70 0x7fd8381e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:30.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.169+0000 7fd837e00640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8380d44c0 0x7fd8381e6230 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fd824000c00 tx=0x7fd8240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.169+0000 7fd834c00640 1 -- 172.21.15.119:0/2628736908 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824041070 con 0x7fd8380d44c0 2024-04-25T10:18:30.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.169+0000 7fd834c00640 1 -- 172.21.15.119:0/2628736908 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82403c460 con 0x7fd8380d44c0 2024-04-25T10:18:30.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.169+0000 7fd834c00640 1 -- 172.21.15.119:0/2628736908 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824037040 con 0x7fd8380d44c0 2024-04-25T10:18:30.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.169+0000 7fd83de00640 1 -- 172.21.15.119:0/2628736908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83815ecd0 con 0x7fd8380d44c0 2024-04-25T10:18:30.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.169+0000 7fd83de00640 1 -- 172.21.15.119:0/2628736908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8380c9030 con 0x7fd8380d44c0 2024-04-25T10:18:30.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.170+0000 7fd83de00640 1 -- 172.21.15.119:0/2628736908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8380749b0 con 0x7fd8380d44c0 2024-04-25T10:18:30.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.171+0000 7fd834c00640 1 -- 172.21.15.119:0/2628736908 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd824044070 con 0x7fd8380d44c0 2024-04-25T10:18:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.172+0000 7fd834c00640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd81806b4d0 0x7fd81806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.172+0000 7fd834c00640 1 -- 172.21.15.119:0/2628736908 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5587+0+0 (secure 0 0 0) 0x7fd8240aa580 con 0x7fd8380d44c0 2024-04-25T10:18:30.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.172+0000 7fd83ca00640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd81806b4d0 0x7fd81806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.173+0000 7fd83ca00640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd81806b4d0 0x7fd81806d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd82c000f60 tx=0x7fd82c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.177+0000 7fd834c00640 1 -- 172.21.15.119:0/2628736908 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd824079500 con 0x7fd8380d44c0 2024-04-25T10:18:30.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.373+0000 7fd83de00640 1 -- 172.21.15.119:0/2628736908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd8380ceda0 con 0x7fd8380d44c0 2024-04-25T10:18:30.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.932+0000 7fd834c00640 1 -- 172.21.15.119:0/2628736908 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v115) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd824078e00 con 0x7fd8380d44c0 2024-04-25T10:18:30.936 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2024-04-25T10:18:30.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.936+0000 7fd83de00640 1 -- 172.21.15.119:0/2628736908 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd81806b4d0 msgr2=0x7fd81806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.936+0000 7fd83de00640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd81806b4d0 0x7fd81806d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd82c000f60 tx=0x7fd82c009040 comp rx=0 tx=0).stop 2024-04-25T10:18:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.936+0000 7fd83de00640 1 -- 172.21.15.119:0/2628736908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8380d44c0 msgr2=0x7fd8381e6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.936+0000 7fd83de00640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8380d44c0 0x7fd8381e6230 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fd824000c00 tx=0x7fd8240046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.936+0000 7fd83de00640 1 -- 172.21.15.119:0/2628736908 shutdown_connections 2024-04-25T10:18:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.936+0000 7fd83de00640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fd81806b4d0 0x7fd81806d990 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.936+0000 7fd83de00640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd83815db70 0x7fd8381e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.936+0000 7fd83de00640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8380d44c0 0x7fd8381e6230 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.936+0000 7fd83de00640 1 --2- 172.21.15.119:0/2628736908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8380b6e80 0x7fd83814ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.936+0000 7fd83de00640 1 -- 172.21.15.119:0/2628736908 >> 172.21.15.119:0/2628736908 conn(0x7fd8380724b0 msgr2=0x7fd8380b5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.937+0000 7fd83de00640 1 -- 172.21.15.119:0/2628736908 shutdown_connections 2024-04-25T10:18:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:30.937+0000 7fd83de00640 1 -- 172.21.15.119:0/2628736908 wait complete. 2024-04-25T10:18:30.963 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:18:31.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.156+0000 7f4fc7a00640 1 Processor -- start 2024-04-25T10:18:31.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.156+0000 7f4fc7a00640 1 -- start start 2024-04-25T10:18:31.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.157+0000 7f4fc7a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4fc00b42c0 0x7f4fc015ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.157+0000 7f4fc7a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc015aef0 0x7f4fc00cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.157+0000 7f4fc7a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc015b370 con 0x7f4fc00b46a0 2024-04-25T10:18:31.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.157+0000 7f4fc7a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc00d05d0 con 0x7f4fc00b42c0 2024-04-25T10:18:31.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.157+0000 7f4fc7a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc015b650 con 0x7f4fc015aef0 2024-04-25T10:18:31.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.158+0000 7f4fc5000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4fc00b42c0 0x7f4fc015ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.158+0000 7f4fc5000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4fc00b42c0 0x7f4fc015ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40944/0 (socket says 172.21.15.119:40944) 2024-04-25T10:18:31.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.158+0000 7f4fc5000640 1 -- 172.21.15.119:0/166110265 learned_addr learned my addr 172.21.15.119:0/166110265 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:31.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.158+0000 7f4fb7e00640 1 --2- 172.21.15.119:0/166110265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc015aef0 0x7f4fc00cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.158+0000 7f4fc5000640 1 -- 172.21.15.119:0/166110265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc015aef0 msgr2=0x7f4fc00cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.158+0000 7f4fc5000640 1 --2- 172.21.15.119:0/166110265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc015aef0 0x7f4fc00cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.158+0000 7f4fc5000640 1 -- 172.21.15.119:0/166110265 >> v1:172.21.15.132:6789/0 conn(0x7f4fc00b46a0 legacy=0x7f4fc01549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.158+0000 7f4fc5000640 1 -- 172.21.15.119:0/166110265 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc015b930 con 0x7f4fc00b42c0 2024-04-25T10:18:31.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.158+0000 7f4fb7e00640 1 --2- 172.21.15.119:0/166110265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc015aef0 0x7f4fc00cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:31.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.159+0000 7f4fc5000640 1 --2- 172.21.15.119:0/166110265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00b42c0 0x7f4fc015ef40 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f4fb00098a0 tx=0x7f4fb002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81d19603c68d381 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.160+0000 7f4fbfe00640 1 -- 172.21.15.119:0/166110265 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0037070 con 0x7f4fc00b42c0 2024-04-25T10:18:31.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.160+0000 7f4fbfe00640 1 -- 172.21.15.119:0/166110265 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb002ae20 con 0x7f4fc00b42c0 2024-04-25T10:18:31.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.160+0000 7f4fbfe00640 1 -- 172.21.15.119:0/166110265 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0032660 con 0x7f4fc00b42c0 2024-04-25T10:18:31.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.160+0000 7f4fc7a00640 1 -- 172.21.15.119:0/166110265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00b42c0 msgr2=0x7f4fc015ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.160+0000 7f4fc7a00640 1 --2- 172.21.15.119:0/166110265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00b42c0 0x7f4fc015ef40 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f4fb00098a0 tx=0x7f4fb002a110 comp rx=0 tx=0).stop 2024-04-25T10:18:31.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.160+0000 7f4fc7a00640 1 -- 172.21.15.119:0/166110265 shutdown_connections 2024-04-25T10:18:31.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.160+0000 7f4fc7a00640 1 --2- 172.21.15.119:0/166110265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc015aef0 0x7f4fc00cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.160+0000 7f4fc7a00640 1 --2- 172.21.15.119:0/166110265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00b42c0 0x7f4fc015ef40 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.160+0000 7f4fc7a00640 1 -- 172.21.15.119:0/166110265 >> 172.21.15.119:0/166110265 conn(0x7f4fc006aba0 msgr2=0x7f4fc006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.160+0000 7f4fc7a00640 1 -- 172.21.15.119:0/166110265 shutdown_connections 2024-04-25T10:18:31.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.160+0000 7f4fc7a00640 1 -- 172.21.15.119:0/166110265 wait complete. 2024-04-25T10:18:31.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.162+0000 7f4fc7a00640 1 Processor -- start 2024-04-25T10:18:31.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.162+0000 7f4fc7a00640 1 -- start start 2024-04-25T10:18:31.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.163+0000 7f4fc7a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00b42c0 0x7f4fc01f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.163+0000 7f4fc7a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc00b46a0 0x7f4fc014f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.163+0000 7f4fc7a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc015aef0 0x7f4fc0158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.163+0000 7f4fc7a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc006eae0 con 0x7f4fc00b42c0 2024-04-25T10:18:31.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.163+0000 7f4fc7a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00cf980 con 0x7f4fc015aef0 2024-04-25T10:18:31.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.163+0000 7f4fc7a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc006ec60 con 0x7f4fc00b46a0 2024-04-25T10:18:31.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.164+0000 7f4fc5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00b42c0 0x7f4fc01f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.164+0000 7f4fc5000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00b42c0 0x7f4fc01f4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40946/0 (socket says 172.21.15.119:40946) 2024-04-25T10:18:31.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.164+0000 7f4fc5000640 1 -- 172.21.15.119:0/3756322852 learned_addr learned my addr 172.21.15.119:0/3756322852 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:31.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.164+0000 7f4fb7e00640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc00b46a0 0x7f4fc014f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.164+0000 7f4fc5000640 1 -- 172.21.15.119:0/3756322852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc00b46a0 msgr2=0x7f4fc014f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.164+0000 7f4fc5a00640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc015aef0 0x7f4fc0158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.164+0000 7f4fc5000640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc00b46a0 0x7f4fc014f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.164+0000 7f4fc5000640 1 -- 172.21.15.119:0/3756322852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc015aef0 msgr2=0x7f4fc0158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.164+0000 7f4fc5000640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc015aef0 0x7f4fc0158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.164+0000 7f4fc5000640 1 -- 172.21.15.119:0/3756322852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc00a8dd0 con 0x7f4fc00b42c0 2024-04-25T10:18:31.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.164+0000 7f4fb7e00640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc00b46a0 0x7f4fc014f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:31.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.164+0000 7f4fc5000640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00b42c0 0x7f4fc01f4ea0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0000c00 tx=0x7f4fb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.165+0000 7f4fbd600640 1 -- 172.21.15.119:0/3756322852 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0037040 con 0x7f4fc00b42c0 2024-04-25T10:18:31.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.165+0000 7f4fbd600640 1 -- 172.21.15.119:0/3756322852 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb0004030 con 0x7f4fc00b42c0 2024-04-25T10:18:31.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.165+0000 7f4fc7a00640 1 -- 172.21.15.119:0/3756322852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc00a90b0 con 0x7f4fc00b42c0 2024-04-25T10:18:31.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.165+0000 7f4fbd600640 1 -- 172.21.15.119:0/3756322852 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0004420 con 0x7f4fc00b42c0 2024-04-25T10:18:31.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.165+0000 7f4fc7a00640 1 -- 172.21.15.119:0/3756322852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc00a9670 con 0x7f4fc00b42c0 2024-04-25T10:18:31.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.166+0000 7f4fc7a00640 1 -- 172.21.15.119:0/3756322852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc00750b0 con 0x7f4fc00b42c0 2024-04-25T10:18:31.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.167+0000 7f4fbd600640 1 -- 172.21.15.119:0/3756322852 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fb003a070 con 0x7f4fc00b42c0 2024-04-25T10:18:31.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.168+0000 7f4fbd600640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4f9c06b4d0 0x7f4f9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.168+0000 7f4fbd600640 1 -- 172.21.15.119:0/3756322852 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f4fb00aa5a0 con 0x7f4fc00b42c0 2024-04-25T10:18:31.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.169+0000 7f4fb7e00640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4f9c06b4d0 0x7f4f9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.169+0000 7f4fb7e00640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4f9c06b4d0 0x7f4f9c06d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8006fd0 tx=0x7f4fa8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.173+0000 7f4fbd600640 1 -- 172.21.15.119:0/3756322852 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fb0041020 con 0x7f4fc00b42c0 2024-04-25T10:18:31.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.382+0000 7f4fc7a00640 1 -- 172.21.15.119:0/3756322852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4fc006e700 con 0x7f4fc00b42c0 2024-04-25T10:18:31.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.390+0000 7f4fbd600640 1 -- 172.21.15.119:0/3756322852 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4fb002ae20 con 0x7f4fc00b42c0 2024-04-25T10:18:31.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.393+0000 7f4fc7a00640 1 -- 172.21.15.119:0/3756322852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4f9c06b4d0 msgr2=0x7f4f9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.393+0000 7f4fc7a00640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4f9c06b4d0 0x7f4f9c06d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8006fd0 tx=0x7f4fa8008040 comp rx=0 tx=0).stop 2024-04-25T10:18:31.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.393+0000 7f4fc7a00640 1 -- 172.21.15.119:0/3756322852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00b42c0 msgr2=0x7f4fc01f4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.393+0000 7f4fc7a00640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00b42c0 0x7f4fc01f4ea0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0000c00 tx=0x7f4fb00046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.393+0000 7f4fc7a00640 1 -- 172.21.15.119:0/3756322852 shutdown_connections 2024-04-25T10:18:31.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.393+0000 7f4fc7a00640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f4f9c06b4d0 0x7f4f9c06d990 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.393+0000 7f4fc7a00640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc015aef0 0x7f4fc0158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.393+0000 7f4fc7a00640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4fc00b46a0 0x7f4fc014f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.393+0000 7f4fc7a00640 1 --2- 172.21.15.119:0/3756322852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00b42c0 0x7f4fc01f4ea0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.393+0000 7f4fc7a00640 1 -- 172.21.15.119:0/3756322852 >> 172.21.15.119:0/3756322852 conn(0x7f4fc006aba0 msgr2=0x7f4fc015bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.393+0000 7f4fc7a00640 1 -- 172.21.15.119:0/3756322852 shutdown_connections 2024-04-25T10:18:31.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.394+0000 7f4fc7a00640 1 -- 172.21.15.119:0/3756322852 wait complete. 2024-04-25T10:18:31.415 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T10:18:31.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.608+0000 7f6da4a00640 1 Processor -- start 2024-04-25T10:18:31.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.609+0000 7f6da4a00640 1 -- start start 2024-04-25T10:18:31.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.610+0000 7f6da4a00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6da015a870 0x7f6da00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.610+0000 7f6da4a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6da00cfc40 0x7f6da015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.610+0000 7f6da4a00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da00cdb10 con 0x7f6da006e2e0 2024-04-25T10:18:31.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.610+0000 7f6da4a00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da00ce720 con 0x7f6da015a870 2024-04-25T10:18:31.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.610+0000 7f6da4a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00cddf0 con 0x7f6da00cfc40 2024-04-25T10:18:31.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.610+0000 7f6d9de00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6da015a870 0x7f6da00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.611+0000 7f6d9de00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6da015a870 0x7f6da00cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40956/0 (socket says 172.21.15.119:40956) 2024-04-25T10:18:31.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.611+0000 7f6d9de00640 1 -- 172.21.15.119:0/3649848866 learned_addr learned my addr 172.21.15.119:0/3649848866 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:31.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.611+0000 7f6d9d400640 1 --1- 172.21.15.119:0/3649848866 >> v1:172.21.15.119:6789/0 conn(0x7f6da006e2e0 0x7f6da00cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36184/0 (socket says 172.21.15.119:36184) 2024-04-25T10:18:31.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.611+0000 7f6d9de00640 1 -- 172.21.15.119:0/3649848866 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6da00cfc40 msgr2=0x7f6da015ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.611+0000 7f6d9de00640 1 --2- 172.21.15.119:0/3649848866 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6da00cfc40 0x7f6da015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.611+0000 7f6d9de00640 1 -- 172.21.15.119:0/3649848866 >> v1:172.21.15.119:6789/0 conn(0x7f6da006e2e0 legacy=0x7f6da00cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.611+0000 7f6d9de00640 1 -- 172.21.15.119:0/3649848866 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da00ce0d0 con 0x7f6da015a870 2024-04-25T10:18:31.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.612+0000 7f6d9de00640 1 --2- 172.21.15.119:0/3649848866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da015a870 0x7f6da00cebb0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009870 tx=0x7f6d9002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae07288a77d3573c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.612+0000 7f6d9ca00640 1 -- 172.21.15.119:0/3649848866 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90037070 con 0x7f6da015a870 2024-04-25T10:18:31.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.612+0000 7f6d9ca00640 1 -- 172.21.15.119:0/3649848866 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d90032440 con 0x7f6da015a870 2024-04-25T10:18:31.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.612+0000 7f6d9ca00640 1 -- 172.21.15.119:0/3649848866 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90032810 con 0x7f6da015a870 2024-04-25T10:18:31.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.613+0000 7f6da4a00640 1 -- 172.21.15.119:0/3649848866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da015a870 msgr2=0x7f6da00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.613+0000 7f6da4a00640 1 --2- 172.21.15.119:0/3649848866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da015a870 0x7f6da00cebb0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009870 tx=0x7f6d9002a250 comp rx=0 tx=0).stop 2024-04-25T10:18:31.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.613+0000 7f6da4a00640 1 -- 172.21.15.119:0/3649848866 shutdown_connections 2024-04-25T10:18:31.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.613+0000 7f6da4a00640 1 --2- 172.21.15.119:0/3649848866 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6da00cfc40 0x7f6da015ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.613+0000 7f6da4a00640 1 --2- 172.21.15.119:0/3649848866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da015a870 0x7f6da00cebb0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.613+0000 7f6da4a00640 1 -- 172.21.15.119:0/3649848866 >> 172.21.15.119:0/3649848866 conn(0x7f6da006aba0 msgr2=0x7f6da006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.613+0000 7f6da4a00640 1 -- 172.21.15.119:0/3649848866 shutdown_connections 2024-04-25T10:18:31.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.613+0000 7f6da4a00640 1 -- 172.21.15.119:0/3649848866 wait complete. 2024-04-25T10:18:31.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.615+0000 7f6da4a00640 1 Processor -- start 2024-04-25T10:18:31.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.615+0000 7f6da4a00640 1 -- start start 2024-04-25T10:18:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.616+0000 7f6da4a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da006e2e0 0x7f6da0141f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.616+0000 7f6da4a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6da00cfc40 0x7f6da01e8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.616+0000 7f6da4a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6da015a870 0x7f6da01e8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.616+0000 7f6da4a00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00cd700 con 0x7f6da006e2e0 2024-04-25T10:18:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.616+0000 7f6da4a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00cd580 con 0x7f6da015a870 2024-04-25T10:18:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.616+0000 7f6da4a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00cd880 con 0x7f6da00cfc40 2024-04-25T10:18:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.616+0000 7f6d9de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da006e2e0 0x7f6da0141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.616+0000 7f6d9de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da006e2e0 0x7f6da0141f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40968/0 (socket says 172.21.15.119:40968) 2024-04-25T10:18:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.616+0000 7f6d9de00640 1 -- 172.21.15.119:0/718731353 learned_addr learned my addr 172.21.15.119:0/718731353 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.617+0000 7f6d9de00640 1 -- 172.21.15.119:0/718731353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6da00cfc40 msgr2=0x7f6da01e8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.617+0000 7f6d9de00640 1 --2- 172.21.15.119:0/718731353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6da00cfc40 0x7f6da01e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.617+0000 7f6d9de00640 1 -- 172.21.15.119:0/718731353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6da015a870 msgr2=0x7f6da01e8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.617+0000 7f6d9de00640 1 --2- 172.21.15.119:0/718731353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6da015a870 0x7f6da01e8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.617+0000 7f6d9de00640 1 -- 172.21.15.119:0/718731353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da00adfb0 con 0x7f6da006e2e0 2024-04-25T10:18:31.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.617+0000 7f6d9de00640 1 --2- 172.21.15.119:0/718731353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da006e2e0 0x7f6da0141f80 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009870 tx=0x7f6d900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.617+0000 7f6d96000640 1 -- 172.21.15.119:0/718731353 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90041070 con 0x7f6da006e2e0 2024-04-25T10:18:31.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.617+0000 7f6d96000640 1 -- 172.21.15.119:0/718731353 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9003c870 con 0x7f6da006e2e0 2024-04-25T10:18:31.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.617+0000 7f6d96000640 1 -- 172.21.15.119:0/718731353 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90037040 con 0x7f6da006e2e0 2024-04-25T10:18:31.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.617+0000 7f6da4a00640 1 -- 172.21.15.119:0/718731353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da01429b0 con 0x7f6da006e2e0 2024-04-25T10:18:31.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.618+0000 7f6da4a00640 1 -- 172.21.15.119:0/718731353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da0142f70 con 0x7f6da006e2e0 2024-04-25T10:18:31.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.619+0000 7f6da4a00640 1 -- 172.21.15.119:0/718731353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da00749b0 con 0x7f6da006e2e0 2024-04-25T10:18:31.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.620+0000 7f6d96000640 1 -- 172.21.15.119:0/718731353 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d90044070 con 0x7f6da006e2e0 2024-04-25T10:18:31.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.621+0000 7f6d96000640 1 --2- 172.21.15.119:0/718731353 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6d8006b4d0 0x7f6d8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.621+0000 7f6d96000640 1 -- 172.21.15.119:0/718731353 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f6d900aac50 con 0x7f6da006e2e0 2024-04-25T10:18:31.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.621+0000 7f6d9d400640 1 --2- 172.21.15.119:0/718731353 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6d8006b4d0 0x7f6d8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.622+0000 7f6d9d400640 1 --2- 172.21.15.119:0/718731353 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6d8006b4d0 0x7f6d8006d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c000f60 tx=0x7f6d8c007a00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.626+0000 7f6d96000640 1 -- 172.21.15.119:0/718731353 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d90079c80 con 0x7f6da006e2e0 2024-04-25T10:18:31.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.894+0000 7f6da4a00640 1 -- 172.21.15.119:0/718731353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f6da00cf6d0 con 0x7f6da006e2e0 2024-04-25T10:18:31.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.950+0000 7f6d96000640 1 -- 172.21.15.119:0/718731353 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v541) v1 ==== 144+0+0 (secure 0 0 0) 0x7f6d90079580 con 0x7f6da006e2e0 2024-04-25T10:18:31.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.953+0000 7f6da4a00640 1 -- 172.21.15.119:0/718731353 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6d8006b4d0 msgr2=0x7f6d8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.953+0000 7f6da4a00640 1 --2- 172.21.15.119:0/718731353 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6d8006b4d0 0x7f6d8006d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c000f60 tx=0x7f6d8c007a00 comp rx=0 tx=0).stop 2024-04-25T10:18:31.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.953+0000 7f6da4a00640 1 -- 172.21.15.119:0/718731353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da006e2e0 msgr2=0x7f6da0141f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.953+0000 7f6da4a00640 1 --2- 172.21.15.119:0/718731353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da006e2e0 0x7f6da0141f80 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009870 tx=0x7f6d900046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.953+0000 7f6da4a00640 1 -- 172.21.15.119:0/718731353 shutdown_connections 2024-04-25T10:18:31.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.953+0000 7f6da4a00640 1 --2- 172.21.15.119:0/718731353 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6d8006b4d0 0x7f6d8006d990 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.953+0000 7f6da4a00640 1 --2- 172.21.15.119:0/718731353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6da015a870 0x7f6da01e8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.953+0000 7f6da4a00640 1 --2- 172.21.15.119:0/718731353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6da00cfc40 0x7f6da01e8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.953+0000 7f6da4a00640 1 --2- 172.21.15.119:0/718731353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6da006e2e0 0x7f6da0141f80 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.953+0000 7f6da4a00640 1 -- 172.21.15.119:0/718731353 >> 172.21.15.119:0/718731353 conn(0x7f6da006aba0 msgr2=0x7f6da015cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.953+0000 7f6da4a00640 1 -- 172.21.15.119:0/718731353 shutdown_connections 2024-04-25T10:18:31.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:31.953+0000 7f6da4a00640 1 -- 172.21.15.119:0/718731353 wait complete. 2024-04-25T10:18:31.975 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:18:31.976 INFO:tasks.cephfs_test_runner: 2024-04-25T10:18:31.976 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T10:18:31.976 INFO:tasks.cephfs_test_runner:Ran 7 tests in 647.781s 2024-04-25T10:18:31.976 INFO:tasks.cephfs_test_runner: 2024-04-25T10:18:31.976 INFO:tasks.cephfs_test_runner:OK 2024-04-25T10:18:31.976 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T10:18:31.986 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T10:18:31.994 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T10:18:31.994 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:18:31.994 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:32.045 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:18:32.045 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-25T10:18:32.045 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:32.100 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:18:32.100 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:18:32.101 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:32.156 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:18:32.157 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-25T10:18:32.157 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:32.211 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:18:32.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:32.212 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:18:32.267 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:18:32.267 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:18:32.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:32.269 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:18:32.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:32.354 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:18:32.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:32.425 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2024-04-25T10:18:32.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:32.552 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-04-25T10:18:32.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:32.593 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2024-04-25T10:18:32.714 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2024-04-25T10:18:32.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:18:32.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:32.718 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-25T10:18:32.776 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:18:32.776 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T10:18:32.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:32.777 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:18:32.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:32.860 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:18:32.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:32.932 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-04-25T10:18:33.028 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:33.036 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:33.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:33.067 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-25T10:18:33.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:33.147 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-04-25T10:18:33.273 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2024-04-25T10:18:33.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:18:33.277 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T10:18:33.286 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:18:33.286 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:18:33.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.490+0000 7f9c26000640 1 Processor -- start 2024-04-25T10:18:33.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.490+0000 7f9c26000640 1 -- start start 2024-04-25T10:18:33.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.492+0000 7f9c26000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9c2015bf00 0x7f9c200cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.492+0000 7f9c26000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c200cc310 0x7f9c2015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.492+0000 7f9c26000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c200ce6c0 con 0x7f9c2015c820 2024-04-25T10:18:33.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.492+0000 7f9c26000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c201455e0 con 0x7f9c2015bf00 2024-04-25T10:18:33.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.492+0000 7f9c26000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2006d450 con 0x7f9c200cc310 2024-04-25T10:18:33.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.492+0000 7f9c1f400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9c2015bf00 0x7f9c200cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.492+0000 7f9c1fe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9c2015c820 0x7f9c2006d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36196/0 (socket says 172.21.15.119:36196) 2024-04-25T10:18:33.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.492+0000 7f9c1fe00640 1 -- 172.21.15.119:0/2602226964 learned_addr learned my addr 172.21.15.119:0/2602226964 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:33.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.492+0000 7f9c1f400640 1 -- 172.21.15.119:0/2602226964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c200cc310 msgr2=0x7f9c2015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:33.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.492+0000 7f9c1f400640 1 --2- 172.21.15.119:0/2602226964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c200cc310 0x7f9c2015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.492+0000 7f9c1f400640 1 -- 172.21.15.119:0/2602226964 >> v1:172.21.15.119:6789/0 conn(0x7f9c2015c820 legacy=0x7f9c2006d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.493+0000 7f9c1f400640 1 -- 172.21.15.119:0/2602226964 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c200cc7b0 con 0x7f9c2015bf00 2024-04-25T10:18:33.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.492+0000 7f9c1e000640 1 -- 172.21.15.119:0/2602226964 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967407933 0 0) 0x7f9c18001490 con 0x7f9c2015c820 2024-04-25T10:18:33.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.494+0000 7f9c1f400640 1 --2- 172.21.15.119:0/2602226964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c2015bf00 0x7f9c200cbdd0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f9c100097a0 tx=0x7f9c1002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3baa4ccee488e156 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.494+0000 7f9c1e000640 1 -- 172.21.15.119:0/2602226964 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10037070 con 0x7f9c2015bf00 2024-04-25T10:18:33.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.494+0000 7f9c1e000640 1 -- 172.21.15.119:0/2602226964 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c10032440 con 0x7f9c2015bf00 2024-04-25T10:18:33.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.494+0000 7f9c1e000640 1 -- 172.21.15.119:0/2602226964 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10032810 con 0x7f9c2015bf00 2024-04-25T10:18:33.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.494+0000 7f9c26000640 1 -- 172.21.15.119:0/2602226964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c2015bf00 msgr2=0x7f9c200cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.494+0000 7f9c26000640 1 --2- 172.21.15.119:0/2602226964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c2015bf00 0x7f9c200cbdd0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f9c100097a0 tx=0x7f9c1002a180 comp rx=0 tx=0).stop 2024-04-25T10:18:33.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.494+0000 7f9c26000640 1 -- 172.21.15.119:0/2602226964 shutdown_connections 2024-04-25T10:18:33.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.494+0000 7f9c26000640 1 --2- 172.21.15.119:0/2602226964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c200cc310 0x7f9c2015c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.494+0000 7f9c26000640 1 --2- 172.21.15.119:0/2602226964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c2015bf00 0x7f9c200cbdd0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.494+0000 7f9c26000640 1 -- 172.21.15.119:0/2602226964 >> 172.21.15.119:0/2602226964 conn(0x7f9c2006aba0 msgr2=0x7f9c2006afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:33.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.495+0000 7f9c26000640 1 -- 172.21.15.119:0/2602226964 shutdown_connections 2024-04-25T10:18:33.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.495+0000 7f9c26000640 1 -- 172.21.15.119:0/2602226964 wait complete. 2024-04-25T10:18:33.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.496+0000 7f9c26000640 1 Processor -- start 2024-04-25T10:18:33.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.496+0000 7f9c26000640 1 -- start start 2024-04-25T10:18:33.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.497+0000 7f9c26000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c200cc310 0x7f9c200ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.497+0000 7f9c26000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c2015bf00 0x7f9c201e4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.497+0000 7f9c26000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c2015c820 0x7f9c2014dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.497+0000 7f9c26000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c200cd050 con 0x7f9c2015bf00 2024-04-25T10:18:33.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.497+0000 7f9c26000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c200ccd70 con 0x7f9c2015c820 2024-04-25T10:18:33.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.497+0000 7f9c26000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2015cca0 con 0x7f9c200cc310 2024-04-25T10:18:33.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.498+0000 7f9c1ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c2015bf00 0x7f9c201e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.498+0000 7f9c1ea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c2015bf00 0x7f9c201e4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40988/0 (socket says 172.21.15.119:40988) 2024-04-25T10:18:33.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.498+0000 7f9c1ea00640 1 -- 172.21.15.119:0/785262 learned_addr learned my addr 172.21.15.119:0/785262 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:33.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.498+0000 7f9c1f400640 1 --2- 172.21.15.119:0/785262 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c200cc310 0x7f9c200ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.498+0000 7f9c1fe00640 1 --2- 172.21.15.119:0/785262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c2015c820 0x7f9c2014dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.498+0000 7f9c1ea00640 1 -- 172.21.15.119:0/785262 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c200cc310 msgr2=0x7f9c200ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.498+0000 7f9c1ea00640 1 --2- 172.21.15.119:0/785262 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c200cc310 0x7f9c200ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.498+0000 7f9c1ea00640 1 -- 172.21.15.119:0/785262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c2015c820 msgr2=0x7f9c2014dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.498+0000 7f9c1ea00640 1 --2- 172.21.15.119:0/785262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c2015c820 0x7f9c2014dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.498+0000 7f9c1ea00640 1 -- 172.21.15.119:0/785262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c201e4a90 con 0x7f9c2015bf00 2024-04-25T10:18:33.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.499+0000 7f9c1ea00640 1 --2- 172.21.15.119:0/785262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c2015bf00 0x7f9c201e4120 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00b480 tx=0x7f9c0c00b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.499+0000 7f9c17400640 1 -- 172.21.15.119:0/785262 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c014070 con 0x7f9c2015bf00 2024-04-25T10:18:33.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.499+0000 7f9c17400640 1 -- 172.21.15.119:0/785262 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c010870 con 0x7f9c2015bf00 2024-04-25T10:18:33.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.499+0000 7f9c17400640 1 -- 172.21.15.119:0/785262 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c010c60 con 0x7f9c2015bf00 2024-04-25T10:18:33.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.499+0000 7f9c26000640 1 -- 172.21.15.119:0/785262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c201e4d70 con 0x7f9c2015bf00 2024-04-25T10:18:33.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.499+0000 7f9c26000640 1 -- 172.21.15.119:0/785262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c201e5330 con 0x7f9c2015bf00 2024-04-25T10:18:33.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.500+0000 7f9c26000640 1 -- 172.21.15.119:0/785262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c200cdfc0 con 0x7f9c2015bf00 2024-04-25T10:18:33.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.501+0000 7f9c17400640 1 -- 172.21.15.119:0/785262 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c0c017070 con 0x7f9c2015bf00 2024-04-25T10:18:33.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.502+0000 7f9c17400640 1 --2- 172.21.15.119:0/785262 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9c0006b4d0 0x7f9c0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.502+0000 7f9c17400640 1 -- 172.21.15.119:0/785262 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f9c0c089d80 con 0x7f9c2015bf00 2024-04-25T10:18:33.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.502+0000 7f9c1f400640 1 --2- 172.21.15.119:0/785262 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9c0006b4d0 0x7f9c0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.503+0000 7f9c1f400640 1 --2- 172.21.15.119:0/785262 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9c0006b4d0 0x7f9c0006d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9c10034000 tx=0x7f9c1002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.507+0000 7f9c17400640 1 -- 172.21.15.119:0/785262 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c200cdfc0 con 0x7f9c2015bf00 2024-04-25T10:18:33.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.699+0000 7f9c26000640 1 -- 172.21.15.119:0/785262 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c20141130 con 0x7f9c0006b4d0 2024-04-25T10:18:33.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.700+0000 7f9c17400640 1 -- 172.21.15.119:0/785262 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13522 (secure 0 0 0) 0x7f9c20141130 con 0x7f9c0006b4d0 2024-04-25T10:18:33.704 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:33.704 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:18:33.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.703+0000 7f9c26000640 1 -- 172.21.15.119:0/785262 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9c0006b4d0 msgr2=0x7f9c0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.703+0000 7f9c26000640 1 --2- 172.21.15.119:0/785262 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9c0006b4d0 0x7f9c0006d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9c10034000 tx=0x7f9c1002e000 comp rx=0 tx=0).stop 2024-04-25T10:18:33.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.703+0000 7f9c26000640 1 -- 172.21.15.119:0/785262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c2015bf00 msgr2=0x7f9c201e4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.703+0000 7f9c26000640 1 --2- 172.21.15.119:0/785262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c2015bf00 0x7f9c201e4120 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00b480 tx=0x7f9c0c00b940 comp rx=0 tx=0).stop 2024-04-25T10:18:33.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.703+0000 7f9c26000640 1 -- 172.21.15.119:0/785262 shutdown_connections 2024-04-25T10:18:33.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.703+0000 7f9c26000640 1 --2- 172.21.15.119:0/785262 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f9c0006b4d0 0x7f9c0006d990 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.703+0000 7f9c26000640 1 --2- 172.21.15.119:0/785262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c2015c820 0x7f9c2014dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.703+0000 7f9c26000640 1 --2- 172.21.15.119:0/785262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c2015bf00 0x7f9c201e4120 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.703+0000 7f9c26000640 1 --2- 172.21.15.119:0/785262 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9c200cc310 0x7f9c200ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.703+0000 7f9c26000640 1 -- 172.21.15.119:0/785262 >> 172.21.15.119:0/785262 conn(0x7f9c2006aba0 msgr2=0x7f9c2013b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:33.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.703+0000 7f9c26000640 1 -- 172.21.15.119:0/785262 shutdown_connections 2024-04-25T10:18:33.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.703+0000 7f9c26000640 1 -- 172.21.15.119:0/785262 wait complete. 2024-04-25T10:18:33.725 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-04-25T10:18:32.158749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":278092,"kb_used_data":28648,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754696628,"statfs":{"total":773094113280,"available":772809347072,"internally_reserved":0,"allocated":29335552,"data_stored":22884631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392254,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-179,"num_write_kb":-8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002335"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":347,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T10:18:29.915494+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:18:29.915494+0000","last_peered":"2024-04-25T10:18:29.915494+0000","last_clean":"2024-04-25T10:18:29.915494+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:18:29.915494+0000","last_undegraded":"2024-04-25T10:18:29.915494+0000","last_fullsized":"2024-04-25T10:18:29.915494+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574990,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2011136,"data_stored":1203234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1417216,"data_stored":612327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32572,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1425408,"data_stored":618483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":618478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574990,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1425408,"data_stored":618483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574991,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36664,"kb_used_data":5484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":5615616,"data_stored":4812782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40752,"kb_used_data":9572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331088,"statfs":{"total":96636764160,"available":96595034112,"internally_reserved":0,"allocated":9801728,"data_stored":9000977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574990,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37252,"kb_used_data":6072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334588,"statfs":{"total":96636764160,"available":96598618112,"internally_reserved":0,"allocated":6217728,"data_stored":5399867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:18:33.726 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:18:33.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.923+0000 7f53f6000640 1 Processor -- start 2024-04-25T10:18:33.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.924+0000 7f53f6000640 1 -- start start 2024-04-25T10:18:33.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.925+0000 7f53f6000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f015ded0 0x7f53f00cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.925+0000 7f53f6000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f53f006fab0 0x7f53f0070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.925+0000 7f53f6000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f00c6540 con 0x7f53f006ff10 2024-04-25T10:18:33.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.925+0000 7f53f6000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f0072bc0 con 0x7f53f006fab0 2024-04-25T10:18:33.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.925+0000 7f53f6000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f00ccfc0 con 0x7f53f015ded0 2024-04-25T10:18:33.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.925+0000 7f53eea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f53f006fab0 0x7f53f0070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.926+0000 7f53efe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f53f006ff10 0x7f53f00cad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36204/0 (socket says 172.21.15.119:36204) 2024-04-25T10:18:33.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.926+0000 7f53eea00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f53f006fab0 0x7f53f0070320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40994/0 (socket says 172.21.15.119:40994) 2024-04-25T10:18:33.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.926+0000 7f53efe00640 1 -- 172.21.15.119:0/1177145246 learned_addr learned my addr 172.21.15.119:0/1177145246 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:33.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.926+0000 7f53ef400640 1 --2- 172.21.15.119:0/1177145246 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f015ded0 0x7f53f00cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.926+0000 7f53eea00640 1 -- 172.21.15.119:0/1177145246 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f015ded0 msgr2=0x7f53f00cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:33.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.926+0000 7f53eea00640 1 --2- 172.21.15.119:0/1177145246 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f015ded0 0x7f53f00cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.926+0000 7f53eea00640 1 -- 172.21.15.119:0/1177145246 >> v1:172.21.15.119:6789/0 conn(0x7f53f006ff10 legacy=0x7f53f00cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.926+0000 7f53eea00640 1 -- 172.21.15.119:0/1177145246 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f0072e10 con 0x7f53f006fab0 2024-04-25T10:18:33.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.926+0000 7f53ee000640 1 -- 172.21.15.119:0/1177145246 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142086224 0 0) 0x7f53f00c6540 con 0x7f53f006ff10 2024-04-25T10:18:33.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.927+0000 7f53eea00640 1 --2- 172.21.15.119:0/1177145246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006fab0 0x7f53f0070320 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a72ca0ae2936b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.927+0000 7f53ee000640 1 -- 172.21.15.119:0/1177145246 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc007740 con 0x7f53f006fab0 2024-04-25T10:18:33.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.927+0000 7f53ee000640 1 -- 172.21.15.119:0/1177145246 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc0078e0 con 0x7f53f006fab0 2024-04-25T10:18:33.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.927+0000 7f53ee000640 1 -- 172.21.15.119:0/1177145246 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc007cb0 con 0x7f53f006fab0 2024-04-25T10:18:33.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.928+0000 7f53f6000640 1 -- 172.21.15.119:0/1177145246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006fab0 msgr2=0x7f53f0070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.928+0000 7f53f6000640 1 --2- 172.21.15.119:0/1177145246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006fab0 0x7f53f0070320 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.928+0000 7f53f6000640 1 -- 172.21.15.119:0/1177145246 shutdown_connections 2024-04-25T10:18:33.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.928+0000 7f53f6000640 1 --2- 172.21.15.119:0/1177145246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f006fab0 0x7f53f0070320 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.928+0000 7f53f6000640 1 --2- 172.21.15.119:0/1177145246 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f015ded0 0x7f53f00cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.928+0000 7f53f6000640 1 -- 172.21.15.119:0/1177145246 >> 172.21.15.119:0/1177145246 conn(0x7f53f00b2370 msgr2=0x7f53f00b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:33.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.928+0000 7f53f6000640 1 -- 172.21.15.119:0/1177145246 shutdown_connections 2024-04-25T10:18:33.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.928+0000 7f53f6000640 1 -- 172.21.15.119:0/1177145246 wait complete. 2024-04-25T10:18:33.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.930+0000 7f53f6000640 1 Processor -- start 2024-04-25T10:18:33.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.930+0000 7f53f6000640 1 -- start start 2024-04-25T10:18:33.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.931+0000 7f53f6000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f006fab0 0x7f53f00adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.931+0000 7f53f6000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f015ded0 0x7f53f014f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.931+0000 7f53f6000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f00aef00 0x7f53f00c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.931+0000 7f53f6000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f00cd4f0 con 0x7f53f015ded0 2024-04-25T10:18:33.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.931+0000 7f53f6000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f00cd210 con 0x7f53f00aef00 2024-04-25T10:18:33.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.931+0000 7f53f6000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f00cdf50 con 0x7f53f006fab0 2024-04-25T10:18:33.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.931+0000 7f53eea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f015ded0 0x7f53f014f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.931+0000 7f53eea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f015ded0 0x7f53f014f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41008/0 (socket says 172.21.15.119:41008) 2024-04-25T10:18:33.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.931+0000 7f53eea00640 1 -- 172.21.15.119:0/2257676322 learned_addr learned my addr 172.21.15.119:0/2257676322 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:33.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.932+0000 7f53ef400640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f006fab0 0x7f53f00adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.932+0000 7f53efe00640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f00aef00 0x7f53f00c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.932+0000 7f53eea00640 1 -- 172.21.15.119:0/2257676322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f006fab0 msgr2=0x7f53f00adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.932+0000 7f53eea00640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f006fab0 0x7f53f00adf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.932+0000 7f53eea00640 1 -- 172.21.15.119:0/2257676322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f00aef00 msgr2=0x7f53f00c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.932+0000 7f53eea00640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f00aef00 0x7f53f00c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.932+0000 7f53eea00640 1 -- 172.21.15.119:0/2257676322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f00af470 con 0x7f53f015ded0 2024-04-25T10:18:33.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.932+0000 7f53ef400640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f006fab0 0x7f53f00adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:33.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.932+0000 7f53efe00640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f00aef00 0x7f53f00c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:33.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.932+0000 7f53eea00640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f015ded0 0x7f53f014f730 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.932+0000 7f53e7400640 1 -- 172.21.15.119:0/2257676322 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc040070 con 0x7f53f015ded0 2024-04-25T10:18:33.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.933+0000 7f53e7400640 1 -- 172.21.15.119:0/2257676322 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc031c40 con 0x7f53f015ded0 2024-04-25T10:18:33.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.933+0000 7f53e7400640 1 -- 172.21.15.119:0/2257676322 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc045460 con 0x7f53f015ded0 2024-04-25T10:18:33.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.933+0000 7f53f6000640 1 -- 172.21.15.119:0/2257676322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f00ad460 con 0x7f53f015ded0 2024-04-25T10:18:33.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.933+0000 7f53f6000640 1 -- 172.21.15.119:0/2257676322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f00ada20 con 0x7f53f015ded0 2024-04-25T10:18:33.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.934+0000 7f53f6000640 1 -- 172.21.15.119:0/2257676322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f00ad290 con 0x7f53f015ded0 2024-04-25T10:18:33.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.935+0000 7f53e7400640 1 -- 172.21.15.119:0/2257676322 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53dc039040 con 0x7f53f015ded0 2024-04-25T10:18:33.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.936+0000 7f53e7400640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f53d006b4d0 0x7f53d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.936+0000 7f53e7400640 1 -- 172.21.15.119:0/2257676322 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f53dc0ab8c0 con 0x7f53f015ded0 2024-04-25T10:18:33.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.936+0000 7f53ef400640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f53d006b4d0 0x7f53d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.937+0000 7f53ef400640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f53d006b4d0 0x7f53d006d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f53e0006fd0 tx=0x7f53e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:33.940+0000 7f53e7400640 1 -- 172.21.15.119:0/2257676322 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53dc07aa20 con 0x7f53f015ded0 2024-04-25T10:18:34.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.136+0000 7f53f6000640 1 -- 172.21.15.119:0/2257676322 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53f00749b0 con 0x7f53d006b4d0 2024-04-25T10:18:34.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.138+0000 7f53e7400640 1 -- 172.21.15.119:0/2257676322 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13522 (secure 0 0 0) 0x7f53f00749b0 con 0x7f53d006b4d0 2024-04-25T10:18:34.142 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:34.142 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:18:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.141+0000 7f53f6000640 1 -- 172.21.15.119:0/2257676322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f53d006b4d0 msgr2=0x7f53d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.141+0000 7f53f6000640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f53d006b4d0 0x7f53d006d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f53e0006fd0 tx=0x7f53e0008040 comp rx=0 tx=0).stop 2024-04-25T10:18:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.141+0000 7f53f6000640 1 -- 172.21.15.119:0/2257676322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f015ded0 msgr2=0x7f53f014f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.141+0000 7f53f6000640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f015ded0 0x7f53f014f730 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc004a40 comp rx=0 tx=0).stop 2024-04-25T10:18:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.141+0000 7f53f6000640 1 -- 172.21.15.119:0/2257676322 shutdown_connections 2024-04-25T10:18:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.141+0000 7f53f6000640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f53d006b4d0 0x7f53d006d990 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.141+0000 7f53f6000640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f00aef00 0x7f53f00c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.141+0000 7f53f6000640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53f015ded0 0x7f53f014f730 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.141+0000 7f53f6000640 1 --2- 172.21.15.119:0/2257676322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f006fab0 0x7f53f00adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.141+0000 7f53f6000640 1 -- 172.21.15.119:0/2257676322 >> 172.21.15.119:0/2257676322 conn(0x7f53f00b2370 msgr2=0x7f53f0074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.142+0000 7f53f6000640 1 -- 172.21.15.119:0/2257676322 shutdown_connections 2024-04-25T10:18:34.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.142+0000 7f53f6000640 1 -- 172.21.15.119:0/2257676322 wait complete. 2024-04-25T10:18:34.165 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-04-25T10:18:32.158749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":278092,"kb_used_data":28648,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754696628,"statfs":{"total":773094113280,"available":772809347072,"internally_reserved":0,"allocated":29335552,"data_stored":22884631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392254,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-179,"num_write_kb":-8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002335"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":347,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T10:18:29.915494+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:18:29.915494+0000","last_peered":"2024-04-25T10:18:29.915494+0000","last_clean":"2024-04-25T10:18:29.915494+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:18:29.915494+0000","last_undegraded":"2024-04-25T10:18:29.915494+0000","last_fullsized":"2024-04-25T10:18:29.915494+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574990,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2011136,"data_stored":1203234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1417216,"data_stored":612327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32572,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1425408,"data_stored":618483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":618478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574990,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1425408,"data_stored":618483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574991,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36664,"kb_used_data":5484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":5615616,"data_stored":4812782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40752,"kb_used_data":9572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331088,"statfs":{"total":96636764160,"available":96595034112,"internally_reserved":0,"allocated":9801728,"data_stored":9000977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574990,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37252,"kb_used_data":6072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334588,"statfs":{"total":96636764160,"available":96598618112,"internally_reserved":0,"allocated":6217728,"data_stored":5399867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:18:34.166 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:18:34.166 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:18:34.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.365+0000 7f1522400640 1 Processor -- start 2024-04-25T10:18:34.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.366+0000 7f1522400640 1 -- start start 2024-04-25T10:18:34.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.367+0000 7f1522400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151c0d3f80 0x7f151c0d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.367+0000 7f1522400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f151c0b9270 0x7f151c0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.367+0000 7f1522400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151c15ef10 con 0x7f151c15db60 2024-04-25T10:18:34.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.367+0000 7f1522400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151c15e390 con 0x7f151c0b9270 2024-04-25T10:18:34.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.367+0000 7f1522400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c15e670 con 0x7f151c0d3f80 2024-04-25T10:18:34.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.368+0000 7f1521a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f151c0b9270 0x7f151c0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.368+0000 7f1521a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f151c0b9270 0x7f151c0d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.119:35228/0 (socket says 172.21.15.119:35228) 2024-04-25T10:18:34.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.368+0000 7f1521a00640 1 -- 172.21.15.119:0/623778183 learned_addr learned my addr 172.21.15.119:0/623778183 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:34.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.368+0000 7f151be00640 1 --2- 172.21.15.119:0/623778183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151c0d3f80 0x7f151c0d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.369+0000 7f1521a00640 1 -- 172.21.15.119:0/623778183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151c0d3f80 msgr2=0x7f151c0d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:34.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.369+0000 7f1521a00640 1 --2- 172.21.15.119:0/623778183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151c0d3f80 0x7f151c0d1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.369+0000 7f1521a00640 1 -- 172.21.15.119:0/623778183 >> v1:172.21.15.132:6789/0 conn(0x7f151c15db60 legacy=0x7f151c0d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.369+0000 7f1521a00640 1 -- 172.21.15.119:0/623778183 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c15e950 con 0x7f151c0b9270 2024-04-25T10:18:34.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.369+0000 7f151b400640 1 -- 172.21.15.119:0/623778183 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625855791 0 0) 0x7f151c15ef10 con 0x7f151c15db60 2024-04-25T10:18:34.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.370+0000 7f1521a00640 1 --2- 172.21.15.119:0/623778183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f151c0b9270 0x7f151c0d24d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1514009870 tx=0x7f151402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ceffcb0ebea201 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.371+0000 7f151b400640 1 -- 172.21.15.119:0/623778183 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514037070 con 0x7f151c0b9270 2024-04-25T10:18:34.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.371+0000 7f151b400640 1 -- 172.21.15.119:0/623778183 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1514032440 con 0x7f151c0b9270 2024-04-25T10:18:34.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.371+0000 7f151b400640 1 -- 172.21.15.119:0/623778183 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514032810 con 0x7f151c0b9270 2024-04-25T10:18:34.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.372+0000 7f1522400640 1 -- 172.21.15.119:0/623778183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f151c0b9270 msgr2=0x7f151c0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.372+0000 7f1522400640 1 --2- 172.21.15.119:0/623778183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f151c0b9270 0x7f151c0d24d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1514009870 tx=0x7f151402a270 comp rx=0 tx=0).stop 2024-04-25T10:18:34.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.372+0000 7f1522400640 1 -- 172.21.15.119:0/623778183 shutdown_connections 2024-04-25T10:18:34.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.372+0000 7f1522400640 1 --2- 172.21.15.119:0/623778183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f151c0b9270 0x7f151c0d24d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.372+0000 7f1522400640 1 --2- 172.21.15.119:0/623778183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151c0d3f80 0x7f151c0d1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.372+0000 7f1522400640 1 -- 172.21.15.119:0/623778183 >> 172.21.15.119:0/623778183 conn(0x7f151c0726c0 msgr2=0x7f151c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:34.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.372+0000 7f1522400640 1 -- 172.21.15.119:0/623778183 shutdown_connections 2024-04-25T10:18:34.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.372+0000 7f1522400640 1 -- 172.21.15.119:0/623778183 wait complete. 2024-04-25T10:18:34.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.374+0000 7f1522400640 1 Processor -- start 2024-04-25T10:18:34.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.374+0000 7f1522400640 1 -- start start 2024-04-25T10:18:34.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.375+0000 7f1522400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f151c0b9270 0x7f151c1f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.375+0000 7f1522400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151c0d3f80 0x7f151c0c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.375+0000 7f1522400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f151c15db60 0x7f151c1e6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.375+0000 7f1522400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0b3120 con 0x7f151c15db60 2024-04-25T10:18:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.375+0000 7f1522400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0b2fa0 con 0x7f151c0b9270 2024-04-25T10:18:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.375+0000 7f1522400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c15df40 con 0x7f151c0d3f80 2024-04-25T10:18:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.375+0000 7f1521a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f151c15db60 0x7f151c1e6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.375+0000 7f1521a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f151c15db60 0x7f151c1e6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41022/0 (socket says 172.21.15.119:41022) 2024-04-25T10:18:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.375+0000 7f1521a00640 1 -- 172.21.15.119:0/1405979335 learned_addr learned my addr 172.21.15.119:0/1405979335 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.376+0000 7f1521000640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f151c0b9270 0x7f151c1f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.376+0000 7f151be00640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151c0d3f80 0x7f151c0c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.376+0000 7f1521a00640 1 -- 172.21.15.119:0/1405979335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151c0d3f80 msgr2=0x7f151c0c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.376+0000 7f1521a00640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151c0d3f80 0x7f151c0c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.376+0000 7f1521a00640 1 -- 172.21.15.119:0/1405979335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f151c0b9270 msgr2=0x7f151c1f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.376+0000 7f1521a00640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f151c0b9270 0x7f151c1f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.376+0000 7f1521a00640 1 -- 172.21.15.119:0/1405979335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c1f7770 con 0x7f151c15db60 2024-04-25T10:18:34.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.376+0000 7f1521000640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f151c0b9270 0x7f151c1f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:34.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.376+0000 7f151be00640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151c0d3f80 0x7f151c0c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:34.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.376+0000 7f1521a00640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f151c15db60 0x7f151c1e6010 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f151402a2b0 tx=0x7f15140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.377+0000 7f1518c00640 1 -- 172.21.15.119:0/1405979335 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514041020 con 0x7f151c15db60 2024-04-25T10:18:34.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.377+0000 7f1518c00640 1 -- 172.21.15.119:0/1405979335 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151403d640 con 0x7f151c15db60 2024-04-25T10:18:34.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.377+0000 7f1518c00640 1 -- 172.21.15.119:0/1405979335 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514037040 con 0x7f151c15db60 2024-04-25T10:18:34.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.377+0000 7f1522400640 1 -- 172.21.15.119:0/1405979335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c0b61e0 con 0x7f151c15db60 2024-04-25T10:18:34.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.377+0000 7f1522400640 1 -- 172.21.15.119:0/1405979335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151c0b3340 con 0x7f151c15db60 2024-04-25T10:18:34.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.378+0000 7f1522400640 1 -- 172.21.15.119:0/1405979335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151c0749b0 con 0x7f151c15db60 2024-04-25T10:18:34.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.379+0000 7f1518c00640 1 -- 172.21.15.119:0/1405979335 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1514044020 con 0x7f151c15db60 2024-04-25T10:18:34.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.380+0000 7f1518c00640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f14f806b4d0 0x7f14f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.380+0000 7f1518c00640 1 -- 172.21.15.119:0/1405979335 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f15140aa750 con 0x7f151c15db60 2024-04-25T10:18:34.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.381+0000 7f1521000640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f14f806b4d0 0x7f14f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.384 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.381+0000 7f1521000640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f14f806b4d0 0x7f14f806d990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1504000f60 tx=0x7f1504009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.385+0000 7f1518c00640 1 -- 172.21.15.119:0/1405979335 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f151404b030 con 0x7f151c15db60 2024-04-25T10:18:34.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.575+0000 7f1522400640 1 -- 172.21.15.119:0/1405979335 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f151c06ae10 con 0x7f14f806b4d0 2024-04-25T10:18:34.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.578+0000 7f1518c00640 1 -- 172.21.15.119:0/1405979335 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13522 (secure 0 0 0) 0x7f151c06ae10 con 0x7f14f806b4d0 2024-04-25T10:18:34.581 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:34.581 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:18:34.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.580+0000 7f1522400640 1 -- 172.21.15.119:0/1405979335 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f14f806b4d0 msgr2=0x7f14f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.580+0000 7f1522400640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f14f806b4d0 0x7f14f806d990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1504000f60 tx=0x7f1504009040 comp rx=0 tx=0).stop 2024-04-25T10:18:34.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.580+0000 7f1522400640 1 -- 172.21.15.119:0/1405979335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f151c15db60 msgr2=0x7f151c1e6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.580+0000 7f1522400640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f151c15db60 0x7f151c1e6010 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f151402a2b0 tx=0x7f15140046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.580+0000 7f1522400640 1 -- 172.21.15.119:0/1405979335 shutdown_connections 2024-04-25T10:18:34.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.580+0000 7f1522400640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f14f806b4d0 0x7f14f806d990 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.580+0000 7f1522400640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f151c15db60 0x7f151c1e6010 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.580+0000 7f1522400640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f151c0d3f80 0x7f151c0c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.580+0000 7f1522400640 1 --2- 172.21.15.119:0/1405979335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f151c0b9270 0x7f151c1f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.580+0000 7f1522400640 1 -- 172.21.15.119:0/1405979335 >> 172.21.15.119:0/1405979335 conn(0x7f151c0726c0 msgr2=0x7f151c0c6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:34.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.580+0000 7f1522400640 1 -- 172.21.15.119:0/1405979335 shutdown_connections 2024-04-25T10:18:34.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.580+0000 7f1522400640 1 -- 172.21.15.119:0/1405979335 wait complete. 2024-04-25T10:18:34.601 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-04-25T10:18:34.159326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":278092,"kb_used_data":28648,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754696628,"statfs":{"total":773094113280,"available":772809347072,"internally_reserved":0,"allocated":29335552,"data_stored":22884631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392254,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-179,"num_write_kb":-8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.635623"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":347,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T10:18:29.915494+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:18:29.915494+0000","last_peered":"2024-04-25T10:18:29.915494+0000","last_clean":"2024-04-25T10:18:29.915494+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:18:29.915494+0000","last_undegraded":"2024-04-25T10:18:29.915494+0000","last_fullsized":"2024-04-25T10:18:29.915494+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574990,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2011136,"data_stored":1203234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1417216,"data_stored":612327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32572,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1425408,"data_stored":618483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":618478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574990,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1425408,"data_stored":618483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574991,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36664,"kb_used_data":5484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":5615616,"data_stored":4812782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40752,"kb_used_data":9572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331088,"statfs":{"total":96636764160,"available":96595034112,"internally_reserved":0,"allocated":9801728,"data_stored":9000977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574990,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37252,"kb_used_data":6072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334588,"statfs":{"total":96636764160,"available":96598618112,"internally_reserved":0,"allocated":6217728,"data_stored":5399867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:18:34.602 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:34.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.798+0000 7fa90bc00640 1 Processor -- start 2024-04-25T10:18:34.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.799+0000 7fa90bc00640 1 -- start start 2024-04-25T10:18:34.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.800+0000 7fa90bc00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa90415d980 0x7fa9040ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.800+0000 7fa90bc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa9040aeb20 0x7fa9040cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.800+0000 7fa90bc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9040cfbd0 con 0x7fa9040cf7c0 2024-04-25T10:18:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.800+0000 7fa90bc00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa904071ed0 con 0x7fa90415d980 2024-04-25T10:18:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.800+0000 7fa90bc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9040712c0 con 0x7fa9040aeb20 2024-04-25T10:18:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.801+0000 7fa909200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa90415d980 0x7fa9040ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.801+0000 7fa909200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa90415d980 0x7fa9040ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41036/0 (socket says 172.21.15.119:41036) 2024-04-25T10:18:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.801+0000 7fa909200640 1 -- 172.21.15.119:0/1511989693 learned_addr learned my addr 172.21.15.119:0/1511989693 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:34.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.801+0000 7fa8fbe00640 1 --2- 172.21.15.119:0/1511989693 >> v2:172.21.15.132:3300/0 conn(0x7fa9040aeb20 0x7fa9040cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.801+0000 7fa909200640 1 -- 172.21.15.119:0/1511989693 >> v2:172.21.15.132:3300/0 conn(0x7fa9040aeb20 msgr2=0x7fa9040cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:34.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.801+0000 7fa909200640 1 --2- 172.21.15.119:0/1511989693 >> v2:172.21.15.132:3300/0 conn(0x7fa9040aeb20 0x7fa9040cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.801+0000 7fa909200640 1 -- 172.21.15.119:0/1511989693 >> v1:172.21.15.132:6789/0 conn(0x7fa9040cf7c0 legacy=0x7fa90415dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:34.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.801+0000 7fa909200640 1 -- 172.21.15.119:0/1511989693 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040715a0 con 0x7fa90415d980 2024-04-25T10:18:34.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.801+0000 7fa8fbe00640 1 --2- 172.21.15.119:0/1511989693 >> v2:172.21.15.132:3300/0 conn(0x7fa9040aeb20 0x7fa9040cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:34.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.802+0000 7fa909200640 1 --2- 172.21.15.119:0/1511989693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d980 0x7fa9040ae5e0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fa904067460 tx=0x7fa8f402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b53d73b2d99684 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.802+0000 7fa903e00640 1 -- 172.21.15.119:0/1511989693 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4037070 con 0x7fa90415d980 2024-04-25T10:18:34.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.803+0000 7fa903e00640 1 -- 172.21.15.119:0/1511989693 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f4032440 con 0x7fa90415d980 2024-04-25T10:18:34.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.803+0000 7fa903e00640 1 -- 172.21.15.119:0/1511989693 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4032810 con 0x7fa90415d980 2024-04-25T10:18:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.803+0000 7fa90bc00640 1 -- 172.21.15.119:0/1511989693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d980 msgr2=0x7fa9040ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.803+0000 7fa90bc00640 1 --2- 172.21.15.119:0/1511989693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d980 0x7fa9040ae5e0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fa904067460 tx=0x7fa8f402a180 comp rx=0 tx=0).stop 2024-04-25T10:18:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.803+0000 7fa90bc00640 1 -- 172.21.15.119:0/1511989693 shutdown_connections 2024-04-25T10:18:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.803+0000 7fa90bc00640 1 --2- 172.21.15.119:0/1511989693 >> v2:172.21.15.132:3300/0 conn(0x7fa9040aeb20 0x7fa9040cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.803+0000 7fa90bc00640 1 --2- 172.21.15.119:0/1511989693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d980 0x7fa9040ae5e0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.803+0000 7fa90bc00640 1 -- 172.21.15.119:0/1511989693 >> 172.21.15.119:0/1511989693 conn(0x7fa90406aba0 msgr2=0x7fa90406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.803+0000 7fa90bc00640 1 -- 172.21.15.119:0/1511989693 shutdown_connections 2024-04-25T10:18:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.803+0000 7fa90bc00640 1 -- 172.21.15.119:0/1511989693 wait complete. 2024-04-25T10:18:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.805+0000 7fa90bc00640 1 Processor -- start 2024-04-25T10:18:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.805+0000 7fa90bc00640 1 -- start start 2024-04-25T10:18:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.806+0000 7fa90bc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9040aeb20 0x7fa90414e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.806+0000 7fa90bc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9040cf7c0 0x7fa9040ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.806+0000 7fa90bc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa90415d980 0x7fa904144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.806+0000 7fa90bc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040adf80 con 0x7fa9040cf7c0 2024-04-25T10:18:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.806+0000 7fa90bc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa904071b60 con 0x7fa9040aeb20 2024-04-25T10:18:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.806+0000 7fa90bc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040ae100 con 0x7fa90415d980 2024-04-25T10:18:34.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.806+0000 7fa8fbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9040cf7c0 0x7fa9040ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.806+0000 7fa8fbe00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9040cf7c0 0x7fa9040ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41052/0 (socket says 172.21.15.119:41052) 2024-04-25T10:18:34.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.806+0000 7fa8fbe00640 1 -- 172.21.15.119:0/1194773725 learned_addr learned my addr 172.21.15.119:0/1194773725 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:34.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.807+0000 7fa8fbe00640 1 -- 172.21.15.119:0/1194773725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa90415d980 msgr2=0x7fa904144850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.807+0000 7fa909c00640 1 --2- 172.21.15.119:0/1194773725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa90415d980 0x7fa904144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.807+0000 7fa909200640 1 --2- 172.21.15.119:0/1194773725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9040aeb20 0x7fa90414e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.807+0000 7fa8fbe00640 1 --2- 172.21.15.119:0/1194773725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa90415d980 0x7fa904144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.807+0000 7fa8fbe00640 1 -- 172.21.15.119:0/1194773725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9040aeb20 msgr2=0x7fa90414e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.807+0000 7fa8fbe00640 1 --2- 172.21.15.119:0/1194773725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9040aeb20 0x7fa90414e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.807+0000 7fa8fbe00640 1 -- 172.21.15.119:0/1194773725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040cdaa0 con 0x7fa9040cf7c0 2024-04-25T10:18:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.807+0000 7fa8fbe00640 1 --2- 172.21.15.119:0/1194773725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9040cf7c0 0x7fa9040ce540 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00c7f0 tx=0x7fa8ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.807+0000 7fa901600640 1 -- 172.21.15.119:0/1194773725 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec015070 con 0x7fa9040cf7c0 2024-04-25T10:18:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.807+0000 7fa901600640 1 -- 172.21.15.119:0/1194773725 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ec007820 con 0x7fa9040cf7c0 2024-04-25T10:18:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.807+0000 7fa901600640 1 -- 172.21.15.119:0/1194773725 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec007bd0 con 0x7fa9040cf7c0 2024-04-25T10:18:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.807+0000 7fa90bc00640 1 -- 172.21.15.119:0/1194773725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90415e6e0 con 0x7fa9040cf7c0 2024-04-25T10:18:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.807+0000 7fa90bc00640 1 -- 172.21.15.119:0/1194773725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9040cdd40 con 0x7fa9040cf7c0 2024-04-25T10:18:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.808+0000 7fa90bc00640 1 -- 172.21.15.119:0/1194773725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9040af550 con 0x7fa9040cf7c0 2024-04-25T10:18:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.809+0000 7fa901600640 1 -- 172.21.15.119:0/1194773725 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8ec018070 con 0x7fa9040cf7c0 2024-04-25T10:18:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.810+0000 7fa901600640 1 --2- 172.21.15.119:0/1194773725 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8e006b4d0 0x7fa8e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.810+0000 7fa901600640 1 -- 172.21.15.119:0/1194773725 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa8ec08a130 con 0x7fa9040cf7c0 2024-04-25T10:18:34.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.811+0000 7fa909200640 1 --2- 172.21.15.119:0/1194773725 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8e006b4d0 0x7fa8e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.811+0000 7fa909200640 1 --2- 172.21.15.119:0/1194773725 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8e006b4d0 0x7fa8e006d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4034000 tx=0x7fa8f402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:34.815+0000 7fa901600640 1 -- 172.21.15.119:0/1194773725 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9040af550 con 0x7fa9040cf7c0 2024-04-25T10:18:35.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:35.001+0000 7fa90bc00640 1 -- 172.21.15.119:0/1194773725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9040749b0 con 0x7fa9040cf7c0 2024-04-25T10:18:35.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:35.003+0000 7fa901600640 1 -- 172.21.15.119:0/1194773725 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+13332 (secure 0 0 0) 0x7fa8ec0591f0 con 0x7fa9040cf7c0 2024-04-25T10:18:35.006 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:35.007 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":115,"fsid":"e0b0f428-b441-4237-a4d1-ed53321f87a5","created":"2024-04-25T10:06:42.868534+0000","modified":"2024-04-25T10:18:30.929747+0000","last_up_change":"2024-04-25T10:06:55.253814+0000","last_in_change":"2024-04-25T10:06:46.230936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:06:55.994187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6587d006-498c-4076-8aa1-6f385b1a5002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6809","nonce":2890625333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6811","nonce":2890625333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6815","nonce":2890625333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2890625333},{"type":"v1","addr":"172.21.15.119:6813","nonce":2890625333}]},"public_addr":"172.21.15.119:6809/2890625333","cluster_addr":"172.21.15.119:6811/2890625333","heartbeat_back_addr":"172.21.15.119:6815/2890625333","heartbeat_front_addr":"172.21.15.119:6813/2890625333","state":["exists","up"]},{"osd":1,"uuid":"7744b5a3-61e9-4e73-9764-ef4e11325b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6801","nonce":298545098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6803","nonce":298545098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6807","nonce":298545098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":298545098},{"type":"v1","addr":"172.21.15.119:6805","nonce":298545098}]},"public_addr":"172.21.15.119:6801/298545098","cluster_addr":"172.21.15.119:6803/298545098","heartbeat_back_addr":"172.21.15.119:6807/298545098","heartbeat_front_addr":"172.21.15.119:6805/298545098","state":["exists","up"]},{"osd":2,"uuid":"edb818b9-6266-4e52-88bf-d3659485dc44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6825","nonce":1001762963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6827","nonce":1001762963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6831","nonce":1001762963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1001762963},{"type":"v1","addr":"172.21.15.119:6829","nonce":1001762963}]},"public_addr":"172.21.15.119:6825/1001762963","cluster_addr":"172.21.15.119:6827/1001762963","heartbeat_back_addr":"172.21.15.119:6831/1001762963","heartbeat_front_addr":"172.21.15.119:6829/1001762963","state":["exists","up"]},{"osd":3,"uuid":"2febff65-e13e-435b-8556-8b0b3399b875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6817","nonce":1142971786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6819","nonce":1142971786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6823","nonce":1142971786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":1142971786},{"type":"v1","addr":"172.21.15.119:6821","nonce":1142971786}]},"public_addr":"172.21.15.119:6817/1142971786","cluster_addr":"172.21.15.119:6819/1142971786","heartbeat_back_addr":"172.21.15.119:6823/1142971786","heartbeat_front_addr":"172.21.15.119:6821/1142971786","state":["exists","up"]},{"osd":4,"uuid":"36485de8-b602-4be0-80ae-e309a8f26352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6801","nonce":1740646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6803","nonce":1740646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6807","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6809","nonce":1740646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1740646323},{"type":"v1","addr":"172.21.15.132:6805","nonce":1740646323}]},"public_addr":"172.21.15.132:6801/1740646323","cluster_addr":"172.21.15.132:6803/1740646323","heartbeat_back_addr":"172.21.15.132:6809/1740646323","heartbeat_front_addr":"172.21.15.132:6805/1740646323","state":["exists","up"]},{"osd":5,"uuid":"5c53c0a2-393c-4d32-8d7d-6ad7a0ac2fa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6815","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6817","nonce":2978152964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6819","nonce":2978152964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6827","nonce":2978152964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2978152964},{"type":"v1","addr":"172.21.15.132:6822","nonce":2978152964}]},"public_addr":"172.21.15.132:6817/2978152964","cluster_addr":"172.21.15.132:6819/2978152964","heartbeat_back_addr":"172.21.15.132:6827/2978152964","heartbeat_front_addr":"172.21.15.132:6822/2978152964","state":["exists","up"]},{"osd":6,"uuid":"c275df55-3aca-4426-8c41-79da30ff0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6808","nonce":353653608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6811","nonce":353653608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6816","nonce":353653608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":353653608},{"type":"v1","addr":"172.21.15.132:6813","nonce":353653608}]},"public_addr":"172.21.15.132:6808/353653608","cluster_addr":"172.21.15.132:6811/353653608","heartbeat_back_addr":"172.21.15.132:6816/353653608","heartbeat_front_addr":"172.21.15.132:6813/353653608","state":["exists","up"]},{"osd":7,"uuid":"37d1ddce-fbb1-498b-aece-09e4831bd340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6823","nonce":2727385616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6825","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6826","nonce":2727385616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6831","nonce":2727385616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2727385616},{"type":"v1","addr":"172.21.15.132:6829","nonce":2727385616}]},"public_addr":"172.21.15.132:6823/2727385616","cluster_addr":"172.21.15.132:6826/2727385616","heartbeat_back_addr":"172.21.15.132:6831/2727385616","heartbeat_front_addr":"172.21.15.132:6829/2727385616","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.598428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.597927+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.632897+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.599428+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.784352+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.612620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.583439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:06:51.909543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/2618740513":"2024-04-26T10:18:27.519554+0000","172.21.15.119:6836/2618740513":"2024-04-26T10:18:27.519554+0000","172.21.15.119:6837/2864575636":"2024-04-26T10:18:09.666363+0000","172.21.15.119:6836/2864575636":"2024-04-26T10:18:09.666363+0000","172.21.15.119:6837/1849584205":"2024-04-26T10:17:56.438859+0000","172.21.15.119:6836/1849584205":"2024-04-26T10:17:56.438859+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:35.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:35.005+0000 7fa90bc00640 1 -- 172.21.15.119:0/1194773725 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8e006b4d0 msgr2=0x7fa8e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:35.005+0000 7fa90bc00640 1 --2- 172.21.15.119:0/1194773725 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8e006b4d0 0x7fa8e006d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4034000 tx=0x7fa8f402e000 comp rx=0 tx=0).stop 2024-04-25T10:18:35.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:35.005+0000 7fa90bc00640 1 -- 172.21.15.119:0/1194773725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9040cf7c0 msgr2=0x7fa9040ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:35.005+0000 7fa90bc00640 1 --2- 172.21.15.119:0/1194773725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9040cf7c0 0x7fa9040ce540 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00c7f0 tx=0x7fa8ec00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:35.005+0000 7fa90bc00640 1 -- 172.21.15.119:0/1194773725 shutdown_connections 2024-04-25T10:18:35.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:35.005+0000 7fa90bc00640 1 --2- 172.21.15.119:0/1194773725 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa8e006b4d0 0x7fa8e006d990 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:35.005+0000 7fa90bc00640 1 --2- 172.21.15.119:0/1194773725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa90415d980 0x7fa904144850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:35.005+0000 7fa90bc00640 1 --2- 172.21.15.119:0/1194773725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9040cf7c0 0x7fa9040ce540 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:35.005+0000 7fa90bc00640 1 --2- 172.21.15.119:0/1194773725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9040aeb20 0x7fa90414e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:35.005+0000 7fa90bc00640 1 -- 172.21.15.119:0/1194773725 >> 172.21.15.119:0/1194773725 conn(0x7fa90406aba0 msgr2=0x7fa90406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:35.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:35.005+0000 7fa90bc00640 1 -- 172.21.15.119:0/1194773725 shutdown_connections 2024-04-25T10:18:35.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:35.005+0000 7fa90bc00640 1 -- 172.21.15.119:0/1194773725 wait complete. 2024-04-25T10:18:36.026 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T10:18:36.026 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:18:36.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.234+0000 7f663cc00640 1 Processor -- start 2024-04-25T10:18:36.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.235+0000 7f663cc00640 1 -- start start 2024-04-25T10:18:36.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.236+0000 7f663cc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f66380b9720 0x7f66380ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.236+0000 7f663cc00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66380cf0f0 con 0x7f6638161090 2024-04-25T10:18:36.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.236+0000 7f663cc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6638161f80 con 0x7f66380cece0 2024-04-25T10:18:36.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.236+0000 7f663cc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66380d21a0 con 0x7f66380b9720 2024-04-25T10:18:36.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.236+0000 7f6635e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6638161090 0x7f6638161730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36214/0 (socket says 172.21.15.119:36214) 2024-04-25T10:18:36.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.236+0000 7f6635e00640 1 -- 172.21.15.119:0/260559588 learned_addr learned my addr 172.21.15.119:0/260559588 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.237+0000 7f6635400640 1 --2- 172.21.15.119:0/260559588 >> v2:172.21.15.132:3300/0 conn(0x7f66380b9720 0x7f66380ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.237+0000 7f6634a00640 1 -- 172.21.15.119:0/260559588 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900625689 0 0) 0x7f6628001110 con 0x7f6638161090 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.237+0000 7f6634a00640 1 -- 172.21.15.119:0/260559588 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f661c003520 con 0x7f6638161090 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.237+0000 7f6634a00640 1 -- 172.21.15.119:0/260559588 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 719284895 0 0) 0x7f661c003520 con 0x7f6638161090 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.237+0000 7f6634a00640 1 -- 172.21.15.119:0/260559588 >> v2:172.21.15.132:3300/0 conn(0x7f66380b9720 msgr2=0x7f66380ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.237+0000 7f6634a00640 1 --2- 172.21.15.119:0/260559588 >> v2:172.21.15.132:3300/0 conn(0x7f66380b9720 0x7f66380ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.237+0000 7f6634a00640 1 -- 172.21.15.119:0/260559588 >> v1:172.21.15.132:6789/0 conn(0x7f66380cece0 legacy=0x7f66380d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.237+0000 7f6634a00640 1 -- 172.21.15.119:0/260559588 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6638162260 con 0x7f6638161090 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.237+0000 7f6634a00640 1 -- 172.21.15.119:0/260559588 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f6628001840 con 0x7f6638161090 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.237+0000 7f6634a00640 1 -- 172.21.15.119:0/260559588 >> v1:172.21.15.119:6789/0 conn(0x7f6638161090 legacy=0x7f6638161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.237+0000 7f6634a00640 1 --2- 172.21.15.119:0/260559588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f661c0039b0 0x7f661c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.238+0000 7f6634a00640 1 --2- 172.21.15.119:0/260559588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f661c004800 0x7f661c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.238+0000 7f6634a00640 1 --2- 172.21.15.119:0/260559588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f661c005210 0x7f661c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.238+0000 7f6634a00640 1 -- 172.21.15.119:0/260559588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66380d21a0 con 0x7f661c005210 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.238+0000 7f6634a00640 1 -- 172.21.15.119:0/260559588 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c00c140 con 0x7f661c0039b0 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.238+0000 7f6634a00640 1 -- 172.21.15.119:0/260559588 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661c00c340 con 0x7f661c004800 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.238+0000 7f6636800640 1 --2- 172.21.15.119:0/260559588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f661c005210 0x7f661c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.238+0000 7f6635e00640 1 --2- 172.21.15.119:0/260559588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f661c0039b0 0x7f661c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.238+0000 7f6635400640 1 --2- 172.21.15.119:0/260559588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f661c004800 0x7f661c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.238+0000 7f6636800640 1 -- 172.21.15.119:0/260559588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f661c004800 msgr2=0x7f661c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.238+0000 7f6636800640 1 --2- 172.21.15.119:0/260559588 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f661c004800 0x7f661c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.238+0000 7f6636800640 1 -- 172.21.15.119:0/260559588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f661c0039b0 msgr2=0x7f661c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.238+0000 7f6636800640 1 --2- 172.21.15.119:0/260559588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f661c0039b0 0x7f661c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.238+0000 7f6636800640 1 -- 172.21.15.119:0/260559588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c003520 con 0x7f661c005210 2024-04-25T10:18:36.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.239+0000 7f6636800640 1 -- 172.21.15.119:0/260559588 reap_dead start 2024-04-25T10:18:36.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.239+0000 7f6636800640 1 --2- 172.21.15.119:0/260559588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f661c005210 0x7f661c00b9c0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f6630001750 tx=0x7f661c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb884d28d512ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.240+0000 7f6634a00640 1 -- 172.21.15.119:0/260559588 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630001370 con 0x7f661c005210 2024-04-25T10:18:36.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.240+0000 7f6634a00640 1 -- 172.21.15.119:0/260559588 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6630005310 con 0x7f661c005210 2024-04-25T10:18:36.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.240+0000 7f6634a00640 1 -- 172.21.15.119:0/260559588 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663002abe0 con 0x7f661c005210 2024-04-25T10:18:36.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.240+0000 7f663cc00640 1 -- 172.21.15.119:0/260559588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f661c005210 msgr2=0x7f661c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.240+0000 7f663cc00640 1 --2- 172.21.15.119:0/260559588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f661c005210 0x7f661c00b9c0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f6630001750 tx=0x7f661c005830 comp rx=0 tx=0).stop 2024-04-25T10:18:36.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.240+0000 7f663cc00640 1 -- 172.21.15.119:0/260559588 shutdown_connections 2024-04-25T10:18:36.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.240+0000 7f663cc00640 1 --2- 172.21.15.119:0/260559588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f661c005210 0x7f661c00b9c0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.240+0000 7f663cc00640 1 -- 172.21.15.119:0/260559588 >> 172.21.15.119:0/260559588 conn(0x7f66380b55e0 msgr2=0x7f66380b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.240+0000 7f663cc00640 1 -- 172.21.15.119:0/260559588 shutdown_connections 2024-04-25T10:18:36.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.240+0000 7f663cc00640 1 -- 172.21.15.119:0/260559588 wait complete. 2024-04-25T10:18:36.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.242+0000 7f663cc00640 1 Processor -- start 2024-04-25T10:18:36.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.242+0000 7f663cc00640 1 -- start start 2024-04-25T10:18:36.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.243+0000 7f663cc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f661c005210 0x7f66380cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.243+0000 7f663cc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66381522a0 0x7f6638159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.243+0000 7f663cc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f663815b310 0x7f663815b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.243+0000 7f663cc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6638161470 con 0x7f661c005210 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.243+0000 7f663cc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66380d1e30 con 0x7f66381522a0 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.243+0000 7f663cc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66380ce7a0 con 0x7f663815b310 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.243+0000 7f6635e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f661c005210 0x7f66380cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.243+0000 7f6635e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f661c005210 0x7f66380cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41068/0 (socket says 172.21.15.119:41068) 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.243+0000 7f6635e00640 1 -- 172.21.15.119:0/444281327 learned_addr learned my addr 172.21.15.119:0/444281327 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.244+0000 7f6635e00640 1 -- 172.21.15.119:0/444281327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f663815b310 msgr2=0x7f663815b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.244+0000 7f6635e00640 1 --2- 172.21.15.119:0/444281327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f663815b310 0x7f663815b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.244+0000 7f6635e00640 1 -- 172.21.15.119:0/444281327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66381522a0 msgr2=0x7f6638159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.244+0000 7f6635e00640 1 --2- 172.21.15.119:0/444281327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66381522a0 0x7f6638159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.244+0000 7f6635e00640 1 -- 172.21.15.119:0/444281327 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663815bd00 con 0x7f661c005210 2024-04-25T10:18:36.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.244+0000 7f6635e00640 1 --2- 172.21.15.119:0/444281327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f661c005210 0x7f66380cd240 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f662800e780 tx=0x7f662800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.244+0000 7f662e000640 1 -- 172.21.15.119:0/444281327 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662800c660 con 0x7f661c005210 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.244+0000 7f662e000640 1 -- 172.21.15.119:0/444281327 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6628018070 con 0x7f661c005210 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.244+0000 7f662e000640 1 -- 172.21.15.119:0/444281327 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662800ca40 con 0x7f661c005210 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.244+0000 7f663cc00640 1 -- 172.21.15.119:0/444281327 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663815bf90 con 0x7f661c005210 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.244+0000 7f663cc00640 1 -- 172.21.15.119:0/444281327 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663815c550 con 0x7f661c005210 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.245+0000 7f663cc00640 1 -- 172.21.15.119:0/444281327 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f66380750b0 con 0x7f661c005210 2024-04-25T10:18:36.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.246+0000 7f662e000640 1 -- 172.21.15.119:0/444281327 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f662800cbe0 con 0x7f661c005210 2024-04-25T10:18:36.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.247+0000 7f662e000640 1 --2- 172.21.15.119:0/444281327 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f661806b500 0x7f661806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.247+0000 7f662e000640 1 -- 172.21.15.119:0/444281327 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f6628089a60 con 0x7f661c005210 2024-04-25T10:18:36.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.248+0000 7f6635400640 1 --2- 172.21.15.119:0/444281327 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f661806b500 0x7f661806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.248+0000 7f662e000640 1 --2- 172.21.15.119:0/444281327 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x7f6618071500 0x7f6618073920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.248+0000 7f662e000640 1 -- 172.21.15.119:0/444281327 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6618073fd0 con 0x7f6618071500 2024-04-25T10:18:36.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.248+0000 7f662e000640 1 -- 172.21.15.119:0/444281327 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f662808e2a0 con 0x7f661c005210 2024-04-25T10:18:36.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.248+0000 7f6636800640 1 --2- 172.21.15.119:0/444281327 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x7f6618071500 0x7f6618073920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.248+0000 7f6635400640 1 --2- 172.21.15.119:0/444281327 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f661806b500 0x7f661806d9c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6638152680 tx=0x7f6624009750 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.248+0000 7f6636800640 1 --2- 172.21.15.119:0/444281327 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x7f6618071500 0x7f6618073920 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.249+0000 7f662e000640 1 -- 172.21.15.119:0/444281327 <== osd.0 v2:172.21.15.119:6808/2890625333 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6618073fd0 con 0x7f6618071500 2024-04-25T10:18:36.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.263+0000 7f663cc00640 1 -- 172.21.15.119:0/444281327 --> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f66380d1700 con 0x7f6618071500 2024-04-25T10:18:36.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.264+0000 7f662e000640 1 -- 172.21.15.119:0/444281327 <== osd.0 v2:172.21.15.119:6808/2890625333 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f66380d1700 con 0x7f6618071500 2024-04-25T10:18:36.267 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:18:36.267 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:18:36.267 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:18:36.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.264+0000 7f663cc00640 1 -- 172.21.15.119:0/444281327 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x7f6618071500 msgr2=0x7f6618073920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.264+0000 7f663cc00640 1 --2- 172.21.15.119:0/444281327 >> [v2:172.21.15.119:6808/2890625333,v1:172.21.15.119:6809/2890625333] conn(0x7f6618071500 0x7f6618073920 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.265+0000 7f663cc00640 1 -- 172.21.15.119:0/444281327 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f661806b500 msgr2=0x7f661806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.265+0000 7f663cc00640 1 --2- 172.21.15.119:0/444281327 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f661806b500 0x7f661806d9c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6638152680 tx=0x7f6624009750 comp rx=0 tx=0).stop 2024-04-25T10:18:36.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.265+0000 7f663cc00640 1 -- 172.21.15.119:0/444281327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f661c005210 msgr2=0x7f66380cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.265+0000 7f663cc00640 1 --2- 172.21.15.119:0/444281327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f661c005210 0x7f66380cd240 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f662800e780 tx=0x7f662800ec40 comp rx=0 tx=0).stop 2024-04-25T10:18:36.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.265+0000 7f6636800640 1 -- 172.21.15.119:0/444281327 reap_dead start 2024-04-25T10:18:36.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.265+0000 7f663cc00640 1 -- 172.21.15.119:0/444281327 shutdown_connections 2024-04-25T10:18:36.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.265+0000 7f663cc00640 1 -- 172.21.15.119:0/444281327 >> 172.21.15.119:0/444281327 conn(0x7f66380b55e0 msgr2=0x7f66380b8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.265+0000 7f663cc00640 1 -- 172.21.15.119:0/444281327 shutdown_connections 2024-04-25T10:18:36.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.265+0000 7f663cc00640 1 -- 172.21.15.119:0/444281327 wait complete. 2024-04-25T10:18:36.287 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T10:18:36.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.487+0000 7fcd2ac00640 1 Processor -- start 2024-04-25T10:18:36.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.487+0000 7fcd2ac00640 1 -- start start 2024-04-25T10:18:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.489+0000 7fcd2ac00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd240b36f0 0x7fcd24163540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.489+0000 7fcd2ac00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd240b3b40 con 0x7fcd240b3240 2024-04-25T10:18:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.489+0000 7fcd2ac00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd2406e3f0 con 0x7fcd24071fc0 2024-04-25T10:18:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.489+0000 7fcd2ac00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2406e6d0 con 0x7fcd240b36f0 2024-04-25T10:18:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.489+0000 7fcd23400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcd240b3240 0x7fcd24162e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36218/0 (socket says 172.21.15.119:36218) 2024-04-25T10:18:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.489+0000 7fcd23400640 1 -- 172.21.15.119:0/997492597 learned_addr learned my addr 172.21.15.119:0/997492597 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.489+0000 7fcd28c00640 1 --2- 172.21.15.119:0/997492597 >> v2:172.21.15.119:3300/0 conn(0x7fcd240b36f0 0x7fcd24163540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.490+0000 7fcd28c00640 1 -- 172.21.15.119:0/997492597 >> v1:172.21.15.132:6789/0 conn(0x7fcd24071fc0 legacy=0x7fcd2406d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.490+0000 7fcd28c00640 1 -- 172.21.15.119:0/997492597 >> v1:172.21.15.119:6789/0 conn(0x7fcd240b3240 legacy=0x7fcd24162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.490+0000 7fcd28c00640 1 -- 172.21.15.119:0/997492597 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2406e9b0 con 0x7fcd240b36f0 2024-04-25T10:18:36.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.491+0000 7fcd28c00640 1 --2- 172.21.15.119:0/997492597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd240b36f0 0x7fcd24163540 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009d90 tx=0x7fcd1c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9bda1851a8a610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.491+0000 7fcd22a00640 1 -- 172.21.15.119:0/997492597 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c007740 con 0x7fcd240b36f0 2024-04-25T10:18:36.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.491+0000 7fcd22a00640 1 -- 172.21.15.119:0/997492597 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c0078e0 con 0x7fcd240b36f0 2024-04-25T10:18:36.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.491+0000 7fcd22a00640 1 -- 172.21.15.119:0/997492597 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c007cb0 con 0x7fcd240b36f0 2024-04-25T10:18:36.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.492+0000 7fcd2ac00640 1 -- 172.21.15.119:0/997492597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd240b36f0 msgr2=0x7fcd24163540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.492+0000 7fcd2ac00640 1 --2- 172.21.15.119:0/997492597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd240b36f0 0x7fcd24163540 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009d90 tx=0x7fcd1c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.492+0000 7fcd2ac00640 1 -- 172.21.15.119:0/997492597 shutdown_connections 2024-04-25T10:18:36.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.492+0000 7fcd2ac00640 1 --2- 172.21.15.119:0/997492597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd240b36f0 0x7fcd24163540 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.492+0000 7fcd2ac00640 1 -- 172.21.15.119:0/997492597 >> 172.21.15.119:0/997492597 conn(0x7fcd2406aba0 msgr2=0x7fcd2406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.492+0000 7fcd2ac00640 1 -- 172.21.15.119:0/997492597 shutdown_connections 2024-04-25T10:18:36.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.492+0000 7fcd2ac00640 1 -- 172.21.15.119:0/997492597 wait complete. 2024-04-25T10:18:36.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.493+0000 7fcd2ac00640 1 Processor -- start 2024-04-25T10:18:36.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.494+0000 7fcd2ac00640 1 -- start start 2024-04-25T10:18:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.494+0000 7fcd2ac00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd24071fc0 0x7fcd24149a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd2ac00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd240b3240 0x7fcd24164450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd2ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd240b36f0 0x7fcd24164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd2ac00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2406e0b0 con 0x7fcd240b3240 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd2ac00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd240723a0 con 0x7fcd24071fc0 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd2ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd24162590 con 0x7fcd240b36f0 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd23400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd240b3240 0x7fcd24164450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd23400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd240b3240 0x7fcd24164450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41094/0 (socket says 172.21.15.119:41094) 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd23400640 1 -- 172.21.15.119:0/1309349775 learned_addr learned my addr 172.21.15.119:0/1309349775 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd23e00640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd24071fc0 0x7fcd24149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd28c00640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd240b36f0 0x7fcd24164990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd23400640 1 -- 172.21.15.119:0/1309349775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd240b36f0 msgr2=0x7fcd24164990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd23400640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd240b36f0 0x7fcd24164990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd23400640 1 -- 172.21.15.119:0/1309349775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd24071fc0 msgr2=0x7fcd24149a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd23400640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd24071fc0 0x7fcd24149a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd23400640 1 -- 172.21.15.119:0/1309349775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd24149f70 con 0x7fcd240b3240 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.495+0000 7fcd23e00640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd24071fc0 0x7fcd24149a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.496+0000 7fcd23400640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd240b3240 0x7fcd24164450 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00c430 tx=0x7fcd0c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.496+0000 7fcd1be00640 1 -- 172.21.15.119:0/1309349775 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c017070 con 0x7fcd240b3240 2024-04-25T10:18:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.496+0000 7fcd1be00640 1 -- 172.21.15.119:0/1309349775 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c005ce0 con 0x7fcd240b3240 2024-04-25T10:18:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.496+0000 7fcd1be00640 1 -- 172.21.15.119:0/1309349775 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c0072f0 con 0x7fcd240b3240 2024-04-25T10:18:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.496+0000 7fcd2ac00640 1 -- 172.21.15.119:0/1309349775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2415bcc0 con 0x7fcd240b3240 2024-04-25T10:18:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.496+0000 7fcd2ac00640 1 -- 172.21.15.119:0/1309349775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2415c280 con 0x7fcd240b3240 2024-04-25T10:18:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.497+0000 7fcd2ac00640 1 -- 172.21.15.119:0/1309349775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd240cd840 con 0x7fcd240b3240 2024-04-25T10:18:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.498+0000 7fcd1be00640 1 -- 172.21.15.119:0/1309349775 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd0c01a070 con 0x7fcd240b3240 2024-04-25T10:18:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.499+0000 7fcd1be00640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd0006b4d0 0x7fcd0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.499+0000 7fcd1be00640 1 -- 172.21.15.119:0/1309349775 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fcd0c0893d0 con 0x7fcd240b3240 2024-04-25T10:18:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.499+0000 7fcd23e00640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd0006b4d0 0x7fcd0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.500+0000 7fcd23e00640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd0006b4d0 0x7fcd0006d990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcd14003eb0 tx=0x7fcd14009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.503+0000 7fcd1be00640 1 -- 172.21.15.119:0/1309349775 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd0c0584a0 con 0x7fcd240b3240 2024-04-25T10:18:36.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.685+0000 7fcd2ac00640 1 -- 172.21.15.119:0/1309349775 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd2415cff0 con 0x7fcd0006b4d0 2024-04-25T10:18:36.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.685+0000 7fcd1be00640 1 -- 172.21.15.119:0/1309349775 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcd14005030 con 0x7fcd0006b4d0 2024-04-25T10:18:36.689 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T10:18:36.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.688+0000 7fcd2ac00640 1 -- 172.21.15.119:0/1309349775 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd0006b4d0 msgr2=0x7fcd0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.688+0000 7fcd2ac00640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd0006b4d0 0x7fcd0006d990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcd14003eb0 tx=0x7fcd14009040 comp rx=0 tx=0).stop 2024-04-25T10:18:36.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.688+0000 7fcd2ac00640 1 -- 172.21.15.119:0/1309349775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd240b3240 msgr2=0x7fcd24164450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.688+0000 7fcd2ac00640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd240b3240 0x7fcd24164450 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00c430 tx=0x7fcd0c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.688+0000 7fcd2ac00640 1 -- 172.21.15.119:0/1309349775 shutdown_connections 2024-04-25T10:18:36.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.688+0000 7fcd2ac00640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fcd0006b4d0 0x7fcd0006d990 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.688+0000 7fcd2ac00640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcd240b36f0 0x7fcd24164990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.688+0000 7fcd2ac00640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd240b3240 0x7fcd24164450 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.688+0000 7fcd2ac00640 1 --2- 172.21.15.119:0/1309349775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd24071fc0 0x7fcd24149a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.688+0000 7fcd2ac00640 1 -- 172.21.15.119:0/1309349775 >> 172.21.15.119:0/1309349775 conn(0x7fcd2406aba0 msgr2=0x7fcd24162e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.689+0000 7fcd2ac00640 1 -- 172.21.15.119:0/1309349775 shutdown_connections 2024-04-25T10:18:36.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.689+0000 7fcd2ac00640 1 -- 172.21.15.119:0/1309349775 wait complete. 2024-04-25T10:18:36.711 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T10:18:36.712 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:18:36.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.907+0000 7f0ff8c00640 1 Processor -- start 2024-04-25T10:18:36.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.907+0000 7f0ff8c00640 1 -- start start 2024-04-25T10:18:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.909+0000 7f0ff8c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ff4144fd0 0x7f0ff415f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.909+0000 7f0ff8c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff415bdf0 con 0x7f0ff40cea40 2024-04-25T10:18:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.909+0000 7f0ff8c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff406d4e0 con 0x7f0ff40cf180 2024-04-25T10:18:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.909+0000 7f0ff8c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff406d7c0 con 0x7f0ff4144fd0 2024-04-25T10:18:36.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.909+0000 7f0ff3e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0ff40cea40 0x7f0ff415c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36220/0 (socket says 172.21.15.119:36220) 2024-04-25T10:18:36.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.909+0000 7f0ff3e00640 1 -- 172.21.15.119:0/2396756150 learned_addr learned my addr 172.21.15.119:0/2396756150 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:36.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.909+0000 7f0ff2a00640 1 -- 172.21.15.119:0/2396756150 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993470287 0 0) 0x7f0ff415bdf0 con 0x7f0ff40cea40 2024-04-25T10:18:36.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff2a00640 1 -- 172.21.15.119:0/2396756150 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd8003520 con 0x7f0ff40cea40 2024-04-25T10:18:36.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff3400640 1 --2- 172.21.15.119:0/2396756150 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ff4144fd0 0x7f0ff415f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff2a00640 1 -- 172.21.15.119:0/2396756150 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1834468734 0 0) 0x7f0fd8003520 con 0x7f0ff40cea40 2024-04-25T10:18:36.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff2a00640 1 -- 172.21.15.119:0/2396756150 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ff4144fd0 msgr2=0x7f0ff415f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff2a00640 1 --2- 172.21.15.119:0/2396756150 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ff4144fd0 0x7f0ff415f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff2a00640 1 -- 172.21.15.119:0/2396756150 >> v1:172.21.15.132:6789/0 conn(0x7f0ff40cf180 legacy=0x7f0ff40ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff2a00640 1 -- 172.21.15.119:0/2396756150 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff406daa0 con 0x7f0ff40cea40 2024-04-25T10:18:36.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff2a00640 1 -- 172.21.15.119:0/2396756150 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f0fec0019f0 con 0x7f0ff40cea40 2024-04-25T10:18:36.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff2a00640 1 -- 172.21.15.119:0/2396756150 >> v1:172.21.15.119:6789/0 conn(0x7f0ff40cea40 legacy=0x7f0ff415c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff2a00640 1 --2- 172.21.15.119:0/2396756150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd80039e0 0x7f0fd8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff2a00640 1 --2- 172.21.15.119:0/2396756150 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ff4144fd0 0x7f0fd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff2a00640 1 --2- 172.21.15.119:0/2396756150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fd8004d40 0x7f0fd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff2a00640 1 -- 172.21.15.119:0/2396756150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff406d7c0 con 0x7f0fd8004d40 2024-04-25T10:18:36.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff2a00640 1 -- 172.21.15.119:0/2396756150 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd800bbd0 con 0x7f0fd80039e0 2024-04-25T10:18:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff2a00640 1 -- 172.21.15.119:0/2396756150 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd800bdd0 con 0x7f0ff4144fd0 2024-04-25T10:18:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.910+0000 7f0ff3e00640 1 --2- 172.21.15.119:0/2396756150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fd8004d40 0x7f0fd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.911+0000 7f0ff3400640 1 --2- 172.21.15.119:0/2396756150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd80039e0 0x7f0fd8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.911+0000 7f0febe00640 1 --2- 172.21.15.119:0/2396756150 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ff4144fd0 0x7f0fd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.911+0000 7f0ff3e00640 1 -- 172.21.15.119:0/2396756150 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ff4144fd0 msgr2=0x7f0fd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.911+0000 7f0ff3e00640 1 --2- 172.21.15.119:0/2396756150 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ff4144fd0 0x7f0fd8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.911+0000 7f0ff3e00640 1 -- 172.21.15.119:0/2396756150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd80039e0 msgr2=0x7f0fd8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.911+0000 7f0ff3e00640 1 --2- 172.21.15.119:0/2396756150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd80039e0 0x7f0fd8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.911+0000 7f0ff3e00640 1 -- 172.21.15.119:0/2396756150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd8003520 con 0x7f0fd8004d40 2024-04-25T10:18:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.911+0000 7f0febe00640 1 --2- 172.21.15.119:0/2396756150 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ff4144fd0 0x7f0fd8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.911+0000 7f0ff3400640 1 --2- 172.21.15.119:0/2396756150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd80039e0 0x7f0fd8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:36.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.912+0000 7f0ff3e00640 1 --2- 172.21.15.119:0/2396756150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fd8004d40 0x7f0fd80051c0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00a760 tx=0x7f0fd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460f642920b53d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.912+0000 7f0ff2a00640 1 -- 172.21.15.119:0/2396756150 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec00a3b0 con 0x7f0fd8004d40 2024-04-25T10:18:36.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.912+0000 7f0ff2a00640 1 -- 172.21.15.119:0/2396756150 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fec02f070 con 0x7f0fd8004d40 2024-04-25T10:18:36.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.912+0000 7f0ff2a00640 1 -- 172.21.15.119:0/2396756150 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec02bba0 con 0x7f0fd8004d40 2024-04-25T10:18:36.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.913+0000 7f0ff8c00640 1 -- 172.21.15.119:0/2396756150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fd8004d40 msgr2=0x7f0fd80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.913+0000 7f0ff8c00640 1 --2- 172.21.15.119:0/2396756150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fd8004d40 0x7f0fd80051c0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00a760 tx=0x7f0fd8005830 comp rx=0 tx=0).stop 2024-04-25T10:18:36.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.913+0000 7f0ff3e00640 1 -- 172.21.15.119:0/2396756150 reap_dead start 2024-04-25T10:18:36.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.913+0000 7f0ff8c00640 1 -- 172.21.15.119:0/2396756150 shutdown_connections 2024-04-25T10:18:36.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.913+0000 7f0ff8c00640 1 -- 172.21.15.119:0/2396756150 >> 172.21.15.119:0/2396756150 conn(0x7f0ff406aba0 msgr2=0x7f0ff406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.913+0000 7f0ff8c00640 1 -- 172.21.15.119:0/2396756150 shutdown_connections 2024-04-25T10:18:36.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.913+0000 7f0ff8c00640 1 -- 172.21.15.119:0/2396756150 wait complete. 2024-04-25T10:18:36.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.914+0000 7f0ff8c00640 1 Processor -- start 2024-04-25T10:18:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.915+0000 7f0ff8c00640 1 -- start start 2024-04-25T10:18:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.915+0000 7f0ff8c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fd8004d40 0x7f0ff413adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.915+0000 7f0ff8c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ff40cd0b0 0x7f0ff413d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.915+0000 7f0ff8c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ff41f3570 0x7f0ff41f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.915+0000 7f0ff8c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41453b0 con 0x7f0fd8004d40 2024-04-25T10:18:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.915+0000 7f0ff8c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff406d190 con 0x7f0ff40cd0b0 2024-04-25T10:18:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.915+0000 7f0ff8c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff4145530 con 0x7f0ff41f3570 2024-04-25T10:18:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.916+0000 7f0ff3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fd8004d40 0x7f0ff413adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.916+0000 7f0ff3400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fd8004d40 0x7f0ff413adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41098/0 (socket says 172.21.15.119:41098) 2024-04-25T10:18:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.916+0000 7f0ff3400640 1 -- 172.21.15.119:0/3456584440 learned_addr learned my addr 172.21.15.119:0/3456584440 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.916+0000 7f0ff3400640 1 -- 172.21.15.119:0/3456584440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ff41f3570 msgr2=0x7f0ff41f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.916+0000 7f0ff3400640 1 --2- 172.21.15.119:0/3456584440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0ff41f3570 0x7f0ff41f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.916+0000 7f0febe00640 1 --2- 172.21.15.119:0/3456584440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ff40cd0b0 0x7f0ff413d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.916+0000 7f0ff3400640 1 -- 172.21.15.119:0/3456584440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ff40cd0b0 msgr2=0x7f0ff413d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.916+0000 7f0ff3400640 1 --2- 172.21.15.119:0/3456584440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ff40cd0b0 0x7f0ff413d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.916+0000 7f0ff3400640 1 -- 172.21.15.119:0/3456584440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff413b710 con 0x7f0fd8004d40 2024-04-25T10:18:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.916+0000 7f0ff3400640 1 --2- 172.21.15.119:0/3456584440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fd8004d40 0x7f0ff413adf0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400e790 tx=0x7f0fe400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.917+0000 7f0feb400640 1 -- 172.21.15.119:0/3456584440 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4017070 con 0x7f0fd8004d40 2024-04-25T10:18:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.917+0000 7f0feb400640 1 -- 172.21.15.119:0/3456584440 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe400c6d0 con 0x7f0fd8004d40 2024-04-25T10:18:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.917+0000 7f0feb400640 1 -- 172.21.15.119:0/3456584440 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe400f040 con 0x7f0fd8004d40 2024-04-25T10:18:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.917+0000 7f0ff8c00640 1 -- 172.21.15.119:0/3456584440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff413b9f0 con 0x7f0fd8004d40 2024-04-25T10:18:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.917+0000 7f0ff8c00640 1 -- 172.21.15.119:0/3456584440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff413bcd0 con 0x7f0fd8004d40 2024-04-25T10:18:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.918+0000 7f0ff8c00640 1 -- 172.21.15.119:0/3456584440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ff413dac0 con 0x7f0fd8004d40 2024-04-25T10:18:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.919+0000 7f0feb400640 1 -- 172.21.15.119:0/3456584440 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fe401a070 con 0x7f0fd8004d40 2024-04-25T10:18:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.920+0000 7f0feb400640 1 --2- 172.21.15.119:0/3456584440 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0fd006b500 0x7f0fd006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.920+0000 7f0feb400640 1 -- 172.21.15.119:0/3456584440 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f0fe4089a80 con 0x7f0fd8004d40 2024-04-25T10:18:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.920+0000 7f0febe00640 1 --2- 172.21.15.119:0/3456584440 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0fd006b500 0x7f0fd006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.920+0000 7f0feb400640 1 --2- 172.21.15.119:0/3456584440 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x7f0fd0071500 0x7f0fd0073920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.921+0000 7f0feb400640 1 -- 172.21.15.119:0/3456584440 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fe4007e50 con 0x7f0fd0071500 2024-04-25T10:18:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.921+0000 7f0feb400640 1 -- 172.21.15.119:0/3456584440 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fe408e2a0 con 0x7f0fd8004d40 2024-04-25T10:18:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.921+0000 7f0ff3e00640 1 --2- 172.21.15.119:0/3456584440 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x7f0fd0071500 0x7f0fd0073920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.921+0000 7f0febe00640 1 --2- 172.21.15.119:0/3456584440 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0fd006b500 0x7f0fd006d9c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc007fd0 tx=0x7f0fdc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.921+0000 7f0ff3e00640 1 --2- 172.21.15.119:0/3456584440 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x7f0fd0071500 0x7f0fd0073920 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.922+0000 7f0feb400640 1 -- 172.21.15.119:0/3456584440 <== osd.1 v2:172.21.15.119:6800/298545098 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0fe4007e50 con 0x7f0fd0071500 2024-04-25T10:18:36.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.936+0000 7f0ff8c00640 1 -- 172.21.15.119:0/3456584440 --> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0ff41f4190 con 0x7f0fd0071500 2024-04-25T10:18:36.939 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.936+0000 7f0feb400640 1 -- 172.21.15.119:0/3456584440 <== osd.1 v2:172.21.15.119:6800/298545098 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0ff41f4190 con 0x7f0fd0071500 2024-04-25T10:18:36.940 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:18:36.940 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:18:36.940 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:18:36.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.937+0000 7f0ff8c00640 1 -- 172.21.15.119:0/3456584440 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x7f0fd0071500 msgr2=0x7f0fd0073920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.937+0000 7f0ff8c00640 1 --2- 172.21.15.119:0/3456584440 >> [v2:172.21.15.119:6800/298545098,v1:172.21.15.119:6801/298545098] conn(0x7f0fd0071500 0x7f0fd0073920 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.937+0000 7f0ff8c00640 1 -- 172.21.15.119:0/3456584440 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0fd006b500 msgr2=0x7f0fd006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.937+0000 7f0ff8c00640 1 --2- 172.21.15.119:0/3456584440 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f0fd006b500 0x7f0fd006d9c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc007fd0 tx=0x7f0fdc009040 comp rx=0 tx=0).stop 2024-04-25T10:18:36.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.937+0000 7f0ff8c00640 1 -- 172.21.15.119:0/3456584440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fd8004d40 msgr2=0x7f0ff413adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.937+0000 7f0ff8c00640 1 --2- 172.21.15.119:0/3456584440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fd8004d40 0x7f0ff413adf0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400e790 tx=0x7f0fe400ec50 comp rx=0 tx=0).stop 2024-04-25T10:18:36.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.937+0000 7f0ff3e00640 1 -- 172.21.15.119:0/3456584440 reap_dead start 2024-04-25T10:18:36.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.937+0000 7f0ff8c00640 1 -- 172.21.15.119:0/3456584440 shutdown_connections 2024-04-25T10:18:36.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.937+0000 7f0ff8c00640 1 -- 172.21.15.119:0/3456584440 >> 172.21.15.119:0/3456584440 conn(0x7f0ff406aba0 msgr2=0x7f0ff415c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.937+0000 7f0ff8c00640 1 -- 172.21.15.119:0/3456584440 shutdown_connections 2024-04-25T10:18:36.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:36.937+0000 7f0ff8c00640 1 -- 172.21.15.119:0/3456584440 wait complete. 2024-04-25T10:18:36.955 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T10:18:37.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.137+0000 7f6a85e00640 1 Processor -- start 2024-04-25T10:18:37.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.138+0000 7f6a85e00640 1 -- start start 2024-04-25T10:18:37.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.139+0000 7f6a85e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a8015b7e0 0x7f6a8015e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.139+0000 7f6a85e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a800b5710 con 0x7f6a800ce860 2024-04-25T10:18:37.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.139+0000 7f6a85e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a800b59f0 con 0x7f6a800d07b0 2024-04-25T10:18:37.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.139+0000 7f6a85e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800b5cd0 con 0x7f6a8015b7e0 2024-04-25T10:18:37.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.139+0000 7f6a7fe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6a800ce860 0x7f6a800d00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36234/0 (socket says 172.21.15.119:36234) 2024-04-25T10:18:37.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.139+0000 7f6a7fe00640 1 -- 172.21.15.119:0/268815960 learned_addr learned my addr 172.21.15.119:0/268815960 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:37.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.139+0000 7f6a84a00640 1 --2- 172.21.15.119:0/268815960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a8015b7e0 0x7f6a8015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 -- 172.21.15.119:0/268815960 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911426664 0 0) 0x7f6a800b5710 con 0x7f6a800ce860 2024-04-25T10:18:37.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 -- 172.21.15.119:0/268815960 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a64003520 con 0x7f6a800ce860 2024-04-25T10:18:37.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 -- 172.21.15.119:0/268815960 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2493144488 0 0) 0x7f6a64003520 con 0x7f6a800ce860 2024-04-25T10:18:37.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 -- 172.21.15.119:0/268815960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a8015b7e0 msgr2=0x7f6a8015e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 --2- 172.21.15.119:0/268815960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a8015b7e0 0x7f6a8015e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 -- 172.21.15.119:0/268815960 >> v1:172.21.15.132:6789/0 conn(0x7f6a800d07b0 legacy=0x7f6a80161960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 -- 172.21.15.119:0/268815960 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a800b5fb0 con 0x7f6a800ce860 2024-04-25T10:18:37.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 -- 172.21.15.119:0/268815960 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f6a700019f0 con 0x7f6a800ce860 2024-04-25T10:18:37.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 -- 172.21.15.119:0/268815960 >> v1:172.21.15.119:6789/0 conn(0x7f6a800ce860 legacy=0x7f6a800d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 --2- 172.21.15.119:0/268815960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a8015b7e0 0x7f6a640039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 --2- 172.21.15.119:0/268815960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a64003f60 0x7f6a64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 --2- 172.21.15.119:0/268815960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a64004d40 0x7f6a640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 -- 172.21.15.119:0/268815960 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800b5cd0 con 0x7f6a64004d40 2024-04-25T10:18:37.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 -- 172.21.15.119:0/268815960 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6400bc00 con 0x7f6a64003f60 2024-04-25T10:18:37.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.140+0000 7f6a7f400640 1 -- 172.21.15.119:0/268815960 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6400be00 con 0x7f6a8015b7e0 2024-04-25T10:18:37.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.141+0000 7f6a85400640 1 --2- 172.21.15.119:0/268815960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a64004d40 0x7f6a640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.141+0000 7f6a84a00640 1 --2- 172.21.15.119:0/268815960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a8015b7e0 0x7f6a640039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.141+0000 7f6a7fe00640 1 --2- 172.21.15.119:0/268815960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a64003f60 0x7f6a64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.141+0000 7f6a85400640 1 -- 172.21.15.119:0/268815960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a8015b7e0 msgr2=0x7f6a640039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.141+0000 7f6a85400640 1 --2- 172.21.15.119:0/268815960 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a8015b7e0 0x7f6a640039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.141+0000 7f6a85400640 1 -- 172.21.15.119:0/268815960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a64003f60 msgr2=0x7f6a64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.141+0000 7f6a85400640 1 --2- 172.21.15.119:0/268815960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a64003f60 0x7f6a64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.141+0000 7f6a85400640 1 -- 172.21.15.119:0/268815960 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64003520 con 0x7f6a64004d40 2024-04-25T10:18:37.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.142+0000 7f6a85400640 1 --2- 172.21.15.119:0/268815960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a64004d40 0x7f6a640051c0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f6a78001720 tx=0x7f6a64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6306c0a494898adf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.142+0000 7f6a7f400640 1 -- 172.21.15.119:0/268815960 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78001370 con 0x7f6a64004d40 2024-04-25T10:18:37.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.142+0000 7f6a7f400640 1 -- 172.21.15.119:0/268815960 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a780052d0 con 0x7f6a64004d40 2024-04-25T10:18:37.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.142+0000 7f6a7f400640 1 -- 172.21.15.119:0/268815960 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7802abc0 con 0x7f6a64004d40 2024-04-25T10:18:37.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.143+0000 7f6a85e00640 1 -- 172.21.15.119:0/268815960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a64004d40 msgr2=0x7f6a640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.143+0000 7f6a85e00640 1 --2- 172.21.15.119:0/268815960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a64004d40 0x7f6a640051c0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f6a78001720 tx=0x7f6a64005830 comp rx=0 tx=0).stop 2024-04-25T10:18:37.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.143+0000 7f6a85400640 1 -- 172.21.15.119:0/268815960 reap_dead start 2024-04-25T10:18:37.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.143+0000 7f6a85e00640 1 -- 172.21.15.119:0/268815960 shutdown_connections 2024-04-25T10:18:37.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.143+0000 7f6a85e00640 1 -- 172.21.15.119:0/268815960 >> 172.21.15.119:0/268815960 conn(0x7f6a800722d0 msgr2=0x7f6a800726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.143+0000 7f6a85e00640 1 -- 172.21.15.119:0/268815960 shutdown_connections 2024-04-25T10:18:37.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.143+0000 7f6a85e00640 1 -- 172.21.15.119:0/268815960 wait complete. 2024-04-25T10:18:37.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.145+0000 7f6a85e00640 1 Processor -- start 2024-04-25T10:18:37.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.145+0000 7f6a85e00640 1 -- start start 2024-04-25T10:18:37.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.146+0000 7f6a85e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a64004d40 0x7f6a801f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.146+0000 7f6a85e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a800b7b50 0x7f6a800b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.146+0000 7f6a85e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a801f3880 0x7f6a801f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.146+0000 7f6a85e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800cee40 con 0x7f6a801f3880 2024-04-25T10:18:37.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.146+0000 7f6a85e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800cecc0 con 0x7f6a64004d40 2024-04-25T10:18:37.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.146+0000 7f6a85e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800cefc0 con 0x7f6a800b7b50 2024-04-25T10:18:37.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.146+0000 7f6a85400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a801f3880 0x7f6a801f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.146+0000 7f6a85400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a801f3880 0x7f6a801f3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41126/0 (socket says 172.21.15.119:41126) 2024-04-25T10:18:37.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.146+0000 7f6a85400640 1 -- 172.21.15.119:0/2816128117 learned_addr learned my addr 172.21.15.119:0/2816128117 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:37.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.147+0000 7f6a7fe00640 1 --2- 172.21.15.119:0/2816128117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a800b7b50 0x7f6a800b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.147+0000 7f6a84a00640 1 --2- 172.21.15.119:0/2816128117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a64004d40 0x7f6a801f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.147+0000 7f6a85400640 1 -- 172.21.15.119:0/2816128117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a800b7b50 msgr2=0x7f6a800b7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.147+0000 7f6a85400640 1 --2- 172.21.15.119:0/2816128117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a800b7b50 0x7f6a800b7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.147+0000 7f6a85400640 1 -- 172.21.15.119:0/2816128117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a64004d40 msgr2=0x7f6a801f0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.147+0000 7f6a85400640 1 --2- 172.21.15.119:0/2816128117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a64004d40 0x7f6a801f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.147+0000 7f6a85400640 1 -- 172.21.15.119:0/2816128117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a801f45e0 con 0x7f6a801f3880 2024-04-25T10:18:37.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.147+0000 7f6a85400640 1 --2- 172.21.15.119:0/2816128117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a801f3880 0x7f6a801f3d00 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f6a78001840 tx=0x7f6a78001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.147+0000 7f6a7cc00640 1 -- 172.21.15.119:0/2816128117 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78005480 con 0x7f6a801f3880 2024-04-25T10:18:37.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.147+0000 7f6a7cc00640 1 -- 172.21.15.119:0/2816128117 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a78041070 con 0x7f6a801f3880 2024-04-25T10:18:37.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.147+0000 7f6a7cc00640 1 -- 172.21.15.119:0/2816128117 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78007250 con 0x7f6a801f3880 2024-04-25T10:18:37.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.147+0000 7f6a85e00640 1 -- 172.21.15.119:0/2816128117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a801f7080 con 0x7f6a801f3880 2024-04-25T10:18:37.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.148+0000 7f6a85e00640 1 -- 172.21.15.119:0/2816128117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a801f1140 con 0x7f6a801f3880 2024-04-25T10:18:37.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.149+0000 7f6a85e00640 1 -- 172.21.15.119:0/2816128117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a801f6e70 con 0x7f6a801f3880 2024-04-25T10:18:37.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.150+0000 7f6a7cc00640 1 -- 172.21.15.119:0/2816128117 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a7803a070 con 0x7f6a801f3880 2024-04-25T10:18:37.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.151+0000 7f6a7cc00640 1 --2- 172.21.15.119:0/2816128117 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6a5c06b500 0x7f6a5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.151+0000 7f6a7cc00640 1 -- 172.21.15.119:0/2816128117 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f6a780a95a0 con 0x7f6a801f3880 2024-04-25T10:18:37.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.151+0000 7f6a84a00640 1 --2- 172.21.15.119:0/2816128117 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6a5c06b500 0x7f6a5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.152+0000 7f6a84a00640 1 --2- 172.21.15.119:0/2816128117 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6a5c06b500 0x7f6a5c06d9c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009950 tx=0x7f6a68008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.156+0000 7f6a7cc00640 1 -- 172.21.15.119:0/2816128117 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a780785d0 con 0x7f6a801f3880 2024-04-25T10:18:37.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.349+0000 7f6a85e00640 1 -- 172.21.15.119:0/2816128117 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a800749b0 con 0x7f6a5c06b500 2024-04-25T10:18:37.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.350+0000 7f6a7cc00640 1 -- 172.21.15.119:0/2816128117 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6a800749b0 con 0x7f6a5c06b500 2024-04-25T10:18:37.353 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T10:18:37.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.352+0000 7f6a85e00640 1 -- 172.21.15.119:0/2816128117 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6a5c06b500 msgr2=0x7f6a5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.352+0000 7f6a85e00640 1 --2- 172.21.15.119:0/2816128117 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6a5c06b500 0x7f6a5c06d9c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009950 tx=0x7f6a68008040 comp rx=0 tx=0).stop 2024-04-25T10:18:37.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.352+0000 7f6a85e00640 1 -- 172.21.15.119:0/2816128117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a801f3880 msgr2=0x7f6a801f3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.352+0000 7f6a85e00640 1 --2- 172.21.15.119:0/2816128117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a801f3880 0x7f6a801f3d00 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f6a78001840 tx=0x7f6a78001120 comp rx=0 tx=0).stop 2024-04-25T10:18:37.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.352+0000 7f6a85e00640 1 -- 172.21.15.119:0/2816128117 shutdown_connections 2024-04-25T10:18:37.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.352+0000 7f6a85e00640 1 --2- 172.21.15.119:0/2816128117 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f6a5c06b500 0x7f6a5c06d9c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.352+0000 7f6a85e00640 1 --2- 172.21.15.119:0/2816128117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a801f3880 0x7f6a801f3d00 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.352+0000 7f6a85e00640 1 --2- 172.21.15.119:0/2816128117 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6a800b7b50 0x7f6a800b7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.352+0000 7f6a85e00640 1 --2- 172.21.15.119:0/2816128117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a64004d40 0x7f6a801f0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.352+0000 7f6a85e00640 1 -- 172.21.15.119:0/2816128117 >> 172.21.15.119:0/2816128117 conn(0x7f6a800722d0 msgr2=0x7f6a80160070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.352+0000 7f6a85e00640 1 -- 172.21.15.119:0/2816128117 shutdown_connections 2024-04-25T10:18:37.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.352+0000 7f6a85e00640 1 -- 172.21.15.119:0/2816128117 wait complete. 2024-04-25T10:18:37.374 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T10:18:37.374 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:18:37.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.578+0000 7fab06400640 1 Processor -- start 2024-04-25T10:18:37.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.578+0000 7fab06400640 1 -- start start 2024-04-25T10:18:37.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.580+0000 7fab06400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab000ceeb0 0x7fab00168a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.580+0000 7fab06400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab00161f80 con 0x7fab000b9720 2024-04-25T10:18:37.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.580+0000 7fab06400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab00162260 con 0x7fab00161090 2024-04-25T10:18:37.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.580+0000 7fab06400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab000d21a0 con 0x7fab000ceeb0 2024-04-25T10:18:37.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.580+0000 7faaffe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab000ceeb0 0x7fab00168a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.580+0000 7faaffe00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab000ceeb0 0x7fab00168a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41134/0 (socket says 172.21.15.119:41134) 2024-04-25T10:18:37.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.580+0000 7faaffe00640 1 -- 172.21.15.119:0/2823678922 learned_addr learned my addr 172.21.15.119:0/2823678922 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:37.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.580+0000 7faafea00640 1 --1- 172.21.15.119:0/2823678922 >> v1:172.21.15.119:6789/0 conn(0x7fab000b9720 0x7fab000ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36250/0 (socket says 172.21.15.119:36250) 2024-04-25T10:18:37.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.581+0000 7faaffe00640 1 -- 172.21.15.119:0/2823678922 >> v1:172.21.15.132:6789/0 conn(0x7fab00161090 legacy=0x7fab00161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.581+0000 7faaffe00640 1 -- 172.21.15.119:0/2823678922 >> v1:172.21.15.119:6789/0 conn(0x7fab000b9720 legacy=0x7fab000ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.581+0000 7faaffe00640 1 -- 172.21.15.119:0/2823678922 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab00162540 con 0x7fab000ceeb0 2024-04-25T10:18:37.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.582+0000 7faaffe00640 1 --2- 172.21.15.119:0/2823678922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab000ceeb0 0x7fab00168a60 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fab000d2320 tx=0x7faaf802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbe414788681f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.582+0000 7faafe000640 1 -- 172.21.15.119:0/2823678922 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8037070 con 0x7fab000ceeb0 2024-04-25T10:18:37.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.582+0000 7faafe000640 1 -- 172.21.15.119:0/2823678922 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf8032440 con 0x7fab000ceeb0 2024-04-25T10:18:37.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.582+0000 7faafe000640 1 -- 172.21.15.119:0/2823678922 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8032810 con 0x7fab000ceeb0 2024-04-25T10:18:37.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.583+0000 7fab06400640 1 -- 172.21.15.119:0/2823678922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab000ceeb0 msgr2=0x7fab00168a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.583+0000 7fab06400640 1 --2- 172.21.15.119:0/2823678922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab000ceeb0 0x7fab00168a60 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fab000d2320 tx=0x7faaf802a250 comp rx=0 tx=0).stop 2024-04-25T10:18:37.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.583+0000 7fab06400640 1 -- 172.21.15.119:0/2823678922 shutdown_connections 2024-04-25T10:18:37.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.583+0000 7fab06400640 1 --2- 172.21.15.119:0/2823678922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab000ceeb0 0x7fab00168a60 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.583+0000 7fab06400640 1 -- 172.21.15.119:0/2823678922 >> 172.21.15.119:0/2823678922 conn(0x7fab000b55e0 msgr2=0x7fab000b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.583+0000 7fab06400640 1 -- 172.21.15.119:0/2823678922 shutdown_connections 2024-04-25T10:18:37.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.583+0000 7fab06400640 1 -- 172.21.15.119:0/2823678922 wait complete. 2024-04-25T10:18:37.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.585+0000 7fab06400640 1 Processor -- start 2024-04-25T10:18:37.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.585+0000 7fab06400640 1 -- start start 2024-04-25T10:18:37.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.586+0000 7fab06400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab000b9720 0x7fab000c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.586+0000 7fab06400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab000ceeb0 0x7fab0015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.586+0000 7fab06400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab00161090 0x7fab0015efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.586+0000 7fab06400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab000d3640 con 0x7fab000ceeb0 2024-04-25T10:18:37.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.586+0000 7fab06400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab00161470 con 0x7fab000b9720 2024-04-25T10:18:37.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.586+0000 7fab06400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab000d37c0 con 0x7fab00161090 2024-04-25T10:18:37.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.586+0000 7faafea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab000ceeb0 0x7fab0015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.587+0000 7faafea00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab000ceeb0 0x7fab0015ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41140/0 (socket says 172.21.15.119:41140) 2024-04-25T10:18:37.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.587+0000 7faafea00640 1 -- 172.21.15.119:0/2178859129 learned_addr learned my addr 172.21.15.119:0/2178859129 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:37.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.587+0000 7faaff400640 1 --2- 172.21.15.119:0/2178859129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab000b9720 0x7fab000c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.587+0000 7faafea00640 1 -- 172.21.15.119:0/2178859129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab00161090 msgr2=0x7fab0015efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.587+0000 7faaffe00640 1 --2- 172.21.15.119:0/2178859129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab00161090 0x7fab0015efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.587+0000 7faafea00640 1 --2- 172.21.15.119:0/2178859129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab00161090 0x7fab0015efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.587+0000 7faafea00640 1 -- 172.21.15.119:0/2178859129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab000b9720 msgr2=0x7fab000c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.587+0000 7faafea00640 1 --2- 172.21.15.119:0/2178859129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab000b9720 0x7fab000c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.587+0000 7faafea00640 1 -- 172.21.15.119:0/2178859129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab000cb6b0 con 0x7fab000ceeb0 2024-04-25T10:18:37.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.587+0000 7faafea00640 1 --2- 172.21.15.119:0/2178859129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab000ceeb0 0x7fab0015ea80 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7faaf000c7a0 tx=0x7faaf000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.587+0000 7faaf7400640 1 -- 172.21.15.119:0/2178859129 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0017070 con 0x7fab000ceeb0 2024-04-25T10:18:37.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.588+0000 7faaf7400640 1 -- 172.21.15.119:0/2178859129 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf000a640 con 0x7fab000ceeb0 2024-04-25T10:18:37.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.588+0000 7faaf7400640 1 -- 172.21.15.119:0/2178859129 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf000a9f0 con 0x7fab000ceeb0 2024-04-25T10:18:37.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.588+0000 7fab06400640 1 -- 172.21.15.119:0/2178859129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab000c85d0 con 0x7fab000ceeb0 2024-04-25T10:18:37.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.588+0000 7fab06400640 1 -- 172.21.15.119:0/2178859129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab000c8b90 con 0x7fab000ceeb0 2024-04-25T10:18:37.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.589+0000 7fab06400640 1 -- 172.21.15.119:0/2178859129 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab000750b0 con 0x7fab000ceeb0 2024-04-25T10:18:37.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.590+0000 7faaf7400640 1 -- 172.21.15.119:0/2178859129 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faaf001a070 con 0x7fab000ceeb0 2024-04-25T10:18:37.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.591+0000 7faaf7400640 1 --2- 172.21.15.119:0/2178859129 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faadc06b4d0 0x7faadc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.591+0000 7faaf7400640 1 -- 172.21.15.119:0/2178859129 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7faaf00894d0 con 0x7fab000ceeb0 2024-04-25T10:18:37.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.591+0000 7faaf7400640 1 --2- 172.21.15.119:0/2178859129 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x7faadc0714d0 0x7faadc0738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.591+0000 7faaff400640 1 --2- 172.21.15.119:0/2178859129 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faadc06b4d0 0x7faadc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.591+0000 7faaf7400640 1 -- 172.21.15.119:0/2178859129 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faadc073fa0 con 0x7faadc0714d0 2024-04-25T10:18:37.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.591+0000 7faaf7400640 1 -- 172.21.15.119:0/2178859129 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaf0090050 con 0x7fab000ceeb0 2024-04-25T10:18:37.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.591+0000 7faaffe00640 1 --2- 172.21.15.119:0/2178859129 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x7faadc0714d0 0x7faadc0738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.592+0000 7faaff400640 1 --2- 172.21.15.119:0/2178859129 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faadc06b4d0 0x7faadc06d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7faae8000f60 tx=0x7faae8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.592+0000 7faaffe00640 1 --2- 172.21.15.119:0/2178859129 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x7faadc0714d0 0x7faadc0738f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.593+0000 7faaf7400640 1 -- 172.21.15.119:0/2178859129 <== osd.2 v2:172.21.15.119:6824/1001762963 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7faadc073fa0 con 0x7faadc0714d0 2024-04-25T10:18:37.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.607+0000 7fab06400640 1 -- 172.21.15.119:0/2178859129 --> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fab000d1700 con 0x7faadc0714d0 2024-04-25T10:18:37.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.607+0000 7faaf7400640 1 -- 172.21.15.119:0/2178859129 <== osd.2 v2:172.21.15.119:6824/1001762963 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fab000d1700 con 0x7faadc0714d0 2024-04-25T10:18:37.611 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:18:37.611 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:18:37.611 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:18:37.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.608+0000 7fab06400640 1 -- 172.21.15.119:0/2178859129 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x7faadc0714d0 msgr2=0x7faadc0738f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.608+0000 7fab06400640 1 --2- 172.21.15.119:0/2178859129 >> [v2:172.21.15.119:6824/1001762963,v1:172.21.15.119:6825/1001762963] conn(0x7faadc0714d0 0x7faadc0738f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.608+0000 7fab06400640 1 -- 172.21.15.119:0/2178859129 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faadc06b4d0 msgr2=0x7faadc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.608+0000 7fab06400640 1 --2- 172.21.15.119:0/2178859129 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7faadc06b4d0 0x7faadc06d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7faae8000f60 tx=0x7faae8009040 comp rx=0 tx=0).stop 2024-04-25T10:18:37.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.608+0000 7fab06400640 1 -- 172.21.15.119:0/2178859129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab000ceeb0 msgr2=0x7fab0015ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.608+0000 7fab06400640 1 --2- 172.21.15.119:0/2178859129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab000ceeb0 0x7fab0015ea80 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7faaf000c7a0 tx=0x7faaf000cc60 comp rx=0 tx=0).stop 2024-04-25T10:18:37.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.608+0000 7faaffe00640 1 -- 172.21.15.119:0/2178859129 reap_dead start 2024-04-25T10:18:37.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.608+0000 7fab06400640 1 -- 172.21.15.119:0/2178859129 shutdown_connections 2024-04-25T10:18:37.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.608+0000 7fab06400640 1 -- 172.21.15.119:0/2178859129 >> 172.21.15.119:0/2178859129 conn(0x7fab000b55e0 msgr2=0x7fab001603a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.609+0000 7fab06400640 1 -- 172.21.15.119:0/2178859129 shutdown_connections 2024-04-25T10:18:37.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.609+0000 7fab06400640 1 -- 172.21.15.119:0/2178859129 wait complete. 2024-04-25T10:18:37.630 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T10:18:37.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.822+0000 7fe5e9600640 1 Processor -- start 2024-04-25T10:18:37.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.822+0000 7fe5e9600640 1 -- start start 2024-04-25T10:18:37.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.823+0000 7fe5e9600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe5e4145260 0x7fe5e40cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.824+0000 7fe5e9600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5e4071910 0x7fe5e40cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.824+0000 7fe5e9600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e40cc9a0 con 0x7fe5e40cc490 2024-04-25T10:18:37.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.824+0000 7fe5e9600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e40cea00 con 0x7fe5e4145260 2024-04-25T10:18:37.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.824+0000 7fe5e9600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40ccc80 con 0x7fe5e4071910 2024-04-25T10:18:37.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.824+0000 7fe5e2000640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe5e40cc490 0x7fe5e4071200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36264/0 (socket says 172.21.15.119:36264) 2024-04-25T10:18:37.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.824+0000 7fe5e2000640 1 -- 172.21.15.119:0/547615499 learned_addr learned my addr 172.21.15.119:0/547615499 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:37.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.824+0000 7fe5e3400640 1 --2- 172.21.15.119:0/547615499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5e4071910 0x7fe5e40cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.824+0000 7fe5e2a00640 1 --2- 172.21.15.119:0/547615499 >> v2:172.21.15.132:3300/0 conn(0x7fe5e4145260 0x7fe5e40cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.824+0000 7fe5e1600640 1 -- 172.21.15.119:0/547615499 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509906387 0 0) 0x7fe5e40cc9a0 con 0x7fe5e40cc490 2024-04-25T10:18:37.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.824+0000 7fe5e1600640 1 -- 172.21.15.119:0/547615499 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5c8003520 con 0x7fe5e40cc490 2024-04-25T10:18:37.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e1600640 1 -- 172.21.15.119:0/547615499 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3026872814 0 0) 0x7fe5c8003520 con 0x7fe5e40cc490 2024-04-25T10:18:37.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e1600640 1 -- 172.21.15.119:0/547615499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5e4071910 msgr2=0x7fe5e40cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e1600640 1 --2- 172.21.15.119:0/547615499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5e4071910 0x7fe5e40cee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e1600640 1 -- 172.21.15.119:0/547615499 >> v2:172.21.15.132:3300/0 conn(0x7fe5e4145260 msgr2=0x7fe5e40cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e1600640 1 --2- 172.21.15.119:0/547615499 >> v2:172.21.15.132:3300/0 conn(0x7fe5e4145260 0x7fe5e40cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e1600640 1 -- 172.21.15.119:0/547615499 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e40ccf60 con 0x7fe5e40cc490 2024-04-25T10:18:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e1600640 1 -- 172.21.15.119:0/547615499 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fe5d0001670 con 0x7fe5e40cc490 2024-04-25T10:18:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e1600640 1 -- 172.21.15.119:0/547615499 >> v1:172.21.15.119:6789/0 conn(0x7fe5e40cc490 legacy=0x7fe5e4071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e1600640 1 --2- 172.21.15.119:0/547615499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c80039b0 0x7fe5c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e1600640 1 --2- 172.21.15.119:0/547615499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5e4071910 0x7fe5c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e1600640 1 --2- 172.21.15.119:0/547615499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5c8004d40 0x7fe5c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e1600640 1 -- 172.21.15.119:0/547615499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40cea00 con 0x7fe5c80039b0 2024-04-25T10:18:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e1600640 1 -- 172.21.15.119:0/547615499 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40ccc80 con 0x7fe5c8004d40 2024-04-25T10:18:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e1600640 1 -- 172.21.15.119:0/547615499 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c800b9f0 con 0x7fe5e4071910 2024-04-25T10:18:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.825+0000 7fe5e2a00640 1 --2- 172.21.15.119:0/547615499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c80039b0 0x7fe5c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.826+0000 7fe5e3400640 1 --2- 172.21.15.119:0/547615499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5c8004d40 0x7fe5c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.826+0000 7fe5e2a00640 1 -- 172.21.15.119:0/547615499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5e4071910 msgr2=0x7fe5c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.826+0000 7fe5e2000640 1 --2- 172.21.15.119:0/547615499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5e4071910 0x7fe5c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.826+0000 7fe5e2a00640 1 --2- 172.21.15.119:0/547615499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5e4071910 0x7fe5c8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.826+0000 7fe5e2a00640 1 -- 172.21.15.119:0/547615499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5c8004d40 msgr2=0x7fe5c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.826+0000 7fe5e2a00640 1 --2- 172.21.15.119:0/547615499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5c8004d40 0x7fe5c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.826+0000 7fe5e2a00640 1 -- 172.21.15.119:0/547615499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c8003520 con 0x7fe5c80039b0 2024-04-25T10:18:37.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.826+0000 7fe5e3400640 1 --2- 172.21.15.119:0/547615499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5c8004d40 0x7fe5c80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:37.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.827+0000 7fe5e2a00640 1 --2- 172.21.15.119:0/547615499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c80039b0 0x7fe5c8003dd0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009ac0 tx=0x7fe5c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9e874c5b1315d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.827+0000 7fe5e1600640 1 -- 172.21.15.119:0/547615499 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4036070 con 0x7fe5c80039b0 2024-04-25T10:18:37.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.827+0000 7fe5e1600640 1 -- 172.21.15.119:0/547615499 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d4031400 con 0x7fe5c80039b0 2024-04-25T10:18:37.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.827+0000 7fe5e1600640 1 -- 172.21.15.119:0/547615499 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4009690 con 0x7fe5c80039b0 2024-04-25T10:18:37.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.828+0000 7fe5e9600640 1 -- 172.21.15.119:0/547615499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c80039b0 msgr2=0x7fe5c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.828+0000 7fe5e9600640 1 --2- 172.21.15.119:0/547615499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c80039b0 0x7fe5c8003dd0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009ac0 tx=0x7fe5c8005830 comp rx=0 tx=0).stop 2024-04-25T10:18:37.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.828+0000 7fe5e3400640 1 -- 172.21.15.119:0/547615499 reap_dead start 2024-04-25T10:18:37.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.828+0000 7fe5e9600640 1 -- 172.21.15.119:0/547615499 shutdown_connections 2024-04-25T10:18:37.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.828+0000 7fe5e9600640 1 -- 172.21.15.119:0/547615499 >> 172.21.15.119:0/547615499 conn(0x7fe5e406aba0 msgr2=0x7fe5e406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.828+0000 7fe5e9600640 1 -- 172.21.15.119:0/547615499 shutdown_connections 2024-04-25T10:18:37.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.828+0000 7fe5e9600640 1 -- 172.21.15.119:0/547615499 wait complete. 2024-04-25T10:18:37.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.830+0000 7fe5e9600640 1 Processor -- start 2024-04-25T10:18:37.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.830+0000 7fe5e9600640 1 -- start start 2024-04-25T10:18:37.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.831+0000 7fe5e9600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c80039b0 0x7fe5e41e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.831+0000 7fe5e9600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5e415b090 0x7fe5e415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.831+0000 7fe5e9600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5e4142e40 0x7fe5e41432a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.831+0000 7fe5e9600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e415c270 con 0x7fe5c80039b0 2024-04-25T10:18:37.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.831+0000 7fe5e9600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e4145640 con 0x7fe5e415b090 2024-04-25T10:18:37.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.831+0000 7fe5e9600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e415c3f0 con 0x7fe5e4142e40 2024-04-25T10:18:37.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.831+0000 7fe5e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c80039b0 0x7fe5e41e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.831+0000 7fe5e2a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c80039b0 0x7fe5e41e64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41150/0 (socket says 172.21.15.119:41150) 2024-04-25T10:18:37.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.831+0000 7fe5e2a00640 1 -- 172.21.15.119:0/58533892 learned_addr learned my addr 172.21.15.119:0/58533892 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:37.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.831+0000 7fe5e2a00640 1 -- 172.21.15.119:0/58533892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5e4142e40 msgr2=0x7fe5e41432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.831+0000 7fe5e2a00640 1 --2- 172.21.15.119:0/58533892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5e4142e40 0x7fe5e41432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.831+0000 7fe5e2a00640 1 -- 172.21.15.119:0/58533892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5e415b090 msgr2=0x7fe5e415b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.831+0000 7fe5e2a00640 1 --2- 172.21.15.119:0/58533892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5e415b090 0x7fe5e415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.831+0000 7fe5e2a00640 1 -- 172.21.15.119:0/58533892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e415ba50 con 0x7fe5c80039b0 2024-04-25T10:18:37.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.832+0000 7fe5e2a00640 1 --2- 172.21.15.119:0/58533892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c80039b0 0x7fe5e41e64e0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4034040 tx=0x7fe5d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.832+0000 7fe5daa00640 1 -- 172.21.15.119:0/58533892 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4040020 con 0x7fe5c80039b0 2024-04-25T10:18:37.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.832+0000 7fe5daa00640 1 -- 172.21.15.119:0/58533892 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d4003b60 con 0x7fe5c80039b0 2024-04-25T10:18:37.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.832+0000 7fe5e9600640 1 -- 172.21.15.119:0/58533892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e4143bc0 con 0x7fe5c80039b0 2024-04-25T10:18:37.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.832+0000 7fe5daa00640 1 -- 172.21.15.119:0/58533892 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4036040 con 0x7fe5c80039b0 2024-04-25T10:18:37.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.832+0000 7fe5e9600640 1 -- 172.21.15.119:0/58533892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e4144180 con 0x7fe5c80039b0 2024-04-25T10:18:37.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.833+0000 7fe5e9600640 1 -- 172.21.15.119:0/58533892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e40750b0 con 0x7fe5c80039b0 2024-04-25T10:18:37.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.835+0000 7fe5daa00640 1 -- 172.21.15.119:0/58533892 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5d4043020 con 0x7fe5c80039b0 2024-04-25T10:18:37.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.835+0000 7fe5daa00640 1 --2- 172.21.15.119:0/58533892 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe5c406b4d0 0x7fe5c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.836+0000 7fe5daa00640 1 -- 172.21.15.119:0/58533892 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fe5d40aa0c0 con 0x7fe5c80039b0 2024-04-25T10:18:37.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.836+0000 7fe5e2000640 1 --2- 172.21.15.119:0/58533892 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe5c406b4d0 0x7fe5c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.836+0000 7fe5e2000640 1 --2- 172.21.15.119:0/58533892 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe5c406b4d0 0x7fe5c406d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0000ff0 tx=0x7fe5d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:37.841+0000 7fe5daa00640 1 -- 172.21.15.119:0/58533892 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5d404a070 con 0x7fe5c80039b0 2024-04-25T10:18:38.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.037+0000 7fe5e9600640 1 -- 172.21.15.119:0/58533892 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5e40ad070 con 0x7fe5c406b4d0 2024-04-25T10:18:38.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.037+0000 7fe5daa00640 1 -- 172.21.15.119:0/58533892 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe5e40ad070 con 0x7fe5c406b4d0 2024-04-25T10:18:38.040 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T10:18:38.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.040+0000 7fe5e9600640 1 -- 172.21.15.119:0/58533892 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe5c406b4d0 msgr2=0x7fe5c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.040+0000 7fe5e9600640 1 --2- 172.21.15.119:0/58533892 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe5c406b4d0 0x7fe5c406d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0000ff0 tx=0x7fe5d0009040 comp rx=0 tx=0).stop 2024-04-25T10:18:38.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.040+0000 7fe5e9600640 1 -- 172.21.15.119:0/58533892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c80039b0 msgr2=0x7fe5e41e64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.040+0000 7fe5e9600640 1 --2- 172.21.15.119:0/58533892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c80039b0 0x7fe5e41e64e0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4034040 tx=0x7fe5d4004730 comp rx=0 tx=0).stop 2024-04-25T10:18:38.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.040+0000 7fe5e9600640 1 -- 172.21.15.119:0/58533892 shutdown_connections 2024-04-25T10:18:38.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.040+0000 7fe5e9600640 1 --2- 172.21.15.119:0/58533892 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe5c406b4d0 0x7fe5c406d990 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.040+0000 7fe5e9600640 1 --2- 172.21.15.119:0/58533892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe5e4142e40 0x7fe5e41432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.040+0000 7fe5e9600640 1 --2- 172.21.15.119:0/58533892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5e415b090 0x7fe5e415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.040+0000 7fe5e9600640 1 --2- 172.21.15.119:0/58533892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c80039b0 0x7fe5e41e64e0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.040+0000 7fe5e9600640 1 -- 172.21.15.119:0/58533892 >> 172.21.15.119:0/58533892 conn(0x7fe5e406aba0 msgr2=0x7fe5e40af360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.040+0000 7fe5e9600640 1 -- 172.21.15.119:0/58533892 shutdown_connections 2024-04-25T10:18:38.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.040+0000 7fe5e9600640 1 -- 172.21.15.119:0/58533892 wait complete. 2024-04-25T10:18:38.062 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T10:18:38.062 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:18:38.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.268+0000 7f96cb800640 1 Processor -- start 2024-04-25T10:18:38.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.268+0000 7f96cb800640 1 -- start start 2024-04-25T10:18:38.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.270+0000 7f96cb800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96c40aedc0 0x7f96c415ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.270+0000 7f96cb800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c406da40 con 0x7f96c40b4900 2024-04-25T10:18:38.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.270+0000 7f96cb800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c406dd20 con 0x7f96c406d260 2024-04-25T10:18:38.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.270+0000 7f96cb800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c415c050 con 0x7f96c40aedc0 2024-04-25T10:18:38.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.270+0000 7f96c8e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96c40aedc0 0x7f96c415ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.270+0000 7f96c8e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96c40aedc0 0x7f96c415ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41156/0 (socket says 172.21.15.119:41156) 2024-04-25T10:18:38.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.270+0000 7f96c8e00640 1 -- 172.21.15.119:0/1211915367 learned_addr learned my addr 172.21.15.119:0/1211915367 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:38.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.270+0000 7f96c9800640 1 --1- 172.21.15.119:0/1211915367 >> v1:172.21.15.119:6789/0 conn(0x7f96c40b4900 0x7f96c415c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36270/0 (socket says 172.21.15.119:36270) 2024-04-25T10:18:38.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.271+0000 7f96c8e00640 1 -- 172.21.15.119:0/1211915367 >> v1:172.21.15.132:6789/0 conn(0x7f96c406d260 legacy=0x7f96c40b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:38.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.271+0000 7f96c8e00640 1 -- 172.21.15.119:0/1211915367 >> v1:172.21.15.119:6789/0 conn(0x7f96c40b4900 legacy=0x7f96c415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.271+0000 7f96c8e00640 1 -- 172.21.15.119:0/1211915367 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c406e000 con 0x7f96c40aedc0 2024-04-25T10:18:38.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.271+0000 7f96c3400640 1 -- 172.21.15.119:0/1211915367 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979766292 0 0) 0x7f96c406da40 con 0x7f96c40b4900 2024-04-25T10:18:38.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.272+0000 7f96c8e00640 1 --2- 172.21.15.119:0/1211915367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c40aedc0 0x7f96c415ae20 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f96c415c1d0 tx=0x7f96b402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2999c50db6498c5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.273+0000 7f96c3400640 1 -- 172.21.15.119:0/1211915367 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4037070 con 0x7f96c40aedc0 2024-04-25T10:18:38.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.273+0000 7f96c3400640 1 -- 172.21.15.119:0/1211915367 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b4032440 con 0x7f96c40aedc0 2024-04-25T10:18:38.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.273+0000 7f96c3400640 1 -- 172.21.15.119:0/1211915367 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4032810 con 0x7f96c40aedc0 2024-04-25T10:18:38.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.273+0000 7f96cb800640 1 -- 172.21.15.119:0/1211915367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c40aedc0 msgr2=0x7f96c415ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.273+0000 7f96cb800640 1 --2- 172.21.15.119:0/1211915367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c40aedc0 0x7f96c415ae20 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f96c415c1d0 tx=0x7f96b402a210 comp rx=0 tx=0).stop 2024-04-25T10:18:38.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.273+0000 7f96cb800640 1 -- 172.21.15.119:0/1211915367 shutdown_connections 2024-04-25T10:18:38.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.273+0000 7f96cb800640 1 --2- 172.21.15.119:0/1211915367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c40aedc0 0x7f96c415ae20 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.274+0000 7f96cb800640 1 -- 172.21.15.119:0/1211915367 >> 172.21.15.119:0/1211915367 conn(0x7f96c406aba0 msgr2=0x7f96c406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.274+0000 7f96cb800640 1 -- 172.21.15.119:0/1211915367 shutdown_connections 2024-04-25T10:18:38.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.274+0000 7f96cb800640 1 -- 172.21.15.119:0/1211915367 wait complete. 2024-04-25T10:18:38.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.275+0000 7f96cb800640 1 Processor -- start 2024-04-25T10:18:38.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.275+0000 7f96cb800640 1 -- start start 2024-04-25T10:18:38.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.276+0000 7f96cb800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96c406d260 0x7f96c414b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.276+0000 7f96cb800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c40aedc0 0x7f96c414da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.276+0000 7f96cb800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96c40b4900 0x7f96c414df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.276+0000 7f96cb800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c406d6f0 con 0x7f96c40aedc0 2024-04-25T10:18:38.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.276+0000 7f96cb800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c40af1a0 con 0x7f96c406d260 2024-04-25T10:18:38.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.276+0000 7f96cb800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c415ced0 con 0x7f96c40b4900 2024-04-25T10:18:38.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.277+0000 7f96c3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c40aedc0 0x7f96c414da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.277+0000 7f96c3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c40aedc0 0x7f96c414da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41160/0 (socket says 172.21.15.119:41160) 2024-04-25T10:18:38.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.277+0000 7f96c3e00640 1 -- 172.21.15.119:0/1667891348 learned_addr learned my addr 172.21.15.119:0/1667891348 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:38.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.277+0000 7f96c8e00640 1 --2- 172.21.15.119:0/1667891348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96c406d260 0x7f96c414b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.277+0000 7f96c3e00640 1 -- 172.21.15.119:0/1667891348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96c40b4900 msgr2=0x7f96c414df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.277+0000 7f96c3e00640 1 --2- 172.21.15.119:0/1667891348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f96c40b4900 0x7f96c414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.277+0000 7f96c3e00640 1 -- 172.21.15.119:0/1667891348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96c406d260 msgr2=0x7f96c414b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.277+0000 7f96c3e00640 1 --2- 172.21.15.119:0/1667891348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96c406d260 0x7f96c414b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.277+0000 7f96c3e00640 1 -- 172.21.15.119:0/1667891348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c414c770 con 0x7f96c40aedc0 2024-04-25T10:18:38.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.277+0000 7f96c3e00640 1 --2- 172.21.15.119:0/1667891348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c40aedc0 0x7f96c414da20 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f96b000c430 tx=0x7f96b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.278+0000 7f96c0c00640 1 -- 172.21.15.119:0/1667891348 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0011790 con 0x7f96c40aedc0 2024-04-25T10:18:38.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.278+0000 7f96c0c00640 1 -- 172.21.15.119:0/1667891348 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b0017070 con 0x7f96c40aedc0 2024-04-25T10:18:38.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.278+0000 7f96cb800640 1 -- 172.21.15.119:0/1667891348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c414ca50 con 0x7f96c40aedc0 2024-04-25T10:18:38.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.278+0000 7f96c0c00640 1 -- 172.21.15.119:0/1667891348 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0011b70 con 0x7f96c40aedc0 2024-04-25T10:18:38.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.278+0000 7f96cb800640 1 -- 172.21.15.119:0/1667891348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c414d010 con 0x7f96c40aedc0 2024-04-25T10:18:38.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.279+0000 7f96cb800640 1 -- 172.21.15.119:0/1667891348 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f96c40750b0 con 0x7f96c40aedc0 2024-04-25T10:18:38.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.280+0000 7f96c0c00640 1 -- 172.21.15.119:0/1667891348 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96b0011d10 con 0x7f96c40aedc0 2024-04-25T10:18:38.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.281+0000 7f96c0c00640 1 --2- 172.21.15.119:0/1667891348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f96a406b4d0 0x7f96a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.281+0000 7f96c0c00640 1 -- 172.21.15.119:0/1667891348 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f96b0089e50 con 0x7f96c40aedc0 2024-04-25T10:18:38.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.281+0000 7f96c8e00640 1 --2- 172.21.15.119:0/1667891348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f96a406b4d0 0x7f96a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.281+0000 7f96c0c00640 1 --2- 172.21.15.119:0/1667891348 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x7f96a40714d0 0x7f96a40738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.281+0000 7f96c0c00640 1 -- 172.21.15.119:0/1667891348 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f96b0020ec0 con 0x7f96a40714d0 2024-04-25T10:18:38.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.281+0000 7f96c0c00640 1 -- 172.21.15.119:0/1667891348 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96b008c470 con 0x7f96c40aedc0 2024-04-25T10:18:38.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.282+0000 7f96c9800640 1 --2- 172.21.15.119:0/1667891348 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x7f96a40714d0 0x7f96a40738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.282+0000 7f96c8e00640 1 --2- 172.21.15.119:0/1667891348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f96a406b4d0 0x7f96a406d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f96b4034000 tx=0x7f96b402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.282+0000 7f96c9800640 1 --2- 172.21.15.119:0/1667891348 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x7f96a40714d0 0x7f96a40738f0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.283+0000 7f96c0c00640 1 -- 172.21.15.119:0/1667891348 <== osd.3 v2:172.21.15.119:6816/1142971786 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f96b0020ec0 con 0x7f96a40714d0 2024-04-25T10:18:38.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.297+0000 7f96cb800640 1 -- 172.21.15.119:0/1667891348 --> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f96c415db00 con 0x7f96a40714d0 2024-04-25T10:18:38.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.298+0000 7f96c0c00640 1 -- 172.21.15.119:0/1667891348 <== osd.3 v2:172.21.15.119:6816/1142971786 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f96c415db00 con 0x7f96a40714d0 2024-04-25T10:18:38.301 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:18:38.301 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:18:38.301 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:18:38.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.298+0000 7f96cb800640 1 -- 172.21.15.119:0/1667891348 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x7f96a40714d0 msgr2=0x7f96a40738f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.298+0000 7f96cb800640 1 --2- 172.21.15.119:0/1667891348 >> [v2:172.21.15.119:6816/1142971786,v1:172.21.15.119:6817/1142971786] conn(0x7f96a40714d0 0x7f96a40738f0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.298+0000 7f96cb800640 1 -- 172.21.15.119:0/1667891348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f96a406b4d0 msgr2=0x7f96a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.298+0000 7f96cb800640 1 --2- 172.21.15.119:0/1667891348 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f96a406b4d0 0x7f96a406d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f96b4034000 tx=0x7f96b402e000 comp rx=0 tx=0).stop 2024-04-25T10:18:38.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.298+0000 7f96cb800640 1 -- 172.21.15.119:0/1667891348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c40aedc0 msgr2=0x7f96c414da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.298+0000 7f96cb800640 1 --2- 172.21.15.119:0/1667891348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96c40aedc0 0x7f96c414da20 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f96b000c430 tx=0x7f96b000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.298+0000 7f96c9800640 1 -- 172.21.15.119:0/1667891348 reap_dead start 2024-04-25T10:18:38.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.298+0000 7f96cb800640 1 -- 172.21.15.119:0/1667891348 shutdown_connections 2024-04-25T10:18:38.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.298+0000 7f96cb800640 1 -- 172.21.15.119:0/1667891348 >> 172.21.15.119:0/1667891348 conn(0x7f96c406aba0 msgr2=0x7f96c40af320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.298+0000 7f96cb800640 1 -- 172.21.15.119:0/1667891348 shutdown_connections 2024-04-25T10:18:38.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.299+0000 7f96cb800640 1 -- 172.21.15.119:0/1667891348 wait complete. 2024-04-25T10:18:38.318 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T10:18:38.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.516+0000 7f682b000640 1 Processor -- start 2024-04-25T10:18:38.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.516+0000 7f682b000640 1 -- start start 2024-04-25T10:18:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.517+0000 7f682b000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6824071200 0x7f6824071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.517+0000 7f682b000640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68240cc9a0 con 0x7f6824145260 2024-04-25T10:18:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.517+0000 7f682b000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68240ccc80 con 0x7f68240cee90 2024-04-25T10:18:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.517+0000 7f682b000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6824071ba0 con 0x7f6824071200 2024-04-25T10:18:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.518+0000 7f6823e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6824145260 0x7f68240cbf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36276/0 (socket says 172.21.15.119:36276) 2024-04-25T10:18:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.518+0000 7f6823e00640 1 -- 172.21.15.119:0/608542700 learned_addr learned my addr 172.21.15.119:0/608542700 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.518+0000 7f681b400640 1 --2- 172.21.15.119:0/608542700 >> v2:172.21.15.119:3300/0 conn(0x7f6824071200 0x7f6824071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.518+0000 7f6823400640 1 -- 172.21.15.119:0/608542700 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140363080 0 0) 0x7f68240cc9a0 con 0x7f6824145260 2024-04-25T10:18:38.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.518+0000 7f6823400640 1 -- 172.21.15.119:0/608542700 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6808003520 con 0x7f6824145260 2024-04-25T10:18:38.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.518+0000 7f681b400640 1 -- 172.21.15.119:0/608542700 >> v1:172.21.15.132:6789/0 conn(0x7f68240cee90 legacy=0x7f68240cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:38.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.518+0000 7f681b400640 1 -- 172.21.15.119:0/608542700 >> v1:172.21.15.119:6789/0 conn(0x7f6824145260 legacy=0x7f68240cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.518+0000 7f681b400640 1 -- 172.21.15.119:0/608542700 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68240ccf60 con 0x7f6824071200 2024-04-25T10:18:38.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.519+0000 7f681b400640 1 --2- 172.21.15.119:0/608542700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824071200 0x7f6824071660 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f680c009d90 tx=0x7f680c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc45554e86bdaca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.520+0000 7f6823400640 1 -- 172.21.15.119:0/608542700 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c007740 con 0x7f6824071200 2024-04-25T10:18:38.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.520+0000 7f6823400640 1 -- 172.21.15.119:0/608542700 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680c0078e0 con 0x7f6824071200 2024-04-25T10:18:38.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.520+0000 7f6823400640 1 -- 172.21.15.119:0/608542700 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c007cb0 con 0x7f6824071200 2024-04-25T10:18:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.520+0000 7f682b000640 1 -- 172.21.15.119:0/608542700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824071200 msgr2=0x7f6824071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.520+0000 7f682b000640 1 --2- 172.21.15.119:0/608542700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824071200 0x7f6824071660 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f680c009d90 tx=0x7f680c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.520+0000 7f682b000640 1 -- 172.21.15.119:0/608542700 shutdown_connections 2024-04-25T10:18:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.520+0000 7f682b000640 1 --2- 172.21.15.119:0/608542700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824071200 0x7f6824071660 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.520+0000 7f682b000640 1 -- 172.21.15.119:0/608542700 >> 172.21.15.119:0/608542700 conn(0x7f682406aba0 msgr2=0x7f682406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.521+0000 7f682b000640 1 -- 172.21.15.119:0/608542700 shutdown_connections 2024-04-25T10:18:38.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.521+0000 7f682b000640 1 -- 172.21.15.119:0/608542700 wait complete. 2024-04-25T10:18:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.522+0000 7f682b000640 1 Processor -- start 2024-04-25T10:18:38.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.522+0000 7f682b000640 1 -- start start 2024-04-25T10:18:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.523+0000 7f682b000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824071200 0x7f68240c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.523+0000 7f682b000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68240cee90 0x7f6824143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.523+0000 7f682b000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6824145260 0x7f68241f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.523+0000 7f682b000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68240cc660 con 0x7f6824071200 2024-04-25T10:18:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.523+0000 7f682b000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68240cea00 con 0x7f68240cee90 2024-04-25T10:18:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.523+0000 7f682b000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6824145640 con 0x7f6824145260 2024-04-25T10:18:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.523+0000 7f6823e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824071200 0x7f68240c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.524+0000 7f6823e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824071200 0x7f68240c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41172/0 (socket says 172.21.15.119:41172) 2024-04-25T10:18:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.524+0000 7f6823e00640 1 -- 172.21.15.119:0/744923915 learned_addr learned my addr 172.21.15.119:0/744923915 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.524+0000 7f681b400640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68240cee90 0x7f6824143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.524+0000 7f6829000640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6824145260 0x7f68241f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.524+0000 7f6823e00640 1 -- 172.21.15.119:0/744923915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6824145260 msgr2=0x7f68241f6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.524+0000 7f6823e00640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6824145260 0x7f68241f6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.524+0000 7f6823e00640 1 -- 172.21.15.119:0/744923915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68240cee90 msgr2=0x7f6824143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.524+0000 7f6823e00640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68240cee90 0x7f6824143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.524+0000 7f6823e00640 1 -- 172.21.15.119:0/744923915 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68241f7750 con 0x7f6824071200 2024-04-25T10:18:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.524+0000 7f6829000640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6824145260 0x7f68241f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.524+0000 7f681b400640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68240cee90 0x7f6824143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.524+0000 7f6823e00640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824071200 0x7f68240c5070 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f681400c460 tx=0x7f681400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.525+0000 7f6820c00640 1 -- 172.21.15.119:0/744923915 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814017070 con 0x7f6824071200 2024-04-25T10:18:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.525+0000 7f6820c00640 1 -- 172.21.15.119:0/744923915 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6814005ce0 con 0x7f6824071200 2024-04-25T10:18:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.525+0000 7f682b000640 1 -- 172.21.15.119:0/744923915 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68241f7a30 con 0x7f6824071200 2024-04-25T10:18:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.525+0000 7f6820c00640 1 -- 172.21.15.119:0/744923915 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814007290 con 0x7f6824071200 2024-04-25T10:18:38.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.525+0000 7f682b000640 1 -- 172.21.15.119:0/744923915 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68240c55b0 con 0x7f6824071200 2024-04-25T10:18:38.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.526+0000 7f682b000640 1 -- 172.21.15.119:0/744923915 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68240c4660 con 0x7f6824071200 2024-04-25T10:18:38.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.527+0000 7f6820c00640 1 -- 172.21.15.119:0/744923915 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f681401a070 con 0x7f6824071200 2024-04-25T10:18:38.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.528+0000 7f6820c00640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f680006b4d0 0x7f680006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.528+0000 7f6820c00640 1 -- 172.21.15.119:0/744923915 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f68140893d0 con 0x7f6824071200 2024-04-25T10:18:38.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.528+0000 7f681b400640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f680006b4d0 0x7f680006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.529+0000 7f681b400640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f680006b4d0 0x7f680006d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f680c034000 tx=0x7f680c033040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.532+0000 7f6820c00640 1 -- 172.21.15.119:0/744923915 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6814058400 con 0x7f6824071200 2024-04-25T10:18:38.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.731+0000 7f682b000640 1 -- 172.21.15.119:0/744923915 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f68240750b0 con 0x7f680006b4d0 2024-04-25T10:18:38.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.732+0000 7f6820c00640 1 -- 172.21.15.119:0/744923915 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f68240750b0 con 0x7f680006b4d0 2024-04-25T10:18:38.735 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T10:18:38.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.734+0000 7f682b000640 1 -- 172.21.15.119:0/744923915 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f680006b4d0 msgr2=0x7f680006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.734+0000 7f682b000640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f680006b4d0 0x7f680006d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f680c034000 tx=0x7f680c033040 comp rx=0 tx=0).stop 2024-04-25T10:18:38.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.734+0000 7f682b000640 1 -- 172.21.15.119:0/744923915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824071200 msgr2=0x7f68240c5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.734+0000 7f682b000640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824071200 0x7f68240c5070 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f681400c460 tx=0x7f681400c920 comp rx=0 tx=0).stop 2024-04-25T10:18:38.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.735+0000 7f682b000640 1 -- 172.21.15.119:0/744923915 shutdown_connections 2024-04-25T10:18:38.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.735+0000 7f682b000640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f680006b4d0 0x7f680006d990 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.735+0000 7f682b000640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6824145260 0x7f68241f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.735+0000 7f682b000640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68240cee90 0x7f6824143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.735+0000 7f682b000640 1 --2- 172.21.15.119:0/744923915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824071200 0x7f68240c5070 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.735+0000 7f682b000640 1 -- 172.21.15.119:0/744923915 >> 172.21.15.119:0/744923915 conn(0x7f682406aba0 msgr2=0x7f68240b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.735+0000 7f682b000640 1 -- 172.21.15.119:0/744923915 shutdown_connections 2024-04-25T10:18:38.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.735+0000 7f682b000640 1 -- 172.21.15.119:0/744923915 wait complete. 2024-04-25T10:18:38.757 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T10:18:38.757 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:18:38.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.952+0000 7fe45b600640 1 Processor -- start 2024-04-25T10:18:38.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.952+0000 7fe45b600640 1 -- start start 2024-04-25T10:18:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.953+0000 7fe45b600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe454071660 0x7fe4540cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.953+0000 7fe45b600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4540cc9a0 con 0x7fe454071200 2024-04-25T10:18:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.953+0000 7fe45b600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4540ccc80 con 0x7fe454145260 2024-04-25T10:18:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.953+0000 7fe45b600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4540cea00 con 0x7fe454071660 2024-04-25T10:18:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.954+0000 7fe453e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe454071200 0x7fe4540cee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36280/0 (socket says 172.21.15.119:36280) 2024-04-25T10:18:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.954+0000 7fe453e00640 1 -- 172.21.15.119:0/2670222643 learned_addr learned my addr 172.21.15.119:0/2670222643 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.954+0000 7fe459600640 1 --2- 172.21.15.119:0/2670222643 >> v2:172.21.15.132:3300/0 conn(0x7fe454071660 0x7fe4540cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.954+0000 7fe453400640 1 -- 172.21.15.119:0/2670222643 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127389723 0 0) 0x7fe4540cc9a0 con 0x7fe454071200 2024-04-25T10:18:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.954+0000 7fe453400640 1 -- 172.21.15.119:0/2670222643 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe438003520 con 0x7fe454071200 2024-04-25T10:18:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.955+0000 7fe453400640 1 -- 172.21.15.119:0/2670222643 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2137968866 0 0) 0x7fe438003520 con 0x7fe454071200 2024-04-25T10:18:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.955+0000 7fe453400640 1 -- 172.21.15.119:0/2670222643 >> v2:172.21.15.132:3300/0 conn(0x7fe454071660 msgr2=0x7fe4540cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.955+0000 7fe453400640 1 --2- 172.21.15.119:0/2670222643 >> v2:172.21.15.132:3300/0 conn(0x7fe454071660 0x7fe4540cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.955+0000 7fe453400640 1 -- 172.21.15.119:0/2670222643 >> v1:172.21.15.132:6789/0 conn(0x7fe454145260 legacy=0x7fe4540cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.955+0000 7fe453400640 1 -- 172.21.15.119:0/2670222643 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4540ccf60 con 0x7fe454071200 2024-04-25T10:18:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.955+0000 7fe453400640 1 -- 172.21.15.119:0/2670222643 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fe440001670 con 0x7fe454071200 2024-04-25T10:18:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.955+0000 7fe453400640 1 -- 172.21.15.119:0/2670222643 >> v1:172.21.15.119:6789/0 conn(0x7fe454071200 legacy=0x7fe4540cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.955+0000 7fe453400640 1 --2- 172.21.15.119:0/2670222643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4380039b0 0x7fe438003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.955+0000 7fe453400640 1 --2- 172.21.15.119:0/2670222643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe438004800 0x7fe438004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.955+0000 7fe453400640 1 --2- 172.21.15.119:0/2670222643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4380051c0 0x7fe43800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.955+0000 7fe453400640 1 -- 172.21.15.119:0/2670222643 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540cea00 con 0x7fe4380039b0 2024-04-25T10:18:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.955+0000 7fe453400640 1 -- 172.21.15.119:0/2670222643 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43800c060 con 0x7fe4380051c0 2024-04-25T10:18:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.955+0000 7fe453400640 1 -- 172.21.15.119:0/2670222643 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43800c260 con 0x7fe438004800 2024-04-25T10:18:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.955+0000 7fe458c00640 1 --2- 172.21.15.119:0/2670222643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4380039b0 0x7fe438003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.956+0000 7fe453e00640 1 --2- 172.21.15.119:0/2670222643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe438004800 0x7fe438004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.956+0000 7fe458c00640 1 -- 172.21.15.119:0/2670222643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe438004800 msgr2=0x7fe438004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.956+0000 7fe459600640 1 --2- 172.21.15.119:0/2670222643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4380051c0 0x7fe43800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.956+0000 7fe458c00640 1 --2- 172.21.15.119:0/2670222643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe438004800 0x7fe438004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.956+0000 7fe458c00640 1 -- 172.21.15.119:0/2670222643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4380051c0 msgr2=0x7fe43800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.956+0000 7fe458c00640 1 --2- 172.21.15.119:0/2670222643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4380051c0 0x7fe43800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.956+0000 7fe458c00640 1 -- 172.21.15.119:0/2670222643 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe438003520 con 0x7fe4380039b0 2024-04-25T10:18:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.956+0000 7fe459600640 1 -- 172.21.15.119:0/2670222643 reap_dead start 2024-04-25T10:18:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.956+0000 7fe453e00640 1 --2- 172.21.15.119:0/2670222643 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe438004800 0x7fe438004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:38.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.957+0000 7fe458c00640 1 --2- 172.21.15.119:0/2670222643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4380039b0 0x7fe438003dd0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fe444001e50 tx=0x7fe438005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca1fe302be6aa8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.957+0000 7fe453400640 1 -- 172.21.15.119:0/2670222643 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444001a70 con 0x7fe4380039b0 2024-04-25T10:18:38.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.957+0000 7fe453400640 1 -- 172.21.15.119:0/2670222643 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44402ae40 con 0x7fe4380039b0 2024-04-25T10:18:38.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.957+0000 7fe453400640 1 -- 172.21.15.119:0/2670222643 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444003bb0 con 0x7fe4380039b0 2024-04-25T10:18:38.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.958+0000 7fe45b600640 1 -- 172.21.15.119:0/2670222643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4380039b0 msgr2=0x7fe438003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.958+0000 7fe45b600640 1 --2- 172.21.15.119:0/2670222643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4380039b0 0x7fe438003dd0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fe444001e50 tx=0x7fe438005830 comp rx=0 tx=0).stop 2024-04-25T10:18:38.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.958+0000 7fe45b600640 1 -- 172.21.15.119:0/2670222643 shutdown_connections 2024-04-25T10:18:38.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.958+0000 7fe45b600640 1 --2- 172.21.15.119:0/2670222643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4380039b0 0x7fe438003dd0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.958+0000 7fe45b600640 1 -- 172.21.15.119:0/2670222643 >> 172.21.15.119:0/2670222643 conn(0x7fe45406aba0 msgr2=0x7fe45406af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.958+0000 7fe45b600640 1 -- 172.21.15.119:0/2670222643 shutdown_connections 2024-04-25T10:18:38.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.958+0000 7fe45b600640 1 -- 172.21.15.119:0/2670222643 wait complete. 2024-04-25T10:18:38.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.960+0000 7fe45b600640 1 Processor -- start 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.960+0000 7fe45b600640 1 -- start start 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.961+0000 7fe45b600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4380039b0 0x7fe45415b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.961+0000 7fe45b600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4540b1de0 0x7fe4540b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.961+0000 7fe45b600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4540b39b0 0x7fe4540b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.961+0000 7fe45b600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe454145640 con 0x7fe4540b1de0 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.961+0000 7fe45b600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe454071a90 con 0x7fe4540b39b0 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.961+0000 7fe45b600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45415c270 con 0x7fe4380039b0 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.961+0000 7fe453e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4540b1de0 0x7fe4540b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.961+0000 7fe453e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4540b1de0 0x7fe4540b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41198/0 (socket says 172.21.15.119:41198) 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.961+0000 7fe453e00640 1 -- 172.21.15.119:0/493217337 learned_addr learned my addr 172.21.15.119:0/493217337 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.961+0000 7fe453e00640 1 -- 172.21.15.119:0/493217337 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4380039b0 msgr2=0x7fe45415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.961+0000 7fe453e00640 1 --2- 172.21.15.119:0/493217337 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4380039b0 0x7fe45415b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.961+0000 7fe453e00640 1 -- 172.21.15.119:0/493217337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4540b39b0 msgr2=0x7fe4540b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.961+0000 7fe453e00640 1 --2- 172.21.15.119:0/493217337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4540b39b0 0x7fe4540b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.961+0000 7fe453e00640 1 -- 172.21.15.119:0/493217337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4540b2830 con 0x7fe4540b1de0 2024-04-25T10:18:38.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.962+0000 7fe453e00640 1 --2- 172.21.15.119:0/493217337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4540b1de0 0x7fe4540b2240 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fe44000d7a0 tx=0x7fe44000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.962+0000 7fe450c00640 1 -- 172.21.15.119:0/493217337 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440019070 con 0x7fe4540b1de0 2024-04-25T10:18:38.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.962+0000 7fe450c00640 1 -- 172.21.15.119:0/493217337 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4400074e0 con 0x7fe4540b1de0 2024-04-25T10:18:38.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.962+0000 7fe45b600640 1 -- 172.21.15.119:0/493217337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4540b2b10 con 0x7fe4540b1de0 2024-04-25T10:18:38.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.962+0000 7fe450c00640 1 -- 172.21.15.119:0/493217337 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440004a00 con 0x7fe4540b1de0 2024-04-25T10:18:38.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.962+0000 7fe45b600640 1 -- 172.21.15.119:0/493217337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4540b30d0 con 0x7fe4540b1de0 2024-04-25T10:18:38.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.963+0000 7fe45b600640 1 -- 172.21.15.119:0/493217337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe4540750b0 con 0x7fe4540b1de0 2024-04-25T10:18:38.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.964+0000 7fe450c00640 1 -- 172.21.15.119:0/493217337 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe44001c070 con 0x7fe4540b1de0 2024-04-25T10:18:38.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.965+0000 7fe450c00640 1 --2- 172.21.15.119:0/493217337 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe43406b500 0x7fe43406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.965+0000 7fe450c00640 1 -- 172.21.15.119:0/493217337 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fe440089a50 con 0x7fe4540b1de0 2024-04-25T10:18:38.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.965+0000 7fe458c00640 1 --2- 172.21.15.119:0/493217337 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe43406b500 0x7fe43406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.965+0000 7fe450c00640 1 --2- 172.21.15.119:0/493217337 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x7fe434071500 0x7fe434073920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.966+0000 7fe450c00640 1 -- 172.21.15.119:0/493217337 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe434073fd0 con 0x7fe434071500 2024-04-25T10:18:38.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.966+0000 7fe450c00640 1 -- 172.21.15.119:0/493217337 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe44008e2a0 con 0x7fe4540b1de0 2024-04-25T10:18:38.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.966+0000 7fe458c00640 1 --2- 172.21.15.119:0/493217337 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe43406b500 0x7fe43406d9c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe444032000 tx=0x7fe44402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.966+0000 7fe459600640 1 --2- 172.21.15.119:0/493217337 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x7fe434071500 0x7fe434073920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.967+0000 7fe459600640 1 --2- 172.21.15.119:0/493217337 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x7fe434071500 0x7fe434073920 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.970+0000 7fe450c00640 1 -- 172.21.15.119:0/493217337 <== osd.4 v2:172.21.15.132:6800/1740646323 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe434073fd0 con 0x7fe434071500 2024-04-25T10:18:38.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.984+0000 7fe45b600640 1 -- 172.21.15.119:0/493217337 --> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe45406efd0 con 0x7fe434071500 2024-04-25T10:18:38.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.985+0000 7fe450c00640 1 -- 172.21.15.119:0/493217337 <== osd.4 v2:172.21.15.132:6800/1740646323 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe45406efd0 con 0x7fe434071500 2024-04-25T10:18:38.989 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:18:38.989 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:18:38.989 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:18:38.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.986+0000 7fe45b600640 1 -- 172.21.15.119:0/493217337 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x7fe434071500 msgr2=0x7fe434073920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.986+0000 7fe45b600640 1 --2- 172.21.15.119:0/493217337 >> [v2:172.21.15.132:6800/1740646323,v1:172.21.15.132:6801/1740646323] conn(0x7fe434071500 0x7fe434073920 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.986+0000 7fe45b600640 1 -- 172.21.15.119:0/493217337 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe43406b500 msgr2=0x7fe43406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.986+0000 7fe45b600640 1 --2- 172.21.15.119:0/493217337 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fe43406b500 0x7fe43406d9c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe444032000 tx=0x7fe44402c000 comp rx=0 tx=0).stop 2024-04-25T10:18:38.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.986+0000 7fe45b600640 1 -- 172.21.15.119:0/493217337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4540b1de0 msgr2=0x7fe4540b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.986+0000 7fe45b600640 1 --2- 172.21.15.119:0/493217337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4540b1de0 0x7fe4540b2240 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fe44000d7a0 tx=0x7fe44000dc60 comp rx=0 tx=0).stop 2024-04-25T10:18:38.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.986+0000 7fe459600640 1 -- 172.21.15.119:0/493217337 reap_dead start 2024-04-25T10:18:38.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.986+0000 7fe45b600640 1 -- 172.21.15.119:0/493217337 shutdown_connections 2024-04-25T10:18:38.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.986+0000 7fe45b600640 1 -- 172.21.15.119:0/493217337 >> 172.21.15.119:0/493217337 conn(0x7fe45406aba0 msgr2=0x7fe4540cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.986+0000 7fe45b600640 1 -- 172.21.15.119:0/493217337 shutdown_connections 2024-04-25T10:18:38.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:38.986+0000 7fe45b600640 1 -- 172.21.15.119:0/493217337 wait complete. 2024-04-25T10:18:39.006 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T10:18:39.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.196+0000 7ffbdd600640 1 Processor -- start 2024-04-25T10:18:39.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.197+0000 7ffbdd600640 1 -- start start 2024-04-25T10:18:39.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.198+0000 7ffbdd600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffbd8161d60 0x7ffbd8166380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.198+0000 7ffbdd600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffbd80cea30 0x7ffbd80b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.198+0000 7ffbdd600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd8162140 con 0x7ffbd806f0c0 2024-04-25T10:18:39.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.198+0000 7ffbdd600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd80b2470 con 0x7ffbd80cea30 2024-04-25T10:18:39.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.198+0000 7ffbdd600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd8165ef0 con 0x7ffbd8161d60 2024-04-25T10:18:39.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.199+0000 7ffbcf400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffbd80cea30 0x7ffbd80b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.199+0000 7ffbcf400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffbd80cea30 0x7ffbd80b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41200/0 (socket says 172.21.15.119:41200) 2024-04-25T10:18:39.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.199+0000 7ffbcf400640 1 -- 172.21.15.119:0/3013391 learned_addr learned my addr 172.21.15.119:0/3013391 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:39.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.199+0000 7ffbdcc00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ffbd806f0c0 0x7ffbd806dcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36286/0 (socket says 172.21.15.119:36286) 2024-04-25T10:18:39.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.199+0000 7ffbd7e00640 1 --2- 172.21.15.119:0/3013391 >> v2:172.21.15.132:3300/0 conn(0x7ffbd8161d60 0x7ffbd8166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.199+0000 7ffbcf400640 1 -- 172.21.15.119:0/3013391 >> v2:172.21.15.132:3300/0 conn(0x7ffbd8161d60 msgr2=0x7ffbd8166380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.199+0000 7ffbcf400640 1 --2- 172.21.15.119:0/3013391 >> v2:172.21.15.132:3300/0 conn(0x7ffbd8161d60 0x7ffbd8166380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.199+0000 7ffbcf400640 1 -- 172.21.15.119:0/3013391 >> v1:172.21.15.119:6789/0 conn(0x7ffbd806f0c0 legacy=0x7ffbd806dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.199+0000 7ffbcf400640 1 -- 172.21.15.119:0/3013391 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd80cf210 con 0x7ffbd80cea30 2024-04-25T10:18:39.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.200+0000 7ffbcf400640 1 --2- 172.21.15.119:0/3013391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd80cea30 0x7ffbd80b2df0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80724f0 tx=0x7ffbc002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa298caf933d39f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.200+0000 7ffbd7400640 1 -- 172.21.15.119:0/3013391 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0037070 con 0x7ffbd80cea30 2024-04-25T10:18:39.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.201+0000 7ffbd7400640 1 -- 172.21.15.119:0/3013391 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc0032440 con 0x7ffbd80cea30 2024-04-25T10:18:39.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.201+0000 7ffbd7400640 1 -- 172.21.15.119:0/3013391 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0032810 con 0x7ffbd80cea30 2024-04-25T10:18:39.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.201+0000 7ffbdd600640 1 -- 172.21.15.119:0/3013391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd80cea30 msgr2=0x7ffbd80b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.201+0000 7ffbdd600640 1 --2- 172.21.15.119:0/3013391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd80cea30 0x7ffbd80b2df0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80724f0 tx=0x7ffbc002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.201+0000 7ffbdd600640 1 -- 172.21.15.119:0/3013391 shutdown_connections 2024-04-25T10:18:39.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.201+0000 7ffbdd600640 1 --2- 172.21.15.119:0/3013391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd80cea30 0x7ffbd80b2df0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.201+0000 7ffbdd600640 1 --2- 172.21.15.119:0/3013391 >> v2:172.21.15.132:3300/0 conn(0x7ffbd8161d60 0x7ffbd8166380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.201+0000 7ffbdd600640 1 -- 172.21.15.119:0/3013391 >> 172.21.15.119:0/3013391 conn(0x7ffbd8066ff0 msgr2=0x7ffbd80b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.201+0000 7ffbdd600640 1 -- 172.21.15.119:0/3013391 shutdown_connections 2024-04-25T10:18:39.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.201+0000 7ffbdd600640 1 -- 172.21.15.119:0/3013391 wait complete. 2024-04-25T10:18:39.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.203+0000 7ffbdd600640 1 Processor -- start 2024-04-25T10:18:39.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.203+0000 7ffbdd600640 1 -- start start 2024-04-25T10:18:39.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.204+0000 7ffbdd600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbd806f0c0 0x7ffbd81f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.204+0000 7ffbdd600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbd80cea30 0x7ffbd8145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.204+0000 7ffbdd600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd8161d60 0x7ffbd8146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.204+0000 7ffbdd600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd81658a0 con 0x7ffbd8161d60 2024-04-25T10:18:39.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.204+0000 7ffbdd600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd80b29a0 con 0x7ffbd806f0c0 2024-04-25T10:18:39.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.204+0000 7ffbdd600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd8165b80 con 0x7ffbd80cea30 2024-04-25T10:18:39.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.204+0000 7ffbdcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd8161d60 0x7ffbd8146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.204+0000 7ffbdcc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd8161d60 0x7ffbd8146320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41206/0 (socket says 172.21.15.119:41206) 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.204+0000 7ffbdcc00640 1 -- 172.21.15.119:0/2865766478 learned_addr learned my addr 172.21.15.119:0/2865766478 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.205+0000 7ffbd7e00640 1 --2- 172.21.15.119:0/2865766478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbd806f0c0 0x7ffbd81f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.205+0000 7ffbdcc00640 1 -- 172.21.15.119:0/2865766478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbd80cea30 msgr2=0x7ffbd8145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.205+0000 7ffbdcc00640 1 --2- 172.21.15.119:0/2865766478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbd80cea30 0x7ffbd8145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.205+0000 7ffbdcc00640 1 -- 172.21.15.119:0/2865766478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbd806f0c0 msgr2=0x7ffbd81f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.205+0000 7ffbdcc00640 1 --2- 172.21.15.119:0/2865766478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbd806f0c0 0x7ffbd81f7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.205+0000 7ffbdcc00640 1 -- 172.21.15.119:0/2865766478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd815a9e0 con 0x7ffbd8161d60 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.205+0000 7ffbdcc00640 1 --2- 172.21.15.119:0/2865766478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd8161d60 0x7ffbd8146320 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c780 tx=0x7ffbd000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.205+0000 7ffbd4c00640 1 -- 172.21.15.119:0/2865766478 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0017070 con 0x7ffbd8161d60 2024-04-25T10:18:39.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.205+0000 7ffbd4c00640 1 -- 172.21.15.119:0/2865766478 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd000a5e0 con 0x7ffbd8161d60 2024-04-25T10:18:39.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.205+0000 7ffbd4c00640 1 -- 172.21.15.119:0/2865766478 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd000a9d0 con 0x7ffbd8161d60 2024-04-25T10:18:39.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.206+0000 7ffbdd600640 1 -- 172.21.15.119:0/2865766478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd8156e60 con 0x7ffbd8161d60 2024-04-25T10:18:39.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.206+0000 7ffbdd600640 1 -- 172.21.15.119:0/2865766478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd815aca0 con 0x7ffbd8161d60 2024-04-25T10:18:39.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.207+0000 7ffbdd600640 1 -- 172.21.15.119:0/2865766478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd80b4610 con 0x7ffbd8161d60 2024-04-25T10:18:39.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.208+0000 7ffbd4c00640 1 -- 172.21.15.119:0/2865766478 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbd001a070 con 0x7ffbd8161d60 2024-04-25T10:18:39.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.209+0000 7ffbd4c00640 1 --2- 172.21.15.119:0/2865766478 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ffbb406b4d0 0x7ffbb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.209+0000 7ffbd4c00640 1 -- 172.21.15.119:0/2865766478 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7ffbd0089450 con 0x7ffbd8161d60 2024-04-25T10:18:39.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.209+0000 7ffbd7e00640 1 --2- 172.21.15.119:0/2865766478 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ffbb406b4d0 0x7ffbb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.210+0000 7ffbd7e00640 1 --2- 172.21.15.119:0/2865766478 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ffbb406b4d0 0x7ffbb406d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8006fd0 tx=0x7ffbc8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.213+0000 7ffbd4c00640 1 -- 172.21.15.119:0/2865766478 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffbd80b4610 con 0x7ffbd8161d60 2024-04-25T10:18:39.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.409+0000 7ffbdd600640 1 -- 172.21.15.119:0/2865766478 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbd806d7c0 con 0x7ffbb406b4d0 2024-04-25T10:18:39.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.410+0000 7ffbd4c00640 1 -- 172.21.15.119:0/2865766478 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ffbd806d7c0 con 0x7ffbb406b4d0 2024-04-25T10:18:39.413 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T10:18:39.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.413+0000 7ffbdd600640 1 -- 172.21.15.119:0/2865766478 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ffbb406b4d0 msgr2=0x7ffbb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.413+0000 7ffbdd600640 1 --2- 172.21.15.119:0/2865766478 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ffbb406b4d0 0x7ffbb406d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8006fd0 tx=0x7ffbc8008040 comp rx=0 tx=0).stop 2024-04-25T10:18:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.413+0000 7ffbdd600640 1 -- 172.21.15.119:0/2865766478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd8161d60 msgr2=0x7ffbd8146320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.413+0000 7ffbdd600640 1 --2- 172.21.15.119:0/2865766478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd8161d60 0x7ffbd8146320 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c780 tx=0x7ffbd000cc40 comp rx=0 tx=0).stop 2024-04-25T10:18:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.413+0000 7ffbdd600640 1 -- 172.21.15.119:0/2865766478 shutdown_connections 2024-04-25T10:18:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.413+0000 7ffbdd600640 1 --2- 172.21.15.119:0/2865766478 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7ffbb406b4d0 0x7ffbb406d990 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.413+0000 7ffbdd600640 1 --2- 172.21.15.119:0/2865766478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbd8161d60 0x7ffbd8146320 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.413+0000 7ffbdd600640 1 --2- 172.21.15.119:0/2865766478 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffbd80cea30 0x7ffbd8145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.413+0000 7ffbdd600640 1 --2- 172.21.15.119:0/2865766478 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbd806f0c0 0x7ffbd81f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.413+0000 7ffbdd600640 1 -- 172.21.15.119:0/2865766478 >> 172.21.15.119:0/2865766478 conn(0x7ffbd8066ff0 msgr2=0x7ffbd8160bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.413+0000 7ffbdd600640 1 -- 172.21.15.119:0/2865766478 shutdown_connections 2024-04-25T10:18:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.413+0000 7ffbdd600640 1 -- 172.21.15.119:0/2865766478 wait complete. 2024-04-25T10:18:39.435 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T10:18:39.436 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:18:39.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.637+0000 7fafd1200640 1 Processor -- start 2024-04-25T10:18:39.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.637+0000 7fafd1200640 1 -- start start 2024-04-25T10:18:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.639+0000 7fafd1200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fafcc161090 0x7fafcc161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.639+0000 7fafd1200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fafcc0b9720 0x7fafcc0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.639+0000 7fafd1200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fafcc0cece0 0x7fafcc06c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.639+0000 7fafd1200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafcc0cf130 con 0x7fafcc161090 2024-04-25T10:18:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.639+0000 7fafd1200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafcc0d21a0 con 0x7fafcc0b9720 2024-04-25T10:18:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.639+0000 7fafd1200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc161f80 con 0x7fafcc0cece0 2024-04-25T10:18:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.639+0000 7fafc9e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fafcc161090 0x7fafcc161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.639+0000 7fafc9e00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fafcc161090 0x7fafcc161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41222/0 (socket says 172.21.15.119:41222) 2024-04-25T10:18:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.639+0000 7fafc9e00640 1 -- 172.21.15.119:0/2013943615 learned_addr learned my addr 172.21.15.119:0/2013943615 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.640+0000 7fafc9400640 1 --2- 172.21.15.119:0/2013943615 >> v2:172.21.15.132:3300/0 conn(0x7fafcc0b9720 0x7fafcc0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.640+0000 7fafca800640 1 --2- 172.21.15.119:0/2013943615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fafcc0cece0 0x7fafcc06c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.640+0000 7fafc9e00640 1 -- 172.21.15.119:0/2013943615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fafcc0cece0 msgr2=0x7fafcc06c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.640+0000 7fafc9e00640 1 --2- 172.21.15.119:0/2013943615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fafcc0cece0 0x7fafcc06c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.640+0000 7fafc9e00640 1 -- 172.21.15.119:0/2013943615 >> v2:172.21.15.132:3300/0 conn(0x7fafcc0b9720 msgr2=0x7fafcc0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.640+0000 7fafc9e00640 1 --2- 172.21.15.119:0/2013943615 >> v2:172.21.15.132:3300/0 conn(0x7fafcc0b9720 0x7fafcc0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.640+0000 7fafc9e00640 1 -- 172.21.15.119:0/2013943615 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc162260 con 0x7fafcc161090 2024-04-25T10:18:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.640+0000 7fafca800640 1 --2- 172.21.15.119:0/2013943615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fafcc0cece0 0x7fafcc06c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.640+0000 7fafc9400640 1 --2- 172.21.15.119:0/2013943615 >> v2:172.21.15.132:3300/0 conn(0x7fafcc0b9720 0x7fafcc0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:39.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.641+0000 7fafc9e00640 1 --2- 172.21.15.119:0/2013943615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc161090 0x7fafcc161730 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fafcc08cf10 tx=0x7fafbc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa88724d1df0aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.641+0000 7fafc8a00640 1 -- 172.21.15.119:0/2013943615 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc037070 con 0x7fafcc161090 2024-04-25T10:18:39.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.642+0000 7fafc8a00640 1 -- 172.21.15.119:0/2013943615 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafbc032440 con 0x7fafcc161090 2024-04-25T10:18:39.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.642+0000 7fafc8a00640 1 -- 172.21.15.119:0/2013943615 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc032810 con 0x7fafcc161090 2024-04-25T10:18:39.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.642+0000 7fafd1200640 1 -- 172.21.15.119:0/2013943615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc161090 msgr2=0x7fafcc161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.642+0000 7fafd1200640 1 --2- 172.21.15.119:0/2013943615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc161090 0x7fafcc161730 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fafcc08cf10 tx=0x7fafbc02a140 comp rx=0 tx=0).stop 2024-04-25T10:18:39.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.642+0000 7fafd1200640 1 -- 172.21.15.119:0/2013943615 shutdown_connections 2024-04-25T10:18:39.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.642+0000 7fafd1200640 1 --2- 172.21.15.119:0/2013943615 >> v2:172.21.15.132:3300/0 conn(0x7fafcc0b9720 0x7fafcc0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.642+0000 7fafd1200640 1 --2- 172.21.15.119:0/2013943615 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fafcc0cece0 0x7fafcc06c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.642+0000 7fafd1200640 1 --2- 172.21.15.119:0/2013943615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc161090 0x7fafcc161730 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.642+0000 7fafd1200640 1 -- 172.21.15.119:0/2013943615 >> 172.21.15.119:0/2013943615 conn(0x7fafcc0b55e0 msgr2=0x7fafcc0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.642+0000 7fafd1200640 1 -- 172.21.15.119:0/2013943615 shutdown_connections 2024-04-25T10:18:39.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.643+0000 7fafd1200640 1 -- 172.21.15.119:0/2013943615 wait complete. 2024-04-25T10:18:39.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.645+0000 7fafd1200640 1 Processor -- start 2024-04-25T10:18:39.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.645+0000 7fafd1200640 1 -- start start 2024-04-25T10:18:39.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.646+0000 7fafd1200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fafcc0b9720 0x7fafcc14f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.646+0000 7fafd1200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fafcc0cece0 0x7fafcc0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.646+0000 7fafd1200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc161090 0x7fafcc15ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.646+0000 7fafd1200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc161470 con 0x7fafcc161090 2024-04-25T10:18:39.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.646+0000 7fafd1200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0d1e30 con 0x7fafcc0b9720 2024-04-25T10:18:39.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.646+0000 7fafd1200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc161c70 con 0x7fafcc0cece0 2024-04-25T10:18:39.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.646+0000 7fafca800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc161090 0x7fafcc15ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.646+0000 7fafca800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc161090 0x7fafcc15ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41236/0 (socket says 172.21.15.119:41236) 2024-04-25T10:18:39.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.646+0000 7fafca800640 1 -- 172.21.15.119:0/3736095322 learned_addr learned my addr 172.21.15.119:0/3736095322 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:39.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.647+0000 7fafc9e00640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fafcc0b9720 0x7fafcc14f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.647+0000 7fafc9400640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fafcc0cece0 0x7fafcc0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.647+0000 7fafca800640 1 -- 172.21.15.119:0/3736095322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fafcc0cece0 msgr2=0x7fafcc0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.647+0000 7fafca800640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fafcc0cece0 0x7fafcc0cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.647+0000 7fafca800640 1 -- 172.21.15.119:0/3736095322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fafcc0b9720 msgr2=0x7fafcc14f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.647+0000 7fafca800640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fafcc0b9720 0x7fafcc14f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.647+0000 7fafca800640 1 -- 172.21.15.119:0/3736095322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc15db80 con 0x7fafcc161090 2024-04-25T10:18:39.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.647+0000 7fafc9e00640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fafcc0b9720 0x7fafcc14f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:39.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.647+0000 7fafca800640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc161090 0x7fafcc15ed20 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fafc400cf20 tx=0x7fafc400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.648+0000 7fafc2000640 1 -- 172.21.15.119:0/3736095322 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc401a070 con 0x7fafcc161090 2024-04-25T10:18:39.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.648+0000 7fafc2000640 1 -- 172.21.15.119:0/3736095322 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc400d840 con 0x7fafcc161090 2024-04-25T10:18:39.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.648+0000 7fafc2000640 1 -- 172.21.15.119:0/3736095322 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc400dc30 con 0x7fafcc161090 2024-04-25T10:18:39.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.648+0000 7fafd1200640 1 -- 172.21.15.119:0/3736095322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc15de20 con 0x7fafcc161090 2024-04-25T10:18:39.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.648+0000 7fafd1200640 1 -- 172.21.15.119:0/3736095322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc15e3e0 con 0x7fafcc161090 2024-04-25T10:18:39.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.649+0000 7fafd1200640 1 -- 172.21.15.119:0/3736095322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fafcc0750b0 con 0x7fafcc161090 2024-04-25T10:18:39.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.650+0000 7fafc2000640 1 -- 172.21.15.119:0/3736095322 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fafc401d070 con 0x7fafcc161090 2024-04-25T10:18:39.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.651+0000 7fafc2000640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fafac06b500 0x7fafac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.651+0000 7fafc2000640 1 -- 172.21.15.119:0/3736095322 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fafc408a8b0 con 0x7fafcc161090 2024-04-25T10:18:39.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.651+0000 7fafc2000640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x7fafac071500 0x7fafac073920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.651+0000 7fafc2000640 1 -- 172.21.15.119:0/3736095322 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafac073fd0 con 0x7fafac071500 2024-04-25T10:18:39.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.651+0000 7fafc2000640 1 -- 172.21.15.119:0/3736095322 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafc4091050 con 0x7fafcc161090 2024-04-25T10:18:39.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.651+0000 7fafc9e00640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fafac06b500 0x7fafac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.652+0000 7fafc9400640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x7fafac071500 0x7fafac073920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.652+0000 7fafc9e00640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fafac06b500 0x7fafac06d9c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fafbc034000 tx=0x7fafbc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.653+0000 7fafc9400640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x7fafac071500 0x7fafac073920 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.655+0000 7fafc2000640 1 -- 172.21.15.119:0/3736095322 <== osd.5 v2:172.21.15.132:6815/2978152964 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fafac073fd0 con 0x7fafac071500 2024-04-25T10:18:39.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.670+0000 7fafd1200640 1 -- 172.21.15.119:0/3736095322 --> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fafcc0b8d20 con 0x7fafac071500 2024-04-25T10:18:39.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.671+0000 7fafc2000640 1 -- 172.21.15.119:0/3736095322 <== osd.5 v2:172.21.15.132:6815/2978152964 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fafcc0b8d20 con 0x7fafac071500 2024-04-25T10:18:39.674 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:18:39.674 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:18:39.674 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:18:39.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.671+0000 7fafd1200640 1 -- 172.21.15.119:0/3736095322 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x7fafac071500 msgr2=0x7fafac073920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.671+0000 7fafd1200640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.132:6815/2978152964,v1:172.21.15.132:6817/2978152964] conn(0x7fafac071500 0x7fafac073920 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.672+0000 7fafd1200640 1 -- 172.21.15.119:0/3736095322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fafac06b500 msgr2=0x7fafac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.672+0000 7fafd1200640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fafac06b500 0x7fafac06d9c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fafbc034000 tx=0x7fafbc02e000 comp rx=0 tx=0).stop 2024-04-25T10:18:39.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.672+0000 7fafd1200640 1 -- 172.21.15.119:0/3736095322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc161090 msgr2=0x7fafcc15ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.672+0000 7fafd1200640 1 --2- 172.21.15.119:0/3736095322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fafcc161090 0x7fafcc15ed20 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fafc400cf20 tx=0x7fafc400a7d0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.672+0000 7fafca800640 1 -- 172.21.15.119:0/3736095322 reap_dead start 2024-04-25T10:18:39.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.672+0000 7fafd1200640 1 -- 172.21.15.119:0/3736095322 shutdown_connections 2024-04-25T10:18:39.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.672+0000 7fafd1200640 1 -- 172.21.15.119:0/3736095322 >> 172.21.15.119:0/3736095322 conn(0x7fafcc0b55e0 msgr2=0x7fafcc0d4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.672+0000 7fafd1200640 1 -- 172.21.15.119:0/3736095322 shutdown_connections 2024-04-25T10:18:39.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.672+0000 7fafd1200640 1 -- 172.21.15.119:0/3736095322 wait complete. 2024-04-25T10:18:39.692 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T10:18:39.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.882+0000 7f342e600640 1 Processor -- start 2024-04-25T10:18:39.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.883+0000 7f342e600640 1 -- start start 2024-04-25T10:18:39.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.884+0000 7f342e600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3428161090 0x7f3428161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.884+0000 7f342e600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f34280ceeb0 0x7f34280d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.884+0000 7f342e600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3428161f80 con 0x7f34280b9720 2024-04-25T10:18:39.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.884+0000 7f342e600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34280d21a0 con 0x7f3428161090 2024-04-25T10:18:39.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.884+0000 7f342e600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3428162260 con 0x7f34280ceeb0 2024-04-25T10:18:39.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.884+0000 7f3427400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3428161090 0x7f3428161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.884+0000 7f3427400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3428161090 0x7f3428161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41244/0 (socket says 172.21.15.119:41244) 2024-04-25T10:18:39.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.884+0000 7f3427400640 1 -- 172.21.15.119:0/2507174725 learned_addr learned my addr 172.21.15.119:0/2507174725 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:39.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.884+0000 7f3426a00640 1 --1- 172.21.15.119:0/2507174725 >> v1:172.21.15.119:6789/0 conn(0x7f34280b9720 0x7f34280ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36296/0 (socket says 172.21.15.119:36296) 2024-04-25T10:18:39.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.885+0000 7f3427e00640 1 --2- 172.21.15.119:0/2507174725 >> v2:172.21.15.132:3300/0 conn(0x7f34280ceeb0 0x7f34280d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.885+0000 7f3427400640 1 -- 172.21.15.119:0/2507174725 >> v2:172.21.15.132:3300/0 conn(0x7f34280ceeb0 msgr2=0x7f34280d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.885+0000 7f3427400640 1 --2- 172.21.15.119:0/2507174725 >> v2:172.21.15.132:3300/0 conn(0x7f34280ceeb0 0x7f34280d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.885+0000 7f3427400640 1 -- 172.21.15.119:0/2507174725 >> v1:172.21.15.119:6789/0 conn(0x7f34280b9720 legacy=0x7f34280ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.885+0000 7f3427400640 1 -- 172.21.15.119:0/2507174725 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3428162540 con 0x7f3428161090 2024-04-25T10:18:39.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.885+0000 7f3426000640 1 -- 172.21.15.119:0/2507174725 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216251972 0 0) 0x7f3428161f80 con 0x7f34280b9720 2024-04-25T10:18:39.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.886+0000 7f3427400640 1 --2- 172.21.15.119:0/2507174725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428161090 0x7f3428161730 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f34280b69d0 tx=0x7f341802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f8afcdb1db4dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.886+0000 7f3426000640 1 -- 172.21.15.119:0/2507174725 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418037070 con 0x7f3428161090 2024-04-25T10:18:39.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.886+0000 7f3426000640 1 -- 172.21.15.119:0/2507174725 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3418032440 con 0x7f3428161090 2024-04-25T10:18:39.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.886+0000 7f3426000640 1 -- 172.21.15.119:0/2507174725 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418032810 con 0x7f3428161090 2024-04-25T10:18:39.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.887+0000 7f342e600640 1 -- 172.21.15.119:0/2507174725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428161090 msgr2=0x7f3428161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.887+0000 7f342e600640 1 --2- 172.21.15.119:0/2507174725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428161090 0x7f3428161730 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f34280b69d0 tx=0x7f341802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.887+0000 7f342e600640 1 -- 172.21.15.119:0/2507174725 shutdown_connections 2024-04-25T10:18:39.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.887+0000 7f342e600640 1 --2- 172.21.15.119:0/2507174725 >> v2:172.21.15.132:3300/0 conn(0x7f34280ceeb0 0x7f34280d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.887+0000 7f342e600640 1 --2- 172.21.15.119:0/2507174725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428161090 0x7f3428161730 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.887+0000 7f342e600640 1 -- 172.21.15.119:0/2507174725 >> 172.21.15.119:0/2507174725 conn(0x7f34280b55e0 msgr2=0x7f34280b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.887+0000 7f342e600640 1 -- 172.21.15.119:0/2507174725 shutdown_connections 2024-04-25T10:18:39.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.887+0000 7f342e600640 1 -- 172.21.15.119:0/2507174725 wait complete. 2024-04-25T10:18:39.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.889+0000 7f342e600640 1 Processor -- start 2024-04-25T10:18:39.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.889+0000 7f342e600640 1 -- start start 2024-04-25T10:18:39.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.890+0000 7f342e600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34280b9720 0x7f3428158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.890+0000 7f342e600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34280ceeb0 0x7f34280c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.890+0000 7f342e600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428161090 0x7f34280c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.890+0000 7f342e600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3428161c70 con 0x7f3428161090 2024-04-25T10:18:39.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.890+0000 7f342e600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3428161470 con 0x7f34280b9720 2024-04-25T10:18:39.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.890+0000 7f342e600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34280d2b70 con 0x7f34280ceeb0 2024-04-25T10:18:39.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.890+0000 7f3427e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428161090 0x7f34280c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.890+0000 7f3427e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428161090 0x7f34280c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41254/0 (socket says 172.21.15.119:41254) 2024-04-25T10:18:39.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.890+0000 7f3427e00640 1 -- 172.21.15.119:0/1799647674 learned_addr learned my addr 172.21.15.119:0/1799647674 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:39.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.891+0000 7f3427e00640 1 -- 172.21.15.119:0/1799647674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34280ceeb0 msgr2=0x7f34280c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.891+0000 7f3427e00640 1 --2- 172.21.15.119:0/1799647674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34280ceeb0 0x7f34280c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.891+0000 7f3427e00640 1 -- 172.21.15.119:0/1799647674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34280b9720 msgr2=0x7f3428158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.891+0000 7f3427400640 1 --2- 172.21.15.119:0/1799647674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34280b9720 0x7f3428158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.891+0000 7f3427e00640 1 --2- 172.21.15.119:0/1799647674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34280b9720 0x7f3428158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.891+0000 7f3427e00640 1 -- 172.21.15.119:0/1799647674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342815d480 con 0x7f3428161090 2024-04-25T10:18:39.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.891+0000 7f3427e00640 1 --2- 172.21.15.119:0/1799647674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428161090 0x7f34280c9650 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f342000d830 tx=0x7f342000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.891+0000 7f341f400640 1 -- 172.21.15.119:0/1799647674 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342000b6c0 con 0x7f3428161090 2024-04-25T10:18:39.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.891+0000 7f341f400640 1 -- 172.21.15.119:0/1799647674 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3420019070 con 0x7f3428161090 2024-04-25T10:18:39.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.891+0000 7f342e600640 1 -- 172.21.15.119:0/1799647674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34280b4cf0 con 0x7f3428161090 2024-04-25T10:18:39.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.891+0000 7f341f400640 1 -- 172.21.15.119:0/1799647674 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342000bac0 con 0x7f3428161090 2024-04-25T10:18:39.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.891+0000 7f342e600640 1 -- 172.21.15.119:0/1799647674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34280b52b0 con 0x7f3428161090 2024-04-25T10:18:39.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.892+0000 7f342e600640 1 -- 172.21.15.119:0/1799647674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34280750b0 con 0x7f3428161090 2024-04-25T10:18:39.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.893+0000 7f341f400640 1 -- 172.21.15.119:0/1799647674 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f342001e460 con 0x7f3428161090 2024-04-25T10:18:39.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.894+0000 7f341f400640 1 --2- 172.21.15.119:0/1799647674 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f340406b4d0 0x7f340406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.894+0000 7f341f400640 1 -- 172.21.15.119:0/1799647674 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f342008a640 con 0x7f3428161090 2024-04-25T10:18:39.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.895+0000 7f3427400640 1 --2- 172.21.15.119:0/1799647674 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f340406b4d0 0x7f340406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.895+0000 7f3427400640 1 --2- 172.21.15.119:0/1799647674 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f340406b4d0 0x7f340406d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3418034000 tx=0x7f341802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:39.898+0000 7f341f400640 1 -- 172.21.15.119:0/1799647674 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3420059700 con 0x7f3428161090 2024-04-25T10:18:40.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.094+0000 7f342e600640 1 -- 172.21.15.119:0/1799647674 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f342815b5e0 con 0x7f340406b4d0 2024-04-25T10:18:40.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.095+0000 7f341f400640 1 -- 172.21.15.119:0/1799647674 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f342815b5e0 con 0x7f340406b4d0 2024-04-25T10:18:40.098 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T10:18:40.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.097+0000 7f342e600640 1 -- 172.21.15.119:0/1799647674 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f340406b4d0 msgr2=0x7f340406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.097+0000 7f342e600640 1 --2- 172.21.15.119:0/1799647674 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f340406b4d0 0x7f340406d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3418034000 tx=0x7f341802e000 comp rx=0 tx=0).stop 2024-04-25T10:18:40.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.097+0000 7f342e600640 1 -- 172.21.15.119:0/1799647674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428161090 msgr2=0x7f34280c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.097+0000 7f342e600640 1 --2- 172.21.15.119:0/1799647674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428161090 0x7f34280c9650 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f342000d830 tx=0x7f342000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.098+0000 7f342e600640 1 -- 172.21.15.119:0/1799647674 shutdown_connections 2024-04-25T10:18:40.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.098+0000 7f342e600640 1 --2- 172.21.15.119:0/1799647674 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f340406b4d0 0x7f340406d990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.098+0000 7f342e600640 1 --2- 172.21.15.119:0/1799647674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3428161090 0x7f34280c9650 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.098+0000 7f342e600640 1 --2- 172.21.15.119:0/1799647674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34280ceeb0 0x7f34280c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.098+0000 7f342e600640 1 --2- 172.21.15.119:0/1799647674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34280b9720 0x7f3428158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.098+0000 7f342e600640 1 -- 172.21.15.119:0/1799647674 >> 172.21.15.119:0/1799647674 conn(0x7f34280b55e0 msgr2=0x7f34280b8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:40.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.098+0000 7f342e600640 1 -- 172.21.15.119:0/1799647674 shutdown_connections 2024-04-25T10:18:40.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.098+0000 7f342e600640 1 -- 172.21.15.119:0/1799647674 wait complete. 2024-04-25T10:18:40.120 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T10:18:40.120 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:18:40.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.322+0000 7f1a82800640 1 Processor -- start 2024-04-25T10:18:40.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.322+0000 7f1a82800640 1 -- start start 2024-04-25T10:18:40.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.323+0000 7f1a82800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a7c0ceeb0 0x7f1a7c168a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.323+0000 7f1a82800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7c161f80 con 0x7f1a7c0b9720 2024-04-25T10:18:40.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.323+0000 7f1a82800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7c162260 con 0x7f1a7c161090 2024-04-25T10:18:40.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.323+0000 7f1a82800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a7c0d21a0 con 0x7f1a7c0ceeb0 2024-04-25T10:18:40.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.324+0000 7f1a7be00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a7c0ceeb0 0x7f1a7c168a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.324+0000 7f1a7aa00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1a7c0b9720 0x7f1a7c0ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36302/0 (socket says 172.21.15.119:36302) 2024-04-25T10:18:40.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.324+0000 7f1a7aa00640 1 -- 172.21.15.119:0/2243504065 learned_addr learned my addr 172.21.15.119:0/2243504065 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:40.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.324+0000 7f1a7be00640 1 -- 172.21.15.119:0/2243504065 >> v1:172.21.15.132:6789/0 conn(0x7f1a7c161090 legacy=0x7f1a7c161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:40.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.324+0000 7f1a7be00640 1 -- 172.21.15.119:0/2243504065 >> v1:172.21.15.119:6789/0 conn(0x7f1a7c0b9720 legacy=0x7f1a7c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.324+0000 7f1a7be00640 1 -- 172.21.15.119:0/2243504065 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c162540 con 0x7f1a7c0ceeb0 2024-04-25T10:18:40.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.325+0000 7f1a7be00640 1 --2- 172.21.15.119:0/2243504065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7c0ceeb0 0x7f1a7c168a60 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0d2320 tx=0x7f1a7402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f390003097bcab7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.325+0000 7f1a7a000640 1 -- 172.21.15.119:0/2243504065 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74037070 con 0x7f1a7c0ceeb0 2024-04-25T10:18:40.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.325+0000 7f1a7a000640 1 -- 172.21.15.119:0/2243504065 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7402ae40 con 0x7f1a7c0ceeb0 2024-04-25T10:18:40.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.325+0000 7f1a7a000640 1 -- 172.21.15.119:0/2243504065 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74032660 con 0x7f1a7c0ceeb0 2024-04-25T10:18:40.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.326+0000 7f1a82800640 1 -- 172.21.15.119:0/2243504065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7c0ceeb0 msgr2=0x7f1a7c168a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.326+0000 7f1a82800640 1 --2- 172.21.15.119:0/2243504065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7c0ceeb0 0x7f1a7c168a60 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0d2320 tx=0x7f1a7402a130 comp rx=0 tx=0).stop 2024-04-25T10:18:40.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.326+0000 7f1a82800640 1 -- 172.21.15.119:0/2243504065 shutdown_connections 2024-04-25T10:18:40.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.326+0000 7f1a82800640 1 --2- 172.21.15.119:0/2243504065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7c0ceeb0 0x7f1a7c168a60 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.326+0000 7f1a82800640 1 -- 172.21.15.119:0/2243504065 >> 172.21.15.119:0/2243504065 conn(0x7f1a7c0b55e0 msgr2=0x7f1a7c0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:40.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.326+0000 7f1a82800640 1 -- 172.21.15.119:0/2243504065 shutdown_connections 2024-04-25T10:18:40.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.326+0000 7f1a82800640 1 -- 172.21.15.119:0/2243504065 wait complete. 2024-04-25T10:18:40.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.328+0000 7f1a82800640 1 Processor -- start 2024-04-25T10:18:40.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.328+0000 7f1a82800640 1 -- start start 2024-04-25T10:18:40.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.329+0000 7f1a82800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7c0b9720 0x7f1a7c151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.329+0000 7f1a82800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a7c0ceeb0 0x7f1a7c1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.329+0000 7f1a82800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a7c161090 0x7f1a7c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.329+0000 7f1a82800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c0d3640 con 0x7f1a7c0b9720 2024-04-25T10:18:40.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.329+0000 7f1a82800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c161470 con 0x7f1a7c0ceeb0 2024-04-25T10:18:40.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.329+0000 7f1a82800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c0d37c0 con 0x7f1a7c161090 2024-04-25T10:18:40.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.329+0000 7f1a7b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7c0b9720 0x7f1a7c151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.329+0000 7f1a7b400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7c0b9720 0x7f1a7c151b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41270/0 (socket says 172.21.15.119:41270) 2024-04-25T10:18:40.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.329+0000 7f1a7b400640 1 -- 172.21.15.119:0/2891715239 learned_addr learned my addr 172.21.15.119:0/2891715239 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:40.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.330+0000 7f1a7aa00640 1 --2- 172.21.15.119:0/2891715239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a7c0ceeb0 0x7f1a7c1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.330+0000 7f1a7b400640 1 -- 172.21.15.119:0/2891715239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a7c161090 msgr2=0x7f1a7c0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.330+0000 7f1a7b400640 1 --2- 172.21.15.119:0/2891715239 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a7c161090 0x7f1a7c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.330+0000 7f1a7b400640 1 -- 172.21.15.119:0/2891715239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a7c0ceeb0 msgr2=0x7f1a7c1552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.330+0000 7f1a7b400640 1 --2- 172.21.15.119:0/2891715239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a7c0ceeb0 0x7f1a7c1552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.330+0000 7f1a7b400640 1 -- 172.21.15.119:0/2891715239 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c154210 con 0x7f1a7c0b9720 2024-04-25T10:18:40.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.330+0000 7f1a7aa00640 1 --2- 172.21.15.119:0/2891715239 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a7c0ceeb0 0x7f1a7c1552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:40.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.330+0000 7f1a7b400640 1 --2- 172.21.15.119:0/2891715239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7c0b9720 0x7f1a7c151b00 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00c7c0 tx=0x7f1a6c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.330+0000 7f1a73400640 1 -- 172.21.15.119:0/2891715239 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c016020 con 0x7f1a7c0b9720 2024-04-25T10:18:40.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.330+0000 7f1a73400640 1 -- 172.21.15.119:0/2891715239 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6c00aa50 con 0x7f1a7c0b9720 2024-04-25T10:18:40.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.330+0000 7f1a82800640 1 -- 172.21.15.119:0/2891715239 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7c1544f0 con 0x7f1a7c0b9720 2024-04-25T10:18:40.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.331+0000 7f1a73400640 1 -- 172.21.15.119:0/2891715239 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c00ae40 con 0x7f1a7c0b9720 2024-04-25T10:18:40.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.331+0000 7f1a82800640 1 -- 172.21.15.119:0/2891715239 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7c154ab0 con 0x7f1a7c0b9720 2024-04-25T10:18:40.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.331+0000 7f1a82800640 1 -- 172.21.15.119:0/2891715239 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a7c0750b0 con 0x7f1a7c0b9720 2024-04-25T10:18:40.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.333+0000 7f1a73400640 1 -- 172.21.15.119:0/2891715239 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a6c019020 con 0x7f1a7c0b9720 2024-04-25T10:18:40.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.334+0000 7f1a73400640 1 --2- 172.21.15.119:0/2891715239 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a5806b4d0 0x7f1a5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.334+0000 7f1a73400640 1 -- 172.21.15.119:0/2891715239 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f1a6c089dd0 con 0x7f1a7c0b9720 2024-04-25T10:18:40.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.334+0000 7f1a7aa00640 1 --2- 172.21.15.119:0/2891715239 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a5806b4d0 0x7f1a5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.334+0000 7f1a73400640 1 --2- 172.21.15.119:0/2891715239 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x7f1a580714d0 0x7f1a580738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.334+0000 7f1a73400640 1 -- 172.21.15.119:0/2891715239 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a58073fa0 con 0x7f1a580714d0 2024-04-25T10:18:40.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.334+0000 7f1a73400640 1 -- 172.21.15.119:0/2891715239 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a6c08d020 con 0x7f1a7c0b9720 2024-04-25T10:18:40.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.335+0000 7f1a7aa00640 1 --2- 172.21.15.119:0/2891715239 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a5806b4d0 0x7f1a5806d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c152040 tx=0x7f1a64009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.335+0000 7f1a7be00640 1 --2- 172.21.15.119:0/2891715239 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x7f1a580714d0 0x7f1a580738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.336+0000 7f1a7be00640 1 --2- 172.21.15.119:0/2891715239 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x7f1a580714d0 0x7f1a580738f0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.338+0000 7f1a73400640 1 -- 172.21.15.119:0/2891715239 <== osd.6 v2:172.21.15.132:6806/353653608 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1a58073fa0 con 0x7f1a580714d0 2024-04-25T10:18:40.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.352+0000 7f1a82800640 1 -- 172.21.15.119:0/2891715239 --> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1a7c0d1700 con 0x7f1a580714d0 2024-04-25T10:18:40.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.353+0000 7f1a73400640 1 -- 172.21.15.119:0/2891715239 <== osd.6 v2:172.21.15.132:6806/353653608 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1a7c0d1700 con 0x7f1a580714d0 2024-04-25T10:18:40.357 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:18:40.357 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:18:40.357 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:18:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.354+0000 7f1a82800640 1 -- 172.21.15.119:0/2891715239 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x7f1a580714d0 msgr2=0x7f1a580738f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.354+0000 7f1a82800640 1 --2- 172.21.15.119:0/2891715239 >> [v2:172.21.15.132:6806/353653608,v1:172.21.15.132:6808/353653608] conn(0x7f1a580714d0 0x7f1a580738f0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.354+0000 7f1a82800640 1 -- 172.21.15.119:0/2891715239 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a5806b4d0 msgr2=0x7f1a5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.354+0000 7f1a82800640 1 --2- 172.21.15.119:0/2891715239 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f1a5806b4d0 0x7f1a5806d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c152040 tx=0x7f1a64009040 comp rx=0 tx=0).stop 2024-04-25T10:18:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.354+0000 7f1a82800640 1 -- 172.21.15.119:0/2891715239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7c0b9720 msgr2=0x7f1a7c151b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.354+0000 7f1a82800640 1 --2- 172.21.15.119:0/2891715239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7c0b9720 0x7f1a7c151b00 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00c7c0 tx=0x7f1a6c00cc80 comp rx=0 tx=0).stop 2024-04-25T10:18:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.354+0000 7f1a7be00640 1 -- 172.21.15.119:0/2891715239 reap_dead start 2024-04-25T10:18:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.354+0000 7f1a82800640 1 -- 172.21.15.119:0/2891715239 shutdown_connections 2024-04-25T10:18:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.354+0000 7f1a82800640 1 -- 172.21.15.119:0/2891715239 >> 172.21.15.119:0/2891715239 conn(0x7f1a7c0b55e0 msgr2=0x7f1a7c06c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.354+0000 7f1a82800640 1 -- 172.21.15.119:0/2891715239 shutdown_connections 2024-04-25T10:18:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.354+0000 7f1a82800640 1 -- 172.21.15.119:0/2891715239 wait complete. 2024-04-25T10:18:40.377 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T10:18:40.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.578+0000 7f51cee00640 1 Processor -- start 2024-04-25T10:18:40.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.579+0000 7f51cee00640 1 -- start start 2024-04-25T10:18:40.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.580+0000 7f51cee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51c80b9580 0x7f51c81618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.580+0000 7f51cee00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c8162040 con 0x7f51c80cef40 2024-04-25T10:18:40.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.580+0000 7f51cee00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c8162320 con 0x7f51c81610b0 2024-04-25T10:18:40.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.580+0000 7f51cee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80b9a30 con 0x7f51c80b9580 2024-04-25T10:18:40.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.580+0000 7f51c7400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f51c80cef40 0x7f51c80d2610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36310/0 (socket says 172.21.15.119:36310) 2024-04-25T10:18:40.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.580+0000 7f51c7400640 1 -- 172.21.15.119:0/2313413072 learned_addr learned my addr 172.21.15.119:0/2313413072 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:40.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.581+0000 7f51cce00640 1 --2- 172.21.15.119:0/2313413072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51c80b9580 0x7f51c81618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.581+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284084008 0 0) 0x7f51c8162040 con 0x7f51c80cef40 2024-04-25T10:18:40.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.581+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51ac003520 con 0x7f51c80cef40 2024-04-25T10:18:40.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.581+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642337164 0 0) 0x7f51c8162320 con 0x7f51c81610b0 2024-04-25T10:18:40.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.581+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51c8162040 con 0x7f51c81610b0 2024-04-25T10:18:40.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.581+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2357989105 0 0) 0x7f51ac003520 con 0x7f51c80cef40 2024-04-25T10:18:40.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.581+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51c80b9580 msgr2=0x7f51c81618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:40.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.581+0000 7f51c6a00640 1 --2- 172.21.15.119:0/2313413072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51c80b9580 0x7f51c81618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.581+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 >> v1:172.21.15.132:6789/0 conn(0x7f51c81610b0 legacy=0x7f51c80ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.581+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c8162600 con 0x7f51c80cef40 2024-04-25T10:18:40.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.581+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f51b0001670 con 0x7f51c80cef40 2024-04-25T10:18:40.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.582+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 >> v1:172.21.15.119:6789/0 conn(0x7f51c80cef40 legacy=0x7f51c80d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.582+0000 7f51c6a00640 1 --2- 172.21.15.119:0/2313413072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ac003b70 0x7f51ac003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.582+0000 7f51c6a00640 1 --2- 172.21.15.119:0/2313413072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51c80b9580 0x7f51ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.582+0000 7f51c6a00640 1 --2- 172.21.15.119:0/2313413072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51ac004d40 0x7f51ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.582+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80b9a30 con 0x7f51ac003b70 2024-04-25T10:18:40.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.582+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac00bb50 con 0x7f51ac004d40 2024-04-25T10:18:40.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.582+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac00bd20 con 0x7f51c80b9580 2024-04-25T10:18:40.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.582+0000 7f51c7e00640 1 --2- 172.21.15.119:0/2313413072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ac003b70 0x7f51ac003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.582+0000 7f51c7e00640 1 -- 172.21.15.119:0/2313413072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51c80b9580 msgr2=0x7f51ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:40.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.582+0000 7f51c7e00640 1 --2- 172.21.15.119:0/2313413072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51c80b9580 0x7f51ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.582+0000 7f51c7e00640 1 -- 172.21.15.119:0/2313413072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51ac004d40 msgr2=0x7f51ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:40.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.582+0000 7f51c7e00640 1 --2- 172.21.15.119:0/2313413072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51ac004d40 0x7f51ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.582+0000 7f51c7e00640 1 -- 172.21.15.119:0/2313413072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac003520 con 0x7f51ac003b70 2024-04-25T10:18:40.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.583+0000 7f51c7e00640 1 --2- 172.21.15.119:0/2313413072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ac003b70 0x7f51ac003f70 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f51b8002000 tx=0x7f51ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278736ce1e21ed64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.584+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8001bf0 con 0x7f51ac003b70 2024-04-25T10:18:40.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.584+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b8003b70 con 0x7f51ac003b70 2024-04-25T10:18:40.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.584+0000 7f51c6a00640 1 -- 172.21.15.119:0/2313413072 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b80050d0 con 0x7f51ac003b70 2024-04-25T10:18:40.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.584+0000 7f51cee00640 1 -- 172.21.15.119:0/2313413072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ac003b70 msgr2=0x7f51ac003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.584+0000 7f51cee00640 1 --2- 172.21.15.119:0/2313413072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ac003b70 0x7f51ac003f70 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f51b8002000 tx=0x7f51ac005830 comp rx=0 tx=0).stop 2024-04-25T10:18:40.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.584+0000 7f51cce00640 1 -- 172.21.15.119:0/2313413072 reap_dead start 2024-04-25T10:18:40.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.584+0000 7f51cee00640 1 -- 172.21.15.119:0/2313413072 shutdown_connections 2024-04-25T10:18:40.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.584+0000 7f51cee00640 1 -- 172.21.15.119:0/2313413072 >> 172.21.15.119:0/2313413072 conn(0x7f51c80b5570 msgr2=0x7f51c80b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:40.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.584+0000 7f51cee00640 1 -- 172.21.15.119:0/2313413072 shutdown_connections 2024-04-25T10:18:40.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.584+0000 7f51cee00640 1 -- 172.21.15.119:0/2313413072 wait complete. 2024-04-25T10:18:40.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.586+0000 7f51cee00640 1 Processor -- start 2024-04-25T10:18:40.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.586+0000 7f51cee00640 1 -- start start 2024-04-25T10:18:40.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.587+0000 7f51cee00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ac003b70 0x7f51c8154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.587+0000 7f51cee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51c8151e80 0x7f51c8153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.587+0000 7f51cee00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51c80c9570 0x7f51c8156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.587+0000 7f51cee00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80b6c00 con 0x7f51ac003b70 2024-04-25T10:18:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.587+0000 7f51cee00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80d1ea0 con 0x7f51c80c9570 2024-04-25T10:18:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.587+0000 7f51cee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c8161490 con 0x7f51c8151e80 2024-04-25T10:18:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.587+0000 7f51c7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ac003b70 0x7f51c8154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.587+0000 7f51c7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ac003b70 0x7f51c8154b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41294/0 (socket says 172.21.15.119:41294) 2024-04-25T10:18:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.587+0000 7f51c7e00640 1 -- 172.21.15.119:0/702448468 learned_addr learned my addr 172.21.15.119:0/702448468 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.588+0000 7f51c7400640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51c8151e80 0x7f51c8153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.588+0000 7f51cce00640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51c80c9570 0x7f51c8156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.588+0000 7f51c7e00640 1 -- 172.21.15.119:0/702448468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51c8151e80 msgr2=0x7f51c8153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.588+0000 7f51c7e00640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51c8151e80 0x7f51c8153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.588+0000 7f51c7e00640 1 -- 172.21.15.119:0/702448468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51c80c9570 msgr2=0x7f51c8156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.588+0000 7f51c7e00640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51c80c9570 0x7f51c8156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.588+0000 7f51c7e00640 1 -- 172.21.15.119:0/702448468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c8153aa0 con 0x7f51ac003b70 2024-04-25T10:18:40.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.588+0000 7f51c7400640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51c8151e80 0x7f51c8153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:40.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.588+0000 7f51c7e00640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ac003b70 0x7f51c8154b40 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f51b8005040 tx=0x7f51b8001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.588+0000 7f51bfe00640 1 -- 172.21.15.119:0/702448468 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8040070 con 0x7f51ac003b70 2024-04-25T10:18:40.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.589+0000 7f51bfe00640 1 -- 172.21.15.119:0/702448468 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b8003970 con 0x7f51ac003b70 2024-04-25T10:18:40.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.589+0000 7f51cee00640 1 -- 172.21.15.119:0/702448468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c8153d80 con 0x7f51ac003b70 2024-04-25T10:18:40.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.589+0000 7f51bfe00640 1 -- 172.21.15.119:0/702448468 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8001bf0 con 0x7f51ac003b70 2024-04-25T10:18:40.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.589+0000 7f51cee00640 1 -- 172.21.15.119:0/702448468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c8154340 con 0x7f51ac003b70 2024-04-25T10:18:40.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.590+0000 7f51cee00640 1 -- 172.21.15.119:0/702448468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c80750b0 con 0x7f51ac003b70 2024-04-25T10:18:40.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.591+0000 7f51bfe00640 1 -- 172.21.15.119:0/702448468 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51b8043070 con 0x7f51ac003b70 2024-04-25T10:18:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.591+0000 7f51bfe00640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f51a406b4d0 0x7f51a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.592+0000 7f51bfe00640 1 -- 172.21.15.119:0/702448468 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f51b80aafd0 con 0x7f51ac003b70 2024-04-25T10:18:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.592+0000 7f51c7400640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f51a406b4d0 0x7f51a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.592+0000 7f51c7400640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f51a406b4d0 0x7f51a406d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f51b0000fb0 tx=0x7f51b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.596+0000 7f51bfe00640 1 -- 172.21.15.119:0/702448468 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51b807a130 con 0x7f51ac003b70 2024-04-25T10:18:40.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.793+0000 7f51cee00640 1 -- 172.21.15.119:0/702448468 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f51c8153840 con 0x7f51a406b4d0 2024-04-25T10:18:40.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.794+0000 7f51bfe00640 1 -- 172.21.15.119:0/702448468 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f51b00070d0 con 0x7f51a406b4d0 2024-04-25T10:18:40.797 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T10:18:40.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.797+0000 7f51cee00640 1 -- 172.21.15.119:0/702448468 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f51a406b4d0 msgr2=0x7f51a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.797+0000 7f51cee00640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f51a406b4d0 0x7f51a406d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f51b0000fb0 tx=0x7f51b0009040 comp rx=0 tx=0).stop 2024-04-25T10:18:40.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.797+0000 7f51cee00640 1 -- 172.21.15.119:0/702448468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ac003b70 msgr2=0x7f51c8154b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.797+0000 7f51cee00640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ac003b70 0x7f51c8154b40 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f51b8005040 tx=0x7f51b8001650 comp rx=0 tx=0).stop 2024-04-25T10:18:40.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.797+0000 7f51cee00640 1 -- 172.21.15.119:0/702448468 shutdown_connections 2024-04-25T10:18:40.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.797+0000 7f51cee00640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f51a406b4d0 0x7f51a406d990 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.797+0000 7f51cee00640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51c80c9570 0x7f51c8156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.797+0000 7f51cee00640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f51c8151e80 0x7f51c8153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.797+0000 7f51cee00640 1 --2- 172.21.15.119:0/702448468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ac003b70 0x7f51c8154b40 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.797+0000 7f51cee00640 1 -- 172.21.15.119:0/702448468 >> 172.21.15.119:0/702448468 conn(0x7f51c80b5570 msgr2=0x7f51c80ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:40.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.797+0000 7f51cee00640 1 -- 172.21.15.119:0/702448468 shutdown_connections 2024-04-25T10:18:40.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:40.797+0000 7f51cee00640 1 -- 172.21.15.119:0/702448468 wait complete. 2024-04-25T10:18:40.817 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T10:18:40.818 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:18:41.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.021+0000 7f242d800640 1 Processor -- start 2024-04-25T10:18:41.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.021+0000 7f242d800640 1 -- start start 2024-04-25T10:18:41.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.023+0000 7f242d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2428071660 0x7f24280cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.023+0000 7f242d800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280cc9a0 con 0x7f2428145260 2024-04-25T10:18:41.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.023+0000 7f242d800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280ccc80 con 0x7f2428071200 2024-04-25T10:18:41.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.023+0000 7f242d800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24280cea00 con 0x7f2428071660 2024-04-25T10:18:41.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.023+0000 7f2426a00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2428145260 0x7f24280cbf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40760/0 (socket says 172.21.15.119:40760) 2024-04-25T10:18:41.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.023+0000 7f2426a00640 1 -- 172.21.15.119:0/135005662 learned_addr learned my addr 172.21.15.119:0/135005662 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:41.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 -- 172.21.15.119:0/135005662 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830069780 0 0) 0x7f24280cc9a0 con 0x7f2428145260 2024-04-25T10:18:41.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 -- 172.21.15.119:0/135005662 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f240c003520 con 0x7f2428145260 2024-04-25T10:18:41.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2427400640 1 --2- 172.21.15.119:0/135005662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2428071660 0x7f24280cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 -- 172.21.15.119:0/135005662 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 758840401 0 0) 0x7f240c003520 con 0x7f2428145260 2024-04-25T10:18:41.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 -- 172.21.15.119:0/135005662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2428071660 msgr2=0x7f24280cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 --2- 172.21.15.119:0/135005662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2428071660 0x7f24280cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 -- 172.21.15.119:0/135005662 >> v1:172.21.15.132:6789/0 conn(0x7f2428071200 legacy=0x7f24280cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 -- 172.21.15.119:0/135005662 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280ccf60 con 0x7f2428145260 2024-04-25T10:18:41.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 -- 172.21.15.119:0/135005662 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7f2418001670 con 0x7f2428145260 2024-04-25T10:18:41.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 -- 172.21.15.119:0/135005662 >> v1:172.21.15.119:6789/0 conn(0x7f2428145260 legacy=0x7f24280cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 --2- 172.21.15.119:0/135005662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f240c0039e0 0x7f240c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 --2- 172.21.15.119:0/135005662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2428071660 0x7f240c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 --2- 172.21.15.119:0/135005662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240c004d40 0x7f240c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 -- 172.21.15.119:0/135005662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280cea00 con 0x7f240c0039e0 2024-04-25T10:18:41.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 -- 172.21.15.119:0/135005662 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c00bad0 con 0x7f240c004d40 2024-04-25T10:18:41.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.024+0000 7f2426000640 1 -- 172.21.15.119:0/135005662 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240c00bcd0 con 0x7f2428071660 2024-04-25T10:18:41.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.025+0000 7f2426a00640 1 --2- 172.21.15.119:0/135005662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f240c0039e0 0x7f240c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.025+0000 7f2427400640 1 --2- 172.21.15.119:0/135005662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240c004d40 0x7f240c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.025+0000 7f2426a00640 1 -- 172.21.15.119:0/135005662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2428071660 msgr2=0x7f240c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.025+0000 7f2426a00640 1 --2- 172.21.15.119:0/135005662 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2428071660 0x7f240c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.025+0000 7f2426a00640 1 -- 172.21.15.119:0/135005662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240c004d40 msgr2=0x7f240c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.025+0000 7f2426a00640 1 --2- 172.21.15.119:0/135005662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240c004d40 0x7f240c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.025+0000 7f2426a00640 1 -- 172.21.15.119:0/135005662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c003520 con 0x7f240c0039e0 2024-04-25T10:18:41.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.026+0000 7f2426a00640 1 --2- 172.21.15.119:0/135005662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f240c0039e0 0x7f240c003e20 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f2418007fd0 tx=0x7f240c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83047b99aa8d98ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.026+0000 7f2426000640 1 -- 172.21.15.119:0/135005662 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418007c20 con 0x7f240c0039e0 2024-04-25T10:18:41.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.026+0000 7f2426000640 1 -- 172.21.15.119:0/135005662 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418005af0 con 0x7f240c0039e0 2024-04-25T10:18:41.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.026+0000 7f2426000640 1 -- 172.21.15.119:0/135005662 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418007070 con 0x7f240c0039e0 2024-04-25T10:18:41.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.027+0000 7f242d800640 1 -- 172.21.15.119:0/135005662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f240c0039e0 msgr2=0x7f240c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.027+0000 7f242d800640 1 --2- 172.21.15.119:0/135005662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f240c0039e0 0x7f240c003e20 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f2418007fd0 tx=0x7f240c005830 comp rx=0 tx=0).stop 2024-04-25T10:18:41.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.027+0000 7f2427400640 1 -- 172.21.15.119:0/135005662 reap_dead start 2024-04-25T10:18:41.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.027+0000 7f242d800640 1 -- 172.21.15.119:0/135005662 shutdown_connections 2024-04-25T10:18:41.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.027+0000 7f242d800640 1 -- 172.21.15.119:0/135005662 >> 172.21.15.119:0/135005662 conn(0x7f242806aba0 msgr2=0x7f242806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.027+0000 7f242d800640 1 -- 172.21.15.119:0/135005662 shutdown_connections 2024-04-25T10:18:41.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.027+0000 7f242d800640 1 -- 172.21.15.119:0/135005662 wait complete. 2024-04-25T10:18:41.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.029+0000 7f242d800640 1 Processor -- start 2024-04-25T10:18:41.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.029+0000 7f242d800640 1 -- start start 2024-04-25T10:18:41.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.030+0000 7f242d800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240c0039e0 0x7f2428141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.030+0000 7f242d800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f242813f6d0 0x7f242815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.030+0000 7f242d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24280ac8b0 0x7f24280acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.030+0000 7f242d800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280cc660 con 0x7f242813f6d0 2024-04-25T10:18:41.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.030+0000 7f242d800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2428071ad0 con 0x7f240c0039e0 2024-04-25T10:18:41.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.030+0000 7f242d800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2428145640 con 0x7f24280ac8b0 2024-04-25T10:18:41.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.030+0000 7f241e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f242813f6d0 0x7f242815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.031+0000 7f241e000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f242813f6d0 0x7f242815b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45996/0 (socket says 172.21.15.119:45996) 2024-04-25T10:18:41.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.031+0000 7f241e000640 1 -- 172.21.15.119:0/477714421 learned_addr learned my addr 172.21.15.119:0/477714421 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:41.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.031+0000 7f2426a00640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240c0039e0 0x7f2428141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.031+0000 7f2427400640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24280ac8b0 0x7f24280acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.031+0000 7f241e000640 1 -- 172.21.15.119:0/477714421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24280ac8b0 msgr2=0x7f24280acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.031+0000 7f241e000640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24280ac8b0 0x7f24280acd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.031+0000 7f241e000640 1 -- 172.21.15.119:0/477714421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240c0039e0 msgr2=0x7f2428141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.031+0000 7f241e000640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240c0039e0 0x7f2428141120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.031+0000 7f241e000640 1 -- 172.21.15.119:0/477714421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280af400 con 0x7f242813f6d0 2024-04-25T10:18:41.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.031+0000 7f2427400640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24280ac8b0 0x7f24280acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:41.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.031+0000 7f241e000640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f242813f6d0 0x7f242815b240 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f241000c530 tx=0x7f241000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.032+0000 7f241f400640 1 -- 172.21.15.119:0/477714421 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24100052d0 con 0x7f242813f6d0 2024-04-25T10:18:41.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.032+0000 7f241f400640 1 -- 172.21.15.119:0/477714421 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2410005470 con 0x7f242813f6d0 2024-04-25T10:18:41.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.032+0000 7f242d800640 1 -- 172.21.15.119:0/477714421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2428141870 con 0x7f242813f6d0 2024-04-25T10:18:41.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.032+0000 7f241f400640 1 -- 172.21.15.119:0/477714421 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2410012870 con 0x7f242813f6d0 2024-04-25T10:18:41.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.032+0000 7f242d800640 1 -- 172.21.15.119:0/477714421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24280af6a0 con 0x7f242813f6d0 2024-04-25T10:18:41.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.033+0000 7f242d800640 1 -- 172.21.15.119:0/477714421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f24280750b0 con 0x7f242813f6d0 2024-04-25T10:18:41.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.034+0000 7f241f400640 1 -- 172.21.15.119:0/477714421 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2410012a10 con 0x7f242813f6d0 2024-04-25T10:18:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.035+0000 7f241f400640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f240406b500 0x7f240406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.035+0000 7f241f400640 1 -- 172.21.15.119:0/477714421 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f2410089bd0 con 0x7f242813f6d0 2024-04-25T10:18:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.035+0000 7f2426a00640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f240406b500 0x7f240406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.035+0000 7f241f400640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x7f2404071500 0x7f2404073920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.035+0000 7f241f400640 1 -- 172.21.15.119:0/477714421 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2404073fd0 con 0x7f2404071500 2024-04-25T10:18:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.035+0000 7f241f400640 1 -- 172.21.15.119:0/477714421 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f241008f2a0 con 0x7f242813f6d0 2024-04-25T10:18:41.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.036+0000 7f2427400640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x7f2404071500 0x7f2404073920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.036+0000 7f2426a00640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f240406b500 0x7f240406d9c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2418032000 tx=0x7f2418009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.037+0000 7f2427400640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x7f2404071500 0x7f2404073920 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.039+0000 7f241f400640 1 -- 172.21.15.119:0/477714421 <== osd.7 v2:172.21.15.132:6821/2727385616 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2404073fd0 con 0x7f2404071500 2024-04-25T10:18:41.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.054+0000 7f242d800640 1 -- 172.21.15.119:0/477714421 --> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2428067d30 con 0x7f2404071500 2024-04-25T10:18:41.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.055+0000 7f241f400640 1 -- 172.21.15.119:0/477714421 <== osd.7 v2:172.21.15.132:6821/2727385616 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2428067d30 con 0x7f2404071500 2024-04-25T10:18:41.058 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-25T10:18:41.058 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:18:41.058 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-25T10:18:41.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.055+0000 7f242d800640 1 -- 172.21.15.119:0/477714421 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x7f2404071500 msgr2=0x7f2404073920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.055+0000 7f242d800640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.132:6821/2727385616,v1:172.21.15.132:6823/2727385616] conn(0x7f2404071500 0x7f2404073920 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.055+0000 7f242d800640 1 -- 172.21.15.119:0/477714421 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f240406b500 msgr2=0x7f240406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.055+0000 7f242d800640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f240406b500 0x7f240406d9c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2418032000 tx=0x7f2418009030 comp rx=0 tx=0).stop 2024-04-25T10:18:41.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.055+0000 7f242d800640 1 -- 172.21.15.119:0/477714421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f242813f6d0 msgr2=0x7f242815b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.055+0000 7f242d800640 1 --2- 172.21.15.119:0/477714421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f242813f6d0 0x7f242815b240 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f241000c530 tx=0x7f241000c9f0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.055+0000 7f2427400640 1 -- 172.21.15.119:0/477714421 reap_dead start 2024-04-25T10:18:41.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.056+0000 7f242d800640 1 -- 172.21.15.119:0/477714421 shutdown_connections 2024-04-25T10:18:41.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.056+0000 7f242d800640 1 -- 172.21.15.119:0/477714421 >> 172.21.15.119:0/477714421 conn(0x7f242806aba0 msgr2=0x7f242815d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.056+0000 7f242d800640 1 -- 172.21.15.119:0/477714421 shutdown_connections 2024-04-25T10:18:41.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.056+0000 7f242d800640 1 -- 172.21.15.119:0/477714421 wait complete. 2024-04-25T10:18:41.079 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T10:18:41.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.278+0000 7fc944e00640 1 Processor -- start 2024-04-25T10:18:41.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.278+0000 7fc944e00640 1 -- start start 2024-04-25T10:18:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.280+0000 7fc944e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc94015ae20 0x7fc9400b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.280+0000 7fc944e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94006da40 con 0x7fc9400b4730 2024-04-25T10:18:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.280+0000 7fc944e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94006dd20 con 0x7fc9400aedc0 2024-04-25T10:18:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.280+0000 7fc944e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc94015c050 con 0x7fc94015ae20 2024-04-25T10:18:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.280+0000 7fc93e800640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc9400b4730 0x7fc94015c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40766/0 (socket says 172.21.15.119:40766) 2024-04-25T10:18:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.280+0000 7fc93e800640 1 -- 172.21.15.119:0/3232411594 learned_addr learned my addr 172.21.15.119:0/3232411594 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.280+0000 7fc937e00640 1 --2- 172.21.15.119:0/3232411594 >> v2:172.21.15.132:3300/0 conn(0x7fc94015ae20 0x7fc9400b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.280+0000 7fc93d400640 1 -- 172.21.15.119:0/3232411594 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890227259 0 0) 0x7fc94006da40 con 0x7fc9400b4730 2024-04-25T10:18:41.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93d400640 1 -- 172.21.15.119:0/3232411594 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc924003520 con 0x7fc9400b4730 2024-04-25T10:18:41.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93d400640 1 -- 172.21.15.119:0/3232411594 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3684849321 0 0) 0x7fc924003520 con 0x7fc9400b4730 2024-04-25T10:18:41.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93d400640 1 -- 172.21.15.119:0/3232411594 >> v2:172.21.15.132:3300/0 conn(0x7fc94015ae20 msgr2=0x7fc9400b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93d400640 1 --2- 172.21.15.119:0/3232411594 >> v2:172.21.15.132:3300/0 conn(0x7fc94015ae20 0x7fc9400b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93d400640 1 -- 172.21.15.119:0/3232411594 >> v1:172.21.15.132:6789/0 conn(0x7fc9400aedc0 legacy=0x7fc94006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93d400640 1 -- 172.21.15.119:0/3232411594 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94006e000 con 0x7fc9400b4730 2024-04-25T10:18:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93d400640 1 -- 172.21.15.119:0/3232411594 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fc9380019f0 con 0x7fc9400b4730 2024-04-25T10:18:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc937e00640 1 --2- 172.21.15.119:0/3232411594 >> v2:172.21.15.132:3300/0 conn(0x7fc94015ae20 0x7fc9400b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93d400640 1 -- 172.21.15.119:0/3232411594 >> v1:172.21.15.119:6789/0 conn(0x7fc9400b4730 legacy=0x7fc94015c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93d400640 1 --2- 172.21.15.119:0/3232411594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9240039b0 0x7fc924003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93d400640 1 --2- 172.21.15.119:0/3232411594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc924004800 0x7fc924004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93d400640 1 --2- 172.21.15.119:0/3232411594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9240052e0 0x7fc92400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93d400640 1 -- 172.21.15.119:0/3232411594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94015c050 con 0x7fc9240052e0 2024-04-25T10:18:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93d400640 1 -- 172.21.15.119:0/3232411594 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92400e3a0 con 0x7fc9240039b0 2024-04-25T10:18:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93d400640 1 -- 172.21.15.119:0/3232411594 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92400e5a0 con 0x7fc924004800 2024-04-25T10:18:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.281+0000 7fc93e800640 1 --2- 172.21.15.119:0/3232411594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9240052e0 0x7fc92400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.282+0000 7fc93de00640 1 --2- 172.21.15.119:0/3232411594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9240039b0 0x7fc924003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.282+0000 7fc937e00640 1 --2- 172.21.15.119:0/3232411594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc924004800 0x7fc924004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.282+0000 7fc93e800640 1 -- 172.21.15.119:0/3232411594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc924004800 msgr2=0x7fc924004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.282+0000 7fc93e800640 1 --2- 172.21.15.119:0/3232411594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc924004800 0x7fc924004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.282+0000 7fc93e800640 1 -- 172.21.15.119:0/3232411594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9240039b0 msgr2=0x7fc924003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.282+0000 7fc93e800640 1 --2- 172.21.15.119:0/3232411594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9240039b0 0x7fc924003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.282+0000 7fc93e800640 1 -- 172.21.15.119:0/3232411594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc924003520 con 0x7fc9240052e0 2024-04-25T10:18:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.283+0000 7fc93e800640 1 -- 172.21.15.119:0/3232411594 reap_dead start 2024-04-25T10:18:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.283+0000 7fc93e800640 1 --2- 172.21.15.119:0/3232411594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9240052e0 0x7fc92400dbd0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc93800a830 tx=0x7fc924005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40ab8bee7a59805 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.283+0000 7fc93d400640 1 -- 172.21.15.119:0/3232411594 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93800a480 con 0x7fc9240052e0 2024-04-25T10:18:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.283+0000 7fc93d400640 1 -- 172.21.15.119:0/3232411594 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93802f070 con 0x7fc9240052e0 2024-04-25T10:18:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.283+0000 7fc93d400640 1 -- 172.21.15.119:0/3232411594 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93802bc50 con 0x7fc9240052e0 2024-04-25T10:18:41.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.284+0000 7fc944e00640 1 -- 172.21.15.119:0/3232411594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9240052e0 msgr2=0x7fc92400dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.284+0000 7fc944e00640 1 --2- 172.21.15.119:0/3232411594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9240052e0 0x7fc92400dbd0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc93800a830 tx=0x7fc924005830 comp rx=0 tx=0).stop 2024-04-25T10:18:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.284+0000 7fc944e00640 1 -- 172.21.15.119:0/3232411594 shutdown_connections 2024-04-25T10:18:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.284+0000 7fc944e00640 1 --2- 172.21.15.119:0/3232411594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9240052e0 0x7fc92400dbd0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.284+0000 7fc944e00640 1 -- 172.21.15.119:0/3232411594 >> 172.21.15.119:0/3232411594 conn(0x7fc94006aba0 msgr2=0x7fc94006af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.284+0000 7fc944e00640 1 -- 172.21.15.119:0/3232411594 shutdown_connections 2024-04-25T10:18:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.284+0000 7fc944e00640 1 -- 172.21.15.119:0/3232411594 wait complete. 2024-04-25T10:18:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.286+0000 7fc944e00640 1 Processor -- start 2024-04-25T10:18:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.286+0000 7fc944e00640 1 -- start start 2024-04-25T10:18:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.287+0000 7fc944e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9240052e0 0x7fc94014e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.287+0000 7fc944e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9400c7340 0x7fc9400b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.287+0000 7fc944e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9400b3920 0x7fc9400ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.287+0000 7fc944e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400af1a0 con 0x7fc9240052e0 2024-04-25T10:18:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.287+0000 7fc944e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400b4b70 con 0x7fc9400c7340 2024-04-25T10:18:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.287+0000 7fc944e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400b41f0 con 0x7fc9400b3920 2024-04-25T10:18:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.287+0000 7fc93de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9240052e0 0x7fc94014e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.287+0000 7fc93de00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9240052e0 0x7fc94014e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46012/0 (socket says 172.21.15.119:46012) 2024-04-25T10:18:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.287+0000 7fc93de00640 1 -- 172.21.15.119:0/4046003227 learned_addr learned my addr 172.21.15.119:0/4046003227 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.288+0000 7fc93de00640 1 -- 172.21.15.119:0/4046003227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9400b3920 msgr2=0x7fc9400ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.288+0000 7fc93de00640 1 --2- 172.21.15.119:0/4046003227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9400b3920 0x7fc9400ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.288+0000 7fc93de00640 1 -- 172.21.15.119:0/4046003227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9400c7340 msgr2=0x7fc9400b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.288+0000 7fc937e00640 1 --2- 172.21.15.119:0/4046003227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9400c7340 0x7fc9400b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.288+0000 7fc93de00640 1 --2- 172.21.15.119:0/4046003227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9400c7340 0x7fc9400b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.288+0000 7fc93de00640 1 -- 172.21.15.119:0/4046003227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94014e880 con 0x7fc9240052e0 2024-04-25T10:18:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.288+0000 7fc93de00640 1 --2- 172.21.15.119:0/4046003227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9240052e0 0x7fc94014e340 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fc93000e490 tx=0x7fc93000e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.288+0000 7fc936000640 1 -- 172.21.15.119:0/4046003227 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9300135f0 con 0x7fc9240052e0 2024-04-25T10:18:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.288+0000 7fc936000640 1 -- 172.21.15.119:0/4046003227 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93001a070 con 0x7fc9240052e0 2024-04-25T10:18:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.288+0000 7fc936000640 1 -- 172.21.15.119:0/4046003227 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9300139d0 con 0x7fc9240052e0 2024-04-25T10:18:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.288+0000 7fc944e00640 1 -- 172.21.15.119:0/4046003227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9400c48b0 con 0x7fc9240052e0 2024-04-25T10:18:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.288+0000 7fc944e00640 1 -- 172.21.15.119:0/4046003227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9400c4e70 con 0x7fc9240052e0 2024-04-25T10:18:41.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.290+0000 7fc944e00640 1 -- 172.21.15.119:0/4046003227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9400750b0 con 0x7fc9240052e0 2024-04-25T10:18:41.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.291+0000 7fc936000640 1 -- 172.21.15.119:0/4046003227 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc930013b70 con 0x7fc9240052e0 2024-04-25T10:18:41.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.292+0000 7fc936000640 1 --2- 172.21.15.119:0/4046003227 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc91c06b550 0x7fc91c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.292+0000 7fc936000640 1 -- 172.21.15.119:0/4046003227 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fc930089e30 con 0x7fc9240052e0 2024-04-25T10:18:41.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.292+0000 7fc937e00640 1 --2- 172.21.15.119:0/4046003227 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc91c06b550 0x7fc91c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.293+0000 7fc937e00640 1 --2- 172.21.15.119:0/4046003227 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc91c06b550 0x7fc91c06da10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc928006fd0 tx=0x7fc928008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.296+0000 7fc936000640 1 -- 172.21.15.119:0/4046003227 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc930024020 con 0x7fc9240052e0 2024-04-25T10:18:41.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.494+0000 7fc944e00640 1 -- 172.21.15.119:0/4046003227 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9400c4660 con 0x7fc91c06b550 2024-04-25T10:18:41.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.495+0000 7fc936000640 1 -- 172.21.15.119:0/4046003227 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc9400c4660 con 0x7fc91c06b550 2024-04-25T10:18:41.498 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T10:18:41.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.497+0000 7fc944e00640 1 -- 172.21.15.119:0/4046003227 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc91c06b550 msgr2=0x7fc91c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.497+0000 7fc944e00640 1 --2- 172.21.15.119:0/4046003227 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc91c06b550 0x7fc91c06da10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc928006fd0 tx=0x7fc928008040 comp rx=0 tx=0).stop 2024-04-25T10:18:41.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.497+0000 7fc944e00640 1 -- 172.21.15.119:0/4046003227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9240052e0 msgr2=0x7fc94014e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.497+0000 7fc944e00640 1 --2- 172.21.15.119:0/4046003227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9240052e0 0x7fc94014e340 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fc93000e490 tx=0x7fc93000e950 comp rx=0 tx=0).stop 2024-04-25T10:18:41.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.498+0000 7fc944e00640 1 -- 172.21.15.119:0/4046003227 shutdown_connections 2024-04-25T10:18:41.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.498+0000 7fc944e00640 1 --2- 172.21.15.119:0/4046003227 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fc91c06b550 0x7fc91c06da10 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.498+0000 7fc944e00640 1 --2- 172.21.15.119:0/4046003227 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc9400b3920 0x7fc9400ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.498+0000 7fc944e00640 1 --2- 172.21.15.119:0/4046003227 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9400c7340 0x7fc9400b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.498+0000 7fc944e00640 1 --2- 172.21.15.119:0/4046003227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9240052e0 0x7fc94014e340 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.498+0000 7fc944e00640 1 -- 172.21.15.119:0/4046003227 >> 172.21.15.119:0/4046003227 conn(0x7fc94006aba0 msgr2=0x7fc9400cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.498+0000 7fc944e00640 1 -- 172.21.15.119:0/4046003227 shutdown_connections 2024-04-25T10:18:41.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.498+0000 7fc944e00640 1 -- 172.21.15.119:0/4046003227 wait complete. 2024-04-25T10:18:41.520 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:18:41.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.715+0000 7fa3e5c00640 1 Processor -- start 2024-04-25T10:18:41.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.715+0000 7fa3e5c00640 1 -- start start 2024-04-25T10:18:41.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.716+0000 7fa3e5c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa3e00b5050 0x7fa3e015fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.717+0000 7fa3e5c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3e006e430 0x7fa3e015bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.717+0000 7fa3e5c00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e00cd440 con 0x7fa3e0144470 2024-04-25T10:18:41.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.717+0000 7fa3e5c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e0145340 con 0x7fa3e00b5050 2024-04-25T10:18:41.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.717+0000 7fa3e5c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00cd720 con 0x7fa3e006e430 2024-04-25T10:18:41.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.717+0000 7fa3dfe00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa3e0144470 0x7fa3e00b4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40770/0 (socket says 172.21.15.119:40770) 2024-04-25T10:18:41.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.717+0000 7fa3dfe00640 1 -- 172.21.15.119:0/1434216944 learned_addr learned my addr 172.21.15.119:0/1434216944 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:18:41.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.717+0000 7fa3d7e00640 1 --2- 172.21.15.119:0/1434216944 >> v2:172.21.15.132:3300/0 conn(0x7fa3e00b5050 0x7fa3e015fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.717+0000 7fa3e5200640 1 --2- 172.21.15.119:0/1434216944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3e006e430 0x7fa3e015bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 -- 172.21.15.119:0/1434216944 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656955274 0 0) 0x7fa3e00cd440 con 0x7fa3e0144470 2024-04-25T10:18:41.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 -- 172.21.15.119:0/1434216944 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c4003520 con 0x7fa3e0144470 2024-04-25T10:18:41.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 -- 172.21.15.119:0/1434216944 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1788238644 0 0) 0x7fa3c4003520 con 0x7fa3e0144470 2024-04-25T10:18:41.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 -- 172.21.15.119:0/1434216944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3e006e430 msgr2=0x7fa3e015bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 --2- 172.21.15.119:0/1434216944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3e006e430 0x7fa3e015bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 -- 172.21.15.119:0/1434216944 >> v2:172.21.15.132:3300/0 conn(0x7fa3e00b5050 msgr2=0x7fa3e015fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 --2- 172.21.15.119:0/1434216944 >> v2:172.21.15.132:3300/0 conn(0x7fa3e00b5050 0x7fa3e015fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 -- 172.21.15.119:0/1434216944 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00cda00 con 0x7fa3e0144470 2024-04-25T10:18:41.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 -- 172.21.15.119:0/1434216944 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7fa3d0001670 con 0x7fa3e0144470 2024-04-25T10:18:41.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 -- 172.21.15.119:0/1434216944 >> v1:172.21.15.119:6789/0 conn(0x7fa3e0144470 legacy=0x7fa3e00b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 --2- 172.21.15.119:0/1434216944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3e006e430 0x7fa3c4003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 --2- 172.21.15.119:0/1434216944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3c4003fa0 0x7fa3c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 --2- 172.21.15.119:0/1434216944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c4004d40 0x7fa3c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 -- 172.21.15.119:0/1434216944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00cd720 con 0x7fa3c4003fa0 2024-04-25T10:18:41.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 -- 172.21.15.119:0/1434216944 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c400b9c0 con 0x7fa3c4004d40 2024-04-25T10:18:41.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3df400640 1 -- 172.21.15.119:0/1434216944 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c400bb70 con 0x7fa3e006e430 2024-04-25T10:18:41.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.718+0000 7fa3d7e00640 1 --2- 172.21.15.119:0/1434216944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3c4003fa0 0x7fa3c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.719+0000 7fa3d7e00640 1 -- 172.21.15.119:0/1434216944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3e006e430 msgr2=0x7fa3c4003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.719+0000 7fa3d7e00640 1 --2- 172.21.15.119:0/1434216944 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3e006e430 0x7fa3c4003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.719+0000 7fa3d7e00640 1 -- 172.21.15.119:0/1434216944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c4004d40 msgr2=0x7fa3c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.719+0000 7fa3d7e00640 1 --2- 172.21.15.119:0/1434216944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c4004d40 0x7fa3c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.719+0000 7fa3d7e00640 1 -- 172.21.15.119:0/1434216944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c4003520 con 0x7fa3c4003fa0 2024-04-25T10:18:41.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.720+0000 7fa3d7e00640 1 --2- 172.21.15.119:0/1434216944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3c4003fa0 0x7fa3c4004800 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009ac0 tx=0x7fa3c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db08a20c3b6eae47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.720+0000 7fa3df400640 1 -- 172.21.15.119:0/1434216944 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8036070 con 0x7fa3c4003fa0 2024-04-25T10:18:41.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.720+0000 7fa3df400640 1 -- 172.21.15.119:0/1434216944 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c8031400 con 0x7fa3c4003fa0 2024-04-25T10:18:41.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.720+0000 7fa3df400640 1 -- 172.21.15.119:0/1434216944 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8009690 con 0x7fa3c4003fa0 2024-04-25T10:18:41.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.721+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1434216944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3c4003fa0 msgr2=0x7fa3c4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.721+0000 7fa3e5c00640 1 --2- 172.21.15.119:0/1434216944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3c4003fa0 0x7fa3c4004800 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009ac0 tx=0x7fa3c4005830 comp rx=0 tx=0).stop 2024-04-25T10:18:41.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.721+0000 7fa3e5200640 1 -- 172.21.15.119:0/1434216944 reap_dead start 2024-04-25T10:18:41.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.721+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1434216944 shutdown_connections 2024-04-25T10:18:41.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.721+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1434216944 >> 172.21.15.119:0/1434216944 conn(0x7fa3e006b1d0 msgr2=0x7fa3e00b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.721+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1434216944 shutdown_connections 2024-04-25T10:18:41.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.721+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1434216944 wait complete. 2024-04-25T10:18:41.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.723+0000 7fa3e5c00640 1 Processor -- start 2024-04-25T10:18:41.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.723+0000 7fa3e5c00640 1 -- start start 2024-04-25T10:18:41.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.724+0000 7fa3e5c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3c4003fa0 0x7fa3e013bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.724+0000 7fa3e5c00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e00ad460 0x7fa3e00ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.724+0000 7fa3e5c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3e00b3090 0x7fa3e00b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.724+0000 7fa3e5c00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00ce200 con 0x7fa3e00ad460 2024-04-25T10:18:41.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.724+0000 7fa3e5c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e0144850 con 0x7fa3e00b3090 2024-04-25T10:18:41.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.724+0000 7fa3e5c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e015bcf0 con 0x7fa3c4003fa0 2024-04-25T10:18:41.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.724+0000 7fa3d7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e00ad460 0x7fa3e00ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.724+0000 7fa3d7e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e00ad460 0x7fa3e00ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46018/0 (socket says 172.21.15.119:46018) 2024-04-25T10:18:41.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.724+0000 7fa3d7e00640 1 -- 172.21.15.119:0/1386979132 learned_addr learned my addr 172.21.15.119:0/1386979132 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:18:41.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.725+0000 7fa3dfe00640 1 --2- 172.21.15.119:0/1386979132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3c4003fa0 0x7fa3e013bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.725+0000 7fa3d7e00640 1 -- 172.21.15.119:0/1386979132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3c4003fa0 msgr2=0x7fa3e013bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.725+0000 7fa3e5200640 1 --2- 172.21.15.119:0/1386979132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3e00b3090 0x7fa3e00b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.725+0000 7fa3d7e00640 1 --2- 172.21.15.119:0/1386979132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3c4003fa0 0x7fa3e013bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.725+0000 7fa3d7e00640 1 -- 172.21.15.119:0/1386979132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3e00b3090 msgr2=0x7fa3e00b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.725+0000 7fa3d7e00640 1 --2- 172.21.15.119:0/1386979132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3e00b3090 0x7fa3e00b3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.725+0000 7fa3d7e00640 1 -- 172.21.15.119:0/1386979132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00adf20 con 0x7fa3e00ad460 2024-04-25T10:18:41.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.725+0000 7fa3d7e00640 1 --2- 172.21.15.119:0/1386979132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e00ad460 0x7fa3e00ad8e0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8034040 tx=0x7fa3c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.725+0000 7fa3dcc00640 1 -- 172.21.15.119:0/1386979132 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8040020 con 0x7fa3e00ad460 2024-04-25T10:18:41.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.725+0000 7fa3dcc00640 1 -- 172.21.15.119:0/1386979132 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c8003a40 con 0x7fa3e00ad460 2024-04-25T10:18:41.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.725+0000 7fa3dcc00640 1 -- 172.21.15.119:0/1386979132 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8036040 con 0x7fa3e00ad460 2024-04-25T10:18:41.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.725+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1386979132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e00ae200 con 0x7fa3e00ad460 2024-04-25T10:18:41.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.725+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1386979132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e013c230 con 0x7fa3e00ad460 2024-04-25T10:18:41.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.727+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1386979132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e0158e60 con 0x7fa3e00ad460 2024-04-25T10:18:41.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.728+0000 7fa3dcc00640 1 -- 172.21.15.119:0/1386979132 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3c8043020 con 0x7fa3e00ad460 2024-04-25T10:18:41.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.729+0000 7fa3dcc00640 1 --2- 172.21.15.119:0/1386979132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3bc06b4d0 0x7fa3bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.729+0000 7fa3dcc00640 1 -- 172.21.15.119:0/1386979132 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa3c80a9ad0 con 0x7fa3e00ad460 2024-04-25T10:18:41.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.729+0000 7fa3dfe00640 1 --2- 172.21.15.119:0/1386979132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3bc06b4d0 0x7fa3bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.730+0000 7fa3dfe00640 1 --2- 172.21.15.119:0/1386979132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3bc06b4d0 0x7fa3bc06d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0007a10 tx=0x7fa3d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.734+0000 7fa3dcc00640 1 -- 172.21.15.119:0/1386979132 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3c8078b00 con 0x7fa3e00ad460 2024-04-25T10:18:41.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.928+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1386979132 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3e00b2090 con 0x7fa3bc06b4d0 2024-04-25T10:18:41.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.930+0000 7fa3dcc00640 1 -- 172.21.15.119:0/1386979132 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13469 (secure 0 0 0) 0x7fa3e00b2090 con 0x7fa3bc06b4d0 2024-04-25T10:18:41.934 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:18:41.934 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:18:41.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.933+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1386979132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3bc06b4d0 msgr2=0x7fa3bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.933+0000 7fa3e5c00640 1 --2- 172.21.15.119:0/1386979132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3bc06b4d0 0x7fa3bc06d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0007a10 tx=0x7fa3d0009040 comp rx=0 tx=0).stop 2024-04-25T10:18:41.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.933+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1386979132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e00ad460 msgr2=0x7fa3e00ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.933+0000 7fa3e5c00640 1 --2- 172.21.15.119:0/1386979132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e00ad460 0x7fa3e00ad8e0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8034040 tx=0x7fa3c8004730 comp rx=0 tx=0).stop 2024-04-25T10:18:41.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.933+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1386979132 shutdown_connections 2024-04-25T10:18:41.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.933+0000 7fa3e5c00640 1 --2- 172.21.15.119:0/1386979132 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7fa3bc06b4d0 0x7fa3bc06d990 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.933+0000 7fa3e5c00640 1 --2- 172.21.15.119:0/1386979132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3e00b3090 0x7fa3e00b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.933+0000 7fa3e5c00640 1 --2- 172.21.15.119:0/1386979132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e00ad460 0x7fa3e00ad8e0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.933+0000 7fa3e5c00640 1 --2- 172.21.15.119:0/1386979132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3c4003fa0 0x7fa3e013bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.933+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1386979132 >> 172.21.15.119:0/1386979132 conn(0x7fa3e006b1d0 msgr2=0x7fa3e00b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.933+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1386979132 shutdown_connections 2024-04-25T10:18:41.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:18:41.933+0000 7fa3e5c00640 1 -- 172.21.15.119:0/1386979132 wait complete. 2024-04-25T10:18:41.956 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-04-25T10:18:40.160941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261848,"kb_used_data":12392,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712872,"statfs":{"total":773094113280,"available":772825980928,"internally_reserved":0,"allocated":12689408,"data_stored":6174728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002852"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":350,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:18:31.230544+0000","last_change":"2024-04-25T10:06:57.510557+0000","last_active":"2024-04-25T10:18:31.230544+0000","last_peered":"2024-04-25T10:18:31.230544+0000","last_clean":"2024-04-25T10:18:31.230544+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:18:31.230544+0000","last_undegraded":"2024-04-25T10:18:31.230544+0000","last_fullsized":"2024-04-25T10:18:31.230544+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:56.280765+0000","last_clean_scrub_stamp":"2024-04-25T10:06:56.280765+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:38.291339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574991,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2031616,"data_stored":1214617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32588,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1437696,"data_stored":624249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32588,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1437696,"data_stored":624249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32588,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1437696,"data_stored":624249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":624249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":624249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":624249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574991,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2031616,"data_stored":1214617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:18:41.959 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T10:06:56.280765+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=6, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=18, tm_sec=35, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:18:41.959 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T10:19:01.960 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:02.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.156+0000 7f68ba200640 1 Processor -- start 2024-04-25T10:19:02.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.157+0000 7f68ba200640 1 -- start start 2024-04-25T10:19:02.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.158+0000 7f68ba200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68b415bf00 0x7f68b40cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.158+0000 7f68ba200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406e8f0 0x7f68b41654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.158+0000 7f68ba200640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b40ce6c0 con 0x7f68b40cc310 2024-04-25T10:19:02.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.158+0000 7f68ba200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b41455e0 con 0x7f68b415bf00 2024-04-25T10:19:02.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.158+0000 7f68ba200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b406d450 con 0x7f68b406e8f0 2024-04-25T10:19:02.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.159+0000 7f68b3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68b415bf00 0x7f68b40cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.159+0000 7f68b3400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68b415bf00 0x7f68b40cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58834/0 (socket says 172.21.15.119:58834) 2024-04-25T10:19:02.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.159+0000 7f68b3400640 1 -- 172.21.15.119:0/57534640 learned_addr learned my addr 172.21.15.119:0/57534640 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:19:02.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.159+0000 7f68b3400640 1 -- 172.21.15.119:0/57534640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406e8f0 msgr2=0x7f68b41654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.159+0000 7f68b3400640 1 --2- 172.21.15.119:0/57534640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406e8f0 0x7f68b41654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.159+0000 7f68b3400640 1 -- 172.21.15.119:0/57534640 >> v1:172.21.15.119:6789/0 conn(0x7f68b40cc310 legacy=0x7f68b415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.159+0000 7f68b3400640 1 -- 172.21.15.119:0/57534640 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b40cc7b0 con 0x7f68b415bf00 2024-04-25T10:19:02.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.160+0000 7f68b3400640 1 --2- 172.21.15.119:0/57534640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b415bf00 0x7f68b40cbdd0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009870 tx=0x7f68a402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed41ccd94114304 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.160+0000 7f68b2a00640 1 -- 172.21.15.119:0/57534640 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4037070 con 0x7f68b415bf00 2024-04-25T10:19:02.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.160+0000 7f68b2a00640 1 -- 172.21.15.119:0/57534640 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a4032440 con 0x7f68b415bf00 2024-04-25T10:19:02.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.160+0000 7f68b2a00640 1 -- 172.21.15.119:0/57534640 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4032810 con 0x7f68b415bf00 2024-04-25T10:19:02.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.161+0000 7f68ba200640 1 -- 172.21.15.119:0/57534640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b415bf00 msgr2=0x7f68b40cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.161+0000 7f68ba200640 1 --2- 172.21.15.119:0/57534640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b415bf00 0x7f68b40cbdd0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009870 tx=0x7f68a402a250 comp rx=0 tx=0).stop 2024-04-25T10:19:02.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.161+0000 7f68ba200640 1 -- 172.21.15.119:0/57534640 shutdown_connections 2024-04-25T10:19:02.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.161+0000 7f68ba200640 1 --2- 172.21.15.119:0/57534640 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b406e8f0 0x7f68b41654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.161+0000 7f68ba200640 1 --2- 172.21.15.119:0/57534640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b415bf00 0x7f68b40cbdd0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.161+0000 7f68ba200640 1 -- 172.21.15.119:0/57534640 >> 172.21.15.119:0/57534640 conn(0x7f68b406aba0 msgr2=0x7f68b406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:02.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.161+0000 7f68ba200640 1 -- 172.21.15.119:0/57534640 shutdown_connections 2024-04-25T10:19:02.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.161+0000 7f68ba200640 1 -- 172.21.15.119:0/57534640 wait complete. 2024-04-25T10:19:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.163+0000 7f68ba200640 1 Processor -- start 2024-04-25T10:19:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.163+0000 7f68ba200640 1 -- start start 2024-04-25T10:19:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.164+0000 7f68ba200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68b406e8f0 0x7f68b413bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.164+0000 7f68ba200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b40cc310 0x7f68b4139dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.164+0000 7f68ba200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b415bf00 0x7f68b413a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.164+0000 7f68ba200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40cd050 con 0x7f68b415bf00 2024-04-25T10:19:02.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.164+0000 7f68ba200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40ccd70 con 0x7f68b406e8f0 2024-04-25T10:19:02.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.164+0000 7f68ba200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b406ee60 con 0x7f68b40cc310 2024-04-25T10:19:02.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.164+0000 7f68b3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b415bf00 0x7f68b413a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.164+0000 7f68b3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b415bf00 0x7f68b413a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58848/0 (socket says 172.21.15.119:58848) 2024-04-25T10:19:02.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.164+0000 7f68b3e00640 1 -- 172.21.15.119:0/1467514322 learned_addr learned my addr 172.21.15.119:0/1467514322 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:19:02.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.165+0000 7f68b3e00640 1 -- 172.21.15.119:0/1467514322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b40cc310 msgr2=0x7f68b4139dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.165+0000 7f68b3e00640 1 --2- 172.21.15.119:0/1467514322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b40cc310 0x7f68b4139dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.165+0000 7f68b3e00640 1 -- 172.21.15.119:0/1467514322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68b406e8f0 msgr2=0x7f68b413bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.165+0000 7f68b3e00640 1 --2- 172.21.15.119:0/1467514322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68b406e8f0 0x7f68b413bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.165+0000 7f68b3e00640 1 -- 172.21.15.119:0/1467514322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b4158210 con 0x7f68b415bf00 2024-04-25T10:19:02.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.165+0000 7f68b3e00640 1 --2- 172.21.15.119:0/1467514322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b415bf00 0x7f68b413a5f0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00b970 tx=0x7f68ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.165+0000 7f68ab400640 1 -- 172.21.15.119:0/1467514322 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac019070 con 0x7f68b415bf00 2024-04-25T10:19:02.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.165+0000 7f68ab400640 1 -- 172.21.15.119:0/1467514322 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ac00cb60 con 0x7f68b415bf00 2024-04-25T10:19:02.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.165+0000 7f68ba200640 1 -- 172.21.15.119:0/1467514322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b41584f0 con 0x7f68b415bf00 2024-04-25T10:19:02.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.165+0000 7f68ab400640 1 -- 172.21.15.119:0/1467514322 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac014440 con 0x7f68b415bf00 2024-04-25T10:19:02.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.165+0000 7f68ba200640 1 -- 172.21.15.119:0/1467514322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b40b2a60 con 0x7f68b415bf00 2024-04-25T10:19:02.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.166+0000 7f68ba200640 1 -- 172.21.15.119:0/1467514322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b40b2c60 con 0x7f68b415bf00 2024-04-25T10:19:02.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.167+0000 7f68ab400640 1 -- 172.21.15.119:0/1467514322 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68ac01c070 con 0x7f68b415bf00 2024-04-25T10:19:02.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.168+0000 7f68ab400640 1 --2- 172.21.15.119:0/1467514322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f689006b4d0 0x7f689006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.168+0000 7f68ab400640 1 -- 172.21.15.119:0/1467514322 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f68ac089da0 con 0x7f68b415bf00 2024-04-25T10:19:02.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.169+0000 7f68b3400640 1 --2- 172.21.15.119:0/1467514322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f689006b4d0 0x7f689006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.169+0000 7f68b3400640 1 --2- 172.21.15.119:0/1467514322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f689006b4d0 0x7f689006d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f68a4034000 tx=0x7f68a402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.173+0000 7f68ab400640 1 -- 172.21.15.119:0/1467514322 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f68b40b2c60 con 0x7f68b415bf00 2024-04-25T10:19:02.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.365+0000 7f68ba200640 1 -- 172.21.15.119:0/1467514322 --> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f68b40749b0 con 0x7f689006b4d0 2024-04-25T10:19:02.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.366+0000 7f68ab400640 1 -- 172.21.15.119:0/1467514322 <== mgr.4106 v2:172.21.15.119:6832/2600847164 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13379 (secure 0 0 0) 0x7f68b40749b0 con 0x7f689006b4d0 2024-04-25T10:19:02.370 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:19:02.370 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-04-25T10:19:02.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.369+0000 7f68ba200640 1 -- 172.21.15.119:0/1467514322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f689006b4d0 msgr2=0x7f689006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.369+0000 7f68ba200640 1 --2- 172.21.15.119:0/1467514322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f689006b4d0 0x7f689006d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f68a4034000 tx=0x7f68a402c000 comp rx=0 tx=0).stop 2024-04-25T10:19:02.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.369+0000 7f68ba200640 1 -- 172.21.15.119:0/1467514322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b415bf00 msgr2=0x7f68b413a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.369+0000 7f68ba200640 1 --2- 172.21.15.119:0/1467514322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b415bf00 0x7f68b413a5f0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00b970 tx=0x7f68ac00be30 comp rx=0 tx=0).stop 2024-04-25T10:19:02.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.369+0000 7f68ba200640 1 -- 172.21.15.119:0/1467514322 shutdown_connections 2024-04-25T10:19:02.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.369+0000 7f68ba200640 1 --2- 172.21.15.119:0/1467514322 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f689006b4d0 0x7f689006d990 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.369+0000 7f68ba200640 1 --2- 172.21.15.119:0/1467514322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b415bf00 0x7f68b413a5f0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.369+0000 7f68ba200640 1 --2- 172.21.15.119:0/1467514322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68b40cc310 0x7f68b4139dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.369+0000 7f68ba200640 1 --2- 172.21.15.119:0/1467514322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68b406e8f0 0x7f68b413bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.369+0000 7f68ba200640 1 -- 172.21.15.119:0/1467514322 >> 172.21.15.119:0/1467514322 conn(0x7f68b406aba0 msgr2=0x7f68b415e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:02.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.370+0000 7f68ba200640 1 -- 172.21.15.119:0/1467514322 shutdown_connections 2024-04-25T10:19:02.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.370+0000 7f68ba200640 1 -- 172.21.15.119:0/1467514322 wait complete. 2024-04-25T10:19:02.391 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-04-25T10:19:02.168322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261836,"kb_used_data":12392,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712884,"statfs":{"total":773094113280,"available":772825993216,"internally_reserved":0,"allocated":12689408,"data_stored":6174728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004126"},"pg_stats":[{"pgid":"1.0","version":"115'119","reported_seq":362,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:18:41.739448+0000","last_change":"2024-04-25T10:18:41.739448+0000","last_active":"2024-04-25T10:18:41.739448+0000","last_peered":"2024-04-25T10:18:41.739448+0000","last_clean":"2024-04-25T10:18:41.739448+0000","last_became_active":"2024-04-25T10:06:57.508076+0000","last_became_peered":"2024-04-25T10:06:57.508076+0000","last_unstale":"2024-04-25T10:18:41.739448+0000","last_undegraded":"2024-04-25T10:18:41.739448+0000","last_fullsized":"2024-04-25T10:18:41.739448+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"115'119","last_scrub_stamp":"2024-04-25T10:18:41.739197+0000","last_deep_scrub":"115'119","last_deep_scrub_stamp":"2024-04-25T10:18:41.739197+0000","last_clean_scrub_stamp":"2024-04-25T10:18:41.739197+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:28.350502+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574996,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33160,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338680,"statfs":{"total":96636764160,"available":96602808320,"internally_reserved":0,"allocated":2031616,"data_stored":1214617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32588,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1437696,"data_stored":624249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32588,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1437696,"data_stored":624249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32588,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1437696,"data_stored":624249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":624249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":624249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32584,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1437696,"data_stored":624249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574996,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33160,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338680,"statfs":{"total":96636764160,"available":96602808320,"internally_reserved":0,"allocated":2031616,"data_stored":1214617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:02.392 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T10:19:02.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.564+0000 7efcb9800640 1 Processor -- start 2024-04-25T10:19:02.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.564+0000 7efcb9800640 1 -- start start 2024-04-25T10:19:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.565+0000 7efcb9800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efcb40d07b0 0x7efcb40cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.566+0000 7efcb9800640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb40b5710 con 0x7efcb40ce860 2024-04-25T10:19:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.566+0000 7efcb9800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb40b59f0 con 0x7efcb415b7e0 2024-04-25T10:19:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.566+0000 7efcb9800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb40b5cd0 con 0x7efcb40d07b0 2024-04-25T10:19:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.566+0000 7efcb3400640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7efcb40ce860 0x7efcb40d00a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38288/0 (socket says 172.21.15.119:38288) 2024-04-25T10:19:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.566+0000 7efcb3400640 1 -- 172.21.15.119:0/488213013 learned_addr learned my addr 172.21.15.119:0/488213013 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:19:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.566+0000 7efcb8e00640 1 --2- 172.21.15.119:0/488213013 >> v2:172.21.15.132:3300/0 conn(0x7efcb40d07b0 0x7efcb40cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.566+0000 7efcb2a00640 1 -- 172.21.15.119:0/488213013 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978794825 0 0) 0x7efcb40b5710 con 0x7efcb40ce860 2024-04-25T10:19:02.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.566+0000 7efcb2a00640 1 -- 172.21.15.119:0/488213013 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc98003520 con 0x7efcb40ce860 2024-04-25T10:19:02.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb2a00640 1 -- 172.21.15.119:0/488213013 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1797051898 0 0) 0x7efc98003520 con 0x7efcb40ce860 2024-04-25T10:19:02.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb2a00640 1 -- 172.21.15.119:0/488213013 >> v2:172.21.15.132:3300/0 conn(0x7efcb40d07b0 msgr2=0x7efcb40cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb2a00640 1 --2- 172.21.15.119:0/488213013 >> v2:172.21.15.132:3300/0 conn(0x7efcb40d07b0 0x7efcb40cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb2a00640 1 -- 172.21.15.119:0/488213013 >> v1:172.21.15.132:6789/0 conn(0x7efcb415b7e0 legacy=0x7efcb415e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb2a00640 1 -- 172.21.15.119:0/488213013 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb40b5fb0 con 0x7efcb40ce860 2024-04-25T10:19:02.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb2a00640 1 -- 172.21.15.119:0/488213013 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3827124844 0 0) 0x7efca0001670 con 0x7efcb40ce860 2024-04-25T10:19:02.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb2a00640 1 -- 172.21.15.119:0/488213013 >> v1:172.21.15.119:6789/0 conn(0x7efcb40ce860 legacy=0x7efcb40d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb2a00640 1 --2- 172.21.15.119:0/488213013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc98003a00 0x7efc98003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb2a00640 1 --2- 172.21.15.119:0/488213013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc98004800 0x7efc98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb2a00640 1 --2- 172.21.15.119:0/488213013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc980052e0 0x7efc9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb2a00640 1 -- 172.21.15.119:0/488213013 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40b5cd0 con 0x7efc98004800 2024-04-25T10:19:02.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb2a00640 1 -- 172.21.15.119:0/488213013 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9800c1e0 con 0x7efc980052e0 2024-04-25T10:19:02.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb2a00640 1 -- 172.21.15.119:0/488213013 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc9800c3e0 con 0x7efc98003a00 2024-04-25T10:19:02.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb3400640 1 --2- 172.21.15.119:0/488213013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc98004800 0x7efc98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.567+0000 7efcb3400640 1 -- 172.21.15.119:0/488213013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc98003a00 msgr2=0x7efc98003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.568+0000 7efcb3e00640 1 --2- 172.21.15.119:0/488213013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc98003a00 0x7efc98003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.568+0000 7efcb3400640 1 --2- 172.21.15.119:0/488213013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc98003a00 0x7efc98003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.568+0000 7efcb8e00640 1 --2- 172.21.15.119:0/488213013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc980052e0 0x7efc9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.568+0000 7efcb3400640 1 -- 172.21.15.119:0/488213013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc980052e0 msgr2=0x7efc9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.568+0000 7efcb3400640 1 --2- 172.21.15.119:0/488213013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc980052e0 0x7efc9800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.568+0000 7efcb8e00640 1 -- 172.21.15.119:0/488213013 reap_dead start 2024-04-25T10:19:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.568+0000 7efcb3400640 1 -- 172.21.15.119:0/488213013 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc98003520 con 0x7efc98004800 2024-04-25T10:19:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.569+0000 7efcb3400640 1 --2- 172.21.15.119:0/488213013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc98004800 0x7efc98004c80 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7efca000c190 tx=0x7efc98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e664d66a33082cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.569+0000 7efcb2a00640 1 -- 172.21.15.119:0/488213013 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0007c60 con 0x7efc98004800 2024-04-25T10:19:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.569+0000 7efcb2a00640 1 -- 172.21.15.119:0/488213013 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca0005ad0 con 0x7efc98004800 2024-04-25T10:19:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.569+0000 7efcb2a00640 1 -- 172.21.15.119:0/488213013 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0007070 con 0x7efc98004800 2024-04-25T10:19:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.569+0000 7efcb9800640 1 -- 172.21.15.119:0/488213013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc98004800 msgr2=0x7efc98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.569+0000 7efcb9800640 1 --2- 172.21.15.119:0/488213013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc98004800 0x7efc98004c80 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7efca000c190 tx=0x7efc98005830 comp rx=0 tx=0).stop 2024-04-25T10:19:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.569+0000 7efcb9800640 1 -- 172.21.15.119:0/488213013 shutdown_connections 2024-04-25T10:19:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.569+0000 7efcb9800640 1 --2- 172.21.15.119:0/488213013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc98004800 0x7efc98004c80 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.569+0000 7efcb9800640 1 -- 172.21.15.119:0/488213013 >> 172.21.15.119:0/488213013 conn(0x7efcb40722d0 msgr2=0x7efcb40726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.570+0000 7efcb9800640 1 -- 172.21.15.119:0/488213013 shutdown_connections 2024-04-25T10:19:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.570+0000 7efcb9800640 1 -- 172.21.15.119:0/488213013 wait complete. 2024-04-25T10:19:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.571+0000 7efcb9800640 1 Processor -- start 2024-04-25T10:19:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.571+0000 7efcb9800640 1 -- start start 2024-04-25T10:19:02.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.572+0000 7efcb9800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc98004800 0x7efcb414ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.572+0000 7efcb9800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcb40b3620 0x7efcb40b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.572+0000 7efcb9800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb40cb420 0x7efcb40cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.572+0000 7efcb9800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40cee70 con 0x7efc98004800 2024-04-25T10:19:02.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.572+0000 7efcb9800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40cecf0 con 0x7efcb40b3620 2024-04-25T10:19:02.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.572+0000 7efcb9800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40ceff0 con 0x7efcb40cb420 2024-04-25T10:19:02.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.573+0000 7efcb3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc98004800 0x7efcb414ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.573+0000 7efcb3e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc98004800 0x7efcb414ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58858/0 (socket says 172.21.15.119:58858) 2024-04-25T10:19:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.573+0000 7efcb3e00640 1 -- 172.21.15.119:0/926896301 learned_addr learned my addr 172.21.15.119:0/926896301 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:19:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.573+0000 7efcb3e00640 1 -- 172.21.15.119:0/926896301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb40cb420 msgr2=0x7efcb40cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.573+0000 7efcb3e00640 1 --2- 172.21.15.119:0/926896301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb40cb420 0x7efcb40cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.573+0000 7efcb3e00640 1 -- 172.21.15.119:0/926896301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcb40b3620 msgr2=0x7efcb40b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.573+0000 7efcb3e00640 1 --2- 172.21.15.119:0/926896301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcb40b3620 0x7efcb40b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.573+0000 7efcb3e00640 1 -- 172.21.15.119:0/926896301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb4160550 con 0x7efc98004800 2024-04-25T10:19:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.573+0000 7efcb3e00640 1 --2- 172.21.15.119:0/926896301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc98004800 0x7efcb414ae10 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7efca400d400 tx=0x7efca400d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.573+0000 7efcabe00640 1 -- 172.21.15.119:0/926896301 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca4018070 con 0x7efc98004800 2024-04-25T10:19:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.574+0000 7efcabe00640 1 -- 172.21.15.119:0/926896301 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca4003610 con 0x7efc98004800 2024-04-25T10:19:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.574+0000 7efcb9800640 1 -- 172.21.15.119:0/926896301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb40ccc10 con 0x7efc98004800 2024-04-25T10:19:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.574+0000 7efcabe00640 1 -- 172.21.15.119:0/926896301 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca4003a00 con 0x7efc98004800 2024-04-25T10:19:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.574+0000 7efcb9800640 1 -- 172.21.15.119:0/926896301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb40cc2e0 con 0x7efc98004800 2024-04-25T10:19:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.575+0000 7efcb9800640 1 -- 172.21.15.119:0/926896301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb40749b0 con 0x7efc98004800 2024-04-25T10:19:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.576+0000 7efcabe00640 1 -- 172.21.15.119:0/926896301 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efca401b070 con 0x7efc98004800 2024-04-25T10:19:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.577+0000 7efcabe00640 1 --2- 172.21.15.119:0/926896301 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc9406b500 0x7efc9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.577+0000 7efcabe00640 1 -- 172.21.15.119:0/926896301 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7efca40895f0 con 0x7efc98004800 2024-04-25T10:19:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.577+0000 7efcb3400640 1 --2- 172.21.15.119:0/926896301 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc9406b500 0x7efc9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.578+0000 7efcb3400640 1 --2- 172.21.15.119:0/926896301 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc9406b500 0x7efc9406d9c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7efca000c160 tx=0x7efca0007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.581+0000 7efcabe00640 1 -- 172.21.15.119:0/926896301 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efcb4160550 con 0x7efc98004800 2024-04-25T10:19:02.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.775+0000 7efcb9800640 1 -- 172.21.15.119:0/926896301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7efcb40d1650 con 0x7efc98004800 2024-04-25T10:19:02.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.781+0000 7efcabe00640 1 -- 172.21.15.119:0/926896301 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7efca4057fd0 con 0x7efc98004800 2024-04-25T10:19:02.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.783+0000 7efcabe00640 1 -- 172.21.15.119:0/926896301 <== mon.0 v2:172.21.15.119:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7efca405c070 con 0x7efc98004800 2024-04-25T10:19:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.784+0000 7efcb9800640 1 -- 172.21.15.119:0/926896301 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc9406b500 msgr2=0x7efc9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.784+0000 7efcb9800640 1 --2- 172.21.15.119:0/926896301 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc9406b500 0x7efc9406d9c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7efca000c160 tx=0x7efca0007000 comp rx=0 tx=0).stop 2024-04-25T10:19:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.784+0000 7efcb9800640 1 -- 172.21.15.119:0/926896301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc98004800 msgr2=0x7efcb414ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.784+0000 7efcb9800640 1 --2- 172.21.15.119:0/926896301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc98004800 0x7efcb414ae10 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7efca400d400 tx=0x7efca400d8c0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.784+0000 7efcb9800640 1 -- 172.21.15.119:0/926896301 shutdown_connections 2024-04-25T10:19:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.784+0000 7efcb9800640 1 --2- 172.21.15.119:0/926896301 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7efc9406b500 0x7efc9406d9c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.784+0000 7efcb9800640 1 --2- 172.21.15.119:0/926896301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efcb40cb420 0x7efcb40cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.784+0000 7efcb9800640 1 --2- 172.21.15.119:0/926896301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcb40b3620 0x7efcb40b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.784+0000 7efcb9800640 1 --2- 172.21.15.119:0/926896301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc98004800 0x7efcb414ae10 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.784+0000 7efcb9800640 1 -- 172.21.15.119:0/926896301 >> 172.21.15.119:0/926896301 conn(0x7efcb40722d0 msgr2=0x7efcb40cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.785+0000 7efcb9800640 1 -- 172.21.15.119:0/926896301 shutdown_connections 2024-04-25T10:19:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.785+0000 7efcb9800640 1 -- 172.21.15.119:0/926896301 wait complete. 2024-04-25T10:19:02.807 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:19:02.807 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:02.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.977+0000 7f648aa00640 1 Processor -- start 2024-04-25T10:19:02.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.977+0000 7f648aa00640 1 -- start start 2024-04-25T10:19:02.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.979+0000 7f648aa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f64840d35b0 0x7f64840b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.979+0000 7f648aa00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f64840b5990 0x7f64840b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.979+0000 7f648aa00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f648415e0c0 con 0x7f648415dce0 2024-04-25T10:19:02.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.979+0000 7f648aa00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64840d2510 con 0x7f64840b5990 2024-04-25T10:19:02.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.979+0000 7f648aa00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64840d27f0 con 0x7f64840d35b0 2024-04-25T10:19:02.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.979+0000 7f6483e00640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f648415dce0 0x7f64840d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38304/0 (socket says 172.21.15.119:38304) 2024-04-25T10:19:02.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.979+0000 7f6483e00640 1 -- 172.21.15.119:0/982315951 learned_addr learned my addr 172.21.15.119:0/982315951 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-04-25T10:19:02.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.979+0000 7f6488a00640 1 --2- 172.21.15.119:0/982315951 >> v2:172.21.15.119:3300/0 conn(0x7f64840b5990 0x7f64840b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.979+0000 7f6483400640 1 --2- 172.21.15.119:0/982315951 >> v2:172.21.15.132:3300/0 conn(0x7f64840d35b0 0x7f64840b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.980+0000 7f6482a00640 1 -- 172.21.15.119:0/982315951 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468774986 0 0) 0x7f648415e0c0 con 0x7f648415dce0 2024-04-25T10:19:02.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.980+0000 7f6482a00640 1 -- 172.21.15.119:0/982315951 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6468003520 con 0x7f648415dce0 2024-04-25T10:19:02.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.980+0000 7f6488a00640 1 -- 172.21.15.119:0/982315951 >> v2:172.21.15.132:3300/0 conn(0x7f64840d35b0 msgr2=0x7f64840b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.980+0000 7f6488a00640 1 --2- 172.21.15.119:0/982315951 >> v2:172.21.15.132:3300/0 conn(0x7f64840d35b0 0x7f64840b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.980+0000 7f6488a00640 1 -- 172.21.15.119:0/982315951 >> v1:172.21.15.119:6789/0 conn(0x7f648415dce0 legacy=0x7f64840d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.980+0000 7f6488a00640 1 -- 172.21.15.119:0/982315951 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64840d2ad0 con 0x7f64840b5990 2024-04-25T10:19:02.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.981+0000 7f6488a00640 1 --2- 172.21.15.119:0/982315951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64840b5990 0x7f64840b6b00 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f64840cdfa0 tx=0x7f647c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c249a20fd7527b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.981+0000 7f6482a00640 1 -- 172.21.15.119:0/982315951 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c037070 con 0x7f64840b5990 2024-04-25T10:19:02.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.981+0000 7f6482a00640 1 -- 172.21.15.119:0/982315951 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f647c032440 con 0x7f64840b5990 2024-04-25T10:19:02.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.981+0000 7f6482a00640 1 -- 172.21.15.119:0/982315951 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c032830 con 0x7f64840b5990 2024-04-25T10:19:02.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.982+0000 7f648aa00640 1 -- 172.21.15.119:0/982315951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64840b5990 msgr2=0x7f64840b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.982+0000 7f648aa00640 1 --2- 172.21.15.119:0/982315951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64840b5990 0x7f64840b6b00 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f64840cdfa0 tx=0x7f647c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.982+0000 7f648aa00640 1 -- 172.21.15.119:0/982315951 shutdown_connections 2024-04-25T10:19:02.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.982+0000 7f648aa00640 1 --2- 172.21.15.119:0/982315951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64840b5990 0x7f64840b6b00 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.982+0000 7f648aa00640 1 --2- 172.21.15.119:0/982315951 >> v2:172.21.15.132:3300/0 conn(0x7f64840d35b0 0x7f64840b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.982+0000 7f648aa00640 1 -- 172.21.15.119:0/982315951 >> 172.21.15.119:0/982315951 conn(0x7f6484072900 msgr2=0x7f64840d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:02.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.982+0000 7f648aa00640 1 -- 172.21.15.119:0/982315951 shutdown_connections 2024-04-25T10:19:02.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.982+0000 7f648aa00640 1 -- 172.21.15.119:0/982315951 wait complete. 2024-04-25T10:19:02.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.984+0000 7f648aa00640 1 Processor -- start 2024-04-25T10:19:02.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.984+0000 7f648aa00640 1 -- start start 2024-04-25T10:19:02.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.985+0000 7f648aa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64840b5990 0x7f6484156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.985+0000 7f648aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64840d35b0 0x7f648415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.985+0000 7f648aa00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648415dce0 0x7f64841f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.985+0000 7f648aa00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64840d2160 con 0x7f648415dce0 2024-04-25T10:19:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.985+0000 7f648aa00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64840d3090 con 0x7f64840b5990 2024-04-25T10:19:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.985+0000 7f648aa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64841f7af0 con 0x7f64840d35b0 2024-04-25T10:19:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.985+0000 7f6488a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648415dce0 0x7f64841f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.985+0000 7f6488a00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648415dce0 0x7f64841f6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58888/0 (socket says 172.21.15.119:58888) 2024-04-25T10:19:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.985+0000 7f6488a00640 1 -- 172.21.15.119:0/2879811925 learned_addr learned my addr 172.21.15.119:0/2879811925 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-25T10:19:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.985+0000 7f6488a00640 1 -- 172.21.15.119:0/2879811925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64840d35b0 msgr2=0x7f648415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.985+0000 7f6488a00640 1 --2- 172.21.15.119:0/2879811925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64840d35b0 0x7f648415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.985+0000 7f6488a00640 1 -- 172.21.15.119:0/2879811925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64840b5990 msgr2=0x7f6484156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.985+0000 7f6488a00640 1 --2- 172.21.15.119:0/2879811925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64840b5990 0x7f6484156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.985+0000 7f6488a00640 1 -- 172.21.15.119:0/2879811925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64841573e0 con 0x7f648415dce0 2024-04-25T10:19:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.986+0000 7f6488a00640 1 --2- 172.21.15.119:0/2879811925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648415dce0 0x7f64841f6bc0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f647c009890 tx=0x7f647c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.986+0000 7f647be00640 1 -- 172.21.15.119:0/2879811925 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c037040 con 0x7f648415dce0 2024-04-25T10:19:02.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.986+0000 7f647be00640 1 -- 172.21.15.119:0/2879811925 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f647c042070 con 0x7f648415dce0 2024-04-25T10:19:02.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.986+0000 7f647be00640 1 -- 172.21.15.119:0/2879811925 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c030920 con 0x7f648415dce0 2024-04-25T10:19:02.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.986+0000 7f648aa00640 1 -- 172.21.15.119:0/2879811925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64841f7530 con 0x7f648415dce0 2024-04-25T10:19:02.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.986+0000 7f648aa00640 1 -- 172.21.15.119:0/2879811925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648415e450 con 0x7f648415dce0 2024-04-25T10:19:02.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.987+0000 7f648aa00640 1 -- 172.21.15.119:0/2879811925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64840749b0 con 0x7f648415dce0 2024-04-25T10:19:02.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.989+0000 7f647be00640 1 -- 172.21.15.119:0/2879811925 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f647c030ac0 con 0x7f648415dce0 2024-04-25T10:19:02.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.991+0000 7f647be00640 1 --2- 172.21.15.119:0/2879811925 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f646006b4d0 0x7f646006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.991+0000 7f647be00640 1 -- 172.21.15.119:0/2879811925 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f647c0aa9b0 con 0x7f648415dce0 2024-04-25T10:19:02.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.991+0000 7f6483e00640 1 --2- 172.21.15.119:0/2879811925 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f646006b4d0 0x7f646006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.992+0000 7f6483e00640 1 --2- 172.21.15.119:0/2879811925 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f646006b4d0 0x7f646006d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f64740012e0 tx=0x7f64740015b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:02.995+0000 7f647be00640 1 -- 172.21.15.119:0/2879811925 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f647c0799d0 con 0x7f648415dce0 2024-04-25T10:19:03.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:03.153 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:03.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:03.221+0000 7f648aa00640 1 -- 172.21.15.119:0/2879811925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64840b83d0 con 0x7f648415dce0 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:03.222+0000 7f647be00640 1 -- 172.21.15.119:0/2879811925 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+3713 (secure 0 0 0) 0x7f647c0792d0 con 0x7f648415dce0 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":114,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/3394261143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3394261143},{"type":"v1","addr":"172.21.15.132:6834","nonce":3394261143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2502874931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2502874931},{"type":"v1","addr":"172.21.15.119:6835","nonce":2502874931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":8694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/2305428003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":2305428003},{"type":"v1","addr":"172.21.15.132:6835","nonce":2305428003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":9339,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3649319143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3649319143},{"type":"v1","addr":"172.21.15.119:6837","nonce":3649319143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[]} 2024-04-25T10:19:03.225 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 114 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:03.224+0000 7f648aa00640 1 -- 172.21.15.119:0/2879811925 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f646006b4d0 msgr2=0x7f646006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:03.224+0000 7f648aa00640 1 --2- 172.21.15.119:0/2879811925 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f646006b4d0 0x7f646006d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f64740012e0 tx=0x7f64740015b0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:03.224+0000 7f648aa00640 1 -- 172.21.15.119:0/2879811925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648415dce0 msgr2=0x7f64841f6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:03.224+0000 7f648aa00640 1 --2- 172.21.15.119:0/2879811925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648415dce0 0x7f64841f6bc0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f647c009890 tx=0x7f647c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:03.225+0000 7f648aa00640 1 -- 172.21.15.119:0/2879811925 shutdown_connections 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:03.225+0000 7f648aa00640 1 --2- 172.21.15.119:0/2879811925 >> [v2:172.21.15.119:6832/2600847164,v1:172.21.15.119:6833/2600847164] conn(0x7f646006b4d0 0x7f646006d990 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:03.225+0000 7f648aa00640 1 --2- 172.21.15.119:0/2879811925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f648415dce0 0x7f64841f6bc0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:03.225+0000 7f648aa00640 1 --2- 172.21.15.119:0/2879811925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f64840d35b0 0x7f648415ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:03.225+0000 7f648aa00640 1 --2- 172.21.15.119:0/2879811925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64840b5990 0x7f6484156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:03.225+0000 7f648aa00640 1 -- 172.21.15.119:0/2879811925 >> 172.21.15.119:0/2879811925 conn(0x7f6484072900 msgr2=0x7f64840b7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:03.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:03.225+0000 7f648aa00640 1 -- 172.21.15.119:0/2879811925 shutdown_connections 2024-04-25T10:19:03.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-25T10:19:03.225+0000 7f648aa00640 1 -- 172.21.15.119:0/2879811925 wait complete. 2024-04-25T10:19:03.244 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T10:19:03.244 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T10:19:03.244 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:19:03.244 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:03.302 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:19:03.302 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:19:03.302 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:03.360 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:19:03.361 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:19:03.361 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:03.384 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:19:03.384 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:19:03.384 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:03.443 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:19:03.443 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T10:19:03.444 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T10:19:03.444 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:03.513 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T10:19:03.514 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T10:19:03.514 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:03.565 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T10:19:03.565 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T10:19:03.565 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:03.627 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T10:19:03.627 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T10:19:03.628 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:03.686 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T10:19:03.687 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T10:19:03.687 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:03.733 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T10:19:03.733 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T10:19:03.733 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:03.779 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T10:19:03.779 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T10:19:03.779 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:03.825 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T10:19:03.825 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T10:19:03.825 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:03.875 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T10:19:03.875 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T10:19:03.875 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T10:19:03.875 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:03.959 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T10:19:03.959 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T10:19:03.959 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:04.045 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T10:19:04.045 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T10:19:04.045 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:19:04.045 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:04.095 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:19:04.095 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:19:04.095 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:04.125 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:19:04.125 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:19:04.125 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:04.202 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:19:04.202 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T10:19:04.203 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-25T10:19:04.300 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:19:04.301 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T10:19:04.567 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:19:04.567 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T10:19:04.703 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:19:04.703 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T10:19:04.839 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:19:04.839 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T10:19:05.036 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-25T10:19:05.036 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T10:19:05.276 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-25T10:19:05.277 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T10:19:05.424 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-25T10:19:05.424 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T10:19:05.551 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-25T10:19:05.551 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T10:19:05.717 INFO:tasks.ceph:Archiving mon data... 2024-04-25T10:19:05.718 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048/data/mon.a.tgz 2024-04-25T10:19:05.718 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-04-25T10:19:05.775 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.5TBwsyhFEr 2024-04-25T10:19:05.775 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.5TBwsyhFEr 2024-04-25T10:19:05.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.5TBwsyhFEr 2024-04-25T10:19:06.117 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.5TBwsyhFEr is 412KB 2024-04-25T10:19:06.225 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.5TBwsyhFEr 2024-04-25T10:19:06.282 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048/data/mon.b.tgz 2024-04-25T10:19:06.282 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-04-25T10:19:06.339 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.0KmiXjlP9u 2024-04-25T10:19:06.339 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.0KmiXjlP9u 2024-04-25T10:19:06.523 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.0KmiXjlP9u 2024-04-25T10:19:06.659 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.0KmiXjlP9u is 424KB 2024-04-25T10:19:06.770 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.0KmiXjlP9u 2024-04-25T10:19:06.826 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048/data/mon.c.tgz 2024-04-25T10:19:06.826 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-04-25T10:19:06.875 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.cJaxL6yNuo 2024-04-25T10:19:06.875 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.cJaxL6yNuo 2024-04-25T10:19:07.049 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.cJaxL6yNuo 2024-04-25T10:19:07.184 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.cJaxL6yNuo is 424KB 2024-04-25T10:19:07.292 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.cJaxL6yNuo 2024-04-25T10:19:07.347 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T10:19:07.347 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:19:07.355 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:19:07.494 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:19:07.550 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:19:07.550 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T10:19:07.552 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048/remote/smithi119/crash 2024-04-25T10:19:07.553 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:19:07.639 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048/remote/smithi132/crash 2024-04-25T10:19:07.640 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:19:07.726 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T10:19:07.726 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:07.734 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:07.878 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:07.963 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:08.052 INFO:tasks.ceph:Compressing logs... 2024-04-25T10:19:08.052 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:19:08.060 DEBUG:teuthology.orchestra.run.smithi132:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:19:08.131 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73828.log 2024-04-25T10:19:08.131 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54202.log 2024-04-25T10:19:08.132 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43263.log 2024-04-25T10:19:08.133 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73828.log.gz 2024-04-25T10:19:08.133 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74564.log 2024-04-25T10:19:08.133 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54202.log.gz 2024-04-25T10:19:08.134 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50315.log 2024-04-25T10:19:08.134 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43263.log.gz 2024-04-25T10:19:08.135 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74564.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74564.log.gz 2024-04-25T10:19:08.136 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40939.log 2024-04-25T10:19:08.136 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47626.log 2024-04-25T10:19:08.137 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50315.log.gz 2024-04-25T10:19:08.138 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52572.log 2024-04-25T10:19:08.138 INFO:teuthology.orchestra.run.smithi119.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40939.log.gz 2024-04-25T10:19:08.138 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68734.log 2024-04-25T10:19:08.139 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47626.log.gz 2024-04-25T10:19:08.139 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40216.log 2024-04-25T10:19:08.140 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68734.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52572.log.gz 2024-04-25T10:19:08.140 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55386.log 2024-04-25T10:19:08.140 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68734.log.gz 2024-04-25T10:19:08.141 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53375.log 2024-04-25T10:19:08.141 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40216.log.gz 2024-04-25T10:19:08.142 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:19:08.142 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55302.log 2024-04-25T10:19:08.143 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53375.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55386.log.gz 2024-04-25T10:19:08.145 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33333.log 2024-04-25T10:19:08.145 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33293.log 2024-04-25T10:19:08.145 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33333.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:19:08.145 INFO:teuthology.orchestra.run.smithi132.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33333.log.gz 2024-04-25T10:19:08.146 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53375.log.gz 2024-04-25T10:19:08.146 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62441.log 2024-04-25T10:19:08.146 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54913.log 2024-04-25T10:19:08.146 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55302.log.gz 2024-04-25T10:19:08.147 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68902.log 2024-04-25T10:19:08.147 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62441.log.gz 2024-04-25T10:19:08.147 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44451.log 2024-04-25T10:19:08.147 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54913.log.gz 2024-04-25T10:19:08.150 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33453.log 2024-04-25T10:19:08.150 INFO:teuthology.orchestra.run.smithi132.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33293.log.gz 2024-04-25T10:19:08.150 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T10:19:08.150 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33453.log: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:19:08.151 INFO:teuthology.orchestra.run.smithi132.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33453.log.gz 2024-04-25T10:19:08.151 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T10:19:08.151 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T10:19:08.151 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T10:19:08.152 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60525.log 2024-04-25T10:19:08.152 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68902.log.gz 2024-04-25T10:19:08.152 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69543.log 2024-04-25T10:19:08.152 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44451.log.gz 2024-04-25T10:19:08.152 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.60525.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60525.log.gz 2024-04-25T10:19:08.152 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68986.log 2024-04-25T10:19:08.152 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57393.log 2024-04-25T10:19:08.153 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69543.log.gz 2024-04-25T10:19:08.153 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47878.log 2024-04-25T10:19:08.153 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68986.log.gz 2024-04-25T10:19:08.153 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.57393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59534.log 2024-04-25T10:19:08.153 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57393.log.gz 2024-04-25T10:19:08.157 INFO:teuthology.orchestra.run.smithi132.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:19:08.158 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33493.log 2024-04-25T10:19:08.158 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39331.log 2024-04-25T10:19:08.158 INFO:teuthology.orchestra.run.smithi132.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T10:19:08.158 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33493.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33493.log.gz 2024-04-25T10:19:08.158 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T10:19:08.158 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.3.39331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.39331.log.gz 2024-04-25T10:19:08.158 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T10:19:08.158 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39335.log 2024-04-25T10:19:08.159 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59152.log 2024-04-25T10:19:08.159 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47878.log.gz 2024-04-25T10:19:08.159 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54964.log 2024-04-25T10:19:08.159 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59534.log.gz 2024-04-25T10:19:08.159 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54869.log 2024-04-25T10:19:08.159 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59152.log.gz 2024-04-25T10:19:08.159 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63669.log 2024-04-25T10:19:08.160 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54964.log.gz 2024-04-25T10:19:08.160 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48176.log 2024-04-25T10:19:08.160 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54869.log.gz 2024-04-25T10:19:08.160 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69196.log 2024-04-25T10:19:08.160 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63669.log.gz 2024-04-25T10:19:08.160 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48176.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48176.log.gz 2024-04-25T10:19:08.160 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41065.log 2024-04-25T10:19:08.163 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33202.log 2024-04-25T10:19:08.163 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.3.39335.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T10:19:08.163 INFO:teuthology.orchestra.run.smithi132.stderr: -- replaced with /var/log/ceph/ceph-client.3.39335.log.gz 2024-04-25T10:19:08.163 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33413.log 2024-04-25T10:19:08.163 INFO:teuthology.orchestra.run.smithi132.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33202.log.gz 2024-04-25T10:19:08.163 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38944.log 2024-04-25T10:19:08.164 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50627.log 2024-04-25T10:19:08.164 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69196.log.gz 2024-04-25T10:19:08.164 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.41065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49651.log 2024-04-25T10:19:08.164 INFO:teuthology.orchestra.run.smithi119.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41065.log.gz 2024-04-25T10:19:08.165 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56407.log 2024-04-25T10:19:08.165 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50627.log.gz 2024-04-25T10:19:08.165 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.49651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42300.log 2024-04-25T10:19:08.165 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56407.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.49651.log.gz 2024-04-25T10:19:08.165 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70284.log 2024-04-25T10:19:08.165 INFO:teuthology.orchestra.run.smithi119.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.56407.log.gz 2024-04-25T10:19:08.168 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33413.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33413.log.gz 2024-04-25T10:19:08.169 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.1.38944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.38944.log.gz 2024-04-25T10:19:08.169 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33242.log 2024-04-25T10:19:08.169 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose 94.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T10:19:08.169 INFO:teuthology.orchestra.run.smithi132.stderr: -- /var/log/ceph/ceph-mds.d.log 2024-04-25T10:19:08.169 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33242.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T10:19:08.169 INFO:teuthology.orchestra.run.smithi132.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33242.log.gz 2024-04-25T10:19:08.170 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68776.log 2024-04-25T10:19:08.170 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42300.log.gz 2024-04-25T10:19:08.170 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.70284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75970.log 2024-04-25T10:19:08.170 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70284.log.gz 2024-04-25T10:19:08.170 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74019.log 2024-04-25T10:19:08.170 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68776.log.gz 2024-04-25T10:19:08.171 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67831.log 2024-04-25T10:19:08.171 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75970.log.gz 2024-04-25T10:19:08.171 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.74019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56908.log 2024-04-25T10:19:08.171 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74019.log.gz 2024-04-25T10:19:08.171 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52222.log 2024-04-25T10:19:08.171 INFO:teuthology.orchestra.run.smithi119.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.67831.log.gz 2024-04-25T10:19:08.173 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38925.log 2024-04-25T10:19:08.174 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.1.38925.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.38925.log.gz 2024-04-25T10:19:08.174 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33373.log 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52722.log 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56908.log.gz 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.52222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56950.log 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52222.log.gz 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52722.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.52722.log.gz --verbose 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi119.stderr: -- /var/log/ceph/ceph-client.admin.53088.log 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39560.log 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56950.log.gz 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi132.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33373.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.33373.log.gz 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50964.log 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53088.log.gz 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51305.log 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39560.log.gz 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68517.log 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50964.log.gz 2024-04-25T10:19:08.179 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46164.log 2024-04-25T10:19:08.179 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51305.log.gz 2024-04-25T10:19:08.179 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68692.log 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68517.log.gz 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.46164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42636.log 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46164.log.gz 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39903.log 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68692.log.gz 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58325.log 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42636.log.gz 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75044.log 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39903.log.gz 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73720.log 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58325.log.gz 2024-04-25T10:19:08.185 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56279.log 2024-04-25T10:19:08.185 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75044.log.gz 2024-04-25T10:19:08.186 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64802.log 2024-04-25T10:19:08.186 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73720.log.gz 2024-04-25T10:19:08.187 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54244.log 2024-04-25T10:19:08.187 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56279.log.gz 2024-04-25T10:19:08.188 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69406.log 2024-04-25T10:19:08.188 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64802.log.gz 2024-04-25T10:19:08.189 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39884.log 2024-04-25T10:19:08.189 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54244.log.gz 2024-04-25T10:19:08.190 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44661.log 2024-04-25T10:19:08.191 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69406.log.gz 2024-04-25T10:19:08.191 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43467.log 2024-04-25T10:19:08.191 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39884.log.gz 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42888.log 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44661.log.gz 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39559.log 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43467.log.gz 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63376.log 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42888.log.gz 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68256.log 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi119.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39559.log.gz 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.63376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44493.log 2024-04-25T10:19:08.197 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63376.log.gz 2024-04-25T10:19:08.197 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75676.log 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi119.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68256.log.gz 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73130.log 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44493.log.gz 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59194.log 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75676.log.gz 2024-04-25T10:19:08.201 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67294.log 2024-04-25T10:19:08.201 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73130.log.gz 2024-04-25T10:19:08.202 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53333.log 2024-04-25T10:19:08.202 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59194.log.gz 2024-04-25T10:19:08.203 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39343.log 2024-04-25T10:19:08.203 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67294.log.gz 2024-04-25T10:19:08.204 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63184.log 2024-04-25T10:19:08.204 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53333.log.gz 2024-04-25T10:19:08.205 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38532.log 2024-04-25T10:19:08.205 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39343.log.gz 2024-04-25T10:19:08.206 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73093.log 2024-04-25T10:19:08.206 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63184.log.gz 2024-04-25T10:19:08.207 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45346.log 2024-04-25T10:19:08.207 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38532.log.gz 2024-04-25T10:19:08.208 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75928.log 2024-04-25T10:19:08.208 INFO:teuthology.orchestra.run.smithi119.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.73093.log.gz 2024-04-25T10:19:08.209 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43851.log 2024-04-25T10:19:08.210 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45346.log.gz 2024-04-25T10:19:08.210 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51759.log 2024-04-25T10:19:08.210 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75928.log.gz 2024-04-25T10:19:08.211 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39301.log 2024-04-25T10:19:08.211 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43851.log.gz 2024-04-25T10:19:08.212 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38436.log 2024-04-25T10:19:08.213 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51759.log.gz 2024-04-25T10:19:08.213 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55988.log 2024-04-25T10:19:08.214 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39301.log.gz 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56473.log 2024-04-25T10:19:08.215 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38436.log.gz 2024-04-25T10:19:08.216 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48345.log 2024-04-25T10:19:08.216 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55988.log.gz 2024-04-25T10:19:08.217 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65662.log 2024-04-25T10:19:08.217 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56473.log.gz 2024-04-25T10:19:08.218 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75508.log 2024-04-25T10:19:08.218 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48345.log.gz 2024-04-25T10:19:08.219 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65353.log 2024-04-25T10:19:08.219 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65662.log.gz 2024-04-25T10:19:08.220 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41203.log 2024-04-25T10:19:08.220 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75508.log.gz 2024-04-25T10:19:08.221 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39891.log 2024-04-25T10:19:08.222 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.65353.log.gz 2024-04-25T10:19:08.222 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.41203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55452.log 2024-04-25T10:19:08.223 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41203.log.gz 2024-04-25T10:19:08.223 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55344.log 2024-04-25T10:19:08.224 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39891.log.gz 2024-04-25T10:19:08.224 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55452.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:19:08.225 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55452.log.gz 2024-04-25T10:19:08.225 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69722.log 2024-04-25T10:19:08.226 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55344.log.gz 2024-04-25T10:19:08.227 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64320.log 2024-04-25T10:19:08.228 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39007.log 2024-04-25T10:19:08.228 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69722.log.gz 2024-04-25T10:19:08.229 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64320.log: 86.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:19:08.229 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40519.log 2024-04-25T10:19:08.229 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64320.log.gz 2024-04-25T10:19:08.230 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55092.log 2024-04-25T10:19:08.230 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39007.log.gz 2024-04-25T10:19:08.231 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68818.log 2024-04-25T10:19:08.232 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40519.log.gz 2024-04-25T10:19:08.233 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64928.log 2024-04-25T10:19:08.233 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55092.log.gz 2024-04-25T10:19:08.234 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59278.log 2024-04-25T10:19:08.234 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68818.log.gz 2024-04-25T10:19:08.235 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74428.log 2024-04-25T10:19:08.235 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64928.log.gz 2024-04-25T10:19:08.236 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72065.log 2024-04-25T10:19:08.236 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59278.log.gz 2024-04-25T10:19:08.237 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46603.log 2024-04-25T10:19:08.237 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74428.log.gz 2024-04-25T10:19:08.238 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.72065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42342.log 2024-04-25T10:19:08.238 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72065.log.gz 2024-04-25T10:19:08.239 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.46603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73580.log 2024-04-25T10:19:08.239 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46603.log.gz 2024-04-25T10:19:08.240 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51433.log 2024-04-25T10:19:08.240 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42342.log.gz 2024-04-25T10:19:08.241 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50357.log 2024-04-25T10:19:08.241 INFO:teuthology.orchestra.run.smithi119.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73580.log.gz 2024-04-25T10:19:08.242 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60264.log 2024-04-25T10:19:08.242 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51433.log.gz 2024-04-25T10:19:08.243 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65065.log 2024-04-25T10:19:08.243 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50357.log.gz 2024-04-25T10:19:08.244 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39217.log 2024-04-25T10:19:08.245 INFO:teuthology.orchestra.run.smithi119.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60264.log.gz 2024-04-25T10:19:08.245 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47229.log 2024-04-25T10:19:08.245 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65065.log.gz 2024-04-25T10:19:08.246 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41023.log 2024-04-25T10:19:08.246 INFO:teuthology.orchestra.run.smithi119.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39217.log.gz 2024-04-25T10:19:08.247 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39175.log 2024-04-25T10:19:08.248 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47229.log.gz 2024-04-25T10:19:08.248 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.41023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54627.log 2024-04-25T10:19:08.249 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41023.log.gz 2024-04-25T10:19:08.249 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54730.log 2024-04-25T10:19:08.249 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39175.log.gz 2024-04-25T10:19:08.250 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43767.log 2024-04-25T10:19:08.251 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54627.log.gz 2024-04-25T10:19:08.251 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45434.log 2024-04-25T10:19:08.252 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54730.log.gz 2024-04-25T10:19:08.252 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43767.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T10:19:08.253 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43767.log.gz 2024-04-25T10:19:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49969.log 2024-04-25T10:19:08.254 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45434.log.gz 2024-04-25T10:19:08.254 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.42003.log 2024-04-25T10:19:08.256 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.49969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65956.log 2024-04-25T10:19:08.256 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49969.log.gz 2024-04-25T10:19:08.257 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.2.42003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70158.log 2024-04-25T10:19:08.257 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.42003.log.gz 2024-04-25T10:19:08.257 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54465.log 2024-04-25T10:19:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.70158.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.65956.log.gz 2024-04-25T10:19:08.259 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71801.log 2024-04-25T10:19:08.259 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70158.log.gz 2024-04-25T10:19:08.260 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39551.log 2024-04-25T10:19:08.260 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54465.log.gz 2024-04-25T10:19:08.261 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.71801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39567.log 2024-04-25T10:19:08.261 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71801.log.gz 2024-04-25T10:19:08.262 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58367.log 2024-04-25T10:19:08.262 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39551.log.gz 2024-04-25T10:19:08.263 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39385.log 2024-04-25T10:19:08.263 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39567.log.gz 2024-04-25T10:19:08.264 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45781.log 2024-04-25T10:19:08.264 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58367.log.gz 2024-04-25T10:19:08.265 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59453.log 2024-04-25T10:19:08.265 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39385.log.gz 2024-04-25T10:19:08.266 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57819.log 2024-04-25T10:19:08.267 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45781.log.gz 2024-04-25T10:19:08.267 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39552.log 2024-04-25T10:19:08.268 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59453.log.gz 2024-04-25T10:19:08.268 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.57819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71675.log 2024-04-25T10:19:08.269 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57819.log.gz 2024-04-25T10:19:08.270 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66022.log 2024-04-25T10:19:08.270 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39552.log.gz 2024-04-25T10:19:08.270 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.71675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40855.log 2024-04-25T10:19:08.271 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71675.log.gz 2024-04-25T10:19:08.272 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38740.log 2024-04-25T10:19:08.272 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66022.log.gz 2024-04-25T10:19:08.273 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51675.log 2024-04-25T10:19:08.273 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40855.log.gz 2024-04-25T10:19:08.274 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42468.log 2024-04-25T10:19:08.274 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38740.log.gz 2024-04-25T10:19:08.275 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66619.log 2024-04-25T10:19:08.275 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51675.log.gz 2024-04-25T10:19:08.276 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53637.log 2024-04-25T10:19:08.276 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42468.log.gz 2024-04-25T10:19:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60180.log 2024-04-25T10:19:08.277 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66619.log.gz 2024-04-25T10:19:08.278 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42174.log 2024-04-25T10:19:08.278 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53637.log.gz 2024-04-25T10:19:08.279 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51477.log 2024-04-25T10:19:08.279 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60180.log.gz 2024-04-25T10:19:08.280 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65576.log 2024-04-25T10:19:08.280 INFO:teuthology.orchestra.run.smithi119.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42174.log.gz 2024-04-25T10:19:08.281 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38655.log 2024-04-25T10:19:08.281 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51477.log.gz 2024-04-25T10:19:08.282 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42720.log 2024-04-25T10:19:08.283 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65576.log.gz 2024-04-25T10:19:08.284 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38655.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38655.log.gz 2024-04-25T10:19:08.284 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45569.log 2024-04-25T10:19:08.284 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65395.log 2024-04-25T10:19:08.284 INFO:teuthology.orchestra.run.smithi119.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.42720.log.gz 2024-04-25T10:19:08.285 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68043.log 2024-04-25T10:19:08.286 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45569.log.gz 2024-04-25T10:19:08.286 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51220.log 2024-04-25T10:19:08.287 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68043.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65395.log.gz 2024-04-25T10:19:08.288 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68043.log.gz 2024-04-25T10:19:08.288 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38868.log 2024-04-25T10:19:08.289 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54669.log 2024-04-25T10:19:08.289 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51220.log.gz 2024-04-25T10:19:08.290 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38868.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38868.log.gz 2024-04-25T10:19:08.290 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72935.log 2024-04-25T10:19:08.291 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54669.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:19:08.291 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.72935.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54669.log.gz 2024-04-25T10:19:08.292 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72935.log.gz 2024-04-25T10:19:08.292 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70030.log 2024-04-25T10:19:08.293 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67741.log 2024-04-25T10:19:08.294 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.70030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56664.log 2024-04-25T10:19:08.295 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70030.log.gz 2024-04-25T10:19:08.295 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67741.log: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:19:08.296 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26345.log 2024-04-25T10:19:08.296 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67741.log.gz 2024-04-25T10:19:08.297 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.56664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72247.log 2024-04-25T10:19:08.297 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56664.log.gz 2024-04-25T10:19:08.298 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.tmp-client.admin.26345.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26345.log.gz 2024-04-25T10:19:08.298 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48218.log 2024-04-25T10:19:08.299 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.72247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58090.log 2024-04-25T10:19:08.299 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.72247.log.gz 2024-04-25T10:19:08.300 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48218.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48218.log.gz 2024-04-25T10:19:08.300 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38912.log 2024-04-25T10:19:08.301 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62085.log 2024-04-25T10:19:08.302 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38912.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.58090.log.gz 2024-04-25T10:19:08.303 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38912.log.gz 2024-04-25T10:19:08.303 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66760.log 2024-04-25T10:19:08.303 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50754.log 2024-04-25T10:19:08.304 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62085.log.gz 2024-04-25T10:19:08.304 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49309.log 2024-04-25T10:19:08.305 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66760.log.gz 2024-04-25T10:19:08.305 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76347.log 2024-04-25T10:19:08.306 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.49309.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50754.log.gz 2024-04-25T10:19:08.307 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49309.log.gz 2024-04-25T10:19:08.307 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50525.log 2024-04-25T10:19:08.307 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65311.log 2024-04-25T10:19:08.308 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76347.log.gz 2024-04-25T10:19:08.309 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64488.log 2024-04-25T10:19:08.309 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50525.log.gz 2024-04-25T10:19:08.310 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74690.log 2024-04-25T10:19:08.310 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1%/var/log/ceph/ceph-client.admin.64488.log: -- replaced with /var/log/ceph/ceph-client.admin.65311.log.gz 2024-04-25T10:19:08.311 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64488.log.gz 2024-04-25T10:19:08.311 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70410.log 2024-04-25T10:19:08.312 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66213.log 2024-04-25T10:19:08.312 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74690.log.gz 2024-04-25T10:19:08.313 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.70410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52035.log 2024-04-25T10:19:08.313 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70410.log.gz 2024-04-25T10:19:08.314 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.66213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66213.log.gz 2024-04-25T10:19:08.314 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60537.log 2024-04-25T10:19:08.315 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40231.log 2024-04-25T10:19:08.315 INFO:teuthology.orchestra.run.smithi119.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52035.log.gz 2024-04-25T10:19:08.316 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.60537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60537.log.gz 2024-04-25T10:19:08.316 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64633.log 2024-04-25T10:19:08.317 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68171.log 2024-04-25T10:19:08.317 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40231.log.gz 2024-04-25T10:19:08.318 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75466.log 2024-04-25T10:19:08.319 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64633.log.gz 2024-04-25T10:19:08.319 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67336.log 2024-04-25T10:19:08.320 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68171.log.gz 2024-04-25T10:19:08.320 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58933.log 2024-04-25T10:19:08.321 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75466.log.gz 2024-04-25T10:19:08.322 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67570.log 2024-04-25T10:19:08.322 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67336.log.gz 2024-04-25T10:19:08.322 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75760.log 2024-04-25T10:19:08.323 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58933.log.gz 2024-04-25T10:19:08.323 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73634.log 2024-04-25T10:19:08.324 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67570.log.gz 2024-04-25T10:19:08.324 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55134.log 2024-04-25T10:19:08.325 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8%/var/log/ceph/ceph-client.admin.73634.log: -- replaced with /var/log/ceph/ceph-client.admin.75760.log.gz 2024-04-25T10:19:08.325 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73634.log.gz 2024-04-25T10:19:08.326 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59703.log 2024-04-25T10:19:08.326 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46785.log 2024-04-25T10:19:08.327 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55134.log.gz 2024-04-25T10:19:08.328 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65830.log 2024-04-25T10:19:08.328 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59703.log.gz 2024-04-25T10:19:08.329 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.46785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52450.log 2024-04-25T10:19:08.329 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46785.log.gz 2024-04-25T10:19:08.330 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53973.log 2024-04-25T10:19:08.330 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65830.log.gz 2024-04-25T10:19:08.331 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67252.log 2024-04-25T10:19:08.331 INFO:teuthology.orchestra.run.smithi119.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.52450.log.gz 2024-04-25T10:19:08.332 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73167.log 2024-04-25T10:19:08.332 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53973.log.gz 2024-04-25T10:19:08.333 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43809.log 2024-04-25T10:19:08.333 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67252.log.gz 2024-04-25T10:19:08.334 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60775.log 2024-04-25T10:19:08.334 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73167.log.gz 2024-04-25T10:19:08.335 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40561.log 2024-04-25T10:19:08.335 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43809.log.gz 2024-04-25T10:19:08.336 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61282.log 2024-04-25T10:19:08.336 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60775.log.gz 2024-04-25T10:19:08.337 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64760.log 2024-04-25T10:19:08.337 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40561.log.gz 2024-04-25T10:19:08.338 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70072.log 2024-04-25T10:19:08.338 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.61282.log.gz 2024-04-25T10:19:08.340 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75802.log 2024-04-25T10:19:08.340 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64760.log.gz 2024-04-25T10:19:08.340 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.70072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60817.log 2024-04-25T10:19:08.341 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70072.log.gz 2024-04-25T10:19:08.341 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57285.log 2024-04-25T10:19:08.342 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75802.log.gz 2024-04-25T10:19:08.343 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47271.log 2024-04-25T10:19:08.343 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60817.log.gz 2024-04-25T10:19:08.344 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.57285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51389.log 2024-04-25T10:19:08.344 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57285.log.gz 2024-04-25T10:19:08.345 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58881.log 2024-04-25T10:19:08.345 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47271.log.gz 2024-04-25T10:19:08.346 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70200.log 2024-04-25T10:19:08.346 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51389.log.gz 2024-04-25T10:19:08.347 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58881.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T10:19:08.347 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58881.log.gz 2024-04-25T10:19:08.348 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.70200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49352.log 2024-04-25T10:19:08.348 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70200.log.gz 2024-04-25T10:19:08.349 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47397.log 2024-04-25T10:19:08.350 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.49352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48484.log 2024-04-25T10:19:08.350 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49352.log.gz 2024-04-25T10:19:08.352 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46645.log 2024-04-25T10:19:08.352 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47397.log.gz 2024-04-25T10:19:08.353 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62936.log 2024-04-25T10:19:08.353 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48484.log.gz 2024-04-25T10:19:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.46645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69154.log 2024-04-25T10:19:08.354 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46645.log.gz 2024-04-25T10:19:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76096.log 2024-04-25T10:19:08.356 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62936.log.gz 2024-04-25T10:19:08.356 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60096.log 2024-04-25T10:19:08.357 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69154.log.gz 2024-04-25T10:19:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74386.log 2024-04-25T10:19:08.358 INFO:teuthology.orchestra.run.smithi119.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76096.log.gz 2024-04-25T10:19:08.359 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47836.log 2024-04-25T10:19:08.359 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60096.log.gz 2024-04-25T10:19:08.360 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48882.log 2024-04-25T10:19:08.360 INFO:teuthology.orchestra.run.smithi119.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.74386.log.gz 2024-04-25T10:19:08.361 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48798.log 2024-04-25T10:19:08.361 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47836.log.gz 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73462.log 2024-04-25T10:19:08.363 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48882.log.gz 2024-04-25T10:19:08.364 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50922.log 2024-04-25T10:19:08.364 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48798.log.gz 2024-04-25T10:19:08.365 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61621.log 2024-04-25T10:19:08.366 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50922.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.73462.log.gz 2024-04-25T10:19:08.367 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53679.log 2024-04-25T10:19:08.367 INFO:teuthology.orchestra.run.smithi119.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50922.log.gz 2024-04-25T10:19:08.367 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.61621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42762.log 2024-04-25T10:19:08.368 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61621.log.gz 2024-04-25T10:19:08.369 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45823.log 2024-04-25T10:19:08.369 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53679.log.gz 2024-04-25T10:19:08.370 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66951.log 2024-04-25T10:19:08.370 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42762.log.gz 2024-04-25T10:19:08.371 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63872.log 2024-04-25T10:19:08.372 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45823.log.gz 2024-04-25T10:19:08.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.66951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66951.log.gz 2024-04-25T10:19:08.372 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59842.log 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59068.log 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63872.log.gz 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45094.log 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59842.log.gz 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46723.log 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59068.log.gz 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40477.log 2024-04-25T10:19:08.378 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45094.log.gz 2024-04-25T10:19:08.378 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.46723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59373.log 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46723.log.gz 2024-04-25T10:19:08.380 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40477.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40477.log.gz 2024-04-25T10:19:08.380 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51529.log 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59884.log 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59373.log.gz 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68428.log 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51529.log.gz 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi132.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47481.log 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi119.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.59884.log.gz 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61240.log 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68428.log.gz 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47481.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.47481.log.gz -5 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46477.log 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62773.log 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61240.log.gz 2024-04-25T10:19:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.46477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72522.log 2024-04-25T10:19:08.389 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46477.log.gz 2024-04-25T10:19:08.390 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42846.log 2024-04-25T10:19:08.391 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.72522.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72522.log.gz 2024-04-25T10:19:08.391 INFO:teuthology.orchestra.run.smithi119.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.62773.log.gz 2024-04-25T10:19:08.391 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69364.log 2024-04-25T10:19:08.392 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74900.log 2024-04-25T10:19:08.393 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42846.log.gz 2024-04-25T10:19:08.394 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69364.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69364.log.gzgzip 2024-04-25T10:19:08.394 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T10:19:08.395 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43389.log 2024-04-25T10:19:08.395 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74900.log.gz 2024-04-25T10:19:08.396 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74606.log 2024-04-25T10:19:08.397 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43389.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43389.log.gz 2024-04-25T10:19:08.397 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54824.log 2024-04-25T10:19:08.398 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45697.log 2024-04-25T10:19:08.399 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74606.log.gz 2024-04-25T10:19:08.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75886.log 2024-04-25T10:19:08.400 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54824.log.gz 2024-04-25T10:19:08.401 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49927.log 2024-04-25T10:19:08.401 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45697.log.gz 2024-04-25T10:19:08.402 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55946.log 2024-04-25T10:19:08.402 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75886.log.gz 2024-04-25T10:19:08.404 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.49927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51927.log 2024-04-25T10:19:08.404 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49927.log.gz 2024-04-25T10:19:08.405 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48095.log 2024-04-25T10:19:08.405 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55946.log.gz 2024-04-25T10:19:08.406 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46920.log 2024-04-25T10:19:08.406 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51927.log.gz 2024-04-25T10:19:08.408 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67378.log 2024-04-25T10:19:08.408 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48095.log.gz 2024-04-25T10:19:08.409 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.46920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44844.log 2024-04-25T10:19:08.409 INFO:teuthology.orchestra.run.smithi119.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.46920.log.gz 2024-04-25T10:19:08.410 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53889.log 2024-04-25T10:19:08.410 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67378.log.gz 2024-04-25T10:19:08.411 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42510.log 2024-04-25T10:19:08.412 INFO:teuthology.orchestra.run.smithi132.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T10:19:08.412 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44844.log.gz 2024-04-25T10:19:08.413 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69623.log 2024-04-25T10:19:08.413 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53889.log.gz 2024-04-25T10:19:08.414 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48840.log 2024-04-25T10:19:08.414 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42510.log.gz 2024-04-25T10:19:08.415 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69623.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T10:19:08.416 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69623.log.gz 2024-04-25T10:19:08.416 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63076.log 2024-04-25T10:19:08.417 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48840.log.gz 2024-04-25T10:19:08.418 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47794.log 2024-04-25T10:19:08.419 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63076.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T10:19:08.420 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63076.log.gz 2024-04-25T10:19:08.421 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65268.log 2024-04-25T10:19:08.421 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47794.log.gz 2024-04-25T10:19:08.422 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74816.log 2024-04-25T10:19:08.423 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53022.log 2024-04-25T10:19:08.424 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65268.log.gz 2024-04-25T10:19:08.425 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57697.log 2024-04-25T10:19:08.425 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74816.log.gz 2024-04-25T10:19:08.426 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76012.log 2024-04-25T10:19:08.427 INFO:teuthology.orchestra.run.smithi119.stderr: 91.1%/var/log/ceph/ceph-client.admin.57697.log: -- replaced with /var/log/ceph/ceph-client.admin.53022.log.gz 2024-04-25T10:19:08.428 INFO:teuthology.orchestra.run.smithi119.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57697.log.gz 2024-04-25T10:19:08.428 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59970.log 2024-04-25T10:19:08.429 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76012.log.gz 2024-04-25T10:19:08.429 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61935.log 2024-04-25T10:19:08.431 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76392.log 2024-04-25T10:19:08.431 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59970.log.gz 2024-04-25T10:19:08.432 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63711.log 2024-04-25T10:19:08.433 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61935.log.gz 2024-04-25T10:19:08.433 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76392.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76392.log.gz 2024-04-25T10:19:08.434 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45052.log 2024-04-25T10:19:08.435 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63711.log: 91.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.63711.log.gz 2024-04-25T10:19:08.435 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47583.log 2024-04-25T10:19:08.436 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42216.log 2024-04-25T10:19:08.437 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45052.log.gz 2024-04-25T10:19:08.438 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47583.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47583.log.gz 2024-04-25T10:19:08.438 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51969.log 2024-04-25T10:19:08.439 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42216.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42216.log.gz 2024-04-25T10:19:08.439 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47710.log 2024-04-25T10:19:08.441 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51571.log 2024-04-25T10:19:08.442 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51969.log: /var/log/ceph/ceph-client.admin.47710.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51969.log.gz 2024-04-25T10:19:08.442 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47710.log.gz 2024-04-25T10:19:08.443 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33056.log 2024-04-25T10:19:08.443 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51571.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51571.log.gz 2024-04-25T10:19:08.444 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T10:19:08.445 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52226.log 2024-04-25T10:19:08.446 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33056.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.33056.log.gz/var/log/ceph/ceph-osd.0.log: 2024-04-25T10:19:08.447 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.52226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52226.log.gz 2024-04-25T10:19:08.448 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50880.log 2024-04-25T10:19:08.449 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40687.log 2024-04-25T10:19:08.451 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50880.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50880.log.gz 2024-04-25T10:19:08.452 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40687.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40687.log.gz 2024-04-25T10:19:08.452 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52938.log 2024-04-25T10:19:08.454 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72851.log 2024-04-25T10:19:08.456 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52938.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.52938.log.gz 2024-04-25T10:19:08.457 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.72851.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72851.log.gz 2024-04-25T10:19:08.457 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53291.log 2024-04-25T10:19:08.459 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44055.log 2024-04-25T10:19:08.460 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53291.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53291.log.gz 2024-04-25T10:19:08.461 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44055.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44055.log.gz 2024-04-25T10:19:08.462 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65439.log 2024-04-25T10:19:08.463 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44764.log 2024-04-25T10:19:08.465 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65439.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65439.log.gz 2024-04-25T10:19:08.466 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44764.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44764.log.gz 2024-04-25T10:19:08.467 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75171.log 2024-04-25T10:19:08.468 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62343.log 2024-04-25T10:19:08.470 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75171.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75171.log.gz 2024-04-25T10:19:08.471 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62343.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.62343.log.gz 2024-04-25T10:19:08.472 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49885.log 2024-04-25T10:19:08.473 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50670.log 2024-04-25T10:19:08.475 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.49885.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49885.log.gz 2024-04-25T10:19:08.476 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50670.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50670.log.gz 2024-04-25T10:19:08.477 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40603.log 2024-04-25T10:19:08.478 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69807.log 2024-04-25T10:19:08.480 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40603.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40603.log.gz 2024-04-25T10:19:08.481 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69807.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69807.log.gz 2024-04-25T10:19:08.481 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44307.log 2024-04-25T10:19:08.483 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75844.log 2024-04-25T10:19:08.484 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44307.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44307.log.gz 2024-04-25T10:19:08.486 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75844.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75844.log.gz 2024-04-25T10:19:08.486 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53805.log 2024-04-25T10:19:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51717.log 2024-04-25T10:19:08.489 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53805.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53805.log.gz 2024-04-25T10:19:08.491 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51717.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51717.log.gz 2024-04-25T10:19:08.491 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55048.log 2024-04-25T10:19:08.493 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56660.log 2024-04-25T10:19:08.494 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55048.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55048.log.gz 2024-04-25T10:19:08.494 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.56660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56660.log.gz 2024-04-25T10:19:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52656.log 2024-04-25T10:19:08.497 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75213.log 2024-04-25T10:19:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52656.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52656.log.gz 2024-04-25T10:19:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41681.log 2024-04-25T10:19:08.501 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75213.log.gz 2024-04-25T10:19:08.501 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61625.log 2024-04-25T10:19:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.41681.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41681.log.gz 2024-04-25T10:19:08.503 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.61625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45390.log 2024-04-25T10:19:08.503 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61625.log.gz 2024-04-25T10:19:08.505 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44535.log 2024-04-25T10:19:08.506 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45390.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45390.log.gz 2024-04-25T10:19:08.507 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44535.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44535.log.gz 2024-04-25T10:19:08.508 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59747.log 2024-04-25T10:19:08.509 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39565.log 2024-04-25T10:19:08.511 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59747.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59747.log.gz 2024-04-25T10:19:08.512 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39565.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39565.log.gz 2024-04-25T10:19:08.513 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47523.log 2024-04-25T10:19:08.514 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51885.log 2024-04-25T10:19:08.516 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47523.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47523.log.gz 2024-04-25T10:19:08.517 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51885.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51885.log.gz 2024-04-25T10:19:08.517 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51631.log 2024-04-25T10:19:08.519 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61977.log 2024-04-25T10:19:08.520 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51631.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51631.log.gz 2024-04-25T10:19:08.522 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61977.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.61977.log.gz 2024-04-25T10:19:08.522 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48526.log 2024-04-25T10:19:08.524 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67569.log 2024-04-25T10:19:08.525 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48526.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48526.log.gz 2024-04-25T10:19:08.527 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46846.log 2024-04-25T10:19:08.528 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67569.log.gz 2024-04-25T10:19:08.529 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42930.log 2024-04-25T10:19:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.46846.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.46846.log.gz 2024-04-25T10:19:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42930.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42930.log.gz 2024-04-25T10:19:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50011.log 2024-04-25T10:19:08.533 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40238.log 2024-04-25T10:19:08.535 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50011.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50011.log.gz 2024-04-25T10:19:08.536 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40238.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40238.log.gz 2024-04-25T10:19:08.537 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46519.log 2024-04-25T10:19:08.538 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47355.log 2024-04-25T10:19:08.540 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.46519.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46519.log.gz 2024-04-25T10:19:08.541 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47355.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47355.log.gz 2024-04-25T10:19:08.541 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55006.log 2024-04-25T10:19:08.543 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62990.log 2024-04-25T10:19:08.544 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55006.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55006.log.gz 2024-04-25T10:19:08.546 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62990.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62990.log.gz 2024-04-25T10:19:08.546 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38613.log 2024-04-25T10:19:08.548 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61326.log 2024-04-25T10:19:08.549 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38613.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38613.log.gz 2024-04-25T10:19:08.551 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61326.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61326.log.gz 2024-04-25T10:19:08.551 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44181.log 2024-04-25T10:19:08.553 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68494.log 2024-04-25T10:19:08.554 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44181.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44181.log.gz 2024-04-25T10:19:08.554 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56824.log 2024-04-25T10:19:08.555 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68494.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68494.log.gz 2024-04-25T10:19:08.556 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33440.log 2024-04-25T10:19:08.557 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56824.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56824.log.gz 2024-04-25T10:19:08.558 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50796.log 2024-04-25T10:19:08.559 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33440.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33440.log.gz 2024-04-25T10:19:08.560 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66439.log 2024-04-25T10:19:08.561 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50796.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50796.log.gz 2024-04-25T10:19:08.563 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66439.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66439.log.gz 2024-04-25T10:19:08.563 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50189.log 2024-04-25T10:19:08.565 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53763.log 2024-04-25T10:19:08.566 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50189.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50189.log.gz 2024-04-25T10:19:08.567 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66209.log 2024-04-25T10:19:08.568 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53763.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53763.log.gz 2024-04-25T10:19:08.569 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64970.log 2024-04-25T10:19:08.569 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.66209.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66209.log.gz 2024-04-25T10:19:08.570 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74858.log 2024-04-25T10:19:08.571 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64970.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64970.log.gz 2024-04-25T10:19:08.572 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70368.log 2024-04-25T10:19:08.573 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74858.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74858.log.gz 2024-04-25T10:19:08.575 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.70368.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70368.log.gz 2024-04-25T10:19:08.575 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48261.log 2024-04-25T10:19:08.577 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48612.log 2024-04-25T10:19:08.578 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48261.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48261.log.gz 2024-04-25T10:19:08.579 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73289.log 2024-04-25T10:19:08.580 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48612.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48612.log.gz 2024-04-25T10:19:08.581 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53196.log 2024-04-25T10:19:08.582 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73289.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73289.log.gz 2024-04-25T10:19:08.583 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74344.log 2024-04-25T10:19:08.583 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53196.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53196.log.gz 2024-04-25T10:19:08.584 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69979.log 2024-04-25T10:19:08.585 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74344.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74344.log.gz 2024-04-25T10:19:08.586 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39259.log 2024-04-25T10:19:08.587 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69979.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69979.log.gz 2024-04-25T10:19:08.588 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59619.log 2024-04-25T10:19:08.589 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39259.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39259.log.gz 2024-04-25T10:19:08.590 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45178.log 2024-04-25T10:19:08.591 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59619.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59619.log.gz 2024-04-25T10:19:08.592 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74732.log 2024-04-25T10:19:08.593 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45178.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45178.log.gz 2024-04-25T10:19:08.594 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64278.log 2024-04-25T10:19:08.595 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74732.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74732.log.gz 2024-04-25T10:19:08.595 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76054.log 2024-04-25T10:19:08.596 INFO:teuthology.orchestra.run.smithi119.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T10:19:08.596 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64278.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64278.log.gz 2024-04-25T10:19:08.597 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56365.log 2024-04-25T10:19:08.598 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56992.log 2024-04-25T10:19:08.598 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76054.log.gz 2024-04-25T10:19:08.603 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47145.log 2024-04-25T10:19:08.603 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56365.log.gz 2024-04-25T10:19:08.603 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56992.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56992.log.gz 2024-04-25T10:19:08.603 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47145.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47145.log.gz 2024-04-25T10:19:08.603 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62749.log 2024-04-25T10:19:08.604 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64236.log 2024-04-25T10:19:08.606 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64362.log 2024-04-25T10:19:08.606 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62749.log.gz 2024-04-25T10:19:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64236.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64236.log.gz 2024-04-25T10:19:08.607 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44925.log 2024-04-25T10:19:08.609 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64362.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48672.log 2024-04-25T10:19:08.609 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64362.log.gz 2024-04-25T10:19:08.610 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57589.log 2024-04-25T10:19:08.610 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44925.log.gz 2024-04-25T10:19:08.612 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48672.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53553.log 2024-04-25T10:19:08.612 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48672.log.gz 2024-04-25T10:19:08.612 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.57589.log: 92.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T10:19:08.613 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57589.log.gz 2024-04-25T10:19:08.613 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54118.log 2024-04-25T10:19:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53553.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53553.log.gz 2024-04-25T10:19:08.615 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68255.log 2024-04-25T10:19:08.616 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54118.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54118.log.gz 2024-04-25T10:19:08.616 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45136.log 2024-04-25T10:19:08.617 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43056.log 2024-04-25T10:19:08.618 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45136.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68255.log.gz 2024-04-25T10:19:08.618 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.45136.log.gz 2024-04-25T10:19:08.618 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68649.log 2024-04-25T10:19:08.619 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43056.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43056.log.gz 2024-04-25T10:19:08.619 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66355.log 2024-04-25T10:19:08.620 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66947.log 2024-04-25T10:19:08.621 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66355.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68649.log.gz 2024-04-25T10:19:08.622 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65483.log 2024-04-25T10:19:08.622 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66355.log.gz 2024-04-25T10:19:08.623 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.66947.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66947.log.gz 2024-04-25T10:19:08.623 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72205.log 2024-04-25T10:19:08.624 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48389.log 2024-04-25T10:19:08.625 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65483.log.gz 2024-04-25T10:19:08.626 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.72205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70452.log 2024-04-25T10:19:08.626 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72205.log.gz 2024-04-25T10:19:08.627 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48389.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T10:19:08.627 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48389.log.gz 2024-04-25T10:19:08.628 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.70452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62301.log 2024-04-25T10:19:08.629 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mon.a.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70452.log.gz 2024-04-25T10:19:08.629 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64676.log 2024-04-25T10:19:08.631 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42804.log 2024-04-25T10:19:08.631 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62301.log.gz 2024-04-25T10:19:08.632 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55639.log 2024-04-25T10:19:08.632 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64676.log.gz 2024-04-25T10:19:08.633 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42804.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42804.log.gz 2024-04-25T10:19:08.634 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70954.log 2024-04-25T10:19:08.634 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.55639.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55639.log.gz 2024-04-25T10:19:08.635 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76264.log 2024-04-25T10:19:08.636 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.70954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45655.log 2024-04-25T10:19:08.636 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70954.log.gz 2024-04-25T10:19:08.638 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62043.log 2024-04-25T10:19:08.638 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76264.log.gz 2024-04-25T10:19:08.639 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67808.log 2024-04-25T10:19:08.639 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45655.log.gz 2024-04-25T10:19:08.640 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62043.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62043.log.gz 2024-04-25T10:19:08.641 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43014.log 2024-04-25T10:19:08.641 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67808.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67808.log.gz 2024-04-25T10:19:08.642 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64058.log 2024-04-25T10:19:08.643 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50567.log 2024-04-25T10:19:08.643 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43014.log.gz 2024-04-25T10:19:08.645 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69028.log 2024-04-25T10:19:08.645 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64058.log.gz 2024-04-25T10:19:08.646 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46957.log 87.8% 2024-04-25T10:19:08.646 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50567.log.gz 2024-04-25T10:19:08.647 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69028.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69028.log.gz 2024-04-25T10:19:08.647 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49717.log 2024-04-25T10:19:08.649 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.46957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43593.log 2024-04-25T10:19:08.649 INFO:teuthology.orchestra.run.smithi119.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.46957.log.gz 2024-04-25T10:19:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.49717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69322.log 2024-04-25T10:19:08.651 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49717.log.gz 2024-04-25T10:19:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43593.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43593.log.gz 2024-04-25T10:19:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70996.log 2024-04-25T10:19:08.653 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69322.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69322.log.gz 2024-04-25T10:19:08.654 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60859.log 2024-04-25T10:19:08.655 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.70996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52506.log 2024-04-25T10:19:08.655 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70996.log.gz 2024-04-25T10:19:08.656 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60859.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60859.log.gz 2024-04-25T10:19:08.657 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5/var/log/ceph/ceph-client.admin.52506.log: --verbose -- /var/log/ceph/ceph-client.admin.49567.log 2024-04-25T10:19:08.658 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52506.log.gz 2024-04-25T10:19:08.659 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60054.log 2024-04-25T10:19:08.660 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.49567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53847.log 2024-04-25T10:19:08.661 INFO:teuthology.orchestra.run.smithi119.stderr: 91.2%/var/log/ceph/ceph-client.admin.60054.log: -- replaced with /var/log/ceph/ceph-client.admin.49567.log.gz 2024-04-25T10:19:08.662 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 88.5% --verbose -- /var/log/ceph/ceph-client.admin.60733.log 2024-04-25T10:19:08.662 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60054.log.gz 2024-04-25T10:19:08.662 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53847.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53847.log.gz 2024-04-25T10:19:08.663 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62875.log 2024-04-25T10:19:08.665 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60733.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60733.log.gz 2024-04-25T10:19:08.665 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44097.log 2024-04-25T10:19:08.666 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42132.log 2024-04-25T10:19:08.668 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44097.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.62875.log.gz 2024-04-25T10:19:08.668 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44097.log.gz 2024-04-25T10:19:08.668 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74984.log 2024-04-25T10:19:08.669 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42132.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70114.log 2024-04-25T10:19:08.669 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42132.log.gz 2024-04-25T10:19:08.670 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45262.log 2024-04-25T10:19:08.670 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74984.log.gz 2024-04-25T10:19:08.672 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.70114.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70114.log.gz 2024-04-25T10:19:08.672 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62567.log 2024-04-25T10:19:08.673 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45262.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45262.log.gz 2024-04-25T10:19:08.673 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.41999.log 2024-04-25T10:19:08.675 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68343.log 2024-04-25T10:19:08.675 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62567.log.gz 2024-04-25T10:19:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.2.41999.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.41999.log.gz 2024-04-25T10:19:08.677 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55218.log 2024-04-25T10:19:08.678 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60979.log 2024-04-25T10:19:08.678 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68343.log.gz 2024-04-25T10:19:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50231.log 2024-04-25T10:19:08.680 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55218.log.gz 2024-04-25T10:19:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60979.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60979.log.gz 2024-04-25T10:19:08.681 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61186.log 2024-04-25T10:19:08.682 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50231.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50231.log.gz 2024-04-25T10:19:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40981.log 2024-04-25T10:19:08.684 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61186.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61186.log.gz 2024-04-25T10:19:08.684 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62235.log 2024-04-25T10:19:08.685 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71200.log 2024-04-25T10:19:08.685 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40981.log.gz 2024-04-25T10:19:08.687 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39899.log 2024-04-25T10:19:08.687 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62235.log.gz 2024-04-25T10:19:08.688 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.71200.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.71200.log.gz 2024-04-25T10:19:08.689 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64016.log 2024-04-25T10:19:08.690 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39899.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39899.log.gz 2024-04-25T10:19:08.690 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63118.log 2024-04-25T10:19:08.691 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61785.log 2024-04-25T10:19:08.692 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64016.log.gz 2024-04-25T10:19:08.693 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63974.log 2024-04-25T10:19:08.694 INFO:teuthology.orchestra.run.smithi119.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.63118.log.gz 2024-04-25T10:19:08.695 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54370.log 2024-04-25T10:19:08.695 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61785.log.gz 2024-04-25T10:19:08.696 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63974.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63974.log.gz 2024-04-25T10:19:08.696 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44619.log 2024-04-25T10:19:08.698 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54370.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54370.log.gz 2024-04-25T10:19:08.698 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69935.log 2024-04-25T10:19:08.699 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44619.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44619.log.gz 2024-04-25T10:19:08.699 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60222.log 2024-04-25T10:19:08.701 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40813.log 2024-04-25T10:19:08.701 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69935.log.gz 2024-04-25T10:19:08.702 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52764.log 2024-04-25T10:19:08.702 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60222.log.gz 2024-04-25T10:19:08.703 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40813.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40813.log.gz 2024-04-25T10:19:08.704 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68085.log 2024-04-25T10:19:08.706 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52764.log: gzip -5 --verbose -- 91.1% /var/log/ceph/ceph-client.admin.73676.log 2024-04-25T10:19:08.706 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52764.log.gz 2024-04-25T10:19:08.707 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44349.log 2024-04-25T10:19:08.707 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68085.log.gz 2024-04-25T10:19:08.708 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40177.log 2024-04-25T10:19:08.708 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73676.log.gz 2024-04-25T10:19:08.709 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58545.log 2024-04-25T10:19:08.710 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44349.log.gz 2024-04-25T10:19:08.710 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40177.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40177.log.gz 2024-04-25T10:19:08.711 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38963.log 2024-04-25T10:19:08.712 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48015.log 2024-04-25T10:19:08.712 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58545.log.gz 2024-04-25T10:19:08.714 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58283.log 2024-04-25T10:19:08.714 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38963.log.gz 2024-04-25T10:19:08.715 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48015.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48015.log.gz 2024-04-25T10:19:08.715 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63914.log 2024-04-25T10:19:08.717 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57763.log 2024-04-25T10:19:08.717 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58283.log.gz 2024-04-25T10:19:08.719 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42426.log 2024-04-25T10:19:08.719 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63914.log.gz 2024-04-25T10:19:08.719 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.57763.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57763.log.gz 2024-04-25T10:19:08.721 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42426.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42426.log.gz 2024-04-25T10:19:08.721 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52830.log 2024-04-25T10:19:08.723 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62609.log 2024-04-25T10:19:08.725 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51262.log 2024-04-25T10:19:08.725 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52830.log.gz 2024-04-25T10:19:08.725 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62609.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62609.log.gz 2024-04-25T10:19:08.726 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47313.log 2024-04-25T10:19:08.728 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58755.log 2024-04-25T10:19:08.728 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51262.log.gz 2024-04-25T10:19:08.729 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46883.log 2024-04-25T10:19:08.729 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47313.log.gz 2024-04-25T10:19:08.730 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54033.log 2024-04-25T10:19:08.730 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58755.log.gz 2024-04-25T10:19:08.732 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.46883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49480.log 2024-04-25T10:19:08.732 INFO:teuthology.orchestra.run.smithi119.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.46883.log.gz 2024-04-25T10:19:08.733 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54033.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54033.log.gz 2024-04-25T10:19:08.734 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.49480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76180.log 2024-04-25T10:19:08.734 INFO:teuthology.orchestra.run.smithi119.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49480.log.gz 2024-04-25T10:19:08.735 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65145.log 2024-04-25T10:19:08.738 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48924.log 2024-04-25T10:19:08.738 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76180.log.gz 2024-04-25T10:19:08.739 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65145.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65145.log.gz 2024-04-25T10:19:08.739 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40217.log 2024-04-25T10:19:08.739 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48568.log 2024-04-25T10:19:08.740 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.48924.log.gz 2024-04-25T10:19:08.741 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60138.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40217.log.gz 2024-04-25T10:19:08.741 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-04-25T10:19:08.742 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48568.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48568.log.gz 2024-04-25T10:19:08.742 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45485.log 2024-04-25T10:19:08.744 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53459.log 2024-04-25T10:19:08.744 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60138.log.gz 2024-04-25T10:19:08.745 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45485.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45485.log.gz 2024-04-25T10:19:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44013.log 2024-04-25T10:19:08.746 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53459.log.gz 2024-04-25T10:19:08.747 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40226.log 2024-04-25T10:19:08.748 INFO:teuthology.orchestra.run.smithi132.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T10:19:08.749 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70326.log 2024-04-25T10:19:08.749 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44013.log.gz 2024-04-25T10:19:08.750 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40226.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40226.log.gz 2024-04-25T10:19:08.751 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58199.log 2024-04-25T10:19:08.751 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.70326.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70326.log.gz 2024-04-25T10:19:08.752 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59236.log 2024-04-25T10:19:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69070.log 2024-04-25T10:19:08.753 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58199.log.gz 2024-04-25T10:19:08.755 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45739.log 2024-04-25T10:19:08.755 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59236.log.gz 2024-04-25T10:19:08.756 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69070.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58839.log 2024-04-25T10:19:08.756 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69070.log.gz 2024-04-25T10:19:08.757 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72313.log 2024-04-25T10:19:08.757 INFO:teuthology.orchestra.run.smithi119.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45739.log.gz 2024-04-25T10:19:08.759 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58839.log: gzip -5 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58839.log.gz 2024-04-25T10:19:08.759 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44577.log 2024-04-25T10:19:08.760 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.72313.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72313.log.gz 2024-04-25T10:19:08.760 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69764.log 2024-04-25T10:19:08.762 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50483.log 2024-04-25T10:19:08.762 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44577.log.gz 2024-04-25T10:19:08.764 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69764.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69764.log.gz 2024-04-25T10:19:08.764 INFO:teuthology.orchestra.run.smithi119.stderr:gzip/var/log/ceph/ceph-client.admin.50483.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59110.log 2024-04-25T10:19:08.764 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50483.log.gz 2024-04-25T10:19:08.766 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53417.log 2024-04-25T10:19:08.768 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53931.log 2024-04-25T10:19:08.768 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59110.log.gz 2024-04-25T10:19:08.769 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53417.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53417.log.gz 2024-04-25T10:19:08.770 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53931.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53931.log.gz 2024-04-25T10:19:08.770 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43635.log 2024-04-25T10:19:08.772 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72767.log 2024-04-25T10:19:08.773 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72119.log 2024-04-25T10:19:08.773 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43635.log.gz 2024-04-25T10:19:08.774 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.72767.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72767.log.gz 2024-04-25T10:19:08.775 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68944.log 2024-04-25T10:19:08.776 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.72119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63453.log 2024-04-25T10:19:08.776 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72119.log.gz 2024-04-25T10:19:08.777 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55904.log 2024-04-25T10:19:08.777 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68944.log.gz 2024-04-25T10:19:08.779 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69849.log 2024-04-25T10:19:08.779 INFO:teuthology.orchestra.run.smithi119.stderr: 90.8%/var/log/ceph/ceph-client.admin.55904.log: -- replaced with /var/log/ceph/ceph-client.admin.63453.log.gz 2024-04-25T10:19:08.780 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55904.log.gz 2024-04-25T10:19:08.780 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60901.log 2024-04-25T10:19:08.782 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69849.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.64718.log 2024-04-25T10:19:08.782 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69849.log.gz 2024-04-25T10:19:08.783 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44967.log 2024-04-25T10:19:08.783 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60901.log.gz 2024-04-25T10:19:08.785 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64718.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64718.log.gz 2024-04-25T10:19:08.785 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73223.log 2024-04-25T10:19:08.786 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44967.log.gz 2024-04-25T10:19:08.786 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55260.log 2024-04-25T10:19:08.788 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38824.log 2024-04-25T10:19:08.788 INFO:teuthology.orchestra.run.smithi119.stderr: 91.1%/var/log/ceph/ceph-client.admin.55260.log: -- replaced with /var/log/ceph/ceph-client.admin.73223.log.gz 2024-04-25T10:19:08.789 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55260.log.gz 2024-04-25T10:19:08.790 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52397.log 2024-04-25T10:19:08.791 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38824.log: gzip -5 88.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.38824.log.gz /var/log/ceph/ceph-client.admin.74648.log 2024-04-25T10:19:08.791 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-04-25T10:19:08.792 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50053.log 2024-04-25T10:19:08.792 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52397.log.gz 2024-04-25T10:19:08.794 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74648.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74648.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45527.log 2024-04-25T10:19:08.794 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-04-25T10:19:08.795 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50053.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50053.log.gz 2024-04-25T10:19:08.796 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45527.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45527.log.gz 2024-04-25T10:19:08.797 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43551.log 2024-04-25T10:19:08.798 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64573.log 2024-04-25T10:19:08.799 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43551.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43551.log.gz 2024-04-25T10:19:08.800 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39877.log 2024-04-25T10:19:08.801 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64573.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64573.log.gz 2024-04-25T10:19:08.801 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47668.log 2024-04-25T10:19:08.802 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54545.log 2024-04-25T10:19:08.803 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39877.log.gz 2024-04-25T10:19:08.804 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58797.log 2024-04-25T10:19:08.804 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47668.log.gz 2024-04-25T10:19:08.805 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72893.log 2024-04-25T10:19:08.806 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54545.log.gz 2024-04-25T10:19:08.807 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58797.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58797.log.gz 2024-04-25T10:19:08.807 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57501.log 2024-04-25T10:19:08.808 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.72893.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72893.log.gz 2024-04-25T10:19:08.809 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63603.log 2024-04-25T10:19:08.809 INFO:teuthology.orchestra.run.smithi132.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T10:19:08.810 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.57501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45973.log 2024-04-25T10:19:08.811 INFO:teuthology.orchestra.run.smithi119.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.57501.log.gz 2024-04-25T10:19:08.812 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44139.log 2024-04-25T10:19:08.812 INFO:teuthology.orchestra.run.smithi119.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63603.log.gz 2024-04-25T10:19:08.813 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45973.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45973.log.gz 2024-04-25T10:19:08.814 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71633.log 2024-04-25T10:19:08.814 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44139.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44139.log.gz 2024-04-25T10:19:08.815 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71453.log 2024-04-25T10:19:08.816 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.71633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63032.log 2024-04-25T10:19:08.816 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71633.log.gz 2024-04-25T10:19:08.817 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.71453.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71453.log.gz 2024-04-25T10:19:08.818 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57327.log 2024-04-25T10:19:08.819 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63032.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.63032.log.gz --verbose 2024-04-25T10:19:08.819 INFO:teuthology.orchestra.run.smithi119.stderr: -- /var/log/ceph/ceph-client.admin.43509.log 2024-04-25T10:19:08.820 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.57327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71759.log 2024-04-25T10:19:08.822 INFO:teuthology.orchestra.run.smithi119.stderr: 90.8%/var/log/ceph/ceph-client.admin.43509.log: -- replaced with /var/log/ceph/ceph-client.admin.57327.log.gz 2024-04-25T10:19:08.822 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43509.log.gz 2024-04-25T10:19:08.822 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.64886.log 2024-04-25T10:19:08.823 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.71759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71912.log 2024-04-25T10:19:08.823 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71759.log.gz 2024-04-25T10:19:08.824 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75087.log 2024-04-25T10:19:08.825 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64886.log.gz 2024-04-25T10:19:08.826 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.71912.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71912.log.gz 2024-04-25T10:19:08.827 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75087.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75087.log.gz 2024-04-25T10:19:08.827 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58629.log 2024-04-25T10:19:08.829 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42552.log 2024-04-25T10:19:08.830 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53721.log 2024-04-25T10:19:08.831 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58629.log.gz 2024-04-25T10:19:08.832 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42552.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42552.log.gz 2024-04-25T10:19:08.832 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74774.log 2024-04-25T10:19:08.833 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53721.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66681.log 2024-04-25T10:19:08.833 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53721.log.gz 2024-04-25T10:19:08.835 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52872.log 2024-04-25T10:19:08.835 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74774.log.gz 2024-04-25T10:19:08.836 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66681.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66681.log.gz 2024-04-25T10:19:08.837 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75634.log 2024-04-25T10:19:08.838 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52354.log 2024-04-25T10:19:08.838 INFO:teuthology.orchestra.run.smithi119.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52872.log.gz 2024-04-25T10:19:08.839 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54286.log 2024-04-25T10:19:08.839 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75634.log.gz 2024-04-25T10:19:08.841 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.52354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67989.log 88.8% 2024-04-25T10:19:08.841 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52354.log.gz 2024-04-25T10:19:08.842 INFO:teuthology.orchestra.run.smithi132.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T10:19:08.842 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45865.log 2024-04-25T10:19:08.842 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54286.log.gz 2024-04-25T10:19:08.843 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45907.log 2024-04-25T10:19:08.843 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67989.log.gz 2024-04-25T10:19:08.845 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45865.log: gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45865.log.gz 2024-04-25T10:19:08.845 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.admin.76138.log 2024-04-25T10:19:08.847 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39049.log 2024-04-25T10:19:08.847 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45907.log.gz 2024-04-25T10:19:08.848 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76138.log: gzip -5 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76138.log.gz 2024-04-25T10:19:08.848 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47920.log 2024-04-25T10:19:08.849 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39049.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39049.log.gz 2024-04-25T10:19:08.849 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48433.log 2024-04-25T10:19:08.851 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69448.log 2024-04-25T10:19:08.851 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47920.log.gz 2024-04-25T10:19:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48433.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48433.log.gz 2024-04-25T10:19:08.853 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.69448.log: -- /var/log/ceph/ceph-client.admin.57435.log 2024-04-25T10:19:08.853 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69448.log.gz 2024-04-25T10:19:08.854 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38326.log 2024-04-25T10:19:08.856 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.57435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45304.log 2024-04-25T10:19:08.856 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9%/var/log/ceph/ceph-client.admin.38326.log: -- replaced with /var/log/ceph/ceph-client.admin.57435.log.gz 2024-04-25T10:19:08.857 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38326.log.gz 2024-04-25T10:19:08.858 INFO:teuthology.orchestra.run.smithi119.stderr: -- /var/log/ceph/ceph-client.admin.51347.log 2024-04-25T10:19:08.858 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45304.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45304.log.gz 2024-04-25T10:19:08.859 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47752.log 2024-04-25T10:19:08.860 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39553.log 2024-04-25T10:19:08.860 INFO:teuthology.orchestra.run.smithi119.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51347.log.gz 2024-04-25T10:19:08.861 INFO:teuthology.orchestra.run.smithi132.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T10:19:08.862 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47752.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47752.log.gzgzip 2024-04-25T10:19:08.862 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68129.log 2024-04-25T10:19:08.863 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39553.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39553.log.gz 2024-04-25T10:19:08.863 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66704.log 2024-04-25T10:19:08.864 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70705.log 2024-04-25T10:19:08.865 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68129.log.gz 2024-04-25T10:19:08.865 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-04-25T10:19:08.865 INFO:teuthology.orchestra.run.smithi132.stderr:real 0m0.754s 2024-04-25T10:19:08.865 INFO:teuthology.orchestra.run.smithi132.stderr:user 0m2.929s 2024-04-25T10:19:08.865 INFO:teuthology.orchestra.run.smithi132.stderr:sys 0m0.339s 2024-04-25T10:19:08.867 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66704.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66704.log.gz 2024-04-25T10:19:08.867 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.70705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59791.log 2024-04-25T10:19:08.867 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70705.log.gz 2024-04-25T10:19:08.868 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64404.log 2024-04-25T10:19:08.870 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74942.log 2024-04-25T10:19:08.870 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59791.log.gz 2024-04-25T10:19:08.870 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64404.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64404.log.gz 2024-04-25T10:19:08.871 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43893.log 2024-04-25T10:19:08.872 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73378.log 2024-04-25T10:19:08.872 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74942.log.gz 2024-04-25T10:19:08.873 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70242.log 2024-04-25T10:19:08.874 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43893.log.gz 2024-04-25T10:19:08.875 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70709.log 2024-04-25T10:19:08.876 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.70242.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.73378.log.gz 2024-04-25T10:19:08.876 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose 88.9% -- /var/log/ceph/ceph-client.admin.66397.log -- replaced with /var/log/ceph/ceph-client.admin.70242.log.gz 2024-04-25T10:19:08.876 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-04-25T10:19:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.70709.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70709.log.gz 2024-04-25T10:19:08.878 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49181.log 2024-04-25T10:19:08.879 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58461.log 2024-04-25T10:19:08.879 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66397.log.gz 2024-04-25T10:19:08.880 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.49181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49181.log.gz 2024-04-25T10:19:08.881 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75592.log 2024-04-25T10:19:08.881 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58461.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58461.log.gz 2024-04-25T10:19:08.882 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73420.log 2024-04-25T10:19:08.883 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50441.log 2024-04-25T10:19:08.884 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75592.log.gz 2024-04-25T10:19:08.886 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73420.log: /var/log/ceph/ceph-client.admin.50441.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.73420.log.gz 2024-04-25T10:19:08.886 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50441.log.gz 2024-04-25T10:19:08.886 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58503.log 2024-04-25T10:19:08.887 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50273.log 2024-04-25T10:19:08.889 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40214.log 2024-04-25T10:19:08.889 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58503.log.gz 2024-04-25T10:19:08.890 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50273.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50273.log.gz 2024-04-25T10:19:08.891 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75129.log 2024-04-25T10:19:08.891 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40214.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40214.log.gz 2024-04-25T10:19:08.892 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72809.log 2024-04-25T10:19:08.893 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63519.log 2024-04-25T10:19:08.893 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75129.log.gz 2024-04-25T10:19:08.894 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.72809.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72809.log.gz 2024-04-25T10:19:08.895 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48756.log 2024-04-25T10:19:08.896 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66528.log 2024-04-25T10:19:08.897 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.63519.log.gz 2024-04-25T10:19:08.897 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48756.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.48756.log.gz 2024-04-25T10:19:08.897 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56066.log 2024-04-25T10:19:08.899 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40729.log 2024-04-25T10:19:08.899 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.66528.log.gz 2024-04-25T10:19:08.900 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56066.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.56066.log.gz 2024-04-25T10:19:08.900 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55820.log 2024-04-25T10:19:08.901 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69112.log 2024-04-25T10:19:08.902 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40729.log.gz 2024-04-25T10:19:08.903 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55820.log: gzip -5 --verbose 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55820.log.gz 2024-04-25T10:19:08.903 INFO:teuthology.orchestra.run.smithi119.stderr: -- /var/log/ceph/ceph-client.admin.65620.log 2024-04-25T10:19:08.904 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69112.log: gzip -5 88.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.69112.log.gz /var/log/ceph/ceph-client.admin.64530.log 2024-04-25T10:19:08.904 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-04-25T10:19:08.905 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46160.log 2024-04-25T10:19:08.906 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65620.log.gz 2024-04-25T10:19:08.907 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64530.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64530.log.gz 2024-04-25T10:19:08.907 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.46160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38782.log 2024-04-25T10:19:08.908 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46160.log.gz 2024-04-25T10:19:08.908 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67657.log 2024-04-25T10:19:08.910 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62833.log 2024-04-25T10:19:08.910 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38782.log.gz 2024-04-25T10:19:08.912 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67657.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.67657.log.gz 2024-04-25T10:19:08.912 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44265.log 2024-04-25T10:19:08.913 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75424.log 2024-04-25T10:19:08.914 INFO:teuthology.orchestra.run.smithi119.stderr: 92.4%/var/log/ceph/ceph-client.admin.44265.log: -- replaced with /var/log/ceph/ceph-client.admin.62833.log.gz 2024-04-25T10:19:08.915 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44265.log.gz 2024-04-25T10:19:08.915 INFO:teuthology.orchestra.run.smithi119.stderr: -- /var/log/ceph/ceph-client.admin.50712.log 2024-04-25T10:19:08.916 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75424.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75424.log.gz 2024-04-25T10:19:08.916 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43107.log 2024-04-25T10:19:08.918 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65746.log 2024-04-25T10:19:08.918 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50712.log.gz 2024-04-25T10:19:08.920 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43107.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43107.log.gz 2024-04-25T10:19:08.920 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54160.log 2024-04-25T10:19:08.920 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65746.log.gz 2024-04-25T10:19:08.921 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67456.log 2024-04-25T10:19:08.923 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42384.log 2024-04-25T10:19:08.923 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54160.log.gz 2024-04-25T10:19:08.924 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67456.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67456.log.gz 2024-04-25T10:19:08.924 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59926.log 2024-04-25T10:19:08.925 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42384.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42384.log.gz 2024-04-25T10:19:08.926 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65534.log 2024-04-25T10:19:08.927 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61368.log 2024-04-25T10:19:08.927 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59926.log.gz 2024-04-25T10:19:08.929 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65534.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65534.log.gz 2024-04-25T10:19:08.929 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59661.log 2024-04-25T10:19:08.930 INFO:teuthology.orchestra.run.smithi119.stderr:gzip 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61368.log.gz 2024-04-25T10:19:08.931 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62483.log 2024-04-25T10:19:08.932 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59576.log 2024-04-25T10:19:08.932 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59661.log.gz 2024-04-25T10:19:08.933 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62483.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62483.log.gz 2024-04-25T10:19:08.934 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72009.log 2024-04-25T10:19:08.934 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59576.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59576.log.gz 2024-04-25T10:19:08.935 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76306.log 2024-04-25T10:19:08.936 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.72009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51059.log 2024-04-25T10:19:08.936 INFO:teuthology.orchestra.run.smithi119.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.72009.log.gz 2024-04-25T10:19:08.937 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76306.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76306.log.gz 2024-04-25T10:19:08.938 INFO:teuthology.orchestra.run.smithi119.stderr:gzip/var/log/ceph/ceph-client.admin.51059.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.69280.log 2024-04-25T10:19:08.939 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51059.log.gz 2024-04-25T10:19:08.939 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58014.log 2024-04-25T10:19:08.941 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40239.log 2024-04-25T10:19:08.941 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69280.log.gz 2024-04-25T10:19:08.942 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.58014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58014.log.gz 2024-04-25T10:19:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64844.log 2024-04-25T10:19:08.944 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50147.log 2024-04-25T10:19:08.944 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40239.log.gz 2024-04-25T10:19:08.945 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74470.log 2024-04-25T10:19:08.945 INFO:teuthology.orchestra.run.smithi119.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64844.log.gz 2024-04-25T10:19:08.947 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50147.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.50147.log.gz 2024-04-25T10:19:08.947 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61119.log 2024-04-25T10:19:08.947 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74470.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74470.log.gz 2024-04-25T10:19:08.948 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33321.log 2024-04-25T10:19:08.949 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69891.log 2024-04-25T10:19:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.33321.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.61119.log.gz 2024-04-25T10:19:08.950 INFO:teuthology.orchestra.run.smithi119.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33321.log.gz 2024-04-25T10:19:08.951 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54076.log 2024-04-25T10:19:08.951 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69891.log: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T10:19:08.952 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69891.log.gz 2024-04-25T10:19:08.952 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74015.log 2024-04-25T10:19:08.953 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39892.log 2024-04-25T10:19:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54076.log: 88.4%/var/log/ceph/ceph-client.0.74015.log: -- replaced with /var/log/ceph/ceph-client.admin.54076.log.gz 2024-04-25T10:19:08.954 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74015.log.gz 2024-04-25T10:19:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65226.log 2024-04-25T10:19:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39892.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39892.log.gz 2024-04-25T10:19:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58984.log 2024-04-25T10:19:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39897.log 2024-04-25T10:19:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65226.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65226.log.gz 2024-04-25T10:19:08.958 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58984.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58984.log.gz 2024-04-25T10:19:08.959 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58241.log 2024-04-25T10:19:08.959 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39897.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39897.log.gz 2024-04-25T10:19:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60012.log 2024-04-25T10:19:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58010.log 2024-04-25T10:19:08.962 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58241.log: /var/log/ceph/ceph-client.admin.60012.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58241.log.gz 2024-04-25T10:19:08.962 INFO:teuthology.orchestra.run.smithi119.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51843.log 2024-04-25T10:19:08.963 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60012.log.gz 2024-04-25T10:19:08.963 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.58010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43347.log 2024-04-25T10:19:08.963 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58010.log.gz 2024-04-25T10:19:08.964 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51843.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51843.log.gz 2024-04-25T10:19:08.965 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61434.log 2024-04-25T10:19:08.965 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45613.log 2024-04-25T10:19:08.966 INFO:teuthology.orchestra.run.smithi119.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43347.log.gz 2024-04-25T10:19:08.967 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71080.log 2024-04-25T10:19:08.967 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61434.log.gz 2024-04-25T10:19:08.967 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65704.log 2024-04-25T10:19:08.968 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45613.log.gz 2024-04-25T10:19:08.969 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.71080.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71080.log.gz 2024-04-25T10:19:08.970 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61041.log 2024-04-25T10:19:08.970 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65704.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65704.log.gz 2024-04-25T10:19:08.971 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61064.log 2024-04-25T10:19:08.972 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47187.log 2024-04-25T10:19:08.972 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61041.log: /var/log/ceph/ceph-client.admin.61064.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61041.log.gz 2024-04-25T10:19:08.973 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67933.log 2024-04-25T10:19:08.973 INFO:teuthology.orchestra.run.smithi119.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.61064.log.gz 2024-04-25T10:19:08.974 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55176.log 2024-04-25T10:19:08.974 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47187.log.gz 2024-04-25T10:19:08.975 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43725.log 2024-04-25T10:19:08.975 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67933.log.gz 2024-04-25T10:19:08.976 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39133.log 2024-04-25T10:19:08.976 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55176.log.gz 2024-04-25T10:19:08.977 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43725.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43725.log.gz 2024-04-25T10:19:08.977 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46994.log 2024-04-25T10:19:08.978 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72518.log 2024-04-25T10:19:08.979 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39133.log.gz 2024-04-25T10:19:08.980 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.46994.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T10:19:08.980 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46994.log.gz 2024-04-25T10:19:08.980 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.72518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72518.log.gz 2024-04-25T10:19:08.981 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73312.log 2024-04-25T10:19:08.982 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57219.log 2024-04-25T10:19:08.983 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67891.log 2024-04-25T10:19:08.983 INFO:teuthology.orchestra.run.smithi119.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.73312.log.gz 2024-04-25T10:19:08.984 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.57219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70518.log 2024-04-25T10:19:08.985 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67891.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57219.log.gz 2024-04-25T10:19:08.986 INFO:teuthology.orchestra.run.smithi119.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.67891.log.gz 2024-04-25T10:19:08.986 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.70518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60330.log 2024-04-25T10:19:08.987 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70518.log.gz 2024-04-25T10:19:08.987 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73056.log 2024-04-25T10:19:08.989 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63777.log 2024-04-25T10:19:08.989 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.60330.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60330.log.gz 2024-04-25T10:19:08.990 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71457.log 2024-04-25T10:19:08.991 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73056.log.gz 2024-04-25T10:19:08.991 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.63777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46561.log 2024-04-25T10:19:08.991 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63777.log.gz 2024-04-25T10:19:08.992 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.71457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65872.log 2024-04-25T10:19:08.992 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71457.log.gz 2024-04-25T10:19:08.993 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.46561.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46561.log.gz -5 2024-04-25T10:19:08.993 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42594.log 2024-04-25T10:19:08.994 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63372.log 2024-04-25T10:19:08.995 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65872.log.gz 2024-04-25T10:19:08.996 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43305.log 2024-04-25T10:19:08.996 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42594.log.gz 2024-04-25T10:19:08.997 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.63372.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63372.log.gz 2024-04-25T10:19:08.997 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56866.log 2024-04-25T10:19:08.998 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55862.log 2024-04-25T10:19:08.999 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43305.log.gz/var/log/ceph/ceph-client.admin.56866.log: 2024-04-25T10:19:08.999 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43971.log 2024-04-25T10:19:08.999 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56866.log.gz 2024-04-25T10:19:09.000 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.55862.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55862.log.gz 2024-04-25T10:19:09.000 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58587.log 2024-04-25T10:19:09.002 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41533.log 2024-04-25T10:19:09.002 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43971.log.gz 2024-04-25T10:19:09.003 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42678.log 2024-04-25T10:19:09.003 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58587.log.gz 2024-04-25T10:19:09.004 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.41533.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41533.log.gz 2024-04-25T10:19:09.004 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69238.log 2024-04-25T10:19:09.006 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39091.log 2024-04-25T10:19:09.006 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42678.log.gz 2024-04-25T10:19:09.007 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.69238.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69238.log.gz 2024-04-25T10:19:09.007 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59026.log 2024-04-25T10:19:09.008 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39091.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39091.log.gz 2024-04-25T10:19:09.008 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71122.log 2024-04-25T10:19:09.009 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40435.log 2024-04-25T10:19:09.010 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.59026.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59026.log.gz 2024-04-25T10:19:09.011 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.71122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49479.log 2024-04-25T10:19:09.011 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71122.log.gz 2024-04-25T10:19:09.015 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40435.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48714.log 2024-04-25T10:19:09.015 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40435.log.gz 2024-04-25T10:19:09.015 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.49479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53595.log 2024-04-25T10:19:09.015 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48714.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.49479.log.gz 2024-04-25T10:19:09.015 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48714.log.gz 2024-04-25T10:19:09.015 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54328.log 2024-04-25T10:19:09.016 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73762.log 2024-04-25T10:19:09.016 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53595.log.gz 2024-04-25T10:19:09.017 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58671.log 2024-04-25T10:19:09.018 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54328.log.gz 2024-04-25T10:19:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73762.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.73762.log.gz 2024-04-25T10:19:09.019 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.58671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47439.log 2024-04-25T10:19:09.020 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58671.log.gz 2024-04-25T10:19:09.020 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64152.log 2024-04-25T10:19:09.022 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47439.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49405.log 2024-04-25T10:19:09.022 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47439.log.gz 2024-04-25T10:19:09.023 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56321.log 2024-04-25T10:19:09.023 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64152.log.gz 2024-04-25T10:19:09.024 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.49405.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49405.log.gz 2024-04-25T10:19:09.024 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62151.log 2024-04-25T10:19:09.025 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56321.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56321.log.gz 2024-04-25T10:19:09.026 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62687.log 2024-04-25T10:19:09.027 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40645.log 2024-04-25T10:19:09.028 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62151.log: /var/log/ceph/ceph-client.admin.62687.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.62151.log.gz 2024-04-25T10:19:09.028 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62687.log.gz 2024-04-25T10:19:09.029 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71266.log 2024-04-25T10:19:09.029 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40645.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40645.log.gz 2024-04-25T10:19:09.030 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68860.log 2024-04-25T10:19:09.031 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43221.log 2024-04-25T10:19:09.031 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.71266.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71266.log.gz 2024-04-25T10:19:09.032 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.68860.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68860.log.gz 2024-04-25T10:19:09.033 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62525.log 2024-04-25T10:19:09.033 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.43221.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43221.log.gz 2024-04-25T10:19:09.033 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66481.log 2024-04-25T10:19:09.035 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57135.log 2024-04-25T10:19:09.035 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.62525.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62525.log.gz 2024-04-25T10:19:09.036 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.66481.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66481.log.gz 2024-04-25T10:19:09.037 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54775.log 2024-04-25T10:19:09.038 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.57135.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57135.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39566.log 2024-04-25T10:19:09.038 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-04-25T10:19:09.039 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.54775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49177.log 2024-04-25T10:19:09.039 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54775.log.gz 2024-04-25T10:19:09.040 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39566.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39566.log.gz 2024-04-25T10:19:09.041 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48990.log 2024-04-25T10:19:09.041 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.49177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49177.log.gz 2024-04-25T10:19:09.042 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40897.log 2024-04-25T10:19:09.043 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.48990.log: /var/log/ceph/ceph-client.admin.67210.log 2024-04-25T10:19:09.043 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48990.log.gz 2024-04-25T10:19:09.044 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41552.log 2024-04-25T10:19:09.044 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40897.log.gz 2024-04-25T10:19:09.045 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.67210.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49760.log 2024-04-25T10:19:09.045 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67210.log.gz 2024-04-25T10:19:09.047 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.41552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71717.log 2024-04-25T10:19:09.047 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41552.log.gz 2024-04-25T10:19:09.047 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.49760.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.49760.log.gz 2024-04-25T10:19:09.048 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64194.log 2024-04-25T10:19:09.050 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.71717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56225.log 2024-04-25T10:19:09.050 INFO:teuthology.orchestra.run.smithi119.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.71717.log.gz 2024-04-25T10:19:09.051 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64194.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64194.log.gz 2024-04-25T10:19:09.051 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45010.log 2024-04-25T10:19:09.051 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56225.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56225.log.gz 2024-04-25T10:19:09.052 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40771.log 2024-04-25T10:19:09.053 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56151.log 2024-04-25T10:19:09.054 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45010.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45010.log.gz 2024-04-25T10:19:09.054 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.40771.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40771.log.gz 2024-04-25T10:19:09.055 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65914.log 2024-04-25T10:19:09.055 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56151.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56151.log.gz 2024-04-25T10:19:09.056 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51801.log 2024-04-25T10:19:09.057 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42258.log 2024-04-25T10:19:09.057 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65914.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65914.log.gz 2024-04-25T10:19:09.059 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51801.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51801.log.gz 2024-04-25T10:19:09.059 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75550.log 2024-04-25T10:19:09.059 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.42258.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42258.log.gz 2024-04-25T10:19:09.060 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45220.log 2024-04-25T10:19:09.061 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71038.log 2024-04-25T10:19:09.062 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75550.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75550.log.gz 2024-04-25T10:19:09.062 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.45220.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45220.log.gz 2024-04-25T10:19:09.063 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73013.log 2024-04-25T10:19:09.063 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.71038.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71038.log.gz 2024-04-25T10:19:09.064 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57069.log 2024-04-25T10:19:09.065 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75718.log 2024-04-25T10:19:09.066 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.73013.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73013.log.gz 2024-04-25T10:19:09.067 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.57069.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57069.log.gz 2024-04-25T10:19:09.067 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.75718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44722.log 2024-04-25T10:19:09.068 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75718.log.gz 2024-04-25T10:19:09.068 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48303.log 2024-04-25T10:19:09.069 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44223.log 2024-04-25T10:19:09.070 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44722.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44722.log.gz 2024-04-25T10:19:09.071 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.48303.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48303.log.gz 2024-04-25T10:19:09.071 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72161.log 2024-04-25T10:19:09.072 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44223.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44223.log.gz 2024-04-25T10:19:09.072 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50838.log 2024-04-25T10:19:09.073 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56128.log 2024-04-25T10:19:09.074 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.72161.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72161.log.gz 2024-04-25T10:19:09.075 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50838.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50838.log.gz 2024-04-25T10:19:09.075 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61869.log 2024-04-25T10:19:09.076 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.56128.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56128.log.gz 2024-04-25T10:19:09.076 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42972.log 2024-04-25T10:19:09.077 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39894.log 2024-04-25T10:19:09.079 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61869.log: /var/log/ceph/ceph-client.admin.42972.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61869.log.gz 2024-04-25T10:19:09.079 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42972.log.gz 2024-04-25T10:19:09.079 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50399.log 2024-04-25T10:19:09.080 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.39894.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39894.log.gz 2024-04-25T10:19:09.080 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47103.log 2024-04-25T10:19:09.081 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38698.log 2024-04-25T10:19:09.082 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.50399.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50399.log.gz 2024-04-25T10:19:09.083 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.47103.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47103.log.gz 2024-04-25T10:19:09.083 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61719.log 2024-04-25T10:19:09.084 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.38698.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38698.log.gz 2024-04-25T10:19:09.084 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58713.log 2024-04-25T10:19:09.085 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55643.log 2024-04-25T10:19:09.087 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.61719.log: /var/log/ceph/ceph-client.admin.58713.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61719.log.gz 2024-04-25T10:19:09.087 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58713.log.gz 2024-04-25T10:19:09.088 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53130.log 2024-04-25T10:19:09.088 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.55643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55643.log.gz 2024-04-25T10:19:09.088 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76222.log 2024-04-25T10:19:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.53130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65788.log 2024-04-25T10:19:09.090 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53130.log.gz 2024-04-25T10:19:09.091 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.76222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51139.log 2024-04-25T10:19:09.091 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76222.log.gz 2024-04-25T10:19:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.65788.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65788.log.gz 2024-04-25T10:19:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.51139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64446.log 2024-04-25T10:19:09.094 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51139.log.gz 2024-04-25T10:19:09.094 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74302.log 2024-04-25T10:19:09.095 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44391.log 2024-04-25T10:19:09.096 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.64446.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64446.log.gz 2024-04-25T10:19:09.097 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.74302.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74302.log.gz 2024-04-25T10:19:09.097 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57655.log 2024-04-25T10:19:09.098 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.44391.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44391.log.gz 2024-04-25T10:19:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.57655.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57655.log.gz 2024-04-25T10:19:09.250 INFO:teuthology.orchestra.run.smithi119.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T10:19:09.260 INFO:teuthology.orchestra.run.smithi119.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T10:19:09.273 INFO:teuthology.orchestra.run.smithi119.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T10:19:09.305 INFO:teuthology.orchestra.run.smithi119.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T10:19:09.433 INFO:teuthology.orchestra.run.smithi119.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T10:19:09.436 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-04-25T10:19:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:real 0m1.335s 2024-04-25T10:19:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:user 0m4.685s 2024-04-25T10:19:09.437 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m2.271s 2024-04-25T10:19:09.438 INFO:tasks.ceph:Archiving logs... 2024-04-25T10:19:09.438 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048/remote/smithi119/log 2024-04-25T10:19:09.439 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:19:10.639 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048/remote/smithi132/log 2024-04-25T10:19:10.640 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:19:11.133 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:19:11.170 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:19:11.171 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:19:11.179 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:19:11.315 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:19:11.324 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:19:11.324 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:19:11.333 DEBUG:teuthology.orchestra.run.smithi132:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:19:11.381 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:19:11.386 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:19:11.391 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:19:11.391 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-04-25T10:19:11.391 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv01.front.sepia.ceph.com 3 6 377 62 +1188us[+1193us] +/- 45ms 2024-04-25T10:19:11.391 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -1050us[-1050us] +/- 51ms 2024-04-25T10:19:11.391 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 -519us[ -514us] +/- 65ms 2024-04-25T10:19:11.391 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:19:11.396 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:19:11.396 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-04-25T10:19:11.396 INFO:teuthology.orchestra.run.smithi132.stdout:^* hv01.front.sepia.ceph.com 3 6 377 62 +1170us[+1191us] +/- 45ms 2024-04-25T10:19:11.396 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 63 -1081us[-1060us] +/- 51ms 2024-04-25T10:19:11.397 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 -553us[ -532us] +/- 65ms 2024-04-25T10:19:11.397 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:19:11.398 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:19:11.407 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:19:11.408 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:19:11.417 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:19:11.629 DEBUG:teuthology.orchestra.run.smithi132:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:19:11.824 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T10:19:11.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:19:11.922 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T10:19:12.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:19:12.017 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:19:12.027 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:19:12.036 INFO:teuthology.task.internal:Duration was 1218.703749 seconds 2024-04-25T10:19:12.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:19:12.045 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:19:12.045 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:19:12.053 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:19:12.185 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:19:12.195 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:19:12.564 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:19:12.564 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2024-04-25T10:19:12.564 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:19:12.625 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi132.front.sepia.ceph.com 2024-04-25T10:19:12.625 DEBUG:teuthology.orchestra.run.smithi132:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:19:12.700 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:19:12.700 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:19:12.708 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:19:12.786 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:19:12.787 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:19:12.792 DEBUG:teuthology.orchestra.run.smithi132:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:19:12.986 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:19:12.997 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:19:12.997 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:19:13.070 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:19:13.156 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:19:13.166 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:19:13.174 DEBUG:teuthology.orchestra.run.smithi132:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:19:13.267 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2024-04-25T10:19:13.273 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = core 2024-04-25T10:19:13.318 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:19:13.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:19:13.365 DEBUG:teuthology.orchestra.run.smithi132:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:19:13.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:19:13.419 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:19:13.430 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:19:13.431 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048/remote/smithi119 2024-04-25T10:19:13.431 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:19:13.533 DEBUG:teuthology.misc:Transferring archived files from smithi132:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673048/remote/smithi132 2024-04-25T10:19:13.534 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:19:13.629 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:19:13.629 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:19:13.638 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:19:13.693 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:19:13.705 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:19:13.705 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:19:13.715 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:19:13.715 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:19:13.721 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:19:13.768 INFO:teuthology.orchestra.run.smithi119.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:19 /home/ubuntu/cephtest 2024-04-25T10:19:13.772 INFO:teuthology.orchestra.run.smithi132.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:19 /home/ubuntu/cephtest 2024-04-25T10:19:13.775 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:19:13.786 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:19:13.810 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} duration: 1218.70374917984 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-04-25T10:19:13.811 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:19:13.887 INFO:teuthology.run:pass